C OMPUTATIONAL P HYSICS 430

PARTIAL D IFFERENTIAL E QUATIONS

Ross L. Spencer and Michael Ware

and

John Colton (Lab 13, Glossary)

Department of Physics and Astronomy

Brigham Young University
 C OMPUTATIONAL P HYSICS 430
PARTIAL D IFFERENTIAL E QUATIONS

Ross L. Spencer and Michael Ware

and

John Colton

Department of Physics and Astronomy

Brigham Young University
Last revised: March 30, 2021

© 2004–2021
Ross L. Spencer, Michael Ware, and Brigham Young University

This is a laboratory course about using computers to solve partial differential

equations that occur in the study of electromagnetism, heat transfer, acoustics,
and quantum mechanics. The course objectives are to

• Solve physics problems involving partial differential equations numerically.

• Better be able to do general programming using loops, logic, etc.

• Have an increased conceptual understanding of the physical implications

of important partial differential equations

You will need to read through each lab before class to complete the exercises
during the class period. The labs are designed so that the exercises can be done in
class (where you have access to someone who can help you) if you come prepared.
Please work with a lab partner. It will take a lot longer to do these exercises if you
are on your own.
To be successful in this class, you should have already taken an introductory
programming class in a standard language (e.g. C++) and be able to use a symbolic
mathematical program such as Mathematica. We will be teaching you Python
based on the assumption that you already know how to program in these other
languages. We also assume that you have studied upper division mathematical
physics (e.g. mathematical methods for solving partial differential equations with
Fourier analysis).
Suggestions for improving this manual are welcome. Please direct them to
Michael Ware ([email protected]).
Contents

Preface i

Table of Contents iii

1 Grids and Numerical Derivatives 1

Introduction to Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Spatial grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Interpolation and extrapolation . . . . . . . . . . . . . . . . . . . . . . . 2
Derivatives on grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Differential Equations with Boundary Conditions 7

More Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Initial conditions vs. boundary conditions . . . . . . . . . . . . . . . . . 7
Solving differential equations with linear algebra . . . . . . . . . . . . . 7
Derivative boundary conditions . . . . . . . . . . . . . . . . . . . . . . . 10
Nonlinear differential equations . . . . . . . . . . . . . . . . . . . . . . . 11

3 The Wave Equation: Steady State and Resonance 13

Python Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Steady state solution by separation of variables . . . . . . . . . . . . . . 13
Resonance and the eigenvalue problem . . . . . . . . . . . . . . . . . . . 15

4 The Hanging Chain and Quantum Bound States 19

Resonance for a hanging chain . . . . . . . . . . . . . . . . . . . . . . . . 19
Quantum bound states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5 Animating the Wave Equation: Staggered Leapfrog 25

The wave equation with staggered leapfrog . . . . . . . . . . . . . . . . . 25
The damped wave equation . . . . . . . . . . . . . . . . . . . . . . . . . . 29
The damped and driven wave equation . . . . . . . . . . . . . . . . . . . 30

6 The 2-D Wave Equation With Staggered Leapfrog 33

Two dimensional grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
The two-dimensional wave equation . . . . . . . . . . . . . . . . . . . . 34
Elliptic, hyperbolic, and parabolic PDEs . . . . . . . . . . . . . . . . . . 37

iii
7 The Diffusion Equation and Implicit Methods 41
Analytic approach to the Diffusion Equation . . . . . . . . . . . . . . . . 41
Numerical approach: a first try . . . . . . . . . . . . . . . . . . . . . . . . 41
Implicit Methods: the Crank-Nicolson Algorithm . . . . . . . . . . . . . 43
The diffusion equation with Crank-Nicolson . . . . . . . . . . . . . . . . 44

8 Schrödinger’s Equation 49
Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Particle in a box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

9 Poisson’s Equation: Iteration Methods 53

Finite difference form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Iteration method on a simple example . . . . . . . . . . . . . . . . . . . 54
Iteration method on Poisson’s equation . . . . . . . . . . . . . . . . . . . 55
Successive over-relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Some Different Geometries . . . . . . . . . . . . . . . . . . . . . . . . . . 60

10 The Continuity Equation 63

The Continuity Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Crank-Nicolson Again . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

11 One Dimensional Gas Dynamics 69

Simultaneously advancing ρ, T , and v . . . . . . . . . . . . . . . . . . . . 69
Continuity Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Conservation of energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
Newton’s second law . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Waves in a closed tube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

12 Solitons: Korteweg-deVries Equation 81

Numerical solution for the Korteweg-deVries equation . . . . . . . . . . 81
Solitons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

13 Machine Learning with Neural Networks 89

Machine Learning Terminology . . . . . . . . . . . . . . . . . . . . . . . 89
Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
Neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Example 1: Predicting exam scores . . . . . . . . . . . . . . . . . . . . . . 94
Example 2: Predicting median home prices . . . . . . . . . . . . . . . . . 95
Example 3: Predicting handwritten digits . . . . . . . . . . . . . . . . . . 96
Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Petals and Sepals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

A Anaconda Environments 103

B Glossary of Terms 107

Index 111
Lab 1
Grids and Numerical Derivatives
Introduction to Python
In this course we will use Python to study numerical techniques for solving some
partial differential equations that arise in Physics. Don’t be scared of this new
language. Most of the ideas, and some of the syntax, that you learned for Matlab
will transfer directly to Python. We’ll work through some brief tutorials about
Python at the beginning of each lab, focusing on the particular ideas that you’ll
need to complete that lab. Pretty soon you will be Python wizards.

P1.1 Work through Chapter 1 of Introduction to Python. There you will learn the
basics of how to write a Python program, how to declare and use entities
called NumPy arrays, and also learn some basic plotting techniques.

With that Python knowledge under our belts, let’s move on to begin our study of
partial differential equations.

Spatial grids
When we solved ordinary differential equations in Physics 330 we were usually
moving something forward in time, so you may have the impression that differ-
ential equations always “flow.” This is not true. If we solve a spatial differential
equation, like the one that gives the shape of a chain draped between two posts,
the solution just sits in space; nothing flows. Instead, we choose a small spatial
step size (think of each individual link in the chain) and seek to find the correct
shape by somehow finding the height of the chain at each link. Cell-Edge Grid
In this course we will solve partial differential equations, which usually means
that the desired solution is a function of both space x, which just sits, and time t , 0 L
which flows. When we solve problems like this we will be using spatial grids, to
represent the x-part that doesn’t flow. The NumPy arrays that you just learned Cell-Center Grid
about above are perfect for representing these kinds of spatial grids.
We’ll encounter three basic types of spatial grids in this class. Figure 1.1 shows 0 L
a graphical representation of these three types of spatial grids for the region
0 ≤ x ≤ L. We divide this region into spatial cells (the spaces between vertical Cell-Center Grid with Ghost Points
lines) and functions are evaluated at N discrete grid points (the dots). In a cell-
edge grid, the grid points are located at the edge of the cell. In a cell-center grid, 0 L
the points are located in the middle of the cell. Another useful grid is a cell-center Figure 1.1 Three common spatial
grid with ghost points. The ghost points (unfilled dots) are extra grid points on grids
either side of the interval of interest and are useful when we need to consider the
derivatives at the edge of a grid.

1
2 Computational Physics 430

P1.2 (a) Write a Python program that creates a cell-edge spatial grid in the
variable x as follows:
4
import numpy as np
3
N=100 # the number of grid points
2 a=0
y(x) b=np.pi
1 x,h = np.linspace(a,b,N,retstep = True)
0 Plot the function y(x) = sin(x) sinh(x) on this grid. Explain the rela-
0 1 2 3
x tionship between the number of cells and the number of grid points
in a cell-edge grid.
Figure 1.2 Plot from 1.2(a)
(b) Explain the relationship between the number of cells and the num-
ber of grid points in a cell-center grid. Then write some code using
NumPy’s arange function to create a cell-centered grid that has ex-
actly 100 cells over the interval 0 ≤ x ≤ 2.
1 Evaluate the function f (x) = cos x on this grid and plot this function.
f(x) Then estimate the area under the curve by summing the products of
0.5 the centered function values f j with the widths of the cells h like this
(midpoint integration rule):
0
np.sum(f)*h
−0.5 Compare this result to the exact answer obtained by integration:
0 0.5 1 1.5 2 Z 2
x ¯2
A= cos x d x = sin(x)¯ = sin(2)
¯
Figure 1.3 Plot from 1.2(b) 0 0

(c) Build a cell-center grid with ghost points over the interval 0 ≤ x ≤ π/2
with 500 cells (502 grid points), and evaluate the function f (x) = sin x
on this grid. Now look carefully at the function values at the first
two grid points and at the last two grid points. The function sin x
has the property that f (0) = 0 and f 0 (π/2) = 0. The cell-center grid
doesn’t have points at the ends of the interval, so these boundary
conditions on the function need to be enforced using more than one
point. Explain how the ghost points can be used in connection with
interior points to specify both function-value boundary conditions
and derivative-value boundary conditions.

Interpolation and extrapolation

Grids only represent functions at discrete points, and there will be times when
we want to find good values of a function between grid points (interpolation) or
beyond the last grid point (extrapolation). We will use interpolation and extrapo-
lation techniques fairly often during this course, so let’s review these ideas.
The simplest way to estimate function values is to use the fact that two points
define a straight line. For example, suppose that we have function values (x 1 , y 1 )
Lab 1 Grids and Numerical Derivatives 3

and (x 2 , y 2 ). The formula for a straight line that passes through these two points
is
(y 2 − y 1 )
y − y1 = (x − x 1 ) (1.1)
(x 2 − x 1 )
Once this line has been established it provides an approximation to the true
function y(x) that is pretty good in the neighborhood of the two data points. To
linearly interpolate or extrapolate we simply evaluate Eq. (1.1) at x values between
or beyond x 1 and x 2 .

P1.3 Use Eq. (1.1) to do the following special cases:

(x2 , y2 )
(a) Find an approximate value for y(x) halfway between the two points (x1 , y1 )
x 1 and x 2 . Does your answer make sense?
Figure 1.4 The line defined by two
(b) Find an approximate value for y(x) 3/4 of the way from x 1 to x 2 . Do
points can be used to interpolate
you see a pattern? between the points and extrapo-
(c) If the spacing between grid points is h (i.e. x 2 − x 1 = h), show that the late beyond the points.
linear extrapolation formula for y(x 2 + h) is

y(x 2 + h) = 2y 2 − y 1 (1.2)

This provides a convenient way to estimate the function value one

grid step beyond the last grid point. Also show that

y(x 2 + h/2) = 3y 2 /2 − y 1 /2. (1.3)

We will use both of these formulas during the course.

Derivatives on grids
When solving partial differential equations, we will frequently need to calculate
derivatives on our grids. In your introductory calculus book, the derivative was
probably introduced using the forward difference formula

f (x + h) − f (x)
f 0 (x) ≈ . (1.4)
h
The word “forward” refers to the way this formula reaches forward from x to x + h
to calculate the slope. The exact derivative represented by Eq. (1.4) in the limit that
h approaches zero. However, we can’t make h arbitrarily small when we represent
a function on a grid because (i) the number of cells needed to represent a region
of space becomes infinite as h goes to zero; and (ii) computers represent numbers
with a finite number of significant digits so the subtraction in the numerator of
Eq. (1.4) loses accuracy when the two function values are very close. But given
these limitation we want to be as accurate as possible, so we want to use the best
derivative formulas available. The forward difference formula isn’t one of them.
4 Computational Physics 430

The best first derivative formula that uses only two function values is usually
the centered difference formula:
f (x + h) − f (x − h)
f 0 (x) ≈ . (1.5)
2h
It is called “centered” because the point x at which we want the slope is centered
between the places where the function is evaluated. Take a minute to study
Fig. 1.5 to understand visually why the centered difference formula is so much
better than the forward difference formula. The corresponding centered second
Figure 1.5 The forward and cen-
derivative formula is
tered difference formulas both
approximate the derivative as the f (x + h) − 2 f (x) + f (x − h)
f 00 (x) ≈ (1.6)
slope of a line connecting two h2
points. The centered difference We will derive both of these formulas later, but for now we just want you to
formula gives a more accurate ap- understand how to use them.
proximation because it uses points
The colon operator provides a compact way to evaluate Eqs. (1.5) and (1.6)
before and after the point where
the derivative is being estimated.
on a grid. Unfortunately for those of you familiar with Matlab, Python’s colon
(The true derivative is the slope of operator acts a little differently from Matlab’s in that the last index is not included
the dotted tangent line). in the range, as we noted in the tutorial. If the function we want to take the
derivative of is stored in an array f, we can calculate the centered first derivative
like this (remember that Python array indexes are zero-based):
fp = np.zeros_like(f)
fp[1:N-1]=(f[2:N]-f[0:N-2])/(2*h)
and the centered second derivative at each interior grid point like this:
fpp = np.zeros_like(f)
fpp[1:N-1]=(f[2:N]-2*f[1:N-1]+f[0:N-2])/h**2
The variable h is the spacing between grid points and N is the number of grid
points. Study this code (focus on the indexing) until you are convinced that it
represents Eqs. (1.5) and (1.6) correctly.
The derivative at the first and last points on the grid can’t be calculated with
Eqs. (1.5) and (1.6) since there are not grid points on both sides of the endpoints.
Instead, we extrapolate the interior values of the two derivatives to the end points.
If we use linear extrapolation then we just need two nearby points, and the
formulas for the derivatives at the end points are found using Eq. (1.2):
fp[0]=2*fp[1]-fp[2]
1
fp[N-1]=2*fp[N-2]-fp[N-3]
f(x)
f  (x)
fpp[0]=2*fpp[1]-fpp[2]
0.5 fpp[N-1]=2*fpp[N-2]-fpp[N-3]
0 P1.4 Create a cell-edge grid with N = 20 on the interval 0 ≤ x ≤ 5. Load f (x) with
−0.5 the Bessel function J 0 (x) and numerically differentiate it to obtain f 0 (x)
f  (x) and f 00 (x). Then make overlaid plots of the numerical derivatives with the
−1
exact derivatives:
0 1 2 3 4 5
x f 0 (x) = −J 1 (x)
1
Figure 1.6 Plots from 1.4 f 00 (x) = (−J 0 (x) + J 2 (x))
2
Lab 1 Grids and Numerical Derivatives 5

Errors in the approximate derivative formulas

We’ll conclude this lab with a look at where the approximate derivative formulas
come from and at the types of the errors that pop up when using them. The
starting point is Taylor’s expansion of the function f a small distance h away from
the point x

1 00 hn
f (x + h) = f (x) + f 0 (x)h + f (x)h 2 + · · · + f (n) (x) + ··· (1.7)
2 n!

Let’s use this series to understand the forward difference approximation to f 0 (x).
If we apply the Taylor expansion to the f (x + h) term in Eq. (1.4), we get

f (x) + f 0 (x)h + 12 f 00 (x)h 2 + · · · − f (x)

£ ¤
f (x + h) − f (x)
= (1.8)
h h
The higher order terms in the expansion (represented by the dots) are smaller
than the f 00 term because they are all multiplied by higher powers of h (which we
assume to be small). If we neglect these higher order terms, we can solve Eq. (1.8)
for the exact derivative f 0 (x) to find

f (x + h) − f (x) h 00
f 0 (x) ≈ − f (x) (1.9)
h 2
From Eq. (1.9) we see that the forward difference does indeed give the first deriva-
tive back, but it carries an error term which is proportional to h. But if h is small
enough then the contribution from the term containing f 00 (x) will be too small to
matter and we will have a good approximation to f 0 (x).
For the centered difference formula, we use Taylor expansions for both f (x+h)
and f (x − h) in Eq. (1.5) to write
h i
0 00 h2 000 h3
f (x + h) − f (x − h) f (x) + f (x)h + f (x) 2 + f (x) 6 + · · ·
= (1.10)
2h h 2h i
2 3
f (x) − f 0 (x)h + f 00 (x) h2 − f 000 (x) h6 + · · ·
−
2h
If we again neglect the higher-order terms, we can solve Eq. (1.10) for the exact
derivative f 0 (x). This time, we find that the f 00 terms exactly cancel to give

f (x + h) − f (x − h) h 2 000
f 0 (x) ≈ − f (x) (1.11)
2h 6
Notice that for the centered formula the error term is much smaller, only of order
h 2 . So if we decrease h in both the forward and centered difference formulas by a
factor of 10, the forward difference error will decrease by a factor of 10, but the
centered difference error will decrease by a factor of 100. This is the reason we try
to use centered formulas whenever possible in this course.
6 Computational Physics 430

P1.5 Write a Python program to compute the forward and centered difference
formulas for the first derivative of the function f (x) = e x at x = 0 with
h = 0.1, 0.01, 0.001. Verify that the error estimates in Eqs. (1.9) and (1.11)
agree with the numerical testing.
Note that at x = 0 the exact values of both f 0 and f 00 are equal to e 0 = 1, so
just subtract 1 from your numerical result to find the error.

In problem 1.5, you found that h = 0.001 in the centered-difference formula

gives a better approximation than h = 0.01. This trend might entice you to try
to keep making h smaller and smaller to achieve any accuracy you want. This
doesn’t work. Figure 1.7 shows a plot of the error you calculated in problem 1.5 as
h continues to decrease (note the log scales). For the larger values of h, the errors
track well with the predictions made by the Taylor’s series analysis. However,
when h becomes too small, the error starts to increase. By about h = 10−16 , and
sooner for the second derivative, the error is the same order as the derivative.
The reason for this behavior is that numbers in computers are represented
with a finite number of significant digits. Most computational languages (includ-
Figure 1.7 Error in the forward ing Python) use double precision variables, which have 15-digit accuracy.1 This is
and centered difference approxi- normally plenty of precision, but look what happens in a subtraction problem
mations to the first derivative and where the two numbers are nearly the same:
the centered difference formula
for the second derivative as a func- 7.38905699669556
tion of h. The function is e x and − 7.38905699191745 (1.12)
the approximations are evaluated
0.00000000477811
for x = 0.
Notice that our nice 15-digit accuracy has disappeared, leaving behind only
6 significant figures. This problem occurs in calculations with floating-point
numbers on all digital computers, and is called roundoff. You can see this effect
by experimenting with the Python console:

h=1e-17
g=1+h
print(g-1)

for different values of h and noting that you don’t always get h back. Also notice
in Fig. 1.7 that this problem is worse for the second derivative formula than it is
for the first derivative formula. The lesson here is that it is impossible to achieve
arbitrarily high accuracy by using arbitrarily tiny values of h.

1 A computer uses 64 bits to represent a double precision number. 53 bits are used to represent

the significant digits. Thus the significant digit part can be any integer between 0 and 253 =
9007199254740992 (almost 16 digits). The exponent is represented by 11 bits. After you add the
possibility of NaN and Inf, the exponent can be −308 to +308. This leaves 1 bit for the overall sign
of the number. Extended precision uses more bits (memory) and computation time, so double
precision is mostly the standard for computational physics.
Lab 2
Differential Equations with Boundary Conditions

More Python
P2.1 Work through Chapter 2 of Introduction to Python, where you will learn
about lists, loops, and logic.

Initial conditions vs. boundary conditions

In Physics 330, we studied the behavior of systems where the initial conditions
were specified and we calculated how the system evolved forward in time (e.g. the
flight of a baseball given its initial position and velocity). In these cases we were
able to use Matlab’s convenient built-in differential equation solvers to model the
system. The situation becomes somewhat more complicated if instead of having
initial conditions, a differential equation has boundary conditions specified at
both ends of the interval (rather than just at the beginning). This seemingly simple
change in the boundary conditions makes it hard to use canned ODE solvers like
Matlab’s ode45. Fortunately, there are better ways to solve these systems.

Solving differential equations with linear algebra

Consider the differential equation

y 00 (x) + 9y(x) = sin(x) ; y(0) = 0, y(2) = 1 (2.1)

Notice that this differential equation has boundary conditions at both ends of the
interval instead of having initial conditions at x = 0. If we represent this equation
on a grid, we can turn this differential equation into a set of algebraic equations
that we can solve using linear algebra techniques. Before we see how this works,
let’s first specify the notation that we’ll use. We assume that we have set up a
cell-edge spatial grid with N grid points, and we refer to the x values at the grid
points using the notation x n , with n = 0..N −1. We represent the (as yet unknown)
function values y(x n ) on our grid using the notation y n = y(x n ).
Figure 2.1 A function y(x) repre-
Now we can write the differential equation in finite difference form as it sented on a cell-edge x-grid with
would appear on the grid. The second derivative in Eq. (2.1) is rewritten using the N = 9.
centered difference formula (see Eq. (1.5)), so that the finite difference version of
Eq. (2.1) becomes:
y n+1 − 2y n + y n−1
+ 9y n = sin(x n ) (2.2)
h2

7
8 Computational Physics 430

Now let’s think about Eq. (2.2) for a bit. First notice that it is not an equation, but
a system of many equations. We have one of these equations at every grid point n,
except the endpoints j = 0 and at j = N −1 where this formula reaches beyond the
ends of the grid and cannot, therefore, be used. Because this equation involves
y n−1 , y n , and y n+1 for the interior grid points n = 1 . . . N − 2, Eq. (2.2) is really a
system of N − 2 coupled equations in the N unknowns y 0 . . . y N −1 . If we had just
two more equations we could find the y n ’s by solving a linear system of equations.
But we do have two more equations; they are the boundary conditions:

y 0 = 0 ; y N −1 = 1 (2.3)

which completes our system of N equations in N unknowns.

Before Python can solve this system we have to put it in a matrix equation of
the form
Ay = b, (2.4)
where A is a matrix of coefficients, y the column vector of unknown y-values,
and b the column vector of known values on the right-hand side of Eq. (2.2). For
the particular case of the system represented by Eqs. (2.2) and (2.3), the matrix
equation is given by

1 0 0 0 ... 0 0 0
    
y0 0
1

 h2
− h22 + 9 1
h2
0 ... 0 0 0 
 y1  
  sin(x 1 )  
1 2 1

 0 h2
− h2 + 9 h2
... 0 0 0 
 y2  
  sin(x 2 )  
. . . . ... . . . . .
    
= .
    
. . . . ... . . . . .
 
    
    
 . . . . ... . . .  .   . 
1
− h22 + 9 1  y
    
 0 0 0 0 ... N −2   sin(x N −2 ) 
h2 h2
0 0 0 0 ... 0 0 1 y N −1 1
(2.5)
Convince yourself that Eq. (2.5) is equivalent to Eqs. (2.2) and (2.3) by mentally
doing each row of the matrix multiply by tipping one row of the matrix up on
end, dotting it into the column of unknown y-values, and setting it equal to the
corresponding element in the column vector on the right.
Once we have the finite-difference approximation to the differential equa-
tion in this matrix form (Ay = b), a simple linear solve is all that is required
to find the solution array y n . NumPy can do this solve with the command
linalg.solve(a, b).

P2.2 (a) Set up a cell-edge grid with N = 30 grid points, like this:
import numpy as np

N=30 # the number of grid points

a=0
b=2
x,h = np.linspace(a,b,N,retstep = True)
Look over this code and make sure you understand what it does before
just using it.
Lab 2 Differential Equations with Boundary Conditions 9

(b) If you solve Eq. (2.1) analytically, you find

16 sin(3x) + cos(6 − x) − cos(6 + x) + cos(2 + 3x) − cos(2 − 3x)

y(x) =
16 sin(6)

Type this solution formula into the Python program that defines the
grid above and plot the exact solution as a blue curve on a cell-edge
grid with N points.
(c) Now create a matrix A filled with zeros using np.zeros, and write a
for loop to load A like the matrix in Eq. (2.5) and do the linear solve to
obtain y n and plot it on top of the exact solution with red dots ('r.')
to see how closely the two agree. Experiment with larger values of N
and plot the difference between the exact and approximate solutions
to see how the error changes with N . We think you’ll be impressed at Figure 2.2 The solution to 2.2(c)
how well the numerical method works, if you use enough grid points. with N = 30

Let’s pause a moment to review how to apply this technique to solve a problem.
First, write out the differential equation as a set of finite difference equations on a
grid, as we did in Eq. (2.2). Then translate this set of finite difference equations
(plus the boundary conditions) into a matrix form analogous to Eq. (2.5). Finally,
build the matrix A and the column vector y in Python and solve for the vector y.
Our example, Eq. (2.1), had only a second derivative, but first derivatives can be
handled using the centered first derivative approximation, Eq. (1.5). Let’s practice
this procedure for a couple more differential equations:

P2.3 (a) Write out the finite difference equations on paper for the differential 10

equation 8
y(x)
6
1 0 1 4
y 00 + y + (1 − 2 )y = x ; y(0) = 0, y(5) = 1 (2.6)
x x 2
0
Then write down the matrix A and the vector b for this equation. Fi-
−2
nally, build these matrices in a Python program and solve the equation 0 1 2 3 4 5
using the matrix method. Compare the solution found using the ma- x
trix method with the exact solution Figure 2.3 Solution to 2.3(a) with
−4 N = 30 (dots) compared to the
y(x) = J 1 (x) + x exact solution (line)
J 1 (5)

J 1 (x) is the first order Bessel function.

HINT: When creating your b vector, you’ll be tempted to write some
code like this: b=x. Don’t. This code just assigns b to be a reference to
the matrix x. What you want is to create a copy of the matrix like this:
b = np.copy(x)
(b) Solve the differential equation

y 00 + sin (x)y 0 + e x y = x 2 ; y(0) = 0, y(5) = 3 (2.7)

10 Computational Physics 430

in Python using the matrix method. Mathematica’s NDSolve com-

mand claims that, to 15-digit accuracy

y(4.5) = 8.720623277763513

You’d run out of memory with the matrix method long before achiev-
ing this level of accuracy, but how many points do you have to use
in your numerical method to get agreement with Mathematica to 3
digits of accuracy (i.e. y(4.5) = 8.72)?
Hint: If your step size variable is h, the index for the for element where
x j ≈ 4.5 can be found this way:
j=int(4.5/h)
Figure 2.4 Solution to 2.3(b) with
The int command rounds the result and casts it as an integer so you
N = 200
can use it to access a specific element.

Derivative boundary conditions

Now let’s see how to modify the linear algebra approach to differential equations
so that we can handle boundary conditions where derivatives are specified instead
of values. Consider the differential equation

y 00 (x) + 9y(x) = x ; y(0) = 0 ; y 0 (2) = 0 (2.8)

We can satisfy the boundary condition y(0) = 0 as before (just use y 1 = 0), but
what do we do with the derivative condition at the other boundary?

P2.4 (a) A crude way to implement the derivative boundary condition is to use
0.25 a forward difference formula for the last two points:
0.2 y(x) y N −1 − y N −2 ¯
= y 0¯
¯
0.15 h x=2
0.1
In the present case, where y 0 (2) = 0, this simply means that we need
0.05
to write the last row of our matrix to enforce
0
0 0.5 1 1.5 2
x y N −1 − y N −2 = 0.

Figure 2.5 The solution to 2.4(a) Think about what the new boundary conditions will do to the final row
with N = 30. The RMS difference of matrix A and the final element of vector b, and then solve Eq. (2.8)
from the exact solution is 8.8 × 10−4 using the matrix method with this boundary condition. Compare the
resulting numerical solution to the analytic solution:

x sin (3x)
y(x) = −
9 27 cos (6)
Lab 2 Differential Equations with Boundary Conditions 11

(b) You can improve the boundary condition formula using quadratic
extrapolation. In this method, you fit a parabola of the form

y(x) = a + bx + c x 2 (2.9)

to the last three points on your grid to find a, b, and c in terms of your
data points. Then you take the derivative of Eq. (2.9) and evaluate it at
the edge x = x N −1 . Normally we’d make you go through the math to
derive this, but since time is short, we’ll just tell you that this process 0.25
gives you the following finite difference approximation for the y 0 (x) at 0.2 y(x)
the end of the grid: 0.15

1 2 3 0.1
y N −3 − y N −2 + y N −1 = y 0 (x N −1 ) (2.10)
2h h 2h 0.05
0
Modify your program from part (a) to include this new condition and 0 0.5 1 1.5 2
show that it gives a more accurate solution than the crude technique of x
part (a). When you check the accuracy, don’t just look at the end of the
Figure 2.6 The solution to 2.4(b)
interval. All of the points are coupled by the matrix A, so you should with N = 30. The RMS difference
use a full-interval accuracy check like the RMS (root-mean-square) from the exact solution is 5.4 × 10−4
error:
np.sqrt(np.mean((y-yexact)**2))

Nonlinear differential equations

Finally, we must confess that we have been giving you easy problems to solve,
which probably leaves the impression that you can use this linear algebra trick
to solve all second-order differential equations with boundary conditions at the
ends. The problems we have given you so far are easy because they are linear
differential equations, so they can be translated into linear algebra problems.
Linear problems are not the whole story in physics, of course, but most of the
problems we will do in this course are linear, so these finite-difference and matrix
methods will serve us well in the labs to come.

P2.5 (a) Here is a simple example of a differential equation that isn’t linear:
0
00
£ ¤
y (x) + sin y(x) = 1 ; y(0) = 0, y(3) = 0 (2.11) −0.5

−1 y(x)
Work at turning this problem into a linear algebra problem to see why
it can’t be done, and explain the reasons to the TA. −1.5

(b) Let’s find a way to use a combination of linear algebra and iteration −2

(initial guess, refinement, etc.) to solve Eq. (2.11) in Python on a grid. 0 1 2 3

First, write the equation as x

Figure 2.7 The solution to 2.5(b).

y 00 (x) = 1 − sin y(x)
£ ¤
(2.12)
12 Computational Physics 430

Make a guess for y(x). It doesn’t have to be a very good guess. In this
case, the guess y(x) = 0 works just fine. Then treat the whole right side
of Eq. (2.12) as known so it goes in the b vector. Then you can solve
the equation to find an improved guess for y(x). Use this better guess
to rebuild b (again treating the right side of Eq. (2.12) as known), and
then re-solve to get and even better guess. Keep iterating until your
y(x) converges to the desired level of accuracy. This happens when
your y(x) satisfies (2.11) to a specified criterion, not when the change
in y(x) from one iteration to the next falls below a certain level. Iterate
until your RMS error is less than 10−5 , as described in the hint below.
HINT: An appropriate error vector would be
err = A@y-(1-np.sin(y))
(remember that @ is the matrix multiplication operator). Compare
this to Eq. (2.12) and convince yourself that the entire vector err will
be zero when the equation is exactly solved. We’ll compute the RMS
error at interior points only, like this
rmserr = np.sqrt(np.mean(err[1:-2]**2))
because the end points don’t satisfy the differential equation. Use
rmserr as your check condition.
Lab 3
The Wave Equation: Steady State and Resonance

Python Mechanics
P3.1 Work through Chapter 3 of Introduction to Python, where you will learn
about user-defined functions and good commenting practices. For the
remainder of the course, you will need to demonstrate good commenting
practices in your code before your exercises will be passed off.

Wave Equation
We are now ready to tackle our first partial differential equation: the wave equa-
tion. For a string of length L fixed at both ends with a force applied to it that varies
sinusoidally in time, the wave equation can be written as

∂2 y ∂2 y
µ = T + f (x) cos ωt ; y(0, t ) = 0, y(L, t ) = 0 (3.1)
∂t 2 ∂x 2
where y(x, t ) is the lateral displacement of the string as a function of position
and time, assuming that y(x, t ) ¿ L.1 We have written Eq. (3.1) in the form of
Newton’s second law, F = ma. The “ma” part is on the left of the equation, except
that µ is not the mass, but rather the linear mass density (mass per length). This
means that the right side should have units of force/length, and it does because
T is the tension (force) in the string and ∂2 y/∂x 2 has units of 1/length. Finally,
f (x) is the amplitude of the driving force (in units of force/length) applied to the
string as a function of position (so we are not necessarily just wiggling the end of
the string) and ω is the frequency of the driving force.
Before calculating, let’s train our intuition to guess how the solutions of this
equation behave. If we suddenly started to push and pull on a string under tension
with force density f (x) cos(ωt ), we would launch traveling waves, which would
reflect back and forth on the string as the driving force continued to launch more
waves. The string motion would rapidly become very messy. But suppose that
there is a bit of damping in the system (not included in the equation above, but in
Lab 5 we will add it). With damping, all of the transient waves due to the initial
launch and subsequent reflections would die away and we would be left with a
steady-state oscillation of the string at the driving frequency ω. This behavior is
the wave equation analog of damped transients and the steady final state of a
driven harmonic oscillator which you studied in Physics 330.
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 87-110.

13
14 Computational Physics 430

Steady state solution by separation of variables

Let’s look for this steady-state solution by guessing that the solution has the form

y(x, t ) = g (x) cos(ωt ) . (3.2)

This function has the form of a spatially dependent amplitude g (x) which oscil-
lates in time at the frequency of the driving force. Substituting this “guess” into the
wave equation and carrying out the derivatives yields (after some rearrangement)

T g 00 (x) + µω2 g (x) = − f (x) ; g (0) = 0, g (L) = 0 (3.3)

This is just a two-point boundary value problem of the kind we studied in Lab 2,
so we can solve it using our matrix technique.
−4
x 10
P3.2 (a) Write a program to solve Eq. (3.3) for N = 100 points with µ = 0.003,
4 T = 127, L = 1.2, and ω = 400. All quantities are in SI units. Find the
3 g(x) steady-state amplitude associated with the driving force density:
2 ½
1 0.73 if 0.8 ≤ x ≤ 1
f (x) = (3.4)
0
0 otherwise
0 0.2 0.4 0.6 0.8 1
x
which is something like grabbing the string toward one end and wig-
Figure 3.1 Solution to P3.2(a) gling. Plot g (x), and properly label the axes of your plot.
(b) Take your code from (a) and turn it into a library with two functions:
i. A function force that accepts the grid x as an argument and
returns the column vector representing f (x)
ii. A function steadySol that accepts the arguments f (x) (returned
from force), h, ω, T , and µ. This function should construct the
matrix, solve the matrix equation, and return g (x).
Save this library as “Lab3Funcs.py” and then write another program
that imports this library. In this program, write a for loop that sweeps
the value of ω from ω = 400 to ω = 1700 in 200 steps keeping the
values of the other parameters constant. Plot g (x) at each frequency
so that you make an animation of what happens when you sweep the
driving frequency. Here is some example code to help you make the
animation, assuming you’ve stored your ω values in wplot:
plt.figure(1)
for n in range(len(wplot)):
w = wplot[n]
g = l3.steadySol(f,h,w,T,u)
plt.clf() # Clear the previous plot
Figure 3.2 Photographs of the first plt.plot(x,g)
three resonant modes for a string plt.title('$\omega={:1.2e}$'.format(w))
fixed at both ends. plt.xlabel('x')
plt.ylim([-0.05, 0.05]) # prevent auto-scaling
plt.draw() # Request to draw the plot now
plt.pause(0.1) # Give the computer time to draw it
Lab 3 The Wave Equation: Steady State and Resonance 15

Maximum Amplitude
0.06
At certain frequencies, you should see distinct resonance modes ap-
0.05
pear as in Fig. 3.2.
0.04
(c) Modify your code from part (b) so that you find and record the maxi- 0.03
mum amplitude of g (x) at each ω. Then plot the maximum amplitude 0.02
as a function of ω to see the resonance behavior of this system. De- 0.01
scribe what the peaks in this plot mean. 0.00
400 600 800 1000 1200 1400 1600

The height of the peaks in this Fig. 3.3 isn’t meaningful—the height of the
actual peak is essentially infinite (since we don’t have any damping yet), while Figure 3.3 Solution to prob-
the height displayed on the plot is just a reflection of how closely our chosen lem 3.2(c).
frequencies happened to line up with the exact resonance frequency. Now we will
learn how to find these resonant frequencies directly without needing to solve
the differential equation over and over again.

Resonance and the eigenvalue problem

The essence of resonance is that at certain frequencies a large steady-state ampli-
tude is obtained with a very small driving force. To find these resonant frequencies
we seek solutions of Eq. (3.3) for which the driving force is zero. With f (x) = 0,
Eq. (3.3) takes on the form

T g 00 (x) + µω2 g (x) = 0 ; g (0) = 0, g (L) = 0 (3.5)

If we rewrite this equation in the form

µω2
µ ¶
g 00 (x) = − g (x) (3.6)
T

then we see that it is in the form of a classic eigenvalue problem:

Ag = λg (3.7)

where A is a linear operator (the second derivative on the left side of Eq. (3.6))
and λ is the eigenvalue (−µω2 /T ).
Equation (3.6) is easily solved analytically, and its solutions are just the familiar
sine and cosine functions. The boundary condition g (0) = 0 tells us to try a sine
function form, g (x) = g 0 sin(kx). If we substitute this form into Eq. (3.6, we find
that it works, provided that k satisfies k = ω µ/T . We the have
p

µ
µ r ¶
g (x) = g 0 sin ω x (3.8)
T

where g 0 is the arbitrary amplitude. When we apply the boundary condition

g (L) = 0, we find that the resonant frequencies take on discrete values given by
s
π T
ω=n (3.9)
L µ
16 Computational Physics 430

where n is an integer. Each value of n gives a specific resonance frequency from

Eq. (3.9) and a corresponding spatial mode g (x) given by Eq. (3.8).
For this simple example we were able to do the eigenvalue problem analyti-
cally, but when the differential equation is not so simple we will need to do the
eigenvalue calculation numerically. Let’s see practice the numerical calculations
in this case where we know the answer. Rewriting Eq. (3.5) in matrix form using
finite differences, as we learned to do in the last lab, yields

Ag = λg (3.10)

where λ = −µω2 T . With finite differencing, this becomes the matrix equation
±
    
? ? ? ? ... ? ? ? g0 ?
 1 −2 1
0 ... 0 0 0   g1   g1 
 h2 h2 h2    
1 2 1
 0 − ... 0 0 0 g g
    
h2 h2 h2
  2   2 
    
 .
 . . . ... . . . 
 . 
 = λ
 . 
 (3.11)
 .
 . . . ... . . . 
 . 


 . 

 .

. . . ... . . . 
 . 


 . 

 0 0 0 0 ... h12 − h22 h12   g N −2   g N −2 
    

? ? ? ? ... ? ? ? g N −1 ?
The question marks in the first and last rows remind us that we have to invent
something to put in these rows to implement the boundary conditions. The
answer we are seeking is the function g, and it appears on both sides of the
equation. Thus, the question marks in the g -vector on the right are a real problem
because without g 0 and g N −1 , we don’t have an eigenvalue problem (i.e. g on left
side of Eq. (3.11) is not the same as g on the right side).
The simplest way to deal with this question-mark problem and to also handle
the boundary conditions is to change the form of Eq. (3.7) to the slightly more
complicated form of a generalized eigenvalue problem, like this:

Ag = λBg (3.12)

where B is another matrix, whose elements we will choose to make the boundary
conditions come out right. To see how this is done, here is the generalized modifi-
cation of Eq. (3.11) with B and the top and bottom rows of A chosen to apply the
boundary conditions g (0) = 0 and g (L) = 0:

A g =λ B g
1 0 0 ... 0 0
     
g0 0 0 0 ... 0 0 g0
1

 h2
− h22 1
h2
... 0 0 
 g1 


 0 1 0 ... 0 0 
 g1 

1

 0 h2
− h22 ... 0 0 
 g2 


 0 0 1 ... 0 0 
 g2 

. . . ... . . . . . . ... . . .
     
 = λ
     
. . . ... . . . . . . ... . . .
   
     
     
 . . . ... . .  .   . . . ... . .  . 
− h22 1  g
     
 0 0 0 ... N −2   0 0 0 ... 1 0   g N −2 
h2
0 0 0 ... 0 1 g N −1 0 0 0 ... 0 0 g N −1
(3.13)
Lab 3 The Wave Equation: Steady State and Resonance 17

Note that B is just the identity matrix (made with NumPy’s np.identity(N))
except that the first and last rows are completely filled with zeros. Take a minute
to do the matrix multiplications corresponding the first and last rows and verify
that they correctly give g 0 = 0 and g N −1 = 0 no matter what the eigenvalue λ turns
out to be.
To numerically solve the generalized eigenvalue problem you do the following:

• Load a NumPy array A with the matrix on the left side of Eq. (3.13) and an
array B with the matrix on the right side.

• Use SciPy’s generalized eigenvalue and eigenvector command:

import scipy.linalg as la
vals,vecs = la.eig(A,B)

which returns the eigenvalues λ as the entries of the matrix vals and the
eigenvectors as the columns of the square matrix vecs. These column
in this array are the amplitude functions g n = g (x n ) associated with each
eigenvalue on the grid x n .

• Convert eigenvalues λ to frequencies via ω2 = −T λ/µ, sort the squared

frequencies in ascending order, like this.
1
import numpy as np
# Compute the eigen-frequencies
g(x)
w = np.sqrt(-T*np.real(vals)/u) 0

# Sort the eigenvalues and eigenvectors n=1

−1
ind = np.argsort(w) 0 0.5 1
w=w[ind] x
vecs = vecs[:,ind] 1
g(x)
The eigenvalues come back as complex numbers, even though the imag-
inary parts are all zero, so we use the np.real(vals) function to switch 0

them back to usual floats. The algorithm np.argsort returns an array of

n=2
indices showing how the array w should be rearranged to be in ascending −1
0 0.5 1
order. The next two lines of code rearrange the w array and the associated x
columns in the vecs array so that they are sorted in ascending order.
1

P3.3 (a) Write a program to numerically find the eigenvalues and eigenvectors g(x)
of Eq. (3.5) using the procedure outlined above. Use N = 30, µ = 0.003, 0

T = 127, and L = 1.2. Write a loop that plots each of the eigenvectors.
n=3
You will find two infinite eigenvalues together with odd-looking eigen- −1
0 0.5 1
vectors that don’t satisfy the boundary conditions. These two show up x
because of the two rows of the B matrix that are filled with zeros. They
are numerical artifacts with no physical meaning, so just ignore them. Figure 3.4 The first three eigen-
The eigenvectors of the higher modes start looking jagged. These functions found in 3.3. The points
must also be ignored because they are poor approximations to the are the numerical eigenfunctions
and the line is the exact solution.
continuous differential equation in Eq. (3.5).
18 Computational Physics 430

(b) A few of the smooth eigenfunctions are very good approximations.

Plot the eigenfunctions corresponding to n = 1, 2, 3 and compare them
with the exact solutions in Eq. (3.8). Since the modes include an
arbitrary scaling g 0 , you won’t find the amplitudes match unless you
choose an appropriate value for g 0 . Calculate the exact values for ω
using Eq. (3.9) and compare them with the numerical eigenvalues.
Now compare your numerical eigenvalues for the n = 20 mode with
the exact solution. What is the trend in the accuracy of the eigenvalue
method?
(c) The first few values for ω should match the resonances that you found
in 3.2(b). Go back to your calculation in 3.2(b) and make two plots
of the steady state amplitude for driving frequencies near these two
resonant values of ω. For each plot, choose a small range of frequen-
cies that brackets the resonance frequency above and below. You
should find very large amplitudes, indicating that you are right on the
resonances.

Finally let’s explore what happens to the eigenmode shapes when we change
the boundary conditions.

1 P3.4 (a) Change your program from problem 3.3 to implement the boundary
condition
0 g(x) g 0 (L) = 0
This boundary condition describes what happens if one end of the
−1 n=1 string is free to slide up and down rather than attached to a fixed
0 0.5 1
x position. Use the approximation you derived in problem 2.4(b) for the
derivative g 0 (L) to implement this boundary condition, i.e.
1

1 2 3
g(x) g 0 (L) ≈ g N −3 − g N −2 + g N −1
0 2h h 2h
Explain physically why the resonant frequencies change as they do.
−1 n=2
0 0.5 1 (b) In some problems mixed boundary conditions are encountered, for
x example
1 g 0 (L) = 2g (L)
g(x) Find the first few resonant frequencies and eigenfunctions for this
0 case. Look at your eigenfunctions and verify that the boundary condi-
tion is satisfied. Also notice that one of your eigenvalues corresponds
−1 n=3 to ω2 being negative. This means that the nice smooth eigenfunction
0 0.5 1 associated with this eigenvalue is not physical in our wave resonance
x
problem with this boundary condition. The code snippet we gave you
Figure 3.5 The first three eigen- above will have trouble with the np.sqrt. You can just look at the
functions for 3.4(a). values of ω2 instead
Lab 4
The Hanging Chain and Quantum Bound States

Resonance for a hanging chain

In the last lab, we studied waves on a string with constant tension and observed ceiling
sinusoidal normal modes of vibration. We’ll start off this lab by studying the
problem of standing waves on a hanging chain. It was the famous Swiss math- x=L
ematician Johann Bernoulli who discovered in the 1700s that a draped hanging
chain has the shape of a “catenary”, or the hyperbolic cosine function. The prob-
lem of the normal mode frequencies of a vertical hanging chain was solved by
Johann’s son, Daniel Bernoulli, and is the first time that the function that later
became known as the J 0 Bessel function showed up in physics.
For a hanging chain, the tension varies with position—the tension at the top
is large (since it supports the weight of the whole chain) and the tension at the
bottom is essentially zero.1 We are going to find its normal modes of vibration of a
hanging chain using the method of Problem 3.3. The wave equation for transverse
waves on a chain with varying tension T (x) and constant linear mass density µ is
given by2
∂2 y ∂ ∂y
µ ¶
µ 2− T (x) =0 (4.1)
∂t ∂x ∂x
We’ll use a coordinate system that defines x = 0 as the bottom of the chain and
x = L as the ceiling. x=0

P4.1 Use the fact that a stationary hanging chain is in equilibrium to draw a
free-body diagram for a link at an arbitrary x. Use this diagram to show that
the tension in the chain as a function of x is given by
Figure 4.1 The first normal mode
for a hanging chain.
T (x) = µg x (4.2)

where µ is the linear mass density of the chain and g = 9.8 m/s2 is the
acceleration of gravity. Then show that Eq. (4.1) reduces to

∂2 y ∂ ∂y
µ ¶
−g x =0 (4.3)
∂t 2 ∂x ∂x

for a freely hanging chain.

1 For more analysis of the hanging chain, see N. Asmar, Partial Differential Equations and

Boundary Value Problems (Prentice Hall, New Jersey, 2000), p. 299-305.

2 Equation (4.1) also works for systems with varying mass density if you replace µ with a function

µ(x), but the equations derived later in the lab are more complicated with a varying µ(x).

19
20 Computational Physics 430

As in Lab 3, we now look for normal modes by separating the variables:

y(x, t ) = f (x) cos (ωt ). We then substitute this form for y(x, t ) into (4.3) and sim-
plify to obtain
d2 f d f ω2
x + = − f (4.4)
d x2 d x g
which is in eigenvalue form with λ = −ω2 g . The boundary condition at the
±

ceiling is f (L) = 0 while the boundary condition at the bottom is obtained by

taking the limit of Eq. (4.4) as x → 0 to find

ω2
f 0 (0) = − f (0) = λ f (0) (4.5)
g

In the past couple labs we dealt with derivative boundary conditions by fitting
a parabola to the last three points on the grid and then taking the derivative of
0 L
the parabola (e.g. Problems 2.4(b) and 3.4). This time we’ll handle the derivative
Figure 4.2 A cell-centered grid boundary condition by using a cell-centered grid with ghost points, as discussed
with ghost points. (The open cir- in Lab 1. Recall that a cell-center grid divides the spatial region from 0 to L into
cles are the ghost points.) N cells with a grid point at the center of each cell. We then add two more grid
points outside of [0, L], one at x 0 = −h/2 and the other at x N +1 = L + h/2. The
ghost points are used to apply the boundary conditions. By defining N as the
number of interior grid points (or cells), we have N + 2 total grid points, which
ceiling may seem weird to you. We prefer it, however, because it reminds us that we are
using a cell-centered grid with N physical grid points and two ghost points.
x=L
With this grid there isn’t a grid point at each endpoint, but rather we have two
grid points that straddle each endpoint. If the boundary condition specifies a
value, like f (L) = 0 in the problem at hand, we use a simple average like this:

f N +1 + f N
=0 . (4.6)
2

For f 0 (L) = 0, a centered derivative around x = L yields

f N +1 − f N
=0 . (4.7)
h

P4.2 (a) On paper, write out the discretized version of Eq. (4.4) and put it in
the form of a generalized eigenvalue problem

A f = λB f (4.8)
x=0
Remember that for the interior points, the matrix B is just the identity
matrix with 1 on the main diagonal and zeros everywhere else. De-
cide on the values needed in the bottom rows of A and B to give the
Figure 4.3 The shape of the sec- boundary condition in Eq. (4.6) at x = L (the ceiling) no matter what
ond mode of a hanging chain λ turns out to be.
Lab 4 The Hanging Chain and Quantum Bound States 21

(b) Now let’s decide how to handle the derivative boundary condition at
x = 0 (the bottom of the chain), given by Eq. (4.5). Since this condition
is already in eigenvalue form we don’t load the top row of B with zeros.
Instead we load A with the left operator f 0 (0) according to Eq. (4.7)
ceiling
and B with the right operator f (0) according to Eq. (4.6). Leave the
eigenvalue λ = −ω2 /g out of the top row of B since the matrix equation x=L
A f = λB f already has the λ multiplying B. Write down the values
needed in the top rows of A and B, perform the matrix multiplication
for this row, and verify that your choices correctly produce Eq. (4.5).
(c) Write a program to load the matrices A and B with L = 2 m (or the mea-
sured length if different). Then solve for the normal modes of vibration
of a hanging chain. As in Lab 3, some of the eigenvectors are not phys-
ical because they don’t satisfy the boundary conditions; ignore them.
Compare your numerical resonance frequencies to measurements
made on the chain hanging from the ceiling in the classroom.
(d) The analytic solution to Eq. (4.4) without any boundary conditions is
³ p ´ ³ p ´
f (x) = c 1 J 0 2ω x/g + c 2 Y0 2ω x/g

where J 0 and Y0 are the Bessel functions of the first and second kind,
respectively. The boundary condition at x = 0 rules out Y0 , since it is
singular at x = 0. Apply the condition f (L) = 0 to find analytically the x=0
mode frequencies ωi in terms of the values x i that satisfy J 0 (x i ) = 0.
Verify that these values agree with the ω values from part (c).
HINT: The scipy.special library has a function jn_zeros(n,i)
that will return the first i values of x for which J n (x) is zero. Figure 4.4 The shape of the third
mode of a hanging chain

Quantum bound states

Now let’s jump forward several centuries in physics history and study bound
quantum states using the same techniques we used to study the modes of a
hanging chain. Schrödinger’s equation for a potential well V (x) is

∂Ψ ħ2 ∂2 Ψ
iħ =− + V (x)Ψ (4.9)
∂t 2m ∂x 2
If we assume a separable solution of the form Ψ(x, t ) = ψ(x)e −i E t /ħ and plug this
into Eq. (4.9), we find the time-independent Schrödinger equation

ħ2 d 2 ψ
− + V (x)ψ = E ψ (4.10)
2m d x 2
For a particle in a one-dimensional harmonic oscillator, with V (x) = kx 2 /2, this
becomes
ħ2 d 2 ψ 1 2
− + kx ψ = E ψ (4.11)
2m d x 2 2
22 Computational Physics 430

with boundary conditions ψ = 0 at ±∞. Note that k is not the wave number; it is
the spring constant, F = −kx, with units of Newtons/meter.
The numbers that go into Schrödinger’s equation are so small that it makes
it difficult to tell what size of grid to use. For instance, using lengths like 2, 5, or
10 meters would be completely ridiculous for the bound states of an atom where
the typical size is on the order of 10−10 m. Some physicists just set ħ, m, and k
to unity, but this is bad practice. If you do this, you won’t know what physical
parameters your numerical results describe. The correct procedure is to “rescale”
the problem.
The goal of rescaling is to replace physical variables like x with unitless vari-
ables like ξ = x/a, where a is a “characteristic” length written in terms of the other
variables in the problem. Since a is a length, ξ is unitless, and since a is scaled
to the problem parameters, ξ will typically have magnitudes around the size of 1.
Let’s practice this procedure for the problem at hand.

P4.3 Substitute x = aξ into Eq. (4.11), and then do some algebra to put the
equation in the form
C d 2ψ 1 2 E
− + ξ ψ= ψ (4.12)
2 dξ 2 2 Ē
where the constants C and Ē involve factors like ħ, m, k, and a.
Now make the differential operator on the left be as simple as possible by
choosing to make C = 1. This determines how the characteristic length a
depends on ħ, m, and k. Once you have determined a in this way, check to
see that it has units of length. You should find

¶1/4 s s
ħ2 k
µ
ħ
a= = , where ω = (4.13)
km mω m

Finally, rescale the energy by writing introducing a new variable ² = E /Ē .

Show that Ē has units of energy, so that ² is unitless. You should find that
s
k
Ē = ħ (4.14)
m

The final scaled version of Schrödinger’s equation then becomes

1 d 2ψ 1 2
− + ξ ψ = ²ψ (4.15)
2 d ξ2 2

When you solve this equation and find results in terms of ² and ξ, you can
use the equations above to translate them into real-world values of E and x.
Figure 4.5 The probability distri-
butions for the ground state and Now that Schrödinger’s equation is in dimensionless form, we are ready to
the first three excited states of the solve it with out standard technique.
harmonic oscillator.
Lab 4 The Hanging Chain and Quantum Bound States 23

P4.4 Discretize Eq. (4.15) on paper. Then write a program to do this eigenvalue
problem, similar to the other programs we’ve written recently. The bound-
ary conditions are that both the value and derivative of ψ should go to zero
at ξ = ±∞. Since you’ve scaled the problem, it makes sense to choose a
cell-edge grid that goes from ξ = −5 to ξ = 5, or some other similar pair
of numbers. These numbers are supposed to approximate infinity in this
problem. Just set the value of ψ to zero at the edge-points and make sure
(by looking at the eigenfunction solutions) that your grid is large enough
that the wave function has zero slope at the edges of the grid.

(a) Plot the first several bound states. As a guide, Figure 4.5 displays the square
of the wave function for the first few excited states. (The amplitude has
been appropriately normalized so that |ψ(x)|2 = 1
R

(b) If you look in a quantum mechanics textbook you will find that the bound
state energies for the simple harmonic oscillator are given by the formula
s
1 k 1
E n = (n + )ħ = (n + )Ē (4.16)
2 m 2

so that the dimensionless energy eigenvalues ²n are given by

1
²n = n + (4.17)
2
Verify that this formula for the bound state energies is correct for n =
0, 1, 2, 3, 4

P4.5 Now redo this entire problem, but with the harmonic oscillator potential
replaced by
V (x) = µx 4 (4.18)
so that we have
ħ2 d 2 ψ
− + µx 4 ψ = E ψ (4.19)
2m d x 2
With this new potential you will need to find new formulas for the character-
istic length and energy so that you can use dimensionless scaled variables as
you did with the harmonic oscillator. Choose a so that your scaled equation
is
1 d 2ψ
− + ξ4 ψ = ²ψ (4.20)
2 d ξ2
with E = ²Ē . Use algebra by hand to show that
¶1/6 ¶1/3
ħ2 ħ4 µ
µ µ
a= Ē = (4.21) Figure 4.6 The probability distri-
mµ m2 butions for the ground state and
the first three excited states for the
Find the first 5 bound state energies by finding the first 5 values of ²n in the potential in Problem 4.5.
formula E n = ²n Ē .
Lab 5
Animating the Wave Equation: Staggered Leapfrog

In the last couple of labs we handled the wave equation by Fourier analysis,
turning the partial differential equation into a set of ordinary differential equa-
tions using separation of variables.1 But separating the variables and expanding
in orthogonal functions is not the only way to solve partial differential equations,
and in fact in many situations this technique is awkward, ineffective, or both. In
this lab we will study another way of solving partial differential equations using a
spatial grid and stepping forward in time. As an added attraction, this method
automatically supplies a beautiful animation of the solution. There are several
algorithms of this type that can be used on wave equations, so this is just an
introduction to a larger subject. The method we will show you here is called
staggered leapfrog; it is the simplest good method that we know.

The wave equation with staggered leapfrog

Consider again the classical wave equation with wave speed c.

∂2 y 2
2∂ y
− c =0 (5.1)
∂t 2 ∂x 2
p
For a string, the wave speed is related to tension and density via c = T /µ. The
boundary conditions are usually either of Dirichlet type (values specified):

y(0, t ) = f left (t ) ; y(L, t ) = f right (t ) (5.2)

or of Neumann type (derivatives specified):

∂y ∂y
(0) = g left (t ) ; (L) = g right (t ) (5.3)
∂x ∂x
Sometimes mixed boundary conditions specify a relation between the value
and derivative, as at the bottom of the hanging chain. In any case, some set of
conditions at the endpoints are required to solve the wave equation. It is also
necessary to specify the initial state of the string, giving its starting position and
velocity as a function of position:

∂y(x, t )
y(x, t = 0) = y 0 (x) ; |t =0 = v 0 (x) (5.4)
∂t
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 87-110.

25
26 Computational Physics 430

Both of these initial conditions are necessary because the wave equation is second
order in time, just like Newton’s second law, so initial displacement and velocity
must be specified at each point to find a unique solution.
To numerically solve the classical wave equation via staggered leapfrog we
approximate both the temporal and spatial derivatives in Eq. (5.1) with centered
finite differences, like this:
n+1 n n−1
∂2 y y j − 2y j + y j
≈
∂t 2 τ2 (5.5)
n n n
∂2 y y j +1 − 2y j + y j −1
≈
∂x 2 h2
In this notation, spatial position is indicated by a subscript j , referring to grid
points x j , while position in time is indicated by superscripts n, referring to time
points t n so that y(x j , t n ) = y nj . The time steps and the grid spacings are assumed
to be uniform with time step called τ and grid spacing called h. The staggered
leapfrog algorithm aims to find y j one time step into the future, denoted by y n+1 j
,
from the current and previous values of y j . To derive the algorithm put Eqs. (5.5)
into Eq. (5.1) and solve for y n+1 j
to find

c 2 τ2 ³ n ´
y n+1
j = 2y nj − y n−1
j + y j +1 − 2y n
j + y n
j −1 (5.6)
h2
P5.1 Derive Eq. (5.6) using the approximate second derivative formulas. This is
really simple, so just do it on paper.

Equation (5.6) can only be used at interior spatial grid points because the
j + 1 or j − 1 indices reach beyond the grid at the first and last grid points. The
behavior of the solution at these two end points is determined by the boundary
conditions. Since we will want to use both fixed value and derivative boundary
conditions, let’s use a cell-centered grid with ghost points (with N cells and N + 2
grid points) so we can easily handle both types without changing our grid. If the
values at the ends are specified we have

y 0n+1 + y 1n+1
= f left (t n+1 ) ⇒ y 0n+1 = −y 1n+1 + 2 f left (t n+1 )
2 (5.7)
n+1 n+1
y N +1 + y N n+1 n+1
= f right (t n+1 ) ⇒ yN +1 = −y N + 2 f right (t n+1 )
2
If the derivatives are specified then we have

y 1n+1 − y 0n+1
= g left (t n+1 ) ⇒ y 0n+1 = y 1n+1 − hg left (t n+1 )
h
n+1 n+1
(5.8)
y N +1 − y N n+1 n+1
= g right (t n+1 ) ⇒ yN +1 = yN + hg right (t n+1 )
h
To use staggered leapfrog, we first advance the solution at all interior points to
the next time step using Eq. (5.6), then we apply the boundary conditions using
Lab 5 Animating the Wave Equation: Staggered Leapfrog 27

the appropriate equation from Eqs. (5.7)-(5.8) to find the values of y at the end
points, and then we are ready to take another time step.
The staggered leapfrog algorithm in Eq. (5.6) requires not just y at the current
time level y nj but also y at the previous time level y n−1
j
. This means that we’ll need
to keep track of three arrays: an array y for the current values y nj , an array yold
for the values at the previous time step y n−1
j
, and an array ynew for the values
at the next time step y n+1
j
. At time t = 0 when the calculation starts, the initial
position condition gives us the current values y nj , but we’ll have to make creative
use of the initial velocity condition to create an appropriate yold to get started.
To see how this works, let’s denote the initial values of y on the grid by y 0j , the
values after the first time step by y 1j , and the unknown previous values (yold) by
y −1
j
. A centered time derivative at t = 0 turns the initial velocity condition from
Eq. (5.4) into
y 1j − y −1
j
= v 0 (x j ) (5.9)
2τ
This gives us an equation for the previous values y −1
j
, but it is in terms of the
still unknown future values y 1j . However, we can use Eq. (5.6) to obtain another
relation between y 1j and y −1
j
. Leapfrog at the first step (n = 0) says that

c 2 τ2 ³ 0 ´
y 1j = 2y 0j − y −1
j + y j +1 − 2y 0
j + y 0
j −1 (5.10)
h2

If we insert this expression for y 1j into Eq. (5.9), we can solve for y −1
j
in terms of
known quantities:

0 c 2 τ2 ³ 0 ´
y −1
j = y j − v 0 (x j )τ + 2
y j +1 − 2y 0j + y 0j −1 (5.11)
2h

P5.2 Derive Eq. (5.11) from Eqs. (5.9) and (5.10). Again, just use paper and pencil.

OK; we are now ready to code the staggered leapfrog algorithm.

P5.3 (a) Start by making a cell-centered x grid with ghost points over the region
0 ≤ x ≤ L, with L = 1 m. Use N = 200 cells, so you have 202 grid
points. Define the initial displacement y as Gaussian bump with 1 cm
amplitude in the middle of the string, like this
y = 0.01 * np.exp(-(x-L/2)**2 / 0.02)
and the initial velocity vy to be zero everywhere along the string. Used
fixed-value boundary conditions, with y(0) = 0 and y(L) = 0. Plot the
initial position just to make sure your grid is right and that your initial
position array looks reasonable
(b) Assume that the wave speed on this string is c = 2 m/s, and pick
the time step as tau = 0.2*h/c. (We’ll explore this choice more
later.) Then create a variable yold using Eq. (5.11), and enforce the
28 Computational Physics 430

boundary conditions on yold. As you write this code, don’t write a

bunch of for loops to iterate through all the points. Instead, assign all
interior points at once using the NumPy array colon indexing method
(e.g. y[1:-1] accesses all the interior points of y) and then set the
boundary conditions explicitly.
(c) Now it is time to code the main staggered leapfrog algorithm and make
an animation of the solution. Since it has been a couple of labs since
we made an animation, here is a framework for the code to remind
you of the basic animation commands:
ynew = np.zeros_like(y)
j = 0
t = 0
1 t=0 tmax = 2
plt.figure(1) # Open the figure window
0
# the loop that steps the solution along
−1 while t < tmax:
0 0.5 1 j = j+1
x t = t + tau
1 t = 0.1
# Use leapfrog and the boundary conditions to load
0
# ynew with y at the next time step using y and yold

# update yold and y for next timestep

−1 # remember to use np.copy
0 0.5 1
x
# make plots every 50 time steps
1 t = 0.2 if j % 50 == 0:
plt.clf() # clear the figure window
0 plt.plot(x,y,'b-')
plt.xlabel('x')
−1 plt.ylabel('y')
0 0.5 1 plt.title('time={:1.3f}'.format(t))
x plt.ylim([-0.03,0.03])
1 t = 0.3
plt.xlim([0,1])
plt.draw() # Draw the plot
plt.pause(0.1) # Give the computer time to draw
0
The actual staggered leapfrog code is missing above. You’ll need to
−1 write that. Run the animations long enough that you can see the
0 0.5 1 reflection from the ends and the way the two pulses add together and
x
pass right through each other.
Figure 5.1 Snapshots of the evo- (d) Once you have it running, experiment with various time steps τ. Show
lution of a wave on a string with by numerical experimentation that if τ > h/c the algorithm blows up
fixed ends and an initial displace-
spectacularly. This failure is called a numerical instability and we
ment but no initial velocity. (See
will be trying to avoid it all semester. This limit is called the Courant-
Problem 5.3(b))
Friedrichs-Lewy condition, or sometimes the CFL condition, or some-
times (unfairly) just the Courant condition.
Lab 5 Animating the Wave Equation: Staggered Leapfrog 29

0.1 t=0

0.1
0 0.5 1
x
0.1 t = 0.2
Figure 5.2 Richard Courant (left), Kurt Friedrichs (center), and Hans Lewy (right) de-
scribed the CFL instability condition in 1928.
0

∂y
(e) Change the boundary conditions so that ∂x = 0 at each end and watch 0.1
how the reflection occurs in this case. 0 0.5 1
x
(f) Change the initial conditions from initial displacement with zero
0.1 t = 0.4
velocity to initial velocity with zero displacement. Use an initial Gaus-
sian velocity pulse just like the displacement pulse you used earlier
0
and use fixed-end boundary conditions. Watch how the wave motion
develops in this case. (You will need to change the y-limits in the axis
0.1
command to see the vibrations with these parameters.) Then find a
0 0.5 1
slinky, stretch it out, and whack it in the middle to verify that the math x
does the physics right. 0.1 t = 0.6

0
The damped wave equation
0.1
We can modify the wave equation to include damping of the waves using a linear 0 0.5 1
damping term, like this: x
∂2 y ∂y 2
2∂ y
+ γ − c =0 (5.12) Figure 5.3 Snapshots of the evo-
∂t 2 ∂t ∂x 2 lution of a wave on a string with
with c constant. The staggered leapfrog method can be used to solve Eq. (5.12) fixed ends and no initial displace-
also. To do this, we use the approximate first derivative formula ment but with an initial velocity.
(See Problem 5.3(c))
n+1 n−1
∂y y j − y j
≈ (5.13)
∂t 2τ
along with the second derivative formulas in Eqs. (5.5) and find an expression for
the values one step in the future:

2c 2 τ2 ³ n
µ ´¶
n+1 1 n n−1 n−1 n n
yj = 4y j − 2y j + γτy j + y j +1 − 2y j + y j −1 (5.14)
2 + γτ h2

P5.4 (a) Derive Eq. (5.14).

(b) Find a new formula for the initial value of yold using Eqs. (5.9) and
(5.14). When you get the answer, ask your TA or instructor to check to
see if you got it right.
30 Computational Physics 430

(c) Modify your staggered leapfrog code to include damping with γ = 0.2.
Max Amplitude vs. Time
Then run your animation with the initial conditions in Problem 5.3(c)
0.06
and verify that the waves damp away. You will need to run for about
25 s and you will want to use a big skip factor so that you don’t have
0.04 to wait forever for the run to finish. Include some code to record the
maximum value of y(x) over the entire grid as a function of time and
0.02
then plot it as a function of time at the end of the run so that you can
0 see the decay caused by γ. The decay of a simple harmonic oscillator
0 5 10 15 20 25
is exponential, with amplitude proportional to e −γt /2 . Scale this time
Figure 5.4 The maximum ampli- decay function properly and lay it over your maximum y plot to see if
tude of oscillation decays expo- it fits. Can you explain why the fit is as good as it is? (Hint: think about
nentially for the damped wave doing this problem via separation of variables.)
equation. (Problem 5.4(c))

The damped and driven wave equation

Finally, let’s look at what happens when we add an oscillating driving force to our
string, so that the wave equation becomes

∂2 y ∂y 2
2∂ y f (x)
+ γ − c = cos(ωt ) (5.15)
∂t 2 ∂t ∂x 2 µ

At the beginning of Lab 3 we discussed the qualitative behavior of this system.

Recall that if we have a string initially at rest and then we start to push and pull on
a string with an oscillating force/length of f (x), we launch waves down the string.
These waves reflect back and forth on the string as the driving force continues
to launch more waves. The string motion is messy at first, but the damping in
the system causes the the transient waves from the initial launch and subsequent
reflections to eventually die away. In the end, we are left with a steady-state
oscillation of the string at the driving frequency ω.
Now that we have the computational tools to model the time evolution of the
system, let’s watch this behavior.

P5.5 (a) Re-derive the staggered leapfrog algorithm to include both driving
and damping forces as in Eq. (5.15).
(b) Modify your code from Problem 5.4 to use this new algorithm. We’ll
have the string start from rest, so you don’t need to worry about finding
y old . Just set y = 0 and y old = 0 and enter the time-stepping loop.
This problem involves the physics of waves on a real guitar string,
so we’ll need to use realistic values for our parameters. Use T = 127,
µ = 0.003, and L = 1.2 (in SI units) and remember that c = T /µ. Use
p

the same driving force as in Problem 3.2(a)

½
0.73 if 0.8 ≤ x ≤ 1
f (x) = (5.16)
0 otherwise
Lab 5 Animating the Wave Equation: Staggered Leapfrog 31

and set the driving frequency at ω = 400. Choose a damping constant x 10

−4

γ that is the proper size to make the system settle down to steady state 5
t = 0.1
after 20 or 30 bounces of the string. (You will have to think about the
value of ω that you are using and about your damping rate result from 0
problem 5.4 to decide which value of γ to use to make this happen.)
−5
Run the model long enough that you can see the transients die away 0 0.5 1
and the string settle into the steady oscillation at the driving frequency. x
−4
You may find yourself looking at a flat-line plot with no oscillation at x 10
5
all. If this happens look at the vertical scale of your plot and remember t = 0.2
that we are doing real physics here. If your vertical scale goes from −1 0
to 1, you are expecting an oscillation amplitude of 1 meter on your
guitar string. Compare the steady state mode to the shape found in −5
Problem 3.2(a) (see Fig. 3.1). 0 0.5 1
x
Then run again with ω = 1080, which is close to a resonance, and again x 10
−4

see the system come into steady oscillation at the driving frequency. 5
t = 0.3

−5
0 0.5 1
x
−4
x 10
5
t = 0.4

−5
0 0.5 1
x

Figure 5.5 Snapshots of the evo-

lution a driven and damped wave
with ω = 400. As the transient
behavior dies out, the oscilla-
tion goes to the resonant mode.
To make the pictures more inter-
esting, the string was not started
from rest in these plots. (In Prob-
lem 5.5 you start from rest for eas-
ier coding.)
Lab 6
The 2-D Wave Equation With Staggered Leapfrog

Two dimensional grids

In this lab we will do problems in two spatial dimensions, x and y, so we need Cell-Edge Grid

to spend a little time thinking about how to represent 2-D grids. For a simple d
rectangular grid where all of the cells are the same size, 2-D grids are pretty
straightforward. We just divide the x-dimension into equally sized regions and
the y-dimension into equally sized regions, and the two one dimensional grids
intersect to create rectangular cells. Then we put grid points either at the corners
of the cells (cell-edge) or at the centers of the cells (cell-centered). On a cell-center
grid we’ll usually want ghost point outside the region of interest so we can get the c
boundary conditions right. a b

NumPy has a nice way of creating rectangular two-dimensional grids using

the meshgrid command. You can create 2-d rectangle defined by x ∈ [a, b] and Cell-Center Grid with Ghost Points
y ∈ [c, d ] and then plot a function on that grid this way:
d
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
import numpy as np

# Make 1D x and y arrays

Nx=20 c
a b
a=-1.5
b=1.5 Figure 6.1 Two types of 2-D grids.
x,hx = np.linspace(a,b,Nx,retstep = True)
Ny=10
c=-1.2
d=1.2
y,hy = np.linspace(c,d,Ny,retstep = True)

# Make the 2D grid and evaluate a function

X, Y = np.meshgrid(x,y,indexing='ij')
Z = X**2 + Y**2

# Plot the function as a surface.

fig = plt.figure(1)
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X, Y, Z, cmap=cm.viridis)
plt.xlabel('x')
plt.ylabel('y')
fig.colorbar(surf)

33
34 Computational Physics 430

The argument indexing='ij' in the meshgrid function switches the ordering

of the elements in the resulting matrices from the matrix indexing convention
Z[row,col] to the function indexing convention Z[x_i,y_y] for representing
Z (x i , y i ).

P6.1 (a) Use the code fragment above in a program to create a 30-point cell-
edge grid in x and a 50-point cell-edge grid in y with a = 0, b = 2, c =
−1, d = 3. Switch back and forth between the argument indexing='ij'
and indexing='xy', and look at the different matrices that result.
Convince the TA that you know what the difference is. (HINT: When
we write matrices, rows vary in the y dimension and columns vary
in the x direction, whereas with Z (x i , y i ) we have a different con-
vention. NumPy’s naming convention seems backward to us, but
we didn’t come up with it. Sorry.) We recommend that you use the
indexing='ij' convention, as it tends to more readable code for
representing Z (x i , y i ).
(b) Using this 2-D grid, evaluate the following function of x and y:
0.5 µ q ¶
−(x 2 +y 2 ) 2 2
0 f (x, y) = e cos 5 x + y (6.1)
−0.5

3
Use the plot_surface command to make a surface plot of this func-
2
tion. Properly label the x and y axes with the symbols x and y, to get
1
2 a plot like Fig. 6.2.
0
1
y
−1 0 x There are a lot more options for plotting surfaces in Python, but we’ll let you
Figure 6.2 Plot from Problem 6.1. explore those on your own. For now, let’s do some physics on a two-dimensional
The graphic in this figure was cre- grid.
ated with Matlab. Python’s graph-
ics engine is sort of privative in
comparison to Matlab’s, so you The two-dimensional wave equation
won’t get something quite so nice.
In particular, getting the scaling The wave equation for transverse waves on a rubber sheet is 1
right is painful in Python.
∂2 z
µ 2
∂ z ∂2 z
¶
µ = σ + (6.2)
∂t 2 ∂x 2 ∂y 2

In this equation µ is the surface mass density of the sheet, with units of mass/area.
The quantity σ is the surface tension, which has rather odd units. By inspecting
the equation above you can find that σ has units of force/length, which doesn’t
seem right for a surface. But it is, in fact, correct as you can see by performing the
following thought experiment. Cut a slit of length L in the rubber sheet and think
about how much force you have to exert to pull the lips of this slit together. Now
imagine doubling L; doesn’t it seem that you should have to pull twice as hard
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 129-134.

Lab 6 The 2-D Wave Equation With Staggered Leapfrog 35

to close the slit? Well, if it doesn’t, it should; the formula for this closing force is
given by σL, which defines the meaning of σ.
We can solve the two-dimensional wave equation using the same staggered
leapfrog techniques that we used for the one-dimensional case, except now we
need to use a two dimensional grid to represent z(x, y, t ). We’ll use the nota-
tion z nj,k = z(x j , y k , t n ) to represent the function values. With this notation, the
derivatives can be approximated as
n+1 n n−1
∂2 z z j ,k − 2z j ,k + z j ,k
≈ (6.3)
∂t 2 τ2
n n n
∂2 z z j +1,k − 2z j ,k + z j −1,k
≈ (6.4)
∂x 2 h x2
n n n
∂2 z z j ,k+1 − 2z j ,k + z j ,k−1
≈ (6.5)
∂y 2 h 2y
where h x and h y are the grid spacings in the x and y dimensions. We insert these
three equations into Eq. (6.2) to get an expression that we can solve for z at the
next time (i.e. z n+1
j ,k
). Then we use this expression along with the discrete version
of the initial velocity condition

z n+1
j ,k
− z n−1
j ,k
v 0 (x j , y k ) ≈ (6.6)
2τ

to find an expression for the initial value of z n−1

j ,k
(i.e. zold) so we can get things
started.

P6.2 (a) Derive the staggered leapfrog algorithm for the case of square cells
with h x = h y = h. Write a program that animates the solution of the
two dimensional wave equation on a square region that is [−5, 5] ×
[−5, 5] and that has fixed edges. Use a cell-edge square grid with
the edge-values pinned to zero to enforce the boundary condition.
Choose σ = 2 N/m and µ = 0.3 kg/m2 and use a displacement initial
condition that is a Gaussian pulse with zero velocity
2
+y 2 )
z(x, y, 0) = e −5(x (6.7)

This initial condition doesn’t strictly satisfy the boundary conditions,

so you should pin the edges to zero.
Getting the animation to work can be tricky, so here is a framework of
code for the animation loop:
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np

# your code to initialize things

36 Computational Physics 430

tfinal=10
t=np.arange(0,tfinal,tau)
skip=10

fig = plt.figure(1)
# here is the loop that steps the solution along
for m in range(len(t)):

# Your code to step the solution

# make plots every skip time steps

if m % skip == 0:
plt.clf()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X,Y,z)
ax.set_zlim(-0.5, 0.5)
plt.xlabel('x')
plt.ylabel('y')
plt.draw()
plt.pause(0.1)
Run the simulation long enough that you see the effect of repeated
reflections from the edges.
(b) You will find that this two-dimensional problem has a Courant condi-
tion similar to the one-dimensional case, but with a factor out front:

µ
r
τ< f h (6.8)
σ

where σ/µ is the wave speed and f is an arbitrary constant. De-

p

termine the value of the constant f by numerical experimentation.

(Try various values of τ and discover where the boundary is between
numerical stability and instability.)
(c) Also watch what happens at the center of the sheet by making a plot
of z(0, 0, t ) there. In one dimension the pulse propagates away from
its initial position making that point quickly come to rest with z = 0.
This also happens for the three-dimensional wave equation. But
something completely different happens for an even number of di-
mensions; you should be able to see it in your plot by looking at the
Figure 6.3 A wave on a rubber behavior of z(0, 0, t ) before the first reflection comes back.
sheet with fixed edges. (d) Finally, change the initial conditions so that the sheet is initially flat
but with the initial velocity given by the Gaussian pulse of Eq. (6.7).
In one dimension when you pulse the system like this the string at
the point of application of the pulse moves up and stays up until
the reflection comes back from the ends of the system. (We did this
experiment with the slinky in Lab 5.) Does the same thing happen
in the middle of the sheet when you apply this initial velocity pulse?
Lab 6 The 2-D Wave Equation With Staggered Leapfrog 37

Answer this question by looking at your plot of z(0, 0, t ). You should

find that the two-dimensional wave equation behaves very differently
from the one-dimensional wave equation.

Elliptic, hyperbolic, and parabolic PDEs

Let’s step back and look at some general concepts related to solving partial differ-
ential equations. Three of the most famous PDEs of classical physics are

(i) Poisson’s equation for the electrostatic potential V (x, y) given the charge
density ρ(x, y)

∂2V ∂2V −ρ
+ = + Boundary Conditions (6.9)
∂x 2 ∂y 2 ²0

(ii) The wave equation for the wave displacement y(x, t )

∂2 y 1 ∂2 y
− =0 + Boundary Conditions (6.10)
∂x 2 c 2 ∂t 2

(iii) The thermal diffusion equation for the temperature distribution T (x, t ) in a
medium with diffusion coefficient D

∂T ∂2 T
=D 2 + Boundary Conditions (6.11)
∂t ∂x

To this point in the course, we’ve focused mostly on the wave equation, but over
the next several labs we’ll start to tackle some of the other PDEs.
Mathematicians have special names for these three types of partial differential
equations, and people who study numerical methods often use these names, so
let’s discuss them a bit. The three names are elliptic, hyperbolic, and parabolic.
You can remember which name goes with which of the equations above by re-
membering the classical formulas for these conic sections:

x2 y 2
ellipse : + =1 (6.12)
a2 b2
x2 y 2
hyperbola : − =1 (6.13)
a2 b2
parabola : y = ax 2 (6.14)

Compare these equations with the classical PDE’s above and make sure you can
use their resemblances to each other to remember the following rules: Poisson’s
equation is elliptic, the wave equation is hyperbolic, and the diffusion equation is
parabolic. These names are important because each different type of equation
requires a different type of algorithm and boundary conditions.
38 Computational Physics 430

Elliptic equations require the same kind of boundary conditions as Poisson’s

equation: V (x, y) specified on all of the surfaces surrounding the region of inter-
est. Notice that there is no time delay in electrostatics. When all of the bounding
voltages are specified, Poisson’s equation says that V (x, y) is determined instantly
throughout the region surrounded by these bounding surfaces. Because of the
finite speed of light this is incorrect, but Poisson’s equation is a good approxima-
tion to use in problems where things happen slowly compared to the time it takes
light to cross the computing region.
To understand hyperbolic boundary conditions, think about a guitar string
described by the transverse displacement function y(x, t ). It makes sense to
give spatial boundary conditions at the two ends of the string, but it makes no
sense to specify conditions at both t = 0 and t = t final because we don’t know the
displacement in the future. This means that you can’t pretend that (x, t ) are like
(x, y) in Poisson’s equation and use “surrounding”-type boundary conditions. But
we can see the right thing to do by thinking about what a guitar string does. With
the end positions specified, the motion of the string is determined by giving it an
initial displacement y(x, 0) and an initial velocity ∂y(x, t )/∂t |t =0 , and then letting
the motion run until we reach the final time. So for hyperbolic equations the
proper boundary conditions are to specify end conditions on y as a function of
time and to specify the initial conditions y(x, 0) and ∂y(x, t )/∂t |t =0 .
Parabolic boundary conditions are similar to hyperbolic ones, but with one
difference. Think about a thermally-conducting bar with its ends held at fixed
temperatures. Once again, surrounding-type boundary conditions are inappro-
priate because we don’t want to specify the future. So as in the hyperbolic case,
we can specify conditions at the ends of the bar, but we also want to give ini-
tial conditions at t = 0. For thermal diffusion we specify the initial temperature
T (x, 0), but that’s all we need; the “velocity” ∂T /∂t is determined by Eq. (6.11),
so it makes no sense to give it as a separate boundary condition. Summarizing:
for parabolic equations we specify end conditions and a single initial condition
T (x, 0) rather than the two required by hyperbolic equations.
If this seems like an arcane side trip into theory, we’re sorry, but it’s important.
When you numerically solve partial differential equations you will spend 10%
of your time coding the equation itself and 90% of your time trying to make the
boundary conditions work. It’s important to understand what the appropriate
boundary conditions are.
Finally, there are many more partial differential equations in physics than
just these three. Nevertheless, if you clearly understand these basic cases you
can usually tell what boundary conditions to use when you encounter a new one.
Here, for instance, is Schrödinger’s equation:

∂ψ ħ2 ∂2 ψ
iħ =− +V ψ (6.15)
∂t 2m ∂x 2
which is the basic equation of quantum (or “wave”) mechanics. The wavy nature
of the physics described by this equation might lead you to think that the proper
boundary conditions on ψ(x, t ) would be hyperbolic: end conditions on ψ and
Lab 6 The 2-D Wave Equation With Staggered Leapfrog 39

initial conditions on ψ and ∂ψ/∂t . But if you look at the form of the equation, it
looks like thermal diffusion. Looks are not misleading here; to solve this equation
you only need to specify ψ at the ends in x and the initial distribution ψ(x, 0), but
not its time derivative.
And what are you supposed to do when your system is both hyperbolic and
parabolic, like the wave equation with damping?

∂2 y 1 ∂2 y 1 ∂y
− − =0 (6.16)
∂x 2 c 2 ∂t 2 D ∂t
The rule is that the highest-order time derivative wins, so this equation needs
hyperbolic boundary conditions.

P6.3 Make sure you understand this material well enough that you are com-
fortable answering basic questions about PDE types and what types of
boundary conditions go with them on a quiz and/or an exam. Then explain
it to the TA to pass this problem off.
Lab 7
The Diffusion Equation and Implicit Methods

Analytic approach to the Diffusion Equation

Now let’s attack the diffusion equation 1

∂T ∂2 T
=D 2 . (7.1)
∂t ∂x
This equation describes how the distribution T (often temperature) diffuses
through a material with a constant diffusion coefficient D. The diffusion equation
can be approached analytically via separation of variables by assuming that T is
of the form T (x, t ) = g (x) f (t ). Plugging this form into the diffusion equation, we
find
1 f˙(t ) g 00 (x)
= (7.2)
D f (t ) g (x)
The left-hand side depends only on time, while the right-hand side depends only
on space, so both sides must equal a constant, say −a 2 . Thus, f (t ) must satisfy

f˙(t ) = −γ f (t ) (7.3)

where γ = a 2 D so that f (t ) is
f (t ) = e −γt . (7.4)
Meanwhile g (x) must satisfy

g 00 (x) + a 2 g (x) = 0 (7.5)

If we specify edge-value boundary conditions so that T (x = 0, t ) = 0 and T (x =

L, t ) = 0 then the solution to Eq. (7.5) is simply

g (x) = sin(ax) (7.6)

and the separation constant can take on the values a = nπ/L, where n is an integer.
Any initial distribution T (x, t = 0) that satisfies these boundary conditions can be
composed by summing these sine functions with different weights using Fourier
series techniques. Notice that higher spatial frequencies (i.e. large n) damp faster,
according to Eq. (7.4). We already studied how to use separation of variables
computationally in the first several labs of this manual, so let’s move directly to
time-stepping methods for solving the diffusion equation.
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 110-129.

41
42 Computational Physics 430

Numerical approach: a first try

Let’s try to solve the diffusion equation on a grid as we did with the wave equation.
If we finite difference the diffusion equation using a centered time derivative and
a centered second derivative in x we get

T jn+1 − T jn−1 D ³ n ´
n n
= T j +1 − 2T j + T j −1 (7.7)
2τ h2

Solving for T jn+1 to obtain an algorithm similar to leapfrog then gives

2Dτ ³ n ´
T jn+1 = T jn−1 + T j +1 − 2T n
j + T n
j −1 (7.8)
h2
There is a problem starting this algorithm because of the need to have T one time
step in the past (T jn−1 ), but even after we work around this problem this algorithm
turns out to be worthless. We won’t make you code it up, but if you did, you’d
find that no matter how small a time step τ you choose, you encounter the same
kind of instability that plagues staggered leapfrog when the step size got too big
(infinite zig-zags). Such an algorithm is called unconditionally unstable, and is an
invitation to keep looking. This must have been a nasty surprise for the pioneers
of numerical analysis who first encountered it.
For now, let’s sacrifice second-order accuracy to obtain a stable algorithm. If
we don’t center the time derivative, but use instead a forward difference we find

T jn+1 − T jn D ³ n ´
n n
= T − 2T + T (7.9)
τ h 2 j +1 j j −1

This algorithm has problems since the left side of Eq. (7.9) is centered at time t n+ 1 ,
2
while the right side is centered at time t n . This makes the algorithm inaccurate,
but it turns out that it is stable if τ is small enough. Solving for T jn+1 yields

Dτ ³ n ´
T jn+1 = T jn + T j +1 − 2T n
j + T n
j −1 (7.10)
h2
P7.1 (a) Modify one of your staggered leapfrog programs that uses a cell-center
grid to implement Eq. (7.10) to solve the diffusion equation on the
0.6
0.4
interval [0, L] with initial distribution
0.2
0 T (x, 0) = sin (πx/L) (7.11)
3
2 2
x 1.5 and boundary conditions T (0) = T (L) = 0. Use D = 2, L = 3, and N =
1 1 t
0 0.5
20. You don’t need to make a space-time surface plot like Fig. 7.1. Just
Figure 7.1 Diffusion of the n = make a line plot that updates each time step as we’ve done previously.
1 sine temperature distribution This algorithm has a CFL condition on the time step τ of the form
given in Problem 7.1(a).
h2
τ≤C (7.12)
D
Lab 7 The Diffusion Equation and Implicit Methods 43

Determine the value of C by numerical experimentation.

Test the accuracy of your numerical solution by overlaying a graph of
the analytic solution. Plot the numerical solution as points and the
exact solution as a line so you can tell the difference. Show that your
grid solution matches the exact solution with increasing accuracy as
the number of grid points N is increased from 20 to 40 and then to 80.
You can calculate the RMS error using something like
error = np.sqrt( np.mean( (T - exact)**2 ))
(b) Get a feel for what the diffusion coefficient does by trying several
different values for D in your code. Give a physical description of this
parameter to the TA.
(c) Now switch your boundary conditions to be insulating, with ∂T /∂x =
0 at both ends. Explain what these two types of boundary conditions 1

mean by thinking about a watermelon that is warmer in the middle

0.5
than at the edge. Tell physically how you would impose both of these
boundary conditions (specifying the value and specifying the deriva- 0
tive) on the watermelon and explain what the temperature history of 10
4 5
the watermelon has to do with your plots of T (x) vs. time. x 5 2 3
0 1 t

Even though this technique can give us OK results, the time step constraint for this Figure 7.2 Diffusion of the Gaus-
method is onerous. The constraint is of the form τ < C h 2 , where C is a constant. sian temperature distribution
Suppose, for instance, that to resolve some spatial feature you need to decrease h given in Problem 7.1(c) with in-
by a factor of 5; then you will have to decrease τ by a factor of 25. This will make sulating boundary conditions.
your code take forever to run, which motivates us to find a better way.

Implicit Methods: the Crank-Nicolson Algorithm

The time-stepping algorithms we have discussed so far are of the same type: at
each spatial grid point j you use present, and perhaps past, values of y(x, t ) at that
grid point and at neighboring grid points to find the future y(x, t ) at j . Methods
like this, that depend in a simple way on present and past values to predict future
values, are said to be explicit and are easy to code. They are also often numerically
unstable, or have severe constraints on the size of the time step.
Implicit methods are generally harder to implement than explicit methods,
but they have much better stability properties. The reason they are harder is that
they assume that you already know the future. To give you a better feel for what
“implicit” means, let’s study the simple first-order differential equation

dy
= −y (7.13)
dt

P7.2 (a) Write a program to solve this equation using Euler’s method:
y n+1 − y n
= −y n . (7.14)
τ
44 Computational Physics 430

4 The program to solve for y using Euler’s method is only a few lines of
y(t) code (like less than 10 lines, including the plot command). Here are
2
the first few lines:
0 tau = 0.5
−2 tmax = 20.
Euler’s Method t = np.arange(0,tmax,tau)
−4 y = np.zeros_like(t)
0 5 10
t Show by numerical experimentation that Euler’s method is unstable
for large τ. You should find that the algorithm that is unstable if τ > 2.
Figure 7.3 Euler’s method is un-
stable for τ > 2. (τ = 2.1 in this Use y(0) = 1 as your initial condition. This is an example of an explicit
case.) method.
(b) Notice that the left side of Eq. (7.14) is centered on time t n+ 1 but the
2
right side is centered on t n . Fix this by centering the right-hand side
at time t n+ 1 by using an average of the advanced and current values
2
of y,
y n + y n+1
yn ⇒ .
2
Modify your program to implement this fix, then show by numerical
1
experimentation that when τ becomes large this method doesn’t blow
0.5 y(t) up. It isn’t correct because y n bounces between positive and negative
values, but at least it doesn’t blow up. The presence of τ in the denom-
0 inator is the tip-off that this is an implicit method, and the improved
stability is the point of using something implicit.
Euler’s Method
−0.5
0 5 10 (c) Now Modify Euler’s method by making it fully implicit by using y n+1
t in place of y n on the right side of Eq. (7.14) (this makes both sides of
the equation reach into the future). This method is no more accurate
Figure 7.4 The implicit method in
than Euler’s method for small time steps, but it is much more stable
7.2(b) with τ = 4.
and it doesn’t bounce between positive and negative values.
Show by numerical experimentation in a modified program that this
1 fully implicit method damps even when τ is large. For instance, see
what happens if you choose τ = 5 with a final time of 20 seconds. The
0.5 y(t) time-centered method of part (b) would bounce and damp, but you
Euler’s Method should see that the fully implicit method just damps. It’s terribly inac-
curate, and actually doesn’t even damp as fast as the exact solution,
0 but at least it doesn’t bounce like part (b) or go to infinity like part (a).
0 5 10 Methods like this are said to be “absolutely stable”. Of course, it makes
t
no sense to choose really large time steps, like τ = 100 when you only
Figure 7.5 The fully implicit want to run the solution out to 10 seconds.
method in 7.2(c) with τ = 2.1.
Lab 7 The Diffusion Equation and Implicit Methods 45

The diffusion equation with Crank-Nicolson

Now let’s look at the diffusion equation again and see how implicit methods can
help us find a stable numerical algorithm. Here’s the equation again:

∂T ∂2 T
=D 2 (7.15)
∂t ∂x
We begin by finite differencing the right side as usual:
∂T j T j +1 − 2T j + T j −1
=D (7.16)
∂t h2
Now we discretize the time derivative by taking a forward time derivative on the
left:
T jn+1 − T jn T jn+1 − 2T jn + T jn−1
=D (7.17) Phyllis Nicolson (1917–1968, English)
τ h2
This puts the left side of the equation at time level t n+ 1 , while the right side is
2
at t n . To put the right side at the same time level (so that the algorithm will be
second-order accurate), we replace each occurrence of T on the right-hand side
by the average
1 T n+1 + T n
T n+ 2 = (7.18)
2
like this:
T jn+1 − T jn T jn+1
+1
+ T jn+1 − 2T jn+1 − 2T jn + T jn+1
−1
+ T jn−1
=D (7.19)
τ 2h 2
If you look carefully at this equation you will see that there is a problem: how are
we supposed to solve for T jn+1 ? The future values T n+1 are all over the place, and
they involve three neighboring grid points (T jn+1
−1
, T jn+1 , and T jn+1
+1
) so we can’t just
solve in a simple way for T jn+1 . This is an example of why implicit methods are John Crank (1916–2006, English)

harder than explicit methods.

In the hope that something useful will turn up, let’s put all of the variables at
time level n + 1 on the left, and all of the ones at level n on the right.
µ 2 ¶ µ 2 ¶
2h 2h
− T jn+1 + + 2 T n+1
j − T n+1
j +1 = T n
j −1 + − 2 T jn + T jn+1 (7.20)
−1
τD τD

We know this looks ugly, but it really isn’t so bad. To solve for T jn+1 we just need to
solve a linear system, as we did in Lab 2 on two-point boundary value problems.
When a system of equations must be solved to find the future values, we say
that the method is implicit. This particular implicit method is called the Crank-
Nicolson algorithm.
To see more clearly what we are doing, and to make the algorithm a bit more
efficient, let’s define a matrix A to describe the left side of Eq. (7.20) and another
matrix B to describe the right side, like this:

AT n+1 = BT n (7.21)
46 Computational Physics 430

T is now a column vector. The elements of A are given by

A j ,k = 0 except for :

2h 2
A j , j −1 = −1 ; A j , j = + 2 ; A j , j +1 = −1 (7.22)
τD
and the elements of B are given by

B j ,k = 0 except for :

2h 2
B j , j −1 = 1 ; B j , j =
− 2 ; B j , j +1 = 1 (7.23)
τD
Once the boundary conditions are added to these matrices, Eq. (7.21) could be
solved symbolically to find T n+1

T n+1 = A−1 BT n . (7.24)

However, since inverting a matrix is computationally expensive we will use Gauss

elimination instead as we did in lab 2 (with SciPy’s linalg.solve function). Here
is a sketch of how you would implement the Crank-Nicolson algorithm in Python.

• Load the matrices A and B as given in Eq. (7.22) and Eq. (7.23) for all of
the rows except the first and last. Since the diffusion coefficient D doesn’t
change with time you can load A and B just once before the time loop starts.

• The first and last rows involve the boundary conditions. Usually it is easier
to handle the boundary conditions if we plan to do the linear solve of our
matrix equation AT n+1 = BT n in two steps, like this:

import scipy.linalg as la

# matrix multiply to get the right-hand side

r = B@T

# set r as appropriate for the boundary conditions

r[0] = ...
r[-1] = ...

# Solve AT = r. The T we get is for the next time step.

# We don't need to keep track of previous T values, so just
# load the new T directly into T itself
T = la.solve(A,r)

With this code we can just load the top and bottom rows of B with zeros, cre-
ating a right-hand-side vector r with zeros in the top and bottom positions.
The top and bottom rows of A can then be loaded with the appropriate
terms to enforce the desired boundary conditions on T n+1 , and the top and
bottom positions of r can be loaded as required just before the linear solve,
as indicated above.
Lab 7 The Diffusion Equation and Implicit Methods 47

For example, if the boundary conditions were T (0) = 1 and T (L) = 5, the top
and bottom rows of A and the top and bottom positions of r would have
been loaded like this (assuming a cell-center grid with ghost points):

# Set the A portion up where you define the matrix

A[0,0] = 0.5
A[0,1] = 0.5
A[-1,-1] = 0.5
A[-1,-2] = 0.5

... # skipped code

# Set the r portion of the boundary condition down in the time

# loop for each iteration
r[0] = 1
r[-1] = 5

so that the equations for the top and bottom rows are

T0 + T1 T N + T N +1
= r0 = r N +1 (7.25)
2 2
The matrix B just stays out of the way (is zero) in the top and bottom rows.

(iii) Once the matrices A and B are loaded, finding the new temperature in-
side the time loop is accomplished in the time loop by solving the matrix
equation using the code fragment listed above.

P7.3 (a) Write program that implements the Crank-Nicolson algorithm with
fixed-edge boundary conditions, T (0) = 0 and T (L) = 0. Test your
program by running it with D = 2 and an initial temperature given
by T (x) = sin (πx/L). Try various values of τ and see how it compares
with the exact solution. Verify that when the time step is too large the
solution is inaccurate, but still stable. To do the checks at large time
step you will need to use a long run time and not skip any steps in the
plotting.
(b) Now study the accuracy of this algorithm by using various values of
the cell number N and the time step τ. For each pair of choices run
for t = 5 s and find the maximum difference between the exact and
numerical solutions. You should find that the time step τ matters less
than N . The number of cells N is the more important parameter for 1

high accuracy in diffusion problems solved with Crank-Nicolson.

0.5
(c) Modify the Crank-Nicolson program to use boundary conditions
∂T /∂x = 0 at the ends. Run with the same initial condition as in 0
10
part (a) (which does not satisfy these boundary conditions) and watch 4 5
x 5 2 3
what happens. Zoom in on the plots early in time to see what happens 0 1 t

in the first few grid points during the first few time steps.
Figure 7.6 Solution to 7.3(c)
Lab 8
Schrödinger’s Equation

Derivations
Here is the time-dependent Schrödinger equation which governs the way a quan- 0.4
tum wave function changes with time in a one-dimensional potential well V (x):1

∂ψ ħ2 ∂2 ψ 0
iħ =− + V (x)ψ (8.1)
∂t 2m ∂x 2
Note that except for the presence of the imaginary unit i , this is very much like 0.4
the diffusion equation. In fact, a good way to solve it is with the Crank-Nicolson
algorithm. Not only is this algorithm stable for Schrödinger’s equation, but it has 0
another important property: it conserves probability. This is very important. If
the algorithm you use does not have this property, then as ψ for a single particle
0.4
is advanced in time you have (after a while) 3/4 of a particle, then 1/2, etc.
The Schrödinger equation usually describes the behavior of very tiny things
on very short timescales, so SI quantities like kilograms, meters, and Joules are 0

essentially infinite in quantum mechanics. Instead, we’ll use a system of units

called atomic units. In this system, lengths are measured in units of a 0 (the Bohr 0.4
radius, a 0 = 5.29 × 10−11 ), masses are measured in units of the electron mass m e ,
energies are measured in units of E 0 (E 0 = α2 mc 2 ≈ 27 eV), and time is measured
0
in units of t 0 (t 0 = 24.2 attoseconds). These base units are chosen so that the −10 −5 0 5 10
constants ħ and m both have the numerical value of 1 (e.g. m = 1m e for an
electron). Figure 8.1 The probability density
|ψ(x)|2 of a particle in a box that
P8.1 Use paper and pencil to derive a Crank-Nicolson algorithm to solve the initially moves to the right and
Schrödinger equation. It will probably be helpful to use the material in then interferes with itself as it re-
Lab 7 as a guide (beginning with Eq. (7.15)). Make sure the V (x)ψ term flects from an infinite potential
enters the algorithm correctly. (Problem 8.2(a)).

10
x
5
Particle in a box
0
Let’s use this algorithm for solving Schrödinger’s equation to study the evolution
−5
of a particle in a box with
−10
½ 0 10 20 30 40 50
0 if − L < x < L t
V (x) = (8.2)
+∞ otherwise
Figure 8.2 The expectation posi-
tion 〈x〉 for the particle in Fig. 8.1
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New
as time progresses and the packet
Jersey, 2000), p. 470-506.
spreads out (Problem 8.2(c)).

49
50 Computational Physics 430

The infinite potential at the box edges is imposed by forcing the wave function to
be zero at these points:
1 t=0 ψ(−L) = 0 ; ψ(L) = 0 (8.3)

0.5 P8.2 Modify one of your programs from Lab 7 to implement the Crank-Nicolson
algorithm derived above for the case of a particle in a box with L = 10.
0 Note we are doing quantum mechanics and the imaginary parts matter
−10 −5 0 5 10
x now. When assigning complex variables in NumPy, you use the engineering
complex number j , like this:
1 t=1
a = 1.0 + 0.5j
0.5
When you allocate your arrays, you’ll need to specify up-front that they will
0
−10 −5 0 5 10 hold complex values, like this:
x
A = np.zeros((N,N),dtype=np.complex_)
1 t=2 B = np.zeros_like(A,dtype=np.complex_)
0.5
(a) Write a script to solve the time-dependent Schrödinger equation using
0
Crank-Nicolson. We find that a cell-edge grid is easiest, but you can
−10 −5 0 5 10 also do cell-center with ghost points if you like. Start with a localized
x
wave packet of width σ and momentum p:
1 t=3
1 2
± 2
ψ(x, 0) = p p e i px /ħ e −x (2σ ) (8.4)
0.5 σ π

0 with p = 2π and σ = 2. (Remember that ħ in (8.4) has the numerical

−10 −5 0 5 10
x value of 1 in our units.) This initial condition does not exactly satisfy
the boundary conditions, but it is very close. Check to see how far
Figure 8.3 The probability density off it is at the boundary, and decide how the sizes of L and σ must
|ψ(x)|2 of a particle that is initially compare in to use this initial condition.
more localized quickly spreads
Run the script with N = 200 and watch the particle (wave packet)
(Problem 8.2(d)).
bounce back and forth in the well. Plot the real part of ψ as an an-
imation to visualize the spatial oscillation of the wave packet, then
10 plot an animation of ψ∗ ψ so that you can visualize the probability
x distribution of the particle. Try switching the sign of p and see what
5
happens.
0
(b) Verify by doing a numerical integral that ψ(x, 0) in the formula given
−5 above is properly normalized. Then run the script and check that
the wave packet stays properly normalized, even though the wave
−10
0 10 20 30 40 50 function is bouncing and spreading within the well. If you are on a
t cell-edge grid you should do the integrals with NumPy’s trapz rather
Figure 8.4 The expectation posi- than sum.
tion of the particle in Fig. 8.3 as (c) Run the script and verify by numerical integration that the expectation
time progresses. value of the particle position
Z L
〈x〉 = ψ∗ (x, t ) x ψ(x, t )d x (8.5)
−L
Lab 8 Schrödinger’s Equation 51

is correct for a bouncing particle. Plot 〈x〉(t ) to see the bouncing

behavior. Run long enough that the wave packet spreading modifies
the bouncing to something more like a harmonic oscillator. (Note:
you will only see bouncing-particle behavior until the wave packet
spreads enough to start filling the entire well.)
(d) You may be annoyed that the particle spreads out so much in time.
Try to fix this problem by narrowing the wave packet (decrease the
value of σ) so the particle is more localized. Run the script and explain
what you see in terms of quantum mechanics.

0.4
t=0
Tunneling
0.2
Now we will allow the pulse to collide with a with a non-infinite potential barrier
0
of height V0 and width ∆x = 0.02L, and study what happens. Classically, the −20 −10 0 10 20
answer is simple: if the particle has a kinetic energy less than V0 it will be unable x
to get over the barrier, but if its kinetic energy is greater than V0 it will slow down t=1
0.4
as it passes over the barrier, then resume its normal speed in the region beyond
the barrier. (Think of a ball rolling over a bump on a frictionless track.) Can 0.2
the particle get past a barrier that is higher than its kinetic energy in quantum
0
mechanics? The answer is yes, and this effect is called tunneling. −20 −10 0 10 20
To see how the classical picture is modified in quantum mechanics we must x
first compute the energy of our pulse so we can compare it to the height of the 0.4
t=2
barrier. The quantum mechanical formula for the expectation value of the energy
is Z ∞ 0.2

〈E 〉 = ψ∗ H ψd x (8.6) 0
−∞ −20 −10 0 10 20
where ψ∗ is the complex conjugate of ψ and where x

0.4
t=3
ħ2 ∂2 ψ
Hψ = − + V (x)ψ(x) (8.7)
2m ∂x 2 0.2

In our case the initial wave function ψ(x, 0) given by Eq. (8.4) is essentially zero at 0
−20 −10 0 10 20
the location of the potential barrier, so we may take V (x) = 0 in the integral when x
we compute the initial energy. Performing the integral, we find
Figure 8.5 The probability distri-
p2 ħ2 bution |ψ(x)|2 for a particle inci-
〈E 〉 = + (8.8) dent on a narrow potential barrier
2m 4mσ2
located just before x = 10 with
Since this is a conservative system, the energy should remain constant throughout V0 > 〈E 〉. Part of the wave tunnels
the propagation. through the barrier and part inter-
feres with itself as it is reflected.
P8.3 (a) Modify your script from Problem 8.2 so that it uses a computing region
52 Computational Physics 430

that goes from −2L to 3L and a potential



 0 if − 2L < x < 0.98L

 V0 if 0.98L ≤ x ≤ L


V (x) = 0 if L < x < 3L (8.9)





+∞ otherwise


so that we have a square potential hill V (x) = V0 between x = 0.98L

and x = L and V = 0 everywhere else in the well.
Note: Since V (x) was just zero in the last problem, this is the first time
to check if your V (x) terms in Crank-Nicolson are right. If you see
strange behavior, you might want to look at these terms in your code.
(b) Run your script several times, varying the height V0 from less than
your pulse energy to more than your pulse energy. Overlay a plot
of V (x)/V0 on your plot of |ψ|2 and look at the behavior of ψ in the
barrier region.
You should do several experiments with your code. (1) Try making
the barrier height both higher than your initial energy and lower than
your initial energy. Explain to your TA how the quantum behavior
differs from classical behavior. You should find that even when the
barrier is low enough that a classical particle could get over it, some
particles still come back. (2) Experiment with the width of your barrier
and see what its effect is on how many particles tunnel through. As
part of this experiment, figure out a way to calculate what fraction of
the particles make it through the barrier.
Lab 9
Poisson’s Equation: Iteration Methods

In three dimensions, Poisson’s equation is given by

∂2V ∂2V ∂2V ρ

+ + 2 =− (9.1)
∂x 2 ∂y 2 ∂z ²0

Poisson’s equation is used to describe the electric potential in a region of space

with charge density described by ρ:1 You can solve the full 3D equation using
the technique we teach you in this lab, but we won’t make you do that here.
Instead, we’ll focus on geometries that are infinitely long in the z-dimension with
a constant cross-section in the x-y plane. In these cases the z derivative goes to
zero, and Poisson’s equation reduces to

∂2V ∂2V ρ
+ =− (9.2)
∂x 2 ∂y 2 ²0

Note that by studying this equation we are also studying Laplace’s equation (Pois-
son’s equation with ρ = 0) and the steady state solutions to the diffusion equation
in two dimensions (∂T /∂t = 0 in steady state).

Finite difference form

The first step in numerically solving Poisson’s equation is to define a 2-D spa-
tial grid. For simplicity, we’ll use a rectangular grid where the x coordinate is
represented by N x values x j equally spaced with step size h x , and the y coor-
dinate is represented by N y values y k equally spaced with step size h y . This
creates a rectangular grid with N x × N y grid points, just as we used in Lab 6 for
the 2-d wave equation. We’ll denote the potential on this grid using the notation
V (x j , y k ) = V j ,k .
The second step is to write down the finite-difference approximation to the
second derivatives in Poisson’s equation to obtain a grid-based version of Poisson’s
equation. In our notation, Poisson’s equation is the represented by
V j +1,k − 2V j ,k + V j −1,k V j ,k+1 − 2V j ,k + V j ,k−1 ρ j ,k
+ =− (9.3)
h x2 h 2y ²0

This set of equations can only be used at interior grid points because on the edges
it reaches beyond the grid, but this is OK because the boundary conditions tell us
what V is on the edges of the region.
1 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 138-150.

53
54 Computational Physics 430

Equation (9.3) plus the boundary conditions represent a set of linear equa-
tions for the unknowns V j ,k , so we could imagine just doing a big linear solve to
find V all at once. Because this sounds so simple, let’s explore it a little to see why
we are not going to pursue this idea. The number of unknowns V j ,k is N x × N y ,
which for a 100×100 grid is 10,000 unknowns. So to do the solve directly we would
have to be working with a 10, 000×10, 000 matrix, requiring 800 megabytes of RAM
to store the matrix. Doing this big solve is possible for 2-dimensional problems
like this because computers with much more memory than this are common.
However, for larger grids the matrices can quickly get out of hand. Furthermore,
if you wanted to do a 3-dimensional solve for a 100 × 100 × 100 grid, this would
require (104 )3 × 8 = 8 × 1012 , or about 8 terabytes of memory. Computers like this
are becoming possible, but this is still a tiny computational grid. So even though
computers with large amounts of RAM are becoming common, people still use
iteration methods like the ones we are about to describe.

Iteration method on a simple example

Consider solving this equation:
x = e −x (9.4)
One method to solve this equation is to make a guess for the solution, call it x 0 ,
and then iterate on the equation like this:

x n+1 = e −xn (9.5)

For large values of n, we find that the process converges to the exact solution
x̄ = 0.567. Let’s do a little analysis to see why it works. Let x̄ be the exact solution
of this equation and suppose that at the n th iteration level we are close to the
solution, only missing it by the small quantity δn like this: x n = x̄ + δn . Let’s
substitute this approximate solution into Eq. (9.5) and expand using a Taylor
series. Recall that the general form for a Taylor’s series is

1 00 hn
f (x + h) = f (x) + f 0 (x)h + f (x)h 2 + · · · + f (n) (x) + ··· (9.6)
2 n!
When we substitute our approximate solution into Eq. (9.5) and expand around
the exact solution x̄ we get

x n+1 = e −x̄−δn ≈ e −x̄ − δn e −x̄ + · · · (9.7)

If we ignore the terms that are higher order in δ (represented by · · · ), then Eq. (9.7)
shows that the error at the next iteration step is δn+1 = −e −x̄ δn . When we are close
to the solution the error becomes smaller every iteration by the factor −e −x̄ . Since
x̄ is positive, e −x̄ is less than 1, and the algorithm converges. When iteration works
it is not a miracle—it is just a consequence of having this expansion technique
result in an error multiplier that is less than 1 in magnitude.
Lab 9 Poisson’s Equation: Iteration Methods 55

P9.1 Write a program to solve the equation x = e −x by iteration and verify that
it converges. Then try solving this same equation the other way round:
x = − ln x and show that the algorithm doesn’t converge. If you were to
use the x̄ + δ analysis above for the logarithm equation, you’d find that the
multiplier is bigger than one. Our time is short today, so we won’t make you
do this analysis now, but it is a good exercise if you want to look at it on
your own time.

Iteration method on Poisson’s equation

Well, what does this have to do with Poisson’s equation? If we solve the finite-
difference version of Poisson’s equation (Eq. (9.3)) for V j ,k , we find

ρ j ,k
à ! ,à !
V j +1,k + V j −1,k V j ,k+1 + V j ,k−1 2 2
V j ,k = + + + (9.8)
h x2 h 2y ²0 h x2 h 2y

With the equation in this form we could just iterate over and over by doing the
following.

1. Choose an initial guess for the interior values of V j ,k .

2. Use this initial guess to evaluate the right-hand side of Eq. (9.8)

3. Replace our initial guess for V j ,k by this right-hand side, and then repeat.

If all goes well, then after many iterations the left and right sides of this equation
will agree and we will have a solution. 2
To see, let’s notice that the iteration process indicated by Eq. (9.8) can be
written in matrix form as
Vn+1 = LVn + r (9.9)
where L is the matrix which, when multiplied into the vector Vn , produces the
V j ,k part of the right-hand side of Eq. (9.8) and r is the part that depends on the
charge density ρ j ,k . (Don’t worry about what L actually looks like; we are just
going to apply general matrix theory ideas to it.) As in the exponential-equation
example given above, let V̄ be the exact solution vector and let δn be the error
vector at the n th iteration. The iteration process on the error is, then,

δn+1 = Lδn (9.10) Carl Friedrich Gauss (1777–1855, Ger-

man)
Now think about the eigenvectors and eigenvalues of the matrix L. If the matrix is
well-behaved enough that its eigenvectors span the full solution vector space of
size N x × N y , then we can represent δn as a linear combination of these eigenvec-
tors. This then invites us to think about what iteration does to each eigenvector.
2 N. Asmar, Partial Differential Equations and Boundary Value Problems (Prentice Hall, New

Jersey, 2000), p. 429-441.

56 Computational Physics 430

The answer, of course, is that it just multiplies each eigenvector by its eigenvalue.
Hence, for iteration to work we need all of the eigenvalues of the matrix L to have
magnitudes less than 1.
So we can now restate the conditions for this approach to work: Iteration on
Eq. (9.8) converges if all of the eigenvalues of the matrix L on the right-hand side
of Eq. (9.8) are less than 1 in magnitude. This statement is a theorem which can be
proved if you are really good at linear algebra, and the entire iteration procedure
described by Eq. (9.9) is known as Jacobi iteration. Unfortunately, even though
all of the eigenvalues have magnitudes less than 1 there are lots of them that have
magnitudes very close to 1, so the iteration takes forever to converge (the error
only goes down by a tiny amount each iteration).
But Gauss and Seidel discovered that the process can be accelerated by making
a very simple change in the process. Instead of only using old values of V j ,k on
the right-hand side of Eq. (9.8), they used values of V j ,k as they became available
during the iteration. (This means that the right side of Eq. (9.8) contains a mixture
of V -values at the n and n + 1 iteration levels.) This change, which is called Gauss-
Seidel iteration is really simple to code; you just have a single array in which to
store V j ,k and you use new values as they become available. Here as a coded
example of Guass-Seidel iteration for a rectangular region grounded on two sides,
with the two other sides held at a potential:
Listing 9.1 (GaussSeidel.py)

from mpl_toolkits.mplot3d import Axes3D

import matplotlib.pyplot as plt
import numpy as np

# Make the grid

xmin = 0
xmax = 2
Nx = 80
x,hx = np.linspace(xmin,xmax,Nx,retstep = True)
hx2 = hx**2

ymin = 0
ymax = 2
Ny = 40
y,hy = np.linspace(ymin,ymax,Ny,retstep = True)
hy2 = hy**2
X,Y = np.meshgrid(x,y,indexing='ij')

# Initialize potential
V = 0.5*np.ones_like(X)

# Enforce boundary conditions

V[:,0] = 0
Lab 9 Poisson’s Equation: Iteration Methods 57

V[:,-1] = 0
V[0,:] = 1
V[-1,:] = 1

# Allow possibility of charge distribution

rho = np.zeros_like(X)

# Iterate
denom = 2/hx2 + 2/hy2
fig = plt.figure(1)
for n in range(200):
# make plots every few steps
if n % 10 == 0:
plt.clf()
ax = fig.gca(projection='3d')
surf = ax.plot_surface(X,Y,V)
ax.set_zlim(-0.1, 2)
plt.xlabel('x')
plt.ylabel('y')
plt.draw()
plt.pause(0.1)

# Iterate the solution

for j in range(1,Nx-1):
for k in range(1,Ny-1):
V[j,k] = ( (V[j+1,k] + V[j-1,k])/hx2
+(V[j,k+1] + V[j,k-1])/hy2
+rho[j,k]) / denom

P9.2 Paste the code above into a program, run it, and watch the solution iterate.
Study the code, especially the part in the loop. When you understand the
code, call a TA over and explain it to them to pass off this part.

Successive over-relaxation
Gauss-Seidel iteration is not the best we can do, however. To understand the next
improvement let’s go back to the exponential example

x n+1 = e −xn (9.11)

and change the iteration procedure in the following non-intuitive way:

x n+1 = ωe −xn + (1 − ω)x n (9.12)

where ω is a number which is yet to be determined.

58 Computational Physics 430

P9.3 Verify quickly on paper that even though Eq. (9.12) looks quite different
from Eq. (9.11), it is still solved by x = e −x .

If we insert x n = x̄ + δn into this new equation and expand as before, the error
changes as we iterate according to the following

δn+1 = −ωe −x̄ + 1 − ω δn

¡ ¢
(9.13)

Notice what would happen if we chose ω so that the factor in parentheses were
zero: The equation says that we would find the correct answer in just one step! Of
course, to choose ω this way we would have to know x̄, but it is enough to know
that this possibility exists at all. All we have to do then is numerically experiment
with the value of ω and see if we can improve the convergence.

P9.4 Write a program that accepts a value of ω and runs the iteration in Eq. (9.12).
Experiment with various values of ω until you find one that does the best
job of accelerating the convergence of the iteration. You should find that
the best ω is near 0.64, but it won’t give convergence in one step. See if you
can figure out why not.
Hint: Think about the approximations involved in obtaining Eq. (9.13).
Specifically go back to the previous derivation and look for terms repre-
sented by dots.

As you can see from Eq. (9.13), this modified iteration procedure shifts the
error multiplier to a value that converges better. So now we can see how to
improve Gauss-Seidel: we just use an ω multiplier like this:

Vn+1 = ω (LVn + r ) + (1 − ω)Vn (9.14)

then play with ω until we achieve almost instantaneous convergence.

Sadly, this doesn’t quite work. The problem is that in solving for N x × N y
unknown values V j ,k we don’t have just one multiplier; we have one for each
eigenvalue of the matrix. So if we shift one of the eigenvalues to zero, we might
shift another one to a value with magnitude larger than 1 and the iteration will
not converge at all. The best we can do is choose a value of ω that centers the
entire range of eigenvalues symmetrically between −1 and 1.
Using an ω multiplier to shift the eigenvalues is called Successive Over-
Relaxation, or SOR for short. Here it is written out so you can code it:

V j +1,k + V j −1,k V j ,k+1 + V j ,k−1 ρ j ,k

à ! ,à !
2 2
V j ,k = ω + + + + (1 − ω)V j ,k
h x2 h 2y ²0 h x2 h 2y
(9.15)
And what value should we use for ω? The answer is that it depends on the values
of N x and N y . In all cases ω should be between 1 and 2, with ω = 1.7 being a
typical value. Some wizards of linear algebra have shown that the best value of ω
Lab 9 Poisson’s Equation: Iteration Methods 59

when the computing region is rectangular and the boundary values of V are fixed
(Dirichlet boundary conditions) is given by

2
ω= p (9.16)
1 + 1 − R2
where
h 2y cos (π/N x ) + h x2 cos (π/N y )
R= . (9.17)
h x2 + h 2y
These formulas usually give a reasonable estimate of the best ω to use. Note,
however, that this value of ω was found for the case of a cell-edge grid with the
potential specified at the edges. If you use a cell-centered grid with ghost points,
and especially if you change to derivative boundary conditions, this value of ω
won’t be quite right. But there is still a best value of ω somewhere near the value
given in Eq. (9.16) and you can find it by numerical experimentation.
Finally, we come to the question of when to stop iterating. It is tempting just
to watch the values of V j ,k and quit when the values stabilize at some level, like
this for instance: quit when ² = |V ( j , k)n+1 − V ( j , k)n | < 10−6 . You will see this
error criterion sometimes used in books, but do not use it. We know of one person
who published an incorrect result in a journal because this error criterion lied.
We don’t want to quit when the algorithm has quit changing V ; we want to quit
when Poisson’s equation is satisfied. (Most of the time these are the same, but
only looking at how V changes is a dangerous habit to acquire.) In addition, we
want to use a relative (%) error criterion. This is easily done by setting a scale
voltage Vscale which is on the order of the biggest voltage in the problem and then
using for the error criterion ¯ ¯
¯ Lhs − Rhs ¯
² = ¯¯ ¯ (9.18)
Vscale ¯
where Lhs is the left-hand side of Eq. (9.8) and Rhs is its right-hand side. Because
this equation is just an algebraic rearrangement of our finite-difference approx-
imation to Poisson’s equation, ² can only be small when Poisson’s equation is
satisfied. (Note the use of absolute value; can you explain why it is important to
use it? Also note that this error is to be computed at all of the interior grid points.
Be sure to find the maximum error on the grid so that you only quit when the
solution has converged throughout the grid.)
And what value should we choose for the error criterion so that we know when
to quit? Well, our finite-difference approximation to the derivatives in Poisson’s
equation is already in error by a relative amount of about 1/(12N 2 ), where N is the
smaller of N x and N y . There is no point in driving ² below this estimate. For more
details, and for other ways of improving the algorithm, see Numerical Recipes,
Chapter 19.

P9.5 Starting with the Gauss-Seidel example above, implement all of the im-
provements described above to write a full successive over-relaxation rou-
tine. Note that you will need to replace the for loop on the variable n
60 Computational Physics 430

into a while loop that is based on the error criterion. Also move the plot
1 command out of the loop so it only executes once, after the solution has
converged. (So that you don’t have to wait for a bunch of plots to be drawn.)
0.5 Run your code for a spatial region of size L x = 4 in the x-dimension and
L y = 2 the y-dimension arranged as shown in Fig. 9.1. Use Nx = Ny = 30
0 and several different values of ω. Note that the boundary conditions on the
2
2 potential V (x, y) are V (−L x /2, y) = V (L x /2, y) = 1 and V (x, 0) = V (x, L y ) = 0.
1 0
y 0 −2 x Set the error criterion to 10−4 . Verify that the optimum value of ω given by
Eq. (9.16) is the best one to use.
Figure 9.1 The electrostatic po-
tential V (x, y) with two sides
grounded and two sides at con-
stant potential. Some Different Geometries
P9.6 (a) Modify your code to model a rectangular pipe with V (x = −2, y) = −1,
1 V (x = 2, y) = 1, and the y = 0 and y = L y edges held at V = 0.
0.5
(b) Modify your code from (a) so that the boundary condition at the x =
0
−L x /2 edge of the computation region is ∂V /∂x = 0 and the boundary
condition on the y = L y edge is ∂V /∂y = 0. You can do this problem
2 either by changing your grid and using ghost points or by using a
2
1 0
quadratic extrapolation technique (see Eq. (2.10)). Both methods
y 0 −2 x work fine, but note that you will need to enforce boundary conditions
inside the main SOR loop now instead of just setting the values at the
Figure 9.2 The potential V (x, y)
edges and then leaving them alone.
from Problem 9.6(a).
You may discover that the script runs slower on this problem. See if
1
you can make it run a little faster by experimenting with the value of
ω that you use. Again, changing the boundary conditions can change
0.5 the eigenvalues of the operator. (Remember that Eq. (9.16) only works
for cell-edge grids with fixed-value boundary conditions, so it only
0
2 gives a ballpark suggestion for this problem.)
2
1 0
y 0 −2 x (c) Study electrostatic shielding by going back to the boundary conditions
of Problem 9.6(a), while grounding some points in the interior of the
Figure 9.3 The potential V (x, y) full computation region to build an approximation to a grounded cage.
with zero-derivative boundary Allow some holes in your cage so you can see how fields leak in. First
conditions on two sides (Prob-
make a rectangular cage similar to Fig. 9.4, but then you can try other
lem 9.6(b).)
geometries.
You will need to be creative about how you build your cage and about
how you make SOR leave your cage points grounded as it iterates. One
thing that won’t work is to let SOR change all the potentials, then set
the cage points back to V = 0 before doing the next iteration. This
takes forever to converge. It is much better to set them to zero and
force SOR to never change them. An easy way to do this is to use a
cell-edge grid with a mask. A mask is an array that you build that is
the same size as V, initially defined to be full of ones like this
Lab 9 Poisson’s Equation: Iteration Methods 61

mask = np.ones_like(V)
Then you go through and set the elements of mask that you don’t want
SOR to change to have a value of zero. (We’ll let you figure out the
logic to do this for the cage.) Once you have your mask built, you
add an if statement to our code so that the SOR stuff inside the j
and k for loops only changes a given point and updates the error if
mask(j,k) is one. This logic assumes you have to set the values of
V for these points before the for loop execute, just like the boundary
conditions. Using this technique you can calculate the potential for
quite complicated shapes just by changing the mask array.

0.5

0.4
0.1
0.2
0
y 0 −0.1 x

Figure 9.4 The potential V (x, y) for an electrostatic “cage” formed by

grounding some interior points. (Problem 9.6(c).)
Lab 10
The Continuity Equation
So far we have studied the numerical solution of partial differential equations
one at a time, but in many interesting situations the problem to be solved involves
coupled systems of differential equations. A “simple” example of such a system
are the three coupled one-dimensional equations of gas dynamics. These are the
equations of acoustics in a long tube with mass density ρ(x, t ), pressure p(x, t ),
and gas velocity v(x, t ) as the dynamic variables. In the next lab we will tackle the
hard problem of simultaneously advancing ρ, T , and v in time and space, which
will require three equations. But for now we will just practice on the continuity
equation to develop the tools we need to do the full problem.

The Continuity Equation

The equation that enforces conservation of mass in acoustics is called the conti-
nuity equation:
∂ρ ∂ ¡ ¢
+ ρv = 0 (10.1)
∂t ∂x
This equation says that as the gas particles are moved by the flow velocity v(x, t ),
the density ρ(x, t ) is carried along with the flow and can be compressed or rarefied,
but mass is not created or destroyed in this process.
Boundary conditions for the continuity equation are a bit different than we’ve
encountered to this point. This is a convection equation, meaning that if you
stand at a point in the flow, the solution at your location arrives (is convected to
you) from further “upwind.” This has a strong effect on the boundary conditions.
Suppose, for instance, that the flow field v(x) is always positive, meaning that the
wind is blowing to the right. At the left-hand boundary it makes sense to specify
ρ because somehow you might arrange to feed density in at that point so that it
can be convected across the grid. But at the right boundary it makes no sense at
all to specify a boundary condition because when the solution arrives there we
just want to let the wind blow it away. (An exception to this rule occurs if v = 0 at
the boundary. In this case there is no wind to blow the solution from anywhere
and it would be appropriate to specify a boundary condition.) We’ll try several
approaches to represent these boundary conditions.

P10.1 Let’s start with something really simple and inaccurate just to see what can
go wrong. If we use a nicely centered difference in x and an inaccurate
forward difference in t , we find

ρ n+1
j
− ρ nj 1 ³ n ´
+ ρ j +1 v j +1 − ρ nj−1 v j −1 = 0 (10.2)
τ 2h
63
64 Computational Physics 430

Solve this equation for ρ n+1

j
and use it in a time-advancing script like the
one you built to do the wave equation in Lab 5. Use a cell-center grid with
ghost points (use about 400 grid points). For initial conditions use
2
ρ(x, 0) = 1 + e −200(x/L−1/2) (10.3)

with x ∈ [0, L], L = 10, and

v(x) = v 0 (10.4)
with v 0 = 1. At the left end use ρ(0, t ) = 1 and at the right end try the
following two things:

(i) Set a boundary condition: ρ(L, t ) = 1.

(ii) Just let mass leave by using linear extrapolation:

ρ(L, t ) = 2ρ(L − h, t ) − ρ(L − 2h, t ) or ρ N +1 = 2ρ N − ρ N −1 (10.5)

Run this algorithm with these two boundary conditions enough times, and
with small enough time steps, that you become convinced that ρ(L, t ) = 1 is
wrong and that the entire algorithm is worthless because it is unstable.

As you might guess from the previous problem, the diffusion equation’s sim-
ple appearance can be deceiving; it is one of the most difficult equations to solve
numerically in all of computational physics because stable methods tend to be in-
accurate and accurate methods tend either to be unstable, or non-conservative (as
time runs mass spontaneously disappears), or unphysical (mass density and/or
pressure become negative.)
Let’s try another method, known as the Lax-Wendroff method. The idea of the
Lax-Wendroff algorithm is to use a Taylor series in time to obtain a second-order
accurate method. Taylor expanding the density function in time gives us

∂ρ τ2 ∂2 ρ
ρ(x, t + τ) = ρ(x, t ) + τ + (10.6)
∂t 2 ∂t 2
We can write the continuity equation as

∂ρ ∂ ¡ ¢
=− ρv (10.7)
∂t ∂x
Substituting into our Taylor expansion then gives

∂ ¡ ¢ τ2 ∂ ∂ ¡ ¢
µ ¶
ρ(x, t + τ) = ρ(x, t ) − τ ρv + − ρv (10.8)
∂x 2 ∂t ∂x

If we reverse the order of the derivatives in the last term and assume that v is not
a function of time, we can use Eq. (10.7) again to obtain.

∂ρv τ2 ∂ ∂ρv
µ ¶
ρ(x, t + τ) = ρ(x, t ) − τ + v . (10.9)
∂x 2 ∂x ∂x
Lab 10 The Continuity Equation 65

Finally, we subtract ρ(x, t ) from both sides and divide by τ to obtain

ρ(x, t + τ) − ρ(x, t ) ∂ρv τ ∂ ∂ρv

µ ¶
=− + v . (10.10)
τ ∂x 2 ∂x ∂x

If we interpret the left-hand side as a time derivative, the second term on the right
looks essentially like the diffusion equation. Since the equation we are solving is
pure convection, the appearance of diffusion is not good news, but at least this
algorithm is better than the horrible one in 10.1. Notice also that the diffusion
coefficient in Eq. (10.10) is proportional to τ (stare at it until you can see that this
is true), so if small time steps are being used diffusion won’t hurt us too much.

P10.2 Finite difference the expression in Eq. (10.10) assuming that v(x) = v 0 =
const, to find the Lax-Wendroff algorithm:

v0τ ³ n ´ v 2 τ2 ³ ´
ρ n+1
j = ρ nj − ρ j +1 − ρ nj−1 + 0 2 ρ nj+1 − 2ρ nj + ρ nj−1 (10.11)
2h 2h

Change your script from 10.1 to use the Lax-Wendroff algorithm. Again,
use a cell-center grid with ghost points and about 400 grid points. Also
use the same initial condition as in Problem 10.1 and use the extrapolated
boundary condition that just lets the pulse leave.
Show that Lax-Wendroff works pretty well unless the time step exceeds a
Courant condition. Also show that it has the problem that the peak density
slowly decreases as the density bump moves across the grid. (To see this use Peter Lax (b. 1926, American) Lax was
a relatively coarse grid and a time step just below the stability constraint. the PhD advisor for Burton Wendroff.

Warning: do not run with τ = h/v 0 . If you do you will conclude that this
algorithm is perfect, which is only true for this one choice of time step.)
This problem is caused by the diffusive term in the algorithm, but since this
diffusive term is the reason that this algorithm is not unstable like the one
in 10.1, we suppose we should be grateful.

Crank-Nicolson Again
Finally, let’s try an implicit method, Crank-Nicolson in fact. As a reminder, the
continuity equation is
∂ρ ∂ ¡ ¢
+ ρv = 0 , (10.12)
∂t ∂x
We can’t solve this equation directly because it has two unknowns (ρ and v). But
if we assume that v is known, then it is possible to solve the equation using Crank-
Nicolson. As usual for Crank-Nicolson, we forward difference in time and center
difference in space to find

ρ n+1
j
− ρ nj v nj+1 ρ nj+1 − v nj−1 ρ nj−1
=− (10.13)
τ 2h
66 Computational Physics 430

Then we use time averaging to put the right side of the equation at the same time
level as the left (i.e. at the n + 1/2 time level):
³ ´ ³ ´
ρ n+1
j − ρ n
j = C 1 ρ n
j +1 + ρ n+1
j +1 −C 2 ρ n
j −1 + ρ n+1
j −1 (10.14)

where
τ ³ n ´
C1 = − v j +1 + v n+1
j +1 (10.15)
8h ³
τ ´
C2 = − v nj−1 + v n+1
j −1 (10.16)
8h
Then we put the ρ n+1 terms on the left and the ρ n terms on the right:

C 2 ρ n+1 n+1
j −1 + ρ j −C 1 ρ n+1 n n n
j +1 = −C 2 ρ j −1 + ρ j +C 1 ρ j +1 (10.17)

Then we write these equations along with the boundary conditions in matrix form

Aρ n+1 = Bρ n (10.18)

which we solve using linear algebra techniques. For this algorithm to calculate
ρ n+1 , we need to feed it values for ρ n , v n , and v n+1 . For now, let’s side-step this
issue by assuming that v(x, t ) is known, and a constant in time. In the next lab
we’ll worry about advancing the velocity solution in parallel with the density.

P10.3 (a) Write a program that implements this algorithm, perhaps starting
from one of your programs from the Schrödinger equation lab. Work
out how to implement the boundary conditions, ρ(0, t ) = 1 and ρ(L, t )
is just allowed to leave, by properly defining the top and bottom rows
of the matrices A and B. This involves multiplying Bρ n to find an
1
6 8
r-vector as you have done before.
4 6
4 (b) Implement this algorithm with a constant convection velocity v =
2
2
t 0 0
x v 0 and show that the algorithm conserves amplitude to very high
precision and does not widen due to diffusion. These two properties
Figure 10.1 A pulse is convected
make this algorithm a good one as long as shock waves don’t develop.
across a region in which the con-
vection velocity v(x) is constant (c) Now use a convection velocity that varies with x:
(Problem 10.3(b)).
v(x) = 1.2 − x/L (10.19)

This velocity slows down as the flow moves to the right, which means
that the gas in the back is moving faster than the gas in the front,
causing compression and an increase in density. You should see the
slowing down of the pulse and the increase in density in your numeri-
cal solution.
(d) Go back to a constant convection velocity v = v 0 and explore the way
this algorithm behaves when we have a shock wave (discontinuous
Figure 10.2 A pulse is convected
density) by using as the initial condition
across a region in which the con-
vection velocity v(x) is decreasing. ½
Note that the pulse narrows and 1.0 if 0 ≤ x ≤ L/2
ρ(x, 0) = (10.20)
grows, conserving mass. (Prob- 0 otherwise
lem 10.3(c))
Lab 10 The Continuity Equation 67

The true solution of this problem just convects the step to the right;
you will find that Crank-Nicolson fails at this seemingly simple task.
(e) For comparison, try the same step-function initial condition in your
Lax-Wendroff script from Problem 10.2.

Our Crank-Nicolson algorithm is both stable and conservative, but it only

works well if the solution doesn’t become too steep. This is a severe limitation,
since we are talking about gas dynamics here and shock waves routinely show up
as solutions in gas dynamics. Numerical methods that properly handle shocks
are much more involved than the ones we will show you here.
Lab 11
One Dimensional Gas Dynamics

Simultaneously advancing ρ, T , and v

Now we are going to use the implicit algorithm that we developed in the previous
lab as a tool to solve the three nonlinear coupled partial differential equations
of one-dimensional gas dynamics. These are the equations of one-dimensional
sound waves in a long tube pointed in the x-direction, assuming that the tube is
wide enough that friction with the walls doesn’t matter.
The three equations we need to advance are the continuity equation

∂ρ ∂ ¡ ¢
+ ρv = 0 , (11.1)
∂t ∂x
the conservation of energy

∂T ∂T ∂v (γ − 1)M κ 1 ∂2 T
+v + (γ − 1)T = , (11.2)
∂t ∂x ∂x kB ρ ∂x 2

and Newton’s second law

∂v ∂v 1 ∂P 4µ ∂2 v
+v =− + (11.3)
∂t ∂x ρ ∂x 3ρ ∂x 2

Here ρ(x, t ) is the density of the gas, v(x, t ) is the velocity of the gas, and T (x, t ) is
the temperature of the gas. The pressure P is given by the ideal gas law

kB
P= ρT (11.4)
M
Because of the nonlinearity of these equations and the fact that they are
coupled we are not going to be able to write down a simple algorithm that will ad-
vance ρ, T , and v in time. But if we are creative we can combine simple methods
that work for each equation separately into a stable and accurate algorithm for
the entire set. We are going to show you one way to do it, but the computational
physics literature is full of other ways, including methods that handle shock waves.
This is still a very active and evolving area of research, especially for problems in
2 and 3 dimensions.
Let’s try a predictor-corrector approach similar to second-order Runge-Kutta
(which you learned about back in 330) by first taking an approximate step in time
of length τ to obtain predicted values for our variables one time step in the future.
We’ll refer to these first-order predictions for the future values as ρ̃ n+1 , T̃ n+1 ,
and ṽ n+1 . In the predictor step we will treat v as constant in time in Eq. (11.1)

69
70 Computational Physics 430

to predict ρ̃ n+1 . Then we’ll use ρ̃ n+1 to help us calculate T̃ n+1 using Eq. (11.2),
while still treating v as fixed in time. Once these predicted values are obtained we
can use them with Eq. (11.3) to obtain a predicted ṽ. With all of these predicted
future values of our variables in hand, we do another round of Crank-Nicolson
on all three equations to step the solution forward in time. We can represent this
procedure schematically as follows:

Step 1 Use the old velocity v n as input for Eqn. (11.5) → Solve for the predicted
density ρ̃.

Step 2 Use v n and ρ̃ as inputs for Eqn. (11.2) → Solve for the predicted tempera-
ture T̃ .

Step 3 Use ρ̃ and T̃ as inputs for Eqn. (11.3) → Solve for the predicted velocity ṽ.

Step 4 Use ṽ as input for Eqn. (11.5) → Solve for the new density ρ n+1 .

Step 5 Use ṽ and ρ n+1 as inputs for Eqn. (11.2) → Solve for the new temperature
T n+1 .

Step 6 Use ρ n+1 and T n+1 as inputs for Eqn. (11.3) → Solve for the new velocity
v n+1 .

This procedure probably seems a bit nebulous at this point, so let’s go through
it in more detail. First we’ll derive the Crank-Nicolson algorithms for our three
equations, then we’ll show how to use these algorithms to solve the system using
the predictor-corrector method.

Continuity Equation
Conservation of mass is governed by the continuity equation

∂ρ ∂ ¡ ¢
+ ρv = 0 , (11.5)
∂t ∂x
We can’t solve this equation directly because it has two unknowns (ρ and v).
But if we assume that v is known as we did in the last lab, then it is possible to
solve the equation using Crank-Nicolson, as we did in the last lab. As usual for
Crank-Nicolson, we forward difference in time and center difference in space to
find
ρ n+1
j
− ρ nj v nj+1 ρ nj+1 − v nj−1 ρ nj−1
=− (11.6)
τ 2h
Then we use time averaging to put the right side of the equation at the same time
level as the left (i.e. at the n + 1/2 time level):
³ ´ ³ ´
ρ n+1
j − ρ n
j = C 1 ρ n
j +1 + ρ n+1
j +1 −C 2 ρ n
j −1 + ρ n+1
j −1 (11.7)
Lab 11 One Dimensional Gas Dynamics 71

where
τ ³ n ´
C1 = − v j +1 + v n+1
j +1 (11.8)
8h ³
τ ´
C2 = − v nj−1 + v n+1
j −1 (11.9)
8h
Then we put the ρ n+1 terms on the left and the ρ n terms on the right:

C 2 ρ n+1 n+1
j −1 + ρ j −C 1 ρ n+1 n n n
j +1 = −C 2 ρ j −1 + ρ j +C 1 ρ j +1 (11.10)

Then we write these equations along with the boundary conditions in matrix form

Aρ n+1 = Bρ n (11.11)

which we solve using linear algebra techniques. For the algorithm represented by
Eq. (11.11) to calculate ρ n+1 , we need to feed it values for ρ n , v n , and v n+1 . Since
the inputs for these variables will be different in the predictor and the corrector
steps, we need to invent some notation. We’ll refer to this Crank-Nicolson algo-
rithm for stepping forward to find ρ n+1 using the notation S ρ ρ n , v n , v n+1 so the
¡ ¢

variables the algorithm needs as inputs are explicitly shown.

Conservation of energy
The temperature of a gas is a macroscopic manifestation of the energy of the
thermal motions of the gas molecules. The equation that enforces conservation
of energy for our system is

∂T ∂T ∂v ∂2 T
+v = −(γ − 1)T + DT (11.12)
∂t ∂x ∂x ∂x 2
where γ is the ratio of specific heats in the gas: γ = C p /C v . This equation says that
as the gas is moved along with the flow and squeezed or stretched, the energy is
convected along with the flow and the pressure goes up and down adiabatically
(that’s why γ is in there). It also says that thermal energy diffuses due to thermal
conduction. Thermal diffusion is governed by the diffusion-like term containing
the thermal diffusion coefficient D T given in a gas by
(γ − 1)M κ
DT = (11.13)
kB ρ
where κ is the thermal conductivity, M is the mass of a molecule of the gas, and
where k B is Boltzmann’s constant.
It is probably easier to conceptualize pressure waves rather than temperature
waves. The ideal gas law gives us a way to calculate the pressure, given a density
ρ and a temperature T , so we’ll use the ideal gas law P = nk B T (where n is the
number of particles per unit volume) to calculate pressure P once the density ρ
and temperature T are known via
kB
P= ρT. (11.14)
M
72 Computational Physics 430

To find a predicted value for T one step in the future, we forward difference
the time derivative and center difference the space derivatives to find

T jn+1 − T jn T jn+1 − T jn−1 v nj+1 − v nj−1 n n n

1 T j +1 − 2T j + T j −1
= −v nj − (γ − 1)T jn +F n
τ 2h 2h ρj h2
(11.15)
where
(γ − 1)M κ
F= (11.16)
kB
We then rearrange Eq. (11.15) into a form that makes the upcoming algebra (and
coding) more readable:

T jn+1 − T jn
= T jn−1 D 1 + T jn D 2 + T jn+1 D 3 (11.17)
τ
where
v nj F
D1 = + (11.18)
2h ρ nj h 2
v nj+1 − v nj−1 2F
D 2 = −(γ − 1) − (11.19)
2h ρ nj h 2
v nj F
D3 = − + (11.20)
2h ρ nj h 2

P11.1 Finish deriving the Crank-Nicolson algorithm for T n+1 by putting the right-
hand side of Eq. (11.17) at the n + 1/2 time level. This means replacing
T n terms with (T n + T n+1 )/2 in Eq. (11.17) and making the replacements
ρ n ⇒ (ρ n + ρ n+1 )/2 and v n ⇒ (v n + v n+1 )/2 in D 1 , D 2 , and D 3 . Then put
your system of equations in the form

AT n+1 = BT n

and write out the coefficients in the A and B matrices so we can code them
later.

When you are finished with Problem 11.1 you will have an algorithm for stepping
T forward in time. We’ll refer to this algorithm as S T T n , v n , v n+1 , ρ n , ρ n+1 , so
¡ ¢

we explicitly see the variables that are required as inputs.

Newton’s second law

Finally, let’s consider Newton’s second law for this system in a form analogous to
a = F /m:
∂v ∂v 1 ∂P 4µ ∂2 v
+v =− + (11.21)
∂t ∂x ρ ∂x 3ρ ∂x 2
Lab 11 One Dimensional Gas Dynamics 73

You should recognize the first term d v/d t as acceleration, and we’ll discuss the
origin of the other acceleration term in a minute. The first term on the right is the
pressure force that pushes fluid from high pressure toward low pressure, with the
pressure P given by the ideal gas law in Eq. (11.14). The second term on the right
represents the force of internal friction called viscosity, and the parameter µ is
referred to as the coefficient of viscosity. (Tar has high viscosity, water has medium
viscosity, and air has almost none.) The 1/ρ factor in the force terms represents
the mass m in a = F /m (but of course we are working with mass density ρ here).
You may be unconvinced that the left side of Eq. (11.21) is acceleration. To
become more convinced, let’s think about this situation more carefully. Newton’s
second law does not apply directly to a location in space where there is a moving
fluid. Newton’s second law is for particles that are moving through space, not for a
location in space that is sitting still with fluid moving through it. This distinction
is subtle, but important. Think, for instance, about a steady stream of honey
falling out of a honey bear held over a warm piece of toast. If you followed a
piece of honey along its journey from the spout down to the bread you would
experience acceleration. But if you watched a piece of the stream at a specific
location (say, 10 cm above the bread) you would see that the velocity of this part
of the stream is constant in time: ∂v ∂t = 0. This is a strong hint that there is
±

more to describing the acceleration of fluids through a region of space than just
the local ∂v ∂t for a given location. You also need to compare the local velocity
±

at your chosen point to the velocities at nearby points in space, which is what the
v∂v ∂x term does in the left side of Eq. (11.21).
±

Notice that Eq. (11.21) has a nonlinear term on the left: v(∂v/∂x). There is
no way to directly represent this nonlinear term using a linear matrix form like
Av n+1 = Bv n , so we’ll have to make an approximation. We’ll assume that the
leading v in the nonlinear term is somehow known and designate it as v̄. (We’ll
deal with finding something to use for v̄ later.) With a forward time derivative
and a centered space derivative, we have

v n+1 − v nj v nj+1 − v nj−1

à n n n n !
k B ρ j +1 T j +1 − ρ j −1 T j −1
à !
j
= −v̄ nj −
τ 2h M ρ nj 2h
à n n n !
4µ v j +1 − 2v j + v j −1
+ n (11.22)
3ρ j h2

Again, we’ll rewrite the equations with named groups of expressions that don’t
depend on v so that our algebra is manageable:

v n+1
j
− v nj
= E 0 + v nj−1 E 1 + v nj E 2 + v nj+1 E 3 (11.23)
τ
74 Computational Physics 430

where
à n n n n !
k B ρ j +1 T j +1 − ρ j −1 T j −1
E0 = − (11.24)
M ρ nj 2h
v̄ nj 4µ
E1 = + (11.25)
2h 3ρ nj h 2
8µ
E2 = − (11.26)
3ρ nj h 2
v̄ nj 4µ
E3 = − + (11.27)
2h 3ρ nj h 2

P11.2 Finish deriving the Crank-Nicolson algorithm for v by making the replace-
ments v n ⇒ (v n+1 + v n )/2 the right-hand side of Eq. (11.23) and ρ n ⇒
(ρ n + ρ̃ n+1 )/2, T n ⇒ (T n + T̃ n+1 )/2, and v̄ n ⇒ (v̄ n + v̄ n+1 )/2 in E 0 , E 1 , E 2 ,
and E 3 . Show that your system of equations needs to be in the form

Av n+1 = Bv n + E 0

where E 0 is a column vector. Write out the coefficients in the A and B

matrices so you can code them later.

We’ll refer to this v-stepping algorithm as S v v n , v̄ n , v̄ n+1 , ρ n , ρ n+1 , T n , T n+1 ,

¡ ¢

where, as usual, we explicitly show the variables that are required as inputs.

Waves in a closed tube

Now that you have algorithms for all three equations, we can restate the predictor-
corrector algorithm using our newly-developed notation.

Predictor Step: First we predict ρ n+1 while treating v as a constant:

ρ̃ n+1 = S ρ ρ n , v n , v n+1 = v n
¡ ¢

Then we predict T n+1 using ρ̃ n+1 , still treating v as a constant

T̃ n+1 = S T T n , v n , v n+1 = v n , ρ n , ρ n+1 = ρ̃ n+1

¡ ¢

Then we predict v n+1 using ρ̃ n+1 and T̃ n+1 , while treating v̄ from the non-
linear term as a constant equal to the current v

ṽ n+1 = S v v n , v̄ n = v n , v̄ n+1 = v n , ρ n , ρ n+1 = ρ̃ n+1 , T n , T̃ n+1

¡ ¢

Corrector Step: Now that we have predicted values for each variable, we step ρ
forward using
ρ n+1 = S ρ ρ n , v n , v n+1 = ṽ n
¡ ¢
Lab 11 One Dimensional Gas Dynamics 75

Then we step T using

T n+1 = S T T n , v n , v n+1 = ṽ n , ρ n , ρ n+1

¡ ¢

And finally, we step v forward using

v n+1 = S v v n , v̄ n = v n , v̄ n+1 = ṽ n , ρ n , ρ n+1 , T n , T n+1

¡ ¢

Now let’s put this algorithm into a script and use it to model waves in a tube
of length L = 10 m with closed ends through which there is no flow of heat. For
disturbances in air at sea level at 20◦ C we have temperature T = 293 K, mass
density ρ = 1.3 kg/m3 , adiabatic exponent γ = 1.4, coefficient of viscosity µ =
1.82 × 10−5 kg/(m·s), and coefficient of thermal conductivity κ = 0.024 J/(m·s·K).
Boltzmann’s constant is k B = 1.38 × 10−23 J/K and the mass of the molecules of
the gas is M = 29 × 1.67 × 10−27 kg for air.

P11.3 (a) As you might guess, debugging the algorithm that we just developed
takes a while to debug because there are so many steps and so many
terms to get typed accurately. (When we wrote the solution code, it
took over an hour to track down two minus sign errors and a factor
of two error.) We’d rather have you use the algorithm than beat your
head on the wall debugging it, so below is the code that implements
the algorithm. Go to the class web site now and download these files.
Study them and make sure you understand how they work.
(b) The one thing we haven’t included in the code is the boundary condi-
tions. The ends are insulating, so we have

∂T /∂x = 0 (11.28)

at both ends. Because the wall ends are fixed and the gas can’t pass
through these walls, the boundary conditions on the velocity are

v(0, t ) = v(L, t ) = 0 (11.29)

Use this fact to obtain the following differential boundary condition

on the density at the ends of the tube:

∂ρ ∂v
+ρ = 0 at x = 0 and x = L (11.30)
∂t ∂x
This condition simply says that the density at the ends goes up and
down in obedience to the compression or rarefaction produced by the
divergence of the velocity.
Write down the finite difference form for all three of these boundary
conditions. Make sure they are properly centered in time and space
for a cell-center grid with ghost points. Then code these boundary
conditions in the proper spots in the code.
76 Computational Physics 430

P11.4 (a) Test the script byqmaking sure that small disturbances travel at the
γk T
M . To do this set T and ρ to their atmospheric
B
sound speed c =
values and set the velocity to
2
v(x, 0) = v 0 e −200(x/L−1/2) (11.31)

with v 0 = c/100. If you look carefully at the deviation of the density

from its atmospheric value you should see two oppositely propagating
signals. Verify that they travel at the speed of sound.
(b) The predictor-corrector algorithm is not as stable as plain Crank-
Nicolson. Vary the time step and find where it begins to be unstable.
Then change N and see how the stability threshold changes. Come up
with an equation that estimates a limit on the step size in terms of h.
(c) Remove the effects of viscosity and thermal conduction by setting
µ = 0 and κ = 0. Increase the value of v 0 to c/10 and beyond and watch
how the pulses develop. You should see the wave pulses develop steep
leading edges and longer trailing edges; you are watching a shock wave
develop. But if you wait long enough you will see your shock wave
develop ugly wiggles; these are caused by Crank-Nicolson’s failure to
properly deal with shock waves.
(d) Repeat part (c) with non-zero µ and κ and watch thermal conduction
and viscosity widen the shock and prevent wiggles. Try artificially
large values of µ and κ as well as their actual atmospheric values.

Listing 11.1 (gas.py)

import Lab11Funcs as S
import matplotlib.pyplot as plt
import numpy as np

# System Parameters
L = 10.0 # Length of tube
T0 = 293. # Ambient temperature
rho0 = 1.3 # static density (sea level)

# speed of sound
c = np.sqrt(S.gamma * S.kB * T0 / S.M)

# cell-center grid with ghost points

N = 100
h = L/N
x = np.linspace(-.5*h,L+.5*h,N+2)

# initial distributions
Lab 11 One Dimensional Gas Dynamics 77

rho = rho0 * np.ones_like(x)

T = T0 * np.ones_like(x)
v = np.exp(-200*(x/L-0.5)**2) * c/100

tau = 1e-4
tfinal = 0.1
t = np.arange(0,tfinal,tau)

skip = 5 #input(' Steps to skip between plots - ')

for n in range(len(t)):

# Plot the current values before stepping

if n % skip == 0:
plt.clf()
plt.subplot(3,1,1)
plt.plot(x,rho)
plt.ylabel('rho')
plt.ylim(1.28, 1.32)
plt.title('time={:1.3f}'.format(t[n]))
plt.subplot(3,1,2)
plt.plot(x,T)
plt.ylabel('T')
plt.ylim(292,294)
plt.subplot(3,1,3)
plt.plot(x,v)
plt.ylabel('v')
plt.ylim(-4,4)
plt.xlabel('x')
plt.pause(0.1)

# 1. Predictor step for rho

rhop = S.Srho(rho,v,v,tau,h)

# 2. Predictor step for T

Tp = S.ST(T,v,v,rho,rhop,tau,h)

# 3. Predictor step for v

vp = S.Sv(v,v,v,rho,rhop,T,Tp,tau,h)

# 4. Corrector step for rho

rhop = S.Srho(rho,v,vp,tau,h)

# 5. Corrector step for T

78 Computational Physics 430

Tp = S.ST(T,v,vp,rho,rhop,tau,h)

# 6. Corrector step for v

v = S.Sv(v,v,vp,rho,rhop,T,Tp,tau,h)

# Now put rho and T at the same time-level as v

rho = rhop
T = Tp

Listing 11.2 (Lab11Funcs.py)

import numpy as np
import scipy.linalg as la

# Physical Constants
gamma = 1.4 # Adiabatic Exponent
kappa = 0.024 # Thermal conductivity
kB = 1.38e-23 # Boltzman Constant
M = 29*1.67e-27 # Mass of air molecule (Average)
mu = 1.82e-5 # Coefficient of viscosity
F = (gamma-1)*M*kappa/kB # a useful constant

def Srho(rho,v,vp,tau,h):
# Step rho forward in time by using Crank-Nicolson
# on the continuity equation

N = len(rho)
A = np.zeros((N,N))
B = np.zeros_like(A)

# Load interior points

const = -tau/8/h
for j in range(1,N-1):
C1 = const * (v[j+1] + vp[j+1])
C2 = const * (v[j-1] + vp[j-1])
A[j,j-1] = C2
A[j,j] = 1
A[j,j+1] = -C1
B[j,j-1] = -C2
B[j,j] = 1
B[j,j+1] = C1

# Apply boundary condition

Lab 11 One Dimensional Gas Dynamics 79

# Write your code here

# Crank Nicolson solve to step rho in time

r = B@rho

return la.solve(A,r)

def ST(T,v,vp,rho,rhop,tau,h):

N = len(T)
A = np.zeros((N,N))
B = np.zeros_like(A)

# Load interior points

for j in range(1,N-1):
D1 = (v[j] + vp[j])/(4*h) + 2*F/(rho[j]+rhop[j])/h**2
D2 = (-(gamma-1) * (v[j+1] + vp[j+1] - v[j-1] - vp[j-1] )/(4*h)
- 4*F/(rho[j] + rhop[j])/h**2 )
D3 = -(v[j] + vp[j])/(4*h) + 2*F/(rho[j]+rhop[j])/h**2

A[j,j-1] = -0.5*D1
A[j,j] = 1/tau - 0.5*D2
A[j,j+1] = -0.5*D3
B[j,j-1] = 0.5*D1
B[j,j] = 1/tau + 0.5*D2
B[j,j+1] = 0.5*D3

# Apply boundary condition

# Insulating: dt/dx = 0

# Write your code here

# Crank Nicolson solve to step rho in time

r = B@T
return la.solve(A,r)

def Sv(v,vbar,vbarp,rho,rhop,T,Tp,tau,h):

N = len(rho)
A = np.zeros((N,N))
80 Computational Physics 430

B = np.zeros_like(A)
E0 = np.zeros_like(v)

# Load interior points

for j in range(1,N-1):
E0[j] = (-kB/4/h/M/(rho[j]+rhop[j]) *
( (rho[j+1] + rhop[j+1]) * (T[j+1] + Tp[j+1])
- (rho[j-1] + rhop[j-1]) * (T[j-1] + Tp[j-1])) )
E1 = (vbar[j] + vbarp[j])/(4*h)+8*mu/3/h**2/(rho[j]+rhop[j])
E2 =-16*mu/3/h**2/(rho[j]+rhop[j])
E3 =-(vbar[j] + vbarp[j])/(4*h) +8*mu/3/h**2/(rho[j]+rhop[j])

A[j,j-1] = -0.5*E1
A[j,j] = 1/tau - 0.5*E2
A[j,j+1] = -0.5*E3
B[j,j-1] = 0.5*E1
B[j,j] = 1/tau + 0.5*E2
B[j,j+1] = 0.5*E3

# Apply boundary condition

# Fixed: v = 0

# Write your code here

# Crank Nicolson solve to step rho in time

r = B@v + E0
return la.solve(A,r)
Lab 12
Solitons: Korteweg-deVries Equation

At the Lagoon amusement park in Utah, there is a water ride called the Log
Flume. It is a standard, old-fashioned water ride where people sit in a 6-seater
boat shaped like a log which slowly travels along a fiberglass trough through some
scenery, then is pulled up a ramp to an upper level. The slow ascent is followed
by a rapid slide down into the trough below, which splashes the passengers a bit,
after which the log slowly makes its way back to the loading area. But you can see
something remarkable happen as you wait your turn to ride if you watch what
happens to the water in the trough when the log splashes down. A large water
wave is pushed ahead of the log, as you might expect. But instead of gradually
dying away, as you might think a single pulse should in a dispersive system like
surface waves on water, the pulse lives on and on. It rounds the corner ahead
of the log that created it, enters the area where logs are waiting to be loaded,
pushes each log up and down in turn, then heads out into the scenery beyond,
still maintaining its shape.
This odd wave is called a “soliton”, or “solitary wave”, and it is an interesting
feature of non-linear dynamics that has been widely studied in the last 30 years, or
so. The simplest mathematical equation which produces a soliton is the Korteweg-
deVries equation
∂y ∂y ∂3 y
+y + α 3 = 0, (12.1)
∂t ∂x ∂x
which describes surface waves in shallow water. In the first two terms of this
equation you can see the convective behavior we studied in Lab 10, but the last
term, with its rather odd third derivative, is something new. We will be studying
this equation in this laboratory.
Numerical solution for the Korteweg-deVries equation
We will begin our study of the Korteweg-deVries equation by using Crank-Nicolson
to finite difference it on a grid so that we can explore its behavior by numerical
experimentation. The first step is to define a grid, and since we want to be able to
see the waves travel for a long time we will copy the trough at Lagoon and make
our computing region be a closed loop. We can do this by choosing an interval
from x = 0 to x = L, as usual, but then we will make the system be periodic by
declaring that x = 0 and x = L are actually the same point, as would be the case
in a circular trough. We will subdivide this region into N subintervals and let Figure 12.1 The log flume ride at
h = L/N and x j = ( j − 1)h, so that the grid is cell-edge. Normally such a cell-edge Lagoon produces a solitary wave
grid would have N + 1 points, but ours doesn’t because the last point ( j = N ) is (marked by arrows in the frames
just a repeat of the first point: x N = x 0 , because our system is periodic. above). The leading edge of the
soliton is where the water begins
to spill over the side of the trough.
81
82 Computational Physics 430

We now do the usual Crank-Nicolson differencing, where we evaluate each

term in the equation at time level n + 1/2. The last term in Eq. (12.1) has a third
derivative, so we’ll need a reasonable approximation for the third derivative.
Suppose you have function values f (x − 3h/2), f (x − h/2), f (x + h/2), and f (x +
3h/2). A good way to do this is to write down four Taylor’s series up to the fifth
derivative for the function at these four points and then solve this system of four
equations to find expressions for f (x), f 0 (x), f 00 (x), and f 000 (x). This procedure
yields the approximate formula

f (x + 3h/2) − 3 f (x + h/2) + 3 f (x − h/2) − f (x − 3h/2)

f 000 (x) ≈ (12.2)
h3

along with an error term on the order of h 2 . When we time average Eq. (12.2), we
find

∂3 y α ³ n+1 n+1 n+1 n+1 n n n n

´
α = y − 3y + 3y − y + y − 3y + 3y − y (12.3)
∂x 3 2h 3 j +2 j +1 j j −1 j +2 j +1 j j −1

Look closely at Eq. (12.3) and also at Eq. (12.2) to convince yourself that they are
not centered on a grid point, but at spatial location j + 1/2. The use of this third
derivative formula thus adds a new twist to the usual Crank-Nicolson differencing:
we will evaluate each term in the equation not only at time level n + 1/2, but also
at spatial location j + 1/2 (at the center of each subinterval) so that the first and
third derivative terms are both properly centered. This means that we will be
using a cell-edge grid, but that the spatial finite differences will be cell centered.
With this wrinkle in mind, we can write the first term in Eq. (12.1) at time level
n + 1/2 and space location j + 1/2 as
Diederik Korteweg (1848–1941, Dutch)
∂y 1 ³ n+1 ´
= y j + y n+1
j +1 − y n
j − y n
j +1 (12.4)
∂t 2τ
Now we have have to decide what to do about the nonlinear convection term
y∂y/∂x. We will assume that the leading y is known somehow by designating it
as ȳ and decide later how to properly estimate its value. Recalling again that we
need to evaluate at time level n + 1/2 and space location j + 1/2, the non-linear
term becomes
∂y ȳ j +1 + ȳ j ³ n+1 ´
y = y j +1 − y n+1
j + y nj+1 − y nj (12.5)
∂x 4h
For now, we’ve ignored the problem that the derivative in Eq. (12.5) is centered in
time at n + 1/2 while the ȳ term isn’t. We’ll have to deal with this issue later.
Each of these approximations in Eqs. (12.3)–(12.5) is now substituted into
Gustav de Vries (1866–1934, Dutch) Eq. (12.1), the y n+1 terms are gathered on the left side of the equation and the y n
Diederik Korteweg was Gustav’s disser- terms are gathered on the right, and then the coefficients of the matrices A and B
tation advisor.
are read off to put the equation in the form

Ay n+1 = By n (12.6)
Lab 12 Solitons: Korteweg-deVries Equation 83

in the usual Crank-Nicolson way. If we denote the four nonzero elements of A

and B like this:
A j , j −1 = a −− A j , j = a−
A j , j +1 = a + A j , j +2 = a ++
(12.7)
B j , j −1 = b −− B j , j = b−
B j , j +1 = b + B j , j +2 = b ++
then the matrix coefficients turn out to be
α 1 3α ( ȳ − + ȳ + )
a −− = − a− = + 3−
2h 3 2τ 2h 4h
1 3α ( ȳ − + ȳ + ) α
a+ = − + a ++ = 3
2τ 2h 3 4h 2h (12.8)
α 1 3α ( ȳ − + ȳ + )
b −− = 3 b− = − 3+
2h 2τ 2h 4h
1 3α ( ȳ − + ȳ + ) α
b+ = + − b ++ = − 3
2τ 2h 3 4h 2h
where y − = y j and where y + = y j +1 , the grid points on the left and the right of the
j + 1/2 spatial location (where we are centering).

P12.1 Derive the formulas in Eq. (12.8) for the a and b coefficients using the
finite-difference approximations to the three terms in the Korteweg-deVries
equation given in Eqs. (12.3)-(12.5).

Now that the coefficients of A and B are determined we need to worry about
how to load them so that the system will be periodic. For instance, in the first row
of A the entry A 1,1 is a − , but a −− should be loaded to the left of this entry, which
might seem to be outside of the matrix. But it really isn’t, because the system is
periodic, so the point to the left of j = 1 (which is also the point j = (N + 1)) is the
point j − 1 = N . The same thing happens in the last two rows of the matrices as
well, where the subscripts + and ++ try to reach outside the matrix on the right.
So correcting for these periodic effects makes the matrices A and B look like this:

a− a + a ++ 0 0 ... 0 a −−
 
 a −− a − a + a ++ 0 ... 0 0 
 
 0
 a −− a − a + a ++ ... 0 0 
A= . . . . ... . . . 
 
 
 0
 ... 0 0 a −− a − a + a ++  
 a ++ 0 ... 0 0 a −− a − a+ 
a + a ++ 0 0 ... 0 a −− a −
(12.9)
b− b + b ++ 0 0 ... 0 b −−
 
 b −− b − b + b ++ 0 ... 0 0 
 
 0
 b −− b − b + b ++ ... 0 0  
B= . . . . ... . . . 
 
 
 0
 ... 0 0 b −− b − b + b ++  
 b ++ 0 ... 0 0 b −− b − b+ 
b + b ++ 0 0 ... 0 b −− b −
84 Computational Physics 430

P12.2 Discuss these matrices with a TA and convince the TA that this structure cor-
rectly models a periodic system (it may help to think about the computing
grid as a circle with x 0 = x N .)

Now we need to settle what to do with ȳ. To properly center Crank-Nicolson

in time between t n and t n+1 we need ȳ = y n+1/2 , but this is not directly possible.
But if we use a predictor-corrector technique like we did in the last lab, we can
approximately achieve this goal. It goes like this.
We will apply Crank-Nicolson twice in each time step. In the first step (the
predictor step) we simply replace ȳ with y n , the present set of values of y, and call
the resulting new value (after Crank-Nicolson is used) ỹ n+1 , the predicted future
value. In the second step we combine this predicted value with the current value
to approximately build ȳ n+1/2 using ȳ n+1/2 ≈ (y n + ỹ n+1 )/2, then rebuild A and B
and do Crank-Nicolson again.
In schematic terms the algorithm looks like this.

ṽ = v n (12.10)

v p = A(v n )−1 B(v n )v n

£ ¤
(12.11)
vp + vn
v∗ = (12.12)
2
n+1 ∗ −1
B(v ∗ )v n
£ ¤
v = A(v ) (12.13)
All right, that’s it. You may have the feeling by now that this will all be a little
tricky to code, and it is. We would rather have you spend the rest of the time in
this lab doing physics instead of coding, so below (and on the course web site)
you will find a copy of a Python program kdv.py that implements this algorithm.
You and your lab partner should carefully study the script to see how each step of
the algorithm described above is implemented, then work through the problems
listed below by running the script and making appropriate changes to it.

Solitons
P12.3 (a) Run kdv.py with α = 0.1, y max = 2, τ = 0.5, t final = 100, and iskip=1.
2
After a while you should see garbage on the screen. This is to convince
you that you shouldn’t choose the time step to be too large.
1
v

(b) Now run (a) again, but with τ = 0.1, then yet again with τ = 0.02. Use
0 t final = 10 for both runs and skip big enough that you can see the
pulse moving on the screen. You should see the initial pulse taking off
−1
0 2 4 6 8 10 to the right, but leaving some bumpy stuff behind it as it goes. The
x
trailing bumps don’t move as fast as the big main pulse, so it laps them
Figure 12.2 A Gaussian pulse after and runs over them as it comes in again from the left side, but it still
1 second of propagation by the mostly maintains its shape. This pulse is a soliton. You should find
Korteweg-deVries equation (Prob- that there is no point in choosing a very small time step; τ = 0.1 does
lem 12.3) pretty well.
Lab 12 Solitons: Korteweg-deVries Equation 85

The standard “lore” in the field of solitons is that the moving bump you saw in
problem 12.3 is produced by a competition between the wave spreading caused
by the third derivative in the Korteweg-deVries equation and the wave steepening
caused by the y∂y/∂x term. Let’s run kdv.py in such a way that we can see the
effect of each of these terms separately.

P12.4 (a) Dispersion (wave-spreading) dominates: Run kdv.py with α = 0.1,

y max = 0.001, τ = 0.1, and t final = 10. The small amplitude makes the −4
x 10
nonlinear convection term y∂y/∂x be so small that it doesn’t matter; 10

only the third derivative term matters. You should see the pulse fall
apart into random pulses. This spreading is similar to what you saw 5
when you solved Schrödinger’s equation. Different wavelengths have

v
different phase velocities, so the different parts of the spatial Fourier
0
spectrum of the initial pulse get out of phase with each other as time
progresses.
(b) Non-linear wave-steepening dominates: Run kdv.py with α = 0.01, −5
0 2 4 6 8 10
x
y max = 2, τ = 0.01, and t final = 10. (If your solution develops short
wavelength wiggles this is an invitation to use a smaller time step. The Figure 12.3 Dispersion dominates
problem is that the predictor-correction algorithm we used on the (Problem 12.4(a).): after 3 seconds
nonlinear term is not stable enough, so we have a Courant condition of time.
in this problem.)
Now it is the dispersion term that is small and we can see the effect
of the non-linear convection term. Where y is large the convection is 2

rapid, but out in front where y is small the convection is slower. This
1
allows the fast peak to catch up with the slow front end, causing wave
v
steepening. (An effect just like this causes ocean waves to steepen and
0
then break at the beach.)
−1
The large pulse that is born out of our initial Gaussian makes it seem like 0 2 4
x
6 8 10

there ought to be a single pulse that the system wants to find. This is, in fact the
case. It was discovered that the following pulse shape is an exact solution of the Figure 12.4 Steepening dominates
Korteweg-deVries equation: (Problem 12.4(b).): after 0.5 sec-
onds of time.
12k 2 α
y(x, t ) = (12.14)
cosh2 (k(x − x 0 − 4αk 2 t ))
where x 0 is the center of the pulse at time t = 0.

P12.5 (a) Use Mathematica to show that this expression does indeed satisfy the
Korteweg-deVries equation.
(b) Now replace the Gaussian initial condition in kdv.py with this pulse
shape, using k = 1.1, x 0 = L/2, and adjusting α so that the height of
the initial pulse is exactly equal to 2, so that it matches the Gaussian
pulse you ran in 12.3. You should find that this time the pulse does
not leave trailing pulses behind, but that it moves without changing
shape. It is a perfect soliton.
86 Computational Physics 430

(c) The formula at the beginning of this problem predicts that the speed
of the pulse should be
c soliton = 4αk 2 (12.15)
Verify by numerical experimentation that your soliton moves at this
speed. To do this, you can just track the position of the maximum
value (the NumPy function argmax can help you do this).

P12.6 One of the most interesting things about solitons is how two of them interact
with each other. When we did the wave equation earlier you saw that left
and right moving pulses passed right through each other. This happens
because the wave equation is linear, so that the sum of two solutions is
also a solution. The Korteweg-deVries equation is nonlinear, so simple
superposition can’t happen. Nevertheless, two soliton pulses do interact
with each other in a surprisingly simple way.
To see what happens keep α = 0.1, but modify your code from 12.5 so that
you have a soliton pulse with k = 1.5 centered at x = 3L/4 and another
soliton pulse with k = 2 centered at x = L/4. Run for about 20 seconds and
watch how they interact when the fast large amplitude pulse in the back
catches up with the slower small amplitude pulse in the front. Is it correct
to say that they pass through each other? If not, can you think of another
qualitative way to describe their interaction?

Listing 12.1 (kdv.py)

import matplotlib.pyplot as plt

import numpy as np
import scipy.linalg as la

# Physical constants
alpha = 0.1

# Make the grid

N = 500
L = 10
h = L/N
x = np.linspace(h/2,L-h/2,N)

# Initial Gaussian centered on the computing region

ymax = 2
y = ymax * np.exp(-(x-.5*L)**2)

# Time range
tau = 0.1
tfinal = 100
t = np.arange(0,tfinal,tau)
Lab 12 Solitons: Korteweg-deVries Equation 87

# Initialize the parts of the A and B matrices that

# do not depend on ybar and load them into At and Bt.
# Make them be sparse so the code will run fast.
At = np.zeros((N,N))
Bt = np.zeros((N,N))

# Function to wrap the column index

def jwrap(j):
if (j < 0):
return j + N
if (j >= N):
return j - N
return j

# load the matrices with the terms that don't depend on ybar
h3 = h**3
for j in range(N):
At[j,jwrap(j-1)] =-0.5*alpha/h3
At[j,j] = 0.5/tau + 1.5*alpha/h3
At[j,jwrap(j+1)] = 0.5/tau - 1.5*alpha/h3
At[j,jwrap(j+2)] = 0.5*alpha/h3

Bt[j,jwrap(j-1)] = 0.5*alpha/h3
Bt[j,j] = 0.5/tau - 1.5*alpha/h3
Bt[j,jwrap(j+1)] = 0.5/tau + 1.5*alpha/h3
Bt[j,jwrap(j+2)] =-0.5*alpha/h3

plt.figure(1)
skip = 10
for n in range(len(t)):
# Predictor step
A = np.copy(At)
B = np.copy(Bt)

# load ybar, then add its terms to A and B

ybar = np.copy(y)
for j in range(N):
tmp = 0.25*(ybar[jwrap(j+1)] + ybar[j])/h
A[j,j] = A[j,j] - tmp
A[j,jwrap(j+1)] = A[j,jwrap(j+1)] + tmp
B[j,j] = B[j,j] + tmp
B[j,jwrap(j+1)] = B[j,jwrap(j+1)] - tmp
88 Computational Physics 430

# do the predictor solve

r = B@y
yp = la.solve(A,r)

# corrector step
A = np.copy(At)
B = np.copy(Bt)

# average current and predicted y values to correct ybar

ybar=.5*(y+yp)
for j in range(N):
tmp = 0.25*(ybar[jwrap(j+1)] + ybar[j])/h
A[j,j] = A[j,j] - tmp
A[j,jwrap(j+1)] = A[j,jwrap(j+1)] + tmp
B[j,j] = B[j,j] + tmp
B[j,jwrap(j+1)] = B[j,jwrap(j+1)] - tmp

# do the final corrected solve

r = B@y
y = la.solve(A,r)

if (n % skip == 0):
plt.clf()
plt.plot(x,y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('time={:1.3f}'.format(t[n]))
plt.ylim(0,3)
plt.pause(.1)
Lab 13
Machine Learning with Neural Networks

Machine Learning Terminology

In this lab, we’ll dip our toes in the field of machine learning. In very loose terms,
machine learning boils down to the process of using general algorithms to model
(sometimes complicated) relationships between system parameters. In a typical
scenario we’ll have an observable quantity which is thought to relate to a set
of input data. For example, suppose you have a bunch of pictures of pets that
you want to classify by the type of animal in the picture (cat, dog, parrot, etc.).
The observable quantity is the type of animal, and the data consists of the pixel
locations and values in the picture.
To get started on this picture classification project, you could manually classify
a bunch of pictures by the type of animal in them and then provide that training
data to the machine learning algorithm. The algorithm uses the training data
to come up with a model that takes a picture as input, and predicts the type
of animal in the pictures. You can then use the output model to automatically
classify the rest of your pet pictures. This process is called supervised learning.

Supervised Learning: The researcher provide a machine learning algorithm with

a “training” data set where the outcomes for each of the given data inputs
are known, and the algorithm comes up with a function relating the inputs
and outputs that can then be used on new data

Model: The output of a machine learning algorithm that represents a relationship

between inputs and outputs.

The picture-sorting procedure outlined above is an example of a classification

problem. Classification algorithms are designed to fit sets of data into discrete
bins (cat, dog, etc.). In contrast, regression algorithms aim to predict a value
based on the input data. For example, you might try to use machine learning
to predict the temperature tomorrow. In this case, the measurable quantity is a
number representing the temperature, and the input data would be things like
the historic temperature record, the date, the location, recent weather, etc.

Classification: The measurable quantity is restricted to a discrete set of values.

Regression: The measurable quantity is allowed to take on a continuous range

of values.

Sometimes, you don’t have a measurable quantity you want to predict from
your data set, but rather want to explore the structure of the data itself. For

89
90 Computational Physics 430

example, say Facebook wants to divide their users into groups of people with
similar demographics so they can target advertising to these groups. They want
to be able to say something like “this ad is likely to be interesting to demographic
group α but uninteresting to group β.” But they don’t know the best way to define
the groups α and β. This is a job for a clustering algorithm, which is a type of
unsupervised learning. Facebook could feed a bunch of information about a set
of people into a clustering algorithm, and the algorithm would return suggestions
for how to assign those people to groups “similar” demographics (i.e. tell you how
to define groups α and β above). Once they’ve classified the data into groups,
they could then use a supervised machine learning algorithm to predict which
groups might be likely to click on which advertisements.

Unsupervised Learning: An algorithm that looks for patterns or groupings in a

set of data without respect to any presuppositions about the data.

Clustering: A type of unsupervised learning algorithm that separates similar

data into groups or categories.

P13.1 Explain the machine learning concepts introduced above to a TA.

There are many, many more ideas in machine learning. You can take many full
courses on the subject. But just to get the flavor of how this works, let’s tackle
some specific problems using a common machine learning technique referred to
as a neural network.

Linear Regression
A computational neural network can be thought of as an extension of linear
regression. Linear regression finds a linear function which maps an input, x, onto
an output, y. Circles are often used to represent the nodes (values) of a neural
network. Using that notation, a linear regression mapping would be represented
this way:
Study Time (h) Score (%)
1.4 20
1.8 30
2.1 37
2.2 45
Questions that linear regression can answer are along the lines of this: given a set
2.3 26 of known inputs with corresponding known outputs, can one predict what the
2.9 86 output will be for a new input that wasn’t part of the original set? As an example,
3.5 67 perhaps you are trying to correlate exam scores with the amount of time students
3.6 100
studied for the exam as shown in Table 13.1.
4.2 72
4.6 82 You could make a plot of the data and fit a line to it as shown in Fig. 13.1.
4.7 99 Then you could use your fit to make a prediction for a student not in the given
data set, such as “Given a time x = 2.7 hours, the expected exam score would
Table 13.1 Table of exam scores vs. be y = 53 percent.” The linear fit is typically done by choosing the slope and
study time. intercept parameters to minimize some cost or loss function, for example the
squared error.
Lab 13 Machine Learning with Neural Networks 91

Simple linear prediction is often insufficient. For example, this linear fit
will predict a negative score for particularly low values of study time, which is
nonsensical. So it would make sense to apply a nonlinear function after the linear
fit, something like y = max(0, z) where we are now using the symbol z to represent
the value that is given from the linear fit, z = w x + b, so we can continue to use
the symbol y for the final output. (We are using the letter w to represent the
slope of the line for reasons made clear below.) Applying that particular nonlinear
function is equivalent to saying, “Set y to 0 if z is negative, otherwise use y = z”.
In a neural network, a function applied after the linear fit is called an ac-
tivation function, and that particular activation function is probably the most
commonly used one. It is called the relu function, which stands for rectified linear
unit. Other activation functions are also frequently used. For example, another Figure 13.1 Simple linear fit of the
common situation is when the output value must be between 0 and 1; in that case test data shown in Table 13.1.
a nonlinear function called a sigmoid function, is used to map all the z values
(potentially ranging from −∞ to +∞) to real values of y between 0 and 1.

Multiple input linear regression

Linear regression can also be done with multiple inputs. Continuing the exam
score example, additional inputs could represent other known quantities such
as how much sleep the student got the night before the exam, the student’s class
attendance, the student’s grades on homework and past exams, etc. The situation
might now look like this; we are using z, y notation at the final node to indicate
that the output node has both pre- and post-activation values.

The linear function which would now be fit is z = w 1 x 1 + w 2 x 2 + w 3 x 3 + w 4 x 4 + b,

and then an activation function could be applied to the value of z to obtain y. The
slopes of the linear regression are now called weights, hence the symbol w. Each
w is represented in the figure by an arrow. The optimal values of the parameters
w 1 , w 2 , w 3 , w 4 , and b will be calculated together using known data to minimize a
cost function.

Neural networks
A neural network is this same idea as multiple regression, just expanded to multi-
ple layers. The layers between input and output are called hidden layers. Each
hidden layer could have its own activation function. Post-activation values of
the nodes of the hidden layers are called the activations of the nodes and are
92 Computational Physics 430

represented by the letter a. Here’s an example of a three layer neural network; by

convention, the input layer of x’s is not counted in the number of layers of the
network.

}
}
}
Input Layer 3,
Layer 1 Layer 2
Layer Output Layer

Figure 13.2 Example of a fully connected three-layer network.

Note that in this example, layer 1 is a collection of 15 parameters (4 w’s and 1

b for each of the three nodes in layer 1), an activation function, and 3 nodes.
This nomenclature is not necessarily uniform; some people may just refer to the
nodes themselves when discussing a layer. Also, sometimes the a and z values are
counted as separate layers, in which case this example would be called a six-layer
neural network. The structure of the network, meaning the number of layers, and
the number of nodes in each layer, and their activation functions, is chosen with
the goal of obtaining the best possible predictions. It is fairly common, however,
to reduce the number of nodes in each layer as one moves from input to output
as in this example here.

More complicated layers

The layers shown in Fig. 13.2 are termed fully connected layers, meaning each
node of the previous layer is connected via an arrow (i.e. a w) to each node of
the next layer. Incidentally, this type of network is called a multilayer perceptron.
More complicated layers can also be used. One type of layer which is frequently
used in image analysis is called a convolutional layer. A color image input is
represented numerically by a 3-dimensional array: 2 dimensions for horizontal
and vertical values, plus a third dimension specifying whether the given numbers
refer to the R, G, or B color channel. A convolutional layer takes a 3-dimensional
array as an input, does an operation on it, and then passes a new 3-d array to
the next layer. One can have multiple convolutional layers in a network, but to
convert from a convolutional layer to a fully connected layer at some point, the
data must be “flattened” from a 3-d array to a 1-d set of values.

Gradient descent
In the specific neural net shown in Fig. 13.2, layer 1 contained 15 fitting parame-
ters. Similarly, layer 2 contains 12 parameters and layer 3 has 4 parameters. (Take
Lab 13 Machine Learning with Neural Networks 93

a moment to look at the figure and make sure these numbers make sense to you.)
Therefore 31 total parameters need to be optimized using the input data. For
more complicated neural nets, it is not uncommon for there to be hundreds of
thousands of fitting parameters! How are these to be optimized? The process is
typically done using the gradient descent method (or a close relative of it).
Gradient descent is an iterative process whereby the location of the minimum
of a function (e.g. the cost function) is found by taking steps along the “downhill”
direction, i.e. the direction of the negative gradient. This is most easily visualized
with a function of two parameters, say w 1 and w 2 , as illustrated in Fig 13.3. cost
At each step in the iteration, the cost function is computed based on the
random starting point
current values of w 1 and w 2 , the gradients of the cost function with respect to w 1
and w 2 are computed at the present location of w 1 and w 2 , and then the values
of w 1 and w 2 are updated according to:

d (cost)
w 1,next = w 1 − α
d w1
Figure 13.3 Taking steps by ad-
d (cost) justing the w 1 and w 2 parameters
w 2,next = w 2 − α
d w2 along the negative gradient direc-
tion to find the minimum of the
For known activation functions and a known cost function, the gradients can
cost function.
be automatically computed by the machine learning software in an interesting
backpropagation process that minimizes calculation time, but is beyond the
scope of this lab.

Hyperparameter tuning and training/validation sets

In those equations, the symbol α is called the learning rate and must be carefully
chosen. It describes how large a step is taken during each iteration. Too small of
a value and the process will take many iterations to converge. Too large a value
and the steps may overshoot the minimum that is being sought. The input α is
an example of a hyperparameter, meaning a parameter that affects the machine
learning but is not automatically optimized by the algorithm itself (unlike the w’s
and b’s). Other examples of hyperparameters include the number of layers, num-
ber of nodes in each layer, the activation functions, and even the cost function
itself. Sometimes these hyperparameters can be tuned in a systematic way, but
often selecting the best hyperparameters involves a degree of trial and error.
How can one decide which hyperparameters are best? It would be tempting
to just look at the cost function and pick the hyperparameters which produce
the lowest possible value. However, with potentially hundreds of thousands of
fitting parameters, the danger of overfitting the data is very real, meaning you
could end up with a set of parameters that do a fantastic job on fitting the given
data, but which make horrible predictions on any new data. To guard against
that, the known data is typically separated into a training set and a validation set.
The validation set is also known as the development set or “dev set”; sometimes it
is also called the “test set,” although at times “test set” refers to a third set used
94 Computational Physics 430

for additional testing after the model is complete so we will avoid that usage.
The training set is used to optimize the network parameters to minimize the
cost function, i.e., to “train the model”, and then the trained neural network is
checked against the validation set to make sure the network can be relied on to
give predictions which make sense.

The Keras library

Fortunately, the many steps involved with training and testing neural networks
are greatly simplified using machine learning libraries, of which one of the most
used for Python is called keras. The examples below all use keras. Use this
pip install command:
pip install keras

Use these additional commands if needed:

pip install tensorflow

pip install -U numpy

If keras still does not work after those commands, you may have a version conflict.
In that case, see Appendix A for some basics about environments in Anaconda
and download an environment file from our website, which should help. Install
and activate it, and then the specified libraries should work together.

Example 1: Predicting exam scores

Here is what the code would look like to use a neural net framework to analyze
the exam score vs. study time data set given above, and then to make a prediction
for x = 2.7 hours. As a quick note about notation, while the letters x and y are
commonly used to refer to a single input data point and its corresponding output,
the collections of all inputs and outputs are usually referred to as capital X and Y .
This code sets up a 1 layer neural network with 1 node in the layer, which
receives a single input from an input layer. The model=Sequential() command
creates a sequential (normal) neural net with name model. The layer is created
via the model.add(Dense(1, activation='linear', input_dim=1)) com-
mand, where the first 1 refers to the number of nodes in the layer and the second
1 designates the size of the input layer (a single input node). We use sgd as the
optimizing routine, which stands for stochastic gradient descent, and a mean
squared error cost (or loss) function. For this simple example we are not bothering
with a validation set and we have used a linear activation function (meaning the
identity, y = z) instead of a relu one. The predicted y for x = 2.7 should be about
y ≈ 54.4.

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
Lab 13 Machine Learning with Neural Networks 95

from keras import optimizers

data = np.array([[1.4, 20.], [1.8, 30.], [2.1, 37.], [2.2, 45.],

[2.3,26.], [2.9, 86.], [3.5, 67.], [3.6, 100.],
[4.2, 72.], [4.6, 82.], [4.7, 99.]])

[X_train,Y_train] = data.transpose()

model = Sequential()
model.add(Dense(1, activation='linear', input_dim=1))
sgd = optimizers.sgd(lr=0.03) #lr is the learning rate
model.compile(loss='mean_squared_error',optimizer=sgd)

#this command does the minimization

model.fit(X_train, Y_train, epochs=100, verbose=1)

#displays some useful info, note how there are only 2 fitting parameters
model.summary()

# the predict function on the next line assumes an

# array of inputs so we need to put 2.7 inside an array
x = np.array([2.7])

#... and predicted_y is likewise an array of predicted outputs

predicted_y = model.predict(x)

print('For x={:f} the predicted y is: {:f}'.format(float(x),float(predicted_y)))

P13.2 Execute the code above, and examine the output. Explain to the TA what
the code does.

Example 2: Predicting median home prices

A commonly used data set for testing machine learning algorithms, and one
built into keras, is that of Boston home prices from the 1970s. It involves 13
metrics taken for 506 different communities in the Boston area, along with the
median home price for the community. Reading it into your code from the
keras.datasets as is done below will automatically sort the data into 404 train-
ing data points and 102 validation data points with the 13 metrics as the X vector
and the median home price as Y . Technically X is a 2-d array, because each x is a
collection of 13 pieces of information, and X is the collection of all x’s for all 404
training data points.
This code sets up a five-layer neural network with 30, 15, 8, 5, and 1 nodes
per layer respectively. There are 1,064 total fittable parameters. Notice how each
layer is just added via a model.add command, and only the first input layer
needs to have input_dim specified (which is the number of nodes in the input
layer). In this example adam is used as the optimizing routine; it is a slightly
96 Computational Physics 430

more sophisticated version of gradient descent and is a popular choice for many
applications.

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras import optimizers
from keras.datasets import boston_housing

(X_train, Y_train), (X_validate, Y_validate) = boston_housing.load_data()

#description here: https://github.com/eric-bunch/boston_housing

print(X_train.shape)
print(Y_train.shape)
print(X_validate.shape)
print(Y_validate.shape)

model = Sequential()
model.add(Dense(30, activation='relu',input_dim=13))
model.add(Dense(15, activation='relu'))
model.add(Dense(8, activation='relu'))
model.add(Dense(5, activation='relu'))
model.add(Dense(1, activation='linear'))
optimizer_choice = optimizers.adam(lr=0.05)
model.compile(loss='mean_squared_logarithmic_error',
optimizer=optimizer_choice, metrics=['mse'])
model.fit(X_train, Y_train, batch_size=32, epochs=50, verbose=1)
model.summary()
score = model.evaluate(X_validate, Y_validate, verbose=0)

print('The loss value and accuracy of the test set is: ' + str(score))

#prediction for a specific input

test_num = 27 #randomly chosen number
x = np.array([X_validate[test_num]])
predicted_y = model.predict(x)
print('For input parameters x = ')
print(X_validate[test_num])
print('the predicted y (median value of home, in $1000s) is:
{:f} '.format(float(predicted_y)))
print('and the actual y value was: {:f}'
.format(float(Y_validate[test_num])))

P13.3 Execute the code above, and examine the output. Explain to the TA what
the code does.
Lab 13 Machine Learning with Neural Networks 97

Example 3: Predicting handwritten digits

Now we will do a more complicated example using image classification with the
MNIST data set which contains images of handwritten digits, each labeled with
the answer for the correct digit. Our neural net will use the data set to learn how
to recognize the digits. Each data point is a 28×28 pixel monochrome image.
Reading the data set into your code from the keras.datasets as is done below will
automatically sort the data into 60,000 training data points and 10,000 validation
data points with the with the pixel values as X and the correct digit as Y .
This code sets up a 4 layer neural network where the first two layers are
of the convolutional type mentioned above (convolutional hyperparameters:
32 convolutional “filters” and a 3 × 3 convolutional grid for each layer), fol-
lowed by two regular (dense) layers of sizes 128 and 10, respectively. The out-
put layer has size 10 because for categorizing data like this, the output of the
neural net needs to be 10 different outputs, each one corresponding to the like-
lihood of the image being a particular digit. So instead of 3, 5, or 6, our outputs
need to be things like [0 0 0 1 0 0 0 0 0 0], [0 0 0 0 0 1 0 0 0 0], and
[0 0 0 0 0 0 1 0 0 0].
The input shape of the X values must be specified for the first layer; after
that, keras is smart enough to determine the needed array sizes and shapes. The
output of the second layer must be flattened prior to going into the third layer.
Some of the layers employ dropout which helps prevent overfitting. Pooling is
a technique to reduce the size of the convolutional arrays.1 The output layer, as
mentioned above, needs to have 10 separate outputs. The softmax activation
choice for the final layer makes it so the 10 output values sum to 1 and can be
interpreted as probabilities. The cost function is categorical_crossentropy
which is the typical choice for sorting data into categories. Because this is a more
complicated example, some additional comments are given in the code.

import numpy as np
from matplotlib import pyplot as plt
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import np_utils
from keras.datasets import mnist

(X_train, Y_train), (X_validate, Y_validate) = mnist.load_data()

#X_train has dimensions (60000, 28, 28): 60,000 images which are 28x28

#number of data points in training set traditionally called m

m = X_train.shape[0]

#view two of the images, images 10 and 11 just chosen randomly

1 Some sources may refer to the pooling, dropout, and flattening functions as existing in their

own layers, but we prefer drawing the boundaries between layers at the convolution and dense
stages only.
98 Computational Physics 430

plt.figure(1)
plt.imshow(X_train[10])
plt.figure(2)
plt.imshow(X_train[11])

#view the image labels corresponding to the two images

print('image 10 label:' + str(Y_train[10]))
print('image 11 label:' + str(Y_train[11]))

#There are three problems with the supplied data.

# First, the convolutional layers will expect the image data

# to be in a "channel", even if it's just a monochrome channel.
# Let's fix that.
X_train = X_train.reshape(m, 28, 28, 1)

#Now the dims are (60000, 28, 28, 1)

# Side note: this would need to be (60000, 28, 28, 3)

# if we had RGB image data

# Secondly, neural nets tend to work best for inputs

# between 0 and 1, so let's normalize the data
X_train = X_train/255

# Thirdly, the y-values are just digits rather than the

# needed arrays of size 10 (corresponding to probabilities
# of digits 0-9, respectively). There's a built-in function
# called "to_categorical" for that.
Y_train = np_utils.to_categorical(Y_train, 10) #now has dims (60000, 10)

#view the new image labels corresponding to the two images

print('new image 10 label:' + str(Y_train[10]))
print('new image 11 label:' + str(Y_train[11]))

#Now to define the neural net structure and hyperparameters.

model = Sequential()

#layer 1:
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28,28,1)))

#layer 2:
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))

#layer 3:
model.add(Flatten())
model.add(Dense(128, activation='relu'))
Lab 13 Machine Learning with Neural Networks 99

model.add(Dropout(0.5))

#layer 4 (output):
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam',
metrics=['accuracy'])

# And now we train the model. This takes a long time, perhaps
# 10 mins depending on your computer, since there are over
# 600,000 adjustable parameters in the 4-layer model we have
# just defined! The bulk of them come from the 4608 outputs
# from layer 2 connecting densely to 128 nodes in layer 3.
# The "batch_size" setting in the command indicates how many
# training inputs it should consider at once as it is adjusting
# the parameters, and the "epochs" number indicates how many
# complete iterations through the entire training set it should do.

model.fit(X_train, Y_train, batch_size=32, epochs=6, verbose=1)

model.summary()

# And now it's done! We can make predictions. To get a general sense as to
# the quality of the NN predictions, we can use the validation set... but we
#must first address the same three issues we did with the training set.

m_test = X_validate.shape[0]

print('Test set has {:d} entries.'.format(m_test))

X_validate = X_validate.reshape(m_test, 28, 28, 1)

X_validate = X_validate/255 # to normalize X values between 0 and 1
Y_validate = np_utils.to_categorical(Y_validate, 10)

#the output of this next command will tell you how good the NN did on the test set
score = model.evaluate(X_validate, Y_validate, verbose=0)
print('The loss value and accuracy of the test set is: ' + str(score))

#It's also fun to look at predictions for single examples. "140" here was just
#a random selection. You can copy and paste these next few lines into the
#console with different test numbers to see an image along with its predicted
#output value.

testimagenumber = 140
singletest=X_validate[testimagenumber]

plt.figure(3)

#must convert back to format imshow can use

plt.imshow(np.round(singletest*255).squeeze())

#model.predict expects an array, so give it an array of length 1

100 Computational Physics 430

singleprediction = model.predict(np.array([singletest]))[0]

#argmax function converts from the length 10 output array back to a single digit
singleprediction = np.argmax(singleprediction)

print('The prediction for image {:d} is: {:d} '

.format(testimagenumber, singleprediction))

P13.4 Execute the code above, and examine the output. Explain to the TA what
the code does.

Other resources
We have just scratched the surface of machine learning and neural networks. Here
are a few more resources which you may find helpful if you want to do additional
exploring.

• How to save and load trained models: keras.io/getting-started/faq/

• List of keras built-in activation functions: keras.io/activations/

• List of keras built-in cost functions: keras.io/losses/

• List of keras built-in optimizers: keras.io/optimizers/

• Some good advice on how to choose a cost function:

machinelearningmastery.com/how-to-choose-loss-functions-when-training-
deep-learning-neural-networks/

Petals and Sepals

Another classic data set used for testing/demonstrating machine learning algo-
rithms is the Iris flower data set. This is a classification problem where four flower
Petal
characteristics (petal length, petal width, sepal length, and sepal width) are used
to determine whether an iris is of the Iris setosa, Iris virginica or Iris versicolor
varieties. Below is some code which loads the dataset and randomly splits it
into the training and validation sets. Note the two sklearn commands; sklearn
Sepal is the implementation of scikit-learn, another free software machine learning
library for Python.

from sklearn.datasets import load_iris

Petal from sklearn.model_selection import train_test_split

iris = load_iris()
X = iris['data'] #150 entries, 4 parameters each
Y = iris['target'] #150 entries, values of 0, 1, or 2
Sepal

Figure 13.4 Petals and a sepals

for Iris versicolor (top) and Iris
virginica (bottom). (Photos from
Lab 13 Machine Learning with Neural Networks 101

names = iris['target_names']
feature_names = iris['feature_names']

X_train, X_validate, Y_train, Y_validate \

= train_test_split(X, Y, test_size=0.333)

You may want to print out any or all of the defined variables to see what you are
working with.

P13.5 Develop a neural net which will learn to separate the irises based on the
four characteristics, with training and validation accuracies both over 95%.

Hint: This problem is much like Example 3 in that it’s a classification problem.
Here are some specific things you should do which are quite similar to that
example:

• You’ll need to import the appropriate keras libraries.

• Use the np_utils.to_categorical function to turn the outputs which

are 0, 1, or 2 into three separate outputs which are each 0 or 1. You will need
to do this for Y_validate as well as for Y_train.

• Use a softmax activation function for the final layer, but with 3 nodes as
opposed to 10 nodes

• Use the categorical_crossentropy cost function

• Use the adam optimizer

• Use metrics=['accuracy'] in your compile command

• Evaluate your model with X_validate and Y_validate

Unlike Example 3, you should not use any convolution layers; instead, do
the problem with a fully connected network like Example 2. You can choose the
number of layers (at least 2, though) and the number of nodes in each layer. Also
like Example 2, be sure to specify the size of the input layer in your first model.add
command. If your network seems to be overfitting the data, as seen by a very high
accuracy with the training data but a low accuracy with the validation data, try
simplifying your network.
A note on the model.compile command: You may have noticed that the
model.compile command was used in two slightly different ways for Examples 2
and 3. In Example 2 it was done like this:
optimizer_choice = optimizers.adam(lr=0.05)

model.compile(loss='mean_squared_logarithmic_error',
optimizer=optimizer_choice, metrics=['mse'])

Whereas in Example 3 it was like this:

102 Computational Physics 430

model.compile(loss='categorical_crossentropy',
optimizer='adam',metrics=['accuracy'])

The difference between the two is that if you want to specify any hyperparameters
for the optimizer (e.g. adam), such as the learning rate, then you must precede
the compile command with an optimizers command as in Example 2. And then
inside the model.compile command you set the optimizer equal to the name
of the optimizer variable you have just created, without quotation marks. If,
however, you are content to use the default hyperparameters for the optimizer,
then inside the model.compile command, you can set the optimizer equal to a
string indicating your choice, in quotes.
Appendix A
Anaconda Environments
With the large number of packages and libraries available for use with Python,
it is not uncommon for them to conflict with each other. (Libraries are collections
of packages.) Additionally, sometimes a package may work with one version of
Python but not with another. To address these types of issues, Anaconda has
implemented the ability to create and load “environments”, and in fact that is
one of the compelling reasons to use the Anaconda version of Python in research
settings where multiple people may be sharing code with each other.
An environment file is basically a list of all of the package versions that are
desired for a given configuration, including the version of Python itself. By loading
an environment file in Anaconda prior to running Python, the user tells Anaconda
to force Python to use those specified versions. That way, for example, someone
working on code can save his or her environment, send the environment file to a
collaborator along with the code, and be sure that the collaborator will be able
to run the code in the exact same way, with no version conflict errors or other
unexpected behavior.
Environments are created and loaded from the Anaconda Prompt, an app
which was installed when you installed Anaconda. The Anaconda Prompt is a
command line interface which lets you run commands related to Anaconda itself
(as opposed to Python commands).
Some of the details below may differ slightly based on your operating system.
For simplicity we will assume a Windows machine. If you open the Anaconda
Prompt from the Windows search menu, you should see the following text:

(base) C:\Users\yourname>

where yourname is your Windows username. The (base) prompt means that it
is using the default environment as opposed to a specially loaded one, and the
given directory (i.e. C:\Users\yourname>) is the default working directory from
which Anaconda will save and load files. After you load an environment, (base)
will be replaced by the name of the environment.
Environment files are often titled “environment.yml”. To load an environment,
first copy the environment.yml file into the working directory and type this into
the command prompt:

conda env create -f environment.yml

The -f option means the next word will be the filename. This command will
cause Anaconda to download all of the packages which are specified inside the
environment.yml file, with specific versions. It may take a while but is a one-time
thing.

103
104 Computational Physics 430

After Anaconda finishes with that command you have an environment you
can use! The name of the environment is specified inside the environment.yml,
and Anaconda will let you know its name when it is finished, by generating a
response like this [where “yourenvironment” is whatever name was specified
inside the file]:

# To activate this environment, use

#
# $ conda activate yourenvironment
#
# To deactivate an active environment, use
#
# $ conda deactivate

At this point you will be able to see that a new folder with the yourenvironment
name has been created in your C:\Users\yourname\Anaconda3\envs direc-
tory, filled with lots of stuff you don’t need to worry about. If you have multiple
environments installed, you can check that directory to see them all.
As the response indicates, to use the environment you should type this into
the command prompt:

conda activate yourenvironment

That tells Anaconda to start using the specified environment and the command
prompt will now look like this:

(yourenvironment) C:\Users\yourname>

For future usages, you can skip the conda env create command and just use the
conda activate command. As also indicated by the response, to go back to the
default environment without exiting and restarting Anaconda, you would type in:

conda deactivate

After activating an environment, type spyder into the Anaconda Prompt com-
mand line to start up Spyder using that environment. Note that starting Spyder
via the Windows search menu at this point will still run Spyder with the default
packages and default Python version, not with the new environment, so you
do need to start Spyder via the Anaconda Prompt. Also, even though you may
activate an environment which includes various packages, when running Python
code you still need to use import packagename-type commands in your Python
file. The difference is that when an environment is active, Python will import the
specified version of the package as opposed to just the default version.
If you would like to create your own environment, you can do so via the conda
create command. As an example, to create an environment named “newenviron-
ment” to run Python version 3.7.6 with the numpy, scipy, keras, tensorflow, and
matplotlib packages, you would type the following into the command prompt:
Lab A Anaconda Environments 105

conda create -n newenvironment python=3.7.6 numpy scipy keras tensorflow matplotlib

The -n option indicates that the next word will be the name of the environment.
The above command will create a new folder in the

C:\Users\yourname\Anaconda3\envs

directory by the name of “newenvironment”, and then that environment will

thereafter be available for use via conda activate commands in the future.
You might want to create an environment like this for yourself if, for example,
you periodically want to use a package that requires a particular version of Python
to run well. If you additionally need to specify a particular version of a package,
you can do so by adding equals signs and numbers to the command, such as
scipy=0.15.0 instead of just scipy. If you don’t explicitly specify the version of
a package, Anaconda will just use the latest version.
To additionally create an environment.yml file you can share with others,
make sure the desired environment is active, then type this command:

conda env export > environment.yml

The environment.yml will be created in the working directory (overwriting any

other environment.yml file which may be present, so be careful). Your colleagues
will then be able to use that file to reproduce your environment on their own com-
puters by running conda env create and conda activate commands as explained
above.
Incidentally, we have created an environment specifically to assist with ver-
sion conflicts in Lab 13 using a conda create command similar to the example
above, and the environment.yml file is available for download on our website. It
creates an environment called pythonML which should allow you to run all of the
machine learning examples from Lab 12.
For more information on environments, see the documentation provided
here.
Appendix B
Glossary of Terms

by Dr. Colton and students in the Winter 2010 semester

Algorithm a set of steps used to solve a problem; frequently these are expressed
in terms of a programming language.

Analytical approach finding an exact, algebraic solution to a differential equa-

tion.

Cell-center grid a grid with a point in the center of each cell. For instance, a
cell-center grid ranging from 0 to 10 by steps of 1 would have points at 0.5,
1.5, . . . , 8.5, 9.5. Note that in this case, there are exactly as many points as
cells (10 in this case).

Cell-center grid with ghost points the same as a regular cell-center grid, but
with one additional point at each end of the grid (with the same spacing as
the rest). A cell-center grid with ghost points ranging from 0 to 10 by steps
of 1 would have points at -0.5, 0.5, . . . , 9.5, 10.5. Note that in this case, there
are two more points than cells. This arrangement is useful when setting
conditions on the derivative at a boundary.

Cell-edge grid a grid with a point at the edge of each cell. For instance, a cell-
edge grid ranging from 0 to 10 by steps of 1 would have points at 0, 1, . . . ,
9, 10. Note that in this case, there are actually N − 1 cells (where N is the
number of points: 11 in this case). The discrepancy between the number
of cell and number of points is commonly called the “fence post problem”
because in a straight-line fence there is one more post than spaces between
posts.

Centered difference formula a formula for calculating derivatives on grids whereby

the slope is calculated at a point centered between the two points where
the function is evaluated. This is typically the best formula for the first
derivative that uses only two values.

f (x + h) − f (x − h)
f 0 (x) ≈
2h

Courant condition also called the Courant-Friedrichs-Lewy condition or the

CFL condition, this condition gives the maximum time step for which an
explicit algorithm remains stable. In the case of the Staggered-Leap Frog
algorithm, the condition is τ > h/c.

107
108 Computational Physics 430

Crank Nicolson method an implicit algorithm for solving differential equations,

e.g. the diffusion equation, developed by Phyllis Nicolson and John Crank.

Dirichlet boundary conditions boundary conditions where a value is specified

at each boundary, e.g. if 0 < x < 10 and the function value at x = 10 is forced
to be 40.

Dispersion the spreading-out of waves

Eigenvalue problem the linear algebra problem of finding a vector g that obeys
Ag = λg.

Explicit algorithm an algorithm that explicitly use past and present values to
calculate future ones (often in an iterative process, where the future values
become present ones and the process is repeated). Explicit algorithms are
typically easier to implement, but more unstable than implicit algorithms.

Extrapolation approximating the value of a function past the known range of

values, using at least two nearby points.

Finite-difference method method that approximates the solution to a differen-

tial equation by replacing derivatives with equivalent difference equations.

Forward difference formula a formula for calculating derivatives on grids whereby

the slope is calculated at one of the points where the function is evaluated.
This is typically less exact than the centered difference formula, although
the two both become exact as h goes to zero.

f (x + h) − f (x)
f 0 (x) ≈
h

Gauss-Seidel method a Successive Over-Relaxation technique with w = 1.

Generalized eigenvalue problem the problem of finding a vector g that obeys

Ag = λBg.

Ghost points see “Cell-center grid with ghost points.”

Grid A division of either a spatial or temporal range (or both), used to numerically
solve differential equations.

Implicit algorithm an algorithm that use present and future values to calculate
future ones (often in a matrix equation, whereby all function points at a
given time level are calculated at once). Implicit algorithms are typically
more difficult to implement, but more stable than explicit algorithms.

Interpolation approximating the value of a function somewhere between two

known points, possibly using additional surrounding points.
Lab B Glossary of Terms 109

Iteration repeating a calculation a number of times, usually until the error is

smaller than a desired amount.

Korteweg-deVries Equation a differential equation that describes the motion of

a soliton.

Lax-Wendroff algorithm an algorithm that uses a Taylor series in time to obtain

a second-order accurate method

Neumann boundary conditions boundary conditions where the derivative of

the function is specified at each boundary, e.g. if 0 < x < 10 and at x = 10
the derivative d x/d t is forced to be 13.

Resonance the point at which a system has a large steady-state amplitude with
very little driving force.

Roundoff an error in accuracy that occurs when dealing with fixed-point arith-
metic on computers. This can introduce very large errors when subtracting
two numbers that are nearly equal.

Second derivative formula this is a centered formula for finding the second
derivative on a grid:

f (x + h) − 2 f (x) + f (x − h)
f 00 (x) ≈
h2

Staggered-Leap Frog an algorithm for solving differential equations (e.g. the

wave equation) whereby the value of a function one time step into the
future is found current and past values of the function. It suffers from the
difficulty at the start of needing to know the function value prior to the
initial conditions.

Steady State solutions to differential equations in which the system reaches an

equilibrium solution that continues on forever in the same fashion.

Successive Over-Relaxation A way to shift the eigenvalues. More specifically, it

is using a multiplier w to shift the eigenvalues., with w > 1.

Successive Over-Relaxation (SOR) An algorithm for solving a linear system such

as Vnew = L × Vol d + r by iterations, by shifting the eigenvalues. This can be
done via solving the equivalent problem of Vnew = w×[RHS of previous equation]+
(1 − w) × Vol d . For good choices of w, this can lead to much quicker conver-
gence.

Transients solutions to differential equations that are initially present but which
quickly die out (due to damping), leaving only the steady-state behavior.
Index

Acoustics, 63 Gauss-Seidel iteration, 56

Generalized eigenvalue problem, 16,
Boundary conditions 20
conservation of mass, 63 Ghost points, 1, 20
Dirichlet, 25 Grids
Neumann, 25 cell-center, 1, 20
PDEs, 37 cell-edge, 1
Cell-center grid, 1, 20 solving differential equations, 7
Cell-edge grid, 1 two-dimensional, 33
Centered difference formula, 4
Hanging chain, 19
CFL condition, 28
Hyperbolic equations, 37
for diffusion equation, 42
Conservation of energy, 71 Implicit methods, 43, 45
Conservation of mass, 63 Initial conditions
Courant condition, 28 wave equation, 25, 27
Crank-Nicolson algorithm, 43, 45 Instability, numerical, 28
Crank-Nicolson, gas dynamics, 66 Interpolation, 2
Iteration, 55
Damped transients, 13
Gauss-Seidel, 56
Derivatives, first and second, 3
Jacobi, 56
Differential equations on grids, 7
Differential equations via linear alge- Jacobi iteration, 56
bra, 8
Diffusion equation, 41 Korteweg-deVries equation, 81
CFL condition, 42
Laplace’s equation, 53
Dirichlet boundary conditions, 25, 26
Lax-Wendroff algorithm, 65
eig (SciPi command), 17 Linear algebra
Eigenvalue problem, 15 using to solve differential equa-
generalized, 16, 20 tions, 8
Eigenvectors, 17 Linear extrapolation, 4
Elliptic equations, 37
Matrix form for linear equations, 8
Explicit methods, 43
meshgrid, 33
Extrapolation, 2, 4

Forward difference formula, 3 Neumann boundary conditions, 25,

26
Gas dynamics, 63 Newton’s second law, 72

111
112 INDEX

Nonlinear Coupled PDE’s, 69

Nonlinear differential equations, 11
Numerical instability, 28

Parabolic equations, 37
Partial differential equations, types,
37
Particle in a box, 50
Poisson’s equation, 53
Potential barrier
Schrödinger equation, 52

Resonance, 15
Roundoff, 6

Schrödinger equation, 38
bound states, 21
potential barrier, 52
time-dependent, 49
Second derivative, 4
Shock wave, 66
Solitons, 81
SOR, 58
Spatial grids, 1
Staggered leapfrog
wave equation, 25
Steady state, 14
Successive over-relaxation (SOR), 53,
58

Taylor expansion, 5
Thermal diffusion, 71
Two-dimensional grids, 33
Two-dimensional wave equation, 33

Wave equation, 13
boundary conditions, 25
initial conditions, 25
two dimensions, 33
via staggered leapfrog, 25