Everything you Never Wanted to Know about Finite Difference Methods for Solving

Partial Differential Equations

c (2003) Elaine Wenderholm

These notes discuss using finite differences to solve physical problems that are ex-
pressed using partial differential equations (PDEs). If a PDE cannot be solved ana-
lytically, then we approximate the solution by solving it numerically. Finite difference
methods are just one way to approximate the solution of PDEs that are linear. The
solutions are motivated by examples; the PDEs are linear equations. Nonlinear PDEs
require different approximation techniques.

Just like the solution to an integral is a family of functions1 , the solution to a PDE
is also a family of functions.

To obtain a unique solution to the PDE we can constrain the solution by specifying
the boundary conditions of the region. and then use these (fixed) boundary values
to determine the values of the interior region. These constraints, usually defined as
functions, are called the Dirichtlet boundary conditions.

1 An elliptic partial-differential equation

The Poisson equation is an example of an elliptic partial-differential equation.

∂2u ∂2u
∇2 u ≡ (x, y) + (x, y) = f (x, y)
∂x2 ∂y 2
2
x2
It is called elliptic because of its similarity to the equation for an ellipse: a2
+ yb2 = r.

This type of PDE is used to describe time-independent physical problems, such as

the steady-state distribution of heat (as temperature) in a plane region.

We want to find the solution to this PDE: the function u(x, y), which denotes the
temperature at position (x, y) in the plane; x and y have physical units.

If the function f (x, y) 6= 0 at x, y, then this positions is a heat source (or sink). When
f (x, y) == 0 for all interior positions, it means that the interior has no heat sources
or sinks:
1
Recall taking integrals: you learned that you had to add a parameter c to the result, called the
constant of integration. This result represents a family of solutions. You make the solution unique
by substituting a value for c.

1
 ∂2u ∂2u
∇2 u ≡ (x, y) + (x, y) = 0
∂x2 ∂y 2
This is called Laplace’s equation, and commonly is denoted ∇2 = 0.

1.1 Physical Problem

Consider a thin metal square plate with dimensions 0.5 meters by 0.5 meters. Two ad-
jacent boundaries are held at a constant 0 deg C. The heat on the other two boundaries
increases linearly from 0 deg C to 100 deg C. We want to know what the temperature
is at each point when the temperature of the metal has reached steady-state,

2 A parabolic partial-differential equation

A (one-dimensional) parabolic PDE of the form

∂u ∂2u
(x, t) − α2 2 (x, t) = 0
∂t ∂x
and is used in the study of gas diffusion and often referred to as the diffusion equation.

It is called parabolic because of its similarity to the parabolic equation: y = ax2 (or,
y − ax2 = 0).

If we again use heat as an example, solution to this equation is the function u(x, t),
where u(x, t) is the temperature of x at time t.

2.1 Physical Problem

Consider a rod of length l that is perfectly insulated. We want to examine the flow
of heat along the rod over time.

The two ends of the rod are held at 0 deg C. We denote heat as temperature, and
seek the solution to the equation u(x, t): the temperature of x at time t > 0.

3 A hyperbolic partial-differential equation

The one-dimensional wave equation is an example of a hyperbolic PDE:

2
 ∂2u ∂2u
α2 (x, t) = (x, t)
∂x2 ∂t2

x2 y2
Again, note the similarity to the equation for a hyperbola: a2
− b2
= r.

3.1 Physical Problem

An elastic string is stretched between two horizontal supports that are separated by
length l. The PDE defines vertical displacement of the string at time t. (We are
assuming that when we “pluck” the string, it only vibrates in one direction.)

4 Finite Differences

We approximate the solution to u(x, y) (and also u(x, t)) by only looking at the values
at only particular points in the problem space. This is done by constructing a grid.
Each point gi,j on the grid (implemented as a 2-dimensional array) represents the
approximation of u(xi , yj ) (and also u(xi , tj )).

4.1 Elliptic PDEs and Central Differences Method

Using the example elliptic PDE problem:

We seek a solution to
∂2u ∂2u
∇2 u ≡ (x, y) + (x, y) = f (x, y)
∂x2 ∂y 2
where, a < x < b, c < y < d, and boundary values are defined by some function
g(x, y), over all ordered pairs (x, y) that denote the borders.

Choose integers n and m so that the interval [a, b] is partitioned into h equal parts, and
interval [c, d] is partitioned into k equal parts. h corresponds to a non-infinitessimal
δx), so the smaller the h(k) the better the approximation. (And similarly for k and
δy.)
xi = a + ih, 0 ≤ i ≤ n
yj = c + jk, 0 ≤ j ≤ m

(From a programming point of view, i is an index, and xi is a distance.)

3
The Taylor series expansion in the variable x about xi , with just the first term gives
the central-difference formula

∂2u u(xi+1 , yj ) − 2u(xi , yj ) + u(xi−1 , yj )

2
(xi , yj ) =
∂x h2
and for the variable y about yj ,

∂2u u(xi , yj+1 ) − 2u(xi , yj ) + u(xi , yj−1 )

2
(xi , yj ) =
∂y k2

Using these central difference formulas, the Poisson equation at the points (xi , yj )
becomes:

u(xi+1 , yj ) − 2u(xi , yj ) + u(xi−1 , yj ) u(xi , yj+1 ) − 2u(xi , yj ) + u(xi , yj−1 )

+ = f (xi , yj )
h2 k2

Using this Central Difference formula to calculate every inner grid point gi,j , (1 ≤ i ≤
n − 1 and 1 ≤ j ≤ m − 1) (which approximates the function u(xi , yj )) results in the
Central Difference Method:

"  #
2  2
h h
2 + 1 gi,j − (gi+1,j + gi−1,j ) − (gi,j+1 + gi,j−1 ) = −h2 f (xi , yj )
k k
where gi,j is computed for every inner grid cell 1 ≤ i ≤ n − 1, 1 ≤ j ≤ m − 1. Hence
gi,j approximates the function u(xi , yj ).

(In the Central Differences Figure, λ = h2 /k 2 ) If h = k = 1 and f (x, y) = 0 for all

internal grid cells, the formula for gi,j simplifies to:
gi,j+1 + gi,j−1 + gi+1,j + gi−1,j
gi,j =
4

The truncation error of the Central Difference Method is O(h2 + k 2 ).

4.2 Parabolic PDEs: Forward-Difference and Backward-Difference

Methods

The Taylor series expansion in t about tj , with just the first term is the formula

4
 ∂u u(xi , tj+1 ) − u(xi , tj )
(xi , tj ) =
∂t k

and in x about xi is

∂2u u(xi+1 , tj ) − 2u(xi , tj ) + u(xi−1 , tj )

2
(x, t) =
∂x h2

The resulting difference formula is

gi,j+1 − gi,j gi+1,j − 2gi,j + gi−1,j
− α2 =0
k h2
where gi,j approximates u(xi , tj ).

Solving for gi,j+1 gives

2α2 k
   
2 k
gi,j+1 = 1− gi,j + α (gi+1,j + gi−1,j )
h2 h2

This is known as the Forward Difference method. The initial conditions must be
defined for all x at t = 0. The Forward Difference method is conditionally stable only
k 1
when h and k are chosen so that α2 2 ≤
h 2
The Backward-Difference method is unconditionally stable. We use, instead, the
backward-difference formula for ∂u
∂t
:

∂u u(xi , tj ) − u(xi , tj−1 )

(xi , tj ) =
∂t k
By substitution, we get
gi,j − gi,j−1 gi+1,j − 2gi,j + gi+1,j
− α2 =0
k h2
and the difference equation becomes (λ = α2 (k/h2 ))

gi,j−1 = (1 + 2λ)gi+1,j − λgi+1,j − λgi,j−1

Unfortunately for us, it cannot be solved explicitly gith an iterative algorithm.

The moral of the story: check first for stability before using Forward Differences.

5
5 Hyperbolic PDEs and 5-star Difference Equa-
tion

The difference equation for

∂2u 2
2∂ u
(x, t) − α (x, t) = 0
∂t2 ∂x2

subject to the initial conditions

u(0, t) = u(l, t) = 0, t > 0 (boundary values held constant),

u(x, 0) = f (x), 0 ≤ x ≤ l (initial temperature),

∂u
(x, 0) = g(x), 0 ≤ x ≤ l (initial rate of change)
∂t

gi,j+1 − 2gi,j + gi,j−1 gi+1,j − 2gi,j + gi−1,j

2
− α2 =0
k h2

where λ = αk/h.

Solving for gi,j+1 gives us

gi,j+1 = 2(1 − λ2 )gi,j + λ2 (gi+1,j + gi−1,j ) − gi,j−1
which requires two previous time steps; and so the initial conditions requires values
not just for t = 0 but also for t = 1.

But the values at t = 1 is not a border value! To make a long story short, we use the
difference equation

λ2 λ2
gi,1 = (1 − λ2 )f (xi ) + f (xi+1 ) + f (xi−1 ) + kg(xi )
2 2
for gi,1 , 1 ≤ i ≤ m − 1 and hope for the best.

6 Solving Laplace’s equation

6.1 Jacobi Iteration

Jacobi iteration uses the values from the previous iteration l − 1 to compute the value
at the current iteration l.

6
(We assume h = k = 1 and ∇2 = 0.)

(gi,j )l = 0.25 (gi−1,j )l−1 + (gi+1,j )l−1 + (gi,j−1 )l−1 + (gi,j+1 )l−1
 

Jacobi iteration is an inherently data-parallel algorithm.

6.2 Gauss-Seidel Iteration

If Jacobi is executed sequentially over the grid, i.e.,

for (int i = 0; i < n; i++ )

for (int j = 0; j < m; j++)
g[i][j] = ... ;

we see that, in iteration l, at g[i][j] we have already computed the values for g[i-1][j]l
and g[i][j-1]l . The values computed in iteration l are “better” than those computed
in iteration l − 1. Gauss-Seidel uses the “best” values:

(GSi,j )l = (gi,j )l = 0.25 (gi−1,j )l + (gi,j−1 )l + (gi+1,j )l−1 + (gi,j+1 )l−1

 

Consequently Gauss-Seidel converges faster. And if it converges faster, it is a good

guess that there is less error propagation than with Jacobi.

Gauss-Seidel is an inherently sequential algorithm since it relies on the fact that

certain neighboring cells have values. This means we can’t use it as is for a parallel
implementation. Instead we need to use red-black tiling, which works great.

6.3 Gauss-Seidel with SOR Iteration

SOR (Successive Over-Relaxation) takes Gauss-Seidel one step further. The idea is
that since the value gi,j in iteration l is better than value gi,j in iteration l − 1, give
additional weight to the value in iteration l. It’s basically a weighted average. Notice,
l
also, that SOR actually uses gi,j ; just plain Gauss-Seidel doesn’t.

Taking (GSi,j )l from the previous equation,

(gi,j )l = ω(GSi,j )l + (1 − ω)(gi,j )l−1

7
0 < ω < 1 is the relaxation parameter. Typical values for ω range between 0.6 . . . 0.8.

(You will also see 1 < ω < 2, in which case the typical range is 1.25 . . . 1.80, and often
ω is 1.25. In this case you multiply the l − 1 value by (2 − ω) and divide the result
by 2).

(For the above central differences we assume h = k = 1 and ∇2 = 0.)

7 Central Differences Iterative Solution

1. • Choose TOLERANCE
• Choose n, m; compute h, k.
• Compute λ = (h2 /k 2 )
• Compute µ = 2(1 + λ)

2. Initialize the border points with the boundary values; initialize the inner points
to 0.

3. Perform either: Jacobi (see 6.1), Gauss-Seidel (see 6.2) or Gauss-Seidel with
SOR (see 6.3) iteration using the appropriate differences equation.

4. If this is a steady-state problem, iterate until, for a < i < b, c < j < d
l l−1
|(gi,j ) − (gi,j )| ≤ T OLERAN CE

Otherwise, iterate until t == tend .

8 Annotated Bibliography

Most of these are “old” books that I’ve accumulated over the years. One thing I’ve
found about old books is that they give warm-and-folksy explanations with lots of
pictures.

C. Ray Wylie and Louis CV. Barrett. Advanced Engineering Mathmatics. 5th Ed.
McGraw-Hill NY 1982. This text was used by the engineering students in GE’s Ad-
vanced Engineering Course.

L. Hopf. Introduction to the Differential Equations of Physics. Dover Publications.

1948. It’s a Dover book. What more needs to be said?

8
Hugh Hildreth Skilling. Fundamentals of Electric Waves. John Wiley & Sons. 1942.
Yup, an oldie that I got years ago at a library book sale. It’s one of my warm-and-fuzzy
books. It gives such a wonderful explanation of pdes, and you realize that basically
everything can be thought of in terms of electricity.

Probably any numerical analysis or numerical methods book that discusses finite
differences.

The following figures graphically depict the various difference methods. Multiply each
value by the formula inside the square (should a formula exist).

Central Differences

j+1 λ

+ / +
j 1 2(λ+1) 1

λ
j−1

i−1 i i+1
Figure 1: Central Differences Method for Two-Dimensional Elliptic PDE

9
 Forward Differences

j+1

+ +
+

j λ 1−2λ λ

i−1 i i+1
Figure 2: Forward Differences Method for One-Dimensional Parabolic PDE

10
 Higher−Order Differences

j+1

+ +
+ +

2
j λ 2(1−λ2 ) λ2

−1
j−1

i−1 i i+1
Figure 3: Higher-Order Differences Method for One-Dimensional Hyperbolic PDE

11