Fourier Transforms

In this chapter I provide a summary of various transform pairs. Notation varies in the literature and I present
here a self-consistent treatment.
Sine Transform
Given a function u(x) on the interval [0, ], the sine transform and its inverse are given by:
Sine Transform: a
m
=
2

_

0
u(x) sin(mx/)dx (1)
Inverse Sine Transform: u(x) =

m=1
a
m
sin(mx/) (2)
To prove that these are indeed inverses of one another, one uses the orthogonality relation for sine functions.
_

0
sin(mx/) sin(m

x/)dx =

2

mm
(3)
which is easy to prove by direct integration. One can also use the completeness relation:

m=1
sin(mx/) sin(mx

/) =

2
(x x

) (4)
which is not easy to prove.
Discrete Sine Transform
There is a discrete form of the sine transformation, the discrete sine transform (DST). In this case we are
given a set of value U
j
for j = 1, . . . , J 1 generally thought of as a discrete sampling of a function
u(x) on [0, ], i.e. U
j
= u(jh), which h = /J. For the DST the function is sampled only on the interior
j = 1, 2, . . . , J 1. Then the discrete sine transform (DST) and its inverse are given by:
33
34
DST: a
m
=
2
J
J1

j=1
U
j
sin(mj/J), m = 1, 2, . . . , J 1 (5)
Inverse DST: U
j
=
J1

m=1
a
m
sin(mj/J), j = 1, 2, . . . , J 1 (6)
One may think of (5) as arising from (1) by replacing the integral by a discrete (trapezoid) approximation.
Note that if one included the cases m = 0 and m = J in (5) one would nd that a
0
= a
J
= 0. Hence
there is not reason to include these. Similarly, U
0
and U
J
would necessarily be zero by (6). Hence it is not
possible to recover non-zero values of U
0
and U
J
via the inverse DST and these are excluded.
The proof that the above pairs are inverses is easy given the orthogonality relation:
J1

j=1
sin(
mj
J
) sin(
m

j
J
) =
J
2

mm
(7)
Note that m and j play exactly the same role in sin(
mj
J
) so that the discrete analog of the completeness
relation (4) holds as a direct consequence of (7) by relabelling j and m:
J1

m=1
sin(
mj
J
) sin(
mj

J
) =
J
2

jj
(8)
Equations (7) and (8) are the same thing.
Cosine Transform
Given a function u(x) on the interval [0, ], the cosine transform and its inverse are given by:
Cosine Transform: a
m
=
2

_

0
u(x) cos(mx/)dx (9)
Inverse Cosine Transform: u(x) =
a
0
2
+

m=1
a
m
cos(mx/) (10)
=

m=0
w
m
a
m
cos(mx/) (11)
where the weight function w
m
is given by:
w
j
=
_
1
2
if j = 0
1 if j > 0
(12)
The proof that these are indeed inverses uses the orthogonality relation for cosine functions.
_

0
cos(mx/) cos(m

x/)dx =

2

mm
(13)
which again is easy to prove by direct integration. These is also a difcult to prove completeness relation:

m=0
w
m
cos(mx/) cos(mx

/) =

2
(x x

) (14)
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
35
Discrete Cosine Transform
For the discrete cosine transform (DST) we consider a set of value U
j
for j = 0, . . . , J again generally
thought of as a discrete sampling of a function u(x) on [0, ]. This time however we sample the function at
the end points j = 0 and j = J. Then the discrete cosine transform (DST) and its inverse are given by:
DCT: a
m
=
2
J
J

j=0
w
j
U
j
cos(mj/J) (15)
Inverse DCT: U
j
=
J

m=0
w
m
a
m
cos(mj/J) (16)
(17)
where now the weight function is:
w
j
=
_
1
2
if j = 0, J,
1 if 0 < j < J
(18)
One may think of (15) as arising from (9) by replacing the integral by a trapezoid approximation. The actual
proof that the above pairs are inverses comes from the following identity.
J

j=0
w
j
w
m
cos(
m

j
J
) cos(
mj
J
) =
J
2

mm
(19)
Note that as the the DST, m and j play exactly the same role in this equation so that the discrete analog of
the completeness relation (14) holds as a direct consequence of (19) by relabelling j and m.
Fourier Transform
Given a function u(x) on the interval [0, ], the Fourier transform (FT) and its inverse are given by:
Fourier Transform: a
m
=
1

_

0
u(x)e
i2mx/
dx (20)
Inverse Fourier Transform: u(x) =

m=
a
m
e
i2mx/
(21)
In this case the function u(x) is often allowed to be complex, but we shall consider only the case of real
u(x).
One has the following orthogonality relation for complex exponentials:
_

0
e
i2mx/
e
i2m

x/
dx =
mm
(22)
which as usual is easy to prove by direct integration.
(Notice that here we have
_

0

m
(x)

m
(x)dx =
mm
, where
m
(x) = e
i2mx/
.)
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
36
One can also use the completeness relation:

m=
e
i2mx/
e
i2mx

/
= (x x

) (23)
which is not easy to prove.
In the case which u(x) is a real-valued function, the a
m
will still be complex, but the following symmetry
holds:
a
m
= a

m
(24)
where denotes complex conjugation. Proof:
a

m
=
_
1

_

0
u(x)e
i2mx/
dx
_
=
1

_

0
u

(x)e
i2mx/
dx =
1

_

0
u(x)e
i2(m)x/
dx = a
m
Using this fact, it is possible to re-write the FT pair in such a way that all quantities are real:
Inverse Fourier Transform: u(x) =
A
0
2
+

m=1
(A
m
cos(2mx/) + B
m
sin(2mx/))
with:
Fourier Transform: A
m
= 2a
r
m
=
2

_

0
u(x) cos(2mx/)dx
B
m
= 2a
i
m
=
2

_

0
u(x) sin(2mx/)dx
where a
r
m
and a
i
m
are the real and imaginary parts of a
m
: a
m
= a
r
m
+ ia
i
m
.
Hence the FT of a real-valued function is equivalent to taking both sine and cosine transforms on the interval
[0, ] but with argument 2mx/ rather than mx/.
The details are left for the reader, but the rst few lines of the derivation are:
u(x) = a
0
+

m=1
a
m
e
i2mx/
+

m=1
a
m
e
i2mx/
= a
0
+

m=1
a
m
e
i2mx/
+

m=1
a
m
e
i2mx/
= a
0
+

m=1
_
a
m
e
i2mx/
+ a

m
e
i2mx/
_
Discrete Fourier Transform
For the discrete Fourier transform (DFT) we again consider a set of value U
j
for j = 0, . . . , J thought of as
a discrete sampling of a function u(x) on [0, ] including end points j = 0 and j = J. In this case we are
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
37
going to consider several forms for the DFT. The rst is given by:
DFT: a
m
=
1
J
J1

j=0
U
j
e
i2mj/J
, m = J/2 + 1, J/2 + 2, . . . , J/2, (25)
Inverse DFT: U
j
=
J/2

m=J/2+1
a
m
e
i2mj/J
j = 0, 1, . . . , J 1, (26)
where we have assumed J is even. This is the form we shall consider most frequently in this course.
The proof that the above pairs are inverses comes from the orthogonality of the complex exponentials:
J1

j=0
e
i2mj/J
e
i2m

j/J
= J
mm
(27)
As with the other discrete transforms, this is also equivalent to the completeness relation in the continuous
case.
Other standard and non-standard forms of the DFT can be derived by noting that if (25) is used to dene a
m
for all integer m, then the a
m
are periodic with period J. Similarly, (26) gives periodic U
j
with period J.
a
m+kJ
= a
m
and U
j+kJ
= U
j
for all integer k. Proof:
a
m+kJ
=
1
J
J1

j=0
U
j
e
i2(m+kJ)j/J
=
1
J
J1

j=0
U
j
e
i2mj/J
e
i2kj
=
1
J
J1

j=0
U
j
e
i2mj/J
= a
m
Using this, the following is an equivalent form for the DFT and its inverse:
DFT: a
m
=
1
J
J

j=0
w
j
U
j
e
i2mj/J
, m = J/2, J/2 + 1, . . . , J/2, (28)
Inverse DFT: U
j
=
J/2

m=J/2
w
m
a
m
e
i2mj/J
j = 0, 1, . . . , J 1, (29)
where:
w
j
=
_
1
2
if j = 0, J,
1 if 0 < j < J
(30)
and
w
m
=
_
1
2
if m = J/2, J/2,
1 if J/2 < m < J/2
(31)
In effect, since U
0
= U
J
, U
0
in (25) can been replaced by (U
0
+ U
J
)/2. Similarly for (29) since a
J/2
=
a
J/2
.
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
38
This is a non-standard form for the DFT. I give it here because it is the form one might guess from the
continuous FT (20)-(21).
Probably the most common form for the DFT is given by using the periodicity in a
m
to shift the sum in (26)
and obtain:
DFT: a
m
=
1
J
J1

j=0
U
j
e
i2mj/J
, m = 0, 1, . . . , J 1, (32)
Inverse DFT: U
j
=
J1

m=0
a
m
e
i2mj/J
j = 0, 1, . . . , J 1, (33)
This form has the advantage that both sums are over the same range. However, it has the disadvantages that
(1) it is not a natural choice given the continuous FT, (2) more importantly, we will consider real U
j
and this
form does not make it as obvious that U
j
obtained by the inverse transformation will be real.
Most libraries that perform discrete Fourier transforms say they compute transforms of type (32)-(33). They
are all equivalent of course.
Finally, we consider the sine/cosine forms of the DFT. As in the continuous case: U
j
real implies a
m
= a

m
.
Then (26) gives:
U
j
=
J/2

m=J/2+1
a
m
e
i2mj/J
(34)
= a
0
+
J/21

m=1
a
m
e
i2mj/J
+
(J/21)

m=1
a
m
e
i2mj/J
+ a
J/2
e
i2(J/2)j/J
(35)
= a
0
+
J/21

m=1
_
a
m
e
i2mj/J
+ a

m
e
i2mj/J
_
+ a
J/2
e
ij
(36)
= a
0
+
J/21

m=1
_
2a
r
m
cos(2mj/J) + (2a
i
m
) sin(2mj/J)
_
+ a
J/2
cos(mj/(J/2)) (37)
=
A
0
2
+
J/21

m=1
(A
m
cos(2mj/J) + B
m
sin(2mj/J)) +
A
J/2
2
cos(mj/(J/2)) (38)
So
Inverse Fourier Transform: U
j
=
J/2

m=0
(w
m
A
m
cos(2mj/J) + B
m
sin(2mj/J))
where
w
m
=
_
1
2
if m = 0, J/2,
1 if 0 < m < J/2
(39)
and A
m
and B
m
are given by:
Fourier Transform: A
m
= 2a
r
m
=
2
J
J

j=0
w
j
U
j
cos(2mj/J)
B
m
= 2a
i
m
=
2
J
J1

j=1
U
j
sin(2mj/J)
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
39
Final remark on counting. It is important to realize that the DFT is simply a change of bases. In the case
of complex U
j
it is a change of bases in C
J
. In the case of real U
j
, our case, it is equivalent to a change
of bases in R
J
. That the counting is correct for this real case (so called real-to-complex-transform) can be
seen by the as follows. Referring to (25), given J real values U
j
, j = 0, 1, . . . , J 1, we obtain J complex
amplitudes a
m
, m = J/2 + 1, J/2 + 2, . . . , J/2. However, there is a symmetry in the a
m
, namely
a
m
= a

m
. Hence the negative half of the spectrum: a
1
, a
2
, . . . , a
J/2+1
is known given the positive
half and thus is should not be counted. This leaves us with the J/2 + 1 values a
0
, a
1
, . . . , a
J/2
. However,
a
0
is necessarily real because a

0
= a
0
= a
0
. Similarly a
J/2
is necessarily real (try to show this). Hence
there are precisely J distinct real quantities in the a
m
m = J/2 + 1, J/2 + 2, . . . , J/2. These are
a
r
0
, a
r
1
, a
i
1
, . . . , a
r
J/21
, a
i
J/21
, a
r
J/2
.
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
40
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
Two and More Space Dimensions
The heat equation in two space dimensions is:
u
t
=
2
u
where

2
u =
_

2
x
2
+

2
y
2
_
u, in Cartesian coordinates
=
_

2
r
2
+
1
r

r
+
1
r
2

2
_
u, in polar coordinates
The nite-difference approach to such an equation is, in principle, very similar to that used in one space
dimension:
Discretize u and spatial operator(s): u(x, y) U
jk
,
2
L
Discretize time:

t
FE, BE, CN, . . .
Solve resulting algebraic equations efciently
New difculties:
1. Geometry and Boundary conditions
2. Grids
3. Solving large linear systems of equations.
41
42
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley
Comments on Testing and Software Design
Suggestions for Testing
Test parts of your program that cannot possibly be wrong.
Test scaling.
Even without exact solutions one can test convergence rates. Use a very high resolution solution as
exact even if it may not be so.
If possible test linear and nonlinear part of the code separately.
Linear Operators.
Generally straightforward but here are some hints.
Build up your operators in pieces that can be tested independently if necessary.
It is usually easy to choose some test functions v and verify that, for example, (I + tL)V
converges to (1 +tL)v with the correct scaling.
Verify that all linear operators are linear.
Verify matrix inverses. Note that (I +tL) and (I tL)
1
are inverses of one another apart
from the sign of t. A forward Euler time step of size t followed by a backward Euler time
step with (t) should be the identity:
U = (I (t)L)
1
(I +tL)U
for any U. This will verify that the explicit and implicit parts of a code are at least consistent.
I strongly recommend this test.
Exact solutions.
Exact solutions are great, but even without one you can (usually) test in at least two ways.
Change the problem.
Given a nonlinear equation
u
t
= Lu +N(u),
choose a suitable test function u
T
for which one can analytically compute
g = (Lu
T
+N(u
T
)).
Add this constant forcing function to the right-hand-side and solve the PDE:
u
t
= Lu +N(u) + g.
u
T
will necessarily be an exact steady solution to this equation.
43
44
Use your pencil or your favorite algebraic manipulation package.
Choose a suitable function v and work out the effect of one time step on this function. Verify
that the code does as expected over one time step.
Often very trivial functions such as linear or constant functions can provide limited testing.
These are surprisingly good at nding mistakes in boundary conditions.
These are powerful, very common, yet largely un-publicized method of testing.
Benchmarks.
Test against benchmarks and other code if available.
Test parts of your program that cannot possibly be wrong.
Leave testing code in place.
You will need if more often than you think. Keep in a separate le if you wish.
For testing strive for complete clarity, not cleverness. Computational cost is not an issue for testing.
Produce a documented test suit which can be easily re-run frequently.
Software Issues
Some thought to consider in writing programs to solve PDEs.
Speed vs readability/maintainability.
Computational speed may be sacriced for readability, reliability, maintainability, and development
time. This is especially true during initial development. Start with good readability/maintainability.
Strive for efciency in algorithms, but do not necessarily strive for the absolute minimum number of
oating point operations per time step.
Later when computation speed becomes a real limit (particularly for production runs) you can im-
prove efciency and reduce computations to the minimum number of operations. Improvement can
usually made in a working code. The slow-but-sure code then can serve as a benchmark for the
improved code.
It is a good idea to comment known inefciencies while you write code.
Note, speed in not simply proportional to the number of computations performed. Memory access is
a major issue.
My rule of thumb: a factor of 2 in speed is not worth worrying about if it means a large sacrice in
any of the other factors: readability, reliability, maintainability.
Generality vs readability/maintainability.
Do not try solve all problems with one code (especially if you write object oriented code). In particular
do not try to solve problems you will never encounter.
Libraries and portability: when to use BLAS/LAPACK libraries and when to write vs self-contained
codes.
Using local libraries (libraries specic the hardware) has the advantage of speed. Furthermore using
the libraries in general can reduce errors and also reduce debugging there are no bugs in BLAS and
you need not look there for errors.
The advantage of self-contained codes written in standard C++ is that you are guaranteed portability
and porting is generally easier.
Numerical methods for PDES Copyright (C) 2002-2005 Dwight Barkley