Chapter 5

Elements of Fourier Analysis

5.1 Fourier Series

In mathematics, we often represent general functions and other objects

using elementary building blocks; the number of these blocks can be finite
or infinite. For example, we write vectors in R 3 as a linear combinations
of three unit basis vectors i, j , K, and we use Taylor series to represent
analytic functions as an infinite linear combination of powers of (z — ZQ).
Similar to powers, sines and cosines can serve as elementary building
blocks of functions, and the Fourier series in sines and cosines provides
the corresponding representation for many functions. This representation
has both theoretical and practical benefits: we will see that, on a bounded
interval, the class of functions that can be represented by a Fourier series
is much larger than the class of functions that can be represented by a
Taylor series, which leads to numerous applications in signal processing,
communications, and other ares.
While the representation of certain functions using sines and cosines
was known to many eighteenth-century mathematicians, it was the French
mathematician J E A N - B A P T I S T E J O S E P H FOURIER (1768-1830) who, in the
early 1800s, developed a general method for solving partial differential equa-
tions using what we now know as Fourier series and Fourier transforms.
The practical importance of this ground-breaking method more than out-
weighed the lack of rigor on the part of Fourier. The first rigorous result
about the convergence of Fourier series appeared only in 1828 and was due
to Dirichlet.

241
242 Fourier Series

5.1.1 Fourier Coefficients

A t r i g o n o m e t r i c polynomial of degree N on the interval [—n, TT] is an ex-
pression PN(X) = a>o+Ylk=i(ak c o s kx+bk sin kx), where a,k, k = 0 , . . . , N,
and bk, k = 1 , . . . , N, are real numbers. Using complex numbers, we can
both simplify and generalize this expression as follows. Recall that, by
the Euler formula (4.3.14), page 214, cos kx = {elkx + e~lkx)/2, sin kx =
(elkx — e~lkx)/(2i). Then the expression for the trigonometric polynomial
becomes PN(X) = J2k=~N ckelkx, where c0 = a0, Ck = (flfe - ibk)/2 for
k > 0, and c/t = (a* + ibk)/2 for A; < 0. It is therefore natural to consider
trigonometric polynomials of the form
N
PN(x)= £ Ckeikx (5.1.1)
fe=-JV

with complex coefficients Cfc. Thus, P/v becomes a complex-valued function

of a real variable x.
The following o r t h o g o n a l i t y r e l a t i o n will be essential in many com-
putations to follow.
EXERCISE 5.1.1. c Verify that

£ eikxdx=r' k =
°
0, fc = ± l , ± 2 , . . . .
(5.1.2)

EXERCISE 5.1.2F (a) Verify that all values of PN{X) are real if and only if
C-k = Cfc for aU k; recall that Ck is the complex conjugate of Ck- Hint: we
already proved the "if" part; for the "only if" part, write P(x) = P(x), multiply
by etkx and integrate from — 7r to TT, using (5.1.2). (b) Verify that

I \PN(x)\2dx = 2n Y, \°k\2- (5-1.3)

•*—" k=-N

Hint: use (5.1.2) and the equality \PN(X)\2 = PN(X)PN(X); remember that the
complex conjugate of elkx is e~%kx.
In what follows, we show that the Fourier coefficients of the function /
are the coefncients of the trigonometric polynomial that is the best mean-
square approximation of / .
Let / be a reasonably good function defined on [—IT, n], for example,
bounded and (Riemann) integrable; it can take complex values. How should
 Fourier Coefficients 243

we choose the coefficients Ck of P/v so that PAT is the best approximation

of / ? Of course, with many different ways to measure the quality of the
approximation, we must specify what "best" means in our case, and relative
simplicity of equality (5.1.3) suggests that we use the mean-square error.
In other words, we want to find the numbers Ck so that the value of
Jin \f(x) ~ PN{X)\2<IX is as small as possible.
To find the corresponding numbers Ck, we write

r \f(x)-pN(X)\2dx= r \f(X)\2dx
J— 7T «/—7T

- J ( / ( z ) P ^ ) + PN{x)W)) dx + J* \PN(x)\2dx,

and define, for k = —N,... ,N, the numbers

<*(/) = ^ f f(x)e-ikxdx; (5.1.4)

271
" J-K

the numbers Ck(f) are called the Fourier c o e f f i c i e n t s of / . Then

algebraic manipulations show that

^ \f(x)-PN(x)\2dx = r \f(x)\2dx-2n JT \ck(f)f

J-Tt J-n k=-N
(5.1.5)
N
+ 27T J2 \Ck(f)-Ck\2.

EXERCISE 5.1.3. C Verify (5.1.5). Hint: you work, back and forth, with the
equality \z~w\ = \z\ +\w\ —zw — wz; keep in mind that, for complex numbers,
(z - w)2 ± \z - w\2.
An immediate consequence of (5.1.5) is that f* \f(x) — P^(x)\2dx is
minimal when the coefficients of Pjv are the Fourier coefficients of / : Ck =
Ck(f) for all k = —N,..., N, with Cfe(/) defined in (5.1.4). Accordingly, we
define a special trigonometric polynomial,

k=-N

and call it the iV-th p a r t i a l sum of the Fourier series for / . Using (5.1.5)
244 Fourier Series

with Cfc = Cfc(/) and PN = 5/,JV, we get

f | / ( i ) - SNJ(x)\2dx = f |/(x)| 2 dx - 2TT £ |c f e (/)| 2 . (5.1.6)

Since the left-hand side of this equality is always non-negative, we conclude

that, for every N, 2irY^k=-N M / ) l 2 < S-* \f{x)\2dx. Thus, if we start
with a bounded integrable function / and, for each k ~ 0, ± 1 , ± 2 , . . . , define
the numbers Ck{f) according to (5.1.4), then the series X^fcl-oo lcfc(/)l2
converges and, in fact,

E M/)l 2 ^ T- / I/Ml2***- (5-1-7)

Inequality (5.1.7) implies that the Fourier coefficients of / tend to zero:

lira | c f e ( / ) | = 0 . (5.1.8)

Inequality (5.1.7) is (a particular case of) B e s s e l ' s i n e q u a l i t y , named

after W. F. Bessel. Equality (5.1.8) is (a particular case of) the
Riemann-Lebesgue Theorem. As we saw earlier, the Ph.D. dissertation
of Riemann was a major contribution to complex analysis. His other dis-
sertation ( H a b i l i t a t i o n ) , was a major contribution to Fourier analysis.
Written in 1854, it also introduced what we now know as the Riemann
integral. The French mathematician HENRI L E O N LEBESQUE (1875-1941)
developed a generalization of the Riemann integral, which allowed him to
generalize Riemann's results about Fourier series. What we now know as
the Lebesgue integral was introduced by Lebesgue in 1902 in his Ph.D. dis-
sertation. Both (5.1.7) and (5.1.8) are extendable to the Lebesgue integral,
but this is beyond the scope of our discussion.
The next natural question is whether we can have the equality in (5.1.7),
and the answer, for sufficiently nice functions f, is positive: if / is bounded
and integrable on [—7r,7r], then

E M/)l 2 = ; r / l/WI2^- (5-L9)

This equality is known as P a r s e v a l ' s i d e n t i t y , after the French math-
ematician M A R C - A N T O I N E PARSEVAL DES CHENES (1755-1836), even
though his original result was not directly connected with Fourier series.
The proof of (5.1.9) is not at all trivial and is too technical to discuss
 Fourier Coefficients 245

here; we will take the result for granted. Below (see page 256), we discuss
the physical interpretation of Parseval's identity in the context of signal
processing. With this interpretation, (5.1.9) means conservation of energy.
Definition 5.1 The Fourier s e r i e s Sf of a bounded, Riemann inte-
g r a t e function / = f(x) on the interval [—IT, IT] is

Sf(x) = Y, ck(f)eikx, where ck(f) = — / f(x)e~ikxdx. (5.1.10)

At this point, we do not know whether Sf(x) = f(x); in fact, it is not

even clear in what sense the infinite sum in (5.1.10) is defined (keep in
mind that convergence of ^2kxL_00 |cfc(/)|2 does not imply convergence of
Sfcl-oo lcfc(/)D- What is certainly true is that, because of the Parseval
identity, the Fourier series converges to / i n the mean square: because
of (5.1.6) and (5.1.9),

lim / \f(x)-SfN(x)\2dx = 0.
N

Unfortunately, this mean-square convergence has nothing to do with

the convergence of Sf}N{x) to f(x) for individual values of x. Indeed,
one can construct a sequence of functions gi, Qi,--- on (0,1) so that
limw-xjo J0 \gN{x)\2dx = 0 but lim./v->oo 9N(^) does not exists for all
x e (0,1), see Problem 6.1, page 437.
EXERCISE 5.1.4? Show that the Fourier series of f is unique: if ak, k > 1,
is a collection of complex numbers with the property lim;v^oo /_ \f{x) —
Efc=-iv akelkx\2dx = 0, then ak = ck(f) for all k. Hint: use (5.1.5).
For some functions, the equality S/(x) = f{x) is easy to prove. Indeed,
if / is a trigonometric polynomial of degree N, then 5/,jv(a;) = f(x) for all
x, because in this case ck(f) = 0 for \k\ > TV. Another example is discussed
in the following exercise.
EXERCISE 5.1.5. B Let F = F(z) be a function, analytic in an annulus
G = {z : ri < \z\ < r2), where 0 < rj. < 1, T2 > 1, and define the function
f(cp) = F(eltp). Show that if F{z) = J2kX=-ooCkZ'C *s ^ e L a u r e n " t s e r i e s
expansion of F in G, then ck = ck(f) for all k and Sf(ip) = X^fcL-oo ck&lkip
for all ip € [0, 2TT], and so in this case we have f(ip) = Sf(ip). Hint: use
formula (4-4-3) on page 217 for the coefficients of the Laurent series, with «o = 0
and p = 1. After changing the variable of integration £ = ellp in (4-4-3), you will
get 5.1.4-
246 Fourier Series

From the practical point of view, neither trigonometric polynomials nor

analytic functions are very interesting to expand in a Fourier series, but
fortunately the class of functions that can be represented by a Fourier
series is much larger. We will see in the next section that, if one can draw
the graph of the function / = f(x), then S/(x) is well-defined for all x and
Sf(x) = f(x) for all x where / is continuous. In other words, the equality
Sf(x) = f(x) holds for most reasonable functions / . Note that the graphs
that can be drawn must be smooth at all but finitely many points, simply
because of the finite thickness of the line.
We conclude this section with an alternative form of the Fourier series
for the REAL-VALUED FUNCTIONS. For such functions, it is natural to write
oo

Sf(x) = ao + /J( a fc cos kx + bk sin kx). (5.1.11)

EXERCISE 5.1.6P (a) Verify that, in (5.1.11), we have

i r i r
oo = — / f(x) dx, afc = — / f(x) cos kxdx, k = 1,2,...;
2 n U W
, r - (5.1-12)
bk = — / f(x) sin kx dx, k = 1,2,

(b) Verify that, for real-valued functions, Parseval's identity (5.1.9) becomes

2a20 + J2(4 + bl) = - f(x)dx, (5.1.13)

fc=i *"J-*

with ak, bk from (5.1.12).

EXERCISE 5.1.7. c Verify the following o r t h o g o n a l i t y r e l a t i o n s for the
trigonometric functions, with integer m,n>l:

f A r • . , JTT, m = n,
/ cos nx cos mx ax = I sin nx sin mx dx = <

cos nx sin mxdx = 0.

J
J —1
Hint: you have at least three options: (1) trigonometric identities; (2) sines
and cosines as complex exponentials; (3) equality emxe-mx = ( c o s m a ; _|_
i sin mx) (cos nx — i sin nx) followed by (5.1.2) on page 242. Treat the cases m = n
and m ^ n separately.
 Convergence 247

5.1.2 Point-wise and Uniform, Convergence

Using formulas (5.1.4) or (5.1.12), we can compute the Fourier series for
specific functions / denned on [—7r, n]. Still, the computations will be much
more efficient, and also make much more sense, once we understand when,
and in what sense, Sf = f.
The aim of the present section is to develop the theory that justifies
all the computations we will perform in the following section, and this
development requires more than the usual number of theorems and proofs;
those who do not like that can move on to the next section: after all, people
have been computing the Fourier series well before the necessary theory was
developed. On the other hand, those who want a more detailed account
of the uniform convergence should consult a book such as Principles of
Mathematical Analysis by W. Rudin, 1976, or Introduction to Analysis by
A. Mattuck, 1998.
But why should one study the point-wise convergence of the Fourier
series? Indeed, Fourier himself believed that the equality Sf (x) = f(x)
was always true, although he did not provide any proofs. It was only in
1828 that Dirichlet published the first rigorous result about the point-wise
convergence of Fourier series (something along the lines of Theorem 5.1.5
below), which put certain restrictions on the function / . After this result,
Dirichlet and many others started to believe that the equality Sf (x) = f(x)
should hold for all continuous functions / ; then, in 1873, PAUL DAVID G U S -
TAV Du BoiS-REYMOND (1831-1889) constructed a continuous function for
which the Fourier series diverges at one point, causing many to suspect that
there could be continuous functions with an everywhere divergent Fourier
series. In the early 1920s, A. N. Kolmogorov, who was not yet 20 year old,
and who had just changed his major from history to mathematics, con-
structed a function for which the Fourier series diverges everywhere, but the
function is not continuous. The question was finally settled in 1964, when
the Swedish mathematician LENNART CARLESON (b. 1928) proved that
the Fourier series of a continuous function converges almost everywhere. It
is a mathematically precise statement; see his paper On convergence and
growth of partial sums of Fourier series in Acta Mathematica, Vol. 116
(1966), pages 137-157. The proof of this result is considered by many to
be one of the hardest in all of analysis.
To summarize, the question of point-wise convergence of Fourier series
is very nontrivial, and our discussion below provides only the most basic
ideas. We will use the complex version of the Fourier series (5.1.10).
248 Fourier Series

For fixed x, the Fourier series Sf(x) is a numerical series; see Section
4.3.1, page 206. Note that \ck(f)elkx\ = \ck(f)\, so the Fourier series will
converge absolutely for all x if ^fcl-oo lcfc(/)l < °°- ^n fact> t n e series in
this case also converges uniformly, which is a special type of convergence
we define next.
Definition 5.2 A sequence of functions f\, fa, • •. on an interval I of the
real line converges uniformly to the function / if, for every e > 0, there
exists an m > 1 such that, for all n > m and all x in the interval I, we
have \fn(x) - f(x)\ < e.
A series ]CfcLi fk(x) converges uniformly if the sequence of partial sums
Sfc=i fk(x) converges uniformly; for the series J2k^=-oo fk(x)> we consider
partial sums of the form ^22=-n fk(x)-
The same definition applies to functions of a complex variable, defined
in a complex domain instead of a real interval.
Recall that a sequence of functions f\, fa,... converges to / at every
point of the interval if for every e > 0, and for every x in the interval, there
exists an m > 1 such that, for all n > m we have \fn(x) — f{x)\ < e. The
difference from uniform convergence is therefore in the possible dependence
of m on x. In other words, a uniformly convergent sequence converges
point-wise, but a point-wise convergent sequence does not need to converge
uniformly. FOR EXAMPLE, the sequence fn(x) = x/n converges uniformly
to zero on every bounded interval, while the sequence fn(x) = x™ converges
to zero on [0,1) point-wise, but not uniformly; the same sequence fn(x) =
xn does converge uniformly on [0, a] for every a < 1.

EXERCISE 5.1.8. B Verify that if a Fourier series converges uniformly on

[—7T, 7r], then the series converges uniformly on K. Hint: use periodicity.
One reason for considering uniform convergence is that the limit of a
uniformly convergent sequence inherits many properties of the individual
functions in the sequence.

Theorem 5.1.1 Assume that a sequence of functions / i , / 2 , ••• on a

closed bounded interval I of the real line converges uniformly to some func-
tion f.

(1) If each fn is continuous on I, then f is continuous on I and

limn^oo Jr fn(x)dx = Jj f(x)dx.
(2) If each fn is differentiable on I and each fn is continuous on I, and
the sequence of derivatives f[, f2,... converges uniformly on I to some
 Convergence 249

function g, then f is also differentiable on I and f = g.

We essentially proved this theorem in the special case of power series,

see Theorem 4.3.4, page 210; an interested reader can easily adjust the
arguments for the general case. The key in the proof is that, by taking n
sufficiently large, we can make the difference \fn(x) — f{x)\ as small as we
want for all x £ I at once.

EXERCISE 5 . 1 . 9 . S(a) State the analog of the above theorem for a series of
functions, (b) Show that if the sequence / i , / 2 , . . . of continuous functions
converges uniformly to f on the closed bounded interval [a,b], then, for all
sufficiently large n, the graph of fn resembles the graph of f. In other words,
show that lim n _ 0 O m a x ^ e ^ ] \fn{x) - f(x)\ = 0. Hint: let max^,;,] \fn{x) -
f(x)\ = \fn(xn) - f(xn)\.
The main test for uniform convergence for a series of functions is
known as W e i e r s t r a s s ' s M-test, in honor of the German mathemati-
cian KARL T H E O D O R WILHELM WEIERSTRASS (1815-1897), who was one
of the founders of modern analysis (both real and complex).

Theorem 5.1.2 WEIERSTRASS'S M-TEST. // \fn(x)\ < Mn for all x in

closed bounded interval I, and if the series of numbers Yln°=i Mn converges,
then the series X^^=i fn{%) converges uniformly in I.
EXERCISE 5.1.10. c (a) Show that if \fn(x)\ < an for all x in the interval,
and the sequence a\,a,2, • • • converges to zero, then the sequence / i , /2, • • •
converges to zero uniformly, (b) By considering the sequence Yl'kLnfki
n — 1,2,..., prove the Weierstrass M-test. (c) Convince yourself that if
Sfe=-oo lcfc(/)l < °°> then the Fourier series converges uniformly on K and
Sf is a continuous function on the whole real line, (d) Consider the power
series Yl'kLo ak(z ~~ zo)k with the radius of convergence R > 0. By taking
Mn = \an\rn, r < R, convince yourself that the power series converges
uniformly inside the closed disk {z : \z — ZQ\ < r}.

We can now establish a rather general result about the convergence of

Fourier series.
T h e o r e m 5.1.3 Assume that the function f is continuous on [—TT,TT],
/( 7 r ) = /(~"")) and the Fourier series Sf converges uniformly on [—IT,TT}.
Then Sf(x) = f(x) for all x G [—TT,TT].

Proof. Let us pass to the limit iV —» oo in equality (5.1.6) on page 244. By

assumption, the sequence {SftN, N > 1} converges uniformly to Sf. Then,
250 Fourier Series

by Theorem 5.1.1 we have

lim /"" |/(z) - SNJ(x)\2dx = P \f(x) - Sf(x)\2dx. (5.1.15)

On the other hand, (5.1.6) and Parseval's identity (5.1.9) imply that
the left-hand side of (5.1.15) is equal to zero. As a result, f* \f(x) —
Sf(x)\2dx = 0, and since both / and Sf are continuous, we conclude that
f(x) = Sf(x) for all x e [-TT, n}. •

Note that the function Sf necessarily satisfies Sf(ir) = Sf(—n), because

elfcir
_ e-jfe7r for a jj fc j n £ a c t ) gj j s a function with period 2ir. As a result,
the statement of the theorem cannot hold without the assumption / ( u ) =
/(—7r). This assumption implies that the function / can be extended to
the whole real line so that the extension has period 2n and is continuous
everywhere.
Sometimes, it is possible to establish the equality Sf(x) = f(x) without
computing the coefficients Ck(f). We can easily prove the following result.
Theorem 5.1.4 Assume that f(iv) = /(—7r) and the function f is differ-
entiable on (—7r, n) and f is bounded and Riemann integrable. Then the
series Ylk^-oo lcfc(/)l converges and Sf(x) = f(x) for all x G [—7r,7r].
Proof. Denote by dk the Fourier coefficients of / ' . Then we integrate by
parts to find

dk f f'{x)e-ikxdx = f{x)eikxX7T +ik[ f{x)e~ikxdx.

J—tr x=—w J_7T
By assumption, the first term on the right-hand side of the last equality is
equal to zero, because f(—ir)e~nk = f(n)emk. As a result, \dk\ = \kck{f)\-
Then the Cauchy-Schwartz inequality (1.2.14) on page 17 implies

oo oo / oo \ oo

E M/)I = M/)I + E M*AI< E i^n E k~2 •

k=-oo fc=-oo \fe=-oo / fc=-oo
fc#0 \ fe^O /
Since ]£fcl-oo \dk\2 converges by the Bessel inequality (5.1.7), and

J2 fc-2 = 2^AT 2 <oo,

fc=-oo fc=l
fe^O
 Convergence 251

we conclude that ^2%L_ao |cfc(/)| < oo. This implies uniform convergence of
the Fourier series, and, together with Theorem 5.1.3, completes the proof.
D

To proceed, let us recall the definitions of the one-sided limits of a

function at a point: the limit from the right / ( a + ) , also denoted by
limx_>a+ f{x), is defined by

f(a+)= l i m / ( a + e).
e—>0,e>0

Similarly, the limit from the left f(a~), also denoted by lim^^Q- f(x), is
defined by

f(a~) = lim /(a-e).

e—>0,£>0

If one can physically draw the graph of a function, then the function must
be piece-wise smooth: it is impossible to draw infinitely many cusps or
discontinuities just because of the finite thickness of the line. In particular,
f{x+) and f(x~) must exist at every point. These are the functions we
encounter in every-day life. The functions can have jump-discontinuities,
that is, points where f(x+) ^ f(x~). The following result shows that we
can represent such functions using a Fourier series. I T IS ALSO THE MAIN
RESULT OF THIS SECTION.

T h e o r e m 5.1.5 Let f be a function with period 2n and assume that, on

every interval of length 2ir, the function has a continuous derivative except
at a finite number of points. Also assume that there exists a number M so
that \f'(x)\ < M for all x where f exists. Then

Sf(x) = ( / ( X ) ' i f f iS c o n t i n u o u s a t
* (5.1.16)
1 {f{x+) + f{x
))/2, if / is not continuous at x.
Moreover, if f is continuous everywhere, then the Fourier series converges
to f uniformly on R.

Note that conditions of the theorem imply that, on every bounded in-
terval, there exist finitely many numbers X\ < Xi < ... < xn such that
the function / is bounded and continuously differentiable on every interval
(xk,Xk+i); this, in particular, implies the existence of f(x+) and f(x~) for
every x. The proof of this theorem is rather long; an interested reader will
find the main steps in Problem 6.2 on page 437.
252 Fourier Series

The next result shows that many convergent Fourier series can be inte-
grated term-by-term even without uniform convergence, and the resulting
integrated sequence will always converge uniformly.

Theorem 5.1.6 Assume that the function f satisfies the conditions of

Theorem 5.1.5 and, in addition, f* f{x)dx = 0. Then the Fourier series
for f can be integrated term-by-term, and the result converges uniformly.
More precisely, if F(x) = f£ f{t)dt, then the Fourier series SF converges
to F uniformly on M. and

k= — oo k= — oo
fe^O fe^O

where Ck(f), k — ± 1 , ± 2 , . . . , are the Fourier coefficients of f.

Proof. We outline the main steps; the details are in the Exercise below.
Step 1. By assumption, co(/) = 0. Integrating by parts for k ^ 0, we
find
"• , / ft-ikx\ x=n rv p-ikx
/

Since both / and / ' are bounded, there exists a positive number A so that
|cfe(/)| < A/\k\ for all k = ± 1 , ± 2 , . . . .
Step 2. Recall that, for x < 0, f* f(t)dt = - f~x f(t)dt. Denote by
Cfc, k = 0, ± 1 , ± 2 , . . . the Fourier coefficients of F. Then

For k ^ 0, we integrate by parts and use / * f(x)dx = F(n) — F(—n) = 0

to conclude that Ck = —ick{f)/k. In particular, |Cfc| < A/k2, and so the
Fourier series of F converges uniformly by Weierstrass's M-test on page
249. Continuity of F implies SF(x) = £fci-oo C^ikx = F(x) for all x. In
particular, SF{0) = EfcL-oo ck = F{0) = 0, and so
OO OO / j>\

k=—oo A:=—oo

which completes the proof. •

 Convergence 253

EXERCISE 5.1.11. 5 Fill in the details in the above proof. In particular, (a)
Verify that condition f* f(x)dx = 0 implies co(f) = 0 and also implies that
the function F is periodic with period 2TT. (b) Verify that F is continuous
everywhere. Hint: for y>x, \F(y) - F(x)\ < f£ \f(t)\dt < A\x - y\. (c) Carry
out the calculations in Step 2 to conclude that Ck = —ick(f)/k. (d) Verify
that SF{X) is indeed the result of the term-by-term integration of Sf(x).
Hint: for k^O, / Q x eiktdt = (eikx - l)/(ik) = i(l - eikx)/k.

Let us emphasize that the construction of the Fourier series 5 / depends

only on the behavior of the function / on the interval (—7r,7r); it does not
even matter how / is denned at the end points of the interval. The Fourier
series S/, if it converges, is a 27r-periodic function and represents the 2-n-
periodic extension of / rather than / itself. As a result, the convergence
of the Fourier series depends on the behavior of this extension of / , and
not on the behavior of original / . In particular, even if / is continuous on
[—7r, 7r] and is infinitely differentiable on (—ir, re), the Fourier series will not
converge uniformly unless /(—n + ) — f(ir~), that is, unless the periodic
extension of / is continuous. In general, the more derivatives the periodic
extension of / has, the faster the Fourier series converges to / ; the rate of
convergence is determined by the rate at which the coefficients Ck (/) tend
to zero.

EXERCISE 5 . 1 . 1 2 . C
(a) Convince yourself that if the periodic extension of
f is not continuous, then the Fourier series cannot converge uniformly on
[—7r,7r]. Hint: if it did converge uniformly, the result would be a continuous
function, which it is not. (b) Assume that the periodic extension of f has N
continuous derivatives. Show that there exists a positive number A so that,
for all k, | c f c ( / ) | < A / ( | f c | + 1 ) ^ . Hint: integrate by parts N times and observe
that the derivatives of the periodic extension, if exist, are also periodic.

The non-uniform convergence of Fourier series can be visualized. Con-

sider the 27r-periodic function g = g{x) so that g(x) = x for |x| < TT,
<7(±7T) = 0; this one is known as the sawtooth function. The function g
is not continuous at the points n + 2nn and therefore the Fourier series
does not converge uniformly. At points 7r and — 7r, the Fourier series of g
converges to (g(7r+) +g(ir~))/2 = 0. Let us investigate what happens near
the point •K.
EXERCISE 5.1.13? (a) Verify that Sg(x) = 2 £ £ l 1 ( - l ) f c + l 2 i ^ . Hint: you
have to integrate by parts.
(b) Writing Sg,N{x) = 2Y^=l{-l)k+ls^^, use a computer algebra
254 Fourier Series

system to plot the graph of S9IN on the interval [—TT, TT] for N = 10,50,100.
(c) Verify that limjv—oo £g,./v (7r(l - 1/iV)) = 2 /^(sin x/x) dx.
(d) Show that 2 J* (sin x/x) dx > 1.177T. Hint: first try it analytically us-
ing the power series expansion of sin x/x at zero; if not successful, try a computer
algebra system.
In other words, even though limjv-KX) Sgtpf(TT — e) — TT — e for every
e £ (0, TT], the maximal value of S3IN(X) on the interval [0, IT], being achieved
at different points, does not converge to IT, the supremum of g on the interval
[0,7r]; by Exercise 5.1.9, this could not have happened had S3,N converged
uniformly.
In applications, we cannot sum infinitely many terms and approximate a
function / with the trigonometric polynomial SfiN(x) = ^2k^_N Ck(f)elkx.
If the periodic extension of the function / is not continuous, then the max-
imal value of the approximation error will not converge to zero. This effect
was first observed in 1898 for the sawtooth function g by A. A. Michelson,
of the Michelson-Morley experiment, who made a machine to recover the
function from the Fourier coefficients. Since it was J. W. Gibbs who pro-
vided the mathematical explanation in 1899, the effect is now known as the
Gibbs phenomenon.

EXERCISE 5.1.14^ Investigate the Gibbs phenomenon for the the 2n-periodic
square wave function h = h{x), defined for x G {—IT, TT] by

h
, if 0 < x < IT;
(x) = { ,
if - TT < X < 0.

5.1.3 Computing the Fourier Series

So far, we considered functions / defined on the interval [—TT, TT], and that is
sufficient from the theoretical point of view: if a function / = f(y) is defined
on the interval [— L, L], then the function F(x) = /(xL/w) is denned on the
interval [—TT, TT], and so f(y) = F(yTr/L), Sf(x) — SF{XTT/L). In practice,
it can be useful to have the explicit formulas for the Fourier coefficients and
the Fourier series of / :
oo
C
Sf(y) = Y, k(fykvy/L, where
fe=
-°° (5.1.17)
ik L ik L
ck{f) = ^f_ f(xLMe- * dx=±J f{y)e- ^ dx.
 Computations 255

Similarly, for real-valued functions / , we have C-k(f) = Cfc(/) and formulas

(5.1.11), (5.1.12), and (5.1.13) become
(X)

Sf(y) = a0 + 2__,iak cos(Ttky/L) + bk sm(ivky/L), where

fc=i

a =
o ^l f{y)dy,ak =- f(y) cos(nky/L)dy, fc = 1,2,...;
-L

bk =
1 fL
Z / ^ sm(-Kky/L) dy, k = 1,2,...;

2a20 + J2(at+bl) = ~ L
f{y)dy.
fc=l
L_, J-L
(5.1.18)
As in the case L = w, if / is piece-wise continuously differentiable then
Sf(y) = f(y) at the points y where the 2L-periodic extension of / is con-
tinuous; the convergence is uniform if this extension of / is continuous
everywhere on R.
EXERCISE 5.1.15. 5 Show that if the function f is extended with period 1L
on M., then all the integrals J_L in (5.1.18) can be replaced with integrals
fa_L for an arbitrary a S M. Hint: if a function g satisfies g(x) = g(x + 2L)
for all x € R, then f^_L g(x)dx — f_Lg(x)dx + f£ g(x)dx — /^~ g{x)dx;
IL+L 9(x)dx = fa_~LL g(y + 2L)dy = fa_~LL g{y)dy.
If the function / is even (f(x) = f(—x)), then bk = 0 for all k and the
Fourier series contains only cosines; if the function is odd (f(x) = —f(—x)),
then dfc = 0 for all k and the Fourier series contains only sines.
If the function is defined on the interval [0,L], then there are three
possibilities for constructing the Fourier series of / :

(1) Extend / on R with period L: f(x + nL) = f(x), x e (0, L); the Fourier
series will, in general, include cos (2itkx/L) and sin(27rfcx/L).
(2) Extend / in an odd way onto (—L,0): f(—x) — —f(x), x e (0,L);
then consider the 2L-periodic extension of the result; the Fourier series
will include only sm(Trkx/L) and is sometimes called the odd half-range
expansion.
(3) Extend / in an even way onto (—L,0): f{—x) = f(x), x e (0,L); then
consider the 2L-periodic extension of the result; the Fourier series will
256 Fourier Series

include only cos{-Kkx/L) and is sometimes called the even half-range

expansion.

If there is a possibility of choice, then the selection is dictated by Exer-

cise 5.1.12: the more derivatives the periodic extension has, the better the
convergence of the Fourier series. For a generic continuous function / , the
even extension is guaranteed continuous and is therefore the best choice.
EXERCISE 5.1.16. (a)c Verify that the even extension of a continuous
function is always continuous everywhere on M.. (b)B Give an example of
a function for which the odd extension is the best choice. (c)c Is there an
example of a function on [0, L] for which the L-periodic extension is the
best? Hint: drawing pictures helps in answering each of these questions.

Let us briefly discuss the CONNECTION BETWEEN FOURIER SERIES AND

SIGNAL PROCESSING. In signal processing, both the period 2L and the
argument of / are measured in units of time, while the function / = f(x)
and each of the Fourier coefficients ck(f) are measured in volts. Then each
u>k = kn/L becomes a frequency. The collection of all u>k corresponding
to Cfc(/) ^ 0 is called the (frequency) spectrum of the signal / , and it
is essential to allow both positive and negative values of the frequencies.
Since k is an integer, a periodic signal has a discrete spectrum. Once
the spectrum of the signal is available, signal processing is carried out by
removing or otherwise modifying the individual numbers Ck{f)eWkt. Since
J-L\ck(f)eiu,kt\2dt = 2L\ck(f)\2, the value 2L\ck{f)\2 is proportional to
the energy of the A;-th frequency component over one period of the signal;
the collection of numbers (w^, |c^(/)| 2 ) is called the power spectrum of
/ . The physical interpretation of the Parseval identity, J_L |/(:r)| 2 d:r =
2£Sfcl-oo l c fc(/)| 2 ' is therefore that the total energy of the signal over the
period is equal to the sum of the energies of all the spectral components.
From this point of view, Parseval's identity makes perfect sense without
any proofs: it just states the conservation of energy.

Sometimes, the Fourier series expansion of one function can be used

to derive the expansions of several other functions. Along the way, we
can also evaluate certain infinite sums using the Parseval identity and the
results about the point-wise convergence of Fourier series. F O R EXAMPLE,
consider the 27r-periodic function / defined by f(x) = 1, 0 < x < it and
f(x) = 0, —7r < x < 0. This is an example of a r e c t a n g u l a r wave; see
Figure 5.1.1(a).
We have c 0 (/) = 1/2 and, for k ^ 0, ck(f) = (27T)"1 / w e~ikxdx =
 Computations 257

/(*) g(x)
1
1 1 » ' 1 ^ I 1 1 1fa1 1 1,1 1 1 1
k '
— 77 •K 2TT 3TT X -3 -1 1 3 5 X

5/Cx) Sg{x)

•
14
— 7T
f1 1 a
•
, 1
2TT

(a)
•
1*

3TT
k
•
1
X
1
U lkl
-3 -1
1

1
• 'k '
3
(b)
'
5
' • ' t. <
X

Fig. 5.1.1 Two Rectangular Waves

(X - e-lk7r)/(2nki). Then, since eikv = e~lkw for all k,

1 pikir \ ikx „—ikx 1 v-^ 1 — C0s(7!"fc) . ., .
SfW = l + Y, nk 2i = o +fc=i
E ^r-
nk
1 sin fc
( *)-
fc=i
We note that 1 — cos(/c7r) = 0 for even k and 1 — cos(fc7r) = 2 for odd k,
and so
sin(2n + l)x
(5.1.19)
*/W-52 + 7T; ^E 2n + 1
71=0
By Theorem 5.1.5, we have Sf{x) = f(x) for x ^ n + 2-nk, and
5/(TT + 2TTA;) = 1/2.
To continue our EXAMPLE, consider another rectangular wave g = g(x)
with period 4, so that g(x) = 1 for — 1 < x < 1, g(x) = 0 for — 2 < x < - 1
and 1 < x < 2; see Figure 5.1.1(b). Then g is a result of horizontal shift
and dilation of / : shift x —> x + 7r/2 makes the function / even, and then
dilation x —-> 7ra;/2 makes the period equal to 4. In other words,

*(*)=/(f*+f)
(verify this!) and therefore
+1
ssM = s,(f* + !K + i£ST<«0 »*)'
n=0
As a bonus, we can evaluate the following two infinite sums:
J X 0 ( - l ) n / ( 2 n + 1) and £,T=o V ( 2 n + x?• Indeed, by Theorem 5.1.5,
258 Fourier Series

S9(0) = g(0) or 1 = 1/2 + (2/TT) £ ~ = 0 ( - l ) n / ( 2 n + 1), which means

(~l)n *
(5.1.20)
^(2n+l) 4'
v
n=0 '

By Parseval's identity (see (5.1.18)),

<0*+^5 (^irH/>><<*='•
which means
oo 1

(5.1.21)
2 n4-+ l ) 2
2^1 (Or, 8'
n=0

EXERCISE 5.1.17? Use the Fourier series expansion (5.1.19) of the function
f from Figure 5.1.1(a), to derive (5.1.20) and (5.1.21).
To complete the EXAMPLE, let us consider the 27r-periodic function h =
h(x) so that h(x) = \x\ for |a;| < n; see Figure 5.1.2(b).

u(x) = 2/(x) - 1 h(x) = f*u(t)dt

,*V 7T

1
1 i i i i N
— •K •IT 2n 3TT X
— TV 7T 2n 3TT X

(a) (b)
Fig. 5.1.2 Rectangle and Triangle

Verify that h(x) = Jo(2f(t) - l)dt, where / = f{x) from Figure 5.1.1(a)
is the original rectangular wave that started our example. By Theorem
5.1.6 about term-by-term integration of Fourier series (see page 252), we
conclude that

cos (2rc + l)a;

:
Sh(x)= / (2Sf(t)-l)dt = ^ f o I n 2 - - nE (2n + l ) 2 '
Jo " t ^ ^ +V ^o
Even if we did not compute the value of the sum £ ^ 1 0 l/(2n + l ) 2 before,
we know that

4>fi i i r , . w i r , TT
—
/ T^ TT? — ao = ;r~ / h(x)dx = — xdx = —,
 Computations 259

and so

*w-?-ii: cos(2n(2n+ +l ) l)x '

TT^;
2

It is true that using these computational tricks requires some experience.

It is also true that using these tricks is much more efficient than computing
the Fourier coefficients of the function h using formulas (5.1.12) on page
246.
EXERCISE 5.1.18? Using Parseval's identity and the formula for Sh, verify
that
7T4
£ (2n + l )
n
x
4
96
n=0 '
EXERCISE 5.1.19. c For the function f{x) = x, x G [0,1], write the three
possible Fourier series expansions (for the extension with period one, and
for the even and odd extensions with period two.) Which expansion con-
verges uniformly, and how can you figure it out without computing the
corresponding Fourier coefficients? Suggestion: use the results of the above
example as much as possible; in particular, try to limit your integration to com-
puting f1xsm(Tmx)dx only.

We finish this section by briefly discussing the APPLICATION OF

FOURIER SERIES TO THE STUDY OF ORDINARY DIFFERENTIAL EQUATIONS.
Consider the linear second-order equation y(t) + w%y{t) = 0, where u> > 0
is a real number and y(t) denotes the second derivative of y with respect
to t. For example, equation (4.1.8) on page 188, describing the current
in a series AC circuit with R = 0 and time-independent E has this form
and wo = 1/y/LC. Recall that the general solution of this equation is
y(t) — Aeiulot + Be~iuot, and the numbers A, B are determined by the
initial conditions; computations are often easier if we use complex expo-
nentials rather than sines and cosines. This equation models a harmonic
o s c i l l a t o r , that is, a system whose free motion (motion without out-
side forcing) is undamped (non-diminishing) oscillations; UJQ represents the
proper frequency of the system.
If an outside forcing / = f(t) is applied to the system, then the corre-
sponding equation becomes y(t) +Woy(t) = f(t). For periodic function / ,
we can find the general solution of this equation by expanding / in a Fourier
series and considering each term of the expansion separately. Indeed, if /
is periodic with period T = 2TT/W, then f(t) = YX=-oockU)^ikt,T =
260 Fourier Series

SfcL-oo ck(f)elkoJt- If ku> ^ (Jo for all k > 0, then, for every k, a particular
solution of y(t) + u0y(t) = exkujt has the form y(t) = Cke%kuJt, which, after
substitution, results in Cfc(—k2u>2 + u>2) = 1 or Cfc = 1/(WQ _ fc2w2). By
linearity of the equation, the general solution of y(t) + u>Qy(t) = f(t) is
therefore

y(t) = Aei«°t + Be-i"°t+ £ - ^ - - e ^ .

fe=—oo

Note that the series on the right-hand side converges uniformly even if the
Fourier series for / does not converge uniformly. Note also that y(t) stays
bounded for all t.
EXERCISE 5 . 1 . 2 0 . C
Find the solution of y(t) + 4y(t) = h(t), where h(t) is
the function shown on Figure 5.1.2, and y(0) = 2/(0) = 0.
If uio = ku> for some k, then we get a resonance and an unbounded
solution: the particular solution of y(t) + u)ay(t) = e%Uot has the form
y(t) = Ctelu'°t. In practical terms, the system can break down if subjected
to the external resonant forcing for sufficiently long time. As a result,
knowledge of the Fourier series expansion of the external forcing is crucial
to ensuring the stability of the system.
EXERCISE 5.1.21. 3 Find the solution of y(t) + 9y(t) = h(t), where h(t) is
the function shown on Figure 5.1.2(b), and y(0) = j)(0) = 0.
Sometimes we know for sure that a given external periodic force will be
acting on the system (example could be a train moving on a bridge and
jumping on the rail joints). There are two ways to ensure that the system
does not break down under this forcing:

• Adjust the proper frequency U>Q of the system so that wo is not a mul-
tiple of u>.
• Introduce damping (friction or other energy loss) into the system.

A damped system is described by the equation y(t) + a?y{t) +u>oy{t) =

f(t), where a > 0; the larger the a, the stronger the damping. In a series
AC circuit, the source of damping is the resistor R; see equation (4.1.8) on
page 4.1.8. For small a, the free motion of the system, that is, the general
solution of y(t) +a2y(t)+cJoy(t) = 0, is exponentially decaying oscillations;
for sufficiently large a, the free motion of the system has no oscillations at
all.
 From Sums to Integrals 261

EXERCISE 5.1.22.'4 Verify that all solutions of y(t) + a2y(t) + 9y{t) = h{t),
where h(t) is the function shown on Figure 5.1.2, remain bounded. How
does the bound depend on a? Hint: the computations are easier with complex
exponentials rather than with sines and cosines; see Section 4.1.3, page 187.
This completes our discussion of the Fourier series. We will revisit the
topic in the following chapter, where we use Fourier series to solve certain
partial differential equations.

5.2 Fourier Transform

5.2.1 From Sums to Integrals

The Fourier series represents a function that is periodic. Is there a similar
representation for functions that are not periodic? The following exercise
provides a clue.
EXERCISE 5.2.1.° For L > 1, consider a 2L-periodic function f = f(x) so
that f(x) - 1, \x\ < 1 and \f{x)\ = 0 for 1 < \x\ < L. Verify that

co(/) = 5I;cfc(/) = ~-± 7 f i , f e = ±l,±2,...

are the Fourier coefficients of f. For three different values of L (L =

5,10, 50,), use a computer algebra system to plot the numbers 2Lck(f) versus
irk/L, k = 0, ± 1 , . . . ± 5L (try not to connect the points on your plots, and
do not forget to multiply Ck{f) by 2L). Compare your plots with the graph
of the function g(x) = sinx/x, |a;| < 57r.
If you indeed do the above exercise (you do not even need a computer
to get the main idea), you will notice that, as L increases, the suitably
re-scaled graph of the Fourier coefficients approaches a continuous curve.
In other words, as we increase the period of the function, the discreteness
of the spectrum becomes less and less visible, and, in the limit, we get the
curve sinx/x, which could be called the continuous spectrum of the non-
periodic function f(x) — 1, |a;| < 1, f(x) = 0, \x\ > 1. In what follows, we
will try to formalize this observation.
Let / = f{x) be a function denned on (—00,00) and let / L ( ^ ) be the
restriction of f(x) to the finite interval [-L, L). We assume that, for every
L > 0, the function fi satisfies the conditions of Theorem 5.1.5 on page 251,
so that the Fourier series S/L(x) of /z,(:r) converges. We rewrite formula
262 Fourier Transform

(5.1.17) on page 254 as follows:

[i;fLfL®e~i™/Ldt)el
knx/L "^_.
/tW= £
the equality holds at all points x where the 2L-periodic extension of //, is
continuous. Let ujk = kTr/L and Aaife = ujk+i — ^fe = TT/L. Then

fdx)= f; (^ J^fL^e-^dtj eiUkX • ^k.

Let us define the function Cx = Cx(u), w € K, by

CLM = ^ | _ f(t)e~iutdt, (5.2.1)

and also the function FL(UJ,X) — C£,(w)elu;x. Then

!L{X)= ^2 FL(uk,x)Aujk= ^2 FL(ujk,x)Au;k + eNtL(x), (5.2.2)

fc=—oo k=—N

where, for every x and L, €N,L{X) —> 0 as TV —> oo. Note that, for each x,
the right-hand side of (5.2.2) is a Riemann sum, and it is natural to pass
to the limit and turn this sum into an integral. This passage to the limit
requires decreasing Awfc = n/L, that is, increasing L. As L —* oo, the
value of Cx(w) will converge to C(w) = (l/27r) J^° f(t)e~lwtdt, provided
IT 1/(0\dt exists. As a result, existence of this integral is the necessary
additional assumption about the function / . With this assumption in place,
we can now expect from (5.2.2) that, for each fixed TV and x,

N
lim fL(x)= lim Y^ FL(cjk,x)AuJk
z + lim eNiL{x)
L—»oo L—»oo —' L—>oo
k=-N
N

I N
F(u),x)du + Ejv(a;)

So far, the only questionable step in our argument is our assumption that
limL-»oo £N,L(X) exists. In fact, we need to go even further and assume that,
similar to EN,L(X), we have limjv->oo£;v(£) = 0 for all x. A more careful
analysis shows that we can ensure these properties of e, for example, with
an additional condition J^ \F(w,x)\dw < C with C independent of x.
 From Sums to Integrals 263

Then, allowing N —> oo, we conclude that f{x) = f_ F(cj,x)du> or

f{x) = J" (± f°° f(t)e-iutdt) e^'du. (5.2.3)

By analogy with Theorem 5.1.5 on page 251, we have the following result.

Theorem 5.2.1 Assume that the function f = f(t) is continuous and

has a continuous bounded derivative everywhere on M except for a finite
number of points. Also assume that J_ \f(t)\dx < oo. Then the function

if{t)= f{s)e iwsds ei tdw (524

/i (^ r ~ ) " - --
is defined for all t € R and

If(t) = { m if / is continuous at i ^ ^
[ ( / ( i + ) + /(* ))/2, if / is not continuous at t.

The proof is somewhat similar to the proof of Theorem 5.1.5, and we

omit it. Notice that we do not have to assume that J_ \F(uj)\dw < oo.
The function / / from (5.2.4) is called the Fourier i n t e g r a l of / . The
Fourier transform of / , denoted either by / or by -F[/], is

~ 1 r°°
f(w) = . F [ / ] M = - 4 = / f(t)e-^dt. (5.2.6)

The inverse Fourier transform of / , denoted either by / or by JF - 1 [/],

is

/ M = T-X{f\{w) = - = / fity^dt. (5.2.7)

The condition f™ \f(x)\dx < oo is sufficient for the existence of both /

and / , because \f{t)e~iujt\ = \f{t)eibJt\ = \f{t)\ for all u.
With the above definitions, 3r~1[J:'[f]}{t) = If(t); under the conditions
of Theorem 5.2.1, we have ^""M^I/IK*) = /(*)> o r
i r00 ~
f(t) = -j= J f(u)e^dw,

if / is continuous at t.
264 Fourier Transform

The reader should keep in mind that other scientific disciplines might
use other d e f i n i t i o n s of the Fourier transform. For example, the
following version of (5.2.4) is popular in engineering:

If{t)= f I T f{s)e-i2-K,JSd^\ei2^tdu. (5.2.8)

The following exercise provides an explanation.

EXERCISE 5.2.2. C Given real numbers A > 0 and B ^ 0, define
~ i r00

fA,B(y) = Jj j~iBxvdx.
(a) Show that, under the assumptions of Theorem 5.2.20, we have

m = ^rfjAMy)eiBxydy
if f is continuous at x. (b) Verify that (5.2.8) corresponds to A = 1,
B = 2TT.

Another definition is related to the class of functions for which the

Fourier transform is denned. We say that / is a b s o l u t e l y i n t e g r a b l e
on K, and write / G L\(R), if f^° \f(x)\dx < oo. More generally, for a
real number p satisfying 1 < p < oo, the set LP(M) is the collection of all
functions / defined on K so that f_ \f(x)\pdx < oo. To summarize,

LP{R) = l f : I \f{x)\pdx<oo\, l<p<oo.

Strictly speaking, the integrals in these definitions are the Lebesgue inte-
grals, as are all the integrals in this and the following two sections, but
this should not stop the reader from proceeding, as the precise definition
of either the Riemann or the Lebesgue integral is not necessary for under-
standing the presentation.
The results of the following exercise are useful to keep in mind, even if
you do not do the exercise.
EXERCISE 5.2.3.^ (a) Give an example of a function f from LP(M) so that
lim| x |_ (00 |/(a:)| ¥" 0- Hint: let your function be zero everywhere except short
intervals Ik around the points Xk = k, k = 2, 3,4,..., where the function is equal
to 1 (draw a picture). For every p, the value of the integral f_ \f(x)\pdx is then
J2k>2 1-^*1) the sum of the lengths of the intervals. Choose \Ik\ so that the sum is
finite, (b) Give an example of a function from L\(R) that is not in L2OR).
 From Sums to Integrals 265

Hint: think \j\fx for 0 < x < 1. (c) Give an example of a function from
Z/2(K) that is not in Li(R). Hint: think 1/x for x > 1.
We now combine Theorem 5.2.1 with Theorem 5.1.5 on page 251
about the convergence of the Fourier series to prove the Nyquist-Shannon
sampling theorem. The theorem shows that a band-limited signal / can
be exactly recovered from its samples at equally spaces time moments; by
definition, the signal / is called band-limited if there exists an fi > 0 such
that the Fourier transform / of / satisfies f{u>) = 0 for \ui\ > fi.

Theorem 5.2.2 If the Fourier transform f = f(uj) of a function f = f(t)

exists and is equal to zero for \w\ > Q, and if both f and f are continuous
everywhere and have continuous derivatives everywhere except at finitely
many points, then

~ sin (il(t - kAt)) „ , 7T

f{hAt) where At (5 2 9)
/W = £ ndkAt) > = n- -'
fc=-oo
Proof. By Theorem 5.2.1 applied to / , with ir/fl — At,

f(t) = ~ [ fMe^dw, (5.2.10)

V27T J-Q

because, by assumption, |/(w)| = 0 for \u)\ > fi. By Theorem 5.1.5 applied
to/,

ku/At

(5.2.11)

where the second equality follows from (5.2.10) with t = —kAt. Sub-
stituting (5.2.11) in (5.2.10) and exchanging the order of summation and
integration, we get

/(*) = Yl I 4 ; / eifcu,At+iwtdw ) f{-kAt). (5.2.12)

m
fc=-oo \ J-n J
It remains to evaluate the integral in (5.2.12) and change the summation
index from k to — k. •
266 Fourier Transform

EXERCISE 5.2.4. c Verify that one can take At < TT/Q. and still have the
result of the theorem. Hint: if |/(w)| = 0 for |w| > fi and Q,\ = {ir/At) > fi,
then \f{uj)\ = 0 for \u\ > n : .
In 1927, the Swedish-born American scientist HARRY NYQUIST (1889-
1976) discovered that, to recover a continuous-time signal from equally-
spaced samples, the sampling frequency 2ir/At should be equal to twice
the bandwidth fl of the signal (that is, At — 27r/(2fi)). In 1948, the Amer-
ican scientist CLAUDE ELWOOD SHANNON (1916-2001) put communication
theory, including the result of Nyquist, on a firm mathematical basis.
Practical implementation of the sampling theorem is not straightfor-
ward, because no signal lasts infinitely long, while a signal that is zero
outside of a bounded interval is not band-limited; see Exercise 5.2.12(a) on
page 271. We discuss some of the related questions on page 277.
We conclude this section with a connection between the samples of a
function and the samples of its Fourier transform. Take a continuously
differentiable function / £ Li(M.) and a real number L > 0, and assume
that, for every x, the function g(x) = X)^L_oc, f(x + 2Ln) is defined and
is also continuously differentiable. This is true, for example, if there exists
an R > 0 so that f(x) = 0 for |x| > R, because in this case the sum
that defines g contains only finitely many non-zero terms. Note that g is
periodic with period 2L, and therefore, by (5.1.17) on page 254, g(x) =
Er=-ooCfc(<?)ei,rfcx/L, where

i rL °°
C f x
^9)=2L S ( + 2Ln)e-™k*/Ldx
~L n = —oo
1 oo „2L(n+l)
= *7 T, f{x)e-i'kx'Ldx (5.2.13)
21y /
n=-oo- 2in

= ^ J°° f(x)e-^Ldx = ^Lfak/L).

Since #(0) = Y^'kL-oo c&(ff)> we

conclude that

J2 f(2Ln) = ^ r £ /(Trfc/L). (5.2.14)

k=—oo

Equality (5.2.14) is called P o i s s o n ' s Summation Formula, after

S. D. Poisson.
 Properties 267

EXERCISE 5.2.5. A (a) Verify the computations in (5.2.13). (b) Taking

x
f(x) = e~ , t > 0, verify the ^-formula:
oo , 0 0
V ^ e -4tir 2 n — V e~ fc2/(4t) .
v /2tk^
n=-oo "" k=-oo

Then explain how to use such a formula for computing approximately the
value o / ^ ^ = _ O Q e~ak for various a > 0; the problem of approximating this
sum arises, for example, in statistics. Hint: how does the value a > 0 control
the rate of convergence of the sequence e~an , n > 1, to zero?

5.2.2 Properties of the Fourier Transform

As in our discussion of Fourier series, we will study the general properties
of the Fourier transform before doing any particular computations. To
begin, let us mention those properties of the Fourier transform that have
counterparts for the Fourier series; while the proofs are beyond the scope
of our discussion, the results are believable given what we know about the
Fourier series and the Fourier transform.
• The Riemann-Lebesgue theorem: If / G £i(I&), then / is continuous on
R a n d lim |/(w)| = 0.
|w|—*oo
• P a r s e v a l ' s i d e n t i t y : If / belongs to both Li(M) and Z/2(R), then
fe L2(R) and
OO /*OG

/
\f(t)\2dt= \f{w)\2du>. (5.2.15)
-oo J — oo
Equality (5.2.15) is also known as P l a n c h e r e l ' s Theorem, after the Swiss
mathematician MICHEL PLANCHEREL (1885-1967), who was the first to
establish it.
EXERCISE 5.2.6. A Verify that (5.2.15) implies
/•OO /-OO

/ f(x)g(x)dx= F[f}{w)F[g){u)dLJ (5.2.16)

for all functions f,g from Z/2(K). As usual, a is the complex conjugate of
the number a. Hint: Apply (5.2.15) with f + g instead of f to show that the real
parts of the two sides of (5.2.16) are the same; then use f + ig to establish the
equality of the imaginary parts; the key relations are \a + b\ = \a\ +\b\ +25R(ab);
\a + ib\2 = \a\2 + \b\2 — 2i SJ(o6), which you should verify too.
268 Fourier Transform

If / represents a time signal, then / is the s p e c t r a l d e n s i t y of / ,

the continuous analog of the discrete spectrum we considered for periodic
signals, see page 256. The values of |/(w)| 2 describe the distribution of
energy among the different frequencies of the signal; Parseval's identity
(5.2.15) represents conservation of energy.
The next property shows that the Fourier transform reduces differenti-
ation to multiplication: if / is continuous and differentiable so that both /
and / ' are in Li(R) and lim| t |_ 0 0 \f(t)\ = 0, then

•F[/'](w)=iw.F[/](a,). (5.2.17)

This follows after integration by parts:

1 r°° 1
V27T J-oo V27T

- ^ J°° f(t)e-^dt = ^.Hf]M.

EXERCISE 5.2.7. c Assume that f has two derivatives so that / , / ' , / " all
belong to Li(R), and also limi^on |/(i)| = limi^oo \f'{t)\ = 0. Show that

T[f"}{w) = -u, 2 .F[/](u;). (5.2.18)

Hint: apply (5.2.17).

There is another operation, called convolution, that the Fourier trans-
form reduces to multiplication. For two functions / , g from Li(R), we define
their convolution f * g so that
oo
f(x - y)g{y)dy. (5.2.19)
/
-00

EXERCISE 5.2.8.c Verify that (f * g)(x) = J^ g(x - y)f{y)dy.

It is easy to show that f * g belongs to Lj(IR):
00 00

J \(f*g)(x)\dx< J j \f(x-y)\\g(y)\dydx
-oo —00
00/00 \ / o o \ / oo \
/ ( I \f(x-y)\dx\ \g(y)\dy=l J \f{x)\dx\ I J \g{y)\dy\
-00 \ — 00
 Properties 269

Similar arguments show that if either / or g is bounded, then so is / * g.

Using an advanced trick called interpolation, one can then prove that, for /
from Lp(M) and g from L 9 (R), the convolution f*g is denned and belongs to
L r (K), where (1/p) + (l/<z) = 1 + (1/V)- The arguments, strictly speaking,
work only with the Lebesgue integral; for an example how things can go
bad with the Riemann integral, see Example C6 on p. 570 in the book
Fourier Analysis by T. W. Korner, 1988.
The Fourier transform of the convolution is, up to a constant factor, the
product of the Fourier transforms: if / , g belong to Li(R), then

f*g = V^Jg. (5.2.20)

Indeed,
I /»0O rOO
/ * g(w) = -f=\ / f(s)9(x - s)e-^dsdx
y Air J-oo J-oo
-l /"OO /*00

= - = / / f(s)g(y)e^y+^dyds
\JITT J-oo J-oo

\^f°° f(s)e-^ds^ (J™ g(y)e-^ydy^ = yfa f(u>) g(w).

The next three properties of the Fourier transform are verified by direct
computation.
LINEARITY O F THE FOURIER TRANSFORM. If f,g belong to Li(R) and
a, b are real numbers, then

F[af + bg] = aF[f\ + br\9]. (5.2.21)

T H E S H I F T FORMULA. If / belongs to L\{ a is a real number, and

g(t) = f{t~a), then

?M 7M- (5.2.22)

T H E DILATION FORMULA. If / belongs to £i( ), a > 0 is a real number,

and h(t) = f{at), then

Mw) = -f(v/a). (5.2.23)

EXERCISE 5.2.9. (a)c Verify the relations (5.2.21), (5.2.22), and (5.2.23).
A
(b) Can we allow complex values of a in (5.2.22) and (5.2.23)?
270 Fourier Transform

The next collection of equalities is as simple as it is useful:

^ - X [ / ] M = .F[/](-w) = -H/1M, ?[f] = I (5-2-24)

where f(x) = f(—x). All equalities in (5.2.24) follow directly from the
definitions (5.2.6) and (5.2.7) after changing the sign of the variable of
integration.
EXERCISE 5.2.10? Verify all equalities in (5.2.24).
Together with the relation ^ r _ 1 [^ r [/]] = / , equalities (5.2.24) have two
practical benefits:
• Two Fourier transforms for the price of one:

if h(u) = f(u), then h(u) = / ( - w ) (5.2.25)

(verify this!) Once the definition of the Fourier transform is modified ac-
cordingly, the second equality will hold even if h is not absolutely integrable.
• Two properties for the price of one: every property of the Fourier trans-
form has a dual property after the inverse Fourier transform is applied to
both sides.
In particular, the dual property of (5.2.20) is

fS=^f*9, (5-2.26)

assuming fg e L\(M). Indeed, by taking the inverse Fourier transform on

both sides of (5.2.20), we get / * g = y/2nJ:~'l[f'g]. Now replace / , g with
/ , <7, respectively, and use (5.2.24) to get

(f*g)(x) = y/2^T-l[fg](x) = y/toF[fg]{-x) = y/toF[fg](x).

Similarly, the dual property of (5.2.17) is

£ / ( w ) = -iF\g](u>), where g(x) = xf(x). (5.2.27)

EXERCISE 5.2.11. Assuming

thatjToo \xf(x)\dx < oo, verify (5.2.18) in
two different ways: (i) By differentiating (5.2.6) and moving the derivative
under the integral sign; (ii) By applying the inverse Fourier transform to
both sides of (5.2.17).
 Computations 271

5.2.3 Computing the Fourier Transform

As the first EXAMPLE, let us compute the Fourier transform of the
rectangular pulse

n L (t) = ft '*'<*' (5.2.28)

10, \t\>L,
where L > 0 is a real number; the definition of n ^ for \t\ = L is not
important. We have

2 sin u>L
y/2n J-L V2^ ™ V n7r u
By Theorem 5.2.1,

w
K J-oo

Notice that the integral on the right-hand side exists even though / does not
belong to Li(R), and, for t = ±L, the value of the integral is 1/2. Setting
t = 0, we find / dw = IT for all L > 0. Using Parseval's identity
J-oo "
OO / • r \ 2

/ I J dw = LTT.
EXERCISE 5.2.12. (a,)B Explain why a signal that is zero outside of a
bounded time interval cannot be band-limited. Hint: if f(t) = /(i)Ili(t) for
some L > 0, then f{u>) is connected to the convolution of f(u>) and sla^L. Can
this convolution be zero outside of a bounded interval in ui ? (b)c Verify that if
f(x) = e _ ' x ', then /(w) = \/2/(\/7r(l + u>2)). Hint: this is straightforward
integration and complex number manipulation; do not try to find any connection
with the rectangular pulse. (c)c Use (5.2.27) to conclude that the Fourier
transform ofte~W is 2iy/2/n w/(l + w 2 ) 2 .
For the next EXAMPLE, consider the function f(t) = e~* I2. Turns out,
the Fourier transform does not change this function:

/(w) = e-" 2 / 2 . (5.2.29)

The rigorous computation is somewhat long and uses complex integration;

see Problem 5.7 on page 436. Here, we will give a more intuitive explana-
272 Fourier Transform

tion. The key is the relation

>o
oo
x2 2
e- ' dx = V2^.
/ -oo

Indeed, writing I = J^° e x l2dx, we have I2 = j ^ J^ e (x +v ^2dxdy,

and, after changing to polar coordinates, I2 = 2ir /Q°° e~r ^2rdr = 2n. As
a result,

f
J —(
e-(x-a) /2dx = ^ (5.2.30)

for every real a. On the other hand,

/(W) = -±= / e~^+2i^'2dt,

\Z2TT J-OO

and t2 + 2ioJt = (t + iuj)2 + w2. Equality (5.2.29) follows immediately, if

we assume that (5.2.30) holds for all complex numbers a (we should be
more careful here because we are actually computing the integral in the
complex plane; see Problem 5.7 for details). In any case, the result (5.2.29)
is certainly worth remembering.
EXERCISE 5.2.13. c Verify that if f(x) = e~ax2, a > 0, then f(w) =
(2a)- / e" /( 4 a ). Hint: use (5.2.23).
1 2 2

To conclude this section, let us discuss the real, as opposed to complex,

form of the Fourier i n t e g r a l . On the right-hand side of (5.2.4), page
263, we rename one of the variables of integration from t to s and use the
Euler formula for the complex exponential:
-1 /*00 /*00

If{t)= {s)e iU,{S t)dsdW

2^joc]J ~ ~
~ o~ / / f(s)l cos (w(s — t)) —ism(u)(s — t)))dsdu}.

If /(£) is real, then the imaginary part of the last expression must be zero.
Therefore, If(t) — (l/27r) f_oo J_oo f(s) cosu>(s — t) dsdu. Since cosw(s — t)
is an even function of u>,
1 />oo /«oo

•TK*) = —
/ / f (s) cos UJ(S — t) dsdio.
7T Jo J-oo
 Computations 273

Using the identity cos (ui(s-t)) = cos u>t cosws + sin wt sin us, we get the
real form of the Fourier integral:
,00 / 1 ,00 x

f(t) = I— / / ( s ) cos ws ds 1 cos wt dio

JO V71" J-00 / (KO-i-W
+ / I— / /(s) sin us ds I sin wt du.
JO \7T i_oo /
For a function / on the half-line (0,oo), define the Fourier cosine
transform

•^c[/](w) = A M = y f / /(*) c o s us ds> (5-2-32)

and the Fourier sine transform

•F.I/1M = A M = \jl [ /(s)sin w s ds

- (5.2.33)
If / is an even function, then f(s) sinws is an odd function of s and the
second term in (5.2.31) is zero. Hence, for even / ,

I t f°° I~2 -
f( ) = / \ -fc{u)coswtduj = Fc{fc}(t). (5.2.34)
Jo V 7T
Thus, T~x = Tc-, and, for functions defined on (0,oo), the Fourier cosine
transform represents the even extension of the function / to M.
If f(s) is an odd function, then f(t) = J0°° y/2/n fs(u>)sm ut du =
Ts[fs](t)- Thus, T~x = Ts, and, for functions defined on (0,00), the Fourier
sine transform represents the odd extension of the function / to R. Other
conditions being equal, the choice between the even and odd extensions is
determined by the smoothness properties of the result: the smoother the
extension, the better.
Similar to (5.2.17), we have the following rules for transforms of deriva-
tives; the reader is encouraged to verify these rules using integration by
/"OO
parts. Assume that / is continuous and / \f(x)\dx < 00. Also assume
Jo
that lim^^oo |/(a;)| = 0 and that, on every bounded interval, the function /
has a bounded derivative everywhere except at finitely many points. Then
/
Fc[f] = toTslf] - V 2 A / ( 0 ) , F.[f] = -w^c[/].
If, in addition, the function / ' has the same properties as / , then

Fc[f"] = -u2Tc[f] - y/2frf'(0), Fslf"} = -u2Ts[f] + #w/(0).

274 Discrete Fourier Transform

These results can be used to compute transforms without integration.

at
F O R EXAMPLE, let us compute Fc[f\ for f(t) = e~ , where a > 0. Since
f'(t) = -ae~at and / " = a2f, we have a2Fc[f) = -w2Tc[f\ + ay/2/^ or

EXERCISE 5.2.14. (a)B Use similar arguments to verify that, for

f(t) = e~ , F,\f](u>) = V2w/(y/^(a2+LJ2)).
at
(b)A Compare the results
with Exercise 5.2.12. Why can't we use the same trick to compute the
Fourier transform of f using (5.2.18)? Hint: If you simply allow x to be neg-
ative, then f £ Li(R); on the other hand neither the even nor the odd extension
of f is differentiable twice.
As a final comment, we mention that the Fourier transforms of rational
functions can often be computed using residue integration, see page 229.

5.3 Discrete Fourier Transform

5.3.1 Discrete Functions

Looking back at our study of Fourier series and transforms, we realize that
we have two parallel theories. One is for periodic functions represented by
the Fourier series. The other is for integrable functions on M represented
by the Fourier integral. Mathematics tries to avoid such separations as
much as possible and aims at a unified theory. How, then, can we com-
bine the Fourier series and Fourier integral? Clearly, a periodic function is
characterized by a countable number of Fourier coefficients, and an Li(R)
function, by its Fourier transform / , which is a function of a continuous
variable a;. In general, there is little hope to represent a general function
by a countably many values, and we therefore should try the other way and
represent the discrete collection of the Fourier coefficients as a function of
a continuous variable.
Given an integrable periodic function / = f(t) with period 2L, the
collection {c/t(/), k = 0, ± 1 , ± 2 , . . . } of the Fourier coefficients is a discrete
function defined only for u) = kn/L. How to define this collection as a
function of the continuous argument w? The obvious definition, f(oj) =
c/t(/) if w = nk/L, and f(u>) = 0 otherwise, is not adequate, because
any attempt to integrate the function / will produce 0. An alternative
definition, setting f(w) = ck{f) for ir{k - 1/2)/L < w < 7r(fc + 1/2)/L, is
 Discrete Functions 275

nicely integrable, but looses the main spirit of the discrete signal: we expect
/ to be zero everywhere except UJ = nk/L. To reconcile these seemingly
irreconcilable objectives, take an integer n > 1 and define the function

\x\
1 1 < l/(2n)
/v
' (5.3.1)
|x| > l/(2n).

For large n, we have fln(a;) = 0 everywhere except in a small neighborhood

of zero. On the other hand, J^° Un(x)dx = n ( l / n ) = 1 for all n. It would
be nice to pass to the limit as n —> oo, but the limit, which should be equal
to oo when x = 0 and zero otherwise, does not look like a function.
Let us try a different approach and consider the sequence of integrals
/„ = / f(x)fln(x)dx. If n is sufficiently large and / is continuous at
zero, then, by the mean-value theorem for integrals (or a version of the
rectangular rule), we get In « /(0), and, in fact,
oo _

/
f{x)Un(x)dx = /(0). (5.3.2)
•oo
EXERCISE 5.3.1. c Verify (5.3.2) for every function that is integrable on
[—1,1] and is continuous at zero. Hint: /(0) = f^° f(0)ti„(x)dx, and, by
continuity, for every e > 0, there exists an N so that, for all n > N, \f(x) —
/(0)|<ei/|;r|<l/(2n).
We can now use (5.3.2) to define linin-joo n „ as a "black box" (or a
rule) that takes a continuous function as an input, and produce the value
of that function at zero as the output. In mathematics, a rule that makes
a number out of a function is called a f u n c t i o n a l . Thus, the limit of
fln(a;), as n —> oo, is an functional. This functional is called D i r a c ' s
d e l t a function, in honor of the British physicist PAUL ADRIEN MAURICE
DlRAC (1902-1984), and is denoted by 5(x). Given what we know about
the functions f[ n , we have the following intuitive description of the delta
function:
oo
S(x)d:\x = l.
/ •OO

The expression
oo
5{x)f{x)dx = /(0) (5.3.3)
/ •OO
276 Discrete Fourier Transform

is both more precise and more convenient for calculations. Similarly, for a
real number a, 6a(x) = 5(x — a) is a functional that takes in a continuous
function and produces the value of the function at the point a:

f f{x)5(x - a)dx = / ( a ) . (5.3.4)

EXERCISE 5.3.2. (a)A Assume that the argument of the delta function is
measured in some physical units, such as time or distance. Show that the
delta function must then be measured in the reciprocal of the corresponding
units. Hint: this follows directly from (5.3.3). (b)c We can use (5.3.2) to
compute the Fourier transform of the delta function. Verify that

T[S\{w) = -—. (5.3.5)

(c)B Verify that

2j feL
TT,*(y-
L / - " V r)=
L J E^ - (*-™)
k——oo k=—oo
Hint: treat both sides as functionals, multiply by a function f = f(y) and in-
tegrate. Each term on the left will produce f(2nk/L); each term on the right,
y/2irf(—kL). Then use the Poisson summation formula (5.2.14) on page 266.
Coming back to the sequence of the Fourier coefficients Cfc(/) of a peri-
odic function / , let
00

/(w) = VfcF Yl ck(f)6{u-kir/L).

fc=—oo

Then f(u) ^ 0 only when ui = hir/L for some k, and, by (5.3.4),

- = / f(w)ei»t<kj= ] T ck(f)eitk"/L = Sf(t),

the Fourier series of / (that was the reason for introducing the extra factor
\/27r in the definition of f(w); recall that if / is the Fourier transform of / ,
then the right-hand side of the above equality is If, the Fourier integral of
/ ) . In other words, this function / is the right choice.
To summarize, the natural representation for a discrete function of a
continuous argument is a linear combination of Dirac's delta functions.
 Discrete Functions 277

T h e resulting computational benefits by far outweigh t h e need t o work

with functionals.

Next, we will look a t t h e spectrum of a discrete signal. Consider a

continuous-time signal / = f(t), sampled a t equally spaced points tk =
kAt, k = 0, ± 1 , ± 2 , T h e corresponding discrete signal is

fd(t) = At J2 f(tk)S(t-tk), (5.3.7)

fc=—oo

where t h e factor At is introduced t o preserve t h e dimension (units of mea-

surement) of / ; see Exercise 5.3.2.

E X E R C I S E 5 . 3 . 3 . B Verify that

OO OO /
27T£A
(5.3.8)
i — ~ > i —= — OO \
~At)
fe=-oo fc

where f is the Fourier transform of f. Hint: the first equality follows imme-
diately from (5.3.4). Then write

Y, f{^)e-iut"= Y }{t)e-iut5{t-tk)dt
k= — oo fe= — oo

and use (5.3.6) with L = 2n/ At; feel free to exchange summation and integration.

fit)

Fig. 5.3.1 Continuous and Discrete Signals

278 Discrete Fourier Transform

Equality (5.3.8) shows that the spectrum of the discrete signal is a pe-
riodic function with period 2n/At, obtained by the periodic repetition of
the original spectrum; see Figure 5.3.1. Recall that the Nyquist-Shannon
sampling theorem on page 265 provides an example of this effect, where
the samples of a signal with bounded spectrum represented the periodic
extension of the spectrum; with the special selection of the sampling fre-
quency, the extension had no overlaps. Exact recovery of the spectrum of /
from the spectrum of fd leads to exact recovery of / from fd. If the function
/ = / ( w ) does not vanish outside of a bounded interval, or if the sampling
frequency is not high enough, then the periodic extension of / contains
overlaps, and the exact recovery of / from fd by the Nyquist-Shannon sam-
pling theorem is not possible. The error of recovery of / from fd that is
due to this spectral overlapping is called the a l i a s i n g e r r o r .

5.3.2 Fast Fourier Transform (FFT)

We derived the Fourier series expansion of a periodic function by comput-
ing the best mean-square approximation of the function by a trigonometric
polynomial. Now let us formulate and solve a similar approximation prob-
lem for a discrete periodic function. Let /o, • • •, /AT-I> with fk = f(2Lk/N),
be N samples of a 2L-periodic function / . In discrete time, it is natural
to replace the best mean-square approximation with the best trigonometric
interpolation: find a trigonometric polynomial P/v(£) = Y^k=o c^emktlL so
that PN(2LTI/N) = fn, n = 0 , . . . , N — 1. Our objective is therefore to find
N numbers Co,..., CAT_I from N relations

N-l
£ Ckei**kn/N = /m n = o , . . . , AT - 1. (5.3.9)
fc=0

To proceed, we take a hint from relation (5.1.2), page 242.

c
EXERCISE 5.3.4. (a) Verify the following d i s c r e t e o r t h o g o n a l i t y
relation

y^e-i2,n(rn-k)/N=\N, m = k + IN, I = 0, ± 1 , ± 2 , . . . ( 5 3 i o )

^0 lo, otherwise.

Hint: e~l 2™£N = 1 for all n, (.; otherwise, you are summing N terms of a geo-
metric series with the first term equal to one, and the ratio equal to e-'2n(rn-k'>/N j
 FFT 279

so use the formula for the sum. (b) Conclude that cm = Fm, where

Fm
1 W_1
= N S fne~i27Tnm/N, (5.3.11)
n=0

and also
JV-l
/ „ = ^ i?fce i2 1 rnfc/JV ) n = 0 ,...,JV-l. (5.3.12)
fc=0

i7ini: to dehue (5.3.11), multiply both sides of (5.3.9) by e-i2^mn/N; sum owr
n, and use (5.3.10); (5.3.12) is the same as (5.3.9).
The collection of numbers Fn, n = 0 , . . . , N — 1, is called the d i s c r e t e
Fourier transform or d i s c r e t e Fourier coeff i c i e n t s of the collection
fn,n = 0,...,N-l.
Let us investigate the connection between the discrete Fourier transform
and the Fourier series. Recall that the Fourier coefficients of / are ck (/) =
we assume that the function / is equal to its
Fourier series for all t, then
oo

fn = f{2Ln/N)= J2 ck(f)ei2"kn/N. (5.3.13)

k= — oo

Substituting (5.3.13) in (5.3.11), we find

1 oo JV —1 oo
F c
m = jj E *(/) E e-' 2 *< m - f c W" = 2 WM/), (5-3-14)
fc= —OO 71=0 (= — oo

where the second equality follows from (5.3.10). If N is even and / is

band-limited, that is, Cfc(/) = 0 for \k\ > N/2, then (5.3.14) implies the
following relation between Fm and c m ( / ) :

Cm{f), 7Tl = 0, . . . , f - 1 ,
(5.3.15)
C-(N-m)(f), m=%,...,N-l.

If c/t(/) ^ 0 for infinitely many k, then the exact recovery of Ck(f) from
Fn is impossible due to the same a l i a s i n g e r r o r we discussed earlier in
connection with the Nyquist-Shannon sampling theorem.
Let us now discuss the computational aspects of formula (5.3.11).
Straightforward computation of all N numbers Fn according to (5.3.11)
280 Discrete Fourier Transform

requires N multiplications and additions of complex numbers to compute

each of the N coefficients. The total operation count is therefore AT2 com-
plex flops (floating point operations), with one complex flop being one
complex multiplication followed by one complex addition; in many applica-
tions this number of operations is inadmissibly large. The Fast Fourier
Transform, or FFT, is a special algorithm that computes Fn in fewer than
N2 operations. There are many versions of FFT, but all of them rely on
the same main idea; we will describe this idea next.
Introduce the following notation for the complex exponentials:

Wkn = e-i2irkn/N _ (5.3.16)

Note that, on both sides of (5.3.16), kn denotes the product of k and n.

Assume that iV is a composite number so that N = N\N2 with integer
Ni,N2 > 1. Then we write

k = Nxk2 + ki, ki=0,...,Ni-l, k2 = 0 , . . . , N2 - 1,

(5.3.17)
n = JV2ni + n2, ni = 0 , . . . , iVi — 1, n2 = 0 , . . . , N2 - 1,

and, using the properties of the exponential function,

Ufkn _ Tj/(-'Vifc2+fci)(Ar2n1+n2)
(5.3.18)
= W%k2niW%lk2n2W%2kiniWftn2 = Wfcn2Wk\niWfcn2.

EXERCISE 5.3.5. c Verify all the equalities in (5.3.18). Hint: W%lk2n2 =

k n2
W \ , etc.
Using (5.3.18), we now find
JV-I 1 JV1-1 /N2-i \
F kW n
- = 77 E f N = N E E M f e 2 + f c l <r KTK1"12-
fc=0 fci=0 Vfe2=0 /
(5.3.19)
For each n, computing the inside sum in the last expression requires N2
complex flops; the sum is then multiplied by W^711, which results in less
than A^2 + 1 complex flops (no addition involved). Finally, computation of
the outside sum requires N\ complex flops. With N coefficients to compute,
the total number M of complex flops involved is M < N(Ni+N2 + l) < N2.
The reduction is noticeable even for moderate values of TV. For example, if
N = 11 • 13, then M < 3575, while TV2 = 20449, that is, N2/M > 5.72. In
other words, when N = 11-13, the above method results in a nearly six-fold
reduction of the number of operations. If iVi or iVa is also composite, then
 Definition 281

a repeated application of this method results in even larger reduction of the

number of operations. With a more careful analysis, one can show that if
N — 2™, then the above FFT requires {N/2) log2 N complex multiplications
and N log2 N complex additions.
Formula (5.3.19) was known to Gauss, who used it in the early 1800s to
simplify some of his astronomical computations. Somehow, the result was
only published in Latin in 1865, and did not get much attention. A century
later, as the advances in electronic computing prompted the development
of digital signal processing, American mathematicians JAMES W. COOLEY
(b. 1926) and JOHN W . TUKEY (1915-2000) re-discovered (5.3.19); see
their paper An algorithm for the machine calculation of complex Fourier
series in the journal Mathematics of Computation, Vol. 19, pages 297-301
(1965).

5.4 Laplace Transform

Recall that the Fourier transform is defined for absolutely integrable func-
tions. For non-integrable functions, for example, those that are positive
increasing, it is often impossible to define the Fourier transform, and the
Laplace transform is used instead.

5.4.1 Definition and Properties

Let f{x) be a real-valued function defined on —oo < x < oo and suppose

f{x) = 0 for x < 0. (5.4.1)

Suppose further that, for some real value Sf > 0 ,

\e-"'xf(x)\dx < oo. (5.4.2)

Jo
sx
Then f£° \e f{x)\dx < oo for every complex number s with 3?s > Sf.

Definition 5.3 The Laplace transform of / is the function F = F{s)

of the complex variable s defined for !Rs > Sf by the formula

F{s] = / e-sxf{x)dx. (5.4.3)

Jo
282 Laplace Transform

In what follows, we will usually use a lower-case letter to denote a function,

and the corresponding upper-case letter to denote the Laplace transform.
Sometimes, we will also write C[f](s) for the Laplace transform of / .
EXERCISE 5.4.1. c Verify that the Laplace transform is an analytic function
in the half-plane {s : Sfts > Sf}.
The reader might be familiar with the Laplace transform from a course
in ordinary differential equations. Even though the transform bears the
name of P.-S. Laplace, it was Oliver Heaviside, co-inventor of vector analy-
sis, who developed the application of this transform to the study of differ-
ential equations; this application is known as the operational, or Heaviside,
calculus.
One reason to mention Laplace transform in our discussion is the connec-
tion with Fourier transform. Indeed, if the function / = f(x) is zero for x <
0 and Sf < 0, then the Fourier transform f(tj) = (27r) -1 / 2 J^° f(x)e~iuJXdx
of / is denned and

F(s) = V^f(-is), / H = -^=F(iw). (5.4.4)

Note that for every function / satisfying (5.4.1) and (5.4.2), and for every
real number so > Sf, the function /o(a;) = f(x)e~s°x' has s/ 0 < 0. As
a result, for functions / = f(x) that are equal to zero when x < 0, the
Laplace transform is a generalization of the Fourier transform.
Below are the main properties of the Laplace transform.

£[afi + bf2] = aFi +bF2, a,be C; (5.4.5)

£{f'}(s) = sF(s) - f(0+), C[xf\(s) = -F'(8); (5.4.6)

£[/<">](«) = snF(s) - (j2 s""fc-1/(fc)(0+)) ; (5.4.7)

\fc=o /

C[xnf](8) = {-l)nFW(s); (5.4.8)

If g{x) - / X f(y)dy then G(s) = ^ s ; (5.4.9)

Jo
If g(x) = f{ax), a > 0, then G{s) = -F(s/a); (5.4.10)
If g(x) = f(x - x0), x0 > 0, then G(s) = e-x°sF{s); (5.4.11)
 Definition 283

If h(x) =
-f Jo
JO
f(x- y)g(y)dy, then H(s) = F(s)G(s)

If/'(a;o) exists and a> Sf, then

(5.4.12)

1 ra+iL (5.4.13)
/ ( i o ) - ^ r - lim / F(s)e9X°ds.
2nr L-oo Ja_iL
In the above properties, all the functions involved must satisfy (5.4.1)
and (5.4.2). Note that, to invert the Laplace transform according to
(5.4.13), we can integrate along any vertical line in the half-plane {s :
5fts > Sf}. Two more properties of the Laplace transform are discussed in
Problem 6.6, page 440.
EXERCISE 5.4.2. B Verify (54.5)-(54.13). Hint: (54.5)-(54.12) are best
verified directly by definition, (54-13), by using (544) and the inversion formula
for the Fourier transform.
EXERCISE 5.4.3. c (a) Let h = h(x) be H e a v i s i d e ' s function h(x) = 1,
x > 0, h(x) = 0, x < 0. Show that H{s) = l/s, Us > 0. (b) Let F(x) =
e~ax, a g R. Show that F(s) = l / ( s + a), Us > -a.
We review the operational calculus by an EXAMPLE. Let us solve the
equation y" — 2y' + y = ex, with initial conditions y(0) = 1, y'(Q) = —1. We
have by (5.4.5) and (5.4.7): s2Y(s)-s+l-2{sY(s)-l) + Y(s) = ( s - 1 ) " 1 ,
3 2
Y(s) = ( s - l ) - + ( s - 3 ) ( s - l ) ~ . Consider the function F(s) = (s-1)-1.
x
We know that this is the Laplace transform of f(x) = e . We also notice
that F'(s) = - ( s - 1 ) " 2 and F"(s) = 2 ( s - 1 ) - 3 . Then the property (5.4.8)
implies that (s — l ) - 3 is the Laplace transform of (x2/2)ex. Next,

s-3 s-1 2 1 2
(s-1)2 (s-1)2 (s-1)2 s-1 (s-1)2'

and we conclude from the above calculations that (s — 3)(s — 1)~ 2 is the
Laplace transform of ex — 2xex. Then the solution of the equation is

V(x) =[-z--2x+l)e

EXERCISE 5.4.4. A Try to solve the following equation using the Laplace
transform: y" + x2y = e~x, y(0) = y'(0) = 0. Hint: do not be surprised if
you do not succeed, but try to understand the reason.
284 Laplace Transform

The two-sided Laplace transform Mf(s) = $^°00f{x)e~sxdx can

also be defined. In the theory of probability, this transform is known
as the moment generating function, and the Fourier transform, as the
c h a r a c t e r i s t i c function.
EXERCISE 5.4.5. B Compute Mf(s) if f(x) = e~x2/2. Hint: see page 272.
Let us now look at the discrete version of the Laplace transform, known
as the Z-transf orm.
Given the samples f(kAt), k = 0 , 1 , . . . , of a continuous signal / = /(£),
t > 0, we construct a discrete signal fd similar to (5.3.4) on page 277:
oo
fd(t) = At £ f(kAt)S(t - kAt), (5.4.14)
fc=o
where 5 is the Dirac delta function (5.3.5); recall that in this section we
always assume f(t) = 0 for t < 0. Taking the Laplace transform on both
sides of (5.4.14) results in
oo
Fd(s) = AtYlf(tk)e-skAt. (5.4.15)
fc=0

Similar to (5.3.8), we have

Fd{s)= J2 W s - i _ ) . (5.4.16)
fc=-oo ^ '

That is, Fd(s) is a periodic function with a purely imaginary period i2n/At.
EXERCISE 5.4.6. B Verify (5.4-16). Hint: the same arguments as for (5.3.8);
since f(t) < 0 for t < 0, integration and summation can start at —oo, when
necessary.
Take a real number a> Sf and write s = a—iuj. For fixed a, the function
H(u>) = Fd(a — iuj) is periodic with period 2-7r/At. Equality (5.4.15), which
can be written as

akAt AkujAt
l
H(LJ) = IAt^Tf(kAt)e-
\KIAL)H Ce
fc=0

becomes a Fourier series expansion of H. Then

Atf(kAt)e-akAt = ^ ^ H(oj)e-ik"Atdu,
27r
J-n/At
 System Theory 285

and, after some algebra, we get

1 ra+iir/At
ra-t-lT!/ l±l
f(kAt) - / Fd(s)eskAtds. (5.4.17)
1X1
Ja-iTt/At
EXERCISE 5.4.7 s Verify (5.4.17).
Let us introduce a new variable z = e s A t . Then the line segment (a —
in/At,a + in/At] becomes the circle Ca with center 0 and radius eaAt.
Also dz = AtesAtds. We now define the sequence x = x{k),k > 0, by
x(k) = f(kAt) and the function X = X(z) so that X(z) = Fd(s) when
z = esAt. Then equalities (5.4.15) and (5.4.17) become
oo .

X(z) = AtY^x{k)z~k, x{k) = -A— <j> X(z)zk-1dz. (5.4.18)

We call the function X = X(z) the Z-transform of the sequence x. By

construction, the Z-transform is defined only for those sequences x that
satisfy x(k) = 0, k < 0, and, in some definitions, At is taken to be I.
EXERCISE 5 . 4 . 8 . B Verify that the circle Ca encloses all singularities of the
function X(z)zk~1 for every k.
The following property of the Z-transform is useful in the study of finite-
differece equations: if y(k) = x(k — m) for some fixed number m, then
Y(z) = z~mX(z). Indeed, keeping in mind that x(k) — 0 for k < 0,
k
Y(z) = EZoy(k)z~ = EZo<k-m)z-k = EZm^ - m)Z-k =
Y,V=ox(k)z~{k+m) = z-mX{z).
We conclude this section with the connection between the Z-transform
and the Discrete Fourier Transform. Let x = x(k), k = 0 , . . . , N — 1, be
a finite sequence. Denote by X = X(z) the Z-transform of this sequence,
and by Xk, k — 0,...,N — 1, the coefficients (5.3.11) on page 279 of the
Discrete Fourier Transform. The the definitions imply
X r e i27rfe / JV ) = NXk. (5.4.19)

EXERCISE 5.4.9.B Verify (5.4.19).

5.4.2 Applications to System Theory

In this section, we will discuss the applications of the Laplace transform
to the analysis of linear autonomous dynamical systems. A l i n e a r system
produces a linear combination of outputs given the corresponding linear
286 Laplace Transform

combination of inputs. A autonomous system does not explicitly involve

the time variable.
A continuous-time autonomous dynamical system connects the input
signal x = x(t) with the output signal y = y(t) via a relation

f(x(t),x(t),x(t),..., x^(t), y(t), y(t),y(t), • • •, 2/(n) (*)) = 0. (5.4.20)

As usual, x denotes the first derivative with respect to t, and x^k\ the
derivative of order k. Given the input signal x(t), t > 0, we can use (5.4.20)
to find the output signal y(t), t > 0, as long as the function / is reasonably
good and we know the initial conditions 1/(0) = (2/(0),y(0),... ,yn"1(0)).
A p h y s i c a l l y implementable system has n > m. In particular, neither
the perfect amplification y(t) = kx(t) nor differentiation y(t) = x(t) is
implementable, even though both have theoretical interest, and can be im-
plemented approximately. Roughly speaking, condition n> m means that
the output at time t depends only on the input up to time t. At the end of
this section we will use the discrete-time analog of (5.4.20) to demonstrate
the necessity of this condition.
The natural assumption about (5.4.20) is that zero input and zero initial
conditions produce zero output, that is, / ( 0 , . . . , 0) = 0. Thus, the system
has an equilibrium point at ( 0 , . . . , 0). If we further assume that the function
/ = f(xo,..., xm, 2/0, • • •, Vn) has two continuous partial derivatives at zero,
then we have a l i n e a r i z a t i o n of (5.4.20) as follows:

a0x(t) + a i i ( t ) + a2x(t) + ... + anx{m)(t)

(5.4.21)
= boy(t) + hy(t) + b2y(t) + ... + bny^(t),

where a^ = fXk(Q, • • •, 0), 6fc = _ /y f c (0, • • •, 0) (recall that fx is another no-

tation for the partial derivative df/dx). The system (5.4.21) is autonomous
if and only if all the coefficients a^, bk do not depend on t. If bn ^ 0, then
y is uniquely determined from (5.4.21) given the input and the initial con-
ditions.
EXERCISE 5.4.10. B Using the Taylor series expansion, verify that
(5.4-21) is an approximation of (54-20) when \x(t)\, \x(t)\,..., |a; ( m '(i)|,
\y{t)\, \y(t)\,..., |t/ n )(£)| are all sufficiently small.
In what follows, we consider the linear autonomous systems of the type
(5.4.21), and assume that bn ^ 0. Note that the non-homogenous n-order
linear differential equation with constant coefficients corresponds to ao = 1,
oi = . . . = am = 0. Taking the Laplace transform of (5.4.21) and assuming
 System Theory 287

that xW{0) = 0 and y(*>(0) = 0 for all k, we find X(s) XT=o akSk =
WELcAAor
Y(s) = W(s)X(s), where W(s) = ^ " ^ V (5.4.22)

The function W = W(s) is called the t r a n s f e r function of the linear

system (5.4.21). In general, cancelling similar factors in the fraction (5.4.22)
changes the transfer function and is therefore not allowed. Often, a linear
system is identified with its transfer function, as in "consider the system
W."
Definition 5.4 The free motion of system (5.4.21) is the solution
y = y(t) of the differential equation YJk=obky^'(t) = 0 with some initial
condition (y(0), y(0),..., 2/ n_1 (0)). The system is called s t a b l e if every free
motion eventually dies out: lim^oo \y(t)\ = 0 for every initial condition.
EXERCISE 5.4.11. (a)B Verify that (54.21) is stable if and only if, for
every bounded function x = x(t) and all initial conditions, the solution
°f YJk=o°ky^(t) = x(t) stays bounded for all t > 0. (b)c Verify that
(5.4.21) is stable if and only if all the roots of the equation X)fc=o bkSk = 0
have negative real parts. Hint (works for both parts): recall that the free motion
of the system is a linear combination of the functions t£eSkt, where Sk is a root
of the above equation, £ = 0,...,£k, and tk is the multiplicity of Sk •
In other words, the system is stable if and only if all the poles of the
transfer function are in the (complex) left half-plane. There are criteria of
the stability of the system in terms of the coefficient bo,...,bn, which fall
outside the scope of our discussion. We only mention that the necessary
condition for stability is either 60 > 0 , . . . , bn > 0 or 60 < 0 , . . . , bn < 0 (that
is, all coefficients must be non-zero and have the same sign). Although not
desirable, unstable systems can be used as long as the input signal and the
initial conditions are periodically re-set to zero.
System (5.4.21) is the basic building block in the construction of more
complicated linear systems. There are two main ways to connect two sys-
tem: cascade and c l o s e d - l o o p . In the CASCADE CONNECTION, the out-
put yi(t) of one system of the type (5.4.21) is the input X2(t) of another,
see the left side of Figure 5.4.1. In terms of the transfer functions, we have
Y2(s) = ^2(5)^2(5) = W2{s)Y1{s) = W2(s)W1(s)X1(s), that is, the trans-
fer function of the cascade system is the product of the individual transfer
functions. One can certainly have a cascade of more than two systems.
288 Laplace Transform

EXERCISE 5.4.12. c Verify that the cascade system is stable if and only if
each of the individual systems is stable. Hint: use the result of the previous
exercise.

?\
T*& 9 x
t. wp
y=

Wx W2
Xl 2/1 = x2 2/2 r xp
wc

Cascade Vc Closed-loop

Fig. 5.4.1 Cascade and Closed-Loop Connections

In the closed-loop or feed-back connection, the two components are

often called the p l a n t , with transfer function Wp(s), and the c o n t r o l l e r ,
with transfer function Wc, see the right side of Figure 5.4.1. The output
y = yp of the plant is the input of the controller; the junction 0 combines
the output yc of the controller with the input x of the system to produce
the input xp of the plant.
If the signals x and yc are added so that xp = x + yc, then
Y(s) = Wp(s){X(s) + Y(s)Wc(s)) and W{s) = Y(s)/X{s) = Wp(s)/(1 -
Wp(s)Wc(s)) is the transfer function of the system. If the signals are sub-
tracted so that xp = x 2/c, then

Wp(s)
W(s) = (5.4.23)
1+ Wp(s)Wc(s)

(verify this). Notice that the signs of Wp and Ws do make a difference. For
example, when the signals x and yc are subtracted, changing the sign of Wc
produces the transfer function Wp(s)/(1 — Wp(s)Wc(s)). Unlike the cas-
cade system, individual stability of the components provides no information
about the stability of the closed-loop system.
A closed-loop connection can approximate certain individual systems
that are not physically implementable, that is, systems of the type (5.4.21)
with n < m. For example, if in (5.4.23) we take Wp(s) = K/(l + s), and
Wc{s) = 1/(1 + s), then, as K -> oo, we get W(s) « 1 + s.
EXERCISE 5 . 4 . 1 3 . C
(a) Give an example of an unstable closed-loop system
with stable plant and controller, (b) Verify that ifWp(s) — 1/(1 + 2s) and
 System Theory 289

Wc = 1/(1 +p), then the real parts of the poles ofW are - 3 / 4 . In other
words, the closed-loop system in this example is more stable than the plant
itself. This is the idea behind negative feedback.
The following table summarizes the common types of controllers.

Type Transfer Function Wc{s)

P K0
I Kts-1
D Kds
PI K0 + KiS~l
PID K0 + Kds + KiS'1

K0,Ki, Kd are positive real numbers. Not surprisingly, the letters P,I,D,
come from "proportional," "integral," and "derivative", respectively.
EXERCISE 5.4.14. B Let x = x(t) be the input of a PID controller and
y = y(t), the output. Assuming x(0) = y(0) — 0, find the relation between
x and y.
We conclude this section with a few words about discrete-time systems.
The discrete analog of (5.4.21) is
M-l N

y( ) = YL ™ ( ~ ) + X] b™y(k - m )-
k a x k m
(5.4.24)

This system is always physically implementable: the input sequence

x(k), k > 0, and the initial conditions y ( 0 ) , . . . , y(N — 1) define the unique
output sequence y(k), k > N. An example of a system that is not physically
implementable is at the end of this section.
Taking the z-transform on both sides of (5.4.24) and assuming that
x(k) = y{k) = 0 for k < 0, we find Y(z) = X(z) E m Jo a™z~m +
Y(z)ZZ=ibmz-m, or

Y{z) = W{z)X(z), where W{z) = ^m=° m

——-. (5.4.25)

We say that (5.4.24) is s t a b l e if, for every bounded input sequence x

and for all initial conditions, the output sequence y stays bounded for all
k > 0. Similar to continuous-time signals, one can show that (5.4.24) is
stable if and only if all poles of the transfer function W are in the unit
disk {z : \z\ < 1}. An unstable system can still be used if the input and
290 Laplace Transform

initial conditions are periodically reset to zero. As an EXAMPLE, consider

the d i g i t a l i n t e g r a t o r y(k) = x(k) + y(k - 1). It is unstable (if x(0) =
y(0) = 0 and x{k) = 1, k > 1, then y(k) = k), and its corresponding
transfer function W(z) = 1/(1 - z - 1 ) has a simple pole at z = 1. Still, a
computer program implementing this system will work just fine as long as
k is not allowed to get very large.
EXERCISE 5 . 4 . 1 5 . S Verify that if all bm = 0, then (5.4.25) is stable.
Given a function W = W(z), the corresponding system that has W
as the transfer function is physically implementable if the number of zeros
of W is less than or equal to the number of poles (both poles and zeros
are counted according to their multiplicities). F O R EXAMPLE, if W(z) =
z — 2, then the corresponding difference equation (5.4.24) is y(k) = x(k +
1) — 2x(k). This system is not implementable, because at time k, only
the values of x(0),..., x(k) are available. On the other hand, the transfer
function W(z) = 1 — 2 z _ 1 corresponds to the difference equation y(k) =
x(k) — 2x(k — 1) and is implementable.