Algorithms of Scientific Computing

FFT on Real Data

Michael Bader
Technical University of Munich
Summer 2023
Fast Fourier Transform – Outline

• Discrete Fourier transform

• Fast Fourier transform
• Special Fourier transforms:
– real-valued FFT
– sine/cosine transform
• Applications:
– Fast Poisson solver (FST)
– Computergraphics (FCT)
• Efficient Implementation

Literature/Reference:
William L. Briggs and Van Emden Henson: The DFT – An Owner’s Manual for the Discrete
Fourier Transform. SIAM, 1995. https://doi.org/10.1137/1.9781611971514

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 2
DFT and Symmetry – Agenda

INPUT TRANSFORM

real symmetry fn ∈ R → Real DFT (RDFT)

even symmetry fn = f−n → Discrete Cosine Transform (DCT)

odd symmetry fn = −f−n → Discrete Sine Transform (DST)

“QUARTER-WAVE” INPUT TRANSFORM

even symmetry fn = f−n−1 → QW-DCT

odd symmetry fn = −f−n−1 → QW-DST

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 3
Real-valued DFT (RDFT)
Consider real-valued input data fn ∈ R, i.e.: fn∗ := fn = fn , then:
N N
2 2     
1 X
−i2πnk/N 1 X 2πnk 2πnk
Fk = fn e = fn cos − i sin .
N N N N
n=− N2 +1 n=− N2 +1

Properties:
N N
2 2
1 2πnk
• Re {Fk } = − N1 2πnk
P
P

N fn cos N , Im {Fk } = fn sin N
n=− N2 +1 n=− N2 +1
• only N independent, real coefficients necessary, since:
1 X ∗ n −nk o∗ 1 X −n(−k)
Fk∗ = fn ωN = fn ωN = F−k
N N
n o∗ n o∗
−nk
Recall: ωN = e−i2πn k/N = cos(−2πnk /N) − i sin(−2πnk/N)

= cos(2πnk/N) + i sin(2πnk/N) = ei2πn k/N = ωN

nk

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 4
Real DFT (2)

Map N values to N coefficients (and vice versa):

n o
f− N +1 , . . . , f0 , . . . , fN
2 2

DFT ⇓ ⇑ IDFT
n o
F0 , Re{F1 }, Im{F1 }, ..., Re{F N −1 }, Im{F N −1 }, F N
2 2 2

Note: real and imaginary parts of F−k correspond to those of Fk

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 5
Real DFT (3)

Situation:
• only N real input values (as all N imaginary parts are 0)
• only N independent, real output values (coefficient components)
due to symmetry F−k = Fk∗

Wanted → new transformation:

N real input values → N distinct real coefficient components
Hence: Insert symmetry F−k = Fk∗ in IDFT!

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 6
Real DFT (4)
Definition of “Real discrete Fourier transform” (RDFT)
• formulation 1
N
2     
1 X 2πnk 2πnk N
Fk = fn cos − i sin , k = 0, . . . ,
N N N 2
n=− N2 +1

• formulation 2
N
2  
1 X 2πnk N
Re{Fk } = fn cos , k = 0, . . . ,
N N 2
n=− N2 +1
N
2  
1 X 2πnk N
Im{Fk } = − fn sin , k = 1, . . . , −1
N N 2
n=− N2 +1

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 7
Inverse Real DFT
Goal: compute a real representation of the DFT (i.e., no e−i2πnk /N ).
We start with an input vector (Fourier coefficients) of length 2N, then:
N
X
fn = Fk ei2πnk /2N
k =−N+1
N−1
X 
= F0 + Fk ei2πnk/2N + F−k e−i2πnk /2N + FN ei2πnN/2N
k=1
N−1
X n o∗ 
= F0 + Fk ei2πnk/2N + Fk ei2πnk/2N + FN eiπn
k=1
N−1
X n o
= F0 + 2 Re Fk ei2πnk/2N + FN eiπn
k=1
N−1
X    
πnk πnk
= F0 + 2 Re{Fk } cos − Im{Fk } sin + FN cos (πn)
N N
k=1

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 8
Inverse Real DFT (2)

Set ak := 2 Re{Fk } and bk := −2 Im{Fk } (but a0 := Re{F0 } and aN := Re{FN })

to get :
N−1
X    
πnk πnk
fn = a0 + ak cos + bk sin + aN cos (πn)
N N
k =1

“Real inverse discrete Fourier transform”

Using the (real-valued) formula for Fk :
N   N  
1 X πnk 1 X πnk
ak = fn cos , bk = fn sin
N N N N
n=−N+1 n=−N+1

1 1
P P
but: a0 = 2N . . . and aN = 2N ...

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 9
Inverse Real DFT – Compare with Textbooks
Alternate (probably more frequent) notation in textbooks:
Set ak := 2 Re{Fk } and bk := −2 Im{Fk } to get
N−1
X    
1 πnk πnk 1
fn = a0 + ak cos + bk sin + aN cos (πn)
2 N N 2
k =1

Using the (real-valued) formula for Fk :

N   N  
1 X πnk 1 X πnk
ak = fn cos , bk = fn sin
N N N N
n=−N+1 n=−N+1

Differences:
• no extra definition of a0 and aN
• but factors 12 in formula for fn → not as nicely related to interpolation

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 10
Real-valued Trigonometric Interpolation

Interpretation of the real DFT as an interpolation problem:

• 2N ansatz functions:

gk (x) := cos(kx) k = 0, . . . , N
hk (x) := sin(kx) k = 1, . . . , N − 1

2πn πn
• 2N supporting points: xn := = n = −N + 1, . . . , N
2N N
• 2N interpolation conditions:

N−1
X    
πnk πnk
fn = a0 + ak cos + bk sin + aN cos (πn)
N N
k=1

(cmp. exercises)

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 11
Fast Real DFT

Computation of a real-valued DFT using complex FFT is inefficient:

• N redundant components computed (symmetry)
• complex arithmetics with lots of real/imaginary parts being 0

Possibilities to improve the efficiency:

1. compute two real DFTs from one complex FFT
2. compute a real DFT of length 2N from one complex FFT of length N
3. “compact” real FFT – use symmetry of the data directly in the algorithm

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 12
Two Real DFTs from one complex FFT

Idea: for real-valued gn and hn , compute DFT of fn := gn + ihn :

1 X
Fk = (gn + ihn )ωN−nk
N n

Comparison with coefficients Gk and Hk of the two real DFTs:

1 X 1 X
Gk = gn ωN−nk Hk = hn ωN−nk
N n N n

Due to linearity of the Fourier transform:

Fk = Gk + iHk

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 13
Two Real DFTs from one complex FFT (2)

Since gn and hn are real data, we have the following symmetry:

∗ ∗
Gk = G−k Hk = H−k .
∗
Hence, we get for F−k :
∗
= (G−k + iH−k )∗ = G−k
∗
+ i ∗ H−k
∗


F−k = Gk − iHk .

Together with Fk = Gk + iHk , we obtain

1 ∗

i ∗


Gk = Fk + F−k and Hk = − Fk − F−k .
2 2

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 14
Two real DFTs from one complex FFT – Algorithm

Algorithm to compute two real DFTs:

(1) set fn := gn + ihn
(2) compute Fk from FFT (using a library, e.g.)
(3) compute Gk and Hk according to

1 ∗

i ∗


Gk = Fk + F−k and Hk = − Fk − F−k
2 2
⇒ “half” the costs compared to using complex FFT

but: additional operations for pre- and postprocessing

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 15
Real DFT of length 2N from complex FFT of
length N
Compute DFT of a real-valued vector (f−N+1 , . . . , fN ):
N
1 X −nk N N
Fk = fn ω2N for k = − + 1, . . . ,
2N 2 2
−N+1

Split up in gn := f2n and hn := f2n−1 ; leads to butterfly scheme:

 
k
Fk = 21 Gk + ω2N Hk ,
 
k
Fk ±N = 12 Gk − ω2N Hk

for k = − N2 + 1, . . . , N2 , respectively.

“Homework”: Do try (to derive) this at home!

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 16
Real 2N-DFT from complex N-FFT

Now: compute Gk and Hk (two real DFTs) from one complex FFT
→ applied to zn := gn + ihn = f2n + if2n−1 :
∗ ∗
Gk = 12 Zk + Z−k Hk = − 2i Zk − Z−k



and

Combine both schemes to:

   
1 k ∗ k
Fk = 4 Zk 1 − iω2N + 14 Z−k 1 + iω2N , k = 0, . . . , N2
   
1 k 1 ∗ k
Fk +N = 4 Zk 1 + iω2N + 4 Z−k 1 − iω2N , k = − N2 + 1, . . . , 0

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 17
Real 2N-DFT from complex N-FFT – Algorithm

Algorithm for a real 2N-DFT:

(1) set zn := f2n + if2n−1
(2) compute Zk from FFT applied on zn (using a library, e.g.)
(3) compute Fk according to
   
k ∗ k
Fk = 14 Zk 1 − iω2N + 14 Z−k 1 + iω2N , k = 0, . . . , N2
   
k ∗ k
Fk+N = 14 Zk 1 + iω2N + 14 Z−k 1 − iω2N , k = − N2 + 1, . . . , 0

Complexity:
• determined by complex N-FFT: O(N log N)
• plus: additional operations for pre- and postprocessing O(N)
• instead of “O(2N log 2N)” for calling a complex FFT of size 2N

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 18
Compact Real FFT

Compute DFT of a real-valued vector (f−N+1 , . . . , fN ):

N
1 X −nk
Fk = fn ω2N for k = 0, . . . , N
2N
−N+1

Split up in gn := f2n and hn := f2n−1 ; leads to butterfly scheme:

 
k
Fk = 21 Gk + ω2N Hk for k = 0, . . . , N2 ,
 
k
Fk+N = 12 Gk − ω2N Hk for k = − N2 + 1, . . . , 0 .

Gk and Hk are coefficients of a real-valued DFT of length N; hence:

∗ ∗
Gk = G−k and Hk = H−k for k = 0, . . . , N2 − 1

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 19
Compact Real-valued FFT (2)

Use symmetry of Gk and Hk for the computation of Fk :

 
k
Fk = 12 Gk + ω2N Hk für k = 0, . . . , N2 ,
 
−k
FN−k = 21 G−k − ω2N H−k
 ∗
k
= 12 Gk − ω2N Hk for k = 0, . . . , N2 − 1

⇒ Computation of Fk (for k = 0, . . . , N) reduced to the computation of Gk

and Hk (for k = 0, . . . , N2 , respectively).

“Edson’s algorithm” (1968)

Michael Bader | Algorithms of Scientific Computing | FFT on Real Data | Summer 2023 20