EEO 401

Digital Signal Processing

Prof. Mark Fowler

Note Set #21

• Using the DFT to Implement FIR Filters
• Reading Assignment: Sect. 7.3 of Proakis & Manolakis
 Motivation: DTFT View of Filtering
There are two views of filtering:
* Time Domain x[ n ] h[n ] y[n ] = h[n ] * x[n ]
* Frequency Domain X f (θ ) H f (θ ) Y f (θ ) = H f (θ ) X f (θ )

The FD viewpoint is indispensable for analysis and design

of filters:
* Passband, Stopband, etc. of |Hf(θ)|
* Linearity of Phase ∠Hf(θ), etc.

Q: What about using DTFT for implementation?

* Compute DTFT of input signal and filter
* Multiply the two and take inverse DTFT

A: NO!!! Can’t compute DTFT – must compute at infinite

many frequency values 2/18
Desired Intention: But Does It Work?
But wait….
• If input signal is finite length, the DFT computes
“samples of the DTFT”
• Likewise, if filter impulse response is finite length

Q: So… can we use this?

x(0) Xf[k] Hf[k]Xf[k]

N-pt N-pt Do we get….
x(1)
DFT IDFT y(n) = h[n]*x[n]??
:
x(N-1)

N-pt
DFT

3/18
h(0) h(1) … h(N-1)
DFT Theory and Cyclic Convolution
A: Not Necessarily!!!!
DFT Theory (Sect. 7.2.2 in Proakis & Manolakis) tells us:
IDFT{H f [k ] X f [k ]} = h[n ] ⊕
* x[ n ]
Circular (Cyclic) Convolution
Thus… this block diagram gives something called cyclic
convolution, not the “ordinary” convolution we want!!!

Q: When does it work???

A: Only when we “trick” the DFT Theory into making
circular = linear convolution!!!!!!

Q: So… when does Cyclic = Linear Convolution???

Easiest to see from an example!!!!!

4/18
Linear Convolution for the Example
What does linear convolution give for 2 finite duration signals:

Original Signals:

x[n] Length N1 = 9

h[n] Length N2 = 5

First Non-Zero Output is at n=0:

x[m]

m
h[-m]
(flip, no shift – since n=0, multiply and add up)
m
5/18
Linear Convolution for the Example (cont.)
Last Non-Zero Output is at n = N1 + N2 – 2 = 12:

x[m]

m
(flip, shift by N1 + N2 – 2 = 12, multiply and add up)
h[12 - m]

The non-zero outputs are for n = 0, 1, … , 12  13 of them

In General: Length of Output of Linear Convolution = N1 + N2 – 1

6/18
Cyclic Convolution for the Example
Now… What does cyclic convolution give for these 2 signals:
“Original” Signals: 1. Zero-Pad Shorter Signal to Length of Longer One
2. Then Periodize Each

“First” Output Sample: 1. Flip periodized version around this point

2. No shift needed to get n = 0 Output Value
3. Sum over one cycle

Same as in Linear Conv!!!! Not Present in Linear Conv!!!! 7/18

Linear Convolution for the Example (cont.)
“Last” Output Sample (i.e., n = 8):
1. Flip periodized version around this point
2. Shift by 8 to get n = 8 Output Value
3. Sum over one cycle

Same as in Linear Conv!!!!

Note: If I try to compute the output for n = 9  Exactly the same case as for n = 0!!
Thus, the output is cyclic (i.e., periodic) with unique values for n = 0, 1, …. 8
In General: Length of Output of Cyclic Convolution = max{N1 , N2}

8/18
Making Cyclic = Linear Convolution
So…. Some of the output values of cyclic conv are different from linear conv!!!
Some of the output values of cyclic conv are same as linear conv
And….
The length of cyclic conv differs from the length of linear conv!!!

From the example above we can verify: N1 = Length of x[n]

If we choose K ≥ N1 + N2 – 1 N2 = Length of h[n]
And Zero-Pad Each Signal to Length K
Then Cyclic = Linear Convolution
Exercise: Verify this for the
above example!!!!

9/18
Simple Frequency-Domain Implementation
of FIR Filtering X(0)H(0)
X(1)H(1) If K is Strictly > N1+N2−1
: Then there will be extra zeros
X(K-1)H(K-1) here that can be ignored
x(0)
x(1) y(0)
: y(1)
K-pt K-pt
x(N1-1) :
FFT IFFT
0 :
: y(K-1)
0 K-pt The DFT of
FFT h(n) is usually
Pre-Computed
Zero-Pad Both to
Length K ≥ N1+N2−1
h(0) h(1) … h(N2-1) 0 … 0

Why Do This?
The FFT’s Efficiency Makes This Faster
Than Time-Domain Implementation
(In Many Cases) 10/18
Problems with the Simple FD Implementation
Q: What if N1 >> N2?
A: Then, need Really Big FFT  Not Good!!!
(Input signal much longer than filter length)
Also… can’t get any output samples until after whole signal is
available and FFT processing is done. Long Delay.

Example: Filter 0.2 sec of a radar signal sampled at Fs = 50 MHz

N1 = (0.2 sec)×(50×106 samples/sec) = 107 samples
FFT Size > 107  Really Big FFT!!!!

Q: What if N1 is unknown in advance?

Example: Filtering a stream of audio
A: FFT size can’t be set ahead of time – difficult programming

Simple FD Implementation Has Serious Limitations!!! 11/18

Better FD-Based FIR Filter Implementations
Two Very Similar Methods Exist Covered Here
• Overlap – Add (OLA)
• Overlap – Save (OLS)

Both methods exploit linearity of filter:

• Break input signal into a sum of blocks  x[n ] = ∑ xi [n ]
• Output = sum of response to each block i


z[n ] = ( x * h )[n ] Use the Simple
= ∑ (x h )[n ]
i *
FD-Based
Method to
i = zi [ n ] Compute Each
= ∑ zi [ n ] Output Block
i

The difference between OLA & OLS

12/18
lies in how the xi[n] blocks are formed
OLA Method for FD-Based FIR
For OLA: Choose xi[n] to be non-overlapped blocks of length NB
(blocks are contiguous)

 x[n ], iN B ≤ n < (i + 1) N B NB is a
xi [ n ] =  Design Choice
0, otherwise

Figure from Porat’s Book

13/18
Q: Now what happens when
each of these length-NB
blocks gets convolved with
the length-N2 filter?
A: The output block has
length N2+ NB – 1 > NB
• Output Blocks are
Bigger than Input
Blocks
• But are separated by
NB points
• Thus… Output Blocks
Overlap
• Total Output = “Sum
of Overlapped Blocks”

“Overlap-Add” Figure from Porat’s Book 14/18

OLA Method Steps
Assume: Filter h[n] length N2 is specified
Choose: Block Size NB & FFT Size NFFT = 2p ≥ NB + N2 – 1
Choose NB such that:
NB + N2 – 1 = 2p
It gives minimal complexity for method (see below)
Run:
Zero-Pad h[n] & Compute NFFT-pt FFT (can be pre-computed)
For each i value (“For Each Block”)
• Compute zi[n] using Simple FD-Based Method
 Zero-Pad xi[n] & Compute NFFT-pt FFT
 Multiply by FFT of h[n]
 Compute IFFT to get zi[n]
• Overlap the zi[n] with previously computed output blocks
• Add it to the output buffer

15/18
 OLA Method Complexity
• The FFT of filter h[n] can be pre-computed  Don’t Count it!
• We’ll measure complexity using # Multiplies/Input Sample
• Use 2NFFTlog2NFFT Real Multiplies as measure for FFT
• Assume input samples are Real Valued
Can do 2 real-signal FFT’s for price of ≈ 1 Complex FFT (Classic FFT Result!)

• For Each Pair of Input Blocks

 One FFT: 2NFFTlog2NFFT Real Multiplies
 Multiply DFT × DFT: 4NFFT Real Multiplies
 One IFFT: 2NFFTlog2NFFT Real Multiplies
 Total: 4NFFT [1 + log2NFFT] Real Multiplies
= 4(NB + N2 – 1) [1 + log2(NB + N2 – 1)]
• The Number of Input Samples = 2 Blocks = 2NB

• # Multiplies/Input Sample = 2(1 + (N2 – 1)/ NB) [1 + log2(NB + N2 – 1)]

16/18
 Comparison to TD Method Complexity
Complexity of TD Method
•Filter h[n] has length of N2
• To get each output sample:
 Multiply each filter coefficient by a signal sample: N2 Multiplies
• # Multiplies/Input Sample = N2 Multiplies

Condition Needed For OLA to Be More Efficient:

 N −1
21 + 2 [1 + log 2 (N 2 + N B − 1)] < N 2 ()
 NB 

Thus… For a given N2, Choose NB to minimize the left-hand side

17/18
FD Complexity vs TD Complexity
Plot of: [Left-Hand Side]/[Right-Hand Side] of ()
5
Contours = (OLA Multiplies)/(TD Multiplies)
10

Curve of
4
10 2.5 0.2 Optimal
0.4 Block
0.6
Sizes
Block Size NB

1
3 0.3
10 0.8
1.5

2
10

1
10
20 40 60 80 100 120 140 160 180 200
Filter Length N2

OLA More Efficient Only For

18/18
Filters Longer than 19