EE123

Digital Signal Processing

Lecture 3B
Properties of DFT
Fast Convolutions

M. Lustig, EECS UC Berkeley

some of the material was based on slides by J.M. Kahn
Announcements

• HW1 due today.

• Homework Slip policy (lowest grade

homework will be dropped)

• Finish reading Ch. 8, start Ch. 9

• ham radio licensing lecture III Tonight

6:30-8pm Cory 521

M. Lustig, EECS UC Berkeley

Last Time

• Discrete Fourier Transform

– Similar to DFS
– Sampling of the DTFT (subtleties....more
later)
– Properties of the DFT
• Today
– Continue with properties
–Linear convolution with DFT
– Overlap-Add / Save method for fast
convolutions

M. Lustig, EECS UC Berkeley

DFT as Matrix Operator

DFT: 0 1
00 0n 0(N 1)
0 1 WN ··· WN ··· W 0 1
X[0] B N C x[0]
B C B . . . CB C
B . C B . . CB . C
B . C B . . . . . CB . C
B C B . . . . . CB C
B . C B CB . C
B X[k] C = B
B WNk0 ··· WNkn ··· W
k(N 1) CB
CB x[n] C
B C B N CB C
B C B CB C
B . C B . . . CB . C
B . C B . . C@ . C
@ . A B . . . . . C . A
@ . . . . . A
X[N 1] (N 1)0 (N 1)n (N 1)(N 1) x[N 1]
W ··· W ··· W
N N N

IDFT: 0 0(N 1) 1
00 0k
0 1 W ··· W ··· W 0 1
x[0] B N N N C X[0]
B C B . . . CB C
B . C B . . CB . C
B . C B . . . . . CB . C
B C B . . . . . CB C
B . C 1 B CB . C
B C = B n0 nk n(N 1) CB C
B x[n] C B W ··· W ··· W CB X[k] C
B C N B
B
N N N CB
CB C
B . C B . . . CB . C
B . C B . . C@ . C
@ . A B . . . . . C . A
@ . . . . . A
x[N 1] (N 1)0 (N 1)k (N 1)(N 1) X[N 1]
W ··· W ··· W
N N N

straightforward implementation requires N2 complex multiplies :-(

M. Lustig, EECS UC Berkeley
DFT as Matrix Operator

• Can write compactly as:

X = WN x
1 ⇤
x = WN X
N
• So,

1 ⇤ 1 ⇤ 1
x = WN X = WN WN x = (N I) x = x
N N N
WHY?

as expected.
M. Lustig, EECS UC Berkeley
Properties of DFT

• Inherited from DFS (EE120) so no need

to be proved

• Linearity

↵1 x1 [n] + ↵2 x2 [n] $ ↵1 X1 [k] + ↵2 X2 [k]

• Circular Sample Shift

j(2⇡/N )km km
x[((n m))N ] $ X[k]e = X[k]WN

M. Lustig, EECS UC Berkeley

Circular shift

x !n "
~ x !n # m"
~

... ... ... ...

n n
0 N #1 0 m N #1

x!n " x!$$n # m %%N "

n n
0 N #1 0 m N #1

M. Lustig, EECS UC Berkeley

Properties of DFT

• Circular frequency shift

j(2⇡/N )nl nl
x[n]e = x[n]WN $ X[((k l))N ]

• Complex Conjugation
⇤ ⇤
x [n] $ X [(( k))N ]

• Conjugate Symmetry for Real Signals

x[n] = x⇤ [n] $ X[k] = X ⇤ [(( k))N ]

Show....
M. Lustig, EECS UC Berkeley
Examples

• 4-point DFT X[k]

–Symmetry
···

0 1 2 3 k

X[k]

• 5-point DFT
···
–Symmetry
0 1 2 3 4 k

M. Lustig, EECS UC Berkeley

Examples
R
• 4-point DFT X[k]
–Symmetry
···

0 1 2 3 k

R X[k]

• 5-point DFT
···
–Symmetry
0 1 2 3 4 k

M. Lustig, EECS UC Berkeley

Properties of DFT

• Parseval’s Identity
N
X1 N
X1
12
|x[n]| = |X[k]|2
n=0
N
k=0

• Proof (in matrix notation)

✓ ◆⇤ ✓ ◆
⇤ 1 ⇤ 1 ⇤ 1 ⇤ ⇤ 1 ⇤
x x= WN X WN X = 2 X WN WN X = X X
N N N | {z } N
N ·I

M. Lustig, EECS UC Berkeley

Circular Convolution Sum

• Circular Convolution:
N
X1
x1 [n] N x2 [n] = x1 [m]x2 [((n m))N ]
m=0

for two signals of length N

• Note: Circular convolution is commutative

x2 [n] N x1 [n] = x1 [n] N x2 [n]

M. Lustig, EECS UC Berkeley

Compute Circular Convolution Sum

x1 [n]
1

0 1 2 3 4 5 6 n
x2 [n]
1

0 1 2 3 n
y[n] = x1 [n] 7 x2 [n] =?

M. Lustig, EECS UC Berkeley

Compute Circular Convolution Sum

x1 [n]
1

0 1 2 3 4 5 6 n
x2 [n]
1

0 1 2 3 4 5 6 n

Circular ‘flip’ y[n] = x1 [n] 7 x2 [n] =?

multiply and add
Here: y[0]

M. Lustig, EECS UC Berkeley

Compute Circular Convolution Sum

x1 [n]
1

0 1 2 3 4 5 6 n
x2 [n]
1

0 1 2 3 4 5 6 n

Equivalent periodic convolution over a period

y[n] = x1 [n] 7 x2 [n] =?

M. Lustig, EECS UC Berkeley
Result

y[n] = x1 [n] 7 x2 [n] =?

4

0 1 2 3 4 5 6 n

M. Lustig, EECS UC Berkeley

Properties of DFT

• Circular Convolution: Let x1[n], x2[n] be length

N

x1 [n] N x2 [n] $ X1 [k] · X2 [k]

Very useful!!! ( for linear convolutions with DFT)

• Multiplication: Let x1[n], x2[n] be length N

1
x1 [n] · x2 [n] $ X1 [k] N X2 [k]
N

M. Lustig, EECS UC Berkeley

Linear Convolution

• Next....
– Using DFT, circular convolution is easy
– But, linear convolution is useful, not circular
– So, show how to perform linear convolution
with circular convolution
– Use DFT to do linear convolution

M. Lustig, EECS UC Berkeley

Linear Convolution

• We start with two non-periodic sequences:

x[n] 0nL 1
h[n] 0  n  P 1
for example x[n] is a signal and h[n] an impulse response of a filter

• We want to compute the linear

convolution: L
X1
y[n] = x[n] ⇤ h[n] = x[m]h[n m]
m=0

y[n] is nonzero for 0 ≤ n ≤ L+P-2 with length M=L+P-1

• Requires L∙P multiplications

M. Lustig, EECS UC Berkeley
Example

x1 [n]
1
L=5

0 1 2 3 4 n
x2 [n]
1

P=4

0 1 2 3 n

M=L+P-1=8

M. Lustig, EECS UC Berkeley

Example: Linear Conv. Via Circular

x1 [n]
1

0 1 2 3 4 5 6 7 n
x2 [n]
1

0 1 2 3 4 5 6 7 n

M=L+P-1=8

M. Lustig, EECS UC Berkeley

Example

x1 [n]
1

0 1 2 3 4 5 6 7 n
x2 [n]
1

0 1 2 3 4 5 6 7 n
Circular ‘flip’
M=L+P-1=8
y[n] = x1 [n] 8 x2 [n] = x1 [n] ⇤ x2 [n]
M. Lustig, EECS UC Berkeley
Linear Convolution via Circular Convolution

• Zero-pad x[n] by P-1 zeros

⇢
x[n] 0  n  L 1
xzp [n] =
0 LnL+P 2

• Zero-pad h[n] by L-1 zeros

⇢
h[n] 0  n  P 1
hzp [n] =
0 P nL+P 2

• Now, both sequences are of length

M=L+P-1

M. Lustig, EECS UC Berkeley

Linear Convolution via Circular Convolution
Both zero-padded sequences xzp [n] and hzp [n] are of length
• Now,
M =both
L + P sequences
1 are of length
M=L+P-1
• WeWecan
can now
computecompute
the linear the linear x[n] ⇤ h[n] = y [n] by
convolution
convolution usingconvolution
computing circular a circular oneM with
xzp [n] hzp [n]:
length M = L+P-1

Linear convolution via circular

(
xzp [n] M hzp [n] 0nM 1
y [n] = x[n] ⇤ y [n] =
0 otherwise

M. Lustig, EECS UC Berkeley

Miki Lustig UCB. Based on Course Notes by J.M Kahn Fall 2012, EE123 Digital Signal Processing
Linear Convolution using DFT

• In practice we can implement a circulant

convolution using the DFT property:
x[n] ⇤ h[n] = xzp [n] M hzp [n]
1
= DFT {DFT {xzp [n]} · DF T {hzp [n]}}

for 0 ≤ n ≤ M-1, M=L+P-1

• Advantage: DFT can be computed with

Nlog2N complexity (FFT algorithm later!)
• Drawback: Must wait for all the samples --
huge delay -- incompatible with real-time
M. Lustig, EECS UC Berkeley
Block Convolution

• Problem:
– An input signal x[n], has very long length
(could be considered infinite)
– An impulse response h[n] has length P
– We want to take advantage of DFT/FFT and
compute convolutions in blocks that are
shorter than the signal
• Approach:
– Break the signal into small blocks
– Compute convolutions
– Combine the results
M. Lustig, EECS UC Berkeley
 olution
Convolution
Convolution
Block Convolution
ple:
ple:
Example:
pulse Response, Length P =Length
Impulse Response, 6 P = 6
h[n] Impulse response, Length P=6
Impulse Response, Length P = 6

0 10 1020 2030 30
0 10 20 30
n n
x[n] Input Signal, Length P=33
Input Signal, Length
Input
n
33 Length 33
Signal,
Input Signal, Length 33
y[n] Output Signal,
LinearLength
Linear Convolution, P=38
Length 38 Length 38
Convolution,
Linear Convolution, Length 38

5 0.5 0.5
0.5

y[n]
0 0
y[n]

y[n]
0

5 -0.5 -0.5
-0.5

0 10 1020 2030 30 0 0 10 10 20 20 30 3
0 10 20 30 0 10 20
n n n n
n n

M. Lustig, EECS UC Berkeley

UCB.
sed onBased onNotes
Course Course
byNotes
J.M by J.M Kahn
Kahn Fall Fall
2012, 2012,Digital
EE123 EE123Signal
DigitalProcessing
Signal Proc
UCB. Based on Course Notes by J.M Kahn Fall 2012, EE123 Digital Signal Pro
Overlap-Add Method
Overlap-Add Method
• Decompose into non-overlapping segments Overlap-Add, Input Segments , Length L = 11

0.5
x0[n]
⇢

x0[n]
0

x[n] rL  n < (r + 1)L

xr [n] = -0.5

0 otherwise
Block Convolution 0 10
n
20 30

0.5
x1[n]
Example:

x1[n]
0

-0.5

Impulse Response, Length P = 6

0 10 20 30

• The input signal is the

n

0.5
0.5
x2[n]
sum of segments
xh[n]
0

[n]
0

2
-0.5
-0.5
0 10 20 30

1
n

X 0 10
n
20

x[n] = x0[n]+x1[n]+x2[n]
Input Signal, Length 33
30
Overlap-Add, Sum of Input Segments

x[n] = xr [n]
0.5

0.5
x[n]

r=0 -0.5
x[n]

0
0 10 20 30
n

Miki Lustig UCB. Based on -0.5

Course Notes byEECS
M. Lustig, J.M Kahn
UC Berkeley
0 10 20 30
Overlap-Add Method
1
X
x[n] = xr [n]
r=0

• The output is:

1
X
y[n] = x[n] ⇤ h[n] = xr [n] ⇤ h[n]
r=0

• Each output segment xr[n]*h[n] is length

N=L+P-1

M. Lustig, EECS UC Berkeley

Overlap-Add Method

• We can compute xr[n]*h[n] using linear

convolution
• Using the DFT:
– Zero-pad xr[n] to length N
– Zero-pad h[n] to length N and
compute DFTN{hzp[n]} (only once) WHY?
– Compute
1
xr [n] ⇤ h[n] = DFT {DFT{xr,zp [n]} · DFT{hzp [n]}}
• Neighboring outputs overlap by P-1
– Add overlaps to get final sequence
M. Lustig, EECS UC Berkeley
Method
Example of overlap and add:
Overlap-Add, Input Segments , Length L = 11 Change to Ys!
Overlap-Add, Output Segments , Length L+P-1 = 16

0.5
x0[n] 0.5
x0[n]

y0[n]
0 0

0.5 -0.5

Convolution
Convolution
0 10
n
20 30 0 10
n
20 30

0.5 x1[n] 0.5 x1[n]

mple:
mple:

y1[n]
0 0

0.5 -0.5

ImpulseResponse,
Impulse Response,Length
LengthPP ==66
0 10 20 30 0 10 20 30
n n

0.5
0.5
0.5
x2[n] 0.5
x2[n]
00

y2[n]
0 0

0.5 -0.5
-0.5
0.5
0 10 20 30 0 10 20 30
n n
00 10
10 20
20 30
30
Overlap-Add, Sum of Input Segments Overlap-Add, Sum of Output Segments

0.5
Input
nn
x[n] = x0[n]+x1[n]+x2[n]
Input Signal,
Signal,Length
Length33
33
0.5
y[n] = y0[n]+y1[n]+y2[n]
LinearConvolution,
Linear Convolution,Length
Length 38
38

0.5
0.5 0.5
0.5
y[n]

0 0

0.5 -0.5
y[n]

00 00
y[n]

0 10 20 30 0 10 20 30
n n

0.5
-0.5 -0.5
-0.5
Course Notes by J.M Kahn Fall 2012, EE123 Digital S

00 10
10 20
20 30
30 00 10
10 2020 30
30
nn nn
Overlap-Save Method

• Basic idea:
• Split input into (P-1) overlapping segments
with length L+P-1
⇢
x[n] rL  n < (r + 1)L + P
xr [n] =
0 otherwise

• Perform circular convolution in each

segment, and keep the L sample portion
which is a valid linear convolution

M. Lustig, EECS UC Berkeley

Recall:
x1 [n]
1

0 1 2 3 4 5 6 n
x2 [n]
1

0 1 2 3 4 5 6 n

4
Valid linear convolution!
2

0 1 2 3 4 5 6 n
M. Lustig, EECS UC Berkeley
veExample
Methodof overlap and save:

Overlap-Save, Input Segments, Length L = 16 Overlap-Save, Output Segments, Usable Length L - P + 1

Usable (y0 [n])

0.5 0.5 Unusable

y0p[n]
x0[n]

0 0

-0.5 -0.5

0 10 20 30 0 10 20 30
n n

Usable (y1 [n])

Unusable
0.5 0.5

y1p[n]
x1[n]

0 0

-0.5 -0.5

0 10 20 30 0 10 20 30
n n
Usable (y2 [n])
Unusable
0.5 0.5

y2p[n]
x2[n)

0 0

-0.5 -0.5

0 10 20 30 0 10 20 30
n n
Overlap-Save, Concatenation of Usable Output Segments

0.5
y[n]

-0.5

0 10 20 30
n

sed on Course Notes by J.M Kahn Fall 2012, EE123 Digital Signal Proce