Introduction to

Digital Signal Processing

slides

Dr. Gal Ben-

Ben-David
Winter 2009
2009//2010
1
 Agenda

Signal and Systems brush-
brush-up, Poisson formulas

Sampling: Point sampling, Impulse Sampling, Band- Band-
pass Sampling, Nyquist Rate, Shannon reconstruction

DFT - Discrete Fourier Transform, Spectrum analysis,
Windows, Zero Padding, Cyclic convolution, Periodic
Signals, FFT

Continuous Phase representation, Linear Phase,
Minimum Phase, All Pass

FIR filters, IRT Method, Windows, Equiripple

IIR Filter, Analog filters, Impulse invariance, Bilinear
Transform

Multirate,, Decimation, Interpolation
Multirate
2
 Course requirements

Theoretic Homework (up to 2 students per submission) - 8%

Matlab exercises (up to 2 students per submission) – 8%

No grade transfer

Final Exam – Moed A, B, Miluim - 84% 84%

Note that for a 170
170++ students class we cannot find personal
“solutions” for grades (i.e., no oral exams, no extra homework, no
special exam dates, ….)

Intensive use of Moodle for announcements, forums, sound files,
submission, HW partners, homework, solutions, ask your lecturer

3
 Literature

Malah
Malah--Raz (Hebrew)

+ Equiripple

+ Multirate

- DFT frequency zero padding

- IIR Cheb
Cheb.,., Elliptic filters will be given in
tutorial only

B.
Porat,
Porat, A Course in Digital Signal
Processing, J. Wiley, 1997

4
 Analog Signal Processing

y (ω ) 1
=
x(t) y(t) x(ω ) 1 + jωRC

In fact R can be purchased (within reasonable price) at 1%

precision, while C is 10% accurate.
The corner frequency is accurate to ~10% (3-4 bit)

5
 Analog Signal Processing

Legacy – Resistors, Capacitors, Operational Amplifiers

Mostly used for filtering

Simple

Limited in scope

Limited in accuracy

Cannot be adapted

Repeatability problems

Aging

Sensitivity to the environment

Uncertain performance in production units

Variation in performance of units

Sensitive analog traces on PCBs

6
 Digital Signal Processing

Platform

General purpose computers

Dedicated Signal Processors

Dedicated hardware

Accuracy only depends on computer registers

Flexible

Repeatable

Adaptive

May be more sophisticated

7
 What is DSP?

Digital Signal Processing – the
processing or manipulation of signals
using digital techniques

Digital
Input Signal Output
Signal
ADC DAC Signal
Processor
Analogue Digital to
to Digital Analogue
Converter Converter
Speech Processing

Speech coding/compression

Speech synthesis

Speech recognition
 Some Properties of Speech
Vowels

“oo” in “blue” “o” in “spot” “ee” in “key” “e” in “again”

•Quasi-periodic
•Relatively high signal power
Consonants

“s” in “spot” “k” in “key”

•Non-periodic (random)
•Relatively low signal power
 Speech Coding

64 kbits/s
22.8 kbits/s

13 kbits/s

BTS
 Speech Coding – Vocoder
Encoder
Original Speech

Analysis:
• Voiced/Unvoiced decision
• Pitch Period (voiced only)
• Signal power (Gain)

Pitch Decoder
Period Signal Power
Pulse Train V/U
G Vocal Tract
Model

Synthesized Speech
Random Noise
 Digital Audio

Standard music CD:

Sampling Rate: 44.
44.1 kHz

16
16--bit samples

2-channel stereo

Data transfer rate = 2×16×
16×44
44,,100 = 1.4
Mbits/s

1 hour of music = 1.4×3,600 = 635 MB
 Audio Coding (Cont’d)

Key standards:

MPEG: Layers I, II, and III (MP3
(MP3); AAC.
• used in DAB, DVD

Dolby AC3
AC3, Dolby Digital, Dolby Surround.

Typical bit rates for 2-channel stereo:

64
64kbits/s
kbits/s to 384 kbits/s.

Subband
Subband--or transform-
transform-based, making
use of perceptual masking properties.
 Image/Video

Still Image Coding:

JPEG (Joint Photographic Experts Group):
• Discrete Cosine Transform (DCT) based

JPEG2000
JPEG 2000:: Wavelet Transform based

Video Coding:

MPEG (Moving Pictures Experts Group):
• DCT
DCT--based,
• Interframe and intraframe prediction,
• Motion estimation.
• MPEG
MPEG--1, MPEG-
MPEG-2, MPEG-
MPEG-4

H.261
H. 261,, H.
H.263
263,H.
,H.264
264

Applications: Digital TV, DVD, Video conference,etc
conference,etc..
 Some Other Application Areas

Image analysis, e.g
e.g::

Human recognition,

Optical Character Recognition (OCR);

Restoration of old image, video, and audio signals;

Analysis of RADAR data;

Analysis of SONAR data;

Data transmission (modems, radio, echo cancellation,
channel equalization, etc.);

Storage and archiving;

Control of electric motors (!).
 DSP Devices & Architectures

Selecting a DSP – several choices:

Fixed-point;
Fixed-

Floating point;

Application--specific devices
Application
(e.g. FFT processors, speech recognizers,etc
recognizers,etc.).
.).

DSP Manufacturers:

Texas Instruments (http://www.ti.com
(http://www.ti.com))

Motorola/Freescale
Motorola/ Freescale (http://www.motorola.com
http://www.motorola.com))

Analog Devices ((http://www.analog.com
http://www.analog.com))

Israel: DSP Group, Zoran
Zoran,, ECI, Rafael, Elbit,
Elbit, IAI, Comverse,
Intel, Microsoft, Elta
Elta,, MAF-
MAF-AT, ....
Typical DSP Operations – MADD/MACC
• Filtering Convolution
• Energy of Signal
• Frequency transforms

Pseudo C code
for (n=0; n<N; n++)
{
s=0;
for (i=0; i<L; i++)
{
s += a[i] * x[n-i];
}
y[n] = s;
}
Traditional DSP Architecture

X RAM x(n-i) ai Y RAM

Multiply/Accumulate

Accumulator

y(n)
 Sampling and quantization
Amplitude

Discrete
Continuous
50
49
48
47
46
45

T 2T Time
20
 Continuous Time Fourier
Transform
Continuous time signal

CTFT

Angular frequency ω = 2πf

Sufficient condition

21
 Inverse CTFT

Reconstruction of non
continuous point

22
 Dirchlet conditions

In each finite section, x(t) has

finite number of non-
non-continuous
points. The “jump” is finite

Bounded variation – Finite

number of extreme points in each
finite section

23
Sinc in time
Gate in frequency

Note:
X(t) is not
absolute
integrable

24
Gate in time
Sinc in frequency

The inverse transform

25
Order 1 Low pass filter

26
Order 1 Low pass filter

27
CTFT

28
CTFT

29
CTFT

30
 Delta or Unit Impulse Function, δ(t)

The delta or unit impulse function, δ(t)

Mathematical definition (non-
(non-pure version)
0 t ≠ t0
δ (t − t0 ) = 
1 ⋅ ∞ t = t0

Graphical illustration

δ(t)
1

0 t0 t

31
 Dirac delta function
Impulse Function
For f() continuous at τ

Impulse function in time contains all frequencies -

same amplitude 32
 DC in time

Impulse in frequency

33
Harmonic exponentials

34
COS
/SIN

35
 SISO – Single Input – Single Output
Systems

A linear
Linear system has a
zero response
for zero
stimulus

Time invariant

36
Linear Time-
Time-Invariant Systems

x(t)=δ(t) System h(t)

T
t t
If a LTI system is excited by an input x(t) = δ(t), the
output is called the impulse response

h(t ) = T {δ (t )}

37
 Unit Impulse

Any continuous
function can be
written as weighted
‘sum’ of impulses as ∞
x(t ) = ∫ x(τ )δ (t − τ )dτ
−∞

38
Response to Arbitrary Input
∞ 
y (t ) = T {x(t )} = T  ∫ x(τ )δ (t − τ )dτ 
− ∞ 
∞
= ∫ x(τ )T {δ (t − τ )}dτ
−∞

But since the system is time invariant

T {δ (t − τ )} = h(t − τ )
∞
y (t ) = ∫ x(τ )h(t − τ )dτ
−∞

y (t ) = x(t ) ∗ h(t )
39
 Convolution
∞
y (t ) = ∫ x(τ )h(t − τ )dτ
−∞

y (t ) = x(t ) ∗ h(t )

x(t) LTI System y(t)

h(t)

40
Properties of the Convolution Integral

Commutative
y (t ) = x(t ) ∗ h(t ) = h(t ) ∗ x(t )

Associative
{x(t ) ∗ h1 (t )}∗ h2 (t ) = x(t ) ∗ {h1 (t ) ∗ h2 (t )}

Distributive
x(t ) ∗ {h1 (t ) + h2 (t )} = x(t ) ∗ h1 (t ) + x(t ) ∗ h2 (t )

41
Eigenfunctions of Continuous-
Continuous-Time LTI Systems

∞
y (t ) = ∫ x(t − τ )h(τ )dτ
x(t)=est LTI System −∞

h(t)

∞ ∞
y (t ) = ∫ h(τ )e s (t −τ ) dτ = e st ∫ h(τ )e − sτ dτ
−∞ −∞
= H ( s )e = λe st
st

λ is the eigenvalue of the system associated with the

eigenfunction est , a complex constant

42
Eigenfunctions of Continuous-
Continuous-Time LTI Systems
Harmonic input
Harmonic output

43
 Transfer function
Define

Harmonic in
Harmonic out

LTI system changes the amplitude and phase of harmonic signals

44
LTI Real impulse response
systems

45
 Periodic function
f (t) f ( t ) = f ( t + kT ) ∀t , ∀k

t
T T

f(t)⇒ a series
of frequencies
multiple of 1/T

46
Orthogonal basis

47
Single Periodic extension

f (t)
period
t
T
CTFT of the single period

The Fourier series coefficients are samples of the CTFT of the single
period function
Samples in frequency – Periodicity in time
48
Poisson Sum formula

We get spectral lines

49
 f(t) F(ω)

t
ω
T

f(t) F(ω)

t
T T ω
ω0

50
 Poisson Sum formula

The formula may be used for arbitrary xI().

For the impulse function

51
 Poisson Sum formula

Impulse train in time – Impulse train in frequency

52
 Sampling “quick and dirty”
x(t) X(f)

t
BW
p(t) P(f)
f

t
T F=1/T

x(t)p(t) X(f)*P(f) f

t f

Harry Nyquist (1889-1976) – Replicas will not overlap if F>2BW

53
 Parseval equation

Sum of Energy of
Single period energy
Fourier coefficients

54
 Symmetry for real x(t)

The Fourier series is composed of cosines

55
 Finite Fourier series
Instead of infinite series, we approximate by using 2N-1 elements

Best approximation for the L2 metric

- Gibbs

56
 Periodic functions and LTI Systems

H(ω)

∞
jkω 0 +ψ ( kω 0 )
∑ k
a H (kω 0 ) ⋅ e
k = −∞

The reponse to a periodic function is also periodic

57
The Discrete time Fourier transform
X f (θ ) = X f (θ + 2π )

Existence sufficient condition

Inverse transform

58
Example

59
Harmonic functions

60
DTFT

61
DTFT

62
DTFT

63
 Unit sample response

x(n)=δ(n) System h(n)

h(n)
t
Convulution

64
 Response to harmonic signal
Eigenfunction

Frequency response

65
 Discrete time periodic signals

Note that cos signal

is periodic only if rational

66
Sampling

67
 Sampling - Aliasing
1.2

1
__ s(t) = sin(2πf t)
0.8 0
0.6

0.4
s(t) @ fS
0.2

0
f0 = 1 Hz, fS = 3 Hz
-0.2 tt
-0.4

-0.6

-0.8
__ s (t) = sin(8πf t)
1 0
-1

-1.2
__ s (t) = sin(14πf t)
2 0

s(t) @ fS represents exactly all sine-waves sk(t) defined by:

sk (t) = sin( 2π (f0 + k fS) t ) , k ∈

68
Sampling of continuous time x(t)

Sampling frequency

Sampling period

What is the relationship between the CTFT of x(t) and

the DTFT of x(n)?

69
 Inverse CTFT

We calculate the infinite integral in sections of 2π/T

XF(ω)

… …
ω
−3π/T -π/T π/T 3π/T
70
Compare to

71
Sampling

72
 Band limited signal.
Nyquist condition is met

Nyquist

73
 Band limited signal.
Nyquist condition is met

2π corresponds to the sampling frequency 74

 Band limited signal.
Nyquist condition not met

Aliasing
75
 Sampling low-
low-pass signals
Continuous spectrum
(a) (a) Band-limited signal:
frequencies in [-B, B] (fMAX = B).

-B 0 B f
Discrete spectrum
(b)
No aliasing
(b) Time sampling frequency
repetition.
fS > 2 B no aliasing.
-B 0 B fS/2 f

Discrete spectrum
(c) Aliasing & corruption
(c) fS 2B aliasing !

Aliasing: signal ambiguity

0 fS/2 f
in frequency domain

76
 Remark
The condition

May be extended to

If no delta component exists at half the sampling frequency

77
Cos at fs/2

π
78
Cos at fs/2

π
79
 Sinc

For

π 80
Non band limited signals

81
 Antialiasing filter
(a) Signal of interest (a),(b) Out-of-band noise can aliase
Out of band
noise
Out of band into band of interest. Filter it before!
noise

-B 0 B f
(c) Antialiasing filter
(b) Passband: depends on bandwidth of
interest.

Attenuation AMIN : depends on

-B 0 B fS/2 • ADC resolution ( number of bits N).
(c) f
AMIN, dB ~ 6.02 N + 1.76
Antialiasing
• Out-of-band noise magnitude.
Passband filter
frequency
Other parameters: ripple, stopband
frequency...
-B 0 B f

82
 Under--sampling
Under
Using spectral replications to Bandpass signal
B
reduce sampling frequency fS centered on fC
requirements.
0 fC
2 ⋅ fC + B 2 ⋅ fC − B f
≤ fS ≤
m +1 m

m∈ , selected so that fS > 2B

-fS 0 fS 2fS
f fC
Example
fC = 20 MHz, B = 5MHz Advantages
Without under-sampling fS > 40 MHz.
Slower ADCs / electronics
With under-sampling fS = 22.5 MHz (m=1);
needed.

= 17.5 MHz (m=2); = 11.66 MHz (m=3).
Simpler antialiasing filters.

83
Sampling of continuous time x(t)

Sampling frequency

Sampling period

What is the relationship between the CTFT of x(t) and

the DTFT of x(n)?

84
 Inverse CTFT

We calculate the infinite integral in sections of 2π/T

XF(ω)

… …
ω
−3π/T -π/T π/T 3π/T
85
Compare to

86
Sampling

87
 Band limited signal.
Nyquist condition is met

Harry
Nyquist
1889-
1976 88
 Band limited signal.
Nyquist condition is met

2π corresponds to the sampling frequency 89

 Band limited signal.
Nyquist condition not met

Aliasing
90
 Remark
The condition

May be extended to

If no delta component exists at half the sampling frequency

91
Cos at fs/2

π
92
Sin at fs/2

93
Sinc

For

π 94
Non band limited signals

95
Antialiasing filter
(a) Signal of interest

Out of band
Out of band
noise
noise

-B 0 B f

(b)

-B 0 B fS/2
(c) f

Antialiasing
Passband filter
frequency

-B 0 B f

96
Example Linear Technology Inc. LTC1564
LTC1564
Digitally Controlled Antialiasing Filter

97
98
99
100
 Successive approximation register A/D
Acquisition Time
VS Data Output Register
½ LSB

VCSH(t)
tACQ
VSH0

t0 tACQ RSW
RS
Time -
VS S1
SAR
CSH +

N-bit DAC
Sample and Hold circuit
“Parasitic” LPF before sampling.
RC Designed to be much faster than Fs.
What happens with Band pass sampling ??
101
 Sampled signal

Let us sample the continuous-time non-periodic signal

x(t) at equal intervals T, to generate the sequence
{…, x(−kT),…, x(−T), x(0), x(T),..., x(kT), ...}

y(t)

x(t)

∞ ∞
y (t ) = ∑ x(kT ) ⋅ δ (t − kT ) = x(t ) ⋅ ∑ δ (t − kT )
k = −∞ k = −∞

102
Impulse sampling

103

x(n) is a discrete
time series

xp(t) is a
continuous time
signal

x(n) and xp(t)
contain the same
information

104
 Poisson formula

Fourier transform of PT(t)

105
Compare to a pervious result

We obtain the dual Poisson formula

106
 Time – Multiplying by an impulse train
Frequency – Convolution with an impulse train

107
Frequency relation between point and
impulse sampling

108
 Shannon reconstruction

x(t) is a narrowband
signal
and sampling is performed at a frequency
higher than Nyquist condition

It is possible to reconstruct the original x(t) from its samples by

Claude Elwood Shannon 1916-2001

109
 Proof
Only the main replica applies because we followed the Nyquist rule

110
 using DTFT

and since

QED
111
Sinc interpolation (non casual)

112
 General interpolation

Interpolation kernel

113
Condition on the interpolation kernel
Note that for Sinc interpolation kernel the
original sample values are kept after
reconstruction sum

114
Ideal reconstruction - frequency domain view

115
Other low pass filters

116
Zero--order hold
Zero

117
Zero order hold

118
ZOH – Frequency domain view

119
ZOH – Frequency domain view

120
ZOH – Frequency domain view
Ideal reconstruction filter

121
 Recommended homework -
Find (time and frequency) properties of the
First--order interpolation kernel
First

Casual ???
122
Discrete Fourier
Transform

123
 The DTFT

We limit ourselves to time limited signals n=0,1,..,N-1

124
Sampling in
frequency

125
DFT frequency resolution

Experiment length

126
 Periodic series
limited in time to

its periodic extension

Since the W-s are has period N

127
Properties

128
Properties

129
 DFT and DTFT

The DFT samples on period of
the discrete-
discrete-time Fourier
transform (DTFT) at N evenly
spaced frequencies fftshift()

130
 Matlab Example: f=0
f=0 (DC), N=32
N=32--point DFT
n=0:31;
n=0 31;
f=0
f=0.0;
x = cos(
cos(2*pi*n*f);
figure(1
figure( 1); stem(
stem(n,x
n,x);
);
figure(2
figure( 2); stem(
stem(n,abs
n,abs((fft
fft(x)));
(x))); % FFT is a fast algorithm to calculate DFT

x(n) |X(k)|
35
1

0.9
30
0.8
25
0.7

0.6
20

0.5
15
0.4

0.3
10

0.2
5
0.1

0 0
0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35

NOTE: Matlab indexes starting from 1 (i.e x(1) to x(N)) not 0 (i.e. x(0) to x(N-1))
131
 Matlab Example: f=0
f=0.25,
25, N=
N=32
32--point DFT
n=0:31;
n=0 31;
f=0
f=0.25;
25;
x = cos(
cos(2*pi*n*f);
figure(1
figure( 1); stem(
stem(n,x
n,x);
);
figure(2
figure( 2); stem(
stem(n,abs
n,abs((fft
fft(x)));
(x)));

x(n) |X(k)|

1 16

0.8 14

0.6
12
0.4
10
0.2

0 8

-0.2
6

-0.4
4
-0.6

2
-0.8

-1 0
0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35

132
DFT of Cosine

133
134
DFT of cosine
Integral number of
cosine periods

135
 Why we do not get “two deltas” ?
We calculate DFT on a limited time cosine
y(θ )
1

θ
-1 −θ0 θ0
0 5 10 15 20 25 30
1

0.5

0
0 5 10 15 20 25 30
*
1

-1
0 5 10 15 20 25 30

136
Why we do not get “two deltas” ?

137
Inverse DFT

138
Zero padding

139
 Zero padding
Improves DFT frequency inter-sampling spacing (“resolution”).
8
1
6
0.5 4
time
0 2
0 4 8 12 16 0
-0.5
0 1 2 3 4 5 bin 6
-1

88
1
66
0.5 44
time
time
0 22
0 10
3 206 309 401250 1560 18
70 21
80 90
24 100
27 110
30120 00
-0.5
00 12
3 24
6 36
9 bin
bin 48
12
-1

140
Discrete Fourier Transform (DFT)

The Discrete Fourier Transform (DFT) of a finite length sequence

N −1 j 2πnk
−
X [k ] = DFTN {x[n ]} = ∑ x[n ]e N
for k ∈ {0,..., N − 1}
n =0

The Inverse Discrete Fourier Transform (IDFT) is defined by

N −1 j 2πnk
1
x[n ] = IDFTN {X [k ]} = ∑ X [k ]e N
for n ∈ {0,..., N − 1}
N k =0

A DFT and IDFT pair is shown as

DFT
x[n ] ←
→ X [k ]
141
 DFT Transformation Matrix

The DFT can be represented as a symmetric matrix
1 1 1 1 1 1 
WN = e j 2π / N
1 W −1 W −2
W −3
... W −( N −1) 

FN =  N N N N 
WNi , j = WN( i −1)( j −1)
: : : : : : 
 − ( N −1) − 2 ( N −1) −3( N −1) − ( N −1) 2 
i, j : row and column indexes
1 WN WN WN ... WN 

This introduces the widely-

widely-used and convenient notations
N −1
 x[0]   X [ 0] 
X [k ] = ∑ x[n]W − nk
N ⇒ X = FN x  x[1]   X [1] 
n=0
x=  X= 
N −1  :   : 
1 −1    
x[n] =
N
∑ X
n =0
[ k ]
W nk
N ⇒ x = FN X  x[ N − 1]  X [ N − 1]

142
The DFT matrix is (almost) unitary

−1 1 T
FN = FN
N

143
 DFT Example

The DFT matrices of dimension 2, 3, 4 are as follows:

 
1  1 1 1 1
1 1
1 1   −1 − j 3 1 − j − 1 j 
F2 =  −1+ j 3 
1 − 1

F3 = 1  F4 =  
 2 2  1 − 1 1 − 1 
 −1 + j 3 −1− j 3   
1  − −
 2 2  1 j 1 j 

x = [1 3 0 − 2]T , 1 1 1 1   1   2   X [0]
1 − j − 1 j   3  1 − 5 j   X [1] 
X = F4 x =    =  = 
1 − 1 1 − 1   0   0   X [ 2]
X = Tx       
1 j − 1 − j −
  
2 1 + 5 j   X [3] 

Where we observe that the real part of X[k] is even-

even-symmetric, and the
imaginary part is odd-
odd-symmetric – the DFT of the real signal.
144
 DFT Computation Using
MATLAB

The functions to compute the DFT and the
IDFT are fft and ifft

These functions make use of Fast Fourier
Transform (FFT) algorithms which are
computationally highly efficient compared
to the direct computation

145
 Simple script file using FFT

%simple example of FFT use

%setting up the signal
t=0:0.001:0.6;
x=sin(2*pi*40*t)+sin(2*pi*100*t) %40 Hz and 100 Hz sine waves added
y=x+2*randn(size(t)); %noise added
%randn(size(t)) gives an array of normally distributed random entries
% that is the same size as t
plot(t(1:50),y(1:50)) %plots first 50 points i.e. to 0.05 s
title('40 Hz plus 100 Hz corrupted by noise')
xlabel('time (seconds)')
pause(2) %waits 2 seconds before continuing

146
Signal plus noise

147
 Simple script file using FFT pt2
pt2

%the fft and output

Y=fft(y,512); %performs the fft. Note 512 elements - power of 2 (2^9)
Pyy=abs(Y)/512;
%Y will have complex elements. The power spectrum is real
%so do an element by element multiplication by complex conjugate
f=1000*(0:256)/512;
plot(f,Pyy(1:257)) %we are only interested in half the data points
%as the same information is provided in the second half of Pyy
title('Frequency content of corrupted signal')
xlabel('Frequency (Hz)')

148
Frequency content

149
 DFT Properties
DFT DFT DFT

Let x[n ] ←→ X [k ], y[n ] ←→ Y [k ], h[n ] ←→ H [k ]
be length-
length-N sequences indexed n=0,…,N ,…,N-1.

DFT Properties:

Linearity: For constant a, b:
DFT
ax[n ] + by[n ] ←→ aX [k ] + Y [k ]

Real Sequences:
Sequences: If x[n] is real
real::

X [k ] = X [ N − k ] and ∠X [k ] = −∠X [ N − k ]

150
Cyclic time shift

151
 DFT Circular Shift Property

x1[n ] = x[(( n + 2)) N ]

152
Cyclic frequency shift

153
 DFT Properties

Duality: DFTN {X [k ]} = Nx[ N − n ]

Parseval’s Theorem N −1 N −1
2 2
∑ X [k ]
k =0
=N 2
∑ x[n]
n =0

154
 Cyclic Convolution
Signals of Length N, n=0,1,…,N-1

Periodic extension

155
 Cyclic Convolution

Why is cyclic convolution
not true linear convolution?

Because a wraparound
effect occurs at the “ends”:

The procedure of each pair
are summed around the
circle.

In a while, it will be seen
that y[n ] = x[n ] * h[n ]
can be computed using
N
y[n ] = x[n ] ⊗ h[n ].

156
Circulant matrix interpretation

157
Cyclic convolution and DFT

158
Cyclic convolution and DFT

159
 Cyclic Convolution Example
→ X 1[n] = WN− kn0
DFT
x1[n] = δ [n − n0 ] ←

X 3 [k ] = X 1[k ] X 2 [k ] = WN− kn0 X 2 [k ]

x3[n ] = x1[n ] ⊗ x2 [n ]
N −1
= ∑ x1[k ]x2 [(( n − k )) N ]
k =0

= x2 [(( n − n0 )) N ]

160
 Linear Convolution by DFT

Linear convolution is desired. The linear convolution can be
computed via DFT, with a minor modification.

Method: To compute the linear convolution
y[n ] = x[n ] * h[n ]
of a sequence x[n] of length-
length- N1 and a sequence h[n] of length-
length- N2
via the DFT, form the length N1 + N2 -1 zero-
zero-padded sequences

 x[n]; 0 ≤ n ≤ N1 − 1 h[n]; 0 ≤ n ≤ N2 −1
xa [ n ] =  ha [n] = 
 0 N1 ≤ n ≤ N1 + N 2 − 2  0 N 2 ≤ n ≤ N1 + N 2 − 2
N1 + N 2 −1
ya [n] = xa [n] ⊗ ha [n]

= x[n ] * h[n ]; 0 ≤ n ≤ N1 + N 2 − 2
161
Linear Convolution
Non zero
contributions at

162
Zero padding to length N=N1+N2-1

 x[n]; 0 ≤ n ≤ N1 − 1
xa [ n ] = 
 0 N1 ≤ n ≤ N1 + N 2 − 2

 y[n]; 0 ≤ n ≤ N2 −1
y a [ n] = 
 0 N 2 ≤ n ≤ N1 + N 2 − 2

163
Linear and Cyclic convolution

Non zero for

164
 Non zero regions

− N − n ≤ −m ≤ N 2 − 1 − N − n
N + n ≥ m ≥ N − N2 +1+ n
N + n ≥ m ≥ N1 + n

The second term is zero QED

165
 DFT of periodic signals*
x(t) with period T0 is samples at T=T0/N
(N samples per period)

* Tip – Topic the lecturer is very fond of. Look at exams from previous semesters. 166
DFT of periodic signals

167
 DFT of periodic signals (aliasing)

For narrowband signals and Nyquist

complying sampling

And the signal may be reconstructed from its DFT coefficients

168
FFT

169
 Top 10

Monte Carlo method or Metropolis algorithm, devised by John von Neumann,
Stanislaw Ulam,
Ulam, and Nicholas Metropolis;

simplex method of linear programming, developed by George Dantzig; Dantzig;

Krylov Subspace Iteration method, developed by Magnus Hestenes,
Hestenes, Eduard
Stiefel,, and Cornelius Lanczos;
Stiefel Lanczos;

Householder matrix decomposition, developed by Alston Householder;

Fortran compiler, developed by a team lead by John Backus;

QR algorithm for eigenvalue calculation, developed by J Francis;

Quicksort algorithm, developed by Anthony Hoare;

Fast Fourier Transform,
Transform, developed by James Cooley and John Tukey; Tukey;

Integer Relation Detection Algorithm, developed by Helaman Ferguson and
Rodney Forcade;
Forcade; (given N real values XI, is there a nontrivial set of integer
coefficients AI so that sum ( 1 <= I <= N ) AI * XI = 0?

Fast Multipole algorithm, developed by Leslie Greengard and Vladimir Rokhlin;Rokhlin; (to
calculate gravitational forces in an N-
N-body problem normally requires N^ N^2
2
calculations. The fast multipole method uses order N calculations, by
approximating the effects of groups of distant particles using multipole expansions)

170
Cooley and Tukey

171
 Performance of the DFT
Algorithm

The DFT requires N2 (NxN
NxN)) complex multiplications:

Each X(k) requires N complex multiplications.

Therefore to evaluate all the values of the DFT ( X(0
X(0) to X(N
X(N--1) )
N2 multiplications are required.

The DFT also requires (N-
(N-1)*N complex additions:

Each X(k) requires N-
N-1 additions.

Therefore to evaluate all the values of the DFT (N-
(N-1)*N additions
are required.

172
 DFT → FFT
N −1
X (k ) = ∑ x[n]WN− nk ; 0 ≤ k ≤ N − 1
n =0

x[n] = x[0
x[0], x[1
x[1], …, x[N-
x[N-1] we assume N even

Divide the sequence x[n] into even and odd
sequences:

x[2
x[2n] = x[
x[0
0], x[
x[2
2], …, x[N
x[N--2]

x[2
x[2n+1
n+1] = x[
x[1
1], x[
x[3
3], …, x[N
x[N--1]

173
Decimation-in-Time Factorization
N −1
− kn
X [k ] = ∑ x[n] ⋅ WN , k = 0,1,L, N − 1 and N even
n =0
N − pt . DFT

= ∑
n , even
+ ∑
n , odd
Note that : W22Nr = WNr

N / 2 −1 N / 2 −1
− 2 rk − ( 2 r +1) k
= ∑ x[2r ]W
r =0
N + ∑ x[2r + 1]W
r =0
N

N / 2 −1 N / 2 −1
− rk −k − rk
= ∑ x[2r ]WN / 2
r =0
+ WN ⋅ ∑ x[2r + 1]WN / 2
r =0
N / 2 − pt . DFT ,G ( k ) N / 2 − pt . DFT , H ( k )

174
 DFT → FFT

The result is that an N-
N-point DFT can be
divided into two N/2
N/2 point DFT’s:
N −1
X (k ) = ∑ x[n ]WN− nk ; 0 ≤ k ≤ N − 1 N-point DFT
n =0

Where Y(k) and Z(k) are the two N/2

N/2 point
DFTs operating on even and odd samples
respectively:
N N
−1 −1
2 2
X (k ) = ∑ x1 [n]W N− nk + WN− k ∑ x2 [n]W N− nk Two N/2
N/2-
n =0 2 n =0 2 point
= Y (k ) + WN− k Z (k ) DFTs
175
 DFT → FFT (k>N/2
(k>N/2)
N N
−1 −1
2 2
X (k ) = ∑ xe [n]W N− nk + WNk ∑ xo [n]W N−nk
n =0 2 n =0 2

M
N N
−1  N −1  N
 N 2 −n k + 
−k −
N 2 −n k + 
X  k +  = ∑ xe [n]W N
 2
+ WN 2
∑ x [n]W o N
 2

 2  n =0 2 n =0 2

N 2π 2π N 2π
−k − −j k −j −j k
− jπ
WN 2
=e N
e N 2
=e N
e = −WN− k
 N 2π 2π N 2π
− k +  −j k −j −j k
W N
 2
=e N 2
e N 22
=e N 2
= W N− k
2 2

176
 DFT → FFT

N N
−1 −1
 N 2 2
X  k +  = ∑ xe [n]W N− nk − WN− k ∑ xo [n]W N− nk
 2  n =0 2 n =0 2

= Y (k ) − WN− k Z (k )

177
 DFT → FFT
N 
X (k ) = Y (k ) + W Z (k ); k = 0,K  − 1
−k
N
2 
 N N 
X  k +  = Y (k ) − WN− k Z (k ); k = 0,K  − 1
 2 2 

Y(k) and WNk Z(k) only need to be calculated once and used for
both equations.

Note: the calculation is reduced from 0 to N
N--1 to 0 to (N/2
(N/2 - 1).

N=2L, Y(k) and Z(k) can also be divided into N/4
If N=2 N/4, N/
N/8
8,… point
DFTs using the same process shown above.

178
 DFT → FFT

The process continues until we reach 1 point DFTs
N=1
N= 1.

0
X [k = 0] = ∑ x(n )W = x(n )
d
1
0

n =0

179
Decimation--in-
Decimation in-time Radix-
Radix-2 FFT
(N=8
(N= 8)
Decimation--in-
Decimation in-time Radix-
Radix-2 FFT
(N=8
(N= 8)
Radix--2 DIT Basic Butterfly
Radix

a (complex) A (complex)

b (complex) B (complex)

W N (complex)
 FFT Implementation

To efficiently implement the FFT algorithm a
few observations are made:

Each stage has the same number of butterflies
(number of butterflies = N/2
N/2, N is number of
points).

The number of DFT groups per stage is equal to
(N/2
(N/ 2stage).

The difference between the upper and lower leg is
equal to 2stage
stage--1.

The number of butterflies in the group is equal to

2stage
stage--1.

Total of 1/2Nlog2N multiplications

183
 Twiddles
 2πk   2πk 
W = cos
k
N  + j sin  
Note that the twiddles  N   N 

are calculated off line and kept in a fixed table

Only the finest set of twiddles is needed.

The rest are taken from the W Nk = W N
2 k

fine table using 2

k 4 k
W N = W N
4

.
.
. 184
 Bit reversal

During the first recursion, we split the signal to odd
and even (according to the Least Significant Bit)

For every stage the LSB becomes the Most
Significant Bit

The order of samples at input should be changed
according to the bit reverse rule (reverse the
“address bus”)

185
Bit reversal

186
Spectral analysis

187
 Signal windowing

y(n) – discrete time signal

We time-
time-limit the signal by multiplying it with a time
limited “window” function – w(n), n=0
n=0,1,…,N
,…,N--1

x(n)=y(n)*w(n)

Example: Rectangle window

x(n ) = y (n )wr (n ) =
1 n = 0,1,..., N − 1
= y (n ) ⋅ 
0 otherwise
 y (n ) n = 0,1,..., N − 1
=
 0 otherwise
188
Frequency domain

x(n)=y(n).w(n)

189
Rectangular window

Dirichlet kernel

“Linear phase” – To be
revisited 190
Dirichlet kernel

191
 Dirichlet kernel

Maximum at

Zeros at

Width of main lobe is

Ratio
between main lobe and
highest side lobe is

192
Analysis of a pure cosine

y (θ )

θ
*
− θ0 θ0

193
 Width of main lobe

Width of main lobe defines the frequency
resolution.

Wide main lobe – poor resolution

Width is proportional to 1/N

194
195
Sampled
amplitudes
do not
perfectly
reflects true
amplitudes

196
Ratio between main and side lobes
– “energy leak”

Side lobes of high amplitude signal may “mask” weaker
signal if its side lobes are higher than the weak signal

The side lobes ratio does not change with N

E.g. Rectangle window may mask signals that are 13 13dB
dB
weaker (not so good…)

197
 Triangle window

Motivation – Higher ratio between main
lobe and side lobes

Method

In the frequency domain -Window with
Dirichlet2 response = 27dB
27dB main
main--side lobe
ratio

In the time domain - Rectangle Window *
Rectangle window = Triangle window

198
Triangle window – N odd
Two rectangle windows – Length (N+1)/2 each

About half compared with rectangle

window – Main lobe is about twice as
wide

199
 Triangle window – N even
Convolution between two
rectangle windows – Length
(N+1)/2 and N/2

About half compared with rectangle

window – Main lobe is twice as wide

200
Rectangle window

8π
Width of main lobe
N 201
 Raised cosine windows

Motivation - Obtain small side lobes

In the frequency domain – Linear
combination of the Dirichlet kernel and
modulated Dirichlet kernels

202
 Raised cosine windows
Rectangle
window

Rectangle window . Cosine

203
Hann Window

204
Hann Window

8π
Width of main lobe
N

205
Hamming Window

206
Hamming Window

8π
Width of main lobe
N

207
Blackman Window

208
Blackman Window

12π
Width of main lobe
N

209
 Kaiser window

A family of windows that depends on a
parameter α.

α continuously tradeoffs between the
width of main lobe and the main-
main-to
to--side
lobes ratio

As α increases the main-
main-to
to--side lobes
ratio gets larger but main lobe width also
increases

210
Kaiser window

211
Kaiser window

212
Kaiser window

213
Window shopping

214
 DFT - Window loss remedial
Smooth data-
data-tapering windows cause information loss near edges.

2 x N samples (input signal)

Solution:
sliding (overlapping) DFTs. DFT #1

• Attenuated inputs get next DFT #2

window’s full gain & leakage
reduced.
DFT #3
• Usually 50% or 75% overlap
(depends on main lobe width).

Drawback: increased
DFT AVERAGING
total processing time.

215
 Example 1 – Main lobe width
For most applications - X axis is plotted in Hz up to ½ sampling rate

Fs=1000;;
Fs=1000

Ts=1
Ts= 1/Fs;

t=0
t=0:Ts:1023
:Ts:1023*Ts;
*Ts;

Freqs=(
Freqs =(00:512
512)/)/512
512**500
500;;

x=cos
x= cos((2*pi*100
*pi*100*t)+
*t)+0
0.5*cos
cos((2*pi*
*pi*102
102*t)
*t) +
0.1*randn(size(t));
randn(size(t));

X=abs(fft
X=abs( fft(x));
(x));

plot(Freqs,20
plot(Freqs, 20*log
*log10
10(X(
(X(11:513
513)));
)));

xlabel('Freq
xlabel ('Freq (Hz)')

ylabel('Amplitude
ylabel ('Amplitude (dB)')
216
Example 1 – Main lobe width
60

50

40

30
Amplitude (dB)

20

10

-10

-20
0 50 100 150 200 250 300 350 400 450 500
Freq (Hz)
217
 Example 1a – Main lobe width

Fs=1000;;
Fs=1000

Ts=1
Ts= 1/Fs;

t=0
t=0:Ts:
:Ts:1023
1023*Ts;
*Ts;

Freqs=(0
Freqs=( 0:512
512)/
)/512
512**500
500;;

x=cos(2
x=cos( 2*pi*
*pi*100
100*t)+
*t)+0
0.5*cos(
*cos(22*pi*
*pi*102
102*t)+
*t)+00.1*randn(size(t));

x=x.*blackman(1024
x=x.*blackman( 1024)';
)';

X=abs(fft(x));

plot(Freqs,20
plot(Freqs, 20*log
*log10
10(X(
(X(1 1:513
513)));
)));

xlabel('Freq (Hz)')

ylabel('Amplitude (dB)')

218
Example 1a – Main lobe width
50

40

30
Amplitude (dB)

20

10

-10

-20
0 50 100 150 200 250 300 350 400 450 500
Freq (Hz)

219
 Example 2 – Main-
Main-to-
to-Side lobes
ratio

Fs=1000;;
Fs=1000

Ts=1
Ts= 1/Fs;

t=0
t=0:Ts:1023
:Ts:1023*Ts;
*Ts;

Freqs=(0
Freqs=( 0:512)/
512)/512
512**500
500;;

x=cos(2
x=cos( 2*pi*
*pi*100
100*t)
*t) + 0.002
002*cos(
*cos(22*pi*
*pi*150
150*t)
*t) +
0.001
001*randn(size(t));
*randn(size(t));

X=abs(fft(x));

plot(Freqs,20
plot(Freqs, 20*log
*log1010(X(
(X(11:513
513)));
)));

xlabel('Freq (Hz)')

ylabel('Amplitude (dB)')

220
Example 2 – Main-
Main-to-
to-Side lobes
ratio
60

50

40
Amplitude (dB)

30

20

10

-10
0 50 100 150 200 250 300 350 400 450 500
Freq (Hz)

221
 Example 2a – Main-
Main-to-
to-Side lobes
ratio

Fs=1000;;
Fs=1000

Ts=1
Ts= 1/Fs;

t=0
t= 0:Ts:
:Ts:1023
1023*Ts;
*Ts;

Freqs=(0
Freqs=( 0:512
512)/
)/512
512**500
500;;

x=cos(2
x=cos( 2*pi*
*pi*100
100*t)
*t) + 0.002
002*cos(
*cos(2
2*pi*
*pi*150
150*t)
*t) +
0.001
001*randn(size(t));
*randn(size(t));

x=x.*blackman(1024
x=x.*blackman( 1024)';
)';

X=abs(fft(x));

plot(Freqs,20
plot(Freqs, 20*log
*log1010(X(
(X(1 1:513)));
513)));

xlabel('Freq (Hz)')

ylabel('Amplitude (dB)')

222
Example 2a – Main-
Main-to-
to-Side lobes
ratio
50

0
Amplitude (dB)

-50

-100
0 50 100 150 200 250 300 350 400 450 500
Freq (Hz)

223
Filter design

224
 RCSR – Real coefficients,
Causal, Stable, Rational)
Continuous time filter

225
Region of Convergence

Re( s ) > −α
jω
complex plane
ROC
pole
x
-α σ

226
 Poles and Zeros
s −1
X ( s) = Re( s ) > −1
( s + 2)( s + 1)
jω
complex plane
ROC
poles
x x o
-2 -1 1 σ

zero

227
 For distinct poles

For non trivial systems, the impulse response is infinite in time

228
Discrete time

229
 Example: Region of
Convergence

∞ ∞
| X ( z ) |= ∑ x (
n = −∞
n ) z −n
= ∑ | x
n = −∞
( n ) || z | −n
<∞

Im
ROC is an annual ring centered on the
origin.

r
RI <| z |< RO
Re
ROC = {z = re jθ | RI < r < RO }
 Stable Systems

A stable system requires that its Fourier

transform is uniformly convergent.

Im
Fourier transform is to evaluate z-

transform on a unit circle.

A stable system requires the ROC of z-
1 transform to include the unit circle.

Re
 Causal Signal
x[n] = a nu[n] ∞
X ( z ) = ∑ az( ) −1 n
=
1
z >a
x[n] 1 a a2 a3 a4 a5 … n =0 1 − az −1

z
n X ( z) = z >a unit circle
z−a

zero pole

o x
a 1

ROC
232
complex z-plane
 Anti--causal Signal
Anti
x[n] = −a nu[−n − 1]
∞ −1
X ( z ) = − ∑ a u (−n − 1) zn −n
= − ∑ a n z −n
n = −∞ n = −∞
∞ ∞
( )
= −∑ a z = 1 − ∑ a −1 z
−1 n
( ) n

n =1 n =0

1
= 1− z<a
1 − a −1 z
z unit circle
X ( z) = z<a
z−a

zero pole
ROC
o x
1 a

233
complex z-plane
For distinct poles

Stable if

Note: The C-coefficients part of the unit sample response is

finite in time. The part due to poles is infinite.

234
 FIR and IIR systems

A discrete system is said to be an
FIR system if its impulse response has
zero--valued samples for n > M > 0
zero

Integer number M is called the
order of the impulse response

IIR system is a discrete system with an
infinite impulse response

FIR = Finite Impulse Response
IIR = Infinite Impulse Response
235
 Representations of discrete time
systems
Z-domain

236
 Difference equation
−1 −q

H z (z ) =
Y ( z ) =
b0 + b1 z + ... + bq z
X ( z ) 1 + a1 z −1 + ... + a p z − p
[ ] [
Y ( z ) ⋅ 1 + a1 z −1 + ... + a p z − p = X ( z ) ⋅ b0 + b1 z −1 + ... + bq z − q ]

Recall that x(n)
X(z) x(n-1)
z-1X(z)

y (n ) + a1 y (n − 1) + ... + a p y (n − p ) =
= b0 x(n ) + b1 x(n − 1) + ... + bq x(n − q )

237
 Difference equation building blocks

y (n ) = − a1 y (n − 1) − ... − a p y (n − p ) +
+ b0 x(n ) + b1 x(n − 1) + ... + bq x(n − q )

Delay

Multiplier

Adder

238
FIR

239
IIR

240
Filter types

241
Specifications of LPF

Transition band
Pass
band
Stop band

242
 Pass band

Nominal gain is 1

Pass band ripple

In dB

Approximation

243
 Transition band

No specific requirements

Response is expected to be monotonic

Important: we shall see that the complexity
of the filter depends heavily on the width
of the transition band

244
Stop band attenuation (ripple)

245
arctan2
arctan2 – arctangent function for all
four quadrants
arctan(y/x) is limited to first two quadrants (sign ambiguity)

246
Frequency response of RCSR
systems

247
 Continuity
For RCSR systems is continuous on the unit circle.

is continuous and the phase is continuous

except for two cases:

1. Function crosses the negative real line

2. Zero amplitude

248
 Case 1 - Function crosses the
negative real line

2π
phase
jump

249
 Example – y(n)=x(n
y(n)=x(n--8)-x(n-
x(n-10)
10)

2 4

1.8
3
1.6
2
1.4
1
1.2
Amplitude

Phase
1 0

0.8
-1
0.6
-2
0.4
-3
0.2

0 -4
0 0.5 1 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5
Freq Freq

2π jumps
250
 can be zero only for a finite
number of points

The numerator (‫ )מונה‬may be zero only at a finite number of points

251
 Case 2 - Zero amplitude

Phase
undefined
at z=0

(Algebra) H(z) = 0 only for a finite number of points – order of

numerator (‫)מונה‬ 252
 Example – y(n)=
x(n)+x(n--1)+…+x(n-
x(n)+x(n )+…+x(n-9)

10 1.5

9 1

8
0.5

7
0
6
Amplitude

-0.5

Phase
5
-1
4
-1.5
3

-2
2

1 -2.5

0 -3
0 0.5 1 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5
Freq Freq

Phase jump is due to sign (+/-) change π jumps

253
Phase jumps can be either π or 2π

Case 1 – always 2π

Case 2 - is zero if
has a zero with multiplicity m on
the unit circle at

254
 Case 2 - continued

Continuous phase
Continuous near
π jump if m odd

255
 Continuous phase representation

We shall allow phase to be any real number
[–π,π)
and not limited to [– π),, to overcome 2π
jumps

We shall define an Amplitude function
instead of absolute value. The new Amplitude
function will allow negative values, to
overcome π jumps

Amplitude – real (may be Continuous phase

negative) function 256
Conversion between Absolute/Phase to
Amplitude/Continuous -Phase

Startat θ=–π and move toward θ
θ= =π
θ=

At each phase discontinuity:

Case 1 – Phase - Add/Subtract 2π

Case 2 – Phase - Add/Subtract π, Multiply
amplitude by -1

257
Example

258
 Formal math.
Discontinuity points

259

At the discontinuity points define as limit

 Remarks

Representation is not unique

Add/Subtract 2π

Add/Subtract π and multiply amplitude by -1

In order to be unique, force

260
Linear phase

2.261
 Delay (number of samples - integer)

Pure delay
X(n) y(n)=x(n-L)
No distortion

Linear phase

262
Delayed sinc L=5
L=5

263
 Non integer delay

X(n) ?

− jθ ( L + δ )
Y (θ ) = X (θ ) ⋅ e

Integer fraction

264
 Non integer delay

1 m = n − L
for δ = 0, sinc(n − m − L ) = 
0 otherwise 265
 Non integer delay
Assume signal is sampled at T=1
∞
y (n ) = ∑ x(m)sinc[t − m]
m = −∞ t = n − L −δ

x(t) x(t − L − δ )
Ideal sinc Continuous Sample
x(n) y(n)
reconstruction time delay

266
Delayed sinc 4.5 samples

267
 Linear phase filters
Continuous phase representation

Phase delay (Unit is

samples)

Linear phase filter has a constant phase delay

Note: The phase properties are relevant mainly to the pass band of the
filter.

268
Generalized Linear phase
During the last section we discussed Linear Phase systems with the form

− jθτ p
H (θ ) = e
We shall now try to generalize this to systems with initial non zero phase

?
jφ 0 − jθτ g
H (θ ) = e
For RCSR systems, the phase is anti-symmetric (phase at DC is zero)

 e jφ0 − jθτ g θ >0

?
− jφ 0 − jθτ g
H (θ ) = e θ <0
 0 θ =0
 269
We relax the requirements to regions where the signal exists

 e jφ0 − jθτ g θ1 < θ < θ 2

 − jφ0 − jθτ g
H (θ ) = e − θ 2 < θ < −θ1
 0 otherwise

Typical case for GLP is a modulated signal

Envelope carrier

we assume that the signal bandwidth is within θ1 < θ < θ 2

270
The modulated signal is within filter
limits

271
Modulation property

x(n ) ↔ X f (θ )
1 f
[
x(n ) ⋅ cos(θ c ⋅ n ) ↔ X (θ − θ c ) + X f (θ + θ c )
2
]
The output

1
[
Y (θ ) = H (θ ) X f (θ − θ c ) + X f (θ + θ c ) =
f

2
]
Positive frequencies negative frequencies

1 f jφ 0 − jθτ g 1 f − jφ 0 − jθτ g
= X (θ − θ c ) ⋅ e + X (θ + θ c ) ⋅ e
2 2

272
 1 f jφ 0 − jθτ g 1 f − jφ 0 − jθτ g
Y (θ ) = X (θ − θ c ) ⋅ e
f
+ X (θ + θ c ) ⋅ e =
2 2

1 f jφ 0 + jθ cτ g − jθ cτ g − jθτ g
= X (θ − θ c ) ⋅ e
2
1 f − jφ 0 + jθ cτ g − jθ cτ g − jθτ g
+ X (θ + θ c ) ⋅ e =
2

1 f − j (θ −θ c )τ g jφ 0 − jθ cτ g
= X (θ − θ c ) ⋅ e ⋅e
2 Constants

1 f − j (θ +θ c )τ g − jφ 0 + jθ cτ g
+ X (θ + θ c ) ⋅ e ⋅e
2

273
 Linear phase delay
x(n ) ↔ X f (θ )
of envelope
define
− jθτ g
v(n ) ↔ X (θ ) ⋅ e
f

Delayed cosine

1 jφ0 − jθ cτ g
cos[θ c ⋅ (n − τ g ) + φ0 ] ↔ e
2
[ ⋅ δ (θ − θ c ) + e
− jφ 0 + jθ cτ g
⋅ δ (θ + θ c ) ]
Convolution in frequency

v(n ) ⋅ cos(θ c ⋅ (n − τ g ) + φ0 ) ↔
1 f − j (θ −θ c )τ g jφ 0 − jθ cτ g
X (θ − θ c ) ⋅ e ⋅e +
2
1 f
Envelope is delayed by τg
− j (θ +θ c )τ g − jφ 0 + jθ cτ g
+ X (θ + θ c )e ⋅e
2
274
The constant phase affects the
carrier but not the envelope

where

275
 GLP -Generalized linear phase

Group delay

For modulated signals -the envelope is delayed by [Group delay] samples

276
Group delay is L or L+0
L+0.5
Is periodic

Real Real

Must be integer
277
Group delay is L/L+0
L/L+0.5

278
h(n) is
real

Real

279
Four types of GLP filters

280
The impulse response of Types I,II is symmetric

281
The impulse response of Types I,II is symmetric

Type I

Type II

Casual GLP filters may only be FIR

282
 The impulse response of Types III,IV is
anti symmetric

Type III

Type IV

Casual GLP filters may only be FIR

283
Minimum phase

Im(z)

Re(z)

284
Same absolute value

285
 Minimum--Phase System
Minimum

The new system has same absolute value, but with
mirrored zero

Theorem: The system with zero inside unit circle has
lower group delay

A system with all poles and zeros inside the unit circle is
called “minimum-
“minimum-phase”. Both the system function and
the inverse is causal and stable

A system with all poles inside the unit circle and zeros
outside is called “maximum-
“maximum-phase”. No inverse system.

Given an RCSR filter, we may build a minimum phase
RCSR filter with the same absolute value.

286
 All-
All -
Using mirrored pole and zero
Pass filter

In general

287
 Minimum-phase and All-
Minimum- All-pass
decomposition

Zero in unit circle
MP

Zero outside

Add mirrored pole
Zero and pole to AP

Add mirrored zero
MP

288
 Example

Consider the following system 1 + 3z −1
H1 (z ) =
1
1 + z −1
2

One pole inside the unit circle:

Make part of minimum-
minimum-phase system

One zero outside the unit circle:

Add an all-
all-pass system to reflect this zero inside the unit circle

1 −1
1− z
1 + 3z −1 1  −1 1  1  −1 1 3
H1 (z ) = =3 z +  = 3 z + 
1 −1 1 −1  3 1 1 −1
1 + z −1 
3
1+ z 1+ z 1− z
2 2 2 3
 1 −1  −1 1 
 1 − z  z + 
H1 (z ) =  3 3  3  = H (z )H (z )
min ap
 1 + z −1  1 − z −1 
1 1
  
 2  3 
289
 Frequency-Response
Frequency-
Compensation

In some applications a signal is distorted by an LTI system

Could filter with inverse filter to recover input signal

Would work only with minimum-
minimum-phase systems

Make use of minimum-
minimum-phase all-
all-pass decomposition

Invert minimum phase part

Assume a distorting system Hd(z)

Decompose it into H z d ( ) = Hd,min (z)Hd,ap (z)

Define compensating system as 1
Hc (z ) =
Hd,min (z )

Cascade of the distorting system and compensating system

1
G(z ) = Hc (z )Hd (z ) = Hd,min (z )Hd,ap (z ) = Hd,ap (z )
Hd,min (z )

290
 FIR Filters

N – filter order
Total of N+1 non-zero coefficients

291
 y (n) = h0 x(n) + h1 x(n − 1) + ... + hM x(n − M )
Direct implementation

Dual implementation

292
 Example 1: Average of two samples
1/2

h(0) = 12 , h(1) = 12 ; 0 1

y (n) = h(0) x(n) + h(1) x(n − 1)

y (n) = 12 x(n) + 12 x(n − 1)

293
 Example 1: Average

h(0) = 12 , h(1) = 12 ;
Equation: y ( n) = 1
2
(x(n) + x(n − 1) )
Transfer function: H ( z ) = 12 + 12 z −1
j θ2 − j θ2 − j θ2 − i θ2
jθ 1
H (e ) = (1 + e
2
− jθ 1
) = (e + e
2 )e = cos( )e
θ
2

294
 Example 2: Difference of two samples
1

h(0) = 1, h(1) = −1;

y (n) = h(0) x(n) + h(1) x(n − 1)
y (n) = x( n) − x( n − 1) −1

295
 Example 2: Differentiator

Equation: y (n) = x( n) − x(n − 1)

h(0) = 1, h(1) = −1;
−1
Transfer function: H ( z) = 1 − z

j θ2 − j θ2 − j θ2 − j θ2 + j π2
jθ
H (e ) = 1 − e − jθ
= (e − e )e = 2 sin ( )e
θ
2

296
 Type I FIR Linear-
Linear-Phase System

Even order

Symmetric

Zero initial phase

297
Frequency response – type I

298
Frequency response – type I
Linear phase amplitude

299
 Type I
Symmetric impulse response Amplitude symmetric with period is 2π

300
 Type II FIR Linear-
Linear-Phase System

Odd order

Symmetric

Zero initial phase

301
Frequency response – type II

302
Frequency response – type II
Linear phase amplitude

Cannot be HP

Relations between h(n) and g(n) may be found in the book

303
 Type II
Symmetric impulse response Amplitude symmetric with period is 4π

304
 Type III FIR Linear-
Linear-Phase
System

Even order
Anti-symmetric
Initial phase

305
Frequency response – type III
Linear phase amplitude

Mainly used for Hilbert filters /

Differentiators
306
 Type III
Anti symmetric impulse response Amplitude anti symmetric, period is 2π

307
 Type IV FIR Linear-
Linear-Phase
System

Odd order
Anti-symmetric
Initial phase
308
Frequency response – type IV
Linear phase amplitude

Mainly used for Hilbert filters /

Differentiators
309
 Type IV
Anti symmetric impulse response Amplitude anti symmetric, period is 4π

310
Summary of the four types

311
 Summary

Less constraints when using types I,IV 312

Location of Zeros for Symmetric
Cases

So if z0 is a zero 1/z0 is also a zero of the system

If h[n] is real and z0 is a zero z0* is also a zero

313
Typical Zero Locations

314
Linear phase Low pass zeros map

2.315
Relation of FIR Linear Phase to
Minimum--Phase
Minimum

In general a linear-
linear-phase FIR system is not minimum
minimum--phase

We can always write a linear-
linear-phase FIR system as

H(z ) = Hmin (z )Huc (z )Hmax (z )

Where

( )
Hmax (z ) = Hmin z −1 z −Mi

And Mi is the number of zeros

Hmin(z) covers all zeros inside the unit circle

Huc(z) covers all zeros on the unit circle

Hmax(z) covers all zeros outside the unit circle

316
 FIR filter design

IRT - Impulse response truncation

IRT 2 –Windows

Equi
Equi--ripple filters

317
 IRT

Given filter specifications, select filter type
and order (group delay is half of the order)

Specify ideal filter response (including linear
phase)

Perform inverse DTFT

Truncate response

Verify design

318
Band pass filter – Type I,II

319
 Special cases

HP only for type I

320
Example

321
Example

322
 Déjà vu

Both examples presented the familiar
Gibbs ripple due to limitation in time

We met that before when we discussed
spectral analysis using a rectangle
windows

Soon we shall try other windows

323
 Differentiator
Continuous time

Limited bandwidth discrete time

approximation

Ideal filter

We shall see that type III will not provide good results due to its zero
at π
324
Differentiator

325
Differentiator

326
Differentiator

327
 Hilbert filter
Very useful with bandpass applications

For example, ultrasound signal processing

Benefits:

Mathematical basis for representing bandpass signals

Easy determination for signal envelope

Half bandwidth - May reduce ADC sampling rates

328
 Hilbert filter 1
xˆ (t ) = x(t ) ∗

Filter impulse response is 1/πt πt
x(τ )
∞
Frequency domain Hilbert transfer 1
∫−∞t − τ dτ

function: =
π

− j , when f > 0
  +90
H ( f ) = − j sgn ( f ) = + j , when f < 0
 0, when f = 0  f
  -90
The signal and its Hilbert transform are
orthogonal ∞
Phase response

∫ x(t )xˆ (t )dt = 0

−∞ 329
 Bandpass Signals

Bandpass signals

Let z(t) be a bandpass signal
centered around some f0f0

z(t) can be expressed as

z (t ) = x(t ) cos(2π f ot ) − y (t )sin (2π f o t )

 In this expression, x(t) and y(t) are
lowpass. z(t) can be written as

z (t ) = a (t ) cos(2π f o t + θ (t ))


where a(t ) = + x 2 (t ) + y 2 (t )
and

θ (t ) = sin −1 ( y (t ) / x(t ))

330
 Analytic signal

Upper graph shows an FFT of the
original signal, x(t).

Analytic signal in the frequency

domain (Single Side Band):

x+ (t ) = x(t ) + j xˆ (t )

2 Z ( f ), when f > 0
 
Z + ( f ) = Z ( f ) + j[− j sgn ( f )]Z ( f ) =  Z (0 ), when f = 0 
 0, when f < 0 
 

331
 Pre--envelope
Pre

Baseband signal: g(t ) = m(t ) cos(2π fc t + θ )

G(f)

G+(f)

332
 Complex Envelope Summary

Complex/pre envelope are related

G+(f)
− j 2πfc t
g˜ (t ) = g+ (t)e
or
Gˆ ( f )
j 2πfc t
g+ (t) = g˜ (t )e

333
Signal

334
Signal + Hilbert

335
Envelope

336
 Discrete time Hilbert filter
Continuous time
(almost “All Pass”)

Discrete time approximation

337
Hilbert filter impulse response

338
Hilbert filter

339
Hilbert filter

340
 L2 Optimality of IRT

Desired Actual

FIR – h(n) is non-zero for n=0,..,N

Time domain (Parseval)

Independent of FIR Optimal when

hd(n)=h(n)
IRT 341
 L2 Optimality of IRT

The IRT is optimal in L2, mean square
error

The integral properties of L2 results in
Gibbs phenomena – 10 10%% error with
smaller and small width as N increases

We shall learn two solutions:

Windows to reduce ripple

Min max equi-
equi-ripple filters

342
Example

343
 Windows

The convolution between the Dirichlet kernel
and the ideal filter response results in the Gibbs
ripple

We already know other windows with better
performance in the frequency domain

Instead of simple truncation, use Window

Window length will be filter order

All windows we know are symmetric, will keep original
symmetric / anti-
anti-symmetric properties

344
Windowing in Frequency Domain
π
1
( )
H e jθ =
2π ∫ H d ( ) (
e jλ
W e )
j (θ − λ )
dλ
−π

345
 Windows

The width of window’s main lobe equals
the width of the transition band. Getting
smaller with N

Ripple is the same for pass and stop band

The ripple is a depends on the window
type and its side lobes.

Unlike DFT, relation is indirect to window’s
side lobes (there is a convolution here).

Ripple figures are different from DFT.

346
 Windowed IRT
 Given filter specifications,
select filter type and order
(group delay is half of the
order)
 Specify ideal filter
response (including linear
phase)
 Perform inverse DTFT
 Truncate h(n) by a
window w(n) h(n ) = hd (n )w(n )
 Verify design

347
Effect of windowing

Gibbs

348
Example:FIR Filter Design by Windowing Method
20 * log10 H ( e jω ) N=55
[dB]

Bartlett Window
Rectangular Window
Hamming Window
ω
349
Example:FIR Filter Design by Windowing Method
20 * log10 H ( e jω ) [dB] Rectangular Window

Kaiser Window: alfa=3

Kaiser Window: alfa=10

Kaiser Window: alfa=15

Kaiser Window: alfa=30

ω 350

Matlab code
close all;
clear all;

fc = 8000/44100; % cut-off frequency

N = 133; % number of taps
n = -((N-1)/2):((N-1)/2);
n = n+(n==0)*eps; % avoiding division by zero

[h] = sin(n*2*pi*fc)./(n*pi); % generate sequence of ideal coefficients

[w] = 0.54 + 0.46*cos(2*pi*n/N); % generate window function
d = h.*w; % window the ideal coefficients

[g,f] = freqz(d,1,512,44100); % transform into frequency domain for plotting

figure(1)
plot(f,20*log10(abs(g))); % plot transfer function
axis([0 2*10^4 -70 10]);

figure(2);
stem(d); % plot coefficient values
xlabel('Coefficient number');
ylabel ('Value');
title('Truncated Impulse Response');

figure(3)
freqz(d,1,512,44100); % use freqz to plot magnitude and phase response
axis([0 2*10^4 -70 10]);
351
 Truncated Impulse Response
0.4

Window 0.3

Method 0.2

Value
0.1

-0.1
0 20 40 60 80 100 120 140
Coefficient number

Magnitude (dB)
-20

-40

-60

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Frequency (Hz) 4
x 10
0
Phase (degrees)

-2000

-4000

-6000
0 0.5 1 1.5 2
Frequency (Hz) 4
x 10

2.352
Windows

Figures are different compared to DFT

2.353
Empirical selection of α an N for
Kaiser Window
Define A

2.354
 Example: Kaiser Window Design of a Lowpass Filter

Specifications ωp = 0.4π, ωp = 0.6π, δ1 = 0.01, δ2 = 0.001

Window design methods assume δ1 = δ2 = 0.001

Determine cut-
cut-off frequency

Due to the symmetry we can choose it to be ωc = 0.5π

Compute
∆ω = ωs − ωp = 0.2π A = −20 log10 δ = 60

And Kaiser window parameters
α = 5.653 M = 37

Then the impulse response is given as
  2
  n − 18 . 5  
I0 5.653 1 −  

h[n] =  sin[0.5π(n − 18.5)]    
18 . 5
 0≤n≤M
 π(n − 18.5) I0 (5.653)

 0 else

355
Approximation Error

356
 Min-Max FIR filter design
Min-
Parks--McClellan algorithm
Parks
Remez exchange

jθ
D (e ) desired frequency response
jθ
H (e ) actual frequency response

357
L2 (RMS of yellow area) vs. Minmax metrics

D(θ )
H (θ ) θp

Minmax

θ
π

θS
358
 Computer--Aided Design of Digital Filters
Computer

Objective - Determine iteratively the
coefficients of H(z), minimizing the
difference between D(e jθ ) and H (e jθ )
over closed subintervals of 0 ≤ θ ≤ π

This difference usually specified as a
weighted error function
E(θ ) = W (e jθ )[ H (e jθ ) − D(e jθ )]
where W (e jθ ) is user-
user-specified weighting
function. Regions with relatively higher
weighting values will result in smaller
relative error.
359
 Weighting function

The higher (relative) weighing function will result in (relative) lower ripple
360
 Computer-Aided Design of
Computer-
Digital Filters

Chebyshev or minimax criterion:
Minimizes the peak absolute value of the
weighted error: ε = max E(θ )
θ ∈Θ

where Θ is a set of disjoint frequency bands

in the range 0 ≤ θ ≤ π , on which D(e jθ ) is
defined

For example, for a lowpass filter design, Θ
is the disjoint union of [0,θ p ] and [θ s , π ]
361
 Design of Equiripple
Linear--Phase FIR Filters
Linear

The linear--phase FIR filter obtained by
linear
minimizing the peak absolute value of
ε = max E(θ )
θ ∈Θ

is usually called the equiripple FIR filter

After ε is minimized, the weighted error
function E(θ) exhibits an equiripple
behavior in the frequency range Θ
362
 Design of Equiripple
Linear--Phase FIR Filters
Linear

The general form of frequency response of a
causal linear-
linear-phase FIR filter of order N:

363
Optimization problem

364
 Alternation Theorem
Sufficient and Necessary condition to optimal solution.
The function

Has K+2 alternate extreme points:

Minimum (-δ), Maximum (+δ), Minimum (-δ), Maximum (+δ), etc.

365
 Equiripple example
1 + δ1
1 − δ1 K =7

δ2 ω
−δ2 0 ωp ωs π

δ2
0 ω
−δ2
ω p ωs
B1 K+2=9
B2
extremal points
366
 Alternation Theorem
K+2 alternate extreme points:
Minimum (-δ), Maximum (+δ), Minimum (-δ), Maximum (+δ), etc.

367
 Iterative approximation
1. Guess initial extreme point set

2. We have K+2 linear equations, one for each θi and K+2

unknowns gk and δ

3. After solving for gk and δ, sample in θ the left-hand side of the equation and
approximate the extreme point. If the extreme points are close to the optimality
condition – stop. Otherwise

4. Update

and goto 2
368
Remez Exchange Algorithm

369
 Equiripple FIR Digital Filter
Design Using MATLAB

Example - Design a linear-
linear-phase FIR
bandpass filter of order 26 with a
passband from 0.3 to 0.5, and
stopbands from 0 to 0.25 and from 0.55
to 1

The pertinent input data here are
N = 26
fpts = [0
[0 0.
0.25 0.
0.3 0.
0.5 0.
0.55 1]
1]
mag = [0[0 0 1 1 0 0]0]
wt = [1
[1 1 1 1]]
370

Computed gain response shown below
where Ap = 1 dB, As = 18.7 dB

N = 26, weight ratio = 1

0
Gain, dB

-20

-40

-60
0 0.2 0.4 0.6 0.8 1
ω /π
371

We redesign the filter with order
increased to 110

Computed gain response shown below
where Ap = 0.024 dB, As = 51.2 dB

Note: Increase in N = 110, weight ratio = 1

order improves 0

gain response at the -20

Gain, dB
expense of increased -40
computational -60

complexity -80
0 0.2 0.4 0.6 0.8 1
372
ω /π

As can be improved at the expenses of a
larger Ap by decreasing the relative passbad
weight ratio W (θ ) N = 110, weight ratio = 1/10

Gain response of 0
bandpass filter of
-20
order 110 obtained

Gain, dB
with a weight vector -40
[1 0.0.1 1]
1] -60

Now Ap = 0.076 dB, -80

0 0.2 0.4 0.6 0.8 1

A = 60.86 dB ω /π
s

Note: For Equiripple filters,

higher N results in smaller ripple
373

Plots of absolute error for 1st design

Absolute error has
same peak value in N = 26, weight ratio = 1
0.2
all bands
Absolute Error 0.1

-0.1

-0.2
0 0.2 0.4 0.6 0.8 1
ω /π
374
 Absolute error in 2nd design is Absolute error in passband of
uniform 3rd design is 10 times the
error in the stopband

-3 N = 110, weight ratio = 1 N = 110, weight ratio = 1/10

x 10
4 0.01

2 0.005
Absolute Error

0 0

-2 -0.005

-4 -0.01
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
ω /π ω /π
375
 Estimating filter order N
Equi-ripple filter order can be estimated by

Transition band (Hz) Sampling Rate (Hz)

376
 IIR Filter Design
Pros

low-order filters can have sharp frequency response
low-

low computational cost

Low phase delay

Cons

Design more difficult

Stability not trivial as FIR

Phase response not easily controlled
(e.g. no casual linear-
linear-phase IIR filters)

Feedback - coefficient sensitivity, quantization noise, etc. can
be a problem
377
 IIR filters
Rational transfer function :

B ( z ) b0 z N + b1 z N −1 + ... + bN b0 + b1 z − 1 + ... + bN z − N
H ( z) = = N N −1
=
A( z ) z + a1 z + ... + a N 1 + a1 z −1 + ... + a N z − N
N poles (zeros of A(z)) , N zeros (zeros of B(z))

infinitely long impulse response

stable iff poles lie inside the unit circle

corresponds to difference equation
y[ k ] + a1 . y[ k − 1] + ... + a N . y[ k − N ] = b0 .u[ k ] + b1 .u[ k − 1] + ... + bN .u[ k − N ]
y[ k ] = b0 .u[ k ] + b1.u[ k − 1] + ... + bN .u[ k − N ]− a1. y[ k − 1] − ... − a N . y[ k − N ]
144444424444443 1444442444443
`MA ' ` AR '

`ARMA’ (autoregressive-
(autoregressive-moving average)
378
 IIR Filter design

We shall study IIR design by conversion of
analog filters to digital filters

Steps

Design specs in the digital domain

Convert specs to analog domain

Design analog filter

Convert analog filter to digital filter

Verify design

379
 Analog filter to digital filter
conversion

Impulse invariance

Step invariance

Bilinear transform

380
 Popular analog filters
Butterworth All pole
No ripples
Maximally flat
Chebyshev I All pole
Pass band ripples
Sharper than Butterworth
Chebyshev II Both poles and zeros
Stop band ripples
Sharper than Butterworth
Elliptic Both poles and zeros
Both pass and stop band ripple
Sharper than Chebyshev
Bessel All pole
No ripples
Close to linear phase
381
Analog Filter frequency response

Trade off between low ripples and sharp transition band

382
 Time domain response

Trade off between low overshoot and sharp transition band

383
 Butterworth Filters

The amplitude |H(j
|H(jωω)| of an nth order Butterworth filter is
given by

Observations:
|H(j0)| is unity

The DC gain |H(j

At ω=ωc the gain is |H(

|H(jjωc)|=
|=00.707 or -3 dB

Monotonic

Asymptotic 2020nn dB/decade

ωc is called the half
half--power frequency
frequency,, or the 3 dB
dB--
cutoff frequency

384
 Normalized Butterworth Filters

We prefer to work with a normalized filter whose cutoff
frequency is 1 rad /sec (ω
(ωc =1), i.e.

Amplitude response of a Butterworth filter is maximally

flat at DC (the first 2n-1 derivatives of |H(j
|H(jωω)| are zero at
ω=0)

The filter gain is 1 (0 dB) at ω=0 and 0.707 (-3 dB) at
ω=1 for all n

For large n, the amplitude response approaches the
ideal characteristic

385
Normalized Butterworth Filters

386
Normalized Butterworth Filters

387
Stop band Low-
Low-Pass Butterworth
Filter Attenuation

388
 Poles location
Butterworth transfer function

Define h(s)

h(s) has 2n poles, evenly spaced on the unit circle (Note: Analog filter – unit
circle has nothing to do with stability)

389
n - even
k=0,1,..,2n-1

n - odd
Pk=n=-1 for odd n

We assign H(s) with the stable poles (left to imaginary axis)

H(-s) is assigned with non stable poles

390
 n=1
n=1 Butterworth Filters
A first order continuous time filter frequency response
is given by
1
H (ω ) = a>0
jω + a
jω complex plane

ROC
pole
x
-a σ

391
 n=2
n=2 2± j 2
1

0.8

X
0.6

0.4

0.2

-0.2

-0.4

-0.6

X
-0.8

-1
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
392
 n=3
n=3
h(s)=

Left side poles

393
 n=3

 H(s) only uses the n poles in the left half plane for
stability.

394
 Butterworth Filter
|H(s)| for n=4
n=4

H(s) = 1/( s4 + 2.6131s3 + 3.4142s2 + 2.6131s + 1)

395
n=10
n=10

396
 Selection of filter order
Select ωc and n to comply with specs.

397
Pass band

Stop band

Solution

where

Select ωc to comply with pass band or stop band inequalities 398

 Normalized Butterworth table

For larger order – search literature

399
 Normalized Butterworth poles

For larger order – search literature

400
 Chebyshev polynomials
Define Nth order Chebyshev polynomial

Continuous
function

Example

T2 ( x ) = cos(2 ⋅ arccos( x )) = 2 cos 2 (arccos( x )) − 1 =

= 2x2 −1
401
402
 Recursive formula

For |x|<1 polynomial has equi-ripples between -1 and +1

Number of ripples increases with N

For |x|>1 |TN(x)| monotonically increases with |x|

Even/Odd N – TN(x) has only even/Odd powers of x

|TN(0)|=1 for even N

TN(0)=0 for odd N

|TN(-,1+1)|=1

403
404
405
Chebyshev type I (Pass band ripple)
ε- parameter
Large ε results in larger
ripples and faster roll off

Pass-band properties

406
407
20N
dB/Decade

408
Poles layout (ellipse)

409
Chebyshev type II (stop band ripple)

Note:
Frequency
axis reversed

410
411
Elliptic filter (Pass and stop band ripple)

412
4th order elliptic filter

413
Bessel filters (linear phase approximation)

θ N (0 )
H N (s ) =
θ N (s ω0 )

θ N (s ) = ∑
N
(2 N − n )! s n
N −n
n =0 2 ⋅ n!⋅( N − n)!

Pole layout

414
Actual 3rd order Bessel low pass 415
 Analog Filter transforms

So far we studied normalized low pass
filters with ωc=1

We shall refer to a normalized low pass
filter as the prototype filter

In order to design non-
non-normalized LPF or
other types (HPF,BPF, etc.) we use filter
transforms

416
 s
s←
ω0

Normalized low pass ω =1

to low pass with cutoff at ω = ω0

ω0
s←
s

Normalized low pass ω =1

to high pass with cutoff at ω = ω0
417
 s 2 + ωH ωL
s←
s + (ω H − ω L )

Normalized low pass ω =1

to band pass with cutoffs at ωL , ωH

s + (ω H − ω L )
s← 2
s + ωH ωL

Normalized low pass ω =1

to band stop with cutoffs at ωL , ωH
418
 Example – design a 3rd order
Butterworth LPF at 1KHz
Prototype

LP
LP

419
 Example – Design a 5th order elliptic 1KHz
high--pass 1dB,-
high dB,-60dB
60dB filter with Matlab
>> [b,a] = ellip(5,1,60,1000,'high','s');
>> [h w]=freqs(b,a,1000);
>> semilogx(w,20*log10(abs(h))),grid,xlabel('Freq (Hz)'),ylabel('Gain (dB)')

-10

-20

-30

-40
Gain (dB)

-50

-60

-70

-80

-90

-100
2 3 4
10 10 10
Freq (Hz)
420
 IIR Filter design
Analog domain Digital domain

Specifications

Specifications
Specifications
Design

Analog
Filter
Digital Filter
421
 Analog filter to Digital
transforms

Given a RCSR analog filter

We would like to transform the
analog filter to a “similar” digital filter

Digital filter requirements:

Similar frequency-
frequency-gain properties

RCSR

Same order

422
 Impulse invariance

Continuous time (CT) and discrete time
(DT) LTI systems are characterized by
their impulse response

Filter response is the convolution between
the impulse response and the input signal

Idea – Given a CT filter with impulse
response h(t), use a DT filter with impuse
response h(nT
h(nT))

mapping: HL(s) -> H(z)
423
 Impulse invariance
The digital filter unit sample response is sampling of the
analog filter impulse response

Given analog filter find its impulse response

by Inverse Laplace transform
Sample impulse response
Calculate digital filter using Z transform

In short
424
T is needed for similar DC gain

2.425
Example – single pole LP

426
The stable analog pole at -α
Was transformed to a stable
digital pole at
(note 2π period)

jω Im(z)
3π/Τ
aliasing

π/Τ

-α
x σ x Re(z)

−π/Τ

unit circle 427


High pass - The analog filter impulse
response may include impulses if
numerator order equals or larger than
denominator order

How to sample the analog impulse ??

428
In general using fraction decomposition and the
previous example

429
ha (t ) → h(n ) = Tha (nT )
1 ∞
 θ − 2πk 
H a (ω ) → ⋅ T ∑ H a  
T k = −∞  T 

The frequency response of the digital

filter is an aliased version of the
frequency response of the
corresponding analogue filter.

430
Impulse invariance may only be
used for narrow band LPF
H L (ω )

“Negligible”
aliasing

ωp ωs ~ 10 ⋅ ω s
431
Design example (Butterworth)
ω p = 500 Hz
ω s = 2750 Hz
Fs = 50 KHz
δ p = δ s = 0.01
θ p = 0.02π
θ s = 0.11π
δ p = δ s = 0.01

432
 Design a LPF using Butterworth analog filter and impulse
invariance method
ωp
T θp
Selectivity k = = = 0.1818
ωs T θ s

Discrimination d =
(1 − δ ) p
−2
−1
=
2.03040506 ⋅10 − 2
= 1.426 ⋅10 −3
δ −2
s −1 9999
− log d 2.846
n≥ = = 3.85 → 4
− log k 0.74
1 rad
H L (ω p ) = 8
≥ 1 − δ p = 0.99 ⇒ 5113 [813Hz ]
 2π 500  sec
1 +  
 ωc 

433
 1
Prototype : H(s) =
s 4 + 2.6131s 3 + 3.4142 s 2 + 2.6131s + 1
s
s←
5113
1
H(s) =
1.4632 ⋅10 −15 s 4 + 1.9549 ⋅10 −11 s 3 + 1.306 ⋅10 −7 s 2 + 5.1107 ⋅10 −4 s + 1

5000
x
4000

3000

2000
x
1000

-1000

-2000 x
-3000

-4000

-5000
x
-5000 -4000 -3000 -2000 -1000 0 1000 2000 3000 4000 5000

434
Partial fractions
4724 − 1957 j 4724 + 1957 j
H L(s) = 0.8535(1 − j ) + 0.8535(1 + j )
s − (− 4724 + 1957 j ) s − (− 4724 − 1957 j )
1957 − 4724 j 1957 + 4724 j
+ 0.3535(− 1 − j ) + 0.3535(− 1 + j )
s − (− 1957 + 4724 j ) s − (− 1957 − 4724 j )
α αT
⇒ T = 2 ⋅ 10 −5

s − (− α ) 1 − e −αT ⋅ z −1
806.4 − 334.1 j 806.4 + 334.1 j
(1 − j ) (1 + j )
H(z) = 100 + 100
 − 9.448 + 3.914 j  −1  − 9.448 − 3.914 j  −1
1 − exp ⋅ z 1 − exp ⋅ z
 100   100 
(− 1 − j )138.4 − 334 j (− 1 + j )138.4 + 334 j
+ 100 + 100
 − 3.914 + 9.448 j  −1  − 3.914 − 9.448 j  −1
1 − exp  ⋅ z 1 − exp  ⋅ z
 100   100 

435
 1 − 2 + 7.657 ⋅ z −1 + 0.475 ⋅ z −2 + 3.437 ⋅ z −3
H (z ) = ⋅
1000 1 − 3.733 ⋅ z −1 + 5.234 ⋅ z − 2 − 3.2664 ⋅ z −3 + 0.7655 ⋅ z − 4

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

-0.8

-1
-1 -0.5 0 0.5 1
436
 0

-20

-40
|H| (dB)

-60

-80

-100

-120
0 0.5 1 1.5 2 2.5
f (Hz) 4
x 10

θs = π

The monotonic roll-off is

not kept due to aliasing
437
 0

-10

-20
|H| (dB)

-30

-40

-50

-60

0 500 1000 1500 2000 2500 3000 3500 4000

f (Hz)

2750Hz
<-40dB

θ s = 0.11π

438
 0.05

-0.05
|H| (dB)

-0.1

-0.15

-0.2
0 100 200 300 400 500
f (Hz)

500Hz
>-0.09dB (δs=0.01)

θ s = 0.02π
439
 Step invariance
Given an analog filter with a known step response,
build a digital filter whose step response is a sampled
version the analog filter

Analog filter

Step response

Sample of step
response 440
 Step invariance
Z domain step response

δ (n ) = u (n ) − u (n − 1)
Using ( )
H (z ) = 1 − z −1 R(z )
where R( z ) is the step respose

 −1 H a (s ) 
Z domain unit (
H (z ) = 1 − z −1
)
Z  L  
sample response  s  t = kT 

441
 Step invariance example
as
H (s ) = High pass
s+a
−1 H (s ) a
L =L −1
= ae u (t )
− at

s s+a

H (z ) =
aT 1 − z (−1
)
− aT −1
1− e z
Pole transform similar to impulse invariance

442
 101
H (s) = 2
L

s + 2 s + 101
 101 
H ( z ) = (1 − z ) Z  2
−1

 s ( s + 2 s + 101) 
z −1 1 s +1 1 
= Z − − 
z  s ( s + 1) 2
+ 100 ( s + 1) 2
+ 100 
z −1  z z 2 − ze −T cos10T ze −T sin 10T 
=  − 2 − 
z  z − 1 z − 2 ze cos10T + e
−T − 2T
10( z 2 − 2 ze −T cos10T + e − 2T ) 

Continuous π
T=
5

443
 Bilinear transform

Algebraic method to convert analog filters

to digital IIR filters

No sampling – no aliasing

Applicable to Low pass, High pass, band
pass, band stop, etc.

Simple – No Laplace or Z transforms

Frequency warp – easy to deal with

444
Integral approximation

445
Trapezoidal approximation

446
Difference equation

447
 Transformation formula
“continuous time” “discrete time”
integrator integrator

Bilinear transform

448
Single pole filters

LP: Zero at z=-

1

HP: Zero at z=1

449
 Bilinear transform
A single analog stable pole is transformed to
a single digital stable pole

Both numerator and denominator have

the same order
450
 In general
Conversion of analog filter

Digital filter has p zeros and poles.

If p>q, p-q zeros are added at z=-1 (LP)

451
Mapping of s-plane into the z-
plane

452
 Bilinear Transformation
Mapping of s-plane into the z-plane.
Analog stable poles are converted to digital
stable poles

453
 Bilinear Transformation

The bilinear transformation

has the quality that every
point in the s plane maps into
a unique point in the z plane,
and vice versa. Also, the left
half of the s plane maps into
the interior of the unit circle
in the z plane so a stable s-
domain system is
transformed into a stable z-
domain system.

454
 Frequency transform (analog
imaginary line to digital unit circle)
Since

We may convert frequency variables

455
 Frequency transform (analog
imaginary line to digital unit circle)

For low frequencies

456
Frequency warp

457
 Bilinear Transformation

Mapping is nonlinear

Complete negative imaginary axis in the
s-plane from ω = −∞ to 0 is mapped
into the lower half of the unit circle in the
z-plane from z = −1 to z = 1

Complete positive imaginary axis in the
s-plane from ω = 0 to ω = ∞ is mapped
into the upper half of the unit circle in the
z-plane from z = 1 to z = −1
458
 Bilinear transform

Transformation can be used only to design
digital filters with prescribed magnitude
response with piecewise constant values

Transformation does not preserve phase
response of analog filter

2.459
Frequency warp

460
 Pre warp
IIR Filter specifications are given in the digital domain.
When analog filter is transformed to digital filter using
bilinear transform frequency warping takes place
To compensate, frequencies in spec. are Pre-Warp
before translated to analog domain.

Note: since we compensate for warping, selection of T is arbitrary

461
 Step 1: Specification of Filter

Specify performance H(θ )
at certain frequencies

E.g. cut-
cut-off frequency

Roll--off frequency
Roll

Low pass filter
example

Desired cut-off
θC frequency

θC 462
 Step 2: Pre
Pre--Warp Frequencies

Warp specified H(ω )
frequencies using

2  θc 
ω C = tan  
T 2

Warped cut-off
ωc frequency

ωc
463
 Step 3: Design Analogue Filter

Design analog filter H A (ω )

Analogue filter Analogue
design produces filter response

HA(s)

ωc
464
 Step 4: Apply Bilinear
Transform

Transform H(θ )
H (s ) H( z ) Digital filter
response

Means replace s by

Result

Transfer function, H(z)
for digital filter ωc
465
 Example

Design a discrete filter, with specifications

Butterworth filter

-3dB cut-
cut-off at 2kHz

Attenuation of at least 10dB
10dB at 4kHz

Sampling frequency 20kHz
20kHz

466

Digital frequencies

Cut--off frequency -3dB
Cut
ωc 2πf c 2π × 2000
θc = = = = 0.2π
fs fs 20,000

Roll--off frequency -10dB
Roll 10dB

ωr 2πf r 2π × 4000
θr = = = = 0.4π
fs fs 20,000
467

Conversion to digital

Pre-warp frequencies
Pre-

Use bilinear transform

Pre
Pre--warp cut
cut--off frequency T=2
T=2
'  θc   0.2π 
ωc = tan   = tan   = 0.325
2  2 

468

Pre
Pre--warp roll
roll--off frequency
'  θr   0.4π 
ωr = tan   = tan   = 0.726
2  2 

Need normalised frequencies

To get normalised Butterworth from table
0.325 0.726
0.325 → =1 0.726 → = 2.234
0.325 0.325
Cut-off frequency Roll-off frequency

Using Butterworth formula N>1.37
N=2

469

Normalised Butterworth for order 2

From table

Normalised Butterworth, order 2 is

1
H c ( BW ) ( s ) = 2
s + 2s + 1

470
Replace s by s
0.325

1
H c ( BW ) (s ) = prototype
s 2 + 2s + 1

1
H c (s ) =
1 s2 2
+ 0.325 s +1
0.3252
471

Actual pre-warped analogue filter is
pre-
2
0.325
H c (s) =
2 2 ×0.325 2 2
s + 0.325
s + 0.325

0.106
H c (s) = 2
s + 0.46 s + 0.106

472

Use bilinear transform z −1
s=
z +1

Filter becomes

0.106
H( z ) =
z −1 2
( ) + 0.46( zz +−11 ) + 0.106
z +1
2
0.068( z +1)
Hd ( z) = 2
z −1.142z + 0.413
473
 =-3dB <-10dB
0

-10

-20

-30
|H| (dB)

-40

-50

-60

-70

-80
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
f (Hz)

474
 Digital Filter Structures

The causal IIR digital filters are
characterized by a real rational transfer
function of z −1 or, equivalently by a
constant coefficient difference equation

From the difference equation
representation, it can be seen that the
realization of the causal IIR digital filters
requires some form of feedback

475
Direct Form IIR Digital Filter
Structures

476
 Direct Form IIR

q – Numerator order p+q delays

p - Denominator order
p+q+1 multipliers
(p+q+1)*Fs multiplication/second 477

We may replace the order of filter sections

478
Canonical form

479
Canonic Form IIR

Max(p,q) delays
p+q+1 multipliers
(p+q+1)*Fs multiplication/second

480
Multi--rate Digital Signal
Multi
Processing

481
Example: TV program conversion
Europe (ex. France)
Temporal and
National Television System spatial Phase Alternating Line
Committee sampling rate 25 frames/second
30 frames/second conversion 625 lines per frame
525 lines per frame QAM Color modulated to
QAM Color modulated to 4.43MHz
3.58MHz
Color
US/Japan
information
sampling rate
conversion

Séquentiel couleur à mémoire

25 frames/second
625 lines per frame
312 Color lines
FM Color modulated to
4.43MHz
France
482
Example: Sampling – Conventional
approach Anti aliasing

Analog
Voice LPF 8Ksps
~3.5KHz 0.99 @3.5KHz A/D
bandwidth 0.01@4Khz

~ Order 45 Butterworth
Analog filter. Not
practical
0

-10

-20
Ha(w)|

-30

-40

-50

0 500 1000 1500 2000 2500 3000 3500 4000

Hz
483
 Oversampling

Analog 3.5KHz
Voice
LPF 800Ksps LPF
~3.5KHz
0.99 @3.5KHz A/D Linear 100
bandwidth
0.01@400Khz Phase FIR

Very simple order 2

Butterworth Analog filter
0

-10

-20
Narrowband
-30
digital signal
-40
Ha(w)|

-50

-60

-70
Note:
-80
Decimation
enhances signal
-90
0 1 2 3 4 5
Hz
6 7 8 9 10
5
resolution
x 10 484
Analog Devices’
AD1870
64X Oversampling

485
Example: Reconstruction– Conventional approach

Analog
Digital voice 8Ksps LPF
~3.5KHz ZOH 0.99 @3.5KHz
bandwidth 0.01@4Khz

~ Order 45 Butterworth
Analog filter. Not
practical

486
 Example: Reconstruction– Up sampling

Digital Analog
x100
voice 800Ksps LPF
~3.5KHz
Digital
ZOH 0.99 @3.5KHz
bandwidth Interpolation 0.01@400KHz

Narrowband Simple 2 pole

digital signal Butterworth Analog filter

487
 Time domain

1 1

0.8 0.8

0.6 0.6

0.4 0.4

0.2 0.2

0 0

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Conventional ZOH X10 Up-sampling

488
Analog Devices’
AD1853
8X Oversampling

489
Now we know what that means…

490
 Multirate Digital Signal
Processing
Basic Sampling Rate Alteration Devices

Interpolator (Up-
(Up-sampler) - Used to
increase the sampling rate by an integer
factor

Decimator (Down-
(Down-sampler) - Used to
decrease the sampling rate by an
integer factor

491
 Up--sampling
Up

Ideal sinc() x(t) Sample at

x[n] reconstructio
L Fs x[m]
n
Fs L Fs
We assume original x[n] was
sampled according to Nyquist
rule.
Ts Sampling at L Fs complies with
Nyquist Ts / L

492
Shannon Reconstruction

 mT 
 − nT 
 T ∞
xu [m] = xa  m  = ∑ x[n]sinc L =
 L  n=−∞  T 
 
 
∞
 m − nL 
= ∑ x[n]sinc 
n=−∞  L 
493
Defin x[l L] l mod L = 0 l = Ln
e y[l ] = 
 0 otherwise
∞
 m−l 
xu [m] = ∑ y[l ]sinc 
l =−∞  L 

Convolution between y[m] and

An ideal non-causal digital [-π/L , +π/L]
Low pass filter

In practice we shall use a practical LPF

494
 y[m]
x[n] L LPF xu[m]
Fs L Fs [-π/L , +π/L] L Fs

Graphs are for L=3

495
 Up--Sampler
Up

Up
Up--sampling operation is implemented by
inserting L-L-1 zero
zero--valued samples
between two consecutive samples of x[n]

Linear and time dependant

 x[m /L], m = 0, ± L, ± 2 L, L
x[n] L y[m] = 
 0, otherwise

496
 Up--Sampler
Up

Interms of the z-transform, the input-
input-
output relation is then given by
∞ ∞
Y ( z) = ∑ y[m] z
m = −∞
−m
= ∑ x[m / L] z
m = −∞
−m
=
m mod L = 0

n=m L
∞
= ∑ x[n] z − nL
( )
=X z L

n = −∞

497
 Up--Sampler
Up

( )
Y ( z) = X z L
Frequency domain is scaled by
ratio L

Y (e ) = X (e )
jθ jθL

L=2

( )
Xe jθ

− 2π 2π

( )
Ye jθ

− 2π −π π 2π
498
 Up--Sampler
Up

As can be seen, a factor-

factor-of-
of-L sampling
rate expansion leads to a compression X ( e jθ )
of by a factor of L and a L-
L-fold
repetition in the baseband [-[-π, π]

This process is called imaging as we
get an additional “image
“image”” of the input
spectrum

499
Interpolation filter

500
 Filter

A practical LPF may be either FIR or IIR

FIR Linear phase is often an advantage

Special computation savings when using
dual form FIR

501
 Dual form FIR
x[n]
L
b2 b1 b0

Z-1 + Z-1 + y[m]

Note: Each multiplier is periodically fed with L-1 zeros and 1 non–zero input.

L = L

502
 Dual form FIR
x[n]

b2 b1 b0

L L L

Z-1 + Z-1 + y[m]

Multiplications are done in the ** low ** sampling

frequency

503
 Interpolation filter

Total of ½*N*F multiplications per second

instead of ½*N*F *L if multipliers are
implemented at higher sampling rate 504
 Example: Reconstruction– Up sampling

Digital Analog
x100
voice 800Ksps LPF
~3.5KHz
Digital
ZOH 0.99 @3.5KHz
bandwidth Interpolation 0.01@4Khz

Simple 2 pole
Butterworth Analog filter
100 LPF
[-π/100,
π/100]

505
LPF 3.5 / 400 π
[-π/100,
π/100]

~
~
π/100 = π
4 / 400 π

Filter is designed
Exampleδ p = δ s = 0.01 using Remezfilter at high sampling
rate but
D∞ (δ p , δ s ) 1.9 multipliers work
N≈ = = 3040 at low sampling
∆f F 500 800,000
rate.
6 multiplications
1/ 2 ⋅ F ⋅ N = 8000⋅ 3040 = 24⋅10
second

We shall talk about better implementations

506
 Down--Sampler
Down

Down
Down--sampling operation is implemented
by keeping every M-th sample of x[n] and
removing M-M-1 in in--between samples to
generate y[n]

Linear and time dependant
y[n] = x[nM
nM]]

x[n] M y[n]

Lower sampling rate will result in aliasing 507

 Decimation

x[ n ] = xa ( nT ) M y[ n ] = xa ( nMT )

Input sampling frequency Output sampling frequency

1 ' FT 1
FT = FT = =
T M T'

508
Frequency-Domain Characterization
Frequency-

Applying the z-transform to the input
input--output
relation of a factor-
factor-of-
of-M down
down--sampler

we get
y[n] = x[Mn]

∞
−n
Y ( z) = ∑ x[ Mn ] z
n = −∞

509

Define a new sequence
xint [n] :
 x[n], n = 0, ± M , ± 2 M ,K
xint [n] = 
 0, otherwise

M M

x[n] y[m] xint [n]

∞ ∞
−n −n
Y ( z) = ∑ x[ Mn ] z = ∑ int
x [ Mn ] z
n = −∞ n = −∞
∞
−k / M 1/ M
= ∑ xint [k ] z = X int ( z )
k = −∞ 510

xint [n] can be formally related to x[n]
through xint [n] = c[n] ⋅ x[n]

1, n = 0, ± M , ± 2 M ,K
where c[n] = 
0, otherwise

Recall that c[n] may be

M −given
1 by
1 kn
c[n] =
M
∑ WM
where k =0
− j 2π / M
WM = e

511

Takingthe z-transform ofxint [n] = c[n] ⋅ x[n]
and making use of
1 M −1
c[n] = ∑ WMkn
M k =0

∞
−n 1 ∞  M −1
kn  −n
X int ( z ) = ∑ c[n]x[n] z = 
∑  ∑ WM  x[n] z 
n = −∞ M n = −∞  k =0 
1 M −1 ∞  M −1
= ∑ 
 ∑ x
M k =0  n =−∞
[ n ]WM
kn − n
z 
 =
1
∑ (
X z WM)
−k
 M k =0
512
 For k=0 – Base spectrum
0<k<M - Aliasing

1 
M −1 1

Y (z ) = X int z( ) 1M
= ∑ X  WM ⋅ z M 
M k =0 
−k

1 M −1  −k j Mθ 
Ye( )jθ
= ∑ X  WM ⋅ e  =
M k =0  
1  (θ −2πk )
M −1
j 
= ∑ X  e M 
M k =0  
513
Example M=2

− 2π −π π 2π

K=0

− 4π − 2π 2π 4π
K=1

− 4π − 2π 2π 4π
Total
aliasin
g
− 4π − 2π 2π 4π
 Decimation

The Nyquist frequency after decimation is
1/M the original Nyquist frequency

aliasing

In order to prevent aliasing signal should
be band limited to -π/M, +π
+π/M before
decimation

515
Decimation filter

516
 Example - Oversampling

Analog 3.5KHz
Voice
LPF 800Ksps LPF
~3.5KHz
0.99 @3.5KHz A/D Linear 100
bandwidth
0.01@400Khz Phase FIR

LPF 100
[-π/100,
π/100]
We already
designed this one
today

517
 Direct implementation

½ when GLP filters are used

518
 Better implementation (FIR)

½ when GLP filters are used

519
 Filters for Fractional Sampling
Rate Alteration

A fractional change in the sampling rate
can be achieved by cascading a factor-
factor-of
of--
M decimator with a factor-
factor-of-
of-L interpolator,
where M and L are positive integers

Such a cascade is equivalent to a
decimator with a decimation factor of M/L
or an interpolator with an interpolation
factor of L/M

520
Filters for Fractional Sampling
Rate Alteration

L H u (z) H d (z) M

521
Filters for Fractional Sampling
Rate Alteration

Hence, the desired configuration for the
fractional sampling rate alteration is as
indicated below where the lowpass filter
H(z) has a stopband edge frequency
given by
π π 
θ s = min  , 
L M 

L H (z) M

522
Decimation system we designed few slides ago

LPF
[-π/100, π/100]
100
3.5 / 400 π 800Ksp
s 8Ksps

~
~
4 / 400 π π

Exampleδ p = δ s = 0.01 using Remezfilter Filter is designed

for high sampling
D∞ (δ p , δ s ) 1.9 rate.
N≈ = = 3040 Multiplications are
∆f F 500 800,000
done in the ** low
1 6 multiplications ** sampling
1/ 2 ⋅ F ⋅ N = 8000⋅ 3040 = 12⋅10 frequency
2 second
523
 Multi--stage decimation system
Multi
Split decimation into few stages
M=M1*M2*M3….. may substantially
reduce processing requirements

LPF 1 50 LPF 2 2

M=100=2*50=50*2=4*25=25*4=2*5*10=5*2*5*2=…..
Optimal selection of Mi may be found by trial-and-error or
by graphs found in
R. E. Crochiere & L. R. Rabiner, Multirate Digital Signal
Processing, Prentice- Hall, 1983
524
 Filters in series H1(z)H2(z)
θp1
1±δp
1
δs1

θs1
1±δp θp2
2
δs2

π
θs2
Transition

Pass
band Stop band

(1±δp1) (1±δp2)≈ δs1(1±δp2)≈ δs1δs2<

1±(δp1+ δp2) δs1 δs1,δs2 525
 In general, N filters in series

Assume

δp 1 = δp 2 = … = δp

δ s1 = δ s2 = … = δ s

Connecting N filters in series we (approximately) get

Pass--band: 1+N
Pass +Nδδp

Stop-band: δs
Stop-

If specifications define δs ,δp for the overall filter, for every
sub filter we define δs ,δp/N

526
Equi-ripple filter order can be estimated
by

Transition band (Hz) Sampling Rate (Hz)

527
 Example M=100
M=100,K=
,K=22, δs =δp=0.01
Fp=
Fp=3.5KHz Fs=4
Fs=4KHz Second filter
16Ksps 8Ksps
3.5KHz LPF 2 2
δp=0.005

δs=0.01
Note: second
7π/16 π/2=π/Μ2 π
filter removes
4KHz 8KHz
frequencies
between 4KHz
Exampleδ p = δ s = 0.01 K = 2 and 8KHz

D∞ (δ p / 2, δ s ) 2.15
N≈ = = 69
∆f F 500 16,000
1 3 multiplications
1/ 2 ⋅ F ⋅ N = 8000⋅ 69 = 276⋅10
2 second
528
 Example M=100
M=100,K=
,K=2
2, δs =δp=0.01
Fp=
Fp=3.5KHz Fs=
Fs=44KHz First filter no aliasing
800Ksps 16Ksps
δp=0.005 50
LPF 1
δs=0.01
3.5KHz
Motivation –
Wider transition Filter replica after
band – shorter 8KHz decimation
filter 4KHz

~
~
Will be removed 16KHz π
by second filter
400Ksps
Post decimation
sampling rate

θp=3.5/400π

θs=8/400π=π/50=π/Μ1
529
 Example M=100
M=100,K=
,K=2
2, δs =δp=0.01
Fp=
Fp =3.5KHz Fs=
Fs=4
4KHz First filter controlled aliasing
800Ksps 16Ksps
δp=0.005 50
LPF 1
δs=0.01
3.5KHz
Motivation –
Wider transition 8KHz Filter replica after
band – shorter decimation
filter

~
~
4KHz 12KHz
Will be removed by
16KHz π
second filter 400Ksps
Post decimation
sampling rate
θp=3.5/400π

θs=12/400π
530
Exampleδ p = δ s = 0.01 K = 2
D∞ (δ p / 2, δ s ) 2.15
N≈ = = 202
∆f F (12 − 3.5) 800
1 6 multiplications
1/ 2 ⋅ F ⋅ N = 16000⋅ 202 = 1.6 ⋅10
2 second

Total of 2M Multiplications/second
Compared with original 12M !

531
Interpolation system

8Ksps LPF 800Ksps

100 [-π/100, π/100]

8Ksps 2 LPF 2 50 LPF 1

800Ksps
16Ksps

532
 8Ksps 2 LPF 2 16Ksps

7π/16
3.5KHz
@Fs=16Ksps
Replica being
Original removed by
spectrum filter

π/2 π 2π
4KHz 8KHz 16KHz

2π
533
16KHz
 16Ksps 50 LPF 1 800Ksp
s

3.5KHz
@Fs=800Ksp
Replica being s
removed by
filter

~
~
4KHz 12KHz 16π/400 π
16KHz 400Ksp
s

~
~
3.5KHz π
400Ksp
534
s
Example
Narrowband filters – Legacy approach
800Ksps LPF
800Ksps
[-π/100, π/100]

Note: No decimation, just filter

Exampleδ p = δ s = 0.01 using Remez filter

D∞ (δ p , δ s ) 1.9
N≈ = = 3040
∆f F 500 800,000
1 multiplications
1 / 2 ⋅ F ⋅ N = 800,000⋅ 3040 = 1.2 ⋅109
2 second

535
 Narrowband filters – multi rate
800Ksps
approach
LPF 1 50 LPF 2 2

8Ksps 2 LPF 2 50 LPF 1

800Ksps

Even tough no sampling rate change is needed, the multi-rate

system consume 4*106 multiplications/sec compared with 109
multiplications/sec for the straight forward approach !!!!

536
 Multi--rate identities
Multi
X(z) L X(zL)

X(z) H(z) L X(zL) H(zL)

X(z) L H(zL) X(zL) H(zL)

537
Identities
L H(zL)
x (n ) y (m)
H(z) L
x (n ) y (m)

M H(z)
x (n ) z (l )
H(zM) M
x (n ) z (l )
538
Polyphase representation of FIR filter
unit sample response
M=3
1

0.8

0.6

0.4

0.2

-0.2

-0.4

-0.6

0 2 4 6 8 10 12 14 16

539
Polyphase representation of FIR filter
unit sample response
M=3
1

-1
0 2 4 6 8 10 12 14 16 18
1

-1
0 2 4 6 8 10 12 14 16 18
1

-1
0 2 4 6 8 10 12 14 16 18
540
H (z ) = h0 + h1z −1 + h2 z −2 + ... + hN z − N =
= [h0 + hM z −M + h2M z −2M + ...]+
+ [h1z −1 + hM +1z −( M +1) + h2M +1z −(2M +1) + ...]+
..... + ...]+
+ [hM −1z −( M −1) + h2M −1z −(2M −1) + h3M −1z −(3M −1) + ...] =

= [h0 + hM z −M + h2M z −2M + ...]+

+ z −1[h1 + hM +1z −M + h2M +1z −2M + ...]+ M x
..... + ... + FIR filters in Z-M

+ z −( M −1)[hM −1 + h2M −1z −M + h3M −1z −2M + ...] =

( ) ( )
= H0 z M + z −1H1 z M + .. + z −( M −1) H M −1 z M ( )
541
 Polyphase decimation filter

( ) ( ) ( )
H (z ) = H0 z M + z −1H1 z M + .. + z −(M −1) H M −1 z M

x(n) H0(zM)

z-1
H1(zM) + M y(m)

z-1
H2(zM)
…

z-1
HM-1(zM)
542
Using an identity
Active only for n=…-M,0,M,2M,..

Active only for n=…-

x(n) M+1,1,M+1,2M+1,..
M H0(z)

z-1
M H1(z) + y(m)

z-1
M… H2(z)

z-1
M Active only for n=…-
HM-1(z) M+2,2,M+2,2M+2,..

Active only for n=…-1,M-1,2M-1,3M-1,.. 543

 Commutator switch
implementation

n=…-M,0,M,2M,.. n=…-M+1,1,M+1,2M+1,..
x(n) H0(z)

H1(z) + y(m)
n=…-M+2,2,M+2,2M+2,..

n=…-1,M-1,2M-1,3M-1,.. H2(z)

…
HM-1(z)

544
Polyphase interpolation filter

( ) ( ) ( )
H (z ) = H0 z L + z −1H1 z L + .. + z −( L−1) H L−1 z L

x(n) y(m)
L H0(zL) +

z-1
H1(zL) +

z-1
H2(zL) +
…

z-1
HL-1(zL)
545
Polyphase interpolation filter

Nonzero for m mod L=0

x(n) y(m)
H0(z) L +

z-1
H1(z) L + Nonzero for m mod L=1

z-1
H2(z) L + Nonzero for m mod L=2
…

z-1
HL-1(z) L Nonzero for m mod L=L-1
546
Polyphase interpolation filter

( ) ( ) ( )
H (z ) = H0 z L + z −1H1 z L + .. + z −( L−1) H L−1 z L

m mod L=0
x(n) y(m)
H0(z)

H1(z)

H2(z)
…

HM-1(z)
547
 Introduction to Digital Signal
processing - conclusion
-Preliminaries
-Fourier analysis
-s and z domain analysis
-Analog filter design (wannabe)
-Signal and systems

548
 Introduction to Digital Signal
processing - topics

Signaland system revisited

Poisson formulas

Sampling, Nyquist
Nyquist,, Shannon

DFT

FFT

Cyclic convolution

Spectral analysis

549
 Introduction to Digital Signal
processing - topics

Filters

Continuous phase

Linear phase

Minimum phase

All pass

Implementations – direct and canonic

FIR

Types

IRT with and without windows

Equi--ripple
Equi
550
 Introduction to Digital Signal
processing - topics

IIR

Analog filter:
• Butterworth
• Chebyshev
Chebyshev,, Elliptic, Bessel – Introduction only

Transformation
• Impulse and step invariance
• Bilinear

Multi--rate
Multi

Decimation

Interpolation

Rate change

Polyphase

Multistage
551
 What’s next ?

Project at the SIPL – ‫המעבדה לעיבוד אותות דיבור ותמונה‬

Radom signals – ‫אותות אקראיים‬

Models of random signals and processes

System manipulation of random signals

Noise sources

Signal Processing – ‫עיבוד אותות‬

Quantization and finite word length

Fixed point and floating point DSP

Spectrum analysis, periodograms

Filter banks, QMF filters

Short time Fourier transform

Wavelet transform

552

Image processing and analysis – ‫ענ"ת‬

Human vision

2D signal processing

Image enhancement

Image coding

Digital coding – ‫קידוד אותות‬

Scalar and vector quantization

Entropy coding

Waveform coding

Frequency domain coding

Linear Prediction

LPC based speech coders

553

Discrete time random signal processing – ‫עיבוד‬
‫אותות אקראיים בזמן בדיד‬

Hilbert space representation of ransom signals

Parametric models

Theoretic bounds

Adaptive signal processing – ‫עיבוד אותות מסתגל‬

Adaptive algorithms – LMS RLS

Adaptive filtering

Linear estimation in dynamic systems -
‫שיערוך לינארי במערכות דינמיות‬

Wiener filter

Kalman filter

Hidden Markov models 554