How to manipulate Frequencies in Discrete-time Domain?

Two Main Approaches

• Difference Equations • Discrete Fourier Transform

(in short, DFT)
• Remember we have introduced
three kinds of Fourier transforms.
x[n]: input, y[n]: output
• However, what we are able to
deal with in the discrete-time
• That is, building a system that domain is usually a finite-
makes use of the current and duration signal.
previous inputs x[n], x[n-1], x[n- • DFT is the final (fourth) Fourier
2] …. transform, where its input is a
as well as the previous outputs discrete-time finite-duration
y[n-1], y[n-2], … signal.

Details will be introduced in later courses

Linear System Using only Current and
Finite Previous Inputs
• Recall that a linear system is referred to as a
system y[n]=T{x[n]} satisfying the following
property for all , .

• A linear system employing only the current and

finite previous inputs is referred to as a kind of
(Finite Impulse Response) FIR system or FIR filter.
• General form:
 Example: running average filters
• 3-point running average filter:

• 7-point running average filter

 Time-invariant System
• Recall that a time-invariant system is referred to a
system y[n]=T{x[n]} satisfying

for all n0. That is, when the input is delayed (shifted)
by n0, the output is delayed by the same amount.
Linear Time-invariant (LTI) System
• It can be easily verified that the FIR filter of the
form

is a time-invariant system.
• A system that is both linear and time-invariant is
referred to as an LTI system.
– So, FIR filter is an LTI system.
 IIR System or IIR Filter
• IIR filter: Linear system using both current and
finite previous inputs as well as previous
outputs.
• General form:

• Recursion: In IIR filter, previous outputs have

been used recursively.
 General form of LTI system
• It can be easily verified that IIR filter is also an LTI
system.
• We have the definition of LTI system, but what is the
general form of an LTI system?
• To answer this question, let us investigate what
happens when we input the simplest signal, the
delta function, into an LTI system.
• Recall: delta function (unit impulse) in discrete-time
domain:
 Impulse Response

• When taking the unit impulse [n] as input to a

system, the output h[n] is called the impulse
response.
• Recall: Any discrete-time signal x[n] can be
represented as the linear combination of
delayed unit-impulse functions:


x n    x k  n  k 
k  
 Impulse Response

• Then, we have the following property for the

impulse response of an LTI system:
  
y n   T   x k  n  k 
 k   
 by Linearity
  x k T  n  k 
k  

• So 
by Time-
y[n ]   x k  h n  k  invariance
k  
 Impulse Response


y[n ]   x k  h n  k 
k  

• Hence, when we know the impulse response

h[n] of a linear system, then the output signal
can be completely determined by the input
signal and the impulse response from the above
equation.
equivalent to
Knowing its
Knowing an
impulse
LTI system
response
 Impulse Response


y[n ]   x k  h n  k 
k  

k m
• Let us further investigate the operation above.
• Basically, for the time index n, the above
equation computes the sum of products of all of
the pairs x[k] and x[m] as long as k+m=n.
• This operation is called convolution for the
discrete-time signals.
 Convolution
• Definition:
y[n ]  x[n ]  h[n ]
 

  x k  h n  k    x n  k  h k 
k   k  

• Convolution is commutative
y[n ]  x[n ]  h[n ]  h[n ]  x[n ]
• Similar to the definition for the continuous-time
signals, but summation is used instead of
integration.
Example
 Example
• h1[n] * h2[n]
 Analogous to the arithmetic
multiplication
• In fact, convolution has an intuitive
explanation: it is very similar to “arithmetic
multiplication”
0 , 0, 5 , 2, 3, 0, 0, ...
• Eg., *) 0, 0, 1, 4, 3, 0, 0, ...
 x[n] = 0, 0, 5, 2, 3, 0, 0…
0, 0, 5 , 2, 3, 0, 0, 0
 h[n] = 0, 0, 1, 4, 3, 0, 0…
 x[n]  h[n]: 0, 0, 0, 20, 8, 12, 0, 0
0, 0, 0, 0, 15, 6, 9, 0

0, 0, 5, 22,26,18, 9, 0

Convolution result
 Interpreted as Polynomial Multiplication

• Convolution can also be explained intuitively by

polynomial multiplication:
• Consider f[n]  h[n]
 Response of LTI System


y[n ]  x[n ]  h[n ]   x k  h n  k 

k  

• Property
• Output of an LTI system is the convolution of the input
sequence and the impulse response of the LTI system.
• Convolution is commutative and associative:
 Equivalent Systems
• Hence, we have the following equivalent systems:
 Equivalent Systems

x[n] y[n]
h1[n] h2[n] h3[n]

x[n] h1[n]h2[n] h3[n] y[n]

 Convolution and Equivalent System
• Property
• Convolution is also distributive over addition:
x[n]  (h1[n] + h2[n]) = x[n]  h1[n] + x[n]  h2[n].
 Impulse Responses of Some LTI
Systems

 Ideal delay: h[n] = [n-nd]

 Moving average  1
  M1  n  M
h n    M 1  M 2 1
2


 Accumulator  0 otherwise
1 n0
h n   
0 otherwise

 Forward difference: h[n] = [n+1][n]

 Backward difference: h[n] = [n][n1]
 Cascading Systems
 An LTI system can be realized in different ways by
separating it into different subsystems.
 The following systems are equivalent:

h n    n  1   n    n  1
  n  1   n  1   n 
  n    n  1
 Equivalent Cascading Systems

 Another example of cascading systems – inverse

system.

h n   u n    n    n  1  u n   u n  1   n 
 Causality
• Recall: A system is causal if it does not
dependent on future inputs.
• That is, if x1[n]=x2[n] when n<n0, then the
output y1[n]=y2[n] when n<n0 for all n0.
– or equivalently, the output y[n0] depends only the
input sequence values for n  n0.
• What is the property of the impulse response of
a causal system?
 Property: An LTI system is causal if and only if

h[n] = 0 for all n < 0.

 General Form of LTI Systems
LTI System 

y[n ]   h k  x n  k 
k  

Causal LTI System 

y[n ]   h k  x n  k 
k 0

FIR Filter M
(namely, Finite
Impulse Response)
y[n ]   h k  x n  k 
k 0

IIR Filter
M N
(namely,
Infinite
Impulse
y[n ]   a k  x n  k    b k  y n  k 
k 0 k 1
Response)
 Difference Equation
• From the above, we can find that both FIR and IIR
filters are causal LTI Systems.
• They are both difference equations.
• Difference equation: (variables are functions.
y: unknown function; x: given)
N M

 a k y n  k    b m x n  m 
k 0 m 0
• Both FIR and IIR filters are solutions of difference
equations.
• IIR filter is the case when a0=1. Difference equation
is merely a more general form.
 Realization of an LTI System
• Difference equation is an LTI system.
• Another important property is that difference
equation is realizable by recursion:
– To realize or implement an LTI system, we cannot but
exploit previously appeared (available) inputs and/or
outputs.
• Question: Are all the LTI systems able to be
implemented by using difference equations?
Illustration: (infinite-long) Matrix  Vector
• A General LTI system is of the form:


y[n ]   h k  x n  k 
k  

• Can be explained as a infinite-dimensional matrix/vector

product
    
    
y [  3] h [0 ] h [  1] h[ 2 ] h [  3] h[ 4 ] h [  5] x [  3]
    
 y [  2 ]  h [1] h [0 ] h [  1] h[ 2 ] h [  3] h[ 4 ]   x [  2 ]
    
y [  1] h [2 ] h [1] h [0 ] h [  1] h[ 2 ] h [  3] x [  1]
    
 y [0 ]   h [3] h [2 ] h [1] h [0 ] h [  1] h[ 2 ]   x [0 ] 
    
 y [1]   h [4 ] h [3] h [2 ] h [1] h [0 ] h [  1]  x [1] 
 y [2 ]   h [5] h [4 ] h [3] h [2 ] h [1] h [0 ]   x [2 ] 
    
    
 Realization of LTI system
• Problem for realizing it: there is an infinite-length
sequence h[k] to both ends (- and ).
• To implement this infinite-length “inner product,”
we either do not have enough ‘buffer’ or will
take infinite time in practice.
 Realization of LTI System
• By recursion, IIR filter can track back and approaches
to one-side-infinity.
• We could allow the system to take future inputs (eg.
future L samples), and delay the output by the same
time-length L. With this idea, “IIR filter” becomes
M N

y[n  L ]   a k  x n  k    b k  y n  k 
k  L k 1 L

• However, the range L is still finite.

• We cannot implement the situation when L
approaches to infinity by difference equation.
• In sum, we cannot realize the LTI system when the
impulse response has double-sided infinite-durations.
 Example: discrete-time ideal low pass
filter
• For instance, we will see later that to perform
ideal low-pass filtering with the cutoff frequency
wc (wc <) in the DTFT domain, the impulse
response is a uniformly sampled sequence of the
sinc function:
sin  c n
h[n ] 
n
• Since h[n]’s duration are from - to , discrete-
time ideal low pass filter cannot be realized by
difference equation.
• Despite ideal low-pass filter is an LTI system.
What Happens in Frequency Domain?
• What is the influence in Frequency domain when
input a discrete-time signal x[n] to an LTI system
of impulse response h[n]?
• To investigate the influence, consider that an LTI
system can be characterized by an infinite-
dimensional matrix as illustrated above.
• Conceptually, this matrix should have “infinite-
long” eigenvectors.
• Eigen function: A discrete-time signal is called
eigenfunction if it satisfies the following property:
When applying it as input to a system, the output
is the same function multiplied by a (complex)
constant, i.e., eigenvalue.
 Eigenfunction of LTI System
 Property: x[n] = ejwn is the eigenfunction of all LTI
systems (wR)
 Pf: Let h[n] be the impulse response of an LTI system,
when ejwn is applied as the input,
 
jw  n  k 
y n    h k e  h k e
jwn  jwk
e
k   k  


 Let H e jw    h k e
 jwk

k  

we can see
y n   H e  e jw jwn
 Frequency Response
 Hence, ejwn is the eigenfunction of the LTI system;
 The associated ‘eigenvalue’ is H(ejw).
 We call H(ejw) the LTI system’s frequency response
 The frequency response is a complex function
consisting of the real and imaginary parts, H(ejw) =
HR(ejw) + jHI(ejw)
 Physical meaning: When input is a signal with
frequency w (i.e., ejwn)), the output is a signal of
the same frequency w, but the magnitude and
phase could be changed (characterized by the
complex amplitude H(ejw))
 Frequency Response
Another explanation of frequency response
• Recall the discrete-time Fourier transform
(DTFT) pair:

• Forward DTFT

• Inverse DTFT
 Frequency Response
• By definition, the frequency response:

H e     h k e
jw  jwk

k  

is just the DTFT of the impulse response h[n].

Impulse DTFT Frequency
Response Inverse DTFT Response

• This is an important reason why we need DTFT for

discrete-time signal analysis, besides the use of
continuous Fourier transform.
 LTI System and Frequency Response
• Now, let’s go back to the problem: When input a
signal x[n] to an LTI system, what happens in the
frequency domain?
• Remember that the output of an LTI system is
y[n]=x[n]*h[n], the convolution of x[n] and h[n].
• In DTFT, we still have the convolution theorem:
time domain convolution is equivalent to
frequency domain multiplication.
• Hence, the output in the frequency domain is
the multiplication,

Y(ejw) = X(ejw)H(ejw)
 Multiplication in Frequency Domain
• In sum, the output sequence’s spectrum is the
multiplication of the input spectrum and the
frequency response.
equivalent to
Knowing its
Knowing an
impulse
LTI system
response
equivalent to equivalent to

Knowing its
frequency
response
 Application Example of Eigenfunctions
• Sinusoidal responses of LTI systems:
x n   A cos  w 0 n      x1 n   x 2 n 
A j jw 0 n A  j  jw 0 n
e e  e e
2 2
– The response of x1[n] and x2[n] are

y1  n   H  e   
jw 0 j ( w n  )
A / 2 e 0

y2 n   H  e   
 jw 0  j ( w n  )
A / 2 e 0

– The total response is y[n] = y1[n] + y2[n]

 Example: Frequency Response of the
Moving-average System
 Impulse response of the moving-average
system is
 1
  M1  n  M
h n    M 1  M 2 1
2


 0 otherwise

 Therefore, the frequency response is

  M
M 2
1
e
jw  jwn
H e
1 M 2  1 nM
1
 Geometric Series Formula

 The following is the geometric series formula:

L 1
L
1
 
k
,
k 0 1

 So m m n

  
k n k

k n k 0
m 1
 
n

 , m n
1
 Frequency Response of the Moving-
average System
M2
1
 M  M 1  e
 jw n
H  e 
jw

1 2 n M1

 jw  M 2  1 
e
jw M 1
1 e
  jw
M1  M2 1 1 e
jw  M 1  M 2  1  / 2  jw  M 1  M 2  1  / 2
1 e e  jw  M 2  M 1  1  / 2
  jw
e
M1  M2 1 1 e
jw  M 1  M 2  1  / 2  jw  M 1  M 2  1  / 2
1 e e  jw  M 2  M 1  / 2
  jw / 2
e
M1  M2 1 e
jw / 2
e
 Frequency Response of the Moving-
average System

 Further evaluation

jw  M 1  M 2  1  / 2  jw  M 1  M 2  1  / 2
1 e e  jw  M 2  M 1  / 2
H  e 
jw
 M  M 1  jw / 2
e
e
jw / 2
1 2
e

sin  w  M 1  M 2  1  / 2   jw  M  M  / 2
1
 e 2 1

M1  M2 1 sin  w / 2 
 
 H e 
jw
jw j H e
exp

(magnitude and phase)

 Spectrum of the Moving-average System
(M1 = 0 and M2 = 4)
 Recall that, in DTFT, the frequency response is repeated
with period 2. “High frequency” is close to .

Amplitude
response

Phase
response
Example: Discrete-time Ideal Low-pass
Filter
• Ideal low-pass filter in DTFT domain
 1, | w | w c
) 
jw
H low pass ( e
 0, w c  | w | 

• Note that we usually depict the frequency

response in the range [-,] only, and bearing
in mind that the spectrum is repeated with
the period 2.
Ideal Low-pass Filter in DTFT domain
 Discrete-time Ideal Low-pass Filter

• The impulse response hlowpass[n] can be found by

inverse Fourier transform:

1 wc
h low pass [ n ]  
jw n
e dw
2  wc

1 1  jw c n
  e
jw n wc jw c n
e |  wc
(e )
2  jn 2  jn
sin w c n

n
Uniform samples of Sinc function
 Approximation of Discrete-time
Ideal Low-pass Filter
• In other words, the forward DTFT of the sampled sync
function recovers the ideal low-pass filter:

sin w c n

 jw n
)
jw
H low pass ( e e
n n  

• However, as analyzed before, the sampled sync

function cannot be realized by difference euation
because it reaches infinity on both ends.
• To approximate the ideal low-pass filter by realizable
approaches, a method is to use the partial sum instead
M
sin w c n

 jw n
)
jw
H M (e e
n M n
 Approximation of Discrete-time
Ideal Low-pass Filter
• Examples of M=1, 3, 7, 19
 Detailed Investigation of DTFT (c.f. Shenoi, 2006)

We will represent the spectrum of DTFT either by H(ejwT) or

more often by H(ejw) for convenience.
 When represented as H(ejw), it has the frequency range [-
,]. In this case, the frequency variable is to be understood as
the normalized frequency. The range [0, ] corresponds to [0,
ws/2] (where wsT=2), and the normalized frequency 
corresponds to the Nyquist frequency (and 2 corresponds to
the sampling frequency).
DC response
When w=0, the complex exponential ejw becomes a constant
signal, and the frequency response X(ejw) is often called the DC
response when w=0.
–The term DC stands for direct current, which is a constant
current.
DTFT Properties Revisited
 Time shifting
 Frequency shifting

Time reversal
 DTFT of (n)
n

   n e
 jwn
e 1
jw 0

n  

DTFT of (n+k)+ (n-k)

According to the time-shifting property,

DTFT of  n  k  is e , DTFT of  n  k  is e
jwk  jwk

Hence

DTFT of  n  k    n  k  is e  2 cos  wk 
 jwk
e
jwk
 DTFT of x(n) = 1 (for all n)
x(n) can be represented as
We prove that its DTFT is
Hence
    jwn

     w  2 k e dw
 k   
   
 
     w  2 k  dw
 k   
   
     w dw  
 
  w dw      w dw     w dw



    w dw  1 for all n

From another point of view
 According to the sampling property: the DTFT of a continuous
signal xa(t) sampled with period T is obtained by a periodic
duplication of the continuous Fourier transform Xa(jw) with a
period 2/T = ws and scaled by T.
Since the continuous F.T. of x(t)=1 (for all t) is (t), the DTFT
of x(n)=1 shall be a impulse train (or impulse comb), and it turns
out to be

 DTFT of anu(n) (|a|<1)

let
Geometric series
then
to infinity

This infinite sequence converges to

when |a|<1.
 DTFT of Unit Step Sequence
Adding these two results, we have the final result
 Differentiation Property
 DTFT of a rectangular pulse
and get
 Suddenly Applied Complex Exponential
Inputs
 In practice, to get the frequency response, we may
not apply the complex exponential inputs ejwn to a
system, but the more practical-appearing inputs of
the form
x[n] = ejwn u[n]
 i.e., complex exponentials that are suddenly applied at an
arbitrary time, which for convenience we choose n=0.
 Consider its output to a causal LTI system:


 0 n0
 n
y n    h k x n  k     
 
 
 jwk e jwn
h k e n0
k   
 k  0 
 Suddenly Applied Complex
Exponential Inputs (continue)
 We consider the output for n  0.
     
y n     h k e    h k e
 jwk e jwn  jwk e jwn
   
 k 0   k  n 1 
  
 H e  e
jw jwn
   h k e

 jwk e

jwn

 k  n 1 
 Hence, the output can be written as y[n] = yss[n] + yt[n],
where
y ss n   H e
jw
 
e
jwn
Steady-state response

y t n     h k e
 jwk jwn
e Transient response
k  n 1
 Suddenly Applied Complex
Exponential Inputs (continue)
 If h[n] = 0 except for 0 n  M (i.e., a FIR system),
then the transient response yt[n] = 0 for n+1 > M.
That is, the transient response becomes zero
since the time n = M. For n  M, only the steady-
state response exists.
 For infinite-duration impulse response (i.e., IIR)
 
y t n    h k e  h k   Q n
 jwk jwn
e 
k  n 1 k  n 1
 For stable system, Qn must become increasingly
smaller as n , and so is the transient response.