Digital Signal Processing (PEC-EE 504/PE07)

Module 1: Discrete-time signals and systems

Discrete time signals and systems: Sequences; representation of signals on orthogonal basis; Representation of discrete
systems using difference equations, Sampling and reconstruction of signals-aliasing; Sampling theorem and Nyquist rate.
Module 2: Discrete Fourier Transform
Frequency Domain Analysis, Discrete Fourier Transform (DFT), Properties of DFT, Convolution of signals, Fast Fourier
Transform Algorithm, Parseval’s Identity, Implementation of Discrete Time Systems.

Module 3: Design of Digital filters

Design of FIR Digital filters: Window method, Park-McClellan's method. Design of IIR Digital Filters: Butterworth,
Chebyshev and Elliptic Approximations; Low-pass, Band-pass, Band-stop and High-pass filters. Effect of finite register
length in FIR filter design. Parametric and non-parametric spectral estimation. Introduction to multi-rate signal
processing.
Module 4: Applications of Digital Signal Processing
Convolution and Correlation Functions and Power Spectra, Stationary Processes, Optimal filtering using ARMA Model,
Linear Mean-Square Estimation, Wiener Filter, Wavelet Transform.

28-10-2022 07:12 1
Text/Reference Books:
1. S. K. Mitra, “Digital Signal Processing: A computer-based approach”, McGraw Hill, 2011.
2. V. Oppenheim and R.W. Schafer, “Discrete Time Signal Processing”, Prentice Hall, 1989.
3. J. G. Proakis and D.G.Manolakis, “Digital Signal Processing: Principles, Algorithms And Applications”, PrenticeHall,1997.
4. L. R. Rabiner and B. Gold, “Theory and Application of Digital Signal Processing”, Prentice Hall, 1992.
5. J. R. Johnson, “Introduction to Digital Signal Processing”, Prentice Hall, 1992.
6. D. J. DeFatta, J. G. Lucas and W. S. Hodgkiss, “Digital Signal Processing”, John Wiley & Sons, 1988.

28-10-2022 07:12 2
 Discrete-Time Signals and Systems
Signals, System and Signal Processing

1. Signal:
 A signal is defined as any physical quantity that various with time, space or any other independent variable or variables.
 Mathematically, we describe a signal as a function of one or more independent variables.
 For example, the functions

f1 (t )  5 t
f 2 (t )  7 t 2
describe two signals, one that varies linearly with the independent variable t (time) and a second that varies
quadratically with t.
 As another example, consider the function
f ( x, y )  3x  2 y  10 y 2

 This function describes a signal of two independent variables x and y that could represent the two spatial coordinate s
in a plane.
28-10-2022 07:12 3
2. System:
 A system may also be defined as a physical device that performs an operation on a signal.
 For example, a filter used to reduce the noise and interference corrupting a desired information-bearing signal is called a
system.
 In this case the filter performs some operation(s) on the signal, which has the effect of reducing (filtering) the noise and
interference from the desired in formation-bearing signal.

3. Signal Processing:
 When we pass a signal through a system, as in filtering, we say that we have processed the signal.
 In this case, the processing of the signal involves filtering the noise and interference from the desired signal.
 In general, the system is characterized by the type of operation that it performs on the signal.
 For example, if the operation is linear, the system is called linear.
 If the operation on the signal is nonlinear, the system is said to be nonlinear, and so forth.
 Such operations are usually referred to as signal processing.

28-10-2022 07:12 4
Basic Elements of a Digital Signal Processing System
 Most of the signals encountered in science and engineering are analog in nature.
 That is the signals are functions of a continuous variable, such as time or space, and usually take on values in a
continuous range.
 Such signals may be processed directly by appropriate analog systems (such as filters or frequency analysers) or
frequency multipliers for the purpose of changing their characteristics or extracting some desired in formation.
 In such a case we say that the signal has been processed directly in its analog form, as illustrated in Fig. 1.
 Both the input signal and the output signal are in analog form.

Figure 1: Block diagram of a digital signal processing system.

28-10-2022 07:12 5
Digital Signal Processing System
 Digital signal processing provides an alternative method for processing the analog signal, as illustrated in Fig. 2.
 To perform the processing digitally, there is a need for an interface between the analog signal and the digital
processor.
 This interface is called an analog-to-digital (A/D) converter.
 The output of the A/D converter is a digital signal that is appropriate as an input to the digital processor.
 The digital output from the digital signal processor is to be given to the user in analog form, such as in speech
communication s, we must provide another interface from the digital domain to the analog domain.
 Such an interface is called a digital-to-analog (D/A) converter. Thus, the signal is provided to the user in analog
form, as illustrated in the block diagram of Fig. 2.

Figure 2: Block diagram of a digital signal processing system.

28-10-2022 07:12 6
1. Analog-to-Digital Conversion
 Most signals of practical interest, such as speech, biological signals, seismic signals, radar signals, sonar signals, and
various communications signals such as audio and video signals, are analog.
 To process analog signals by digital means, it is first necessary to convert them into digital form, that is, to convert
them to a sequence of numbers having finite precision.
 This procedure is called analog-to-digital (A/D) conversion, and the corresponding devices are called A/D converters
(ADCs).
 Conceptually, we view A/D conversion as a three-step process.
 This process is illustrated in Fig. 3.

Figure 3: Basic parts of an analog-to-digital (A /D) converter.

28-10-2022 07:12 7
a. Sampling:
 This is the conversion of a continuous-time signal in to a discrete time signal obtained by taking “samples” of the
continuous-time signal at discrete-time instants.
 Thus, if xa(t) is the input to the sampler, the output is xa(nT) = x(n), where T is called the sampling interval.

b. Quantization:
 This is the conversion of a discrete-time continuous-valued signal in to a discrete-time, discrete-valued (digital)
signal.
 The value of each signal sample is represented by a value selected from a finite set of possible values.
 The difference between the unquantized sample x(n) and the quantized output xq(n) is called the quantization
error.

c. Coding:
 In the coding process, each discrete value xq(n) is represented by a 6-bit binary sequence.

28-10-2022 07:12 8
2. Digital-to-Analog Conversion
 The process of converting a digital signal into an analog signal is known as digital-to-analog (D/A) conversion.
 All D/A converters “connect the dots” in a digital signal by performing some kind of interpolation, whose accuracy
depends on the quality of the D/A conversion process.

 Figure 4 illustrates a simple form o f D/A

conversion, called a zero-order hold or
a staircase approximation.
 Other approximations are possible,
such as linearly connecting a pair of
successive samples (linear
interpolation), fitting a quadratic
through three successive samples
(quadratic interpolation), and so on.
 For signals having a limited frequency
content (finite bandwidth), the
sampling theorem is introduced in the
optimum form of interpolation.
Figure 4: Zero-order hold digital-to-analog (D /A) conversion.
28-10-2022 07:12 9
Sampling of Analog Signals
 There are many ways to sample an analogy signal.
 The periodic or uniform sampling are used most often in practice.
 This is described by the relation

x(n)  xa (nT ) n (1)

where x(n) is the discrete-time signal obtained

by “taking samples” of the analogy signal xa(t)
every T second.
 This procedure is illustrated in Fig. 5.
 The time interval T between successive
samples is called the sampling period or
sample interval and its reciprocal 1/T = Fs is
called the sampling rate (samples per second)
or the sampling frequency (hertz).

Figure 5: Periodic sampling of an analog signal.

28-10-2022 07:12 10
Sampling of Analog Signals
 Periodic sampling establishes a relationship between the time variables t and n of continuous-time and discrete-time
signals, respectively.
 These variables are linearly related through the sampling period T or, equivalently, through the sampling rate Fs = 1/T,
as
n
t  nT 
Fs (2)

 As a consequence of (2), there exists a relationship between the frequency variable F (or Ω) for analog signals and the
frequency variable f (or 𝜔) for discrete-time signals.
 To establish this relationship, consider an analog sinusoidal signal of the form

xa (t )  A cos(2 Ft   ) (3)
which, when sampled periodically at a rate Fs = 1/T as samples per second, yields

 2 nF  (4)
xa (nT )  x(n)  A cos(2 FnT   )  A cos   
 Fs 

28-10-2022 07:12 11
Sampling of Analog Signals
 We note that the frequency variables F and f are linearly related as
F
f  (5)
Fs
or, equivalently, as

  T (6)

where,
Fs = sampling frequency
F = frequency of analog signal
f = frequency of digital signal
ω = relative or normalized frequency

 The range of the frequency variable F or Ω for continuous-time sinusoids are

  F  
(7)
    

28-10-2022 07:12 12
Sampling of Analog Signals
 The situation is different for discrete-time sinusoids. We know that
1 1
 f
2 2 (8)
   
 By substituting from (5) and (6) in to (8), we find that the frequency of the continuous-time sinusoid when sampled at
a rate Fs = 1/T must fall in the range

1 F F 1
  s F  s  (9)
2T 2 2 2T

or, equivalently,

 
   Fs     Fs  (10)
T T
 From these relations we observe that the fundamental difference between continuous-time and discrete-time signals
is in their range of values of the frequency variables F and f, or Ω and ω.
 Periodic sampling of a continuous-time signal implies a mapping of the infinite frequency range for the variable F (or
Ω) into a finite frequency range for the variable F (or Ω).
 Since the highest frequency in a discrete -time signal is ω = π or f = ½, it follows that, with a sampling rate Fs, the
corresponding
28-10-2022 07:12 highest values of F and Ω are 13
Sampling of Analog Signals

 Since the highest frequency in a discrete -time signal is ω = π or f = ½, it follows that, with a sampling rate Fs, the
corresponding highest values of F and Ω are
Fs 1
Fmax  
2 2T
(11)

 max   Fs 
T
 Therefore, sampling introduces an ambiguity, since the highest frequency in a continuous-time signal that can be
uniquely distinguished when such a signal is sampled at a rate Fs = 1/T is

Fmax = Fs/2
or Ωmax = πFs .

28-10-2022 07:12 14
The Sampling Theorem

 Let us suppose that any analog signal can be represented as a sum of sinusoids of different amplitudes, frequencies,
and phases, that is,
N
xa (t )   Ai cos(2 Fi t   i ) (12)
i 1

where N denotes the number of frequency components.

 From our knowledge of Fmax, we can select the appropriate sampling rate.
 Any frequency above Fs/2 or below - Fs/2 results in samples that are identical with a corresponding frequency in the
range – Fs/2 ≤ F ≤ Fs/2.
 We must select Fs/2 to be greater than Fmax.
 Thus, to avoid the problem of aliasing, Fs is selected so that

Fs > 2 Fmax

where Fmax is the largest frequency component in the analog signal.

28-10-2022 07:12 15
The Sampling Theorem

 With the sampling rate selected in this manner, any frequency component, say |Fi| < Fmax, in the analog signal is
mapped in to a discrete-time sinusoid with a frequency
1 F 1
  fi  i 
2 Fs 2
or, equivalently,

  i  2 f i  

 Since, |f| =1/2 or |ω| = π is the highest (unique) frequency in a discrete-time signal.
 The condition Fs > 2Fmax ensures that all the sinusoidal components in the analog signal are mapped in to
corresponding discrete-time frequency components with frequencies in the fundamental interval.
 All the frequency components of the analog signal are represented in sampled form without ambiguity, and
hence the analog signal can be reconstructed without distortion from the sample values using an “appropriate”
interpolation (digital-to-analog conversion) method.
 The “appropriate” or ideal interpolation formula is specified by the sampling theorem.

28-10-2022 07:12 16
 Sampling Theorem
 If the highest frequency contained in an analog signal xa(t) is Fmax = B and the signal is sampled at a rate Fs > 2 Fmax = 2B.
 Then xa(t) can be exactly recovered from its sample values using the interpolation function
sin 2 Bt (12)
g (t ) 
2 Bt
 Thus, xa(t) may be expressed as

 n   n 
xa (t )   aF x g
  t   (13)
n    s   Fs 
where xa(n/Fs) = xa(nT) = x(n) are the samples of xa(t).

 When the sampling of xa(t) is performed at the minimum sampling rate Fs = 2B, the reconstruction formula in (1.4.23)
becomes

 n  sin 2 B (t  n / 2 B )
xa (t )   a  2 B  2 B (t  n / 2 B )
n
x (14)

 The sampling rate FN = 2B = 2 Fmax is called the Nyquist rate.

28-10-2022 07:12 17
Sampling Theorem

 As can be observed from either (1.4.23) or (1.4.24), the reconstruction of xa(t) from the sequence xa(n) is a
complicated process, involving a weighted sum of the interpolation function g(t) and its time-shifted versions g(t - nT)
for - ∞ < n < ∞, where the weighting factors are the samples x(n).
 Because of the complexity and the infinite number of samples required in (1.4.23) or (1.4.24), these reconstruction
formulas are primarily of theoretical interest.

28-10-2022 07:12 18
Signal Reconstruction
The sampling process produces a discrete time signal from a continuous time signal by examining the value of the
continuous time signal at equally spaced points in time.
 Reconstruction, also known as interpolation, attempts to perform an opposite process that produces a continuous
time signal coinciding with the points of the discrete time signal.
 Because the sampling process for general sets of signals is not invertible, there are numerous possible reconstructions
from a given discrete time signal, each of which would sample to that signal at the appropriate sampling rate.
 This module will introduce some of these reconstruction schemes.

Reconstruction Process
 The process of reconstruction, also commonly known as interpolation, produces a continuous time signal that would
sample to a given discrete time signal at a specific sampling rate.
 Reconstruction can be mathematically understood by first generating a continuous time impulse train

ximp (t )  
n  
xs (n)  (t  nTs )

from the sampled signal xs with sampling period Ts and then applying a lowpass filter G that satisfies certain conditions
to produce an output signal x .
28-10-2022 07:12 19
Reconstruction Process
 If G has impulse response g, then the result of the reconstruction process illustrated in Figure 2.
 It is given by the following computation, the final equation of which is used to perform reconstruction in practice.

Figure 2: Block diagram of reconstruction process for a given lowpass filter G.

 It is given by the following computation, the final equation of which is used to perform reconstruction in practice.

x (t )  ( ximp  g )(t )   ximp ( ) g (t   )d


 



n  
xs (n) (  nTs ) g (t   )d

 
 
n  
xs (n)   (  nTs ) g (t   ) d



 
n  
xs (n) g (t  nTs )

28-10-2022 07:12 20
Reconstruction Filters

 In order to guarantee that the reconstructed signal x samples to the discrete time signal xs from which it was
reconstructed using the sampling period Ts, the lowpass filter G must satisfy certain conditions.
 These can be expressed well in the time domain in terms of a condition on the impulse response g of the lowpass
filter G.
 The sufficient condition to be a reconstruction filters that we will require is that, for all n  
1 n0
g (nTs )     (n).
 0 n  0
 This means that gg sampled at a rate Ts produces a discrete time unit impulse signal.
 Therefore, it follows that sampling x with sampling period Ts results in

x (nTs )  
m  
xs ( m) g ( nTs  mTs )


 
m  
xs (m) g ((n  m)Ts )


 
m  
xs (m)  (n  m)

 xs ( n )
which is the desired result for reconstruction filters.
28-10-2022 07:12 21
Perfect Reconstruction

 In order to understand the conditions for perfect reconstruction and the filter it employs, consider the following.
 As a beginning, a sufficient condition under which perfect reconstruction is possible will be discussed.
 Subsequently, the filter and process used for perfect reconstruction will be detailed.
 Recall that the sampled version xs of a continuous time signal x with sampling period Ts has a spectrum given by

1 
   2 k 
X s ( ) 
Ts
 X 
T
.
k    s 
 As before, note that if xx is bandlimited to (−π/Ts, π/Ts), meaning that X is only nonzero on (−π/Ts, π/Ts), then each period
of Xs has the same form as X.
 Thus, we can identify the original spectrum X from the spectrum of the samples Xs and, by extension, the original
signal x from its samples xs at rate Ts if x is bandlimited to (−π/Ts, π/Ts).

28-10-2022 07:12 22
Perfect Reconstruction
 If a signal x is bandlimited to (−B, B), then it is also bandlimited to (−π/Ts, π/Ts) provided that Ts < π/B.
 Thus, if we ensure that x is sampled to xs with sufficiently high sampling angular frequency ωs = 2π/Ts > 2B and have a
way of identifying the unique (−π/Ts, π/Ts) bandlimited signal corresponding to a discrete time signal at sampling
period Ts, then xs can be used to reconstruct x  x exactly.
 The frequency 2B is known as the angular Nyquist rate.
 Therefore, the condition that the sampling rate ωs = 2π/Ts >2B be greater than the Nyquist rate is a sufficient condition
for perfect reconstruction to be possible.
 The correct filter must also be known in order to perform perfect reconstruction.
 The ideal lowpass filter defined by G ( )  Ts (u (   / Ts )  u (   / Ts )), which is shown in Figure 3 removes all signal
content not in the frequency range (−π/Ts, π/Ts).


 Now, application of this filter to the impulse train n  
xs ( n)  (t  nTs ) results in an output bandlimited to (−π/Ts, π/Ts).

Figure 3: The above plots show the ideal lowpass filter and its inverse Fourier transform, the sinc function.
28-10-2022 07:12 23
Perfect Reconstruction

 We now only need to confirm that the impulse response g of the filter G satisfies our sufficient condition to be a
reconstruction filter.
 The inverse Fourier transform of G(ω) is
 1 t0
g (t )  sin c (t / Ts )   sin(2 / Ts ,
  t / Ts t0
which is shown in Figure 4. Hence,

 1 n0 1 n0
g (nTs )  sin c(n)   sin( n )    (n).
 n n0 0 n0 Figure 4: The plots show an example discrete
time signal and its Whittaker-Shannon sinc
 Therefore, the ideal lowpass filter G is a valid reconstruction filter. reconstruction.

 Since it is a valid reconstruction filter and always produces an output that is band limited to (−π/Ts, π/Ts), this filter
always produces the unique (−π/Ts, π/Ts) band limited signal that samples to a given discrete time sequence at sampling



period Ts when the impulse train n  
xs (n)  (t  nTs ) is input.

28-10-2022 07:12 24
Perfect Reconstruction

 Therefore, we can always reconstruct any (−π/Ts, π/Ts) bandlimited signal from its samples at sampling period Ts by
the formula

X (T )  
n 
xs sin c (t / Ts  n).

 This perfect reconstruction formula is known as the Whittaker-Shannon interpolation formula and is sometimes also
called the cardinal series. In fact, the sinc function is the infinite order cardinal basis spline η∞.
 Consequently, the set { sinc(t/Ts− n)| n ∈  } forms a basis for the vector space of (−π/Ts, π/Ts) bandlimited signals
where the signal samples provide the corresponding coefficients.
 It is a simple exercise to show that this basis is, in fact, an orthogonal basis.

28-10-2022 07:12 25
Aliasing Phenomena

 Through discussion of the Nyquist-Shannon sampling theorem and Whittaker-Shannon reconstruction formula, it has
already been shown that a (−B, B)(−B, B) continuous time signal can be reconstructed from its samples at rate ωs =
2π/Ts via the sinc interpolation filter if ωs > 2B.
 Now, this module will investigate a problematic phenomenon, called aliasing, that can occur if this sufficient condition
for perfect reconstruction does not hold.
 When aliasing occurs the spectrum of the samples has different form than the original signal spectrum, so the
samples cannot be used to reconstruct the original signal through Whittaker-Shannon interpolation.

Aliasing
 Aliasing occurs when each period of the spectrum of the samples does not have the same form as the spectrum of
the original signal.
 Given a continuous time signals x with continuous time Fourier transform X, recall that the spectrum Xs of sampled
signal xs with sampling period Ts is given by

1 
   2 k 
X s ( ) 
Ts
 X
T
.
k    s 
28-10-2022 07:12 26
Aliasing
 As has already been mentioned several times, if xx is bandlimited to (−π/Ts, π/Ts) then each period of Xs has the same
form as X.
 However, if x is not bandlimited to (−π/Ts, π/Ts), then the X   T2s k  can overlap and sum together.
 This is illustrated in Figure 5 in which sampling above the Nyquist frequency produces a samples spectrum of the same
shape as the original signal, but sampling below the Nyquist frequency produces a samples spectrum with very different
shape.

Figure 5: The spectrum of a bandlimited signals is

shown as well as the spectra of its samples at rates
above and below the Nyquist frequency. As is shown,
no aliasing occurs above the Nyquist frequency, and
the period of the sample spectrum centred about
the origin has the same form as the spectrum of the
original signal scaled in frequency. Below the Nyquist
frequency, aliasing can occur and causes the
spectrum to take a different than the original
spectrum.
28-10-2022 07:12 27
Aliasing

 Whittaker-Shannon interpolation of each of these sequences produces different results.

 The low frequencies not affected by the overlap are the same, but there is noise content in the higher frequencies
caused by aliasing.
 Higher frequency energy masquerades as low energy content, a highly undesirable effect.

 Unlike when sampling above the Nyquist frequency, sampling below the Nyquist frequency does not yield an injective
(one-to-one) function from the (−B, B) bandlimited continuous time signals to the discrete time signals.
 Any signal x with spectrum X which overlaps and sums to Xs samples to xs.
 It should be intuitively clear that there are very many (−B, B) bandlimited signals that sample to a given discrete time
signal below the Nyquist frequency, as is demonstrated in Figure 6.
 It is quite easy to construct uncountably infinite families of such signals.

28-10-2022 07:12 28
Aliasing

Figure 6: The spectrum of a discrete time signal xs,

taken from Figure 5, is shown along with the
spectra of three (−B, B) signals that sample to it at
rate ωs < 2B. From the sampled signal alone, it is
impossible to tell which, if any, of these was
sampled at rate ωs to produce xs. In fact, there are
infinitely many (−B, B) bandlimited signals that
sample to xs at a sampling rate below the Nyquist
rate.

28-10-2022 07:12 29
Aliasing

 Aliasing obtains it name from the fact that multiple, in fact infinitely many, (−B, B) bandlimited signals sample to the
same discrete sequence if ωs < 2B.
 Thus, information about the original signal is lost in this noninvertible process, and these different signals effectively
assume the same identity, an “alias”.
 Hence, under these conditions the Whittaker-Shannon interpolation formula will not produce a perfect
reconstruction of the original signal but will instead give the unique (−ωs/2, ωs/2) bandlimited signal that samples to
the discrete sequence.

28-10-2022 07:12 30
28-10-2022 07:12 31