77un/ Edition

DIGITAL
SIGNAL
PROCESSING
Principles, Algorithms, m l Applications

J o h n G. Proakis
Dimitris G. M anolakis

Digital Signal
Processing
Principles, Algorithms, and Applications
Third E dition

John G. Proakis
Northeastern U niversity

Dimitris G. Manolakis
Boston C ollege

PRENTICE-HALL INTERNATIONAL, INC.

This edition may be sold only in those countries to which it is consigned by Prentice-Hall International.
It is not to be reexported and it is not for sale in the U .S.A ., Mexico, or Canada.
1996 by Prentice-Hall, Inc.
Simon & Schuster/A Viacom Company
U pper Saddle River, New Jersey 07458

All rights reserved. No part of this book may be

reproduced, in any form or by any means,
without permission in writing from the publisher.

The author and publisher of this book have used their best efforts in preparing this book. These
efforts include the development, research, and testing of the theories and programs to determine their
effectiveness. The author and publisher make no warranty of any kind, expressed or implied, with
regard to these programs or the documentation contained in this book. The author and publisher shall
not be liable in any event for incidental or consequential damages in connection with, or arising out
of. the furnishing, performance, or use of these programs.

Printed in the United States of America

10

ISBN 0-13-3TM33fl-cl

Prentice-Hall International (U K ) Limited. L ondon

Prentice-Hall of Australia Pty. Limited, Sydney
Prentice-Hall Canada, Inc., Toronto
Prentice-Hall Hispanoamericana. S.A., M exico
Prentice-Hall of India Private Limited, N ew D elhi
Prentice-Hall of Japan, Inc., T okyo
Simon & Schuster Asia Pie, Ltd., Singapore
Editora Prentice-Hall do Brasil, Ltda., R io de Janeiro
Prentice-Hall, Inc, Upper Saddle River, N ew Jersey

Contents

PREFACE

xiii
1

1 INTRODUCTION
1.1

S ignals, S ystem s, and S ignal P ro cessin g 2

1.1.1
Basic Elements of a Digital Signal Processing System. 4
1.1.2 A dvantages of Digital over Analog Signal Processing, 5

1.2

C lassificatio n o f Signals 6
1.2.1
Multichannel and Multidimensional Signals. 7
1.2.2 Continuous-Time Versus Discrete-Time Signals. 8
1.2.3 Continuous-Valued Versus Discrete-Valued Signals. 10
1.2.4 Determ inistic Versus Random Signals, 11

1.3

T h e C o n c e p t o f F re q u e n c y in C o n tin u o u s -T im e an d
D isc re te -T im e S ignals 14
1.3.1
Continuous-Time Sinusoidal Signals, 14
1.3.2 Discrete-Time Sinusoidal Signals. 16
1.3.3 Harmonically Related Complex Exponentials, 19

1.4

A n a lo g -to -D ig ita l an d D ig ita l-to -A n a lo g C o n v e rs io n 21

1.4.1
Sampling of Analog Signals, 23
1.4.2 The Sampling Theorem , 29
1.4.3 Q uantization of Continuous-Am plitude Signals, 33
1.4.4 Quantization of Sinusoidal Signals, 36
1.4.5 Coding of Quantized Samples, 38
1.4.6 Digital-to-Analog Conversion, 38
1.4.7 Analysis of Digital Signals and Systems Versus Discrete-Time
Signals and Systems, 39
S u m m a ry a n d R e fe re n c e s

Problems

39

40
iii

iv

Contents

DISCRETE-TIME SIGNALS AND SYSTEMS

2.1

D isc rete-T im e S ignals 43

2.1.1 Some Elem entary Discrete-Time Signals, 45
2.1.2 Classification of Discrete-Time Signals, 47
2.1.3 Simple Manipulations of Discrete-Time Signals, 52

2.2

D isc re te -T im e S ystem s 56
2.2.1
Input-O utput Description of Systems, 56
2.2.2 Block Diagram Representation of Discrete-Time Systems, 59
2.2.3 Classification of Discrete-Time Systems, 62
2.2.4 Interconnection of Discrete-Tim e Systems, 70

2.3

A n alysis o f D isc re te -T im e L in e a r T im e -In v a ria n t S ystem s 72

2.3.1 Techniques for the Analysis of Linear Systems, 72
2.3.2 Resolution of a Discrete-Time Signal into Impulses, 74
2.3.3 Response of LTI Systems to A rbitrary Inputs: The Convolution
Sum, 75
2.3.4 Properties of Convolution and the Interconnection of LTI
Systems, 82
2.3.5 Causal Linear Tim e-Invariant Systems. 86
2.3.6 Stability of Linear Tim e-Invariant Systems, 87
2.3.7 Systems with Fim te-D uration and Infinite-Duration Impulse
Response. 90

2.4

D isc rete-T im e System s D e s c rib e d by D iffe re n c e E q u a tio n s 91

2.4.1
Recursive and Nonrecursive Discrete-Tim e Systems, 92
2.4.2 Linear Time-Invariant Systems Characterized by
Constant-Coefficient Difference Equations, 95
2.4.3 Solution of Linear Constant-Coefficient Difference Equations. 100
2.4.4 The Impulse Response of a Linear Tim e-Invariant Recursive
System, 108

2.5

Im p le m e n ta tio n o f D isc re te -T im e S ystem s 111

2.5.1
Structures for the Realization of Linear Tim e-Invariant
Systems, 111
2.5.2 Recursive and Nonrecursive Realizations of FIR Systems, 116

2.6

C o rre la tio n of D isc re te -T im e S ignals 118

2.6.1
Crosscorrelation and A utocorrelation Sequences, 120
2.6.2 Properties of the A utocorrelation and Crosscorrelation
Sequences, 122
2.6.3 Correlation of Periodic Sequences, 124
2.6.4 Com putation of Correlation Sequences, 130
2.6.5 Input-O utput Correlation Sequences, 131

2.7

S u m m ary a n d R e fe re n c e s

Problems

135

134

43

Contents

THE Z-TRANSFORM AND ITS APPLICATION TO THE ANALYSIS

OF LTI SYSTEMS
3.1

T h e r-T ra n sfo rm
151
3.1.1
The Direct ^-Transform. 152
3.1.2 The inverse : -Transform, 160

3.2

P ro p e rtie s o f th e ; -T ra n sfo rm

3.3

R a tio n a l c-T ran sfo rm s 172

3.3.1
Poles and Zeros, 172
3.3.2 Pole Location and Time-Domain Behavior for Causal Signals. 178
3.3.3 The System Function of a Linear Tim e-Invariant System. 181

3.4

In v e rs io n o f th e ^ -T ra n sfo rm 184
3.4.1
The Inverse ; -Transform by Contour Integration. 184
3.4.2 The Inverse ;-Transform by Power Series Expansion. 186
3.4.3 The Inverse c-Transform by Partial-Fraction Expansion. 188
3.4.4
Decomposition of Rational c-Transforms. 195

3.5

T h e O n e -sid e d ^ -T ra n sfo rm
197
3.5.1
Definition and Properties, 197
3.5.2
Solution of Difference Equations. 201

3.6

A n aly sis o f L in e a r T im e -In v a ria n t S ystem s in th e --D o m a in

3.6.1
Response of Systems with Rational System Functions. 203
3.6.2 Response of P ole-Z ero Systems with Nonzero Initial
Conditions. 204
3.6.3 Transient and Steady-State Responses, 206
3.6.4 Causality and Stability. 208
3.6.5 P ole-Z ero Cancellations. 210
3.6.6 M ultiple-Order Poles and Stability. 211
3.6.7 The Schur-C ohn Stability Test, 213
3.6.8 Stability of Second-Order Systems. 215

3.7

S u m m ary an d R e fe re n c e s
P ro b le m s

151

161

203

219

220

FREQUENCY ANALYSIS OF SIGNALS AND SYSTEMS

4.1

F re q u e n c y A n aly sis o f C o n tin u o u s-T im e Signals 230

4.1.1 The Fourier Series for Continuous-Time Periodic Signals. 232
4.1.2 Power Density Spectrum of Periodic Signals. 235
4.1.3 The Fourier Transform for Continuous-Time Aperiodic
Signals, 240
4.1.4 Energy Density Spectrum of Aperiodic Signals. 243

4.2

F re q u e n c y A n aly sis o f D isc re te -T im e Signals 247

4.2.1 The Fourier Series for Discrete-Time Periodic Signals, 247

230

Contents

V)

4.2.2
4.2.3
4.2.4
4.2.5
4.2.6
4.2.7
4.2.8

Power Density Spectrum of Periodic Signals. 250

The Fourier Transform of Discrete-Time Aperiodic Signals. 253
Convergence of the Fourier Transform. 256
Energy Density Spectrum of Aperiodic Signals, 260
Relationship of the Fourier Transform to the i-Transform , 264
The Cepstrum, 265
The Fourier Transform of Signals with Poles on the Unit
Circle, 267
4.2.9 The Sampling Theorem Revisited, 269
4.2.10 Frequency-Domain Classification of Signals: The Concept of
Bandwidth, 279
4.2.11 The Frequency Ranges of Some N atural Signals. 282
4.2.12 Physical and M athematical Dualities. 282
4.3

P ro p e rtie s of th e F o u rie r T ra n s fo rm fo r D isc re te -T im e

S ignals 286
4.3.1
Symmetry Properties of the Fourier Transform, 287
4.3.2 Fourier Transform Theorems and Properties, 294

4.4

F re q u e n c y -D o m a in C h a ra c te ristic s of L in e a r T im e -In v a ria n t

S ystem s 305
4.4.1
Response to Complex Exponential and Sinusoidal Signals: The
Frequency Response Function. 306
44.2
Steady-State and Transient Response to Sinusoidal Input
Signals. 314
4.4.3 Steady-State Response to Periodic Input Signals, 315
4.4.4 Response to Aperiodic Input Signals. 316
4.4.5 Relationships Between the System Function and the Frequency
Response Function. 319
4.4.6 Com putation of the Frequency Response Function. 321
4.4.7
Input-O utput Correlation Functions and Spectra, 325
4.4.8 Correlation Functions and Power Spectra for Random Input
Signals. 327

4.5

L in e a r T im e -In v a ria n t S ystem s as F re q u e n c y -S e le c tiv e

F ilters 330
Ideal Filter Characteristics, 331
4.5.1
4.5.2 Lowpass, Highpass, and Bandpass Filters, 333
4,5.3 Digital Resonators, 340
4.5.4
Notch Filters, 343
4.5.5
Comb Filters. 345
4.5.6 All-Pass Filters. 350
4.5.7
Digital Sinusoidal Oscillators, 352

4.6

In v e rse S y stem s an d D e c o n v o lu tio n 355

4.6.1
Invertibility of Linear Tim e-Invariant Systems, 356
4.6.2 Minimum-Phase. Maximum-Phase, and Mixed-Phase Systems. 359
4.6.3 System Identification and Deconvolution, 363
4.6.4 Hom om orphic Deconvolution. 365

vii

Contents

4.7

S u m m ary a n d R e fe re n c e s
P ro b le m s

368

THE DISCRETE FOURIER TRANSFORM: ITS PROPERTIES AND

APPLICATIONS
5.1

F re q u e n c y D o m a in Sam pling: T h e D isc re te F o u rie r

T ra n s fo rm 394
5.1.1
Frequency-Dom ain Sampling and Reconstruction of
Discrete-Time Signals. 394
5.1.2 The Discrete Fourier Transform (DFT). 399
5.1.3 The D FT as a Linear Transform ation. 403
5.1.4 Relationship of the DFT to O ther Transforms, 407

5.2

P ro p e rtie s o f th e D F T 409
5.2.1
Periodicity. Linearity, and Symmetry Properties, 410
5.2.2
Multiplication of Two DFTs and Circular Convolution. 415
5.2.3
Additional DFT Properties, 421

5.3

L in e a r F ilte rin g M e th o d s B ased on th e D F T

5.3.1
Use of the DFT in Linear Filtering. 426
5.3.2 Filtering of Long Data Sequences. 430

5.4

F re q u e n c y A n aly sis o f S ignals U sing th e D F T

5.5

S u m m ary an d R e fe re n c e s
P ro b le m s

367

394

425

433

440

440

EFFICIENT COMPUTATION OF THE DFT: FAST FOURIER

TRANSFORM ALGORITHMS

448

6.1

E fficien t C o m p u ta tio n of th e D F T : F F T A lg o rith m s 448

6.1.1
Direct Com putation of the DFT, 449
6.1.2 D ivide-and-Conquer Approach to Com putation of the DFT. 450
6.1.3 Radix-2 FFT Algorithms. 456
6.1.4 Radix-4 FFT Algorithms. 465
6.1.5
Split-Radix FFT Algorithms, 470
6.1.6 Im plem entation of FFT Algorithms. 473

6.2

A p p lic a tio n s o f F F T A lg o rith m s 475

6.2.1
Efficient Com putation of the D FT of Two Real Sequences. 475
6.2.2 Efficient Com putation of the D FT of a Z N -Point Real
Sequence, 476
6.2.3 Use of the FFT Algorithm in Linear Filtering and Correlation, 477

6.3

A L in e a r F ilte rin g A p p ro a c h to C o m p u ta tio n o f th e D F T

6.3.1 The Goertzel Algorithm, 480
6.3.2 The Chirp-z Transform Algorithm, 482

479

viii

Contents

6.4

Q u a n tiz a tio n E ffects in the C o m p u ta tio n o f th e D F T 486

6.4.1
Quantization Errors in the Direct Com putation of the DFT. 487
6.4.2 Quantization Errors in FFT Algorithms. 489

6.5

S u m m ary an d R e fe re n c e s
P ro b le m s

493

494

500

7 IMPLEMENTATION OF DISCRETE-TIME SYSTEMS

7.1

S tru c tu res fo r th e R e a liz a tio n o f D isc re te -T im e S ystem s

7.2

S tru c tu res fo r F IR System s 502

7.2.1
Direcl-Form Structure, 503
7.2.2
Cascade-Form Structures. 504
7.2.3
Frequency-Sampling S tructures1. 506
7.2.4 Lattice Structure. 511

500

S tru c tu re s for IIR S ystem s 519

7.3.1
Direct-Form Structures. 519
7.3.2 Signal Flow Graphs and Transposed Structures. 521
7.3.3 Cascade-Form Structures, 526
7.3.4 Parallel-Form Structures. 529
7.3.5
Lattice and Lattice-Ladder Structures for IIR Systems, 531
S tate-S p a ce System A n aly sis a n d S tru c tu re s 539
7.4.1
State-Space Descriptions of Systems Characterized by Difference
Equations. 540
7.4.2 Solution of the State-Space Equations. 543
7.4.3 Relationships Between Input-O utput and State-Space
Descriptions, 545
7.4.4 State-Space Analysis in the z-Domain, 550
7.4.5 Additional State-Space Structures. 554
R e p re s e n ta tio n of N u m b e rs 556
7.5.1 Fixed-Point Representation of Numbers. 557
7.5.2 Binary Floating-Point R epresentation of Numbers. 561
7.5.3 E rrors Resulting from R ounding and Truncation. 564
Q u a n tiz a tio n of F ilte r C o e fficien ts 569
7.6.1
Analysis of Sensitivity to Quantization of Filter Coefficients. 569
7.6.2 Q uantization of Coefficients in FIR Filters. 578
7.7

R o u n d -O ff E ffects in D igital F ilte rs 582

7.7.1 Limit-Cycle Oscillations in Recursive Systems. 583
7.7.2 Scaling to Prevent Overflow, 588
7.7.3 Statistical Characterization of Q uantization Effects in Fixed-Point
Realizations of Digital Filters. 590

7.8

S u m m ary a n d R e fe re n c e s
P ro b le m s

600

598

ix

Contents

8.1

G e n e ra l C o n s id e ra tio n s 614
8.1.1
Causality and Its Implications. 615
8.1.2 Characteristics of Practical Frequency-Selective Filters. 619

8.2

D e sig n o f F IR F ilters 620

8.2.1
Symmetric and Antisym m eiric FIR Filters, 620
8.2.2 Design of Linear-Phase FIR Filters Using Windows, 623
8.2.3 Design of Linear-Phase FIR Filters by the Frequency-Sampling
M ethod, 630
8.2.4 Design of Optimum Equiripple Linear-Phase FIR Filters, 637
8.2.5 Design of FIR Differentiators, 652
8.2.6 Design of Hilbert Transformers, 657
8.2.7 Comparison of Design M ethods for Linear-Phase FIR Filters, 662

8.3

D esig n o f I I R F ilters F ro m A n a lo g F iiters 666

8.3.1
IIR Filter Design by Approxim ation of Derivatives. 667
8.3.2 IIR Filter Design by Impulse Invariance. 671
8.3.3 IIR Filter Design by the Bilinear Transform ation, 676
8.3.4 The M atched-; Transform ation, 681
8.3.5
Characteristics of Commonly Used Analog Filters. 681
8.3.6 Some Examples of Digital Filter Designs Based on the Bilinear
Transform ation. 692

8.4

F re q u e n c y T ra n s fo rm a tio n s 692
8.4.1
Frequency Transform ations in the Analog Dom ain, 693
8.4.2 Frequency Transform ations in the Digital Dom ain. 698

8.5

D esig n o f D ig ital F ilters B a sed on L e a st-S q u a re s M e th o d

8.5.1
Pade Approxim ation Method, 701
8.5.2 Least-Squares Design Methods, 706
8.5.3 FIR Least-Squares Inverse (W iener) Filters, 711
8.5.4 Design of IIR Filters in the Frequency Dom ain, 719

8.6

S u m m ary an d R e fe re n c e s
P ro b le m s

614

DESIGN OF DIGITAL FILTERS

701

724

726

SAMPLING AND RECONSTRUCTION OF SIGNALS

9.1

S am p lin g o f B a n d p a ss S ignals 738

9.1.1
R epresentation of Bandpass Signals, 738
9.1.2 Sampling of Bandpass Signals, 742
9.1.3 Discrete-Time Processing of Continuous-Time Signals, 746

9.2

A n a lo g -to -D ig ita l C o n v e rsio n 748

9.2.1 Sample-and-Hold. 748
9.2.2 Quantization and Coding, 750
9.2.3 Analysis of Q uantization Errors, 753
9.2.4 Oversampling A /D Converters, 756

738

Contents

9.3

D ig ita l-to -A n a lo g C o n v e rsio n 763

9.3.1
Sample and Hold, 765
9.3.2 First-Order Hold. 768
9.3.3 Linear Interpolation with Delay, 771
9.3.4
Oversampling D/A Converters, 774

9.4

S u m m ary an d R e fe re n c e s
P ro b le m s

774

775

10 MULTIRATE DIGITAL SIGNAL PROCESSING

782

10.1

In tro d u c tio n

10.2

D e c im a tio n by a F a c to r D

10.3

In te rp o la tio n by a F a c to r /

10.4

S am p lin g R a te C o n v e rsio n by a R a tio n a l F a c to r I ID

10.5

F iite r D esig n an d Im p le m e n ta tio n for S a m p lin g -R ate

C o n v e rsio n 792
10.5.1 Direct-Form FIR Filter Structures, 793
10.5.2 Polyphase Filter Structures, 794
10.5.3 Time-Variant Filter Structures. 800

10.6

M u ltistag e Im p le m e n ta tio n o f S a m p lin g -R a te C o n v e rs io n

10.7

S a m p lin g -R a te C o n v e rsio n o f B a n d p a ss S ignals 810

10.7.1 Decim ation and Interpolation by Frequency Conversion, 812
10.7.2 M odulation-Free Method for Decimation and Interpolation. 814

10.8

S a m p lin g -R a te C o n v e rsio n by an A rb itra ry F a c to r 815

10.8.1 First-O rder Approxim ation, 816
10.8.2 Second-Order Approximation (Linear Interpolation). 819

10.9

A p p lic a tio n s o f M u ltira te Signal P ro c essin g 821

10.9.1 Design of Phase Shifters. 821
10.9.2 Interfacing of Digital Systems with Different Sampling Rates, 823
10.9.3 Im plem entation of Narrowband Lowpass Filters, 824
10.9.4 Im plem entation of Digital Filter Banks. 825
10.9.5 Subband Coding of Speech Signals, 831
10.9.6 Q uadrature M irror Filters. 833
10.9.7 Transmultiplexers. 841
10.9.8 Oversampling A/D and D /A Conversion, 843

10.10

S u m m ary an d R e fe re n c e s
P ro b le m s

846

783
784
787

844

790

806

xi

Contents

11 LINEAR PREDICTION AND OPTIMUM LINEAR FILTERS

11.1

In n o v a tio n s R e p re s e n ta tio n o f a S ta tio n a ry R a n d o m

P ro c e ss 852
11.1.1 Rational Power Spectra. 854
11.1.2 Relationships Between the Filter Param eters and the
Autocorrelation Sequence, 855

11.2

F o rw a rd an d B a ck w ard L in e a r P re d ictio n 857

11.2.1 Forw ard Linear Prediction, 857
11.2.2 Backward Linear Prediction, 860
11.2.3 The Optimum Reflection Coefficients for the Lattice Forward and
Backward Predictors, 863
11.2.4 Relationship of an A R Process to Linear Prediction. 864

11.3

S o lu tio n o f th e N o rm al E q u a tio n s 864

11.3.1 The Levinson-Durbin Algorithm. 865
11.3.2 The Schiir Algorithm. 868

11.4

P ro p e rtie s o f th e L in e a r P re d ic tio n -E rro r F ilte rs

11.5

A R L attice an d A R M A L a ttic e -L a d d e r F ilters 876

11.5.1 AR LaLtice Structure. 877
11.5.2 A RM A Processes and Lattice-Ladder Filters. 878

11.6

W ie n e r F ilters fo r F ilterin g a n d P re d ictio n 880

11.6.1 FIR W iener Filter, 881
11.6.2 Orthogonality Principle in Linear M ean-Square Estimation, 884
11.6.3 IIR W iener Filter. 885
11.6.4 Noncausal Wiener Filter. 889

11.7

S u m m ary an d R e fe re n c e s
P ro b le m s

852

873

890

892

12 POWER SPECTRUM ESTIMATION

12.1

E stim a tio n o f S p e c tra from F in ite -D u ra tio n O b s e rv a tio n s o f

Signals 896
12.1.1 Com putation of the Energy Density Spectrum. 897
12.1.2 Estim ation of the Autocorrelation and Power Spectrum of
Random Signals: The Periodogram. 902
12.1.3 The Use of the DFT in Power Spectrum Estim ation, 906

12.2

N o n p a ra m e tric M e th o d s fo r P o w er S p ectru m E s tim a tio n 908

12.2.1 The B artlett Method: Averaging Periodograms, 910
12.2.2 The Welch Method: Averaging Modified Periodogram s, 911
12.2.3 The Blackman and Tukey Method: Smoothing the
Periodogram, 913
12.2.4 Perform ance Characteristics of N onparam etric Power Spectrum
Estim ators, 916

896

xii

Contents

12.2.5 Com putational Requirem ents of Nonparam etric Power Spectrum

Estimates, 919
12.3

P a ra m e tric M e th o d s fo r P o w er S p e c tru m E stim a tio n 920

12.3.1 Relationships Between the A utocorrelation and the Model
Param eters, 923
12.3.2 The Y ule-W alker M ethod for the A R Model Param eters, 925
12.3.3 The Burg M ethod for the A R Model Param eters, 925
12.3.4 Unconstrained Least-Squares M ethod for the A R Model
Param eters, 929
12.3.5 Sequential Estim ation M ethods for the A R Model Param eters, 930
12.3.6 Selection of A R Model O rder, 931
12.3.7 MA Model for Power Spectrum Estim ation, 933
12.3.8 A R M A Model for Power Spectrum Estim ation, 934
12.3.9 Some Experim ental Results, 936

12.4

M in im u m V a rian ce S p ectral E stim a tio n

12.5

E ig e n an aly sis A lg o rith m s fo r S p e c tru m E stim a tio n 946

12.5.1 Pisarenko Harm onic Decom position M ethod, 948
12.5.2 Eigen-decomposition of the A utocorrelation Matrix for Sinusoids
in White Noise, 950
12.5.3 MUSIC Algorithm. 952
12.5.4 ESPR IT Algorithm, 953
12.5.5 O rder Selection Criteria. 955
12.5.6 Experim ental Results, 956

12.6

S u m m ary an d R e fe re n c e s
P ro b le m s

942

959

960

A RANDOM SIGNALS CORRELATION FUNCTIONS, AND POWER

SPECTRA

A1

B RANDOM NUMBER GENERATORS

B1

C TABLES OF TRANSITION COEFFICIENTS FOR THE DESIGN OF

LINEAR-PHASE FIR FILTERS
D LIST OF MATLAB FUNCTIONS
REFERENCES AND BIBLIOGRAPHY
INDEX

C1
D1
R1
11

Lj_ Preface

T h is b o o k w as d e v e lo p e d b ased on o u r te ach in g o f u n d e rg ra d u a te and g ra d u

a te level co u rse s in d ig ital signal p ro cessin g o v er th e p a s t several y ears. In this
b o o k w e p re se n t th e fu n d a m e n ta ls o f d isc re te -tim e signals, system s, and m o d e rn
d ig ital p ro cessin g a lg o rith m s an d a p p lic a tio n s fo r stu d e n ts in electrical e n g in e e r
ing. c o m p u te r en g in eerin g , a n d c o m p u te r science. T h e b o o k is su itab le fo r e ith e r
a o n e -se m e s te r o r a tw o -se m e ste r u n d e rg ra d u a te level c o u rse in d isc re te system s
a n d dig ital signal p ro cessin g . It is also in te n d e d fo r use in a o n e -se m e s te r first-year
g ra d u a te -le v e l co u rse in digital signal processing.
It is a ssu m ed th a t th e s tu d e n t in electrical and c o m p u te r e n g in e e rin g has h ad
u n d e rg ra d u a te c o u rses in a d v an ce d calculus (in clu d in g o rd in a ry d iffe re n tia l e q u a
tio n s). an d lin ear sy stem s fo r c o n tin u o u s-tim e signals, including an in tro d u c tio n
to th e L ap lace tran sfo rm . A lth o u g h the F o u rie r se ries a n d F o u rie r tra n sfo rm s of
p e rio d ic an d a p e rio d ic signals a re d escrib ed in C h a p te r 4, we ex p ect th a t m any
s tu d e n ts m ay have h ad th is m a te ria l in a p rio r course.
A b ala n c e d co v erag e is p ro v id e d of b o th th e o ry an d p ra c tic a l ap p licatio n s.
A larg e n u m b e r o f w ell d esigned p ro b le m s a re p ro v id e d to h e lp th e s tu d e n t in
m a ste rin g th e su b ject m a tte r. A so lu tio n s m a n u a l is av ailab le fo r th e b en efit o f
th e in stru c to r an d can be o b ta in e d fro m th e p u b lish er.
T h e th ird e d itio n o f th e b o o k covers basically th e sa m e m a te ria l as th e se c
o n d e d itio n , b u t is o rg an ized d ifferen tly . T h e m a jo r d ifferen ce is in th e o rd e r in
w hich th e D F T a n d F F T alg o rith m s are co v ered . B a sed o n su g g estio n s m a d e by
se v era l rev iew ers, w e n o w in tro d u c e th e D F T a n d d esc rib e its efficient c o m p u ta
tio n im m e d ia te ly fo llo w ing o u r tr e a tm e n t of F o u rie r analysis. T his re o rg a n iz a tio n
h as also allo w ed us to elim in a te re p e titio n o f so m e to p ics c o n cern in g th e D F T and
its ap p licatio n s.
In C h a p te r 1 w e d escrib e th e o p e ra tio n s in v o lv ed in th e an alo g -to -d ig ital
c o n v ersio n o f an alo g signals. T h e p ro cess o f sa m p lin g a sin u so id is d escrib ed in
so m e d e ta il an d th e p ro b le m o f aliasing is ex p lain ed . Signal q u a n tiz a tio n an d
d ig ita l-to -a n a lo g co n v ersio n a re also d escrib ed in g e n e ra l term s, b u t th e analysis
is p re s e n te d in su b s e q u e n t c h a p te rs.
C h a p te r 2 is d e v o te d e n tire ly to th e c h a ra c te riz a tio n a n d analysis o f lin e a r
tim e -in v a ria n t (sh ift-in v arian t) d isc re te -tim e system s a n d d isc re te -tim e signals in
th e tim e d o m a in . T h e co n v o lu tio n sum is d e riv e d a n d system s a re categ o rized
a c co rd in g to th e d u ra tio n of th e ir im p u lse re sp o n s e as a fin ite -d u ra tio n im p u lse
xiii

xiv

Preface

re sp o n se (F IR ) an d as an in fin ite -d u ra tio n im pulse re sp o n se ( II R ) . L in e a r tim ein v a ria n t sy stem s c h a ra c te riz e d by d ifferen ce e q u a tio n s are p r e s e n te d an d th e so
lu tio n o f d ifferen ce e q u a tio n s w ith initial c o n d itio n s is o b ta in e d . T h e c h a p te r
co n clu d es w ith a tre a tm e n t o f d isc re te -tim e c o rre la tio n .
T h e z -tra n sfo rm is in tro d u c e d in C h a p te r 3. B o th th e b ila te ra l an d th e
u n ila te ra l z -tra n sfo rm s are p re se n te d , a n d m e th o d s fo r d e te rm in in g th e in v erse
z -tra n sfo rm are d esc rib e d . U se o f the z -tra n s fo rm in the analysis o f lin ear tim ein v a ria n t sy stem s is illu stra te d , an d im p o rta n t p ro p e rtie s o f system s, su c h as c a u s a l
ity a n d stab ility , a re re la te d to z-d o m ain ch aracteristics.
C h a p te r 4 tr e a ts th e analysis o f signals and sy stem s in th e fre q u e n c y d o m ain .
F o u rie r se ries an d th e F o u rie r tra n sfo rm a re p re s e n te d fo r b o th co n tin u o u s-tim e
an d d isc rete-tim e signals. L in e a r tim e -in v a ria n t (L T I) d isc rete sy stem s are c h a r
a c terized in th e fre q u e n c y d o m a in by th e ir freq u e n c y resp o n se fu n c tio n an d th e ir
re sp o n se to p e rio d ic an d a p e rio d ic signals is d e te rm in e d . A n u m b e r of im p o rta n t
ty p es o f d isc re te -tim e system s are d esc rib e d , in clu d in g re s o n a to rs , n o tc h filters,
co m b filters, all-p ass filters, a n d o scillato rs. T h e desig n of a n u m b e r of sim ple
F IR a n d IIR filters is also co n sid ered . In a d d itio n , th e stu d e n t is in tro d u c e d to
th e co n c e p ts o f m in im u m -p h a se , m ix ed -p h ase, an d m a x im u m -p h a se system s an d
to th e p ro b le m o f d e c o n v o lu tio n .
T h e D F T . its p ro p e rtie s an d its a p p licatio n s, a re th e topics c o v e re d in C h a p
te r 5. T w o m e th o d s a re d e sc rib e d fo r using th e D F T to p e rfo rm lin e a r filtering.
T h e use o f th e D F T to p e rfo rm fre q u e n c y analysis o f signals is also d escrib ed .
C h a p te r 6 co v ers th e efficient c o m p u ta tio n o f th e D F T . In c lu d e d in this c h a p
te r are d e sc rip tio n s o f radix-2, ra d ix -4, a n d sp lit-ra d ix fast F o u rie r tra n sfo rm (F F T )
alg o rith m s, a n d a p p lic a tio n s o f th e F F T a lg o rith m s to th e c o m p u ta tio n o f c o n v o
lu tio n a n d c o rre la tio n . T h e G o e rtz e l alg o rith m a n d the ch irp -z tra n sfo rm are
in tro d u c e d as tw o m e th o d s fo r c o m p u tin g th e D F T using lin e a r filtering.
C h a p te r 7 tre a ts th e re a liz a tio n o f I I R an d F IR system s. T h is tre a tm e n t
in clu d es d irect-fo rm , cascad e, p a ra lle l, lattice, a n d la ttic e -la d d e r re a liz a tio n s. T h e
c h a p te r in clu d es a tr e a tm e n t o f sta te -sp a c e analysis an d s tru c tu re s fo r d isc rete-tim e
system s, an d ex am in es q u a n tiz a tio n effects in a d igital im p le m e n ta tio n o f F IR and
I IR system s.
T e c h n iq u e s fo r d esign o f digital F IR a n d IIR filters are p r e s e n te d in C h a p
te r 8. T h e d esign te c h n iq u e s in clu d e b o th d irect design m e th o d s in d isc re te tim e
an d m e th o d s involv in g th e co n v ersio n o f an a lo g filters in to digital filters by v ario u s
tra n sfo rm a tio n s. A lso tre a te d in this c h a p te r is th e d esig n o f F I R a n d IIR filters
by le a st-sq u a re s m e th o d s.
C h a p te r 9 fo cu ses o n th e sam pling o f c o n tin u o u s-tim e sig n a ls a n d th e r e
c o n s tru c tio n o f such signals fro m th e ir sam ples. In th is c h a p te r, w e d eriv e th e
sam p lin g th e o re m fo r b a n d p a ss co n tin u o u s-tim e -sig n a ls an d th e n co v e r th e A /D
an d D /A co n v ersio n te c h n iq u e s, including o v e rsam p lin g A /D a n d D /A co n v erters.
C h a p te r 10 p ro v id e s an in d e p th tre a tm e n t o f sa m p lin g -ra te c o n v ersio n and
its a p p lic a tio n s to m u ltira le d ig ital signal p ro cessin g . In a d d itio n to d escrib in g d e c
im atio n a n d in te rp o la tio n by in te g e r facto rs, we p re s e n t a m e th o d o f sa m p lin g -rate

Preface

xv

co n v e rsio n by an a rb itra ry facto r. S ev eral a p p licatio n s to m u ltira te signal p ro c e ss

ing a re p re s e n te d , in clu d in g th e im p le m e n ta tio n o f d igital filters, su b b a n d cod in g
o f sp e ech sig n als, tra n sm u ltip le x in g , an d o v ersam p lin g A /D a n d D /A c o n v e rte rs.
L in e a r p re d ic tio n an d o p tim u m lin e a r (W ien er) filters a re tr e a te d in C h a p
te r 11. A lso in clu d ed in this c h a p te r are d escrip tio n s o f th e L e v in s o n -D u rb in
alg o rith m a n d Schiir a lg o rith m fo r solving th e n o rm a l e q u a tio n s , as w ell as th e
A R la ttic e a n d A R M A la ttic e -la d d e r filters.
P o w e r sp e c tru m e stim a tio n is th e m ain to p ic of C h a p te r 12. O u r co v erag e
in clu d es a d e s c rip tio n o f n o n p a ra m e tric an d m o d el-b ased (p a ra m e tric ) m e th o d s.
A lso d e s c rib e d a re e ig e n -d e c o m p o sitio n -b a se d m e th o d s, in clu d in g M U S IC an d
E S P R IT .
A t N o r th e a s te r n U n iv ersity , w e h av e u se d th e first six c h a p te rs o f this b o o k
fo r a o n e -se m e s te r (ju n io r level) c o u rse in d isc rete sy stem s a n d d ig ital signal p r o
cessing.
A o n e -s e m e s te r se n io r level c o u rse fo r stu d e n ts w h o h av e h a d p rio r e x p o su re
to d isc rete sy stem s can u se th e m a te ria l in C h a p te rs 1 th ro u g h 4 for a q u ick rev iew
a n d th e n p ro c e e d to co v er C h a p te r 5 th ro u g h 8.
In a first-v ear g ra d u a te level c o u rse in digital signal p ro cessin g , th e first five
c h a p te rs p ro v id e th e s tu d e n t w ith a goo d rev iew of d isc re te -tim e system s. T h e
in stru c to r can m o v e q u ick ly th ro u g h m o st o f th is m aterial a n d th e n co v e r C h a p te rs
6 th ro u g h 9, fo llo w ed by e ith e r C h a p te rs 10 and 11 o r by C h a p te rs 11 an d 12.
W e h a v e in c lu d e d m an y ex am p les th ro u g h o u t th e b o o k an d a p p ro x im a te ly
500 h o m e w o rk p ro b le m s. M an y o f th e h o m ew o rk p ro b le m s can b e so lv ed n u m e r
ically on a c o m p u te r, using a so ftw are p ack ag e such as M A T L A B . T h e se p r o b
lem s a re id e n tifie d by an asterisk . A p p e n d ix D co n tain s a list o f M A T L A B fu n c
tio n s th a t th e s tu d e n t can use in solving th e se p ro b lem s. T h e in s tru c to r m ay also
w ish to c o n s id e r th e u se o f a s u p p le m e n ta ry b o o k th a t c o n ta in s c o m p u te r b ased
exercises, su c h as th e b o o k s Digilal Signal Processing Us ing M A T L A B (P.W .S.
K e n t, 1996) by V. K. In g le a n d J. G . P ro a k is a n d C o m p u te r- B a s e d Exercises f o r
S ignal P ro cessing Using M A T L A B (P re n tic e H all, 1994) by C. S. B u rru s e t al.
T h e a u th o rs a re in d e b te d to th e ir m an y facu lty c o lleag u es w ho h av e p ro v id e d
v alu ab le su g g e stio n s th ro u g h review s o f the first an d se co n d ed itio n s o f this b o o k .
T h e se in clu d e D rs. W . E . A le x a n d e r, Y. B re sle r, J. D e lle r, V. Ingle, C. K eller,
H . L e v -A ri, L. M e ra k o s , W. M ik h a e l, P. M o n ticcio lo , C. N ikias, M . S ch etzen ,
H . T ru ssell, S. W ilso n , a n d M. Z o lto w sk i. W e a re also in d e b te d to D r. R , P ric e fo r
re c o m m e n d in g th e in clu sion o f sp lit-ra d ix F F T alg o rith m s a n d re la te d su g g estio n s.
F in ally , w e w ish to ac k n o w le d g e th e su g g e stio n s an d c o m m e n ts o f m an y fo rm e r
g ra d u a te s tu d e n ts , a n d especially th o se by A . L. K ok, J. L in an d S. S rin id h i w ho
assisted in th e p r e p a r a tio n o f several illu stra tio n s an d th e so lu tio n s m an u al.
J o h n G . P ro a k is
D im itris G , M a n o lak is

Introduction

D ig ital signal p ro cessin g is an are a o f science a n d e n g in e e rin g th a t h a s d ev e lo p e d

rap id ly o v e r th e p ast 30 y ears. T his rap id d e v e lo p m e n t is a resu lt o f th e signif
ican t ad v an ce s in digital c o m p u te r tech n o lo g y an d in te g ra te d -c irc u it fab rica tio n .
T h e digital c o m p u te rs an d asso ciated digital h ard w are of th re e d e c a d e s ago w ere
relativ ely larg e an d ex p en siv e and, as a co n seq u en ce, th e ir use w as lim ited to
g e n e ra l-p u rp o s e n o n -re a l-tim e (o ff-line) scientific c o m p u ta tio n s an d business a p
p licatio n s. T h e ra p id d ev e lo p m e n ts in in te g ra te d -c irc u it te c h n o lo g y , sta rtin g with
m ed iu m -scale in te g ra tio n (M S I) an d p ro g ressin g to large-scale in te g ra tio n (L S I),
a n d now , v ery -larg e-scale in te g ra tio n (V L S I) of e le c tro n ic circuits has sp u rre d
th e d e v e lo p m e n t o f p o w erfu l, sm a ller, faster, an d c h e a p e r digital c o m p u te rs an d
sp e cial-p u rp o se d igital h a rd w a re . T h e se in ex p en siv e an d re lativ ely fast digital c ir
cuits h av e m a d e it p o ssib le to co n stru c t highly so p h istic a te d digital system s cap ab le
o f p e rfo rm in g co m p lex digital signal p ro cessin g fu n ctio n s a n d tasks, w hich are u su
ally to o difficult a n d /o r to o expensive to be p e rfo rm e d by an a lo g circuitry or a n alo g
signal p ro cessin g system s. H e n c e m an y of th e signal p ro cessin g task s th a t w ere
c o n v en tio n ally p e rfo rm e d by an alo g m e a n s a re realized to d a y by less ex p en siv e
an d o fte n m o re re lia b le digital h a rd w a re .
W e do n o t w ish to im ply th a t digital signal p ro cessin g is th e p ro p e r so lu
tio n fo r all signal p ro cessin g p ro b lem s. In d e e d , fo r m a n y signals w ith e x tre m e ly
w ide b a n d w id th s, real-tim e p ro cessin g is a re q u ire m e n t. F o r such signals, a n a
log o r, p e rh a p s, o p tical signal p ro cessin g is th e only p o ssib le so lu tio n . H o w ev er,
w h ere dig ital circuits are av ailab le an d h av e sufficient sp e e d to p e rfo rm th e signal
p ro cessin g , th ey a re usually p re fe ra b le .
N o t only d o d igital circuits yield c h e a p e r an d m o re re lia b le system s fo r signal
p ro cessin g , th e y h av e o th e r a d v an tag es as w ell. In p a rtic u la r, digital pro cessin g
h a rd w a re allow s p ro g ra m m a b le o p e ra tio n s. T h ro u g h so ftw are, on e can m o re easily
m o d ify th e sig n al p ro cessin g fu n ctio n s to b e p e rfo rm e d by th e h a rd w a re . T h u s
dig ital h a rd w a re a n d a s so ciated so ftw are p ro v id e a g re a te r d eg re e o f flexibility in
sy stem d esign. A lso , th e re is o ften a h ig h e r o rd e r of p re c isio n ach iev ab le w ith
d ig ital h a rd w a re an d so ftw are c o m p a re d w ith an alo g circu its a n d an alo g signal
p ro cessin g system s. F o r all th e se re a so n s, th e re h as b e e n an explosive grow th in
d ig ital signal p ro cessin g th e o ry a n d a p p licatio n s o v e r th e p ast th re e decades.

Introduction

Chap. 1

In this b o o k o u r o b jectiv e is to p re se n t an in tro d u c tio n o f th e basic analysis

to ols an d te c h n iq u e s fo r d igital p ro cessin g o f signals. W e b eg in by in tro d u c in g
so m e o f th e n ecessa ry term in o lo g y an d by d escrib in g th e im p o rta n t o p e ra tio n s
asso ciated w ith th e p ro cess of c o n v ertin g an an alo g signal to d ig ital fo rm su itab le
fo r d igital p ro cessin g . A s we shall se e, digital p ro cessin g o f a n a lo g signals has
som e d raw b ack s. F irst, an d fo re m o st, c o n v ersio n o f an a n a lo g signal to digital
fo rm , acco m p lish ed by sa m p lin g th e signal an d q u a n tiz in g th e sa m p le s, resu lts in a
d isto rtio n th a t p re v e n ts us fro m re c o n stru c tin g th e o rig in a l a n a lo g signal fro m the
q u a n tiz e d sam p les. C o n tro l o f th e a m o u n t o f th is d isto rtio n is ach ie v e d by p ro p e r
choice o f th e sam p lin g ra te a n d th e p recisio n in th e q u a n tiz a tio n p ro cess. S eco n d ,
th e re a re finite p re c isio n effects th a t m u st be c o n s id e re d in th e d igital pro cessin g
o f th e q u a n tiz e d sam p les. W hile th e se im p o rta n t issues are c o n s id e re d in som e
d etail in this b o o k , th e em p h asis is on th e analysis a n d d esig n o f digital signal
p ro cessin g sy stem s a n d c o m p u ta tio n a l te ch n iq u es.

1.1 SIGNALS, SYSTEMS, AND SIGNAL PROCESSING

A signal is d efin ed as any physical q u a n tity th a t varies w ith tim e, sp ace, o r any
o th e r in d e p e n d e n t v ariab le o r variables. M a th em atic ally , we d e sc rib e a signal as
a fu n ctio n o f o n e o r m o re in d e p e n d e n t variab les. F o r e x am p le, th e fu n ctio n s
* i( r ) = 5/

(1.1.1)
S2(t) = 20 r
d escrib e tw o signals, o n e th a t varies lin early w ith the in d e p e n d e n t v ariab le t (tim e)
an d a seco n d th a t v aries q u a d ra tic a lly w ith t. A s a n o th e r ex a m p le , co n sid e r the
fu n ctio n
v) = 3x + 2 x y + 1 0 y 2
(1.1.2)
T his fu n ctio n d escrib es a signal o f tw o in d e p e n d e n t v a riab les x a n d y th a t could
r e p re s e n t th e tw o sp a tia l c o o rd in a te s in a p lan e.
T h e signals d e sc rib e d by (1.1.1) an d (1.1.2) b e lo n g to a class o f signals th a t
are p recisely d efin ed by specifying th e fu n c tio n a l d e p e n d e n c e on th e in d e p e n d e n t
v ariab le. H o w ev er, th e re are cases w h ere such a fu n c tio n a l re la tio n sh ip is u n k n o w n
o r to o highly c o m p licated to be o f any p ractical use.
F o r ex am p le, a sp e ech signal (see Fig. 1.1) c a n n o t be d e s c rib e d fu n ctio n ally
by ex p ressio n s such as (1.1.1). In g e n eral, a se g m e n t o f sp e ech m ay be re p re se n te d
to a high d eg re e o f accu racy as a sum of se v era l sin u so id s o f d iffe re n t am p litu d e s
a n d freq u e n cies, th a t is, as
N
A j ( t ) s i n [ 2 ; r f } ( r ) f + #,(/)]

(1.1.3)

i=i
w h ere {/!,(/)}, {F ,(r)j, a n d {t9,(r)} a re th e se ts of (p o ssib ly tim e -v a ry in g ) a m p litu d es,
freq u e n cies, an d p h a se s, resp ectiv ely , o f th e sinusoids. In fact, o n e w ay to in te rp re t
th e in fo rm a tio n c o n te n t o r m essag e co n v ey ed by an y sh o rt tim e se g m e n t o f th e

Sec. 1.1

#
S

i j ^

Signals, Systems, and Signal Processing

Th

... ^

ft

---------- ,|* y y y > v y y m w

'r r m

'

w m

' W W W ......................

1
Figure 1.1

Example of a speech signal.

sp e e c h signal is to m e a s u re the a m p litu d es, freq u e n cies, a n d p h a se s c o n ta in e d in

th e sh o rt tim e se g m e n t o f the signal.
A n o th e r ex am p le o f a n a tu ra l signal is an e le c tro c a rd io g ra m (E C G ). Such a
signal p ro v id e s a d o c to r w ith in fo rm a tio n a b o u t th e co n d itio n o f the p a tie n t's h e a rt.
S im ilarly, an e le c tro e n c e p h a lo g ra m (E E G ) signal p ro v id es in fo rm a tio n a b o u t th e
activ ity o f th e b rain .
S p eech , e le c tro c a rd io g ra m , a n d e le c tro e n c e p h a lo g ra m signals a re ex am p les
o f in fo rm a tio n -b e a rin g signals th a t evolve as fu n ctio n s o f a single in d e p e n d e n t
v ariab le, n am elv , tim e. A n ex am p le o f a signal th at is a fu n ctio n o f tw o in d e
p e n d e n t v ariab les is an im age signal. T h e in d e p e n d e n t v ariab les in th is case are
th e sp atial c o o rd in a te s. T h e se a re b u t a few ex am p les o f th e co u n tless n u m b e r of
n a tu ra l signals e n c o u n te re d in practice.
A s so c ia te d w ith n a tu ra l signals are the m ean s by w hich such signals are g e n
e ra te d . F o r ex am p le, sp e ech signals are g e n e ra te d by fo rcin g air th ro u g h th e vocal
co rd s. Im ag es a re o b ta in e d by ex p o sin g a p h o to g ra p h ic film to a scene o r an o b
ject. T h u s signal g e n e ra tio n is usually asso ciated w ith a sy stem th a t re sp o n d s to a
stim u lu s o r fo rce. In a sp e ech signal, th e system consists o f th e vocal cords a n d
th e vocal tra c t, also called th e vocal cavity. T h e stim ulus in c o m b in a tio n w ith th e
sy stem is called a signal source. T h u s w e have sp eech so u rces, im ag es so u rces, an d
v ario u s o th e r ty p es o f signal sources.
A sy stem m ay also be defin ed as a physical device th a t p e rfo rm s an o p e r a
tio n on a signal. F o r e x am p le, a filter u sed to red u c e th e n o ise an d in te rfe re n c e
co rru p tin g a d e s ire d in fo rm a tio n -b e a rin g signal is called a system . In this case th e
filter p e rfo rm s so m e o p e ra tio n (s ) on th e signal, w hich h as th e effect o f red u cin g
(filterin g ) th e n o ise a n d in te rfe re n c e from th e d e sire d in fo rm a tio n -b e a rin g signal.
W h en w e pass a signal th ro u g h a system , as in filterin g , w e say th a t we h av e
p ro c e sse d th e signal. In this case th e p ro cessin g of th e signal involves filtering th e
n o ise an d in te rfe re n c e fro m th e d e s ire d signal. In g e n e ra l, th e system is c h a ra c
te riz e d by th e ty p e o f o p e ra tio n th a t it p e rfo rm s on th e signal. F o r ex am p le, if
th e o p e ra tio n is lin ear, th e system is called linear. If th e o p e ra tio n o n th e signal
is n o n lin e a r, th e system is said to be n o n lin e a r, a n d so fo rth . S uch o p e ra tio n s a re
u su a lly re fe rre d to as signal processing.

Introduction

Chap. 1

F o r o u r p u rp o se s, it is c o n v en ien t to b r o a d e n th e d efin itio n o f a system to

include n o t o n ly physical devices, b u t also so ftw are re a liz a tio n s o f o p e ra tio n s on
a signal. In d igital p ro cessin g o f signals on a digital c o m p u te r, th e o p e ra tio n s p e r
fo rm e d on a signal co n sist of a n u m b e r of m a th e m a tic a l o p e ra tio n s as specified by
a so ftw are p ro g ram . In this case, th e p ro g ra m r e p re s e n ts an im p le m e n ta tio n o f the
system in software. T h u s we h ave a system th a t is re a liz e d on a d igital c o m p u te r
by m ean s o f a se q u en ce o f m a th e m a tic a l o p e ra tio n s; th a t is, w e h av e a digital
signal p ro cessin g system realized in so ftw are. F o r e x am p le, a d ig ital c o m p u te r can
be p ro g ra m m e d to p e rfo rm digital filtering. A lte rn a tiv e ly , th e d igital processing
o n th e signal m ay be p e rfo rm e d by digital h ard w a re (logic circu its) co nfigured to
p e rfo rm th e d e sire d specified o p e ra tio n s. In such a re a liz a tio n , w e h av e a physical
d ev ice th a t p e rfo rm s th e specified o p e ra tio n s. In a b r o a d e r se n se, a digital system
can be im p le m e n te d as a c o m b in a tio n o f digital h a rd w a re an d so ftw are, each of
w hich p e rfo rm s its ow n set of specified o p e ra tio n s.
T h is b o o k d eals w ith th e p ro cessin g o f signals by digital m e a n s, e ith e r in so ft
w are o r in h a rd w a re . Since m an y of the signals e n c o u n te re d in p ra c tic e are analog,
w e will also co n sid er th e p ro b lem of c o n v ertin g an a n a lo g signal in to a digital sig
n al fo r pro cessin g . T h u s we will be d ealin g p rim a rily w ith d ig ital system s. T he
o p e ra tio n s p e rfo rm e d by such a system can u su ally be specified m ath em atically .
T h e m e th o d o r set o f ru les for im p le m e n tin g th e sy stem by a p ro g ra m th a t p e r
fo rm s th e c o rre sp o n d in g m a th e m a tic a l o p e ra tio n s is called an algorithm. U sually,
th e re are m an y w ays o r alg o rith m s by w hich a system can be im p le m e n te d , e ith e r
in so ftw are o r in h a rd w a re , to p e rfo rm th e d e sire d o p e ra tio n s a n d c o m p u tatio n s.
In p ra c tic e , we h av e an in te re st in devising a lg o rith m s th a t are c o m p u ta tio n a lly
efficient, fast, an d easily im p lem en ted . T h u s a m a jo r to p ic in o u r stu d y o f digi
tal signal p ro cessin g is th e discussion o f efficient a lg o rith m s fo r p e rfo rm in g such
o p e ra tio n s as filterin g , c o rre la tio n , an d sp e c tra l analysis.

1.1.1 Basic Elements of a Digital Signal Processing

System
M o st o f th e signals e n c o u n te re d in science an d e n g in e e rin g a re a n a lo g in n a tu re.
T h a t is. th e signals a re fu n ctio n s of a c o n tin u o u s v a ria b le , such as tim e o r space,
an d u su ally ta k e o n v alues in a co n tin u o u s ran g e. S uch signals m ay be p ro cessed
directly by a p p ro p ria te an alo g system s (such as filters o r fre q u e n c y an aly zers) or
fre q u e n c y m u ltip lie rs for th e p u rp o se of ch an g in g th e ir c h a ra c te ristic s o r ex tractin g
so m e d esired in fo rm a tio n . In such a case w e say th a t th e signal h as b e e n p ro cessed
d irectly in its an alo g fo rm , as illu strated in Fig. 1.2. B o th th e in p u t signal a n d the
o u tp u t signal a re in an a lo g form .

Analog
input
signal

Analog
signal
processor

Analog
output
signal
Figure 1.2

A nalog signal processing.

Sec. 1.1

Signals, Systems, and Signal Processing

5
Analog
output
signal

Analog
input

signal
Digital
input
signal
Figure 1.3

Digital
output
signal

Block diagram of a digital signal processing system.

D ig ital signal p ro cessin g p ro v id e s an a lte rn a tiv e m e th o d fo r p ro cessin g th e

a n a lo g signal, as illu stra te d in Fig. 1.3. T o p e rfo rm th e p ro cessin g digitally, th e re
is a n e e d fo r an in te rfa c e b e tw e e n th e an a lo g signal a n d th e digital p ro cesso r.
T h is in te rfa c e is called an analog-to-digital ( A / D ) converter. T h e o u tp u t of th e
A /D c o n v e rte r is a d ig ital signal th a t is a p p ro p ria te as an in p u t to th e d igital
p ro cesso r.
T h e dig ital signal p ro c e ss o r m ay be a larg e p ro g ra m m a b le digital c o m p u te r
o r a sm all m ic ro p ro c e s so r p ro g ra m m e d to p e rfo rm th e d e s ire d o p e ra tio n s on th e
in p u t signal. It m ay also be a h a rd w ire d digital p ro c e ss o r co n fig u red to p e rfo rm
a specified se t o f o p e ra tio n s on th e in p u t signal. P ro g ra m m a b le m ach in es p r o
v id e th e flexibility to ch an g e th e signal p ro cessin g o p e ra tio n s th ro u g h a ch an g e
in th e so ftw are, w h e re a s h a rd w ire d m ach in es a re difficult to reco n fig u re. C o n s e
q u e n tly , p ro g ra m m a b le signal p ro c e ss o rs a re in very c o m m o n use. O n th e o th e r
h an d , w h en signal p ro cessin g o p e ra tio n s are w ell d efin ed , a h a rd w ire d im p le m e n
ta tio n o f th e o p e ra tio n s can be o p tim ized , re su ltin g in a c h e a p e r signal p ro c e sso r
a n d , u su ally , o n e th a t ru n s fa ste r th a n its p ro g ra m m a b le c o u n te rp a rt. In a p p li
c atio n s w h e re th e d ig ital o u tp u t fro m th e d igital signal p ro c e sso r is to be given
to th e u se r in an alo g form , such as in sp e ech co m m u n icatio n s, w e m ust p r o
vid e a n o th e r in te rfa c e fro m th e digital d o m a in to th e a n a lo g d o m ain . S uch an
in te rfa c e is called a digital-to-analog ( D / A ) converter. T h u s th e signal is p r o
v id ed to th e u se r in an a lo g form , as illu stra te d in th e b lo ck d iag ram o f Fig. 1.3.
H o w e v e r, th e re a re o th e r p ractical a p p lic a tio n s involving signal analysis, w h ere
th e d e s ire d in fo rm a tio n is co n v ey ed in digital form a n d n o D /A c o n v e rte r is
re q u ire d . F o r ex am p le, in th e d igital p ro cessin g o f r a d a r signals, th e in fo rm a
tio n e x tra c te d fro m th e ra d a r signal, such as th e p o sitio n o f th e aircra ft a n d its
sp e ed , m ay sim ply b e p rin te d on p a p e r. T h e re is n o n e e d fo r a D /A c o n v e rte r in
th is case.

1.1.2 Advantages of Digital over Analog Signal

Processing
T h e re a re m an y re a so n s w hy d ig ital signal p ro cessin g o f an an alo g signal m ay be
p re fe ra b le to p ro cessin g th e signal directly in th e an a lo g d o m ain , as m e n tio n e d
briefly e a rlie r. F irst, a digital p ro g ra m m a b le sy stem allow s flexibility in r e c o n
figuring th e digital signal p ro cessin g o p e ra tio n s sim ply by ch anging th e p ro g ra m .

Introduction

Chap. 1

R e c o n fig u ra tio n o f an an a lo g system usually im plies a re d e sig n o f th e h a rd w a re

follow ed by te stin g a n d v erification to see th a t it o p e ra te s p ro p e rly .
A ccu racy c o n s id e ra tio n s also p lay an im p o rta n t role in d e te rm in in g th e fo rm
o f th e signal p ro cesso r. T o le ra n c e s in an alo g c ircu it c o m p o n e n ts m a k e it e x tre m e ly
difficult fo r th e system d esig n er to co n tro l th e accu racy o f an an a lo g signal p r o
cessing system . O n th e o th e r h an d , a digital system p ro v id e s m uch b e tte r c o n tro l
o f accu racy re q u ire m e n ts . Such re q u ire m e n ts , in tu rn , re s u lt in specifying th e a c
cu racy r e q u ire m e n ts in th e A /D c o n v e rte r a n d th e d igital sig n a l p ro c e sso r, in te rm s
o f w ord le n g th , flo atin g -p o in t v ersu s fix ed -p o in t arith m e tic , a n d sim ilar facto rs.
D ig ita l signals are easily sto re d o n m a g n e tic m ed ia (ta p e o r disk) w ith o u t d e
te rio ra tio n o r loss o f signal fidelity b e y o n d th a t in tro d u c e d in th e A /D co n v ersio n .
A s a c o n se q u e n c e , th e signals b e c o m e tra n s p o rta b le an d can b e p ro cessed off-line
in a re m o te la b o ra to ry . T h e digital signal p ro cessin g m e th o d also allow s for th e im
p le m e n ta tio n o f m o re so p h istic a te d signal p ro cessin g alg o rith m s. It is usually very
difficult to p e rfo rm p recise m a th e m a tic a l o p e ra tio n s on signals in a n a lo g fo rm b u t
th ese sam e o p e ra tio n s can b e ro u tin e ly im p le m e n te d on a d ig ital c o m p u te r using
so ftw are.
In so m e cases a d igital im p le m e n ta tio n of th e signal p ro cessin g system is
c h e a p e r th a n its an a lo g c o u n te rp a rt. T h e lo w er cost m ay be d u e to th e fact th a t
th e dig ital h a rd w a re is c h e a p e r, o r p e rh a p s it is a re su lt o f th e flexibility fo r m o d
ifications p ro v id e d by th e digital im p le m e n ta tio n .
A s a c o n se q u e n c e o f th ese ad v a n ta g e s, d igital signal p ro c e ssin g has b e e n
a p p lied in p ractical sy stem s co v erin g a b ro a d ra n g e of d iscip lin es. W e cite, fo r ex
am p le, th e a p p licatio n o f d igital signal p ro cessin g te c h n iq u e s in sp e ech p ro cessin g
an d signal tran sm issio n o n te le p h o n e ch an n els, in im age p ro c e ssin g an d tra n sm is
sio n , in seism o lo g y an d geophysics, in oil e x p lo ra tio n , in th e d e te c tio n of n u c le a r
ex p lo sio n s, in th e p ro cessin g of signals receiv ed fro m o u te r sp a ce, an d in a vast
v ariety o f o th e r a p p licatio n s. S om e o f th e se a p p lic a tio n s a re cited in su b s e q u e n t
ch ap ters.
A s a lre a d y in d icated , h o w ev er, digital im p le m e n ta tio n has its lim itatio n s.
O n e p ractical lim ita tio n is th e sp e ed o f o p e ra tio n o f A /D c o n v e rte rs a n d digital
signal p ro cesso rs. W e shall see th a t signals hav in g e x tre m e ly w id e b a n d w id th s re
q u ire fa st-sam p lin g -rate A /D c o n v e rte rs an d fast d igital signal p ro cesso rs. H e n c e
th e re a re an alo g signals w ith larg e b a n d w id th s fo r w hich a digital p ro cessin g a p
p ro a c h is b ey o n d th e s ta te of th e a rt o f digital h a rd w a re .

1.2 CLASSIFICATION OF SIGNALS

T h e m e th o d s we use in p ro cessin g a signal o r in an aly zin g th e re s p o n s e o f a system
to a sig n al d e p e n d h eavily on th e ch a ra c te ristic a ttr ib u te s o f th e specific signal.
T h e re a re te c h n iq u e s th a t ap p ly only to specific fam ilies o f signals. C o n seq u en tly ,
an y in v estig atio n in signal p ro cessin g sh o u ld sta rt w ith a classification o f th e signals
in v o lv ed in th e specific ap p licatio n .

Sec. 1.2

Classification of Signals

1.2.1 Multichannel and Multidimensional Signals

A s e x p lain ed in S ectio n 1.1, a signal is d escrib ed by a fu n c tio n o f o n e o r m o re
in d e p e n d e n t v ariab les. T h e v alue of th e fu n ctio n (i.e., th e d e p e n d e n t v ariab le) can
be a re a l-v a lu e d sc alar q u a n tity , a co m p lex -v alu ed q u a n tity , o r p e rh a p s a v ecto r.
F o r e x am p le, th e signal
si( r ) = A sin37rr
is a re a l-v a lu e d signal. H o w e v e r, th e signal
s2(f) = A e ji7Tt = A cos 37t t

j'A sin 3:r r

is co m p lex v alu ed .
In so m e a p p lic a tio n s, signals a re g e n e ra te d by m u ltip le so u rces or m u ltip le
sen so rs. Such signals, in tu rn , can be re p re s e n te d in v e c to r fo rm . F ig u re 1.4 show s
th e th re e c o m p o n e n ts of a v e c to r signal th a t re p re se n ts th e g ro u n d a c c e le ra tio n
d u e to an e a r th q u a k e . T h is a c c e le ra tio n is the re su lt of th re e basic ty p es of elastic
w aves. T h e p rim a ry (P ) w aves an d th e se co n d a ry (S) w aves p ro p a g a te w'ithin th e
b o d y o f rock a n d a re lo n g itu d in al a n d tra n sv e rsa l, resp ec tiv ely . T h e th ird ty p e
o f elastic w ave is called th e su rface w ave, b e c a u se it p ro p a g a te s n e a r th e g ro u n d
su rface. If $*(/). k = 1. 2. 3. d e n o te s th e electrical signal from th e th se n so r as a
fu n ctio n o f tim e, th e se t of p = 3 signals can be re p re se n te d by a v e c to r S?(f )< w h ere
r si (O '
S;,(r) =

Si(t)

-Sl(t) J
W e re fe r to such a v e c to r o f signals as a m u ltich a n n el signal. In e le c tro c a rd io g ra
p hy. for ex am p le, 3 -lead an d 12-lead e le c tro c a rd io g ra m s (E C G ) are o ften used in
p ractice, w hich resu lt in 3 -ch an n el a n d 12-channel signals.
L e t us n o w tu rn o u r a tte n tio n to th e in d e p e n d e n t v a ria b le (s). If the signal is
a fu n ctio n o f a single in d e p e n d e n t v ariab le, th e signal is called a o ne-d im en sio n a l
signal. O n th e o th e r h a n d , a signal is called M -d i m e n s i o n a l if its v alu e is a fu n ctio n
of M in d e p e n d e n t v ariab les.
T h e p ic tu re sh o w n in Fig. 1.5 is an ex am p le of a tw o -d im e n sio n al signal, since
th e in ten sity o r b rig h tn e ss I ( x . y) a t each p o in t is a fu n ctio n of tw o in d e p e n d e n t
v ariab les. O n th e o th e r h a n d , a b la c k -a n d -w h ite telev isio n p ic tu re m ay be r e p
r e se n te d as I ( x . y . t ) since th e b rig h tn e ss is a fu n ctio n of tim e. H e n c e th e T V
p ic tu re m ay b e tr e a te d as a th re e -d im e n s io n a l signal. In c o n tra st, a co lo r T V p ic
tu re m ay b e d e sc rib e d by th re e in te n sity fu n ctio n s of th e fo rm Ir (x, y. ?), Is (x. y. t ),
a n d I i , ( x . y , t ) , c o rre sp o n d in g to th e b rig h tn e ss of the th re e p rin cip al colors (red .
g re e n , b lu e) as fu n ctio n s o f tim e. H e n c e th e co lo r T V p ic tu re is a th re e -c h a n n e l,
th re e -d im e n s io n a l signal, w hich can b e re p re s e n te d by th e v e c to r
-/,(* ,> . O '
I U , y. t)
. l b(x, v ,r ) _
In this b o o k we d e a l m ainly w ith sin g le-ch an n el, o n e -d im e n sio n a l real- or
co m p lex -v alu ed signals a n d w e re fe r to th e m sim ply as signals. In m a th e m a tic a l

Introduction

Chap. 1

Up

/ % East

7jJL______
South

bouth

I____ i

]____ I.

f S waves

_4 P waves

1____ 1____ I

t Surface waves

i_______ I ,

10

r1 -2

I____ _ J _______I_______ I----------- 1-----------1----------- 1-----------1----------- 1-----------1

12

14

16

18

20

22

24

26

28

30

Time (seconds)
(b)
Figure 1.4 Three components of ground acceleration measured a few kilometers
from the epicenter of an earthquake. (From Earthquakes, by B. A . Bold. 1988
by W. H. Freeman and Company. Reprinted with permission of the publisher.)

te rm s th ese signals are d escrib ed by a fu n ctio n o f a single in d e p e n d e n t v ariable.

A lth o u g h th e in d e p e n d e n t variab le n e e d n o t be tim e, it is c o m m o n p ractice to use
t as th e in d e p e n d e n t v ariab le. In m an y cases th e signal p ro c e ssin g o p e ra tio n s and
a lg o rith m s d e v e lo p e d in this tex t for o n e -d im e n sio n a l, sin g le-ch an n el signals can
b e e x te n d e d to m u ltic h a n n e l an d m u ltid im e n sio n a l signals.

1.2.2 Continuous-Time Versus Discrete-Time Signals

Signals can b e fu rth e r classified in to fo u r d iffe re n t c a te g o rie s d e p e n d in g on the
ch a ra c te ristic s o f th e tim e (in d e p e n d e n t) v a ria b le an d th e v alu es th ey tak e.
Con tin u o u s -tim e signals o r a nalog signals a re d e fin ed for ev e ry value o f tim e an d

Sec. 1.2

Classification of Signals

Figure 1.5

Example of a two-dimensional signal.

th ey ta k e on v alu es in the co n tin u o u s in terv al (a . b ). w h e re a can be oc a n d b

can be oc. M a th em atic ally , th ese signals can be d e sc rib e d by fu n ctio n s o f a c o n
tin u o u s v ariab le. T h e sp eech w av efo rm in Fig. 1.1 an d th e signals x i(r) = c o s 7i t ,
x j { t ) = e ^ 1' 1, oc < t < oq are ex am p les o f an alo g signals. Discrete-time signals
a re d efin ed o n ly at c e rta in specific v alu es o f tim e. T h e se tim e in sta n ts n eed n o t be
e q u id ista n t, b u t in p ractice th ey are usually ta k e n a t e q u a lly sp a ced in terv als fo r
c o m p u ta tio n a l c o n v en ien ce an d m a th e m a tic a l tra c ta b ility . T h e signal x(t) =
n = 0, 1 , 2 , . . . p ro v id es an ex am p le o f a d isc re te -tim e signal. If we use th e
in d ex n o f th e d isc rete-tim e in sta n ts as th e in d e p e n d e n t v ariab le, th e signal v alu e
b eco m es a fu n ctio n o f an in te g e r v ariab le (i.e., a se q u e n c e of n u m b e rs). T h u s a
d isc re te -tim e signal can be re p re s e n te d m a th e m a tic a lly by a se q u e n c e of real o r
c o m p lex n u m b ers. T o em p h asize the d isc rete-tim e n a tu r e o f a signal, w e sh all
d e n o te such a signal as x{n) in ste a d o f x ( t ) . If th e tim e in stan ts t are e q u ally
sp a ced (i.e., t = n T ), th e n o ta tio n x ( n T ) is also used. F o r ex am p le, th e se q u e n c e
x(n)

if n > 0
o th erw ise

( 1 .2 . 1 )

is a d isc re te -tim e signal, w hich is r e p re s e n te d g rap h ically as in Fig. 1.6.

In ap p licatio n s, d isc rete-tim e signals m ay arise in tw o ways:
1. B y se lectin g v alu es o f an an alo g signal a t d isc re te -tim e in stan ts. T his p ro c e ss
is called s am plin g an d is discussed in m o re d etail in S ectio n 1.4. A ll m e a s u r
ing in stru m e n ts th a t ta k e m e a s u re m e n ts at a re g u la r in te rv a l o f tim e p ro v id e
d isc rete-tim e signals. F o r ex am p le, th e signal x ( n ) in Fig. 1.6 can be o b ta in e d

10

Introduction

Chap. 1

x{n)

I I T
Figure 1.6 Graphical representation of the discrete time signal x[n) = 0.8" for
n > 0 and x(n) = 0 for n < 0.

bv sa m p lin g th e an a lo g signal x ( t ) 0 .8 ', t > 0 an d x ( t ) = 0. t < 0 once

ev ery seco n d .

2. By accu m u latin g a v a ria b le o v e r a p e rio d o f tim e. F o r e x a m p le , c o u n tin g th e

n u m b e r o f cars using a given s tre e t every h o u r, o r re c o rd in g th e v alu e of gold
ev ery day, resu lts in d isc re te -tim e signals. F ig u re 1.7 show s a g rap h o f the
W o lfer su n sp o t n u m b e rs. E a c h sam ple o f this d isc re te -tim e signal p ro v id es
th e n u m b e r o f su n s p o ts o b se rv e d d u rin g an in te rv a l o f 1 y e a r.

1.2.3 Continuous-Valued Versus Discrete-Valued Signals

T h e v alu es o f a c o n tin u o u s-tim e or d isc re te -tim e signal can be c o n tin u o u s or d is
crete. If a signal ta k e s on all po ssib le values on a finite or an infinite ran g e, it

Year
Figure 1.7

W olfer annual sunspot num bers (1770-1869).

Sec. 1.2

Classification of Signals

11

is said to b e c o n tin u o u s-v a lu e d signal. A lte rn a tiv e ly , if th e signal ta k e s on v alu es

fro m a finite se t o f p o ssib le values, it is said to be a d isc re te -v a lu e d signal. U su ally ,
th e s e v alu es a re e q u id ista n t a n d h en ce can be e x p ressed as an in te g e r m u ltip le of
th e d istan c e b e tw e e n tw o successive values. A d isc re te -tim e signal hav in g a set of
d isc rete v alu es is called a digital signal. F ig u re 1,8 show s a d igital signal th a t ta k e s
o n o n e o f fo u r p o ssib le values.
In o r d e r fo r a signal to be p ro c e sse d digitally, it m u st be d isc rete in tim e
a n d its v alu es m u st b e d isc re te (i.e., it m ust b e a digital sig n al). If th e signal to
b e p ro c e sse d is in an a lo g fo rm , it is c o n v e rte d to a digital signal by sam pling th e
an alo g signal at d isc rete in stan ts in tim e, o b ta in in g a d isc re te -tim e signal, and th e n
by q ua n tizin g its v alu es to a set o f d isc re te v alu es, as d e sc rib e d la te r in th e c h a p te r.
T h e p ro cess o f co n v e rtin g a c o n tin u o u s-v a lu e d signal in to a d isc re te -v a lu e d signal,
called quan tizatio n, is basically an a p p ro x im a tio n p ro cess. It m ay be acco m p lish ed
sim ply bv ro u n d in g o r tru n c a tio n . F o r ex am p le, if th e allo w ab le signal v alu es
in th e d ig ital signal are in teg ers, say 0 th ro u g h 15, the c o n tin u o u s-v a lu e signal is
q u a n tiz e d in to th ese in te g e r values. T h u s th e signal v alue 8.58 will be a p p ro x im a te d
by th e v alu e 8 if th e q u a n tiz a tio n p ro cess is p e rfo rm e d by tru n c a tio n o r by 9 if
th e q u a n tiz a tio n p ro c e ss is p e rfo rm e d by ro u n d in g to th e n e a re s t in teg er. A n
ex p la n a tio n o f th e a n a lo g -to -d ig ita l co n v ersio n p ro cess is given la te r in th e c h a p te r.

Figure 1.8

Digital signal with four different amplitude values.

1.2.4 Deterministic Versus Random Signals

T h e m a th e m a tic a l an aly sis an d p ro cessin g o f signals re q u ire s th e availability o f a
m a th e m a tic a l d e sc rip tio n fo r th e signal itself. T h is m a th e m a tic a l d e scrip tio n , o fte n
re fe rre d to as th e signal m o d e l, lead s to a n o th e r im p o rta n t classification of signals.
A n y signal th a t can b e u n iq u ely d esc rib e d by an explicit m a th e m a tic a l ex p ressio n ,
a tab le o f d a ta , o r a w ell-defined ru le is called deterministic. T his te rm is used to
em p h asize th e fact th a t all p ast, p re se n t, an d fu tu re values o f th e signal are kno w n
precisely , w ith o u t an y u n c e rta in ty .
In m a n y p ractical ap p lic a tio n s, h o w ev er, th e re are sig n a ls th a t e ith e r c a n n o t
b e d esc rib e d to an y re a so n a b le d e g re e o f accu racy by explicit m a th e m a tic a l fo r
m u las, o r such a d e sc rip tio n is to o c o m p licated to b e of any p ractical use. T h e lack

12

Introduction

Chap. 1

o f such a re la tio n sh ip im plies th a t such signals ev o lv e in tim e in an u n p re d ic ta b le

m a n n e r. W e re fe r to th ese signals as ra n d o m . T h e o u tp u t o f a n oise g e n e ra to r,
th e seism ic signal o f Fig. 1.4, an d th e sp e ech signal in Fig. 1.1 are ex am p les of
ra n d o m signals.
F ig u re 1.9 show s tw o signals o b ta in e d fro m th e sa m e n o ise g e n e ra to r an d
th e ir asso ciated h isto g ram s. A lth o u g h th e tw o signals do n o t re s e m b le each o th e r
visually, th e ir h isto g ra m s re v e a l som e sim ilarities. T his p ro v id e s m o tiv a tio n fo r

3>4 1
---------------------------------------------------------- 1

200

400

600

-------*-------------------

800

1000

----- ------ ---------------------------------------

1200

1400

1600

(a)

400 f ----------------- ------------------- - -------------------------- ------------------- ------------------- 1

350 r

300 -

-j

250 <200 -

150 100 -

(b)

Figure 1.9
tograms.

Two random signals from the same signal generator and their his

Sec. 1.2

Classification of Signals

13

----- '
200

------------ '------------------------------------------------------------- --------1

400
600
800
1000
1200
1400
1600

<c)

Figure 1.9

C ontinued

th e an aly sis a n d d e sc rip tio n of ra n d o m signals using statistical te c h n iq u e s in stea d

o f ex p licit fo rm u las. T h e m ath e m a tic a l fra m e w o rk fo r th e th e o re tic a l analysis of
ra n d o m sig n als is p ro v id e d by th e th e o ry of p ro b a b ility a n d sto c h astic processes.
S o m e b asic e le m e n ts o f this a p p ro a c h , a d a p te d to th e n e e d s o f this bo o k , are
p re s e n te d in A p p e n d ix A .
It sh o u ld b e em p h a siz e d a t th is p o in t th a t th e classification o f a real-world
sig n al as d e te rm in is tic o r ra n d o m is n o t alw ays clear. S o m etim es, b o th a p p ro a c h e s
le a d to m e a n in g fu l resu lts th a t p ro v id e m o re insight in to signal b eh av io r. A t o th e r

Introduction

14

Chap. 1

tim es, th e w ro n g classification m ay le a d to e rro n e o u s resu lts, since som e m a th e

m atical to o ls m ay ap p ly o n ly to d e te rm in istic signals w hile o th e rs m ay apply only
to ra n d o m signals. T h is will b e c o m e c le a re r as w e ex am in e specific m a th e m a tic a l
tools.

1.3 THE CONCEPT OF FREQUENCY IN CONTINUOUS-TIME AND

DISCRETE-TIME SIGNALS
T h e co n cep t o f fre q u e n c y is fam iliar to stu d e n ts in e n g in e e rin g a n d th e sciences.
T h is co n cep t is basic in. fo r ex am p le, th e design of a rad io re c e iv e r, a high-fidelity
sy stem , o r a sp e c tra l filter fo r co lo r p h o to g ra p h y . F ro m physics w e k n o w th a t
freq u e n cy is closely re la te d to a specific ty p e o f p e rio d ic m o tio n called h arm o n ic
o scillatio n , w hich is d e s c rib e d by sin u so id al fu n ctio n s. T h e c o n c e p t o f freq u e n cy
is d irectly re la te d to th e c o n c e p t o f tim e. A ctu ally , it has th e d im en sio n of inverse
tim e. T h u s w e sh o u ld e x p ect th a t th e n a tu re of tim e (c o n tin u o u s o r d isc rete) w ould
affect th e n a tu re o f th e fre q u e n c y accordingly.

1.3.1 Continuous-Time Sinusoidal Signals

A sim ple h a rm o n ic o sc illatio n is m a th e m a tic a lly d escrib ed
c o n tin u o u s-tim e sin u so id al signal:
x a(t) = A cos(Q t + 0). oc < t < oc

by th e follow ing
(1.3.1)

show n in Fig. 1.10. T h e su b sc rip t a u se d w ith x { t ) d e n o te s an an a lo g signal. T his

signal is co m p letely c h a ra c te riz e d by th re e p a ra m e te rs: A is th e a m p litu d e of the
sin u so id . ft is th e fre q u e n c y in ra d ia n s p e r se co n d (rad /s), a n d 6 is th e p h a se in
rad ian s. In ste a d o f ft, we o fte n use th e fre q u e n c y F in cycles p e r seco n d o r h e rtz
(H z), w h ere
Q In F

(1.3.2)

In term s o f F. (1.3.1) can be w ritte n as

x a(t) = A cos(2n F t + 6 ), oo < t < oc

(1.3.3)

W e will use b o th fo rm s, (1.3.1) an d (1.3.3), in re p re s e n tin g sin u so id al signals.

x j t ) = A cos(2nFt + 8)

Figure 1.10 Example of an analog

sinusoidal signal.

Sec. 1.3

Frequency Concepts in Continuous-Discrete-Tim e Signals

15

T h e an a lo g sin u so id al signal in (1.3.3) is c h a ra c te riz e d by th e follow ing p r o p

erties:
A L F o r ev ery fixed v alu e o f th e fre q u e n c y F, x a(r) is p erio d ic. In d e e d , it can
easily b e sh o w n , using e le m e n ta ry trig o n o m e try , th a t
x a(.t + Tp ) = A(r)
w h e re Tp = 1 / F is th e fu n d a m e n ta l p e rio d o f the sin u so id al signal.
A 2 . C o n tin u o u s -tim e sin u so id al signals w ith d istin ct (d iffe re n t) fre q u e n c ie s a re
th e m se lv e s d istin ct.
A 3 . In c re a sin g th e fre q u e n c y F resu lts in an in crease in th e ra te o f o sc illatio n
o f th e signal, in th e sense th a t m o re p e rio d s are included in a given tim e
in terv al.
W e o b se rv e th a t fo r F = 0. th e value Tp oc is co n sisten t w ith the fu n
d a m e n ta l re la tio n F = 1 / T r . D u e to co n tin u ity o f th e tim e v a ria b le r, w e can
in crease th e fre q u e n c y F, w ith o u t lim it, w ith a c o rre sp o n d in g increase in th e ra te
o f oscillatio n .
T h e re la tio n sh ip s we h ave d e sc rib e d fo r sin u so id al signals carry o v er to th e
class o f co m p lex e x p o n e n tia l signals
xa {t) - A e JlSi,+{"

(1.3.4)

T h is can easily b e seen by e x p ressin g th e se signals in te rm s o f sinusoids using th e

E u le r id en tity
ej4: = cos <p i j sin <p

(1.3.5)

B y d efin itio n , freq u e n c y is an in h e re n tly p o sitiv e physical q u an tity . T his

is o b v io u s if we in te r p re t fre q u e n c y as the n u m b e r o f cycles p e r u n it tim e in a
p e rio d ic signal. H o w e v e r, in m any cases, only fo r m a th e m a tic a l co n v en ien ce , w e
n e e d to in tro d u c e n e g a tiv e freq u e n cies. T o see this w e recall th a t th e sin u so id al
signal (1.3.1) m ay be e x p ressed as
xa (t) = A c o s (^ r + 6 ) = j

eJ(Q,+f>) + ~ e - J(a+9)

(1.3.6)

w hich follow s fro m (1.3.5). N o te th a t a sin u so id al signal can be o b ta in e d by ad d in g

tw o e q u a l-a m p litu d e c o m p lex -co n ju g ate e x p o n e n tia l signals, so m e tim e s called p h aso rs, illu stra te d in Fig. 1.11. A s t i m e p ro g re sse s th e p h a s o rs ro ta te in o p p o site
d ire c tio n s w ith a n g u la r fre q u e n c ie s 2 ra d ia n s p e r se co n d . Since a positive f r e
q u e n c y c o rre sp o n d s to co u n te rc lo c k w ise u n ifo rm a n g u la r m o tio n , a negative f r e
q u e n c y sim ply c o rre sp o n d s to clockw ise a n g u la r m o tio n .
F o r m a th e m a tic a l c o n v en ien ce , w e use b o th n e g a tiv e a n d po sitiv e freq u e n cies
th ro u g h o u t th is b o o k . H e n c e th e fre q u e n c y ra n g e fo r a n a lo g sinusoids is oo <
F < oo.

16

Introduction

Chap. 1

Re

Figure 1.11 Representation of a cosine

function by a pair of complex-conjugate
exponentials (phasors).

1.3.2 Discrete-Time Sinusoidal Signals

A d isc rete-tim e sin u so id al signal m ay be e x p ressed as
x ( n ) A cos (ton + 8), oo < n < oc

(1.3.7)

w h ere n is an in te g e r v ariab le, called th e sam p le n u m b e r. A is th e am plitu de o f the

sin u so id , co is th e fr e q u e n c y in rad ian s p e r sa m p le , and 8 is th e p h a s e in radians.
If in ste a d o f a> we use the freq u e n cy v ariab le / defin ed by
a; = 2 n f

(1.3.8)

x( n ) A cos(2n f n + 8). oc < n < oc

(1.3.9)

th e re la tio n (1.3.7) b eco m es

T h e freq u e n cy / h as d im en sio n s o f cycles p e r sa m p le . In S ectio n 1.4. w here

we co n sid e r th e sam p lin g o f an alo g sinusoids, w e re la te th e fre q u e n c y v ariab le
/ o f a d isc re te -tim e sin u so id to th e fre q u e n c y F in cycles p e r se co n d fo r the
an a lo g sin u so id . F o r th e m o m e n t we co n sid er th e d isc re te -tim e sin u so id in (1.3.7)
in d e p e n d e n tly of th e co n tin u o u s-tim e sinusoid given in (1.3.1). F ig u re 1.12 show s
a sin u so id w ith fre q u e n c y co n /6 ra d ia n s p e r sa m p le ( f ~ ~ cycles p e r sam ple)
a n d p h ase 8 n / 3 .
x(n) - A cos (urn + 8)

Figure 1.12 Example of a discrete-time

sinusoidal signal (w 7t / 6 and b 7r/3).

Sec. 1.3

Frequency Concepts in Continuous-Discrete-Tim e Signals

17

In c o n tra s t to co n tin u o u s-tim e sinusoids, th e d isc re te -tim e sin u so id s are c h a r

a c te riz e d by th e fo llow ing p ro p e rtie s:
B l . A discrete-time sinu so id is p er iod ic o n ly i f its fr e q u e n c y f is a rational n u m b e r .
B y d efin itio n , a d isc rete-tim e signal x( n ) is p erio d ic w ith p e rio d N ( N > 0) if
a n d only if
x ( n + N ) = x(n )

fo r all n

(1.3.10)

T h e sm a llest v alu e o f N fo r w hich (1.3.10) is tru e is called th e f u n d a m e n ta l p eriod .

T h e p r o o f o f th e p erio d icity p ro p e rty is sim ple. F o r a sin u so id w ith fre q u e n c y
/o to b e p e rio d ic , we sh o u ld have
cos[27t /o( A7 + n) + 8} co s(2 ,t/o + 6)
T h is re la tio n is tru e if an d only if th e re exists an in te g e r k such th a t
2 n f ) N = 2kn
o r, eq u iv alen tly .
/o = 4
N

(1.3.11)

A cc o rd in g to (1.3.11). a d isc rete-tim e sin u so id al signal is p e rio d ic only if its f re

q u e n c y /o can be ex p re sse d as th e ra tio o f tw o in teg ers (i.e.. / () is ra tio n a l).
T o d e te rm in e th e fu n d a m e n ta l p e rio d N o f a p e rio d ic sin u so id , w e e x p ress its
fre q u e n c y /o as in (1.3.11) an d cancel com m on facto rs so th a t k a n d N are relativ ely
p rim e . T h e n th e fu n d a m e n ta l p e rio d o f the sin u so id is e q u a l to N. O b serv e th a t a
sm all ch an g e in fre q u e n c y can resu lt in a large change in th e p erio d . F o r ex am p le,
n o te th a t f \ = 3 1 /6 0 im plies th at N\ = 60, w h ereas f i = 3 0 /6 0 resu lts in Nz = 2.
B 2. Disc rete-tim e sinu so ids whose fre q u en cies are separa ted by an integer m ultiple
o f 2 n are identical.
T o p ro v e this assertio n , let us c o n sid er th e sin u so id cos(oo + 0). It easily
fo llow s th a t
cos[(wo + 2 n )n + B\ = cos(wo/i + 2 n n + 9) co%{a)Qn + 9)

(1.3.12)

A s a re su lt, all sin u so id al se q u en ces

x k(n) A cos(a>tn + 8).

k = 0 ,1 ,2 ,...

(1.3.13)

w h ere
Wk = cl>c + 2k n ,

7r < o>o < n

are in distinguishable (i.e., identical). O n th e o th e r h a n d , th e se q u e n c e s o f any tw o

sin u so id s w ith fre q u e n c ie s in th e ra n g e - n < a> < n or ^ < f < ~ a re distinct.
C o n s e q u e n tly , d isc rete-tim e sinusoidal signals w ith fre q u e n c ie s M < n o r | / | < \

Introduction

18

Chap. 1

are u n iq u e. A n y se q u e n c e re su ltin g fro m a sin u so id w ith a fre q u e n c y M > n , or

| / | > j , is id en tical to a se q u e n c e o b ta in e d fro m a sin u so id al signal w ith fre q u e n c y
\co\ < n . B e c a u se o f th is sim ilarity, we call th e sin u so id h av in g th e freq u e n cy M >
tt an alias o f a c o rre sp o n d in g sinusoid w ith fre q u e n c y jwj < n . T h u s w e re g a rd
freq u e n cies in th e ra n g e tt < a> < tt, o r 1 < / < 1 as u n iq u e a n d all freq u e n cies
|o>[ > t t , o r | / | > ~, as aliases. T h e r e a d e r sh o u ld n o tice th e d iffe re n c e b e tw e e n
d isc rete-tim e sin u so id s an d co n tin u o u s-tim e sinusoids, w h e re th e la tte r re su lt in
d istin ct signals fo r 2 o r F in th e e n tir e ran g e oc < 2 < oc o r oc < F < oc.
B 3. The highest rate o f oscillation in a discrete-time sin u s o i d is attained whe n
to 7i (or cu = tt) or, eq uivalently, f \ (or f = \ ) .
T o illu stra te th is p ro p e rty , let u s in v estig ate th e c h a ra c te ristic s of th e sin u
so id a l signal se q u e n c e
x ( n ) = cos c^on
w h en th e fre q u e n c y v aries fro m 0 to tt. T o sim plify th e a rg u m e n t, we ta k e values
o f (l>
o = 0, 7t / 8, tt/4 , jt/2 , n c o rre sp o n d in g to / = 0,
5,
w hich resu lt in
p erio d ic se q u e n c e s h av in g p e rio d s N = oc, 16, 8, 4, 2. as d e p ic te d in Fig. 1.13. W e
n o te th a t th e p e rio d o f th e sinusoid d e c re a se s as th e fre q u e n c y in creases. In fact,
we can see th a t th e ra te o f o scillatio n in crease s as th e fre q u e n c y increases.
,, xin)

Figure 1.13

Signal x ( n ) = c o s cu^n for various values of the frequency

cdq.

Sec. 1.3

Frequency Concepts in Continuous-Discrete-Time Signals

19

T o see w h at h a p p e n s for tt < ioq < 2tt . we co n sid e r th e sinusoids w ith

fre q u e n c ie s a>\ = a>(> a n d 0J2 2 n ojq. N o te th at as co\ varies from tt to 2n . a>z
v aries fro m ir to 0. it can be easily se en th at
= A cos co} n A cos won
X2 (n) = A cos uhn A cos(27r coo)n

(1.3.14)

= A c o s (coqii) x \ (h)
H e n ce ur is an alias o f w\. If we h ad u sed a sine fu n ctio n in stea d o f a cosine fu n c
tio n , th e resu lt w ou ld basically be th e sam e, ex cep t fo r a 180' p h a se d ifferen ce
b etw een th e sin u so id s A]() and xi ( n ) . In any case, as we increase th e re lativ e
freq u e n cy coo o f a d isc re te -tim e sin u so id from tt to 27r. its ra te of o sc illatio n d e
creases. F o r coo = 2 tt th e re su lt is a c o n sta n t signal, as in th e case fo r oju = 0.
O b v io u sly , fo r co{) = tt (o r f = k) w e h ave th e hig h est ra te o f oscillation.
A s fo r th e case o f co n tin u o u s-tim e signals, n eg ativ e fre q u e n c ie s can be in
tro d u c e d as w ell for d isc rete-tim e signals. F o r this p u rp o se w e use th e id en tity
A

x (n ) = Acos(con + 0) = (>Jiwn+0) +

(1.3.15)

Since d isc re te -tim e sinusoidal signals w ith fre q u e n c ie s th a t are se p a ra te d by

an in teg er m u ltip le o f 27r are iden tical, it follow s th a t th e fre q u e n c ie s in any in terv al
co] < a> < co\ + 2 tt c o n stitu te all the existing d isc rete-tim e sinusoids o r com plex
e x p o n en tials. H en ce th e fre q u e n c y ran g e fo r d isc rete-tim e sinusoids is finite with
d u ra tio n 2 n . U sually, we choose th e ran g e 0 < co < 2n o r tt < co < tt ({) < f < 1.
1 < / < | ) , w hich we call th e f u n d a m e n t a l range.

1.3.3 Harmonically Related Complex Exponentials

S in u so id al signals a n d com plex e x p o n e n tia ls play a m a jo r role in th e analysis o f
signals an d system s. In so m e cases we deal w ith sets of h arm on ic a lly related c o m
plex e x p o n e n tia ls (o r sin u so id s). T h e se are sets of p e rio d ic co m p lex e x p o n e n tia ls
w ith fu n d a m e n ta l fre q u e n c ie s th a t are m u ltip le s o f a single positive freq u e n cy .
A lth o u g h we confine o u r discussion to com plex e x p o n e n tia ls, th e sam e p r o p e r
ties clearly hold fo r sin u so id al signals. W e co n sid e r h a rm o n ically re la te d co m p lex
e x p o n e n tia ls in b o th co n tin u o u s tim e and d isc rete tim e.

Continuous-time exponentials.

T h e basic signals for co n tin u o u s-tim e ,

h arm o n ically re la te d e x p o n e n tia ls are

sk(t) = ejkno' = e ll7TkFn'

jt = 0 . l . 2 . . . .

(1.3.16)

W e n o te th a t for ea c h value o f k, s^U) is p erio d ic w ith fu n d a m e n ta l p e rio d

1 /(kFo) = Tp/ k o r fu n d a m e n ta l freq u e n cy kFo. Since a signal th a t is p e rio d ic
w ith p e rio d Tp / k is also p e rio d ic w ith p erio d k ( T p/ k ) = Tp fo r any positive in te g e r
k, w e see th a t all o f th e s*(r) h av e a co m m o n p e rio d of Tp, F u rth e rm o re , acco rd in g

20

Introduction

Chap. 1

to S ectio n 1.3.1, Fo is allo w ed to ta k e any v alu e an d all m e m b e rs of th e set are

d istin ct, in th e se n se th a t if k\ ^ k2, th en 5*1 (7) ^
F ro m th e basic signals in (1.3.16) we can c o n s tru c t a lin e a r c o m b in a tio n of
h arm o n ically re la te d co m p lex ex p o n e n tia ls o f th e form
cc

SC
=

x ( ' ) =

Ckelkiltit

k oc

( 1 -3 -1 7 )

k ~ oc

w h ere ck, k = 0, 1 , 2 . . . . are a rb itra ry co m p lex co n sta n ts. T h e signal x a(t)

is p erio d ic w ith fu n d a m e n ta l p erio d Tp = l / f o , a n d its r e p re s e n ta tio n in term s
o f (1.3.17) is called th e F ourier series ex p an sio n fo r x a (t). T h e co m p lex -v alu ed
co n sta n ts are th e F o u rie r se rie s coefficients a n d th e signal sk (r) is called th e fcth
h a rm o n ic o f x (l(t).

Discrete-time exponentials. Since a d isc re te -tim e co m p le x e x p o n e n tia l is

p erio d ic if its relativ e fre q u e n c y is a ra tio n a l n u m b e r, w e ch o o se f Q 1/A' an d we
d efine th e sets o f h arm o n ically re la te d co m p lex e x p o n e n tia ls by
k = 0. 1 . 2 , . . .

sk (n) = ej2* kf" \

(1.3.18)

In c o n tra st to th e c o n tin u o u s-tim e case, we n o te th at

sk+Nln) = eJ7* n'k+N,/N = e ^ s k (n) = sk (n)
T his m ean s th a t, c o n siste n t w ith (1.3.10), th e re are only N d istin ct p e rio d ic com plex
e x p o n en tials in th e se t d e sc rib e d by (1.3.18). F u rth e rm o re , all m e m b e rs of the set
h av e a co m m o n p e rio d o f N sam ples. C learly, w e can ch o o se a n y co n secu tiv e A'
co m p lex e x p o n e n tia ls, say from k = no to k no 4- N 1 to fo rm a h arm o n ically
re la te d set w ith fu n d a m e n ta l freq u e n cy /(, = 1 / N . M o st o fte n , fo r co n v en ien ce ,
we ch o o se th e set th a t c o rre sp o n d s to no = 0 , th a t is, th e set
* = 0 . 1 . 2 .........N - 1

sk(n) = ejl n k n / s .

(1.3.19)

A s in th e case o f c o n tin u o u s-tim e signals, it is o b v io u s th a t th e lin e a r co m

b in atio n
N- 1

\-l

x ( n ) = c * s * ( n ) = Y L ckei2nkn' N
k= 0

Jt= 0

resu lts in a p e rio d ic signal w ith fu n d a m e n ta l p e rio d N . A s w e shall see later,

this is th e F o u rie r series re p re s e n ta tio n for a p e rio d ic d isc re te -tim e se q u e n c e w ith
F o u rie r co efficien ts {q}. T h e se q u e n c e sk (n) is called th e /tth h a rm o n ic o f x(n ).
Example 1.3.1
Stored in the memory of a digital signal processor is one cycle of the sinusoidal signal
. ( 2nn

x ( n ) = sin I

+ 6

where 6 2 n q / N , where q and N are integers.

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

21

(a) Determ ine how this table of values can be used to obtain values of harmonically
related sinusoids having the same phase.
(b) D eterm ine how this table can be used to obtain sinusoids of the same frequency

but different phase.

Solution

(a) Let

denote the sinusoidal signal sequence

( 2yrnk
xk(n) = sin I --------- !
V N

This is a sinusoid with frequency f k = k / N. which is harmonically related to

x{n). But xk{n) may be expressed as
xk{n) = sin

2 tt ( k n )

= x(kn)
Thus we observe that ,vt (0) = .v(0). **(1) = x ( k ) . x k (2) = x ( 2 k ) . and so on.
Hence the sinusoidal sequence
can be obtained from the table of values
of x ( n ) by taking every k th value of * ( ) . beginning with .v(0). In this m anner we
can generate the values of all harmonically related sinusoids with frequencies
fk = k / N for k = 0. 1....... N - 1.
(b) We can control the phase 8 of the sinusoid with frequency j\ k / N by taking

the first value of the sequence from memory location q 9 N/ 2 tt. where q is
an integer. Thus the initial phase 6 controls the starling location in the table
and we wrap around the table each time the index (kn) exceeds N.

1.4 ANALOG-TO-DIGITAL AND DIGITAL-TO-ANALOG CONVERSION

M o st sig n als o f p ractical in terest, such as sp e ech , bio lo g ical signals, seism ic signals,
ra d a r signals, so n a r signals, and v ario u s co m m u n ic a tio n s signals such as au d io a n d
v id eo signals, are an alo g . T o p ro cess an a lo g signals by digital m e a n s , it is first
n ecessa ry to c o n v e rt th e m into digital form , th a t is, to c o n v e rt th e m to a se q u e n c e
o f n u m b e rs h av in g finite precision. T his p ro c e d u re is called analog-to-digital ( A / D )
co n v e r sio n , an d th e c o rre sp o n d in g devices a re called A / D converters ( A D C s ) .
C o n c e p tu a lly , w e view A /D co n v ersio n as a th re e -s te p p ro cess. T his p ro cess
is illu stra te d in Fig. 1.14.
1. S am p lin g . T his is th e c o n v ersio n o f a c o n tin u o u s-tim e signal in to a d is c re te
tim e signal o b ta in e d by tak in g sa m p le s" o f th e c o n tin u o u s-tim e signal at
d isc re te -tim e in stan ts. T h u s, if x a(t) is th e in p u t to th e sa m p le r, th e o u tp u t
is x a ( n T ) = x ( n ), w h ere T is called th e s a m p lin g interval.
2. Q ua ntiza tio n . T h is is th e co n v ersio n o f a d isc re te -tim e c o n tin u o u s-v a lu e d
signal in to a d isc rete-tim e, d isc rete-v alu ed (d ig ital) signal. T h e v alu e o f ea c h

Introduction

22

Chap. 1

A/D converter

01011...
"7

Analog
signal

Discrete-time
signal

Quantized
signal

Digital
signal

Figure 1.14 Basic parts of an analog-to-digital (A /D ) converter.

signal sam p le is re p re s e n te d by a v alu e se lected from a finite set o f p o ssi
b le values. T h e d ifferen ce b e tw e e n th e u n q u a n tiz e d sa m p le x ( n ) an d the
q u a n tiz e d o u tp u t x q(n) is called th e q u a n tiz a tio n erro r.
3. Coding. In th e co d in g p ro cess, each d isc rete v alu e x q{n) is re p re s e n te d by a
6 -b it b in ary se q u en ce.
A lth o u g h w e m o d e l th e A /D c o n v e rte r as a sa m p le r follow ed by a q u a n tiz e r
an d co d er, in p ractice th e A /D co n v ersio n is p e rfo rm e d by a single device th at
ta k e s x a(t) an d p ro d u c e s a b in a ry -c o d e d n u m b e r. T h e o p e ra tio n s of sa m p lin g an d
q u a n tiz a tio n can be p e rfo rm e d in e ith e r o rd e r b u t. in p ractice, sa m p lin g is alw ays
p e rfo rm e d b e fo re q u a n tiz a tio n .
In m an y cases o f p ractical in te re st (e.g., sp e ech p ro cessin g ) it is d esirab le
to co n v ert th e p ro cessed digital signals in to an a lo g form . (O b v io u sly , we can n o t
listen to th e se q u e n c e of sa m p le s re p re se n tin g a sp e ech signal o r see th e n u m
b ers co rre sp o n d in g to a T V signal.) T h e p ro cess o f co n v e rtin g a digital signal
in to an an alo g signal is kno w n as digital-to-analog ( D / A ) co nversion. A ll D /A
c o n v e rte rs c o n n ect th e d o ts " in a digital signal by p e rfo rm in g so m e kind of in te r
p o la tio n , w hose accu racy d e p e n d s on th e q u ality of th e D /A c o n v ersio n process.
F ig u re 1.15 illu strates a sim ple fo rm o f D /A c o n v ersio n , called a z e ro -o rd e r hold
o r a sta ircase a p p ro x im a tio n . O th e r a p p ro x im a tio n s a re p o ssib le, such as lin early
co n n ectin g a p a ir o f successive sa m p le s (lin e a r in te rp o la tio n ), fittin g a q u a d ra tic
th ro u g h th re e successive sa m p le s (q u a d ra tic in te rp o la tio n ), an d so on. Is th e re an
o p tim u m (ideal) in te rp o la to r? F o r signals hav in g a limited f r e q u e n c y co ntent (finite
b an d w id th ), th e sa m p lin g th e o re m in tro d u c e d in th e follow ing se c tio n specifies the
o p tim u m form o f in te rp o la tio n .
S am p lin g a n d q u a n tiz a tio n are tr e a te d in th is sectio n . In p a rtic u la r, we
d e m o n s tra te th a t sa m p lin g d o e s n o t re su lt in a loss of in fo rm a tio n , n o r d o es it
in tro d u c e d isto rtio n in th e signal if th e signal b a n d w id th is finite. In p rin cip le , th e
an alo g signal can b e re c o n stru c te d from th e sam ples, p ro v id e d th a t th e sam p lin g
ra te is sufficiently high to avoid th e p ro b le m co m m o n ly called aliasing. O n th e
o th e r h an d , q u a n tiz a tio n is a n o n in v e rtib le o r irre v e rsib le p ro c e ss th a t resu lts in
signal d isto rtio n . W e shall sh o w th a t th e a m o u n t o f d isto rtio n is d e p e n d e n t on

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

Original
Signal

23

Staircase
Approximation

/
/
/

V
*o

/
/

/
-/
I
I
o

67Time

Figure 1.15

Zero-ordcr hold digital-to-analog (D /A ) conversion.

th e accu racy , as m e a s u re d by th e n u m b e r of bits, in th e A /D c o n v ersio n process.

T h e facto rs affe ctin g the choice of th e d esired accu racy of th e A /D c o n v e rte r are
cost an d sam p lin g ra te . In g en eral, th e cost in crease s w ith an in crease in accuracy
a n d /o r sa m p lin g rate.

1.4.1 Sampling of Analog Signals

T h e re are m an y ways to sa m p le an an a lo g signal. W e lim it o u r discussion to
p er iod ic o r u n i f o r m s a m p li n g , w hich is th e ty p e of sa m p lin g used m o st o ften in
p ractice. T h is is d escrib ed by the re la tio n
x(n) = xa (nT).

o c < n < c c

(1.4.1)

w h ere x ( n ) is th e d isc re te -tim e signal o b ta in e d by ta k in g sa m p le s o f th e an alo g

signal x aU) ev ery T se co n d s. T his p ro c e d u re is illu stra te d in Fig. 1.16. T h e tim e
in te rv a l T b etw e e n successive sa m p les is called th e sa m p li n g p er io d o r sa m p le
interval an d its recip ro c al 1 / 7 Fs is called the s a m p lin g rate (sam p les p e r second)
o r th e sa m p lin g f r e q u e n c y (h ertz).
P e rio d ic sa m p lin g estab lish es a re la tio n sh ip b e tw e e n th e tim e v ariab les t an d
n o f c o n tin u o u s-tim e an d d isc re te -tim e signals, resp ec tiv ely . In d e e d , th e s e v a ri
ab les a re lin early re la te d th ro u g h th e sa m p lin g p e rio d T or, e q u iv alen tly , th ro u g h
th e sa m p lin g ra te Fs l / 7 \ as
(1.4.2)
A s a c o n s e q u e n c e o f (1.4.2), th e re exists a re la tio n sh ip b e tw e e n th e freq u e n cy
v a ria b le F (o r Q) fo r an a lo g signals a n d the freq u e n c y v a ria b le / (o r co) for
d isc re te -tim e signals. T o estab lish th is re la tio n sh ip , co n sid e r an an alo g sinusoidal
signal o f th e fo rm
x B{t) = A c o s ( 2 t t F t + 8 )

(1.4.3)

24

Introduction

*(n) = xa(nT)

Analog
signal

Fs = 1IT

Chap. 1

Discrete-time
signal

Sampler

Figure 1.16

Periodic sampling of an analog signal.

w hich, w h en sa m p le d p erio d ically at a ra te Fs 1 / 7 sa m p le s p e r se co n d , yields

x a( n T ) == x{ n) = A cos(27 t F n T -f 9)
/2 n n F
\
= A cos (
+ 9 \

(1-4.4)

If we c o m p are (1.4.4) w ith (1.3.9). w e n o te th a t th e fre q u e n c y v ariab les F

an d / a re lin early re la te d as

Is
or, eq u iv alen tly , as
co = Q T

(1.4.5)
Fs=sampling frequency
F=frequency of analoa
f=frequency of digital signal
(1.4.6)
=relative or normalized frequency

T h e re la tio n in (1.4.5) ju stifies th e n am e relative o r n o r m a li z e d f r e q u e n c y , w hich is

so m e tim e s u sed to d esc rib e th e fre q u e n c y v a ria b le / . A s (1.4.5) im p lies, w e can use
/ to d e te rm in e th e fre q u e n c y F in h e rtz only if th e sa m p lin g fre q u e n c y Fs is k now n.
W e recall fro m S ectio n 1.3.1 th a t th e ran g e o f th e fre q u e n c y v a ria b le F o r 2
fo r co n tin u o u s-tim e sin u so id s are
oc < F < oo
oc < 2 < 00

( I .4 .7 )

H o w e v e r, th e situ a tio n is d iffe re n t fo r d isc re te -tim e sinusoids. F ro m S ectio n 1.3.2

w e recall th a t
2

c / < \
2

(1.4.8)

n < co < n

B y su b stitu tin g fro m (1.4.5) a n d (1.4.6) in to (1.4.8), w e find th a t th e freq u e n cy

o f th e co n tin u o u s-tim e sin u so id w h e n sa m p le d a t a ra te Fs = 1 / 7 m u st fall in

Sec. 1.4

25

Analog-to-Digitai and Digital-to-Analog Conversion

th e ra n a e
(1.4.9)
or, e q u iv alen tly .

. -Q < _Ti F,

n F, <

-----------=

(1.4.10)

T h e se re la tio n s a re su m m a riz e d in T ab le 1.1.

RELATIO NS AM ONG FR E Q U E N C Y VARIABLES
Discrete-time sienals

Continuous-time sienals

co = 2,t f
radians
cycles
sample
sample

F
u ..
\

u> = nT ,f=F /F,

/

'

Q =
radians
sec

1A
5
IA

TA B LE 1.1

- /

= to/T.F~f- Fs

.................. - ..... ..

tt/T < Q < ,T /r

-f-2,/2 5 F < /-;/-

- o c < fi < oc
oc < f-' < oc

F ro m th ese re la tio n s w e o b se rv e th at the fu n d a m e n ta l d ifferen ce b etw een

c o n tin u o u s-tim e an d d isc rete-tim e signals is in th e ir ran g e of v alu es o f th e fre
q u en cy v a riab les F an d / , o r Q and w. P erio d ic sa m p lin g o f a c o n tin u o u s-tim e
signal im p lies a m a p p in g of th e infinite freq u e n cy ran g e fo r th e v ariab le F (o r 2)
in to a finite fre q u e n c y ran g e for the v ariab le / (o r a>). Since th e highest freq u e n cy
in a d isc re te -tim e signal is co tt o r / = *, it follow s th a t, w ith a sa m p lin g rate
Fs, th e c o rre sp o n d in g h ig h est values o f F and 2 are

_J_

~ 2T

(1.4.11)

^max ft Fs
T h e re fo re , sa m p lin g in tro d u c e s an am b ig u ity , since th e h ig h est fre q u e n c y in a
co n tin u o u s-tim e signal th a t can be u n iq u ely d istin g u ish ed w h en such a signal is
sa m p le d at a ra te Fs = l / T is Fm&li F J 2 , o r Qmax = n F s. T o see w h at h a p p e n s
to fre q u e n c ie s a b o v e F J 2, let us co n sid er the follow ing ex am p le.
Example 1.4.1
The implications of these frequency relations can be fully appreciated by considering
the two analog sinusoidal signals
xi (!) cos 2ji(l0)t
xi(t) cos2;r(50)f

(1.4.12)

26

Introduction

Chap. 1

which are sampled at a rate Fs = 40 Hz. The corresponding discrete-time signals or

sequences are

(1.4.13)

However. cos5,t/2 = cos(2^n + 7rn/2) co s7rn /2 . Hence

= *i(n)- Thus the
sinusoidal signals are identical and, consequently, indistinguishable. If we are given
the sampled values generated by cos(7r/'2)n, there is some ambiguity as to whether
these sampled values correspond to x\{i) or xz(D- Since x2(r) yields exactly the same
values as
when the two are sampled at F, 40 samples per second, we say that
the frequency F2 50 Hz is an alias of the frequency F\ = 10 Hz at the sampling
rate of 40 samples per second.
It is im portant to note that F2 is not the only alias of F]. In fact at the sampling
rate of 40 samples per second, the frequency F3 = 90 Hz is also an alias of F], as is
the frequency F4 = 130 Hz, and so on. All of the sinusoids cos2tt(Fj -f 40k)i. k = 1.
2. 3. 4 . . . . sampled at 40 samples per second, yield identical values. Consequently,
they are all aliases of F\ = 10 Hz.
In g en eral, th e sam pling o f a c o n tin u o u s-tim e sin u so id al signal
x a(t) = A cos(27rF{)/ + 8)

(1 .4 .1 4 )

w ith a sam pling rate F v = 1 / T resu lts in a d isc re te -tim e signal

x ( n ) = A cos(27r/ 0 -f 6)

(1.4.15)

w h ere /o = F()/F , is th e re la tiv e fre q u e n c y o f th e sin u so id . If w e assum e th a t

- F s /2 < Fo < F J 2 . th e fre q u e n c y f 0 o f x ( n ) is in th e ran g e ^ < /o < L w hich is
th e freq u e n cy ran g e fo r d isc re te -tim e signals. In this case, the re la tio n sh ip b etw een
Fo an d f {) is o n e -to -o n e , a n d h en ce it is p o ssib le to identify (o r re c o n stru c t) th e
a n alo g signal xa (t) from th e sa m p les x ( n) .
O n th e o th e r h a n d , if th e sinusoids
x a {t) = A

c o s (27 z F kt

+ 6)

(1.4.16)

w h ere
Fk = F0 + k F s .

k = l. 2 .

(1.4.17)

are sam p led a t a ra te F,, it is c le a r th a t th e fre q u e n c y F* is o u tsid e th e fu n d a m e n ta l

fre q u e n c y ran g e Fs / 2 < F < F J 2 . C o n se q u e n tly , th e sa m p le d signal is

= A

c o s (27zn F o / F s

+ 6 + 2nkn)

A c o s ( 2 n f o n + 6)

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

27

w hich is id en tical to th e d isc re te -tim e signal in (1.4.15) o b ta in e d by sam p lin g .

(1.4.14). T h u s an infinite n u m b e r of c o n tin u o u s-tim e sin u so id s is re p re se n te d by
sa m p lin g th e sa m e d isc re te -tim e signal (i.e.. by th e sam e se t o f sam p le s). C o n
se q u e n tly , if w e a re given th e se q u en ce
an am b ig u ity exists as to w hich
c o n tin u o u s-tim e signal x a(t) th e se v alu es re p re se n t. E q u iv a le n tly , we can say th a t
th e fre q u e n c ie s Fk F o + k F s, oo < k < oo (k in te g e r) a re in d istin g u ish a b le fro m
th e fre q u e n c y Fo a fte r sa m p lin g an d h e n c e th ey are aliases o f Fo. T he re la tio n sh ip
b e tw e e n th e fre q u e n c y v a riab les o f th e c o n tin u o u s-tim e a n d d isc rete-tim e signals
is illu stra te d in Fig. 1.17.
A n ex a m p le o f aliasing is illu stra te d in Fig. 1.18. w h e re tw o sinusoids w ith
fre q u e n c ie s F 0 = | H z an d Fj = | H z yield id en tical sa m p le s w hen a sam p lin g
ra te o f Fs 1 H z is used. F ro m (1.4.17) it easily follow s th a t fo r k = - 1 , Fo =
F, + Fs = ( | + 1) H z = i H z.

Figure 1.17 Relationship between the continuous-time and discrete-time fre

quency variables in the case of periodic sampling.

Figure 1.18

Illustration of aliasing.

28

Introduction

Chap. 1

Since Fsf2. w hich c o rre sp o n d s to w = tt, is th e hig h est fre q u e n c y th a t can be

r e p re s e n te d u n iq u ely w ith a sam p lin g ra te Fs , it is a sim ple m a tte r to d e te rm in e
th e m ap p in g o f any (alias) fre q u e n c y above Fs/2 (co tt) in to th e e q u iv a le n t
freq u e n cy b elo w Fs /2. W e can use F J 2 or a) t t as the p iv o ta l p o in t a n d reflect
or fo ld th e alias freq u e n c y to the ran g e 0 < w < tt. Since th e p o in t o f reflectio n
is Fsj 2 (co = t t ) , th e fre q u e n c y F J 2 (cu = re) is called th e f o l d i n g frequency.
Example 1.4.2
Consider the analog signal
Xait) = 3 cos IOOtt/
(a) Determ ine the minimum sampling rate required to avoid aliasing.

(b) Suppose that the signal is sampled at the rate Fs = 200 Hz.

What is the
discrete-time signal obtained after sampling?
(c) Suppose that the signal is sampled at the rate Fs ~ 75 Hz. W hat is the discretetime signal obtained after sampling?

(d) What is the frequency 0 < F < FJ2 of a sinusoid that yields samples identical
to those obtained in part (c)?
Solution
(a) The frequency of the analog signal is F 50 Hz. Hence the minimum sampling
rate required to avoid aliasing is Ff = 100 Hz.
(b) If the signal is sampled at Fs = 200 Hz. the discrete-time signal is
lOOtf
TT
x{n) = j cos n j cos n
200
2
(c) If the signal is sampled at F, = 75 Hz. the discrete-time signal is
100?r
A tt
x(n) 3 cos j ^ n 3 cos n

3 cos ~n
3
(d) For the sampling rate of Fs ~ 75 Hz. we have
F = f F, = 7 5 /
The frequency of the sinusoid in part (c) is / |. Hence
F = 25 Hz
Clearly, the sinusoidal signal
ya(i) 3 cos I n Ft
3 cos 507ti
sampled at Fs 75 samples/s yields identical samples. H ence F 50 Hz is an
alias of F = 25 Hz for the sampling rate Fs = 75 Hz.

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

29

1.4.2 The Sampling Theorem

G iv en an y an a lo g signal, how sh o u ld w e select th e sa m p lin g p e rio d T or, eq u iv
alen tly , th e sa m p lin g ra te FJ. T o an sw er this q u e stio n , w e m u st h ave som e in
fo rm a tio n a b o u t th e c h aracteristics of th e signal to be sa m p le d . In p a rtic u la r, we
m u st h av e so m e g e n e ra l in fo rm a tio n c o n cern in g th e fr e q u e n c y con tent o f th e sig
nal. S uch in fo rm a tio n is g en erally av ailab le to us. F o r e x am p le, w e k n o w g en erally
th a t th e m a jo r fre q u e n c y co m p o n e n ts o f a sp eech signal fall b elo w 3000 H z. O n
th e o th e r h a n d , telev isio n signals, in g e n eral, c o n ta in im p o rta n t fre q u e n c y co m
p o n e n ts u p to 5 M H z. T h e in fo rm a tio n c o n te n t of such signals is c o n ta in e d in
th e a m p litu d e s, fre q u e n c ie s, an d p h ases o f the v ario u s fre q u e n c y co m p o n e n ts, b ut
d e ta ile d k n o w le d g e o f th e c h aracteristics of such signals is n o t a v a ilab le to us p rio r
to o b ta in in g th e signals. In fact, the p u rp o se of p ro cessin g the signals is usually to
e x tra c t th is d e ta ile d in fo rm a tio n . H o w ev er, if w e k n o w th e m ax im u m freq u e n cy
c o n te n t o f th e g e n e ra l class o f signals (e.g.. th e class of sp e ech signals, the class
o f v id eo signals, etc.). w e can specify th e sam pling ra te n ecessa ry to co n v ert the
a n alo g signals to dig ital signals.
L et us su p p o se th a t any an alo g signal can be r e p re s e n te d as a sum o f sin u so id s
o f d iffe re n t a m p litu d e s, freq u e n cies, a n d p h ases, th a t is.
N

(1.4.18)
w h ere N d e n o te s th e n u m b e r o f freq u e n cy c o m p o n e n ts. A ll signals, such as speech
an d v id eo , len d th em se lv e s to such a re p re s e n ta tio n o v er an y sh o rt tim e segm ent.
T h e a m p litu d e s, freq u e n cies, a n d p h ases usually ch an g e slow ly w ith tim e from one
tim e se g m en t to a n o th e r. H o w e v e r, su p p o se th a t th e fre q u e n c ie s do n o t exceed
som e k n o w n fre q u e n c y , say Fmax. F o r ex am p le, F max = 3000 H z fo r th e class
o f sp e ech signals a n d Fmax = 5 M H z fo r telev isio n signals. Since th e m ax im u m
freq u e n cy m ay v ary slightly fro m d iffe re n t re a liz a tio n s am o n g signals of any given
class (e.g., it m ay vary slightly from s p e a k e r to sp e a k e r), w e m ay wish to e n su re
th a t Fmax d o e s n o t ex ceed som e p re d e te rm in e d v alue by passin g th e an a lo g signal
th ro u g h a filter th a t se v ere ly a tte n u a te s freq u e n cy c o m p o n e n ts ab o v e Fmax. T hus
we a re c e rta in th a t no signal in the class co n tain s fre q u e n c y c o m p o n e n ts (having
significant a m p litu d e o r p o w e r) above Fmax. In p ra c tic e , such filtering is com m only
u sed p rio r to sam p lin g .
F ro m o u r k n o w led g e o f Fmax, w e can se lect th e a p p ro p ria te sam pling rate.
W e k n o w th a t th e h ig h est freq u e n cy in an an alo g signal th a t can be u n a m b ig u
ously re c o n s tru c te d w h en th e signal is sa m p le d a t a ra te F, = 1 / T is F J 7. A ny
fre q u e n c y a b o v e Fsf 2 o r b elo w - F J 2 resu lts in sa m p le s th a t a re id en tical w ith a
c o rre sp o n d in g fre q u e n c y in th e ra n g e F J 2 < F < Fs/2. T o avoid th e am b ig u ities
re su ltin g fro m aliasin g , we m u st se lect th e sa m p lin g ra te to be sufficiently high.
T h a t is, w e m u st select F J 2 to be g re a te r th an Fmax. T h u s to avoid th e p ro b le m
o f aliasin g , Fs is se le c te d so th a t
Fs > 2 Fmax

(1.4.19)

30

Introduction

Chap. 1

w h ere Fmax is th e larg est fre q u e n c y c o m p o n e n t in the a n a lo g signal. W ith the

sa m p lin g ra te se le c te d in this m a n n e r, any fre q u e n c y c o m p o n e n t, say |F ;| < Fmax,
in th e an alo g signal is m a p p e d in to a d isc re te -tim e sinusoid w ith a freq u e n cy

Fs ~ 2

(1.4.20)

o r, eq u iv alen tly ,
tt < a)j = 2n f < 7r

(1.4.21)

S ince, | / | = \ o r \co\ = n is th e h ig h est (u n iq u e ) freq u e n c y in a d isc re te -tim e signal,

th e choice o f sam p lin g ra te acco rd in g to (1.4.19) avoids th e p ro b le m of aliasing.
In o th e r w o rd s, th e co n d itio n Fs > 2 Fmax e n s u re s th a t all th e sin u so id al c o m p o
n e n ts in th e a n a lo g signal are m a p p e d in to c o rre sp o n d in g d isc re te -tim e freq u e n cy
c o m p o n e n ts w ith fre q u e n c ie s in th e fu n d a m e n ta l in terv al. T h u s all the freq u e n cy
c o m p o n e n ts o f th e a n alo g signal a re re p re s e n te d in sa m p le d fo rm w ith o u t am b i
guity, a n d h en ce th e an alo g signal can be re c o n stru c te d w ith o u t d isto rtio n from
th e sa m p le v alu es u sing an a p p r o p ria te in te rp o la tio n (d ig ita l-to -a n a lo g c o n v e r
sio n ) m e th o d . T h e ap p ro p riate o r ideal in te rp o la tio n fo rm u la is specified by the
s a m p ling theorem.
Sam pling T h eorem .
If the h ig h est fre q u e n c y c o n ta in e d in an an alo g signal
x a (t) is Fmax = B a n d th e signal is sa m p le d at a ra te F, > 2 F max = 2 B. th e n A.u(r)
can b e exactly re c o v e re d fro m its sa m p le values using th e in te rp o la tio n fu n ctio n
s i n 2 jr B /
8(t) =

2n B t

_ ,
(

T h u s jcfl(f) m ay be e x p ressed as
* . ( ) * ( ' - )

(1-4.23)

w h ere x a(n / F s ) = x a( n T ) = Jt(rc) a re th e sa m p les o f x a(t).

W h en th e sa m p lin g of x a(t) is p e rfo rm e d at the m in im u m sam p lin g ra te
Fs = 2 B , th e re c o n stru c tio n fo rm u la in (1.4.23) b eco m es
^

/ n \ sin2nB (t n flB )
(

zb)

, ,

(1'4 2 4 )

T h e sa m p lin g r a te F ^ = 2 B = 2 Fmax is called th e N y q u is t rate. F ig u re 1.19 illus

tra te s th e ideal D /A c o n v ersio n p ro cess using th e in te rp o la tio n fu n c tio n in (1.4.22).
A s can b e o b se rv e d from e ith e r (1.4.23) o r (1.4.24), th e re c o n stru c tio n o f x a(t)
fro m th e se q u e n c e x ( n ) is a c o m p lic a te d p ro cess, involving a w e ig h te d sum o f the
in te rp o la tio n fu n ctio n g (t) an d its tim e -sh ifte d v ersio n s g ( t n T ) fo r oo < n < oo,
w h ere th e w eig h tin g facto rs a re th e sa m p le s x ( n ) . B e c a u se o f th e co m p lex ity an d
th e infinite n u m b e r o f sa m p les re q u ire d in (1.4.23) o r (1.4.24), th e s e re c o n stru c tio n

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

31

sample of ,v(n

(/[ ^ /

{ri l )l

fll

\t -r l 11

Figure 1.19 Ideal D /A conversion

(interpolation).

fo rm u ias are p rim a rily o f th e o re tic a l in te re st. P ra ctical in te rp o la tio n m e th o d s are

given in C h a p te r 9.
Example 1.4.3
Consider the analog signal
xu(r) = 3cos50;rz

10sin300;n cos 100tt?

What is the Nvquist rate for this signal?

Solution

The frequencies present in the signal above are

F = 25 Hz.

F: = 150 Hz.

F, = 50 Hz

Thus Fnm = 150 Hz and according to (1.4.19),

F > 2 Fmax = 300 Hz
The Nvquist rale is FA = 2 Fm;. Hence
Fs = 300 Hz
Discussion It should be observed that the signal component 10sin300;r/. sampled at
the Nvquist raie FA- = 300, results in the samples 10 sin 7r/j. which are identically zero.
In other words, we are sampling the analog sinusoid at its zero-crossing points, and
hence we miss this signal component completely. This situation would not occur if the
sinusoid is offset in phase by some am ount 8. In such a case we have lOsinGOOin -ffl)
sampled at the Nvquist rate FA- = 300 samples per second, which yields the samples
-c o
10 sin(7rn+ ^) = 10(sin n n cos & -+

stth

sin f>)

= lOsin 6 cos nn

Thus if 6 ^ 0 or tt, the samples of the sinusoid taken at the

zero. However, we still cannot obtain the correct amplitude
the phase 9 is unknown. A simple rem edy that avoids this
situation is to sample the analog signal at a rate higher than

Nvquist rate are not all

from the samples when
potentially troublesome
the Nvquist rate.

Example 1.4.4
Consider the analog signal
*a(t) = 3cos2000irf + 5 sin6000;rr + lOcos 12.000;?;

Introduction

Chap. 1

(a) What is the Nvquist rate for this signal?

(b) Assume now that we sample this signal using a sampling rate Fs = 5000
samples/s. What is the discrete-time signal obtained after sampling?
(c) What is the analog signal y(r) we can reconstruct from the samples if we use
ideal interpolation?

= 2.5 kHz
2

and this is the maximum frequency that can be represented uniquely by the
sampled signal. By making use of (1.4.2) we obtain

= 3 cos 2jt( j )h + 5 sin 2n- (^ ) + 10 cos 2n(^)n

= 3 c o s2 ;r(|)n + 5 sin 2 7 r(l ) 4- 10cos2,t(1 4- ^)u
= 3cos2jr({)n 4- 5sin27r(1) 4- 1 0 c o s 2 ^ (|)
Finally, we obtain
x(n) = 13 cos2^({)/i - 5sin27r( = )fl
The same result can be obtained using Fig. 1.17. Indeed, since F, = 5 kHz.
the folding frequency is FJ2 = 2.5 kHz. This is the maximum frequency that
can be represented uniquely by the sampled signal. From (1.4.17) we have
Fti = Fk kFs. Thus Fo can be obtained by subtracting from Fk an integer
m ultiple of Fs such that F t/2 < F0 < F J 2. The frequency F: is less than Fsf2
and thus it is not affected by aliasing. However, the other two frequencies are
above the folding frequency and they will be changed by the aliasing effect.
Indeed.
F'2 = Fj ~ Fs = - 2 kHz
f;

= Fj Fs = 1 kHz

From (1.4.5) it follows that /] =

with the result above.

and h = ^ which are in agreem ent

Sec. 1.4

A nalog-toD igital and Digital-to-Analog Conversion

33

(c) Since only the frequency components at 1 kHz and 2 kHz are present in the
sampled signal, the analog signal we can recover is
x(t)

= 13 cos 2000^r - 5sin400()-Tf

which is obviously different from the original signal xU). This distortion of the
original analog signal was caused by the aliasing effect, due to the low sampling
rate used.
A lth o u g h aliasin g is a pitfall to be av o id ed , th e re are tw o useful p ractical
a p p licatio n s b ased on th e ex p lo itatio n of the aliasing effect. T h e se a p p licatio n s
are th e stro b o sc o p e an d the sam pling oscilloscope. B o th in stru m e n ts are d esigned
to o p e ra te as aliasin g devices in o rd e r to re p re se n t high fre q u e n c ie s as low f re
q u en cies.
T o e la b o ra te , co n sid er a signal w ith h ig h -freq u en cy c o m p o n e n ts confined to
a given fre q u e n c y b an d B\ < F < B2. w h ere Bz B\ = B is d efined as the
b a n d w id th o f th e signal. W e assum e th a t B < < B\ < B 2. T h is co n d itio n m ean s
th a t th e fre q u e n c y c o m p o n e n ts in the signal are m uch larg er th an th e b an d w id th
B of th e signal. Such signals are usually called p a ssb a n d or n a rro w b a n d signals.
N ow . if this signal is sa m p le d at a rate Fs > 2B. b u t F^ << B\. th e n all th e f re
q u en cy c o m p o n e n ts c o n ta in e d in the signal will be aliases of fre q u e n c ie s in the
ran g e 0 < F < F J 2 . C o n seq u en tly , if we o b se rv e the freq u e n cy c o n te n t of the
signal in th e fu n d a m e n ta l range 0 < F < F J 2 . we k now precisely the freq u e n cy
co n te n t o f th e an a lo g signal since we k now the fre q u e n c y b an d B\ < F < B2 u n d e r
c o n sid e ra tio n . C o n se q u e n tly , if the signal is a n a rro w b a n d (p a ss b a n d ) signal, we
can re c o n stru c t th e o riginal signal from the sam p le s, p ro v id e d th a t the signal is
sa m p le d at a ra te Fs > 2 B. w h ere B is th e b an d w id th . T h is s ta te m e n t c o n stitu te s
a n o th e r fo rm o f th e sam pling th e o re m , w hich we call the p a s s b a n d f o r m in o rd e r
to d istin g u ish it fro m th e p rev io u s form o f the sa m p lin g th e o re m , w hich ap p lies in
g en eral to all ty p es of signals. T he la tte r is so m e tim es called th e bas e ban d f or m.
T h e p a s s b a n d f o r m o f th e sam pling th e o re m is d escrib ed in d e ta il in S ectio n 9.1.2.

1.4.3 Quantization of Continuous-Amplitude Signals

A s w e h av e se en , a dig ital signal is a se q u en ce of n u m b e rs (sa m p le s) in w hich each
n u m b e r is re p re s e n te d by a finite n u m b e r of digits (finite p recisio n ).
T h e p ro c e ss o f co n v ertin g a d isc rete-tim e c o n tin u o u s-a m p litu d e signal in to a
dig ital signal by ex p ressin g each sa m p le value as a finite (in ste a d of an infinite)
n u m b e r o f d igits, is called quant izati on. T he e rro r in tro d u c e d in re p re se n tin g th e
c o n tin u o u s-v a lu e d signal by a finite set o f d isc rete v alu e levels is called quant i zat i on
error o r quant i zat i on noise.
W e d e n o te th e q u a n tiz e r o p e ra tio n o n th e sa m p le s x ( n ) as Q[x{n)] an d let
x q{n) d e n o te th e se q u en ce o f q u a n tiz e d sam ples a t th e o u tp u t o f th e q u a n tiz e r.
H e n ce
Xq(n)

= Q[x(n)]

34

Introduction

Chap. 1

T h e n th e q u a n tiz a tio n e r ro r is a se q u e n c e eq (n) d efin ed as th e d iffe re n c e b etw e e n

th e q u a n tiz e d v alu e a n d th e actu al sa m p le value. T h u s
eq (n) = x q {n) - x ( n )
W e illu strate th e q u a n tiz a tio n p ro cess w ith a n ex am p le.
d isc rete-tim e signal

(1.4.25)
L e t us co n sid e r the

o b ta in e d by sa m p lin g th e an a lo g e x p o n e n tia l signal x a( t) = 0 .9 ', t > 0 w ith a

sam p lin g freq u e n cy f , = 1 H z (see Fig. 1.20(a)). O b se rv a tio n o f T a b le 1.2, w hich
show s th e v alu es o f th e first 10 sa m p les o f x ( n ) , rev eals th a t th e d e sc rip tio n o f the
sam p le v alu e x{n) re q u ire s n significant digits. It is o b v io u s th a t th is signal can n o t

(a)

Figure 1.20

Illustration of quantization.

Sec. 1.4

35

Analog-to-Digital and Digital-to-Analog Conversion

TA B LE 1.2

NU M ER IC A L ILLU S TR A TIO N O F Q U A N TIZA TIO N W ITH ONE

S IG N IF IC A N T D IG IT USING T R U N C A T IO N O R R O UN DING

x,(n )
(Truncation)

(Rounding)

1
0.9

1.0
0.9

1.0
0.9

3
4
5
6
7

0.81
0.729
0.6561
0.59049
0.531441
0.4782969

0.8
0.7

0.8
0.7
0.7

8
9

0.43046721
0.387420489

0.4

x ( n)
n

0
1
2

D iscrete-tim e signal

.voi)

0.6
0.5
0.5
0.4
0.3

(Rounding)
0.0

0.0
0.01
0.029

0.0439
0.00951
0.031441

0.6
0.5
0.5
0.4
0.4

0.0217031
0.03046721
0.012579511

be p ro cessed by u sing a c a lcu lato r o r a digital c o m p u te r since only the first few
sam p les can be sto re d an d m a n ip u la te d . F o r ex am p le, m ost c alcu lato rs process
n u m b e rs w ith only eig h t significant digits.
H o w e v e r, let us assum e th a t w e w an t to use only o n e significant digit. To
elim in ate th e excess digits, w e can e ith e r sim ply d iscard th em (tru n ca tion ) o r dis
card th e m bv ro u n d in g th e resu ltin g n u m b e r (ro un din g). T h e resu ltin g q u an tized
signals x q (n) a re show n in T ab le 1.2. W e discuss only q u a n tiz a tio n by ro u n d in g ,
alth o u g h it is ju st as easy to tr e a t tru n c a tio n . T h e ro u n d in g p ro c e ss is g raphically
illu stra te d in Fig. 1.20b. T h e values allo w ed in th e digital signal are called the
quan tizatio n levels, w h ereas the d istan c e A b etw een tw o successive q u a n tiz a tio n
levels is called th e q uantization step siz e o r resolution. T h e ro u n d in g q u a n tiz e r
assigns each sa m p le of x ( n ) to th e n e a re s t q u a n tiz a tio n level. In c o n tra st, a q u a n
tizer th a t p e rfo rm s tru n c a tio n w ould have assigned each sa m p le of jc(/z) to the
q u a n tiz a tio n level b elo w it. T h e q u a n tiz a tio n e r ro r eq (n) in ro u n d in g is lim ited to
th e ra n g e of A /2 to A /2 , th a t is,
A

- y <<?,()<f

(1A26)

In o th e r w o rd s, th e in sta n ta n e o u s q u a n tiz a tio n e r ro r c a n n o t exceed half of the

q u a n tiz a tio n ste p (see T a b le 1.2).
If jcmjn an d j:max r e p re s e n t th e m in im u m an d m ax im u m v alu e of x (n ) a n d L
is th e n u m b e r o f q u a n tiz a tio n levels, th en
A = Xmax ~ ^
L - 1

(1.4.27)

W e d efin e th e d y n a m i c range of th e signal as jrmax -*min- 1 o u r ex am p le we

h av e Jtmax = 1, Jtmjn = 0, a n d L 11, w hich leads to A = 0.1. N o te th a t if the
d y n am ic ran g e is fixed, in creasin g th e n u m b e r o f q u a n tiz a tio n levels, L resu lts in a
d e c re a se o f th e q u a n tiz a tio n ste p size. T h u s th e q u a n tiz a tio n e r ro r d e c re a s e s and
th e accu racy o f th e q u a n tiz e r in crease s. In p ra c tic e w e can re d u c e th e q u a n tiz a tio n

36

Introduction

Chap. 1

error to an insignificant am ount by choosing a sufficient num ber o f quantization

levels.
T h eoretically, quantization o f analog signals always resu lts in a loss o f in
form ation. T his is a result o f the am biguity introduced by quantization. Indeed,
quantization is an irreversible or noninvertible process (i.e., a m any-to-on e m ap
ping) since all sam ples in a distance A /2 about a certain quantization level are
assigned the sam e value. This am biguity m akes the exact quantitative analysis of
quantization extrem ely difficult. This subject is discussed further in C hapter 9,
where w e use statistical analysis.

1.4.4 Quantization of Sinusoidal Signals

Figure 1.21 illustrates the sam pling and quantization o f an an alog sinusoidal signal
x a (/) = A cos
using a rectangular grid. H orizontal lines w ithin the range of the
quantizer indicate the allow ed levels o f quantization. V ertical lines indicate the
sam pling tim es. Thus, from the original analog signal x a{t) w e obtain a discrete
tim e signal x ( n ) = x a {nT) by sam pling and a discrete-tim e, discrete-am plitude
signal x q ( nT) after quantization. In practice, the staircase sign al xv (r) can be
obtained by using a zero-order hold. This analysis is useful b ecau se sinusoids are
used as test signals in A /D converters.
If the sam pling rate Fs satisfies the sam pling theorem , quantization is the only
error in the A /D conversion process. Thus w e can evalu ate the quantization error

Time
Figure L21

Sampling and quantization of a sinusoidal signal.

Sec. 1.4

Analog-to-Digital and Digital-to-Analog Conversion

37

bv q u a n tiz in g th e an alo g signal x(t) in stead of th e d isc re te -tim e signal a ( ) =

x a( nT) . In sp e c tio n o f Fig. 1.21 in d icates th at the signal x u {t) is alm o st lin ear
b e tw e e n q u a n tiz a tio n levels (see Fig. 1.22). T he c o rre sp o n d in g q u a n tiz a tio n e rro r
eq (t) x u(t) x q(t) is show n in Fig. 1.22. In Fig. 1.22. r d e n o te s th e tim e th at
x a {t) stay s w ithin th e q u a n tiz a tio n levels. T he m e a n -sq u a re e r ro r p o w e r Pq is
Pq = 2 t /
Since eq (i) = ( A / 2 r ) t . -

< t <

t.

e{!)cJl = 7 j

(1.4.28)

we have

If th e q u a n tiz e r has b bits of accuracy an d the q u a n tiz e r co v ers the e n tire range
2A . th e q u a n tiz a tio n step is A = 2 A / 2 h. H en ce
A 2/ 3

P., =

(1.4.30)

T h e a v erag e p o w e r o f the signal xu(D is

1 f 1'
,
A2
P, = / (A cos Qi,i r d i = ~
TP Jo
2

(1.4.31)

T h e q u ality o f th e o u tp u t o f the A /D c o n v e rte r is usually m e a s u re d by th e signalto- quanti zati on noise ratio ( S Q N R ) . w hich pro v id es th e ratio o f th e signal p o w er
to th e no ise po w er:
S Q N R = = - 22b
P
">
' i/
E x p re s se d in d ecib els (d B ), th e S Q N R is
S Q N R (d B ) = 101og1(l S Q N R = 1.76 + 6.026

(1.4.32)

T his im p lies th a t th e S Q N R in crease s a p p ro x im ately 6 dB fo r ev ery bit a d d e d to

th e w o rd le n g th , th a t is. fo r each d o u b lin g of th e q u a n tiz a tio n levels.
A lth o u g h fo rm u la (1.4.32) was d eriv ed fo r sin u so id al signals, w e shall see in
C h a p te r 9 th a t a sim ilar resu lt holds fo r every signal w hose d y n am ic ran g e sp a n s the
ran g e o f th e q u a n tiz e r. T h is re la tio n sh ip is ex trem ely im p o rta n t b e c a u se it d ictates

Figure 1.22

T he quantization error eq (t) x a (t) - x q (t).

38

Introduction

Chap. 1

th e n u m b e r of bits re q u ire d by a specific a p p lic a tio n to assu re a given signal-ton o ise ratio . F o r ex am p le, m o st co m p a c t disc p lay ers use a sa m p lin g freq u e n cy
o f 44.1 k H z an d 16-bit sa m p le re so lu tio n , w hich im plies a S Q N R of m o re th an
96 dB .

1.4.5 Coding of Quantized Samples

T h e co d in g p ro cess in an A /D c o n v e rte r assigns a u n iq u e b in a ry n u m b e r to each
q u a n tiz a tio n level. If we h av e L levels w e n e e d at least L d iffe re n t binary' n u m b ers.
W ith a w ord len g th o f b bits w e can c re a te 2b d iffe re n t b in ary n u m b e rs. H e n c e we
h av e 2h > L. o r e q u iv alen tly , b > log 2 L. T h u s th e n u m b e r of bits re q u ire d in the
co d e r is th e sm allest in te g e r g re a te r th a n o r eq u al to log 2 L. In o u r ex am p le it can
easily be seen th a t we n eed a c o d e r w ith b = 4 bits. C o m m ercially av ailab le A /D
c o n v e rte rs m ay be o b ta in e d w ith finite p recisio n of b 16 o r less. G e n e ra lly , the
h ig h er th e sam p lin g sp e ed a n d th e fin er th e q u a n tiz a tio n , th e m o re ex p en siv e the
d evice becom es.

1.4.6 Digital-to-Analog Conversion

Am plifude

T o co n v ert a d igital signal in to an an a lo g signal we can use a d ig ital-to -an alo g

(D /A ) co n v erter. A s sta te d p rev io u sly , th e task o f a D /A c o n v e rte r is to in te rp o la te
b e tw e e n sam ples.
T h e sam p lin g th e o re m specifies th e o p tim u m in te rp o la tio n fo r a bandlim ited signal. H o w ev er, this ty p e o f in te rp o la tio n is to o c o m p lic a te d an d . h en ce
im p ractical, as in d icated p rev io u sly . F ro m a p ractical v iew p o in t, the sim p lest D /A
co n v e rte r is th e z e ro -o rd e r h o ld show n in Fig. 1.15. w hich sim p ly holds c o n sta n t
th e v alu e o f o n e sam p le u n til th e n ex t o n e is receiv ed . A d d itio n a l im p ro v e m e n t
can b e o b ta in e d by u sing lin e a r in te rp o la tio n as show n in Fig. 1.23 to c o n n e c t
successive sa m p les w ith stra ig h t-lin e se g m en ts. T h e z e ro -o rd e r hold an d lin ear
in te rp o la to r are an aly zed in S ectio n 9.3. B e tte r in te rp o la tio n can be ach iev ed by
u sing m o re so p h isticated h ig h e r-o rd e r in te rp o la tio n tech n iq u es.
In g en eral, su b o p tim u m in te rp o la tio n te c h n iq u e s re su lt in p assin g fre q u e n c ie s
ab o v e th e fo ld in g freq u e n cy . S uch fre q u e n c y c o m p o n e n ts are u n d e s ira b le a n d are
u su ally rem o v ed by p assin g th e o u tp u t o f th e in te rp o la to r th ro u g h a p ro p e r an alo g

Sec. 1.5

Summ ary and References

39

filter, w hich is called a postfilter or sm o o th in g filter. T h u s D /A c o n v ersio n usually

involves a su b o p tim u m in te rp o la to r follow ed by a p ostfilter. D /A c o n v e rte rs are
tre a te d in m o re d e ta il in S ectio n 9.3.

1.4.7 Analysis of Digital Signals and Systems Versus

Discrete-Time Signals and Systems
W e h av e seen th a t a d igital signal is defin ed as a fu n ctio n o f an in te g e r in d e p e n d e n t
v ariab le an d its v alu es are ta k e n from a finite set of po ssib le values. T he usefulness
of such signals is a c o n s eq u en ce of th e possibilities o ffe re d by digital co m p u ters.
C o m p u te rs o p e ra te on n u m b ers, w hich are re p re s e n te d by a strin g of 0 's an d l's .
T h e len g th of th is strin g (w o r d length) is fixed an d finite an d usually is 8. 12. 16. or
32 bits. T h e effe cts o f finite w ord len g th in c o m p u ta tio n s cause co m p licatio n s in
th e an aly sis of d ig ital signal p ro cessin g system s. T o avoid th ese co m p licatio n s, we
neg lect th e q u a n tiz e d n a tu re of digital signals an d system s in m uch o f o u r analysis
an d c o n sid e r th em as d isc rete-tim e signals an d system s.
In C h a p te rs 6 . 7. and 9 we in v estig ate th e c o n se q u e n c e s o f using a finite w ord
len g th . T h is is an im p o rta n t topic, since m any digital signal p ro cessin g p ro b lem s are
solved w ith sm all c o m p u te rs o r m icro p ro cesso rs th at em p lo y fix ed -p o in t arith m etic.
C o n se q u e n tly , o n e m ust look carefully at the p ro b le m of fin ite-p recisio n arith m e tic
an d a c c o u n t for it in th e design of so ftw are an d h a rd w a re th at p e rfo rm s the d esired
signal p ro cessin g tasks.

1.5 SUMMARY AND REFERENCES

In th is in tro d u c to ry c h a p te r w e have a tte m p te d to p ro v id e the m o tiv a tio n for digital
signal p ro cessin g as an a lte rn a tiv e to a n a lo g signal pro cessin g . W e p re se n te d the
basic e le m e n ts o f a digital signal p ro cessin g system an d d efin ed th e o p e ra tio n s
n e e d e d to c o n v e rt an an alo g signal in to a digital signal re a d y fo r processing. O f
p a rtic u la r im p o rta n c e is the sam pling th e o re m , w hich w as in tro d u c e d by N vquist
(1928) an d la te r p o p u la riz e d in the classic p a p e r by S h a n n o n (1949). T h e sam pling
th e o re m as d e sc rib e d in S ection 1.4.2 is d eriv ed in C h a p te r 4. S in u so id al signals
w ere in tro d u c e d p rim a rily fo r the p u rp o se of illu stra tin g th e aliasin g p h e n o m e n o n
an d fo r th e s u b s e q u e n t d e v e lo p m e n t o f th e sa m p lin g th e o re m .
Q u a n tiz a tio n effects th a t are in h e re n t in the A /D co n v e rsio n of a signal w ere
also in tro d u c e d in th is c h a p te r. Signal q u a n tiz a tio n is b est tre a te d in statistical
term s, as d esc rib e d in C h a p te rs 6 , 7. an d 9.
F in ally , th e to p ic o f signal re c o n stru c tio n , o r D /A co n v e rsio n , w as d escrib ed
briefly. Signal re c o n stru c tio n b ased on sta ircase o r lin e a r in te rp o la tio n m eth o d s is
tre a te d in S ectio n 9.3.
T h e re a re n u m e ro u s p ractical a p p lic a tio n s of d igital signal processing. T he
b o o k e d ite d by O p p e n h e im (1978) tre a ts a p p lic a tio n s to sp e ech p rocessing, im age
p ro cessin g , ra d a r signal pro cessin g , so n a r signal p ro cessin g , a n d g eophysical signal
p ro cessin g .

40

Introduction

Chap. 1

PROBLEMS

LI Classify the following signals according to whether they are (1) one- or multi
dimensional; (2) single or multichannel, (3) continuous time or discrete time, and
(4) analog or digital (in amplitude). Give a brief explanation.
(a) Closing prices of utility stocks on the New York Stock Exchange.
(b) A color movie.
(c) Position of the steering wheel of a car in motion relative to cars reference frame.
(d) Position of the steering wheel of a car in motion relative to ground reference
frame.
(e) Weight and height measurem ents of a child taken every month.
1.2 D eterm ine which of the following sinusoids are periodic and com pute their funda
mental period.
30n \
( 62m
(a) cosO.OIjt/i
(b) cs n - I
(c) cos 3
(d) sin3
(e) sin
105 /
'
'
V 10
1 3 Determ ine whether or not each of the following signals is periodic. In case a signal
is periodic, specify its fundamental period.
(a) xu(r) 3cos(5r + 7r/6)
(b)
= 3 cos(5n + ;r/6)
(c) j:(h) = 2 e x p [j(n /6 - 7i)]
(d) x(n) = cos(/8) cos(?rn/8)
(e) x(n) = cos(7rn/2) sin(7rn/8) + 3cos(jrn/4 + 7t / 3)
1.4 (a) Show that the fundamental period Nr of the signals
,s>() = ei2nkr,IN.

* = 0 .1 .2 ,...

is given by Np = N / C C D ( k . N ), where GCD is the greatest common divisor of k

and N.
(b) What is the fundam ental period of this set for N =71
(c) What is it for N = 16?
1.5 Consider the following analog sinusoidal signal:
xa(t) 3 sin(1007rr)
(a) Sketch the signal xa{t) for 0 < t < 30 ms.
(b) The signal xa(t) is sampled with a sampling rate Fs = 300 samples/s. Determ ine
the frequency of the discrete-time signal x{n) = xa( nT), T = 1 /F. and show that
it is periodic.
(c) Compute the sample values in one period of .x(n). Sketch _*<n) on the same
diagram with x(t). W hat is the period of the discrete-time signal in milliseconds?
(d) Can you find a sampling rate Fs such that the signal x(n) reaches its peak value
of 3? W hat is the minimum Fs suitable for this task?
L6 A continuous-time sinusoid xa(t) with fundamental period Tp = 1/F0 is sampled at a
rate F, = 1/ T to produce a discrete-time sinusoid x(n) = x(,nT).
() Show that x(n) is periodic if T / T p = k / N (i.e., T/ T p is a rational num ber).
(b) If x(n) is periodic, what is its fundam ental period Tp in seconds?

Chap. 1

41

Problem s

(c) Explain the statement: ,r(n) is periodic if its fundamental period Tr . in seconds,

is equal to an integer number of periods of .vu).

1.7 An analog signal contains frequencies up to 10 kHz.
(a) W hat range of sampling frequencies allows exact reconstruction of this signal
from its samples?
(b ) Suppose that we sample this signal with a sampling frequency F, = 8 kHz. Ex
amine what happens to the frequency F| = 5 kHz.
(c) Repeat part (b) for a frequency F> = 9 kHz.
1.8 An analog electrocardiogram (ECG) signal contains useful frequencies up to 100 Hz.
(a) W hat is the Nvquist rate for this signal?
(b ) Suppose that we sample this signal at a rate of 250 samples/s. What is the highest
frequency that can be represented uniquely at this sampling rate?
1.9 An analog signal a u ) = sin(480;rr) + 3sin(720:rr) is sampled 600 times per second.
(a) D eterm ine the Nvquist sampling rate for x a{t).
(b ) D eterm ine the folding frequency.
(c) What are the frequencies, in radians, in the resulting discrete time signal t (/j )?
(d) If
is passed through an ideal D/A converter, what is the reconstructed signal
v(n?
1.10 A digital communication link carries binary-coded words representing samples of an
input signal
(t) 3 cos 600.7 r

2 cos 1800jt /

The link is operated at 10.000 bits/s and each input sample is quantized into 1024
different voltage levels.
(a) W hat is the sampling frequency and the folding frequency?
( b ) W hat is the Nvquist rate for the signal .*(;)?
(c) What are the frequencies in the resulting discrete-time signal x(n)7
(d) W hat is the resolution A?
1.11 Consider the simple signal processing system shown in Fig. P I.11. The sampling
periods of the A/D and D /A converters are T = 5 ms and T' = 1 ms. respectively.
Determ ine the output vU) of the system, if the input is
a:(n = 3 cos 100;rf -+- 2 sin 250;rt

(t in seconds)

The postfilter removes any frequency component above F J 2.

Figure P l . l l

1.12 (a) Derive the expression for the discrete-time signal .r(n) in Example 1.4.2 using the
periodicity properties of sinusoidal functions.
(b) W hat is the analog signal we can obtain from x(n) if in the reconstruction process
we assume that Fs = 10 kHz?

42

Introduction

Chap. 1

1.13 The discrete-time signal x(n) = 6.35cos(jr/10)n is quantized with a resolution (a) A =
0.1 or (b) A = 0.02. How many bits are required in the A/D converter in each case?
1.14 D eterm ine the bit rate and the resolution in the sampling of a seismic signal with
dynamic range of 1 volt if the sampling rate is Fs = 20 samples/s and we use an S-bit
A/D converter? W hat is the maximum frequency that can be present in the resulting
digital seismic signal?
1.15* Sampling o f sinusoidal signals: aliasing Consider the following continuous-time si
nusoidal signal
XoO) = sin 2jr/r0f,

oc < t < oo

Since xa(t) is described m athematically, its sampled version can be described by values
every T seconds. The sampled signal is described by the formula

where Fs = l / T is the sampling frequency.

(a) Plot the signal j:(n), 0 < n < 99 for F, = 5 kHz and Fu = 0.5, 2, 3, and 4.5 kHz.
Explain the similarities and differences am ong the various plots.
(b) Suppose that F0 = 2 kHz and Fs = 50 kHz.
(1) Plot the signal x(n). W hat is the frequency / (l of the signal je(n)?
(2) Plot the signal v(n) created by taking the even-numbered samples of x(n).
Is this a sinusoidal signal? Why? If so, what is its frequency?
1.16* Quantization error in A /D conversion o f a sinuoidal signal Let xq(n) be the signal
obtained by quantizing the signal x(n) = sin27r/on. The quantization error power PQ
is defined by

The quality of the quantized signal can be measured by the signal-to-quantization

noise ratio (SQ NR) defined by
SQ N R = 10 log,0
a
where Px is the power of the unquantized signal x (n).
() For /o = 1/50 and N = 200, write a program to quantize the signal Jt(n), using
truncation, to 64, 128, and 256 quantization levels. In each case plot the signals
x (n), Xq(n), and e(n) and com pute the corresponding SQNR.
(b) Repeat part (a) by using rounding instead of truncation.
(c) Comment on the results obtained in parts (a) and (b).
(d) Compare the experimentally m easured SQNR with the theoretical SQNR pre
dicted by formula (1.4.32) and com m ent on the differences and similarities.

Discrete-Time Signals and

Systems

In C h a p te r 1 w e in tro d u c e d th e re a d e r to a n u m b e r o f im p o rta n t ty p es o f signals

an d d escrib ed th e sa m p lin g p ro cess bv w hich an a n a lo g signal is c o n v e rte d to a
d isc rete-tim e signal. In a d d itio n , we p re s e n te d in so m e d etail th e c h a racteristics
o f d isc re te -tim e sin u so id al signals. T h e sin u so id is an im p o rta n t e le m e n ta ry signal
th a t se rv es as a b asic b u ild in g block in m o re co m p lex signals. H o w e v e r, th e re are
o th e r e le m e n ta ry signals th a t are im p o rta n t in o u r tr e a tm e n t o f signal processing.
T h ese d isc re te -tim e signals a re in tro d u c e d in this c h a p te r a n d are used as basis
fu n ctio n s o r b u ild in g b locks 1o d escrib e m o re com plex signals.
T h e m a jo r e m p h asis in this c h a p te r is th e c h a ra c te riz a tio n o f d isc rete-tim e
sy stem s in g en era! a n d the class o f lin ear tim e -in v a ria n t (L T I) system s in p articu lar.
A n u m b e r o f im p o rta n t tim e-d o m ain p ro p e rtie s o f L T I sy stem s a re d efined and
d e v e lo p e d , an d an im p o rta n t fo rm u la, called th e c o n v o lu tio n fo rm u la, is d eriv ed
w hich allow s us to d e te rm in e th e o u tp u t of an L T I system to any given a rb itra ry
in p u t signal. In a d d itio n to th e c o n v o lu tio n fo rm u la , d iffe re n c e e q u a tio n s are in
tro d u c e d as an a lte rn a tiv e m e th o d fo r describ in g th e in p u t- o u tp u t re la tio n sh ip of
an L T I sy stem , an d in a d d itio n , recursive an d n o n re c u rsiv e re a liz a tio n s of LTI
sy stem s are tre a te d .
O u r m o tiv a tio n fo r th e em p h asis on th e stu d y o f L T I sy stem s is tw ofold. F irst,
th e re is a larg e co llectio n o f m a th e m a tic a l te c h n iq u e s th a t can be ap p lied to the
an aly sis o f L T I system s. S eco n d , m an y p ractical system s a re e ith e r L T I system s
o r can b e a p p ro x im a te d by L T I system s. B ecau se of its im p o rta n c e in digital
signal p ro cessin g a p p licatio n s an d its close re se m b la n c e to th e co n v o lu tio n form ula,
we also in tro d u c e th e c o rre la tio n b e tw e e n tw o signals. T h e a u to c o rre la tio n and
c ro ssc o rre la tio n o f signals a re defined an d th e ir p ro p e rtie s a re p re se n te d .

2.1 DISCRETE-TIME SIGNALS

A s w e d iscu ssed in C h a p te r 1, a d isc re te -tim e signal x{n) is a fu n ctio n o f an in d e
p e n d e n t v a ria b le th a t is an in teg er. It is g rap h ically re p re s e n te d as in Fig. 2.1. It
is im p o rta n t to n o te th a t a d isc re te -tim e signal is n o t defined at in sta n ts b etw een
43

44

Discrete-Time Signals and Systems

Figure 2-1

Chap. 2

Graphical representation of a discrete-time signal.

tw o successive sam p les. A lso , it is in c o rre c t to th in k th a t .v(n) is e q u a l to z e ro if n

is n o t an in teg er. S im ply, th e signal x ( n ) is n o t defin ed fo r n o n in te g e r v alu es o f n.
In th e se q u el w e will assu m e th a t a d isc re te -tim e signal is d efin ed fo r every
in te g e r value n fo r oo < n < oc. By tra d itio n , w e re fe r to x( n) as th e n th sa m p le
o f th e signal ev en if th e signal x( n) is in h e re n tly d isc re te tim e (i.e., n ot o b ta in e d
by sam p lin g an a n a lo g signal). If, in d e e d , x ( n) w as o b ta in e d fro m sa m p lin g an
a n alo g signal x a( t ), th e n .i(n ) = x a( nT) , w h ere T is th e sa m p lin g p e rio d (i.e., th e
tim e b etw e e n successive sam p les).
B e sid es th e g rap h ical re p re s e n ta tio n of a d isc re te -tim e signal o r se q u e n c e as
illu strated in Fig. 2.1. th e re a re som e a lte rn a tiv e re p re s e n ta tio n s th at are o ften
m o re co n v e n ie n t to use. T h e se are:
1. F u n ctio n al re p re s e n ta tio n , such as
fo r n = 1, 3
fo r n = 2
else w h e re

f 1,

x(n)

I4,

[ 0,

(2 . 1.1)

2. T a b u la r r e p re s e n ta tio n , such as
n

-2

x ( n)

-1
0

3. S eq u en ce re p re s e n ta tio n
A n in fin ite -d u ra tio n signal o r se q u e n c e w ith th e tim e o rig in (n = 0) in d ic a te d
by th e sy m b o l | is re p re s e n te d as
*<n) = { . . . 0 . 0 . 1 . 4 , 1 . 0 , 0 , . . . }
T

(2.1.2)

A se q u e n c e j:(n ), w hich is z e ro fo r n < 0, can be re p re s e n te d as

jc()

= { 0 ,1 .4 .1 .0 .0 ....}
T

(2.1.3)

T h e tim e o rig in fo r a se q u e n c e x ( n ) , w hich is z e ro fo r n < 0, is u n d e rs to o d to be

th e first (le ftm o st) p o in t in th e seq u en ce.

Sec. 2.1

45

Discrete-Time Signals

A fin ite -d u ra tio n se q u e n c e can be re p re se n te d as

x i n) = {3. - 1 . - 2 . 5 .0 .4 . -1 }

(2.1.4)

w h ereas a fin ite -d u ra tio n se q u e n c e th a t satisfies the c o n d itio n x(/i) ~ 0 for n < 0
can be r e p re s e n te d as

jc(n) = { 0 .1 .4 . 1)

(2.1.5)

T h e signal in (2.1.4) co n sists of seven sa m p le s or p o in ts (in tim e), so it is called or

id en tified as a se v e n -p o in t se q u e n c e . S im ilarly, the se q u e n c e given by (2.1.5) is a
f o u r-p o in t se q u e n c e .

2.1.1 Some Elementary Discrete-Time Signals

In o u r stu d y o f d isc re te -tim e signals an d system s th e re a re a n u m b e r o f b asic signals
th at a p p e a r o ften a n d play an im p o rta n t role. T h ese signals a re d efin ed below .
1. T h e unit s a m p l e se quence is d e n o te d as <5(n) an d is defin ed as
<5( / i )

0.

for n - 0
for n ^ 0

( 2 . 1. 6 )

In w o rd s, th e u n it sa m p le se q u e n c e is a signal th at is zero e v erv w h ere, ex cep t

a t n 0 w h e re its value is unity. T his signal is so m e tim e s re fe rre d to as a
unit impul se. In c o n tra st to the an alo g signal 8(t). w hich is also called a
u n it im p u lse an d is d efin ed to be ze ro ev ery w h ere ex cep t / = 0. an d has unit
a re a , th e u n it sa m p le se q u e n c e is m uch less m ath e m a tic a lly c o m p licated . T he
g rap h ical re p re s e n ta tio n o f <5(n ) is show n in Fig. 2.2.
2. T h e unil step signal is d e n o te d a s w (n ) an d is defin ed as
u(n) =

1.
0.

fo r n > 0
fo r n < 0

(2.1.7)

F ig u re 2.3 illu stra te s th e u nit ste p signal.

3. T h e uni t r a m p signal is d e n o te d as u r (n) an d is d efin ed as
u r (n) -

fo r n > 0
fo r n < 0

( 2 . 1.

T h is signal is illu stra te d in Fig. 2.4.

fi(n)

Figure 2.2 G raphical rep resen tatio n of

the unit sample signal.

46

Discrete-Tim e Signals and Systems

Chap. 2

u(n)

12

3 4 5 6 7

Figure 2 3 G raphical rep resen tatio n of

the unit step signal.

ur(n)

Figure 2,4 G raphical rep resen tatio n of

the unit ram p signal.

4. T h e exponent i al signal is a se q u e n c e o f th e fo rm
jr(n) = a

fo r all n

(2.1.9)

If th e p a r a m e te r a is real, th e n jr(n) is a real signal. F ig u re 2.5 illu stra te s x ( n)

fo r v ario u s v alu es o f th e p a r a m e te r a.
W h en th e p a ra m e te r a is co m p lex v a lu e d , it can b e e x p re ss e d as
a s rejf1
w h e re r an d 6 a re n o w th e p a ra m e te rs. H e n c e w e c a n e x p ress x ( n ) as
x ( n ) = r nej0n

(2 . 1. 10 )

= r n (cos On -j- y s in # n )

TiinilU

Figure 2.5

G raphical representation of exponential signals.

Sec. 2.1

47

Discrete-Time Signals

Since x (/j ) is now co m plex valu ed , it can be re p re s e n te d g rap h ically by p lo ttin g

th e real p a rt
x K(n) = r" cos6#fi

(2.1.11)

as a fu n ctio n of n. an d se p a ra te ly p lo ttin g th e im ag in ary p a rt

xi (n) = r 1sin 6n

(2.1.12)

as a fu n ctio n o f n. F ig u re 2.6 illu strates th e g ra p h s o f x R(n) a n d x / ( n ) for r 0.9

an d 6 = tt/1 0 . W e o b se rv e th a t th e signals x R(?i) a n d x / ( n ) a re a d a m p e d (decaying
e x p o n e n tia l) co sin e fu n ctio n an d a d a m p e d sine fu n c tio n . T h e angle v ariab le 6
is sim ply th e freq u e n c y of th e sinusoid, p rev io u sly d e n o te d by th e (n o rm alized )
fre q u e n c y v ariab le w. C learly, if r 1. th e d a m p in g d is a p p e a rs an d x K(n). x/ ( n).
an d A'(n) h av e a fixed am p litu d e, w hich is unity.
A lte rn a tiv e ly , th e signal .*(/?) given by (2.1.10) can be re p re s e n te d g raphically
by th e a m p litu d e fu n ctio n
|.i(h )| = A{n) = r

(2.1.13)

_ v (/;) = <f>{n) = Bn

(2.1.14)

an d th e p h ase fu n ctio n

F ig u re 2.7 illu stra te s -4(/?) an d 0 (/i) fo r r 0.9 a n d B = ,t/1 0 . W e o b se rv e th at

th e p h ase fu n ctio n is lin ear w ith n. H o w ev er, th e p h ase is d efin e d only o v e r the
in terv al n < B < t t o r. eq u iv alen tly , o v er th e in terv al 0 < 6 < 2 t t . C o n seq u en tly ,
by co n v e n tio n 4>(n) is p lo tte d o v er th e finite in terv al n < B < t t o t Q < $ < 2tt.
In o th e r w o rd s, w e su b tra c t m u ltip lies o f I n fro m <p(n) b e fo re p lo ttin g . In one
case. <p(n) is c o n s tra in e d to th e range n < 0 < n a n d in th e o th e r case <p(n) is
c o n stra in e d to th e ran g e 0 < fi < 2n. T h e su b tra c tio n o f m u ltip le s o f 2n from <p(n)
is e q u iv a le n t to in te rp re tin g th e fu n ctio n 4>(n) as 4>{n), m o d u lo 2n. T he g rap h for
<p{n). m o d u lo 2n . is show n in Fig. 2.7b.

2.1.2 Classification of Discrete-Time Signals

T h e m a th e m a tic a l m e th o d s e m p lo y ed in th e analysis of d isc re te -tim e signals and
system s d e p e n d on th e c h aracteristics o f th e signals. In this se ctio n we classify
d isc rete-tim e signals acco rd in g to a n u m b e r of d iffe re n t c h aracteristics.

Energy signals and power signals.

T h e en e rg y of a signal x( n ) is

d efin ed as
OC

E=

|jc(n)|2

(2.1.15)

n=-oc

W e h av e u se d th e m a g n itu d e -sq u a re d v alu es o f jr(n), so th a t o u r d efin itio n applies

to c o m p le x -v a lu e d signals as w ell as re a l-v a lu e d signals. T h e e n e rg y of a signal can
be fin ite o r in fin ite. If E is finite (i.e., 0 < E < oo), th e n x ( n ) is called an energy

Figure 2.6
signal.

G raph of the real and im aginary com ponents of a com plex-valued exponential

Sec. 2.1

49

Discrete-Time Signals

-1 0

1 2

11

~ S

(a* Gra ph of A {n I = r" . r = 0.9

( b t G r a p h ol

n . m o d u l o 2ir p l o n e d in t h e r a n g e It t , it I

Figure 2.7 G r a p h o f a m p l i tu d e and p h a s e function of a co m p l e x -v a l u e d e x p o n e n

tial sicnal: ( a ) gr ap h of A i m = r " . 4 = 0 . 9 : ( b ) g rap h of c/>in) = I . t / I O v j . m o d u lo
2ji p lo tt e d in the ran ge i - j i . t t J .

signal. S o m etim es w e add a su b scrip t .v to an d w rite , to em p h asize th a t , is

the en e rg y o f th e signal x( n).
M anv signals th a t possess infinite en erg y , h av e a finite av e ra g e p o w er. T he
av erag e p o w e r o f a d isc re te -tim e signal x( n) is defin ed as
P =

1
lim
** 2 N -j- 1 n -
-,\

(2 .1.1 6)

If we define th e signal en e rg y o f x(?i) o v er th e finite in terv al N < n < N as

(2.1.17)

th e n w e can e x p ress th e signal energy as

=

lim /v'
,\ oc

(2.1.18)

an d th e a v erag e p o w e r o f th e signal x ( n ) as
1
En
P = lim
n -+x 2 N + 1

(2.1.19)

50

Discrete-Tim e Signals and Systems

Chap. 2

C learly , if E is finite. P = 0. O n th e o th e r h a n d , if E is infinite, th e av erag e

p o w e r P m ay be e ith e r finite o r infinite. If P is finite (an d n o n z e ro ), th e signal is
called a p o w e r signal. T h e follow ing e x am p le illu stra te s such a signal.
Example 2.1.1
Determ ine the power and energy of the unit step sequence. The average power of
the unit step signal is

N + 1
lim --------------

A1-** 2 N

1 + l/N
1
lim ------------- --- 4- l / N
2

a9c 2

Consequently, the unit step sequence is a power signal. Its energy is infinite.
Sim ilarly, it can b e show n th a t th e co m p lex e x p o n e n tia l se q u e n c e x(n') =
A e JUJan h as av erag e p o w e r A 2, so it is a p o w e r signal. O n th e o th e r h an d , th e unit
ra m p se q u en ce is n e ith e r a p o w e r signal n o r an en e rg y signal.

Periodic signals and aperiodic signals. A s d efin e d o n S ection 1.3, a

signal x( n ) is p erio d ic w ith p e rio d N ( N > 0) if a n d only if
x( n + N ) = x( n ) fo r all n

(2 . 1.20 )

T h e sm allest v alu e o f N fo r w hich (2.1.20) h o ld s is called th e (fu n d a m e n ta l) p erio d .

If th e re is no v alu e o f N th a t satisfies (2.1.20), th e signal is called nonper i odi c or
aperiodic.
W e h av e a lread y o b se rv e d th a t th e sin u so id al signal o f th e form
x ( n) = A sin 2n f o n

( 2 . 1 .21 )

is p e rio d ic w h en /J, is a ra tio n a l n u m b e r, th a t is, if /o can be e x p re sse d as

(2 .1.22 )

w h ere k an d N a re integers.
T h e e n erg y o f a p erio d ic signal x ( n ) o v e r a single p e rio d , say. o v er th e in terv al
0 5 n < N - 1, is fin ite if x () ta k e s on finite v alu es o v e r th e p e rio d . H o w ev er, the
e n e rg y o f th e p e rio d ic signal fo r oc < n < oo is infinite. O n th e o th e r h an d , the
a v e ra g e p o w e r o f th e p erio d ic signal is finite an d it is e q u a l to th e av erag e p o w er
o v e r a single p e rio d . T h u s if x () is a p e rio d ic signal w ith fu n d a m e n ta l p e rio d N
an d ta k e s o n fin ite v alues, its p o w e r is g iven by

(2.1.23)
n=0
C o n s e q u e n tly , p erio d ic signals are p o w e r signals.

Sec. 2.1

Discrete-Time Signals

51

Symmetric (even) and antisymmetric (odd) signals.

A real v a lu ed sig

n al x ( n ) is called sy m m etric (ev en ) if

jr(-n ) =

()

(2.1.24)

O n th e o th e r h a n d , a signal x ( n ) is called a n tisy m m etric (o d d ) if

.v( - h ) = - x ( n )

(2.1.25)

W e n o te th a t if .v(/7) is odd, th e n x(0) = 0. E x am p le s o f signals w ith ev en an d odd

sy m m etry are illu stra te d in Fig. 2.8.
W e w ish to illu strate th a t any a rb itra ry signal can be e x p re sse d as th e sum of
tw o signal c o m p o n e n ts , o n e o f w hich is even an d th e o th e r o d d . T h e ev en signal
co m p o n e n t is fo rm e d by ad d in g x(/i) to x ( n) and div id in g by 2. th a t is.
= j[.v{7!) + x ( - ) ]

(2.1.26)

.v(n]

<'
4

!I
j T
] !

l M ! ....................

ll
-4-3-2-I

12

3 4

i,

(a)

.r(n)

- 5 - 4 - 3 - 2 -1

.ill'
12

3 4 5

<>

(b)
Figure 2.8

Exam ple of even (a) and odd (b) signals.

rt

52

Discrete-Time Signals and Systems

Chap. 2

Clearly, x e(n) satisfies the sym m etry con d ition (2.1.24). Sim ilarly, w e form an odd
signal com p onent x(n) according to the relation
x(n) = j[.x(n) - * ( - / i ) ]

(2.1.27)

A gain , it is clear that x 0(n) satisfies (2.1.25); hence it is in d eed odd. N ow , if we

add the tw o signal com p onents, defined by (2.1.26) and (2.1.27), w e ob tain ;c(n),
that is,
*(n ) = x e(n) + x n{n)

(2.1.28)

Thus any arbitrary signal can be exp ressed as in (2.1.28).

2.1.3 Simple Manipulations of Discrete-Time Signals

In this section w e consider som e sim ple m odifications or m anipulations involving
the in d ep en dent variable and the signal am plitude (depend en t variable).

Transformation of the independent variable (time). A signal x ( n ) may

be shifted in tim e by replacing the in d ep en dent variable n by n k, w here k is an
integer. If A: is a positive integer, the tim e shift results in a delay of the signal by
k units o f tim e. If k is a negative integer, the tim e shift results in an advance of
the signal by \k\ units in time.

Example 2.L2
A signal x ( n ) is graphically illustrated in Fig. 2.9a. Show a graphical representation
of the signals x ( n 3) and x ( n -I- 2).

Solution The signal x (/i 3) is obtained by delaying ;t(n) by three units in time. The
result is illustrated in Fig. 2.9b. On the other hand, the signal x(n + 2 ) is obtained by
advancing x ( n ) by two units in time. The result is illustrated in Fig. 2.9c. Note that
delay corresponds to shifting a signal to the right, whereas advance implies shifting
the signal to the left on the time axis.
If the signal x ( n ) is stored on m agnetic tape or on a disk or, perhaps, in the
m em ory o f a com puter, it is a relatively sim ple operation to m odify the base by
introducing a delay or an advance. O n the other hand, if the signal is not stored but
is b ein g generated by som e physical p h en om en on in real tim e, it is not p ossible
to advance the signal in tim e, since such an op eration in volves signal sam ples
that have not yet b een generated. W hereas it is alw ays possib le to insert a delay
into signal sam ples that have already b een generated, it is physically im possible
to view the future signal sam ples. C on sequ en tly, in real-tim e signal processing
applications, the operation o f advancing the tim e base o f the signal is physically
unrealizable.
A n o th er useful m odification o f the tim e base is to replace th e in d ep en dent
variable n by n. T h e result o f this op eration is a f o l d i n g or a reflection o f the
signal about the tim e origin n = 0.

Sec. 2.1

53

Discrete-Time Signals
xin)
4 i

j 1

I 4 3 2 1 0

1 2

x i n - 3)

il
T -l

1 2

3 4

S 6

(b)

xin +2)

- 6 - 5 - 4 -3 - 2 - 1

Figure 2.9 Graphical representation of

a signal, and its delayed and advanced
versions.

Example 2.1.3
Show the graphical representation of the signal x { - n ) and x i - n + 2). where x ( n ) is
the signal illustrated in Fig. 2.10a.
The new signal yin) = x ( n) is shown in Fig. 2.10b. Note that y(0) = *(0).
v ( l ) = x( 1). y(2) = _t( 2). and so on. Also, y (1) = jc(1 ) , v(2) = x(2), and so on.

Solution

Therefore, yin) is simply xin) reflected or folded about the lime origin n = 0. The
signal yin) x ( n
+
2) is simply x ( n) delayed by two units in time. The resulting
signal is illustrated in Fig. 2.10c. A simple way to verify that the result in Fig. 2.10c
is correct is to compute samples, such as y(0) = *(2), y (l) = jc(1 >, v(2) = ;t(0),
v(1) = jr(3). and so on.

It is im p o rta n t to n o te th a t th e o p e ra tio n s o f folding an d tim e d elay in g (o r

ad v an cin g ) a signal a re n o t co m m u ta tiv e . If we d e n o te th e tim e-d e la y o p e ra tio n
by T D a n d th e fo ld in g o p e ra tio n by F D . we can w rite
T D i[jc(n )l = jc(n - k)

k > 0
(2.1.29)

FD [jc(/j)] = x ( n)
N ow

T D A(FD[.r(n)]l = T D *[.t(-n)] = x ( - n + k)

(2.1.30)

54

Discrete-Time Signals and Systems

Chap. 2

y(n) = x(-n + 2)

-2 -1

in

0 1 2 3 4 5

(c)

Figure 2.10 Graphical illustration of

the folding and shifting operations.

w hereas
F D {T D i[j:(n )]} = FD[j:(/i &)] = x ( n k )

(2.1.31)

N o te that becau se the signs o f n and k in x { n k) and Jt(-n + ik ) are different, the re
sult is a shift o f the signals x ( n ) and x ( n) to the right by k sam p les, corresponding
to a tim e delay.
A third m odification o f the in d ep en dent variable in volves replacing n by fin,
w here /x is an integer. W e refer to this tim e-b ase m odification as time scaling or
dow nsam pling.

Example 2.L4
Show the graphical representation of the signal y(n) = x(2n), where x(n) is the signal
illustrated in Fig. 2.11a.

Solution We note that the signal y(n) is obtained from x(n) by taking every other
sample from jc(), starting with x(0). Thus y(0) = x(0), y{l) = x(2), y(2) = jc(4),
and y ( - l ) = x(~ 2 ), y ( -2 ) = jc(4), and so on. In other words, we have skipped

Sec. 2.1

55

Discrete-Time Signals

v(n) = .v(2n)

-4

! -2

0 I 2 3

(b)
Figure 2.11

Graphical illustration ot down-samplinc operation.

the odd-num bered samples in *() and retained the even-num bered samples. The
resulting signal is illustrated in Fig. 2.11b.
If th e signal ,\ () w as o riginally o b ta in e d by sa m p lin g an a n a lo g signal x a(t),
th e n jc() = Xa(nT), w h ere T is the sa m p lin g in terv al. Nowr. v(n) = x ( 2n) =
x a(2Tn). H e n c e th e tim e-scalin g o p e ra tio n d escrib ed in E x a m p le 2.1.4 is e q u iv a le n t
to ch an g in g th e sam p lin g ra te from 1 /T to 1/27". th a t is, to d e c re a s in g the ra te by
a facto r o f 2. T h is is a d o w n s a mp l i n g o p e ra tio n .

Addition, multiplication, and scaling of sequences. A m p litu d e m o d ifi

catio n s in clu d e addition, multiplication, an d scaling o f d isc re te -tim e signals.
A m p l i t u d e scaling o f a signal by a c o n sta n t A is acco m p lish ed by m u ltiplying
th e v alu e o f ev ery signal sa m p le by A. C o n se q u e n tly , w e o b ta in
v(n) = Ax ( n )

oc < fi < oc

T h e s u m o f tw o signals xj ( n) a n d xz i n) is a signal _v(n), w h o se v alu e at any

in stan t is equal to th e sum o f th e values o f th e se tw o signals a t th a t in sta n t, th a t is.
y(/t) = jq (n) + X2 <n)

oc < n < oc

T h e p r o d u c t o f tw o signals is sim ilarly defin ed o n a sa m p le -to -sa m p le basis as

v(n) = X](n)X 2 (n)

oo < n < oo

56

Discrete-Time Signals and Systems

Chap. 2

2.2 DISCRETE-TIME SYSTEMS

In m a n y ap p lic a tio n s o f d ig ital signal p ro cessin g w e w ish to desig n a d ev ice or
an alg o rith m th a t p e rfo rm s so m e p re sc rib e d o p e ra tio n o n a d isc re te -tim e signal.
S uch a d evice o r a lg o rith m is called a d isc re te -tim e system . M o re specifically, a
discrete-time s y s t em is a d ev ice o r a lg o rith m th a t o p e ra te s on a d isc re te -tim e signal,
called th e i nput o r excitation, acco rd in g to so m e w ell-defined ru le , to p ro d u c e a n
o th e r d isc rete-tim e signal called th e out p u t o r response of th e system . In g en eral,
we view a system as an o p e r a tio n o r a se t o f o p e ra tio n s p e rfo rm e d on th e in p u t
sig n al x( n) to p ro d u c e th e o u tp u t signal _v(n). W e say th a t th e in p u t signal x( n) is
t r ans f ormed by th e sy stem in to a signal >(), an d ex p ress th e g e n e ra l re la tio n sh ip
b e tw e e n jc () a n d y ( n ) as
y( n) = T[x{n) }
(2 .2 .1)
w h ere th e sym bol T d e n o te s th e tra n s fo rm a tio n (also called an o p e r a to r) , o r p r o
cessing p e rfo rm e d by th e sy stem on ;c(n) to p ro d u c e y(n). T h e m a th e m a tic a l
re la tio n sh ip in (2.2.1) is d e p ic te d g rap h ically in Fig. 2.12.
T h e re are v ario u s w ays to d escrib e th e c h a ra c te ristic s of th e system an d th e
o p e ra tio n it p e rfo rm s on x( n ) to p ro d u c e y(n ). In this c h a p te r w e shall b e c o n
c e rn e d w ith th e tim e -d o m a in c h a ra c te riz a tio n o f system s. W e shall begin w ith
an in p u t-o u tp u t d e sc rip tio n o f th e system . T h e in p u t-o u tp u t d e sc rip tio n focuses
on th e b e h a v io r at th e te rm in a ls of th e system a n d ignores th e d e ta ile d in te rn a l
co n stru ctio n o r re a liz a tio n o f th e system . L a te r, in S ection 7.5. w e in tro d u c e th e
sta te -sp a c e d e sc rip tio n o f a system . In this d e sc rip tio n w e d e v e lo p m a th e m a ti
cal e q u a tio n s th a t n o t o nly d esc rib e th e in p u t- o u tp u t b e h a v io r o f th e sy stem b u t
specify its in te rn a l b e h a v io r a n d stru c tu re .

2.2.1 Input-Output Description of Systems

T h e in p u t-o u tp u t d e sc rip tio n o f a d isc re te -tim e sy stem consists o f a m a th e m a tic a l
ex p ressio n o r a ru le, w hich explicitly d efin es th e re la tio n b e tw e e n th e in p u t an d
o u tp u t signals ( i n p u t - o u t p u t relationship). T h e ex act in te rn a l s tru c tu re o f th e sys
tem is e ith e r u n k n o w n o r ig n o red . T h u s th e o n ly w ay to in te ra c t w ith th e sy stem is
by u sin g its in p u t an d o u tp u t te rm in a ls (i.e., th e system is a ssu m ed to be a black
b o x to th e u se r). T o reflec t th is p h ilo so p h y , w e u se th e g ra p h ic a l re p re se n ta -

x (n J

Input signal
or excitation
Figure 2.12

Discrete-time
System
Output signal
or response

Block diagram representation of a discrete-tim e system.

Sec. 2.2

57

Discrete-Time Systems

tio n d e p ic te d in Fig. 2.12, an d th e g e n e ra l in p u t-o u tp u t re la tio n sh ip in (2.2.1) or,

a lte rn a tiv e ly , th e n o ta tio n
Jt(n)

y(n)

(2.2.2)

w hich sim ply m e a n s th a t v(n) is th e re sp o n se of the system T to th e e x c ita tio n

x{n). T h e fo llo w in g e x am p les illu stra te se v era l d iffe re n t system s.
Example 2.2.1
D eterm ine the response of the following sytems to the input signal
x(n) = , A
u,

-3 < n < 3
otherwise

y(n) = x{n)
v() = x in i)
y(n) = x i n 4- i)
y i n ) = j[A-(n + 1) + x ( n ) + x i n - D]
(e) y ( n) = m a x { x( n + 1), x ( n ) . x ( n 1)1
(0 y ( n ) = Z L . x x ( k ) = x ( n ) + x ( n 1) + x{n 2) -t

(a)
(b)
(c)
(d)

Solution

(2.2.3)

First, we determ ine explicitly the sample values of the input signal
xin) = ( ....0 .3 ,2 .1 .0 .1 .2 , 3 ,0 ,...)
T

Next, we determ ine the output of each system using its input-output relationship.
(a) In this case the output is exactly the same as the input signal. Such a system is
known as the identity system.
(b) This system simply delays the input by one sample. Thus its output is given by
x{n) = { ...,0 ,3 .2 .1 ,0 ,1 .2 .3 ,0 ,..,)
t
(c) In this case the system advances the input one sample into the future. For
example, the value of the output at time n = 0 is y(0) = *(1). The response of
this system to the given input is
x(n) = { ...,0 ,3 . 2 .1 .0 ,1 ,2 . 3 ,0 ....}
t
(d) The output of this system at any time is the mean value of the present, the
im m ediate past, and the immediate future samples. For example, the output at
time n = 0 is
y(0) =

+ x(0) + jr(l)] = |[1 + 0 + 1] = |

R epeating this com putation for every value of n, we obtain the output signal
>() = { ...0 ,1 , f , 2 , l j . l . 2 , , 1 .0 ,...)
t

58

Discrete-Time Signals and Systems

Chap. 2

(e> This system selects as its output at time n the maximum value of the three input

samples x(n - l.l, .v(n). and ,r(n + 1). Thus the response of this system to the
input signal .\{n) is
v(n) = {0.3. 3. 3. 2 .1 .2 . 3, 3, 3 . 0 . . . . )
t

(f) This system is basically an accumulator that computes the running sum of all
the past input values up to present time. The response of this system to the
given input is
v(n) = {.. ..0 .3 . 5. 6. 6, 7, 9. 1 2 .0 ....}
T
W e o b se rv e th a t fo r several of th e sy stem s c o n sid e re d in E x a m p le 2.2.1 the
o u tp u t at tim e n no d e p e n d s n ot only on th e v alu e of the in p u t at n = n (, [i.e.,
jc(o)]- b u t also on th e values o f the in p u t a p p lie d to th e system b e fo re an d after
n = n (). C o n sid er, fo r in stan ce, th e a c c u m u la to r in th e ex a m p le . W e see th at the
o u tp u t at tim e n = ?i() d e p e n d s n ot only on th e in p u t a t tim e n = no. b u t also on
x ( n ) a t tim es n = no 1. no - 2, and so on. By a sim ple a lg e b ra ic m a n ip u latio n
th e in p u t-o u tp u t re la tio n o f th e a c c u m u la to r can b e w ritte n as

= y(/i - 1) + x(ti)
w hich justifies th e te rm accumul at or. In d e e d , th e system c o m p u te s th e c u rre n t
v alu e o f th e o u tp u t by a d d in g (a ccu m u latin g ) th e c u rre n t v alu e o f th e in p u t to th e
p rev io u s o u tp u t value.
T h e re are so m e in te re stin g co n clu sio n s th a t can be d raw n by tak in g a close
lo o k in to this a p p a re n tly sim ple system . S u p p o se th a t we are given th e in p u t signal
x(rt ) fo r n > no. a n d we wish to d e te rm in e th e o u tp u t v(/i) o f th is system fo r n > no.
F o r n = no. no + 1........ (2.2.4) gives
v (n n) = v(o - 1) -f x (/i0)
_v(no + l l

v ( o ) + x ( n o + 1)

an d so on. N o te th a t we h ave a p ro b le m in c o m p u tin g y ( n a), since it d e p e n d s on

y(o - 1). H o w ev er,
y(t io - 1) =

x(k)

k X .

th a t is. y(no - 1) sum m arizes* th e effect on th e system from all the in p u ts w hich
h ad b e e n ap p lied to th e system b efo re tim e no- T h u s th e re sp o n s e of th e system
fo r n > no to th e in p u t x(/7j th a t is a p p lie d a t tim e no is th e c o m b in e d resu lt of this
in p u t an d all in p u ts th a t h ad b e e n a p p lie d p re v io u sly to th e sy stem . C o n seq u en tly .
y(/i), n > no is n o t u n iq u ely d e te rm in e d by th e in p u t x ( n ) fo r n > no.

Sec. 2.2

59

Discrete-Time Systems

T he additional inform ation required to d eterm ine y ( n) for n > no is the initial
condi t i on y(no - 1). T his value sum m arizes the effect o f all p reviou s inputs to the.
system . Thus the initial con d ition y(o - 1) togeth er with the input seq u en ce x ( n )
for n > no uniquely d eterm ine the output sequ en ce y(n ) for n > n0.
If the accum ulator had no excitation prior to n 0, the initial con d ition is y(no
1) = 0. In such a case w e say that the system is initially relaxed. S ince y(no 1) = 0,
the output seq u en ce y(n) d ep en d s only on the input seq u en ce x ( n ) for n > n0.
It is custom ary to assum e that every system is relaxed at n = oo. In this
case, if an input x ( n ) is applied at n = co, the corresponding output y( n) is solely
and uni qu e l y determ ined by the given input.

Example 2.2.2
T he accum ulator described by (2.2.3) is excited by the sequence x(n) = nu(n). D e
term ine its output under the condition that:

(a) It is initially relaxed [i.e., v ( - l ) = 0].

<b) Initially, y ( 1 )= 1.
Solution

The output of the system is defined as

tl
-]
y(n) = ^
x(k) =
x(k) +
*=-oc

*=-oc

r
i= < l

x(k)

= y (-l) +
k=o
But

n(n -f 1)

(a) If the system is initially relaxed, v(1) = 0 and hence

n(n + l)
v (n)

-------- 2 --------

"

(b) On the other hand, if the initial condition is y ( - l ) = 1, then

n(n -I-1)
n2 + n + 2
v(n) = 1 + ---- - - = -------------

n > 0

2.2.2 Block Diagram Representation of Discrete-Time

Systems
It is useful at this point to introduce a block diagram representation o f d iscrete
tim e system s. For this purpose w e n eed to define som e basic building blocks that
can b e intercon n ected to form com p lex system s.

An adder. F igure 2.13 illustrates a system (adder) that perform s the addi
tion o f tw o signal seq u en ces to form another (th e sum ) seq u en ce, w hich w e d en ote

Discrete-Time Signals and Systems

60

Chap. 2

x|(n )

y(n) = i,( n ) + x2(n)

Figure 2.13 Graphical representation

of an adder.

as y (n ). N o te th a t it is n o t n ecessa ry to sto re e ith e r o n e o f th e se q u e n c e s in o rd e r

to p e rfo rm th e a d d itio n . In o th e r w ords, th e a d d itio n o p e ra tio n is memor yl ess .

A constant multiplier. T his o p e ra tio n is d e p ic te d by Fig. 2.14, an d sim ply

re p re s e n ts ap p ly in g a scale fa c to r on th e in p u t x ( n) . N o te th a t th is o p e ra tio n is
also m em oryless.
a
------------------------ -

v(n) = o i( n )

Figure 2.14 Graphical representation

of a constant multiplier.

A signal multiplier. F ig u re 2.15 illu stra te s th e m u ltip lic a tio n o f tw o sig

nal se q u en ces to fo rm a n o th e r (th e p ro d u c t) se q u e n c e , d e n o te d in th e figure as
y (n ). A s in th e p re c e d in g tw o cases, w e can view th e m u ltip lic a tio n o p e ra tio n as
m em o ry less.
A|(n)

v(n) = jT|(n)Ai(n)

---- -0 ^

Figure 2.1S Graphical representation

of a signal multiplier.

x2( n )

A unit delay element. T h e u n it d elay is a sp e cial sy stem th a t sim ply d elay s

th e signal passing th ro u g h it by one sam p le. F ig u re 2.16 illu stra te s such a system .
If th e in p u t signal is x ( n) , th e o u tp u t is x( n 1). In fact, the sa m p le x{n 1) is
sto re d in m em o ry at tim e n 1 a n d it is recalled fro m m e m o ry a t tim e n to form
v ( n ) = x( n - 1)
T h u s th is basic b u ild in g b lock re q u ire s m em o ry . T h e use o f th e sym bol ; _1 to
d e n o te th e u n it o f d eiay will b eco m e a p p a r e n t w h e n w e discuss th e z -tra n sfo rm in
C h a p te r 3.
x(n)
------------------ -

y (n ) = jr ( n 1)

_____

Figure 2,16 Graphical representation

of the unit delay element.

A unit advance element. In c o n tra st to th e u n it d e la y , a u nit ad v an ce

m o v es th e in p u t x ( n ) a h e a d by o n e sa m p le in tim e to yield x ( n + 1). F ig u re 2.17
illu stra te s th is o p e ra tio n , w ith th e o p e r a to r ; b ein g used to d e n o te th e u n it advance.

Sec. 2.2

Discrete-Time Systems

61
y( n ) = x( n + I )

x( n)

Figure 2.17 Graphical representation

of the unit advance element.

W e o b se rv e th a t an y such ad v an ce is physically im possible in real tim e, since, in

fact, it in v o lv es lo o k in g in to th e fu tu re o f th e signal. O n th e o th e r h an d , if we store
th e signal in th e m em o ry o f th e c o m p u te r, w e can recall any sa m p le a t any tim e.
In such a n o n re a l-tim e ap p lic a tio n , it is p o ssible to advance th e signal jr(?r) in tim e.
Example 2.2.3
Using basic building blocks introduced above, sketch the block diagram representa
tion of the discrete-time system described by the input-output relation.
v(n) = 3.v( - 1) + | x(n) + \x(n - 1)

where x(n) is the input and y(n) is the output of the system.
Solution According to (2.2.5), the output v(n) is obtained by multiplying the input
x(n) by 0.5, multiplying the previous input jr ( n - l) by 0.5. adding the two products, and
then adding the previous output v(n 1) multiplied by j. Figure 2.18a illustrates this
block diagram realization of the system. A simple rearrangem ent of (2.2.5). namely.
v ( ) =

.v(n -

)+

[jc(k) + x ( n -

)|

( 2 . 2 .6 )

leads to the block diagram realization shown in Fig. 2.18b. Note that if we treat "the
system from the viewpoint of an input-output or an external description, we are
not concerned about how the system is realized. On the other hand, if we adopt an
Black box
0.5

-i
x( n )

(a)
Black box

-i
x( n)

Figure 2.18 Block diagram realizations of the system y(n) = 0.25y(n 1) +

0.5 x{n) + 0.5j(n 1).

Discrete-Time Signals and Systems

62

Chap. 2

internal description of the system, we know exactly how the system building blocks
are configured. In terms of such a realization, we can see that a system is relaxed at
time n = no if the outputs of all the delays existing in the system are zero at n = n{)
(i.e., all memory is filled with zeros).

2.2.3 Classification of Discrete-Time Systems

In th e analysis as w ell as in th e design o f system s, it is d e s ira b le to classify the
sy stem s acco rd in g to th e g e n e ra l p ro p e rtie s th a t th e y satisfy. In fact, th e m a th e
m atical te c h n iq u e s th a t w e d e v e lo p in th is an d in s u b s e q u e n t c h a p te rs fo r analyzing
an d d esig n in g d isc rete-tim e system s d e p e n d h eav ily on th e g e n e ra l ch aracteristics
of th e system s th a t are b ein g c o n sid e re d . F o r th is re a so n it is n ecessa ry for us
to d ev elo p a n u m b e r o f p ro p e rtie s o r c a te g o rie s th a t can be u se d to d escrib e th e
g e n e ra l ch aracteristics o f system s.
W e stress th e p o in t th a t fo r a sy stem to po ssess a given p ro p e rty , th e p ro p e rty
m u st h o ld fo r ev ery p o ssible in p u t signal to th e system . If a p ro p e rty holds for
so m e in p u t signals b u t n ot fo r o th e rs, th e system d o e s n ot p o ssess th a t p ro p erty .
T h u s a c o u n te re x a m p le is sufficient to p ro v e th a t a system d o e s n ot possess a
p ro p e rty . H o w ev er, to p ro v e th a t th e system has so m e p ro p e rty , we m ust prove
th a t th is p ro p e rty h o ld s fo r every po ssib le in p u t signal.

Static versus dynamic systems. A d isc re te -tim e sy stem is called static

o r m em o ry le ss if its o u tp u t at any in sta n t n d e p e n d s at m o st o n the in p u t sam ple
at th e sam e tim e, b u t n o t o n p a st o r fu tu re sa m p le s o f th e in p u t. In any o th e r case,
th e system is said to b e d y n a mi c o r to h av e m em o ry . If th e o u tp u t o f a system at
tim e n is co m p letely d e te rm in e d by th e in p u t sa m p le s in th e in te rv a l fro m n - N
to n ( N > 0), th e system is said to h av e m e m o r y o f d u ra tio n N . U N 0. th e
sy stem is static. H 0 < N < oo, th e sy stem is said to have f ini te m e m o r y , w h ereas
if N = oo, th e system is said to have infinite m e m o r y .
T h e system s d escrib ed by th e follow ing in p u t- o u tp u t e q u a tio n s
y(n) = a x {n)

(2.2.7)

y ( n ) = nx ( n ) + b x 3(n)

(2.2.8)

a re b o th static o r m em o ry less. N o te th a t th e re is n o n e e d to s to re any o f th e past

in p u ts o r o u tp u ts in o rd e r to c o m p u te th e p re se n t o u tp u t. O n th e o th e r h an d , th e
sy stem s d escrib ed by th e follow ing in p u t-o u tp u t re la tio n s
y( n) = x ( n ) + 3 x ( n 1)
y(n) = J ^ x ( n - k )
k=0

(2.2.9)
(2.2.10)

y( n) = J 2 x ( n - k )
Jt=0

(2.2.11)

are dynamic systems or systems with memory. The systems described by (2.2.9)

Sec. 2.2

63

Discrete-Time Systems

an d (2.2.10) h av e fin ite m em o ry , w h ereas the sy stem d e sc rib e d by (2.2.11) has

infinite m em o ry .
W e o b se rv e th a t sta tic o r m em oryless system s are d e sc rib e d in g e n e ra l by
in p u t-o u tp u t e q u a tio n s o f th e form
y(n) = T [ x ( n ) , n]

(2.2.12)

a n d th e y do n o t in clu d e d elay e le m e n ts (m em o ry ).

Time-invariant versus time-variant systems. W e can su b d iv id e th e g en

eral class o f sy stem s in to th e tw o b ro a d c a teg o ries, tim e -in v a ria n t system s and
tim e -v a ria n t sy stem s. A system is called tim e -in v a ria n t if its in p u t- o u tp u t c h a ra c
teristics d o n o t c h a n g e w ith tim e. T o e la b o ra te , su p p o se th a t w e h av e a system T
in a re la x e d s ta te w hich, w h en ex cited by an in p u t signal x ( n) , p ro d u c e s an o u tp u t
signal y(n). T h u s w e w rite
y( n) = T [ x { n) )

(2.2.13)

N o w su p p o se th a t th e sam e in p u t signal is d e la y e d by k u n its o f tim e to yield

x (n - &), an d ag ain a p p lied to th e sam e system . If th e c h a ra c te ristic s of th e system
d o n o t ch an g e w ith tim e, th e o u tp u t o f th e relax ed system will b e y( k). T h at is,
th e o u tp u t will b e th e sa m e as th e resp o n se to x ( n) . ex cep t th a t it will be d elay ed
by th e sam e k u n its in tim e th a t the in p u t w as d elay ed . T his lead s us to define a
tim e -in v a ria n t o r sh ift-in v a ria n t system as follow s.

Definition.

A relax ed system T is time i nvariant o r shift i nvariant if and

o n ly if
x( n)

y(n)

im p lies th a t
x {n k)

y( n k)

(2.2.14)

fo r ev ery in p u t signal x (n ) a n d every tim e shift k.

T o d e te rm in e if an y given system is tim e in v a ria n t, w e n e e d to p e rfo rm the

te st specified b y th e p re c e d in g definition. B asically, we ex cite th e system w ith an
a rb itra ry in p u t se q u e n c e x ( n) , w hich p ro d u c e s an o u tp u t d e n o te d as y ( n) . N ext
w e d elay th e in p u t se q u e n c e by sam e a m o u n t k an d re c o m p u te th e o u tp u t. In
g e n eral, w e can w rite th e o u tp u t as
y(, k) = T [ x ( n <:)]
N o w if th is o u tp u t y{n, k) = y{n k), for all p o ssib le v alu es o f k, th e system is
tim e in v a ria n t. O n th e o th e r h a n d , if th e o u tp u t y( n, k ) ^ y ( n k), ev en fo r o n e
v alu e o f k, th e sy stem is tim e v arian t.

Discrete-Time Signals and Systems

64

Chap. 2

VI 71) = X( 77I- V<71 - ] I

xin )

D ifferentiator"

- B

x(/t)
Time" multiplier

v( n ) = xl - n )
Folder"

v(n J = .u n ) cos oiii

Figure 2.19 Examples of a

lime-invariant (a) and some time-variant
systems (h)-(d).

Example 2.2.4
Determ ine if the systems shown in Fig. 2.19 are time invariant or time variant.
Solution
(a) This system is described by the input^output equations
y(7i) = T\ xin)} = x(n\ x(n - 1)

(2.2.15)

Nov, if the input is delayed by k units in time and applied to the system, it is
clear from the block diagram that the output will be
y i n . k) = x i n - k) x i n k 1)

(2.2.16)

On the other hand, from (2.2.14) we note that if we delay y (n) by k units in
time, we obtain
yin k) = x(n k) xin k 1)

(2.2.17)

Since the right-hand sides of (2.2.16) and (2.2.17) are identical, it follows that
v(n. k) = yin - k). Therefore, the system is time invariant.

Sec. 2.2

Discrete-Time Systems

65

(b) The input-output equation for this system is

y(n) = T[x(n)] = nx(n)

(2.2.18)

The response of this system to x(n - Jt) is

y(n, k) = nx(n - k)

(2.2.19)

Now if we delay ;y(n) in (2.2.18) by k units in time, we obtain

y(n - k) = (n k)x(n k)
(2 .2.20)
= nx(n k) - kx(n - k)

This system is time variant, since y(n, k) ^ y(n - k).

(c) This system is described by the input-output relation
>() = T[x(n)\ = x ( - n )

(2.2.21)

The response of this system to jr(n - k) is

;y(n, A;) = T[x(n - *)] = x ( - n - k)

(2.2.22)

Now, if we delay the output ;y(n), as given by (2.2.21), by k units in time, the
result will be
y(n - k) = x ( - n + k)
(2.2.23)
Since y(n, k) ^ y{n - it), the system is time variant.

(d) The input-output equation for this system is

y(n) = j:(n) costDon

(2.2.24)

The response of this system to x(n - k) is

y(n, k) = x(n - k) cos o\)n

(2.2.25)

If the expression in (2.2.24) is delayed by k units and the result is compared to

(2.2.25), it is evident that the system is time variant.

Linear versus nonlinear systems. The general class o f system s can also
be subdivided into linear system s and nonlinear system s. A linear system is one
that satisfies the su pe rp ositio n princ iple. Sim ply stated, the principle o f su p erp osi
tion requires that the resp onse o f the system to a w eighted sum o f signals b e equal
to the corresponding w eighted sum of the responses (outp u ts) of the system to each
of the individual input signals. H en ce w e have the follow ing definition o f linearity.
Definition.

A r e la x e d T s y s te m is lin e a r if a n d only if
T [ a i x i ( n ) + azx2{n)] = a\ T [ x \ ( n ) ] + a i T [ x 2{n)]

(2.2.26)

for any arbitrary input seq u en ces x\ ( n) and x 2(n), and any arbitrary constants aj
and 0 2 Figure 2.20 gives a pictorial illustration of the superposition principle.

66

Discrete-Time Signals and Systems

Chap. 2

if and only if v(n) = v'(n).

T h e su p e rp o sitio n p rin cip le e m b o d ie d in th e re la tio n (2.2.26) can be s e p a

r a te d in to tw o p arts. F irst, su p p o se th a t a2 = 0. T h e n (2.2.26) re d u c e s to
T{a\ X\ ( n) ] = a\ T [ x \ { n ) } = a\ vi(n)

(2.2.27)

w h ere
vi (fl) = T [ x x(n)}

T h e re la tio n (2.2.27) d e m o n s tra te s the mul ti pli cat i ve o r scaling p r o p e r t y of a lin ear
system . T h a t is, if th e re sp o n se o f th e system to th e in p u t x i ( n ) is vi(n ), the
re sp o n se to a\X](n) is sim ply a i j i(n ). T h u s any scaling of th e in p u t resu lts in an
id en tical scaling o f th e c o rre sp o n d in g o u tp u t.
S eco n d , su p p o se th a t ai = a2 = 1 in (2.2.26). T h e n
T [ x \ ( n ) + x 2 (n)] = T [ x \ { n ) ] + T [ x \ ( n ) }

(2.2.28)
= yi ( n) + yz(n)
T his re la tio n d e m o n s tra te s th e additivity pr ope r t y o f a lin e a r sy stem . T h e ad ditivity
an d m u ltip lic ativ e p ro p e rtie s c o n stitu te th e su p e rp o s itio n p rin c ip le as it ap p lies to
lin ear system s.
T h e lin earity co n d itio n em b o d ie d in (2.2.26) can b e e x te n d e d a rb itra rily to
any w eig h ted lin e a r c o m b in a tio n o f signals by in d u ctio n . In g e n e ra l, we h av e
M- 1
x( n) = ^ 2 GkXk(n)

M- 1
y ( n ) = ^ akyk (n)

k=l

(2.2.29)
i= l

w h ere
^ ( n ) = T [ x k(n)}

k = 1, 2, , . . , M 1

(2.2.30)

Sec. 2.2

67

Discrete-Time Systems

W e o b se rv e fro m (2.2.27) th a t if a i 0, th e n y (n ) = 0. In o th e r w ords, a r e

lax ed , lin e a r sy stem w ith z e ro in p u t p ro d u c e s a z e ro o u tp u t. If a system p ro d u c e s
a n o n z e ro o u tp u t w ith a zero in p u t, th e system m ay be e ith e r n o n re la x e d o r n o n
lin ear. If a re la x e d sy stem d o e s n o t satisfy th e su p e rp o s itio n p rin cip le as given by
th e d efin itio n ab o v e, it is called nonlinear.
Exam ple 2.2^
D eterm ine if the systems described by the following input-output equations are linear
or nonlinear.
(a) y(n) = ttx(n)

(b) y(n) = *(n2)

<d) y( n) = Ax{n) + B

(c) v(n) = v2(n)

(e) y(n) = ex[n]

Solution
(a) For two input sequences jti(n) and

the corresponding outputs are

Vi(n) = n.ifi(n)

(2.2.31)

y2{n) = nx2(n)
A iinear combination of the two input sequences results in the output
Vj() = T[a\Xy (n) + oijMh)] =

() +

(/i)]
(2.2.32)

= ainxi (n) + a2nx2(n)

On the other hand, a linear combination of the two outputs in (2.2.31) results
in the output
a\ V] (n) + a2y 2(n) = ainx\(n) + a2n,x2(n)

(2.2.33)

Since the right-hand sides of (2.2.32) and (2.2.33) are identical, the system is
iinear.
(b) As in part (a), we find the response of the system to two separate input signals
*i(n) and x 2(n). The result is
v,(n) = X\(n2)
(2.2.341
y2(rt) = X2(n2)
The output of the system to a linear combination of Xi(n) and *;(?) is
y3(n) = T\a\X\ () + a 2x 2(n)] = a hx,(n2) + a2x2(n2)

(2.2.35)

Finally, a linear combination of the two outputs in (2.2.36) yields

O] \>i(n) + c 2V2(n) = a i JCi (n2) + <i2X2(n2)

(2.2.36)

By comparing (2.2.35) with (2.2.36). we conclude that the system is linear.

(c) The output of the system is the square of the input. (Electronic devices that
have such an input-output characteristic and are called square-law devices.)
From our previous discussion it is clear that such a system is memoryless. We
now illustrate that this system is nonlinear.

68

Discrete-Time Signals and Systems

Chap. 2

The responses of the system to two separate input signals are

v,() = .vf(n)
'
y2(n) = x;(n)

(2.2.37)

The response of the system to a linear combination of these two input signals is
>'3<n ) = T[a 1*1 (n) + a2x2(n)}
= [fli*i(n) + a2x2(n)]2

(2.2.38)

= cfA 'fln) -I- 2a-la 2x i ( n ) x 2(n) + a 2x 2 { n )

On the other hand, if the system is linear, it would produce a linear combination
of the two outputs in (2.2.37). namely,
ai_Vi(n) + fl2.V2(n) =

(rt) + a2x2(n)

(2.2.39)

Since the actual output of the system, as given by (2.2.38). is not equal to
(2.2.39), the system is nonlinear.

(d) Assuming that the system is excited by x\(n) and x2in) separately, we obtain
the corresponding outputs
V](n) = AX](rt) + B

(2.2.40)
y2(n) = A x 2(n) + B
A linear combination of X\(n) and x 2{n) produces the output
V i( ) =

T[u^X ]

(/i) + a 2x 2(n>]

= A[a,x,(/i) + a 2x 2(n)} + B

(2.2.41)

= A a \ X \ ( n ) -I- a2A x 2(n) + B

On the other hand, if the system were linear, its output to the linear com bina
tion of Ji(n) and x 2 (n) would be a linear combination of vj i n) and y2(n). that is.
ai yi (n ) + a 2y 2(n) = a ] A x ] { n ) + a \ B

a 2A x 2{ n ) + a 2 B

(2.2.42)

Clearly. (2.2.41) and (2.2.42) are different and hence the system fails to satisfy
the linearity test.
The reason that this system fails to satisfy the linearity test is not that the
system is nonlinear (in fact, the system is described by a linear equation) but
the presence of the constant B. Consequently, the output depends on both the
input excitation and on the param eter B ^ 0. Hence, for B ^ 0. the system is
not relaxed. If we set B = 0, the system is now relaxed and the linearity test is
satisfied.
(e) Note that the system described by the input-output equation
y(n) = e,1',)

(2.2.43)

is relaxed. If x(n) = 0, we find that y(n) = 1. This is an indication that the

system is nonlinear. This, in fact, is the conclusion reached when the linearity
test, is applied.

Causal versus noncausal systems.

d isc re te -tim e system s.

W e b eg in w ith th e d efin itio n o f causal

Sec. 2.2

69

Discrete-Time Systems

D e fin itio n ,
a system is said to b e causal if th e o u tp u t o f th e system at any
tim e n [i.e., v(n)] d e p e n d s oniy on p re s e n t an d p ast in p u ts [i.e., x { n ), x(tt - 1),
x(rt 2 ) , . . . ] , b u t d o e s n o t d e p e n d on fu tu re in p u ts [i.e., x( n + 1), x ( n + 2 ) , . . . ] . In
m a th e m a tic a l te rm s, th e o u tp u t o f a cau sal sy stem satisfies an e q u a tio n o f th e form
v(n) = F[x{n), x ( n - 1), x ( n - 2 ) , . . . ]

(2.2.44)

w h ere /'[] is so m e a rb itra ry function.

If a system d o e s n o t satisfy this d efin itio n , it is called noncausal . S uch a
sy stem has an o u tp u t th a t d e p e n d s n o t oniy on p re s e n t a n d p ast in p u ts b u t also
o n fu tu re in p u ts.
It is a p p a re n t th a t in real-tim e signal p ro cessin g ap p licatio n s w e c a n n o t o b
serv e fu tu re v alu es o f th e signal, and h e n c e a n o n cau sal system is physically u n re a l
izab le (i.e., it c a n n o t b e im p le m e n te d ). O n th e o th e r h an d , if th e signal is re c o rd e d
so th a t th e p ro cessin g is d o n e off-line (n o n re a l tim e ), it is p o ssible to im p lem en t
a n o n cau sal sy stem , since all v alu es o f th e signal are av ailab le a t th e tim e o f p r o
cessing. T h is is o fte n th e case in th e p ro cessin g o f g eophysical signals an d im ages.
Example 2.2.6
D eterm ine if the systems described by the following input-output equations are causal
or noncausal.
(a) y(n) = x(n) - x(n - 1)
(d) y(n') = x(n) + 3jr(n + 4)

(b) y(n) =
(e) y(n) = x( n2)

x(k)

(c) y(n) = ax(n)

(t) y(n) = x(2n)

(g) }'(n) = x ( -n )
Solution The systems described in parts (a), (b), and (c) are clearly causal, since the
output depends only on the present and past inputs. On the other hand, the systems
in parts (d). (e), and (f) are clearly noncausal, since the output depends on future
values of the input. The system in (g) is also noncausal, as we note by selecting, for
example, n = - 1 , which yields v(1) = * 0 ) Thus the output at n = - 1 depends on
the input at n = 1, which is two units of time into the future.

Stable versus unstable systems. S tab ility is an im p o rta n t p ro p e rty th a t

m u st b e c o n s id e re d in an y p ractical ap p lic a tio n o f a system . U n s ta b le system s
u su ally ex h ib it e rra tic an d ex tre m e b e h a v io r an d cause overflow in an y p ractical
im p le m e n ta tio n . H e re , w e define m a th e m a tic a lly w h at w e m e a n by a sta b le system ,
a n d la te r, in S ectio n 2.3.6, w e ex p lo re th e im plicatio n s o f this definition fo r lin ear,
tim e -in v a ria n t system s.

Definition. A n a rb itra ry re la x e d system is said to be b o u n d e d in p u t-b o u n d e d

o u tp u t (B IB O ) sta b le if a n d only if ev ery b o u n d e d in p u t p ro d u ces a b o u n d e d
o u tp u t.
T h e c o n d itio n s th a t th e in p u t se q u e n c e x{n) a n d th e o u tp u t se q u e n c e y ( n) are
b o u n d e d is tra n s la te d m a th e m a tic a lly to m e a n th a t th e re exist som e finite n u m b ers,

Discrete-Time Signals and Systems

70

Chap. 2

say M x an d M v. such th at
j.v(ri)! < M K < oc

< M x < dc

(2.2.45)

fo r all n. If. fo r so m e b o u n d e d in p u t se q u e n c e ,v(), the o u tp u t is u n b o u n d e d

(in fin ite), th e sy stem is classified as u n sta b le .
Exam ple 2.2.7
Consider the nonlinear system described by the input-output equation
V(/I ) =

1)

,V(/i )

As an input sequence we select the hounded signal

xin ) = C&(n )
where C is a constant. We also assume that y(1) = 0. Then the output sequence is
y(0) = C,

y (l) = C \

y(2) = Cd............

y(n) = C : "

Clearly, the output is unbounded when ] < ICl < oc. Therefore, the system is BIBO
unstable, since a bounded input sequence has resulted in an unbounded output.

2.2.4 Interconnection ot Discrete-Time Systems

D isc rete-tim e sy stem s can be in te rc o n n e c te d to form larg er sy stem s. T h e re are
tw o b asic ways in w hich system s can be in te rc o n n e c te d : in c a scad e (series) o r in
p arallel. T h ese in te rc o n n e c tio n s are illu strated in Fig. 2.21. N o te th at th e tw o
in te rc o n n e c te d sy stem s are d ifferen t.
In th e cascad e in te rc o n n e c tio n the o u tp u t of th e first system is
yiO?) = 7j[;r(/j)]

xin)

(2.2.46)

y(n)

:
r

T\

T,

7,
(a)

v | (n )

(b)

Figure 2.21 Cascade (a) and parallel

(b) interconnections of systems.

Sec. 2.2

Discrete-Time Systems

71

an d th e o u tp u t o f th e second system is
v(n) = T2[\\(n)]
(2.2.47)
=

r 2{7 i[* (n )]}

W e o b se rv e th a t sy stem s 7"j a n d T2 can be co m b in ed o r c o n s o lid a te d in to a single

o v e ra ll sy stem
% = T271

(2.2.48)

C o n s e q u e n tly , w e can ex p ress th e o u tp u t o f th e co m b in ed sy stem as

y( n) = Tc[x(n)]
In g e n e ra l, th e o rd e r in w hich th e o p e ra tio n s T\ a n d T2 a re p e rfo rm e d is
im p o rta n t. T h a t is,
T27I # T,T2
fo r a rb itra ry system s. H o w ev er, if th e system s 7j a n d T2 a re iin e a r a n d tim e
in v a ria n t, th e n (a) % is tim e in v arian t an d (b ) T2T\ = T \ T2, th a t is, th e o r d e r in
w hich th e sy stem s p ro cess th e signal is n o t im p o rta n t. 7^71 a n d T \ T 2 yield id en tical
o u tp u t se q u e n c e s.
T h e p ro o f o f (a) follow s. T h e p ro o f o f (b ) is given in S e c tio n 2.3.4. T o p ro v e
tim e in v a ria n c e , su p p o se th a t Tj and T2 a re tim e in v arian t; th e n
x( n k )

vi(/i - k)

an d
Vi(n - k) '+ y( n - k)
Thus
x{n k) Tf

y( n k )

an d th e re fo re , Tc is tim e in v arian t.
In th e p a ra lle l in te rc o n n e c tio n , th e o u tp u t of th e sy stem T\ is ^ ( n ) an d the
o u tp u t o f th e sy stem T2 is y2(n). H e n c e th e o u tp u t o f th e p a ra lle l in te rc o n n e c tio n is
v3(n) = .V] ( n) + >>2(n)
= Ti[x{n)\ + T2[x(n)\
= (T\ + T2)[x(n)}
= Tp[x(n)\
w h e re Tp = T\ + T2.
In g e n e ra l, w e can u se p a rallel an d cascade in te rc o n n e c tio n o f sy stem s to
c o n s tru c t la rg e r, m o re com plex system s. C o n v e rsely , w e can ta k e a la rg e r system
a n d b re a k it d o w n in to sm a ller su b sy stem s fo r p u rp o se s o f an aly sis a n d im p le
m e n ta tio n . W e sh all u se th e s e n o tio n s la te r, in th e design a n d im p le m e n ta tio n of
d ig ital filters.

Discrete-Time Signals and Systems

72

Chap. 2

2.3 ANALYSIS OF DISCRETE-TIME LINEAR TIME-INVARIANT

SYSTEMS
In S ectio n 2.2 we classified system s in a c c o rd a n c e w ith a n u m b e r of c h aracteristic
p ro p e rtie s o r categ o ries, nam ely: lin e a rity , cau sality , stab ility , a n d tim e in variance.
H av in g d o n e so. we now tu rn o u r a tte n tio n to th e analysis o f th e im p o rta n t class
o f lin ear, tim e -in v a ria n t (L T I) system s. In p a rtic u la r, we shall d e m o n s tra te th a t
such system s are c h a ra c te riz e d in th e tim e d o m ain sim ply bv th e ir resp o n se to a
u n it sam p le se q u en ce. W e shall also d e m o n s tra te th a t any a rb itra ry in p u t signal
can b e d eco m p o sed an d r e p re s e n te d as a w eig h ted sum of u n it sa m p le seq u en ces.
A s a c o n s eq u en ce o f th e lin e a rity a n d tim e-in v arian ce p ro p e rtie s of the system ,
th e resp o n se o f th e system to any a rb itra ry in p u t signal can be e x p ressed in term s
of th e u n it sam p le re sp o n se of th e system . T h e g en eral form o f the ex p ressio n
th a t re la te s th e u n it sam ple re sp o n se o f the system an d th e a rb itra ry in p u t signal
to th e o u tp u t signal, called th e c o n v o lu tio n sum o r th e c o n v o lu tio n fo rm u la, is also
d eriv ed . T h u s we are ab le to d e te rm in e the o u tp u t o f any lin e a r, tim e-in v arian t
system to any a rb itra ry in p u t signal.

2.3.1 Techniques for the Analysis of Linear Systems

T h e re are tw o basic m e th o d s for an aly zin g th e b e h a v io r o r re sp o n s e of a lin ear
system to a given in p u t signal. O n e m e th o d is b a se d on th e d ire c t so lu tio n o f the
in p u t-o u tp u t e q u a tio n for th e system , w hich, in g e n e ra l, has th e form
v(ji) - F [v ( n - 1), v (?7 2 ) ........ y(n - N) , x ( n ) . x ( n 1).......... x ( n - M)]
w h ere F[-] d e n o te s so m e fu n ctio n o f th e q u a n titie s in b ra c k e ts. Specifically, fo r
an L T I system , w e shall see la te r th a t th e g e n e ra l form o f th e in p u t-o u tp u t re la
tio n sh ip is
N

M
(2.3.1)

w h ere
an d {b*,.} are c o n sta n t p a ra m e te rs th a t specify th e sy stem an d a re in
d e p e n d e n t o f x( n) a n d y( n) . T h e in p u t-o u tp u t re la tio n sh ip in (2.3.1) is called
a d ifferen ce e q u a tio n a n d re p re se n ts o n e w ay to c h a ra c te riz e th e b e h a v io r of a
d isc re te -tim e L T I system . T h e so lu tio n o f (2.3.1) is th e su b je ct o f S ection 2.4.
T h e seco n d m e th o d fo r analyzing th e b e h a v io r o f a lin e a r sy stem to a given
in p u t signal is first to d e c o m p o se o r reso lv e th e in p u t signal in to a sum o f e le
m e n ta ry signals. T h e e le m e n ta ry signals a re se le c te d so th a t th e re sp o n se o f the
system to each signal c o m p o n e n t is easily d e te rm in e d . T h e n , u sin g th e lin earity
p ro p e rty o f th e sy stem , th e re sp o n se s o f th e sy stem to th e e le m e n ta ry signals are
ad d e d to o b ta in th e to ta l re s p o n s e of th e sy stem to th e given in p u t signal. T h is
seco n d m e th o d is th e o n e d e sc rib e d in th is se ctio n .

Sec. 2.3

Analysis o f Discrete-Time Linear Tim e-Invariant Systems

73

T o e la b o ra te , su p p o se th a t th e in p u t signal x (n ) is reso lv ed into a w eig h ted

sum o f elem en tary ' signal c o m p o n e n ts {*/. ()} so th a t

w h ere th e {ct} a re th e set of a m p litu d e s (w eighting coefficients) in the d e c o m

p o sitio n o f th e signal x( n) . N ow su p p o se th a t th e re sp o n se o f th e system to the
e le m e n ta ry signal c o m p o n e n t x k{n) is y k(n). T hus.
y*(n) = T [j.t (n)]

(2.3.3)

assu m in g th a t th e sy stem is re la x e d an d th a t th e re sp o n se to ckx k(n) is cky k(n). as

a c o n s e q u e n c e o f th e scaling p ro p e rty o f th e lin e a r system .
F in ally , th e to ta l re sp o n se to th e in p u t x( n ) is

(2.3.4)

In (2.3.4) we u se d th e ad d itiv ity p ro p e rty o f th e lin e a r system .

A lth o u g h to a larg e e x te n t, th e ch o ice o f th e e le m e n ta ry signals a p p e a rs to
b e a rb itra ry , o u r se lectio n is heavily d e p e n d e n t on th e class of input signals that
we w ish to co n sid er. If w e place n o re stric tio n on th e c h aracteristics of th e input
signals, its re so lu tio n in to a w eig h ted sum of u n it sam ple (im p u lse) se q u en ces
p ro v e s to b e m a th e m a tic a lly c o n v e n ie n t an d c o m p letely g en eral. O n th e o th e r
h a n d , if w e re stric t o u r a tte n tio n to a subclass o f in p u t signals, th e re m ay be
a n o th e r set o f e le m e n ta ry signals th a t is m o re c o n v e n ie n t m a th e m a tic a lly in the
d e te rm in a tio n o f th e o u tp u t. F o r ex am p le, if th e in p u t signal x( n) is p erio d ic
w ith p e rio d N , w e have a lre a d y o b se rv e d in S ectio n 1.3.5 th a t a m a th em atically
c o n v e n ie n t set o f elem en tary 7 signals is th e set o f e x p o n en tials
x k (n) = eJluin

k = 0. l , . . . , i V - l

(2.3.5)

w h e re th e fre q u e n c ie s {cok } a re h arm o n ic a lly re la te d , th a t is.

k = 0. 1.........N - 1

(2.3.6)

T h e fre q u e n c y 2 n / N is called th e fu n d a m e n ta l fre q u e n c y , an d all h ig h er-freq u en cy

c o m p o n e n ts a re m u ltip le s o f th e fu n d a m e n ta l fre q u e n c y c o m p o n e n t. T h is subclass
o f in p u t sig n als is c o n s id e re d in m o re d e ta il later.
F o r th e re so lu tio n o f th e in p u t signal in to a w eig h ted su m o f u n it sam ple
se q u en ces, w e m u st first d e te rm in e th e re sp o n se o f th e system to a u n it sa m
p le se q u e n c e a n d th e n use th e scaling a n d m u ltip lic ativ e p ro p e rtie s of th e lin ear

74

Discrete-Time Signals and Systems

Chap. 2

sy stem to d e te rm in e th e fo rm u la fo r th e o u tp u t given any a rb itra ry input, T his

d e v e lo p m e n t is d escrib ed in d etail as follow s.

2.3.2 Resolution of a Discrete-Time Signal into Impulses

S u p p o se w e h av e an a rb itra ry signal x ( n ) th a t we w ish to reso lv e in to a sum of unit
sa m p le seq u en ces. T o utilize th e n o ta tio n e sta b lish e d in th e p re c e d in g se ctio n , we
select th e e le m e n ta ry signals x k (n) to be
(2.3.7)

x k(n) = 8{n - k)

w h ere k re p re se n ts th e d elay o f th e u n it sam p le se q u e n c e . T o h a n d le an a rb itra ry

signal x ( n ) th a t m ay h ave n o n z e ro v alu es o v er an infinite d u ra tio n , th e set of unit
im p u lses m u st also b e infinite, to en co m p ass th e infinite n u m b e r of delays.
N o w su p p o se th a t we m u ltip ly th e tw o se q u e n c e s x( n) a n d <5(n - k). Since
8{n k) is z e ro e v ery w h ere ex cep t a t n = k , w h e re its v alu e is u nity, the result
o f th is m u ltip lic atio n is a n o th e r se q u e n c e th a t is z e ro e v ery w h ere e x c e p t at n k ,
w h ere its v alu e is x ( k) , as illu stra te d in Fig. 2,22. T h u s
x( n) 8( n k) = x ( k) 8( n k)

(2.3.8)

Jt(n)
T TT 111 i l l
i
[ -2-10113
(a)

i , 1 I , Tt I
i
i 1
JC(Jc)

6(/i-Jt)

(b)
*(*) 6(n k )

k
0

Figure 2.22

M ultiplication of a signal x i n ) with a shifted unit sam ple sequence.

Sec. 2.3

75

Analysis of Discrete-Time Linear Time-Invariant Systems

is a se q u e n c e th a t is z e ro e v e ry w h e re ex cep t at n = k , w h e re its v a lu e is x( k ) . If we
w ere to re p e a t th e m u ltip lic a tio n of x ( n ) w ith <5(a? m ), w h ere m is a n o th e r d elay
(im =6 k), th e re su lt will b e a se q u en ce th a t is z e ro e v e ry w h e re e x cep t at n = m,
w h ere its v alu e is x ( m ) . H e n c e
x( n ) 5 ( n m) = x ( m) 8( n m)

(2.3.9)

In o th e r w o rd s, each m u ltip lic a tio n o f th e signal x( n ) by a u n it im p u lse at som e

d elay k, [i.e., <5(n it)], in essen ce picks o u t th e single v alu e x ( k ) o f th e signal jc(n)
at th e d e la y w h e re th e u n it im pulse is n o n z e ro . C o n s e q u e n tly , if w e re p e a t this
m u ltip lic a tio n o v e r all p o ssib le delays, - o o < k < oo, a n d su m all th e p ro d u c t
se q u e n c e s, th e re su lt will be a se q u e n c e e q u a l to th e se q u e n c e x ( n ) , th a t is,
PCx( n) = ^
x ( k) 8( n k)
k=oc
W e e m p h a s iz e th a t th e rig h t-h an d side of (2.3.10) is th e
infinite n u m b e r o f u n it sa m p le se q u en ces w h ere th e u n it sa m p le
h as an a m p litu d e value o f x( k ) . T h u s th e rig h t-h a n d sid e o f
re so lu tio n o f o r d e c o m p o s itio n o f any a rb itra ry signal jc(n) in to a
sum o f sh ifted u n it sam p le seq u en ces.

(2.3.10)
su m m a tio n of an
se q u e n c e 6(n - k)
(2.3.10) gives th e
w e ig h te d (scaled)

Exam ple 2.3.1

Consider the special case of a finite-duration sequence given as
jc()

= (2, 4, 0,3)
T

Resolve the sequence x(n) into a sum of weighted impulse sequences.

Solution Since the sequence x(n) is nonzero for the time instants n = 1, 0. 2, we
need three impulses at delays k = 1. 0, 2, Following (2.3.10) we find that
x<n) = 2(5(n + 1) + 4<5(n) + 3<5(n 2)

2.3.3 Response of LTI Systems to Arbitrary Inputs: The

Convolution Sum
H av in g re so lv e d an a rb itra ry in p u t signal x ( n ) in to a w eig h te d su m o f im pulses,
w e a re no w re a d y to d e te rm in e th e re sp o n se of an y re la x e d lin e a r sy stem to any
in p u t signal. F irs t, w e d e n o te th e re sp o n se v(n, k) o f th e sy stem to th e in p u t unit
sa m p le se q u e n c e a t n = it by th e special sym bol h(n, k), oo < k < oo. T h a t is,
y( n, k) = h(n. k ) = T[ S( n )]

(2.3.11)

In (2.3.11) w e n o te th a t n is th e tim e index a n d k is a p a r a m e te r sh ow ing th e

lo c a tio n o f th e in p u t im p u lse. If th e im pulse at th e in p u t is sc aled by an a m o u n t
ct = jc(it), th e re sp o n s e of th e system is th e c o rre sp o n d in g ly sc aled o u tp u t, th a t is,

Ckh(n, k) = x(k)h(n, k)

(2.3.12)

Discrete-Time Signals and Systems

76

Chap. 2

F in ally , if th e in p u t is th e a rb itra ry signal x(/t) th a t is e x p re ss e d as a sum of

w eig h ted im p u lses, th a t is.
(2.3.13)
th e n th e resp o n se o f the system to x(/i) is the c o rre sp o n d in g sum o f w eig h ted
o u tp u ts, th a t is,
y(rt) = 7"[.r(/;)] = T

x( k ) S( n k)

x ( k ) T [ 5 (m - k)]

(2.3.14)

ii= oc
C learly , (2.3.14) follow s from th e su p e rp o sitio n p ro p e rty of lin e a r system s, and is
k n o w n as th e superposit i on s u mma t i o n .
W e n o te th a t (2.3.14) is an e x p ressio n for th e resp o n se o f a lin e a r system to
any a rb itra ry in p u t se q u e n c e x( n) . T his ex p ressio n is a fu n ctio n of b o th .v() and
th e resp o n ses h(n. k) of the system to th e unit im pulses Sin k) fo r oc < k < oc.
In d eriv in g (2.3.14) w e used th e lin earity p ro p e rty o f th e system but not its tim ein v arian ce p ro p e rty . T h u s th e ex p ressio n in (2.3.14) ap p lies to any relax ed lin ear
(tim e -v a ria n t) system .
If. in a d d itio n , th e system is tim e in v a ria n t, th e fo rm u la in (2.3.14) sim plifies
c o n sid erab ly . In fact, if the resp o n se o f th e L T I system to th e u n it sa m p le seq u en ce
<5(rc) is d e n o te d as h(n). th a t is.
h(n) = T [ b( n ) \

(2.3.15)

th en by th e tim e-in v arian ce p ro p e rty , th e resp o n se o f the system to the delay ed

u n it sa m p le se q u e n c e <5(n - k) is
h(n k) = T [ S( n A')]

(2,3.16)

C o n seq u en tly , th e fo rm u la in (2.3.14) re d u c e s to

(2.3.17)
k=-oc
N ow we o b serv e th a t th e relax ed L T I system is co m p letely c h a ra c te riz e d by a
single fu n ctio n h(n), n am ely , its resp o n se to th e u n it sam p le se q u e n c e
In
c o n tra st, th e g en eral c h a ra c te riz a tio n of th e o u tp u t o f a tim e -v a ria n t, lin e a r sys
tem re q u ire s an in fin ite n u m b e r o f u n it sa m p le re sp o n s e fu n ctio n s, h{n, k), o n e for
ea c h p o ssib le d elay .
T h e fo rm u la in (2.3.17) th a t gives th e re sp o n se y( n) of th e L T I system as a
fu n c tio n o f th e in p u t signal x ( n ) a n d th e u n it sa m p le (im p u lse) re sp o n se h(n) is
called a convol ut i on s um. W e say th a t th e in p u t jt(n ) is c o n v o lv ed w ith th e im p u lse

Sec. 2.3

Analysis of Discrete-Time Linear Tim e-Invariant Systems

77

response h(n) to yield the output y in ). W e shall now explain the procedure for
com puting the resp onse y (n ). both m athem atically and graphically, given the input
x ( n ) and the im pulse response h(n) o f the system .
Suppose that we wish to com pute the output of the system at som e time
instant, say n = n 0. A ccordin g to (2.3.17), the resp onse at n = no is given as
OC

y (n 0) =

^ 2 x ( k ) h ( n 0 - k)

(2.3.18)

Jc=-oc

O ur first observation is that the index in the sum m ation is k , and h en ce both the
input signal x ( k ) and the im pulse resp onse h(no - k) are fun ction s o f k. Second,
w e ob serve that the sequ en ces x ( k ) and h(nQ k) are m ultiplied togeth er to form
a product seq u en ce. T h e output >(o) is sim ply the sum over all valu es o f the
product sequ en ce. T he seq u en ce h ( n 0 k) is ob tain ed from h ( k ) by, first, folding
h (k) about k = 0 (the tim e origin), w hich results in the seq u en ce h ( k). The
folded seq u en ce is then shifted by no to yield h(no k). T o sum m arize, the process
o f com p utin g the con volu tion b etw een x ( k ) and h (k) in volves the follow in g four
steps.
1. Folding. F old h(k) about k = 0 to obtain h ( ~ k ) .
2. Shifting, Shift h ( k) by n 0 to the right (left) if n o is p ositive (n egative), to
obtain h(no ).
3. Multiplication. M ultiply
v Jk) = x ( k ) h ( n 0 - k).

by h (no k) to obtain the product sequ en ce

4. S u m m a t i o n . Sum all the values o f the product seq u en ce vnt)( k ) to obtain the
value o f the output at tim e n = n 0.
W e note that this p rocedure results in the resp onse o f the system at a sin
gle tim e instant, say n = n 0. In gen eral, we are interested in evaluating the
response o f the system over all tim e instants - o o < n < oo. C onsequently,
steps 2 through 4 in the sum m ary m ust be rep eated , for all p ossible tim e shifts
oo < n < oo.
In order to gain a b etter understanding o f the procedure for evaluating the
convolution sum , w e shall dem onstrate the p rocess graphically. T he graphs will
aid us in explaining the four steps in volved in the com p utation o f the convolution
sum.

Example 2.3.2
The impulse response of a linear time-invariant system is
/!() = [1 .2 ,1 ,-1 }

(2.3.19)

Determine the response of the system to the input signal

x(n) = {1,2.3,1}
t

(2.3.20)

Discrete-Time Signais and Systems

Chap. 2

Solution We shall com pute the convolution according to the formula (2.3.17). but
we shall use graphs of the sequences to aid us in the com putation. In Fig. 2.23a we
illustrate the input signal sequence x(k) and the impulse response h{k) of the system,
using k as the time index in order to be consistent with (2.3.17),
The first step in the com putation of the convolution sum is to fold h(k). The
folded sequence h(~k) is illustrated in Fig. 2.23b. Now we can compute the output
at n = 0. according to (2.3.17), which is

v(0) =

(2.3.21)
*=-cx

Since the shift n = 0, we use h( k) directly without shifting it. The product sequence
= x(k)h(-k)

(2.3.22)

h(k)

x(k
3

4i

T
1

* -i

-1 0 ! j

10

h(-k)

1 2 3

L'n(k 1

2
.

. -2

T t . . .
-1 0 1 2

.
(b)

Shift

vAk)

Product

,,.
L |{t)

Product
sequence

h(\-k)

111

To

Br

T
7

(c)

T
.

~3 T !
j-2 -1 0 1

1
k

0 12
(d)

Figure 2.23

G raphical com putation of convolution.

Sec. 2.3

Analysis of Discrete-Time Linear Time-Invariant Systems

79

is also shown in Fig. 2.23b. Finally, the sum of all the terms in the product sequence
yields
() =

vott) = 4

We continue the com putation by evaluating the response of the system at n = 1.

According to (2.3.17),
(2.3.23)
The sequence h(\ k) is simply the folded sequence h( k) shifted to the right by one
unit in time. This sequence is illustrated in Fig. 2.23c. The product sequence
V] (k} = x(k)h{l k)

(2.3.24)

is also illustrated in Fig. 2.23c. Finally, the sum of all the values in the product
sequence yields
y(l) =

ui(*) = 8

In a similar m anner, we obtain y(2) by shifting h ( - k ) two units to the right,

forming the product sequence ih(A) = x(k)h(2 k) and then summing all the terms
in the product sequence obtaining y(2) = 8. By shifting h(k) farther to the right,
multiplying the corresponding sequence, and summing over all the values of the re
sulting product sequences, we obtain v(3) = 3. v(4) = - 2 , y(5) = - 1 . For u > 5, we
find that v(n) = 0 because the product sequences contain all zeros. Thus we have
obtained the response y(n) for n > 0.
Next we wish to evaluate v(n) for n < 0. We begin with n =
Then
(2.3.25)
Now the sequence h (1 k) is simply the folded sequence h ( k ) shifted one time
unit to the left. The resulting sequence is illustrated in Fig. 2.23d. The corresponding
product sequence is also shown in Fig. 2.23d. Finally, summing over the values of the
product sequence, we obtain
V (-1) = 1
From observation of the graphs of Fig. 2.23, it is clear that any further shifts of
h (1 - k) to the left always results in an all-zero product sequence, and hence
y(n) = 0

for n 5 2

Now we have the entire response of the system for oc < n < oc. which we
summarize below as
y(n) =

0 ,0,1, 4. 8, 8. 3, - 2 , - 1 , 0 . 0 . . . .)
t

(2.3.26)

80

Discrete-Time Signals and Systems

Chap. 2

In E x am p le 2.3.2 w e illu stra te d th e c o m p u ta tio n o f th e co n v o lu tio n sum .

using g rap h s o f th e se q u en ces to aid us in visualizing th e ste p s in volved in th e
c o m p u ta tio n p ro c e d u re .
B e fo re w o rk in g o u t a n o th e r ex a m p le , w e w ish to show th a t th e co n v o lu
tio n o p e ra tio n is c o m m u ta tiv e in th e se n se th a t it is irre le v a n t w hich of th e tw o
se q u e n c e s is fo ld ed a n d shifted. In d e e d , if w e b eg in w ith (2.3.17) a n d m ak e a
ch an g e in th e v ariab le o f th e su m m a tio n , fro m k to m , by defin in g a new index
m n k, th e n k = n m an d (2.3.17) b e co m es
CC

y(n) =

^2

x( n m ) h { m )

(2.3.27)

Since m is a d u m m y in dex, w e m ay sim ply re p la c e m by k so th a t

y{n) =

x(n-k)h(k)

(2.3.28)

T h e ex p ressio n in (2.3.28) involves leav in g th e im p u lse re sp o n s e h( k) u n a lte re d ,

w hile th e in p u t se q u en ce is fo ld ed a n d sh ifted . A lth o u g h th e o u tp u t v(n) in (2.3.28)
is id en tical to (2.3.17), th e p ro d u c t se q u e n c e s in th e tw o fo rm s o f th e co n v o lu tio n
fo rm u la are not id en tical. In fact, if w e define th e tw o p ro d u c t se q u e n c e s as
v(k) = x ( k ) h ( n k)
w n(k) = x( n k) h(k)
it can b e easily show n th at
un() = w n (n k)
an d th e re fo re ,
CC

v(n) =

CC

~ k)

koc

oc

since b o th se q u en ces co n tain th e sam e sa m p le v alu es in a d iffe re n t a rra n g e m e n t.

Example 2.3.3
Determ ine the output y(n) of a relaxed linear tim e-invariant system with impulse
response
h(ri) = a"u(n), \a\ < 1
when the input is a unit step sequence, that is,
x (n) = u(n)
Solution In this case both /j(n) and jc(n) are infinite-duration sequences. We use
the form of the convolution formula given by (2.3.28) in which x (k) is folded. The

Sec. 2.3

81

Analysts of Discrete-Time Linear Tim e-Invariant Systems

h{k\

x(k)

TT i f
1

(b)
I'oCA')

x(- k)

l
* 1

-3

-2 -1

-1

0
A(1

(c)

-k)

>

i'i(i-)
a

1
- 1 0

v( 2k)

1
1':(!>

II

- 2 - 1 0 1 2 3 4 5
Figure 2.24

Graphical computation of convolution in Example 2.3.3.

sequences h(k), x(k). and x{k) are shown in Fig. 2.24. The product sequences vo(k).
v\(k), and v2(k) corresponding to x ( k)h(k), x (l k)h(k), and x(2 - k)h(k) are illus
trated in Fig. 2.24c, d. and e. respectively. Thus we obtain the outputs
v(0) = 1
y(l) = 1 + a
y( 2) = 1 + a + a 1

Discrete-Time Signals and Systems

82

Chap. 2

Clearly, for n > 0, the output is

y(n) = 1 + a 4- a2 + + a
1 _ an+1

(2.3,29)

=
1-a
On the other hand, for n < 0, the product sequences consist of all zeros. Hence
v(n) = 0

n < 0

A graph of the output y(n) is illustrated in Fig. 2.24f for the case 0 < a < 1.
Note the exponential rise in the output as a function of n. Since |a| < 1, the final
value of the output as n approaches infinity is
v(oo) = lim v(n) = ------n-*oc '
1 a

(2.3.30)

T o su m m arize, th e co n v o lu tio n fo rm u la p ro v id e s us w ith a m ean s fo r co m

p u tin g th e re sp o n s e o f a re lax ed , lin ear tim e -in v a ria n t system to an y a rb itra ry in p u t
signal x( n). It ta k e s o n e o f tw o e q u iv a le n t form s, e ith e r (2.3.17) o r (2,3.28), w h ere
jt(n ) is th e in p u t sig n al to th e system , h (n ) is th e im p u lse re s p o n s e of th e system ,
an d y (n ) is th e out p u t o f th e system in re sp o n s e to th e in p u t signal x (n ). T he
e v a lu a tio n o f th e co n v o lu tio n fo rm u la involves fo u r o p e ra tio n s , n am ely: f ol di ng
e ith e r th e im p u lse re sp o n s e as specified by (2.3.17) o r th e in p u t se q u e n c e as sp ec
ified by (2.3.28) to yield e ith e r h ( k) o r x { k). resp ec tiv ely , shifting th e folded
se q u e n c e by n u n its in tim e to yield e ith e r h{n k ) o r x { n k ). mul t i pl yi ng the
tw o se q u e n c e s to yield th e p ro d u c t se q u en ce, e ith e r x { k) h{ n k) o r x ( n - k ) h ( k ) ,
a n d finally s u m m i n g all th e v alu es in th e p ro d u c t se q u e n c e to y ield th e o u tp u t v (n )
o f th e sy stem a t tim e n. T h e folding o p e ra tio n is d o n e only o n ce. H o w ev er, th e
o th e r th re e o p e ra tio n s a re re p e a te d fo r all p o ssib le shifts oc < n < oo in o rd e r
to o b ta in y (n ) fo r oo < n < oc.

2.3.4 Properties of Convolution and the Interconnection

of LTI Systems
In th is se ctio n w e in v estig ate so m e im p o rta n t p ro p e rtie s o f co n v o lu tio n an d in
te r p re t th e se p r o p e rtie s in te rm s o f in te rc o n n e c tin g lin ear tim e -in v a ria n t system s.
W e sh o u ld stress th a t th ese p ro p e rtie s h o ld fo r e v e ry in p u t signal.
It is c o n v e n ie n t to sim plify th e n o ta tio n by using an a s te ris k to d e n o te the
c o n v o lu tio n o p e ra tio n . T h u s
OC

y( n) = x{n) * h(n) = ^

x ( k ) h ( n k)

(2.3.31)

Jt = - O C

In th is n o ta tio n th e se q u e n c e follow ing th e aste risk [i.e., th e im p u lse re sp o n se /i()]

is fo ld e d an d sh ifted . T h e in p u t to th e sy stem is ;c(n). O n th e o th e r h a n d , we also
sh o w ed th a t
OC

>>(n) = h{n) * x( n) = ^

k=-oc

h ( k ) x ( n - k)

(2.3.32)

Sec. 2.3

83

Analysis of Discrete-Time Linear Time-Invariant Systems

hin)
h(n)
Figure 2.25

<

>

v(n )

xin)

Interpretation of the commutative property of convolution.

In th is fo rm o f th e co n v o lu tio n fo rm u la, it is the in p u t signal th a t is fo ld ed . A lte r

n ativ ely . we m ay in te r p re t this fo rm o f the c o n v o lu tio n fo rm u la as re su ltin g from
an in te rc h a n g e o f th e ro les o f j:(n) an d h(n). In o th e r w ords, w e m ay re g a rd x( n)
as th e im p u lse re sp o n se o f th e system an d h( n) as the e x c ita tio n o r in p u t signal.
F ig u re 2.25 illu stra te s th is in te rp re ta tio n .
W e can view co n v o lu tio n m o re ab stractly as a m a th e m a tic a l o p e ra tio n b e
tw een tw o signal se q u e n c e s, say x( n ) a n d h(n), th a t satisfies a n u m b e r o f p ro p e rtie s .
T h e p ro p e rty e m b o d ie d in (2.3.31) an d (2.3.32) is called th e c o m m u ta tiv e law'.

Commutative law
jf(n) * h{n) = h(n) * x( n)

(2.3.33)

V iew ed m a th e m a tic a lly , the c o n v o lu tio n o p e ra tio n also satisfies th e asso cia
tive law , w hich can be sta te d as follow s.

Associative law
[-v(/i) * /?[()] * h 2(/i) x( n) * [/*!() *

(2.3.34)

F ro m a physical p o in t o f view, we can in te rp re t x (n) as the in p u t signal to

a lin e a r tim e -in v a ria n t system w ith im pulse re sp o n se /j|(/i). T h e o u tp u t o f this
sy stem , d e n o te d as v i(n ), b eco m es the in p u t to a seco n d lin e a r tim e -in v a ria n t
sy stem w ith im p u lse re sp o n se hzin). T h en th e o u tp u t is
y( n) V] (n) * h 2(n)
= [j:(fi) * h\(n)] * h2(n)
w hich is p recisely th e left-h an d side of (2.3.34). T h u s th e le ft-h a n d side o f (2.3.34)
c o rre sp o n d s to h av in g tw o lin ear tim e -in v a ria n t system s in cascad e. N ow th e righth an d side o f (2.3.34) in d icates th a t th e in p u t x ( n ) is a p p lied to an e q u iv a le n t system
h av in g an im p u lse re sp o n se , say h(n), w hich is e q u a l to the co n v o lu tio n o f th e tw o
im p u lse resp o n se s. T h a t is,
h(n) = h](n) * h?(n)
an d
y(n) = x( n ) * h(n)
F u rth e rm o re , sin ce th e co n v o lu tio n o p e ra tio n satisfies th e c o m m u ta tiv e p ro p e rty ,
o n e can in te rc h a n g e th e o r d e r o f th e tw o system s w ith re sp o n s e s h \ ( n ) a n d hzi n)
w ith o u t a lte rin g th e o v erall in p u t-o u tp u t re la tio n sh ip . F ig u re 2.26 g rap h ically il
lu stra te s th e asso ciativ e p ro p e rty .

84

Discrete-Time Signals and Systems

xin)

jr(n)

y()

h(n) =

Chap. 2

y{/i)

h\(n) * h2(n)

(a)

x(n)

h,(n)

y</i)

v(n)

h2(n)

h t(n)

(b)

Figure Z 2 6

Implications of the associative (a) and the associative and commuta

tive (b) properties of convolution.

Example 2.3.4
D eterm ine the impulse response for the cascade of two iinear time-invariant systems
having impulse responses
h \ () = ( j W " )
and

Solution To determ ine the overall impulse response of the two systems in cascade,
we simply convolve h}{n) with h2(n). Hence

where h2(n) is folded and shifted. We define the product sequence

v(k) = h\(k)h2(n - k)
= ()*(*)"-*
which is nonzero for k > 0 and n - k > 0 or n > k > 0. On the o th er hand, for n < 0,
we have v(k) = 0 for all k, and hence
h{n) = 0, n < 0
For n > k > 0. the sum of the values of the product sequence v(k) over all k yields
hin) =
tscO

= Hr
= (i)"(2"+l - 1 )

Sec. 2.3

Analysis of Discrete-Time Linear Tim e-Invariant Systems

85

T h e g e n e ra liz a tio n o f the associative law to m o re th a n tw o system s in cascade

follow s easily fro m th e d iscu ssio n given ab o v e. T h u s if w e h av e L lin e a r tim ein v arian t sy stem s in cascade w ith im p u lse re sp o n se s h\ ( u) . h ^ i n ) ........ h L (n). th ere
is an e q u iv a le n t lin e a r tim e -in v a ria n t sy stem hav in g an im p u lse re sp o n se th a t is
e q u a l to th e (L l)-fo id co n v o lu tio n of th e im p u lse resp o n se s. T h a t is.
h( n) = h \ { n ) * hzi n) * * h L (n)

(2.3.35)

T h e c o m m u ta tiv e law im p lies th a t th e o rd e r in w hich the c o n v o lu tio n s a re p e r

fo rm e d is im m a te ria l. C o n v e rsely , an y lin e a r tim e -in v a ria n t system can be d e c o m
p o se d in to a cascad e in te rc o n n e c tio n o f subsystem s. A m e th o d for accom plishing
th e d e c o m p o s itio n will be d escrib ed later.
A th ird p ro p e rty th a t is satisfied by th e c o n v o lu tio n o p e ra tio n is the d istrib u
tive law, w hich m ay be sta te d as follow s.

Distributive law
xin) *

4-

= .xin) * h\ {n) 4- x i n ) * hzi n)

(2.3.36)

I n te rp r e te d physically, this law im plies th a t if we h ave tw o lin ear tim ein v a ria n t sy stem s w ith im p u lse re sp o n se s h \ i n ) a n d /?;() ex cited by the sam e
in p u t signal .r(/;), th e sum of the tw o resp o n ses is identical to the resp o n se of an
o v erall system w ith im pulse resp o n se
h i n ) = )i\ in) 4- //:(/;)
T h u s th e o v erall system is view ed as a p arallel co m b in a tio n of the tw o linear
tim e -in v a ria n t sy stem s as illu stra te d in Fig. 2.27,
T h e g e n e ra liz a tio n o f (2.3.36) to m o re th an tw o lin e a r tim e-in v arian t sys
tem s in p a rallel follow's easily by m a th e m a tic a l in d u ctio n . T h u s th e in te rc o n n e c
tio n of L lin ear tim e -in v a ria n t system s in p a ra lle l w ith im p u lse resp o n ses h\ i n) .
h z i n ) .........h L{n) a n d ex cited by the sa m e in p u t x i n ) is e q u iv a le n t to o n e overall
system w ith im p u lse re sp o n se
L

h(n) ^ hj i n)
(=i

(2.3.37)

C o n v e rsely , an y lin ear tim e -in v a ria n t system can be d e c o m p o se d into a p arallel
in te rc o n n e c tio n o f su b sy stem s.

Figure 2.27 Interpretation of the distributive property of convolution: two LTI

systems connected in parallel can be replaced by a single system with h(n) =

(n) + k2{n).

Discrete-Time Signals and Systems

86

Chap. 2

2.3.5 Causal Linear Time-Invariant Systems

In S ectio n 2.2.3 w e d efin ed a causal system as o n e w hose o u tp u t at tim e n d e p en d s
o n ly on p re s e n t an d p ast in p u ts b u t d o es n o t d e p e n d on f u tu re in p u ts. In o th e r
w o rd s, th e o u tp u t o f the. sy stem at so m e tim e in s ta n t n, say n = no, d e p e n d s only
on v alu es o f jc() fo r n < n 0In th e case o f a lin e a r tim e -in v a ria n t system , cau sality can b e tra n sla te d
to a c o n d itio n o n th e im p u lse resp o n se. T o d e te rm in e this re la tio n sh ip , le t us
co n sid e r a lin ear tim e -in v a ria n t system having an o u tp u t a t tim e n = no given by
th e c o n v o lu tio n fo rm u la
OC
v ( o ) =

^2

h ( k ) x (n o ~ k )

k - o c

S u p p o se th a t w e su b d iv id e th e sum in to tw o sets o f term s, o n e se t involving p re se n t

a n d p a st v alu es o f th e in p u t [i.e.. x{n) for n < n 0] a n d o n e se t involving fu tu re
valu es o f th e in p u t [i.e.,
n > no]. T h u s we o b ta in

OC

-I

y (n u) ^ h ( k ) x ( n o - k) +
i=0

h( k) x( nu - k)

^
k oc

= [/ ?(0)x (n ()) + h( \ ) x( r tu -

1) + h ( 2 ) x ( n 0 - 2 ) + ]

+ [/?( 1 )x((] + 1) + h ( - 2 ) x ( n o + 2) + ]

W e o b se rv e th a t th e term s in th e first sum involve jr(no), x( no 1).........w hich are

th e p re s e n t a n d p ast values of th e in p u t signal. O n th e o th e r h a n d , th e te rm s in
th e se co n d sum in volve th e in p u t signal c o m p o n e n ts ;c(no + l) , x { n o -f 2 ) .........N ow ,
if th e o u tp u t a t tim e n = n 0 is to d e p e n d only on th e p re s e n t a n d p a st in p u ts, th en ,
clearly , th e im p u lse re sp o n se o f th e system m u st satisfy th e c o n d itio n
ft(n) = 0

n < 0

(2.3.38)

Since h{n) is th e re sp o n se o f th e re lax ed lin e a r tim e -in v a ria n t sy stem to a u nit

im p u lse a p p lied a t n = 0, it follow s th a t h(n) = 0 fo r n < 0 is b o th a necessary
an d a sufficient c o n d itio n fo r causality. H e n c e an L T I syst em is causal i f a n d onl y
i f its i mpul se respons e is z e r o f o r negative values o f n.
Since fo r a cau sal sy stem , h(n) = 0 fo r n < 0, th e lim its on th e su m m a tio n of
th e co n v o lu tio n fo rm u la m ay be m odified to reflect th is re stric tio n . T h u s w e h ave
th e tw o e q u iv a le n t form s
OC

y ( n ) = ^ h ( k ) x ( n - k)
Jt=0
n
= ^
x{k) h{n k)
k=~oc

(2.3.39)

(2.3.40)

A s in d icated p rev io u sly , cau sality is re q u ire d in an y re a l-tim e signal p ro c e ss

ing a p p licatio n , since at a n y given tim e n w e have n o access to f u tu re v alu es o f th e

Sec. 2.3

Analysis of Discrete-Time Linear Time-Invariant Systems

87

in p u t signal. O n ly th e p re s e n t a n d p ast v alu es o f the in p u t signal are av ailab le in

c o m p u tin g th e p re s e n t o u tp u t.
It is so m e tim e s c o n v e n ie n t to call a se q u en ce th a t is z e ro fo r n < 0, a causa!
s e q u e n c e , an d o n e th a t is n o n z e ro fo r n < 0 a n d n > 0. a n on c aus al sequence. T his
te rm in o lo g y m e a n s th a t su c h a se q u e n c e could be th e u n it sa m p le re sp o n se of a
causal o r a n o n c a u sa l system , resp ectiv ely .
If th e in p u t to a causal lin e a r tim e -in v a ria n t system is a causal se q u e n c e [i.e.,
if jr(n) = 0 fo r n < 0]. th e lim its on th e co n v o lu tio n fo rm u la a re fu rth e r restricted .
In th is case th e tw o e q u iv a le n t form s o f th e c o n v o lu tio n fo rm u la b eco m e
n
y ( n ) = ^ h( k ) x ( n k)
(2.3.41)
*=o
n
=
-k)
(2.3.42)
*=<)
W e o b se rv e th a t in th is case, th e lim its on the su m m a tio n s fo r the tw o a lte rn a tiv e
fo rm s are id en tical, a n d th e u p p e r lim it is grow ing w ith tim e. C learly , th e resp o n se
o f a cau sal sy stem to a causal in p u t se q u e n c e is causal, since y( n) 0 fo r n < 0.
Example 2.3.5
Determ ine the unit step response of the linear time-invariant system with impulse
response
h{n) = a " u( n )

\a\ < 1

Solution Since the input signal is a unit step, which is a causal signal, and the system
is also causal, we can use one of the special forms of the convolution formula, either
(2.3.41) or (2.3.42). Since x(n) = 1 for n > 0. (2.3.41) is simpler to use Because of the
simplicity of this problem, one can skip the steps involved with sketching the folded
and shifted sequences. Instead, we use direct substitution of the signals sequences in
(2.3.41) and obtain
y(n) = y ~ v
*=(I
1 - a"*1
1 -ci
and y(n) = 0 for n < 0. We note that this result is identical to that obtained in Ex
ample 2.3.3. In this simple case, however, we computed the convolution algebraically
without resorting to the detailed procedure outlined previously.

2.3.6 Stability of Linear Time-Invariant Systems

A s in d ic a te d p rev io u sly , sta b ility is an im p o rta n t p ro p e rty th a t m u st be co n sid e re d
in an y p ra c tic a l im p le m e n ta tio n o f a system . W e d efin ed an a rb itra ry relax ed
system as B IB O sta b le if a n d only if its o u tp u t se q u e n c e y () is b o u n d e d fo r every
b o u n d e d in p u t x ( n) .

88

Discrete-Time Signals and Systems

Chap. 2

If x ( n ) is b o u n d e d , th e re exists a c o n s ta n t M x such th a t

<00

l* (n )l < M x

S im ilarly, if th e o u tp u t is b o u n d e d , th e re exists a c o n s ta n t M y such th a t

| ^ ( n ) | < My < OO

fo r all n.
N ow , given such a b o u n d e d in p u t se q u e n c e x ( n ) to a lin e a r tim e -in v a ria n t
sy stem , le t us in v e stig a te th e im p licatio n s of th e d efin itio n o f sta b ility o n th e c h a r
acteristics o f th e system . T o w a rd th is en d , w e w o rk again w ith th e c o n v o lu tio n
fo rm u la
OO
y( n) =
h{k) x{n - k )
k=oc
If w e ta k e th e a b so lu te value of b o th sides o f th is e q u a tio n , w e o b tain
h( k ) x ( n k)

Lv()| =

* = -O C

N ow , th e ab so lu te v alu e of th e sum o f term s is alw ays less th a n o r e q u a l to the

sum o f th e a b so lu te v alu es o f th e term s. H e n c e
OC

\y(n)\ < Y

IM*)II*(/1 - fc)l

i = oc

If th e in p u t is b o u n d e d , th e re exists a finite n u m b e r M x such th a t |x(n)[ < M t , By

su b s titu tin g th is u p p e r b o u n d fo r x( n ) in th e e q u a tio n ab o v e, w e o b ta in
OC'

|y (n )| < M X Y

W * )l

k = -o c

F ro m th is ex p ressio n w e o b se rv e th a t th e o u tp u t is b o u n d e d if th e im p u lse resp o n se

o f th e sy stem satisfies th e co n d itio n
OO

Sh =

Y
k = -o c

IAWI < 00

<2-3-43)

T h a t is, a linear time-invari ant s y s t em is stable i f its i mpu l s e respons e is absolutely

s u m m a b l e . T h is c o n d itio n is n o t only sufficient b u t it is also n e c e ssa ry to e n su re th e
sta b ility o f th e system . In d e e d , w e shall show th a t if 5* = 00, th e r e is a b o u n d e d
in p u t fo r w hich th e o u tp u t is n o t b o u n d e d . W e c h o o se th e b o u n d e d in p u t

\h*(-n)\
0,

h{n) ? 0
h(n) = 0

w h e re h*(n) is th e co m p lex c o n ju g a te o f h(n). I t is sufficient to show th a t th e re is

o n e v alu e o f n fo r w h ich y( n) is u n b o u n d e d . F o r n = 0 w e h av e
,< 0 ,=

;
*= 00^

k = (x. 1 v /l

Thus, if Si, = 00, a bounded input produces an unbounded output since y(0) = 00.

Sec. 2.3

Analysis of Discrete-Time Linear Tim e-Invariant Systems

T h e c o n d itio n in (2.3.43) im plies th a t th e im p u lse re s p o n s e h(n) goes to

as n a p p ro a c h e s infinity. A s a co n se q u e n c e , th e o u tp u t o f th e system g o es to
as n a p p ro a c h e s infinity if th e in p u t is set to z e ro b ey o n d n > n 0. T o p ro v e
su p p o se th a t |j ( h ) | < M x fo r n < no an d x ( n ) = 0 fo r n > no- T h e n , at n = no
th e system o u tp u t is
(no + AO =

^
h ( k ) x ( n 0 + N - k) + ^
*-=-oc

89
zero
zero
this,
+ N,

h { k ) x ( n 0 + N - k)

B u t th e first su m is z e ro since * () = 0 fo r n > no- F o r th e rem ain in g p a rt, we

ta k e th e a b so lu te v alu e o f th e o u tp u t, w hich is
j oc

y (0 + AO| =

sc

h( k)x(no + N )j <

< Mx

|/i(/:)||jf(no + N )|

W *)!
k=N

N ow , as N a p p ro a c h e s infinity.

an d hen ce

lim ]\ {/i() + AO! = 0

T h is re su lt im p lies th a t any ex citatio n at th e in p u t to th e system , w hich is of a finite
d u ra tio n , p ro d u c e s an o u tp u t th a t is tra n s ie n t in n a tu re ; th a t is, its am p litu d e
d ecay s w ith tim e an d dies o u t e v en tu ally , w h en th e system is stab le.
Example 2.3.6
Determ ine the range of values of the param eter a for which the linear time-invariant
system with impulse response
hin) = a"u(n)

is stable.
Solution First, we note that the system is causal. Consequently, the lower index on
the summation in (2.3.43) begins with k = 0. Hence

Clearly, this geom etric series converges to

provided that \a\ < 1 . Otherwise, it diverges. Therefore, the system is stable if |a| < 1.
Otherwise, it is unstable. In effect, h(.n) must decay exponentially toward zero as n
approaches infinity for the system to be stable.

90

Discrete-Time Signals and Systems

Chap. 2

Example 23.7
D eterm ine the range of values of a and b for which the linear time-invariant system
with impulse response
...
( a", n > 0
h(-n) = [ iw.f , n < n0
is stable.

Solution This system is noncasual. The condition on stability given by (2.3.43) yields
OC

t :

-1

OC

i*()f=

n= -o c

ia i" +
fi=s(J

y i

|fe,n

n= -o c

From Example 2.3.6 we have already determ ined that the first sum converges for
|a| < 1. The second sum can be manipulated as follows:

= p( l + p + p 2 + ) =

I - p

where p = \f\b\ must be less than unity for the geometric series to converge. Conse
quently, the system is stable if both \a\ < 1 and |fc| > 1 are satisfied,

2.3.7 Systems with Finite-Duration and Infinite-Duration

Impulse Response
U p to this p oin t w e have characterized a linear tim e-invariant system in term s of
its im pulse resp onse h(n). It is also con ven ien t, how ever, to subdivide the class
o f linear tim e-invariant system s in to tw o types, those that h ave a finite-duration
im pulse resp onse (F IR ) and those that have an infinite-duration im pulse response
(IIR ). Thus an F IR system has an im pulse resp onse that is zero ou tsid e o f som e
finite tim e interval. W ithout loss o f generality, w e focus our atten tion on causal
FIR system s, so that
h (n) = 0

n < 0 and n > M

The con volu tion form ula for such a system reduces to
u-1
? ( n ) = H h ^ x(^n ~ k )
t=0
A useful interpretation o f this exp ression is ob tain ed by ob serving that th e output
at any tim e n is sim ply a w eigh ted iinear com b ination o f the in p ut signal sam ples
x (n ), x { n - 1 ) , , x ( n - M + 1). In other words, the system sim ply w eigh ts, by
the values o f the im pulse resp onse h(k), k = 0, 1,
1, the m ost recent
M signal sam ples and sum s the resulting M products. In e ffe ct, th e system acts
as a w in d o w that view s on ly the m ost recent M input signal sam p les in form ing
the output. It n eg lects or sim ply forgets all prior input sam p les [i.e., x ( n M ),

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

91

x ( n M 1). . . .] . T h u s we say th a t an F IR svstem has a finite m e m o ry o f len g th -M

sam p les.
In c o n tra st, an I I R lin e a r tim e-in v arian t system has an in fin ite -d u ra tio n im
pu lse resp o n se. Its o u tp u t, b ased on th e c o n v o lu tio n fo rm u la, is
v(n) = ^ >2h(k) x( n - k )
w h ere cau sality h as b e e n assu m ed , a lth o u g h this a ssu m p tio n is n o t n ecessary. N ow .
th e system o u tp u t is a w eig h te d [by the im p u lse re sp o n se *(/:}] lin e a r co m b in a tio n
o f th e in p u t sig n al sa m p le s .r(n), x{n - 1), x( n 2 ) ........ S ince this w eig h te d sum
in v olves th e p re s e n t an d all th e p ast in p u t sam ples, w e say th a t th e sy stem has an
infinite m em o ry .
W e in v e stig a te th e c h aracteristics of F IR an d IIR sy stem s in m o re d etail in
su b s e q u e n t c h a p te rs.

2.4 DISCRETE-TIME SYSTEMS DESCRIBED BY DIFFERENCE

EQUATIONS
U p to this p o in t w e h ave tre a te d linear and tim e-in v arian t sy stem s th a t are c h a r
a c terized by th e ir u n it sa m p le resp o n se h(n). In tu rn . h(n) allow s us to d e te rm in e
th e o u tp u t v{n) o f th e system for any given in p u t se q u e n c e jc(/i) by m e a n s o f the
c o n v o lu tio n su m m a tio n .
(2.4.1)
In g e n e ra l, th e n , we h av e show n th a t any linear tim e -in v a ria n t system is c h a r
a c terized by th e in p u t- o u tp u t rela tio n sh ip in (2.4.1). M o re o v e r, th e co n v o lu tio n
su m m a tio n fo rm u la in (2.4.1) suggests a m ean s fo r th e re a liz a tio n o f th e system .
In th e case o f F IR sy stem s, such a realizatio n in v o lv es a d d itio n s, m u ltip lic atio n s,
a n d a finite n u m b e r o f m e m o ry locations. C o n se q u e n tly , an F IR system is read ily
im p le m e n te d d irectly , as im p lied by the c o n v o lu tio n su m m atio n .
If th e sy stem is IIR , h o w ev er, its practical im p le m e n ta tio n as im p lied by
c o n v o lu tio n is clearly im p o ssib le, since it re q u ire s an infinite n u m b e r o f m e m
o ry lo catio n s, m u ltip lic a tio n s, an d ad d itio n s. A q u e stio n th a t n a tu ra lly arises,
th e n , is w h e th e r o r n o t it is possible to realize IIR sy stem s o th e r th a n in the
form su g g e sted by th e c o n v o lu tio n su m m atio n . F o rtu n a te ly , th e an sw er is yes.
th e re is a p ra c tic a l an d c o m p u ta tio n a lly efficient m e a n s fo r im p le m e n tin g a
fam ily o f IIR sy stem s, as will b e d e m o n s tra te d in th is se ctio n . W ith in th e g en
eral class o f IIR sy stem s, this fam ily of d isc re te -tim e sy stem s is m o re c o n
v en ien tly d e s c rib e d by d ifferen ce eq u atio n s. T h is fam ily o r subclass of IIR
sy stem s is v ery u sefu l in a v ariety o f p ractical ap p licatio n s, in clu d in g th e im p le
m e n ta tio n o f d ig ita l filters, a n d the m o d elin g o f physical p h e n o m e n a a n d physical
system s.

92

Discrete-Time Signals and Systems

Chap. 2

2.4.1 Recursive and Nonrecursive Discrete-Time Systems

A s in d icated ab o v e, th e co n v o lu tio n su m m a tio n fo rm u la e x p re sse s th e o u tp u t of
th e lin e a r tim e -in v a ria n t sy stem explicitly a n d only in te rm s o f th e in p u t signal.
H o w e v e r, th is n e e d n o t b e th e case, as is sh o w n h e re . T h e re a re m an y system s
w h ere it is e ith e r n ecessa ry o r d e sirab le to ex p re ss th e o u tp u t o f th e system n o t
on ly in term s o f th e p re s e n t a n d p a st v alu es o f th e in p u t, b u t also in te rm s o f the
a lre a d y av ailab le p a st o u tp u t values. T h e follow ing p ro b le m illu stra te s th is point.
S u p p o se th a t w e w ish to c o m p u te th e cumul at i ve average o f a signal x ( n ) in
th e in te rv a l 0 < k < n, d efin ed as
1
"
v(n) = ------- = 0 . 1, . . .
n + 1 ~f.

(2.4.2)

A s im p lied b y (2.4.2), th e c o m p u ta tio n o f _v(n) re q u ire s th e sto ra g e o f all th e in p u t

sa m p le s x (k ) fo r 0 < k < n. Since n is in creasin g , o u r m em o ry r e q u ire m e n ts grow
lin early w ith tim e.
O u r in tu itio n suggests, h o w ev er, th a t y( n) can be c o m p u te d m o re efficiently
by utilizin g th e p re v io u s o u tp u t v alue y( n 1). In d e e d , by a sim p le alg eb raic
re a rra n g e m e n t o f (2.4.2), we o b ta in
itl
( + l)y (n ) =

x(k) + x(n)

= ny( n - 1) + x {n)
a n d h en ce
y( n) =

1
-x(n)
y(n - 1) +
n + 1'
n + 1'

(2.4.3)

N ow , th e cu m u lativ e a v erag e v(n) can b e c o m p u te d recu rsiv ely b y m u ltip ly in g th e

p re v io u s o u tp u t v a lu e y( n - 1) by n/ ( n 4-1 ), m u ltip ly in g th e p r e s e n t in p u t x ( n ) by
1 / (n + 1), an d a d d in g th e tw o p ro d u cts. T h u s th e c o m p u ta tio n o f y (n ) by m ean s
o f (2.4.3) re q u ire s tw o m u ltip lic atio n s, o n e a d d itio n , a n d o n e m e m o ry lo catio n , as
illu stra te d in Fig. 2.28. T his is an ex a m p le of a recursive s yst em. In g e n e ra l, a
sy stem w hose o u tp u t v(n) a t tim e n d e p e n d s o n a n y n u m b e r o f p a s t o u tp u t values
y( n - 1), y ( n - 2 ) , . . . is called a recu rsiv e system .
tin)

Figure 2.28

Realization of a recursive cum ulative averaging system.

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

93

T o d e te rm in e th e c o m p u ta tio n of the recu rsiv e sy stem in (2.4.3) in m ore

d etail, su p p o se th a t we begin th e p ro cess w ith n = 0 a n d p ro c e e d fo rw a rd in tim e.
T h u s, acco rd in g to (2.4.3). we o b ta in
y(0) -

jc(0)

v (l) = 5.v(0) + U ( l )
y(2) = y ( l ) + i* ( 2 )
an d so on. If o n e grow s fatig u ed w ith this c o m p u ta tio n an d w ishes to pass the
p ro b le m to so m e o n e else at som e tim e, say n = no. th e only in fo rm a tio n th a t one
n e e d s to p ro v id e his o r h e r su ccesso r is the p a st value y(o - 1) a n d the new' input
sa m p le s jr(n), j (/j + 1 )........ T h u s the successor b eg in s w ith

an d p ro c e e d s fo rw ard in tim e until som e tim e, say n = n j. w h en he or she b e

co m es fatig u ed an d passes the c o m p u ta tio n a l b u rd e n to so m e o n e else w ith the
in fo rm a tio n o n th e value _v(/?i 1). an d so on.
T h e p o in t wc wish to m ak e in th is discussion is th a t if o n e w ishes to co m p u te
th e re sp o n se (in this case, the cu m u lativ e av e ra g e ) of the sy stem (2,4.3) to an input
signal x( n ) a p p lied at n tiu. we n eed th e value y (n (, - 1) an d th e input sam ples
x ( n ) for /? > /in. T h e term y(>i() 1) is called th e initial condi t i on for the system in
(2.4.3) an d c o n ta in s all the in fo rm a tio n n e e d e d to d e te rm in e th e re sp o n se o f the
sy stem for n > () to th e in p u t signal x (n ). in d e p e n d e n t o f w h at has o ccu rred in
th e p ast.
T h e fo llow ing ex am p le illu strates th e use o f a (n o n lin e a r) recu rsiv e system
to co m p u te th e sq u a re ro o t of a n u m b e r.
Exam ple 2.4.1

Square-Root Algorithm

Many computers and calculators compute the square root of a positive number A.
using the iterative algorithm

where j_i is an initial guess (estimate) of \/~A. As the iteration converges we have
= J~A.
Consider now the recursive system

sn ~ s_j. T hen it easily follows that

(2.4.4)
which is realized as in Fig, 2.29. If we excite this system with a step of amplitude
A [i.e.. x(n) = A u ( n )] and use as an initial condition y(1) an estimate of
the
response v() of the system will tend toward
as n increases. Note that in contrast
to the system (2.4.3), we do not need to specify exactly the initial condition. A rough
estim ate is sufficient for the proper perform ance of the system. For example, if we

94

Discrete-Time Signals and Systems

Chap. 2

-0---- -0

4n)

v(n - I)

Figure Z 2 9

Realization of the square-root system.

let A = 2 and y ( - l ) = 1, we obtain j(0 ) =

>-(1) = 1,4166667, v(2) = 1,4142157.
Similarly, for y ( 1) = 1.5, we have v(0) = 1,416667, y (l) = 1.4142157. Compare
these values with the %/2, which is approximately 1.4142136.
W e h av e no w in tro d u c e d tw o sim ple recu rsiv e system s, w h ere th e o u tp u t vf/i)
d e p e n d s o n th e p re v io u s o u tp u t v alu e y( n 1) a n d th e c u rre n t in p u t ;r(n). B o th
system s a re causal. In g e n e ra l, w e can fo rm u la te m o re co m p lex cau sal recu rsiv e
system s, in w hich th e o u tp u t y ( n ) is a fu n ctio n o f se v era l p ast o u tp u t v alu es an d
p re se n t a n d p ast in p u ts. T h e system sh o u ld h av e a finite n u m b e r o f d elay s o r,
eq u iv alen tly , sh o u ld re q u ire a finite n u m b e r o f sto ra g e lo catio n s to be p ractically
im p le m e n te d . T h u s th e o u tp u t o f a cau sal a n d p ractically re a liz a b le recursive
system can be e x p ressed in g e n e ra l as
y( n) = F[y( n 1), y( n 2 ) , . . . , y( n N ) , x ( n) , x ( n 1 ) , . . . , x ( n M ) \

(2.4.5)

w h ere F [ ] d e n o te s so m e fu n ctio n of its a rg u m e n ts. T h is is a re c u rsiv e e q u a tio n

specifying a p ro c e d u re fo r co m p u tin g th e system o u tp u t in term s o f p rev io u s v alu es
o f th e o u tp u t a n d p re s e n t an d p ast inputs.
In c o n tra st, if y{n) d e p e n d s only o n th e p re s e n t an d p a st in p u ts, th e n
y (n ) = F[ x ( n ) , x ( n 1)........ x(rt M) ]

(2.4.6)

S uch a sy stem is called nonrecursi ve. W e h a s te n to ad d th a t th e c a u s a l F IR system s

d escrib ed in S ectio n 2.3.7 in te rm s o f th e co n v o lu tio n sum fo rm u la h av e th e fo rm
o f (2.4.6). In d e e d , th e c o n v o lu tio n su m m a tio n fo r a cau sal F IR sy stem is
M

y ( n) =

- *)
i=0

= h( 0 ) x ( n ) -)- h ( \ ) x ( n 1) + - + h ( M ) x ( n M )
F [j:(n ), x( n 1), . . . , x ( n M )]
w h ere th e fu n ctio n F[-] is sim ply a lin e a r w eig h ted su m o f p re s e n t a n d p ast in p u ts
a n d th e im p u lse re sp o n s e values h( n), 0 < n < M , c o n s titu te th e w eig h tin g c o e f
ficients. C o n se q u e n tly , th e cau sal lin e a r tim e -in v a ria n t F IR sy stem s d e s c rib e d by
th e c o n v o lu tio n fo rm u la in S ectio n 2.3.7, a re n o n re c u rsiv e . T h e b a sic d ifferen ces
b e tw e e n n o n re c u rsiv e a n d re c u rsiv e system s a re illu stra te d in Fig. 2.30. A sim ple
in sp e ctio n o f th is figure re v e a ls th a t th e fu n d a m e n ta l d iffe re n c e b e tw e e n th e s e tw o

Sec. 2.4

x(n)

Discrete-Time Systems Described by Difference Equations

95

V(H )

F[Mn). xin - 1),

x{n)

1
^

|
|

........ ti n - .Wl]

K n ) ........... v(/i -

y{n)

M)\
]

1--------------------------------- 1
(b)

Figure 2.30 Basic form for a causal

and realizable (a) nonrecursive and
(b) recursive system.

system s is the fe e d b ack loop in th e recu rsiv e system , w hich feed s b ack th e o u tp u t
o f the system in to th e in p u t. T h is feed b ack loop co n tain s a d e la y ele m e n t. T he
p resen c e o f this d elay is crucial for the realizab ility of the system , since the ab sen ce
of this d e la y w ould force the system to c o m p u te yi n) in term s o f v(n). w hich is
n o t po ssib le fo r d isc re te -tim e system s.
T h e p re se n c e o f the fe ed b ack loop o r, eq u iv alen tly , the recu rsiv e n a tu re of
(2.4.5) c re a te s a n o th e r im p o rta n t d ifferen ce b etw een recursive an d n o n re c u rsiv e
system s. F o r e x am p le, su p p o se th a t we wish to c o m p u te th e o u tp u t y(o) of a
system w h en it is ex cited by an in p u t ap p lied at tim e n = 0. If th e system is
recu rsiv e, to co m p u te y ( 0). we first n e e d to c o m p u te all the p re v io u s v alu es y(0).
y ( l ) ........ y(o - 1)- In c o n tra st, if the system is n o n recu rsiv e. we can c o m p u te the
o u tp u t y (n 0) im m e d ia te ly w ith o u t having y(no - 1), y(o 2 )........ In co n clu sio n ,
th e o u tp u t o f a recu rsiv e system sh o u ld be c o m p u te d in o rd e r [i.e., v(0), y ( l) ,
y ( 2 ) . . . w h e re a s for a n o n re c u rsiv e system , th e o u tp u t can b e c o m p u te d in any
o rd e r [i.e., y(200). y (15). y{3). y(300). etc.]. T his fe a tu re is d e sira b le in so m e
p ractical a p p licatio n s.

2.4.2 Linear Time-Invariant Systems Characterized by

Constant-Coefficient Difference Equations
In S ectio n 2.3 w e tre a te d lin e a r tim e -in v a ria n t system s a n d c h a ra c te riz e d th em
in te rm s o f th e ir im p u lse resp o n ses. In this su b sectio n w e focus o u r a tte n tio n
o n a fam ily o f iin e a r tim e -in v a ria n t system s d escrib ed by an in p u t- o u tp u t r e la
tio n called a d iffe re n c e e q u a tio n w ith c o n s ta n t c o e ffic ie n ts . S ystem s d e sc rib e d
by c o n s tan t-co efficien t lin e a r d ifferen ce e q u a tio n s are a subclass o f the recu rsiv e
a n d n o n re c u rsiv e sy stem s in tro d u c e d in th e p re ced in g su b sectio n . T o b rin g o u t
th e im p o rta n t id eas, w e b eg in by tre a tin g a sim ple re c u rsiv e sy stem d e sc rib e d by
a first-o rd e r d iffe re n c e e q u a tio n .

96

Discrete-Time Signals and Systems

Chap. 2

<*>
Figure 131 Block diagram realization
of a simple recursive system.

S u p p o se th a t w e h a v e a recu rsiv e system w ith an in p u t- o u tp u t e q u a tio n

y( n) = ay( n - 1) + x ( n )

(2.4.7)

w h ere a is a c o n sta n t. F ig u re 2.31 show s a block d ia g ra m re a liz a tio n o f th e system .

In co m p a rin g th is sy stem w ith th e cu m u la tiv e a v erag in g sy stem d e sc rib e d by the
in p u t-o u tp u t e q u a tio n (2.4.3), w e o b se rv e th a t th e system in (2.4.7) h as a c o n sta n t
co effic ie n t (in d e p e n d e n t o f tim e ), w h e re a s th e sy stem d e sc rib e d in (2.4.3) h as tim ev a ria n t coefficients. A s w e will show , (2.4.7) is an in p u t- o u tp u t e q u a tio n fo r a
lin e a r tim e -in v a ria n t sy stem , w h e re a s (2.4.3) d escrib es a lin e a r tim e -v a ria n t system .
N ow , su p p o se th a t w e ap p ly an in p u t signal x ( n ) to th e sy stem fo r n > 0.
W e m a k e n o assu m p tio n s a b o u t th e in p u t sig n a l fo r n < 0, b u t w e d o assum e
th e ex iste n ce o f th e in itial c o n d itio n v ( 1). S ince (2.4.7) d e s c rib e s th e system
o u tp u t im plicitly, w e m u st solve this e q u a tio n to o b ta in an ex p licit ex p ressio n for
th e sy stem o u tp u t. S u p p o se th a t w e co m p u te successive valu es o f y(n) fo r n > 0,
b eg in n in g w ith y(0), T h u s
y (0) = o j ( - l ) + x(0)
v (l) = ay (0 ) + jc(3 ) = a 2y ( - 1) + o j:( 0 ) + *(1)
y(2) = o j ( l ) + x ( 2 ) = o 3y (1) + a 2jr(0) + a ^ ( l ) + x ( 2 )

y( n) = a y( n - 1) + x( n)
= e " +1y ( - l ) + a" x( 0) + fl" 1Jt(l) + + a x ( n - 1) + x( n)
o r, m o re co m p actly ,
n > 0

(2.4.8)

T h e re sp o n se y ( n ) o f th e system as given by th e rig h t-h a n d sid e o f (2.4.8)

co n sists o f tw o p arts. T h e first p a rt, w hich c o n ta in s th e te rm y ( 1), is a re su lt of
th e in itial c o n d itio n y ( 1) o f th e system . T h e se co n d p a r t is th e re sp o n se o f the
sy stem to th e in p u t sig n al x( n) .
I f th e sy stem is initially re lax ed a t tim e n = 0, th e n its m e m o ry (i.e., the
o u tp u t o f th e d e la y ) sh o u ld b e zero . H e n c e y ( 1) = 0. T h u s a rec u rsiv e system is
re la x e d if it sta rts w ith z e ro in itial c o n d itio n s. B e c a u se th e m e m o ry o f th e system
d escrib es, in so m e sen se, its sta te , w e say th a t th e system is a t z e ro s ta te an d
its c o rre sp o n d in g o u tp u t is called th e zero-state respons e o r f o r c e d respons e, an d

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

97

is d e n o te d by yzs(n). O b v iously, the z e ro -sta te re sp o n se o r fo rc e d resp o n se o f the

system (2.4.7) is given by
V/jittf) Y ^ a kx( n k)
*=()

n > 0

(2.4.9)

It is in te re stin g to n o te th at (2.4.9) is a co n v o lu tio n su m m a tio n involving the

in p u t signal co n v o lv ed w ith the im p u lse re sp o n se
h(n) = a':u ( n )

(2.4.10)

W e also o b se rv e th a t the sy stem d e sc rib e d by th e first-o rd e r d ifferen ce e q u atio n

in (2.4.7) is cau sal. A s a resu lt, th e low er lim it on th e c o n v o lu tio n su m m atio n in
(2.4.9) is k = 0. F u rth e rm o re , th e co n d itio n v (1) = 0 im plies th a t the in p u t signal
can be a ssu m ed cau sal an d h en ce th e u p p e r lim it on th e co n v o lu tio n su m m atio n
in (2.4.9) is n. since x( n - k) = 0 for k > n. In effect, we have o b ta in e d th e resull
th a t th e re lax ed recu rsiv e system d esc rib e d by th e firs t-o rd e r d ifferen ce e q u atio n
in (2.4.7), is a lin e a r tim e-in v arian t IIR svstem w ith im p u lse resp o n se given bv
(2.4.10).
N ow . su p p o se th at th e system d e sc rib e d by (2.4.7) is in itially n o n re la x e d [i.e..
y ( 1) ^ 0] an d th e in p u t x(/i) = 0 for all //. T h e n the o u tp u t of th e system with
zero in p u t is called the zero- i nput response or nat ural r espons e an d is d e n o te d by
yZj(/j). F ro m (2.4.7). w ith ,v() = 0 for oc < n < oc . we o b ta in
\Y,OM = ',+ l y ( 1)

>

(2.4.11)

W e o b se rv e th a t a recu rsiv e system w ith n o n z e ro initial c o n d itio n is n o n relax ed

in th e se n se th a t it can p ro d u c e an o u tp u t w ith o u t b ein g ex cited . N o te th at the
z e ro -in p u t re sp o n se is d u e to th e m e m o ry of th e system .
T o su m m a riz e , th e z e ro -in p u t re sp o n se is o b ta in e d by se ttin g the in p u t signal
to z e ro , m ak in g it in d e p e n d e n t of the in p u t. It d e p e n d s only on th e n a tu re of the
system a n d th e in itial co n d itio n . T h u s th e z e ro -in p u t re sp o n s e is a ch a ra c te ristic of
th e sy stem itself, a n d it is also know n as th e nat ural o r f ree r esponse of th e svstem .
O n th e o th e r h a n d , th e z e ro -sta te resp o n se d e p e n d s on th e n a tu re of th e system
an d th e in p u t signal. Since th is o u tp u t is a re sp o n se fo rce d u p o n it by th e input
signal, it is u su ally called th e f o r c e d response of th e system . In g en eral, th e total
re sp o n se o f th e system can b e ex p ressed as v(n) = y Zi ( n ) + .Vzs( n).
T h e sy stem d escrib ed by th e first-o rd e r d iffe re n c e e q u a tio n in (2.4.7) is the
sim p lest p o ssib le recu rsiv e system in th e g en eral class o f re c u rsiv e system s d e
sc rib ed by lin e a r co n stan t-co efficien t d iffe re n c e e q u a tio n s . T h e g e n eral form for
such an e q u a tio n is
S

v(/i) = ^ ai,y{n - k) -j- ^ bkx ( n - k)

i-=l
k=0

(2.4,12)

or, e q u iv alen tly ,

Ar

' Y a ky{n - k) = ' Y ^ b kx( n - k)

Jc=0
*=0

a0 m 1

(2.4.13)

98

Discrete-Time Signals and Systems

Chap. 2

T h e in te g e r N is called th e order o f the d ifferen ce e q u a tio n o r th e o r d e r of the

system . T h e n eg ativ e sign on th e rig h t-h a n d side o f (2.4.12) is in tro d u c e d as a
m a tte r o f co n v en ien ce to allow us to ex p ress th e d ifferen ce e q u a tio n in (2.4.13)
w ith o u t any n eg ativ e signs.
E q u a tio n (2.4.12) ex p re sse s th e o u tp u t of the system at tim e rt d irectly as
a w eig h ted sum o f p ast o u tp u ts \ {n - 1), y( n - 2 ).........y(/i - N ) as well as past
an d p re se n t in p u t signals sam p les. W e o b se rv e th a t in o rd e r to d e te rm in e y( n)
fo r n > 0, we n e e d th e in p u t x ( n ) for all n > 0, an d th e initial c o n d itio n s y ( 1),
y ( 2), y ( N ) . In o th e r w o rd s, th e initial c o n d itio n s su m m a riz e all th a t we
n eed to k n o w a b o u t th e p a s t h isto ry o f th e re sp o n se o f th e sy stem to c o m p u te
th e p re se n t an d fu tu re o u tp u ts. T h e g e n e ra l so lu tio n o f th e /V -order c o n stan tco efficien t d ifferen ce e q u a tio n is c o n sid e re d in th e follow ing su b se c tio n .
A t this p o in t we r e s ta te th e p ro p e rtie s of lin earity , tim e in v a ria n c e , an d
stab ility in th e c o n te x t o f recu rsiv e system s d e sc rib e d by lin ear c o n s ta n t-c o e ffic ie n t
d ifferen ce e q u a tio n s. A s we h av e o b se rv ed , a recu rsiv e system m a y b e relax ed o r
n o n re la x e d , d e p e n d in g o n th e initial co n d itio n s. H e n c e th e d e fin itio n s o f th ese
p ro p e rtie s m u st ta k e in to a c c o u n t th e p re se n c e o f the initial co n d itio n s.
W e begin w ith th e d e fin itio n o f lin earity . A system is lin e a r if it satisfies th e
follow ing th re e re q u ire m e n ts:
1. T h e to tal re sp o n se is e q u a l to th e sum o f th e z e ro -in p u t a n d z e ro -sta te r e
sp o n ses [i.e.. y ( n) = y 7.\(n) + yzs(n)].
2. T h e p rin cip le o f su p e rp o s itio n ap p lies to th e z e ro -sta te re s p o n s e (zero-state
linear).
3. T h e p rin cip le o f su p e rp o s itio n ap p lies to the z e ro -in p u t re s p o n s e (zero- i nput
linear).
A system th a t d o es n o t satisfy all three se p a ra te r e q u ire m e n ts is by d efin itio n
n o n lin ear. O b v io u sly , fo r a re lax ed system , yZj(n) = 0, an d th u s re q u ire m e n t 2,
w hich is th e d efin itio n o f lin e a rity given in S ection 2.2.4, is sufficient.
W e illu strate th e a p p lic a tio n o f th ese re q u ire m e n ts by a sim p le ex am p le.
Example 2.4.2
D eterm ine if the recursive system defined by the difference equation
v(n) = av(n 1)4- x(n)
is linear.
Solution
as

By combining (2.4.9) and (2.4.11), we obtain (2.4.8), which can be expressed

y(n) = yzi(n) + y*(n)

Thus the first requirem ent for linearity is satisfied.

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

To check for the second requirem ent, let us assume that ,t(n) =
Then (2.4.9) gives

99
+

*={}

= ('iy ^ (n) + C2V^'(n)

Hence y(/i> satisfies the principle of superposition, and thus the system is zero-state
linear.
Now let us assume that y(1) = q vj(1) 4- f 2y;( 1). From (2.4.11) we obtain
1) 4- C;V;(1)]
=

vi (1)4-

v; ( 1)

= Ci v'i '(n) + r ;y ^(/i)

Hence the system is zero-input linear.
Since the system satisfies all three conditions for linearity, it is linear.
A lth o u g h it is so m e w h at ted io u s, the p ro c e d u re used in E x am p le 2.4,2 to
d e m o n s tra te lin e a rity for th e system d escrib ed by th e first-o rd e r d ifferen ce e q u a
tio n , ca rrie s o v er d irectly to th e g en eral recu rsiv e system s d e sc rib e d by th e c o n stan tcoefficient d ifferen ce e q u a tio n given in (2.4.13).
H en ce , a recu rsiv e system
d escrib ed by th e lin ear d iffe re n c e e q u a tio n in (2.4.13) also satisfies all th re e c o n
d itio n s in th e d efin itio n o f lin earity , a n d th e re fo re it is linear.
T h e n ex t q u e s tio n th a t arises is w h e th e r o r n o t the cau sal lin ear system
d e sc rib e d by th e lin e a r c o n stan t-co efficien t differen ce e q u a tio n in (2.4.13) is tim e
in v arian t. T h is is fairly easy, w h en d e alin g w ith system s d e sc rib e d by explicit
i n p u t-o u tp u t m a th e m a tic a l re la tio n sh ip s. C learly, th e system d e sc rib e d by (2.4.13)
is tim e in v a ria n t b ecau se th e coefficients ak an d bk are c o n stan ts. O n th e o th e r
h a n d , if o n e o r m o re o f th e s e coefficients d e p e n d s on tim e, th e system is tim e
v a ria n t, since its p ro p e rtie s ch an g e as a fu n ctio n of tim e. T h u s w e co n clu d e th at
the recursive sy s t em descri bed b y a linear constant-coefficient difference equat i on is
linear a n d t ime invariant.
T h e final issu e is th e sta b ility of th e recursive system d e sc rib e d by th e lin ear,
c o n stan t-co efficien t d iffe re n c e e q u a tio n in (2.4.13). In S ection 2.3.6 w e in tro d u c e d
th e c o n c e p t o f b o u n d e d in p u t-b o u n d e d o u tp u t (B IB O ) sta b ility fo r re la x e d sys
tem s. F o r n o n re la x e d system s th a t m ay b e n o n lin e a r, B IB O sta b ility sh o u ld be
view ed w ith so m e care. H o w e v e r, in th e case o f a lin e a r tim e -in v a ria n t recursive
system d e sc rib e d by th e lin e a r co n stan t-co efficien t d iffe re n c e e q u a tio n in (2.4.13),
it suffices to s ta te th a t such a system is B IB O sta b le if a n d only if fo r every
b o u n d e d in p u t a n d ev ery b o u n d e d initial c o n d itio n , th e to ta l system re sp o n se is
bounded.

100

Discrete-Time Signals and Systems

Chap. 2

Example 2.43
Determ ine if the linear tim e-invariant recursive system described by the difference
equation given in (2.4.7) is stable.
Solution
i*(n)l <

Let us assume that the input signal x(n) is bounded in amplitude, that is,
< oc for all n > 0. From (2.4.8) we have
j v{n J | < |a',+l_y(1)| + ^

akx(n - k) ,

n>0

If n is finite, the bound M v is finite and the output is bounded independently of the
value of a. However, as n -* oo, the bound My remains finite only if |aj < 1 because
|a |B - 0 as n -* oc. Then M y = Ms j(\ - |o|).
Thus the system is stable only if \a\ < 1.
For the sim ple first-order system in E xam ple 2.4,3, w e w ere able to express
the con d ition for B IB O stability in term s o f the system param eter a. nam ely \a\ < 1.
W e should stress, how ever, that this task b ecom es m ore difficult for higher-order
system s. F ortunately, as we shall see in su bsequent chapters, other sim ple and
m ore efficient tech n iqu es exist for investigating the stability o f recursive system s.

2.4.3 Solution of Linear Constant-Coefficient Difference

Equations
G iven a linear con stan t-coefficien t d ifferen ce eq u ation as the in p u t-o u tp u t rela
tionship describing a linear tim e-invariant system , our objective in this subsection
is to determ ine an explicit exp ression for the output y{n). T h e m eth od that is
d ev elo p ed is term ed the direct m e t h o d . A n alternative m eth od based on the ztransform is described in Chapter 3. For reasons that will b eco m e apparent later,
the z-transform approach is called the indirect m e th o d .
Basically, the goal is to determ ine the output y(n ), n > 0, o f the system given
a specific input x ( n ), n > 0, and a set o f initial con d ition s. T h e direct solution
m ethod assum es that the total solu tion is the sum o f tw o parts:
y ( n ) = y h(n) + }'r (n)
T he part y/,(n) is know n as the h o m o g e n e o u s or c o m p l e m e n ta r y solu tion , w hereas
yp(n) is called the particular solution.

The homogeneous solution of a difference equation. W e begin the

problem o f solving the linear con stan t-coefficien t d ifferen ce eq u ation given by

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

101

(2.4.13) bv assu m in g th a t th e in p u t x (n ) = 0. T h u s w e will first o b ta in th e so lu tio n

to th e h o m o g e n e o u s di ff erence equati on
\
J 2 a ky ( n - k ) = 0
*=(>

(2.4.14)

T h e p ro c e d u re fo r solving a lin e a r c o n s tan t-co efficien t d ifferen ce e q u a tio n

d irectly is very sim ilar to th e p ro c e d u re fo r solving a iin e a r c o n stan t-co efficien t
d iffe re n tia l e q u a tio n . B asically, we assum e th a t th e so lu tio n is in th e fo rm o f an
e x p o n e n tia l, th a t is.
yi,{n) = a"

(2.4.15)

w h ere th e su b scrip t h o n \ {n) is used to d e n o te th e so lu tio n to th e h o m o g e n e o u s

d ifferen ce e q u a tio n . If w e su b stitu te this assu m ed so lu tio n in (2.4.14), w e o b tain
th e p o ly n o m ial e q u a tio n
=0
fc=U
or
k" A (k* + ci)k^ ! + a2k^ * + - +

q n \ k

+ aw) = 0

(2.4.16)

T h e p o ly n o m ial in p a re n th e se s is called th e characteristic p o l y n o m i a l o f the

system . In g e n e ra l, it h as N roots, w hich we d e n o te as Xi. k 2.........k ^ . T h e ro o ts
can b e real o r co m p lex v alued. In p ra c tic e the co efficien ts a \ , a 2........ are usually
re a l. C o m p le x -v a lu e d ro o ts o ccu r as c o m p le x -c o n ju g a te p airs. S om e o f th e N ro o ts
m ay be id en tical, in w hich case w e have m u ltip Je -o rd e r ro o ts.
F o r th e m o m e n t, let us assum e th a t th e ro o ts a re d istin ct, th a t is, th e re are
n o m u ltip le -o rd e r ro o ts. T h e n th e m o st g e n eral so lu tio n to th e h o m o g e n e o u s
d iffe re n c e e q u a tio n in (2.4.14) is
yh(n) = C \ k \ + C2k 2 + + C^ k ^ ,

(2.4.17)

w h ere C i. C 2.........C s are w eig h tin g coefficients.

T h e se co efficien ts are d e te rm in e d fro m th e in itial c o n d itio n s specified fo r th e
sy stem . Since th e in p u t jr(n) = 0. (2.4.17) can b e u sed to o b ta in th e z ero- i nput
response o f th e system . T h e follow ing ex am p les illu stra te th e p ro c e d u re .
Example 2.4.4
D eterm ine the hom ogeneous solution of the system described by the first-order dif
ference equation
_v(n) + a\y(rt 1) = x(n)
Solution

The assumed solution obtained by setting x(n) = 0 is

y*(n) = V

(2.4.18)

D iscrete-Time Signals and System s

102

Chap. 2

When we substitute this solution in (2.4.18), we obtain [with x(n) = 0]

X -f-12]An ^ = 0
A" *(A + ai) = 0
A = O)
Therefore, the solution to the hom ogeneous difference equation is
= CA" = C ( - a ,)"

(2.4.19)

The zero-input response of the system can be determ ined from (2.4,18) and
(2.4.19). With x(n) = 0, (2.4.18) yields
v(0) =

-a, v ( - l )

On the other hand, from (2.4,19) we have

v*(0) = C
and hence the zero-input response of the system is
Vii() = (i)n+1 v{1)

n >0

(2.4.20)

With a = ai, this result is consistent with (2.4,11) for the first-order system, which
was obtained earlier by iteration of the difference equation.

Example 2AS
Determ ine the zero-input response of the system described by the homogeneous
second-order difference equation
y() - 3y(n - 1) - 4y(n - 2) = 0

(2.4.21)

Solution First we determ ine the solution to the homogeneous equation. We assume
the solution to be the exponential
yh(n) = X"
U pon substitution of this solution into (2.4.21), we obtain the characteristic equation
a"

- 3xn- ' - 4 r ~ 2 = o

A"-2(X2 3A 4) = 0
Therefore, the roots are X = - 1 , 4, and the general form of the solution to the
homogeneous equation is
}7i(n) = CjX" + C 2X2

(2.4.22)

= C1( - i r + C2(4)"
The zero-input response of the system can be obtained from the homogenous
solution by evaluating the constants in (2.4.22), given the initial conditions y (1) and
y ( ~ 2). From the difference equation in (2.4.21) we have
y(0) = 3y(1) + 4y(2)
y(l) = 3v(0) + 4_y(1)
= 3 [3 y (-l)+ 4y(-2)] + 4y(-l)
= 13y(1) + l 2y( 2)

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

103

On the other hand, from (2.4.22) we obtain

v(0> = C) + C;

V( 1 I = -C l +4C;
By equating these two sets of relations, we have
C i + C ; = 3_v( 1) + 4y(2)
- C , + 4 C ; = 13 v{ 1) + 12 y( 2)
The solution of these two equations is
Ci = 7 v( 1 ) + ? v ( - 2 )
C; = X .V (-l) + T.v( -2 )
Therefore, the zero-input response of the system is

v,i(n) = [5 v( 1 >- i v(-2)](-l)''

(2.4.23)
n >0

+ [ x . v ( - l + t.v(-2)](4>"

For example, if v{2) = 0 and y ( - l ) = 5. then C| = 1, C: = 16. and hence

y-,i(H) = ( -1 ) " * 1 + (4)"+:

n >0

T h e se e x am p les illu strate the m e th o d fo r o b ta in in g the h o m o g e n e o u s so lu tio n

and th e z e ro -in p u t re sp o n se o f th e system w hen the ch a ra c te ristic e q u a tio n co n tain s
d istin ct ro o ts. O n th e o th e r h an d , if th e c h a ra c te ristic e q u a tio n co n ta in s m u ltip le
ro o ts, th e form o f th e so lu tio n given in (2.4.17) m ust be m odified. F o r ex am p le, if
ai is a ro o t o f m u ltip licity m, th en (2.4.17) b eco m es

\h(ii) = Ci A.'.' + C2/iX! + C v;:a',' + + Cm>7n'~lX'!

(2.4.24)
+ Cm+\)"m+\ + + C^Kl

The particular solution of the difference equation. T h e p a rtic u la r so

lu tio n \ p (n) is re q u ire d to satisfy th e d ifferen ce e q u a tio n (2.4.13) fo r th e specific
in p u t signal x (n ). n > 0. In o th e r w ords, y p(n) is any so lu tio n satisfying
N

' Y ^ a ky p (n - k) ' Y ^ b kx ( n - k)
A-0
*=0

an 1

(2.4.25)

T o solve (2.4.25). w e assu m e fo r yp (n), a fo rm th at d e p e n d s on th e fo rm o f th e

in p u t j:(h ). T h e fo llow ing ex am p le illu strates the p ro c e d u re .
Example 2.4.6
D eterm ine the particular solution of the first-order difference equation
y(n) + fliy(n - 1) = x(n).

| f l i | <l

when the input x(n) is a unit step sequence, that is.

x(n) = u (n )

(2,4.26)

104

Discrete-Time Signals and System s

Chap. 2

Solution Since the input sequence x(n) is a constant for n > 0. the form of the solu
tion that we assume is also a constant. Hence the assumed solution of the difference
equation to the forcing function x(n), called the particular solution of the difference
equation, is
vr (n) = Ku(n)
where A- is a scale factor determined so that (2.4.26) is satisfied. U pon substitution
of this assumed solution into (2.4.26). we obtain
Ku i n ) + d]Ku{,n 1) = u ( n )

To determine K, we must evaluate this equation for any n > 1. where none of the
terms vanish. Thus
K -t- O] K 1

1
K = -------l+^i
Therefore, the particular solution to the difference equation is
v (n) = --------u(n)
'
1 +fii

(2.4.27)

In th is ex am p le, th e in p u t * ( ). n > 0. is a c o n s ta n t an d th e fo rm assu m ed

for th e p a rtic u la r so lu tio n is also a c o n sta n t. If x{n) is an e x p o n e n tia l, w e w ould
assu m e th a t th e p a rtic u ia r so lu tio n is also an e x p o n e n tia l. If x {n) w e re a sin u so id ,
th e n >/,() w o uld also be a sinusoid. T h u s o u r assu m ed fo rm fo r th e p a rtic u la r
so lu tio n tak es th e basic fo rm of th e signal x{n). T a b le 2.1 p ro v id e s th e g en eral
fo rm o f th e p a rtic u la r so lu tio n for sev eral ty p es of excitatio n .
Exam ple 2.4.7
Determ ine the particuiar solution of the difference equation
v(n) = | y(n - 1) - v(w - 2) + x(n)
when the forcing function x(n) = 2". n > 0 and zero elsewhere.
TABLE 2.1 GENERAL FORM OF THE PARTICULAR
SOLUTION FOR SEVERAL TYPES OF INPUT
SIGNALS

Input Signal,
x(n)

Particular Solution,
vr (n)

A (constant)
AM"
AnM

K
KM"
Kan + K ^ " - ' 1 + . . . + KM

A cos Wf>n
A sin a>on

Kj cos toon -I- K2 sin won

AnnM

An(Ki)nM+ K\ttM 1

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

Solution

105

The form of the particular solution is

yp (n ) = K2n

n >0

Upon substitution of yP(n) into the difference equation, we obtain

K2"u( n) =

1)

- I K 2 n- 2u(n - 2 ) + 2 " ( )

To determ ine the value of K, we can evaluate this equation for any n > 2, where
none of the terms vanish. Thus we obtain
4A" = \ ( 2K) - i f f + 4
and hence K =

Therefore, the particular solution is

yn(n) = ^2"

n > 0

W e h av e n o w d e m o n s tra te d how to d e te rm in e th e tw o c o m p o n e n ts o f the

so lu tio n to a d ifferen ce e q u a tio n w ith c o n s ta n t coefficients. T h e se tw o c o m p o n e n ts
are th e h o m o g e n e o u s so lu tio n and th e p a rtic u la r so lu tio n . F ro m th e s e tw o co m
p o n e n ts, we c o n s tru c t the to tal so lu tio n fro m w hich w e can o b ta in th e ze ro -sta te
re sp o n se .

The total solution of the difference equation. T h e lin e a rity p ro p e rty o f

th e lin ear c o n stan t-co efficien t d ifferen ce e q u a tio n allow s u s to a d d th e h o m o g e
n e o u s so lu tio n an d the p a rtic u la r so lu tio n in o rd e r to o b ta in th e total solution. T h u s

v() =

y h(n)

+ \ p( n)

T h e r e s u lta n t sum v(n) co n tain s th e c o n s ta n t p a r a m e te r s {C/} e m b o d ie d in th e

h o m o g e n e o u s so lu tio n c o m p o n e n t >v,(n). T h e se c o n s ta n ts can be d e te rm in e d to
satisfy th e in itial co n d itio n s. T h e follow ing ex a m p le illu stra te s th e p ro c e d u re ,
Exam ple 2.4.8
D eterm ine the total solution y(/i), n > 0, to the difference equation
y(n) +a i y ( n - 1) = x(n)

(2.4.28)

when x( n) is a unit step sequence [i.e., x(n) = ()] and y (1) is the initial condition.
Solution

From (2.4,19) of Example 2.4.4, the hom ogeneous solution is

y*(n) = C(-fli)"
and from (2.4.26) of Example 2.4.6, the particular solution is

Consequently, the total solution is

y(n) = C ( - a i)" + -
1 + fli

n >0

where the constant C is determ ined to satisfy the initial condition y ( - l ) .

(2.4.29)

106

Discrete-Time Signals and Systems

Chap. 2

In particular, suppose that we wish to obtain the zero-siate response of the

system described by the first-order difference equation in (2.4.28). T hen we set
y( 1) = 0. To evaluate C. we evaluate (2.4.28) at n = 0 obtaining
y ( 0 ) + Oi y ( 1 ) =

y(0) = 1
On the other hand, (2.4.29) evaluated at n = 0 yields
v(0) = C 4- i
1 + ax
Consequently.
1
C + -------- = 1
1 + i
ai
1 + ai

Substitution for C into (2.4.29) yields the zero-state response of the system
l-f-fli)" * '
Vi*(/i) = ---- -----
1 + i

n >0

If we evaluate the param eter C in (2,4.29) under the condition that y( 1) ^ 0. the
total solution will include the zero-input response as well as the zero-state response
of the system. In this case (2.4.28) yields
y(0) + a]V( 1) = 1
y (0) = fliy( 1) 4- 1

On the other hand. (2.4.29) yields

]
1 + U\
By equating these two relations, we obtain
C + ^ = ai v(1) 4- 1
1 4 - a,

C = -g ] v( 1) +

-
1 4- a ]

Finally, if we substitute this value of C into (2.4.29). we obtain

y(n) = (a , r +1y(1) + -
=

^
1 + a)

n >0

(2.4.30)

Vzs( n)

W e o b se rv e th a t th e system re sp o n se as given by (2.4.30) is c o n s iste n t w ith

th e re sp o n se y ( n) given in (2.4.8) for th e first-o rd e r system (w ith a = - o i ) . w hich
was o b ta in e d by solv in g th e d ifferen ce e q u a tio n iterativ e ly . F u rth e rm o r e , we n o te
th a t th e v alu e o f th e c o n s ta n t C d e p e n d s b o th o n th e initial co n d itio n y (1) an d
o n th e ex citatio n fu n ctio n . C o n se q u e n tly , th e v alue o f C in flu en ces b o th th e zeroin p u t re sp o n se a n d th e z e ro -sta te re sp o n se . O n th e o th e r h a n d , if w e w ish to

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

107

obtain the zero-state resp onse only, w e sim ply solve for C under th e con d ition
that y ( - l ) = 0, as d em onstrated in E xam ple 2.4.8.
W e further ob serve that the particular solution to the d ifferen ce equation can
b e o b tain ed from the zero-state resp onse o f the system . In d eed , if |o]| < 1, which
is the con d ition for stability o f the system , as w ill be show n in S ection 2.4.4, the
lim iting valu e o f >zs() as n approaches infinity, is the particular solu tion , that is,
1
( n ) = lim >zs() = --------n-*Oo
1+ai
Since this com p onent o f the system response d o es not go to zero as n approaches
infinity, it is usually called the steady-state re spons e o f the system . T his response
persists as lon g as the input persists. T he com p onent that d ies out as n approaches
infinity is called the transient re sponse o f the system .

Example 2.4.9
D eterm ine the response y(n), n > 0, of the system described by the second-order
difference equation
v(n) - 3y(n - 1) - 4v(n - 2) = x( n ) + 2x(n - 1)

(2.4.31)

when the input sequence is

x(n) =4"u(n)

Solution We have already determ ined the solution to the homogeneous difference
equation for this system in Example 2.4.5. From (2.4.22) we have
y*(n) = C , ( - l ) n + C 2(4)n

(2.4.32)

The particular solution to (2.4.31) is assumed to be an exponential sequence of the

same form as x(n). Normally, we could assume a solution of the form
yp(n) = K(4,)au(n)
However, we observe that ( n ) is already contained in the hom ogeneous solution,
so that this particular solution is redundant. Instead, we select the particular solution
to be linearly independent of the terms contained in the homogeneous solution. In
fact, we treat this situation in the same manner as we have already treated multiple
roots in the characteristic equation. Thus we assume that
yp(n) = Kn{4)"u(n)

(2.4.33)

Upon substitution of (2.4.33) into (2.4.31), we obtain

Kn(4)"u(n) - 3 K ( n - l)(4 ),' - 1u(n - 1) - 4 K{n - 2)(4)n- 2u(n - 2)
= (4)"u(n) + 2(4)"-1 m(/i 1)
To determ ine K , we evaluate this equation for any n > 2, where none of the
unit step term s vanish. To simplify the arithmetic, we select n = 2, from which we
obtain K = | . Therefore,

yP{n) |rt(4)n(/l)

(2.4.34)

108

Discrete-Time Signals and System s

Chap. 2

The total solution to the difference equation is obtained by adding (2.4.32) to

(2.4.34). Thus

y() C ](1)" + C;(4)n + |w(4)"

n > 0

(2.4.35)

where the constants C\ and C2 are determined such that the initial conditions are
satisfied. To accomplish this, we return to (2.4.31), from which we obtain
v(0) = 3 y (- l) + 4y (2) + 1
y (1) = 3v(0) + 4_v(-l) + 6
= 1 3 y (- l) + 12y(-2) + 9
O n the other hand, (2.4.35) evaluated at n = 0 and n = 1 yields
y(0) = Ci + C2
y (l) = -C] + 4C2 + f
W e can now equate these two sets of relations to obtain C\ and C 2. In so doing, we
have the response due to initial conditions y (1) and y (2) (the zero-input response),
and the zero-state or forced response.
Since we have already solved for the zero-input response in Exam ple 2.4.5. we
can simplify the computations above by setting v (1) = y (2) = 0. Then we have
C, + C; = 1
- C l + 4 C2 + f

=9

Hence C : = ^ and C 2 =
Finally, we have the zero-statc response to the forcing
function
= (4)"(n) in the form

vB(n) =

+5<4)" +H 4)"

nZ0

(2.4.36)

The total response of the system, which includes the response to arbitrary initial
conditions, is the sum of (2.4.23) and (2.4.36).

2.4.4 The Impulse Response of a Linear Time-Invariant

Recursive System
T h e im p u lse re sp o n se o f a lin e a r tim e -in v a ria n t sy stem w as p re v io u sly defin ed as
th e re sp o n se o f th e sy stem to a u n it sa m p le e x citatio n [i.e., x (n ) = <5(n)]. In th e
case o f a recu rsiv e sy stem , h ( n ) is sim ply e q u a l to th e z e ro -s ta te re sp o n s e o f the
system w h en th e in p u t j:(n ) = <5(n) an d th e system is initially re la x e d .
F o r ex am p le, in th e sim ple first-o rd e r re c u rsiv e system g iv en in (2.4.7), th e
z e ro -sta te re sp o n se g iv en in (2.4.8), is
n

(2.4.37)
*=o
W ith jr(n) = i ( ) is substituted into (2.4.37), we obtain
n
yzs(n) = Y 2 a k8(n - k)
=

an

n > 0

Sec. 2.4

Discrete-Time Systems Described by Difference Equations

109

H e n c e th e im p u lse re sp o n se o f th e first-o rd e r rec u rsiv e system d e sc rib e d by

(2.4.7) is
h(n) = a nu(n)

(2.4.38)

as in d ic a te d in S ectio n 2.4.2.
In th e g e n e ra l case o f an a rb itra ry , lin e a r tim e -in v a ria n t rec u rsiv e system , the
z e ro -s ta te re sp o n s e ex p re sse d in te rm s o f th e co n v o lu tio n su m m a tio n is
>zs(n) ^ 2 h ( k ) x ( n k)

n > 0

(2.4.39)

i=0

W h e n th e in p u t is an im p u lse [i.e., * ( ) = <5()], (2.4.39) re d u c e s to

Vzs(n) = h(n)

(2.4.40)

N o w , let us c o n s id e r th e p ro b le m o f d e te rm in in g th e im p u lse re sp o n se h i n ) given a

lin e a r c o n s tan t-co efficien t d ifferen ce e q u a tio n d e sc rip tio n o f th e system . In term s
o f o u r d isc u ssio n in th e p re c e d in g su b sectio n , we h av e e s ta b lis h e d th e fact th a t the
to ta l re sp o n s e o f th e sy stem to any e x citatio n fu n ctio n co n sists o f th e sum o f tw o
so lu tio n s o f th e d ifferen ce e q u atio n : th e so lu tio n to the h o m o g e n e o u s e q u atio n
p lu s th e p a rtic u la r so lu tio n to th e e x c itatio n fu n ctio n . In th e case w h ere the exci
ta tio n is an im p u lse, th e p a rtic u la r so lu tio n is z e ro , since x ( n ) = 0 for n > 0. th at is.
yp (n) = 0
C o n s e q u e n tly , th e re sp o n se o f th e system to an im p u lse co n sists only o f th e so lu
tio n to th e h o m o g e n e o u s e q u a tio n , w ith the ( Q ) p a r a m e te r s e v a lu a te d to satisfy
th e in itial c o n d itio n s d ic ta te d by th e im pulse. T h e follow ing ex am p le illu strates
th e p ro c e d u re fo r o b ta in in g h(n) given th e d ifferen ce e q u a tio n fo r the system .
Example 2.4.10
D eterm ine the impulse response h(n) for the system described by the second-order
difference equation
y(n ) 3v(n 1) 4 y ( 2) = x ( n ) + 2 x i n 1)

(2.4.41)

Solution W e have already determ ined in Example 2.4.5 that the solution to the
hom ogeneous difference equation for this system is
^ ( n ) = C, (-1 )" + C2(4)"

n > 0

(2.4.42)

Since the particular solution is zero when x(n) = 6(n), the impulse response of the sys
tem is simply given by (2.4.42), where C] and C2 must be evaluated to satisfy (2.4.41).
For n = 0 and n = 1, (2.4.41) yields
v(0) = 1
y (1) = 3 y (0 )+ 2 = 5
where we have imposed the conditions y (1) = y (2) = 0. since the system must be
relaxed. On the other hand, (2.4.42) evaluated at n = 0 and n = 1 yields
y (0) = C, + C2

y (1) = Ci + 4C2

Discrete-Time Signals and Systems

110

Chap. 2

By solving these two sets of equations for C] and C2, we obtain

=

C: = 5

Therefore, the impulse response of the system is

h{n) = [i ( 1)" + f (4)-}u(n)
W e m ak e th e o b se rv a tio n th a t b o th th e sim ple firs t-o rd e r recu rsiv e system
an d th e se c o n d -o rd e r recu rsiv e system h av e im pulse re sp o n se s th a t a re infinite in
d u ra tio n . In o th e r w o rds, b o th o f th e s e recu rsiv e system s a re IIR system s. In
fact, d u e to th e recu rsiv e n a tu re of th e system , an y recu rsiv e sy stem d e sc rib e d by
a lin e a r co n stan t-co efficien t d iffe re n c e e q u a tio n is an I IR system . T h e co n v erse
is n o t tru e, h o w ev er. T h a t is, n o t ev ery lin e a r tim e -in v a ria n t I I R sy stem can be
d e sc rib e d by a lin e a r c o n s tan t-co efficien t d ifferen ce e q u a tio n . In o th e r w ords,
recu rsiv e sy stem s d e sc rib e d by lin ear co n sta n t-c o e ffic ie n t d iffe re n c e e q u a tio n s are
a su bclass o f lin e a r tim e -in v a ria n t I I R system s.
T h e ex ten sio n o f th e a p p ro a c h th a t w e h av e d e m o n s tra te d fo r d e te rm in
ing th e im p u lse re sp o n se o f th e first- a n d s e c o n d -o rd e r system s, g en e ra liz e s in a
stra ig h tfo rw a rd m a n n e r. W h e n th e system is d e sc rib e d by an A 'th -o rd e r lin e a r
d ifferen ce e q u a tio n o f th e ty p e given in (2.4.13), th e so lu tio n o f th e h o m o g e n e o u s
e q u a tio n is

*=i
w h en th e ro o ts {a*} o f th e c h a ra c te ristic p o ly n o m ial are d istin ct. H e n c e th e im pulse
re sp o n se o f th e sy stem is id en tical in fo rm , th a t is.
(2.4.43)
w h ere th e p a ra m e te rs {Ctl are d e te rm in e d by se ttin g th e initial c o n d itio n s v ( 1) =
. . . = y ( - N ) = 0.
T h is fo rm o f h{n) allow s us to easily re la te th e stab ility of a system , d escrib ed
by an N th -o rd e r d iffe re n c e e q u a tio n , to th e values o f th e ro o ts o f th e ch a ra c te ristic
p o ly n o m ial. In d e e d , since B IB O sta b ility re q u ire s th a t th e im p u lse re sp o n s e be
ab so lu te ly su m m ab le, th e n , fo r a causal system , w e h av e
x
oc
N
N
oo
oo
> (* )! =
Y ^ C kXnk < | C * | | A * | "
=0
n
=0
n=0 Ik=-l
k=\
n=0

N ow if |

j < 1 fo r all k, th e n

an d h en ce
OC

IM*)[ < oo

Sec. 2.5

im plementation of Discrete-Time Systems

111

O n th e o th e r h an d , if o n e o r m o re o f th e
> 1, h{n) is n o lo n g e r ab so lu te ly
su m m ab le, an d c o n se q u e n tly , th e sy stem is u n sta b le . T h e re fo re , a n ecessa ry an d
sufficient c o n d itio n fo r th e sta b ility o f a causal IIR system d e sc rib e d by a lin ear
c o n s tan t-co efficien t d iffe re n c e e q u a tio n , is th a t al! ro o ts o f th e c h a ra c te ristic p o ly
n o m ial b e less th a n u n ity in m a g n itu d e. T h e re a d e r m ay verify th a t th is c o n d itio n
ca rrie s o v er to th e case w h ere th e system h as ro o ts o f m u ltip lic ity m.

2.5 IMPLEMENTATION OF DISCRETE-TIME SYSTEMS

O u r tre a tm e n t o f d isc re te -tim e sy stem s h as b e e n fo cu sed on th e tim e -d o m a in c h a r
ac te riz a tio n an d analysis o f lin ear tim e -in v a ria n t system s d esc rib e d by c o n stan tco efficien t lin e a r d ifferen ce e q u a tio n s. A d d itio n a l an aly tical m e th o d s a re d e v e l
o p e d in th e n ex t tw o c h a p te rs, w h ere we c h a ra c te riz e an d an aly ze L TI sy stem s in
th e fre q u e n c y d o m ain . T w o o th e r im p o rta n t to p ics th a t will b e tr e a te d la te r are
th e d esig n an d im p le m e n ta tio n o f th e se system s.
In p ractice, system design a n d im p le m e n ta tio n a re usually tr e a te d jo in tly
r a th e r th a n se p a ra te ly . O fte n , th e system design is d riv en by th e m e th o d o f
im p le m e n ta tio n an d by im p le m e n ta tio n c o n stra in ts, such as cost, h a rd w a re lim
itatio n s, size lim ita tio n s, a n d p o w e r re q u ire m e n ts . A t th is p o in t, we h av e n o t
as y e t d e v e lo p e d th e n ecessa ry analysis an d design tools to tr e a t such com plex
issues. H o w ev er, we h ave d e v e lo p e d sufficient b a c k g ro u n d to c o n sid e r so m e b a
sic im p le m e n ta tio n m e th o d s for re a liz a tio n s o f L TI sy stem s d escrib ed by lin ear
c o n s tan t-co efficien t d iffe re n c e e q u atio n s.

2.5.1 Structures for the Realization of Linear

Time-Invariant Systems
In th is su b sectio n we d esc rib e s tru c tu re s fo r th e re a liz a tio n o f system s d escrib ed
by lin e a r c o n s tan t-co efficien t d ifferen ce e q u a tio n s. A d d itio n a l stru c tu re s fo r th ese
sy stem s a re in tro d u c e d in C h a p te r 7.
A s a b eg in n in g , let us c o n sid e r th e first-o rd e r system
y(n) = a iy (n 1) + box ( n) + b \ x ( n - 1)

(2.5.1)

w hich is re a liz e d as in Fig. 2.32a. T h is re a liz a tio n uses s e p a ra te delays (m em o ry )

fo r b o th th e in p u t a n d o u tp u t signal sa m p le s a n d it is called a direct f o r m I structure.
N o te th a t th is sy stem can b e view ed as tw o lin e a r tim e -in v a ria n t system s in cascad e.
T h e first is a n o n re c u rsiv e , sy stem d e sc rib e d by th e e q u a tio n
u(n) = Zoj;(n) + 6 j j ( n 1)

(2.5.2)

w h e re a s th e se c o n d is a rec u rsiv e system d e sc rib e d by th e e q u a tio n

y ( n ) = - a i y ( n - 1) + t>{)

(2.5.3)

H o w e v e r, as we h a v e se e n in S ectio n 2.3.4, if w e in te rc h a n g e th e o r d e r o f the

c a s cad ed lin e a r tim e -in v a ria n t system s, th e o v erall system re sp o n s e re m a in s th e

112

Discrete-Time Signals and Systems

Chap. 2

sam e. T h u s if w e in te rc h a n g e the o rd e r o f the recu rsiv e an d n o n re c u rsiv e system s,

we o b ta in an a lte rn a tiv e s tru c tu re fo r th e re a liz a tio n o f th e sv stem d e sc rib e d by
(2.5.1). T h e re su ltin g system is show n in Fig. 2.32b. F ro m th is figure we o b tain
th e tw o d ifferen ce e q u a tio n s
w( n) fl] w( n 1) + jt()

(2-5.4)

v(n) = t>nw(n) + b\ w{ n - 1)

(2.5.5)

w hich p ro v id e an a lte rn a tiv e a lg o rith m for co m p u tin g th e o u tp u t o f the system

d e sc rib e d by th e single d ifferen ce e q u a tio n given in (2.5.1). In o th e r w o rd s, the
tw o d ifferen ce e q u a tio n s (2.5.4) an d (2.5.5) are e q u iv a le n t to th e single differen ce
e q u a tio n (2.5.1).
A close o b se rv a tio n o f Fig. 2.32 re v e a ls th at th e tw o d elay e le m e n ts co n tain
th e sam e in p u t w( n) an d h en c e the sam e o u tp u t w( n 1). C o n s e q u e n tly , these
tw o e le m e n ts can be m e rg ed in to o n e delay, as show n in Fig. 2.32c. In c o n tra st
x(n)

b{,

r(n)

vim

-----------------~

(a)

(b.)

(c)

Figure 132 Steps in converting from the direct form I realization in (a) to the
direct form II realization in (c).

Sec. 2.5

Implementation of Discrete-Time Systems

113

to th e d ire c t fo rm I s tru c tu re , th is new re a liz a tio n re q u ire s o n ly o n e d elay fo r

th e au x iliary q u a n tity ui(n), an d h en c e it is m o re efficient in te rm s o f m em o ry
re q u ire m e n ts . It is called th e direct f o r m I I structure a n d it is u sed ex ten siv ely in
p ra c tic a l a p p licatio n s.
T h e se stru c tu re s can read ily be g e n e ra liz e d fo r th e g e n e ra l lin e a r tim ein v a ria n t recu rsiv e sy stem d escrib ed by th e d iffe re n c e e q u a tio n
N

y (n ) = - y ^ a ^ .y ( - k) +
*=l

- k)

(2.5.6)

k=0

F ig u re 2.33 illu stra te s th e d ire c t fo rm I s tru c tu re fo r th is sy stem . T h is stru c tu re

re q u ire s M + N d elay s a n d N + M + 1 m u ltip lic atio n s. I t c a n be v iew ed as the
c ascad e o f a n o n re c u rsiv e system
M
i;() = Y bkx{n - k)

(2.5.7)

i=U

a n d a recu rsiv e system

s

y(n) = -

v(n ~ k) + v(n)

(2.5.8)

By rev ersin g th e o r d e r o f th ese tw o system s as was p rev io u sly d o n e fo r th e

first-o rd e r sy stem , w e o b ta in th e d ire c t form II s tru c tu re sh o w n in Fig. 2.34 fo r

Figure 1 3 3

Direct form I structure of the system described by (2.5.6).

114

Discrete-Time Signals and Systems

w(n)

b0

Chap. 2

N
y(n )

-1
UJ{/1 - 1)
J
-1

Figure 2 3 4

w(n - 2)

b2

w(n - 3)

by

Direct form II structure for the system described by (2.5.6).

N > M. T h is stru c tu re is the cascad e o f a recu rsiv e system

N

w( n) = ^

ai W(n - k ) -f x( n)

(2.5.9)

*=1
fo llo w ed by a n o n re c u rsiv e sy stem
M
v(n) =

vu(n - k )

(2.5.10)

W e o b serv e th a t if N > M . this s tru c tu re re q u ire s a n u m b e r o f d elay s e q u a l to

th e o rd e r N o f th e sy stem . H o w e v e r, if M > N , th e re q u ire d m e m o ry is specified
by M . F ig u re 2.34 can easily by m o d ified to h a n d le th is case. T h u s th e d ire c t form
II stru c tu re re q u ire s M + N + 1 m u ltip lic a tio n s a n d m ax{M , jV} d elay s. B e cau se it
re q u ire s th e m in im u m n u m b e r o f d elay s fo r th e re a liz a tio n o f th e system d e sc rib e d
by (2.5.6), it is so m e tim es ca lle d a canoni c f o r m .

Sec. 2.5

Implementation of Discrete-Time Systems

115

A special case o f (2.5.6) occurs if w e set th e system p a ra m e te rs ak 0.

it = 1.........N. T h e n th e in p u t-o u tp u t re la tio n sh ip fo r th e sy stem re d u c e s to
M
v(n) =

- k)

(2.5.11)

k=0
w hich is a n o n re c u rsiv e lin e a r tim e -in v a ria n t system . T his sy stem view s only the
m o st re c e n t M + 1 in p u t signal sa m p les a n d , p rio r to a d d itio n , w eights each sam ple
by th e a p p ro p ria te co efficien t bk from th e set {i^}. In o th e r w ords, th e system
o u tp u t is b asically a weight ed m o v i n g average of th e in p u t signal. F o r this reaso n
it is so m e tim e s called a m o v i n g average ( M A ) syst em. S uch a system is an F IR
sy stem w ith an im p u lse re sp o n s e h (k ) e q u a l to th e coefficients bk, th at is.

* '* ) = { o ! '

o to rw l"

'5 1 2

If w e r e tu rn to (2.5.6) a n d se t M = 0, th e g e n eral lin e a r tim e -in v a ria n t system

red u ces to a p u re ly re c u rsiv e system d escrib ed by th e d ifferen ce e q u a tio n
N
y ( n ) = ~ Y a O (n ~ k) + bux(n)
i =1

(2.5.13)

In th is case th e sy stem o u tp u t is a w eig h ted lin ear c o m b in a tio n o f N past o u tp u ts

a n d th e p re s e n t in p u t.
L in e a r tim e -in v a ria n t system s d esc rib e d by a se c o n d -o rd e r d ifferen ce e q u a
tio n a re an im p o rta n t subclass o f th e m o re g e n eral system s d e sc rib e d by (2.5.6)
o r (2.5.10) o r (2.5.13). T h e re a so n fo r th e ir im p o rta n c e will be ex p lain ed later
w h en w e discuss q u a n tiz a tio n effects. Suffice to say at this p o in t th a t se c o n d -o rd e r
sy stem s a re u su ally u sed as b a sic b u ild in g b lo ck s fo r realizin g h ig h e r-o rd e r system s.
T h e m o st g e n e ra l se c o n d -o rd e r sy stem is d escrib ed by th e d ifferen ce e q u a tio n
y (n ) =

- a[v( n - 1) - a2v(n - 2) 4- b(,x(n)

(2.5.14)
+ b\ x ( n 1) +- b 2x ( n 2)

w hich is o b ta in e d fro m (2.5.6) b y se ttin g N = 2 an d M = 2. T h e d irect form II

s tru c tu re fo r realizin g th is sy stem is sh o w n in Fig. 2.35a. If we se t a\ =
= 0.
th e n (2.5.14) re d u c e s to
y( n) = box(n) + b\x(rt 1) + b 2x ( n - 2)

(2.5.15)

w hich is a sp ecial case o f th e F IR system d esc rib e d by (2.5.11). T h e stru c tu re

fo r realizin g th is sy stem is sh o w n in Fig. 2.35b. F inally, if w e set b\ = bz = 0
in (2.5.14), w e o b ta in th e p u re ly recu rsiv e se c o n d -o rd e r sy stem d esc rib e d by th e
d iffe re n c e e q u a tio n
y( n) = a \ y ( n 1) a 2y(n - 2) 4- box(rt)

(2.5.16)

w hich is a sp e cial case o f (2.5.13). T h e stru c tu re fo r realizin g th is system is show n

in Fig. 2.35c.

Discrete-Time Signals and Systems

116

Chap. 2

(a)

----- -0

Y[n)

(b)

x i n)

-^-1 H

02

I _a'
-1

-1
<c)

Figure 2.35 Structures for the realization of second-order systems: (a) general
second-order system; (b) FIR system; (c) purely recursive system

2.5.2 Recursive and Nonrecursive Realizations of FIR

Systems
W e h a v e a lread y m a d e th e d istin ctio n b etw e e n F IR an d IIR system s, b ased on
w h e th e r th e im p u lse re sp o n se h( n) o f th e system h a s a finite d u ra tio n , o r an infi
n ite d u ra tio n . W e h av e also m a d e th e d istin c tio n b e tw e e n rec u rsiv e a n d n o n re c u r
sive system s. B asically, a causal recu rsiv e system is d esc rib e d by an in p u t-o u tp u t
e q u a tio n o f th e fo rm
y(n) = F[v(^i 1).........y{n N ) , x ( n ) , ------x ( n - M)]

(2.5.17)

an d fo r a lin ear tim e -in v a ria n t system specifically, by th e d iffe re n c e e q u a tio n

N

y { n ) = - Y ak>(n ~ k ) + Y , bkX(n ~ k )
*=i
*=o

(2.5.18)

Sec. 2.5

Implementation of Discrete-Time Systems

117

O n th e o th e r h a n d , cau sal n o n recu rsiv e system s d o n o t d e p e n d o n p a st values of

th e o u tp u t a n d h e n c e a re d escrib ed by an in p u t-o u tp u t e q u a tio n o f th e form
y (n) = F [x (n ), jc(n - 1).........x ( n Mj ]

(2.5.19)

a n d fo r iin e a r tim e -in v a ria n t system s specifically, by th e d iffe re n c e e q u a tio n in

(2.5.18) w ith ak: = 0 fo r k = 1, 2 , . . . , N.
In th e case o f F IR system s, we h av e a lre a d y o b se rv e d th a t it is aiw ays possible
to re a liz e such sy stem s n o n recu rsiv ely . In fact, w ith at. = 0, k = 1, 2 , . . . , N , in
(2.5.18), w e h av e a sy stem w ith an in p u t-o u tp u t e q u a tio n
v() = ] p 6 * ;t( n ~

(2.5.20)

T h is is a n o n re c u rsiv e a n d F IR system . A s in d ic a te d in (2.5.12), th e im pulse

re sp o n s e o f th e sy stem is sim ply e q u a l to th e coefficients {&*). H e n c e every F IR
sy stem can b e re a liz e d n o n recu rsiv ely . O n th e o th e r h a n d , an y F IR system can
also b e realized recu rsiv ely. A lth o u g h the g e n eral p ro o f o f th is s ta te m e n t is given
la te r, w e shall give a sim ple ex am p le to illu stra te th e p o in t.
S u p p o se th a t w e h av e an F IR system o f th e form
1
m
y( n) = M + If-'

- k)

(2.5.21)

fo r co m p u tin g th e mo v i n g average of a signal jc(h). C learly , th is sy stem is F IR w ith

im p u lse re sp o n se
h( n) ~

0 < n < M

M + 1

F ig u re 2.36 illu stra te s th e stru c tu re of th e n o n re c u rsiv e re a liz a tio n o f th e system .

N ow , su p p o se th a t w e ex p ress (2.5.21) as
1
M
y (n ) = --------- x ( n 1 k)
M
m +
^ 1 ^Jt=0
+

. [x(n) ~ x ( n - 1 - M )]
M + 1

y(n - 1) +

Figure 2 3 6

1
M + V

[x(n) x (n 1 Af)]

N onrecursive realization of an FIR moving average system.

(2.5.22)

118

Discrete-Time Signals and Systems

Chap. 2

N ow , (2.5.22) re p re se n ts a recu rsiv e re a liz a tio n of th e F IR system . T h e stru c tu re

o f th is recu rsiv e re a liz a tio n of th e m oving av erag e system is illu stra te d in Fig. 2.37.
In su m m ary , w e can th in k of th e te rm s F IR a n d IIR as g e n e ra l ch aracteristics
th a t distin g u ish a ty p e of lin e a r tim e -in v a ria n t system , and of th e term s recursive
an d nonrecursi ve as d e sc rip tio n s of th e stru c tu re s fo r realizin g o r im p lem en tin g
th e system .

vt/> - 1)
Figure 2 3 7

Recursive realization of an FIR moving averapc svstem.

2.6 CORRELATION OF DISCRETE-TIME SIGNALS

A m a th em atical o p e ra tio n th a t closely rese m b le s co n v o lu tio n is c o rre la tio n . Just
as in th e case o f c o n v o lu tio n , tw o signal se q u e n c e s are in v o lv ed in c o rre la tio n .
In c o n tra st to c o n v o lu tio n , h o w ev er, o u r o bjective in co m p u tin g th e c o rre la tio n
b etw een th e tw o signals is to m e a su re th e d e g re e to w hich th e tw o signals are
sim ilar an d th u s to e x tra c t so m e in fo rm a tio n th a t d e p e n d s to a large e x te n t on
th e ap p licatio n . C o rre la tio n o f signals is o ften e n c o u n te re d in ra d a r, so n a r, digital
co m m u n icatio n s, geolo g y, an d o th e r a re a s in science an d en g in eerin g .
T o b e specific, le t us su p p o se th a t we have tw o signal se q u e n c e s x ( n ) an d
y( n) th a t we w ish to co m p a re . In r a d a r an d active so n a r a p p lic a tio n s. x( n ) can
re p re s e n t th e sa m p le d v ersio n o f th e tra n s m itte d signal and y{n) can re p re s e n t th e
sam p led v ersio n o f th e receiv ed signal at th e o u tp u t o f the a n a lo g -to -d ig ita l (A /D )
c o n v erter. If a ta rg e t is p re se n t in th e space b e in g se a rc h e d by th e ra d a r o r so n a r,
th e receiv ed signal y( n) consists of a d e la y e d v ersio n o f th e tra n sm itte d signal,
reflec te d from th e ta rg e t, an d c o rru p te d b y a d d itiv e noise. F ig u re 2.38 d e p ic ts the
ra d a r signal re c e p tio n p ro b le m .
W e can re p re s e n t th e re c e iv e d signal se q u e n c e as
y ( n ) = a x ( n D) + w( n )

(2.6.1)

w h ere a is som e a tte n u a tio n fa c to r re p re s e n tin g th e signal loss involved in th e

ro u n d -trip tran sm issio n o f th e signal x (n ), D is th e ro u n d -trip d elay , w hich is

Sec. 2.6

Correlation of Discrete-Time Signals

119

assum ed to be an integer m ultiple o f the sam pling interval, and w (n) represents
the additive n oise that is picked up by the antenna and any n oise generated by the
electron ic com p onents and amplifiers contained in the front end o f the receiver.
O n the other hand, if there is no target in the space searched by the radar and
sonar, the received signal y(n ) consists o f noise alone.
H avin g the tw o signal sequ en ces, x ( n ) , which is called the reference signal or
transm itted signal, and y ( n ) , the received signal, the problem in radar and sonar
d etection is to com p are y(n) and x ( n ) to determ ine if a target is present and, if
so, to determ in e the tim e d elay D and com p ute the distance to the target. In
practice, the signal x ( n D) is h eavily corrupted by the additive n oise to the point
w here a visual inspection o f y ( n ) d oes n ot reveal the p resen ce or absence o f the
desired signal reflected from the target. Correlation provides us with a m eans for
extracting this im portant inform ation from y( n).
D igital com m unications is an oth er area w here correlation is o ften used. In
digital com m unications the inform ation to be transm itted from on e p oin t to an
other is usually con verted to binary from , that is, a seq u en ce o f zeros and ones,
which are then transm itted to the in tend ed receiver. T o transm it a 0 w e can trans
m it the signal seq u en ce xo(n) for 0 < n < L 1, and to transm it a 1 w e can transmit
the signal seq u en ce jti(n) for 0 < n < L 1, w here L is som e integer that d en otes
the num ber o f sam p les in each o f the tw o sequ en ces. V ery often , x\ (n) is selected
to be th e negative o f xo(n). T h e signal received by the in tend ed receiver m ay be
represented as
y (n ) = x,-(n) + w (n )
* = 0 ,1
0 < n < L 1
(2.6.2)
w here now the uncertainty is w hether x 0(n) or *](n ) is the signal com p on en t in
>(n), and w (n ) rep resents the additive n oise and other interferen ce inherent in

120

Discrete-Time Signals and Systems

Chap. 2

any co m m u n icatio n system . A g ain , such noise has its o rig in in th e e le ctro n ic
c o m p o n e n ts c o n ta in e d in th e fro n t e n d o f th e receiv er. In an y case, th e receiv er
k n o w s th e p o ssib le tra n sm itte d se q u e n c e s xo(n) a n d
(n) a n d is fa c e d w ith the task
o f co m p a rin g th e receiv ed signal y( n) w ith b o th xo(n) a n d Jti(n) to d e te rm in e w hich
o f th e tw o signals b e tte r m a tc h e s y(n). T h is c o m p a riso n p ro c e ss is p e rfo rm e d by
m ean s o f th e c o rre la tio n o p e ra tio n d e sc rib e d in th e follow ing su b se c tio n .

2.6.1 Crosscorrelation and Autocorrelation Sequences

S u p p o se th a t we h av e tw o real signal se q u e n c e s x ( n ) an d y ( n ) ea c h o f w hich has
finite en erg y . T h e crosscorrelation o f x ( n ) and v(n) is a se q u e n c e rxy(l), w hich is
d efin ed as
r.tv(l)

~ 0

l = 0. 1 , 2 , . . .

(2.6.3)

riy(t) Y , X ^n + 0 y ( )

1 = 0, 1 , 2 , . . .

(2.6.4)

n ~ x

or, eq u iv alen tly , as

OC

n= -oc

T h e in d ex I is th e (tim e ) sh ift (o r lag) p a r a m e te r an d th e su b scrip ts x y on th e c ro ss

c o rre la tio n se q u e n c e rxy(l) in d icate the se q u e n c e s b ein g c o rre la te d . T h e o rd e r of
th e su b scrip ts, w ith .x p re c e d in g y, in d ic a te s th e d ire c tio n in w h ich o n e se q u en ce
is sh ifted , re lativ e to th e o th e r. T o e la b o ra te , in (2.6.3), th e se q u e n c e x ( n ) is left
u n sh ifte d an d y (n ) is sh ifted by / u n its in tim e, to th e rig h t fo r / p o sitiv e a n d to
th e left for I n eg ativ e. E q u iv a le n tly , in (2.6.4), th e se q u en ce y ( ) is left u n sh ifted
a n d x{n) is sh ifted by I u n its in tim e, to th e left fo r / p o sitiv e a n d to th e rig h t fo r
/ n eg ativ e. B u t sh iftin g x (n ) to th e left by / u n its relativ e to y (n ) is e q u iv a le n t
to sh iftin g y() to th e rig h t by / u n its re lativ e to x ( n) . H e n c e th e c o m p u ta tio n s
(2.6.3) an d (2.6.4) yield id en tical c ro ssc o rre la tio n se q u en ces.
If w e rev erse th e ro les o f jr() an d y (n) in (2.6.3) an d (2.6.4) a n d th e re fo re
re v e rse th e o rd e r o f th e indices xy. we o b ta in th e c ro ss c o rre la tio n se q u e n c e
OC

ryx(I) =

y ( n ) x ( n I)

(2.6.5)

y ( n + l ) x ( n)

(2.6.6)

n=0C
o r, e q u iv alen tly ,
OC

ryx(l) = Y
n d c

By c o m p a rin g (2.6.3) w ith (2.6.6) o r (2.6.4) w ith (2.6.5), we c o n c lu d e th a t

rxy(l) = ryx( ~l )

(2.6.7)

T h e re fo re , r y i (l) is sim ply th e fo ld ed v ersio n o f rxy(l), w h e re th e fo ld in g is d o n e

w ith re sp e c t to / = 0. H e n c e , ryx(l) p ro v id e s exactly th e sam e in fo rm a tio n as rxv(l),
w ith re sp e c t to th e sim ilarity o f x ( n ) to y(n ).

Sec. 2.6

Correlation of Discrete-Time Signals

121

Exam ple 2.6.1

D eterm ine the crosscorrelation sequence rxv(l) of the sequences
x(n) = { ....0 .0 .2 . 1 .3 .7 .1 .2 . 3. 0. 0 ....}
t

v(n) = { . . . . 0 . 0 . 1 . - 1 . 2 . - 2 . 4 . 1 . - 2 . 5 .0 .0 ,...]
t

Solution

.(/). For I = 0 w e have

Let us use the definition in (2.6.3) to compute

r rv(0) = Y

x(ri)v(n)

The product sequence u()(n) = x ( n ) y ( r ) is

u(n) = { ..., 0. 0. 2. 1. 6. -1 4 . 4. 2, 6, 0, 0. . . .)
t

and hence the sum over all values of n is

r,v(0) = 7
For I > 0, we simpiy shift v() to the right relaLive to a ' ( h ) hy / units, compute
the product sequence v/(n) = jr(n)_v(n I), and finally, sum o v er all va lu es o f the
product sequence. Thus we obtain
r t ,( l) = 13,

rJV(2) = -1 8 .

r vv(3) = 16.

r,,(4 ) = - 7

rM5) = 5.

r ,v(6) = - 3 ,

rxy (/) = 0.

I >1

For / < 0, we shift y(n) to the left relative to jr(n) by / units, compute the product
sequence v/(n) = j(n ) v(n I), and sum over all values of the product sequence. Thus
we obtain the values of the crosscorrelation sequence
rIV(1) = 0,

riy( - 2 ) = 33.

rly{ - 3) = -1 4 .

rTV( - 4 ) = 36

rJV( -5 ) = 19,

fxv(-6) = - 9 ,

rIV( - 7) = 10,

rxv(l) = 0, I < - 8

Therefore, the crosscorrelation sequence of x{n) and y(n) is

r,AD = (10, - 9 ,1 9 , 36, -1 4 , 33,0, 7,13, -1 8 ,1 6 . - 7 , 5, - 3 )
t

T h e sim ilarities b e tw e e n th e c o m p u ta tio n o f th e c ro ss c o rre la tio n o f tw o se

q u e n c e s a n d th e c o n v o lu tio n o f tw o se q u e n c e s is a p p a re n t. In th e c o m p u ta tio n of
c o n v o lu tio n , o n e o f th e se q u e n c e s is fo ld ed , th e n sh ifted , th e n m u ltip lie d by th e
o th e r se q u e n c e to fo rm th e p ro d u c t se q u e n c e fo r th a t shift, a n d finally, th e values
o f th e p r o d u c t se q u e n c e a re su m m ed . E x c e p t fo r th e fo ld in g o p e ra tio n , th e co m
p u ta tio n o f th e c ro ss c o rre la tio n se q u e n c e involves th e sa m e o p e ra tio n s: shifting
o n e o f th e se q u e n c e s , m u ltip lic atio n o f th e tw o se q u e n c e s, an d su m m in g o v er all
v alu es o f th e p r o d u c t se q u e n c e . C o n se q u e n tly , if w e h av e a c o m p u te r p ro g ra m
th a t p e rfo rm s c o n v o lu tio n , w e can use it to p e rfo rm c ro ss c o rre la tio n by p ro v id in g

122

Discrete-Time Signals and Systems

Chap. 2

as in p u ts to th e p ro g ra m , th e se q u e n c e jc() an d th e fo ld ed se q u e n c e y ( n). T h e n
th e c o n v o lu tio n o f x ( n ) w ith y ( n) yields th e c ro ss c o rre la tio n r rv(/). th a t is,
rxv(D = x ( l ) * y ( - l )

(2.6.8)

In th e special case w h ere y( n) = x( n ) , we h av e th e aut ocorrel ati on o f *( ),

w hich is d efined as th e se q u e n c e

OC

rXx(l)=

x(n)j:(n - 0

(2.6.9)

ft ~ OC

o r, eq u iv alen tly , as

OO
rxx{l) =

^ 2 , x (n +

(2.6.10)

n= oc

In d ealin g w ith fin ite -d u ra tio n se q u e n c e s, it is cu sto m a ry to ex p ress th e a u to

c o rre la tio n an d c ro ss c o rre la tio n in te rm s of th e finite lim its on th e su m m a tio n . In
p a rtic u la r, if x ( ) a n d v(n) a re causal se q u e n c e s o f le n g th N [i.e., .v(n) = y(n) = 0
for n < 0 an d n > N], th e c ro ssc o rre la tio n an d a u to c o rre la tio n se q u e n c e s m ay be
ex p ressed as
rxy(l) =

x(n)y(n-l)

(2.6.11)

an d
A'-1*1-1

rxx( l ) =

(2.6.12)

n=f
w h ere i = I, k = 0 fo r / > 0, a n d / = 0, k = I fo r / < 0.

2.6.2 Properties of the Autocorrelation and

Crosscorrelation Sequences
T h e a u to c o rre la tio n an d c ro ssc o rre la tio n se q u e n c e s h av e a n u m b e r of im p o rta n t
p ro p e rtie s th a t w e n o w p re se n t. T o d e v e lo p th e s e p ro p e rtie s , le t us assu m e th a t
we h av e tw o se q u e n c e s x ( n ) a n d y (n) w ith finite en erg y from w hich w e fo rm the
lin e a r c o m b in a tio n ,
ax(n)

bv( n I)

w h ere a an d b a re a rb itra ry c o n s ta n ts a n d I is so m e tim e shift. T h e en erg y in this

signal is

OC

Y
fj -oc

OC

OC

[ax(n) + by( n - I)]2 = a 2 ^

x 2( n ) + b 2 ^

n = oc

y 2{n - I)

/i oc

-j l ^V""' x (in )\ y (tn I)

n
+. 2ab
n = oc

= a 2rxx(0) + b2r yy( 0) + l a b r xy{l)

(2 '6 -13)

Sec. 2.6

123

Correlation of Discrete-Time Signals

F irst, we n o te th a t rx x (0) = E x a n d /-vy(0) = v, w hich a re th e en e rg ie s o f

x{n) an d y (n ), resp ectiv ely . It is o b v io u s th a t
a 2rxx(0) -I- b 2r y>.(0) + 2abrxy(l) > 0

(2.6.14)

N ow , assu m in g th a t b ^ 0, w e can divide (2.6.14) by b 2 to o b ta in

r (0 )

(^)2

+ 2rxy(l) Q

+ r v_v(0) > 0

W e view th is e q u a tio n as a q u a d ra tic w ith coefficients r XJ(0), 2rxv(l), a n d ^ ,( 0 ) .

Since th e q u a d ra tic is n o n n e g a tiv e , it follow s th a t th e d isc rim in a n t of this q u a d ra tic
m u st b e n o n p o sitiv e , th a t is,
4 [ r ; v(/) - r , , ( 0 K v(0)] < 0
T h e re fo re , th e c ro ss c o rre la tio n se q u e n c e satisfies th e c o n d itio n th a t
|r,v(/>| < y r ( 0 ) r , v(0) =

(2.6.15)

In th e sp ecial case w h ere v(n) = x ( n ), (2.6.15) re d u c e s to

\rxx( D \ < r xx(0) = E x

(2.6.16)

T his m ean s th a t th e a u to c o rre la tio n se q u e n c e o f a signal a tta in s its m ax im u m value

at z e ro lag. T h is re su lt is c o n siste n t w ith th e n o tio n th a t a sig n a l m a tc h e s p erfe ctly
w ith itself at z e ro shift. In th e case o f th e c ro ssc o rre la tio n se q u en ce, th e u p p e r
b o u n d o n its v alu es is given in (2.6.15).
N o te th a t if an y o n e o r b o th o f th e signals involved in th e c ro ssc o rre la tio n
a re scaled , th e sh a p e o f th e c ro ss c o rre la tio n se q u e n c e d o e s n o t ch an g e, only the
a m p litu d e s o f th e c ro ss c o rre la tio n se q u e n c e a re sc aled accordingly. S ince scaling
is u n im p o rta n t, it is o ften d e s ira b le , in p ra c tic e , to n o rm a liz e th e a u to c o rre la tio n
a n d c ro ss c o rre la tio n se q u e n c e s to th e ran g e fro m - 1 to 1. In th e case o f the
a u to c o rre la tio n se q u e n c e , w e can sim ply d iv id e by ^ ( 0 ) . T h u s th e n o rm aliz ed
a u to c o rre la tio n se q u e n c e is d efin ed as
P. A D =

rixiO)

(2-6 -17)

Sim ilarly, we d efin e th e n o rm a liz e d c ro ssc o rre la tio n se q u e n c e

pXY(l) =

r ' v(l)
:
v/ r xx(0 )rvv(0)

(2.6.18)

N ow \pXI{l)\ < 1 a n d |/oXv(0! < 1, a n d h e n c e th ese se q u e n c e s a re in d e p e n d e n t of

signal scaling.
F in ally , as we h av e a lre a d y d e m o n s tra te d , th e c ro ss c o rre la tio n se q u e n c e sa t
isfies th e p ro p e rty
r Xy ( l ) = f y x ( - 0

Discrete-Time Signals and Systems

124

Chap. 2

W ith y(n) = x ( n) , th is re la tio n resu lts in th e follow ing im p o rta n t p ro p e rty fo r the
a u to c o rre la tio n se q u en ce
(2.6.19)

r { l ) = rx s ( . - l )

H e n c e th e a u to c o rre la tio n fu n ctio n is an ev en fu n ctio n . C o n s e q u e n tly , it suffices

to co m p u te rx x (l) fo r / > 0.
Example 2.6.2
Compute the autocorrelation of the signal
x(n) = a"u(n), 0 < a < 1
Solution Since x (n) is an infinite-duration signal, its autocorrelation also has infinite
duration. We distinguish two cases.
If I > 0. from Fig. 2.39 we observe that
r t J (/) =

x ( n ) x ( n - /) =

' = a

n= l

n =i

1
n~l

Since a < 1, the infinite series coti crges and we obtain

r tl (/) ~

I> 0

^ -V
1 a-

For / < 0 we have

=
n=(l

x(n)x(/i /) = a -1'S~'(rr)" = ------- -ti~!

I - a-

/ < 0

n=(l

But when / is negative, c r 1

into the following expression:

a ' 1'.

Thus the two relations for

rx,(!) =

,,a ,t:
oc < / < oc
1 a~
The sequence rxx(l) is shown in Fig. 2.42(d). We observe that

r i t (!)

can be combined

(2.6.20)

r ( ~ / | = rxAD
and

rlt(0) =

1 a2

Therefore, the normalized autocorrelation sequence is

r (/)
p s,(!) - cr|,:
~ oc < I < oc
rxxW)

(2.6.21)

2.6.3 Correlation of Periodic Sequences

In S ectio n 2.6.1 w e d efin ed th e c ro ss c o rre la tio n an d a u to c o r re la tio n se q u en ces of
en erg y signals. In this se ctio n we co n sid e r th e c o rre la tio n se q u e n c e s o f p o w er
signals an d , in p a rtic u la r, p e rio d ic signals.
L et x( n ) a n d y(rc) b e tw o p o w e r signals. T h e ir c ro ss c o rre la tio n se q u e n c e is
d efin ed as
1
M
rx \ 0 )

lim

----- - Y ]

M -oc 2 M + 1

x(n)y(n~l)

( 2 .6 .2 2 )

Sec. 2.6

125

Correlation of Discrete-Time Signals

xi n)

) i'

in .

-2-10123
(a)

xin-I)
/ >o

o
(b)

x(n - I )

l< 0

(c)

r(!) =

- 2 - 1 0
Figure 139

1 2
(d)

' , a1'1
1 - a2

Compulation of the autocorrelation o f the signal xin)

a",

0 < a < 1.
If x ( n ) = y(tt), we have the definition o f the autocorrelation seq u en ce of a
p ow er signal as
1
M
rxx(I) = iim . . . , 1 Y ] x ( n ) x ( n - l )
(2.6.23)
Af-oo 2M + 1
In particular, if x ( n ) and y ( n ) are two periodic seq u en ces, each with period
the averages indicated in (2.6.22) and (2.6.23) o ver the infinite interval, are identical

126

D iscrete-Time Signals and Systems

Chap. 2

to th e av erag es o v er a single p e rio d , so th a t (2.6.22) an d (2.6.23) re d u c e to

(2.6.24)
an d
(2.6.25)
It is c lear th a t r ry(l) an d rxx(l) are p e rio d ic c o rre la tio n
T h e fa c to r 1 / N can b e v iew ed as a n o rm a liz a tio n scale
In som e p ractical ap p licatio n s, c o rre la tio n is u se d
an o b se rv e d physical signal w hich m ay be c o rru p te d by
ex am p le, c o n sid er a signal se q u e n c e y( n) o f th e form

se q u e n c e s w ith p e rio d N .
facto r.
to id en tify p erio d icitie s in
ra n d o m in te rfe re n c e . F o r

y{n) = * (n ) + w(n)

(2.6.26)

w h ere jc(/i) is a p erio d ic se q u e n c e o f so m e u n k n o w n p e rio d N a n d w( n) re p re se n ts

an ad d itiv e ran d o m in te rfe re n c e . S u p p o se th a t w e o b se rv e M sa m p le s o f y(n ), say
0 < n < M 1, w h ere M > > N. F o r all p ractical p u rp o se s, w e can assum e th a t
y( n) 0 fo r n < 0 an d n > M. N ow th e a u to c o rre la tio n se q u e n c e of y(n), using
th e n o rm aliz atio n facto r o f \ / M . is
(2.6.27)
If we su b stitu te for y(n) fro m (2.6.26) in to (2.6.27) w e o b ta in
__1
ry.T(/) =
j M- 1
Y ] x ( n ) x i n - /)

M
j

M J

H - y ^ [ .r ( n ) itj (/2 /) + w{ n) x{n /)]

M

(2.6.28)

T h e first fa c to r on th e rig h t-h a n d sid e of (2.6.28) is th e a u to c o rre la tio n se

q u e n c e o f x i n) . S ince x( n) is p e rio d ic , its a u to c o rre la tio n s e q u e n c e ex h ib its th e
sam e p erio d icity , th u s co n ta in in g relativ ely larg e p e a k s at / = 0, N , 2N , a n d so
on. H o w ev er, as th e shift I a p p ro a c h e s M , th e p e a k s a re re d u c e d in a m p litu d e
d u e to th e fact th a t w e h av e a finite d a ta re c o rd o f M sa m p le s so th a t m any o f th e
p ro d u c ts ;t(n)j:(rt /) a re zero . C o n s e q u e n tly , w e sh o u ld av o id c o m p u tin g r VT(/)
fo r larg e lags, say, I > M f l .

Sec. 2.6

Correlation of Discrete-Time Signals

127

T h e c ro ss c o rre la tio n s rxu,(/) an d rwx(l) b etw e e n th e signal

a n d th e a d
ditiv e ra n d o m in te rfe re n c e a re e x p ected to be relativ ely sm all as a resu lt of the
e x p e c ta tio n th a t ;r(/i) a n d w ( n ) will be to ta lly u n re la te d . F in ally , the last term on
th e rig h t-h a n d sid e of (2.6.28) is th e a u to c o rre la tio n se q u e n c e o f th e ra n d o m se
q u e n c e w( n) . T h is c o rre la tio n se q u en ce will c e rta in ly c o n tain a p e a k a t / = 0, but
b ecau se o f its r a n d o m ch aracteristics, r ww(l) is ex p e c te d to d ecay rap id ly to w ard
zero . C o n s e q u e n tly , o nly rxx{l) is e x p e c te d to h av e large p e a k s fo r / > 0. T his
b e h a v io r allow s us to d e te c t th e p re se n c e of th e p e rio d ic signal a (/ j ) b u rie d in the
in te rfe re n c e u>(n) a n d to id e n tify its p e rio d .
A n e x am p le th a t illu stra te s th e use of a u to c o rre la tio n to identify a h id d e n
p e rio d ic ity in an o b se rv e d physical signal is show n in Fig. 2.40. T h is figure illus
tra te s th e a u to c o rre la tio n (n o rm a liz e d ) se q u e n c e fo r th e W o lfe r su n sp o t n u m b e rs
fo r 0 < / < 20, w h ere an y v alu e o f / c o rre sp o n d s to o n e y ear. T h e se n u m b e rs are
given in T a b le 2.2 fo r th e 100-year p e rio d 1770-1869. T h e re is c lear ev id en ce in
th is fig u re th a t a p e rio d ic tre n d exists, w ith a p e rio d o f 10 to 11 years.
Example 2.6.3
Suppose that a signal sequence jr(n) = sin(7r/5)/i, for 0 < n < 99 is corrupted by
an additive noise sequence Kn), where the values of the additive noise are selected
independently from sample to sample, from a uniform distribution over the range
TABLE 2.2
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794

YEARLY WOLFER SUNSPOT NUMBERS

101
82
66
35
31
7
20
92
154
125
85
68
38
23
10
24
83
132
131
118
90
67
60
47
41

1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819

21
16
6
4
7
14
34
45
43
48
42
28
10
8
2
0
1
5
12
14
35
46
41
30
24

1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844

16
7
4
2
8
17
36
50
62
67
71
48
28
8
13
57
122
138
103
86
63
37
24
11
15

1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869

40
62
9X
124
96
66
64
54
39
21
7

4
23
55
94
96
77
59
44
47
30
16
7
37
74

128

Discrete-Time Signals and Systems

Chap. 2

Year

(a)

Figure 2.40 Identification of periodicity in the Wolfer sunspot numbers: (a) an

nual Wolfer sunspot numbers; (b) autocorrelation sequence.

(A /2, A /2), where A is a param eter of the distribution. The observed sequence is
y(n) = x(n) + w(n). D eterm ine the autocorrelation sequence rvt(n) and thus determine
the period of the signal x(rt).
Solution The assumption is that the signal sequence x(n) has some unknown period
that we are attem pting to determ ine from the noise-corrupted observations {y(n)).
Although x(n) is periodic with period 10, we have only a ftmte-duration sequence of

Sec. 2.6

129

Correlation of Discrete-Time Signals

length M = 100 [i.e.. 10 periods of jcm )]. The noise power level P in the sequence
u'(n) is determ ined by the param eter A. We simply state that Pu = A: /12. The signal
power level is P, = I. Therefore, the signal-to-noise ratio (SNR) is defined as

P,

~P~U ~ A : /1 2 A :

Usually, the SNR is expressed on a logarithmic scale in decibels (dB) as 101ogui

( PJPv) .
Figure 2.41 illustrates a sample of a noise sequence w(n), and the observed
sequence y(n) x(n) + it(n) when the SNR = 1 dB. The autocorrelation sequence
'

J '
t
I l l
Lt ttIiIt. Tllrii!
ill ..tI
1. I i i h i
1
p p T T J ljw ^
IT
1 i ([Ijj-liii i|
4
* 4
i
j

la]

(c)
Figure 2.41

noise.

Use of autocorrelation to detect the presence of a periodic signal corrupted by

Discrete-Tim e Signals and Systems

130

L T TTTT. Tt TJ IT.TTUTtTT ._L T .T. I .T.

pp V|Vvli

w(n)

'

Chap. 2

ttT .1.

(a)

(b)

rM .

r![It
xij|il

rl It

i o1

r!lfit

1
tTT
1 [1* ?

(c)
Figure 2.42 Use of autocorrelation to detect the presence of a periodic signal
corrupted by noise.

r vv(/) is illustrated in Fig. 2.41c. W e observe that the periodic signal *(/?), embedded
in y(n), results in a periodic autocorrelation function rZI(l) with period N 10. The
effect of the additive noise is to add to the peak value at / = 0. but for I ^ 0, the
correlation sequence rwu,(l) = ^ 0 a s a result of the fact that values of w(ri) were gen
erated independently. Such noise is usually called white noise. The presence of this
noise explains the reason for the large peak at I = 0. The smaller, nearly equal peaks
at I = 10, 2 0 ,... are due the periodic characteristics of x(n).
Figure 2.42 illustrates the noise sequence w(n), the noise-corrupted signal y(n),
and the autocorrelation sequence r vv(/) for the same signal, within which is embedded
a signal at a smaller noise level. In this case, the SNR = 5 dB. E ven with this relatively
small noise level, the periodicity of the signal is not easily determ ined from observa
tion of y( n). However, it is clearly evident from observation of the autocorrelation
sequence ryy(n).

2.6.4 Computation of Correlation Sequences

A s indicated on Section 2.6.1, the procedure for com puting the crosscorrelation
seq u en ce b etw een x ( n ) and y ( n ) in volves shifting on e o f the seq u en ces, say x(n),

Sec. 2.6

Correlation of Discrete-Time Signals

131

to o b ta in x( n - /), m u ltip ly in g th e sh ifted se q u e n c e by v(n) to o b ta in th e p r o d

u ct se q u e n c e y ( n ) x ( n - /), an d th en su m m in g all th e v alu es o f th e p ro d u c t se
q u en ce to o b ta in r yx(l). T his p ro c e d u re is r e p e a te d fo r d iffe re n t values o f th e
lag /. E x c e p t fo r th e fo ld in g o p e ra tio n th a t is involved in c o n v o lu tio n , th ese b a
sic o p e ra tio n s fo r co m p u tin g th e c o rre la tio n se q u e n c e a re id e n tic a l to th o se in
co n v o lu tio n .
T h e p ro c e d u re fo r co m p u tin g th e co n v o lu tio n is directly a p p licab le to c o m
p u tin g th e c o rre la tio n o f tw o se q u en ces. S pecifically, if w e fo ld y (n) to o b ta in
y ( n), th e n th e c o n v o lu tio n o f x ( n ) w ith v ( n) is id en tical to th e cro ssc o rre la tio n
o f x ( n ) w ith y(n). T h a t is.
rxy(l) = x(rt) * y ( n ) |n=/

(2.6.29)

A s a c o n se q u e n c e , th e c o m p u ta tio n a l p ro c e d u re d escrib ed for c o n v o lu tio n can be

a p p lied d irectly to th e c o m p u ta tio n o f th e c o rre la tio n se q u en ce.
W e no w d e sc rib e an a lg o rith m th a t can b e easily p ro g ra m m e d to c o m p u te
th e c ro ss c o rre la tio n se q u e n c e of tw o fin ite -d u ra tio n signals * ( ), 0 < n < N ~ I,
an d y ( n) , 0 < n < M 1.
T h e alg o rith m c o m p u te s rJV(/) fo r p o sitiv e lags. A cc o rd in g to th e re la tio n
rxy(l) = r YX(I), th e v alu es o f rXY(l) fo r n eg ativ e lags can be o b ta in e d by using th e
sam e a lg o rith m fo r p o sitiv e lags, an d in te rc h a n g in g th e roles o f jt( ) an d v(n). W e
o b se rv e th a t if M < N, rXY{i) can be c o m p u te d by th e re la tio n s
M - \ + l

Y ' , x ( n ) y ( n /),

0 < 1 < N M
(2.6.30)

O n th e o th e r h a n d , if M > N , th e fo rm u la fo r th e c ro ss c o rre la tio n becom es

rxy(l) = Y

x ( n ) v ( n - 1)

0 < 1< N -1

(2.6.31)

T h e fo rm u la s in (2.6.30) a n d (2.6.31) c a n b e c o m b in ed a n d c o m p u te d by m ean s

o f th e fo llo w in g sim p le alg o rith m illu stra te d in th e flow chart in Fig. 2.43. By
in terch an g in g th e ro les o f j ( ) a n d y(n) an d re c o m p u tin g th e cro ssc o rre la tio n
se q u e n c e , w e o b ta in th e v alu es o f rXY(l) c o rre sp o n d in g to n eg ativ e shifts I.
If w e w ish to c o m p u te th e a u to c o rre la tio n se q u e n c e rxx(l), w e se t y (n ) = jr(n)
an d M = N in (2.6.31). T h e co m p u ta tio n o f rxx{l) can be d o n e by m e a n s of the
sa m e a lg o rith m fo r p o sitiv e shifts only.

2.6.5 Input-Output Correlation Sequences

In th is se ctio n w e d e riv e tw o i n p u t- o u tp u t re la tio n sh ip s fo r L T I system s in th e
c o rre la tio n d o m a in . L e t us assu m e th a t a signal x ( n ) w ith k n o w n a u to c o r re la
tio n rxx(I) is a p p lied to an L T I system w ith im p u lse re sp o n s e h( n), p ro d u c in g th e

132

Discrete-Tim e Signals and Systems

C j lD

Chap. 2

Figure 2A3 Flowchart for software

implementation of crosscorrelation.

Sec. 2.6

133

Correlation of Discrete-Time Signals

o u tp u t signal
v(n) = h( n) * .x (/?) =

h(k)x(>i k)

k=oc
T h e c ro ss c o rre la tio n b e tw e e n th e o u tp u t a n d th e in p u t signal is
r VJ-(7) = y d ) * x ( - l ) h ( l ) * [*(/) * * ( - / ) ]
or
ryx(l) = /;(/) * rxx(l)

(2.6.32)

w h ere w e h av e u se d (2.6.8) a n d th e p ro p e rtie s o f co n v o lu tio n . H e n ce th e c ro ssco r

re la tio n b e tw e e n th e in p u t an d the o u tp u t of th e sy stem is th e co n v o lu tio n of the
im p u lse re sp o n s e w ith the a u to c o rre la tio n of th e in p u t se q u e n c e . A lte rn a tiv e ly .
ryx(l) m av b e view ed as th e o u tp u t of th e L T I sy stem w hen th e in p u t se q u en ce is
rxx (/). T h is is illu stra te d in Fig. 2.44. If we rep lace / by - / in (2.6.32), w e o b tain
r.tyU) = h ( - D * rxx(l)
T h e a u to c o rre la tio n of th e o u tp u t signal can be o b ta in e d by using (2.6.8) w ith
x(/i) = y(/i) an d th e p ro p e rtie s o f co n v o lu tio n . T h u s we have
r lv(/) = v(/) * v (/)
= [/;(/) * * (/)] * [/j ( / J * * ( /)]
(2.6.33)
= [/?(/) * h ( /)] * [*(/) * * ( /)]
= rhh(l) * rxx(l)
T h e a u to c o rre la tio n rhh(l) of th e im pulse resp o n se h(n) exists if th e system is stable.
F u rth e rm o re , th e stab ility in su res th a t th e system d o e s n ot ch an g e th e ty p e (en erg y
o r p o w e r) o f th e in p u t signal. By ev alu atin g (2.6.33) fo r / = 0 we o b ta in
CC

(2.6.34)
w hich p ro v id e s th e en e rg y (o r p o w er) of th e o u tp u t signal in te rm s o f a u to c o r re
latio n s. T h e se re la tio n sh ip s h o ld fo r b o th en erg y a n d p o w e r signals. T h e direct
d e riv a tio n o f th e s e re la tio n sh ip s for en erg y an d p o w e r signals, a n d th e ir e x ten sio n s
to co m p lex sig n als, are left as exercises fo r the stu d e n t.

Input
rxx<n )

LTI
SYSTEM

Output

ft(n)

rvr(n)
Figure 2.44 Input-output relation for
crosscorrelation ryx(n).

134

Discrete-Time Signals and Systems

Chap. 2

2.7 SUMMARY AND REFERENCES

T h e m a jo r th e m e o f th is c h a p te r is th e c h a ra c te riz a tio n o f d isc re te -tim e signals an d
sy stem s in th e tim e d o m ain . O f p a rtic u la r im p o rta n c e is th e class o f lin e a r tim ein v a ria n t (L T I) sy stem s w hich a re w idely u se d in th e design a n d im p le m e n ta tio n
o f d ig ital signal p ro cessin g system s. W e c h a ra c te riz e d L T I sy ste m s by th e ir u n it
sa m p le re sp o n se h{n) an d d e riv e d th e co n v o lu tio n su m m a tio n , w hich is a fo rm u la
fo r d e te rm in in g th e re sp o n se y( n) o f th e system c h a ra c te riz e d by h( n) to an y given
in p u t se q u e n c e x(n).
T h e class o f L T I system s c h a ra c te riz e d by lin e a r d ifferen ce e q u a tio n s w ith
c o n s ta n t coefficients is by fa r th e m o st im p o rta n t of th e L T I sy stem s in th e th e o ry
an d a p p licatio n o f d ig ital sig n a l p ro cessin g . T h e g e n e ra l s o lu tio n of a lin e a r dif
fe re n c e e q u a tio n w ith c o n s ta n t coefficients w as d eriv e d in th is c h a p te r an d show n
to co n sist of tw o c o m p o n en ts: th e so lu tio n o f th e h o m o g e n e o u s e q u a tio n w hich
r e p re s e n ts th e n a tu ra l re sp o n se o f th e sy stem w h en th e in p u t is z e ro , an d th e p a r
tic u la r so lu tio n , w hich re p re s e n ts th e re sp o n s e o f th e system to th e in p u t signal.
F ro m th e d ifferen ce e q u a tio n , w e also d e m o n s tra te d h o w to d e riv e th e u n it sam ple
re sp o n s e of th e L T I system .
L in e a r tim e -in v a ria n t sy stem s w ere g en erally su b d iv id ed in to F IR (finited u ra tio n im p u lse re sp o n se ) a n d I I R (in fin ite -d u ra tio n im pulse re sp o n s e ) d e p e n d
ing o n w h e th e r h(n) h as finite d u ra tio n o r infinite d u ra tio n , resp ec tiv ely . T he
re a liz a tio n s o f such system s w ere briefly d escrib ed . F u rth e rm o re , in the re a liz a
tio n o f F IR system s, we m a d e th e d istin ctio n b e tw e e n recu rsiv e a n d n o n recu rsiv e
re a lizatio n s. O n th e o th e r h a n d , w e o b se rv e d th a t I I R system s can be im p le m e n te d
recu rsiv ely , only.
T h e re are a n u m b e r o f te x ts o n d isc re te -tim e signals a n d system s. W e m e n
tio n as ex am p les th e b o o k s by M c G illem a n d C o o p e r (1984), O p p e n h e im a n d W illsky (1983), an d S ie b e rt (1986). L in e a r c o n sta n t-c o e ffic ie n t d iffe re n c e e q u a tio n s are
tr e a te d in d e p th in th e b o o k s by H ild e b ra n d (1952) a n d L evy a n d L essm an (1961).
T h e last to p ic in this c h a p te r, o n c o rre la tio n o f d isc re te -tim e signals, plays an
im p o rta n t ro le in d ig ital signal p ro cessin g , esp ecially in a p p lic a tio n s d ealin g w ith
digital c o m m u n ic a tio n s, ra d a r d e te c tio n a n d e stim a tio n , so n a r, a n d geophysics. In
o u r tr e a tm e n t o f c o rre la tio n se q u e n c e s, w e a v o id e d th e use o f sta tis tic a l concepts.
C o rre la tio n is sim ply d efin ed as a m a th e m a tic a l o p e r a tio n b e tw e e n tw o se q u en ces,
w hich p ro d u c e s a n o th e r se q u e n c e , called e ith e r th e crosscorrelat ion s e quence w hen
th e tw o se q u e n c e s a re d iffe re n t, o r th e aut ocorrel ati on sequence w h e n th e tw o se
q u e n c e s are id en tical.
In p ractical a p p lic a tio n s in w hich c o rre la tio n is u sed , o n e ( o r b o th ) o f th e
se q u e n c e s is (a re ) c o n ta m in a te d by n o ise a n d , p e rh a p s , by o th e r fo rm s o f in te rfe r
en ce. In such a case, th e noisy se q u e n c e is c alled a r a n d o m se q u e n c e a n d is c h a r
a c te riz e d in sta tistical term s. T h e c o rre sp o n d in g c o rre la tio n se q u e n c e b e c o m e s a
fu n ctio n o f th e sta tistical c h a ra c te ristic s of th e n o ise an d an y o th e r in te rfe re n c e .
T h e statistical c h a ra c te riz a tio n o f se q u e n c e s a n d th e ir c o rre la tio n is tr e a te d in
A p p e n d ix A . S u p p le m e n ta ry re a d in g o n p ro b a b ilis tic an d sta tistical c o n c e p ts deal-

Chap. 2

135

Problems

ing w ith c o rre la tio n can be fo u n d in th e b o o k s by D a v e n p o rt (1970). H e lstro m

(1990). P ap o u lis (1984). an d P eeb les (1987).

PROBLEMS
2.1 A discrete-time signal x(n) is defined as

l + j,

3 < n < 1

1.

0 < n < 3

0,

elsewhere

(a) Determ ine its values and sketch the signal .v(n).
(b) Sketch the signals that result if we:

(1) First fold x{n) and then delay the resulting signal by four samples.
(2) First delay xin) by four samples and then fold the resulting signal
(c) Sketch the signal x ( n + 4 ).
(d) Com pare the results in parts (b) and (c) and derive a rule for obtaining the signal
,v(n -t- k) from
(e) Can you express the signal ,r(n) in term s of signals S(n) and u{n)l
2.2 A discrete-time signal ,v(n) is shown in Fig. P2.2. Sketch and label carefully each of
the following signals.
.v(n)

J_

J_

L L _ ________ _
- 2 - 1 0 1 2 3 4

FigUre P2.2

(a ) x(n - 2)

(b) x(4 n) ( c)x(n + 2) (d) x(n)u(2 n)

(e) x(n - 1 )8{n - 3) (F) x( n2) (g) even part of x{n)
(h) odd part of x ( n )
2 3 Show that
(a ) &(n) = u(n) u(n 1)
(b) u(n) =

8(k) =

2.4 Show that any signal can be decomposed into an even and an odd component. Is the
decomposition unique? Illustrate your arguments using the stgnal
x(n) = {2. 3, 4. 5. 6)
t

2.5 Show that the energy (power) of a real-valued energy (power) signal is equal to the
sum of the energies (powers) of its even and odd components.
2.6 Consider the system
vf) = T[x( n) ] = x ( n 2)

(a) Determ ine if the system is time invariant.

136

Discrete-Time Signals and Systems

Chap. 2

(b) To clarify the result in part (a) assume that the signal
_fl,
\ 0,

O 5 /1 < 3
elsewhere

is applied into the system.

(1) Sketch the signal *().
(2) D eterm ine and sketch the signal y ( n ) = T[x(n)},
(3) Sketch the signal y'2(n) = y(n 2).
(4) D eterm ine and sketch the signal x2(n) = x(n - 2).
(5) Determ ine and sketch the signal
= T \ x 2(n)].
(6) Com pare the signals ^ ( n ) and y(n - 2). W hat is your conclusion?
(c) R epeat part (b) for the system
y(rt) = x(n) - x(n 1)

Can you use this result to make any statem ent about the time invariance of this
system? Why?
(d) Repeat parts (b) and (c) for the system
y(rt) = T [ x ( n )] = nx(n)
2.7 A discrete-time system can be
(1) Static or dynamic
(2) Linear or nonlinear
(3) Time invariant or time varying
(4) Causal or noncausal
(5) Stable o r unstable
Examine the following systems with respect to the properties above.
(a) v() = cos[jc(n)]
v(n) =
x(k>
v(n) = x(n)cos(^n)
y(n) ~ x ( n + 2)
y(n) = Trun[;c(n)], where Trun[jc(n)] denotes the integer part of x(n), obtained
by truncation
(f) y(n) = Round[jc(n)], where Round[;c(n)] denotes the integer part of Jt(n) obtained
by rounding
Remark: The systems in parts (e) and (f) are quantizers that perform truncation and
rounding, respectively.
(g) y(B) = |*(n)|
(h) v(rt) = x(n)u(n)
(I) y(n) = x(n) + nx{n + 1)
(j) y ( n ) = x ( 2 n )

(b)
(c)
(d)
(e)

(I) y( n) = x ( - n )
(m) y(n) = sign[j:(n)]
(n) The ideal sampling system with input xaU) and output x(n) = x a(nT), oc <
n < oo
2J8 Two discrete-time systems 7] and T2 are connected in cascade to form a new system
T as shown in Fig. P2.8. Prove or disprove the following statements.

Chap. 2

137

Problems

yin)

xin )

Ti

T;
T - T-. T 2

Figure P2.8

(a) If T\ and % are linear, then T is linear (i.e.. the cascade connection of two linear
systems is linear).
(b) If T\ and
are time invariant, then T is time invariant.
(c) If T[ and 7? are causal, then T is causal.
<d) If T] and T2 are linear and time invariant, the same holds for T .
(e) If 7] and T2 are linear and time invariant, then interchanging their order does not
change the system T.
(0 As in part (e) except that 7J, T2 are now time varying. (Hint: Use an example.)
(g) If 7] and T2 are nonlinear, then T is nonlinear.
(h) If T< and T2 are stable, then T is stable.
(i) Show by an example that the inverse of parts (c) and (h) do not hold in general.
2.9 Let T be an LTI, relaxed, and BIBO stable system with input x{n) and output y(n).
Show that:
(a) If x(n) is periodic with period N [i.e., jr(n) = x{n + N) for all n > 0], the output
y(n) tends to a periodic signal with the same period.
(b) If x(n) is bounded and tends 10 a constant, the output will also tend to a constant.
(c) If x{n) is an energy signal, the output y(n) will also be an energy signal.
2.10 The following in p u t-output pairs have been observed during the operation of a umeinvariam system:
x,(n) = {1.0,2} ^
t

y,(fi) = (0, 1.2}

t

x,(n) = {0.0,3} ^
t

v; (n) = (0, 1.0,2}

t

x-\(n) = {0. 0, 0. 1}

vj(n) = (1,2, 1}

Can you draw any conclusions regarding the linearity of the system. W hat is the
impulse response of the system?
2.11 The following input-output pairs have been observed during the operation of a linear
system:
Xi(n) = {-1. 2. 1}
t

x2{n) = {1, - 1 , -1}

y,(/i) = (1, 2. - 1 , 0. 1}
t

\'2in) = {-1. 1, 0, 2}

t
x 3(n) = {0, 1, 1)
t

t
y i ( n) = {1, 2. 1}
r

Can you draw any conclusions about the time invariance of this system?
2.12 The only available information about a system consists of N input-output pairs, of
signals y,(rc) = T[xj(n)], / = 1, 2........N.

Discrete-Time Signals and Systems

138

Chap. 2

(a) What is the class of input signals for which we can determ ine the output, using
the information above, if the system is known to be linear?
(b) The sam e as above, if the system is known to be tim e invariant.
2.13 Show that the necessary and sufficient condition for a relaxed LTI system to be BIBO
stable is
y .

i/i()i < Mh < oo

for some constant Mn.

2.14 Show that;
(a) A relaxed linear system is causal if and only if for any input x(n) such that
for n < no

jc(n) = 0 for n < no => y(n) = 0

(b) A relaxed LTI system is causal if and only if
h(n) = 0

for n < 0

2.15 (a) Show that for any real or complex constant a, and any finite integer num bers M
and N, we have
n
a M a h
. if * !
1 <3

N - M + 1,

if a = 1

(b) Show that if |o| < 1 , then

y v

= - i -

l - a

2.16 (a) If y(n) = x (n) * h(n). show that v =

/ .- where ^
_w
(b) Compute the convolution y(n) = x(n) * h(n) of the following signals and check
the correctness of the results by using the test in (a).
(1) jr(n) (1,2, 4), /)(n) = (1 ,1 ,1 ,1 ,1 }
(2) x(n) = {1, 2, - 1 ) , h(n) = x (n)
(3) x(n) = (0,1, - 2 , 3. - 4 ) . h(n) = {, i , 1, 1}
(4) jc(n )= :{1 .2 .3.4.5J.A (n) = {l)
(5) x(n) = (1, -2 ,3 } , h(n) = (0, 0 .1 .1 ,1 ,1 )
t
t
(6) x(n) = { 0 ,0 ,1 ,1 ,1 ,1 ), h(n) = { 1 ,-2 . 3}
t
t
(7) jr(u) = {0,1, 4, -31. h(n) = [1,0, - 1 , -1}
t
t
(8)

= [1,1,2], h(n) = u(n)

t

(9) jt(n) = [1,1. 0,1,11, h(n) = {1, - 2 , - 3 , 4}

t
t
(10) jc(n) = (1,2, 0,2 , l}/i(n) = Jt(n)
t
(11) *{n) = (i)"u(n), h(rt) = ( j ) nM(n)

2.17 Compute and plot the convolutions x(n) * h(n) and h(n) *x(n) for the pairs of signals
shown in Fig. P2.17.

Chap. 2

139

Problems

trln)

b TI f
0 12 3

0 l 2 3 4 5 b

h\n)

j x(n)

Iiit.
1 2 3

-3-2-10 I 2 3

hin)

x(n)
i

..III!..
3 4 5 6

II

htnl

J (n )
]

111

II

2-1

2 3 4 5
<di

2.18

Figure P2.17

Determ ine and sketch the convolution y(n) of the signals

-V(/l) =
h(n) -

0 < Ji < 6
elsewhere

0.
1,
0.

2 < n < 2
elsewhere

(a) Graphicallv
(b) Analytically

2.19 Compute the convolution y(n) of the signals

x (n) =

h(n) =

o'".

3 < n < 5

0.

elsewhere

1,
0,

0< n< 4
elsewhere

2.20 Consider the following three operations.

(a) Multiply the integer numbers: 131 and 122.
(b) Compute the convolution of signals: {1. 3.1) * (1,2. 2}.
(c) Multiply the polynomials: 1 4- 3; + z2 and 1 4- 2z 4- 2z2.
(d> Repeat part (a) for the numbers 1.31 and 12.2.
(e) Comment on your results.
2.21 Com pute the convolution y(n) = x ( n ) * h(n) of the following pairs of signals.
(a) x(n) = a"u(n), h(n) = b"u{n) when a ^ b and when a ~ b
(b) x ( n) =

1.
2,

. 0,

n = 2, 0, 1
n = 1

elsewhere

h ( n ) = S(n) S (n 1) + S(n 4) + S(n 5)

140

Discrete-Time Signals and System s

Chap. 2

(c) x{n) = u(n + 1) u(n 4) <5(n 5)

h(n) = [u(n +2) u(n - 3)] (3 - |n|)
(d ) x ( n ) = u ( n ) - u( n - 5)
h(n) = u(n 2) u(n 8) 4- u(n 11) u(n 17)
2.22 Let x(n) be the input signal to a discrete-time filter with impulse response ht(n) and
let y,(n) be the corresponding output.
(a) Compute and sketch x(n) and \ j ( n ) in the following cases, using the same scale
in all figures.
x(n) = {1,4, 2. 3, 5, 3, 3. 4. 5. 7. 6. 9}
h](n) = (1,1)
h2(n) = {1,2.1}
]ij(n) = {i, j)
*4() = {? { j)

(b)
(c)
(d)
(e)

Sketch x(n), yi(n), y 2(n) on one graph and *(). y3(n), y,j(n), y.s(n) on another
graph
W hat is the difference between yi() and \2(n). and between y^(n) and y^(n)?
Comment on the smoothness of v2(/?) and v4(n). Which factors affect the sm ooth
ness?
Compare y4(n) with ysfn). What is the difference? Can you explain it?
Let h(,(n) = {^, - j } . Com pute y6<n). Sketch v(n), y 2(n), and yft(n) on the same
figure and comment on the results.

2.23 The discrete-time system

v(n) = ny(n 1) + jr(n)

n > 0

is at rest [i.e., v(1) = 0]. Check if the system is linear time invariant and BIBO stable.
2.24 Consider the signal y(n) = a"u(n), 0 < a < 1.
(a) Show that any sequence x{n) can be decomposed as

and express ck in terms of x(n).

(b) Use the properties of linearity and time invariance to express the output y(n) =
T[x(n)] in term s of the input x (n) and the signal g(n) = T[y(n)], where T [ ] is
an LTI system.
(c) Express the impulse response h(n) = T[B{n)} in terms of g(rt).
2.25 D eterm ine the zero-input response of the system described by the second-order dif
ference equation
x(n) - 3y(n - 1) - 4y(n - 2) = 0
2.26 Determ ine the particular solution of the difference equation
y(n) = jv(ii - 1) - y(n - 2) +x ( n)
when the forcing function is x(n) = 2"u(n).

Chap. 2

Problems

141

2 2 1 D eterm ine the response yin). n > 0. of the system described by the second-order
difference equation
yin) - 3v(n 1) 4y(/i 2) = xin) + 2x(n 1)
to the input xin) = 4"w(n).
2.28 Determ ine the impulse response of the following causal system:
y(n) 3y(n 1) 4v(n 2) = jr(n) + 2x(n 1)
2.29 Let xin). A'i < n < N2 and h(n),
< n < M2 be two finite-duration signals.
(a) Determ ine the range L\ < n < L 2 of their convolution, in term s of N\, N2, M\
and M2.
(b) Determ ine the limits of the cases of partial overlap from the left, full overlap,
and partial overlap from the right. For convenience, assume that h(n) has shorter
duration than jc().
(c) Illustrate the validity of your results by computing the convolution of the signals
-2 < n < 4
elsewhere
-1 < n < 2
elsewhere
2.30 Determ ine the impulse response and the unit step response of the systems described
by the difference equation
(a) yin) = ().6y(;i - 1) - ().08v(n - 2) + xin)
(b) _v( ) = 0.7y(;; - 1) - 0.1 yin - 2) -f 2xin) - xin - 2)
231 Consider a svstem with impulse response
A,) = H r '
( 0.

- n - 4
elsewhere

Determ ine the input xin) for 0 < n < S that will generate the output sequence
v(n) = 1 1 .2 .2 .5 .3 .3 .3 .2 .1 .0 ....}

t
232 Consider the interconnection of LTI systems as shown in Fig. P2.32.
(a) Express the overall impulse response in terms of h \ (n), h2(n), h^in). and h^in).
(b) D eterm ine h{n) when
M ) = {j. 3 . 7 }
h2{n) hy(n) = (n + 1 )u(n)
fi4(n) = S(n 2)

Figure P 2J2

Discrete-Time Signals and Systems

142

Chap. 2

(c) Determ ine the response of the system in part (b) if

x(n) = &(n + 2) + 3S(n - 1) - 4 S(n - 3)

2 3 3 Consider the system in Fig. P2.33 with h(n) = a"u(n), 1 < a < 1. Determ ine the
response y(n) of the system to the excitation
*(n) = (n + 5) u(n 10)

x(n)

Figure P233
2 3 4 Com pute and sketch the step response of the system

U_ 1

2 3 5 Determ ine the range of values of the param eter a for which the linear time-invariant

system with impulse response

(Hint: The solution can be obtained easily and quickly by applying the linearity and
tim e-invariance properties to the result in Exam ple 2.3.5.)
2 3 7 D eterm ine the response of the (relaxed) system characterized by the impulse response

h(n) = ( l ) u(n)
to the input signal
| 1,
x(n) = { ^
10,

0 < n < 10
~
otherwise

2 3 8 D eterm ine the response of the (relaxed) system characterized by the impulse response

h(n) = ( j ) Hu{n)
to the input signals
(a) x(n) = 2nu(n)
(b) x(n) = u ( - n )

Chap. 2

143

Problems

239 T hree systems with impulse responses h\(n) 5(n) &(n 1). h2(rt) = h( n}. and
= u(n), are connected in cascade.
(a) W hat is the impulse response.
of the overall system?
(b ) Does the order of the interconnection affect the overall system?
2.40 (a) Prove and explain graphically the difference between the relations
x(n )5(n no) =

o)

x(n) *&(n - n0) = x(n - nu)

and

(b ) Show that a discrete-time system, which is described by a convolution summation,

is LTI and relaxed,

(c) W hat is the impulse response of the system described by y(n) = x(n n())?
2.41 Two signals 5(n) and u(n) are related through the following difference equations
j(n) + a\ j(n 1) +

N) = bi)v(n)

Design the block diagram realization of:

(a) The system that generates x(n) when excited by v(n).
(b ) The system that generates u(n) when excited by s(n).
(c) What is the impulse response of the cascade interconnection of systems in parts
(a) and (b)?
2.42 Com pute the zero-state response of the system described by the difference equation
y(n ) + ^ v(n 1) = x(n ) + 2x{n 2)
to the input
xin) = (1.2. 3. 4, 2, 1)

T
by solving the difference equation recursively.
2 .43 Determ ine the direct form II realization for each of the following LTI systems.
(a) 2v(n) + y(n 1 ) - 4 v(n 3) = x(n) + 3x(n 5)
(b) y(n) = Jr(n) x(n 1) + 2x(n 2) 3x(n - 4)
2 .4 4 Consider the discrete-time system shown in Fig. P2.44.

Figure P2.44

(a) Com pute the 10 first samples of its impulse response.

(b ) Find the input-output relation.

(c) Apply the input x(n) = { 1 .1 .1 ....} and com pute the first 10 samples of the output,
t

144

Discrete-Time Signals and Systems

Chap. 2

(d) Com pute the first 10 samples of the output for the input given in part (c) by using
convolution.
(e) Is the system causal? Is it stable?
2 ^ 5 Consider the system described by the difference equation
y(n) = ay(n - 1) + bx(n)

(a) Determ ine b in terms of a so that

(b ) Com pute the zero-state step response s(n) of the system and choose b so that

j(oo) = 1.
(c) Com pare the values of b obtained in parts (a) and (b). W hat did you notice?
2*46 A discrete-time system is realized by the structure shown in Fig. P2.46.
(a) D eterm ine the impulse response.
(b) Determ ine a realization for its inverse system, that is, the system which produces
x( n) as an output when y(n) is used as an input.
x in )

-o

-0

v(n)

0.8

Figure P2.46

2 .4 7 Consider the discrete-time system shown in Fig. P2.47.

v(n)

Figure P2^f7

(a) Com pute the first six values of the impulse response of the system.
(b ) Com pute the first six values of the zero-state step response of the system.

(c) Determ ine an analytical expression for the impulse response of the system.
1 4 8 D eterm ine and sketch the impulse response of the following systems for n 0,
1........9.
(a) Fig. P2.48(a).
(b ) Fig. P 2 .4 8 (b ).
(c) Fig. P2.48(c).

Chap. 2

Problems

145

Cc)

Figure P2.48

(d) Classify the systems above as FIR or IIR.

(e) Find an explicit expression for the impulse response of the system in part (c).

2.49 Consider the systems shown in Fig. P2.49.

(a) Determ ine and sketch their impulse responses /i|(n), h2(n), and h3(n).
(b) Is it possible to choose the coefficients of these systems in such a way that
h\(n) = h2(n) = h3(n)
2.50 Consider the system shown in Fig. P2.50.
(a) Determ ine its impulse response h(n).
(b) Show that h(n) is equal to the convolution of the following signals.
h] (n) = 6(n) + 6(n - 1)
M " ) = (^)"u(n)

Discrete-Time Signals and Systems

146

Chap. 2

v (n )

yin)

2.51 Com pute the sketch the convolution y,(n) and correlation r,(n) sequences for the
following pair of signals and comment on the results obtained.
(a) *,{) = (1.2.4)
A ,(n )= (1 ,1 .1 .1 . If
t

(b) x2(n) = (0,1. - 2 . 3, - 4 ]

h2(n) = U. 1. 2 . 1 , i}

(c)

jt-j( b )

= (1.2. 3, 4}
t

(d) x4(n) = {1. 2, 3,4)

t

A3 (u )

= (4. 3. 2, 1)
t

hA(n) = (1.2.3. 4)
t

2.52 The zero-state response of a causal LTI system to the input x ( n ) = {1,3, 3,1) is
y(n) = (1,4, 6 ,4 ,1 ). Determ ine its impulse response.
t

Chap. 2

Problems

147

2.53 Prove by direct substitution the equivalence of equations (2.5.9) and (2.5.10), which
describe the direct form II structure, to the relation (2.5.6), which describes the direct
form I structure.
2.54 D eterm ine the response y(n). n > 0 of the system described by the second-order
difference equation
y ( n) 4 y(n - 1) + 4v(/i 2) = x( n) x( n 1)

when the input is

x(n) = (-l)" u (n )
and the initial conditions are v(1) = y ( -2 ) = 0.
2.55 D eterm ine the impulse response h(n) for the system described by the second-order
difference equation
v(ni 4v(;i 1 > + 4y(n 2) = x(r?) x(n 1)

2.56 Show that any discrete-tim e signal x(n) can be expressed as

[.v(A') x(k 1)]u(n - k)

c(n) =

where (/i - k ) is a unit step delayed by k units in time, that is,

u(/i -

1.
I 0,

k) =

n >k
otherwise

2.57 Show that the output of an LTI system can be expressed in term s of its unit step
response v(n) as follows.
i'(n) = y " ' [,v(A:) x(k 1)]jr(fl k)

[x(AQ - x ( k - l) ] s ( /i - k)

c
2.58 Com pute the correlation sequences rIX(l) and rtv(l) for the following signal sequences.
j _

P nu - N < n < n {, + N

I 0,
f 1.
v(n)

10,

otherwise
-N <n < N

otherwise

2.59 D eterm ine the autocorrelation sequences of the following signals.

(a) x(n) = {1. 2.1.1)
t

(b) v(n) = il. 1.2.1}

t

W hat is your conclusion?

2.60 W hat is the normalized autocorrelation sequence of the signal x(n) given by
1,
x(n) = ,
0,

-jV < n < N

otherwise

Discrete-Time Signals and Systems

148

Chap. 2

2.61 An audio signal j(r) generated by a loudspeaker is reflected at two different walls
with reflection coefficients r ] and r2. The signal *(/) recorded by a microphone close
to the loudspeaker, after sampling, is
x (n) = s(n) + rxs(n k\) + r2s(n k2)
where kt and k2 are the delays of the two echoes.
(a) Determ ine the autocorrelation rzx(I) of the signal x(n).
Can we obtain ri, r2, k\, and k2 by observing r,s (1)1
(c) W hat happens if r2 = 0?

(b)

2.62* Time-delay estimation in radar Let xa(t) be the transm itted signal and yfl(r) be the
received signal in a radar system, where
y(r) = axa(t - id) + vu(f)
and va(t) is additive random noise. The signals xa(t) and >(/) are sampled in the
receiver, according to the sampling theorem , and are processed digitally to deter
mine the time delay and hence the distance of the object. The resulting discrete-time
signals are
jr(n) = xa(nT)
y(n) = y(nT) = axu(nT - DT) + vu(nT)
= ax(n D) + u(n)
(a) Explain how we can measure the delay D by computing the crosscorrelation r*,.(/).
(b) Let x(n) be the 13-point Barker sequence
X

(n) = (+ 1,+1,+1,+1,+1, -1, -l.+l.+l, -1,+1. -1,+1)

and u(n) be a Gaussian random sequence with zero mean and variance a 2 = 0.01.
Write a program that generates the sequence v(n), 0 < n < 199 for a = 0.9 and
D = 20. Plot the signals jt(), y(n), 0 < n < 199.
(c) Compute and plot the crosscorrelation rTV(/), 0 < / < 59. Use the plot to estimate
the value of the delay D.
(d) Repeat parts (b) and (c) for a 2 = 0.1 and a 2 = 1.
<e) Repeat parts (b) and (c) for the signal sequence
jt(n) = j _ l , _ l , - l , + i , + i , + i . + i , - i ,
+ 1 . - l . + l . + l , - 1 ,- 1 ,+ 1 }
which is obtained from the four-stage feedback shift register shown in Fig. P2.62,

Figure P2.61
register.

Linear feedback shift

Chap. 2

Problems

149

Note that x(n) is just one period of the periodic sequence obtained from the
feedback shift register.
(f) Repeat parts (b) and (c) for a sequence of period N 27 1, which is obtained
from a seven-stage feedback shift register. Table 2.3 gives the stages connected
to the modulo-2 adder for (maximal-length) shift-register sequences of length
N =2"
TABLE 2.3 SHIFT-REGISTER
CONNECTIONS FOR GENERATING
MAXIMAL-LENGTH SEQUENCES
m

Stages Connected to Modu)o-2 Adder

1. 2
1. 3
1. 4
S. 4
L6
1. 7
1, 5, 6. 7
1.6
1. K
1. 10
i. 7. y, 12
1. H), 11. 13
1. 5. 9. 14
1. 15
1. 5. 14, 16
1. 15

3
4
5
6
7
H
y
id
li
12
13
14
15
16
17

2.63* Implementation o f L T I systems

by the difference equation

Consider the recursive discrete-time system described

_v(n) = U\ v( n 1) a;v (n 2) 4- b^x(rt)

where a\ - 0.8, u? = 0.64. and b() = 0.866.

(a) Write a program to compute and plot the impulse response h{n) of the system
for 0 < n < 49.
(b) Write a program to com pute and plot the zero-state step response s(n) of the
system for 0 < n < 100.
(c) Define an F IR system with impulse response ^ fir (h ) given by
,

, .

1 h(n),

10.

0 < n < 19
elsewhere

where h(n) is the impulse response computed in part (a). W rite a program to
compute and plot its step response.
(d) Com pare the results obtained in parts (b) and (c) and explain their similarities
and differences.

150

Discrete-Time Signals and Systems

Chap. 2

2jS4* W rite a com puter program that computes the overall impulse response h(n) of the sys
tem shown in Fig. P2.64 for 0 < n < 99. The systems TU T2, T 3, and % are specified by
Ti : hi(n) = {1 . 5 .
t

g.

55)

T2 : h 2(n) = {1,1,1,1,11
t
Ts :

+ 5*(" - 1) +

- 2)

T4 : y(n) = 0.9y(n 1) 0.81y{n 2) + v(n) + u(n 1)

Plot h(n) for 0 < n < 99.

Figure P2.64

The Z -Transform and Its

Application to the Analysis of
LTI Systems

T ra n s fo rm te c h n iq u e s a re an im p o rta n t tool in the analysis o f signals a n d lin

e a r tim e -in v a ria n t (L T I) system s. In this c h a p te r w e in tro d u c e th e ^ -tran sfo rm ,
d ev elo p its p ro p e rtie s , and d e m o n s tra te its im p o rta n c e in th e analysis an d c h a ra c
te riz a tio n o f lin ear tim e -in v a ria n t system s.
T h e : -tra n sfo rm plays th e sam e role in th e analysis o f d isc re te -tim e signals
an d L T I sy stem s as th e L ap lace tran sfo rm d o e s in th e analysis o f c o n tin u o u s-tim e
signals a n d L T I svstem s. F o r ex am p le, w e sh a d see th a t in th e ^ -d o m ain (com plex
z -p lan e) th e c o n v o lu tio n of tw o tim e-d o m ain signals is e q u iv a le n t to m u ltip lic atio n
o f th e ir c o rre sp o n d in g ^ -tran sfo rm s. T h is p ro p e rty g reatly sim plifies the analysis
o f th e re sp o n s e o f an L TI system to v ario u s signals. In ad d itio n , th e c-tran sfo rm
p ro v id es us w ith a m ean s of ch a ra c te riz in g an L T I system , a n d its resp o n se to
v ario u s signals, by its p o le - z e r o locations.
W e b eg in th is c h a p te r by defining th e c-tran sfo rm . Its im p o rta n t p ro p e rtie s
a re p re s e n te d in S ectio n 3.2. In S ection 3.3 the tra n sfo rm is u se d to ch aracterize
signals in te rm s o f th e ir p o le - z e r o p a tte rn s. S ection 3.4 d escrib es m e th o d s fo r
in v ertin g th e z-tra n sfo rm o f a signal so as to o b ta in th e tim e -d o m a in re p re s e n ta
tio n o f th e signal. T h e o n e-sid ed :-tra n s fo rm is tr e a te d in S ectio n 3.5 a n d used
to solve lin e a r d ifferen ce e q u a tio n s w ith n o n z e ro in itial co n d itio n s. T h e c h a p te r
c o n clu d es w ith a d iscussion o n th e use of th e z -tra n sfo rm in th e analysis o f L TI
system s.

3.1 THE Z-TRANSFORM

In th is sectio n w e in tro d u c e th e z -tra n sfo rm of a d isc re te -tim e signal, in v estig ate
its c o n v e rg e n c e p ro p e rtie s , an d briefly discuss th e in v erse z-tran sfo rm .
151

152

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

3.1.1 T h e D ire c t z - T r a n s f o r m
T h e ^ -tran sfo rm o f a d isc re te -tim e signal jt(fi) is d efin ed as th e p o w e r series
OC

X(z) =

n= oo

x { n ) z ~'

(3-u )

w h ere z is a co m p lex v ariab le. T h e re la tio n (3.1.1) is so m e tim e s called th e direct

z - t ransform b ecau se it tra n sfo rm s th e tim e -d o m a in signal x ( n ) in to its co m p lex p la n e re p re s e n ta tio n X (z). T h e in v erse p ro c e d u re [i.e., o b ta in in g x ( n ) fro m X (z)]
is called th e i nverse z- t ransf orm a n d is e x a m in e d briefly in S e ctio n 3.1.2 a n d in
m o re d etail in S ectio n 3.4.
F o r c o n v en ien ce , th e z -tra n sfo rm o f a signal x ( n ) is d e n o te d by
X (z) - Z U ( )}

(3.1.2)

w h ereas th e re la tio n sh ip b e tw e e n x ( n ) a n d X ( z ) is in d ic a te d by
jc()

< -^ * (;)

(3.1.3)

S ince th e z-tra n sfo rm is an infinite p o w e r se ries, it exists only fo r th o se v alu es of

z fo r w hich th is se ries co n v erg es. T h e region o f conver gence ( R O C ) o f X (z) is th e
set o f all v alu es o f z fo r w hich X ( z ) a tta in s a finite value. T h u s an y tim e w e cite
a z-tra n sfo rm w e sh o u ld also in d icate its R O C .
W e illu strate th e se co n c e p ts by so m e sim ple ex am p les.
Example 3.1.1
D eterm ine the ^-transforms of the following finite-duration signals.
(a) jf,(n) = (1 ,2 .5 .7 ,0 ,1 }
(b) jf,() = ( I . 2 . 5 . 7 . 0 . 1)
t

(c) jc3(n) = (0 ,0 ,1 ,2 , 5, 7,0.1}

(d) *4(h) = (2 .4 ,5 .7 .0 ,1 )
t

(e) x j ( n ) = S( n)

(f) x$(n) = <S(n k), k > 0

(g) x j ( n ) = &(n + k) , k > 0
Solution

From definition (3.1.1), we have

(a) X](z) = 1 + 2z~' + 5z~2 + 7 z '3 + z~5, ROC: entire z-plane except z = 0
(b) X 2(z) = z2 + 2z + 5 + 7c-1 + z-3, ROC: entire z-plane except z = 0 and z = oo
(c) Xj(z) = z~2 + 2z-3 + 5z-4 + 7z-5 -I- z-7, ROC: entire z-plane except z = 0
(d) X4(z) = 2z2 -I- 4z -I- 5 4- 7z_1 -I- z-3, ROC: entire z-plane except z = 0 and z = oo
(e) X;(z) = l[i.e S(n)

* 1], ROC: entire z-plane

(f) Xb(z) = z_ t[i.e &(n k) z_t], k > 0, ROC: entire z-plane except z = 0
(g) Xy(z) = zk[i.e &(n + k)

z*], k > 0, ROC: entire z-plane except z = oo

Sec. 3.1

153

The z-Transform

F ro m th is ex a m p le it is easily se en th a t th e R O C o f a f i ni t e-durat i on signal

is th e e n tire ;- p la n e , e x cep t p ossibly th e p o in ts z = 0 a n d /o r z oo. T h e se p o in ts
a re e x c lu d ed , b e c a u s e z t (k > 0} b eco m es u n b o u n d e d fo r z = oc an d z ~ k01 > 0)
b ec o m e s u n b o u n d e d fo r z = 0.
F ro m a m a th e m a tic a l p o in t of view th e z-tra n sfo rm is sim p ly an a lte rn a tiv e
r e p re s e n ta tio n o f a signal. T h is is nicely illu stra te d in E x a m p le 3.1.1, w h e re w e
see th a t th e co effic ie n t o f z~", in a given tra n sfo rm , is th e v a lu e o f th e signal at
tim e n. In o th e r w o rd s, th e e x p o n e n t o f z co n tain s th e tim e in fo rm a tio n w e n eed
to id en tify th e sa m p le s o f th e signal.
In m an y cases w e can ex p ress th e sum of th e finite o r infinite se rie s fo r th e
z-tra n s fo rm in a c lo sed -fo rm e x p ressio n . In such cases th e z -tra n s fo rm o ffers a
co m p act a lte rn a tiv e r e p re s e n ta tio n of th e signal.
Example 3.1.2
D eterm ine the z-transform of the signal
Jf(n) = (5
Solution

The signal jc(n) consists of an infinite num ber of nonzero values

x(n)= ( l . a u ^ . U )'1....
The z-transform of x(n) is the infinite power series
X( z ) = 1 + U ' + ( ^) 2z - 2 + (| ) "z"" + ---

ft=<)

nail

This is an infinite geometric series. We recall that

1 + A + ,42 + A3 - t - - - - =

1 A

Consequently, for | 1

if | A 1 < 1

< 1, or equivalently* for \z\ >

X(z) =

X(z) converges to

ROC: (zl > |

jZ

We see that in this case, the z-transform provides a compact alternative representation
of the signal x(n).
L e t us e x p ress th e co m p lex v a ria b le z in p o la r fo rm as
z = r e }6
w h ere r = |z| a n d 6 = i^z- T h e n X ( z ) can be e x p ressed as
OC

X ( z ) \ t- r ' = Y
x ( n ) r ~ ne - j en
n*-00

(3.1.4)

154

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

In th e R O C o f X U ). |X ( ;) | < oc. B ut

(3.1.5)
5

\ M n ) r - ne - ' * n \ =

\ x( n) r ~n\

H e n c e |X (z)| is finite if th e se q u en ce x ( n ) r ~ ' is a b s o lu te ly su m m a b le .

T h e p ro b le m o f finding th e R O C fo r X ( z ) is e q u iv a le n t to d e te rm in in g the
ra n g e o f v alu es o f r fo r w hich th e se q u e n c e x ( n ) r ~ " is a b so lu te ly su m m ab le. T o
e la b o ra te , let us e x p ress (3.1.5) as

(3.1.6)

If X ( z ) co n v erg es in so m e region of the co m p lex p la n e , b o th su m m a tio n s in (3.1.6)

m u st be finite in th a t region. If the first sum in (3.1.6) co n v erg es, th e re m ust exist
v alu es o f r sm all e n o u g h such th at the p ro d u c t se q u e n c e x ( n) r" . 1 < /; < oc, is
a b s o lu te ly su m m ab le. T h e re fo re , the R O C for th e first sum co n sists o f all p o in ts
in a circle of so m e rad iu s r^, w here /] < oc, as illu stra te d in Fig. 3.1a. O n the
o th e r h a n d , if th e seco nd sum in (3.1.6) co n v erg es, th e re m u st exist v alu es o f r
larg e en o u g h such th a t th e p ro d u c t se q u e n c e x ( n ) / r " . 0 < n < oc, is a b so lu te ly
su m m ab le. H en ce th e R O C fo r the se co n d sum in (3.1.6) co n sists o f all p o in ts
o u tsid e a circle o f rad iu s r > r2. as illu stra te d in Fig. 3.1b.
Since th e co n v erg en ce of X (c) re q u ire s th a t b o th sum s in (3.1.6) b e finite, it
follow s th a t th e R O C o f X ( z ) is g en erally specified as th e a n n u la r region in th e
;- p la n e , r: < r < r\. w hich is the co m m o n reg io n w h e re b o th su m s are finite. T his
reg io n is illu stra te d in Fig. 3.1c. O n th e o th e r h a n d , if
> r\, th e r e is no co m m o n
reg io n o f co n v erg en ce fo r th e tw o sum s an d h en ce X ( ;) d o es n o t exist.
T h e follow ing ex am p les illu strate th e se im p o rta n t co n cep ts.
Example 3.1.3
Determ ine the e-transform of the signal

Solution

From the definition (3.1.1) we have

If \az 11 < 1 or equivalently, |z| > |a |, this power series converges to 1/(1 - a ; -1).

Sec. 3.1

The z-Transform

155

Im(z)

Im(;)

Im(z)

Figure 3.1 R egion of convergence for

X (z) and its corresponding causal and
anticausa! components.

156

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

lm(;)

Figure 3.2 The exponential signal xi n) = txnu{n) (a), and the ROC of its
transform (b).

Thus we have the z-transform pair

x(n) =a" u( n )

X (;) =

ROC: |z| > |cr|

1 - a ;-1

(3.1.7)

The R O C is the exterior of a circle having radius |a |. Figure 3.2 shows a graph of the
signal .x(n) and its corresponding ROC. Note that, in general, or need not be real.
If we set or = 1 in (3.1.7), we obtain the z-transform of the unit step signal
x(n) - u(n)

X(z) =

ROC: |zl > 1

(3.1.6

Example 3.1.4
Determ ine the z-transform of the signal
x(n) = a"u( n 1) =
Solution

n >0
n < -1

0,

From the definition (3.1.1) we have

-I

oc

n---oc

/= i

where / = n. Using the formula

A + A: + A3 + = A(1 + A + A2 + ) =

A
1 - A

when | >1f < 1 gives

* ( ;) = - -

1
1 a ~ lz

l - a ; '1

provided that |cr_1z| < 1 or, equivalently, |z| < jar j. Thus
1
(3.1.9)
ROC: [z| < |a |
1-azThe R O C is now the interior of a circle having radius |a|. This is shown in Fig. 3.3.
x(n) = a"u( n 1)

X(Z) = -

Sec. 3.1

The z-Transform

157
Im(c)

Figure 3.3 Anticausal signal jt(h) = -crnu( n - 1) (a), and the ROC of its
transform (b).

E x a m p le s 3.1.3 an d 3.1.4 illu stra te tw o very im p o rta n t issues. T h e first c o n

cern s th e u n iq u e n e s s o f th e "-tran sfo rm . F ro m (3.1.7) an d (3.1.9) we see that
th e cau sal signal a nu ( n ) a n d th e an ticau sal signal a " u ( n 1) have id en tical
clo sed -fo rm e x p re ssio n s fo r th e ^ -tran sfo rm , th a t is,
Z ( o " w ( ) ) = Z { a nu ( n -

1) ) -

------ --------

1 a c ' 1

T h is im p lies th a t a clo sed -fo rm e x p ressio n fo r th e z-tra n sfo rm d o e s n o t u n iq u ely

specify th e signal in th e tim e do m ain . T h e am b ig u ity can b e reso lv ed only if
in a d d itio n to th e c lo sed -fo rm ex p ressio n , th e R O C is specified. In su m m ary , a
discrete-time signal x ( n ) is uni quel y d et er mi ned b y its z- t rans f orm A' (;) a n d the
region o f conver gence o f X( z ) . In this te x t th e te rm z -tra n s fo rm " is u se d to re fe r
to b o th th e clo sed -fo rm e x p ressio n an d th e c o rre sp o n d in g R O C . E x a m p le 3.1.3
also illu stra te s th e p o in t th a t the R O C o f a causal signal is the exterior o f a circle
o f s o m e radius r 2 whi l e the R O C o f an ant icausal signal is the interior o f a circle o f
s o m e radi us rj. T h e fo llow ing e x am p le co n sid ers a se q u en ce th a t is n o n z e ro for
00 < n < 00.
Example 3.1.5
D eterm ine the z-transform of the signal
x (n) = a"u(n) + bnu( n 1)
Solution

From definition (3.1.1) we have

b"z " =

X(z) =
n=0

n = oc

+ Y ib -'z)1
n=0

i= l

The first pow er series converges if locz-11 < 1 or |z| > |a |. The second power series
converges if \b~xz\ < 1 or |z| < |6j.
In determ ining the convergence of XCz), we consider two different cases.

158

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Case 1 |b| < |a |:

In this case the two R O C above do not overlap, as shown
in Fig. 3.4(a). Consequently, we cannot find values of z for which both power series
converge simultaneously. Clearly, in this case, X(c) does not exist.
Case 2 |f>| > |a |:
In this case there is a ring in the z-plane where both power
series converge simultaneously, as shown in Fig. 3.4(b). Then we obtain
X(z) =

1 ccz 1
1 bz 1
b a

(3.1.10)

a + b z abz~l
The R O C of X(z) ts |cr| < \z\ < \b\.
T h is e x am p le show s th a t i f there is a R O C f o r an infinite durat i on two-si ded
signal, it is a ring ( annul ar region) in the z-plane. F ro m E x a m p le s 3.1.1, 3.1.3, 3.1.4,
an d 3.1.5. w e see th a t th e R O C of a signal d e p e n d s o n b o th its d u ra tio n (finite
or in fin ite) an d o n w h e th e r it is cau sal, a n tic a u sa l, o r tw o -sid ed . T h e se facts are
su m m a riz e d in T a b le 3.1.
O n e special case of a tw o -sid ed signal is a signal th a t h as infinite d u ra tio n
on th e rig h t sid e b u t n o t on th e left [i.e., x( n ) = 0 fo r n < (l < 0], A sec
on d case is a signal th a t has infinite d u ra tio n o n th e left side b u t n o t on the

plane

Ifcl < tot

X(z) does not exisl

Im(;)
krl < IAI

ReU)

ROC for X(z)

Figure 3.4 R O C fo r z-transform in
E xam ple 3.1.5.

Sec. 3.1

159

The ^-Transform
TABLE 3.1

CH A R A C TE R IS TIC FAM ILIES O F SIGN ALS W IT H T H E IR

C O R R E S P O N D IN G ROC

Signal

ROC
Finite-Duration Signals

Two-sided

. . TTT l i t ,

Jnfinite-Duration
Causal

l l T t*

Anltcausal

T] T1
Two-sided

I . I t.....
rig h t [i.e., x{ n ) = 0 fo r n > n\ > 0]. A th ird sp ecial case is a signal th a t has
finite d u ra tio n o n b o th th e left a n d rig h t sides [i.e., x ( n ) = 0 fo r n < no < 0
a n d n > n\ > 0]. T h e se ty p e s o f signals a re so m e tim e s c alled right-sided, left
sided, a n d finite-d uration two-sided, signals, resp ec tiv ely . T h e d e te rm in a tio n o f th e
R O C fo r th e s e th r e e ty p es o f signals is left as an e x ercise fo r th e r e a d e r (P r o b
lem 3.5).
F in ally , w e n o te th a t th e z -tra n sfo rm d efin ed b y (3.1.1) is so m e tim e s re fe rre d
to as th e tw o-sided o r bilateral z-transform , to d istin g u ish it fro m th e o ne-sided o r

160

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

unilateral z- t ransf orm given by

X~(z) = ^ * ( ) z ~ "

(3.1.11)

T h e o n e-sid ed z -tra n sfo rm is ex am in ed in S ection 3.5. In this tex t w e use the

ex p ressio n z-tra n sfo rm exclusively to m ean th e tw o -sid ed z -tra n sfo rm d efined by
(3.1.1). T h e te rm 'tw o -sid e d ' will b e u se d oniy in cases w h ere w e w an t to resolve
any am b ig u ities. C learly , if x ( n ) is causal [i.e., * ( 72) = 0 for n < 0], th e o n e-sid ed
and tw o -sid ed z-tran sfo rm s a re eq u iv a le n t. In any o th e r case, th e y a re d ifferen t.

3.1.2 The Inverse z-Transform

O ften , we h av e th e z-tra n sfo rm X ( z ) of a signal a n d w e m ust d e te rm in e th e signal
se q u en ce. T h e p ro c e d u re fo r tra n sfo rm in g from th e z-d o m ain to th e tim e dom ain
is called th e inverse z- t ransform. A n in v ersio n fo rm u la for o b ta in in g x( n) from
X (z) can b e d eriv ed by using th e Cau c hy integral t h e o r e m , w hich is an im p o rta n t
th e o re m in th e th e o ry o f co m p lex variables.
T o b egin, we h av e th e z-tra n sfo rm defin ed by (3.1.1) as

S u p p o se th a t we m u ltip ly b o th sides o f (3.1.12) by z"~' an d in te g ra te both sides

o v er a closed c o n to u r w ithin the R O C o f X ( z ) w hich en clo ses th e origin. Such a
c o n to u r is illu stra te d in Fig. 3.5. T h u s we have
(3.1.13)
w h e re C d e n o te s th e clo sed c o n to u r in th e R O C o f A'(z). ta k e n in a c o u n te rc lo c k
w ise d irectio n . Since th e se rie s c o n v erg es on th is c o n to u r, w e can in te rc h a n g e
th e o rd e r o f in te g ra tio n an d su m m atio n o n th e rig h t-h a n d side o f (3,1.13). T h u s
im (o

Figure 3.5
(3.1.13).

Contour C for integral in

Sec. 3.2

Properties of the z-Transform

161

(3.1.13) b eco m es
x ( z ) z n- ' d z =
t= 00

x ( k ) ( h z n- l~kd z
*

(3.1.14)

N o w w e can in v o k e th e C a u ch y in te g ra l th e o re m , w hich s ta te s th a t

ftM5)
w h ere C is an y c o n to u r th a t en clo ses th e origin. B y ap p ly in g (3.1.15), th e righth a n d side o f (3.1.14) re d u c e s to 2 n j x ( n ) a n d h en c e th e d e sire d in v e rsio n fo rm u la
x ( n ) = ^ - ^ X ( z ) z n_I d z

(3.1.16)

A lth o u g h th e c o n to u r in te g ra l in (3.1.16) p ro v id es th e d e s ire d in v ersio n fo r

m u la fo r d e te rm in in g th e se q u e n c e * ( ) fro m th e z-tra n sfo rm , w e sh all n o t use
(3.1.16) d irectly in o u r e v a lu a tio n o f in v erse z -tran sfo rm s. In o u r tre a tm e n t w e d eal
w ith signals a n d sy stem s in th e z-d o m ain w hich h av e ra tio n a l z -tra n s fo rm s (i.e., ztra n sfo rm s th a t a re a ra tio o f tw o p o ly n o m ials). F o r such z -tra n sfo rm s w e d e v e lo p a
s im p ler m e th o d fo r in v ersio n th a t ste m s from (3.1.16) and e m p lo y s a ta b le lo o k u p .

3.2 PROPERTIES OF THE Z-TRANSFORM

T h e z -tra n sfo rm is a v ery p o w erfu l to o l fo r th e stu d y o f d isc re te -tim e signals an d
system s. T h e p o w e r o f th is tra n sfo rm is a c o n s eq u en ce o f so m e v ery im p o rta n t
p ro p e rtie s th a t th e tra n sfo rm possesses. In th is sectio n w e e x am in e som e o f th e se
p ro p e rtie s .
In th e tre a tm e n t th a t follow s, it sh o u ld b e re m e m b e re d th a t w h e n w e c o m b in e
sev eral z -tra n sfo rm s, th e R O C o f th e o v erall tra n sfo rm is, at least, th e in te rse c tio n
o f th e R O C o f th e in d iv id u al tra n sfo rm s. T h is will b eco m e m o re a p p a re n t la te r,
w h en w e discu ss specific ex am p les.

Linearity.

If
Jt,(n)

Xi ( z )

an d
x 2(n) < -U X 2(z)
th e n
x( n) = a \ x \ ( n ) + a 2x 2(n)

X( z ) = a j Xj ( z ) -h o2X 2(z)

(3.2.1)

fo r an y c o n s ta n ts a i a n d a 2. T h e p ro o f o f th is p ro p e rty follow s im m e d ia te ly from

th e d efin itio n o f lin e a rity a n d is left as an ex ercise fo r th e re a d e r.
T h e lin e a rity p ro p e rty can easily be g e n e ra liz e d fo r an a r b itra r y n u m b e r o f
signals. B asically , it im p lies th a t th e z -tra n sfo rm o f a lin e a r c o m b in a tio n o f signals
is th e sa m e lin e a r c o m b in a tio n o f th e ir z-tran sfo rm s. T h u s th e lin e a rity p ro p e rty
h e lp s u s to find th e z -tra n s fo rm o f a signal by ex p ressin g th e signal as a su m o f
e le m e n ta ry signals, fo r ea c h o f w hich, th e z -tra n sfo rm is a lre a d y k n o w n .

162

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Example 3.2.1
Determ ine the z-transform and the R O C of the signal
*() = [3(2") -4 (3 ")] ( )
Solution

If we define the signals

jti() = 2"u(n)

and
x2(n) = 3"u(n)
then

jc()

can be written as
x(n) = 3xi(n) 4 x2(n)

According to (3.2.1), its z-transform is

X(;) = 3 X ,( c ) - 4 X 2(z)
From (3.1.7) we recall that
a'u(n)

1 - az

ROC: |z| > |a|

(3.2.2)

By setting a = 2 and or = 3 in (3.2.2). we obtain

Ai(n) = 2u(n)

X,(z) = ^

x2(n) = 3nu(ri)

X 2(z) ~ ^ __

ROC: |zj > 2

,

ROC: |zl > 3

The intersection of the ROC of X,(z) and X2(:) is |z| > 3. Thus the overall transform
X (z) is
ROC: |;| > 3

Example 3.2.2
Determ ine the z-transform of the signals
(a) x(n) = (COSoon)u(n)
(b) x(n) = (sin w^n)u(n)
Solution
(a) By using E uler's identity, the signal .t(n) can be expressed as
x (n) = (cosa>on)u(n) = \ e Jaln"u(.n) + ^e~JUJnu(n)
Thus (3.2.1) implies that
X( z ) = \ Z { e ^ u ( n ) ) + ^ { g - ^ u i n ) }

Sec. 3.2

Properties of the z-Transform

163

If we set a = e^^O ori = \ei<u\ = 1) in (3.2.2), we obtain

eJUV"u(fi)

---------------

R O C : |z[ > 1

e - ,wnnu(n) ^

-------- T T T

R O C : l; l >

and
1-

Thus
X(z) = -------- ----------h I ------- - ------r
2 1 ^ f z -1
2 1 e~J^lz~I

RO C : |;1 > 1

A fter some simple algebraic manipulations we obtain the desired result, namely,
:

1 C_ ! COSW()

(coscO(}f?)u(n) < - - ;---------------- 1 - 2z~1cos ton +

ROC: |-1 > 1

,, ,, ,

L r2.j)

(b) From E ulers identity,

x(n) = (sino<o)(n) = ^-\eiun" u(n) e~,w""u()]
2j

Thus
X(-) = ^ - f - ------1------- - ------- ------- - |
2j \ 1
r _1
1

ROC: |:| > 1

and finally.
z sm OH)
(sin coi,n)u(n) x - ^ ----- - -----------------1 - 2z_1 cos wo + z~-

Time shifting.

ROC: |;| > 1

(3.2.4)

If
x( n)

X (c)

th e n
x{n - k ) z ~ kX{ z )

(3.2.5)

T h e R O C o f z ~kX ( z ) is th e sam e as th a t o f X ( z ) e x c e p t fo r j = 0 if k > 0 and

z = oo if k < 0. T h e p r o o f o f th is p ro p e rty follow s im m e d ia te ly fro m the definition
o f th e ^ -tra n sfo rm given in (3.1.1)
T h e p r o p e rtie s o f lin e a rity an d tim e shifting a re th e k ey fe a tu re s th a t m ake
th e z -tra n s fo rm e x tre m e ly u se fu l fo r th e analysis o f d isc re te -tim e L T I system s.
Exam ple 3*23
By applying the time-shifting property, determ ine the z-transform of the signals .T2(n,i
and xi(n) in Exam ple 3.1.1 from the Z'transform of Jti(n).
Solution

It can easily be seen that

x 2( n ) = * ]( + 2)

164

The ^-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

and

*i(n) = j:i(n - 2)
Thus from (3.2.5) we obtain
X2(z) = z2X, (z) = r + 2; + 5 + 7z ' 1 + z3
and
X 3(z) = r 2X,(z) = z"2 + 2z"3 + 5z-4 + 7z"s + z-7
Note that because of the multiplication by z2, the R O C of X2(z) does not include the
point z = oc, even if it is contained in the R O C of A^i(z).
E x am p le 3.2.3 p ro v id es a d d itio n a l insight in u n d e rs ta n d in g th e m e an in g of
th e shiftin g p ro p e rty . In d e e d , if we recall th a t th e coefficient o f z ~ n is th e sam ple
v alu e at tim e n, it is im m e d ia te ly se en th a t d elay in g a signal by k( k > 0) sam ples
[i.e., x ( n )
x ( n A')] c o rre sp o n d s to m u ltip ly in g all te rm s o f th e z -tra n sfo rm by
z~ k. T h e co efficien t o f z~" b ec o m e s th e coefficient o f ~~tn+k).
Example 3.2.4
Determ ine the transform of the signal
f 1,
' " H o .

0 < n < jV - 1
e lse w h e r e

,
<316>

Solution We can determ ine the z-transform of this signal by using the definition
(3.1.1). Indeed,
* -i
f N,
if z = l
X(z) = ^ l z ^ = l + ; - ' + - - - + z - (Af- l l =
l - r *
-f . , ,
(3.2.7)
*=<'
I 1 - z-1' 1
Since .v(n) has finite duration, its R O C is the entire z-plane, except z 0.
Let us also derive this transform by using the linearity and time shifting prop
erties. Note that x in) can be expressed in terms of two unit step signals
x(n) = u(n) u(n N)
By using (3.2.1) and (3.2.5) we have
X(z) = Z{u(n)} - Z{u(n - N)) = (1 - z"*')Z{u ()}

(3.2.8)

However, from (3.1.8) we have

Z[u(n)) = ^

ROC: jzl > 1

which, when combined with (3.2.8), leads to (3.2.7).

E x a m p le 3.2,4 h e lp s to clarify a v ery im p o rta n t issue re g a rd in g th e R O C
o f th e c o m b in a tio n o f se v era l z-tran sfo rm s. If th e lin ear c o m b in a tio n of several
sig n als h as finite d u ra tio n , th e R O C o f its z -tra n sfo rm is exclusively d ic ta te d by the
fin ite -d u ra tio n n a tu re o f th is signal, n o t by th e R O C o f th e in d iv id u al tran sfo rm s.

Scaling in the z-domain.

x{n)

If
X{z )

ROC: ri < [z| < r2

Sec. 3.2

165

Properties of the z-Transform

th en
a nx ( n )

X (c ~ z)

R O C : \a\r\ < |z| < \a\r2

(3.2.9)

for a n y c o n s ta n t a, re a l o r com plex.

Proof. F ro m th e d efin itio n (3.1.1)
OC

Z{a" x( n) } =

OC

a nx ( n ) z " =

x( n) ( a z) n

\o\r i < I;I < \a\r2

T o b e tte r u n d e rsta n d th e m ean in g a n d im plicatio n s o f th e scaling p ro p e rty ,
w e e x p re ss a a n d z in p o la r form as a = rae->a", z = r e i<0, a n d w e in tro d u c e a new
co m p lex v a ria b le w = a~^z- T h u s Z { x ( n ) ) = X ( z ) an d Z{a" x( n) } = X (if). It can
easily b e seen th a t

T h is ch a n g e o f v a ria b le s re su lts in e ith e r sh rin k in g (if r 0 > 1) o r e x p a n d in g (if

r 0 < 1) th e z -p lan e in co m b in a tio n w ith a ro ta tio n (if too # 2i-jr ) o f th e z-p lan e
(see Fig. 3.6). T h is e x p lain s w hy w e h av e a ch an g e in th e R O C o f th e n ew tra n sfo rm
w h e re |a | < 1. T h e case \a\ = 1, th a t is, a = e^w" is o f special in te re st b e c a u se it
c o rre sp o n d s o n ly to ro ta tio n o f th e z-p lan e.
Exam ple 3.2.5
Determ ine the z-transforms of the signals
(a) x(n) = a"(cosu\in)u(n)
(b) x(n) = tf"(sinion)u(n)
r-planc

w-plane

Im(-)

(W-C^o
0

Figure 3.6
o _ 1 Z,

Re(z)

Re(w)

Mapping of the r-plane to the u -plane via the transformation ui =

a roe^.

166

The z-Transform and Its Application to the Analysis of LTI S ystem s

Chap. 3

Solution
(a) From (3.2.3) and (3.2.9) we easily obtain
1 - a ; -1 cos wn
a

(C O S a i |, ) u ( r t )

(3.2.10)

*-------* --------- ---------- ;--------------------------

1 l a z r 1 co s ton -f a

(b) Similarly, (3.2.4) and (3.2.9) yield

a r -1 sin to<i
1 2az~ cos to,, + a

a (sin wii/i)u(n) * --- ---- ;---------

Time reversal.

|z! > la I

(3.2.11)

If
a (//) *-> X ( z )

R O C : r\ <

< r;

th en
j r ( - n ) <-i-> X { z ~ ])

R O C : < |z < r2

(3.2.12)

r\

P r o o f F ro m th e d efin itio n (3.1.1), w e have

Z{a (/;)) =

Y
h= - x.

-x { - !i ) z~" Y
' ( h ( z ~ t )~l Ar (z~ ')
/= ->;

w h ere th e ch an g e o f v ariab le / = n is m ad e. T h e R O C o f A '( ;_1) is

< |c 1f < r-i

o r e q u iv alen tly

< ];| <

r2
n

N o te th a t th e R O C fo r x( n) is the inverse o f th a t fo r x ( n). T h is m ean s th a t if co

b elo n g s to th e R O C o f x( n) , th en l/" o is in th e R O C fo r x ( n).
A n in tu itiv e p ro o f o f (3.2.12) is th e follow ing. W h en w e fold a signal, the
co efficien t o f z ~n b e co m es th e coefficient o f z n. T h u s, fo ld in g a signal is eq u iv alen t
to rep lacin g ; by
in th e z-tra n sfo rm fo rm u la. In o th e r w o rd s, reflectio n in the
tim e d o m ain c o rre sp o n d s to inversion in th e z-d o m ain .
Exam ple 3.2.6
Determ ine the z-transform of the signal
x i n ) = u( n)

Solution

It is known from (3.1.8) that

u(n) <-U i

ROC:

|zl

> 1

By using (3.2.12), we easily obtain

u(n)

* ------

Differentiation in the z-domain.

ROC: |z < 1

If

(3.2.13)

167

Properties of the z-Transform

Sec. 3.2

then
n x { n ) < >- z ^ ^ y
dz

(3.2.14)

Proof . B y differentiating both sid es o f (3.1.1), w e have

dX{z)
n~oc

n=oc

- z ~ l Z{nx(n)}
N o te that b oth transform s have the sam e RO C .

Example 3.2.7
D eterm ine the z-transform of the signal
jr(n) = na"u(n)

Solution The signal j:(n) can be expressed as njcitn), where Xj(n) = a"u(n). From
(3.2.2) we have that
jri(rt) = aKu(n) < - X,(z) = - ----- 1 az~'

ROC: |z| > \a\

Thus, by using (3.2.14), we obtain

n<j"ii(n)

dXji z)
a : '1
X(z) = - z ^ - = ----------- r-r
az

(1 - az~' V

ROC: |z| > |a|

(3.2.15)

If we set a = 1 in (3.2.15), we find the z-transform of the unit ram p signal

ntt(n)

ROC: jz| > 1

(3.2.16)

Example 3.2.8
D eterm ine the signal x(n) whose z-transform is given by
X(z) = log(l -t- o z '1)

|z! > |o(

Solution By taking the first derivative of X(z), we obtain

dX{z)
- a z ~2
dz 1 + az~l
Thus
dX(z)
dz

= az

1 - (- a ) z ->

>

\a]

The inverse z-transform of the term in brackets is (-a) ". The multiplication by
z _1 implies a time delay by one sample (time shifting property), which results in
( - a ) " _ 1u(n 1). Finally, from the differentiation property we have
nx(n ) = a ( a)n~lu(n 1)

168

The z -Transfomn and Its Application to the Analysis of LTI Systems

Chap. 3

x ( n) = ( - 1 } " +1 u(n - 1)

Convolution of two sequences.

If

Xi (rt)

X^z)

x 2(n)

X 2 {z)

th e n

x( n) = jfi(n) * x 2 (n)

X ( z ) = X \ ( z ) X 2 (z)

(3.2.17)

T h e R O C o f X (;) is, at least, th e in te rse c tio n of th a t fo r ATi(c) an d AS(z).

Proof. T h e co n v o lu tio n o f x i(n ) a n d x 2 (n) is d efin e d as
OC

x(n) =

x i ( k ) x 2(n - k)

i' = oc

T h e z-tra n sfo rm o f x( n ) is
oc

AT{;) =

oc

Y 2 x( n) z ~ " =

ac

Y ,

x \ ( k ) x 2(n - k)

n= - o c |_Jt = -cc

n- x

U p o n in te rch an g in g th e o rd e r o f the su m m a tio n s a n d ap p ly in g th e tim e-sh iftin g

p ro p e rty in (3.2.5). we o b tain
X{z) =

x 2(n k) z "

x ' {k)

= X 2 (z) Y

^ ( k ) z ~ k = X 2 ( z ) X ](z)

Example 3.2.9
Compute the convolution x(rt) of the signals
jt,(n) = { 1 .-2 ,1 )
_ f 1.

0 < n < 5

10 ,

elsewhere

Solution From (3.1.1), we have

* ,( ; ) = 1 - 2 ; ' + z ~ 2
X 2(z) = I + z ~ ' + z~2 + j ' 3 + z"4 +
According to (3.2.17), we carry out the multiplication of X\(z) and X 2(z). Thus
X(z) = X 1 (z)X 1 (z) = 1 - z "1 - z ~6 + z "7
Hence
x ( n) = { 1 . - 1 . 0 , 0, 0, 0 , - 1 , 1 }
t

Sec. 3.2

Properties of the z-Transform

169

The same result can also be obtained by noting that

X,U) = (1 - ; ~ 'r
x 2w

1 - ; -6
= 3 p

Then
- z "6 + :" 7

X(z) = (1 - ; - )(l - z~*) = 1 -

The reader is encouraged to obtain the same result explicitly by using the convolution
summation formula (tim e-domain approach).
T h e co n v o lu tio n p ro p e rty is o n e o f th e m o st p o w erfu l p ro p e rtie s o f th e ztra n sfo rm b ec a u se it c o n v e rts th e co n v o lu tio n o f tw o signals (tim e d o m a in ) to
m u ltip lic a tio n o f th e ir tra n sfo rm s. C o m p u ta tio n of th e c o n v o lu tio n o f tw o signals,
using th e z -tra n sfo rm , re q u ire s th e follow ing steps:
1. C o m p u te th e z -tra n s fo rm s o f th e signals to be co n v o lv ed .
X i(z) = Z{ x \ ( n ) \
(tim e d o m ain * -.-dom ain)
X 2 (z) = Z { x 2 (n) ]

2 . M u ltip ly th e tw o z -tran sfo rm s.

X (z) = X ,(z )X 2(;)

(z-d o m ain )

3. F in d th e in v e rse z -tra n s fo rm o f X (z).

x ( n ) = Z _ , {X(z))

(z-d o m ain tim e d o m a in )

T h is p ro c e d u re is, in m a n y cases, c o m p u ta tio n a lly e a s ie r th a n th e d irect e v a l

u a tio n o f th e co n v o lu tio n su m m atio n .

Correlation of two sequences.

If

xj(n)

X i(z)

x 2 (n)

X 2 (z)

th e n
OC

Rx,J2 (z) = X 1 (Z)X 2(Z"1)

OC

Proof . We recall that

rXix2U) = x\ ( l ) * x 2( - l )

(3.2.18)

170

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

U sing th e co n v o lu tio n an d tim e-rev ersa l p ro p e rtie s , we easily o b ta in

= Z{x] ( / ) }Z{xz ( /)} = X l ( z ) X 2 ( z ~1)
T h e R O C o f R X}X2(z) is at least the in te rse c tio n o f th a t fo r X i(c ) an d X 2("- 1 )A s in th e case o f co n v o lu tio n , th e c ro ssc o rre la tio n o f tw o signals is m o re
easily d o n e via p o ly n o m ial m u ltip lic atio n acco rd in g to (3.2.18) a n d th en inverse
tra n sfo rm in g th e resu lt.
Exam ple 3.2.10
Determ ine the autocorrelation sequence of the signal
x (n ) =

a " u {n ).

1 <

< 1

Since the autocorrelation sequence of a signal is its correlation with itself,

(3.2.18) gives
S o lu tio n

Rxs(z) = Z{rxJ I ) ) = X (z)X (;~l )

From (3.2.2) we have
ROC: |z| > \a\

Ale) = ---------- J - az~'

(causal signal)

and by using (3.2.15). we obtain

| = -....
1 - az

ROC: |;| <

(a|

(anticausal signal)

Thus
= - = ------------ ------------ 1 az 1 1 - az
1 a (; + ; ) + fl

ROC: \a\ < |;| <

|c|

Since the R O C of Rxx(z) is a ring, rIX{i) is a two-sided signal, even if x(n) is causal.
To obtain rsJ l ) , we observe that the ^-transform of the sequence in Exam
ple 3.1.5 with b = 1 /a is simply (1 a1)Rxx(z). Hence it follows that
rXI(l) = -- a ^

oc < I < oo

\ a-

The reader is encouraged to compare this approach with the time-dom ain solution of
the same problem given in Section 2.6.

Multiplication of two sequences.

If

x \ ( n)

ATi(z)

x 2 (n)

X 2 {z)

th e n
r(n) = x i ( n ) x 2 (n)

X(z) =

( ) v~ ^ v

(3.2.19)

w h ere C is a clo sed c o n to u r th a t e n clo ses th e o rig in a n d lies w ith in th e reg io n of

co n v erg en ce c o m m o n to b o th X](i>) a n d ^ ( l / u ) .

Sec. 3.2

171

Properties of the z-Transform

Proof . T h e z -tra n sfo rm o f J 3 (n) is

OC
X(z) = Y

OC

x( n)z ~n = Y

x ] ( n) x 2 ( n) z ~n

n = oc

ft

L e t us s u b s titu te th e in v erse tra n sfo rm

* i(n ) =
fo r
in th e z -tra n sfo rm X (z) a n d in te rc h a n g e th e o r d e r o f su m m a tio n an d
in te g ra tio n . T h u s w e o b ta in
1

" oc

X (z)= 2

v~ ^ v

T h e su m in th e b ra c k e ts is sim ply th e tra n sfo rm X 2 (:) e v a lu a te d a t z / v . T h e re fo re ,

X(z) = ^ - 6 x ^ x 2
2 n j Jc
w hich is th e d e s ire d result.
T o o b ta in th e R O C o f X (z) w e n o te th a t if X i(u ) co n v e rg e s fo r ru < |u| < r\
an d X 2 iz) c o n v erg es fo r r2! < |z| < r2ll, th e n th e R O C o f X 2 {z/v) is

H e n c e th e R O C fo r A"(z) is at least
r\ir2i < |z[ < r u r2u

(3.2.20)

A lth o u g h this p ro p e rty will n o t be used im m e d ia te ly , it will p ro v e u se fu l later,

esp ecially in o u r tr e a tm e n t o f filter design b ased o n th e w in d o w te c h n iq u e , w h ere
w e m u ltip ly th e im p u lse re sp o n s e o f an IIR system by a fin ite -d u ra tio n w in d o w
w h ich se rv es to tru n c a te th e im p u lse re sp o n se o f th e I I R sy stem .
F o r c o m p le x -v a lu e d se q u e n c e s .ti(rt) an d x 2 (n) w e c a n define th e p ro d u c t
s e q u e n c e as x( n ) = Jti (nj x^i n) . T h e n th e c o rre sp o n d in g c o m p lex co n v o lu tio n
in te g ra l b eco m es
x ( n ) = x i ( n ) x 2 {n)

X(z) =

v~ldv

(3.2.21)

T h e p r o o f o f (3.2.21) is left as an ex ercise fo r th e re a d e r.

Parsevals relation.
y

If jti(n ) an d x 2 (n) a re c o m p le x -v a lu e d se q u en ces, th e n

* i(n )x 2 (n) = l j ( j ^ X i ( v ) X 2

v ~ 'd v

(3.2.22)

p ro v id e d th a t r ^ r y < 1 < n ur2u, w h e re ry < |z| < r \ u a n d r ^ < |z| < r2u a re th e
R O C o f X ](z) a n d X 2(z). T h e p ro o f o f (3.2.22) follow s im m e d ia te ly by ev alu atin g
X ( z ) in (3.2.21) at z = 1.

172

The z-Transform and Its Application to the Analysis of LTI Systems

The Initial Value Theorem.

Chap. 3

If j ( ;i ) is causal [i.e.. x ( n ) = 0 fo r n < 0], th e n

.r(0) = lim X ( z )
:*sc

(3.2.23)

Proof. Since x{n) is causal. (3.1.1) gives

X (z) y x {n )z x (0) + x (1 )z 1 + x (2)z
=o

O b v iou sly , as z oc. z ~" 0 since n > 0 an d (3.2.23) follow s.

A ll th e p ro p e rtie s o f th e z -tra n sfo rm p r e s e n te d in th is s e c tio n are su m m arized
in T a b le 3.2 fo r easy re fe re n c e . T h ey are listed in th e sam e o rd e r as th ey have
b een in tro d u c e d in th e tex t. T h e c o n ju g atio n p ro p e rtie s a n d P a rse v a l's relatio n
are left as ex ercise s fo r th e re a d e r.
W e have now' d e riv e d m o st o f th e z -tra n sfo rm s th a t are e n c o u n te re d in m any
p ractical ap p licatio n s. T h e se z -tra n sfo rm pairs a re su m m a riz e d in T ab le 3.3 for
easy re fe re n c e . A sim ple in sp e ctio n o f th is ta b le show s th a t th e s e z-tran sfo rm s
are all rational f u n c t i o n s (i.e., ratio s o f p o ly n o m ials in z _1). A s will soon b ecom e
a p p a re n t, ratio n al z -tra n sfo rm s are e n c o u n te re d n o t only as th e z-tran sfo rm s of
v ario u s im p o rta n t signals b u t also in th e c h a ra c te riz a tio n of d isc re te -tim e lin ear
tim e -in v a ria n t sy stem s d esc rib e d by c o n s tan t-co efficien t d iffe re n c e e q u a tio n s.

3.3 RATIONAL Z-TRANSFORMS

A s in d ic a te d in S ectio n 3.2, an im p o rta n t fam ily o f z-tra n sfo rm s a re th o se fo r w hich
X ( z ) is a ra tio n a l fu n ctio n , th a t is. a ra tio of tw o p o ly n o m ials in z _l (o r z). In
this sectio n w e discuss som e very im p o rta n t issues re g a rd in g th e class o f ra tio n a l
z-tran sfo rm s.

3.3.1 Poles and Zeros

T h e zeros of a z -tra n sfo rm X (z) a re th e v alu es of z fo r w hich X (z) = 0. T h e pol es
o f a z-tra n sfo rm are th e v alu es o f z fo r w hich X (z) = oc. If X ( z ) is a ratio n al
fu n ctio n , th e n

J K ,) .
D( z)

ao + a i z 1 + ----- \ - aNz ~ h

= --------*

(3.3.1)

k=o
If ao / 0 an d bo ^ 0, w e can avoid th e n eg ativ e p o w e rs o f z by fa cto rin g o u t the
te rm s boz~M a n d a$z~N as follow s:

X(z)

N( z)

b 0z ~ M z M + {bx/ bo) z M~ x + + b M/ b 0

D( z)

a0z N Z N + ( a \ / a a) z fJ~ l H--------\ - aN/ a Q

TABLE 3.2

PROPERTIES OF THE Z-TRANSFORM

Property

Time Domain

z-Domain

R OC

Notation

Linearity

*(n)
xt (n)
^(n)
aixy(n) + a2x 2(n)

X(z)
Xi(z)
X 2(z)
<t\* l(z ) + 2X 2(z)

Time shifting

x{n - k)

z~kX(z)

Scaling in the z-domain

a"x(n)

X(a~' z)

ROC: r2 < |z) < r\

ROC,
ROC2
At least the intersection of R O Q
and R O C 2
T hat of X (z), except z = 0 if k > 0
and z = oo if k < 0
\a\r2 < |z| < |a|r[

Time reversal

x(~rj)

X(z~')

Conjugation
Real part
Imaginary part

x'(rt)
Relx(n)}
lm{x(n))

X' ( z ' )

ROC
Includes ROC
Includes ROC

Differentiation in the
z-domain
Convolution

nx(n)
x i ( n ) * x 2(n)

Jf|(z)X 2(z)

Correlation

rx,x2(l) = * i(0 * x 2 (~l)

/ W z ) = Xi<z)* 2<z_l)

Initial value theorem

If x(n) causal

.r(O) = lim X(z)

Multiplication

Xi(n)x 2(n)

2~J^X,(v)X2^ J v - ' d v

Parsevals relation

i[X (z) + X*U*)]

i[X (z )-* (; )]
dX( z )
Z dz

= 2? i i

5X ,( t)) ^ (l/tJ ) v - ' dv

r1

< |z| <

r2

r2 < |z| < r.

A t least, the intersection of ROCi
and R O C 2
A t least, the intersection of ROC of
X,(z) and -K^z 1)

A t least rj/ry < |z| < r]r2 a

174

The z-Transform and Its Application to the Analysis of LTI Systems

TABLE 3.3

SOME COMMON Z-TRANSFORM PAIRS

Signal. x(n)
1

u(n)

a"u(n)

A ll ;
1:1 > 1

\z\ > \a\

1 az~l

a:" 1

na"u(n )

Izt > !a|

( l - a z ^ )2

au(n 1)

na'u(n 1)
(cos aion)u(n)
(sin a>nn)u(n)
( a" cosiH ) n ) u ( n )

10

1 - r-1
1

ROC

: -Transform, X (c)

5{n)

Chap. 3

(a'1sini*in)u{n)

1
k l < la|

1 a : -1

a z~l

|;| < N

(1 - c : - 1)2
1 - Z '1 COSf^o

1 - 2; _1 costal + z ~2
sin

2 ;" 1 cos tun + z ~2

1 az~] cos a*)

1-

kl > 1

\z\ > 1

1 2az~ cos wo + a 2z ~2
a ; -1 sinaiii
1-

2a ; " 1 cos a*,

a 2z ~2

Since N ( z ) and D( z ) are polynom ials in z, they can be expressed in factored form as
X (-)

b() - - M + N

D( z )

~
~ Z2) (Z ~
(z - p i ) ( z - P i ) (z -

Zm )
p n

u
n " z*}

(3.3.2)

X ( z ) = G z N~ M^A ------------

n<* ~ pjt)

*=i

w here G = 6o/ao- Thus X (r) has M finite zeros at z = z\, zi , . , z m (th e roots o f
the num erator p olyn om ial), N finite p oles at z = p \ , p i .........P n (th e roots o f the
d enom inator p olyn om ial), and \N M\ zeros (if N > M ) or p o le s (if N < M ) at
the origin z = 0. P o les or zeros m ay also occur at z = 0 0 . A zero exists at z = oc if
X ( 0 0 ) = 0 and a p o le exists at z = oc if X ( 0 0 ) = oc. If w e count the p oles and zeros
at zero and infinity, w e find that X (z) has exactly the sam e num ber o f p o les as zeros.
W e can represent X ( z ) graphically by a p o l e - z e r o p l o t (or pat t ern) in the
com plex plane, which show s the location o f p oles by crosses ( x ) and the location
o f zeros by circles (o). T he m ultiplicity of m ultipie*order p o les or zeros is indicated
by a num ber clo se to the corresponding cross or circle. O b viou sly, by definition,
the R O C o f a z-transform should not contain any poles.

Sec. 3.3

175

Rational z-Transform s

Exam ple 33.1

D eterm ine the pole-zero plot for the signal
a >0

x(n) = a"u(n)
Solution

From Table 3.3 we find that

1_____

X(z) =

ROC; \z\ > a

- a

1 a ; -1

Thus
has one zero at n = 0 and one pole at pi = a. The pole-zero plot is
shown in Fig. 3.7. Note that the pole p\ = a is not included in the R O C since the
z-transform does not converge at a pole.

Re(;)

Figure 3.7

Pole-zero plot for the

causal exponential signal ,v() = a"imu

Exam ple 3 3 .2
D eterm ine the pole-zero plot for the signal

1 0,

Of n < W - 1
elsewhere

where a > 0.
Solution

From the definition (3.1.1) we obtain

^
X(z) = X W
"

ln

l - t a ; ' 1) "
z - a u
1 - az 1 = Sz T T t--------T
(z - a)

1}" = - r r ^ ------ r

Since a > 0, the equation z M = a M has M roots at

Z t =

a e ' 1* * ' "

it

0 , 1 . . . . . . M

The zero zo = a cancels the pole at z = a. Thus

w

(Z - Z\ ) ( Z - Zl ) (Z - Z j t f - i )

X(z) = ---------------- ^ - ----------------

which has M 1 zeros and M - 1 poles, located as shown in Fig. 3.8 for M = 8. Note
th at the R O C is the entire z-plane except z = 0 because of the M - 1 poles located
at the origin.

176

The z-Transfonm and Its Application to the Analysis of LTI Systems

Chap. 3

Im(c)

Red)

Figure 3.8 Pole-zero patlem for

the finite-duration signal x(n) = a",
0 < n < M l(a > 0). for M = 8.
C learly , if we are given a p o le - z e r o p lo t, w e can d e te rm in e X ( ), by using
(3.3.2), to w ithin a scaling fa c to r G. T his is illu stra te d in th e follow ing ex am p le.
Example 3.3.3
Determine the c-transform and the signal that corresponds to the pole-zero plot of
Fiji. 3.9.
Solution There are two zeros (M = 2) at Z] = 0, Z2 = r cosojo and two poles (N = 2)
al p] reJ , Pz = re~iw". By substitution of these relations into (3.3.2), we obtain
X(z) = (

Z(Z r C O S ttH i)

= G(c - re}wu)(z - re~>w")

(; - Pi)(c - pi)

ROC:

A fter some simple algebraic manipulations, we obtain

C O S a><)

Xiz) = G-.
1 - 2 r - ~ l costou + r1

ROC: 1-1 > r

From Table 3.3 we find that

x(n) = G(r" cosa>on)u(n)
F ro m E x a m p le 3.3.3, w e see th a t th e p r o d u c t (; p \ ) ( z P 2 ) resu lts in a
p o ly n o m ial w ith re a l coefficients, w h en p\ a n d p 2 a re c o m p lex co n ju g ates. In
Im(2)

Figure 3.9 P ole-zero p attern for

Exam ple 3.3.3.

Sec. 3.3

177

Rational z - T ransforms

g e n e ra l, if a p o ly n o m ial h a s re a l coefficients, its ro o ts a re e ith e r real o r o ccu r in

c o m p lex -co n ju g ate pairs.
A s w e h av e seen , th e ; -tran sfo rm X ( z ) is a com plex fu n c tio n of th e com plex
v ariab le z = R e ( z ) + j Im (r). O b v io u sly , |X (c )|, th e m a g n itu d e o f X ( z ), is a real
and p o sitiv e fu n ctio n o f c. S ince : re p re se n ts a p o in t in th e co m p lex p la n e , |X ( )|
is a tw o -d im e n sio n a l fu n c tio n an d d esc rib e s a su rfa c e . T h is is illu stra te d in
Fig. 3.10(a) fo r th e z -tra n sfo rm
7 - 1

- - 2

(a)

Figure 3.10 Graph of |X (;)| for the

;;-Transform in (3.3.3). [Reproduced with
permission fr om Introduction to Systems
Analysis, by T. H. Glisson, 1985 by
McGraw-Hill B ook Company.]

178

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

w hich h as o n e z e ro a t z\ = 1 a n d tw o p o le s a t p \ , p 2 = 0. 9eJ" /4. N o te the

high p e a k s n e a r th e sin g u larities (p o les) a n d th e d e e p v alley close to th e zero.
F ig u re 3.10(b) illu stra te s th e g rap h o f lX (z)j fo r z = eJO>.

3.3.2 Pole Location and Time-Domain Behavior for

Causal Signals
In th is su b sectio n w e c o n sid e r th e re la tio n b e tw e e n th e z -p la n e lo catio n o f a pole
p a ir a n d th e fo rm (sh a p e ) o f th e c o rre sp o n d in g signal in th e tim e d o m ain . T h e dis
cu ssio n is b a s e d g en erally o n th e collectio n o f z -tra n s fo rm p a irs given in T a b le 3.3
an d th e resu lts in th e p re c e d in g su b sectio n . W e d e a l exclusively w ith real, causal
signals. In p a rtic u la r, w e se e th a t th e c h a ra c te ristic b e h a v io r o f cau sal signals d e
p e n d s o n w h e th e r th e p o les o f th e tra n sfo rm a re c o n ta in e d in th e reg io n |z| < 1 ,
o r in th e reg io n |z| > 1, o r on th e circle |z| = 1. S ince th e circle jz| = 1 h as a
ra d iu s o f 1 , it is called th e unit circle.
If a real signal h as a z-tran sfo rm w ith o n e p o le , th is p o le h a s to b e real. T he
o n ly su ch signal is th e re a l e x p o n e n tia l
x ( n ) = a nu{n) ^

X ( z ) = ------ r
1 az

R O C : |z] > |o|

h av in g o n e z e ro a t zi = 0 an d o n e p o le a t pi a on th e re a l axis. F ig u re 3.11
x (n)

11

x(n )

1 1 ! TTt

x(n

x(n)

i l l

rt

I I I
x(n)

'1

n i

1
-oi
'

Figure 3.11 Time-domain behavior of a single-real pole causal signal as a function

of the location of the pole with respect to the unit circle.

Sec. 3.3

179

Rational z-Transform s

illu stra te s th e b e h a v io r o f th e signal w ith re sp e c t to th e lo c a tio n o f th e p o le r e l

ative to th e u n it circle. T h e signal is d ecay in g if th e p o le is inside the unit
circle, fixed if th e p o le is o n th e u n it circle, an d gro w in g if th e p o le is o u t
side th e u n it circle. In a d d itio n , a n eg ativ e p o le resu lts in a signal th a t a lte r
n a te s in sign. O b v io u sly , causal signals w ith p o les o u tsid e th e u n it circle b e
co m e u n b o u n d e d , cau se overflow in d igital system s, a n d in g e n e ra l, sh o u ld be
av o id ed .
A cau sal re a l signal w ith a d o u b le re a l p o le h as th e fo rm
x ( n ) = n a nu ( n )

(see T a b le 3.3) an d its b e h a v io r is illu stra te d in Fig. 3.12. N o te th a t in c o n tra s t to

th e sin g le-p o le signal, a d o u b le real p o le o n th e u n it circle re su lts in an u n b o u n d e d
signal.
F ig u re 3.13 illu stra te s th e case of a p a ir o f c o m p le x -c o n ju g a te poles. A c c o rd
ing to T a b le 3.3, th is co n fig u ratio n of p o le s resu lts in an e x p o n e n tia lly w eig h ted
sin u so id al signal. T h e d istan c e r of th e p o les fro m th e o rigin d e te rm in e s th e e n v e
lope o f th e sin u so id al signal an d th e ir angle w ith th e real p o sitiv e axis, its relative
freq u e n cy . N o te th a t th e a m p litu d e o f th e signal is gro w in g if r > 1, c o n stan t if
r 1 (sin u so id a l sig n als), a n d d ecaying if r < 1 .
x(n)

<

T T ....

xin)

T
0

!
1

Figure 3.12 Time-domain behavior of causal signals corresponding to a double (m = 2) real

pole, as a function of the pole location.

180

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Figw e 3.13 A pair of complex-conjugate poles corresponds to causal signals with

oscillatory behavior.

Finally, Fig. 3.14 show s the b eh avior o f a causal signal w ith a d ou b le pair of
p o les on the unit circle. T his reinforces the corresponding results in Fig. 3.12 and
illustrates that m ultiple p o les on the unit circle should b e treated with great care.
T o sum m arize, causal real signals w ith sim ple real p oles or sim ple com plexconjugate pairs o f p oles, w hich are in sid e or on the unit circle are always bounded
in am plitude. Furtherm ore, a signal with a p o le (or a com p lex-con ju gate pair
o f p o les) near the origin decays m ore rapidly than on e a ssociated with a pole
n ear (but inside) the unit circle. Thus the tim e behavior o f a signal depends
strongly on the location o f its p o les relative to th e unit circle. Z eros also af
fect the behavior o f a signal but n ot as strongly as p oles. F or exam ple, in the

Sec. 3.3

181

Rational z-Transform s

Figure 3.14 Causal signal corresponding to a double pair of complex-conjugate

poles on the unit circle.
case o f sin u so id al signals, th e p resen c e an d lo catio n o f ze ro s affects only th e ir
p h ase.
A t this p o in t, it sh o u ld be stressed th a t e v e ry th in g w e h av e said a b o u t causal
signals ap p lies as w ell to causal L TI system s, since th e ir im p u lse re sp o n se is a causal
signal. H en ce if a p o le of a system is o u tsid e the unit circle, th e im pulse resp o n se
of th e system b eco m es u n b o u n d e d and. co n se q u e n tly , th e sy stem is u n sta b le .

3.3.3 The System Function of a Linear Time-Invariant

System
In C h a p te r 2 w e d e m o n s tra te d th a t th e o u tp u t o f a (re la x e d ) lin e a r tim e -in v a ria n t
sy stem to an in p u t se q u e n c e x( n) can be o b ta in e d by co m p u tin g th e co n v o lu tio n
of jc(h) w ith th e u n it sa m p le resp o n se o f th e system . T h e co n v o lu tio n p ro p e rty ,
d eriv e d in S ectio n 3.2, allow s us to ex p ress th is re la tio n sh ip in th e z-d o m ain as
Y( z ) = m z ) X ( z )

(3.3.4)

w h ere Y( z ) is th e z -tra n sfo rm o f th e o u tp u t se q u en ce v(n), X (z) is th e z-tra n sfo rm

o f th e in p u t se q u e n c e x ( n ) an d H( z ) is th e z -tra n sfo rm o f th e u n it sa m p le resp o n se
h{n).
If w e k n o w h( n) an d x( n ) , w e can d e te rm in e th e ir c o rre sp o n d in g z-tra n sfo rm s
H ( z ) a nd X ( z ) , m u ltip ly th e m to o b ta in Y(z), a n d th e r e fo r e d e te rm in e y( n) by
e v a lu a tin g th e in v erse z -tra n sfo rm of K(z). A lte rn a tiv e ly , if w e k now x ( n ) an d we
o b se rv e th e o u tp u t y( n) of th e system , w e can d e te rm in e th e u n it sa m p le resp o n se
by first solv in g fo r H( z ) fro m th e re la tio n

a n d th e n e v a lu a tin g th e in v erse z -tra n sfo rm o f H( z ) .

Since
OC

(3.3.6)

182

The z-Transform and Its Application to the Analysis of LTI System s

Chap. 3

it is c le a r th a t H ( z ) re p re s e n ts th e z -d o m ain c h a ra c te riz a tio n o f a system , w h ereas

h{n) is th e c o rre sp o n d in g tim e -d o m a in c h a ra c te riz a tio n of th e system . In o th e r
w o rd s, H{z ) an d h( n) are e q u iv a le n t d e s c rip tio n s o f a system in th e tw o d o m ain s.
T h e tra n sfo rm H{ z ) is called th e s yst em f unct i on.
T h e re la tio n in (3.3.5) is p a rtic u la rly u sefu l in o b ta in in g H{ z ) w h en th e system
is d e s c rib e d b y a lin e a r c o n s tan t-co efficien t d iffe re n c e e q u a tio n o f th e fo rm
M
N
(3.3.7)
In th is case th e sy stem fu n ctio n can be d e te rm in e d d irectly fro m (3.3.7) by com
p u tin g th e z -tra n s fo rm o f b o th sid es o f (3.3.7). T h u s, by a p p ly in g th e tim e-sh iftin g
p ro p e rty , w e o b ta in
M
y u

) = - J 2 a*Y ^ z ~k + E

b iX (z )r

M
Y(z)
X(z)

o r, eq u iv alen tly ,
> z-*
(3.3.8)

T h e re fo re , a lin e a r tim e -in v a ria n t system d e s c rib e d by a c o n s ta n t-c o e ffic ie n t dif

fere n ce e q u a tio n h a s a ra tio n a l system fu n ctio n .
T h is is th e g e n e ra l fo rm fo r th e system fu n c tio n o f a sy ste m d e sc rib e d by a
lin e a r c o n s tan t-co efficien t d ifferen ce e q u a tio n . F ro m th is g e n e ra l fo rm w e o b tain
tw o im p o rta n t sp e cial fo rm s. F irst, if ajt = 0 fo r 1 < k < N , (3.3.8) re d u c e s to
M
H ( z ) = J 2 bkz ~k
k=0

(3.3.9)

In th is case, H ( z ) c o n ta in s M z ero s, w h o se valu es a re d e te r m in e d by the

system p a ra m e te rs {*}, a n d an M th -o rd e r p o le a t th e o rigin z = 0. S ince the
sy stem c o n ta in s o n ly trivial p o le s (a t z = 0) an d M n o n triv ia l z e ro s, it is called

Sec. 3.3

183

Rational z-Transform s

an all -zero syst em. C learly, such a system h as a fin ite -d u ra tio n im p u lse resp o n se
(F IR ), a n d it is called an F IR system o r a m oving av erag e (M A ) system .
O n th e o th e r h an d , if bk 0 fo r 1 < k < M, th e system fu n c tio n re d u c e s to

A'

*= 1

k
(3.3.10)

boz"

<io = l

at*.
In this case H ( z ) co n sists of N poles, w hose v alu es are d e te r m in e d by th e system
p a ra m e te rs {a*} a n d an /V th-order zero at th e o rig in z = 0- W e usually do not
m a k e re fe re n c e to th e se trivial zeros. C o n se q u e n tly , th e sy stem fu n c tio n in (3.3.10)
c o n ta in s o n ly n o n triv ia l p o les an d th e c o rre sp o n d in g sy stem is called an all-pole
syst em. D u e to th e p re se n c e o f poles, th e im p u lse re sp o n s e o f such a system is
infinite in d u ra tio n , a n d h e n c e it is an I IR system .
T h e g e n e ra l fo rm o f th e sy stem fun ctio n given by (3.3.8) c o n ta in s b o th poles
a n d zero s, and h e n c e th e c o rre sp o n d in g sy stem is called a p o l e - z e r o s y s t e m , with
N p o le s a n d M z ero s. P o les a n d /o r zero s at c = 0 an d z = oc a re im p lied b u t are
n o t c o u n te d ex p licitly. D u e to th e p re se n c e o f p o les, a p o le - z e r o system is an IIR
system .
T h e fo llo w in g ex am p le illu strates th e p ro c e d u re fo r d e te rm in in g th e system
fu n ctio n a n d th e u n it sa m p le resp o n se from the d iffe re n c e e q u a tio n .
E xam ple 3.3.4
D eterm ine the system function and the unit sample response of the system described
by the difference equation
v(n) =

Solution

1y(n -

1) + 2 x ( n )

By computing the -transform of the difference equation, we obtain

Y(z) = z - >Y(z) + 2X(z)

H ence the system function is

xt:l

I - Jz -1

This system has a pole at z = \ and a zero at the origin. Using Table 3.3 we obtain
the inverse transform
h(n) = 2(i)"u()

This is the unit sample response of the system.

184

The z-T ransform and Its Application to the Analysis of LTI Systems

Chap. 3

W e h av e n o w d e m o n s tra te d th a t ra tio n a l z -tra n s fo rm s a re e n c o u n te re d in

c o m m o n ly u se d sy stem s a n d in th e c h a ra c te riz a tio n o f lin e a r tim e -in v a ria n t sys
tem s. In S ectio n 3.4 w e d esc rib e se v e ra l m e th o d s fo r d e te rm in in g th e inverse
z -tra n sfo rm o f ra tio n a l fu n ctio n s.

3.4 INVERSION OF THE Z-TRANSFORM

A s w e saw in S ectio n 3.1.2, th e in v erse z -tra n s fo rm is fo rm ally given by
x ( n ) = - < x ( z ) z n~ 1d z
2
njjt

(3.4.1)

w h ere th e in te g ra l is a c o n to u r in te g ra l o v e r a clo sed p a th C th a t en clo ses the

o rig in an d lies w ith in th e reg io n o f c o n v e rg e n c e o f ^ ( z ) . F o r sim plicity, C can be
ta k e n as a circle in th e R O C o f X (z) in th e z-p lan e.
T h e re a re th re e m e th o d s th a t a re o fte n u se d fo r th e e v a lu a tio n o f th e inverse
z-tran sfo rm in practice:
1. D ire c t e v a lu a tio n o f (3.4.1), by c o n to u r in te g ra tio n .
2. E x p a n sio n in to a se rie s o f te rm s, in th e v a ria b le s z, an d z _1.
3. P a rtia l-fra c tio n ex p an sio n a n d ta b le lo o k u p .

3.4.1 The Inverse z-Transform by Contour Integration

In th is se ctio n w e d e m o n s tra te th e use o f th e C au ch y re sid u e th e o r e m to d e te rm in e
th e in v erse z -tra n sfo rm d irectly fro m th e c o n to u r in teg ral.

Cauchy residue theorem. L e t / ( z ) b e a fu n c tio n o f th e c o m p lex v ariab le

z an d C b e a clo sed p a th in th e z -p lan e. If th e d e riv a tiv e d f ( z ) / d z exists o n and
inside th e c o n to u r C a n d if / ( z ) has no p o le s a t z = zo, th e n

- L (f) J ! d z

2njjcz-zo

= |^
<Zl,)'
10,

(3.4.2)

if zo is ou tsid e C

M o re g en erally , if th e (k + l ) - o r d e r d e riv a tiv e o f / ( z) exists a n d / ( z ) h a s n o p o les

a t z = zo, th e n
1

-!>'
0,

d k- ' f ( z )

04.3)

if zo is o u ts id e C

T h e v alu es o n th e rig h t-h a n d sid e o f (3.4.2) a n d (3.4.3) a re c a lle d th e re sid u e s of

th e p o le a t z = zo- T h e re su lts in (3.4.2) a n d (3.4.3) a re tw o fo rm s o f th e Cauc hy
residue t heorem.
W e can ap p ly (3.4.2) a n d (3.4.3) to o b ta in th e v a lu e s o f m o re g en eral c o n to u r
in teg rals. T o b e specific, su p p o s e th a t th e in te g ra n d o f th e c o n to u r in te g ra l is

Sec. 3.4

Inversion of th e

2 - T ra n s fo rm

185

P(z) = f ( z ) / g ( z ) ~ w h e re f ( z ) h a s no p o les inside th e c o n to u r C an d g (z) is a

p o ly n o m ial w ith d istin ct (sim p le) ro o ts c i, ^ 2 . ___-n inside C. T h e n

(3.4.4)
n

1=1
w h ere
f(z)
(3.4.5)
A l (z) = ( z - z i ) P{ z ) = ( z - z l) -J - 1x
g( z)
T h e v alu es (A, (-;,)} a re re sid u e s o f th e c o rre sp o n d in g p o le s at z =
/ = 1, 2 , . . . . n.
H e n c e th e v alu e o f th e c o n to u r in te g ra l is e q u a l to th e sum o f th e resid u es o f all
th e p o le s in sid e th e c o n to u r C.
W e o b se rv e th a t (3.4.4) w as o b ta in e d by p e rfo rm in g a p a rtia l-fra c tio n e x p a n
sion o f th e in te g ra n d an d ap p ly in g (3.4.2). W h en g(z) has m u ltip le -o rd e r ro o ts
as w ell as sim p le ro o ts inside th e c o n to u r, th e p a rtia l-fra c tio n e x p a n sio n , w ith a p
p ro p ria te m o d ifica tio n s, an d (3.4.3) can b e used to e v a lu a te th e resid u es at th e
c o rre sp o n d in g p o les.
In th e case o f th e in v erse z -tra n sfo rm , w e h ave

[resid u e of X (z )z n 1 a t z
a ll p o l e s

U t)

in s id e

(3.4.6)

p ro v id e d th a t th e p o les {z,} a re sim ple. If X (z )r "_1 has no p o le s inside th e c o n to u r

C fo r o n e o r m o re v alu es o f n, th e n x (n) = 0 fo r th e s e values.
T h e fo llo w in g ex am p le illu stra te s th e e v a lu a tio n o f th e in v erse z-tra n sfo rm
by u se o f th e C a u ch y re sid u e th e o re m .
Exam ple 3.4.1
Evaluate the inverse z-transform of
X(z) = ---------- r
1 az~
using the complex inversion integral.
Solution

kl > kil

We have

where C is a circle at radius greater than |a|. We shall evaluate this integral using
(3.4.2) with f ( z ) = z". We distinguish two cases.

186

The ^-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

L If n > 0, f ( z ) has only zeros and hence no poles inside C. T he only pole inside
C is z = a. Hence
*(*) = /(zo) = a n

n > 0

2. If n < 0, / ( z ) = z" has an nth-order pole at z = 0, which is also inside C. Thus

there are contributions from both poles. For n = - 1 we have
x ( - l ) -=

1 S)
1
dz =
1
2 nj j c z ( z a)
z a

+ -

= 0

If n = 2, we have

2)

2 n j z H z - a ) dZ

dz ( z - f l )

= 0

By continuing in the same way we can show that *(n) = 0 for n < 0. Thus
x (n) = a"u(n)

3.4.2 The Inverse z-Transform by Power Series

Expansion
The basic idea in this m eth od is the follow ing: G iven a z-transform X ( z ) with its
corresponding R O C , w e can expand X (z) into a p ow er series o f the form
OO

X(z) =

cz~n

(3.4.7)

co

w hich con verges in the given R O C . T h en , by the u n iqu en ess o f the z-transform,
x ( n ) = c for all n. W hen X ( z ) is rational, the exp an sion can b e perform ed by
long division.
T o illustrate this tech n iqu e, w e w ill invert som e z-transform s involving the
sam e expression for X ( z ) , but different R O C . T his w ill also serve to em phasize
again the im portance o f the R O C in d ealing with z-transform s.
Exam ple 3A 2
D eterm ine the inverse z-transform of
1 1.5z_1 + 0.5z 2
when
(a) ROC: |z| > 1
(b) ROC: |z| < 0.5
Solution
(a) Since the R O C is the exterior of a circle, we expect x(n) to be a causal signal.
Thus we seek a power series expansion in negative powers of z. By dividing

Sec. 3.4

187

Inversion of the z-Transform

the num erator of X{z) by its denom inator, we obtain the power series
AUI = ! _ 3 _ -; +

= 1+

+ TE; "4 + " '

By com paring this relation with (3.1.1), we conclude that

Note that in each step of the long-division process, we eliminate the lowestpower term of c~*.
(b) In this case the ROC is the interior of a circle. Consequently, the signal x(n)
is anticausal. To obtain a power series expansion in positive powers of c. we
perform the long division in the following way:
2: 2 + 6c3 + 14c4 + 30cs + 62c* +
+ ill
1 - 3: + 2c:
3c - 2z 2
3c - 9c: + 6c3
l z 2 - 6c3
7 r - 21 c3 + 14c4
15c3 - 14c4
15c3 - 45c4 + 30cs
31c4 - 30c5
Thus
X (c) =

1_

1------:------= 2c: + 6c3 + 14c4 + 30c5 + 62cfi +

In this case x(n) = 0 for n > 0. By comparing this result to (3.1.1), we conclude
that
Jt(n) = {

62. 30. 14.6,2, 0. 0}

t

We observe that in each step of the long-division process, the lowest-power

term of c is eliminated. We emphasize that in the case of anticausal sig
nals we simply carry out the long division by writing down the two poly
nomials in reverse order (i.e., starting with the most negative term on the
left).

F ro m th is e x a m p le w e n o te th a t, in g e n eral, th e m e th o d o f long d ivision will

n o t p ro v id e a n sw ers fo r x( n) w h en n is larg e b e c a u se th e lo n g division b eco m es
ted io u s. A lth o u g h , th e m e th o d p ro v id es a d irect e v a lu a tio n o f x( n ) , a clo sed -fo rm
so lu tio n is n o t p o ssib le , ex cep t if th e resu ltin g p a tte r n is sim p le e n o u g h to infer
th e g e n e ra l te rm x ( n ) . H e n c e th is m e th o d is used only if o n e w ish e d to d e te rm in e
th e v a lu e s o f th e first few sa m p le s o f th e signal.

188

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Example 3.4.3
Determ ine the inverse z-transform of
X(z) = log(l + az_1)
Solution

|zj > |a|

Using the power series expansion for log(l

^

+ x ),

with

|jc| < 1,

we have

( - r v r "

Thus

n < 0

0,

Expansion of irrational functions into power series can be obtained from tables.

3.4.3 The Inverse z-Transform by Partial-Fraction

Expansion
In th e tab le lo o k u p m e th o d , w e a tte m p t to e x p ress th e fu n c tio n X (z) as a linear
c o m b in a tio n
X ( z ) = a j X] (;) + 02 X 2 ( 1 ) + - - + c/Kx fc(z)

(3.4.8)

w h ere X] ( ; ) , . . . . X k (z ) a re ex p ressio n s w ith in v erse tra n sfo rm s x \ ( n ) , . . . , x k W

av ailab le in a ta b le o f z-tra n sfo rm p airs. If such a d e c o m p o s itio n is possible,
th e n x( n) , th e in v erse z -tra n sfo rm o f X ( z ) , can easily b e fo u n d using th e lin earity
p ro p e rty as
x ( n) = at]Xi(n) + a 2x 2 (n) H--------\ - a Kx K (n)

(3.4.9)

T h is a p p ro a c h is p a rtic u la rly useful if X ( z ) is a ra tio n a l fu n ctio n , a s in (3.3.1). W ith

o u t loss o f g e n erality , w e assu m e th a t ao = 1, so th a t (3.3.1) can b e e x p ressed as
_ t o
D(z)

_ b +b,r' + -

+ t r

l + f l i Z - 1 H----- + a u z ~ N

N o te th a t if a 0 ^ 1. w e can o b ta in (3.4.10) fro m (3.3.1) by div id in g b o th n u m e ra to r

an d d e n o m in a to r by aoA ra tio n a l fu n ctio n o f th e form (3.4.10) is called p r o p e r if a N ^ 0 an d M < N.
F ro m (3.3.2) it fo llo w s th a t th is is e q u iv a le n t to saying th a t th e n u m b e r o f finite
zero s is less th a n th e n u m b e r o f fin ite p o les.
A n im p ro p e r ra tio n a l fu n ctio n ( M > N ) can alw ays b e w ritte n as th e sum of
a p o ly n o m ial an d a p r o p e r ra tio n a l fu n ctio n . T h is p ro c e d u re is illu stra te d by the
fo llow ing ex am p le.
Example 3.4.4
Express the im proper rational transform
l + 3 z - ' + n z - 2 + l 2-3
1

1 + 6Z

+
in terms of a polynomial and a proper function.

Sec. 3.4

189

Inversion of the z-Transform

Solution First, we note that we should reduce the num erator so that the term s ; -2
and c- *' are eliminated. Thus we should carry out the long division with these two
polynomials written in reverse order. We stop the division when the order of the
rem ainder becomes
Then we obtain
= 1 + 2: - i +

In g e n e ra l, an y im p ro p e r ra tio n a l fu n ctio n (M > N ) can b e e x p ressed as

X ( ;) = W

i = Co + C i:" 1 + ' ' ' + Cu~n Z ~im~N) +

(3 A ll)

T h e in v erse z -tra n s fo rm o f the p o ly n o m ial can easily b e fo u n d by in sp ectio n .

W e fo cu s o u r a tte n tio n on th e inversion o f p r o p e r ra tio n a l tra n sfo rm s, since any
im p ro p e r fu n c tio n can b e tra n sfo rm e d in to a p ro p e r fu n c tio n by using (3.4.11).
W e carry o u t th e d e v e lo p m e n t in tw o steps. F irst, we p e rfo rm a p a rtia l fra c
tio n e x p a n sio n o f th e p r o p e r ra tio n a l fu n ctio n a n d th e n w e in v ert each o f th e
term s.
L e t A"(c) b e a p ro p e r ra tio n a l fu n ctio n , th a t is,
* (.-) =
= - " - - ' " T ,1 +
+buZ^
D( z)
1 + ^ iC
+ * -t~

(3.4.12)

w h ere
aN ^ 0

M < N

an d

T o sim p lify o u r discu ssion w e e lim in ate n eg ativ e p o w ers of c by m ultip ly in g b o th

th e n u m e r a to r a n d d e n o m in a to r of (3,4,12) by z N. T h is resu lts in
L

_ y v -l

N -M

X{z) =

CA 4- a \ z N

w hich c o n ta in s only p o sitiv e p o w e rs o f

(3.4.13,

+ ------(- <a/v

Since N > M , th e fu n ctio n

,N -2 _i_____ l
(3,4.14)

: w + tiic w- 1 + - - - + flW

is also alw ays p ro p e r.

O u r ta sk in p e rfo rm in g a p a rtia l-fra c tio n ex p an sio n is to ex p ress (3.4.14)
o r, e q u iv a le n tly , (3.4.12) as a sum o f sim ple fractio n s. F o r th is p u rp o se w e first
fa c to r th e d e n o m in a to r p o ly n o m ial in (3.4.14) in to facto rs th a t c o n tain th e poles
P i, p 2, . . . , p n o f X (z). W e d istinguish tw o cases.

Distinct poles. S u p p o se th a t th e p o le s p \ , p 2 ........ p/v a re all d iffe re n t (dis

tin ct). T h e n w e se e k an ex p an sio n of th e fo rm
z

z p\

z P2

+ -. - +

( 3 .
z Pn

T h e p ro b le m is to d e te rm in e th e coefficients A i , A 2 , . - . , A s - T h e re a re tw o w ays
to so lv e th is p ro b le m , as illu stra te d in th e follow ing exam p le.

190

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Example 3.4 .5
D eterm ine the partial-fraction expansion of the proper function
(3'U 6 ,
Solution First we elim inate the negative powers, by multiplying both num erator and
denom inator by z2. Thus
.2

X i z ) = zV2 - 11.5z
< +i 0n.5
The poles of X(z) are p\ = 1 and P2 = 0.5. Consequently, the expansion of the form
(3.4,15) is
X(z)

z
Cz l)(z 0.5)

A\
z

A2
z 0.5

(3.4.17)

A very simple method to determ ine A[ and A 2 is to multiply the equation by the
denom inator term (z - l)(z - 0.5). Thus we obtain
z = ( z - 0 .5 M i + ( z - 1 ) A 2

(3.4.18)

Now if we set z = p\ = 1 in (3.4.18), we eliminate the term involving A2. Hence

1 = (1 -0 .5 )A ,
Thus we obtain the result A i = 2. Next we return to (3.4.18) and set z = p 2 = 0.5,
thus eliminating the term involving Ai, so we have
0.5 = ( 0 .5 - 1)A2
and hence Ai = 1. Therefore, the result of the partial-fraction expansion is
X(z)

2
z- 1

1
z - 0.5

(3.4.19)

T h e exam ple given ab ove suggests that w e can determ ine the coefficients A \,
A i , . . . , Afj , by m ultiplying b oth sides o f (3.4.15) by each o f the term s (z - Pk).
k = 1 , 2 , , . . , N , and evaluating the resulting exp ression s at the corresp on d ing pole
p osition s, p \ , p i .........P n T h u s w e have, in general,

(Z- Pl)X(;) = (z-wM.i+ ... + /lt+,..+ fa-P)^

z

z - PI

C onsequently, w ith z = Pk,

Ak =

(3.4.20) yield s the

( z ~ ^ )X (; )i
z

(3420)

z - Pn
Jtth coefficient as
k =

1, 2. .

(3.4.21)

\z-pl

Exam ple 3.4.6

Determ ine the partial-fraction expansion of
1 17- ^ 0 .5 ,-

,3A22)

Sec. 3.4

191

inversion of the z-Transform

Solution To eliminate negative powers o f ; in (3.4.22), we multiply both num erator

and denom inator by
Thus
X(z)
z+ 1
The poles of X(z) are complex conjugates
Px =

\ + J i

Pi

\ ~ j \

z2 - ; + 0.5

and
=

Since p\ ^ p 2- we seek an expansion of the form (3.4.15). Thus

* (:)

; + l

A]

Ai

(z-p i)(:-p 2)

z-P\

z-pi

To obtain A , and A2, we use the formula (3,4.21), Thus we obtain

(z-pi)X(z)
At = -----------

; + 1 I
- P

( z - p 2)X(z)
A- = ------------

2 \ ^ P1

Pi

; + l
-

?+

3+ J5-5+./5
1

:=P2

T h e ex p a n sio n (3,4.15) a n d the fo rm u la (3.4.21) h o ld fo r b o th real a n d c o m

p lex p o les. T h e o n ly c o n s tra in t is th a t all p o les be d istin ct. W e also n o te th at
A; = A*. It can b e easily seen th a t th is is a c o n s e q u e n c e o f th e fact th a t p 2 = p ' .
In o th e r w o rd s, comp l ex- conj ugat e pol es result in c o mpl ex- con j ugat e coefficients in
the part ial-fraction expansi on. T h is sim ple resu lt will p ro v e v ery u se fu l la te r in o u r
d iscu ssio n .
M u ltip le - o rd e r p o l e s .
If X U) h a s a p o le o f m u ltip lic ity /, th a t is, it co n ta in s
in its d e n o m in a to r th e fa c to r (z - pk)1, th e n th e e x p a n s io n (3.4.15) is n o lo n g er
tru e. In th is case a d iffe re n t ex p an sio n is n e e d e d . F irst, w e in v e stig a te th e case of
a d o u b le p o le (i.e., 1 = 2).
Exam ple 3.4.7
D eterm ine the partial-fraction expansion of

Solution

First, we express (3.4,23) in terms of positive powers of

in the form

* ( ;) =
z2
:
(z + 1)(; - l ) 2
X (z) has a simple pole at p\ = - 1 and a double pole pi = p$ = 1. In such a case the
appropriate partial-fraction expansion is

= ______ t ______ =

z
(z + l)(z - 1)2
z+ 1 z - l
(z -1 )2
The problem is to determ ine the coefficients A 1, A2, and A3.

(3424)

192

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

We proceed as in the case of distinct poles. T o determ ine Aj, we multiply both
sides of (3.4.24) by (z + 1) and evaluate the result at z = - 1 . Thus (3.4.24) becomes
(z + l)X (z)

z+ 1

z+ 1

z-1

( z - 1)2

------------------ = Ai + ------ - A 2 + ------- r - z A i

which, when evaluated at z = - 1 , yields

1
4

(z + 1)X(z)

A i = ------------------

Next, if we multiply both sides of (3.4.24) by (z - l ) 2, we obtain

=
Z

<3.4.25,

z+ 1

Now, if we evaluate (3.4.25) at z = 1, we obtain A 3. Thus

(z l) 2X(z) I

= 1

*
Li
2
The remaining coefficient Az can be obtained by differentiating both sides of
(3.4.25) with respect to z and evaluating the result at z = 1. Note that it is not
necessary formally to carry out the differentiation of the right-hand side of (3.4.25),
since all term s except A2 vanish when we set z = 1. Thus
d
A2 =
dz

(z l ) 2X(z)

T h e gen eralization o f the p rocedure in the exam p le ab ove to the case o f an

/th-order p ole ( z p k)! is straightforward. T h e partial-fraction expansion must
contain the terms
Mk

Alt
Z - Pk

( z - Pk)2

(z - Pk)1

T h e coefficients {A,*} can b e evalu ated through d ifferen tiation as illustrated in

E xam p le 3.4.7 for / = 2.
N ow that w e have perform ed the partial-fraction exp an sion , w e are ready to
take the final step in the inversion o f X (z). First, let us con sid er th e case in which
X ( z ) contains distinct poles. F rom the partial-fraction exp an sion (3.4.15), it easily
follow s that
X ( z ) A \ -------------r + A i -------------- + -)- A n --------------1 -

P\Z~1

1 - PlZ

(3.4.27)

1 - P n Z~ 1

T he inverse z-transform , x ( n ) = Z ~ i { X( z ) ) , can b e ob tain ed by inverting each

term in (3.4.27) and taking the corresponding linear com b ination . From T ab le 3.3
it fo llo w s that these term s can be inverted using th e form ula
( p k)nu{n),
- ( P k ) nu ( - n - 1),

if RO C : |*| > Ip*J

(causal signals)
if RO C: |z| < \P k \
(anticausal sign als)

Sec. 3.4

Inversion of the ^-Transform

193

If th e signal jc(n) is cau sal, the R O C is |r | > p max- w h e re p max = m ax{|/>i|,

\p2\.........IpnII- In th is case all te rm s in (3.4.27) re su lt in causal signal c o m p o n e n ts
an d th e signal x ( n ) is given by
jr(/i) = (A i + A -Pz --------f" A Np nN )u(n)

(3.4.29)

If all p o le s a re real. (3.4.29) is th e d e sire d e x p ressio n fo r th e signal j (). T h u s a

causal sig n al, h av in g a ; -tra n s fo rm th a t co n tain s real a n d d istin ct p o les, is a linear
c o m b in a tio n o f re a l e x p o n e n tia l signals.
S u p p o se n o w th a t all p o les a re d istin ct b u t som e o f th e m a re co m p lex . In
this case so m e o f th e te rm s in (3.4.27) re su lt in com plex e x p o n e n tia l c o m p o n en ts.
H o w e v e r, if th e signal x ( n) is real, w e sh o u ld b e ab le to re d u c e th e se te rm s in to
real c o m p o n e n ts. If x ( n ) is real, th e p o ly n o m ials a p p e a rin g in X (z) have real co
efficients. In th is case, as w e h av e seen in S ectio n 3.3, if pj is a p o le , its com plex
c o n ju g a te p j is also a p o le. A s w as d e m o n s tra te d in E x am p le 3.4.6, th e c o rre s p o n d
ing coefficien ts in th e p a rtia l-fra c tio n e x p an sio n a re also co m p lex co n ju g ates. T h u s
th e c o n trib u tio n o f tw o c o m p lex -co n ju g ate p o les is of th e form
x k (n) = \ A k {pt )n + A H p l )"]()

(3.4.30)

T h e se tw o te rm s can be c o m b in ed to form a real signal c o m p o n e n t. F irst,

we ex p re ss Aj an d Pj in p o la r form (i.e., a m p litu d e an d p h a se ) as
A* = \ Ak \eja'

(3.4.31)

Pi: =

(3.4.32)

w h ere a k an d fik a re th e p h a s e c o m p o n e n ts o f A k a n d p k. S u b stitu tio n o f th ese

re la tio n s in to (3.4.30) gives
x k{n) = IAi

+ e - -'(A"+ l]u(n)

or, eq u iv alen tly ,

x k(n) = 2|A *|r" c o s($ tn + a k)u(n)

(3.4.33)

T h u s w e co n clu d e th a t
Z - 1 ( - r + - ~ r ) = 2 \ A k \rR
k c o s ( f tn + a k) u(n)
\i - p kz ~ l
i - p;z~ v

(3.4.34)

if th e R O C is |zj > \ pk \ = rk .
F ro m (3.4.34) we o b se rv e th a t ea c h p a ir o f c o m p le x -c o n ju g a te p o le s in th e
z -d o m ain resu lts in a causal sin u so id al signal c o m p o n e n t w ith an e x p o n e n tia l e n
v elo p e. T h e d ista n c e rk o f th e p o le fro m th e o rig in d e te rm in e s th e e x p o n e n tia l
w eig h tin g (g ro w in g if r k > 1, d ecay in g if r k < 1, c o n s ta n t if rk = 1). T h e angle of
th e p o le s w ith re sp e c t to th e p o sitiv e re a l axis p ro v id e s th e fre q u e n c y o f th e sin u
so id a l signal. T h e zero s, o r e q u iv alen tly th e n u m e ra to r o f th e ra tio n a l tran sfo rm ,
affect o n ly in d ire c tly th e a m p litu d e an d th e p h a se o f x k (n) th ro u g h A k.
In th e case o f mul t i pl e p o les, e ith e r re a l o r co m p lex , th e in v erse tra n sfo rm
o f te rm s o f th e fo rm A j ( z p k)n is re q u ire d . In th e case o f a d o u b le p o le the

194

The 2 -Transform and Its Application to the Analysis of LTI Systems

Chap. 3

fo llow ing tra n sfo rm p a ir (see T a b le 3.3) is q u ite useful:

pz~x
(1 - p z ] )2

= n p nu( n)

(3.4.35)

p ro v id e d th a t th e R O C is |z| > \p\. T h e g e n e ra liz a tio n to th e case o f p o le s w ith

h ig h e r m u ltip licity is left as an exercise fo r th e re a d e r.
Example 3.4.8
Determ ine the inverse z-transform of
X U) =

1
1 - 1 .5 ;-' +0.5z~ 2

(a) RO C III > 1

(b) RO C Izl < 0.5
(c) RO C 0.5 < |z| < 1
Solution This is the same problem that we treated in Exam ple 3.4.2. The partialfraction expansion for X(z) was determined in Example 3.4.5. The partial-fraction
expansion of X(z) yields

<3-4-36>

To invert X(z) we should apply (3.4,28) for pi 1 and p 2 = 0.5. However, this
requires the specification of the corresponding ROC.

(a) In case when the R O C is |z| > 1, the signal x(n) is causal and both term s in
(3.4.36) are causal terms. According to (3.4.28), we obtain
x(n) = 2 (l)n(n) (0.5)"u(n) = (2 0.5 )u(n)

(3.4.37)

which agrees with the result in Example 3.4.2(a).

(b) When the R O C is

|z| < 0.5, the signal x(n) is anticausal. Thus both term s in
(3.4.36) result in anticausal components. From (3.4.28) we obtain

x(n) = [2 + (0.5)'I]u(n 1)

(3.4.38)

(c) In this case the ROC 0.5 < |z| < 1 is a ring, which implies that the signal x(n) is
two-sided. Thus one of the term s corresponds to a causal signal and the other
to an anticausal signal. Obviously, the given ROC is the overlapping of the
regions (z| > 0.5 and |z| < 1. Hence the pole p 2 = 0.5 provides the causal part
and the pole p\ = 1 the anticausal. Thus
x(n) = -2 (1 ) "u( - n - 1) - (0.5)"(n)

Example 3.4.9
D eterm ine the causal signal jc(n) whose z-transform is given by

(3.4.39)

Sec, 3.4

Inversion of the z-Transtorm

Solution

195

In Exam ple 3.4.6 we have obtained the partial-fraction expansion as

where
A, = Al = j - j
and

Pi = p = i
Since we have a pair of complex-conjugate poles, we should use (3.4.34). The
polar forms of Aj and p, are

Hence

Example 3.4.10
D eterm ine the causal signal x(n) having the ;-transiorm
X(z) =
Solution

(1 + ; - )(]

From Example 3.4.7 we have

X(Z)

3
+
+ T
41

41

.-1

+ 2 ( 1 - ; - 1)2

By applying the inverse transform relations in (3.4.28) and (3.4.35), we obtain

1
3
1
f 1
x( n ) = - ( l)"(n) + t ( ) + - n u ( n ) =
t(-1)

3
w*l
+ - + u(n)

3.4.4 Decomposition of Rational z-Transforms

A t th is p o in t it is a p p ro p ria te to discuss som e a d d itio n a l issues c o n c e rn in g th e
d e c o m p o s itio n o f ra tio n a l z-tran sfo rm s, w hich will p ro v e v ery u se fu l in th e im p le
m e n ta tio n o f d isc re te -tim e system s.
S u p p o se th a t w e h ave a ra tio n a l z-tran sfo rm X ( z ) e x p re ss e d as

X( z) =

(3.4.40)

196

The z-T ransform and Its Application to the Analysis of LTI Systems

Chap. 3

w h ere, fo r sim plicity, w e h av e a ssu m ed th a t ao = 1. If M > N [i.e., X ( z ) is

im p ro p e r], w e c o n v e rt X (z) to a sum o f a p o ly n o m ial a n d a p r o p e r fu n c tio n
M-H
X ( z ) = J 2 c t z ~ k + X pA z )
k=o

(3.4.41)

If th e p o le s o f X pr( z ) are d istin ct, it can b e e x p a n d e d in p a rtia l fra c tio n s as

pT(z) = A \ - ------------- + A 2 ------------- r + --- + ^ jv -------------- r
l-p \z~ l
l ~ P 2Z~l
1 - PnZ~1

(3.4.42)

A s w e h av e a lre a d y o b se rv e d , th e re m ay b e so m e c o m p le x -c o n ju g a te p airs of
p o le s in (3.4.42). S in ce we u su a lly d eal w ith real signals, w e sh o u ld av o id com plex
co efficien ts in o u r d eco m p o sitio n . T h is can b e ach iev ed by g ro u p in g a n d co m b in in g
te rm s co n ta in in g co m p lex -co n ju g ate p o les, in th e follow ing w ay:
A A p * z ~ l + A* - A * p z ~ l
1 - pz~]

1 - p z 1 - p*z ~' + PP*z ~2

1 /5*z-1

(3.4.43)

bo + b -iZ ~ l

1 + a i z ~ l + 02 z ~2
w h ere
>o = 2 R e ( A ) ,

a \ = 2 R e ( p )

b\ = - 2 Re (Ap*),

a 2 = \ p \2

(3.4.44)
a re th e d e sire d co efficients. O b v io u sly , a n y ra tio n a l tra n sfo rm o f th e fo rm (3.4.43)
w ith co efficien ts given by (3.4.44), w hich is th e case w h en a 2 4 a 2 < 0, can be
in v e rte d using (3.4.34). B y c o m b in in g (3.4.41), (3.4.42), a n d (3.4.43) w e o b ta in a
p a rtia l-fra c tio n e x p a n sio n fo r th e z -tra n s fo rm w ith distinct p o les th a t co n ta in s real
coefficients. T h e g e n e ra l re su lt is
MN
Xiz)

K\

= 3 C Z - * + E

1.

1 l.

, /
f'l
-2
t l l + a u z 1+auZ 2

<3 A 4 S >

w h ere K\ + 2 AS = N . O b v io u sly , if Af = TV, th e first te rm is ju s t a c o n stan t,

a n d w h e n M < N , th is te rm v an ish es. W h e n th e r e a re also m u ltip le p oles, som e
a d d itio n a l h ig h e r-o rd e r te rm s sh o u ld b e in clu d ed in (3.4.45).
A n a lte rn a tiv e fo rm is o b ta in e d by ex p ressin g X ( z ) as a p r o d u c t o f sim ple
te rm s as in (3.4.40). H o w e v e r, th e co m p le x -c o n ju g a te p o le s a n d z e ro s sh o u ld be
co m b in e d to av o id co m p lex co efficien ts in th e d e c o m p o sitio n . S u ch c o m b in atio n s
re su lt in se c o n d -o rd e r ra tio n a l te rm s o f th e follow ing form :
(1 - Z*Z-1 )(1 - z*kz ~ ' )

1 + b\kZ~l + b u z ~2

(1 - />*z- 1 ) ( l - p*kz ~ l )

1 -I- a u z -1 + a u z ~2

(3.4.46)

w h ere
b\ k = - 2 R t ( z k ) ,

flu = 2 R e (p * )

b u = jz i l ,

02*

(3.4.47)
= \Pk\

Sec. 3.5

The One-sided 2 -Transform

197

A ssu m in g fo r sim p licity th a t M = N, w e se e th a t X (z) can be d e c o m p o s e d in the

follow ing way:

1 + a kz 1

1 + a u z 1+ axz 2

<3 -4 -48>

w h ere N = K\ + 2 / ^ * W e will r e tu rn to th e s e im p o rta n t fo rm s in C h a p te rs 7 a n d 8.

3.5 THE ONE-SIDED Z-TRANSFORM

T h e tw o -sid ed z -tra n sfo rm re q u ire s th a t th e c o rre sp o n d in g signals be specified
fo r th e e n tire tim e ran g e oo < n < oo. T his re q u ire m e n t p re v e n ts its u se for
a v ery u se fu l fam ily o f p ra c tic a l p ro b lem s, n am ely th e e v a lu a tio n o f th e o u tp u t
o f n o n re la x e d system s. A s we recall, th e s e system s a re d e sc rib e d by d ifferen ce
e q u a tio n s w ith n o n z e ro initial c o n d itio n s. Since th e in p u t is a p p lie d a t a finite
tim e, say n ()l b o th in p u t a n d o u tp u t signals are specified fo r n > no, b u t by no
m e a n s a re z e ro fo r n < no- T h u s th e tw o -sid ed z-tran sfo rm c a n n o t b e used. In this
se c tio n w e d e v e lo p th e o n e -sid e d z-tra n sfo rm w hich can be u se d to solve differen ce
e q u a tio n s w ith in itial c o n d itio n s.

3.5.1 Definition and Properties

T h e one- si ded o r unilateral z -tra n sfo rm o f a signal x ( n ) is d efin ed by
CC

* + (;) n=0

W e also u se th e n o ta tio n s Z +{x(n)} a n d

*{n )

X + (z)

T h e o n e -sid e d z -tra n sfo rm d iffers fro m th e tw o -sid ed tra n sfo rm in th e low er

lim it o f th e su m m a tio n , w hich is aiw ays z e ro , w h e th e r or n o t th e signal x ( n ) is zero
fo r n < 0 (i.e., cau sal). D u e to th is choice o f lo w er lim it, th e o n e -sid e d z-tran sfo rm
h as th e fo llow ing ch aracteristics:
1. It d o e s n o t c o n ta in in fo rm a tio n a b o u t th e signal jc(n) fo r n e g a tiv e v alu es o f
tim e (i.e., fo r n < 0).

2. It is un i q u e o n ly fo r ca u sa l signals, becau se only th ese signals a re z e ro for

n < 0.
3. T h e o n e -sid e d z -tra n s fo rm A,+(z) o f x( n ) is id en tical to th e tw o -sid ed ztra n s fo rm o f th e signal x( n) u( n) . S ince x ( n ) u ( n ) is causal, th e R O C o f its
tra n sfo rm , a n d h en c e th e R O C of X + {z), is alw ays th e e x te rio r o f a circle.
T h u s w h en w e d e a l w ith o n e-sid ed z-tran sfo rm s, it is n o t n ecessa ry to re fe r
to th e ir R O C .

198

The /-T ran sform and Its Application to the Analysis of LTI Systems

Chap. 3

Example 3.5.1
Determ ine the one-sided z-transform of the signals in Exam ple 3.1.1.
Solution

From the definition (3.5.1), we obtain

x,(n) = {1, 2 ,5 ,7 ,0 ,1 }
t

X f ( z ) = 1 + 2 ; ' 1 + 5z~2 + 7z~3 + ;~ 5

x 2 (n) = {1.2, 5, 7, 0,1}

t

Jt:+ (z) = 5 + 7z ~l + z ~3

x 3(n) = (0 ,0 ,1 ,2 , 5,7, 0,1}

t
xA(n) = {2,4, 5, 7 ,0 ,1 )
t

X^(z ) = z ~2 + 2z"3 + 5z~4 + l z ' s + z~7

X4+ (z) = 5 + 7 ; ' 1 + z 3

x 5(n) ~ S(n) -*-1- * X+(z) = 1

x6(n) = 6(n - A),

k > 0 -e*

*700 = <S(k -I- k),

k > 0

(z) = z~k
Xy (z) = 0

Note that for a noncausal signal, the one-sided z-transform is not unique. Indeed,
X 2 (z) = X^(z) but X2 (n) ^ x 4(n). Also for anticausal signals,
(z) is always zero.
A lm ost all properties we have studied for the tw o-sid ed z-transform carry o ver to
the on e-sid ed z-transform with the excep tion of the shi ft i ng property.

Shifting Property
C a se 1: T im e D e la y

If
x(n)

X +(z)

then
k
X (n - k ) X * z~*[Ar+(z) + ] [ ] j c ( - n ) z n]

k> 0

(3.5.2)

n=l

In case jc(n) is causal, then

x{n ~ k )

(3.5.3)

z~k X +(z)

Proof. F rom the definition (3.5.1) w e have

Z +{x(n - k)} = z~

f S ( 0 z -'+

Y t x( l ) z -

-k

By changing the in d ex from / to n = t he result in (3.5.2) is easily obtained.

Sec. 3.5

199

The One-sided z-Transform

Example 3.5.2
Determ ine the one-sided ^-transform of the signals
(a) x( n ) = a nu(n)
(b) Ai(n) = xin 2) where x( n) = a "
Solution
(a) From (3.5.1) we easily obtain

(b) We will apply the shifting property for k = 2. Indeed, we have

Z +{ x ( n ~ 2)1 = ; " 2[X + ( ; ) + j : ( - 1 ) ; + j : ( - 2 ) - 2]
= r 2^ + (:) + J T ( - l) ;- 1 + * ( -2 )
Since x( 1) = a ~ '. x ( 2) = a~2. we obtain
^
1 - az ~l

+ <T2

T h e m e a n in g o f th e shifting p ro p e rty can be in tuitively ex p la in e d if w e w rite (3.5.2)

as follow s:
Z +[x (?i &)} = [jc (k)

x ( + 1 ); ' +

1 ); *+* ]
(3.5.4)

+ :.~kX +(.z)

k> 0

T o o b ta in x { n - k ) ( k > 0) fro m ;t(r?), w e sh o u ld shift x ( n ) by k sa m p le s to th e right.

T h e n k n e w sa m p le s, x ( - k ) , x { k + 1), * ( - 1 ) , e n te r th e p o sitiv e tim e axis
w ith x ( k) lo c a te d at tim e zero . T h e first te rm in (3.5.4) sta n d s fo r th e z-tran sfo rm
o f th e s e sam p les. T h e o ld sa m p les o f x( n k) a re th e sa m e as th o se o f .r(n)
sim ply sh ifted by k sa m p le s to th e right. T h e ir z-tra n sfo rm is o b v io u sly z _iX + (z),
w hich is th e se co n d te rm in (3.5.4).

Case 2: Time advance

If
xin)

X +(z)

th e n
X+( z ) - Y x ( n ) z - n k > 0

x ( n + k) z

(3.5.5)

Proof . F ro m (3.5.1) we h ave

oc

Z +[x(n + *)} =

oc

+ k)z~n = zk Y m z ~ l
n=0

l= k

w h e re w e h a v e c h a n g e d th e in d ex o f su m m atio n fro m n t o 1 = n + k. N o w , from

200

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

(3.5.1) w e o b ta in
* + (z) = j r * ( / ) : ' = Y x { l ) z ~l +
/=o
i=0
i=t
By co m b in in g th e last tw o re la tio n s, w e easily o b ta in (3.5.5).
Exam ple 3.5.3
With x(n), as given in Example 3.5.2, determ ine the one-sided ^-transform of the
signal
xi (n) = x(n + 2 )
Solution
obtain

We will apply the shifting theorem for k = 2. From (3.5.5), with k = 2, we

Z +{x(n + 2)| = -2X+(z) - x (0 );2 - *(1);

But jc(0) = 1, jc(1) = a. and X + (z) = 1/(1 - a ; -1). Thus

,2
Z +{x(n + 2)) = - ----- - - z 2 - az
1 az~
T h e case o f a tim e ad v an ce can be in tu itiv e ly e x p la in e d as follow s. T o o b tain
x( n- i - k) , k > 0, we sh o u ld shift jc (n) by k sa m p le s to th e left. A s a re su lt, th e sam ples
* (0 ). * ( 1 ) , ___x ( k 1) le a v e th e p o sitiv e tim e axis. T h u s w e first re m o v e th e ir
co n trib u tio n to th e X +(z), an d th e n m u ltip ly w h at re m a in s by z k to co m p e n sa te
fo r th e shifting o f th e signal by k sam p les.
T h e im p o rta n c e o f th e shifting p ro p e rty lies in its a p p lic a tio n to th e so lu tio n
of d ifferen ce e q u a tio n s w ith c o n s ta n t co efficien ts a n d n o n z e ro in itial co n d itio n s.
T h is m a k e s th e o n e-sid ed z -tra n sfo rm a v ery useful to o l fo r th e an aly sis o f recu rsiv e
lin e a r tim e -in v a ria n t d isc re te -tim e system s.
A n im p o rta n t th e o re m useful in th e analysis o f signals a n d system s is th e
final v alu e th e o re m .
F in al V a lu e T h e o re m .

If
x{n)

X +{z)

th e n
lim x( n ) = lim (z - l ) X + (z)
n- 00
7-l

(3.5.6)

T h e lim it in (3.5.6) exists if th e R O C o f (z - l)A '+ (z) in clu d es th e u n it circle.

T h e p ro o f o f th is th e o re m is left as an ex ercise fo r th e re a d e r.
T h is th e o re m is u sefu l w h en w e a re in te re s te d in th e a s y m p to tic b e h a v io r of
a signal x( n) a n d w e k n o w its z -tra n sfo rm , b u t n o t th e signal itself. In such cases,
esp ecially if it is co m p licated to in v e rt X + (z), w e can u se th e final v alue th e o re m
to d e te rm in e th e lim it o f x{n) as n goes to infinity.

Sec. 3.5

201

The One-sided z-Transform

Example 3.5.4
The impulse response of a relaxed linear time-invariant system is k(n) = a"u(n),
| | < 1. D eterm ine the value of the step response of the system as n oo.
Solution

The step response of the system is

y{n) = h(n) * x(n)

where
Jt(n) = u(n)
Obviously, if we excite a causal system with a causal input the output will be causal.
Since h(n), x(n), v(n) are causal signals, the one-sided and two-sided z-transforms are
identical. From the convolution property (3.2.17) we know that the z-transforms of
h(n) and *(n) must be multiplied to yield the z-transform of the output. Thus
= .

, .

- , = 7----- tt? ------- r

1 - az 1 1 - z 1

(z - l ) ( z - cr)

ROC: |z| > |ar|

Now
(z - l)y (z) -

Z a

ROC: |z| > |a|

Since |a | < 1 the R O C of (z - l)K(z) includes the unit circle. Consequently, we can
apply (3.5.6) and obtain
lim v(n) = lim - =
noc'
1 a

3.5.2 Solution of Difference Equations

T he on e-sid ed z-transform is a very efficient tool for the solu tion of d ifference
eq u a tio n s w ith n on zero initial conditions. It ach ieves that by reducing the dif
feren ce eq u ation relating the tw o tim e-d om ain signals to an eq u ivalen t algebraic
eq u ation relating their on e-sid ed z-transform s. T h is eq u ation can b e easily solved
to obtain the transform o f the desired signal. T he signal in the tim e dom ain is
o b ta in ed by inverting the resulting z-transform . W e will illustrate this approach
with tw o exam ples.
Exam ple 3-5.5
The well-known Fibonacci sequence of integer num bers is obtained by computing
each term as the sum of the two previous ones. The first few terms of the sequence are
1 ,1 ,2 , 3,5. 8 ,...
D eterm ine a closed-form expression for the n th term of the Fibonacci sequence.
Solution Let y(n) be the nth term of the Fibonacci sequence. Clearly, y(n) satisfies
the difference equation
y (n ) = y(n - 1) + y(n - 2)

(3.5.7)

202

The z -Transform and Its Application to the Analysis of LTI System s

Chap. 3

with initial conditions

v(0) = v(11 + v(2) = 1

(3.5.8a)

y (l) = y(0) + V(-1) = 1

(3.5.8b)

From (3.5.8b) we have y (1) = 0. Then (3.5.8a) gives v(2) = 1. Thus we have to
determ ine y(n), n > 0, which satisfies (3.5.7), with initial conditions y (1) = 0 and
y(2) = 1.
By taking the one-sided ^-transform of (3.5.7) and using the shifting property
(3.5.2). we obtain
y+(z) = [ ; - 'y +(-) + y (1) ] + [z ~2Y ^ ( z ) + y (-2 ) + . v ( - l ) ; - 1]
or
1

K + (c) =

"

(3.5.9)

where we have used the fact that y{ 1) = 0 and v(2) = 1.

We can invert K+(;) by the partial-fraction expansion m ethod. The poles of
y'+(;) are
1 -I- V5

P2 =

Pi -

l-V s

and the corresponding coefficients are A i = p i/V 5 and A 2 = -/> ;/V 5 . Therefore,
v(n) =

'l + V5 / 1 + v'S \ "

1 - V5 / I - n / 5 \ "

2V5

u(rt)

2 ^5

or, equivalently.
u{n)

(3.5.10)

Example 3.5.6
Determ ine the step response of the system
v(n) = ay(n 1) + x(rt)

1<

<

(3.5.11)

when the initial condition is y (1) = 1.

Solution

By taking the one-sided ^-transform of both sides of (3.5.11), we obtain

K+(;) = a [ ;- 'y * (c ) + v (- D ] + X +(z)

Upon substitution for v ( - l ) and X+(;) and solving for y + (;). we obtain the result
Y+(z) =

1
(1 a ; - l )(l - ; _l)

1 a ; -1

(3.5.12)

By performing a partial-fraction expansion and inverse transform ing the result, we

have
y(n) = a"

,u(n)

1 or"+I
+ ;-------- u(n)
1 a

( l - a " +2) u ( n )

1 a

( 3 .5 . 13)

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z-D om ain

203

3.6 ANALYSIS OF LINEAR TIME-INVARIANT SYSTEMS IN THE

Z-DOMAIN
In S e ctio n 3.4.3 w e in tro d u c e d th e sy stem fu n ctio n o f a lin e a r tim e -in v a ria n t sys
tem a n d re la te d it to th e u n it sa m p le re sp o n se a n d to th e d iffe re n c e e q u a tio n
d e sc rip tio n o f sy stem s. In th is sectio n w e d escrib e th e use o f th e sy stem fu n c
tio n in th e d e te rm in a tio n o f th e re sp o n s e of th e system to so m e ex c ita tio n signal.
F u rth e rm o re , w e e x te n d th is m e th o d o f a nalysis to n o n re la x e d sy stem s. O u r a tte n
tio n is fo c u se d o n th e im p o rta n t class o f p o le - z e r o sy stem s r e p re s e n te d by lin e a r
c o n s tan t-co efficien t d iffe re n c e e q u a tio n s w ith a rb itra ry in itial co n d itio n s.
W e also c o n s id e r th e to p ic o f sta b ility o f lin e a r tim e -in v a ria n t sy stem s an d
d esc rib e a test fo r d e te rm in in g th e sta b ility o f a sy stem b a s e d o n th e co efficien ts
o f th e d e n o m in a to r p o ly n o m ial in th e system fu n c tio n . F in ally , w e p ro v id e a
d e ta ile d an aly sis o f se c o n d -o rd e r system s, w hich fo rm th e b asic b u ild in g b lo ck s in
th e re a liz a tio n o f h ig h e r-o rd e r system s.

3.6.1 Response of Systems with Rational System

Functions
L e t us c o n s id e r a p o le - z e r o system d e s c rib e d by th e g e n e ra l lin e a r c o n s ta n tc o efficien t d iffe re n c e e q u a tio n in (3.3.7) a n d th e c o rre sp o n d in g system fu n ctio n
in (3.3.8). W e r e p re s e n t H ( z ) as a ra tio o f tw o p o ly n o m ials B ( z ) / A ( z ) , w h ere
B (z) is th e n u m e r a to r p o ly n o m ia l th a t c o n ta in s th e z e ro s o f H( z ) , a n d A ( z ) is the
d e n o m in a to r p o ly n o m ia l th a t d e te rm in e s th e p o le s o f H ( z ) . F u rth e rm o r e , let us
a ssu m e th a t th e in p u t signal x ( n ) h as a ra tio n a l z -tra n sfo rm X (z) o f th e fo rm
X(z) =
Q( z)

(3.6.1)

T h is a s su m p tio n is n o t o v erly restrictiv e, since, as in d ic a te d p rev io u sly , m o st signals

o f p ra c tic a l in te re s t h av e ra tio n a l z -tran sfo rm s.
If th e sy stem is in itially re lax ed , th a t is, th e in itia l c o n d itio n s fo r th e d iffe re n c e
e q u a tio n are z e ro , y ( 1) = y ( 2) = = y ( N ) = 0, th e z -tra n s fo rm o f th e
o u tp u t o f th e sy stem h a s th e fo rm
Y(z) = H ( z ) X ( z ) =

Mz)Q(z)

(3.6.2)

N o w su p p o s e th a t th e sy stem c o n ta in s sim ple p o le s p \ , p j .........p s a n d th e ztra n sfo rm o f th e in p u t signal co n ta in s p o le s <71, qt, , q u w h e re p t ^ qm fo r all
it = 1, 2
a n d m = 1, 2 , . . . , L. In a d d itio n , w e a ssu m e th a t th e z e ro s o f
th e n u m e r a to r p o ly n o m ia ls B( z ) a n d N ( z ) d o n o t co in cid e w ith th e p o les {p t } an d
{<7i}, so th a t th e re is n o p o le - z e r o c a n c e lla tio n . T h e n a p a rtia l-fra c tio n ex p an sio n
o f K(z) yield s

t o 1 - PkZ 1

*-* 1 - qkZ 1

204

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

T h e in v erse tra n sfo rm o f ^ (z ) y ield s th e o u tp u t signal fro m th e sy stem in th e form

N

y (n ) = Y A t ( P k ) n u ( n ) + 5 2 2*0?*)'1n (n)
*=i
*-i

(3.6.4)

W e o b se rv e th a t th e o u tp u t se q u e n c e y( n) can b e su b d iv id e d in to tw o p a rts. T he
first p a r t is a fu n ctio n o f th e p o les {pt} o f th e system an d is called th e natural
response o f th e sy stem . T h e in flu e n ce o f th e in p u t signal o n th is p a r t o f th e
re sp o n s e is th ro u g h th e scale facto rs {A*}. T h e se c o n d p a r t o f th e re sp o n se is a
fu n ctio n o f th e p o le s {qk} o f th e in p u t signal an d is called th e f o r c e d response of
th e sy stem . T h e in flu e n ce o f th e sy stem o n th is re sp o n s e is e x e rte d th ro u g h th e
scale facto rs { Qk } W e sh o u ld e m p h asize th a t th e scale fa c to rs {A*} an d { Q k } a re fu n ctio n s o f
b o th se ts o f p o le s {pk } a n d { ^ ) . F o r e x am p le, if AXz) = 0 so th a t th e in p u t is
zero , th e n K(z) = 0, a n d c o n s e q u e n tly , th e o u tp u t is z e ro . C lea rly , th e n , th e
n a tu r a l re sp o n se o f th e sy stem is zero . T h is im p lies th a t th e n a tu ra l re sp o n se of
th e sy stem is d iffe re n t fro m th e z e ro -in p u t re sp o n se .
W h en X (z) a n d H ( z ) h av e o n e o r m o re p o les in c o m m o n o r w h e n X (z)
a n d /o r H{ z ) c o n ta in m u ltip le -o rd e r p o les, th e n K(z) will h av e m u ltip le -o rd e r poles.
C o n se q u e n tly , th e p a rtia l-fra c tio n ex p a n sio n o f Y( z ) will c o n ta in fa c to rs o f th e form
1/(1 - p / z ~ l )t , k = 1, 2 , . . . , m, w h ere m is th e p o le o rd e r. T h e in v ersio n o f th ese
facto rs will p ro d u c e te rm s o f th e fo rm n k~ xp* in th e o u tp u t y( n ) o f th e system , as
in d ic a te d in S ectio n 3.4.2.

3.6.2 Response of Pole-Zero Systems with Nonzero

Initial Conditions
S u p p o se th a t th e sig n al x ( n ) is a p p lie d to th e p o le - z e r o sy stem at n = 0. T hus
th e sign al x ( n ) is a ssu m ed to be cau sal. T h e effe cts o f all p re v io u s in p u t signals to
th e system a re reflec te d in th e in itial c o n d itio n s y ( 1), y ( ~ 2 ) .........y ( N ) . Since
th e in p u t x ( n ) is ca u sa l a n d since w e a re in te re s te d in d e te rm in in g th e o u tp u t y ( n )
fo r n > 0, w e can u se th e o n e -sid e d z-tra n s fo rm , w hich allow s us to d e a l w ith th e
in itial co n d itio n s. T h u s th e o n e -sid e d z -tra n s fo rm o f (3.4.7) b e c o m e s

f t)

akz

Y +{z) + Y j y (' - n ) z ''

+ f c * z - * X + (z)

(3.6.5)

S ince x ( n) is cau sal, we can se t X + (z) = X (z). In an y case (3.6.5) m ay b e e x p ressed

as

52 ^
Y +(z) =

*=0
s

-X(z)-

1+52<:
= H(z)X(z) +

akz
No(z)
A (z)

(3.6.6)

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z-D om ain

205

w h e re
N

No(z) = - 5 Z a *z_,t
k= 1
n=l

(3.6.7)

F ro m (3.6.6) it is a p p a r e n t th a t th e o u tp u t o f th e sy stem w ith n o n z e ro initial

c o n d itio n s can b e su b d iv id e d in to tw o p a rts. T h e first is th e z e ro -sta te re sp o n se of
th e sy stem , d e fin ed in th e z -d o m ain as
Ya (z) = H ( z ) X ( z )

(3.6.8)

T h e se c o n d c o m p o n e n t c o rre sp o n d s to th e o u tp u t re su ltin g fro m th e n o n z e ro initial

co n d itio n s. T h is o u tp u t is th e z e ro -in p u t re sp o n se o f th e system , w hich is defined
in th e z -d o m ain as

(z) = TTT
A(z)

a 6 9)

H e n c e th e to ta l re sp o n se is th e su m o f th ese tw o o u tp u t c o m p o n e n ts , w hich can

b e e x p ressed in th e tim e d o m a in by d e te rm in in g th e in v erse z -tra n s fo rm s o f Kzs(;)
a n d Y A z ) s e p a ra te ly , a n d th e n ad d in g th e resu lts. T h u s
y ( n) = >zs() + >'zi(n)

(3.6.10)

S ince th e d e n o m in a to r of Fz|( z ) , is A(z), its p o le s are p i , P 2 ........ Ps- C o n se

q u e n tly , th e z e ro -in p u t re sp o n se h a s th e form
N

>'zi(n) =

52

(3.6.11)

T h is can b e a d d e d to (3.6.4) a n d th e te rm s involving th e p o les {/?*} can be co m b in ed

to yield th e to ta l re sp o n s e in th e form
y( n) =

k= 1

k=\

52 A k' ( Pk) nu( n) + 51 Q k i q k T u ( n )

(3.6.12)

w h e re , by d efin itio n ,
A'k =

+ Dk

(3.6.13)

T h is d e v e lo p m e n t in d icates clearly th a t th e effect o f th e initial co n d itio n s

is to a lte r th e n a tu ra l re sp o n s e o f th e system th ro u g h m o d ifica tio n o f th e scale
f a c to rs {Ak}. T h e re a re n o n ew p o le s in tro d u c e d by th e n o n z e ro in itial co n d itio n s.
F u rth e rm o r e , th e r e is no effect on th e fo rce d re sp o n s e o f th e system . T h ese
im p o rta n t p o in ts a re re in fo rc e d in th e follow ing e x am p le.
Exam ple 3.6.1
D eterm ine the unit step response of the system described by the difference equation
y{n) = 0.9y(n 1) - 0.81y(n - 2) + x( n)
under the following initial conditions:
(a) y ( - l ) = > (-2 ) = 0

(b) y ( - 1) = y ( - 2 ) = 1

206

The z-Transform and Its Application to the Analysis of LTI Systems

Solution

Chap. 3

The system function is

H {z) =

1
1 - 0 . 9 : - 1 + 0 .8 1 ;- 2

This system has two complex-conjugate poles at

P 2 = G. 9e- jn/i

p l = Q.9e^

The z-transform of the unit step sequence is

1
X(z) =

1-z -1

Therefore,
1
(1 - 0 . 9 e j ^ z - l K l - 0 . 9 e - j '1z -'l ) ( l - z ~ ' )

0.542 - /0 .0 4 9
1 - 0 .9 e ^ f h ~ l

0.542 + y0.049
+ :---- r +

1 - 0 .9e->nPz - 1

1.099
1 - z~l

and hence the zero-state response is

y^(n) =

l.099

1.088(0.9)" cos

(~ n

- 5.2C) J

u(n)

(a) Since the initial conditions are zero in this case, we conclude that v(n) = >'is(fl).
(b) For the initial conditions v ( - l ) = v (-2 ) = 1, the additional com ponent in the
z-transform is
Mi(z)
A(z)

I'ziW

0.09 - 0.S ir " 1

1 - 0 . 9 ; - ' + 0.81r2

0.026 + j0.4936 i 0.026 - jO.4936

1-

0 H e i ' f i z - 1 + 1 - 0 . 9 e - '* P z ~ '

Consequently, the zero-input response is

yzi(n) = 0.988(0.9)'' cos ^ ~ n + 87^J u(n)
In this case the total response has the z-transform
y (z )

= J ^ w + y^ u )
1.099
0.568 + y'0.445
0.568 - /0.445
+ :---- r +
1 - z "1
1 - Q.9eJ*Vz- 1
1 - 0.9e' ff/3; -1

The inverse transform yields the total response in the form

y(n) = 1.099u(n) + 1.44(0.9)" cos ( ^ n + 3 8 ^ u (n)

3.6.3 Transient and Steady-State Responses

A s w e h av e seen fro m o u r p re v io u s d iscussion, th e re sp o n se o f a sy ste m to a given
in p u t can b e se p a ra te d in to tw o c o m p o n e n ts , th e n a tu r a l re sp o n s e a n d th e fo rce d

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z-Dom ain

207

re sp o n se . T h e n a tu ra l re sp o n s e o f a cau sal system h a s th e fo rm

N
>nr(n) =

52 A k i p k ) nu{ n)

(3.6.14)

*=1
w h e re {pk), k = 1, 2,
N a re th e p o le s o f th e sy stem a n d {A*} a re sc ale fac
to rs th a t d e p e n d o n th e in itial c o n d itio n s an d on th e c h a ra c te ristic s o f th e in p u t
se q u en ce.
If |/>*| < 1 fo r all k, th e n , y nT(n) d ecay s to z e ro as n a p p ro a c h e s infinity. In
such a case w e re fe r to th e n a tu ra l re sp o n se of th e system as th e t ransient response.
T h e ra te a t w hich >'nr(n) d ecay s to w a rd z e ro d e p e n d s on th e m a g n itu d e o f th e p o le
p o sitio n s. I f all th e p o le s h av e sm all m a g n itu d e s, th e d ec a y is v ery ra p id . O n the
o th e r h a n d , if o n e o r m o re p o le s a re lo c a te d n e a r th e u n it circle, th e c o rre sp o n d in g
te rm s in >nr(n) w ill d e c a y slow ly to w a rd z e ro a n d th e tra n s ie n t will p ersist fo r a
relativ ely lo n g tim e.
T h e fo rc e d re sp o n s e o f th e system h as th e fo rm
i
Vfr() = 5 2 <2*(<?*)"()
k=\

(3.6.15)

w h e re {^t), k = 1, 2 , . . . , L a re th e p o le s in th e forcing fu n c tio n a n d { Qk } a re

scale fa c to rs th a t d e p e n d o n th e in p u t se q u e n c e an d o n th e c h a ra c te ristic s o f th e
system . If all th e p o le s o f th e in p u t signal fall in sid e th e u n it circle, ^ ( n ) will d ecay
to w a rd z e ro as n a p p ro a c h e s infinity, ju st as in th e case o f th e n a tu ra l resp o n se.
T h is sh o u ld n o t b e su rp risin g since th e in p u t signal is also a tra n s ie n t signal. O n
th e o th e r h a n d , w h e n th e cau sal in p u t signal is a sin u so id , th e p o le s fall o n th e unit
circle a n d c o n s e q u e n tly , th e fo rce d re sp o n se is also a sin u so id th a t p ersists fo r all
n > 0. In th is case, th e fo rce d re sp o n se is called th e steady-state respons e o f th e
system . T h u s, fo r th e sy stem to sustain a ste a d y -s ta te o u tp u t fo r n > 0, th e in p u t
sig n al m u st p e rsist fo r all n > 0.
T h e fo llo w in g ex a m p le illu stra te s th e p re se n c e o f th e s te a d y -s ta te resp o n se.
Exam ple 3*6.2
D eterm ine the transient and steady-state responses of the system characterized by
the difference equation
>{n) = 0.5;y(n - 1) + jc(n)
when the input signal is x ( n ) = 10cos(jrn/4)u(n). The system is initially at rest (i.e.,
it is relaxed).
Solution

The system function for this system is

and therefore the system has a pole at z = 0.5. The z-transform of the input signal is
(from Table 3.3)
10(1 ( l/y / 2 ) z ~ 1)
X (z ) ---------------f ---------------1 - y / 2 _1 + Z~1

208

The z-Transform and Its Application to the Analysis of LTI S ystem s

Chap. 3

Consequently.
K (o = H (:)X (z )

10(1 - ( l / v '2 ) ; - 1)
(1 0.5~- )(1 - e - w 4; - 1)!!
6.78e~J2&1

6.3
1 - 0.5;- 1

,"T

1-

i)
6.7&ej2K1

1 - e-J*iAz~x

The natural or transient response is

ynr(n) = 6.3(0.5)"u()
and the forced or steady-state response is
Vfr (n) =

+6. 1&eJ2*J e - inn!A]u(},)

= 13.56cos (^~^n 2 8 . 7 w()

Thus we see that the steady-state response persists for all n > 0. just as the input
signal persists for all n > 0.

3.6.4 Causality and Stability

A s d efin ed p rev io u sly , a causal lin e a r tim e -in v a ria n t system is o n e w hose unit
sa m p le resp o n se h (n ) satisfies the co n d itio n
h (n ) = 0

n < 0

W e h av e also show n th at the R O C o f th e ;;-tra n sfo rm o f a cau sal se q u e n c e is the

e x te rio r o f a circle. C o n se q u e n tly , a lin e a r t im e -in v a r ia n t sy ste m is c a u s a l i f an d
o n ly i f the R O C o f the system f u n c t io n is the e x te rio r o f a c ir c le o f ra d iu s r < 00,
in c lu d in g the p o in t z

do.

T h e stab ility o f a lin ear tim e -in v a ria n t system can also be e x p re ss e d in term s
o f th e ch a ra c te ristic s o f th e system fu n ctio n . A s w e recall fro m o u r prev io u s
d iscu ssio n , a n ecessa ry an d sufficient co n d itio n fo r a lin e a r tim e -in v a ria n t system
to b e B IB O sta b le is

52

n=- x
In tu rn , this c o n d itio n im plies th a tt H ( z ) m
u st c o n ta in th e u n it circle w ith in its R O C .
musi
In d e e d , since
OC

H (z) =

h (n)z

it follow s th a t
OC

OC

n oc

n= oc

W h en e v a lu a te d o n th e u n it circle (i.e., |z| = 1),

OC

\ h (z )\ <

52

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the 7 -Domain

209

H en ce, if the system is B IB O stable, the unit circle is con tained in the R O C o f
H(z)- T h e con verse is also true. T h erefore, a linear tim e-in va ria n t sy stem is B IB O
stable i f a n d o n ly i f th e R O C o f the sy stem fu n c tio n includes the u n it circle.
W e should stress, how ever, that the con d ition s for causality and stability are
d ifferent and that o n e d oes not im ply the other. F or exam p le, a causal system
m ay b e stable or unstable, just as a noncausal system m ay b e stable or unstable.
Sim ilarly, an unstable system m ay be eith er causal or n oncausal, just as a stable
system m ay be causal or noncausal.
For a causal system , h ow ever, the con d ition on stability can be narrowed
to so m e exten t. In d eed , a causal system is characterized by a system function
H ( z ) having as a R O C the exterior o f som e circle o f radius r. For a stable
system , the R O C m ust include the unit circle. C on sequ en tly, a causal and sta
ble system m ust have a system function that con verges for |z| > r < 1. Since
the R O C cannot contain any p oles o f H ( z ) , it follow s that a causal linear tim ein va ria n t sy stem is B I B O stable i f a n d o n ly i f all the p o le s o f H ( z ) are inside the
u n it circle.
Example 3.63
A linear tim e-invariant system is characterized by the system function
3 - 4 z-'
H(z) ~ 1 - 3.5z-> + 1.5: ' 2

1
"

l - ' i z - 1 + 1 - 3 z- 1

Specify the R O C of H(z) and determ ine h(n) for the following conditions:
(a) The system is stable.
( b ) The system is causal.

(c) The system is anticausal.

Solution

T he system has poles at z = 5 and z = 3.

(a) Since the system is stable, its R O C must include the unit circle and hence it is
\ < \z\ < 3. Consequently, h(n) is noncausal and is given as
h(n) = (i)"n() - 2(3)-it ( - n - 1)
(b) Since the system is causal, its R O C is jz| > 3. In this case
/i(n) = ( i r ( n)+ 2 (3 )"u (n )
This system is unstable.
(c) If the system is anticausal, its R O C is |z| < 0.5. Hence
h{n) = - { ( \ y + 2 ( 3 T ) u ( - n - l )

In this case the system is unstable.

210

The z - T ransform and Its Application to the Analysis of LTI Systems

Chap. 3

3.6.5 Pole-Zero Cancellations

W h en a z -tra n sfo rm h as a p o le th a t is a t th e sam e lo catio n as a z e ro , th e pole
is ca n c e le d by th e z e ro an d , c o n s e q u e n tly , th e te rm c o n ta in in g th a t p o le in the
in v erse z-tra n sfo rm v an ish es. S uch p o le - z e r o c a n ce llatio n s a re v ery im p o rta n t in
th e an alysis o f p o le - z e r o system s.
P o le -z e ro c a n ce llatio n s can o c c u r e ith e r in th e sy stem fu n c tio n itself o r in
th e p ro d u c t o f th e sy stem fu n ctio n w ith th e z -tra n sfo rm o f th e in p u t signal. In the
first case w e say th a t th e o r d e r o f th e sy stem is re d u c e d by o n e . In th e la tte r case
w e say th a t th e p o le o f th e system is su p p re s se d by th e z e ro in th e in p u t signal,
o r vice v ersa. T h u s, by p ro p e rly se lectin g th e p o sitio n o f th e ze ro s o f th e in p u t
signal, it is p o ssib le to su p p re ss o n e o r m o re sy stem m o d es (p o le fa c to rs) in th e
re sp o n se o f th e sy stem . S im ilarly, by p r o p e r se le c tio n o f th e z e ro s o f th e system
fu n ctio n , it is p o ssib le to su p p re s s o n e o r m o re m o d e s o f th e in p u t signal fro m the
re sp o n se o f th e system .
W h e n th e z e ro is lo c a te d v ery n e a r th e p o le b u t n o t ex actly a t th e sa m e loca
tio n , th e te rm in th e re sp o n se h as a v ery sm all a m p litu d e . F o r e x a m p le , n o n ex act
p o ie - z e r o c an ce llatio n s can o ccu r in p ra c tic e as a re su lt o f in su ffician t n u m erical
p recisio n u sed in re p re se n tin g th e co efficien ts o f th e system . C o n s e q u e n tly , one
sh o u ld n o t a tte m p t to stab ilize an in h e re n tly u n sta b le system by p lacing a z e ro in
th e in p u t signal at th e lo catio n o f th e pole.
Example 3.6.4
Determ ine the unit sample response of the system characterized by the difference
equation
v(n) = 2.5 v(n 1) >(n 2) + jr(n) 5jr(n 1) + 6x(n 2)
Solution

The system function is

1 - 5*-1 + 6z ' 2
(1 - i z - ') ( l - 2 z -
This system has poles at p\ = 2 and p x = ~, Consequently, at first glance it appears
that the unit sample response is
Y(z) = H( z) X( z) =

1 - 5 z ~ ' + 6z2
(1 - j z _ ,)(l - 2z_l

By evaluating the constants at z = j and z = 2, we find that

A = *

B =0

The fact that 5 = 0 indicates that there exists a zero at z = 2 which cancels
the pole at z = 2. In fact, the zeros occur at z = 2 and z = 3. Consequently, H{z)

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z-Dom ain

211

reduces to
H(z) =

1 - 3 : -1
1- k -1
2 .5 ;-1

= 1and therefore
h(rt) = S{n) -

2 .5 ( 5)"

~ ^

The reduced*order system obtained by canceling the common pole and zero is char
acterized by the difference equation
y(n) = iy(n - 1) + x(n) - 3x{n - 1)
Although the original system is also BIBO stable due to the pole-zero cancellation,
in a practical im plementation of this second-order system, we may encounter an
instability due to imperfect cancellation of the pole and the zero.
Example 3.6.5
D eterm ine the response of the system
v(n) = jjy(>i - 1) - y(n - 2) + x(n)
to the input signal x{n) = S(n) ^&(n 1).
Solution

The system function is

1
WU) =

(1 -

(1 - 1 c-1)

This system has two poles, one at z = ^ and the other at : =

the input signal is

The ^transform of

X {z ) = 1 - j i ' 1

In this case the input signal contains a zero at ; = i which cancels the pole at ; =
Consequently,
Y(z) = HU) X( z )
m

i r p

and hence the response of the system is

y(n) = ( 5)n(n)
Clearly, the m ode ( |)" is suppressed from the output as a result of the pole-zero
cancellation.

3.6.6 Multiple-Order Poles and Stability

A s w e h a v e o b se rv e d , a n ecessa ry an d sufficient c o n d itio n fo r a causal lin e a r tim ein v a ria n t sy stem to b e B IB O sta b le is th a t all its p o le s lie in sid e th e u n it circle.
T h e in p u t sig n al is b o u n d e d if its z -tra n s fo rm c o n ta in s p o le s {qk}, k = 1 , 2 , L,

212

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

w hich satisfy th e c o n d itio n \qk \ < 1 fo r all k. W e n o te th a t th e fo rc e d re sp o n se of

th e system , given in (3.6.15). is also b o u n d e d , ev en w h e n th e in p u t signal contains
o n e o r m o re d istin ct p o les on th e u n it circle.
In view o f th e fact th a t a b o u n d e d in p u t signal m ay h av e p o le s on th e unit
circle, it m ight a p p e a r th a t a sta b le system m ay also h av e p o le s on th e u n it circle.
T his is n o t th e case, h o w ev er, since such a sy stem p ro d u c e s an u n b o u n d e d resp o n se
w h en ex cited by an in p u t signal th a t also has a p o le a t th e sa m e p o sitio n o n the
u n it circle. T h e follow ing ex a m p le illu stra te s th is p o in t.
Example 3.6.6
D eterm ine the step response of the causal system described by the difference equation
v(n) = y{n - 1) + x{n)
Solution

The system function for the system is

We note that the system contains a pole on the unit circle at c = 1. The ^-transform
of the input signal x(n) = u{n) is

which also contains a pole at ; = 1. Hence the output signal has the transform
K(=) = H {z)X {z)

(1 - ; - 1)2

which contains a double pole at z = 1.

The inverse z-transform of Y(z) is
>-()= (n + l)n(B)
which is a ramp sequence. Thus y(n) is unbounded, even when the input is bounded.
Consequently, the system is unstable.
E x am p le 3.6.6 d e m o n s tra te s clearly th a t B IB O stab ility r e q u ire s th a t th e sys
tem p o le s b e strictly in side th e u n it circle. If th e sy stem p o les a re all inside th e unit
circle an d th e e x c ita tio n se q u e n c e x ( n ) c o n ta in s o n e o r m o re p o le s th a t coincide
w ith th e p o les o f th e sy stem , th e o u tp u t Y(z ) w ill c o n ta in m u ltip le -o rd e r poles. As
in d ic a te d p rev io u sly , such m u ltip le -o rd e r p o les re su lt in an o u tp u t se q u e n c e th at
co n tain s term s o f th e form
A kn b( p k)nu(n)
w h ere 0 < b < m 1 an d m is th e o r d e r o f th e p o le. If |p*| < 1, th e s e te rm s decay
to z e ro as n a p p ro a c h e s infinity b e c a u s e th e e x p o n e n tia l f a c to r (pk) n d o m in ates
th e te rm n b. C o n se q u e n tly , n o b o u n d e d in p u t signal can p r o d u c e an u n b o u n d e d
o u tp u t signal if th e sy stem p o les a re all in sid e th e u n it circle.

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z - D o m a in

213

F in ally , w e sh o u ld s ta te th a t th e o n ly u sefu l system s w h ich c o n ta in p o les

on th e u n it circle a re th e d ig ital o sc illato rs discussed in C h a p te r 4. W e call such
sy stem s mar gi nal l y stable.

3.6.7 The Schur-Cohn Stability Test

W e h av e sta te d p rev io u sly th a t th e sta b ility o f a sy stem is d e te rm in e d by th e
p o sitio n o f th e p o les. T h e p o les o f th e system are th e ro o ts o f th e d e n o m in a to r
p o ly n o m ia l o f H ( z ), n am ely ,
A(z) = 1 + a \ z 1 + Q-2Z 2 + + QpfZ ^

(3.6.16)

W h en th e sy stem is cau sal all th e ro o ts o f A (z) m u st lie inside th e u n it circle fo r

th e sy stem to b e sta b le .
T h e re a re se v e ra l c o m p u ta tio n a l p ro c e d u re s th a t aid u s in d e te rm in in g if any
o f th e ro o ts o f A (z) lie o u tsid e th e u n it circle. T h e se p ro c e d u re s a re called stability
criteria. B e io w w e d esc rib e th e S c h u r-C o h n test p ro c e d u re fo r th e sta b ility o f a
sy stem c h a ra c te riz e d by th e system fu n c tio n H( z ) = B ( z ) / A { z).
B e fo re w e d e sc rib e th e S c h u r-C o h n te st w e n e e d to e sta b lish so m e useful
n o ta tio n . W e d e n o te a p o ly n o m ia l o f d e g re e m by
Am(z) = a m(* )z - A
i=0

a m(0) = l

(3.6.17)

T h e reci procal o r reverse p o l y n o m i a l Bm(z) o f d e g ree m is d e fin e d as

Bm(z) =

z-')
(3.6.18)
UmV"

k*=0

W e o b se rv e th a t th e coefficients o f B m( z) are th e sa m e a s th o se o f Am(z), b u t

in re v e rse o rd e r.
In th e S c h u r-C o h n sta b ility te s t, to d e te rm in e if th e p o ly n o m ia l A (z) has all
its r o o ts in sid e th e u n it circle, w e c o m p u te a set o f coefficients, called reflection
coefficients, K \ , K i .........K n fro m th e p o ly n o m ials A m(z). F irst, w e set
A N (z) = A( z )
and

(3.6.19)
ATjV = a # ( N )

T h e n w e c o m p u te th e lo w e r-d e g re e p o ly n o m ials Am(z), m = N , N 1, N 2 , . . . , 1,

a c c o rd in g to th e rec u rsiv e e q u a tio n
A
^
^m (z) K mB m(z)
A m- i ( z ) = --------1
--------

^ c
(3.6.20)

w h e re th e c o effic ie n ts K m a re d e fin ed as

Km = am{m)

(3.6.21)

214

The z-Transform and Its Application to the Analysis of LTI Systems

Chap, 3

T h e S c h u r-C o h n sta b ility test sta te s th a t the p o l y n o m i a l A (;) gi en by (3.6.16)

has all its roots inside the unii circle i f a n d onl y i f the coefficients K m satisfy the
condit i on \Km \ < 1 f o r all m 1, 2 ........ N.
W e shall n o t p ro v id e a p r o o f o f th e S c h u r-C o h n te s t at th is p o in t. The
th e o re tic a l ju stificatio n fo r th is te s t is given in C h a p te r 11. W e illu stra te th e com
p u ta tio n a l p ro c e d u re w ith th e follow ing exam ple.
Example 3.6.7
Determ ine if the system having the system function

is stable.
Solution

We begin with A ;(;), which is defined as

A 2 (z ) =

1 -

lz~] -

k " 2

Hence

Now
#>(:) =
and
/\2(r) 1 - K;

Therefore.
Ki = - I
Since |ATi | > 1 il follows that the system is unstable. This fact is easily estab
lished in this example, since the denom inator is easily factored to yield the two poles
at pi = 2 and p 2
However, for higher-degree polynomials, the Schur-Cohn
test provides a simpler test for stability than direct factoring of //(- ) .
T h e S c h u r-C o h n sta b ility te s t can b e easily p ro g ra m m e d in a d igital c o m p u ter
an d it is very efficien t in te rm s o f a rith m e tic o p e ra tio n s. S pecifically, it req u ires
o n ly N 2 m u ltip lic a tio n s to d e te rm in e th e co efficien ts {Km}, m = 1 , 2 .........N. The
recu rsiv e e q u a tio n in (3.6.20) can b e e x p ressed in te rm s o f th e p o ly n o m ial coef
ficients by e x p a n d in g th e p o ly n o m ia ls in b o th sides o f (3.6.20) a n d e q u a tin g the
co efficien ts c o rre sp o n d in g to e q u a l p o w ers. In d e e d , it is easily e s ta b lis h e d that
(3.6.20) is e q u iv a le n t to th e follow ing alg o rith m : Set
a N (k) = ak

it

= 1 ,2 .........N

(3.6.23)

Kfj = a f j ( N)
T h e n , fo r m = N , N 1 , . . . , 1, c o m p u te
K m = a m(rn)

(3.6.22)

<jm_ i(0 ) = l

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the z-D om ain

215

it = 1, 2 , . . . , m 1

(3.6.24)

an d

w h ere
bm(k) = am(m k)

k = 0,1,...,m

(3.6.25)

T h is recu rsiv e a lg o rith m fo r th e c o m p u ta tio n o f th e c o effic ie n ts {ATm} finds

a p p lic a tio n in v a rio u s signal p ro cessin g p ro b le m s, especially in sp e e c h signal p r o
cessing.

3.6.8 Stability of Second-Order Systems

In th is se c tio n w e p ro v id e a d e ta ile d an aly sis o f a sy stem h av in g tw o p o les. A s
w e sh all se e in C h a p te r 7, tw o -p o le system s fo rm th e b asic b u ild in g b lo ck s fo r th e
r e a liz a tio n o f h ig h e r-o rd e r system s.
L e t us c o n s id e r a cau sal tw o -p o le sy stem d e s c rib e d by th e s e c o n d -o rd e r dif
fe re n c e e q u a tio n
y (n ) = - a ^ y ( n - I) - a 2y ( n - 2) + b 0x ( n )

(3.6.26)

T h e sy stem fu n ctio n is

X (z)

1 4- a i r 1 + a 2z ~ :

(3.6.27)

boz2
z 2 + a \z + 2
T h is sy stem h a s tw o z ero s at th e origin a n d p o les a t

(3.6.28)
T h e sy stem is B IB O sta b le if th e p o le s lie in sid e th e u n it circle, th a t is, if
|P i| < 1 a n d \ p2\ < 1. T h e se c o n d itio n s can b e re la te d to th e v alu es o f th e
co effic ie n ts a\ a n d a 2. In p a rtic u la r, th e ro o ts o f a q u a d ra tic e q u a tio n satisfy th e
re la tio n s
fil = (P5 + pi )

(3.6.29)
(3.6.30)

F ro m (3.6.29) a n d (3.6.30) w e easily o b ta in th e c o n d itio n s th a t a\ a n d a 2 m u st

satisfy fo r sta b ility . F irst, aj m u st satisfy th e c o n d itio n
\ai\ = \ p\ pi \ - \p\Wpi\ < 1

(3.6.31)

T h e c o n d itio n fo r a\ can b e e x p re sse d as

1 1 < 1 + 2

(3.6.32)

216

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

T h e c o n d itio n s in (3.6.31) a n d (3.6.32) can also be d e riv e d fro m th e S c h u rC o h n stab ility te st. F ro m th e recu rsiv e e q u a tio n s in (3.6.22) th r o u g h (3.6.25), we
find th a t
(3.6.33)
an d
K 2 = 02

(3.6.34)

T h e sy stem is sta b le if a n d o n iy if [AT]| < 1 an d lA^I < 1. C o n s e q u e n tly ,

1 < 2 < 1
o r eq u iv a le n tly \a%\ < 1, w hich a g rees w ith (3.6.31). A lso,

or, eq u iv alen tly ,

ai < 1 ~b 02
d\ > 1 a 2
w hich a re in a g re e m e n t w ith (3.6.32). T h e re fo re , a tw o -p o le sy stem is sta b le if and
only if th e co efficien ts a\ a n d a ; satisfy th e c o n d itio n s in (3.6.31) a n d (3.6.32).
T h e stab ility c o n d itio n s given in (3.6.31) an d (3.6.32), d efin e a reg io n in the
co efficien t p lan e (o i. ai), w hich is in th e fo rm o f a tria n g le , as sh o w n in Fig. 3.15.
T h e sy stem is sta b le if an d only if th e p o in t (o j, q t) lies inside th e tria n g le , w hich
w e call th e stability triangle.
T h e ch a ra c te ristic s o f th e tw o -p o le system d e p e n d o n th e lo catio n o f the
p o les o r. eq u iv alen tly , on th e lo catio n o f th e p o in t (e i, 02) in th e sta b ility triangle.
T h e p o le s o f th e sy stem m ay be re a l o r co m p lex c o n ju g a te , d e p e n d in g on the
v alu e o f th e d isc rim in a n t A = a* Aa2- T h e p a ra b o la a 2 = a \ f 4 splits th e stability

Figmre 3 .15 Region of stability

(stability triangle) in the ( a i, a?)
coefficient plane for a second-order
system.

Sec. 3.6

Analysis of Linear Tim e-Invariant Systems in the ^-D om ain

217

tria n g le in to tw o reg io n s, as illu stra te d in Fig. 3.15. T h e re g io n b elo w th e p a ra b o la

(,a\ > 4 d 2 ) c o rre sp o n d s to re a l an d d istin ct poles. T h e p o in ts o n th e p a ra b o la
( af = 4*22) re su lt in re a l a n d e q u a l (d o u b le ) poles. F inally, th e p o in ts ab o v e th e
p a ra b o la c o rre sp o n d to co m p lex -co n ju g ate poles.
A d d itio n a l in sig h t in to th e b e h a v io r o f th e system can be o b ta in e d fro m th e
u n it sa m p le re sp o n s e s fo r th e s e th re e cases.

Real and distinct poles (af = 4a2)- Since p 1 ,

system fu n c tio n can b e ex p re sse d in th e fo rm
A\

1 - Piz 1

pi are re a l a n d p\ ^ p2. th e

A2

(3.6.35)

1 - P 2Z' 1

w h ere
koPi

Ai ~

-boP2

(3.6.36)

P 1 - P2

Pi - P2

C o n se q u e n tly , th e u n it sa m p le re sp o n se is
b

(3.6.37)

' ( P T ' - P ? l )u(n)

Pi P 2

T h e re fo re , th e u n it sa m p le re sp o n se is th e d ifferen ce of tw o d e c ay in g e x p o n e n tia l
se q u en ces. F ig u re 3.16 illu stra te s a ty p ical g rap h for h( n) w h en th e p o les are
distinct.

Real and equal poles (af = 4a2).

In th is case p\

P 2 p = a \ / 2. T he

system fu n ctio n is
( 1 - p z - 1)2

(3.6.38)

an d h e n c e th e u n it sa m p le re sp o n se o f th e system is
h{n) = b0(n + 1 ) p nu(n)
h(n)

Figure 3.16 Plot of h(n) given by (3.6.37) with p\ = 0.5, pz = 0.75; h(n) =
~ P2)](P+1 - P 2 +I)u(n).

(3.6.39)

218

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

h(n)

Figure 3.17

Plot of h(rr) given by (3.6.39) with p =

ti(n) = (n + 1

W e o b serv e th a t h( n) is th e p ro d u c t o f a ra m p se q u e n c e a n d a real decaying

e x p o n e n tia l se q u en ce. T h e g ra p h o f h( n) is show n in Fig. 3.17.

Complex-conjugate poles (af < 4a2). Since the p o le s a re co m p lex c o n

ju g a te , th e system fu n ctio n can be fa c to re d and ex p re sse d as
A

1 p : -1

A
1 - re^'z' 1

1 p*z~l

(3.6.40)

A'

1 - r e ~ )tUi' z ~x

w h ere p = r e Jai an d 0 < coq < tt. N o te th a t w hen th e p o le s are co m p lex co n ju g ates,
th e p a ra m e te rs a\ a n d 02 a re re la te d to r an d a>o acco rd in g to
a\ = 2 r cos coo
(3.6.41)
az = r
T h e c o n stan t A in th e p a rtia l-fra c tio n ex p an sio n o f H( z ) is easily sh o w n to be
A =

bo p

boreJW"

p p*

r ( e i 0 e~JW0)

b 0eJ<I*

(3.6.42)

j 2 sin ti>o
C o n seq u en tly , th e u n it sa m p le re sp o n se o f a system w ith c o m p ie x -c o n ju g a te poles
is
h{n) =

sin coo
born
sin coo

2j

-(n)
(3.6.43)

sin(rt -t- l)cuou(n)

In this case h ( n ) h a s an o sc illato ry b e h a v io r w ith an e x p o n e n tia lly decaying

e n v elo p e w hen r < 1. T h e an g le wo o f th e p o les d e te rm in e s th e fre q u e n c y of
o scillatio n an d th e d istan c e r o f th e p o le s from th e origin d e te rm in e s th e ra te of

Sec. 3.7

Summ ary and References

219

ft(n)

Figure 3.18

Plot of h(n) given by (3.6.43) with bo = 1, a*> = n/4, r = 0.9;

sin[(n + l)twn)u(n).

h(n) = [fcor"/(sin

decay . W h e n r is clo se to u n ity , th e d ecay is slow . W h en r is close to th e origin,

th e d ecay is fast. A typical g ra p h o f h(n) is illu stra te d in Fig. 3.18.

3.7 SUMMARY AND REFERENCES

T h e z -tra n s fo rm p lay s th e sa m e ro le in d isc re te -tim e signals a n d sy stem s as th e
L a p la c e tra n sfo rm d o e s in c o n tin u o u s-tim e signals a n d system s. In th is c h a p te r we
d e riv e d th e im p o rta n t p ro p e rtie s o f th e z-tra n sfo rm , w hich a re e x tre m e ly useful in
th e an aly sis o f d isc re te -tim e system s. O f p a rtic u la r im p o rta n c e is th e co n v o lu tio n
p ro p e rty , w hich tra n sfo rm s th e c o n v o lu tio n o f tw o se q u e n c e s in to a p ro d u c t o f
th e ir z-tran sfo rm s.
In th e c o n te x t o f L T I system s, th e co n v o lu tio n p ro p e rty re su lts in th e p ro d u c t
o f th e z -tra n s fo rm X ( z ) o f th e in p u t signal w ith th e sy stem fu n c tio n H ( z ) , w h e re
th e la tte r is th e z -tra n s fo rm o f th e u n it sa m p le re sp o n s e o f th e sy stem . T his
re la tio n sh ip allo w s u s to d e te rm in e th e o u tp u t o f an L T I sy stem in re s p o n s e to an
in p u t w ith tra n s fo rm X ( z ) b y c o m p u tin g th e p ro d u c t Y ( z ) = H ( z ) X ( z ) a n d th e n
d e te rm in in g th e in v e rse z -tra n sfo rm of Y ( z ) to o b ta in th e o u tp u t se q u e n c e y(n).
W e o b se rv e d th a t m an y signals o f p ra c tic a l in te re s t h a v e r a tio n a l z-tran sfo rm s.
M o re o v e r, L T I sy stem s c h a ra c te riz e d b y c o n s tan t-co efficien t lin e a r d ifferen ce

220

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

e q u a tio n s , also p o ssess r a tio n a l system fu n ctio n s. C o n s e q u e n tly , in determ in in g

th e in v erse z -tran sfo rm , w e n a tu ra lly e m p h a siz e d th e in v ersio n o f ra tio n a l trans
fo rm s. F o r such tra n sfo rm s, th e p a rtia l-fra c tio n ex p an sio n m e th o d is relatively
easy to apply, in c o n ju n ctio n w ith th e R O C , to d e te rm in e th e c o rre sp o n d in g se
q u e n c e in th e tim e d o m ain . T h e o n e-sid ed z -tra n sfo rm w as in tro d u c e d to solve for
th e resp o n se o f cau sal system s ex cited by causal in p u t signals w ith n o n z e ro initial
co n d itio n s.
F inally, w e c o n s id e re d th e c h a ra c te riz a tio n of L T I sy stem s in th e z-transform
d o m a in . In p a rtic u la r, w e re la te d th e p o le - z e r o lo catio n s of a sy stem to its timed o m a in c h a racteristics an d re sta te d th e re q u ire m e n ts fo r sta b ility a n d cau sality of
L T I sy stem s in te rm s o f th e p o le lo catio n s. W e d e m o n s tra te d th a t a causal system
h as a system fu n ctio n H( z ) w ith a R O C |z| >
w h e re 0 < r\ < oc. In a stable
an d cau sal system , th e p o les o f H( z ) lie inside th e u n it circle. O n th e o th e r hand,
if th e system is n o n cau sal, th e c o n d itio n for stab ility re q u ire s th a t th e u n it circle be
c o n ta in e d in th e R O C o f H( z ) . H en ce a n o n cau sal sta b le L T I sy stem has a system
fu n c tio n w ith p o les b o th in sid e an d o u tsid e th e u n it circle w ith an a n n u la r R O C
th a t in clu d es th e u n it circle. T h e S c h u r-C o h n test fo r th e sta b ility o f a causal LTI
sy stem w as d escrib ed and th e stab ility o f se c o n d -o rd e r system w as co n sid e re d in
so m e d etail.
A n ex cellen t c o m p reh en siv e tre a tm e n t o f the z -tra n s fo rm a n d its application
to th e analysis o f L TI system s is given in the tex t by Ju ry (1964). T h e S churC o h n test for sta b ility is tr e a te d in se v era l texts. O u r p r e s e n ta tio n w as given in
th e c o n tex t o f reflec tio n co efficien ts w hich are used in lin e a r p re d ic tiv e cod in g of
sp e ech signals. T h e te x t by M a rk e l an d G ra y (1976) is a go o d re fe re n c e for the
S c h u r-C o h n te st a n d its a p p licatio n to sp e ech signal processing.

PROBLEMS
3.1 Determ ine the z-transform of the following signals,
(a) x(n) = {3. 0. 0. 0, 0, 6, 1. 4}
(} )\ > 5
0.
n <4
3.2 D eterm ine the z-transforms of the following signals and sketch the corresponding
pole-zero patterns.
(a) x ( n ) = (1 + n ) u ( n )
(b) x ( n ) = (a" + a ' ) u( n) , a real
(b) x(n) =

(c)
W)
(e )
(I)

x(n)
x(n)
x(n)
x (n)

=
=
=
=

( 1 )n2 -n u (r)
( n a " sinaon)w(rc)
(na" CQSwon) u( n)
Ar " c o s (w^n + <j>)u(n). 0 < r < 1

(g) *(n) = j( n : i ~
1)
(h) jr(rt) = ( i ) n[(n) - u(n - 10)]

Chap. 3

221

Problems

33 Determine the z-transforms and sketch the ROC of the following signals.
f(i)\

(a) x,(n)

1 ( f) " " .

n>
_0

"<

( i ) B- 2 \
0,
(c) *j() = x 1(n + 4 )
<d) x4(n) = j t , ( - n )
(b) x 2(n) =

3.4 D eterm ine

(a) x(n) =
(b) x(n) =
(c) x(n) =
(d) x(n) =
<e) x(n) =
(f) jr(n) =

n >0
n < 0

the z-transform of the following signals.

n(l)"w(n)
n2u(n)
nanu( n 1)
(-1 )" (cos ~n) u(n)
(l)"u(n)
( 1 ,0 .- 1 ,0 . 1 ,- 1 , . . . }
t

3.5 D eterm ine the regions of convergence of right-sided, left-sided, and finite-duration
two-sided sequences.
3.6 Express the z-transform of
y(n) = Y x(k)
k=*oc
in term s of X (-). [Him: Find the difference y(n) - y(n - 1).]
3.7 Com pute the convolution of the following signals by m eans of the z-transform.
*i(n)

= f ()".
1 ()"".

n >0
n<0

*2 (n) = ( j ) nH(n)
3.8 Use the convolution property to:
(a) Express the z-transform of
y(n) = 5 2 x
tt-oc
in terms of X(z).

(b) Determine the z-transform of x(n) = (n + l)u(n). [Hint. Show first that x ( n ) =
u(n) * n(n).]
3.9 The z-transform X(z) of a real signal x(n) includes a pair of complex-conjugate zeros
and a pair of complex-conjugate poles. What happens to these pairs if we multiply
x(n) by eJWnl (Hint. Use the scaling theorem in the z-domain.)
3.10 Apply the final value theorem to determine
1,
10,

jt( oo )

for the signal

if n is even
otherwise

3.11 Using long division, determine the inverse z-transform of

1 + 2Z-1
1 - 2z 1 + z 2
if (a) jr(n) is causal and (b) x(n) is anticausal.

222

The z-Transform and Its Application to the Analysis of LTI Systems

3.12 Determ ine the causal signal

Chap. 3

having the z-transform

*(z) =

1
( l - Z z - ' H l - z - 1)2

3.13 Let ;t(n) be a sequence with z-transform X(z)- Determ ine, in terms of X(z), the
z-transforms of the following signals.

i,n

(a) j:i(n) =

even

II

if n odd
0,
(b) x2(n) = x(2n)
3.14 D eterm ine the causal signal x (n) if its z-transform X(z) is given by:
l+ 3 z -
(a)
1 + 3 z- + 2 z-2
1
(b)
1 - z - ' + ^ z -2
II

><

II

(d)

><

II

z -6 + z^7

(c)

(e) X(z) =
(0 X(z) =

1 + 6z + z " 2

4 (1 - 2 ; - 1 + 2 z '2)(l - O.Sz"1)
2

1.5z-1

1 - 1.5--1 + C.5z^:
1 + 2z + z-2

II

(g)

11 + 2z-2
1+ :-2

II

(j)

II

1 + 4z_1 + 4z-2
(h) X(z) is specified by a pole-zero p atten
1- k -1
(i)
1 - a z -1

Figure P3.14

3.15 Determ ine all possible signals x(n) associated with the z-transform
X U ) = {1 - 2 z - ') ( 3 - z - ])

3.16 Determ ine the convolution of the following pairs of signals by means of the ztransform.

Chap. 3

(a) x\(ft) =
(b)

223

Problems

Xi(n)

= u(n),

- 1), x 2(n) = [1 +
jc2(n) = 5(n) + (i)"u (n )

(c) x\ (n) = ( i ) nM(n),

(d)

ari(n) = nu(n),

*2(71) = c-Osnnu(n)
x 2(n) = 2 "u(n 1)

3.17 Prove the final value theorem for the one-sided z-transform.
3.18 If X(z) is the z-transform of x(n), show that:
(a) Z { x m(n)} = X ' ( z ' )
(b) Z{Re[jr(n}]} = j[X (z) + X*(z*)]
(c) Z{Im[jr()]l = |[X (z) - **(=*)]
(d) If
* * ( )= { * (? )
1 0,

if " A integer
otherwise

then
X t U) = X (z k)

(e)
= X(ze~Jw)
3.19 By first differentiating X(z) and then using appropriate properties of the z-transform.
determ ine x(n) for the following transforms.

(a) X(z) = l o g ( l 2 z),

\z\<{

(b) X(z) = log(l - z-1), |z! > 5

3.20 (a) Draw the pole-zero pattern for the signal
jcj(n) = (r" sin a>on)u(fi)

0 < r < 1

(b) Com pute the z-transform A^tz), which corresponds to the pole-zero pattern in
part (a).
(c) Com pare X]{z) with X 2(z). Are they indentical? If not. indicate a m ethod lo
derive Xi(z) from the pole-zero pattern.
3.21 Show that the roots of a polynomial with real coefficients are real or form complexconjugate pairs. The inverse is not true, in general.

3.22 Prove the convolution and correlation properties of the z-transform using only its
definition.

3.23 D eterm ine the signal x(n) with z-transform

X (z) = e: + e l/z

|z|^0

3.24 D eterm ine, in closed form, the causal signals x(n) whose z-transforms are given by:
(a) X(z) = T T T 5 ^ T

o3

(b) X(Z) = 1 - 0.5z~! + 0.6z 2

Partially check your results by computing *(0), x (l), *(2), and
m ethod.

jt( oo)

by an alternative

3.25 D eterm ine all possible signals that can have the following z-transforms.
1

224

The z - T ransform and Its Application to the Analysis of LTI Systems

Chap. 3

3.26 Determ ine the signal x(n) with z-transform

XU) =

+ z- 2

1-

if X(z) converges on the unit circle.

3 .2 7 Prove the complex convolution relation given by (3.2.22).
3.28 Prove the conjugation properties and Parse vals relation for the z-transform given in

Table 3.2.
3.29 In Example 3.4.1 we solved for .r(n), n < 0, by perform ing contour integrations for
each value of n. In general, this procedure proves to be tedious. It can be avoided by
making a transform ation in the contour integral from z-plane to the uj = 1/z plane.
Thus a circle of radius R in the z-plane is mapped into a circle of radius 1/ R in the wplane. As a consequence, a pole inside the unit circle in the z-plane is mapped into a
pole outside the unit circle in the m-plane. By making the change of variable w = 1/z
in the contour integral, determ ine the sequence x ( n ) for n < 0 in Example 3.4.1,
n < N 1 be a finite-duration sequence, which is also real-valued and
even. Show that the zeros of the polynomial X(z) occur in mirror-image pairs about
the unit circle. That is. if z = rej/> is a zero of X(z), then z = (1 / r ) e J" is also a zero.

3.30 Let *(n), 0 <

3.31 Compute the convolution of the following pair of signals in the time domain and by

using the one-sided z-transform.

(a) .v,(n) = {1. 1. 1. 1. 1).
x 2(n) = (1. 1. 1)
t

x 2(n) = (i)"u(n)

(b) Xi (n) = ( j)"u(h).

x 2(n) =

(c) .ii(n) = (1.2, 3.4}.

(4,

3, 2. 1}

(d) *,() = {1.1. 1.1.1}.

Jr2 ( )

= {1.1,1}

Did you obtain the same results by both methods? Explain.

3.32 D eterm ine the one-sided z-transform of the constant signal x ( n ) = 1. oo < n < 00 .
3 .3 3 Prove that the Fibonacci sequence can be thought of as the impulse response of the
system described by the difference equation ,y(/i) = v(n - 1) + y ( n - 2) 4- x(n). Then

determ ine h(n) using z-transform techniques.

3 .3 4 Use the one-sided z-transform to determ ine y(n), n > 0 in the following cases.

(a) y(n) + \ y( n - 1) - \ y( n - 2) = 0;

y ( - l ) = y (-2 ) = 1

(b) y(n) - 1.5y(n - 1) + 0.5y(n - 2) = 0;

y ( - l ) = 1. v(2) = 0

(c) v(n) = \ y ( n - 1) + x ( n )
x ( n ) = ()"u(rt).

y(-l) = 1

(d) _v(r) = j v(n - 2) + x(n)

x ( n ) = u{n)

>(1) = 0;

y(2) = 1

3.3 5 Show that the following systems are equivalent.

(a) y ( n ) = 0 .2 y (n - 1) + x { n ) - 0.3.r(n - 1) + 0.02jt(n - 2)

(b) _y{) = x ( n ) - 0 .1 x (n - 1)

Chap. 3

225

Problems

3.36 Consider the sequence xin) = ci"uin). 1 < a < 1. D eterm ine at least two sequences
that are not equal to xin) but have the same autocorrelation.
3.37 Com pute the unit step response of the system with impulse response
f3\

n < 0
n >0

3.38 Com pute the zero-state response for the following pairs of systems and input signals.
(a) hin) =

,v(h) = <j)"^cos w r j ;<('?)

(b) h(n) = ( | Yu Oi ). xi n) = { \ ) u{n) + ( \ r " u ( - n - 1)

(c) yin) = 0.1 yin 1) + 0.2y(>i - 2) + xin) + xin - 1)
,v ( / i) =

( ^ V m (i i )

(d) yin) = |.v() -

1)

x{n) = lO^Cos n'ju(n)

(e) yin) = yin 2 ) + lOjr(n)
,v(/7) = lO^cos n^jnin)
(f) h{n) = i ^ Y ii(n). x i n ) i/ln) it in 7)

(g) hin) = (|V'if(H). xin) = ( 1)", 3C < n < x

(h) hin) = U) ' i

= in + 1)(j)"h<h)

3.39 Consider the system

1 - 2 r ' + 2:~: ----------;---- ----------- :-----------------(
1
- 0 . 5 ; - ) ( l - 0. 2: - ' )

H(Z) =

ROC: 0.5 < |d < l

(a) Sketch the pole-zero pattern. Is the system stable?

(b) D eterm ine the impulse response of the system.
3.40 Com pute the response of the system
ytn) = 0.7v(n - 1) 0.12y(n - 2) + x(n 1) + xin 2)
to the input v(/?) = nuin), Is the system stable?
3.41 D eterm ine the impulse response and the step response of the following causal systems.
Plot the pole-zero patterns and determ ine which of the systems are stable.
(a ) y i n ) = j v( - 1) - jr v( n - 2) + x ( n )

(b) y(n) = vin 1) 0.5y(n - 2) + x(n) + xin I )

; ~ ' f l

(c> =
(d) v(n) = 0.6y(n 1) 0.08v(n 2) -f- x(n)
(e) v() = 0.7y(n 1) 0.1y(/3 2) + 2xin) x(n 2)
3.42 Let xin) be a causal sequence with ;-transform X(z) whose pole-zero plot is shown
in Fig. P3.42. Sketch the pole-zero plots and the R O C of the following sequences;
(a) Xt(n) = x ( ~ n -)- 2)
(b) x 2(n) = eim/iu,x(n)

226

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Im(z)

Re(;)

Figure P3.42
3.43 W e want to design a causal discrete-time LTI system with the property that if the

input is
x( n ) = ( l ) nu ( n) -

- 1)

then the output is

V(lt) = (|)"tt(n)
(a) D eterm ine the impulse response h(n) and the system function H(z) of a system
that satisfies the foregoing conditions.
(b) Find the difference equation that characterizes this system.
(c) D eterm ine a realization of the system that requires the minimum possible amount
of memory.
(d) D eterm ine if the system is stable.
3 .4 4 D eterm ine the stability region for the causal system
H{z) =

1 + o ir -1 + a2z~2
by computing its poles and restricting them to be inside the unit circle.
3.45 Consider the svstem

H{z) =

_L J?i-r-2
I1 _ l --1 +
Determine:
(a) The impulse response
(b) The zero-state step response
(c) The step response if y(1) = 1 and y ( -2 ) = 2
3 .4 6 D eterm ine the system function, impulse response, and zero-state step response of the
system shown in Fig P3.46.
3 .4 7 Consider the causal system

y in ) = - a i y i n - 1) + b0x(n) + b^xin - 1)

Determine:
(a) The impulse response

Chap. 3

227

Problems

Xi.fl)

y(n)

a
Figure P3.46

(b) The zero-state step response

(c) The step response if y ( 1) = A / 0
(d) The response to the input
x (n ) ~

CO S a> u f!

<

<

oo

3.48 Determ ine the zero-state response of the system

y(n) = i v(h - 1) + 4*(n) + 3x(n 1)

to the input
jt(n) = em 'nu{n)
W hat is the steady-state response of the system?
3.49 Consider the causal system defined by the pole-zero pattern shown in Fig. P3.49.
(a) Determ ine the system function and the impulse response of the system given that
W U )U i = 1.
(b) Is the system stable?
(c) Sketch a possible im plementation of the system and determ ine the corresponding
difference equations.
Im(;)

3.50 An FIR LTI system has an impulse response h(n), which is real valued, even, and
has finite duration of 2/V + 1. Show that if ;i = rejaJa is a zero of the system, then
d = ( l / r ) e j a *> is also a zero.
3.51 Consider an LTI discrete-time system whose pole-zero pattern is shown in Fig. P3.51.
(a) D eterm ine the R O C of the system function H(z) if the system is known to be
stable.

228

The z-Transform and Its Application to the Analysis of LTI Systems

Chap. 3

Im(;)

Re<z)

-0 .5

Figure P3.51

(b ) It is possible for the given pole-zero plot to correspond to a causal and stable

system? If so, what is the appropriate ROC?

(c) How many possible systems can be associated with this pole-zero pattern?
3.52 Let x(n) be a causal sequence.
(a) What conclusion can you draw about the value of its z-transform A(c) at ; = oo?
(b ) Use the result in part (a) to check which of the following transform s cannot be
associated with a causal sequence.

3.53 A causal pole-zero system is BIBO stable if its poles are inside the unit circle. Con
sider now a pole-zero system that is BIBO stable and has its poles inside the unit
circle. Is the system always causal? [Hint: Consider the systems h\(n) = anu(n) and
ti2 (n) = anu{n + 3), |a| < 1.]
3 3 4 Let *(/i) be an anticausal signal [i.e., x (n) = 0 for n > 0]. Form ulate and prove an
initial value theorem for anticausal signals.
3.55 The step response of an LTI system is
J(w) =

+ 2)

(a) Find the system function H(z) and sketch the pole-zero plot.
(b ) D eterm ine the impulse response ft(n).

(c) Check if the system is causal and stable.

3 3 6 Use contour integration to determ ine the sequence jc(n) whose z-transform is given

Chap. 3

229

Problems

; a
1 a:
1-

(c) X'U) = --------<d) A'C)

3.57 Let ,v(h) be a sequence with c-transform

X d = --- ---------
(1 - t;r)(l - a ; " 1)

R O C : a < |;| < 1ja

with 0 < a < 1. Determ ine ,v() by using contour integration

3.58 The ^-transform of a sequence .v(n) is given by

X(;> =

(: - i ) ( ; - 2)5<- + ?): (r + 3)

Furthermore it is known that X (:) converges for |;| = 1.

(a ) Determ ine the R O C of X(c).
(b) Determ ine xin ) at n = -18. {Hint: Use contour integration.)

34

Frequency Analysis of Signals

and Systems

T h e F o u rie r tra n sfo rm is o n e o f se v e ra l m a th e m a tic a l to o ls th a t is useful in the

an alysis an d d esign o f L T I system s. A n o th e r is th e F o u rie r se ries. T h ese signal
re p re se n ta tio n s basically involve th e d e c o m p o sitio n o f th e sig n als in te rm s o f sinu
so id al (o r co m p lex e x p o n e n tia l) c o m p o n e n ts. W ith such a d e c o m p o s itio n , a signal
is said to be re p re se n te d in th e f r e q u e n c y domai n .
A s w e shall d e m o n s tra te , m ost signals o f p ractical in te re st can be d eco m p o sed
in to a sum o f sin u so id al signal c o m p o n e n ts. F o r th e class of p e rio d ic signals, such
a d eco m p o sitio n is called a Fouri er series. F o r th e class o f finite e n e rg y signals, the
d eco m p o sitio n is called th e Fouri er t ransf orm. T h e se d e c o m p o s itio n s are ex trem ely
im p o rta n t in th e an aly sis o f L T I system s becau se th e re sp o n se o f a n L T I system to
a sin u so id al in p u t signal is a sin u so id o f th e sam e freq u e n c y b u t o f d iffe re n t am pli
tu d e a n d p h ase. F u rth e rm o re , th e lin e a rity p ro p e rty o f th e L T I sy stem im plies th a t
a lin ear sum o f sin u so id al c o m p o n e n ts at th e in p u t p ro d u c e s a sim ilar lin e a r sum
o f sin u so id al c o m p o n e n ts a t th e o u tp u t, w hich d iffer only in th e a m p litu d e s and
p h ases fro m th e in p u t sin u so id s. T h is c h a ra c te ristic b e h a v io r o f L T I sy stem s ren
d e rs th e sin u so id al d e c o m p o sitio n o f signals v ery im p o rta n t. A lth o u g h m an y o th er
d e c o m p o sitio n s o f signals a re p o ssib le, only th e class o f sin u so id al (o r co m p lex ex
p o n e n tia l) signals p o ssess th is d e sira b le p ro p e rty in passin g th ro u g h an L T I system .
W e begin o u r stu d y o f fre q u e n c y analysis o f signals w ith th e re p re se n ta tio n
o f co n tin u o u s-tim e p e rio d ic a n d a p e rio d ic signals by m e a n s of th e F o u rie r series
an d th e F o u rie r tra n sfo rm , resp ec tiv ely . T h is is follow ed by a p a ra lle l tre a tm e n t
o f d isc rete-tim e p erio d ic a n d a p e rio d ic signals. T h e p r o p e rtie s o f th e F o u rie r
tra n sfo rm a re d e sc rib e d in d e ta il an d a n u m b e r of tim e -fre q u e n c y d u alities are
p re se n te d .

4.1 FREQUENCY ANALYSIS OF CONTINUOUS-TIME SIGNALS

It is w ell k n o w n th a t a prism can be u sed to b re a k u p w h ite light (su n lig h t) in to the
co lo rs o f th e ra in b o w (see Fig. 4.1a). In a p a p e r su b m itte d in 1672 to th e R oyal
Society, Isaac N e w to n u se d th e te rm spe ct r um to d escrib e th e c on t i n u o u s b an d s
230

Sec. 4.1

231

Frequency Analysis of Continuous-Time Signals

Glass prism

Figure 4.1

(a) Analysis and

( b ) sy n th e sis o f th e w h ite light (s u n lig h t)

u sin g glass p rism s.

o f co lo rs p ro d u c e d by this a p p a ra tu s. T o u n d e rsta n d this p h e n o m e n o n , N ew to n

p laced a n o th e r p rism u pside-dow rn w ith resp ec t to th e first, an d sh o w ed th a t th e
co lo rs b le n d e d back into w hite light, as in Fig. 4.1b, By in se rtin g a slit b etw een
the tw o p rism s a n d blo cking o n e o r m o re colors from h ittin g the se co n d prism ,
he sh o w ed th a t th e rem ix ed light is no lo n g er w hite. H e n c e th e light passing
th ro u g h th e first p rism is sim ply an aly zed into its c o m p o n e n t co lo rs w ith o u t any
o th e r ch an g e. H o w e v e r, o n ly if we mix again all o f th ese c o lo rs d o we o b ta in the
o rig in a l w hite light.
L a te r, Jo s e p h

F r a u n h o f e r ( 1 7 8 7 - 1 8 2 6 ) . in m a k in g m e a s u r e m e n ts o f lig h t

e m itt e d b y th e su n a n d s ta r s , d is c o v e r e d th a t th e s p e c tr u m o f th e o b s e r v e d lig h t
c o n s is ts o f d is tin c t c o l o r lin e s . A fe w y e a r s la te r ( m i d - ] 8 0 0 s ) G u s ta v K i r c h h o f f a n d
R o b e r t B u n s e n fo u n d th a t e a c h c h e m ic a l e le m e n t, w h e n h e a te d to in c a n d e s c e n c e ,
r a d ia te d its o w n d is tin c t c o lo r o f lig h t. A s a c o n s e q u e n c e , e a c h c h e m ic a l e le m e n t
c a n b e id e n tifie d b y its o w n lin e sp e ctru m .
F r o m p h y s ic s w e k n o w th a t e a c h c o lo r c o r r e s p o n d s to a s p e c ific f r e q u e n c y o f
th e v is ib le s p e c tr u m . H e n c e th e a n a ly sis o f lig h t in to c o lo r s is a c tu a lly a f o r m o f

fr e q u e n c y a n a ly s is .
F r e q u e n c y a n a ly s is o f a s ig n a l in v o lv e s th e r e s o lu tio n o f th e s ig n a l in to its
fr e q u e n c y ( s in u s o id a l) c o m p o n e n ts .

In s te a d o f lig h t, o u r s ig n a l w a v e fo r m s a re

b a s ic a lly fu n c tio n s o f tim e . T h e r o le o f th e p ris m is p la y e d b y th e F o u r i e r a n a ly sis

to o ls th a t w e w ill d e v e lo p :

th e F o u r ie r s e r ie s a n d th e F o u r i e r t r a n s f o r m .

The

r e c o m b in a tio n o f th e s in u s o id a l c o m p o n e n ts to r e c o n s tr u c t th e o r ig in a l s ig n a l is
b a s ic a lly a F o u r i e r s y n th e s is p r o b le m . T h e p r o b le m o f s ig n a l a n a ly s is is b a s ic a lly
th e s a m e fo r th e c a s e o f a s ig n a l w a v e fo rm a n d f o r th e c a s e o f th e lig h t f r o m h e a te d
c h e m ic a l c o m p o s itio n s .

J u s t a s in th e c a s e o f c h e m ic a l c o m p o s itio n s , d if f e r e n t

s ig n a l w a v e fo r m s h a v e d iffe r e n t s p e c tr a . T h u s th e s p e c tr u m p r o v id e s a n id e n tity

232

Frequency Analysis of Signals and Systems

Chap. 4

o r a s ig n a tu r e f o r th e s ig n a l in th e s e n s e t h a t n o o t h e r s ig n a l h a s th e s a m e sp e c tru m .
A s w e w ill s e e , th is a ttr ib u te is r e la te d to th e m a th e m a tic a l t r e a t m e n t o f fre q u e n c y d o m a in te c h n iq u e s .
I f w e d e c o m p o s e a w a v e fo r m in to s in u s o id a l c o m p o n e n ts , in m u c h th e sa m e
w ay t h a t a p ris m s e p a r a te s w h ite lig h t in to d iffe r e n t c o lo r s , th e su m o f th e s e
s in u s o id a l c o m p o n e n ts re s u lts in th e o r ig in a l w a v e fo r m . O n th e o t h e r h a n d , if any
o f th e s e c o m p o n e n ts is m is s in g , th e r e s u lt is a d if f e r e n t s ig n a l.
I n o u r tr e a t m e n t o f fr e q u e n c y a n a ly s is , w e w ill d e v e lo p th e p r o p e r m a th e
m a tic a l to o ls ( p r is m s ) f o r th e d e c o m p o s itio n o f s ig n a ls ( l i g h t ) in to s in u so id a l
f r e q u e n c y c o m p o n e n ts ( c o l o r s ) . F u r t h e r m o r e , th e t o o ls ( in v e r s e p r is m s " ) f o r sy n
th e s is o f a g iv e n s ig n a l f r o m its f r e q u e n c y c o m p o n e n ts w ill a ls o b e d e v e lo p e d .
T h e b a s ic m o t iv a tio n f o r d e v e lo p in g th e fr e q u e n c y a n a ly s is to o ls is to p ro v id e
a m a th e m a tic a l a n d p ic to r ia l r e p r e s e n t a t io n f o r th e f r e q u e n c y c o m p o n e n ts th a t a re
c o n ta in e d in a n y g iv e n s ig n a l. A s in p h y s ic s , th e te r m s pe ct r um is u s e d w h e n r e f e r
rin g t o th e fr e q u e n c y c o n t e n t o f a s ig n a l. T h e p r o c e s s o f o b ta in in g th e s p e c tru m
o f a g iv e n sig n a l u sin g th e b a s ic m a th e m a tic a l t o o ls d e s c r ib e d in th is c h a p te r is
k n o w n a s f re q u e n c y o r spectral analysis.

In c o n tr a s t, th e p r o c e s s o f d e te r m in in g

th e s p e c tr u m o f a s ig n a l in p r a c tic e , b a s e d o n a c tu a l m e a s u r e m e n ts o f th e sig n a l,
is c a lle d spect rum estimation.

T h is d is tin c tio n is v e r y im p o r ta n t.

In a p r a c tic a l

p r o b le m th e s ig n a l to b e a n a ly z e d d o e s n o t le n d it s e lf to a n e x a c t m a th e m a tic a l
d e s c r ip tio n . T h e s ig n a l is u s u a lly s o m e i n f o r m a tio n - b e a r in g s ig n a l fr o m w h ich we
a r e a tte m p tin g t o e x t r a c t th e r e le v a n t in f o r m a tio n . I f th e in f o r m a tio n th a t w e wish
to e x t r a c t c a n b e o b ta in e d e i t h e r d ir e c tly o r in d ir e c tly fr o m th e s p e c tr a l c o n te n t o f
th e s ig n a l, w e c a n p e r fo r m spe ct rum est imation o n th e in f o r m a t io n - b e a r in g sig n a l,
a n d th u s o b ta in a n e s tim a te o f th e s ig n a l s p e c tr u m . In f a c t, w e c a n v iew s p e c tra l
e s tim a tio n as a ty p e o f s p e c tr a l a n a ly s is p e r fo r m e d o n s ig n a ls o b t a i n e d f r o m p h y si
ca l s o u r c e s (e .g ., s p e e c h , E E G , E C G , e t c .) . T h e in s tr u m e n ts o r s o f tw a r e p r o g ra m s
u se d t o o b ta in s p e c tr a l e s tim a te s o f s u c h s ig n a ls a r e k n o w n a s sp e c t r u m analyzers.
H e r e , w e w ill d e a l w ith s p e c tr a l a n a ly s is . H o w e v e r , in C h a p t e r 12 w e sh all
t r e a t th e s u b je c t o f p o w e r s p e c tr u m e s tim a tio n .

4.1.1 The Fourier Series for Continuous-Time Periodic

Signals
In th is s e c tio n w e p r e s e n t th e f r e q u e n c y a n a ly s is to o ls fo r c o n tin u o u s - tim e p e
r io d ic s ig n a ls .

E x a m p le s o f p e r io d ic s ig n a ls e n c o u n te r e d in p r a c t i c e a r e s q u a re

w a v e s , r e c ta n g u la r w a v e s , tr ia n g u la r w a v e s , a n d o f c o u r s e , s in u s o id s a n d c o m p le x
e x p o n e n tia ls .
T h e b a s ic m a th e m a tic a l r e p r e s e n ta tio n o f p e r io d ic s ig n a ls is th e F o u r i e r s e
r ie s , w h ic h is a lin e a r w e ig h te d su m o f h a r m o n ic a lly r e la te d s in u s o id s o r c o m p le x
e x p o n e n tia ls . J e a n B a p t is t e J o s e p h F o u r i e r ( 1 7 6 8 - 1 8 3 0 ) , a F r e n c h m a th e m a tic ia n ,
u se d s u c h t r ig o n o m e t r ic s e r ie s e x p a n s io n s in d e s c r ib in g th e p h e n o m e n o n o f h ea t
c o n d u c tio n a n d te m p e r a tu r e d is tr ib u tio n th r o u g h b o d ie s . A lth o u g h h is w o r k was
m o t iv a te d b y th e p r o b le m o f h e a t c o n d u c tio n , th e m a th e m a tic a l te c h n iq u e s th a t

Sec. 4.1

Frequency Analysis of C ontinuous-Tim e Signals

233

h e d e v e lo p e d d u rin g th e e a r ly p a r . o f th e n in e t e e n t h c e n tu r y n o w fin d a p p lic a

tio n in a v a r ie ty o f p r o b le m s e n c o r r .r \ is s in g m a n y d if f e r e n t f ie ld s , in c lu d in g o p tic s ,
v ib r a tio n s in m e c h a n ic a l s y s te m s , s y s t e m th e o r y , a n d e le c t r o m a g n e t ic s .
F r o m C h a p t e r 1 w e r e c a ll t h .i : a lin e a r c o m b in a t io n o f h a r m o n ic a lly r e la te d
c o m p le x e x p o n e n tia ls o f th e fo r m

x {r) =

cke j 2 * kF

( 4 .1 .1 )

i = -3C
is a p e r io d ic s ig n a l w ith f u n d a m e n t a l p e r io d Tp =

1/Fo.

H e n c e w e c a n th in k o f

th e e x p o n e n t ia l s ig n a ls

{ e i i x k p k = Q i i 2 l
a s th e b a s ic b u ild in g b lo c k s f r o m

w h ic h w e c a n c o n s t r u c t p e r io d ic s ig n a ls o f

v a r io u s ty p e s b y p r o p e r c h o ic e o f t h e fu n d a m e n ta l f r e q u e n c y a n d th e c o e f f ic ie n ts
( q ). F o d e te r m in e s th e f u n d a m e n ta l p e r i o d o f x ( t ) a n d th e c o e f f ic ie n t s { }

s p e c ify

th e s h a p e o f th e w a v e fo rm .
S u p p o s e th a t w e a r e g iv e n a p e r io d i c s ig n a l x { i) w ith p e r io d Tp .

W e ca n

r e p r e s e n t th e p e r io d ic sig n a l by t h e s e r ie s ( 4 .1 .1 ) , c a lle d a F o u r ie r series, w 'here

th e f u n d a m e n ta l fr e q u e n c y Fo is s e l e c t e d to b e th e r e c ip r o c a l o f th e g iv e n p e r io d

Tp . T o d e te r m in e th e e x p r e s s io n t o r th e c o e f f i c i e n t s ( q ) , w e firs t m u ltip ly b o th
sid e s o f ( 4 .1 .1 ) b y th e c o m p le x e x p o n e n t i a l

Fltl!
w h e r e I is a n in t e g e r an d th e n in t e g r a t e b o th s id e s o f th e r e s u ltin g e q u a tio n o v e r
a s in g le p e r io d , s a y fr o m 0 to T r , o r m o r e g e n e r a lly , fr o m fo to r0 + T p, w h e r e i o is
a n a r b itr a r y b u t m a th e m a tic a lly c o n v e n i e n t s ta r t in g v a lu e . T h u s w e o b ta in
fh>+Tr

J'

r'o+Tr

x ( t ) e - j2*IF,''dt = J'

g- j t oi Kt /

oc

cke+J2nkF"' J di

(4.1.2)

T o e v a lu a te th e in te g r a l o n th e r ig h t- h a n d s id e o f ( 4 .1 .2 ) , w e in te r c h a n g e th e o r d e r
o f th e s u m m a tio n a n d in te g r a tio n a n d c o m b in e th e tw o e x p o n e n tia ls . H e n c e

rl

sc

,, + r .

c*

OC

ei2* F{k-'"dt =

i = oc

k= oc

p j27rF < M -hl

Ck

- |'n + r ,

J l n F o i k - /)_

( 4 .1 .3 )

F o r k ^ I, th e r ig h t-h a n d s id e o f ( 4 .1 .3 ) e v a lu a te d a t th e lo w e r a n d u p p e r lim its , Zo

a n d t0 + Tp , r e s p e c tiv e ly , y ie ld s z e r o . O n th e o t h e r h a n d , if k = /, w e h a v e

fJtn

di i

Consequently, (4.1.2) reduces to

'o+Tp

fJ /ft

x ( t ) e - j2 * ,Fo'd t = c ,T p

Frequency Analysis of Signals and System s

234

Chap. 4

a n d t h e r e f o r e th e e x p r e s s io n f o r th e F o u r i e r c o e f fic ie n ts in t e r m s o f th e g iv e n
p e r io d ic s ig n a l b e c o m e s

fto+TP

I
c, =

x ( l ) e - jlw,F"'dt

Tp
S in c e fo is a r b itr a r y , th is in te g r a l c a n b e e v a lu a te d o v e r a n y i n te r v a l o f le n g th Tp,
th a t is, o v e r a n y in te r v a l e q u a l t o th e p e r io d o f th e s ig n a l jr ( r ) . C o n s e q u e n tly , th e
in te g r a l fo r th e F o u r i e r s e r ie s c o e f f ic ie n t s w ill b e w r itte n as

c, = f

x ( t ) e ~ j2nlF,>'d t

( 4 .1 .4 )

Tp J t

A n im p o r ta n t is s u e th a t a r is e s in th e r e p r e s e n ta tio n o f th e p e r io d ic s ig n a l

x ( t ) b y th e F o u r ie r s e r ie s is w h e t h e r o r n o t th e s e r ie s c o n v e r g e s t o * ( f ) f o r e v e ry
v a lu e o f r, th a t is, if th e s ig n a l x (t ) a n d its F o u r ie r s e r ie s r e p r e s e n t a t io n
OC

c ke j2 * kFtt'

( 4 .1 .5 )

A=-oc
a r e e q u a l a t e v e r y v a lu e o f t.

T h e s o -c a lle d D ir ic h le t c o n d it io n s g u a r a n te e th a t

th e s e r ie s ( 4 .1 .5 ) w ill b e e q u a l to x ( t ), e x c e p t a t th e v a lu e s o f i f o r w h ich jc (r ) is
d is c o n tin u o u s .

A t th e s e v a lu e s o f r, ( 4 .1 .5 ) c o n v e r g e s t o th e m id p o in t (a v e r a g e

v a lu e ) o f th e d is c o n tin u ity . T h e D i r ic h l e t c o n d itio n s a r e :

1 . T h e s ig n a l .r ( f ) h a s a fin ite n u m b e r o f d is c o n tin u itie s in a n y p e r io d .
2 . T h e sig n a l x ( t ) c o n ta in s a fin ite n u m b e r o f m a x im a an d m in im a d u rin g an y
p e rio d .
3 . T h e s ig n a l x (t ) is a b s o lu te ly in te g r a b le in a n y p e r io d , t h a t is.

\x (t )\d t < oo

( 4 .1 .6 )

J Tr
A lt p e r io d ic s ig n a ls o f p r a c tic a l i n te r e s t s a tis fy th e s e c o n d itio n s .
T h e w e a k e r c o n d itio n , t h a t th e s ig n a l h a s fin ite e n e r g y in o n e p e r io d ,

\x ( t ) \2d i < o c

( 4 .1 .7 )

J tp

g u a r a n te e s th a t th e e n e r g y in th e d if f e r e n c e s ig n a l
OC

e (t) = x { t ) -

Ckej2kF"'
k=oc

is z e r o , a lth o u g h * ( ; ) a n d its F o u r i e r s e r ie s m a y n o t b e e q u a l f o r a ll v a lu e s o f t.
N o te th a t ( 4 .1 .6 ) im p lie s ( 4 .1 .7 ) , b u t n o t v ic e v e r s a .

A l s o , b o t h ( 4 .1 .7 ) a n d th e

D ir i c h le t c o n d itio n s a r e s u f fic ie n t b u t n o t n e c e s s a r y c o n d itio n s ( i .e ., th e r e a r e s ig

n a ls t h a t h a v e a F o u r ie r s e r ie s r e p r e s e n t a t i o n b u t d o n o t s a tis fy t h e s e c o n d itio n s ) .
In s u m m a r y , i f x ( t ) is p e r io d ic a n d s a tis fie s th e D i r ic h l e t c o n d itio n s , it ca n
b e r e p r e s e n te d in a F o u r i e r s e r ie s a s in ( 4 .1 .1 ) , w h e r e th e c o e f f i c i e n t s a r e s p e c ifie d
b y ( 4 .1 .4 ) . T h e s e r e la t io n s a r e s u m m a r iz e d b e lo w .

Sec. 4.1

Frequency Analysis of Continuous-Time Signals

235

FR E Q U E N C Y A N A LY S IS O F C O N T IN U O U S -TIM E PE R IO D IC S IG N A LS

Synthesis equation

-vU) = ^

Analysis equation

ct =

(4.1.8)

cke,2~kl'"'

j
1r Jrp

(4.1.9)

In g e n e r a l, th e F o u r i e r c o e ff ic ie n ts c k a r e c o m p le x v a lu e d .
e a s ily sh o w n t h a t if th e p e r io d ic s ig n a l is r e a l, c k a n d

M o r e o v e r , it is

a r e c o m p le x c o n ju g a te s .

A s a r e s u lt, if

ct = \ck\e}b
th e n

C-U ~~
C o n s e q u e n tly , th e F o u r i e r s e r ie s m ay a ls o b e r e p r e s e n te d in t h e fo r m

rv
a (M =

(-(, +

k i I c o s 0 -7 1 k

F()t

6k )

( 4 .1 .1 0 )

n-i
w h e r e t,, is r e a l v a lu e d w h e n x ( i) is r e a l.
F in a lly , w e s h o u ld in d ic a te th a t y e t a n o th e r fo r m f o r th e F o u r ie r s e r ie s ca n
b e o b ta in e d by e x p a n d in g th e c o s in e fu n c tio n in ( 4 .1 .1 0 ) as
c o s ( 2 jt A / v + 0k) c o s 2 n k F o i c o s 0t s i n l n k F ^ r s in fy
C o n s e q u e n tly , w e c a n re w r ite ( 4 .1 .1 0 ) in th e fo rm

3t
ji ( r ) ao + Y 2 ( a ic c o $ 2 n k F o t bk s i n 2 -n k F o i)

( 4 .1 .1 1 )

1=1
w h ere
<3() = Co
fl* = 2|C|- [ c o s 0k

bk = 2|q| sin # *
T h e e x p r e s s io n s in ( 4 .1 .8 ) , ( 4 ,1 .1 0 ) , a n d ( 4 .1 ,1 1 ) c o n s t itu te t h r e e e q u iv a le n t fo r m s
f o r th e F o u r i e r s e r ie s r e p r e s e n ta tio n o f a re a l p e r io d ic s ig n a l.

4.1.2 Power Density Spectrum of Periodic Signals

A p e r io d ic s ig n a l h a s in fin ite e n e r g y a n d a fin ite a v e r a g e p o w e r , w h ic h is g iv e n a s

Px = ~

f
P 'T .

\x ( t ) \2d t

( 4 .1 .1 2 )

Frequency Analysis of Signals and Systems

236

Chap. 4

I f w e ta k e th e c o m p le x c o n ju g a te o f ( 4 .1 .8 ) an d s u b s t itu te f o r * * ( / ) in ( 4 .1 .1 2 ) . we
o b ta in

OC

( 4 .1 .1 3 )
DC

E
k~ 3C

T h e r e f o r e , w e h a v e e s ta b lis h e d th e r e la tio n
( 4 .1 .1 4 )
w h ich is c a lle d P a r s e v a l's re la tio n fo r p o w e r s ig n a ls .
T o illu s tr a te th e p h y s ic a l m e a n in g o f ( 4 .1 .1 4 ) , s u p p o s e th a t .v(r) c o n s is ts o f a
s in g le c o m p le x e x p o n e n tia l

x (t) = c , e j27!tFn'
In th is c a s e , all th e F o u r ie r s e r ie s c o e f f ic ie n ts e x c e p t c* a r e z e r o .

C o n s e q u e n tly ,

th e a v e r a g e p o w e r in th e sig n a l is

It is o b v io u s th a t |q |: r e p r e s e n ts th e p o w e r in th e Ath h a r m o n ic c o m p o n e n t o f th e
sig n a l. H e n c e th e to ta l a v e r a g e p o w e r in th e p e r io d ic sig n a l is s im p ly th e su m o f
th e a v e r a g e p o w e r s in a ll th e h a r m o n ic s .
I f w e p lo t th e |q|2 as a fu n c t io n o f th e f r e q u e n c ie s kF o, k = 0 , 1 , 2 ..........th e
d ia g r a m th a t w e o b ta in sh o w s h o w th e p o w e r o f th e p e r io d ic s ig n a l is d is tr ib u te d
a m o n g th e v a r io u s f r e q u e n c y c o m p o n e n ts .

T h is d ia g r a m , w h ic h is illu s tr a te d in

F ig . 4 .2 , is c a lle d th e p o w e r d e n sity sp e ctru m * o f th e p e r io d ic s ig n a l x ( t ). S in c e th e

Pow er density spectrum

- 4 F 0 - 3 F 0 2Fn F0
Figure 4.2

lct P

Fn

2F0 3F0 4F0

Frequency. F

Pow er density spectrum of a continuous-tim e periodic signal.

This function is also called the power spectral density or. simply, the power spectrum.

237

Frequency Analysis of Continuous-Time Signals

Sec. 4.1

p o w e r in a p e r io d ic s ig n a l e x is ts o n ly at d is c r e te v a lu e s o f f r e q u e n c ie s ( i .e .. F 0.

F o . 2 F q . . . . ) . th e s ig n a l is s a id to h a v e a lin e sp e ctru m . T h e s p a c in g b e tw e e n
tw o c o n s e c u tiv e s p e c tr a l lin e s is e q u a l to th e r e c ip r o c a l o f th e fu n d a m e n ta l p e rio d

Tp . w h e r e a s th e s h a p e o f th e s p e c tr u m ( i.e .. th e p o w e r d is tr ib u tio n o f th e s ig n a l),

d e p e n d s o n th e tim e -d o m a in c h a r a c t e r is t ic s o f th e sig n a l.
A s in d ic a te d in th e p r e c e d in g s e c t i o n , th e F o u r ie r s e r ie s c o e f f ic ie n ts ( q ) a re
c o m p le x v a lu e d , th a t is. th e y c a n b e r e p r e s e n te d as

Ck = \ck\eJhL
w h ere

6k = 4-Q
In s te a d o f p lo ttin g th e p o w e r d e n sity s p e c tr u m , w e c a n p lo t th e m a g n itu d e v o lta g e
s p e c tr u m {|ot|} a n d th e p h a s e s p e c tr u m {(?*} as a fu n c tio n o f f r e q u e n c y . C le a r ly , th e
p o w e r s p e c tr a l d e n s ity in th e p e r io d ic s ig n a l is s im p ly th e s q u a r e o f th e m a g n itu d e
s p e c tr u m . T h e p h a s e in fo r m a tio n is to ta lly d e s tr o y e d ( o r d o e s n o t a p p e a r ) in th e
p o w e r s p e c tr a l d e n s ity .
I f th e p e r io d ic sig n a l is re a l v a lu e d , th e F o u r ie r s e r ie s c o e f f ic ie n ts { c * } sa tis fy
th e c o n d itio n

c -k =
C o n s e q u e n tly . Ki|: = |q|: . H e n c e th e p o w e r s p e c tru m is a s y m m e tr ic fu n c tio n o f
f r e q u e n c y . T h i s c o n d itio n a ls o m e a n s th a t th e m a g n itu d e s p e c tr u m is s y m m e tr ic
( e v e n f u n c t io n ) a b o u t th e o rig in an d th e p h a s e s p e c tru m is a n o d d fu n c tio n .

As

a c o n s e q u e n c e o f th e s y m m e tr y , it is s u ffic ie n t to s p e c ify th e s p e c tr u m o f a re a l
p e r io d ic sig n a l f o r p o s itiv e f r e q u e n c ie s o n ly . F u r th e r m o r e , th e to ta l a v e r a g e p o w e r
c a n b e e x p r e s s e d as

Px

+ 2

1q |

( 4 .1 .1 5 )

( 4 .1 .1 6 )
w h ich fo llo w s d ir e c t ly fro m th e r e la tio n s h ip s g iv e n in S e c t io n 4 .1 .1 a m o n g { aa },
{bn }. a n d ( q ) c o e f fic ie n ts in th e F o u r ie r s e r ie s e x p r e s s io n s .

Example 4.1.1
Determ ine the Fourier series and the power density spectrum of the rectangular pulse
train sienal illustrated in Fie. 4.3.
x(t)

-T

Figure 4 J Continuous-time periodic

tram of rectangular pulses.

238

Chap. 4

Frequency Analysis of Signals and System s

Solution The signal is periodic with fundam ental period Tp and. clearly, satisfies the
Dirichlet conditions. Consequently, we can represent the signal in the Fourier series
given by (4.1.8) with the Fourier coefficients specified by (4.1.9).
Since x(t) is an even signal [i.e.. x(t) = * ( - ;) ] , it is convenient to select the
integration interval from ~ T p/2 to Tp/2. Thus (4.1.9) evaluated for k = 0 yields
(4.1.17)
The term c(I represents the average value (dc com ponent) of the signal .r (r). For k ^ 0
we have
Tp l ~j2TTkFn\ _ jri
A

ej*yf-i,r

tt FkTp

(4.1.18)

)2

A t sin TrkF^r

Tp

_
k = 1 . 2 . ...

n k F\\T

It is interesting to note that the right-hand side of (4.1.18) has the form (sin 4>)/4>,
where <p = n k F ^ . In this case <t>takes on discrete values since F(, and r are fixed and
the index k varies. However, if we plot (sin $)/</> with 0 as a continuous param eter
over the range oc < <t> < oc, we obtain the graph shown in Fig. 4.4. We observe
that this function decays to zero as 0 < x . has a maximum value of unity at <p = 0,
and is zero at multiples of tt (i.e., at <
p= mn. m = 1, 2 ,...) . It is clear that the
Fourier coefficients given by (4.1.18) are the sample values of the (sin <p)/<p function
for <$> xkFut and scaled in amplitude by A t / T p.
Since the periodic function x{t) is even, the Fourier coefficients c* are real.
Consequently, the phase spectrum is either zero, when c* is positive, or t t when c k is
negative. Instead of plotting the m agnitude and phase spectra separately, we may sim
ply plot |c*} on a single graph, showing both the positive and negative values ck on the
graph. This is commonly done in practice when the Fourier coefficients {c*} are real.
Figure 4.5 illustrates the Fourier coefficients of the rectangular pulse train when
Tp is fixed and the pulse width t is allowed to vary. In this case Tp = 0.25 second, so
that F() = \ j T p = 4 Hz and t = 0.057),, t = 0.17},, and r = 0.27),. We observe that
the effect of decreasing r while keeping Tp fixed is to spread out the signal power
over the frequency range. The spacing between adjacent spectral lines is F{) = 4 Hz,
independent of the value of the pulse width r.
sin <p

- I n 6n -57T 4n 3jt 2n n

2n

3n

0
Figure 4.4

The function (sin <p)/<p.

4k

5n

bn

In

<j>

Sec. 4.1

ck

,mTrnit

239

Frequency Analysis of Continuous-Time Signals

J lU jii "

vfffilllll I l k ,

t = 0.1 Tp

u -itrnrm-i.

JJiLLUTT

j ct

F
T

= 0.05 Tr

..................rrnTfniniiin|fnijnnfiiifiii........... ...
F

and the pulse width r varies.

On the other hand, it is also instructive to fix t and vary the period Tp when
Tp > r. Figure 4 .6 illustrates this condition when T,, ~ 5r. Tr = lOr. and Tp = 2 0 t .
In this case, the spacing between adjacent spectral lines decreases as Tp increases. In
the limit as Tr
oc, the Fourier coefficients q approach zero due to the factor of
Tp in the denom inator of (4 .1 .1 8 ). This behavior is consistent with the faci that as
Tp < oc and r remains fixed, the resulting signal is no longer a power signal. Instead,

Cl
...... ...................IMn 1'

..,.((111111 lllllllli,..
0

Tp = 20r
'M illin '"

Figure 4.6 Fourier coefficient of a rectangular pulse train with fixed pulse width
t and varying period Tp .

240

Frequency Analysis of Signals and Systems

Chap.

it becomes an energy signal and its average power is zero. The spectra of finite energy
signals are described in the next section.
We also note that if k / 0 and sm(7ikFnx) = 0. then ct = 0. The harmonics
with zero power occur at frequencies kF0 such that n{kFG)r = m n , m = 1, 2 ,.
o r at JtF0 = m jx. For example, if F() = 4 Hz and t = Q.2TP, it follows that the spectral
components at 20 Hz, 40 H z , . .. have zero power. These frequencies correspond
to the Fourier coefficients
k = 5, 10, 15........O n the other hand, if r = 0.1Tp,
the spectral components with zero power are k = 10, 20, 3 0 ........
The power density spectrum for the rectangular pulse train is

4.1.3 The Fourier Transform for Continuous-Time

Aperiodic Signals
In Section 4.1.1 w e d evelop ed the Fourier series to represent a periodic signal
as a linear com bination o f harm onically related com p lex exp on en tials. A s a con
seq u en ce o f the periodicity, w e saw that these signals possess line spectra with
equidistant lines. T h e line spacing is equal to the fundam ental frequency, which
in turn is the inverse o f the fundam ental period of the signal. W e can view the
fundam ental period as providing the num ber o f lin es per unit o f frequency (line
d en sity), as illustrated in Fig. 4.6.
W ith this interpretation in m ind, it is apparent that if w e allow the period to
increase w ithout limit, the line spacing tends toward zero. In the limit, when the
period b eco m es infinite, the signal b eco m es aperiodic and its spectrum becom es
continuous. This argum ent suggests that the spectrum of an ap eriod ic signal will
b e the en v elo p e o f the line spectrum in the corresponding p eriod ic signal obtained
by repeating the aperiodic signal with som e period Tp.
L et us consider an aperiodic signal x ( t ) with finite duration as show n in
Fig. 4.7a. From this aperiodic signal, w e can create a periodic signal * , , ( 0 with p e
riod Tp, as shown in Fig. 4.7b. Clearly, x r (t) = x ( t ) in the limit as Tp
oo, that is,
x{t) =

lim x p(t)

This interpretation im plies that w e should be able to obtain the spectrum of * (/)
from the spectrum o f x p(i) sim ply by taking the limit as Tp -* oo.
W e begin with the Fourier series representation o f x p(t).
xp{t) =

ckej l ^ ' 1

F0 = y

(4.1.20)

where

(4.1.21)

Sec. 4.1

241

Frequency Analysis o1 Continuous-Time Signals

,r(n

- T

-TJ2

TJ2

TJ2

Figure 4.7

(a) Aperiodic signal ,v(/)

and (b) periodic signal xr U) constructed

bv repeating x(t) with a period Tr .

(h)

S in c e x p (t) x (/ ) f o r ~ T r f 2 < t < Tp/ 2 , ( 4 .1 .2 1 ) c a n b e e x p r e s s e d as

1 'Tr /2
= /
x ( t ) e ~ )27' kF'', d t
Tp J-Trfl

( 4 .1 .2 2 )

It is a ls o tr u e th a t .* (r ) = 0 fo r |r| > Tpj2 . C o n s e q u e n tly , th e lim its o n th e in te g r a l

in ( 4 .1 .2 2 ) c a n b e r e p la c e d b y o c an d o c . H e n c e

ct =

x { t ) e ~ j2 * kF" d t

( 4 .1 .2 3 )

Tp J-oc

L e t us n o w d e fin e a fu n c tio n X ( F ) , c a lle d th e F o u r ie r tra n sfo rm o f * { / ) , as

X (F) =

x (t)e ~ lln F , dt

( 4 .1 .2 4 )

J-oc
A' ( F )

F q.

is a fu n c t io n o f th e c o n tin u o u s v a r ia b le F .

I t d o e s n o t d e p e n d o n Tp o r

H o w e v e r , i f w e c o m p a r e ( 4 .1 .2 3 ) a n d ( 4 .1 .2 4 ) , it is c l e a r th a t th e F o u r ie r

c o e f f ic ie n t s ct c a n b e e x p r e s s e d in te r m s o f X ( F ) as

c* = ^ X ( k F o )
1p
o r e q u iv a le n tly .

Tpc k = X ( k F 0) = X

( 4 A .2 5 )

T h u s th e F o u r i e r c o e ff ic ie n ts a r e s a m p le s o f X ( F ) ta k e n a t m u ltip le s o f f o a n d
s c a le d b y F 0 (m u ltip lie d b y \ / T p).

S u b s titu tio n f o r c t f r o m ( 4 .1 .2 5 ) in to ( 4 .1 .2 0 )

y ie ld s

V O

= ^r 'jh x ( ^ r ) er M

(4 .1 .2 6 )

242

Frequency Analysis of Signals and Systems

W e w ish to ta k e th e lim it o f ( 4 .1 .2 6 ) a s T r a p p r o a c h e s in fin ity .

Chap, 4

F i r s t , w e d e fin e

A F = 1/7},. W ith th is s u b s t itu tio n , ( 4 .1 .2 6 ) b e c o m e s

xpU ) =

Y2

X (k & F )e

JlxkAFi

( 4 .1 .2 7 )

k=-x
I t is c l e a r th a t in th e lim it as Tp a p p r o a c h e s in fin ity , x p {t) r e d u c e s to jc(/ ). A ls o , A F
b e c o m e s th e d iff e r e n tia l d F a n d k A F b e c o m e s th e c o n tin u o u s f r e q u e n c y v a ria b le

F.

In tu r n , th e s u m m a tio n in ( 4 .1 .2 7 ) b e c o m e s a n in te g r a l o v e r th e fr e q u e n c y

v a r ia b le F . T h u s

( 4 .1 .2 8 )

T h is in te g r a l r e la tio n s h ip y ie ld s x ( t ) w h e n X ( F )

is k n o w n , a n d it is c a lle d th e

in erse F o u r ie r tra n sfo rm .

T h is c o n c lu d e s o u r h e u r is tic d e r iv a tio n o f th e F o u r i e r tr a n s f o r m p a ir g iv e n
b y ( 4 .1 .2 4 ) an d ( 4 .1 .2 8 ) f o r an a p e r io d ic s ig n a l x ( t ).

A lth o u g h th e d e r iv a tio n is

n o t m a th e m a tic a lly rig o r o u s , it le d to th e d e s ir e d F o u r i e r t r a n s f o r m re la tio n s h ip s

w ith r e la tiv e ly s im p le in tu itiv e a r g u m e n ts . In s u m m a r y , th e f r e q u e n c y a n a ly s is o f
c o n tin u o u s -tim e a p e r io d ic s ig n a ls in v o lv e s th e fo llo w in g F o u r i e r tr a n s f o r m p a ir.
FR EQ U EN C Y ANALYSIS O F C O N T IN U O U S -T IM E A P ER IO DIC SIGN ALS

Synthesis equation
inverse transform

(4 .1 .2 9 )

Analysis equation
direct transform

( 4 .1 .3 0 )

I t is a p p a r e n t th a t th e e s s e n tia l d if f e r e n c e b e tw e e n th e F o u r i e r s e r ie s a n d th e
F o u r i e r tr a n s fo r m is th a t th e s p e c tr u m in th e l a t t e r c a s e is c o n tin u o u s a n d h e n c e
th e s y n th e s is o f an a p e r io d ic s ig n a l fr o m its s p e c tr u m is a c c o m p lis h e d b y m e a n s o f
in te g r a tio n in s te a d o f s u m m a tio n .
F in a lly , w e w ish to in d ic a te th a t th e F o u r ie r t r a n s f o r m p a ir in ( 4 .1 .2 9 ) and
( 4 .1 .3 0 ) c a n b e e x p r e s s e d in te r m s o f th e r a d ia n fr e q u e n c y v a r ia b le Q =

2nF.

S in c e d F = d S l / l n . ( 4 .1 .2 9 ) a n d ( 4 .1 .3 0 ) b e c o m e
( 4 .1 .3 1 )

( 4 .1 .3 2 )
T h e s e t o f c o n d itio n s th a t g u a r a n te e th e e x is t e n c e o f th e F o u r i e r tr a n s f o r m is th e

Sec. 4.1

Frequency Analysis of Continuous-Time Signals

243

D ir ic h le t c o n d it io n s , w h ich m a y b e e x p r e s s e d as:
1 . T h e s ig n a l v (r) h as a fin ite n u m b e r o f fin ite d is c o n tin u itie s .
2 . T h e s ig n a l x ( t ) h a s a fin ite n u m b e r o f m a x im a a n d m in im a .
3 . T h e s ig n a l x ( t ) is a b s o lu te ly in te g r a b le , th a t is.

\x (t )\d t < o c

i:

( 4 .1 .3 3 )

T h e th ir d c o n d itio n fo llo w s e a s ily fr o m th e d e fin itio n o f th e F o u r i e r tr a n s f o r m ,

g iv e n in ( 4 .1 .3 0 ) . In d e e d .

\ X(F)\

jj

x ( t ) e - J2 F ,dt

< j

\x ( t )\d t

H e n c e ] X ( F ) ! < o c if ( 4 .1 .3 3 ) is s a tis fie d .

A w e a k e r c o n d itio n f o r th e e x is te n c e o f th e F o u r i e r t r a n s f o r m is th a t x {t)
h a s fin ite e n e r e v ; th a t is.
|.v(/)|^r < o c

( 4 .1 .3 4 )

N o te th a t if a s ig n a l x ( i) is a b s o lu te ly in te g r a b le . it w ill a ls o h a v e fin ite e n e rg y .

T h a t is. if

|.v(/)!^r < o c

J CM
th e n

rx
|.v(/)i^/ < o c

H o w e v e r , th e c o n v e r s e is n o t tr u e .

( 4 .1 .3 5 )

T h a t is. a s ig n a l m a y h a v e fin ite e n e r g y b u t

m a y n o t b e a b s o lu te ly in te g r a b le . F o r e x a m p le , th e s ig n a l
sin 2 n t
x {t) = -
7

( 4 .1 .3 6 )

Tt

is s q u a r e in te g r a b le b u t is n o t a b s o lu te ly in te g r a b le .

T h is s ig n a l h a s th e F o u r i e r

tr a n s fo r m

f1

* ( F ) = { o ;

\FI < 1

(4 -1 .3 7 )

S in c e th is s ig n a l v io la te s ( 4 .1 .3 3 ) , it is a p p a r e n t th a t th e D i r i c h l e t c o n d itio n s a re
s u ffic ie n t b u t n o t n e c e s s a r y f o r th e e x is te n c e o f th e F o u r i e r tr a n s f o r m . In a n y c a s e ,
n e a r ly all fin ite e n e r g y s ig n a ls h a v e a F o u r ie r tr a n s f o r m , s o t h a t w e n e e d n o t w o rry
a b o u t th e p a th o lo g ic a l s ig n a ls , w h ich a r e s e ld o m e n c o u n t e r e d in p r a c tic e .

4.1.4 Energy Density Spectrum of Aperiodic Signals

L e t x (t ) b e a n y fin ite e n e r g y s ig n a l w ith F o u r i e r t r a n s fo r m X ( F ) . Its e n e r g y is

244

Frequency Analysis of Signals and Systems

Chap. 4

w hich, in turn, may be expressed in term s o f X ( F ) as follows:

Ex -

x ( t ) x '( i) d t

J OC

1oo

X '( F ) d F \

r /*oc
/
x { t ) e ~ j2 n F 'd t

'OC

\ X ( F ) \ 2d F

T h erefore, w e con clu d e that

(4.1.38)
This is P a r s e v a ls re la tio n for aperiodic, finite energy signals and expresses the
principle o f conservation of energy in the tim e and frequency dom ains.
T he spectrum X ( F ) o f a signal is in general, com p lex valued. C onsequently,
it is usually expressed in polar forms as
X ( F ) = | X ( F ) | ^ W(f)

where |X ( F ) | is the m agnitude spectrum and (F ) is the phase spectrum .

(F ) - i U ( F )
O n the other hand, the quantity
SXX( F ) = \X ( F ) |2

(4.1.39)

which is the integrand in (4.1.38), represents the distribution of en ergy in the signal
as a function o f frequency. H en ce SXX( F ) is called the e n e rg y d e n sit y s p e ctru m of
x ( t ). T he integral o f S XX( F ) over all freq u en cies gives the total en ergy in the signal.
V iew ed in another way, the energy in the signal x ( t ) over a band o f frequencies
F \ < F < F [ + A F is

From (4.1.39) w e observe that S XX( F ) d o es not con tain any p h ase inform ation
[i.e., SXX( F ) is purely real and n on n egative]. Since the phase spectrum of ;t(r) is
not contained in SXX( F ) , it is im possible to reconstruct the signal given S XX( F ) .
Finally, as in the case of Fourier series, it is easily show n that if the signal
x (t ) is real, then
\X ( -F ) \ =
X { -F )

\X ( F ) \

(4.1.40)

= -* X (F )

(4.1.41)

Sec. 4.1

Frequency Analysis of Continuous-Time Signals

245

By com bining (4.1.40) and (4.1.39), w e obtain

(4.1.42)

SXX( ~ F ) = S xA F )

In other words, the energy density spectrum o f a real signal has even sym m etry.
Exam ple 4.1.2
D eterm ine the Fourier transform and the energy density spectrum of a rectangular
pulse signal defined as

and illustrated in Fig. 4.8(a).

Solution Clearly, this signal is aperiodic and satisfies the Dirichlet conditions. Hence
its Fourier transform exists. By applying (4.1.30), we find that
(4.1.44)
We observe that X( F) is real and hence it can be depicted graphically using only
one diagram, as shown in Fig. 4.8(b). Obviously, X( F) has the shape of the (sin0)/<?
function shown in Fig. 4.4. Hence the spectrum of the rectangular pulse is the en
velope of the line spectrum (Fourier coefficients) of the periodic signal obtained by

~>

2
(a)
X<F)

Ar

F
(b)
Figure 4.8

(a) R ectangular pulse and (b) its F ourier transform .

246

Frequency Analysis of Signals and System s

Chap. 4

periodically repeating the pulse with period Tp as in Fig. 4.3. In other words, the
Fourier coefficients ck in the corresponding periodic signal xp{t) are simply samples
of X( F) at frequencies kF0 = k/ Tp. Specifically,

From (4.1.44) we note that the zero crossings of X( F ) occur at multiples of 1 /r.
Furtherm ore, the width of the main lobe, which contains most of the signal en
ergy, is equal to 2/z. As the pulse duration t decreases (increases), the main
lobe becomes broader (narrow er) and m ore energy is moved to the higher (lower)
frequencies, as illustrated in Fig. 4.9. Thus as the signal pulse is expanded (com
pressed) in time, its transform is compressed (expanded) in frequency. This be
havior, between the time function and its spectrum, is a type of uncertainty
principle that appears in different forms in various branches of science and engi
neering.
Finally, the energy density spectrum of the rectangular pulse is
, /sm 7 rj
S ( F ) = ( A t )1 ( " ~ ' ^ T )
} ( ttF-

(4.1.46)

*(r)

X 0 I

x (r)

X(F)

- L

X(l)

Figure 4.9

F o u rier transform of a re d a n g u la r pulse for various w idth values.

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

247

4.2 FREQUENCY ANALYSIS OF DISCRETE-TIME SIGNALS

In Section 4.1 we d ev elop ed the Fourier series representation for continuous-tim e
periodic (pow er) signals and the Fourier transform for finite energy aperiodic
signals. In this section we repeat the d evelop m en t for the class o f discrete-tim e
signals.
A s we have ob served from the discussion o f Section 4.1, the Fourier series
representation o f a con tinu ou s-tim e periodic signal can consist o f an infinite num
ber o f frequency com p onents, where the frequency spacing b etw een tw o successive
harm onically related freq u en cies is 1 / T p, and w here Tp is the fundam ental period.
Since the frequency range for continuous-tim e signals exten d s from oo to oc, it
is p ossib le to have signals that contain an infinite num ber o f frequency com p o
nents. In contrast, the frequency range for discrete-tim e signals is unique over the
interval ( - r t . T i ) or (0 .2 ^ ). A discrete-tim e signal o f fundam ental period N can
consist o f frequency co m p on en ts separated by 2n / N radians or / = 1/jV cycles.
C onsequently, the Fourier series representation o f the discrete-tim e periodic signal
will contain at m ost N frequency com ponents. This is the basic d ifference b etw een
the Fourier series representations for continuous-tim e and discrete-tim e periodic
signals.

4.2.1 The Fourier Series for Discrete-Time Periodic

Signals
Suppose that w e are given a periodic sequ en ce a (/ i ) with period N , that is, x i n ) =
x(n + N ) for all n. T he Fourier series representation for x(h) consists o f N har
m onically related exp on en tial functions
k = 0 A ........ N 1
and is expressed as
A'-l
X () = Y ^ c kejlKkn/N
*=o

(4.2.1)

where the {<:*) are the coefficients in the series representation.

T o derive the expression for the Fourier coefficients, w e use the follow ing
formula:
= (
I 0,
1

y '1
n=0

* = 0. N , 2 N, . . .
otherw ise

2 2

N o te the sim ilarity o f (4.2.2) with the con tinu ou s-tim e counterpart in (4.1.3). T he
p roof o f (4.2.2) follow s im m ediately from the application of the geom etric sum
m ation form ula
jv-i

a = l

248

Frequency Analysis of Signals and System s

Chap. 4

T h e e x p r e s s io n f o r th e F o u r i e r c o e f f ic ie n ts c* c a n b e o b t a i n e d b y m u ltip ly in g
b o th s id e s o f ( 4 .2 .1 ) b y th e e x p o n e n tia l e ~ j2nin//v a n d s u m m in g th e p r o d u c t fro m
= 0 t o n

= yV 1. T h u s
N 1 A- ]

A'-l

^ 2 x ( n ) e ~ il!rln /N

c ke J2n<k~ l)n/N

n=(J

(4 .2 .4 )

n=0 t=0

I f w e p e r fo r m th e s u m m a tio n o v e r n first, in th e r ig h t-h a n d s id e o f (4 .2 .4 ) ,

w e o b ta in
y ^ (fj2.TU-/)n/A' _

k - I = 0, N , 2 N , . . .
o th e r w is e

| N,
I 0,

w h e r e w e h a v e m a d e u se o f ( 4 .2 .2 ) .

(4 2 5)

T h e r e f o r e , th e rig h t-h a n d s id e o f ( 4 .2 .4 )

r e d u c e s to N c / a n d h e n c e
j A'-l
q = -

J ' x ( n ) e ~ J2*" /tl

1 = 0. 1 .......... N - 1

(4 .2 .6 )

/T = (J

T h u s w e h a v e th e d e s ir e d e x p r e s s io n fo r th e F o u r i e r c o e f f ic ie n t s in te r m s o f th e
s ig n a l x ( ) .
T h e r e la tio n s h ip s ( 4 .2 .1 ) a n d ( 4 .2 .6 ) fo r th e f r e q u e n c y a n a ly s is o f d is c r e te *
tim e s ig n a ls a r e s u m m a r iz e d b e lo w .
FR E Q U E N C Y ANALYSIS O F D IS C R E TE -TIM E P E R IO D IC SIGN ALS

Synthesis equation

iW l

-1 %
II

Analysis equation

A'-l
X(/! ) = ^ Ck(j2jTt"!r'
*=0

(4.2.7)

(4.2.8)

E q u a t i o n ( 4 .2 .7 ) is o ft e n c a lle d th e d isc re te -tim e F o u r ie r s e rie s ( D T P S ) . T h e

F o u r i e r c o e ff ic ie n ts {c* }. k =

0 . 1 .......... N 1 p r o v id e th e d e s c r ip tio n o f jc ( ) in

th e f r e q u e n c y d o m a in , in th e s e n s e th a t c k r e p r e s e n t s th e a m p litu d e a n d p h a s e
a s s o c ia t e d w ith th e fr e q u e n c y c o m p o n e n t

sk(n) = e ^ kn^ ' = e jWtB

w h e r e to* = 2 n k / N .
W e re c a ll f r o m S e c tio n 1 .3 .3 th a t th e f u n c t io n s s k (n ) a r e p e r i o d i c w ith p e r io d

N . H e n c e sk(n) = sk {n + N ) . In v iew o f th is p e r io d ic ity , it fo llo w s t h a t th e F o u r ie r

c o e f f ic ie n ts c k, w h e n v ie w e d b e y o n d th e r a n g e k ~

0 , 1 , ____A ' - l , a ls o s a tis fy a

p e r io d ic ity c o n d itio n . In d e e d , fro m ( 4 .2 .8 ) , w h ich h o ld s f o r e v e r y v a lu e o f k , w e

have

Ck+ N = ^ J ^ x ( n ) e - ^ (k+N)n/N = ~ J 2 x { n ) e - ^ kn/N = c t

n=0

n=0

( 4 .2 .9 )

Sec. 4.2

249

Frequency Analysis of Discrete-Time Signals

T h e r e f o r e , th e F o u r i e r s e r ie s c o e f f ic ie n ts { q } fo r m a p e r io d ic s e q u e n c e w h e n e x
te n d e d o u ts id e o f t h e r a n g e k = 0 , 1 ..........jV 1. H e n c e

Ck+N = <^k
th a t is , { c t } is a p e r io d ic s e q u e n c e w ith f u n d a m e n ta l p e r io d N.

T hu s the s p e ctru m
o f a signa l x(n ), w h ich is per iod ic with p e r i o d N , is a p er io d ic s e qu en ce with p e r io d
N . C o n s e q u e n tly , a n y N c o n s e c u tiv e s a m p le s o f th e s ig n a l o r its s p e c tr u m p r o v id e
a c o m p le t e d e s c r ip tio n o f th e s ig n a l in th e tim e o r f r e q u e n c y d o m a in s .
A lth o u g h th e F o u r i e r c o e f fic ie n ts fo r m a p e r io d ic s e q u e n c e , w e w ill fo c u s o u r
a tte n tio n o n th e s in g le p e r io d w ith ra n g e k = 0 , 1 .......... N 1. T h i s is c o n v e n ie n t,
s in c e in th e f r e q u e n c y d o m a in , th is a m o u n ts to c o v e r in g th e f u n d a m e n ta l ra n g e
0 < a>t =

2 n k /N

< 2 n , fo r 0 < k < N 1.

In c o n t r a s t , th e f r e q u e n c y ra n g e

it < a)* = 2 7 i k / N

< j t , c o r r e s p o n d s to N / 2 < k < N / 2 , w h ic h c r e a t e s an

in c o n v e n ie n c e w h e n N is o d d . C le a r ly , i f w e u se a s a m p lin g f r e q u e n c y F s , th e
r a n g e 0 < k < N 1 c o r r e s p o n d s to th e f r e q u e n c y ra n g e 0 < F < F , .

Example 4.2.1
D eterm ine the spectra of the signals
(a) jr() = cos -Ji nn
(b) x(n) = cos nn/ 3
(c) x( n ) is periodic with period N = 4 and
x(n) = {1, 1.0.0}
T

Solution
(a) For m = J i n , we have / = 1j - Jl . Since f , is not a rational number, the signal
is not periodic. Consequently, this signal cannot be expanded in a Fourier series.
N evertheless, the signal does possess a spectrum. Its spectral content consists
of the single frequency component at id = wo = -Jin.
(b) In this case / (l = | and hence x{n) is periodic with fundam ental period N = 6.
From (4.2.8) we have
5
It = 0 . 1 ........5
However, x(n) can be expressed as
x(n) = cos
" r =
+ \ e~i2*nib
6
*
which is already in the form of the exponential Fourier series in (4.2.7). In
comparing the two exponential terms in x{n) with (4.2.7), it is apparent that
ci = j. The second exponential in x(n) corresponds to the term Jt = 1 in
(4.2.7), However, this term can also be written as
- j 2 j r n / 6 _

^ j2jr (5 n> /6

which means that c_i = c$. But this is consistent with (4.2.9), and our previous
observation that the Fourier series coefficients form a periodic sequence of

250

Chap. 4

Frequency Analysis of Signals and Systems

period N . Consequently, we conclude that

= C\ = f 4 = 0

C(, =
c, = ^

(c) From (4.2.8). we have

1

A-= 0 . 1 ,2 ,3

Ck

or
Ct = l ( l + e - i ^ )

A-= 0 . 1.2.3

For k = 0, 1, 2, 3 we obtain
f'l j ( l j )

C'd = S

f'2 = 0

Cl = j { l + j }

The m agnitude and phase spectra are

K'd I

ki

4-Co = 0

4 f| = --

2tr- = undefined

4_o =

4
4
Figure 4.10 illustrates the spectral content of the signals in (b) and (c).

4.2.2 Power Density Spectrum of Periodic Signals

T h e a v e r a g e p o w e r o f a d is c r e te - tim e p e r io d ic sig n a l w ith p e r io d N w as d efin ed
in ( 2 .1 .2 3 ) as

P, = -

M )|

( 4 .2 .1 0 )

W e s h a ll n ow d e riv e a n e x p r e s s io n fo r P x in te r m s o f th e F o u r i e r c o e f f ic ie n t {c<J.
I f w e u se th e r e la tio n ( 4 .2 .7 ) in ( 4 .2 .1 0 ) , w e h a v e

j v -i
Pi =

x(rt)x*(n)

n = ()

A'-l

N
N o w . w e c a n in te r c h a n g e t h e o r d e r o f th e tw o s u m m a tio n s a n d m a k e u s e o f ( 4 .2 .8 ) ,
o b ta in in g

p * = Y , ct
*=<j
IV1
A'-l

, A-i
- y x ( n ) e - j2kn/N
-i A I

( 4 . 2 . 11 )

Sec. 4.2

Frequency Analysis of Discrete-Tim e Signals

251

(a)

Zc t
Jr

"4
-3

1
- 2

- 1

... t

2 3 4

71
4
Figure 4.10 Spectra of the periodic
signals discussed in Example 4.2.1 (b)
and (c).

(c)

which is the desired exp ression for the average p ow er in the p eriod ic signal. In
other w ords, the average pow er in the signal is th e sum o f th e pow ers o f the
individual frequency com p onents. W e view (4.2.11) as a P arsevals relation for
d iscrete-tim e period ic signals. T h e seq u en ce | a |: for k = 0, 1 , N - 1 is the
distribution o f p ow er as a function o f freq u en cy and is called the p o w e r density
spectrum o f the periodic signal.
If w e are interested in the energy o f th e se q u en ce Jt(n) o v er a single period,
(4.2.11) im plies that
/VI

N -l

n=0

k()

(4.2.12)
which is consistent with our p reviou s results for con tin u ou s-tim e p eriodic signals.
If the signal x ( n ) is real [i.e., x (n) = jr(n)], then, p roceed in g as in S ec
tion 4.2.1, w e can easily sh ow that
ct = c . k

(4.2.13)

252

Frequency Analysis of Signals and Systems

Chap. 4

or equivalently,
|c_*| =

|c * f

- 4 c_t =

(even sym m etry)

(4.2.14)

(odd sym m etry)

(4.2.15)

T h ese sym m etry properties for the m agnitude and phase spectra o f a periodic sig
nal, in conjunction with the periodicity property, have very im portant im plications
on the frequency range o f discrete-tim e signals.
In d eed , by com bining (4.2.9) with (4.2.14) and (4.2.15), w e obtain
Iq

(4.2.16)

I =

and
4-c*

(4.2.17)

M ore specifically, w e have

\c N p \

k jV /2 |,

A -c N[2

k ( A f + l)/2 l<

4-C (N -1)/2

II

|c il

4 .c 0 =

4 - c' l

- % - c' n

% -C N -l

(4.2.18)

k (/V -l)/2

if N is even

~ 4 - C ( /V + l)/2

if N is odd

Thus, for a real signal, the spectrum c*, k = 0, 1 ,. . . , N [ 2 for N even, or

k = 0. 1........ ( N l ) / 2 for N odd, com p letely specifies the signal in the frequency
dom ain. Clearly, this is consistent with the fact that the highest relative frequency
that can be represented by a discrete-tim e signal is eq u al to n . In d eed , if 0 < a>k =
2 n k / N < jt, then 0 < k < N f 2.
By m aking use o f these sym m etry p roperties o f the Fourier series coefficients
o f a real signal, the F ourier series in (4.2.7) can also be exp ressed in the alternative
forms
x ( n ) = co + 2

= ao + ^

lc*l cos

( ^ k cos kn - b k sin kn 'j

(4.2.19)

(4.2.20)

w here a0 = c0. <*k = 2 [ct|co s0 * . bk = 2|c*|sin 0*, and L = N [2 if N is even and

L ( N ~ l ) / 2 if N is odd.
F inally, we n o te that as in the case o f continuous-tim e signals, the power
density spectrum |ct |2 d oes n ot contain any phase inform ation. Furtherm ore, the
spectrum is discrete and periodic with a fundam ental period eq u al to that o f the
signal itself.

Example 4.2.2 Periodic Square-Wave Signal

Determ ine the Fourier series coefficients and the power density spectrum of the
periodic signal shown in Fig. 4.11.

Sec. 4.2

253

Frequ ency A nalysis of Discrete-Time Signals

Figure 4.11 Discrete-time periodic

square-wave signal.

A'

Solution By applying the analysis equation (4.2.8) to the signal shown in Fig. 4.11.
we obtain
=

1 L~l
= -L Y Ae- P^n/
AJ i/

1
A j

k = 0. 1.

N - 1

which is a geom etric sum mation. Now we can use (4.2.3) to simplify the summation
above. T hus we obtain
AL
TT
A 1 - e- j2*kL/s
I N 1'

k= 0
k = 1.2.

N - 1

The last expression can be simplified further if we note that

1

e -j2xkL;S

1 - <-

-jzk L tN

I.ikl/K

e -ink;'\

( rrl. \ _ ,-jxkl.\

-jxHt.-u.'K sin(xkL. /N )
s in (7 r /A /

T herefore.
AL

k = 0. +A'. 2 N ,

, s i ni n kL/ N)
sin(7zk/N)

I A'

(4.2.21)
otherwise

The pow er density spectrum of this periodic signal is

/ AL_

k*l" =

k = 0, +N. 2 A'.

V N
A V / sinnkL/N
Ar / V s i n n k / N

(4.2.22)
otherwise

Figure 4.12 illustrates the plots of jct |2 for L = 5 and 7, A' = 40 and 60. and A = 1.

4.2.3 The Fourier Transform of Discrete-Time Aperiodic

Signals
Just as in the case o f con tinu ou s-tim e aperiodic energy signals, the frequency anal
ysis o f d iscrete-tim e aperiodic finite-energy signals in volves a Fourier transform of
the tim e-d om ain signal. C on sequ en tly, th e d evelop m en t in this section parallels
to a large ex ten t, that given in Section 4.1.3.

254

Frequency Analysis of Signals and Systems

Chap. 4

L = 5, N = 40

I 1II I 1. . I I I I

_xlJ
-2 0

-1 0

10

20

L = 7, N = 60

LJ i l
-3 0

- 20

-1 0

10

20

30

= 5. N =

-3 0

- 20

10

10

20

30

Figure 4.12 Plot of the pow er density

spectrum given by (4.2.22).

T he Fourier transform o f a finite-energy discrete-tim e signal x ( n ) is defined as

OC

X (co) =

x ( n ) e ~ Jwr

(4.2.23)

FI OC

Physically, X(co) represents the frequency con ten t o f the signal x ( n ) . In other
words. X{(o) is a d ecom p osition o f x ( n ) into its frequency com p onents.
W e observe tw o basic d ifferen ces b etw een the Fourier transform of a discrete
tim e finite-energy signal and the F ourier transform o f a finite-energy analog signal.
First, for continuous-tim e signals, the Fourier transform , and h en ce the spectrum
o f the signal, have a frequency range o f {0 0 , 0 0 ). In contrast, the frequency
range for a discrete-tim e signal is unique over the frequency interval o f (n , n)
or, equivalently, (0. 2 tt). T h is property is reflected in the Fourier transform o f the

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

255

signal. In d eed . X(co) is period ic with period 2 n . that is.

X(a> + 2 7 T k )=

22

j{u>^-27ik)u

x { n ) e 'i{w^ 7' k)n

cc
oc

J 2 x ( n ) e - ' ame - JZ7,kn

(4.2.24)

/l = OC
OC

2 2

x ( n ) e - , Mn

= X(w)

H en ce X ( t o ) is p eriod ic with period 2 t t . B u t this property is just a con seq u en ce of

the fact that the frequency range for any discrete-tim e signal is lim ited to (n , n )
or (0 , 27r). and any frequency outside this interval is eq u ivalen t to a frequency
within the interval.
T h e secon d basic d ifference is also a con seq u en ce o f the discrete-tim e nature
o f the signal. Since the signal is discrete in tim e, the Fourier transform o f the
signal in v o lv es a sum m ation o f term s instead o f an integral, as in the case of
continuous-tim e signals.
Since X (w ) is a periodic function o f the frequency variable a), it has a Fourier
series exp an sion , provided that the con d ition s for the existen ce o f the Fourier
series, described previously, are satisfied. In fact, from th e definition o f the
Fourier transform X (co) o f the sequ en ce x(n), given by (4.2.23), w e ob serve that
X (a >) has the form o f a Fourier series. T he Fourier coefficients in this series
expansion are the valu es o f the seq u en ce x ( n ) .
T o dem onstrate this point, let us evaluate the seq u en ce x ( n ) from X(co). First,
we m ultiply both sides (4.2.23) by ej<um and integrate over the interval ( tt, t t ) .
Thus w e have
(4.2.25)
The integral on the right-hand side o f (4.2.25) can be evaluated if w e can inter
change the order o f sum m ation and integration. This interchange can be m ade if
the series
N

X N (a>) = 2 2 x ( n ) e - llon
n=-N

con verges uniform ly to X(o)) as A1 -> oc. U niform con vergen ce m ean s that, for
every w, Xh((d) * X(ca), as /V -* oo. T he con vergen ce o f the Fourier transform
is discussed in m ore detail in the follow in g section. For the m om en t, let us as
sum e that the series con verges uniform ly, so that w e can interchange the order of
sum m ation and integration in (4.2.25). Then

256

Frequency Analysis of Signals and Systems

Chap.

C onsequently,

x{n)

rj = OC

2:Tx(m).
0.

n'da) =

fJ 71

(4.2.26)

By com bining (4.2.25) and (4.2.26). w e obtain the desired result that
1

x (n )

fn

.7T J jt

X ( o j) e J

do)

(4.2.27)

If w e com pare the integral in (4.2.27) with (4,1.9), we n o te that this is just
the expression for the Fourier series coefficient for a function that is periodic with
period 2 n . The only difference b etw een (4.1.9) and (4.2.27) is the sign on the
exp on en t in the integrand, which is a con sequ en ce o f our definition o f the Fourier
transform as given by (4.2.23). T h erefore, the Fourier transform of the sequence
x ( n ) , defined by (4.2.23), has the form o f a Fourier series expansion.
In summary, the Fourier tra nsform p a ir f o r discrete-time signals is as follows.
FR EQ U EN C Y ANALYSIS OF DtS C R E TE -TIM E AP ER IO D IC S IG N A LS

Synthesis equation
inverse transform

x(

Analysis equation
direct transform

X {io )=

(4 .2 .2 8 )

X < w ) i lu ' ' ' d w

OC
2 2

(4.2.29)

x ( n ) e ~ , w '1

4.2.4 Convergence of the Fourier Transform

In the derivation o f the inverse transform given by (4.2.28), w e assum ed that the
series

Xjv(cw) =

22 xWe~

(4.2.30)

converges uniform ly to X(a>), given in the integral o f (4.2.28), as N -* oc. By

uniform convergence w e m ean that for each
lim { s u p X (oj) ~ X s (>>)'} = 0
(4.2.31)
N> * <*>
U niform convergence is guaranteed if ;c(/j) is ab solu tely sum m able. Indeed, if
DC
2 2 |j c )i < oc
(4.2.32)
n=-oc
then

|X M l =

22

x (n )e '

<

2 2

<

00

H en ce (4.2.32) is a sufficient condition for the existen ce o f the d iscrete-tim e Fourier

transform. W e n o te that this is the discrete-tim e counterpart of th e third Dirich-

Sec. 4.2

257

Frequency Analysis of Discrete-Time Signals

let condition for the Fourier transform o f con tinu ou s-tim e signals. T he first two
conditions d o n ot apply due to the discrete-tim e nature o f [*()}.
S om e se q u en ces are not absolutely sum m able, but they are square sum m able.
That is, they have finite energy
OC

Ex =

|jc(n ) |2 < oo

(4.2.33)

n = DC

which is a w eak er condition than (4.2.32). W e w ould like to d efine the Fourier
transform o f finite-energy sequ en ces, but w e m ust relax the con d ition o f uniform
con vergen ce. For such seq u en ces w e can im pose a m ean-square con vergen ce co n
dition:
lim f |X M - X N ((ti)\2dto = 0
Af<* J-71

(4.2.34)

Thus the energy in the error X(a>) - X/v(oj) tends toward zero, but the error
|X (w ) - Xw(a))j d o es not necessarily tend to zero. In this way w e can include
finite-energy signals in the class o f signals for which the Fourier transform exists.
Let us con sid er an exam p le from the class o f finite-energy signals. Suppose
that
X M = ( 1
[ U.

H ^

|< 7 r

(4-2.35)

< ja>| < 7T

T he reader should rem em ber that X(a>) is periodic with p eriod 2 n . H en ce (4.2.35)
represents only on e period o f X(co). The inverse transform o f X { cj) results in the
sequence
i
jr() =

-~

In

r
I

X { w ) e ja>nda)

eJ

sin<u,-n

dm = ------------

n^O

Jin

For n = 0, w e have
*( 0 )
H en ce

x (n ) =

n
(j)c sin a)cn
n

cocn

n = 0
(4.2.36)
n

This transform pair is illustrated in Fig. 4.13.

S om etim es, the seq u en ce {*()) in (4.2.36) is exp ressed as
sin cocn
x ( n ) = ----------

oo < n < oo

(4.2.37)

258

Frequency Analysis of Signals and Systems

Chap. 4

X(w)

1
-JT

UJt

(i),

7T

(b)
Figure 4.13

Fourier transform pair in (4.2.35) and (4.2.36j.

with the understanding that at n = 0, x ( n ) = w j n . W e should em p hasize, however,

that (sina)(7i)/jrfi is not a con tinu ou s function, and hence L H osp ital's rule cannot
be used to determ ine jr(0 ).
N ow iet us consider the determ ination o f the Fourier transform o f the se
q uence given by (4.2.37), T he seq u en ce {jr(n)j is not absolutely sum m able. H ence
the infinite series
(4.2.38)

does not converge uniform ly for all w. H ow ever, the seq u en ce {jc()} has a finite
energy E x = o)c/ tc as will be show n in Section 4.3. H en ce the sum in (4.2.38) is
guaranteed to converge to the X (a>) given by (4.2.35) in the m ean-square sense.
T o elaborate on this point, let us con sid er the finite sum

X N (a>)=

(4.2.39)

Figure 4.14 show s the function X N{w) for several values of N . W e n ote that there
is a significant oscillatory oversh oot at co = coc, in d ep en dent o f the value o f N . As

Sec. 4.2

259

Frequency Analysis of Discrele-Ttme Signals

x , s(w)

^50<)

A"7o(ul)

Figure 4.14 Illustration of convergence of the F ourier transform and the G ibbs
phenom enon at the point of discontinuity

N increases, the oscillation s b ecom e m ore rapid, but the size o f the ripple rem ains
the sam e. O ne can sh ow that as N -* oo, the oscillations con verge to the point
o f the discontinuity at to a>t . but their am plitude d oes not go to zero. H ow ever,
(4.2.34) is satisfied, and therefore
converges to X ( w ) in the m ean-square
sense.
T he oscillatory behavior o f the approxim ation X^ic o) to the function X(a>) at
a point o f discontinuity o f
is called the G ib b s p h e n o m e n o n . A sim ilar effect
is ob served in the truncation o f the F ourier series o f a con tinu ou s-tim e periodic
signal, given by the syn th esis eq u ation (4.1.8). For exam ple, the truncation o f the
Fourier series for the period ic square-w ave signal in E xam ple 4.1.1, gives rise to
the sam e oscillatory behavior in the finite-sum approxim ation o f x ( t ) . T he G ibbs
ph en om en on will be en cou n tered again in the design o f practical, discrete-tim e
FIR system s considered in C hapter 8 .

260

Frequency Analysis of Signals and Systems

Chap. 4

4.2.5 Energy Density Spectrum of Aperiodic Signals

R ecall that the energy o f a discrete-tim e signal x ( n ) is defined as
OC

Ex = 2 2 l*(n )l2
n=oc

(4.2.40)

Let us n o w e x p r e s s th e e n e r g y E x in te r m s o f th e s p e c tr a l c h a r a c t e r i s t i c X (w). F irst

we have

Ex =

00
22 x(n)x*(n)=
n ------- -v

I" J rJT
2 2 x(n)
X*
n
-v _
J JT

If we interchange the order o f integration and sum m ation in the equation above,
we obtain
dw

|X (co)\~du>
T h erefore, the energy relation b etw een x ( n ) and X(a>) is
E,=

oc
\
2 2 \x^)\2 =
n=-oc

fn
\X (c o )l2dco

(4.2.41)

This is Parseval's relation for discrete-tim e aperiodic signals with finite energy.
T he spectrum X(a>) is, in general, a com p lex-valu ed function of frequency.
It may be expressed as
X(u>) = \X ( c u ) \e J* M

(4.2.42)

where
Q(co) - ^ X ( c o )
is the phase spectrum and \X (a > )\ is the m agnitude spectrum .
A s in the case o f continuous-tim e signals, the quantity
S(o>) = | X M |2

(4.2.43)

represents the distribution of energy as a function o f frequency, and it is called

the ener gy density sp ectrum o f x ( n ) . Cleariy, Sxx(a>) d oes not contain any phase
inform ation.
Suppose now that the signal x ( n ) is real. T hen it easily follow s that
X*(o>) = X ( - oj)

(4.2.44)

or equivalently,
|X (ti>)| = |X(tt>)|

(even symmetry)

(4.2.45)

Sec. 4.2

261

Frequency Analysis of Discrete-Time Signals

(4.2.46)

(4.2.47)
From these sym m etry properties we con clu d e that the frequency range of
real d iscrete-tim e sign als can be lim ited further to the range 0 < co < n (i.e.,
on e-half o f the p erio d ). Indeed, if we know X ( a >) in the range 0 < w < n , we
can d eterm in e it for the range - n < co < 0 using the sym m etry p roperties given
above. A s w e have already ob served , sim ilar results hold for discrete-tim e periodic
signals. T h erefo re, the frequency-dom ain description o f a real discrete-tim e signal
is co m p letely sp ecified by its spectrum in the frequency range 0 < co < n .
U su a lly , w e w ork with the fundam ental interval 0 < a > < 7r o r 0 < F < F J 2,
exp ressed in H ertz. W e sketch m ore than half a period only w hen required by the
specific application.
Example 4.2.3
D eterm ine and sketch the energy density spectrum 5,,r (w) of the signal
= a"u(/i)

1 < a < 1

Solution Since \a\ < 1. the sequence x(n) is absolutely summable, as can be verified
by applying the geometric summation formula.

H ence the Fourier transform of x(n) exists and is obtained by applying (4.2.29). Thus

Since \ae~J,Jl = |a| < 1. use of the geom etric summation formula again yields

The energy density spectrum is given by

Sl x(w) = |X(w)|2 = X(a>)X'(w) =

(1 ae~Ju,)( 1 aeJW)

or, equivalently, as
1
1 2a cos a>+ a
Note that S (-o> ) = Ss i (w) in accordance with (4.2.47).
Figure 4.15 shows the signal *() and its corresponding spectrum for a = 0.5
and a = -0 .5 . Note that for a = -0 .5 the signal has more rapid variations and as a
result its spectrum has stronger high frequencies.

Frequency Analysis of Signals and Systems

262

Chap. 4

x(n) - (-0.5)"u(!

Figure 4.15
spectra.

(a) Sequence .\in) =

and mii I = i

Figure 4.16
pulse.

(b) ih eir energy density

D iscrete-tim e rectangular

Example 4.2.4
Determ ine the Fourier transform and the energy density spectrum of the sequence
xin) =

0 < n <L otherwise

(4.2.48)

which is illustrated in Fig. 4.16.

Solution

Before computing the Fourier transform, we observe that

Hence xin) is absolutely summable and its Fourier transform exists. Furthermore,
we note that Jt(n) is a finite-energy signal with
= \A\: L,
The Fourier transform of this signal is

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

263

1 - e~uL

= Ae- ' ,u/2'a - 11

s m( w / 2 )

(4.2.49)

For w = 0 the transform in (4.2.49) yields X(0) = AL, which is easily established
by setting w = 0 in the defining equation for X(a>), or by using L H ospitals rule in
(4.2.49) to resolve the indeterm inate form when w = 0.
The m agnitude and phase spectra of ;t(n) are

f \A\L,
|X<)I =

w= 0

sin(L /2)
sin(aj/2)

otherwise

(4-2.50)

and
w
2

$X (a>) = $ A - - ( L - l ) + $

sin(ctiZ./2)
'
sin(w/2)

(4.2.51)

where we should rem em ber that the phase of a real quantity is zero if the quantity is
positive and n if it is negative.
The spectra |X(o>)| and ^.X(w) are shown in Fig. 4.17 for the case A 1 and
L = 5. The energy density spectrum is simply the square of the expression given in
(4.2.50).
T here is an interesting relationship that exists b etw een the F ourier transform
o f the constant am plitude pulse in E xam p le 4.2.4 and the period ic rectangular
lX(w)l

Figure 4.17 Magnitude and phase of

Fourier transform of the discrete-time
rectangular pulse in Fig. 4.16.

264

Frequency Analysis of Signals and Systems

Chap. 4

w ave considered in E xam ple 4.2.2. If w e evaluate the F ourier transform as given
in (4.2.49) at a set o f equally spaced (harm onically related) freq u en cies

w e obtain
(4.2.52)
If w e com pare this result with the expression for the F ourier series coefficients
given in (4.2.21) for the period ic rectangular w ave, w e find that
k = 0. 1 , . . . , N 1

(4.2.53)

T o elab orate, w e have established that the Fourier transform o f the rectangular
p ulse, which is identical with a single p eriod o f the period ic rectangular pulse
train, evaluated at the freq u en cies co = I n k / N , k = 0, 1 , ___ N 1, which are
identical to the harm onically related frequency com p onents u sed in the Fourier
series representation o f the periodic signal, is sim ply a m ultiple o f the Fourier
coefficients f a ) at the corresponding frequencies.
T he relationship given in (4.2.53) for the F ourier transform o f the rectangular
pulse evaluated at co 2 n k / N , k = 0. 1........ A ' - l , and the F ourier coefficients
o f the corresponding periodic signal, is not only true for these tw o signals but, in
fact, holds in general. This relationship is d evelop ed further in Chapter 5.

4.2.6 Relationship of the Fourier Transform to the

z-Transform
T h e z-transform o f a seq u en ce * () is defined as
OC

RO C: r2 < \z\ < r\

w here ri < |;) < rj is the region o f con vergen ce o f X (z).
com plex variable z in polar form as

(4.2.54)
L et us express the

(4.2.55)
where r = |z| and co = 4 ;. T h en , w ithin the region o f con vergen ce of X (z), we
can substitute z = r e jai into (4.2.54), T his yields
(4.2.56)
From the relationship in (4.2.56) w e n ote that X ( z ) can be interpreted as
the Fourier transform o f the signal seq u en ce x ( n ) r ~ . T he w eigh tin g factor r ~ n is
growing with n if r < 1 and decaying if r > 1 . A ltern atively, if X ( z ) con verges for

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

265

1-1 = 1 , then

X (:)U , = X H =

2 2 x { , ! ) e - 1,0,1

(4.2.57)

T h erefore, the Fourier transform can be view ed as the z-transform o f the sequ en ce
evaluated on the unit circle. If X ( z ) d oes not converge in the region |z| 1 [i.e.. if
the unit circle is n o t contained in the region o f con vergen ce o f X (z)], the Fourier
transform X ( i o ) d o es not exist.
W e should n o te that the existen ce of the z-transform requires that the s e
quence {jr ( )r"} be ab solu tely sum m able for som e value o f r. that is.
OC

22

| < oc

(4.2.58)

n~-oc

H en ce if (4.2.58) con verges only for values o f r > ro > 1. the z-transform exists,
but the Fourier transform d oes not exist. This is the case, for exam p le, for causal
seq u en ces o f the form jr(rc) = a " u(n ), where jo > 1 .
T here are seq u en ces, how ever, that do not satisfy the requirem ent in (4.2.58).
for exam p le, the seq u en ce
sin avrc

x ( n ) ----------

~ oo < n < oc

(4.2,5^)

Tin

This seq u en ce d o es not have a .--transform. Since it has a finite energy, its Fourier
transform con verges in the m ean-square sense to the d iscon tin uou s function X {(jo ).
defined as
M <*v
[ (J,

(4 2 6 0 )

<w( < \
a>\ < 71

In con clu sion , the existen ce of the z-transform requires that (4,2.58) be sat
isfied for so m e region in the z-plane. If this region contains the unit circle, the
Fourier transform X(cu) exists. H ow ever, the existen ce o f the Fourier transform,
which is defined for finite energy signals, d oes not n ecessarily ensure the existence
o f the z-transform .

4.2.7 The Cepstrum

Let us con sid er a seq u en ce {jc(/j)} having a z-transform X (z). W e assum e that
(jr(n)) is a stable seq u en ce so that X ( z ) con verges on the unit circle. The c o m p l e x
ce pstru m o f the seq u en ce {jc()} is defined as the seq u en ce (cr(n)}, which is the
inverse z-transform o f Cjj(z), w here
Cjciz) = In X (z)

(4.2.61)

T h e com p lex cepstrum exists if C^tz) con verges in the annular region n <
\z\ < n , w here 0 < r\ < 1 and r2 > 1. W ithin this region o f con vergen ce, C t (z)
can be represen ted by the Laurent series
Cx(z) = In X (z) =

c^

z ~n

(4.2.62)

266

Frequency Analysis of Signals and Systems

Chap. 4

where

cAn) =

f In X ( z ) z n' 1d z
Jc

(4.2.63)

C is a closed contour about th e origin and lies w ithin the region o f convergence.
Clearly, if C, (z) can be rep resen ted as in (4.2.62), the com plex cepstrum sequence
{cr(rt)} is stable. F urtherm ore, if the com p lex cepstrum exists, Cx (z) converges on
the unit circle and hence w e have
CC

CAat) = lnX(cu) =

2 2 c A n ) e ~ JU,n

(4.2.64)

n-OC

where (cv(/i)} is the sequ en ce ob tain ed from the inverse F ourier transform of
In X(to). that is,
1 r
c r(n) = /
ln X (a >)eJwnd w
2tt J _ n

(4.2.65)

If w e express X(co) in term s of its m agnitude and p h ase, say

X(a>) = \X (co)\eJf>iw)

(4.2.66)

in X(co) = In |X(ct))| + jd(a>)

(4.2.67)

th en

By substituting (4.2.67) into (4.2.65), w e obtain the com plex cepstrum in the form
1
fn
cx (n) I [In |X ( a >)| + j6(cL>)]eJU,ndco
2jt J _ n

(4,2.68)

W e can separate the inverse F ourier transform in (4.2.68) into the inverse Fourier
transforms o f In |X (w )| and 9(a>)\
cm(n) = 2 -

ln\X(a>)\eJl^dcL>

(4.2.69)

ce(n) = - ^

6(co)eJa>ndco

(4.2.70)

In som e applications, such as sp eech signal processing, only the com p onent c(n)
is com puted. In such a case the phase o f X (a>) is ignored. T h erefore, the sequence
{*()} cannot b e recovered from {cm(n)j. That is, the transform ation from (jr(n)}
to {cm(n)} is not invertible.
In speech signal p rocessing, the (real) cepstrum has b een used to separate
and thus to estim ate the spectral con ten t o f the sp eech from the pitch frequency
of the speech. T he com plex cepstrum is used in practice to separate signals that
are con volved . T h e process o f separating tw o con volved signals is called d e c o n
volution and the use o f the com p lex cepstrum to perform the separation is called
h o m o m o r p h i c d econ vo lutio n. T his topic is discussed in Section 4.6.

Sec. 4.2

Frequency Analysts of Discrete-Time Signals

267

4.2.8 The Fourier Transform of Signals with Poles on the

Unit Circle
A s was show n in Section 4.2.6, the Fourier transform o f a seq u en ce jc(h) can be
determ ined by evaluating its z-transform X (z) on the unit circle, provid ed that the
unit circle lies within the region o f con vergen ce o f X (z). O therw ise, the Fourier
transform d o es not exist.
T h ere are som e aperiodic sequ en ces that are neither ab solu tely sum m able
nor square sum m able. H en ce their Fourier transform s do not exist. O n e such
seq u en ce is the unit step seq u en ce, which has the z-transform

A n oth er such seq u en ce is the causal sinusoidal signal seq u en ce x ( n ) = (coscoon)

u(n). T his seq u en ce has the z-transform
y , , _

1 - Z ~ COS (at)
1 - 2 z _i

cosojo

+ z -2

N o te that both o f these seq u en ces have p oles on the unit circle.
For seq u en ces such as these tw o exam ples, it is som etim es useful to extend
the Fourier transform representation. T his can be accom plished, in a m ath em ati
cally rigorous w ay, by allow ing the Fourier transform to contain im pulses at certain
frequencies corresponding to the location o f the p oles o f X ( z ) that lie on the unit
circle. T he im pu lses are functions o f the con tinu ou s frequency variable co and
have infinite am plitude, zero width, and unit area. A n im pulse can be view ed as
the lim iting form o f a rectangular pulse of height 1 /a and width a, in the limit
as a -+ 0. Thus, by allow ing im pulses in the spectrum of a signal, it is p ossible
to exten d the Fourier transform representation to som e signal seq u en ces that are
neither absolutely sum m able nor square sum m able.
T h e follow in g exam ple illustrates the exten sion of the F ourier transform rep
resentation for three sequ en ces.
Exam ple 4.2.5
D eterm ine the Fourier transform of the following signals.
(a) x i(n) = u(n)
(b) jr2(n) = (-l)" (n )

(c) jc3(n) = (coswon)u(n)

by evaluating their z-transforms on the unit circle.
Solution
(a) From Table 4.3 we find that
XiU) = r - t - j = - ^ r
ROC: |c| > 1
1 - z '1
z -1
Xi(z) has a pole, p\ = 1, on the unit circle, but convenges for |z| > 1.

268

Frequency Analysis of Signals and Systems

Chap. 4

If we evaluate A'l (:i on the unit circle, except at : = 1. we obtain

X \ ( w ) = ------ :------------ = - :--------- e

2i/ sin(w/2>

to = 2 r r k

2sm(j/2)

k = 0 . J . . ..

At a) = 0 and multiples of 2,t , A|( w ) contains impulses of area .t .

Hence the presence of a pole at ; = 1 (i.e.. at w = 0) creates a problem
only when we want to compute A'i(w) at w = 0 .because |A](tu)i * oc as
to > 0. For any other value of to. X \ (to) is finite (i.e.. well behaved). Although,
at first glance one might expect the signal to have zero-frequency components
at all frequencies except at as = 0. this is not the case. This happens because
the signal .\|(n) is not a constant for all oc < < oc. Instead, it is turned
on at n = 0. This abrupt jump creates all frequency com ponents existing in
the range 0 < t o < tt. Generally, all signals which start at a finite time have
nonzero-frequency com ponents everywhere in the frequency axis from zero up
to the folding frequency.
(b ) From Table 3.3 we find that the .'-transform of a''uin) with a = 1 reduces to
1

XA:)

---------- r =

1+ : !

;+ l

R O C :

|;i >

which has a pole at : = - 1 = c-n . The Fourier transform evaluated at frequen

cies other than a> = tt and multiples of 2tt is
A'lfw)

- -------

2 cos (to/2)

a> *

In this case the impulses occurs at w =

Hence the magnitude is
I X-> ( t o ) | =

-------- --

2[ cos(to/2)

tt

t!)

2 , t (k r

|)

k =

0.

1.

. ..

+ 2rrk.

2 tZ k +

TT

^ = 0 . 1 . . . .

and the phase is

X; (to) =

if cos < 0
Note that due to the presence of the pole at a = 1 (i.e.. at frequency w = tt),
the m agnitude of the Fourier transform becomes infinite. Now \X(w)\ * oc as
w >
n. We observe that (])nu(n) = (cosTrn)u(n), which is the fastest possible
oscillating signal in discrete time.
(c) From the discussion above, it follows that A?(w) is infinite at the frequency
com ponent w = too. Indeed, from Table 3.3. we find that
:

1 C_1 COS to,)

1 - 2: ~ cos wo +

) = (coStonfi)tt(n) < A-i(;) = --- - ------------

R O C : |-| > 1

The Fourier transform is

1 - e ' 1" cos cuo

X 3 (to) - --------- :------------------------------(1 -

to ^ ito o 4- 27tk

k = 0. 1 . . . .

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

269

The magnitude of X3 M is given by

|1 e~,<u cos tool

wu -I- 2 n k

k = 0, 1 ....

Now if w = cm or w = tuo. |X 3 (a>)| becomes infinite. For all other frequencies,

the Fourier transform is well behaved.

4.2.9 The Sampling Theorem Revisited

T o p rocess a con tinu ou s-tim e signal using digital signal processing techniques, it is
necessary to convert the signal into a seq u en ce o f num bers. A s w as discussed in
Section 1.4, this is usually d on e by sam pling the analog signal, say x a(t), periodically
every T secon ds to produce a discrete-tim e signal x ( n ) given by
00 < n < 00

x ( n ) = x a( n T )

(4.2.71)

T h e relationship (4.2.71) describes the sam pling process in the tim e dom ain.
A s discussed in C hapter 1, the sam pling frequency Fs = l / T m ust be selected large
enough such that the sam pling d oes not cause any loss o f spectral inform ation (no
aliasing). In d eed , if the spectrum of the analog signal can be recovered from the
spectrum o f the discrete- tim e signal, there is no loss of inform ation. C onsequently,
w e investigate the sam pling p rocess by finding the relationship b etw een the spectra
o f signals x a(t) and x ( n ).
If x a (t) is an aperiodic signal with finite energy, its (voltage) spectrum is given
by the F ourier transform relation
(4.2.72)
w hereas the signal x a (i) can be recovered from its spectrum by the inverse Fourier
transform
(4.2.73)
N o te that u tilization o f all frequency com p on en ts in the infinite frequency range
00 < F < 00 is necessary to recover the signal x(t) if the signal x(t) is not
bandlim ited.
T h e spectrum o f a discrete-tim e signal x ( n ) , ob tain ed by sam pling x a (t), is
given by the Fourier transform relation
OO
(4.2.74)
or, equivalently,
OO

X (f) = 2 2 x ( n ) e - ^ n
dc

(4.2.75)

270

Frequency Analysis of Signals and Systems

Chap. 4

The sequ en ce x(n) can be recovered from its spectrum X(a>) or X ( / ) by the inverse
transform

-*{) -

2 tt J . x

X(co)eJU,ndco
(4.2.76)

= /

X { f ) e j2nJnd f

J-\/2
In order to determ ine the relationship b etw een the spectra of the discrete
time signal and the analog signal, w e n ote that period ic sam pling im poses a rela
tionship betw een the in d ep en dent variables t and n in the signals x a {t) and x(n),
respectively. That is,
r = nT =
F,

(4.2.77)

This relationship in the tim e dom ain im plies a corresponding relationship betw een
the frequency variables F and / in Xa(F) and X( f ) . respectively.
Indeed, substitution o f (4.2.77) into (4.2.73) yields

x(n) = xa( " T ) =

Xtl(F)ej2,TnF/F'd F

(4.2.78)

If we com pare (4.2.76) with (4.2.78), we conclude that

r 1/2
1/2

f oc
X ( f ) e j2nf " d f =

X u ( F ) e )2nnFlFd F

(4.2.79)

J -oc

\a

From the d evelop m en t in C hapter 1 w e know that periodic sam pling im poses a
relationship b etw een the frequency variables F and / of the corresponding analog
and discrete-tim e signals, respectively. That is,
/ =

(4.2.80)

With the aid o f (4.2.80), w e can m ake a sim p le change in variable in (4.2,79), and
obtain the result
y

X ^ - 0 ej2nF/F' d F =

X a ( F ) e j2,,nF/F- d F

(4.2.81)

W e now turn our atten tion to the integral on the right-hand side o f (4.2.81)T he integration range of this integral can be divided into an infinite num ber of
intervals o f width F,. Thus the integral over the infinite range can be expressed
as a sum o f integrals, that is,

oc

3C

X a ( F ) e J2,TnF/F' d F =
x

[{k+XflsF,

/
Jt= -o c J ( k - \ f l ) F s

X a ( F ) e J2jrnF/Fd F

(4.2.82)

Sec. 4.2

271

Frequency Analysis of Discrete-Time Signals

W e o b serve that X a {F) in the frequency interval (k I ) F t to (k + ~)FS is identical

to X a( F k F s) in the interval Fs/ 2 to Fs/2. C onsequently.
oc
V
Jk=-oc

p{k+\l2)F,

x.

X a( F ) e i27TnF/F' d F =

J ^ \fl\Fs

Y2

* F\ :2

X a ( F - k F , ) e p^ nF!f d F

k=-?c
'FJ2
J2

X a( F - k F < )

' ~"r ! d F

J-FJ2

(4.2.83)
where w e have used the p eriodicity of the exp on en tial, nam ely.
e j 2 n n { F + k F s )/Ft _ ^jTnnF/F,

Com paring (4.2.83). (4.2.82), and (4.2.81), w e conclude that

X ( t ) = F' T . X A F - k F . )
' *t
k=--XL

(4.2.84)

or, equivalently.
OC

X(f) = F

X a [ ( f - k ) F s]

(4.2.85)

This is the desired relationship betw een the spectrum X ( F / F , ) or X ( f ) o f the

discrete-tim e signal and the spectrum X a( F) o f the analog signal. The righl-hand
side o f (4.2.84) or (4.2.85) consists o f a periodic repetition of the sealed spectrum
Fs X a( F) with period F,. This periodicity is necessary because the spectrum X ( f )
or X ( F / F ,) o f the discrete-tim e signal is periodic with period f p = 1 or Fp = Fs .
For exam p le, su p p ose that the spectrum o f a band-lim ited analog signal is
as sh ow n in Fig. 4.18(a). T h e spectrum is zero for [FI > B. N ow . if the sam
pling frequency Fs is selected to be greater than 2 5 . the spectrum X ( F / F S) of the
discrete-tim e signal will appear as show n in Fig. 4.18(b). Thus, if the sam pling
frequency
is selected such that Fv > 2 B. where 2 B is the N yquist rate, then
= FsX a(F )

|f | < F J 2

(4.2.86)

In this case there is no aliasing and therefore, the spectrum o f the discrete-tim e
signal is identical (w ithin the scale factor F,) to the spectrum o f the analog signal,
within the fundam ental frequency range |F | < Fsf 2 or [ f \ <
O n the other hand, if the sam pling frequency Fs is selected such that Fs <
2 B, the periodic continuation o f X a( F ) results in spectral overlap, as illustrated
in Fig. 4.18(c) and (d). T h u s the spectrum X { F / F S) o f the discrete-tim e signal
contains aliased frequency com p onents o f the analog signal spectrum X a(F). The
end result is that the aliasing which occurs prevents us from recovering the original
signal x (r) from the sam ples.
G iven the discrete-tim e signal x(n) with the spectrum X ( F / F S), as illustrated
in Fig. 4.18(b ), w ith no aliasing, it is n ow p ossible to reconstruct the original analog

Figure 4.18 Sampling of an analog bandlimited signal and aliasing of spectral

components.

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

273

signal from the sam ples j ( n ) . Since in the absence of aliasing

*.<-(*()

(4.2.87)

r -

\ F \ > F 5/ 2

lo ,

and by the Fourier transform relationship (4.2.75).

f )-
/

(4.2.88)

n= -o c

the inverse Fourier transform o f X a ( F ) is

xAD =

f F' p-

X a ( F ) e s~*F ' d F

(4.2.89)

Let us assum e that F s = 2 B . W ith the substitution o f (4.2.87) in to (4.2.89), we

have
Xa(t)

F, J - F J

x { n ) e ~ j2 * Fn/F

dF

(4.2.90)
s\n(n/T )(t - nT)
(ir/T W -n T )
where x (n ) = x a( n T ) and w here T = \ / F s 1 /2 B is the sam pling interval. This
is the reconstruction form ula given by (1.4.24) in our discussion o f the sam pling
theorem .
T h e reconstruction form ula in (4.2.90) in volves the function
i KO =

s in (jr /7 );

s in 2 ^ B f

(n/T)t

2n B t

(4.2.91)

appropriately shifted by n T , n = 0 , 1 , 2 ........ and m ultiplied or w eigh ted by

the corresponding sam ples x (n T ) o f the signal. W e call (4.2.90) an in terp ola
tion form ula for reconstructing x a U) from its sam ples, and g (f). given in (4.2.91),
is the interpolation function. W e note that at t = k T , the in terp olation function
g(t n T ) is zero excep t at k = n. C onsequently. xa (t) evalu ated at t = k T is simply
the sam ple x 0{k T ). A t all other tim es the w eigh ted sum o f the tim e shifted versions
o f the interpolation function com b ine to yield exactly x a (t). T his com b ination is
illustrated in Fig. 4.19.
T he form ula in (4.2.90) for reconstructing the analog signal xa (t) from its
sam ples is called the ideal interpolation fo rm u la . It form s the basis for the sa m p lin g
theorem, which can b e stated as follow s.
Sam pling T h eo rem . A bandlim ited con tinu ou s-tim e signal, with highest fre
quency (bandw idth) B H ertz, can be uniquely recovered from its sam ples provided
that the sam pling rate Fs > 2 B sam ples per second.

274

Frequency Analysis of Signals and Systems

Chap, 4

A ccordin g to the sam pling theorem and the reconstruction form ula in (4.2,90),
the recovery o f x a(t) from its sam ples ;c(), requires an infinite num ber o f sam
ples. H ow ever, in practice w e use a finite num ber o f sam ples o f the signal and
deal with finite-duration signals. A s a con seq u en ce, w e are con cern ed only with
reconstructing a finite-duration signal from a finite num ber o f sam ples.
W hen aliasing occurs due to to o low a sam pling rate, the effect can be de
scribed by a m ultiple folding o f the frequency axis o f the frequency variable F for
the analog signal. Figure 4.20(a) show s the spectrum X a(F ) o f an analog signal.
A ccordin g to (4.2.84), sam pling of the signal with a sam pling frequency Fs results
in a periodic repetition o f X a( F ) with period Fs . If Fs < 2 B , the shifted replicas of
X g {F) overlap. T he overlap that occurs within the fundam ental frequency range
F.J2 < F < Fs/2, is illustrated in Fig. 4.20(b ). T h e corresponding spectrum of
the discrete-tim e signal w ithin the fundam ental frequency range, is obtained by
adding all the shifted portions within the range | / | < j , to yield the spectrum
shown in Fig. 4.20(c).
A careful inspection of Fig. 4.20(a) and (b) reveals that the aliased spectrum
in Fig. 4.20(c) can be obtained by folding the original spectrum lik e an accordian
with pleats at every odd m ultiple o f Fs/2. C on sequ en tly, the frequency F J 2 is
called the f o l d i n g fre q u e n c y , as indicated in C hapter 1. Clearly, then, periodic
sam pling autom atically forces a folding o f the frequency axis o f an analog signal
at odd m ultiples o f Fs/2, and this results in the relationship F = f Fs b etw een the
frequencies for con tinu ou s-tim e signals and discrete-tim e signals. D u e to the fold
ing o f the frequency axis, the relationship F f F, is not truly linear, but piecew ise
linear, to accom m odate for the aliasing effect. T his relationship is illustrated in
Fig. 4.21.
If the analog signal is bandlim ited to B < Fs/2, the relationship betw een /
and F is linear and o n e-to-on e. In other words, there is no aliasing. In practice,
prefiltering with an antialiasing filter is usually em p loyed prior to sam pling. This
ensures that frequency com p onents o f the signal above F > B are sufficiently
attenuated so that, if aliased, they cause n egligib le distortion on th e desired signal.
T h e relationships am ong the tim e-d om ain and freq u en cy-d om ain functions
x a (t), x ( n ) , X ( F ), and X ( f ) are sum m arized in Fig, 4.22. T h e relationships for

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

XJFi

0
(a)

o
T

T
(c)

Figure 4.20

Illustration of aliasing around the folding frequency.

Figure 4.21

R elationship betw een frequency variables F and / .

275

276

Frequency Analysis of Signals and Systems

Chap. 4

XJF)=
Fourier transform
pair
Xa(t)

\.

Xa(F)
xa(t) = j ~ Xa(F )ei -^' dF
a:0(F)

Reconstruction:

= tfrt)

F,
IFI < - j

sin 7r(r - nT)!T

7r(t - nT)!T

Sampling-,
x(n) - x a(nT)

x(n)

X (/)
Fourier transform
pair
x(n) = J

Figure 4.22

X(f)eill,f"df

Time-domain and frequency-domain relationships for sampled sig

nals.
recovering the con tinu ou s-tim e functions, x 0{t) and X a{F ), from th e discrete-tim e
quantities x ( n ) and X ( f ) , assum e that the analog signal is bandlim ited and that it
is sam pled at the N yquist rate (or faster).
T h e follow in g exam p les serve to illustrate the problem o f the aliasing of
frequency com ponents.

Example 4.2.6 Aliasing in Sinusoidal Signals

The continuous-time signal

xa(r) ~ cos 2nF^t

LgjZxFai

has a discrete spectrum with spectral lines at F = F U> as shown in Fig. 4.23(a). The
process of sam pling this signal with a sam pling frequency Fs introduces replicas of the
spectrum about multiples of Fs. This is illustrated in Fig. 4.23(b) fo r Fs/2 < F0 < F,To reconstruct the continuous-time signal, we should select the frequency com
ponents inside the fundam ental frequency range \F\ < Fs f l . The resulting spectrum

Sec. 4.2

277

Frequency Analysis of Discrete-Time Signals

Spectrum

_|
0

-Fo

F0

U)
Spectrum

' it

-F t

Fu - Fs 0 F,. - F0
(b)
Spcctrum

F, -<FS- /(,) f-,~F0

^
2
(c)
Spectrum

2T\
- F q - Fs

Fs F, - F0 0 F0 - F, F,
(d)
Spectrum

2
F,

-y

f3

(e)
Figure 4.23

F 0 + F,

A liasing of sinusoidal signals.

278

Frequency Analysis of Signals and Systems

Chap. 4

is shown in Fig. 4.23(c). The reconstructed signal is

* ( /) = c o s 2 j t ( F , -

F (t)i

Now. if Fv is selected such that Fs < F(l < 3F 5/2, the spectrum of the sampled
signal is shown in Fig. 4.23(d). The reconstructed signal, shown in Fig. 4.23(e). is
xu(t) = cos2jr(F (1 - Fs)i
In both cases, aliasing has occurred, so that the frequency of the reconstructed signal
is an aliased version of the frequency of the original signal.

E x a m . e 4.2.7

S am p lin g a N o n b a n d lim ited Signal

Consider the continuous-time signal

*(/) = e - AI,i

A>0

whose spectrum is given hv

X J F ) = ----- -------A- 4- (2;r F Y
Determ ine the spectrum of the sampled signal ,r(n) = xu{nT).
Solution

If we sample x(t) with a sampling frequency F, = 1/7", we have

jr(n) xuinT) e~A1'"' = (e~A1

oc < n < oc

The spectrum of x(n) can be found easily if we use a direct com putation of the Fourier
transform. We find that
F\
F, J

1 - e~1AT
1 - 2e~A1 cos I n FT + e~2AT

T =

1
Fs

Clearly, since c o s 2 x F T = c o s 2 t t ( F/ Fs ) is periodic with period Fs, so is X ( F / F S),

Since X a(F) is not bandlim ited. aliasing cannot be avoided. The spectrum of
the reconstructed signal i(r) is
i - i .

\F\<!

( t.

Xa(F) =
[o .

I F I > J

Figure 4.24(a) shows the original signal xa(t) and its spectrum X a( F ) for A = 1The sampled signal x(n) and its spectrum X ( F / F S) are shown in Fig. 4.24(b) for
= 1 Hz. The aliasing distortion is clearly noticeable in the frequency domain. The
reconstructed signal xa(r) is shown in Fig. 4.24(c). The distortion due to aliasing can
be reduced significantly by increasing the sampling rate. For example, Fig. 4.24(d)
illustrates the reconstructed signal corresponding to a sampling rate Fs = 20 Hz. It
is interesting to note that in every case xa( nT) = xa(nT), but x{t) / xa(t) at other
values of time.

Sec. 4.2

279

Frequency Analysis of Discrete-Time Signals

x (n :

.*,,(/) = e~A>11 .A = I

14
A - + (2 ttF) 2

(al

1.0

-2

-1

1 t
[

.
<b)

(c }

(d)

Figure 4.24 (a) A nalog signal xa(i ) and its spectrum Xa(Fy. (b) xin i = x a ( ti T)
and the spectrum of *(n) for A = 1 and F, = 1 Hz; (c) reconstructed signal x(i)
for F, - 1 Hz: (dj reconstructed signal i aU) for Fs = 20 Hz.

4.2.10 Frequency-Domain Classification of Signals: The

Concept of Bandwidth
Just as w e have classified signals according to their tim e-dom ain characteristics, it
is also desirable to classify signals according to their frequency-dom ain character
istics. It is com m on practice to classify signals in rather broad terms according to
their frequency content.
In particular, if a p ow er signal (or energy signal) has its p ow er density sp ec
trum (or its energy density spectrum ) concentrated about zero frequency, such
a signal is called a low -freq ue ncy signal. Figure 4.25(a) illustrates the spectral
characteristics o f such a signal. O n the other hand, if the signal p ow er density

Frequency Analysis of Signals and Systems

280
Xa{F)

Chap. 4

X{a>)

fa)
X(w)

Xa(F)

(b)
XJF)

X(oj)

(c)
Figure 4.25 (a) Low-frequency, (b) high-frequency, and (c) medium-frequency
signals.
spectrum (or the energy density spectrum ) is concentrated at high frequencies,
the signal is called a h igh -fr eq uency signal. Such a signal spectrum is illustrated
in Fig. 4.25(b). A signal having a p ow er density spectrum (or an energy density
spectrum ) concentrated som ew h ere in the broad frequency range b etw een low fre
q uencies and high frequencies is called a m e d i u m - f r e q u e n c y signal or a bandpass
signal. Figure 4.25(c) illustrates such a signal spectrum .
In addition to this relatively broad frequency-dom ain classification o f signals,
it is often desirable to express q uantitatively the range o f freq u en cies over which
the p ow er or energy density spectrum is concentrated. This q u an titative m easure
is called the b a n d w i d t h o f a signal. For exam p le, su p p ose that a con tinu ou s
tim e signal has 95% o f its p ow er (or energy) density spectrum con cen trated in the
frequency range F\ < F < F 2. T hen the 95% bandw idth o f the signal is F2 F 1 . In
a sim ilar m anner, w e may define the 75% or 90% or 99% bandw idth o f the signal.

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

281

In the case o f a bandpass signal, the term n a r r o w b a n d is used to describe

the signal if its bandw idth Fz - F\ is m uch sm aller (say, by a factor o f 10 or m ore)
than th e m edian frequency {Fi + F \) f2. O therw ise, the signal is called wideband.
W e shall say that a signal is b a n d lim ite d if its spectrum is zero ou tsid e the
freq u en cy range
> B. For exam ple, a con tinu ou s-tim e finite-energy signal x (t)
is bandlim ited if its Fourier transform X ( F ) = 0 for m > B. A d iscrete-tim e
finite-en ergy signal x ( n ) is said to be (periodically) b a n d lim ited if
for a>o < M < x

jX(ti>)| = 0

Sim ilarly, a p eriod ic co n tinu ou s-tim e signal x p (r) is p eriodically bandlim ited if its
F ourier coefficien ts c* = 0 for || > M , w here M is som e p ositive integer. A
p erio d ic discrete-tim e signal w ith fundam ental period N is periodically bandlim ited
if the Fourier coefficients ck = 0 for kG < |j < N . Figure 4.26 illustrates the four
types o f bandlim ited signals.
B y exploiting the duality b etw een the frequency dom ain and the tim e dom ain,
we can provide sim ilar m ean s for characterizing signals in the tim e dom ain. In
particular, a signal x ( t ) will be called tim e- lim ited if
x(r) =

|f|

>

If the signal is p eriod ic with period Tn, it will be called periodic ally time- lim ited if
x p it)

<

\t\

<

T p f l

If w e have a d iscrete-tim e signal x( n ) o f finite duration, that is,

.r(n) = 0

In I > N

it is also called tim e-lim ited. W hen the signal is period ic with fundam ental period
it is said to be p eriod ically tim e-lim ited if
x(n} = 0

Figure 4 J 6

no < )n\ < N

Some exam ples of bandlim ited signals.

282

Frequency Analysis of Signals and Systems

Chap. 4

W e state, w ithout proof, that no sign al can be time-lim ited a n d ban dlimited
sim ultaneously. Furtherm ore, a reciprocal relationship exists b etw een the time
duration and the frequency duration o f a signal. T o elaborate, if w e have a shortduration rectangular pulse in the tim e dom ain, its spectrum has a width that is
inversely proportional to the duration o f the tim e- dom ain pulse. T he narrower
the pulse b eco m es in the tim e dom ain, the larger the bandwidth of the signal
b ecom es. C onsequently, the product o f the tim e duration and the bandwidth of
a signal cannot be m ade arbitrarily sm all. A short-duration signal has a large
bandwidth and a sm all bandwidth signal has a lon g duration. T hus, for any signal,
the tim e-b an dw id th product is fixed and cannot b e m ade arbitrarily small.
Finally, w e n o te that w e have discussed frequency analysis m eth od s for peri
odic and aperiodic signals with finite energy. H o w ev er, there is a family o f deter
m inistic aperiodic signals with finite pow er. T h ese signals consist o f a linear super
p osition o f com plex exp on en tials with nonharm onically related frequencies, that is,
M
x{n) = Y l A ke ^ n
k= 1
w here &>i, a s , . . . , a >m are nonharm onically related. T h ese signals have discrete
spectra but the distances am ong the lin es are nonharm onically related. Signals
with discrete nonharm onic spectra are som etim es called quasi-periodic.

4.2.11 The Frequency Ranges of Some Natural Signals

T he frequency analysis tools that w e have d ev elo p ed in this chapter are usually
applied to a variety o f signals that are en cou n tered in practice (e.g., seism ic, b io lo g
ical, and electrom agn etic signals). In gen eral, the frequency analysis is perform ed
for the purpose o f extracting inform ation from the ob served signal. For exam ple,
in the case o f biological signals, such as an E C G signal, the analytical tools are
used to extract inform ation relevant for d iagn ostic purposes. In th e case o f seism ic
signals, w e may b e interested in detectin g th e p resen ce o f a nu clear exp losion or in
d eterm ining the characteristics and location o f an earthquake. A n electrom agnetic
signal, such as a radar signal reflected from an airplane, con tains inform ation on
the p osition o f the plane and its radial velocity. T h ese param eters can be estim ated
from observation o f the received radar signal.
In processing any signal for the p urpose o f m easuring param eters or ex
tracting other types o f inform ation, o n e m ust know approxim ately the range o f
freq u en cies contained by the signal. For referen ce, T ables 4.1, 4.2, and 4.3 give
approxim ate lim its in the frequency d om ain for b iological, seism ic, and electro
m agnetic signals.

4.2.12 Physical and Mathematical Dualities

In the p revious section s o f the chapter w e have introduced several m eth od s for the
frequency analysis o f signals. Several m eth od s w ere n ecessary to accom m odate the

Sec. 4.2

Frequency Analysis of Discrete-Time Signals

TABLE 4.1

283

FR EQ U E N C Y R A NG ES OF SO M E BIO LOG ICAL

SIGN ALS

F requency R ange (Hz)

Type of Signal

0-20
0-20

E lectroretinogram 8
Electronystagtnogram h
P neum ogram '
Electrocardiogram (EC G )
E lectroencephalogram (E E G )
Electrom yogram d
Sphygm om anogram e
Speech
"A
A
CA
dA
CA

0-40
0-100
0-100
10-200
0-200
100-^000

graphic recording of retina characteristics.

graphic recording of involuntary m ovem ent of the eyes.
graphic recording of respiratory activity.
graphic recording of m uscular action, such as m uscular contraction.
recording of blood pressure.

TABLE 4.2

FR E Q U E N C Y R A NG ES OF SO M E S E IS M IC SIGNALS

Type o f Signal

F requency R ange (Hz)

W ind noise
Seismic exploration signals
E arthquake and nuclear explosion signals
Seismic noise

TABLE 4.3

100-KXX)
10-11X1
(1.01-1(1
0.1-1

F R E Q U E N C Y RANGES O F E L E C T R O M A G N ETIC SIGN ALS

W avelength (m )

F requency R ange (Hz)

10M 02
io-- i o - :

3 x lO4^ x JO6
3 x 1(^-3 x 10U1

Type of Signal
R adio broadcast
S hortw ave radio signals
R adar, satellite com m unications.
space com m unications.
com m on-carrier microwave
Infrared
Visible light
U ltraviolet
G am m a rays and x-rays

3.9

1-10
u rM o ^ 6
10_7-8.1
10~7
10 -7-]O - 8
u r M o - 10

3 x
3x
3.7 x
3 x
3 x

10^-3 x
10 113 x
10w-7.7
1015-3 x
10173 x

1010
I0 i4
x 1014
lO16
101K

differen t types o f signals. T o sum m arize, the follow in g frequency analysis tools
have b een introduced:
1. T he F ourier series for con tinu ou s-tim e p eriod ic signals.
2. T he F ourier transform for con tinu ou s-tim e ap eriod ic signals.
3. T he F ourier series for discrete-tim e periodic signals.

4. T he F ourier transform for discrete-tim e aperiodic signals.

284

Frequency Analysis of Signals and Systems

Chap. 4

Figure 4.27 sum m arizes the analysis and synthesis form ulas for these types of
signals.
A s w e have already indicated several tim es, there are two tim e-dom ain char
acteristics that determ ine the type of signal spectrum w e obtain. T h ese are whether
the tim e variable is con tinu ou s or discrete, and w h eth er the signal is periodic or
aperiodic. Let us briefly sum m arize the results o f the previous sections.

Continuous-time signals have aperiodic spectra. A close inspection of

the F ourier series and Fourier transform analysis form ulas for continuous-tim e
signals d o es not reveal any kind of periodicity in the spectral dom ain. This lack of
p eriodicity is a con sequ en ce of the fact that the com p lex ex p on en tial exp(y2jr Ft)
is a function o f the continuous variable t, and hence it is not p eriod ic in F . Thus
the frequency range o f continuous-tim e signals exten d s from F = 0 to F = 0 0 .
Discrete-time signals have periodic spectra. In d eed , both the Fourier
series and the Fourier transform for discrete-tim e signals are p eriod ic with period
co = 2n . A s a result o f this periodicity, the frequency range o f d iscrete-tim e signals
is finite and exten d s from co = 1r to w tt radians, where w = n corresponds to
the highest p ossible rate o f oscillation.
Periodic signals have discrete spectra. A s we have ob served , periodic
signals are described by m eans o f Fourier series. T h e Fourier series coefficients
provide the lines" that constitute the discrete spectrum . T h e line spacing A F
or A f is equal to the inverse o f the period Tp or N , resp ectiveiy, in the time
dom ain. That is. A F = 1/7,, for continuous-tim e period ic signals and A f = 1 / N
for discrete-tim e signals.
Aperiodic finite energy signals have continuous spectra. This prop
erty is a direct con seq u en ce o f the fact that both X ( F ) and X ( w ) are functions
o f exp { j 2 n F t ) and exp {jeon), respectively, which are con tinu ou s functions o f the
variables F and co. The continuity in frequency is necessary to break the harmony
and thus create aperiodic signals.
In sum m ary, w e can conclude that periodic ity with p e r i o d a in o n e domain
automatically im plies discretization with s p a c i n g " o f 1 jot in the o th e r do m ain , and
vice versa.
If w e k eep in m ind that p eriod in the frequency d om ain m eans the fre
quency range, spacing'' in the tim e dom ain is the sam pling p eriod T , line spacing
in the frequency dom ain is A F , then a = Tp im plies that 1 /a = \ j T p = A F , a = N
im plies that A f = \ / N , and a = Fs im plies that T = 1 /F S.
T h ese tim e-frequency dualities are apparent from observation o f Fig. 4.27.
W e stress, how ever, that the illustrations u sed in this figure do n ot correspond to
any actual transform pairs. Thus any com parison am ong them sh ou ld be a v o i d e d .
A careful in sp ection o f Fig. 4 .2 7 also reveals som e m athem atical s y m m e tr ie s
and dualities am ong the several frequency analysis relationships. In particular,

Figure 4.27

Summary of analysis and synthesis form ulas.

286

Frequency Analysis of Signals and Systems

Chap. 4

we observe that there are dualities b etw een the follow in g analysis and synthesis
equations:
1. T he analysis and synthesis eq u ation s o f the con tinu ou s-tim e Fourier trans
form.
2. T he analysis and synthesis eq u ation s of the discrete-tim e F ourier series.
3. T he analysis eq u ation o f the con tinu ou s-tim e F ourier series and the synthesis
equation o f the discrete-tim e Fourier transform.

4. T he analysis equation o f the discrete-tim e F ourier transform and the synthesis

equation o f the continuous-tim e Fourier series.
N o te that all dual relations differ on ly in the sign of the exp on en t of the
corresponding com p lex exp on en tial. It is interesting to note that this change in
sign can be thought o f either as a folding of the signal or a folding o f the spectrum ,
since
e - j 2 n Fl _ e j 2n( - FU _

If w e turn our atten tion now to the spectral density o f signals, w e recall that
we have used the term ener gy density sp e ctru m for characterizing finite-energy
aperiodic signals and the term p o w e r density sp e ctru m for period ic signals. This
term inology is con sistent with the fact that periodic signals are p ow er signals and
aperiodic signals w ith finite energy are energy signals.

4.3 PROPERTIES OF THE FOURIER TRANSFORM FOR

DISCRETE-TIME SIGNALS
The Fourier transform for aperiodic finite-energy discrete-tim e signals described
in the preceding section p ossesses a num ber of properties that are very useful in
reducing the com p lexity o f frequency analysis problem s in m any practical appli
cations. In this section w e d evelop the im portant p roperties o f the Fourier trans
form. Similar p roperties hold for the Fourier transform of ap eriod ic finite-energy
continuous-tim e signals.
For co n v en ien ce, w e adopt the notation
OC

X(a>) = F {x{)} =

x ( n ) e ~ Jcun

(4.3.1)

/J CC

for the direct transform (analysis eq u ation ) and

x ( n ) = F ~ 1{X(tv)] =

X{u>)ejmndoo

(4.3.2)

2 * J 2n

for the inverse transform (syn thesis eq u ation ). W e also refer to x ( n ) and X(o)) as
a Fourier tra nsfo rm p a i r and d en ote this relationship with the n otation

Sec. 4.3

Properties of the Fourier Transform for Discrete-Time Signals

287

R ecall that X (a>) is periodic with period 2 n . C on sequ en tly, any interval
o f length 2 n is sufficient for the specification o f the spectrum . U sually, we plot
the spectrum in the fundam ental interval [jr. tt J. W e em p hasize that all the
spectral inform ation contained in the fundam ental interval is necessary for the
co m p lete d escription or characterization o f the signal. For this reason, the range
o f integration in (4.3.2) is always 2jt, in d ep en dent o f the specific characteristics o f
the signal w ithin the fundam ental interval.

4.3,1 Symmetry Properties of the Fourier Transform

W hen a signal satisfies som e sym m etry p roperties in the tim e dom ain, these prop
erties im pose so m e sym m etry conditions on its Fourier transform . E xp loitation
of any sym m etry characteristics leads to sim pler form ulas for both the direct and
inverse F ourier transform. A discussion o f various sym m etry properties and the
im plications o f these properties in the frequency dom ain is given here.
Suppose that both the signal x(n) and its transform X(co) are com plex-valued
functions. T hen they can be expressed in rectangular form as
jr(n) = x K(n) + jX f(n )
X'(co) = X k {(o ) + j X [ ( co)

(4.3.4)
(4.3.5)

By substituting (4.3.4) and e~>w cosu> - / s in co into (4.3.1) and separating the
real and im aginary parts, we obtain
OC

Xk(lo) =

22

costu/i -f- xj{n) sin con]

(4.3.6)

ft = OC

SC
X/(a>) = 2 2 [*jfOO sin a>n.*/() cos&inj
n=-oc

(4.3.7)

In a sim ilar m anner, by substituting (4.3.5) and eJm = cos co + j s m u > into (4.3.2),
w e obtain
x R(n) = - f [X/?(a>) cosam X/(co) sin con]dco
2 n J2n

(4.3.8)

X/(n)

(4.3.9)

f [X ft (co) sincon + X/ ( w) cos con]dco

"T J2k
N o w , let us in vestigate som e special cases.

Real signals. If x(n) is real, then x/t(n) = x(n) and xi(n) = 0.

(4.3.6) and (4.3.7) reduce to

H en ce

OC

X R(co) 2 2 x ( n ) cos con

(4.3.10)

Frequency Analysis of Signals and System s

288

Chap. 4

and
X i { w) =

* ( )s in o jrt

(4.3.11)

Since c o s (con) = cos ton and sin (con) = sin cun, it follow s from (4.3.10) and
(4.3.11) that

X R( - w ) = X R(co)

(ev en )

(4.3.12)

X, ( -co) = -X /(o > )

(o d d )

(4.3.13)

If w e com bine (4.3.12) and (4.3.13) into a single eq u ation , w e have

X*(co) = X ( - w )

(4.3.14)

In this case we say that the spectrum o f a real signal has H e r m itia n sy m m etry.
W ith the aid o f Fig. 4.28, w e observe that the m agnitude and phase spectra
for real signals are
X(to)\ =

J X\{co)

+ Xj(co)

(4.3.15)
(4.3.16)

4 _X|oj| = tan
X

( co)

A s a con sequ en ce o f (4.3.12) and (4.3.13), the m agnitude and p h ase spectra also
p ossess the sym m etry properties
|,Y(co)| = \ X { co)\
liX (-co) = -& X (to )

(ev en )

(4.3.17)

(od d )

(4.3.18)

In the case o f the inverse transform o f a real-valued signal [i.e., Jt(n) = jcj?(n)],
(4.3.8) im plies that
(n) =

[X/f(a>) cosa>n X /(a>)sinam ]dw

(4.3.19)

Jin

R(co)
Since both products X R
(to) cos ton and X /( oj) sin con are even fun ction s o f co, we
have
1 r
= I [X^(cu) cos con X/(co) sin con]dco

x Ja

Imaginary axis

functions.

(4.3.20)

Sec. 4.3

Properties of the Fourier Transform for Discrete-Time Signals

289

Real and even signals. If .*<) is real an d ev e n [i.e., x ( ~ n ) = * ( )], th en

jf(/i)cosw ?i is ev en a n d x ( n )s in w n is od d . H e n c e , fro m (4.3.10). (4.3.11). an d
(4.3.20) w e o b ta in
Xff(a>) = x (0 ) + 2

x(n)coswn

(e v e n )

(4.3.21)

X i ( w) = 0

(4.3.22)

jr(n) = j f * X R(o>) cos cun da>

(4.3.23)

T h u s real a n d ev en signals po ssess real-v a lu e d sp e c tra , w hich, in ad d itio n , a re even

fu n ctio n s o f th e freq u e n cy v ariab le a>.

Real and odd signals. If x(/i) is real an d o d d [i.e., x ( n) = - x ( ) ] , th e n

x ( n )c o s w n is o d d an d .vODsinwn is even. C o n se q u e n tly . (4.3.10). (4.3.11) and
(4.3.20) im ply th a t
A'k(oi) = 0

(4.3.24)
(o d d )

(4.3.25)

(4.3.26)
T h u s re a l-v a lu e d o d d signals possess p u rely im ag in arv -v alu ed sp e ctral c h a ra c te ris
tics. w hich, in a d d itio n , a re o d d f unct i ons o f th e freq u e n cy v ariab le co.

Purely imaginary signals. In th is case x K(n) = 0 an d x ( n ) - j x f (n). T h u s

(4.3.6). (4.3.7), a n d (4.3.9) re d u c e to
(o d d )

(4.3.27)

(ev en )

(4.3.28)

(4.3.29)
If x / ( n ) is o d d [i.e., x / ( n) = x/ (n)], th e n
(o d d )
X/(a>) = 0

(4.3.30)
(4.3.31)
(4.3.32)

290

Frequency Analysis of Signals and Systems

Chap. 4

S im ilarly, if xj ( n) is ev en [i.e.. x / ( n) .v/()]. we h ave

X *M -

(4.3.33)

X
X [ ( oj) = x /(0 ) + 2 Y ^ x i ( n ) cos con

(ev en )

(4.3.34)

n= 1

1 r
x ; ( n) = I

X Jo

X 1 (co) cos cun d w

(4.3.35)

A n a rb itra ry , possibly co m p lex -v alu ed signal x (n ) can be d e c o m p o s e d as

x ( n ) = Xfi(n) + j x i ( n ) = x R
e (n) + x ^ ( n) + j [ x ] ( n ) + *"( )]
(4.3.36)
= Ar (/i) + Jr(rt)
w h ere, by d efin itio n ,
x f (n) = x K
f ( n) - h j Xf ( n ) = j[at() + jr* (- n )]
x(n) = x'x(n) + j x/ ( r i ) - |[ jr( n ) - x * ( - n ) ]
T h e su p e rscrip ts e a n d o d e n o te th e even an d o d d signal c o m p o n e n ts , respectively.
W e n o te th a t x e(n) x e(n) a n d x r,(n) = - a :(). F ro m (4.3.36) a n d the F o u rie r
tra n sfo rm p ro p e rtie s e s tab lish ed ab o v e, w e o b ta in th e follow ing relatio n sh ip s:
x{n) = [*() + j x l (n)\ + [.i*(/0+y'A'/(n)]
]

xito) = [x(R(co) + j x f u o ) 1 +

'

(4.3.37)
+ j x ) \ w) ]

T h e se sy m m e try p ro p e rtie s of th e F o u rie r tra n sfo rm are su m m a riz e d in T a

ble 4.4 an d in Fig. 4.29. T h ey a re o ften used to sim plify F o u rie r tra n sfo rm calcu
latio n s in p ractice.

Example 4.3.1
Determ ine and sketch X R(w), X , ( w ), l^ioOi. and ^X(co) for the Fourier transform
X(a>) =

- 1< a < 1

(4.3.38)

1 a e ~ ,w

Solution By multiplying both the num erator and denom inator of (4 .3 .3 8 ) by the
complex conjugate of the denom inator, we obtain
1 ae!w
1 a co s co j a sin co
X(w) - ----------------------------- = ------------------ ------
(1 a e ~ }U,) ( 1 - a e JU>)

1 2a c o s o j + a "

This expression can be subdivided into real and imaginary parts. Thus we obtain
1

X r (w )
1

a cos co
a cos o j + a 1

a sin w
X,(u>) = -
1 2a cos co + a 2
Substitution of the last two equations into (4.3.15) and (4.3.16) yields the mag
nitude and phase spectra as

|X(<u)i -

1 - = =

v l 2a cos co

a2

(4.3.39)

Sec. 4.3

Properties of the Fourier Transform for Discrete-Time Signals

TABLE 4.4

SYMMETRY PROPERTIES OF THE DISCRETE-TIME

FOURIER TRANSFORM

Sequence

DTFT

xin)

A' (w)
Xi-w)
X (a>)

x (n )

j 'l - n )
J(n)

A>(n) =

"I-

4 [ j r ( n } + a ' ( n ) ]

X R\.u

- , v ( n)]

j X, { t o)

x(n) =

^)]

A'/jlo)) =: ^[X(t^) -V*(aj)j

jx /[n )

Real Sienals
A n y real signal
x (n )

X (w ) = X ' ( - u i )
X r (w ) = X r { id)
Xfiuj)
IX ( oj)| = ] X ( <y)|

\ t.(n)

i (a (>>)

,r( n )]

(r e a l an d e v e n )
x.An)

i|.r(n) -

r l n)]

(r e a l a n d o d d )

F igure 4.29

= -iX (-w )
An(to)
(real and even)

]X
(imaginary and odd)

Sum m ary of symmetry properties for the F o u rier transform .

291

Frequency Analysis of Signals and Systems

292

Chap, 4

and
XrX(w) =

- tan

(4.3.40)

1
1 a cos w

Figures 4.30 and 4.31 show the graphical representation of these spectra foT
a = 0.8. The reader can easily verify that as expected, all symmetry properties for
the spectra of real signals apply to this case.
Example 4.3.2
D eterm ine the Fourier transform of the signal
| A.

M < n < M

1 0,

elsewhere

Solution CleaTly, x { n) = x{n). Thus *() is a

we obtain

Teal

(4.3.41)
and even signal. From (4.3.21)

X(a>) = X K(a>) = A I 1 4- 2 ^ c o s

Xg((i>)

(a)

(b)
Figure 4 J 0

G raph of X r ( o>) and X / ( w ) for the transform in Exam ple 4,3.1.

Sec. 4.3

Properties of the Fourier Transform for Discrete-Time Signals

293

IXlcoM

(a)

^X{w)

Figure 4.31

Magnitude and phase spectra of the transform in Example 4.3.1.

If we use the identity given in Problem 4.13, we obtain the simpler form
X(u>) = A

sinfAf + l)oj
. - -J sin(tu/2)

Since X (a>) is real, the magnitude and phase spectra are given by
sin(Af +
IX M I =

sin(a)/2)

I
|

(4,3.42)

and
0,
jt,

Figure 4,32 shows the graphs for X(w).

if X{w) > 0
if X(a>) < 0

(4.3.43)

294

Frequency Analysis of Signals and Systems

Chap. 4

X(n)

- MOM
X(u>)

IXMI

ZX(w)

4.3.2 Fourier Transform Theorems and Properties

In th is se ctio n w e in tro d u c e se v era l F o u rie r tra n sfo rm th e o re m s a n d illu stra te th eir
u se in p ra c tic e b y ex am p les.

Linearity.

If

Sec. 4,3

295

Properties of the Fourier Transform for Discrete-Time Signais

an d
F

X2 (n) * X ;(a/)
th en
(4.3.44)
Sim ply sta te d , th e F o u rie r tra n sfo rm a tio n , view ed as an o p e ra tio n on a signal
jt(rt), is a iin e a r tra n sfo rm a tio n . T h u s th e F o u rie r tra n sfo rm o f a lin e a r c o m b in a tio n
o f tw o o r m o re signals is eq u al to th e sam e lin e a r co m b in a tio n o f th e F o u rie r
tra n sfo rm s o f th e in d iv id u al signals. T h is p ro p e rty is easily p ro v e d by using (4.3.1).
T h e lin earity p ro p e rty m a k e s th e F o u rie r tra n sfo rm su ita b le fo r th e stu d y o f lin ear
system s.
Example 4.3.3
Determ ine the Fourier transform of the signal
(4.3.45)

Beginning with the definition of the Fourier transform in (4.3.1), we have

Xi(a>) =

22 M'Oe- -""" = 22a

'e
i=D

The summation is a geometric series that converges to

1 ae~)w

provided that
\ae ""I = Icij |e ""| = \a] < 1

which is a condition that is satisfied in this problem. Similarly, the Fourier transform
of X j i n ) is

296

Frequency Analysis of Signals and Systems

Chap, 4

By combining these two transforms, we obtain the Fourier transform of x(n) in the
form
X ((i>) =
l _ a:
1 -

(4.3.46)

2a co so j + a 2

Figure 4.33 illustrates *(n) and X(w) for the case in which a = 0.8.

Time shifting.

If
X (a))

x (n )
th e n
x ( n - k)

e - ja,kX ( w )

(4.3.47)

T h e p ro o f o f th is p ro p e rty follow s im m e d ia te ly from (he F o u rie r tra n sfo rm of

x ( n k) by m ak in g a change in th e su m m a tio n index. T h u s
F[x{n - k ) } = X(a>)e~J<uk
= \X(aj )\ej[^ Xuu]- <,,k]

X(a>)

Figure 4.33

Sequence x I n ) and its F ourier transform in Exam ple 4.3.3 with

Sec. 4.3

Properties of the Fourier Transform for Discrete-Time Signals

297

T h is re la tio n m ean s th a t if a signal is sh ifted in th e tim e d o m a in by k sam

p les, its m a g n itu d e sp e c tru m re m a in s u n c h a n g e d . H o w e v e r, th e p h ase sp e c tru m is
ch an g ed by an a m o u n t -cok. T his re su lt can easily b e e x p la in e d if w e recall th a t
th e fre q u e n c y c o n te n t o f a signal d e p e n d s only on its sh a p e. F ro m a m a th e m a tic a l
p o in t o f view , w e can say th a t shifting by k in th e tim e d o m a in , is e q u iv a le n t to
m u ltip ly in g th e sp e c tru m by e~i'ok in th e fre q u e n c y d o m ain .
T im e r e v e r s a f .

If
X(co)

j:()
th e n
x(-n)

X ( - a >)

(4.3.48)

T h is p ro p e rty can be e s tab lish ed by p e rfo rm in g th e F o u rie r tra n sfo rm a tio n

o f jt ( n) an d m ak in g a sim ple ch an g e in th e su m m a tio n in d ex . T h u s
OC
F (x (-n )j =

22 x ( l '>e>Wi =
f= OC

If x ( n ) is real, th e n from (4.3.17) an d (4.3.18) w e o b ta in

F ( * ( - n ) | = X{-a>) =
= \ X( w) \ e ~ / ^ XiM'
T h is m e a n s th a t if a signal is folded a b o u t th e origin in tim e, its m a g n itu d e sp e ctru m
re m a in s u n c h a n g e d , a n d th e p h a se sp e c tru m u n d e rg o e s a c h a n g e in sign (p h ase
re v e rsa l).
C o n v o lu tio n t h e o r e m .

If
F
x\ (n) 4 X ](w)

an d
x 2 (n)

X 2 {oj)

th e n
x (n ) = Xi (h) * X2 (n)

X(co) = X] (a))X 2 (&>)

(4.3.49)

T o p ro v e (4.3.49), w e recall th e co n v o lu tio n fo rm u la

OC
x ( n ) = ^ i(n ) * x 2 (n) = Y
x \ ( k ) x 2 (n - k)
k=oc

B y m u ltip ly in g b o th sides o f this e q u a tio n by th e e x p o n e n tia l e x p ( jeon) a n d

su m m in g o v e r all n, w e o b ta in
OC
X(a>)= ^
x ( n ) e ~ Jwn =

OC
^

" OC
^
x \ { k ) x 2(n k)

e~]a>n

298

Frequency Analysis of Signals and Systems

Chap. 4

A fte r in te rc h a n g in g th e o rd e r o f th e su m m a tio n s a n d m ak in g a sim ple ch an g e in

th e su m m atio n in d ex , th e rig h t-h a n d side o f this e q u a tio n re d u c e s to th e p ro d u ct
Xi(a>)X 2 (io). T h u s (4.3.49) is estab lish ed .
T h e co n v o lu tio n th e o re m is o n e o f th e m o st p o w erfu l to o ls in lin e a r system s
analysis. T h a t is, if w e co nvolve tw o signals in th e tim e d o m a in , th e n this is
e q u iv a le n t to m u ltip ly in g th e ir sp e c tra in th e fre q u e n c y d o m ain . In la te r c h ap ters
we will see th a t th e co n v o lu tio n th e o re m p ro v id es an im p o rta n t c o m p u tatio n al
to o l fo r m an y d ig ital signal p ro cessin g ap p licatio n s.
Example 4.3.4
By use of (4.3.49). determ ine the convolution of the sequences
A'l(/1) = A:(h ) = {1. 1. 1)

t
Solution

By using (4.3.21). we obtain

X i ((o) = X 2(w ) = 1 + 2 cos w

Then
X (co) = A't (cijjASfa)) = (1 + 2coso>)
= 3 + 4 cos w -i- 2 cos ho
= 3 4- 2 (c"" + ( "") + ic' 2"' +
Hence the convolution of

with

is

.v (ii) = ( 1 2 3 2 1 )

Figure 4.34 illustrates the foregoing relationships.

The correlation theorem.

If
x i(n )

X \(a>)

A;( )

Xzito))

an d

th en
ri,x2(m )

S ,; (w) = X \ ( c d) X 2 (oj)

(4.3.50)

T h e p ro o f o f (4.3.50) is sim ilar to th e p ro o f o f (4.3.49). In th is case, w e have

X
r^xAn) =

x ] ( k ) x 2( k - n )

k=~-'X.
B y m u ltip ly in g b o th sides o f this e q u a tio n by th e e x p o n e n tia l ex p (jeon) and
su m m in g o v er all n , w e o b ta in
OC

Sxll2(w) =

oc

x i ( k )x 2(k - n )

e ~ JmR

X2M

Figure 4 3 4

Graphical representation of the convolution property.

F in ally , w e in te rc h a n g e th e o rd e r o f th e su m m a tio n s an d m a k e a ch an g e in th e
su m m a tio n in d ex . T h u s w e find th a t th e rig h t-h a n d sid e o f th e e q u a tio n above
re d u c e s to X \ ( u ) ) X 2 ( a>). T h e fu n ctio n SXlX2(a)) is called th e cross-energy density
s pe c t r um o f th e sig n a ls x\ (n) an d x 2 (n).

The W iener-Khintchine theorem.

rxz(l)

L e t jc(n) be a re a l signal. T h e n
s(a)

(4-3.51)

T h a t is, th e e n e rg y sp e c tra l d en sity o f an en erg y signal is th e F o u rie r tra n sfo rm o f

its a u to c o rre la tio n se q u e n c e . T h is is a sp ecial case o f (4.3.50).
T h is is a v ery im p o rta n t resu lt. It m ean s th a t th e a u to c o rre la tio n se q u e n c e
o f a sig n al a n d its en e rg y sp e c tra l d e n sity c o n tain th e sam e in fo rm a tio n a b o u t th e
signal. S ince, n e ith e r o f th e s e c o n ta in s any p h ase in fo rm a tio n , it is im p o ssib le to
u n iq u e ly re c o n stru c t th e signal fro m th e a u to c o rre la tio n fu n ctio n o r th e en erg y
d e n sity sp e ctru m .

Example 4JJ
D eterm ine the energy density spectrum of the signal
x(n) = tf^ufn)

- 1<a < 1

300

Chap. 4

Frequency Analysts of Signals and Systems

Solution
signal is

From Example 2.6.2 we found that the autocorrelation function for this

f a il)

= --------- oc < / < oo

1 a*

By using the result in (4.3.46) for the Fourier transform of a 1'1, derived in Exam
ple 4.3.3. we have
= ----- ----- - 1 2a cos w + a-

1 0

Thus, according to the W iener-K hintchine theorem ,

1
1 2a cos w + a 2

Sxx(w) =

Frequency shifting.

If
x( n) < F > X( u) )

th en
e i<onnx ( n )

X(cu-m )

(4.3.52)

T h is p ro p e rty is easily p ro v e d by d ire c t s u b s titu tio n in to th e analysis e q u a tio n

(4.3.1). A cco rd in g to this p ro p e rty , m u ltip lic a tio n o f a se q u e n c e x( n) by e iu>-)n is
eq u iv a le n t to a fre q u e n c y tra n sla tio n o f th e sp e c tru m X(co) by co<>. T h is freq u e n cy
tra n sla tio n is illu stra te d in Fig. 4.35. S ince th e s p e c tru m X(co) is p e rio d ic, th e shift
a\) ap p lies to th e sp e c tru m o f th e signal in every p erio d .

The modulation theorem.

If
x( n) < F > X (a>)

X(u>)

_2

1
2

(a)

X( uj-

- 2w

- 2* + wo

cjq)

2i

2 + cjo

(b)
Figure 4 3 5
form.

Illustration of the frequency-shifting property of the Fourier trans

Sec. 4.3

Properties o f the Fourier Transform for Discrete-Time Signals

301

th e n
+ cuo) + X(w c^o)]

x ( n ) cos coqt i

(4.3.53)

T o p ro v e th e m o d u la tio n th e o re m , w e first ex p ress th e signal coswo'J as

+ e - J^'")

cos won =

U p o n m u ltip ly in g x (n ) by th e se tw o e x p o n e n tia ls a n d using th e freq u e n cy -sh iftin g

p ro p e rty d e sc rib e d in th e p re c e d in g sectio n , w e o b ta in th e d e sire d re su lt in (4.3.53).
A lth o u g h th e p ro p e rty given in (4.3.52) can also be v iew ed as (com plex)
m o d u la tio n , in p ractice w e p re fe r to use (4.3.53) b ecau se th e signal j (h ) c o s a ^ ?
is real. C learly , in th is case th e sy m m etry p ro p e rtie s (4.3.12) and (4.3.13) are
p re se rv e d .
T h e m o d u la tio n th e o re m is illu stra te d in Fig. 4.36, w hich c o n ta in s a plot of
th e sp e c tra o f th e signals jr(/;). vi(n) = ;c (n )c o s 0 .5 ;rn an d y ifn ) = .* ( )cos7n;.

2
(a)

(b)

2
(c)

Figure 4.36

G raphical representation of the m odulation theorem .

302

Frequency Analysis of Signals and Systems

Parsevals theorem.

Chap. 4

If
jri(n)

an d
X 2 (a>)

X2 (n)
th en
oc

/ ' ,7

xi(n)jL*(n) = - /

X\((i>)X*

(4.3.54)

2jr J - *

T o p ro v e this th e o re m , w e use (4.3.1) to elim in a te X\(<o) on th e rig h t-h an d

side o f (4.3.54). T h u s we h ave

h i

T ,

X](n)e'

X^i^dto
x
=

22

n= -3 c

^
( w ) e ~ Jujnda> =

*' 2n-

22
n ~ oc

In th e special case w h ere x 2 (n) = X](/i) = x ( n) , P a rse v a ls re la tio n (4.3.54)

red u ces to

yx

k ( )| 2 = r

/f

2 TT J 2,

(4.3.55)

\X (a > )\2dw

W e o b se rv e th a t th e le ft-h a n d side o f (4.3.55) is sim ply th e e n e rg y E x o f th e signal

x(fj). It is also e q u a l to th e a u to c o rre la tio n o f * ( ), rxx(l), e v a lu a te d a t / = 0.
T h e in te g ra n d in th e rig h t-h a n d side o f (4.3.55) is e q u a l to th e en erg y density
sp e c tru m , so th e in te g ra l o v e r th e in terv al it < a> < it y ields th e to ta l signal
en erg y . T h e re fo re , w e co n clu d e th a t
E x = r xA ) =

i
\x ( n )\2 =
2jt

J2n\X(a))\ 2d w = 2it

cn
Ss x (w)da>

Multiplication of two sequences (Windowing theorem).

(4.3.56)

If

x\ ( n) < Xi(cu)
an d
x 2 (n) < X 2 (co)

then
f
i
x j i n ) = x \ { n ) x 2 {n) * X j ( w) =

r
/ X] ( k ) X 2 (a) - k ) d k
J_n

(4.3.57)

Sec. 4.3

Properties of the Fourier Transform f o r Discrete-Time Signals

303

T h e in te g ra l on th e rig h t-h a n d side of (4.3.57) re p re se n ts th e co n v o lu tio n of the

F o u rie r tra n sfo rm s Xi(cl>) a n d X 2(w). T h is re la tio n is th e d u a l o f th e tim e-d o m ain
c o n v o lu tio n . In o th e r w ords, th e m u ltip lic atio n o f tw o tim e -d o m a in se q u en ces is
e q u iv a le n t to th e co n v o lu tio n of th e ir F o u rie r tran sfo rm s. O n th e o th e r h a n d , the
co n v o lu tio n o f tw o tim e -d o m a in se q u e n c e s is e q u iv a le n t to th e m u ltip lic a tio n o f
th e ir F o u rie r tran sfo rm s.
T o p ro v e (4.3.57) we b egin w ith th e F o u rie r tra n sfo rm o f
(n) = jci(rt)x 2(n)
an d u se th e fo rm u la fo r th e in v erse tra n sfo rm , nam ely,

T h u s, w e h av e
CC

X3(oj) =

CC

x 3(n)e Jwn =

X\(k)

2 2 x ' ( n )x 2 (n )e

k)d k

T h e co n v o lu tio n in teg ral in (4.3.57) is kn o w n as th e per i od i c convol ut i o n o f

Xi(o>) a n d X 2 (to) b e c a u se it is th e co n v o lu tio n o f tw o p e rio d ic fu n ctio n s h av in g th e
sam e p e rio d . W e n o te th a t th e lim its o f in te g ra tio n e x te n d o v e r a single p erio d .
F u rth e rm o re , w e n o te th a t d u e to th e p erio d icity o f th e F o u rie r tra n sfo rm fo r
d isc re te -tim e signals, th e re is no p e rfe c t" d u ality b e tw e e n th e tim e an d freq u e n c y
d o m a in s w ith re sp e c t to th e c o n v o lu tio n o p e ra tio n , as in th e case o f c o n tin u o u s
tim e signals. In d e e d , co n v o lu tio n in th e tim e d o m ain (a p e rio d ic su m m a tio n ) is
e q u iv a le n t to m u ltip lic a tio n o f co n tin u o u s p e rio d ic F o u rie r tra n sfo rm s. H o w ev er,
m u ltip lic a tio n o f a p e rio d ic se q u e n c e s is e q u iv a le n t to p e rio d ic co n v o lu tio n o f th e ir
F o u rie r tra n sfo rm s.
T h e F o u rie r tra n sfo rm p a ir in (4.3.57) will p ro v e u se fu l in o u r tr e a tm e n t of
F IR filter d esign b a se d on th e w indow te c h n iq u e .

Differentiation in the frequency domain.

If

x( n) X(a>)

then
f

nx(n)

. ii X( ( o)
da)

(4.3.58)

304

Frequency Analysis of Signals and System s

Chap. 4

T o p ro v e this p ro p e rty , we use th e d efin itio n o f the F o u rie r tra n sfo rm in

(4.3.1) an d d iffe re n tia te th e series te rm by te rm w ith resp ec t to w. T h u s we
o b ta in
d X ( u>)

doj

du>

7 2 n x ( n ) e Ju

N ow w e m u ltip ly b o th sides of th e e q u a tio n bv j to o b ta in th e d esired resu lt in

(4.3.58).
T h e p ro p e rtie s d eriv ed in this section are su m m a riz e d in T a b le 4.5, which
serv es as a co n v e n ie n t refe ren ce . T ab le 4.6 illu stra te s som e useful F o u rie r tran s
form p a irs th a t will be e n c o u n te re d in la te r c h a p te rs.

TABLE 4.5

PR O P E R TIE S OF TH E FO U R IE R T R A N S F O R M FOR D IS C R E TE -TIM E

S IG N ALS

Property
Notation

Linearity
Time shifting
Time reversal
Convolution
C orrelation

W iener-Khm tchine
theorem
Frequency shifting
M odulation

Time Domain

r (/)

A'(co)
X](w)
A'iUu)
a\X](a)) + 02 X 2 (01)
e_ ""l X()
X ( a>)
Xi(u>)X;(oj)
5,,.,, iw) = X, ( w)X2(-to)
- A j (ul>)X ,(iu)
[if X2 (h) is real]
-S.t.r (w)

eja,0"x(n)
x(n) cosa>nr

X( w o^j)
\ X ( w + tut,) + i X ( w - wn)

,v()
A](ii)
x2(n)
a\X] (n) +
x(rt ~ k)
x(~n)
x \ { n ) * x 2 (n)
r llt,(/) = ^, (/) * j z( - / )

Multiplication
D ifferentiation in the
frequency domain
Conjugation

Frequency Domain

1 f
I X |(/.)X (w X)dk
2jr
d X (u)
du>

J-n

nx(n)
x*{n)

X ( - w)
X\ ( w) X*(io)doj

Sec. 4.4
TABLE 4.6

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

305

SO M E U S EFU L F O U R IE R TR A N S FO R M PAIRS FO R D IS C R E TE -TIM E A P ER IO D IC

SIGNALS

4.4 FREQUENCY-DOMAIN CHARACTERISTICS OF LINEAR

TIME-INVARIANT SYSTEMS
In th is se ctio n w e d e v e lo p th e c h a ra c te riz a tio n o f lin e a r tim e -in v a ria n t system s in
th e fre q u e n c y d o m a in . T h e basic e x c ita tio n signals in th is d e v e lo p m e n t a re th e
co m p lex e x p o n e n tia ls a n d sin u so id al fu n ctio n s. T h e c h a ra c te ristic s o f th e system
a re d e s c rib e d by a fu n ctio n o f th e fre q u e n c y v a ria b le co called th e fre q u e n c y r e
sp o n se, w hich is th e F o u rie r tra n sfo rm o f th e im p u lse re sp o n s e h( n) o f th e system .
T h e fre q u e n c y re sp o n s e fu n ctio n c o m p le te ly c h a ra c te riz e s a lin e a r tim ein v a ria n t sy stem in th e fre q u e n c y d o m a in . T h is allow s us to d e te rm in e th e

306

Frequency Analysis of Signals and System s

Chap. 4

ste a d y -s ta te re sp o n se o f th e system to an y a rb itra ry w eig h ted lin e a r co m b in atio n

o f sin u so id s o r co m p lex e x p o n en tials. Since p e rio d ic se q u e n c e s, in p a rtic u la r, lend
th em se lv e s to a F o u rie r series d e c o m p o sitio n as a w eig h te d sum o f h a rm o n ically re
lated co m p lex e x p o n e n tia ls, it b eco m es a sim ple m a tte r to d e te rm in e th e response
o f a lin e a r tim e -in v a ria n t system to this class o f signals. T his m e th o d o lo g y is also
a p p lied to a p e rio d ic signals since such signals can b e view ed as a su p e rp o s itio n of
infin itesim al size co m p lex ex p o n en tials.

4.4.1 Response to Complex Exponential and Sinusoidal

Signals: The Frequency Response Function
In C h a p te r 2, it w as d e m o n s tra te d th a t th e re sp o n se o f any re la x e d lin e a r tim ein v arian t sy stem to an a rb itra ry in p u t signal jr(n), is given by th e c o n v o lu tio n sum
fo rm u la
X

yin)-

ft(k)x(n-k)

(4.4.1)

OC
In th is in p u t- o u tp u t re la tio n sh ip , th e sy stem is c h a ra c te riz e d in th e tim e dom ain
by its u n it sam p le re sp o n s e {h ( n ). - o c < n < oo}.
T o d ev e lo p a fre q u e n c y -d o m a in c h a ra c te riz a tio n o f th e sy stem , let us excite
th e sy stem w ith th e co m p lex e x p o n e n tia l
00 < n < cc

jr(n) = A e Jam

(4.4.2)

w h ere A is th e a m p litu d e and a> is an y a rb itra ry fre q u e n c y c o n fin ed to th e freq u en cy

in terv al [ - t t . j t ] . By su b stitu tin g (4.4.2) in to (4.4.1), w e o b ta in th e re sp o n se
v(n) =

22

h ( k ) [ A e Jb,in- k}]

k~ ~ X OC
= A

(4.4.3)

2 2 h ( k ) e ~ Juk

W e o b se rv e th a t th e te rm in b ra c k e ts in (4.4.3) is a fu n ctio n o f th e freq u en cy

v ariab le a>. In fact, th is te rm is th e F o u rie r tra n sfo rm o f th e u n it sa m p le resp o n se
h{k) o f th e system . H e n c e w e d e n o te th is fu n ctio n as

00
H(w)=

22

h ( k ) e ~ ja>k

(4.4.4)

C learly , th e fu n ctio n H(co) exists if th e sy stem is B IB O sta b le , th a t is, if

OC
y

|/i(rc)| <

00

n =-oc

W ith th e d efin itio n in (4.4.4), th e re sp o n se o f th e system to th e com plex

e x p o n e n tia l giv en in (4.4.2) is
y(n) = AH(oo)eJwn

(4.4.5)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

307

W e n o te th a t th e re sp o n se is also in th e fo rm o f a co m p lex e x p o n e n tia l w ith the

sam e fre q u e n c y as th e in p u t, b u t a lte re d by th e m u ltip lic ativ e fa c to r H ( m ).
A s a re su lt o f th is c h a ra c te ristic b e h a v io r, th e e x p o n e n tia l signal in (4.4.2) is
called an ei genf unct i on o f th e system . In o th e r w o rd s, an eig e n fu n c tio n o f a system
is an in p u t signal th a t p ro d u c e s an o u tp u t th a t differs fro m th e in p u t by a c o n s ta n t
m u ltip lic ativ e facto r. T h e m u ltip lic ativ e fa c to r is called an ei genval ue o f th e system .
In th is case, a c o m p lex e x p o n e n tia l signal o f th e fo rm (4.4.2) is an e ig en fu n ctio n of
a lin e a r tim e -in v a ria n t system , a n d H(u>) e v a lu a te d at th e fre q u e n c y o f the in p u t
signal is th e c o rre sp o n d in g eig en v alu e.
Example 4.4.1
D eterm ine the output sequence of the system with impulse response
(4.4.6)

k{n) = (iY u(/i)

when the input is the complex exponential sequence
xin) = Ac^"' -

oo < n < oc

Solution First we evaluate the Fourier transform of the impulse response hin), and
then we use (4.4.5) to determ ine v(n). From Example 4.2.3 we recall that
(4.4.7 >

At ui = 7t/2, (4.4.7) yields

and therefore the output is

(4.4.8)
oc < n < oc

T h is ex a m p le clearly illu stra te s th a t th e only effe ct o f th e system on th e in p u t

signal is to scale th e a m p litu d e by 2 /\/5 a n d shift th e p h ase by - 2 6 .6 . T h u s th e
o u tp u t is also a co m p lex e x p o n e n tia l o f fre q u e n c y n / 2 , a m p litu d e 2 A / v /5. an d
p h a se - 2 6 .6 C.
If w e a lte r th e fre q u e n c y of th e in p u t signal, th e effe ct o f th e system on
th e in p u t also c h an g es a n d h en c e th e o u tp u t ch an g es. In p a rtic u la r, if th e in p u t
se q u e n c e is a c o m p lex e x p o n e n tia l o f fre q u e n c y tt. th a t is,
x(n) = A e ^ n

oo < n < oc

then, at co = jr.
1

2
3

(4.4.9)

308

Frequency Analysts of Signals and Systems

Chap. 4

an d th e o u tp u t o f th e sy stem is
v(n) = 5 A e jnn

00 < n < 00

(4.4.10)

W e n o te th a t H ( n ) is p u re ly real [i.e., th e p h a se a s so ciated w ith H{u>) is zero at

co t t ] . H e n c e , th e in p u t is scaled in a m p litu d e by th e fa c to r H ( n ) = =, b u t the
p h ase shift is zero .
In g e n e r a l H(co) is a co m p lex -v alu ed fu n ctio n o f th e fre q u e n c y variable w.
H e n c e it can b e e x p ressed in p o la r form as
H(to) = \H(a>)\eJH{w)

(4.4.11)

w h ere |//(a> )| is th e m ag n itu d e o f H(a>) and

@(w) ^ H (c o)

w hich is th e p h a se shift im p a rte d on th e in p u t signal by the system at th e fre

quen cy CO.
Since H( w) is th e F o u rie r tra n sfo rm o f i/i()}, it follow s th a t H(to) is a peri
od ic fu n ctio n w ith p e rio d 2 n . F u rth e rm o re , w e can view (4.4.4) as th e ex ponential
F o u rie r series ex p an sio n for H(co), w ith h( k) as th e F o u rie r se rie s coefficients. C on
se q u e n tly , th e u n it im p u lse h( k) is re la te d to H(co) th ro u g h th e in te g ra l expression
1

h( k) =

H( t o) elwkdco

(4.4.12)

F o r a lin ear tim e -in v a ria n t system w ith a re a l-v a lu e d im p u lse resp o n se, the
m ag n itu d e an d p h a s e fu n ctio n s possess sy m m etry p ro p e rtie s w h ich are d eveloped
as follow s. F ro m th e d efin itio n o f H(co). w e have
H(co) =

h( k)e'

52

ju / k

h(k) coscok j 5 2

kDc

h (jt)s in w

(4 4 13)

k=oc

= H R{io) -f j H 1 (to)
= y j H l i t o ) + H f ( c o ) e J tan- 1["/<")/"*<-)]

w h ere H K(to) an d Hi(co) d e n o te th e real a n d im ag in aary c o m p o n e n ts o f H(a>), d e

fined as
DC

Hn(co) =

52

h ( k ) c os c ok
(4.4.14)

H/(co) = 5 2 h( k) sin cok

k = -x

It is c lear from (4.4.12) th a t th e m a g n itu d e an d p h a s e o f H(co), ex p re sse d in term s

o f H r (co) an d Hi(co), are
\H(co)\ = J H 2r(co) + Hf(co)

(4.4.15)
(w ) = tan

--------H R(to)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

309

W e n o te th a t H R (a>) = H R( - t o ) a n d Hi {a>) = / / / ( oj). so th a t H r ( c o ) is an

ev en fu n c tio n o f oj a n d Hi (co) is an o d d fu n ctio n o f w. A s a c o n s e q u e n c e , it follow s
th a t
is an ev en fu n ctio n of w an d 0(o>) is an o d d fu n c tio n o f a>. H e n c e ,
if w e k n o w j / f ( a > ) J a n d 0 ( a > ) fo r 0 < w < n . w e also k n o w th e s e fu n c tio n s for
jr < w < 0.
Example 4.4.2

Moving Average Filter

D eterm ine the m agnitude and phase of H(w) for the three-point moving average
(MA) system
y ( n ) = ^fjc(n + 1) + x( n ) + x( n - 1)]
and plot these two functions for 0 < a> < n.
Solution

Since
h{n) = [ i i i }
t

it follows that
H(co) =

1 +

1 ( 1

2cos<w)

Hence
i 11 + 2 cos a)

IW M I

' 0.
B ( w )

<

(4.4.16)
a> <

2 jt / 3

<

2?r/3
w

<

jt

Figure 4.37 illustrates the graphs of the m agnitude and phase of H(w). As indicated
previously, |W(w)| is an even function of frequency and C-)(a>) is an odd function of

3tt
4
Figure 4.37

Magnitude and phase

responses for the M A system in

Example 4.4.2.

310

Frequency Analysis of Signals and Systems

Chap. 4

frequency. It is apparent from the frequency response characteristic H(u>) that this
moving average filter smooths the input data, as we would expect from the inputoutput equation.
T h e sy m m etry p ro p e rtie s satisfied by th e m a g n itu d e a n d p h a se fu n ctio n s of
H ( u >), an d th e fact th a t a sinusoid can be e x p re sse d as a su m o r d ifferen ce of
tw o co m p lex -co n ju g ate e x p o n e n tia l fu n ctio n s, im ply th at th e re sp o n s e o f a linear
tim e -in v a ria n t sy stem to a sin u so id is sim ilar in fo rm to th e re sp o n se w hen the
in p u t is a co m p lex e x p o n e n tia l. In d e e d , if th e in p u t is
Jfi (n) Ae-' w'

th e o u tp u t is
vi() = A \ H( o j ) \ e j<rnw' eilM'
O n th e o th e r h a n d , if th e in p u t is
j::( )

th e re sp o n se o f th e system is
y : ( / j) =

A\H{~

A \ H (aj)\e~j H ""' e ~J'l>"

w h ere, in th e last ex p ressio n , w e h av e m a d e use of the sy m m etry p ro p e rtie s

\H(to)\ = \ H ( cd)\ an d (~)(a;) = (-)(co). N ow , by ap p ly in g the su p e rp o sitio n
p ro p e rty o f th e lin e a r tim e -in v a ria n t system , w e find th at th e re sp o n se o f th e sys
tem to th e in p u t
i[jt] (n) + _V2 (/;)] = A cos con
is
}(>0 = j[ v , (n) + \2(/r)]
(4.4.17)
y i n ) = A \ H( c o) \ cosjw/? + (-)(o>)]
Sim ilarly, if th e in p u t is
x i n ) = r[jf |(n ) jt-*(/t) 1 = A sin ton
J2
th e resp o n se o f th e system is
v W = i [ v l( ,, )

(4 4 1 8 )

v(h) = A \ H { w ) \ sin[cu + 0 (tn )]

I t is a p p a r e n t fro m th is discussion th a t H{co), o r e q u iv a le n tly . |//(c u )| and
0(cw), co m p le te ly c h a ra c te riz e the effect o f th e system on a sin u so id a l in p u t signal
o f any a rb itra ry freq u e n cy . In d e e d , w e n o te th a t \H(a>)\ d e te rm in e s th e am plifi
catio n ( | / / M | > 1) o r a tte n u a tio n (\H(co)\ < 1) im p a rte d by th e system o n the
in p u t sin u so id . T h e p h a se @(co) d e te rm in e s th e a m o u n t o f p h a s e shift im p a rte d

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

311

by th e sy stem o n th e in p u t sinusoid. C o n se q u e n tly , by k n o w in g H{aj), w e are

ab le to d e te rm in e th e re sp o n s e o f the system to an y sin u so id a l in p u t signal. Since
H ( w ) specifies th e re sp o n s e o f th e sy stem in th e fre q u e n c y d o m a in , it is called th e
f r e q u e n c y response o f th e system . C o rre sp o n d in g ly , \H{to) \ is called th e ma g ni t ude
respons e a n d 0(o>) is c a lle d th e phas e response o f th e system .
I f th e in p u t to th e sy stem co n sists o f m o re th a n o n e sin u so id , th e s u p e rp o
sitio n p ro p e rty o f th e lin e a r system can b e u se d to d e te rm in e th e re sp o n se . T h e
fo llo w in g e x a m p le s illu stra te th e use o f th e su p e rp o s itio n p ro p e rty .
Exam ple 4.43
D eterm ine the response of the system in Example 4.4.1 to the input signal
;c(n) = 10 5 sin n + 2 0 co snn

oc < n < oc

Solution

The frequency response of the system is given in (4.4.7) as

H(w) = ------* 1-

The first term in the input signal is a fixed signal com ponent corresponding to w = 0.
Thus
H( 0) =
The second term in
response of the system is

=2

has a frequency jr/2.

At this frequency the frequency

Finally, the third term in x ( n ) has a frequency w =

H { tt) =

tt,

At this frequency

Hence the response of the system to *(n) is

10 . / jt
\
40
y (n ) = 2 0
j= . sm ^ 2^ 2 6 .6 J + co sttw

o c < n < oc

Example 4.4.4
A linear tim e-invariant system is described by the following difference equation:
y(n) = ay(n 1) + bx(n)

0 < a < 1

(a) D eterm ine the magnitude and phase of the frequency response H(w) of the
system,
(b ) Choose the param eter b so that the maximum value of \H{u>)\ is unity, and

sketch fH(<d)\ and 4 . H( w) for a = 0,9.

Frequency Analysis of Signals and Systems

312

Chap. 4

(c) D eterm ine the output of the system to the input signal
x (n )

Solution

+ 12 sin

2 0 cos

(^ z n

+ ^

The impulse response of the system is

h(n) = ba"u(n)
Since |a| < 1. the system is BIBO stable and hence H(co) exists,
(a) The frequency response is
oc
H (w) =

22, h ^

e ~iam

b
1 ae~JW
Since

1 ae~, = (1 a cos co) 4- j a sin co

it follows that
[1 ae J'| = ^ /(l a cos co)2 4- (a sin co)2

= y/\ + a2 2a cos co
and
4 (1 - ae J) = tan

osinct)
1 a cos co

T herefore,
\b\

\H(co)\ =

V l 4- a2 2a cos 1u
^H(co) = &(co) = 4-b - tan 1

a sin co
1 a cos (

(b) Since the param eter a is positive, the denom inator of \H(w)\ attains a minimum
at w = 0. Therefore, |//(a>)| attains its maximum value at a> = 0. A t this
frequency we have
1*1
|ff(0)| = r U - = ' l
1 a
which implies that b = (1 a). We choose b = 1 a, so that
l o
|W M I -

J\ + a 2 2a cos

and
&(w) = tan"

asinoj
1 a cos co

The frequency response plots for |tf(co)| and 0(o>) are illustrated in
Fig. 4.38. W e observe that this system attenuates high frequency signals.

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

313

Figure 4.38

Magnitude and
phase responses for the system in
Example 4.4.4 with a =

(c) The input signal consists of components of frequencies to = 0. tt/2, and n. For
co = 0. |//(0)| = 1 and 0 (0) = 0. For to = jr/2,
1 a
(-)l =
_________
V2 / 1
y/\ -f a 2

0.1
-

______

= 0.074

-s/1.81

-- - tan-1 a = - 4 2

For co = tt,

1 -a
0.1
\H(n)\ = = = 0.053
l+ o
1.9
0(7r) = 0
Therefore, the output of the system is
y(n ) = 5|ff(0)| + 1 2 | / / ( | ) | s i i i | j + e ( ! ) ]
2 0 |/ / ( jt ) |

= 5 + 0.888sin

cos

^ 7 rn + + 0 ( 7 r ) j

42^ 1.06 cos ^jrn + ^

oc < n < oc

314

Frequency Analysis ol Signals and Systems

Chap. 4

In th e m o st g e n e ra l case, if th e in p u t to th e system co n sists o f an arbitrary

lin ear co m b in a tio n o f sinusoids o f th e form
L

x ( n ) = 5 2 A, cos(co,n 4 4>i)

oc < < oc

1=1
w h ere (A, | an d {<,} a re th e am p litu d e s a n d p h ases o f th e c o rre sp o n d in g sinusoidal
c o m p o n e n ts, th e n th e re sp o n s e of th e sy stem is sim ply
L

v{n) = 5 2 Aj \ H{ a),)| cos[oj, 4- <p, 4- 0(ci>,)]

(4.4.19)

1= 1

w h ere \H(u>i)\ an d (oj,) are th e m a g n itu d e an d p h a se , re sp e c tiv e ly , im p a rte d by

th e system to th e in d iv id u al fre q u e n c y c o m p o n e n ts o f th e in p u t signal.
It is clear th a t d e p e n d in g on th e fre q u e n c y re sp o n se H ( uj) o f th e system , input
sinu soid s o f d iffe re n t fre q u e n c ie s will be affe c te d d ifferen tly by th e system . F o r ex
am p le, so m e sin u so id s m ay be co m p letely su p p re sse d by th e sy stem if H ( w ) = 0 at
th e fre q u e n c ie s o f th e s e sinusoids. O th e r sin u so id s m ay receiv e n o a tte n u a tio n (or
p e rh a p s, som e am p lifica tio n ) by th e system . In effect, w e can view th e lin ear timein v arian t system fu n c tio n in g as a filter to sin u so id s o f d iffe re n t fre q u e n c ie s, passing
som e o f th e fre q u e n c y c o m p o n e n ts to th e o u tp u t a n d su p p re s sin g o r preventing
o th e r freq u e n cy c o m p o n e n ts fro m re ach in g th e o u tp u t. In fact, as discussed in
C h a p te r 8, th e basic dig ital filter design p ro b le m involves d e te rm in in g th e p a ra m e
ters o f a lin e a r tim e -in v a ria n t system to achieve a d e sire d fre q u e n c y re sp o n s e H (co).

4.4.2 Steady-State and Transient Response to Sinusoidal

Input Signals
In th e discu ssio n in th e p re ced in g sectio n , w e d e te rm in e d th e re s p o n s e o f a linear
tim e -in v a ria n t sy stem to e x p o n e n tia l a n d sin u so id al in p u t sig n als a p p lied to the
system at n = oc. W e usually call such signals e te rn a l e x p o n e n tia ls o r etern al
sin u so id s, b ecau se th e y w ere a p p lied at n = - o c . In such a case, th e re sp o n se that
w e o b se rv e a t th e o u tp u t o f th e system is th e ste a d y -s ta te re s p o n s e . T h e re is no
tra n sie n t re sp o n se in th is case.
O n th e o th e r h a n d , if th e e x p o n e n tia l o r sin u so id al signal is a p p lie d a t som e
finite tim e in sta n t, say at n = 0, th e re sp o n se of th e sy stem co n sists of tw o term s, the
tra n sie n t re sp o n se a n d the ste a d y -sta te re sp o n se . T o d e m o n s tra te this b ehavior,
let us co n sid er, as an e x am p le, th e sy stem d e sc rib e d by th e firs t-o rd e r d ifferen ce
e q u a tio n
y( n) = av( n 1 )4- j:(/i)

(4.4.20)

T his sy stem w as c o n s id e re d in S ectio n 2.4.2. Its re sp o n se to an y in p u t x ( n) applied

at n = 0 is given by (2.4.8) as
y( n) = a n+ly ( - l ) + J 2 a kx ( n - k)

*=0
w h ere y ( 1) is th e in itial co n d itio n .

n> 0

(4.4.21)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

315

N o w , let us assu m e th a t th e in p u t to th e system is th e c o m p le x e x p o n e n tia l

x ( n ) - A e Jan

n> 0

(4.4.22)

a p p lied a t n = 0. W h e n w e s u b s titu te (4.4.22) in to (4.4.21), w e o b ta in

n
v(n) = a " +1y ( 1) +

= a

v ( 1) + A

k=0
1 _ gn+le ~juiUi +\)
= o "+1v ( - l ) + A ----- --------------------e jmn
1 -ae-J
A/jn^

(4.4.23)
n > 0
A

= an+' v ( - l ) ---------- :
---eJua + ------ - e ia*
1 a e ~ JW

1 ~ a e JW

n>0

W e recall th a t th e system in (4.4.20) is B IB O sta b le if |a | < 1 . In th is case

th e tw o te rm s in v o lv in g ^ " +1 in (4.4.23) d ecay to w a rd z e ro as n a p p ro a c h e s infinity.
C o n s e q u e n tly , w e a re left w ith th e ste a d y -s ta te re sp o n se
Vss(n) =
=

A
i
hm v() = ------------ e1
l-ae-J*

(4.4.24)

AH(a))eJwn

T h e first tw o te rm s in (4.4.23) c o n stitu te th e tra n s ie n t re s p o n s e o f th e svstem ,

th a t is,
Aan + '
I
Vlr(n) = a n+1 v ( - l ) ------- ;----------- :----- e JU,n
n > 0
(4.4.25)
1 - a e ~ JW
w hich d e c a y to w a rd z e ro as n a p p ro a c h e s infinity. T h e first te r m in th e tra n s ie n t
re sp o n se is th e z e ro -in p u t re sp o n se o f th e system an d th e se c o n d te rm is th e
tra n s ie n t p ro d u c e d by th e e x p o n e n tia l in p u t signal.
In g e n e ra l, all lin e a r tim e -in v a ria n t B IB O system s b e h a v e in a sim ilar fashion
w h e n e x c ite d by a co m p lex e x p o n e n tia l, o r by a sin u so id a t n = 0 o r at so m e o th e r
finite tim e in sta n t. T h a t is, th e tra n s ie n t re sp o n s e d ecay s to w a rd z e ro as n oo,
leav in g o n ly th e s te a d y -s ta te re sp o n s e th a t w e d e te rm in e d in th e p re c e d in g section.
In m an y p ractical a p p lic a tio n s, th e tra n s ie n t re sp o n se of th e sy ste m is u n im p o rta n t,
a n d th e r e fo r e it is u su ally ig n o re d in d e a lin g w ith th e re sp o n s e o f th e system to
sin u so id al in p u ts.

4.4.3 Steady-State Response to Periodic Input Signals

S u p p o se th a t th e in p u t to a sta b le lin e a r tim e -in v a ria n t system is a p e rio d ic signal
x ( n ) w ith fu n d a m e n ta l p e rio d N. S ince such a signal exists fro m co < n < oc,
th e to ta l re sp o n s e o f th e sy stem a t an y tim e in s ta n t n, is sim p ly e q u a l to the
s te a d y -s ta te resp o n se.

316

Frequency Analysis of Signals and Systems

Chap. 4

T o d e te rm in e th e resp o n se v(n) o f th e system , we m a k e use o f th e Fourier

se ries re p re s e n ta tio n of th e p e rio d ic signal, w hich is
A- l

jr(n) = 5 2 c ^ j27rkn/N
k=0

k = 0, 1........ N - 1

(4.4.26)

w h e re th e {c*} are th e F o u rie r se ries coefficients. N ow th e re sp o n s e of th e system

to th e co m p lex e x p o n e n tia l signal
Xk (n)

ckej27rkn/N

k=

0 , 1 ............ N

is
v* ( n ) = t \ H

k^j

fc = 0. 1.........A ' - l

(4.4.27)

w h ere
H

/ Ink \
(
J = H ( w ) L = 2, i7A-

k = 0.

........ A ' - l

B y u sin g th e su p e rp o s itio n prin cip le fo r lin e a r system s, w e o b ta in th e resp o n se of

th e system to th e p erio d ic signal x( n) in (4.4.26) as
A1
v() = 5 2

/ ?t k \
gjlxknlf*

oc < 7; < OC

(4.4.28)

T his resu lt im p lies th a t th e re sp o n se of th e system to th e p e rio d ic in p u t signal

x ( n ) is also p e rio d ic w ith th e sam e p e rio d N . T h e F o u rie r se rie s coefficients for
y( n) a re
dk = ckH ( ^ ^ j

* = 0 , 1 .........A ' - l

(4.4.29)

H e n c e , th e lin e a r system can ch an g e th e sh a p e o f th e p e rio d ic input signal by

scaling th e a m p litu d e an d shifting th e p h ase o f th e F o u rie r se rie s c o m p o n en ts, but
it d o e s n o t affe ct th e p e rio d of the p e rio d ic in p u t signal.

4.4.4 Response to Aperiodic Input Signals

T h e c o n v o lu tio n th e o re m , given in (4.3.49). p ro v id e s th e d e s ire d freq u e n cy -d o m ain
re la tio n sh ip fo r d e te rm in in g the o u tp u t of an L T I system to an ap e rio d ic finiteen erg y signal. If {jt(n)} d e n o te s th e in p u t se q u e n c e , (v(n)} d e n o te s th e o u tp u t
se q u e n c e , an d (/?()} d e n o te s th e u n it sa m p le re sp o n se o f th e system , th e n from
th e co n v o lu tio n th e o re m , w e have
Y(w) = H(a>)X(w)

(4.4.30)

w h e re Y(to), X (o j), an d H(co) are th e c o rre sp o n d in g F o u rie r tra n sfo rm s o f {v()J.

(*()}, an d {h( n) ), resp ectiv ely . F ro m th is re la tio n sh ip w e o b se rv e th a t th e sp e c
tru m o f th e o u tp u t signal is e q u a l to th e sp e c tru m o f th e in p u t signal m ultiplied
by th e fre q u e n c y resp o n se o f th e system .

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

317

If w e e x p re ss K(cu),
an d X(o>) in p o la r fo rm , th e m a g n itu d e a n d p h ase
o f th e o u tp u t signal can b e ex p re sse d as
(4.4.31)

|y M ! = I / / M I I X M 1
4-Y (o>)

(4.4.32)

+ 4 - H ( co)

w h ere |//(a> )| a n d
a re th e m a g n itu d e an d p h ase re sp o n s e s o f th e system .
B y its v ery n a tu r e , a fin ite-en erg y a p e rio d ic signal c o n ta in s a c o n tin u u m of
fre q u e n c y c o m p o n e n ts . T h e lin e a r tim e -in v a ria n t system , th ro u g h its fre q u e n c y
re sp o n s e fu n c tio n , a tte n u a te s so m e fre q u e n c y c o m p o n e n ts of th e in p u t signal an d
am p lifies o th e r fre q u e n c y co m p o n e n ts. T h u s th e system acts as a filter to th e in p u t
signal. O b s e rv a tio n o f th e g ra p h o f \H(to)\ show s w hich fre q u e n c y c o m p o n e n ts
a re am p lified a n d w h ich a re a tte n u a te d . O n th e o th e r h a n d , th e angle o f H(a>)
d e te rm in e s th e p h a s e sh ift im p a rte d in th e c o n tin u u m o f fre q u e n c y c o m p o n e n ts of
th e in p u t signal as a fu n c tio n o f freq u e n cy . If th e in p u t signal s p e c tru m is ch an g ed
by th e sy stem in an u n d e s ira b le way, we say th a t th e system h a s cau sed mag ni t ude
a n d p h a s e distortion.
W e also o b se rv e th a t the o ut put o f a linear ti me-i nvariant sy s t em c annot c o n
tain f r e q u e n c y c o m p o n e n t s that are n o t cont ai ned in the input signal. It tak es e ith e r
a lin e a r tim e* v aria n t sy stem o r a n o n lin e a r system to c re a te fre q u e n c y c o m p o n e n ts
th a t a re n o t n ecessa rily c o n ta in e d in th e in p u t signal.
F ig u re 4.39 illu stra te s th e tim e-d o m ain an d fre q u e n c y -d o m a in relatio n sh ip s
th a t can b e u se d in th e an aly sis o f B IB O -s ta b le L T I system s. W e o b se rv e th a t
in tim e -d o m a in an aly sis, w e d eal w ith th e co n v o lu tio n o f th e in p u t signal w ith
th e im p u lse re sp o n s e o f th e system to o b ta in th e o u tp u t s e q u e n c e o f th e system .
O n th e o th e r h a n d , in fre q u e n c y -d o m a in analysis, w e deal w ith th e in p u t signal
sp e c tru m X(a)) a n d th e fre q u e n c y re sp o n s e H(co) o f th e sy stem , w hich a re re la te d
th ro u g h m u ltip lic a tio n , to yield th e sp e c tru m o f th e signal a t th e o u tp u t o f th e
system .
W e c a n u se th e re la tio n in (4.4.30) to d e te rm in e th e sp e c tru m Y( u>) o f th e
o u tp u t signal. T h e n th e o u tp u t se q u e n c e {v()} can b e d e te rm in e d fro m th e in v erse
F o u rie r tra n sfo rm
(4.4.33)
H o w e v e r, th is m e th o d is se ld o m used. In ste a d , th e z -tra n s fo rm in tro d u c e d in
C h a p te r 3 is a s im p le r m e th o d fo r solving th e p ro b le m o f d e te rm in in g th e o u tp u t
se q u e n c e {y(n)}.

X(n)
X( w)

Input

Linear
time-invariant
system
hin), H(a>)

Output

v(n) = h(n)+x(n)
KM = ma>)X(a>)

F,gre 4 3 9

Tlm e' and

frequency-domain inpul-output
relationships in LTI systems.

318

Frequency Analysis of Signals and Systems

Chap. 4

L e t us r e tu rn to th e b asic in p u t-o u tp u t re la tio n in (4,4.30) a n d c o m p u te the

s q u a re d m a g n itu d e o f b o th sides. T h u s w e o b ta in
\Y(co)\2 = \H(oj)\ 2 \X(co )\2
(4.4.34)
Syy(u}) = \ H{ co)\2S xx {0>)
w h ere Sxx(oo) a n d S vv(cu) a re th e en erg y d en sity sp e c tra o f th e in p u t a n d o u tp u t
signals, resp ec tiv ely . By in te g ra tin g (4.4.34) o v e r th e fre q u e n c y ran g e ( - n , n ) , we
o b ta in th e e n e rg y o f th e o u tp u t signal as
r

(cu)da>
(4.4.35)

r
27r

J-y,

H ( co ) |2 S j x ( co) d co

Example 4.4.5
A linear time-invariant system is characterized by its impulse response
h( n ) = (y)" u (n )

Determ ine the spectrum and the energy density spectrum of the output signal when
the system is excited by the signal
*() = ( j )"()
Solution

The frequency response function of the system

OC
H(w)

1
1 - \e-^
Similarly, the input sequence U(n)) has a Fourier transform
1

X(w) =

1 - e ~ JUI

Hence the spectrum of the signal at the output of the system is

Y (co) = H( w) X( c o )

1
(1 -

1-

The corresponding energy density spectrum is

= \Y(<o)\2 = \H(o>)\2\X(o>)\2

1
( j - coso))(j| - 1 cosa>)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

319

4.4.5 Relationships Between the System Function and

the Frequency Response Function
F ro m th e d iscu ssio n in S ectio n 4.2.6 w e k now th a t if th e system fu n ctio n H( z )
co n v erg es on th e u n it circle, w e can o b ta in th e freq u e n c y re sp o n s e o f th e system
by e v a lu a tin g H ( z ) o n th e u n it circle. T h u s
(4.4.36)
In th e case w h e re H ( z ) is a ra tio n a l fu n ctio n of th e fo rm H{ z ) = B ( z ) / A ( z ) . w e have

(4.4.37)

M
(4.4.38)
]~~[U - pk? /<")
w h e re th e (fli) a n d {>*} a re real, b u t {ct } and {pk} m ay be c o m p lcx -v alu cd .
It is so m e tim e s d e s ira b le to ex p ress th e m a g n itu d e s q u a re d o f H(a>) in term s
o f H( z ) . F irst, w e n o te th a t
|f f( a O r = H{ w) H*( u>)
F o r th e ra tio n a l sy stem fu n ctio n given by (4.4.38). we have
M

ft

(4.4.39)

It fo llo w s th a t H*( w) is o b ta in e d by e v a lu a tin g H*( 1/ z*) on th e unit circle, w h ere

fo r a ra tio n a l sy stem fu n ctio n .
M
(4.4.40)
*=i
H o w e v e r, w h e n {*()} is re a l or, e q u iv alen tly , th e coefficients {a*} and {bk} a re
real, c o m p lex -v alu ed p o le s a n d z e ro s o ccu r in c o m p le x -c o n ju g a te pairs. In this

320

Frequency Analysis of Signals and Systems

case. H*{ 1/;* ) = H ( z ~ l ). C o n se q u e n tly , H'(a>) =

Chap. 4

an d

| H(co )\2 = H{oj)H*{oj) = H ( w ) H ( - o > ) =

(4.4.41)

A cc o rd in g to th e c o rre la tio n th e o re m fo r th e z -tra n sfo rm (see T a b le 3.2), the

fu n ctio n H ( z ) H ( z ~ l ) is th e z -tra n sfo rm o f th e a u to c o rre la tio n se q u e n c e {rhh( m)}
o f th e u n it sa m p le re sp o n se (/i(n)J. T h e n it follow s fro m th e W ie n e r-K h in tc h in e
th e o re m th a t \ H { w )\2 is th e F o u rie r tra n sfo rm o f
Sim ilarly, if H ( z ) = B ( z ) / A ( z ) , th e tra n sfo rm s D( z ) = B tz J B tz - 1 ) an d C( z) =
/4 (;)A (c ') are th e z-tra n sfo rm s o f th e a u to c o rre la tio n s e q u e n c e s {c/} an d \di\,
w h ere

\rhh(m)}.

N - |J i

akak+i

Q =

N < I < N

(4.4.42)

t=o
M - \t\

d, = 2 2 bkbk+i
~ M < / <M
(4.4.43)
Jk=0
Since th e system p a ra m e te rs (at) a n d {*} a re re a l valu ed , it follow s th a t c, c_/
an d di = d-i . B y using th is sy m m etry p ro p e rty , \ H( u >)\2 m ay be ex p ressed as
do + 2 2^2 dk cos kco
\H(co )\2 = ----------^ -------------(4.4.44)
ri
co 4- 2 ^ ' c'jt cos k.u)
i =1
F inally, w e n o te th a t c o s k w can be e x p re ss e d as a p o ly n o m ia l fu n ctio n of
cosoj. T h a t is,
t
cos kco = ^ / U c o s a , ) "
(4.4.45)
m=0
w h ere [j3m] a re th e coefficients in th e ex p an sio n . C o n s e q u e n tly , th e n u m e ra to r
an d d e n o m in a to r o f \H(u ))\2 can b e v iew ed as p o ly n o m ial fu n c tio n s o f coso;. T he
fo llow ing e x am p le illu stra te s th e fo reg o in g re la tio n sh ip s.
Example 4.4.6
Determ ine

for the system

y(n) = 0.1 v(n 1) + 0.2 y(n 2 ) + x (n) + x(n 1)

Solution

The system function is

14

:~ l

H(Z) ~ 1 4 0 . 1 - - 0 . 2 : - 2
and its R O C is |z| > 0.5. Hence H{ od) exists. Now
1 + z

1+ z

1 + 0 .1 Z '1 - 0 .2 z - 2

1 4- O.lz - 0.2z2

2 4- z + z 1
1.05 + 0.08(z 4- z_1) - 0.2(z~2 + z-2)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

321

By evaluating H{z)H(z~' ) on the unit circle, we obtain

2 -j- 2 cos w
\H(u>)~ = --------------------------------------1.0^ -i- 0.16 cos w 0.4 cos 2w
However, cos2w = 2cos: w 1. Consequently. | H(u>)\2 may be expressed as
2(1 cos u>)
1.45 4 0.16cosw - 0 .8 cos:
W e n o te th a t given H ( z ), it is stra ig h tfo rw a rd to d e te rm in e H ( z ~ l ) an d th e n
|//{ w ) |2. H o w e v e r, th e inverse p ro b lem o f d e te rm in in g H( z ) given 1/ / (cu)]2 or th e
c o rre sp o n d in g im p u lse resp o n se {/?())- is n o t stra ig h tfo rw a rd . Since |//(a > )l: d o es
n o t c o n tain th e p h ase in fo rm atio n in H(a>), it is n o t p o ssible to u n iq u ely d e te rm in e
H (:).
T o e la b o ra te on th e p oint, let us assum e th a t th e N p o les an d M ze ro s of
H{z) are [pk] an d {;.*). respectively. T h e c o rre sp o n d in g p o les and zero s o f H ( z ~ ])
are (1 j p^ j an d ll/c*. }, respectively. G iv en \ H( u >)\2 o r, eq u iv a le n tly . H ( z ) H ( z ~ l ). w e
can d e te rm in e d iffe re n t system fu n ctio n s H( z ) by assigning to H( z ) . a pole pt o r
its recip ro c al 1f p k . an d a z e ro zt o r its recip ro c al 1 fzk- F o r ex am p le, if A" = 2 a n d
M = 1. th e p o les an d z ero s of H l z ) H ( : _ l ) are [ p j. p z , 1/ p \ , l / p z ) an d (~ i, 1 /c i }. If
p i a n d pz are real, th e possible p o les for H(z.) a re {pi.
1. U /P i 1 /P :K i / 'i 1/ / :}.
a n d [/52. 1//>]} an d th e possible ze ro s are {~i) o r {1 / - 1 }, T h e re fo re , th e re are eight
p o ssib le ch o ices o f sy stem functions, all o f w hich re su lt in th e sam e j//(a> )|: . E ven
if we restrict th e p o les of H( z ) to be inside th e unit circle, th e re a re still tw o
d iffe re n t ch o ices for H( z ) . d e p e n d in g on w h e th e r w e pick th e zero j;i} o r { l/:.i).
T h e re fo re , we c a n n o t d e te rm in e H{z ) u n iq u ely given only th e m a g n itu d e resp o n se
\H(w)\.

4.4.6 Computation of the Frequency Response Function

In e v a lu a tin g th e m a g n itu d e re sp o n se an d th e p h a s e re sp o n se as fu n c tio n s o f f r e
q u en cy , it is c o n v e n ie n t to ex p ress H ( oj) in te rm s o f its p o le s an d zeros. H e n c e
we w rite H(co) in fa c to re d form as
M
n

H(co) =

a - z t e - i wk )

--------------------

(4.4.46)

f ] ( 1 - p ke->*)
k=1
or, e q u iv alen tly , as
M
H M = b0eJU^ - M) ------------------]"~[(eJ - p k)

(4.4.47)

Frequency Analysis of Signals and Systems

322

Chap. 4

L e t us ex p re ss th e c o m p lex -v alu ed facto rs in (4.4.47) in p o la r form as

ej a - Z k = Vk (co)ejB*M

(4.4.48)

e J0J - p k = Uk (a>)e] * k(m)

(4.4.49)

an d

w h ere
V*(w) s \eJW - z*|,

&k (co) = 4. (eJW - zk)

(4.4.50)

Uk(co) = \eJ - p kI,

= %.(eJW - p k)

(4.4.51)

an d

T h e m a g n itu d e o f H ( w ) is eq u al to th e p ro d u c t o f m a g n itu d e s of all te rm s in

(4.4.47). T h u s, u sin g (4.4.48) th ro u g h (4.4.51), w e o b ta in
Vi() VM(co)
-------
V\ (co)U2 ((o) Un(co)

|fr0l------

\H(co)\ =

(4.4.52)

since th e m a g n itu d e o f eJM(N~M) is 1.

T h e p h ase o f H(to) is th e sum o f th e p h a s e s o f th e n u m e r a to r facto rs, m i
n us th e p h ases o f th e d e n o m in a to r facto rs. T h u s, by co m b in in g (4.4.48) th ro u g h
(4.4.51), we h av e

2^ H

(co)

4-bo

co(N -

) +

0 i (co)

0T(dti)

&m(oj)
(4.4.53)

[<t>i(a>) + <I>2(^) + + $>(o>)]

T h e p h ase o f th e gain te rm
is z e ro o r jt, d e p e n d in g on w h e th e r bo is po sitiv e or
n eg ativ e. C learly , if w e k n o w th e z ero s an d th e p o les o f th e sy ste m fu n ctio n H(z),
we can ev a lu a te th e fre q u e n c y re sp o n se fro m (4.4.52) an d (4.4.53).
T h e re is a g e o m e tric in te r p re ta tio n of th e q u a n titie s a p p e a rin g in (4.4.52)
a n d (4.4.53). L e t us c o n sid e r a p o le p k a n d a z e ro z* lo c a te d a t p o in ts A a n d B
o f th e z-p lan e, as sh o w n in Fig. 4 .40(a). A ssu m e th a t w e w ish to c o m p u te H{io)
a t a specific v alu e o f fre q u e n c y co. T h e given v alu e o f co d e te rm in e s th e an g le of
ejw w ith th e p o sitiv e real axis. T h e tip o f th e v e c to r e
specifies a p o in t L o n the
u n it circle. T h e e v a lu a tio n o f th e F o u rie r tra n sfo rm fo r th e g iv en v alue of co is
e q u iv a le n t to e v a lu a tin g th e z -tra n sfo rm at th e p o in t L o f th e co m p lex p la n e . L et
us d raw th e v ecto rs A L a n d B L from th e p o le an d z e ro lo c a tio n s to th e p o in t L, at
w hich w e w ish to c o m p u te th e F o u rie r tra n sfo rm . F ro m Fig. 4 .4 0 (a) it follow s th at
CL = CA + A L
an d
CL = CB + BL
H o w ev er, C L = ej<u, C A = p k a n d C B = zk. T h u s

AL = ej* - p k

(4.4.54)

BL = e JW - z k

(4.4.55)

an d

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

323

ImU)

(a)
im(r)

Figure 4.40 Geom etric interpretation

of the contribution of a pole and a zero
to the Fourier transform (1) magnitude:
the factor V* / Vk, (2) phase; the factor

* - <t>*B y c o m b in in g th e s e re la tio n s w ith (4.4.48) a n d (4.4.49), w e o b ta in

A L = eJ<u - p k = Uk {co)eJ* kM

(4.4.56)

BL = eJ* Zk = Vk (aj)ejStUu)

(4.4.57)

T h u s Uk(a>) is th e len g th o f AL, th a t is, th e d istan c e o f th e p o le p k fro m th e p o in t

L c o rre sp o n d in g to e }w, w h e re a s Vk(cu) is th e d ista n c e o f th e z e ro z k fro m th e sam e
p o in t L. T h e p h a se s <!>*(&>) a n d Q*(a>) a re th e an g les o f th e v e c to rs A L an d BL

324

Frequency Analysis of Signals and Systems

Chap. 4

Im(c)

Pt = e>^

Zt =

mz)

Figure 4.41 A zero on the unit circle

causes |f/((u)| - 0 and w = 4-Zk- In
contrast, a pole on the unit circle results
in \H{w)\ = 0 0 at w = 4-Pt-

w ith th e p o sitiv e re a l axis, resp ec tiv ely . T h e s e g e o m e tric in te r p re ta tio n s a re show n

in Fig. 4.40(b).
G e o m e tric in te rp re ta tio n s are very useful in u n d e rs ta n d in g how th e location
o f p o le s an d ze ro s affects th e m ag n itu d e an d p h a s e o f th e F o u rie r tran sfo rm .
S u p p o se th a t a zero , say z*. an d a pole, say p k, a re o n th e u n it circle as sh o w n in
Fig. 4.41. W e n o te th a t at w = 4 z* , Vk (co) an d c o n s e q u e n tly | H(co)\ b eco m e zero.
S im ilarly, at to = 4-pk th e len g th Uk(w) b eco m es z e ro a n d h e n c e \H(co)\ b ecom es
infinite. C learly , th e e v a lu a tio n o f p h a s e in th e s e case s h a s n o m ean in g .
F ro m th is discu ssion we can easily see th a t th e p re se n c e o f a ze ro close to
th e u n it circle cau ses th e m a g n itu d e o f th e fre q u e n c y re sp o n s e , a t freq u e n cies
th a t c o rre sp o n d to p o in ts o f th e u n it circle close to th a t p o in t, to b e sm all. In
c o n tra st, th e p re se n c e o f a p o le close to th e u n it circle c au ses th e m a g n itu d e of
th e fre q u e n c y re sp o n s e to b e larg e a t fre q u e n c ie s close to th a t p o in t. T h u s poles
h av e th e o p p o site effe ct o f zero s. A lso , p lacin g a z e ro close to a p o le cancels
th e effe ct o f th e p o le, an d vice v ersa. T h is can b e also se en fro m (4.4.47), since
if Zk = Pk, th e te rm s eJW - z* an d ejb> pk cancel. O b v io u sly , th e p re se n c e of
b o th p o les an d z e ro s in a tra n sfo rm resu lts in a g r e a te r v a rie ty o f sh a p e s for
\H(to)\ a n d ^ H( c o ) . T his o b se rv a tio n is very im p o rta n t in th e desig n o f digital
filters. W e co n clu d e o u r discussion w ith th e follow ing ex a m p le illu stra tin g th ese
co n cep ts.
Example 4.4.7
Evaluate the frequency response of the system described by the system function

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

Clearly. H(z) has a zero at ; = 0 and a pole at p = 0.8.

frequency response of the system is

S o lu tio n

H (w) =

325

Hence the

el- - 0.8

The m agnitude response is

| H (to) j =

1
\e-lw - 0.8|

Vl-64 1.6cose

and the phase response is

B(co) = ui tan

sin oj
cos oj - 0.8

The magnitude and phase responses are illustrated in Fig. 4.42. Note that the peak
of the m agnitude response occurs at a s = 0. the point on the unit circle closest to the
pole located at 0.8.
If th e m a g n itu d e re sp o n se in (4.4.52) is ex p ressed in d ecib els,
M
\
\H(co)\lllt = 2 0 lo g ,(l |/j()| + 2 0 5 2 lg.jci v/a M - 2 o 5 2 logui
A=1
i-l

(4.4.58)

T h u s th e m a g n itu d e resp o n se is ex p ressed as a sum o f th e m ag n itu d e facto rs in

| H i w )!.

4.4.7 Input-Output Correlation Functions and Spectra

In S ectio n 2.6.5 w e d e v e lo p e d several c o rre la tio n re la tio n sh ip s b etw een the input
an d th e o u tp u t se q u e n c e s o f an L TI system . S pecifically, w e d eriv ed th e follow ing
e q u a tio n s:
r vv(m) = rhh(m) * r xx(m)

(4.4.59)

ryx( m ) = h( m) * rxx(m)

(4.4.60)

w h ere rxx(m) is th e a u to c o rre la tio n se q u e n c e of th e in p u t signal {-*(/?)), ryv(m) is

th e a u to c o rre la tio n se q u e n c e o f th e o u tp u t {.y(n)K rhh(m ) is th e a u to c o rre la tio n se
q u e n c e o f th e im p u lse re sp o n se {Ai(/t)}. a n d ryx(m) is th e c ro ss c o rre la tio n se q u e n c e
b e tw e e n th e o u tp u t an d th e in p u t signals. Since (4,4.59) an d (4.4.60) involve th e
co n v o lu tio n o p e ra tio n , th e z-tra n sfo rm o f th e s e e q u a tio n s yields
Svvf;) = 5 m ( : ) 5 ( c )
(4.4.61)
= H ( z ) H ( z ~ 1 )Sxx(z)
S VJr(z) = H ( z ) S ( z )

(4.4.62)

If we su b s titu te z = e JW in (4,4.62), we o b tain

S y jc M

H(os)Sxx(oj)
(4.4.63)

= H(o>)\X(o>)\2

326

Frequency Analysis of Signals and Systems

- T

_ t
2

I
2

Chap. 4

Figure 4.42 Magnitude and phase of

system with H(z) = 1/(1 - 0 .8 ; - 1).

w h ere Syx(u>) is th e cro ss-en e rg y d en sity sp e c tru m o f [y(n)} a n d (jc(/7)}. Sim ilarly,
ev alu atin g Svv(z) o n th e u n it circle yields th e en erg y d e n sity s p e c tru m o f th e o u tp u t
signal as
S v v M = | / / ( w ) |2S M

(4.4.64)

w h ere Sxx(a>) is th e en erg y d en sity sp e c tru m o f th e in p u t signal.

S ince ryy(m) an d Syy(co) a re a F o u rie r tra n sfo rm p air, it fo llo w s th at
ryy(m) = ~

Syy(co)ejumdco

(4.4.65)

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-Invariant Systems

327

T h e to ta l e n erg y in th e o u tp u t signal is sim ply

Ey --- f

Syy{CO)dlO = T,.v(0 >

~jT
i r
= I \ H(to)\' S xx(cv)dcv
J-*

(4.4.66)

T h e re su lt in (4.4.66) m ay be used to easily p rove th a t v > 0.

F in ally , we n o te th a t if the in p u t signal has a flat sp e c tru m [i.e.. 5 v., (w) =
, = c o n s ta n t fo r n < co < t t ] , (4.4.63) red u ces to
5 , v(oj) = H { w ) E x

(4.4.67)

w h ere , is th e c o n s ta n t v alu e o f the sp e ctru m . H e n c e

H ( w ) = 5vv(w)

(4.4.68)

h(n) r yxim)

(4.4.69)

E\

o r . e q u iv a le n tly .

T h e re la tio n in (4,4.69) im plies th at Inn) can be d e te rm in e d by exciting th e input

to th e system by a sp e ctrally flat signal ja ()), and c ro ssc o rre la tin g th e in p u t w ith
th e o u tp u t o f th e system . T his m e th o d is usef ul in m e a su rin g th e im pulse resp o n se
o f an u n k n o w n svstem .

4.4.8 Correlation Functions and Power Spectra for

Random Input Signals
T his d e v e lo p m e n t p a ra lle ls th e d e riv a tio n s in S ection 4.4.7, w ith th e e x cep tio n th a t
we no w d e a l w ith sta tistical m o m e n ts o f th e in p u t an d o u tp u t signals o f an L TI
system . T h e v ario u s sta tistical p a ra m e te rs are in tro d u c e d in A p p e n d ix A .
L et us c o n s id e r a d isc re te -tim e lin e a r tim e -in v a ria n t sy stem w ith u n it sam ple
re sp o n se (/z(n)} an d fre q u e n c y re sp o n se H ( f ) . F o r this d e v e lo p m e n t w e assum e
th a t {/?()} is real. L et _*() be a sam ple fu n ctio n of a sta tio n a ry ra n d o m process
X ( n ) th a t ex cites th e system an d let y(/;) d e n o te th e re sp o n se o f th e system to xi n) .
F ro m th e c o n v o lu tio n su m m atio n th a t re la te s th e o u tp u t to th e in p u t w e have
OC

v(n) =

5 2 h ( k ) x ( n k)

(4.4.70)

Jt = - oc

S ince jc(n) is a ra n d o m in p u t signal, th e o u tp u t is also a ra n d o m se q u en ce. In o th e r

w o rd s, fo r each sa m p le se q u e n c e x ( n ) o f th e p ro cess X ( n ), th e r e is a c o rre sp o n d in g
sa m p le se q u e n c e v(n) o f th e o u tp u t ra n d o m p ro c e ss Y(n) . W e w ish to relate
th e sta tistical c h a ra c te ristic s o f th e o u tp u t ra n d o m p ro cess Y( n) to th e statistical
c h a ra c te riz a tio n o f th e in p u t p ro c e ss a n d th e c h aracteristics o f th e system .

328

Frequency Analysis of Signals and Systems

Chap. 4

T h e ex p e c te d v alu e o f th e o u tp u t y(n) is
OC

22

m v s= [y (n )] = E[

h(k)x{rt fc)]

k ~ rx.
oc

22

A (* ) 0 (n - * ) ]

(4.4.71)

A=^oc
cc

22

my = mx

h( k)

k oc

F ro m th e F o u rie r tra n sfo rm re la tio n sh ip

CC

H{u>)=

22

* (*)*"-'*

(4.4.72)

k = - oc

we have

OC

22 h (*)

H ( 0) =

(4.4.73)

A DC

w hich is th e dc gain o f th e system . T h e re la tio n sh ip in (4.4.73) allow s us to express

th e m e a n v alu e in (4.4.71) as
m y = m xH{ 0)
(4.4.74)
T h e a u to c o rre la tio n se q u en ce fo r th e o u tp u t ra n d o m p ro c e ss is
yyy(m) [ y * ( n ) y ( + m ) ]
-- E

2 2 h( k)x*(n ~ k) Y 2
_kOQ
OC

22

+ m - j)

OC

(4.4.75)

V .

h ( k ) h ( j ) E [ x * { n - k)x(rt -f- m - _/)]

= C C _/ = OC
OC

cc

22 Y2 h(k'>hU)Yxx(k - j + m)
k= co

cc

T h is is th e g en eral fo rm fo r th e a u to c o rre la tio n o f th e o u tp u t in term s of the

a u to c o rre la tio n o f th e in p u t a n d th e im p u lse re sp o n s e o f th e system .
A sp ecial fo rm o f (4.4.75) is o b ta in e d w h en th e in p u t ra n d o m p ro c e ss is w hite,
th a t is, w h en m , = 0 an d
Yxx(m) = oj S ( m )
(4.4.76)
w h ere o x = ^ j(O ) is th e in p u t signal p o w er. T h e n (4.4.75) re d u c e s to
OC

Yyy(m)

= <72

22

h{k) h{k + m)

(4.4.77)

t= -o c

U n d e r th is co n d itio n th e o u tp u t p ro cess h a s th e a v e ra g e p o w e r
MO) = ^

h ^ n) =
n=-e

w h ere w e h av e a p p lie d P a rse v a ls th e o re m .

(4-4.78)
J - 1/2

Sec. 4.4

Frequency-Domain Characteristics of Linear Tim e-invariant Systems

329

T h e re la tio n sh ip in (4.4.75) can b e tra n sfo rm e d in to th e fre q u e n c y d o m ain

by d e te rm in in g th e p o w e r d e n sity sp e c tru m of yvv(m ). W e h ave
r vvM

22

/vv

h ( k ) h ( l ) Y, A k - I + m)

k ^ o o i =dc

(4.4.79)

h{k)h(l )

Yxx(k I + m) e

k = CC I = DC
OC

= r r, ( / )

2 2 h{k) eJwk

h { l ) e - ]wl

^k= - c c

= tw (tu )|-r^ .v(w)

T h is is th e d e sire d re la tio n sh ip fo r th e p o w e r d en sity sp e c tru m o f th e o u tp u t p r o
cess, in te rm s o f th e p o w e r d en sity sp e c tru m o f th e in p u t p ro cess an d th e freq u e n cy
re sp o n s e o f th e system .
T h e e q u iv a le n t e x p ressio n fo r co n tin u o u s-tim e system s w ith ra n d o m in p u ts is
r vv( F ) = \ H{ F ) \ 2 r xA F )

(4.4.80)

w h ere th e p o w e r d e n sity s p e c tra T w ^ ) and T ( F ) are th e F o u rie r tra n sfo rm s

o f th e a u to c o rre la tio n fu n c tio n s y v v ( T ) a n d yJ t ( r ) , resp ec tiv ely , a n d w h e re H { F )
is th e fre q u e n c y re sp o n s e of th e system , w hich is re la te d to th e im p u lse re sp o n se
by th e F o u rie r tra n sfo rm , th a t is.
H(F)

h( t ) e

- J-C
f

- j 2n Fi

dt

(4.4.81)

A s a final ex ercise , w e d e te rm in e th e c ro ssc o rre la tio n o f th e o u tp u t v(n) w ith

th e in p u t signal jcOi). If w e m u ltip ly b o th sides o f (4.4.70) by x*(n m) a n d ta k e
th e e x p e c te d v a lu e , w e o b ta in
y

[ v (n )-**( wz)] = E

h( k)x*{n m ) x ( n k)

= OC

y VJ(m) =

2 2 h( k) E[ x * ( n m ) x ( n k)]

(4.4.82)

= CC
OC

22

h ( k ) Y x x ( m - k)

Since (4.4.82) h as th e fo rm o f a c o n v o lu tio n , th e fre q u e n c y -d o m a in e q u iv a le n t

e x p re ssio n is

r(w ) = ff(<u)r(<u)

(4.4.83)

In the special case where x ( n ) is white noise, (4.4.83) reduces to

r y, M

= a x2 H { a > )

(4.4.84)

330

Frequency Analysis of Signals and Systems

Chap. 4

w h ere a 2 is th e in p u t n o ise p o w er. T h is re su lt m e a n s th a t an u n k n o w n sy stem w ith

fre q u e n c y resp o n se H ( w ) can be id en tified by exciting th e in p u t w ith w h ite noise,
c ro ssco rrelatin g th e in p u t se q u e n c e w ith th e o u tp u t se q u e n c e to o b ta in y Vjr(m ), and
finally, c o m p u tin g th e F o u rie r tra n sfo rm o f yyx(m). T h e re su lt o f th e s e c o m p u ta
tio n s is p ro p o rtio n a l to H{co).

4.5 LINEAR TIME-INVARIANT SYSTEMS AS FREQUENCY-SELECTIVE

FILTERS
T h e te rm filter is co m m o n ly used to d escrib e a d ev ice th a t d isc rim in a te s, a c co rd
ing to so m e a ttr ib u te o f th e o b jects a p p lied at its in p u t, w h at passes th ro u g h it.
F o r ex am p le, an a ir filter allow s a ir to p ass th ro u g h it b u t p re v e n ts d u st p a r
ticles th a t are p r e s e n t in th e a ir from passing th ro u g h . A n oil filter p erfo rm s
a sim ilar fu n ctio n , w ith th e e x cep tio n th a t oil is th e su b sta n c e allo w ed to pass
th ro u g h th e filter, w h ile p a rtic le s o f dirt are c o llected a t th e in p u t to th e filter
a n d p re v e n te d fro m p assin g th ro u g h . In p h o to g ra p h y , an u ltra v io le t filter is of
ten u sed to p re v e n t u ltra v io le t light, w hich is p re s e n t in su n lig h t a n d w hich is not
a p a rt o f visible light, from passin g th ro u g h an d affe ctin g th e chem icals on the
film.
A s w e h av e o b se rv e d in th e p re c e d in g se ctio n , a lin e a r tim e -in v a ria n t system
also p e rfo rm s a ty p e o f d isc rim in atio n o r filtering am o n g th e v ario u s freq u e n cy
c o m p o n e n ts a t its in p u t. T h e n a tu re o f th is filtering actio n is d e te rm in e d by the
fre q u e n c y re sp o n se c h a ra c te ristic s H{co), w hich in tu r n d e p e n d s o n th e ch o ice of
th e sy stem p a ra m e te rs (e.g., th e coefficients (at ) a n d [bk ] in th e d iffe re n c e e q u a tio n
ch a ra c te riz a tio n o f th e sy stem ). T h u s, by p r o p e r selectio n o f th e coefficients, we
can d esig n freq u e n cy -selectiv e filters th a t p ass signals w ith fre q u e n c y co m p o n e n ts
in so m e b an d s w h ile th e y a tte n u a te signals co n ta in in g fre q u e n c y c o m p o n e n ts in
o th e r fre q u e n c y b an d s.
In g e n eral, a lin e a r tim e -in v a ria n t system m odifies th e in p u t signal sp e c
tru m X((o) a cco rd in g to its fre q u e n c y re sp o n s e H(co) to yield an o u tp u t signal
w ith sp e c tru m Y(to) = H(o>)X(cl>). In a sense, H(co) acts as a wei ghti ng f u n c
tion o r a spectral s h a pi ng f un c t i o n to th e d iffe re n t fre q u e n c y c o m p o n e n ts in th e
in p u t signal. W h e n v iew ed in th is c o n te x t, any lin e a r tim e -in v a ria n t system can
b e c o n sid e re d to b e a fre q u e n c y -sh a p in g filter, ev en th o u g h it m ay n o t n ecessa r
ily c o m p letely b lo ck a n y o r all fre q u e n c y c o m p o n e n ts. C o n s e q u e n tly , th e term s
lin e a r tim e -in v a ria n t sy stem a n d filte r a re sy n o n y m o u s a n d a re o fte n used
in terch an g eab ly .
W e use th e te r m filter to d escrib e a lin e a r tim e -in v a ria n t system u se d to
p e rfo rm sp e c tra l sh a p in g o r freq u e n cy -selectiv e filtering. F ilte rin g is u se d in dig
ital sig n al p ro cessin g in a v a rie ty o f w ays. F o r e x am p le, re m o v a l o f u n d e sira b le
n o ise fro m d e s ire d signals, sp e c tra l sh a p in g such as e q u a liz a tio n o f c o m m u n icatio n
c h a n n e ls, signal d e te c tio n in r a d a r, so n a r, a n d c o m m u n ic a tio n s, a n d fo r p e rfo rm in g
sp e c tra l an aly sis o f signals, a n d so on.

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

331

4.5.1 Ideal Filter Characteristics

F ilters a re usu ally classified acco rd in g to th e ir fre q u e n c y -d o m a in ch aracteristics
as low pass, h ig h p ass. b a n d p a ss, and b a n d s to p o r b a n d -e lim in a tio n filters. T he
id eal m a g n itu d e re sp o n s e ch a ra c te ristic s o f th ese ty p e s o f filters are illu strated
in Fig. 4.43. A s sh o w n , th ese ideal filters h ave a c o n sta n t-g a in (usually ta k e n as
u n itv -g a in ) p a ssb a n d c h a ra c te ristic an d z e ro gain in th e ir sto p b a n d .

Lowpass

IHu)\

I
Hiphpa*

\HUo)I

Bandpass

-n

a>0

ai]

cu, iii(|Wi

t
Bandstop

~o>

All-pass

Figure 4.43 Magnitude responses

for some ideal frequency-selective
discrete-time filters.

332

Frequency Analysis of Signals and Systems

Chap. 4

A n o th e r ch a ra c te ristic o f an id eal filter is a lin e a r p h ase re sp o n s e . T o dem on

stra te this p o in t, let us assum e th a t a signal se q u e n c e (*(/?)} w ith fre q u e n c y com
p o n e n ts co n fin ed to th e fre q u e n c y ran g e coj < w < un is p a s se d th ro u g h a filter
w ith freq u e n cy resp o n se
H(co) = J
[
w h ere C an d

JWnv'

< co<c^
o th erw ise

(J,

(4_5

a re c o n stan ts. T h e signal at th e o u tp u t o f th e filte r h as a spectrum

Y( co) = X(co)H(co)
u>\ < co < co2

- CX(co)e }tL""

(4.5.2)

By app ly in g th e scaling a n d tim e-sh iftin g p ro p e rtie s o f the F o u rie r tran sfo rm , we
o b ta in th e tim e -d o m a in o u tp u t
v(ri) = Cx ( n - h 0)

(4.5.3)

C o n se q u e n tly , th e filter o u tp u t is sim ply a d e la y e d an d a m p litu d e -s c a le d v ersio n of

th e in p u t signal. A p u re d elay is usually to le ra b le a n d is n o t c o n s id e re d a distortion
o f th e signal. N e ith e r is a m p litu d e scaling. T h e re fo re , ideal filters have a linear
p h ase c h a ra c te ristic w ithin th e ir p a ssb a n d , th a t is.
0(a>) = cori(i

(4.5.4)

T h e d eriv ativ e o f th e p h a se w ith re sp e c t to fre q u e n c y has th e u n its o f delay.

H e n c e we can defin e th e signal d elay as a fu n ctio n o f fre q u e n c y as

d&(a>)
( o j) =

--------- ----------

aco

.
(4.5.5)

Zg(co) is usually called th e e nvel ope del ay o r th e g r o u p del ay o f th e filter. W e

in te r p re t zg(co) as th e tim e d elay th a t a signal c o m p o n e n t o f fre q u e n c y co u n d erg o es
as it p asses fro m th e in p u t to th e o u tp u t o f the system . N o te th a t w h en 0(co) is
lin e a r as in (4.5.4), r s (a)) no = c o n s ta n t. In th is case all fre q u e n c y co m p o n en ts
o f th e in p u t signal u n d e rg o th e sam e tim e delay.
In co n clu sio n , id eal filters h av e a c o n s ta n t m a g n itu d e c h a ra c te ristic an d a
lin ear p h ase c h a ra c te ristic w ith in th e ir p assb an d . In all cases, such filters a re not
p h ysically re a liz a b le b u t se rv e as a m a th e m a tic a l id e a liz a tio n o f p ra c tic a l filters.
F o r ex am p le, th e id eal low pass filter h a s an im p u lse re sp o n se

sin coc7Tn
hip(n) -----------nn

00 < /7 < 00

(4.5.6)

W e n o te th a t th is filter is n o t causal an d it is n o t a b so lu te ly su m m a b le an d th e re fo re
it is also u n sta b le . C o n se q u e n tly , this id eal filter is physically u n re a liz a b le . N ev
e rth e le ss, its fre q u e n c y re sp o n se c h a racteristics can be a p p ro x im a te d v ery closely
by p ractical, ph y sically re a liz a b le filters, as will be d e m o n s tra te d in C h a p te r 8.
In th e fo llo w in g d iscu ssio n , w e tr e a t th e design o f so m e sim p le d igital filters
by th e p la c e m e n t o f p o les a n d zero s in th e z-p lan e. W e h a v e a lre a d y d escrib ed
h o w th e lo catio n o f p o les a n d zero s affects th e fre q u e n c y re s p o n s e characteristics

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

333

o f th e system . In p a rtic u la r, in S ection 4.4.6 w e p re s e n te d a g ra p h ic a l m e th o d for

c o m p u tin g th e fre q u e n c y re sp o n se c h a racteristics fro m th e p o le - z e r o p lo t. T his
sam e a p p ro a c h can be u sed to design a n u m b e r o f sim ple b u t im p o rta n t digital
filters w ith d e s ira b le freq u e n c y re sp o n se ch aracteristics.
T h e basic p rin cip le u n d e rly in g th e p o le - z e r o p la c e m e n t m e th o d is to lo cate
p o les n e a r p o in ts o f th e u n it circle c o rre sp o n d in g to fre q u e n c ie s to be em p h asized ,
a n d to p lace ze ro s n e a r th e fre q u e n c ie s to be d e e m p h a siz e d . F u rth e rm o re , the
fo llo w in g c o n s tra in ts m ust be im posed:
1. A ll p o le s sh o u ld be p laced inside th e unit circle in o r d e r fo r the filter to be
stab le. H o w e v e r, ze ro s can be p laced an y w h ere in th e z-p lan e.
2. A ll co m p lex zero s an d p o les m ust o ccu r in c o m p le x -c o n ju g a te p airs in o rd e r
fo r th e filter coefficients to be real.
F ro m o u r p re v io u s discussion w e recall th a t fo r a given p o le - z e r o p a tte rn ,
th e sy stem fu n ctio n H i : ) can be ex p ressed as

(4.5.7)

w h ere bi} is a jiain c o n sta n t selected to n o rm aliz e th e freq u e n cy re sp o n se at som e

sp ecified freq u e n cy . T h a t is, bo is se le c te d such th at
| H ( cl>o) | = 1

(4.5.8)

w h ere
is a fre q u e n c y in th e p a ssb a n d o f the filter. U sually, N is se le c te d to
eq u al o r ex cee d M , so th a t th e filter h as m o re n o n triv ial p o le s th a n zeros.
In th e n ex t se ctio n , we illu strate th e m e th o d o f p o le - z e r o p la c e m e n t in the
d esig n o f so m e sim p le low pass. highpass. an d b an d p a ss filters, d igital re so n a to rs,
a n d co m b filters. T h e d esign p ro c e d u re is facilita ted w h en c a rrie d o u t in te ra c tiv e ly
on a d ig ital c o m p u te r w ith a g raphics term in a l.

4.5.2 Lowpass, Highpass, and Bandpass Filters

In th e d esig n o f lo w pass digital filters, th e p oles sh o u ld be p la c e d n e a r th e unit
circle at p o in ts c o rre sp o n d in g to low fre q u e n c ie s (n e a r cm = 0) and ze ro s sh ould
b e p la c e d n e a r o r on th e u n it circle at p o in ts c o rre sp o n d in g to high freq u e n cies
( n e a r co = t t ) . T h e o p p o site h o ld s tru e fo r h ighpass filters.
F ig u re 4.44 illu strates th e p o le - z e r o p la c e m e n t o f th re e low pass a n d th re e
h ig h p ass filters. T h e m a g n itu d e an d p h a se re sp o n se s fo r th e sin g le-p o /e filter w ith
sy stem fu n ctio n

334

Frequency Analysis of Signals and System s

Chap. 4

Highpass

Figure 4.44

Pole-zero patterns for several lowpass and highpass fillers.

are illu stra te d in Fig. 4.45 fo r a = 0.9. T h e gain G w as se le c te d a s 1 a, so th at

th e filter h as u n ity gain a t co = 0. T h e gain of this filter a t h ig h fre q u e n c ie s is
relativ ely sm all.
T h e a d d itio n o f a ze ro a t z = 1 f u rth e r a tte n u a te s th e re s p o n s e o f th e filter
a t high freq u e n cies. T h is lead s to a filter w ith a sy stem fu n ctio n
H 2 (z)

= 1- ~ ~ ~ ~l~~' .
2 1 az~

(4.5.10)

an d a freq u e n cy re sp o n se c h a ra c te rstic th a t is also illu stra te d in Fig. 4.45. In this

case th e m ag n itu d e o f H 2 (co) g o es to z e ro a t co = n.
S im ilarly, w e can o b ta in sim p le h ig h p ass filters by reflec tin g (fo ld in g ) the
p o le -z e ro lo catio n s o f th e low pass filters a b o u t th e im ag in ary axis in th e z-plane.
T h u s w e o b tain th e sy stem fu n ctio n
H i ( z ) = -l ~ a
\
2 1 -f a z ~ [

(4.5.11)

w hich has th e freq u e n cy re sp o n s e c h a ra c te ristic s illu stra te d in Fig. 4.46 fo r a = 0.9.

Example 4.5.1
A two-pole lowpass filter has the system function

W(z) =

b0
(1 -

pz

-')2

Sec. 4.5

Linear Tim e-invariant Systems as Frequency-Selective Filters

335

Figure 4.45 Magnitude and phase

response of (1) a singie-pole filter
and (2) a one-pole, one-zero
filter; Wi(;) = (1 - a ) /( 1 a ; ' 1),
H j(z) = [(1 a )/2 ][(l + r ~ * )/(l - a ; - 1 )]
and a = 0.9.

D eterm ine the values of h{>and p such that the frequency response H(w) satisfies the
conditions
H(0) = 1
and
I / K \ I2
1

r W I =2

Solution

A t o) = 0 we have
H( 0) =

(1 - p )2

Hence
bo = (1 - p Y

= 1

336

Chap. 4

20 log|0]HM|

Frequency Analysis of Signals and Systems

Figure 4.46 Magnitude and phase

response of a simple highpass fitter;
H(C) = [(1 - a )/2 ][(l - z - ' ) / ( l + a z - 1)]

with a = 0.9.
At w = tt/4.
(1 ) =
(1 ~ P?
v4 /
(1 pe~i*/4)2
(1 ~ P )2
(1 - p cos(;r/4) + j p sin (tt/4))2
(1 - P ) 2

(1 - pj s / 2 + jp/ ^/ 2)2
Hence
(1 - p t

[(1 - p / - j2 ) 2 + p 212]2

1
2

Sec. 4.5

Unear Tim e-Invariant Systems as Frequency-Selective Filters

337

or, equivalently.
V ^d - p y = 1 + p1 - y'lp
The value of p = 0.32 satisfies this equation. Consequently, the system function for
the desired filter is
0.46
H( z)
(1 - 0.32c 1)=
T h e sam e p rin cip le s can be a p p lied fo r th e desig n of b a n d p a s s filters. B asi
cally, th e b a n d p a s s filter sh o u ld c o n tain o n e o r m o re p airs o f co m p lex -co n ju g ate
p o les n e a r th e u n it circle, in the vicinity of th e freq u e n cy b a n d th a t c o n stitu te s the
p assb a n d o f th e filter. T h e follow ing ex am p le serves to illu strate th e basic ideas.
E x a m p le 4.5.2

Design a two-pole bandpass filler that has the center of its passband at w = n72.
zero in its frequency response characteristic at w = 0 and at = n . and its magnitude
response is 1 /V 2 at w = 4w /9 .
S o lu tio n

Clearly, the filter must have poles at

Pl : = r i b

and zeros at

1 and : = 1. Consequently, the system function is

(: - 1)<: + 1)
(: - j r)(: + j r)

/z o = a

- 1

= c-

The gain factor is determ ined by evaluating the frequency response H(w) of the filler
at u> = jr/2. Thus we have

" ( ) - cr b - >
G

1 - r2

The value of r is determ ined by evaluating H( w) at w = 4tt/9. Thus we have

4tt \

(1 - r1)2

9 /|

1 4-

2 - 2 c o s ( 8 7 t/9 )
r4 + 2r2 c o s ( 8 jt/9 )

1
2

or. equivalently,
1.94(1

The value of r2 =
desired filter is

0.7

- r 1)1 =

1 .8 8 r2 + r 4

satisfies this equation. Therefore, the system function for the

"<:l = a,5IT5&
Its frequency response is illustrated in Fig.

4.47.

338

Frequency Analysis of Signals and Systems

_ T

x
~2

r
2

Chap. 4

Figure 4.47 Magnitude and

phase response of a simple
bandpass filter in Example 4.5.2;
H(z) = 0.15[(1 - z ~ 2) / a + 0 . 7 ; - 2)].

It sh o u ld b e e m p h a siz e d th a t th e m ain p u rp o se of th e fo re g o in g m eth o d o lo g y

fo r d esig n in g sim p le digital filters b y p o le - z e r o p la c e m e n t is to p ro v id e insight
in to th e effect th a t p o les a n d z e ro s have on th e fre q u e n c y re sp o n s e characteristic
o f system s. T h e m e th o d o lo g y is n o t in te n d e d as a goo d m e th o d fo r designing
d igital filters w ith w ell-specified p a s sb a n d an d sto p b a n d ch aracteristics. System atic
m e th o d s fo r th e d esig n of so p h is tic a te d d ig ital filters fo r p ra c tic a l ap p lic a tio n s are
d iscussed in C h a p te r 8.

A simple lowpass-to-highpass filter transformation. S u p p o se th a t we

h av e d esig n ed a p r o to ty p e low pass filter w ith im p u lse re sp o n se h\p(n). B y us-

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

339

ing th e fre q u e n c y tra n sla tio n p ro p e rty o f th e F o u rie r tra n sfo rm , it is p o ssib le to
co n v ert th e p ro to ty p e filter to e ith e r a b a n d p a ss or a h ig h p ass filter. F re q u en cy
tra n sfo rm a tio n s fo r c o n v ertin g a p ro to ty p e low pass filter in to a filter of a n o th e r
ty p e are d e sc rib e d in d etail in S ection 8.3. In th is sectio n w e p re se n t a sim plefre q u e n c y tra n sfo rm a tio n fo r co n v e rtin g a low pass filter in to a h ig h p ass filter, and
vice v ersa.
If /i|p(n) d e n o te s th e im p u lse resp o n se of a low pass filter w ith freq u e n cy
re sp o n se H\r (co). a h ighpass filter can be o b ta in e d by tra n sla tin g H]P(a>) by t t rad ian s
(i.e., rep lacin g co by co n ) . T h u s
^ h p ( ^ )

H\p(co

TT )

(4.5.12)

w h e re H hP(w) is th e fre q u e n c y re sp o n se o f th e h ig h p ass filter. Since a freq u e n cy

tra n sla tio n o f t t ra d ia n s is e q u iv a le n t to m u ltip lic atio n o f th e im pulse resp o n se
/iip(;i) by e 177", th e im p u lse resp o n se o f th e highpass filter is
/ihp(n) (p 7't )''/7|p (/7) = ( 1 )"/ ii p( )

(4.5.13)

T h e re fo re , th e im p u lse resp o n se of th e h ig h p ass filter is sim ply o b ta in e d from the

im pulse re sp o n se o f the low pass filter by ch an g in g th e signs o f th e o d d -n u m b e re d
sa m p le s in inp(n). C o n v ersely .
= ( - 1 )"/ihp()

(4.5.14)

If th e lo w pass filter is d e sc rib e d by th e d ifferen ce e q u a tio n

(4.5.15)

its fre q u e n c y re sp o n s e is
M
k0

(4.5.16)

N ow , if w e re p la c e co by co n , in (4.5.16). th en
M

tfhpM =

--------------

(4.5.17)

1 + ( - 1 )kake ~ iak
w hich c o rre sp o n d s to th e d iffe re n c e e q u a tio n
( 4 . 5 . 18 )

Frequency Analysis of Signals and Systems

340

Chap. 4

Example 4.5.3
Convert the lowpass filter described by the difference equation
v(n) = 0.9v(n 1) + 0 .1 x 0 )
into a highpass filter.
Solution

The difference equation for the highpass filter, according to (4.5.18), is

y(n) = -0.9v(n - 1) + 0.1x(n)

and its frequency response is

0.1

- 1+ Q9e_ju
The reader may verify that H^((o) is indeed highpass.

4.5.3 Digital Resonators

A digital resonat or is a sp ecial tw o-pole b a n d p a ss filter w ith th e p a ir o f com plexc o n ju g ate p o les lo c a te d n e a r th e u n it circle as show n in Fig. 4 .4 8 (a). T h e m ag n itu d e
of th e fre q u e n c y re sp o n se of th e filter is show n in Fig. 4 .48(b). T h e n am e re so n a to r
re fe rs to th e fact th a t th e filter has a large m a g n itu d e re sp o n se (i.e., it re so n a te s ) in
th e v icinity o f th e p o le lo catio n . T h e a n g u lar p o sitio n of th e p o le d e te rm in e s th e
re so n a n t freq u e n c y o f th e filter. D igital re so n a to rs are useful in m a n y applications,
in clu d in g sim p le b a n d p a ss filterin g an d sp e ech g e n e ra tio n .
In th e d esig n o f a digital r e s o n a to r w ith a re so n a n t p e a k at o r n e a r to o>o,
w e se lect th e c o m p lex -co n ju g ate p o les at
Pi .2 = r e ja*

0 < r < 1

In a d d itio n , we can select u p to tw o zeros. A lth o u g h th e re a re m an y possible

choices, tw o cases a re o f special in terest. O n e choice is to lo cate th e z ero s a t the
origin. T h e o th e r ch o ice is to locate a z e ro a t z = 1 an d a z e ro a t z = 1. T his
choice co m p letely e lim in a te s th e re sp o n s e o f th e filter a t fre q u e n c ie s co = 0 and
co = n , an d it is u se fu l in m an y p ractical a p p licatio n s.
T h e sy stem fu n ctio n o f th e digital r e s o n a to r w ith ze ro s a t th e origin is
H ( z ) = ----------- :------ r ~ -----------:-------r
(1 r e ^ z
)(1 re -' "z- 1 )

(4.5.19)

H ( z ) = ------- ----------(4.5.20)
1 (2 r coscwo);-1 + r 2z ~2
S ince \H(co)\ h as its p e a k at o r n e a r co = coo, w e select th e gain bo so th a t
|W(too)I = 1- F ro m (4.5.19) w e o b ta in
b0

H{(Oo) =

---------:------- ----- ^-----------:------- :---(1 - r e j<*,e -J*>)0. - r e - w e - w )

_______________
(1 r ) ( l r e - -'2'00)
an d h e n c e
bo

\H(coo)\ = ------;---I - === = 1

(1 r)V 1 + r 2 2 r cos2a)o

(4 5 21)

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

341

2
(b)

tt

_ n

r
2

(c)

Tt

Figure 4.48 (a) Pole-zero pattern and

(b) the corresponding magnitude and
phase response of a digital resonator
with (1) r 0.8 and (2) r = 0.95.

342

Frequency Analysis of Signals and System s

Chap. 4

T h u s th e d esired n o rm a liz a tio n fa c to r is

i?o = (1 r ) > /l + r 2 2r cos2a>o

(4.5.22)

T h e freq u e n cy re sp o n se o f th e r e s o n a to r in (4.5.19) can b e e x p re ss e d as

bo
\H(a>)\ = ----------------Ux(to)U2 ((o)

(4.5.23)

(w ) = 2 to <J>i (tfj) <t>2 (co)

w h ere U\ (w) a n d 1/ 2 (01) are th e m a g n itu d e s of th e v e c to rs fro m p\ an d p 2 to the
p o in t w in th e u n it circle a n d <t>i(<w) an d <J>2 (d) a re th e c o rre sp o n d in g angles of
th e se tw o v ecto rs. T h e m a g n itu d e s Ui(to) a n d U2 (co) m ay be e x p re ss e d as
Ui(co) = J \ + r 2 2 r cos(a>o to)
(4.5.24)
U 2 (<o) = y / 1 + r 2 2 r c o s (w q + co)

F o r any v alu e o f r , U\(to) ta k e s its m in im u m v alu e (1 r ) a t to = too- T he

p ro d u c t U\((o)U2 ({o) re a c h e s a m in im u m v alu e at th e fre q u e n c y
air = co s-1

coswo^j

(4.5.25)

w hich defin es p recisely th e r e s o n a n t fre q u e n c y o f th e filter. W e o b se rv e th a t w hen

r is very close to u n ity , tor
a>o, w hich is th e a n g u la r p o sitio n o f th e p o le. W e also
o b se rv e th a t as r a p p ro a c h e s unity, th e re so n a n c e p e a k b ec o m e s s h a rp e r becau se
U\(to) ch an g es m o re rap id ly in relativ e size in th e vicinity o f ojo- A q u a n tita tiv e
m e a su re o f th e sh a rp n e s s o f th e re so n a n c e is p ro v id e d by th e 3-dB b a n d w id th A w
o f th e filter. F o r v alu es o f r close to u nity.
Aw % 2(1 - r )

(4.5.26)

F ig u re 4.48 illu stra te s th e m a g n itu d e a n d p h a s e o f d igital re s o n a to rs w ith

coq = n / 3 , r = 0.8 an d too ~ *73, r = 0.95. W e n o te th a t th e p h a se resp o n se
u n d erg o es its g re a te st ra te o f c h a n g e n e a r th e re s o n a n t freq u e n cy .
If th e zero s o f th e digital r e s o n a to r a re p laced a t z = 1 an d z = 1, th e
re s o n a to r has th e sy stem fu n ctio n
H(z) = G

= G

(1 - z - ' X l + z " 1)

(1 re>aK,z ~ l )( 1 r e - -'a*2 -1 )

(4.5.27)

1 -z -2
1 (2r coscdo)z-1 + r 2z ~2

an d a fre q u e n c y re sp o n s e c h a ra c te ristic
H{ w) = b [ \ _

(*-->][! _ r e - n o +*>)J

(4.5.28)

W e o b se rv e th a t th e z e ro s a t z = 1 affe ct b o th th e m a g n itu d e a n d p h a se resp o n se

o f th e re so n a to r. F o r ex a m p le , th e m a g n itu d e re sp o n s e is
\H(to)\=b0

U\(to)U2(to)

( 4 -5 2 9 )

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

343

Figure 4.49 Magnitude and phase

response of digital resonator with zeros
at 10 = 0 and
- t and ( I ) r = O.N and
(2) r = 0.95.

to=

w h ere N { oj) is d efin ed as

N ( w) 7 2 (1 - cos2a>)
D u e to th e p re se n c e o f th e zero fa c to r, th e re s o n a n t fre q u e n c y is a lte re d from
th at given by th e ex p ressio n in (4.5.25). T h e b a n d w id th of th e filter is also a lte re d .
A lth o u g h ex act v alu es for th e s e tw o p a ra m e te rs are ra th e r te d io u s to d eriv e, we
can easily c o m p u te th e fre q u e n c y re sp o n se in (4.5.28) an d c o m p a re th e re su lt with
th e p re v io u s case in w hich th e zeros a re lo c a te d at th e origin.
F ig u re 4.49 illu strates th e m a g n itu d e an d p h a s e c h a ra c te ristic s fo r o>q n / 3 .
r = 0.8 an d a>o tt/ 3, r = 0.95. W e o b se rv e th a t th is filter h a s a slightly sm a ller
b a n d w id th th a n th e re s o n a to r, w hich h as zero s a t th e origin. In a d d itio n , th e re
a p p e a rs to be a v ery sm all shift in th e re so n a n t fre q u e n c y d u e to th e p re se n c e of
th e zero s.

4.5.4 Notch Filters

A n o tc h filter is a filter th a t c o n ta in s o n e o r m o re d e e p n o tc h e s o r, ideally, p e rfe c t
nulls in its fre q u e n c y re sp o n se ch a ra c te ristic . F ig u re 4.50 illu stra te s th e freq u e n cy
re sp o n se c h a ra c te ristic o f a n o tc h filter w ith nulls a t fre q u e n c ie s cl>o an d
. N otch
filters are useful in m an y a p p lic a tio n s w h e re specific fre q u e n c y c o m p o n e n ts m ust
be elim in a te d . F o r ex am p le, in stru m e n ta tio n a n d re c o rd in g sy stem s re q u ire th a t
th e p o w er-lin e fre q u e n c y o f 60 H z a n d its h a rm o n ic s b e e lim in ated .

344

Frequency Analysis of Signals and System s

Chap. 4

Figure 4,50 Frequency response

characteristic of a notch filter.

T o c re a te a null in th e fre q u e n c y re sp o n se o f a filter at a fre q u e n c y ojo, we

sim ply in tro d u c e a p a ir of c o m p lex -co n ju g ate ze ro s on th e u n it circle at an angle
too. T h a t is,
S,.2 = f j
T h u s th e system fu n ctio n fo r an F IR n o tch filter is sim ply
H( z ) = boQ - eJlo" z ~] Ml (4.5.30)
bo (1 2 C O S tr>()c '

z ~)

A s an illu stratio n . Fig. 4.51 show s th e m a g n itu d e re sp o n se fo r a n o tc h filter having

a null a t a> = tt/4 .
T h e p ro b lem w ith th e F IR notch filter is th a t th e n o tch has a relativ ely large
b an d w id th , w hich m ean s th a t o th e r fre q u e n c y c o m p o n e n ts a ro u n d th e d esired null
are se v ere ly a tte n u a te d . T o re d u c e th e b an d w id th o f th e null, w e can re so rt to
a m o re so p h isticated , lo n g er F IR filter d esig n ed acco rd in g to c rite ria describ ed
in C h a p te r 8. A lte rn a tiv e ly , w e could, in an ad h o c m a n n e r, a tte m p t to im prove
on th e freq u e n cy re sp o n se ch a ra c te ristic s by in tro d u c in g p oies in th e system func
tion.
S u p p o se th a t w e p lace a p a ir o f c o m p lex -co n ju g ate p o les at

Ph2 = r e i m
T h e effect o f th e p o les is to in tro d u c e a re so n a n c e in th e vicinity o f th e null and
th u s to red u ce th e b a n d w id th o f th e n o tch . T h e sy stem fu n ctio n fo r th e resulting
filter is
H

(c) =

1 - 2 cos wo; 1 + z~
bo1 - 2 r cosojoZ-1 + r 2z ~2

(4.5.31)

T h e m ag n itu d e re sp o n se \H(u>)\ o f th e filter in (4.5.31) is p lo tte d in Fig. 4.52 for

coo = t t /4 , r = 0.85, an d fo r ti>o = n / 4 , r = 0.95. W h en c o m p a re d w ith the
freq u e n cy resp o n se o f th e F IR filter in Fig. 4.51, w e n o te th a t th e effect o f the
p o les is to red u ce th e b an d w id th o f th e no tch .
In a d d itio n to red u cin g th e b a n d w id th o f th e n o tc h , th e in tro d u c tio n o f a
p o le in th e vicinity o f th e null m ay re su lt in a sm all rip p le in th e p a s sb a n d o f th e
filter d u e to th e re so n a n c e c re a te d by th e pole. T h e effe ct o f th e rip p le can be
re d u c e d by in tro d u c in g a d d itio n a l p o le s a n d /o r ze ro s in th e sy ste m fu n c tio n o f the
n o tch filter. T h e m a jo r p ro b le m w ith th is a p p ro a c h is th a t it is b a sic a lly an ad hoc,
tria l-a n d -e rro r m e th o d .

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

345

Figure 4.51 Frequency response

characteristics of a notch filter with
a notch at co = n / 4 or / = 1/8;
H(z) = G[1 - 2cosooz-1 + z-2 ]-

4.5.5 Comb Filters

In its sim p lest fo rm , a c o m b filter can be v iew ed as a n o tc h filter in w h ich th e
n u lls o c c u r p e rio d ic a lly ac ro ss th e fre q u e n c y b a n d , h e n ce th e an a lo g y to an o rd i
n a ry c o m b th a t h as p erio d ically sp a ced te e th . C o m b filters find ap p lic a tio n s in a
w ide ra n g e o f p ra c tic a l sy stem s such as in th e re je c tio n o f p o w e r-lin e h arm o n ics,
in th e se p a ra tio n o f so la r a n d lu n a r c o m p o n e n ts fro m io n o sp h e ric m e a s u re m e n ts
o f e le c tro n c o n c e n tra tio n , a n d in th e su p p re ssio n o f c lu tte r fro m fixed o b je c ts in
m o v in g -ta rg e t-in d ic a to r (M T I) ra d a rs.

346

Frequency Analysis of Signals and Systems

Chap. 4

Figure 4.52

Frequency response
characteristics of two notch filters with
poles at ( I ) r = 0,85 and 12) r (1.95:

H(z) = />n|(l - - cos to u r'1

2r cosgju:-1 + r2z ~~)].

r " ")/(1 -

T o illu strate a sim ple form of a co m b filler, c o n sid er a m oving a v erag e (F IR )

filter d escrib ed by th e d ifferen ce e q u a tio n
xin - k )

v() =

(4.5.32

T h e system fu n ctio n o f this F IR filter is

Hiz

+ * Jt=0
1

[ l - c - * ^ 11]

M + 1

( 1 - - - )

(4.5.33)

an d its freq u e n cy re sp o n se is
S]na)( j d )
H(a>) =

(4.5.34)
M + 1

sin(w /2)

F ro m (4.5.33) we o b se rv e th a t th e filter has ze ro s on th e u n it circle at

k = 1 , 2 . 3 .........M

(4.5.35)

N o te th a t th e p o le a t ; = 1 is actu ally can ce le d by th e zero at ; = 1, so th a t in

effect th e F IR filter d o e s n ot c o n tain p o le s o u tsid e z = 0.
A p lo t o f th e m ag n itu d e c h a ra c te ristic o f (4.5.34) clearly illu stra te s th e ex
isten ce o f th e p erio d ically sp a c e d zero s in fre q u e n c y at cot = 2 n k / ( M + 1) for
= 1 , 2 , -----M. F ig u re 4,53 sh o w s \ H( w) \ fo r M = 10.

Sec. 4.5

Linear Tim e-Invariant Systems as Frequency-Selective Filters

347

Figure 4.53 Magnitude response

characteristic for the comb filter given
by (5,4.32) with M = 10.

In m o re g e n eral te rm s, w e can c re a te a com b filter by ta k in g an F I R filter

w ith sy stem fu n ctio n
M
H( z ) = 2 2 h ( k ) z ~ k
(4.5.36)
k=0
an d rep la c in g z by z L, w h ere L is a po sitiv e in teg er. T h u s th e new F IR filter has
a sy stem fu n ctio n
H l (z ) = Y , h ( k ) z

(4.5.37)

If th e fre q u e n c y re sp o n se of th e original F IR filter is //(w ), th e freq u e n c y resp o n se

o f th e F IR in (4.5.37) is
M
H l (u>) = 2 2 h ( k ) e ~ jkLa
(4.5.38)
k=I)
= H(Lco)
C o n s e q u e n tly , th e fre q u e n c y re sp o n se c h a ra c te ristic H l {oj) is sim ply an L -o rd e r
re p e titio n o f H(co) in th e ra n g e 0 < co < 2 n . F ig u re 4.54 illu stra te s th e re la tio n sh ip
b e tw e e n H l ( c o) a n d H (w) fo r L 5.
N o w , su p p o se th a t th e o rig in al F I R filter w ith system fu n ctio n H ( z ) h as a
sp e c tra l n ull (i-e -, a z e ro ), at so m e fre q u e n c y coo. T h e n th e filter w ith system
fu n ctio n H L(z) h a s p erio d ically sp a ced nulls a t av = coo + I n k j L , k = 0, 1 , 2 , . . . ,
L 1. A s an illu stra tio n , F ig. 4.55 show s an F IR co m b filter w ith Af = 3 an d
L = 3. T h is F IR filter can b e v iew ed as an F IR filter o f le n g th 10, b u t only fo u r
o f th e 10 filter coefficients are n o n zero .
L e t us no w r e tu rn to th e m oving a v erag e filter w ith system fu n ctio n given by
(4.5.33). S u p p o se th a t we re p la c e z by z L. T h e n th e re su ltin g co m b filter h as th e
sy stem fu n ctio n
1
1_
1)
Hl(z) = T 7~~r
/
(4-5.39)
M + 1
l-z~ L

and a frequency response

H l ( co) =

sin[a>L(M + l ) /2 ]

M + 1

sin(coL/2)

(4.5.40)

348

Frequency Analysis of Signals and Systems

Chap. 4

H (u > )

(a)
Hl (co)

(b)
Figure 4.54

Comb fiJter with frequency response W/Joi) obtained from H(a>).

Figure 4.55

Realization of an FIR comb filter having M = 3 and L ~ 3.

T his filter has zero s on th e u n it circle at

Zk = e j 2* k / L l M+ h

(4.5.41)

fo r alt in te g e r v alu es o f k e x cep t k 0, L, 2 L .........M L . F ig u re 4.56 illu strates

\ H l {(d )\ fo r L = 5 an d M = 10.
T h e co m b filter d esc rib e d by (4.5.39) finds a p p lic a tio n in th e s e p a ra tio n of
so la r a n d lu n a r sp e c tra l c o m p o n e n ts in io n o sp h e ric m e a s u re m e n ts o f e le c tro n c o n
c e n tra tio n as d e sc rib e d in th e p a p e r by B e rn h a rd t e t al. (1976). T h e so la r p e rio d
is Ts = 24 h o u rs an d resu lts in a so la r c o m p o n e n t o f o n e cycle p e r d ay a n d its
h arm o n ics. T h e lu n a r p e rio d is Tl = 24.84 h o u rs a n d p ro v id e s sp e c tra l lin es at
0.96618 cycle p e r d ay an d its h arm o n ics. F ig u re 4.57a show s a p lo t o f th e p o w e r
d en sity sp e c tru m o f th e u n filtered io n o sp h e ric m e a s u re m e n ts o f th e e le c tro n con-

Sec. 4.5

Linear Time-Invariant Systems as Frequency-Selective Filters

349

Figure 4.56 Magnitude response

characteristic for a comb filter given by
(4.5.40). with L 3 anti M = II).

c e n tra tio n . N o te th a t th e w eak lu n a r sp e ctral c o m p o n e n ts are alm o st h id d en by

th e stro n g so la r sp e c tra l co m p o n en ts.
T h e tw o se ts o f sp e c tra l c o m p o n e n ts can be se p a ra te d by th e use o f com b
filters. If w e w ish to o b ta in th e so lar co m p o n e n ts, we can use a co m b filter w ith
a n a rro w p a s sb a n d a t m u ltip le s of o n e cycle p e r day. T h is can be ach iev ed by
selectin g L such th a t Fs/ L = 1 cycle p e r day. w h ere Fs is th e c o rre sp o n d in g
sa m p lin g freq u e n cv . T h e resu lt is a filter th a t has p e a k s in its freq u e n cy resp o n se
at m u ltip le s o f o n e cycle p e r day. By se lectin g M = 58. the filter will h ave nulls
at m u ltip le s o f ( F J L ) I ( M + 1) = 1/59 cycle p e r day. T h e se nulls a re very close
to th e lu n a r c o m p o n e n ts a n d result in good rejectio n . F ig u re 4.57(b) illu strates

Freq ue ncy (cycles/day)

<c)

Figure 4.57 (a) Spectrum of unfiltered electron content data; (b) spectrum of out
put of solar filter; (c) spectrum of output of lunar filter. [From paper by Bernhardt
et al. (1976). Reprinted with permission of the American Geophysical Union.]

Frequency Analysis of Signals and Systems

350

Chap. 4

the p ow er spectral density of the output o f the com b filter that isolates the solar
com ponents. A com b filter that rejects the solar com p onents and passes the lunar
com ponents can be d esigned in a sim ilar m anner. Figure 4.57(c) illustrates the
pow er spectral density at the output o f such a lunar filter.

4.5.6 All-Pass Filters

An all-pass filter is defined as a system that has a constant m agnitude resp onse for
all frequencies, that is.
|ff{oj)l = l

0<o><7r

(4.5.42)

The sim plest exam ple o f an all-pass filter is a pure delay system with system func
tion
H(z) = z~k
This system passes all signals w ithout m odification except for a d elay of k sam ples.
This is a trivial all-pass system that has a linear phase response characteristic.
A m ore interesting all-pass filter is described by the system function
a x + a /v '-i" ^ 1 + + a \ Z

n(z) = -

1 + 0\Z
,.

' + + Qn Z

(4.5.43)

.- N + k

----- IT

a = 1

where all the filter coefficients \ak ) are real. If w e define the polyn om ial /\(~) as
A (;) = Y ^ a kz k
k=U

an 1

then (4.5.43) can be expressed as

H(z) = z ~ n M

(4.5.44)

A(z)

Since
\H(a>)\2 =

= 1

the system given by (4.5.44) is an all-pass system . Furtherm ore, if zo is a pole

o f H( z ) . then 1/zu is a zero o f H ( z ) (i.e., the p oles and zeros are reciprocals of
o n e another). Figure 4.58 illustrates typical p o le -z e r o patterns for a single-pole,
single-zero filter and a tw o-p ole, tw o-zero filter. A plot o f the p h ase characteristics
o f these filters is shown in Fig. 4.59 for a = 0.6 and r = 0.9, too = tt/4.
The m ost genera! form for the system function o f an all-pass system with real
coefficients, expressed in factored form in term s of p oles and zeros, is
Nr
H, f ^ - T T - _____
- 1J i -

n
Pk
l \ a - A z - x i - P i z ~ x)

(4 5 45)
(4 5 '

where there are N R real p o les and zeros and N c com p lex-con ju gate pairs o f poles
and zeros. For causal and stable system s w e require that - 1 < a* < 1 and |/S*| < 1.

Sec. 4.5

Linear Time-Invariant Systems as Frequency-Selective Filters

351

(a)

Figure 4.58 Pole-zero patterns of (a) a

first-order and (b) a second-order
all-pass filter.

10

20 tog I H M I

0
-10
-20

-3 0

0(u)

-4 0

Figure 4.59 Frequency response

characteristics of an all-pass
filter with system functions
(1) H(z) = (0.6 + z - ') / ( l + 0 . 6 Z '1),
(2) H(z) = (r2 - 2tcoswqz~1 + Z ~ 2 ) /

(1 2rcoscuoz-i + r2z~2), r = 0.9,

(jo n/A.

352

Frequency Analysis of Signals and Systems

Chap. 4

E xp ression s for the phase response and group delay o f all-pass system s can
easily be obtained using the m ethod described in Section 4.4.6. For a single p o lesingle zero all-pass system w e have
Manioc)

Ju

H en ce
r sin(a) 6}

C-)ap(w) = cd 2 tan'

r COS(w - 6 )

and
di~).Ap(ct>)

1 - r2

ciaj

1 + i - 2r cosiw 0)

(4.5.46)

W e n ote that for a causal and stable system , r < 1 and hence rt, (w) > 0. Since the
group delay o f a higher-order p o le-zero system consists o f a sum o f positive terms
as in (4.5.46), the group delay will always be positive.
A ll-pass filters find application as phase equalizers. W hen placed in cascade
with a system that has an undesired phase response, a phase eq u alizer is designed
to com pensate for the poor phase characteristics of the system and therefore to
p roduce an overall linear-phase response.

4.5.7 Digital Sinusoidal Oscillators

A digital si nusoi dal oscillator can be view ed as a lim iting form o f a tw o-p ole res
onator for which the com plex-conjugate p oles He on the unit circle. From our
previous discussion o f secon d-ord er system s, we recall that a system with system
function
H ( z ) = -----------------------1 + a j z " 1 + ai z

(4.5.47)

and param eters

a i = 2rcosa>o

and

a2 = r 2

(4.5.48)

has com plex-conjugate poles at p = r e m \ and a unit sam ple response

h(n) = ~ sin(n + l)fDow(n)
sin cun

(4.5.49)

If the p oles are placed on the unit circle (r = 1) and bo is set to A sincuo, then
h(n) = A s i n ( n + l)u>ou(n)

(4.5.50)

Thus the im pulse response o f the second-order system with com plex-conjugate
poles on the unit circle is a sinusoid and the system is called a digital sinusoidal
oscillator or a digital si nusoi dal generator. A digital sinusoidal gen erator is a basic
com ponent o f a digital frequency synthesizer.

Sec. 4.5

Linear Time-Invariant Systems as Frequency-Selective Filters

353

T h e block diagram representation o f the system function given by (4.5.47) is

illustrated in Fig. 4.60. The corresponding difference equation for this system is
v ( n) = a i y0? 1) y( n 2) + boS(n)

(4.5.51)

w here the param eters are a\ = 2cosa>o and bo = Asincwo, and the initial con d i
tions are y ( 1) = v (2) = 0. By iterating the differen ce eq u ation in (4.5.51), we
obtain
y (0) = ^ sin w o
y ( l ) = 2cos<woy(0) - 2 A sin too cos tt>o = Asin2tL>o
y(2) 2costL>oy(l) - y(0)
= 2 A cos ao sin 2wo A sin
- A {4 cos2 coq 1) sin

loq

wq

3 A sin too ~ 4 sin 3 coq = A sin Ixoq

and so forth. W e n ote that the application of the im pulse at n = 0 serves the
purpose o f beginning the sinusoidal oscillation. T hereafter, th e oscillation is selfsustaining b ecau se the system has no dam ping (i.e., r = 1).
It is in terestin g to note that the sinusoidal oscillation ob tain ed from the sys
tem in (4.5.51) can also be ob tain ed by setting the input to zero and setting the
initial co n d ition s to y ( - l ) = 0, v (2) = -A sin a> o. Thus the zero-input response
to the secon d-ord er system described by the h om ogen eou s differen ce equation
y{n) = - a i y ( n - 1) - y( n - 2)

(4.5.52)

with initial con d ition s y ( - l ) 0 and >(2) = A sin two, is exactly the sam e as
the resp onse o f (4.5.51) to an im pulse excitation. In fact, the d ifferen ce equation
in (4.5.52) can b e ob tain ed directly from the trigonom etric identity

a +8

a - B

sin a + sin fi = 2 sin - cos -

(4.5.53)

w here, by definition, or = (n + !)&*>, 0 = (n l)^ o , and y( n) = s i n( n + l)a>o-

354

Frequency Analysts of Signals and Systems

Chap. 4

In som e practical applications involving m odulation of tw o sinusoidal carrier

signals in phase quadrature, there is a need to generate the sinusoids A sin ojqn
and A cosaion. T hese signals can be gen erated from the so-called coupl ed- f orm
oscillator, which can be ob tain ed from the trigonom etric form ulas
co s(a + ft) = cos a cos ft sin a sin ft
sin (a + ft) = sin a cos ft + cos a sin ft
where, by definition, a =

ft = u>o, and

yr (n) = COsnajoK(rc)

(4.5.54)

y , (n ) = sinna>t)U(fj)

(4.5.55)

Thus we obtain the tw o cou p led d ifference equations

)'(() = (COS a>o)}<( - 1) - (s in a jo )y , ( - 1)

(4.5.56)

y.,() = (sin o>o)yt-(n - 1) + (cosw o)yr(n - 1)

(4.5.57)

which can also be expressed in matrix form as

yAn)

_vt(n) _

cos co{) sin too

sin t?o cos lo{)

yt.(n - 1)

(4.5.58)

yAn ~

T h e structure for the realization of the coupled-form oscillator is illustrated in

Fig. 4.61. We note that this is a tw o-output system which is not driven by any input,
but which requires the initial con d ition s y<( ]) = /Icosaio and y.f ( 1) = -A sin a> o
in order to begin its self-sustaining oscillations.
Finally, it is interesting to note that (4.5.58) corresponds to vector rotation
in the tw o-dim ensional coordinate system with coordinates yc(n) and y.T(n). A s a
con sequ en ce, the coupled-form oscillator can also be im plem en ted by use o f the
so-called C O R D IC algorithm [see the b ook by K ung et al. (1985)].

Figure 4.61 Realization of the

coupled-form oscillator.

Sec. 4.6

inverse Systems and Deconvolution

355

4.6 INVERSE SYSTEMS AND DECONVOLUTION

A s we have seen , a linear tim e-invariant system takes an input signal v(/j) and
produces an output signal y (n ), which is the con volu tion of * ( ) with the unit
sam ple response h{n) of the system . In m any practical applications w e are given
an output signal from a system w hose characteristics are unknow n and we are
asked to determ ine the input signal. For exam p le, in the transm ission of digital
inform ation at high data rates over telep h on e channels, it is w ell known that the
channel distorts the signal and causes intersym bol interference am ong the data
sym bols. T h e intersym bol interference m ay cause errors when w e attem pt to re
cover the data. In such a case the problem is to design a corrective system which,
w hen cascaded with the channel, produces an output that, in som e sense, corrects
for the distortion caused by the channel, and thus yields a replica of the desired
transm itted signal. In digital com m unications such a corrective system is called
an equalizer. In the general con text of linear system s theory, how ever, w e call
the corrective system an inverse s y s t e m , because the corrective system has a fre
quency resp onse which is basically the reciprocal o f the frequency response o f
the system that caused the distortion. F urtherm ore, since the distortivc system
yields an output y i n ) that is the con volu tion of the input x ( n ) wiih the im pulse
response h(n). the inverse system operation that takes y(/i) and produces a ( / i ) is
called deconvol ut i on.
If the characteristics o f the distorlive system are unknow n, it is often nec
essary, when p ossib le, to excite the system with a known signal, observe the
output, com pare it with the input, and in som e m anner, determ in e the charac
teristics o f the system . For exam ple, in the digital com m unication problem just
described, w here the frequency response of the channel is unknow n, the m ea
surem ent o f the channel frequency response can be accom p lish ed by transm itting
a set o f equal am plitude sinusoids, at different freq u en cies with a specified set
o f phases, within the frequency band o f the channel. The channel will atten
uate and phase shift each o f the sinusoids. By com paring the received signal
with the transm itted signal, the receiver obtains a m easu rem en t o f the channel
frequency resp onse which can be used to design the inverse system . The pro
cess o f determ ining the characteristics of the unknow n system , either h( n) or
H(co), by a set o f m easurem ents perform ed on the system is called syst em identi
fication.
T h e term decon volu tion " is often used in seism ic signal p rocessing, and
m ore generally, in geophysics to describe the op eration of separating the input
signal from the characteristics o f the system which w e wish to m easure. T h e decon volu tion operation is actually in tend ed to identify the characteristics o f the
system , which in this case, is the earth, and can also be view ed as a system id en
tification problem . T he inverse system , in this case, has a frequency response
that is the reciprocal o f the input signal spectrum that has b een used to excite the
system .

356

Frequency Analysis of Signals and Systems

Chap. 4

4.6.1 Invertibility of Linear Time-Invariant Systems

A system is said to be invertible if there is a o n e-to -o n e corresp on d en ce betw een
its input and output signals. This definition im plies that if w e know the output
sequ en ce y(n), oc < n < oc, of an invertible system T , w e can un iqu eiv determ ine
its input A(n), oc < n < oo. T he inverse syst em with input v() and output x{n)
is d en oted by T ~ l . Clearly, the cascade con n ection o f a system and its inverse is
equivalent to the identity system , since
w( n) = 7

1[>'()] = T ~ x [T[x{n)]) - x(n )

(4.6.1)

as illustrated in Fig. 4.62. For exam ple, the system s defined by the input-output
relations y{n) = ax ( n ) and y( n) = x ( n 5) are invertible, w h ereas the input-output
relations y( n) = x 2(n) and y{n) = 0 represent noninvertible system s.
A s indicated above, inverse system s are im portant in m any practical appli
cations. including geop h ysics and digital com m unications. Let us begin by con
sidering the problem of determ ining the inverse o f a given system . W e limit our
discussion to the class o f linear tim e-invariant discrete-tim e system s.
N ow . suppose that the linear tim e-invariant system T has an im pulse response
h(n) and let h t (n) d en ote the im pulse response o f the inverse system T _ l . Then
(4.6.1) is equivalent to the con volu tion equation
U(n) = h / ( n) * h( n) * x ( n ) = x( n)

(4.6.2)

h(n) * h/ ( n) = S(n)

(4.6.3)

But (4.6.2) im plies that

The convolution equation in (4.6.3) can be used to solve for h/ ( n) for a given
h(n). H ow ever, the solution o f (4.6.3) in the tim e dom ain is usually difficult. A
sim pler approach is to transform (4.6.3) into the z-dom ain and so lv e for T _1. Thus
in the z-transform dom ain, (4.6.3) becom es
H{z )H, (z) = 1
and therefore the system function for the inverse system is

*,(:)=^

(4.6.4)

If H ( z ) has a rational system function

H( z) =

(4.6.5)
/1(c)

Identity system

v( n )
T

T -i

Direct
system

Inverse
system

n '(n ) = x(n)

Figure 4.62 System T in cascade with

its inverse T _1.

Sec. 4.6

357

Inverse Systems and Deconvolution

then
(4.6.6)
Thus the zeros o f H ( z ) b ecom e the p oles of the inverse system , and vice versa.
Furtherm ore, if H{ z ) is an F IR system , then Hi ( z) is an all-p ole system , or if H{z )
is an all-p ole system , then H s {z) is an F IR system .
Example 4.6.1
Determine the inverse of the system with impulse response
h(n) = UyuOO
Solution

The system function corresponding to h(n) is

H(z) -

1-

This system is both causal and stable. Since H(z) is an all-pole system, its inverse is
FIR and is given by the system function
h ,{ z)

= i - h-1

Hence its impulse response is

= &(n)

1)

Example 4.6.2
Determine the inverse of the system with impulse response
h(n) = <5(n) j<5(h 1)

Solution

This is an FIR system and its system function is

H(z) = 1 - k ~ '

ROC: |z| > 0

The inverse system has the system function

Thus H/ ( z) has a zero at the origin and a pole at z = j- In this case there are two
possible regions of convergence and hence two possible inverse systems, as illustrated
in Fig. 4.63. If we take the ROC of Hi(z) as |j| > j, the inverse transform yields

which is the impulse response of a causal and stable system. On the other hand, if
the ROC is assumed to be |j| < | , the inverse system has an impulse response

In this case the inverse system is anticausal and unstable.

358

Frequency Analysis of Signals and Systems

Chap. 4

Figure 4.63 Two possible regions of

convergence for H (z) = z/(z - Y>-

(b)

W e observe that (4.6.3) cannot be solved uniquely by using (4.6.6) unless we

specify the region of con vergen ce for the system function o f the inverse system.
In som e practical applications the im pulse response h(n) d o es not possess a
z-transform that can be expressed in closed form. A s an alternative w e may solve
(4.6.3) directly using a digital com puter. Since (4.6.3) d oes not. in general, possess
a unique solution, we assum e that the system and its inverse are causal. Then
(4.6.3) sim plifies to the equation
h(k)hi(rt k) S(/i)

(4.6.7)

By assum ption, h/ ( n) = 0 for n < 0. For n = 0 we obtain

h,(0 ) = l / h ( 0 )

(4.6.8)

T he values o f hi {n) for n > 1 can be ob tain ed recursively from the equation
^

= ~

h(k)h/(n-k)

mo)

n21

This recursive relation can easily be program m ed on a digital com puter.

( A -6

Sec. 4.6

359

Inverse Systems and Deconvolution

T here are two problem s associated with (4.6.9). First, the m ethod d oes not
work if h(0) = 0. H ow ever, this problem can easily be rem ed ied by introducing
an appropriate delay in the right-hand side o f (4.6.7), that is, by replacing <50) by
S(n m) , where m = 1 if *(0) = 0 and /i( l) ^ 0, and so on. Second, the recursion
in (4.6.9) gives rise to rou n d -off errors which grow with n and, as a result, the
num erical accuracy o f h(n) d eteriorates for large n.
Example 4.6.3
D eterm ine the causal inverse of the FIR system with impulse response
h(n) = S(n) aS(n 1)

Solution

Since /i(0) = 1. h(1) = a, and h(n) = 0 for n > a , we have

MO) = 1/MO) = 1
and
h/(n) = a M n _ 1 )

n 1

Consequently,

M l) = a .

M 2) = a 2..........

h,(n) =a"

which corresponds to a causal IIR system as expected,

4.6.2 Minimum-Phase, Maximum-Phase, and

Mixed-Phase Systems
T h e invertibility o f a linear tim e-invariant system is intim ately related to the char
acteristics o f the phase spectral function o f the system . T o illustrate this point, let
us consider tw o FIR system s, characterized by the system functions
H\ {z) = 1 + j ; -1 = z_1(z + \ )

(4.6.10)

Hj i z ) =

(4,6.11)

z ! ( |z + 1)

The system in (4.6.10) has a zero at z =

and an im pulse response fc(0) = 1,
/i(l) = 1/2. T h e system in (4.6.11) has a zero at z = 2 and an im pulse response
h(0) = 1/2, /i(1) = 1, which is the reverse of the system in (4.6.10). This is due to
the reciprocal relationship b etw een the zeros o f H\ ( z) and
In the frequency dom ain, the tw o system s are characterized by their fre
quency response functions, which can be expressed as
|ffi(<y)| = |H 2(o))! = y | + cos to

(4.6.12)

sin w
-j------------| + cos OJ

(4.6.13)

_i

sin to
------------2 + cos to

(4.6.14)

and
i((w) = to + tan

2 (cl>) = to + tan

The m agnitude characteristics for the tw o system s are identical becau se the zeros
o f Hi ( z ) and Hi ( z ) are reciprocals.

360

Frequency Analysis of Signals and Systems

Chap. 4

fr|I co)

94a.)
i

(b)

Figure 4.64 Phase response

charactcnslics for the systems in (4.6.10)
and (4.6.11).

The graphs o f 0 ] (co) and (~)2 (cv) are illustrated in Fig. 4.64. W e observe that
the phase characteristic 0i(cd ) for the first system begin s at zero phase at the fre
quency w = 0 and term inates at zero phase at the frequency cv 7t. H en ce the net
phase change, 0 i( ;r ) - 0i(O ) is zero. On the other hand, the p h ase characteristic
for the system with the zero outside the unit circle undergoes a n et phase change
0 ;(;r ) 2(0) = n radians. A s a con seq u en ce o f these different p h ase character
istics, w e call the first system a m i n i mu m - p h a s e s y s t e m and the secon d system is
called a m a x i mu m - p h a s e system.
Th ese definitions are easily exten d ed to an F IR system o f arbitrary length.
T o be specific, an F IR system o f length M + 1 has M zeros. Its frequency response
can be expressed as
H(co) = bQ(\ - z \ e ~ iw)( 1 - z 2e ~ n ( ! - z Me ~ J)

(4.6.15)

where {;,} d en ote the zeros and bo is an arbitrary constant. W h en all the zeros
are inside the unit circle, each term in the product o f (4.6.15), corresponding to
a real-valued zero, will undergo a net phase change o f zero b etw een a> = 0 and
(*) = n . A lso , each pair of com p lex- conjugate factors in H(a>) will undergo a net
phase change o f zero. T herefore,
^ H ( n ) - ^H(O) = 0

(4.6.16)

and h en ce the system is called a m inim um -phase system . O n the o th er hand, when
all the zeros are outside the unit circle, a real-valued zero will con trib ute a net

Sec. 4.6

361

Inverse Systems and Deconvolution

phase change o f tt radians as the frequency varies from w = 0 to co it. and each
pair o f com p lex-con ju gate zeros will contribute a net phase change of 2 tt radians
over the sam e range o f ai. T herefore.
iL H ( jt )

- ^ H ( O ) = M tt

(4.6.17}

which is the largest possible phase change for an FIR system with M zeros. H ence
the svstem is called m axim um phase. It follow s from the discussion above that
4- Hmax{7T) > 4 tf m,n(jr)

(4.6.18)

If the FIR system with M zeros has som e o f its zeros inside the unit circlc
and the rem aining zeros ou tsid e the unit circle, it is called a mi x e d - p h a s e system
or a n o n m i n i m u m - p h a s e syst em.
Since the derivative o f the phase characteristic o f the system is a m easure
of the tim e d elay that signal frequency com p onents undergo in passing through
the system , a m inim um -phase characteristic im plies a m inim um delay function,
while a m axim um -phase characteristic im plies that the delay characteristic is also
m axim um .
N ow suppose that we have an FIR system with real coefficients. .Then the
m agnitude square value of its frequency response is
|t f ( w ) |: =

)|r^ ,-

(4.6.19)

This relationship im plies that if we replace a zero

o f the system by its inverse
1 /zk- the m agnitude characteristic o f the system d oes not change. Thus if we re
flect a zero zt that is inside the unit circle into a zero I/:* ou tsid e the unit circle,
we se e that the m agnitude characteristic o f the frequency response is invariant to
such a change.
It is apparent from this discussion that if \ H ( c o ) \ 2 is the m agnitude square
frequency resp onse o f an F IR system having M zeros, there are 2 M possible con
figurations for the M zeros, som e of which are inside the unit circle and the re
m aining are ou tsid e the unit circle. Clearly, one configuration has all the zeros
inside the unit circle, which corresponds to the m inim um -phase system . A sec
ond configuration has all the zeros outside the unit circle, which corresponds to
the m axim um -phase system . T h e rem aining 2 W 2 configurations correspond to
m ixed-phase system s. H ow ever, not all 2 M - 2 m ixed-phase configurations n ec
essarily correspond to F IR system s with real-valued coefficients. Specifically, any
pair o f com p lex-con ju gate zeros result in on ly two possib le configurations, w hereas
a pair o f real-valued zeros yield four possib le configurations.
Example 4.6.4
D eterm ine the zeros for the following FIR systems and indicate whether the system
is minimum phase, maximum phase, or mixed phase.
tf,(z) = 6 + z_1 - z~2
H 2{ z )

1 - r

1 - 6 ;" 2

362

Frequency Analysis of Signals and Systems

Chap. 4

H:Az) = 1 - ^c"1 - $z~z

HAZ) =

Solution

1+

f r ' -

By factoring the system functions we find the zeros for the four systems

are
H]{z) * Ci.: = * { * minimum phase
H 2 (z )

- ci,: = - 2 , 3 maximum phase

H:,(z) * ci.: = - J. 3 * mixed phase

Hi\z) *Ci.: = 2, t * mixed phase
Since the zeros of the fouT systems are reciprocals of one another, it follows that all
four systems have identical m agnitude frequency response characteristics but different
phase characteristics.
The m inim um -phase property o f F I R system s carries over to I I R system s that
have rational system functions. Specifically, an I I R system with system function
B(z)
H( z ) =

(4.6.20)

A (c )

is called m i n i m u m p h a s e if all its poles and zeros are inside the unit circle. For a
stable and causal system [all roots of A (c) fall inside the unit circle] the system is
called m a x i m u m pha s e if all the zeros are outside the unit circle, and m i x e d phase
if som e, but not all. o f the zeros are ou tsid e the unit circle.
This discussion brings us to an im portant point that should be em phasized.
That is. a stable p o le-zero system that is m inim um phase has a stab le inverse which
is also minim um phase. The inverse system has the system function
= d(z)

(4 -6 -21)

H en ce the m inim um -phase property of H ( z ) ensures the stability o f the inverse

system H ~ l (z) and the stability o f H( z ) im plies the m inim um -phase property of
H ~ \ z ) . M ixed-phase system s and m axim um -phase system s result in unstable in
verse system s.

D ecom p osition

of

n on m in im u m -p h ase

p o le -z e r o

sy stem s.

Any

nonm inim um -phase p o le -z e r o system can be expressed as

(z) = /W z)tfa p (z)

(4.6.22)

w here
is a m inim um -phase system and / / ap(z) is an all-pass system . We
dem onstrate the validity o f this assertion for the class o f causal and stable systems
w-ith a rational system function H ( z ) = B ( z ) / A ( z ) . In general, if B(z) has one
or m ore roots ou tsid e the unit circle, w e factor B(z) in to the product B i( z ) # 2 (z)>
w here B i(z) has all its roots inside the unit circle and B 2(z) has all its roots outside

Sec. 4.6

Inverse Systems and Deconvolution

the unit circle. T h en

m inim um -phase system

363

has all its roots inside the unit circle. W e define the
B i i - J B j i z - 1)
Mz)

and the all-pass system

Hw (z) =

Biiz)
B 2(z - 1)

Thus H ( z ) Fiminiz)Hap(z) N ote that / / ap(z) is a stable, all-pass, m axim um -phase

system .

Group delay of nonminimum-phase system. B ased on the d ecom p osi

tion o f a nonm inim um -phase system given by (4.6.22), w e can express the group
delay o f H( z ) as
Tg(o>) = rin(co) + ^apM

(4,6.23)

Since r r (u>) > 0 for 0 < to < n , it follow s that 1 ^( 0;) > rn(o;), 0 < a>< tt. From
(4.6.23) w e co n clu d e that am ong all p o le -z e r o system s having the sam e m agnitude
response, the m inim um -phase system has the sm allest group delay.

Partial energy o f nonminimum-phase system.

T he partial energy o f a

causal system with im pulse response h(n) is defined as

Cn) = W * ) I 2
t=u

(4.6.24)

It can be show n that am ong all system s having the sam e m agnitude response and
the sam e total en ergy ( 0 0 ), the m inim um -phase system has the largest partial
energy [i.e., E min(n) > E( n) , where Emj(n) is the partial energy o f the m inim um phase system ].

4.6.3 System Identification and Deconvolution

Suppose that w e excite an unknow n linear tim e-invariant system with an input se
quen ce x ( n ) and w e observe the output seq u en ce y( n). From the output sequ en ce
w e wish to determ ine the im pulse resp onse o f the unknow n system . This is a prob
lem in s y s t em identification, which can be solved by deconvol ut i on. Thus w e have
y(n) = h( n) * *(n)
^

(4-6.25)
h( k ) x ( n k)

=
k=oo

A n analytical solution o f the d econ volu tion problem can b e obtained by

w orking with the z-transform o f (4.6.25). In the z-transform dom ain w e have
Y( z ) = H( z ) X( z )

364

Frequency Analysis of Signals and Systems

Chap, 4

an d h en ce
K(;)
HrJ = -
* (;)

(4.6.26)

X u ) and
are th e ^ -tra n sfo rm s of the av ailab le in p u t signal x(/;) a n d the
o b se rv e d o u tp u t signal yi n) , resp ectiv ely . T his a p p ro a c h is a p p r o p ria te only w hen
th e re are clo sed -fo rm ex p ressio n s fo r X ( z ) an d F (c).
Example 4.6.5
A causal system produces the output sequence
n=0
n= 1
otherwise

f 1.
yin) = |
.
I 0.
when excited by the input sequence

n= 0

(1 .
x(n) = i
H i'

I 0,

otherwise

Determ ine its impulse response and its input-outpul equation.

Solution The system function is easily determ ined by taking the .--transforms of ,v(n)
and yin). Thus we have
no

H( z ) = ----- =

1 + 77,:"

1 - 17i-

-------:----- 7

+ i7>"~

1-r
(1 -

)(1 -

Since the system is causal, its ROC is |;| > i. The system is also stable since its poles
lie inside the unit circle.
The input-output difference equation for the system is
y i n ) = -jj;y(/i - 1 ) - ^ y i n - 2 ) + ,r() + y^.v(n - 1 )
Its impulse response is determined by performing a partial-fraction expansion of H(z)
and inverse transforming the result. This com putation yields
hin) = [4(i)" - 3({ )n]u(n)
W e observe that (4.6.26) d eterm in es the unknow n system uniquely if it is
known that the system is causal. H ow ever, the exam ple above is artificial, since
the system response {>()} is very likely to be infinite in duration. C onsequently,
this approach is usually im practical.
A s an alternative, we can deal directly with the tim e-dom ain expression given
by (4.6.25). If the system is causal, w e have
n
y( n) = ^ h{ k) x( n k)
n> 0
*=o

Sec. 4.6

365

Inverse Systems and Deconvolution

and h ence
n-l
y { n )

Hn) =

(4.6.27)

y ~ ^ h ( k ) x ( n

k )
n

>

x(0)

This recursive solu tion requires that jr(0) ^ 0. H ow ever, we n ote again that w hen
(/i(n)) has infinite duration, this approach m ay not be practical unless w e truncate
the recursive solu tion at sam e stage [i.e., truncate {/z()}].
A n o th er m eth od for identifying an unknow n system is b ased on a crosscor
relation technique. R ecall that the in p ut-ou tp u t crosscorrelation function derived
in Section 2.6.5 is given as
oc
r y x ( r 77)

h ( k ) r ( X ( m

k )

h ( n )

r x ! ( m )

(4.6.28)

t=0
where r y x ( m ) is the crosscorrelation seq u en ce o f the input {x(^)J to the system
with the output {>()} o f the system , and r x x { m ) is the autocorrelation sequ en ce
o f the input signal. In the frequency dom ain, the corresponding relationship is
S vx((d)

J-!(io)S.fX (co)

H ( c o ) \X (u >)\2

H en ce
Svi(a>)
H ( w )

---------- =

SX i ( w )

5 Vj(w )
-

1 -

\ X ( a >)\2

(4.6.29)

T h ese relation s suggest that the im pulse response (/?(k)1 or the frequency re
sp on se o f an unknow n system can be determ ined (m easu red ) by crosscorrelating
the input seq u en ce {*()} with the output seq u en ce (y(n)}, and then solvin g the
d econ volu tion problem in (4.6.28) by m eans o f the recursive eq u ation in (4.6,27).
A ltern atively, w e cou ld sim ply com pute the Fourier transform o f (4.6.28) and d e
term ine the freq u en cy response given by (4.6.29). Furtherm ore, if w e select the
input seq u en ce (jc(n)} such that its autocorrelation seq u en ce { ^ ( n ) } , is a unit sam
ple seq u en ce, or eq u ivalen tly, that its spectrum is flat (con stan t) over the passband
o f H(a>), the valu es o f the im pulse resp onse {/?()} are sim ply equal to the values
o f the crosscorrelation seq u en ce {rVJ()}.
In general, the crosscorrelation m eth od described above is an effective and
practical m eth od for system identification. A n oth er practical approach based on
least-squares optim ization is described in C hapter 8.

4.6.4 Homomorphic Deconvolution

T h e com p lex cepstrum , introduced in Section 4.2.7, is a useful to o l for perform ing
d econ volu tion in so m e applications such as seism ic signal processing. T o describe
this m eth od , let us su p p ose that {>>()} is the output seq u en ce o f a linear timeinvariant system w hich is excited by the input seq u en ce (x(n )f. Then
Y( z) = X{ z ) H( z )

(4.6.30)

366

Frequency Analysis of Signals and Systems

Chap. 4

w here H( z ) is the system function. T h e logarithm o f Y ( z ) is

C A z ) = In Y (c)
= in X (c) + In H( z )

(4.6.31)

= C ,(c) + C>,U)
C onsequently, the com plex cepstrum o f the output sequence (y(n)} is expressed
as the sum o f the cepstrum o f |x(n )} and {/i(n)J, that is,
c y( n) = cx (n) + ch(n)

(4.6.32)

Thus w e observe that con volu tion of the tw o seq u en ces in the tim e dom ain corre
sponds to the sum m ation o f the cepstrum seq u en ces in the cepstral dom ain. The
system for perform ing these transform ations is called a h o m o r m o r p h i c sy s t em and
is illustrated in Fig. 4.65.
In som e applications, such as seism ic signal processing and speech signal
processing, the characteristics of the cepstral sequ en ces (c,(/j)} and {c>(n)} are suf
ficiently different so that they can be separated in the cepstral dom ain. Specifically,
suppose that {c* (/?)} has its main com p on en ts (m ain energy) in th e vicinity o f small
values o f n, w hereas |c r(n)} has its com p on en ts concentrated at large values of n.
W e m ay say that |c(n)} is lowpass" and {cx()l is highpass. W e can then sepa
rate {cy,(n)} from {c,(r;)) using appropriate low p ass and h igh pass w indow s, as
illustrated in Fig. 4.66. Thus
ch(n) = c v(rt)wir (rr)

(4.6.33)

cx (n) = c_v(n )Whp(n)

(4.6.34)

and

z-Transform

logarithm

c j.)

;-iransform

Figure 4.65 Homomorphic system for obtaining the cepstrum (cv(n)} of the se
quence (y(n)l-

F igure 4.66 S ep aratin g th e two

cepstral com ponents by low pass" and
highpass windows.

Sec. 4.7

367

Summary and References

w here
U ) p ( J l)

1.
0.

< N,
otherw ise

(4.6.35)

0,

ll < N\
\n\ >

(4.6.36)

UhpO) = 1.1

O nce w e have separated the cepstrum sequ en ces ( o ,( ) } and (c,-(n)} by w indow ing,
the seq u en ces {x(n)} and {/i(n)( are obtained b y p a ssin g (o ,( )| and (c.v(n)) through
the inverse hom om orphic system , shown in Fig. 4.67.
In practice, a digital com puter w ould be used to com p ute the cepstrum o f the
seq u en ce {v()}. to perform the w indow ing functions, and to im plem ent the inverse
h om om orphic system shown in Fig. 4.67. In place o f the --transform and inverse
z-transform . we w ould substitute a special form of the Fourier transform and its
inverse. T his special form , called the discrete Fourier transform, is described in
C hapter 5.
C,<M)

C,(.v)
.--Transform

XO
Com plex
exponential
W(c)

| .v(n)
Inverse

"-rnmt l^nT> (
ir
i /iijn

Figure 4.67 Inverse homomorphic system for recovering the sequences {.kii )| mid
|/j()) from the corresponding cepstru.

4.7 SUMMARY AND REFERENCES

T he Fourier series and the Fourier transform are the m athem atical tools lor an
alyzing the characteristics o f signals in the frequency dom ain, T he Fourier series
is appropriate for representing a periodic signal as a w eigh ted sum of harm oni
cally related sinusoidal com p onents, w here the w eighting coefficients represent the
strengths o f each o f the harm onics, and the m agnitude squared of each w eighting
coefficient represents the pow er of the corresponding harm onic. A s we have in
dicated, the Fourier series is on e o f m any possible orthogonal series expansions
for a p eriodic signal. Its im portance stem s from the characteristic behavior o f LTI
system s, as we shall see in Chapter 5.
T h e Fourier transform is appropriate for representing the spectral charac
teristics o f aperiodic signals with finite energy. T he im portant properties of the
F ourier transform were also presented in this chapter.
There are m any excellen t texts on Fourier series and Fourier transforms.
F or reference, w e include the texts by B racew ell (1978), D avis (1963), Dvm and
M cK ean (1972). and P apoulis (1962).
In this chapter w e also considered the frequency-dom ain characteristics o f
LTI system s. W e show ed that an LTI system is characterized in the frequency
d om ain by its frequency response function H ( ai), w hich is the Fourier transform

368

Frequency Analysis of Signals and Systems

Chap. 4

o f the im pulse response o f the system . W e also observed that the frequency
response function d eterm ines the effect o f the system on any input signal. In fact,
by transform ing the input signal into the frequency dom ain, w e ob served that it is a
sim ple matter to determ ine the effect of the system on the signal and to determ ine
the system output. W hen view ed in the frequency dom ain, an LTI system performs
spectral shaping or spectral filtering on the input signal.
The design o f som e sim ple IIR filters was also considered in this chapter from
the view point o f p o le-zero placem ent. B y m eans o f this m eth od , we w ere able
to design sim ple digital resonators, notch filters, com b filters, ail-pass filters, and
digital sinusoidal generators. T h e design o f m ore com p lex IIR filters is treated in
detail in Chapter 8. which also includes several references. D igital sinusoidal gen
erators find use in frequency synthesis applications. A com p reh en sive treatm ent of
frequency synthesis tech n iqu es is given in the text edited by G orski-P op iel (1975).
Finally, w e characterized LTI system s as either m inim um -phase, maximumphase, or m ixed-phase, d ep en d ing on the p osition o f their p o les and zeros in the
frequency dom ain. U sing these basic characteristics of LTI system s, w e considered
practical problem s in inverse filtering, d econ volu tion , and system identification.
W e concluded with the description o f a d econ volu tion m eth od based on cepstral
analysis o f the output signal from a linear system .
A vast am ount o f technical literature exists on the topics o f inverse filter
ing. d econ volu tion , and system identification. In the context o f com m unications,
svstem identification, and inverse filtering as they relate to channel equalization
are treated in the book by Proakis (1995). D econ volu tion tech n iqu es are widely
used in seism ic signal processing. For reference, w e suggest the papers by W ood
and Treitel (1975), P eacock and T reitel (1969), and the b ook s by R obinson and
Treitel (1978, 1980). H om om orphic d econ volu tion and its ap p lication s to speech
processing is treated in the book by O p p en heim and Schafer (1989).

PROBLEMS
4.1 Consider the full-wave rectified sinusoid in Fig. P4.1.
(a ) Determine its spectrum X a(F).
(b) Compute the power of the signal.
Xa( ! )

Figure P4.1

Chap. 4

369

Problems

(c) Plot the power spectral density.

(d) Check the validity of Parseval's relation for this signal.
4.2 Com pute and sketch the m agnitude and phase spectra tor the following signals (a > 0).
Ae~a' ,
i > 0
la) V- , ' , = '0 .
,< 0
(b) xu(t) = Ae~'r
4.3 Consider the signal
1 -

l r |h .

|r | < t

' 0.

elsewhere

(a) Determ ine and sketch its magnitude and phase spectra. |X (F)| and 2^ X a(F),
respectively.
( b) Create a periodic signal x,.(t) with fundam ental period Tr > 2r. so that ,v(/) =
.v,,(n for |f| < T;,/2. What are the Fourier coefficients ci for the signal x;,u)?
(c) Using the results in p an s (a) and (b). show that a = (1/X;,)X (k/Tr ).
4.4 Consider the following periodic signal:

xin) = { ..., I. (I. 1.2. 3.2. 1.0. I. . . ,|

t

(a) Sketch the signal vt/i) and its magnitude and phase spectra.
( b) Using the results in pari (a), verily Parseval's relation by computing the power
in the time and frequency domains.
4.5 Consider the signal
rr/i
nn
1
3nn
xin ) = 2 -f- 2 cos ------Hcos h c o s ----4
2
2
4
(a) Determ ine and sketch its power density spectrum.
( b) Evaluate the power of the signal.

4.6 Determ ine and sketch the magnitude and phase spectra of the following periodic
signals.
n(n - 2)
(a) ,v<(t)=4sin
3
In
. In
(b) ,v(n) = cos n + sin n
3
^
2n
. 2n
(c) x(n) = cos n sin n

(d) x( n)

= {..., - 2 . - 1 . 0 . 1 . 2 . - 2 . - 1 . 0 . 1.2.... )
t

( e) x i n ) = ( . . . . - 1 . 2 . 1. 2. - 1 . 0 . - 1 . 2 . 1 . 2 . . . . J
(f) x (n) = ( . . . . 0 , 0 . 1 . 1 . 0 . 0 . 0 . 1. 1 . 0 . 0 , . . . )

t
(g) x i n) = 1 . oc < n < oc
( h) x(n) = ( - 1 ) " . - o c < n < oc
4.7 D eterm ine the periodic signals * 0 ), with fundam ental period N = 8. if their Fourier
coefficients are given by:
kn
3kn

370

Frequency Analysis of Signals and Systems

Chap. 4

kn
(b) c * = { Slny -

Q < k <6

k= 7

0,

(c) {c*} = { . . . . O . i . i . l . 2 . 1 . i , i . O .
t

4.8 Two DT signals. j*() and s{(n), are said to be orthogonal over an interval [N\, A^] if
k =t

Y2st ( n) s ?( n) = j 0 *'

k / I

If At = 1. the signals are called orthonorm al,

(a) Prove the relation
N '

ej2*kn/* =

' N,
0.

k = 0, jV, 2N,
otherwise

(b) Illustrate the validity of the relation in part (a) by plotting for every value of

k ~ 1 ,2 ........ 6. the signals st (n) = eJI2,,/('}k" , n = 0, 1 ,___ 5. [Aro/e: For a given k,

n the signal
can be represented as a vector in the complex plane.]
(c) Show that the harmonically related signals
i*(n) = ejan,K'kB
are orthogonal over any interval of length N.
4.9 Compute the Fourier transform of the following signals.
(a) x(n) = u(n) u(n 6)
(b) x(n) = 2"u{-n)
(c) x(n) = (j)"u(n + 4)
(d) x(n) = (a" sin a>on)u(n)
|a | < 1
(e) x(n) = laTsiniuiin
jar| < 1
(f) x(n) =

2 -U )n .
0,

|| 5 4
elsewhere

(g) xin) = { - 2 . - 1 . 0. 1 . 2 )
t

\ n\ <M

^
.
U (2 A / + l - | n | ) .
(h) x(n) =
| 0.

. .
|n| > M

Sketch the magnitude and phase spectra for parts fa), (f), and (g).
4.10 D eterm ine the signals having the following Fourier transforms.
0,
0 < \a>| < <U()
(a) X(a>) = ,
I.

W() < \a>\ < 7T

(b) X (u>) = cos2 a>

(n\ Yt \ - \
O* - &i/2 < M < O*) + to /2
W
1 0,
elsewhere
(d) The signal shown in Fig. P4.10.
4 .11 Consider the signal
x{n) = {1 , 0,- 1 ,2 .3 }
t

Chap. 4

371

Problems
Xitu)

3tt

jt

7r

3n

6 jt

7tt

tt

Figure P4.10

with Fourier transform Xtw) = X's (ct>) + j ( X t (a>)). D eterm ine and sketch the signal
y(n) with Fourier transform
Y(w) = X,(o>) + X R( ^) ei2,il
4.12 Determ ine the signal

8k
10

jc(h )

if its Fourier transform is as given in Fig. P4.12.

0
(a)

X(ai)

(b)
X(u>)

(c )

Figure P4.12

8rr

971

10

To

7T

372

Frequency Analysis of Signals and Systems

Chap. 4

4.13 In Example 4.3.3. the Fourier transform of the signal

-M < n < M
otherwise

1,
0.

x (n) =

was shown to be
M
X (a>) = 1 + 2 2 2 cos am
f!=1
Show that the Fourier transform of
X|(n) =

1.
0.

0< n <M
otherwise

and
x 2(n) =

1.
0,

M < n < 1
otherwise

are, respectively.
] _

Xi(a>) =
X 2{to) =
Thus prove that
X (to) - X 1 (tt>) + X 2 (tt)}

sin(M + \) w
sintw /2)
and therefore.
coswn =

sin(M + \ ) w
sin(a/2 j

4.14 Consider the signal

x(n) = ( - 1 ,2 , - 3 ,2 , -1}
t

with Fourier transform X((u). Compute the following quantities, without explicitly
computing X(a>):
(a) X(0)
(b) AX(co)
(c) f * n X(w) dw
(d) X (jt)
(e) f *JX( co ) \ 2 dco
4.15 The center of gravity of a signal x(w) is defined as
T , nx(n)

yx(n)
7I = OC

and provides a measure of the time delay of the signal.

Chap. 4

373

Problems
Xui

Figure P4.15

(a) Express c in terms of X{<o).

(b) Compute c for the signal x(n) whose Fourier transform is shown in Fig. P4.15.
4.16 Consider the Fourier transform pair
a"u(n)

|ti) < 1

-----------1 at

Use the differentiation in frequency theorem and induction to show that

< + /-!> !
i
1
_v(/i) = --------------- a u(n} - * \ Uo) ----------------/i!(f-l)!
(1 - f
4.17 Let vUil he an arbitrary signal, not necessarily real-valued, with Fourier transform
Express the Fourier transforms of the following signals in terms of X(a>).
(a) _v i)
(h) x ' ( - n )

(f) yin) = vf/1 ) - ,v(/; - 1 )

(d) v(fi I = ' y

xik i

(L) V|H)=.V(2)
_
I x(n/2).
n even
(0 v(n) = L
,.
( 0.
n odd
4.18 Determ ine and sketch the Fourier transforms Xiiw), X 2(co). and Xi(a>) of the following
signals.
(a) -V!(fi) = (1. 1. 1. 1.1)
(b) x2(n) = ( 1. 0. 1 . 0. 1 , 0. 1 . 0, 1 )
(c) x:j n ) = (1 . 0. 0. 1 . 0. 0. 1 . 0. 0, 1 , 0. 0. 1 )
t

(d) Is there any relation between Xi(w). X:(g;). and X3(w)? What is its physical
meaning?
(e) Show that if
**() =
then

, ( r ).

if n j k integer

0.

otherwise

Xt(a>) = X (ka>)

4.19 Let x(n) be a signal with Fourier transform as shown in Fig. P4.19. D eterm ine and
sketch the Fourier transform s of the following signals.

Frequency Analysis of Signals and Systems

374

Chap. 4

Figure P4.19

(b ) A;(n) = x(n) sin(jrn/2)

(d) a4(/7) = x(n) cos nn

(a ) jri(fl) = jr(n) cos(;rn/4)

(c) xy(n) = x( n)cos(nn/2)

Note that these signal sequences are obtained by amplitude modulation of a carrier
co swrn or sinw,,/! by the sequence x(n).
4.20 Consider an aperiodic signal *(n) with Fourier transform X(u>). Show that the Fourier
series coefficients CA
' of the periodic signal

are eiven bv

c;

I .
N

k
N

k = 0. 1........A '- l

4.21 Prove that

X v (w ) ^ '

'
n=~ N

sin w,n
nn

may be expressed as
X v (o j )

1
= ^

r sinf(2A^ + 1 ){w 8f lj \
db
- 0)/2\

.L ,

4.22 A signal jfn ) has the following Fourier transform:

1

X (oj) = ---------------

1 - ae~JW

Determine the Fourier transforms of the following signals:

(b) e*nf2x(n + 2 )
(b) x { 2n)
(d) x(n) cos(0.37rn)
(c) jt() * x(n 1 )
(0 x(n) * x (n)
4.23 From a discrete-time signal x ( n ) with Fourier transform X(cv), shown in Fig. P4.23,
determine and sketch the Fourier transform of the following signals:
,
1 x(n).
n even
( a )v i( n ) =
I 0,
n odd
(b) yiiri) - x(2n)
[ x( nj 2 ).
n even
(c) y,() =
,,
I 0,
n odd
Note that vj (n) = x(n)s(n), where s(n) = {... 0, 1. 0, 1. 0. 1. 0. 1. ...}
t
(a) x(2n + \ )

Chap. 4

Problems

375
Xlw)

ai
Figure P4.23

4.24 The following inpul-output pairs have been observed during the operation of various
svstems:

(C) .t(H) C --- * V(IJ) = _V' " '

(d) .v(u) = e n u(ii) yin) j r ' "

7*
(e) ,v(/i = ,v(/i + Ari ) vin) = yin 4- N21
A',
A;. A't. N2 prime
Determ ine their frequency response if each of the above systems is LTI.
4.25 (a) D eterm ine and sketch the Fourier transform W/ffw) of the rectangular sequence
[I.
1 0.

(I < n < M
otherwise

(b) Consider the triangular sequence

( ) < / i < M/2
Mj2 < 2 < M
otherwise
Determ ine and sketch the Fourier transform Wr (a)) of u'70)) by expressing it as
the convolution of a rectangular sequence with itself.
(c) Consider the sequence
, in) = ^ (l -t- cos ^f-) u'R{n)
Determ ine and skctch Wricu) by using
4.26 Consider an LTI system with impulse response h i n ) =
u(n).
(a) Determ ine and skctch the magnitude and phase response \H(co)\ and
respectively.
(b) D eterm ine and sketch the magnitude and phase spectra for the input and output
signals for the following inputs:

(2) jr(n) = ( . . . . 1.0.0. 1. 1. 1.0. 1. 1. 1,0, 1. ...}

r
4.27 D eterm ine and sketch the magnitude and phase response of the following systems:
(a) y(n) = ^[.v(/i) + xin - 1 )]
(b) yfn) = 4[v(/i) - xin - 1 )]
(c) v(n) =

+ 1) - x(n - 1)]

376

Frequency Analysis of Signals and Systems

(d) y(n) =

+ 1 ) + x(n - 1 )]

(e) v (n )= i[jr(rt) +
(f) y(n) =
(g)
(h)
(i)
(j)
(k)
(I)
(m)
(n)

Chap. 4

jc(/t - 2 ) ]

- x(n - 2 )]

v(n) = j[jr(n) + x(n - 1 ) + x{n - 2 )]

y(n) = x(n) x(n 8)
y(n) = 2x(n 1 ) x(n 2 )
v(n) =
+ x(n - 1) + x(n - 2) + x( - 3)]
y()
y(n)
y(n)
y(n)

=
+ 3x(n - 1) + 3x(n - 2) + x(n - 3)]
= x(n - 4)
= x(n + 4)
= j[x(n) - 2x(n - 1 ) + xin - 2 )]

4.28 An FIR filter is described by the difference equation

y(n) = x(n) + xin - 10)
(a) Compute and sketch its magnitude and phase response.
(b) Determ ine its response to the inputs

TT

/ TT

IT '

(1) x(n) = cos n + 3sm \ ^- n + ^

(2 ) jr(n) = 10 + 5 cos ( ^ - n + y

oc < n < oc
OO <

< oc

4.29 D eterm ine the transient and steadv-statc responses of the FIR filler shown in Fig. P4.29
to the input signal A(n) = 10ejr"',~uin). Let b = 2 and v ( - l ) = v (-2 ) = v(3) =
v(4) = 0.

Figure P4.29
4.30 Consider the FIR filter
v(n) = x ( n ) + x ( n - 4)
(a) Compute and sketch its m agnitude and phase response.
(b) Compute its response to the input
x (n) = cos n + cos n
2
4

oc < n < oc

(c) Explain the results obtained in part (b) in terms of the m agnitude and phase
responses obtained in part (a).

4.31 Determ ine the steady-state and transient responses of the system
y(n) = j [*() x ( n - 2)]

Chap. 4

377

Problems

to the input signal

x(n) = 5 + 3 cos

oo < n < oc

+ 60^

4.32 From our discussions it is apparent that an LTI system cannot produce frequencies
at its output that are different from those applied in its input. Thus, if a system
creates new" frequencies, it must be nonlinear and/or time varying. D eterm ine the
frequency content of the outputs of the following systems to the input signal
4
(a) v (/i) = .v(2n)
(b) y(n) = x 2(n)
<c) y(n) = (cos nn)x(n)
4.33 D eterm ine and sketch the m agnitude and phase response of the systems shown in
Fig. P4.33(a) through (c).

(a)

(b)
8

(c)
Figure P4J3
4.34 Determ ine the magnitude and phase response of the m ultipath channel
y( n) = x (n) + xfn M)
A t what frequencies does H(co) = 0?
4.35 Consider the filter
v() = 0.9v(n - 1) + bx(n)
(a) D eterm ine b so that |H (0)| = 1.
(b) D eterm ine the frequency at which j/ / (cu)| = l/%/2.

378

Frequency Analysis of Signals and Systems

Chap. 4

(c) Is this filter lowpass, bandpass, or highpass?

(d) Repeat parts (b) and (c) for the filter v(n) = 0.9y(n - 1) + O.l.r(n).
4.36* Harmonic distortion in digital sinusoidal generators An ideal sinusoidal generator
produces the signal
x(n) = cos 2nf)n
oc < n < cc
which is periodic with fundam ental period N if /<i = ki,/N and ky, N are relatively
prime numbers. The spectrum of such a pure" sinusoid consist of two lines at k = ko
and k = N ka (we limit ourselves in the fundam ental interval 0 < k < N - 1).
In practice, the approximations made in com puting the samples of a sinusoid of
relative frequency f t result in a certain am ount of power falling into other frequencies.
This spurious power results in distortion, which is referred to as harmonic distortion.
Harmonic distortion is usually m easured in terms of the total harmonic distortion
(TH D), which is defined as the ratio
THD =

spurious harmonic power

total power

(a) Show that

iQn t2
T H D == 1 - 2 P,
where

*-j

n = (I

(b) By using the Taylor approximation

COS 0 = 1 -------- ---------
2!
4!
6!
compute one period of jr(n) for / 0 = 1/96, 1/32, 1/256 by increasing the number
of terms in the Taylor expansion from 2 to 8.
(c) Compute the THD and plot the power density spectrum for each sinusoid in
part (b) as well as for the sinusoids obtained using the com puter cosine function.
Comment on the results.
4.37* Measurement o f the total harmonic distortion in quantized sinusoids

Let x(n) be a

periodic sinusoidal signal with frequency fo = k / N , that is,

x(n) = sin 2jr/on
(a) Write a computer program that quantizes the signal x(n) into b bits or equivalently
into L = 2h levels by using rounding. The resulting signal is denoted by xq(n).
(b) For f a = 1/50 compute the THD of the quantized signals xq(n) obtained by using
b = 4, 6, 8, and 16 bits.
(c) Repeat part (b) for / = 1/100.
(d) Comment on the results obtained in parts (b) and (c).
438* Consider the discrete-time system
y(n) = ay(n 1 ) + (1 a)x(n)
where a 0.9 and y(1) = 0.

n > 0

Chap. 4

379

Problems

(a) Compute and sketch the output y, (n) of the system to the input signals
.v ,(n ) = s in 2 jT /,n

0 ft < 100

where / , =
./? =
(b) Compute and sketch the magnitude and phase response of the system and use
these results to explain the response of the system to the signals given in part (a).
4.39* Consider an LTI system with impulse response h{n) = ( t )1"1
(a) Determ ine and sketch the magnitude and phase response Hi m) and
respectively.
(b) D eterm ine and sketch the magnitude and phase spectra for the input and output
signals for the following inputs:
3,t ii

( 1 ) .v(n ) = cos

. rc < n < cc

(2) .xin) = (....-1, 1. - 1 . 1 . - 1 . 1 . - 1 . 1 . - I . 1, - 1 . 1...-I

T
4.40* Time-domain sampling

(a)
(b)
(c)
(d)
(e)

Consider the continuous-time signal

Compute analytically the spectrum X(F) of a U ) Compute analytically the spectrum of the signal a (/?> = x{nT). T = 1 /F ,.
Plot the magnitude spectrum |X (F)| for Ft, = K) Hz.
Plot the magnitude spectrum (X(F)I for F, = 10. 20, 40. and 100 Hz.
Explain the results obtained in part (d) in terms ol the aliasing effect.

4.41 Consider (he digital filter shown in Fig. P4.41.

(a) Determ ine the input-output relation and the impulse response h(n).
(b) Determ ine and sketch the magnitude ]W(w)| and the phase response 2^ H ( a of
the filter and find which frequencies are completely blocked by the filter.
(c) When w,, = t / 2 , determ ine the output yin) to the input
.v(n) = 3 cos

-*

1 ---- ( +

u - - 2 cos aif,

+ 30 ^

cc < n < oc

y(n>

Figure P4.41

4.42 Consider the FIR filter

y(n) = xin) - xin - 4 )
(a) Compute and sketch its m agnitude and phase response.
(b) Com pute its response to the input
n
n
x ( n ) = cos n + cos n
c c < n < oc
2
4

Frequency Analysts of Signals and Systems

380

Chap. 4

(c) Explain the results obtained in part (b) in terms of the answer given in part (a).
4.43 Determ ine the steady-state response of the system
y(n) = i[.v(n) - x(n - 2 )]
to the input signal
oc < n < oc
4.44 Recall from Problem 4.32 that an LTI system cannot produce frequencies at its output
that are different from those applied in its input. Thus if a system creates new
frequencies, it must be nonlinear and/or time varying. Indicate w hether the following
systems are nonlinear and/or time varying and determ ine the output spectra when the
input spectrum is

(a) _v(n) = j:(2nt

(b) y ( ) = x : (n)
(c) v(n) = (cos nn)xin)
4.45 Consider an LTI svstem with impulse response

(a) Determ ine its system function H(z).

(b) Is it possible to implement this system using a finite number of adders, multipliers,
and unit delays? If yes. how?
(c) Provide a rough sketch of \ H(w)| using the pole-zero plot.
(d) Determine the response of the system to the input
x(n) = (
4.46 An FIR filter is described by the difference equation
yin) = x(n) xin 10)
(a) Compute and sketch its m agnitude and phase response.
(b) Determ ine its response to the inputs
oc < n < oc
oc < n < oc4.47 The frequency response of an ideal bandpass filter is given by

Chap. 4

381

Problems

(a) D eterm ine its impulse response

(b) Show that this impulse response can be expressed as the product of cos(n7t / 4)
and the impulse response of a lowpass filter.
4.48 Consider the system described by the difference equation
v(n) = jy(n I ) + .r(n) 4- |.r(n - 1 >
(a) D eterm ine its impulse response.
(b) D eterm ine its frequency response:
(1) From the impulse response
(2) From the difference equation
(c) D eterm ine its response to the input
I TT

x {n > = cos y r +

.T \

- oc < n < oc

4.49 Sketch roughly the m agnitude |A"u<>)! of the Fourier transforms corresponding to the
pole-zero patterns given in Fig. P4.49.

Figure P4.49
4.50 Design an F IR filter that completely blocks the frequency a*, = rr/4 and then compute
its output if the input is
x(n)

^sin

u(n)

for n = 0, 1 , 2, 3, 4. Does the filter fulfill your expectations? Explain.

382

Frequency Analysis of Signals and Systems

Chap. 4

4.51 A digital filter is characterized by the following properties:

(1) It is highpass and has one pole and one zero.
(2) The pole is at a distance r = 0.9 from the origin of the ; -plane.
(3) Constant signals do not pass through the system.
(a) Plot the pole-zero pattern of the filtei and determ ine its system function H(z).
(b) Compute the m agnitude response \H(cu}\ and the phase response ^ H( t n) of the
filter,
(c) Normalize the frequency response H( cd) so that
= 1.
(d) Determ ine the input-output relation (difference equation) of the filter in the time
domain.
(e) Compute the output of the system if the input is
ex: < n < oc

(You can use either algebraic or geom etncal arguments.)

4.52 A causal first-order digital filter is described by the system function

(a) Sketch the direct form I and direct form II realizations of this filter and find the
corresponding difference equations.
(b) For a = 0.5 and b 0.6, sketch the pole-zero pattern, is the system stable?
Why?
(c) For a = 0.5 and b = 0.5, determ ine bu. so that the maximum value of \H(w)\ is
equal to 1 .
(d) Sketch the m agnitude response \H{co)\ and the phase response 2t//(o>) of the
filter obtained in part (c).
(e) In a specific application it is known that a 0.8. Does the resulting filter amplify
high frequencies or low frequencies in the input? Choose the value of b so as to
improve the characteristics of this filter (i.e., m ake it a better lowpass or a better
highpass filter).
4.53 Derive the expression for the resonant frequency of a two-pole filter with poles at
Pi = r e 6 and p 2 = p*x. given by (4.5.25).
4.54 Determ ine and sketch the magnitude and phase responses of the Hanning filter char
acterized by the (moving average) difference equation
y(n) ~ jx (n ) + \ x( n - 1 ) + j*( - 2 )
4.55 A causal LTI system excited by the input
x(rt') = (j)"(n) + u( n - 1)
produces an output v(n) with r-transform
_ 3 -l

(a) D eterm ine the system function H(z) and its ROC.
(b) Determ ine the output y(n) of the system.
(Hint: Pole cancellation increases the original ROC.)

Chap. 4

383

Problems

4.56 Determ ine the coefficients of a linear-phase FIR filter

v(n) = b^x(n) + btx(n 1) + thx(n 2)
such that:
(a) It rejects completely a frequency com ponent at a* = 2jt/3.
(b) Its frequency response is normalized so that H{0) ~ 1.
(c) Compute and sketch the magnitude and phase response of the filter to check if
it satisfies the requirements.
4.57 D eterm ine the frequency response H(w) of the following moving average filters.
-v(") = 2F

t t

> - * )

1
1
(b) y(ff) = r-x(n + Af) +
AM
2M

J
x(n - k) + ~r~rzx(n - M)
AM
k=- M + i
Y

Which filter provides better smoothing? Why?

4.58 The convolution *(r) of two continuous-time signals *((/) and x2(t), from which at
least one is nonperiodic, is defined by
x(t) = X] (/) * xj(t) j

x\ {X)x2(t k)dk

(a) Show that X(F) = X\ ( F) X2(F), where X, (F) and X 2(F) are the spectra of *1 (r)
and a:(/), respectively.
= ( I'
< X
p'
10.
elsewhere
(c) Determ ine the spectrum of x(t) using the results in part (a).

<b> Com pute .r(r) if jrt(r) =

4.59 Com pute the magnitude and phase response of a filter with system function
H(z) = l + c
If the sampling frequency is F, = 1 kHz, determ ine the frequencies of the analog
sinusoids that cannot pass through the filter.
4.60 A second-order system has a double pole at p li2 = 0.5 and two zeros at
Z\.2 = es3*,A
Using geom etric arguments, choose the gain G of the filter so that |tf(0 )| = 1.
4.61 In this problem we consider the effect of a single zero on the frequency response of
a system. Let ; = re9 be a zero inside the unit circle (r < 1). Then
H,(w) = 1 re]0e~iu>
= 1 r cos(a) B) + j r sin(ti) (?)
(a) Show that the m agnitude response is
\HZ((D)\ = [ 1 - 2 r cosito - 6 ) + r 2f n

or, equivalently,
201og10 |//:(cd)| = 101og10[l 2r cos{ct> 6) + r2]

384

Frequency Analysis of Signals and Systems

Chap. 4

(b) Show that the phase response is given as

0 : (ct>) = tan

-i

rsin(w - 8)
------------------1 - r cos(ct) -

(c) Show that the group delay is given as

r - rcos(w - 8)
1 + r1 2r COS(oj
(d) Plot the magnitude
()]dB. the phase ( oj) and the group delay
and 0 = 0, n j 2, and n.

for r = 0.7

4.62 In this problem we consider the effect of a single pole on the frequency response of
a system. Hence, we let
'<> =

' =1

Show that
[Wr (a>)ljB = ~ |W-(fi))|dD
4 // (u > ) = - Z - H . ( c o )

where H:(<x>) and

are defined in Problem 4.61.

4.63 In this problem we consider the effect of complex-conjugate pair of poles and zeros
on the frequency response of a system. Let
H.{u>) = (1 - reJ*'e~J'")(l - re~il'e~-")
(a) Show that the magnitude response in decibels is
| W - M | d B

1 0 1 o g 1()[ l

2 r

c o s ( o )

# ) ]

+ 101og,[l + r 2 - 2r cos(a) + 6)]

(b) Show that the phase response is given as
: (co) = tan '

rsinUo 8)
l-r c o s (tu -# )

Asin(w + fi)
l- r c o s ( o > + 0)

(c) Show that the group delay is given as

r2 r cos(o> 8)
r2 r c o s ( c j + 8)
= r ------ +
1 + r 2 - 2r cos(to 9 )
1 4- r2 - 2r cos(a> +
(d) If Hp(w) = l / H : (a>), show that

&p(w) - - .(oj)
Tg(a>) =
(e) Plot

p(ci>) and

(w) for r = 0.9, and 8 0, jt/2.

Chap. 4

385

Problems

4.64 D e te rm in e th e 3-dB b a n d w id th o f the filte rs (0 < a < 1)

Which is a better lowpass filter?

4.65 Design a digital oscillator with adjustable phase, that is. a digital filter which produces
the signal
y(n) = cos(<W|, + ft)u(n)

4.66 This problem provides another derivation of the structure for the coupled-form os
cillator by considering the system
y i n ) = av(ri 1 ) 4 xi / i )

for a e' ".

Let xi/i) be real. Then yi n} is complex. Thus
yin) =

./'/()

(a) Determ ine the equations describing a system with one input x i n) and the two
outputs y t i U t ) and y f Ui ) .
(b) Determ ine a block diagram realization
(c) Show that if ,v() = bin), then
y^in) = (cosw()'i ii/tn I
y/ {ti) = (sin mull )u(n)
(d) Compute ynin), \/in). n = 0. 1....... 9 for &j(, = tt/6. Compare these with the true
values of the sine and cosine,
4.67 Consider a filter with system function
(1 -

1 -

H(Z) = h\ --------:------j--- ;-------------- r( 1 reJU

I"z )(1 re~,w"z )
(a) Sketch the pole-zero pattern.
<b) Using geom etric arguments, show that for r ^ 1, the system is a notch filter and
provide a rough sketch of its m agnitude response if ttx> = 60 .
(c) For (i>n = 60 , choose bo so that the maximum value of \Hia>)\ is 1.
(d) Draw a direct form II realization of the system
(e) D eterm ine the approximate 3-dB bandwidth of the system.
4.68* Design an FIR digital filter that will reject a very strong 60-Hz sinusoidal interference
contaminating a 200-Hz useful sinusoidal signal. Determ ine the gain of the filter so
that the useful signal does not change amplitude. The filter works at a sampling
frequency F, = 500 samples/s. Compute the output of the filter if the input is a 60-Hz
sinusoid or a 200-Hz sinusoid with unit amplitude. How does the perform ance of the
filter compare with your requirem ents?
4.69 Determ ine the gain bo for the digital resonator described by (4.5.28) so that
I//(n) I = 1.

386

Frequency Analysis of Signals and Systems

Chap. 4

4.70 D em onstrate that the difference equation given in (4.5.52) can be obtained by apply
ing the trigonometric identity
a + [i
a
cos or + cos p = 2 cos - cos -
where a (n-t-Dwu, ~ (n l)a>o, and v(n) = coswon. Thus show that the sinusoidal
signal x(n) A co sco^n can be generated from (4.5.52) by use of the initial conditions
v(1) = A cos an) and y (2) = j4cos2a>o.
4.71 Use the trigonometric identity in (4.5.53) with a = na>o and (i = (n 2)a^t to derive the
difference equation for generating the sinusoidal signal y(n) = A sin no*,. Determine
the corresponding initial conditions.
4.72 Using the --transform pairs 8 and 9 in Table 3.3. determine the difference equations
for the digital oscillators that have impulse responses h(n) = A cosna>it(n) and h(n) =
A sin ncDf>u(n), respectively.
4.73 Determ ine the structure for the coupled-form oscillator by combining the structure
for the digital oscillators obtained in Problem 4.72.
4.74 Convert the highpass filter with system function
H(z) =

az

a < 1

into a notch filter that rejects the frequency a*, = tt/ 4 and its harmonics.
(a) Determ ine the difference equation.
(b) Sketch the pole-zero pattern.
(c) Sketch the magnitude response for both filters.
4.75 Choose L and M for a lunar niter that must have narrow passbands at (k AF)
cycles/dav. where k = i. 2, 3 ,... and A F = 0.067726.
4.76 (a) Show that the systems corresponding to the pole-zero patterns of Fig. 4.58 are
all-pass.
(b) What is the number of delays and multipliers required for the efficient implemen
tation of a second-order all-pass system?
4.77 A digital notch filter is required to remove an undesirable 60-Hz hum associated with
a power supply in an ECG recording application. The sampling frequency used is
Fs = 500 samples/s. (a) Design a second-order FIR notch filter and (b) a secondorder pole-zero notch filter for this purpose. In both cases choose the gain by so that
\H(w)\ = 1 for w = 0.
4.78 Determ ine the coefficients {/?()} of a highpass linear phase FIR filter of length M =
4 which has an antisymmetric unit sample response h{n) = h(M - I - n) and a
frequency response that satisfies the condition

4.79 In an attem pt to design a four-pole bandpass digital filter with desired magnitude
response

0,

elsewhere

Chap. 4

Problems

387

we select the four poles at

/>,, = 0 .8 e-MT"
and four zeros at

(a) Determ ine the value of the gain so that

HDb
(b) Determ ine the system function H(:).

(c) Determ ine the magnitude of the frequency response H(a>) for 0 < a> < tt and
compare it with the desired response \Hd(w)[.
4.80 A discrete*time system with input
and output v ( n ) is described in the frequency
domain by the relation
V(oj) v

dXiuj)
~ 1 X (as) -t-

das

(a) Compute the response of the svstem to the input x i n ) =

(b ) C heck if the system is LTI and stable.

4.81 Consider an ideal lowpass filter with impulse response It in) and frequency response
I 1.

K"! 5

a),

I 0,

CIJ,

|< D | <

Hiw) = |

<

TT

What is the frequency response of the iiher defined by

X<n) =

f).

n ev e n

n odd

4.82 Consider the system shown in Fig. P4.S2. Determ ine its impulse response and its
frequency response if the system H(a>) is:
(a) Lowpass with cutoff frequency w,.
(b) Highpass with cutoff frequency o j , .

xi n )
Hiw)

I'

4.83 Frequency inverters have been used for many years for speech scrambling. Indeed,
a voice signal x(n) becomes unintelligible if we invert its spectrum as shown in
Fig. P4.S3.
(a) Determ ine how' frequency inversion can be perform ed in the time domain.
(b) Design an unscrambler. (Hint: The required operations are very simple and can
easily be done in real time.)

388

Frequency Analysis of Signals and Systems

Chap. 4

X(a>)

-x

0
(b)

Figure P4.83

(a) Original spectrum;

(b) frequency-inverted spectrum.

4.84 A lowpass filter is described by the difference equation

v(n) = 0.9v(/i - 1) + 0.1x(n)
(a) By performing a frequency translation of n f l . transform the filter into a bandpass
filter.
(b) What is the impulse response of the bandpass filter?
(c) What is the major problem with the frequency translation m ethod for transform
ing a prototype lowpass filter into a bandpass filter?
4.85 Consider a system with a real-valued impulse response h(n) and frequency response
H(w) = \H (a>)\em -'
The quantity
D =

2 2 2^ ( n )
n = ~ oc

provides a m easure of the effective duration of h(n).

(a) Express D in terms of H(w).
(b) Show that D is minimized for 0(a>) = 0.
4.86 Consider the lowpass filter
v(n) = a y { n 1) + fejr(n)

(a)
(b)
(c)
<d)

0 < a < 1

D eterm ine b so that |//{0)| = 1.

D eterm ine the 3-dB bandwidth an, for the normalized filter in part (a).
How does the choice of the param eter a affect uyf!
Repeat parts (a) through (c) for the highpass filter obtained by choosing - 1 <
a < 0.

4.87 Sketch the magnitude and phase response of the m ultipath channel
y (n ) = x ( n ) + a jr(n M )
fo r a < < 1.

a > 0

Chap. 4

389

Problems

4.88 Determ ine the system functions and the pole-zero locations for the systems shown in
Fig. P4.88(a) through (c). and indicate w hether or not the systems are stable.
l

lc)
Figure P4.88

4.89 D eterm ine and sketch the impulse response and the magnitude and phase responses
of the FIR filter shown in Fig. P4.89 for b = 1 and b = -1 .

4.90 Consider the system

v(n) = x(n) - 0.95.r(fl - 6)
(a)
(b)
(c)
(d)

Sketch its pole-zero pattern.

Sketch its magnitude response using the pole-zero plot.
D eterm ine the system function of its causal inverse system.
Sketch the magnitude response of the inverse system using the pole-zero plot.

4.91 Determ ine the impulse response and the difference equation for all possible systems
specified by the system functions

390

Frequency Analysis of Signals and Systems

(a)

H( Z)

1 _ rZ
_'_

Chap. 4

,_2

0 " W = ! _ eL z-4

0< < 1

4.92 D eterm ine the impulse response of a causal LTI system which produces the response
v(n) = {1. - 1 .3 . - 1 ,6 )
t
when excited by the input signal
x(n) = (1,1,2}
t
4.93 The system
y(n) = i_v(n - 1) + x(n)
is excited with the input
x(n) = (j)"(n)
Determ ine the sequences aj t (/), rhh(l), r,y(i). and rvy(l).
4.94 Determ ine if the following FIR systems are minimum phase.
(a) h(n) = {10,9. - 7 , - 8 , 0 , 5 . 3}
r
(b) h(n) = (5,4, - 3 . - 4 , 0 ,2 ,1 )
t
4.95 Can you determ ine the coefficients of the all-pole svstem

*=i
if you know its order N and the values /?(0), /i(l)........h ( L - l ) of its impulse response?
How? W hat happens if you do not know N1
4.96 Consider a system with impulse response
h(n) = baS(n) + bi$(n D) + ^<5(n 2D)
(a ) Explain why the system generates echoes spaced D samples apart.

(b) Determ ine the m agnitude and phase response of the system.
(c) Show that for \b0 + b2\ < < |>]|, the locations of maxima and minima of \H(a>)
are at
k
w = n
k = 0. 1, 2. . . .
D
(d) Plot |//(w )| and
for b$ 0.1, b\ = 1, and b2 = 0.05 and discuss the results.
4.97 Consider the pole-zero system
B(z)
1 + bz * v'
W(z) = A(z) = T
1T
+ -----a z 1r = Y
l h^ z
(a) Determine >i(0), >i(l), h(2), and h(3) in terms of a and b.
(b) Let rhH(l) be the autocorrelation sequence of h(n). D eterm ine rkh{0), /-^(l), rw,(2),
and r/,h(3) in terms of a and b.

Chap. 4

391

Problems

4.98 Let
be a real-valued minimum-phase sequence. Modify xin I to obtain another
real-valued minimum-phase sequence y(/i) such that y(0) = x(0) and y(n) = |jr(n)|.
4.99 The frequency response of a stable LTI system is known to be real and even. Is the
inverse system stable?
4.100 Let h(n) be a real filter with nonzero linear or nonlinear phase response. Show that
the following operations are equivalent to filtering the signal x(n) with a zero-phase
filter.
(a) g(n) Ji(n) * ,v(n)
/ (h ) = h (/ t ] * g t - i i )

yon = f( ~n)
(b)

g(n) h(n) *
/ (r?) = h{n) * x ( n)

y(nl = ain) + / ( - i i )
(Hint: D eterm ine the frequency response of the composite system _v(/i) = //[.v()].)
4.101 Cheek the validity of the following statements:
(a) The convolution of two minimum-phase sequences is always mimmum-phase se
quence.
(h) The sum of two minimum-phase sequences is always minimum phase.
4.102 Determ ine the minimum-phase syslem whose squared magnitude response is given
by:
- cos w
^

(a) i H(</)}'- =

(b )

I H (u>)',: =

(i

a - ) - 2a cos w

4.103 Consider an FIR syslem with the following system function:

H(z) = (1 - 0 . 8 r '7V ' K l

- l..v " T,V 1HI - L S ^ ' V 1)

(a) Determ ine all systems that have the same magnitude response.
minimum-phase system?
(b) Deierm ine the impulse response of al) systems in part (a).
(c) Plot the partial energy

Which is the

i=t>
for every system and use il lo identify the minimum- and maximum-phase systems.
4.104 The causal system
//(.-) =

,v

392

Frequency Analysis of Signals and Systems

Chap, 4

(a) Show that by properly choosing A we can obtain a new stable system.
(b) What is the difference equation describing the new system?
4.105* Given a signal x(n), we can create echoes and reverberations by delaying and scaling
the signal as follows
>() = 22 Skx (n ~ kD)
where D is positive integer and gk > g i i > 0.
(a) Explain why the comb filter
H(z) =

1-az-

can be used as a reverberator (i.e.. as a device to produce artificial reverberations).

(Hint: Determ ine and sketch its impulse response.)
(b) The all-pass comb filter
H(z) =

-a
1 - a z - 1

is used in practice to build digital reverberators by cascading three to five such

filters and properly choosing the param eters a and D. Com pute and plot the
impulse response of two such reverberators each obtained by cascading three
sections with the following parameters.
UNIT 1

UNIT 2

Seclion

I)

Section

1
2
3

50
40
32

0.7
0.665
0.63175

1
2
3

50
17
6

0.7
0.77
0.847

(c) The difference between echo and reverberation is that with pure echo there are
clear repetitions of the signal, but with reverberations, there are not. How is this
reflected in the shape of the impulse response of the reverberator? Which unit
in part (b) is a better reverberator?
(d) If the delays D\, D2, Dj in a certain unit are prime numbers, the impulse response
of the unit is more dense." Explain why.
(e) Plot the phase response of units 1 and 2 and comm ent on them.
(f) Plot h(n) for D |, D2, and
being nonprime. W hat do you notice?
More details about this application can be found in a paper by J. A. M oorer, Signal
Processing Aspects of Com puter Music: A Survey," Proc, IEEE, vol. 65, No. 8, Aug.
1977, pp. 1108-1137.
4.106* By trial-and-error design a third-order lowpass filter with cutoff frequency at wc = Jt/9
radians/sample interval. Start your search with
(a) zi = Z2 = Z3 = 0, pi = r, p2J = re,tu' . r = 0.8
(b) r = 0.9, zi = Z2 = Z3 = - 1

Chap. 4

393

Problems

4.107* A speech signal with bandwidth B = 10 kHz is sampled at F2 = 20 kHz. Suppose

that the signal is corrupted by four sinusoids with frequencies
F, = 10, 000 Hz.

F3 = 7778 Hz

F2 = 8889 Hz,

F4 = 6667 Hz

(a) Design a FIR filter that eliminates these frequency components.

(b) Choose the gain of the filter so that |H (0)| = 1 and then plot the log m agnitude
response and the phase response of the filter.
(c) Does the filter fulfill your objectives? Do you recom m end the use of this filter in
a practical application?
4.108* Com pute and sketch the frequency response of a digital resonator with co = t t / 6 and
r = 0.6, 0.9, 0.99. In each case, compute the bandwidth and the resonance frequency
from the graph, and check if they are in agreem ent with the theoretical results.
4.109* The system function of a communication channel is given by
H(z) = (1

- 0 . 9 f - 'u4'T;;-1)(l - 1.5^'afor; - 1)(l - 1.5<- '" tez - 1)

D eterm ine the system function Ht.(;) of a causal and stable compensating system so
that the cascade interconnection of the two systems has a flat magnitude response.
Sketch the pole-zero plots and the m agnitude and phase responses of all systems in
volved into the analysis process. [Hint: Use the decomposition H(z) = Wap(;) Wn,in(;)-]

The Discrete Fourier

Transform: Its Properties and
Applications

F requency analysis o f discrete-tim e signals is usually and most con ven ien tly per
form ed on a digital signal processor, which may be a general-purpose digital com
puter or specially d esigned digital hardware. T o perform frequency analysis on a
d iscrete-tim e signal
w e convert the tim e-dom ain sequ en ce to an equivalent
frequencv-dom ain representation. We know that such a representation is given by
the Fourier transform X(cu) of the seq u en ce |a(>;)). H ow ever, A'u<j) is a contin
uous function o f frequency and therefore, it is not a com p utationally convenient
representation o f the sequence {.v(/i)).
In this section we consider the representation o f a sequ en ce
by sam ples
o f its spectrum X ( uj). Such a frequency-dom ain representation lead s to the discrete
Fourier transform (D F T ), which is a pow erful com putational tool for perform ing
frequency analysis o f discrete-tim e signals.

5.1 FREQUENCY DOMAIN SAMPLING: THE DISCRETE FOURIER

TRANSFORM
B efore w e introduce the D F T , w e consider the sam pling of the F ourier transform of
an aperiodic discrete-tim e sequ en ce. Thus, w e establish the relationship betw een
the sam pled Fourier transform and the D FT.

5.1.1 Frequency-Domain Sampling and Reconstruction of

Discrete-Time Signals
W e recall that aperiodic finite-energy signals have con tinu ou s spectra. Let us
consider such an aperiodic discrete-tim e signal x ( n ) with Fourier transform

(5.1.1)

394

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

395

S uppose that w e sam ple X(a>) periodically in frequency at a spacing o f Sco radians
b etw een successive sam ples. Since X(a>) is period ic with period 2tt, only sam ples
in the fundam ental frequency range are necessary. For con ven ien ce, w e take N
equidistant sam ples in the interval 0 < w < 2tt with spacing Sco = 2 ttf N , as shown
in Fig. 5.1. First, w e consider the selection of N , the num ber o f sam ples in the
frequency dom ain.
If w e evaluate (5.1.1) at u> = 2 n k / N , w e obtain
(5.1.2)
T h e sum m ation in (5.1.2) can be subdivided in to an infinite n um ber of sum m ations,
where each sum con tains N term s. Thus
-i

oc

N-1

IN + N - 1

-)2nkn/\
}=-~yL f}=i A

If we change the index in the inner sum m ation from n to n I N and interchange
the order o f the sum m ation, we obtain the result
(5.1.3)
for k = 0, 1, 2 .........N 1.
T h e signal
OC

(5.1.4)
ob tain ed by the periodic repetition o f x{n) every N sam ples, is clearly periodic
with fundam ental period N. C onsequently, it can be expanded in a Fourier

0
F igure 5.1

kSa)

tt

Swh

Frequency-dom ain sam pling of the F ourier transform .

396

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

series as
A '-l

x p (n) = Y ^ c ke,2nkn/N

n = 0 , 1 .........N - 1

(5.1.5)

k=Q

with Fourier coefficients

l A '-l

k = 0 , 1 ........ N 1

ck = - - ' x p ( n ) e - j2,rkn/N
N n=0

(5.1.6)

U p on com paring (5.1.3) with (5.1.6), we conclude that

1
f 2* \
ck = X l k )
N
\ N J

k = 0 . 1.........A ' - l

(5.1.7)
J

T herefore,
1 ^ ~ ^ / ^7T \
x.,(n) = ' y * ( * ) e llnkntN
\ N /

n = 0, 1........ A ' - l

(5.1.8)

The relationship in (5.1.8) provides the reconstruction of the periodic signal

x r (u) from the sam ples of the spectrum X ( oj). H ow ever, it d o es not imply that
we can recover X(u>) or x{n) from the sam ples. T o accom plish this, we need to
consider the relationship betw een x p (n) and j:().
Since x p(n) is the periodic exten sion of x{n) as given by (5.1.4). it is clear
that x (/ j) can be recovered from x p (n) if there is no aliasing in the tim e domain,
that is, if x( n) is tim e-lim ited to less than the period N of x p(n). This situation is
illustrated in Fig. 5.2, w here without loss of generality, we consider a finite-duration
rtn)

Hitt,....
L

xp(n)
N > L

ITI

i t t t t

IITI

t t t

.
L

JTTTTttt
N

N< L

IT- I TNt t t O I TN T t

TTTt 1-

Figure 5.2 Aperiodic sequence x(n) of length L and its periodic extension for
N > L (no aliasing) and N < L (aliasing).

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

397

sequ en ce x( n) , which is n on zero in the interval 0 < n < L 1. W e ob serve that

when N > L.
x(ti) = Xp(n)

0 < n < N 1

so that x ( n ) can be recovered from x r (n) w ithout am biguity. On the other hand,
if N < L, it is not possib le to recover
from its periodic exten sion due to timed o m a i n aliasing. Thus, w e conclude that the spectrum of an aperiodic discrete-tim e
signal with finite duration L . can be exactly recovered from its sam ples at frequen
cies tot: = 2j rk/ N. if N > L. The procedure is to com pute x p (n). n = 0, 1.........N - 1
from (5.1.8); then
0 < n < N
elsew h ere
and finally, X(io) can be com p uted from (5.1.1).
A s in the case o f con tinu ou s-tim c signals, it is p ossible to express the spectrum
X ( w ) directly in term s o f its sam ples X Q n k / N ) , k = 0. 1........ N 1. T o derive
such an interpolation formula for X{co), we assum e that N > L and begin with
(5.1.8). Since x( n) = x,,(n) for 0 < /; < A' - 1,

k ]

()<//< N - I

(5.1.1(1)

If w e use (5.1.1) and substitute for v(/;), we obtain

A'-I

X(a>) =
n = (l

(5.1.11)

A '-l

2>

The inner sum m ation term in the brackets o f (5.1.11) represents the basic
interpolation function shifted by 2 ttk / N in frequency. Indeed, if w e define
v/tuA-'
N 1 - e-J
sin(o>Ar/2 )
N sin(w /2)

then (5.1.11) can be expressed as

jiuiN

(5.1.12)

398

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

The interpolation function P(co) is not the familiar ( si n 8 ) / 6 but instead, it

is a periodic counterpart of it, and it is due to the periodic nature of A(co). The
phase shift in (5.1.12) reflects the fact that the signal j:(n) is a causal, finite-duration
seq u en ce o f length N. The function sln( a>N/ 2)/ ( N sin(a>/2)) is plotted in Fig. 5.3
for N = 5. We observe that the function P(to) has the property

k =
* = 1 .2 .........A ' - l

(5.1.14)

C onsequently, the interpolation form ula in (5.1,13) gives exactly the sam ple val
ues X ( 2 j r k / N ) for oj = 2 i r k / N . A t all other freq u en cies, the form ula provides a
properly w eighted linear com bination o f the original spectral sam ples.
T h e follow ing exam ple illustrates the frequency-dom ain sam pling of a
discrete-tim e signal and the tim e-dom ain aliasing that results.
Example 5.1.1
Consider the sisinal
xin ) = a"uin)

0 < a < 1

The spectrum of this signal is sampled at frequencies a>t = l i r k / S . k 0. 1....... A '- l.

Determ ine the reconstructed spectra for a = 0.8 when A' = 5 and N = 50.
Solution

The Fourier transform of the sequence x(n) is

Suppose thai we sample X(w) at N equidistant frequencies w*. = 2 x k / N , k = 0,

A' - 1. Thus we obtain the spectral samples

Xiw)
1.0

JV= 5

Figure S.3

Plot o f the function

[sin(ct>W/2)]/[jty sin(tu/2)].

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

399

The periodic sequence xp(n), corresponding to the frequency samples X ( 2 n k / N ) ,

k = 0, 1 , . . . . N 1, can be obtained from either (5.1.4) or (5.1.8). Hence
0

oc

x p(n) = ^

jc(n - IN) = ^

a~IN

0 <n < N - 1
where the factor 1/(1 - a N) represents the effect of aliasing. Since 0 < a < 1, the
aliasing error tends toward zero as N -+ oo.
For a = 0.8, the sequence x(n) and its spectrum X{w) are shown in Fig. 5,4a
and b, respectively. The aliased sequences xp(n) for N = 5 and N = 50 and the
corresponding spectral samples are shown in Fig. 5.4c and d, respectively. We note
that the aliasing effects are negligible for N = 50.
If we define the aliased finite-duration sequence x(n) as
xp{n),
0,

0 < n < N 1
otherwise

then its Fourier transform is

X(w) = y ^ i ( n ) e
1
1 a N

1
1 ae~,w

Note that although X(ou) ^ X(a>), the sample values at a>t = I n k f N are identical.
That is,
1
1- a N

1-a N
l - a e - ' 21"

5.1.2 The Discrete Fourier Transform (DFT)

The d ev elo p m en t in the preceding section is concerned with the frequency-dom ain
sam pling o f an aperiodic finite-energy sequ en ce j:(n). In general, the equally
spaced freq u en cy sam ples X (2n k / N ) , k = 0 , 1 ___ _ N 1, do n ot uniquely represent
the original seq u en ce x ( n ) when x(n) has infinite duration. Instead, the frequency
sam ples X ( 2 n k / N ) , k = 0, 1........ N - I, correspond to a period ic seq u en ce x p(n)
o f period N , w here x p (n) is an aliased version o f *{), as indicated by the relation
in (5.1.4), that is,
(5.1.15)
W hen the seq u en ce x ( n ) has a finite duration o f length L < N , then x p{n)
is sim ply a periodic repetition o f x ( n ) , w h ere x p (n) over a single period is

400

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

IAWH

1.0?

Tfrrmnx.
(a)

(b)

x(n)
1.0*

x \ f k)

1W

50

50

(1

fdi
Figure 5.4 (a) Plot of sequence xin) = (0.X)"h (h ): (b) its Fourier transform (magnitude
only): (c) effect of aliasing with A' = 5: (d) reduced effect of aliasing with A' = 50.

given as
x r (n) =

x( n) .

0.

0 < n < L 1
L < n < N 1

(5.1.16)

C onsequently, the frequency sam ples X ( 2 i r k / N ) , k = 0. 1.........N 1, uniquely

represent the finite-duration seq u en ce *{/;) Since x ( n ) = x p (n) over a single pe
riod (padded by N L zeros), the original finite-duration seq u en ce x ( n ) can be
obtained from the frequency sam ples \ X ( 2 n k / N \ by m eans o f the formula (5.1.8)It is im portant to n ote that zero p a d d i n g d oes not provide any additional
inform ation about the spectrum X(a>) o f the seq u en ce \x(n)}. T he L equidis-

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

401

tant sam ples o f X(u>) are sufficient to reconstruct X(co) using the reconstruction
form ula (5.1.13). H o w ever, padding the seq u en ce {jc( )} with N L zeros and
com puting an A'-point D F T results in a better display" of the Fourier transform
X(o>).
In sum m ary, a finite-duration seq u en ce x( n ) o f length L [i.e., x(n ) = 0 for
n < 0 and n > L\ has a Fourier transform
JL1
X(u>) = ^ 2 x ( n ) e ~ Jwn
n=0

0 < a> < 2jt

(5.1.17)

where the upper and low er indices in the sum m ation reflect the fact that x ( n ) = 0
outside the range 0 < n < L 1. W hen w e sam ple X{a>) at equally spaced
freq u en cies u>k = 2 n k / N . k ~ 0, 1, 2 ........ N 1, where N > L. the resultant
sam ples are
X(k) = X
= J 2 x ( n ) e ^ j27,k"/N
N_ ) N 7
"=
X(k) = y

x ( n ) e - p^ h,IN

(5.1.18)

k = 0, 1, 2, , . . , N - 1

n=<l

where for co n v en ien ce, the upper index in the sum has been increased from L 1
to /V - 1 since x ( n ) = 0 for n > L.
T he relation in (5.1.18) is a form ula for transform ing a seq u en ce {jc()} of
length L < N in to a seq u en ce o f frequency sam ples ( X()) o f length N . Since
the frequency sam ples are ob tain ed by evaluating the Fourier transform X (a>)
at a set o f N (eq ually spaced) discrete frequencies, the relation in (5.1.18) is
called the discrete Fouri er t rans f or m (D F T ) o f jc(). In turn, the relation given
by (5.1.10), which allow s us to recover the seq u en ce jr(n) from the frequency
sam ples
1 A'-l
x ( n ) = - X f c ) e ,23tknlN

n = 0 . 1 .........N ~ 1

(5.1.19)

is called the i nverse D F T (ID F T ). Clearly, when x ( n ) has length L < N , the Ap
point ID F T yield s x (n ) = 0 for L < n < N 1. T o sum m arize, the form ulas for
the D F T and ID F T are

DFT
A'-l
X ( k ) = J ^ x ( n ) e - j2nkn/N
^70

k = 0 , 1, 2,

(5.1.18)

IDFT
, A'-l
x( n) = Y * X ( k ) e JZnkn/N
N
*=o

n = 0 , 1 , 2 .........N - 1

(5.1.19)

402

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

Example 5.1.2
A finite-duration sequence of length L is given as
x(n) I
1 0,

0 < n <L -l
otherwise

Determine the /V-point DFT of this sequence for N > L.

Solution

The Fourier transform of this sequence is

-1

L-l

The magnitude and phase of X (w) are illustrated in Fig. 5.5 for L = 10. The Appoint
DFT of x (n) is simply X(w) evaluated at the set of N equally spaced frequencies
wt = 2it k / N , k = 0, 1........N 1. Hence
p-jkrkLiN

sm(*kL/N)

, - j x k l L - 1 I/A/

sm(i t k/ N)
IX(w)l

1"

7C

2n

W
|H 0

0(a>)
TT

ID

Figure SS Magnitude and phase

characteristics of the Fourier transform
for signal in Exam ple 5.1.2.

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

403

If N is selected such that N = L, then the D FT becomes

L,
* (* ) = 10.

k =0
* = 1 .2 ........L - l

Thus there is only one nonzero value in the DFT. This is apparent from obser
vation of X(w), since X(a>) = 0 at the frequencies an = I n k f L , k ^ 0. The
reader should verify that x(n) can be recovered from X(k) by perform ing an Z.-point
IDFT.
Although the L-point D FT is sufficient to uniquely represent the sequence x{ n)
in the frequency domain, it is apparent that it does not provide sufficient detail to yield
a good picture of the spectral characteristics of x(n). If we wish to have better picture,
we must evaluate (interpolate) X(a>) at more closely spaced frequencies, say wt, =
2n k / N , where N > L. In effect, we can view this com putation as expanding the size
of the sequence from L points to N points by appending A '- L zeros to the sequence
x ( n ). that is, zero padding. Then the A*-point DFT provides finer interpolation than
the L-point DFT.
Figure 5.6 provides a plot of the Appoint DFT, in magnitude and phase, for
L = 10, N = 50, and N = 100. Now the spectral characteristics of the sequence
are more clearly evident, as one will conclude by comparing these spectra with the
continuous spectrum XUo).

5.1.3 The DFT as a Linear Transformation

T he form ulas for the D F T and ID F T given by (5.1.18) and (5.1.19) m ay be ex
pressed as
N -1

X ( k ) = J ^ j r O i) ^ "
FT(I
] JV-1
x( n) = J ] x a ) ^ in
^ k=o

A- = 0 , 1 , . . . , A/ 1

n = 0 , 1 .........N - 1

(5.1.20)

(5.1.21)

w here, by definition,
W N = e ~ ^ IN

(5.1.22)

which is an N th root o f unity.

W e n o te that the com putation o f each point o f th e D F T can be accom plished
by N com p lex m ultiplications and ( N ~ 1) com plex additions. H en ce the W-point
D F T valu es can b e com p uted in a total o f N 2 com p lex m ultiplications and N ( N 1)
com p lex additions.
It is instructive to view the D F T and ID F T as linear transform ations on
seq u en ces {jc()} and {X(fc)}, respectively. Let us d efine an Af-point vector %N o f
th e signal se q u en ce x( n) , n 0, 1 , . . . , N 1, an N -p oin t vector X N o f frequency

404

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

Figure 5.6 Magnitude and phase of an N-poini DFT in Example 6.4.2; (a) L =
N = 50; (b) L = 10, N = 100.

sam ples, and an N x N m atrix

as

- *(0)
X* =

x(l)

X,v

.x(N-l).
'1

w*

1!

3
*

.1

<

_1

-I

- X ( N - 1 ).

1. ..

x(0)
X ( l)

w*

W%N~ 0

(5.1.23)
N
y y 2 ( N 1)

(* -!)< A '-l)

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

405

W ith th ese definitions, the W-point D F T m ay b e exp ressed in matrix form as

X* =

(5.1.24)

w h ere
is the m atrix o f the linear transform ation. W e ob serve that Wjv is a
sym m etric matrix. If w e assum e that the inverse o f W * exists, then (5.1.24) can
be inverted by prem ultiplying both sid es by W ^1. T hus w e ob tain
x/v =

(5.1.25)

B u t this is just an exp ression for the ID F T .

In fact, th e ID F T as given by (5.1.21), can b e exp ressed in m atrix form as
** =

(5.1.26)

The Discrete Fourier Transform: Its Properties and Applications

406

where
d en o tes the com p lex conjugate o f the m atrix W A.
(5.1.26) with (5.1.25) leads us to con clu d e that

Chap. 5

C om parison of

W *1 = -W * w

(5.1.27)

W W ;, = N l N

(5.1.28)

w hich, in turn, im plies that

w here I * is an N x N identity matrix. T h erefore, the m atrix

in the trans
form ation is an orthogonal (unitary) m atrix. F urtherm ore, its inverse exists and
is given as W *n / N . O f course, the existen ce o f the inverse o f W,v w as established
p reviously from our derivation o f the ID FT .
Example 5.L3
Compute the D FT of the four-point sequence
1

x(n) = (0

3)

Solution The first step is to determ ine the matrix W4. By exploiting the periodicity
property of W4 and the symmetry property
= -v v
the matrix W4 may he expressed as

W< =

~w"
w"

w"

-1
1
1
.1

w
w1.
w9

w44

Wl

w;

-W
1

-j
-1
j

1
- 1
1

'1

< 1
1

1
J

1
w4

W;
w*

1
M'i
W?

I"
W4-

1
j
-1
-j

Then
T
X4 = W 4X4 =

-2 + 2;

-2

L-2-2JJ
The IDFT of X4 may be determ ined by conjugating the elements in W 4 to obtain WJ
and then applying the formula (5.1.26).
T h e D F T and ID F T are com putational to o ls that play a very im portant role
in m any digital signal processing applications, such as frequency analysis (spectrum
analysis) o f signals, p ow er spectrum estim ation , and lineaT filtering. T h e im por
tance o f the D F T and ID F T in such practical ap p lication s is du e to a large extent
on the ex isten ce o f com putationally efficient algorithm s, know n co llectiv ely as fast

Sec. 5.1

Frequency Domain Sampling: The Discrete Fourier Transform

407

F ourier transform (F F T ) algorithm s, for com puting the D F T and ID F T . T his class
o f algorithm s is d escrib ed in Chapter 6.

5.1.4 Relationship of the DFT to Other Transforms

In this d iscu ssion w e h ave indicated that the D F T is an im portant com putational
to o l for perform ing frequency analysis o f signals on digital signal p rocessors. In
v iew o f the other frequency analysis to o ls and transform s that w e have d e v e l
o p ed , it is im portant to establish the relationships b etw een the D F T to th ese other
transform s.

Relationship to the Fourier series coefficients of a periodic sequence.

A p eriod ic se q u en ce
F ourier series o f th e form

with fundam ental period N can b e represented in a

iV~l
x p (n) ^ kei2l,nk,N
t=o

oo < n < oo

(5.1.29)

w h ere the F ourier series coefficients are given by the expression

1 A'-l
Ct = ~ J 2 XP(/1 )r~j2"k/N
^ n=(l

Jt = 0 , 1 .........AT - 1

(5.1.30)

If w e com p are (5.1.29) and (5.1.30) with (5.1.18) and (5.1.19), w e ob serve that the
form ula for the F ourier series coefficients has the form o f a D F T . In fact, if we
d efine a se q u en ce x(rt) = x p(n), 0 < n < N - 1 , the D F T o f this se q u en ce is sim ply
X(k) = Nc k

(5.1.31)

F urtherm ore, (5.1.29) has the form o f an ID F T . T hus th e N -p oin t D F T provid es

the exact line spectrum o f a p eriod ic seq u en ce with fundam ental period N.

Relationship to the Fourier transform of an aperiodic sequence. W e

have already sh ow n that if -t(n) is an aperiodic finite en ergy seq u en ce w ith Fourier
transform X(a>), w h ich is sam pled at N eq u ally spaced freq u en cies
= 2n k / N ,
k = 0 , 1 , . . . , N 1, the spectral com p onents
OO
X(k) =

x ( n ) e - j2* nk,N

k = 0,1,..., N - 1

(5.1.32)

n oo

are th e D F T coefficien ts o f the period ic seq u en ce o f period N , given by

OO
J;p (n) = ^

x(n-lN )

(5.1.33)

/* OO
Thus x p (n) is d eterm in ed by aliasing {jc(n)J o ver th e interval 0 < n < N - 1. T h e
finite-duration seq u en ce

408

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

bears no resem blance to the original seq u en ce {x(n )), u nless ;c(n) is o f finite dura
tion and length L < N, in which case
x( n) = x{n)

0 < n < N 1

(5.1.35)

O nly in this case will the ID F T o f {X(jfc)} yield the original seq u en ce {*()}.

Relationship to the z*transform.

Let us consider a seq u en ce x( n) having

the ^-transform
(5.1.36)
with a R O C that includes the unit circle. If X ( z ) is sam pled at the N equally
spaced points on the unit circle zt = e i2*k/ N, 0, 1, 2 , . . . , N 1, w e obtain
X( k ) = X(z)U=, it

k = 0, 1 , . . . , N - 1
(5.1.37)

x ( n ) e ~ j2nnk/N

T he expression in (5.1.37) is identical to the Fourier transform X(io) evalu ated at

the N equally spaced freq u en cies <ot = 2 n k / N , k = 0, 1 , ___ N ~ 1, which is the
topic treated in Section 5.1.1.
If the seq u en ce x( n) has a finite duration o f length N or less, the seq u en ce can
be recovered from its /V-point D F T . H en ce its z-transform is un iqu ely determ ined
by its N -p oin t D F T . C onsequently, X ( z ) can be expressed as a function o f the
D F T fX (k)} as follow s
N~ 1
* (Z ) = ^ * ( ) 2 n
n=0
/V-l
X(z) =
n=0 L

*=0

-N N- 1

X ( Z) =

(5.1.38)

X( k)
ej2*k/Nz - i

W hen evaluated on the unit circle, (5.1.38) yield s the F ourier transform o f the
finite-duration seq u en ce in term s o f its D F T , in th e form

k=0 1
T his expression for th e Fourier transform is a p olyn om ial (L agrange) interpolation
form ula for X ( w ) exp ressed in term s o f the valu es {X (Jt)) o f th e polyn om ial at a
set o f equally spaced d iscrete freq u en cies <ok = 2 n k / N , k = 0, 1.........N - 1. With

Sec. 5.2

Properties of the DFT

409

so m e algebraic m anipulations, it is p ossib le to reduce (5.1.39) to the interpolation

form ula given p reviously in (5.1.13).

Relationship to the Fourier series coefficients of a continuous-time

signal. S uppose that xa (t) is a con tinu ou s-tim e periodic signal with fundam ental
p eriod Tp = 1 /F 0. T he signal can be expressed in a Fourier series
OC

xaU) =

CteJ2,TkF"

t = 3C

(5-1-40)

w here { q ) are the Fourier coefficients. If we sam ple xc,(t) at a uniform rate
Fs = N / T p = 1 / T , w e obtain the discrete-tim e sequ en ce
x ( n ) = x a( nT) =

y
Cl;ej2,rkF",T =
ckej2nt,l/N
k='\,
k=cc
N - 1
J 2 x k n /N

- E

l~~CX.

It is clear that (5.1.41) is in the form o f an ID F T form ula, where

rv
X{ k) = N j 2 Ck-iN = N c t
/=--v
and
V
Q = y
t'k-iN
/=-ac

(5.1.42)

(5.1.43)

T hus the {q } seq u en ce is an aliased version o f the seq u en ce (cA}.

5.2 PROPERTIES OF THE DFT

In S ection 5.1.2 w e introduced the D F T as a set o f N sam p les {X(Jt)} of the
F ourier transform X(a>) for a finite-duration seq u en ce |jr(n)} o f length L < N.
T h e sam pling o f X (to) occurs at the N equally spaced freq u en cies cd* = 2 n k / N ,
k 0, 1, 2 .........N 1. W e dem onstrated that the N sam p les (X(A)} uniquely
represent the seq u en ce (;c(n)} in the frequency dom ain. R ecall that the D F T and
inverse D F T (ID F T ) for an //-p o in t seq u en ce {*()} are given as
N- 1
DFT: X( k ) = J ^ x ^n ) W N
n=0

* - 0 , 1 .........A ' - l

(5.2.1)

1 N- 1

ID FT: jc(n) = X ( k ) W ~ kn
i=0

n = 0 , 1 .........N - 1

(5.2.2)

where Wn is defined as
WN = e~ j2n,N

(5.2.3)

410

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

In this section w e present the im portant p rop erties o f the D F T . In view o f the
relationships estab lish ed in Section 5.1.4 b etw een the D F T and Fourier series,
and Fourier transform s and --transform s o f d iscrete-tim e signals, w e exp ect the
p roperties o f the D F T to resem ble the properties o f these o th er transform s and
series. H ow ever, som e im portant d ifferen ces exist, o n e o f w hich is the circular
con v o lu tio n property derived in the follow in g section . A good understanding of
these properties is extrem ely helpful in the application o f the D F T to practical
p roblem s.
T h e notation used b elow to d en o te the N -point D F T pair x ( n ) and AT(Jt) is

*() S

x(k)

5.2.1 Periodicity, Linearity, and Symmetry Properties

Periodicity.

If x(n) and X (k ) are an Af-point D F T pair, then

x(n + N) = x( n)

for all n

(5.2.4)

X( k + N) = X( k )

for all k

(5.2.5)

T h ese periodicities in .r(n) and A' (A:) follow im m ed iately from form ulas (5.2.1) and
(5.2.2) for the D F T and ID FT , respectively.
W e previously illustrated the periodicity property in the seq u en ce x (n) for a
given D FT. H ow ever, we had not p reviously view ed the D F T X( k ) as a periodic
seq u en ce. In so m e applications it is ad van tageou s to do this.

Linearity.

If
DFT

* ,( )

X l (k)

and
x 2(n) K
N

X 2(k)

then for any real-valued or com p lex-valu ed con stan ts a\ and a2,
DFT

a\ X] ( n) + a 2x 2(n) < + a {X\ ( k) + a2X 2(k)

(5.2.6)

This property follow s im m ediately from the definition o f the D F T given by (5.2.1).

Circular Symmetries of a Sequence. A s w e h ave seen , the Appoint D FT

o f a finite duration seq u en ce, x(n) o f length L < N is eq u ivalen t to the W-point
D F T o f a periodic seq u en ce xp (n), o f period N, w hich is ob tain ed by periodically
exten d ing jc(n), that is,
OO

xp (n) =

x( n - I N)
/=OO

(5.2.7)

Sec. 52.

Properties of the DFT

411

N o w su p p ose that w e shift the periodic seq u en ce x p (n) by k units to the right.
Thus w e obtain a n oth er period ic sequ en ce
OC

x'p (n) = x p (n - k) =

^ x( n - k - IN)
/=-OC

T h e finite-duration seq u en ce
1 0,

otherw ise

(5.2.8)

(5.2.9)

is related to the original seq u en ce x ( n ) by a circular shift. T h is relationship is

illustrated in Fig. 5.7 for N = 4.
In gen eral, th e circular shift of the seq u en ce can b e rep resen ted as the index
m o d u lo N . T h u s w e can w rite
x'(n) = x(n k, m odu lo N )
(5.2.10)
s x((n ~ k) ) N
For exam p le, if k = 2 and N = 4, w e have
x'(n) = * (( - 2))4
which im plies that
jc'(0) = x ( ( - 2 ) ) 4 = x ( 2 )
x' { l ) =

jc (

( - 1))4 = ;c ( 3 )

* '( 2 ) = jt<(0))4 = j (0)

x

'( 3 ) =

j t ( ( 1))4

j t (1 )

H en ce x'(n) is sim ply x (n) shifted circularly by tw o units in tim e, w here the cou n
terclock w ise direction has b een arbitrarily selected as the p ositive direction. Thus
w e con clu d e that a circular shift o f an
-point seq u en ce is eq u ivalen t to a linear
shift o f its p eriod ic ex ten sion , and vice versa.
T h e in h eren t p eriod icity resulting from the arrangem ent o f the Af-point se
q u en ce o n th e circum ference o f a circle dictates a differen t definition o f even and
od d sym m etry, and tim e reversal o f a sequ en ce.
A n Af-point seq u en ce is called circularly even if it is sym m etric ab ou t the
p oin t zero on th e circle. T h is im plies that
x ( N n ) = x ( n )

1 < n < N 1

(5.2.11)

A n W -point seq u en ce is called circularly o d d if it is antisym m etric ab ou t the point

zero o n the circle. T h is im plies that
x ( N n) = x( n)

1 < n < N 1

(5.2.12)

T h e tim e reversal o f an Af-point seq u en ce is attained by reversing its sam ples

about the p o in t ze r o on the circle. T h u s th e seq u en ce jc((n ) ) # is sim ply given as
* ((- ))* = x ( N - n )

0 < n < N - l

(5.2.13)

T h is tim e reversal is eq u ivalen t to plottin g j:(/i ) in a clock w ise direction on a circle.

412

The Discrete Fourier Transform: Its Properties and Applications

4<

j(n)

3
2t
'
<a)

4'

xM

41

T2J
-4

r l l '

. . t

-3 - 2 - I

(b)

Jr,,(n - 2)
4<
3 <

- t6 - 5I

-4

-3

4'

4'

, - t " l 1 m i I
-2

-I

(O
4
3

TI

*(0)

*'(2)

<e)
Figure 5.7

Circular shift o f a sequence.

Chap. 5

Sec. 5.2

Properties of the DFT

413

A n eq u ivalen t definition o f even and od d seq u en ces for th e associated peri

odic seq u en ce x p (n) is given as follow s
even:

x(n) =

= x p( N - n)
(5.2.14)

odd:

x p (n) = - x p( n) - - x p ( N - n)

If the periodic seq u en ce is com p lex-valu ed , w e have

conjugate even:

x (n) = x U N n)
P
x p (n) = x * ( N n )

conjugate odd:

(5.2.15)

T h ese relationships suggest that w e d ecom p ose the seq u en ce x p (n) as

xp (n) = xP'(n) + x ^ i n )

(5.2.16)

xPAn) = \ [ x p (n) + x p( N - n)]

,
P
x p{n) = \ [ x P(n) - x * ( N - n ) ]

(5.2.17)

w here

Symmetry properties of the DFT. T h e sym m etry properties for the D F T

can be o b tain ed by applying the m eth od ology p reviously used for the Fourier
transform . Let us assum e that the N -p oin t seq u en ce jc(n) and its D F T are both
com plex valued. T h en the seq u en ces can be exp ressed as
jr(n) = XR(n) + j x / ( n )

0 < n < N 1

(5.2.18)

X( k ) = Xg ( k ) + j X , ( k )

0 < k < N - 1

(5.2.19)

B y substituting (5.2.18) in to the expression for the D F T given by (5.2.1), w e obtain

* * (* ) =

j*J?(") cos

X,(k) = E

+ x , ( n ) sin

(5.2.20)

[**(" )sin ~ J p ' -*/() cos

n=0 L

(5-2.21)
J

Sim ilarly, by substituting (5.2.19) into the expression for the ID F T given by (5.2.2),
w e obtain
x K(n) = i

1
* /(n ) =

[* * ( * ) cos
*=0 L
f

Real-valued sequences.

- X ,( fc ) s in ^ ^ j
J

. 2nkn
sin ^ h

2n kn "I
cos ^ J

(5.2.22)

(5.2.23)

If the seq u en ce jc(n) is real, it follow s directly

from (5.2.1) that

X ( N - k ) = X m(k) = X(-Jfc)

(5.2.24)

414

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

C onsequently, \ X ( N ~ jfc)| = |AT(A:)| and I X ( N - k) = l X ( k ) . Furtherm ore,

xi (n) = 0 and therefore ;r(rj) can b e d eterm in ed from (5.2.22). which is another
form for the ID F T .

Real and even sequences.

If x (n) is real and even , that is,

x(n ) x ( N n)

0 < n < N 1

then (5.2.21) yield s X / ( k ) = 0. H en ce the D F T reduces to

Cy
2nkn
X ( k ) = y x ( n ) c o s -------

0 < k < N \

(5.2.25)

which is itself real-valued and even .

reduces to

Furtherm ore, since X / ( k ) = 0, the ID FT

jc () ^ X (k ) c o s -------*=o

Real and odd sequences.

0 < n < N 1

(5.2.26)

If x ( n ) is real and odd, that is,

x ( n ) = x ( N n)

0 < n < N ~ 1

then (5.2.20) yields X R(k) = 0. H en ce

X( k ) j Y x ( n ) sin
t z

o < k < N 1

(5.2.27)

which is purely im aginary and odd. Since X K(k) = 0, the ID F T reduces to

* ( ) = j ~ y

X (k ) sin

0 < n < N - 1

(5.2.28)

Purely imaginary sequences.

In this case, x( n) = j x f (n). C onsequently,

(5.2.20) and (5.2.21) reduce to

**< *) -

E
n=0

x >(k) -

JC,(n)sin

(5.2.29)
N

* / ( " ) 0 0 8 77n=0

(5.2.30)

W e ob serve that Xn( k) is odd and X } (k) is even.

If x / ( n ) is odd, then X / ( k ) = 0 and h en ce X(Jt) is purely real. O n th e other
hand, if x r (n ) is ev en , then X * (Jt) = 0 and h en ce X(i fc) is purely im aginary.

Sec. 5.2

Properties of the DFT

TABLE 5.1

415

SYMMETRY PROPERTIES OF THE DFT

yV-Point Sequence x(n).

0< n < N - 1

/V-Point DFT
X(k)
X ( N k)
X' ( k)

x(n)
x*(n)
x*( N - n )
xx(n)

X t,
*)]
X,-,<*) = HX( k ) - X ' ( N - *)]
X K(k)

JXj (n)
xcf(n) = i[jr(n) + x*(N - )]
xcJ n ) = |[T (n) - x*( N - /j)]

j X/ ( k )
Real Signals
X(k) = X*( N - k )
X K(k) = X W - k )
X, (k) = - X , i N - k )
\X(k)\ = \ X( N - k ) \
I X( k) = - I X { N - k)
X K{k)
jX/tt)

A ny real signal
jr{n)

x, An) = j[* (n ) + x ( N - n)]

x.(n) = 5(a-() - x( N - )]

T h e sym m etry properties given ab ove may be sum m arized as follow s:

Mu) - xjf(n) +

x 'kOi )

+ j x j{n) + jx'/in)

\
x(k) = x H
(k) + x ; (k) + j x ;t (k) + j x l<,(k)

(5-2-3D

AH the sym m etry properties o f the D F T can easily be d ed u ced from (5.2.31). For
exam p le, the D F T o f the seq u en ce
x pr(n) = j ^ f n ) + x * ( N - n)]
is
* * ( * ) = X'K(k) + XK(k)
T h e sym m etry p rop erties of the D F T are sum m arized in T able 5.1. E x
p loitation o f th e se properties for the efficient com putation o f the D F T o f special
se q u en ces is con sid ered in so m e o f the problem s at the end o f the chapter.

5.2.2 M ultiplication o f Two DFTs and Circular C on volu tion

S u p pose that w e have tw o finite-duration seq u en ces o f length N, Jti(n) and
T h eir resp ective Appoint D F T s are
JV-1
X, (k ) = Y L jc, (n)e~i2* HklN
iV-1
X 2(k) = Y x 2( n ) e - j2*nk/N
n=0

k = 0,1,..., N 1

(5.2.32)

k = 0,1,..., N - 1

(5.2.33)

The Discrete Fourier Transform: Its Properties and Applications

416

Chap. 5

If w e m ultiply the tw o D F T s togeth er, the result is a D F T , say

o f a se
quen ce jf3(n) o f length N. L et us determ in e the relationship b etw een x3(n) and
the sequ en ces X] ( n) and jr2(n).
W e have
Xi ( k) = X t ( k ) X2(k)

k = 0 , 1 .........N - 1

(5.2.34)

T h e ID F T o f {X3(Jt)} is
1 N-1
x 3 (m) = ~
X j ( k ) e i2* km/N
*=o

(5.2.35)

A'-l

Suppose that we substitute for X\ ( k ) and X 2(k) in (5.2.35) using the D F T s given
in (5.2.32) and (5.2.33). T hus w e obtain

j27rkn/N

k=0

-j2nkl/N

j2nkm/N

/=0

(5.2.36)

Af-l

T he inner sum in the brackets in (5.2.36) has the form

-i

f N.

a = 1
(5.2.37)

y > A= l l - a "
*-

0751

w here a is defined as
Q _ e j2n(m-n-l)/N

W e observe that a = 1 w hen m n I is a m ultiple o f N. O n the other hand,

a N = 1 for any value o f a = 0. C on sequ en tly, (5.2.37) reduces to
/V- 1

ks=0

t
a

,
I N,
1 0,
1

I = m - n + p N = ((m - n ) ) N,
otherw ise

p an in teger

2 381

If w e substitute the result in (5.2.38) in to (5.2.36), w e obtain the desired expression

for ^ ( m ) in the form
xi ( m) = Y

x \ ( n) x 2((m - n ) ) N

m = 0,1,..., N - 1

(5.2.39)

T h e expression in (5.2.39) has th e form o f a con volu tion sum . H ow ever, it is

not the ordinary linear con volu tion that w as introduced in C hapter 2, which relates
the output seq u en ce y( n) o f a linear system to th e input seq u en ce x( n) and the
im pulse response h(n). Instead, the con volu tion sum in (5.2.39) in volves the index

Sec. 5.2

417

Properties of the DFT

((m fl))w and is called circular co nvolution. Thus w e con clu d e that m ultiplication
o f the D F T s o f tw o seq u en ces is eq u ivalen t to the circular con volu tion o f the tw o
seq u en ces in the tim e dom ain.
T h e fo llow in g exam p le illustrates the op eration s in volved in circular con vo
lution.

Example Si.1
Perform the circular convolution of the following two sequences:
*!() = { 2,1,2( 1}
t
x 2(n) = {1,2,3,4}
t

Solution Each sequence consists of four nonzero points. For the purposes of illus
trating the operations involved in circular convolution, it is desirable to graph each
sequence as points on a circle. Thus the sequences x\{n) and x 2(n) are graphed as
illustrated in Fig. 5.8(a). We note that the sequences are graphed in a counterclock
wise direction on a circle. This establishes the reference direction in rotating one of
the sequences relative to the other.
Now, xi(m) is obtained by circularly convolving jci<) with J 2() as specified by
(5.2.39). Beginning with m = 0 we have
jrj(O ) = y ^ j r i ( n ) j r 2( ( - n ) ) y

n=U
jc2(()>4 is simply the sequence x 2(n) folded and graphed on a circle as illustrated in
Fig. 5.8(b). In other words, the folded sequence is simply xz(n) graphed in a clockwise
direction.
The product sequence is obtained by multiplying jci(h) with * j( ( - n ) ) 4, point by
point. This sequence is also illustrated in Fig. 5.8(b). Finally, we sum the values in
the product sequence to obtain
*3(0) = 14

For m = 1 we have
3

*3(1) = ^ x l ( n ) x 2( ( l - 4

*0
It is easily verified that *2((1 n ))4 is simply the sequence * 2((~ n ))4 rotated coun
terclockwise by one unit in time as illustrated in Fig. 5.8(c). This rotated sequence
multiplies x\ (n) to yield the product sequence, also illustrated in Fig. 5.8(c). Finally,
we sum the values in the product sequence to obtain *3(1). Thus
* 3d) = 16
F or m = 2 we have
3

*3(2) = E

Xl (n )*2((2 - n ))

n0
Now *2 ((2 n ))4 is the folded sequence in Fig. 5.8(b) rotated two units of time in
the counterclockwise direction. The resultant sequence is illustrated in Fig. 5.8(d)

*|0) = 1

jr2( l) = 2

* ,(0) = 2

jri(2> - 2

*2(2) = 3

*2( 0 ) =1

(a)

jc2 ( 2 )

* 2( 0 ) = 1

=3

jc2(I) = 2
Folded sequence

Product sequence
(b)

x2(0) = 1

jr2(3 )= 4

x2(l) = 2

jc2(2> = 3
Folded sequence rotated by one unit in time

x 2( l )

(c)

=2

*2(2)= 3

*2(0) = 1

*2(3) = 4
Folded sequence rotated by two units in time
(d)
*j(2) = 3

*2(3) = 4

*j(0)=l
Folded sequence rotated by three units in time
Figure 5.8

Product sequence

(e)

Circular oonvolutioa o f tw o sequences.

Sec. 5.2

Properties of the DFT

419

along with the product sequence x l (n)xi((2 - n))A. By summing the four term s in the
product sequence, we obtain
* j ( 2 ) = 14

For m = 3 we have
3

X30 ) = y^xi(n)x;((3 - n ) ) 4
i,=Q
The folded sequence X2((n))4 is now rotated by three units in time to yield jc2(<3n))4
and the resultant sequence is multiplied by *j(n) to yield the product sequence as
illustrated in Fig. 5.8(e). The sum of the values in the product sequence is
x 3( 3) = 16

We observe that if the com putation above is continued beyond m = 3, we

simply repeat the sequence of four values obtained above. Therefore, the circular
convolution of the two sequences x \ ( n ) and x2(n) yields the sequence
xi(n) = {14,16,14,16)
t
From this exam p le, w e observe that circular con volu tion involves basically
the sam e four step s as the ordinary linear co n vo lu tio n introduced in C hapter 2:
fo l d i n g (tim e reversing) on e seq u en ce, shifting the fold ed seq u en ce, m u ltip ly in g the
tw o seq u en ces to obtain a product seq u en ce, and finally, s u m m i n g the valu es o f the
product seq u en ce. T h e basic d ifference b etw een th ese tw o types o f con volu tion
is that, in circular con v olu tion , the foldin g and shifting (rotatin g) op eration s are
p erform ed in a circular fashion by com puting the index o f o n e o f the seq u en ces
m o d u lo N . In linear co n volu tion , there is n o m odu lo N op eration .
T h e reader can easily sh ow from our previous d evelop m en t that eith er on e
o f the tw o seq u en ces m ay b e fold ed and rotated w ithou t changing the result o f the
circular con volu tion . Thus
N- 1
x$(m) = Y ' X 2 ( n )x i ( ( m n ) ) s
m = 0 , 1 , ... t N 1
(5.2.40)
n=0
T h e fo llo w in g exam p le serves to illustrate the com p utation o f x 3 (n) by m eans
o f the D F T and ID F T .
Exam ple SJJ1
By m eans of the D FT and IDFT, determ ine the sequence xj(n) corresponding to the
circular convolution of the sequences x\ (n) and X2 (n) given in Exam ple 5.2.1.
Solution
xi(n) is

First we com pute the DFTs of xi(n) and * 2(0 ). T he four-point D FT of

3

Xi<*) = Y Xl
ft-0

k = 0 ,1 ,2 ,3

420

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

Thus
*1(0) = 6

JCid) = 0

Jlf,(2) = 2

A'1(3) = 0

The D FT o f Jt2(n) is
3

X 2(.k) = Y
fl0

k = 0 ,1 ,2 ,3

= 1 + 2 e - iltkri + 3e~>*k + 4e~J^ kri

Thus
X2(0) = 10

X 2a ) = - 2 + j 2

X2( 2 ) = - 2

X 2Q) = - 2 - j 2

When we multiply the two DFTs, we obtain the product

X3(*) = X t (k)X 2(k)
or, equivalently,
X3(0) = 60

* 3(1) = 0

X3( 2 ) = - 4

Xj (3) = 0

Now, the ID FT of X 3(k) is

jr3(n) = Y X3(Jt)cj7,rn*/4
*41

n = 0, 1, 2, 3

= j(6 0 - 4eJn)
Thus
j 3(0> = 14

jc3(1)

= 16

x3(2) = 14

;c3(3) = 16

which is the result obtained in Exam ple 5.2.1 from circular convolution.
W e conclude this section by form ally stating this im portant property o f the
DFT.

Circular co n v o lu tio n .

If
* i(n )

DFT

x m

and

then
DFT

*1 (H) (g> * 2 (n) ^

(*)X 2(Jt)

(5.2.41)

w here x\ (n) (N) x2(n) d en o te s th e circular con volu tion o f the se q u en ce x i(n ) and
x%(n).

Sec. 5.2

Properties of the DFT

421

42)

*6)

*(6)

jt(2)

Figure 5.9 Tim e reversal of a sequence.

5.2.3 Additional DFT Properties

Time reversal of a sequence.

If
DFT

th e n
x((-n))v = x ( N - n )

X ( ( - k ) ) N = X ( N - k)

(5.2.42)

H e n c e re v e rsin g th e /V-point se q u e n c e in tim e is e q u iv a le n t to rev ersin g th e D F T

values. T im e rev ersal of a s e q u e n c e x(rt) is illu stra te d in Fig. 5.9.
Proof . F ro m th e d efin itio n of th e D F T in (5.2.1) w e h a v e
JV-I
D F T { * W - )} = x ( N ~ n ) e ~ j2nkn/N
n=0
I f w e c h a n g e th e in d ex fro m n to m = N - n, th e n
N -l
D F T [ x ( N - n ) } = x ( m ) < T j7,r*(* - m)/,v
=o
A '-l

= Y , x ( m ) e > 2*kmIN
m=0
= Y x ( m ) e - i2*m(N- k)/N = X ( N k)
m=0
W e n o te th a t X ( N - k) = X { ( - k ) ) N, 0 < k < N - l .

Circular time shift of a sequence.

If

422

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

th e n
* n - l))N

X ( k ) e ~ j2* k!/N

(5.2.43)

Proof. From the definition o f the D F T w e have

N- 1

D F T { * ((# i- / ) ) * } = * ( ( n - / ) ) e - ' ' 2ir*"/N

n=0
= > ( ( - l ) ) Ne ~ * * k/N
n=0
+ Y x ( n - l ) e - j kn/N
n=l
B ut x( (n - 1)) h = x ( N - l + n). C onsequently,
~ 1 + n ) e ' j2jrkn/N

x ( ( n - l ) ) Ne~j2*kn/N = ] T
n0
rr=s()
=

x ( m ) e ' ^ k^
m=N-l

F urtherm ore,
y x { n - l ) e - J2,Tkn/N =
/=/

x ( m ) e - j2kim+IWN

T herefore,
N~ 1
DFT{jc(( - / ) ) } = ^ x{m)e~-'2xk(m+,)/N
m=0

= X ( k ) e ' ^ u/N

Circular freq u en cy sh ift.

If
DFT

x( n)

X( k)

then
x ( n ) e j2*In/N 5 - X ( ( k - / *

(5.2.44)

H en ce, the m ultiplication o f the seq u en ce jc(/i) w ith the com p lex exp on en tial se
quence eP***1* is eq u ivalen t to th e circular shift o f the D F T by I u n its in frequency.
This is th e dual to the circular tim e-shifting p roperty and its p r o o f is sim ilar to the
latter.

Sec. 5.2

Properties of the DFT

423

C o m p lex -co n ju g a te p roperties.

If
X(k)

jr(n) K
th e n
DFT

x\n)

* * ( ( - * ) ) * = * * ( * - k)

(5.2.45)

T h e p r o o f o f th is p r o p e rty is left as an ex ercise fo r th e re a d e r. T h e ID F T o f X m(k)

is
Y

X * ( k ) e J2nkn/N =

x ( J k ) e ' 2**(A'- " )' A'

*=o

T h e re fo re ,
x * ( ( - n ) ) N = x+( N - b)

Circular correlation.

A/

X'(k)

(5.2.46)

In g e n e ra l, fo r c o m p le x -v a lu e d se q u e n c e s x ( n ) and

V(n), if
DF!'

x(n)

/v

X(A-)

an d
v()

y<*)

th e n
FJV(/)

R ( k ) = X ( k ) Y r (k)

(5.2.47)

w h ere r ,,.(/) is th e (u n n o rm a liz e d ) c irc u la r c ro ssc o rre la tio n se q u e n c e , d efin e d as

N- 1

^ v (0 = ^ * ( n ) / ( ( n - l))N
F r o o / W e c a n w rite f*v(/) as th e circ u la r co n v o lu tio n o f x ( n ) w ith y*(n),
th a t is,

T h e n , w ith th e a id o f th e p r o p e rtie s in (5.2.41) a n d (5.2.46), th e W -point D F T o f

rxy(l) is
R i y (k) = X (k)F *(k)
In th e sp e c ia l case w h e re y (n ) = x ( n ) , w e h a v e th e c o rre sp o n d in g e x p ressio n
fo r th e c irc u la r a u to c o r re la tio n o f x ( n ) ,

424

The Discrete Fourier Transform: Its Properties and Applications

Multiplication of two sequences.

Chap. 5

If

jr,(n )

* ,( * )

x 2(n)

X 2(k)

and

then
x A n ) x 2(n) K

^ X ,( J t ) ( N ) X 2(*)

(5.2.49)

This property is the dual o f (5.2.41). Its p ro o f follow s sim ply by interchanging
the roles o f tim e and frequency in the exp ression for the circular con volu tion of
tw o sequ en ces.

Parsevals theorem.

For com p lex-valu ed seq u en ces * (n ) and _y(n), in gen

eral, if

and

then

A N- 1
/V 1

/V 1

Jt(n)y*(n) = - X ( J t ) r (/:)
N *=0
n=0

(5-2 50)

Proof. T h e property follow s im m ed iately from the circular correlation prop

erty in (5.2.47). W e have
N-1

^ x ( n ) y * ( n ) = r JV(0)
and

k~0
H en ce (5.2.50) fo llo w s by evalu atin g the ID F T at I = 0.
T he exp ression in (5.2.50) is th e gen eral form o f P arsevals theorem . In th e
sp ecial case w h ere ;y(n) = x ( n ) , (5.2.50) red u ces to

/11=0

\x{n)\2 = i

JtseO

|X ( * ) | 2

(5.2.51)

Sec. 5.3

Linear Filtering Methods Based on the DFT

TABLE 5.2

425

PROPERTIES OF THE DFT

Property
Notation
Periodicity
Linearity
Time reversal
Circular time shift
Circular frequency shift
Complex conjugate
Circular convolution
Circular correlation
Multiplication of two sequences

Time Dom ain

Frequency Domain

x(n), yOi)
x (n) =s x(rt + iV)
a]Xi(n) + a2x 2(n)
x ( N n)
*((" - D) n
x(n)ei2jr,n/N
x "(n)
xi (n )@ jr2(n)

*<*), Y(k)
X( k) = X( k + N)
0\ Xi ( k) + a2X 2(k)
X(N - k )
X(k)e~J2*kl/N
X ((k-l))N
X*(N k)
xm xiik)

x (n ) y * (-n )

X(k) Y' ( k)

jri(n)x2(n)
/V1

Parsevals theorem
n=(!

*3=0

w hich ex p resses the energy in the finite-duration seq u en ce x ( n) in term s o f the

frequency co m p o n en ts {X()l T h e properties o f the D F T given ab ove are sum m arized in T able 5.2.

5.3 LINEAR FILTERING METHODS BASED ON THE DFT

Since the D F T provides a discrete frequency rep resen tation o f a finite-duration
seq u en ce in the frequency dom ain, it is in terestin g to exp lore its use as a com
p utational to o l for linear system analysis and, especially, for linear filtering. W e
have already estab lish ed that a system with freq u en cy resp on se H { w ) y w hen e x
cited with an input signal that has a spectrum X(a>), p o ssesses an output spectrum
Y(a>) = X ( oj) H ( w ). T he output seq u en ce y ( n) is d eterm in ed from its spectrum via
the inverse F ourier transform . C om putationally, the p rob lem with this frequencydom ain approach is that X(a>), H(a>), and Y(a>) are fun ction s o f the continuous
variable o>. A s a con seq u en ce, the com p utations can n ot be d o n e on a d igital com
puter, since th e com puter can on ly store and perform com p utations on quantities
at discrete frequencies.
O n the other hand, th e D F T d o es len d itself to com p utation on a digital
com puter. In th e discussion that follow s, w e d escrib e h ow th e D F T can b e used
to perform lin ear filtering in the frequency dom ain. In particular, w e p resent
a com p utational p rocedure that serves as an alternative to tim e-d om ain co n v o
lution. In fact, the frequency-dom ain approach b ased o n the D F T , is com pu
tationally m ore efficien t than tim e-dom ain con volu tion d u e to the existen ce o f
efficient algorithm s for com p utin g the D F T . T h ese algorithm s, w hich are d e
scribed in C hapter 6, are collectively called fast F ourier transform (FFT ) algo
rithms.

426

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

5.3.1 Use of the DFT in Linear Filtering

In the p receding section it w as d em onstrated that the product o f tw o D F T s is
equivalent to th e circular con volu tion o f the corresponding tim e-d om ain sequ en ces.
U nfortunately, circular con volu tion is o f no use to us if our ob jectiv e is to deter
m ine th e output o f a linear filter to a given input seq u en ce. In this case w e seek
a frequency-dom ain m eth od ology eq u ivalen t to lin ear con volu tion .
S u ppose that w e have a finite-duration seq u en ce x( n) o f length L which
excites an F IR filter o f length Af. W ithout loss o f generality, let
jc(n) = 0,
h(n) =

n < 0 and n > L

0,

n <

and n > M

where h(n) is the im pulse resp onse o f the F IR filter.

The output seq u en ce y ( n ) o f the FIR filter can be exp ressed in the tim e
dom ain as the con volu tion o f x( n) and h(n), that is
y(n)

M- 1

h(k) x( n k )

(5.3.1)

Since h(n) and jc(n) are finite-duration seq u en ces, their con volu tion is also finite
in duration. In fact, the duration o f y( n) is L + M 1.
T h e frequency-dom ain equivalent to (5.3.1) is
Y(co) = X( w) H( ( o )

(5.3.2)

If the seq u en ce y (n ) is to be represented u n iqu ely in the freq u en cy dom ain by

sam ples o f its spectrum Y (to) at a set o f d iscrete freq u en cies, th e n um ber o f distinct
sam ples m ust equal or exceed L + M - 1. T h erefore, a D F T o f size N > L + M I,
is required to represent [y(n)\ in the frequency dom ain.
N ow if
Y(k) = Y{a>)\m 2 ,tkfN

= X M H M U h t /N

k = 0 , 1 ,. . . , N 1
it = 0 ,1 ....... N - 1

then
Y(k) = X ( k ) H ( k )

Jfc = 0 , l , . . . , t f - 1

(5.3.3)

w h ere {X(Jfc)} and {f/(Jt)} are the N -p oin t D F T s o f th e corresp on d ing sequences
x( n) and h (n), resp ectively. Since the seq u en ces x ( n) and h ( n ) have a duration
less than N , w e sim ply pad th ese seq u en ces w ith zeros to in crease their length to
N. T his increase in th e size o f the seq u en ces d o e s n ot alter their spectra X(o>) and
H(a>), which are con tin u ou s spectra, sin ce the seq u en ces are ap eriod ic. H ow ever,
by sam pling their spectra at N equally sp aced p oin ts in freq u en cy (com p u ting the
JV-point D F T s), w e have increased the num ber o f sam p les that represent these
seq u en ces in the frequency dom ain b eyon d the m inim um n u m b er (L or M, re
sp ectively).

Sec. 5.3

427

Linear Filtering Methods Based on the DFT

Since the N = L + M 1-point D F T o f the output seq u en ce y ( n ) is sufficient

to represent y ( n ) in the frequency dom ain, it follow s that the m ultiplication o f the
N -point D F T s X( k ) and H{ k) , according to (5.3.3), follow ed by the com putation
o f the Appoint ID F T , m ust yield the seq u en ce {,v(n)J. In turn, this im plies that
the Appoint circular con volu tion o f x( n) with h(n) m ust b e eq u ivalen t to th e linear
con v o lu tio n o f x ( n ) with h(n). In other words, by increasing th e length o f the
seq u en ces x( n) and h(n) to N points (by appending zeros), and then circularly
con volvin g the resulting seq u en ces, w e obtain the sam e result as w ou ld have b een
ob tain ed with linear con volu tion . Thus with zero padding, the D F T can b e used
to perform linear filtering.
T h e follow in g exam p le illustrates the m eth od ology in the u se o f the D F T in
linear filtering.
Example 5.3.1
By m eans of the D FT and IDFT, determine the response of the FIR filter with impulse
response
h(n) = 11.2.3}
t
to the input sequence

x(n) = |1. 2, 2.1)

t
Solution The input sequence has length L = 4 and the impulse response has length
M = 3. Linear convolution of these two sequences produces a sequence of length
N = 6. Consequently, the size of the DFTs must be at least six.
For simplicity we com pute eight-point DFTs. We should also m ention that the
efficient com putation of the DFT via the fast Fourier transform (FFT) algorithm is
usually perform ed for a length N that is a power of 2. Hence the eight-point D FT of
jr(n) is
7

Jkn/8

= 1 + 2e~W

+ le->*k2 -I- e~**k,A

This com putation yields

X (4) = 0

X to -i+ J

XC7

k = 0 .1 ........7

The Discrete Fourier Transform: Its Properties and Applications

428

Chap. 5

The eight-point DFT of h(n) is

7

H(k) =
= 1 + 2e~J*k/* + 3>e- jnkr2
Hence
H(0) = 6,

H (l) = l + V 2 - > (3 + V 5 ) ,

H( 2 ) = - 2 - j 2

HO) = 1 - ^ 2 + > (3 - V 2 ) ,

H( 4) = 2

H(5) = 1 - 7 2 - y (3 - J 2 ) ,

H( 6) = ~2 + >2

W(7) = 1 + V2 + j ( 3 + J 2 )
The product of these two DFTs yields Y{k), which is
Y (0) = 36,
y(4) = 0,

Y (2) = >4

y (l) = - 1 4 .0 7 -> 1 7 .4 8
V(5) = 0.07 y 0.515

Y( 6) = ~ j 4

X(3) = 0.07 + ,0.515

Y(7) = -14.07 + >17.48

Finally, the eight-point ID FT is

7

v(n) = Y

y(k)ej2nk"/H

n = 0, 1........7

*=0
This computation yields the Tesult
>() = (1 ,4 ,9 ,1 1 .8 ,3 ,0 ,0 }
t

We observe that the first six values of y(rt) constitute the set of desired output
values. The last two values are zero because we used an eight-point D FT and IDFT,
when, in fact, the minimum number of points required is six.
A lthough the m ultiplication o f tw o D F T s corresp on d s to circular convolution
in the tim e dom ain, w e have ob served that padding the seq u en ces x ( n ) and h(n)
with a sufficient num ber o f zeros forces the circular con volu tion to yield the sam e
output sequ en ce as linear con volu tion . In the case o f the F IR filtering problem
in E xam p le 5.3.1, it is a sim ple m atter to d em onstrate that the six-point circular
convolution o f the sequ en ces
h( n) = {1, 2 , 3 , 0 , 0, 0}
t

(5.3.4)

x ( n ) = {1 ,2 , 2 , 1 , 0 , 0}
t

(5.3.5)

results in the output sequ en ce

y ( n) = {1, 4, 9 , 1 1 , 8 , 3}
t
w hich is the sam e seq u en ce ob tain ed from linear con volu tion .

(5.3.6)

Sec. 5.3

Linear Filtering Methods Based on the DFT

429

I t is im p o rta n t f o r u s to u n d e rs ta n d th e aliasing th a t re su lts in th e tim e d o m a in

w h e n th e size o f th e D F T s is sm a lle r th a n L + M I. T h e fo llo w in g e x am p le focuses
o n th e aliasin g p ro b le m .
Exam ple 5.3.2
D eterm ine the sequence v(n) that results from the use of four point DFTs in Exam
ple 5.3.1.
Solution

The four-point D F T of h(n) is

t*)jre.-jink*!*
H(k) = ^ A (fi n\
'
ff-0
H(k) = \ + 2 e - ink/1 + 3 e - ikn

k = 0 ,1 ,2 , 3

Hence
H(0) = 6,

H( \ ) = - 2 - j 2 ,

H( 2) = 2,

H ( 3 ) = - 2 + j2

The four-point DFT of x (} is

X(k) = \ + 2c ~jnk/2 + 2e~,7,k + 3e ~J%7,t/2

k = 0, 1, 2, 3

Hencc
*(()) = 6,

X (l) = - l - j ,

X(2) = 0,

Jf(3) = 1 + y

The product of these two four-point DFTs is

K(0) = 36,

f ( l ) = ;4 ,

Y( 2) = 0.

K(3) = ~ j 4

The four-point ID FT yields

y(n) = \ Y ^ k^eJ2xk,A
*=o

= 0 , 1 ,2 ,3

= 1(36 + j 4e Jn"^ - j 4e J1*na)

Therefore,
v(n) = {9,7, 9,11}
t
The reader can verify that the four-point circular convolution of h(n) with x(n)
yields the same sequence y(n).
I f w e c o m p a re th e re su lt y (n ), o b ta in e d fro m fo u r-p o in t D F T s w ith th e s e
q u e n c e y (n ) o b ta in e d fro m th e use o f e ig h t-p o in t (o r six -p o in t) D F T s, th e tim ed o m a in aliasin g effe cts d e riv e d in S ectio n 5.2.2 a re clearly e v id e n t. In p a rtic u la r,
>(4) is a liased in to y (0) to yield
y (0) = y(( + y ( 4) = 9
S im ilarly, _y(5) is a liased in to _y(l) to yield

?(1) = ? (!) + ?(5) = 7

430

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

A ll other aliasing has n o effect since y(n ) = 0 for n > 6 . C on sequ en tly, w e have
y ( 2) = y ( 2) = 9
y(3) = y(3) = 1 1
T h erefore, on ly the first tw o p oints o f y ( n ) are corrupted by th e effect o f aliasing
[i.e., y(0) 7^ y(0) and v( l ) ^ y (l)]. T his observation has im portan t ram ifications
in the discussion o f the follow in g section , in which w e treat th e filtering o f long
sequ en ces.

5.3.2 Filtering of Long Data Sequences

In practical applications involving linear filtering o f signals, the input sequ en ce
x ( n ) is often a very long sequ en ce. T his is especially true in so m e real-tim e signal
processing applications concerned with signal m onitorin g and analysis.
Since linear filtering perform ed via the D F T in volves op eration s on a block
o f data, w hich by necessity m ust be lim ited in size due to lim ited m em ory o f a
digital com puter, a long input signal seq u en ce m ust be se g m en ted to fixed-size
blocks prior to processing. Since the filtering is linear, su ccessive blocks can be
p rocessed on e at a tim e via the D F T and the output blocks are fitted togeth er to
form the overall output signal sequ en ce.
W e now d escribe tw o m eth od s for linear F IR filtering a lon g seq u en ce on a
block-by-block basis using the D F T . T h e input seq u en ce is segm en ted in to blocks
and each block is p rocessed via the D F T and ID F T to prod u ce a block o f output
data. T h e output blocks are fitted togeth er to form an overall output sequ en ce
which is identical to the seq u en ce ob tain ed if the lon g block had b een processed
via tim e-dom ain con volu tion .
T h e tw o m eth od s are called the overlap-save m e t h o d and the o verlap-a dd
m eth o d . For b oth m eth od s w e assum e that the F IR filter has duration M . The
input data seq u en ce is segm en ted in to blocks o f L points, w h ere, by assum ption,
L
M w ithout lo ss o f generality.

Overlap-save method. In this m eth od the size o f the in p ut data blocks is

N L 4- M 1 and th e size o f the D F T s and ID F T are o f len gth N . E ach data
block consists o f the last M - 1 data points o f the p reviou s data b lock follow ed by
L new data p oin ts to form a data seq u en ce o f len gth N = L 4- M 1. A n N -point
D F T is com p uted for each data block. T h e im pulse resp onse o f the F IR filter is
increased in length by appending L - l zero s and an Appoint D F T o f the sequ en ce
is com p uted on ce and stored. T h e m ultiplication o f the tw o Af-point D F T s {//(Jt)}
and {Xm(Jt)} for the m th b lock o f data yields
Ym(k) = H ( k ) X m(k)

k = 0 , 1 .........N - 1

(5.3.7)

T hen the Appoint ID F T yield s the result

L ( n ) = { ^ (0 )y ffl(l) ym(M - 1)ym{M ) ym(N - 1)}

(5.3.8)

Sec. 5.3

Linear Filtering Methods Based on the DFT

431

Sin ce the data record is o f length N, the first Af 1 p oin ts o f >m(n) are corrupted
by aliasing and m ust be discarded. T h e last L p oints o f y( n) are exactly the sam e
as the result from linear con volu tion and, as a con seq u en ce,
>(n) = y( n) , n = M, M + 1.........N - 1

(5.3.9)

T o avoid lo ss o f data du e to aliasing, the last Af 1 p oin ts o f each data record

are saved and th ese p oin ts b ecom e the first Af 1 data p oin ts o f the subsequent
record, as indicated above. T o begin the processing, the first Af 1 p oints o f the
first record are set to zero. Thus the blocks o f data seq u en ces are
jci(n) = (0, 0 ........ 0, x(0), x ( l ) .......... x ( L - 1)}

(5.3.10)

M 1 points

x2(n) = {x ( L - M + 1).........x ( L 1 ) , x ( L ) , . . . , x ( 2 L - l) j
M - 1 data points
from i|(n)

(5.3.11)

L new data points

x3(rt) = {x ( 2 L - M + l ) .........x ( 2 L 1), x ( 2 L ) .......... x ( 3 L - l } )

M - 1 data points
from

(5.3.12)

I new data points

and so forth. T h e resulting data sequ en ces from the ID F T are given by (5.3.8),
w here the first M 1 points are discarded due to aliasing and the rem aining L
p oin ts con stitute the d esired result from linear con volu tion . T h is segm en tation o f
th e input data and the fitting o f the output data blocks togeth er to form the output
seq u en ce are graphically illustrated in Fig. 5.10.

Overlap-add method. In this m eth od the size o f the input data block is L
p oin ts and the size o f th e D F T s and ID F T is N ~ L + Af - 1. T o each data block
w e append Af 1 zeros and com p ute the N -point D F T . T h u s the data b lock s may
be rep resen ted as
jti(n) = {jc(0),

jc(1 )......... jc(

- 1 ) , 0 , 0 _____0}
M -l

(5.3.13)

zeros

X2(n) = [ x( L) , x ( L + 1), . . , X(2L - 1), 0, 0 , . . . , 0}

(5.3.14)

M-1 zeros

jt3(n) = { x ( 2 L ) .........x( 3 L - 1), 0 , 0 , . . . , 0)

M -\

(5.3.15)

zeros

and so on . T h e tw o Appoint D F T s are m ultiplied togeth er to form

Ym(k) = H ( k ) X m(k)

* = 0 , 1 .........N - 1

(5.3.16)

T h e ID F T y ield s data blocks o f length N that are free o f aliasing sin ce the size o f
th e D F T s and ID F T isW = Z, + Af 1 and the seq u en ces are increased to N -p oin ts
b y ap p en d in g zero s to each block.

432

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

Output signal

points

/
Discard
Pints

Figure 5.10 L in ear F IR filtering by the

overlap-save m ethod.

Since each data block is term inated with M 1 zeros, the last M 1 points
from each output block m ust be overlap p ed and added to the first M - 1 p oin ts of
the succeeding block . H en ce this m eth od is called th e overlap-add m ethod. This
overlapping and adding yields the output seq u en ce
y ( ) = {y i(0 ), ^ i ( l ) , . . . , y \ ( L - l ) , y i ( L ) + y2(0 ), ^ ( L + 1) +

(5.3.17)
>2( 1 ) .........y \ ( N - 1) + y i ( M 1), y 2 ( M ) , . . . }
T he segm en tation o f the input data in to b lock s and th e fitting o f th e output data
blocks to form the output seq u en ce are graphically illustrated in F ig. 5.11.
A t this poin t, it m ay appear to the reader that th e use o f th e D F T in linear
F IR filtering is n ot o n ly an indirect m eth od o f com puting the o u tp u t o f an FIR
filter, b ut it m ay a lso be m ore exp en sive com p utationally since th e input data must
first b e converted to the frequency d om ain via th e D I T , m ultip lied by th e D F T
o f th e F IR filter, and finally, converted back to th e tim e d om ain via the ID FT.
O n th e contrary, h ow ever, by using the fast F ourier transform algorithm , as will
b e show n in C hapter 6, th e D F T s and ID F T require few er com p utations to com
pu te th e output seq u en ce than th e direct realization o f the F IR filter in the time

Sec. 5.4

Frequency Analysis of Signals Using the DFT

433

Input data

\
MA

Output data

M-1 point.*;/ PZJ

add
V/y
together
M-l points^
add

together

Figure 5.11 Linear FIR filtering by the

overlap-add method.

dom ain. This com putational efficiency is the basic advantage o f using the D F T to
com p ute the output o f an F IR filter.

5.4 FREQUENCY ANALYSIS OF SIGNALS USING THE DFT

T o com p ute the spectrum o f either a con tinu ou s-tim e or d iscrete-tim e signal, the
valu es o f the signal for all tim e are required. H ow ever, in practice, w e observe
signals for on ly a finite duration. C on sequ en tly, the spectrum o f a signal can
on ly b e approxim ated from a finite data record. In this section w e exam ine the
im plications o f a finite data record in frequency analysis using the D F T .
If the signal to b e analyzed is an analog signal, w e w ou ld first pass it through
an an tialiasing filter and th en sam ple it at a rate F, > 2 5 , w h ere B is the band
width o f the filtered signal. T h u s th e highest frequency that is contained in the
sam pled signal is Fs f l . F inally, for practical purposes, w e lim it the duration o f
the signal to th e tim e interval To L T , w h ere L is the num ber o f sam ples and T

The Discrete Fourier Transform: Its Properties and Applications

434

Chap. 5

is the sam ple interval. A s w e shall ob serve in the follow in g d iscu ssion , the finite
observation interval for the signal places a limit on the freq u en cy resolution; that
is, it lim its our ability to distinguish tw o frequency com p on en ts that are separated
by less than 1 /To = 1/Z-7" in frequency.
L et {*()} d en o te the seq u en ce to b e analyzed. L im iting th e duration o f the
seq u en ce to L sam ples, in the interval 0 < n < L 1, is eq u ivalen t to m ultiplying
{jc(/i)} by a rectangular w indow w ( n ) o f length L. That is,
x (n) = x ( n ) w ( n )

(5.4.1)

w h ere
u;w

= {J ;
, v,,

0
< n < L - 1
otherw ise

(5.4.2)

N ow su p p ose that the sequ en ce x ( n ) consists o f a single sin u soid , that is,
x( n ) = coscoon

(5.4.3)

Then the Fourier transform o f the finite-duration seq u en ce x ( n ) can be expressed

X(u) = $[W '(cu-<u0) + W 'to + wo)]

(5.4.4)

where W(a>) is the Fourier transform o f the w indow seq u en ce, w h ich is (for the
rectangular w indow )
W{0}) =

D/2
sin(df/ 2 )

(5.4.5)

T o com pute X(a>) w e use the D F T . B y padding the seq u en ce x ( n ) w ith N L zeros,
we ca n c o m p u te the N -point D F T o f the truncated (L p oin ts) seq u en ce {Jt(n)J.
T he m agnitude spectrum |A W I = |X(t*) | for o* = 2 n k / N , k = 0, l , . . . , A f , is
illustrated in Fig. 5.12 for L = 25 and N = 2048. W e n ote that the w indow ed
spectrum X((o) is n ot localized to a single frequency, but instead it is spread out
over the w h ole frequency range. Thus the p ow er o f the origin al signal sequence
{jr(n)} that was concentrated at a single frequency has b een spread by the window
into the entire freq u en cy range. W e say that the p ow er has lea k ed o u t in to the
entire frequency range. C onsequently, this p h en om en on , which is a characteristic
o f w indow ing the signal, is called leakage.

Frequency

Figure 5.12 Magnitude spectrum for

I = 25 and n = 2048, illustrating the
occurrence of Leakage.

Sec. 5.4

435

Frequency Analysis of Signals Using the DFT

W in d o w in g n o t o n ly d isto rts th e sp e ctral e stim a te d u e to th e le a k a g e effects,

it also re d u c e s s p e c tra l re so lu tio n . T o illu stra te th is p ro b le m , let us c o n s id e r a
signal s e q u e n c e co n sistin g o f tw o fre q u e n c y co m p o n e n ts,
x ( n ) = cos co\ n + c o s a ^ n

(5.4.6)

W h e n th is se q u e n c e is tr u n c a te d to L sam ples in th e ra n g e 0 < n < L 1, th e

w in d o w e d s p e c tru m is
X ( w ) \ [ W (to u>i) + W (a> a>2 ) + W((o + ati) + W(a> + a>2)]

(5.4.7)

T h e s p e c tru m W (to) o f th e re c ta n g u la r w indow se q u e n c e h as its first z e ro crossing

a t co = 2 n / L . N o w if |<yi a ^ l < 2n / L , th e tw o w in d o w fu n c tio n s W(co a>\) a n d
V/(o) an) o v e rla p a n d , as a co n se q u e n c e , th e tw o sp e c tra l lin es in x ( n ) a re n o t
d istin g u ish a b le . O n ly if (a>\ a>i) > 2 n / L will w e se e tw o s e p a ra te lo b es in th e
sp e c tru m X(a>). T h u s o u r a b ility to reso lv e sp e c tra l lines o f d iffe re n t fre q u e n c ie s
is lim ite d b y th e w in d o w m ain lo b e w idth. F ig u re 5.13 illu s tra te s th e m a g n itu d e
sp e c tru m |X (a)|, c o m p u te d via th e D F T , fo r th e se q u e n c e
+ cosa^n

(5.4.8)

Magnitude

x ( n ) = costuo n + cos

Frequency

Frequency

(a)

<b)

(c)

Figure 5.13 Magnitude spectrum for the signal given by (5.4.8), as observed through a

rectangular window.

436

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

w here ato = 0.2n ,

= 0. 22n, and coi = 0.6jt. T h e w indow len gth s selected are
L = 25, 50, and 100. N o te that o>o and
are not resolvab le for L = 25 and 50,
but th ey are resolvab le for L = 100.
T o reduce leak age, w e can select a data w indow w( n) that h as low er sid elob es
in the frequency d om ain com pared with the rectangular w in d ow . H ow ever, as we
d escribe in m ore detail in Chapter 8, a reduction o f the sid elo b es in a w indow
W( w) is obtained at the ex p en se o f an increase in the width o f th e m ain lo b e o f
W(a>) and h en ce a loss in resolution. T o illustrate this point, let us consider the
H anning w indow , w hich is specified as
w( n ) =

^(1 - cos jTTj-n),

0 < n < i. 1

0,

otherw ise

(5.4.9)

Figure 5.14 show s |A"(<y)| for the w indow o f (5.4.9). Its sid elo b es are significantly
sm aller than th o se o f the rectangular w in d ow , but its m ain lob e is approxim ately
tw ice as w ide. Figure 5.15 sh ow s the spectrum o f the signal in (5.4.8), after it is
w indow ed by the H anning w indow , for L = 50, 75, and 100. T h e reduction o f
the sid elo b es and th e d ecrease in the resolu tion , com pared with the rectangular
w indow , is clearly evident.
For a general signal seq u en ce ljr(n)}, the frequency-dom ain relationship be
tw een the w indow ed seq u en ce i ( n ) and the original seq u en ce x ( n ) is given by the
con volu tion form ula
(5.4.10)
T h e D F T o f the w in d ow ed seq u en ce x( n) is the sam pled version o f the spectrum
X(a>). T hus w e have
X( k ) = X ( ) U W
(5.4.11)
k = 0,1,..., N - 1
Just as in the case o f th e sinusoidal seq u en ce , if the spectrum o f th e w indow is
relatively narrow in w idth com pared to the spectrum X (to) o f th e signal, the win
dow function has o n ly a sm all (sm ooth in g) effect on the spectrum X (w). O n the
other hand, if the w in d ow function has a w ide spectrum com pared to the w idth of
6
25

0
Frequency

Figure 5.14 Magnitude spectrum of tbe

Hanning window.

437

Frequency Analysis of Signals Using the DFT

Magnitude

Sec. 5.4

L = 100

; 6

-r

- T

Frequency

(O
Figure 5.15 Magnitude spectrum of the signal in (5.4.8) as observed through a Hanning
window.
X(a>), as w ould b e the case w hen the num ber o f sam ples L is sm all, the w indow
spectrum m asks the signal spectrum and, con sequ en tly, the D F T o f the data re
flects th e spectral characteristics o f the w in d ow function. O f course, this situation
should b e avoided.
Example 5.4.1
The exponential signal
*..(0

H r

t >o
t <0

is sampled at the rate F, = 20 samples per second, and a block of 100 samples is used
to estimate its spectrum. Determine the spectral characteristics of the signal x(t) by
computing the DFT of the finite-duration sequence. Compare the spectrum of the
truncated discrete-time signal to the spectrum of the analog signal.
Solution

The spectrum of the analog signal is

Xa(F) =

1
1 + ;2 jtF

The exponential analog signal sampled at the rate of 20 samples per second yields

438

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

the sequence
x(n) = <r"r = c - '/ 20,

n > 0

Now, let
0 < n < 99
(0.95)",
0,
otherwise
The A/-point D F T of die L = 100 point sequence is
99

X(k) = Y

i<-n '>e~i2*k/N

* = 0 ,1 ........A ' - l

To obtain sufficient detail in the spectrum we choose N = 200. This is equivalent to

padding the sequence x ( n) with 100 zeros.
The graph of the analog signal xa(t) and its magnitude spectrum |Xa(F )| are
illustrated in Fig. 5.16(a) and (b), respectively. The truncated sequence xin) and its
N = 200 point D FT (m agnitude) are illustrated in Fig. 5.16(c) and (d), respectively.

1.0

0.8
0.6
0.4
0.2

(a)

_
-5 0

-4 0

-3 0

-2 0

-1 0

JL
0

i
10

20

30

40

50

(b)
Figure 5.1ti Effect o f windowing (truncating) the sampled version o f the analog
signal in Example 5.4.1.

Sec. 5.4

Frequency Analysis of Signals Using the DFT

439

(d)

(*)
Figure 5.16 Continued
In this case the D FT {X (Jt)} bears a close resem blance to the spectrum of the analog
signal. The effect of the window function is relatively small.
O n the other hand, suppose that a window function of length L = 20 is selected.
T hen the truncated sequence x(n) is now given as
.() _ [ ( - 95)"1 0,

0 < n < 19
otherwise

440

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

Its N = 200 point DFT is illustrated in Fig. 5.16(e). Now the effect of the wider
spectral window function is dearly evident. First, the main peak is very wide as a
result of the wide spectral window. Second, the sinusoidal envelope variations in the
spectrum away from the main peak are due to the large sidelobes of the rectangular
window spectrum. Consequently, the DFT is no longer a good approximation of the
analog signal spectrum.

5.5 SUMMARY AND REFERENCES

T h e m a jo r focus o f th is c h a p te r w as o n th e d isc re te F o u rie r tra n s fo rm , its p ro p e rtie s
an d its ap p licatio n s. W e d e v e lo p e d th e D F T by sa m p lin g th e sp e c tru m X (&>) o f
th e se q u en ce x(n').
F re q u e n c y -d o m a in sa m p lin g o f th e sp e c tru m o f a d is c re te -tim e signal is p a r
ticu larly im p o rta n t in th e p ro cessin g o f d ig ital signals. O f p a r tic u la r significance
is th e D F T , w hich w as show n to u n iq u e ly re p re s e n t a fin ite -d u ra tio n se q u e n c e in
th e fre q u e n c y d o m a in . T h e ex iste n ce o f c o m p u ta tio n a lly efficien t a lg o rith m s fo r
th e D F T , w hich a re d e sc rib e d in C h a p te r 6, m a k e it p o ssib le to d ig itally p ro cess
sig n als in th e fre q u e n c y d o m a in m u ch fa ste r th a n in th e tim e d o m a in . T h e p ro
cessin g m e th o d s in w hich th e D F T is esp ecially su ita b le in clu d e lin e a r filtering as
d e sc rib e d in th is c h a p te r a n d c o rre la tio n , an d sp e c tru m an aly sis, w h ich a re tre a te d
in C h a p te rs 6 a n d 12. A p a rtic u la rly lucid an d co n cise tre a tm e n t o f th e D F T and
its ap p licatio n to fre q u e n c y an aly sis is given in th e b o o k by B rig h a m (1988).

PROBLEMS
5.1 The first five points of the eight-point DFT of a real-valued sequence are (0.25,
0.125 - j 0.3018, 0, 0.125 - y0.0518, 0}. Determine the remaining three points.

5.2 Compute the eight-point circular convolution for the following sequences.
(a) *,() = (1,1,1,1,0,0,0,0}
. 3jt
x2(n) = sin -n
8

_
0 < n < 7

0 <n <7
3n
.
_
*2(n) = cos n
0< n < 7
O
(c) Compute the DFT of the two circular convolution sequences using the DFTs of
*i(n) and X2 (n).

(b) *,() = (i)"

S 3 Let X (Jt), 0 <

We define

Jt <

N 1, be the Appoint DFT of the sequence *(n), 0 < n < N 1.

y /ia _ f
{)
1 0,

0 < k < k c, N - k c < k < N - 1

kc < k < N ke

and we compute the inverse //-point DFT of X(k), 0 < k < N - 1. What is the effect
of this process on the sequence x ()? Explain.

Chap. 5

441

Problems

5.4 F or the sequences

jci(n) = cos ^ - n
N

jc2() = sin

0 <n <N - 1

determ ine the N -point:

(a) Circular convolution jc^n) (N)x 2(n)
(b) Circular correlation of *i(n) and x 2(n)
(c) Circular autocorrelation of xi(n)
(d) Circular autocorrelation of x 2(n)
5.5 Com pute the quantity
N-l
y ^ x ](n)x 2(n)
=0
for the following pairs of sequences.
(a) JC](n) = x 2 (n) = cos ~ n
N

0 < n < N 1

x 2(n) = sin n
0 < n < N 1
N
N
(c) *](n) = 6(n) + 5(n - 8)
*2() = ( ) - u(n - N)

(b) Xi (n) = cos n

5.6 D eterm ine the A/-point D FT of the Blackman window

if(n) = 0.42 0.5 cos - -I- 0.08c o s ------ Ar 1
A '- l

0 < n < N 1

5.7 If X (Jt) is the D FT of the sequence *(n), determ ine the Af-point D FTs of the sequences
2nkn
xc(n) x(n) c o s ------N

. . .
0< n < N - 1

. 2nkn
x,(n) = x ( n ) sin
Ar

0 < n < N 1

and

in terms of X (Jt).
5.8 Determine the circular convolution of the sequences
*i(n) = {1,2,3,1}
t
x2(n) = {4,3,2,2}
t
using the time-domain formula in (5.2.39).
5.9 Use the four-point DFT and IDFT to determine the sequence
X3(n) = *i(n)()*2(*)
where xi(n) and x 2(n) are the sequence given in Problem 5.8.
5.10 Compute the energy of the N -point sequence

442

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

5.11 Given the eight-point D FT of the sequence

*() =

1.
0,

0 <n < 3
4 <n < 7

compute the D FT of the sequences:

1,
n= 0
(a) x\(n) = 0,
1< n < 4
1.
5 <n < 7
0,
0 <n < 1
(b) jr2(n) = 1 ,
2 <n < 5
0,
6< n < 7
5.12 Consider a finite-duration sequence
jc(n) = {0 ,1 .2 ,3 .4 )
t
(a) Sketch the sequence .t(n) wilh six-poini DFT
(*) = Wj XOfc)

k = 0 .1 ........6

(b) D eterm ine the sequence y(n) with six-point D FT K(jt) = Re |X(A)|.
(c) Determ ine the sequence u(w) with six-point D FT V(k) = Im |X(A-)|.
5.13 Let x p(n) be a periodic sequence with fundamental period N. Consider the following
DFTs:
DFT

x p(n) * * *,(*)
N

x ()

D FT

3^

X ,(k)

(a) W hat is the relationship between Xi(^) and XiOt)?

(b) Verify the result in part (a) using the sequence
xp(n) = { 1. 2 ,1 ,2 ,1 ,2 ,1 .2 - ]
t
5.14 Consider the sequences
X] (n) = {0,1, 2,3,4}
t

x 2(n) = {0 ,1 ,0 ,0 .0 ]
t

s(n) = {1. 0, 0 .0 ,0 )
t

and their 5-point DFTs.

(a) Determine a sequence y( n ) so that Y(k) = A-!
fJt).
(b) Is there a sequence x3(n) such that S(k) = X^(k)X^(k)'!
5.15 Consider a causal LTI system with system function

The output y(n) of the system is known for 0 < n < 63. Assuming that H(z) is
available, can you develop a 64-point DFT method to recover the sequence x{n),
0 < n < 63? Can you recover all values of x(n) in this interval?
5.16* The impulse response of an LTI system is given by h(n) = S(n) - ji(n - /to). To
determine the impulse response g(n) of the inverse system, an engineer computes the
Af-point D FT H(k), N = 4ko, of h(n) and then defines g(n) as the inverse DFT of

Chap. 5

443

Problems

G (k) = 1j H ( k ), k = 0, 1, 2 , . . . , A' - l . Determine g(n) and the convolution h(n)*g(n),

and comment on whether the system with impulse response g(n) is the inverse of the
system with impulse response h(n).
5.17* Determine the eight-point DFT of the signal
*() = [ 1 , 1 , 1 . 1 . 1 , 1 , 0 , 0 }
and sketch its magnitude and phase.
5.18 A linear time-invariant system with frequency response H(u>) is excited with the
periodic input
OO
jc(n) = ^ S(n kN)
*=-oo
Suppose that we compute the Af-point DFT Y(k) of the samples >(), 0 < it < N - 1
of the output sequence. How is ^(Jt) related to //(w)?
5.19 D F T o f real sequences with special symmetries
(a) Using the symmetry properties of Section 5.2 (especially the decomposition prop
erties), explain how we can compute the DFT of two real symmetric (even) and
two real antisymmetric (odd) sequences simultaneously using an W-point DFT
only.
(b) Suppose now that we are given four real sequences *,(), i = 1, 2, 3, 4, that are
all symmetric [i.e., X j ( n ) = x j ( N n ), 0 < n < N 1], Show that the sequences
j,-(n) = X j ( n + 1) X j ( n 1)

are antisymmetric [i.e., s, (n) = s;(N - n) and 5, (0 ) = 0 ].

(c) Form a sequence *(/i) using Jti(n), JC2 (n), ^(n), and ^(n) and show how to compute
the DFT Xj(k) of x,(n), i = 1, 2, 3, 4 from the N-point DFT X(k) of x(n).
(d) Are there any frequency samples of X, (k) that cannot be recovered from X(k)?
Explain.
5.20 D F T o f real sequences with o dd harmonics only Let x(n) be an A'-point real sequence
with Af-point DFT X(k) (N even). In addition, x(n) satisfies the following symmetry
property:
/
N \
x ( n + y j = -x (n )

N
n = 0 , 1 ........y - 1

that is, the upper half of the sequence is the negative of the lower half.
(a) Show that
X (k) = 0

k even

that is, the sequence has a spectrum with odd harmonics.

(b) Show that the values of this odd-harmonic spectrum can be computed by evaluat
ing the A/2-point DFT of a complex modulated version of the original sequence
x(n).
5.21 Let x(t) be an analog signal with bandwidth B = 3 kHz. We wish to use a N = 2"point DFT to compute the spectrum of the signal with a resolution less than or equal
to 50 Hz. Determine (a) the minimum sampling rate, (b) the minimum number of
required samples, and (c) the minimum length of the analog signal record.

444

The Discrete Fourier Transform: Its Properties and Applications

Chap. 5

5.22 Consider the periodic sequence

2x

oo < n < oo

x p(n) = COS n

with frequency /o = ^ and fundamental period N = 10. Determine the 10-point

DFT of the sequence x ( n ) = x p(n), 0 < n < Af - 1.

5.23 Compute the W-point DFTs of the signals

(a) >:(") = ^()
(b) x ( n) = S(n no)
(c) Jt(n) = a "

(d) x(n)

0 < fiq < N

0 < n < N 1

1,
, Q

0 < n < N/ 2 - l ( N even)

N p.< n< N

(e) x(n) = ea*IN)k<> 0 < n < N - 1

2x
(I) x(n) = cos kon 0 < n < N 1
N
(g) j:(/i) = sin -j^kort

0< n < N - 1

i
I
n even
) ^
10,
odd 0 < n < Af 1
5.24 Consider the finite-duration signal
x(n) = {1, 2, 3, 1}

(a) Compute its four-point DFT by solving explicitly the 4-by-4 system of linear
equations defined by the inverse DFT formula.
(b) Check the answer in part (a) by computing the four-point DFT, using its defini
tion.

5.25 (a) Determine the Fourier transform X (tu) of the signal

x(n) =

{ 1 ,2 ,3 ,2 ,1 ,0 1

(b) Compute the 6-point DFT V (k) of the signal

i(n) = { 3 ,2 ,1 ,0 ,1 ,2 }

(c) Is there any relation between X(w) and V(Jt)? Explain.

5.26 Prove the identity

Y & ( n + l N ) = Y eJQ*/,'*K
JOC
kmC
(Hint: Find the DFT of the periodic signal in the left-hand side.)
5 J 7 Computation o f the even and odd harmonics using the DFT
sequence with an Appoint DFT X(k) ( N even)
(a) Consider the time-aliased sequence
x ( n + I M ),

Let x(n) be an Appoint

0 < n < M 1

fv-00
0,
elsewhere
What is the relationship between the Af-point DFT K(Jt) of y(/i) and the Fourier
transform X(w) of x(n)?

Chap. 5

Problems

445

(b) Let

0 < n < N 1

J x (n ) + x ( n + ~ l '

(-*?)

I 0,

elsewhere

and
y(n)

DFT
K(Jt)
N/l

Show that AT(Jt) = Y(k/2), k = 2, 4 , . . . , N - 2.

(c) Use the results in parts (a) and (b) to develop a procedure that computes the
odd harmonics of X(k) using an jV/2-point DFT.
528"' Frequency-domain sampling

Consider the following discrete-time signal

C( n ) = h ' " '
} 0,

|n| > L

where a = 0.95 and L = 10

(a) Com pute and plot the signal x(n).
(b) Show that
x(n)e

X (w )=

= x(0) + 2 ^ ^ x(n) cos wn

Plot X{i) by computing it at w = jri/1 0 0 , k = 0, 1........100.

(c) Com pute
N

\N

for N = 30.
(d) D eterm ine and plot the signal
x(n) =
*=o
W hat is the relation between the signals x(n) and x ( n)l Explain.
(e) Com pute and plot the signal ii(n ) =
X x(n - IN), - L < n < L for N = 30.
Com pare the signals x(rt) and X](n).
(f) R epeat parts (c) to (e) for N = 15.

5.29* Frequency-domain sampling The signal x(n) = a 1"1, - 1 < a < 1 has a Fourier
transform
X(w) =

1
1 2a cos <o + a2

(a) Plot X(w) for 0 < w < 2jt, a = 0.8.

Reconstruct and plot X(w) from its samples X Qj r k / N) , 0 < k < N - 1 for:
(b) N = 20
(c) N = 100
(d) Com pare the spectra obtained in parts (b) and (c) with the original spectrum
X(a>) and explain the differences.
(e) Illustrate the time-domain aliasing when N = 20.

The Discrete Fourier Transform: Its Properties and Applications

446

5 J0 * Frequency analysis o f amplitude-modulated discrete-time signal

(a)
(b)
(c)
(d)
(e)

Chap. 5

T he discrete-time

Sketch the signals jr(n), xc(/i), and *m(n), 0 < n < 255.
Com pute and sketch the 128-point D FT of the signal
0 < n < 127.
Compute and sketch the 128-point D FT of the signal xtm(n), 0 < n < 99.
Com pute and sketch the 256-point D FT of the signal jcani{n), 0 < n < 179.
Explain the results obtained in parts (b) through (d), by deriving the spectrum of
the am plitude-m odulated signal and comparing it with the experim ental results.

5.31* The sawtooth waveform in Fig. P5.31 can be expressed in the form of a Fourier series
as

)
(a) D eterm ine the Fourier series coefficients c*.
(b) Use an N -point subroutine to generate samples of this signal in the time domain
using the first six term s of the expansion for N 64 and N = 128. Plot the signal
x(f) and the samples generated, and com m ent on the results.

Figure P531
5 3 2 Recall that the Fourier transform of x (r) = eJ0* is X ( j i 2) = 2jtS(2 - i2o) and the
Fourier transform of
0 < t < To
otherwise
is
e-jOT<i/l

(a) D eterm ine the Fourier transform Y ( j n ) of

y(r) = p(t)eja'
and roughly sketch |K0 '2)| versus 2.

Chap. 5

Problems

447

(b) Now consider the exponential sequence

jr(n) =
where <uo is some arbitrary frequency in the range 0 < ojo < tt radians. Give the
most general condition that a>o must satisfy in order for x(n) to be periodic with
period P (P is a positive integer).
(c) Let y(n) be the finite-duration sequence
v (n ) =

x ( n ) w N (n)

e iw " u i s ( n )

where w^( n) is a finite-duration rectangular sequence of length N and where

x(n) is not necessarily periodic. D eterm ine Y(a)) and roughly sketch \Y(a>)\ for
0 < to < 2n. W hat effect does N have in | y ( o j ) | ? Briefly com ment on the
similarities and differences between jY(a>)\ and ]y(j'S2)|.
(d> Suppose that
*(n) = e>{1' ! p)n

p a positive integer

and
y(n) = w N(n)x(n)
where N = IP, I a positive integer. Determ ine and sketch the N-point D FT of
y(n). R elate your answer to the characteristics of |K{w)|.
Is the frequency sampling for the D FT in part (d) adequate for obtaining a rough
approxim ation of |K(w)| directly from the magnitude of the D FT sequence |K(/t)|?
If not. explain briefly how the sampling can be increased so that it will be possible
to obtain a rough sketch of |K(o)| from an appropriate sequence |y (Jt) | .

Efficient Computation of the

DFT: Fast Fourier Transform
Algorithms

A s w e h av e o b se rv e d in th e p reced in g c h a p te r, th e D isc re te F o u rie r T ra n sfo rm

(D F T ) play s an im p o rta n t ro le in m an y a p p lic a tio n s o f digital signal processing,
in clu d in g lin e a r filterin g , c o rre la tio n an aly sis, a n d sp e c tru m analysis. A m ajor
re a so n fo r its im p o rta n c e is th e ex isten ce o f efficient a lg o rith m s fo r c o m p u tin g the
DFT.
T h e m ain to p ic o f this c h a p te r is th e d e sc rip tio n o f c o m p u ta tio n a lly efficient
a lg o rith m s fo r e v a lu a tin g th e D F T . T w o d iffe re n t a p p ro a c h e s a re d e sc rib e d . O n e is
a d iv id e -a n d -c o n q u e r a p p ro a c h in w hich a D F T o f size N , w h e re jV is a c o m p o site
n u m b e r, is re d u c e d to th e c o m p u ta tio n o f sm a lle r D F T s fro m w hich th e larg er
D F T is co m p u te d . In p a rtic u la r, w e p r e s e n t im p o rta n t c o m p u ta tio n a l alg o rith m s,
called fast F o u rie r tra n sfo rm (F F T ) a lg o rith m s, fo r c o m p u tin g th e D F T w h en the
size N is a p o w e r o f 2 a n d w h e n it is a p o w e r o f 4.
T h e se co n d a p p ro a c h is b a s e d o n th e fo rm u la tio n o f th e D F T as a lin ear
filterin g o p e ra tio n o n th e d a ta . T h is a p p ro a c h le a d s to tw o a lg o rith m s, th e G o ertzel
alg o rith m an d th e ch irp -z tra n sfo rm a lg o rith m fo r co m p u tin g th e D F T via linear
filterin g o f th e d a ta se q u e n c e .

6.1 EFFICIENT COMPUTATION OF THE DFT: FFT ALGORITHMS

In th is sectio n w e p re s e n t se v e ra l m e th o d s fo r c o m p u tin g th e D F T efficiently.
In view o f th e im p o rta n c e o f th e D F T in v a rio u s d ig ital sig n a l p ro cessin g ap
p licatio n s, such as lin e a r filtering, c o rre la tio n an aly sis, a n d s p e c tru m analysis, its
efficien t c o m p u ta tio n is a to p ic th a t h a s re c e iv e d c o n s id e ra b le a tte n tio n by m any
m a th e m a tic ia n s, e n g in e e rs, a n d a p p lie d scien tists.
B asically , th e c o m p u ta tio n a l p ro b le m fo r th e D F T is t o c o m p u te th e se q u en ce
{X(*)} o f N c o m p lex -v alu ed n u m b e rs g iv en a n o th e r se q u e n c e o f d a ta (x(n)} of
448

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

449

le n g th N , a c c o rd in g to th e fo rm u la
N- 1

* (* ) = V

Jt(n) W N
kn

0 < k < N -l

(6.1.1)

w h ere
WN = e - }7* ,N

(6.1.2)

In g e n e ra l, th e d a ta se q u e n c e x ( n ) is also a s su m e d to b e c o m p lex v alu ed .

S im ilarly , th e I D F T b ec o m e s
1 n -i
jr(n ) = ^ X ( J k ) W ^ " *
^ *=o

0 < n < N I

(6.1.3)

Since th e D F T a n d ID F T involve b asically th e sa m e ty p e o f c o m p u ta tio n s, o u r

d iscu ssio n o f efficien t c o m p u ta tio n a l a lg o rith m s fo r th e D F T a p p lie s as w ell to th e
efficien t c o m p u ta tio n o f th e ID F T .
W e o b se rv e th a t fo r each v alu e o f k , d ire c t c o m p u ta tio n o f X ( k ) involves
N co m p lex m u ltip lic a tio n s ( 4 N real m u ltip lic a tio n s) a n d N 1 co m p lex a d d itio n s
(4 JV -2 re a l a d d itio n s). C o n se q u e n tly , to c o m p u te all N v alu es o f th e D F T re q u ire s
jV2 c o m p lex m u ltip lic atio n s a n d N 2 N c o m p lex ad d itio n s.
D ire c t c o m p u ta tio n o f th e D F T is b asically in efficien t p rim a rily b e c a u se it
d o e s n o t e x p lo it th e sy m m etry a n d p e rio d ic ity p ro p e rtie s o f th e p h a s e fa c to r WV
In p a rtic u la r, th e s e tw o p r o p e rtie s are:
S y m m etry p ro p e rty :

Wk
N+N/2 = W N
L

(6.1.4)

P e rio d ic ity p ro p e rty :

W#+N = W N
k

(6.1.5)

T h e c o m p u ta tio n a lly efficient a lg o rith m s d e s c rib e d in th is se c tio n , k n o w n collec

tiv ely as fa st F o u rie r tra n sfo rm (F F T ) a lg o rith m s, e x p lo it th e s e tw o b asic p ro p e rtie s
o f th e p h a s e facto r.

6.1.1 Direct Computation of the DFT

F o r a co m p le x -v a lu e d se q u e n c e x ( n ) o f N p o in ts, th e D F T m a y b e ex p re sse d as
X R{k) = Y

|* * ( n ) c o s ^ ~ p - + x / ( n ) s i n ^ ^ - j

Xi(k) = - Y

|x ,f ( n ) s in

- x , ( n ) co s

T h e d ire c t c o m p u ta tio n o f (6.1.6) a n d (6.1.7) re q u ire s :

L 2 N 2 e v a lu a tio n s o f trig o n o m e tric fu n ctio n s.
2. 4 N 2 re a l m u ltip lic atio n s.

(6.1.6)

(6.1.7)

450

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

3. AN ( N 1) real additions.

4. A num ber o f indexing and addressing op eration s.

T hese operations are typical o f D F T com putational algorithm s. T he operations
in item s 2 and 3 result in the D F T values X K(k) and X i ( k ) . T h e indexing and
addressing op eration s are necessary to fetch the data x(n'), 0 < < Ar 1, and
the phase factors and to store th e results. T h e variety o f D F T algorithm s optim ize
each o f these com p utational p rocesses in a different way.

6.1.2 Divide-and-Conquer Approach to Computation of

the DFT
T h e d evelop m en t o f com p utationally efficient algorithm s for the D F T is m ade pos
sible if w e adopt a divide-and-conquer approach. T his approach is based on the
d ecom p osition o f an Af-point D F T in to su ccessively sm aller D F T s. T his basic ap
proach leads to a fam ily o f com putationally efficient algorithm s k n ow n collectively
as FFT algorithm s.
T o illustrate the basic n otion s, let us consider the com p utation o f an Appoint
D F T , where N can be factored as a product o f tw o integers, that is,
N = LM

(6.1.8)

T h e assum ption that N is not a prim e num ber is n ot restrictive, sin ce w e can pad
any sequ en ce with zeros to ensure a factorization o f the form ( 6 . 1 .8 ).
N o w the seq u en ce j ( n ) , 0 < n < N 1, can be stored in either a one
dim ensional array in d exed by n or as a tw o-d im en sion al array in d exed by I and
m, w here 0 < / < L 1 and 0 < m < A / - l a s illustrated in Fig. 6.1. N o te that / is
the row index and m is the colum n index. T hus, the se q u en ce x (n ) can b e stored
in a rectangular array in a variety o f w ays, each o f which d ep en d s on the mapping
o f index n to the in d exes (/, m).
For exam p le, su p p ose that w e select the m apping
n = Ml + m

(6.1.9)

T his leads to an arrangem ent in which the first row consists o f th e first M elem ents
o f x ( n ) , the secon d row consists o f the n ext M elem en ts o f x ( n ) , and so on, as
illustrated in Fig. 6.2 (a ). O n the other hand, the m apping
n l + mL

(6.1.10)

stores the first L elem en ts o f x ( n ) in the first colu m n, the n ext L elem en ts in the
second colum n, and so on , as illustrated in Fig. 6.2(b ).
A sim ilar arrangem ent can be u sed to store th e com p u ted D F T valu es. In
particular, the m apping is from the in d ex it to a pair o f in d ices (p , q), where
0 < p < L 1 and 0 < q < M 1. If w e se lect the m apping

k Mp

(6 .1 .1 1 )

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

n ----------

451

...

x(\)

M2)

- 1

JcCW-1)

(a)
column index
row index

*(0,0)

x(0,1)

* 1 .0 )

* 1 ,1 )

*(2,0)

*2,1)

Af-1

L- 1
(b)

Figure 6.1
N -l.

T w o dim ensional data array for storing the sequence

x(n).

0 < n

the D F T is stored on a row -w ise basis, w here the first row contains the first M
elem en ts o f the D F T X ( k ) , the second row contains the next set of M elem en ts,
and so on . O n the other hand, th e m apping

(6 . 1. 12)

k = qL + p

results in a colu m n-w ise storage o f X (Jt), w here the first L elem en ts are stored in
the first colu m n, th e secon d se t o f L elem en ts are stored in the secon d colum n,
and so on.
N o w su p p ose that x ( n) is m apped in to the rectangular array x ( l , m ) and X( k )
is m app ed in to a corresp on d ing rectangular array X ( p , q). T h en the D F T can be
exp ressed as a d o u b le sum o ver th e elem en ts o f the rectangular array m ultiplied
b y th e corresp on d ing p h ase factors. T o b e specific, let us ad op t a colum n-w ise
m apping fo r x ( n ) g iven by (6.1.10) and th e row -w ise m apping for the D F T given
by (6.1.11). T hen
(6.1.13)

X ( p , q) = Y Y x{1' m ) W < p+qHmL+t)

rrtacO 1=0
But
p+ q){m L+ i)

^M L rnp ^m L q

H o w ev er, W%mp = 1, W%*L = W $ L = W**, and W * pl = W?'

(6.1.14)
= W[l

452

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Row-wise

Chap.

Ml + m

M- 1
0

*0)

MM)

M 2M )

*2)

M D

MM

+ 1)

M 2M +

1)

M M + 2)

M 2M

1)

M M -

+ 2)

L - 1 x( (L- IW) M(L - 1 )Af + 1) x L -l)M + 2)

M2M

- 1)

M 'iM -

1)

x{L M -

1)

(a)
Column-wise

M- 1
0

MO )

jt(1)

M L+

M2)

M L + 2)

L -l

x(L

- I)

M U -

x((M

M 2L)

ML)

1)

1)

x(2L

+ 1)

M 2 L + 2)

M IL

- 1)

x((M -

1) L )
I )L + 1)

M (M -l)L + 2 )

MLM -

1)

(b)

Figure <L2 Two arrangements for the data arrays.

W ith th ese sim p lification s, (6.1.13) can be exp ressed as
L- l
X( p, q) = Y
/*0

(6.1.15)
^|R0

The expression in (6.1.15) in volves the com p utation o f D F T s o f length M and

length L. T o elab orate, let us subdivide th e com p utation into th ree steps:
L First, w e com p ute th e M -point D F T s
M-l
F(l,q) = x ( / ,m ) W ^ \

for each of the rows I = 0 ,1 ........ L - l .

0 < q < M - 1

( 6 .1 .1 6 )

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

453

2 . S econ d , w e com p ute a new rectangular array G ( l , q ) defined as

(6.1.17)
3. F inally, w e com p ute the L -point D F T s
L-l
X ( p , q ) = J 2 G ^l ' ^ WL

(6.1.18)

for each colum n q = 0 , 1 , . . . , M 1, o f the array G (l, q).

O n the surface it m ay appear that the com putational procedure outlined
above is m ore com p lex than the direct com p utation o f the D F T . H ow ever, let
us evalu ate the com putational com p lexity o f (6.1.15). T h e first step in volves the
com p utation o f L D F T s, each o f M points. H en ce this step requires L M 2 com
plex m ultiplications and L M { M 1) com p lex additions. T h e secon d step requires
L M com p lex m ultiplications. Finally, the third step in the com p utation requires
M L 2 com p lex m ultiplications and M L ( L 1) com p lex additions. T h erefore, the
com p utational com plexity is
C om plex m ultiplications:

N ( M + L + 1)

C om plex additions:

N ( M + L 2)

(6.1.19)
w here N = M L . T hus the num ber o f m ultiplications has b een reduced from N 2
to N ( M + L + 1 ) and the num ber o f additions has b een reduced from N ( N 1) to
N ( M + L 2).
For exam p le, suppose that N = 1000 and w e select L = 2 and M = 500.
T h en , instead o f having to perform 106 com p lex m ultiplications via direct com pu
tation o f the D F T , this approach leads to 503,000 com p lex m ultiplications. This
rep resents a reduction by approxim ately a factor o f 2. T h e num ber o f additions is
also red u ced by ab ou t a factor o f 2.
W h en N is a highly com p osite num ber, that is, N can be factored in to a
product o f prim e num bers o f th e form
N = r\ r 2 - r v

(6.1.20)

then the d ecom p osition ab ove can b e rep eated (v - 1 ) m ore tim es. T his procedure
results in sm aller D F T s, w hich, in turn, lead s to a m ore efficient com putational
algorithm .
In effect, th e first segm en tation o f th e seq u en ce x ( n ) in to a rectangular array
o f M colu m ns w ith L elem en ts in each colu m n resu lted in D F T s o f sizes L and M .
Further d eco m p o sitio n o f th e data in effect in volves th e segm en tation o f each row
(or colu m n ) into sm aller rectangular arrays w hich result in sm aller D F T s. This
p roced u re term inates w h en N is factored in to its prim e factors.

Example 6.1.1
To illustrate this computational procedure, let us consider the computation of an
N = 15 point DFT. Since N = 5 x 3 = 15, we select L = 5 and M = 3. In other

454

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

words, we store the 15-point sequence *(n) column-wise as follows:

Row
Row
Row
Row
Row

1:
2:
3:
4:
5:

*(0, 0)
*(1,0)
*(2,0)
*(3,0)
* (4,0)

= *(0)
= * (1 )
= * (2 )
= * (3 )
= * (4 )

*(0.1)
*(1, 1)
x ( 2 , 1)
*(3,1)
*(4,1)

= * (5 )
= jc(6)
= x(7)
= * (8 )
= * (9 )

*(0, 2) = *(10)
*(1,2) = *(11)
x(2,2)=x(12)
*(3,2) = .r{13)
*(4,2) = *(14)

Now. we com pute the three-point DFTs for each of the five rows. This leads
to the following 5 x 3 array:
F(0, 0)
F a , o)
F( 2. 0)
F(3, 0)
F(4. 0)

F (O .l)
F O .l)
F( 2,1)
FO. 1)
F (4.1)

F( 0.2)
F (1.2)
F( 2.2)
FO- 2)
F( 4.2)

The next step is to multiply each of the terms F(l, q) by the phase factors
= M/jj. 0 < / < 4 and 0 < q < 2. This computation results in the 5 x 3 array:
Column 1

Column 2

Column 3

G (0.0)
G (1,0)
G(2, 0)
G (3.0)
G (4 .0)

C(0. 1)
C ( l.l)
C(2. 1)
G (3 ,1)
G (4 .1)

C (0.2)
G (l. 2)
C ( 2 .2)
G (3 .2)
G (4.2)

The final step is to com pute the five-point DFTs for each of the three columns.
This com putation yields the desired values of the D FT in the form
X (0.0)
X (1,0)
X (2.0)
X (3,0)
X (4,0)

=
=
=
=
=

X(0)
X(3)
X (6)
X (9)
X (12)

X (0 ,1) =
X (1.1) =
X (2 .1) =
X (3,1) =
X (4,1) =

X (l)
X (4)
X(7)
X(10)
X (13)

X (0,2)
X (1,2)
X (2,2)
X (3 ,2)
X (4 ,2)

=
=
=
=
=

X(2)
X(5)
X(8>
X ( ll)
X(14)

Figure 6.3 illustrates the steps in the computation.

It is interesting to view the segm ented data sequence and the resulting D FT in
term s of one-dim ensional arrays. When the input sequence x(n) and the output DFT
X(jfc) in the two-dimensional arrays are read across from row 1 through row 5, we
obtain the following sequences:
IN PU T A R R A Y
*(0) x{5) *(10) *(1) *(6) *(11) x(2) *(7) *(12) *(3) *(8) *(13) x(4) *{9) *(14)
O U T PU T A R R A Y
X(0) X (l) X(2) X(3) X(4) X(5) X(6) X(7) X(8) X(9) X(10) X ( ll) X(12) X(13) X(14)
W e observe that the input data sequence is shuffled from the norm al order
in the com putation of the D FT. On the other hand, the output sequence occurs in
norm al order. In this case the rearrangem ent of the input data array is due to the

Sec. 6.1

455

Efficient Computation of the DFT: FFT Algorithms

Figure 63
DFTs.

Computation of N = 15-point DFT by means of 3-point and 5-point

segm entation of the one-dim ensional array into a rectangular array and the order in
which the D FTs are com puted. This shuffling of either the input data sequence or
the output D FT sequence is a characteristic of most FFT algorithms.
T o sum m arize, the algorithm that w e have introduced in volves the follow in g
com putations:

Algorithm 1
1. S tore the signal colu m n-w ise.
2. C om pute the Af-point D F T o f each row.
3. M ultiply the resulting array by the p h ase factors
4. C om p ute the L -point D F T o f each colum n
5. R ea d the resulting array row -w ise.
A n additional algorithm with a sim ilar com putational structure can b e o b
tained if th e input signal is stored row -w ise and the resulting transform ation is.
colu m n-w ise. In this case w e select as
n = Ml + m
k

(6.1.21)

= qL + p

This ch o ice o f in d ices lead s to the form ula for the D F T in the form
X {p ,q )

= * (/,
msO 1*0

(6. 1.22)

M- l
= E > c
Thus w e obtain a secon d algorithm .

urmp
WN

456

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

A lg o rith m 2
1. S to re th e sig n al row -w ise.
2. C o m p u te th e L -p o in t D F T a t each co lu m n .
3. M u ltip ly th e re su ltin g a rra y by th e fa c to rs W%m.
4. C o m p u te th e A f-point D F T o f e a c h row .
5. R e a d th e re su ltin g a rra y colum n-w ise.
T h e tw o a lg o rith m s given a b o v e h a v e th e sa m e c o m p lex ity . H o w e v e r, they
d iffer in th e a rra n g e m e n t o f th e c o m p u ta tio n s. In th e follow ing se c tio n s w e exploit
th e d iv id e -a n d -c o n q u e r a p p ro a c h to d e riv e fast a lg o rith m s w h e n th e size o f the
D F T is re stric te d to b e a p o w e r o f 2 o r a p o w e r o f 4.

6.1.3 Radix-2 FFT Algorithms

In th e p re c e d in g se ctio n w e d e sc rib e d fo u r a lg o rith m s fo r efficien t c o m p u ta tio n of
th e D F T b ased o n th e d iv id e -a n d -c o n q u e r a p p ro a c h . S uch an a p p ro a c h is applica
b le w h en th e n u m b e r N o f d a ta p o in ts is n o t a p rim e . In p a rtic u la r, th e a p p ro ach
is v ery efficien t w h en N is highly c o m p o s ite , th a t is, w h en N can b e fa c to re d as
N = r\r2ry rv, w h e re th e {r,} are prim e.
O f p a rtic u la r im p o rta n c e as th e case in w hich r i = r2 = r v = r , so th at
N = r ' \ In such a case th e D F T s a re o f size r , so th a t th e c o m p u ta tio n o f the
N -p o in t D F T h a s a re g u la r p a tte rn . T h e n u m b e r r is called th e rad ix o f th e F F T
alg o rith m .
In th is se c tio n w e d escrib e rad ix -2 a lg o rith m s, w hich a re by far th e m ost
w idely u sed F F T alg o rith m s. R ad ix -4 a lg o rith m s a re d e sc rib e d in th e follow ing
sectio n .
L e t us c o n s id e r th e c o m p u ta tio n o f th e N 2 V p o in t D F T by th e dividea n d -c o n q u e r a p p ro a c h specified by (6.1.16) th ro u g h (6.1.18). W e select M = N / 2
a n d L = 2. T h is se lectio n re su lts in a sp lit o f th e N -p o in t d a ta se q u e n c e in to two
// /2 - p o in t d a ta se q u e n c e s f \ ( n ) a n d f 2(ri), c o rre sp o n d in g to th e ev en -n u m b ered
a n d o d d -n u m b e re d sa m p le s o f x ( n ), resp ec tiv ely , th a t is,
/ i ( n ) = x ( 2 n)
f i ( n ) = x (2n + 1),

N
n = 0 ,1 ,..., ~ 1

(6.1.23)

T h u s f i ( n ) a n d f j ( n ) a re o b ta in e d by d e c im a tin g x ( n) b y a f a c to r o f 2, a n d hence
th e re su ltin g F F T a lg o rith m is called a d e c im a tio n -in -tim e a lg o rith m .
N o w th e Af-point D F T c a n b e e x p re ss e d in te rm s o f th e D F T s o f th e deci
m a te d se q u e n c e s as follow s:
A'-l
X(k) = Y x ^
n*0

wn

* = 0 ,1 ,..., A f-1

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

=

x (n )H # + 5 3

n even

(6.1.24)

n odd

<W/2)-l

(Af/2)-l

J ! x ( 2 m ) W ] f k 4- 5 3 *(2m + l)W * (2'n+1>

m=0
m=0

=
B ut

457

= W ^/ 2- W ith this su b stitu tio n , (6.1.24) c a n b e e x p re ss e d as

(N/Zi-i

x(k)=

5 3

(N/2)l

M m )w *ra + K

= fi(Jfc) + W * f 2(*)

it = 0 , 1 , . . . , W - l

w h e re F i(it) a n d F2 (k) a re th e N /2 -p o in t D F T s o f th e se q u e n c e s f \ ( m ) an d f 2 (m),

resp ec tiv ely .
S ince F\ ( k) a n d F 2 (k) a re p e rio d ic, w ith p e rio d N f l , w e h a v e F](A -f N / 2) =
F i(* ) a n d /^(jfc + N f l ) = / i t * ) - 1 a d d itio n , th e fa c to r W ^ Nfl = Wfa. H e n c e
(6.1.25) can b e ex p re sse d as
X ( k ) = f i(J t) + W* F2(fc)

= F , ( * ) - < F 2(/:)

* = 0 , 1 .........~ 1

(6.1.26)

* = 0 , 1 ....... y - 1

(6.1.27)

W e o b se rv e th a t th e d ire c t c o m p u ta tio n o f F\ ( k) re q u ire s ( N /2 )2 com plex

m u ltip lic a tio n s. T h e sa m e a p p lie s to th e c o m p u ta tio n o f F2 (k). F u rth e rm o r e , th e re
a re N f l a d d itio n a l co m p lex m u ltip lic a tio n s re q u ire d to c o m p u te W kN F 2 (k). H en ce
th e c o m p u ta tio n o f X ( k ) re q u ire s 2 ( N f l )1 4- N f l = N 2/ 2 + N f l c o m p lex m u ltip li
c a tio n s. T h is first s te p resu lts in a re d u c tio n o f th e n u m b e r o f m u ltip lic a tio n s from
N 2 to N 1 f l + N f l , w h ich is a b o u t a fa c to r o f 2 fo r N large.
T o b e c o n s iste n t w ith o u r p rev io u s n o ta tio n , w e m ay define
</!(*) = F l(* )

* = 0 ,l,...,y - l

G 2 (k) = W N
l F2 (k)

* = 0 , l , . . . , y 1

T h e n th e D F T X (it) m ay b e ex p re sse d as
X ( k ) = G x(k) + G 2 (k)

it = 0 , 1 ____ y - 1
(6.1.28)

X(k + j )

= G x( k ) - G 2 (k)

* = 0 , 1 .........y - 1

T h is c o m p u ta tio n is illu stra te d in Fig. 6.4.

H a v in g p e rfo rm e d th e d e c im a tio n -in -tim e o n ce, w e c a n r e p e a t th e p ro cess
fo r e a c h o f th e se q u e n c e s f \ ( n ) a n d f 2 (n). T h u s f \ ( n ) w o u ld re s u lt in th e tw o

458

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

* 0 ) x{2) x(4)

Chap. 6

JrW -2)

Figure 6.4 F irst step in the decim ation-in-tim e algorithm .

/V /4-point se q u e n c e s
v n (n ) = / ] ( 2n)
v\ 2 (n) = f \ Q n + 1)

N
= 0 , 1 ................... 1
4
n = 0, 1.........j

(6.1.29)

- 1

an d f 2 (n) w o u ld yield
V2\(n) = f 2 (2 n)

N
n = 0, 1......... - 1
4

V22(n) = /2(2n + 1)

N
n = 0 , 1......... - 1

(6.1.30)

B y co m p u tin g jV /4-point D F T s , w e w o u ld o b ta in th e ///2 - p o in t D F T s Fi(Jfc) and

F2 (k) fro m th e re la tio n s
FiOfc) = V(Jfc) + W k
Nf2 Vn (k)

k = 0 ,1 ,

1
(6.1.31)

Fx (* + t ) = Vl1{k) - KpynW
F 2 (k) = V21(*) + W N
k / 2 V22(k)

k=

1..... 7 - 1

k = 0 ,1 ,..., J

- 1
(6.1.32)

F2 ( * + j )

= V2i (*) -

N
k = 0, . , , , j - l

where the (Vi; (jt)} are the ///4 -p o in t D F T s o f th e seq u en ces {u,;(n)}.

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

459

COMPARISON OF COMPUTATIONAL COMPLEXITY FOR THE

DIRECT COMPUTATION OF THE DFT VERSUS THE FFT ALGORITHM

TABLE 6.1

Number of
Points,
N

Complex Multiplications
in Direct Computation,
N2

Complex Multiplications
in FFT Algorithm,
(JV/2) log2 N

Speed
Improvement
Factor

4
8
16
32
64
128
256
512
1,024

16
64
256
1,024
4,096
16,384
65,536
262.144
1,048,576

4
12
32
80
192
448
1,024
2,304
5,120

4.0
5.3
8.0
12.8
21.3
36.6
64.0
113.8
204.8

W e o b se rv e th a t th e c o m p u ta tio n o f {V(J(*)} re q u ire s 4{W /4)2 m u ltip lic a tio n s

a n d h e n c e th e c o m p u ta tio n o f F\ ( k) a n d F 2(Jt) can b e a c c o m p lish ed w ith N 2/ 4 +
N f l c o m p lex m u ltip lic a tio n s. A n a d d itio n a l N f l co m p lex m u ltip lic a tio n s a re re
q u ire d to c o m p u te X ( k ) fro m F i(it) a n d Fi{k), C o n se q u e n tly , th e to ta l n u m b e r o f
m u ltip lic a tio n s is re d u c e d a p p ro x im a te ly by a fa c to r o f 2 ag ain to N 2/ 4 + N.
T h e d e c im a tio n o f th e d a ta se q u e n c e can b e re p e a te d ag ain a n d ag ain until
th e re su ltin g se q u e n c e s a re re d u c e d to o n e -p o in t se q u en ces. F o r N = 2 V, this
d e c im a tio n can b e p e rfo rm e d v = log2 N tim es. T h u s th e to ta l n u m b e r o f co m p lex
m u ltip lic a tio n s is re d u c e d to { N f l ) log2 N . T h e n u m b e r o f co m p lex a d d itio n s is
N log2 N . T a b le 6.1 p re s e n ts a c o m p a riso n o f th e n u m b e r o f co m p lex m u ltip lic a
tio n s in th e F F T a n d in th e d ire c t c o m p u ta tio n o f th e D F T .
F o r illu stra tiv e p u rp o se s , Fig. 6.5 d e p ic ts th e c o m p u ta tio n o f an N = 8 p o in t
D F T . W e o b se rv e th a t th e c o m p u ta tio n is p e rfo rm e d in th r e e stag es, b eg in n in g
w ith th e c o m p u ta tio n s o f fo u r tw o -p o in t D F T s, th e n tw o fo u r-p o in t D F T s , an d

Figure <L5 Three stages in the computation o f an N = 8-point DFT.

460

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Stage

Stage 2

Chap. 6

Stage 3
X (0 )

X(l)

X(2)

X(3)

X(4)

X{5)

*(6)

X p )

finally, o n e eigh t-p oin t D F T . T he com bination o f the sm aller D F T s to form the
larger D F T is illustrated in Fig. 6. 6 for N = 8 .
O bserve that the basic com putation perform ed at every stage, as illustrated
in Fig. 6 .6 , is to take tw o com p lex num bers, say th e pair (a, b), m ultiply b by W N
r,
and then add and subtract the product from a to form tw o new com p lex numbers
(A, B ). This basic com putation, which is show n in Fig. 6.7, is called a butterfly
b ecau se the flow graph resem bles a butterfly.
In general, each butterfly involves on e com p lex m ultiplication and tw o com
plex additions. F or N = 2 V, there are N f l butterflies per stage o f th e com putation
p rocess and log 2 N stages. T h erefore, as previously indicated th e total num ber of
com plex m ultiplications is ( N f l ) log 2 N and com p lex additions is Arlog 2 N .
O nce a butterfly operation is perform ed on a pair o f com p lex num bers (a, b)
to p roduce ( A , B ) , there is no n eed to 'sa v e the input pair ( a , b ) . H en ce w e can
>A = a + W^b

B=a-Wt/b

F igure 6.7 Basic butterfly com putation

in th e decim ation-in-tim e F FT
algorithm .

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

461

store th e result (A , B ) in the sam e location s as ( a , b ) . C on sequ en tly, w e require

a fixed am ount o f storage, n am ely, 2 N storage registers, in order to store the
results ( N com p lex num bers) o f the com p u tation s at each stage. Since th e sam e
2 N storage loca tio n s are used throughout the com p utation o f the JV-point D F T ,
w e say that the c o m p u ta tio n s are d o n e in place.
A secon d im portant observation is con cern ed with the ord er o f the input
data seq u en ce after it is d ecim ated (v - 1 ) tim es. For exam p le, if w e consider
the case w h ere N = 8 , w e know that th e first d ecim ation yield s the seq u en ce
jc(0), x ( 2 ) , x (4 ), * ( 6 ), * (1 ), Jt(3), jr(5), jc(7), and the secon d d ecim ation results in
the seq u en ce jc(0), x (4 ), x (2 ), x ( 6 ), jt(1), x (5 ), jc(3), jc(7). T h is sh u fflin g o f the
input data seq u en ce has a w ell-d efin ed order as can b e ascertained from observing
Fig. 6 .8 , w hich illustrates the d ecim ation o f th e eigh t-p oin t seq u en ce. B y expressing
the in d ex n, in the seq u en ce x ( n ) , in binary form , w e n o te that th e order o f the
d ecim ated d ata seq u en ce is easily ob tain ed by reading the binary representation
o f th e index n in reverse order. T hus the data p oin t jt(3) = *(011) is placed in
position m = 110 or m = 6 in the d ecim ated array. Thus w e say that the data x ( n )
after d ecim ation is stored in bit-reversed order.
W ith th e input data seq u en ce stored in bit-reversed order and the butterfly
com p utations perform ed in p lace, the resulting D F T seq u en ce X ( k ) is ob tain ed
in natural order (i.e., k = 0 , 1 , . . . , N 1). O n the oth er hand, w e should indi
ca te that it is p ossib le to arrange the F F T algorithm such that the input is left
in natural order and the resulting output D F T will occur in bit-reversed order.
Furtherm ore, w e can im pose the restriction that b oth the input data x ( n ) and the
output D F T X ( k ) be in natural order, and d erive an FFT algorithm in which the
com p utations are not d on e in place. H en ce such an algorithm requires additional
storage.
A n o th e r im portant radix-2 FFT algorithm , called th e decim ation -in-freq u en cy
algorithm , is o b tain ed by using the divide-and-conquer approach described in S ec
tion 6.1.2 w ith th e ch oice o f M = 2 and L = N f l . T his ch oice o f param eters
im plies a colu m n-w ise storage o f the input data seq u en ce. T o d erive the algo
rithm, w e begin by splitting the D F T form ula in to tw o sum m ations, o n e o f which
in v o lv es the sum over the first N t 2 data p oin ts and th e secon d sum in volves the
last N I 2 data points. Thus w e obtain
W 2 > -1

X (k) =

N- 1

x( n)W *? + Y

x(n)W %
(6.1.33)

Since WN/2 = (1)*, the exp ression (6.1.33) can b e rew ritten as

(6.1.34)

462

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

Data
decimation 1
Memory

Memory address
(decimal) (binary)

000

(ninino)
(0 0 0)
(0 0 1)
(0 1 0)
(0 1 1)
(1 0 0)
(1 0 !)
(1 1 0)
(1 1 1)

(/To"2 ()

(nnn in ;)

(0 0 0 )
(1 0 0 )
(0 0 1)
(1 0 1 )
(0 10)
(1 1 0 )
(0 1 1)
(1 1 1 )

-
-
-
*
-
-

(0 0 0 )
(1 0 0)
(0 10)
(t 10)
(0 0 1)
(1 0 1)
(0 1 1)
(1 1 ))

-
*
-
-
-*>
-*>
4

(b)
Figure 6Jt

Shuffling of the data and bit reversal.

N ow , let us split (d e c im a te ) X ( k ) in to th e ev en - a n d o d d - n u m b e re d sam p les. Thus

w e o b ta in
(W /2)-l r

x(n) + x

v tr k n

N/2

k = Q, 1 , . . . , y - 1

(6.1.35)

an d
(A 72)-l

X{2k + \) =

JV \"1

"
w h ere w e h av e u se d th e fact th a t Wf, = 'Wsp.-

N
* = 0 , 1 ......y - 1
(6.1.36)

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

463

If w e d e fin e th e N /2 -p o in t se q u e n c e s gi ( n) a n d gz(n) as
g i( n ) = * ( ) + *
(6.1.37)
g 2 (n) = |* ( n ) - x

+ y^)

n = 0 , 1 , 2 .........J

- 1

th e n
(N/2)-l
x ( 2 k) =

s m K )2

n=0

(6.1.38)

(AT/2)1

& W WN/2

X (2* + l ) =
n=0

T h e c o m p u ta tio n o f th e s e q u e n c e s g i(n ) a n d g 2 (n) a c co rd in g to (6.1.37) a n d th e

su b s e q u e n t u se o f th e s e se q u e n c e s to c o m p u te th e N /2 -p o in t D F T s a re d e p ic te d in
Fig. 6.9. W e o b se rv e th a t th e b asic c o m p u ta tio n in th is figure in v o lv es th e b u tte rfly
o p e r a tio n illu stra te d in Fig. 6.10.
T h is c o m p u ta tio n a l p ro c e d u re can b e re p e a te d th ro u g h d e c im a tio n o f th e
N /2 -p o in t D F T s , X ( 2 k ) a n d X ( 2 k + 1). T h e e n tire p ro cess in v o lv es v = log2 N

464

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

A = a + b

wi,
B = ( a b)W f/
-

Figure 6.10 Basic butterfly com putation

in the decim ation-in-frequency F FT
algorithm .

stages o f decim ation , w here each stage in volves N t l butterflies o f the type shown in
Fig. 6.10. C onsequently, the com putation o f the Af-point D F T via the decim ationin-frequency FFT algorithm , requires ( N / 2) iog 2 N com p lex m ultiplications and
N lo g 2 N com p lex additions, just as in the d ecim ation -in-tim e algorithm . For il
lustrative purposes, the eight-point d ecim ation-in-frequency algorithm is given in
Fig. 6.11.
W e observe from Fig. 6.11, that the input data x ( n ) occurs in natural order,
but the output D F T occurs in bit-reversed order. W e also n ote that the com puta
tions are perform ed in place. H ow ever, it is p ossib le to reconfigure the decim ationin-frequency algorithm so that the input seq u en ce occurs in bit-reversed order
w hile the output D F T occurs in norm al order. F urtherm ore, if w e abandon the
requirem ent that the com putations b e d on e in place, it is also p ossib le to have
both the input data and the output D F T in norm al order.

Figure 6.11

N=

8-point decimation-in-frequency FFT algorithmn.

Sec. 6.1

Efficient Computation of the DFT: FFT Algorithms

465

6.1.4 Radix-4 FFT Algorithms

W h e n th e n u m b e r o f d a ta p o in ts N in th e D F T is a p o w e r o f 4 (i.e., N = 4 l ), w e
can , o f c o u rse, alw ays use a radix-2 a lg o rith m fo r th e c o m p u ta tio n . H o w e v e r, fo r
th is case, it is m o re efficien t c o m p u ta tio n a lly to em p lo y a rad ix -4 F F T alg o rith m .
L e t u s b eg in by d escrib in g a rad ix -4 d e c im a tio n -in -tim e F F T a lg o rith m , w hich
is o b ta in e d by se lectin g L = 4 an d M = N / 4 in th e d iv id e -a n d -c o n q u e r a p p ro a c h
d e s c rib e d in S ectio n 6.1.2. F o r this ch o ice o f L a n d M , w e h av e /, p 0 ,1 , 2, 3: m,
q = 0, 1.........N J4 - 1; n = 4m + /; a n d k = ( N / 4) p + q. T h u s w e sp lit o r d e c im a te
th e W -point in p u t se q u e n c e in to f o u r su b s e q u e n c e s, x ( 4 n ), jc(4n + 1), x( 4n + 2),
x ( 4 n -f 3), n = 0, 1.........N / 4 1By a p p ly in g (6.1.15) w e o b ta in
3

* ( /> . ) =

[w ^F il'q ^W ?

0 ,1 .2 .3

(6.1.39)

w h ere F ( I . q ) is giv en by (6.1.16), th a t is.

(iV/4 |!

F(l.q)=

mq
x ( l - n i ) W N/A

I = 0 .1 , 2. 3.
N
.........4 - 1

(6.1.40)

an d
(6.1.41)

x(l . m ) = x ( 4 m -j- /)
(N
X(p.q) = X / p + q

(6.1.42)

T h u s, th e fo u r ///4 -p o in t D F T s o b ta in e d fro m (6.1.40) a re c o m b in e d a cco rd in g

to (6.1.39) to yield th e W -point D F T . T h e ex p re ssio n in (6.1.39) fo r co m b in in g
th e ///4 -p o in t D F T s d efin es a rad ix -4 d e c im a tio n -in -tim e b u tterfly , w hich can be
e x p re ss e d in m atrix fo rm as
~X( Q , q )
X(\,q)
X (2,q)
-X(3,q)J

- 1 1 1 1
1

' j

-1

W F(0,<7)
j

1 - 1 1 - 1
Li j -i - j

WF(Lq)
W % F(2 ,q)

(6.1.43)

w l qF { X q )

T h e ra d ix -4 b u tte rfly is d e p ic te d in Fig. 6 .1 2 (a) a n d in a m o re co m p a c t fo rm

in Fig. 6 .1 2 (b ). N o te th a t since W = 1, e a c h b u tte rfly in v o lv es th r e e co m p lex
m u ltip lic a tio n s , a n d 12 c o m p lex a d d itio n s.
T h is d e c im a tio n -in -tim e p ro c e d u re can b e re p e a te d recu rsiv ely v tim es. H e n c e
th e re su ltin g F F T alg o rith m co n sists o f v sta g es, w h e re e a c h sta g e c o n ta in s A74
b u tte rflie s . C o n s e q u e n tly , th e c o m p u ta tio n a l b u r d e n fo r th e a lg o rith m is 3 v N / 4 =
(3jV /8) lo g ; N c o m p lex m u ltip lic a tio n s a n d O N f l ) log2 N co m p lex a d d itio n s. W e
n o te th a t th e n u m b e r o f m u ltip lic a tio n s is re d u c e d by 2 5 % , b u t th e n u m b e r o f
a d d itio n s h a s in c re a se d b y 50% fro m N log2 N to O N f l ) log2 N .

466

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

Figure 6.12 Basic butterfly computation in o radix-4 FFT algorithm.

It is in te re stin g to n o te , h o w ev er, th a t by p e rfo rm in g th e a d d itio n s in tw o

step s, it is p o ssib le to re d u c e th e n u m b e r o f a d d itio n s p e r b u tte rfly fro m 12 to 8.
T h is can b e acco m p lish ed by e x p ressin g th e m atrix o f th e lin e a r tra n s fo rm a tio n in
(6.1.43) as a p ro d u c t o f tw o m atric es as follow s:
m ? n
X ( l,9 )
X ( 2,q)

-1
0
1
.0

0
1
0
1

1
0
-1
0

0 -j
0
j

1
1
0
.0

0
0
1
1

1
-1
0
0

0
0
1
-1

w jjm g )
WqF(\,q)
W % F ( 2 .q)

(6.1.44)

lW *F(3,q).

N ow ea c h m atrix m u ltip lic a tio n involves fo u r a d d itio n s fo r a to ta l o f e ig h t ad d i

tio n s. T h u s th e to ta l n u m b e r o f co m p lex a d d itio n s is re d u c e d to N log2 N , w hich
is id en tical to th e ra d ix -2 F F T a lg o rith m . T h e c o m p u ta tio n a l sa v in g s re su lts from
th e 25% re d u c tio n in th e n u m b e r o f co m p lex m u ltip lic atio n s.
A n illu stra tio n o f a rad ix -4 d e c im a tio n -in -tim e F F T a lg o rith m is sh o w n in
Fig. 6.13 fo r N = 16. N o te th a t in th is a lg o rith m , th e in p u t se q u e n c e is in norm al
o r d e r w h ile th e o u tp u t D F T is shuffled. In th e rad ix -4 F F T a lg o rith m , w here
th e d e c im a tio n is b y a f a c to r o f 4, th e o r d e r o f th e d e c im a te d se q u e n c e can be
d e te rm in e d b y re v e rsin g th e o r d e r o f th e n u m b e r th a t re p re s e n ts th e in d ex n
in a q u a te rn a ry n u m b e r sy stem (i.e., th e n u m b e r sy stem b a s e d o n th e digits 0,
1, 2, 3).
A rad ix -4 d e c im a tio n -in -fre q u e n c y F F T a lg o rith m can b e o b ta in e d by se lect
ing L = N / 4 , M = 4; /, p = 0, 1.........N / 4 - 1; m, q = 0, 1, 2, 3; n = {N / 4 ) m + /;
a n d k = 4 p + q. W ith th is ch o ice o f p a ra m e te rs , th e g e n e ra l e q u a tio n g iven by

Efficient Computation of the DFT: FFT Algorithms

467

Figure 6.13 Sixteen-point radix-4 decimation-in-time algorithm with input in nor

mal order and output in digit-reversed order.
(6.1.15) can be exp ressed as
(A y 4 )- l

X(p,q) =

Ip

C ( l , q) W' NfA
l

(6.1.45)

1=0

w here
q = 0 , 1 , 2, 3
G (l,q) =

w hF (l,

q)

(6.1.46)
i - 0 . 1 ......... - 1
4

and
q
F(l,q) = Y x ( l , m ) W ?

=0,1,2,3

N
/ = 0 , 1 , 2 , 3 .........- - 1
4

(6.1.47)

W e n o te that X ( p , q ) = X ( 4 p + q ), q = 0, 1, 2, 3. C on sequ en tly, the Af-point

D F T is d ecim a ted into four N /4 -p o in t D F T s and h en ce w e have a decim ationin -frequency F F T algorithm . T h e com putations in (6.1.46) and (6.1.47) define

468

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

Figure 6.14 Sixteen-point, radix-4 decimation-in-frequency algorithm with input

in normal order and output in digit-reversed order.
the basic radix-4 butterfly for the d ecim ation-in-frequency algorithm . N o te that
the m ultiplications by the factors W% occur after the com b ination o f the data
p oin ts x (/, m), just as in the case o f th e radix-2 decim ation -in-freq u en cy algo
rithm.
A 16-point radix-4 decim ation -in-freq u en cy F F T algorithm is show n in
Fig. 6.14. Its input is in norm al order and its output is in digit-reversed order.
It has exactly the sam e com putational com p lexity as the d ecim ation -in-tim e radix4 F F T algorithm .
For illustrative purposes, let us rederive the radix-4 decim ation-in-frequency
algorithm by breaking the jV-point D F T form ula in to four sm aller D F T s. We
have
N- 1

X( k) = T x ( n ) W N
kn
n=0
=

JV/4-1
N/Z-1
3N/4-1
fif-l

x { n ) W kNn +
x{n)W%' +
* (* )< " +
x(n)W N
kn
n=0
n=N/4
n=Nfi
n=JN/4

Sec. 6.1

469

Efficient Computation of the DFT: FFT Algorithms

/A 74/ * - 1J

h= (I

=o

A'/4 1

E*("

A// 4t -- iI
A

,< ,< + <

< V/2 E

A/ N

'

\i \

N / A \

* ( + y ) <

+ < A'/4 E

'l \ ! \

A (" + t )
(6.1.48)

F ro m th e d efin itio n o f th e tw id d le facto rs, w e h av e

lNk/4
w\N

(6.1.49)

(jf

A fte r su b s titu tio n o f (6.1.49) in to (6.1.48). we o b ta in

N/ 4-1

xa)=

x() + (

(6.1.50)
N
+ { -1 f x [ n + - J + ( ;)

W\

T h e re la tio n in (6.1.50) is n o t an N /4 -p o in t D F T b e c a u se th e tw id d le facto r

d e p e n d s o n N a n d n o t on N / 4 . T o c o n v ert it in to an A '/4-point D F T , wc su b d iv id e
th e D F T se q u e n c e in to four /V /4-point su b se q u e n c e s, X ( 4 k ) . X ( 4 k + !), X (4 + 2),
an d X { 4 k + 3), k 0, 1........ N / 4 1. T h u s we o b ta in th e rad ix -4 d ecim atio n -in fre q u e n c y D F T as
x ( n ) + .v

(6.1.51)

+? )

+a(', + i ) +j:(" + t
X (4k + l ) =

*( ) ~

ix

( +

0 urkn
w"w
N r r N /4

(6.1.52)

"j}
IV " w kn

N w yv/4

X ( 4 k + 2) =

T ; 1r
(
n \
E
* ( ) * ( + j J

- K
r
X ( 4 k + 3) =

(6.1.53)

' +t )

/
x (n '>+ J x ( " +

W 2" w
kn
w Nf4

\
J

~ x ( n ~h j ) - Jx (b+ t

(6.1.54)

) ] w^

kn
S/4

470

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

w h ere w e h av e u se d th e p ro p e rty W^ kn = W^"4. N o te th a t th e in p u t to ea c h N/ 4p o in t D F T is a lin e a r co m b in a tio n o f fo u r signal sa m p le s sc aled by a tw id d le factor.

T h is p ro c e d u re is r e p e a te d v tim es, w h ere v = log,, N.

6.1.5 Split-Radix FFT Algorithms

A n in sp e ctio n o f th e radix-2 d e c im a tio n -in -fre q u e n c y flo w g rap h show n in Fig. 6.11
in d icates th a t th e e v e n -n u m b e re d p o in ts o f th e D F T can b e c o m p u te d in d e p e n
d en tly o f th e o d d -n u m b e re d p o in ts. T h is suggests th e p o ssib ility o f using d ifferen t
c o m p u ta tio n a l m e th o d s fo r in d e p e n d e n t p a rts o f th e a lg o rith m w ith th e ob jectiv e
o f re d u c in g th e n u m b e r o f c o m p u ta tio n s. T h e sp lit-ra d ix F F T (S R F F T ) alg o rith m s
ex p lo it th is id ea by u sing b o th a radix-2 a n d a radix-4 d e c o m p o s itio n in th e sam e
F I T alg o rith m .
W e illu stra te th is a p p ro a c h w ith a d e c im a tio n -in -fre q u e n c y S R F F T alg o rith m
d u e to D u h a m e l (1986). F irst, w e recall th a t in th e rad ix -2 d ec im a tio n -in -fre q u e n c y
F F T alg o rith m , th e e v e n -n u m b e re d sa m p le s o f th e /V -point D F T a re given as

N o te th a t th ese D F T p o in ts can b e o b ta in e d fro m an N /2 -p o in t D F T w ith o u t any

a d d itio n a l m u ltip lic atio n s. C o n se q u e n tly , a radix-2 suffices fo r th is c o m p u ta tio n .
T h e o d d -n u m b e re d sa m p le s {X{2k + 1)) o f th e D F T re q u ire th e p re m u ltip li
catio n o f th e in p u t se q u e n c e w ith th e tw id d le fa c to rs W N
n . F o r th e s e sa m p les a
rad ix -4 d eco m p o sitio n p ro d u c e s so m e c o m p u ta tio n a l efficiency b e c a u se th e fourp o in t D F T h as th e larg est m u ltip lic a tio n -fre e b u tterfly . I n d e e d , it can b e show n
th a t usin g a rad ix g r e a te r th a n 4, d o e s n o t re su lt in a significant re d u c tio n in com
p u ta tio n a l co m p lex ity .
I f we u se a rad ix -4 d e c im a tio n -in -fre q u e n c y F F T a lg o rith m fo r th e oddn u m b e re d sa m p le s o f th e /V -point D F T , w e o b ta in th e fo llo w in g N /4 -p o in t D FTs:
N/4-1

(6.1.56)
- j [ x ( n + N / 4 ) - x( n + 3 N / 4 )]}
A74-1

(6.1.57)
+ j [ x ( n + N / 4 ) - x{ n + 3 N / 4 ) ] } W ^
T h u s th e N -p o in t D F T is d e c o m p o se d in to o n e N /2 -p o in t D F T w ith o u t ad d itio n al
tw id d le facto rs a n d tw o N /4 -p o in t D F T s w ith tw id d le facto rs. T h e /V-p o in t D F T
is o b ta in e d by su ccessiv e u se o f th e s e d e c o m p o s itio n s u p to th e la st stag e. T hus
w e o b ta in a d e c im a tio n -in -fre q u e n c y S R F F T alg o rith m .
F ig u re 6.15 sh o w s th e flow g ra p h fo r a n in -p la ce 3 2 -p o in t d ecim atio n in -freq u en cy S R F F T a lg o rith m . A t stag e A of th e c o m p u ta tio n fo r N = 32, the

Sec. 6.1

471

Efficient Computation of the DFT: FFT Algorithms

Figure 6,15 L ength 32 split-radix F F T algorithm s from p ap er by D uham el (1986); rep rin ted
w ith perm ission from the IE E E .

to p 16 p o in ts c o n s titu te th e se q u e n c e
go() = x ( n ) + x ( n + N / 2)

0 < n < 15

(6.1.58)

T h is is th e s e q u e n c e re q u ire d fo r th e c o m p u ta tio n o f X ( 2 k ) . T h e n e x t 8 p o in ts
c o n s titu te th e se q u e n c e

gi(n) = x(n) - x(n + N/ 2)

0<n<7

(6.1.59)

472

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

T h e b o tto m e ig h t p o in ts c o n s titu te th e se q u e n c e j g 2 (71). w h ere

82 (n) = x ( n + N / 4 ) x ( n + 3 N / 4 )

0 < < 7

(6.1.60)

T h e se q u e n c e s gj ( n) a n d gi ( n) a re u sed in th e c o m p u ta tio n o f X ( 4 k 4 - 1) and

A'( 4 * + 3). T h u s, a t stag e A w e h ave c o m p le te d th e first d e c im a tio n fo r th e radix-2
c o m p o n e n t o f th e alg o rith m . A t sta g e B , th e b o tto m eig h t p o in ts c o n stitu te the
c o m p u ta tio n o f [# i(n ) + 7 ( )]^32 ->0 < /i < 7, w hich is u sed to c o m p u te X (4A -f 3),
0 < k < 7. T h e n ex t eig h t p o in ts fro m th e b o tto m c o n s titu te th e c o m p u ta tio n of
[tfi(n) j g 2(h)] VVjj, 0 < n < 7, w hich is u se d to c o m p u te X ( 4 k 4 -1 ), 0 < k < 7.
T h u s a t stag e B , w e h av e c o m p le te d th e first d e c im a tio n fo r th e ra d ix -4 alg o rith m ,
w hich resu lts in tw o 8 -p o in t se q u e n c e s. H e n c e th e b asic b u tte rfly c o m p u ta tio n fo r
th e S R F F T a lg o rith m h a s th e L -s h a p e d form illu stra te d in Fig. 6.16.
N ow w e r e p e a t th e ste p s in th e c o m p u ta tio n a b o v e. B e g in n in g w ith th e to p
16 p o in ts a t stag e A , w e r e p e a t th e d e c o m p o s itio n f o r th e 1 6 -p o in t D F T . In o th e r
w o rd s, w e d e c o m p o se th e c o m p u ta tio n in to an e ig h t-p o in t, rad ix -2 D F T an d tw o
fo u r-p o in t, rad ix -4 D F T s. T h u s a t sta g e B , th e to p eig h t p o in ts c o n s titu te the
se q u e n c e (w ith N = 16)

g'o(*) = 8o(n) + go(n 4- N / 2)

<n<7

(6.1.61)

an d th e n ex t eig h t p o in ts c o n s titu te th e tw o fo u r-p o in t se q u e n c e s g[(n) a n d jg'2(n),

w h ere
g[ (n) = go(n) ~ go(n + N f l )
0 < n < 3
(6.1.62)
82 () = 8o(n + N / 4 ) - g0(n + 3 N / 4 )
0 < n < 3
T h e b o tto m 16 p o in ts o f sta g e B a re in th e fo rm o f tw o e ig h t-p o in t D F T s. H en ce
ea c h e ig h t-p o in t D F T is d e c o m p o s e d in to a fo u r-p o in t, rad ix -2 D F T a n d a fourp o in t, rad ix -4 D F T . In th e final stag e, th e c o m p u ta tio n s in v o lv e th e co m b in atio n
o f tw o -p o in t se q u en ces.
T a b le 6.2 p r e s e n ts a c o m p a riso n o f th e n u m b e r o f nont ri vi al re a l m u ltip li
ca tio n s a n d a d d itio n s re q u ire d to p e rfo rm a n jY -point D F T w ith co m p lex -v alu ed

Sec. 6 .1

473

Efficient Computation of the DFT: FFT Algorithms

TABLE 6.2 NUMBER OF NONTRIVIAL REAL MULTIPLICATIONS AND
ADDITIONS TO COMPUTE AN N-POINT COMPLEX DFT
Real M ultiplications
Radix

R adix
4

24
88
264
712
1,800
4.360
10.248

20

16
32
64
128
256
512
1,024

208

Radix
8

204

1.392
3.204
7,856

Real A dditions
Split
Radix

Radix
2

20
68
196
516
1.284
3.076
7,172

152
408
1.032
2.504
5,896
13.566
30.728

Radix
4

R adix
8

148
976

972

5,488
12,420
28.336

Split
Radix
148
388
964
2308
5.380
12.292
27,652

Source: E xtracted from D uham el (1986).

d a ta , using a rad ix -2 , ra d ix -4, radix-8, a n d a sp lit-ra d ix F F T . N o te th a t th e S R F F T

alg o rith m re q u ire s th e lo w est n u m b e r o f m u ltip lic a tio n a n d a d d itio n s. F o r this
re a so n , it is p re fe ra b le in m an y p ra c tic a l a p p licatio n s.
A n o th e r ty p e o f S R F F T a lg o rith m has b e e n d e v e lo p e d by P rice (1990). Its
re la tio n to D u h a m e ls a lg o rith m d e sc rib e d p rev io u sly can b e seen by n o tin g th a t
th e rad ix -4 D F T te rm s X ( 4 k 4- 1) an d X ( 4 k + 3) involve th e N /4 -p o in t D F T s o f th e
se q u e n c e s [g i(n ) a n d [ # i( '0 + ,/ 2( )]W $ \ resp ec tiv ely . In effect, th e
se q u e n c e s g i(/i) a n d g 2 (n) are m u ltip lie d by th e fa c to r (v e c to r) (1, j ) = (1, H ^ )
an d by WJJ fo r th e c o m p u ta tio n o f X ( 4 k + 1), w hile th e c o m p u ta tio n o f X (4k + 3)
in v o lv es th e fa c to r (1 , j ) = (1, W{2*) an d W j f .

In s te a d , o n e can r e a rra n g e th e

c o m p u ta tio n so th a t th e fa c to r fo r X ( 4 k + 3) is ( j , 1) = (W 8, 1). A s a resu lt

o f th is p h a s e ro ta tio n , th e tw id d le fa c to rs in th e c o m p u ta tio n o f X (4k -f 3) b eco m e
ex actly th e sam e as th o se fo r X ( 4 k + 1), e x cep t th a t th e y o c c u r in m irr o r im age
o rd e r. F o r e x am p le, at sta g e B o f Fig. 6.15, th e tw id d le fa c to rs W21, W 18, . . . , W3
a re re p la c e d by (V1, W 2, . . . , W 1, resp ec tiv ely . T h is m irro r-im a g e sy m m etry occurs
a t ev ery s u b s e q u e n t sta g e o f th e alg o rith m . A s a c o n s e q u e n c e , th e n u m b e r o f
tw id d le facto rs th a t m u st b e c o m p u te d a n d s to re d is r e d u c e d by a fa c to r o f 2 in
c o m p a riso n to D u h a m e ls a lg o rith m . T h e re su ltin g a lg o rith m is called th e m irr o r
F F T (M F F T ) alg o rith m .
A n a d d itio n a l facto r-o f-2 savings in sto ra g e o f tw id d le fa c to rs can be o b ta in e d
b y in tro d u c in g a 90 p h a s e o ffset a t th e m id p o in t o f ea c h tw id d le a rra y , w hich can
b e re m o v e d if n ecessa ry a t th e o u tp u t o f th e S R F F T c o m p u ta tio n . T h e in co r
p o r a tio n o f th is im p ro v e m e n t in to th e S R F F T ( o r th e M F F T ) re su lts in a n o th e r
a lg o rith m , also d u e to P ric e (1990), c a lle d th e p h a s e F F T (P F F T ) alg o rith m .

6.1.6 Implementation of FFT Algorithms

N o w th a t w e h av e d e s c rib e d th e b asic rad ix -2 a n d rad ix -4 F F T a lg o rith m s, let
us c o n s id e r so m e o f th e im p le m e n ta tio n issues. O u r r e m a rk s ap p ly d ire c tly to

474

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

rad ix -2 a lg o rith m s, a lth o u g h sim ilar c o m m e n ts m ay b e m a d e a b o u t rad ix -4 a n d

h ig h er-rad ix alg o rith m s.
B asically, th e rad ix -2 F F T a lg o rith m co n sists o f ta k in g tw o d a ta p o in ts a t a
tim e fro m m e m o ry , p e rfo rm in g th e b u tte rfly c o m p u ta tio n s a n d r e tu rn in g th e r e
su ltin g n u m b e rs to m em o ry . T h is p ro c e d u re is r e p e a te d m an y tim e s ( ( N log2 N) [ 2
tim es) in th e c o m p u ta tio n o f an JV-point D F T .
T h e b u tterfly c o m p u ta tio n s re q u ire th e tw id d le fa c to rs {W^} a t v a rio u s stages
in e ith e r n a tu ra l o r b it-re v e rse d o rd e r. In an efficien t im p le m e n ta tio n o f th e algo
rith m , th e p h ase fa c to rs a re c o m p u te d o n ce a n d s to re d in a ta b le , e ith e r in n o rm a l
o r d e r o r in b it-re v e rse d o rd e r, d e p e n d in g o n th e specific im p le m e n ta tio n o f th e
alg o rith m .
M e m o ry re q u ire m e n t is a n o th e r fa c to r th a t m u st b e c o n s id e re d . If th e c o m
p u ta tio n s a re p e rfo rm e d in p lace, th e n u m b e r o f m e m o ry lo c a tio n s re q u ire d is 2 N
since th e n u m b e rs a re com plex. H o w e v e r, w e can in ste a d d o u b le th e m e m o ry to
4N , th u s sim p lifying th e in d ex in g a n d c o n tro l o p e r a tio n s in th e F F T a lg o rith m s. In
th is case w e sim p ly a lte rn a te in th e use o f th e tw o se ts o f m e m o ry lo c a tio n s from
o n e sta g e o f th e F F T a lg o rith m to th e o th e r. D o u b lin g o f th e m e m o ry also allow s
us to h av e b o th th e in p u t se q u e n c e a n d th e o u tp u t se q u e n c e in n o rm a l o rd e r.
T h e re are a n u m b e r o f o th e r im p le m e n ta tio n issues re g a rd in g ind ex in g , bit
rev ersal, an d th e d e g re e o f p arallelism in th e c o m p u ta tio n s. T o a larg e ex ten t,
th e se issues a re a fu n ctio n o f th e specific a lg o rith m a n d th e ty p e o f im p le m e n ta
tio n , n am ely , a h a rd w a re o r so ftw are im p le m e n ta tio n . In im p le m e n ta tio n s b ased
o n a fix ed -p o in t a rith m e tic , o r flo atin g -p o in t a rith m e tic o n sm a ll m a ch in es, th e re
is also th e issue o f ro u n d -o ff e rro rs in th e c o m p u ta tio n . T h is to p ic is co n sid e re d
in S ectio n 6.4.
A lth o u g h th e F F T a lg o rith m s d e sc rib e d p re v io u sly w e re p re s e n te d in th e
c o n te x t o f co m p u tin g th e D F T efficiently, th e y can also b e u s e d to c o m p u te th e
ID F T , w hich is
j

A '- l

* (") = 7 T ] C * (
*=0

(6' ll63)

T h e o n ly d iffe re n c e b e tw e e n th e tw o tra n sfo rm s is th e n o rm a liz a tio n fa c to r l / N

a n d th e sign o f th e p h a se fa c to r WN. C o n s e q u e n tly , a n F F T a lg o rith m fo r co m
p u tin g th e D F T , c a n b e c o n v e rte d to an F F T a lg o rith m fo r c o m p u tin g th e ID F T
by ch an g in g th e sign o n all th e p h a se fa c to rs a n d d iv id in g th e final o u tp u t o f th e
alg o rith m by N.
In fact, if w e ta k e th e d e c im a tio n -in -tim e a lg o rith m t h a t w e d e sc rib e d in
S ectio n 6.1.3, re v e rse th e d ire c tio n o f th e flow g ra p h , c h a n g e th e sign o n th e p h ase
facto rs, in te rc h a n g e th e o u tp u t a n d in p u t, a n d finally, d iv id e th e o u tp u t by N , w e
o b ta in a d e c im a tio n -in -fre q u e n c y F F T a lg o rith m fo r c o m p u tin g th e ID F T . O n th e
o th e r h a n d , if w e b eg in w ith th e d e c im a tio n -in -fre q u e n c y F F T a lg o rith m d escrib ed
in S ectio n 6.1.3 a n d r e p e a t th e ch an g es d e s c rib e d a b o v e , w e d b ta in a d ecim atio n in -tim e F F T a lg o rith m fo r c o m p u tin g th e ID F T . T h u s it is a sim p le m a tte r to devise
F F T a lg o rith m s fo r co m p u tin g th e ID F T .

Sec. 6.2

Applications of FFT Algorithms

475

F in ally , w e n o te th a t th e em p h asis in o u r discussion o f F F T a lg o rith m s w as

o n rad ix -2 , rad ix -4 , a n d sp lit-ra d ix a lg o rith m s. T h e se are by fa r th e m o st w idely
u se d in p ra c tic e . W h e n th e n u m b e r o f d a ta p o in ts is n o t a p o w e r o f 2 o r 4. it is a
sim p le m a tte r to p a d th e se q u e n c e x ( n ) w ith zero s such th a t /V = 2 1 o r N = 4 '.
T h e m e a s u re o f co m p lex ity fo r F F T alg o rith m s th a t w e h av e e m p h a siz e d
is th e r e q u ire d n u m b e r o f a rith m e tic o p e ra tio n s (m u ltip lic a tio n s an d a d d itio n s).
A lth o u g h this is a v ery im p o rta n t b e n c h m a rk fo r c o m p u ta tio n a l co m p lex ity , th e re
a re o th e r issues to b e c o n s id e re d in p ractical im p le m e n ta tio n o f F F T a lg o rith m s.
T h e se in clu d e th e a rc h ite c tu re o f th e p ro cesso r, th e av ailab le in stru c tio n set, th e
d a ta s tru c tu re s fo r s to rin g tw id d le facto rs, an d o th e r c o n sid e ra tio n s.
F o r g e n e ra l-p u rp o s e c o m p u te rs, w h e re th e cost o f th e n u m e ric a l o p e ra tio n s
d o m in a te , rad ix -2 , rad ix-4, an d sp lit-ra d ix F F T a lg o rith m s a re go o d c a n d id a te s.
H o w e v e r, in th e case o f sp e c ia l-p u rp o se digital signal p ro c e ss o rs , fe a tu rin g sin g le
cycle m u ltip ly -a n d -a c c u m u la te o p e ra tio n , b it-re v e rse d ad d re ssin g , a n d a high d e
g ree o f in stru c tio n p a ra lle lism , th e stru c tu ra l re g u la rity o f th e a lg o rith m is eq u ally
im p o rta n t as a rith m e tic co m p lex ity . H e n c e fo r D S P p ro c e sso rs, rad ix -2 o r radix4 d e c im a tio n -in -fre q u e n c y F F T a lg o rith m s are p re fe ra b le in te rm s o f sp e e d an d
a ccu racy . T h e irre g u la r stru c tu re o f th e S R F F T m ay r e n d e r it less su ita b le fo r
im p le m e n ta tio n o n d ig ital signal pro cesso rs. S tru c tu ra l re g u la rity is also im p o rta n t
in th e im p le m e n ta tio n o f F F T a lg o rith m s on v e c to r p ro c e sso rs, m u ltip ro c e sso rs,
a n d in V L SI. I n te rp ro c e s s o r co m m u n icatio n is an im p o rta n t c o n s id e ra tio n in such
im p le m e n ta tio n s o n p a rallel pro cesso rs.
In co n clu sio n , we h av e p re s e n te d several im p o rta n t c o n s id e ra tio n s in th e
im p le m e n ta tio n o f F F T a lg o rith m s. A d v an ce s in digital signal p ro cessin g te c h n o l
ogy, in h a rd w a re a n d so ftw are, will c o n tin u e to influence th e ch o ice a m o n g F F T
a lg o rith m s fo r v a rio u s p ractical ap p licatio n s.

6.2 APPLICATIONS OF FFT ALGORITHMS

T h e F F T a lg o rith m s d e sc rib e d in th e p re c e d in g se ctio n find a p p lic a tio n in a v ariety
o f a re a s , in clu d in g lin e a r filtering, c o rre la tio n , a n d s p e c tru m analysis. B asically,
th e F F T a lg o rith m is u sed as a n efficient m e a n s to c o m p u te th e D F T a n d th e ID F T .
In th is se c tio n w e c o n s id e r th e u se o f th e F F T a lg o rith m in lin e a r filterin g
a n d in th e c o m p u ta tio n o f th e c ro ssco rrelatio n o f tw o se q u e n c e s. T h e use o f th e
F F T in s p e c tru m an aly sis is c o n sid e re d in C h a p te r 12. In a d d itio n w e illu strate
h o w to e n h a n c e th e efficiency o f th e F F T a lg o rith m by fo rm in g co m p le x -v a lu e d
se q u e n c e s fro m re a l-v a lu e d se q u e n c e s p rio r to th e c o m p u ta tio n o f th e D F T .

6.2.1 Efficient Computation of the DFT of Two Real

Sequences
T h e F F T a lg o rith m is d esig n e d to p e rfo rm co m p lex m u ltip lic a tio n s a n d a d d itio n s,
ev en th o u g h th e in p u t d a ta m ay b e re a l v alued. T h e b asic re a s o n fo r th is s itu a tio n is

476

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

th a t th e p h a se fa c to rs a re co m p lex a n d h e n c e , a fte r th e first sta g e o f th e alg o rith m ,

all v a riab les a re b asically co m p lex -v alu ed .
In view o f th e fact th a t th e a lg o rith m can h a n d le c o m p le x -v a lu e d in p u t se
q u en ces, w e c a n ex p lo it th is cap ab ility in th e c o m p u ta tio n o f th e D F T o f tw o
re a l-v a lu e d se q u e n c e s.
S u p p o se th a t * i(n ) a n d x 2(n) are tw o re a l-v a lu e d se q u e n c e s o f len g th N , and
let x( n ) b e a co m p le x -v a lu e d se q u e n c e d efin ed as
x ( n ) = X] (n) + j x 2 (n)

0 < n < N 1

(6.2.1)

T h e D F T o p e ra tio n is lin e a r a n d h en ce th e D F T o f x ( n ) can be ex p re sse d as

X ( k ) = X ](k) + j X 2(k)

(6.2.2)

T h e se q u e n c e s ^ i( ) an d J 2 OO can be ex p re sse d in te rm s o f x ( n ) as follow s:

, 4
ac(H) + J:*(n)
* i( ) = -------- 2--------

(6.2.3)

x(n)-x*(n)
*:() = ------- tt.--------

(6.2.4)

H e n c e th e D F T s o f jri(n ) an d x 2(n) are

* ,( * ) =

-] \ D F T [ x { n ) } + D F T [ x \ n ) } )

X 2(k) = j - \ D F T [ x ( n )] - DF T [ x * ( n ) ] )

(6.2.5)
(6.2.6)

R ecall th a t th e D F T o f x*( n) is X * ( N k). T h e re fo re ,

X] (k) = i[X (* r) + X * ( N - jt)]

(6.2.7)

X 2(k) = -^ [X (* > - X * ( N - *)]

;2

(6.2.8)

T h u s, by p e rfo rm in g a single D F T o n th e co m p le x -v a lu e d se q u e n c e x ( n ), we
h av e o b ta in e d th e D F T o f th e tw o re a l se q u e n c e s w ith only a sm all a m o u n t of
a d d itio n a l c o m p u ta tio n th a t is involved in co m p u tin g Xi (Jt) a n d X 2 (k) fro m X(k)
by u se o f (6.2.7) a n d (6.2.8).

6.2.2 Efficient Computation of the DFT of a 2/V-Point

Real Sequence
S u p p o se th a t g( n) is a re a l-v a lu e d se q u e n c e o f 2 N p o in ts. W e n o w d e m o n s tra te
h o w to o b ta in th e 2 N -p o in t D F T o f g( n) fro m c o m p u ta tio n o f o n e A ppoint D F T
involving c o m p le x -v a lu e d d a ta . F irst, w e define
* i(n )

g(2 n)

(6.2.9)
*2(n) = g ( 2 n + 1)

Sec. 6.2

477

Applications of FFT Algorithms

T h u s w e h a v e su b d iv id e d th e 2 N -p o in t re a l se q u e n c e in to tw o W -point real se
q u e n c e s. N o w w e can ap p ly th e m e th o d d escrib ed in th e p re c e d in g sectio n .
L et jc(n) b e th e A7-p o in t c o m p lex -v alu ed se q u e n c e
A-(n) = * i ( n ) + j x i i n )

(6 .2 .10)

F ro m th e re su lts o f th e p re c e d in g se ctio n , w e h av e
x m

= ^ [* (* ) + * * ( * - * ) ]
j

(6.2.11)

X 2(k) = [ X( k) - X * ( N - k)]
F inally, w e m u st ex p re ss th e 2/V -point D F T in te rm s o f th e tw o /V -point D F T s,
Xi(A) a n d X 2(k). T o acco m p lish this, w e p ro c e e d as in th e d e c im a tio n -in -tim e F F T
a lg o rith m , n am ely ,
N -1

N-1

C( k ) = s < 2 h ) H $ * + J 2 s ( 2 n + ^ W7 N ^ k
n=tl

n=0

N- l

N-1

=()

n=()

C o n s e q u e n tly ,
G( k ) = X t (k) + W i N X 2(k)

k = 0 . 1 ..........N - 1
( 6 . 2 . 12 )

G( k + N ) = X i ( k ) - W%N X 2(k)

k = Q . \ ..........N - l

T h u s w e h av e c o m p u te d th e D F T o f a 2/V -point real se q u e n c e from o n e jV-point

D F T an d so m e a d d itio n a l c o m p u ta tio n as in d icated by (6.2.11) an d (6.2.12).

6.2.3 Use of the FFT Algorithm in Linear Filtering and

Correlation
A n im p o rta n t ap p lic a tio n o f th e F F T a lg o rith m is in F IR lin e a r filterin g o f lo n g
d a ta se q u e n c e s. In C h a p te r 5 w e d e sc rib e d tw o m e th o d s, th e o v e rla p -a d d an d th e
o v e rla p -sa v e m e th o d s fo r filterin g a lo n g d a ta se q u e n c e w ith an F I R filter, b a s e d
o n th e u se o f th e D F T . In th is se ctio n w e c o n sid e r th e u se o f th e s e tw o m e th o d s
in c o n ju n c tio n w ith th e F F T a lg o rith m fo r co m p u tin g th e D F T an d th e ID F T .
L e t h( n), 0 < n < M - 1 , b e th e u n it sa m p le re sp o n s e o f th e F IR filter an d let
x ( n ) d e n o te th e in p u t d a ta se q u e n c e . T h e block size o f th e F F T alg o rith m is N ,
w h e re N = L + M 1 an d L is th e n u m b e r o f new d a ta sa m p le s b e in g p ro cessed
by th e filter. W e a ssu m e th a t fo r a n y given v alu e o f Af, th e n u m b e r L o f d a ta
sa m p le s is se le c te d so th a t N is a p o w e r o f 2. F o r p u rp o se s o f th is discussion, w e
c o n s id e r o n ly rad ix -2 F F T alg o rith m s.
T h e /V -point D F T o f h(n), w hich is p a d d e d b y L 1 z e ro s, is d e n o te d as H( k ) .
T h is c o m p u ta tio n is p e rfo rm e d o n c e via th e F F T an d th e re su ltin g N co m p lex
n u m b e rs a r e sto re d . T o be specific w e a ssu m e th a t th e d e c im a tio n -in -fre q u e n c y

478

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

F F T a lg o rith m is u se d to c o m p u te H( k ) . T h is y ields H ( k ) in b it-re v e rse d o rd e r,

w hich is th e w ay it is s to re d in m em ory.
In th e o v e rlap -sav e m e th o d , th e first M 1 d a ta p o in ts o f e a c h d a ta b lo ck are
th e last M 1 d a ta p o in ts o f th e p rev io u s d a ta b lo ck . E a c h d a ta b lo c k c o n ta in s L
new d a ta p o in ts, su ch th a t N = L + M 1. T h e N -p o in t D F T o f ea c h d a ta block
is p e rfo rm e d by th e F F T alg o rith m . If th e d e c im a tio n -in -fre q u e n c y alg o rith m is
e m p lo y ed , th e in p u t d a ta b lo ck re q u ire s n o shuffling a n d th e v a lu e s o f th e D F T
o ccu r in b it-re v e rse d o rd e r. S ince th is is ex actly th e o r d e r o f H ( k ) . w e can m ultiply
th e D F T o f th e d a ta , say Xm(fc), w ith //(Jt) a n d th u s th e re su lt
Ym(k) = H ( k ) X m(k)
is also in b it-re v e rse d o rd e r.
T h e in v erse D F T (ID F T ) can b e c o m p u te d by use o f an F F T alg o rith m th a t
ta k e s th e in p u t in b it-re v e rse d o r d e r a n d p ro d u c e s an o u tp u t in n o rm al o rd er.
T h u s th e r e is n o n e e d to shuffle any b lo ck o f d a ta e ith e r in c o m p u tin g th e D F T
o r th e ID F T .
If th e o v e rla p -a d d m e th o d is used to p e rfo rm th e lin e a r filterin g , th e co m p u
ta tio n a l m e th o d u sin g th e F F T a lg o rith m is basically th e sa m e. T h e only differen ce
is th a t th e N -p o in t d a ta b lo ck s consist o f L new d a ta p o in ts a n d M 1 a d d itio n a l
zero s. A fte r th e I D F T is c o m p u te d fo r ea c h d a ta b lo ck , th e W -point filtered blocks
a re o v e rla p p e d as in d ic a te d in S ectio n 5.3.2, a n d th e M - 1 o v e rla p p in g d a ta p o in ts
b e tw e e n successive o u tp u t re c o rd s a re a d d e d to g e th e r.
L et u s assess th e c o m p u ta tio n a l co m p lex ity o f th e F F T m e th o d fo r lin e a r fil
terin g . F o r th is p u rp o se , th e o n e -tim e c o m p u ta tio n o f H ( k ) is in sig n ifican t an d can
b e ig n o red . E ach F F T re q u ire s ( N / 2) log2 N co m p lex m u ltip lic a tio n s an d N Iog2 N
a d d itio n s. Since th e F F T is p e rfo rm e d tw ice, o n ce fo r th e D F T a n d o n ce fo r th e
ID F T , th e c o m p u ta tio n a l b u rd e n is N log2 N co m p lex m u ltip lic a tio n s an d 2 N log2 N
a d d itio n s. T h e re a re also N co m p lex m u ltip lic a tio n s a n d N 1 a d d itio n s re q u ire d
to c o m p u te ym(Jfc). T h e re fo re , w e h av e ( N \ o g 2 2 N ) / L co m p lex m u ltip lic a tio n s p er
o u tp u t d a ta p o in t a n d a p p ro x im a te ly ( 2 N \ o g 2 2 N ) / L a d d itio n s p e r o u tp u t d ata
p o in t. T h e o v e rla p -a d d m e th o d re q u ire s an in c re m e n ta l in c re a se o f ( M \ ) / L in
th e n u m b e r o f ad d itio n s.
B y w ay o f c o m p a riso n , a d ire c t fo rm re a liz a tio n o f th e F I R filter involves M
real m u ltip lic atio n s p e r o u tp u t p o in t if th e filter is n o t lin e a r p h a s e , a n d M / 2 if it
is lin e a r p h ase (sy m m etric ). A lso , th e n u m b e r o f a d d itio n s is M - 1 p e r o u tp u t
p o in t (see Sec. 8.2).
I t is in te re stin g to co m p a re th e efficiency o f th e F F T a lg o rith m w ith th e direct
fo rm re a liz a tio n o f th e F IR filter. L e t us focus o n th e n u m b e r o f m ultip lic atio n s,
w h ich a re m o re tim e co n su m in g th a n a d d itio n s. S u p p o se th a t M = 128 = 27 an d
N = 2 V. T h e n th e n u m b e r o f co m p lex m u ltip lic a tio n s p e r o u tp u t p o in t fo r an F F T
size o f N = 2 V is

Sec. 6.3

A Linear Filtering Approach to Computation of the DFT

TABLE 6.3

479

COMPUTATIONAL COMPLEXITY

Size of FFT
i) log2 N

f(v)
Number of Complex Multiplications
per Output Point

9
10
11
12
14

13.3
12.6
12.8
13.4
15.1

T h e v alu es o f c( v) fo r d iffe re n t v alu es o f i> are given in T a b le 6.3. W e o b se rv e

th a t th e re is an o p tim u m v a lu e o f i< w h ich m in im iz es c(u ). F o r th e F IR filter of
size M = 128, th e o p tim u m o ccu rs at d = 10.
W e sh o u ld e m p h asize th a t c ( f ) r e p re s e n ts th e n u m b e r o f co m p lex m u ltip lic a
tio n s fo r th e F F T -b a se d m e th o d . T h e n u m b e r o f re a l m u ltip lic a tio n s is fo u r tim es
th is n u m b e r. H o w e v e r, ev en if th e F IR filter has lin e a r p h a s e (see Sec. 8.2), th e
n u m b e r o f c o m p u ta tio n s p e r o u tp u t p o in t is still less w ith th e F F T -b a se d m eth o d .
F u rth e rm o r e , th e efficiency o f th e F F T m e th o d can be im p ro v e d by c o m p u tin g
th e D F T o f tw o successive d a ta b lo ck s sim u lta n e o u sly , ac c o rd in g to th e m eth o d
ju st d e sc rib e d . C o n s e q u e n tly , th e F F T -b a se d m e th o d is in d e e d su p e rio r from a
c o m p u ta tio n a l p o in t o f view w h en th e filter len g th is re lativ ely large.
T h e c o m p u ta tio n o f th e cross c o rre la tio n b e tw e e n tw o se q u e n c e s by m e a n s o f
th e F F T a lg o rith m is sim ilar to th e lin e a r F IR filtering p ro b le m ju st d esc rib e d . In
p ractical a p p lic a tio n s involving c ro ssc o rre la tio n , a t least o n e o f th e se q u e n c e s has
finite d u ra tio n an d is a k in to th e im p u lse re sp o n s e o f th e F IR filter. T h e seco n d
s e q u e n c e m ay be a lo n g se q u e n c e w hich c o n ta in s th e d e s ire d se q u e n c e c o rru p te d
b y a d d itiv e n o ise. H e n c e th e se co n d se q u e n c e is a k in to th e in p u t to th e F I R filter.
B y tim e rev e rsin g th e first se q u e n c e a n d co m p u tin g its D F T , w e h av e r e d u c e d th e
cro ss c o rre la tio n to an e q u iv a le n t co n v o lu tio n p ro b le m (i.e.. a lin e a r F I R filtering
p ro b le m ). T h e re fo re , th e m e th o d o lo g y w e d e v e lo p e d fo r lin e a r F IR filterin g by
u se o f th e F F T a p p lie s directly.

6.3 A LINEAR FILTERING APPROACH TO COMPUTATION OF THE

DFT
T h e F F T alg o rith m ta k e s N p o in ts o f in p u t d a ta a n d p ro d u c e s an o u tp u t se q u e n c e
o f N p o in ts c o rre sp o n d in g to th e D F T o f th e in p u t d a ta . A s w e h a v e show n,
th e rad ix -2 F F T a lg o rith m p e rfo rm s th e c o m p u ta tio n of th e D F T in ( N f l ) log2 N
m u ltip lic a tio n s a n d N log2 N a d d itio n s fo r a n N -p o in t se q u e n c e .
T h e re a re so m e a p p lic a tio n s w h e re o n ly a se le c te d n u m b e r o f valu es o f
th e D F T a re d e s ire d , b u t th e e n tire D F T is n o t re q u ire d . In such a case, th e
F F T a lg o rith m m ay n o lo n g e r be m o r e efficien t th a n a d ire c t c o m p u ta tio n o f
th e d e s ire d v alu es o f th e D F T . In fact, w h e n th e d e s ire d n u m b e r o f valu es o f

480

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

th e D F T is less th a n log2 N , a d ire c t c o m p u ta tio n o f th e d e s ire d v alu es is m o re

efficient.
T h e d irect c o m p u ta tio n o f th e D F T can b e fo rm u la te d as a lin e a r filtering
o p e ra tio n o n th e in p u t d a ta se q u en ce. A s w e will d e m o n s tra te , th e lin e a r filter
ta k e s th e fo rm o f a p a ra lle l b a n k o f re so n a to rs w h e re ea c h r e s o n a to r se lects o n e
o f th e fre q u e n c ie s a>k = 2 n k / N , k = 0, 1 , . . . , N 1, c o rre sp o n d in g to th e N
fre q u e n c ie s in th e D F T .
T h e re a re o th e r a p p lic a tio n s in w hich w e re q u ire th e e v a lu a tio n o f th e ztra n sfo rm o f a fin ite -d u ra tio n se q u e n c e a t p o in ts o th e r th a n th e u n it circle. If
th e se t o f d e sire d p o in ts in th e z-p lan e po ssesses so m e re g u la rity , it is possible
to also ex p ress th e c o m p u ta tio n o f th e z -tra n s fo rm a s a lin e a r filte rin g o p e ra tio n .
In th is c o n n e c tio n , w e in tro d u c e a n o th e r alg o rith m , called th e c h irp -z tra n sfo rm
alg o rith m , w hich is su ita b le fo r e v a lu a tin g th e z -tra n s fo rm o f a se t o f d a ta o n a
v ariety o f c o n to u rs in th e z-p lan e. T h is alg o rith m is also fo rm u la te d as a lin ear
filtering o f a set o f in p u t d a ta . A s a co n se q u e n c e , th e F F T a lg o rith m can b e used
to c o m p u te th e ch irp -z tra n sfo rm a n d th u s to e v a lu a te th e z -tra n s fo rm at various
c o n to u rs in th e z -p la n e , in clu d in g th e u n it circle.

6.3.1 The Goertzel Algorithm

T h e G o e rtz e l a lg o rith m ex p lo its th e p erio d icity o f th e p h ase fa c to rs {W} an d
allow s us to ex p re ss th e c o m p u ta tio n o f th e D F T as a lin e a r filterin g o p e ra tio n .
Since W # kN = 1, w e can m u ltip ly th e D F T by th is fa c to r. T h u s
(6.3.1)
W e n o te th a t (6.3.1) is in th e fo rm of a c o n v o lu tio n .
se q u e n c e yk(n) as

In d e e d , if w e d efin e the

(6.3.2)
m=0

th e n it is c le a r th a t ( n ) is th e co n v o lu tio n o f th e fin ite -d u ra tio n in p u t se q u en ce

x( n ) o f len g th N w ith a filter th a t h as an im pulse re sp o n s e
h k(n) = W ~ knu ( n )

(6.3.3)

T h e o u tp u t o f th is filter a t n = N y ields th e v alu e o f th e D F T a t th e freq u e n cy

an = h r k / N . T h a t is,
X ( k ) = >*(n)|n=JV

(6.3.4)

as can b e verified b y c o m p a rin g (6.3.1) w ith (6.3.2).

T h e filter w ith im p u lse re s p o n s e h k (n) h a s th e sy stem fu n c tio n

(6.3.5)

Sec. 6.3

A Linear Filtering Approach to Computation of the DFT

481

T h is filter h as a p o le o n th e u n it circle a t th e fre q u e n c y cd* = 2n k / N . T h u s, the

e n tire D F T can b e c o m p u te d by passin g th e block o f in p u t d a ta in to a p a ra l
lel b a n k o f N sin g le-p o le filters (re s o n a to rs), w h ere each filter h as a p o le at the
c o rre sp o n d in g fre q u e n c y o f th e D F T .
I n s te a d o f p e rfo rm in g th e c o m p u ta tio n o f th e D F T as in (6.3.2), via co n v o lu
tio n , w e can use th e d iffe re n c e e q u a tio n c o rre sp o n d in g to th e filter given by (6.3.5)
to c o m p u te y k(ir) recu rsiv ely . T h u s we h av e
y t (n) = W ^ kyt ( n - 1) + x i n )

V i-(-l) = 0

(6.3.6)

T h e d e sire d o u tp u t is X ( k ) = y k( N) , fo r k = 0, 1 , . . . , N 1. T o p e rfo rm this

c o m p u ta tio n , w e can c o m p u te o n ce a n d sto re th e p h a s e facto rs W # k.
T h e co m p lex m u ltip lic a tio n s an d a d d itio n s in h e re n t in (6.3.6) can be av o id ed
by co m b in in g th e p airs o f re so n a to rs p o ssessin g c o m p le x -c o n ju g a te p oles. T his
lead s to tw o -p o le filters w ith system fu n c tio n s o f th e form
] _ iy*
HkL ) ~ 1 - 2 c o s ( 2 t i k / N ) : ~ l + C ' 2

(6'3 '7)

T h e d irect form II re a liz a tio n o f th e system illu stra te d in Fig. 6.17 is d e sc rib e d by
th e d iffe re n c e e q u a tio n
2:rk

v k(n) = 2 cos

v*.(zi 1) - vk(n - 2) + x( i t )

Vi(h) = vk in) - W N
k vk (n - 1)

(6.3.8)
(6.3.9)

w ith in itial c o n d itio n s iv - ( - l) = vk{ - 2 ) = 0.

T h e recu rsiv e re la tio n in (6.3.8) is ite ra te d for n = 0, 1.........N , b u t th e e q u a
tio n in (6.3.9) is c o m p u te d o n ly o n ce a t tim e n = N. E ach ite ra tio n re q u ire s o n e
real m u ltip lic a tio n a n d tw o a d d itio n s. C o n se q u e n tly , fo r a re a l in p u t se q u e n c e
x ( n) . th is a lg o rith m re q u ire s N + 1 re a l m u ltip lic a tio n s to yield n o t o n ly X ( k ) b ut
also, d u e to sy m m etry , th e v a lu e o f X ( N k).
T h e G o e rtz e l alg o rith m is p a rtic u la rly a ttra c tiv e w h en th e D F T is to b e c o m
p u te d at a re lativ ely sm all n u m b e r M o f values, w h e re M < Iog2 N . O th erw ise,
th e F F T a lg o rith m is a m o re efficient m e th o d .

Figure 6.17 Direct form It realization

of two-pole resonator for computing the
DFT.

482

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

6.3.2 The Chirp-z Transform Algorithm

T he D F T o f an W -point data seq u en ce x(n ) has b een view ed as the z-transform
o f x i n ) evaluated at N equally spaced p oin ts on the unit circle. It has also been
view ed as N equally spaced sam ples o f the Fourier transform o f th e data sequ en ce
x (n ). In this section w e consider the evaluation o f X ( z ) on other contours in the
z-plane, including th e unit circle.
S u ppose that w e wish to com p ute the values o f the z-transform o f jc(n) at a
set o f p oints {z*}. T hen,
A '- l

X i z k) = J 2 x ( n ) z r

* = 0 , 1 .........L - 1

(6.3.10)

n=0

For exam ple, if the contour is a circle o f radius r and the z* are N equally spaced
points, then
Zk = r e j 2*"/"
2=1
X ( z k) = J 2 i x M r ~n}e
n=0

k =

0, 1,2 ..... N - 1
(6.3.11)

n/N

k = 0 , 1 , 2 .........N - 1

In this case the FFT algorithm can be applied on the m odified seq u en ce x { n ) r ~ n.
M ore generally, suppose that the p oin ts z* in the z-plane fall on an arc which
begins at som e point
Zo

= r0eJlk

and spirals either in toward the origin or out away from the origin such that the
points
are defined as
zk = rQe je(Roei *)i

k = 0 ,1 ,..., L - 1

(6.3.12)

N o te that if R0 < 1, the points fall on a con tour that spirals tow ard th e origin and if
R0 > 1, the contour spirals away from the origin. If Ro 1, the con tou r is a circular
arc o f radius ro. If r0 = 1 and Ro = l , the con tour is an arc o f th e unit circle. The
latter contour w ould allow us to com p ute the frequency con ten t o f the sequence
x ( n ) at a dense set o f L freq u en cies in the range covered by the arc w ithout having
to com pute a large D F T , that is, a D F T o f the seq u en ce x ( n ) pad d ed with many
zeros to obtain the desired resolution in frequency. Finally, if r0 = Ro = 1,
= 0,
0o = 2n / N , and L = N , the contour is the entire unit circle and the frequencies
are those o f the D F T . T h e various contours are illustrated in Fig. 6.18.
W hen points {z*J in (6.3.12) are substituted in to the exp ression for the ztransform, w e obtain
* ( z t ) = X ! -* ( > z r i
n=0
n=
N-1

= j > ( n ) ( r 0e j * ) ~ V

(6.3.13)

Sec. 6.3

A Linear Filtering Approach to Computation of the DFT

lm(r)

ImU)

lm (;l

Im(;)

483

n=0

Figure 6.18 Some examples of contours on which we may evaluate the ztransform.
w here, by definition.
V = R veJ^

(6.3.14)

W e can exp ress (6.3.13) in the form o f a con volu tion , by n oting that
nk = j[n 2 + k 2 (k n) 2]

(6.3.15)

Substitution o f (6.3.15) into (6.3.13) yield s

N- 1

X(C*) = V -

l ' Z /2

J 2 [ x ( n ) ( r 0eJlk' ) - nV - n2f2] V (k- n)2fZ

(6.3.16)

Let us define a n ew seq u en ce g ( n ) as

g (n) = x ( n )( r ^ e j<hr n V - n^

(6.3.17)

T h en (6.3.16) can b e exp ressed as

X ( z k) = V - k2/2y g ( n ) V {k-',)1/2

(6.3.18)

484

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

T he sum m ation in (6.3.18) can be interpreted as the con volu tion o f the sequ en ce
g (n) with the im pulse resp onse h (n) o f a filter, where
h(n) = V n2/2

(6.3.19)

C onsequently, (6.3.18) m ay b e expressed as

X(zt) = V -*Vy(k)
(6.3.20)

w here y(Jt) is the ou tp u t o f the filter

s 1
y ( k ) = Y ' g ( n ) h ( k n)
n=0

k = 0. 1.........L 1

(6.3.21)

W e observe that b oth h (n) and g(n) are com p lex-valu ed seq u en ces.
T he sequ en ce h (n) with R 0 = 1 has the form o f a com plex exp on en tial with
argum ent (on = n 2<(>o /2 = (n 0 o /2 )n. T h e quantity
rep resents the frequency
o f the com plex exp on en tial signal, which increases linearly with tim e. Such signals
are used in radar system s and are called chi rp signals. H en ce th e z-transform
evaluated as in (6.3.18) is called the chi rp-z t ransf orm.
T h e linear con volu tion in (6.3.21) is m ost efficien tly d on e by use o f the FFT
algorithm . T he seq u en ce g( n) is o f length N . H ow ever, h{n) has infinite du
ration. Fortunately, only a portion h{n) is required to co m p u te the L values
o f X (z).
Since w e will com p ute the con volu tion in (6.3.1) via the F FT, let us consider
the circular con volu tion o f the W-point seq u en ce g{n) with an M -p oint section of
/i(n), w here M > N . In such a case, w e k n ow that the first N 1 p oin ts contain
aliasing and that the rem aining M N + 1 p oints are identical to the result that
would b e obtained from a linear con volu tion o f h( n) with g(n). In view o f this, we
should select a D F T o f size
M - L + N - 1
which would yield L valid p oin ts and N - 1 points corrupted by aliasing.
T he section o f h(n) that is n eed ed for this com putation corresp on d s to the
values o f h{ri) for ( N - 1) < n < (L 1), which is o f length M = L + N 1, as
observed from (6.3.21). Let us define the seq u en ce h \{n ) o f length M as
/ii(n ) = h(n N -f 1)

n 0 , 1 .........M 1

(6.3.22)

and com p ute its Af-poin t D F T via the FFT algorithm to obtain H \ ( k ) . F rom x (n )
w e com p ute g ( n ) as specified by (6.3.17), pad g(n ) w ith L 1 zeros, and com
pute its Af-point D F T to yield G(Jfc). T h e ID F T o f th e product y i(* ) = G ( k ) H \( k )
yields the Af-point seq u en ce > i(n ), n = 0, 1 , . . . , Af 1. T h e first N 1 p oints of
y i( ) are corrupted by aliasing and are discarded. T h e desired valu es are yi(n)
f o r N 1 < n < M 1, w hich correspond to the range 0 < n < L l i n (6.3.21),

Sec. 6.3

A Linear Filtering Approach to Computation of the DFT

485

that is,
y(n) = y t ( n + N 1)

n = 0, 1.........L 1

(6.3.23)

A ltern atively, w e can define a seq u en ce ft 2 (n) as

h 2(n) =

h (n),
h ( n ~ N - L + l),

< n < L 1
L < ti < M 1

(6.3.24)

The A f-point D F T o f h 2{n) yields H2(k), which w hen m ultiplied by G( k ) yields

Y2(k) = G( k ) Hz ( k ) . T he ID F T o f Y2(k) yield s the seq u en ce y2( n) for 0 < n < A f - 1 .
N o w the desired valu es o f >2 (") are in the range 0 < n < L 1, that is,
y ( n ) = y 2(n)

n = 0, 1 , . . . , L 1

(6.3.25)

Finally, the com p lex valu es X(Zi) are com p uted by dividing y( k) by h ( k ),
k = 0, 1.........L 1, as specified by (6.3.20).
In gen eral, the com p utational com p lexity o f the chirp-z transform algorithm
described ab ove is o f the order of Af log 2 M com plex m ultiplications, where M =
N + L ~ 1. T h is num ber should be com pared with the product, N L, the num ber
o f com p utations required by direct evaluation o f the z-transform . Clearly, if L is
sm all, direct com p utation is m ore efficient. H ow ever, if L is large, then the chirp-z
transform algorithm is m ore efficient.
T h e chirp-z transform m eth od has b een im plem ented in hardware to com pute
the D F T o f signals. For the com putation of the D FT, w e select ro = /?(i = 1, 6\j = 0,
</>o = 2n / N , and L = N. In this case
y-ir/2 _

e -jjin -/N

nn2
. Tin2
= c o s --------- j s i n ------N
N

<6 '3 '26 >

T he chirp filter with im pulse response

h( n) = V nlfl
t2
. nn2
= c o s -------i n ---- (- j/ ssin

N
N

(6.3.27)

= h r(n) + jh , { n )
has b een im p lem en ted as a pair o f F IR filters w ith coefficients h r (n) and A,(n),
resp ectively. B o th su rface acous tic w av e (SAW ) d evices and charge co u p l e d d e
vices (C C D ) h ave b een u sed in practice for the F IR filters. T h e cosine and sine
seq u en ces given in (6.3.26) n eed ed for the prem ultiplications and postm ultiplica
tion s are usually stored in a read-only m em ory (R O M ). Furtherm ore, w e n ote that
if o n ly the m agnitu d e o f the D F T is desired, the postm ultiplications are u n n eces
sary. In this case,
|X (z*)l = \y(k)\

k = 0 ,1 ,...,

-1

(6.3.28)

as illustrated in Fig. 6.19. T hus the linear F IR filtering approach using th e chirp-z
transform has b een im p lem en ted for the com putation o f the D F T .

486

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

Chirp Fillers

Figure 6.19 Block diagram illustrating the implementation of the chirp-z transform for com
puting the DFT (magnitude only).

6.4 QUANTIZATION EFFECTS IN THE COMPUTATION OF THE DFT*

A s w e have ob served in our p reviou s discussions, the D F T plays an im portant role
in m any digital signal p rocessing applications, including F IR filtering, the com pu
tation o f the correlation betw een signals, and spectral analysis. For this reason
it is im portant for us to kn ow th e effect o f quantization errors in its com puta
tion. In particular, w e shall consider the effect o f rou n d -off errors due to the
m ultiplications perform ed in the D F T with fixed-point arithm etic.
T h e m odel that w e shall adopt for characterizing rou n d -off errors in m ulti
plication is the additive w hite n oise m o d el that w e use in the statistical analysis
o f Tound-off errors in IIR and F IR filters (see Fig. 7.34). A lth ou gh the statistical

*It is recommended that the reader review Section 7.5 prior to reading this section.

Sec. 6.4

487

Quantization Effects in the Computation of the DFT

analysis is perform ed for rounding, the analysis can be easily m odified to apply to
truncation in tw o's-com p lem en t arithm etic (see Sec. 7.5.3).
O f particular interest is the analysis o f rou n d -off errors in the com putation
o f the D F T via the FFT algorithm . H ow ever, w e shall first establish a benchm ark
by determ ining the round-off errors in the direct com p utation o f the D F T .

6.4.1 Quantization Errors in the Direct Computation of

the DFT
G iven a finite-duration seq u en ce (jt(n)], 0 < n < N 1, the D F T o f {jc(h)1 is
defined as
A/-1

* (* ) = Y l x ( n ) w "'
j,=0

= 0 , 1 ........ N - 1

(6.4.1)

w here IVyv = c ~ )2r,/N. W e assum e that in general, {*()] is a com p lex-valu ed se

quence. W e also assum e that the real and im aginary com p on en ts o f {a (h)I and
{VV^"] are represented by b bits. C onsequently, the com putation o f the product
requires four real m ultiplications. Each real m ultiplication is rounded
from 2b bits to b bits, and hence there are four quantization errors for each
com p lex-valu ed m ultiplication.
In the direct com putation o f the D F T , there are N com p lex-valu ed m ultiplica
tions for each point in the D FT. T herefore, the total num ber o f real m ultiplications
in the com putation o f a single point in the D F T is 4 N. C on sequ en tly, there are
4 N quantization errors.
Let us evaluate the variance o f the quantization errors in a fixed-point com
putation o f the D F T . First, w e m ake the follow in g assum ptions about the statistical
properties o f the quantization errors.
1. T h e quantization errors due to rounding are uniform ly distributed random
variables in the range (A /2 , A /2 ) where A = 2~ b.
2. T h e 4 N quantization errors are m utually uncorrelated.

3. T h e 4 N quantization errors are uncorrelated with the seq u en ce |jc{}}.

Since each o f the quantization errors has a variance
A 2

7~2b

" ' = 1 2 = 1 2

<6A2>

the variance o f the quantization errors from the 4 N m ultiplications is

<r2 = 4 N o ]

(6A3)
3
H en ce the variance o f the quantization error is p roportional to the size o f D FT.
N o te that w hen Af is a p ow er o f 2 (i.e., N = 2 1), the variance can be expressed

488

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

2 2(h1*/2
a ] = -------------

Chap. 6

(6.4.4)

This expression im plies that every fourfold increase in the size N o f the D F T
requires an additional bit in com putational precision to offset the additional quan
tization errors.
T o prevent overflow , the input seq u en ce to the D F T requires scaling. Clearly,
an upper bound on | X (A:) | is
A '- l

[* ()! < Y

|*(n )|

(6.4.5)

n=0

If the dynam ic range in addition is ( - 1 , 1 ) , then |X (/:)| <

requires that

A '-l

|jr(/i)| < 1

(6.4.6)

n=0

If U (/i)| is initially scaled such that |a (/j)| < 1 for all n, then each point in the
seq u en ce can be divided by N to ensure that (6.4.6) is satisfied.
T h e scaling im plied by (6.4.6) is extrem ely severe. For exam p le, su p p ose
that the signal seq u en ce {*()} is white and. after scaling, each valu e |.r(n)l o f the
seq u en ce is uniform ly distributed in the range (-1 /7 V , I/ N ) . T h en the variance of
the signal sequ en ce is
<2 / * > 2 = 1 o ,2 = ----------

3N1

12

tzA-n
(6.4.7)

>

and the variance o f the output D F T coefficients |Jf(/t)l is

al = N a2
1

(6.4.8)

~ 3 ~N
Thus the signal-to-noise p ow er ratio is
(6.4.9)
W e observe that the scaling is responsible for reducing th e S N R by N and
the com bination o f scaling and quantization errors result in a total reduction that
is proportional to N 2. H en ce scaling the input seq u en ce (j(n )} to satisfy (6.4.6)
im poses a severe p en alty on the signal-to-noise ratio in the D F T .
Exam ple 6.4.1
Use (6.4.9) to determ ine the num ber of bits required to com pute the D FT of a 1024point sequence with a SNR of 30 dB.
Solution

The size of the sequence is N = 210. Hence the SNR is

Sec. 6.4

Quantization Effects in the Computation of the DFT

489

For an SNR o f 30 dB, we have

3(2* - 20) = 30
b = 15 bits
N ote that the 15 bits is the precision for both multiplication and addition.
Instead o f scaling the input sequ en ce {Jt(n)}, suppose w e sim ply require that
|x(n)l < 1. T h en w e m ust provide a sufficiently large dynam ic range for addition
such that |* ( * ) l < N . In such a case, the variance o f the seq u en ce {|jc(n)|) is
a 2 = 5 , and h en ce th e variance o f |X (* )| is
(6.4.10)
C on sequ en tly, the S N R is
(6.4.11)
If w e repeat the com putation in E xam ple 6.4,1, w e find that the num ber o f
bits required to a ch ieve a S N R o f 30 dB is b = 5 bits. H ow ever, w e n eed an
additional 1 0 bits for the accum ulator (th e adder) to accom m odate the increase
in the dynam ic range for addition. A lthou gh w e did not ach ieve any reduction
in the dynam ic range for addition, we have m anaged to reduce the p recision in
m ultiplication from 15 bits to 5 bits, which is highly significant.

6.4.2 Quantization Errors in FFT Algorithms

A s w e have sh ow n , the F F T algorithm s require significantly few er m ultiplications
than the direct com p utation o f the D F T . In view o f this w e m ight con clu d e that the
com p utation o f the D F T via an FFT algorithm w ill result in sm aller quantization
errors. U n fortu n ately, that is n ot the case, as w e will dem onstrate.
L et us con sid er the use o f fixed-point arithm etic in th e com putation o f a
radix-2 F F T algorithm . T o be specific, w e select the radix-2, decim ation-in-tim e
algorithm illustrated in Fig. 6.20 for the case N = S. T h e results on quantiza
tion errors that w e ob tain for this radix-2 FFT algorithm are typical o f th e results
o b ta in ed w ith o th er radix - 2 and higher radix algorithm s.
W e o b serv e that each butterfly com putation in volves o n e com plex-valued
m ultiplication or, eq u ivalen tly, four real m ultiplications. W e ignore the fact that
so m e butterflies con tain a trivial m ultiplication by 1 . If w e consider th e but
terflies that affect the com p utation o f any on e valu e o f the D F T , w e find that,
in gen eral, there are N /2 in the first stage of the FFT , N / 4 in the secon d stage,
N / 8 in the third state, and so on , until the last stage, w here there is on ly on e.
C on sequ en tly, th e num ber o f butterflies per output point is

2 " - '+ 2 " - 2 + --- + 2 + l = 2v [ l + ( ! ) + + ( j ) ']

= 2 [ l - ( j n

= W -l

490

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Stage 1

Stage 2

Chap. 6

Stage 3

For exam ple, the butterflies that affect the com p utation o f A"(3) in the eight-point
FFT algorithm o f Fig. 6.20 are illustrated in Fig. 6.21.
T h e quantization errors introduced in each butterfly propagate to the output.
N o te that the quantization errors introduced in the first stage p ropagate through
(v - 1 ) stages, th ose introduced in the second stage propagate through (v - 2 )
stages, and so on. A s these quantization errors propagate through a num ber of
su bsequent stages, th ey are phase shifted (ph ase rotated) by th e phase factors
W^n. T h ese phase rotations do not change the statistical p rop erties o f the quan
tization errors and, in particular, the variance o f each q uantization error remains
invariant.
If w e assum e that the quantization errors in each butterfly are uncorrelated
with the errors in other butterflies, then there are 4(W - 1 ) errors that affect the
output o f each point o f the FFT. C on sequ en tly, th e variance o f the total quanti
zation error at the output is
A

f f| = 4 (A r- ! )

(6.4.13)

Sec. 6.4

Quantization Effects in the Computation of the DFT

Figure 6.21

491

Butterflies that affect the computation o f X (3).

w here A = 2 h. H en ce
a2= j

2"

(6.4.14)

T his is exactly the sam e result that w e ob tain ed for the direct com p utation o f the
DFT.
T h e result in (6.4.14) should n ot b e surprising. In fact, the FFT algorithm
d o es not reduce the num ber o f m ultiplications required to com p ute a single point
o f the D F T . It d o es, h ow ever, exp loit the p eriod icities in W^n and thus reduces
the num ber o f m ultiplications in the com p utation o f the entire block o f N points
in the D F T .
A s in the case o f the direct com p utation o f the D F T , w e m ust scale the
input seq u en ce to prevent overflow . R ecall that if |jc(n) | < \ / N , 0 < n < N
1, then |X (* )| < 1 for 0 < k < N 1. T hus overflow is avoided. W ith this
scaling, the relation s in (6.4.7), (6.4.8), and (6.4.9), ob tain ed previously for the
direct com p utation o f the D F T , apply to the F F T algorithm as w ell. C onsequently,
the sam e S N R is o b tain ed for the FFT.
Since the F F T algorithm consists o f a seq u en ce o f stages, w h ere each stage
con tains butterflies that in volve pairs o f points, it is p ossib le to d evise a differ
en t scaling strategy that is n ot as severe as dividing each input p oin t by N . This

492

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

alternative scaling strategy is m otivated by the observation that the in term edi
ate values [Xn(/r)| in the n = 1, 2,..., u stages o f th e F F T algorithm satisfy the
conditions (see P roblem 6.35)
m ax[|X n+1 ( * ) U X n+1(/)|] > m ax[|X n( * ) U X B( 0 |]
(6.4.15)
m ax [|X n+1 ( * ) |,|X B+1(/)|] <

m ax[jX (Jt)|,|X n(/)|]

In view o f th ese relations, w e can distribute the total scalin g o f 1 / N in to each

o f the stages o f the F F T algorithm . In particular, if |jr(n)| < 1, w e apply a scale
factor o f 5 in the first stage so that |jr(n)| <
T h en the output o f each subsequent
stage in the FFT algorithm is scaled by | , so that after v stages w e have achieved
an overall scale factor o f ( j ) 1' = 1 //V. Thus overflow in the com p utation o f the
D F T is avoided.
This scaling procedure d o es not affect the signal level at the output o f the
FFT algorithm , but it significantly reduces the variance o f the quantization errors
at the output. Specifically, each factor o f ^ reduces the variance o f a quantization
error term by a factor o f

Thus the 4 ( N / 2 ) quantization errors introduced in

the first stage are reduced in variance by (^ V - 1 , the 4 ( N / 4 ) quantization errors

introduced in the second stage are reduced in variance by ( j ) 1- 2 . and so on. C on
sequ en tly, the total variance o f the quantization errors at the output o f the FFT
algorithm is

w here the factor (^ )tJ is negligible.

W e now o b serve that (6.4.16) is n o longer proportional t o N . O n th e other
hand, the signal has th e variance a \ = 1 /3 N , as given in (6.4.8). H e n c e the S N R is

f l = _ L .2 ^
_

2N
22bv\

(6.4.17)

Thus, by distributing th e scaling o f l / N uniform ly throughout th e FFT algorithm ,

w e have achieved an S N R that is inversely proportional to N in stead o f N 2.
Example 6.4.2
Determine the number of bits required to compute an FFT of 1024 points with an
SNR of 30 dB when the scaling is distributed as described above.

Sec. 6.5

Summary and References

Solution

493

The size of the FFT is N = 210. Hence the SNR according to (6.4.17) is
101gio 22h~v~l = 30
3(2b - 11) = 30
b =

bits)

This can be com pared with the 15 bits required if all the scaling is perform ed in the
first stage of the FFT algorithm.

6.5 SUMMARY AND REFERENCES

T h e fo cu s o f this chapter w as on the efficien t com putation o f the D F T . W e d em on
strated that by taking advantage o f the sym m etry and p eriodicity p roperties o f the
ex p on en tial factors W#", w e can reduce the num ber o f com p lex m ultiplications
n eed ed to com p ute the D F T from N 2 to N log 2 N w hen Af is a p ow er o f 2. A s w e
indicated, any seq u en ce can be augm ented with zeros, such that N 2''.
For d ecad es, FFT -type algorithm s were o f interest to m athem aticians w ho
w ere con cern ed w ith com p utin g values o f F ourier series by hand. H ow ever, it
w as not until C o o ley and T u k ey (1965) published their w ell-k now n paper that the
im pact and significance o f the efficient com putation o f the D F T was recognized.
Since then the C o o le y -T u k e y FFT algorithm and its various form s, for exam ple,
the algorithm s o f S in gleton (1967, 1969), have had a trem en dou s influence on the
use o f the D F T in con v olu tion , correlation, and spectrum analysis. For a historical
p erspective on the F FT algorithm , the reader is referred to the paper by C ooley
et al. (1967).
T h e split-radix FFT (SR F F T ) algorithm d escribed in Section 9.3.5 is due
to D u h a m el and H ollm an n (1 9 8 4 ,1 9 8 6 ). T he m irror F F T (M FF T ) and p h ase
F F T (PF FT ) algorithm s w ere described to the authors by R. Price. T he exp loitation
o f sym m etry p rop erties in the data to reduce the com putation tim e are described
in a paper by Sw arztrauber (1986).
O ver the years, a num ber o f tutorial papers have b een published on FFT
algorithm s. W e cite the early papers by Brigham and M orrow (1967), Cochran et
al. (1967), B ergland (1969), and C ooley et al. (1967, 1969).
T h e reco g n itio n that the D F T can b e arranged and com p uted as a linear
con volu tion is also highly significant. G o ertzel (1968) indicated that the D F T
can b e com p uted via linear filtering, although the com p utational savings o f this
approach is rath er m odest, as w e have observed. M ore significant is the work
o f B lu estein (1 9 7 0 ), w ho d em onstrated that the com p utation o f the D F T can be
form u lated as a chirp linear filtering operation. T his w ork led to the d evelop m en t
o f th e chirp-z transform algorithm by R ab in er et al. (1969).
In addition to the F F T algorithm s describ ed in this chapter, there are other
efficien t algorithm s for com p utin g the D F T , som e o f w hich further reduce the

494

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

num ber o f m ultiplications, but usually require m ore additions. O f particular im

portance is an algorithm due to R ader and B renner (1976), the class o f prim e factor
algorithm s, such as the G o o d algorithm (1971), and the W inograd algorithm (1976,
1978). For a d escription o f these and related algorithm s, the reader m ay refer to
the text by Blahut (1985).

PROBLEMS
6.1 Show that each of the numbers
eja*/NH

o < /t < W - 1

corresponds to an Wth root of unity. Plot these numbers as phasors in the complex
plane and illustrate, by m eans of this figure, the orthogonality property
-ja ir/N M n

n=<)

N,
I 0,
I

if k ~ 1
if k ^ l
1

6.2 (a) Show that the phase factors can be com puted recursively by
W$ =
(b) Perform this com putation once using single-precision floating-point arithmetic
and once using oniy four significant digits. Note the deterioration due to the
accumulation of round-off errors in the later case.
(c) Show how the results in part (b) can be improved by resetting the result to the
correct value - j . each time gl = N/4.
6 3 Let x(n) be a real-valued N -point (N = 2' ) sequence. Develop a method to compute
an N -point D FT X (k), which contains only the odd harmonics [i.e., X'(k) = 0 if Jt is
even] by using only a real A,/2-spoint DFT.
6.4 A designer has available a num ber of eight-point FFT chips. Show explicitly how he
should interconnect three such chips in order to com pute a 24-point DFT.
6.5 The ^-transform of the sequence x(n) = u(n) - u(n - 7) is sampled at five points on
the unit circle as follows
x(k) = X(z) 1- = eJ'2jr*/5

Jt* 0,1 ,2 ,3 ,4

Determ ine the inverse D FT x'(n) of X (Jt). Com pare it with *(/t) and explain the
results.
6.6 Consider a finite-duration sequence x(n), 0 < n < 7, with z-transform X(z). We wish
to com pute X (:) at the following set of values:
zk = 0.8ejf|(2)r*/*,+(' /8)]

0 < Jfc < 7

(a) Sketch the points {z*} in the complex plane.

D eterm ine a sequence s ( n ) such that its D F T provides the desired samples of

( b )

*(z).

Chap. 6

495

Problems

6.7 Derive the radix-2 decimation-in-time FFT algorithm given by (6.1.26) and (6.1.27)
as a special case of the more general algorithmic procedure given by (6.1.16) through
(6.1.18).
6.8 Com pute the eight-point D FT of the sequence
1.
I 0,

X^

0 < n < 7
otherwise

by using the decimation-in-frequency FFT algorithm described in the text.

6.9 Derive the signal flow graph for the N = 16 point, radix-4 decimation-in-time FFT
algorithm in which the input sequence is in norm al order and the computations are
done in place.
6.10 D erive the signal flow graph for the N = 16 point, radix-4 decimation-in-frequency
FFT algorithm in which the input sequence is in digit-reversed order and the output
D FT is in normal order.
6.11 Com pute the eight-point DFT of the sequence
x{n) = U . i

1,0. 0,0.0

12 2 2 2

using the in-placc radix-2 dccimation-in-time and radix-2 decimation-in-frequency al

gorithms. Follow exactly the corresponding signal flow graphs and keep track of all
the interm ediate quantities by putting them on the diagrams,
6.12 Com pute the 16-point D FT of the sequence
x(n) = cos

0 < n < 15

using the radix-4 decimation-in-time algorithm.

6.13 Consider the eight-point decimation-in-tim e (D IT) flow graph in Fig. 6.6.
(a) What is the gain of the signal p ath that goes from x(7) to X(2)7
(b) How many paths lead from the input to a given output sample? Is this true for
every output sample?
(c ) Com pute X (3) using the operations dictated by this flow graph.
6.14 Draw the flow graph for the decimation-in-frequency (D IF) SRFFT algorithm for
N = 16. W hat is the num ber of nontrivial multiplications?
6.15 D erive the algorithm and draw the N = 8 flow graph for the D IT SRFFT algorithm.
Com pare your flow graph with the D IF radix-2 FFT flow graph shown in Fig. 6.11.
6.16 Show that the product of two complex numbers (a + j b ) and ( C + j d ) can be perform ed
with three real multiplications and five additions using the algorithm
x R

(a

b)d

+ (c -

d)a

xi

(a

b)d

+ (c +

d)b

where
X =

x R

jx ,

= (a +

jb )(c

jd )

6.17 Explain how the D FT can be used to com pute N equispaced samples of the ztransform , of an iV-point sequence, on a circle of radius r.

496

Efficient Computation of the DFT: Fast Fourier Transform Algorithms

Chap. 6

6.18 A real-valued A/-point sequence Jt(n) is called D FT bandlim ited if its D FT X(k) = 0
for ko < k < N An. We insert (L 1)N zeros in the middle of A'(Jt) to obtain the
following L N -point DFT
X(k),
X'(k) = { 0,
X(k + N - LN) .

0 < i < Ao 1
Jt0 < Jk < L N - kt,
L N - J t + 1 < Jt < L N 1

Show that
Lx'(Ln) = x(n)

0 < n < N 1

where
x\n)

X (k)
LN

Explain the meaning of this type of processing bv working out an example with N = 4,
L = 1. and A ( J t ) = { 1 , 0 . 0 . 1 ) .
6.19 Let X(k) be the A'-point DFT of the sequence .v(n). 0 < n < A - 1. What is the
A-point DFT of the sequence s(n) = X(n). 0 < n < N - 1?
6.20 Let X(k) be the A-point DFT of the sequence \(n ), (I < < N 1. We define a
2 N -point sequence _v() as
V{;I)

1-v^V

--vcn
n odd

Express the 2 Appoint DFT of y(n) in terms of X(k).

6.21 (a) Determ ine the ^-transform W (-) of the Hanning window w () = (1 - cos
fl.
(b) Determ ine a formula to compute the JV-point DFT X(k) of ihe signal .r.(n) =
w(n)x(n)< 0 < n < N 1. from the JV-point DFT A'(A ) of the signal .r(n).
6.22 Create a DFT coefficient table that uses only N/4 memory locations to store the first
quadrant of the sine sequence (assume N even).
6.23 D eterm ine the com putational burden of the algorithm given by (6.2.12) and compare
it with the com putational burden required in the 2N -point DFT of g(n). Assume that
the FFT algorithm is a radix-2 algorithm.
6.24 Consider an IIR system described by the difference equation
S'

v(n) = - ^

ai-v<n ~

+ y , b k x ( n - k)

*= 1

Describe a procedure that com putes the frequency response H

Jt = 0, 1........

A ' - l using the FFT algorithm ( N = 2).

6.25 Develop a radix-3 decimation-in-time FFT algorithm for N = 3 and draw the corre
sponding flow graph for N = 9. W hat is the num ber of required com plex multiplica
tions? Can the operations be perform ed in place?
6.26 Repeat Problem 6.25 for the D IF case.
6.27 FFT input and output pruning In many applications we wish to com pute only a few
points M of the Appoint D FT of a finite-duration sequence of length L (i.e., M N
and I < < N).

Chap. 6

Problems

497

(a) Draw the flow graph of the radix-2 D IF FFT algorithm for N = 16 and eliminate
[i.e., prune] all signal paths that originate from zero inputs assuming that only
jc(0) and jc(1) are nonzero.
(b) R epeat part (a) for the radix-2 D IT algorithm.
(c) Which algorithm is better if we wish to com pute all points of the D FT? What
happens if we want to compute only the points X (0), X (l), X (2), and X (3)?
Establish a rule to choose between D IT and D IF pruning depending on the
values of M and L.
(d) Give an estim ate of saving in computations in term s of M, L, and N.
6.28 Parallel computation o f the D F T Suppose that we wish to com pute an N = 2P2V
point D FT using 2P digital signal processors (DSPs). For simplicity we assume that
p = v = 2. In this case each DSP carries out all the com putations that are necessary
to com pute 2V D FT points.
(a) Using the radix-2 D IF flow graph, show that to avoid data shuffling, the entire
sequence x(n) should be loaded to the memory of each DSP.
(b) Identify and redraw the portion of the flow graph that is executed by the DSP
that com putes the D FT samples X(2), X(10), X(6), and X(14).
(c) Show that, if we use M = 2'' DSPs, the com putation speed-up S is given by
S= M

log-, N
log2 N - log, M + 2(M - 1)

6.29 Develop an inverse radix-2 D IT FFT algorithm starting with the definition. Draw the
flow graph for com pulation and com parc with the corresponding flow graph for the
direct FFT. Can the IFFT flow graph be obtained from the one for the direct FFT?
6.30 R epeat Problem 6.29 for the D IF case.
6.31 Show that an FFT on data with H erm itian symmetry can be derived by reversing the
flow graph of an FFT for real data.
6.32 D eterm ine the syslem function H(z) and the difference equation for the system that
uses the G oertzel algorithm to compute ihe DFT value X ( N - k).
6.33 (a) Suppose that x(n) is a finite-duration sequence of N = 1024 points. It is desired
to evaluate the z-transform X (;) of the sequence at the points
Zk

eiQ * * *

k = 0 . 100,200 ......1000

by using the most efficient m ethod or algorithm possible. Describe an algorithm

for perform ing this com putation efficiently. Explain how you arrived at your
answer by giving the various options or algorithms that can be used.
(b) R epeat part (a) if X (z) is to be evaluated at
zt = 2(0.9) V '1(2,r/5000*+r/21

k = 0 ,1 ,2 ........999

634 R epeat the analysis for the variance of the quantization error, carried out in Sec
tion 6.4.2, for the decimation-in-frequency radix-2 FFT algorithm.

635 The basic butterfly in the radix-2 decimation-in-time FFT algorithm is

*+,<*) = Xn( * )+ W X(l)

498

Efficient Computation of ttie DFT: Fast Fourier Transform Algorithms

Chap. 6

(a) If we require that !X(Jt)| < j and |Jf(/)| < 5 , show that
|Re[X +,(*)]| < 1,

|/te[X n+1(/)]| < 1

| I m [ X + 1 (*)]l <

| / m [ A B + 1 (/)]| <

1,

Thus overflow does not occur.

(b) Prove that
max[|X,,+i(A)|. |X+i(/)|] > m ax[|Xn(*)l.l*,,(/)|]
max[|X+1(*)UX,,+1(/)|] < 2 m ax [|X (* )|.|* n (/)l]
636* Computation o f the D F T Use an FFT subroutine to compute the following DFTs
and plot the m agnitudes |X(Jt)| of the DFTs.
(a) The 64-point D FT of the sequence
n = 0 ,1 ........15
otherwise

, ,
[1 ,
x(n) = I
10,

(Ni ~ 16)

(b) The 64-point D FT of the sequence

1,
10,

n = 0 ,1 ........... 7
otherwise

(N\ = 8)

(c) The 128-point DFT of the sequence in part (a).

(d) The 64-point D FT of the sequence
| 10f >'*/*>",
x (n) {
J 0.

= 0 ,1 ........63
otherwise

(Ni = 64)

Answer the following questions.

(1) What is the frequency interval between successive samples for the plots in
parts (a), (b). (c), and (d)?
(2) What is the value of the spectrum at zero frequency (dc value) obtained
from the plots in parts (a), (b), (c), (d)?
From the formula
X(k) = Y ^ x (n ) e '
compute the theoretical values for the dc value and check these with the
com puter results.
(3) In plots (a), (b), and (c), what is the frequency interval betw een successive
nulls in the spectrum ? W hat is the relationship between N 1 of the sequence
x( n ) and the frequency interval between successive nulls?
(4) Explain the difference between the plots obtained from parts (a) and (c).
637* Identification o f pole positions in a system
difference equation

Consider the system described by the

y(n) = r2y(n 2) + x(n)

(a) Let r = 0.9 and x(n) = &(n). G enerate the output sequence y(n) for 0 < n < 127.
Compute the N = 128 point D FT [ I'M ) and plot {|y(Jt)t).

Chap. 6

Problems

499

(b) Com pute the N = 128 point DFT of the sequence

ui(n) = (0.92)- " v(n)
where y(/t) is the sequence generated in part (a). Plot the D FT values | W (t)|.
W hat can you conclude from the plots in parts (a) and (b)?
(c) Let r = 0.5 and repeat part (a).
(d) R epeat part (b) for the sequence
ui(n) = (0.55)~nv(n)
where y( n) is the sequence generated in part (c). W hat can you conclude from
the plots in parts (c) and (d)?
(e) Now let the sequence generated in part (c) be corrupted by a sequence of mea
surem ent" noise which is Gaussian with zero mean and variance a 2 = 0.1. Repeat
parts (c) and (d) for the noise-corrupted signal.

Implementation of
Discrete-Time Systems

The focus o f this chapter is on the realization o f linear tim e-invariant discrete
tim e system s in eith er softw are or hardware. A s w e noted in C h ap ter 2, there are
various configurations or structures for the realization o f any F IR and IIR discrete
tim e system . In C hapter 2 w e described the sim plest o f these structures, nam ely,
the direct-form realizations. H ow ever, there are other m ore practical structures
that offer som e distinct advantages, especially w hen q uantization effects are taken
into consideration.
O f particular im portance are the cascade, parallel, and lattice structures,
which exhibit robustness in finite-w ord-length im plem en tation s. A lso described
in this chapter is the frequency-sam pling realization for an F IR system , which
often has the advantage o f being com putationally efficient w hen com pared with
alternative FIR realizations. O ther im portant filter structures are ob tain ed by
em ploying a state-sp ace form ulation for linear tim e-invariant system s. A n analysis
o f system s characterized by th e state-variable form is p resen ted in both the tim e
and frequency dom ains.
In addition to describing the various structures for the realization o f discrete
tim e system s, w e a lso treat problem s associated with q uantization effects in the
im plem en tation o f digital filters using finite-precision arithm etic. T his treatm ent
includes the effects on the filter frequency response characteristics resulting from
coefficient quantization and the round-off noise effects inherent in the digital im
p lem entation o f d iscrete-tim e system s.

7.1 STRUCTURES FOR THE REALIZATION OF DISCRETE-TIME

SYSTEMS
Let us consider the im portant class o f linear tim e-invariant d iscrete-tim e system s
characterized by the general linear constant-coefficient d ifferen ce eq u ation
N

(7.1.1)
500

Sec. 7.1

Structures for the Realization of Discrete-Time Systems

501

A s w e have sh ow n by m eans o f the z-transform , such a class o f linear tim e-invariant

d iscrete-tim e system s are also characterized by the rational system function
M
H(z) =

(7.1.2)

which is a ratio o f tw o polyn om ials in z - 1 . From the latter characterization, we

obtain the zero s and p o les o f the system function, which d ep en d on the ch oice o f
the system p aram eters {>*} and {a*} and which determ ine the frequency response
characteristics o f the system .
O ur fo cu s in this chapter is on the various m eth od s o f im plem en ting (7.1.1)
or (7.1.2) in eith er hardware, or in softw are on a program m able digital com puter.
W e shall sh ow that (7.1.1) or (7.1.2) can be im plem en ted in a variety o f ways
d ep en d ing on the form in which th ese tw o characterizations are arranged.
In g en eral, w e can view (7.1.1) as a com putational p rocedure (an algorithm )
for determ in in g the output seq u en ce v(/t) o f the system from the input sequ en ce
x ( n ) . H o w ev er, in various ways, the com putations in (7.1.1) can be arranged into
eq u ivalen t sets o f d ifferen ce equations. Each set o f eq u ation s defines a com pu
tational procedure or an algorithm for im plem enting the system . From each set
o f eq u ation s w e can construct a block diagram consisting o f an interconnection o f
d elay elem en ts, m ultipliers, and adders. In Section 2.5 w e referred to such a block
diagram as a realization o f the system or, equivalently, as a structure for realizing
the system .
If the system is to be im plem en ted in softw are, the block diagram or, eq u iv
alently, the set o f eq u ation s that are obtained by rearranging (7.1.1), can be co n
verted in to a program that runs on a digital com puter. A ltern atively, the structure
in block diagram form im plies a hardware configuration for im plem enting the
system .
Perhaps, the o n e issue that m ay not be clear to the reader at this point
is w hy w e are con sid erin g any rearrangem ents o f (7.1.1) or (7.1.2). W hy not
just im plem en t (7.1.1) or (7.1.2) directly w ithout any rearrangem ent? If either
(7.1.1) or (7.1.2) is rearranged in som e m anner, what are the benefits gained in the
corresponding im plem en tation ?
T h ese are the im portant q u estion s which are answ ered in this chapter. A t
this p oin t in our d ev elop m en t, w e sim ply state that the m ajor factors that influ
en ce our ch o ice o f a specific realization are com putational com p lexity, m em ory
requirem ents, and finite-w ord-length effects in the com putations.
C o m p u t a t i o n a l c o m p le x ity refers to the num ber o f arithm etic op eration s (m u l
tiplications, d ivisions, and additions) required to com p ute an output valu e y (n ) for
the system . In the past, th ese w ere the on ly item s used to m easure com putational
com p lexity. H o w ev er, w ith recent d evelop m en ts in the design and fabrication o f
rather sop h isticated program m able digital signal p rocessin g chips, oth er factors,

502

Implementation of Discrete-Time Systems

Chap. 7

such as the num ber o f tim es a fetch from m em ory is p erform ed or the num ber of
tim es a com parison betw een tw o num bers is perform ed per output sam ple, have
b ecom e im portant in assessing the com putational com p lexity o f a given realization
o f a system .
M e m o r y requ irem ents refers to the num ber o f memory locations required
to store the system param eters, past inputs, past outputs, and any interm ediate
com puted values.
F inite- wor d-length effects or finite-precision effects refer to the quantization
effects that are inherent in any digital im plem en tation o f the system , either in
hardware or in softw are. T h e param eters o f the system m ust necessarily be repre
sented with finite precision. T h e com p utations that are p erform ed in the process
o f com puting an output from the system must be rounded- o ff or truncated to fit
within the lim ited precision constraints o f the com p uter or the hardware used in
the im plem en tation . W hether the com p utations are perform ed in fixed-point or
floating-point arithm etic is an oth er consideration. A ll these p rob lem s are usually
called finite-w ord-length effects and are extrem ely im portant in influencing our
ch oice o f a system realization. W e shall see that different structures o f a system,
which are equivalent for infinite precision, exhibit different behavior when finiteprecision arithm etic is used in the im plem entation. T h erefore, it is very important
in practice to select a realization that is not very sensitive to finite-w ord-length
effects.
A lthou gh these three factors are the major o n e s in influencing our ch oice o f
the realization o f a system o f the type described by either (7.1.1) or (7.1.2), other
factors, such as w hether the structure or the realization len d s itself to parallel
processing, or w h eth er the com p utations can b e pip elined, m ay play a role in
our selection o f the specific im plem entation. T h ese additional factors are usually
im portant in the realization o f m ore com p lex digital signal processin g algorithm s.
In our discussion o f alternative realizations, w e con cen trate on the three
m ajor factors just outlined. O ccasionally, w e will include som e additional factors
that m ay be im portant in som e im plem entations.

7.2 STRUCTURES FOR FIR SYSTEMS

In general, an F IR system is described by the differen ce equation
Af-1

bkx (n ~ k)

y(n) =

(7.2.1)

*=0
or, equivalently, by the system function
A f-1

H{z)=*YibkZ~k
*=o

{122)

Furthermore, the unit sample response of the FIR system is identical to the coef-

Sec. 7.2

Structures for FIR Systems

503

ficients {>*}, that is,

,, ,

\ b,

'K,) = | o .

0 < n < Af 1

,7 , , ,

otherwise

<7-2 '3>

T h e length o f the F IR filter is selected as M to conform w ith the established

n otation in the technical literature.
W e shall present several m eth od s for im plem en ting an F IR system , b egin
ning w ith the sim p lest structure, called the direct form. A secon d structure is
the cascade-form realization. T h e third structure that w e shall d escribe is the
frequency-sam pling realization. Finally, w e present a lattice realization o f an FIR
system . In this discussion w e follow the con ven tion often used in the technical
literature, which is to use
for the param eters o f an F IR system .
In addition to the four realizations indicated ab ove, an F IR system can be
realized by m eans o f the D F T , as described in S ection 6.2. From on e point o f view,
the D F T can be considered as a com putational procedure rather than a structure
for an F IR system . H ow ever, when the com putational procedure is im plem ented
in hardw are, there is a corresponding structure for the F IR system . In practice,
hardware im plem en tation s o f the D F T are based on the use o f the fast Fourier
transform (FFT ) algorithm s described in C hapter 6 .

7.2.1 Direct-Form Structure

T he direct-form realization follow s im m ed iately from the nonrecursive difference
eq u ation given by (7.2.1) or, equivalently, by the con volu tion sum m ation
M- 1
y ( n ) = ^ h ( k ) x ( n - k)
t=o

(7.2.4)

T h e structure is illustrated in Fig. 7.1.

W e o b serve that this structure requires Af 1 m em ory locations for stor
ing th e Af 1 previous inputs, and has a com p lexity o f Af m ultiplications and
M 1 additions p er output point. Since the output con sists o f a w eigh ted linear
com b ination o f Af 1 past values o f the input and the w eigh ted current value o f
th e input, th e structure in Fig. 7.1, resem b les a tapped d elay line or a transversal

Figwre 7.1

Direct-form realization of FIR system.

504

Implementation of Discrete-Time Systems

Chap. 7

Figure 7.2 Direct-form realization of linear-phase FIR system (Af odd).

system . C on sequ en tly, the direct-form realization is often called a transversal or
tapped-delay-line filter.
W hen the F IR system has linear phase, as described in S ection 8.2, the unit
sam ple response o f the system satisfies either the sym m etry o r asym m etry condition

h(n) = h ( M - 1 - n)

(7.2.5)

For such a system the num ber o f m ultiplications is reduced from M to M f l for Af
even and to (Af l ) / 2 for M odd. For exam p le, the structure that takes advantage
o f this sym m etry is illustrated in Fig. 7.2 for the case in which M is odd.

7.2.2 Cascade-Form Structures

T he cascade realization follow s naturally from the system fun ction given by (7.2.2).
It is a sim ple m atter to factor H ( z ) into secon d-ord er F IR system s so that
K

0-2.6)

fl(z) = [] fl* ( z )
*=i
where
H k (z) = bk0 + bk]Z~l + bk2z~ 2

* = 1 ,2 .........K

(7.2.7)

and K is the in teger part o f ( M + l ) /2 . T h e filter param eter bo m ay be equally

distributed am ong the K filter section s, such that bo = biobw b Ka or it m ay be
assigned to a sin gle filter section . T h e zeros o f H ( z ) are grou p ed in pairs to pro
d uce the secon d-ord er F IR system s o f the form (7.2.7). It is alw ays desirable to
form pairs o f com p lex-con ju gate roots so that th e coefficien ts \bki} in (7.2.7) are
real valued. O n th e other hand, real-valued roots can b e paired in any arbitrary
m anner. T h e cascade-form realization alon g with th e basic secon d -ord er section
are sh ow n in Fig, 7.3.

Sec. 7.2

505

Structures for FIR Systems

jr(n)=jr,(n)
H](z)

>'[(") =

Vjr(n) = ,Y(n)

>'2<n) =
H2(z)

*2< )

jr,(n)
(a)

Figure 7.3 Cascade realization of an FIR system.

In the case o f linear-phase F IR filters, the sym m etry in h (n) im plies that the
zeros o f H ( z ) a lso exhibit a form o f sym m etry. In particular, if Zk and z*k are a pair
o f co m p lex-con ju gate zeros then 1 f z t and 1 / z \ are also a pair o f com plex-conjugate
zero s (see Sec. 8.2). C on sequ en tly, w e gain som e sim plification by form ing fourthorder section s o f the FIR system as follow s
Hk(z) = c ,( l - z / t z - K l - c ^ ' K l - z ~ l / z k )(\ - z - ' / z V ,
(7.2.8)
=

Q o + Q - l t " 1 + Ck2Z~ " + Q t Z - + Z ~ A

w here the coefficien ts {ct i } and (c^ f are functions o f zt- Thus, by com bining
the tw o pairs o f p o les to form a fourth-order filter section , w e have reduced the
n um ber o f m ultiplications from six to three (i.e., by a factor o f 50% ). Figure 7.4
illustrates the b asic fourth-order F IR filter structure.

Figure TA Fourth-order section in a

cascade realization o f an FIR system.

506

Implementation of Discrete-Time Systems

Chap. 7

7.2.3 Frequency-Sampling Structures*

T he frequency-sam pling realization is an alternative structure for an F IR filter
in which the param eters that characterize the filter are the valu es o f the desired
frequency resp onse instead o f the im pulse response h(n). T o d erive the frequencysam pling structure, w e specify the desired frequency response at a set o f equally
spaced frequencies, nam ely
2 tt
cot = (.k + a )
M

M - 1
A: = 0 , 1 , . . . , -
2

M od d

M
A: = 0, 1 , . . . , -------1

M even

a =

or j

and so lv e for the unit sam ple response h( n) from these equally spaced frequency
specifications. Thus we can write the frequency response as
M- 1
h ( n ) e ~ jum
n 0

and the values o f H(a>) at freq u en cies a>t = ( 2 n / M ) { k + a ) are sim ply
H (k + a ) = H

+ cr)^
(7.2.9)

h (n )e -j2*lk+aWM

k _ 0< 1____ A/ - 1

T he set o f values { //( -(-a )} are called the frequency sam ples o f H(a)). In the case
w here a = 0, j//(Jt)} corresponds to th e M -point D F T o f (A(n)}.
It is a sim ple m atter to invert (7.2.9) and express h(n) in term s o f the fre
quency sam ples. T he result is
i M- 1
h(n) = J ' H ( k + a ) e j2*(i+a)n/M

n = 0, 1 , . . . , M - 1

(7.2.10)

W hen a = 0, (7.2.10) is sim ply the ID F T o f {//(& )}. N ow if w e use (7.2.10) to

substitute for h (n) in the z-transform H ( z ) , w e have
u- \

H( z ) = > ( " ) ; "

n=0
M- 1

-L

i u - 1
Y \ H ( k + a ) e j2* il'H' )"/"
M *=0

(7.2.11)

^The reader may also refer to Section 8.2.3 for additional discussion o f frequency-sampling FIR
filters.

Sec. 7.2

507

Structures for FIR Systems

B y interchanging the order o f the tw o sum m ations in (7.2.11) and perform ing
the sum m ation over the index n w e obtain
I M-1
^ ^ej2n(i+a)/M - \ y

H( z ) = ^ H ( k + a)

(7.2.12)
} - Z- Mej2na y l
2. 1

H ( k + a)
_ e j2nik+a)/M z - 1

Thus the system function H ( z ) is characterized by the set o f frequency sam ples
(W(Jt-t-cr)j instead o f {h()).
W e v iew this F IR filter realization as a cascade o f tw o filters [i.e., H { z ) =
/ / i ( z ) / / 2 (s)]- O ne is an all-zero filter, or a com b filter, with system function
H x(z) = (1 - r wf y2l )
M

(7.2.13)

Its zeros are located at equally spaced points on the unit circle at
Zt =

jt = 0 , 1 .........A f - 1

T he second filter with system function

Af 1

H l i z ) 2 ^ -J _
Jt=0

j2n(t+a)/M

(7.2.1 )
*

consists o f a parallel bank o f sin gle-p ole filters with resonant frequencies
Pl = e}2* ik+ayM

k = 0 , l .........M -

N o te that the p o le locations are identical to the zero locations and that both
occur at a)k = 2 k (k + a ) / M , which are the freq u en cies at which the desired fre
quency resp onse is specified. T he gains o f the parallel bank o f resonant filters
are sim ply the com p lex-valu ed param eters \ H ( k + a )}. T h is cascade realization is
illustrated in Fig. 7.5.
W hen the desired frequency resp onse characteristic o f the F IR filter is nar
row band, m ost o f the gain param eters \ H ( k + a )) are zero. C on sequ en tly, the
corresponding resonant filters can be elim inated and on ly the filters with nonzero
gains n eed be retained. T h e n et result is a filter that requires few er com p uta
tion s (m ultiplications and additions) than the corresponding direct-form realiza
tion. Thus w e ob tain a m ore efficient realization.
T h e frequency-sam pling filter structure can be sim plified further by exploiting
the s y m m e t r y in H ( k + a ) , nam ely, H ( k ) = H * ( M k) for a = 0 and
H (* + i ) = H* ( M - k - | )

for a = \

T h ese relations are easily deduced from (7.2.9). A s a result o f this sym m etry, a
pair o f sin g le-p o le filters can b e com b ined to form a sin gle tw o -p o le filter with

508

Implementation of Discrete-Time Systems

Chap. 7

real-valued param eters. Thus for a = 0 the system function Hz (z) reduces to
H ( 0) ,
fiiiz) = t _ l +
H i 0)

^
t=1

A ik ) + B(lc)z- 1
----- 7
2 c o s { 2 n k / M ) z ~ ' + z~2

, H iM /2)

, lM^ ~ ]

H^ > = r r p r + T T P - +

A{k) + B { k ) z ~ x

2cos(2;rfc/Af)z -1 + z ~ 2

w
M odd

M even

(7.2.15)

Sec. 7.2

509

Structures for FIR Systems

w h ere, by definition,
A( k ) = H ( k ) + H ( M - k )

(7.2.16)

B( k) = H { k ) e ~ i2nklM + H ( M Sim ilar exp ression s can b e ob tain ed for a =

Example 7.2.1
Sketch the block diagram for the direct-form realization and the frequency-sampling
realization of the M = 32, a = 0, linear-phase (symmetric) F IR filter which has
frequency samples
1,

(f)-

* = 0 ,1 ,2

5-

4= 3

0,

* = 4 . 5 ........15

Compare the computational complexity of these two structures.

Solution Since the filter is symmetric, we exploit this symmetry and thus reduce the
num ber of multiplications per output point by a factor of 2, from 32 to 16 in the
direct-form realization. The number of additions per output point is 31. The block
diagram of the direct realization is illustrated in Fig. 7.6.
We use the form in (7.2.13) and (7.2.15) for the frequency-sampling realization
and drop all terms that have zero-gain coefficients |H(k)}. T he nonzero coefficients
are H(k) and the corresponding pairs are H( M - k), for k = 0 ,1 , 2, 3. The block
diagram of the resulting realization is shown in Fig. 7.7. Since H( 0) = 1, the single
pole filter requires no multiplication. The three double-pole filter sections require
three multiplications each for a total of nine multiplications. The total num ber of
additions is 13. Therefore, the frequency-sampling realization of this FIR filter is
computationally more efficient than the direct-form realization.

Figure 7.6

Direct-form realization of Af = 3 2 FIR filter.

510

Implementation of Discrete-Time Systems

Figare 7.7

Frequency-sampling realization for the FIR filter in Exam ple 7.2.1.

Chap. 7

Sec. 7.2

Structures for FIR Systems

511

7.2.4 Lattice Structure

In this sectio n w e introduce an oth er F IR filter structure, called the lattice filter or
lattice realization. Lattice filters are used exten sively in digital sp eech processing
and in the im plem en tation o f adaptive filters.
L et us b egin the d evelop m en t by considering a seq u en ce o f F IR filters with
system functions
Hm(z) = A m(z)

m = 0 . 1 , 2 .........M - 1

w here, by definition, A m(z) is the polyn om ial

m
A m(z) = 1 + a m(k)z~ k
*=i

m > 1

(7.2.17)

(7.2.18)

and /lo(^) = ! T he unit sam ple resp onse o f the m th filter is /j,(0) = 1 and
h m{k) = a m(k), k = 1, 2 , . . . , m. T he subscript m on the p olyn om ial Am(c) d en otes
the d egree o f the polynom ial. For m athem atical con ven ien ce, w e define a, (0) = 1.
If (*(n)} is the input seq u en ce to the filter A,{z) and (y(n )( is the output
seq u en ce, w e have
m
v() = -*() + ^ o r m(A:)A-(?f k)
(7.2.19)
*=l
T w o direct-form structures o f the F IR filter are illustrated in Fig. 7.8.

Figure 1A

Direct-form realization of the FIR prediction filter.

512

Implementation of Discrete-Time Systems

Chap. 7

In C hapter 11, w e show that the F IR structures sh ow n in Fig. 7.8 are inti
m ately related w ith the topic o f linear prediction, w here
m

x ( n ) = ~ ^ 2 a m( k )x (n - k)
*=i

(7.2.20)

is the o n e-step forward predicted value o f x (n), based on m past inputs, and
y ( n ) = x (n) x (n ), given by (7.2.19), rep resents the p red iction error sequ en ce.
In this context, the top filter structure in Fig. 7.8 is called a p re d ictio n er ror filter.
N o w suppose that w e have a filter of order m = 1. T h e ou tp u t o f such a filter
is
>-() = x ( n ) - f a i( l)j r ( n - 1)

(7.2.21)

This output can also be ob tain ed from a first-order or sin gle-stage lattice filter,
illustrated in Fig. 7.9, by exciting both o f the inputs by x ( n ) and selectin g the output
from the top branch. Thus the output is exactly (7.2.21), if w e select /li = ori (1).
T he param eter K i in the lattice is called a reflection coefficient and it is identical
to the reflection coefficient introduced in the S ch u r-C ohn stability test described
in Section 3.6.7.
N ext, let us consider an FIR filter for w hich m = 2. In this case the output
from a direct-form structure is
y (n ) = x ( n ) + ff;>(l)jr(/i -

) + ct2(2)x (n -

(7.2.22)

By cascading tw o lattice stages as show n in Fig. 7.10, it is p ossib le to obtain the

sam e output as (7.2.22). Indeed, the output from the first stage is

f i (fl) = -*(n) + K]X{n - 1)

(7.2.23)
i(n ) = K {x ( n ) + x ( n -

T h e output from the secon d stage is

flin ) = f\(n ) +

)
(7.2.24)

g 2(n) = K 2f i ( n ) + g i ( n - 1)

/o(") = &>(") =*()

/i() =/o(") + *io{n - 1) = *00 + X , x ( n

1)

j,{n) = tf,/oOO + SoO1 - 1) = fCix(n) + x(n - 1)

Figirc 7.9

Single-stage lattice filter.

Sec. 7.2

Structures for FIR Systems

513

Figure 7.10 Two-stage lattice filter.

If w e focus our atten tion on f 2{n) and substitute for f \ ( n ) and g\ ( n 1) from
(7.2.23) in to (7.2.24), w e obtain
f 2(n) = jr(n) + K \ x ( n - 1) +

- 1) + x ( n - 2)]
(7.2.25)

= x ( n ) + ^ i ( l + K 2) x( n - 1) + K 2x {n 2)
N o w (7.2.25) is identical to the output of the direct-form F IR filter as given by
(7.2.22), if w e eq u ate the coefficients, that is,
a 2 (2) = K 2

a 2( l) = ATi(l + K 2)

(7.2.26)

or, eq u ivalen tly,

K 2 = a 2(2)

Kx =

1 + a 2(2)

(7.2.27)

Thus the reflection coefficients K\ and K 2 o f the lattice can be ob tain ed from the
coefficien ts {am()} o f th e direct-form realization.
B y con tinu in g this process, o n e can easily d em onstrate by induction, the
eq u iv a len ce b etw een an m th-order direct-form FIR filter and an w -ord er or m stage lattice filter. T h e lattice filter is gen erally d escribed by the follow in g set o f
order-recursive equations:
/o (n ) = gain) = x ( n )

(7.2.28)

U i n ) = / ffl_ j( ) + K mgm- X{n - 1)

g m(n) = ^ / B- i W + i . - i ( i i - l )

m = 1 ,2 .........M - 1

(7.2.29)

m = 1,2,..., M 1

(7.2.30)

T h en the ou tp u t o f th e (A fl)-sta g e filter corresponds to the output o f an (A f1)order F IR filter, that is,
y (n ) =
Figure 7.11 illustrates an (Af - l)-sta g e lattice filter in b lock diagram form along
w ith a typical stage that show s the com p utations sp ecified by (7.2.29) and (7.2.30).
A s a co n seq u en ce o f the eq u ivalen ce b etw een an F IR filter and a lattice filter,
th e ou tp u t f m(n) o f an m -stage lattice filter can b e exp ressed as
m

f m( n) = <xm( k) x( n - k)
*=o

m(0) = 1

(7.2.31)

Sin ce (7.2.31) is a con v olu tion sum , it follow s that th e z-transform relationship is

Fm(z) = Am(z)X(z)

Implementation of Discrete-Time Systems

514

Chap. 7

(a)

Figure 7.11 (Af l)-stage lattice filter.

or, equivalently,
7}

F (?)

X(z)

Fo(z)

A m(z) =

(7.2.32)

T h e other output com p on en t from the lattice, nam ely, g m(n), can also be
exp ressed in the form o f a con volu tion sum as in (7.2.31), by using another set
o f coefficients, say {^m(Jt)}. T hat this in fact is the case, b ecom es apparent from
ob servation o f (7.2.23) and (7.2.24). From (7.2.23) w e n ote that the filter coeffi
cients for the lattice filter that produces / i ( n ) are {1 , Al} = { 1 , ofi( 1 )} w hile the
coefficients for the filter with output g \ ( n ) are (AT], 1} = | a i ( l ) , 1}. W e n ote that
these tw o sets o f coefficients are in reverse order. If w e con sid er the tw o-stage
lattice filter, with th e output given by (7.2.24), w e find that g 2 (n) can be expressed
in the form
g 2(n) = K 2f \ ( n ) + i(n - 1)
= K 2[x in) + K \ x { n 1)] + K \ x { n - 1) + x{ n 2)
= K 2x{ n ) + K \ ( l + K 2)x{n - 1) -I- x i n - 2)
= cr2 ( 2 )jc(n) + *2 ( 1 ) * (n -

) + xin -

C onsequently, the filter coefficients are {a 2 (2), a 2 ( l ) , 1}, w h ereas the coefficients
for the filter that produces the output f i i n ) are {1, a 2( \ ) , a 2 (2)}. H ere, again, the
tw o sets o f filter coefficients are in reverse order.
F rom this d ev elo p m en t it follow s that the ou tp u t gm{n) from an m -stage
lattice filter can b e exp ressed by the con volu tion sum o f the form
m

gmin) =

A *(*M - k)

(7.2.33)

Sec. 7.2

Structures for FIR Systems

515

w h ere the filter coefficients {& ,(*)} are associated with a filter that produces
f m(n) = y ( n ) but op erates in reverse order. C onsequently,
f}m (k) = a m (m k)

* = 0 , 1 .........m

(7.2.34)

with p m(m) = 1 .
In the co n tex t o f linear prediction, su p p ose that the data x(n), x{n - 1), . . . ,
x ( n m + 1) is u sed to linearly predict th e signal valu e x ( n m ) by u se o f a linear
filter w ith coefficients {fim(k)}. Thus the predicted value is
m-l
(7.2.35)
Since the data are run in reverse order through the predictor, the prediction per
form ed in (7.2.35) is called b a c k w a r d predictio n. In contrast, the F IR filter with
system function Am(z) is called a f o r w a r d predictor.
In the z-transform dom ain, (7.2.33) b ecom es
G m(z) = B m( z ) X ( z )

(7.2.36)

or, eq u ivalen tly,

(7.2.37)
w here Bm(z) represents the system function o f the F IR filter with coefficients
{& ,(*)}, that is,
(7.2.38)
4=0

Since fim(k) = a m(m *), (7.2.38) m ay b e exp ressed as

m

ffl
(7.2.39)
m

T h e relationship in (7.2.39) im plies that the zeros o f the F IR filter w ith system
function B m(z) are sim ply the reciprocals o f the zeros o f A ( z ) . H en ce B m(z) is
called the reciprocal or reverse p olyn om ial o f A m(z).
N o w that w e have estab lish ed th ese interesting relationships b etw een the
direct-form F IR filter and th e lattice structure, let us return t o th e recursive lattice
eq u a tio n s in (7.2.28) through (7.2.30) and transfer them to th e z-dom ain. Thus

516

Implementation of Discrete-Time Systems

Chap. 7

w e have
(7.2.40)

F0(z) = G 0(z) = X ( z )
Fm(z) = Fm_ i(z ) + J:mz - 1 Gm_ i(z )

m = 1, 2 , . . . , M - 1

(7.2.41)

G m(z) =

m =

(7.2.42)

, 2 , . . . , M 1

If w e divide each eq u ation by X (z), w e obtain the desired results in the form
(7.2.43)

A0(z) = B0(z) = 1
A m{z)

A m- X(z)

K z ~ } Bm- i ( z )

Bm(z) = K n A n - i W + z - ' B ^ i z )

(7.2.44)

m = l,2 ,...,M -l

(7.2.45)

m = 1 , 2 .........M -

T hus a lattice stage is described in the z-dom ain by the m atrix equation
(7.2.46)
B efo re concluding this discussion, it is desirable to d ev elo p the relationships
for converting the lattice param eters
that is, the reflection coefficients, to the
direct-form filter coefficients (a m(Jt)), and vice versa.

Conversion of lattice coefficients to direct-form filter coefficients. The

direct-form F IR filter coefficients {am(Jk)} can b e obtained from the lattice coeffi
cien ts {AT;} by using the follow ing relations:
(7.2.47)

A 0 (z) = B 0(z) = 1
A m(z) = A m- \ ( z ) + K mz ~ xBm~\(z)
Bm(z) = Z- mA m(z~ l )

m =

m = 1 ,2 .........M -

1,2- -

-M -

(7.2.48)
(7.2.49)

T h e solu tion is o b tain ed recursively, beginning with m = 1 . Thus w e obtain a

seq u en ce o f (M 1) F IR filters, o n e for each valu e o f m. T h e procedure is best
illustrated by m eans o f an exam ple.

Example 12J2
Given a three-stage lattice filter with coefficients AT] = j, K 2 =
the FIR filter coefficients for the direct-form structure.

, Ky =

, determine

Solution We solve the problem recursively, beginning with (7.2.48) for m = 1. Thus
we have
Ai(z) = A<z) + tfiZ ^ B oW
= 1 + K n - 1 = 1 + Jz "1
Hence the coefficients of an FIR filter corresponding to the single-stage lattice are
ori(0) = 1 , ari(l) = Ki = j. Since Bm(z) is the reverse polynomial of A(z), we have

Sec. 7.2

517

Structures for FIR Systems

Next we add the second stage to the lattice. For m = 2 , (7.2.48) yields
AjU) = ^ i(i) + KiZ~* B\(z)
1 4- J - -1 4- 1 ,- 2

1 + Rc T 2'-

Hence the FIR filter param eters corresponding to the two-stage lattice are *2(0) = 1,
2(1) = | , *2(2) =
Also,

*2(z) = J + i = +z-2
Finally, the addition of the third stage to the lattice results in the polynomial
M( z ) = * z (;)+ * :3 z _ ,ft(z )
= 1+

z _1 + %z~2 + | z -3

Consequently, the desired direct-form FIR filter is characterized by the coefficients

*3(0) = 1

a 3(l) = g

of3(2) = 2

o3{3) = i

A s this exam ple illustrates, the lattice structure with param eters K \ , K i .........
K m, corresp on d s to a class o f m direct-form F IR filters with system functions A\ (z),
y4i ( z) , . . . , A,(z). It is interesting to note that a characterization o f this class o f m
FIR filters in direct form requires m( m + l ) / 2 filter coefficients. In contrast, the
lattice-form characterization requires on ly the m reflection coefficients {A',}. T he
reason that the lattice provides a m ore com pact representation for the class o f m
F IR filters is sim ply du e to the fact that the addition o f stages to the lattice d o es
n ot alter the param eters o f the previous stages. O n the other hand, the addition
o f the mth stage to a lattice with ( m - 1 ) stages results in a F IR filter with system
function A m(z) that has coefficients totally d ifferent from the coefficients o f the
low er-order F IR filter with system function Am_ i(z ).
A form ula for determ ining the filter coefficients [am(Jt)} recursively can be
easily d erived from p olyn om ial relationships in (7.2.47) through (7.2.49). From the
relationship in (7.2.48) w e have
Am(z) = Am_ i(z ) + K mz ~ l B m- i ( z )
m
_i
m-t
^ a m (k ) z ~ k = ^ a m-] (k)z~ k + K m ^ a m- \ ( m - 1 - k ) z (i+1)
k=0

k=0

(7.2.50)

*=0

B y eq u atin g the coefficients o f equal pow ers o f z - 1 and recalling that a m(0) = 1
for m = 1, 2 .........M - 1, w e obtain the desired recursive eq u ation for the F IR filter
coefficients in the form
*m(0) = 1

(7.2.51)

a m(m) = K m
a m(k) = ctm- \ ( k ) +

(7.2.52)
- *)

= *mi(k) + a m(rn)am-i(rn - k)

(7-2. 53)

Implementation of Discrete-Time Systems

518

Chap. 7

W e n o te that (7.2.51) through (7.2.53) are sim ply the L ev in so n -D u rb in recursive

equations given in Chapter 11.

Conversion of direct-form FIR filter coefficients to lattice coefficients.

S u p pose that w e are given the F IR coefficien ts for the direct-form realization or,
equivalently, the polyn om ial A m(z), and w e w ish to determ in e the corresponding
lattice filter param eters { ,} . For the m -stage lattice w e im m ed iately obtain the
param eter K m = a m (m). T o obtain K m- \ w e n eed the p olyn om ials Am_ i(z ) since,
in general, K m is ob tain ed from the p olyn om ial A m (z) for m = M 1, Af 2 , . . . , 1.
C onsequently, w e need to com p ute the p olyn om ials A m(z) starting from m = Af 1
and stepping d o w n su ccessively t o m = 1 .
T h e d esired recursive relation for the polyn om ials is easily determ ined from
(7.2.44) and (7.2.45). W e have
A m(z) = A m- \ ( z ) + K mz ~ x B m- \ { z )
= A m- \ ( z ) + K m[Bm(z) - ^ mAm_ i( ;) ]
If w e solve for Am_i (z), w e obtain
A m- i ( z ) =

m = M - h M - 2 .........1

-*"*"<*>

(7.2.54)

which is just the step-dow n recursion used in the S ch u r-C ohn stability test d e
scribed in S ection 3.6.7. T hus w e com p ute all low er-d egree p olyn om ials A m(z)
beginning with A M- \ ( z ) and obtain the desired lattice coefficien ts from the rela
tion K m = a m(m). W e ob serve that th e procedure works as lon g as \ Km \
1 for
m = 1, 2 .........A f - 1 .
Example 7.23
Determ ine the lattice coefficients corresponding to the FIR filter with system function
H(z) = A3(z) = 1 -ISolution

First we note that

# 3 (z) =

-I- f z 2 -I- z-J

= a 3(3) = | . Furtherm ore,

5

+ jU- 1 +

z~2 + z~y

The step-down relationship in (7.2.54) with m = 3 yields

A2(z) =

A3(z) - K i B ^ z )
1- K

= l + f z - ^ z1 , - 2
Hence K2 a 2(2) = 5 and B i (z) =
recursion in (7.2.51), we obtain
, , ,

+ | z _1 + z_1- By repeating the step-down

A i b ) ~ K2B2(z )

Sec. 7.3

519

Structures for HR Systems

From the step -d ow n recursive eq u ation in (7.2.54), it is relatively easy to

obtain a form ula for recursively com puting K m, begin n ing with m = M 1 and
stepping dow n to m = 1. For m = M 1, M 2 , . . . , 1 w e h ave
K m = a m( m )

Qfml (0) = 1

(7.2.55)

a m(k) - K mfim(k)

J __ g 2
a m(k) -

otm ( r n ) a m ( m

- k)

1 -al(m )

1 5 k < m 1

(7.2.56)

w hich is again the recursion w e introduced in the S ch u r-C oh n stability test.

A s indicated ab ove, the recursive eq u ation in (7.2.56) breaks dow n if any
lattice param eters | Armj = 1. If this occurs, it is ind icative o f the fact that the
polyn om ial A m- \ ( z ) has a root on the unit circle. Such a root can b e factored out
from Am_](z) and the iterative p rocess in (7.2.56) is carried out for th e reducedorder system .

7.3 STRUCTURES FOR IIR SYSTEMS

In this section w e consider different IIR system s structures d escribed by the dif
ference eq u ation in (7.1.1) or, eq u ivalen tly, by the system function in (7.1.2). Just
as in the case o f F IR system s, there are several types o f structures or realizations,
including direct-form structures, cascade-form structures, lattice structures, and
lattice-ladder structures. In addition, IIR system s lend th em selv es to a parallelform realization. W e begin by describing tw o direct-form realizations.

7.3.1 Direct-Form Structures

T h e rational system function as given by (7.1.2) that ch aracterizes an IIR system
can be v iew ed as tw o system s in cascade, that is,
H ( z ) = H i (z ) H 2( z )

(7.3.1)

w h ere Hi (z) con sists o f th e zeros o f H ( z ) , and H z (z ) con sists o f the p o les o f H {z),
M
H l (z ) = Y ! f bkZ~k
*=o

(7-3.2)

H 2 ( z ) = ------- -- ---------

(7.3.3)

and

1+ X > z"*
*=i
In S ection 2.5.1 w e describe tw o different direct-form realizations, character
ized by w h eth er H \{z) p reced es H j i z ) , or vice versa. Since H \ ( z ) is an F IR system ,
its direct-form realization w as illustrated in Fig. 7.1. B y attaching th e all-p ole

520

Implementation of Discrete-Time Systems

All-zero system

Chap. 7

All-pole system

Figure 7.12 Direct form I realization.

system in cascade with H\ ( z) , w e obtain the direct form I realization d epicted in
Fig. 7.12. T his realization requires M + N + 1 m ultiplications, M + N additions,
and M + N + 1 m em ory locations.
If th e all-p ole filter f y i z ) is placed b efore th e all-zero filter H i(z), a m ore
com pact structure is ob tain ed as illustrated in S ection 2.5.1. R ecall that the differ
en ce eq u ation for th e all-pole filter is
N

w(n) = - ^

ai(W{n - k) + jr(n)

(7.3.4)

Since w ( n ) is the input to the all-zero system , its ou tp u t is

M
yW =

bk w (n - k)

(7.3.5)

*=o
W e n o te that both (7.3.4) and (7.3.5) in volve d elayed versions o f the sequ en ce
(u>(n)}- C on sequ en tly, on ly a single d elay line or a sin gle set o f m em ory locations
is required for storing the past values o f {u>(n)}. T h e resulting structure that
im plem en ts (7.3.4) and (7.3.5) is called a direct form II realization and is depicted
in Fig. 7.13. T his structure requires M + N + 1 m ultiplications, M + N additions,

Sec. 7.3

Structures for IIR Systems

Figure 7.13

D irect form II realization

521

(N

Af).

and the m axim um o f {M, N } m em ory locations. Since the direct form II realization
m inim izes the num ber o f m em ory locations, it is said to b e canonic. H ow ever, w e
should indicate that other IIR structures also p ossess this property, so that this
term in ology is perhaps unjustified.
T h e structures in Figs. 7.12 and 7.13 are both called direct form realiza
tions b ecau se th ey are ob tain ed directly from the system function H ( z ) w ithout
any rearrangem ent o f H ( z ) . U n fortu n ately, both structures are extrem ely sen si
tive to param eter quantization, in gen eral, and are not recom m en d ed in practical
applications. T his to p ic is discussed in detail in Section 7.6, w h ere w e dem onstrate
that w h en N is large, a sm all change in a filter coefficient d u e to param eter quan
tization, results in a large change in the location o f th e p o les and zeros o f the
system .

7.3.2 Signal Flow Graphs and Transposed Structures

A signal flow graph provides an alternative,Nbut eq u ivalen t, graphical rep resen
tation to a b lock diagram structure that w e have b een using to illustrate various
system realization s. T h e basic elem en ts o f a flow graph are branches and n od es.
A signal flow graph is basically a set o f directed branches that con n ect at n od es.
B y d efinition , th e signal ou t o f a branch is equal to th e branch gain (system func
tio n ) tim es the signal in to the branch. Furtherm ore, the signal at a n od e o f a
flow graph is eq u al to the sum o f the signals from all branches con n ectin g to the
node.
T o illustrate th e se basic n otion s, let us con sid er the tw o -p o le and tw o-zero
IIR system d ep icted in b lock diagram form in Fig. 7.14a. T h e system block

522

Implementation of Discrete-Time Systems

Chap. 7

(a)
Source node

Sink node

5
(b)

Fignre 7.14 Second-order filter structure (a) and its signal flow graph (b).
diagram can be converted to the signal flow graph show n in F ig. 7.14b. W e note
that the flow graph contains five n od es lab eled 1 through 5. T w o o f the nodes
( 1 ,3 ) are sum m ing n od es (i.e., they contain ad d ers), w hile the other three nodes
represent branching points. Branch transm ittances are ind icated for the branches
in the flow graph. N o te that a d elay is indicated by the branch transm ittance
z- 1 . W h en the branch transm ittance is unity, it is left u n lab eled . T h e input to
the system originates at a so ur ce n o d e and the ou tp u t signal is extracted at a sink
node.
W e observe that the signal flow graph con tains th e sam e b asic inform ation
as the block diagram realization o f the system . T h e on ly ap p aren t d ifference is
that b o th branch p oints and adders in the b lock diagram are rep resen ted by nodes
in th e signal flow graph.
T h e subject o f linear signal flow graphs is an im portant o n e in the treatm ent
o f netw ork s and m any interesting results are available. O n e b asic n otion involves
the transform ation o f o n e flow graph in to a n oth er w ithout changing the basic
in p u t-ou tp u t relationship. Specifically, o n e tech n iq u e that is usefu l in deriving
new system structures for F IR and IIR system s stem s from th e transposition or
flo w - g ra p h reversal theorem. T his th eorem sim ply states that if w e reverse the

Sec. 7.3

523

Structures for IIR Systems

directions o f all brancb transm ittances and interchange the input and output in
the flow graph, the system function rem ains unchanged. T h e resulting structure is
called a transposed s tructure or a tran sp osed f o r m .
F or exam p le, the transposition o f the signal flow graph in Fig. 7.14b is illus
trated in Fig. 7.15a. T h e corresponding b lock diagram realization o f the transposed
form is d ep icted in Fig. 7.15b. It is interesting to n ote that the transposition o f the
original flow graph resulted in branching n od es b ecom in g ad d er n od es, and vice
versa. In Section 7.5 w e p rovid e a p ro o f o f the transposition theorem by using
state-sp ace techniques.
L et us apply the transposition theorem to the direct form II structure. First,
w e reverse all the signal flow d irections in Fig. 7.13. Second, w e change n od es
into adders and adders into n od es, and Anally, w e interchange the input and the
output. T h ese op eration s result in the transposed direct form II structure show n
in Fig. 7.16. T his structure can b e redrawn as in Fig. 7.17, which show s the input
on the left and the output on the right.

~ ai

s^\

(b)

Figure 7.15 Signal flow graph of

transposed structure (a) and its
realization (b).

524

Implementation of Discrete-Time Systems

Chap. 7

Figure 7.16 Transposed direct form II

structure.
T h e transposed direct form II realization that w e have ob tain ed can be d e
scribed by the set o f d ifference equations
(7.3.6)

y ( n ) = wr f n - l ) + b o x ( n )

Wt(n) = wt+i(n - 1)

+ b kx(n)

Jt = 1 ,2 ......... N - 1

w N(n ) = bNx(n) - a Ny(n)

(7.3.7)
(7.3.8)

W ithout loss o f generality, w e have assum ed that Af = W in w riting equ ation s. It

is also clear from observation o f Fig. 7.17 that this set o f d ifferen ce eq u ation s is
equ ivalen t to the sin gle differen ce equation

y ( n ) = ~ ^ 2 a ky (n - k ) + ^ b kx ( n - k )

(7.3.9)

Sec. 7.3

525

Structures for IIR Systems

Figure 7.17 Transposed direct form II

structure.

Finally, w e o b serv e that th e transposed direct form II structure requires the sam e
n um ber o f m ultip lication s, additions, and m em ory locations as the original direct
form II structure.
A lth o u g h our discussion o f transposed structures has b een concerned with
the general form o f an IIR system , it is interesting to n o te that an F IR system ,
ob tain ed from (7.3.9) by setting the a* = 0, k = 1, 2 , . . . , N , also has a transposed
direct form as illustrated in Fig. 7.18. T his structure is sim ply ob tain ed from
Fig. 7.17 by settin g a* = 0, k = 1, 2 , . . . , N . T his transposed form realization m ay

Figure 7.18

Transposed FIR structure.

526

Implementation of Discrete-Time Systems

Chap. 7

be described by the set o f differen ce eq u ation s

w M(n) = b t f x ( n )

(7.3.10)

u>*(/i) = u>*+i(n 1) + bkx(n)

k = M 1, M ~ 2, . . . , 1

y ( n ) = w i ( n - 1) +/>&*()

(7.3.11)
(7.3.12)

In sum m ary, T ab le 7.1 illustrates the direct-form structures and the corresponding
d ifference eq u ation s for a basic tw o-p ole and tw o-zero IIR system with system
function
x
b0 + b i z ~ l + b 2z ~ 2
H ( z ) = ----------- ------------ 3 -

1 + fli z 1 +azz 1

(7.3.13)

This is th e basic building block in the cascade realization o f h igh-order IIR system s,
as described in the follow in g section. O f the three direct-form structures given in
T ab le 7.1, th e direct form II structures are p referable due to the sm aller number
o f m em ory locations required in their im plem entation.
Finally, w e n o te that in the z-dom ain, the set o f d ifferen ce eq u ation s describ
ing a linear signal flow graph con stitute a linear set o f equations. A n y rearrange
m ent o f such a set o f eq u ation s is equ ivalen t to a rearrangem ent o f the signal flow
graph to obtain a n ew structure, and vice versa.

7.3.3 Cascade-Form Structures

Let us con sid er a high-order IIR system with system function given by (7.1.2).
W ithout loss o f generality w e assum e that N > M . T h e system can be factored
into a cascade o f secon d-ord er subsystem s, such that H (z) can b e exp ressed as
K

*=i
w here K is the in teger part o f (N + l ) /2 . Hk (z) has th e general form
rr , ,
bto + b u z ' 1 + bk2Z ' 2
Hk (z) = --------------------- 3 5 1 + fljtiz 1 + a k2z 2

(7.3.15)

A s in the case o f F IR system s b ased on a cascad e-form realization, the param eter
bo can be distributed equally am ong the K filter se ctio n s so that bo = >1 0 ^ 2 0 bxoT he coefficients {a*j} and {*>*, }in the secon d -ord er su b system s are real. This
im plies that in form ing th e secon d-ord er su b system s or quadratic factors in (7.3.15),
w e should group to g eth er a pair o f com p lex-con ju gate p o les and w e should group
togeth er a pair o f com p lex-con ju gate zeros. H ow ever, th e pairing o f tw o com plexconjugate p o les with a pair o f com p lex-con ju gate zeros or real-valu ed zeros to form
a subsystem o f the type given by (7.3.15), can b e d o n e arbitrarily. Furtherm ore,
any tw o real-valued zeros can b e paired to g eth er to form a quadratic factor and,
likew ise, any tw o real-valued p o les can b e paired togeth er to form a quadratic
factor. C on sequ en tly, th e quadratic factor in th e num erator o f (7.3.15) m ay consist

TABLE 7.1

SOME SECOND-ORDER MODULES FOR DISCRETE-TIME SYSTEMS

Structure

Implementation Equations

System Function

xin)

v(n) = fc().t(n) +fc]X(n - I)

-I- i>2.t(rt - 2)
- oi v(n - 1) <t2\(n 2)

H( z) =

bft + bjZ 1 +biz 2

1 + a u -1 +aiz~2

H( Z) =

bp + bjZ 1 +bjz
1 + a u _1 +aiz~2

H( Z)

bp -t-fri? 1 +bjz 2
1 + c u - * +aiz~2

x<n)

u.( n )

~ d i w ( n I ) a 2 w ( n 2)

+ x(n)

,v(n) = bow(n) -+- fcj utr( 1)

+ b2U)(n 2)
x<n)

v(n) = bu.x(n) + wt (n - 1)
= ht x( n) ai_v(n)
-t- w 2 ( n -

527

w 2 (n)

b ix(n) -

1)

a 2y ( n )

528

Implementation of Discrete-Time Systems

Chap. 7

o f either a pair o f real roots or a pair o f com p lex-con ju gate roots. T h e sam e
statem ent applies to the den om in ator o f (7.3.15).
If N > M , so m e o f the secon d-ord er subsystem s h ave num erator coefficients
that are zero, that is, eith er bk2 = 0 or bk\ = 0 or b oth bk2 = bk\ = 0 for som e k. Fur
therm ore, if N is odd, o n e o f the subsystem s, say Hk (z), m ust h ave a k2 = 0, so that
the subsystem is o f first order. T o p reserve the m odularity in th e im plem en tation
o f H ( z ) , it is o ften preferable to use the basic secon d-ord er su b system s in the cas
cade structure and h ave som e zero-valu ed coefficients in so m e o f the subsystem s.
E ach o f the secon d-ord er subsystem s with system function o f the form (7.3.15)
can be realized in eith er direct form I, or direct form II, or tran sp osed direct form
II. Since there are m any w ays to pair the p oles and zeros o f H (z ) into a cascade
o f secon d-ord er section s, and several w ays to order the resulting subsystem s, it is
p ossib le to obtain a variety o f cascade realizations. A lth ou gh all cascade realiza
tions are equ ivalen t for infinite precision arithm etic, the various realizations may
differ significantly w hen im plem en ted with finite-precision arithm etic.
T he general form o f th e cascade structure is illustrated in Fig. 7.19. If we
use the direct form II structure for each of the subsystem s, the com putational
algorithm for realizing the IIR system with system function H ( z ) is described by
the follow in g set o f equations.
>>o(n) = x ( n )

(7.3.16)

w k(n) = - a ki w k(n - 1) - a k2w k (n - 2) + y*_i(n)

k = 1 ,2 ...........K

(7.3.17)

y k(n) = bk0w k(n) + bt i w k(n - 1) -I- bk2w t (n - 2)

k = 1 ,2 ..........K

(7.3.18)

y (n ) = ytcin)

(7.3.19)

(a)
**0

-*l

S ~

> '* ( * ) = X t + 1 ( n )

^ -0

0 -

(b)

Ftg*rc 7.19 Cascade structure of second-order systems and a realization of each

second-order section.

Sec. 7.3

529

Structures for IIR Systems

Thus this set o f eq u ation s p rovides a com p lete description o f the cascade structure
based on direct form II section s.

7.3.4 Parallel-Form Structures

A parallet-form realization o f an IIR system can be ob tain ed by p erform ing a
partial-fraction expansion o f H( z ) . W ithout loss o f generality, w e again assum e
that N > M and that the p o le s are distinct. T hen, by perform ing a partial-fraction
expansion o f H( z ) , w e obtain the result
N
Ai
H ( z ) = C + V ------- r
t t 1 - P*z~'

(7.3.20)

w here {p*} are the p oles, {A t \are the coefficients (residues) in the partial-fraction
exp an sion , and the constant C is defined as C = b s / a s i . T h e structure im plied
by (7.3.20) is sh ow n in Fig. 7.20. It consists o f a parallel bank o f sin gle-p ole
filters.
In gen eral, som e o f the p oles o f H ( z ) may be com plex valued. In such a case,
the corresp on d ing coefficients A t are also com plex valued. T o avoid m ultiplica
tions by com p lex num bers, w e can com b ine pairs o f com p lex-con ju gate p oles to
form tw o -p o le subsystem s. In addition, w e can com b ine, in an arbitrary m anner,

Figure 7.20

Parallel structure of IIR system.

530

Implementation of Discrete-Time Systems

Figure 7*21

Chap. 7

Structure of second-order section in a parallel IIR system realization.

pairs o f real-valued p o les to form tw o-p ole subsystem s. Each o f th ese subsystem s
has the form
Hk{z) =
1

(7.3.21)

+ a kiz ] + a k2r

w here the coefficients {bk,} and (at;] are real-valued system param eters. T he over
all function can n ow be expressed as

(7.3.22)

H( z) = C + J 2 h*<*>

w here K is the in teger part o f ( N + 1)/2. W hen N is odd, on e o f the H k (z) is really
a sin gle-p ole system (i.e., bk\ = a k 2 = 0 ).
T h e individual second-order section s which are th e basic b uilding blocks for
H ( z ) can be im plem en ted in eith er o f the direct form s or in a transposed direct
form. T h e direct form II structure is illustrated in Fig. 7.21. W ith this structure as
a basic building block, the parallel-form realization o f the F IR system is described
by the follow in g set o f equations

wk(n) = aki w k(n - 1) - ak2wk(n - 2) + x(n)

* = 1,2,...,

(7.3.23)

yt(n) = bk0wk(n ) + bktwt (n - 1)

k = 1 ,2 .........K

(7.3.24)

K
y(n) = Cx ( n ) + ^ y t ( n )

(7.3.25)

Example 7.3.1
Determine the cascade and parallel realizations for the system described by the system
function
10(1 -

H(z) =

1)(1 l z - 1 ) ( l +

z-1)

(1 - |z _1)(l - 5Z-1)[1 - (j + ; j ) z -1][l - (5 -

Sec. 7.3

Structures for IIR Systems

531

Solution The cascade realization is easily obtained from this form. One possible
pairing of poles and zeros is
1 - lz~*
= i _ 2 .- i + 2, z- 2
1

8'

32

l + l z - '- z " 2
H2<c) = i1 - z ~ l
+ \rz~~2;
and hence
H(z) = 10//,(z)Jfe(z)
The cascade realization is depicted in Fig. 7.22a.
To obtain the parallel-form realization, H(z) must be expanded in partial frac
tions. Thus we have
Ai
Ai
" (2 ) =
+
1-fz-1

1-iz-1

i - ( i + yi)z-i

i - ( l - y i );-i

where j4t , A 2, A 3, and A j are to be determ ined. A fter some arithmetic we find that
A { = 2.93,

A, = -17.68,

A3 = 12.25 - yl4.57,

A\ = 12.25 + >14.57

upon recom bining pairs of poles, we obtain

% 14.75 12.90z_l
24.50 + 26.82;"'
;----- 5 i I- 1 _ -- I -L J.--2
H (z) -------1
1 _ Z--1 _1_ 2 . 7 - 1
x*- + 324
*
2^
The parallel-form realization is illustrated in Fig. 7.22b.

7.3.5 Lattice and Lattice-Ladder Structures for IIR

Systems
In Section 7.2.4 w e d ev elop ed a lattice filter structure that is eq u ivalen t to an F IR
system . In this sectio n w e exten d the d evelop m en t to IIR system s.
L et us b egin with an all-p ole system with system function
H (z) = ------- ^ ----------- =
A
t
a n (z )
1+ ^cis(k)z
k=l

(7. 3. 26)

T h e direct form realization o f this system is illustrated in Fig. 7.23. T h e difference

eq u ation for this IIR system is
N

y( n ) = ~ ^ a N (k) y( n - k) + x { n)
*=i

(7.3.27)

It is in terestin g to n ote that if w e interchange the roles o f input and output

[i.e., interchange x ( n ) w ith y(n ) in (7.3.27)], w e obtain

x(n) = - ^ T a N(k)x(n - k) 4- y(ri)

Implementation of Discrete-Time Systems

532

Chap. 7

(a)

(b)
Figure 7.22 Cascade and parallel realizations for the system in Example 7.3.1.
or, equivalently,
N

y ( n ) = x ( n ) 4- ^ a w(fc)*(n - k)
*=i

(7.3.28)

W e n o te that the eq u ation in (7.3.28) describes an F IR system having the

system function H ( z ) = A N (z), w h ile the system describ ed by th e differen ce equa
tion in (7.3.27) rep resen ts an IIR system with system function H ( z ) = lM w(z)*

Sec. 7.3

533

Structures for IIR Systems

yi.n)

*()

Figure 7.23 Direct-form realization of an all-pole system.

O n e system can b e o b tain ed from the other sim ply by interchanging th e roles o f
th e input and output.
B ased on this o b servation , w e shall use the all-zero (F IR ) lattice describ ed in
S ection 7.2.4 to ob tain a lattice structure for an all-p ole IIR system by interchanging
the ro les o f the input and output. First, w e take the all-zero lattice filter illustrated
in Fig. 7.11 and th en redefine the input as
x ( n ) = f N(n)

(7.3.29)

v(n) = fo(n)

(7.3.30)

and the ou tp u t as

T h ese are exactly the o p p osite o f the definitions for the all-zero lattice filter. T h ese
d efinitions d ictate that the qu an tities { / m(n)l be com p uted in d escen din g ord er [i.e.,
/ v ( n ) , / v _ i ( n ) , . . . ) . T h is com p utation can be accom plished by rearranging the
recursive eq u ation in (7.2.29) and thus solvin g for / m_\ (n) in term s o f f m(n ), that is,
K mgm- i ( n - 1 )

/ m - i ( n ) = f m( n) -

m =

N, N - 1 ,..., 1

T h e eq u a tio n (7.2.30) for gm(n) rem ains unchanged.

T h e result o f th ese ch an ges is th e set o f equations
f s ( n ) = x (n)

(7.3.31)

f m - \ ( n ) = f m(n) - K mgm. ] ( n - 1)
gm(n) = K mf m- i ( n ) + g m- \ ( n - 1)

m = N , N - 1,.

(7.3.32)

m = N , N - 1,

(7.3.33)
(7.3.34)

y ( n ) = /o (n ) = go(n)
w hich corresp on d to the structure sh ow n in Fig. 7.24.
Input

Figure 7.24

Lattice structure for an all-pole

IIR system.

Implementation of Discrete-Time Systems

534

Chap. 7

T o d em onstrate that the set o f eq u ation s (7.3.31) through (7.3.34) represent

an all-pole IIR system , let us con sid er th e case w h ere N = 1. T h e eq u ation s
reduce to
* (n ) = /i( n )

fain) = h i n ) - K\goin - 1)
g i(n ) = ATi/o(n) + go(n - 1)

(7.3.35)

?() = foin)
= xin ) - K \yin - 1 )
Furtherm ore, the eq u ation for g i(/i) can be exp ressed as
i(n ) = ATi^n) + y in - 1)

(7.3.36)

W e observe that (7.3.35) represents a first-order all-p ole IIR system w hile (7.3.36)
represents a first-order F IR system . T h e p o le is a result o f the feed b ack introduced
by the solution o f th e ( / m(n)} in descen din g order. T his feed b ack is dep icted in
Fig. 7.25a.
Forward

(a)
Forward

Reverse
(b)

Figure 125

Single-pole and two-pole lattice system.

Sec. 7.3

535

Structures for IIR Systems

N ex t, let us con sid er the case N = 2, which corresponds to the structure in

Fig. 7.25b. T h e eq u ation s corresponding to this structure are
f 2(n) = x (n)
f i ( n ) = f 2(n) - K 2g i(n - 1)
g2(n) = K 2f i ( n ) + g i(n -

)
(7.3.37)

/o (n ) = / i ( n ) - K igo in - 1 )
gi (n) = K ifo (n ) + g o ( n - l )
y (n ) = /o (n ) = go()
A fter so m e sim ple substitutions and m anipulations w e obtain
y (n ) =

+ K 2)y (n 1) - K 2y (n - 2) + x ( n )

g2(n) = K 2v(n ) + K XQ + K 2)y (n - 1) + v(n - 2)

(7.3.38)
(7.3.39)

C learly, the differen ce equation in (7.3.38) represents a tw o-p ole IIR system , and
the relation in (7.3.39) is the in p u t-ou tp u t equation for a tw o-zero F IR system .
N o te that the coefficients for the F IR system are identical to th ose in the IIR
system excep t that they occu r in reverse order.
In general, th ese con clu sion s hold for any N . Indeed, with the definition of
Am( i) given in (7.2.32). the system function for the all-p ole IIR system is
, ,
Y (z)
Fo(z)
1
Ha (z) = -------= ----------= ---------X (z )
Fm(z)
Am( z)

7 , ,,
(7.3.40)

Sim ilarly, the system function o f the all-zero (F IR ) system is

H b(z) =

Y (z )

G 0(Z)

= B m(z) = z~ mA m( z ~ l )

(7.3.41)

w h ere w e used the p reviously established relationships in (7.2.36) through (7.2.42).

Thus the coefficients in the F IR system H b(z) are identical to the coefficients in
Am(z), excep t that they occur in reverse order.
It is interesting to n o te that the all-p ole lattice structure has an all-zero path
with input go(n) and output g x ( n ), w hich is identical to its counterpart all-zero
path in th e all-zero lattice structure. T h e polynom ial Bm (z), w hich represents the
system function o f the all-zero path com m on to b oth lattice structures, is usually
called the b a c k w a rd sy stem fu n c tio n , b ecau se it provid es a backward path in the
a ll-p ole lattice structure.
From this discussion th e reader should ob serve that the all-zero and all-pole
lattice structures are characterized by th e sam e set o f lattice param eters, nam ely,
K \, K 2, . . K n . T h e tw o lattice structures differ on ly in the in tercon n ection s o f
their signal flow graphs. C on sequ en tly, the algorithm s for con vertin g b etw een the
system param eters (a m(fc)} in th e direct form realization o f an F IR system , and the
p aram eters o f its lattice counterpart apply as w ell to the all-p ole structure.

536

implementation of Discrete-Time Systems

Chap. 7

W e recall that the roots o f the polynom ial A N (z) lie in sid e th e unit circle if
and o n ly if the lattice param eters | ^ m| < 1 for all m = 1, 2 .........N . T h erefore, the
all-p o le lattice structure is a stable system if and on ly if its param eters \K m\ < 1
for all m.
In practical applications the all-pole lattice structure has b een used to m odel
the hum an vocal tract and a stratified earth. In such cases the lattice param eters,
{K m) have the physical significance o f b ein g identical to reflection coefficients in
the physical m edium . T his is the reason that the lattice param eters are o ften called
reflection coefficients. In such applications, a stable m od el o f the m edium requires
that the reflection coefficients, obtained by perform ing m easu rem en ts o n output
signals from the m edium , b e less than unity.
T h e all-p ole lattice provides the basic b uilding block for lattice-type structures
that im plem en t IIR system s that contain both p oles and zeros. T o d evelop the
appropriate structure, let us consider an IIR system with system function
M

H (z ) = -------------------------=
A
,
1 + J 2 a N(k)z ~ L
k= 1

* n (z )

(1 3 A 2 )

w here the notation for the num erator polynom ial has b een changed to avoid con
fusion with our previous develop m en t. W ithout loss o f generality, w e assum e that
N > M.
In the direct form II structure, the system in (7.3.42) is described by the
difference equations
N

w (n ) = ^ a tf(fc )u > (n k) + x (n )
*=i
M
y (n ) = c M(fc)w(n - k)
*=o

(7.3.43)

(7.3.44)

N o te that (7.3.43) is the in p ut-ou tp u t o f an all-p ole IIR system and that (7.3.44) is
the in p u t-o u tp u t o f an all-zero system . Furtherm ore, w e ob serve that the output of
the all-zero system is sim ply a linear com bination o f delayed ou tp u ts from the all
p o le system . This is easily seen by observing th e direct form II structure redrawn
as in Fig. 7.26.
Since zeros result from form ing a linear com bination o f previous outputs we
can carry o ver this observation to construct a p o le -z e r o IIR system using the all
p ole lattice structure as the basic building block. W e have already observed that
gm(n ) is a linear com bination o f present and past outputs. In fact, the system
Hh(z) =

= Bm(z)
Y (z)

Sec. 7.3

537

Structures for IIR Systems

0>

^ - 0 - ------0 -------0
w (n )

w(n - 1 )

w(n - 2)

w(n - Af + 1) ------

w ( n - M)

t - M ------------- ------

e 0 )

c*<2 )

cu ( M -

1)

c^M)
yirt)

- 0 - -------0 ------- 0 ^

Figure 7.26 D irect form II realization of IIR system.

is an all-zero system . T h erefore, any linear com bination o f {gm(n)} is also an

all-zero system .
Thus w e begin with an all-p ole lattice structure with param eters K m, 1 <
m < N , and w e add a la dder part by taking as the output a w eigh ted linear
com bination o f ( * (n )}. T h e result is a p o le-zero IIR system w hich has the latticela d d e r structure show n in Fig. 7.27 for M = N . Its output is
M
(7.3.45)
msrO
w here (um) are the param eters that determ ine the zeros o f the system . T h e system

Figure 7.27

Lattice-ladder structure for the realization o f a p ole-zero system.

Implementation of Discrete-Time Systems

538

Chap. 7

function corresponding to (7.3.45) is

H(z) =

Y (z )
X (z)
(7.3.46)

-Z>

G m(z)
X{z)

Since X ( z ) = F N(z) and fb (z) = G q( z ), (7.3.46) can be written as

rr/ ^
H(z) =

G m{z) Fq( z )
> u m 7 ^
G 0 (z) FN (z)
Bm{z)
a n (z )

(7.3.47)

M
y , vmBm{z)
A \(z)
If w e com pare (7.3.41) with (7.3.47), w e con clu d e that
M
C M(z) = ' / vmBm(z)
m=0

(7.3.48)

T his is the desired relationship that can be used to determ in e the w eighting coef
ficients {um). Thus, w e have dem onstrated that the coefficients o f the num erator
polyn om ial C*f(z) determ in e the ladder param eters {um}, w h ereas the coefficients
in the d en om in ator polyn om ial A ^(z) determ ine the lattice p aram eters {Km\.
G iven the polyn om ials C (z ) and A n ( z ), w here N > M , th e param eters of
the all-pole lattice are d eterm ined first, as described p reviou sly, by the conver
sion algorithm given in Section 7.2.4, which con verts the direct form coefficients
in to lattice param eters. B y m ean s o f the step -d ow n recursive relations given by
(7.2.54), w e obtain the lattice param eters {Km} and th e p olyn om ials Bm(z), m = 1,
2 ,..., N.
T h e ladder p aram eters are d eterm in ed from (7.3.48), w hich can be expressed
as
TO1
C m( z) = vk B k(z) + vmB m(z)
(7.3.49)
or, equivalently, as
C m(z) = Cm_ ,(z ) + vmB m( z)

(7.3.50)

Thus Cm(z) can b e com p uted recursively from the reverse p olyn om ials Bm(z), m =
1 , 2 , . . . , M . Since
1 for all m, th e param eters vm, m = 0, 1 , . . . , M can be
determ ined by first noting that
vm = cm(m)

m = 0,1,..., M

(7.3.51)

Sec. 7.4

State-Space System Analysis and Structures

539

T h en , by rew riting (7.3.50) as

Cm- \ (z) = Cm (z ) - iv, Bm(z)

(7.3.52)

and running this recursive relation backward in m (i.e., m = M , M 1 , . , , , 2 ), w e

o b tain cm(m ) and therefore the ladder param eters according to (7.3.51).
T h e lattice-ladder filter structures that w e h ave p resen ted require the m in
im um am ount o f m em ory but not the m inim um num ber o f m ultiplications. A l
thou gh lattice structures w ith only on e m ultiplier per lattice stage exist, the tw o
m ultiplier-per-stage lattice that w e have described, is by far th e m ost w idely used in
practical applications. In con clu sion , th e m odularity, the built-in stability charac
teristics em b o d ied in the coefficients {ATm}, and its robustness to finite-w ord-length
effe cts m ake th e lattice structure very attractive in m any practical applications,
including sp eech processing system s, adaptive filtering, and geophysical signal pro
cessing.

7.4 STATE-SPACE SYSTEM ANALYSIS AND STRUCTURES

U p to this point our treatm ent o f linear tim e-invariant system s has b een lim ited
to an in p u t-o u tp u t or external description of the characteristics o f the system . In
other w ords, the system was characterized by m athem atical eq u ation s that relate
the input signal to the output signal. In this section w e introduce the basic concepts
in the state-sp ace description o f linear tim e-invariant causal system s. A lth ou gh the
state-space or in tern a l description of the system still in volves a relationship betw een
the input and output signals, it also in volves an additional set o f variables, called
state variables. Furtherm ore, the m athem atical eq u ation s describing the system ,
its input, and its output are usually divided into tw o parts:
1. A set o f m athem atical eq u ation s relating th e state variables to the input
signal.
2. A seco n d set o f m athem atical eq u ation s relating the state variables and the
current input to the output signal.
T h e state variables provide inform ation ab ou t all th e internal signals in the
system . A s a result, the state-sp ace description provides a m ore d etailed descrip
tion o f the system than the in p u t-ou tp u t description. A lth o u g h our treatm ent o f
state-sp ace analysis is confined prim arily to sin gle in p u t-sin g le output linear tim einvariant causal system s, the state-sp ace tech n iqu es can also b e applied to non
linear system s, tim e-variant system s, and m ultip le in p u t-m u ltip le ou tp u t system s.
In fact, it is in the characterization and analysis o f m ultip le in p u t-m u ltip le output
system s that th e p ow er and im portance o f state-sp ace m eth od s are clearly evident.
B o th in p u t-o u tp u t and state-variable descriptions o f a system are useful in
practice. T h e description w e use d ep en d s on the p rob lem , the available inform a
tion, and the q u estion s to b e answ ered. In our p resen tation , the em phasis is on

540

Implementation of Discrete-Time Systems

Chap. 7

the use o f state-sp ace tech n iqu es in system analysis, and in the d evelop m en t o f
state-sp ace structures for th e realization o f discrete-tim e system s.

7.4.1 S ta te -S p a ce D e sc rip tio n s o f S y s te m s C haracterized

by D ifference E qu ations
A s w e have already o b served , the determ ination o f the output o f a system requires
that w e know the input signal and the set o f initial con d ition s at th e tim e the input
is applied. If a system is n ot relaxed initially, say at tim e no, th en k n ow led ge o f
the input signal x ( n ) for n > n Q is not sufficient to un iqu ely d eterm in e the output
y(n) for n > no- T h e initial con d ition s o f the system at n = no m ust also b e know n
and taken in to account. T his set o f initial con d ition s is called th e state o f the
system at n = no- H en ce we defin e the state o f a sy ste m a t tim e no a s the a m o u n t o f
in fo rm a tio n that m u s t be p r o v id e d at tim e n0, w hich, together w ith th e in p u t signal
x ( n ) f o r n > no, u n iq u e ly d eterm in e the o u tp u t o f th e sy stem f o r a ll n > noFrom this definition w e infer that the con cep t o f state lead s to a d ecom p o
sition o f a system in to tw o parts, a part that contains m em ory, and a m em oryless
com p onent. T he inform ation stored in the m em ory co m p on en t con stitu tes the set
o f initial conditions and is called the state o f the system . T h e current ou tp u t o f the
system then b eco m es a function o f the current value o f the input and the current
state. T hus, to d eterm in e the output o f the system at a given tim e, w e n eed the
current value o f the state and the current input. Since the current value o f the
input is available, w e only n eed to provide a m echanism for upd atin g the state o f
the system recursively. C on sequ en tly, the state o f the system at tim e no + 1 should
d epend on the state o f the system at tim e n 0 and the value o f th e input signal x (n)
at n = noT h e follow in g exam p le illustrates the approach in form ulating a state-space
description o f a system . L et us con sid er a linear tim e-invariant causal system
described by the d ifferen ce equation
3

(7.4.1)
T h e direct form II realization for the system is show n in Fig. 7.28.
A s state variables, w e u se the con ten ts o f the system m em ory registers, cou n t
ing them from the b o ttom , as show n in Fig. 7.28. W e recall that th e output o f a
delay elem en t rep resen ts the present value stored in th e register and the input
represents the next v alu e to b e stored in th e m em ory. C on seq u en tly, with the aid
o f Fig. 7.28, w e can w rite
t>i(n + 1 ) = V2 (n)
v i (n +

) = U3 ()

v$(n + 1 ) = - a 3U i(n ) - 0 2 2 (n ) - o m 3( n ) + x(n)

(7.4.2)

Sec. 7.4

541

State-Space System Analysis and Structures

<*>
o

- l

v2()
0

U,()
" 3

Figure 7.28 Direct form II realization of system described by the difference equa
tion in (7.5.1).

It is in terestin g to n o te that the state-variable form ulation for the third-order

system o f (7.4.1) in v o lv es three first-order difference eq u ation s given by (7.4.2). In
general, an n th-order system can be described by n first-order differen ce equations.
T h e ou tp u t eq u ation , w hich exp resses y ( n ) in term s o f the state variables and
the presen t input v alu e x (n ), can also be ob tain ed by referring to Fig. 7.28. W e have
y (n ) = i>oVi(n + 1 ) + />3 Ui(n) + b2v2 (n) + 6 it>3 (/i)
W e can elim in a te v$(n + 1 ) by using the last equation in (7.4.2). Thus w e obtain
the desired ou tp u t eq u ation
y (n ) = (f >3 - boa2) v i (n ) + ( 2 boa2) v2(n) + (b\ - i>o<ai)u3 (n) + b0x ( n )

(7.4.3)

If w e put (7.4.2) and (7.4.3) in to m atrix form w e have

V[(n + 1 )-

V2 (n + 1) =
_ v 3 (n + 1 ) .

.-0 3

-a 2

t>] ( n ) -

1
-a \.

V2 (n)
-i> 3 (n )-

and
y ( n ) = [(b3 - b0a 3) ( ^ - M

) (i>i ~ M l ) ]

-0 0

x(n)

(7.4.4)

.1 .

~ v i(n )
v2(n) +box(n)

(7.4.5)

-V 3 ( * ) -

T h e eq u a tio n s (7.4.4) and (7.4.5) provide a com p lete description o f the sys
tem . F urtherm ore, th e variables v i(n ), V2 (n), and 113( 0 ), w hich sum m arize all the
n ecessary past inform ation, are the state variables o f the system . W e also observe
that as in d icated previou sly, eq u ation s (7.4.4) and (7.4.5) split the system in to tw o
co m p o n en t parts, a dynam ic (m em ory) subsystem and a static (m em oryless) sub
system . W e say that this se t o f eq u ation s provides a state-space description o f the
system .

Implementation of Discrete-Time Systems

542

Chap. 7

B y generalizing the previous exam p le, it can easily be se en that the A'th-order
system described by
N

Af
,( ) = - a ty(n ~
*=i

bkX^n ~ k)

(7.4.6)

Jt=0

can be expressed as a linear tim e-invariant state-space realization by the relations

State equation
v(n -j- 1 ) = Fv(n) + q x(n )

(7.4.7)

>() = gf v ( n ) + d x ( n )

(7.4.8)

Output equation

w here the elem en ts o f F, q, g, and d are con stan ts (i.e., they d o n ot change as a
function o f the tim e index n ), given by
-

~a -1

-a2

~a\

"0 "

q=

F =
.-a s

(7.4.9)
0

b N boaN
b fj - 1 bo a n-\

g=
b\ boa\
A n y discrete-tim e system w h ose input x (n ), ou tp u t y( n ) , and state v{n), for
all n > no, are related by the state-sp ace eq u ation s ab ove, w h ere F, q, g, and d are
arbitrary but fixed quantities, will be called linear and tim e invariant. If at least
o n e o f the quantities in F, q, g, or d d ep en d s on tim e, the system b ecom es time
variant.
W e will refer to (7.4.7) through (7.4.8) as the linear tim e-in va rian t state-space
m o d el, which can b e represented by the sim ple vector-m atrix block diagram in
Fig. 7.29. In this figure the d ou b le lin es represent vector qu an tities and the blocks
represent the v ector or m atrix coefficients.
Example 7.4.1
Determine the state-space equations for the transposed direct form II structure shown
in Fig. 7.30.
Solution

The validity of this structure can be seen if we rewrite (7.4.1) as

y(n) =

- k) - aky(n - * )] + box(n)

Sec. 7.4

State-Space System Analysis and Structures

543

Figure 7.29 General state-space description of a linear time-invariant system.

*()

Figure 7JO State-space realization for the system described by (7.4.1).

Due to the linearity and time invariance of the system, instead of first delaying the
signals x(n) and y(n) and then computing the terms bkx(n - k) aky(n k) as in
Fig. 7.28, we first compute the terms bkx(n) at y(n) and then delay them.
If we use the state variables indicated in Fig. 7.30, we obtain
'v i(n + 1 )"
Vi (n + 1 ) =
. v3(n + 1 ) .

.0

'M n )
bi boaj "
-a 2
i>2 (n) = * 2 - M 2 x(n)
-b\ bod\ .
- a i . .t>B(n).
03

(7.4.10)

Di(n)
y(n) = [0

1 ]

l> 2 (fl)

-f box(n)

(7.4.11)

V3(n) J
T h e state-sp ace description specified by (7.4.4) and (7.4.5) is know n as a ty p e
1 state-sp ace realization, w h ereas the o n e described by (7.4.10) and (7.4.11) is
ca lled a typ e 2 state-sp ace realization.

7.4 .2 S o lu tio n o f th e S ta te -S p a ce E q u ation s

T h ere are several m eth od s for solvin g the state-sp ace eq u ation s. H ere w e discuss
a recursive solu tion w hich m akes u se o f th e fact that the state-sp ace eq u ation s are
a se t o f linear first-order differen ce equations.

544

Implementation of Discrete-Time Systems

Chap. 7

For the jV-dim ensional state-sp ace m odel

v( + 1) = F v(/i) + qjr(n)

(7.4.12)

y(n) = g'v(n) + d x(n )

(7.4.13)

and given the initial co n d ition v ( 0), we have for n > n0,
v(n 0 + 1) = Fv(n0) + q*(n)
v(no + 2 ) = F v(n 0 + 1 ) + qjr(n0 + 1 )
= F 2 v(n0) + F q j( o ) + q-^(o + 1)
w here F 2 represents the matrix product FF and Fq is the product o f the m atrix F
and the vector q. If w e continue as in the on e-dim en sion al case, w e obtain, for
n > n0,
(7.4.14)
The matrix F is defined as the N x N identity matrix, having unity on the
main diagonal and zeros elsew h ere. T he m atrix F'- -' is often d en o te d as 4>(/ - j ) ,
that is,
* (/ - j) = F ~ J

(7.4.15)

for any p ositive in tegers i > j . T his m atrix is called the state transition matrix of
the system .
T h e output o f the system is obtained by substituting (7.4.14) in to (7.4.13).
T he result o f this substitution is
y(n) = g'F" nov(n0) +

g'F" 1 kq x ( k ) + d x { n )

*=0

(7.4.16)
n- 1

- floM no) + ^

1 - &)q*(*) + d x ( n )

From this general result, w e can determ ine the output for tw o special cases.
First, the zero-input resp onse o f the system is
yzM ) = g'F" nv(n0) = g '$ ( n - floM no)

(7.4.17)

O n the other hand, the zero-state response is

n1
yzs(n) =

g' $ { n - 1 - fc)q*(/:) + d x ( n )

(7.4.18)

Clearly, the A'-dim ensional state-sp ace system is zero-in p u t linear, zero-state
linear, and since y ( n ) = y Zi(n) + y zs(n), it is linear. F urtherm ore, sin ce any system
described by a linear con stan t-coefficien t d ifferen ce eq u ation can be put in the
state-space form, it is linear, in agreem en t with the results ob tain ed in S ection 2.4.

Sec. 7.4

State-Space System Analysis and Structures

545

7.4.3 Relationships Between Input-Output and

State-Space Descriptions
From our p reviou s discussion w e h ave seen that there is n o unique ch oice for the
state variables o f a causal system . Furtherm ore, d ifferent ch o ices for th e state
vecto r lead to differen t structures for the realization o f the sam e system . H en ce,
in g en eral, the in p u t-o u tp u t relationship d oes not un iqu ely d escrib e the internal
structure o f th e system .
T o illu strate these assertions, let us consider an N -dim en sion al system with
the sta te-sp a ce rep resen tation
v(n + 1) = Fv(rt) 4- qjf(n)

(7.4.19)

y(n ) = g'v(n) -1- d x ( n )

(7.4.20)

L et P b e any N x N matrix w h ose inverse m atrix

state v ecto r v(n) as

P-1 exists. W e d efine a new

(7.4.21)
(7.4.22)

(7.4.23)

(7.4.24)
N o w , w e d efine a n ew system param eter matrix

F and the vectors q and g as

F = PFP-1
(7.4.25)

W ith th ese d efinition s, th e state eq u ation s can b e exp ressed in term s o f th e new
system qu an tities as

v(rt 4 1) Fv(n) 4 - q *(n )

(7.4.26)

y ( n ) = fv(n) -j- d x ( n )

(7.4.27)

If w e com p are (7.4.19) and (7.4.20) with (7.4.26) and (7.4.27), w e ob serve
that by a sim p le linear transform ation o f the state variables, w e have gen erated
a n ew set o f sta te eq u ation s and an output eq u ation , in w hich th e input x ( n ) and
the ou tp u t y ( n ) are unchanged. Since there is an infinite n um ber o f ch oices o f the
transform ation m atrix P, there is also an infinite num ber o f state-sp ace eq u ation s

Implementation of Discrete-Time Systems

546

Chap. 7

and structures for a system . S om e o f these structures are different, w hile som e
others are very sim ilar, differing o n ly by scale factors.
A ssociated with any state-sp ace realization o f a system is the con cep t o f a
m i n i m a l realization. A state-sp ace realization is said to b e m i n i m a l if the d im ension
o f the state sp a ce (th e num ber o f state variab les) is th e sm allest o f all p ossible
realizations. Since each state variable rep resen ts a quantity that m ust be stored
and updated at every tim e instant n , it fo llo w s that a m inim al realization is o n e
that requires the sm allest num ber o f d elays (storage registers). W e recall that the
direct form II realization requires the sm allest num ber o f storages registers, and
con sequ en tly, a state-sp ace realization based on the con ten ts o f the d elay elem en ts
results in a m inim al realization. Sim ilarly, an F IR system realized as a direct form
structure leads to a m inim al state-sp ace realization if th e valu es o f the storage
registers are defined as the state variables. O n the other hand, the direct form I
realization o f an IIR system d oes n ot lead to a m inim al realization.
N o w , let us determ in e the im pulse resp on se o f the system from the statespace realization. T he im pulse resp onse provid es o n e o f the links betw een the
in p u t-o u tp u t and state-space description o f system s.
B y definition the im pulse resp onse h ( n ) o f a system is the zero-state re
sponse o f the system to the excitation x ( n ) = 6 (n). H e n c e it can be obtained from
equation (7.4.16) if w e set no = 0 (th e tim e w e apply th e input), v(0) = 0, and
x ( n ) S(n). T hus the im pulse resp onse o f the system describ ed by (7.4.19) and
(7.4.20) is given by
h (n) g, Fn~ 1 q(n 1) + d 8(n)
(7.4.28)
= g '$ ( n l)qw (n 1 ) + d 8 ( n )
G iven a state-sp ace description, it is straightforward to d eterm in e the im pulse re
sp on se from (7.4.28). H ow ever, the inverse is n ot easy since there is an infinite
number o f state-sp ace realizations for the sam e in p u t-o u tp u t description.

The transpose system. T h e transpose o f a m atrix F is ob tain ed by inter

changing its colum ns and rows, and it is d en oted by F . F or exam p le,
/12

f\N '

/21

/2 2

-/v 1

fs2

r /n

f2N

fsN

F* =
-

r /11

/21

/12

/2 2

/V 2

-flN

flN

f NN -

fm '

N o w define the tra nspose s y s t e m (7 .4 .1 9 )-(7 .4.20) as

v'tn + 1) = F v * (rt) + g x ( n )

(7.4.29)

y '( n ) = q V (n ) - \-d x ( n )

(7.4.30)

A ccording to (7.4.28), the im pulse resp on se o f this sy stem is given as

t i ( n ) = qr(F')"-1 g(n - 1) + d S ( n )

(7.4.31)

Sec. 7.4

State-Space System Analysis and Structures

547

From m atrix algebra w e know that (F )" 1 = ( F -1 )'- H en ce

h '( n ) ~ q ' ( F _ 1 ), gw(n - 1 ) + d5(n)
W e claim that h'(n) = h(n). In d eed , the term q ' ( F -1 )'g is a scalar. H en ce it
is eq u al to its transpose. C onsequently,
[ q ' ( F - 1 )'g]' = g'(F y - ' q
S ince this is true, it fo llow s that (7.4.31) is identical to (7.4.28) and, therefore,
h '(n) = h (n). T hus a single in p ut-sin g le o u t p u t sy stem a n d its trans po se have i d e n
tical im p u lse respons es a n d h en ce the s a m e i n p u t - o u t p u t relationship. T o support
this claim further, w e n ote that the type 1 and type 2 state-sp ace realizations,
d escribed by (7.4.3), (7.4.4), (7.4.10), and (7.4.11) are transpose structures, which
stem from the sam e in p ut-ou tp u t relationship (7.4.1).
W e have introduced the transpose structure b ecau se it provides an easy
m eth o d for generating a new structure. H ow ever, som etim es this new structure
m ay either differ trivially or be identical to the original on e.

The diagonal system. A closed -form solu tion o f the state-space equations
is easily ob tain ed w hen the system m atrix F is diagonal. H en ce, by finding a m atrix
P so that F = P F P - 1 is diagonal, the solu tion of the state eq u ation s is sim plified
considerably. T h e d iagonalization o f the m atrix F can be accom plished by first
d eterm ining the eigen valu es and eigen vectors o f the matrix.
A num ber A is an eigenvalue o f F and a n on zero vector u is the associated
eigen vecto r if
Fu = Au

(7.4.32)

T o determ ine the eigen valu es o f F, w e n ote that

(F - Xl)u = 0

(7.4.33)

T his eq u ation has a (nontrivial) non zero solu tion u if the m atrix F - XI is singular
[i.e., if (F Al) is n oninvertible], which is the case if the determ inant o f (F /.I)
is zero, that is, if
d et (F - XI) = 0

(7.4.34)

This determ inant in (7.4.34) yield s the characteristic p o l y n o m i a l o f the m atrix

F. For a n N x N m atrix F, the characteristic polyn om ial o f F is degree N and h en ce
it has jV roots, say X,, i = 1, 2 .........N . T h e roots m ay b e distinct or som e roots
m ay b e repeated. In any case, for each root A,, w e can d eterm ine a vector u (,
called the eigen vector corresponding to the eigen valu e X,, from the equation
FU; = X(U;
T h ese eigen vectors are orthogonal, that is, uju, = 0, for i ^ j .
If w e form a m atrix U w h ose colum ns consist o f the eigen vectors {u, }, that is,

f t
U =

Ul
- 1 1

t
U2

UjV
i J

548

Implementation of Discrete-Time Systems

Chap. 7

then the m atrix F = U -1 F U is diagonal. Thus w e have solved for the m atrix that
diagonalizes F.
T h e follow in g exam ple illustrates the procedure o f d iagon alizin g F.
Example 7.4.2
The Fibonacci sequence, which is the sequence {1,1,2, 3, 5 ,8 .1 3 ....} , can be gener
ated as the impulse response of the system that satisfies the state-space equations
v ( + 1) =

j j v < n ) + j ^ J .x ( n )

y(rt) = [ 1

1 ] v ( n ) + x(n)

Determine the impulse response {/()} of the system.

Solution

Now we wish to determine an equivalent system

v(n + 1) = Fv(n) + qx(n)
y(n) = g'v(n) + dx(n)

such that the matrix F is diagonal. From (7.4.25) we recall that the two systems are
equivalent if
F = PFP'

q = P?

g '= g 'P " 1

Given F. the problem is to determine a matrix P such that F = PFP - 1 is a diagonal

matrix.
First, we compute the determinant in (7.4.34). We have
d e t(F -A l) = d e t [ j

X 1 = 0

or
1 + V5
2

x, = -

1 - Vs

X2 = - 1 -

To find the eigenvector U] corresponding to A.], we have

[?

r '=[i]

Similarly, we obtain

We observe that ur,u 2 = 1 +

= 0 (i.e., the eigenvectors are orthogonal). Now
matrix U, whose columns are the eigenvectors of F, is

Then the matrix U -1FU is diagonal. Indeed, it easily follows that

Xj

0 1

Sec. 7.4

State-Space System Analysis and Structures

549

and since the transformation matrix is P = U -1, we have

p

-M
= __ !__ r ^
A.2 A.i LA-i
1 J

Thus the diagonal matrix F has the form

M o':]
where the diagonal elements are the eigenvalues of the characteristic polynomial.
Furthermore, we obtain
_1_
vl

q = Pq =

L 'V 5 J

and
r =

'3 + V s 3 - V 5
2

The impulse response of this equivalent diagonal system is

fi(n) = g'Fqu(n - 1) +d&(n)

(^K^y
m

u(n - 1 ) + 5(n)

which is the general formula for the Fibonacci sequence.

An alternative expression can be found by noting that the Fibonacci sequence
can be considered as the zero-input response of the system described by the difference
equation
y(n) = y(n - 1 ) + y(n - 2 ) + x (n)
with initial conditions j><1) = 1, y (2) = 1. From the type 1 state-space realization,
we note that U](0) = y { - 2 ) = - 1 and t^(0) = ;y (-l) = 1. Hence
r * ( n
Lt>2 (0 ) J

p r ^ n
Lv2 <0 )J

zi
5

r - 3 + V5 n
2

3 + VS

and the zero-input response is

y n(") = r ^ ^ ( 0 )
()
This is the more familiar form for the Fibonacci sequence, where the first term of the
sequence is zero, that is, the sequences is {0 , 1 , 1 , 2 , 3 , 5 , 8 , . . .)

Implementation of Discrete-Time Systems

550

Chap. 7

This exam ple illustrates the m eth od for diagonalizing the matrix F. The
diagonal system y ield s a set o f N d ecou p led , first-order linear d ifferen ce equations
that are easily solved to yield the state and the ou tp u t o f the system .
It is im portant to n ote that the eigen valu es o f the matrix F are identical to the
roots o f the characteristic polynom ial, which are ob tain ed from th e h om ogen eou s
d ifferen ce eq u ation that characterizes the system . F or exam p le, the system that
g en erates the F ibonacci seq u en ce is characterized by the h o m o g e n e o u s difference
equation
y(n)

y(n

I)

y{n - 2) = 0

(7.4.35)

R ecall that the solu tion is ob tain ed by assum ing that the h o m o g e n e o u s solution
has the form
yh(n) = kn
Substitution o f this solu tion into (7.4.35) yield s the characteristic polynom ial
A2 - / - 1 = 0
B ut this is exactly the sam e characteristic polynom ial ob tain ed from the determ i
nant o f (F - XI).
Since the state-variable realization o f the system is not u n iq u e, the matrix
F is also not unique. H ow ever, the eigen valu es o f the system are unique, that is,
they are invariant to any nonsingular linear transform ation o f F. C onsequently,
the characteristic polynom ial o f F can be determ in ed eith er from evaluating the
determ inant o f ( F - A l ) or from the d ifferen ce eq u ation characterizing the system .
In conclusion, th e state-sp ace description provides an alternative character
ization o f the system that is eq u ivalen t to the in p u t-ou tp u t description. O n e ad
vantage o f the state-variable form ulation is that it provid es us with the additional
inform ation concerning the internal (state) variables o f the system , inform ation
that is not easily ob tain ed from the in p u t-ou tp u t description. Furtherm ore, the
state-variable form ulation o f a linear tim e-invariant system allow s us to represent
the system by a set o f (usually cou p led ) first-order differen ce eq u ation s. T he d e
coupling o f the eq u ation s can be ach ieved by m eans o f a linear transform ation that
can b e ob tain ed by solvin g for the eigen valu es and eigen vectors o f the system . The
d ecou p led eq u ation s are then relatively sim ple to solve. M ore im portant, how ever,
the state-space form ulation provides a pow erful, yet straightforward m eth od for
d ealing with system s that have m ultiple inputs and m ultiple ou tp u ts (M IM O ). A l
though w e have n ot con sid ered such system s in our study, it is in the treatm ent of
M IM O system s w h ere the true p ow er and the b eau ty o f the sp ace-sp ace form ula
tion can be fully appreciated.

7.4.4 State-Space Analysis in the z-Domain

T h e state-sp ace analysis in th e previous section s has b een perform ed in the time
d om ain. H o w ev er, as w e have ob served previously, the analysis o f linear timeinvariant discrete-tim e system s can also b e carried ou t in the z-transform

Sec. 7.4

State-Space System Analysis and Structures

551

d om ain , often w ith greater ease. In this section w e treat the state-sp ace rep
resen tation o f linear tim e-invariant d iscrete-tim e system s in the z-transform d o
main.
L et us con sid er the state-sp ace eq u ation
v(rt + 1) = Fv(n) + <pr(n)

(7.4.36)

If w e define the v ecto r V (z) as

* \(z ) 1
V2(z)
V (z) =

(7.4.37)

LVV(z).
then (7.4.36) can b e ex p ressed in m atrix form as
zV (z) = F V (z) + q * ( z )
T h e tw o term s in volving
can b e used to so lv e for

V(z) can be
V(z). Thus

(7.4.38)

collected togeth er and the resulting equation

(zi - F)V(z) = q*(z)

(7.4.39)

V(z) = ( z I - F r V ( z )
T h e inverse z-transform o f (7.4.39) yield s the solu tion for the state equations.
N ex t, w e turn our atten tion to the output eq u ation , which is given as
j (n ) = g*v (n ) + d x ( n )

(7.4.40)

y ( z ) = g ' V ( z ) + d X (z)

(7.4.41)

T h e z-transform o f (7.4.40) is

B y using the solu tion in (7.4.39) w e can elim in ate th e state vector V (z) in
(7.4.41). Thus w e obtain
y (z ) = [g, ( z I - F ) ~ 1q + d ]X (z)
w hich is the z-transform o f the zero-state resp onse o f th e system .
fun ction is easily ob tain ed from (7.4.42) as
H (z ) = y = g ' ( z I - F ) - 1q + <f

(7.4.42)
T h e system

(7.4.43)

T h e state eq u ation given by (7.4.39), th e ou tp u t eq u ation given by (7.4.42) and the

system function given by (7.4.43) all h ave in com m on the factor (z i F )- 1 . This
is a fun d am en tal quantity that is related to the z-transform o f the state transition
m atrix o f th e system . T h e relationship is easily estab lish ed by com puting the

552

Implementation of Discrete-Time Systems

Chap. 7

z-transform o f the im pulse resp on se h (n), which is g iv en by (7.4.28). T hus w e

have
00

H (z) = X > ( )z ~ "

n=0
oc
- 1) + d i ( n ) \ z ~ n

(7.4.44)

T h e term in p a ren theses in (7.4.44) can b e w ritten as

00

= z - H l + F z 1 + F 2 z - 2 + ---)

(7.4.45)

= z -H l-F z " 1)" 1 = ( z I - F ) ' 1

If w e substitute the result in (7.4.45) into (7.4.44), w e obtain the expression for
H ( z ) as given in (7.4.43).
Since the state transition m atrix is given by
(/i) = F
the z-transform o f

(7.4.46)

(n) is
00

V z~" = I + F z - 1 + F 2 z - 2 + F V 3 +
= ( I - F z - 1) - 1 = z ( z I - F ) - 1

T h e relation in (7.4.47) p rovides a sim p le m eth od for determ ining the state
transition m atrix by m eans o f z-transform s. W e recall that
(z I-F T 1 =

d e t(z l - F)

(7.4.48)

w here a d j(A ) d en o te s the a djo int m a trix o f A and d et ( A ) d en o te s the determ inant
o f the matrix A . Substitution o f (7.4.48) in to (7.4.43) yield s the result

(7A49>
C on sequ en tly, the den om in ator D ( z) o f th e system fun ction H ( z ) , w hich contains
the p o les o f the system is sim ply
D ( z ) = d e t(z l - F)

(7.4.50)

But the d e t(z l F) is just the characteristic p olyn om ial o f F. Its roots, w hich are
the p o les o f system , are th e eigen valu es o f th e m atrix F.

Sec. 7.4

State-Space System Analysis and Structures

553

Example 7A 3
Determine the system function H(z), the impulse response h(n), and the state tran
sition matrix 3>(n) of the system that generates the Fibonacci sequence. This system
is described by the state-space equation
v(n + l) = j^
y(n) = [ 1
Solution

j j v(n) +

x(n)

1 ] v(n) + jr(n)

First, we determine H(z) and h(n) by computing (zl - F)_t. We have

Hence

..[71 !][?] +'

1

z2 - ; - 1

l- z - '-z - 2

By inverting / / ( ), we obtain h(n) in the form

h(n) =

(H)

y/5

We note that the poles of H(z) are />] = (] + \/5 )/2 and p 2 = (1 - \/5)/2. Since
| Pi | > 1, the system that generates the Fibonacci sequence is unstable.
The state transition matrix # (n ) has the z-transform
z(zl - F) " 1 =

2 1
[ ;2 ~ Z
z2 - z - 1 L z
z J

The four elements of ^(n ) are obtained by computing the inverse transform of the
four elements of z(zI - F)_1. Thus we obtain

L<fci(n) <h2(n)l
where

<hi(n) = ~7 =

H T -(4 T >

()

We note that the impulse response h{n) can also be computed from (7.4.28) by using
the state transition matrix.

554

Implementation of Discrete-Time Systems

Chap. 7

This analysis m ethod applies specifically to the com p utation o f the zero-state
response o f the system . T his is the con seq u en ce o f the fart that w e have used the
tw o-sided z-transform .
If w e wish to determ in e the total resp on se o f the system , beginning at a
nonzero state, say v(/i0), w e m ust use the on e-sid ed z-transform . Thus, for a given
initial state v(n0) and a given input * ( ) for n > no, w e can d eterm in e th e state
vector v(n) for n > no and the output y ( n ) for n > no, by m ean s o f the one-sided
z-transform.
In this d ev elo p m en t w e assum e that no = 0, w ithou t loss o f generality. T hen,
given ;r(n) for n > 0 , and a causal system , d escrib ed by th e state eq u ation s in
(7.4.36), the on e-sid ed z-transform o f the state eq u ation s is
zV + (z) - zv(0) = F V + (z) + q X (z)
or, equivalently,
V + (z) = z (z l - F r V O ) + (z l - F ) - 1 q X (z)

(7.4.51)

N o te that X+ (z) = X (z), since x ( n ) is assum ed to b e causal.

Similarly, the z-transform o f the output eq u ation given by (7.4.40) is
Y +(z) = i V + (z) + d X ( z )

(7.4.52)

If w e substitute for V + (z) from (7.4.51) in to (7.4.52), w e obtain the result

Y +(z) = z g ' ( z I - F r 1 v(0) + [ g ' ( z I - F r 1q + rf]X (z)

(7.4.53)

O f the terms on the right-hand side o f (7.4.53), the first rep resen ts the zero-input
response o f the system due to the initial con d ition s, w hile the secon d represents
the zero-state resp onse o f the system that w e ob tain ed previously. C onsequently,
(7.4.53) constitutes the total resp onse o f th e system , which can b e exp ressed in the
tim e dom ain by inverting (7.4.53). T h e result o f this inversion y ield s the form for
y(n) given p reviously by (7.4.16).

7.4.5 Additional State-Space Structures

In Section 7.4.2 w e described h ow state-sp ace eq u ation s can b e ob tain ed from a
given structure and, con versely, h ow to obtain a realization o f the system given
the state equations. In this section w e revisit the parallel-form and cascade-form
realizations described previously and con sid er th ese structures in the con text o f a
state-space form ulation.
T h e parallel-form state-sp ace structure is ob tain ed by exp an d ing the system
function H ( z ) in to a partial-fraction exp an sion , d ev elo p in g the state-sp ace form u
lation for each term in the expansion and the corresponding structure, and finally,
connecting all the structures in parallel. W e illustrate the p roced u re under the
assum ption that the p o les are distinct and N M .
T he system fun ction H ( z ) can be exp ressed as
N
D
H (z) = C + Y

(7.4.54)

Sec. 7.4

555

State-Space System Analysis and Structures

N o te that this is a d ifferent exp an sion from that given in (7.3.20). T h e output o f
the system is
Y (z ) = H ( z ) X ( z ) = C X ( z ) +

(7.4.55)

Bk Yk(z)
k=1

w here, by definition,
X(Z)
it = 1 ,2 .........N
Z - Pt
In the tim e dom ain, the eq u ation s in (7.4.56) b ecom e

(7.4.56)

Yk (z) =

y t (n + 1) = p kyk(n) + * ( )

k = 1 , 2 , , N

(7.4.57)

W e define the state variables as

(7.4.58)

k = 1 ,2 ,..., N

vt(n ) = yt (n)

T h en the d ifferen ce eq u ation s in (7.4.57) b ecom e

Vk(n + 1 ) = PkVkin) + x ( n )

* = 1 , 2 .........N

(7.4.59)

T h e state eq u ation s in (7.4.59) can b e exp ressed in m atrix form as

"Pi
0

o
pj

0 0

"1"
1
v(n) +

v(n + 1) =
_

jc(rt)

(7.4.60)

.1 .

Pn -

and the output equation is

y (n ) = [ B]

B2

B n ] v(n ) + C x ( n )

(7.4.61)

This parallel-form realization is called the n o r m a l f o r m representation, b e

cause the m atrix F is d iagonal, and h en ce the state variables are uncoupled. A n
alternative structure is ob tain ed by pairing com p lex-con ju gate p oles and any tw o
real-valued p o les to form secon d-ord er section s, which can be realized by using
eith er type 1 or type 2 state-sp ace structures.
T h e ca scade-fo rm state-sp ace structure can b e ob tain ed by factoring H ( z ) in to
a product o f first-order and secon d-ord er section s, as described in S ection 7.2.2,
and then im plem en ting each section by using eith er type 1 or type 2 state-sp ace
structures.
L et us con sid er the state-sp ace representation o f a single secon d-ord er section
in volving a pair o f com p lex-con ju gate p oles. T h e system function is
H (z) =

bo + b i z ~ l + f a z ~ 2
1 + O iZ - 1 + 0 2 Z ' 2

boz2 + b i z + b 2

(7.4.62)

z 2 + a i z + 02
A
A*
+
z - p
Z -p *
The output of this system can be expressed as
= bo +

t ( z ) +, ----------AX& h
. -------- y(z) = b0x
z - p

Z -p

(7.4.63)

556

Implementation of Discrete-Time Systems

Chap. 7

W e define the quantity

S (z) =

(7.4.64)

z -p

This relationship can be exp ressed in the tim e dom ain as

s{n + 1) = p s ( n ) + A x { n )

(7.4.65)

Since s(n ), p , and A are com p lex valued, w e define .y(n) as

s(n ) =

(/i) + j v 2(n)

P = i + ;' 2
A -

(7.4.66)

q \+ jq2

U p o n substitution o f these relations into (7.4.65) and separating its real and
im aginary parts, w e obtain
i>i ( + ] ) = cfiui (n) ct2V2( n ) + q \ x { n )
(7.4.67)
U2( + 1) = <*2i>i(n) + a ] U 2 ( n ) + q2x ( n )

W e ch oose vi(n) and v2(n) as the state variables and thus ob tain the coupled pair
o f state equations w hich can be expressed in matrix form as
v(n + 1 ) =

ai
a2

a 2
ai

v() +
L</2 j

j:(n)

(7.4.68)

T he output eq u ation can be exp ressed as

y ( n ) = b0x ( n ) + s ( n ) + j*(n )

(7.4.69)

U p on substitution for s(n ) in (7.4.69), w e obtain the desired result for the output
in the form
y ( n ) = [2

0 ] v ( ) + b0x ( n )

(7.4.70)

A realization for the secon d-ord er section is show n in Fig. 7.31. It is simply
called the co u p le d - fo rm state-sp ace realization. T his structure, w hich is used as the
building block in the im plem en tation o f cascade-form realizations for higher-order
IIR system s, exh ib its low sensitivity to finite-w ord-length effects.

7.5 REPRESENTATION OF NUMBERS

U p to this point w e h ave con sid ered the im plem en tation o f d iscrete-tim e systems
without being co n cern ed ab ou t the finite-w ord-length effects that are inherent in
any digital realization, w h eth er it b e in hardware or in softw are. In fact, w e have
analyzed system s that are m od eled as linear w h en , in fact, digital realizations of
such system s are inherently nonlinear.
In this and th e fo llow in g tw o section s, w e con sid er th e variou s form s of
quantization effects that arise in digital signal processing. A lth o u g h w e describe

Sec. 7.5

Representation of Numbers

557
i>o

i
Figure 7 3 1
system.

Coupled-form state-space realization of a two-pole, two-zero IIR

floating-point arithm etic operations briefly, our m ajor concern is with fixed-point
realizations o f digital filters.
In this sectio n w e con sid er the representation o f n um bers for digital com pu
tations. T h e m ain characteristic o f digital arithm etic is the lim ited (usually fixed)
nu m b er o f digits used to represent num bers. T his constraint leads to finite n u
m erical p recision in com putations, w hich leads to rou n d -off errors and nonlinear
effects in th e p erform ance o f digital filters. W e n ow provide a brief introduction
to digital arithm etic.

7.5.1 Flxed-Point Representation of Numbers

T h e rep resen tation o f n um bers in a fixed-point form at is a generalization o f the
fam iliar d ecim al rep resen tation o f a num ber as a string o f digits w ith a decim al
p oin t. In this n otation , th e digits to th e left o f th e d ecim al point represent the
in teger part o f th e num ber, and the digits to the right o f the decim al p oin t represent
the fractional part o f th e num ber. T hus a real num ber X can be represented as
X = {b~A ........ b ^ M M ..........b B)r
=

>,r~

0 < ,< ( r - l)

<7'5 1 )

i rzA

w h ere bt rep resen ts the digit, r is th e radix or base, A is th e num ber o f integer

sse

Implementation of Discrete-Time Systems

Chap. 7

digits, and B is the num ber o f fractional digits. A s an exam p le, th e decim al num ber
(123.45)io and the binary num ber ( 1 0 1 .0 1 ) 2 r e p r e se n t th e fo llo w in g sums:
(123.45)io =

x 10 2 + 2 x 10 1 + 3 x 10 + 4 x 10_I + 5 x 10 ~ 2

( 1 0 1 .0 1 ) 2 = 1 x 2 2 + 0 x 2 1 + 1 x 2 + 0 x 2 _ 1 + 1 x 2 2
L et us focus our atten tion on the binary representation sin ce it is the m ost
im portant for digital signal processing. In this case r = 2 and the digits {,} are
called binary digits or bits and take the valu es {0 ,1 }. T h e binary digit b - A is called
the m ost significant bit (M S B ) o f th e num ber, and the binary d igit b B is called the
least significant bit (L S B ). T h e binary p o in t b etw een the digits bo and b\ d oes
not exist physically in the com puter. Sim ply, the logic circuits o f the com puter
are design ed so that the com p utations result in num bers that correspond to the
assum ed location o f this point.
B y using an n-bit integer form at (A = n 1, B = 0), w e an represent
unsigned integers with m agnitude in the range 0 to 2" - 1. U sually, w e use the
fraction format (A = 0, B = n - 1), with a binary p oin t b etw een bo and b \, that
perm its num bers in the range from 0 to 1 - 2~n. N o te that any in teger or m ixed
num ber can be represented in a fraction form at by factoring o u t the term r A in
(7.5.1). In the sequ el w e focu s our atten tion on the binary fraction form at because
m ixed num bers are difficult to m ultiply and the num ber o f b its representing an
integer cannot be reduced by truncation or rounding.
T here are three w ays to represent n egative num bers. T h is leads to three
form ats for the representation o f signed binary fractions. T h e form at for positive
fractions is the sam e in all three representations, nam ely,
B

X = 0.bib2 bB = Y l b <- 2 ', X > 0

;i

(7.5.2)

N o te that the M SB bo is set to zero to represent th e positive sign. C onsider now

the negative fraction
B

X = - O M h bB = - J 2 b i - 2 ~

(7 -5 -3>

T his num ber can b e represented using o n e o f th e follow in g th ree form ats.

S ign -m agn itu d e form at.

In this form at, th e M SB is set to 1 to represent

the n egative sign,

.Xsm l.b \b 2 - b B

O n e's-com p lem en t form at.

for X < 0

(7.5.4)

In this form at th e negative num bers are rep

resen ted as
X iC = l b i h - b B

X < 0

(7.5.5)

w here fc; = 1 - i>, is th e o n e s com p lem en t o f bt . T h u s if X is a p ositive number,

the corresponding negative num ber is d eterm in ed by com p lem en tin g (changing l s

Sec. 7.5

Representation of Numbers

559

to 0 s and 0 s to l s) all th e bits. A n alternative definition for X ic can b e obtained

by notin g that
B

X 1C = 1 x 2 + ( 1 - bi) 2 -' = 2 - 2 ~ b \X\

(7.5.6)

r=l

Twos-complement format. In this form at a negative num ber is repre

sen ted by form ing the tw o s com p lem en t o f the corresponding p ositive num ber.
In o th er w ords, the negative num ber is ob tain ed by subtracting the p ositive num
ber from 2.0. M ore sim ply, the tw o s com p lem en t is form ed by com p lem en tin g
the p ositive num ber and adding on e L SB . Thus
X 2c = l . b i b 2 - - b B + 0 0 - - 0 1

X < 0

(7.5.7)

w here + represents m o d u lo -2 addition that ignores any carry gen erated from the
sign bit. For exam p le, the num ber 2 is sim ply ob tain ed by com p lem en tin g 0011
( | ) to obtain 1100 and then adding 0001. T his yield s 1101, which rep resents j
in tw o s com p lem en t.
From (7.5.6) and (7.5.7) is can easily be seen that
X 2Q = X ]C + 2 ~ H = 2 - \X\

(7.5.8)

T o d em onstrate that (7.5.7) truly represents a n egative num ber, w e use the identity
B

l = J 2 2 ~i + 2 ' B
i=i

(7 -5 -9 )

T h e negative num ber X in (7.5.3) can b e exp ressed as

B

X 2C = - J 2 b i - 2 "' + 1 - 1
1= 1
B

= - 1 + J ] ( l - bi)2~' + 2~B
i=i
= - l + ^ b i -2 -1 + 2~b
i= i

w hich is exactly th e tw o s-com p lem ent rep resen tation o f (7.5.7).

In sum m ary, the value o f a binary string b o h b B d ep en d s on the form at
used. For p ositive num bers, bo = 0, and the num ber is given by (7.5.2). For
negative num bers, w e use th ese corresponding form ulas for the three form ats.
Example 7.5.1
Express the fraction g and | in sign-magnitude, twos-complement, and onescomplement format.

560

Implementation of Discrete-Time Systems

Solution

Chap. 7

X = | is rep resen ted as 2~' + 2 - 2 + 2~3, s o that X = 0 .1 1 1 .

In sign-

m agnitude form at, X = | is rep resen ted as 1.111. In o n e 's co m p lem e n t, we have

X,c = 1.0 0 0
In tw os co m p lem en t, th e result is

X 2C = 1.0 0 0 + 0.00 1 = 1 .0 0 1
T he basic arithm etic op eration s o f addition and m ultiplication d epend on
the form at used. For o n e s-com p lem ent and tw o s-com p lem ent form ats, addition
is carried out by adding the num bers bit by bit. T h e form ats differ only in the way
in which a carry bit affects the M SB. F or exam p le, f = g. In tw o s com p lem en t,
we have
0 1 0 0 1101 = 0 0 0 1
where indicates m o d u lo-2 addition. N o te that the carry bit, if p resent in the
M SB. is dropped. O n the o th er hand, in o n e s- com p lem en t arithm etic, the carry in
the M SB, if present, is carried around to the LSB. T h u s the com p utation g | = g
b ecom es

0100 1100 = 0000 0001 = 0001

A ddition in the sign-m agnitude form at is m ore com p lex and can in volve sign
checks, com plem enting, and the generation o f a carry. O n the other hand, di
rect m ultiplication o f tw o sign- m agnitude num bers is relatively straightforward,
w hereas a special algorithm is usually em p loyed for o n e s co m p lem en t and tw os
com plem ent m ultiplication.
M ost fixed-point digital signal processors use tw o s-com p lem en t arithm etic.
H en ce, the range for ( B + l ) bit num bers is from 1 to 12 - f i . T h ese num bers can
be view ed in a w h eel form at as show n in Fig. 7.32 for B = 2. T w o s-com p lem ent
arithm etic is basically arithm etic m odu lo- 2 B +1 [i.e., any num ber that falls ou tsid e
0.0

-2

0.5

-4
(a)

1.0

(b)

Figure 7.32 Counting wheel for 3-bit twos-complement numbers (a) integers and
(b) functions.

Sec. 7.5

561

Representation of Numbers

the range (overflow or underflow ) is reduced to this range by subtracting an appro*

priate m ultiple o f 2 B+1]. T h is type o f arithm etic can be view ed as cou n tin g using
the w h eel o f Fig. 7.32. A very im portant property o f tw o s- com p lem en t addition
is that if the final sum o f a string o f num bers X \ , X i , . . . , X N is w ithin the range,
it w ill be com p uted correctly, even if individual partial sum s result in overflow s.
T h is and other characteristics o f tw o s-com p lem ent arithm etic are con sid ered in
Problem 7.44.
In g en eral, th e m ultiplication o f tw o fixed-point num bers each o f b bits in
length results in a product o f 2b bits in length. In fixed-point arithm etic, the
product is eith er truncated or rounded back to b bits. A s a result w e have a
truncation or rou n d -off error in the b least significant bits. T he characterization
o f such errors is treated b elow .

7.5.2 Binary Floating-Point Representation of Numbers

A fixed-point representation o f num bers allow s us to cover a range o f num bers,
say* *max *min with a resolution
.

-*max

-Xmin

w h ere m = 2* is the num ber o f levels and b is the num ber o f bits. A basic character*
istic o f the fixed-point represen tation is that the resolution is fixed. Furtherm ore,
A increases in direct p roportion to an increase in the dynam ic range.
A floating-point representation can be em p loyed as a m ean s for covering a
larger dynam ic range. T h e binary floating-point representation com m on ly used
in practice, con sists o f a m antissa M , which is the fractional part o f the num ber
and falls in the range | < M < 1, m ultiplied by the exp on en tial factor 2 E, w here
the exp on en t E is either a p ositive or n egative integer. H en ce a num ber X is
represented as
X = M -2E
T h e m antissa requires a sign bit for representing p ositive and negative num bers,
and the ex p o n en t requires an additional sign bit. Since th e m antissa is a signed
fraction, w e can use any o f the four fixed-point rep resen tation s just described.
For exam p le, the num ber X i = 5 is represented by the follow in g m antissa
and exponent:
M i = 0 .10 10 0 0
= 0 11
w hile the num ber X 2 = | is represented by th e follow in g m antissa and exp on en t
M2 -

0.110000

E 2 = 101

where the leftmost bit in the exponent represents the sign bit.

562

Implementation of Discrete-Time Systems

Chap. 7

If the tw o num bers are to b e m ultiplied, the m antissas are m ultiplied and the
exp on en ts are added. Thus the product o f th e se tw o num bers is
X i X 2 = M \ M 2 2 El+E2
=

( 0 .011110) - 2010

( 0 . 111100) - 2001

O n the other hand, the addition o f the tw o floating-point n um bers requires that
the exp on en ts be equal. T his can b e accom p lish ed by sh iftin g th e m antissa o f the
sm aller num ber to the right and com p en sating by increasing the corresponding
exp on en t. Thus the num ber X 2 can b e exp ressed as
M 2 = 0 .0 0 0 0 1 1
E 2 = O il
W ith E 2 = E], w e can add the tw o num bers X \ and X 2. T h e result is
X i + X 2 = (0.101011) -2 011
It should be ob served that the shifting operation required to equalize the
exp on en t o f X 2 with that for X 1 results in loss o f precision, in general. In this
exam p le the six-bit m antissa w as sufficiently lon g to a ccom m od ate a shift o f four
bits to the right for M 2 w ithout dropping any o f the on es. H ow ever, a shift o f five
bits w ould have caused the loss o f a single bit and a shift o f six bits to the right
w ould have resulted in a m antissa o f M 2 = 0.000000, u n less w e round upward after
shifting so that M 2 = 0.000001.
O verflow occurs in the m ultiplication o f tw o floatin g-poin t num bers w hen the
sum o f the exp on en ts exceed s the dynam ic range o f the fixed -p oin t representation
o f the exp on en t.
In com paring a fixed-point represen tation w ith a floatin g-poin t representa
tion, each w ith the sam e num ber o f total bits, it is apparent that the floating
p oin t representation allow s us to cover a larger dynam ic range by varying the
resolution across the range. T h e resolu tion d ecreases w ith an increase in the
size o f successive num bers. In other words, the distance b etw een tw o successive
floating-point num bers increases as the num bers increase in size. It is this vari
able resolution that results in a larger dynam ic range. A ltern atively, if w e wish
to co v er the sam e dynam ic range with b oth fixed-point and floating-point rep
resentations, th e floating-point representation provid es finer resolu tion for sm all
num bers but coarser resolution for th e larger num bers. In contrast, the fixedp oin t representation provides a uniform resolu tion through ou t the range o f num
bers.
For exam p le, if w e have a com p uter with a w ord size o f 32 bits, it is p ossible
to represent 2 32 num bers. If w e wish to represent the p o sitiv e integers beginning
w ith zero, th e largest p ossib le in teger that can b e accom m od ated is
2 32 1 = 4,294,967,295

Sec. 7.5

563

Representation of Numbers

T h e distance b etw een su ccessive num bers (th e resolu tion ) is 1. A ltern atively, w e
can d esign ate the leftm ost bit as the sign bit and u se the rem aining 31 bits for
th e m agnitu d e. In such a case a fixed-point rep resen tation allow s us to cover the
range
(2 31 - 1) = 2,147,483,647

to

(2 31 - 1) = 2,147,483,647

again with a resolu tion o f 1 .

O n the other hand, su p p ose that w e increase the resolu tion by allocating 10
bits for a fractional part, 21 bits for the in teger part, and 1 bit for the sign. T hen
this representation allow s us to cover the d ynam ic range

( 2 31 1 ) 2 ~ 10 = ( 2 21 2 -t0 )

to

( 2 31 1 ) 2 - 1 0 = 2 21 2 ~ 10

or, eq u ivalen tly,

-2 ,0 9 7 ,1 5 1 .9 9 9

to

2,097,151.999

In this case, the resolution is 2 ~ 10. Thus, th e dynam ic range has b een decreased
b y a factor o f approxim ately 1 0 0 0 (actually 2 10), w hile th e resolution has b een
increased by the sam e factor.
For com parison, su p p ose that the 32-bit w ord is u sed to represent floating
poin t num bers. In particular, let the m antissa be rep resen ted by 23 bits plus a sign
bit and let th e exp on en t b e represented by 7 bits plus a sign bit. N ow , the sm allest
num ber in m agnitude will have the representation,
sign
0,

23 bits
- 0

sign

7 bits

1111111

10 0

= i x 2 127 ^ 0 . 3 x 1 0 ' 38

A t the o th er extrem e, the largest num ber that can be represented w ith this floating
p oin t representation is
sign
0

23 bits
111 -- 1

sign
0

7 bits
1111111 = ( l - 2 23) x 2 m 1.7 x 1038

Thus, w e h ave ach ieved a dynam ic range o f ap p roxim ately 1076, but with varying
resolu tion . In particuiar, w e have fine resolu tion for sm all num bers and coarse
resolu tion for larger num bers.
T h e representation o f zero p o ses so m e special p rob lem s. In general, on ly
th e m antissa has to be zero, but not the exp on en t. T h e ch oice o f M and ,
the representation o f zero, th e handling o f overflow s, and other related issu es
h ave resu lted in various floating-point rep resen tation s o n different digital co m
puters. In an effort to define a com m on floating-point form at, the Institute o f
E lectrical and E lectron ic E n gin eers (IE E E ) in trodu ced th e IE E E 754 standard,
w hich is w id ely used in practice. For a 32-bit m achine, th e IE E E 754 standard
sin gle-p recision , floating-point num ber is rep resen ted as X = ( 1)* 2 - l 27 (Af),
w here

8 9

31

564

Implementation of Discrete-Time Systems

Chap. 7

T his num ber has th e follow in g interpretations:

If E = 255 and M ^ 0, then X is n ot a num ber
If E 255 and M 0, then X ( - 1 ) 5 oo
If 0 < E < 255, then X = ( - 1 ) 5 2 E~ l21( l . M )
If E - 0 and M ^ 0, then X = ( - 1 ) 5 2"I26 (0.Af)
If E = 0 and M = 0, then X = ( - 1 ) 5 0
w here 0 . M is a fraction and l . M is a m ixed num ber with on e in teger bit and 23
fractional bits. F or exam p le, th e num ber

has th e value X =

1 0 0 0 0 0 10

10 1 0

00

1 x 2 13 0 -12 7 x 1 .1 0 1 0 ...0 = 2 3 x y

13.

T h e m agni

tude range o f the 32-bit IE E E 754 floating-point num bers is from 2 ~ m x 2 -2 3 to

(2 2 23) x 2 127 (i.e., from 1,18 x 10 ~ 38 to 3.40 x 1038). C om p utations with num bers
ou tsid e this range result in eith er underflow or overflow .

7.5.3 Errors Resulting from Rounding and Truncation

In perform ing com p utations such as m ultiplications with eith er fixed-point or
floating-point arithm etic, w e are usually faced with the p rob lem o f quantizing a
num ber via truncation or rounding, from a given level o f p recision to a lev el o f
low er precision. T h e effect o f rounding and truncation is to in trodu ce an error
w h ose value d ep en d s on th e num ber o f bits in th e original num ber relative to
the num ber o f bits after quantization. T h e characteristics o f th e errors introduced
through eith er truncation or rounding d ep en d on th e particular form o f num ber
representation.
T o be specific, let us con sid er a fixed-point representation in w hich a num ber
x is quantized from b u bits to b bits. Thus the num ber
bu
x = 0 .1 0 1 1 0 1
consisting o f bu bits prior to quantization is rep resen ted as
b

x = o.ioi i
after quantization, w h ere b < bu. F or exam p le, if x rep resen ts th e sam p le o f
an analog signal, then bu m ay b e taken as infinite. In any ca se if th e quantizer
truncates th e value o f x , th e truncation error is defined as
Et = Q , ( x ) - x

(7.5.10)

Sec. 7.5

Representation of Numbers

565

First, w e con sid er th e range o f valu es o f the error for sign-m agnitude and
tw o s-com p lem ent representation. In both o f th ese representations, the positive
num bers have identical rep resen tation s. For p ositive num bers, truncation results in
a num ber that is sm aller than the unquantized num ber. C on sequ en tly, the trunca
tion error resulting from a reduction o f the num ber o f significant bits from b to b is
- Q T b - 2 ~ b) < E, < 0

(7.5.11)

w h ere the largest error arises from discarding bu b bits, all o f which are ones.
In the case o f n egative fixed-point num bers based on the sign-m agnitude
representation, the truncation error is p ositive, since truncation basically reduces
the m agnitude o f the num bers. C on sequ en tly, for negative num bers, w e have
0 < E, < ( X b - 2 ~ b-)

(7.5.12)

In the tw o s-com p lem en t rep resen tation , the negative o f a num ber is obtained
by subtracting the corresp on d ing p ositive num ber from 2. A s a con seq u en ce, the
effect o f truncation on a negative num ber is to increase the m agnitude o f the
negative num ber. C on sequ en tly, x > Q t (x) and hence
(2~h 2~b) < E, < 0

(7.5.13)

H en ce w e con clu d e that the truncation er ror f o r the sig n-m ag nitu de representation
is sy m m e t r i c a b o u t ze ro a n d falls in the range
- ( 2 ~ h - 2~b) < , < ( 2~h - 2 ~b")

(7.5.14)

O n the other hand, f o r tw o 's-co m p lem e n t representation, the trunc ation error is
a lw a ys neg ative a n d falls in the range
(2~h 2 ~ b ) < E, < 0

(7.5.15)

N ex t, let us con sid er the quantization errors due to rounding o f a num ber. A
num ber x , rep resen ted by bu bits b efore quantization and b bits after quantization,
incurs a q uantization error
r = Q r(x ) - X

(7.5.16)

B asically, rounding in v o lves on ly the m agnitude o f the num ber and, con sequ en tly,
the rou n d -off error is in d ep en d en t o f the type o f fixed-point representation. T he
m axim um error that can be introduced through rounding is (2~ b 2~b,) /2 and this
can be eith er p ositive or n egative, d ep en d ing on the valu e o f x . T h erefore, the
r o u n d - o f f er ro r is s y m m e t r i c a b o u t z e r o a n d falls in the ra nge
- \ { 2 ~ b - 2~ h ) < E r < \ { 2 ~ b - 2~b' )

(7.5.17)

T h ese relation sh ip s are sum m arized in Fig. 7.33 w hen x is a continuous signal
am plitude (b u = oo).
In a floating-point rep resen tation , the m antissa is either rounded or truncated.
D u e to the n onuniform resolu tion , the corresponding error in a floating-point
rep resen tation is p rop ortional to the num ber b ein g quantized. A n appropriate

566

Implementation of Discrete-Time Systems

Chap. 7

QM)

E, - QM) - *
2'
(a )

,= Q,M - x
2-b < , < 0

, = (2,M - x
- 2 -* 5 , < 2 -

(b)

(c )

Figure 733 Quantization errors in rounding and truncation: (a) rounding; (b) truncation
in twos complement; (c) truncation in sign-magnitude.
represen tation for th e quantized value is
Q ( x ) = x + ex

(7.5.18)

w here e is called the relative error. N o w

Q ( x ) - x = ex

(7.5.19)

Sec. 7.5

Representation of Numbers

567

In the case o f truncation based on tw o s-com p lem en t rep resen tation o f the
m antissa, w e have
- 2 E2~b < e,x < 0

(7.5.20)

for p ositiv e num bers. Since 2 _ 1 < x < 2 , it follow s that

- 2~ m

< e, < 0

x > 0

(7.5.21)

O n the other hand, for a negative num ber in tw o s-com p lem en t representation,
the error is
0 < e,x < 2E2~b

and h en ce
0 < e, < 2~ b+l

x <0

(7.5.22)

In the case where the m antissa is rounded, the resulting error is sym m etric
relative to zero and has a m axim um valu e o f 2 ~ hf l . C on sequ en tly, the round-off
error b eco m es
- 2 e 2~hf l < erx < 2 E 2~ hf l

(7.5.23)

A gain , since x falls in the range 2 _ 1 < x < 2 , w e divide through by 2 E~ ] so that
- 2~h < er < 2~b

(7.5.24)

In arithm etic com putations in volving q uantization via truncation and round
ing, it is con ven ien t to adopt a statistical approach to the characterization o f such
errors. T h e quantizer can be m od eled as introducing an additive n oise to the
unquantized value x . T hus w e can w rite
Q(x )

= X

w h ere e = E r for rounding and e = E, for truncation. T h is m od el is illustrated in

Fig. 7.34.
S ince x can be any num ber that falls w ithin any o f the levels o f the quan
tizer, the quantization error is usually m odeled as a random variable that falls
I Quantizer
CM

(a)

{*)---(b)

Figure 734 Additive noise model for

the nonlinear quantization process:
(a) actual system; (b) model for
quantization.

568

Implementation of Discrete-Time Systems

Chap. 7

within the lim its specified. T his random variable is assum ed to be uniform ly
distributed w ithin th e ranges sp ecified for th e fixed-point rep resen tation s. Fur
therm ore, in practice, bu > > b, so that w e can n eglect th e factor o f 2~ b- in
the form ulas given b elow . U n d er th ese con d ition s, the probability density func
tions for the rou n d -off and truncation errors in the tw o fixed-point representations
are illustrated in Fig. 7.35. W e n ote that in the case o f truncation o f the tw o scom plem ent representation o f the num ber, the average value o f the error has a
bias o f 2~ bf2, w h ereas in all other cases just illustrated, th e error has an average
value o f zero.
W e shall use this statistical characterization o f the q uantization errors in our
treatm ent o f such errors in digital filtering and in the com p utation o f the D F T for
fixed-point im plem entation.
p(r)

A=

2~h

Er

2
(a)

P(E,)

2A

A = 2-*
-A

E,

(b)
P<|)

A
A = 2~b
0

(c)

Figure 7J5 Statistical characterization

of quantization errors: (a) round-off
error, (b) truncation error for
sign-magnitude; (c) truncation error for
twos complement.

Sec. 7.6

Quantization of Filter Coefficients

569

7.6 QUANTIZATION OF FILTER COEFFICIENTS

In th e realization o f F IR and IIR filters in hardware or in softw are on a generalpu rp ose com puter, th e accuracy w ith w hich filter coefficients can b e specified is
lim ited by th e w ord length o f the com puter or the length o f the register provided
to store the co efficients. Since the coefficients used in im plem en ting a given filter
are n ot exact, the p o les and zeros o f the system function w ill, in general, be
d ifferent from th e d esired p o les and zeros. C onsequently, w e obtain a filter having
a frequency resp on se that is d ifferent from the frequency resp onse o f the filter with
unquantized coefficients.
In S ection 7.6.1, w e d em onstrate that the sensitivity o f th e filter frequency
resp onse characteristics to quantization o f the filter coefficien ts is m inim ized by
realizing a filter having a large num ber o f p o les and zeros as an interconnection
o f secon d -ord er filter section s. T his lead s us to the parallel-form and cascadeform realizations in which the b asic building blocks are secon d-ord er filter
section s.

7.6.1 Analysis of Sensitivity to Quantization of Filter

Coefficients
T o illustrate the effect o f quantization o f the filter coefficients in a direct-form
realization o f an IIR filter, let us consider a general IIR filter with system
function
M
I > * -
Jt = 0

(7.6.1)

T h e direct-form realization o f the IIR filter w ith quantized coefficients has the
system function

*3=0

(7.6.2)

w h ere th e quantized coefficients {jt}and {a*) can b e related to the unquantized

coefficien ts {*} and [ak ] by th e relations
a t ~ ok + A a*

jfc = 1 ,2

b t = bt + A b t

k = 0, 1 ,,.., M

(7.6.3)

and {Aa*} and {Ai*} represent the quantization errors.

570

Implementation of Discrete-Time Systems

Chap. 7

T h e d en om in ator o f H ( z ) m ay be exp ressed in the form

d (z )

w h ere
H { z ) as

= i + Y l akZ * = n a

~ PkZ ^

(7.6.4)

are th e p o les o f H ( z ) . Sim ilarly, w e can express th e d en om in ator o f

N
(7.6.5)

w here ~pk = p k + A p k, k = 1 , 2 , . . . , N , and A p t is the error or perturbation resulting

from the quantization o f the filter coefficients.
W e shall now relate th e perturbation A p t to the q u an tization errors in the
fa tl.
T he perturbation error Ap, can b e expressed as

w here dp,/dan, the partial derivative o f

with respect to ak, rep resen ts the incre
m ental change in the p ole p, d u e to a change in the coefficien t a k. Thus the total
error Ap, is exp ressed as a sum o f the increm ental errors d u e to ch an ges in each
o f the coefficients {ak }.
T h e partial d erivatives d p t / d a k, k = 1 , 2 , . . . , N , can be ob tain ed by differen
tiating D (z) with respect to each o f the {a*}. First w e have
(7.6.7)

T hen
dPi _ { d D ( z ) / d a k)z=Pi
dak

O D ( z ) / d z ) z=Pl

(7.6.8)

T he num erator o f (7.6.8) is

(7.6.9)
T h e den om in ator o f (7.6.8) is

Sec. 7.6

571

Quantization of Filter Coefficients

(7.6.10)

7 * n (n -p i)
* /=i

T h erefore, (7.6.8) can be expressed as

(7.6.11)

1=1
Substitution o f the result in (7.6.11) into (7.6.6) yield s the total perturbation
error A p, in the form
N

N-k

(7.6.12)

/=]
Mt
T his exp ression provides a m easure o f the sensitivity o f the /th p ole to changes in
the coefficients {a*). A n analogous result can b e ob tain ed for the sensitivity o f the
zero s to errors in the param eters ( it ) .
T h e term s (p,- pi) in th e denom inator o f (7.6.12) represent vectors in the
z -plane from the p o les {/?/} to the p ole p,. If th e p oles are tightly clustered as
th ey are in a narrowband filter, as illustrated in Fig. 7.36, the lengths |p, - p t \ are
sm all for th e p o les in the vicinity o f p, . T h ese sm all len gth s will contribute to large
errors and h en ce a large perturbation error Ap, results.
T h e error Ap, can b e m inim ized by m axim izing the lengths |p,- p/|. T his can
be accom p lish ed by realizing the high-order filter with eith er sin gle-p ole or d o u b le
p o le filter section s. In general, how ever, sin gle-p ole (and sin gle-zero) filter section s
have com p lex-valu ed p o les and require com p lex-valu ed arithm etic op eration s for
their realization. T h is problem can b e avoided by com b inin g com plex-valued p o les
(and zero s) to form secon d-ord er filter section s. Since th e com p lex-valu ed p oles
are usually sufficiently far apart, the perturbation errors {A p,} are m inim ized. A s
a co n seq u en ce, the resulting filter with quantized coefficien ts m ore closely ap
proxim ates th e freq u en cy resp onse characteristics o f th e filter w ith unquantized
coefficients.
It is interesting to n ote that ev en in the case o f a tw o -p o le filter section , the
structure used to realize the filter section plays an im portant role in the errors

572

implementation of Discrete-Time Systems

Chap. 7

Im(z)

Figure 7 3 6

Pole positions for a

bandpass HR filter.

caused by coefficient quantization. T o be specific, let us con sid er a tw o-p ole filter
with system function
H{z) =

1
1 (2 /- c o s 0 )z - 1 + r 2z~ 2

(7.6.13)

This filter has p o les at z = r e j<i. W hen realized as shown in Fig. 7.37, it has two
coefficients, a\ 2 r cos 6 and a 2 = r 2. W ith infinite precision it is p ossib le to
achieve an infinite num ber o f p ole p osition s. Clearly, with finite precision (i.e.,
q uantized coefficients oi and a2), th e p ossib le p ole p osition s are also finite. In
fact, w hen b bits are used to represent the m agnitudes o f a\ and a2, there are at
m ost (2h l ) 2 p ossib le p osition s for the p o les in each quandrant, exclu d in g the
case a\ = 0 and a2 = 0 .
F or exam ple, su p p ose that b = 4. T hen there are 15 p ossib le n on zero values
for a\. T here are a lso 15 p ossib le valu es for r 2. W e illustrate th e se p ossib le valu es
in Fig. 7.38 for the first quandrant o f the z-plane only. T here are 169 p ossib le pole
Mn)

-y(n)

<*>
2rcos 6

Figure 7.37
IIR filter.

Realization of a two-pole

Sec. 7.6

573

Quantization of Filter Coefficients

Figure 7.38 Possible pole positions for two-pole IIR filter realization in Fig. 7.37.
p o sition s in this case. T h e nonuniform ity in their p osition s is d u e to th e fact that
w e are quantizing r 2, w h ereas the p ole p osition s lie on a circular arc o f radius r. O f
particular significance is the sparse set o f p oles for valu es o f 6 near zero and, due to
sym m etry, near 6 = n . T h is situation w ould b e highly unfavorable for low pass fil
ters and highpass filters w hich norm ally h ave p o les clustered near 6 = 0 and 8 = jr.
A n alternative realization o f the tw o*pole filter is th e coupled-form realiza
tion illustrated in Fig. 7.39. T h e tw o cou p led eq u ation s are
> i( ) = x i n ) + r c o s # >j( 1 ) - r s i n # y (n 1 )
(7.6.14)
y (n ) = r sin # y \( n 1 ) + r c o s 0 y (n 1 )
B y transform ing these tw o eq u ation s into the z-d om ain , it is a sim p le m atter to
sh ow that
y (z)
X (z ) ~

( r s in 0 ) z - 1
1 - (2 r c o s 6 )z -

+ r2 z~ 2

574

Implementation of Discrete-Time Systems

. 0

Chap. 7

>i(n)

rcos 9

V|{n - 1 )
rsin 6
t sin 6

V|(n - 1)

Figure 7.39 Coupled-form realization

o f a two-pole IIR filter.

In the cou p led form w e ob serve that there are also tw o coefficients, a i
r s in # and a 2 = r c o s 9. Since they are both linear in r, th e possib le p o le positions
are now equally spaced points on a rectangular grid, as sh ow n in Fig. 7.40. A s
a co n seq u en ce, the p ole p osition s are now uniform ly distributed inside the unit
circle, which is a m ore d esirable situ ation than the p reviou s realization, especially
for low pass filters. (T h ere are 198 possib le p o le p osition s in this case.) H ow ever,
the price that w e pay for this uniform distribution o f p ole p osition s is an increase
in com putations. T h e cou p led -form realization requires four m ultiplications per
ou tp u t point, w h ereas the realization in Fig. 7.37 requires o n ly tw o m ultiplications
p er output point.
It is interesting to com pare the coupled-form realization o f Fig. 7.39 with
the cou p led (or norm al) form state-sp ace structure o f Fig. 7.31. T h e p oles o f the
state-space structure are directly related to its coefficients, sin ce on and a 2 are
the real and im aginary parts o f the roots. Since aj = r co s and a 2 r sin ,
it is clear that quantizing cri and a 2 results in a rectangular grid o f possib le p o le
p osition s, as sh ow n in Fig. 7.40.
Since there are various w ays in which o n e can realize a secon d-ord er filter
section , there are o b v iou sly m any p ossib ilities for d ifferen t p o le locations with
quantized coefficients. Ideally, w e should select a structure that provides us with
a d en se set o f p o in ts in th e region s w here th e p o les b e. U n fortu n ately, how ever,
there is n o sim ple and system atic m eth od for determ ining th e filter realization that
yield s this d esired result.
G iven that a higher-order IIR filter should be im p lem en ted as a com bination
o f secon d-ord er section s, w e still m ust d ecid e w h eth er to em p lo y a parallel config
uration or a cascade configuration. In other w ords, w e m ust d ecid e b etw een the
realization

(7.6.16)

Sec. 7.6

Quantization of Filter Coefficients

575

and the realization

h

(z ) = T

1
CM + C 1Z'
1 + fltiZ"1 + a*2Z-2

(7.6.17)

If the IIR filter has zeros on the unit circle, as is gen erally the case with ellip tic
and C h eb ysh ev type II filters, each secon d-ord er sectio n in th e cascade configu
ration o f (7.6.16) contains a pair o f com p lex-con ju gate zeros. T h e coefficients
{&*} directly d eterm in e the location o f th ese zeros. If th e [by] are quan tized , the
sen sitivity o f th e system resp onse to th e q uantization errors is easily and directly
con trolled by allocating a sufficiently large num ber o f b its to th e representation
o f th e {bki }. In fact, w e can easily evalu ate th e perturbation effe ct resulting from
quan tizin g the coefficients
to so m e sp ecified p recision. T h u s w e h ave direct
con trol o f both th e p o les and th e zeros that result from th e quantization p rocess.
O n th e other hand, the p arallel realization o f H ( z ) provid es direct con trol
o f th e p o le s o f the system only. T h e num erator co efficien ts {chi} and {c*i} d o not

576

Implementation of Discrete-Time Systems

Chap. 7

specify the location o f the zeros directly. In fact, the {c*o) and fc*i} are ob tain ed
by perform ing a partial-fraction exp an sion o f H ( z ). H en ce th ey d o not directly
influence the location o f th e zeros, but on ly indirectly through a com b in ation o f all
th e factors o f H ( z ) . A s a co n seq u en ce, it is m ore difficult to d eterm in e the effect
o f quantization errors in the coefficients {c*, }, on the location o f the zeros o f the
system .
It is apparent that quantization o f the param eters {c*, J is lik ely to produce a
significant perturbation o f the zero p osition s and usually, it is sufficiently large in
fixed-point im plem en tation s to m ove the zeros o ff the unit circle. T his is a highly
undesirable situation, w hich can b e easily rem edied by use o f a floating-point
representation. In any case the cascade form is m ore robust in th e p resen ce o f co
efficient quantization and should b e the preferred ch oice in practical applications,
especially w here a fixed-point representation is em p loyed .
Example 7.6.1
Determine the effect of parameter quantization on the frequency response of the
7-order elliptic filter given in Table 8.11 when it is realized as a cascade of secondorder sections.
Solution The coefficients for the elliptic filter given in Table 8.11 are specified for
the cascade form to six significant digits. We quantized these coefficients to four and
then three significant digits (by rounding) and plotted the magnitude (in decibels)
and the phase of the frequency response. The results are shown in Fig. 7.41 along the
frequency response of the filter with unquantized (six significant digits) coefficients.
We observe that there is an insignificant degradation due to coefficient quantization
for the cascade realization.
Example 7.(L2
Repeat the computation of the frequency response for the elliptic filter considered in
Example 7.6.1 when it is realized in the parallel form with second-order sections.
Solution

The system function for the 7-order elliptic filter given in Table 8.11 is
0 .2 7 8 1 3 0 4 + 0 .0 0 5 4 3 7 3 1 0 8 Z -1
(2) "

1 - 0 .7 9 0 1 0 3 Z '1
- 0 .3 8 6 7 8 0 5 + 0 .3 3 2 2 2 2 9 ;- '
+ 1 - 1 .5 1 7 2 2 3 j-1 + 0 .7 1 4 0 8 8 z ~ 2
0.1 2 7 7 0 3 6 - 0 .1 5 5 8 6 9 6 :- 1
+ 1 - 1 .4 2 1 7 7 3 * -1 + 0 .8 6 1 8 9 5 z 2
- 0 .0 1 5 8 2 4 1 8 6 + 0 .3 8 3 7 7 3 5 6 z " ]

+ 1 - 1.387447*-1 +0.962242z-i
The frequency response of this filter with coefficients quantized to four digits
is shown in Fig. 7.42a. When this result is compared with the frequency response in
Fig. 7.41, we observe that the zeros in the parallel realization have been perturbed
sufficiently so that the nulls in the magnitude response are now at - 8 0 , - 8 5 , and
92 dB. The phase response has also been perturbed by a small amount.

Quantization of Filter Coefficients

577

Phase (degree)

Gain (db)

Sec. 7.6

Figure 7.41 Effect of coefficient quantization of the magnitude and phase response of an
N 7 elliptic filter realized in cascade form.
When the coefficients are quantized to three significant digits, the frequency
response characteristic deteriorates significantly, in both magnitude and phase, as
illustrated in Fig. 7.42b. It is apparent from the magnitude response that the zeros
are no longer on the unit circle as a result of the quantization of the coefficients. This
result clearly illustrates the sensitivity of the zeros to quantization of the coefficients
in the parallel form.
When compared with the results of Example 7.6.1, it is also apparent that the
cascade form is definitely more robust to parameter quantization than the parallel
form.

Implementation of Discrete-Time Systems

Chap. 7

Phase (degree)

Gain (db)

578

(a) Quantization to 4 digits

Figure 7.42 Effect of coefficient quantization of the magnitude and phase response of an
N = 7 elliptic filter realized in cascade form: (a) quantization to four digits; (b) quantization
to three digits.

7.6.2 Quantization of Coefficients In FIR Filters

A s ind icated in the preceding section , the sensitivity analysis p erform ed o n the
p o le s o f a system a lso ap p lies directly to th e zeros o f th e IIR filters. C on sequ en tly,
an exp ression analogou s to (7.6.12) can b e ob tain ed for th e zeros o f an F IR filter.
In effect, w e should gen erally realize F IR filters w ith a large n u m b er o f zeros as

Quantization of Filter Coefficients

579

Phase (degree)

Gain (db)

Sec. 7.6

(b) Quantization to 3 digits

Figure 7.42 Continued

a cascad e o f secon d-ord er and first-order filter section s to m inim ize the sensitivity
to coefficient quantization.
O f particular interest in practice is the realization o f linear p h ase F IR filters.
T h e direct-form realizations show n in Figs. 7.1 and 7.2 m aintain th e linear-phase
property ev en w h en th e coefficients are quantized. T his follo w s easily from the o b
servation that the system function o f a linear-phase F IR filter satisfies th e property
H( z ) =

Implementation of Discrete-Time Systems

580

Chap. 7

in d ep en dent o f w h eth er the coefficien ts are quantized or unq uan tized (see S ec
tion 8.2). C on sequ en tly, co efficien t quantization d o es not affect th e p h ase charac
teristic o f the F IR filter, but affects o n ly the m agnitude. A s a result, coefficient
quantization effects are n ot as severe on a linear-phase F IR filter, since the on ly
effect is in the m agnitude.
Example 7.63
Determine the effect of parameter quantization on the frequency response of an
Af = 32 linear-phase FIR bandpass filter. The filter is realized in the direct form.
Solution The frequency response of a linear-phase FIR bandpass filter with unquantized coefficients is illustrated in Fig. 7.43a. When the coefficients are quantized to
four significant digits, the effect on the frequency response is insignificant. However,
when the coefficients are quantized to three significant digits, the sidelobes increased
by several decibels, as illustrated in Fig. 7.43b. This result indicates that we should use
a minimum of 10 bits to represent the coefficients of this FIR filter and, preferably,
12 to 14 bits, if possible.
From this exam ple w e learn that a m inim um o f 10 bits is required to represent
the coefficients in a direct-form realization o f an F IR filter o f m od erate length. A s
the filter length increases, the num ber o f bits per coefficient m ust be increased to
m aintain the sam e error in the freq u en cy resp onse characteristic o f the filter.
For exam ple, suppose that each filter coefficient is rounded to {b + 1) bits.
T hen the m axim um error in a coefficien t value is bounded as
_ 2 -<fr+t> < e h(n) < 2 ~ (b+l)
Since the quantized valu es m ay b e rep resen ted as h(n) = h (n) + eh(n), th e error in
the frequency resp onse is
= y ^ f ek {n)e'
Since eh(n) is zero m ean, it follow s that E m (to) is also zero m ean . A ssu m in g that
the coefficient error seq u en ce eh(), 0 < n < M - 1, is uncorrelated, the variance
o f the error E m {co) in the frequency resp on se is just the sum o f the variances o f
the M terms. Thus w e have
2 - 2 (i>+i)
2~2(b+2>
cr| = : M = -----------M

12
3
H ere w e note that the variance o f the error in H(a>) increases linearly with M .
H en ce the standard d eviation o f the error in H { a>) is
2 ~(H-2 )
a =
\ /3

C onsequently, for every factor o f 4 in crease in M , th e precision in the filter coeffi

cients m ust b e increased by o n e ad d ition al bit to m aintain th e standard deviation
fixed. This result, taken togeth er w ith th e results o f E xam p le 7.6.3, im plies that

Quantization of Filter Coefficients

581

Gain (dB)

Sec. 7.6

Relative frequency

Gain (dB)

(a) No quantization

(b) Quantization to 3 digits

Figure 7.43 Effect of coefficient quantization of the magnitude of an M = 32 linear-phase
FIR filter realized in direct form: (a) no quantization; (b) quantization to three digits.
th e freq u en cy error rem ains tolerab le for filter lengths up to 256, provided that
filter coefficients are rep resen ted by 12 to 13 bits. If th e w ord length o f the d igi
tal signal processor is less than 12 bits or if th e filter len gth excee d s 256, the filter
sh ou ld b e im plem en ted as a cascad e o f sm aller length filters to reduce the precision
requirem ents.

582

Implementation of Discrete-Time Systems

Chap. 7

In a cascade realization o f the form

K
H ( z ) = G ]" [//* ( 0
i=l

(7.6.18)

w here the secon d-ord er section s are given as

H k(z) = 1 + bklz ~ l + b k2z~ 2
the coefficients o f com p lex-valu ed zeros are exp ressed as b

(7 A 1 9 )
= 2rk cos 6k and

bk2 = r\. Q uantization o f bki and bk2 result in zero location s as sh ow n in Fig. 7.38,
ex cep t that the grid exten d s to p oin ts ou tsid e th e unit circle.
A problem m ay arise, in this case, in m aintaining the lin ear-ph ase property,
b ecau se th e quantized pair o f zeros at z = ( l / r k)ej6k m ay n ot b e th e m irror im age
o f the quantized zero s at z = rke j6k. T his problem can be avoid ed by rearranging
the factors corresponding to the m irror-im age zero. That is, w e can write the
m irror-im age factor as
f l co s0 * z _ 1 + -~rz- 2 ) = -^-(r? 2 rk cos 0 *7 + z ~ 2)

rk

r\

(7.6.20)

T h e factors {1 / r 2} can be com bined with the overall gain factor G, or they can
be distributed in each o f the secon d-ord er filters. T h e factor in (7.6.20) contains
exactly the sam e param eters as the factor ( 1 2rk c o s 0 *z - 1 + r^z- 2 ), and co n se
quently, the zeros now occur in m irror-im age pairs even w hen th e param eters are
quantized.
In this brief treatm ent w e have given the reader an introduction to the
problem s o f coefficient q uantization in IIR and F IR filters. W e have dem onstated that a high-order filter should b e reduced to a cascad e (for F IR or IIR
filters) or a parallel (for IIR filters) realization to m inim ize th e effects o f quan
tization errors in the coefficients. T his is esp ecially im portant in fixed-point re
alizations in w hich the coefficients are represented by a relatively sm all num ber
o f bits.

7.7 ROUND-OFF EFFECTS IN DIGITAL FILTERS

In Section 7.5 w e characterized the q uantization errors that occu r in arithm etic
op eration s p erform ed in a digital filter. T h e p resen ce o f o n e or m ore quantiz
ers in the realization o f a d igital filter results in a n on linear d evice w ith char
acteristics that m ay be significantly different from the id eal linear filter. For
exam p le, a recursive digital filter m ay exh ib it undesirable oscillation s in its out
put, as show n in the follow in g section , even in the ab sen ce o f an input
signal.
A s a result o f the finite-precision arithm etic op eration s perform ed in the
digital filter, so m e registers m ay overflow if th e input signal le v e l b ecom es large.

Sec. 7.7

583

Round-Off Effects in Digital Filters

O verflow rep resents another form o f undesirable n on linear distortion on the d e

sired signal at the output o f the filter. C on sequ en tly, sp ecial care m ust be exercised
to scale th e input signal properly, eith er to p revent overflow com p letely or, at least,
to m inim ize its rate o f occurrence.
T h e non linear effects due to finite-precision arithm etic m ake it extrem ely
difficult to precisely analyze the p erform ance o f a digital filter. T o perform an
analysis o f q uantization effects, w e adopt a statistical characterization o f qu an ti
zation errors w hich, in effect, results in a linear m od el for th e filter. Thus w e are
able to quantify the effects o f quantization errors in th e im plem en tation o f d igi
tal filters. O ur treatm ent is lim ited to fixed-point realization s where quantization
effects are very im portant.

7.7.1 Limit-Cycle Oscillations in Recursive Systems

In the realization o f a digital filter, either in digital hardware or in software on
a digital com p uter, the quantization inherent in the finite- precision arithm etic
op era tio n s render the system nonlinear. In recursive system s, the nonlinearities
due to the finite-precision arithm etic op eration s o ften cau se periodic oscillation s
to occur in the output, even w hen the input seq u en ce is zero or som e n on zero
con stan t value. Such oscillations in recursive system s are called limit cycles and
are directly attributable to round-off errors in m ultiplication and overflow errors
in addition.
T o illustrate the characteristics o f a lim it-cycle oscillation , let us con sid er a
sin gle-p ole system described by the linear d ifferen ce eq u ation
y(n) = ay( n - 1) + x ( n )

(7.7.1)

w h ere th e p o le is at z = a. T h e ideal system is realized as show n in Fig. 7.44. O n

the other hand, the actual system , which is described by the non linear differen ce
eq u ation
v( n) = Q[ av( n - 1)] + x ( n )

(7.7.2)

is realized as show n in Fig. 7.45.

S u p pose that the actual system in Fig. 7.45 is im p lem en ted w ith fixed-point
arithm etic based on four bits for the m agnitude plus a sign bit. T he quantization
that takes p lace after m ultiplication is assum ed to round the resulting product
upward.

Figure 7.44
system.

Idea] single-pole recursive

584

Implementation of Discrete-Time Systems

Chap. 7

Wn)

Figure 7.45 Actual nonlinear system.

In T able 7.2 w e list the resp onse o f the actual system for four different
locations o f the p o le z a, and an input x(n ) = /9S(n), w here fi = 15/16, which
has the binary representation 0.1111. Ideally, the resp onse o f th e system should
decay toward zero exp on en tially [i.e., y( n ) = a" * 0 as n * oo]. In the actual
system , how ever, the resp onse v(n) reaches a steady-state p eriod ic output sequ en ce
with a period that d ep en d s on the value o f the p ole. W hen the p o le is p ositive, the
oscillations occur with a period N p = 1, so that the output reach es a con stan t value
of ^ for o = j and g for a =
O n th e other hand, w hen the p o le is n egative, the
output sequ en ce o scillates b etw een p ositive and negative valu es ( ^ for a = |
for a =
He n c e the p eriod is N p = 2.
T h ese limit cycles occur as a result o f the quantization effe cts in m ultipli
cations. W hen the input seq u en ce x ( n ) to the filter b ecom es zero, the output o f
the filter then, after a num ber o f iterations, enters into the lim it cycle. T h e ou t
put rem ains in the lim it cycle until an oth er input o f sufficient siz e is applied that
drives the system ou t o f the limit cycle. Sim ilarly, zero-input lim it cycles occur
from nonzero initial con d ition s with the input x ( n ) = 0 . TTie am plitudes o f the
output during a lim it cycle are confined to a range o f values that is called the d ead
b a n d o f the filter.
and

TABLE 7.2

a =
~

0 .1 1 1 1

0 .1 0 0 0

LIMIT CYCLES FOR LOWPASS SINGLE-POLE FILTER

0 .1 0 0 0

a =

1 .1 0 0 0
1
2

\ 16 /
(f t)

0 .1 1 1 1
1 .1 0 0 0

0 .0 1 0 0

0 .0 0 1 0

0 .0 0 0 1

(f t)

0 .0 0 0 1

(f t)

0 .0 0 0 1

0 .0 0 0 1

(f t)

1 .0 0 0 1

0 .0 0 0 1

(ft)

0 .0 0 0 1

(f t)
(f t)

(f t)

0 .0 1 0 0
1 .0 0 1 0
0 .0 0 0 1
1 .0 0 0 1
0 .0 0 0 1

(if)
( - ft)

( - ft)
(f t)
(-ft)
(f t)
(- ft)

(ft )

a =
_

0 .1 0 1 1
0 .1 0 0 0
0 .0 1 1 0
0 .0 1 0 1
0 .0 1 0 0
0 .0 0 1 1
0 .0 0 1 0
0 .0 0 1 0
0 .0 0 1 0

0 .1 1 0 0
3
4

(S )
(f t)

(ft)
(f t)
(f t)
(f t)
(f t)
(f t)

(ft)

a =

1 .1 1 0 0
3

0 .1 0 1 1

(B )

1 .1 0 0 0

( - ft)

0 .0 1 1 0
1 .0 1 0 1
0 .0 1 0 0
1 .0 0 1 1
0 .0 0 1 0
1 .0 0 1 0
0 .0 0 1 0

(f t)
(- ft)
(f t)
(-ft)
(f t)
(- ft)

(ft)

Sec. 7.7

Round-Off Effects in Digital Fitters

585

It is interesting to n ote that w h en the resp onse o f the sin gle-p ole filter is in
the lim it cycle, the actual n onlinear system op erates as an equivalent linear system
w ith a p o le at z = 1 w h en the p o le is p ositive and z = 1 w h en the p o le is negative.
That is,
(7.7.3)
Since the qu an tized product av(n 1) is ob tain ed by rounding, it fo llo w s that the
quantization error is b ou n ded as
\Q r[av(n - 1)] - a v ( n - 1)| < \ 2~b

(7.7.4)

w h ere b is th e num ber o f bits (exclu sive o f sign) used in th e representation o f the
p o le a and u(n). C on sequ en tly, (7.7.4) and (7.7.3) leads to
|v(n - 1 )| - \ctv{n - 1 )| < \ 2~b
and h en ce
1

2~b

N " ~ DI < f 7
1 - \a\

(7.7.5)

T he exp ression in (7.7.5) d efines the dead band for a sin gle-p ole filter. F or
ex a m p le, w hen b = 4 and ja| = 1, w e have a dead band with a range o f am plitudes
^ ) . W hen b = 4 and \a\ = 2, th e dead band increases to (g, | ) .
T h e lim it-cycle behavior in a tw o-p ole filter is m uch m ore com plex and a
larger variety o f o scillation s can occur. In this case the ideal tw o-p ole system is
described by the linear differen ce eq u ation ,
(

>>(n) = ai)'(n - 1) + a2y ( n - 2) + jc(n)

(7.7.6)

w h ereas the actual system is described by the non linear d ifferen ce eq u ation
v(n) = Qr [a iv( n - 1)] + Q r f a v i n - 2)] + x ( n )

(7.7.7)

W hen th e filter coefficients satisfy the con d ition a \ < 4o2, the p oles o f the
system occur at
z = r e ie
w h ere a 2 = r 2 and ai = 2 r c o s 0 . A s in the case o f th e sin gle-p ole filter, w hen
the system is in a zero-input or zero-state lim it cycle,
Qr[a2v(n - 2)] = - v ( n - 2)

(7.7.8)

In other w ords, th e system b eh aves as an oscillator with com p lex-con ju gate p o les
on the unit circle (i.e., a2 r 2 = 1). R ou n d ing the product a v ( n 2) im plies
that
| Q r[a2v(n - 2)] - a2v{n - 2 ) \ < \ - 2~ b
U p o n substitution o f (7.7.8) into (7.7.9), w e obtain the result
|u(n - 2 )| - |a2v( n - 2 )| < \ 2~b

(7.7.9)

586

Implementation of Discrete-Time Systems

Chap. 7

or equivalently,
(7.7.10)
T h e expression in (7.7.10) d efines the dead band o f th e tw o-p ole filter with com plexcon gu gate p oles. W e ob serve that the dead-band lim its d ep en d on ly on |ci2 1- T he
param eter a\ = 2r cos 8 d eterm in es th e frequency o f oscillation.
A n oth er p o ssib le lim it-cycle m od e with zero input, w hich occurs as a result
o f rounding the m ultiplications, corresponds to an eq u ivalen t secon d -ord er system
with p o les at z = 1 . In this case it w as show n by Jackson (1969) that the tw o-p ole
filter exh ib its oscillation s with an am plitude that falls in the d ead band b ounded
by 2 - * / ( l - | il - 0 2 )It is interesting to n ote that these lim it cycles result from rounding the prod
uct o f the filter coefficients with the previous outputs, t>( 1 ) and v(n 2 ).
Instead o f rounding, w e m ay ch o o se to truncate the products to b bits. W ith trun
cation, w e can elim in ate m any, although not all, o f the lim it cycles as show n by
Claasen et al. (1973). H ow ever, recall that truncation results in a biased error
unless the sign-m agnitude representation is u sed, in which case the truncation er
ror is sym m etric about zero. In general, this bias is undesirable in digital filter
im plem entation.
In a parallel realization o f a high-order IIR system , each secon d-ord er filter
section exhibits its ow n lim it-cycle behavior, with n o interaction am ong the secondorder filter section s. C on sequ en tly, the output is the sum o f the zero-input limit
cycles from the individual section s. In the case o f a cascade realization for a highorder IIR system , the lim it cycles are m uch m ore difficult to an alyze. In particular,
w hen the first filter section exhibits a zero-input limit cycle, the output lim it cycle
is filtered by the su cceed in g section s. If the frequency o f the lim it cycle falls near a
resonance frequency in a su cceed in g filter section , th e am plitu d e o f the seq u en ce
is en h an ced by the reson ance characteristic. In general, w e m ust be careful to
avoid such situations.
In addition to lim it cycles caused by rounding the result o f m ultiplications,
there are limit cycles caused by overflow s in addition. A n o v erflow in addition
o f tw o or m ore binary num bers occurs w hen the sum e x cee d s the w ord size
available in the digital im plem en tation o f the system . F or exam p le, let us con
sider the secon d-ord er filter section illustrated in Fig. 7.46, in w hich th e addi
tion is perform ed in tw os-com p lem ent arithm etic. Thus w e can w rite the output
y i n ) as
y i n ) = g [ a i y in - 1 ) + a 2y i n - 2 ) + -c(/i)]

(7.7.11)

w h ere the function g [ ] represents th e tw os-com p lem ent ad d ition . It is easily

verified that th e function g(u ) versus v is described by the graph in Fig. 7.47.
R ecall that th e range o f valu es o f the param eters (0 1 , 0 2 ) for a stab le filter
is g iven by the stability triangle in Fig. 3.15. H ow ever, th e se con d ition s are no
longer sufficient to p reven t overflow oscillation w ith tw os-com p lem en t arithm etic.

Sec. 7.7

587

Round-Off Effects in Digital Filters

x(n)

y(n)

<>
Q>

02

Figure 7.46 Two-pole filter realization.

g\v\

Figure 7.47 Characteristic functional relationship for two's complement addition

of two or more numbers.
In fact, it can easily be show n that a necessary and sufficient con d ition for ensuring
that n o zero-input overflow limit cycles occur is
|oi | + \ai | < 1

(7.7.12)

w hich is extrem ely restrictive and h en ce an u n reason ab le constraint to im pose on

an y secon d -ord er section.
A n effectiv e rem edy for curing the prob lem o f overflow oscillation s is to
m o d ify the a d d er characteristic, as illustrated in Fig. 7.48, so that it perform s sat
uration arithm etic. T h u s w h en an overflow (or underflow ) is sen sed , th e output
o f the ad d er w ill be the full-scale valu e o f 1 . T h e distortion caused by this
nonlinearity in the adder is usually sm all provid ed that saturation occurs in fre
q u en tly. T h e u se o f such a nonlinearity d o es n ot p reclu d e the n eed for scal
ing o f the signals and th e system param eters, as d escrib ed in the follow in g se c
tion.

586

Implementation of Discrete-Time Systems

Chap. 7

g(v)

Figure 7.48 Characteristic functional

relationship for addition with clipping at
1.

Saturation arithm etic as just d escribed elim in ates lim it cycles d u e to overflow , on
the o n e hand, but o n the other hand, it cau ses undesirable signal distortion due to
the nonlinearity o f the clipper. In order to limit the am ount o f n on linear distortion,
it is im portant to scale the input signal and the unit sam ple resp onse, b etw een the
input and any internal sum m ing n od e in the system , such that overflow b ecom es
a rare event.
For fixed-point arithm etic, let us first con sid er the extrem e con d ition that
overflow is not perm itted at any nod e o f th e system . Let yk (n ) d en o te the response
o f the system at the Jtth n od e w hen the input seq u en ce is x{n) and the unit sam ple
response b etw een the n od e and the input is h t i n ) . T hen
!>*() I =

h k( m ) x ( n - m )
-oo

oo

S u ppose that jc(n) is upper b ou n ded by A x . T hen

OC
!>*()! < A z

\hk (m)\

for all n

(7.7.13)

N o w , if the dynam ic range o f th e com p uter is lim ited to (1 ,1 ) , th e con d ition

!>*(") I < 1
can be satisfied by requiring that the input x ( n ) b e scaled such that
1

(7.7.14)

IM m)l
for all p ossib le n o d es in th e system . T h e con d ition in (7.7.14) is b oth necessary
and sufficient to p revent overflow .

Sec. 7.7

589

Round-Off Effects in Digital Filters

T h e con d ition in (7.7.14) is overly con servative, h ow ever, to the point w here
the input signal m ay be scaled to o m uch. In such a case, m uch o f the precision
used to represent x ( n ) is lost. T his is especially true for narrowband sequ en ces,
such as sinusoids, w here the scaling im plied by (7.7.14) is extrem ely severe. For
narrowband signals w e can use the frequency resp on se characteristics o f the system
in determ ining the appropriate scaling. Since |//(a>)| rep resents the gain o f the
system at frequency co, a less severe and reason ably ad eq u ate scaling is to require
that
m ax |

(7-7 -15)

(ct>) |

0 <o><;r

w h ere Hk(w) is the Fourier transform o f {/t*(n)).

In the case o f an F IR filter, the con d ition in (7.7.14) reduces to
A, < ^

-------

(7.7.16)

l*t(m )|
m=0
which is now a sum over the M n on zero term s o f the filter unit sam ple response.
A n oth er approach to scaling is to scale the input so that
OO

OO

\yt(n )\2 < C 2

\ x(n)\2 = C 2E x
n=
=
-00
OC
= 00

(7.7.17)

F rom P arsevals theorem

;m w e have
f]

|>-t( ) | 2 =

n ^oo

\H(co)X(co)\2da>

2 * J -*

(7.7.18)

^ /I
B y com b inin g (7.7.17) with (7.7.18), w e obtain
fn

00

T
\hk(n )\2
t^o o

( 1 /2 * ) /
\H(a>)\2da)
J ~n

If w e com pare the d ifferent scaling factors given ab ov e, w e find that

I 1 /2

n = -o o

I M n )|2

00

< m ax|flt(o> )| <

IM )I

(7-7-20)

n-oo

C learly, (7.7.14) is th e m ost pessim istic constraint.

In the fo llo w in g section w e ob serve the ram ifications o f this scaling on the
o u tp u t sign al-to-n oise (pow er) ratio (S N R ) from a first-order and a second-order
filter section.

590

Implementation of Discrete-Time Systems

Chap. 7

7.7.3 Statistical Characterization of Quantization Effects

in Fixed-Point Realizations of Digital Filters
It is apparent from our treatm ent in the previou s section that an analysis o f quan
tization errors in digital filtering, based on d eterm inistic m o d e ls o f quantization
effects, is not a very fruitful approach. T h e basic problem is that the nonlinear
effects in quantizing th e products o f tw o num bers and in clipping the sum o f tw o
num bers to prevent overflow are n ot easily m od eled in large system s that contain
many m ultipliers and m any sum m ing nodes.
T o obtain m ore general results on the q uantization effects in digital filters, w e
shall m od el the quantization errors in m ultiplication as an ad d itive n oise sequ en ce
e ( n ), just as w e did in characterizing th e q uantization errors in A /D con version o f
an analog signal. For addition, w e con sid er the effect o f scaling the input signal
to prevent overflow .
L et us begin our treatm ent w ith the characterization o f th e rou n d -off n oise in
a sin gle-p ole filter w hich is im plem en ted in fixed-point arithm etic and is described
by the nonlinear d ifferen ce eq u ation
v(n) = Qr[av( n 1)] + x ( n )

(7.7.21)

The effect o f rounding the product a v ( n 1) is m od eled as a n oise seq u en ce e(n)

added to the actual product a v(n 1 ), that is,
Q r[av(n 1)] = a v ( n 1) + e ( n )

(7.7.22)

W ith this m odel for the q uantization error, the system under consideration is
described by the linear difference eq uation
v( n) = a v ( n 1) + x ( n ) + e(n)

(7.7.23)

T he corresponding system is illustrated in b lock diagram form in Fig. 7.49.

It is apparent from (7.7.23) that the output seq u en ce v( n ) o f the filter can
be separated in to tw o com p on en ts. O n e is th e resp onse o f th e system to the
input seq u en ce x( n ) . T h e secon d is the resp onse o f the system to the additive
quantization n o ise e( n). In fact, w e can express the output se q u en ce u(n) as a sum
o f th ese tw o com p onents, that is,
v ( n ) = y( n ) + q ( n )

(7.7.24)

Figure 7.49 Additive noise model for

the quantization error in a single-pole
filter.

Sec. 7.7

Round-Off Effects in Digital Filters

591

w h ere y ( n ) rep resen ts the response o f the system to * (n ), and q ( n ) rep resen ts the
resp onse o f th e system to the quantization error e(n). U p o n substitution from
(7.7.24) fo r v(n) in to (7.7.23), w e obtain
y ( n ) + q(rt) = a y(n - 1 ) + a q (n - 1) + je(n) + e(n)

(7.7.25)

T o sim p lify the analysis, w e m ake the follow in g assu m p tion s ab ou t the error
seq u en ce e(n).
1. For any n, th e error sequ en ce {e(n)\ is uniform ly distributed o v er the range
( _ i . 2 ~b, i - 2~b). T his im plies that the m ean value o f e(n) is zero and its
variance is
2 - 2b
e = - f i 0 -1.26 )
2 . T h e error {()) is a stationary w hite n oise seq u en ce. In o th er words, the
error e( n) and the error e(m ) are uncorrelated for n ^ m.
3. T h e error seq u en ce {e(n)} is uncorrelated with the signal se q u en ce {*()}.
T h e last assum ption allow s us to separate the differen ce eq u ation in (7.7.25)
into tw o uncou p led differen ce equations, nam ely,
y (n ) = a y(n - 1) + x (n)

(7.7.27)

q (n) = aq(tt - 1) + e ( n )

(7.7.28)

T h e differen ce equation in (7.7.27) represents th e in p u t-ou tp u t relation for the

desired system and the d ifferen ce equation in (7.7.28) rep resen ts the relation for
th e q uantization error at the output o f the system .
T o co m p lete the analysis, w e m ake use o f tw o im portant relation sh ip s d evel
op ed in A p p en d ix A . T h e first is the relationship for the m ean valu e o f the output
q ( n ) o f a linear shift-invariant filter with im pulse resp onse h{h) w h en excited by a
random seq u en ce e( n) h aving a m ean value m e. T h e resu lt is
00

mq = me

h(rt)

(7.7.29)

fl=00

or, eq u ivalen tly,

m q ~ m eH { 0)

(7.7.30)

w h ere H { 0 ) is the v alu e o f the frequency resp onse H(co) o f th e filter evalu ated at
(o = 0.
T h e seco n d im portant relationship is the exp ression for th e au tocorrelation
seq u en ce o f th e output q ( n ) o f the filter with im pulse resp on se h ( n ) w h en the input
random se q u en ce e{ri) has an autocorrelation y ( n ) . T h is result is
=

i oo/=oo

h ( k ) h ( l ) y te(k l + n)

(7.7.31)

592

Implementation of Discrete-Time Systems

Chap. 7

In th e im portant special case w h ere the random seq u en ce is w h ite (spectrally flat),
the autocorrelation yee(n) is a unit sam p le seq u en ce scaled by the variance a 2,
that is,
Yee(n) = cr2S (n)

(7.7.32)

U p o n substituting (7.7.32) in to (7.7.31), w e ob tain th e desired result for the au to

correlation seq u en ce at the ou tp u t o f a filter excited by w hite n o ise, nam ely,
00

y(n) =

Y , h i k >h ^k + n)

(7 7 33)

fc=00

T he variance a 2 o f the output n oise is sim ply ob tain ed by evaluating y ^ i n ) at

n = 0. Thus
OO

q2 = < Y , h2^

<7 -7-34>

* = s -O C

and w ith the aid o f P arsevals th eorem , w e have the alternative expression
a] = ^

\H(a>)\2du>

(7.7.35)

In the case o f the sin gie-p ole filter under consideration, the unit sam ple
resp onse is
h (n) = a nu ( n )

(7.7.36)

Since the quantization error d u e to rounding has zero m ean, th e m ean value o f
the error at the ou tp u t o f the filter is m q = 0. T h e variance o f the error at the
output o f the filter is
00

*=

(7.7.37)

,2
1 a2

W e observe that the n oise p ow er a 2 at the output o f the filter is enhanced

relative to the input n oise p ow er a 2 by the factor 1 / ( 1 a 2). T his factor increases
as the p o le is m oved closer to th e unit circle.
T o obtain a clearer picture o f the effect o f the quantization error, w e should
also co n sid er the effect o f scaling th e input. L et us assum e that th e input seq u en ce
(.t(n)} is a w hite n o ise sequ en ce (w ideband signal), w h ose am plitude has been
scaled according to (7.7.14) to p revent overflow s in addition. T h en
A , < 1 - |a|
If w e assum e that x i n ) is uniform ly distributed in th e range ( - A * , A x), then,
according to (7.7.31) and (7.7.34), the signal p ow er at the ou tp u t o f the filter

Sec. 7.7

Round-Off Effects in Digital Fitters

593

is
OO
*

(7.7.38)

1 - a2

w h ere a 2 = (1 - |a |) 2/3 is the variance o f the input signal. T h e ratio o f the signal
p ow er a 2 to the quantization error p ow er a 2, w hich is called the signal-to-noise
ratio (S N R ), is sim ply

= d
<*}

(7.7.39)

= ( 1 - | s |)2 2 2(6+l)
T his exp ression for th e output S N R clearly illustrates the severe penalty paid
as a con seq u en ce o f the scaling o f the input, expecially w h en the p ole is near the
unit circle. B y com parison, if the input is not scaled and the adder has a sufficient
num ber o f bits to avoid overflow , then th e signal am plitude m ay b e confined to
the range ( 1 , 1 ). In this case, cr2 =
which is in d ep en d en t o f the p ole position.
T hen
2

% = 2 1(b+l)

(7.7.40)

T h e d ifferen ce b etw een th e S N R s in (7.7.40) and (7.7.39) clearly dem onstrates the
need to use m ore bits in addition than in m ultiplication. T h e num ber o f additional
bits d ep en d s on the p osition o f the p ole and sh ou ld b e increased as the p ole is
m oved closer to the unit circle.
N ex t, let us consider a tw o-p ole filter w ith infinite p recision which is described
by the linear d ifferen ce equation
3>(n) = a i y ( n - 1) + a 2y ( n - 2) + .x(n)

(7.7.41)

w here a\ = 2r co s 6 and a2 = r 2. W h en the tw o products are rounded, w e have

a system which is d escribed by th e non linear differen ce eq u ation
u(n) = Q r[a\v{n - 1)] + Q r[a2v(n - 2)] + x (n )

(7.7.42)

This system is illustrated in block diagram form in Fig. 7.50.

N o w there are tw o m ultiplications, and h en ce tw o quantization errors are
produced for ea ch output. C on sequ en tly, w e sh ou ld introduce tw o n o ise seq u en ces
e\ (n) and e2(n), w hich correspond to the quantizer outputs
Q r[aiv(n - 1)] = a i v ( n - 1) + *i(n )
(7.7.43)
Q r[a2v(n - 2 )] = a2v(n ~ 2 ) + e2(n )

594

Implementation of Discrete-Time Systems

Chap. 7

Figure 7.50 Tw o-pole digital filter with

rounding quantizers.

A block diagram for the corresponding m od el is show n in Fig. 7.51. N o te that the
error sequ en ces e\ (n) and e 2 (n) can b e m oved directly to the input o f the filter.
A s in the case o f the first-order filter, the output o f the secon d -ord er filter
can be separated in to tw o com p onents, the desired signal com p on en t and the
quantization error com p onent. T he form er is described by the differen ce equation
y(n) = a t y ( n - 1) + a 2 y(n 2) + x ( n )

(7.7.44)

w hile the latter satisfies the d ifferen ce eq u ation

q ( n ) = a \ q ( n - l ) + a 2 q (n - 2) + e i ( n ) + e 2 (n)

(7.7.45)

It is reasonable to assum e that the tw o seq u en ces e\( n ) and e 2 (n) are uncorrelated.
N o w the secon d-ord er filter has a unit sam ple resp onse
h ( n ) = sin(/t +
sin 6

) 9 u (n)

(7.7.46)

H en ce
^

X j

j(n)

O'

+ r2

1 _ r 2 r 4 + J __ 2 r 2 c o s 26

(7.7.47)

u(n)

<l(n)

O'0e2(n)

Fijjnre 7.51 A dditive noise model for

the quantization errors in a two-pole
filter realization.

Sec. 7.7

595

Round-Off Effects in Digital Filters

B y applying (7.7.34), w e ob tain the variance o f the quantization errors at the output
o f the filter in th e form
k2

(7.7.48)
In the case o f the signal com p on en t, if w e scale the input as in (7.7.14) to
avoid overflow , the p o w er in the output signal is
a l = o 2l ' Y J h 2 {n)
where the p ow er in the input signal

jc()

(7.7.49)

is given by the variance

1

(7.7.50)

X > ( )l
C on sequ en tly, th e S N R at the output o f the tw o-p ole filter is

_\,
0

cr,:

o:

(^+1 )

(7.7.51]
X

> ( n ) l

A lth ou gh it is difficult to determ in e the exact value o f the den om in ator term
in (7.7.51), it is easy to obtain an upper and a low er bound. In particular, |/7 (n)| is
upper b ounded as
\H n )\ < - J r n
sm 6

n>

(7.7.52)

so that
OO

A;
n=o

OC

-I

|/i(n)j < - Y V = --------1

( 11 - r ) s in e

- s in 0

(7.7.53)

T h e low er bound m ay be ob tain ed by notin g that

I //M l =

y^h(n)e

jam

< X

> ( ) 1

B ut
1

H(a>) =
(1

- r e j s e-j>)( 1 - r e ~ j ee ~ im)

A t a> = $, w hich is th e reson ant freq u en cy o f the filter, w e ob tain the largest value
o f \HU d )\. H en ce
1

(1 r) V 1 + r 2 2r cos 26

(7.7.54)

596

implementation of Discrete-Time Systems

Chap. 7

T h erefore, the S N R is b ounded from a b ove and b elo w according to the relation
2
2 2

>+i)(i _

r ) 2 s in 2

<

2<6+1)( l - r ) 2( 1 + r 2 -

r c o s 2 9)

( 7 . 7 .5 5 )

F or exam ple, w hen 9 = n [ 2, the exp ression in (7.7.55) red u ces to

2
22

<i>+i>( i _

r ) 2

< f l < 22<i,+1>(l - r )2( l + r ) 2

(7.7.56)

T h e dom inant term in this bound is (1 r ) 2 which acts to reduce the S N R

dram atically as the p o les m ove tow ard the unit circle. H en ce the effect o f scaling
in the secon d-ord er filter is m ore severe than in the sin gle-p ole filter. N o te that
if d = 1 - r is the distance o f the p ole from the unit circle, the S N R in (7.7.56)
is reduced by d 2, w hereas in the sin gle-p ole filter the reduction is proportional
to d. T h ese results serve to reinforce the earlier statem ent regarding the use of
m ore bits in addition than in m ultiplication as a m echanism for avoiding the severe
penalty due to scaling.
T he analysis o f the quantization effects in a secon d-ord er filter can be ap
plied directly to higher-order filters b ased on a parallel realization. In this case
each secon d-ord er filter section is in d ep en d en t o f all the other section s, and there
fore the total quantization n oise p ow er at the output o f the p arallel bank is sim ply
the linear sum o f the quantization n oise p ow ers o f each o f th e individual sections.
On the other hand, the cascade realization is m ore difficult to analyze. For the
cascade interconnection, the n oise gen erated in any secon d-ord er filter section is
filtered by the succeeding section s. A s a co n seq u en ce, there is the issue o f how to
pair togeth er real-valued p oles to form secon d -ord er section s and h ow to arrange
the resulting secon d-ord er filters to m inim ize the total n oise p ow er at the output
o f the high-order filter. T h is gen eral top ic was investigated by Jackson (1970a, b),
w h o sh ow ed that p o les close to the unit circle sh ou ld be paired w ith nearby zeros
to reduce the gain o f each secon d-ord er section . In ordering the secon d-ord er
sectio n s in cascade, a reasonable strategy is to place the section s in the order o f
decreasin g m axim um frequency gain. In this case the n oise p o w er gen erated in
the early high-gain section is not b o o sted significantly by the latter sections.
T h e follow in g exam p le illustrates th e p oin t that proper ordering o f section s
in a cascade realization is im portant in con trollin g th e rou n d -off n o ise at th e output
o f th e overall filter.

Example 7.7.1
Determine the variance of the round-off noise at the output of the two cascade
realizations of the filter with system function

H{z) = Hi (z)H2(z)
where
1

Sec. 7.7

597

Round-Off Effects in Digital Filters

m z) =
i- iz -

SolntioD Let h(n), hj(n), and h 2 (n) represent the unit sample responses correspond
ing to the system functions H(z),

and H2 (z), respectively. It follows that

M ) = ( jV ^ n )

M ) = (;)*<()

h(n) = [2 ( 1 )" - (i)"](n)

The two cascade realizations are shown in Fig. 7.52.
In the first cascade realization, the variance of the output is
OO
00
V ( n) + 5 2 ^ (n)
B=eO

nxO

In the second cascade realization, the variance of the output noise is

OO

OO

5 2 h 2 (n) + ^

*i<)

xM

>

vOO

-O
e\(")

K *)
e2(n)

(a) Cascade realization I

(b) Cascade realizationII

Figwe 7.52 Two cascade realizations in Example 7.8.1: (a) cascade realization I;
(b) cascade realization II.

598

Implementation of Discrete-Time Systems

Chap. 7

Now
X > i< " ) =
RbO

l-l

>

(") =
1

16

- 15
4

iYs V ( n ) = i _ i r -
i _ iT + i _ j. = 1-83

Therefore,
= 2.90er?

a22 = 3.16ct;
and the ratio of noise variances is
a\

= 1.09

Consequently, the noise power in the second cascade realization is 9% larger than
the first realization.

7.8 SUMMARY AND REFERENCES

From the treatm ent in this chapter w e have seen that there are various realizations
o f discrete-tim e system s. FIR system s can be realized in a direct form, a cascade
form , a frequency sam pling form, and a lattice form. IIR system s can also be
realized in a direct form , a cascade form , a lattice or a lattice-lad d er form, and in
a parallel form.
For any given system described by a linear con stan t-coefficien t differen ce
equation, these realizations are eq u ivalen t in that they represent the sam e system
and produce the sam e output for any given input, provided that the internal com
putations are perform ed w ith infinite precision. H ow ever, th e various structures
are n o t equivalent w hen they are realized with finite- p recision arithm etic.
T h e state-sp ace form ulation provides an internal d escrip tion o f a system and,
as a con seq u en ce, w e o b tain ed additional system realizations, called state-space re
alizations. T h ese realizations represent additional p ossib le structures that provide
good alternative candidate realizations for th e system .
T hree im portant factors are p resen ted for ch oosin g am on g the various FIR
and IIR system realizations. T h ese factors are com p utational com p lexity, m em
ory requirem ents, and finite-w ord-length effects. D ep en d in g o n eith er th e tim ed om ain or the frequency-dom ain characteristics o f a system , so m e structures m ay
require less com p utation and/or less m em ory than others. H e n c e our selection
m ust co n sid er th e se tw o im portant factors.
M uch research has b een d on e o v er th e p ast tw o d ecad es o n state-sp ace rep
resentation and realization o f system s. For referen ce, w e cite the b ook s by Chen

Sec. 7.8

Summary and References

599

(1970), D e R u sso et al. (1965), Z ad eh and D eso er (1963), and G u p ta (1966). T h e

use o f state-sp ace filter structures in the realization o f IIR system s has b een p ro
p o sed by M ullis and R ob erts (1976a,b), and further d ev elo p ed by H w an g (1977),
Jackson et al. (1979), Jackson (1979), M ills et al. (1981), and B om ar (1985).
In deriving the transposed structures in S ection 7.3, w e introduced several
con cep ts and op eration s on signal flow graphs. Signal flow graphs are treated in
d ep th in th e b ook s by M ason and Zim m erm an (1960) and C how and Cassignol
(1962).
A n o th e r im portant structure for IIR system s, a w a v e digital filter, has b een
in vestigated by F ettw eis (1971) and further d ev elo p ed by S ed lm eyer and F ettw eis
(1973). A treatm ent o f this filter structure can also b e found in the b ook by
A n to n io u (1979).
F in ite-w ord -len gth effects are an im portant factor in th e im plem en tation o f
digital signal processing system s. In this chapter w e described the effects o f a finite
w ord length in digital filtering. In particular, w e con sid ered the follow in g problem s
dealin g with finite-w ord length effects:
1. P aram eter quantization in digital filters
2. R o u n d -o ff n oise in m ultiplication
3. O verflow in addition
4. Lim it cycles
T h ese four effects are internal to the filter and influence th e m eth od by w hich
the system w ill be im plem en ted . In particular, w e d em onstrated that high-order
system s, especially IIR system s, should be realized by using secon d-ord er section s
as building blocks. W e advocated the use o f the direct form II realization, eith er
th e co n v en tio n a l or th e transposed form.
E ffects o f rou n d -off errors in fixed-point im plem en tation s o f F IR and IIR
filter structures have b een in vestigated by m any researchers. W e cite the papers
by G old and R ad er (1966), R ader and G old (1967b), Jackson (1970a,b), Liu (1971),
C han and R ab in er (1973a,b,c), and O p p en heim and W ein stein (1972).
A s an alternative to the use o f direct form II secon d-ord er filters as building
b lock s for high-order filters, w e can use secon d-ord er state-variable form s. Such
state-variable form s can b e op tim ized with respect to th e state transition m atrix
to m inim ize rou n d -off errors. T he op tim ization lead s to m inim u m -rou n d -off-n oise
secon d -ord er state-variable filters that are highly robust for im plem en ting both
narrow band and w ideband filters.
F or a treatm ent o f m inim um -round-off-noise secon d -ord er state-sp ace real
izations, th e reader can refer to the papers o f M ullis and R ob erts (1976a,b), H w an g
(1977), Jackson et al. (1979), M ills et al. (1981), B om ar (1985), and the b o o k by
R ob erts and M ullis (1987).
L im it-cycle o scillation s occur in IIR filters as a result o f quantization effects
in fixed-point m ultiplication and rounding. Investigation o f lim it cycles in d igital
filtering and their characteristic behavior is treated in the p apers by Parker and

600

Implementation of Discrete-Time Systems

Chap. 7

H ess (1971), B rubaker and G ow d y (1972), Sandberg and K aiser (1972), and Jack
son (1969, 1979). T he latter paper deals with lim it cycles in state-sp ace structures.
M eth od s have also b een d evised to elim in ate lim it cycles cau sed by round-off er
rors. For exam ple, the papers by B arnes and Fam (1977), F am and B arnes (1979),
C hang (1981), B utterw eck et al. (1984), and A u e r (1987) discuss this problem .
O verflow oscillation s have b een treated in the pap er by E bert et al. (1969).
T he effects o f param eter quantization has b een treated in a num ber o f papers.
W e cite for reference the work o f R ad er and G old (1967b), K n ow les and O lcayto
(1968), A v en h a u s and S chuessler (1970), H errm ann and Sch u essler (1970b), Chan
and R abiner (1973c), and Jackson (1976).
Finally, w e m en tion that the lattice and lattice-ladder filter structures are
know n to b e robust in fixed-point im plem en tation s. For a treatm ent o f th ese types
o f filters, the reader is referred to the p apers o f G ray and M arkel (1973), M akhoui
(1978), and M orf et al. (1977) and to the b ook by M arkel and Gray (1976).

PROBLEMS
7.1 Determine a direct form realization for the following linear phase filters.
(a) h(n) = {1,2,3,4,3,2,1)
t

(b) h(n) = {1,2 ,3,3,2 ,1}

t

12. Consider an FIR filter with system function

H(z) = 1 + 2.88Z"1 + 3.4048Z"2 + 1.74z"3 + 0.4z~4
Sketch the direct form and lattice realizations of the filterand determine in detail the
corresponding input-output equations. Is the system minimum phase?
73 Determine the system function and the impulse response of the system shown in
Fig. P7.3.
3

Figwe P7J

1A Determine the system function and the impulse response of the system shown in
Fig. P7.4.
7.5 Determine the transposed structure of the systems in Fig. P7.4 and verify that both
the original and the transposed system have the same system function.

Chap. 7

Problems

601

Figure P7.4

7.6 Determine a,, a2, and fi and c0 in terms of b\ and b2 so that the two systems in
Fig. P7.6 are equivalent.
7.7 Consider the filter shown in Fig. P7.7.
(a) Determine its system function.
(b) Sketch the pole-zero plot and check for stability if
(3)
= ^2 = 1,
(2) b{) = >2 = 1,

b\ = 2,

b] = 2,

! = 1.5, fl2 = -0 .9
flj = 1,^2 = 2

(c) Determine the response to x(n) = cos(;rn/3) ifbo = 1, b\ = >2 = 0, a, = 1, and

02 = 0.99.
7.8 Consider an LTI system, initially at rest, described by the difference equation
y(n) = \y(n - 2) + x(n)

(a) Determine the impulse response, h(n), of the system.

(b) What is the response of the system to the input signal
*() = [(j)" + (-f)" ] (n )

(c) Determine the direct form II,parallel-form, and cascade-form realizations for this
system.
<d) Sketch roughly the magnitude response |//(w)| of this system.
7.9 Obtain the direct form I, direct form II, cascade, and parallel structures for the fol
lowing systems.
(a) y(n) = |y (n - 1) - g>>(n - 2) + x(n) + ijc(n - 1)

(b) y(n) = - O .ly O i - 1) + 0.72y(n - 2) + 0.7z(n) - 0.252*(n - 2)

(c) y(n) = -0.1;y(n -1)4- 0.2y( - 2) + 3x(n) + 3.6x(n - 1) + 0.6jt(n - 2)
(d ) H( z ) =

2(1 z-1)(l + -J2z~x + z~2)

(1 + O-Sz-Od - 0.9t-1 + 0.81z-2)

602

Implementation of Discrete-Time Systems

Chap. 7

Figure P7.6
*()

(e) y(n) = \y(n - 1)+ iy(n - 2)+ x(n) + x{n - 1)

(f) y(n) = y(n - l ) - \ y ( n - 2 ) + x ( n ) - x ( n - 1) + x(n - 2)
Which of the systems above are stable?
7.10 Show that the systems in Fig. P7.10 are equivalent.
7.11 Determine all the FIR filters which are specified by the lattice parameters Ki =
K2 = 0.6, K3 = -0.7, and KA= i.
7.12 Determine the set of difference equations for describing a realization of an IIR system based on the use of the transposed direct form II structure for the second-order
subsystems.

Chap. 7

Problems

603

Figure P7.10
7.13* Write a program that implements a parallel-form realization based on transposed
direct form II second-order modules.

7.14* Write a program that implements a cascade-form realization based on regular direct
form II second-order modules.

7.15 Determine the parameters {

of the lattice filter corresponding to the FIR filter

described by the system function

ff(z) = A 2(z ) = 1 + 2 :7 1 + z~2

7.16 (a) Determine the zeros and sketch the zero pattern for the FIR lattice filter with
parameters

Ki = \,
( b )

The same as in part (a) but with

K2 = - l

*3 = 1

= -1 .

(c) You should have found that all the zeros lie exactly on the unit d rd e. Can this
( d )

result be generalized? How?

Sketch the phase response of the filters in parts (a) and (b). What did you notice?
Can this result be generalized? How?

7.17 Consider an FIR lattice filter with coefficients K i = 0.65, K 2 = -0 .3 4 , and K 3 = 0.8.
(a) Find its impulse response by tracing a unit impulse input through the lattice
structure.
( b )
Draw the equivalent direct-form structure.

604

Implementation of Discrete-Time Systems

Chap. 7

7.18 Consider a causal IIR system with system function

1 + 2z-1 + 3z ~2 + 2z ~ 3
1 + 0.9z-1 - 0.8z-2 + 0.5z-3

(a) Determine the equivalent lattice-ladder structure.

(b) Check if the system is stable.
7.19 Determine the input-output relationship, the system function, and plot the pole-zero
pattern for the discrete-time system shown in Fig. P7.19.
rcos $

v(n)

rcos 8

Figure P7.19

7.20 Determine the coupled-form state-space realization for the digital resonator
1

H(z) =
1

- (2 r cos o)z - 1 + r 2 z ~2

7.21 (a) Determine the impulse response of an FIR lattice filter with parameters Aj = 0.6,
Ki = 0.3,
= 0.5, and KA = 0.9.
(b) Sketch the direct form and lattice all-zero and all-pole filters specified by the
^-parameters given in part (a).
7.22 (a) Sketch the lattice realization for the resonator
1

H(z) =
1

- (2 r cos a> o )z-:i + r 2 z ~2

(b) What happens if r = 1?

7.23 Sketch the lattice-ladder structure for the system
1 - 0.8z _ 1 + 0.15z-2
1 + O.lz"1 - 0.72z~2
7.24 Determine a state-space model and the corresponding realization for the following
FIR system:

Chap. 7

605

Problems

7.25 Determine the state-space model for the system described by

y(n)= y(n - 1)+ O.ll^n - 2)+ x(n)

and sketch the type 1 and type 2 state-space realizations.
1J26 Determine the type 1 and type 2 state-space realizations for the Fibonacci system and
its diagonal form.
7.27 By means of the z-transform, determine the impulse response of the system described
by the state-space parameters

ro 0.111

F = Li

ro.11-1

J'

i J-

roi
=LiJ-

. .
<i=1

7.28 Determine the characteristic polynomial of the coupled-form state-space structure

described by (7.4.68) and solve for the roots.
729 Determine the transpose structure for the coupled-form state-space structure shown
in Fig. 7.31.
730 Consider a pole-zero system with system function
(1

(1

- O.Se^'V'Xl - 0.5e - J*/4 z~l )

- 0.8 e J ^ z ~ l ) a - 0 . 8

(a) Sketch the regular and transpose direct form II realizations of the system.
(b) Determine and sketch the type 1 and type 2 state-space realizations.
(c) Determine the impulse response of the system by inverting H(z) and by using
state-space techniques.
(d) Determine the coupled-form state-space realization.
(e) Repeat parts (a) through (d) for the system obtained by changing the angle of
the poles from jt/3 to n/4.
731 (a) Determine a parallel and a cascade realization of the system
(1

- z - ^ l - O -^ ^ Z -O d - O .& r-^ Z -1)

(b) Determine the type 1 and type 2 state-space descriptions of the system in part (a).
7 3 2 Show how to use a lattice structure to implement the following all-pass filter

0.5 + 0.2z _1 - 0.6z - 2 + z - 3

1 - 0.6z- + 0.2z - 2 + 0.5z - 3
Is the system stable?
7 3 3 Consider a system described by the following state-space equations:

T<" + 1 ) = [-0 8 1
y(n) = [-1 .8 1

! ] ' <") + [?]*<">

l]v (n ) + * (n )

(a) Determine the characteristic polynomial and the eigenvalues of the system.
(b) Determine the state transition matrix $ (n ) for n > 0.
(c) Determine the system function and the impulse response of the system.
( d )
Compute the step response of the system if v(0) = [0 1 ]'.
(e) Sketch a state-space re alizatio n for the system.

606

Implementation of Discrete-Time Systems

Chap. 7

7 3 4 Repeat Problem 7.33 if the system is described by the state-space equations

^ J v(n) +

v(n + 1 ) =
y(n) =

[1

j x{n)

] v(n)

7 3 5 Repeat Problem 7.33 for the system described by the state-space equations

, ( " + 1 ) - [ c u
y (n )

= [1

] * (") + [ o ] - ' (")

1 ] v (n ) + x ( n )

7 3 6 Consider the system

y ( n ) = 0 .9 y (n -

1) - 0 .0 8 y (n 2 ) + jc(r) + x ( r t - 1)

(a) Determine the type 1 and type 2 state-space realizations of the system.
(b) Determine the parallel and cascade state-space realizations of the system.
(c) Determine the impulse response of the system by at least two different methods,
7 3 7 Consider the causal system

y(n) = J:Kn - 1) - gyfn - 2)+ x(n) +

- 1)

(a) Determine its system function.

(b) Determine the type 1 state-space model.
(c) Determine the state transition matrix &(n) = F , for any n, using z-transform
techniques.
(d) Determine the system function using the formula

tf<2) = g'UI - F)_1q + <*

Compare the answer with that in part (a).
(e) Compute the characteristic polynomial det(zlF) and check if the system is stable.
7 3 8 Determine the impulse response of the system

- n - .-[?]
using the z-transform approach.
7 3 9 A discrete-time system is described by the following state-space model:

v(/i + 1) = Fv(n) + qx(n)

y{n) = g'v(n) +dx(n)

where

F=[-0ji
i] -[!] -[?] -

(a) Sketch the corresponding state-space structure.

(b) Calculate the impulse response for n = 0, 1 , . . . , 5 and for n == 17 by using the
state-space approach
(c) Find the difference equation description of the system.
(d) Repeat part (b) by using the difference equation.
(e) Sketch the direct form II implementation o f the system.

Chap. 7

607

Problems

7.40 Determine the state-space parameters F, q, g, and d for:

(a) the all-zero lattice structure

(b) the all-pole lattice structure

7.41 The generic floating-point format for a DSP microprocessor is the following:
| sign bit

exponent

The value of the number X is given by

Ol.M x 2
10.M x 2 e

if 5 = 0
if 5 = 1

0
if is the most negative twos-complement value
Determine the range of positive and negative numbers for the following two formats:

12

15

()

24

31

(b)

11

10

23
S

0
short format

M
0

22
M

single-precision format

7.42 Consider the IIR recursive filter shown in Fig. P7.42 and let h F(n), h K(n), and h(n)
denote the impulse responses of the FIR section, the recursive section, and the overall
filter, respectively,
(a) Find all the causal and stable recursive second-order sections with integer coef
ficients (ax, a2) and determine and sketch their impulse responses and frequency
responses. These filters do not require complicated multiplications or quantiza
tion after multiplications.

Figure P7.42

608

Implementation of Discrete-Time Systems

Chap. 7

(b) Show that three of the sections obtained in part (a) can be obtained by intercon
nection of other sections.
(c) Find a difference equation that describes the impulse response h{n) of the filter
and determine the conditions for the overall filter to be FIR.
(d) Rederive the results in parts (a) to (c) using z-domain considerations.
7.43 This problem illustrates the development of digital filter structures using Horners
rule for polynomial evaluation. To this end consider the polynomial
p(x) = atpXp + ap- 1*'-1 -I--- h aix + ao
which computes p(x) with the minimum cost of p multiplications and p additions.
(a) Draw the structures corresponding to the factorizations
Hr(z) = b0(l + b i z - l (l + b2 z - l a + b i z ^ ) ) )
H(z) = bv(z ~ 3 + (] z ~ 2 + (b2 Z~ 2 +!%)))

and determine the system function, number of delay elements, and arithmetic
operations for each structure
(b) Draw the Horner structure for the following linear-phase system:
3

H( z ) = Z ~ l

*o +

7.44 Let x\ and xi be (b + l)-bit binary numbers with magnitude less than 1. To compute
the sum of xi and x2 using twos-complement representation we treat them as (b + 1)bit unsigned numbers, we perform addition modulo-2 and ignore any carry after the
sign bit.
(a) Show that if the sum of two numbers with the same sign has the opposite sign,
this corresponds to overflow.
(b) Show that when we compute the sum of several numbers using twos-complement
representation, the result will be correct, even ifthere are overflows, ifthe correct
sum isless than 1 in magnitude. Illustrate this argument by constructing a simple
example with three numbers.
7.45 Consider the system described by the difference equation
y(n) = ay(n - 1) - ax(n) -I-x(n 1)
(a) Show that it is all-pass.
(b) Obtain the direct form II realization of the system
(c) If you quantize the coefficients of the system in part (b), is it still all- pass?
(d) Obtain a realization by rewriting the difference equation as
y(n) = a[y{n - 1)- x(n)] + x(n - 1)
(e) If you quantize the coefficients of the system in part (d), is it still all-pass?
7.46 Consider the system
y(n) = \y ( n - 1) + *()

(a) Compute its response to the input x(n) = (j)"n(n) assuming infinite-predsion
arithmetic.

Chap. 7

Problems

609

(b) Compute the response of the system y(n), 0 < n < 5 to the same input, assuming
finite-precision sign-and-magnitude fractional arithmetic with five bits (i.e., the
sign bit plus four fractional bits). The quantization is performed by truncation.
(c) Compare the results obtained in parts (a) and (b).
7 .4 7 The input to the system

y(n) = 0.999y(n - 1) + x(n)

is quantized to b = 8 bits. What is the power produced by the quantization noise at
the output of the filter?
7 .4 8 Consider the system

y(n) = 0.875y(n 1) 0.125;y(n 2) -I- x(n)

(a) Compute its poles and design the cascade realization of the system.
(b) Quantize the coefficients of the system using truncation, maintaining a sign bit
plus three other bits. Determine the poles of the resulting system.
(c) Repeat part (b) for the same precision using rounding.
(d) Compare the poles obtained in parts (b) and (c) with those in part (a). Which
realization is better? Sketch the frequency responses of the systems in parts (a),
(b), and (c).
7 .4 9 Consider the system

(a) Draw all possible realizations of the system.

(b) Suppose that we implement the filter with fixed-point sign-and- magnitude frac
tional arithmetic using (b + 1) bits (one bit is used for the sign). Each resulting
product is rounded into b bits. Determine the variance of the round-off noise
created by the multipliers at the output of each one of the realizations in part (a).
7 .5 0 The first-order filter shown in Fig. P7.50 is implemented in four-bit (including sign)

fixed-point twos-complement fractional arithmetic. Products are rounded to four-bit

representation. Using the input x(n) = 0.105(n), determine:

(a) The first five outputs if a = 0.5. D oes the filter go into a limit cycle?
(b) The first five outputs if a = 0.75. Does the filter go into a limit cycle?
7 .5 1 The digital system shown in Fig. P7.51 uses a six-bit (including sign) fixed-point twos-

complement A /D converter with rounding, and the filter H(z) is implemented us

ing eight-bit (including sign) fixed-point twos-complement fractional arithmetic with
rounding. The input x(t) is a zero-mean uniformly distributed random process hav
ing autocorrelation yZJ(t) = 3(r). Assume that the A /D converter can handle input
values up to 1 . 0 without overflow.

610

Implementation of Discrete-Time Systems

Chap. 7

Figure P7.51

(a) What value of attenuation should be applied prior to the A /D converter to assure
that it does not overflow?
(b) With the attenuation above, what is the signal-to-quantization noise ratio (SNR)
at the A /D converter output?
(c) The six-bit A /D samples can be left-justified, right-justified, or centered in the
eight-bit word used as the input to the digital filter. What is the correct strategy
to use for maximum SNR at the filter output without overflow?
(d) What is the SNR at the output of the filter due to all quantization noise sources?
7 .5 2 Shown in Fig. P7.52 is the coupled-form implementation of a two-pole filter with

poles at x = reje. There are four real multiplications per output point. Let c, (n),
i = 1, 2, 3, 4 represent the round-off noise in a fixed-point implementation of the
filter. Assume that the noise sources are zero-mean mutually uncorretated stationary
white noise sequences. For each n the probability density function p(e) is uniform in
the range - A f l < e < A /2 , where A 2~h.
(a) Write the two coupled difference equations for y(/i) and t>(n), including the noise
sources and the input sequence x(rt).
(b) From these two difference equations, show that the filter system functions H\(z)
and H2 (z) between the input noise terms ej(n) + e2 (n) and e3 (n) + et (n) and the
output y(n) are:
rsin<?z-1

Hiiz)
1

~lrCO& 6 z~v + r 2 z ~2

r cos 6 z~l
r c o s# z - 1 + r 2 z ~2

m t)

We know that
H(z) = - --------~ 2 j => Hri) =
1 - 2 TCOS0 Z- 1 + r 2 z ~2

sin(n + l)0u(/i)
sin#

Determine hi(n) and h%(n).

(c ) Determine a closed-form expression for the variance of the total noise from

i = 1, 2, 3, 4 at the output of the filter.

(*)>

Chap. 7

Problems

611

7.53 Determine the variance of the round-off noise at the output of the two cascade real
izations of the filter shown in Fig. P7.53, with system function
H{z) =
where
" =

H 2(z ) =

7.54 Quantization effects in direct-form FIR filters

Consider a direct-form realization of

an FIR filter of length Af. Suppose that the multiplication of each coefficient with
the corresponding signal sample is performed in fixed-point arithmetic with b bits and
each product is rounded to b bits. Determine the variance of the quantization noise
at the output of the filter by using a statistical characterization of the round-off noise
as in Section 7.7.3.

7.55* Consider the system specified by the system function

*(*>

A(z)

la

(1 - Q-8ej," V 1)(l L 1
(1 - h - W + 3Z-1)

1r
(1 + i z - H i - sz~l)
1
JL 2( l - 0 . ^ - > ) ( l - 0 . 8 e- > ^ - 1)J

(a) Choose Gi and G 2 so that the gain of each second-order section at to = 0 is equal
to 1.

612

Implementation of Discrete-Time Systems

Chap. 7

x (n)

Xn)

k>-f
c,(n)

*2 (n)
(a) Cascade realization I

(b) Cascade realization n

Figure P7.53

(b) Sketch the direct form 1 , direct form 2, and cascade realizations of the system.
(c) Write a program that implements the direct form 1 and direct form 2, and compute
the first 1 0 0 samples of the impulse response and the step response of the system.
(d) Plot the results in part (c) to illustrate the proper functioning of the programs.
736* Consider the system given in Problem 7.55 with Gi = G 2 = 1.
(a) Determine a lattice realization for the system
H(z) = B(z)

(b) Determine a lattice realization for the system

1

A{z)
(c) Determine a lattice-ladder realization for the system H(z) = B(z)/A(z).

(d) Write a program for the implementation of the lattice-ladder structure in part (c).
(e) Determine and sketch the first 100 samples of the impulse responses of the sys
tems in parts (a) through (c) by working with the lattice structures.
(f) Compute and sketch the first 100 samples of the convolution of impulse responses
in parts (a) and (b). What did you find? Explain your results.

Chap. 7

Problems

613

7.57* Consider the system given in Problem 7.55.

(a) Determine the parallel-form structure and write a program for itsimplementation.
(b) Sketch a parallel structure using second-order coupled-form state-space sections.
(c) Write a program for the implementation of the structure in part (b).
(d) Verify the programs in parts (a) and (c) by computing and sketching the impulse
response of the system.

Design of Digital Filters

W ith the background that w e have d evelop ed in the p reced in g chapters, w e are
n ow in a position to treat the subject o f digital filter d esign . W e shall describe
several m eth od s for designing F IR and IIR digital filters.
In the design o f freq u en cy-selective filters, the desired filter characteristics
are specified in th e frequency dom ain in term s o f the desired m agnitude and phase
resp onse o f the filter. In the filter design p rocess, w e d eterm in e th e coefficients of a
causal F IR or IIR filter that closely approxim ates the desired frequency response
specifications. T h e issue o f which type o f filter to design, F IR or IIR , depends
on the nature o f the p roblem and on the specifications o f th e desired frequency
response.
In practice, F IR filters are em p loyed in filtering p rob lem s w here there is
a requirem ent for a linear-phase characteristic w ithin the passband o f the filter.
If there is n o requirem ent for a linear-phase characteristic, eith er an IIR or an
F IR filter m ay b e em p loyed . H ow ever, as a gen eral rule, an IIR filter has lower
sid elo b es in the stopband than an F IR filter having th e sam e n u m b er o f parameters.
F or this reason, if som e p h ase distortion is either tolerab le or unim portant, an IIR
filter is preferable, prim arily because its im plem en tation in v o lv es few er parameters,
requires less m em ory and has low er com putational com p lexity.
In conjunction w ith our discussion o f d igital filter d esign , w e describe fre
quency transform ations in b oth the analog and digital d om ain s for transforming a
low pass p rototyp e filter in to either an oth er low p ass, bandpass, bandstop, or highpass filter.
T oday, F IR and IIR digital filter design is greatly facilitated by the availability
o f num erous com puter softw are program s. In describing th e various digital filter
design m eth od s in this chapter, our primary objective is to give the reader the
background n ecessary to select the filter that b est m atches th e application and
satisfies the design requirem ents.

8.1 GENERAL CONSIDERATIONS

In Section 4 .5 , w e d escribed the characteristics o f ideal filters and d e m o n s t r a t e d
that such filters are n ot causal and therefore, are n ot physically realizable. In t 5
614

Sec. B.1

615

General Considerations

section , the issu e o f causality and its im plications is con sid ered in m ore detail.
F ollo w in g this discussion, w e presen t the frequency resp on se characteristics o f
causal F IR and IIR digital filters.

8.1.1 Causality and Its Implications

Let us co n sid er the issue o f causality in m ore d etail by exam ining the im pulse
resp o n se h ( n ) o f an ideal low pass filter with frequency resp onse characteristic
H(a>) =

C0 c < O) < 7T

(8. 1.1)

T h e im pulse resp onse o f this filter is

Ct)c
h(n) =

n
coc sin a>cn

n -

(8.1.2)
n^O

A p lot o f h{n) for <oc = n / 4 is illustrated in Fig. 8.1. It is clear that the ideal
low p ass filter is noncausal and h en ce it cannot be realized in practice.
O ne p ossib le solu tion is to introduce a large delay no in h (n) and arbitrarily
to set h in ) = 0 for n < no- H ow ever, the resulting system no longer has an
id eal frequency resp onse characteristic. In d eed , if w e set h ( n ) 0 for n < n0,
the F ourier series exp an sion o f H(to) results in the G ibbs p h en om en on , as will be
described in Section 8.2.
A lth o u g h this discussion is lim ited to the realization o f a low pass filter, our
con clu sion s hold, in gen eral, for all th e other ideal filter characteristics. In brief,
non e o f the id eal filter characteristics previously illustrated in Fig. 4.43 are causal,
h en ce all are physically unrealizable.
A q u estion that naturally arises at this p oin t is the follow ing: W hat are the
necessary and sufficient con d ition s that a frequency resp onse characteristic H(co)
h (n )

Figure 8.1

Unit sample response of an ideal lowpass filter.

616

Design of Digital Fitters

Chap. 8

m ust satisfy in order for th e resulting filter to b e causal? T h e answ er to this

question is given by the P a ley -W ien er theorem , w hich can b e stated as follows:

Paley-Wiener Theorem. If h(n) has finite energy and h( n ) = 0 for n < 0 ,

then [for a reference, se e W ien er and P aley (1934)]
f | I n \H(to)\\dco < oo
J ir

(8.1.3)

Conversely, if |//(a>)I is square integrable and if th e integral in (8.1.3) is finite, then

w e can associate w ith \H{a>)\ a phase resp onse 0 (o>), so that the resulting filter
w ith frequency response
H{a>) = \H(a>)\em w ]
is causal.
O ne im portant con clu sion that w e draw from the P a le y -W ien er theorem is
that the m agnitude function |//(&>) I can b e zero at som e freq u en cies, but it cannot
be zero over any finite band o f freq u en cies, since the integral th en b ecom es infinite.
C onsequently, any ideal filter is noncausal.
A pparently, causality im poses so m e tight constraints on a linear tim einvariant system . In addition to the P a ley -W ien er con d ition , causality also im plies
a strong relationship b etw een H r {cd) and H r (a>), the real and im aginary com p o
nents o f the frequency resp onse H(o>). T o illustrate this d ep en d en ce, w e d ecom
p o se h (n) into an even and an odd seq u en ce, that is,
h(n) = h e(n) + h a(n)

(8.1.4)

h e(n) = \ [ h ( n ) + h ( - n ) ]

(8.1.5)

h 0 (n) = \ [ h ( n ) - h { - n )]

( 8 . 1 .6 )

w here

and

N ow , if h ( n ) is causal, it is p ossib le to recover h(n) from its ev en part h e{n) for

< n < oo or from its odd co m p on en t h 0 (n) for 1 < n < oo.
Indeed, it can b e easily seen that

h{n) = 2 h r(n)u (n) h e{0)S{n)

n> 0

(8.1.7)

h ( n ) = 2h 0 (n )u (n ) + h (0)S (n)

n > 1

(8.1.8)

and

Since h 0 {n) = 0 for n = 0, w e cannot recover ft(0) from h 0 (n) and h en ce w e also
m ust know /i(0). In any case, it is apparent that h 0 (n) = h e(n) for n > 1, so there
is a strong relationship b etw een h 0 (n) and h t (n).
If h ( n ) is ab solu tely sum m able (i.e., B IB O stab le), th e frequency response
H{co) exists, and

H(o>) = HR(a>) + j H , (co)

(8.1.9)

Sec. 8.1

General Considerations

617

In addition, if h (n) is real valued and causal, the sym m etry p rop erties o f the F ourier
transform im ply that
h e(n)

H R(a>)

h 0 (n)

Hi (to)

( 8 . 1. 10)

Since h(n) is co m p letely specified by h e(n), it follow s that H(to) is com p letely
d eterm in ed if w e k n ow H R(to). A ltern atively, H(to) is com p letely determ in ed
from H j(a >) and fc(0). In short, H R(to) and Hi(to) are in terd ep en d en t and cannot
b e specified in d ep en d en tly if the system is causal. E q u ivalen tly, the m agnitude
and phase resp onses o f a causal filter are in terd ep en dent and h en ce cannot b e
specified ind ep en dently.
G iven H g(w) for a corresponding real, even , and ab solu tely sum m able s e
q u en ce h e(rt), w e can determ in e H(to). T h e follow in g exam p le illustrates the p ro
cedure.
Example 8 .L 1
Consider a stable LTI system with real and even impulse response h(n). Determine
H(w) if
1 a cos to
HR(w) SB ----- ----------- - J<*j < 1
1 2 a cos w + a 1
Solution

The first step is to determine he(n). This can be done by noting that
H r (w ) = H r ( z )

where
H ( )=
RZ

1 ~ Q(z +
_ z - a(z 2 + l ) / 2
1 - a(z + z_1) + a 2
(z - a ) ( l ~ a z )

The ROC has to be restricted by the poles at pi = a and pj = \ / a and should include
the unit circle. Hence the ROC is (oj < |z| < l/|o |. Consequently, ht (n) is a twosided sequence, with the pole at z = a contributing to the causal part and p 2 = 1 /a
contributing to the anticausal part. By using a partial-fraction expansion, we obtain

ht(n) =

ifl11+

\&(h)

(8.1.11)

By substituting (8.1.11) into (8.1.7), we obtain h(n) as

k(n)

a"()

Finally, we obtain the Fourier transform of h(n) as

1
1

ae~lto

T h e relationship b etw een the real and im aginary com p on en ts o f th e Fourier

transform o f an ab solu tely sum m able, causal, and real seq u en ce can be easily
established from (8.1.7). T h e F ourier transform relationship for (8.1.7) is
H(a>) = H r (co) + j H ] ( t o ) = -

H R ( k ) U ( t o - k ) d k ~ h e ( 0)

(8.1.12)

618

Design of Digital Fitters

Chap. 8

w h ere U(u>) is th e Fourier transform o f the unit step seq u en ce u (n). A lth ou gh the
unit step seq u en ce is n ot ab solu tely sum m able, it has a F ourier transform (see
S ection 4.2.8).
U(u)) = n&(a>)+

1
1

- e-J*

(8.1.13)

r/ x + -1 y r1 cot
"

7TS(a))

J2

n < a> < n

B y substituting (8.1.13) in to (8.1.12) and carrying ou t the in tegration , w e obtain

the relation b etw een H R(to) and Hj(a>) as

H,(to) =

H (k) cot

(8.1.14)

T hus HiUo) is uniquely determ in ed from H r (uj) through this integral relationship.
T h e integral is called a discrete H ilb ert tra nsform . It is left as an exercise to the
reader to establish the relationship for H R(a>) in term s o f the discrete H ilbert
transform o f
T o sum m arize, causality has very im portant im plications in the design o f
freq u en cy-selective filters. T h ese are: (a) th e frequency resp on se H(a>) cannot
b e zero, excep t at a finite set o f p oin ts in frequency; (b) th e m agnitude }tf(ti>)|
cannot be con stan t in any finite range o f freq u en cies and the transition from pass
band to stopband cannot b e infinitely sharp [this is a co n seq u en ce o f the G ibbs
p h en o m en o n , w hich results from the truncation o f h (n) to a ch iev e causality]; and
(c) the real and im aginary parts o f H(a>) are in terd ep en dent and are related by the
discrete H ilb ert transform . A s a con seq u en ce, the m agnitu d e \H(a>)\ and phase
0 (<y) o f H(a>) cannot be ch osen arbitrarily.
N o w that w e k n ow th e restrictions that causality im p oses on the frequency
resp onse characteristic and th e fact that ideal filters are not ach ievab le in practice,
w e lim it our atten tion to the class o f linear tim e-invariant system s specified by the
differen ce eq u ation
N

3,(n ) = ~ H ak y(n - *) +
*= 1

bkX<Jl ~ k )
*= 0

w hich are causal and physically realizable. A s w e h ave d em onstrated , such system s
have a frequency response

H(co) =

------------a ke -jwk

(8.1.15)

Jc=l

T h e basic digital filter d esign prob lem is to approxim ate any o f th e ideal frequency
resp onse characteristics w ith a system that has the freq u en cy resp on se (8.1.15), by
properly selectin g the coefficients {ak} and {bt}. T h e approxim ation problem is

Sec. 8.1

General Considerations

619

treated in d etail in Sections 8.2 and 8.3, w h ere w e discuss tech n iqu es for digital
filter design.

8.1.2 Characteristics of Practical Frequency-Selective

Filters
A s w e ob served from our discussion o f the preced in g section , ideal filters are
n oncausal and h en ce physically unrealizable for real-tim e signal processing appli
cations. Causality im plies that the frequency resp onse characteristic H ( a>) o f the
filter cannot be zero, except at a finite set o f p oin ts in the frequency range. In
addition, H ( a>) cannot have an infinitely sharp cu toff from passband to stopband,
that is, H(oj) cannot drop from unity to zero abruptly.
A lth o u g h the frequency resp onse characteristics p ossessed by ideal filters m ay
b e desirable, they are not absolutely necessary in m ost practical applications. If w e
relax th ese con d ition s, it is p ossib le to realize causal filters that approxim ate the
ideal filters as closely as w e desire. In particular, it is n ot necessary to insist that the
m agnitude \H{u>)\ b e constant in the entire passband o f th e filter. A sm all am ount
o f ripple in the passband, as illustrated in Fig. 8.2, is usually tolerab le. Similarly, it
is not necessary for the filter response
(</>)|to be zero in the stopband. A sm all,
n on zero value or a sm all am ount of ripple in the stopband is also tolerable.
T he transition o f the frequency response from passband to stopband defines
the transition b a n d or transition region o f the filter, as illustrated in Fig. 8.2. T he
b an d -ed ge frequency wp defines the ed ge o f the passband, w hile the frequency ojs
d en o tes the beginning o f the stopband. Thus the width o f the transition band is
aij o)p . T h e w idth o f the passband is usually called the b a n d w i d t h o f the filter. For
exam p le, if the filter is low pass with a passband ed ge frequency w p, its bandwidth
is wp .

Figure &2

Magnitude characteristics of physically realizable filters.

620

Design of Digital Filters

Chap. 8

If there is ripple in th e passband o f the filter, its v alu e is d en o te d as 8 lt and

the m agnitude |//(a ))| varies b etw een the lim its 1 i i . T h e ripple in the stopband
o f the filter is d en o ted as S2.
T o accom m odate a large dynam ic range in the graph o f the frequency re
sp o n se o f any filter, it is com m on practice to use a logarithm ic scale for th e m ag
nitude | H(a)) |. C on sequ en tly, the ripple in the passband is 2 01og 10 5] d ecib els, and
that in the stopband is 2 0 1 o g 10 &2 In any filter design p roblem w e can sp ecify (1) the m axim um tolerab le pass
band ripple, (2) the m axim um tolerab le stopband ripple, (3) the passband edge
frequency a>p, and (4) th e stopband ed ge freq u en cy cos. B ased on th e se speci
fications, w e can select the param eters {a*} and {>*} in the freq u en cy response
characteristic, given by (8.1.15), w hich best ap proxim ates th e d esired specification.
T he d egree to which H(co) approxim ates the sp ecification s d ep en d s in part on the
criterion used in th e selection o f the filter coefficien ts {a*} and {&*) as w ell as on
th e num bers ( M , N ) o f coefficients.
In the follow in g section w e presen t a m ethod for designing linear-phase F IR
filters.

8.2 DESIGN OF FIR FILTERS

In this section w e describe several m eth od s for design in g F IR filters. O ur treatm ent
is fo cu sed o n the im portant class o f linear-phase F IR filters.

8.2.1 Symmetric and Antisymmetric FIR Filters

A n F IR filter o f length M with input x ( n ) and output y ( n ) is d escribed by the
differen ce equation
y ( n ) s= box(n) + b \ x ( n -

) -I-------- H bM- \ x ( n - M +

)
(8.2.1)

w here {*} is the set o f filter coefficients. A ltern atively, w e can exp ress the output
seq u en ce as the con volu tion o f the unit sam p le resp onse h (n) o f th e system with
the input signal. T hus w e have
y(n) =

h (k)x(n-k)

(8 .2 .2 )

w here th e low er and upper lim its on the con volu tion sum reflect the causality and
finite-duration characteristics o f the filter. Clearly, (8.2.1) and (8.2.2) are identical
in form and h ence it fo llow s that i>* = h ( k ), k = 0 , 1 , . . . , M ~ 1 .
T h e filter can a lso be characterized by its system function
M-

(8.2.3)

Sec. 8.2

621

Design of FIR Filters

w hich w e view as a p olyn om ial o f d egree Af 1 in the variable z _1. T h e roots o f

this p oly n o m ia l con stitu te the zeros o f the filter.
A n F IR filter has linear p h ase if its unit sam ple resp onse satisfies the con d i
tion
h{n) = h { M - 1 - n)

n = 0,1,..., M - 1

(8.2.4)

W hen th e sym m etry and antisym m etry conditions in (8.2.4) are incorporated into
(8.2.3), w e have
H ( z ) = /i(0) + M D z "

+ h ( 2 ) z ' z + --- + h ( M - 2) z ~(M~2) + h ( M - 1)

M od d

(M /2 ) - l

-m

Y ,

h ( n )[ ziM- l~ 2k)/2 z - ^ - 2k}/2]

M even

(8.2.5)

N o w , if w e sub stitu te z 1 for z in (8.2.3) and m ultiply both sides o f the resulting
eq u ation by z_tw_1), w e obtain
z- (w - i ) H ( z - i ) _ H { Z )
(82>6)
T h is result im plies that the roots o f the polyn om ial H ( z ) are identical to the roots
o f the p olyn om ial H { z - 1 ). C on sequ en tly, the roots o f H ( z ) m ust occur in reciprocal
pairs. In other w ords, if zj is a root or a zero o f H ( z ), then \ / z \ is also a root.
Furtherm ore, if the unit sam ple resp onse h(n) o f the filter is real, com p lex-valu ed
roots m ust occur in com p lex-con ju gate pairs. H en ce, if zi is a com p lex-valu ed
root, z* is a lso a root. A s a con sequ en ce o f (8.2.6), H ( z ) also has a zero at 1 /z j .
Figure 8.3 illustrates the sym m etry that exists in the location of th e zeros o f a
linear-phase F IR filter.
T he frequency resp onse characteristics o f linear-phase F IR filters are o b
tained by evaluating (8.2.5) on th e unit circle. This substitution yield s the exp res
sion for H ( w ) .

Figure &3 Symmetry o f zero locations

for a linear-phase FIR filter.

622

Design of Digital Filters

Chap. 8

W hen h i n ) = h ( M - 1 n), H{a>) can be expressed as

H(to) = H r (co)e-

(8.2.7)

jm iM ~ 1)/2

w here H r(a>) is a real function o f co and can be expressed as

fu / __ i \

(M - 3 ) f2

^2
n= 0

^ ( n ) c o s a ) ^ -------- n j

__ i

M odd

( 8 .2 .8)

(Af/2)1

Hr (co) = 2

L7
( M 1
\
^
h (n) cosco ( -------- n j
=o
\
'

M even

(8.2.9)

T he phase characteristic o f the filter for both M odd and M ev en is

-to

to

i f t f r fcu) > 0

(8.2.10)

@(<o) =
71 *

i i H

r ( a > ) < 0

W hen
h(n) = h ( M -

- n)

the unit sam ple resp onse is antisym m etric. For M odd, the cen ter p oin t o f the
antisym m etric h(n) is n = ( M - l ) / 2 . C onsequently,

H ow ever, if M is ev en , each term in h( n ) has a m atching term o f o p p o site sign.

It is straightforward to sh ow that the frequency response o f an F IR filter with
an antisym m etric unit sam ple resp onse can be exp ressed as
H(to) = Hr (co)e*-a>{M- ' )rz+1' f t

(8.2.11)

where
( -3 )/2

/ M

- 1

Hr (co) 2

^
o

A( n ) s i n a > ( -------- n l
\
2
/

M od d

(8.2.12)

H r(io) = 2

^
=o

A ( n ) s i n o ; f -------- n J
\
2
/

M ev en

(8.2.13)

The phase characteristic o f the filter for b oth Af od d and Af ev e n is

n
2

( M - 1\
"_ < w ( 2 )

'&Hr (a>)>

(o>) =

(8.2.14)
T ~ w( ~

~ )

H H r (co)

< 0

T h ese general freq u en cy resp onse form ulas can b e used to design linearphase F IR filters w ith sym m etric and antisym m etric unit sam ple resp onses. W e

Sec. 8.2

Design of FIR Filters

623

n o te that, for a sym m etric h ( n ), the num ber o f filter coefficien ts that specify th e
frequency resp onse is (Af + l ) / 2 w hen M is odd or M / 2 w h en M is even . O n the
other hand, if the unit sam ple resp onse is antisym m etric,

so that there are (Af l ) / 2 filter coefficients w hen M is odd and M / 2 coefficients
w h en M is ev en to be specified.
T he ch o ice o f a sym m etric or antisym m etric unit sam p le resp on se dep en d s
on the application. A s w e shall se e later, a sym m etric unit sam ple resp onse is
suitable for so m e applications, w h ile an antisym m etric unit sam p le resp onse is
m ore su itab le for other applications. For exam p le, if h(n) = h ( M - 1 - ri) and M
is odd, (8.2.12) im plies that Hr (0) = 0 and Hr ( n ) = 0. C on sequ en tly, (8.2.12) is n ot
suitable as eith er a low pass filter or a highpass filter. Sim ilarly, the antisym m etric
unit sam ple resp onse w ith M even also results in H r (0) = 0, as can b e easily verified
from (8.2.13). C onsequently, w e w ould not use the antisym m etric con d ition in the
design o f a low pass linear-phase F IR filter. O n the other hand, the sym m etry
con d ition h(n) = h ( M 1 - n) yields a linear-phase F IR filter w ith a nonzero
resp onse at to = 0 , if desired, that is,
/ Af 1 \
w -W
tfr(0) = / i ( - 1 + 2
h{n),

M odd

(8.2.15)

Hr (0) = 2

M even

(8.2.16)

2 /

n=0

J
n= 0

h ( n ),

In sum m ary, the problem o f F IR filter design is sim ply to d eterm in e the Af
coefficients A( n) , n = 0 , l , . . . , A f 1 , from a specification o f the desired frequency
resp onse Hj(to) o f th e F IR filter. T he im portant param eters in the specification o f
Hd (co) are given in Fig. 8.2.
In the follow in g subsections w e describe design m eth od s based on specifica
tion o f Hjico).

8.2.2 Design of Linear-Phase FIR Filters Using Windows

In this m eth od w e b egin with the desired frequency resp onse specification Hj(to)
and determ in e the corresponding unit sam ple resp onse h j ( n ) . In d eed , hd {n) is
related to Ha(a>) by the F ourier transform relation

ff*(o>) = f > ( " ) * '"

(8.2.17)

ItsO

where
1

f*

hd {n) = /
2j t

Hd {co)ejamd(o

(8.2.18)

T hus, given Hj((o), w e can d eterm in e th e unit sam p le resp on se h j ( n ) by evaluating

the integral in (8 .2 .1 8 ).

624

Design of Digital Filters

Chap. 8

In general, the unit sam ple resp onse h d {n) ob tain ed from (8.2.17) is infinite
in duration and m ust be truncated at som e p oin t, say at n = M 1 , to yield an
F IR filter o f length Af. Truncation o f h d (n) to a length M 1 is equivalent to
m ultiplying h d (n) by a rectangular w in d ow , defined as

" " - { i

othervrise " ^

<8 '2 1 9 )

- 1

(8 .2 .2 0 )

Thus the unit sam ple response o f the F IR filter b ecom es

h{n) = h d (n )w in )
_ | h d in),

n =

{0,

, 1 .........M

otherw ise

It is instructive to con sid er the effect o f the w indow fun ction on the d e
sired frequency response Hd ito). R ecall that m ultiplication o f th e w indow function
w i n ) with h d (n) is equivalent to con volu tion o f H d (w) with Wi<o), w here W(<o) is
the frequency-dom ain representation (Fourier transform ) o f th e w indow function,
that is,
A f-1

W i w ) = Y i w ( n ) e - jwn
n= 0

(8.2.21)

T hus the con volu tion o f H d i<o) with W i w ) yields the freq u en cy resp onse of the
(truncated) F IR filter. That is,
1
f*
H i w ) = / H d( v ) W ( w - v ) d v
2n J.

(8.2.22)

T h e Fourier transform o f the rectangular w indow is

Af-1

W (w ) = Y e ~ jan
n* 0

(8.2.23)
=

=
1

sin(a>M/2 )
sin(a>/2 )

e~ iw

T his w indow function has a m agnitude response

\W(<o)\ =

|sm (<u/ 2 )|

- 7t < w < 7t

(8.2.24)

and a p iecew ise linear phase

^ ~ 2~ )

w h en sin(uAf/2) > 0

}M 1\
& I - I + n ,

w h en sm (tuAf/2) < 0

co

( oj) =

(8.2.25)

T h e m agnitude resp onse o f th e w in d ow function is illustrated in F ig. 8.4 for M = 31

and 61. T he w idth o f the m ain lob e [width is m easured to th e first zero o f W(tu)]

Sec. 8.2

625

Design of FIR Filters

Figure 8.4 Frequency response

for rectangular window of lengths
(a) M = 31, (b) M = 61.

is 4j t / M . H en ce, as M increases, the m ain lob e b ecom es narrower. H ow ever, the

sid elo b es o f |W(a>)| are relatively high and rem ain u n affected by an increase in M .
In fact, even though the w idth o f each sid elob e d ecreases with an increase in M ,
the height o f each sid elo b e increases with an increase in M in such a m anner that
the area under each sid elob e rem ains invariant to changes in M . T h is character
istic b eh avior is not evid en t from observation o f Fig. 8.4 b ecau se W ( oj) has b een
norm alized by M such that the norm alized peak valu es o f the sid elob es rem ain
invariant to an increase in M .
T h e characteristics o f the rectangular w in d ow play a significant role in d eter
m ining the resulting frequency resp onse o f the F IR filter obtained by truncating
hd (n) to length M . Specifically, th e con volu tion o f Hd {u)) with W(a)) has the effect
o f sm ooth in g
A s M is increased, W(a) b ecom es narrower, and the sm o o th
ing provided by W(cv) is reduced. O n the other hand, th e large sid elob es o f W(a>)
result in so m e undesirable ringing effects in th e F IR filter frequency resp onse
H(co), and also in relatively larger sid elob es in H(a>). T h ese undesirable effects
are b est alleviated by th e use o f w indow s that d o not con tain abrupt discon tin u
ities in their tim e-d om ain characteristics, and h ave correspondingly low sid elob es
in their frequency-dom ain characteristics.
T ab le 8.1 lists several w in d ow functions that p ossess desirable frequency re
sp on se characteristics. Figure 8.5 illustrates th e tim e-d om ain characteristics o f th e
w indow s. T h e frequency resp onse characteristics o f the H anning, H am m ing, and
B lackm an w in d ow s are illustrated in Figs. 8 . 6 through 8 .8 . A ll o f th ese w indow
fun ction s h ave significantly low er sid elob es com pared w ith the rectangular w in
dow . H o w ev er, for the sam e value o f M , the width o f the m ain lob e is also w ider
for th e se w in d ow s com pared to the rectangular w indow . C on sequ en tly, these w in
d ow fun ction s provide m ore sm ooth in g through the con volu tion operation in the
frequency dom ain, and as a result, th e transition region in th e F IR filter resp onse
is w ider. T o reduce the w idth o f this transition region, w e can sim ply increase the
length o f th e w indow w hich results in a larger filter. T ab le 8.2 sum m arizes th ese
im portant frequency-dom ain featu res o f th e various w in d ow functions.
T he w in d ow tech n iqu e is b est described in term s o f a specific exam ple. Sup
p o se that w e w ant to design a sym m etric low p ass linear-phase F IR filter having a

626

Design of Digital Filters

TABLE 8.1

Chap. 8

WINDOW FUNCTIONS FOR FIR FILTER DESIGN

Name of
window

Time-domain sequence,
h (n ),

0<

n < M

M -l
Bartlett (triangular)

M-

Blackman

0.42 0.5 cos

Hamming

+ 0.08 cos ^

M -l '

0.54 0.46 cos

1 /.

M -l

2n n
M -l

2n n \

- 11 c o s ------- r
2 \
M-lJ

Hanning

Kaiser

* [.(!? )]

Lanczos

L >0

, 1

A/ 1 ]

1. ------- <
I
2
I
Tukey
2 L

M -l

-------

0 < or <

) ( M l)/2
(1 aot)(M
l) /2

a(A / l) /2 < n

Af-1 I

)\
A /-1

d esired frequency response

tfrfM = (

1 0,

s M < o>c

(8

.2 6 )

o t h e r w is e

A d elay o f ( M l ) / 2 units is incorporated in to Hd (w) in anticipation o f forcing

the filter to be o f length M . T he corresp on d ing unit sam ple resp on se, ob tain ed by
evaluating the integral in (8.2.18), is
h d (n) = ^ - f
e ^ - ^ d c o
2 n J _ Uc

- _1
/

M - l\

2 )

Clearly, h d (n) is noncausal and infinite in duration.

n *

(*2.27,

Sec. 8.2

Design of FIR Fitters

------------------------------- 7--------/ /

0.8

f t

f t
* Z Hanning

0.6

0.4

0.2

627

// /
// *.

Blackman

Rectangular
\

y Hamming
\

\ \\
' *\
\ \\
\ W
V <v.

M- 1

Shapes of several window

Magnitude (dB )

Figure 8J5

Magnitude (dB)

Figure 8.6 Frequency responses of

Hanning window for (a) M 31 and
(b) M = 61.

Figure 8.7 Frequency responses for

Hamming window for (a) M = 31 an<
(b) M = 61.

628

Design of Digital Filters

Chap. 8

Figure &8 Frequency responses for

Blackman window for (a) M = 31 and
(b) M = 61.
TABLE 8.2 IMPORTANT FREQUENCY-DOMAIN
CHARACTERISTICS OF SOME WINDOW FUNCTIONS

Type of window

Approximate
transition width of
main lobe

Peak sidelobe
(dB)

Ait/M
&7T/M
%ixjM
&JI/M
YI tz/ M

-1 3
-2 7
-3 2
-4 3
-5 8

Rectangular
Bartlett
Hanning
Hamming
Blackman

If w e m ultiply h j ( h ) by the rectangular w indow seq u en ce in (8.2.19), w e

obtain an F IR filter o f length M having the unit sam ple resp onse
sin a),
h (n) =

< n < M 1

(8.2.28)

If M is selected to b e odd, the value o f h ( n ) at n = ( M Y)/2 is

. (

) , =

(8.2.29)

T h e m agnitude o f the frequency resp onse H(o)) o f this filter is illustrated

in Fig. 8.9 for M = 61 and M = 101. W e o b serve that relatively large oscilla
tion s or ripples occur near the band ed ge o f th e filter. T h e oscillation s increase in
freq u en cy as M increases, but they d o n ot dim inish in am plitude. A s indicated pre
viously, these large oscillation s are the direct result o f the large sid elo b es existing
in th e frequency characteristic W (a>) o f th e rectangular w in d ow . A s this window
fun ction is co n v o lv ed with th e desired freq u en cy resp onse characteristic
the oscillation s occur as the large con stan t area sid elob es o f W ( cd) m ove across
th e discontinuity that exists in
Since (8.2.17) is basically a F ourier series
represen tation o f
the m ultiplication o f h j ( n ) with a rectangular w in d ow is
id en tical to truncating the Fourier series rep resen tation o f th e desired filter char-

Sec. 8.2

Design of FIR Fitters

629

(a)

0.8
n
a
0.6
oc
2 0.4
o
O 1

0.2

M = 101

0.2

0.3

Normalized frequency
(b)

0.4

Figure 8.9 Lowpass filter designed with

a rectangular window (a) M = 61 and
(b) M = 101.

acteristic H j (co). T he truncation o f th e F ourier series is know n to introduce ripples

in the frequency resp onse characteristic H(co) du e to th e nonuniform con vergen ce
o f the F ourier series at a discontinuity. T h e oscillatory b eh avior near the band
ed g e o f the filter is called the G i b b s p h e n o m e n o n .
T o a lleviate the p resence o f large oscillation s in b oth the passband and th e
stopband, w e should use a w in d ow function that contains a taper and decays to
ward zero gradually, instead o f abruptly, as it occurs in a rectangular w indow .
F igures 8.10 through 8.13, illustrate the frequency resp onse o f the resulting filter
w h en so m e o f the w indow fun ction s listed in T ab le 8.1 are used to taper h d (n). A s
illustrated in Figs. 8.10 through 8.13, the w in d ow functions d o indeed elim inate the
ringing effects at the band ed ge and d o result in low er sid elo b es at the exp en se o f
an increase in the w idth o f the transition band o f the filter.

Fignv 8.10 Lowpass FIR filter

designed with rectangular window
(M = 61).

630

Design of Digital Filters

Chap. 8

Figure 8.11 Lowpass FIR filter

designed with Hamming window
(Af = 61).

Figure 8-12 Lowpass FIR filter

designed with Blackman window
(Af = 61).

Figure 8.13 Lowpass FIR filter

designed with a = 4 Kaiser wind*
(Af = 61).

8.2.3 Design of Linear-Phase FIR Filters by the

Frequency-Sampling Method
In the frequency sam pling m eth od for F IR filter design, w e specify the desired
frequency resp onse Hj(co) at a set o f equally sp aced freq u en cies, nam ely
Vk = tj-(* + )

* =

,1,

M -l
M

M od d
(8.2.30)

and solve for the unit sample response h ( n ) of the FIR filter from these equally

Sec. 8.2

Design of FIR Filters

631

spaced frequency specifications. T o reduce sid elob es, it is desirable to optim ize the
frequency specification in the transition band o f the filter. T his optim ization can be
accom plished num erically on a digital com p uter by m eans o f linear program m ing
tech n iqu es as show n by R abiner et al. (1970).
In this section w e exploit a basic sym m etry property o f th e sam pled frequency
resp onse function to sim plify the com putations. L et us begin with the desired
frequency resp onse o f the F IR filter, w hich is [for sim plicity, w e drop the subscript
in Hd (a>)],
M-

H(a>) =

h ( n )e ~ iwn

(8.2.31)

=o
Su p pose that w e specify the frequency resp on se o f the filter at the freq u en cies
given by (8.2.30). T hen from (8.2.31) w e obtain
H (k + a) = H ^ - i k + a y j

H (k + a) s

M -1
Y h ( n ) e ~ i2n{k+a)n,M
n= 0

k = 0 , 1 , . . . . Af 1

(8.2.32)

It is a sim ple m atter to invert (8.2.32) and express h{n) in term s o f H ( k + a ) .

If w e m ultiply both sid es of (8.2.32) by the exp on en tial, z x p ( j 2 x k m / M ) , m 0,
1 . . . . , M 1, and sum over k = 0, 1 , Af ~ 1, the right-hand side o f (8.2.32)
reduces to A f h ( m ) e x p ( j 2 n a m / A f ) . Thus w e obtain
i M -1
= 77 Y L H ( k + ct)ej2,T(k+a)nlM
*=o

n = 0,1,...,A/ 1

(8.2.33)

T h e relationship in (8.2.33) allow s us to com p ute the valu es o f th e unit sam p le

resp onse h (n) from the specification o f the frequency sam ples H ( k + a ) , k = 0 ,
1 . . . . , M 1. N o te that w hen a = 0, (8.2.32) reduces to the discrete F ourier
transform (D F T ) o f the seq u en ce [h(n)} and (8.2.33) red u ces to the inverse D F T
(ID F T ).
Since (fc(/i)} is real, w e can easily show that the freq u en cy sam ples { H ( k + a ) }
satisfy the sym m etry condition
H (k + a) = H*(Af k a)

(8.2.34)

T h is sym m etry co n d ition , along with the sym m etry con d ition s for \h (n )), can b e
u sed to reduce the frequency specifications from M p oin ts to ( M + l ) / 2 p oin ts for
M od d and M f l p oin ts for M even . T hus the linear eq u ation s for determ ining
{h(n)} from [ H ( k + a )} are considerably sim plified.
In particular, if (8.2.11) is sam pled at the freq u en cies
= 2n ( k + a ) / M ,
k = 0, 1 , . . . , Af 1, w e obtain
/ 2 jt
H ( k + a ) = Hr l (k + a ) J

(8.2.35)

632

Design of Digital Filters

Chap. 8

w here j3 = 0 w h en
is sym m etric and p = 1 w hen {h(n)} is antisym m etric. A
sim plication occurs by defining a set o f real frequency sam ples { G (k + m)}
G ( k + a ) = ( ~ l ) kH r

k = 0 , 1 .........M - l

(8.2.36)

W e u se (8.2.36) in (8.2.35) to elim in ate Hr (a>k). T h u s w e ob tain

H (k + a) = G ik +

(8.2.37)

N o w the sym m etry con d ition for H ( k + a ) given in (8.2.34) tran slates into a corre
sponding sym m etry con d ition for G (k + a ), which can be ex p lo ited by substituting
into (8.2.33), to sim plify th e expressions for the F IR filter im pu lse resp onse {h(n))
for the four cases ar = 0, a = j ,
= 0, and f$ = \ . T h e results are sum m arized in
T able 8.3. T h e d etailed derivations are left as exercises for the reader.
A lth o u g h the frequency sam pling m eth od provides us w ith another m ean s for
designing linear-phase F IR filters, its major advantage lies in th e efficien t frequency
sam pling structure, w hich is obtained w hen m ost o f the freq u en cy sam ples are zero,
as dem onstrated in S ection 7.2.3.
T h e follow in g exam p les illustrate the design o f linear-phase F IR filters based
on the frequency sam pling m ethod. T h e optim um valu es for th e sam ples in the
transition band are ob tain ed from the tables in A p p en d ix C w h ich are taken from
the paper by R abiner et al. (1970).
Example 8 .2 . 1
Determine the coefficients of a linear-phase FIR filter of length M 15 which has a
symmetric unit sample response and a frequency response that satisfies the conditions
1,
0.4,
0.

* = 0 ,1 , 2, 3
k=4
k = 5 ,6 , 7

Solution Since h(n) is symmetric and the frequencies are selected to correspond to
the case a = 0, we use the corresponding formula in Table 8.3 to evaluate h(n). In
this case
G(k) = ( ~ l ) kHr

' 2n k \
)
u j

k = 0 , 1 ........7

The result of this computation is

*(0) = A(14)
A(l) = A(13)
h (2) = A(12)
A(3) = A (ll)
h( 4) = *(10)
h( 5) = A(9)
h{6) = A(8)
A(7)

= -0.014112893
= -0.001945309
= 0.04000004
= 0.01223454
-0.09138802
= -0.01808986
= 0.3133176
= 0.52

Sec. 8.2

633

Design of FIR Filters

TABLE 8.3 UNIT SAMPLE RESPONSE: h(n) = h{M ~ 1 - n)

Symmetric
H(k) = G(k)e>*k/M

k = 0,1...., M - 1

G(Jt) C l)*r
a =

<?(*) = ~G(M - *)

G(0) + 2

= *

G(*) cos

Jk=l
M -l

+ i)

M odd

f->-

A/ even

tf (* + i) = G (k + A) e->*/2e>0*+D/2*f

|(+
^ = EC(* Sin (*

G ( * + i ) = ( - l) * f f r

G (i + i) = G (M - k - i)
2

. ,

,v

jt

+ 5) (" + 5)

Antisymmetric

H(k) = G(k)eJ,r/2eJ,lk/M

k = 0,1,..., M - 1

G{k) = (-1)kHr ( j p j

G(k) = G(M - k)

h{n) = -^7 V
G(fc)sin^^ (n + ;)
Af
'
Af v
*() =

{ l)"+1G(A//2) - 2

G(&)sin

Af odd
( + j)

(* + i) = G (* + i) e 0k+l)fUI
G{k + \) = ( - l ) kHr \ ~ (* + l)j
G (k + i) = - G (Af - k - \) ;G(M/2) = 0 for Af odd
* (n )=

j; 12 G(k+5) 005
Af - 3

V =

( * + j ) (w + 2)

Af odd
Af even

Af even

634

Design of Digital Filters

Chap. 8

3ir
4

Figure 8.14 Frequency response of linear-phase FIR filter in Example 8.2.1.

The frequency response characteristic of this filter is shown in Fig. 8.14. We should
emphasize that Hr{w) is exactly equal to the values given by the specifications above
at a)k = 2jr)t/15.

Example HJ- 2

Determine the coefficients of a linear-phase FIR filter of length M = 32 which has a

symmetric unit sample response and a frequency response that satisfies the condition
k ~ 0 , 1,2,3,4, 5
k=

* = 7, 8,.... 15
where T\ = 0.3789795 for a = 0, and 7\ = 0.3570496 for a = 5 . These values of T\
were obtained from the tables of optimum transition parameters given in Appendix C.
The appropriate equations for this computation are given in Table 8.3 for
a = 0 and a =
These computations yield the unit sample responses shown in
Table 8.4. The corresponding frequency response characteristics are illustrated in
Figs. 8.15 and 8.16, respectively. Note that the bandwidth of the filter for a = 5 is
wider than that for a = 0.
Solution

T h e optim ization o f th e frequency sam p les in th e tran sition region o f the

frequency resp onse can b e explained by evalu atin g th e system fun ction H ( z ) , given
by (7.2.12), on th e unit circle and using th e relation sh ip in (8.2.37) to express H ( u >)

Sec. 8.2

Design of FIR Filters

635

TABLE 8.4
M = 32
ALPHA 0 .
T1 * 0 . 3 7 8 9 7 9 5 E + 0 0
= -0 .7 1 4 1 9 7 8 E -0 2
= - 0 . 3 0 7 0 8 0 I E - 02
0 . 5B 91327E-02
= 0 . 1349923E-01
= 0 . 8087033E-02
* -0 .1 1 0 7 2 5 8 E -0 1
- 0 . 2420687E-01
- 0 . 9446550E-02
= 0 . 2544464E-01
= 0 . 3985050E-01
0 . 2753036E-02
= - 0 . 5913959E-01
= - 0 . 6841660E-01
= 0 . 3 175741E-01
= 0.2080981E+00
= 0 . 3471138E+00

h ( 0)
h ( 1)
h ( 2)
h ( 3)
h ( 4)
h ( 5)
h ( 6)
h ( 7)
h ( 8)
h ( 9)
h(10)
h (1 1 )
h (1 2)
h(13)
h (14)
h(15)

= - 0 . 40B9120E-02
= - 0 . 9973779E-02
= - 0 . 7379891E-02
= 0 . 5949799E-02
= 0 . 1727056E-01
= 0 . 7878412E-02
= - 0 . 1798590E-01
= -0 .2 6 7 0 5 8 4 E -0 1
= 0 . 3778549E-02
= 0 . 4191022E-01
= 0.283 9 3 4 4 E -0 1
= - 0 . 4 1 6 3 144E-01
= - 0 . 8254962E-01
= 0 . 2802212E-02
= 0 . 2013655E+00
= 0 . 3717532E+00

Magnitude (d B )

h ( 0)
h ( 1)
h ( 2)
h ( 3)
h ( 4)
h ( 5)
h ( 6)
h ( 7)
h ( 8)
h ( 9)
h(10)
h (ll)
h (1 2)
h(13)
h (1 4)
h (15)

M * 32
ALPHA 0 . 5
T1 = 0 . 3 5 7 0 4 9 6 E + 0 0

Figure 8.15 Frequency response oflinear-phase FIR filterinExample 8.2.2 (M =

32 and a = 0).

636

Design of Digital Filters

Figure 8.16
32 and a =

Chap. 8

Frequency response o f linear-phase FIR filter in Example 8.2.2 (M =

5).

in term s o f G{k + a ) . T hus for th e sym m etric filter w e obtain

sin

a)M
M-

G (k + a )

l)/2

(8.2.38)

w here
[-G (M -k),
G{k + or) _ | C (M _ k _ j ^

a = 0
asci

(8.2.39)

Similarly, for the antisym m etric linear-phase F IR filter w e obtain

.

sin
H(co) =

(w M

\
G ( k + a)

u> IX
2

~ M (k +

a)\

where
r(lr,

f G ( M k) ,

a = 0

(8.2.41)

W ith these ex p ression s for the frequency resp onse H(co) given in term s o f the
desired frequency sam p les {G (*+<*)}, w e can easily explain th e m eth o d for selectin g
the param eters {G(k + a )} in th e transition band w hich result in m inim isin g the
peak sid elo b e in th e stopband. In brief, th e valu es o f G ( k + a ) in the passband are

Sec. 8.2

Design of FiR Filters

637

set to (1)* and th ose in th e stopband are set to zero. F or any ch oice o f G (k +ot)
in the transition band, the value o f H ( w ) is com puted at a d en se set o f frequencies
(e.g., at co = 2n n / K , n = 0, 1 , . . . , K 1, w h ere, for exam p le, K = 10M ) . T he
valu e o f the m axim um sid elo b e is d eterm ined, and the valu es o f the param eters
[G (k + a)} in th e transition band are changed in a d irection o f steep est descent,
w hich, in effect, red u ces th e m axim um sid elob e. T h e com p utation o f H(a>) is now
rep eated w ith the n ew ch oice o f {G(k + a )}. T h e m axim um sid elob e o f H ( a>) is
again determ in ed and the values o f the param eters {G (ik+a)} in the transition band
are adjusted in a direction o f steep est d escen t that, in turn, reduces the sidelobe.
T his interactive process is perform ed until it con verges to the optim um choice o f
the param eters {G(jfc + a )} in the transition band.
T h ere is a p otential p roblem in the frequency-sam pling realization o f the FIR
linear-phase filter. T h e frequency sam pling realization o f the F IR filter introduces
p o les and zeros at equally spaced p oints on the unit circle. In the ideal situation, the
zeros can cel the p o les and, con sequ en tly, the actual zeros o f H ( z ) are determ ined
by the selectio n o f the frequency sam ples { / / ( i k - f a ) } . In a practical im plem entation
o f the frequency-sam pling realization, how ever, q uantization effects preclude a
perfect can cellation o f the p oles and zeros. In fact, the location o f p oles on the
unit circle provide no dam ping o f th e rou n d -off n oise that is introduced in the
com putations. A s a result, such n oise tends to increase w ith tim e and, ultim ately,
m ay destroy the norm al operation o f the filter.
T o m itigate this problem , w e can m ove b oth the p o le s and zeros from the
unit circle to a circle just inside the unit circle, say at radius r = 1 e, w here e
is a very sm all num ber. T hus the system function o f the linear-phase F IR filter
b eco m es
(8.2.42)
T h e corresponding tw o -p ole filter realization given in Section 7.2.3 can be m odified
accordingly. T h e dam ping provided by selecting r < 1 ensures that rou n d off noise
will be b ou n ded and thus instability is avoided.

8.2.4 Design of Optimum Equiripple Linear-Phase FIR

Filters
T h e w indow m eth od and th e frequency-sam pling m eth od are relatively sim ple
tech n iqu es for d esigning linear-phase F IR filters. H ow ever, th ey also p ossess som e
m inor disadvantages, described in Section 8.2.6, which m ay render them undesir
able for so m e applications. A major probiem is the lack o f p recise control o f the
critical freq u en cies such as cop and ws .
T h e filter d esign m eth od described in this section is form ulated as a C hebyshev approxim ation problem . It is view ed as an optim um design criterion in the
sen se that the w eigh ted approxim ation error b etw een th e desired frequency re
sp on se and the actual frequency resp onse is spread even ly across the passband

638

Design of Digital Filters

Chap. 8

and even ly across th e stopband o f the filter m inim izing the m axim um error. T he
resulting filter designs have ripples in b oth the passband and th e stopband.
T o describe the design procedure, let us con sid er the design o f a lowpass
filter with passband ed ge frequency a>p and stop b an d ed ge frequency oos. From
the general specifications given in Fig. 8.2, in th e passband, the filter frequency
resp onse satisfies the condition
1 - i < Hr (w) < 1 + Si

M < o)p

(8.2.43)

Similarly, in the stopband, the filter frequency resp on se is sp ecified to fall betw een
the lim its 5 2 > that is,
8 2 < Hr ((D) < S2

M > OJ,

(8.2.44)

Thus Si represents the ripple in the passband and &2 rep resen ts th e attenuation or
ripple in the stopband. T h e rem aining filter param eter is M , th e filter length or
the num ber o f filter coefficients.
L et us focus on the four different cases that result in a linear-phase F IR filter.
T h ese cases w ere treated in Section 8.2.2 and are sum m arized b elow .

Case 1: Symmetric unit sample response h(n) = h (M 1 n) and M Odd.

In this case, the real-valued frequency resp onse characteristic Hr (io) is
H r {co) = h i -

h(ri)

COS

c u f -------- n )

(8.2.45)

If w e let k = ( M l ) / 2 n and define a new set o f filter p aram eters (a(Jk)} as

a (k) =

* V( ^ ) -

(8.2.46)

( ^ - 0 then (8.2.45) reduces to the com pact form

Hr (a>)

(M - 1 ) /2
^
a (k) cos (ok

(8.2.47)

kse0

Case 2: Symmetric unit sample response /i(n ) = h (M 1 n ) and M Even.

In this case, H r(co) is exp ressed as
H r{(d ) = 2

/ i ( ) c o s g j | -------- n l

n= o

(8.2.48)

A g a in , w e change the sum m ation in d ex from n t o k = M / 2 n and define a new

set o f filter param eters {>(*)} as

(M

Sec. 8.2

Design of FIR Filters

639

W ith th ese substitutions (8.2.48) b ecom es

up.
H r(w) = X

/
1\
b(k) cos to [ k - - j

(8.2.50)

In carrying o u t the optim ization, it is co n ven ien t to rearrange (8.2.50) further in to

the form
H r (o>) = cos

(A//2 ) 1 _
H k)co sa > k
o

(8.2.51)

w h ere the coefficients {(*)] are linearly related to the coefficients {i>(fc)}. In fact,
it can b e show n that the relationship is
m

= \ h i)

b(k) = 2b(k) - b(k - 1)

k = 1,2,3,..., y

-2

(8.2.52)

C ase 3: A n tisym m etric unit sam p le resp onse h ( n ) = h ( M 1 n ) and

M O dd. T he real-valued frequency resp onse characteristic H r (co) for this case is
(M'i) /2
H r (to) = 2
n=o

/ M _ 1
\
A( ) s i n a > f -------- n l
\
2
/

(8.2.53)

If w e ch an ge th e sum m ation in (8.2.53) from n to k = ( M I ) / 2 n and define a

n ew set o f filter param eters {c(*)} as
c(jk) = 2h

* = L 2 .........(Af l ) / 2

(8.2.54)

c(Jfc) sin (ok

(8.2.55)

then (8.2.53) b eco m es

(* f-l)/2

H r (co )=

5Z
*=i

A s in th e p reviou s case, it is con ven ien t to rearrange (8.2.55) in to the form

(W-3)/2

Hr (u>) = sin a> ^

c(k) cos tok

(8.2.56)

w h ere th e coefficients {c(Jfc)} are linearly related to the p aram eters {c(Jfc)}. T h is d e
sired relationship can b e derived from (8.2.55) and (8.2.56) and is sim ply

640

Design of Digital Filters

Chap. 8

given as

' M - l

m - -m
c(k 1) c(k 4- 1) = 2c(k)

(8.2.57)

2 < k <

M -5

5(0) + \ c ( 2 ) = c( l )

Case 4: Antisymmetric unit sample response h(ri) - h ( M 1 n) and

M Even.

In this case, the real-valued frequency response characteristic Hr (co) is

H r (a>) = 2

(Mf2 ) 1
/ M _ 1
\
X
/i(n)sincti ( - - n \
=o
\
2
/

(8,2.58)

A change in the sum m ation index from n to k = M f l n com b ined with a definition
o f a n ew set o f filter coefficients {^(*)), related to {/i(n)} according to
d (k)~ 2 h (^-k^j

* = l,2,...,y

(8.2.59)

results in th e expression

M/2

j \

H r ((o) = X ^ ( i) s in & > ( k - 1

'
'

(8.2.60)

A s in the previous tw o cases, w e find it con ven ien t to rearrange (8.2.60) in to the
form
(j)

Hr (a>) = s i n

ft

*=o

mm

d(k )c osa >k

(8.2.61)

w here the n ew filter p aram eters {d(fc)} are related to {<*()} as follow s:

H
d(k -

t
1

- ' H

) - d(k) =

2 d(k)

M
< k <

(8.2.62)

d(0 ) - \ d ( 1 ) = d( 1 )
T h e exp ression s for H r (a>) in these four cases are sum m arized in T ab le 8.5.
W e n o te that the rearrangem ents that w e m ade in ca ses 2, 3, an d 4 h ave allow ed

Sec. 8.2

Design of FIR Filters

641

TABLE 8.5 REAL-VALUED FREQUENCY RESPONSE

FUNCTIONS FOR LINEAR-PHASE FIR FILTERS

Filter type
h(n) ~ h(M M odd
(case 1 )

h(n) = h(M Af even

(case 2 )

Q(<o)

P(co)

y ' a(k)coscok
k=0
(M/2 J- 1
b(k) cos cok
M

- n)

- n)

CO
COS
2

h(n) = ~ h ( M - I - n)
Af odd
(case 3)

sin to

h(n) = h{M 1 n)
M even
(case 4)

. co
sin

(M-31/2

Y
km0

cfc) cos cok

(*/2 )-l
d(k) cos cok

kmO

us to express H r (co) as
H r (co) = Q(co)P(co)

(8.2.63)

where

Q(<o) =

case

COS

case

sin co
.
CO
sin

case 3

(8.2.64)

case 4

and P(co) has th e com m on form

P(co) = 7 > ( f c ) c o s ^

(8.2.65)

with {(*)} representing the param eters o f the filter, w hich are linearly related to
th e unit sam ple resp onse h (n) o f th e F IR filter. T h e upper lim it L in the sum is
L = ( M - l ) / 2 for Case 1, L = (Af 3 )/2 for C ase 3, and L = A f/2 1 for C ase 2
and C ase 4.
In addition to the com m on fram ew ork given ab ove for the representation
o f H r (eo), w e also define th e real-valued desired frequency resp onse H j r ((o) and
the w eigh ting function W(co) on the approxim ation error. T h e real-valued d e
sired frequency resp onse Hdr (co) is sim ply defined to b e unity in the passband and
zero in the stopband. For exam p le, Fig. 8.17 illustrates several d ifferen t typ es
o f characteristics for Hdr(co). T he w eigh ting function on the approxim ation er
ror allow s u s to ch o o se the relative size o f the errors in the different frequency
b ands (i.e., in th e passband and in the stopband). In particular, it is con ven ien t to
norm alize W(co) to unity in the stopband and set W(a>) = B2 /& 1 in the passband,

642

Design of Digital Filters

Chap. 8

HdJ&)

(a)

(b)
H dr{w )

(o]

o>2
(c)

tlj 4lj (t) 4

<d)

tu

Figure 8.17 D esired frequency

response characteristics for different
types of filters.

that is,
W(a>)

I S2 /S 1 ,

co in the passband
& in the stopband

(8.2.66)

Then w e sim ply select W(a>) in the passband to reflect our em p h asis o n the relative
size o f the ripple in the stopband to the ripple in the passband.
W ith th e specification o f Hdr(oi) and W O ), w e can n ow d efin e the w eighted
approxim ation error as
E(a>) = W ( o ) [ H dr(co) - Hr (a>)\
= W (w )[ H dr((o) - Q(co)P(w)]

Sec. 8.2

Design of FIR Filters

643

= W(co)Q(co)

Hdr(co)
. Q(C0 )

(8.2.67)

P(co)

For m athem atical con ven ien ce, w e define a m odified w eigh ting function W (co) and
a m odified desired frequency resp onse Hdr(co) as
W(a>) = W(co)Q(co)
Hdr(co) =

( 8 .2 .68)

Hdr(co)

Q(co)
T h en the w eigh ted approxim ation error m ay b e exp ressed as
E(co) = W(co)[Hdr(co) ~ P(o>)\

(8.2.69)

for all four d ifferent types o f linear-phase F IR filters.

G iven the error function E(co), the C h eb ysh ev approxim ation problem is
basically to determ in e the filter param eters (of(Jt)} that m inim ize the m aximum
ab solu te value o f E(co) over the frequency bands in which the approxim ation is to
be p erform ed . In m athem atical term s, w e seek the solu tion to the problem
L

min

m ax [ ( oj)|

min

m ax \W(co)[Hdr(co) y ^ g ( * ) c o s cok]\
. WfS
*=o
(8.2.70)
w here 5 rep resents the set (disjoint union) o f frequency bands over which the
op tim ization is to be perform ed. B asically, the set 5 consists o f the passbands and
stopbands o f the desired filter.
T h e solu tion to this problem is due to Parks and M cC lellan (1972a), w ho
applied a th eorem in the theory of C h eb ysh ev approxim ation. It is called the
al ternation theo rem , which w e state w ithout proof.
over (a ( i) J

toeS

over jo<(t)}

Alternation Theorem: Let 5 b e a com pact subset o f the interval [0,

n ecessary and sufficient con d ition for

tt).

i
P(co) = y ^ g ( f c )c o s cok
k= 0
to b e the unique, best w eigh ted C hebyshev approxim ation to Hdr(co) in S, is that
the error function E (co) exhibit at least L + 2 extrem al freq u en cies in S. That is,
there m ust exist at least L + 2 freq u en cies {co,} in 5 such that co\ < c&i < < coL+2,
E(coj) = E(a>i+j), and
jfO oi)! = m a x \E(cv)\
axS

i = 1,2,..., L + 2

W e n o te that the error function E(co) alternates in sign b etw een tw o su cces
sive extrem al frequencies. H en ce the theorem is called the alternation theorem .
T o elab orate on the alternation theorem , let us con sid er the design o f a
low p ass filter with passband 0 < co < cop and stopband cos < co < t t . Since the

644

Design of Digital Filters

Chap. 8

desired frequency resp onse Hdr(&) and the w eigh ting function W (w ) are p iecew ise
constant, w e have
dE{w)
d
~ = ( W O ) [Hdr(w) - H r {w) }
dw
aw
_

d H r (w) = Q
dw

C onsequently, the freq u en cies {oj,} corresponding to the p eak s o f E (w ) also cor
respond to peaks at w hich H r (w) m eets the error toleran ce. Since H r(w) is a
trigonom etric polyn om ial o f d egree L , for C ase 1, for exam p le,
L

H r (w) Y _ a ( k ) co s wk
k=0
l

]P a r (* )

X ^ni(C O SW )n

i=0
L

= ][V (Jfc)(cosa> )k
*=o

(8.2.71)

it follow s that Hr (w) can have at m ost L 1 local m axim a and m inim a in the op en
interval 0 < w < n . In addition, w = 0 and w = n are usually extrem a o f H r (w) and,
also, o f E{w). T h erefore, H r {w) has at m ost L - l - l extrem al frequencies. Further
m ore. the ban d -ed ge freq u en cies w p and ws are also extrem a o f E (w ), since |(a>)|
is m axim um at w = wp and w = ws . A s a con seq u en ce, there are at m ost L + 3 ex
trem al frequencies in ( w) for the unique, best approxim ation o f the ideal low pass
filter. O n the other hand, the alternation theorem states that there are at least L + 2
extrem al freq u en cies in E (w ). T hus the error function for th e low pass filter design
has either L + 3 or L + 2 extrem a. In gen eral, filter designs that contain m ore than
L + 2 alternations or ripples are called extra ripple filters. W h en the filter design
contains the m axim um num ber o f alternations, it is called a m a x i m a l ripple filter.
T he alternation th eorem guarantees a unique solu tion for the C hebyshev
optim ization problem in (8.2.70). A t the desired extrem al freq u en cies {w}, w e
have the set o f equations
W (w n)[Hdr(wn) - P ( w n)] = ( - i y \ 5

= 0 , 1 .........L + 1

(8.2.72)

w here & represents the m axim um valu e o f th e error function E (w ). In fact, if we

select W (w ) as indicated by (8.2.66), it follow s that S = S2.
The set o f linear eq u ation s in (8.2.72) can b e rearranged as
( - 1 )"$
P((On) + V - = Hdr(wn)
W(o>)

n = 0 , 1 .........L + 1

or, equivalently, in the form

a(k)cosw k +
to

^
= H dr((On)
W (v*)

= 0,1,..., L + 1

(8.2.73)

Sec. 8.2

645

Design of FIR Filters

I f w e treat the {()} and S as the param eters to be determ in ed , (8.2.73) can be
exp ressed in matrix form as
1

co s COo

co s 2 cdo

cos Lwo

a (0 )

Hdr (<*>0 )

W(coo)
1

co s cd\

co s 2 o>]

cos Lcoi

- 1

a(l)
W(<ui)

c o s w t+ i

c o s 2 a>z,+i

cosZ.gj+ i

a(L)

(-l)i

Hdr(0 h )

_ H dr (&1 +1 ) _

(8.2.74)
Initially, w e k n ow neither the set o f extrem al freq u en cies {io} nor the p a
ram eters {(*()} and S. T o solve for the param eters, w e use an iterative algorithm ,
called th e R e m e z e xch an ge a lgorithm [see R abiner et al. (1975)], in which w e
begin by guessing at the set o f extrem al freq u en cies, d eterm in e P (w ) and <5, and
then co m p u te the error function E (w ). From (w ) w e d eterm in e an oth er set o f
L + 2 extrem al freq u en cies and repeat the process iteratively until it con verges to
the optim al set o f extrem al freq u en cies. A lth ou gh the m atrix eq u ation in (8.2.74)
can b e used in the iterative procedure, matrix inversion is tim e consum ing and
inefficient.
A m ore efficient procedure, su ggested in the paper by R abiner et al. (1975),
is to co m p u te <5 analytically, according to the form ula
S=

yoHdr((oo) + y\ Hdr (<oi) H--------1- yL+\Hdr O l + i)

tf>

Y\

( - r V u i

W(o)o)

W (w i)

W { a>t + i)

(8.2.75)

w here

co s (Ok cos (on

nykk

(8.2.76)

T h e expression for S in (8.2.75) fo llo w s im m ed iately from th e m atrix equation in

(8.2.74). Thus with an initial guess at th e L + 2 extrem al freq u en cies, w e com pute 5.
N o w since P(a>) is a trigom etric p olyn om ial o f the form
L
P(co) = Y a ( k ) x k
*o

x = cosa)

and since w e k n ow that th e polynom ial at the p oin ts x = co s ( o , n ~ 0 , 1 , . . . , L + 1,

has th e corresponding values
( 1)"5

P(a>n) - Hjr((On) ~ -----W(con)

n = 0 , 1.....L + 1

(8.2.77)

646

Design of Digital Filters

Chap. 8

w e can use the Lagrange interpolation form ula for P(a>). T h u s P(to) can be ex
pressed as [see H am m ing (1962)]
L
> ( o * )[& /(*

JC *)]

(8.2.78)

w here P(a>n) is given by (8.2.77), x = co sw , x k = cosw *, and

(8.2.79)

H aving the solu tion for P(co), w e can n ow com p ute the error function E(co) from
(o>) = W{u>)[Hdr{a>) - P(co)]

(8.2.80)

on a den se set o f frequency points. U sually, a num ber o f p oints equal to 16Af,
w here M is the length o f the filter, suffices. If |(&))| > 8 for so m e freq u en cies on
the den se set, then a new set o f frequencies corresponding to the L + 2 largest peaks
o f |(<y)| are selected and the com putational procedure b egin n ing with (8.2.75)
is repeated. Since the new set o f L + 2 extrem al freq u en cies are selected to
correspond to the peaks o f the error function |(a>)|, the algorithm forces 5 to
increase in each iteration until it con verges to the upper b ou n d and h en ce to the
optim um solu tion for the C h eb ysh ev approxim ation problem . In other words,
w hen [(a>)| < 5 for all frequencies on the d en se set, the optim al solu tion has
b een found in term s o f the polynom ial H(co).
A flowchart o f the algorithm is sh ow n in Fig. 8.18 and is d u e to R em ez (1957).
O nce the optim al solu tion has b een ob tain ed in term s o f P(co), the unit
sam ple resp onse h (n) can b e com p uted directly, w ithout h aving to com p ute the
p aram eters (ar(fc)}. In effect, w e have d eterm ined
H r(co) = Q ( oj) P ( co)
w hich can be evalu ated at co = 2n k / M , k = 0, 1 , . . . , (Af l ) /2 , for Af odd, or
A f/2 for M ev en . T hen, d ep en d ing on the type o f filter b ein g d esign ed , h (n) can
b e determ in ed from th e form ulas given in T able 8.3.
A com puter program written by Parks and M cC lellan (1972b) is available for
d esigning iinear phase F IR filters b ased on the C h eb ysh ev approxim ation criterion
and im plem en ted with the R em ez exch an ge algorithm . T h is program can be used
to design low pass, highpass or bandpass filters, differentiators, and H ilbert trans
form ers. T he latter tw o types o f filters are described in the follow in g section s. A
num ber o f softw are packages for designing equiripple lin ear-ph ase F IR filters are
n ow available.
T h e P ark s-M cC lellan program requires a num ber o f in p u t param eters which
determ in e th e filter characteristics. In particular, the follow in g param eters must

Sec. 8.2

Design of FIR Filters

647

Input filter parameters

Initial guess of
M + 2 extremal freq.

Fifnre 8.18

Flowchart o f R em ez algorithm.

b e specified:
L IN E 1

N F 1 LT:

T h e filter length, d en oted ab ove as M .

JT Y P E :

T yp e o f filter
JT Y P E = 1 results in a m ultiple passband/stopband filter.

648

Design of Digital Fitters

Chap. 8

JT Y P E = 2 results in a differentiator.
JT Y P E = 3 results in a H ilb ert transform er.
N BA NDS:

The num ber o f frequency bands from 2 (for a low pass filter) to
a m axim um o f 1 0 (for a m ultiple-band filter).

L G R ID :

T he grid density for interpolating the error function E{u>). T he

d efault valu e is 16 if left unspecified.

L IN E 2
EDGE:

T he freq u en cy bands specified by low er and upper cu toff fre

qu en cies, up to a m axim um o f 1 0 bands (an array o f size 2 0 ,
m axim um ). T he freq u en cies are given in term s o f the variable
/ = a ) / 2 n , w h ere / = 0.5 corresponds to the folding frequency.

L IN E 3
FX:

A n array o f m axim um size 10 that sp ecifies the desired fre

q uency resp onse Hdr (&>) in each band.

L IN E 4
W TX;

A n array o f m axim um size 10 that specifies the w eight function

in each band.

The follow in g exam p les d em onstrate the use o f this program to design a
low pass and a bandpass filter.
Example 8.23
Design a lowpass filter of length M = 61 with a passband edge frequency f p = 0.1
and a stopband edge frequency f s = 0.15.
Solution The lowpass filter is a two-band filter with passband edge frequencies
(0,0.1) and stopband edge frequencies (0.15,0.5). The desired response is (1, 0) and
the weight function is arbitrarily selected as ( 1 , 1 ).
61, 1, 2
0.0, 0.1,0.15,0.5
1. 0. 0.0
1. 0, 1.0
The result of this design is illustrated in Table 8 .6 , which gives the filter coefficients.
The frequency response is shown in Fig. 8.19. The resulting filter has a stopband
attenuation of - 5 6 dB and a passband ripple of 0.0135 dB.
If w e increase the length o f the filter to M = 101 w h ile m aintaining all the
other param eters given ab ove the sam e, the resulting filter h as the frequency re
sp o n se characteristic sh ow n in Fig. 8.20. N o w , th e stopband atten u ation is 85 dB
and the passband ripple is reduced to 0.00046 dB .
W e should indicate that it is p ossib le to in crease the atten u ation in the stop
band by k eep in g th e filter len gth fixed, say at Af = 61, and decreasin g the weighting
function W(co) = S2 / S 1 in th e passband. W ith Af = 61 and a w eigh ting function

TABLE 8.6 PARAMETERS FOR LOWPASS FILTER DESIGN IN

EXAMPLE 8.2.3
F I N I T E I M P U L S E R E S P O N S E (FIR)
LINEAR PHASE DIGITAL FILTER DESIGN
REMEZ EXCHANGE ALGORITHM
FILTER LENGTH = 6 1
I M P U L S E R E S P O N S E *** **
H ( 1) = - 0 . 1 2 1 0 9 3 5 1 E - 0 2 = H ( 61)
H ( 2) = - 0 . 6 7 2 7 0 6 8 7 E - 0 3 = H ( 60)
H ( 3) = 0 . 9 8 0 9 0 2 4 0 E - 0 4 = H ( 59)
H( 4) = 0 . 1 3 5 3 6 6 6 4 E - 0 2 = H ( 58)
0. 22969784E-02
H ( 5)
H ( 57)
H ( 6) = 0 . 1 9 9 6 3 4 9 5 E - 0 2 = H ( 56)
0 . 9 7 0 2 6 0 9 5 E - 0 4 = H( 55)
H ( 7) =
H ( 8) = - 0 . 2 6 4 6 6 6 9 5 E - 0 2 = H ( 54)
H ( 9) = - 0 . 4 5 1 3 3 1 0 3 E - 0 2 = H ( 53)
H (10) = - 0 . 3 7 7 0 4 9 4 4 E - 0 2 = H ( 52)
H (11) = 0 . 1 3 0 7 9 6 5 5 E - 0 4 = H ( 51)
HS12) = 0 . 5 1 7 913 5 6 E - 0 2 = H( 50)
H (13) =
0 . 8 4 8 8 3 4 7 8 E - 0 2 = H ( 49)
H (14) = 0 . 6 9 5 3 2 1 1 0 E - 0 2 = H ( 48)
H (15) = 0 . 7 1 0 3 7 0 5 9 E - 0 4 = H ( 47)
H (16) = - 0 . 9 0 4 07 8 9 7 E - 0 2 = H ( 46)
H (17) = - 0 . 1 4 7 2 3 0 4 7 E - 0 1 = H ( 45)
H (18) = - 0 . 1 1 9 5 8 9 4 5 E - 0 1 = H ( 44)
H (19) = - 0 . 2 9 7 9 9 2 1 4 E - 0 4 = H ( 43)
H (20) = 0 . 1 5 7 1 3 4 2 2 E - 0 1 = H ( 42)
H (21) =
0 . 2 5 6 5 7 1 5 1 E - 0 1 = H ( 41)
H (22) = 0 . 2 1 0 5 7 3 7 3 E - 01 = H { 40)
H (23) =
0. 6 8 6 3 7 7 6 8 E - 0 4 = H ( 39)
H (24) = -0. 2 8 9 0 2 0 5 4 E - 0 1 = H ( 38)
H (25) = - 0 . 4 9 1 1 8 5 4 1 E - 0 1 = H ( 37)
H (26)
- 0 . 4 2 7 1 3 9 7 0 E - 01 = H ( 36)
H (27) = - 0 . 5 0 1 1 4 3 0 4 E- 0 4 = H ( 35)
H (28) =
0 . 7 3 5 7 4 2 1 5 E - 0 1 = H ( 34)
H (29) = 0 . 1 5 7 8 2 0 4 0 E + 0 0 = H( 33)
H (30) =
0 . 2 2 4 6 5 5 1 2 E + 0 0 = H ( 32)
H (31) =
0 . 25007001E+'p0 = H ( 31)
BAND 1
BAND 2
LOWER BAND EDGE
0.0000000
0.1500000
UPPER BAND EDGE
0.1000000
0.5000000
DESIRED VALUE
1.0000000
0.0000000
WEIGHTING
1.0000000
1.0000000
DEVIATION
0.0015537
0.0015537
D E V I A T I O N IN D B
0.0134854
-56.1724014
EXTREMAL FREQUENCIES--MAXIMA OF THE ERROR CURVE
0.0000000
0.0252016
0.0423387
0.0584677
0.0735887
0.0866935
0.0957661
0.1000000
0.1500000
0.1540323
0.1631048
0.1762097
0.1903225
0.2054435
0.2215725
0.2377015
0.2538306
0.2699596
0.2860886
0.3022176
0.3183466
0.3354837
0.3516127
0.3677417
0.3848788
0.4010078
0.4171368
0.4342739
0.4504029
0.4665320
0.4836690
0.5000000

649

Design of Digital Fillers

Chap. 8

Magnitude (dB)

650

Figure 8.19

Figure

Frequency response o f M = 61 FIR filter in Example 8.2.3.

Frequency response o f M 101 FIR filter in Exam ple 8.2.3.

(0 .1 ,1 ), w e obtain a filter that has a stopband attenuation o f 65 dB and a pass

band ripple o f 0.049 dB.

Example 8*2,4
Design a bandpass filter of length M = 32 with passband edge frequencies f pi = 0.2
and fpi = 0.35 and stopband edge frequencies of f M\ = 0.1 and f ,2 = 0.425.

Sec. 8.2

Design of FIR Filters

651

Solution This passband filter is a three-band filter with a stopband range of (0,0.1), a
passband range of (0.2,0.35), and a second stopband range of (0.425,0.5). The weight
ing function is selected as (1 0 .0 , 1 .0 , 1 0 .0 ), or as ( 1 .0 , 0 .1 , 1 .0 ), and the desired response
in the three bands is (0.0,1.0,0.0). Thus the input parameters to the program are
32, 1, 3
0.0,0.1,0.2,0.35,0.425,0.5
0 . 0, 1. 0, 0.0
10.0, 1. 0, 10.0
The results of this design are shown in Table 8.7, which gives the filter coefficients.
We note that the ripple in the stopbands Bz is 10 times smaller than the ripple in
TABLE 8.7

PARAMETERS FOR BANDPASS FILTER IN EXAMPLE 8.2.4

F I N I T E I M P U L S E R E S P O N S E (FIR)
LINEAR PHASE DIGITAL FILTER DESIGN
REMEZ EXCHANGE ALGORITHM
BANDPASS FILTER
FILTER LENGTH = 3 2
***** I M P U L S E R E S P O N S E *****
H ( 1) = -0 .5 7 5 3 4 0 2 6 E - 0 2 = H ( 32)
H ( 2) = 0 .9 9 0 2 6 6 9 1 E - 0 3 = H ( 31)
H< 3) = 0 . 7 5 7 3 3 4 7 1 E - 0 2 = H ( 30)
-0 .6 5 1 4 1 2 0 4 E - 0 2 = H ( 29)
H ( 4)
H ( 5) = 0 .1 3 9 6 0 5 0 9 E - 0 1 = H ( 28)
H ( 6) = 0 .2 2 9 5 1 6 4 4 E - 0 2 = H ( 27)
H ( 7) = -0 .1 9 9 9 4 0 4 1 E - 0 1 = H( 26)
H ( 8) =
0,. 7 1 3 6 9 6 5 6 E - 0 2 = H ( 25)
H< 9} = - 0 ..3 9 6 5 7 3 7 3 E - 0 1
H( 24)
H (10) e 0 .,1 1 2 6 0 0 6 6 E - 0 1 = H ( 23)
H( l l > = 0 ..6 6 2 3 3 6 3 5 E - 0 1 = H ( 22)
H (12) * -0.,1 0 4 9 7 2 0 2 E - 0 1 = H ( 21)
H
H
H
H

(13)
(14)
(15)
(16)

= 0 .,8 5 1 3 6 1 6 0 E - 0 1 = H(
= -0..1 2 0 2 4 9 8 8 E + 0 0 = H (
= - 0 .,2 9 6 7 8 5 8 0 E + 0 0 * = H(
= 0 . 3 0 4 1 0 9 1 3 E + 0 0 = H(
BAND 1
BAND 2

20)
19)
18)
17)

LOWER BAND EDGE

0.0000000
0.2000000
0.4250000
UPPER BAND EDGE
0.1000000
0.3500000
0.5000000
DESIRED VALUE
0.0000000
1.0000000
0.0000000
WEIGHTING
10.0000000
1.0000000
10.0000000
DEVIATION
0.0015131
0.0151312
0.0015131
DEVIATION IN DB
-56.4025536
0.1304428
-56.4025536
EXTREMAL FREQUENCIES MAXIMA OF THE ERROR CURVE
0.0000000
0.0273438
0.0527344
0.0761719
0.0937500
0.1000000
0.2000000
0.2195313
0.2527344
0.2839844
0.3132813
0.3386719
0.3500000
0.4250000
0.4328125
0.4503906
0.4796875

Design of Digital Fitters

Chap. 8

Magnitude (d B )

652

Figure JL21

Frequency response of M = 32 FIR filter in Example 8.2.4.

the passband due to the fact that errors in the stopband were given a weight of 10
compared to the passband weight of unity. The frequency response of the bandpass
filter is illustrated in Fig. 8.21.
T h ese exam p les serve to illustrate the relative ease w ith which optim al low pass, highpass, bandstop, bandpass, and m ore general m ultiband linear-phase FIR
filters can be designed based on the C h eb ysh ev approxim ation criterion im ple
m ented by m eans o f the R em ez exch an ge algorithm . In the n ext tw o sections w e
consider the design o f differentiators and H ilbert transform ers.

8.2.5 Design of FIR Differentiators

D ifferen tiators are used in m any analog and digital system s to take the derivative
o f a signal. A n ideal differentiator has a frequency resp onse that is linearly pro
portional to frequency. Sim ilarly, an ideal digital differentiator is defined as one
that has the frequency resp onse
Hd(ao) = j t o

7r < u) < n

(8.2.81)

T h e unit sam ple response corresponding to Hj(co) is

M ) =
=

Hd((*>)eiwnda>

2n

cosnn

j(jDeJconda>
oo < n < oo, n ^ 0

(8.2.82)

Sec. 8.2

653

Design of FIR Filters

W e ob serve that the ideal differentiator has an antisym m etric unit sam ple resp onse
[i.e., h d (n) = h d( n)]. H en ce, h d (0) = 0.
In this section w e consider the design o f lin ear-ph ase F IR differentiators
b ased o n the C h eb ysh ev approxim ation criterion. In view o f the fact that the
ideal differentiator has an antisym m etric unit sam p le resp onse, w e shall con fine
our atten tion to F IR designs in which h ( n ) = h ( M 1 n). H en ce w e con sid er
the filter types classified in the p receding section , as C ase 3 and C ase 4.
W e recall that in C ase 3, w h ere Af is od d , the real-valu ed frequency resp onse
o f the F IR filter H r(co) has the characteristic that H r (0) = 0. A zero resp onse at
zero freq u en cy is just the con d ition that the d ifferen tiator should satisfy, and w e
se e from T ab le 8.5 that b oth filter types satisfy this con d ition . H ow ever, if a fullband differentiator is desired, this is im possib le to ach ieve with an F IR filter having
an od d num ber o f coefficients, since H r ( n ) = 0 for Af odd. In practice, h ow ever,
full-band differentiators are rarely required.
In m ost cases o f practical interest, the desired freq u en cy resp onse character
istic n eed o n ly be linear over the lim ited frequency range 0 < co < 2 n f p , w h ere f p
is called the bandw idth o f the differentiator. In the frequency range 2n f p < co < n ,
the desired resp onse m ay be either left unconstrained or constrained to b e zero.
In the design o f F IR differentiators b ased on the C h eb ysh ev approxim ation
criterion, the w eigh ting function W(co) is sp ecified in th e program as
(8.2.83)
W (co) =
0 < co < 2 n f p
co
in order that the relative ripple in the passband be a constant. T hus the ab solu te
error b etw een the desired resp onse co and th e approxim ation Hr(co) increases as
co varies from 0 to 2n f p. H ow ever, the w eigh ting fun ction in (8.2.83) ensures that
the relative error
8

{ W (co )[(o - Hr (a>)]}

m ax
O s a><2nfp

H r (CO)

m ax

1 -

(8.2.84)

is fixed within the passband o f the differentiator.

Example 8^5
Use the Remez algorithm to design a linear-phase FIR differentiator of length hi =
60. The passband edge frequency is 0.1 and the stopband edge frequency is 0.15.

Solution The input parameters to the program are

60,
0.0,
1.0,
1.0,

2 ,
0.1,
0.0
1.0

0.15,

0.5

The results of this design including the filter coefficients are shown in Table 8 .8 . The
frequency response characteristic is illustrated in Fig. 8.22. Also shown in the same
figure is the approximation error over the passband 0 < / < 0 . 1 of the filter.

TABLE 8.8

PARAMETERS FOR FIR DIFFERENTIATOR IN EXAMPLE 8.2.5

F I N I T E I M P U L S E R E S P O N S E (FIR)
LINEAR-PHASE DIGITAL FILTER DESIGN
REMEZ EXCHANGE ALGORITHM
DIFFERENTIATOR
FILTER LENGTH = 6 0
IMPULSE RESPONSE '
H ( 1) = - 0 . 1 2 4 7 8 0 7 5 E - 0 2 = -H{ 60)
- 0 . 1 5 7 1 3 5 6 0 E - 02 = -H( 59)
H ( 2)
H ( 3) = 0 . 3 6 8 4 6 7 3 7 E - 0 2 = -H ( 58)
0 . 1 9 2 9 8 0 2 0 E - 0 2 = - H ( 57)
H ( 4) =
H( 5) = 0 . 1 4 2 6 4 1 4 I E - 02 = - H ( 56)
H ( 6) = - 0 . 1 7 6 1 5 2 7 7 E - 0 2 = -H( 55)
H ( 7) = - 0 . 4 3 1 1 0 5 7 3 E ~ 02 = -H( 54)
- 0 . 4 6 9 5 3 4 0 5 E - 0 2 = -H( 53)
H ( 8)
H ( 9) = - 0 . 1 4 1 0 5 2 4 4 E - 02 * -H( 52)
0 . 4 1 6 9 4 2 2 2 E - 0 2 = -H( 51)
H (10)
H (11) = 0 . 8 5 7 3 6 2 1 5 E - 0 2
- H ( 50)
H (12) =
0 . 7 9 8 1 3 0 3 1 E - 0 2 = -H( 49)
0 . 1 1 8 3 3 3 8 5 E - 0 2 = H ( 48)
H(13)
H (14) = - 0 . 8 7 3 9 6 0 6 5 E - 0 2 = -H( 47)
H (15) = - 0 . 1 5 4 0 1 8 4 7 E - 0 1 = H ( 46)
H (16) K - 0 . 1 2 8 7 8 4 4 5 E - 0 1 = -H ( 45)
H (17) = - 0 . 1 8 8 2 6 8 7 2 E - 0 3 = ~H( 44)
H (18) = 0 . 1 6 6 2 0 5 0 6 E - 0 1 = -H ( 43)
H (19) ss 0 . 2 6 7 4 1 5 2 3 E - 0 1 = -H( 42)
H (20) = 0.2 0 8 9 2 0 1 8 E - 0 1 = -H ( 41)
H(21) = - 0 . 1 8 5 8 4 0 9 5 E - 02 = -H( 40)
H (22) = - 0 . 3 1 1 0 9 9 0 9 E - 0 1 = -H ( 39)
H (23) = - 0 . 4 8 8 2 2 1 7 6 E - 0 1 = -H( 38)
H (24) = - 0 . 3 8 6 7 3 4 5 3 E - 01 = " H ( 37)
H (2 5) = 0 . 3 6 7 6 0 1 2 2 E - 0 2 = -H( 36)
H (26) =
0 . 6 5 4 6 2 4 7 8 E - 0 1 = -H ( 35)
H (27) = 0 . 1 2 0 6 6 3 1 7 E + 0 0 = -H( 34)
H (28) = 0 . 1 4 1 8 2 1 3 4 E + 0 0 = -H( 33)
H (29) = 0 . 1 1 4 0 3 7 5 7 E + 0 0 * -H( 32)
H ( 3 0 > = 0 . 4 3 6 2 0 0 8 0 E - 0 1 = - H ( 31)
BAND 1
BAND 2
LOWER BAND EDGE
0.0000000
0.1500000
UPPER BAND EDGE
0.1000000
0.5000000
DESIRED SLOPE
10.0000000
0.0000000
WEIGHTING
1.0000000
1.0000000
DEVIATION
0.0073580
0.0073580
E X T R E M A L F R E Q U E N C I E S --- M A X I M A O F T H E E R R O R C U R V E
0.0010417
0.0156250
0.0312500
0.0468750
0.0614583
0.0750000
0.0875000
0.0968750
0.1000000
0.1500000
0.1552083
0.1666667
0.1822916
0.1979166
0.2156249
0.2322916
0.2489582
0.2666668
0.2843754
0.3020839
0.3187508
0.3364594
0.3541680
0.3718765
0.3906268
0.4083354
0.4260439
0.4447942
0.4625027
0.4812530
0.5000000

654

655

Design of FIR Filters

Normalized Error
X .001195

Error X .000736

Magnitude

Sec. 8.2

Normalized frequency
Fifure
Frequency response and approximation error for M = 60 FIR differentiator of
Example 8.2.5.

The im portant p aram eters in a differentiator are its len gth Af, its bandwidth
{band-edge frequency) f p, and the p eak relative error S o f th e approxim ation. T he
in terrelationship am ong th ese three param eters can be easily displayed param et
rically. In particular, th e value o f 20 log 10 S versus f p w ith Af as a param eter is
sh ow n in Fig. 8.23 for Af even and in Fig. 8.24 for Af odd. T h ese results, due to
R abiner and Schafter (1974a), are useful in the selection o f th e filter length, given
specifications on the inband ripple and the cu toff frequency f p.
A com parison o f th e graphs in Figs. 8.23 and 8.24 reveals that even -len gth
differentiators result in a significantly sm aller approxim ation error S than com pa
rable od d -len gth d ifferentiators. D esig n s based o n Af odd are particularly p oor if

656

Design of Digital niters

Chap. 8

Passband cutoff frequency {fp)

Figure &23 Curves of 20 log 105 versus f p for M = 4, 8, 16, 32, and 64. [From
paper by Rabiner and Schafer (1974a). Reprinted with permission o f AT&T.]

the bandw idth ex ceed s f p = 0.45. T h e problem is b asically th e zero in the fre
q u en cy resp onse at co = n ( f = 1 /2 ). W h en f p < 0.45, g o o d d esign s are obtained
for M odd, but com parable-length d ifferen tiators with M even are always better
in the sense that the approxim ation error is sm aller.
In view o f the ob viou s advantage o f even -len gth over od d -len gth differentia
tors, a conclusion m ight b e that even -len gth differentiators are alw ays preferable in
practical system s. T his is certainly true for m any applications. H ow ever, w e should
n o te that th e signal d elay introduced by any linear-phase F IR filter is ( M 1)/2,
w hich is n ot an integer w hen M is even . In m any practical applications, this is
unim portant. In so m e applications w here it is d esirable to h ave an integer-valued
d elay in the signal at th e output o f th e differentiator, w e m ust se lect M to be odd.
T h ese num erical results are b ased on d esign s resulting from the C hebyshev
approxim ation criterion. W e w ish to indicate it is also p o ssib le and relatively

Sec. 8.2

Design of FIR Filters

657

Passband cutoff frequency (fp)

Figure 8-24 Curves of 201ogto S versus Fp for M 5, 9, 17, 33 and 65. [From
paper by Rabiner and Schafer (1974a). Reprinted with permission o f AT&T.]

easy to design linear phase F IR differentiators based o n the frequency sam pling
m eth od . F or exam p le, Fig. 8.25 illustrates the frequency resp onse characteristics o f
a w ideband ( f p = 0.5) differentiator, o f length M 30. T h e graph o f the ab solu te
valu e o f th e approxim ation error as a function o f freq u en cy is also show n in this
figure.

8.2.6 Design of Hilbert Transformers

A n ideal H ilb ert transform er is an all-pass filter that im parts a 90 phase shift
on th e signal at its input. H en ce th e frequency resp on se o f th e ideal H ilbert
transform er is specified as

(8.285)

Design of Digital Filters

Chap. 8

Error (dB)

Magnitude

658

Figure fL25 Frequency response and approximation error for M = 30 FIR differentiator
designed by frequency sampling method.

H ilbert transform ers are frequently used in com m u n ication system s and signal
processing, as, for exam ple, in the gen eration o f sin gle-sid eb an d m odulated signals,
radar signal processin g, and sp eech signal p rocessing.
T h e unit sam ple resp onse o f an idea! H ilbert transform er is

hd(n) = J

Hd(u>)eja,ndco

Sec. 8.2

659

Design of FIR Filters

2 sin2(jrn/2)

(8.2.86)

n
0,

A s ex p ected , hd(n) is infinite in duration and n oncausal. W e n ote that hd(n) is

antisym m etric [i.e., hd (n) = k d( n)]. In view o f this characteristic, w e focus our
atten tion on th e design o f linear-phase F IR H ilb ert transform ers w ith antisym m et
ric unit sam ple resp onse [i.e., h(n) h ( M 1 n)]. W e also ob serve that our
ch oice o f an antisym m etric unit sam ple resp onse is con sisten t with having a purely
im aginary frequency resp onse characteristic Hd (co).
W e recall o n ce again that w h en h{n) is antisym m etric, the real-valued fre
quency resp onse characteristic H,(co) is zero at co = 0 for b oth M od d and ev en
and at co = n w h en M is odd. C learly, then, it is im p ossib le to design an all-pass
digital H ilb ert transform er. Fortunately, in practical signal processing applications,
an all-pass H ilbert transform er is unnecessary. Its bandw idth n eed on ly cover the
bandwidth o f the signal to b e p h ase shifted. C on sequ en tly, w e specify the desired
real-valued frequency resp onse o f a H ilbert transform filter as
Hdr(co) =

2 :z

f < co <

2n f u

(8.2.87)

w here f t and / are the low er and upper cu toff freq u en cies, respectively.
It is interesting to n ote that the ideal H ilbert transform er with unit sam ple
resp onse hd (n) as given in (8.2.86) is zero for n even . T his property is retained by
the F IR H ilbert transform er under som e sym m etry con d ition s. In particular, let
us consider the C ase 3 filter type for which
(W-l)/2

(8.2.88)

and su p p ose that ft = 0.5 f u. This ensures a sym m etric passband about the
m idpoint frequency / = 0.25. If w e have this sym m etry in the frequency resp onse,
Hr (co) = Hr ( n co) and h en ce (8.2.88) yields
(*

IM-D/2
J ' c(k) sin<w/k c o s 7T/k
fc-i
(M-D /2
7 ! c ( k) ( 1)*+1 sini>*
or equivalently,
(M- 1)/2

(8.2.89)
Clearly, c(k) m ust b e eq u al to zero for jfc 0, 2, 4,

660

Design of Digital Filters

Chap. 8

N o w , th e relationship b etw een (c()} and the unit sam p le resp on se {/?() j is,
from (8.2.54),

or, equivalently,
(8.2.90)
If c(k) is zero for k = 0, 2, 4 , . . . , then (8.2.90) yield s

0,

0 ,2 , 4 , . . . . for

h{k) =

0,

k 1, 3 , 5 , . . . , for ^

ev en
(8.2.91)

^ od d

U n fortu n ately, (8.2.91) h old s only for M odd. It d o es n ot h old for M even . This
m ean s that for com parable values o f M , the case M odd is preferab le since the
com p utational com p lexity (num ber o f m ultiplications and ad d ition s per output
p oin t) is roughly o n e half o f that for M even.
W hen th e design o f th e H ilbert transform er is perform ed b y the C hebyshev
approxim ation criterion using the R em ez algorithm , w e se lect th e filter coefficients
to m inim ize the peak approxim ation error
S =

max

[Hdr(ai) Hr (co) ]

2jtft <a><2xfm

(8.2.92)
=

m ax

2jrfi<w<2x / ,

[1 - Hr ((t>)]

Thus the w eighting function is set to unity and th e op tim ization is perform ed over
the single frequency band (i.e., the passband o f th e filter).
Example 8.2.6
Design a Hilbert transformer with parameters M = 31, // = 0.05, and / , = 0.45.
Solution We observe that the frequency response is symmetric, since f u = 0.5 - fiThe parameters for executing the Remez algorithm are
31.
0.05,

3,
0.45

1.0

1.0
The result of this design is the unit sample response coefficients and the peak ap
proximation error & = 0.0026803 or 51.4 dB given in Table 8.9. We observe that,
indeed, every other value of h(n) is essentially zero (these values are of the order of
10~7). The frequency response of the Hilbert transformer is shown in Fig. 826.

Sec. 8.2

Design of FIR Filters

661

TABLE 8.9 PARAMETERS FOR FIR HILBERT TRANSFORM FILTER IN

EXAMPLE 8.2.6
F I N I T E I M P U L S E R E S P O N S E (FIR)
L INEAR PHASE DIGITAL FILTER DESIGN
REMEZ EXCHANGE ALGORITHM
HILBERT TRANSFORMER
FILTER LENGTH = 3 1
* * * I M P U L S E R E S P O N S E * * * *
H { 1)
H ( 2)
H ( 3)
H ( 4)
H ( 5)
H ( 6)
H { 7)
H ( 8)
H( 9)
H {10)

0 .4 1 9 5 7 5 1 6 E - 0 2 = -H< 31)
0 .6 4 3 1 0 2 5 7 E - 0 7 = ~H( 30)
0 . 9 2 8 2 2 4 4 4 E - 0 2 = -H ( 29)
0 .5 2 6 9 3 9 2 7 E - 0 7 = -H< 28)
0,.1 8 8 3 5 9 8 8 E - 0 1 = -H ( 27)
0 .8 2 3 0 8 2 8 3 E - 0 7 = -H( 26)
= 0..3 4 4 0 1 1 9 0 E - 0 1 = -H ( 25)
= 0..9 3 3 2 8 7 9 4 E - 0 7 = -H( 24)
0.,5 9 5 5 1 7 3 8 E - 0 1 = -H( 23)
=
= 0..5 0 8 2 1 1 7 1 E - 0 7 = -H( 22)
H (11) = 0. 1 0 3 0 3 7 8 2 E + 0 0 = ~H( 21)
H (12) = 0. 1 7 6 1 2 1 3 8 E - 0 7 = -H( 20)
H (13) = 0. 1 9 6 8 3 1 6 7 E + 0 0 = -H< 19)
H (14) = -0. 2 3 9 7 7 6 0 6 E - 0 7 = -H ( 18)
H {15) = 0. 6 3 1 3 5 3 7 4 E + 0 0 = -H ( 17)
H (16) = 0. 0
BAND 1
LOWER BAND EDGE
0.0500000
UPPER BAND EDGE
0.4500000
DESIRED VALUE
1.0000000
WEIGHTING
1.0000000
DEVIATION
0.0026803
E X T R E M A L F R E Q U E N C I E S --- M A X I M A O F T H E E R R O R C U R V E
0.0500000
0.1583333
0.3124998

=
=
=
=
=

0.0562500
0.1874999
0.3416664

0.0750000
0.2187499
0.3708331

0.1000000
0.2499998
0.3999997

0.1291666
0.2812498
0.4249997

R abiner and Schafer (1974b) h ave investigated the characteristics o f H ilbert

transform er designs for b oth M odd and M ev en . If the filter design is restricted
to a sym m etric frequency response, then there are basically three param eters o f
interest, M, 8 , and f\. Figure 8.27 is a p lot o f 201og10<5 versus f i with M as a
param eter. W e ob serve that for com parable valu es o f M, th ere is no perform ance
advantage o f using M od d over M even , and vice versa. H ow ever, the com puta
tional com p lexity in im plem enting a filter for M odd is less by a factor o f 2 over
M even as previou sly indicated. T herefore, M odd is p referab le in practice.
For design purposes, the graphs in Fig. 8.27 suggest that, as a rule o f thum b,

Mfi % -0 .6 1 log10 8

(8.2.93)

662

Chap. 8

Magnitude (d B )

Design of Digital Filters

Figure 8.26

Frequency response of FIR Hilbert transform filter in Example

8.6.6.

H en ce this form ula can be used to estim ate the size o f on e o f th e three basic filter
param eters w h en the other tw o param eters are specified.
In concluding this section , w e wish to sh ow that H ilb ert transform ers can
also be designed by the w indow m eth od and th e freq u en cy sam pling m ethod.
F or exam p le, Fig. 8.28 illustrates the frequency resp onse o f an M = 31 H ilbert
transform er design ed using the frequency sam pling m ethod. T h e corresponding
values o f the unit sam ple resp onse are given in T ab le 8.10. A com parison o f these
filter param eters with th ose given in T able 8.9 indicates so m e sm all differences.
In particular, it appears that the C h eb ysh ev approxim ation criterion gives signifi
cantly sm aller valu es for the filter coefficients that should b e zero. In general, the
C h eb ysh ev approxim ation criterion results in b etter filter designs.

8.2.7 C om parison of D esign M ethods for L inear-P hase

FIR Filters
H istorically, the design m eth od based on the use o f w in d ow s to truncate the
im pulse resp onse hd(n) and obtaining the d esired spectral shaping, was the first
m eth od p rop osed for designing linear-phase F IR filters. T h e frequency-sam pling
m eth od and the C h ebyshev approxim ation m eth od w ere d ev elo p ed in the 1970s
and have since b eco m e very popular in the design o f practical linear-phase FIR
filters.
T he major disadvantage o f the w in d ow d esign m eth od is the lack o f precise
control o f th e critical frequencies, such as cop and tos , in th e design o f a lowpass
F IR filter. T h e valu es o f w p and cos, in general, d ep en d on the typ e o f w indow and
the filter length M.

Sec. 8.2

Design of FIR Filters

663

-10
-20

-3 0

I
I

-4 0

-50

1 - 60

o
s
o
"

-7 0

-8 0

-9 0

-100
-1 1 0

-1 2 0

0.02

0.04

0.06

0.08

0.10

Transition width ( A f )
Figure JL27 Curves o f 201og 10S versus A f for M 3, 4, 7, 8, 15, 16, 31, 32, 63,
64. [From paper by Rabiner and SchafeT (1974b). Reprinted with permission of
AT&T.]

T he frequency sam pling m eth od provides an im provem en t o ver the w in

dow design m eth od , sin ce Hr (ui) is specified at the freq u en cies cok 2 n k / M or
cok = n( 2k + 1 )/M and th e transition band is a m ultiple o f 2n / M . This filter d e
sign m eth od is particularly attractive w hen the F IR filter is realized eith er in the
frequency dom ain by m eans o f th e D F T or in any o f the frequency sam pling re
alizations. T h e attractive feature o f th ese realizations is that Hr (cok) is either zero
or unity at all freq u en cies, excep t in the transition band.
T h e C h eb ysh ev approxim ation m ethod provides total control o f the filter
specifications, and, as a co n seq u en ce, it is usually p referable o v er the other tw o
m eth od s. For a low pass filter, the specifications are given in term s o f the param
eters <Dp, (ds, $i, 52, and Af. W e can specify th e param eters cdp, u>s, Af and 8 , and
o p tim ize th e filters relative to 82- B y spreading th e approxim ation error over the

664

Chap. 8

Magnitude ( d B )

Design of Digital Filters

Figure 8-28 Frequency response of M 31 FIR Hilbert transform filter designed by the
frequency sampling method.

passband and the stopband o f the filter, this m eth od results in an optim al filter d e
sign, in the sense that for a given set o f specifications just d escrib ed , the m axim um
sid elob e level is m inim ized.
T h e C hebyshev design p rocedure based on the R em ez exch an ge algorithm
requires that w e sp ecify the length o f the filter, the critical freq u en cies iop and ws,
and the ratio S2/Si- H o w ever, it is m ore natural in filter design to specify iop , cos , 5j,
and S2 and to d eterm in e th e filter length that satisfies the sp ecification s. A lth ou gh
there is n o sim ple form ula to determ in e the filter length from th e se specifications,
a num ber o f approxim ations h ave b een prop osed for estim ating M from cop , ojs ,
Si, and S2. A particularly sim p le form ula attributed to K aiser for approxim ating
M is
A = - 2 0 1 o g ,,( ^ ) - 1 3 +1
14.6 A f
where A f is the transition band, defined as A f = (cos - a)p) f l u . This form ula
has b een given in the paper by R abiner et al. (1975). A m ore accurate form ula
proposed by H errm ann et al. (1973) is
M =

>oo(Si, S2) - f ( S u S2 ) ( A f ) 2
*
2n }> + 1
A/

(8.2.95)

w here, by definition,
>00( 51, S2) = [0.005309(loglo <5; )2 + 0 .0 7 1 14(log105i) - 0.4761 J(Iog 10 S2)
- [0.00266(log10 Sx)2 + 0.5941 lo g 10 5i + 0.4278]

/(S i, 52) = 11.012 + 0.51244(log10 5i - log10 S2)

(8.2.96)

(8.2.97)

Sec. 8 2

Design of FIR Filters

665

TABLE 8.10 PARAMETERS A FOR M = 31 HILBERT

TRANSFORM FILTER DESIGNED BY THE FREQUENCYSAMPLING METHOD
LINEAR PHASE FIR HILBERT TRANSFORM
FREQUENCY SAMPLING METHOD
F I L T E R L E N G T H =31
LOWER CUTOFF FREQUENCY
UPPER CUTOFF FREQUENCY

(RELATIVE)
(RELATIVE)

= 0.5000000E-01
= 0.4500000E+00

IMPULSE RESPONSE:
H ( 0)
H ( 1)
H ( 2)
H ( 3)
H ( 4)
H ( 5)
H< 6)
H ( 7)
H ( 8)
H ( 9)
H (10)
H (11)
H(12)
H (13)
H(14)

= -0.1342662E-03
=
0.2133148E-02
0.4848863E-02
= 0.2286159E-02
0.1423532E-01
= 0.1517075E-02
=
0.3001805E-01
= 0.5263533E-03
=
0 .5 5 7 4 7 2 1 E - 0 1
= -0.2281570E-03
= 0.1001032E+00

H (22)
H(23)
H(24)
H(25)
H (26)
H (27)

=
=
=
=
=

-0. 5 3 3 8 3 2 6 E - 0 3
= 0.1949848E+00
= -0.3994641E-03
= 0.6307253E+00
H {15) = - 0 . 9 3 3 5 9 5 6 E - 0 6
H (16) s - 0 . 6 3 0 7 2 4 5 E + 0 0
H ( 1 7 ) = 0.3996222E-03
H (18) = - 0 . 1 9 4 9 8 5 3 E + 0 0
H (19) = 0 . 5 3 4 1 3 0 7 E - 0 3
H (20) = - 0 . 1 0 0 1 0 3 5 E + 0 0
H (21) = 0 . 2 2 8 5 3 3 8 E - 0 3
-0.5574735E-01
-0.5263340E-03
-0.3001794E-01
-0.1517240E-02

-0.1423557E-01
= -0.2285915E-02
H (28) = - 0 . 4 8 4 8 2 1 5 E - 0 2
H (29) = - 0 . 2 1 3 3 8 0 0 E - 0 2
H (30) = 0 . 1 3 4 4 1 6 2 E - 0 3

666

Design of Digital Filters

Chap. 8

T h ese form ulas are extrem ely useful in obtaining a g o o d estim ate o f the filter
length required to ach ieve the given specifications A / , 61, and 82 . T he estim ate
is used to carry out the design and if the resulting 8 ex ceed s th e specified 82 , the
length can b e increased until w e obtain a sid elo b e level that m eets the specifica
tions.

8.3 DESIGN OF IIR FILTERS FROM ANALOG FILTERS

Just as in the design o f F IR filters, there are several m eth od s that can be used to
design digital filters having an infinite-duration unit sam ple resp onse. T he tech
niques described in this section are all based on converting an analog filter into
a digital filter. A n a lo g filter design is a m ature and w ell d ev elo p ed field, so it is
n ot surprising that w e begin the design o f a digital filter in the analog dom ain and
then convert the design into the digital dom ain.
A n analog filter can be described by its system function.
M
(8.3.1)

0
w here {a*} and {/S*} are the filter coefficients, or by its im pulse resp onse, which is
related to Ha (s) by the L aplace transform
(8.3.2)
A ltern atively, the analog filter having the rational system fun ction H ( s ) given in
(8.3.1), can b e described by the linear con stan t-coefficien t d ifferen tial equation
(8.3.3)
w here x ( t ) d en o tes the input signal and ;y(f) d en o tes the ou tp u t o f the filter.
E ach o f th ese three eq u ivalen t characterizations o f an an alog filter leads to
alternative m eth od s for converting the filter in to the digital dom ain, as will be
described in S ection s 8.3.1 through 8.3.4. W e recall that an analog linear tim einvariant system with system function H ( s ) is stable if all its p o le s lie in the left
half o f the j-p la n e. C on sequ en tly, if the con version tech n iqu e is to be effective, it
should p ossess th e follow in g d esirable properties:
L T h e j Q axis in the s-p lan e should m ap in to the unit circle in the z-plane.
T hus there w ill b e a direct relationship b etw een the tw o frequency variables
in the tw o dom ains.

Sec. 8.3

Design of IIR Filers From Analog Fitters

667

2. T h e left-h alf plane (L H P ) o f the 5-plane should m ap into the inside o f the
unit circle in the z-plane. T hus a stable analog filter w ill b e con verted to a
stable digital filter.
W e m en tio n ed in the p receding section that physically realizable and stable
IIR filters cannot have linear phase. R ecall that a linear-phase filter m ust have a
system function that satisfies the con d ition
H ( z ) = z ~ n H ( z ~ 1)

(8.3.4)

w here z ~ N represents a d elay o f N units o f tim e. B u t if this w ere the case, the
filter w ou ld have a m irror-im age p o le ou tsid e the unit circle for every p o le inside
th e unit circle. H en ce the filter w ou ld be unstable. C on sequ en tly, a causal and
stable IIR filter cannot have linear phase.
If the restriction o n physical realizability is rem oved , it is p ossib le to obtain
a linear-phase IIR filter, at least in principle. T his approach in volves perform ing a
tim e reversal o f the input signal x ( n ), passing x ( n) through a digital filter H( z ) ,
tim e-reversing the output o f H( z ) , and finally, passing th e result through H( z )
again. T his signal processing is com putationally cu m bersom e and appears to offer
no advantages over linear-phase F IR filters. C on sequ en tly, w hen an application
requires a linear-phase filter, it should be an F IR filter.
In the design o f IIR filters, w e shall specify the desired filter characteristics
for the m agnitude response only. T his d oes not m ean that w e con sid er the phase
resp onse unim portant. Since the m agnitude and phase characteristics are related,
as indicated in Section 8.1, w e specify the desired m agnitude characteristics and
accept the phase response that is ob tain ed from the design m eth od ology.

8.3.1 IIR Filter Design by Approximation of Derivatives

O n e o f the sim plest m eth od s for converting an analog filter in to a digital filter is to
approxim ate the differential eq u ation in (8.3.3) by an eq u ivalen t d ifferen ce eq u a
tion. T his approach is often used to solve a linear con stan t-coefficien t differential
eq u ation num erically on a digital com puter.
For the derivative d y { t ) / d t at tim e t n T , w e substitute the b a c k w a r d di f
ference ^ ( n T ) y ( n T l ) ] / 7 \ Thus
dy ( t )

y ( n T ) - y i n T - T)

dt
(g.3.5)
w h ere T represents the sam pling interval and y(n) = y { n T ) . T h e analog differ
en tiator w ith output d y ( t ) / d t has th e system function H ( s ) = s, w hile the digi
tal system that produces the output [y(n) >-(n 1 ) ] /T has the system function
H ( z ) = (1 z ~ y) / T . C on sequ en tly, as show n in Fig. 8.29, th e frequency-dom ain

668

Design of Digital Filters

yit)

Chap. 8

dyjt)
dt

H(s) = s
(a)

y(n)

y(n) - v(n -l)

i-z

H(Z)

FigHre fL29 Substitution of the

backward difference for the derivative
implies the mapping s (1 z~l )/T.

(b)

equ ivalen t for the relationship in (8.3.5) is

1 - z "1

(8.3.6)

T h e second derivative d 2 y ( t ) / d t 2 is replaced by the secon d differen ce, which

is derived as follow s:
d

d 2y{t)
dt2

i= nT

'dy(t)

dt

t=nT

\ y ( n T) - y ( n T - T ) ] / T - [ y ( n T - T) - y { n T - 2 T ) ] / T
y ( n ) - 2 y ( n - 1) + y( n - 2)

(8.3.7)

r2

In the frequency dom ain, (8.3.7) is equ ivalen t to

2

2 z ] +

Z~2

T2

( \

~ \

- I n
z _1

2
(8.3.8)

It easily fo llow s from th e discussion that th e substitution for th e Jfcth derivative

of y ( t ) results in the eq u ivalen t frequency-dom ain relationship
k
s~ = t =

(8.3.9)

C onsequently, th e system function for the digital IIR filter ob tain ed as a result of
the approxim ation o f the d erivatives by finite d ifferen ces is
H(z) =

(8.3.10)

w here Ha(s) is th e system function o f th e analog filter characterized by th e differ

ential equation given in (8.3.3).
L et us investigate the im plications o f the m apping from th e s-p lan e to the
z-plane as given by (8.3.6) or, equivalently,

1
2

1-sT

If w e substitute s = jQ, in (8.2.11), w e find that

(8.3.11)

Sec. 8.3

669

Design of IIR Filters From Analog Filters

1
. QT
1+ &T2+ J \ + &T2

(8.3.12)

A s Q varies from oo to oo, the corresponding locus o f p oin ts in th e z-p lane is a

circle o f radius \ and with cen ter at z
as illustrated in Fig. 8.30.
It is easily d em onstrated that the m apping in (8.3.11) takes p oin ts in the
L H P o f the 5-plane in to corresponding p oints inside this circle in the z-plane and
p oints in the R H P o f the 5-plane are m apped in to p oints ou tsid e this circle. C on
sequ en tly, this m apping has the desirable property that a stable analog filter is
transform ed into a stable digital filter. H ow ever, the p ossib le location o f the p oles
o f the d igital filter are confined to relatively sm all freq u en cies and, as a co n se
q u en ce, the m apping is restricted to the design o f low p ass filters and bandpass
filters having relatively sm all resonant frequencies. It is n ot p ossib le, for exam
p le, to transform a highpass analog filter into a corresponding highpass digital
filter.
In an attem pt to overcom e the lim itations in the m apping given ab ove, m ore
com p lex substitutions for the derivatives have b een p rop osed . In particular, an
Lth-order d ifferen ce o f the form

y(nT + kT) y(nT kT)

_ _

(8.3.13)

has b een p rop osed , w h ere {a*} are a set o f param eters that can b e selected to
op tim ize the approxim ation. T h e resulting m apping b etw een the 5-plan e and the
z-plane is n ow
(8.3.14)

Unit circle

j-plane

Figure 8 3 0

The mapping s = (1 z l ) / T takes LHP in the J-plane into points

inside the circle o f radius j and center r = j in the 2-plane.

670

Design of Digital Filters

W h en z =

Chap. 8

w e have
L

j Y ak sin w *
*=i

(8.3.15)

w hich is purely im aginary. T hus

2 x^
1 = ^ a* sin<uJt
* k=l

(8.3.16)

is the resulting m apping b etw een the tw o frequency variables. B y proper choice
o f the coefficients {a*} it is p ossib le to m ap the j l - axis in to th e unit circle. Fur
therm ore, p oin ts in the L H P in 5- can b e m apped into points in sid e the unit circle
in z.
D esp ite achieving the tw o d esirable characteristics w ith th e m apping of
(8.3.16), the prob lem o f selectin g the set o f coefficients (a*} rem ains. In general,
this is a difficult problem . Since sim pler tech n iqu es exist for converting analog
filters in to IIR digital filters, w e shall not em p hasize the u se o f the Lth-order
d ifferen ce as a sub stitu te for the derivative.

Example 83.1
Convert the analog bandpass filter with system function
H{S) = (*'+0.1 )2 + 9
into a digital IIR filter by use of the backward difference for the derivative.

Solution Substitution for s from (8.3.6) into H(s) yields

H(z) =

( i ^ + 0.l) +9

r 2/(i + o . 2 r + 9.oir2)
2(1 + 0.17)
,
1
1 + 0 . 2 7 + 9.01 f 2

l+ 0 .2 7 + 9 .0 ir 2

The system function H(z) has the form of a resonator provided that T is selected
small enough (e.g., T < 0 .1 ), in order for the poles to be near the unit circle. Note
that the condition af < 4ai is satisfied, so that the poles are complex valued.
For example, if T = 0.1 , the poles are located at
P i,2 = 0.91 jO.27

= 0.949eyi6'5
W e n o te that th e range o f reson ant freq u en cies is lim ited to lo w freq u en cies, due to
the characteristics o f the m apping. T h e reader is en cou raged to p lo t the frequency
resp o o se H(co) o f th e digital filter for d ifferen t valu es o f T and com p are th e results
w ith th e frequency resp onse o f the an alog filter.

Sec. 8.3

Design of IIR Filters From Analog Filters

671

Example 8-3.2
Convert the analog bandpass filter in Example 8.3.1 into a digital IIR filter by use of
the mapping

s = j(z~ z-')

Solution By substituting for j in H (j), we obtain

H{z) *

1
+9
z2T

~ z4 + 0.27 V + (2 + 9.01T2)z2 - 02Tz + 1

W e observe that this mapping has introduced two additional poles in the con
version from Ha(s) to H(z). As a consequence, the digital filter is significantly more
complex than the analog filter. This isa major drawback to the mapping given above.
8.3.2 IIR Filter Design by Impulse Invariance
In the im pulse invariance m ethod, our objective is to d esign an IIR filter having a
unit sam ple response h(n) that is the sam pled version o f th e im pulse response o f
the analog filter. That is,
h(n) = h( n T)

n = 0,1,2,...

(8.3.17)

w h ere T is th e sam pling interval.

T o exam in e the im plications o f (8.3.17), w e refer back to S ection 4.2.9. R ecall
that w hen a con tinu ou s tim e signal xa (t) with spectrum X a ( F) is sam pled at a
rate Fs \ / T sam ples per second, the spectrum o f th e sam pled signal is the
period ic rep etition o f the scaled spectrum FsX a( F) w ith period Fs . Specifically,
the relationship is
OO
X ( f ) = Fs
X a[ ( f r k)F, ]
(8.3.18)

**-00
w h ere / = F / F s is the norm alized frequency. A liasin g occurs if the sam pling rate
Fj is less than tw ice th e highest frequency con tained in X a (F).
E xp ressed in the con text o f sam pling th e im pulse resp onse o f an analog
filter with frequency response Ha (F), the digital filter w ith unit sam ple response
h(n) s ha{ nT) has the frequency resp onse
OO

W ) = F,

Ha[ ( f - k)Fs]

(8.3.19)

H a [ (to - 2 n k ) F s]

(8.3.20)

**
00
or, eq u ivalen tly,

00
H(a>) = Fs Y

672

Design of Digital Fitters

Chap. 8

H J C iT )

Figure 8J 1 Frequency response H(&) of the analog filter and frequency re

sponse of the corresponding digital filterwith aliasing.
or
H&T) = J

Ha ( n -

(8.3.21)

Figure 8.31 depicts the frequency resp onse o f a low pass an alog filter and the
frequency response o f the corresponding digital filter.
It is clear that the digital filter w ith frequency resp onse H(<d ) has the fre
quency response characteristics o f th e corresponding analog filter if the sam pling
interval T is selected sufficiently sm all to com p letely avoid or at least m inim ize
the effects o f aliasing. It is also clear that the im pulse invariance m eth od is in
appropriate for d esigning highpass filters due the to spectrum aliasing that results
from the sam pling process.
T o investigate the m apping o f p oin ts b etw een the z-p lane and the j-p la n e
im plied by the sam pling process, w e rely on a generalization o f ( 8.3 .2 1 ) which
relates the z-transform o f h(n) to the L aplace transform o f ha (t). This relation
ship is
fl(z )i * = j Y

H ( * -

^ 3 -22)

Sec. 8.3

673

Design of IIR Filters From Analog Filters

w here

OO
H(z) = X > ( n ) z - n
73=0
OO
* Y

h M e ' sTn

<8 -3-23)

N o te that w hen s = j Q , (8.3.22) reduces to (8.3.21), where th e factor o f j in Ha (Q)

is suppressed in our notation.
L et us con sid er the m apping o f p oin ts from the s-p lan e to the z-p lane im plied
by the relation
z - esT

(8.3.24)

If w e sub stitu te j = a + j Q. and express the com p lex variable z in polar form as
e = r e jai, (8.3.24) b eco m es
r e j = eaTejnT
Clearly, w e m ust have
r = eaT
(8.3.25)

co = Q.T
C on sequ en tly, a < 0 im plies that 0 < r < 1 and a > 0 im plies that r > 1. W hen
a = 0, w e have r ~ 1. T h erefore, the L H P in s is m apped inside the unit circle in
z and the R H P in s is m apped ou tsid e the unit circle in z.
A lso , the j Q- ax\ s is m apped into th e unit circle in z as indicated above. H o w
ever, the m apping o f the y'fi-axis in to the unit circle is n ot on e-to -o n e. Since co
is unique over th e range (tt, tt), the m apping co = Q.T im p lies that the interval
n / T < 2 < n / T m aps into the corresponding values o f tt < co < n. Fur
therm ore, the frequency interval n / T < Cl < 3 n / T also m aps in to the interval
n < co < 7i and, in general, so d o es the interval (2k - V ) n / T < 2 < (2k -(- \ ) t t / T ,
w hen k is an integer. T hus the m apping from th e analog frequency 2 to the fre
q uency variable co in the digital dom ain is m any-to-on e, w hich sim ply reflects the
effects o f aliasing due to sam pling. Figure 8.32 illustrates the m apping from the
j-p la n e to the z-p lane for the relation in (8.3.24).
T o exp lore further the effect o f the im pulse invariance design m eth od on
th e characteristics o f the resulting filter, let us express the system function o f the
analog filter in partial-fraction form. On the assum ption that the p o les o f the
analog filter are distinct, w e can write
N

Ha(s) = Y

(8.3.26)

w here {/>*} are the p o les o f the analog filter and {c*} are the coefficients in the
partial-fraction exp an sion . C onsequently,
N

M O = Y CkePkt
k- i

(8.3.27)

674

Design of Digital Filters

Chap. 8

Figure 8_32 The mapping of z e*T

maps strips of width 2n / T (for cr < 0) in
the 5-plane into points in the unit circle
in the z-plane.

If w e sam ple h a(t) p eriodically at t ~ nT, w e have

h(n) = h A n T )

(8.3.28)
N ow , with the substitution o f (8.3.28), the system fun ction o f the resulting digital
IIR filter b eco m es
H( z ) = ] T / i ( n ) z - n
w*0
oo / N

= E
n=0 \*=1
N

(8.3.29)
i= l

=o

T he inner sum in (8.3.29) con verges b ecau se p* < 0 an d yields

(8.3.30)
T herefore, the system function o f the digital filter is
H (z) = Y

f t

- ------- f

1 - e Tz - 1

(8.3.31)

W e observe that the digital filter has p o les at

_

ffPkT

Zk = e

k = 1 , 2 , . . . , AT

(8.3.32)

Sec. 8.3

675

Design of HR Fitters From Analog Filters

A lth o u g h the p o les are m apped from the .y-plane to th e z-p lane by the relationship
in (8.3.32), w e should em phasize that the zeros in th e tw o dom ains do not satisfy the
sam e relationship. T herefore, the im pulse invariance m eth od d o es not correspond
to the sim p le m apping o f points given by (8.3.24).
T h e d ev elo p m en t that resulted in H ( z ) given by (8.3.31) was based on a filter
having distinct p oles. It can b e generalized to in clu d e m ultiple-order p oles. For
brevity, h ow ever, w e shall not attem pt to gen eralize (8.3.31).

Example 833
Convert the analog filter with system function

into a digital IIR filter by means of the impulse invariance method.

Solution W e note that the analog filterhas a zero at s = -0.1 and a pair of complexconjugate poles at
pk = - 0.1 j 3
as illustrated in Fig. 8.33.
W e do not have to determine the impulse response ha{t) in order to design
the digital IIR filter based on the method of impulse invariance. Instead, we directly
determine H{z), as given by (8.2.31), from the partial-fraction expansion of Ha(s).
Thus we have
l
2

Then
l

Figure 8 3 3 Pole-zero locations for

analog filter in Example 83.3.

676

Design of Digital Filters

Chap. 8

Since the two poles are complex conjugates, we can combine them to form a
single two-pole filterwith system function
H{z) =

1 (e 017 cos3T)z~
1 (2e~0 lT cos 3T)z~l + e~2Tz~

The magnitude of the frequency response characteristic of this filter is plotted

in Fig. 8.34 for T = 0.1 and T = 0.5. For purpose of comparison, we have also plotted
the magnitude of the frequency response of the analog filter in Fig. 8.35, W e note
that aliasing is significantly more prevalent when T 0.5 than when T = 0.1. Also,
note the shift of the resonant frequency as T changes.

Figure 8.35 Frequency response of

analog filler in Exam ple 8.3.3.

T he preceding exam ple illustrates the im portance o f selectin g a sm all value

for T to m inim ize the effect o f aliasing. D u e to the presen ce o f aliasing, the
im pulse invariance m ethod is appropriate for the design o f low p ass and bandpass
filters only.

8.3.3 IIR Filter Design by the Bilinear Transformation

T h e IIR filter design tech n iqu es describ ed in the p receding tw o section s have a
severe lim itation in that they are appropriate only for low pass filters and a lim ited
class o f bandpass filters.
In this section w e describe a m apping from th e J-plane to th e z-plane, called
the bilinear transform ation, that overcom es the lim itation o f th e other tw o design

Sec. 8.3

Design of IIR Filters From Analog Filters

677

m eth od s d escribed previously. T h e bilinear transform ation is a conform al m apping

that transform s the yft-axis in to the unit circle in the z-p lane only on ce, thus
avoiding aliasing o f frequency com p onents. Furtherm ore, all points in the L H P o f
s are m app ed inside th e unit circle in th e z-p lane and all p oin ts in the R H P o f s
are m apped in to corresponding p oin ts ou tsid e the unit circle in the z-plane.
T he bilinear transform ation can b e linked to the trapezoidal form ula for
num erical integration. For exam ple, let us con sid er an analog linear filter with
system function
H( s ) =
s + a

(8.3.33)

T his system is also characterized by the differential equation

dy(t)
------h ay( t ) bx( t )
(8.3.34)
at
Instead o f substituting a finite d ifferen ce for th e d erivative, suppose that w e in
tegrate the derivative and approxim ate the integral by th e trapezoidal form ula.
Thus
y O ) = / y ' < j ) d r + y ( t 0)
= J/o
/> '

(8.3.35)

w here y'(t) d en o tes the derivative o f y( t ). T he approxim ation o f the integral in

(8.3.35) by the trapezoidal form ula at r ~ nT and t 0 = nT T yields
y ( n T ) = j [ y \ n T ) + y' ( nT - T)] + y { n T - T)

(8.3.36)

N o w the differential eq u ation in (8.3.34) evalu ated at t = n T yields

y ' ( n T ) - a y ( n T ) + bx ( nT )

(8.3.37)

W e u se (8.3.37) to substitute for th e derivative in (8.3.36) and thus ob tain a dif

feren ce eq u ation for the equivalent discrete-tim e system . W ith y(n) = y(n T ) and
x( n) = x( n T ) , w e obtain the result
^1 +

'J y ( n) ~

~ f ) y(~n

~ f l x (n ) + x (n ~

(8.3.38)

T h e z-transform o f this differen ce equation is

( i + ~ ) y(z) - ( ! ~

) z ~l y ( z ) = T

(1 +

Z1)X(Z)

C on sequ en tly, th e system function o f th e equivalent digital filter is

=
{Z)

X( z )

( bT pL)(\ + z~')
1 + a T / 2 ~ (1 a T pL)z~l

or, equivalently,
H(z) =

------

(8.3.39)

678

Design of Digital Fitters

Clearly, the m apping from the

j -p lane

Chap. 8

to th e z-p lane is

5= H r r )

(8'3'40>

T his is called the bilinear transformation.

A lth ou gh our derivation o f the bilinear transform ation w as p erform ed for a
first-order d ifferential eq u ation , it holds, in general, for an N th -ord er differential
equation.
T o investigate the characteristics o f the bilinear transform ation, let
z = r e i(u
s = a + ji2

T h en (8.3.40) can b e exp ressed as

=

2 z ~
Tz +

2 r e Ja> - 1
T re>w + 1
r2 1
2r sin co
= - ( + jT \ 1 + r 2 + 2r co s co
J 1 + r 2 + 2r cos co,
C onsequently,

_
A =

2
r2-
T 1 + r2 + 2 r cos <

(8.3.41)

2
2 r sin co
-------------
---------- --

( 8 .3 .4 2 )

T l+ r

2r

cos co

First, w e n o te that if r < 1, then a < 0, and if r > 1, th en a > 0. C on se

quently, the L H P in s m aps into the in sid e o f the unit circle in the z-plane and the
R H P in s m aps in to the ou tsid e o f the unit circle. W hen r = 1, then a = 0 and
^

Q =

sin to

T 1 + c o s co

= - tan 2
T

(8.3.43)
K

oT
co=2tan~1

(8.3.44)

or, equivalently,

T h e relationship in (8.3.44) b etw een the frequency variables in the tw o dom ains
is illustrated in Fig. 8.36. W e ob serve that the entire range in 2 is m apped only
o n ce in to the range n < a> < n . H ow ever, the m apping is high ly n onlinear. W e
o b serve a frequency com p ression or f r equency warping, as it is usually called , due
to the n onlinearity o f the arctangent function.
It is also interesting to n o te that th e b ilinear transform ation m aps th e point
s = oo into th e p oin t z 1. C on sequ en tly, th e sin gle-p ole low p ass filter in

Sec. 8.3

Design of IIR Fitters From Analog Filters

679

Figure 836

Mapping between the frequency variables to and Q resulting from

the bilinear transformation.

(8.3.33), which has a zero at s = oo, results in a digital filter that has a zero at
z = 1.
Example 83.4
Convert the analog filter with system function

into a digital IIR filter by means of the bilinear transformation. The digital filter is
to have a resonant frequency of wr = jt/2 .
Solution First, we note that the analog filter has a resonant frequency
= 4. This
frequency is to be mapped into cur = jt/2 by selecting the value of the parameter T.
From the relationship in (8.3.43), we must select T = \ in order to have cor - it f l .
Thus the desired mapping is

The resulting digital filter has the system function

0.128 + 0.006Z 1 - 0 .1 2 2 r 2
(Z)

1 + 0 .0 0 0 6 z - 5 + 0 .9 7 5 z -2

W e note that the coefficient of the z~l term in the denominator of H(z) is extremely
small and can be approximated by zero. Thus we have the system function
0.128 + 0 .0 0 6 z-1 - 0 .1 2 2 z -2

;---------T h is filter h as p o le s at
p i .2 = 0.987e,r/2

680

Design of Digital Filters

Chap. 8

and ze ro s at
zi

.2 =

- 1 , 0.95

Therefore, we have succeeded in designing a two-pole filter that resonates near u) =

nfl .
In this exam ple the param eter T w as selected to m ap th e resonant frequency
o f the analog filter in to the desired reson ant frequency o f the digital filter. U sually,
the design o f the digital filter begins w ith specifications in the digital dom ain, which
in volve the frequency variable <d . T h ese sp ecification s in frequency are converted
to the analog dom ain by m eans o f the relation in (8.3.43). T h e analog filter is then
d esign ed that m eets these sp ecification s and con verted to a digital filter by m eans
o f the bilinear transform ation in (8.3.40). In this procedure, the param eter T is
transparent and m ay b e set to any arbitrary value (e.g., T = 1). T he follow ing
exam p le illustrates this point.
Example 8.3.5
Design a single-pole lowpass digital filter with a 3-dB bandwidth of 0.2jt, using the
bilinear transformation applied to the analog filter
His) =
where

is the 3-dB bandwidth of the analog filter.

Solution The digital filter is specified to have its -3-d B gain at wc = 0.2n. In the
frequency domain of the analog filter we 0 .2 n corresponds to

fZc = tan0.l7r
_

065
7

Thus the analog filter has the system function

0.65/7
j + 0 .6 5 / 7

This represents our filter design in the analog domain.

Now, we apply the bilinear transformation given by (8.3.40) to convert the
analog filter into the desired digital filter. Thus we obtain
II(z)
( ) ~

1 - 0 .5 0 9 Z - 1

where the parameter 7 has been divided o u t

The frequency response of the digital filter is
0.245(1 + e~ju)
]

1 - 0.5 0 9 c-.'* '

At w = 0, H( 0) = 1, and at a> = 0.2?r, we have |//(0.2jr)| = 0.707, which is the desired

response.

Sec. 8.3

Design of IIR Filters From Analog Filters

681

8.3.4 The Matched-z Transformation

A n o th er m eth od for converting an analog filter into an eq u ivalen t digital filter is to
m ap the p o les and zeros o f H ( s ) directly into p o les and zeros in the z-plane. Sup
p o se that the system function of the analog filter is exp ressed in the factored form
M
Y [ ( S ~ z k)
H ( s ) = *=!-----------r > *=i

(8.3.45)

pa

w here {z*} are the zeros and {p*} are the p o les of the filter . T hen the system
function for the digital filter is
M
Y [ ( l - e ^ Tz ~ x)
(8.3.46)
f ] a - ^ V ' )
*=i
w here T is the sam pling interval. Thus each factor o f the form (s a) in H( s )
is m apped into the factor (1 eaTz ~ l ). This m apping is called the matched- z
transformation.
W e ob serve that the p oles ob tain ed from the m atched-z transform ation are
identical to the p oles ob tain ed with the im pulse invariance m eth od . H ow ever, the
tw o techniques result in d ifferent zero positions.
T o preserve the frequency resp onse characteristic o f the analog filter, the
sam pling interval in th e m atched-z transform ation m ust b e properly selected to
yield the p o le and zero locations at the eq u ivalen t position in the z-plane. Thus
aliasing m ust be avoid ed by selectin g T sufficiently sm all.

8.3.5 Characteristics of Commonly Used Analog Filters

A s w e have seen from our discussion ab ove, IIR digital filters can easily b e o b
tained by b eginning with an analog filter and then using a m apping to transform the
s -plane into the z-plane. Thus the design o f a digital filter is reduced to designing
an appropriate analog filter and then p erform ing the con version from H( s ) to H( z ) ,
in such a w ay so as to preserve as m uch as p ossib le, the desired characteristics o f
the analog filter.
A n a lo g filter design is a w ell-d evelop ed field and m any b ook s have b een
written on the subject. In this section w e briefly describe the im portant character
istics o f com m on ly used analog filters and introduce the relevan t filter param eters.
O ur discussion is lim ited to low pass filters. S u b seq u en tly, w e describe several
frequency transform ations that convert a low p ass p rototyp e filter in to either a
bandpass, highpass, or band-elim ination filter.

682

Design of Digital Filters

Chap. 8

Butterworth filters. L ow pass B utterw orth filters are all-p ole filters char
acterized by the m agnitude-squared freq u en cy response
1H (S2) 12 = l + ( 2 / n c)2* =

i + cH n/np)M

(8'3
47)

where N is the order o f the filter, 2C is its 3-dB freq u en cy (usually called the
cu toff freq u en cy), Qp is the passband ed ge frequency, and 1/(1 + e 2) is th e banded ge valu e o f |H (2)|2. Since H ( s ) H ( s) evaluated at j = j l is sim ply equal to
|//( 2)|2, it fo llo w s that

H(sW-s) - t

t f W

<8-3'48)

T he p o les o f H ( s ) H ( s) occur on a circle o f radius 2C at equally spaced points.

From (8.3.48) w e find that
= ( - l ) 1//v = e j & + * / *

k = 0,1,..., N - 1

and h ence
sk - n ce j 7r/2e j(2k+l)7r/2N

* = 0 , 1 .........N 1

(8.3.49)

For exam ple, Fig. 8.37 illustrates the p ole p osition s for an N = 4 and N 5
Butterw orth filters.
T he frequency response characteristics o f the class o f B utterw orth filters are
show n in Fig. 8.38 for several valu es o f N. W e n ote that
(2)|2 is m on oton ic
in both the passband and stopband. T h e ord er o f the filter required to m eet an
attenuation 82 at a specified frequency 2, is easily d eterm in ed from (8.3.47). Thus
at 2 = 2* w e have

*
i

-2

e H n I / a p ) 2N

61

and hence

_ tog [ ( l / g ) - 1 ] _ . W
O
21og(G,/W
log(S J,/n,)

w here, by definition, <52 = l / V l + 8 2. T hus th e B utterw orth filter is com p letely

characterized by the param eters N, 82, e, and the ratio Qs / l p.
Example 8.3.6
Determine the order and the poles of a lowpass Butterworth filter that has a -3-d B
bandwidth of 500 Hz and an attenuation of 40 dB at 1000 Hz.
Solution The critical frequencies are the 3-dB frequency Qc and the stopband
frequency 2,, which are
= IOOOjt
2, =

2000jt

Sec. 8.3

683

Design of HR Filters From Analog Filters

Figure 8.37

Pole positions for Butterworth filters.

For an attenuation of 40 dB, S2 = 0.01. Hence from (8.3.50) we obtain

log,o ( 10* -

1)

21ogio 2
= 6.64
To meet the desired specifications, we select N = 7. The pole positions are
st = \0007rei{nfl+cu+l,*/u]

0,1.2.....

Chebyshev filters. T h ere are tw o types o f C h eb ysh ev filters. T yp e I

C h eb ysh ev filters are all-pole filters that exhibit equiripple b eh avior in th e passband and a m o n o to n ic characteristic in the stopband. O n the other hand, the
fam ily o f type II C h eb ysh ev filters contains both p o le s and zeros and exh ib its a

684

Design of Digital Filters

Chap. 8

l#(2)P

m on o to n ic b eh avior in the passband and an eq u irip p le b eh avior in the stopband.

T h e zeros o f this class o f filters lie on the im aginary axis in th e s -plane.
T he m agnitude squared o f the frequency resp onse characteristic o f a type I
C h ebyshev filter is given as
|t f ( f i )|2 = ------- J-----------i + 2r 2 ( n / n , )

(8.3.51)

w here e is a param eter o f the filter related to the ripple in th e passband and Tn (x )
is the W th-order C hebyshev polyn om ial defined as
t

\ cos(Af c o s- 1 * ),

c o s h w c o d .- 1* ),

|x |

<1

|X|>1

, ox

( 8 J '52)

Sec. 8.3

685

Design of IIR Filters From Analog Filters

T he C h eb ysh ev p olyn om ials can be generated by the recursive equation

Tn+ i (.x ) = 2xTN(x) - Tn i ( x )

AT = 1 , 2 , . . .

(8.3.53)

w h ere 7 o U ) = 1 and T\(x) = x. From (8.3.53) w e obtain T^C*) = 2*2 1- T-six)

4 r 3 3 x , and so on.
Som e o f the p roperties o f these polyn om ials are as follow s:

1 . | 7 VU) | < 1 for all 1*1 < 1.

2 . 7V(1) = 1 for all N.
3. A ll the roots o f th e p olyn om ial T?j(x) occur in th e interval 1 < * < 1.
T h e filter param eter is related to the ripple in th e passband, as illustrated
in Fig. 8.39, for N odd and N even. For N odd, TN(0) = 0 and h en ce | / / ( 0 )|2 = 1.
O n th e other hand, for N even, TN(0) = 1 and h en ce | / / ( 0 )|2 = 1/(1 -f e 2). A t the
band ed ge frequency SI = Qp, w e have T,y(l) = 1 , so that

or, equivalently.

w h ere <$i is the value o f the passband ripple.

The p o les o f a type I C hebyshev filter lie on an ellip se in the j-p la n e with
m ajor axis
(8.3.55)
iffrti)!*

N odd
Figure 8-39

N even
Type I Chebyshev filter characteristic.

686

Design of Digital Filters

Chap. 8

and m inor axis

-1

(8.3.56)

20

w here

is related to e according to the equation

vT

+72 + 1

1/ N

(8.3.57)

T he p ole locations are m ost easily d eterm in ed for a filter o f ord er N by first locating
the p o les for an equ ivalen t iVth-order B utterw orth filter that lie on circles o f radius
rj or radius r2, as illustrated in Fig. 8.40. If w e d en o te the angular p osition s o f the
p o les o f the B utterw orth filter as
(2k +

1)7T

(8.3.58)
2N
then the p osition s o f the p oles for the C hebyshev filter lie on the ellip se at the
coordinates (xk, yt ) , k = 0, 1 , . . . , N 1 , w here
xk = r 2 cos <pk,

k = Q ,l,...,N -l

yk = r i sin <pk.

k = 0,1, . . . , N - 1

(8.3.59)

Figure 8.40

Determination o f the pole locations for a Chebyshev filter.

Sec. 8.3

687

Design of IIR Filters From Analog Filters

A type II C hebyshev filter contains zeros as w ell as p oles. T h e m agnitude

squared o f its frequency resp onse is given as
\ H (2)|2 =

(8.3.60)

w here Ty( x) is, again, the N th -ord er C hebyshev p olyn om ial and 2* is the stopband
frequency as illustrated in Fig. 8.41. T h e zeros are located on the im aginary axis
at th e points
k = 0,1,..., N - 1

Sk = j sin f a

(8.3.61)

T h e p o les are located at the points (vk, w k), w here

vk =

Slsxk

0, 1 ,

-1

(8.3.62)

* = 0,1,..., A f-1

(8.3.63)

* =

yf*l + Vk
wk

Qyk
y/xk + y k2

where j**} and {yk} are defined in (8.3.59) with 0 now related to the ripple in the
stopband through the equation

1/jV
1

+ J l - S 2'
(8.3.64)

From this description, w e o b serve that the C hebyshev filters are characterized
by the param eters N, e, S2, and the ratio Q.s / Sl p . For a given set o f specifications
IWDI2

|tf(n)P

n , n,
N odd

N even
Figure 8.41

Type II Chebyshev filters.

Design of Digital Filters

688

Chap. 8

on (, S2, and Q.s/Q.p, we can detennine the order of the filter from the equation
log

l(v

N =

fl-Sl + y /l-S ld + ^ j/e S !

log (ns/ Q p) + y/ ( n s / n p)2 - 1

(8.3.65)

cosh (5/e)
co sh -1 (2, / ^ )
w here, by definition, S2 =

1 / V l + <52.

Example 83.7
Determine the order and the poles of a type I lowpass Chebyshev filter that has a
1-dB ripple in the passband, a cutoff frequency 2P = IOOOjt, a stopband frequency
of 2000jt, and an attenuation of 40 dB or more for 2 > S2s.
Solution First, we determine the order of the filter. W e have
101og10(l + e2) = 1

1 + 2 = 1.259
2 = 0.259

e = 0.5088
Also,
20 log10<52 = -40
82

0.01

Hence from (8.3.65) we obtain

N =

^Sio 196.54
log10(2 +V3)

= 4.0
Thus a type I Chebyshev filter having four poles meets the specifications.
The pole positions are determined from the relations in (8.3.55) through (8.3.59).
First, we compute /?,n, and r2. Hence
P = 1.429
ri = 1.062P

r2 = 0.3650,
The angles {0 *} are
<Pt =

2+

(2k + 1)7T
8

k = 0 ,1,2 ,3

Therefore, the poles are located at

*1 + j y 1 = ~0.13972P j0.979np

+ jyi = 0.3372,, _/'0.4056Qp

Sec. 8.3

689

Design of IIR Filters From Analog Filters

T h e filter specifications in E xam p le 8.3.7 are very sim ilar to the specifications
g iven in E xam p le 8.3.6, w hich in volved the design o f a B utterw orth filter. In
that case th e num ber o f p o les required to m eet th e sp ecification s was seven . O n
the other hand, the C hebyshev filter required on ly four. T h is result is typical o f
such com parisons. In general, the C hebyshev filter m eets the specifications with a
few er num ber o f p o les than the corresponding B utterw orth filter. A ltern atively, if
w e com p are a B utterw orth filter to a C h eb ysh ev filter having th e sam e num ber
o f p o les and the sam e passband and stopband specifications, the C hebyshev filter
will h a v e a sm aller transition bandwidth. For a tabulation o f the characteristics o f
C h eb ysh ev filters and their p o le-zero locations, the in terested reader is referred
to th e h an d b ook o f Z verev (1967).

Elliptic filters. E llip tic (or C auer) filters exhibit eq uiripple b eh avior in b oth
the passband and the stopband, as illustrated in Fig. 8.42 for N odd and N even.
T h is class o f filters contains both p o les and zeros and is characterized by the
m agnitude-squared frequency response

|H (^ )|2 = 1

u.

277~; 0 / o ,

(8-3 -66)

w h ere UN{:r) is the Jacobian elliptic function o f order N , w hich has b een tabulated
by Z v erev (1967), and e is a param eter related to the passband ripple. T h e zeros
lie on the ;'fi-axis.
W e recall from our discussion o f F IR filters that the m ost efficient designs
occur w hen w e spread the approxim ation error equally over the passband and the
stop b an d . E llip tic filters accom plish this ob jective and, as a con seq u en ce, are the
m ost efficien t from the view p oin t o f yielding the sm allest-ord er filter for a given
IW(Q)|2

1 + e2

\S

N odd
Figure 8.42

Magnitude-squared frequency characteristics o f elliptic filters.

690

Design of Digital Filters

Chap. 8

set o f specifications. E quivalently, w e can say that for a given order and a given
set o f specifications, an elliptic filter has the sm allest transition bandwidth.
T he filter order required to ach ieve a given set o f specifications in passband
ripple Si, stopband ripple 62, and transition ratio ftp /ft* is given as
K ( n pM ) K U i - f i / s * ) )
N = --------------- v
L
K W K ( y i - ( 8 , / n , )2j

(8.3.67)

w h ere K ( x ) is the co m p lete elliptic integral o f the first kind, defined as

K (X ) =

r a

'0

d$

- ........... .
v

1 jc 2 s u v

(8.3.68)
6

and 62 = 1 / V l + 62. V alu es o f this integral have b een tabu lated in a num ber
o f texts [e.g., th e b ook s by Jahnke and E m d e (1945) and D w igh t (1957)]. T h e
passband ripple is 10 lo g 10( l + e2).
W e shall n ot attem pt to describe ellip tic functions in any d etail b ecau se such
a discussion w ould take us to o far afield. Suffice to say that com p u ter program s are
available for designing ellip tic filters from the frequency specifications indicated
above.
In v iew o f the optim ality o f ellip tic filters, the reader m ay qu estion the reason
for considering the class o f Butterw orth or th e class o f C h eb ysh ev filters in practical
applications. O n e im portant reason that th ese other types o f filters m ight be prefer
able in som e applications is that they p ossess b etter phase resp on se characteristics.
T h e phase resp onse o f ellip tic filters is m ore non linear in the passband than a com
parable B utterw orth filter or a C h eb ysh ev filter, especially near the band edge.

Bessel filters. B e sse l filters are a class o f all-p ole filters that are charac
terized by the system function
H( s ) =

( 8. 3. 69)

B n (s )

w h ere B n (s ) is the N th -ord er B e sse l polynom ial. T h ese p olyn om ials can b e ex
p ressed in the form
N

B * (j) = a * j *

(8-3.7)

w h ere the coefficients {a*} are given as

&N - t y

' 2 - > * ! ( * - * ) !

,
* - ( U .... N

<8-371)

A ltern atively, the B essel p olyn om ials m ay b e gen erated recursively from the rela
tion
B n (s) = (2N - 1)B*_j(i) + s 2B n . 2(s)
with Bo(s) = 1 and fli(s ) = s + 1 as initial conditions.

(8.3.72)

Sec. 8.3

Design of IIR Filters From Analog Filters

691

Magnitude

Phase

Figure 8.43 Magnitude and phase responses o f Bessel and Butterworth filters of
order N = 4.

A n im portant characteristic o f B essel filters is the linear-phase response o ver

the passband o f the filter. For exam p le, Fig. 8.43 sh ow s a com parison o f th e
m agnitude and phase resp onses o f a B e sse l filter and B utterw orth filter o f order
N = 4. W e n o te that the B essel filter has a larger transition bandwidth, but
its phase is linear w ithin the passband. H ow ever, w e should em p hasize that the

692

Design of Digital Filters

Chap. 8

linear-phase chacteristics o f the analog filter are d estroyed in th e p rocess o f con

verting the filter into the digital dom ain by m eans o f the transform ations decribed
previously.

8.3.6 Some Examples of Digital Filter Designs Based on

the Bilinear Transformation
In this section w e presen t several exam p les o f digital filter d esign s ob tain ed from
analog filters by applying the bilinear transform ation to con vert H( s ) to H(z).
T h ese filters design s are p erform ed with the aid o f o n e o f several softw are packages
now available for use on a p ersonal com puter.
A low pass filter is design ed to m eet specifications o f a m axim um ripple o f
- dB in the passband, 60-dB attenuation in th e stopband, a passband ed ge fre
quency o f (i)p = 0.25;r, and a stopband ed ge frequency o f cos = 0.30tt.
A B utterw orth filter o f order N = 37 is required to satisfy the specifications.
Its frequency response characteristics are illustrated in Fig. 8.44. If a C heby
shev filter is used, a filter o f order N = 13 satisfies the sp ecification s. T he fre
quency resp onse characteristics for a type I and type II C h eb ysh ev filters are
show n in Figs. 8.45 and 8.46, respectively. T he type I filter has a passband rip
p le o f 0.31 dB. Finally, an elliptic filter o f order N 7 is d esign ed which also
satisfied the specifications. For illustrative purposes, w e sh ow in T able 8.11, the
num erical valu es for the filter param eters and the resulting frequency specifica
tions are show n in Fig. 8.47. T he follow in g n otation is used for th e param eters in
the function H(z):
,

r r W ' > + W

' 1)z_1 + k(i, 2) z ~2

/C Q w
(8.3.73)

A lth o u g h w e have described only low pass analog filters in th e p reced in g section, it
is a sim ple m atter to convert a low pass analog filter into a ban d pass, bandstop, or
highpass analog filter by a frequency transform ation, as is d escrib ed in Section 8.4.
T he bilinear transform ation is then applied to convert the an alog filter into an
equivalent digital filter. A s in the case o f the low p ass filters describ ed ab ove, the
entire design can b e carried ou t on a com puter.

8.4 FREQUENCY TRANSFORMATIONS

T he treatm ent in th e p receding section is focu sed prim arily on the d esign o f lowpass IIR filters. If w e wish to design a highpass or a bandpass or a bandstop filter,
it is a sim ple m atter to take a low pass prototyp e filter (B u tterw orth , Chebyshev,
elliptic, B e sse l) and perform a frequency transform ation.
O n e possib ility is to perform the frequency transform ation in th e analog
dom ain and then to convert the analog filter into a corresp on d in g digital filter
by a m apping o f the 5-plane in to the z-plane. A n alternative approach is first to

Sec. 8.4

Frequency Transformations

Figure 8.44

693

Frequency response characteristics of a 37-order Butterworth filter.

convert the analog lowpass filter into a lowpass digital filter and then to transform
the lowpass digital filter into the desired digital filter by a digital transformation.
In general, these two approaches yield different results, except for the bilinear
transformation, in which case the resulting filter designs are identical. These two
approaches are described below.
8.4.1 Frequency Transformations in the Analog Domain

First, we consider frequency transformations in the analog domain. Suppose that

we have a lowpass filter with passband edge frequency Qp and we wish to convert

694

Design of Digital Filters

Figure 8.45

Chap. 8

Frequency response characteristics o f a 13-order type I Chebyshev filter.

it to another low pass filter with passband ed ge frequency 2^,. T h e transform ation
that accom plishes this is
j y s

(low pass to low pass)

(8.4.1)

Thus w e obtain a low pass filter w ith system function Hi ( s) = Hp [(Qp/ t t p)s],
w here Hp(s) is th e system function o f the p rototyp e filter w ith passband ed ge
frequency Qp.

Sec. 8.4

Frequency Transformations

Figure M

695

Frequency response characteristics o f a 13-order type II Chebyshev filter.

If w e w ish to convert a low pass filter in to a highpass filter with passband

ed g e frequency Q!p, the desired transform ation is
s -------(low pass to highpass)
(8.4.2)
s
T h e system function o f th e highpass filter is Hk(s) Hp (Q.pQ!p/s).
T h e transform ation for converting a low pass analog filter with passband ed ge
frequency S2P in to a band filter, having a lo w er band ed ge frequency Qj and an
upper band ed g e frequency 2, can b e accom plished by first converting the low pass

696

Design of Digital Filters

TABLE 8.11

Chap. 8

FILTER COEFFICIENTS FOR A 7-ORDER ELLIPTIC FILTER

I N F I N I T E I M P U L S E R E S P O N S E (IIR)
ELLIPTIC LOWPASS FILTER

UNQUANTIZED COEFFICIENTS
FILTER ORDER = 7
SAMPLING FREQUENCY = 2.000 KILOHERTZ
I.

A(I,

1)

A(I,

2)

B (I , 0}

B (I , 1)

B (I , 2)

I
2

-.790103
.000000
.104948
. 104948
.000000
.714 0 8 8
.102450
-.007817
-1.517223
.102232
3
-1.421773
.861895
.420 1 0 0
-.399842
.419864
4
.962252
-1.387447
-.826743
.714841
. 714929
*** C H A R A C T E R I S T I C S O F D E S I G N E D F I L T E R ***
BAND 1
BAND 2
LOWER BAND EDGE
.30000
.00000
1.00000
UPPER BAND EDGE
.25000
NOMINAL GAIN
1.00000
.00000
NOMINAL RIPPLE
.00100
.05600
MAXIMUM RIPPLE
.00071
. 04910
R I P P L E I N DB
.41634
-63.00399

filter in to another low pass filter having a band ed ge frequency Sl'p =

perform ing the transform ation
s S
s(lu S2/)

(low pass to bandpass)

1 and then

(8.4.3)

E quivalently, w e can accom plish the sam e result in a single step by m ean s o f the
transform ation
s 2 + 2/ S2
s y Qp ------
j( 2u 2i)

(low pass to bandpass)

(8.4.4)

where
2/ = low er band ed ge frequency
S2U = upper band ed ge frequency
Thus w e obtain
(

s 2 + 2i 2B \

Finally, if w e w ish to convert a low pass analog filter w ith band ed g e frequency
2p in to a bandstop filter, th e transform ation is sim ply th e in verse o f (8.4.3) with
the additional factor Slp serving to norm alize for th e band ed ge freq u en cy o f the
low pass filter. T h u s th e transform ation is

j lpS-2 - ^ -

(lowpass to bandstop)

(8.4.5)

Sec. 8.4

Frequency Transformations

Figure 8.47

697

Frequency response characteristics o f a 7-order elliptic filter.

which leads to

T h e m appings in (8.4.1), (8.4.2), (8.4.3), and (8.4.5) are sum m arized in

T able 8.12. T h e m appings in (8.4.4) and (8.4.5) are non linear and m ay appear
to distort the frequency response characteristics o f the low p ass filter. H ow ever,
the effects o f the nonlinearity on the frequency resp onse are m inor, prim arily af
fecting the frequency scale but preserving the am plitude resp onse characteristics
o f the filter. Thus an equiripple low p ass filter is transform ed in to an equiripple
b andpass or b andstop or highpass filter.

698

Design of Digital Filters

Chap. 8

TABLE 8.12

FREQUENCY TRANSFORMATIONS FOR

ANALOG FILTERS (PROTOTYPE LOWPASS FILTER
HAS BAND EDGE FREQUENCY ftp)

Type of
transformation

Transformation

Band edge
frequencies o f
new filter

np

Lowpass

s *

ftp

^ ^

Highpass

Bandpass

_ s2 + Sit
S --- iin -------------
p s ( n u - n ()

ft/, ftp

Bandstop

s ---

Q t, Clu

----------
p s2 + n u n .

Example 8.4.1
Transform the single-pole lowpass Butterworth filter with system function
H ^s) = - T 7 T

into a bandpass filter with upper and lower band edge frequencies 2 and
tively.

respec

Solution The desired transformation is given by (8.3.4). Thus we have

H (s) =

1
-f-

i(n - a,) +
(K K/)j
s 2 + (2 S2j)s +

The resulting filter has a zero at s = 0 and poles at

8.4.2 Frequency Transformations in the Digital Domain

A s in the analog dom ain, frequency transform ations can be perform ed on a digital
low pass filter to convert it to either a bandpass, bandstop, or highpass filter. T h e
transform ation in volves replacing th e variable z -1 by a rational function g (z -1 ),
w hich must satisfy the follow in g properties.
L T he m apping z _1 g (z -1 ) m ust m ap p oin ts inside th e unit circle in the
z-plane in to itself.
2. The unit circle m ust also b e m apped into itself.

Sec. 8.4

699

Frequency Transformations

C ondition (2) im plies that for r = 1,

e~Ja -

g ( e - J,) = g(u})

It is clear that w e m ust have |g(ti>)| = 1 for all

all-pass. H en ce it is o f the form
Z

oj .

That is, the m apping m ust b e

-ak

(8.4.6)

S ( z - ) = n f r akz-

w h ere \ak \ < 1 to ensure that a stable filter is transform ed into another stable filter
(i.e., to satisfy co n d ition 1 ).
From the general form in (8.4.6), w e obtain the desired set o f digital trans
form ations for converting a prototyp e digital low pass filter in to either a bandpass,
a bandstop, a highpass, or an oth er low pass digital filter. T h ese transform ations
are tabulated in T able 8.13.
TABLE 8.13 FREQUENCY TRANSFORMATION FOR DIGITAL FILTERS
(PROTOTYPE LOWPASS FILTER HAS BAND EDGE FREQUENCY a>P)
Type of
transformation

Transformation

Parameters
(o'p band edge frequency
of new filter

Lowpass

sin[(^p - w'p ) p \

1 - az'

sin[(a>p + w'p)p.]
w'p = band edge frequency
new filter

Highpass

cos[(top + to'p ) /2]

1 + az~x

cos[(a>p - a>p)/2]
wi = lower band edge frequency
toy = upper band edge frequency

Bandpass

z 1 aiz + a ;
a2z~2 - a\z~l + 1

0l = 2aK/ ( K + 1)
a2 = ( K - 1 )/(K + 1)
COs[ (tD, + Ci>;)/2]
c o s K c D j, c d / ) / 2 ]

K as COt

0>1

tan -r-

o>i = lower band edge frequency

co = upper band edge frequency

Bandstop

a{ = - 2a/(K + 1)
02 = {I - K) / ( 1 + K)
ai z- 1 + 1

cos[(i>u + <u;)/2]

cos[(a)a <u/)/2]
K tan

a>M a>i

CL

700

Design of Digital Filters

Chap. 8

Example 8.42
Convert the single-pole lowpass Butterworth filter with system function
0 .2 4 5 ( 1 + Z ' 1)

H(z)

1 - 0 .5 0 9 Z -1

into a bandpass filter with upper and lower cutoff frequencies a>u and a>i, respectively.
The lowpass filter has 3-dB bandwidth cop = 0.2jt (see Example 8.3.5).

Solution The desired transformation is

Z

_i

z~2 - a i z ~ l + a 2
a 2z ~2 - ai z ' 1 + 1

---- ----------z------------;----- -

where a\ and a 2 are defined in Table 8.13. Substitution into H(z) yields
0.245

z~l ~ aiz~' + a2
a2z ~2 - fliZ-1 + 1

H(z) =

1 + o.509

\ a 2z~2 -

+1/

^ ________0.245(1 - a2)( 1 - r 2)________

(1 + 0.509o2) - 1.509fliz-1 + (a2 + 0.509)z~2
Note that the resulting filter has zeros at z = 1 and a pair of poles that depend on
the choice of o>u and a>t.
For example, suppose that o>u = 3jt/5 and a>i = 2tt/5. Since wn = 0.2?r, we find
that K = 1, a 2 = 0, and ay = 0. Then
=

-245(1 ~ Z~2)

()
l+0.509z~2
This filter has poles at z = y'0.713 and hence resonates at <d = np..
Since a frequency transform ation can be perform ed eith er in the analog d o
m ain or in the digital dom ain, the filter design er has a ch o ice as to w hich ap
proach to take. H o w ever, som e caution m ust b e exercised d ep en d in g on the
types o f filters b ein g d esign ed . In particuiar, w e know that the im pulse invari
an ce m ethod and th e m apping o f d erivatives are inappropriate to use in designing
highpass and m any bandpass filters, due to the aliasing p rob lem . C on sequ en tly,
o n e w ould n ot em p loy an analog frequency transform ation follow ed by con ver
sio n o f the result into the digital d om ain by use o f these tw o m appings. Instead,
it is m uch b etter to perform the m apping from an analog low pass filter in to a
digital low pass filter by eith er o f th ese m appings, and th en to perform the fre
q uency transform ation in the digital dom ain. Thus the p rob lem o f aliasing is
avoided.
In the case o f the b ilinear transform ation, w here aliasing is not a problem , it
d o es n ot m atter w h eth er the frequency transform ation is p erform ed in the analog
dom ain or in th e digital dom ain. In fact, in this case on ly, th e tw o approaches
result in identical d igital filters.

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

701

8.5 DESIGN OF DIGITAL FILTERS BASED ON LEAST-SQUARES

METHOD
E xcep t for th e im pulse invariance m eth od , the design tech n iqu es for IIR filters d e
scribed in S ection 8.3 in volved the conversion o f an analog filter in to a digital filter
by so m e m apping from the ,y-plane to the z-plane. A s an alternative, on e can design
digital IIR filters directly in the z-dom ain w ithou t referen ce to th e analog dom ain.
W e n ow describe several m eth od s for designing digital filters directly. In th e
first three tech n iqu es, the P ade approxim ation m eth o d and least-squares design
m eth od s, the specifications are given in the tim e d om ain and th e design is carried
out in th e tim e dom ain. The final section describes a least-squares technique in
w hich the design is carried out in the frequency dom ain.

8.5.1 Pade Approximation Method

Suppose that the desired im pulse resp onse hj { n) is sp ecified for n > 0. T he filter
to b e d esign ed has the system function
M

Y bkZ k
H(z) =

(8.5.1)
where h(k) is its unit sam ple response. T he filter has L = M + N + 1 param eters,
nam ely, th e coefficients [ak} and {&*}, w hich can b e selected to m inim ize so m e
error criterion.
T h e least-sq u ares error criterion is often u sed in optim ization problem s o f
this type. Su p pose that w e m inim ize the sum o f the squared errors
u
(8.5.2)
with respect to th e filter p aram eters {a*} and {>*}, w h ere U is som e p reselected
upper lim it in the sum m ation.
In g en eral, h(n) is a non linear function o f the filter param eters and h en ce th e
m inim ization o f in v o lves th e solu tion o f a set o f n on linear equ ation s. H ow ever,
if w e select th e upper lim it as U = L 1, it is p ossib le to m atch h(n) p erfectly
to the desired resp onse hd (n) for 0 < n < Af + N. T h is can b e achieved in th e
follow in g m anner.
T h e differen ce eq u ation for th e d esired filter is
y (n ) = - a i y ( n -

1 ) - a 2y( n - 2) ---------- a Ny{n - N )

+ box(n) + b\ x( n - 1) H-------- 1- b Mx( n - M)

(8.5.3)

702

Design of Digital Filters

Chap. 8

Suppose that the input to the filter is a unit sam ple [i.e., x( n) S(n)]. T hen the
resp onse o f the filter is y(/i) = h(n) and h en ce (8.5.3) b eco m es
h(n) = a\ h{n 1) ai hi n 2) -------a s h { n N)
-)- boS(n) -I- b\S(n 1) H--------1- b MS(n Af)

(8.5.4)

Since S(n Jt) = 0 excep t for n = k, (8.5.4) red u ces to

h(n) = a \ h ( n ~ 1) aj h(n 2 ) ----------a \ h ( n N ) + bn

0 < n < M

(8.5.5)

For n > Af, (8.5.4) b eco m es

h(n) a\ h(n 1 ) ai h(n 2 )

a Nh(n - N)

(8.5.6)

T he set o f linear eq u ation s in (8.5.5) and (8.5.6) can b e used to solve for the
filter param eters {a*} and {b*}. W e set h(n) = hd {n) for 0 < n < Af -I- N, and
use the linear eq u ation s in (8.5.6) to solve for the filter param eters {a*}. T hen
w e use values for the {a*} in (8.5.5) and solve for the param eters {>*}. Thus we
obtain a perfect m atch b etw een h(n) and the desired resp onse hj ( n) for the first
L values o f th e im pulse response. T his design tech n iqu e is u su ally called the Pade
approxi mat i on procedure.
T h e d egree to which this design tech n iqu e produces accep tab le filter designs
d ep en d s in part on the num ber o f filter coefficients se lecte d . Since the design
m eth od m atches /i</(n) o n ly up to the num ber o f filter param eters, the m ore com
p lex the filter, the b etter th e approxim ation to h j ( n ) for 0 < n < M + N . H ow ever,
this is also the m ajor lim itation with the P ade approxim ation m eth od , nam ely, the
resulting filter m ust contain a large num ber o f p o les and zeros. For this reason,
the P ade approxim ation m ethod has found lim ited use in filter designs for practical
applications.

Example 8.5.1
Suppose that the desired unit sample response is

hj(n) = 2(i)"w(n)
Determine the parameters of the filter with system function

using the Pad6 approximation technique.

Solution In this simple example, H(z) can provide a perfect match to Hjiz), by
selecting bo = 2, f>i = 0, and
we indeed obtain the same result.

Let us apply the Pad6 approximation to see if

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

703

With the substitution for hd(ri), we obtain ai = j. To solve for b^ and b\, we use
the form (8.5.5) with h(n) = hj(n). Thus
h<t(n) = ~ M - 1) + b08(n) + b xS(n - 1)

For n = 0 this equation yields b0 = 2. For n = 1 we obtain the result b\ = 0. Thus

H(z) HAz).
This exam ple illustrates that the P ade approxim ation results in a perfect
m atch to Hd(z) w hen the desired system function is rational and w e have prior
k n ow led ge o f the num ber o f p oles and zeros in the system . In general, how ever,
this is n ot the case in practice, since hd (n) is d eterm in ed from som e desired fre
quency resp onse specifications
a>). In such a case th e P ade approxim ation m ay
n ot result in a good filter design. T o illustrate a p oten tial p roblem and suggest a
solu tion , let us consider the follow in g exam ples.

Example 8.5.2
A fourth-order Butterworth filter has the system function
_
4.8334 x 10-3(; + l)4
A:} ~ (z2 - 1.3205- + 0.6326)(z2 - 1.0482z + 0.2959)
The unit sample response corresponding to /^(z) is illustrated in Fig. 8.48. Use the
Pade approximation method to approximate //rf(~).
Solution We observe that the desired filter has M = 4 zeros and jV = 4 poles. It is
instructive to determine the coefficients in the Pade approximation when the number
of zeros and/or poles are not identical to the desired number of filter parameters.

704

Design of Digital Filters

Chap. 8

Magnitude
response

4
Figure 8.49

Filter designs based on Pade approximation (Example 8.5.2).

In Fig. 8.49 we plot the frequency response of the filter obtained by the Padd
approximation method. We have considered four cases: M = 3, N = 5; M = 3, N = 4;
M = 4, N = 4; M = 4, N = 5. W e observe that when M = 3, the resulting frequency
response is a relatively poor approximation to the desired response. However, an
increase in the number of poles from A' = 4 to N = 5 appears to compensate in part
for the lack of the one zero. When M is increased from three to four, we obtain a
perfect match with the desired Butterworth filter not only for N = 4 but for N = 5,
and, in fact, for larger values of N.
Example 8.5.3

A three-pole and three-zero type II lowpass Chebyshev digital filter has the system
function
"

0.3060(1 + z)(0.2652 - 0.09z-' + 0.2652z~2)

(1 - OJSSOz-^d - 1.1318z-> + 0.5387z-2)

Its unit sample response is illustrated in Fig. 8.50. Use the Padf approximation
method to approximate Hj(z).
Solution By following the same procedure as in Example 8.5.2, we determined the
Pad6 approximation of H</(z) based on the selection of M = 2, N = 3; M = 2, N = 4;
M = 3, N = 3; M = 3, N = 4. The frequency responses of the resulting designs are
illustrated in Fig. 8.51.

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

Figure fL50 Impulse response h j i n) of type II Chebyshev digital filter given in

Example 8.5.3.

Magnitude

705

706

Design of Digital Filters

Chap. 8

As in Example 8.5.2, we note that when we underestimate the number of zeros

we obtain a relatively poor design, as evidenced by the two cases in which M = 2 .
On the other hand, if M = 3, we obtain a perfect match for N = 3 and N = 4.
T h ese tw o exam ples suggest that an effectiv e approach in using the Pade
approxim ation is to try different values o f M and N until the frequency resp onses o f
the resulting filters con verge to the d esired frequency resp onse w ithin som e sm all,
acceptable approxim ation error. H ow ever, in practice, this approach appears to
be cum bersom e.

8.5.2 Least-Squares Design Methods

A gain , let us assum e that hd {n) is specified for n > 0. W e begin with the sim ple
case in w hich th e digital filter to b e d esign ed contains only p oles, that is,
H(z) =

(8.5.7)

N ow , con sid er the cascade con n ection o f the desired filter Hd (z) w ith the reciprocal,
all-zero filter 1 / / / (z), as illustrated in Fig. 8.52. N ow suppose that the cascade
configuration in Fig. 8.52 is excited by the unit sam ple seq u en ce S(n). Thus the
input to the inverse system 1/ H { z ) is hd (n) and the output is v(n). Ideally, y d (n) =
S(n). T he actual output is
(8.5.8)
T h e condition that >v(0) = _y(0) 1 is satisfied by selectin g bo h d {0). For
n > 0, y( n) represents the error b etw een th e desired output yd {n) 0 and the
actual output. H en ce the param eters {a*} are selecte d to m inim ize the sum of
yin)

M )
H d ( z)

/~ns

H(z)
T

error

Minimize
the sum of
squared errors

Figure 8J?2

Least-squares inverse filter design m ethod.

S(n)

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

707

squares o f th e error sequence,

= X > 2()
n=l

Z.
n=l

;v
hd ( n ) + Y akhdin - k)
k=\

(8.5.9)

h 2A 0 )
B y differen tiatin g with respect to the param eters {a*}, it is easily estab lish ed
that w e ob tain the set o f linear equations o f the form
N

1 , 2, . . . , W

(8.5.10)

= m * ) M + * - n = rhhik - l )
n=0

(8.5.11)

J 2 w hh( k, l ) = - r hh( l , 0 )

/ =

k-\

w h ere, by definition,
OO

rkh(k,l) =
rt=l

00

T h e solu tion o f (8.5.10) yields th e desired param eters for th e inverse system
1 / H ( z ) . Thus w e obtain the coefficients o f the all-p ole filter.
In a practical design problem , the desired im pu lse resp onse hd (n) is specified
for a finite set o f points, say 0 < n < L, w h ere L > > N. In such a case, the
correlation sequ en ce rdd(k) can be com p uted from the finite seq u en ce hd (n) as
L \kl\

?hh (k I)

hd(n)hd(n + k - I)

0 < k - l < N

(8.5.12)

and these valu es can b e used to solve the set o f linear eq u ation s in (8.5.10).
T he least-squares m eth od can also b e used in a p o le -z e r o approxim ation for
Hd (z). If the filter H( z ) that approxim ates Hd (z) has b oth p oles and zeros, its
resp onse to the unit im pulse <5(n) is
S

^ ) = - J f l i A(fi t ) + ^ M ( - i t )
* -i
*=o

n > 0

(8.5.13)

or, eq u ivalen tly,

h(n) - Y a*h(n - V + b *
**i

0 < n < M

(8.5.14)

n> M

(8.5.15)

F or n > M , (8.5.13) red u ces to

N

h(n) = ^ a * h ( n k)

708

Design of Digital Filters

Chap. 8

Clearly, if H j i z ) is a p o le-zero filter, its resp onse to Bin) w ou ld satisfy the sam e
eq u ation s (8.5.13) through (8.5.15). In gen eral, h ow ever, it d o es not. N everth eless,
w e can use the desired response h j ( n) for n > M to construct an estim ate o f hd(n),
according to (8.5.15). That is,

hd(n) -

- *)

(8.5.16)

T hen w e can select the filter param eters {a*} to m inim ize the sum o f squared errors
b etw een the desired response hd(n) and the estim ate hd (n) for n > M. Thus we
have
OO
n=M+1

ih d^

~ hd(n )]2

(8.5.17)
The m inim ization o f \, with respect to the p o le param eters {a*}, leads to the set
o f linear equations
Y

a irhh(k,l) = ~ r hh( k , 0 )

k =

1 , 2, . . . , N

(8.5.18)

where rhh( k, l ) is n ow defined as

r>, h( k, l )=

Y
hd (n - k) hd (n - I)
n~M+1

(8.5.19)

Thus these linear eq u ation s yield the filter p aram eters {a*}. N o te that these eq u a
tions reduce to the all-pole filter approxim ation w h en M is set to zero.
T h e param eters {bk} that determ in e the zeros o f the filter can be obtained
sim ply from (8.5.14), w here h(n) = hd (rt), by sub stitu tion o f th e values [ak] ob
tained by solving (8.5.18). Thus
bn = hd(n) + ^

akhd (n k)

0<n<M

(8.5.20)

T h erefore, the param eters {a*} that d eterm in e the p o les are obtained by the
m ethod o f least squares w hile the p aram eters {bk} that d eterm in e the zeros are
ob tain ed by the the P ade approxim ation m eth od . T he foregoin g approach for
determ ining the p o les and zeros o f H ( z ) is so m etim es called P r o n y s m e t h o d .
T h e least-squares m eth od provid es go o d estim ates for th e p o le param eters
{ak}. H ow ever, P ron ys m eth od m ay n ot b e as effective in estim ating the pa
ram eters {>*}, prim arily b ecau se the com p utation in (8.5.20) is not based on the
least-squares m ethod.

Sec. 8.5

Design of Digital Fitters Based on Least-Squares Method

(n)

all-pole
filter

all-zero
filter

W ,( z )

Figure 8^ 3

709

M")

H 2(z)

Least-squares method for determining the poles and zeros of a filter.

A n alternative m eth od in which both sets o f param eters {fl*) and {*} are d e
term ined by application o f the least-squares m eth od has b een prop osed by Shanks
(1967). In Shanks m eth od , the param eters {a*) are com p uted on the basis o f the
least-squares criterion, according to (8.5.18), as indicated ab ove. This yields the
estim ates {a*}, which allow us to syn th esize the all-pole filter.
Hi (z) = ------- rr---------

(8-5.21)

i+

akz k

T h e resp onse o f this filter to the im pulse <5(n) is

u(rc) = ^

fl;v(n k) + S(n)

n > 0

(8.5.22)

If the seq u en ce (u(n)} is used to excite an all-zero filter with system function
(8.5.23)

H 2( z ) =

as illustrated in Fig. 8.53, its resp onse is

(8.5.24)

hd(n) = ^ & * u ( n - k)
N o w w e can define an error seq u en ce e ( n ) as
e{n) = hd {n) - hd{n)
= hd (n) - Y

(8.5.25)

b*v (n ~ k)

and, con sequ en tly, the param eters {&*} can also be determ in ed by m eans o f the
least-squares criterion, nam ely, from the m inim ization of
-l 2
M ) - J ^ M ( ~k)

(8.5.26)

Thus w e obtain a set o f linear eq u ation s for the param eters {bk}, in the form
M
Y bt rvv(k,l) = r * (0

/ = 0 ,1 ........ Af

(8.5.27)

710

Design of Digital Filters

Chap. 8

w here, by definition,
OO
r vv(k, I) = Y v (n ~ k ^ n - *>
r*=0
oo
rhvik) =
hd (n)v(n - Jt)
n=0

(8.5.28)

(8.5.29)

Example 8.5.4
Approximate the fourth-order Butterworth filter given in Example 8.5.2 by means of
an all-pole filter using the least-squares inverse design method.
Solution From the desired impulse response hd(n), which is illustrated in Fig. 8.48,
we computed the autocorrelation sequence rkh(kj) = rhh(k - /) and solved to set
of linear equations in (8.5.10) to obtain the filter coefficients. The results of this
computation are given in Table 8.14 for N = 3, 4, 5, 10, and 15. In Table 8.15 we list
the poles of the filter designs for N = 3, 4, and 5 along with the actual poles of the
fourth-order Butterworth filter. W e note that the poles obtained from the designs
are far from the actual poles of the desired filter.
ESTIMATES OF FILTER COEFFICIENTS
IN LEAST-SQUARES INVERSE FILTER DESIGN
METHOD

TABLE 8.14
{a k}

iV =
0) =

02 =
a-s =
N =
a{ =
a2 =
a3 =
a4 =
JV =
fl! =
a2 =
a2 =
aj, =
a5 =

3
0.254295E + 01
0.241800E + 01
0.853829E + 00
4
0.319047 + 01
0.425176 + 01
0.278234 + 01
0.758375E + 00
5
0.368733E + 01
0.607422 + 01
0.556726E + 01
-0.284813 + 01
0.654996E + 00

N = 10
ai =
5.008451
a2 - -12.660761
= 21.557365
04 = -27.804110
as = 28.683949
6 = -24.058558
a7 = 16.156847
flg ss -8.247148
ag =
2.854789
flio -0.502956

N 15
=
2.993620
-1.143053
a2 =
= -12.132861
39.663433
a4 ~
ai = -75.749001
109.247757
06 =
a7 = -129.513794
a8 =
131.026794
a9 ~ -114.905266
87.449211
flio =
an = -57.031906
30.915134
<312 =
a 13 = -13.124536
3.879295
014 =
-0.597313
015 =
ai

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

711

TABLE 8.15 ESTIMATES OF POLE

POSITIONS IN LEAST-SQUARES
INVERSE FILTER DESIGN METHOD
(EXAMPLE 8.5.4)
Number of
poles
N =3
N = 4
N = 5

N =4
Butterworth
filter

Pole positions
0.9305
0.8062 yO.5172
0.8918 j'0.2601
0.7037 yO.6194
0.914
0.8321 y0.4307
0.5544 j'0.7134
0.6603 y0.4435
0.5241 yo.1457

The frequency responses of the filter designs are plotted in Fig. 8.54. W e note
that when N issmall, the approximation to the desired filterispoor. As N isincreased
to N = 10 and N = 15, the approximation improves significally. However, even for
N = 15, there are large ripples in the passband of the filter response. It is apparent
that this method, which isbased on an all-pole approximation, does not provide good
approximations to filters that contain zeros.
Example 8.5.5

Approximate the type II Chebyshev lowpass filter given in Example 8.5.3 by means
of the three least-squares methods described above.
Solution The results of the filter designs obtained by means of the least-squares
inverse method, Pronys method and Shanks method, are illustrated in Fig. 8.55.
The filter parameters obtained from these design methods are listed in Table 8.16.
The frequency response characteristics in Fig. 8.55 illustrate that the leastsquares inverse (all-pole) design method yields poor designs when the filter contains
zeros. On the other hand, both Pronys method and Shanksmethod yield very good
designs when the number of poles and zeros equals or exceeds the number of poles
and zeros in the actual filter. Thus the inclusion of zeros in the approximation has a
significant effect in the resulting filter design.

8.5.3 FIR Least-Squares Inverse (Wiener) Filters

In th e preced in g sectio n w e described the use o f the least-squares error criterion
in the d esign o f p o le -z e r o filters. In this section w e u se a sim ilar approach to
d eterm in e a least-squares F IR inverse filter to a d esired filter.
T h e inverse to a linear tim e-invariant system w ith im pulse resp onse h(n) and
system fun ction H(z) is defined as the system w h ose im pu lse resp onse hr(n) and

712

Chap. 8

Magnitude (dB )

Design of Digital Filters

Magnitude (dB)

(a)

Figure &54
method.

Magnitude responses for filter designs based on the least-squares inverse filter

system function H / ( z ), satisfy the resp ective equ ation s.

h ( n ) * h , ( n ) = 5(o)
H ( z ) H , (z) = 1

(8.5.30)
(8.5.31)

In gen eral, H, { z) is IIR , unless H { z ) is an all-p ole system , in w hich case H, { z) is

HR.

Magnitude (dB)
Magnitude (dB)
Magnitude (dB)

F ifu n &55 Filter designs based on least-squares methods (Example 8-5.5):

(a) least-squares design; (b) Pronys method; (c) Shanks m ethod.

714

Design of Digital Filters

Chap. 8

TABLE 8.16 POLE-ZERO LOCATIONS FOR FILTER DESIGNS IN EXAMPLE 8.5.5

Chebyshev Filter:
Zeros: -1 .0.1738311 ;0.9847755
Poles: 0.3880,0.5659 y'0.467394
Filter
Order

Poles in
Least-Squares Inverse

N = 3

0.8522
0.6544 j0.6224
N = 4
0.7959 y0.3248
____________________ ___________________0.4726 j'0.7142___________________
Filter
Order

Pronys Method
Shanks M ethod
________________ 1__________________________________________________
Poles
Zeros
Poles
Zeros

N = 3
M = 2

0.5332
0.6659 j 0.4322

N = 4
M = 2

0.7092
-0 .2 9 1 9
0.6793 y0.4863

N - 3
M = 3

0.3881
0.5659 J0.4671

N = 4

-0 .00014
0.388
0.5661 y0.4672

M = 3

-0 .1497 j0.4925

0.5348
0.6646 j0.4306

-0 .2 4 3 7 y0.5918

-0 .1 9 8 2 ;0.37

0.7116
-0.2921
0.6783 _/0.4855

0.306 y0.4482

0.3881
0.5659 _/0.4671

0.1738 y'0.9848

-1
0.1736 /0.9847

-1
0.1738 ;0.9848

-0.00014
0.388
0.566 ;0.4671

-1
-1
0.1738 y0.9848

In m any practical applications, it is d esirable to restrict th e inverse filter to

b e F IR . O bviously, o n e sim ple m eth od is to truncate
In so doing, w e incur
a total squared approxim ation error equal to
CO

,=

Y
* / ( ")
fiAf+1

(8.5.32)

where Af + 1 is the length o f the truncated filter and E, rep resen ts the energy in
the tail o f the im pulse resp onse /i/(n ).
A ltern atively, w e can use the least-squares error criterion to op tim ize the
Af + 1 coefficients o f the F IR filter. First, let d( n) d en o te the desi red out put
sequence o f the F IR filter o f length Af -I-1 and let h(n) be th e input sequence.
T hen, if y( n) is the output seq u en ce o f the filter, as illustrated in Fig. 8.56, the
error seq u en ce b etw een the desired output and the actual ou tp u t is
M
e ( n) = d( n) ^ * / i ( n k)
k=0

where the {*} are the FIR filter coefficients.

(8.5.33)

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

715

d(n) = S(n)

m -

FIR
filter

v(")

(M
~ ~ r~

e(n)

Minimize
the sum of
squared errors

Figure 8.56

Least-squares FIR inverse filter.

T h e sum o f squares o f the error seq u en ce is

2

OC

d{n) - ^ b k h i r i - k)

n=0 l

(8.5.34)

i -0

W hen is m inim ized with respect to the filter coefficients, w e obtain the set o f
linear eq u ation s
M
Y , b k r hh{ k - l ) = rdh{l)
*=o

I = 0, 1.........M

(8.5.35)

w here rhh(l) is the au tocorrelation o f h(n), defined as

00
rhhd) = Y ^ h { n ) h { n - I)

(8.5.36)

=0
and rdh{n) is the crosscorrelation b etw een the desired output d(n) and the input
seq u en ce h(rt), defined as
OO
rdh{l) = Y , d ( n ) h { n - l )
(8.5.37)
=o
T h e optim um , in the least-squares sen se, F IR filter that satisfies the linear
eq u ation s in (8.5.35) is called the Wiener filter, after th e fam ou s m athem atician
N orbert W ien er, w h o introduced optim um least-squares filtering m eth od s in en g i
n eerin g [see b o o k by W iener (1949)].
If the optim um least-squares F IR filter is to b e an approxim ate inverse filter,
the desired resp onse is
d( n) = S(n)

(8.5.38)

T h e crosscorrelation b etw een d(n) and h(n) red u ces to

^ ( / ) = { J (0)

10

/ = .
otherw ise

(8.5.39)

Design of Digital Filters

716

Chap. 8

T h erefore, the coefficients o f the least-squares F IR filter are ob tain ed from the
solution o f the linear eq u ation s in (8.5.35), w hich can b e exp ressed in m atrix form
as
~ r hh (0 )

^ a(I)
. r hh{ M )

rhh (2 )
rhh(l)

rhhi 1 )
rhh(0)

r hh{ M ~ l )

r hfl( M - l )

" bo ~
b\

rhhi 0 )

-bu -

rhh(M)

'H O )!

(8.5.40)

W e observe that the matrix is n ot only sym m etric but it also has the special
p roperty that all the elem en ts along any d iagon al are equal. Such a matrix is
called a T oeplitz matrix and lends itself to efficient inversion by m eans o f an
algorithm due to L evin son (1947) and D urbin (1959), which requires a num ber o f
com putations p roportional to M 2 instead o f the usual M 3. T h e L evin son -D u rb in
algorithm is described in Chapter 11.
T h e m inim um value o f the least-squares error ob tain ed w ith th e optim um
F IR filter is
d ( n ) - ^ b k h i n - k)
o

= L

d{n)

M
^ ^ ( n ) - ^ 2 b krdh(k)

(8.5.41)

In the case w here the F IR filter is the least-squares inverse filter, d ( n ) = S(n) and
rdh(n) h(0)8(n). T herefore,
Erin = 1 - h( 0)bo

(8.5.42)

Example 8.5.6
Determine the least-squares FIR inverse filter of length 2 to the system with impulse
response
h(n) -

1,
a,
. 0,

n=0
n = 1
otherwise

where |a| < 1. Compare the least-squares solution with the approximate inverse
obtained by truncating hj(n).
Solution Since the system has a system function H(z) = 1 a z ~ \ the exact inverse
is IIR and is given by
H,{z) =

1 - az"

or, equivalently,
hi(n) = a"u(n)

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

717

If this is truncated after n terms, the residual energy in the tail is

, =

a 2^ ! + a 2 + a 4 H------- )

From (8.5.40) the least-squares FIR filter of length 2 satisfies the equations

which have the solution

1 + a 2 4- a 4

For purposes of comparison, the truncated inverse filter of length 2 has the
coefficients bo = 1 , b\ = a.
The least-squares error is
^

~ 1 +

which compares with

for the truncated approximate inverse. Clearly, , > , so that the least-squares
FIR inverse filter is superior.
In this exam p le, th e im pulse resp onse h( n ) o f the system w as m inim um phase.
In such a case w e selected the d esired resp onse to b e J (0) = 1 and d (n) = 0, n > 1.
O n the other hand, if the system is nonm inim um phase, a d elay should b e inserted
in th e desired resp onse in order to obtain a g o o d filter design. T he value o f the ap
propriate d elay d ep en d s on the characteristics o f h(n). In an y case w e can com pute
the least-squares error filter for d ifferen t d elays and se lect the filter that produces
the sm allest error. T h e follow in g exam p le illustrates the effect o f the delay.

Example 8-5.7
Determine the least-squares FIR inverse of length 2 to the system with impulse response
a,
h(n) =

1,
0,

where |a| < 1.

n= 0
n= 1
otherwise

718

Design of Digital Filters

Chap. 8

Solution

T h is is a m axim um -phase system . If we select d(n) = [ 1 0 ] w e obtain the

sam e solu tion as in E x am p le 8.5.6, with a m inim um least-squ ares erro r

E m = 1 - h( 0 )bo

= 1 + a

l+ a 2
1 + a2 + a4

If 0 < a < 1 , then

> 1, which rep resen ts a p o o r inverse filter. If 1 < a < 0, then
mjn < 1. In p articu lar, fo r a
we obtain min = 1.57. F o r a = - 5 . m = 0 .8 1 ,
which is still a very large value fo r th e squared erro r.
N ow suppose that the desired response is specified as d(n) = & ( n 1). T h e n
th e set o f eq u atio ns fo r the filter coefficien ts, o btain ed from (8 .5 .3 5 ), are th e solution
to the eq uations
fl + a2
L a

-a

-I p o ] _ [ M D ] _ r 1 1

l + a 2 J Lf c 1J

L / i(0)J

L -o fJ

T h e solu tion o f th ese eq uations is

a 2 +

O'3
1 + a 2 + or4
T h e least-sq u ares erro r, given by (8.5.41), is

mm = 1 - b(,rdh(0) - birdh(l)
= 1 - M ( 1 ) - M ( 0)
^min 1

1
^
. "t"
1 + a2 + a4
I + a2 + a4
1 - a4
1 + a2 + a4

In p articu lar, suppose th at a = j . T h en min = 0.29. C o n sequ en tly, the desired

response d(n) = S(n - 1) results in a significantly b e tte r in v erse filter. F u rth e r im
provem en t is possib le by increasing th e length o f th e inverse filter.

In gen eral, w hen the desired resp onse is specified to con tain a delay D , then
the crosscorrelation rd/,(l), defined in (8.5.37), b ecom es
r dh(l) = h ( D - I)

l =

T h e set o f linear eq u ation s for the coefficients o f the least-squares F IR inverse

filter given by (8.5.35) reduce to
M
] T V a / . ( * - 0 = h ( D - 1)
*=o

1 = 0 , 1 .........M

(8.5.43)

T h en the exp ression for the corresponding least-squares error, given in general by

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

719

(8.5.41), b eco m es
u
ftr in s l - J ^ b M D - k )
*=o

(8.5.44)

L east-squares F IR in verse filters are often used in m any practical applications for
d eco n v o lu tio n , including com m unications and seism ic signal p rocessing.

8.5.4 Design of HR Filters in the Frequency Domain

T h e IIR filter design m eth od s described in S ection s 8.5.1 through 8.5.3 are carried
out in the tim e dom ain. T h ere are also direct design tech n iqu es for IIR filters
that can b e p erform ed in the frequency dom ain. In this section w e d escribe a
filter param eter optim ization tech n iqu e carried ou t in the frequency d om ain that
is rep resen tative o f frequency-dom ain design m ethods.
T h e design is m ost easily carried out with the system function for the IIR
filter exp ressed in the cascade form as

+ a k2Z~2

1+

(8.5.45)

w here the filter gain G and the filter coefficients {a*i], {**2 ], {#ti}, {Pki} are to b e
determ in ed . T h e frequency response o f the filter can be exp ressed as
H{to) = GA(w)e>B(a,)

(8.5.46)

w here
A((D) - ]""[

1 + Pk\Z~l + f$k2Z~2

(8.5.47)

1 + a * U - 1 + a * 2 Z 2

and (a>) is the phase response.

In stead o f d ealin g with the phase o f the filter, it is m ore co n ven ien t to deal
w ith the en v elo p e delay as a function o f frequency, w hich is
</(*)

(8.5.48)

or, eq u ivalen tly,

Zg (u>) =

Tg ( z ) I z > -

r d Q (z )~[
L dz

dz

J WJ db)

(8.5.49)

It can b e show n that r?(z) can b e exp ressed as

zg (z) = R e

K r Atiz + 2/5*2

a*iz + 2a*2

(8.5.50)

i l[ L

w here R e ( u ) d en o tes the real part o f the com p lex-valu ed quantity u.

N o w su p p ose that the d esired m agnitude and d elay characteristics A(a>) and
Tg(co) are specified at arbitrarily ch osen discrete freq u en cies coi,
in

720

Design of Digital Filters

Chap. 8

the range 0 < M < n . T hen the error in m agnitude at the frequency cok is
GA(cok) A d (cok) w here A d (cok) is the desired m agnitude resp onse at cok. Sim i
larly, the error in delay at a>k can b e defined as zg(a>k) zd (a>k), w here zd (cok) is
the desired d elay response. H ow ever, the ch oice o f zd (a>k) is com plicated by the
difficulty in assigning a nom inal delay to th e filter. H en ce, w e are led to define the
error in d elay as zg(cok) rg(wo) zd (a>k), w h ere rg(wo) is the filter d elay at som e
nom inal center frequency in the passband o f the filter and zd (wk) is the desired
delay response o f the filter relative to r?(o^). B y defining th e error in delay in
this m anner, w e are w illing to accept a filter having w h atever nom inal delay zg (a>o)
results from the op tim ization procedure.
A s a perform ance index for determ ining th e filter param eters, o n e can ch oose
any arbitrary function o f the errors in m agnitude and d elay. T o be specific, let
us select the total w eigh ted least-squares error o ver all freq u en cies &i, a>2 , . . . , <oL,
that is,
L
( p ,

G)

( 1

A )

wn[GA(co) - A d (con )

] 2

n 1

v[zg(a)) -

(o>o) - zd (con)]2

(8.5.51)

71= 1
w here p d en o tes the 4 K -dim ensional vector o f filter coefficients {a*i}, {<**2 }, (An),
and {/ta h and A , {u>}, and {t>} are w eigh ting factors se lecte d by the designer.
Thus the em p hasis on the errors affecting the design m ay b e placed entirely on the
m agnitude ( A = 0), or on the delay ( A = 1) or, perhaps, eq u ally w eigh ted betw een
m agnitude and d elay ( A = 1/2). Similarly, th e w eigh ting factors in frequency {u;}
and {u} determ in e the relative em phasis on the errors as a fun ction o f frequency.
T h e squared-error function E (p , G ) is a non linear fun ction o f (AK + 1) pa
ram eters. T h e gain G that m inim izes is easily d eterm in ed and given by the
relation
L

Y ^ u M v n) A d (con)
G = ^ L---------------------

(8.5.52)

^ i o nA 2 (a))
n=l

T h e optim um gain C can b e substituted in (8.5.51) to yield

L
(p , G) = (1 *) v)n[GA(a>) - A d (co)]2
n* 1
L

+ k

vK

> ~ zs ( t o n ) ] 2

(8.5.53)

n 1

D u e to th e nonlinear nature o f (p, G) , its m inim ization o v er the rem aining

4 K param eters is perform ed by an iterative n um erical op tim ization m eth od such

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

721

as the F letch er and P o w ell m ethod (1963). O n e begins the iterative process by
assum ing an initial set o f param eter values, say p (0). W ith the initial values su b sti
tuted in (8.5.51), we obtain the least-squares error ( p (0), G). If w e also evaluate
the partial derivatives d / d a k\, d / d a k2, 3/3/J*i, and b / b 0 k2 at the initial value
p (0), w e can u se this first derivative inform ation to change the initial values o f the
p aram eters in a direction that lead s toward the m inim um o f the function (p . G)
and thus to a new set o f param eters p (1).
R ep etitio n o f the ab ove step s results in an iterative algorithm which is d e
scribed m athem atically by the recursive eq u ation
p (m+l) _ p (m)

A (m)Q()g (m)

m _ 0, 1, 2, . . .

w here A (m) is a scalar representing the step size o f the iteration, Q (m) is a (4K x 4 K )
m atrix, w hich is an estim ate o f the H essian , and g (m) is a (4K x 1) vector consisting
o f the four AT-dimensional vectors o f gradient com p on en ts o f (i.e., d / d a \ n ,
d / d a k l, d / d p kU d / d p k2), evaluated at a kl = or" \ a k2 = a ^ \ /3k] =
fik2 =

and

Th*s iterative p rocess is term inated w hen the gradient com p on en ts

are nearly zero and the value o f the function (p , G) d o es not change appreciably
from o n e iteration to another.
T he stability constraint is easily incorporated into the com puter program
through the param eter vector p. W hen la ^ l > 1 for any k = 1 , . . . , K , the param
eter a k2 is forced back inside the unit circle and the iterative p rocess continued. A
sim ilar p rocess can be used to force zeros in sid e the unit circle if a m inim um -phase
filter is desired.
T h e m ajor difficulty with any iterative p roced u re that searches for the param
eter valu es that m inim ize a nonlinear function is that the process m ay con verge
to a local m inim um instead o f a global m inim um . Our on ly recourse around this
p rob lem is to start th e iterative process w ith d ifferent values for the param eters
and o b serve the end result.
Example 8.5.8
Let us design a lowpass filter using the Fletcher-Powell optimization procedure just
described. The filter is to have a bandwidth of 0.3n and a rejection band commencing
at 0.45jr. The delay distortion can be ignored by selecting the weighting factor X = 0.
Solution We have selected a two-stage (K = 2) or four-pole and four-zero filter
which we believe is adequate to meet the transition band and rejection requirements.
The magnitude response is specified at 19 equally spaced frequencies, which is con
sidered a sufficiently dense set of points to realize a good design. Finally, a set of
uniform weights is selected.
This filter has the response shown in Fig. 8.57. It has a remarkable resemblance
to the response of the elliptic lowpass filter shown in Fig. 8.58, which was designed
to have the same passband ripple and transition region as the computer-generated
filter. A small but noticeable difference between the elliptic filter and the computer
generated filter is the somewhat flatter delay response of the latter relative to the
former.

722

Design of Digital Filters

Frequency
(a)

(b)
Figure (L57 Filter designed by Fletcher-Powell optimization method (Exam
ple 8.5.8).
Example 8.5.9
Design an IIR filter with magnitude characteristics
smm,
Arf(a>) =
0,

and a constant envelope delay in the passband

<\(0 \ < 7 t

Chap. 8

Sec. 8.5

Design of Digital Filters Based on Least-Squares Method

723

Frequency
Figure &58 Amplitude and delay response for elliptic filter.
Solution The desired filter is called a modified duobinary filter and finds application
in high-speed digital communications modems. The frequency response was specified
at the frequencies illustrated in Fig. 8.59. The envelope delay was left unspecified
in the stopband and selected to be flat in the passband. Equal weighting coefficients
{ujn} and {t>} were selected. A weighting factor of A = 1/2 was selected.
A two-stage (four-pole, four-zero) filter is designed to meet the foregoing spec
ifications. The result of the design is illustrated in Fig. 8.60. We note that the
magnitude characteristic is reasonably well matched to sin a> in the passband, but the
stopband attenuation peaks at about - 2 5 dB, which is rather large. The envelope
delay characteristic is relatively flat in the passband.

724

Design of Digital Filters

n
2

Chap. 8

Figure 8^9 Frequency response of an ideal modified duobinary filter.

A four-stage (eight-pole, eight-zero) filter having the same frequency response
specifications was also designed. This design produced better results, especially in
the stopband where the attenuation peaked at 36 dB. The envelope delay was also
considerably flatter.

8.6 SUMMARY AND REFERENCES

W e have described in so m e detail the m ost im portant tech n iq u es for designing FIR
and IIR digital filters based on eith er frequency-dom ain specifications expressed
in term s o f a d esired frequency resp onse Hd (to), or in term s o f the desired im pulse
response h d (n).
A s a general rule, F IR filters are used in applications w h ere there is a need
for a linear-phase filter. T his requirem ent occurs in m any applications, especially in
telecom m u n ication s, w h ere there is a requirem ent to separate (dem u ltiplex) signals
such as data that have b een frequency-division m ultip lexed , w ithou t distorting
th ese signals in the process of dem ultiplexing. O f the several m eth od s described
for designing F IR filters, the frequency sam pling design m eth od and the optim um
C hebyshev approxim ation m ethod yield the b est designs.
IIR filters are generally used in applications w here so m e phase distortion
is tolerab le. O f the class o f IIR filters, ellip tic filters are th e m ost efficient to
im plem en t in th e sen se that for a given set o f specifications, an elliptic filter has a
low er order or few er coefficients than any other IIR filter type. W hen com pared
w ith F IR filters, elliptic filters are also considerably m ore efficient. In view o f
this, on e m ight con sid er the use o f an elliptic filter to ob tain th e desired frequency
selectivity, fo llo w ed then by an all-pass phase equ alizer th a t com p en sates for the
p h ase distortion in th e ellip tic filter. H ow ever, attem pts to accom plish this have
resulted in filters w ith a num ber o f coefficients in the cascad e com bination that

Sec. 8.6

Summary and References

725

Frequency (kHz)

4.000
3.500
3.000
| 2.500
'I
S' 2 000
2 1.500
u

tt

1.000

0.500
0.000

0.00

0.48

0.96
. 1.44
Frequency (kHz)

1.92

2.40

Figare &60 Frequency response of filter in Example 8.5.9. Designed by the

Fletcher-Powell optimization method.

eq u aled or ex ceed ed th e num ber o f coefficients in an eq u ivalen t linear-phase F IR

filter. C on sequ en tly, no reduction in com p lexity is ach ievab le in using phaseeq u alized ellip tic filters.
In addition to th e filter d esign m eth od s based on th e transform ation o f an alog
filters in to th e digital dom ain, w e also p resen ted several m eth od s in w hich th e
d esign is d o n e directly in the discrete-tim e dom ain. T h e least-squares m ethod is
particularly appropriate for designing IIR filters. T h e least-squares m ethod is also
u sed for th e design o f F IR W iener filters.

726

Design of Digital Filters

Chap. 8

Such a rich literature n ow exists on the design o f digital filters that it is not
p ossible to cite all the im portant referen ces. W e shall cite on iy a few . S om e of
the early w ork on digital filter design w as d on e by K aiser (1963, 1966), Steiglitz
(1965), G old en and K aiser (1964), R ad er and G old (1967a), Shanks (1967), H elm s
(1968), G ibbs (1969, 1970), and G old and R ader (1969).
T h e design o f analog filters is treated in the classic b ook s by Storer (1957),
G u illem in (1957), W einberg (1962), and D an iels (1974).
T h e frequency sam pling m ethod for filter design was first p rop osed by G old
and Jordan (1968, 1969), and op tim ized by R abiner et al. (1970). A d d ition al
results w ere published by H errm ann (1970), H errm ann and S chuessler (1970a),
and H ofstetter et al. (1971). T h e C h eb ysh ev (m inim ax) approxim ation m eth od for
d esigning linear-phase F IR filters w as p rop osed by Parks and M cC lellan (1972a,b)
and discussed further by R ab in er et al. (1975). T h e design o f ellip tic digital filters is
treated in the b ook by G old and R ad er (1969) and in the paper b y Gray and M arkel
(1976). T he latter includes a com puter program for designing digital ellip tic filters.
T h e use o f frequency transform ations in the digital d om ain w as prop osed by
C onstantinides (1 9 6 7 ,1 9 6 8 , 1970). T h ese transform ations are appropriate on ly for
IIR filters. T he reader should n ote that w hen these transform ations are applied
to a low pass F IR filter, the resulting filter is IIR.
D irect design techniques for digital filters have b een con sid ered in a num
ber o f papers, including Shanks (1967), Burras and Parks (1970), S teiglitz (1970),
D eczk y (1972), B rophy and Salazar (1973), and B andler and Bardakjian (1973).

PROBLEMS
8.1 Design an FIR linear phase, digital filter approximating the ideal frequency response
1,

for \a)\ <

0,

for < \w\ < n

HAco) =

(a) Determine the coefficients of a 25-tap filter based on the window method with a
rectangular window.
(b) Determine and plot the magnitude and phase response of the filter.
(c) Repeat parts (a) and (b) using the Hamming window.
(d) Repeat parts (a) and (b) using a Bartlett window.
8.2 Repeat Problem 8.1 for a bandstop filter having the ideal response

HAo>) =

for M <
6
t
n
i .
n
for | < M < j
for < \w\ < jt

8 J Redesign the filter of Problem 8.1 using the Hanning and Blackman windows.
8.4 Redesign the filter of Problem 8.2 using the Hanning and Blackman windows.

Chap. 8

727

Problems

8 ^ D eterm ine the unit sample response {h(n)} of a linear-phase F IR filter of length M = 4
for which the frequency response at u> = 0 and w = n/ 2 is specified as
HA 0) = 1

H,

--I

8.6 D eterm ine the coefficients {A(n)} of a linear-phase FIR filter of length M = 15 which
has a symmetric unit sample response and a frequency response that satisfies the
condition
/2 7 T * \
fl,
* = 0 ,1 .2 ,3
r V 15 j ~ I 0,
* = 4, 5, 6, 7
8.7 R epeat the filter design problem in Problem 8.6 with the frequency response specifi
cations
1
* = 0 .1 ,2 ,3
0.4
*= 4
, i f
0
* = 5 .6 ,7
8.8 The ideal analog differentiator is described by
dxa(r)
y. ( 0 =
where *(/) is the input and ya(t) the output signal.
(a) D eterm ine its frequency response by exciting the system with the input jc (/) =
e jlxFt

(b ) Sketch the magnitude and phase response of an ideal analog differentiator band-

limited to B hertz.
(c) The ideal digital differentiator is defined as
H(w) sc ja>

|w| < x

Justify this definition by comparing the frequency response \H(w)|, 4. H(<d) with
that in part (b).
(d) By computing the frequency response H(o>), show that the discrete-time system
y(n) = x ( n ) - x ( n - 1)
is a good approximation of a differentiator at low frequencies.
(e) Com pute the response of the system to the input
x(n) = A cos(a>on + 6)
8.9 Use the window m ethod with a Hamming window to design a 21-tap differentiator
as shown in Fig. P8.9. Com pute and plot the magnitude and phase response of the
resulting filter.
W,f()l

Figure P8.9

Design of Digital Filters

728

Chap. 8

8.10 Use the matched-z transform ation to convert the analog filter with system function
5 + 0 .1
H ( s ) ~ (s + 0.1)2 + 9
into a digital IIR filter. Select T = 0.1 and com pare the location of the zeros in H (z)
with the locations of the zeros obtained by applying the impulse invariance m ethod
in the conversion of H(s).
8.11 Convert the analog bandpass filter designed in Example 8.4.1 into a digital filter by
means of the bilinear transformation. Thereby derive the digital filter characteristic
obtained in Example 8.4.2 by the alternative approach and verify that the bilinear
transform ation applied to the analog filter results in the same digital bandpass fil
ter.
8.12 A n ideal analog integrator is described by the system function Ha(s) = 1fs. A digital
integrator with system function //(z) can obtained by use of the bilinear transform a
tion. T hat is,
7 1 + z-1
W(z) 2 1 - - i =
(a) W rite the difference equation for the digital integrator relating the input x ( n ) to
the output v(n).
(b) Roughly sketch the magnitude
and phase (ft) of the analog integra
tor.
(c) It is easily verified that the frequency response of the digital integrator is
H

(a>) =

. T cos(<u/2)
; =
2 sin(a>/2)

- J-

.T
u)
COt 2
2

- J

Roughly sketch |W(w)| and 6(w).

(d) Com pare the magnitude and phase characteristics obtained in parts (b) and (c).
How well does the digital integrator m atch the m agnitude and phase character
istics of the analog integrator?
(e) The digital integrator has a pole at z = 1. If you im plem ent this filter on a digital
com puter, what restrictions might you place on the input signal sequence x ( n ) to
avoid com putational difficulties?
8.13 A z-plane pole-zero plot for a certain digital filter is shown in Fig. P8.13. The filter
has unity gain at dc.
(a) D eterm ine the system function in the form
H( z)

(1 + aiz~1)(l + b\Z 1 + biz

(1 +CiZ_)(l +<fiz - 1 + d2z 3

giving numerical values for the parameters A, ai, b\, bj, c\, d\, and d2.
(b) Draw block diagrams showing numerical values for path gains in the following
forms:
(1) Direct form II (canonic form)
(2) Cascade form (make each section canonic, with real coefficients)
8.14 Consider the pole-zero plot shown in Fig. P8.14.
(a) D oes it represent an FIR filter?
(b) Is it a linear-phase system?

Chap. 8

729

Problems

Figure P8.13

Figure P8.14
(c) Give a direct form realization that exploits all symmetries to minimize the number
of multiplications. Show all path gains.
8.15* A digital low-pass filter is required to meet the following specifications:
Passband ripple: < 1 dB
Passband edge: 4 kHz
Stopband attenuation: > 40 dB
Stopband edge: 6 kHz
Sample rate: 24 kHz
The filter is to be designed by performing a bilinear transformation on an analog
system function. Determine what order Butterworth, Chebyshev, and elliptic analog
designs must be used to meet the specifications in the digital implementation.

Design of Digital Filters

730

Chap. 8

8.16* An IIR digital low-pass filter is required to meet the following specifications:
Passband ripple (or peak-to-peak ripple): < 0.5 dB
Passband edge: 1.2 kHz
Stopband attenuation: > 40 dB
Stopband edge: 2.0 kHz
Sample rate: 8.0 kHz
Use the design formulas in the book to determine the required filter order for
(a) A digital Butterworth filter
(b) A digital Chebyshev filter
(c) A digital elliptic filter
8.17* Determine the system function H(z) of the lowest-order Chebyshev digital filter that
meets the following specifications:
(a) 1-dB ripple in the passband 0 < \co\ < 0 .3 jt.
(b) At least 60 dB attentuation in the stopband 0.35jt < |w] < jr. Use the bilinear
transformation.
8.18* Determine the system function H(z) of the lowest-order Chebyshev digital filter that
meets the following specifications:
(a) |-d B ripple in the passband 0 < |a>) < 0.24 jt.
(b) At least 50-dB attenuation in the stopband 0 .3 5 jt < |u| < jr. Use the bilinear
transformation.
8.19* An analog signal x (r) consists of the sum of two components jri(r) and jr2 (r). The
spectral characteristics of x(t) are shown in the sketch in Fig. P8.19. The signal x{t) is
bandlimited to 40 kHz and it is sampled at a rate of 100 kHz to yield the sequence *(n).
It is desired to suppress the signal x 2(f) by passing the sequence *(n) through a
digital lowpass filter. The allowable amplitude distortion on |X i(/) | is 2% (Si = 0.02)
over the range 0 < |F | < 15 kHz. Above 20 kHz, the filter must have an attenuation
of at least 40 dB (52 = 0.01).
(a) Use the Remez exchange algorithm to design the minim um -order linear-phase
FIR filter that meets the specifications above. From the plot of the magni
tude characteristic of the filter frequency response, give the actual specifications
achieved by the filter.
(b) Compare the order M obtained in part (a) with the approximate formulas given
in equations (8.2.94) and (8.2.95).
(c) For the order M obtained in part (a), design an FIR digital lowpass filter using the
window technique and the Hamming window. Compare the frequency response
characteristics of this design with those obtained in part (a).
\X(F)\

Frequency in kilohertz

Figure P8.14

Chap. 8

Problems

731

(d) Design the minimum-order elliptic filter that meets the given amplitude specifica
tions. Compare the frequency response of the elliptic filter with that of the FIR
filter in part (a).
(e) Compare the complexity of implementing the FIR filter in part (a) versus the
elliptic filter obtained in part (d). Assume that the FIR filter is implemented in
the direct form and the elliptic filter is implemented as a cascade of two-pole
filters. Use storage requirements and the number of multiplications per output
point in the comparison of complexity.
8.20 The impulse response of an analog filter is shown in Fig. P8.20.

Figure P&20
(a) Let h(n) = h(nT), where T = 1, be the impulse response of a discrete-time filter.
Determine the system function H(z) and the frequency response H{w) for this
FIR filter.
(b) Sketch (roughly) \H{w) and compare this frequency response characteristic with
(c) The FIR filter with unit sample response h{n) given above is to be approximated
by a second-order IIR filter of the form
G(z) =

bgz

a2z '
Use the least-squares inverse design procedure to determine the values of the
coefficients &o, ai, and a2.
1 - ayz~

8.21 In this problem you will be comparing some of the characteristics of analog and digital
implementations of the single-pole low-pass analog system
HAs) = J <M O = e~t"
s -(- Of.
(a) What is the gain at dc? At what radian frequency is the analog frequency re
sponse 3 dB down from its dc value? At what frequency is the analog frequency
reponse zero? At what time has the analog impulse response decayed to 1/e of
its initial value?
(b) Give the digital system function H(z) for the impulse-invariant design for this
filter. What is the gain at dc? Give an expression for the 3-dB radian frequency.
A t what (real-valued) frequency is the response zero? How many samples are
there in the unit sample time-domain response before it has decayed to 1/e o f its
initial value?
(c) Prewarp the parameter a and perform the bilinear transformation to obtain
the digital system function H(z) from the analog design. What is the gain at dc?
At what (real-valued) frequency is the response zero? Give an expression for
the 3-dB radian frequency. How many samples in the unit sample time-domain
response before it has decayed to 1 /e of its initial value?

732

Design of Digital Filters

Chap. 8

JL22 We wish to design a FTR bandpass filter having a duration M = 201. Hj(co) represents
the ideal characteristic of the noncausal bandpass filter as shown in Fig. P8.22.

------------ ------------------- --------------- ------------

jt

O.Sjt

0.4 k

0.4 k

0.5 k

jt

Figure P&22
(a) Determine the unit sample (impulse) response hd(n) corresponding to Hj(a>),

(b) Explain how you would use the Hamming window

/ 2* \
u>(n) = 0.54 + 0.46 cos j ------ - )
\M -1 J

M -l
M -l
------ - < n < -
2
~ ~
2

to design a FIR bandpass filter having an impulse response h(n) for 0 < n <
200.
(c) Suppose that you were to design the FIR filter with M = 201 by using the fre
quency sampling technique in which the DFT coefficients H(k) are specified
instead of h(n). Give the values of H(k) for 0 < k < 200 corresponding to
Hd{eJU) and indicate how the frequency response of the actual filter will differ
from the ideal. Would the actual filter represent a good design? Explain your
answer.
6 2 3 We wish to design a digital bandpass filter from a second-order analog lowpass But
terworth filter prototype using the bilinear transformation. The specifications on the
digital filter are shown in Fig. P8.23(a). The cutoff frequencies (measured at the half
power points) for the digital filter should lie at w = 5jt/12 and co ln/12.
The analog protoype is given by
H(s) ----------- l- -----s2 + v 2 j + 1
with the half-power point at 2 = 1 .

(a) Determine the system function for the digital bandpass filter.
(b) Using the same specs on the digital filter as in part (a), determine which of the
analog bandpass prototype filters shown in Fig. P.8.23(b) could be transform ed
directly using the bilinear transformation to give the proper digital filter. Only
the plot of the magnitude squared of the frequency is given.
6 2 4 Figure P8.24 shows a digital filter designed using the frequency sampling method.
(a) Sketch a z-plane pole-zero plot for this filter.
(b) Is the filter lowpass, highpass, or bandpass?
(c) Determine the magnitude response \H(a>) at the frequencies wk ~ n k / 6 for k = 0,
1, 2, 3, 4, 5, 6 .

Chap. 8

Problems

733

ma>)\2
TT
6

t\
J

m m 2

Figure P&23

TT

7r

734

Design of Digital Filters

Chap. 8

(d) Use the results of part (c) to sketch the magnitude response for 0 < a> < n and
confirm your answer to part (b).
8.25 An analog signal of the form xa(t) = a(t) cos 2000^/ is bandlimited to the range
900 < F < 1100 Hz. It is used as an input to the system shown in Fig. P8.25.
(a) Determine and sketch the spectra for the signals x(n) and w(n).
(b) Use a Hamming window of length M = 31 to design a lowpass linear phase FIR
filter H(w) that passes (c(n)}.
(c) Determine the sampling rate of the A /D converter that would allow us to elim
inate the frequency conversion in Fig. P8.25.
8.26 System identification Consider an unknown LTI system and an FIR system model
as shown in Fig. P8.26. Both systems are excited by the same input sequence (x(n)}.
The problem is to determine the coefficients (A(n), 0 < n < M - 1} of the FIR model

Chap. 8

Problems

Rx

= ~ = 2500

735

cos (0.8

nn)

Figure P8.25

Figure P8.26
of the system to minimize the average squared error between the outputs of the two
systems.
(a) Use the least-squares criterion to determ ine the equations for the optimum F IR
filter coefficients.
(b) R epeat part (a) if the output of the unknown system is corrupted by an additive
white noise {u>()} sequence with variance cr2.
8.27 D eterm ine the least-squares F IR inverse of length 3 to the system with impulse re
sponse
2,

h(n) =

1.
0.

n= 0
n= 1
otherwise

Also, determ ine the minimum squared error min.

8.28 D eterm ine the least-squares FIR inverse filter of length 3 for the system with im
pulse response h(n) given in Example 8.5.6, when a = | and the desired response is
specified as d(n) = 8(n 2). Also compute the minimum least-squares error.

736

Design of Digital Filters

Chap. 8

8.29* A linear time-invariant system has an input sequence x(n) and an output sequence
y(n). The user has access only to the system output y(n). In addition, the following
information is available.
(a ) The input signal is periodic with a given fundamental period N and has a flat
spectral envelope, that is,
N- 1
jt(n) = ^ 2 ct e Ji2xmk"
all n
k=0
where c*k 1 for all k.
(b) The system H(z) is all-pole, that is,
H(z) = ------- -------i+ X > -*
*=4
but the order p and the coefficients (a*, 1 < k < p) are unknown. Is it possible to
determine the order p and the numerical values of the coefficients {ak, 1 < k < p\
by taking measurements on the output y(n)7 If yes, explain how. Is this possible
for every value of p i
(c) Repeat Problem 8 . 3 1 for a system with system function
M
Y , b*: ~k
H (z) =

-----------

k=\
(d) FIR system modeling Consider an unknown FIR system with impulse response
h ( n ) , 0 < n < 11, given by
h(0)

/i( 1 1 )

0.309828

x
x

1 0 ~

h(l)

/i(10)

0.416901

h ( 2)

A (9)

-0.577081

l O -1

h(3) = h ( 8 )

- 0 . 8 52502

1 0 " 1

h( 4 ) = h( 7 )

0.147157 x

10

h(5)

0.449188

10

h(6)

1 0 ~

A potential user has access to the input and output of the system but does not
have any information about its impulse response other than that it is FIR. In
an effort to determine the impulse response of the system, the user excites it
with a zero mean, random sequence x(n) uniformly distributed in the range
[ 0 . 5 , 0 . 5 ] , and records the signal x(n) and the corresponding output y(n) for
0

<

<

199.

(1) By using the available information that the unknown system is FIR, the
user employs the method of least-squares to obtain an FIR model h(n),
0 < n < M 1. Set up the system of linear equations, specifying the param
eters / i ( 0 ) , A ( l),. . . , h(M 1 ) . Specify formulas we should use to determine
the necessary autocorrelation and crosscorrelation values.

Chap. 8

737

Problems

(2) Since the order of the system is unknown, the user decides to try models of
different orders and check the corresponding total squared error. Clearly,
this error will be zero (or very close to it if the order of the model be
comes equal to the order of the system). Com pute the F IR models hM{n),
0 < n < M - 1 for M = 8, 9, 10, 11, 12, 13, 14 as well as the corre
sponding total squared errors E M, M = 8, 9........14. What do you ob
serve?
(3) D eterm ine and plot the frequency response of the system and the models
for M = 1 1 ,1 2 , 13. Comment on the results.
(4) Suppose now that the output of the system is corrupted by additive noise,
so instead of the signal v(n), 0 < n < 199, we have available the sig
nal
v(n) = y(n) + O.Oliy(n)
where w(n) is a Gaussian random sequence with zero mean and variance
ff2 = 1 .
R epeat part (b) by using u(n) instead of _v(n) and comment on the results. The
quality of the model can be also determined by the quantity
OC

J2h
2(n)
n=0

Sampling and Reconstruction

of Signals

In C hapters 1 and 4 w e treated the sam pling o f con tin u ou s-tim e signals and d em on
strated that if the signals are bandlim ited, it is p ossib le to reconstruct the original
signal from the sam ples, provided that the sam pling rate is at least tw ice the highest
frequency contained in th e signal. W e also briefly describ ed th e sub seq u en t o p er
ations o f quantization and coding that are n ecessary to con vert an analog signal
to a digital signal appropriate for digital processing.
In this chapter w e con sid er tim e-d om ain sam pling, analog-to-digital (A /D )
con version (quantization and cod in g), and d igital-to-an alog ( D /A ) conversion (sig
nal reconstruction) in greater depth. First, w e con sid er th e sam pling o f the sp e
cial class o f signals that are characterized as bandpass signals. T h en w e treat
an alog-to-digital converters and their characteristics. O f particular interest is
the use o f oversam pling and sigm a-delta m odu lation in th e design o f high p re
cision A /D converters. T h e final top ic o f th e chapter is d igital-to-an alog con ver
sio n or, sim ply, the reconstruction o f th e con tinu ou s-tim e sign al from its sam pled
values.

9.1 SAMPLING OF BANDPASS SIGNALS

O ur m ain focus in this section is th e sam pling o f bandpass signals. W e begin by
describing th e tim e and frequency dom ain rep resen tation s o f bandpass signals.

9.1.1 Representation of Bandpass Signals

S u p pose that a real-valued signal x ( t ) has a freq u en cy co n ten t concentrated in a
narrow band o f freq u en cies in the vicinity o f a frequency Fc, as show n in Fig. 9.1.
O ur ob jective is to d ev elop a m athem atical rep resen tation o f such signals. First,
w e construct a signal that contains o n ly the p ositive freq u en cies in x ( t ) . Such a
signal can b e exp ressed as
X+(F) = 2 V ( F ) X ( F )
738

(9.1.1)

Sec. 9.1

Sampling of Bandpass Signals

739

|X(F)|

Figure 9.1 Spectrum of a bandpass

signal.

-Fr

w here X ( F ) is the Fourier transform o f x ( t ) and V ( F ) is the unit step function.

T he equivalent tim e-dom ain expression for (9.1.1) is
x +(t)

OC

X + ( F ) e i2* f ' d F
-OC

(9.1.2)
= f - 1 [2 V ( F ) ] * F , [ ^ ( ^ ) ]
T h e signal A+(r) is called the analytic signal or the pre -en v elo p e o f jr(r). W e n ote
that F ~ l [X ( F ) ] = x ( t ) and
/ r [2 V < /)] = <$(r) + ^ 7T/

(9.1.3)

H en ce,
-M O =

*x(t)

SU) +

TT t

x(t) + j * x ( t)

(9.1.4)

TT 1

W e define x { t ) as
x ( t ) = x ( t )
nt
=

iH Jroc ^t

(9.1.5)
^

T h e signal x ( t ) can be view ed as the output o f the filter with im pulse response
00 < t < oo

hit) = ,
nt

(9.1.6)

w hen excited by the input signal * ( /). Such a filter is called a Hilber t trans form er .
T he frequency resp onse of this filter is sim ply
/ OO
h ( t ) e ~ j2,lF,d t
*00

e
- nI Jf.oc i.
t

dt

( F > 0)

(F = 0)

(9.1.7)

(F < 0)

We observe that |tf(F )| = 1 and that the phase response (F) = - I n for F > 0

740

Sampling and Reconstruction of Signals

Chap. 9

and @ (F) = jjr for F < 0. T h erefore, this filter is basically a 90 p h ase shifter
for all frequencies in the input signal, and it is akin to the discrete-tim e H ilbert
transform filter described in Section 8.2.6.
T he analytic signal * + (/) is a bandpass signal. W e can obtain an equivalent
low pass representation by perform ing a frequency translation o f X + ( F ) . Thus, w e
define X ,(F ) as
X , ( F ) = X + ( F + Fc)

(9.1.8)

T h e equivalent tim e-dom ain relation is

* /(/) = x + ( l )e ~ ]2 Fr'
= [*(f) + ; x ( r ) K j2* fr'

(9.1.9)

or, equivalently,
* (/) + j x ( t ) = x , ( t ) e j2nF''

(9.1.10)

In general, the signal x/(t) is com p lex-valu ed (see P roblem 9.3), and can be
expressed as
jc, ( 0

= M / ) + jf' M 0

(9.1.11)

If w e substitute for x/(t) in (9.1.10) and eq u ate real and im aginary parts on each
side, we obtain the relations
x(r) = u< (t) c o s 2 n F ct - u s (t) s i n 2 j t F ct

(9.1.12)

x ( t ) = ,.(/) sin 2?r Fct + u s {t) cos 27rF,t

(9.1.13)

T he expression (9.1.12) is the desired form for the representation o f a band

pass signal. T h e low -frequency signal com p on en ts uc (t) and u s (t) can be view ed
as am plitude m odulations im pressed on the carrier com p on en ts cos 2 n F ct and
s i n 2 n F ct, respectively. Since th ese carrier com p onents are in phase quadrature,
u c(t) and u s (t) are called the q ua dra ture c o m p o n e n t s o f the bandpass signal x ( t ) .
A n o th er representation o f the signal in (9.1.12) is
x ( t ) = R e{[Wf(r) + y.t ( r ) y 2jr^ }
= R e M f W 2^ ' ]

(9.1.14)

w here R e d en o tes the real part o f the com p lex-valu ed quantity in the brackets
follow ing. T h e low pass signal x/(r) is usually called the c o m p l e x envelo pe o f the
real signal j(? ), and is basically the eq uivalent l ow p a ss signal.
Finally, a third p ossib le representation o f a bandpass signal is ob tain ed by
expressing * ,(/) as
x,U ) = a ( t ) e Jfnn

(9.1.15)

a(t ) = yj u l ( t ) + u](t )

(9.1.16)

where

Sec. 9.1

Sampling of Bandpass Signals

741

Then
.r(r) = R e[x,(r)e-/2irF'']
= R e [ a ( f ) ^ 23rF,+^>J]
= a (r )c o s[ 2 jr /v / + 0 ( 0 ]

(9.1.18)

T h e signal a ( t ) is called the en velop e o f jc(r), and 6 ( t) is called the p h a s e o f jc(r).

T h erefore, (9.1.12), (9.1.14), and (9.1.18) are eq u ivalen t rep resen tation s o f b and
pass signals.
T h e Fourier transform o f x ( t ) is

[R e[ xi(t)ej27fFr' ] } e - i2,tF,d t

(9.1.19)

U se o f the identity
R e ( f) = i ( * + n

(9.1.20)

in (9.1.19) yield s the result

f X [x,(t)eJ2*F' ' + x ; ) e - J2* F' ' ] e - J2* F'd t

X ( F ) = -l
J

-oc

= \ [ X t ( F Fc) + X j i F F(.)]

(9.1.21)

w here X i ( F ) is the Fourier transform o f x/(t). T his is the basic relationship b etw een
the spectrum o f the real bandpass signal x ( t ) and the spectrum o f the equivalent
low pass signal xi(t).
It is apparent from (9.1.21) that the spectrum o f the bandpass signal x ( t )
can b e ob tain ed from the spectrum o f the com p lex signal x t (t) by a frequency
translation. T o be m ore precise, su p p ose that the spectrum o f the signal *;(/)
is as sh ow n in Fig. 9.2(a). T hen the spectrum X ( F ) for p ositive freq u en cies is
sim ply X f ( F ) translated in frequency to the right by Fc and scaled in am plitude by
5 . T he spectrum X ( F ) for negative freq u en cies is ob tain ed by first foldin g X / ( F )
about F = 0 to obtain X i ( - F ), conjugating X t ( ~ F ) to ob tain X * ( - F ) , translating
X * ( - F ) in frequency to the left by /> , and scaling the result by
T h e folding
and conjugation o f X i ( F ) for the negative-frequency co m p on en t o f the spectrum
result in a m agnitude spectrum |X (F )| that is even and a phase spectrum 2^ X ( F )
that is od d as show n in Fig. 9.2(b). T h ese sym m etry p roperties m ust hold since
the signal x ( t ) is real valued. H ow ever, they do not apply to the spectrum o f the
equ ivalen t com plex signal X i(t).
T h e d ev elo p m en t ab ove im plies that a n y b a nd p ass signal x ( t ) can b e re pre
se nted b y an eq u ivalent lo w pa ss signal x/(t). In general, the equivalent low pass
signal x /( t) is com p lex valued, w hereas the bandpass signal x ( t ) is real. T h e latter
can b e ob tain ed from the form er through the tim e-d om ain relation in (9.1.14) or
through the frequency-dom ain relation in (9.1.21).

742

Sampling and Reconstruction of Signals

Chap. 9

|*,(F)|

-f2

F,

&i(F)

(P

(a)
|X(F)|

-F-F,

-F-F+F,

F - F-, - Fr F + F,

K X (n

Figure 9.2 (a) Spectrum of the lowpass signal and (b) the corresponding spectrum
for the bandpass signal.

9.1.2 Sampling of Bandpass Signals

W e have already dem onstrated that a con tin u ou s-tim e signal with highest fre
q uency B can be uniquely represented by sam p les taken at the m inim um rate
(N yq uist rate) o f 2 B sam ples per secon d . H ow ever, if the signal is a bandpass
signal w ith frequency com p onents in the band B\ < F < B i, as show n in Fig. 9.3,
a blind ap plication o f the sam pling theorem w ould have us sam p lin g the signal at
a rate o f 2 B 2 sam ples per secon d.
If that w ere the case and B 2 w as an ex trem ely high freq u en cy, it would
certainly be ad van tageou s to perform a freq u en cy shift o f the bandpass signal by

Sec. 9.1

Sampling of Bandpass Signals

743

X(F)

B-,

-B,

Figure 9.3 Bandpass signal with

frequency components in the range
B\ < F 5 B2.

an am ount
Fc =

B x + B2

(9.1.22)

and sam pling the equivalent low pass signal. Such a frequency shift can be achieved
by m ultiplying the bandpass signal as given in (9.1.12) by the quadrature carriers
co s 2 jt Fc[ and sin 2 n F ct and low pass filtering the products to elim inate the signal
co m p o n en ts at 2 Fc. Clearly, the m ultiplication and the su b seq u en t filtering are first
perform ed in the analog dom ain and then the ou tp u ts o f the filters are sam pled.
T h e resulting eq u ivalen t low pass signal has a bandw idth B /2 , w h ere B = B2 B\.
T h erefore, it can be represented uniquely by sam ples taken at the rate o f B sam p les
per secon d for each o f the quadrature com p onents. Thus the sam pling can be
p erform ed on each o f the low pass filter outputs at the rate o f B sam p les per secon d,
as indicated in Fig. 9.4. T herefore, the resulting rate is 2 B sam ples per second.
In v iew o f the fact that frequency con version to low pass allow s u s to reduce
the sam pling rate to I B sam ples per secon d, it should be p ossib le to sam ple the
bandpass signal at a com parable rate. In fact, it is.
S u p p ose that the upper frequency Fc + B p , is a m ultiple o f th e bandwidth B
(i.e., Fc + B / 2 = k B ) , w here k is a p ositive integer. If w e sam ple * ( /) at the rate

Figure 9.4 Sampling of a bandpass signal by first converting to an equivalent

low-pass signal.

744

Sampling and Reconstruction of Signals

Chap. 9

2 B = 1 / r sam ples per secon d , w e have

x ( n T ) = u c( n T ) o o s 2 n Fcn T us ( n T ) s i n 2 j i Fcn T
n n ( 2 k - 1)
. nn{2k - 1 )
= u c( n T ) c o s ------- ------------us { n T ) s i n -------- --------

(9.1.23)

w here the last step is ob tain ed by substituting Fc ~ k B B [2 and T = 1 /2 B.

For n ev en , say n = 2m , (9.1.23) red u ces to
x ( 2 m T ) = x ( m T \ ) = u c(m T i) cos 7rm(2k 1) = (l ) mu c(mT ])

(9.1.24)

w here T\ = 2 T = 1 /B . For n odd, say n = 2m 1, (9.1.23) red u ces to

x(2 m T - T ) = x (m Ti - ^

= u s (' m T j - ^

( - l ) m + t+ 1

(9.1.25)

T h erefore, the even -n um b ered sam ples o f x{ t), which occur at the rate o f B sam
p les per secon d, produce sam p les o f the low p ass signal com p on en t u c(t). T he
odd-num bered sam ples o f x ( t ) , which also occu r at the rate o f B sam ples per
secon d , produce sam ples o f the low pass signal com p onent K ,(f).
N o w , the sam p les {uc{mT\)} and the sam ples (w,(m 7i 7 \/2 )) can b e used
to reconstruct the equivalent low pass signals. Thus, according to the sam pling
theorem for low p ass signals with T\ = 1 / B ,
^
, .,s in ( 7 r /r i) (/ - m 7 j )
2 ^ u ^ m T 0 , / T Ul------- = rr( x / T \ ) ( t m T \)

uA t) =
f.\
M

V'
)

^ ^ sin (?r/7i)(r mT\ + 7 \/2 )

t
n

,n 1 ^
(9.1.26)

* T

r +

r m

( M

2 7 )

Furtherm ore, th e relation s in (9.1.24) and (9.1.25) allow us to express u c(t) and
3 (i) directly in term s o f sam p les o f x ( t ) . N ow , sin ce x ( t ) is exp ressed as
x ( t ) = u c(t) c o s 2 n F ct u s (t) s m 2 n Fct

(9.1.28)

substitution from (9.1.27), (9.1.26), (9.1.25), and (9.1.24) in to (9.1.28) yield s

'O
=

\,

^ s in (jr /2 r )(r -2 m 7 )

XQa

) ( n / 2 T W - 2 r , T ) - C S 2 * F '

(9.1.29)
B ut
( l ) m c o s 2 ti Fct = c o s 2 j r Fc{t 2m T )

and
(l ) m+* sin27r Fct =

cos2nrFc(r 2m T + T )

Sec. 9.1

Sampling of Bandpass Signals

745

W ith th ese substitutions, (9.1.29) reduces to

^
T

x (i)=

sin (jr/27')(/ - m T )
------ - c o s l n F c{ t - m T )
x(m T)
[n/2T )(t - m T)

(9.1.30)

w here T = 1 /2 B. T his is the desired reconstruction form ula for the bandpass signal
x ( i ) , with sam p les taken at the rate o f I B sam ples per secon d , for the special case
in w hich the upper band frequency Fc + B f 2 is a m ultiple o f the signal ban d
w idth B.
In the general case, where only the condition Fc > B f 2 is assum ed to hold,
let us define the in teger part o f the ratio Fc + B / 2 to B as
Fc + B / 2

(9.1.31)

W hile holding the upper cu toff frequency Fc + B / 2 constant, w e increase the

bandwidth from B to B' such that
Fc + B / 2
= r

(9.1.32)

Furtherm ore, it is con ven ien t to define a new center frequency for the increased
bandwidth signal as
+

(9.1.33)

Clearly, the increased signal bandw idth B' includes the original signal spectrum of
bandwidth B.
N o w the upper cu to ff frequency Fc + B / 2 is a m ultip le o f B', C onsequently,
the signal reconstruction form ula in (9.1.30) h old s with Fc replaced by F'c and T
replaced by T \ w here T ' = 1 /2 B \ that is,
/2 T '){t m T ')

^
* > - ^

r ) l d r w

- r ) . c m 2 7 , F {' ~ m T )

This p roves that x{ t) can b e represented by sam ples taken at the uniform rate
1 / T 2 B r ' / r , w here r' is the ratio
,

Fc + B / 2
j

Fc
-

1
+

( 9 .1 .3 5 )

and r = \r'J.
W e o b serve that w hen the upper cu toff frequency Fc + B / 2 is n ot an integer
m ultiple o f the bandwidth B , the sam pling rate for the bandpass signal m ust be
increased by the factor r ' f r . H ow ever, n ote that as Fc/ B increases, the ratio r ' f r
ten d s tow ard unity. C on sequ en tly, the p ercent increase in sam pling rate tends to
zero.
T he derivation given ab ove also illustrates the fact that the low pass signal
co m p o n en ts u c(t) and u s(t) can b e exp ressed in term s o f sam p les o f the bandpass

Sampling and Reconstruction of Signals

746

Chap. 9

signal. In d eed , from (9.1.24), (9.1.25), (9.1.26), and (9.1.27), w e obtain the result

mo -

j l <-ir,

and
, > =

- r ) Sjn(" / ? r ') ( , ~ 2 " r + n

(ff /2 r ')( f - 2 n T ' + V )

( 9 .U 7 )

w here r = Lr'J.
In con clu sion , w e have dem onstrated that a bandpass signal can b e repre
sen ted u n iqu ely by sam ples taken at a rate
2 B < Fs < 4 B
w here B is the bandw idth o f the signal. T h e low er lim it ap p lies w h en the upper
frequency Ft. + B / 2 is a m ultiple o f B, T he upper lim it on Fs is ob tain ed under
w orst-case con d ition s w hen r = 1 and r' % 2 .

9.1.3 Discrete-Time Processing of Continuous-Time Signals

A s indicated in our introductory rem arks in C hapter 1, th ere are num erous ap
p lications w here it is advantageous to p rocess con tin u ou s-tim e (analog) signals on
a digital signal processor. Figure 9.5 illustrates the general configuration of the
system for d igital processing o f an analog signal. In design in g the processing to
be p erform ed, w e m ust first select the bandw idth o f the signal to be processed
since the signal bandw idth determ ines the m inim um sam pling rate. For exam ple,
a speech signal, w hich is to be transm itted digitally, can con tain frequency com p o
nen ts ab ove 3000 H z, but for p u rp oses o f sp eech intelligibility and sp eak er id en
tification, the preservation o f frequency com p on en ts b elow 3000 H z is sufficient.
C on sequ en tly, it w ould be inefficient from a processin g view p oin t to preserve the
h igher-frequency com p on en ts and w asteful o f channel bandw idth to transm it the
extra bits n eed ed to represent these h igher-frequency co m p o n en ts in the speech
signal. O nce th e desired frequency band is selected w e can sp ecify the sam pling
rate and the characteristics o f the prefilter, w hich is also called an antialiasing filter.

Antialiasing filte r T he antialiasing filter is an an alog filter which has a

tw ofold purpose. First, it ensures that the bandw idth o f the signal to b e sam pled
is lim ited to the desired frequency range. Thus any freq u en cy com p on en ts o f
th e signal ab o v e th e foldin g frequency Fs f l are sufficiently atten u ated so that the

Figure 9.5

Configuration of system for digital processing of an analog signal.

Sec. 9.1

Sampling of Bandpass Signals

747

am ount o f signal distortion due to aliasing is negligible. F or exam ple, the sp eech
signal to be transm itted digitally over a telep h o n e channel w ould be filtered by
a low p ass filter having a passband exten d ing to 3000 H z, a transition band o f
approxim ately 400 to 500 H z, and a stopband ab ove 3400 to 3500 H z. T he speech
signal m ay be sam pled at 8000 H z and h en ce the fold in g frequency w ould be
4000 H z. Thus aliasing would be n egligible.
A n o th e r reason for using an antialiasing filter is to lim it the additive n oise
spectrum and other interference, w hich often corrupts the desired signal. U su
ally, ad d itive n oise is w ideband and exceed s the bandw idth o f the desired signal.
B y prefiltering w e reduce the additive noise p ow er to that which falls w ithin the
bandwidth o f the desired signal and w e reject the ou t-of-b an d noise.
Ideally, w e w ould like to em p loy a filter with steep cu toff frequency resp onse
characteristics and with no delay distortion w ithin the passband. Practically, h ow
ever, w e are constrained to em p loy filters that have a finite-w idth transition region,
are relatively sim ple to im plem ent, and introduce som e tolerab le am ount o f delay
distortion. V ery stringent filter specifications, such as a narrow transition region,
result in very com plex filters. In practice, w e may c h o o se to sam p le the signal well
ab ove the Nyquist rate and thus relax the design sp ecification s on the antialiasing
filter.
O n ce w e have specified the prefilter requirem ents and have selected the d e
sired sam pling rale, w e can p roceed with the design o f the digital signal processing
op eration s to be perform ed on the discrete-tim e signal. T h e selection o f the sam
pling rate Fs = 1 / 7 , where T is the sam pling interval, not only determ ines the
highest frequency (Fsf 2) that is preserved in the analog signal, but also serves as a
scale factor that influences the design specifications for d igital filters and any other
discrete-tim e system s through w hich the signal is processed .
For exam ple, su p p ose that w e have an analog signal to be differentiated that
has a bandw idth o f 3000 H z. A lth ou gh differentiation can be perform ed directly
on the analog signal, w e ch oose to do it digitally in discrete tim e. H en ce w e
sam ple th e signal at the range Fs = 8000 H z and d esign a digital d ifferentiator as
described in Sec. 8.2.4. In this case, the sam pling rate Fs = 8000 H z estab lish es
the foldin g frequency o f 4000 H z, which corresponds to the frequency w = n
in the discrete-tim e signal. H en ce the signal bandwidth o f 3000 H z corresponds
to the frequency wc = 0.75tt. C on sequ en tly, the discrete-tim e differentiator for
p rocessing the signal would be designed to have a passband o f 0 < \w\ < 0.75jt.
A s an oth er exam ple o f digital p rocessing, the sp eech signal that is b andlim
ited to 3000 H z and sam pled at 8000 H z m ay be separated in to tw o or m ore
frequency subbands by digital filtering, and each subband o f sp eech is digitally e n
cod ed with different precision, as is d on e in subband cod in g (see Section 10.9.5 for
m ore d etails). T h e frequency resp onse characteristics o f the digital filters for sep
arating th e 0- to 3000-H z signal in to subbands are sp ecified relative to the folding
frequency o f 4000 H z, which corresponds to the frequency co = n for the d iscrete
tim e signal. Thus w e m ay process any con tinu ou s-tim e signal in the discrete-tim e
dom ain by perform ing equivalent op eration s in discrete tim e.

746

Sampling and Reconstruction of Signals

Chap. 9

T he o n e im plicit assum ption that w e h ave m ade in this discussion on the

eq u ivalen ce o f continuous-tim e and d iscrete-tim e signal p rocessin g is that the quan
tization error in analog-to-digital con version and rou n d -off errors in digital signal
processing are negligible. T h ese issu es are further discussed in this chapter. H o w
ever, w e should em p hasize that analog signal processin g op eration s cannot be
d on e very p recisely either, since electron ic com p on en ts in analog system s have
toleran ces and th ey introduce n oise during their operation. In general, a d igi
tal system designer has b etter con trol o f toleran ces in a digital signal processing
system than an analog system d esign er w h o is designing an eq u ivalen t analog
system .

9.2 ANALOG-TO-DIGITAL CONVERSION

T h e discussion in Section 9.1 focu sed on the con version o f con tinu ou s-tim e signals
to discrete-tim e signals using an id eal sam pler and ideal in terp olation . In this
section w e deal with the d evices for p erform ing th ese con version s from analog to
digital.
R ecall that the p rocess o f con vertin g a con tin u ou s-tim e (analog) signal to
a digital seq u en ce that can b e p rocessed by a digital system requires that w e
quantize the sam pled values to a finite num ber o f levels and represent each level
by a num ber o f bits. The electron ic d evice that perform s this conversion from an
analog signal to a digital seq u en ce is called an an alog-to-digital ( A / D ) converter
(A D C ). On the other hand, a digital-to-analog ( D / A ) co nverter (D A C ) takes a
digital seq u en ce and produces at its ou tp u t a voltage or current proportional to
the size o f th e digital word applied to its input. D /A con version is treated in
Section 9.3.
Figure 9.6(a) show s a block diagram o f the basic elem en ts o f an A /D co n
verter. In this section w e con sid er the perform ance requirem ents for these e l
em ents. A lth o u g h we focu s m ainly on ideal system characteristics, w e shall also
m en tion som e k ey im perfection s en cou n tered in practical d evices and indicate how
th ey affect the perform ance o f the con verter. W e con cen trate on th ose aspects that
are m ore relevant to signal processing applications. T h e practical aspects o f A /D
converters and related circuitry can b e foun d in the m anufacturers specifications
and data sheets.

9.2.1 Sample-and-Hold
In practice, the sam pling o f an analog signal is p erform ed b y a sam ple-and-hold
(S /H ) circuit. T h e sam pled signal is th en q uantized and con verted to digital form.
U sually, the S/H is integrated in to the A /D converter.
T he S/H is a digitally con trolled analog circuit that tracks the analog input
signal during the sam ple m od e, and then h old s it fixed during th e hold m ode to
the instantaneous value o f the signal at the tim e th e system is sw itched from the

Sec. 9.2

749

Analog-to-Digital Conversion

(a)
Tracking

Figure 9.6 (a) Block diagram of basic elements of an A/D converter; (b) timedomain response of an ideal S/H circuit.

sam ple m o d e to the hold m ode. Figure 9.6(b) show s the tim e-dom ain resp onse o f
an ideal S/H circuit (i.e., a S/H that responds instantaneously and accurately).
T he goal o f the S/H is to continuously sam ple the input signal and then to
hold that value constant as lon g as it takes for the A /D converter to ob tain its
digital representation. T h e use o f an S/H allow s the A /D con verter to op erate
m ore slow ly com pared to the tim e actually used to acquire the sam ple. In the
ab sen ce o f a S/H , the input signal must n ot change by m ore than on e-h alf o f the
quantization step during the conversion, which m ay be an im practical constraint.
C on sequ en tly, the S/H is crucial in h igh-resolution (12 bits per sam ple or higher)
digital conversion o f signals that have large bandw idths (i.e., they change very
rapidly).
A n ideal S/H introduces n o distortion in the conversion p rocess and is ac
curately m o d eled as an ideal sam pler. H ow ever, tim e-related degradations such
as errors in the p eriodicity o f the sam pling p rocess ( jitter), n onlinear variations
in the duration o f the sam pling aperture, and changes in the voltage held during
con version ( d ro o p ) d o occur in practical devices.
T h e A /D converter begins the conversion after it receives a convert co m
m and. T h e tim e required to com p lete the con version should b e less than the
duration o f the hold m o d e o f the S/H . F urtherm ore, the sam pling p eriod T should
b e larger than the duration o f the sam ple m od e and the hold m ode.
In th e follow in g section s w e assum e that the S/H introduces n egligib le errors
and w e fo cu s on the digital conversion o f the analog sam ples.

750

Sampling and Reconstruction of Signals

Chap. 9

9.2.2 Quantization and Coding

T h e basic task o f the A /D converter is to con vert a con tin u ou s range o f input
am plitudes in to a discrete set o f digital cod e w ords. T his con version in volves the
processes o f q ua ntiza tio n and coding. Q uantization is a n on linear and noninvertible p rocess that m aps a given am plitude x( n ) = x ( n T ) at tim e t = n T into an
am plitude x k, taken from a finite set o f values. T h e p roced u re is illustrated in
Fig. 9.7(a), w h ere the signal am plitude range is divided in to L intervals
h = {** < x( n ) < x * + i}

k = 1 ,2 ,..., L

(9.2.1)

by the L + 1 decision levels x \ , x L, . . . , x L+\- T h e p ossib le ou tputs o f the quantizer

(i.e., the quantization levels) are d en oted as x\ , x 2, . . . , x i . T h e op eration o f the
quantizer is defined by the relation
*<?() = G [*(n )j = *k

if * (n ) Ik

(9.2.2)

In m ost digital signal processing operations the m apping in (9.2.2) is in d ep en

dent o f n (i.e., th e quantization is m em oryless and is sim ply d e n o te d as x q = Q[x]).
Furtherm ore, in signal processing w e o ften use u n i f o r m or linea r q u an tizer s defined
by
4+i - x k = A
k = l,2 ,...,L -l
(9.2.3)
Xk+i - x k = A
for finite x k, x t+]
w here A is the q u a n tize r step size. U n iform quantization is usually a requirem ent
if the resulting digital signal is to b e p rocessed by a digital system . H ow ever,
in transm ission and storage applications o f signals such as sp ee ch , n onlinear and
tim e-variant quantizers are frequently used.
If a zero is assigned a quantization level, the quantizer is o f the m id tre a d
type. If zero is assigned a decision level, the quantizer is called a m id rise type.
Quantization
levels

Decision
levels

\
*3

*3

<

\
*4 H

x5

r~

xk xk xk*\

Instantaneous amplitude ------- (a)

x >~

xl

! - 4A

>

x2

*2

- 3A

x3

x5

- 2A

*4 *4 -*5
-

x5

x6

X1

X1

2A

Instantaneous amplitude-------

xi

-*8 -*9= 00
3A

!--------------------------- Range of quantizer------------------------------ -I

(b)
Figure 9.7

Quantization process and an example o f a midtread quantizer.

Sec. 9.2

Analog-to-Digital Conversion

751

Figure 9 .7 (b ) illustrates a m idtread quantizer with L = 8 levels. In theory, the

extrem e d ecisio n lev els are taken as *i = oo and x l +i = oo, to cover the total
d y n a m i c ra n g e o f the input signal. H ow ever, practical A /D converters can handle
on ly a finite range. H en ce w e define the range R o f th e quantizer by assum ing
that /] = I I = A. For exam ple, the range o f the quantizer show n in Fig. 9.7(b) is
equal to 8 A , In practice, the term full-scale range (F S R ) is used to describe the
range o f an A /D converter for bipolar signals (i.e., signals with b oth p ositive and
negative am plitu d es). T h e term f u l l scale (F S ) is used for unipolar signals.
It can b e easily seen that the q uantization error eq (n) is always in the range
- A / 2 to A /2 :
A
A
- < eq{n) <
(9.2.4)
In other w ords, the instantaneous quantization error cannot exceed half o f the
q uantization step. If the dynam ic range o f the signal, defined as x max jcmjn, is
larger than th e range o f the quantizer, the sam ples that exceed the quantizer
range are clip p ed , resulting in a large (greater than A /2 ) quantization error.
T h e op era tio n o f the quantizer is b etter described by the quantization char
acteristic fun ction , illustrated in Fig. 9.8 for a m idtread quantizer with eight

Figure 9.8

Example of a midtread quantizer.

752

Sampling and Reconstruction of Signals

Chap. 9

quantization levels. This characteristic is preferred in practice over the midriser

b ecau se it provides an output that is in sen sitive to infinitesim al changes o f the
input signal about zero. N o te that the input am plitu d es o f a m idtread quantizer
are rounded to the nearest quantization levels.
T h e c od ing process in an A /D con verter assigns a unique binary num ber to
each quantization level. If w e have L levels, w e n eed at least L different binary
num bers. W ith a w ord length o f b + 1 bits w e can represent 2 b+l distinct binary
num bers. H en ce w e should have 2b+1 > L or, eq u ivalen tly, b + 1 > log 2 L. T hen
the step size or the re solution o f the A /D converter is given by

where R is the range o f the quantizer.

There are various binary cod in g schem es, each with its advantages and dis
advantages. Table 9.1 illustrates som e existin g sch em es for 3-bit binary coding.
T h ese num ber representation schem es w ere d escribed in detail in Section 7.5.
The tw o s-com p lem ent representation is used in m ost digital signal pro
cessors. Thus it is con ven ien t to u se the sam e system to represent digital sig
nals becau se w e can operate on them directly w ithou t any extra format conver-

TABLE 9.1

C O M M O N LY US ED BIPOLAR C O D E S
Decimal Fraction

Number

Positive
Reference

+7

+i

+6

+i

+5

+5

+4

+i

+3

+1

+2

+1

+1
0
0
-1
-2

0+
01
B
2
s

Negative
Reference
7
8
h
K
5
*
4
5
3
K
2
S
1
8
00+

Sign +
Magnitude

Two's
Complement

Offset
Binary

Ones
Complement

0111

0111

1111

0111

0 110

0110

1110

0 110

0 10 1

0 10 1

110 1

0 10 1

0 1 00

0 10 0

1100

0 1 00
00 1 1

00 1 1

0 0 11

10 11

0010

0 0 10

10 10

00 1 0

000 1

000 1

100 1

000 1

+s

0 00 0
1000
100 1

0000
(0 0 0 0)
1111

1000
( 1 0 0 0)
0111

0000
1111
1110

10 10

1110

0 110

110 1

110 1

0 10 1

1100

1100

0 10 0

10 11

-3

+1

10 11

-4

~S

+i!

1100

-5

~s
6
~s
7
~g
8
S

+!

110 1

10 11

00 1 1

10 10

+!

1110

10 10

00 1 0

10 0 1

+1

1111

10 0 1

000 1

1000

(1 0 0 0)

(0 0 0 0)

-6
-7
-8

+i

Sec. 9.2

Analog-to-Digital Conversion

sion. In gen eral, a (b + 1) -bit binary fraction o f the form

value

753
P b has the

- A > - 2 + f t 2 -1 + 02 2~2 H--------h f o 2~ b

if w e use th e tw o s-com p lem ent representation. N o te that 0 q is the m ost signifi
cant bit (M S B ) and 0 b is the least significant bit (L S B ). A lth ou gh the binary cod e
used to represent the quantization levels is im portant for the design o f the A /D
converter and the su bsequent n um erical com putations, it d o e s n ot h ave any effect
in the p erform ance o f the quantization process. Thus in our sub seq u en t discus
sions w e ign ore the p rocess o f cod in g when w e analyze the perform ance o f A /D
converters.
Figure 9.9(a) sh ow s the characteristic o f an ideal 3-bit A /D converter. The
only degradation introduced by an ideal con verter is the q uantization error, which
can b e reduced by increasing the num ber o f bits. This error, that d om in ates the
p erform ance o f practical A /D converters, is analyzed in the next section.
Practical A /D con verters differ from id eal con verters in several w ays. V ariou s
d egradations are usually en cou n tered in practice. A num ber o f these perform ance
degradations are illustrated in Fig. 9 .9 (b )-(e ). W e n ote that practical A /D con vert
ers m ay have offset error (the first transition m ay not occur at exactly + |L S B ) ,
scale-factor (or gain) error (the d ifferen ce b etw een the values at which the first
transition and the last transition occur is n ot equal to FS 2L S B ), and linearity
error (th e differen ces b etw een transition valu es are n ot all eq u al or uniform ly
changing). If the differential linearity error is large en ou gh , it is possib le for on e
or m ore cod e words to b e m issed. P erform ance data on com m ercially available
A /D con verters are specified in the m anufacturers data sh eets.

9.2.3 Analysis of Quantization Errors

T o d eterm in e the effects o f quantization on the p erform ance o f an A /D converter,
w e adopt a statistical approach. T h e d ep en d en ce o f the q uantization error on the
characteristics o f the input signal and the n onlinear nature o f the quantizer m ake
a determ in istic analysis intractable, excep t in very sim ple cases.
In the statistical approach, w e assum e that the q uantization error is random
in nature. W e m odel this error as n oise that is added to the original (unquan
tized ) signal. If the input analog signal is within the range o f the quantizer, the
q uantization error eq(n) is b ou n ded in m agnitu d e [i.e., \eq {n)\ < A /2 ], and the
resulting error is called gra nular noise. W hen the input falls ou tsid e the range o f
the quantizer (clipping), eq (n) b eco m es unb oun d ed and results in o verloa d noise.
T his type o f n o ise can result in severe signed distortion w h en it occurs. O ur on ly
rem ed y is to scale the in p ut signal so that its dynam ic range falls within the range
o f the quantizer. T h e follow in g analysis is b ased on the assum ption that there is
n o overload n oise.
T h e m athem atical m od el for th e quantization error ev (n) is sh ow n in Fig. 9.10.
T o carry ou t th e analysis, w e m ake th e follow in g assum ptions about the statistical

Sampling and Reconstruction of Signals

Chap. 9

Output digital number (code and fractional value)

754

Normalized analog input

(a)

(b)

(c)

(d)

(e)

Figure 9.9 Characteristics of ideal and

practical A/D converters.

properties o f eq( n ):
L T h e error eq(n) is uniform ly distributed over the range A /2 < eq{n) < A /2.
2. T h e error seq u en ce
is a stationary w h ite n o ise seq u en ce. In other
w ords, the error eq (n ) and the error eq (m) for m
are uncorrelated.

Sec. 9.2

Analog-to-Digital Conversion
Quantizer

x(n)

755

xAn)

GM>]
(a) Actual system

x(n)-

X(fn) = x (n) + e^n)

&

eq{n)

Figure 9.10 M athem atical m odel of

(b) Mathematical model

quantization noise.

3. T h e error se q u en ce {eQ(n)} is uncorrelated with th e signal seq u en ce x ( n ).

4. T h e signal se q u en ce x ( n ) is zero m ean and stationary.
T h ese assu m p tion s do not hold, in general. H ow ever, they do hold w hen
the quan tization step size is sm all and the signal seq u en ce x ( n ) traverses several
quan tization levels b etw een tw o successive sam ples.
U n d er these assum ptions, the effect o f the additive n o ise eq(n) on the d esired
signal can b e quantified by evaluating the signal-to-quantization n oise (pow er)
ratio (S Q N R ), w hich can be exp ressed on a logarithm ic scale (in decib els or
d B ) as
S Q N R = 101og 10
MI

(9.2.6)

w here Px = a j = E [ x 2{n)} is the signal p ow er and P = a ] E[e^(n)] is the p ow er

o f the qu an tization n oise.
If th e quantization error is uniform ly distributed in the range (A /2 , A /2 )
as sh ow n in Fig. 9.11, th e m ean valu e o f th e error is zero and th e variance (the
quantization n o ise p ow er) is

A/2

f A/2

. , / m d e = z L A/2

A2

(9.2.7)

Pie)

1
A

Figure 9.11 Probability density

function for the quantization error.

756

Sampling and Reconstruction of Signals

Chap. 9

By com bining (9.2.5) w ith (9.2.7) and substituting the result in to (9.2.6), the
exp ression for the S Q N R b ecom es
S Q N R = 10 log = 20 log
P

a,

(9.2.8)
= 6.02* + 16.81 - 20 log
ox

dB

T he last term in (9.2.8) d ep en d s o n the range R o f the A /D converter and the

statistics o f the input signal. For exam p le, if w e assum e that jf(n) is Gaussian
distributed and the range o f the quantizer exten d s from 3ax to 3ax (i.e., R = 6 ax ),
then less than 3 o u t o f every 1000 input signal am plitudes w ou ld result in an
overload on the average. F or R = 6ax , (9.2.8) b eco m es
S Q N R = 6.02* + 1.25 dB
T h e form ula in (9.2.8) is frequently used to specify the p recision n eed ed in an
A /D converter. It sim ply m ean s that each additional bit in the quantizer increases
the signal-to-quantization n oise ratio by 6 dB. (It is interesting to n ote that the
sam e result w as derived in Section 1.4 for a sinusoidal signal using a determ inistic
approach.) H o w ev er, w e should bear in m ind the con d ition s under which this
result has b een derived.
D u e to lim itations in the fabrication o f A /D converters, their perform ance
falls short o f the theoretical value given by (9.2.8). A s a result, the effective number
o f bits m ay b e som ew h at less than the num ber o f bits in the A /D converter. For
instance, a 16-bit converter m ay have on ly an effective 14 bits o f accuracy.

9.2.4 Oversampling A/D Converters

T he basic idea in oversam pling A /D converters is to increase the sam pling rate
o f the signal to th e point w here a low -resolu tion quantizer suffices. B y oversam
pling. w e can reduce the dynam ic range o f the signal valu es b etw een successive
sam ples and thus reduce the resolu tion requirem ents on the quantizer. A s w e
have ob served in the p receding section , the variance o f the q uantization error in
A /D conversion is cr,2 = A 2 /1 2 , w here A = R f l b+l. Since the dynam ic range o f
the signal, which is proportional to its standard deviation ox , should m atch the
range R o f the quantizer, it follow s that A is p roportional to ax . H en ce for a given
num ber o f bits, the p ow er o f the quantization n oise is proportional to the variance
o f the signal to b e quantized. C on sequ en tly, for a given fixed S Q N R , a reduction
in th e variance o f the signal to be q uantized allow s us to red u ce the num ber o f
bits in the quantizer.
T h e basic id ea for reducing the dynam ic range leads us to con sid er differential
quantization. T o illustrate this p oin t, let us evalu ate th e variance o f the difference
b etw een tw o su ccessive signal sam ples. Thus w e have
d ( n ) x (n ) jc (n 1)

(9.2.9)

Sec. 9.2

757

Analog-toDigital Conversion

T h e variance o f d ( n ) is
u l = E [ d 2(n)] = {[x (n ) - x ( n - l ) ] 2}
= E [ x \ n ) } - 2 E [ x ( n ) x ( n - 1)] + E [ x 2(n - 1)]
= 2<7j[1 -

y* A 1

(9.2.10)

)]

w here y xx( l ) is the value o f the autocorrelation seq u en ce yxx(m) o f x ( n ) evalu ated
at m = 1. If Yx.r (l) > 0.5, w e ob serve that aJ < cr^. U n d er this condition, it
is better to quantize the d ifferen ce d{n) and to recover x( n ) from the quantized
valu es {d9(n)}. T o obtain a high correlation b etw een successive sam ples o f the
signal, w e require that the sam pling rate be significantly higher than the N yquist
rate.
A n ev en b etter approach is to quantize the d ifference
d ( n ) = x ( n ) a x ( n 1)

(9.2.11)

w here a is a param eter selected to m inim ize the variance in d{n). This leads to
the result (see Problem 9.7) that the optim um ch oice o f a is
a =

Yxx^

Yxx^

Yxx (0)

and
a d2 = a 2x [\ - a 2]

(9.2.12)

In this case, ctJ < a 2, since 0 < a < 1. T he quantity a x ( n 1 ) is called a first-order
predictor o f jc().
Figure 9.12 sh ow s a m ore general differential predictiv e signal quantizer sys
tem . This system is u sed in sp eech en cod in g and transm ission over telep h on e
channels and is know n as differential p u lse co d e m odulation (D P C M ). T h e goal
o f the predictor is to provide an estim ate x ( n ) o f x ( n ) from a linear com bination
o f past values o f x ( n ) , so as to red u ce the dynam ic range o f the difference signal
d ( n ) = x { n ) x ( n ). T hus a predictor o f order p has the form
p
x ( n ) = ^ 2 akx (n - k )

Figure 9.12 Encoder and decoder for differential predictive signal quantizer
system.

(9.2.13)

758

Sampling and Reconstruction of Signals

Chap. 9

T he use o f the feed b ack loop around the quantizer as sh ow n in Fig. 9.12 is n ec
essary to avoid the accum ulation o f quantization errors at the d ecoder. In this
configuration, the error e ( n ) = d ( n ) dq (n) is
e(n) = d ( n ) - dq(n) = x(n ) - x (n) - dq (n) = Jf(n) - x q (n)
Thus the error in the reconstructed quantized signal x q (n) is equal to the quan
tization error for the sam ple d (n). T h e d ecod er for D P C M that reconstructs the
signal from the q uantized values is also show n in Fig. 9.12.
T h e sim p lest form o f differential predictive quantization is called delta m o d
ulation (D M ). In D M , the quantizer is a sim ple 1-bit (tw o -lev el) quantizer and the
predictor is a first-order predictor, as shown in Fig. 9.13(a). B asically, D M pro
vid es a staircase approxim ation o f the input signal. A t every sam pling instant, the
sign o f the d ifferen ce b etw een the input sam ple x ( n ) and its m ost recent staircase
approxim ation x ( n ) = a x q (n 1 ) is determ ined, and then the staircase signal is
updated by a step A in the direction o f the d ifference.
From Fig. 9.13(a) w e ob serve that
x q (n) = a x g(n - 1) + dq {n)

(9.2.14)

which is the d iscrete-tim e equivalent of an analog integrator. If a = 1, w e have

an ideal accum ulator (integrator) w hereas the ch oice a < 1 results in a leaky
integrator. Figure 9.13(c) show s an analog m od el that illustrates the basic prin
ciple for the practical im plem en tation o f a D M system . T h e analog low pass filter
is necessary for the rejection o f out-of-band com p on en ts in the frequency range
b etw een B and Fs f l , since Fs > > B due to oversam pling.
T he crosshatched areas in Fig. 9.13(b) illustrate tw o typ es o f quantization
error in D M , slo p e-o v erload distortion and granular n oise. Since the m axim um
slop e A ( T in x ( n ) is lim ited by the step size, slop e-overload distortion can be
avoided if m ax \ d x ( t ) / d t \ < A / T . T h e granular n oise occurs w h en the D M tracks
a relatively flat (slow ly changing) input signal. W e n ote that increasing A reduces
overload distortion but increases th e granular n oise, and vice versa.
O ne way to reduce these tw o types o f distortion is to use an integrator in
front o f the D M , as sh ow n in Fig. 9.14(a). This has tw o effects. First, it em phasizes
the low freq u en cies o f x (t) and increases the correlation o f th e signal in to the D M
input. S econd, it sim plifies the D M d ecod er b ecau se the d ifferen tiator (inverse
system ) required at th e d ecod er is can celed by the D M integrator. H en ce the
d ecod er is sim ply a low pass filter, as show n in Fig. 9.14(a). Furtherm ore, the two
integrators at the en co d er can b e replaced by a single in tegrator placed before
the com parator, as show n in Fig. 9.14(b). T h is system is know n as sigma-delta
m o d u l a t i o n (S D M ).
S D M is an ideal candidate for A /D con version . Such a con verter takes
advantage o f the high sam pling rate and spreads the quantization n oise across the
band up to Fs f2 . Since Fs > > B, the n oise in the signal-free band B < F < F , f l
can b e rem oved by appropriate digital filtering. T o illustrate this principle, let

Sec. 9.2

Analog-to-Digital Conversion

759

Coder

Decoder
(a)

(b)
T

(c)

Figure 9.13 Delta modulation system and two types of quantization errors.
us con sid er the discrete-tim e m od el o f S D M , sh ow n in Fig. 9.15, where w e have
assum ed that the com parator ( 1 -bit quantizer) is m od eled by an additive w hite
n o ise source w ith variance a f = A 2 /12. T h e integrator is m od eled b y the d iscrete
tim e system with system function
(9.2.15)

760

Sampling and Reconstruction of Signals

Figure 9.14

Chap. 9

Sigm a-detta m odulation system.

H{z)

ein)

T h e z-transform o f the sequ en ce {dq (n)} is

1

1 + H {z )

1 + H (z)

E (z )
(9.2.16)

= H A z)X (z) + H(z)E(z)

w h ere H s {z) and H{z) are the signal and n oise system fun ction s, respectively. A
go o d S D M system has a flat frequency resp onse Hs {co) in th e signal frequency

Sec. 9.2

761

Analog-to-Digrtal Conversion

band 0 < F < B . O n the other hand, H(z) should have high attenuation in the
frequency band 0 < F < B and low attenuation in the band B < F < Fs/2.
For th e first-order S D M system with th e integrator specified by (9.2.15), w e
have
Hs (z) = z ~ 1

(9.2.17)

H n(z) = l - z ~ 1

T hus H s (z) d o es n o t distort the signal. T h e p erform ance o f the S D M system is

therefore d eterm in ed by the n oise system function H n(z), w hich has a m agnitude
frequency response
J

(/^) I = 2 sin

(9.2.18)
Fs

as show n in Fig. 9.16. T h e in-band quantization n oise variance is given as

(9.2.19)
w here Se( F ) = a ] j F s is the p ow er spectral density o f the quantization n oise.
From this relationship w e n ote that doubling Fs (increasing the sam pling rate by
a factor o f 2), w hile k eep in g B fixed, reduces the p ow er o f the quantization n oise
by 3 dB. T his result is true for any quantizer. H ow ever, additional reduction m ay
be possib le by properly ch oosin g the filter H ( z ) .
For the first-order S D M , it can be sh ow n (see P roblem 9.10) that for Fs > >
2 B, the in-band q uantization n oise p ow er is
(9.2.20)
N o te that doubling th e sam pling frequency reduces the n oise p ow er by 9 dB o f
which 3 dB is due to the reduction in St ( F ) and 6 dB is due to the filter charac
teristic H (F). A n additional 6 -dB reduction can be ach ieved by using a d ou b le
integrator (see P rob lem 9.11).
In sum m ary, th e n oise p ow er
can be reduced by increasing the sam
pling rate to spread the quantization n oise ppw er over a larger frequency band
( - F J 2, Fs/ 2), and then shaping th e n oise p ow er spectral density by m eans o f an

F,
~T
Figure 9.16

F,
2
Frequency (magnitude) response o f noise system function.

762

Sampling and Reconstruction of Signals

Analog section

Chap. 9

Digital section

SDM - to - PCM Converter

Figure 9.17

Basic elem ents o f an oversam pling A /D converter.

appropriate filter. Thus, S D M provides a 1-bit quantized signal at a sam pling fre
q uency Fs = 2I B , w here the oversam pling (interp olation) factor I d eterm in es the
SN R o f the S D M quantizer.
N ext, we explain how to convert this signal in to a b-bit q uantized signal at
the N yquist rate. First, w e recall that the S D M d ecod er is an an alog low p ass filter
with a cu toff frequency B. T h e output o f this filter is an approxim ation to the
input signal x(r). G iven the 1-bit signal dq {n) at sam pling frequency Fs , w e can
obtain a signal x q {n) at a low er sam pling frequency, say the N yquist rate o f 2 B
or som ew hat faster, by resam pling the output o f the low pass filter at the 2 B rate.
T o avoid aliasing, w e first filter out the out-of-band (fl, F J 2) n o ise by processing
the w ideband signal. T h e signal is then passed through the low pass filter and
resam pled (dow nsam pled ) at the low er rate. T he dow nsam pling process is called
decimation and is treated in great detail in Chapter 10.
For exam p le, if the interpolation factor is I = 256, the A /D con verter output
can be obtained by averaging su ccessive non-overlapping blocks o f 128 bits. This
averaging w ould result in a digital signal with a range o f valu es from zero to
256{b as 8 bits) at the N yquist rate. T he averaging p rocess also provid es the
required antialiasing filtering.
Figure 9.17 illustrates the basic elem en ts o f an oversam p lin g A /D converter.
O versam pling A /D con verters for voice-band (3-k H z) signals are currently fab
ricated as integrated circuits. T ypically, they op erate at a 2 -M H z sam pling rate,
dow nsam ple to 8 kH z, and provide 16-bit accuracy.

Sec. 9.3

Digital-to-Analog Conversion

763

9-3 DIGITAL-TO-ANALOG CONVERSION

In Section 4.2.9 w e dem onstrated that a bandlim ited low p ass analog signal, which
has b een sam p led at the N yquist rate (or faster), can be reconstructed from its
sam ples w ithou t distortion. T h e ideal reconstruction form ula or ideal interpolation
form ula derived in Section 4.2.9 is
sin(:rr/T)(r n T )

(9.3.1)

(n/T )(t - n T )

w here the sam pling interval T l / F s = 1 /2 B , Fs is th e sam pling frequency and

B is the bandwidth o f the analog signal.
W e have view ed the reconstruction o f the signal x { t ) from its sam ples as an
in terp olation prob lem and have described the function
s in ^ r /T )
g ( t) =

(9.3.2)

n t/T

as the ideal interpolation function. T he interpolation form ula for x(r), given by
(9.3.1), is basically a linear superposition o f tim e-sh ifted versions o f g (t). with each
g (t n T ) w eigh ted by the corresponding signal sam ple x ( n T ) .
A ltern atively, w e can view the reconstruction o f th e signal from its sam ples as
a linear filtering p rocess in which a discrete-tim e seq u en ce o f short pulses (ideally
im pulses) with am plitudes equal to the signal sam ples, excites an analog filter, as
illustrated in Fig. 9.18. T h e analog filter corresponding to the ideal interpolator
has a frequency response
T,

IFf

0,

|F | >

<

H (F) =

Fs

2T

(9.3.3)

2T

H ( F ) is sim ply the F ourier transform of the in terp olation function g{t). In other
w ords, H { F ) is the frequency resp onse o f an analog reconstruction filter w h ose
Ideal analog
lowpass filter
H (F)

Reconstructed signal

Input signal
F.

Y .

x(nT)& (t ~ nT)

x (t)=

oc

sin

0 - riT)

Y l

x(nT )~

--------------

n=-oc

Figure 9.18 Signal reconstruction viewed as a filtering process.

(! nT)

764

Sampling and Reconstruction of Signals

Chap. 9

H (F )

L
'

Fi
2

(a)
hU)

(b)
Figure 9.19 Frequency response (a) and the impulse response (b) of an ideal
low-pass filter.
im pulse response is h(t) = g(t). A s show n in Fig. 9.19, the id eal reconstruction
filter is an ideal low pass filter and its im pulse resp on se exten d s for all tim e. H ence
the filter is noncausal and physically nonrealizable. A lth ou gh th e interpolation
filter with im pulse resp onse given by (9.3.1) can be approxim ated closely with
som e delay, the resulting function is still im practical for m ost ap p lication s where
D /A con version is required.
In this sectio n w e p resent som e practical, alb eit nonideat. in terp olation tech
niques and interpret them as linear filters. A lth ou gh m any sophisticated poly
nom ial interpolation tech n iqu es can b e d evised and analyzed, our discussion is
lim ited to constant and linear interpolation. Q uadratic and h igh er polyn om ial in-

Sec. 9.3

Digital-to-Analog Conversion

765

terp olation is often used in num erical analysis, but is it less likely to be used in
digital signal processing.

9.3.1 Sample and Hold

In practice, D /A conversion is usually perform ed by com bining a D /A converter
w ith a sam ple-and-hold (S/H ) and follow ed by a low pass (sm ooth in g) filter, as
show n in Fig. 9.20. T h e D /A converter accepts at its input, electrical signals that
correspond to a binary w ord, and p rod u ces an output voltage or current that is
proportional to the valu e o f the binary w ord. Ideally, its in p u t-ou tp u t characteristic
is as show n in Fig. 9.21(a) for a 3-bit bipolar signal. T he line connecting the d ots is
a straight line through the origin. In practical D /A converters, the line connecting
the d ots m ay d eviate from the ideal. Som e o f the typical d eviation s from ideal
are offset errors, gain errors, and nonlinearities in the in p u t-ou tp u t characteristic.
T h ese types o f errors are illustrated in Fig. 9.21(b).
A n im portant param eter o f a D /A converter is its settling tim e, which is
defined as the tim e required for the output o f the D /A converter to reach and
rem ain w ithin a given fraction (usually, i^ L S B ) o f the final value, after applica
tion o f the input cod e word. O ften , the ap plication o f the input cod e word results
in a high-am plitude transient, called a glitch . This is especially the case w hen
tw o co n secu tive cod e words to the A /D differ by several bits. T h e usual w ay to
rem edy this problem is to use a S/H circuit d esigned to serve as a d eglitcher.
H en ce the basic task o f the S/H is to hold th e output o f the D /A converter c o n
stant at the p revious output value until the n ew sam ple at the output o f the D /A
reach es steady state, then it sam ples and h olds the new valu e in the next sam pling
interval. Thus the S/H approxim ates the analog signal by a series o f rectangular
p u lses w h o se height is equal to the corresponding value o f the signal pulse. F ig
ure 9.22(a) illustrates the approxim ation o f the analog signal x ( t ) by a S/H . A s
sh ow n , the approxim ation, d en oted as x ( t ), is basically a staircase function which
ta k es the signal sam ple from the D /A con verter and h old s it for T secon ds. W hen
the n ext sam ple arrives, it jum ps to the n ext value and h old s it for T secon ds, and
so on.
W hen view ed as a linear filter, as show n in Fig. 9.22(b), the S/H has an
im pulse response
h ( t) =

Digital
input

signal

Digitalto-analog
convener

Figure 9.20

1,
0,

Sample

and
hold

0 < t< T
otherw ise

(9.3.4)

Lowpass
smoothing
filter

Basic operations in converting a digital signal into an analog signal.

Analog
output
signal

766

Sampling and Reconstruction of Signals

Analog output voltage

Ideal D/A

3A

2A

/
_l_______1_______L.
100

101

110

111

001

000

010

011

Input code words

/
-A

-2 A

-4A

Offset :
error

JL,

//

sy

r /1 i i i

i i i

Z x "

Scale factor
or gain error

L_i_u-iiii
Nonmonoionicity

Figure 9.21 (a) Ideal D/A converter characteristic and (b) typical deviations from
ideal performance in practical D/A converters.

Chap. 9

Sec. 9.3

767

Digital-to-Analog Conversion

Sampled signal -

-x(t)

S /H

(b)

*(/)

Figure 9.22 (a) Approximation of an

analog signal by a staircase; (b) linear
filtering interpretation; (c) impulse
response of the S/H.

(c)

T his is illustrated in Fig. 9.22(c). T h e corresponding freq u en cy resp onse is

OO
h{t)e->2F'd t
00

[ T e - j2* F,d t
Jo

r ( sin n F T \
\ 7TF T )

(9.3.5)
,-jxFT

T h e m agnitude and p h ase o f H ( F ) are p lotted in Figs. 9.23. For com parison, the
freq u en cy resp o n se o f th e id eal interpolator is su p erim p osed o n th e m agnitude
characteristics.
It is apparent that the S/H d o es not p ossess a sharp cu toff frequency re
sp on se characteristic. T his is du e to a large exten t to the sharp transitions o f
its im pulse resp onse h ( t). A s a con seq u en ce, the S /H p asses undesirable aliased
frequency co m p o n en ts (freq u en cies ab ove Fsf 2) to its output. T o rem edy this
problem , it is com m on practice to filter i( r ) by passing it through a low pass filter

768

Sampling and Reconstruction of Signals

Chap. 9

l(F)|

9(F)

which highly atten u ates frequency com p onents ab ove F J 2. In effect, the low pass
filter follow ing the S/H sm ooth s the signal x ( t ) by rem oving th e sharp discontinu
ities.

9.3.2 First-Order Hold

A first-order hold approxim ates x ( t ) by straight-line segm en ts w hich have a slop e
that is d eterm ined by the current sam ple x { n T ) and the previou s sam ple x ( n T T).
A n illustration o f this signal reconstruction tech n iqu es is given in Fig. 9.24.
T h e m athem atical relationship b etw een the input sam p les and the output
w aveform is
s(r)

= x(nT) +

T)

X
^ n T ~ T - (t

raf)

nr<K(n

l)r

(9.3.6)

Sec. 9.3

Digital-to-Analog Conversion

769

W hen view ed as a linear filter, the im pulse resp onse o f the first-order hold is
1 + T'

h(t) =

t
1
0,

0 <t <T
(9.3.7)

T < t < IT
otherw ise

T his im pulse response is dep icted in Fig. 9.25(a). T h e Fourier transform o f h ( t)

y ield s the frequency response, which can be exp ressed in the form
jm F )
e}

H ( F ) = T ( \ + A n F 2T 2) X f l ( ^ ^ ^ j

(9.3.8)

w here the phase 0 (F ) is

(F ) = n F T + tan - 1 2 n F T

(9.3.9)

T h ese frequency resp onse characteristics are graphically illustrated in Fig. 9.25(b)
and (c).
Since this reconstruction tech n iqu e also suffers from distortion due to passage
o f frequency com p o n en ts ab ove Fs/ 2, as can be ob served from Fig. 9.25(b), it is
fo llo w ed by a low pass filter that significantly atten u ates freq u en cies ab ove the
fold in g frequency Fs f l .
T h e p eak s in H { F ) within the band |F | < Fsf l m ay be undesirable in som e
applications. In such a case it is possib le to m odify th e im pulse response by
reducing the slop e by so m e factor 0 < 1. T his results in th e im pulse response h (t)
illustrated in Fig. 9.26(a). T he corresponding frequency resp onse is given by

-r[i-

0 + 0(1 + j l n F T )

sin n F T , - j xFT I sin jrF T

nFT *
n FT

(9.3.10)

T h e m agnitude |/ / ( F ) | is illustrated in Fig. 9.26(b) for 0 = 0.5, 0 = 0.3, and

0 = 0.1. W e n o te that the peak in H ( F ) is relatively sm all for 0 = 0.3 and

k(l)

\H {F )\

(b)
9(F)

Figure 9.25 Impulse response (a) and frequency response characteristics (b) and
(c) for a first-order hold.
770

Sec. 9.3

771

Digital-to-Analog Conversion
h(t)

MF)!

2T

IT

(b)
Figure 9.26 Impulse response (a) and frequency (magnitude) response (b) for a
modified first-order hold.
d o es not exist w h en 0 = 0.1. T hus this m odified first-order hold exhibits b etter
frequency resp onse characteristics in the frequency range |F | < Fs/ 2.

9.3.3 Linear Interpolation with Delay

T h e first-order hold perform s signal reconstruction by com p utin g th e slo p e o f the
straight lin e b ased on th e current sam p le x ( n T ) and th e past sam p le x ( n T T ) o f

772

Sampling and Reconstruction of Signals

Chap. 9

the signal. In effect, this tech n iqu e linearly extrapolates or attem p ts to linearly p r e
dict the next sam ple o f the signal based on the sam ples x ( n T ) and x { n T - T). A s
a co n sequ en ce, the estim ated signal w aveform i ( r ) contains jum ps at the sam ple
points.
The jum ps in x ( t ) can be avoided by providing a o n e-sa m p le delay in the re
construction process. Then successive sam ple p oin ts can b e con n ected by straightlin e segm ents. Thus the resulting interpolated signal x ( t ) can b e exp ressed as
x(t) = x{nT - T ) +

x ( n T ) x ( n T 7")

(t n T )

n T < t < (n + 1 )T

(9.3.11)

W e ob serve that at t n T , x { n T ) = x ( n T T ) and at t = n T + T , x ( n T + T ) =

x ( n T ) . T herefore, x ( t ) has an inherent delay o f T secon d s in interpolating the
actual signal jc(r). Figure 9.27 illustrates this linear in terp olation technique.
V iew ed as a linear filter, the linear interpolator with a 7 -se c o n d d elay has
an im pulse response
0 < t < T
i t IT.
(9.3.12)
hit) =
2 -t/T .
T < t < 2T
otherw ise
10 .
T he corresponding frequency response is
( 2

dt
(9.3.13)

-j2x Fi

T h e im pulse response and frequency resp onse characteristics o f this interpolation

filter are illustrated in Fig. 9.28. W e observe that the m agnitude characteristic
falls o ff rapidly and contains sm all sid elo b es b eyon d the sam p lin g frequency Fs.
Furtherm ore, its p h ase characteristic is linear due to the d elay T. B y follow ing
this interpolator with a low pass filter that has a sharp cu toff b eyon d the frequency
Fs/2, the high-frequency com p on en ts in x ( t ) can b e further reduced.

Figure 9.27

Linear interpolation of x{t) with a 7-second delay.

Sec. 9.3

773

Digital-to-Analog Conversion
A (r)

(a)

IW )I

_3
T

__2
T

_ J _ __ 1_ 0
T
IT

J_
IT

J_
T

2_

2
T

(b)
9(F)

(c)

Figure 9.28 Impulse response (a) and frequency response characteristics (b) and
(c) for the linear interpolator with delay.

774

Sampling and Reconstruction of Signals

Digital section

Chap. 9

Analog section

Figure 93.9 Elements of an oversampling D/A converter.

This con clu d es our discussion o f signal reconstruction b ased on sim ple inter
p olation techniques. T he tech n iqu es that w e have described are easily incorporated
into the design o f practical D /A converters for the reconstruction o f analog signals
from digital signals. W e shall con sid er interpolation again in C hapter 10 in the
context o f changing the sam pling rate in a digital signal processin g system .

9.3.4 Oversampling D/A Converters

T he elem en ts o f an oversam pling D /A converter are shown in Fig. 9.29. A s we
ob serve, it is subdivided in to a digital front end follow ed by an an alog section. The
digital section con sists o f an interpolator w h ose function is to increase th e sam pling
rate by som e factor / , and th en is follow ed by a SD M . T he in terp olator sim ply
increases the digital sam pling rate by inserting I 1 zeros b etw een successive
low rate sam ples. T he resulting signal is then processed by a digital filter with
cu toff frequency Fc = B / F s in order to reject the im ages (replicas) o f the input
signal spectrum . T his higher rate signal is fed to the S D M , w hich creates a n o ise
shaped 1-bit sam ple. E ach 1-bit sam ple is fed to the 1-bit D /A . w hich provides
the analog interface to th e an tialiasing and sm ooth in g filters. T h e output analog
filters have a passband o f 0 < F < B hertz and serve to sm ooth the signal and to
rem ove the quantization n oise in the frequency band B < F < Fs/2. In effect, the
oversam pling D /A converter uses S D M with the roles of th e analog and digital
section s reversed com pared to the A /D converter.
In practice, oversam pling D /A (and A /D ) converters have m any advantages
over the m ore co n v en tion al D /A (and A /D ) converters. First, the high sam pling
rate and the su b seq u en t digital filtering m inim ize or rem ove the n eed for com plex
and exp en sive analog an tialiasing filters. Furtherm ore, any an alog n o ise introduced
during the con version p h ase is filtered out. A lso , there is no n eed for S/H circuits.
O versam pling S D M A /D and D /A con verters are very robust with resp ect to vari
ations in the analog-circuit param eters, are inherently linear, and have low cost.

9.4 SUMMARY AND REFERENCES

T h e m ajor focus o f this chapter w as on the sam pling and reconstruction o f sig
nals. In particular, w e treated the sam pling o f con tinu ou s-tim e signals and the
subsequent op eration o f A /D con version . T h ese are necessary op eration s in the

Chap. 9

Problems

775

digital p rocessin g o f a n alog signals, either on a gen eral-p u rp ose com puter or on a
cu stom -d esign ed digital signal processor. T he related issue o f D /A conversion was
also treated. In addition to the con ven tional A /D and D /A con version techniques,
w e also described a n oth er type o f A /D and D /A con version , based on the principle
o f oversam p lin g and a type o f w aveform en cod in g called sigm a-delta m odulation.
S igm a-d elta con version tech n ology is especially suitable for audio band signals due
to their relatively sm all bandwidth (less than 20 k H z) and in som e applications,
th e requirem ents for high fidelity.
T h e sam pling th eorem was introduced by N yquist (1928) and later p opular
ized in the classic paper by Shannon (1949). D /A and A /D con version techniques
are treated in a b ook by Sheingold (1986). O versam pling A /D and D /A con ver
sion has b een treated in the technical literature. Specifically, w e cite the work o f
C andy (1986), C andy et al. (1981) and Gray (1990).

PROBLEMS

9.1 Consider the sampling of the bandpass signal whose spectrum is illustrated in Fig. P9.1.
Determine the minimum sampling rate Fx to avoid aliasing.
X(F)

9 2 Consider the sampling of the bandpass signal whose spectrum is illustrated in Fig. P9.2.
Determine the minimum sampling rate Fs to avoid aliasing.
X(F)

9.3 Prove that xi(t) is generally a complex-valued signal and give the condition under
which it is real. Assume that x(t) is a real-valued bandpass signal.
9.4 Consider the two systems shown in Fig. P9.4.
(a) Sketch the spectra of the various signals if xa(t) has the Fourier transform shown
in Fig. 9.4(b) and F, = 2 B. How are >](f) and ^ (z) related to xa(t)f
( b ) Determine >i(r) and ^(z) if xa(t) = cos2nFot, Fo = 20 Hz, and F, = 50 Hz or
Fs = 30 Hz.

776

Sampling and Reconstruction of Signals

F,

Chap. 9

F,

(a)

XJF)

(b)

Figure P9.4

9.5 A contin uou s-tim e signal xa(t) with bandw idth B and its ech o xa(t - r) arrive sim ul
taneously at a T V receiver. T h e received an aiog signal

sa(t) = xa(r) + a x a( t - T )

|| < 1

is processed by the system shown in F ig, P 9.5. Is it possible to specify F , and H (z )

so that ya(t) = xa(t) [i.e., rem ov e th e g h ost xa(i - r ) from th e received signal]?

F,

Figure P9J
9 j6 A bandlim ited co n tin u ou s-tim e signal x(t) is sam pled at a sam plin g frequen cy F s >
2 B . D ete rm in e th e en ergy E d o f th e resulting d iscrete -tim e signal x(n) as a function
o f th e energy o f th e an alog signal, E a, and th e sam pling period T = 1 / F S.
9 .7 L e t x(n) b e a zero-m ean statio n ary p rocess with v arian ce

and a u to co rrela tio n y j l ) .

Chap. 9

777

Problems

a j = a 2[l + a2 - 2apx(l)]
where px(l) = j'x(l)/?'* (0) is the normalized autocorrelation sequence,
(b) Show that <rj attains its minimum value

for a = Kv(l)/Xr() = ArC1)

(c) U nder what conditions is a j < a ;?
(d) R epeat steps (a) to (c) for the second-order prediction error
d(n) = x(n) a^x(rt 1) aixln 2)
9 .8 Consider a DM coder wilh input ,v(n) = A cos(2n n F/ F, ). W hat is the condition for

avoiding slope overload? Illustrate this condition graphically.

9.9 Let x(t) be a bandlimited signal with fixed bandwidth B and variance rr*.
(a) Show that the signal-lo-quantization noise ratio. SQNR = 101og,n(fl7/<v). in
creases by 3 dB each time we double the sampling frequency F,. Assume that
the quantization noise model discussed in Section 9.2.3 is valid.
(b ) If we wish to increase the SQNR of a quantizer by doubling its sampling fre
quency, what is the most efficient way to do it? Should we choose a linear
muUibil A/D converter or an oversampling one?
9.10 Consider the first-order SDM model shown in Fig. 9.15.
(a) Show that the quantization noise power in the signal band {- B . B ) is given by

(b ) Using a two-term Taylor series expansion of the sine function and assuming that

Fs > > B, show that

9.11 Consider the second-order SDM model shown in Fig. P 9 .ll.

(a) D eterm ine the signal and noise system functions H,(z) and H(z), respectively.
(b ) Plot the magnitude response for the noise system function and compare it with
the one for the first-order SDM, Can you explain the 6-dB difference from these
curves?
(c) Show that the in-band quantization noise power a 2 is given approximately by

which implies a 15-dB increase for every doubling of the sampling frequency.

778

Sampling and Reconstruction of Signals

Chap. 9

e(n)

Figure P 9 .ll

9.12 Figure P9.12 illustrates the basic idea for a lookup table based sinusoidal signal gen
erator. The samples of one period of the signal
'2n

x ( n ) = cos ( n

n = 0 ,1 ........N - l

are stored in memory. A digital sinusoidal signal is generated by stepping through

the table and wrapping around at the end when the angle exceeds 2x. This can be
done by using modulo-N addressing (i.e.. using a circular buffer). Samples of x ( n )
are feeding the ideal D /A converter every T seconds.
(a) Show that by changing F, we can adjust the frequency F{, of the resulting analog
sinusoid.
(b) Suppose now that Fs = l / T is fixed. How many distinct analog sinusoids can be
generated using the given lookup table? Explain.

xa(t) = cos 2 ttF0t

Figure P9.12

9.13 Suppose that we represent an analog bandpass filter by the frequency response
H( F) = C( F - Fc) + C*{F - Fc)
where C ( f ) is the frequency response of an equivalent lowpass filter, as shown in
Fig. P9.13.

Chap. 9

Problems

779

(a) Show that the impulse response c(r) of the equivalent lowpass filter is related to
the impulse response h(t) of the bandpass filter as follows:
h(t) = 2Re[c(t)ej2jrF{']
(b) Suppose that the bandpass system with frequency response H ( F ) is excited by a
bandpass signal of the form
x(t) = Re[u(f)e-'2rF,''3
where u(r) is the equivalent lowpass signal. Show that the filter output may be
expressed as
y(r) = Re[r(/)e-'2)rfV']
where

(Hint: Use the frequency domain to prove this result.)

C(F)

-8

Figure P9.13

9.14* Consider the sinusoidal signal generator in Fig. P9.14, where both the stored sinusoidal
data

and the sampling frequency Fs = 1/ T are fixed. An engineer wishing to produce a

sinusoid with period 2N suggests that we use either zero-order or first-order (linear)
interpolation to double the num ber of samples per period in the original sinusoid as
illustrated in Fig. P9.14(a).
(a) Determ ine the signal sequences v(n) generated using zero-order interpolation
and linear interpolation and then com pute the total harm onic distortion (TH D )
in each case for N = 32, 64. 128.
(b) R epeat part (a) assuming that all sample values are quantized to 8 bits.
(c) Show that the interpolated signal sequences y(n) can be obtained by the system
shown in Fig. P9.14(b). The first module inserts one zero sample between suc
cessive samples of x ( n ). D eterm ine the system H(z) and sketch its magnitude
response for the zero-order interpolation and for the linear interpolation cases.
Can you explain the difference in perform ance in term s of the frequency response
functions?

780

Sampling and Reconstruction of Signals

Chap. 9

Interpolated values

Interpolated values

(a>

Insert
zeros

jc/n)

Figure P9.13

(a)

nU(r\
\i)

Rpire P9.13 (b)

(d) Determine and sketch the spectra of the resulting sinusoids in each case both
analytically [using the results in part (c)] and evaluating the DFT of the resulting
signals.
(e) Sketch the spectra of x, (n) and j(n), if x(n) has the spectrum shown in Fig. P9.14(c)
for both zero-order and linear interpolation. Can you suggest a better choice for

Chap. 9

781

Problems

XM

0
3

n
3

(c)

Figure P9.13

(c)

9.15 Let xa(t) be a time-limited signal: that is, xa(t) = 0 for |/| > t , with Fourier transform
X ( F ) . The function X ( F ) is sampled with sampling interval &F = 1/T<.
(a) Show that the function
CC

can be expressed as a Fourier series with coefficients

1
ct = y X J k & F )

(b ) Show that X ( F) can be recovered from the samples X a(kSF), - o c < k < oc if
T, >

2t.

(c) Show lhal if Tx < 2r. there is time-domain aliasing that prevents exact recon

struction of X U(F).
(d) Show that if 7, > 2 r, perfect reconstruction of X a(F) from the samples X(k5F)
is possible using the inlerpolation formula
OO
XJF) = ^ 2

X a (k&F)

sin { ( j i / S F ) ( . F ~ k S F ) ]
(n /8 F )(F - k S F )

10

Multirate Digital Signal

Processing

In m an y p ractical a p p lic a tio n s o f d igital signal pro cessin g , o n e is faced w ith th e

p ro b le m o f c h an g in g th e sa m p lin g ra te o f a signal, e ith e r in c re a sin g it o r d e creasin g
it by so m e a m o u n t. F o r e x a m p le , in te le c o m m u n ic a tio n sy stem s th a t tra n sm it an d
receiv e d ifferen t ty p e s o f signals (e.g., te le ty p e , facsim ile, sp e e c h , v id eo , e tc .), th e re
is a re q u ire m e n t to p ro cess th e v ario u s signals at d iffe re n t ra te s c o m m e n s u ra te w ith
th e c o rre sp o n d in g b a n d w id th s o f th e signals. T h e p ro cess o f c o n v e rtin g a signal
fro m a given ra te to a d iffe re n t ra te is called samp l i n g rate conversi on. In tu rn ,
sy stem s th a t em p lo y m u ltip le sa m p lin g ra te s in th e p ro cessin g o f d ig ital signals a re
called multirale digital si gnal proc es s i ng systems.
S am p lin g ra te c o n v ersio n o f a d igital signal can b e a c c o m p lish ed in o n e of
tw o g en eral m e th o d s. O n e m e th o d is to p ass th e digital signal th ro u g h a D /A
co n v e rte r, filter it if n ecessa ry , a n d th e n to re sa m p le th e re su ltin g a n a lo g signal at
th e d esired ra te (i.e., to p ass th e a n a lo g signal th ro u g h an A /D c o n v e rte r). T h e
se co n d m e th o d is to p e rfo rm th e sa m p lin g ra te c o n v ersio n e n tire ly in th e d igital
d o m ain .
O n e a p p a re n t a d v a n ta g e o f th e first m e th o d is th a t th e new sa m p lin g ra te
can b e a rb itra rily se le c te d a n d n e e d n o t h av e a n y special re la tio n s h ip to th e old
sa m p lin g ra te . A m a jo r d isa d v a n ta g e , h o w ev er, is th e signal d is to rtio n , in tro d u c e d
by th e D /A c o n v e rte r in th e signal re c o n stru c tio n , a n d by th e q u a n tiz a tio n effects
in th e A /D co n v e rsio n . S am p lin g ra te c o n v ersio n p e rfo rm e d in th e d igital d o m ain
av o id s this m a jo r d isa d v a n ta g e .
In th is c h a p te r w e d e sc rib e sa m p lin g r a te co n v ersio n a n d m u ltira te signal
p ro cessin g in th e d ig ital d o m a in . F irs t w e d esc rib e sa m p lin g r a te c o n v ersio n by a
ra tio n a l fa c to r a n d p re s e n t se v e ra l m e th o d s fo r im p le m e n tin g th e ra te c o n v e rte r, in
clu d in g sin g le-stag e a n d m u ltista g e im p le m e n ta tio n s. T h e n , w e d e sc rib e a m e th o d
fo r sa m p lin g ra te co n v e rsio n by an a rb itra ry fa c to r a n d discuss its im p le m e n ta tio n .
F in ally , w e p re s e n t se v era l a p p lic a tio n s o f sa m p lin g r a te c o n v e rsio n in m u ltira te
signal p ro cessin g sy stem s, w hich in c lu d e th e im p le m e n ta tio n o f n a rro w b a n d fil
ters, d igital filter b a n k s, a n d q u a d r a tu re m irro r filters. W e a lso discuss th e use o f
782

Sec. 10.1

783

Introduction

q u a d r a tu re m irro r filters in su b b a n d coding, tra n sm u ltip le x e rs, a n d finally o v ersam p lin g A /D an d D /A co n v e rte rs.

10.1 INTRODUCTION
T h e p ro c e ss o f sa m p lin g ra te co n v ersio n in th e d ig ital d o m a in can b e v iew ed as
a lin e a r filterin g o p e ra tio n , as illu stra te d in Fig. 10.1(a). T h e in p u t signal jr(n)
is c h a ra c te riz e d b y th e sa m p lin g ra te Fx = \ / T x a n d th e o u tp u t signal y ( m ) is
c h a ra c te riz e d by th e sa m p lin g ra te F v = 1/7V, w h e re Tx a n d Ty are th e c o r re
s p o n d in g sam p lin g in terv als. In th e m ain p a r t o f o u r tr e a tm e n t, th e ra tio Fy/ F x is
co n s tra in e d to b e ra tio n a l,

Fx =L
D

w h ere D a n d / a re relativ ely p rim e in teg ers. W e shall show th a t th e lin e a r filter
is c h a ra c te riz e d by a tim e -v a ria n t im pulse re sp o n se , d e n o te d as h ( n , m ) . H e n c e
th e in p u t x ( n ) an d th e o u tp u t v( m) a re re la te d by th e c o n v o lu tio n su m m a tio n fo r
tim e -v a ria n t system s.
T h e sam p lin g ra te co n v ersio n p ro cess can also b e u n d e rsto o d fro m th e p o in t
o f view o f d igital re sam p lin g o f th e sam e an a lo g signal. L et x ( i ) b e th e a n a
log signal th a t is sa m p le d at th e first ra te Fx to g e n e ra te x ( n) . T h e goal o f
ra te co n v ersio n is to o b ta in a n o th e r se q u e n c e y(m ) d ire c tly from jr(n). w hich
is eq u a l to th e sa m p le d v alu es o f x ( t ) at a se co n d ra te Fy. A s is d e p ic te d in
Fig. 10.1(b), y(m ) is a tim e-sh ifted version o f * (n ). S uch a tim e shift can be
x(n)

Linear filter
k(n, m)

Tx

y{i")
Fx mT
ky

(a)

(b)
Figure 10.1

Sampling rate conversion viewed as a linear filtering process.

7B4

Multirate Digital Signal Processing

Chap. 10

r e a liz e d by u sin g a lin e a r filter th a t h as a flat m a g n itu d e re sp o n s e a n d a lin e a r

p h a s e re sp o n s e (i.e., it h a s a fre q u e n c y re sp o n s e o f e~>u>r, w h e re r, is th e tim e
d e la y g e n e ra te d by th e filter). If th e tw o sa m p lin g r a te s a r e n o t e q u a l, th e r e
q u ire d a m o u n t o f tim e sh iftin g will vary fro m sa m p le to sa m p le , a s show n in
Fig. 10.1(b). T h u s th e ra te c o n v e rte r can b e im p le m e n te d u sin g a se t o f lin ear
filters th a t h av e th e sa m e flat m a g n itu d e re sp o n s e b u t g e n e r a te d iffe re n t tim e
d elays.
B e fo re co n sid e rin g th e g e n e ra l case o f sa m p lin g r a te co n v e rsio n , w e shall
c o n s id e r tw o sp ecial cases. O n e is th e case o f sa m p lin g ra te r e d u c tio n by an in te g e r
fa c to r D, a n d th e se co n d is th e case o f a sa m p lin g ra te in c re a se b y an in te g e r facto r
I . T h e p ro c e ss o f re d u c in g th e sa m p lin g ra te b y a fa c to r D (d o w n sa m p lin g by D)
is called deci mati on. T h e p ro c e ss o f in c re a sin g th e sa m p lin g r a te by an in te g e r
fa c to r I (u p sam p lin g b y I ) is ca lle d interpolation.

10.2 DECIMATION BY A FACTOR D

L e t us assu m e th a t th e signal x ( n ) w ith sp e c tru m X(co) is t o be d o w n sam p led
by a n in te g e r fa c to r D. T h e sp e c tru m X(a>) is assu m ed to b e n o n z e ro in th e
freq u e n cy in te rv a l 0 < )co\ < n o r, e q u iv a le n tly , \F\ < Fx /2. W e k n o w th a t if w e
re d u c e th e sam p lin g ra te sim p ly by se lectin g ev ery D th v alu e o f x ( n ) , th e resu ltin g
sig n al will b e an alia se d v ersio n o f jr(n), w ith a fo ld in g f re q u e n c y o f FX/ 2 D . T o
av o id aliasing, w e m u st first re d u c e th e b a n d w id th o f x ( n ) to Fmax = FX/ 2 D or,
eq u iv alen tly , to a>max = n j D . T h e n we m ay d o w n sam p le by D a n d th u s avoid
aliasing.
"Hie d e c im a tio n p ro c e ss is illu stra te d in Fig. 10.2. T h e in p u t s e q u e n c e x{n) is
p a s se d th ro u g h a lo w p ass filter, c h a ra c te riz e d by th e im p u lse re s p o n s e h ( n ) an d a
fre q u e n c y re sp o n se H D(a>), w hich id eally satisfies th e c o n d itio n
H d (o>) = ( J
10,

M * * ! D
o th erw ise

(10.2.1)

T h u s th e filter e lim in a te s th e sp e c tru m o f X (a>) in th e ra n g e n / D < w < n . O f

c o u rse, th e im p licatio n is th a t o n ly th e fre q u e n c y c o m p o n e n ts o f x i n ) in th e ran g e
M < tc/ D a re o f in te re st in f u rth e r p ro cessin g o f th e signal.
T h e o u tp u t o f th e filter is a s e q u e n c e v(n) given as
OC

v(n) = Y ^ ^ ) x ( n - k)
t=o
X(fl)

h(n)
1

i\

Figure 103.

i<n)

1!

Downsampler
W

D ecimation by a factor D.

(10.2.2)

yim)
Fx

Sec. 10.2

Decimation by a Factor

785

w hich is th e n d o w n sa m p le d by th e fa c to r D to p ro d u c e y ( m) . T h u s
y(m) = v ( m D )
(10.2.3)
=

h ( k ) x { m D k)

A lth o u g h th e filtering o p e ra tio n o n x ( n ) is iin e a r an d tim e in v a ria n t, th e

d o w n sam p lin g o p e r a tio n in c o m b in a tio n w ith th e filterin g resu lts in a tim e -v a ria n t
sy stem . T h is is easily verified. G iv en th e fact th a t x (n) p ro d u c e s y ( m) , w e n o te
th a t x ( n no) d o e s n o t im ply y( n no) u n less no is a m u ltip le o f D. C o n seq u en tly ,
th e o v e ra ll lin e a r o p e ra tio n (lin e a r filtering fo llo w ed by d o w n sam p lin g ) on x ( n ) is
n o t tim e in v arian t.
T h e fre q u e n c y -d o m a in c h a racteristics o f th e o u tp u t se q u e n c e y ( m ) can be
o b ta in e d b y re la tin g th e sp e c tru m o f y (m ) to th e s p e c tru m o f th e in p u t se q u e n c e
x ( n ) . F irs t, it is c o n v e n ie n t to define a se q u e n c e v(n) as
n = 0, D , 2 D , . . .
o th erw ise

(10.2.4)

C learly , v(n) can b e view ed as a se q u e n c e o b ta in e d by m u ltiplying v(n) w ith a

p e rio d ic tr a in o f im p u lses p (n ), w ith p e rio d D, as illu stra te d in Fig. 10.3. T h e
d isc re te F o u rie r se ries re p re s e n ta tio n o f p( n) is
(10.2.5)
H ence
v(n) = v ( n ) p ( n )

( 10.2 .6 )

y( m) = v (mD) = v ( m D ) p ( m D ) v( mD)

(10.2.7)

an d

n
-6

-5 -4

-3

- 2 (-1 0

3 4

Pin)

-6
Figure 1 0 3

D = 3.

-3

Multiplication o f v(n) with a periodic impulse train pin) with period

786

Multirate Digital Signal Processing

Chap. 10

N o w th e z -tra n sfo rm o f th e o u tp u t se q u e n c e y ( m ) is
OO

y(z) =

m=oc

oo
J2

v( r nD) z ~m

(10.2.8)

ms=oc

00
]T

Y(z) =

v ( m ) z - m/D

m=oc

w h ere th e last ste p follow s fro m th e fact th a t v ( m) = 0, e x c e p t a t m u ltip le s o f D.

B y m ak in g use o f th e re la tio n s in (10.2.5) a n d (10.2.6) in (10.2.8), w e o b ta in
1

Y( z ) =

__

v
m=cc

u ~ l

ejlxmk/D | -m/D

<t=0

j D1

oc

j2nk/D 1/D^-m

*=0 m=-oo

(10.2.9)

"* -0
= 7J H D ( e - i2* t /Dz l/D) X ( e - W > z l / D)
*=0
w h ere th e last ste p fo llow s fro m th e fact th a t V (z) = H q { z ) X ( z ).
B y e v a lu a tin g y ( 2) in th e u n it circle, w e o b ta in th e s p e c tru m o f th e o u tp u t
signal y{m) . S ince th e ra te o f y ( m ) is Fy = 1/TV, th e fre q u e n c y v a ria b le , w hich w e
d e n o te as w v, is in ra d ia n s a n d is re la tiv e to th e sa m p lin g r a te F v,
ti)y =

= 2 n F T s.

(10.2.10)

Fy
S ince th e sa m p lin g ra te s a re r e la te d by th e ex p ressio n
Fy = ^

(10.2.11)

it fo llow s th a t th e fre q u e n c y v a ria b le s wy a n d

2tr F
cox = - = 2 n F T x
Fx

(10.2.12)

a>v = D ojx

(10.2.13)

a re re la te d by

T h u s, as e x p ected , th e fre q u e n c y ra n g e 0 < \ojx \ < n / D is stre tc h e d in to th e

c o rre sp o n d in g fre q u e n c y ra n g e 0 < \a>y l < n by th e d o w n sam p lin g p ro cess.

Sec. 10.3

Interpolation by a Factor I

787

W e c o n clu d e th a t th e sp e c tru m Y (wv), w hich is o b ta in e d by e v a lu a tin g (10.2.9)

o n th e u n it circle, can b e e x p ressed as
(10.2.14)
W ith a p ro p e rly d esig n e d fitter H D(a)), th e aliasin g is e lim in a te d a n d , c o n seq u en tly ,
all b u t th e first te rm in (10.2.14) vanish. H e n c e

(10.2.15)

fo r 0 < |w v| < tt. T h e s p e c tra fo r th e se q u en ces x i n ) , v{n), a n d y ( m) a re illu stra te d

in Fig. 10.4.

10.3 INTERPOLATION BY A FACTOR I

A n in crease in th e sam p lin g rate by an in te g e r fa c to r o f I can be acco m p lish ed
b y in te rp o la tin g I 1 new sam ples b etw e e n successive v alu es o f th e signal. T h e
in te rp o la tio n p ro cess can b e acco m p lish ed in a v ariety o f w ays. W e sh all d escrib e
a p ro cess th a t p re se rv e s th e sp e c tra l sh a p e o f th e signal se q u e n c e x ( n ) .
L et u(m ) d e n o te a se q u e n c e w ith a ra te F v = I Fx , w hich is o b ta in e d fro m
x i n ) by a d d in g / - 1 z e ro s b e tw e e n successive v alu es o f x{n) . T h u s
(10.3.1)
a n d its sa m p lin g ra te is id en tical to th e ra te o f y{m) .
tra n sfo rm
V( z ) =

Y ,

T h is se q u e n c e has a z-

v^ z ~ m

m = oc
oc

(10.3.2)
= Xiz')
T h e c o rre sp o n d in g sp e c tru m of v ( m) is o b ta in e d by e v a lu a tin g (10.3.2) o n th e u nit
circle. T h u s
Vicoy) = X { w y I)

(10.3.3)

w h e re <uv d e n o te s th e fre q u e n c y v a ria b le re lativ e to th e n ew sa m p lin g ra te F , (i.e.,

o)y = 2 n F / F y ) . N o w th e re la tio n sh ip b e tw e e n sa m p lin g ra te s is F y = 1 Fx an d
h e n c e , th e fre q u e n c y v a ria b le s (oI a n d toy a re re la te d a c c o rd in g to th e fo rm u la

768

Multirate Digital Signal Processing

Chap. 10

l* K )l

I Via,)I

Figure 10.4 Spectra of signals in the

decimation of x( n ) by a factor D.

T h e s p e c tra X(a>x) a n d V(a>y) a re illu stra te d in Fig. 10.5. W e o b se rv e th a t the

sa m p lin g ra te in c re a se , o b ta in e d by th e a d d itio n o f / 1 z e ro sa m p le s b etw e e n
successiv e v alu es o f x ( n) , re su lts in a sig n a l w h o se sp e c tru m V(<wv) is an /-fo ld
p e rio d ic re p e titio n o f th e in p u t signal sp e c tru m X
S ince o n ly th e fre q u e n c y c o m p o n e n ts o f x (n) in th e ra n g e 0 < coy < n j l
a re u n iq u e , th e im ag es o f X ( oj) ab o v e coy = n f l sh o u ld b e r e je c te d b y passing
th e se q u e n c e v ( m ) th ro u g h a low pass filter w ith fre q u e n c y r e s p o n s e Hi(a>y) th a t

Sec. 10.3

Interpolation by a Factor /

789

l* K )l

Figure 10.5 Spectra of .run and run

where V(a>v) - X(wvl).

id eally has th e ch a ra c te ristic

Hl((i}y) =

0 < |a>v| < n / I

o th erw ise

C.
0.

(10.3.5)

w h e re C is a scale fa c to r re q u ire d to p ro p e rly n o rm a liz e th e o u tp u t se q u e n c e v(/7?).

C o n s e q u e n tly , th e o u tp u t sp e c tru m is
(10.3.6)
T h e scale fa c to r C is se le c te d so th a t th e o u tp u t y ( m ) = jc ( m/ 1 ) fo r m = 0,
/ , + 2 1 .........F o r m a th e m a tic a l co n v e n ie n c e , w e select th e p o in t m = 0. T h u s

y( 0) =

2~

.Jr_n
~
-

(10.3.7)

X {(I)vI)d(Dx

S in ce a>y = tox / I , (10.3.7) can be e x p re ss e d as

c i r
v(0) = /
I 2n

X ( w x) dwx
(10.3.8)

C
= -x (0 )
T h e re fo re , C = / is th e d esired n o rm a liz a tio n facto r.
F in ally , w e in d ic a te th a t th e o u tp u t se q u e n c e y ( m ) c a n b e ex p re sse d as a
c o n v o lu tio n o f th e s e q u e n c e v(n) w ith th e u n it sa m p le re sp o n s e h( n) o f th e low pass

790

Multirate Digital Signal Processing

Chap. 10

filter. T h u s
OC

y( m) =

Y 2 h( m k) v( k)

(10.3.9)

k = oc

S in ce v(k) = 0 e x cep t at m u ltip le s o f / , w h e re v ( k l ) = x ( k) , (10.3.9) b ec o m e s

y (m )=

OO
Y2 h{m -kl)x(k)

(10.3.10)

10.4 SAMPLING RATE CONVERSION BY A RATIONAL FACTOR U D

H a v in g discussed th e sp ecial cases o f d e c im a tio n (d o w n sa m p lin g by a fa c to r D )
a n d in te rp o la tio n (u p sa m p lin g by a fa c to r / ) , w e no w c o n s id e r th e g e n e ra l case
o f sam p lin g ra te co n v e rsio n by a ra tio n a l fa c to r I / D . B asically, w e can ach iev e
th is sa m p lin g ra te c o n v ersio n by first p e rfo rm in g in te rp o la tio n by th e fa c to r I a n d
th e n d e c im atin g th e o u tp u t o f th e in te r p o la to r by th e fa c to r D. In o th e r w o rd s, a
sa m p lin g ra te co n v ersio n by th e ra tio n a l fa c to r I ( D is ac c o m p lish e d by cascad in g
an in te rp o la to r w ith a d e c im a to r, as illu stra te d in Fig. 10.6.
W e em p h asize th a t th e im p o rta n c e o f p e rfo rm in g th e in te rp o la tio n first an d
th e d e c im atio n se co n d , is to p re se rv e th e d e sire d sp e c tra l c h a ra c te ristic s o f x(n).
F u rth e rm o re , w ith th e cascad e co n fig u ra tio n illu stra te d in Fig. 10.6, th e tw o filters
w ith im p u lse re sp o n se {/(/)} a n d {hd (l)\ a re o p e ra te d at th e sa m e r a te , n am ely I F X
an d h e n c e can b e c o m b in e d in to a single low pass filter w ith im p u lse re sp o n s e h(l)
as illu stra te d in Fig. 10.7. T h e fre q u e n c y re sp o n s e H( t ov) of th e co m b in e d filter
m u st in c o rp o ra te th e filtering o p e ra tio n s fo r b o th in te rp o la tio n an d d ecim atio n ,
a n d h e n c e it sh o u ld id eally p o ssess th e fre q u e n c y re sp o n s e c h a ra c te ristic
0 < \&v\ < m m ( n / D , n / 1 )
o th e rw ise

(10.4.1)

w h e re cut. = 2x F / F v = 2n F / I Fx = a)x/ / .

Rate = / F t

Figure 10.6

Rate = Fx = Fv
D

M ethod for sampling rate conversion by a factor f/D .

Sec. 10.4

x(n)
Rate = F,

Sampling Rate Conversion by a Rational Factor

Lowpass
filter
h(D

u(t)

Upsampler
T/

w([)

IID

Down sampler
4D

791

v(m)

Rate =

Rate = IFX = Fv
Figure 10.7

Method for sampling rate conversion by a factor 1/D.

In th e tim e d o m a in , th e o u tp u t o f th e u p sa m p le r is th e se q u e n c e
( ///) ,
u (0 = ( j (
[ 0.

/ = 0, / , 2 / , . . .
,
o th erw ise

(10.4.2)

a n d th e o u tp u t o f th e lin e a r tim e -in v a ria n t filter is

OO
w( l ) = ^
h(! k) v( k)
k=-oc

(10.4.3)

oc
h(l k l ) x ( k )

F inally, th e o u tp u t o f th e sa m p lin g ra te c o n v e rte r is th e se q u e n c e (y(m )}, w hich is

o b ta in e d b y d o w n sa m p lin g th e se q u e n c e {>(/)} by a fa c to r o f D. T h u s
v(m) = w{mD)
~
2_j h ( m D k l ) x ( k )
k
oc

(10.4.4)

It is illu m in a tin g to ex p ress (10.4.4) in a d iffe re n t fo rm by m a k in g a ch an g e

in v ariab le. L e t

mD

k =

(10.4.5)

w h e re th e n o ta tio n [ r j d e n o te s th e la rg e st in te g e r c o n ta in e d in r. W ith th is ch an g e
in v ariab le, (10.4.4) b ec o m e s
y( m) =

oc

h ( m D ----V
- ^ -

V- ^ _

~ 1
/

(10.4.6)

W e n o te th a t
mD

mD

I -

m o d u lo I

mD

= ( mD) !
C o n s e q u e n tly , (10.4.6) can b e e x p re sse d as

00
y(m )-

n=- oc

L)

h(~n I + (m D ) i ) x (

j -

~ n)

VL

(10.4.7)

792

Multirate Digital Signal Processing

Chap. 10

It is a p p a r e n t fro m th is fo rm th a t th e o u tp u t y ( m ) is o b ta in e d by p assin g th e
in p u t se q u e n c e x ( n ) th ro u g h a tim e -v a ria n t filter w ith im p u lse re sp o n s e
g(n,m) h(n l + (mD)i)

oo< m ,n< oo

(10.4.8)

w h e re h(k) is th e im p u lse re sp o n s e o f th e tim e -in v a ria n t lo w p ass filter o p e ra tin g

at th e sam p lin g r a te 1FX. W e fu rth e r o b se rv e , th a t fo r any in te g e r k,
g ( n , m -I- k l ) = h ( n l -I- ( m D -I- k D I ) , )
= h(nl + (mD),)

(10.4.9)

= g(n,m)
H e n c e g(n, m) is p e rio d ic in th e v a ria b le m w ith p e rio d I.
T h e fre q u e n c y -d o m a in re la tio n sh ip s can b e o b ta in e d by c o m b in in g th e resu lts
o f th e in te rp o la tio n a n d d e c im a tio n p ro cesses. T h u s th e s p e c tru m a t th e o u tp u t of
th e lin e a r filter w ith im p u lse re sp o n s e h(l) is
V (a) v) = H( ai v)X(a>vl )
I X ( w vI ) ,
0,

0 < IojJ < m in (7 r /0 , n / I )

o th e rw ise

(10.4.10)

T h e sp e ctru m o f th e o u tp u t se q u e n c e y (m ), o b ta in e d by d e c im a tin g th e se q u e n c e
v(n) by a fa c to r o f D , is

a o .4 .1 1 )

w h e re w v = D(ov. S in ce th e lin e a r filter p re v e n ts aliasin g as im p lie d by (10.4.10),

th e sp e c tru m o f th e o u tp u t se q u e n c e given by (10.4.11) re d u c e s to

o th e rw ise

10.5 FILTER DESIGN AND IMPLEMENTATION FOR SAMPLING-RATE

CONVERSION
A s in d ic a te d in th e discu ssion a b o v e , sa m p lin g ra te c o n v e rsio n b y a fa c to r I } D can
b e a c h iev ed by first in creasin g th e sa m p lin g r a te b y 1, ac c o m p lish e d by in se rtin g
7 - 1 z e ro s b e tw e e n su ccessive v alu es o f th e in p u t signal x( n ) , fo llo w e d by lin e a r
filterin g o f th e re su ltin g se q u e n c e to elim in a te th e u n w a n te d im a g e s o f X (co), an d
finally, by d o w n sam p lin g th e filte re d signal by th e fa c to r D. I n th is se c tio n we
c o n s id e r th e d esig n a n d im p le m e n ta tio n o f th e lin e a r filter.

Sec. 10.5

Filter Design and Implementation for Sampling-Rate Conversion

793

10.5.1 Direct-Form FIR Filter Structures

In p rin c ip le , th e sim p lest re a liz a tio n o f th e filter is th e d ire c t-fo rm F IR s tru c tu re
w ith sy stem fu n ctio n
M -l

H{z ) = Y b ( k ] z ~k
t =o

(10-5 -1)

w h e re {/j {/:)} is th e u n it sam ple re sp o n se o f th e F IR filter. T h e low pass filter

c an b e d e sig n e d to h av e lin e a r p h a s e , a specified p a s sb a n d rip p le a n d sto p b a n d
a tte n u a tio n . A n y o f th e sta n d a rd , w ell k n o w n F IR filter design te c h n iq u e s (e.g.,
w in d o w m e th o d , freq u e n cy sa m p lin g m e th o d ) can be u se d to c a rry o u t this design.
T h u s w e w ill h av e th e filter p a ra m e te rs {/j ()}, w hich allo w us to im p le m e n t th e
F IR filter d ire c tly as sh ow n in Fig. 10.8.
A lth o u g h th e d ire c t-fo rm F IR filter re a liz a tio n illu stra te d in Fig. 10.8 is sim
p le, it is also v ery in efficient. T h e inefficiency resu lts fro m th e fact th a t th e u p sa m p lin g p ro cess in tro d u c e s / 1 ze ro s b e tw e e n successive p o in ts o f th e in p u t
sig n al. If I is larg e, m o st o f th e signal c o m p o n e n ts in th e F IR filter are zero. C o n
se q u e n tly , m o st o f th e m u ltip lic atio n s an d a d d itio n s re su lt in zero s. F u rth e rm o re ,
th e d o w n sam p lin g p ro cess at th e o u tp u t o f th e filter im p lies th a t only on e o u t o f

Figure 10.8
factor I/D.

Direct-form realization of FIR filter in sampling rate conversion by

794

Mullirate Digital Signal Processing

Chap. 10

ev ery D o u tp u t sa m p le s is re q u ire d a t th e o u tp u t o f th e filter. C o n s e q u e n tly , only

o n e o u t o f ev ery D p o ssib le v a lu e s a t th e o u tp u t o f th e filter sh o u ld be co m p u te d .
T o d ev elo p a m o re efficien t filter s tru c tu re , let us b eg in w ith a d e c im a to r th a t
re d u c e s th e sa m p lin g ra te by an in te g e r fa c to r D. F ro m o u r p re v io u s discussion,
th e d e c im a to r is o b ta in e d by p assin g th e in p u t se q u e n c e x ( n ) th r o u g h an F IR filter
a n d th e n d o w n sam p lin g th e filter o u tp u t by a fa c to r >, as illu stra te d in Fig. 10.9a.
In th is co n fig u ratio n , th e filter is o p e ra tin g a t th e high sa m p lin g ra te Fx, w hile
o n ly o n e o u t o f ev ery D o u tp u t sa m p le s is ac tu a lly n e e d e d . T h e logical so lu tio n
to th is inefficiency p ro b le m is to e m b e d th e d o w n sam p lin g o p e ra tio n w ith in th e
filter, as illu stra te d in th e filter re a liz a tio n given in Fig. 10.9b. In this filter stru c
tu re , all th e m u ltip lic a tio n s an d a d d itio n s a re p e rfo rm e d a t th e lo w er sam p lin g
ra te Fx / D. T h u s w e h av e ach ie v e d th e d e s ire d efficiency. A d d itio n a l re d u c tio n in
c o m p u ta tio n can b e ach ie v e d by e x p lo itin g th e sy m m etry c h a ra c te ristic s of
F ig u re 10.10 illu stra te s an efficient re a liz a tio n o f th e d e c im a to r in w hich th e F IR
filter h as lin ear p h a s e , a n d h e n c e {/i{&)} is sym m etric.
N ex t, let us co n sid e r th e efficien t im p le m e n ta tio n o f an in te rp o la to r, w hich
is realized by first in se rtin g I l z e ro s b e tw e e n sa m p le s o f x ( n ) a n d th e n filtering
th e re su ltin g se q u en ce. T h e d ire c t-fo rm re a liz a tio n is illu stra te d in Fig. 10.11. T h e
m a jo r p ro b lem w ith th is s tru c tu re is th a t th e filter c o m p u ta tio n s a re p e rfo rm e d at
th e hig h sam p lin g ra te 1FX. T h e d e s ire d sim p lificatio n is a c h ie v e d by first using th e
tra n sp o s e d fo rm o f th e F IR filter, as illu stra te d in Fig. 10.12a, a n d th e n e m b ed d in g
th e u p sa m p le r w ithin th e filter, as show n in Fig. 10.12b. T h u s, all th e filter m u ltip li
c a tio n s a re p e rfo rm e d at th e low ra te Fx, w hile th e u p sa m p lin g p ro c e ss in tro d u c e s
1 1 ze ro s in ea c h o f th e filte r b ra n c h e s o f th e stru c tu re show n in Fig. 10.12b. T h e
r e a d e r can easily verify th a t th e tw o filteT s tru c tu re s in Fig. 10.12 a re e q u iv a le n t.
It is in te re stin g to n o te th a t th e s tru c tu re o f th e in te rp o la to r, sh o w n in
Fig. 10.12b, can b e obtained by tra n sp o s in g th e stru c tu re o f th e d e c im a to r sh o w n
in Fig. 10.9. W e o b se rv e th a t th e tra n s p o s e o f a d e c im a to r is a n in te rp o la to r, an d
vice v ersa. T h ese re la tio n sh ip s a re illu stra te d in Fig. 10.13, w h e re (b ) is obtained
by tra n sp o sin g (a ) a n d (d) is obtained by tra n sp o s in g (c). C o n s e q u e n tly , a d e c i
m a to r is th e d u al o f an in te rp o la to r, a n d vice v ersa. F ro m th e s e re la tio n sh ip s, it
fo llow s th a t th e re is an in te r p o la to r w h o se s tru c tu re is th e d u a l o f th e decimatoT
sh o w n in Fig, 10.10, w hich ex p lo its th e sy m m etry in h(n).

10.5.2 Polyphase Filter Structures

T h e c o m p u ta tio n a l efficiency o f th e filter s tru c tu re show n in Fig. 10.12 can also
b e ach iev ed by re d u c in g th e larg e F IR filter o f le n g th M in to a se t o f sm a ller
filters o f len g th K = M / 1 , w h ere M is se le c te d to b e a m u ltip le o f I . T o d e m o n
s tr a te th is p o in t, le t u s c o n s id e r th e in te r p o la to r given in F ig. 10.11. S ince th e
u p sa m p lin g p ro cess in se rts 1 1 z e ro s b e tw e e n successive v a lu e s o f x( n) , o n ly K
o u t o f th e M in p u t v alu es sto re d in th e F IR filter a t an y o n e tim e a re n o n z e ro .
A t o n e tim e in sta n t, th e s e n o n z e ro v a lu e s co in cid e a n d a re m u ltip lie d by th e fil
te r co efficien ts h{0), h ( I ) , h ( 2 l ) , . . . , h ( M I ) . In th e fo llo w in g tim e in sta n t, th e

Sec. 10.5

Filter Design and Implementation for Sampling-Rate Conversion

(a)

Figure 10.9

Decimation by a factor D.

795

Multirate Digital Signal Processing

Chap. 10

Sec. 10.5

Rlter Design and Implementation for Sampling-Rate Conversion

797

JT(n)

h(M - 2)

MM-

Figure 10.11 Direct-form realization of

FIR filter in interpolation by a factor /,

1)

n o n z e ro v a lu e s o f th e in p u t se q u e n c e co in cid e a n d a re m u ltip lie d by th e filter co

efficien ts /?(!). h ( I + 1), h ( 2 I + 1)___ h ( M - 7 + 1), a n d so o n . T h is o b se rv a tio n
le a d s us to d efin e a se t o f sm a ller filters, called p o ly p h a s e filters, w ith u n it sa m p le
re sp o n se s
p t ( n ) = h( k + n ! )

k = 0 ,1 ,..., 7 1
(10.5.2)
n = 0,

~ 1

w h e re K = A //7 is an in teg er.

F ro m th is d iscussion it follow s th a t th e se t o f 7 p o ly p h a se filters can b e
a rra n g e d as a p a ra lle l re a liz a tio n , a n d th e o u tp u t o f e a c h filter can be se le c te d
b y a c o m m u ta to r as illu stra te d in Fig. 10.14. T h e ro ta tio n o f th e c o m m u ta to r is
in th e co u n te rc lo c k w ise d ire c tio n b e g in n in g w ith th e p o in t a t m ~ 0. T h u s, th e
p o ly p h a s e filters p e rfo rm th e c o m p u ta tio n s a t th e low sa m p lin g ra te Fx , a n d th e
r a te c o n v e rsio n resu lts fro m th e fact th a t 7 o u tp u t sa m p le s a re g e n e ra te d , o n e
fro m each o f th e filters, fo r each in p u t sa m p le .
T h e d e c o m p o s itio n o f {/r{)} in to th e se t o f 7 su b filters w ith im p u lse re sp o n se
Pk(n), k = 0, 1 , . . . , 7 1, is c o n siste n t w ith o u r p re v io u s o b s e rv a tio n th a t th e in p u t
sig n a l w as b e in g filte re d by a p erio d ically tim e -v a ria n t lin e a r filter w ith im p u lse
re sp o n se
g ( n , m ) = h ( n l + ( mD) [ )
(10.5.3)
w h e re D = 1 in th e c a se o f th e in te rp o la to r. W e n o te d p rev io u sly th a t g( n, m)
v a rie s p e rio d ic a lly w ith p e rio d 7. C o n se q u e n tly , a d iffe re n t se t o f coefficients a re
u se d to g e n e ra te th e se t o f 7 o u tp u t sa m p le s y ( m ) , m = 0, 1 , . . . , I - 1.
A d d itio n a l in sig h t can b e g ain e d a b o u t th e c h a ra c te ristic s o f th e se t o f p o ly
p h a s e su b filte rs b y n o tin g th a t pk(n) is o b ta in e d fro m h( n ) by d e c im a tio n w ith a
f a c to r 7. C o n s e q u e n tly , if th e o rig in a l filter fre q u e n c y re s p o n s e H {co) is flat o v e r
th e ra n g e 0 < M < c o/ I , e a c h o f th e p o ly p h a se su b filters p o ssesses a re lativ ely flat

(b)
Figure 10.12

Efficient realization of an interpolator.

Sec. 10.5

Filter Design and Implementation for Sampling-Rate Conversion

799

<b)

(a)

(O
(d)
Figure 10.13 Duality relationships obtainedthroughtransposition.

Rate - Fr - IFX

Pi- i<")
Rate=Fx
Figure 10.14

Rate=F,
Interpolation byuse of polyphase filters.

re sp o n s e o v e r th e ra n g e 0 < |a>| < n (i.e., th e p o ly p h a s e su b filters a re b asically

all-p ass filters a n d d iffer p rim a rily in th e ir p h a se c h a ra c te ristic s). T h is ex p lain s th e
re a so n fo r th e te rm p o ly p h a s e in d e sc rib in g .th e s e filters.
T h e p o ly p h a se filter can also b e view ed as a se t o f I su b filters c o n n e c te d to a
co m m o n d e la y line. Id eally, th e fcth su b filter will g e n e ra te a fo rw a rd tim e sh ift o f
(.k / I ) T x, fo r k = 0, 1. 2 ........./ 1, re la tiv e to th e z e ro th subfilter. T h e re fo re , if th e
z e ro th filter g e n e ra te s z e ro d elay , th e fre q u e n c y re sp o n s e o f th e Jtth su b filter is

pk((o) = e)wk/I
A tim e sh ift o f an in te g e r n u m b e r o f in p u t sa m p lin g in te rv a ls (e.g., ITX) can b e
g e n e ra te d by sh iftin g th e in p u t d a ta in th e d e la y line by I sa m p le s an d using th e
sa m e su b filters. B y c o m b in in g th e s e tw o m e th o d s, w e c a n g e n e ra te an o u tp u t th a t
is sh ifted fo rw a rd b y an a m o u n t (/ + i / I ) T x re la tiv e to th e p re v io u s o u tp u t.
B y tra n sp o s in g th e in te rp o la to r stru c tu re in Fig. 10.14, w e o b ta in a c o m m u
ta to r stru c tu re fo r a d e c im a to r b a s e d o n th e p a ra lle l b a n k o f p o ly p h a se filters, as
illu stra te d in Fig. 10.15. T h e u n it sa m p le re sp o n s e s o f th e p o ly p h ase filters a re

800

Multirate Digital Signal Processing

Figure 10.15

Chap. 10

Decimation by use of polyphase filters.

n o w d efin ed as
Pk{n) = h( k + n D )

k = 0, 1____ O - l
(10.5.4)
n = 0 , 1 .........K - l

w h e re K = M / D is an in te g e r w h en M is se le c te d to be a m u ltip le o f D. T h e
c o m m u ta to r ro ta te s in a c o u n terclo ck w ise d ire c tio n sta rtin g w ith th e filter po(n) at

0.

A lth o u g h th e tw o c o m m u ta to r s tru c tu re s fo r th e in te r p o la to r a n d th e d e c i
m a to r ju st d escrib ed r o ta te in a c o u n te rc lo c k w ise d ire c tio n , it is also p o ssib le to
d e riv e an e q u iv a le n t p a ir o f c o m m u ta to r s tru c tu re s hav in g a clockw ise ro ta tio n .
In th is a lte rn a tiv e fo rm u la tio n , th e se ts o f p o ly p h a se filters a re d efin ed to h ave
im p u lse resp o n ses
P k ( n ) = h ( n l k)

k = 0 ,1 ,...,/- 1

(10.5.5)

Pk( n ) = h { n D - k )

* = 0 , 1 .........D - 1

(10.5.6)

f o r th e in te rp o la to r a n d d e c im a to r, resp ectiv ely .

10.5.3 Time-Variant Filter Structures

H av in g d esc rib e d th e filter im p le m e n ta tio n fo r a d e c im a to r a n d an in te rp o la to r,
let us no w c o n sid e r th e g e n e ra l p ro b le m o f sa m p lin g ra te c o n v e rsio n by th e fa c to r
I / D . In th e g e n e ra l case o f sa m p lin g r a te c o n v e rsio n by a fa c to r I / D , th e filtering
can b e acco m p lish ed by m e a n s o f th e lin e a r tim e -v a ria n t filte r d e s c rib e d by th e
re sp o n se fu n ctio n
g( n, m) = h ( n l (m D )/)
(10.5.7)
w h ere h(n) is th e im p u lse re sp o n s e o f th e low -pass F IR filter, w hich id eally , has
th e fre q u e n c y re sp o n s e specified by (10.4.1). F o r c o n v e n ie n c e w e se lect th e length
o f th e F I R filter {/i(n)} to a m u ltip le o f I (i.e., M = K I ) . A s a c o n s e q u e n c e , th e
s e t o f coefficien ts {#(, m)} fo r e a c h m = 0 , 1, 2
, 1, c o n ta in s K elem en ts.

Sec. 10.5

Filter Design and Implementation for Sampling-Rate Conversion

801

S in ce g( n, m ) is also p e rio d ic w ith p e rio d I , a s d e m o n s tr a te d in (10.4.9), it follow s

th a t th e o u tp u t y { m) can b e e x p re ss e d as

y(m ) = X I#
n0

- [_ y j l ) x

-n )
\L

(10.5.8)

C o n c e p tu a lly , w e can th in k o f p e rfo rm in g th e c o m p u ta tio n s specified by

(10.5.8) b y p ro cessin g blocks o f d a ta o f le n g th AT b y a se t o f K filter coefficients
g ( n , m j_ m / l \ I ) , n = 0 , 1 , . . . , K 1. T h e re a re I such se ts o f coefficients, o n e
se t fo r e a c h b lo ck o f I o u tp u t p o in ts o f y{m) . F o r e a c h b lock o f 1 o u tp u t p o in ts,
th e r e is a c o rre sp o n d in g b lo ck o f D in p u t p o in ts o f x ( n ) th a t e n te r in th e c o m p u
ta tio n .
T h e b lo ck p ro cessin g a lg o rith m fo r co m p u tin g (10.5.8) can b e visualized as
illu stra te d in Fig. 10.16. A b lo ck o f D in p u t sa m p le s is b u ffe re d a n d sh ifted in to
a se co n d b u ffe r o f len g th K , o n e sa m p le a t a tim e. T h e sh iftin g fro m th e in p u t
b u ffe r to th e se co n d b u ffe r o ccu rs a t a ra te o f o n e sa m p le ea c h tim e th e q u a n tity
\ m D / I J in c re a se s by o n e. F o r ea c h o u tp u t sa m p le y( f ) , th e sa m p le s fro m th e
se co n d b u ffe r a re m u ltip lie d by th e c o rre sp o n d in g se t o f filter coefficients g( n, I)
fo r n = 0, 1 , ___ K 1, an d th e K p ro d u c ts a re a c c u m u la te d to give y (/), fo r 1 = 0,
x(n)

Coefficient storage

Figure 10.16
cessing.

Efficient implementation of sampling-rate conversion by block pro

Multirate Digital Signal Processing

802

Chap. 10

1
, 1. T h u s th is c o m p u ta tio n p ro d u c e s / o u tp u ts. It is th e n re p e a te d fo r a
new se t o f D in p u t sam p les, a n d so on.
A n a lte rn a tiv e m e th o d fo r co m p u tin g th e o u tp u t o f th e sa m p le ra te c o n v e rte r,
sp ecified by (10.5.8), is b y m e a n s o f an F IR filter stru c tu re w ith p e rio d ic a lly varying
filter coefficients. Such a stru c tu re is illu stra te d in Fig. 10.17. T h e in p u t sam ples
x{n) a re p a sse d in to a sh ift re g iste r th a t o p e ra te s a t th e sa m p lin g ra te Fx an d is of
le n g th K = M / 1 , w h e re M is th e le n g th o f th e tim e -in v a ria n t F I R filter specified by
th e fre q u e n c y re sp o n s e g iv en by (10.4.1). E a c h sta g e o f th e re g is te r is c o n n e c te d to
a h o ld -a n d -sa m p le dev ice th a t se rv es to co u p le th e in p u t sa m p le ra te Fx to th e o u t
p u t sa m p le r a te Fy = (I / D ) F X. T h e sa m p le at th e in p u t to ea c h h o ld -a n d -sa m p le
d ev ice is h eld u n til th e n e x t in p u t sa m p le arriv es a n d th e n is d isc a rd e d . T h e o u tp u t
sa m p les o f th e h o ld -a n d -sa m p le d ev ice a re ta k e n a t tim es m D / l , m = 0, 1, 2 ........
W h e n b o th th e in p u t a n d o u tp u t sa m p lin g tim e s co in cid e (i.e.. w h en m D / I is an
in te g e r), th e in p u t to th e h o ld -a n d -sa m p le is ch an g e d first a n d th e n th e o u tp u t
sa m p le s th e n ew in p u t. T h e K o u tp u ts fro m th e K h o ld -a n d -s a m p le devices a re
m u ltip lie d by th e p e rio d ic a lly tim e-v a ry in g co effic ie n ts g( n, m - \ m / I \ l ) , fo r n = 0,
1 , . . . , K - 1, an d th e re su ltin g p ro d u c ts a re su m m e d to yield y(m ). T h e c o m p u
ta tio n s a t th e o u tp u t o f th e h o ld -a n d -sa m p le d ev ices are r e p e a te d a t th e o u tp u t
sa m p lin g ra te o f F v = ( I / D ) F X.
F inally, ra te co n v ersio n by a ra tio n a l fa c to r 1 / D can also be p e rfo rm e d by
use o f a p o ly p h ase filter h av in g 1 subfilters. If w e assu m e th a t th e m th sam p le
n(0.l).l = 0. I......./ - I

Figure 10.17

Efficient realization of sampling-rate conversion by a factor //>.

Sec. 10.5

Fitter Design and Implementation for Sampling-Rate Conversion

803

y ( m ) is com p uted by taking the output o f the imth subfilter with input data x( n ) ,
x ( n - 1 ) , . . . , x ( n K + 1 ) , in the delay line, the n ext sam p le y ( m + 1) is taken from
the (i + i)st subfilter after shifting lm+j new sam ples in the d elay lin es w here i m+i =
(i + D )mod / and l m+1 is the in teger part o f (im + D ) / I . T h e in teger im+i should be
saved to b e used in determ ining the subfilter from w hich the n ext sam p le is taken.
L et us now dem onstrate the filter design procedure, first in the design o f a
d ecim ator, secon d in th e design o f an interpolator, and finally, in th e design o f a
rational sam ple-rate converter.

Example 10.5.1
Design a decim ator that downsamples an input signal x(n) by a factor D = 2. Use
the Rem ez algorithm to determ ine the coefficients of the F IR filter that has a 0.1-dB
ripple in the passband and is down by at least 30 dB in the stopband. Also determ ine
the polyphase filter structure in a decim ator realization that employs polyphase filters.

Solution A filter of length M = 30 achieves the design specifications given above.

The impulse response of the FIR filter is given in Table 10.1 and the frequency
response is illustrated in Fig. 10.18. Note that the cutoff frequency is wc = jr/2.
The polyphase filters obtained from h{n) have impulse responses
p*(n) = h(2n + k)

k = 0,1;

n = 0 ,1 , ,.. ,1 4

N ote that po(n) = h(2n) and p\{tt) = h(2n + 1). Hence one filter consists of the evennum bered samples o f h(n) and the other filter consists of the odd-num bered samples
of h(n).

Example 10.5.2
Design an interpolator that increases the input sampling rate by a factor of I = 5. Use
the R em ez algorithm to determ ine the coefficients of the F IR filter that has 0.1-dB
ripple in the passband and is down by at least 30 dB in the stopband. Also, determ ine
the polyphase filter structure in an interpolator realization based on polyphase filters.

Solution A filter o f length M = 30 achieves the design specifications given above.

The frequency response of the F IR filter is illustrated in Fig. 10.19 and its coefficients
are given in Table 10.2. The cutoff frequency-is wc = n/5.
The polyphase filters obtained from h(n) have impulse responses
pk{n) = h(Sn + jt)

* = 0 ,1 ,2 ,3 ,4

Consequently, each filter has length 6.

Example 10.53
Design a sam ple-rate converter that increases the sampling rate by a factor 2.5. Use
the Rem ez algorithm to determ ine the coefficients of the F IR filter that has 0.1-dB
ripple in the passband and is down by at least 30 dB in the stopband. Specify the
sets of time-varying coefficients g( n, m) used in the realization of the sampling-rate
converter according to the structure in Fig. 10.17.

Solution The F IR filter that m eets the specifications of this problem is exactly the
same as the filter designed in Example 10.5.2. Its bandwidth is tt/5.

Muttirate Digital Signal Processing

804

Chap. 10

TABLE 10.1 COEFFICIENTS OF LINEAR-PHASE FIR FILTER IN

EXAMPLE 10.5.1
FINITE

IMPULSE RESPONSE

(FIR)

LINEAR-PHASE DIGITAL FILTER DESIGN

REMEZ EXCHANGE ALGORITHM
FILTER LENGTH

H { 1)
H ( 2)
H ( 3)
H( 4)
H(
H(

5)
6)
H ( 7)

H(
H(

8)

9)
H (10)
H (11)
H (12 )
H(13)
H (14)
H (15)

=30

IMPULSE RESPONSE
0 .6 0 2 5 6 1 6 5 E - 0 2 = H ( 30)
= -0 .1 2 8 1 7 1 4 3 E - 0 1 = H ( 29)
-0..2 8 5 8 2 0 6 6 E - 0 2 = HI 28)
=
0..1 3 6 6 3 3 4 6 E - 0 1
H ( 27)
= -0,,4 6 6 8 8 9 6 1 E - 0 2 = H ( 26)
= -0,.1 9 7 0 4 4 1 5 E - 0 1 = H ( 25)
=
0,.1 5 9 8 4 6 2 3 E - 0 1 = H ( 24)
=

=
0 .21384886E-01
= -0..3 4 9 7 9 4 4 0 E - 0 1
= -0 . 1 5 6 1 5 5 2 2 E - 0 1
=
0..6 4 0 0 6 1 1 3 E - 0 1
= - 0 .7 3 4 5 1 7 7 2 E - 0 2
= -0. 1 1 8 7 3 1 8 5 E + 0 0
=
0. 9 8 0 4 7 8 4 5 E - 0 1
0. 4 9 2 2 5 0 6 8 E + 0 0
=
BAND

LOWER BAND EDGE

UPPER BAND EDGE
DESIRED VALUE
WEIGHTING
DEVIATION
DEVIA T I O N IN DB

= H (
= H(
= H (
= H(
= H (
H(
= H{
= H(

23)
22)
21)
20)
19)
18)
17)
16)
BAND 2

0.0000000
1.0000000
2.0000000

0.. 3 1 0 0 0 0 0
0.. 5 0 0 0 0 0 0
0.. 0 0 0 0 0 0 0
1.. 0 0 0 0 0 0 0

0.0107151
0.0925753

0.. 0 2 1 4 3 0 2
33. 3 7 9 4 7 4 6

0.2500000

EXTREMAL FREQUENCIES-MAXIMA OF THE ERROR CURVE

0.0000000
0.0416667
0.0791667
0.1166666
0.1854166
0.2145832
0.2395832
0.2500000

0.1520833
0.3100000

0.3225000

0.4474999

0.3495833

0.3808333

0.4141666

0.4829165

The coefficients of the filter are given by (10.4.8) as

g( n, m) ~ h(rtl + (mD)/)
= h ^ nill + mD

mDA
I

By substituting 1 = 5 and D =
= 2, we obtain
g(n,
x,m)
m) = h {^5
5n + 2 m - 5

2m \

y\)

By evaluating g(n, m) for n = 0, 1 , . . . . 5 and m = 0 , 1 , . . . , 4 we obtain the following

Sec. 10.5

Filter Design and Implementation for Sampling-Rate Conversion

805

Figure 10.18 Magnitude response o f linear-phase FIR filter of length M = 30 in

Example 10.5.1.

Figure 10.19 Magnitude response o f linear-phase FIR filter of length M = 30 in

Example 10.5.2.

coefficients for the time-variant filter:

*(0,m ) = {*(0)
* (l,m ) = {*(5)
g ( 2 , m ) = {*(10)
g ( 3 , m ) = {*(15)
g ( 4 , m ) = {*(20)
g ( 5 , m ) = {*(25)

*(2)
*(7)
*(12)
*(17)
*(22)
*(27)

h{ 4)
*( 9)
*(14)
*(19)
*(24)
*(29)

*(1)
*( 6)
*(11)
*(16)
*(21)
*(26)

*(3)}
A(8)J
*(13)}
*(18)}
*(23)}
*(28)}

A polyphase filter im plementation would employ five subfilters, each of length

six. To decim ate the output of the polyphase filters by a factor of D = 2 simply means

806

Multirate Digital Signal Processing

Chap. 10

COEFFICIENTS OF LINEAR-PHASE FIR FILTER IN

EXAMPLE 10.5.2

TABLE 10.2

F I N I T E I M P U L S E R E S P O N S E (FIR)
LINEAR-PHASE DIGITAL FILTER DESIGN
REMEZ EXCHANGE ALGORITHM
FILTER LENGTH = 3 0

H ( 1)
H ( 2)
H< 3)
H ( 4)
H ( 5)
H( 6)
H ( 7)
H< 8)
H( 9)
H (10)
H {11)
H (12)
H (13)
H (14)
H(15)

IMPULSE RESPONSE
0 .6 3 9 8 7 2 1 6 E - 0 2 = H (
=
= -0..1 4 7 6 1 3 0 4 E - 0 1 = H (
= -0..1 0 8 8 6 5 7 7 E - 0 2 = H (
= -0..2 8 7 1 4 9 5 7 E - 0 2 = H {
0 .1 0 4 8 6 4 3 0 E - 0 1 = H<
0 .2 1 4 7 7 1 4 2 E - 0 1 = H (
=
0,.1 9 4 7 9 3 6 2 E - 0 1
=
H(
= -0,.3 1 0 6 7 4 3 1 E - 0 3
= - 0 ,.3 0 0 5 3 0 3 3 E - 0 1
= - 0 .4 9 8 7 7 0 2 9 E - 0 1
= - 0 .3 7 3 7 1 2 8 5 E - 0 1
=
0. 1 8 4 8 2 8 9 6 E - 0 1
=
0. 1 0 7 4 7 1 4 1 E + 0 0
0..1 9 9 5 1 0 9 8 E + 0 0
0 .2 5 7 9 4 8 2 8 E + 0 0
=

LOWER BAND EDGE

UPPER BAND EDGE
DESIRED VALUE
WEIGHTING
DEVIATION
DEVIATION

:I N D B

BAND 1
0.0000000
0.1000000
1.0000000
3.0000000
0.0097524
0.0842978

= H(
= H (
= H (
= H {
= H(
= H {
= H (

30!
29)
28)
27)
26)
25)
24)
23)
22)
21)
20)
19)
18)
17)

= H ( 16)
BAND 2
0 . 1600000
0. 5 0 0 0 0 0 0
0., 0 0 0 0 0 0 0
1 .0000000
0,.0 2 9 2 5 7 2
30. 6 7 5 3 3 4 9

EXTREMAL FREQUENCIES-MAXIMA OF THE ERROR CURVE

0.0000000
0.1600000

0.0333333
0.1745833

0.0645834
0.2016666

0.0895833
0.2370833

0.1000000
0.2704166

0.3058332
0 .4829164

0.3412498

0.3766665

0.4120831

0.4474997

that we take every other output from the polyphase filters. Thus the first output v(0)
is taken from pQ(n), the second output y (l) is taken from P2(n), the third from p*(n),
the fourth from pi(n), the fifth from p-i(n). and so on.

10.6 MULTISTAGE IMPLEMENTATION OF SAMPLING-RATE

CONVERSION
In practical applications o f sam pling-rate con version w e often en cou n ter decim a
tion factors and in terp olation factors that are m uch larger than unity. For exam
ple, su p p ose that w e are given the task o f altering th e sam pling rate by the factor

Sec. 10.6

Multistage Implementation of Sampling-Rate Conversion

807

I / D = 130/63. A lth o u g h , in th e o ry , th is ra te a lte ra tio n can b e a c h iev ed exactly,

th e im p le m e n ta tio n w o u ld re q u ire a b a n k o f 130 p o ly p h a se filters a n d m ay b e
c o m p u ta tio n a lly in efficien t. In th is section w e c o n s id e r m e th o d s f o r p e rfo rm in g
sa m p lin g -ra te c o n v e rsio n fo r e ith e r D 1 a n d /o r 7 > 1 in m u ltip le stages.
F irst, let us c o n s id e r in te rp o la tio n by a fa c to r 7 1 a n d le t u s a ssu m e th a t
7 can b e fa c to re d in to a p r o d u c t o f p o sitiv e in te g e rs as

'-fl*

( 10 . 6 . 1)

T h e n , in te rp o la tio n by a fa c to r 7 c a n b e a c c o m p lish e d b y cascad in g L sta g es o f

in te rp o la tio n a n d filterin g , as sh o w n in Fig. 10.20. N o te th a t th e filter in ea c h o f
th e in te rp o la to rs e lim in a te s th e im ag es in tro d u c e d by th e u p sa m p lin g p ro cess in
th e c o rre sp o n d in g in te rp o la to r.
In a sim ilar m a n n e r, d e c im a tio n by a fa c to r D, w h e re D m ay b e fa c to re d in to
a p ro d u c t o f p o sitiv e in te g e rs as

( 10.6 .2)

D = \\D .

can be im p le m e n te d as a cascad e o f J stag es o f filtering an d d e c im atio n as illu s

tr a te d in Fig. 10.21. T h u s th e sa m p lin g ra te a t th e o u tp u t o f th e ith sta g e is
F, =

Fj-i
D,

i = 1 ,2 ........ J

(10.6.3)

w h ere th e in p u t ra te fo r th e se q u e n c e {*()] is F0 = Fx .
T o e n s u re th a t n o aliasing o ccu rs in th e o v erall d e c im a tio n p ro cess, w e can
d esig n each filter stag e to avoid aliasin g w ithin th e fre q u e n c y b a n d o f in te re st. T o

*(n)
F,
Stage 1
Figure 10.20

Stage 1

Stage 2

h h Fi

Stage L

Multistage implementation of interpolation by a factor /,

Stage 2

Fx

Staged

D\Dt
Figure 10.21

Multistage implementation of decimation by a factor D.

808

Multirate Digital Signal Processing

Chap. 10

e la b o ra te , let us d efin e th e d e s ire d p a s sb a n d an d th e tra n sitio n b a n d in th e overall

d e c im a to r as
P assb an d : 0 < F <
(10.6.4)
T ra n s itio n b a n d : Fpc 2; F < Fsc
w h e re Fx < FX/ 2 D . T h e n , aliasing in th e b a n d 0 < F < Fx is a v o id e d by selectin g
th e fre q u e n c y b a n d s o f ea c h filter sta g e a s follow s:
P a ssb a n d : 0 < F < F ^
T ra n s itio n b an d : Fpc < F < Fj - Fx

^^

F ,
S to p b a n d : Fj - Fx < F < ~ F o r ex am p le, in th e first filter sta g e we h av e F\ = Fx / D \ . a n d th e filter is
d esig n ed to h av e th e fo llow ing fre q u e n c y bands:
P assb an d : 0 < F < F ^
T ra n s itio n b an d : F ^ < F < Fy - Fsz

(10 6 6)

fT
S to p b a n d : F\ - Fx < F < - y
A f te r d e c im atio n b y D\ , th e re is aliasing from th e signal c o m p o n e n ts th a t fall in
th e filter tra n sitio n b a n d , b u t th e aliasin g o ccu rs at fre q u e n c ie s ab o v e Fsc. T h u s
th e re is n o aliasin g in the fre q u e n c y b a n d 0 < F < Fx . B y d esig n in g th e filters in
th e su b s e q u e n t sta g es to satisfy th e sp ecificatio n s given in (10.6.5). w e e n su re th at
n o aliasin g o ccu rs in th e p rim a ry fre q u e n c y b a n d 0 < F < Fsc.
Example 10.6.1
Consider an audio-band signal with a nominal bandwidth of 4 kH z that has been
sampled at a rate of 8 kHz. Suppose that we wish to isolate the frequency components
below 80 Hz with a filter that has a passband 0 < F < 75 and a transition band
15 < F < 80. H ence F ^ = 75 Hz and FK = 80. The signal in the band 0 < F < 80
may be decim ated by the factor D = F J 2 F X = 50. We also specify that the filter
have a passband ripple 8, = 102 and a stopband ripple of <52 = 10-4.
The length o f the linear phase F IR filter required to satisfy these specifications
can be estim ated from one of the well known formulas given in the literature. R e
call that a particularly simple formula for approxim ating the length M , attributed to
Kaiser, is
^ = - ' 0 lo ^ - 1 3
14.6 A /

+ 1

(1 M 7 )

where A/ is the norm alized (by the sampling rate) width of the transition region [i.e.,
A f = (Fx Fpc)/Fs]. A m ore accurate formula proposed by H errm ann et al. (1973)
is
^

l M
A/

+ 1

(10.6.8)

Sec. 10.6

Multistage Implementation of Sampling-Rate Conversion

809

where D^i Si , ^2) and /( 5 i, 52) are defined as

A (S i.fc) = [0.005309(logini i ) 2 + G.07114(log]0Si)
-0.4761] log1052
-[0.00266(logi0 5i)2 + 0.5941 log10Si + 0.4278]
f ( S 1, i 2) = 11.012 + O.51244[log105, - log102]

(10.6.9)
(10.6.10)

Now a single FIR filter followed by a decim ator would require (using the Kaiser
form ula) a filter of (approximate) length
14.6(5/8000) 1 3 + ^ ^ 2
As an alternative, let us consider a two-stage decimation process with D\ = 25 and
Di = 2. In the first stage we have the specifications F\ = 320 Hz and
Passband: 0 < F < 75
Transition band: 15 < F < 240
J61
A^ 8000
fin = y

^21 = 2

Note that we have reduced the passband ripple <5[ by a factor of 2, so that the total
passband ripple in the cascade of the two filters does not exceed Si. On the other
hand, the stopband ripple is maintained at S2 in both stages. Now the Kaiser formula
yields an estim ate of A/, as

Ml =

~ 101ogn > ^ 2 1 - 1 3 + J % 167

14.6A/

For the second stage, we have F2 = F\fL 160 and the specifications
Passband: 0 < F < 75
Transition band: 75 < F < 80
A f = 320
<5i2 ~

&n = h

Hence the estim ate of the length M2 of the second filter is

M2 ~ 220
Therefore, the total length of the two F IR filters is approxim ately M\ + M2 = 387.
This represents a reduction in the filter length by a factor of more than 13.
The reader is encouraged to repeat the com putation above with D\ = 10 and
>2 = 5.
It is apparent from the com p utations in E xam p le 10.6.1 that the reduction
in th e filter length results from increasing the factor A f , w hich appears in the
d en om in ator in (10.6.7) and (10.6.8). By decim ating in m ultiple stages, w e are

810

Multirate Digital Signal Processing

Chap. 10

ab le to in c re a se th e w id th o f th e tra n sitio n reg io n th ro u g h a re d u c tio n in th e

sa m p lin g rate.
In th e case o f a m u ltista g e in te rp o la to r, th e sa m p lin g r a te a t th e o u tp u t of
th e / t h stag e is
//_! = ItFi
i = J , J - l .........1
a n d th e o u tp u t ra te is Fo = I F j w h e n th e in p u t sa m p lin g ra te is Fj . T h e c o r re
sp o n d in g freq u e n cy b a n d sp e cificatio n s a re
P a ssb a n d : 0 < F < Fp
T ra n s itio n b an d : Fp < F < F; Fsc
T h e fo llow ing e x am p le illu stra te s th e a d v a n ta g e s o f m u ltista g e in te rp o la tio n .
Example 10.6.2
Let us reverse the filtering problem described in Exam ple 10.6.1 by beginning with a
signal having a passband 0 < F < 75 and a transition band of 75 < F < 80. We wish
to interpolate by a factor of 50. By selecting I\ = 2 and l2 = 25. we have basically a
transposed form of the decim ation problem considered in Exam ple 10.6.1. Thus we
can simply transpose the two-stage decim ator to achieve the two-stage interpolator
with /, = 2 , i2 = 25. Mi 220. and M2 % 167.

10.7 SAMPLING-RATE CONVERSION OF BANDPASS SIGNALS

A b an d p a ss signal is a signal w ith fre q u e n c y c o n te n t c o n c e n tra te d in a n a rro w
b an d o f fre q u e n c ie s ab o v e z e ro freq u e n cy . T h e c e n te r fre q u e n c y Fc o f th e signal
is g en erally m uch la rg e r th a n th e b a n d w id th B (i.e., Fc B). B a n d p a ss signals
arise fre q u e n tly in p ra c tic e , m o st n o ta b ly in c o m m u n ic a tio n s, w h e re in fo rm a tio n
b e a rin g signals su c h as sp e ech an d v id eo a re tra n s la te d in fre q u e n c y a n d th e n
tra n sm itte d o v e r such c h a n n e ls as w ire lines, m icro w av e ra d io , a n d sa tellites.
In th is se ctio n w e c o n s id e r th e d e c im a tio n a n d in te rp o la tio n o f b a n d p a ss
signals. W e b eg in b y n o tin g th a t an y b a n d p a s s signal has an e q u iv a le n t low pass
re p re se n ta tio n , o b ta in e d by a sim ple fre q u e n c y tra n sla tio n o f th e b a n d p a s s signal.
F o r ex am p le, th e b a n d p a s s signal w ith sp e c tru m X ( F ) sh o w n in Fig. 10.22a can
b e tra n s la te d to lo w p ass by m e a n s o f a fre q u e n c y tra n sla tio n o f Fc, w h e re Fc
is an a p p ro p ria te ch o ice o f fre q u e n c y (u su ally , th e c e n te r fre q u e n c y ) w ith in th e
b a n d w id th o c c u p ied b y th e b a n d p a s s signal. T h u s w e o b ta in th e e q u iv a le n t low pass
signal as illu stra te d in Fig. 10.22b.
F ro m S ectio n 9.1 w e re c a ll th a t an a n a lo g b a n d p a s s signal c a n b e re p re se n te d
as
x ( t ) = ,4(/) cos[27rFcr + ?(/)]
= A( t ) cosf?(r) c o s 2 jrF cr - 4 (/) s in 0 ( /)s in 2 7 rF c/
(10.7.1)
= u c(t) cos 2 n F ct us (t) s i n 2 n F ct
= R & [x i( t)e J7* Fcl]

Sec. 10.7

Sampling-Rate Conversion of Bandpass Signals

811

|W)|

Bandpass signal
(a)

Equivalent lowpass signal

<b)

Figure 10.22

Bandpass signal and its equivalent lowpass representation.

w h ere, by d efin itio n ,

u c( t ) = y 4 (r)c o s# (0

(10.7.2)

u s (t) ~ j4 (O sin 0 (O

(10.7.3)

x t (t) = u c(t) + j u s (t)

(10.7.4)

A( t ) is called th e a mp l i t u d e o r envel ope o f th e signal, O(t') is th e p h a se , a n d u c(t)

a n d u s( t ) a re called th e quadrat ur e c o mp o n e n t s o f th e signal.
P h y sically , th e tra n s la tio n o f x {t ) to low pass in volves m u ltip ly in g (m ixing)
x (r) by th e q u a d r a tu re c a rrie rs cos 2 tz Fct a n d s i n 2 ^ F c; a n d th e n low pass filtering
th e tw o p ro d u c ts to e lim in a te th e fre q u e n c y c o m p o n e n ts g e n e ra te d a ro u n d th e
f re q u e n c y 2FC (th e d o u b le freq u e n c y te rm s). T h u s all th e in fo rm a tio n c o n te n t
c o n ta in e d in th e b a n d p a s s signal is p re se rv e d in th e low pass signal, a n d h en ce th e
la tte r is e q u iv a le n t to th e fo rm er. T h is fact is o b v io u s fro m th e sp e c tra l r e p re s e n
ta tio n o f th e b a n d p a s s signal, w hich can b e w ritte n as
X ( F ) = \ [ X , { F - Fc) + X ; { - F - Fc)]

(10.7.5)

w h e re X i ( f ) is th e F o u rie r tra n sfo rm o f th e e q u iv a le n t low pass sig n al xi{t) a n d

X { F ) is th e F o u rie r tra n s fo rm o f ;c(f).

Multirate Digital Signal Processing

812

Chap. 10

It w as sh o w n in S ectio n 9.1 th a t a b a n d p a s s signal o f b a n d w id th B can be

u n iq u e ly re p re s e n te d by sa m p le s ta k e n at a ra te o f 2 B sa m p le s p e r se co n d , p r o
v id ed th a t th e u p p e r b a n d (h ig h e st) fre q u e n c y is a m u ltip le o f th e signal b a n d
w id th B. O n th e o th e r h a n d , if th e u p p e r b a n d fre q u e n c y is n o t a m u ltip le of
B, th e sa m p lin g r a te m u st b e in c re a se d by a sm all a m o u n t to a v o id aliasing. In
an y case, th e sa m p lin g ra te fo r th e b a n d p a s s sig n a l is b o u n d e d fro m a b o v e a n d
b elo w as
2 B < Fs < 4 B

(10.7.6)

T h e re p re s e n ta tio n o f d isc re te -tim e b a n d p a ss signals is b asically th e sa m e as

th a t fo r an alo g sig n als given by (10.7.1) w ith th e su b s titu tio n o f t = n T , w h e re T
is th e sa m p lin g in terv al.

10.7.1 Decimation and Interpolation by Frequency

Conversion
T h e m a th e m a tic a l eq u iv a le n c e b e tw e e n th e b a n d p a s s signal x ( t ) a n d its e q u iv a le n t
lo w p ass re p re s e n ta tio n xi( t ) p ro v id e s o n e m e th o d fo r a lte rin g th e sa m p lin g rate
o f th e signal. S pecifically, w e can ta k e th e b a n d p a s s signal w h ich h as b e e n sa m
p led a t ra te Fx , c o n v e rt it to low pass th ro u g h th e fre q u e n c y c o n v e rsio n p ro cess
illu stra te d in Fig. 10.23, a n d p e rfo rm th e sa m p lin g -ra te c o n v e rsio n o n th e low pass
sig n al u sing th e m e th o d s d e sc rib e d prev io u sly . T h e low pass filters fo r o b ta in in g
th e tw o q u a d r a tu re c o m p o n e n ts can b e d esig n e d to h av e lin e a r p h a s e w ithin the

Figure 10.23

Conversion of a bandpass signal to lowpass.

Sec. 10.7

Sampling-Rate Conversion of Bandpass Signals

813

b a n d w id th o f th e sig n al an d to a p p ro x im a te th e id eal fre q u e n c y re sp o n s e c h a ra c

te ristic

*<>-(

<m7-7)

w h e re coB is th e b a n d w id th o f th e d isc re te -tim e b a n d p a s s sig n al (a>B < n ) .

I f d e c im a tio n is to b e p e rfo rm e d by a n in te g e r fa c to r D, th e an tialiasin g
filter p re c e d in g th e d e c im a to r can be c o m b in e d w ith th e low pass filter u se d fo r
fre q u e n c y c o n v e rsio n in to a single filter th a t a p p ro x im a te s th e id eal freq u e n c y
r e sp o n s e
H D(a>) =

h
0,

M <a >p / D
o th e rw ise

(10.7.8)

w h e re coD is an y d e s ire d freq u e n cy in th e ra n g e 0 < coD < n . F o r e x am p le, w e m ay

se le c t <
d d = m b / 2 if w e a re in te re ste d only in th e fre q u e n c y ran g e 0 < co < coB/ 2 D
o f th e o rig in a l signal.
If in te rp o la tio n is to be p e rfo rm e d by an in te g e r fa c to r I o n th e fre q u e n c y tra n s la te d sig n al, th e filter used to re je c t th e im ag es in th e sp e c tru m sh o u ld b e
d e sig n e d to a p p ro x im a te th e low pass filter c h a ra c te ristic

"'<->-{

(10-7-9>

W e n o te th a t in th e case o f in te rp o la tio n , th e low pass filte r n o rm ally u se d to re je c t

th e d o u b le -fre q u e n c y c o m p o n e n ts is re d u n d a n t an d m ay b e o m itte d . Its fu n ctio n
is e sse n tia lly se rv e d by th e im age re je c tio n filter H[(co).
F in ally , w e in d ic a te th a t sa m p lin g -ra te co n v e rsio n by an y ra tio n a l fa c to r //>
can b e acco m p lish ed o n th e b a n d p a s s signal as illu stra te d in Fig. 10.24. A g a in ,
th e lo w p ass filter fo r re je c tin g th e d o u b le -fre q u e n c y c o m p o n e n ts g e n e ra te d in th e
fre q u e n c y -c o n v e rsio n p ro cess can b e o m itte d . Its fu n c tio n is sim ply se rv e d by th e
im a g e -re je c tio n /a n tia lia s in g filter follow ing th e in te r p o la to r , w hich is d esig n ed to
a p p ro x im a te th e id eal fre q u e n c y re sp o n s e c h a ra c te ristic :
=

[ 0,

o th erw ise

(10.7.10)

O n c e th e sa m p lin g ra te o f th e q u a d r a tu re signal c o m p o n e n ts h a s b e e n a lte re d

by e ith e r d e c im a tio n o r in te rp o la tio n o r b o th , a b a n d p a s s signal c a n b e r e g e n e r
a te d by a m p litu d e m o d u la tin g th e q u a d r a tu re c a rrie rs cos<wcn a n d sin cocn by th e
c o rre s p o n d in g signal c o m p o n e n ts a n d th e n a d d in g th e tw o signals. T h e c e n te r

Figure 10.24

Sampling rate conversion of a bandpass signal.

Multirate Digital Signal Processing

814

fre q u e n c y

Chap. 10

is an y d e s ira b le fre q u e n c y in th e ran g e

mi n(u>B/ 2D, (oB/ 2 l ) < coc < n

(10.7.11)

10.7.2 Modulation-Free Method for Decimation and

Interpolation
B y re stric tin g th e fre q u e n c y ra n g e fo r th e signal w hose fre q u e n c y is to be a lte re d ,
it is p o ssib le to av o id th e c a rrie r m o d u la tio n p ro c e ss a n d to ach iev e freq u e n cy
tra n sla tio n d irectly . In th is case w e e x p lo it th e freq u e n cy tra n sla tio n p ro p e rty
in h e re n t in th e p ro c e ss o f d e c im a tio n an d in te rp o la tio n .
T o b e specific, let us c o n s id e r th e d e c im a tio n o f th e sa m p le d b an d p a ss signal
w h o se sp e ctru m is sh o w n in Fig. 10.25. N o te th a t th e signal sp e c tru m is confined
to th e freq u e n c y ran g e
mir
(m + l ) 7 r
, _
----- < co < :---------
(10.7.12)
D ~
~
D
w h ere m is a p o sitiv e in te g e r. A b a n d p a s s filter w o u ld n o rm a lly b e u sed to elim
in ate signal fre q u e n c y c o m p o n e n ts o u tsid e th e d e s ire d fre q u e n c y ran g e. T h e n
d irect d e cim atio n o f th e b a n d p a s s signal by th e fa c to r D re su lts in th e sp e c tru m
sh o w n in Fig. 10.26a, fo r m o d d , a n d Fig. 10.26b fo r m even. In th e case w h ere
m is o d d . th e re is an in v ersio n o f th e sp e c tru m o f th e signal. T h is in v ersio n can
b e u n d o n e by m u ltip ly in g ea c h sa m p le o f th e d e c im a te d signal by ( - 1 ) " , n = 0.
1, N o te th a t v io la tio n o f th e b a n d w id th c o n s tra in t given by (10.7.12) re su lts in
signat aliasing,
M o d u la tio n -fre e in te rp o la tio n o f a b a n d p a s s signal by an in te g e r fa c to r I can
b e acco m p lish ed in a sim ilar m a n n e r. T h e p ro c e ss o f u p sa m p lin g by in se rtin g zero s
b e tw e e n sa m p les o f x( n ) p ro d u c e s I im ag es in th e b a n d 0 < w < i t . T h e d e sire d
im ag e can b e se le c te d by b a n d p a s s filtering. N o te th a t th e p ro c e ss o f in te r p o la
tio n also p ro v id es u s w ith th e o p p o rtu n ity to ach iev e fre q u e n c y tra n sla tio n o f th e
sp e ctru m .
F inally, m o d u la tio n -fre e sa m p lin g r a te co n v e rsio n fo r a b a n d p a s s signal by
a ra tio n a l fa c to r I / D can b e acco m p lish ed b y cascad in g a d e c im a to r w ith an in
te r p o la to r in a m a n n e r th a t d e p e n d s on th e ch o ice o f th e p a r a m e te r s D a n d I.
A b a n d p a s s filter p re c e d in g th e sa m p lin g c o n v e rte r is u su ally re q u ire d to isolate
th e signal freq u e n c y b an d o f in te re st. N o te th a t th is a p p ro a c h p ro v id e s us w ith a
m o d u la tio n -fre e m e th o d fo r ach iev in g fre q u e n c y tra n sla tio n o f a signal by selecting
D = 1.

(m + i) n

mTi

Figure 10.25

mn

{m + 1)n

Spectrum o f a bandpass signal.

Sec. 10.8

Sampling-Rate Conversion by an Arbitrary Factor

815

m odd

(>

m even

(b>
Figure 10.26

Spectrum of decimated bandpass signal.

10.8 SAMPLING-RATE CONVERSION BY AN ARBITRARY FACTOR

In the previou s sectio n s o f this chapter, w e have show n h ow to perform sam pling
rate con version exactly by a rational num ber I / D . In som e applications, it is either
inefficient or, so m etim es im possib le to use such an exact rate con version schem e.
W e first con sid er the fo llow in g tw o cases.

Case 1. W e n eed to perform rate con version by the rational num ber I / D ,
w h ere / is a large in teger (e.g., l / D = 1023/511). A lth ou gh w e can ach ieve
exact rate con version by this num ber, w e w ould n eed a p olyp hase filter with 1023
subfilters. Such an exact im plem en tation is ob viou sly inefficient in m em ory usage
b ecau se w e n eed to store a large num ber o f filter coefficients.
Case 2. In so m e applications, th e exact con version rate is n ot know n w hen
w e design the rate converter, or th e rate is con tinu ou sly changing during the co n
version p rocess. For exam p le, w e m ay en cou n ter the situation w here th e input and
ou tp u t sam p les are con trolled by tw o in d ep en dent clocks. E ven though it is still
p ossib le to d efine a n om in al con version rate that is a rational num ber, the actual

816

Multirate Digital Signal Processing

Chap. 10

ra te w o u ld b e slig h tly d iffe re n t, d e p e n d in g on th e fre q u e n c y d iffe re n c e b etw een

th e tw o clocks. O b v io u sly , it is n o t p o ssible to design an e x a c t ra te c o n v e rte r in
th is case.
T o im p le m e n t sa m p lin g r a te c o n v ersio n fo r a p p lic a tio n s sim ilar to th e se
cases, we re s o rt to n o n e x a c t ra te c o n v ersio n sc h em es. U n a v o id a b ly , a n o n ex act
sc h em e will in tro d u c e so m e d isto rtio n in th e c o n v e rte d o u tp u t signal. (It sh ould
be n o te d th a t d isto rtio n exists ev en in an ex a c t ra tio n a l r a te c o n v e rte r because
th e p o ly p h a se filter is n e v e r id eal.) S uch a c o n v e rte r will b e a d e q u a te , as long
as th e to ta l d isto rtio n d o e s n o t ex cee d th e specification r e q u ire d in th e a p p li
catio n .
D e p e n d in g on th e ap p lic a tio n re q u ire m e n ts a n d im p le m e n ta tio n c o n stra in ts,
w e can use first-o rd e r, se c o n d -o rd e r, o r h ig h e r-o rd e r a p p ro x im a tio n s . W e shall d e
sc rib e first-o rd e r a n d se c o n d -o rd e r a p p ro x im a tio n m e th o d s a n d p ro v id e an analysis
o f th e resu ltin g tim in g e rro rs.

10.8.1 First-Order Approximation

L e t u s d e n o te th e a rb itra ry c o n v e rsio n ra te by r a n d su p p o se th a t th e in p u t to th e
ra te c o n v e rte r is th e se q u e n c e (jr(n)}. W e n e e d to g e n e ra te a se q u e n c e o f o u tp u t
sa m p le s se p a ra te d in tim e by Tx f r , w h e re Tx is th e sa m p le in te rv a l for {x(n)i. By
co n stru ctin g a p o ly p h a se filter w ith a larg e n u m b e r o f su b filters as ju s t d esc rib e d ,
we can a p p ro x im a te such a se q u e n c e w ith a n o n u n ifo rm ly sp a c e d se q u e n c e . W ith
o u t loss o f g e n e ra lity , w e can e x p ress 1 j r as

w h ere k a n d I a re p o sitiv e in te g e rs a n d 0 is a n u m b e r in th e ra n g e
0 < &< y
C o n se q u e n tly , 1j r is b o u n d e d fro m ab o v e an d b e lo w as
k

<

<

jfc + 1

-------------

I
r
I
I c o rre sp o n d s to th e in te rp o la tio n fa c to r, w hich w ill b e d e te r m in e d to satisfy th e
specificatio n on th e a m o u n t o f to le ra b le d isto rtio n in tro d u c e d b y ra te c o n v ersio n .
/ is also eq u a l to th e n u m b e r o f p o ly p h a se filters.
F o r e x am p le, su p p o se th a t r = 2.2 a n d th a t w e h a v e d e te r m in e d , as we
will d e m o n s tra te , th a t 1 = 6 p o ly p h a se filters a re r e q u ire d to m e e t th e d isto rtio n
specificatio n . T h e n
k ^ 2
1
3
k + 1
/
6
r
6
I
so th a t k = 2. T h e tim e sp a cin g b e tw e e n sa m p le s o f th e in te r p o la te d se q u e n c e is
Tx / I . H o w e v e r, th e d e s ire d c o n v ersio n ra te r = 2.2 fo r / = 6 c o rre sp o n d s to a
d e c im a tio n fa c to r o f 2.727, w h ich falls b e tw e e n k = 2 a n d k = 3. In th e first-o rd e r
a p p ro x im a tio n , w e ach iev e th e d e s ire d d e c im a tio n r a te b y se le c tin g th e o u tp u t

Sec. 10.8

Sampling-Rate Conversion by an Arbitrary Factor

Figure 10.27

817

Sample rate conversion by use of first-order approximation.

sa m p le fro m th e p o ly p h a se filter closest in tim e to th e d e s ire d sa m p lin g tim e. T h is

is illu stra te d in Fig. 10.27 fo r 1 = 6.
In g e n e ra l, to p e rfo rm ra te c o n v ersio n by a fa c to r r, w e em p lo y a p o ly p h ase
filter to p e rfo rm in te rp o la tio n an d th e re fo re to in c re a se th e fre q u e n c y o f th e o rig
in al s e q u e n c e o f a fa c to r o f I. T h e tim e sp acin g b e tw e e n th e sa m p le s o f th e
in te r p o la te d se q u e n c e is e q u a l to Tx/ J . If th e id eal sa m p lin g tim e o f th e m th s a m
p le , y ( m) , o f th e d e sire d o u tp u t se q u e n c e is b e tw e e n th e sa m p lin g tim e s o f tw o
sa m p le s o f th e in te rp o la te d se q u en ce, w e se lect th e sa m p le clo se r to y ( m ) as its
ap p ro x im a tio n .
L e t u s a ssu m e th a t th e m th se le c te d sa m p le is g e n e ra te d b y th e (im)th su b filter
u sin g th e in p u t sa m p le s x( n) , x ( n 1 ) , . . . , x ( n K + 1) in th e d e la y line. T h e
n o rm a liz e d sa m p lin g tim e e rro r (i.e., th e tim e d iffe re n c e b e tw e e n th e se le c te d
sa m p lin g tim e a n d th e d e s ire d sam p lin g tim e n o rm a liz e d b y Tx) is d e n o te d by tm.
T h e sign o f tm is p o sitiv e if th e d e s ire d sa m p lin g tim e le a d s th e se le c te d sam pling
tim e , a n d n e g a tiv e o th e rw ise . It is easy to show th a t \tm | < 0 .5 / / . T h e n o rm a liz e d
tim e a d v a n c e fro m th e m th o u tp u t y ( m ) to th e (m + l ) s t o u tp u t y ( m + 1) is e q u a l
t o ( 1 / r ) + tm.
T o c o m p u te th e n e x t o u tp u t, w e first d e te rm in e a n u m b e r clo sest to im/ J +
+ km/ I th a t is o f th e fo rm /m_] + im+ i / I , w h e re b o th /m+1 a n d i m+1 a re
in te g e rs a n d im+i < I. T h e n , th e (m + l ) s t o u tp u t y(m + 1) is c o m p u te d using th e
(jm+i) th su b filte r a f te r sh iftin g th e signal in th e d e la y line by lm+i in p u t sam p les.
T h e n o rm a liz e d tim in g e r r o r fo r th e (m + l ) th sa m p le is tm+1 = ( i m/ I + l / r + t m) ~
(lm+] + im+i / I ) . I t is sa v ed fo r th e c o m p u ta tio n o f th e n e x t o u tp u t sam ple.

818

Multirate Digital Signal Processing

Chap. 10

B y in creasin g th e n u m b e r o f su b filters u se d , w e can a rb itra rily in c re a se th e

co n v ersio n accu racy . H o w e v e r, w e also re q u ire m o re m e m o ry to sto re th e larg e
n u m b e r of filter coefficients. H e n c e it is d e s ira b le to use as few su b filters as possible
w hile k e e p in g th e d isto rtio n in th e c o n v e rte d signal below th e sp ecificatio n . T he
d isto rtio n in tro d u c e d d u e to th e sa m p lin g -tim e a p p ro x im a tio n is m o st c o n v en ien tly
e v a lu a te d in th e fre q u e n c y d o m a in .
S u p p o se th a t th e in p u t d a ta se q u e n c e {x( n)\ h a s a flat s p e c tru m fro m - t o x
to cox , w h e re u>x <
w ith a m a g n itu d e A. Its to ta l p o w e r can b e c o m p u te d using
P a rse v a ls th e o re m , n am ely ,

Ssi
r \X{(o)\2d ( o =
5
2?r ; _ tUj

(10.8.1)

F ro m th is d iscussion given, w e k n o w th a t fo r each o u tp u t v(m ), th e tim e d ifferen ce

b e tw e e n th e d e s ire d filter a n d th e filter actu ally u se d is tm, w h e re | r j < 0 .5 / / .
H e n c e th e fre q u e n c y re sp o n se o f th e s e filters c a n b e w ritte n as eJa>T a n d
resp ectiv ely . W h e n / is la rg e , wtm is sm all. By ig n o rin g h ig h -o rd e r e rro rs, we can
w rite th e d ifferen ce b e tw e e n th e fre q u e n c y re sp o n s e s as
e jur _ eja>U-l) _

1 _ e - j w,)
( 10.8 .2 )

= e i0J1 (1 cos cot,,, + j sin colm) % j e JU'Tcot,,,

B y using th e b o u n d |irn\ < 0 .5 / / , w e o b ta in an u p p e r b o u n d fo r th e to ta l e rro r
p o w e r as
Pe = - L

' \ X ( c o ) e ^ T ~ X(co)eJU,lT~ ^ \ 2d w ^
[ ' \X(<o)jeJ^a>tm\2dco
J
ojx
2-7T J
(u,

- ^ L a2 {t) 2dw=^

( 1 0 '8 '3 )

T h is b o u n d show s th a t th e e r r o r p o w e r is in v ersely p ro p o rtio n a l to th e s q u a re o f th e

n u m b e r o f su b filters I. T h e re fo re , th e e r r o r m a g n itu d e is in v e rse ly p ro p o rtio n a l
to I. H e n c e w e call th e a p p ro x im a tio n o f th e ra te c o n v ersio n m e th o d d e sc rib e d
a b o v e a first-o rd e r a p p ro x im a tio n . B y u sin g (10.8.3) a n d (10.8.1), th e r a tio o f th e
sig n a l-to -d isto rtio n d u e to a sa m p lin g -tim e e r r o r fo r th e firs t-o rd e r a p p ro x im a tio n ,
d e n o te d as S D rR l , is lo w e r b o u n d e d as
S D ,R 1 =

P
pe

M l2
> ~
(4

(10.8.4)

It can be seen fro m (10.8.4) th a t th e sig n a l-to -d isto rtio n r a tio is p ro p o rtio n a l
to th e sq u a re o f th e n u m b e r o f su b filters.
Exam ple 10.8.1
Suppose that the input signal has a flat spectrum betw een - 0 .8 jt and 0.8 tt. Determ ine
the num ber of subfilters to achieve a signal-to-distortion ratio of 50 dB.

Sec. 10.8

Sampling-Rate Conversion by an Arbitrary Factor

819

Solution T o achieve an SDrR > 105, we set S D iR l = 12/2/wJ equal to 105. Thus
we find that
fw
1 s w ,,/ ~ ~ ~ 230 subfilters

10.8.2 Second-Order Approximation (Linear Interpolation)

T h e d isa d v a n ta g e o f th e first-o rd e r a p p ro x im a tio n m e th o d is th e larg e n u m b e r o f
su b filte rs n e e d e d to ach iev e a specified d isto rtio n re q u ire m e n t. In th e follow ing
d iscu ssio n , w e d esc rib e a m e th o d th a t u se s lin e a r in te rp o la tio n to ach iev e th e sa m e
p e rfo rm a n c e w ith a r e d u c e d n u m b e r o f subfilters.
T h e im p le m e n ta tio n o f th e lin e a r in te rp o la tio n m e th o d is v ery sim ilar to th e
firs t-o rd e r a p p ro x im a tio n d iscu ssed ab o v e. In ste a d o f u sin g th e sa m p le fro m th e
in te rp o la tin g filter clo sest to th e d e s ire d c o n v ersio n o u tp u t as th e a p p ro x im a tio n ,
w e c o m p u te tw o a d ja c e n t sa m p le s w ith th e d e s ire d sa m p lin g tim e falling b e tw e e n
th e ir sa m p lin g tim es, as is illu stra te d in Fig. 10.28. T h e n o rm a liz e d tim e spacing

y(+1M i
a m=Ilm
Figure 10.28

Sample rale conversion by use o f linear interpolation.

820

Multirate Digital Signal Processing

Chap. 10

b e tw e e n th e se tw o sa m p le s is 1 j l . A ssu m in g th a t th e sa m p lin g tim e o f th e first

sa m p le lags th e d e s ire d sa m p lin g tim e by jm, th e sa m p lin g tim e o f th e se c o n d sa m
p le is th e n lead in g th e d e s ire d sa m p lin g tim e by ( 1 / / ) tm. If w e d e n o te th ese
tw o sa m p le s by vi (m) a n d y i ( m ) a n d use lin e a r in te rp o la tio n , w e can c o m p u te the
a p p ro x im a tio n to th e d e s ire d o u tp u t as
v(m ) = (1 - a m)vi(m ) + a n,y2(m)

(10.8.5)

w h e re a m = l t m. N o te th a t 0 < a, < 1 .
T h e im p le m e n ta tio n o f lin e a r in te rp o la tio n is sim ilar to th a t fo r th e firsto r d e r a p p ro x im a tio n . N o rm a lly , b o th y i(m ) a n d yz i m) a re c o m p u te d using th e /th
an d (/ + l) th su b filters, re sp e c tiv e ly , w ith th e sa m e se t o f in p u t d a ta sa m p le s in
th e d elay line. T h e o n ly e x c e p tio n is in th e b o u n d a ry case, w h e re / = / - 1. In
th is case w e u se th e ( / l ) t h su b filte r to c o m p u te >'i(/w), b u t th e se co n d sam ple
yi ( m) is c o m p u te d u sin g th e z e ro th su b filter a fte r new in p u t d a ta a re sh ifted into
th e d e la y line.
T o an aly ze th e e r r o r in tro d u c e d by th e se c o n d -o rd e r a p p ro x im a tio n , we first
w rite th e fre q u e n c y re sp o n s e s o f th e d esired filter an d th e tw o su b filters u se d to
c o m p u te vi(m ) a n d V2(w ), as e-/Wr,
ancj ja>u-t+'\/i)^ resp ec tiv ely . B e cau se
lin e a r in te rp o la tio n is a iin e a r o p e ra tio n , w e can also use lin e a r in te rp o la tio n to
co m p u te th e fre q u e n c y re sp o n se o f th e filter th a t g e n e ra te s y(w?) as

= e Jtur[(l a m)e~JW'"' + ameJ'i~!"+l / n ]

( 10 . 8 .6 )

- ejmT0 - a,)(cos coin, - ;s in o tf m)

-h e ^ Ta [ c o s a j ( l / / - tm) + j sin c o (l/7 - r , ) ]
B y ig n o rin g h ig h -o rd e r e rro rs, w e can w rite th e d ifferen ce b e tw e e n th e d esired
fre q u e n c y resp o n se s an d th e o n e given by (10.8.6) as
ei m - (1 - a m)e Jtu<t ," ) -

\ - (i

cim) c o s u)tm am costL>(l// - tm)]

+ j [ ( l - a m) sin (Dtm - a m s in a > ( l// - rm)]}

U sin g (1 - a m) am <
1
Pe = = - /

(10.8.7)

w e o b ta in a n u p p e r b o u n d fo r th e to ta l e r r o r p o w e r as

\X(u>)[ejwT - (1 -

- a me ^ - ' ^ n ]\2dco

(10.8.8)

Sec. 10.9

Applications of Multirate Signal Processing

821

T his result indicates that the error m agnitude is inversely proportional to I 2. H en ce

w e call the approxim ation using linear interpolation a s e c ond- or d e r appr ox i ma t i o n.
U sin g (10.8.8) and (10.8.1), the ratio o f signal-to-distortion d u e to a sam pling tim e
error for the secon d-ord er approxim ation, d en oted by S D ,R 2 , is b ou n ded from
b elo w as
S D jR 2 =

P
Pe

80/ 4
>
arx

(10.8.9)

T h erefore, th e signal-to-distortion ratio is p roportional to the fourth p ow er o f the

num ber o f subfilters.
Exam ple 10.8.2
D eterm ine the num ber of subfilters required to meet the specifications given in E x
ample 10.8.1 when linear interpolation is employed.
Solution
obtain

To achieve S D ,R > 105, we set S D iR 2 = 80I A/w* equal to 10s. Thus we

From this exam ple we see that the required num ber o f subfilters for the
secon d-ord er approxim ation is reduced by a factor o f ab ou t 15 com pared to the
first-order approxim ation. H ow ever, w e now n eed to co m p u te tw o interpolated
sam p les in this case, instead o f o n e for the first-order approxim ation. H en ce w e
have d ou b led the com putational com plexity.
L inear interpolation is the sim plest case o f the class o f approxim ation m eth
od s based o n Lagrange p olynom ials. It is also p ossib le to use higher-order L a
grange polyn om ial approxim ations (interp olation) to further reduce the num ber
o f subfilters required to m eet specifications. H ow ever, th e secon d-ord er approx
im ation seem s sufficient for m ost practical applications. T h e in terested reader is
referred to the paper by R am stad (1984) for higlier-order L agrange interpolation
m ethods.

10.9 APPLICATIONS OF MULTIRATE SIGNAL PROCESSING

T here are num erous practical applications o f m ultirate signal processing. In this
sectio n w e d escribe a few o f these applications.

10.9.1 Design of Phase Shifters

S u p p ose that w e w ish to design a netw ork that delays th e signal x ( n ) by a fraction
o f a sam ple. L et us assum e that the d elay is a rational fraction o f a sam pling

822

Multirate Digital Signal Processing

x(n)
Ft

71

If

Figure 10.29

Lowpass
filler

Delay by
k samples

IF,

Chap. 10

1
j

Method for generating a delay in a discrete-time signal.

in te rv a l Tx [i.e.. d = (k / I ) T x, w h e re k a n d / a re relativ ely p rim e po sitiv e in teg ers].

In th e freq u e n cy d o m ain , th e d elay c o rre sp o n d s to a lin e a r p h a s e shift o f th e fo rm
@(a>) =

kio

(10.9.1)

T h e d esign o f an all-p ass lin e a r-p h a s e filter is relativ ely difficult. H o w e v e r,

w e can use th e m e th o d s o f sa m p le -ra te co n v e rsio n to ach iev e a d elay o f (k / l ) T x,
ex actly , w ith o u t in tro d u c in g an y significant d isto rtio n in th e sig n al. T o be specific,
let us co n sid e r th e sy stem sh o w n in Fig. 10.29. T h e sa m p lin g r a te is in crease d by a
fa c to r / using a s ta n d a rd in te rp o la to r. T h e lo w p ass filter e lim in a te s th e im ages in
th e sp e ctru m o f th e in te rp o la te d signal, a n d its o u tp u t is d e la y e d by k sa m p le s a t
th e sam p lin g r a te I F x . T h e d e la y e d sig n a l is d e c im a te d by a fa c to r D = I. T h u s
we h av e ach iev ed th e d e sire d d e la y o f (k / l ) T x .
A n efficient im p le m e n ta tio n o f th e in te r p o la to r is th e p o ly p h a s e filter illu s
tra te d in Fig. 10.30. T h e d e la y o f k sa m p le s is ach iev ed by p lacing th e initial
p o sitio n o f th e c o m m u ta to r a t th e o u tp u t o f th e Jtth su b filter. S in ce d e c im a tio n by

Figure 10.30
Fig. 10.29.

Polyphase filter structure for implementing the syslem shown in

Sec. 10.9

Applications of Multirate Signal Processing

823

D = I m e a n s th a t w e ta k e o n e o u t o f ev ery / sa m p le s fro m th e p o ly p h a s e filter, th e

c o m m u ta to r p o sitio n c a n b e fixed to th e o u tp u t o f th e fcth su b filter. T h u s a d e la y
in k / I can b e a c h iev ed by using o n ly th e k th su b filter o f th e p o ly p h ase filter. W e
n o te th a t th e p o ly p h a se filter in tro d u c e s an a d d itio n a l d e la y o f (Af l ) /2 sam ples,
w h ere M is th e le n g th o f its im p u lse re sp o n se .
F in ally , w e m e n tio n th a t if th e d e s ire d d elay is a n o n ra tio n a l fa c to r o f th e
sa m p le in te rv a l Tx , e ith e r th e firs t-o rd e r o r se c o n d -o rd e r a p p ro x im a tio n m e th o d
d e s c rib e d in S ectio n 10.8 can b e u se d to o b ta in th e delay.

10.9.2 Interfacing of Digital Systems with Different

Sampling Rates
In p ra c tic e w e fre q u e n tly e n c o u n te r th e p ro b le m o f in terfacin g tw o d igital sy stem s
th a t a re c o n tro lle d b y in d e p e n d e n tly o p e ra tin g clocks. A n an a lo g so lu tio n to
th is p ro b le m is to c o n v e rt th e signal fro m th e first sy stem to a n a lo g form an d
th e n re sa m p le it a t th e in p u t to th e se c o n d sy stem u sin g th e clock in th is system .
H o w e v e r, a sim p le r a p p ro a c h is o n e w h e re th e in te rfa c in g is d o n e by a digital
m e th o d u sin g th e b asic sa m p le -ra te c o n v ersio n m e th o d s d e sc rib e d in this c h a p te r.
T o b e specific, le t us c o n s id e r in te rfa c in g th e tw o sy stem s w ith in d e p e n d e n t
clocks as sh o w n in F ig. 10.31. T h e o u tp u t o f system A a t ra te Fx is fed to an
in te rp o la to r w hich in c re a se s th e sa m p lin g ra te by I . T h e o u tp u t o f th e in te rp o la to r
is fed a t th e ra te I Fx to a d igital sa m p le -a n d -h o ld w hich se rv es as th e in te rfa c e to
sy stem B a t th e h ig h sa m p lin g r a te 1FX. S ignals fro m th e d igital sa m p le -a n d -h o ld
a re re a d o u t in to sy stem B at th e clock ra te D F y of sy stem B. T h u s th e o u tp u t
r a te fro m th e sa m p le -a n d -h o ld is n o t sy n c h ro n iz e d w ith th e in p u t rate.
In th e sp ecial case w h e re D = I a n d th e tw o clock ra te s a re c o m p a ra b le
b u t n o t id en tical, so m e sa m p le s a t th e o u tp u t o f th e sa m p le -a n d -h o ld m ay be
r e p e a te d o r d ro p p e d a t tim es. T h e a m o u n t o f signal d isto rtio n re su ltin g from th is
m e th o d can b e k e p t sm a ll if th e in te rp o la to r/d e c im a to r fa c to r is large. B y using
lin e a r in te rp o la tio n in p la c e o f th e d ig ital sa m p le -a n d -h o ld , as w e d e sc rib e d in
S ectio n 10.8, w e can f u rth e r re d u c e th e d isto rtio n a n d th u s re d u c e th e size o f th e
in te rp o la to r facto r.

Figure 1031

Interfacing of two digital systems with different sampling rates.

Multirate Digital Signal Processing

824

Chap. 10

10.9.3 Implementation of Narrowband Lowpass Filters

In S ectio n 10.6 w e d e m o n s tra te d th a t a m u ltista g e im p le m e n ta tio n o f sam plingra te co n v ersio n o fte n p ro v id es fo r a m o re efficient re alizatio n , esp ecially w h en th e
filter sp ecificatio n s a re v ery tig h t (e.g., a n a rro w p a s sb a n d a n d a n a rro w tra n sitio n
b a n d ). U n d e r sim ilar c o n d itio n s, a low pass, lin e a r-p h a s e F I R filte r m ay b e m o re
efficien tly im p le m e n te d in a m u ltista g e d e c im a to r-in te rp o la to r co n fig u ratio n . T o
b e m o re specific, w e can e m p lo y a m u ltistag e im p le m e n ta tio n o f a d e c im a to r o f
size D, fo llo w ed by a m u ltista g e im p le m e n ta tio n o f an in te r p o la to r o f size / , w h ere
I = D.
W e d e m o n s tra te th e p ro c e d u re b y m ean s o f a n ex a m p le fo r th e desig n o f
a lo w p ass filter w hich h as th e sa m e sp ecificatio n s as th e filter th a t is given in
E x a m p le 10.6.1.
Example 10.9.1
Design a linear-phase FIR filter that satisfies the following specifications:
Sampling frequency:
Passband:
Transition band:
Stopband
Passband ripple:
Stopband ripple:

8000 Hz
0 < F < 75
75 < F < 80
80 < F < 4000
5] = 10-2
&2 = 10~4

Solution If this filter were designed as a single-rate linear-phase F IR filter, the length
of the filter required to m eet the specifications is (from Kaisers form ula)
M

5152

Now, suppose that we employ a multirate im plem entation of the lowpass filter
based on a decimation and interpolation factor of D = / = 100. A single-stage
implem entation of the decim ator-interpolator requires an FIR filter of length

^ ^ - O l o g W ;/ 2 ) - 1 3 + l a 5 4 a )
14.6 A /

However, there is a significant savings in computational complexity by implementing

the decimator and interpolator filters using their corresponding polyphase filters. If
we employ linear-phase (symmetric) decimation and interpolation filters, the use of
polyphase filters reduces the multiplication rate by a factor of 100.
A significantly more efficient im plementation is obtained by using two stages
of decimation followed by two stages of interpolation. For example, suppose that we
select D\ = 50, D2 = 2, h = 2, and I2 = 50. Then the required filter lengths are
-lO lo g C JA /4 ) - 1 3 _____
W l..........

1 4 .6 V ---------+

-101oglo( W 4 ) - 1 3 . , _
---------- u Z T f ----------+ 1 ^ 233
Thus we obtain a reduction in the overall filter length of 2(5480)/2(177+233) ~ 13.36In addition, we obtain further reduction in the multiplication rate by using polyphase

Sec. 10.9

Applications of Multirate Signal Processing

825

filters. For the first stage of decimation, the reduction in multiplication rate is 50, while
for the second stage the reduction in multiplication rate is 100. F urther reductions
can be obtained by increasing the num ber of stages of decim ation and interpolation.

10.9.4 Implementation of Digital Filter Banks

F ilte r b a n k s a re g en erally c a teg o rized as tw o types, analysis filter b a n k s a n d s y n
thesis filter ban k s . A n analysis filter b a n k co n sists o f a se t o f filters, w ith system
fu n c tio n s {#;(&)}, a rra n g e d in a p a ra lle l b a n k as illu s tra te d in Fig. 10.32a. T h e
f re q u e n c y re sp o n s e c h a ra c te ristic s o f th is filter b a n k sp lits th e signal in to a c o r re
sp o n d in g n u m b e r o f su b b a n d s. O n th e o th e r h a n d , a sy n th e sis filter b a n k co n sists
o f a se t o f filters w ith system fu n ctio n s {G *(')}, a rra n g e d as sh o w n in Fig. 10.32b,
w ith c o rre sp o n d in g in p u ts {v*(/i)}- T h e o u tp u ts o f th e filters a re su m m e d to fo rm
th e sy n th e siz e d signal {*(/;))
F ilte r b a n k s a re o fte n u se d fo r p e rfo rm in g sp e c tru m analysis a n d signal syn
th esis. W h en a filter b a n k is em p lo y ed in th e c o m p u ta tio n o f th e d isc re te F o u rie r

Analysis fitter bank

(a)

Synthesis filter bank

(b)
Figure 10.32

A digital filter bank.

826

Multirate Digital Signal Processing

Chap. 10

tra n sfo rm (D F T ) o f a se q u e n c e {*()}, th e filter b a n k is called a D F T filter b an k .

A n an aly sis filter b an k co n sistin g o f N filters {Hk(z), Jt = 0 , 1 , ___ N 1} is called a
u n ifo rm D F T filter b a n k if Hk (z), it = 1, 2 , . . . , N 1, a re d e riv e d fro m a p ro to ty p e
filter H q(z ), w h ere
(10.9.2)
H e n c e th e fre q u e n c y re sp o n se c h a ra c te ristic s o f th e filters { //* ( '), Jt = 0, 1 , ,
/ / l} a re sim ply o b ta in e d b y u n ifo rm ly sh iftin g th e fre q u e n c y re sp o n s e o f th e p ro
to ty p e filter b y m u ltip le s of 2n / N . In th e tim e d o m a in th e filters a re c h a ra c te riz e d
by th e ir im p u lse resp o n ses, w hich can b e e x p re ss e d as
h k (n) = h 0( n)ej2nnk/N

k = 0 , l ____ N - l

(10.9.3)

w h e re {/iq()} is th e im pulse re sp o n s e o f th e p ro to ty p e filter.

T h e u n ifo rm D F T analysis filter b a n k can be re a liz e d as sh o w n in Fig. 10.33a,
w h e re th e fre q u e n c y c o m p o n e n ts in th e se q u e n c e {jr(n)} a re tra n s la te d in freq u e n cy
to lo w p ass by m u ltip ly in g x( n) w ith th e co m p lex e x p o n e n tia ls e x p ( j l n n k j N ) , k =
1___ N - 1, an d th e re su ltin g p ro d u c t signals a re p assed th ro u g h a low pass filter
w ith im p u lse re sp o n se {ho(n)). Since th e o u tp u t o f th e low pass filter is relativ ely
n a rro w in b an d w id th , th e signal can b e d e c im a te d by a fa c to r D < N . T h e resu ltin g
d e c im a te d o u tp u t signal can b e e x p re ss e d as
h o ( mD n)x(/i)e', -jlxnk/N

X k(m)

k = 0, 1, . . . , N - 1
(10.9.4)

n
m = 0 , 1,...

w h e re {X*(m)} a re sa m p les o f th e D F T a t fre q u e n c ie s a>k = 2 n k / N .

T h e c o rre sp o n d in g sy n th e sis filte r fo r ea c h e le m e n t in th e filter b a n k can
b e v iew ed as show n in Fig. 10.33b, w h e re th e in p u t signal se q u e n c e s {l*(m ), k =
0, 1 , . . . , N 1} a re u p sa m p le d by a fa c to r o f I = D, filte re d to re m o v e th e
im ag es, a n d tra n sla te d in fre q u e n c y by m u ltip lic a tio n by th e c o m p lex ex p o n e n tia ls
{ e x p i j l n n k / N ) , k = 0, 1, . . N 1). T h e re su ltin g fre q u e n c y -tra n sla te d signals
fro m th e N filters are th e n su m m ed . T h u s w e o b ta in th e se q u e n c e

ej2"klN

i n

= !> ( *
wn

Y , y^ m ^8o(n - m l )
_

- m l)

Jlnnk/N
1 Y d m ) e ^ nklN
1* I__ n

(10.9.5)

w h e re th e fa c to r 1 / N is a n o rm a liz a tio n fa c to r, {y(m)} re p re s e n t sa m p le s o f th e

in v e rse D F T se q u e n c e c o rre sp o n d in g to {}*(m)}, {go()} is th e im p u lse resp o n se
o f th e in te rp o la tio n filter, a n d I ~ D.

Sec. 10.9

Applications of Muttirate Signal Processing

627

e-jainn

Analysis
(a)

e)<w

Synthesis

(b)
Figure 1033

w* =
N

A uniform D FT filter bank.

T h e re la tio n sh ip b e tw e e n th e o u tp u t (X*(n)} o f th e analysis filter b a n k a n d

th e in p u t {y*(m )} to th e sy n th e sis filter b a n k d e p e n d s o n th e a p p lic a tio n . U s u
ally, {y*(rn)| is a m o d ified v ersio n o f {X*(m)), w h e re th e specific m o d ifica tio n is
d e te rm in e d b y th e ap p licatio n .
A n a lte rn a tiv e re a liz a tio n o f th e an aly sis a n d sy n th e sis filter b a n k s is illu s
tr a te d in Fig. 10.34. T h e filters a re re a liz e d as b a n d p a s s filters w ith im p u lse r e
sp o n ses
hk(n) = h0(n)eJ7* nk/N

k = 0,1,..., N - 1

(10.9.6)

Multirate Digital Signal Processing

828

Chap. 10

p-jtiitfnD

Analysis
(a)
ejm<jnD

Synthesis
(b)
Figure 10.34

Alternative realization of a uniform DFT filter bank.

T h e output o f each bandpass filter is d ecim ated by a factor D and m ultiplied by

e x p i j l n m k / N ) to p roduce the D F T seq u en ce {X*(m)}. T h e m odu lation by the
com plex exp on en tial allow s us to shift the spectrum o f the signal from
= 2nk/N
to oo = 0. H en ce this realization is equ ivalen t to the realization given in Fig. 10.33.
T h e filter bank output can b e written as

X k( m) =

j c ( n ) * o ( m D - n ) e j2nk(mD~n)/N

"-jlnmkD/N

(10.9.7)

T h e corresponding filter bank syn th esizer can be realized as show n in

Fig. 10.34b, w h ere the input seq u en ces are first m ultiplied by the exp on en tial
factors [ e x p ( j 2 n k m D / N) ] , upsam pled by the factor I = D, and the resulting se-

Sec. 10.9

Applications of Multirate Signal Processing

829

q u e n c e s a re filtered by th e b a n d p a s s in te rp o la tio n filters w ith im p u lse re sp o n se s

g k ( n) = g 0( n ) e j2j!nk/N

(10.9.8)

w h e re {o()) is th e im p u lse re sp o n se o f th e p r o to ty p e filter. T h e o u tp u ts o f th e se

filters are th e n su m m e d to yield
N-1
1
< " ) = j j i .

(10.9.9)

k=0

w h e re I D.
In th e im p le m e n ta tio n of d igital filters b a n k s, c o m p u ta tio n a l efficiency can be
ac h iev ed by u se o f p o ly p h a se filters fo r d e cim atio n a n d in te rp o la tio n . O f p a rtic u la r
in te re st is th e case w h e re th e d e c im a tio n fa c to r D is se le c te d to be e q u a l to th e
n u m b e r N o f fre q u e n c y b an d s. W h en D = N, w e say th a t th e filter b a n k is critically
sampled.
F o r th e an aly sis filter b an k , let us define a se t o f N = D p o ly p h a se filters
w ith im p u lse resp o n ses
p t ( n ) = ho ( n N k)

k = 0. 1.........jV 1

(10.9.10)

a n d th e c o rre sp o n d in g se t o f d e c im a te d in p u t se q u e n c e s
x k (n) = x ( n N + k)

* = 0 . 1 .........N - 1

(10.9.11)

N o te th a t th is d efin itio n o f {/;*()) im plies th a t th e c o m m u ta to r fo r th e d e c im a to r

r o ta te s clockw ise.
T h e s tru c tu re o f th e analysis filter b an k b ased o n th e use o f p o ly p h ase filters
can b e o b ta in e d by su b stitu tin g (10.9.10) a n d (10,9.11) in to (10.9.7) a n d re a rra n g in g
th e s u m m a tio n in to th e fo rm
-jZ n n k / N

k - 0 , 1 ____ D - 1

(10.9.12)

w h e re N = D. N o te th a t th e in n e r su m m a tio n r e p re s e n ts th e co n v o lu tio n o f
i p n(l)} w ith {*(/)). T h e o u te r su m m atio n re p re s e n ts th e W -point D F T o f th e
filter o u tp u ts. T h e filter s tru c tu re c o rre sp o n d in g to th is c o m p u ta tio n is illu strated
in Fig. 10.35. E a c h sw eep o f th e c o m m u ta to r re su lts in N o u tp u ts, d e n o te d as
( r ( m ), n = 0, 1.........A ' - l )
from th e N p o ly p h ase filters. T h e / V -p o in t D F T o f
th is se q u e n c e yield s th e sp e c tra l sa m p le s (Jft(m )]. F o r la rg e valu es o f N , th e F F T
alg o rith m p ro v id e s an efficient m e a n s fo r co m p u tin g th e D F T .
N o w su p p o se th a t th e sp e ctral sa m p le s { ^ ( m ) } a re m o d ified in so m e m a n n e r,
p re sc rib e d by th e ap p lic a tio n , to p ro d u c e |y t.(m)}. A filter b a n k sy n th e sis filter
b a s e d o n a p o ly p h a se filter s tru c tu re can b e re a liz e d in a sim ilar m a n n e r. F irst,
w e d efin e th e im p u lse re sp o n se o f th e N {D = I N ) p o ly p h a se filters fo r th e
in te rp o la tio n filter as
qk(n) = go( nN + k)

k = 0 ,1 ........N - 1

(10.9.13)

830

Multirate Digital Signal Processing

Figure 1035

Chap. 10

Digital filter bank structure for the computation of (10.9.12).

a n d th e c o rre sp o n d in g set o f o u tp u t signals as

vt () = v(n N + k)

* = 0 , 1 .........A ' - l

(10.9.14)

N o te th a t th is d efin itio n o f {qt(n)} im plies th a t th e c o m m u ta to r fo r th e in te rp o la to r

r o ta te s co u n terclo ck w ise .
B y su b stitu tin g (10.9.13) in to (10.9.5), w e can ex p ress th e o u tp u t v/(n) o f th e
/th p o ly p h a se filter as
j N -\
V i(n)

^ < ? /(

m )

-]T n(m )e

j2Kkl/H

i =0,1...., N - 1

(10.9.15)

T h e te rm in b ra c k e ts is th e /V -point in v e rse D F T o f {% ("*)}, w h ich w e d e n o te as

{y; (m)}. H en ce
Vi (tt) = ^

qt(n m) yi {m)

I 0, 1, . .. , N 1

(10.9.16)

T h e sy n th esis s tru c tu re c o rre sp o n d in g to (10.9.16) is sh o w n in Fig. 10.36. It is

in te re stin g to n o te th a t by d efin in g th e p o ly p h a s e in te rp o la tio n filte r as in (10.9.13),
th e stru c tu re in Fig. 10.36 is th e tra n sp o s e o f th e p o ly p h a se an aly sis filter show n
in Fig. 10.35.
In o u r tr e a tm e n t o f d ig ital filter b a n k s w e c o n s id e re d th e im p o rta n t case o f
critically sa m p le d D F T filter b a n k s, w h e re D ~ N . O th e r ch o ic e s o f D a n d N
c an b e e m p lo y ed in p ra c tic e , b u t th e im p le m e n ta tio n o f th e filters b e c o m e s m o re
c o m p lex . O f p a rtic u la r im p o rta n c e is th e o v e rsa m p le d D F T filte r b a n k , w h ere
N = K D, D d e n o te s th e d e c im a tio n f a c to r a n d K is an in te g e r th a t specifies the

Sec. 10.9

Applications of Multirate Signal Processing

Figure 10,36

831

Digital filter bank structure for the computation of (10.9.16).

o v e rsam p lin g facto r. In this case it can be sh o w n t hat th e p o ly p h ase filter b a n k

s tru c tu re s for th e an alysis an d sy n th e sis filters can be im p le m e n te d by use o f N
su b filters a n d /V-point D F T s an d in v erse D F T s.

10.9.5 Subband Coding of Speech Signals

A v ariety o f te c h n iq u e s have b e e n d e v e lo p e d to efficiently r e p re s e n t sp e ech signals
in dig ital fo rm fo r e ith e r tran sm issio n o r sto ra g e . S ince m o st o f th e sp e ech en e rg y
is c o n ta in e d in th e lo w er freq u e n cies, w e w ould like to e n c o d e th e lo w er-freq u en cy
b a n d w ith m o re b its th a n th e h ig h -fre q u e n c y b a n d . S u b b a n d coding is a m e th o d ,
w h ere th e sp e ech signal is su b d iv id ed in to se v e ra l fre q u e n c y b an d s a n d each b a n d
is d ig itally e n c o d e d se p a ra te ly .
A n e x am p le o f a fre q u e n c y su b d iv isio n is show n in Fig. 10.37a. L e t us a s
su m e th a t th e sp eech signal is sa m p le d a t a ra te Fs sa m p le s p e r se co n d . T h e
first fre q u e n c y su b d iv ision splits th e signal sp e c tru m in to tw o eq u al-w id th se g
m e n ts, a lo w p ass sig n al (0 < F < Fxj 4) a n d a h ig h p ass signal (Fs/ 4 < F < Fs / 2).
T h e se co n d fre q u e n c y subdivision sp lits th e low pass signal from th e first stag e
in to tw o e q u a l b an d s, a low pass signal (0 < F < Fs/ 8) an d a h ig h p ass signal
( Fs/& < F < F J 4). F inally, th e th ird fre q u e n c y su b d iv isio n splits th e low pass
signal fro m th e se co n d sta g e in to tw o e q u a l b a n d w id th signals. T h u s th e sig
n a l is su b d iv id e d in to fo u r fre q u e n c y b an d s, co v erin g th re e o ctav es, as sh o w n in
Fig. 10.37b.
D e c im a tio n by a fa c to r o f 2 is p e rfo rm e d a fte r fre q u e n c y su b division. By
a llo catin g a d iffe re n t n u m b e r o f b its p e r sa m p le to th e signal in th e fo u r su b b a n d s,
w e can a c h iev e a re d u c tio n in th e b it ra te o f th e d ig italized sp e ech signal.

832

Multirate Digital Signal Processing

Chap. 10

(a)

j 1
0

JT

JT

4
7T

[
JT

2
(b)

Figure 10.37

Block diagram of a subband speech coder.

F ilte r d esign is p a rtic u la rly im p o rta n t in ach iev in g goo d p e rfo rm a n c e in s u b

b a n d coding. A liasin g re su ltin g fro m d e c im a tio n o f th e su b b a n d signals m u st be
n eg ligible. It is c le a r th a t w e c a n n o t use brickw all filter c h a ra c te ristic s as sh o w n in
Fig. 10.38a, since such filters a re ph y sically u n re a liz a b le . A p a rtic u la rly p ractical
so lu tio n to th e aliasin g p ro b le m is to u se quadrat ur e mi r r or filters (Q M F ), w hich
h a v e th e fre q u e n c y re sp o n se c h a ra c te ristic s show n in Fig. 10.38b. T h e se filters are
d e sc rib e d in th e fo llo w ing sectio n .
T h e syn th esis m e th o d for th e su b b a n d e n c o d e d sp e ech signal is b asically th e
rev erse o f th e e n c o d in g p ro cess. T h e signals in a d ja c e n t lo w p ass a n d h ig h p ass
freq u e n cy b an d s a re in te rp o la te d , filte re d , a n d c o m b in e d as sh o w n in Fig. 10.39.
A p a ir o f Q M F is u se d in th e signal sy n th e sis fo r ea c h o ctav e o f th e signal.
S u b b a n d co d in g is also an effe ctiv e m e th o d to ach iev e d a ta co m p re ssio n in
im ag e signal p ro cessin g . B y c o m b in in g su b b a n d co d in g w ith v e c to r q u a n tiz a tio n
fo r each su b b a n d sig n al, S a fra n e k e t al. (1988) h av e o b ta in e d c o d e d im ag es w ith
a p p ro x im a te ly ^ b it p e r pixel, c o m p a re d w ith 8 b its p e r p ix el fo r th e u n c o d e d
im age.
In g e n eral, s u b b a n d co d in g o f signals is an effe ctiv e m e th o d fo r achieving
b an d w id th c o m p ressio n in a d ig ital r e p re s e n ta tio n o f th e signal, w h en th e signal
e n erg y is c o n c e n tra te d in a p a rtic u la r re g io n o f th e fre q u e n c y b a n d . M u ltira te
signal p ro cessin g n o tio n s p ro v id e efficient im p le m e n ta tio n s o f th e su b b a n d e n
co d er.

Sec. 10.9

Applications of Multirate Signal Processing

833

(a)

QMF
(b)
Figure 10.38

Filter characteristics for subband coding.

10.9.6 Quadrature Mirror Filters

T h e b asic b u ild in g b lo c k in a p p licatio n s o f q u a d r a tu re m irro r filters (Q M F ) is
th e tw o -c h a n n e l Q M F b a n k show n in Fig. 10.40. T h is is a m u ltira te digital filter
s tru c tu re th a t em p lo y s tw o d e c im a to rs in th e signal an a ly sis se c tio n a n d tw o
in te r p o la to r s in th e sig nal sy n th e sis sectio n . T h e lo w p ass a n d h ig h p ass filters in
th e an aly sis se c tio n h a v e im p u lse resp o n ses ho(n) a n d h \ ( n ) , resp ec tiv ely . S im ilarly,
th e lo w p ass a n d h ig h p ass filters c o n ta in e d in th e sy n th e sis se c tio n h a v e im p u lse
re sp o n s e s go(n) a n d g i( n ), respectively.

834

Multirate Digital Signal Processing

Chap. 10

Outpa

Figure 10J9

Synthesis o f subband-encoded signals.

Analysis
section
Figure 10.40

Synthesis
section
Two-channel QM F bank.

T h e F o u rie r tra n sfo rm s o f th e sig n a ls a t th e o u tp u ts o f th e tw o d ecim ato rs

X a0(co) = -

(10.9.17)
X aX(co) = -

If Xsq(m) and Xlt(<u) represent the two inputs to the synthesis section, the output

Sec. 10.9

Applications of Multirate Signal Processing

835

is sim ply
X(co) = Xfo(2(o)Go(co) + X s\(2(o)G\(co)

(10.9.18)

N o w , su p p o s e th a t we c o n n ect th e analysis filter to th e c o rre sp o n d in g sy nthesis

filter, so th a t X ao{(o) =
a n d X a\(oo) = -X*i(ew). T h e n , by su b stitu tin g fro m
(10.9.17) in to (10.9.18), w e o b ta in
X(co) = ^ \ H q{(o ) G q((o ) + H\(a))G\{( d ) ]X {w)

+ \[H0(u>7t)Go(<o) + Hi(co 7t)G\(u))]X(o} tt)

(10.9.19)

T h e first te rm in (10.9.19) is th e d e sire d signal o u tp u t fro m th e Q M F b an k .

T h e seco n d te rm re p re s e n ts th e effect o f aliasing, w hich w e w o u ld like to elim in ate.
H e n c e w e re q u ire th a t
H 0((o - n)Gu(a>) + Hi(co -

7t)Gx(co) =

(10.9.20)

T h is co n d itio n can be sim ply satisfied by selectin g Go(co) a n d G i M as

G q(cd) = Hiico -

tt)

G x(a>) = - H a( o > - n )

(10.9.21)

T h u s th e se co n d term in (10.9.19) vanishes.

T o e la b o ra te , let us assum e th a t Hu(co) is a lo w p ass filter an d H x(io) is a
m irro r-im a g e h ig h p ass filter. T h e n w e can e x p ress Hti(u>) a n d H\(u>) as
H0(a>) = H(to)
(10.9.22)
H\ (w) = H ( oj tt)
w h e re H( w) is th e fre q u e n c y re sp o n se o f a lo w p ass filter. In th e tim e d o m ain , th e
c o rre sp o n d in g re la tio n s a re
hnin) = h(n)
(10.9.23)
h\ ( n) = ( - 1 )nh( n)
A s a c o n s e q u e n c e , Hn(a>) a n d H x(u>) h ave m irro r-im a g e sy m m e try a b o u t th e fre
q u e n c y a) tt/2, as sh o w n in Fig. 10.38b. T o b e c o n siste n t w ith th e c o n s tra in t in
(10.9.21), w e se lect th e low pass filter G0(to) as
G 0(co) = 2H(co)

(10.9.24)

G x (co) = - 2 H(co - n )

(10.9.25)

a n d th e h ig h p ass filter G x(a>) as

In th e tim e d o m ain , th e s e re la tio n s b eco m e

go(n) = 2h( n)
(10.9.26)
g\( n) = 2 (1 )nh{n)
T h e scale fa c to r o f 2 in g0(n) a n d g x(n) c o rre sp o n d s to th e in te rp o la tio n fa c to r
u se d to n o rm a liz e th e o v erall fre q u e n c y re sp o n s e o f th e Q M F . W ith th is choice o f

836

Multirate Digital Signal Processing

Chap. 10

th e filter c h aracteristics, th e c o m p o n e n t d u e to aliasin g v an ish es. T h u s th e aliasing

resu ltin g fro m d e c im a tio n in th e analysis se ctio n o f th e Q M F b a n k is p e rfe c tly
can ce le d by th e im ag e signal sp e c tru m th a t arises d u e to in te rp o la tio n . A s a
resu lt, th e tw o -ch an n el Q M F b e h a v e s as a lin ear, tim e -in v a ria n t system .
If we su b s titu te fo r Ho(a>), H-\(co), Go(co), a n d Gi(a>) in to th e first te rm o f
(10.9.19), w e o b ta in
X(co) = [ H 2(co) - H 2(u) -

tt) ] X ( co)

(10.9.27)

Id eally , th e tw o -c h a n n e l Q M F b a n k sh o u ld h ave u n ity gain,

\ H 2{oj) H 2(co n ) \ = 1

fo r all co

(10.9.28)

w h ere H(co) is th e fre q u e n c y re sp o n se o f a low pass filter. F u rth e rm o r e , it is also

d e s ira b le fo r th e Q M F to h av e lin e a r p h ase.
N ow , let u s c o n sid e r th e use o f a lin e a r p h a se filter H(co). H e n c e H(co) m ay
be ex p ressed in th e form
H ( w ) = Hr (co)e~Jai<N- h/2

(10.9.29)

w h ere N is th e filter len g th . T h en

H 2 (co) = H 2( w ) e - j,' N - h

(10.9.30)
=

\ H ( w ) \ 2e~J(N- ])

an d
H 2( a>- tt) = H ? ( o > - j r ) e - jia,- ,niN- ' )
(10.9.31)
= ( - l ) N~ l \H(co - jz)\2e - JwiN- l)
T h e re fo re , th e o v erall tra n s f e r fu n ctio n o f th e tw o -ch an n el Q M F w hich em p lo y s
lin e a r-p h a se F IR filters is

X(co)

= [\H(a>)\2 - ( - l f - ]\ H( c o - 7 T ) \ 2] e - JU,'N- ])

(10.9.32)

N o te th a t th e o v erall filter h a s a d elay o f N 1 sa m p le s an d a m a g n itu d e c h a ra c

te ristic
A(co) = \H(co)\2 - { - \ ) N- ' \ H { c o - n ) \ 2

(10.9.33)

W e also n o te th a t w h en N is o d d , A ( n f l ) = 0, b e c a u se \H{7ijT)\ = |// ( 3 j t/ 2 ) |.

T h is is an u n d e sira b le p r o p e rty fo r a Q M F design. O n th e o th e r h a n d , w h e n N is
ev en ,
A((D) = \H{co)\2 + \ H { c o - n ) \ 2

(10.9.34)

w h ich av o id s th e p ro b le m o f a z e ro at a) = n f l . F o r N ev en , th e id e a l tw o -ch an n el
Q M F sh o u ld satisfy th e c o n d itio n
A(<d) = \ H( w) \ 2 -I-1H(C0 - n ) I2 = 1

for all to

(10.9.35)

Sec. 10.9

Applications of Multirate Signal Processing

837

w h ich fo llo w s fro m (10.9.33). U n fo rtu n a te ly , th e only filter fre q u e n c y re sp o n se

fu n c tio n th a t satisfies (10.9.35) is th e trivial fu n ctio n \H(u>)\2 = co s2 aw. C o n s e
q u e n tly , a n y n o n triv ia l lin e a r-p h a se F IR filter H(a>) in tro d u c e s so m e a m p litu d e
d isto rtio n .
T h e a m o u n t o f a m p litu d e d isto rtio n in tro d u c e d by a n o n triv ia l lin e a r p h a s e
F IR filter in th e Q M F c a n be m in im ized by o p tim izin g th e F IR filter coefficients.
A p a rtic u la rly effe ctiv e m e th o d is to select th e filter coefficients o f H(<o) such th a t
A ((d) is m a d e as flat as p o ssib le w hile sim u ltan eo u sly m in im iz in g ( o r c o n strain in g )
th e s to p b a n d en erg y o f H(u>). T his a p p ro a c h lead s to th e m in im iz atio n o f th e
in te g ra l sq u a re d e r ro r
(10.9.36)
w h e re w; is a w eig h tin g fa c to r in th e ran g e 0 < w < 1. In p e rfo rm in g th e o p ti
m izatio n , th e filter im p u lse re sp o n se is c o n s tra in e d to b e sy m m e tric (lin e a r p h a se ).
T h is o p tim iz a tio n is easily d o n e n u m erically on a d igital c o m p u te r. T h is a p p ro a c h
h as b e e n u se d by J o h n s to n (1980), an d Ja in a n d C ro c h ie re (1984) to design tw och a n n e l Q M F s. T a b le s o f o p tim u m filter coefficients h ave b e e n ta b u la te d by J o h n
sto n (1980).
A s an a lte rn a tiv e to lin e a r-p h a se F IR filters, w e can design an I I R filter th a t
satisfies th e all-p ass c o n s tra in t given by (10.9.28). F o r th is p u rp o s e , e llip tic filters
p ro v id e esp ecially efficient designs. Since th e Q M F w o u ld in tro d u c e so m e p h ase
d isto rtio n , th e signal a t th e o u tp u t o f th e Q M F can be p a sse d th ro u g h an all-pass
p h a s e e q u a liz e r d esig n e d to m inim ize p h ase d isto rtio n .
In a d d itio n to th e s e tw o m e th o d s fo r Q M F design, o n e can also design th e
tw o -c h a n n e l Q M F s to e lim in a te co m p letely b o th a m p litu d e a n d p h a s e d isto rtio n
as w ell as c a n c e lin g aliasin g d isto rtio n . Sm ith a n d B a rn w e ll (1984) h a v e show n
th a t su ch p er f ect reconstruct i on Q M F can be d esig n ed by relax in g th e lin e a r-p h a se
co n d itio n o f th e F IR lo w pass filter H{(o). T o ach iev e p e rfe c t re c o n stru c tio n , w e
b eg in by d esig n in g a lin e a r-p h a se F IR h a lfb a n d filter o f le n g th 2 N 1.
A h a lf-b a n d filter is defin ed as a z e ro -p h a se F I R filter w h o se im p u lse resp o n se
{b(n}} satisfies th e c o n d itio n
b(2n) =

co n stan t,
0,

n = 0
n 7^0

(10.9.37)

H e n c e all th e e v e n -n u m b e re d sa m p le s are z e ro e x c e p t a t n = 0. T h e z e ro -p h a se
r e q u ire m e n t im p lies th a t b( n) = b( n). T h e fre q u e n c y re sp o n s e o f such a filter is
K
(10.9.38)
w h ere K is o d d . F u rth e rm o r e , B(co) satisfies th e c o n d itio n B( w) + B(7i ~ co) is e q u a l
to a c o n s ta n t fo r all freq u e n cies. T h e typical fre q u e n c y re sp o n s e c h a ra c te ristic o f a
h a lf-b a n d filter is sh o w n in Fig. 10.41. W e n o te th a t th e filter re sp o n s e is sym m etric
w ith re sp e c t to n j 2, th e b a n d e d g es freq u e n cies cup a n d a)s a re sy m m etric a b o u t

838

Multirate Digital Signal Processing

Chap. 10

w = n / 2 , a n d th e p e a k p a s s b a n d a n d s to p b a n d e r r o r s a r e e q u a l. W e a ls o n o te th a t
th e f ilte r c a n b e m a d e c a u s a l b y in tr o d u c in g a d e la y o f K s a m p le s .
N o w , s u p p o s e th a t w e d e s ig n an F I R h a lf- b a n d filte r o f le n g th 2jV 1, w h e r e

N is e v e n , w ith f r e q u e n c y r e s p o n s e a s s h o w n in F ig . 1 0 .4 2 ( a ) .

F ro m

B(a>) w e

c o n s t r u c t a n o th e r h a lf-b a n d filte r w ith fr e q u e n c y r e s p o n s e

B+(w) = B(co) +
as s h o w n in F ig . 1 0 .4 2 ( b ) .

( 1 0 .9 .3 9 )

N o t e th a t B +(a>) is n o n n e g a tiv e a n d h e n c e it h a s th e

s p e c tr a l fa c to r iz a tio n
fl+ ( z ) = H { z ) H { z - \ - (N~ h

( 1 0 .9 .4 0 )

B+(a>) = \H(co)\2e~ ja>(N~1)

(1 0 .9 .4 1 )

o r , e q u iv a le n tly ,

w h e r e H (to) is th e f r e q u e n c y r e s p o n s e o f a n F I R

f ilt e r o f le n g th N

w ith re a l

c o e f f ic ie n ts . D u e t o th e s y m m e tr y o f B+(co ) w ith r e s p e c t to co = n / 2 , w e a ls o h a v e

B+(z) + ( - 1 ) " - ' B + i - z ) = c t z - ^ - V

( 1 0 .9 .4 2 )

o r , e q u iv a le n tly ,

B+{a>)

-I- ( - D ^ f l + O w -

n) = a

( 1 0 .9 .4 3 )

w h e r e a is a c o n s t a n t . T h u s , b y s u b s t itu tin g ( 1 0 .9 .4 0 ) in to ( 1 0 .9 .4 2 ) , w e o b ta in

H(z)H(z~l) + H ( - z ) H ( ~ z ~ l ) = a

( 1 0 .9 .4 4 )

S in c e H ( z ) s a tis fie s ( 1 0 .9 .4 4 ) a n d s in c e a lia s in g is e lim in a te d w h e n w e h a v e G o (z ) =

H \ ( z) a n d G i ( z ) =

H q ( z ) , it fo llo w s t h a t t h e s e c o n d itio n s a r e s a tis fie d b y

Sec. 10.9

Applications of Multirate Signal Processing

839

Bffft)

amplitude
response of
G(z)

-S

BJta)
amplitude
response of
* (*)

Figure 10.42 Frequency response characteristics of half-band filters B(a>) and

B+(<u>). (From Vaidyanathan (1987))

840

Multirate Digital Signal Processing

Chap. 10

c h o o s in g H \ ( z ) , G o ( z ) , a n d G \ ( z ) as

H o (z) = H { z)
H \{ z ) = - z - iN - h H 0( - Z ~ ')
( 1 0 .9 .4 5 )

G 0{z) = z - (N- u H q {z - 1)
G ,( z ) = z ^ - ^ H ^ r 1 ) = - H o ( - z )
T h u s a lia s in g d is to r tio n is e lim in a te d a n d s in c e X { a > ) /X { a ) ) is a c o n s t a n t , t h e Q M F
p e r f o r m s p e r f e c t r e c o n s tr u c tio n s o t h a t x ( n ) = a x ( n N + 1 ). H o w e v e r , w e n o te
th a t H ( z ) is n o t a l i n e a r -p h a s e f ilte r .
T h e F I R filte r s H o (z), H ] ( z ), G o (z ), a n d G i ( z ) in th e tw o - c h a n n e l Q M F b a n k
a r e e f fic ie n tly r e a liz e d as p o ly p h a s e filte r s . S in c e I = D = 2 , tw o p o ly p h a s e filte r s
a r e im p le m e n te d f o r e a c h d e c im a t o r a n d tw o f o r e a c h in te r p o la to r . H o w e v e r , if
w e e m p lo y lin e a r -p h a s e F I R filte r s , th e s y m m e tr y p r o p e r tie s o f th e a n a ly s is filte r s
a n d s y n th e s is f ilte r s a llo w u s to s im p lify th e s tr u c tu r e a n d r e d u c e th e n u m b e r o f
p o ly p h a s e filte r s in th e a n a ly sts s e c tio n t o tw o filte r s a n d t o a n o t h e r tw o f ilte r s in
th e s y n th e s is s e c tio n .
T o d e m o n s tr a t e th is c o n s t r u c tio n , le t u s a s s u m e th a t th e filte r s a r e lin e a r p h a s e F I R filte r s o f le n g th N

( N e v e n ) , w h ich h a v e im p u ls e r e s p o n s e s g iv e n by

( 1 0 .9 .2 3 ) . T h e n th e o u tp u ts o f th e a n a ly s is f ilte r p a ir , a f t e r d e c im a tio n b y a fa c to r
o f 2 , ca n b e ex p ressed as

X a k (m ) =

Y ^ ( l ) knh ( n ) x ( 2 m - n )

* = 0 ,1

f l = OC
1

OO

=
i'= 0

( - 1 )k<l2J+l)h { 2 l + l) x ( 2 m - 2 1 - i)

l=oc
N \

N -1

J2

h ( 2 i) x ( 2 m - H ) + ( - 1 ) *

1=0

Y i^ (2 !

+ l) x ( 2 m - 2 1 - 1 )

( 1 0 .9 .4 6 )

1 =0

N o w le t u s d e fin e th e im p u ls e r e s p o n s e o f tw o p o ly p h a s e filte r s o f le n g th N ( 2 as
P i i m ) = h {2m + i)

i = 0 ,1

( 1 0 .9 .4 7 )

T h e n ( 1 0 .9 .4 6 ) c a n b e e x p r e s s e d as
N/2-

X ak{m) =

Y2

P o (m )x (2 (m - /))

1=0

( 1 0 .9 .4 8 )
AT/2-1

P \{ m ) x ( 2 m 2 1 1 )

+ ( 1 )*

Jt = 0 , 1

1=0

T h is e x p r e s s io n c o r r e s p o n d s t o th e p o ly p h a s e f ilt e r s tr u c tu r e f o r th e a n a ly s is
s e c t i o n sh o w n in F ig . 1 0 .4 3 . N o t e t h a t t h e c o m m u t a t o r r o t a t e s c o u n te r c lo c k w is e

Sec. 10.9

841

Applications of Multirate Signal Processing

A nalysis____________
section
F ig u re 10.43

11

___________ Synthesis
section

Polyphase filter structure for the Q M F bank.

a n d th a t th e filte r w ith im p u ls e r e s p o n s e (p o (ro )} p r o c e s s e s th e e v e n - n u m b e r e d

s a m p le s o f th e in p u t s e q u e n c e a n d th e f ilte r w ith im p u ls e r e s p o n s e {/ ?i(m )} p r o
c e s s e s th e o d d -n u m b e r e d s a m p le s o f th e in p u t s ig n a l.
In a s im ila r m a n n e r , b y u sin g ( 1 0 .9 .2 6 ) , w e c a n o b ta in th e s tr u c tu r e f o r th e
p o ly p h a s e s y n th e s is s e c tio n , w h ich is a ls o sh o w n in F ig . 1 0 .4 3 . T h is d e r iv a tio n is
le ft a s an e x e r c is e fo r th e r e a d e r (P r o b le m 1 0 .1 6 ) , N o te t h a t th e c o m m u t a t o r a ls o
r o ta te s c o u n te r c lo c k w is e .
F in a lly , w e o b s e r v e th a t th e p o ly p h a s e filte r s tr u c tu r e s h o w n in F ig . 1 0 ,4 3 is
a p p r o x im a te ly fo u r tim e s m o r e e f fic ie n t th a n th e d ir e c t -fo r m F I R f ilte r r e a liz a tio n .

10.9.7 Transmultiplexers
A n o t h e r a p p lic a t io n o f m u ltir a te s ig n a l p r o c e s s in g is in t h e d e s ig n a n d im p le m e n
t a tio n o f d ig ita l tr a n s m u lt ip le x e r s w h ic h a r e d e v ic e s f o r c o n v e r tin g b e tw e e n tim e d iv is io n -m u ltip le x e d

( T D M ) s ig n a ls a n d f r e q u e n c y - d iv is io n - m u ltip le x e d

(F D M )

s ig n a ls .
In a tr a n s m u lt ip le x e r fo r T D M - t o - F D M c o n v e r s io n , t h e in p u t s ig n a l {jr(/?)}
is a tim e -d iv is io n m u ltip le x e d s ig n a l c o n s is tin g o f L s ig n a ls , w h ich a r e s e p a r a te d
b y a c o m m u t a t o r s w itc h . E a c h o f th e s e L s ig n a ls a r e th e n m o d u la te d o n d if f e r e n t
c a r r i e r f r e q u e n c ie s t o o b t a in an F D M s ig n a l f o r tr a n s m is s io n . In a t r a n s m u lt ip le x e r
fo r F D M - t o - T D M

c o n v e r s io n , th e c o m p o s ite s ig n a l is s e p a r a te d b y filte r in g in to

t h e L s ig n a l c o m p o n e n ts w h ic h a r e th e n tim e -d iv is io n m u ltip le x e d .
I n te le p h o n y , s in g le - s id e b a n d tr a n s m is s io n is u s e d w ith c h a n n e ls s p a c e d a t
a n o m in a l 4 - k H z b a n d w id th .

T w e lv e c h a n n e ls a r e u s u a lly s ta c k e d in f r e q u e n c y

t o f o r m a b a s ic g ro u p c h a n n e l, w ith a b a n d w id th o f 4 8 k H z .

L a r g e r b a n d w id th

F D M s ig n a ls a r e f o r m e d b y f r e q u e n c y tr a n s la tio n o f m u ltip le g ro u p s in to a d ja c e n t
f r e q u e n c y b a n d s . W e s h a ll c o n fin e o u r d is c u s s io n to d ig ita l t r a n s m u lt ip le x e r s f o r
1 2 - c h a n n e l F D M a n d T D M s ig n a ls .
L e t us firs t c o n s id e r F D M - t o - T D M c o n v e r s io n .

T h e a n a lo g F D M s ig n a l is

p a s s e d th r o u g h an A / D c o n v e r t e r a s sh o w n in F ig . 1 0 .4 4 a . T h e d ig ita l s ig n a l is th e n

842

Multirate Digital Signal Processing

Chap. 10
TD M

(a)

(b)
Figure 10.44 Block diagram of FDM-to-TDM transmultiplexer.
d e m o d u la te d t o b a s e b a n d b y m e a n s o f s in g le -s id e b a n d d e m o d u la to r s . T h e o u tp u t
o f e a c h d e m o d u la to r is d e c im a te d a n d fe d to c o m m u t a t o r o f t h e T D M s y s te m .
T o b e s p e c ific , le t us a s s u m e t h a t th e 1 2 -c h a n n e l F D M s ig n a l is s a m p le d a t
th e N y q u is t r a t e o f 9 6 k H z a n d p a s s e d th r o u g h a f ilte r - b a n k d e m o d u la to r .

The

b a s ic b u ild in g b lo c k in th e F D M d e m o d u la to r c o n s is ts o f a f r e q u e n c y c o n v e r te r , a
lo w p a s s filte r , a n d a d e c im a to r , a s illu s tr a te d in F ig . 1 0 .4 4 b . F r e q u e n c y c o n v e r s io n
ca n b e e ff ic ie n tly im p le m e n te d b y th e D F T f ilte r b a n k d e s c r ib e d p r e v io u s ly . T h e
lo w p a s s filte r a n d d e c im a to r a r e e ff ic ie n tly i m p le m e n te d b y u s e o f th e p o ly p h a s e
filte r s tr u c tu r e . T h u s th e b a s ic s tr u c tu r e f o r th e F D M - t o - T D M c o n v e r t e r h a s th e
f o r m o f a D F T f ilte r b a n k a n a ly z e r . S in c e th e s ig n a l in e a c h c h a n n e l o c c u p ie s a
4 - k H z b a n d w id th , its N y q u is t r a t e is 8 k H z , a n d h e n c e th e p o ly p h a s e f ilte r o u t
p u t c a n b e d e c im a te d b y a f a c t o r o f 1 2 . C o n s e q u e n tly , th e T D M c o m m u t a t o r is
o p e r a tin g a t a r a t e o f 1 2 x 8 k H z o r 9 6 k H z .
In T D M - t o - F D M c o n v e r s io n , th e 1 2 -c h a n n e l T D M s ig n a l is d e m u ltip le x e d
in to th e 1 2 in d iv id u a l s ig n a ls , w h e r e e a c h s ig n a l h a s a r a t e o f 8 k H z . T h e sig n a l

Sec. 10.9

Applications of Multirate Signal Processing

843

Figure 10.45 Block diagram of TDM-to-FDM transmultiplexer.

in e a c h c h a n n e l is in te r p o la te d b y a fa c t o r o f 1 2 a n d -fre q u e n c y c o n v e r te d b y a
s in g le -s id e b a n d m o d u la to r , as s h o w n in F ig . 1 0 .4 5 . T h e s ig n a l o u tp u ts fro m th e 1 2
s in g le -s id e b a n d m o d u la to r s a r e s u m m e d an d fe d to th e D / A c o n v e r te r . T h u s w e
o b ta in th e a n a lo g F D M s ig n a l fo r tr a n s m is s io n . A s in th e c a s e o f F D M - t o - T D M
c o n v e r s io n , th e in t e r p o la t o r an d th e m o d u la to r f ilte r a r e c o m b in e d a n d e f f ic ie n tly
im p le m e n te d b y u se o f a p o ly p h a s e filte r . T h e f r e q u e n c y tr a n s la tio n c a n b e a c
c o m p lis h e d b y th e D F T . C o n s e q u e n tly , th e T D M - t o - F D M c o n v e r t e r e n c o m p a s s e s
th e b a s ic p r in c ip le s in tr o d u c e d p r e v io u s ly in o u r d is c u s s io n o f D F T f ilte r b a n k
s y n th e s is .

10.9.8 Oversampling A/D and D/A Conversion

O u r tr e a t m e n t o f o v e r s a m p lin g A / D a n d D / A c o n v e r te r s in C h a p te r 9 p r o v id e s
a n o t h e r e x a m p le o f m u ltir a te s ig n a l p r o c e s s in g . R e c a l l t h a t a n o v e r s a m p lin g A /D
c o n v e r t e r is im p le m e n te d b y a c a s c a d e o f a n a n a lo g s ig m a - d e lta m o d u la to r ( S D M )
fo llo w e d b y a d ig ita l a n tia lia s in g d e c im a tio n filte r a n d a d ig ita l h ig h p a s s f ilte r a s
s h o w n in F ig . 1 0 .4 6 . T h e a n a lo g S D M p r o d u c e s a 1 -b it p e r s a m p le o u tp u t a t a v e r y
h ig h s a m p lin g r a t e . T h is 1 -b it p e r s a m p le o u tp u t is p a s s e d th ro u g h a d ig ita l lo w p a ss
f ilte r , w h ic h p r o v id e s a h ig h -p r e c is io n ( m u ltip le -b it) o u tp u t th a t is d e c im a te d to
a lo w e r s a m p lin g r a t e . T h is o u tp u t is th e n p a s s e d to a d ig ita l h ig h p a s s f ilte r th a t
s e r v e s to a t te n u a te th e q u a n tiz a tio n n o is e a t th e lo w e r fr e q u e n c ie s .
T h e r e v e r s e o p e r a tio n s t a k e p la c e in a n o v e r s a m p lin g D / A c o n v e r te r , a s
s h o w n in F ig . 1 0 .4 7 . A s illu s tr a te d in th is fig u re , th e d ig ita l s ig n a l is p a s s e d th ro u g h
a h ig h p a s s f ilt e r w h o s e o u tp u t is fe d t o a d ig ita l in t e r p o l a t o r (u p s a m p le r a n d a n t i
im a g in g f ilte r ) . T h is h ig h -s a m p lin g -r a te s ig n a l is th e in p u t t o th e d ig ita l S D M th a t
p r o v id e s a h ig h -s a m p lin g -r a te 1 -b it p e r s a m p le o u tp u t. T h e 1 - b it p e r s a m p le o u tp u t

FigDre 10.46 Diagram of oversampling A/D converter

844

Multirate Digital Signal Processing

precision digital signal

Chap. 10

per sam ple

Figure 10.47 Diagram of oversampling D/A converter

is th e n c o n v e r te d to a n a n a lo g s ig n a l b y lo w p a s s filte r in g a n d f u r t h e r s m o o th in g
w ith a n a lo g filte r s .
F ig u r e 1 0 .4 8 illu s tr a te s t h e b lo c k d ia g ra m o f a c o m m e r c ia l ( A n a l o g D e v ic e s
A D S P - 2 8 m s p 0 2 ) c o d e c ( e n c o d e r a n d d e c o d e r ) f o r v o ic e -b a n d s ig n a ls b a s e d o n
s ig m a -d e lta A / D a n d D / A c o n v e r t e r s a n d a n a lo g fr o n t- e n d c ir c u its n e e d e d a s a n
in t e r f a c e to th e a n a lo g v o ic e -b a n d s ig n a ls . T h e n o m in a l s a m p lin g r a t e ( a f t e r d e c
i m a tio n ) is 8 k H z a n d th e s a m p lin g r a t e o f th e S D M is 1 M H z . T h e c o d e c h a s a
6 5 - d B S N R a n d h a r m o n ic d is to r tio n p e r f o r m a n c e .

10.10 SUMMARY AND REFERENCES

T h e n e e d f o r s a m p lin g r a t e c o n v e r s io n a r is e s fr e q u e n t ly in d ig ita l s ig n a l p r o c e s s in g
a p p lic a tio n s . I n th is c h a p t e r w e firs t t r e a t e d s a m p lin g r a t e r e d u c tio n ( d e c im a tio n )
a n d s a m p lin g r a t e in c r e a s e ( in t e r p o l a t i o n ) b y in t e g e r fa c to r s a n d th e n d e m o n
s tr a t e d h o w th e tw o p r o c e s s e s c a n b e c o m b in e d t o o b t a i n s a m p lin g r a t e c o n v e r s io n
b y a n y r a t io n a l f a c t o r . L a t e r , in S e c t i o n 1 0 .8 , w e d e s c r ib e d a m e t h o d to a c h ie v e
s a m p lin g r a t e c o n v e r s io n b y a n a r b itr a r y f a c t o r .
I n g e n e r a l, th e im p le m e n ta tio n o f s a m p lin g r a t e c o n v e r s io n r e q u ir e s t h e u s e
o f a lin e a r tim e - v a r ia n t filte r . W e d e s c r ib e d m e t h o d s f o r im p le m e n tin g s u c h filte r s ,
in c lu d in g th e c la s s o f p o ly p h a s e f ilte r s tr u c tu r e s , w h ic h a r e e s p e c ia lly s im p le to
im p le m e n t. W e a ls o d e s c r ib e d th e u s e o f m u ltis ta g e im p le m e n ta tio n s o f m u ltir a te
c o n v e r s io n as a m e a n s o f s im p lify in g th e c o m p le x ity o f th e f ilte r r e q u ir e d t o m e e t
t h e s p e c ific a tio n s .
In th e s p e c ia l c a s e w h e r e t h e s ig n a l t o b e r e s a m p le d is a b a n d p a s s s ig n a l, w e
d e s c r ib e d tw o m e t h o d s f o r p e r f o r m in g th e s a m p lin g r a t e c o n v e r s io n , o n e o f w h ich
in v o lv e s fr e q u e n c y c o n v e r s io n , w h ile th e s e c o n d is a d ir e c t c o n v e r s io n m e t h o d th a t
d o e s n o t e m p lo y m o d u la tio n .
F in a lly , w e d e s c r ib e d a n u m b e r o f a p p lic a t io n s t h a t e m p lo y m u ltir a te s ig n a l
p r o c e s s in g , in c lu d in g th e im p le m e n ta tio n o f n a r r o w b a n d filte r s , p h a s e s h ifte r s , fil
t e r b a n k s , s u b b a n d s p e e c h c o d e r s , a n d tr a n s m u lt ip le x e r s . T h e s e a r e ju s t a fe w o f
th e m a n y a p p lic a t io n s e n c o u n t e r e d in p r a c t ic e w h e r e m u ltir a te s ig n a l p r o c e s s in g
is u se d .
T h e firs t c o m p r e h e n s iv e t r e a t m e n t o f m u ltir a te s ig n a l p r o c e s s in g w a s g iv e n
in th e b o o k b y C r o c h i e r e a n d R a b in e r ( 1 9 8 3 ) . In t h e te c h n ic a l l i t e r a t u r e , w e c ite
th e p a p e r s b y S c h a f e r a n d R a b i n e r ( 1 9 7 3 ) , a n d C r o c h i e r e a n d R a b i n e r ( 1 9 7 5 , 1 9 7 6 ,
1 9 8 1 , 1 9 8 3 ) . T h e u s e o f i n te r p o la tio n m e t h o d s t o a c h i e v e s a m p lin g r a t e c o n v e r s io n

Serial

output

D igital
control
select
Sclk

Serial
input

Figure 10.48 Diagram of Analog Devices ADSP-28 codec

845

Multirate Digital Signal Processing

846

Chap. 10

b y a n a r b itr a r y f a c t o r is t r e a t e d in a p a p e r b y R a m s t a d ( 1 9 8 4 ) . A th o r o u g h tu to r ia l
tr e a t m e n t o f m u ltir a te d ig ita l filte r s a n d f ilt e r b a n k s , in c lu d in g q u a d r a tu r e m ir r o r
filte r s , is g iv e n b y V e t t e r l i ( 1 9 8 7 ) , a n d b y V a id y a n a t h a n ( 1 9 9 0 , 1 9 9 3 ) , w h e r e m a n y
r e f e r e n c e s o n v a r io u s a p p lic a tio n s a r e c ite d .

A c o m p r e h e n s iv e s u r v e y o f d ig ita l

tr a n s m u lt ip le x in g m e t h o d s is fo u n d in t h e p a p e r b y S c h e u e r m a n n a n d G o c k l e r
( 1 9 8 1 ) . S u b b a n d c o d in g o f s p e e c h h a s b e e n c o n s id e r e d in m a n y p u b lic a tio n s . T h e
p io n e e r in g w o r k o n th is to p ic w a s d o n e b y C r o c h i e r e ( 1 9 7 7 , 1 9 8 1 ) a n d b y G a r la n d
a n d E s t e b a n ( 1 9 8 0 ) . S u b b a n d c o d in g h a s a ls o b e e n a p p lie d t o c o d in g o f im a g e s .
W e m e n tio n th e p a p e r s b y V e t t e r l i ( 1 9 8 4 ) , W o o d s a n d O N e il ( 1 9 8 6 ) , S m ith a n d
E d d in s ( 1 9 8 8 ) , a n d S a f r a n e k e t a l. ( 1 9 8 8 ) a s ju s t a fe w e x a m p le s . I n c lo s in g , w e
w ish t o e m p h a s iz e th a t m u ltir a te s ig n a l p r o c e s s in g c o n tin u e s t o b e a v e r y a c tiv e
re s e a r c h a re a .

PROBLEMS
10.1 An analog signal x a(t) is bandlimited to the range 90 0 < F < 1 1 0 0 Hz. It is used as
an input to the system shown in Fig. P10.1. In this system, H(&) is an ideal lowpass
filter with cutoff frequency Fc = 125Hz.

jr = 2500

= 1 = 250

Tx

Ty

Fignre P10.1
(a) Determine and sketch the spectra for the signals x(n), w(n), v(n), and y(n).
(b) Show that it is possible to obtain y(n) by sampling x*(t) with period T = 4

milliseconds.
10.2 Consider the signal x(n) = a*u(rt), \a\ < 1.
(a) Determine the spectrum X(w).
(b) The signal x(n) is applied to a decimator that reduces the rate by a factor of 2.
Determine the output spectrum.
(c ) Show that the spectrum in part (b) is simply the Fourier transform of x(2n).
10-3 The sequence x(n) is obtained by sampling an analog signal with period T. From
this signal a new signal is derived having the sampling period T/2 by use of a linear
interpolation method described by the equation
x{n j 2),

n even

Chap. 10

847

Problems

(a) Show that this linear interpolation scheme can be realized by basic digital signal
processing elements.
(b) Determine the spectrum of y(n) when the spectrum of x(n) is
f 1,
1 0,

0 < M < 0.2*

otherwise

(c) Determine the spectrum of y(.n) when the spectrum of x(n) is

* ( ,) = ! ''

0.7* < M <0.9*

10,

otherwise

10.4 Consider a signal x(n) with Fourier transform

X (co) = 0 for

w, < |o| < n

fm < I/I < \

(a) Show that the signal x( n) can be recovered from its samples x(mD) if the sampling
frequency ws = 2tt/ D 2uim( f s = 1/ D > 2f m).
(b) Show that x(n) can be reconstructed using the formula
OC

x(n) =

x( k D) h r(n kD)

w here
sin (2^/ rn)

hr(n) = ---------

fm
COm < (t)c < Cl)J co

2n n

(c) Show that the bandlimited interpolation in part (b) can be thought as a two-step
process: First, increasing the sampling rate by a factor of D by inserting (> 1)
zero samples between successive samples of the decimated signal xa(rt) = x(nD)
and second, filtering the resulting signal using an ideal lowpass filter with cutoff
frequency cuc.
10.5 In this problem we illustrate the concepts of sampling and decimation for discretetime signals. To this end consider a signal x ( n ) with Fourier transform X (w) as in
Fig. P10.5.
X(o>)

L m

- 2 - 1 0 1 2

...

Figure P10.5
(a) Sampling x(n) with a sampling period D = 2 results to the signal
, ,
[ x(n),
'W - l o ,

n = 0 , 2 , 4 , . ..

n = 1 , 3, 5 . . . .

Compute and sketch the signal x,(n) and its Fourier transform X s(w). Can we
reconstruct jc(n) from xt (n)? How?

848

Multirate Digital Signal Processing

(b)

Chap. 10

D ecim atin g x(n) by a facto r o f D = 2 produ ces the signal

x<i(n) = x(2n)

all

Show th at X d{a)) = X x(a>/2). P lo t the signal x j( n ) and its transfo rm X d(cj). D o

we lose any in form ation w hen we d ecim ate th e sam pled signal xr (n)?
1 0 .6 * D esign a d ecim ato r th at dow nsam ples an input signal x(n) by a fa cto r D = 5. U se
th e R e m ez algorithm to d eterm in e th e co efficien ts o f the F I R filter that has 0.1 -d B
ripple in th e passband (0 < w < 7t/5) and is dow n by at least 3 0 d B in th e stopband.
A lso determ in e the correspond ing polyphase filter stru ctu re fo r im plem enting th e
decim ator.
1 0 .7 * D esig n an in terp o lato r th at in creases th e input sam pling rate by a fa cto r o f / = 2. U se
the R e m ez algorithm to determ in e th e coefficien ts o f the F I R filter th at has a 0.1-d B
ripple in th e passband (0 < w < n /2 ) and is down by at least 3 0 d B in the stopband.
A iso , determ in e the correspond ing polyphase filter stru ctu re fo r im plem enting the
in terp olato r.
1 0 .8 * D esign a sam p le-rate co n v e rte r th at redu ces the sam pling rate by a fa cto r \ . U se the
R e m ez algorithm to d eterm in e th e co efficien ts o f the F I R filter th at has a
0 .1 -d B ripple in the passband and is dow n by at least 30 d B in th e stopband. Specify
the sets o f tim e-v ariant co efficien ts g (n ,m ) and th e co rresp o nd ing co efficien ts in the
polyphase filter realization o f the sam p le-rate co n v erter.
10.9 C onsid er the two d ifferen t ways o f cascad ing a d ecim ato r with an in terp o la to r shown
in Fig. P I 0.9.

x(n)

>' i()
(a)

x(n)

ID

17

---------------- -

y2(n)

(b)

Figure P10.9

(a)

If D = I , show th at th e outpu ts o f th e tw o con figu rations are different. H en ce,

in g en eral, the two system s are n o t identical.

(b)

Show th at th e tw o system s are id en tical if and only if D and I a re relatively prim e.

10.10 Prove th e eq u iv alen ce o f th e two d ecim ato r and in terp o lato r con fig u rations shown
in Fig. P 10.10. T h e se equivalen t relation s are called the n ob le id en tities (see
V aid y anathan , 1990).
10.11 C onsid er an arb itrary digital filter with tran sfer fun ction
OO

H (z) =

h (n )z~n
n*-00

(a)

P erfo rm a tw o-com p on en t polyphase d eco m p osition o f H ( z ) by grouping the

even -nu m bered sam ples Ao(n) = h(2n ) and th e odd-n um bered sam ples h \(n ) =

Chap. 10

849

Problems

(a)

x(n)

(b)

Figure P1G.10
h(2n + 1). T hu s show that H ( z ) can b e exp ressed as
H ( i ) = H o ( z 2) + r 1H ,(zJ )
and d eterm in e N 0(z) and N ,(~).

(b)

G en era liz e the result in part (a ) by showing that H ( z ) can be decom p osed in to
an >-com ponent polyphase filter structure with tran sfer function

D ete rm in e //*(;).
(c) F o r th e I I R filter with transfer function

H {z) =

1 a z~ l

d eterm in e Ho(z) and H \ (z) fo r the tw o-com p on en t d ecom p osition .

10.12 D esign a tw o-stage d ecim ato r for th e follow ing specification s

Passband:
T ran sitio n band:
In pu t sam pling rate:
R ip p le:

D = 100
0 < F < 50
5 0 < F < 55
1 0,000 H z
8i = 1 0 " ', 5 2 = 1 0 " 3

1 0 ,1 3 D esign a lin ear ph ase F I R filter th a t satisfies th e follow ing sp ecification s based on a
single-stage and a tw o-stage m ultirate stru ctu re.
Sam pling rate:
Passband:
T ran sitio n band:

1 0,000 H z
0 < F <60
60 < F < 65

R ip p le:

5, = 1 0 - 1,S 2 = 1 0 - 3

1 0 .1 4 Prov e th at th e h alf-ban d filter th at satisfies (1 0 .9 .4 3 ) is alw ays odd and the even
coefficien ts are zero.

850

Multirate Digital Signal Processing

Chap. 10

10.15 D esign o n e-stag e and tw o-stage in terp olato rs to m ee t the follow ing specification :

/
Input sam pling rate:
Passband:
T ran sitio n band:
R ip p le:

=20

10.000 H z
0 < F < 90
90 < F < 100
a, = lO "2. ^ = lO "3

10.16 B y using (1 0 .9 .2 6 ) derive the equations correspond ing to the stru ctu re fo r the poly
phase synthesis section shown in Fig. 10.43.
10 .1 7 Show th at th e transpose o f an L-stage in terp o lato r fo r increasin g th e sam pling rate by
an in teg er facto r I is equivalen t to an L -stag e d ecim ato r th at d e cre a ses the sam pling
rate by a facto r D = I.
1 0.18 Sk etch the polyphase filter structure fo r achieving a tim e ad v an ce o f ( k / l ) T , in a
seq u en ce x(n).
10.19 Prov e th e follow ing exp ressions fo r an in terp olato r o f o rd e r /.
(a) T h e im pulse response h{n) can be expressed as

where

n = 0. I . 2 1 , . . .
o therw ise

(b) H (z)

may be expressed as

( ;) =

i-i

1 0 .2 0 * Zoom -frequertcy an alysis

C o nsid er the system in Fig. P I0 .2 0 a .

(a) Sketch th e spectrum o f the signal _v(n) = y*(/i) +

jy i( n )

if th e input signal jc(n)

has the spectru m shown in Fig. P I 0.20b.

(b)

Suppose th at we are in terested in th e analysis o f the fre q u e n cies in th e band

fo 5 / fo + A / , w here /o = jt/6 and A / = tt/3. D e te rm in e th e cu to ff of

a lowpass filter and the decim ation fa cto r D requ ired to re ta in th e inform ation
con tain ed in this band o f frequ en cies.
(c) A ssum e th at

w here p = 4 0 and /* = k/ p, Jt = 0 , 1 , ,
D F T o f x(n ).

p 1. C om pute and p lo t th e 1024-point

Chap. 10

Problems

851

(a)

X(a>)

(b)

Figure P10.19
(d ) R e p e a t p art (b ) for the signal x(n) given in part (c ) by using an appropriately
designed low pass lin ear phase F I R filter to d eterm in e th e decim ated signal *(> =
**{")

j s i (n ).

( e ) C o m p u te th e 1024-p o in t D F T o f s(n ) and in v estig ate to see if you h av e obtain ed

th e exp ected results.

11

Linear Prediction and

Optimum Linear Filters

T h e d e s ig n o f filte r s to p e r f o r m s ig n a l e s tim a tio n is a p r o b le m

th a t f r e q u e n tly

a r is e s in th e d e s ig n o f c o m m u n ic a tio n s y s te m s , c o n t r o l s y s te m s , in g e o p h y s ic s , a n d
in m a n y o th e r a p p lic a t io n s a n d d is c ip lin e s . In th is c h a p t e r w e t r e a t th e p r o b le m
o f o p tim u m f ilte r d e s ig n fr o m a s ta tis tic a l v ie w p o in t. T h e filte r s a r e c o n s tr a in e d
to b e lin e a r a n d th e o p tim iz a tio n c r ite r io n is b a s e d o n th e m in im iz a t io n o f th e
m e a n -s q u a r e e r r o r . A s a c o n s e q u e n c e , o n ly th e s e c o n d - o r d e r s ta tis tic s ( a u t o c o r
r e la tio n a n d c r o s s c o r r e la t io n f u n c t io n s ) o f a s ta tio n a r y p r o c e s s a r e r e q u ir e d in th e
d e te r m in a tio n o f th e o p tim u m filte r s .
In c lu d e d in th is t r e a t m e n t is th e d e s ig n o f o p tim u m filte r s f o r lin e a r p r e d ic
tio n . L i n e a r p r e d ic t io n is a p a r tic u la r ly im p o r ta n t to p ic in d ig ita l s ig n a l p r o c e s s in g ,
w ith a p p lic a t io n s in a v a r ie ty o f a r e a s , s u c h as s p e e c h s ig n a l p r o c e s s in g , im a g e p r o
c e s s in g , a n d n o is e s u p p r e s s io n in c o m m u n ic a tio n s y s te m s .

A s w e s h a ll o b s e r v e ,

d e te r m in a tio n o f t h e o p tim u m lin e a r f ilt e r f o r p r e d ic t io n r e q u ir e s th e s o lu tio n o f

a s e t o f lin e a r e q u a tio n s th a t h a v e s o m e s p e c ia l s y m m e tr y . T o s o lv e th e s e lin e a r
e q u a tio n s , w e d e s c r ib e tw o a lg o r ith m s , t h e L e v in s o n - D u r b in a lg o r it h m a n d th e
S c h u r a lg o r ith m , w h ich p r o v id e th e s o lu tio n to th e e q u a tio n s th r o u g h c o m p u ta
tio n a lly e ff ic ie n t p r o c e d u r e s t h a t e x p lo it th e s y m m e tr y p r o p e r tie s .
T h e la s t s e c tio n o f th e c h a p t e r t r e a t s a n im p o r ta n t c la s s o f o p tim u m filte rs
c a lle d W i e n e r filte r s . W ie n e r filte r s a r e w id e ly u se d in m a n y a p p lic a t io n s in v o lv in g
th e e s tim a tio n o f s ig n a ls c o r r u p te d w ith a d d itiv e n o is e .

11.1 INNOVATIONS REPRESENTATION OF A STATIONARY RANDOM

PROCESS
In th is s e c tio n w e d e m o n s tr a t e t h a t a w id e -s e n s e s ta tio n a r y r a n d o m p r o c e s s c a n b e
r e p r e s e n te d a s th e o u tp u t o f a c a u s a l a n d c a u s a lly in v e r t ib le l i n e a r s y s te m e x c ite d
b y a w h ite n o is e p r o c e s s . T h e c o n d itio n t h a t th e s y s t e m is c a u s a lly i n v e r t ib le a ls o
a llo w s us to r e p r e s e n t t h e w id e -s e n s e s ta tio n a r y r a n d o m p r o c e s s b y th e o u tp u t o f
th e in v e r s e s y s te m , w h ic h is a w h ite n o i s e p r o c e s s .

852

Sec. 11.1

Innovations Representation of a Stationary Random Process

853

L e t us c o n s id e r a w id e -s e n s e s ta tio n a r y p r o c e s s {jc ( ) } w ith a u t o c o r r e la t io n

seq u en ce

{yxx(m)}

a n d p o w e r s p e c tr a l d e n s ity r ( / ) , \ f \ <

W e a s s u m e th a t

!%.<(/) is r e a l a n d c o n tin u o u s f o r a ll |/| < j . T h e z -tr a n s fo r m o f th e a u t o c o r r e l a

tio n s e q u e n c e {y xx(m )} is
^ (

2) =

( 1 1 .1 .1 )

fr o m w h ic h w e o b ta in th e p o w e r s p e c tr a l d e n s ity b y e v a lu a tin g 1 % ,^ ) o n th e u n it
c ir c le [i.e . b y s u b s t itu tin g z = e x p ( y 2 jr / ) ] .
N o w , l e t u s a s s u m e th a t lo g I V , (z) is a n a ly tic (p o s s e s s e s d e r iv a tiv e s o f a ll
o r d e r s ) in a n a n n u la r r e g io n in th e z -p la n e th a t in c lu d e s th e u n it c ir c le ( i .e ., n <

\z\ < r 2 w h e r e n < 1 a n d r 2 > 1 ). T h e n , l o g r ^ z ) c a n b e e x p a n d e d in a L a u r e n t

s e r ie s o f th e fo r m
lo g

r (z ) =

( 11. 1.2 )

v(m )z~

w h e r e th e (u (m )} a r e th e c o e f f ic ie n t s in th e s e r ie s e x p a n s io n . W e c a n v ie w {u (m )}
a s th e s e q u e n c e w ith z -tr a n s fo r m V ( z ) = l o g T ( z ) . E q u iv a le n tly , w e c a n e v a lu a te
lo g T J r (z ) o n th e u n it c ir c le ,

J2

lo g r (/ )=

v ( m ) e - J23lfm

( 1 1 .1 .3 )

s o t h a t th e { r ( m ) } a r e th e F o u r i e r c o e f f ic ie n ts in th e F o u r i e r s e r ie s e x p a n s io n o f
th e p e r io d ic fu n c t io n l o g r ^ ( / ) . H e n c e

v( m) =

[ \ g T xx{ f ) ] e j l f md f

m = 0 , d b l------

( 1 1 .1 .4 )

s in c e T x l ( f ) is a re a l a n d e v e n fu n c tio n o f / .

W e o b s e r v e th a t v {m ) =

F r o m ( 1 1 .1 .2 ) it fo llo w s th a t
r ^ f z ) = exp

_m=oc

( 1 1 .1 .5 )

^2 u t -,\
w h e r e , b y d e f in it io n , a 2 = e x p [i? (0 )] a n d

H ( z ) = exp

(11.1.6)

T . v(m )z~

I f ( 1 1 .1 .5 ) is e v a lu a te d o n th e u n it c i r c l e , w e h a v e th e e q u iv a le n t r e p r e s e n ta tio n o f
th e p o w e r s p e c tr a l d e n s ity a s

(11.1.7)

r ( / ) = <r2 \ H ( f ) \ 2

We note that
lo g Txx{ f )

= lo g a i+

lo g H ( f ) + lo g

OC

v ( m ) e - j2* fm

//* (/)

854

Linear Prediction and Optimum Linear Filters

Chap. 11

F r o m th e d e fin itio n o f H ( z ) g iv e n b y ( 1 1 .1 .6 ) , it is c l e a r t h a t th e c a u s a l p a r t o f
th e F o u r i e r s e r ie s in ( 1 1 .1 .3 ) is a s s o c ia t e d w ith H ( z ) a n d th e a n tic a u s a l p a r t is
a s s o c ia t e d w ith H ( z ~ l ). T h e F o u r i e r s e r ie s c o e f f ic ie n t s { v ( m ) } a r e th e ce p stra l
c o e fficie n ts a n d t h e s e q u e n c e { u (m ) } is c a lle d th e c e p s tr u m o f th e s e q u e n c e {y xx(m )},
a s d e fin e d in S e c t i o n 4 .2 .7 .
T h e filte r w ith s y s te m fu n c t io n H ( z ) g iv e n b y ( 1 1 .1 .6 ) is a n a ly tic in t h e r e g io n
|z| > r \ < 1. H e n c e , in th is r e g io n , it h a s a T a y l o r s e r ie s e x p a n s io n a s a c a u s a l
s y s te m o f th e f o r m
OC

H ( z ) = J 2 h in ) z ~ n
m=0

( 1 U '8 )

T h e o u tp u t o f th is filte r in r e s p o n s e t o a w h ite n o is e in p u t s e q u e n c e w (n ) w ith

p o w e r s p e c tr a l d e n s ity a 2 is a s ta tio n a r y r a n d o m p r o c e s s { * ( ) } w ith p o w e r s p e c
tr a l d e n s ity r xx( f ) = o 2 \ H ( f ) \ 2 . C o n v e r s e ly , th e s ta tio n a r y r a n d o m p r o c e s s |a-(/j)}
w ith p o w e r s p e c tr a l d e n s ity T xx( f ) c a n b e tr a n s f o r m e d in to a w h ite n o is e p r o c e s s
b y p a s s in g (jc ( ti)} th r o u g h a lin e a r f ilte r w ith s y s te m f u n c t io n 1 / H ( z ) .

W e c a ll

th is f ilte r a n o is e w h ite n in g filte r. It s o u tp u t, d e n o te d as (w ( n ) } is c a lle d th e in n o

v a tio n s p ro c e s s a s s o c ia t e d w ith th e s ta tio n a r y r a n d o m p r o c e s s { * ( ) } . T h e s e tw o

r e la tio n s h ip s a r e illu s tr a te d in F ig . 1 1 .1 .
T h e r e p r e s e n t a t i o n o f th e s ta tio n a r y s t o c h a s t ic p r o c e s s {.v fn )} a s t h e o u tp u t
o f a n I I R filte r w ith s y s te m fu n c tio n H ( z ) g iv e n b y ( 1 1 .1 .8 ) a n d e x c ite d b y a w h ite
n o is e s e q u e n c e (u ^ n )} is c a lle d th e W o ld re p re se n ta tio n .

11.1.1 Rational Power Spectra

L e t u s n o w r e s t r ic t o u r a t t e n t io n t o th e c a s e w h e r e th e p o w e r s p e c tr a l d e n s ity o f
th e s ta tio n a r y r a n d o m p r o c e s s { * ( ) ) is a r a t io n a l fu n c t io n , e x p r e s s e d a s
r ^ ( z ) = g u,7 7\ J

A {z)A (z~i )

r\ < \ z \ < r 2

( 1 1 .1 .9 )

w h e r e th e p o ly n o m ia ls B ( z ) a n d A ( z ) h a v e r o o t s t h a t fa ll in s id e th e u n it c ir c le in
th e s - p la n e . T h e n th e l in e a r f ilt e r H ( z ) f o r g e n e r a tin g th e r a n d o m p r o c e s s U ( ) }

w(rr)
W hile noise

L inear
causal
filter

x(n ) = Y , h(i)w(n - Jc)

k= 0

H{z)
(a)

-r(n)

L inear
causal
filter

w(n)
W hite noise

VH(z)

(b)

Figure 11.1 Filters for generating

(a) the random process Jt(n) from white
noise and (b) the inverse filter.

Sec. 11.1

Innovations Representation of a Stationary Random Process

855

fr o m th e w h ite n o is e s e q u e n c e {iu (n )} is a ls o r a t io n a l a n d is e x p r e s s e d a s

(11.1 .10)

w h e r e {bk } a n d [ak } a r e th e filte r c o e f f ic ie n ts t h a t d e te r m in e th e lo c a t io n o f th e

z e r o s a n d p o le s o f H ( z ) , re s p e c tiv e ly . T h u s H { z ) is c a u s a l, s ta b le , a n d m in im u m
p h ase.
s y s te m .

I t s r e c ip r o c a l 1 / H { z ) is a ls o a c a u s a l, s ta b le , a n d m in im u m - p h a s e lin e a r
T h e r e f o r e , t h e ra n d o m p r o c e s s {;c (n )} u n iq u e ly r e p r e s e n t s t h e s ta tis tic a l

p r o p e r tie s o f t h e in n o v a tio n s p r o c e s s {u j(/ i)}, a n d v ic e v e r s a .

F o r t h e li n e a r s y s t e m w ith th e r a t io n a l s y s t e m f u n c t io n H (z ) g iv e n b y ( 1 1 .1 .1 0 ) ,
i h e o u tp u t x ( n ) is r e la te d to t h e in p u t u>(n) b y th e d if f e r e n c e e q u a tio n
( 1 1 .1 .1 1 )
W e w ill d is tin g u is h a m o n g t h r e e s p e c ific c a s e s .

Autoregressive (AR) process.

b0 =

1. bk = 0 , k > 0 .

I n th is c a s e , t h e

lin e a r f ilte r H ( z ) = 1 / A ( z ) is a n a ll-p o le filte r a n d th e d if f e r e n c e e q u a t io n f o r th e

in p u t - o u t p u t r e la tio n s h ip is

(11.1 .12)
k= 1
In tu r n , th e n o is e -w h ite n in g filte r f o r g e n e r a tin g th e in n o v a tio n s p r o c e s s is a n
a ll- z e r o filte r .

Moving average (MA) process.

a k = 0 , k > 1. In th is c a s e , th e l i n e a r f ilte r

H { z ) = B { z ) is a n a ll-z e r o filte r a n d th e d if f e r e n c e e q u a tio n f o r th e i n p u t- o u tp u t

r e la tio n s h ip is
( 1 1 .1 .1 3 )
T h e n o is e -w h it e n in g filte r f o r th e M A p r o c e s s is a n a ll- p o le f ilte r .

Autoregressive, moving average (ARMA) process.

I n th is c a s e , th e lin

e a r f ilte r H ( z ) = B { z ) / A { z ) h a s b o th fin ite p o le s a n d z e r o s in t h e z - p la n e a n d th e

c o r r e s p o n d in g d if f e r e n c e e q u a tio n is g iv e n b y ( 1 1 .1 .1 1 ) .

T h e i n v e r s e s y s te m f o r

g e n e r a tin g t h e in n o v a tio n p r o c e s s f r o m x ( n ) is a ls o a p o l e - z e r o s y s te m o f th e fo r m
1 / H ( z ) = A ( z ) /B { z ) .

11.1.2 Relationships Between the Filter Parameters and

the Autocorrelation Sequence
W h e n th e p o w e r s p e c tr a l d e n s ity o f th e s ta tio n a r y r a n d o m p r o c e s s is a r a t io
n a l f u n c t io n , t h e r e is a b a s ic r e la tio n s h ip b e tw e e n t h e a u t o c o r r e l a t i o n s e q u e n c e

856

Linear Prediction and Optimum Linear Filters

Chap. 11

{XjcjcC"t)} a n d th e p a r a m e te r s {a * } a n d {>*} o f th e l i n e a r f ilte r H ( z ) th a t g e n e r a te s

th e p r o c e s s b y filte r in g th e w h ite n o is e s e q u e n c e w (n ). T h is r e la tio n s h ip c a n b e o b
ta in e d b y m u ltip ly in g th e d if f e r e n c e e q u a tio n in ( 1 1 .1 .1 1 ) b y x * (n m ) a n d ta lcin g
th e e x p e c te d v a lu e o f b o th s id e s o f th e re s u ltin g e q u a tio n . T h u s w e h a v e

p
E [ x { n ) x * { n m )] = ^ f l t [ . x ( n k )x * (n m )]
( 1 1 .1 .1 4 )
+ ^T^f>i[u>(n k )x * (n m )]
H en ce

<?

Yxx ( m ) =

~ f t + Y 2 btYwx^w ~ f t ( H . 1 . 1 5 )
1=1

i= 0

w h e r e y w.r (m) is th e c r o s s - c o r r e la t io n s e q u e n c e b e tw e e n u;(/j) a n d x ( /i) .

T h e c r o s s c o r r e la tio n y wx(m ) is r e la te d to th e f ilte r im p u ls e r e s p o n s e . T h a t is,

ywxOn) = [ A * ( f i) w ( + m )]

= E

h (k )w * (n k ) w ( n -i- m )

( 1 1 .1 .1 6 )

= a lh ( -m )
w h e r e , in th e la s t s te p , w e h a v e u se d th e fa c t th a t th e s e q u e n c e w (n) is w h ite .
H en ce
I < 0
B y c o m b in in g ( 1 1 .1 .1 7 ) w ith ( 1 1 .1 .1 5 ) , w e o b ta in th e d e s ir e d r e la tio n s h ip

p
~ Y 2 , a tYxx{m - k ),
k=\

Yxxim) =

m > q

( 1 1 .1 .1 8 )

kYx* (m ~ k ^

h tt' lbk+m ,

k=l

0 <m <q

k= 0

- Yxx

m < 0

T h is r e p r e s e n ts a n o n lin e a r r e la tio n s h ip b e tw e e n yJX (m ) a n d th e p a r a m e t e r s {a * } ,

{**}
T h e r e la tio n s h ip in ( 1 1 .1 .1 8 ) a p p lie s , in g e n e r a l, t o th e A R M A p r o c e s s . F o r
a n A R p r o c e s s , ( 1 1 .1 .1 8 ) s im p lifie s to

p
~ 2 2 akY ( m ~ k ) ,

m > 0

k =i

Yxx(m) =

p
- 2 2 a k Yx*(m ~ W + a l

(11.1.19)
m = 0

k=i

y A (~ m )'

m <o

Sec. 11.2

Forward and Backward Linear Prediction

857

T h u s w e h a v e a lin e a r r e la tio n s h ip b e tw e e n y xx (m ) a n d th e { a t } p a r a m e te r s . T h e s e
e q u a tio n s , c a lle d th e

Y u le - W a lk e r e q u a tio n s , c a n b e e x p r e s s e d in th e m a tr ix

fo rm

~Yxx(0)

y (l)
-Yxx(p)

Yxx( ~ 2 )
YxA ~ l)

y , , ( - 1)
K(0)
YxA

- 1)

- 1 a\

Yxx(-P)
Yxx(~P + l)

Y x x i p - 2)

-a p _

- YxAO)

( 11. 1.20)

_0 _

T h i s c o r r e l a t i o n m a tr ix is T o e p lit z , a n d h e n c e it c a n b e e ff ic ie n tly in v e r t e d b y u s e
o f t h e a lg o r it h m s d e s c r ib e d in S e c t i o n 1 1 .3 .
F in a lly , b y s e tt in g a k 0 , 1 < k < p , a n d h (k ) bk, 0 < k < q , in ( 1 1 .1 .1 8 ) ,
w e o b ta in th e r e la tio n s h ip fo r th e a u t o c o r r e la t io n s e q u e n c e in th e c a s e o f a M A
p r o c e s s , n a m e ly ,
<?

YxAm ) =

cru , 2 2 bkbk+m'
n *=<>

0,

y*x (m ),

- m - ?

m > q
m < 0

( 1 1 - 1 .2 1 )

11.2 FORWARD AND BACKWARD LINEAR PREDICTION

L i n e a r p r e d ic t io n is an i m p o r ta n t to p ic in d ig ita l s ig n a l p r o c e s s in g w ith m a n y p r a c
tic a l a p p lic a t io n s .

In th is s e c tio n w e c o n s id e r t h e p r o b le m o f lin e a r ly p r e d ic tin g

th e v a lu e o f a s ta tio n a r y r a n d o m p r o c e s s e it h e r fo r w a rd in tim e o r b a c k w a r d in
tim e .

T h is fo r m u la tio n le a d s t o la t t ic e filte r s tr u c tu r e s a n d t o s o m e in te r e s tin g

c o n n e c t io n s t o p a r a m e tr ic s ig n a l m o d e ls .

11.2.1 Forward Linear Prediction

L e t u s b e g in w ith th e p r o b le m o f p r e d ic t in g a fu tu r e v a lu e o f a s ta tio n a r y r a n d o m
p r o c e s s fr o m o b s e r v a t io n o f p a s t v a lu e s o f t h e p r o c e s s . I n p a r tic u la r , w e c o n s id e r
th e o n e -ste p f o r w a r d lin e a r p r e d ic to r , w h ic h fo r m s th e p r e d ic t io n o f t h e v a lu e x ( n )
b y a w e ig h te d lin e a r c o m b in a tio n o f th e p a s t v a lu e s x ( n 1 ), x ( n - 2 ) , . . . , x ( n p ).
H e n c e t h e lin e a r ly p r e d ic t e d v a lu e o f x ( n ) is

p
x (n ) = ^ a p( i ) x (n k)

( 1 1 .2 .1 )

*= l
w h e r e th e { a p {k)} r e p r e s e n t t h e w e ig h ts in t h e lin e a r c o m b in a tio n . T h e s e w e ig h ts
a r e c a lle d th e p r e d ic t io n c o e fficie n ts o f t h e o n e - s t e p fo r w a r d l i n e a r p r e d ic t o r o f

o r d e r p . T h e n e g a tiv e s ig n in th e d e fin it io n o f x ( n ) is f o r m a th e m a tic a l c o n v e n ie n c e

a n d c o n f o r m s w ith c u r r e n t p r a c tic e in th e te c h n ic a l lite r a tu r e .

858

Linear Prediction and Optimum Linear Filters

F ig u re 1 L 2

Chap. 11

F o r w a r d lin e a r p re d ic tio n

T h e d if f e r e n c e b e tw e e n th e v a lu e x ( n ) a n d th e p r e d ic t e d v a lu e x ( n ) is c a lle d
th e f o r w a r d p r e d ic t io n e r r o r , d e n o te d a s f p (n):

f p (n) = x ( n ) - x ( n )
p

= x (n ) + ^

( 11. 2 .2 )
a p (k )x (n - k)

W e v iew li n e a r p r e d ic tio n as e q u iv a le n t t o li n e a r filte r in g w h e r e t h e p r e d ic t o r

is e m b e d d e d in th e lin e a r filte r , as s h o w n in F ig . 1 1 .2 . T h is is c a lle d a p r e d ic t io n -

e rro r filt e r w ith in p u t s e q u e n c e {jc(ai)} a n d o u tp u t s e q u e n c e { f r (n )}. A n e q u iv a le n t

r e a liz a tio n f o r th e p r e d ic t io n - e r r o r filte r is s h o w n in F ig . 1 1 .3 . T h i s r e a liz a tio n is
a d ir e c t -f o r m F I R f ilte r w ith s y s te m fu n c tio n

p
( 1 1 .2 .3 )

A p (z) =
i-=0
w h e r e , b y d e f in it io n , a ^ (0 ) = 1.

A s sh o w n in S e c tio n 7 .2 .4 , th e d ir e c t -f o r m F I R f ilte r is e q u iv a le n t t o a n a llz e r o la ttic e f ilte r . T h e l a t t i c e filte r is g e n e r a lly d e s c r ib e d b y t h e fo llo w in g s e t o f

p( P)

1
----/,(")
Figure 11-3 Prediction-error filter

Sec. 11.2

Forward and Backward Linear Prediction

859

o r d e r -r e c u r s iv e e q u a tio n :
fo ( n ) = go(n) - x {n )
fm (n ) = / m -i(n ) + K mg m~ i( n - 1)

m = l,2 ,...,p

g m(n) = A r*/m_ i ( n ) + g m- i ( n - 1)

m = 1 . 2 ........... p

( 1 1 .2 .4 )

w h e r e { m) a r e th e r e f le c tio n c o e f f ic ie n ts a n d g m(n) is th e b a c k w a r d p r e d ic tio n

e r r o r d e fin e d in th e fo llo w in g s e c tio n .

N o te th a t f o r c o m p le x - v a lu e d d a ta , th e

c o n ju g a t e o f K m is u se d in th e e q u a tio n f o r g m(n).

F ig u r e

1 1 .4 illu s tr a te s a

/7-stage l a t t i c e filte r in b lo c k d ia g ra m fo r m a lo n g w ith a ty p ic a l s ta g e sh o w in g

th e c o m p u ta tio n s g iv e n b y ( 1 1 .2 .4 ) .
A s a c o n s e q u e n c e o f th e e q u iv a le n c e b e tw e e n th e d ir e c t -f o r m p r e d ic t io n e r r o r F I R filte r a n d th e F I R la t t ic e filte r , th e o u tp u t o f th e />-stage l a t t i c e filte r is
e x p r e s s e d as

f p (n) -

a p(f:^x (n ~ f t

a p (0) = l

( 1 1 .2 .5 )

S in c e ( 1 1 .2 .5 ) is a c o n v o lu tio n su m , th e z - tr a n s fo r m r e la t io n s h ip is

( 11 .2 .6 )

F r (z) = A p ( z ) X ( z )
o r , e q u iv a le n tly .

A (z) =
'

X (z)

( 1 1 .2 .7 )

F 0(z)

T h e m e a n -s q u a r e v a lu e o f th e fo r w a rd lin e a r p r e d ic t io n e r r o r f p {n) is

fp = E [ \ f p ( n ) \2]

(11.2 .8)

p
= > 'jj(0 ) + 2 R e

a * ( l) a p( k ) y xx(l - k )

Figure 1L4

p -stage lattice filter

860

Linear Prediction and Optimum Linear Filters

Chap. 11

<?/ is a q u a d r a tic fu n c t io n o f th e p r e d ic t o r c o e f f ic ie n t s a n d its m in im iz a tio n le a d s

t o th e s e t o f lin e a r e q u a tio n s

p
yx A d

= - Y

*= i

, a r {k )y * * v ~ k)

l = l ' 2 .......... p

( n -2 -9 )

T h e s e a r e c a lle d th e n o r m a l e q u a tio n s f o r th e c o e f f ic ie n ts o f th e lin e a r p r e d ic t o r .

T h e m in im u m m e a n - s q u a r e p r e d ic t io n e r r o r is s im p ly
m in [ / ] s

E { = y ( 0 ) + Y , a p { k ) YxA - k )

( 1 1 .2 .1 0 )

*= i
In th e fo llo w in g s e c tio n w e e x t e n d th e d e v e lo p m e n t a b o v e to th e p r o b le m o f
p r e d ic tin g th e v a lu e o f a tim e s e r ie s in th e o p p o s ite d ir e c t io n , n a m e ly , b a c k w a r d
in tim e .

11.2.2 Backward Linear Prediction

L e t u s a s s u m e th a t w e h a v e th e d a ta s e q u e n c e x ( n ) , x ( n - 1 ) .......... x(rt p + 1) fro m a
s ta tio n a r y ra n d o m p r o c e s s a n d w e w ish t o p r e d ic t th e v a lu e x ( n p ) o f th e p r o c e s s .
In th is c a s e w e e m p lo y a o n e -ste p b a c k w a rd lin e a r p re d ic t o r o f o r d e r p . H e n c e
p

-i

x ( n p ) = Y 2 bp (k )x {n k)
k=0

( 11 .2 . 11 )

T h e d if f e r e n c e b e tw e e n th e v a lu e x ( n p ) a n d th e e s tim a te x ( n - p ) is c a lle d th e

b a c k w a r d p re d ic tio n e r r o r , d e n o te d a s g p( n ):
P- \
g p (n) = x ( n - p ) + 2 2 bp (k)x (n ~ f t

i=

( 11.2 .12)

' Y l bP ^ x ^n ~ f t

bp (p ) = 1

t=o
T h e b a c k w a r d lin e a r p r e d ic t o r c a n b e r e a liz e d e i t h e r b y a d ir e c t -f o r m F I R
f ilte r s tr u c tu r e s im ila r t o th e s tr u c tu r e s h o w n in F ig . 1 1 .2 o r a s a la ttic e s tr u c tu r e .
T h e l a t t ic e s tr u c tu r e sh o w n in F ig . 1 1 .4 p r o v id e s t h e b a c k w a r d lin e a r p r e d ic t o r as
w e ll a s th e fo r w a rd lin e a r p r e d ic to r .
T h e w e ig h tin g c o e ffic ie n ts in th e b a c k w a r d li n e a r p r e d i c t o r a r e th e c o m p le x
c o n ju g a te s o f th e c o e f f ic ie n ts f o r th e fo r w a r d l i n e a r p r e d ic t o r , b u t th e y o c c u r in
re v e rse o rd e r. T h u s w e hav e

bp(k) a * ( p k )

k = 0 ,l,...,p

( 1 1 .2 .1 3 )

In th e z -d o m a in , th e c o n v o lu tio n s u m in ( 1 1 .2 .1 2 ) b e c o m e s

G p(z) = Bp (z)X (z)

or, equivalently,
G p (z)

G p (z)

(11.2.14)

Sec. 11.2

861

Forward and Backward Linear Prediction

w h e r e B p {z) r e p r e s e n ts th e s y s te m fu n c t io n o f th e F I R

filte r w ith c o e f f ic ie n ts

bP (k).
S in c e bp (k) = a * ( p k ), G r (z) is r e la te d to A r (z)

B p (z) =

= X >
t=o

>

-*)*"*
( 1 1 .2 .1 6 )

= z - ' fl; ( * ) z *
t=0
= Z-p A*p (z - ')
T h e r e la tio n s h ip in ( 1 1 .2 .1 6 ) im p lie s th a t th e z e r o s o f th e F I R f ilte r w ith s y s te m
fu n c t io n B p(z) a r e s im p ly th e ( c o n ju g a t e ) r e c ip r o c a ls o f th e z e r o s o f A ,,(z ). H e n c e

B p (z) is c a lle d th e r e c ip r o c a l o r re v e rse p o ly n o m ia l o f A p (z).

N o w th a t w e h a v e e s ta b lis h e d t h e s e in te r e s tin g r e la tio n s h ip s b e tw e e n th e
f o r w a r d a n d b a c k w a r d o n e - s te p p r e d ic t o r s , le t us re tu r n to th e re c u r s iv e l a t t ic e
e q u a tio n s in ( 1 1 .2 .4 ) a n d t r a n s fo r m th e m to th e z -d o m a in . T h u s w e h a v e

F 0(z) = G q ( z ) = X ( z )
F m(z) = F m- i ( z ) + /rm; - ' G ffl_ , W
G m(z ) =

K " F m- i { z ) + z _ 1 G m_ i ( ; )

m = 1 ,2 ,..., p

( 1 1 .2 .1 7 )

m = 1, 2 , . . . , p

I f w e d iv id e e a c h e q u a tio n b y X ( z ) , w e o b ta in th e d e s ir e d r e s u lts in th e f o r m
4 0 (z ) =

5 o (z ) = 1

A m{z) = A m^ ( z ) + K mz ~ l B m^ { z )

m = 1 , 2 .......... p

B m(z) =

in = 1 , 2 .......... p

K*mA m^ ( z ) + z ' 1 B m- i ( z )

( 1 1 .2 .1 8 )

T h u s a la t t i c e filte r is d e s c r ib e d in th e z -d o m a in b y th e m a tr ix e q u a tio n

A m( z ) '

'1

K mZ

_ B m(z) _

A m- i ( z )
B m -l( z )

( 1 1 .2 .1 9 )

T h e r e la t io n s in ( 1 1 .2 .1 7 ) f o r A m(z) a n d B m(z) a llo w u s t o o b ta in t h e d ir e c t -f o r m

F I R f ilte r c o e f f ic ie n ts (a m(A)] fr o m t h e r e f le c tio n c o e f f ic ie n t s ( K m), a n d v ic e v e r s a .
T h e s e r e la tio n s h ip s w e r e g iv e n in S e c t io n 7 .2 .4 b y ( 7 .2 .5 1 ) th r o u g h ( 7 .2 .5 3 ) .
T h e l a t t i c e s t r u c tu r e w ith p a r a m e t e r s K i , K 2........ K p c o r r e s p o n d s t o a c la s s
o f p d ir e c t -f o r m F I R f ilte r s w ith s y s te m f u n c t io n s A i ( z ), A i ( z ) , . . , A p (z).

I t is

in te r e s tin g t o n o t e t h a t a c h a r a c te r iz a tio n o f th is c la s s o f p F I R filte r s in d ir e c t f o r m

r e q u ir e s p ( p + l ) / 2 f ilte r c o e f f ic ie n ts . I n c o n t r a s t , th e l a t t i c e - f o r m c h a r a c te r iz a tio n
r e q u ir e s o n ly t h e p r e f le c t i o n c o e f f ic ie n t s

T h e r e a s o n th e l a t t i c e p r o v id e s a

m o r e c o m p a c t r e p r e s e n t a t io n f o r t h e c la s s o f p F I R filte r s is b e c a u s e a p p e n d in g

862

Linear Prediction and Optimum Linear Filters

Chap. 11

s ta g e s to th e la t t ic e d o e s n o t a l t e r th e p a r a m e t e r s o f th e p r e v io u s s ta g e s . O n th e
o th e r h a n d , a p p e n d in g th e /?th s ta g e t o a la t t ic e w ith (p 1 ) s t a g e s is e q u iv a le n t
t o in c r e a s in g th e le n g th o f a n F I R filte r b y o n e c o e f f ic ie n t . T h e r e s u ltin g F I R f ilte r
w ith s y s te m fu n c t io n A p (z) h a s c o e f f ic ie n ts t o ta lly d if f e r e n t f r o m th e c o e f f ic ie n ts
o f th e lo w e r - o r d e r F I R f ilte r w ith s y s te m f u n c tio n
T h e fo r m u la f o r d e te r m in in g th e f ilt e r c o e f f ic ie n ts {ap (k ) \ r e c u r s iv e ly is e a s ily
d e riv e d fr o m p o ly n o m ia l r e la tio n s h ip s ( 1 1 .2 .1 8 ) . W e h a v e

A m(z) =
m
y ' a m(k )z k *=0

1( ' )

- f K mz ~ ] B m-

m -\

-i

( 1 1 .2 .2 0 )

* + K m

Y ' a m_ i ( i ) c
A=0

a+ 1)

i= 0
a n d r e c a llin g th a t a m( 0 ) = 1

B y e q u a tin g th e c o e f f ic ie n t s o f e q u a l p o w e r s o f
fo r m =

1 - k)z

1, 2 , ___ p , w e o b ta in th e d e s ir e d r e c u r s iv e e q u a tio n f o r th e F I R filte r

c o e f f ic ie n ts in th e fo r m
Am(0 ) =

a m(m) = K m
a,(k) = a m~ \{ k ) + K ma*n_ } (m - k)
= a m~ \( k ) + a m (m )a*

( 1 1 .2 .2 1 )

(m - k)

1 < k < m -

m = 1 .2 ,.... p
T h e c o n v e r s io n fo r m u la f r o m th e d ir e c t -fo r m F I R filte r c o e f f i c i e n t s (a^(/:)} to
th e l a t t ic e r e fle c tio n c o e f f ic ie n t s |Af, } is a ls o v e r y s im p le . F o r th e />-s ta g e la ttic e w e
im m e d ia te ly o b ta in th e r e fle c tio n c o e f f ic ie n t K p = a n (p ). T o o b t a i n K p~ i ,
w e n e e d th e p o ly n o m ia ls A m(z) f o r m = p 1 . ___ 1. F r o m ( 1 1 . 2 . 1 9 ) w e o b ta in
.

. x

A m(z) K mB m(z)

A m- i( z ) = ------ ------

~ 2

------

m = p ........1

... ,

(11.2.22)

I& m I

w h ich is ju s t a s te p -d o w n r e c u r s io n . T h u s w e c o m p u te a ll lo w e r - d e g r e e p o ly n o m ia ls

A m(z) b e g in n in g w ith A p- i ( z ) a n d o b ta in t h e .d e s ir e d l a t t ic e r e f l e c t i o n c o e f f ic ie n ts
a m(m ). W e o b s e r v e th a t th e p r o c e d u r e w o rk s a s lo n g

f r o m t h e r e la tio n K m =
a s |ATmi ^

1 fo r m =

1, 2 , . . . , p

1.

F ro m

th is s te p -d o w n r e c u r s io n f o r th e

p o ly n o m ia ls , it is r e la tiv e ly e a s y to o b t a in a fo r m u la f o r r e c u r s iv e ly a n d d ir e c tly
co m p u tin g K m, m = p 1 , . . . , 1. F o r m p - 1 , . . . , 1, w e h a v e

K m = a m(m )
Qm(k) K mbm(k)

/r

=
_

i t

iU 2 2 3 )

l - l

a m(k) - a m{m)a*m(m - k)
1 -

\am(m )\2

w h ic h is ju s t t h e r e c u r s io n in th e S c h u r - C o h n s ta b ility t e s t f o r t h e p o ly n o m ia l

A m(z).

Sec. 11.2

Forward and Backward Linear Prediction

863

A s ju s t in d ic a te d , th e r e c u r s iv e e q u a tio n in ( 1 1 .2 .2 3 ) b r e a k s d o w n i f a n y o f
t h e la t t ic e p a r a m e t e r s \ K m \ = 1. I f th is o c c u r s , it is in d ic a tiv e th a t th e p o ly n o m ia l
Am_ i ( z ) h a s a r o o t l o c a t e d o n th e u n it c ir c le . S u c h a r o o t c a n b e f a c t o r e d o u t f r o m
A m_ i ( z ) a n d th e ite r a tiv e p r o c e s s in ( 1 1 .2 .2 3 ) c a r r ie d o u t f o r th e r e d u c e d - o r d e r
s y s te m .
F in a lly , le t us c o n s i d e r th e m in im iz a tio n o f t h e m e a n - s q u a r e e r r o r in a b a c k
w a rd lin e a r p r e d ic t o r . T h e b a c k w a r d p r e d ic t io n e r r o r is

p -1

g p (n) x (n - p ) - f 2 2 bP (k )x (n - k)
( 1 1 .2 .2 4 )

= j(h - p) + ^

a* ()*(/1 ~ p + k)

k=i
a n d its m e a n -s q u a r e v a lu e is

b
p = [ M ) 1 2]

( 1 1 .2 .2 5 )

T h e m in im iz a tio n o f b
p w ith r e s p e c t to th e p r e d ic t io n c o e f f ic ie n t s y ie ld s th e s a m e
s e t o f lin e a r e q u a tio n s a s in ( 1 1 .2 .9 ) . H e n c e th e m in im u m m e a n - s q u a r e e r r o r is
m in [ { ] = E j = E ;

( 1 1 .2 .2 6 )

w h ic h is g iv e n b y ( 1 1 .2 ,1 0 ) .

11.2.3 The Optimum Reflection Coefficients for the

Lattice Forward and Backward Predictors
I n S e c t io n s 1 1 .2 .1 a n d 1 1 .2 .2 w e d e r iv e d th e s e t o f lin e a r e q u a t i o n s w h ic h p r o v id e
t h e p r e d i c t o r c o e f f ic ie n ts t h a t m in im iz e t h e m e a n - s q u a r e v a lu e o f th e p r e d ic t io n
e r r o r . In th is s e c t io n w e c o n s id e r th e p r o b le m o f o p tim iz in g t h e r e f le c tio n c o e f f i
c ie n ts in th e l a t t ic e p r e d ic t o r a n d e x p r e s s in g th e r e f le c tio n c o e f f ic ie n t s in te r m s o f
t h e fo r w a r d a n d b a c k w a r d p r e d ic t io n e r r o r s .
T h e fo r w a r d p r e d ic t io n e r r o r in th e l a t t ic e filte r is e x p r e s s e d as

f m(n) = f m^ ( n ) + K mgm- X(n - 1 )

( 1 1 .2 .2 7 )

T h e m in im iz a t io n o f E[|/m(n)|2] w ith r e s p e c t t o th e r e f le c t io n c o e f f i c i e n t K m y ie ld s
th e re s u lt

- [ / m- i ( n ) g ; _ ! ( n - l ) ]
rTi
'(
TTm
[ | g _ i ( n - 1)|2]

( 1 1 ./ .Z o ;

o r , e q u iv a le n tly ,

K m --

E [ f m- \( n ) g *

A n 1)1

, gm~ 1 -------- --

( 1 1 .2 .2 9 )

i-.
w h e r e E fm_ x = E b
m_ x = E [ \ g m^ ( n -

1)|2] = E[|/m_ i(n )| 2].

W e o b s e r v e t h a t th e o p tim u m c h o ic e o f th e r e f le c t io n c o e f f ic ie n ts in th e
l a t t i c e p r e d i c t o r is th e n e g a tiv e o f t h e ( n o r m a liz e d ) c r o s s c o r r e la t io n c o e f f ic ie n ts

864

Linear Prediction and Optimum Linear Filters

Chap. 11

b e tw e e n th e fo r w a r d a n d b a c k w a r d e r r o r s in t h e la t t ic e .* S i n c e it is a p p a r e n t fr o m
( 1 1 .2 .2 8 ) th a t |AT, | < 1 . it fo llo w s t h a t t h e m in im u m m e a n - s q u a r e v a lu e o f th e
p r e d ic tio n e r r o r , w h ich c a n b e e x p r e s s e d r e c u r s iv e ly as

E { = {1 - |

( 11. 2. 30)

is a m o n o to n ic a lly d e c r e a s in g s e q u e n c e .

11.2.4 Relationship of an AR Process to Linear

Prediction
T h e p a r a m e te r s o f a n A R (/ ? ) p r o c e s s a r e i n tim a te ly r e la te d t o a p r e d ic t o r o f o r d e r

p f o r th e s a m e p r o c e s s .

T o s e e th e r e la tio n s h ip , w e r e c a ll th a t in a n A R ( p )

p r o c e s s , th e a u t o c o r r e la t io n s e q u e n c e { y ^ { m ) } is r e la te d to t h e p a r a m e te r s {a *}
b y t h e Y u l e - W a l k e r e q u a tio n s g iv e n in ( 1 1 .1 .1 9 ) o r ( 1 1 .1 .2 0 ) . T h e c o r r e s p o n d in g
e q u a tio n s f o r th e p r e d ic t o r o f o r d e r p a r e g iv e n b y ( 1 1 .2 .9 ) a n d ( 1 1 .2 .1 0 ) .
A d ir e c t c o m p a r is o n o f t h e s e tw o s e ts o f r e l a t i o n s r e v e a ls t h a t t h e r e is a o n e t o - o n e c o r r e s p o n d e n c e b e tw e e n th e p a r a m e te r s { a * } o f th e A R ( p ) p r o c e s s a n d th e
p r e d ic t o r c o e ff ic ie n ts {ap (k)} o f th e p t h - o r d e r p r e d ic t o r .

In f a c t , i f th e u n d e rly

in g p r o c e s s l x ( n ) } is A R ( p ) , th e p r e d ic t io n c o e f f i c ie n t s o f th e p t h - o r d e r p r e d ic t o r
a r e id e n tic a l to {ak}. F u r t h e r m o r e , th e m in im u m M S E in th e / ?th -o rd e r p r e d ic t o r
/ is id e n tic a l to crj,, th e v a r ia n c e o f th e w h ite n o is e p r o c e s s .

In th is c a s e , th e

p r e d ic t io n - e r r o r filte r is a n o is e -w h ite n in g f ilte r w h ic h p r o d u c e s th e in n o v a tio n s

s e q u e n c e (u ;{n )).

11.3 SOLUTION OF THE NORMAL EQUATIONS

In th e p r e c e d in g s e c tio n w e o b s e r v e d th a t th e m in im iz a t io n o f th e m e a n - s q u a r e
v a lu e o f th e fo r w a r d p r e d ic t io n e r r o r r e s u lte d in a s e t o f l in e a r e q u a tio n s f o r th e
c o e f f ic ie n ts o f th e p r e d ic t o r g iv e n b y ( 1 1 .2 .9 ) . T h e s e e q u a t i o n s , c a lle d t h e n o r m a l
e q u a tio n s , m a y b e e x p r e s s e d in th e c o m p a c t fo r m

Y 2 ,ap ^ y ^ 1

= 0

* =0

1 = 1 , 2 .......... p

(n .3 .1 )

0 ,( 0 ) = 1

T h e r e s u ltin g m in im u m M S E ( M M S E ) is g iv e n b y ( 1 1 .2 .1 0 ) . I f w e a u g m e n t
( 1 1 .2 .1 0 ) t o t h e n o r m a l e q u a tio n s g iv e n b y ( 1 1 .3 .1 ) w e o b ta in t h e s e t o f a u g m en te d

n o r m a l e q u a tio n s, w h ic h m a y b e e x p r e s s e d as
; :

2 .......... P

(1 U .2 ,

W e a ls o n o te d t h a t i f th e r a n d o m p r o c e s s is a n A R ( p ) p r o c e s s , t h e M M S E E p = a 2 .

The normalized crosscorrelation coefficients between the forward and backward error in the
lattice (i.e.,
are often called the partial correlation (PARCOR) coefficients.

Sec. 11.3

865

Solution of the Normal Equations

In th is s e c tio n w e d e s c r ib e tw o c o m p u ta tio n a lly e f f ic ie n t a lg o r it h m s f o r s o lv

in g th e n o r m a l e q u a tio n s . O n e a lg o r it h m , o r ig in a lly d u e t o L e v in s o n ( 1 9 4 7 ) a n d
m o d ifie d b y D u r b in ( 1 9 5 9 ) , is c a lle d th e L e v in s o n - D u r b in a lg o r ith m . T h i s a l g o
r ith m is s u ita b le f o r s e r ia l p r o c e s s in g a n d h a s a c o m p u ta tio n c o m p le x ity o f 0 ( p 2 ).
T h e s e c o n d a lg o r it h m , d u e t o S c h ii r ( 1 9 1 7 ) , a ls o c o m p u te s t h e r e f le c tio n c o e f f i
c ie n ts in 0 ( p 2) o p e r a tio n s b u t w ith p a r a lle l p r o c e s s o r s , th e c o m p u ta tio n s c a n b e
p e r f o r m e d in O ( p ) tim e . B o t h a lg o r it h m s e x p lo it th e T o e p lit z s y m m e tr y p r o p e r ty
in h e r e n t in th e a u t o c o r r e la t io n m a tr ix .
W e b e g in b y d e s c r ib in g th e L e v in s o n - D u r b in a lg o r ith m .

11.3.1 The Levinson-Durbin Algorithm

T h e L e v in s o n - D u r b in a lg o r it h m is a c o m p u ta tio n a lly e f fic ie n t a lg o r it h m f o r s o lv in g
th e n o r m a l e q u a tio n s in ( 1 1 .3 .1 ) f o r t h e p r e d ic t io n c o e f f ic ie n ts .

T h i s a lg o r ith m

e x p lo its th e s p e c ia l s y m m e tr y in th e a u t o c o r r e la t io n m a tr ix
> x ,( 0 )

k; , (

y ,,( D

. Y x x i p - 1)
N o te th a t

rp(i,j) =
j) ,
rp(i,j) = T*(j, i ) ,

m a trix . S in c e
The

key

to

th e

i)

* A p ~ 1)

Yxx(O)

Y*x i P ~ 2 )

Y x x (p ~ 2)

y^O )

( 1 1 .3 .3 )

s o t h a t t h e a u t o c o r r e la t i o n m a tr ix is a T o e p lit z
th e m a tr ix is a ls o H e r m itia n .

L e v in s o n - D u r b in

m eth o d

o f s o lu tio n ,

t h a t e x p lo its th e

T o e p l it z p r o p e r ty o f th e m a tr ix , is to p r o c e e d r e c u r s iv e ly , b e g in n in g w ith a p r e
d ic t o r o f o r d e r m = 1 ( o n e c o e f f i c i e n t ) a n d th e n t o in c r e a s e th e o r d e r r e c u r s iv e ly ,
u s in g th e lo w e r - o r d e r s o lu tio n s to o b ta in t h e s o lu tio n t o th e n e x t- h ig h e r o r d e r .
T h u s th e s o lu tio n t o th e f ir s t- o r d e r p r e d ic t o r o b t a in e d b y s o lv in g ( 1 1 .3 .1 ) is

o i(l) = -

Yxx( 1)

( 1 1 .3 .4 )

y*.t ( 0 )

a n d th e re s u ltin g M M S E is
e{

= yj (0 )+ f l i ( l ) y ( - l )
( 1 1 .3 .5 )

= y ^ (0)[l - jfli (1)I2]

R e c a l l th a t a j ( l ) = K \ , th e first r e fle c tio n c o e f f i c ie n t in th e l a t t ic e f ilte r .
T h e n e x t s te p is t o s o lv e f o r th e c o e f f ic ie n t s {o 2( I ) , ^ 2 (2 )} o f th e s e c o n d - o r d e r
p r e d ic t o r a n d e x p r e s s th e s o lu tio n in te r m s o f f l i ( l ) . T h e tw o e q u a tio n s o b ta in e d
fr o m ( 1 1 .3 .1 ) a r e

a 2 ( l ) y xx( 0 ) + a 2 (2)y * x ( l ) ~ Y x x ( 1)

(11.3.6)

02( l ) y ^ ( l )

+ a 2 ( 2 ) y xx(0) = - y xx( 2 )

866

Linear Prediction and Optimum Linear Filters

Chap. 11

B y u s in g th e s o lu tio n in ( 1 1 .3 .4 ) to e lim in a te yJ r ( l ) , w e o b ta in th e s o lu tio n

a 2 (2) = -

y xx(2) + a i ( 1 ) ^ ( 1 )
X r*(0 )[l -

| fll(l)i2]

Y jcjc( 2 ) - f a \ { l ) y xx( l )

^2(1)

fli d )

( 1 1 .3 .7 )

+ a 2(2)aUl)

T h u s w e h a v e o b ta in e d th e c o e ffic ie n ts o f th e s e c o n d - o r d e r p r e d ic t o r . A g a in , w e
n o t e t h a t ct2(2) = K 2 , th e s e c o n d r e f le c tio n c o e f f ic ie n t in th e la t t i c e f ilte r .
P r o c e e d in g in th is m a n n e r , w e c a n e x p r e s s th e c o e f f i c i e n t s o f th e m th - o r d e r
p r e d ic t o r in te r m s o f th e c o e f f ic ie n ts o f th e (m l ) s t - o r d e r p r e d ic t o r . T h u s w e c a n
w rite th e c o e f f ic ie n t v e c t o r

am as

th e su m o f tw o v e c to r s , n a m e ly .

~ a m( 1) *
a m(2)

" d m_ , "

_ f l m( m) w h ere

a,_i

( 1 1 .3 .8 )

am =
0

_Km _

is th e p r e d ic t o r c o e f f ic ie n t v e c t o r o f th e (m ~ l ) s t - o r d e r p r e d ic t o r a n d

th e v e c t o r d m_ j a n d th e s c a la r K m a r e to b e d e te r m in e d . L e t u s a ls o p a r titio n th e

m x m a u t o c o r r e la t io n m a tr ix T r* as

r#n-l

r m=

7m-1

(1 1 .3 .9 )

y ^ (0 )
= [y xx(m -

w h ere

1)

yxx(m -

2)

y xx( 1 )] = ( 7 * _ ] ) r , th e a s te r is k ( * ) d e

n o te s th e c o m p le x c o n ju g a t e , a n d y rm d e n o te s th e tr a n s p o s e o f y m. T h e s u p e r s c r ip t

b on

7 m_ ,

d e n o t e s th e v e c t o r

7^

, = [y (l)

y xx(2 )

y xx (m - 1 ) ] w ith e l e

m e n ts ta k e n in r e v e r s e o r d e r .
T h e s o lu tio n to th e e q u a tio n r ma m = y m c a n b e e x p r e s s e d a s
1 f
.1

a m- i
_0

um
k

] ) [ * > ]

<i m o >

T h is is th e k e y s te p in th e L e v in s o n - D u r b in a lg o r it h m . F r o m ( 1 1 .3 .1 0 ) w e o b ta in
tw o e q u a tio n s , n a m e ly ,
+ K my b*_x 7 m - i a m -1 + 7 t '_ i d m - i + K myxx{0 ) =

- y m_ x

( 1 1 .3 .1 1 )

- y xx(m )

( 1 1 .3 .1 2 )

S in c e r m_ i a m_ i = - y m_ : , ( 1 1 .3 .1 1 ) y ie ld s th e s o lu tio n

dm_!
But

7 **_ j

is ju s t

7 m_ !

= - K mT - \ xy b
;_ ,

( 1 1 .3 .1 3 )

w ith e le m e n t s ta k e n in r e v e r s e o r d e r a n d c o n ju g a te d . T h e r e -

Sec. 11.3

867

Solution of the Normal Equations

f o r e , th e s o lu tio n in ( 1 1 .3 .1 3 ) is sim p ly
'< C _ ] ( " J - 1 )
d m_ , = K ma * * .! = K m

flmi

-2 )

( 1 1 .3 .1 4 )

T h e s c a la r e q u a tio n ( 1 1 .3 .1 2 ) c a n n o w b e u s e d to s o lv e f o r K m. I f w e e lim in a te
d m_ i in ( 1 1 .3 .1 2 ) b y u s in g ( 1 1 .3 .1 4 ) , w e o b ta in

Km [Yxx(Q) +

+ 'Ym^ia m -l = ~ Y x x ( )

H en ce

T h e r e f o r e , b y s u b s titu tin g th e s o lu tio n s in ( 1 1 .3 .1 4 ) a n d ( 1 1 .3 .1 5 ) i n t o ( 1 1 .3 .8 ) ,

w e o b ta in th e d e s ir e d r e c u r s io n f o r th e p r e d ic t o r c o e f f i c i e n t s in th e L e v i n s o n D u r b in a lg o r ith m as
, ,
^
Yxx(m) +
a m_ ,
y xx(m ) +
a m(m ) = K m = --------------------- j- ------ T-. = --------------------- ---------------

K,,<0)

( 1 1 .3 .1 6 )

a ( k ) = a m- i ( k ) + K ma*m_ x(m - k)
= a m_ i( k ) + a m{ m ) a ^ ( m ~ k )

k -\2 ,...,m
tn = i , z , . . . , p

^1 1 3

T h e r e a d e r s h o u ld n o t e t h a t th e r e c u r s iv e r e la tio n in ( 1 1 .3 .1 7 ) is id e n tic a l to th e
r e c u r s iv e r e la t io n in ( 1 1 .2 .2 1 ) f o r th e p r e d ic t o r c o e f f ic ie n t s , o b ta in e d f r o m th e
p o ly n o m ia ls A m( z ) a n d B m(z). F u r t h e r m o r e , K m is t h e r e f le c t io n c o e f f ic ie n t in th e
m th s ta g e o f t h e l a t t ic e p r e d ic t o r .

T h i s d e v e lo p m e n t c le a r ly illu s tr a te s th a t th e

L e v in s o n - D u r b in a lg o r it h m p r o d u c e s th e r e f le c tio n c o e f f i c ie n t s f o r th e o p tim u m
l a t t i c e p r e d ic t io n f ilte r a s w e ll a s th e c o e f f ic ie n ts o f th e o p tim u m d ir e c t -f o r m F I R
p r e d ic t o r .
F in a lly , le t u s d e te r m in e th e e x p r e s s io n f o r th e M M S E . F o r t h e m th -o r d e r
p r e d ic t o r , w e h a v e
m

E fm = yjrjr(O) + 2 2 a >(k ) Y x x ( -lc )

yxx(O ) + ^ [ o m^ i ( / c ) + f l m ( m ) f l * _ 1( r n - * ) ] y ( - / c )
i= l

E fm_ l [ \ - \ a m(m )\2\ = E ftn^ \ - \ K

m\2)

( 1 1 .3 .1 8 )

m = 1,2,...,

w h e r e E ^ = y ( 0 ) . S in c e th e r e fle c tio n c o e f f ic ie n t s s a tis fy t h e p r o p e r ty t h a t |ATm| <

1 , th e M M S E f o r t h e s e q u e n c e o f p r e d ic t o r s s a tis fie s th e c o n d itio n

E { > E { > E f2 > - - - > E pf

(11.3.19)

Linear Prediction and Optimum Linear Filters

868

Chap. 11

T h is c o n c lu d e s th e d e r iv a tio n o f th e L e v in s o n - D u r b in a lg o r it h m f o r so lv in g
th e lin e a r e q u a tio n s T ma m = y m, f o r m = 0 , 1

p . W e o b s e r v e th a t th e lin e a r

e q u a tio n s h a v e th e s p e c ia l p r o p e r ty th a t th e v e c t o r o n th e r ig h t- h a n d s id e a ls o
a p p e a r s as a v e c t o r in

rm.

In th e m o r e g e n e r a l c a s e , w h e r e th e v e c t o r o n th e

rig h t-h a n d s id e is s o m e o t h e r v e c to r , s a y c m, th e s e t o f li n e a r e q u a tio n s c a n b e

s o lv e d r e c u rs iv e ly b y in tr o d u c in g a s e c o n d r e c u r s iv e e q u a t io n t o s o lv e th e m o r e
g e n e r a l lin e a r e q u a tio n s

rmbm= cm. T h e

re s u lt is a g e n e r a liz e d L e v in s o n - D u r b i n

a lg o r ith m ( s e e P r o b le m 1 1 .1 2 ) .
T h e L e v in s o n - D u r b in r e c u r s io n g iv e n b y ( 1 1 .3 .1 7 ) r e q u ir e s 0 ( m ) m u ltip lic a
tio n s a n d a d d itio n s ( o p e r a t i o n s ) to g o fr o m s ta g e m to s ta g e m + 1. T h e r e f o r e , f o r

p s ta g e s it ta k e s o n th e o r d e r o f 1 + 2 + 3-1-------- (- p { p + 1)/ 2, o r 0 ( p 2 ), o p e r a tio n s to

s o lv e f o r th e p r e d ic t io n f ilte r c o e ff ic ie n ts , o r th e r e f le c tio n c o e f f i c i e n t s , c o m p a r e d
w ith O ( p ^ ) o p e r a tio n s if w e d id n o t e x p lo it th e T o e p lit z p r o p e r ty o f th e c o r r e la tio n
m a tr ix .
I f th e L e v in s o n - D u r b in a lg o r ith m is im p le m e n te d o n a s e r ia l c o m p u te r o r
s ig n a l p r o c e s s o r , th e r e q u ir e d c o m p u ta tio n tim e is o n th e o r d e r o f 0 ( p 2 ) tim e
u n its . O n th e o t h e r h a n d , i f th e p r o c e s s in g is p e r fo r m e d o n a p a r a lle l p r o c e s s in g
m a c h in e u tiliz in g a s m a n y p r o c e s s o r s a s n e c e s s a r y t o e x p lo it th e fu ll p a r a lle lis m
in th e a lg o r ith m , th e m u ltip lic a tio n s a s w e ll a s th e a d d itio n s r e q u ir e d to c o m p u te
( 1 1 .3 .1 7 ) c a n b e c a r r ie d o u t s im u lta n e o u s ly . T h e r e f o r e , th is c o m p u ta tio n c a n b e
p e r f o r m e d in O ( p ) tim e u n its . H o w e v e r , th e c o m p u ta tio n in ( 1 1 .3 .1 6 ) f o r th e r e
f le c tio n c o e ff ic ie n ts ta k e s a d d itio n a l tim e . C e r ta in ly , th e in n e r p r o d u c ts in v o lv in g
th e v e c to r s a m_ i a n d
p ro cesso rs.

c a n b e c o m p u te d s im u lta n e o u s ly b y e m p lo y in g p a r a lle l

H o w e v e r , th e a d d itio n o f th e s e p r o d u c ts c a n n o t b e d o n e s im u lta n e

o u s ly , b u t in s te a d , r e q u ir e 0 ( \ o g p ) tim e u n its .

H e n c e , th e c o m p u ta tio n s in th e

L e v in s o n - D u r b in a lg o r it h m , w h e n p e r f o r m e d b y p p a r a lle l p r o c e s s o r s , c a n b e a c
c o m p lis h e d in 0 ( p \ o g p ) tim e .
In th e f o llo w in g s e c tio n w e d e s c r ib e a n o t h e r a lg o r it h m , d u e to S c h iir ( 1 9 1 7 ) ,
th a t a v o id s th e c o m p u ta tio n o f in n e r p r o d u c ts , a n d t h e r e f o r e is m o r e s u ita b le f o r
p a r a lle l c o m p u ta tio n o f th e r e f le c tio n c o e ff ic ie n ts .

11.3.2 The Schur Algorithm

T h e S c h u r a lg o r ith m is in tim a te ly r e la te d to a r e c u r s iv e t e s t f o r d e te r m in in g th e
p o s it iv e d e fin it e n e s s o f a c o r r e la t io n m a tr ix .

T o b e s p e c ific , l e t u s c o n s id e r th e

a u t o c o r r e la tio n m a tr ix T ^ + i a s s o c ia t e d w ith th e a u g m e n t e d n o r m a l e q u a tio n s g iv e n

b y ( 1 1 .3 .2 ) . F r o m th e e le m e n ts o f th is m a tr ix w e f o r m t h e fu n c t io n

y(l)z_I +Y*A2)z- 2 + --- + yxAp)z-p

^ (0 ) + r ( l ) z - V

- + y ( P n-<>

( 1 U '2 0 )

a n d th e s e q u e n c e o f fu n c t io n s R m(z) d e fin e d r e c u r s iv e ly as
D i \

R m(z) -

Rm - 1 ( z ) R m - l ( o o )
------ rr

z *[1 - / ?*_ ! ( o o ) ( z ) J

m = 1,2,...

/1 1 o - i n

( 1 1 .3 .2 1 )

Sec. 11.3

869

Solution of the Normal Equations

S c h i i r s t h e o r e m s t a t e s t h a t a n e c e s s a r y a n d s u ffic ie n t c o n d itio n f o r th e c o r
r e l a t i o n m a t r ix to b e p o s itiv e d e f in it e is t h a t |/?m(oo)| < 1 f o r m = 1, 2 , . . . , p .
L e t u s d e m o n s tr a t e th a t th e c o n d itio n f o r p o s itiv e d e fin ite n e s s o f th e a u t o c o r
r e la t io n m a t r ix I ^ + i is e q u iv a le n t t o th e c o n d itio n t h a t th e r e f le c tio n c o e f f ic ie n ts
in t h e e q u iv a le n t la t t ic e f ilt e r s a tis f y t h e c o n d itio n |K m \ < 1, m = 1, 2 , . . . , p .
F ir s t , w e n o t e th a t R o(oo) = 0 . T h e n , fr o m ( 1 1 .3 .2 1 ) w e h a v e

Yxx(l) +

X x x (2 )z _1 + + Yxx(p)z~p+l
-----------------------------------

R \ ( 2) ---------^

Kr*(0) + y (l)z -1 + + Yxx(p)z~p

^
( 1 1 .3 .2 2 )

H e n c e r t i( o o ) = yxx( l ) / y Xx(0 ). W e o b s e r v e t h a t R \( o o ) = K \ .
S e c o n d , w e c o m p u te R j( z ) a c c o r d in g to ( 1 1 .3 .2 1 ) a n d e v a lu a te th e r e s u lt a t

z = 00 . T h u s w e o b ta in

Yxx(2) + K ^ x x d )

x(0)(l - l^ii2)
A g a in , w e o b s e r v e t h a t # 2( 00) = K j . B y c o n tin u in g th is p r o c e d u r e , w e fin d th a t

R m(oo) = K m, f o r m = 1, 2 , . . . , p . H e n c e th e c o n d itio n |J?m(oo)| < 1 f o r m = 1,

2 , . . . , p , is id e n tic a l to th e c o n d itio n |

| < 1 f o r m = 1, 2 , . . . , p , a n d e n s u r e s th e

p o s it iv e d e f in it e n e s s o f th e a u t o c o r r e l a t i o n m a tr ix r p+i.
S i n c e th e r e fle c tio n c o e f f ic ie n t s c a n b e o b ta in e d fr o m th e s e q u e n c e o f f u n c
tio n s R m( 2), m =

1, 2 , . . . , p , w e h a v e a n o t h e r m e th o d f o r s o lv in g th e n o r m a l

e q u a tio n s . W e c a ll th is m e t h o d th e S c h u r a lg o rith m .

Schur algorithm.

L e t us firs t r e w r ite R m(z) as

*<z) = 7Q mfrr
(z)

m= l ' P

( 1 1 .3 .2 3 )

w h ere

P o (z) = Y x x O ) Z ~ X + Y x x ( 2 ) z ~ 2 H--------- h Y x x (p )z ~ p
( 1 1 .3 .2 4 )

Q a (z) = Y xxi 0 ) + Y x x ( l) z ~ l + -------h Y x x (p )z ~ p

S in c e

^0

0 a n d K m R m(oo) f o r m =

1 , 2 , . . . , p , th e r e c u r s iv e e q u a tio n

( 1 1 .3 .2 1 ) im p lie s th e fo llo w in g r e c u r s iv e e q u a t io n s f o r th e p o ly n o m ia ls P m(z) a n d

Qm(z):
[Q m (z)j

Z_1

r p m_ i ( z )
[_ Q m \ (z )

T h u s w e have

P l( z ) =

P o (z) = X t x ( l ) z _1 -I- Y x x (2 )z ~ 2 + + Y x x (p )z ~ p
( 1 1 .3 .2 6 )

f i i ( z ) = z -1 Q o (z ) = Y xx(0 )z - 1 + y ( l ) z - 2 + - + y xx( p ) z ~ p- 1

and

K\ =

P i( z )
Q iiz )

Yxx(1)
Vxx(0)

(11.3.27)

870

Linear Prediction and Optimum Linear Filters

Chap. 11

N e x t th e r e fle c tio n c o e f f ic ie n t K i is o b ta in e d b y d e te r m in in g P 2 (z) a n d Q 2 U ) fro m

( 1 1 .3 .2 5 ) , d iv id in g P 2 (z) b y Q i( z ) a n d e v a lu a tin g th e re s u lt a t z = o o . T h u s w e fin d
th a t

P i( z ) =
=

P \( z ) + K xQ x{z)
[Yxx( 2 ) + K i y xx( l ) \ z ~ 2 +
+ [Y x A p ) + * 1 Y x x ip -

1 )]z~ p
( 1 1 .3 .2 8 )

Q i( z ) =
=

+
[x rJf(0 ) + Js:1v ( i ) ] z - 2 + - - + [Y x x ip -

2 ) + K * y xx( p - 1 ) } z ~ p

w h e r e th e te r m s in v o lv in g z ~ p~ l h a v e b e e n d r o p p e d .

T h u s w e o b s e r v e th a t th e

re c u r s iv e e q u a tio n in ( 1 1 .3 .2 5 ) is e q u iv a le n t to ( 1 1 .3 .2 1 ) .
B a s e d o n th e s e r e la tio n s h ip s , th e S c h u r a lg o r ith m is d e s c r ib e d b y th e fo llo w
in g re c u r s iv e p r o c e d u r e .

In it ia liz a t io n . F o r m th e 2 x ( p + 1) g e n e r a t o r m a tr ix
0
y ^ (0 )

Gn =

Kv-r(l)
y^O )

Yxx( 2 )
y xx (2 )

^ ( Z 7) ]

y xx{ p ) \

w h e r e th e e le m e n t s o f th e firs t ro w a r e th e c o e f f i c i e n t s o f P q( z ) a n d th e
e le m e n ts o f th e s e c o n d ro w a r e th e c o e f f ic ie n ts o f ?o(z).

Step 1.

S h ift th e s e c o n d ro w o f th e g e n e r a t o r m a tr ix t o th e rig h t by o n e

p la c e a n d d is c a r d th e la s t e le m e n t o f th is ro w . A z e r o is p la c e d in th e v a c a n t
p o s itio n . T h u s w e o b ta in a n e w g e n e r a to r m a tr ix ,
G, =

Yxx( 1)
y xx (0 )

0
.0

Y xxO )
y xx( 1)

Yx A 2 )
yxx{ 1)

y xA p )
y xx(p - 1 )

( 1 1 .3 .3 0 )

T h e (n e g a tiv e ) r a t io o f t h e e le m e n ts in th e s e c o n d c o lu m n y ie ld th e r e f le c tio n
c o e f f ic ie n t K \ = - y xA l ) / y xx {0).

Step 2. M u ltip ly th e g e n e r a t o r m a tr ix b y th e 2 x 2 m a tr ix
Vi =

Ki

(1 1 .3 .3 1 )

T h u s w e o b ta in

V!G1=

y < 2 ) + K xYxx{ \ )

y (0 ) + ^ * y ( l)

Y x x ip ) + K \ y xx{ p - 1)
Y x x ip -

1 ) + K * y xx(p )_
( 1 1 .3 .3 2 )

Step 3. S h ift th e s e c o n d ro w o f V i G i b y o n e p la c e t o th e r ig h t a n d th u s fo rm
th e n e w g e n e r a t o r m a tr ix
G: =

yxx(2) + K i y xx( l )

y xx(p ) + K \ y X I (p - 1)

L0

y xA 0 ) + K ; y xx( \ )

Yxx( p - 2 ) + K \ y xx( p - l )
( 1 1 .3 .3 3 )

T h e n e g a tiv e r a t io o f th e e le m e n ts in th e th ir d c o lu m n o f G

y ie ld s K

2.

Sec. 11.3

871

Solution of the Normal Equations

S te p s 2 a n d 3 a r e r e p e a t e d u n til w e h a v e s o lv e d f o r a ll p r e f le c tio n c o e f f ic ie n ts .
In g e n e r a l, th e 2 x 2 m a tr ix in s te p 2 is
1

K*

( 1 1 .3 .3 4 )

a n d m u ltip lic a tio n o f V m b y G m y ie ld s V mG m. In s te p 3 w e s h ift th e s e c o n d ro w

o f V mG m o n e p la c e t o th e rig h t a n d o b ta in th e n e w g e n e r a t o r m a tr ix G m+i .
W e o b s e r v e th a t th e s h iftin g o p e r a tio n o f th e s e c o n d ro w in e a c h ite r a tio n
is e q u iv a le n t t o m u ltip lic a tio n b y th e d e la y o p e r a t o r z ~ x in t h e s e c o n d re c u r s iv e
e q u a tio n in ( 1 1 .3 .2 5 ) . W e a ls o n o te t h a t th e d iv is io n o f th e p o ly n o m ia l P m(z) b y
th e p o ly n o m ia l Q m(z) a n d th e e v a lu a tio n o f t h e q u o tie n t a t z = oo is e q u iv a le n t
to d iv id in g th e e le m e n ts in th e (m + l ) s t c o lu m n o f G m.

T h e c o m p u ta tio n o f

th e p r e fle c tio n c o e f f ic ie n ts c a n b e a c c o m p lis h e d b y u se o f p a r a lle l p r o c e s s o r s in

0 { p ) tim e u n its . B e lo w w e d e s c r ib e a p ip e lin e d a r c h it e c t u r e f o r p e r f o r m in g t h e s e

c o m p u ta tio n s .
A n o t h e r w ay o f d e m o n s tr a t in g th e r e la tio n s h ip o f th e S c h u r a lg o r it h m t o th e
L e v in s o n - D u r b in a lg o r ith m a n d th e c o r r e s p o n d in g l a t t ic e p r e d i c t o r is to d e te r m in e
th e o u tp u t o f th e la ttic e filte r o b ta in e d w h e n th e in p u t s e q u e n c e is th e c o r r e la tio n
s e q u e n c e {y.r.v(m ), m = 0 . 1 , . . . ) , T h u s , th e first in p u t to th e l a t t i c e f ilte r is yx_v( 0 ) ,

yxx(n )].

th e s e c o n d in p u t is y ( l ) , a n d s o o n [i.e ., / ( ) =
th e firs t s ta g e , w e h a v e g o(n -

1) =

yxx(n ~ \ ) .

A f t e r th e d e la y in

H e n ce , fo r n =

1, th e r a t io

./ ii(l)/ g o(0) = K r,(l)/ y jt.v (0 K w h ich is th e n e g a tiv e o f th e r e f le c tio n c o e f f ic ie n t K \ .

A lt e r n a t iv e ly , w e c a n e x p r e s s th is r e la tio n s h ip a s

/o(D + ^igo(O ) = X n ( l ) + ^ iyjjc(O ) = 0

F u r t h e r m o r e , # o (0 ) = yxx( 0 ) =

. A t tim e n = 2 , th e in p u t to th e s e c o n d s ta g e

is, a c c o r d in g to ( 1 1 .2 .4 ) ,
/ ,( 2 ) = M 2 ) + K ^ g o G ) = y xx( 2 ) + K lY x x ( l )
a n d a f t e r th e u n it o f d e la y in th e s e c o n d s ta g e , w e h a v e
g,(l)

K * M l ) + g o ( 0 ) = K l y xx(1)

y ( 0 )

N o w th e r a t io f \ { 2 ) / g \ ( \ ) is
/ i(2 )

y xx(2) + K lY x x ( l )

si(l)

y x x(0) + ^ K ( l )

yxx(2) + K \ y xx{ \ )
e

= - k 2

H en ce
/i (2 ) + K i g \ ( \ ) =
51 (1 ) =

E{

B y c o n tin u in g in th is m a n n e r , w e c a n s h o w t h a t a t t h e in p u t t o th e m th l a t t i c e s ta g e ,
th e r a t io f m- \ ( m ) / g m- i ( m

1) = - K m a n d g m- i ( m

~ 1) = E }m_ v

C o n s e q u e n tly ,

th e la t t ic e f ilt e r c o e f f ic ie n ts o b ta in e d f r o m th e L e v in s o n a lg o r it h m a r e id e n tic a l
to t h e c o e f f i c i e n t s o b ta in e d in th e S c h u r a lg o r it h m . F u r t h e r m o r e , th e l a t t ic e filte r

872

Linear Prediction and Optimum Linear Filters

Chap. 11

s tr u c tu r e p r o v id e s a m e t h o d f o r c o m p u tin g th e r e f le c t io n c o e f f i c i e n t s in th e la ttic e
p r e d ic to r .

A pipelined architecture for implementing the Schur algorithm.

Kung

a n d H u ( 1 9 8 3 ) d e v e lo p e d a p ip e lin e d la ttic e - ty p e p r o c e s s o r f o r im p le m e n tin g th e

S c h u r a lg o r ith m . T h e p r o c e s s o r c o n s is ts o f a c a s c a d e o f p l a t t i c e - t y p e s ta g e s , w h e r e
e a c h s ta g e c o n s is ts o f tw o p r o c e s s in g e le m e n t s ( P E s ) , w h ic h w e d e s ig n a te a s u p p e r
P E s d e n o te d a s A \ , A z . . . . , A p , a n d lo w e r P E s d e n o te d a s S j , B 2 .......... B p, a s sh o w n
in F ig . 1 1 .5 . T h e P E d e s ig n a te d a s A \ is a s s ig n e d t h e ta s k o f p e r f o r m in g d iv isio n s.
T h e r e m a in in g P E s p e r fo r m o n e m u ltip lic a tio n a n d o n e a d d itio n p e r i te r a tio n ( o n e
c lo c k c y c le ) .
I n itia lly , t h e u p p e r P E s a r e lo a d e d w ith th e e le m e n ts o f t h e firs t ro w o f th e
g e n e r a t o r m a tr ix G o . as illu s tr a te d in F ig . 1 1 .5 . T h e lo w e r P E s a r e lo a d e d w ith th e
e le m e n ts o f th e s e c o n d ro w o f th e g e n e r a t o r m a tr ix G o- T h e c o m p u t a t io n a l p r o c e s s
b e g in s w ith th e d iv is io n P E , A \, w h ic h c o m p u te s t h e firs t r e f l e c t i o n c o e f f ic ie n t as

K \ = yJ J :( l ) / K t j( 0 ) . T h e v a lu e o f K \ is s e n t s im u lta n e o u s ly to a ll th e P E s in th e
u p p e r b r a n c h a n d lo w e r b r a n c h .
T h e s e c o n d s te p in th e c o m p u ta tio n is to u p d a te th e c o n t e n t s o f a ll p r o c e s s in g
e le m e n ts s im u lta n e o u s ly . T h e c o n te n ts o f th e u p p e r a n d l o w e r P E s a r e u p d a te d
a s fo llo w s ;
P E A m: A m < A m + K i B m

m = 2 , 3 .......... p

P E B m: B m * B m + K \ A m

m = 1,2,.... p

T h e th ird s te p in v o lv e s th e s h iftin g o f th e c o n te n ts o f th e u p p e r P E s o n e
p la c e t o th e le ft. T h u s w e h a v e
P E A m\ A - i + - A m

m = 2 , 3 .......... p

A t th is p o in t, P E A \ c o n ta in s yxx( 2 ) + K \ y xx( \ ) w h ile P E

B \ c o n ta in s y ( 0 ) +

K * y xx( \ ) . H e n c e th e p r o c e s s o r A \ is r e a d y t o b e g in th e s e c o n d c y c le b y c o m p u tin g

Figure 11.5 Pipelined parallel processor for computing the reflection coefficients

Sec. 11.4

Properties of the Linear Prediction-Error Filters

873

th e s e c o n d r e f le c tio n c o e f f i c i e n t K

b e g in n in g w ith th e d iv is io n A \ / B \

a r e r e p e a t e d u n til all p r e f le c tio n c o e f f ic ie n ts

-A \/B \.

T h e t h r e e c o m p u ta tio n a l s te p s

a r e c o m p u te d . N o te th a t P E B \ p r o v id e s th e m in im u m m e a n - s q u a r e e r r o r e L f o r
e a c h i te r a tio n .
I f t <j d e n o t e s th e tim e f o r P E A \ to p e r f o r m a ( c o m p l e x ) d iv is io n a n d rma
is th e tim e r e q u ir e d to p e r fo r m a ( c o m p le x ) m u ltip lic a tio n a n d an a d d itio n , th e
tim e r e q u ir e d t o c o m p u te a ll p r e fle c tio n c o e f f ic ie n ts is p (z^ + zma) f o r th e S c h u r
a lg o r ith m .

11.4 PROPERTIES OF THE LINEAR PREDICTION-ERROR FILTERS

L i n e a r p r e d ic t io n filte r s p o s s e s s s e v e r a l im p o r ta n t p r o p e r tie s , w h ich w e n o w d e
s c r ib e . W e b e g in b y d e m o n s tr a t in g th a t th e f o r w a r d p r e d ic t io n - e r r o r f ilte r is m in
im u m p h a s e .

Minimum-phase property of the forward prediction-error filter.

We

h a v e a lr e a d y d e m o n s tr a t e d th a t th e r e fle c tio n c o e f f ic ie n ts { AT,} a r e c o r r e la tio n c o

e f f ic ie n ts , a n d c o n s e q u e n tly , |A", | <
= (1 \ K

1 fo r all i .

T h is c o n d itio n a n d th e re la tio n

b e u se d to s h o w th a t th e z e r o s o f th e p r e d ic t io n - e r r o r

f ilt e r a r e e it h e r all in s id e th e u n it c ir c le o r th e y a r e all o n th e u n it c ir c le .

F ir s t , w e s h o w th a t i f e J, > 0 , th e z e r o s |z,| < 1 f o r e v e r y i. T h e p r o o f is by
in d u c tio n . C le a r ly , fo r p = 1 th e s y s te m fu n c t io n f o r th e p r e d i c t i o n - e r r o r f ilte r is

A A z ) = 1 + K i z -1

( 1 1 .4 .1 )

H e n c e z \ = K \ a n d e { = (1 - |K \ \ 2) E l > 0 . N o w , s u p p o s e th a t th e h y p o th e s is is
tr u e f o r p -

1. T h e n , i f n is a r o o t o f A p { z ) , w e h a v e f r o m ( 1 1 .2 .1 6 ) a n d ( 1 1 .2 .1 8 ) ,

A p (Zi) = A p - \ ( z i ) + K p z ~ l Bp
( 1 1 .4 .2 )

H en ce

z - pA ; _ x( i / z , )

Kp

A p ^ i (Zi )

Q U i)

( 1 1 .4 .3 )

W e n o t e th a t th e fu n c tio n Q( z) is a ll p a s s . In g e n e r a l, a n a ll-p a s s fu n c t io n o f th e
fo rm
p ( z ) = n ^

u *i< i

a i.4 .4 )

z + z*
s a tis f ie s th e p r o p e r ty th a t l ^ t e ) ) >

1 f o r [zl <

1, [/ (z )! =

1 f o r |z| =

1, a n d

|/*(z)| < 1 f o r |z| > 1. S i n c e Q ( z ) = - P { z ) f z , it fo llo w s th a t |z,| < 1 i f |f?(z)| > 1.

C le a r ly , th is is th e c a s e s in c e j2 ( z ,) = 1/ATP a n d e {, > 0 .

874

Linear Prediction and Optimum Linear Filters

> 0 and

O n th e o t h e r h a n d , s u p p o s e th a t

Chap. 11

= 0 . I n th is c a s e \ K p \ - = \

a n d \Q(Zi)\ = 1. S in c e th e M M S E is z e r o , th e r a n d o m p r o c e s s jr ( n ) is c a lle d p r e

d ic t a b le o r d e te rm in istic .

S p e c ific a lly , a p u r e ly s in u s o id a l r a n d o m p r o c e s s o f th e

fo rm

M
x ( n ) = 2 2 a ke jinoi*+et)
k=l

( 1 1 .4 .5 )

w h e r e th e p h a s e s { 8 k] a r e s ta tis tic a lly in d e p e n d e n t a n d u n ifo r m ly d is tr ib u te d o v e r

(0 , 2 n ) , h a s th e a u t o c o r r e la t io n

M
YxA m ) = 2 2 a h Jmwi

( H - 4 .6 )

*= i
a n d th e p o w e r d e n s ity s p e c tr u m

r xA f ) =

h = ?

- /*>
*= 1

(114-7)

Z7r

T h is p r o c e s s is p r e d ic t a b le w ith a p r e d ic t o r o f o r d e r p > M .
T o d e m o n s tr a t e th e v a lid ity o f th e s t a te m e n t, c o n s id e r p a s s in g th is p r o c e s s
th r o u g h a p r e d ic t io n e r r o r f ilt e r o f o r d e r p > M . T h e M S E a t th e o u tp u t o f th is
f ilte r is
M/2
/ =

r xA f ) \ A p ( f ) \ 2d f

J -\n

r 1/2 [
=

j-i/2

* (/ -/ *)

\A p{ f ) \ 2d f

( 1 1 .4 .8 )

= * 2i v / * ) i 2
k=i
B y c h o o s in g M o f t h e p z e r o s o f t h e p r e d ic t io n - e r r o r filte r t o c o in c id e w ith th e
fr e q u e n c ie s {/ * } , th e M S E / c a n b e fo r c e d to z e r o . T h e r e m a in in g p M z e r o s
c a n b e s e le c t e d a r b itr a r ily to b e a n y w h e r e in s id e th e u n it c ir c l e .
F in a lly , t h e r e a d e r c a n p r o v e t h a t i f a r a n d o m p r o c e s s c o n s is ts o f a m ix tu r e
o f a c o n tin u o u s p o w e r s p e c tr a l d e n s ity a n d a d is c r e te s p e c tr u m , th e p r e d ic t io n e r r o r filte r m u s t h a v e a ll its r o o t s in s id e th e u n it c ir c le .

Maximum-phase property of the backward prediction-error filter.

The

s y s te m fu n c t io n f o r t h e b a c k w a r d p r e d ic t io n e r r o r f ilt e r o f o r d e r p is

B p (z) = z - pA ; ( z ~ 1)

( 1 1 .4 .9 )

C o n s e q u e n tly , th e r o o t s o f B p ( z ) a r e th e r e c ip r o c a ls o f th e r o o t s o f th e fo r w a rd
p r e d ic t io n - e r r o r f ilt e r w ith s y s te m fu n c t io n A p(z ).

H e n c e i f A p (z) is m in im u m

p h a s e , th e n B p (z) is m a x im u m p h a s e . H o w e v e r , i f th e p r o c e s s x ( ) is p r e d ic t a b le ,
a ll th e r o o t s o f B p ( z ) lie o n t h e u n it c ir c le .

Sec. 11.4

Properties of the Linear Prediction-Error Filters

Whitening property.

875

S u p p o s e th a t th e ra n d o m p r o c e s s x(rt) is a n A R (p )

s ta tio n a r y r a n d o m p r o c e s s th a t is g e n e r a te d b y p a s s in g w h ite n o is e w ith v a r ia n c e

a th r o u g h a n a ll-p o le f ilt e r w ith s y s te m fu n c t io n

H( Z) = --------- y -----------

( 1 1 .4 .1 0 )

k= 1
T h e n th e p r e d i c t i o n - e r r o r f ilte r o f o r d e r p h a s th e s y s te m fu n c t io n
p

A p (z) 1 + 2 2 a p ( f t z ~ k

( 1 1 .4 .1 1 )

*= i
w h e r e th e p r e d ic t o r c o e f f ic ie n ts a p (k) = a
f ilte r is a w h ite n o is e s e q u e n c e { to (n ) }.

T h e r e s p o n s e o f th e p r e d i c t i o n - e r r o r
I n th is c a s e th e p r e d ic t io n - e r r o r f ilte r

w h ite n s t h e in p u t r a n d o m p r o c e s s x ( n ) a n d is c a lle d a w h ite n in g filte r , a s in d ic a te d

in S e c t i o n 1 1 .2 .
M o r e g e n e r a lly , e v e n i f th e in p u t p r o c e s s x ( n ) is n o t a n A R p r o c e s s , th e
p r e d i c t io n - e r r o r f ilt e r a tte m p ts t o r e m o v e th e c o r r e l a t i o n a m o n g th e s ig n a l s a m p le s
o f th e in p u t p r o c e s s .

A s th e o r d e r o f th e p r e d ic t o r is in c r e a s e d , th e p r e d ic t o r

o u tp u t x ( n ) b e c o m e s a c lo s e r a p p r o x im a tio n to x ( n ) a n d h e n c e th e d if f e r e n c e

f ( n ) = x ( n ) x ( n ) a p p r o a c h e s a w h ite n o is e s e q u e n c e .

Orthogonality of the backward prediction errors.

T h e backw ard p re

d ic t io n e r r o r s { # ( ) ) fr o m d iff e r e n t s ta g e s in t h e F I R la t t i c e f ilte r a r e o r th o g o n a l.
T h a t is,

E [ g m(n )g i (n)] =

t m

, t i

( 1 1 .4 .1 2 )

T h is p r o p e r ty is e a s ily p r o v e d b y s u b s titu tin g f o r gm(n) a n d g j( n ) in to ( 1 1 .4 .1 2 )

a n d c a r r y in g o u t th e e x p e c ta t io n . T h u s

E [g m (n )g/*(n )] = i > m ( * )

j =0

( 1 1 .4 .1 3 )

> ,* ( ;) [ * ( - k )x * (n - j ) ]

J= 0

i-0

b * a ) ] C bm ^

Yxx a ~ f t

B u t t h e n o r m a l e q u a tio n s f o r th e b a c k w a r d l in e a r p r e d ic t o r r e q u ir e th a t

> m (*)X U - *) =

k=0

f
[

m'

Jj Z 2 .......... m ~ 1

( 1 1 .4 .1 4 )

T h e re fo re ,

E [ g m( n )? g!( ;* () }n =) } ={ \J !" ; - E '

( 1 1 .4 .1 5 )
0 < I < m 1

876

Linear Prediction and Optimum Linear Filters

Additional properties.

Chap. 11

T h e r e a r e a n u m b e r o f o t h e r in t e r e s t in g p r o p e r tie s

r e g a r d in g th e fo r w a rd a n d b a c k w a r d p r e d ic t io n e r r o r s in t h e F I R

la ttic e f ilte r .

T h e s e a r e g iv e n h e r e f o r r e a l-v a lu e d d a ta . T h e i r p r o o f is l e f t a s a n e x e r c is e f o r
th e re a d e r.
( a ) E [ f m(n )x (n - /')] = 0 ,

1 < i < m

( b ) E [ g m{n )x (n - /)] = 0 ,

0 < i < m -

( c ) [ / m( n ) x ( n ) ] = [ g m(n )jr (n - m )] = E m
( d ) E [ f i( n ) f j ( n ) ] = EtnajO 'i j )
(e ) E [ f ,( n ) f j{ n - f)] = 0 , fo r

( f ) E [ g i ( n)g j (n - 0 ] = 0 ,

for

>

i < j

J ^ J :

(g ) E [ M n + i ) f j ( n + j ) } = ^

) = j

J. + ^

! *

J.

(h) E[gi(n + i ) gj ( n + ; ) ] = max {iJ)

i J ~ 0'

f f l [ / < < > < > ] - { '* '

* "

( j ) E [ f i( n ) g i { n - 1 )] = - K i + \ E i
( k ) [ g ,( r t - l ) jt ( n ) ] = E [ f j { n + 1 )jc(u -

(I) [ / , ( B > a < - D J -

j \ +l(

1 )] = - * , + ] ,

11.5 AR LATTICE AND ARMA LATTICE-LADDER FILTERS

In S e c t io n 1 1 .2 w e s h o w e d th e r e la tio n s h ip o f t h e a ll-z e r o F I R

l a t t i c e t o lin e a r

p r e d ic t io n . T h e lin e a r p r e d i c t o r w ith t r a n s f e r f u n c t io n ,
p

A P(z) = 1 + ' ^ a p ( k) z ~ k

( 1 1 .5 .1 )

*= l
w h e n e x c ite d b y a n in p u t ra n d o m p r o c e s s {jc ( n ) } , p r o d u c e s a n o u tp u t th a t a p
p r o a c h e s a w h ite n o is e s e q u e n c e a s p -> o o.

O n th e o t h e r h a n d , i f th e in p u t

p r o c e s s is a n A R ( p ) , th e o u tp u t o f A p {z) is w h ite . S in c e A p{z) g e n e r a t e s a M A ( p )

p r o c e s s w h e n e x c it e d w ith a w h ite n o is e s e q u e n c e , t h e a ll- z e r o l a t t i c e is s o m e tim e s
c a lle d a M A la t t ic e .
I n th e fo llo w in g s e c t i o n , w e d e v e lo p th e la t t i c e s tr u c tu r e f o r t h e in v e r s e filte r

1 / A p (z),
p rocess.

c a lle d th e A R

l a t t ic e , a n d th e l a t t i c e - l a d d e r s tr u c tu r e f o r a n A R M A

Sec. 11.5

AR Lattice and ARMA Lattice-Ladder Filters

877

11.5.1 AR Lattice Structure

L e t us c o n s id e r a n a ll-p o le s y s te m w ith s y s te m fu n c tio n

H ( z ) = --------- j l -------------

( 1 1 .5 .2 )

i= l

T h e d if f e r e n c e e q u a tio n f o r th is I I R s y s te m is

y(n) = - ^ P p ( ) y ( n - k ) + x ( n )

( 1 1 .5 .3 )

k= l
N o w s u p p o s e t h a t w e in te r c h a n g e th e r o le s o f t h e in p u t a n d o u tp u t [ i.e ., i n t e r
c h a n g e x (n ) w ith y ( n ) in ( 1 1 .5 .3 ) ] o b ta in in g th e d i f f e r e n c e e q u a tio n

~ft +

*(n) = k= 1
o r, e q u iv a le n tly ,

p
v ( ) = *(/?) + 2 2 , a p ( f t x (n ~ f t
*= i

( 1 1 .5 .4 )

W e o b s e r v e th a t ( 1 1 .5 .4 ) is a d if f e r e n c e e q u a tio n f o r an F I R s y s te m w ith
s y s te m fu n c t io n A p(z ).

T h u s a n a ll-p o le I I R s y s te m c a n b e c o n v e r te d to an a ll

z e r o s y s te m b y in te r c h a n g in g th e r o le s o f th e in p u t a n d o u tp u t.
B a s e d o n th is o b s e r v a t io n , w e c a n o b ta in th e s tr u c tu r e o f a n A R { p ) la ttic e
f r o m a M A ( p ) la t t ic e b y in te r c h a n g in g t h e in p u t w ith th e o u tp u t. S in c e th e M A ( p )
l a t t i c e h a s v (n ) = f P (n) as its o u tp u t a n d x ( n ) = / o(n ) is t h e in p u t, w e le t

x { n ) = f p {n)
( 1 1 .5 .5 )

y (n ) =

fo (n )

T h e s e d e fin itio n s d ic t a te t h a t th e q u a n t it ie s { f m{n )} b e c o m p u te d in d e s c e n d in g o r

d e r. T h is c o m p u ta tio n c a n b e a c c o m p lis h e d b y r e a r r a n g in g th e r e c u r s iv e e q u a tio n
f o r { / * ( ) } in ( 1 1 .2 .4 ) a n d s o lv in g f o r /m_ j( n ) in t e r m s o f f m{n ). T h u s w e o b ta in

f m~ \( n ) -

f m{n) - K mg m- } (n - 1 )

m = p , p - 1 .......... 1

T h e e q u a tio n f o r g m(n) r e m a in s u n c h a n g e d . T h e r e s u lt o f th e s e c h a n g e s is th e s e t
o f e q u a tio n s
*(n ) =

f p (n)

f m- i ( n ) = f m(n) - K mg m- i ( n - 1)
(1 1 .5 .6 )

gm(n) =

y(n) = M n ) = gain)
T h e c o r r e s p o n d in g s tr u c tu r e f o r t h e A R ( p ) l a t t i c e is s h o w n in F ig . 1 1 .6 . N o te th a t
th e a ll-p o le la t t ic e s tr u c tu r e h a s a n a ll-z e r o p a th w ith in p u t g o ( ) a n d o u tp u t g p(n ),

878

Linear Prediction and Optimum Linear Filters

Chap. 11

Figure 11.6 Lattice structure for an all-pole system

w h ic h is id e n tic a l to th e a ll-z e r o p a th in th e M a ( p ) la t t ic e s tr u c tu r e .

T h is is n o t

s u r p ris in g , s in c e th e e q u a tio n f o r g m (n ) is id e n tic a l in th e tw o l a t t i c e s tr u c tu r e s .

W e a ls o o b s e r v e th a t th e A R ( p ) a n d M A (/ ?) la t t ic e s t r u c tu r e s a r e c h a r a c
te r iz e d b y th e s a m e p a r a m e te r s , n a m e ly , th e r e fle c tio n c o e f f i c i e n t s {/if,). C o n s e
q u e n tly , th e e q u a tio n s g iv e n in ( 1 1 .2 .2 1 ) a n d ( 1 1 .2 .2 3 ) f o r c o n v e r t in g b e tw e e n th e
s y s te m p a r a m e te r s {o^ O t)} in th e d ir e c t -fo r m r e a liz a tio n s o f th e a ll- z e r o s y ste m
A p(z ) a n d th e l a t t ic e p a r a m e te r s { ; ] o f th e M A ( p ) la t t ic e s t r u c t u r e , a p p ly a s w e ll
to th e a ll- p o le s tr u c tu r e s .

11.5.2 ARMA Processes and Lattice-Ladder Filters

T h e a ll-p o le l a t t ic e p r o v id e s th e b a s ic b u ild in g b lo c k f o r la t t ic e - t y p e s tr u c tu r e s
th a t im p le m e n t I I R s y s te m s th a t c o n ta in b o th p o le s a n d z e r o s . T o c o n s t r u c t th e
a p p r o p r ia te s tr u c tu r e , le t u s c o n s id e r a n I I R s y s te m w ith s y s te m fu n c t io n

c 9(*)z _*

H ( z ) = ------------------- =
,

v -

( 1 1 .5 .7 )

A p {z)

*= i
W ith o u t lo s s o f g e n e r a lity , w e a s s u m e th a t p > q.
T h is s y s te m is d e s c r ib e d b y th e d if f e r e n c e e q u a tio n s

v (n ) = - ' ^ T a p (k )v (n - k) + x (n )
k= 1

<7
y (n ) =

( 1 1 .5 .8 )

J 2 c ^ ( f t v (n

o b ta in e d b y v ie w in g th e s y s te m a s a c a s c a d e o f a n a ll-p o le s y s t e m f o llo w e d b y a n
a ll- z e r o s y s te m . F r o m ( 1 1 .5 .8 ) w e o b s e r v e th a t th e o u tp u t y ( n ) is s im p ly a lin e a r
c o m b in a tio n o f d e la y e d o u tp u ts fr o m th e a ll- p o le s y s te m .
S in c e z e r o s r e s u lt fr o m fo r m in g a lin e a r c o m b in a tio n o f p r e v io u s o u tp u ts , w e
c a n c a r r y o v e r th is o b s e r v a t io n t o c o n s t r u c t a p o l e - z e r o s y s t e m u s in g t h e a ll- p o le

Sec. 11.5

AR Lattice and ARMA Lattice-Ladder Fitters

879

l a t t ic e s t r u c tu r e a s th e b a s ic b u ild in g b lo c k . W e h a v e c le a r ly o b s e r v e d th a t gm(n)
in th e a ll-p o le l a t t ic e c a n b e e x p r e s s e d a s a lin e a r c o m b in a tio n o f p r e s e n t a n d p a s t
o u tp u ts . In f a c t , th e s y s te m
( 1 1 .5 .9 )

is a n a ll-z e r o s y s te m .

T h e r e f o r e , a n y lin e a r c o m b in a tio n o f { m(n ) } is a ls o a n

a ll- z e r o f ilte r .
L e t u s b e g in w ith a n a ll- p o le l a t t ic e filte r w ith c o e f f ic i e n t s K m, 1 < m < p ,
a n d a d d a la d d e r p a r t by ta k in g a s th e o u tp u t, a w e ig h te d l i n e a r c o m b in a tio n o f
{ m (n )}. T h e r e s u lt is a p o l e - z e r o filte r t h a t h a s th e la t t ic e -la d d e r s t r u c tu r e s h o w n
in F ig . 1 1 .7 . I t s o u tp u t is

v (n ) = 2 2 p k g k ( n )

( 1 1 .5 .1 0 )

w h e r e { / y a r e th e p a r a m e te r s th a t d e te r m in e th e z e r o s o f th e s y s te m . T h e s y s te m
fu n c tio n c o r r e s p o n d in g t o ( 1 1 .5 .1 0 ) is

( 1 1 .5 .1 1 )

Inpul
x(n)

fp -iW

fp-M )

/>()

f iW

/o(")

Stage

Stage

8P- ()_

Stage
gP- 2< )

P~ 1

o(n)

|( )

'p
O utput

(a) P o le -z e ro system

f m - 1<")

.-I
(b)

m1*1stage of lattice

Figure 11.7 Lattice-ladder structure for pole-zero system

880

Linear Prediction and Optimum Linear Filters

Chap. 11

S in c e X ( z ) = F p (z) a n d F q ( z ) = G o (z ), ( 1 1 .5 .1 1 ) , c a n b e e x p r e s s e d a s

G t i z ) F o (z)

-2 P*

k= 0

G0(z) Fp{z)
( 1 1 .5 .1 2 )

a p (z) *=o

T h e re fo re ,
<?
C 9 (z) = f t * ( z )
t=o

( 1 1 .5 .1 3 )

T h is is th e d e s ire d r e la tio n s h ip th a t c a n b e u se d t o d e te r m in e th e w e ig h tin g c o e f

f ic ie n ts {fik} as p r e v io u s ly s h o w n in S e c t i o n 7 .3 .5 .
G iv e n th e p o ly n o m ia ls C q ( z ) a n d A p (z), w h e r e p > q , t h e r e f le c tio n c o e f
f ic ie n ts { , } a r e d e te r m in e d firs t fr o m th e c o e f f ic ie n t s { ^ ( / t ) } .

B y m e a n s o f th e

s te p -d o w n r e c u r s iv e r e la tio n g iv e n b y ( 1 1 .2 .2 2 ) , w e a ls o o b t a i n th e p o ly n o m ia ls

B k ( z ) , k = 1 , 2 , . . . , p . T h e n th e la d d e r p a r a m e t e r s c a n b e o b t a i n e d f r o m ( 1 1 .5 .1 3 ) ,
w h ic h c a n b e e x p r e s s e d as
m -1

C , (z ) = ^

to

P k B k iz ) + fimB m(z)
( 1 1 .5 .1 4 )

= C m- i ( z ) + fimB m(z)
o r , e q u iv a le n tly ,

C m- i ( z ) = C m(z) - P mB m{z)

m = p, p - 1 , . . . , 1

( 1 1 .5 .1 5 )

B y ru n n in g th is r e c u r s iv e r e la tio n b a c k w a r d , w e c a n g e n e r a t e a ll th e lo w e r - d e g r e e
p o ly n o m ia ls , C m( z ), m =

p 1 .......... 1. S in c e bm(m ) = 1, t h e p a r a m e t e r s f5m a r e

d e te r m in e d fr o m ( 1 1 .5 .1 5 ) b y s e tt in g

p m = c m(m )

m = p , p - 1 ------, 1 , 0

W h e n e x c ite d b y a w h ite n o is e s e q u e n c e , th is la t t i c e - l a d d e r f i l t e r s tr u c tu r e
g e n e r a t e s a n A R M A ( p , ^ ) p r o c e s s th a t h a s a p o w e r d e n s ity s p e c tr u m

\C q( f ) \
IV / ) l
a n d a n a u t o c o r r e la tio n fu n c t io n th a t s a tis fie s ( 1 1 .1 .1 8 ) , w h e r e a 2 in th e v a ria n c e
o f t h e in p u t w h ite n o is e s e q u e n c e .

11.6 WIENER FILTERS FOR FILTERING AND PREDICTION

I n m a n y p r a c tic a l a p p lic a t io n s w e a r e g iv e n a n in p u t s ig n a l { * ( ) } , c o n s is tin g o f
th e s u m o f a d e s ir e d s ig n a l {.s(/i)} a n d a n u n d e s ir e d n o is e o r i n t e r f e r e n c e {u>(n)},
a n d w e a r e a s k e d t o d e s ig n a f itte r th a t s u p p r e s s e s th e u n d e s ir e d i n te r f e r e n c e

Sec. 11.6

Wiener Filters for Filtering and Prediction

881

d (n )

----------- t { + )-------<()

Figure l l j t
problem
c o m p o n e n t.

Model for linear estimation

In s u c h a c a s e , th e o b je c t i v e is t o d e s ig n a s y s te m t h a t f ilte r s o u t

th e a d d itiv e in t e r f e r e n c e w h ile p r e s e r v in g th e c h a r a c t e r is t ic s o f th e d e s ir e d s ig n a l
{* ( )}
In th is s e c t io n w e t r e a t th e p r o b le m o f s ig n a l e s tim a tio n in th e p r e s e n c e o f
a n a d d itiv e n o is e d is tu r b a n c e .

T h e e s tim a to r is c o n s t r a in e d to b e a li n e a r f ilte r

w ith im p u ls e r e s p o n s e {/ i(n )}, d e s ig n e d s o th a t its o u tp u t a p p r o x im a te s s o m e s p e c

ifie d d e s ire d s ig n a l s e q u e n c e \d ( n ) } .

F ig u r e 1 1 .8 illu s tr a te s th e lin e a r e s tim a tio n

p r o b le m .
T h e in p u t s e q u e n c e t o th e f ilte r is x ( n ) = s ( n ) + w (n ), a n d its o u tp u t s e q u e n c e
is v (n ). T h e d iffe r e n c e b e tw e e n t h e d e s ir e d s ig n a l a n d th e filte r o u tp u t is th e e r r o r
s e q u e n c e e(n ) = d (n ) y (n ) .
W e d is tin g u is h t h r e e s p e c ia l c a s e s :
1 . I f d {n ) = s ( n ) , th e lin e a r e s tim a tio n p r o b le m is r e f e r r e d to as, filte rin g .
2 . I f d (n ) = s {n + D ), w h e r e D > 0 , th e li n e a r e s tim a tio n p r o b le m is r e f e r r e d to
a s s ig n a l p re d ic t io n . N o te t h a t th is p r o b le m is d if f e r e n t th a n th e p r e d ic tio n
c o n s id e r e d e a r l i e r in th is c h a p te r , w h e r e d {n ) = x i n + D ) , D > 0 .
3 . I f d {n ) s ( n D ) , w h e r e D > 0 , th e li n e a r e s tim a tio n p r o b le m is r e f e r r e d
t o a s s ig n a l s m o o th in g .
O u r tr e a t m e n t w ill c o n c e n t r a t e o n filte r in g a n d p r e d ic t io n .
T h e c r ite r io n s e le c te d f o r o p tim iz in g t h e - f i l t e r im p u ls e r e s p o n s e {/t(n)} is
th e m in im iz a t io n o f th e m e a n -s q u a r e e r r o r . T h is c r it e r i o n h a s th e a d v a n ta g e s o f
s im p lic ity a n d m a th e m a tic a l tr a c ta b ility .
T h e b a s ic a s s u m p tio n s a r e t h a t th e s e q u e n c e s { $ ( ) } , {

a n d {/(n)} a r e

z e r o m e a n a n d w id e -s e n s e s ta tio n a r y . T h e lin e a r f ilte r w ill b e a s s u m e d to b e e it h e r

F I R o r I I R . I f it is I I R , w e a s s u m e th a t th e in p u t d a ta (jc (n )} a r e a v a ila b le o v e r
th e in f in it e p a s t. W e b e g in w ith t h e d e s ig n o f t h e o p tim u m F I R filte r . T h e o p t i
m u m lin e a r f ilte r , in t h e s e n s e o f m in im u m m e a n - s q u a r e e r r o r ( M M S E ) , is c a lle d
a W ie n e r filte r.

11.6.1 FIR Wiener Filter

S u p p o s e th a t t h e filte r is c o n s t r a in e d t o b e o f le n g th M w ith c o e f f ic ie n ts {A*, 0 <

k <

M - l ) .

H e n c e its o u tp u t y ( n ) d e p e n d s o n th e fin ite d a ta r e c o r d x ( n ) ,

882

Linear Prediction and Optimum Linear Fitters

Chap. 11

x (n 1 ) , , x (n M + 1 ),
Af -1

h ( k )x (n - k)

y (n ) =

( 1 1 .6 .1 )

t=o
T h e m e a n -s q u a r e v a lu e o f th e e r r o r b e tw e e n th e d e s ir e d o u tp u t d ( n ) a n d y in ) is

Z M = E \e { n ) \2
^
=

d (n ) -

( 1 1 .6 .2 )

h ( k ) x ( n k)
k =o

S in c e th is is a q u a d r a tic f u n c t io n o f th e filte r c o e f f ic ie n t s , th e m in im iz a t io n o f u
y ie ld s th e s e t o f lin e a r e q u a tio n s
M -l
5 > ( * W
*=o

/ = 0 , 1 .......... M

/ - ) = ^ ,(/ )

- l

( 1 1 .6 .3 )

w h e r e yxx(k) is th e a u t o c o r r e l a t io n o f th e in p u t s e q u e n c e { * ( ) } a n d ydx(k) =

E [ d ( n ) x * ( n fc)] is th e c r o s s c o r r e l a t i o n b e tw e e n th e d e s ir e d s e q u e n c e [d (n )} a n d
th e in p u t s e q u e n c e { x {n ), 0 < n < M 1 ). T h e s e t o f lin e a r e q u a t i o n s th a t s p e c ify
th e o p tim u m filte r is c a lle d th e W ie n e r - H o p f e q u a tio n . T h e s e e q u a tio n s a r e a ls o
c a lle d th e n o r m a l e q u a tio n s , e n c o u n te r e d e a r l ie r in th e c h a p t e r in th e c o n t e x t o f
lin e a r o n e - s te p p r e d ic t io n .
In g e n e r a l, th e e q u a tio n s in ( 1 1 .6 .3 ) c a n b e e x p r e s s e d in m a tr ix f o r m a s
r wh = y d
w h ere r

( 1 1 .6 .4 )

is a n M x M ( H e r m i t i a n ) T o e p lit z m a tr ix w ith e l e m e n t s T/* = Y * A l ~ k )

a n d y d is th e M x 1 c r o s s c o r r e l a t io n v e c t o r w ith e l e m e n t s Yd x(0 , i = 0 , 1 , . . . , M - \ .
T h e s o lu tio n f o r th e o p tim u m f ilte r c o e f f ic ie n ts is
hopt = Y M y d

( 1 1 .6 .5 )

a n d th e r e s u ltin g m in im u m M S E a c h ie v e d b y th e W i e n e r f ilte r is

M -l
M M S E M = m in M = a 2d h

^opt( * ) ? ( )

( 1 1 .6 .6 )

o r , e q u iv a le n tly ,
M M S E M = a 2 - y J T ^ y d

( 1 1 .6 .7 )

w h ere a j = E \d ( n ) \2 .
L e t us c o n s id e r s o m e s p e c ia l c a s e s o f ( 1 1 .6 .3 ) . I f w e a r e d e a lin g w ith filte rin g ,
t h e d (n ) = s ( n ) . F u r t h e r m o r e , i f s ( n ) a n d w (n ) a r e u n c o r r e la te d r a n d o m s e q u e n c e s ,
a s is u s u a lly th e c a s e in p r a c t ic e , th e n

Yxx(k) =

y ss(k) + Yww(k)

Ydx(k)

Yss(k)

(11.6.8)

Sec. 11.6

Wiener Fitters for Filtering and Prediction

883

a n d th e n o r m a l e q u a tio n s in ( 1 1 .6 .3 ) b e c o m e
M -l

Y , W

) M

- V

+ Y w w V - k ) ] = y ss(l)

1 = 0 , 1 .......... M

-l

( 1 1 .6 .9 )

i= 0

I f w e a r e d e a lin g w ith p r e d ic t io n , th e n d (n ) = s (n + D ) w h e r e D

> 0. A s

s u m in g th a t s ( n ) a n d w (n ) a r e u n c o r r e la te d ra n d o m s e q u e n c e s , w e h a v e

Y d ,(k ) = y ss(l + D )

( 1 1 .6 .1 0 )

H e n c e th e e q u a tio n s f o r th e W ie n e r p r e d ic t io n f ilt e r b e c o m e

M-l
* ( * ) [ y ( / - * ) + yv.tt. ( / - * ) ] = y</ + z>)
i= 0

/= 0,1...., w - l

( 1 1 .6 .1 1 )

In all th e s e c a s e s , th e c o r r e la t io n m a tr ix t o b e in v e r t e d is T o e p lit z . H e n c e th e
( g e n e r a liz e d ) L e v in s o n - D u r b in a lg o r ith m m a y b e u s e d to s o lv e f o r th e o p tim u m
f ilte r c o e ffic ie n ts .

Example 11.6.1
Lei us consider a signal x( n) = j(n) + w(n), where s(n) is an AR(1) process that
satisfies the difference equation
s(n ) = 0.6.v(n 1) + u(n)

where lu(n)l is a white noise sequence with variance <tl2 = 0.64, and (u>(n)) is a white
noise sequence with variance en = 1. We will design a Wiener filter of length M = 2
to estimate (j(h)}.
Solution Since (j(n)) is obtained by exciting a single-pole filter by white noise, the
power spectra! density of
is
r ( / ) = < r ;\H { f ) \2

0.64
|1 0.6e~-i*nf \ 2'

0.64
1.36 1.2cos2jt/
The corresponding autocorrelation sequence (y(m)} is
Y,Am ) = (0.6)|m|

The equations for the filter coefficients are

2h(0) + 0.6A (1) = 1
0 .6 A (0 )+ 2 A (1 ) = 0 .6

Solution of these equations yields the result

h(0) = 0.451

h(l) = 0.165

884

Linear Prediction and Optimum Linear Filters

Chap. 11

The corresponding minimum MSE is

MMSE2 = 1 - A(0)y(0) - /i(l)>v,(l)
= 1 - 0.451 - (0.165X0.6)
= 0.45
This error can be reduced further by increasing the length of the Wiener filter (see
Problem 11.27).

11.6.2 Orthogonality Principle in Linear Mean-Square

Estimation
T h e n o r m a l e q u a tio n s f o r th e o p tim u m filte r c o e f f ic ie n t s g iv e n b y ( 1 1 .6 .3 ) c a n b e
o b ta in e d d ir e c tly b y a p p ly in g th e o r th o g o n a lity p r in c ip le in lin e a r m e a n - s q u a r e
e s tim a tio n . S im p ly s ta te d , th e m e a n - s q u a r e e r r o r m in ( 1 1 .6 .2 ) is a m in im u m if
th e f ilte r c o e f f ic ie n ts {/ ;()) a r e s e le c t e d s u ch th a t th e e r r o r is o r th o g o n a l t o e a c h
o f th e d a ta p o in ts in th e e s tim a te ,

E \e ( n ) x * ( n /)] = 0

/ = 0 , 1 .......... M 1

( 11.6 .12)

h ( k ) x ( n k)

( 1 1 .6 .1 3 )

w h ere
M -l

e (n ) = d (n )
i =o

C o n v e r s e ly , i f th e f ilte r c o e f f ic ie n t s s a tis fy ( 1 1 .6 .1 2 ) , th e r e s u ltin g M S E is a m in i

m um .
W h e n v ie w e d g e o m e tr ic a lly , th e o u tp u t o f t h e filte r , w h ic h is th e e s tim a te
M -l

d (n )

h (k )x (n k)

(1 1 .6 .1 4 )

*=o
is a v e c to r in th e s u b s p a c e s p a n n e d b y th e d a ta { * ( & ) , 0 < k < M 1 }. T h e e r r o r

e (n ) is a v e c to r f r o m d (n ) to <?(n) [ i.e ., d (n ) = e ( n ) + d (n )], a s s h o w n in F ig . 1 1 .9 .

T h e o r th o g o n a lity p r in c ip le s ta te s th a t th e le n g th M =

E M / i) !2 is a m in im u m

w h e n e (n ) is p e r p e n d ic u la r t o th e d a ta s u b s p a c e [ i.e ., e(rt) is o r th o g o n a l to e a c h
d a ta p o in t x ( k ) , 0 < k < M 1 ],
W e n o te t h a t th e s o lu tio n o b t a in e d fr o m th e n o r m a l e q u a t i o n s in ( 1 1 .6 .3 )
is u n iq u e i f th e d a ta { * ( ) } in th e e s t im a t e d (n ) a r e lin e a r ly in d e p e n d e n t. In th is
c a s e , th e c o r r e la t io n m a tr ix

is n o n s in g u la r . O n th e o t h e r h a n d , i f th e d a ta a r e

lin e a r ly d e p e n d e n t, th e r a n k o f T M is le s s th a n M a n d t h e r e f o r e th e s o lu tio n is n o t
u n iq u e . In th is c a s e , th e e s t im a t e d ( n ) c a n b e e x p r e s s e d a s a lin e a r c o m b in a tio n
o f a r e d u c e d s e t o f lin e a r ly in d e p e n d e n t d a ta p o in t s e q u a l to th e r a n k o f T w .
S in c e th e M S E is m in im iz e d b y s e le c tin g t h e f ilte r c o e f f i c i e n t s to s a tis fy th e
o r th o g o n a lity p r in c ip le , th e r e s id u a l m in im u m M S E is s im p ly

MMSE*f = E[e(n)d*()]

which yields the result given in (11.6.6).

(11.6.15)

Sec. 11.6

885

Wiener Filters for Filtering and Prediction

Figure 11.9

Geometric interpretation

of linear M S E problem

11.6.3 IIR Wiener Filter

In th e p r e c e d in g s e c tio n w e c o n s t r a in e d th e f ilte r t o b e F I R a n d o b ta in e d a s e t o f

M lin e a r e q u a tio n s f o r th e o p tim u m filte r c o e f f ic ie n ts . In th is s e c tio n w e a llo w th e

f ilte r to b e in fin ite in d u ra tio n ( I I R ) a n d th e d a ta s e q u e n c e to b e in fin ite a s w e ll.
H e n c e th e f ilte r o u tp u t is

( 11 .6 . 16 )

y ( ) = / 2 ( ) * ( n - k)

T h e f ilt e r c o e f f ic ie n ts a r e s e le c te d to m in im iz e th e m e a n -s q u a r e e r r o r b e tw e e n th e
d e s ir e d o u tp u t d (n ) a n d v ( n ). th a t is,

x =

E \ e { n ) \2

( 1 1 .6 .1 7 )

A p p lic a tio n o f th e o r th o g o n a lity p r in c ip le le a d s to th e W i e n e r - H o p f e q u a

tion.
( 1 1 .6 .1 8 )
T h e r e s id u a l M M S E is s im p ly o b t a in e d b y a p p lic a t io n o f th e c o n d itio n g iv e n b y
( 1 1 .6 .1 5 ) . T h u s w e o b ta in
M M S E s c = m in ^

= aj -

f c opt ( * ) > ( )

( 1 1 .6 .1 9 )

T h e W i e n e r - H o p f e q u a tio n g iv e n b y ( 1 1 .6 .1 8 ) c a n n o t b e s o lv e d d ir e c tly w ith

z - tr a n s f o r m te c h n iq u e s , b e c a u s e th e e q u a tio n h o ld s o n ly f o r 1 > 0 . W e s h a ll s o lv e

886

Linear Prediction and Optimum Linear Filters

Chap. 11

f o r th e o p tim u m I I R W ie n e r filte r b a s e d o n th e in n o v a tio n s r e p r e s e n t a t io n o f th e

s ta tio n a r y r a n d o m p r o c e s s { * ( ) } .
R e c a l l th a t a s ta tio n a r y r a n d o m p r o c e s s U ( n ) ) w ith a u t o c o r r e l a t i o n yxx(k)
a n d p o w e r s p e c tr a l d e n s ity r ( / ) c a n b e r e p r e s e n te d b y a n e q u iv a le n t in n o v a
tio n s p r o c e s s , ( (n )} b y p a s s in g { * ( ) } th r o u g h a n o is e -w h ite n in g f ilt e r w ith s y ste m
fu n c t io n 1 / G ( z ) , w h e r e G ( z ) is th e m in im u m -p h a s e p a r t o b t a in e d f r o m th e s p e c tr a l
fa c to r iz a tio n o f r xJt(z ):
r ( z ) = <T,2 G ( z ) G ( z - 1)

( 1 1 .6 .2 0 )

H e n c e G i z ) is a n a ly tic in t h e r e g io n jz| > n , w h e r e n < \ .

N o w , th e o p tim u m W i e n e r f i lt e r c a n b e v ie w e d a s th e c a s c a d e o f th e w h ite n
in g filte r 1 / G (z ) w ith a s e c o n d f ilte r , s a y Q ( z ) , w h o s e o u tp u t v ( ) is id e n tic a l to
th e o u tp u t o f t h e o p tim u m W i e n e r filte r . S in c e
OC

<?(*)/( - k)

j( n ) =

( 1 1 ,6 .2 1 )

*=o
a n d e(rt) = d { n ) > ( n ), a p p lic a t io n o f th e o r th o g o n a lity p r in c ip le y ie ld s th e n e w
W i e n e r - H o p f e q u a tio n as
OC

22<}(k'>y"V - k ) = ydi(D

/>

( 11.6. 22)

B u t s in c e { i( n ) ) is w h ite , it fo llo w s th a t y(/ k ) = 0 u n le s s / = k. T h u s w e o b ta in

th e s o lu tio n as

Yd, (I)

Ydi(0

q d ) = ZZLLL =

/> 0

( 1 1 .6 .2 3 )

Ya(0)

T h e z - tr a n s f o r m o f t h e s e q u e n c e l g ( i ) } is

Q iz ) = Y l q ( k ) z ~ k
k0
=

Y lr d i( lc ) z ~ k

a <

t=o

( 1 1 .6 .2 4 )

I f w e d e n o te th e z -tr a n s fo r m o f th e tw o -s id e d c r o s s c o r r e la tio n s e q u e n c e Y d iik ) by

r\ ,,(z ):
OC

r di( z ) =

Y i

W *( * ) * *

( 1 1 .6 .2 5 )

* = -0 0
a n d d e fin e [r\/,-(z)]+ a s
[ I \ ,,( z ) ] + = 2 2 Y * W z ~ k
*=o

( 1 1 .6 .2 6 )

Q iz ) = ~ [ r d iiz ) ) +
of

( 1 1 .6 .2 7 )

then

Sec. 11.6

Wiener Filters for Filtering and Prediction

887

T o d e te r m in e [r</;(z)]+ , w e b e g in w ith th e o u tp u t o f th e n o is e - w h it e n in g f ilte r ,

w h ic h c a n b e e x p r e s s e d a s
OC

i{ n ) = ^

v ( );c (n k)

( 1 1 .6 .2 8 )

*=o
w h e r e (u (Jt), k > 0 } is t h e im p u ls e r e s p o n s e o f th e n o is e -w h it e n in g filte r ,
( 1 1 .6 .2 9 )
Then

Yd i{k) = E [ r f ( n ) i * ( n - * ) ]
OC

2 2 v ( m ) E [ d { n ) x * ( n m Jt)]

22 v (m ')ydAk

( 1 1 .6 .3 0 )

+m )

m=0
T h e z - tr a n s fo r m o f th e c r o s s c o r r e la tio n Y d iik ) is
oc

r ji( z ) =

2 2

oc

' Y 2 v ^ Y d x { k + m)

k=oo m=0
=

u ( m ) 2 2 Y d x (k + m )z~
m=0
*= -o c
oc

( 1 1 .6 .3 1 )

oc

u (m )z m 2 2

V iz -'W iA z ) =

Y * * (k )z~ k
r dx(z)
G i z - 1)

T h e re fo re ,

Q iz ) = A

r^ (z ) I

( 1 1 .6 .3 2 )

F in a lly , th e o p tim u m I I R W ie n e r f ilt e r h a s th e s y s te m f u n c t io n

H op\ iz ) =

Q iz )
G iz )
( 1 1 .6 .3 3 )

In s u m m a r y , th e s o lu tio n f o r th e o p tim u m I I R W i e n e r f ilte r r e q u ir e s t h a t

w e p e r fo r m a s p e c t r a l fa c to r iz a tio n o f

(z ) t o o b ta in G i z ) , th e m in im u m -p h a s e

c o m p o n e n t , a n d th e n w e s o lv e f o r th e c a u s a l p a r t o f T dxi z ) / G i z ~ 1 ). T h e f o llo w in g
e x a m p le illu s tr a te s th e p r o c e d u r e .

Linear Prediction and Optimum Linear Filters

888

Chap. 11

Example 11.6.2
L e t us d eterm in e the optim um I I R W ien e r filter fo r th e signal given in E x a m p le 11.6.1.

Solution

F o r this signal we have

1.8(1 - k - ' X l - h )
+ i = - _ ^ . T . ( 1 - (|^ .

r ^ ) - r w
w here a f = 1.8 and

1 - |z-1
ca) = rro fe ^
T h e z-transform o f th e crossco rrelation ydx (m ) is
0 .6 4

r dJ(z) = r ( z )
H en ce

(1 - 0 . 6 z " > ) ( l - 0 . 6 z )

rr^a)! =r__

Lg(z-)J+ L a - 1

0 .64
z X l - 0 . 6 ; - 1)

0.8

[ 1 - 0.6z~'

0 .2 6 6 ;
i _ i zj

0.8
1 - 0 .6 z - ]
T h e optim um I I R filter has th e system function
1

/ l-0.6r'\

H-'w = r8(TTTFrj(rro^j

0. 8

and an im pulse response

* o p t( n ) =

5( j ) "

n >

W e c o n c lu d e th is s e c t io n b y e x p r e s s in g th e m in im u m M S E g iv e n b y ( 1 1 .6 .1 9 )
in te r m s o f th e f r e q u e n c y -d o m a in c h a r a c te r is tic s o f th e filte r . F i r s t , w e n o t e th a t

a j = E\ d{ n) \ 2 is s im p ly th e v a lu e o f th e a u t o c o r r e l a t i o n s e q u e n c e \ydd(k)} e v a lu
a te d a t it = 0. S in c e

Vdd(k) = ^ y ^ r ^ ( z ) z * ^ z

( 1 1 .6 .3 4 )

it fo llo w s th a t

a ] = ydd(0 ) =

( 1 1 .6 .3 5 )

w h e r e t h e c o n t o u r in te g r a l is e v a lu a te d a lo n g a c lo s e d p a th e n c i r c l i n g th e o rig in
in th e r e g io n o f c o n v e r g e n c e o f

Sec. 11.6

Wiener Filters for Filtering and Prediction

889

T h e s e c o n d te r m in ( 1 1 .6 .1 9 ) is a ls o e a s ily tr a n s f o r m e d t o t h e f r e q u e n c y
d o m a in b y a p p lic a t io n o f P a r s e v a ls t h e o r e m . S in c e h opl( k ) = 0 f o r k < 0 , w e h a v e

^ S fio p d z W .A z - ^ d z

fto p ,( * ) } ( * ) =
cc

( 1 1 .6 .3 6 )

J *

w h e r e C is a c lo s e d c o n to u r e n c ir c lin g th e o r ig in t h a t lie s w ith in t h e c o m m o n

r dx(z~l).

r e g io n o f c o n v e r g e n c e o f / / optU ) a n d

B y c o m b in in g ( 1 1 .6 .3 5 ) w ith ( 1 1 .6 .3 6 ) , w e o b ta in th e d e s ir e d e x p r e s s io n f o r
th e M M S E c c in th e fo r m
M M S E ^ = ~ < ^ [ r dd(z) -

H opd z W dA z - l ) ] z - l d z

( 1 1 .6 .3 7 )

E x a m p le 1 1 .6 3

For the optimum Wiener filter derived in Example 11.6.2, the minimum MSE is
m m sem =

i fi r

2 x iJ f

o 35ss

-J)(l-0.fe)J

There is a single pole inside the unit circle at z = j. By evaluating the residue at the
pole, we obtain
MMSE^ - 0.444
We observe that this MMSE is only slightly smaller than that for the optimum two-tap
Wiener filter in Example ] 1.6.1.

11.6.4 Noncausal Wiener Filter

I n th e p r e c e d in g s e c tio n w e c o n s t r a in e d th e o p tim u m W i e n e r f ilte r t o b e c a u s a l
[ i.e ., h opt(n) = 0 f o r n < 0 ]. In th is s e c t io n w e d r o p th is c o n d itio n a n d a llo w th e
f ilte r t o in c lu d e b o th th e in fin ite p a s t a n d th e in fin ite f u tu r e o f th e s e q u e n c e { * ( ) }
in f o r m in g th e o u tp u t y ( n ) , th a t is,
OO
y( n) =

2 2

h ( k ) x (n k)

( 1 1 .6 .3 8 )

oc

T h e re s u ltin g filte r is p h y s ic a lly u n r e a liz a b le . I t c a n a ls o b e v ie w e d a s a s m o o th in g

filt e r in w h ic h th e in fin ite fu tu r e s ig n a l v a lu e s a r e u s e d t o s m o o th t h e e s tim a te

d (n ) = y ( n ) o f th e d e s ir e d s ig n a l d { n ) .
A p p lic a tio n o f th e o r th o g o n a lity p r in c ip le y ie ld s th e W i e n e r - H o p f e q u a tio n
f o r t h e n o n c a u s a l filte r in th e fo r m
OO

2 2
k=oc

- k ) = y dx( l )

- oo < I <

00

( 1 1 .6 .3 9 )

a n d t h e r e s u ltin g M M S E nf as
OO
M M S E nc =

22
Jt=-00

h (* ) r lc ^

d 1 -6 -4 )

890

Linear Prediction and Optimum Linear Filters

Chap. 11

S in c e ( 1 1 .6 .3 9 ) h o ld s f o r oo < I < oo, th is e q u a tio n c a n b e tr a n s f o r m e d

d ir e c tly to y ie ld th e o p tim u m n o n c a u s a l W i e n e r filte r as
^ (z ) = p ^ ~

I jrjrv)

( 1 1 .6 .4 1 )

T h e M M S E ( c a n a ls o b e s im p ly e x p r e s s e d in t h e z -d o m a in a s
M M S E nf = J - ^ [ r dd(z) -

H nc( z ) r dA t - l ) ] z - l d z

( 1 1 .6 .4 2 )

In th e fo llo w in g e x a m p le w e c o m p a r e th e fo r m o f th e o p tim a l n o n c a u s a l filte r

w ith th e o p tim a l c a u s a l f ilt e r o b ta in e d in th e p r e v io u s s e c tio n .

Example 11.6.4
The optimum noncausal Wiener filter for the signal characteristics given in Exam
ple 11.6.1 is given by (11.6.41), where
0.64

r^ a> F-"(

(1 - 0 .6 ; - )(l -0.6c)

and
r ( s ) = r (;) + l
2(1 - 0 .3 --1 - 0.3;)
(1 - 0 . 6 - - ' )(1 - 0.6c)

Then.
0.3555
H ", U ~ ( l - i c - ' K l - i c )

This filter is clearly noncausal.

The minimum MSE achieved by this filter is determined from evaluating
(11.6.42). The integrand is

The only pole inside the unit circle is c = |. Hence the residue is
0.35551

0.3555

= 0.40

Hence the minimum achievable MSE obtained with the optimum noncausal Wiener
filter is
MMSEnr = 0.40
Note that this is lower than the MMSE for the causal filler, as expected.

11.7 SUMMARY AND REFERENCES

T h e m a jo r f o c a l p o in t in th is c h a p t e r is th e d e s ig n o f o p tim u m l i n e a r s y s te m s fo r
lin e a r p r e d ic t io n a n d f ilte r in g .

T h e c r i t e r i o n f o r o p tim a lity is t h e m in im iz a tio n

o f th e m e a n -s q u a r e e r r o r b e t w e e n a s p e c ifie d d e s ir e d f ilt e r o u tp u t a n d t h e a c tu a l
f ilte r o u tp u t.

Sec. 11.7

Summary and References

891

I n th e d e v e lo p m e n t o f lin e a r p r e d ic t io n , w e d e m o n s tr a t e d th a t th e e q u a tio n s
f o r th e fo r w a r d a n d b a c k w a r d p r e d ic t io n e r r o r s s p e c ifie d a la ttic e f ilte r w h o s e
p a r a m e te r s , th e r e fle c tio n c o e f fic ie n ts [ K m), w e r e s im p ly r e la te d to th e f ilte r c o e f
fic ie n ts (a m(A:)} o f t h e d ir e c t fo r m F I R lin e a r p r e d ic t o r a n d th e a s s o c ia t e d p r e d ic t io n
e r r o r filte r . T h e o p tim u m f ilte r c o e f f ic ie n ts { K m) a n d (a m( ) } a r e e a s ily o b ta in e d
f r o m th e s o lu tio n o f th e n o r m a l e q u a tio n s .
W e d e s c r ib e d tw o c o m p u ta tio n a lly e ff ic ie n t a lg o r it h m s f o r s o lv in g th e n o r m a l
e q u a tio n s , th e L e v in s o n - D u r b in a lg o r ith m a n d th e S c h u r a lg o r it h m .

B o t h a lg o

rith m s a r e s u ita b le f o r s o lv in g a T o e p lit z s y s te m o f li n e a r e q u a tio n s a n d h a v e a

c o m p u ta tio n a l c o m p le x ity o f 0 ( p 2 ) w h e n e x e c u te d o n a s in g le p r o c e s s o r .

H ow

e v e r , w ith fu ll p a r a lle l p r o c e s s in g , th e S c h u r a lg o r ith m s o lv e s th e n o r m a l e q u a tio n s

in O ( p ) tim e , w h e r e a s th e L e v in s o n - D u r b in a lg o r ith m r e q u ir e s O ( p l o g p ) tim e .
In a d d itio n t o th e a ll- z e r o la ttic e filte r r e s u ltin g f r o m lin e a r p r e d ic t io n , w e
a ls o d e riv e d th e A R la t t ic e ( a ll- p o le ) filte r s tr u c tu r e a n d th e A R M A l a ttic e - la d d e r
( p o l e - z e r o ) filte r s tr u c tu r e . F in a lly , w e d e s c r ib e d th e d e s ig n o f th e c la s s o f o p t i
m u m lin e a r filte r s , c a lle d W ie n e r filte r s .
L in e a r e s tim a tio n th e o r y h a s h a d a lo n g a n d r ic h h is to r y o f d e v e lo p m e n t
o v e r th e p a s t fo u r d e c a d e s . K a ila th (1 9 7 4 ) p r e s e n ts a h is to r ic a l a c c o u n t o f th e firs t
t h r e e d e c a d e s . T h e p io n e e r in g w o rk o f W ie n e r ( 1 9 4 9 ) o n o p tim u m lin e a r filte r in g
f o r s ta tis tic a lly s ta tio n a r y s ig n a ls is e s p e c ia lly s ig n ific a n t.

T h e g e n e r a liz a tio n o f

th e W i e n e r filte r t h e o r y t o d y n a m ic a l s y s te m s w ith ra n d o m in p u ts w a s d e v e lo p e d
b y K a lm a n ( 1 9 6 0 ) a n d K a lm a n a n d B u c y ( 1 9 6 1 ) .

K a lm a n filte r s a r e tr e a t e d in

th e b o o k s b y M e d itc h ( 1 9 6 9 ) , B r o w n ( 1 9 8 3 ) , a n d C h u i a n d C h e n ( 1 9 8 7 ) .

The

m o n o g r a p h b y K a ila t h ( 1 9 8 1 ) tr e a ts b o th W i e n e r a n d K a lm a n f ilte r s .
T h e r e a r e n u m e r o u s r e f e r e n c e s o n lin e a r p r e d ic t io n a n d la ttic e f ilte r s .

Tu

t o r ia l t r e a t m e n t s o n t h e s e s u b je c t s h a v e b e e n p u b lis h e d in th e jo u r n a l p a p e r s b y
M a k h o u l ( 1 9 7 5 , 1 9 7 8 ) a n d F r ie d la n d e r (1 9 8 2 a , b ) . T h e b o o k s b y H a y k in ( 1 9 9 1 ) ,
M a r k e l a n d G r a y 1 9 7 6 ) , a n d T r e t t e r ( 1 9 7 6 ) p r o v id e c o m p r e h e n s iv e tr e a t m e n t s o f
th e s e s u b je c t s . A p p lic a tio n s o f li n e a r p r e d ic t io n t o s p e c tr a l a n a ly s is a r e fo u n d in
th e b o o k s b y K a y ( 1 9 8 8 ) an d M a r p le ( 1 9 8 7 ) , to g e o p h y s ic s in th e b o o k R o b in s o n
a n d T r e i t e l ( 1 9 8 0 ) , a n d to a d a p tiv e filte r in g in th e b o o k b y H a y k in ( 1 9 9 1 ) .
T h e L e v in s o n - D u r b in a lg o r ith m f o r s o lv in g th e n o r m a l e q u a tio n s r e c u r s iv e ly
w a s g iv e n b y L e v in s o n ( 1 9 4 7 ) a n d la t e r m o d ifie d b y D u r b in ( 1 9 5 9 ) .

V a r ia t io n s

o f th is c la s s ic a l a lg o r it h m , c a lle d s p lit L e in s o n a lg o r it h m s , h a v e b e e n d e v e lo p e d
b y D e l s a r t e a n d G e n i n ( 1 9 8 6 ) a n d b y K r is h n a ( 1 9 8 8 ) .

T h e s e a lg o r it h m s e x p lo it

a d d itio n a l s y m m e tr ie s in th e T o e p l it z c o r r e la tio n m a tr ix a n d s a v e a b o u t a f a c t o r
o f 2 in th e n u m b e r o f m u ltip lic a tio n s .
T h e S c h u r a lg o r it h m w as o r ig in a lly d e s c r ib e d b y S c h u r ( 1 9 1 7 ) in a p a p e r
p u b lis h e d in G e r m a n .

A n E n g lis h tr a n s la tio n o f th is p a p e r a p p e a r s in th e b o o k

e d ite d b y G o h b e r g ( 1 9 8 6 ) . T h e S c h u r a lg o r ith m is in tim a te ly r e la te d t o th e p o ly

n o m ia ls

w h ich c a n b e i n te r p r e te d a s o r th o g o n a l p o ly n o m ia ls . A t r e a t m e n t

o f o r t h o g o n a l p o ly n o m ia ls is g iv e n in th e b o o k s b y S z e g o ( 1 9 6 7 ) , G r e n a n d e r a n d
S z e g o ( 1 9 5 8 ) , a n d G e r o n im u s ( 1 9 5 8 ) . T h e th e s is o f V i e i r a ( 1 9 7 7 ) a n d th e p a p e r s
b y K a ila t h e t a l. ( 1 9 7 8 ) , D e l s a r t e e t a l. ( 1 9 7 8 ) , a n d Y o u l a a n d K a z a n jia n ( 1 9 7 8 )

892

Linear Prediction and Optimum Linear Filters

p r o v id e a d d itio n a l re s u lts o n o r th o g o n a l p o ly n o m ia ls .

Chap. 11

K a il a t h ( 1 9 8 5 , 1 9 8 6 ) p r o

v id e s t u t o r ia l t r e a t m e n t s o f th e S c h u r a lg o r it h m a n d its r e la tio n s h ip to o r th o g o n a l
p o ly n o m ia ls a n d t h e L e v in s o n - D u r b in a lg o r ith m . T h e p ip e lin e d p a r a lle l p r o c e s s
in g s tr u c tu r e f o r c o m p u tin g t h e r e fle c tio n c o e f f ic ie n t s b a s e d o n t h e S c h u r a lg o r ith m
a n d th e r e la te d p r o b le m o f s o lv in g T o e p l it z s y s te m s o f lin e a r e q u a t i o n s is d e s c r ib e d
in th e p a p e r b y K u n g a n d H u ( 1 9 8 3 ) . F in a lly , w e s h o u ld m e n t i o n t h a t s o m e a d
d itio n a l c o m p u ta tio n a l e f fic ie n c y c a n b e a c h ie v e d in th e S c h u r a lg o r it h m , b y fu r
t h e r e x p lo itin g s y m m e tr y p r o p e r tie s o f T o e p l it z m a t r ic e s , a s d e s c r i b e d b y K r is h n a
( 1 9 8 8 ) . T h is le a d s t o t h e s o -c a lle d s p lit- S c h iir a lg o r it h m , w h ic h is a n a lo g o u s to th e
s p lit- L e v in s o n a lg o r ith m .

PROBLEMS
11.1 T h e pow er density spectrum o f an A R process [x (n )) is given as
r jr M

g- |A{oj)|2
25
|1 - e~>w + i e - J '2" ! 2

w h ere al, is th e v arian ce o f th e input seq uen ce.

(a) D ete rm in e th e d ifferen ce equation fo r g en eratin g the A R p ro ce ss w hen the e x
citatio n is w hite noise.
(b) D eterm in e th e system function fo r th e w hitening filter.
1 L 2 A n A R M A p rocess has an au to co rrelatio n [yIX (m)} w hose z-tran sfo rm is given as

(a)

D ete rm in e th e filter H ( z ) for g en eratin g (x (n )) fro m a w hite n o ise input seq uen ce.
Is H ( z ) un iqu e? E xp lain .

(b)

D ete rm in e a stab le lin ear w hitening filter for th e seq u en ce {*())

1 L 3 C o n sid er th e A R M A p ro cess g en erated by th e d ifferen ce eq u a tio n

x(n ) = 1.6x(n 1) 0 .6 3 * ( - 2) +

(a)
(b)

+ 0.9w (n - 1)

D eterm in e th e system fun ction o f th e w hitening filter and its p o les and zeros.
D ete rm in e th e pow er density spectrum o f {jc(n )}.

11.4 D ete rm in e the lattice coefficien ts correspond ing to th e F I R filter w ith system function

H(z) = Ai(z) = 1 + j j z ' 1 + f z~2 + jz~ 3

1 L 5 D ete rm in e th e reflection co efficie n ts { K m} o f the lattice filter co rresp o n d in g to the
F I R filter described by the system fun ction

Chap. 11

893

Problems

1 1 .6 (a) D ete rm in e the zeros and sk etch the zero p attern fo r the F I R la ttice filter with
reflectio n co efficien ts

Ki = \

K2 = - \

*3 = 1

(b ) R e p e a t part (a ) but with Ky = 1.

(c ) Y o u should have found that th e zeros lie on th e unit circle. C an this result be
g en eralized ? H ow ?
1 1 .7 D ete rm in e th e im pulse response o f th e F I R filter th at is described by the lattice
co efficien ts K \ = 0.6, K 2 = 0.3, K 3 = 0 .5 , and K A = 0.9.
1 1 .8 In S ec tio n 11.2.4 we indicated th at th e n oise-w hitening filter A p{z ) for a causal A R ( p )
process is a forw ard lin ear p red iction -erro r filter o f o rd e r p. Show th a t th e backw ard
lin ear p red ictio n -erro r filter o f o rd er p is the n oise-w hiten in g filter o f th e co rresp o nd
ing an ticau sal A R (p) process.
1 1 .9 U se th e o rth og o nality principie to d eterm in e the norm al eq u atio ns and the resulting
m inim um M S E fo r a forw ard p red ictor o f ord er p th at p red icts m sam ples {m > 1)
into th e futu re (m -step forw ard p red icto r). S k etc h th e p red iction erro r filter.
11 .1 0 R e p e a t P rob lem 11.9 fo r an m -step backw ard predictor.
11.11 D ete rm in e a L e v in so n -D u rb in recu rsive algorithm fo r solving fo r the coefficien ts o f
a backw ard pred iction -erro r filter. U se th e result to show th at co efficien ts o f th e
forw ard and backw ard predictors can be exp ressed recu rsively as

1 1 .1 2 T h e L e v in so n -D u rb in algorithm d escrib ed in S ectio n 11.3.1 solved the lin ear eq u a

tions
r a m= - y m
where the right-hand side o f this eq u atio n has elem en ts o f th e au toco rrelatio n se
qu en ce th a t are also elem en ts o f th e m atrix T .
problem o f solving th e lin ear eq uations

L e t us co n sid er th e m ore general

r mb m
w here cm is an arb itrary v ector. (T h e v ecto r bm is n ot related to th e coefficien ts o f
th e backw ard p red icto r.) Show th at th e solu tion to r mb = c can b e obtain ed from
a generalized L e v in s o n -D u rb in algorithm which is given recursively as

bm(k) =

- bm{m )a' ^(m - k)

'" "

m = 1 ,2 ........p

w here M l ) = c ( l) / y j,( 0 ) = c(1)/q and a(k) is given by (11,3.17). T hu s a second

recu rsion is required to solve th e eq u atio n I^ b * , = c * .
1 L 1 3 U se th e g eneralized L e v in so n -D u rb in algorithm to solve th e norm al eq uations recu r
sively fo r th e m -step forw ard and backw ard p redictors.

894

Linear Prediction and Optimum Linear Filters

Chap. 11

11.14 Show that the transform ation

in the Sch ur algorithm satisfies the special property

V ,JV = (1 - \ K J 2) i
w here

T hu s V m is called a 7 -ro ta tio n m atrix. Its ro le is to ro tate o r h y p erbolate th e row o f

G m to lie alon g the first co o rd in ate d irectio n (K a ila th , 1985).
11.15 Prov e the additional p rop erties (a ) through (1) o f th e p red iction - e rro r filters given in
Sectio n 11.4.
11.16 E x te n d the add ition al prop erties (a ) through (1) o f th e p red iction erro r filters given
in Sectio n 11.4 to com plex-valued signals.
1 1.17 D eterm in e th e reflection co efficie n t K 3 in term s o f th e a u to co rrela tio n s {yTr (m )} from
th e Schur algorithm and co m p are your result with the ex p ressio n fo r K 3 o btain ed
from th e L e v in so n -D u rb in algorithm .
1 1.18 C onsid er a in finite-length (p = oo) o n e-step forw ard p red ictor fo r a statio nary random
process (x(n )) with a pow er density spectru m o f
Show th a t th e m ean -squ are
erro r o f th e p red ictio n -erro r filter can b e expressed as

1 1 .1 9 D ete rm in e th e output o f an in finite-len gth (p = oo) m -step forw ard p red ictor and
th e resulting m ean -squ are e rro r w hen th e input signal is a first-o rd er autoregressive
process o f the form

x(n ) = ax(n 1) + w(n)

1 L 2 0 A n A R (3 ) process {x(n)} is ch aracterized by the au toco rrelatio n seq u en ce yJX (0) = 1,
(a ) U se the Schtir algorithm to d eterm in e the th ree reflection co efficien ts K u K 2, and
(b ) Sk etch th e lattice filter fo r synthesizing {x (n )} from a white n oise ex citation .
1 L 2 1 T h e purpose o f this p roblem is to show th at the polynom ials (A (z )}, which a re the
system functions o f th e forw ard p red ictio n -erro r filters o f o rd e r m, m = 0 , 1 , . . . , p,
can be in terp reted as o rth og o n al on th e unit circle. T ow ard th is end, suppose that
r j x ( f ) is the pow er spectral density o f a zero-m ean rand om p ro cess (* { n ) } and let
M m (z)}, m = 0, 1 . . . . , p ), b e the system functions o f th e co rresp o n d in g prediction erro r filters. Show th at th e polyn o m ials {A m(z)} satisfy the o rth og o n ality property

1 L 2 2 D eterm in e th e system fun ction o f th e all-p ole filter d escrib ed by th e la ttice co efficien ts
J fi = 0.6, K 2 = 0 .3 , K } = 0 .5 , and K A = 0.9.

Chap. 11

895

Problems

11.23 Determine the parameters and sketch the lattice-ladder filter structure for the system
with system function

H{Z)
11.24 Consider a signal

jc ( )

1 + 0 .1 ;- i- 0 .7 2 ;- 2

= s(n) + u ;(n), where s(n) is an A R ( 1 ) process that satisfies

the difference equation

s (n ) = Q .8s(n 1) + v(n)

where {u(n)) is a white noise sequence with variance er2 = 0.49 and {w(n)) is a
white noise sequence with variance er2 = 1. The processes {t>{n)) and {u)(n)J are
uncorrelated.
(a) Determine the autocorrelation sequences {y(m)) and {yXJ(m)].
( b ) Design a Wiener filter of length M = 2 to estimate |j{n)}.
(c) Determine the MMSE for M = 2.
11.25 Determine the optimum causal IIR Wiener filter for the signal given in Problem 11.24
and the corresponding MMSE^.
11.26 Determine ihe system function for the noncausal IIR Wiener filter for the signal given
in Problem 11.24 and the corresponding MMSEnr.
11.27 Determine the optimum FIR Wiener filter of length M 3 for the signal in Ex
ample 11.6.1 and the corresponding MMSEj. Compare MMSEj with MMSEi and
comment on the difference.
11.28 An AR(2) process is defined by the difference equation
= x(n 1) 0.6jr(n 2) + u>(ri)
where {u>(n)} is a white noise process with variance <t2, Use the Yule-Walker equa
tions to solve for the values of the autocorrelation y(0), ^ x(l). and yxx(2).
11.29 An observed random process (*()) consists of the sum of an AR{j>) process of the
form
p

and a white noise process {u>{n)} with variance tr2. The random process {u()} is also
white with variance er2. The sequences {u(n)} and fio(n)} are uncorrelated.
Show that the observed process
= j() + u>()} is ARM A(p, p) and de
termine the coefficients of the numerator polynomial (MA component) in the corre
sponding system function.

a 12

Power Spectrum Estimation

In th is c h a p te r w e a r e c o n c e r n e d w ith th e e s tim a tio n o f th e s p e c t r a l c h a r a c te r is tic s

o f s ig n a ls c h a r a c te r iz e d as r a n d o m p r o c e s s e s . M a n y o f th e p h e n o m e n a th a t o c c u r
in n a tu r e a r e b e s t c h a r a c te r iz e d s ta tis tic a lly in te r m s o f a v e r a g e s .

F o r e x a m p le ,

m e t e o r o lo g ic a l p h e n o m e n a s u c h a s th e flu c tu a tio n s in a ir t e m p e r a t u r e a n d p r e s s u re
a r e b e s t c h a r a c te r iz e d s ta tis tic a lly a s ra n d o m p r o c e s s e s .

T h e r m a l n o is e v o lta g e s

g e n e r a te d in r e s is to r s a n d e le c t r o n i c d e v ic e s a r e a d d itio n a l e x a m p le s o f p h y s ic a l
s ig n a ls th a t a r e w e ll m o d e le d a s ra n d o m p r o c e s s e s .
D u e to th e r a n d o m f lu c tu a tio n s in s u ch s ig n a ls , w e m u s t a d o p t a s ta tis ti
c a l v ie w p o in t, w h ich d e a ls w ith th e a v e r a g e c h a r a c t e r is t ic s o f r a n d o m s ig n a ls . In
p a r tic u la r , th e a u t o c o r r e la tio n fu n c t io n o f a r a n d o m p r o c e s s is th e a p p r o p r ia te
s ta tis tic a l a v e r a g e th a t w e w ill u se fo r c h a r a c te r iz in g ra n d o m s ig n a ls in th e tim e
d o m a in , a n d th e F o u r i e r tr a n s fo r m o f th e a u t o c o r r e la t io n f u n c t io n , w h ic h y ie ld s
t h e p o w e r d e n s ity s p e c tr u m , p r o v id e s th e t r a n s fo r m a tio n fr o m t h e tim e d o m a in t o
th e f r e q u e n c y d o m a in .
P o w e r s p e c tr u m e s tim a tio n m e t h o d s h a v e a r e la tiv e ly lo n g h is to r y .

For a

h is to r ic a l p e r s p e c tiv e , th e r e a d e r is r e f e r r e d to th e p a p e r b y R o b i n s o n ( 1 9 8 2 ) a n d
t h e b o o k b y M a r p le ( 1 9 8 7 ) . O u r t r e a t m e n t o f th is s u b je c t c o v e r s t h e c la s s ic a l p o w e r
s p e c tr u m e s tim a tio n m e th o d s b a s e d o n th e p e r io d o g r a m , o r ig in a lly in tr o d u c e d by
S c h u s t e r ( 1 8 9 8 ) , a n d b y Y u l e ( 1 9 2 7 ) , w h o o r ig in a te d t h e m o d e m m o d e l- b a s e d o r
p a r a m e t r ic m e th o d s . T h e s e m e t h o d s w e r e s u b s e q u e n tly d e v e lo p e d a n d a p p lie d b y
W a lk e r (1 9 3 1 ), B a r tle tt (1 9 4 8 ), P a rz e n (1 9 5 7 ), B la c k m a n an d T u k e y (1 9 5 8 ), B u rg
( 1 9 6 7 ) , a n d o th e r s . W e a ls o d e s c r ib e t h e m e t h o d o f C a p o n ( 1 9 6 9 ) a n d m e t h o d s
b a s e d o n e ig e n a n a ly s is o f th e d a ta c o r r e l a t io n m a tr ix .

12.1 ESTIMATION OF SPECTRA FROM FINITE-DURATION

OBSERVATIONS OF SIGNALS
T h e b a s ic p r o b le m th a t w e c o n s id e r in th is c h a p t e r is th e e s t im a t io n o f th e p o w e r
d e n s ity s p e c tr u m o f a s ig n a l fr o m th e o b s e r v a t io n o f th e s ig n a l o v e r a fin ite tim e
in te r v a l. A s w e w ill s e e , th e fin ite r e c o r d le n g th o f t h e d a ta s e q u e n c e is a m a jo r

896

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

lim ita tio n o n th e q u a lity o f th e p o w e r s p e c tr u m e s t i m a t e .

897

W h e n d e a lin g w ith

s ig n a ls t h a t a r e s ta tis tic a lly s ta tio n a r y , th e lo n g e r th e d a ta r e c o r d , t h e b e t t e r th e

e s t im a t e t h a t c a n b e e x t r a c t e d fr o m th e d a ta .

O n t h e o t h e r h a n d , i f th e s ig n a l

s ta tis tic s a r e n o n s ta tio n a r y , w e c a n n o t s e le c t a n a r b itr a r ily lo n g d a ta r e c o r d to

e s tim a te t h e s p e c tr u m .

In s u c h a c a s e , th e le n g th o f t h e d a ta r e c o r d t h a t w e

s e l e c t is d e te r m in e d b y th e ra p id ity o f th e tim e v a r ia tio n s in th e s ig n a l s ta tis tic s .

U l t i m a t e l y , o u r g o a l is t o s e le c t a s s h o r t a d a ta r e c o r d a s p o s s ib le t h a t s till a llo w s
u s t o r e s o lv e t h e s p e c tr a l c h a r a c te r is tic s o f d if f e r e n t s ig n a l c o m p o n e n ts in th e d a ta
r e c o r d t h a t h a v e c lo s e ly s p a c e d s p e c tr a .
O n e o f t h e p r o b le m s th a t w e e n c o u n t e r w ith c la s s ic a l p o w e r s p e c tr u m e s t im a
tio n m e t h o d s b a s e d o n a fin ite -le n g th d a ta r e c o r d is t h e d is to r tio n o f th e s p e c tr u m
t h a t w e a r e a tte m p tin g t o e s tim a te . T h i s p r o b le m o c c u r s in b o th th e c o m p u ta tio n
o f t h e s p e c tr u m f o r a d e te r m in is tic s ig n a l a n d th e e s t im a t io n o f th e p o w e r s p e c
tr u m o f a r a n d o m s ig n a l. S in c e it is e a s ie r to o b s e r v e t h e e f f e c t o f th e fin ite le n g th
o f th e d a ta r e c o r d o n a d e te r m in is tic s ig n a l, w e t r e a t th is c a s e firs t. T h e r e a f t e r , w e
c o n s id e r o n ly r a n d o m s ig n a ls a n d th e e s tim a tio n o f t h e i r p o w e r s p e c tr a .

12.1.1 Computation of the Energy Density Spectrum

L e t us c o n s id e r th e c o m p u ta tio n o f th e s p e c tr u m o f a d e te r m in is tic s ig n a l f r o m
a f in ite s e q u e n c e o f d a ta . T h e s e q u e n c e jc (n ) is u s u a lly t h e re s u lt o f s a m p lin g a
c o n tin u o u s - tim e s ig n a l x a (t) a t s o m e u n ifo rm s a m p lin g r a t e F s . O u r o b je c t i v e is
to o b t a in a n e s tim a te o f t h e tr u e s p e c tr u m fr o m a f in ite - d u r a tio n s e q u e n c e x ( n ) .
R e c a l l t h a t if x ( t ) is a fin ite -e n e r g y s ig n a l, t h a t is,

CC

\xa (t)\2d t < oo

-00

th e n its F o u r i e r t r a n s f o r m e x is ts a n d is g iv e n a s

X a(F ) =

Xo( t ) e - j2 * F ,d t

J -OO

F r o m P a r s e v a ls t h e o r e m w e h a v e
oo

\xa ( t ) \2dt =

roc
/
\ X a ( F ) l 2d F

( 1 2 .1 .1 )

J oc

00

T h e q u a n t it y |Xa (F )| 2 r e p r e s e n ts th e d is tr ib u tio n o f s ig n a l e n e r g y a s a f u n c
tio n o f fr e q u e n c y , a n d h e n c e it is c a lle d th e e n e r g y d e n s ity s p e c tr u m o f th e s ig n a l,
t h a t is,

SXA F ) = \ X e ( F ) \ 2

( 1 2 .1 .2 )

a s d e s c r ib e d in C h a p te r 4 . T h u s t h e to t a l e n e r g y in t h e s ig n a l is s im p ly th e in te g r a l
o f SXX{ F ) o v e r a ll F [ i.e ., th e to t a l a r e a u n d e r SJtJt( F ) ] .
I t is a ls o in te r e s tin g t o n o te t h a t S XX( F ) c a n b e v ie w e d a s t h e F o u r i e r t r a n s
fo rm

o f a n o t h e r fu n c t io n ,

R IX { t ) ,

c a lle d

th e

a u to c o rr e la tio n fu n c t io n o f th e

898

Power Spectrum Estimation

Chap. 12

fin ite - e n e r g y s ig n a l x a (t), d e fin e d as

RxA*)= f

J OC

x * { t) x a { t + x ) d t

( 1 2 .1 .3 )

I n d e e d , it e a s ily fo llo w s th a t

R x x ( r ) e - J2jrFrd x = SXA F ) = l * ( F ) | 2
J C
s o t h a t R x x (?) a n d SXX( F ) a r e a F o u r i e r t r a n s fo r m p a ir.

( 1 2 .1 .4 )

N o w s u p p o s e th a t w e c o m p u te th e e n e r g y d e n s ity s p e c tr u m o f th e s ig n a l x a (t )
f r o m its s a m p le s t a k e n a t th e r a t e F s s a m p le s p e r s e c o n d . T o e n s u r e th a t th e r e is
n o s p e c tr a l a lia s in g re s u ltin g f r o m th e s a m p lin g p r o c e s s , th e s ig n a l is a s s u m e d to
b e p r e filte r e d , s o th a t, f o r p r a c tic a l p u r p o s e s , its b a n d w id th is lim ite d to B h e r tz .
T h e n th e s a m p lin g fr e q u e n c y F s is s e le c te d s u c h th a t F s > 2 B .
T h e s a m p le d v e r s io n o f x a (t) is a s e q u e n c e * { ) , - o c < n < o c , w h ich h a s a
F o u r i e r tr a n s fo r m ( v o lta g e s p e c tr u m )
OC

X { a i) =

22

ns-cc

x (n )e ~ J<"

o r, e q u iv a le n tly ,

X (f) =

22

X ( n ) e - j2xfn

( 1 2 .1 .5 )

A! = CC

R e c a l l th a t X { f ) c a n b e e x p r e s s e d in te r m s o f th e v o lta g e s p e c tr u m o f th e a n a lo g
s ig n a l jco ( 0 as
*

( )

X ( F ~ k F ^

( l 2 1 '6)

w h e r e / = F / F s is th e n o r m a liz e d f r e q u e n c y v a r ia b le .
I n th e a b s e n c e o f a lia s in g , w ith in th e f u n d a m e n ta l ra n g e l^ l < F J 2 , w e h a v e

X a ( F ) , |Fj < F J 2

( 1 2 .1 .7 )

H e n c e th e v o lta g e s p e c tru m o f th e s a m p le d s ig n a l is id e n tic a l to t h e v o lta g e s p e c

tr u m o f th e a n a lo g s ig n a l. A s a c o n s e q u e n c e , th e e n e r g y d e n s it y s p e c tr u m o f th e
s a m p le d s ig n a l is

= F } \X a ( F ) \2

( 1 2 .1 .8 )

W e c a n p r o c e e d f u r t h e r b y n o tin g t h a t th e a u t o c o r r e la t io n o f t h e s a m p le d
s ig n a l, w h ich is d e fin e d as
OO

rxx ( k ) =

2 2

* * (

+ *)

(1 2 - 1 .9 )

#*= 00
h a s th e F o u r i e r tr a n s fo r m ( W i e n e r - K h i n t c h i n e t h e o r e m )

S (/)=

j t

r*Ak )e~j2*kf

(12.1.10)

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

899

H e n c e th e e n e rg y d e n s ity sp e c tru m c a n b e o b ta in e d by th e F o u rie r tra n s fo rm o f

th e a u to c o rre la tio n o f th e se q u e n c e {Jt(n)}.
T h e r e la tio n s a b o v e le a d us to d istin g u ish b e tw e e n tw o d istin ct m e th o d s fo r
co m p u tin g th e en e rg y d e n s ity sp e c tru m o f a signal x(t) fro m its sa m p le s Jt(n). O n e
is th e direct method, w hich involves co m p u tin g th e F o u rie r tra n s fo rm o f {*()], a n d
th e n

SxA f ) = l * ( / ) l 2
2
OO
2 2 X( n) e~j2*fn

(12.1.11)

T h e se co n d a p p ro a c h is c a lle d th e indirect method b e c a u s e it re q u ire s tw o ste p s.

F irst, th e a u to c o r re la tio n rxx(k) is c o m p u te d fro m x(rt) a n d th e n th e F o u rie r tr a n s
fo rm o f th e a u to c o rre la tio n is c o m p u te d as in (12.1.10) to o b ta in th e e n e rg y d e n sity
sp e ctru m .
In p ra c tic e , h o w e v e r, only th e fin ite -d u ra tio n se q u e n c e x (n ), 0 < n < N - 1 , is
a v ailab le fo r c o m p u tin g th e sp e c tru m o f th e signal. In effe ct, lim itin g th e d u ra tio n
o f th e se q u e n c e x ( n ) t o N p o in ts is e q u iv a le n t to m u ltip ly in g x ( n ) b y a re c ta n g u la r
w ind o w . T h u s w e h av e
0 < n < N 1
o th e rw ise

(12.1.12)

F ro m o u r d iscu ssio n o f F I R filter design b ased o n th e u se o f w indow s to lim it th e

d u ra tio n o f th e im p u lse re sp o n s e , w e recall th a t m u ltip lic a tio n o f tw o se q u e n c e s is
e q u iv a le n t to co n v o lu tio n o f th e ir v o ltag e sp e c tra . C o n s e q u e n tly , th e freq u e n cy d o m a in r e la tio n c o rre sp o n d in g to (12.1.12) is

X i f ) = * ( / ) * W{ f )
(12.1.13)
- 1/2

R e call fro m o u r d iscussion in S ectio n 8.2.1 th a t c o n v o lu tio n o f th e w indow

f u n c tio n W ( f ) w ith X ( f ) s m o o th s th e sp e c tru m X ( f ) , p ro v id e d th a t th e sp e c tru m
W ( f ) is re la tiv e ly n a rro w c o m p a re d to X ( / ) . B u t th is c o n d itio n im plies th a t
th e w in d o w w i n ) b e su fficiently long (i.e., N m u st b e sufficiently la rg e ) such th a t
W ( f ) is n a rro w c o m p a re d to X ( / ) . E v e n if W ( f ) is n a rro w c o m p a re d to X ( f ) ,
th e co n v o lu tio n o f X ( f ) w ith th e sid e lo b e s o f W ( f ) re su lts in sid e lo b e en erg y in
X ( f ) , in fre q u e n c y b a n d s w h e re th e tr u e signal sp e c tru m X ( / ) = 0. T h is sid e lo b e
en e rg y is called leakage. T h e follow ing ex a m p le illu stra te s th e le a k a g e p ro b le m .
Example 12.1.1
A signal with (voltage) spectrum
l/l< 0 .1
otherwise
is convolved with the rectangular window of length N = 61. D eterm ine the spectrum
o f X ( f ) g iv e n b y (1 2 .1 .1 3 ).

900

Power Spectrum Estimation

Chap. 12

Figure 1Z1 Spectrum obtained by convolving an M = 61 rectan g u lar window

with the ideal lowpass spectrum in Exam ple 12.1.1.

Solution The spectral characteristic W ( f ) for the length N = 61 rectangular window

is illustrated in Fig. 8.2(b). Note that the width of the main lobe of the window
function is Aw = 4^/61 or A/ = 2/61, which is narrow compared to X ( f ) .
The convolution of X ( f ) with W ( f ) is illustrated in Fig. 12.1. We note that
energy has leaked into the frequency band 0.1 < | / | < 0.5. where X ( f ) = 0. A part
of this is due to the width of the main lobe in VV(/), which causes a broadening or
smearing of X ( f ) outside the range \ f \ < 0.1. However, the sidelobe energy in X ( f )
is due to the presence of the sidelobes of W (/), which are convolved with X ( f ) .
The smearing of X ( f ) for | / | > 0.1 and the sidelobes in the range 0.1 < | / | < 0.5
constitute the leakage.
Ju s t as in th e case o f F IR filter d esig n , w e can re d u c e sid e lo b e le a k a g e by
selectin g w in d o w s th a t h av e low sid e lo b e s. T h is im p lies th a t th e w in d o w s h av e a
sm o o th tim e -d o m a in cu to ff in ste a d o f th e a b ru p t cu to ff in th e re c ta n g u la r w indow .
A lth o u g h such w in d o w fu n ctio n s re d u c e sid e lo b e le a k a g e , th ey re s u lt in an in crease
in sm o o th in g o r b ro a d e n in g o f th e s p e c tra l c h a ra c te ristic X ( f ) . F o r ex am p le, the
use o f a B lac k m a n w in d o w o f le n g th N = 61 in E x a m p le 12.1.1 re su lts in th e
sp e c tra l c h a ra c te ristic X ( / ) sh o w n in F ig. 12.2. T h e sid e lo b e le a k a g e h as c e rta in ly
b e e n re d u c e d , b u t th e sp e c tra l w id th h a s b e e n in c re a se d by a b o u t 5 0 % .
T h e b ro a d e n in g o f th e s p e c tru m b e in g e s tim a te d d u e to w in d o w in g is p a rtic u
larly a p ro b le m w h e n w e w ish to reso lv e signals w ith closely sp a c e d fre q u e n c y co m
p o n e n ts . F o r ex a m p le , th e signal w ith sp e ctral c h a ra c te ristic X ( / ) =
{ f ) + X i { f ),
as sh o w n in Fig. 12.3, c a n n o t b e reso lv ed as tw o se p a ra te sig n a ls u n le ss th e w idth
o f th e w in d o w fu n c tio n is significantly n a r ro w e r th a n th e fre q u e n c y se p a ra tio n A f .
T h u s w e o b se rv e th a t u sing sm o o th tim e -d o m a in w indow s re d u c e s le ak ag e a t th e
ex p en se o f a d e c re a s e in fre q u e n c y re so lu tio n .

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

901

Figure 12.2 Spectrum obtained by convolving an M = 61 B lackm an window w ith

the ideal lowpass spectrum in Exam ple 12.1.1.

I t is c le a r fro m th is discussion th a t th e e n e rg y d en sity s p e c tru m o f th e w in

d o w ed se q u e n c e {*()} is an a p p ro x im a tio n o f th e d e s ire d sp e c tru m o f th e se q u en ce
{jc(n)J. T h e sp e c tra l d en sity o b ta in e d fro m {Jc()} is
2

Sisif) = W ) \ 2 =

j2nfn

(12.1.14)

T h e sp e c tru m given by (12.1.14) can be c o m p u te d n u m erically a t a set of N

fre q u e n c y p o in ts by m e a n s o f th e D F T . T h u s
X ( k ) = 2 2 , x ( n ) e - J2nkn/N
ji=Q

(12.1.15)

Then
|X(fc)|2 = Si-Af)\j-k/N = Sri

(12.1.16)

902

Power Spectrum Estimation

Chap. 12

and hence
j 2 n k n / N

(12.1.17)

w hich is a d isto rte d v ersio n o f th e tru e s p e c tru m Sxx( k / N ) .

12.1.2 Estimation of the Autocorrelation and Power

Spectrum of Random Signals: The Periodogram
T h e fin ite-en erg y signals c o n s id e re d in th e p re c e d in g se c tio n possess a F o u rie r
tra n sfo rm an d a re c h a ra c te riz e d in th e sp e c tra l d o m a in by th e ir e n e rg y d en sity
sp e ctru m . O n th e o th e r h a n d , th e im p o rta n t class o f signals c h a ra c te riz e d as s ta
tio n a ry ra n d o m p ro c e sse s d o n o t h a v e finite e n e rg y a n d h e n c e d o n o t po ssess a
F o u rie r tran sfo rm . S uch signals h a v e finite a v e ra g e p o w e r a n d h en c e a re c h a ra c
te riz e d by a p o w e r density spect rum. If x ( t ) is a s ta tio n a ry ra n d o m p ro cess, its
a u to c o rre la tio n fu n ctio n is
yxx(r) = E [ x * ( t ) x ( t +

t )]

(12.1.18)

w h ere [] d e n o te s th e sta tistical a v e ra g e . T h e n , via th e W ie n e r-K h in tc h in e th e

o re m , th e p o w e r d e n sity sp e c tru m o f th e sta tio n a ry ra n d o m p ro c e ss is th e F o u rie r
tra n sfo rm o f th e a u to c o rre la tio n fu n c tio n , th a t is,
/ OC
yxA T ) e ~ j2* FTd t
(12.1.19)
-oc
In p ra c tic e , w e d e a l w ith a single re a liz a tio n o f th e ra n d o m p ro cess fro m
w hich w e e stim a te th e p o w e r sp e c tru m o f th e p ro cess. W e d o n o t k n o w th e tru e
a u to c o rre la tio n fu n ctio n yxx( t ) a n d as a c o n se q u e n c e , w e c a n n o t c o m p u te th e
F o u rie r tra n sfo rm in (12.1.19) to o b ta in rjx(F). O n th e o th e r h a n d , fro m a single
re a liz a tio n o f th e ra n d o m p ro c e ss w e c a n c o m p u te th e tim e -a v e ra g e a u to c o rre la
tio n fu n ctio n
R x A r ) = r ^ r f x *{ t ) x ( t + x) dt
J - T0

(12.1.20)

w h ere 27o is th e o b se rv a tio n in te rv a l. I f th e sta tio n a ry r a n d o m p ro cess is ergodic

in th e first a n d se co n d m o m e n ts (m e a n a n d a u to c o rre la tio n fu n c tio n ), th e n
X ut(r) =

lim R x x i r )

lim
To-+oc

To-*oo

7b

(12.1.21)

27b /_ jc*(r)jc (r + r )dt

To

T h is re la tio n ju stifies th e u se o f th e tim e -a v e ra g e a u to c o rre la tio n Rxx( r ) as

a n e stim a te o f th e sta tistical a u to c o r re la tio n fu n c tio n yxx(r ) . F u rth e rm o r e , th e
F o u rie r tra n sfo rm o f R xx(r ) p ro v id e s a n e s tim a te PXX( F ) o f th e p o w e r d en sity

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

903

s p e c tru m , th a t is,
7b

Pxx(F) = [ R xx{ T ) e - j:brFrd r

J- T7b
q
i

frTo
T rr rr7b
*

= t t /
/
x ^ x ^ + T^d t
-^ o J-T q U - T q
1

e-pxFx

dT

(12.1.22)

I f T

= r /
x ( t ) e - * F' d t
-iio \J-To
T h e a c tu a l p o w e r d e n sity sp e c tru m is th e e x p e c te d v a lu e o f PXX( F) in th e lim it as
7b - oo,
r (F ) = lim [ / > ( ) ]
Til- oo
=

(12.1.23)

[ / :;

lim j J I /
To-^oc
a i

x( t ) e j27rFdt

F ro m (12.1.20) an d (12.1.22) w e again n o te th e tw o p o ssib le a p p ro a c h e s to

co m p u tin g PXX(F) , th e d ire c t m e th o d as given by (12.1.22) o r th e in d ire c t m e th o d ,
in w hich w e o b ta in ^ ( r ) first a n d th e n c o m p u te th e F o u rie r tra n sfo rm .
W e sh all c o n s id e r th e e stim a tio n o f th e p o w e r d e n s ity sp e c tru m fro m sa m p le s
o f a sin g le re a liz a tio n o f th e ra n d o m process. In p a rtic u la r, w e assu m e th a t x a ( t )
is sa m p le d a t a ra te Fs > 2 B, w h e re B is th e h ig h est fre q u e n c y c o n ta in e d in th e
p o w e r d e n sity sp e c tru m o f th e ra n d o m p ro cess. T h u s w e o b ta in a fin ite -d u ra tio n
s e q u e n c e x ( n ), 0 < n < N 1, by sa m p lin g x a(t). F ro m th e s e sa m p le s w e can
co m p u te th e tim e -a v e ra g e a u to c o rre la tio n se q u e n c e
j

A '- m - l

r' (m) = ---------N m

x

AA

= T,-----r 7
A/ lm I

Y
x*(n)x{n+m)
n=Q

m = 0 ,1 ,

1
(12.1.24)

N- 1

Jl

x (n)x(n+m)

m = -1 , - 2 ,..., 1 - N

a n d th e n c o m p u te th e F o u rie r tra n sfo rm

A '- l

P^ ( f ) =

E
r'x x { m ) e - M *
m=N+l

(12.1.25)

T h e n o rm a liz a tio n fa c to r N |m | in (12.1.24) re su lts in an e s tim a te w ith m e a n

v a lu e

N- m- l

E [r x' A m )] = r; r~,
N ~

22
to

E[x*(n)x{n + m)]
(12.1.26)

= Yxx(m)
w h e re yxx( m) is th e tr u e (statistical) a u to c o rre la tio n s e q u e n c e o f x ( n) . H e n c e
rxx(m) is a n u n b ia s e d e s tim a te o f th e a u to c o rre la tio n fu n c tio n yxx( m). T h e v a ria n c e

904

Power Spectrum Estimation

Chap. 12

o f th e e stim a te rxx(m) is a p p ro x im a te ly
N
v a r [r i j ( m )] ^

_ |m |]2

22

[\Yx*(n)\2 + y * A n - m ) y xx(n + m) ]

(12.1.27)

w hich is a resu lt given by J e n k in s a n d W a tts (1968). C learly,

lim v a r [r ' (m)] = 0

N oc

(12.1.28)

p ro v id e d th a t
3C
22

\Yxx(n)\2 < oo

n-oc

S ince E[r'xx{m)\ = y xx(m) a n d th e v a ria n c e o f th e e stim a te c o n v e rg e s to z e ro as

N
oc, th e e stim a te r'xx(m) is said to b e consistent.
F o r larg e v alu es o f th e lag p a r a m e te r m, th e e s tim a te r'xx(m) g iv en by (12.1.24)
h as a larg e v arian ce, esp ecially as m a p p ro a c h e s N . T h is is d u e to th e fact th a t
few er d a ta p o in ts e n te r in to th e e stim a te fo r larg e lags. A s an a lte rn a tiv e to
(12.1.24) w e can u se th e e s tim a te
I w - -i
rxx{m) = Y
jt*(n)jt(ii + m)
N t=o
| N-1
^ r(m ) = y
x *( n ) x ( n + m)
N
n=\m\

0 < m < JV 1
(12.1.29)
m = 1. 2 ........ 1 N

w hich h a s a b ias o f \m\yxx( m ) / N , since its m e a n v alu e is

E[rIX(m)] -

A '- m - l

22

E[ x* ( n) x ( n + m)]

n=0
W -_M yjrjr(m)
, , = ^
= _

(12.1.30)
| m |j\ yx A m )

_ _

H o w e v e r, th is e stim a te h a s a sm a ller v arian ce, given a p p ro x im a te ly as

1 1X1
v a r[rJJt(m )] 2 2 [Xr.r(n)|2 + Yx x (n - m ) y xx(n + m )]

(12.1.31)

n~oo

W e o b se rv e th a t rxx(m) is asympt ot i cal l y u n b i a s e d , th a t is,

lim E[rxx{m)] = yxx(m)

H-+x

(12.1.32)

an d its v arian ce c o n v erg es to z e ro as N -* oo. T h e re fo re , th e e s tim a te rxx(m) is

also a consi st ent est imate o f yxx(m).
W e sh all u se th e e stim a te rxx( m ) given by (12.1.29) in o u r tr e a tm e n t o f p o w e r
sp e c tru m e s tim a tio n . T h e c o rre sp o n d in g e s tim a te o f th e p o w e r d e n s ity sp e c tru m is
N -1

/ (/)=

22
m = - ( N - 1)

r* A m ) e - M m

(12.1.33)

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

905

I f w e su b s titu te fo r rxx(m) fro m (I2 .I.2 9 ) in to (12.1.33), th e e s tim a te Pxx( f ) can

also b e e x p re ss e d as
2
Pxx ( / ) -

-jln fn

(12.1.34)

w h e re X ( / ) is th e F o u rie r tra n sfo rm o f th e sam p le se q u e n c e x{n) . T h is w ell kno w n

fo rm o f th e p o w e r d en sity sp e c tru m e s tim a te is called th e periodogram. I t w as o rig
in ally in tro d u c e d by S c h u ste r (1898) to d e te c t a n d m e a s u re h id d e n p e rio d ic itie s
in d a ta .
F ro m (12.1.33), th e av e ra g e v alu e o f th e p e rio d o g ra m e s tim a te PxA f ) is

22

rx A m ) e ,2jrfn

m = - ( N - 1)

22

E [ p xx( f ) ] =

J2

E[rxA m ) ] e - * " '

m= (/V1)

f 1 J r )

m = - ( N - 1) ^

Y xim )e

J2nfm

(12.1.35)

'

T h e in te r p re ta tio n th a t w e give to (12.1.35) is th a t th e m e a n o f th e e stim a te d

sp e c tru m is th e F o u rie r tra n sfo rm o f th e w in d o w ed a u to c o rre la tio n fu n ctio n
Yxx(m)

Y xx(m )

(12.1.36)

w h e re th e w in d o w fu n c tio n is th e (tria n g u la r) B a rtle tt w in d o w . H e n c e th e m e a n

o f th e e s tim a te d sp e c tru m is
E[PxA f ) } =

J 2 YA m ) e - i2nfm
m=oc0

(12.1.37)

1/2
if*

r xx( a ) W B( f - a ) d a
1/2

w h e re W B( f ) is th e sp e c tra l ch a ra c te ristic o f th e B a rtle tt w indow . T h e re la tio n

(12.1.37) illu stra te s th a t th e m e a n o f th e e s tim a te d sp e c tru m is th e co n v o lu tio n o f
th e tr u e p o w e r d en sity sp e c tru m T ( / ) w ith th e F o u rie r tra n sfo rm W B( f ) o f th e
B a rtle tt w in d o w . C o n se q u e n tly , th e m e a n o f th e e s tim a te d s p e c tru m is a sm o o th e d
v e rsio n o f th e tr u e s p e c tru m a n d su ffers fro m th e sa m e sp e c tra l le a k a g e p ro b le m s
w h ich a re d u e to th e fin ite n u m b e r o f d a ta p o in ts.
W e o b s e rv e th a t th e e stim a te d sp e c tru m is asy m p to tically u n b ia s e d , th a t is,
N -1

lim E
N-t-cc

2 2

rx x ( m ) e ' j2nfm

Yxx{m)e-J2* fm = TxA f )

H o w e v e r, in g e n e ra l, th e v a ria n c e o f th e e s tim a te Pxx( f ) d o e s n o t d ec a y to zero

as N - oo. F o r ex a m p le , w h e n th e d a ta se q u e n c e is a G a u s sia n ra n d o m process,

906

Power Spectrum Estimation

Chap. 12

the variance is easily shown to be (see Problem 12.4)

(12.1.38)
w hich , in th e lim it as N > oo, b ec o m e s
lim v a r [ P ( / ) ] = ^ l x ( f )

(12.1.39)

N - + oo

H e n c e w e c o n clu d e th a t th e p e r i o d o g r a m is n o t a consi st ent est imate o f the true

p o w e r density spe ct r um (i.e., it d o e s n o t co n v e rg e to th e tru e p o w e r d en sity sp e c
tru m ).
In su m m ary , th e e s tim a te d a u to c o rre la tio n rxx(m) is a c o n s iste n t e stim a te o f
th e tr u e a u to c o rre la tio n fu n ctio n yxx(m). H o w e v e r, its F o u rie r tra n sfo rm Pxx( f ) ,
th e p e rio d o g ra m , is n o t a c o n siste n t e s tim a te o f th e tru e p o w e r d e n sity sp e ctru m .
W e o b se rv e d th a t Pxx( f ) is an a s y m p to tically u n b ia s e d e stim a te o f Vxx{ f ) , b u t fo r
a fin ite -d u ra tio n se q u en ce, th e m e a n v alu e o f Pxx{ f ) c o n ta in s a bias, w hich from
(12.1.37) is e v id e n t as a d isto rtio n o f th e tru e p o w e r d en sity sp e c tru m . T h u s th e
e s tim a te d sp e c tru m su ffers fro m th e sm o o th in g effe c ts a n d th e le a k a g e e m b o d ie d
in th e B a rtle tt w ind o w . T h e sm o o th in g an d le a k a g e u ltim ately lim it o u r a b ility to
reso lv e closely sp a ced sp e ctra.
T h e p ro b le m s o f leak ag e an d fre q u e n c y re so lu tio n th a t w e h ave ju st d e
sc rib ed as w ell as th e p ro b le m th a t th e p e rio d o g ra m is n o t a c o n s iste n t estim a te
o f th e p o w e r sp e c tru m , p ro v id e th e m o tiv a tio n fo r th e p o w e r sp e c tru m e s tim a
tio n m e th o d s d e sc rib e d in S ectio n s 12.2, 12.3, a n d 12.4. T h e m e th o d s d escrib ed
in S ectio n 12.2 a re classical n o n p a ra m e tric m e th o d s, w hich m a k e n o assu m p tio n s
a b o u t th e d a ta se q u en ce. T h e em p h asis o f th e classical m e th o d s is on o b ta in in g a
c o n s iste n t e stim a te o f th e p o w e r sp e c tru m th ro u g h so m e av e ra g in g o r sm o o th in g
o p e ra tio n s p e rfo rm e d d irectly o n th e p e rio d o g ra m o r o n th e a u to c o rre la tio n . A s
w e will see, th e effe ct o f th e se o p e ra tio n s is to re d u c e th e fre q u e n c y re so lu tio n
fu rth e r, w hile th e v a ria n c e o f th e e s tim a te is d e c re a se d .
T h e sp e c tru m e stim a tio n m e th o d s d e sc rib e d in S ectio n 12.3 a re b a se d on
so m e m o d e l o f h o w th e d a ta w ere g e n e ra te d . In g e n e ra l, th e m o d e l-b a se d m e th o d s
th a t h av e b een d e v e lo p e d o v e r th e p a s t tw o d e c a d e s p ro v id e significantly h ig h e r
re so lu tio n th a n d o th e classical m e th o d s.
A d d itio n a l m e th o d s a re d e sc rib e d in S ectio n s 12.4 a n d 12.5. O n e o f th ese
m e th o d s, d u e to C a p o n (1969), is b a s e d o n m in im iz in g th e v a ria n c e in th e sp e c
tra l e stim a te . T h e m e th o d s d e sc rib e d in S ectio n 12.5 a re b a s e d on an e ig e n
v a lu e /e ig e n v e c to r d e c o m p o sitio n o f th e d a ta c o rre la tio n m atrix .

12.1.3 The Use of the DFT in Power Spectrum Estimation

A s giv en b y (12.1.14) a n d (12.1.34), th e e s tim a te d en e rg y d e n s ity sp e c tru m , , ( / )
a n d th e p e rio d o g ra m Pxx( f ), resp ec tiv ely , can b e c o m p u te d b y u se o f th e D F T ,
w hich in tu rn is efficiently c o m p u te d by a F F T a lg o rith m . I f w e h a v e N d a ta p o in ts,

Sec. 12.1

Estimation of Spectra from Finite-Duration Observations of Signals

907

w e c o m p u te as a m in im u m th e A ppoint D F T . F o r ex am p le, th e c o m p u ta tio n yields

sa m p le s o f th e p e rio d o g ra m
* = 0 ,1 ,..., JV -1

(12.1.40)

a t th e fre q u e n c ie s /* = k / N .
In p ra c tic e , h o w ev er, such a sp a rs e sa m p lin g o f th e sp e c tru m d o e s n o t p ro v id e
a v ery g o o d re p re s e n ta tio n o r a g o o d p ic tu re o f th e c o n tin u o u s sp e c tru m estim ate
* > ( /). T h is is easily re m e d ie d b y e v a lu a tin g Pxx( f ) a t a d d itio n a l freq u e n cies.
E q u iv a le n tly , w e can effectiv ely in c re a se th e le n g th o f th e se q u e n c e by m e a n s o f
z e ro p a d d in g a n d th e n e v a lu a te Pxx{ f ) at a m o re d en se se t o f freq u e n cies. T h u s
if w e in crease th e d a ta se q u e n c e le n g th to L p o in ts by m e a n s o f z e ro p a d d in g an d
e v a lu a te th e L -p o in t D F T , we h av e
(12.1.41)
W e em p h a siz e th a t z e ro p a d d in g an d e v a lu a tin g th e D F T at L >
d o e s n o t im p ro v e th e fre q u e n c y re so lu tio n in th e sp e c tra l estim a te .
p ro v id e s us w ith a m e th o d fo r in te rp o la tin g th e valu es o f th e m e a s u re d
a t m o re fre q u e n c ie s. T h e fre q u e n c y re so lu tio n in th e s p e c tra l e stim a te
d e te rm in e d by th e len g th N o f th e d a ta reco rd .

N p o in ts
It sim ply
sp e ctru m
PXI{.f) is

Exam ple 12.1.2

A sequence of N = 16 samples is obtained by sampling an analog signal consisting of
two frequency components. The resulting discrete-time sequence is
x( n) = sin2jr(0.135)w + cos2;r(0.135 + Af ) n

n = 0 , 1 , . . . , 15

where A/ is the frequency separation. Evaluate the power spectrum P ( f ) =

(1/7V )|X (/)|2 at the frequencies f k = k/ L, k = 0, 1 ,__ L 1, for L = 8, 16, 32,
and 128 for values of A/ = 0.06 and A/ = 0.01.
Solution By zero padding, we increase the data sequence to obtain the power spec
trum estim ate PXI(k/L). The results for A/ = 0.06 are plotted in Fig. 12.4. Note that
zero padding does not change the resolution, but it does have the effect of interpo
lating the spectrum Pxl ( f ) . In this case the frequency separation A/ is sufficiently
large so that the two frequency components are resolvable.
The spectral estimates for A/ = 0.01 are shown in Fig. 12.5. In this case the
two spectral com ponents are not resolvable. Again, the effect of zero padding is to
provide more interpolation, thus giving us a better picture of the estim ated spectrum.
It does not improve the frequency resolution.
W h e n o n ly a few p o in ts o f th e p e rio d o g ra m a re n e e d e d , th e G o e rtz e l alg o
rith m d e s c rib e d in C h a p te r 6 m ay p ro v id e a m o re efficient c o m p u ta tio n . Since th e
G o e rtz e l a lg o rith m h a s b e e n in te r p re te d as a lin e a r filterin g a p p ro a c h to c o m p u t
in g th e D F T , it is c le a r th a t th e p e rio d o g ra m e s tim a te can b e o b ta in e d by passing

908

Power Spectrum Estimation

Chap. 12

_S___ B___ S_

L I ..

T
32

^mTTTTnK/f

I^WrffTllnaa
128

Figure 12.4

Spectra of two sinusoids with frequency separation A f 0.06.

th e signal th ro u g h a b a n k o f p a ra lle l tu n e d filters a n d sq u a rin g th e ir o u tp u ts (see

P ro b le m 12.5).

12.2 NONPARAMETRIC METHODS FOR POWER SPECTRUM

ESTIMATION
T h e p o w e r sp e c tru m e stim a tio n m e th o d s d e sc rib e d in th is se c tio n a re th e classical
m e th o d s d e v e lo p e d by B a rtle tt (1948), B la c k m a n a n d T u k e y (1958), an d W elch
(1967). T h e se m e th o d s m a k e n o a ssu m p tio n a b o u t h o w th e d a ta w ere g e n e ra te d
a n d h e n c e a re called nonparamet ri c.

Sec. 12.2

Nonparametric Methods for Power Spectrum Estimation

F igure 1X5

909

Spectra of two sinusoids with frequency separation A/ = 0.01.

S ince th e e stim a te s a re b ased e n tire ly on a fin ite re c o rd o f d a ta , th e fre q u e n c y

re so lu tio n o f th e s e m e th o d s is, at b est, e q u a l to th e sp e c tra l w id th o f th e re c ta n g u la r
w in d o w o f le n g th N , w hich is a p p ro x im a te ly 1( N a t th e - 3 - d B p o in ts. W e shall b e
m o re p recise in sp ecify in g th e fre q u e n c y re so lu tio n o f th e specific m e th o d s. A ll th e
e s tim a tio n te c h n iq u e s d e sc rib e d in th is section d e c re a s e th e fre q u e n c y re so lu tio n
in o r d e r to re d u c e th e v a rian ce in th e sp e c tra l e stim a te .
F irst, w e d e sc rib e th e e stim ates an d d e riv e th e m e a n a n d v aria n c e o f e ach . A
c o m p a riso n o f th e th re e m e th o d s is given in S ectio n 12.2.4. A lth o u g h th e sp e ctral
e stim a te s a re e x p ressed a s a fu n ctio n o f th e c o n tin u o u s fre q u e n c y v a ria b le / , in
p ra c tic e , th e e stim a te s a re c o m p u te d a t d isc rete fre q u e n c ie s via th e F F T alg o rith m .
T h e F F T -b a se d c o m p u ta tio n a l re q u ire m e n ts a re c o n s id e re d in S ectio n 12.2.5.

910

Power Spectrum Estimation

Chap. 12

12.2.1 The Bartlett Method: Averaging Periodograms

B a r tle tts m e th o d fo r re d u c in g th e v arian ce in th e p e rio d o g ra m involves th re e
step s. F irst, th e N -p o in t se q u e n c e is su b d iv id e d in to K n o n o v e rla p p in g se g m en ts,
w h e re ea c h se g m e n t h as le n g th M, T h is re su lts in th e K d a ta se g m e n ts
x,(n) = x( n + i M )

i = 0, I, . . . . K 1

(12.2.1)

n = 0 , 1 .........M - l
F o r ea c h se g m en t, w e c o m p u te th e p e rio d o g ra m
p i 'v
"

M-l
= _L E * . '< n ) e - J2nfn\
M

i= 0 ,l,...,J f - l

(12.2.2)

F in ally , w e av erag e th e p e rio d o g ra m s fo r th e K se g m e n ts to o b ta in th e B a rtle tt

p o w e r sp e c tru m e stim a te [B a rtle tt (1948)]
1 * -1
p xBA f ^ = T Z y ( / )
K

( 12-2 -3 >

,=v>

T h e statistical p ro p e rtie s of this e s tim a te a re easily o b ta in e d . F irst, th e m ean

v alu e is
(1 2 2 4 )
= [* (/)]
F ro m (12.1.35) an d (12.1.37) w e h ave th e e x p e c te d v a lu e fo r th e single p e rio d o g ra m
as
Af-1

E f1-

y ^ e~s2nfm
(12.2.5)

M J _-i]/2
r-

s in ;r ( / - a) J

w h ere
1 / s in jr/M V
B(f) = t ;
:
M V s ni f f / /

. _ .
( 12. 2. 6)

is th e fre q u e n c y c h aracteristics o f th e B a rtle tt w indow

m,
w B(n) =

] ------ ,

0,

|m|

< M 1

11 ~
"
o th e rw ise

(12.2.7)

F ro m (12.2.5) w e o b se rv e th a t th e tru e sp e c tru m is no w c o n v o lv e d w ith th e

fre q u e n c y c h a ra c te ristic W a i f ) o f th e B a rtle tt w indow . T h e e ffe c t o f re d u c in g th e
le n g th o f th e d a ta fro m N p o in ts to M = N / K re su lts in a w in d o w w hose sp e c tra l

Sec. 12.2

Nonparametric Methods for Power Spectrum Estimation

911

w id th h a s b e e n in c re a se d b y a fa c to r o f K . C o n s e q u e n tly , th e fre q u e n c y re so lu tio n

h as b e e n re d u c e d b y a fa c to r K.
In r e tu rn fo r th is re d u c tio n in re so lu tio n , w e h a v e r e d u c e d th e v arian ce. T h e
v a ria n c e o f th e B a rtle tt e s tim a te is
var[ / ( / ) ] =

K2 I=U

( 12.2 .8 )

= ^ v a r [/> (/)]
If w e m a k e u se o f (12.1.38) in (12.2.8), w e o b ta in
v ar [ / > * ( / ) ] = j T l x { f )

1+

/ sin I n f M ' V
\M s in 2 ;r/ J

(12.2.9)

T h e re fo re , th e v a rian ce o f th e B a rtle tt p o w e r sp e c tru m e s tim a te h a s b e e n re d u c e d

by th e fa c to r K.

12.2.2 The Welch Method: Averaging Modified

Periodograms
W elch (1967) m ad e tw o b asic m o d ificatio n s to th e B a rtle tt m e th o d . F irst, h e a l
lo w ed th e d a ta se g m en ts to o v erlap . T h u s th e d a ta se g m e n ts c a n b e r e p re s e n te d as
Xj ( n) = x( n + i D )

n = 0 ,1 , . . . , M 1

(12.2.10)

i = 0 . 1 .........L - l
w h e re i D is th e sta rtin g p o in t fo r th e /th se q u e n c e . O b s e rv e th a t if D = M ,
th e se g m en ts d o n o t o v e rla p an d th e n u m b e r L o f d a ta se g m e n ts is id e n tic a l to
th e n u m b e r K in th e B a rtle tt m e th o d . H o w e v e r, if D = M /2 , th e r e is 50%
o v e rla p b e tw e e n su ccessive d a ta se g m e n ts a n d L = 2 K se g m e n ts a re o b ta in e d .
A lte rn a tiv e ly , w e can fo rm K d a ta se g m en ts ea c h o f le n g th 2 M .
T h e se c o n d m o d ifica tio n m a d e by W elch to th e B a rtle tt m e th o d is to w indow
th e d a ta se g m e n ts p rio r to c o m p u tin g th e p e rio d o g ra m . T h e re su lt is a m o d ifie d
p e rio d o g ra m
1
* * (/) =

~MU n=0

( 12.2. 11)

w h e re U is a n o rm a liz a tio n fa c to r f o r th e p o w e r in th e w in d o w fu n c tio n a n d is

se le c te d as
1 M-l
u /(n )
( 12. 2 . 12)
T h e W elch p o w e r sp e c tru m e stim a te is th e a v e ra g e o f th e se m o d ified p e rio d o g ra m s,
th a t is,
1 L~x
^ ( /) =
(12.2.13)

912

Power Spectrum Estimation

Chap. 12

The mean value of the Welch estimate is

(12.2.14)

^ . =o
= * ( /)]
B u t th e e x p e c te d v alu e o f th e m od ified p e rio d o g ra m is
i

M -l M-\

[ ^ ( / ) ] = T777 E E '( " ) w ( * ) k ( " K ( ) ] < A T /l* ' " )

=0 m=0

(12.2.15)

M - l Af-1

Since
r ia

Vjcx(n) = j

r xx(a)el27!anda

(12.2.16)

J - 1/2
s u b s titu tio n fo r y ^ n ) fro m (12.2.16) in to (12.2.15) yields

*?</>]=

1/2

M - 1 M -l

j- 1/2l r w

n = 0 m =0

da

(12.2.17)

1 /2

f
= /
r (a )W (/-a )rfa
'-1/2
w h e re , by d efin itio n ,

3 # 17

E u>(n)e
n=0

-jZnfn

(12.2.18)

T h e n o rm a liz a tio n fa c to r U e n s u re s th a t

1/2

W (/W / = 1

(12.2.19)

J 1/2

T h e v arian ce o f th e W elch e s tim a te is

v a r l O / ) ] = Tr E

^ ~ ( f ) P g ( f ) ] ~ {E[P?Af)])2

(12.2 .20)

In th e case o f n o o v e rla p b e tw e e n successive d a ta se g m e n ts ( L = K ) , W elch has

sh o w n th a t
v a r |P * ( / ) J = i v a r [ P ' ( / ) l

* zr(/)

(12.2.21)

Sec. 12.2

Nonparametric Methods for Power Spectrum Estimation

913

In th e case o f 5 0% o v e rla p b e tw e e n successive d a ta se g m en ts (L = 2 AT),

th e v a ria n c e o f th e W elch p o w e r sp e c tru m e s tim a te w ith th e B a rtle tt (tria n g u la r)
w indow , also d e riv e d in th e p a p e r by W elch, is
v a r [ / > ( / ) ] * ^ r * T( / )

(12.2.22)

A lth o u g h w e c o n s id e re d only th e tria n g u la r w in d o w in th e c o m p u ta tio n o f

th e v a rian ce, o th e r w in d o w fu n c tio n s m ay be used. In g e n e ra l, th ey will yield
a d iffe re n t v arian ce. In ad d itio n , o n e m ay v ary th e d a ta se g m e n t o v e rla p p in g
b y e ith e r m o re o r less th a n th e 50% co n sid e re d in th is se c tio n in a n a tte m p t to
im p ro v e th e re le v a n t c h a ra c te ristic s o f th e estim ate.

12.2.3 The Blackman and Tukey Method: Smoothing the

Periodogram
B lac k m a n a n d T u k e y (1958) p ro p o se d an d an aly zed th e m e th o d in w hich th e
sa m p le a u to c o rre la tio n se q u e n c e is w in d o w ed first a n d th e n F o u rie r tra n sfo rm e d
to yield th e e s tim a te o f th e p o w e r sp e ctru m . T h e ra tio n a le fo r w in d o w in g th e
estim a te d a u to c o rre la tio n se q u e n c e rxx(m) is th a t, fo r larg e lags, th e e s tim a te s
are less re lia b le b ecau se a sm a ller n u m b e r (N m) o f d a ta p o in ts e n te r in to th e
estim ate. F o r v alu es o f m a p p ro a c h in g N , th e v a rian ce o f th e s e e stim a te s is very
h igh, an d h en c e th e se e s tim a te s sh o u ld be given a sm a lle r w eig h t in th e fo rm a tio n
o f th e e stim a te d p o w e r sp e c tru m . T h u s th e B la c k m a n -T u k e y e s tim a te is
A f-1

E
r , A m ) w { m ) e - J2*fm
m=(M1)

(12.2.23)

w h e re th e w in d o w fu n c tio n w( n) h as length 2 M 1 a n d is z e ro fo r \m\ > M .

W ith th is d e fin itio n fo r w( n) , th e lim its o n th e sum in (12.2.23) can b e e x te n d e d to
( oo, oo). H e n c e th e fre q u e n c y -d o m a in e q u iv a le n t ex p re ssio n fo r (12.2.23) is th e
c o n v o lu tio n in teg ral
yl/2

* * //(/)= /

J- 1/2

P*A<x)W{f-a)da

(12.2.24)

w h e re Pxx( f ) is th e p e rio d o g ra m . It is clear from (12.2.24) th a t th e effe ct o f w in

d o w in g th e a u to c o r re la tio n is to sm o o th th e p e rio d o g ra m e s tim a te , th u s d e creasin g
th e v a rian ce in th e e s tim a te a t th e e x p en se o f re d u c in g th e re so lu tio n .
T h e w in d o w s e q u e n c e w( n) sh o u ld be sy m m e tric (e v e n ) a b o u t m = 0 to
e n s u re th a t th e e stim a te o f th e p o w e r sp e c tru m is re a l. F u rth e rm o r e , it is d e sirab le
to se lect th e w in d o w sp e c tru m to b e n o n n eg ativ e, th a t is,
W (f) > 0

| / | < 1 /2

(12.2.25)

T h is c o n d itio n e n s u re s th a t PXJ ( f ) > 0 fo r | / | < 1/2, w hich is a d e s ira b le p ro p e rty

fo r an y p o w e r sp e c tru m e s tim a te . W e sh o u ld in d ic a te , h o w ev er, th a t so m e o f th e
w in d o w fu n c tio n s w e h a v e in tro d u c e d d o n o t satisfy th is c o n d itio n . F o r exam p le,

914

Power Spectrum Estimation

Chap. 12

in sp ite o f th e ir low sid e lo b e levels, th e H a m m in g a n d H a n n ( o r H a n n in g ) w indow s

d o n o t satisfy th e p ro p e rty in (12.2.25) a n d , c o n s e q u e n tly , m ay re su lt in n eg ativ e
sp e c tru m e stim a te s in so m e p a rts o f th e fre q u e n c y ran g e.
T h e e x p e c te d v alu e o f th e B la c k m a n -T u k e y p o w e r s p e c tru m e s tim a te is

, 1/2
[* * //( /) ]= /
E [ P xA a ) ] W ( f - a ) d a
J - 1/2

(12.2.26)

w h e re fro m (12.1.37) w e h av e
,1/2
[/> ( )]= /
r xA 0 ) W B( a - 0 ) d e
J -1/2
a n d WB{ f ) is th e F o u rie r tra n sfo rm o f th e B a rtle tt w in d o w .
(12.2.27) in to (12.2.26) yields th e d o u b le c o n v o lu tio n in teg ral

(12.2.27)
S u b stitu tio n of

r 1/2 ,1/2
E[P?rT ( / ) ] = /
/
r ( e ) W JJ( a - 0 ) W ( / - a ) r f a r f e
/1/2 J \[2

(12.2.28)

E q u iv a le n tly , by w o rk in g in th e tim e d o m a in , th e e x p e c te d v alu e o f th e

B la c k m a n -T u k e y p o w e r sp e c tru m e stim a te is
M- 1

22

=
m

E [ r * A m ) ] w ( m ) e - J 27lfm

(iWll
(12.2.29)

Af-1

22

y , A m ) w B( m ) w ( m ) e - j 2nfm

m=(M-l)
w h ere th e B a rtle tt w in d ow is

wB(m)

0,

o th e rw ise

C learly , w e sh o u ld select th e w in d o w le n g th fo r w( n) such th a t M << N , th a t

is, w( n) sh o u ld b e n a rro w e r th a n w B(m) to p ro v id e a d d itio n a l sm o o th in g o f the
p e rio d o g ra m . U n d e r th is c o n d itio n , (12.2.28) b e c o m e s
, 1/2
E lP f/if)}*

VxA 0 ) W { f - d ) d d

(12.2.31)

J - 1/2
since

1/2

J - 1/2

1/2

WB(a - d ) W { f - a ) d a =

W B( a ) W ( f - 9 - a)doc

J - 1/2

(12.2.32)

^ w (f-e)
T h e v arian ce o f th e B la c k m a n -T u k e y p o w e r sp e c tru m e s tim a te is
v a r [ / > / / ( / ) ] = { [ P / / ( / ) 2]} - { [ / > / / ( / ) ] } 2

(12.2.33)

Sec. 12.2

Nonparametric Methods for Power Spectrum Estimation

w h e re th e m e a n can b e a p p ro x im a te d as in (12.2.31).
(11.2.33) is

915

T h e se c o n d m o m e n t in

fl/2

rl/2

E{[PxBxT ( f ) ] 2) = /

E [ P xx( a ) P xx( . e ) ] W( f

-6)dade

(12.2.34)

J - 1/2 J-\f2
O n th e a s su m p tio n th a t th e ra n d o m p ro c e ss is G a u ssia n (see P ro b le m 12.5), w e
find th a t

r (a )r (0) 1 +

E [ P xA a ) P xA&)] =

sin7r(0 + a ) N
_Ns i n7 r ( 8 + a)_

sin7r(0 a ) N
N s i n n ( 6 a) _
(12.2.35)

S u b stitu tio n o f (12.2.35) in to (12.2.34) yields

-1/2

-.2

r xA 0 ) W ( f - e ) d e

E { [ P ? / ( f ) ] 2} =
- 1/2

,1 [2

+ /

1/2

r xA<*)rxA e ) W ( f - a ) W ( f - d )

J - 1/2 J -l/2

s in jr(# -l-oON

si nn( & a ) N

N s in jr(0 + a)_

N sin7r(8 or)

d a d$

(12.2.36)

T h e first te rm in (12.2.36) is sim ply th e sq u a re o f th e m e a n o f P f J ( / ) , w hich

is to b e s u b tra c te d o u t a c co rd in g to (12.2.33). T h is le a v e s th e se c o n d te rm in
(12.2.36), w h ich c o n s titu te s th e v arian ce. F o r th e case in w hich N Af, th e
fu n c tio n s s i n n ( 9 + a ) N / N s i n ^ ( 0 + a ) a n d s m
n ( 8 a ) N / N sin jr ( 0 a ) a re relativ ely
n a rro w c o m p a re d to W ( f ) in th e vicinity o f 0 = a a n d 9 = a , resp ec tiv ely .
T h e re fo re ,

y _ i /2

{[L N s m n ( 8

-a)N

+ a;)J

r
[ _ ^ sin 7 r(0 ~a j

de
(12.2.37)

% r xA - a ) W ( f + c c ) + r xA<x)W( f - a)

N
W ith th is a p p ro x im a tio n , th e v a ria n c e o f PXJ i f ) b ec o m e s

Cxn

ar[ / " ( / ) ] -

" J - 1/2
1/2

r ( a ) W ( / - a ) [ r ( - a ) W ( / + a) + r (o )W (/-o ()]rfa

r lfl

J i L 1/2 rZ^
-

w^ f ~ a)da

(12.2.38)

w h e re in th e last s te p w e m a d e th e a p p ro x im a tio n

1/2

r z, ( a ) r ( - c r ) V ( / - a ) W ( f + a ) d a 0
1/2

(12.2.39)

Power Spectrum Estimation

916

Chap. 12

W e shall m a k e o n e a d d itio n a l a p p ro x im a tio n in (12.2.38). W h e n W ( f ) is

n a rro w c o m p a re d to th e tru e p o w e r sp e c tru m r ( / ) , (12.2.38) is f u rth e r a p p ro x
im a te d as
v a r[/> //(/)] *

r2
xx(f)

f '* W2(0)d0j
1/2

(12.2.40)

M-l
H A f)

w 2( m )

(Af1)

12.2.4 Performance Characteristics of Nonparametric

Power Spectrum Estimators
In th is sectio n w e co m p a re th e q u ality o f th e B a rtle tt, W elch, a n d B la c k m a n an d
T u k e y p o w e r sp e c tru m estim ates. A s a m e a s u re o f q u ality , w e u se th e ra tio o f its
v arian ce to th e s q u a re o f th e m e a n o f th e p o w e r sp e c tru m e s tim a te th a t is,
Q

iE[P?Af)}}2

(12.2.41)

t>ar [ P ( f ) ]
w h e re A = B , W , o r B T fo r th e th re e p o w e r s p e c tru m estim ates. T h e recip ro c al o f
th is q u a n tity , called th e variability, can also be u se d as a m e a s u re o f p e rfo rm a n c e .
F o r re fe re n c e , th e p e rio d o g ra m has a m e a n a n d v arian ce

E[P*Af)] = f r(0)WW - 8)de

(12.2.42)

J- l / 2

v a r [/> ( /) ]= r ,V /)[ l + ( | ^
) 2'
\N sin2nf/

(12.2.43)

w h ere
1 / sin n f N \ 2
W B( f ) = ~
~
Lr
N V sin n f J

(12.2.44)

i-l/2
E [ P * i ( f ) ] -+

r (/) /

w B( d ) d e =

u>,(0) r ( / ) = r ( / )
(12.2.45)

J - 1/2

var[/>(/)] - r l x( f )
H e n c e , as in d ic a te d p rev io u sly , th e p e rio d o g ra m is an asy m p to tically u n b ia se d
e stim a te o f th e p o w e r sp e c tru m , b u t it is n o t c o n siste n t becau se its v arian ce d o es
n o t a p p ro a c h z e ro as N in c re a se s to w a rd infinity.
A sy m p to tically , th e p e rio d o g ra m is c h a ra c te riz e d by th e q u a lity fa c to r
r\

Qp

2 (f )}
_ r xx^J

= z.1-,

= 1

(12.2.46)

rlA f)
T h e fact th a t Qp is fixed a n d in d e p e n d e n t o f th e d a ta le n g th N is a n o th e r in d ic a tio n
o f th e p o o r q u ality o f th is e stim a te .

Sec. 12.2

Nonparametric Methods for Power Spectrum Estimation

Bartlett power spectrum estimate.

917

T h e m e a n a n d v a ria n c e o f th e B a rtle tt

p o w e r s p e c tru m e s tim a te a re
E[P?Af)]=

J-

(12.2.47)

r xxm w B( f - 6) dd

1/2

/ s in 2 jr /A f V
v a r[/ (/)] = - r 1 ( f )

^ \ M sin 2 n f )

(12.2.48)

an d
1 / sin n f M \ 2
W B( f ) =
. J .
M \ s in ^ / /

(12.2.49)

A s N y oo a n d M y oo, w hile K = N / M rem ain s fixed, w e find th a t

r 1/2

[ * * < /) ]

r ( /)

w ,(/)r f/

= r xx( f ) w B(0) =

r (/)

7 - -1/2
1 /2

(12.2.50)

v a r [P /x( / ) ]

rL (/)

W e o b se rv e th a t th e B a rtle tt p o w e r sp e c tru m e s tim a te is asy m p to tically u n b i

ased a n d if K is allo w ed to in crease w ith an in crease in N , th e e s tim a te is also c o n
sisten t. H e n c e , asy m p to tically , th is e stim a te is c h a ra c te riz e d b y th e q u a lity fa c to r
N
Qb = K =
(12.2.51)
M
T h e fre q u e n c y re so lu tio n o f th e B a rtle tt e s tim a te , m e a s u re d by tak in g th e
3-dB w id th o f th e m ain lo b e o f th e re c ta n g u la r w indow , is
0.9
A/ =
J
M
H e n c e , M = 0 .9 /A / a n d th e q u ality fa c to r b eco m es
N
Qb =

0 .9 /A /

Welch power spectrum estimate.

p o w e r sp e c tru m e stim a te are

= l.IN A f

(12.2.52)

(12.2.53)

T h e m e a n a n d v a ria n c e o f th e W elch

1/2

[ * ( / ) ] =

f r^mw(f-B)de

(12.2.54)

2 2 w ( n ) e - j2*f *

(12.2.55)

w h ere

and
fo r no o v e rla p

v a r [ j ( /)] =

9 ,
r 2 (f),

8L xx

fo r 50% o v e rla p a n d
tria n g u la r w indow

(12.2.56)

918

As N

Power Spectrum Estimation

Chap. 12

oo an d M - * oo, th e m e a n co n v e rg e s to
[* (/> ]-> r ( / )

(12.2.57)

an d th e v arian ce c o n v erg es to z e ro , so th a t th e e s tim a te is c o n s iste n t.

U n d e r th e tw o co n d itio n s given by (12.2.56) th e q u ality fa c to r is
N
L = ,
Qw =

fo r n o o v e rla p

S L = 16N

fo r 5 0 % o v e rla p a n d

9 9M '

(12.2.58)

tria n g u la r w indow

O n th e o th e r h a n d , th e sp e ctral w idth o f th e tr ia n g u la r w indow a t th e 3-dB p o in ts is

A/ =
M

(12.2.59)

C o n se q u e n tly , th e q u ality fa c to r e x p re ss e d in te rm s o f A f a n d N is

0.78N A /,

fo r n o o v e rla p

1 39NAf

fo r ^ 0/ o v e rla P a n d
tr ia n g u la r w indow

Blackman-Tukey power spectrum estimate.

(12.2.60)

T h e m e a n a n d v a ria n c e o f

th is e s tim a te a re a p p ro x im a te d as
- 1/2

E[Pf / ( f ) ] * f

r xxm w ( f - d ) d 8

J --M
1/2l

- r
1

v a r[P //(/)] *

r;x(f)
L

(12.2.61)

>71
w (m)

m=-(W -1)

w h e re u>(m) is th e w in d ow se q u e n c e u se d to ta p e r th e e s tim a te d a u to c o rre la tio n

se q u e n c e . F o r th e re c ta n g u la r an d B a rtle tt (tria n g u la r) w in d o w s w e h av e
1
N

r * '1

2/ s
(2 M/N,
< ")= { 2U/3N.

re c ta n g u la r w in d o w
tr ia n g u la r w in d o w

^
(12'2 '62)

I t is c le a r fro m (12.2.61) th a t th e m e a n v a lu e o f th e e s tim a te is asym ptotically

u n b ia se d . Its q u a lity fa c to r fo r th e tr ia n g u la r w in d o w is
Q bt = 1 - 5 ^
M

(12.2.63)

S ince th e w in d o w le n g th is 2M 1, th e fre q u e n c y re so lu tio n m e a s u re d a t th e 3-dB

p o in ts is
A/ = ^
=
2M
M

(12.2.64)

and hence
Q b t = x r i N & f = 2 .3 4 W A /
U.64

(12.2.65)

Sec. 12.2

N onparametric Methods for Power Spectrum Estimation

919

TABLE 12.1 QUALITY OF POWER

SPECTRUM ESTIMATES
Estimate

Quality Factor

Bartlett

l.UNAf

Welch
(50% overlap)
Blackman-Tukey

1.39NA/

2.34AfA/

T h e se re su lts a re su m m a riz e d in T a b le 12.1. It is a p p a re n t fro m th e resu lts

w e h av e o b ta in e d th a t th e W elch a n d B la c k m a n -T u k e y p o w e r sp e c tru m e stim a te s
a re so m e w h a t b e tte r th a n th e B a rtle tt e stim a te . H o w e v e r, th e d iffe re n c e s in p e r
fo rm a n c e a re relativ ely sm all. T h e m ain p o in t is th a t th e q u a lity fa c to r in crease s
w ith an in c re a se in th e le n g th N o f th e d a ta . T h is c h a ra c te ristic b e h a v io r is n o t
s h a re d by th e p e rio d o g ra m estim a te . F u rth e rm o re , th e q u a lity fa c to r d e p e n d s on
th e p ro d u c t o f th e d a ta len g th N a n d th e fre q u e n c y re so lu tio n A f . F o r a d e
sire d level o f q u ality , A f can b e d e c re a s e d (fre q u e n c y re so lu tio n in c re a se d ) by
in creasin g th e len g th N o f th e d a ta , a n d vice versa.

12.2.5 Computational Requirements of Nonparametric

Power Spectrum Estimates
T h e o th e r im p o rta n t a s p e c t o f th e n o n p a ra m e tric p o w e r sp e c tru m e stim a te s is
th e ir c o m p u ta tio n a l re q u ire m e n ts . F o r th is c o m p a riso n w e assu m e th e e stim a te s
a re b a s e d o n a fixed a m o u n t o f d a ta N a n d a specified re so lu tio n A f . T h e radix2 F F T a lg o rith m is a ssu m ed in all th e c o m p u ta tio n s. W e shall c o u n t only th e
n u m b e r o f c o m p lex m u ltip lic a tio n s re q u ire d to c o m p u te th e p o w e r sp e c tru m e sti
m a te .

Bartlett power spectrum estimate

F F T len g th = M = 0 .9 / A /
N u m b e r o f F F T s = = 1.11W A f
M
J
N u m b e r o f c o m p u ta tio n s =

log2 M j ~ log2

Welch power spectrum estimate (50% overlap)

F F T len g th = M 1 .2 8 /A /
N u m b e r o f F F T s = = 1. 56N A f
M

Number of computations = I( y' olog2

feM
*)

920

Power Spectrum Estimation

Chap. 12

In a d d itio n to th e 2 N /Af F F T s, th e r e a re a d d itio n a l m u ltip lic a tio n s re q u ire d

fo r w in d o w in g th e d a ta . E a c h d a ta re c o rd re q u ire s M m u ltip lic atio n s. T h e re fo re ,
th e to ta l n u m b e r o f c o m p u ta tio n s is
1 28
5 12
T o ta l c o m p u ta tio n s = 2 N + N log2 = N lo g ,
Af
- Af

Blackman-Tukey power spectrum estimate. In th e B ia c k m a n -T u k e y

m e th o d , th e a u to c o rre la tio n rxx{m) can b e c o m p u te d efficiently via th e F F T a l
g o rith m . H o w e v e r, if th e n u m b e r o f d a ta p o in ts is larg e, it m ay n o t b e p o ssib le to
c o m p u te o n e A p p oint D F T . F o r ex a m p le , we m ay h ave N = 105 d a ta p o in ts b ut
o nly th e cap ac ity to p e rfo rm 1024-point D F T s. S ince th e a u to c o rre la tio n se q u e n c e
is w in d o w ed to 2 M 1 p o in ts w h e re M <: N , it is p o ssible to c o m p u te th e d esired
2 M 1 p o in ts o f rxx(m) by se g m e n tin g th e d a ta in to K = N f l M re c o rd s a n d th e n
c o m p u tin g 2 A /-p o in t D F T s a n d o n e 2 M -p o in t I D F T via th e F F T alg o rith m . R a d e r
(1970) h as d e sc rib e d a m e th o d fo r p e rfo rm in g th is c o m p u ta tio n (se e P ro b le m 12.7).
If w e b ase th e c o m p u ta tio n a l co m p lex ity o f th e B la c k m a n -T u k e y m e th o d on
th is a p p ro a c h , we o b ta in th e follow ing c o m p u ta tio n a l re q u ire m e n ts .
F F T len g th = 2 M = 1 .2 8 /A /
/ N \
N
N um ber o f FFT s = 2 K + 1 = 2 { ) + 1 ^
\2M J
M
N u m b e r o f c o m p u ta tio n s =

log2 2 M ) = N log2

W e can neg lect th e a d d itio n a l M m u ltip lic a tio n s re q u ire d to w in d o w th e a u to c o r

re la tio n se q u e n c e rxx(m), since it is a relativ ely sm all n u m b e r. F in ally , th e re is th e
a d d itio n a l c o m p u ta tio n re q u ire d to p e rfo rm th e F o u rie r tra n s fo rm o f th e w in d o w ed
a u to c o rre la tio n se q u en ce. T h e F F T a lg o rith m can b e used fo r th is co m p u ta tio n
w ith so m e zero p a d d in g fo r p u rp o se s o f in te rp o la tin g th e s p e c tra l e stim a te . A s a
re su lt o f th e se a d d itio n a l c o m p u ta tio n s, th e n u m b e r o f c o m p u ta tio n s is in crease d
by a sm all a m o u n t.
F ro m th e se resu lts w e c o n c lu d e th a t th e W elch m e th o d r e q u ire s a little m o re
c o m p u ta tio n a l p o w e r th a n d o th e o th e r tw o m e th o d s. T h e B a rtle tt m e th o d a p p a r
en tly re q u ire s th e sm a llest n u m b e r o f c o m p u ta tio n s. H o w e v e r, th e d ifferen ces in
th e c o m p u ta tio n a l re q u ire m e n ts of th e th re e m e th o d s a re re la tiv e ly sm all.

12.3 PARAMETRIC METHODS FOR POWER SPECTRUM ESTIMATION

T h e n o n p a ra m e tric p o w e r sp e c tru m e stim a tio n m e th o d s d e s c rib e d in th e p re c e d in g
se ctio n a re re lativ ely sim p le, w ell u n d e rsto o d , an d easy to c o m p u te using th e F F T
alg o rith m . H o w ev er, th e s e m e th o d s re q u ire th e av ailab ility o f lo n g d a ta re c o rd s in
o r d e r to o b ta in th e n ecessa ry fre q u e n c y re so lu tio n re q u ire d in m a n y ap p licatio n s.
F u rth e rm o re , th e s e m e th o d s su ffer fro m sp e c tra l le a k a g e effects, d u e to w indow -

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

921

ing, th a t a re in h e re n t in fin ite-len g th d a ta reco rd s. O fte n , th e sp e c tra l leak ag e

m ask s w eak sig n als th a t a re p re s e n t in th e d a ta .
F ro m o n e p o in t o f view , th e basic lim ita tio n o f th e n o n p a ra m e tric m e th o d s is
th e in h e re n t a ssu m p tio n th a t th e a u to c o rre la tio n e stim a te rxx(m) is z e ro fo r m > N,
as im p lied by (12.1.33). T h is a ssu m p tio n se v ere ly lim its th e fre q u e n c y re so lu tio n
an d th e q u a lity o f th e p o w e r sp e c tru m e s tim a te th a t is ach iev ed . F ro m a n o th e r
v iew p o in t, th e in h e re n t a ssu m p tio n in th e p e rio d o g ra m e s tim a te is th a t th e d a ta
are p e rio d ic w ith p e rio d N . N e ith e r o n e o f th e s e a s su m p tio n s is realistic.
In th is se ctio n w e d esc rib e p o w e r sp e c tru m e s tim a tio n m e th o d s th a t d o n o t
re q u ire such a s su m p tio n s. In fact, th e s e m e th o d s ext rapolate th e v alu es o f th e
a u to c o rre la tio n fo r lags m > N. E x tra p o la tio n is p o ssib le if w e h av e so m e a pri ori
in fo rm a tio n o n h o w th e d a ta w ere g e n e ra te d . In such a case a m o d e l fo r th e signal
g e n e ra tio n can b e c o n s tru c te d w ith a n u m b e r o f p a ra m e te rs th a t can b e e s tim a te d
fro m th e o b se rv e d d a ta . F ro m th e m o d e l an d th e e s tim a te d p a ra m e te rs , w e can
c o m p u te th e p o w e r d e n sity sp e c tru m im plied by th e m odel.
In effect, th e m o d e lin g a p p ro a c h e lim in a te s th e n e e d fo r w indow fu n ctio n s
an d th e a s su m p tio n th a t th e a u to c o rre la tio n se q u e n c e is z e ro fo r \m\ > N . A s a
c o n s e q u e n c e , paramet ri c (m o d e l-b a s e d ) p o w e r sp e c tru m e s tim a tio n m e th o d s avoid
th e p ro b le m o f le a k a g e a n d p ro v id e b e tte r freq u e n c y re so lu tio n th a n d o th e F F T b ased , n o n p a ra m e tric m e th o d s d e sc rib e d in th e p re c e d in g se ctio n . T h is is e s p e
cially tru e in a p p lic a tio n s w h e re sh o rt d a ta re c o rd s a re a v a ila b le d u e t o tim e -v a ria n t
o r tra n sie n t p h e n o m e n a .
T h e p a ra m e tric m e th o d s c o n s id e re d in th is section a re b ased o n m o d elin g
th e d a ta se q u e n c e * ( ) as th e o u tp u t o f a lin ear system c h a ra c te riz e d by a ra tio n a l
sy stem fu n ctio n o f th e fo rm

(12.3.1)

T h e c o rre sp o n d in g d iffe re n c e e q u a tio n is

p
(12.3.2)
w h e re w( n) is th e in p u t se q u e n c e to th e system a n d th e o b se rv e d d a ta , x( n ) ,
re p re s e n ts th e o u tp u t se q u e n c e .
In p o w e r sp e c tru m e stim a tio n , th e in p u t se q u e n c e is n o t o b se rv a b le . H o w
e v e r, if th e o b se rv e d d a ta a re c h a ra c te riz e d as a s ta tio n a ry ra n d o m p ro cess, th e n
th e in p u t se q u e n c e is also assu m ed to b e a s ta tio n a ry ra n d o m process. In such a
case th e p o w e r d e n s ity sp e c tru m o f th e d a ta is
r ( / ) = \ n ( f ) \ 2r ww( f )
w h e re r ,**,(/) is th e p o w e r d en sity sp e c tru m o f th e in p u t se q u e n c e a n d H ( f ) is
th e fre q u e n c y re sp o n s e o f th e m o d el.

9 22

Power Spectrum Estimation

Chap. 12

S ince o u r o b jectiv e is to e stim a te th e p o w e r d en sity s p e c tru m r ( / ) , it is

c o n v e n ie n t to assu m e th a t th e in p u t se q u e n c e w( n ) is a z e ro -m e a n w h ite noise
se q u e n c e w ith a u to c o rre la tio n
Yww( m) = ^ S ( m )

w h e re a 2 is th e v a ria n c e (i.e., a 2 = [|u>()|2]). T h e n th e p o w e r d en sity sp e c tru m

o f th e o b se rv e d d a ta is sim ply
T (/) = ol\H {f)\2 =

(12.3.3)

In S e ctio n 11.1 w e d e sc rib e d th e r e p re s e n ta tio n o f a sta tio n a ry ra n d o m p ro c e ss as

given b y (12.3.3).
In th e m o d e l-b a se d a p p ro a c h , th e sp e c tru m e s tim a tio n p ro c e d u re consists
o f tw o step s. G iv e n th e d a ta se q u e n c e x ( n) , 0 < n < JV - 1, w e e stim a te th e
p a r a m e te r s {a*} a n d (6*) o f th e m o d el. T h e n fro m th e s e e s tim a te s , w e c o m p u te
th e p o w e r sp e c tru m e s tim a te acco rd in g to (12.3.3).
R e c a ll th a t th e ra n d o m p ro c e ss x ( n ) g e n e ra te d b y th e p o le - z e r o m o d e l in
(12.3.1) o r (12.3.2) is called an aut or e gr es s i ve- movi ng average (A R M A ) pr oc es s o f
o r d e r (p , q) a n d it is u sually d e n o te d as A R M A (p , q). I f q = 0 a n d fro =
th e
re su ltin g system m o d e l h as a system fu n c tio n H ( z ) = 1 /A (z) a n d its o u tp u t x( n)
is called a n autoregressive (A R ) process o f o r d e r p. T h is is d e n o te d as A R(/>).
T h e th ird p o ssib le m o d e l is o b ta in e d by se ttin g A ( z ) = 1, so th a t H( z ) B{z). Its
o u tp u t x ( n ) is called a m o v i n g average (M A ) p r o c e s s o f o r d e r q a n d d e n o te d as
M A (^ ).
O f th e se th re e lin e a r m o d els th e A R m o d e l is by fa r th e m o st w idely used.
T h e re a so n s a re tw o fo ld . F irst, th e A R m o d e l is su ita b le fo r re p re s e n tin g sp e ctra
w ith n a rro w p e a k s (re so n a n c e s). S eco n d , th e A R m o d e l re su lts in v ery sim ple
lin e a r e q u a tio n s fo r th e A R p a ra m e te rs . O n th e o th e r h a n d , th e M A m o d e l, as
a g e n e ra l ru le , re q u ire s m an y m o re co effic ie n ts to r e p re s e n t a n a rro w sp ectru m .
C o n s e q u e n tly , it is ra re ly u se d by itse lf as a m o d e l f o r s p e c tru m e s tim a tio n . By
co m b in in g p o le s a n d zero s, th e A R M A m o d e l p ro v id e s a m o re efficien t re p re s e n
ta tio n , fro m th e v iew p o in t o f th e n u m b e r o f m o d e l p a ra m e te rs , o f th e sp e c tru m
o f a ra n d o m p ro cess.
T h e d e c o m p o sitio n th e o re m d u e to W o ld (1938) a s se rts th a t an y A R M A o r
M A p ro c e ss can b e re p re s e n te d u n iq u ely by an A R m o d e l o f p o ssib ly infinite o rd e r,
a n d a n y A R M A o r A R p ro c e ss c a n b e r e p re s e n te d by a M A m o d e l o f possibly
in fin ite o rd e r. In v iew o f this th e o re m , th e issue o f m o d e l se le c tio n re d u c e s to
selectin g th e m o d e l th a t re q u ire s th e sm a llest n u m b e r o f p a r a m e te r s th a t a re also
easy to c o m p u te . U su ally , th e choice in p ra c tic e is th e A R m o d e l. T h e A R M A
m o d e l is u sed to a le s se r ex ten t.
B e fo re d escrib in g m e th o d s fo r e s tim a tin g th e p a r a m e te r s in a n A R ( p ) , MA(<?),
a n d A R M A ( p , q) m o d els, it is useful to e sta b lish th e b asic re la tio n sh ip s b etw een
th e m o d e l p a r a m e te r s a n d th e a u to c o rre la tio n s e q u e n c e yxx( m). In a d d itio n , we
re la te th e A R m o d e l p a ra m e te rs to th e co efficien ts in a lin e a r p r e d ic to r fo r the
p ro c e ss x( n) .

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

9 23

12.3.1 Relationships Between the Autocorrelation and

the Model Parameters
In S ectio n 11.1.2 w e esta b lish e d th e b asic re la tio n sh ip s b e tw e e n th e a u to c o rre la tio n
{ ^ ( m ) } a n d th e m o d e l p a r a m e te r s {a*} a n d {bk}. F o r th e A R M A ( p , q) process,
th e re la tio n sh ip g iv en by (11.1.18) is

~22QkYxx>jn Yxx (m) =

m > q

k),

(12.3.4)

~ 2 2 ot Yxxi m - ^
*=l
y*x (m) ,

+ <*122
*=o

0 <m <q
m < 0

T h e re la tio n sh ip s in (12.3.4) p ro v id e a fo rm u la fo r d e te rm in in g th e m o d el
p a ra m e te rs {ak} by re stric tin g o u r a tte n tio n to th e case m > q. T h u s th e se t o f
lin e a r e q u a tio n s
~Yxx(q)
Yxxiq + 1)

Yxxiq ~ 1)
Yxxiq)

-Yxx(q + / ? - ! )

Yxx(q + p - 2)

Yxxiq ~ p + 1 ) '
Yxx (q + P + 2)

Yxxiq)
Yxxiq + 1)
yxx(q + 2)
(12.3.5)
- Yx xi q + P ) -

can b e u se d to so lv e fo r th e m o d e l p a ra m e te rs {ak} by u sin g e s tim a te s o f th e

a u to c o rre la tio n se q u e n c e in place o f Yxx
) fo r m > q. T h is p ro b le m is d iscussed
in S e ctio n 12.3.8.
A n o th e r in te r p re ta tio n o f th e re la tio n sh ip in (12.3.5) is th a t th e v alu es o f
th e a u to c o rre la tio n Yxx ) iox m > q a re u n iq u e ly d e te rm in e d fro m th e p o le
p a ra m e te rs {ak} a n d th e v alu es o f y xx(m) fo r 0 < m < p. C o n s e q u e n tly , th e lin e a r
sy stem m o d e l a u to m a tic a lly e x te n d s th e v alu es o f th e a u to c o rre la tio n se q u e n c e
Yxx(m) fo r m > p.
If th e p o le p a ra m e te rs {a*} a re o b ta in e d fro m (12.3.5), th e re su lt d o e s n o t
h e lp u s in d e te rm in in g th e M A p a r a m e te r s {i*}, b e c a u se th e e q u a tio n
q-m

l 2 2 h ( f t bk+m = Yxx(m) + ' Y 2 a kYxx{m - k)

0 < m < q

d e p e n d s o n th e im p u lse re sp o n s e h( n). A lth o u g h th e im p u lse re sp o n s e can be

e x p re ss e d in te rm s o f th e p a ra m e te rs {*} by lo n g division o f B(z) w ith th e kn o w n
A( z ) , th is a p p ro a c h re su lts in a se t o f n o n lin e a r e q u a tio n s fo r th e M A p a r a m
eters.

Power Spectrum Estimation

924

Chap. 12

I f w e a d o p t a n A R ( p ) m o d e l fo r th e o b se rv e d d a ta , th e re la tio n s h ip b e tw e e n
th e A R p a ra m e te rs a n d th e a u to c o rre la tio n se q u e n c e is o b ta in e d by se ttin g q = 0
in (12.3.4). T h u s w e o b ta in
p

~ Y 2 akYxx(m - k),
*=i

m> 0

Y xx(m ) =

(12.3.6)

- J 2 akYxx(m - k ) + ct^,
Jt=i
Yxx( ~ m )-

m = 0
m < 0

In th is case, th e A R p a r a m e te r s {a*) a re o b ta in e d fro m th e so lu tio n o f th e Y u le W a lk e r o r n o rm a l e q u a tio n s

Yxx (0)

Y x x ( ~ 1)

Yxx(l)

Y x x ( 0)

- Y x x i p - 1)

Y x x ip -

Y x x i - P + 1)
Yxx(~ P

2)

+ 2)

~a\ "

Y x x i 1) "

a2

Yxx

(2)

(12.3.7)

. Y xx( P ) .

Krjr (0)

a n d th e v aria n c e a 2 can b e o b ta in e d fro m th e e q u a tio n

p

(12.3.8)

= Yxxi ) + Y 2 a k Y x x i ~ k )

T h e e q u a tio n s in (12.3.7) a n d (12.3.8) a re u su ally co m b in e d in to a single m atrix

e q u a tio n o f th e fo rm
Yxx ( 0 )
Y xx(l)

L y(p)

Yx x (

- 1 -

1)

Yxx ( 0 )

Y x x ip -

1)

Y x x ( - p + 1)

y (0 )

a\
-ap_

~a l ~
0
(12.3.9)

_0 _

S ince th e c o rre la tio n m atrix in (12.3.7), o r in (12.3.9), is T o e p litz , it can b e effi

cien tly in v e rte d b y u se o f th e L e v in s o n - D u rb in a lg o rith m .
T h u s all th e sy stem p a r a m e te r s in th e A R (/?) m o d e l a r e easily d e te rm in e d
fro m k n o w le d g e o f th e a u to c o rre la tio n se q u e n c e yx x (m) fo r 0 < m < p. F u rth e r
m o re , (12.3.6) can b e u se d to e x te n d th e a u to c o rre la tio n s e q u e n c e f o r m > p, o n ce
th e {at) a re d e te rm in e d .
F in ally , f o r c o m p le te n e s s, w e in d ic a te th a t in a M A (^ ) m o d e l f o r th e o b se rv e d
d a ta , th e a u to c o rre la tio n se q u e n c e yjjr(m ) is re la te d to th e M A p a r a m e te r s [bk] by
th e e q u a tio n
9
< ? l Y ^ b kbk+m,
0 <m <q
Yxx(m )

1=0
0,
y*x( - m ) ,

which was established in Section 11.1.

m > q
m < 0

(12.3.10)

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

925

W ith th is b a c k g ro u n d esta b lish e d , w e no w d esc rib e th e p o w e r sp e c tru m e s

tim a tio n m e th o d s fo r th e A R (/?), A R M A (/?,< ?), a n d MA(<?) m o d els.

12.3.2 The Yule-Walker Method for the AR Model

Parameters
In th e Y u le - W a lk e r m e th o d we sim ply e stim a te th e a u to c o rre la tio n fro m th e
d a ta an d u se th e e stim a te s in (12.3.7) to solve fo r th e A R m o d e l p a ra m e te rs.
In th is m e th o d it is d e s ira b le to use th e b iased fo rm o f th e a u to c o rre la tio n e sti
m a te ,
^ N - m 1

rxx(m) =
N

y '
n = 0

jc*(n)jc( + m)

m> 0

(12.3.11)

to e n s u re th a t th e a u to c o rre la tio n m atrix is p o sitiv e se m id e fin ite. T h e re su lt is

a sta b le A R m o d el. A lth o u g h sta b ility is n o t a critical issue in p o w e r sp e c
tru m e s tim a tio n , it is c o n je c tu re d th a t a sta b le A R m o d e l b e st re p re s e n ts th e
d ata.
T h e L e v in s o n -D u rb in alg o rith m d e sc rib e d in C h a p te r 11 w ith rxx( m ) su b sti
tu te d fo r yxx(m) yield s th e A R p a ra m e te rs. T h e c o rre sp o n d in g p o w e r sp e c tru m
e s tim a te is
2
P r w ( f ) = --------------- ^ ---------------(12 3 12)
"
II + L i a P( * ) e - J2*/ l l2
w h e re a p(k) a re e s tim a te s o f th e A R p a ra m e te rs o b ta in e d fro m th e L e v in s o n D u rb in re c u rsio n s a n d
<7l p = / = rxx(0) f [ [ l - M
*=i

) | 2]

(12.3.13)

is th e e s tim a te d m in im u m m e a n -sq u a re v alue fo r th e p th - o rd e r p re d ic to r. A n

ex a m p le illu stra tin g th e fre q u e n c y re so lu tio n ca p a b ilitie s o f th is e s tim a to r is given
in S ectio n 12.3.9.
In e stim a tin g th e p o w e r sp e c tru m o f sin u so id al signals via A R m o d els, L acoss
(1971) sh o w ed th a t sp e c tra l p e a k s in an A R sp e c tru m e s tim a te a re p ro p o rtio n a l
to th e s q u a re o f th e p o w e r o f th e sin u so id al signal. O n th e o th e r h a n d , th e a re a u n
d e r th e p e a k in th e p o w e r d en sity sp e c tru m is lin early p ro p o rtio n a l to th e p o w e r o f
th e sin u so id . T h is c h a ra c te ristic b e h a v io r h o ld s fo r all A R m o d e l-b a se d estim atio n
m e th o d s.

12.3.3 The Burg Method for the AR Model Parameters

T h e m e th o d d ev ised b y B u rg (1968) fo r e stim a tin g th e A R p a r a m e te r s can be
v iew ed as an o rd e r-re c u rsiv e le a st-sq u a re s lattice m e th o d , b a s e d on th e m in im iz a
tio n o f th e fo rw a rd a n d b ac k w a rd e rro rs in lin e a r p re d ic to rs , w ith th e c o n s tra in t
th a t th e A R p a r a m e te r s satisfy th e L e v in s o n -D u rb in re c u rsio n .

Pow er Spectrum Estimation

926

Chap. 12

T o d e riv e th e e s tim a to r, su p p o s e th a t w e a re given th e d a ta * (n ), n = 0,

1.........N 1, a n d let us c o n sid e r th e fo rw a rd a n d b a c k w a rd lin e a r p re d ic tio n e sti
m a te s o f o r d e r m , given as
m

x(n) = y

am(fc)x (rt fc)

*=1

(12.3.14)

x ( n - m) =

(fc)x(n + k m)
k= 1

a n d th e c o rre sp o n d in g fo rw a rd a n d b a c k w a rd e rro rs f m{n) a n d gm(n) defin ed as

f m{n) = x(n) x(n) a n d gm(n) = x(n m) x(n m ) w h e re am(k), 0 < k <
m 1, m = 1, 2 , . . . , p, are th e p re d ic tio n coefficients. T h e le a st-sq u a re s e r ro r
is
A'-l

+ \ g M \ 2]

(12.3.15)

n=/n
T h is e r r o r is to b e m in im ized by se lectin g th e p re d ic tio n coefficients, su b je ct
to th e c o n s tra in t th a t th e y satisfy th e L e v in s o n -D u rb in re c u rsio n given by
am(k) = a m- i( * ) -I-

-k)

1 < fc < m - 1

(12.3.16)

1 < m < p
w h ere K m = a m(m) is th e m th reflec tio n c o effic ie n t in th e la ttic e filter re a liz a tio n
o f th e p re d ic to r. W h e n (12.3.16) is s u b s titu te d in to th e ex p re ssio n s fo r f m(n)
a n d gm(n), th e re su lt is th e p a ir o f o rd e r-re c u rsiv e e q u a tio n s fo r th e fo rw a rd a n d
b ac k w a rd p re d ic tio n e rro rs given by (11.2.4).
N ow , if w e su b s titu te fro m (11.2.4) in to (12.3.16) a n d p e rfo rm th e m in im iza
tio n o f m w ith re sp e c t to th e co m p lex -v alu ed re fle c tio n co effic ie n t K m, w e o b ta in
th e resu lt
N-1

fm-i(n)g*m- i ( n - D
Km =

------------------------------------

m = 1 ,2 , . . . , p

(12.3.17)

n=m

T h e te rm in th e n u m e r a to r o f (12.3.17) is a n e s tim a te o f th e c ro ssc o rre la tio n b e

tw e e n th e fo rw a rd a n d b ac k w a rd p re d ic tio n e rro rs. W ith th e n o rm a liz a tio n facto rs
in th e d e n o m in a to r o f (12.3.17), it is a p p a r e n t th a t |AT,J < 1, so th a t th e all-pole
m o d e l o b ta in e d fro m th e d a ta is sta b le . T h e r e a d e r sh o u ld n o te th e sim ilarity o f
(12.3.17) to its sta tistical c o u n te rp a rts given b y (11.2.29).
W e n o te th a t th e d e n o m in a to r in (12.3.17) is sim ply th e le a s t-s q u a re s e s tim a te
o f th e fo rw a rd a n d b a c k w a rd e rro rs,

a n d E ^ _ v re sp e c tiv e ly . H e n c e (12.3.17)

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

927

can b e e x p re ss e d as

- 1)
m = 1, 2 , . . . , p

Km =

(12.3.18)

w h e re
+ E bm- \ is an e stim a te o f th e to ta l sq u a re d e r ro r E m. W e leave it as
a n ex ercise fo r th e r e a d e r to verify th a t th e d e n o m in a to r te r m in (12.3.18) can be
c o m p u te d in an o rd e r-re c u rsiv e fash io n a c co rd in g to th e re la tio n
E m = (1 - \ Km |2) m_i - |/m -] (m - 1)[2 - |gm_ ,(m - 2 ) |2

(12.3.19)

w h e re E, = E ^ + E b is th e to ta l le a st-sq u a re s e rro r. T h is re su lt is d u e to A n d e rse n

(1978).
T o su m m arize, th e B u rg a lg o rith m c o m p u te s th e reflec tio n coefficients in
th e e q u iv a le n t lattice s tru c tu re as specified by (12.3.18) a n d (12.3.19). a n d th e
L e v in s o n -D u rb in a lg o rith m is u se d to o b ta in th e A R m o d e l p a ra m e te rs. F ro m
th e e s tim a te s o f th e A R p a ra m e te rs, w e fo rm th e p o w e r sp e c tru m e s tim a te

(12.3.20)

1 + j 2 a p( k ) e - W

T h e m a jo r a d v a n ta g e s o f th e B u rg m e th o d fo r e stim a tin g th e p a ra m e te rs o f
th e A R m o d e l a re (1) it resu lts in high fre q u e n c y re so lu tio n , (2) it yields a sta b le
A R m o d el, an d (3) it is c o m p u ta tio n a lly efficient.
T h e B u rg m e th o d is k n o w n to h av e se v e ra l d isa d v a n ta g e s, h o w ev er. F irst,
it ex h ib its sp e c tra l lin e sp littin g a t high sig n a l-to -n o ise ratio s, [see th e p a p e r by
F o u g e re e t al. (1976)]. B y line sp littin g , w e m e a n th a t th e sp e c tru m o f x( n) m ay
h a v e a sin g le sh a rp p e a k , b u t th e B u rg m e th o d m ay re s u lt in tw o o r m o re closely
sp a ced p e a k s. F o r h ig h -o rd e r m o d els, th e m e th o d also in tro d u c e s sp u rio u s p eak s.
F u rth e rm o r e , fo r sin u so id al signals in n o ise , th e B u rg m e th o d ex h ib its a sensitivity
to th e in itial p h a se o f a sinusoid, esp ecially in s h o rt d a ta reco rd s. T h is sensitivity
is m a n ife st as a fre q u e n c y shift fro m th e tru e freq u e n cy , re su ltin g in a p h a se d e
p e n d e n t fre q u e n c y b ias. F o r m o re d etails o n so m e o f th e s e lim ita tio n s th e r e a d e r
is r e fe rr e d to th e p a p e rs o f C h e n a n d S te g e n (1974), U lry c h a n d C lay to n (1976),
F o u g e re e t al. (1976), K ay a n d M a rp le (1979), S w ingler (1979a, 1980), H e rrin g
(1980), a n d T h o rv a ld s e n (1981).
S ev eral m o d ifica tio n s h av e b e e n p ro p o se d to o v e rc o m e so m e o f th e m o re
im p o rta n t lim ita tio n s o f th e B u rg m e th o d : n a m ely , th e lin e sp littin g , sp u rio u s
p e a k s , a n d fre q u e n c y bias. B asically, th e m o d ifica tio n s inv o lv e th e in tro d u c tio n
o f a w eig h tin g (w in d o w ) se q u e n c e o n th e sq u a re d fo rw a rd a n d b ack w a rd e r
ro rs . T h a t is, th e le a st-sq u a re s o p tim iz a tio n is p e rfo rm e d o n th e w e ig h te d s q u a re d

928

Power Spectrum Estimation

Chap. 12

e rro rs
N-1

b =

wm( n) [ \ f m(n)\2 + |gm(n)[2]

(12.3.21)

w hich , w h en m in im iz ed , re su lts in th e reflec tio n coefficient e s tim a te s

A '- l

']Tl Wm--i(n)fm- i ( n ) g Z _ l (n - 1)
Km = ~1 ~

L--------------------------------------------------

1 n=m

(12.3.22)

+ |g m- i ( n - 1 )|2]

In p a rtic u la r, w e m e n tio n th e use o f a H a m m in g w in d o w u se d by S w ingler

(1979b), a q u a d ra tic o r p a ra b o lic w in d o w u sed by K av eh a n d L ip p e rt (1983), th e
e n erg y w eig h tin g m e th o d u se d by N ik ias a n d S co tt (1982), a n d th e d a ta -a d a p tiv e
en e rg y w eig h tin g u se d by H e lm e an d N ik ias (1985).
T h e se w in d o w in g a n d en erg y w eig h tin g m e th o d s h av e p ro v e d effe ctiv e in
re d u cin g th e o c c u rre n c e o f line sp littin g a n d sp u rio u s p eak s, a n d a re also effective
in re d u c in g fre q u e n c y bias.
T h e B u rg m e th o d fo r p o w e r sp e c tru m e s tim a tio n is u su a lly asso ciated w ith
m a x i m u m ent r opy spe ct r um est i mati on, a c rite rio n u sed by B u rg (1967, 1975) as a
basis f o r A R m o d elin g in p a ra m e tric s p e c tru m e s tim a tio n . T h e p ro b le m co n sid e re d
by B u rg w as h o w b e st to e x tra p o la te fro m th e given v alu es o f th e a u to c o rre la tio n
se q u e n c e yxx{m), 0 < m < p , th e v alu es fo r m > p, such th a t th e e n tire a u to c o r re
latio n se q u e n c e is p o sitiv e se m id efin ite. S ince an in fin ite n u m b e r o f e x tra p o la tio n s
a re p o ssib le, B u rg p o s tu la te d th a t th e e x tra p o la tio n s b e m a d e o n th e b asis o f m ax i
m izing u n c e rta in ty (e n tro p y ) o r ra n d o m n e ss , in th e se n se th a t th e sp e c tru m Txx{ f )
o f th e p ro cess is th e fla tte st o f all sp e c tra w hich h a v e th e g iv en a u to c o rre la tio n
valu es yxx(m), 0 < m < p. In p a rtic u la r th e e n tro p y p e r sa m p le is p ro p o rtio n a l to
th e in te g ra l [see B u rg (1975)]
r 1/2

/
In
J-l/2

r xx( f ) d f

(12.3.23)

B u rg fo u n d th a t th e m ax im u m o f th is in te g ra l su b je ct to th e ( p + 1) c o n stra in ts
rlfl
/
J - 1/2

= Yxx(m)

0 < m < p

(12.3.24)

is th e A R ( p ) p ro c e ss fo r w hich th e given a u to c o rre la tio n se q u e n c e yxx( m), 0 <

m < p is re la te d to th e A R p a ra m e te rs by th e e q u a tio n (12.3.6). T h is so lu tio n
p ro v id e s an a d d itio n a l ju stificatio n fo r th e u se o f th e A R m o d e l in p o w e r sp e c tru m
e stim a tio n .
In view o f B u rg s b asic w o rk in m ax im u m e n tro p y s p e c tra l es tim a tio n , th e
B u rg p o w e r sp e c tru m e s tim a tio n p ro c e d u re is o fte n called th e m a x i m u m ent ropy
m e t h o d (M E M ). W e sh o u ld em p h asize, h o w ev er, th a t th e m a x im u m e n tro p y sp e c
tru m is id en tical to th e A R -m o d e l sp e c tru m o n ly w h e n th e e x a c t a u to c o rre la tio n

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

929

yx x (m) is k n o w n . W h e n o n ly an e s tim a te o f yxx(m) is a v a ilab le fo r 0 < m < p, th e

A R -m o d e l e s tim a te s o f Y u le -W a lk e r a n d B u rg a re n o t m ax im u m e n tro p y sp e c
tra l e s tim a te s. T h e g e n e ra l fo rm u la tio n fo r th e m ax im u m e n tro p y sp e c tru m b a se d
on e stim a te s o f th e a u to c o rre la tio n se q u e n c e resu lts in a se t o f n o n lin e a r e q u a
tio n s. S o lu tio n s fo r th e m ax im u m e n tro p y sp e c tru m w ith m e a s u re m e n t e rro rs in
th e c o rre la tio n se q u e n c e h a v e b e e n o b ta in e d by N ew m an (1981) a n d S c h o tt a n d
M cC lellan (1984).

12.3.4 Unconstrained Least-Squares Method for the AR

Model Parameters
A s d e sc rib e d in th e p re c e d in g se ctio n , th e B u rg m e th o d f o r d e te rm in in g th e p a
r a m e te rs o f th e A R m o d e l is basically a le a st-sq u a re s la ttic e a lg o rith m w ith th e
a d d e d c o n s tra in t th a t th e p re d ic to r coefficients satisfy th e L e v in so n re c u rsio n . A s
a re su lt o f th is c o n s tra in t, a n in crease in th e o r d e r of th e A R m o d e l re q u ire s only
a sin g le p a r a m e te r o p tim iz a tio n a t e a c h stage. In c o n tr a s t to th is a p p ro a c h , w e
m ay use an u n c o n s tra in e d le a st-sq u a re s alg o rith m to d e te rm in e th e A R p a r a m e
ters.
T o e la b o ra te , w e fo rm th e fo rw a rd an d b ac k w a rd lin e a r p re d ic tio n e stim a te s
an d th e ir c o rre sp o n d in g fo rw a rd a n d b ac k w a rd e r ro r s as in d ic a te d in (12.3.14) a n d
(12.3.15). T h e n w e m in im ize th e sum o f sq u a re s o f b o th e rro rs , th a t is,
N -1

p -

Y l ^ f p ^ 2+

n=p

x ( n ) + ' Y \ ap ^ x ^n ~ k )
*=i

2"

+ x ( n - p) + 2 2 Op(k ) x( n + k - p)
k=1

(12.3.25)
w h ich is th e sa m e p e rfo rm a n c e in d ex as in th e B u rg m e th o d . H o w e v e r, w e d o n o t
im p o se th e L e v in s o n - D u rb in re c u rsio n in (12.3.25) fo r th e A R p a ra m e te rs. T h e
u n c o n s tra in e d m in im iz atio n o f p w ith re sp e c t to th e p re d ic tio n co effic ie n ts yields
th e se t o f lin e a r e q u a tio n s
M * ) ^ ( U ) = - r ( l , 0)
k=l

l = l,2,...,p

(12.3.26)

w h e re , by d efin itio n , th e a u to c o rre la tio n rxx( l , k ) is

JV-1
r ( / , k) =

~ k ) x *(n 0 + x ( n - P + l)x*(n p + )]

(12.3.27)

np
T h e re su ltin g re sid u a l le a st-sq u a re s e r r o r is
p
e pL s = r ( 0 ,0 ) + t f , ( * ) r ( 0 , k )
t-i

(12.3.28)

930

Power Spectrum Estimation

Chap. 12

H e n c e th e u n c o n s tra in e d le a st-sq u a re s p o w e r s p e c tru m e s tim a te is

ls
P ^ i f ) = ----------------E---------------T
1 + i 2 a p( k ) e - J^
k=l

(12.3.29)

T h e c o rre la tio n m a trix in (12.3.27), w ith e le m e n ts rxx(l, k) , is n o t T o e p litz , so

th a t th e L e v in s o n -D u rb in alg o rith m c a n n o t b e a p p lie d . H o w e v e r, th e c o rre la tio n
m atrix h a s su fficien t s tru c tu re to m a k e it p o ssib le to d ev ise c o m p u ta tio n a lly effi
cien t a lg o rith m s w ith c o m p u ta tio n a l co m p lex ity p ro p o rtio n a l to p 2. M a rp le (1980)
d ev ised such an a lg o rith m , w hich h a s a lattice s tru c tu re a n d em p lo y s L e v in so n D u rb in -ty p e o r d e r re c u rsio n s a n d a d d itio n a l tim e recu rsio n s.
T h is fo rm o f th e u n c o n s tra in e d le a st-sq u a re s m e th o d d e s c rib e d h a s also b e e n
called th e u n w i n d o w e d data le a st-sq u a re s m e th o d . I t h a s b e e n p ro p o s e d fo r sp e c
tru m e s tim a tio n in se v era l p a p e rs, in clu d in g th e p a p e rs b y B u rg (1967), N u tta ll
(1976), an d U lry ch an d C la y to n (1976). Its p e rfo rm a n c e c h a ra c te ristic s h av e b e e n
fo u n d to b e s u p e rio r to th e B u rg m e th o d , in th e se n se th a t th e u n c o n s tra in e d leastsq u a re s m e th o d d o e s n o t e x h ib it th e sa m e sen sitiv ity to such p ro b le m s as line sp lit
ting, fre q u e n c y b ias, a n d sp u rio u s p eak s. In view o f th e c o m p u ta tio n a l efficiency o f
M a rp le s a lg o rith m , w hich is c o m p a ra b le to th e efficiency o f th e L e v in s o n -D u rb in
alg o rith m , th e u n c o n s tra in e d le a st-sq u a re s m e th o d is very a ttr a c tiv e . W ith this
m e th o d th e re is n o g u a ra n te e th a t th e e stim a te d A R p a r a m e te r s yield a sta b le
A R m o d el. H o w e v e r, in sp e c tru m e s tim a tio n , th is is n o t c o n s id e re d to be a
p ro b le m .

12.3.5 Sequential Estimation Methods for the AR Model

Parameters
T h e th r e e p o w e r sp e c tru m e s tim a tio n m e th o d s d e sc rib e d in th e p re c e d in g se ctio n s
fo r th e A R m o d e l can b e classified as b lo ck p ro cessin g m e th o d s. T h e se m e th o d s
o b ta in e stim a te s o f th e A R p a ra m e te rs fro m a b lo ck o f d a ta , say jr(n), n = 0,
I , . , N I. T h e A R p a ra m e te rs , b a s e d o n th e b lo ck o f N d a ta p o in ts, a re th e n
u sed to o b ta in th e p o w e r sp e c tru m estim a te .
In situ a tio n s w h e re d a ta a re av ailab le on a c o n tin u o u s b asis, w e can still
se g m en t th e d a ta in to b lo ck s o f N p o in ts an d p e rfo rm sp e c tru m e stim a tio n on
a b lo ck -b y -b lo ck basis. T h is is o fte n d o n e in p ra c tic e , fo r b o th re a l-tim e an d
n o n -re a l-tim e ap p licatio n s. H o w e v e r, in such a p p lic a tio n s, th e r e is an a lte rn a tiv e
a p p ro a c h b ased o n se q u e n tia l (in tim e) e s tim a tio n o f th e A R m o d e l p a ra m e te rs as
each n ew d a ta p o in t b ec o m e s av ailab le. B y in tro d u c in g a w e ig h tin g fu n ctio n in to
p a s t d a ta sam p les, it is p o ssib le to d e e m p h a siz e th e effect o f o ld e r d a ta sam ples
as n ew d a ta a re receiv ed .
S e q u e n tia l la ttic e m e th o d s b ased o n recu rsiv e le a s t sq u a re s can b e em p lo y ed
to o p tim a lly e s tim a te th e p re d ic tio n a n d re fle c tio n c o effic ie n ts in th e lattice re
alizatio n o f th e fo rw a rd a n d b ac k w a rd lin e a r p re d ic to rs . T h e re c u rsiv e e q u a

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

931

tio n s fo r th e p re d ic tio n coefficients re la te directly to th e A R m o d el p a ra m e te rs.

In a d d itio n to th e o rd e r-re c u rsiv e n a tu re o f th e s e e q u a tio n s , as im p lied by th e
la ttic e stru c tu re , w e can also o b ta in tim e -re c u rsiv e e q u a tio n s fo r th e reflectio n
co effic ie n ts in th e lattice an d fo r th e fo rw a rd a n d b a c k w a rd p re d ic tio n coeffi
cien ts.
S e q u e n tia l rec u rsiv e le a st-sq u a re s a lg o rith m s a re e q u iv a le n t to th e u n c o n
stra in e d le a st-sq u a re s, b lo ck p ro cessin g m e th o d d e sc rib e d in th e p re c e d in g se c
tio n . H e n c e th e p o w e r sp e c tru m e stim a te s o b ta in e d by th e s e q u e n tia l recursive
le a s t-s q u a re s m e th o d re ta in th e d e s ira b le p r o p e rtie s o f th e b lo ck p ro cessin g alg o
rith m d e s c rib e d in S ectio n 12.3.4. S ince th e A R p a ra m e te rs a re b ein g co n tin u o u sly
e s tim a te d in a se q u e n tia l e stim a tio n a lg o rith m , p o w e r sp e c tru m e stim a te s can b e
o b ta in e d as o fte n as d e s ire d , fro m o n ce p e r sa m p le to o n c e ev ery N sam ples. By
p ro p e rly w e ig h tin g p a s t d a ta sam p les, th e s e q u e n tia l e s tim a tio n m e th o d s a re p a r
ticu larly su ita b le fo r e stim a tin g a n d tra c k in g tim e -v a ria n t p o w e r s p e c tra resu ltin g
fro m n o n s ta tio n a ry signal statistics.
T h e c o m p u ta tio n a l co m p lex ity o f s e q u e n tia l e s tim a tio n m e th o d s is g en erally
p ro p o rtio n a l to p, th e o r d e r o f th e A R p ro cess. A s a c o n se q u e n c e , se q u e n tia l
e s tim a tio n a lg o rith m s a re c o m p u ta tio n a lly efficient an d , fro m this v iew p o in t, m ay
o ffe r so m e a d v a n ta g e o v e r th e b lo ck p ro cessin g m e th o d s.
T h e re a re n u m e ro u s re fe re n c e s on se q u e n tia l e stim a tio n m e th o d s. T he p a
p e rs by G riffith s (1975), F rie d la n d e r (1982a, b ), a n d K a lo u p tsid is a n d T h e o d o rid is
(1987) a re p a rtic u la rly re le v a n t to th e sp e c tru m e s tim a tio n p ro b le m .

12.3.6 Selection of AR Model Order

O n e o f th e m o st im p o rta n t asp ects of th e use o f th e A R m o d e l is th e se lectio n
o f th e o r d e r p. A s a g e n e ra l ru le , if w e se lect a m o d e l w ith to o low an o r
d e r, w e o b ta in a highly sm o o th e d sp e c tru m . O n th e o th e r h a n d , if p is se le c te d
to o hig h , w e r u n th e risk o f in tro d u c in g sp u rio u s low -level p e a k s in th e sp e c
tru m . W e m e n tio n e d p rev io u sly th a t o n e in d ic a tio n o f th e p e rfo rm a n c e o f th e
A R m o d e l is th e m e a n -sq u a re v a lu e o f th e re sid u a l e rro r, w hich, in g en eral, is
d iffe re n t fo r each o f th e e stim a to rs d e sc rib e d a b o v e . T h e c h a ra c te ristic of th is
re sid u a l e r r o r is th a t it d e c re a s e s as th e o r d e r o f th e A R m o d e l is in crease d .
W e can m o n ito r th e r a te o f d e crease an d d e cid e to te r m in a te th e p ro cess w h en
th e ra te o f d e c re a s e b e c o m e s re la tiv e ly slow . I t is a p p a re n t, h o w ev er, th a t th is
a p p ro a c h m ay b e im p re cise an d ill-defined, an d o th e r m e th o d s sh o u ld b e in v esti
g ated .
M u ch w o rk h as b e e n d o n e by v a rio u s re s e a rc h e rs o n th is p ro b le m a n d m any
e x p e rim e n ta l re su lts h a v e b e e n given in th e lite ra tu r e [e.g., th e p a p e rs by G ersch
a n d S h a rp e (1973), U lry ch an d B ish o p (1975), T o n g (1975, 1977), J o n e s (1976),
N u tta ll (1976), B e rry m a n (1978), K a v e h a n d B ru z z o n e (1979), a n d K ash y ap
(1980)].
T w o o f th e b e tte r k n o w n c rite ria fo r se lectin g th e m o d e l o rd e r h av e b e e n
p r o p o s e d b y A k a ik e (1969, 1974). W ith th e first, c a lle d th e f i na l pr edi ct ion error

932

Power Spectrum Estimation

Chap. 12

( FPE) criterion, th e o rd e r is se le c te d to m in im iz e th e p e rfo rm a n c e in d ex

FPE(',=<(^373t)

d2'3-30*

w h e re a 2p is th e e s tim a te d v a rian ce o f th e lin e a r p re d ic tio n e r ro r . T h is p e rfo r

m a n ce in d ex is b a s e d o n m in im izin g th e m e a n -s q u a re e r ro r fo r a o n e -ste p p re d ic to r.
T h e seco n d c rite rio n p r o p o s e d by A k a ik e (1974), called th e A k a i k e i n f o r ma
t ion criterion (A IC ), is b a se d o n se lectin g th e o r d e r th a t m in im iz es
A I C ( p ) = In a l p + 2 p / N

(12.3.31)

N o te th a t th e te rm a 2p d e c re a s e s a n d th e r e fo r e In er2p also d e c re a s e s as th e o rd e r
o f th e A R m o d e l is in crease d . H o w e v e r, 2 p / N in c re a se s w ith an in crease in p.
H e n c e a m in im u m v alu e is o b ta in e d fo r so m e p.
A n a lte rn a tiv e in fo rm a tio n c rite rio n , p ro p o s e d by R is sa n e n (1983), is b ased
o n se lectin g th e o r d e r th a t m i n i mi z e s the description l ength (M D L ), w h e re M D L
is d efin ed as
M D L (/j) = N \ r \ a 2p + p i n N

(12.3.32)

A fo u rth c rite rio n has b e e n p ro p o s e d by P a rz e n (1974). T h is is called th e

criterion aut oregressive transf er (C A T ) fu n ctio n a n d is d e fin ed as
C A T (p) = ( I

_L ) - J -

(12.3.33)

w h ere
(12.3.34)
T h e o rd e r p is se le c te d to m in im ize C A T (/?).
In ap p ly in g th is crite ria , th e m e a n sh o u ld b e re m o v e d fro m th e d a ta . Since
a 2k d e p e n d s o n th e ty p e o f sp e c tru m e s tim a te w e o b ta in , th e m o d e l o rd e r is also
a fu n ctio n o f th e c riterio n .
T h e e x p e rim e n ta l re su lts given in th e re fe re n c e s ju s t c ite d in d ic a te th a t
th e m o d e l-o rd e r se lectio n c rite ria d o n o t y ield d efin itiv e re su lts. F o r ex am p le,
U lry ch an d B ish o p (1975), J o n e s (1976), a n d B e rry m a n (1978), fo u n d th a t th e
F P E ( p ) c rite rio n te n d s to u n d e re s tim a te th e m o d e l o rd e r. K a s h y a p (1980) sh o w ed
th a t th e A IC c rite rio n is sta tistically in c o n s is te n t as N oo. O n th e o th e r
h a n d , th e M D L in fo rm a tio n c rite rio n p ro p o s e d b y R issan n is statistically c o n
sisten t. O th e r e x p e rim e n ta l re su lts in d ic a te th a t fo r sm a d a ta len g th s, th e o r
d e r o f th e A R m o d e l sh o u ld b e se le c te d to b e in th e ra n g e N /3 to N f l fo r
g o o d resu lts. I t is a p p a r e n t th a t in th e a b se n c e o f any p r io r in fo rm a tio n r e
g ard in g th e p h y sical p ro c e ss th a t re su lte d in th e d a ta , o n e sh o u ld tr y d iffer
e n t m o d e l o r d e rs a n d d iffe re n t c rite ria an d , finally, c o n s id e r th e d iffe re n t
resu lts.

Sec. 12.3

933

Parametric Methods for Power Spectrum Estimation

12.3.7 MA Model for Power Spectrum Estimation

A s show n in S ectio n 12.3.1, th e p a r a m e te r s in a M A (^ ) m o d e l a re re la te d to th e
sta tistical a u to c o rre la tio n yxx(m) by (12.3.10). H o w e v e r,

(12.3.35)
w h e re th e co efficien ts [dm] a re re la te d to th e M A p a r a m e te r s by th e ex p ressio n
g-\m\

(12.3.36)

(12.3.37)
a n d th e p o w e r sp e c tru m fo r th e MA(< 7 ) p ro cess is
(12.3.38)
It is a p p a r e n t fro m th e se e x p ressio n s th a t w e d o n o t h av e to solve fo r th e M A
p a ra m e te rs {>*} to e stim a te th e p o w e r sp e c tru m . T h e e stim a te s o f th e a u to c o r re
latio n yxx( m) fo r |?| < q suffice. F ro m such e stim a te s w e c o m p u te th e e s tim a te d
M A p o w e r sp e ctru m , g iven as

(12.3.39)
w h ich is id e n tic a l to th e classical (n o n p a ra m e tric ) p o w e r sp e c tru m e stim a te d e
sc rib ed in S ectio n 12.1.
T h e re is an a lte rn a tiv e m e th o d fo r d e te rm in in g {bk} b a se d o n a h ig h -o rd e r
A R a p p ro x im a tio n to th e M A p ro cess. T o b e specific, let th e MA(<?) p ro cess b e
m o d e le d by an A R ( p ) m o d el, w h e re p > > q. T h e n B( z ) = 1 / A( z ) , o r eq u iv alen tly ,
B ( z ) A ( z ) = 1. T h u s th e p a ra m e te rs {bk} a n d {ak} a re r e la te d by a co n v o lu tio n sum ,
w h ich can b e e x p ressed as
n = 0
n t^O

(12.3.40)

w h ere {a} a re th e p a r a m e te r s o b ta in e d by fittin g th e d a ta to an A R ( p ) m odel.

A lth o u g h th is set o f e q u a tio n s c a n be easily solved fo r th e {*}, a b e tte r fit is
o b ta in e d b y u sin g a le a st-sq u a re s e r r o r c rite rio n . T h a t is, w e fo rm th e sq u a re d e r ro r
2

ao = l,

ak = 0,

k <0

(12.3.41)

which is minimized by selecting the MA(?) parameters {*}. The result of this

9 34

Power Spectrum Estimation

Chap. 12

minimization is
b = - R ~i
J r aa
w h ere th e e le m e n ts o f

(12.3.42)

a n d raa a re given as

p-tf-jl
R aa(\i ~ y |) =

22

i,j = l,2,...,q

n=0

(12.3.43)

p-i
raa 0 ) = 2 2

n+ '

I = 1, 2, . . . ,

T h is least sq u a re s m e th o d fo r d e te rm in in g th e p a r a m e te r s o f th e MA(<j)
m o d el is a ttr ib u te d to D u rb in (1959). I t h as b e e n sh o w n by K a y (1988) th a t this
e s tim a tio n m e th o d is a p p ro x im a te ly th e m ax im u m lik e lih o o d u n d e r th e a ssu m p tio n
th a t th e o b se rv e d p ro c e ss is G au ssian .
T h e o r d e r q o f th e M A m o d e l m a y b e d e te rm in e d e m p iric a lly b y se v era l
m e th o d s. F o r e x a m p le , th e A IC fo r M A m o d els h a s th e sa m e fo rm as fo r A R
m o d els,
A lC (tf) = l n < 4 + ^

(12.3.44)

w h ere a 2q is an e s tim a te o f th e v arian ce o f th e w h ite noise. A n o th e r a p p ro a c h ,

p ro p o se d by C h o w (1972b), is to filter th e d a ta w ith th e in v e rse MA(<y) filter an d
test th e filtered o u tp u t fo r w h iten ess.

12.3.8 ARMA Model for Power Spectrum Estimation

T h e B u rg alg o rith m , its v a riatio n s, a n d th e le a s t-s q u a re s m e th o d d e sc rib e d in th e
p rev io u s sectio n s p ro v id e re lia b le h ig h -re so lu tio n sp e c tru m e s tim a te s b a s e d o n
th e A R m o d el. A n A R M A m o d e l p ro v id e s us w ith an o p p o r tu n ity to im p ro v e on
th e A R sp e c tru m e s tim a te , p e rh a p s, by using fe w e r m o d e l p a ra m e te rs .
T h e A R M A m o d e l is p a rtic u la rly a p p r o p ria te w h e n th e sig n a l h as b e e n co r
ru p te d b y no ise. F o r e x am p le, su p p o s e th a t th e d a ta x ( n ) a re g e n e r a te d by an
A R sy stem , w h ere th e sy stem o u tp u t is c o rru p te d by ad d itiv e w h ite n o ise . T h e
z-tra n sfo rm o f th e a u to c o rre la tio n o f th e re s u lta n t signal can b e e x p re sse d as

r Az) = J u m ? T )+ "
_

a l +<t2A ( z ) A ( z

(12.3.45)
j)

w h ere cr2 is th e v a ria n c e o f th e a d d itiv e noise. T h e re fo re , th e p ro c e ss x ( n ) is

A R M A (/j, p), w h e re p is th e o rd e r o f th e a u to c o rre la tio n p ro c e ss . T h is re la
tio n sh ip p ro v id e s so m e m o tiv a tio n fo r in v estig atin g A R M A m o d e ls fo r p o w er
sp e c tru m estim a tio n .
A s w e h a v e d e m o n s tra te d in S ectio n 12.3.1, th e p r a m e te rs o f th e A R M A
m o d el a re re la te d to th e a u to c o rre la tio n b y th e e q u a ' on in (12.3.4). F o r lags

Sec. 12.3

Parametric Methods for Power Spectrum Estimation

935

|m | > q, th e e q u a tio n involves o n ly th e A R p a r a m e te r s {a*}. W ith e stim a te s

s u b s titu te d in p lace o f yxx(m), w e can solve th e p e q u a tio n s in (12.3.5) to o b ta in
o*. F o r h ig h -o rd e r m o d e ls, h o w ev er, th is a p p ro a c h is likely to yield p o o r e stim a te s
o f th e A R p a ra m e te rs d u e to th e p o o r e stim a te s o f th e a u to c o rre la tio n fo r larg e
lags. C o n s e q u e n tly , th is a p p ro a c h is n o t re c o m m e n d e d .
A m o re re lia b le m e th o d is t o c o n s tru c t an o v e rd e te rm in e d se t o f lin ear e q u a
tio n s fo r m > q, a n d to u se th e m e th o d o f le a st s q u a re s o n th e se t o f o v e r d e te r
m in e d e q u a tio n s , as p ro p o se d by C a d zo w (1979). T o e la b o ra te , su p p o se th a t
th e a u to c o rre la tio n se q u e n c e can b e a c c u ra te ly e s tim a te d u p to lag M, w h ere
M > p + q. T h e n w e can w rite th e follow ing se t o f lin e a r e q u a tio n s:

rxx(q + 1)

rxx(q ~ 1)
rxx(q)

rXx(q - p + 1) -] ~a\
o2
rxx(q - p + 2)

,rxx( M - 1)

rxx( M - 2 )

rxx( M - p)

rxx(q)

= _

~rxx(q + 1)~
rxx(q + 2 )

. r xx( M )

_
(12.3.46)

o r eq u iv a le n tly ,
(12.3.47)

Tr

R r r S ---

Since
is o f d im en sio n ( M q) x p, an d M q > p w e c a n use th e le a st-sq u a re s
c rite rio n to so lv e fo r th e p a r a m e te r v e c to r a. T h e re su lt o f th is m in im iz atio n is
a = -(R ' R r

(12.3.48)

T h is p ro c e d u re is called th e least-squares mo d i f i e d Y u l e - W a l k e r me t hod. A

w eig h tin g fa c to r can also b e a p p lie d to th e a u to c o rre la tio n se q u e n c e to d e e m p h a size th e less re lia b le e s tim a te s for larg e lags.
O n c e th e p a ra m e te rs fo r th e A R p a r t o f th e m o d e l h av e b e e n e stim a te d as
in d ic a te d ab o v e , w e h a v e th e system
p
A( z ) = 1 + ^
t= i

(12.3.49)

T h e se q u e n c e x ( n ) can n o w b e filte re d by th e F IR filter A ( z ) to yield th e se q u e n c e

p
v(n) = x (n ) + 2 2 & tx (n k)
*=i

n = 0 ,1 ,..., N 1

(12.3.50)

T h e c a scad e o f th e A R M A ( p , q) m o d e l w ith A ( z ) is a p p ro x im a te ly th e MA(<?)

p ro c e ss g e n e r a te d by th e m o d e l B{z). H e n c e w e can ap p ly th e M A e s tim a te given
in th e p re c e d in g se ctio n to o b ta in th e M A sp e c tru m . T o b e specific, th e filtered
se q u e n c e v(n) f o r p < n < N 1 is u se d to fo rm th e e s tim a te d c o rre la tio n se q u e n c e s
ruv( m), fro m w h ich w e o b ta in th e M A sp e c tru m

^ A( / ) = L

rw ( m ) e ~ ^

(12.3.51)

936

Power Spectrum Estimation

Chap. 12

F irst, w e o b se rv e th a t th e p a ra m e te rs {>*} a re n o t r e q u ire d to d e te r m in e th e p o w e r

sp e ctru m . S eco n d , w e o b se rv e th a t r(m ) is an e s tim a te o f th e a u to c o r re la tio n fo r
th e M A m o d e l g iv en by (12.3.10). In fo rm in g th e e stim a te rvv( m) , w eig h tin g (e.g.,
w ith th e B a rtle tt w in d o w ) m ay b e u se d to d e e m p h a s iz e c o rre la tio n e s tim a te s fo r
larg e lags. In a d d itio n , th e d a ta m ay b e filte re d by a b a c k w a rd filter, th u s cre a tin g
a n o th e r se q u e n c e , say v b(n), so th a t b o th v(n) a n d v b(n) can b e u se d in fo rm in g
th e e stim a te o f th e a u to c o rre la tio n r(m ), as p ro p o s e d by K a y (1980). F inally,
th e e s tim a te d A R M A p o w e r sp e c tru m is
pMA/
P * RMA( f ) = -----------v j L- l l l -----(12.3.52)

1+E

T h e p ro b le m o f o rd e r se lectio n fo r th e A R M A ( p , g ) m o d e l h as b e e n in v es
tig a te d by C how (1972a, b) a n d B ru z z o n e a n d K a v e h (1980). F o r th is p u rp o s e th e
m in im u m o f th e A IC in d ex
A I C ( p , q) = I n a l " + ~

(12.3.53)

can b e u sed , w h e re 6 l pq is a n e s tim a te o f th e v a ria n c e o f th e in p u t e rro r. A n

a d d itio n a l te s t o n th e a d e q u a c y o f a p a rtic u la r A R M A ( p , q) m o d e l is to filter th e
d a ta th ro u g h th e m o d e l a n d te s t fo r w h iten ess o f th e o u tp u t d a ta . T h is re q u ire s th a t
th e p a ra m e te rs o f th e M A m o d e l b e c o m p u te d fro m th e e s tim a te d a u to c o rre la tio n ,
u sing sp e ctral fa c to riz a tio n to d e te rm in e B(z) fro m D( z ) =
F o r a d d itio n a l re a d in g o n A R M A p o w e r s p e c tru m e s tim a tio n , th e r e a d e r is
re fe rre d to th e p a p e rs by G ra u p e e t al. (1975), C a d zo w (1981, 1982), K ay (1980),
an d F rie d la n d e r (1982b).

12.3.9 Some Experimental Results

In th is se ctio n w e p re s e n t so m e e x p e rim e n ta l re su lts on th e p e rfo rm a n c e o f A R
a n d A R M A p o w e r s p e c tru m e s tim a te s o b ta in e d b y u sin g artificially g e n e ra te d d ata.
O u r o b jectiv e is to c o m p a re th e s p e c tra l e stim a tio n m e th o d s o n th e basis o f th e ir
fre q u e n c y re so lu tio n , bias, a n d th e ir ro b u stn e s s in th e p re se n c e o f ad d itiv e noise.
T h e d a ta c o n sist o f e ith e r o n e o r tw o sin u so id s a n d a d d itiv e G a u s sia n noise.
T h e tw o sin u so id s a re sp a c e d A f a p a rt.
C lea rly , th e u n d e rly in g p ro cess is
A R M A ( 4 ,4). T h e re su lts th a t a re sh o w n em p lo y a n A R ( p) m o d e l fo r th ese
d a ta . F o r h ig h sig n a l-to -n o ise ra tio s (S N R s) w e e x p e c t th e A R ( 4 ) to b e a d e q u a te .
H o w e v e r, fo r low S N R s, a h ig h e r-o rd e r A R m o d e l is n e e d e d to a p p ro x im a te th e
A R M A ( 4 ,4) p ro cess. T h e re su lts given b e lo w a re c o n s iste n t w ith th is sta te m e n t.
T h e S N R is d efin ed as 101og10
w h e re a 2 is v a ria n c e o f th e ad d itiv e n oise
a n d A is th e a m p litu d e o f th e sinusoid.
In F ig. 12.6 w e illu stra te th e re su lts fo r N = 20 d a ta p o in ts b a s e d o n an A R (4 )
m o d e l w ith a S N R = 20 dB a n d A f = 0.13. N o te th a t th e Y u le - W a lk e r m e th o d
gives an ex tre m e ly sm o o th ( b ro a d ) sp e c tra l e s tim a te w ith sm a ll p e a k s . I f A f is

Sec. 12.3

937

Parametric Methods for Power Spectrum Estimation

Burg

-1 0

//II

II

LS

20 points
4 poles
SNR = 20 dB

E
3 5

.20

N-------- Yule-Walker ^

r I i i i i m j i i i i l i i i i j j n j i l - I i l l n i i i i i . i j l i i l i i - L l i i i x i j . j . i l i j i l j i i i i i i j i l i m i i i i i l n m i n | J | n t u n i ___ f

.22

.24

Figure 12.6

.26

.28
.30
.32
.34
Frequency (cycles/sample)

.36

.38

.40

Comparison of A R spectrum estimation methods.

d e c re a s e d to A f = 0.07, th e Y u le -W a lk e r m e th o d n o lo n g e r reso lv es th e p e a k s as
illu stra te d in Fig. 12.7. S o m e b ias is also e v id e n t in th e B u rg m e th o d . O f c o u rse , by
in c re a sin g th e n u m b e r o f d a ta p o in ts th e Y u le -W a lk e r m e th o d e v e n tu a lly is able
to reso lv e th e p eak s. H o w e v e r, th e B u rg a n d le a st-s q u a re s m e th o d s a re clearly
s u p e rio r fo r s h o rt d a ta reco rd s.
T h e effe ct o f ad d itiv e n oise o n th e e s tim a te is illu stra te d in Fig. 12.8 fo r th e
le a s t-s q u a re s m e th o d . T h e effect o f filter o rd e r o n th e B u rg a n d le a st-sq u a re s
m e th o d s is illu stra te d in F igs. 12.9 a n d 12.10, resp ec tiv ely . B o th m e th o d s ex h ib it
s p u rio u s p e a k s as th e o r d e r is in crease d to p = 12.
T h e effe ct o f in itial p h a se is illu stra te d in Figs. 12.11 a n d 12.12 fo r th e B u rg
an d le a s t-s q u a re s m e th o d s. I t is clear th a t th e le a st-sq u a re s m e th o d ex h ib its less
se n sitiv ity to in itial p h a s e th a n th e B u rg alg o rith m .
A n e x a m p le o f lin e sp littin g fo r th e B u rg m e th o d is sh o w n in Fig. 12.13 w ith
p = 12. It d o e s n o t o c c u r fo r th e A R (8 ) m o d el. T h e le a st-sq u a re s m e th o d d id
n o t e x h ib it lin e sp littin g u n d e r th e sa m e co n d itio n s. O n th e o th e r h a n d , th e line
sp littin g o n th e B u rg m e th o d d isa p p e a re d w ith an in crease in th e n u m b e r o f d a ta
p o in ts N .
F ig u re s 12.14 a n d 12.15 illu stra te th e re so lu tio n p r o p e rtie s o f th e B u rg an d
le a st-s q u a re s m e th o d s f o r A f = 0.07 a n d N = 20 p o in ts a t low S N R (3 dB ).
S ince th e ad d itiv e n o ise p ro c e ss is A R M A , a h ig h e r-o rd e r A R m o d e l is re q u ire d
t o p ro v id e a g o o d a p p ro x im a tio n a t low S N R . H e n c e th e fre q u e n c y re so lu tio n
im p ro v e s as th e o r d e r is in crease d .

Power Spectrum Estimation

Normalized power spectrum (dB)

938

Frequency (cycles/sample)

Normalized power spectrum

(dB)

Figure 12.7 Comparison of A R spectrum estimation methods.

Frequency (cycles/sample)
Figure 1Z8

Effect of additive noise on LS method.

Chap. 12

Parametric Methods for Power Spectrum Estimation

Normalized power spectrum (dB)

Sec. 12.3

Effect of filler order of Burg method.

Normalized power spectrum

(dB)

Figure 12.9

Figure 12.10

Effect of filter order

od

LS method.

939

Chap. 12

/
Frequency (cycles/sample)
Figure 12.11

Effect of initial phase on Burg method.

20 points
4 poles
SNR = 15 dB
/ , = -23
f l = -36

-6

16 sinusoid phases

-1 2

_ J4 r Hi ! I11111l1
.20
.22

.26

Figure 12.12

.28
.30
.32
.34
Frequency (cycles/sample)

.36

Effect of initial phase on LS m ethod.

.38

.40

Parametric Methods for Power Spectrum Estimation

Normalized power spectrum

(dB)

Sec. 12.3

Frequency (cycles/sample)
Line splitting in Burg method.

Normalized power spectrum (dB)

Figure 12.13

FigHre 12.14

Frequency resolution o f B urg m ethod w ith N = 20 points.

941

942

Power Spectrum Estimation

Chap. 12

Frequency (cycles/sample)
Figure 12,15

Frequency resolution of LS method with N 20 points.

T h e F P E fo r th e B u rg m e th o d is illu stra te d in Fig. 12.16 f o r a n S N R = 3 dB .

F o r th is S N R th e o p tim u m v a lu e is p = 12 a c co rd in g to th e F P E crite rio n .
T h e B u rg a n d le a st-sq u a re s m e th o d s w ere also te s te d w ith d a ta fro m a n a r
ro w b a n d p ro cess, o b ta in e d by exciting a fo u r-p o le (tw o p a irs o f c o m p le x -c o n ju g a te
p o les) n a rro w b a n d filter a n d se lectin g a p o rtio n o f th e o u tp u t se q u e n c e fo r th e d a ta
re c o rd . F ig u re 12.17 illu stra te s th e su p e rp o s itio n o f 20 d a ta re c o rd s o f 20 p o in ts
e ach . W e o b se rv e a relativ ely sm all v ariab ility . In c o n tra st, th e B u rg m e th o d
e x h ib ite d a m u ch la rg e r v ariab ility , a p p ro x im a te ly a fa c to r o f 2 c o m p a re d to th e
le a st-sq u a re s m e th o d . T h e re su lts sh o w n in F igs. 12.6 th ro u g h 12.17 a re ta k e n
fro m P o o le (1981).
F in ally , w e sh o w in Fig. 12.18 th e A R M A ( 1 0 ,10) sp e c tra l e s tim a te s o b ta in e d
by K ay (1980) fo r tw o sin u so id s in n o ise using th e le a s t-s q u a re s A R M A m e th o d
d e sc rib e d in S ectio n 12.3.8, as a n illu stra tio n o f th e q u a lity o f p o w e r sp e c tru m
e s tim a tio n o b ta in e d w ith th e A R M A m odel.

12.4 MINIMUM VARIANCE SPECTRAL ESTIMATION

T h e sp e c tra l e s tim a to r p r o p o s e d by C a p o n (1969) w as in te n d e d fo r u se in large
seism ic a rra y s fo r fre q u e n c y -w a v e n u m b e r e stim a tio n . I t w as la te r a d a p te d to
sin g le-tim e-series s p e c tru m e s tim a tio n b y L aco ss (1971), w h o d e m o n s tr a te d th a t

Sec. 12.4

943

Minimum Variance Spectral Estimation

l.l
1.0

0.9
0.8

t 0.7

|3 0.6
c

fe.0.5

| 0.4
0.3
0.2

0.1
0

6 8 10 12 14 16 18
Numberof poles

Normalized power spectrum (dB)

Figure 12.16 Final prediction error for

Burg estimate.

20points
4poles
; 1.;t = .95f^ - 23'
Zi

= ,95e *

20datasequences

-12

.20

.22

.24

Figure 12.17

.26

.28 .30 .32 .34

Frequency(cycles/sample)

.36

.38

Effect of starting point in sequence on LS m ethod.

.40

944

Chap. 12

Normalized power spectrum (dB)

Power Spectrum Estimation

Normalized power spectrum (dB)

Frequency (cycles/sample)

Figure 12.18 A R M A (10, 10) pow er

spectrum estim ates from p ap er by Kay
(1980). R eprin ted w ith perm ission from
the IE E E .

(b)

th e m e th o d p ro v id es a m in im u m v a ria n c e u n b ia s e d e stim a te o f th e sp e c tra l c o m

p o n e n ts in th e signal.
F o llo w in g th e d e v e lo p m e n t o f L aco ss, le t us c o n s id e r an F I R filter w ith co ef
ficients a*, 0 < k < p, to b e d e te rm in e d . U n lik e th e lin e a r p re d ic tio n p ro b le m , w e
d o n o t c o n stra in ao to b e u n ity . T h e n , if th e o b se rv e d d a ta * ( ) , 0 < n < N 1,
a re p assed th ro u g h th e filter, th e re sp o n s e is

p
y(n) = 21<ikx(n - k ) = X '(n )a
k=0

(12.4.1)

Sec. 12.4

Minimum Variance Spectral Estimation

9 45

X l (n) = [jr(n ) x ( n 1)
x ( n - p) ] is th e d a ta v e c to r a n d a is th e filter c o e f
ficien t v e c to r. If w e assu m e th a t [ * ( )] = 0, th e v arian ce o f th e o u tp u t se q u e n c e is
a 2 = [ |y ( n ) |2] = [a * 'X * (n )X ' (n)a]
(12.4.2)
= a*T a
w h e re T xx is th e a u to c o rre la tio n m a trix o f th e se q u e n c e jc(n), w ith e le m e n ts yxx(m).
T h e filter co efficien ts a re se le c te d so th a t a t th e fre q u e n c y f s, th e fre q u e n c y
re sp o n s e o f th e F I R filter is n o rm a liz e d to u n ity , th a t is,
; a ke - j2*k}' = 1
k= 0

T h is c o n s tra in t can also b e w ritte n in m a trix fo rm as

E * '( //) a = 1

(12.4.3)

w h e re
E '( / , ) = [ l

e*2**

ej2*pf' ]

B y m in im izin g th e v a ria n c e a 2 su b je c t to th e c o n s tra in t (12.4.3), w e o b ta in an

F IR filter th a t p a sse s th e fre q u e n c y c o m p o n e n t / / u n d is to rte d , w hile c o m p o n e n ts
d is ta n t fro m / / a re se v e re ly a tte n u a te d . T h e re su lt o f th is m in im iz a tio n is sh o w n
by L aco ss to le a d to th e co efficien t v e c to r
a = r - 1E*

E* ( /,)

(12.4.4)

If a is su b s titu te d in to (12.4.2), w e o b ta in th e m in im u m v arian ce

?
1
CTmin = ------------- i---------E 'o n r - 'E 'a , )

(12.4.5)

T h e ex p re ssio n in (12.4.5) is th e m in im u m v aria n c e p o w e r sp e c tru m e s tim a te

a t th e fre q u e n c y f s. B y ch an g in g / / o v e r th e ra n g e 0 < f t < 0.5, w e can o b ta in
th e p o w e r s p e c tru m e s tim a te . I t sh o u ld b e n o te d th a t a lth o u g h E ( / ) c h an g es w ith
th e ch o ice o f fre q u e n c y , F ^ 1 is c o m p u te d o n ly o n ce. A s d e m o n s tr a te d by L acoss
(1971), th e c o m p u ta tio n o f th e q u a d ra tic fo rm E '
c
a
n
be d o n e w ith a
sin g le D F T .
W ith an e s tim a te R o f th e a u to c o rre la tio n m a trix su b s titu te d in p la c e o f
T j j , w e o b ta in th e m in im u m v a ria n c e p o w e r s p e c tru m e s tim a te o f C a p o n as
p W ( f ) = ----------------------P (/)R ^ E * (/)

(12.4.6)

I t h a s b e e n sh o w n b y L aco ss (1971) th a t th is p o w e r sp e c tru m e s tim a to r yields

e s tim a te s o f th e sp e c tra l p e a k s p ro p o rtio n a l to th e p o w e r a t th a t fre q u e n c y . In
c o n s tra s t, th e A R m e th o d s d e sc rib e d in S ectio n 12.3 re s u lt in e s tim a te s o f th e
sp e c tra l p e a k s p ro p o rtio n a l to th e s q u a re o f th e p o w e r a t th a t freq u e n cy .
T h is m in im u m v a ria n c e m e th o d is basically a filter b a n k im p le m e n ta tio n fo r
th e sp e c tru m e s tim a to r. I t d iffers basically fro m th e filter b a n k in te r p re ta tio n o f
th e p e rio d o g ra m in th a t th e filter coefficients in th e C a p o n m e th o d a re o p tim ized .

94 6

Power Spectrum Estimation

Chap. 12

E x p e rim e n ts o n th e p e rfo rm a n c e o f this m e th o d c o m p a re d w ith th e p e r

fo rm a n c e o f th e B u rg m e th o d h av e b e e n d o n e by L acoss (1971) an d o th e rs. In
g e n e ra l, th e m in im u m v a ria n c e e stim a te in (12.4.6) o u tp e rfo rm s th e n o n p a ra m e tric
sp e c tra l e s tim a to rs in fre q u e n c y re so lu tio n , b u t it d o e s n o t p ro v id e th e high fre
q u en cy re so lu tio n o b ta in e d fro m th e A R m e th o d s o f B u rg a n d th e u n c o n s tra in e d
least sq u a res. E x te n siv e c o m p a riso n s b e tw e e n th e B u rg m e th o d a n d th e m ini
m u m v arian ce m e th o d h a v e b e e n m a d e in th e p a p e r by L aco ss. F u rth e rm o re ,
B u rg (1972) d e m o n s tra te d th a t fo r a k n o w n c o rre la tio n s e q u e n c e , th e m in im u m
v a rian ce sp e c tru m is re la te d to th e A R m o d e l sp e c tru m th ro u g h th e e q u a tio n
(12.4.7)
w h ere r*xR(f, k) is th e A R p o w e r sp e c tru m o b ta in e d w ith an A R (it) m o d el. T h u s
th e recip ro c al o f th e m in im u m v a ria n c e e stim a te is e q u a l to th e a v e ra g e o f th e
recip ro c als o f all sp e c tra o b ta in e d w ith A R ( k ) m o d e ls fo r 1 < k < p. Since
lo w -o rd e r A R m o d els, in g e n e ra l, d o n o t p ro v id e g o o d re so lu tio n , th e av erag in g
o p e ra tio n in (12.4.7) re d u c e s th e fre q u e n c y re so lu tio n in th e sp e c tra l estim a te .
H e n c e w e c o n clu d e th a t th e A R p o w e r sp e c tru m e stim a te o f o r d e r p is s u p e rio r
to th e m in im u m v a ria n c e e stim a te o f o rd e r p + 1.
T h e re la tio n sh ip given by (12.4.7) re p re se n ts a fre q u e n c y -d o m a in re la tio n sh ip
b e tw e e n th e C a p o n m in im u m v a ria n c e e s tim a te an d th e B u rg A R e stim a te . A
tim e -d o m a in re la tio n sh ip b e tw e e n th e se tw o e s tim a te s also can b e e s ta b lis h e d as
show n by M u sicu s (1985). T h is has led to a c o m p u ta tio n a lly efficien t alg o rith m
fo r th e m in im u m v a ria n c e e s tim a te .
A d d itio n a l re fe re n c e s to th e m e th o d o f C a p o n a n d c o m p a riso n s w ith o th e r
e s tim a to rs can b e fo u n d in th e lite ra tu re . W e cite th e p a p e rs o f C a p o n a n d G o o d
m a n (1971), M a rz e tta (1983), M a rz e tta a n d L a n g (1983, 1984), C a p o n (1983), a n d
M c D o n o u g h (1983).

12.5 EIGENANALYSIS ALGORITHMS FOR SPECTRUM ESTIMATION

In S ectio n 12.3.8 w e d e m o n s tra te d th a t an A R ( p ) p ro c e ss c o r ru p te d by ad d itiv e
(w h ite) n o ise is e q u iv a le n t to a n A R M A (/? , p ) p ro cess. In th is se c tio n w e c o n
sid e r th e sp ecial case in w hich th e signal c o m p o n e n ts a re sin u so id s c o r ru p te d by
ad d itiv e w h ite n o ise . T h e a lg o rith m s a re b a se d o n an e ig e n -d e c o m p o sitio n o f th e
c o rre la tio n m atrix o f th e n o ise -c o rru p te d signal.
F ro m o u r p re v io u s d isc u ssio n o n th e g e n e ra tio n o f sin u so id s in C h a p te r 4,
w e recall th a t a re a l sin u so id al signal can b e g e n e ra te d via th e d iffe re n c e e q u a tio n ,
x (n) = d]X(n 1) 02x ( n 2)

(12.5.1)

w h e re a\ = 2 c o s 2 jt/* , 02 = 1, a n d initially, x ( - l ) = 1, x ( 2) = 0. T h is system

h as a p a ir o f c o m p le x -c o n ju g a te p o le s ( a t f = f a n d f = f t ) a n d th e re fo re
g e n e ra te s th e sin u so id x(n) cos 2 n f kn, fo r n > f

Sec. 12.5

Eigenanalysis Algorithms for Spectrum Estimation

947

In g e n e ra l, a signal co n sistin g o f p sin u so id al c o m p o n e n ts satisfies th e d iffe r

e n c e e q u a tio n
2P

x( n) = - 2 2 a x (n ~ m )

(12.5.2)

m=l

a n d c o rre sp o n d s to th e sy stem w ith sy stem fu n ctio n

H{ z ) = --------------------

2p

(12.5.3)

..tZ

1
T h e p o ly n o m ia l

2p
A( z ) = 1 +

(12-5 '4 >

m=l

h a s 2 p ro o ts o n th e u n it circle w hich c o rre sp o n d to th e fre q u e n c ie s o f th e sinusoids.

N ow , su p p o s e th a t th e sinusoids a re c o rru p te d by a w h ite n o ise seq u en ce
ic(rt) w ith [ t i f ( n ) |2] = <r2. T h e n w e o b se rv e th a t
y( n) = x ( n ) + w( n )

( 12.5.5)

If w e s u b s titu te x( n ) = y ( n ) w( n) in (12.5.2), w e o b ta in
2p

y ( n ) - w( n) ~ - ^ [ . v ( n - m) - w( n - m) ]am
m=1
o r, eq u iv a le n tly ,

2p

2p
T

amy( n m ) Y J omw( n m)

m=0

(12.5.6)

m=0

w h e re , by d efin itio n , a0 = 1.
W e o b se rv e th a t (12.5.6) is th e d ifferen ce e q u a tio n fo r a n A R M A ( p , p ) p r o
cess in w h ich b o th th e A R a n d M A p a ra m e te rs a r e id e n tic a l. T h is sy m m etry is a
c h a ra c te ristic o f th e sin u so id al signals in w h ite n o ise. T h e d iffe re n c e e q u a tio n in
(12.5.6) m ay b e e x p re sse d in m atrix fo rm as
Y 'a = W 'a

(12.5.7)

w h e re Y r = [y (n ) y ( n 1) y ( n - 2 p ) ] is th e o b s e rv e d d a ta v e c to r o f d i
m e n s io n (2p 4 - 1 ), W ' = [ wi n ) w( n 1) w( n - 2p ) ] is th e n o ise vector,
a n d a = [ 1 a\ ct2P ] is th e c o effic ie n t v e cto r.
If w e p re m u ltip ly (12.5.7) by Y a n d ta k e th e e x p e c te d v a lu e , w e o b ta in
E ( Y Y > = ( Y W ') a = [ ( X -I- W )W r]a
(12.5.8)
r yya = < a
w h e re w e h a v e u se d th e a ssu m p tio n th a t th e s e q u e n c e w( n ) is z e ro m e a n a n d
w h ite , a n d X is a d e te rm in is tic signal.

Power Spectrum Estimation

9 48

Chap. 12

T h e e q u a tio n in (12.5.8) is in th e fo rm o f a n e ig e n e q u a tio n , th a t is,

( I V - < r 2I ) a = 0

(12.5.9)

w h e re c 2 is an eig e n v a lu e o f th e a u to c o rre la tio n m a trix T >7. T h e n th e p a ra m e te r

v e c to r a is an e ig e n v e c to r a s so ciated w ith th e eig e n v a lu e cr2. T h e e ig e n e q u a tio n
in (12.5.9) form s th e basis fo r th e P isa re n k o h a rm o n ic d e c o m p o s itio n m e th o d .

12.5.1 Pisarenko Harmonic Decomposition Method

F o r p ra n d o m ly -p h a se d sin u so id s in ad d itiv e w h ite n o ise , th e a u to c o r re la tio n val
u es are
y ( 0 ) = l + Y 2 Pi
(12.5.10)
Yyy(k) = 2 2

COS^ n f , k

k it 0

w h e re Pj = A ] / 2 is th e a v e ra g e p o w e r in th e ith sin u so id an d A, is th e c o rre sp o n d

in g am p litu d e. H e n c e w e m ay w rite
c o s 2 jt/i

c o s 2 tt f i

cos 4 ;r/i

c o s 4 j t /2

.c o s 2 n p f i

cos 2 n f p n r p r
cos 4 n f p
P2

cos 2 ttp /2

cos 2:

-Pp-

r y .w d ) n
Yyy( 2)
(12.5.11)
- Y y y (p ) -

If w e k n o w th e fre q u e n c ie s f j , 1 < i < p , w e can u se this e q u a tio n to d e te rm in e

th e p o w e rs o f th e sin u so ids. In p la c e o f yxx(m), w e use th e e s tim a te s rxx(m). O n c e
th e p o w e rs a re k n o w n , th e n o ise v a ria n c e c a n b e o b ta in e d fro m (12.5.10) as
p

''v v ( O )

y.p,

(12.5.12)

1=1

T h e p ro b le m th a t re m a in s is to d e te rm in e th e p fre q u e n c ie s /, 1 < 1 <

p, w hich , in tu rn , re q u ire k n o w led g e o f th e e ig e n v e c to r a c o rre sp o n d in g to th e
e ig e n v a lu e cr2. P isa re n k o (1973) o b se rv e d [see also P a p o u lis (1984) a n d G re n a n d e r
an d S zego (1958)] th a t fo r an A R M A p ro c e ss co n sistin g o f p sin u s o id s in ad d itiv e
w h ite n o ise , th e v a ria n c e a 2 c o rre sp o n d s to th e m in im u m e ig e n v a lu e o f r vv w hen
th e d im en sio n o f th e a u to c o rre la tio n m a trix e q u a ls o r e x cee d s (2p + 1 ) x (2p + 1 ) .
T h e d e s ire d A R M A co efficien t v e c to r c o rre sp o n d s to th e e ig e n v e c to r asso ciated
w ith th e m in im u m eig en v alu e. T h e re fo re , th e fre q u e n c ie s / , , 1 < i < p are
o b ta in e d fro m th e ro o ts o f th e p o ly n o m ia l in (12.5.4), w h e re th e c o effic ie n ts are
th e e le m e n ts o f th e e ig e n v e c to r a c o rre sp o n d in g to th e m in im u m eig e n v a lu e <r2.
In su m m ary , th e P isa re n k o h a rm o n ic d e c o m p o s itio n m e th o d p ro c e e d s as
follow s. F irst w e e s tim a te r yy fro m th e d a ta (i.e., w e fo rm th e a u to c o rre la
tio n m a trix R yy). T h e n w e find th e m in im u m e ig e n v a lu e a n d th e c o rre sp o n d in g
m in im u m e ig en v ecto r. T h e m in im u m e ig e n v e c to r y ield s th e p a r a m e te r s o f th e

Sec. 12.5

Eigenanalysis Algorithms for Spectrum Estimation

949

A R M A ( 2 /? ,2 p ) m o d el. F ro m (12.5.4.) w e can c o m p u te th e ro o ts th a t c o n s titu te

th e fre q u e n c ie s {/,}. B y using th e se fre q u e n c ie s, w e can solve (12.5.11) fo r th e
sig n al p o w e rs {/,} by su b stitu tin g th e e stim a te s r yy(m) fo r y vy(m).
A s w ill b e se e n in th e follow ing ex am p le, th e P is a re n k o m e th o d is b a s e d o n
th e u se o f a n o ise su b sp ace e ig e n v e c to r to e stim a te th e fre q u e n c ie s o f th e sinusoids.
Exam ple 125.1
Suppose that we are given the autocorrelation values yvv,(0) = 3, yv,( l) = 1, and
Yyy (2) = 0 for a process consisting of a single sinusoid in additive white noise. D eter
mine the frequency, its power, and the variance of the additive noise.
Solution

The correlation matrix is

r3
1
-0

i o3 1
1 3J

The minimum eigenvalue is the smallest root of the characteristic polynomial

-3 -A

g ( k ) =

0
X

= (3 - k)(k2 - 6X + 7} = 0

3- k j

T herefore, the eigenvalues are A.) = 3, k2 = 3 + ->/2. ^ = 3 - \ f l .

The variance of the noise is
a] = Amjn = 3 - V 2
The corresponding eigenvalue is the vector that satisfies (12.5.9), that is.
-V 2
1
. 0

y/2

y /2 .

- 1 -

'

-o=

0
.0 .

-02-

The solution is ai = - y / 2 . and a2 = \.

The next step is to use the value aj and a2 to determ ine the roots of the
polynomial in (12.5.4). W e have
s 2 - n/2 z +

1= 0

Thus
1 . . 1
V2
J V2

Zl Z2

Note that |zi| = |z2| = 1, so that the roots are on the unit circle. The corresponding
frequency is obtained from
Z; =*'"

1
s / 2 + y V2

which yields / i = | . Finally, the pow er of the sinusoid is

P \ COS 2 n f l

Yyy ( 1 ) =

/>! = V 2

and its amplitude is A = J 2P X = \/2v^.

950

Power Spectrum Estimation

Chap. 12

As a check on our computations, we have

a l = y,.v(0) - Pi
= 3 - -Jl
which agrees with An,jn.

12.5.2 Eigen-decomposition of the Autocorrelation Matrix

for Sinusoids in White Noise
In th e p rev io u s d iscu ssio n w e assu m ed th a t th e sin u so id al signal co n sists o f p real
sinusoids. F o r m a th e m a tic a l co n v e n ie n c e w e shall n o w a ssu m e th a t th e signal
co n sists o f p co m p lex sin u so id s o f th e fo rm
p

x(n) =

22

A ,eJ(2,t/n+,f,

(12.5.13)

;=i
w h ere th e a m p litu d e s {A;} a n d th e fre q u e n c ie s j / ;- } a re u n k n o w n a n d th e p h ases
{<f>, } a re statistically in d e p e n d e n t ra n d o m variab les u n ifo rm ly d is trib u te d o n (0, 2n ) .
T h e n th e ra n d o m p ro cess x ( n ) is w id e-sen se s ta tio n a ry w ith a u to c o r re la tio n fu n c
tion
y ,,( m ) =

22

Piei2nf'm

(12.5.14)

/=!
w h ere, fo r co m p lex sin u so id s, P, = A ? is th e p o w e r of th e / th sin u so id .
Since th e se q u e n c e o b se rv e d is y( n) = x ( m ) + w ( n ), w h e re w{n) is a w h ite
n o ise se q u en ce w ith sp e c tra l d e n sity erj, th e a u to c o rre la tio n fu n c tio n fo r v (n) is
Yyy(m) = yIX( m ) + a l S ( m )

m = 0, 1 , . . . , { M - 1)

(12.5.15)

H e n c e th e M x M a u to c o rre la tio n m a trix fo r y ( n ) can be e x p re ss e d as

r Vv = r +

ctu
2i

(12.5.16)

w h ere r fJt is th e a u to c o rre la tio n m atrix fo r th e signal x ( n ) a n d a l l is th e a u to c o r

re la tio n m atrix fo r th e n o ise. N o te th a t if select M > p , T IX w hich is o f d im en sio n
M x M is n o t o f full ra n k , b e c a u se its ra n k is p. H o w e v e r, T vv is full ra n k b ecau se
<7*1 is o f ra n k M.
In fact, th e signal m a trix T** can be re p re s e n te d as
p

T xx =

22

(12.5.17)

1=1

w h ere H d e n o te s th e c o n ju g a te tra n sp o s e an d s, is a signal v e c to r o f d im e n sio n M

d efin ed as
s,- = f l , ei7* f \ e iA*{\

l!* ]

(12.5.18)

Since ea c h v e c to r ( o u te r p ro d u '0 s ,s f is a m atrix o f ra n k 1 a n d sin ce th e re a re p

v e c to r p ro d u c ts, th e m a trix T *, is o f ra n k p. N o te th a t if th e sin u so id s w ere real,
th e c o rre la tio n m a trix T JJf h a s ra n k 2 p.

Sec. 12.5

Eigenanalysis Algorithms for Spectrum Estimation

951

N o w , le t u s p e rfo rm an eig en -d e c o m p o sitio n o f th e m atrix

L e t th e
eig e n v a lu e s {A/} b e o r d e re d in d e creasin g v a lu e w ith k \ > Aj > A,3 > - > k M an d
le t th e c o rre sp o n d in g e ig e n v e c to rs b e d e n o te d as {v,, i = 1 , . . . , M}. W e assu m e
th a t th e e ig e n v e c to rs a re n o rm a liz e d so th a t vj 1 v, =
. In th e a b se n c e o f n oise
th e eig e n v a lu e s A., , i = 1, 2 , . . . , p, a re n o n z e ro w hile k p+i = k p+2 = =
=
0. F u rth e rm o re , it follow s th a t th e signal c o rre la tio n m a trix can b e e x p ressed
as
p

r =

(12.5.19)
/=i

T h u s, th e e ig e n v e c to rs v(, i = 1, 2 , . . . , p sp a n th e sig n a l su b sp ace as do th e

sig n al v ecto rs s, , i = 1, 2 , . , . , p. T h e se p e ig en v ecto rs fo r th e signal su b sp ace a re
called th e pr i nc i pal eigenvectors a n d th e c o rre sp o n d in g e ig e n v a lu e s a re called th e
pri nc i pal eigenvalues.
In th e p re se n c e o f n o ise , th e n o ise a u to c o rre la tio n m a trix in (12.5.16) can b e
re p re s e n te d as
M
o]\ = al 2 2 W ?
(12.5.20)
/=i
B y s u b s titu tin g (12.5.19) a n d (12.5.20) in to (12.5.16), w e o b ta in

r w = 2 2 k *iVil + 2 2 ^ y y ^
1=1
1=1
(12.5.21)
= 2 2 (k +
1=1

2 2 fT- v' v/ /

i=p+i

T h is e ig e n -d e c o m p o sitio n se p a ra te s th e eig e n v e c to rs in to tw o sets. T h e se t {v,, i =

1, 2 , . . . , p], w h ich a re th e p rin cip al eig en v ecto rs, span th e signal su b sp ace, w hile
th e se t {v,, i = p + \ .........M] , w hich a re o rth o g o n a l to th e p rin c ip a l e ig en v ecto rs,
a re said to b e lo n g to th e n o ise su b sp ace. S ince th e signal v e c to rs {s,, / = 1 ,2 , , p]
a re in th e sig n al su b sp ace, it follow s th a t th e {s, } a re sim p ly lin e a r co m b in a tio n s
o f th e p rin cip al e ig e n v e c to rs a n d a re also o rth o g o n a l to th e v e c to rs in th e n o ise
su b sp ace.
I n th is c o n te x t w e se e th a t th e P isa re n k o m e th o d is b a s e d o n an e s tim a tio n o f
th e fre q u e n c ie s by u sin g th e o rth o g o n a lity p ro p e rty b e tw e e n th e signal v ecto rs a n d
th e v ecto rs in th e n o ise su b sp ace. F o r com plex sin u so id s, if w e se lect M = p + 1
(fo r re a l sin u so id s w e se lect M = 2 p + 1), th e r e is only a single e ig e n v e c to r in
th e n o ise su b sp a c e (c o rre s p o n d in g to th e m in im u m e ig e n v a lu e ) w hich m u st b e
o rth o g o n a l to th e signal vectors. T h u s w e h av e
p
s f vp+i = 2 2 i p + i t t + l ) e ' j2nf'k = 0
i=0

i = 1,2,..., p

(12.5.22)

But (12.5.22) implies that the frequencies {/;} can be determined by solving for

Power Spectrum Estimation

9 52

Chap. 12

th e z e ro s o f th e p o ly n o m ial
p

V ( z ) = J 2 vp + ^ k + 1^ ~ k
n=0

(12.5.23)

T h e an g les o f th e s e r o o ts a re 2n f , , i = 1,

all o f w hich lie o n th e u n it circle.

2....... PW h e n th e n u m b e r o f sin u so id s is u n k n o w n , th e d e te rm in a tio n o f p m ay

p ro v e to b e d ifficult, esp ecially if th e signal level is n o t m u ch h ig h e r th a n th e noise
level. In th e o ry , if M > p + 1, th e re is a m u ltip lic ity (M - p ) o f th e m in im u m
eig en v alu e. H o w ev er, in p ra c tic e th e (Afp) sm all e ig en v alu es o f R vv will p ro b a b ly
b e d ifferen t. B y co m p u tin g all th e eig en v alu es it m ay b e p o ssib le to d e te rm in e p
by g ro u p in g th e M p sm all (n o ise) e ig en v alu es in to a se t a n d a v e rag in g th e m to
o b ta in an e stim a te o f cr*. T h e n , th e a v erag e v alu e can b e u se d in (12.5.9) along
w ith R yj, to d e te rm in e th e c o rre sp o n d in g eig en v ecto r.

12.5.3 MUSIC Algorithm

T h e m u ltip le sig n al classification (M U S IC ) m e th o d is also a n o ise su b sp ace fre
q u en cy estim ato r. T o d e v e lo p th e m e th o d , le t us first c o n s id e r th e w e ig h te d
sp e c tra l estim ate
M
P (f)=

2 2 w*ls " ( / ) v*i2

*=/>+i

w h ere {v*, jt = p + 1 , ___ M } are th e e ig e n v e c to rs in th e n o ise su b sp ace,

a se t o f p o sitiv e w eig h ts, a n d s ( / ) is th e co m p lex sin u so id al v e c to r
s( / ) =

(12.5.24)
{u;*)

a re

(12.5.25)

N o te th a t at / = / ; , s ( / ,) = s,, so th a t a t an y o n e o f th e p sin u so id a l fre q u e n c y

c o m p o n e n ts o f th e signal, w e h av e
/> (/,) = 0

i = 1,2,...,/?

(12.5.26)

H e n c e , th e recip ro c al o f P ( f ) is a sh a rp ly p e a k e d fu n c tio n o f fre q u e n c y a n d p r o

v id es a m e th o d fo r e stim a tin g th e fre q u e n c ie s o f th e sin u so id al c o m p o n e n ts . T h u s
1

1
22
*=p + i

(12.5.27)

I ( / ) n |

A lth o u g h th e o re tic a lly 1 / P ( f ) is infinite a t / = / , , in p ra c tic e th e e s tim a tio n e rro rs

re su lt in fin ite v a lu e s fo r 1 / P ( f ) a t all fre q u e n c ie s.
T h e M U S IC sin u so id al fre q u e n c y e s tim a to r p ro p o s e d by S ch m id t (1981,
1986) is a sp ecial case o f (12.5.27) in w h ich th e w eig h ts wk 1 f o r all k. H e n c e
^ m u s i c ( / ) = j j --------------------

22
k=p+1

|s H( / ) * |2

(12.5.28)

Sec. 12.5

Eigenanaiysis Algorithms for Spectrum Estimation

9 53

T h e e s tim a te o f th e sin u so id al fre q u e n c ie s a re th e p e a k s o f P m u sic C/)- O n c e th e

sin u so id a l fre q u e n c ie s a re e s tim a te d , th e p o w e r o f ea c h o f th e sin u so id s can b e
o b ta in e d by so lv in g (12.5.11).

12.5.4 ESPRIT Algorithm

E S P R IT (e s tim a tio n o f signal p a ra m e te rs via ro ta tio n a l in v a ria n c e te c h n iq u e s ) is
y e t a n o th e r m e th o d fo r e stim a tin g fre q u e n c ie s o f a su m o f sin u so id s by use o f
a n e ig e n -d e c o m p o sitio n a p p ro a c h . A s w e o b se rv e fro m th e d e v e lo p m e n t th a t
follow s, w h ich is d u e to R o y e t al. (1986), E S P R IT ex p lo its an u n d e rly in g r o t a
tio n a l in v a ria n c e o f sig nal su b sp a c e s sp a n n e d by tw o te m p o ra lly d isp lac ed d a ta
v ecto rs.
W e ag ain c o n sid e r th e e stim a tio n o f p c o m p lex -v alu ed sin u so id s in ad d itiv e
w h ite n o ise . T h e re c e iv e d se q u e n c e is given by th e v e c to r
y( n) = [?( ). y( n + 1 ) , . . . , y ( n + M - 1)]'
(12.5.29)

= x(n) + w (n)
w h e re x (n ) is th e signal v e c to r a n d w (n) is th e n oise v ecto r. T o e x p lo it th e d e
te rm in is tic c h a ra c te r o f th e sinusoids, w e d efin e th e tim e-d isp laced v e c to r z( n) =
y (n + 1). T h u s
z( n) = [z(n), z(n + 1).........z( n + M - 1)]'
(12.5.30)
= [;y(n + 1), y( n + 2 ).........y( n + Af)]'
W ith th e s e d efin itio n s we can ex p ress th e v ecto rs y(n) a n d z(n ) as
y (rt) = S a + w (n)
(12.5.31)
i (n ) = S $ a + w (n)
w h e re a = [ a \ , a i , . . ap]r, cii = A
, , a n d $ is a d ia g o n a l p x p m a trix c o n sist
in g o f th e re la tiv e p h a s e b e tw e e n a d ja c e n t tim e sa m p le s o f each o f th e co m p lex
sin u so id s,
$ = d ia g [ e '2,r/l, ej2jrf2, . . . , ei 2nf>]

(12.5.32)

N o te th a t th e m a trix & re la te s th e tim e-d isp laced v e c to rs y( n) a n d z( n) a n d can

b e called a r o ta tio n o p e ra to r. W e also n o te th a t <l> is u n ita ry . T h e m a trix S is th e
Af x p V a n d e rm o n d e m a trix specified b y th e co lu m n v ecto rs
s, = [1, e}2*fl , ejAnf , . . . ,

i = 1 , 2 .........p

(12.5.33)

N o w th e a u to c o v a ria n c e m a trix fo r th e d a ta v e c to r y (n) is

r y, = E [y (n )y " (n )]

(12.5.34)
= S P S W+ a l l

954

Power Spectrum Estimation

Chap. 12

w h e re P is th e p x p d ia g o n a l m atrix co n sistin g o f th e p o w e rs o f th e co m p lex

sin u so id s,
P = d ia g [|a i |2, \a2\2, - . . , \ap |2]
(12.5.35)
= d ia g [ f i, P2.........Pp]
W e o b se rv e th a t P is a d iag o n al m a trix since co m p lex sin u so id s o f d iffe re n t
fre q u e n c ie s a re o rth o g o n a l o v e r th e in fin ite in terv al. H o w ev er, w e sh o u ld e m p h a
size th a t th e E S P R IT a lg o rith m d o e s n o t re q u ire P to b e d ia g o n a l. H e n c e th e
alg o rith m is a p p lic a b le to th e case in w hich th e c o v a ria n c e m a tr ix is e s tim a te d
fro m fin ite d a ta reco rd s.
T h e cro ssco v arian ce m a trix o f th e signal v e c to rs y(n) a n d z (tt) is
T v. = [ y ( n ) z ff(n)] = S P $ " S " + T w

(12.5.36)

w h ere
I V = [ w (n ) w w(n + 1)]

(12.5.37)

0 0

T h e a u to an d cro ssco v arian ce m a tric e s r vv a n d T v; a re given as

r

rvv =

yw (0)
y ;v(D

yyy(l)
Yyy(0)

yvv(A/ - 1) "1
Yyyi M - 2 )
(12.5.38)

- y ; y ( M - 1)
r

K w (i)
yyy(0)

Yyy(M - 2)

Yyy (2)
yyy(l)

Yyy( 0)

Yyv(M) -1
Yyy(M -l)
(12.5.39)

- Yyy ( M 2)

Y'yy( M - 3)

yyy(l)

w h ere Yyy(m) = [y * (n )y (n + m )]. N o te th a t b o th T yy a n d 1%, a re T o e p litz m atric es.

B a se d o n th is fo rm u la tio n , th e p ro b le m is to d e te rm in e th e fre q u e n c ie s {ft)
a n d th e ir p o w ers {/*,} fro m th e a u to c o rre la tio n se q u e n c e [yyy(m)}.
F ro m th e u n d e rly in g m o d e l, it is c le a r th a t th e m a trix S P S W h a s ra n k p.
C o n se q u e n tly , T vv given by (12.5.34) h a s (M - p) id en tical e ig e n v a lu e s e q u a l to
a 2. H e n c e
T yy - a l l = S P S W = C yy

(12.5.40)

F ro m (12.5.36) w e also h av e
- a 2T w = S P * HS H m C

(12.5.41)

N o w , le t us c o n s id e r th e m a trix C yy XCyz, w h ich can b e w ritte n as

Cyy - k C yi = SP(I - \ * H)S H

(12.5.42)

Sec. 12.5

Eigenanalysis Algorithms for Spectrum Estimation

955

C lea rly , th e c o lu m n sp a ce o f S P S W is id en tical to th e co lu m n space o f S P $ HS W.

C o n s e q u e n tly , th e ra n k o f C vv - XCy: is e q u a l to p. H o w e v e r, w e n o te th a t if
X = ex p (;'2 7 r/(), th e rth ro w o f (I
is z e ro an d , h e n c e th e ra n k o f [I
ex p O 2 ^ - /) ] is p 1. B u t A, = e x p { j 2 n f i ) , i = 1, 2 , . . . , p, a re th e g en eralized
e ig en v alu es o f th e m a trix p a ir (C yy, C n ). T h u s th e p g e n e ra liz e d eig en v alu es {A, }
t h a t lie o n th e u n it circle c o rre sp o n d to th e e le m e n ts o f th e r o ta tio n o p e ra to r
3>. T h e re m a in in g M p g e n e ra liz e d eig en v alu es o f th e p a ir {Cvv, C v;} w hich
c o rre sp o n d to th e c o m m o n null sp a ce o f th e se m a tric e s, a re z e ro [i.e., th e ( M p)
e ig e n v a lu e s a re a t th e o rig in in th e co m p lex p lan e].
B a sed o n th e s e m a th e m a tic a l re la tio n sh ip s w e can fo rm u la te an alg o rith m
( E S P R IT ) fo r e s tim a tin g th e fre q u e n c ie s {/]. T h e p ro c e d u re is as follow s:
1. F ro m th e d a ta , c o m p u te th e a u to c o rre la tio n valu es r yy(nt), m = 1 , 2 ........ M ,
a n d fo rm th e m a tric e s R vv a n d R v- c o rre sp o n d in g to e stim a te s o f r vv an d

rv:
2. C o m p u te th e e ig en v alu es o f R vv. F o r M > p, th e m in im u m eig en v alu e is an
e stim a te o f a 2.
3. C o m p u te C vv = R vv a * I an d C v- = R vc <7*Q, w h e re Q is defin ed in
(12.5.37).
4. C o m p u te th e g e n e ra liz e d eig en v alu es o f th e m a trix p a ir (C TT, C v- |. T he p
g e n e ra liz e d e ig en v alu es o f th e s e m atric es th a t lie o n (o r n e a r) th e u n it circle
d e te rm in e th e ( e stim a te ) e le m e n ts of $ a n d h e n c e th e sin u so id al freq u en cies.
T h e re m a in in g M p eig e n v a lu e s will lie at ( o r n e a r) th e origin.
O n e m e th o d fo r d e te rm in in g th e p o w e r in th e sin u so id al c o m p o n e n ts is to
solve th e e q u a tio n in (12.5.11) w ith ryy{m) s u b s titu te d fo r y yy(m).
A n o th e r m e th o d is b a se d on th e c o m p u ta tio n o f th e g e n e ra liz e d eig en v ecto rs
{v,} c o rre sp o n d in g to th e g e n eralized e ig en v alu es {A, }. W e h av e
(C vv - A., c v:)v, = S P (I - X/

= 0

(12.5.43)

S in ce th e co lu m n sp a ce o f ( Cyyk ( - Cyz) is id en tical to th e co lu m n space sp a n n e d by

th e v e c to rs {S;, j ^ /} g iven by (12.5.33), it follow s th a t th e g e n e ra liz e d eig e n v e c to r
v, is o rth o g o n a l to sj5 j ^ i. S ince P is d iag o n al, it follow s fro m (12.5.43) th a t th e
sig n al p o w e rs a re
yh C v
Pi = ~ n r i r
|V " S ,|2

i =

( 12 .5 .44)

12.5.5 Order Selection Criteria

T h e eig e n a n a ly sis m e th o d s d e sc rib e d in th is se ctio n fo r e stim a tin g th e freq u e n cies
a n d th e p o w ers o f th e sin u so id s, also p ro v id e in fo rm a tio n a b o u t th e n u m b e r o f
sin u so id al c o m p o n e n ts . If th e r e a re p sin u so id s, th e e ig e n v a lu e s a s so ciated w ith th e

956

Power Spectrum Estimation

Chap. 12

signal su b sp ace a re [kj+cr*, i 1 , 2 . . . , / >} w hile th e re m a in in g {M p ) eig en v alu es

are all eq u al to a 2 . B a sed o n th is e ig e n v a lu e d e c o m p o sitio n , a te s t can b e d esigned
th a t c o m p a re s th e eig en v alu es w ith a specified th re sh o ld . A n a lte rn a tiv e m e th o d
also u ses th e e ig e n v e c to r d e c o m p o s itio n o f th e e s tim a te d a u to c o rre la tio n m atrix
of th e o b se rv e d sig n al a n d is b a se d o n m a trix p e rtu rb a tio n analysis. T h is m e th o d
is d e c rib e d in th e p a p e r by F u c h s (1988).
A n o th e r a p p ro a c h b a se d o n an e x te n sio n a n d m o d ifica tio n o f th e A I C crite
rio n to th e e ig e n -d e c o m p o sitio n m e th o d , h a s b e e n p ro p o se d by W ax an d K ailath
(1985). If th e eig en v alu es o f th e sa m p le a u to c o rre la tio n m a trix a r e ra n k e d so th a t
^i > k 2 > > k M , w h e re M > p, th e n u m b e r o f sin u so id s in th e signal su b sp ace
is e s tim a te d b y se lectin g th e m in im u m v alu e o f M D L (p ), given as
M D L (p ) = log

G(p)
M

p)

+ E(p)

(12.5.45)

w h ere
g (p

)=

p = 0 , 1 .........M - l

M p

A(p) =

1
M

//>+]

(12.5.46)

1
E ( P) = 2 p(-2 M ~ P')loZ N
N: n u m b e r o f sa m p le s u sed to e s tim a te th e M
a u to c o rre la tio n lags
S o m e resu lts o n th e q u ality o f th is o r d e r se le c tio n c rite rio n a re given in th e p a p e r
b y W ax an d K ailath (1985). T h e M D L c rite rio n is g u a ra n te e d to b e co n sisten t.

12.5.6 Experimental Results

In th is sectio n w e illu stra te w ith an ex a m p le , th e re so lu tio n c h a ra c te ristic s o f th e
eig en an aly sis-b ase d sp e c tra l e s tim a tio n a lg o rith m s a n d c o m p a re th e ir p e rfo rm a n c e
w ith th e m o d el-b ased m e th o d s an d n o n p a ra m e tric m e th o d s. T h e signal se q u e n c e
is
4

x ( n ) = 2 2 A iej a n f 'n+4,') + w( n)
i=l
w h e re A, = 1, i = 1, 2, 3, 4, {tp,} a re sta tistically in d e p e n d e n t ra n d o m v ariab les
u n ifo rm ly d istrib u te d on (0, 2j t ), {w(rt)) is a z e ro -m e a n , w h ite n o ise se q u e n c e w ith
v a ria n c e a 2, an d th e fre q u e n c ie s a re f \ = 0.222, f a 0.166, f s = 0.10, an d
f i = 0.122. T h e se q u e n c e {jt(n), 0 < n < 1023} is u se d to e s tim a te th e n u m b e r
o f fre q u e n c y c o m p o n e n ts a n d th e c o rre sp o n d in g v alu es o f th e ir fre q u e n c ie s for
a 2 = 0 . 1 , 0.5, 1.0, a n d M 12 (le n g th o f th e e s tim a te d a u to c o rre la tio n ).

Sec. 12.5

Eigenanalysis Algorithms for Spectrum Estimation

957

Frequency
Figure 12.19

Power spectrum estimates from Blackman-Tukey method.

F ig u re s 12.19, 12.20, 12.21, a n d 12.22 illu stra te th e e s tim a te d p o w e r sp e c tra

o f th e signal u sin g th e B la c k m a n -T u k e y m e th o d , th e m in im u m v a ria n c e m e th o d
o f C a p o n , th e A R Y u le -W a lk e r m e th o d , a n d th e M U S IC a lg o rith m , resp ec tiv ely .
T h e re su lts fro m th e E S P R IT a lg o rith m a re given in T a b le 12.2. F ro m th e s e resu lts
it is a p p a r e n t th a t (1) th e B la c k m a n -T u k e y m e th o d d o e s n o t p ro v id e sufficient

Figure H 2 0

Power spectrum estimates from minimum variance method.

958

Power Spectrum Estimation

Figure 1X21

Chap. 12

Power spectrum estimates from Yule-Walker A R method.

re so lu tio n to e s tim a te th e sin u so id s fro m th e d a ta ; (2) th e m in im u m v arian ce

m e th o d o f C a p o n reso lv es o n ly th e fre q u e n c ie s f \ , f 2 b u t n o t / 3 a n d / 4; (3) th e
A R m e th o d s reso lv e all fre q u e n c ie s fo r cr2 = 0.1 a n d cr2 = 0.5; a n d (4) th e M U S IC
a n d E S P R IT a lg o rith m s n o t o n ly re c o v e r all fo u r sin u so id s, b u t th e ir p e rfo rm a n c e
fo r d iffe re n t v a lu e s o f cr2 is esse n tia lly in d istin g u ish a b le . W e f u r th e r o b se rv e th a t

Frequency
Figure 12.22

Power spectrum estimates from MUSIC algorithm.

Sec. 12.6

Summary and References

TABLE 12.2

0.1
0.5
1.0
True values

959

ESPRIT ALGORITHM

Ia

-0.2227
-0.2219
-0.222
-0.222

-0,1668
-0.167
-0.167
-0.166

-0.1224
-0.121
0.1199
0.122

0.10071
0.0988
0.1013
0.100

th e re so lu tio n p r o p e rtie s o f th e m in im u m v a ria n c e m e th o d a n d th e A R m e th o d is

a fu n c tio n o f th e n o ise v a rian ce. T h e s e re su lts clearly d e m o n s tr a te th e p o w e r o f
th e e ig e n a n a ly sis-b a se d a lg o rith m s in reso lv in g sin u so id s in a d d itiv e n oise.
In co n clu sio n , w e sh o u ld em p h asize th a t th e h ig h -re s o lu tio n , eig en an aly sisb a s e d sp e c tra l e s tim a tio n m e th o d s d e sc rib e d in th is se ctio n , n a m ely M U S IC an d
E S P R IT , a re n o t o n ly ap p licab le to sin u so id al signals, b u t ap p ly m o re g en erally
to th e e stim a tio n o f n a rro w b a n d signals.

12.6 SUMMARY AND REFERENCES

P o w e r sp e c tru m e s tim a tio n is o n e o f th e m o st im p o rta n t a r e a s o f re se a rc h a n d a p
p lic a tio n s in d ig ita l sig n a l p ro cessin g . In th is c h a p te r w e h a v e d e sc rib e d th e m o st
im p o rta n t p o w e r sp e c tru m e s tim a tio n te c h n iq u e s a n d a lg o rith m s th a t h av e b e e n
d e v e lo p e d o v e r th e p a s t cen tu ry , b e g in n in g w ith th e n o n p a ra m e tric o r classical
m e th o d s b a s e d o n th e p e rio d o g ra m a n d c o n clu d in g w ith th e m o re m o d e rn p a r a
m e tric m e th o d s b a s e d o n A R , M A , a n d A R M A lin e a r m o d e ls. O u r tr e a tm e n t
is lim ite d in sc o p e to sin g le-tim e-series sp e c tru m e s tim a tio n m e th o d s, b a se d on
se co n d m o m e n ts (a u to c o rre la tio n ) o f th e sta tistical d a ta .
T h e p a ra m e tric a n d n o n p a ra m e tric m e th o d s th a t w e d e s c rib e d h a v e b e e n
e x te n d e d to m u ltic h a n n e l a n d m u ltid im e n sio n al sp e c tru m es tim a tio n . T h e tu to ria l
p a p e r by M cC lellan (1982) tr e a ts th e m u ltid im e n sio n a l sp e c tru m e s tim a tio n p r o b
lem , w h ile th e p a p e r by Jo h n s o n (1982) tr e a ts th e m u ltic h a n n e l s p e c tru m e s tim a
tio n p ro b le m . A d d itio n a l sp e c tru m e s tim a tio n m e th o d s h a v e b e e n d e v e lo p e d fo r
u se w ith h ig h e r-o rd e r c u m u la n ts th a t involve th e b isp e c tru m a n d th e trisp e c tru m .
A tu to ria l p a p e r o n th e s e to p ics h a s b e e n p u b lish e d by N ik ias a n d R a g h u v e e r
(1987).
A s e v id e n c e d fro m o u r p rev io u s d iscussion, p o w e r sp e c tru m e s tim a tio n is an
a r e a th a t h a s a ttr a c te d m a n y re s e a rc h e rs a n d , a s a re su lt, th o u s a n d s o f p a p e rs h av e
b e e n p u b lish e d in th e te c h n ic a l lite ra tu re on th is su b je ct. M u c h o f th is w o rk h as
b e e n c o n c e rn e d w ith n ew a lg o rith m s a n d te c h n iq u e s, a n d m o d ifica tio n s o f ex ist
in g te c h n iq u e s . O th e r w o rk h as b e e n c o n c e rn e d w ith o b ta in in g a n u n d e rsta n d in g
o f th e c a p a b ilitie s a n d lim ita tio n s o f th e v ario u s p o w e r s p e c tru m m e th o d s. In
th is c o n te x t th e sta tistical p ro p e rtie s a n d lim ita tio n s o f th e classical n o n p a ra m e tric
m e th o d s h av e b e e n th o ro u g h ly an aly zed a n d a re w ell u n d e rs to o d . T h e p a ra m e tric
m e th o d s h a v e also b e e n in v e stig a te d b y m an y re se a rc h e rs , b u t th e an aly sis o f th e ir

960

Power Spectrum Estimation

Chap. 12

perform ance is difficult and, con sequ en tly, few er results are availab le. S om e o f the
papers that have addressed the problem o f p erform ance characteristics o f param et
ric m eth od s are th o se of K rom er (1969), L acoss (1971), B erk (1974), B aggeroer
(1976), Sakai (1979), Sw ingler (1980), L ang and M cC lellan (1980), and Tufts and
K um aresan (1982).
In addition to the referen ces already given in this chapter on the various
m eth od s for spectrum estim ation and their perform ance, w e sh ou ld include for
referen ce som e o f th e tutorial and survey papers. In particular, w e cite the tutorial
paper by K ay and M arple (1981), w hich in clu d es ab ou t 280 referen ces, the paper
b y B rillinger (1974), and the Special Issue on Spectral E stim ation o f the I E E E
P ro ceed in g s, Sep tem b er 1982. A n oth er indication o f the w id esp read interest in
the subject o f spectrum estim ation and analysis is the recent p u b lication o f texts
by G ardner (1987), K ay (1988), and M arple (1987), and the IE E E b ook s edited
by Childers (1978) and K esler (1986).
M any com puter program s as w ell as softw are packages that im plem ent the
various spectrum estim ation m eth od s described in this chapter are available. O ne
softw are package is available through the IE E E (P ro g ra m s f o r D ig ita l S ig n a l P ro
cessing, IE E E Press, 1979); others are available com m ercially.

PROBLEMS
12.1 (a) By expanding (12.1.23), taking the expected value, and finally taking the limit as
To -+ oo, show that the right-hand side converges to

(b) Prove that

12 2 For zero mean, jointly Gaussian random variables,

[see Papoulis (1984)] that

X 2, X 3, X*, it is well known

E W x X iX 3X t ) = (X iX 2 ) (X 3 X4) + (X 1 X3) (X 2 X4) + E (X iX t )E (X 2X 3)

Use this result to derive the mean-square value of r^(m), given by (12.1.27) and the
variance, which is
v a r ^ fm )] = E[\r'xz(m)\2] - \E[r'xx(m)]{
1 2 3 By use of the expression for the fourth joint moment for Gaussian random variables,

show that
(a) E [P (fx)P * A h )] = a* 1 +
+

sin jr(/i + f i) N

[Afsin7r(/i + / 2) 1

Chap. 12

Problems

961

sin 7T(fi + f 2)N

N sin jt(/i + / 2)

(b) co v [/> (/,)/> (/2)] =

+

sin ^ (/i - f 2)N 1

A? s i n n - ( / i

(c) var[/> (/)] = crj

h )\

j+
|

/ s i n 2 j r / W \ 2]
under the condition that the sequence x(n)
y N sin 2jt/ 1) }

is a zero-mean white Gaussian noise sequence with variance a],

1 2 .4 Generalize the results in Problem 12.3 to a zero-mean Gaussian noise process with

power density spectrum r ( / ) . Then derive the variance of the periodogram Pxr( f ),
as given by (12.1.38). {Hint: Assume that the colored Gaussian noise process is the
output of a linear system excited by white Gaussian noise. Then use the appropriate
relations given in Appendix A.)
12.5 Show that the periodogram values at frequencies /* = k /L , k = 0, 1 , . . . , L - 1, given
by (12.1.41) can be computed by passing the sequence through a bank of N IIR filters,
where each filter has an impulse response
M ) = e -j2*nk,f/u(n)
and then compute the magnitude-squared value of the filter outputs at n = N. Note
that each filter has a pole on the unit circle at the frequency f t .
12.6 Prove that the normalization factor given by (12.2.12) ensures that (12.2.19) is satisfied.
12.7 Let us consider the use of the DFT (computed via the FFT algorithm) to compute
the autocorrelation of the complex-valued sequence x (n ), that is,
1

i'u(m ) =
N

N -m -l

x*(n)x(n + m), m > 0

/I sbO

Suppose the size M of the FFT is much smaller than that of the data length N.
Specifically, assume that N = K M .
(a) Determine the steps needed to section x(n) and compute rxx(m) for (M /2) + 1 <
m < (M /2) - 1, by using 4K A/-point DFTs and one M-point IDFT
(b) Now consider the following three sequences x\(n), x2(n), and x^(n), each of du
ration M. Let the sequences x\(n) and x2(n) have arbitrary values in the range
0 < n < (M/2) - 1, but are zero for (M /2) < n < M 1. The sequence xi(n) is
defined as
xi(n).
x i(n) =
xi

H>

"2

< rt < M - 1

Determine a simple relationship among the M-point DFTs X i(k), X 2(k), and
* 3 (J t).

(c) By using the result in part (b), show how the computation of the DFTs in part
(a) can be reduced in number from AK to 2 K.
12J5 The Bartlett method is used to estimate the power spectrum of a signal x(n). We
know that the power spectrum consists of a single peak with a 3-dB bandwidth of
0 .0 1 cycle per sample, but we do not know the location of the peak.

962

Power Spectrum Estimation

Chap. 12

(a) Assuming that N is large, determine the value of M = N / K so that the spectral

window is narrower than the peak.

(b) Explain why it is not advantageous to increase M beyond the value obtained in
pan (a).
12.9 Suppose we have N = 1000 samples from a sample sequence of a random process.
(a) Determine the frequency resolution of the Bartlett, Welch (50% overlap), and
Blackman-Tukey methods for a quality factor Q = 10.
(b) Determine the record lengths (Af) for the Bartlett, Welch (50% overlap), and
Blackman-Tukey methods.
12.10 Consider the problem of continuously estimating the power spectrum from a sequence
x (n) based on averaging periodograms with exponential weighting into the past. Thus
with P \ f ) = 0, we have
1

l M _1

l Y ' xm(n)e-

C (/) =

where successive periodograms are assumed to be uncorrelated and w is the (expo

nential) weighting factor.
(a) Determine the mean and variance of
for a Gaussian random process.
(b) Repeat the analysis of part (a) for the case in which the modified periodogram
defined by Welch is used in the averaging with no overlap.
12.11 The periodogram in the Bartlett method can be expressed as
M -l

-J2n/n
m=-<Af-1 ) V

where r J( *(m) is the estimated autocorrelation sequence obtained from the j'th block
of data. Show that P ^ ( f ) can be expressed as
/ *> (/ ) = E ' ! ( f ) R ^ E ( f )

where
(/) = [l

eila}

]'

and therefore,

12.12 Derive the recursive order-update equation given in (12.3.19).

12.13 Determine the mean and the autocorrelation of the sequence jcfn), which is the output
of a ARMA (1,1) process described by the difference equation
x(n) =

(n 1) + w(n) - w(n - 1)

where u>(n) is a white noise process with variance crj.

12.14 Determine the mean and the autocorrelation of the sequence
M A(2) process described by the difference equation
x(n) = w(n) - 2 w(n - 1) + w(n 2)
where w(n) is a white noise process with variance a*.

jc

(n) generated by the

Chap. 12

Problems

963

12.15 An M A(2) process has the autocorrelation sequence

6cr2
Yxx (m ) =

m = 0

4ct2 ,

m= 1

2 a 2,

m = 2

0,

otherwise

(a) Determine the coefficients of the MA(2) process that have the foregoing auto
correlation.

(b) Is the solution unique? If not, give all the possible solutions.
12.16 An MA(2) process has the autocorrelation sequence

y(m) =

al
35 ,
- cr2.
62

m =0
m = 1

m = 2
7*al
62
(a) Determine the coefficients of the minimum-phase system for the MA(2) process.
(b) Determine the coefficients of the maximum-phase system for the MA(2) process,
(c) Determine the coefficients of the mixed-phase system for the MA(2) process.
12.17 Consider the linear system described by the difference equation
v(n) = 0.8v(n 1) + x(n) + x(n 1)
where x(n) is a wide-sense stationary random process with zero mean and autocorre
lation
* ,( > = G ) 1" 1

(a) Determine the power density spectrum of the output y(n).

(b) Determine the autocorrelation yyy(m) of the output.
(c) Determine the variance o\2 of the output.
12.18 From (12.3.6) and (12.3.9) we note that an A R (p) stationary random process satisfies
the equation

f 2

p
Yxx(m) +

{ .q

= 0,

1 < m < p.

where ap(k) are the prediction coefficients of the iinear predictor of order p and a 2
is the minimum mean-square prediction error. If the (p + 1) x (p + 1) autocorrelation
matrix Txx in (12.3.9) is positive definite, prove that:
(a) The reflection coefficients |K| < 1 for 1 < m < p.
(b) The polynomial
p
A p(z) = 1 + y ^ a p(k)z~k
*i
has all its roots inside the unit circle (i.e., it is minimum phase).

12.19 Consider the A R(3) process generated by the equation

x(n) = %x(n - 1) + %x(n - 2) - %x(n - 3) + w(n)
where w(n) is a stationary white noise process with variance a*.

964

Power Spectrum Estimation

Chap. 12

(a ) D eterm in e th e co efficien ts o f the optim um p = 3 lin ear p red ictor.

D eterm in e th e au toco rrelatio n seq u en ce y ( m ) , 0 < m < 5.
(c ) D eterm in e th e reflection co efficien ts co rresp o nd ing to th e p = 3 lin ear predictor.

(b)

1 2 2 0 * A n A R (2 ) p rocess is d escrib ed by th e d ifferen ce eq u atio n

x (n) = 0.81jc (n 2) + w(n)

w here w(n) is a white n oise p rocess with variance er2 .
(a ) D eterm in e th e p aram eters o f th e M A (2 ), M A (4 ), and M A (8 ) m od els which p ro
vide a m inim um m ean -squ are e rro r fit to th e data x (n ).
(b ) P lo t th e true spectrum and th ose o f the MA(<?), q = 2, 4, 8 , sp ec tra and com pare
the results. C o m m en t on how well the M A (g ) m od els ap p ro x im a te th e A R ( 2 )
process.
1 2.21 A n M A (2 ) p rocess is described by th e d ifferen ce equation

x () = w(n) + 0.81u>(n 2)
w here w(n) is a w hite n oise p rocess with v arian ce a 2 .
(a ) D ete rm in e th e p aram eters o f th e A R ( 2 ) , A R ( 4 ) , and A R ( 8 ) m od els that provide
a m inim um m ean -squ are erro r fit to the data x(n).
(b ) P lo t th e true spectra m and th ose o f th e A R ( p ) , p = 2. 4 , 8. and com pare
the results. C o m m en t on how w ell the A R ( p ) m odels a p p roxim ate the M A (2 )
process.

1 2 2 2 T h e z-transform o f th e a u to co rrelatio n yT I(m )of an A R M A ( 1 ,1) process is

TIX(z) = a l H ( z ) H ( z ~ ' )

(a) Determine the minimum-phase system function H(z).

(b ) Determine the system function H (z ) for a mixed-phase stable system.

12.23 Consider a FIR filter with coefficient vector

[1

2 r COS 6

t2 ]

(a) Determine the reflection coefficients for the corresponding FIR lattice filter.
(b ) Determine the values of the reflection coefficients in the limit as r

1 2 2 4 An AR(3) process is characterized by the prediction coefficients

o3 (l) = 1.25.

a3(2) = 1.25,

a 3 (3) = - 1

(a) Determine the reflection coefficients.

(b ) Determine yxx(m) for 0 < m < 3.

(c) Determine the mean-square prediction error.

1 2 2 5 The autocorrelation sequence for a random process is
1,
0.5,
0.625,
0.6875,
0

m = 0
m = 1
m = 2
m = 3
otherwise

1.

Chap. 12

965

Problems

D ete rm in e th e system fun ction s A (z) for th e p red ictio n -erro r filters for m = 1, 2, 3,
th e reflectio n coefficien ts { K m}, and the correspond ing m ean -squ are p red iction errors.
1 2 .2 6

(a)

D e te rm in e th e p o w er spectra fo r th e ran d om p rocesses g en erated by the follow ing

d iffe re n ce eq uations.
(1 ) x (n ) = 0.81jr(n 2) + ui(n) w(n 1)
(2 ) x(n) = u >{n) w(n 2)
(3 ) x (n ) = 0 .8 1 * (n 2) + ui(n)
w here w{n) is a w hite noise process with v arian ce cr2 .
(b) S k etc h th e sp ectra fo r th e p rocesses given in part (a ).
(c) D ete rm in e th e au to co rrelatio n yX I(m ) fo r th e p rocesses in (2 ) and (3 ).

1 2 .2 7 T h e au to co rrelatio n seq u en ce fo r an A R process x(n ) is

(a)
(b)

D ete rm in e th e d iffe re n ce eq u atio n for i ( n ) .

Is y ou r answ er uniqu e? I f n o t, give any o th e r possible solutions.

1 2 .2 8 R e p e a t P ro b lem 12.27 fo r an A R p rocess with a u to co rrelatio n

Vi:r(ffi) = a' ' cos

nm

w here 0 < a < 1.

1 2 2 9 T h e B a r tle tt m ethod is used to estim ate th e pow er spectru m o f a signal from a s e
q u en ce x (n) consisting o f N = 2 4 0 0 sam ples.
(a) D ete rm in e the sm allest length M o f each segm ent in th e B a rtle tt m ethod th at
yields a frequ en cy resolu tion o f A/ = 0.01.
(b) R e p e a t part (a ) fo r A / = 0.02.
(c) D ete rm in e th e qu ality factors Q b fo r parts (a ) and (b ).
1 2 3 0 P rov e th at a F I R filter with system function
p

A(Z) =
*>1
and reflectio n co efficien ts |AT* | < 1 fo r 1 < k < p 1 and |ATP | > 1 is m axim um phase
[all th e ro o ts o f A p ( z ) lie outside th e unit circle].
1 2 3 1 A random p rocess x (n ) is ch aracterized by th e p o w er density spectrum

U ) ~

\e W -Q .9 ?
_ JQ 9 |2|cjr2T/ + jQ 9l2

w h ere cr2 is a co n stan t (scale facto r).

(a) I f w e view r ( / ) as th e pow er spectrum a t th e o utpu t o f a lin ear p o le -z e ro
system H ( z ) driven by white n oise, d eterm in e H (z).
(b) D ete rm in e th e system function o f a stab le system (noise-w hiten in g filter) that
p rod u ces a w hite n oise output w hen ex cited by x(n).
1 2 3 2 T h e W -point D F T o f a random seq u en ce x(n) is
N -1

X (k ) = y '^ x ( n ) e - j2*nk/"

HkO
A ssu m e th at E [x {n )] = 0 and E [x (n )x (n + m ) ] = cr25(m ) [i.e., x(n) is a w hite noise
process].

966

Power Spectrum Estimation

Chap. 12

(a) Determine the variance of X (k).

(b) Determine the autocorrelation of X (k).
1233 Suppose that we represent an ARM A(p, q) process as a cascade of a MA(<?) followed
by an AR(p) model. The input-output equation for the MA(<y) model is
v(n) = 2 2 b t w(n - k)
*=o
where w(n) is a white noise process. The input-output equation for the AR(p) model
is

x(n) + 22a/cX(n - k) = v(n)

*=i

(a) By computing the autocorrelation of v(n), show that

qm

Yw(m) = o l ' y ' b l b i+m

Jt=0

(b) Show that

p

Yw(m) = 2 2 aky vx(m + k)

a = 1

*s=0

where yM(m) = [u{n + m)jr*(n)]:

1234 Determine the autocorrelation yxx(m) of the random sequence
x(n) = A cos(ti>i + <p)
where the amplitude A and the frequency w\ are (known) constants and <j> is a uni
formly distributed random phase over the interval (0 , 2 t t ) .
1235 Suppose that the AR(2) process in Problem 12.20 is corrupted by an additive white
noise process v(n) with variance crjf. Thus we have
y(n) = x(n) + v(n)

(a) Determine the difference equation for y(n) and thus demonstrate that y(n) is an
ARMA(2, 2) process. Determine the coefficients of the A R M A process.

(b) Generalize the result in part (a) to an AR(p) process

p

x(n) = - ^ 2 ak(xn ~ k) + w(n)

t-i
and
>(ti) = x(n) + v(n)
1236 (a) Determine the autocorrelation of the random sequence
K
X (n ) = 2 2 A t COS(<ot n + </>k) + ttf(/j)
*l
where (A*} are constant amplitudes, [cot] are constant frequencies, and {4>t) are
mutually statistically independent and uniformly distributed random phases. The
noise sequence w(n) is white with variance
(b) Determine the power density spectrum of x(n).

Chap. 12

967

Problems

1237 The harmonic decomposition problem considered by Pisarenko can be expressed as

the solution to the equation
a"'rvva = tr2 a"a
The solution for a can be obtained by minimizing the quadratic form a*'rvva subject to
the constraint that a*'a = 1. The constraint can be incorporated into the performance
index by means of a Lagrange multiplier. Thus the performance index becomes
= a * T vva 4- A(1 a*'a)
By minimizing with respect to a, show that this formulation is equivalent to the
Pisarenko eigenvalue problem given in (12.5.9) with the Lagrange multiplier play
ing the role of the eigenvalue. Thus show that the minimum of is the minimum
eigenvalue a 2.
12.38 The autocorrelation of a sequence consisting of a sinusoid with random phase in noise
is
y(m ) = P cos2jr/im
where f\ is the frequency of the sinusoidal, P is its power, and
is the variance of
the noise. Suppose that we attempt to fit an AR(2) model to the data.
(a) Determine the optimum coefficients of the AR(2) model as a function of a*
and fi.
(b) Determine the reflection coefficients K\ and K 2 corresponding to the AR(2)
model parameters.
(c) Determine the limiting values of the AR(2) parameters and (K\, K 2) as <xu2 -* 0 .
1239 This problem involves the use of crosscorrelation to detect a signal in noise and esti
mate the time delay in the signal. A signal x (n) consists of a pulsed sinusoid corrupted
by a stationary zero-mean white noise sequence. That is,
x(n) = y(n no) + ui(n)
where w(n) is the noise with variance

0 < n < N 1

and the signal is

y(n) = A cos twon,

= 0,

0 < n < M 1
otherwise

The frequency a>o is known but the delay h0, which is a positive integer, is unknown,
and is to be determined by crosscorrelating x(n) with _y(n). Assume that N > M + n0.
Let
N- 1
ri y (.m) = Y
n* 0

y (n

- m )x (n )

denote the crosscorrelation sequence between x(n) and y(n). In the absence of noise
this function exhibits a peak at delay m = n0. Thus n0 is determined with no error.
The presence of noise can lead to errors in determining the unknown delay.
(a) For m = no, determine E[riy(n(,)]. Also, determine the variance, var[riV(rtB)],
due to the presence of the noise. In both calculations, assume that the double
frequency term averages to zero. That is, M 3> 2n/wo.
(b) Determine the signal-to-noise ratio, defined as
SNR = l % < ^
var[r^(n0)j

968

Power Spectrum Estimation

Chap. 12

(c) What is the effect of the pulse duration M on the SNR?

1 2.40* Generate 100 samples of a zero-mean white noise sequence w(n) with variance a j =

j 2,
(a)
(b)
(c)

by using a uniform random number generator.

Compute the autocorrelation of u>(n) for 0 < m < 15.
Compute the periodogram estimate Pz l ( f ) and plot it.
Generate 10 different realizations of w(n) and compute the corresponding sample
autocorrelation sequences n (m ), 1 < k < 10 and 0 < m < 15.
(d) Compute and plot the average periodogram for part (c):
r>v(m) = -L 2 * , r*(m)
(e) Comment on the results in parts (a) through (d).

12.41* A random signal is generated by passing zero-mean white Gaussian noise with unit
variance through a filter with system function

(1 + az~l + 0 .9 9 i- 2 ) ( l - a z - 1 + 0 . 9 8 ; - ; )
(a) Sketch a typical plot of the theoretical power spectrum r { / ) for a small value
of the parameter a (i.e., 0 < a < 0.1). Pay careful attention to the value of the
two spectral peaks and the value of Pzx(a>) for co = n/2.
(b) Let a = 0.1. Determine the section length M required to resolve the spectral
peaks of T ( / ) when using Bartletts method.
(c) Consider the Blackman-Tukey method of smoothing the periodogram. How
many lags of the correlation estimate must be used to obtain resolution compa
rable to that of the Bartlett estimate considered in part (b)? How many data
must be used if the variance of the estimate is to be comparable to that of a
four-section Bartlett estimate?
(d) For a 0.05, fit an AR(4) model to 100 samples of the data based on the
Yule-Walker method and plot the power spectrum. Avoid transient effects by
discarding the first 200 samples of the data.
(e) Repeat part (d) with the Burg method.
(f) Repeat parts (d) and (e) for 50 data samples and comment on similarities and
differences in the results.

Appendix
Random Signals, Correlation
Functions, and Power Spectra

In this appendix w e provide a brief review o f the characterization o f random

signals in term s o f statistical averages expressed in both the tim e dom ain and the
frequency dom ain. T he reader is assum ed to have a background in probability
theory and random p rocesses, at the level given in the b ook s o f H elstrom (1990)
and P eeb les (1987).

Random Processes
M any physical p h en om en a en cou n tered in nature are best characterized in statis
tical term s. For exam p le, m eteorological p h en om en a such as air tem perature and
air pressure fluctuate random ly as a function o f tim e. Therm al n oise voltages gen
erated in the resistors o f electron ic d evices, such as a radio or television receiver,
are a lso random ly fluctuating p h en om en a. T h ese are just a few exam p les o f ran
dom signals. Such signals are usually m odeled as infinite-duration infinite-energy
signals.
Suppose that w e take the set o f w aveform s corresponding to the air tem p er
ature in differen t cities around the w orld. For each city there is a corresponding
w aveform that is a function o f tim e, as illustrated in Fig. A .I . T h e set o f all p ossible
w aveform s is called an en sem b le o f tim e functions or, equivalently, a ra n d o m p r o
cess. T h e w aveform for the tem perature in any particular city is a single realization
or a sa m p le fu n c tio n o f the random process.
Sim ilarly, th e therm al n oise voltage gen erated in a resistor is a single real
ization or a sam p le function o f the random p rocess con sistin g o f all n oise voltage
w aveform s gen erated by the set o f all resistors.
T h e set (en sem b le) o f all p ossib le noise w aveform s o f a random process
is d en o te d as X (r, S ), w h ere t rep resents the tim e index and S represents the
set (sam p le sp a ce) o f all p ossib le sam ple functions. A single w aveform in the
en sem b le is d en o te d by x ( t , s). U sually, w e drop the variable s (or S ) for n otational
co n v en ien ce, so that the random p rocess is d en oted as X (r) and a single realization
is d en o ted as x ( t) .
A1

A2

Random Signals, Correlation Functions, and Power Spectra

App. A

Figure A.1

H aving defined a random process X ( t) as an en sem b le o f sam p le functions, let

us consider the v alu es o f the p rocess for any set o f tim e instants ti > t2 > > t,
w here n is any p ositive integer. In general, the sam p les X,. = x(r, ), i = 1, 2 , . . . , n
are n random variables characterized statistically by their joint probability density
function (P D F ) d en o ted as p ( x tl, x,2, . . . , x , j for any n.

Stationary Random Processes

Suppose that w e have n sam ples o f th e random p rocess X ( t ) at t =

i = 1,

2 , . . . , n, and another set o f n sam p les disp laced in tim e from th e first set by an

am ount z . Thus the secon d se t o f sam p les are X ,i+T = X (tj + z ) , i = 1, 2 , . . . , n , as

App. A

Random Signals, Correlation Functions, and Power Spectra

A3

show n in Fig. A .I . T h is secon d set of n random variables is characterized by the

joint p robability density function p (x ,i+T, . . . , x lii+T). T he jo in t P D F s o f the tw o sets
o f random variables m ay or m ay not be identical. W hen th ey are id en tical, then
P (x h , x l2........ x ,J = p ( x lt+T, x,1+x, . . . . x,+z)

( A .l)

for all r and all n, then the random process is said to b e stationary in the strict
sense. In other words, the statistical properties o f a stationary random p rocess are
invariant to a translation o f the tim e axis. O n the other hand, w h en th e joint P D F s
are different, the random process is nonstationary.

Statistical (Ensemble) Averages

Let us con sid er a random process X ( t ) sam pled at tim e instant t t,. Thus X (ti)
is a random variable with P D F p { x ti). T he / th m o m e n t o f the random variable is
d efined as the exp ected value o f X l (tj), that is,
(A .2)
In general, the value o f the /th m om en t dep en d s on the tim e instant tit if the P D F
o f X,, dep en d s on t,. W h en the process is stationary, h ow ever, p { x u+T) p { x r.) for
all x. H en ce th e P D F is in d ep en dent o f tim e and, con seq u en tly, the /th m om en t
is in d ep en dent o f tim e (a constant).
N ext, let us con sid er the two random variables X ,t = X (?,), i = 1 , 2, corre
sponding to sam p les o f X (t) taken at t = t] and t = 12. T h e statistical (en sem b le)
correlation b etw een X t] and X ,2 is m easured by the joint m om en t
(A .3)
Since the joint m om en t d epends on the tim e instants ti and ti, it is d en oted as
Y x x i h , h ) and is called the autocorrelation fu n c tio n o f the random process. W hen

the process X (/) is stationary, the joint P D F o f the pair (X tj, X ,2) is identical to
the joint P D F o f the pair (X fl+r, X ,1+T) for any arbitrary r. T his im plies that the
autocorrelation function o f X ( t ) dep en d s on the tim e d ifferen ce t \ ti = x. H en ce
for a stationary real-valued random process the au tocorrelation function is
yxx( r) = E [ X tl+TX h ]

(A .4)

y ( - r ) = ( * - , * ) = ( * , ; * , ;+r) = ^ ( r )

(A .5)

O n the other hand,

T h erefore, yXI{x) is an even function. W e also n o te that yz x (0) = E ( X 2 ') is the

a verage p o w e r o f the random process.
There exist nonstationary processes with the property that the m ean value
o f th e p rocess is a constant and the au tocorrelation function satisfies the property
Yxz(t 1, *2) = Y x x i h h ) - Such a p rocess is called w ide-sense stationary. Clearly,

A4

Random Signals, Correlation Functions, and Power Spectra

App. A

w id e-sen se stationarity is a less stringent con d ition than strict-sen se stationarity.

In our treatm ent w e shall require on ly that the p rocesses b e w id e-sen se stationary.
R ela ted to the autocorrelation fun ction is the au tocovarian ce function, which
is defined as
C ( l i , l 2 ) = E { [ X tl - m ( f i ) ] [ X , j - m ( r 2)]}
(A .6)
= Y x x (h ,t2) m ( t\)m ( t2)
w here m (t\) = E { X tl) and m (t2) = E (X ,2) are the m ean v alu es o f X h and X n ,
respectively. W hen the process is stationary,
cXx ( h , h ) = cxx(fi - t2) =

cxx ( t )

= yxx(r) - m \

(A .7)

where r = h t2. Furtherm ore, the variance o f the p rocess is a } = <^,(0) =

X(0 ) - m 2.

Statistical Averages for Joint Random Processes

Let X ( t ) and Y ( t ) be tw o random p rocesses and let X tj = X(r,-), i = 1, 2 , . . . , n ,
and Y,< = y{rj), j = 1, 2 , . . . , m , represent the random variables at tim es h > h >
> / and t { > t'2 > > t'm, resp ectively. T h e tw o sets o f ran d om variables are
characterized statistically by the joint P D F

......>,;)
for any set o f tim e instants {/,} and \t'} and for any p ositive in teger valu es o f m
and n.
T h e crosscorrelation fu n c tio n o t X ( t ) and Y (t), d en oted as yxv (t\ , t2)> is defined
by the joint m om en t

OO

00

j:tly,2p ( x t], y,2) d x tldy,2

(A .8)

00 * ' 00

and the crosscovariance is

C x y (tu h ) = yxy( t \ , t 2) - m x (ti)m y (t2)

(A .9)

W hen the random p rocesses are jointly and individually stationary, w e have
YxyVi, h ) = Yxyih - t2) and cxy(ti, t2) = cxy(h - t2). In this case
^ ( - r ) = E( XtlYll+r ) = E (X ,r r Yt) = yyx( r)

(A .10)

T h e random processes X ( t ) and Y (t) are said to be statistically in d ep en dent

if and on ly if
p(xh

x<., y ,,

.........y t>
m) = p ( x h , . . . , x tt )p(y,> , - . . , y , m)

for all ch oices o f

t- and for all p ositive in tegers n and m. T h e p rocesses are said
to b e uncorrelated if
Yxy(h, t2) = E (X ,t)E {Y ,t)

so that cxy(ti, t2) = 0.

( A .ll)

App. A

Random Signals, Correlation Functions, and Power Spectra

A5

A co m p lex-valu ed random p rocess Z( /) is defined as

Z { t) = X ( t) + j Y ( t )

(A .12)

w here X( t ) and K(/) are random p rocesses. T he joint P D F o f the com plex-valued
random variables Z,, = Z(r,), i = 1, 2 , , is given by the joint P D F o f the
co m p o n en ts ( X l;, Yti), i = 1, 2 ........ n. Thus the P D F that characterizes Z,,, i = 1,
2 .........n is
p ( x n , x l2.........x t ,y ,y l2, . . . , y , m)
A com p lex-valu ed random p rocess Z (t) is en cou n tered in the representation
o f the in-phase and quadrature com p onents o f th e low p ass equivalent o f a nar
row band random signal or noise. A n im portant characteristic o f such a process is
its autocorrelation function, which is defined as
= E { Z h Z* )
= E [ ( X t] + j Y tl)(X ,2 - j Y ,2)]

(A .13)

= Y xxih, h ) + yvv('i, h )t+ j [ Y y A h , h ) Yxy(t\- h ) \

W hen the random p rocesses X (r ) and Y (t) are jointly and individually stationary,
the autocorrelation function o f Z (t) b ecom es
YzztiU*2) = Yzz0 1 - t2) = /;;(T )
w here

ti t2. T h e com plex conjugate of (A .1 3 ) is

Y*z (r) = (Z * Z ,i_r) = y ( - r )

(A .14)

N ow , su p p ose that Z(r) = X ( t) + j Y ( t ) and W(/) = U (t) + j V ( t ) are tw o

com p lex-valu ed random processes. T heir crosscorrelation function is defined as
Y zv d u ti) =

( z w ;j

= E [ ( X h + j Y li)(U l2 ~ j V t2)}

( A .15)

= Yxuihi h ) + Y y v U \,h ) + j[Y y u (h , h ) Yxvih* h)]

W hen X ( t) , Y (t) , U (t), and V ( t ) are pairwise stationary, th e crosscorrelation func
tion s in (A .1 5 ) b eco m e functions o f the tim e differen ce z
t2. In addition,
w e have
y; j r) = E (Z * W ,t. T) = E (Z * + tWh ) = y wz( - T )

( A .16)

Power Density Spectrum

A stationary random p rocess is an infinite-energy signal and h en ce its Fourier
transform d o e s n o t exist. T h e spectral characteristic o f a random p rocess is o b
tained according to the W ien er-K h in ch in e th eorem , by com puting the Fourier
transform o f th e autocorrelation function. That is, the distribution o f pow er with

A6

Random Signals, Correlation Functions, and Power Spectra

App. A

frequency is given by the function

(A.17)
T he inverse F ourier transform is given as
(A .18)
W e observe that
y ( 0 ) = f Z c r * * (F )d F
(A .19)
= (* ? ) > 0
Since E (X ? ) = yxx (0) rep resents the average p ow er o f the random p rocess, which
is the area under r ^ F ) , it follow s that r ^ F ) is the distribution o f p ow er as a
function o f frequency. For this reason, TXX(F ) is called the p o w e r d en sity sp e ctru m
o f the random process.
If the random process is real, y IX( z ) is real and even and h en ce r(F) is real
and even. If the random p rocess is com p lex valued, yxx(r) = yx x ( z ) and, h ence
a cc

aoc

n jF ) =

y*x (z )e j2nfrdT = /

y ; j - z ) e - ' 2" F'd z

T herefore, r xx(F) is always real.

T h e definition o f the p ow er d en sity spectrum can be exten d ed to tw o jointly
stationary random p rocesses X ( t ) and Y (t) , which have a crosscorrelation function
yxv(z). The Fourier transform o f yxy(z) is

(A.20)
which is called the cro ss-p o w er d en sity sp e ctru m . It is easily sh ow n that T*V(F ) =
r v,( - F ) . For real random p rocesses, th e condition is P yx(F ) = I \ v( F).

Discrete-Time Random Signals

This characterization o f con tinu ou s-tim e random signals can b e easily carried over
to discrete-tim e signals. Such signals are usually ob tain ed by uniform ly sam pling
a con tinu ou s-tim e random process.
A discrete-tim e random p rocess X {n) con sists o f an en sem b le o f sam p le s e
q u en ces x ( n ). T h e statistical p rop erties o f X { n ) are sim ilar to th e characterization
o f X (/), with the restriction that n is n ow an in teger (tim e) variable. T o be specific,
w e state the form for the im portant m om en ts that w e use in th is text.

App. A

Random Signals, Correlation Functions, and Power Spectra

A7

T he /th m om en t o f X( n) is defined as
E { X n ) = f

x'nP (xn)d x n

(A .21)

J -O C

and the autocorrelation seq u en ce is

OC

/ 00

x nx kp {x nyx k)dx dxk

/
oc . / - 0

(A .22)

Sim ilarly, the autocovariance is

c ( n , t ) = Y x x in .k ) - E { X ) E ( X k)

(A .23)

For a stationary p rocess, w e have the special form s (m = n k)

Yxx(n - k ) = yxx(m)
Cxx(n - k) = cxx(m) =

(A .24)
- m]

w here m , = E (X ) is the m ean o f th e random process. T h e variance is defined as

a2=
= yjr ( 0 ) - m \.
For a com p lex-valu ed stationary process Z(n) = X (n ) + j Y ( n ) , w e have
y (m ) = yxx (m ) + y vv (m) + j [yyx (m ) - yxx (m )]

(A .25)

and the crosscorrelation seq u en ce o f tw o com p lex-valu ed stationary seq u en ces is

Y zA m ) = Yxu(m) -I- y yv(m) + j [ y yu{m) - yxv(m )]

(A .26)

A s in the case o f a continuous-tim e random p rocess, a discrete-tim e random

p rocess has infinite en ergy but a finite average p ow er and is given as
E ( X 2n) = yxx (0)

(A .27)

B y use o f the W ien er-K h in ch in e theorem , w e obtain the p ow er d en sity spectrum

o f the discrete-tim e random process by com puting the F ourier transform o f the
au tocorrelation seq u en ce yxx(m ), that is,
00

22

r,,( /)=

Y xx{m )e~ M m

(A .28)

T x x ( f ) e iln fm d f

(A .29)

m = oc

T h e inverse transform relationship is

r i/2
Y x x (m )=

1/2

J-\a

W e ob serve that the average p ow er is

f\a
YxA 0) = /
r xx( f ) d f
J 1/2

(A .30)

so that r xx( f ) is the distribution o f p ow er as a function o f frequency, that is,

Vxx( f ) is the pow er d en sity spectrum o f the random p rocess X (n ), T h e properties
w e have stated for Trx(F ) also hold for Vxx( f ) .

A8

Random Signals, Correlation Functions, and Power Spectra

App. A

Time Averages for a Discrete-Time Random Process

A lth o u g h w e have characterized a random process in term s o f statistical averages,
such as the m ean and the autocorrelation seq u en ce, in practice, w e usually have
available a single realization o f th e random process. L et us con sid er the problem
o f obtaining th e averages o f the random process from a sin gle realization. T o
accom plish this, the random p rocess m ust be ergodic.
B y definition, a random p rocess X ( n ) is ergod ic if, with p robability 1, all
the statistical averages can b e d eterm in ed from a sin gle sam p le function of the
process. In effect, the random process is ergod ic if tim e averages ob tain ed from
a single realization are equal to the statistical (en sem b le) averages. U n d er this
con d ition w e can attem pt to estim ate the en sem b le averages u sin g tim e averages
from a single realization.
T o illustrate this point, let us con sid er the estim ation o f th e m ean and the
au tocorrelation o f the random process from a sin gle realization x ( n ). Since w e
are in terested o n ly in these tw o m om en ts, w e define en god icity with respect to
th ese param eters. F or additional details on the requirem ents for m ean ergodicity
and autocorrelation ergodicity w hich are given b elo w , the reader is referred to the
b o o k o f P apoulis (1984).

Mean-Ergodic Process
G iven a stationary random p rocess X ( n ) with m ean

let us form the tim e a erage

(A .31)
In general, w e view m x in (A .3 1 ) as an estim ate o f the statistical m ean w h ose
value w ill vary w ith the different realizations o f the random p rocess. H en ce m x is
a random variable with a P D F p ( m x). L et us com p u te the ex p ected valu e o f m ,
over all p ossible realizations o f X (n ). Sin ce the sum m ation and th e expectation
are linear operations w e can interchange them , so that

Since th e m ean value o f the estim ate is equal to the statistical m ean , w e say that
the estim a te m , is unbiased.
N ex t, w e com p ute the variance o f rhx. W e have
v a r ( / n j = E(\mx \2) - \mx \2

App. A

Random Signals, Correlation Functions, and Power Spectra

A9

But
( |/ ,|2 ) = ( 2 /v'+ T )2 5 1
'

n = A/ k=N

T h erefore,

(A .33)

If var(m x) -* 0 as A' -* oo, the estim ate con verges with probability 1 to the
statistical m ean m x. T h erefore, the p rocess X ( n ) is m ean ergod ic if
(A .34)
U n d er this con d ition , the estim ate m x in the lim it as N > oc b ecom es equal to
the statistical m ean, that is,
(A .35)
T h u s th e tim e-average m ean, in the lim it as TV -* oo, is equal to the en sem b le
m ean.
A sufficient con d ition for (A .3 4 ) to hold is .if
00

(A .36)
w hich im plies that cxx(m ) 0 as m -* oo. T h is condition h old s for m ost zero-m ean
p rocesses en co u n tered in th e physical world.

Correlation-Ergodic Processes
N o w , let us con sid er the estim ate o f the autocorrelation yxx(m ) from a single
realization o f the p rocess. F ollow in g ou r previou s n otation , w e d en ote the estim ate
(for a com p lex-valu ed signal, in gen eral) as

(A.37)

A10

Random Signals, Correlation Functions, and Power Spectra

App. A

A g a in , w e regard rxx(m ) as a random variable for any given lag m, since it is a

function o f the particular realization. T h e exp ected value (m ean valu e over all
realizations) is
N
V

E [r (m )] -

2N + 1

E [x * (n )x (n + m)]
(A .38)

= 2
T h erefore, the exp ected value o f the tim e-average autocorrelation is eq u al to the
statistical average. H en ce w e have an u nbiased estim ate o f yxx (m).
T o determ ine the variance o f the estim ate rxx(m), w e co m p u te the exp ected
value o f \rxx(m )\2 and subtract the square o f th e m ean value. T hus
v ar[r(m )] = E [\rxx(m )\2] - |y (m ) |2

(A .39)

But
j

E [\rXI(m ) |2] = t t t
T
Y
E [x*{n)x(.n + m )x (k )x * (k + m )]
(2N -I-1)2 n ? Nkt ? N

(A .40)

T h e exp ected value o f the term ;c*(n)jc(n + m )x (k )x * (k + m ) is just the autocorre

lation sequ en ce o f a random process defined as
vm(n) = x*(n)x(n + m )
H en ce (A .4 0 ) m ay be exp ressed as

E
E t f O - W
( 2 N + l ) 2 n=-Nk=-N
(A .41)
2N

T
2N + 1

( i - J s L W
)
V
2 N + l ) rvv V '

and the variance is

v ar[r (m )] =

(* ~

a v

+t )

<A A Z )

If var[rxx(m)] -* 0 as N -* oo, the estim ate rxx(m) con verges with probability
1 to the statistical autocorrelation yxx(m). U n d er these con d ition s, the process is

correlation ergodic and th e tim e-average correlation is identical to the statistical

average, that is,

lim
r Y ] x * ( n ) x ( n + m ) = yxx( m)
N-COO 2 N + 1 n Ar

(A .43)

In our treatm ent o f random signals, w e assum e that the ran d om p rocesses are
m ean ergodic and correlation ergod ic, so that w e can deal w ith tim e averages o f
the m ean and the au tocorrelation ob tain ed from a sin gle realization o f the process.

Random Number Generators

In som e o f the exam p les given in the text, random num bers are generated to sim
ulate the effect o f n oise on signals and to illustrate how the m ethod o f correlation
can be used to d etect the presen ce o f a signal buried in n oise. In the case o f
periodic signals, the correlation tech n iqu e also allow ed us to estim ate the period
o f the signal.
In practice, random num ber generators are often used to sim ulate the effect
o f noiselik e signals and other random p h en om en a en cou n tered in the physical
world. Such n oise is p resent in electron ic d evices and system s and usually limits
our ability to com m u n icate over large distances and to be able to d etect relatively
w eak signals. B y generating such n oise on a com puter, w e are able to study
its effects through sim ulation o f com m unication system s, radar d etection system s,
and the lik e and to a ssess the p erform ance o f such system s in the presen ce o f
noise.
M ost com puter softw are libraries include a uniform random num ber gen er
ator. Such a random num ber gen erator gen erates a num ber b etw een zero and
1 with equal probability. W e call the output o f the random num ber generator a
random variable. If A d en otes such a random variable, its range is the interval
0 < A < 1.
W e know that the num erical output o f a digital com p uter has lim ited preci
sion, and as a co n seq u en ce, it is im possible to rep resen t the continuum o f num bers
in the interval 0 < A < 1. H ow ever, w e can assum e that our com puter represents
each output by a large num ber o f bits in either fixed point or floating point. C on se
quently, for all practical purposes, the num ber o f outputs in the interval 0 < A < 1
is sufficiently large, so that w e are justified in assum ing that any value in the
interval is a possib le ou tp u t from th e generator.
T he uniform probability density function for the random variable A , d en oted
as p ( A ) , is illustrated in Fig. B .la . W e n ote that the average valu e or m ean value
o f A , d en oted as m A, is m A =
T h e integral o f th e probability density function,
w hich represents the area under p ( A ) , is called the p robability distribution function

B1

B2

Random Number Generators

App. B

P(A)

2
(a)

(b)

Figure B .l

o f the random variable A and is defined as

F (A ) = f

p (x )d x

( B. l )

J OC

For any random variable, this area must always b e unity, w hich is the m axim um
value that can be achieved by a distribution function. H en ce
(B .2)
and the range o f F( A) is 0 < F (A ) < 1 for 0 < A < 1.
If w e wish to generate uniform ly distributed n oise in an interval (b , b + 1 )
it can sim ply be accom plished by using the output A o f the random num ber gen
erator and shifting it by an am ount b. Thus a new random variable B can be
defined as
B = A + b
w hich n ow has a m ean value m s = b +

For exam p le, if b =

(B .3)
the random

variable B is uniform ly distributed in the interval ( | , | ) , as sh ow n in Fig. B.2a.

Its probability distribution function F ( B ) is sh ow n in Fig. B.2b.
A uniform ly distributed random variable in the range ( 0 , 1 ) can b e used
to gen era te random variables with other probability distribution functions. For
exam p le, suppose that w e wish to gen erate a random variable C with probability
distribution function F ( C ) , as illustrated in Fig. B.3. Since the range o f F i C ) is
the interval (0 , 1 ), w e begin by generating a uniform ly distributed random variable
A in the range ( 0, 1) . If we set
F (C ) = A

(B .4)

C = F~l (A)

(B.5)

then

App. B

Random Number Generators

B3

Figure B.2

Figure B3
T hus w e so lv e (B .4 ) for C and the solution in (B .5) provides the value o f C
for w hich F ( C ) = A. B y this m eans w e obtain a n ew random variable C with
probability distribution F (C ). This inverse m apping from A to C is illustrated in
Fig. B.3.
Example B.1
Generate a random variable C that has the linear probability density function shown
in Fig. B.4a, that is,

p(C)={2'
0,

Solution

05 C - 2
otherwise

This random variable has a probability distribution function

0,

F( C)

iC
A1- 2 '
1,

C< 0
0< C < 2
C > 2

which is illustrated in Fig. B.4b. We generate a uniformly distributed random variable

A and set F( C) = A. Hence
F( C) j C 1 = A

B4

Random Number Generators

App. B

Figure B.4

Upon solving for C, we obtain

c

Thus we generate a random variable C with probability function F (C ). as shown in

Fig. B.4b.
In E xam ple B .l the inverse m apping C = F - 1 (A) was sim ple. In som e cases
it is not. This problem arises in trying to gen erate random num bers that h ave a
norm al distribution function.
N o ise encou n tered in physical system s is o ften characterized by th e norm al or
G aussian probability distribution, which is illustrated in Fig. B.5. T h e probability

Figure B.5

App. B

B5

Random Number Generators

density function is given by

oo < C < oo

(B . 6 )

w here ct2 is the variance o f C , w hich is a m easure o f the spread o f the probability
density function p {C ). T he probability distribution function F (C ) is the area under
p (C ) over the range ( 0 0 , C ). Thus
(B.7)
U n fortu n ately, the integral in (B .7) can n ot be expressed in term s o f sim ple func
tions. C onsequently, the inverse m apping is difficult to ach ieve.
A w ay has b een found to circum vent this problem . F rom probability th e
ory it is know n that a (R ayleigh distributed) random variable R , with probability
distribution function
0,
1

R < 0
R > 0

(B.8)

is related to a pair o f G aussian random variables C and D , through the transfor

m ation
C = R cos

(B .9)

D R sin

(B .10)

w here is a uniform ly distributed variable in the interval (0, 2 n ). T h e param eter

ct2 is the variance o f C and D . Since (B . 8 ) is easily inverted, w e have
F (R ) = 1 - e ~ R2f2n2 = A
and h en ce
(B.ll)
w here A is a uniform ly distributed random variable in the interval ( 0, 1) . N ow if
w e gen erate a secon d uniform ly distributed random variable B and define
s=

2nB

(B.12)

then from (B .9 ) and (B .10), w e obtain tw o statistically in d ep en d en t G aussian dis

tributed random variables C and D .
T he m eth od described ab ove is o ften used in practice to gen erate G aussian
distributed random variables. A s sh ow n in Fig. B.5, these random variables have
a m ean value o f zero and a variance a 2. If a n on zero m ean G aussian random
variable is desired, then C and D can be translated by the addition o f the m ean
value.
A subroutine im plem enting this m eth od for generating G aussian distributed
random variables is given in Fig. B .6 .

Random Number Generators

B6
C

SUBROUTINE GAUSS CONVERTS A UNIFORM RANDOM

SEQUENCE XIN IN

S E Q U E N C E W I T H G ( 0 , S I G M A * *2)

PARAMETERS

XIN

SIGMA

YOUT

[0,1]

:U N I F O R M

TO A GAUSSIAN RANDOM

IN

0,1]

RANDOM NUMBER

:U N I F O R M I N

[0, 1 ]

RANDOM NUMBER

:S T A N D A R D D E V I A T I O N O F T H E G A U S S I A N
:O U T P U T F R O M T H E G E N E R A T O R

C
SUBROUTINE GAUSS
PI=4.0*ATAN

9XIN,B,SIGMA,YOUT)

( 1 .0)

B=2.0*PI*B
R=SQRT

(2,0*(S I G M A * * 2 ) * A L O G ( 1 . 0 / (1.0-XIN) ) )

Y O U T = R * C O S (B }
RETURN
END
C

NOTE:

C
C

YOU MUST PROVIDE AS

C
C

TO USE THE ABOVE

SUBROUTINE

FOR A

GAUSSIAN RANDOM NUMBER GENERATOR

INPUT TWO UNIFORM RANDOM NUMBERS
XIN AND B
XIN AND B MUST BE STATISTICALLY

INDEPENDENT

Figure B.6

S u b ro u tin e fo r g e n e ra tin g G a u s s ia n ra n d o m v a ria b le s

App. B

Appendix
Tables of Transition
Coefficients for the Design of
Linear-Phase FIR Filters

In Section 8.2.3 w e described a design m ethod for linear-phase F IR filters that

in volved the specification of H r (a>) at a set o f equally spaced frequencies (ok =
2 n ( k + cr)/M, w here a = 0 or a = 3 , k = 0, 1 , . . . , (M l ) / 2 for M od d and k = 0,
1 , 2 ___ _ (M /2 ) 1 for M even , w h ere M is the length o f the filter. W ithin the
passband o f the filter, w e select Hr (a>k) = 1, and in the stopband, H r (a)k ) 0. For
freq u en cies in the transition band, the values o f H r (cok) are op tim ized to m inim ize
the m axim um sid elob e in the stopband. This is called a m in im a x o p tim iza tio n
criterion.
T h e optim ization o f the values o f Hr (cu) in the transition band has been p er
form ed by R ab in er et al. (1970) and tables o f transition valu es have b een provided
in the published paper. A selected num ber o f the tables for low p ass FIR filters
are included in this appendix.
Four tables are given. T ab le C .l lists the transition coefficients for the
case a = 0 and o n e coefficient in the transition band for both M odd and M
ev en . T ab le C.2 lists th e transition coefficients for the case a = 0, and tw o
coefficients in the transition band for M od d and M ev en . T ab le C.3 lists the
transition coefficients for the case a = \ , M even and o n e coefficient in the
transition band. Finally, T able C.4 lists the transition coefficients for the case
a = 5 , M ev en , and tw o coefficients in the transition band. T h e tables also in
clude the level o f the m axim um sid elob e and a bandwidth param eter, denoted
as BW .
T o use the tables, w e begin with a set o f specifications, including (1) the
bandw idth o f the filter, which can be defined as (2 jt/A /)(B W + a ) , w here B W is
the num ber o f con secu tive frequencies at w hich H(a)k) = 1, (2) the width o f the
transition region, w hich is roughly 2 n / M tim es the n um ber o f transition coeffi
cients, and (3) the m axim um tolerab le sid elob e in the stopband. T he length o f the
filter can be selected from the tables to satisfy th e specifications.

C1

TABLE C.1

TRANSITION COEFFICIENTS FOR a = 0

Af Odd

BW

1
2
3
4
5
6
1
2
3
4
6
8
10
12
14
15
1
2
3
4
5
6
10
14
18
22
26
30
31
1
2
3
4
6
8
10
18
26
34
42
50
58
59
60
61

Minimax
Af = 15
-42.30932283
-41.26299286
-41.25333786
-41.94907713
-44.37124538
-56.01416588
Af = 33
43.03163004
-42.42527962
-42.40898275
-42.45948601
-42.52403450
-42.44085121
-42.11079407
-41.92705250
-44.69430351
-56.18293285
Af = 65
-43.16935968
-42.61945581
-42.70906305
-42.86997318
-43.01999664
-43.14578819
-43.44808340
-43.54684496
-43.48173618
-43.19538212
-42.44725609
-44.76228619
-59.21673775
Af = 125
-43.20501566
-42.66971111
-42.77438974
-42.95051050
-43.25854683
-43.47917461
-43.63750410
-43.95589399
-44.05913115
-44.05672455
-43.94708776
-43.58473492
-42.14925432
-42.60623264
-44.78062010
-56.22547865

Af Even
Ti

BW

Minimax

7i

0.43378296
0.41793823
0.41047636
0.40405884
0.39268189
0.35766525

1
2
3
4
5
6

0.42994995
0.41042481
0.40141601
0.39641724
0.39161377
0.39039917
0.39192505
0.39420166
0.38552246
0.35360718

1
2
3
4
5
8
10
12
14

Af = 16
-39.75363827
-37.61346340
-36.57721567
-35.87249756
-35.31695461
-35.51951933
Af = 32
-42.24728918
-41.29370594
-41.03810358
-40.93496323
-40.85183477
-40.75032616
-40.54562140
-39.93450451
-38.91993237

0.42919312
0.40903320
0.39920654
0.39335937
0.38950806
0.38679809
0.38129272
0.37946167
0.37955322
0.38162842
0.38746948
0.38417358
0.35282745

1
2
3
4
5
6
10
14
18
22
26
30

M = 64
-42.96059322
-42.30815172
-42.32423735
-42.43565893
-42.55461407
-42.66526604
-43.01104736
43.28309965
-43.56508827
43.96245098
-44.60516977
-43.81448936

0.42882080
0.40830689
0.39807129
0.39177246
0.38742065
0.38416748
0.37609863
0.37089233
0.36605225
0.35977783
0.34813232
0.29973144

0.42899170
0.40867310
0.39868774
0.39268189
0.38579101
0.38195801
0.37954102
0.37518311
0.37384033
037371826
0.37470093
0.37797851
0.39086304
0.39063110
0.38383713
0.35263062

1
2
3
4
5
7
10
18
26
34
42
50
58
62

Af = 128
-43.15302420
-42.59092569
-42.67634487
-42.84038544
-42.99805641
-43.25537014
-43.52547789
-43.93180990
-44.18097305
-44.40153408
-44.67161417
-45.17186594
-46.92415667
-49.46298973

0.42889404
0.40847778
0.39838257
0.39226685
0.38812256
0.38281250
0.3782638
0.37251587
0.36941528
0.36686401
0.36394653
0.35902100
0.34273681
0.28751221

0.42631836
0.40397949
0.39454346
0.38916626
0.38840332
0.40155639
0.42856445
0.40773926
0.39662476
0.38925171
0.37897949
0.36990356
0.35928955
0.34487915
0.34407349

Source: Rabiner ct al. (1970); 1970 IEEE; reprinted with permission.

App. C

Transition Coefficients for the Design of Linear-Phase FIR Filters

C3

TABLE C.2 TRANSITION COEFFICIENTS FOR a = 0

M Odd

BW

1
2
3
4
5
1
2
3
5
7
9
11
13
14
1
2
3
4
5
9
13
17
21
25
29
30
1
2
3
5
7
9
17
25
33
41
49
57
58
59
60

Minimax
Af
-70.60540585
-69.26168156
-69.91973495
-75.51172256
-103.45078300

M Even
T\

= 15
0.09500122
0.10319824
0.10083618
0.08407953
0.05180206
M = 33
-70.60967541
0.09497070
-68.16726971
0.10585937
0.10937500
-67.13149548
-66.53917217
0.10965576
-67.23387909
0.10902100
0.10502930
-67.85412312
-69.08597469
0.10219727
-75.86953640
0.08137207
104.04059029
0,05029373
M = 65
70,66014957
0.09472656
68,89622307
0.10404663
-67.90234470
0.10720215
0.10726929
67.24003792
-66.86065960
0.10689087
0.10548706
-66.27561188
-65.96417046
0.10466309
0.10649414
-66.16404629
0.10701904
-66.76456833
-68.13407993
0.10327148
-75.98313046
0.08069458
-104.92083740
0.04978485
M = 125
-70.68010235
0.09464722
0.10390015
-68.94157696
-68.19352627
0,10682373
0.10668945
-67.34261131
0.10587158
-67.09767151
0.10523682
-67.058012%
-67.17504501
0.10372925
-67.22918987
0.10316772
-67.11609936
0.10303955
-66.71271324
0.10313721
-66.62364197
0.10561523
-69.28378487
0.10061646
-70.35782337
0.09663696
-75.94707718
0.08054886
-104.09012318
0.04991760

Ti

BW

Minimax

Tx

t2

M = 16

0.58995418
0.59357118
0.58594327
0.55715312
0.49917424

1
2
3
4
5

0.58985167
0.59743846
0.59911696
0.59674101
0.59417456
0.58771575
0.58216391
0.54712777
0.49149549

1
2
3
5
7
9
11
13

0.58945943
059.476127
0,59577449
0.59415763
0.59253047
0.58845983
0.58660485
0.58862042
0.58894575
0.58320831
0.54500379
0.48965181

1
2

-65.27693653
-62.85937929
-62.96594906
-66.03942485
-71.73997498

0.10703125
0.12384644
0.12827148
0.12130127
0.11066284
M = 32
-67.37020397
0.09610596
-63.93104696
0.11263428
-62.49787903
0.11931763
0.12541504
-61.28204536
-60.82049131
0.12907715
-59.74928167
0.12068481
-62.48683357
0.13004150
-70.64571857
0.11017914

0.60559357
0.62201631
0.62855407
0.61952704
0.60979204
0.59045212
0.60560235
0.61192546
0.61824023
0.62307031
0.60685586
0.62821502
0.60670943

M = 64

3
4
5
9
13
17
21
25
29

-70.26372528
-67.20729542
-65.80684280
-64.95227051
-64.42742348
-63.41714096
-62.72142410
-62.37051868
-62.04848146
-61.88074064
-70.05681992

1
2
3
4
6
9
17
25
33
41
49
57
61

-70.58992958
-68.62421608
-67.66701698
-66.95196629
-66.32718945
-66,01315498
-65.89422417
-65.92644215
-65.95577812
-65.97698021
-65.67919827
-64.61514568
-71.76589394

0.09376831
0.10411987
0.10850220
0.11038818
0.11113281
0.10936890
0.10828857
0.11031494
0.11254273
0.11994629
0.10717773

0.58789222
0.59421778
0.59666158
0.59730067
0.59698496
0.59088884
0.58738641
0.58968142
0.59249461
0.60564501
0.59842159

M --= 128

0.58933268
0.59450024
0.59508549
0.59187505
0,59821869
0.58738706
0.58358265
0.58224835
0.58198956
0.58245499
0.58629534
0.57812192
0.57121235
0.54451285
0.48963264

Source: Rabiner et al. (1970); 1970 IEEE; reprinted with permission.

0.09445190
0.10349731
0.10701294
0.10685425
0.10596924
0.10471191
0.10288086
0.10182495
0.10096436
0.10094604
0.09865112
0.09845581
0.10496826

0.58900996
0.59379058
0.59506081
0.59298926
0.58953845
0.58593906
0.58097354
0.57812308
0.57576437
0.57451694
0.56927420
0.56604486
0.59452277

C4

Transition Coefficients for the Design of Linear-Phase FIR Filters

TABLE C.3 TRANSITION

COEFFICIENTS FOR a = \
BW
1
2
3
4
5
6
1
2
3
4
6
8
10
12
14
1
2
3
4
5
6
10
14
18
22
26
30
1
2
3
4
5
7
10
18
26
34
42
50
58
62

Minimax
Af = 16
-51.60668707
-47.48000240
-45.19746828
-44.32862616
-45.68347692
-56.63700199
M = 32
-52.64991188
-49.39390278
-47.72596645
-46.68811989
-45.33436489
-44.30730963
-43.11168003
-42.97900438
-56.32780266
Af = 64
-52.90375662
-49.74046421
-48.38088989
-47.47863007
-46.88655186
-46.46230555
-45.46141434
-44.85988188
-44.34302616
-43.69835377
-42.45641375
-56.25024033
Af = 128
-52.96778202
-49.82771969
-48.51341629
-47.67455149
-47.11462021
-46.43420267
-45.88529110
-45.21660566
-44.87959814
-44.61497784
-44.32706451
-43.87646437
-42.30969715
-56.23294735

h
0.26674805
0.32149048
0.34810181
0.36308594
0.36661987
0.34327393
0.26073609
0.30878296
0.32984619
0.34217529
0.35704956
0.36750488
0.37810669
0.38465576
0.35030518
0.25923462
0.30603638
0.32510986
0.33595581
0.34287720
0.34774170
0.35859375
0.36470337
0.36983643
0.37586059
0.38624268
0.35200195
0.25885620
0.30534668
0.32404785
0.33443604
0.34100952
0.34880371
0.35493774
0.36182251
0.36521607
036784058
037066040
037500000
0.38807373
035241699

Source: Rabiner el al. (1970); 1970

IEEE; reprinted with permission.

App. C

App. C

Transition Coefficients for the Design of Linear-Phase FIR Filters

TABLE C.4 TRANSITION COEFFICIENTS FOR

1

BW
1
2
3
4
5
1
2
3
5
7
9
11
13
1
2
3
4
5
9
13
17
21
25
29
1
2
3
4
6
9
17
25
33
41
49
57
61
Source;

Minimart

Ti

M = 16
-77.26126766 0.05309448
-73.81026745 0.07175293
-73.02352142 0.07862549
-77.95156193 0.07042847
-105.23953247 0.04587402
M = 32
-80.49464130 0.04725342
-73.92513466 0.07094727
-72.40863037 0.08012695
-70.95047379 0.08935547
-70.22383976 0.09403687
-69.94402790 0.09628906
-70.82423878 0.09323731
-104.85642624 0.04882812
M = 64
-80.80974960 0.04658203
-75.11772251 0.06759644
-72.66662025 0.07886963
-71.85610867 0.08393555
-71.34401417 0.08721924
-70.32861614 0.09371948
-69.34809303 0.09761963
-68.06440258 0.10051880
-67.99149132 0.10289307
-69.32065105 0.10068359
-105.72862339 0.04923706
M == 128
-80.89347839 0.04639893
-77.22580583 0.06295776
-73.43786240 0.07648926
-71.93675232 0.08345947
-71.10850430 0.08880615
-70.53600121 0.09255371
-69.95890045 0.09628906
-69.29977322 0.09834595
-68.75139713 0.10077515
-67.89687920 0.10183716
-66.76120186 0.10264282
-69.21525860 0.10157471
-104.57432938 0.04970703
Rabiner et al.

reprinted with permission.

T2
0.41784180
0.49369211
0.51966134
0.51158076
0.46967784
0.40357383
0.49129255
0.52153983
0.54805908
0.56031410
0.56637987
0.56226952
0.48479068
0.40168723
0.48390015
0.51850058
0.53379876
0.54311474
0.56020256
0.56903714
0.57543691
0.58007699
0.57729656
0.48767025
0.40117195
0.47399521
0.51361278
0.53266251
0.54769675
0.55752959
0.56676912
0.57137301
0.57594641
0.57863142
0.58123560
0.57946395
0.48900685

(1970); 1970 IEEE;

05

C6

Transition Coefficients for the Design of Linear-Phase FIR Filters

App. C

A s an illustration, the filter design for which Af = 15 and

corresponds to a = 0, B W = 4, sin ce H r (p>k) = 1 at th e four con secu tive fre

quencies a>k = I n k 115, k 0, 1, 2, 3, and the transition coefficien t is T\ at the
frequency a>k = 8 n / l 5 . T he valu e given in T ab le C .l for M = 15 and B W = 4 is
7\ = 0.40405884. T h e m axim um sid elo b e is at 41.9 d B , according to T ab le C .l.

Appendix D
List of MATLAB Functions

In this A p p en d ix, w e list several M A T L A B functions that th e student can use to

so lv e som e o f the problem s num erically. T h e list includes th e m ost relevant M A T
L A B functions for each o f the chapters, but it is not exh au stive. H ow ever, this list
is cum ulative in the sense that on ce a function is listed in any chapter, it is not re
p eated in subsequent chapters. T h ese M A T L A B functions are obtained from two
sources: (1) the student version o f M A T L A B and (2) the b o o k en titled D igital S ig
n a l P ro cessin g U sing M A T L A B . (PW S K ent 1996), by V .K . Ingle and J.G . Proakis.
O ur primary objective in listing these M A T L A B fun ction s is to inform the
student w ho is not fam iliar with M A T L A B o f the existen ce o f these functions and
to encourage the student to use them in the solution o f som e o f the hom ew ork
problem s.
CH APTER 1
sin (x ), c o s (x ), ta n (x )

trigonom etric functions

a b s(x )

absolute values o f a vector x with real or com plex

com ponents.

r e a l(x )

takes the real part o f each com p on en ts o f the vector

x.

im a g (x )

takes the im aginary part o f each com p onent o f the

vector x .

c o n j(x )

com plex-conjugate o f each com p on en t o f x .

e x p (z )

ex(co s^ + j sin >), w here z = x + j y ,

su m (x )

sum o f the (real or com p lex) com p onents o f the vec

tor x .

p ro d (x )

product o f the (real or com p lex) com p on en ts of the

vector x .

a n g le(x )

com p utes the phase angles o f each com p onent o f the

vector x .

log(a;>

com p utes the natural logarithm o f each o f the e le

m en ts o f x .
D1

D2

List of MATLAB Functions

App. D

loglO(x)

com p utes the logarithm to the base 1 0 o f the elem en ts

of x.

sq r t(x )

com p utes the square root o f the elem en ts o f x .

CH APTER 2

conv(x, h )

con volu tion o f the tw o (vector) seq u en ce s x and h .

fliplr(x)

folds the (vector) seq u en ce x .

filter(b, a , x )

solves the differen ce eq u ation with coefficients

a = [a0, a \ , . . . , a ]
b = [fr0, b i , . . . , b m]
x = input seq u en ce

filler(6.1, x )

im plem ents an F IR filter with input x and coeffi

cien ts b.

r a n d (l, N )

gen erates a length N random seq u en ce that is uniform

in the interval ( 0 , 1 ).

r a n d n (l, N )

gen erates a length N seq u en ce o f G uassian random

variables with zero m ean and unit variance,

xcorr(x, y )

com p utes the crosscorrelation o f th e tw o seq u en ces x

and y.

xcorr(x)

com p utes the autocorrelation o f the seq u en ce x .

CH APTER 3

roots(a)

com p utes the roots o f the polyn om ial w ith coefficients

a = [ao ,

residuez(b, a )

, q n \

com p utes the residues in a partial fraction expansion,

where
b = coefficients o f num erator polyn om ial
[i>o, b \ , . . . b M]
a = coefficients o f d en om in ator polyn om ial
[ a o , >i, -<3w]

deconv(i>, a)

com p utes the result o f dividing b by a in a polyn om ial

part p and a rem ainder r.

poly(r)

com p utes the coefficients o f the p olyn om ial p with

roots r.

pzplotz(b, a )

plots the p o les and zeros in the z-p lane given th e co

efficient vectors b and a .

App. D

List of MATLAB Functions

fi)ter(b, a , x , x ic )

D3

im plem en ts the filter given by a d ifferen ce eq u ation

with coefficient vector b and a, input x and initial
con d ition s x ir.

CHAPTER 4
freqz(6, a , N )

com p utes an //-p o in t com p lex frequency resp onse v ec

tor and an Appoint frequency vector o j , uniform over
the interval 0 < co < n , for filter with coefficient v e c
tor & and a.

freqz(6, a , N , whole)

sam e com p utation as freqz(, a, N), excep t that the

freq u en ce range is 0 < co < 2n.

freqz(b, a , u>)

com p utes the frequency resp on se o f the system at the

freq u en cies specified by th e vector

grpdelay(t>, a, N )

com p utes the group d elay o f the filter w ith num era
tor polyn om ial having coefficients b and denom inator
p olyn om ial with coefficients o , at N p oints over the
interval (0 , jr).

grpdelay(b, a , TV, whole)

sam e as ab ove, ex cep t that the frequency range is 0 <

oj <

In .

CHAPTER 5
dfs(x, N )

com p utes the discrete fourier series (D F S ) coefficient

array for the period ic signal seq u en ce x with period

Jdfs ( y , N )

com p utes the signal seq u en ce from the D F S co effi

cient array y.

N.

rem (n. N )

determ in es the rem ainder after dividing n by N.

m od(n , N )

com p utes n m od N.

dft(x, N )

com p utes the Appoint D F T o f the data seq u en ce x.

idft(AT, N )

com p utes the N -point inverse D F T o f X .

ovrlpsav(x, h, N )

im plem ents the overlap -save m eth od to perform block

con volu tion w here N is the block length.

CHAPTER 6
ffl(x , N )

im plem ents a radix-2, W -point F F T algorithm ,

i (X , N )

im plem en ts a radix-2, N -p oin t inverse F F T algorithm ,

fftshift(x)

rearranges the ou tputs o f fft so that the zero fre

quency com p on en t is the cen ter o f the spectrum .

CHAPTER 7
dir2cas(b, a)

converts a direct form structure to the cascade form.

D4

List of MATLAB Functions

App. D

cas2dir(bo, B , A )

converts a cascade structure to the direct form struc

ture.

dir2par(b, a )

converts direct form to the parallel form structure,

par2dir(C0B, A )

converts a parallel form to the direct form structure,

dir21atc(b)

converts a F IR direct form structure to an all-zero

latice structure.

latc2dlr(AT)

converts an all-zero lattice structure to the direct form

structure.

dir2ladr(b, a)

converts direct form IIR structure to p ole-zero latticeladder structure.

Iadr2dir(AT, C )

converts a lattice-ladder structure to the direct form

IIR structure.

casfiltr(60, B , A , x )

im plem ents the cascade form IIR and F IR realization

o f a filter with input seq u en ce x .

parfiltr(C, B , A , x )

im plem ents the parallel form IIR realization o f a dilter with input seq u en ce x .

latcfilt(i, x )

im plem ents the F IR lattice filter realization with input

seq u en ce x .

ladrfilt(K, C . x )

im plem ents the lattice-ladder realization o f a filter

with input seq u en ce x .

round(x)

rounds the com p on en ts o f the vector x to the nearest

integer

fix(x)

rounds (truncates) the com p on en t o f th e vector x to

the nearest in teger tow ard zero,

sign(x)

each com p onent o f x is set to + 1 if it is positive and

1 if it is negative.

ss2tf(J4 , jB, C , D , i u )

com p utes the transfer function H ( x ) o f a system given

the state-sp ace description o f the form
x A x + Bu
y = Cx + Du
from the i'wth input.

ss2zp(J4, D , C , D , tu )

com p utes the transfer function H (s ) and exp resses it

in factored form , thus, giving the p o les and zeros o f
H (s).

CH APTER 8

boxcar(M)

gen erates an A f-point rectangular w indow ,

bartlett(M )

gen erates an M -p oint B artlett w indow ,

hanning(M)

gen erates an Af-point H an n ing w indow .

App. D

List of MATLAB Functions

ham m iitg(M )

gen erates an M -p oint H am m ing w indow ,

blackm an(M )
kaiser(M )
buttap(iV)

gen erates an M -point B lackm an w indow ,

D5

gen erates an M -point kaiser w in d ow ,

p rovides th e coefficients o f an analog low pass B u tter
w orth filter o f order N , w ith norm alized frequency, in
cascade form.

chebiap(iV, R )

provides the coefficients o f an analog low p ass C heby

shev filter o f order N , with norm alized frequency and
passband ripple R p, in cascade form ,

ellipap(7V, R P, A S)

p rovides the coefficients o f an an alog low pass elliptic

filter o f order N , passband ripple R p, stopband atten u
ation A 2, with norm alized frequency, in cascad e form,

freqs(fc>, a , u>)

com p utes th e frequency resp onse o f an analog filter,

with co in rad/sec.

butter(iV, u n )

designs a digital low pass B utterw orth filter o f order

N and cu toff frequency con.

chebyl(7V, R p , urn)

designs a digital low pass C h eb ysh ev filter o f order N

passband ripple R p, and cu toff frequency con.

cheby2(AT, A S i u>n)

designs a T ype 2 low pass C h eb ysh ev filter or order

N , stopband ripple A and cu to ff frequency con.

e l l i p ( N , R p , A s ,u>n)

designs a digital low pass elliptic filter o f order N , passband ripple R p , stopband ripple A s, and cu to ff fre
quency cun.

bilinear(z, p. fc, fs)

uses the bilinear transform ation to convert an analog

filter with zeros z, p o les p , and gain k, in to a digital
filter, with fs being the sam ple frequency in H z.

bilinear(num, den, fs)

uses th e bilinear transform ation to con vert an ana

log filter with num erator p olyn om ial coefficients num,
den om in ator p olyn om ial coefficients den, and sam ple
frequency fs, in to a digital filter,

remez(7V, / , m )

uses the R em ez algorithm to d eterm in e the coeffi

cients o f an optim um equiripple, linear p h ase F IR fil
ter o f length N + 1, from freq u en cy specifications /
and gains m for each band.

remez ( N , / , m , ftype')

sam e description as ab ove with fty p e used to specify

a H ilbert transform or differentiator,

butter(7V, w n , high)

designs a highpass B utterw orth filter o f order N and

3-dB cu toff frequency con.

botter(iV, w n , bandpass)

designs a 2 N -order bandpass B utterw orth filter, with

3-dB passband col < co < co2, w h ere con = [twl, co2\.

chebyl(iV , i i,,, w n , high)

designs a highpass C hebyshev filter o f order N , pass

band ripple R p, and cu toff frequency con.

List of MATLAB Functions

D6

App. D

ellilp(iV, R p , A s , w n )

designs an ellip tic bandpass filter o f order N , pass

band ripple Rp , stopband atten u ation A s , and cu toff
freq u en cies a>n = [a>l, w2],

lp2bp(num, den, u>o, B u t)

transform s an analog low pass filter to an analog band

pass filter.

.B u r)

transform s an analog low pass filter to an analog b and

stop filter.

lp2bs(num, den,

u jo

lp2hp(num, den, u o )

transform s an analog low pass filter to an analog highpass filter.

Ip2lp(num, den, u>o)

transform s an analog low p ass filter to an analog lowpass filter with cu toff frequency coo.

polyfit(x, y , n )

finds a p olyn om ial p such that p ( x ) fits the data in a

vector y in a least squares sense.

CH APTER 9

spline(x, y , x i )
spline(jTit, x , t)

cubic spline data interpolation,

uses cubic spline interp olation , w h ere x and n t s are
arrays containing sam ples x ( n ) at m s , and t is an ar
ray that contains a fine grid at w hich the function is
evaluated.

C H A P T E R 10

dnsam ple(x, M )

dow n sam p les the seq u en ce x by th e factor M .

L:_ References and Bibliography

A k a ik e , H. 1969. P o w er Spectru m E stim atio n T hrou gh A u toreg ressio n M o d el F ittin g ,"

A n n . Inst. Stat. M ath., V o l. 21, pp. 4 0 7 -1 4 9 .

A k a ik e , H. 1974. A New L o o k at the S tatistical M odel Id en tificatio n , I E E E Trans. A u to

m atic Control, V o l. A C -1 9 , pp. 7 1 6 -7 2 3 , D ecem b e r.

A n d e r s e n , N. O. 1978. C om m en ts on th e p erform an ce o f M axim um E n tro p y A lg o rith m .

Proc. I E E E , V o l. 66, pp. 1 5 8 1 -1 5 8 2 , N ovem ber.

A n t o n i o u , A . 1979. D ig ita l Filters: A n a ly s is and D e s ig n , M cG raw -H ili, New Y o rk .
A u e r , E. 1987. A D igital F ilte r Structu re F re e o f L im it C y cles, Proc. 19 8 7 I C A S S P ,

pp. 2 1 .1 1 .1 -2 1 .1 1 .4 , D allas, T e x ., A pril.

A v e n h a u s , E ., and S c h u e s s l e r , H. W . 1970. O n the A p p ro xim atio n P ro b lem in th e D esign

o f D igital F ille rs with L im ited W o rd len g th , A rch . E le k . Ubertragung, V o l. 2 4 , pp. 571

572.
B a g g e r o e r , A . B . 1976. C onfid en ce In tervals fo r R egression (M E M ) S p ectra ! E stim a te s,

I E E E Trans. Inform atio n Th e o ry , V ol. IT -2 2 , pp. 5 3 4 -5 4 5 , S ep tem b er.

B a n d l e r , J. W ., and B a r d a k j ia n , B . J. 1973. L e ast p ih O ptim ization o f R e cu rsiv e D igital

F ilte rs, I E E E Trans. A u d io and Electroacoustics, V o l. A U -2 1 , pp. 4 6 0 4 7 0 . O cto b er.

B a r n e s , C. W and F a m , A . T . 1977. M inim um N orm R ecu rsiv e D ig ital F ilte rs T h a t A re

F re e o f O verflow L im it C y cles, I E E E Trans. C ircu its an d Systems, V o l. C A S -2 4 , pp. 5 6 9 5 7 4 , O cto b er.

B a r t l e t t , M. S. 1948. Sm oo th in g Periodogram s from T im e S eries with C ontin u ou s S p e c

tr a , N ature (L o n d o n ), V o l. 161, pp. 6 8 6 -6 8 7 , M ay.

B a r t l e t t , M . S. 1961. Stochastic processes, C am bridge U niversity P ress, C am bridge.
B e r g l a n d , G. D. 1969. A G uided T o u r o f th e F ast F o u rie r T ra n sfo rm , I E E E Spectrum,

V o l. 6, pp. 4 1 - 5 2 , Ju ly .
B e r k , K. N. 1974. C o nsisten t A u toreg ressiv e Sp ectral E stim a te s, A n n . Stat., \ o \ . 2, p p . 4 8 9 502.
B e r n h a r d t , P. A., A n t o n i a d is , D. A., and D a R o s a , A. V. 1976. L u n ar P ertu rb ation s in

C o lu m n ar E le c tr o n C o n ten t and T h e ir In te rp retatio n in T erm s o f D y n am o E le ctro sta tic

F ie ld s, /. Geo ph ys. R es., V o l. 81, pp. 5 9 5 7 -5 9 6 3 , D ecem b e r.
B e r r y m a n , J. G. 1978. C h o ice o f O p e ra to r L en gth for M axim um E n tro p y S p e ctra l A n aly

sis, G e o p h ysics, V o l. 4 3 , pp. 1 3 8 4 -1 3 9 1 , D ecem b e r.

B l a c k m a n , R. B ., and T u k e y , J. W. 1958. The Measurem ent o f P o w er Spectra, D o v e r. New

Y o rk .

R1

R2

References and Bibliography

B l a h u t , R. E. 1985. Fast A lg o rithm s fo r D ig ita l S ig n a l Processing, A d d ison -W esley , R ead in g,

M ass.
B l u e s t e i n , L. I. 1970. A L in ear F ilterin g A p p ro ach to th e C o m p u tatio n o f the D iscrete
F o u rier T ra n sfo rm , I E E E Trans. A u d io an d Electro acou stics, V o l. A U -1 8 , pp. 4 5 1 -4 5 5 ,
D ecem b e r.
B o l t , B . A. 1988. Earth qu akes , W . H . F re em a n and C o ., N ew Y o rk .
B o m a r, B. W. 1985. New Seco n d -O rd e r S ta te -S p a ce Stru ctu res for R e a liz in g Low R o u n d o ff
N oise D igital F ilte r s, I E E E Trans. Acoustics, Speech, a n d Sig n a l Processing, V ol. A S S P 33, pp. 1 0 6 -1 1 0 , F eb ru ary .
B r a c e w e l l , R. N. 1978. T he Fo u rie r Transform an d Its A p p licatio n s, 2nd ed ., M cG raw -H ill,
New Y o rk .
B r ig h a m , E. O. 1988. The Fast Fo u rie r Transform a n d Its A p p licatio n s. P ren tice H all.
E n glew ood C liffs, N J .
B r ig h a m , E. 0 . , and M o r r o w , R. E. 1967. T h e F a st F o u rie r T ra n sfo rm , I E E E Spectrum ,
V o l. 4. pp. 6 3 -7 0 , D ecem b er.
B r i l l i n g e r , D. R. 1974. F o u rier A nalysis o f Statio n ary P ro cesses, Pro c. I E E E . V o l. 62,
pp. 1 6 2 8 -1 6 4 3 , D ecem b e r.
B r o p h y , F., and S a l a z a r , A. C. 1973, C o nsid eratio n s o f th e Pad e A p p ro xim an t T ech n iq u e
in the Synthesis o f R ecu rsiv e D igital F ilte r s, I E E E Trans. A u d io an d Electroacoustics ,
V ol. A U -2 1 . pp. 5 0 0 -5 0 5 . D ecem b er.
B r o w n . J. L J r ., 1980. F irst-O rd e r Sam plin g o f B an d p ass S ig n a ls -A N ew A p p ro a ch ,
I E E E Trans. Inform ation Theory. V o l. IT -2 6 , pp. 6 1 3 -6 1 5 , S ep tem b er.
B r o w n , R. C. 1983. Introduction to R an d o m S ig n a l A n a ly s is an d K a lm a n Filte rin g , W iiey,
New Y o rk .
B r u b a k e r , T. A., and G o w d y , J. N. 1972. L im it C ycles in D ig ital F ilte r s, I E E E Trans.
Autom atic Co n trol. V o l. A C -1 7 , pp. 6 7 5 -6 7 7 , O cto b er.
B r u z z o n e , S. P., and K a v e h , M. 1980. O n S o m e Suboptim u m A R M A Spectral E stim a
to rs . I E E E Trans. Acoustics, Speech, an d S ig n a l P ro cessing, V ol. A S S P -2 8 , pp. 7 5 3 -7 5 5 ,
D ecem b e r.
B u r g , J. P. 1967. M axim um E n trop y Sp e ctral A n aly sis, Proc. 37th M eeting o f the S o c i

ety o f Exp lo ratio n Geophysicists, O k lah om a C ity, O k la ., O cto b er. R e p rin te d in M odern
Spectrum A n a ly sis, D . G . C hildres, ed. I E E E press, N ew Y o rk .
B u r g , J. P. 1968. A New A nalysis T ech n iq u e fo r T im e S eries D a ta . N A T O A d vanced
Study Institu te on Signal Processin g with E m p h asis on U n d erw ater A cou stics, A ugust
1 2 -2 3 . R ep rin ted in M o d e m Spectrum A n a ly s is , D . G . C h ilders, ed ., I E E E P ress, New
Y o rk .
B u r g , J. P. 1972. T h e R elatio n sh ip B e tw ee n M axim um E n tro p y and M axim u m L ik elih o od
S p e ctra , G e o ph ysics, V o l. 37, pp. 3 7 5 -3 7 6 , A p ril.
B u r g , J. P. 1975, M axim um E n tro p y S p ectral A n aly sis, P h .D . d issertatio n , D ep artm en t
o f G eo p h ysics, Stan ford U niversity, Stan fo rd , C alif., M ay .
B u r r u s , C. S ., and P a r k s , T. W . 1970. T im e -D o m a in D esig n o f R e cu rsiv e D igital F ilte rs.
I E E E Trans. A u d io an d Electroacoustics, V o l. 18, pp. 1 3 7 -1 4 1 , Ju n e .
B u r r u s , C. S. and P a r k s , T . W . 1985. D F T / F F T an d Co n v o lu tio n A lgo rith m s, W iley, New
Y o rk .

References and Bibliography

R3

B u t t e r w e c k , H. J., v a n M e e r , A. C. P., and V e r k r o o s t , G. 1984. New Second-Order Digital

Filter Sections Without Limit Cycles," IE E E Trans. Circuits and Systems, Vol. CAS-31,
pp. 141-146, February.
C a d z o w , J. A . 1979. ARM A Spectral Estimation: An Efficient Closed-Form Procedure,

Proc. R A D C Spectrum Estimation W orkshop, pp. 81-97, Rome, N.Y., October.

C a d z o w , J. A. 1981. Autoregressive-Moving Average Spectral Estimation: A Model Equa

tion Error Procedure, IE E E Trans. Geoscience Remote Sensing, Vol. GE-19, pp. 24-28,
January.
C a d z o w , J. A. 1982. Spectral Estimation: An Overdetermined Rational Model Equation

Approach, Proc. IE E E , Vol. 70, pp. 907-938, September.

C a n d y , J. C . 1986. Decimation for Sigma Delta Modulation, IE E E Trans. Communica

tions, Vol. COM-34, pp. 72-76, January.

C a n d y , J. C , W o o le y , B. A., and B e n ja m in , D. J. 1981. A Voiceband Codec with Digital

Filtering, IE E E Trans. Communications, Vol. COM-29, pp. 815-830, June.

C a p o n , J. 1969. High-Resolution Frequency-Wavenumber Spectrum Analysis, Proc. IE E E ,

Vol. 57, pp. 1408-1418, August.

C a p o n , J. 1983. Maximum-Likelihood Spectral Estimation, in Nonlinear M ethods o f Spec

tral Analysis, 2nd ed., S. Haykin, ed., Springer-Verlag, New York.

C a p o n , J., and G o o d m a n , N. R. 1971. Probability Distribution for Estimators of the

Frequency-Wavenumber Spectrum, Proc. IE E E , Vol. 58, pp. 1785-1786, October.

C h a n , D. S. K and R a b i n e r , L. R. 1973a. Theory o f Roundoff Noise in Cascade Realiza

tions of Finite Impulse Response Digital Filters, Bell Syst. Tech. J., Vol. 52, pp. 329-345,
March.
C h a n , D. S. K., and R a b i n e r , L. R. 1973b. An Algorithm for Minimizing Roundoff Noise

in Cascade Realizations of Finite Impulse Response Digital Filters, Bell Sys. Tech. J.,
Vol. 52, pp. 347-385, March.
C h a n , D. S. K., and R a b i n e r , L. R. 1973c. Analysis of Quantization Errors in the Direct

Form for Finite Impulse Response Digital Filters, IE E E Trans. A udio and Electroacous
tics, Vol. AU-21, pp. 354-366, August.
C h a n g , T. 1981. Suppression of Limit Cycles in Digital Filters Designed w ith One

Magnitude-Truncation Quantizer, IE E E Trans. Circuits and Systems, Vol. CAS-28,

pp. 107-111, February.
C h e n , C . T. 1970. Introduction to Linear System Theory, Holt, Rinehart and Winston, New

York.
C h e n , W. Y., and S te g e w , G. R. 1974. Experiments with Maximum Entropy Power Spectra

of Sinusoids, J. Geophys. Res., Vol. 79, pp. 3019-3022, July.

C h i l d e r s , D. G ed. 1978. M odem Spectrum Analysis, IEEE Press, New York.
C h o w , J. C. 1972a. On the Estimation of the Order of a Moving-Average Process, IE E E

Trans. Autom atic Control, Vol. AC-17, pp. 386-387, June.

C h o w , J. C. 1972b. On Estimating the Orders of an Autoregressive-Moving Average Pro

cess with Uncertain Observations, IE E E Trans. A utom atic Control, Vol. AC-17, pp. 707709, October.
C h o w , Y., and C a s s ig n o l, E. 1962. Linear Signal Flow Graphs and Applications, Wiley, New

York.

R4

References and Bibliography

C h u i, C. K ., a n d C h e n , G. 1987. K a lm a n Filterin g , Sp rin g er-V erlag , N ew Y o rk .

C la a S e n , T . A. C. M ., M e c k l e n b r a u k e r , W. F. G ., and P e e k , J. B. H. 1 973. S eco n d -O rd er
D ig ital F ilte r with O n ly O n e M ag n itu d e-T ru ncation Q u an tizer and H aving P ractically No
L im it C y cles, Electron. Lett., V o l. 9, N ovem ber.
C o c h r a n , W . T., C o o l e y , J. W ., F a v in , D. L ., H e lm s, H. D., K a e n e l , R. A ., L a n g , W. W .,
M a l i n g , G. C., N e l s o n , D. E ., R a d e r . C. E and W e lc h , P. D. 1967. W h a t Is the F ast
F o u rier T ran sfo rm , I E E E Trans. A u d io an d Electroacoustics, V o l. A U -1 5 , pp. 4 5 -5 5 ,
Ju n e.
C o n s t a n t i n i d e s , A. G. 1967. F req u en cy T ran sfo rm atio n s for D ig ital F ilte r s, Electron. Lett.,
V o l. 3, pp. 4 8 7 -4 8 9 , N ovem ber.
C o n s t a n t i n i d e s , A. G. 1968. F req u en cy T ran sfo rm atio n s for D ig ital F ilte r s , Electron. Lett.,
V ol. 4 , pp. 1 1 5 -1 1 6 , A pril.
C o n s t a n t i n i d e s , A. G. 1970. Sp ectral T ran sfo rm atio n s fo r D ig ital F ilte r s , Proc. I E E ,
V o l. 117, pp. 1 5 8 5 -1 5 9 0 . A ugust.
C o o le y . J. W ., and T u k e y , J. W. 1965. A n A lgorithm for the M ach ine C om putation o f
C om plex F o u rier S e rie s , Math. C o m p ., V ol. 19. pp. 2 9 7 -3 0 1 . A pril.
C o o le y , J. W L e w is, P., and W e lc h , P. D. 1967. H istorical N otes on th e F ast F o u rier
T ra n sfo rm , I E E E Trans. A u d io and Electroacoustics, V o l. A U -1 5 , pp. 7 6 -7 9 , Ju n e .
C o o le y , J. W ., L e w is, P.. and W e lc h , P. D. 1969. T h e F ast F o u rier T ran sform and Its
A p p licatio n s, I E E E Trans. Ed u catio n . V ol. E -1 2 pp. 2 7 -3 4 , M arch.
C r o c h i e r e , R. E. 1977. O n the D esign o f Sub~band C od ers for Low B it R a te Speech
C o m m u n ication ," B e l l S y s t. T e c h . J., V ol. 56, pp. 7 4 7 -7 1 1 , M a v -Ju n e .
C r o c h i e r e , R. E. 1981. Sub-band C oding, B e ll Syst. Tech. J.. V o l. 6 0 . pp. 1 6 3 3 -1 6 5 4 ,
S ep tem b er.
C r o c h i e r e . R. E., and R a b i n e r , L. R. 1975. O ptim um F I R D ig ital F ilte r Im p lem en tatio n s
for D ecim atio n , In te rp o latio n , and N arrow band F ilterin g , I E E E Trans. A co ustics , Speech
and S ig n a l Processing, V o l. A S S P -2 3 , pp. 444456, O cto b er.
C r o c h i e r e , R. E., and R a b i n e r , L. R. 1976. F u rth e r C o nsid eratio n s in th e D esign o f
D ecim ato rs and In te rp o la to rs, I E E E Trans. A coustics, Speech, and S ig n a l Processing,
V ol. A S S P -2 4 , pp. 2 9 6 -3 1 1 , August.
C r o c h i e r e , R. E and R a b i n e r , L. R. 1981. In terp o lation and D ecim a tio n o f D ig ital S ig n a lsA T u to rial R ev iew , Proc. I E E E , V ol. 69, pp. 3 0 0 -3 3 1 , M arch .
C r o c h i e r e , R. E., and R a b i n e r , L. R . 1983. M ultirate D ig ita l S ig n a l P ro cessing, P ren tice H all,
E n gelw ood C liffs. N .J.
D a n ie ls , R. W , 1974. A p p ro xim atio n M ethods fo r the D e sign o f Passive, A ctiv e an d D ig ita l
Filters, M cG raw -H ill, New Y o rk .
D a v e n p o r t , W . B J r . 1970. P ro b ab ility and R an d o m Processes: A n Introduction fo r A p p lie d

Scientists an d Engineers. M cG raw -H ill, N ew Y o rk .

D a v is , H. F. 1963. F o u rie r Series an d O rtho go n al Functions. A llyn and B a c o n , B o sto n .
D e c z k y , A . G. 1972. Sy n th esis o f R ecu rsiv e D ig ital F ilters U sing th e M inim um p -E r ro r

C riterio n , I E E E Trans. A u d io an d Electroacoustics, V o l. A U -2 0 , pp. 2 5 7 -2 6 3 , O cto b er.

D e l s a r t e , P., and G e n i n , Y . 1986. T h e Split Levin son A lg o rith m , I E E E Trans. Acoustics,

Speech, an d Sig n al P ro ce ssin g , V o l. A S S P -3 4 , pp. 4 7 0 -4 7 8 , Ju n e .

References and Bibliography

R5

P., G e n i n , Y., and K a m p , Y. 1978. Orthogonal Polynomial Matrices on the Unit

Circle, IE E E Trans. Circuits and Systems. Vol. CAS-25, pp. 149-160. January.

D e ls a rte ,

D e R u s so , P. M., R o y , R. J., and C lo s e , C. M. 1965. State Variables fo r Engineers, Wiley, New

York.
D u h a m e l, P. 1986. Implementation of Split-Radix FFT Algorithms for Complex, Real, and

Real-Symmetric Data, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP34, pp. 285-295, April.
D u h a m e l, P., and H o l l m a n n , H. 1984. Split-Radix FFT Algorithm, Electron. Lett., Vol. 20,

pp. 14-16, January.

D u r b in , J. 1959. Efficient Estimation of Parameters in Moving-Average Models, Biome-

trika, Vol. 46, pp. 306-316.

D w i g h t, H. B. 1957. Tables o f Integrals and Other Mathematical Data, 3rd ed., Macmillan,

New York.
D ym , H., a n d M c K e a n , H . P. 1972. Fourier Series and Integrals, Academic, New York.
E b e r t , P. M M a z o , J. E., and T a y l o r , M . G. 1969. Overflow Oscillations in Digital Filters,

Bell Syst. Tech. J., Vol. 48, pp. 2999-3020, November.

F am , A. T., and B a r n e s , C. W. 1979. Non-minimal Realizations of Fixed-Point Digital Filters

That Are Free of All Finite Wordlength Limit Cycles, IE E E Trans. Acoustics, Speech,
and Signal Processing, Vol. ASSP-27, pp. 149-153, April.
F e t t w e i s , A. 1971. Some Principles of Designing Digital Filters Imitating Classical Filter

Structures, IE E E Trans. Circuit Theory, Vol. CT-18, pp. 314-316, March.

F l e t c h e r , R and P o w e l l , M. J. D. 1963. A Rapidly Convergent Descent Method for
Minimization, Comput. J., Vol. 6 , pp. 163-168.
F o u g e r e , P. F., Z a w a l ic k , E. J., a n d R a d o s k i, H. R. 1976. Spontaneous Line Splitting in

Maximum Entropy Power Spectrum Analysis, Phys. Earth Planet, Inter., Vol. 12, 201207, August.
F r i e d l a n d e r , B. 1982a. Lattice Filters for Adaptive Processing, Proc. IE E E , Vol. 70,

pp. 829-867, August.

F r i e d l a n d e r , B. 1982b. Lattice Methods for Spectral Estimation, Proc. IE E E , Vol. 70,

pp. 990-1017, September.

Fuchs, J. J. 1988. "Estimating the Number of Sinusoids in Additive White Noise, IE E E
Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-36, pp. 1846-1853, December.
G a n t m a c h e r , F. R. 1960. The Theory o f Matrices, Vol. I., Chelsea, New York.
G a r d n e r , W. A. 1987. Statistical Spectral Analysis: A Nonprobabilistic Theory, Prentice Hall,

Englewood Cliffs, N.J.

G a r l a n , C., and E s te b a n , D. 1980. 16 Kbps Real-Time QMF Sub-band Coding Implemen

tation, Proc. 1980 International Conference on Acoustics, Speech, and Signal Processing,
pp. 332-335, April.
L. Y. 1958. Orthogonal Polynomials (in Russian) (English translation
sultant's Bureau, New York, 1961).

G e r o n im u s ,

by

Con

G e r s c h , W., and S h a r p e , D. R. 1973. Estimation of Power Spectra with Finite-Order A u

toregressive Models, IE E E Trans. A utom atic Control, Vol. AC-18, pp. 367-369, August.
A. J. 1969. An Introduction to Digital Filters, A u s t Tellcommun. Res., Vol. 3,
pp. 3-14, November.

G ib b s ,

R6

References and Bibliography

A. J. 1970. The Design of Digital Filters, A use. Telecommun. Res., Vol. 4, pp. 29-34,
May.

G ib b s ,

1968. An Algorithm for the Evaluation of Finite Trigonometric Series, A m .

Math. Monthly, Vol. 65, pp. 34-35, January.

G o e rtz e l, G.

I., ed. 1986.1. Schiir Methods in Operator Theory and Signal Processing, Birkhauser
Verlag, Stuttgart, Germany.

G o h b e rg ,

B., and J o r d a n , K. L J r . 1986. A Note on Digital Filter Synthesis, Proc. IE E E ,

Vol. 56, pp. 1717-1718, October.

G o ld ,

B., and - J o r d a n , K. L., Jr. 1969. A Direct Search Procedure for Designing Finite
Duration Impulse Response Filters, IE E E Trans. A u d io and Electroacoustics, Vol. AU17, pp. 33-36, March.

G o ld ,

B., and R a d e r , C. M . 1966. Effects of Quantization Noise in Digital Filters. Proc.

AF1PS 1966 Spring Joint Computer Conference, Vol. 2 8 , pp. 2 1 3 -2 1 9 .

G o ld ,

B., and

G o ld ,

R a d e r,

C. M. 1969. Digital Processing o f Signals, McGraw-Hill, New York.

R. M., and K a i s e r , J. F . 1964. Design of Wideband Sampled Data Filters, Bell

Syst. Tech. J., Vol. 43, pp. 1533-1546, July.

G o ld e n ,

I. J. 1971. The Relationship Between Two Fast Fourier Transforms, IE E E Trans.

Computers, Vol. C-20, pp. 310-317.

G ood,

G o r s k i-P o p ie l,

J., ed. 1975. Frequency Synthesis: Techniques and Applications, IEEE Press,

New York.
D., K r a u s e , D. J., and M o o r e , J. B. 1975. Identification of Autoregressive-Moving
Average Parameters of Time Series, IE E E Trans. Autom atic Control, Vol. AC-20,
pp. 104-107, February.

G rau p e,

A. H., and M a r k e l , J. D. 1973. Digital Lattice and Ladder Filter Synthesis, IE E E

Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-21, pp. 491-500, December.

G ray ,

A. H., and M a r k e l , J. D. 1976. A Computer Program for Designing Digital Elliptic

Filters, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-24, pp. 529
538, December.

G ra y ,

G ray ,

R. M. 1990. Source Coding Theory, Kluwer, Boston, MA.

O,, and S z e g o , G . 1958. Teoplitz Forms and Their Applications, University of

California Press, Berkeley, Calif.

G re n a n d e r,

L. J. 1975. Rapid Measurements of Digital Instantaneous Frequency, IE E E

Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-23, pp. 207-222, April.

G r iff ith s ,

G u ille m in ,
G u p ta ,

E. A. 1957. Synthesis o f Passive Networks, Wiley, New York.

S. C. 1966. Transform and State Variable M ethods in Linear Systems, Wiley, New

York.
H a m m in g ,

R. W. 1962. Numerical Methods fo r Scientists and Engineers, McGraw-Hill, New

York.
H a y k in ,

S. 1991. Adaptive Filter Theory, 2nd ed., Prentice Hall, Englewood Cliffs, N J.

and N i k i a s , C. S. 1985. Improved Spectrum Performance via a Data-Adaptive

Weighted Burg Technique, IE E E Trans. Acoustics, Speech, and Signal Processing,
Vol. ASSP-33, pp. 903-910, August,

H e l m e , B .,

References and Bibliography

R7

D. 1967. Fast Fourier Transforms Method of Computing Difference Equations

and Simulating Filters, IE E E Trans. A udio and Electroacoustics, Vol. AU-15, pp. 85-90.
June.

H e lm s , H .

1968. Nonrecursive Digital Filters: Design Methods for Achieving Specifi

cations on Frequency Response, IE E E Trans. A u d io and Electroacoustics, Vol. AU-16,
pp. 336-342, September.

H e lm s , H . D .

C. W. 1990. Probability and Stochastic Processes fo r Engineers, 2nd ed., Macmil

lan, New York.

H e ls tro m ,

R. W. 1980. The Cause of Line Splitting in Burg Maximum-Entropy Spectral

Analysis, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-28, pp. 692701, December.

H e r r in g ,

O. 1970. Design of Nonrecursive Digital Filters with Linear Phase, Electron.

Lett., Vol. 6 , pp. 328-329, November.

H erm an n ,

O., and S c h u e s s l e r , H . W. 1970a. Design of Nonrecursive Digital Filters with

Minimum Phase, Electron. Lett., Vol. 6 , pp. 329-330, November.
H e r r m a n n , O., and S c h u e s s l e r , H . W. 1970b. On the Accuracy Problem in the Design of
Nonrecursive Digital Filters, Arch. Elek. Ubertragung, Vol. 24, pp. 525-526.
H e r r m a n n , O., R a b i n e r , L. R ., and C h a n , D. S. K. (1973). Practical Design Rules for
Optimum Finite Impulse Response Lowpass Digital Filters, Bell Syst. Tech. J., Vol. 52,
pp. 769-799. July-August.
H erm an n ,

H ild e b ra n d ,

F. B. 1952. M ethods o f A pplied Mathematics, Prentice Hall, Englewood Cliffs,

N.J.
E., O p p e n h e i m , A. V., and S i e g e l , J. 1971. A New Technique for the Design
of Nonrecursive Digital Filters, Proc. 5th A nnual Princeton Conference on Information
Sciences and Systems, pp. 64-72.

H o f s te tte r ,

H o u s e h o l d e r , A . S.

1964. The Theory o f Matrices in Numerical Analysis, Blaisdell, Waltham,

Mass.
S. Y, 1977. Minimum Uncorrelated Unit Noise in State Space Digital Filter
ing. IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-25, pp. 273-281.
August.
J a c k s o n , L. B. 1969. An Analysis of Limit Cycles Due to Multiplication Rounding in
Recursive Digital (Sub) Filters, Proc. 7th A nnual Allerton Conference on Circuit and
System Theory, pp. 69-78.
H w ang,

L. B . 1970a. On the Interaction of Roundoff Noise and Dynamic Range in Digital

Filters, Bell Syst. Tech. J., Vol. 49, pp. 159-184, February.

Jack so n ,

L. B. 1970b. Roundoff Noise Analysis for Fixed-Point Digital Filters Realized

in Cascade or Parallel Form, IE E E Trans. A udio and Electroacoustics, Vol. AU-18,
pp. 107-122, June.
J a c k s o n , L. B. 1976. Roundoff Noise Bounds Derived from Coefficients Sensitivities in
Digital Filters, IE E E Trans. Circuits and Systems, Vol. CAS-23, pp. 481-485, August.
J a c k s o n , L. B. 1979. Limit Cycles on State-Space Structures for Digital Filters, IE E E
Trans. Circuits and Systems, Vol. CAS-26, pp. 67-68, January.
Jackson,

B L i n d g r e n , A. G., and K jm , Y. 1979. Optimal Synthesis of Second-Order

State-3pace Structures for Digital Filters, IE E E Trans. Circuits and Systems,
Vol. CAS-26, pp. 149-153, March.

Ja c k s o n , L.

R8

References and Bibliography

J a h n k e . E .. a n d E m d e , F.

1945. Tables o f Functions, 4th ed.. Dover, New York.

V. K.. and C r o c h i e r e , R. E. 1984. Quadrature Mirror Filter Design in the Time

Domain," IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-32, pp. 353361, April.

J a in .

G. M., and
San Francisco.

J e n k in s ,

W a tts ,

D. G. 1968. SpectraI Analysis and Its Applications, Holden-Day,

D. 1980. A Filter Family Designed for Use in Quadrature Mirror Filter

Banks, IE E E International Conference on Acoustics, Speech, and Signal Processing,
pp. 291-294, April.

J o h n s t o n , J.

1982. The Application of Spectral Estimation Methods to Bearing Estima

tion Problems, Proc. IE E E , Vol. 70, pp. 1018-1028, September.

Jo h n so n , D. H.

R. H. 1976. Autoregression Order Selection, Geophysics, Vol. 41, pp. 771-773,

August.

Jo n es,

J u r y . E . I.

1964. Theory and Applications o f the z-Transform M ethod, Wiley, New York.

T. 1974. A View of Three Decades of Linear Filter Theory." IE E E Trans. Infor

mation Theory, Vol. IT-20, pp. 146-181, March.

K a ila th ,

T. 1981. Lectures on Wiener and Kalman Filtering, 2nd Printing, Springer-Verlag,

New York.

K a ila th ,

T. 1985. Linear Estimation for Stationary and Near-Stationary Processes. in

M odem Signal Processing, T. Kailath, ed.. Hemisphere Publishing Corp., Washington,
D.C.

K a ila th .

T. 1986. A Theorem of I. Schur and Its Impact on Modern Signal Processing,"

in Gohberg (1986).

K a ila th ,

T.. V i e i r a , A. C. G, and M o r f , M. 1978. Inverses of Toeplitz Operators, Innova

tions, and Orthogonal Polynomials. S IA M Rev., Vol. 20, pp. 1006-1019.

K a ila th ,

K a is e r . J. F. 1963. Design Methods for Sampled Data Filters." Proc. First Allerton Con
ference on Circuit System Theory, pp. 2 2 1 -2 3 6 , November.

J. F. 1966. Digital Filters, in System Analysis by Digital Computer, F. F. Kuo and

J. F. Kaiser, Eds., Wiley, New York.

K a is e r,

N., and T h e o d o r i d i s , S. 1987. Fast Adaptive Least-Squares Algorithms for

Power Spectral Estimation, IE E E Trans. Acoustics, Speech, and Signal Processing,
Vol. ASSP-35, pp. 661-670, May.

K a lo u p ts id is ,

R. E. 1960. A New Approach to Linear Filtering and Prediction Problems." Trans.

A SM E , J. Basic Eng., Vol. 82D, pp. 35-45, March.

K a lm a n ,

R. E., and B u c y , R. S. 1961. New Results in Linear Filtering Theory, Trans.

ASM E, J. Basic Eng., Vol. 83, pp. 95-108.

K a lm a n ,

R. L. 1980. Inconsistency of the AIC Rule for Estimating the Order of Autore
gressive Models, IE E E Trans. Autom atic Control, Vol. AC-25, pp. 996-998, October

K ashyap.

J., and B r u z z o n e , S. P. 1979. Order Determination for Autoregressive Spectral Es

timation." Record o f the 1979 R A D C Spectral Estimation W orkshop, pp. 139-145, Griffin
Air Force Base, Rome, N.Y.

K aveh,

M., and L i p p e r t , G. A . 1983. An Optimum Tapered Burg Algorithm for Linear

Prediction and Spectral Analysis, IE E E Trans. Acoustics, Speech, and Signal Processing,
Vol. ASSP-31, pp. 438-444, April.

K aveh,

References and Bibliography

R9

S. M. 1980. A New ARMA Spectral Estimator, IE E E Trans. Acoustics, Speech, and

Signal Processing, Vol. ASSP-28, pp. 585-588, October.

K ay,

K ay,

S. M. 1988. M odern Spectral Estimation, Prentice Hall, Englewood Cliffs, N. J.

K ay , S, M ., and M a r p l e , S. L ., J r . 1979. Sources of and Remedies for Spectral Line Splitting

in Autoregressive Spectrum Analysis, Proc. 1979 IC A SSP , pp. 1 5 1 -1 5 4 .

S. M ., and M a r p l e , S. L., J r . 1981. Spectrum Analysis: A Modern Perspective, Proc.

IE E E , Vol. 69, pp. 1380-1419, November.

K ay,

K e s le r,

S. B., ed. 1986. M odem Spectrum Analysis II, IEEE Press, New York.

B., and O l c a y t o , E . M. 1968. Coefficient Accuracy and Digital Filter R e

sponse, IE E E Trans. Circuit Theory, Vol. CT-15, pp. 31-41, March.

K n o w le s , J.

H. 1988. New Split Levinson, Schur, and Lattice Algorithms for Digital Signal
Processing, Proc. 1988 International Conference on Acoustics, Speech, and Signal Pro
cessing, pp. 1640-1642, New York, April.

K rish n a ,

R. E. 1969. Asymptotic Properties of the Autoregressive Spectral Estimator,

Ph.D. dissertation, Department of Statistics, Stanford University, Stanford, Calif.

K ro m er,

and Hu, Y . H. 1983. A Highly Concurrent Algorithm and Pipelined Architec

ture for Solving Toeplitz Systems, IE E E Trans. Acoustics, Speech, and Signal Processing,
Vol. ASSP-31, pp. 66-76, January.

K u n g , S. Y

S. Y., W h i t e h o u s e , H. J ., and K a i l a t h , T., eds. 1985. V L S I and Modern Signal

Processing, Prentice Hall, Englewood Cliffs, N.J.

K ung,

R. T. 1971. Data Adaptive Spectral Analysis Methods, Geophysics, Vol. 36,

pp. 661-675, August.

L aco ss,

S. W ., and M c C l e l l a n , J. H. 1980. Frequency Estimation with Maximum Entropy

Spectral Estimators, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP28, pp. 716-724, December.

L ang,

1947. The Wiener RMS Error Criterion in Filter Design and Prediction,
J. Math. Phys., Vol. 25, pp. 261-278.

L e v in so n , N.

L evy,

H and

L essm an,

F. 1961. Finite Difference Equations, Macmillan, New York.

B. 1971. Effect of Finite Word Length on the Accuracy of Digital Filters A Review,
IE E E Trans. Circuit Theory, Vol. CT-18, pp. 670-677, November.

L iu ,

J. 1975. Linear Prediction: A Tutorial Review, Proc. IE E E , Vol. 63, pp. 561
580, April.

M a k h o u l,

1978. A Class of All-Zero Lattice Digital Filters: Properties and Applica

tions, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-26, pp. 304-314,
August.
M arkel, J. D., and G r a y , A. H., Jr. 1976. Linear Prediction o f Speech, Springer-Verlag, New
York.
M a k h o u l, J.

S. L ., J r . 1980. A New Autoregressive Spectrum Analysis Algorithm. IE E E

Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-28, pp. 441-454, August.

M a rp le ,

S. L., J r . 1987. Digital Spectral Analysis with Applications, Prentice Hall, Englewood
Cliffs, N J.
a r z e t t a , T . L . 1983. A New Interpretation for Capons Maximum Likelihood Method
of Frequency-Wavenumber Spectral Estimation, IE E E Trans. Acoustics, Speech, and
Signal Processing, Vol. ASSP-31, pp. 445-449, April.

M a rp le ,

R10

References and Bibliography

T. L., and Lang, S. W. 1983. New Interpretations for the MLM and DASE
Spectral Estimators, Proc. 1983 IC A SSP , pp. 844-846, Boston, April.

M a rz e tta ,

T. L ., and L a n g , S. W. 1984. Power Spectral Density Bounds. IE E E Trans.

Information Theory, Vol. IT-30, pp. 117-122, January.

M a rz e tta ,

S. J., and
New York.

M aso n ,

Z im m e rm a n ,

H. J. 1960. Electronic Circuits, Signals and Systems, Wiley,

J. H. 1982. Multidimensional Spectral Estimation, Proc. IE E E , Vol. 70,

pp. 1029-1039, September.

M c C le lla n ,

R. N. 1983. Application of the Maximum-Likelihood Method and the Maxi

mum Entropy Method to Array Processing, in Nonlinear M ethods o f Spectral Analysis,
2nd ed., S. Haykin, ed., Springer-Verlag, New York.

M cD onough,

D., and C o o p e r , G . R. 1984. Continuous and Discrete Signal and System

Analysis, 2nd ed.. Holt Rinehart and Winston, New York.

M c G ille m , C .

M e d itc h ,

J. E. 1969. Stochastic Optimal Linear Estimation and Control. McGraw-Hill, New

York.
L., M u l l i s , C . T ., and R o b e r t s , R . A . 1981. Low Roundoff Noise and Normal
Realizations of Fixed-Point IIR Digital Filters. IE E E Trans. Acoustics, Speech, and
Signal processing, Vol. ASSP-29, pp. 893-903, August.

M ills , W .

A., and L e e , D. T. 1977. Ladder Forms for Identification and Speech

Processing, Proc. 1977 IE E E Conference Decision and Control, pp. 1074-1078, New
Orleans, La., December.

M o r f , M V ie ir a ,

C. T ., and R o b e r t s , R . A . 1976a. Synthesis of Minimum Roundoff Noise Fixed*

Point Digital Filters, IE E E Trans. Circuits and Systems, Vol. CAS-23, pp. 551-561,
September.

M u llis ,

C. T and R o b e r t s , R . A . 1976b. Roundoff Noise in Digital Filters: Frequency

Transformations and Invariants, IE E E Trans. Acoustics, Speech and Signal Processing,
Vol. ASSP-24, pp. 538-549, December

M u llis ,

1985. Fast M L M Power Spectrum Estimation from Uniformly Spaced Corre

lations, IE E E Trans. Acoustics, Speech, and Signal Proc., Vol. ASSP-33, pp. 1333-1335,
October.

M u s ic u s , B .

W. I. 1981. Extension to the Maximum Entropy Method III, Proc. 1st A SSP
Workshop on Spectral Estimation, pp. 1.7.11.7.6, Hamilton, Ontario, Canada, August.

N ew m an,

and R a g h u v e e r , M . R . 1987. Bispectrum Estimation:

cessing Framework, Proc. IE E E , Vol. 75, pp. 869-891, July.

N i k i a s , C . L .,

Digital Signal Pro

C. L and S c o t t , P. D. 1982. Energy-Weighted Linear Predictive Spectral Es

timation: A New Method Combining Robustness and High Resolution. IE E E Trans.
Acoustics, Speech, and Signal Processing, Vol. ASSP-30, pp. 287-292, April.

N ik ia s ,

A. H. 1976. Spectral Analysis of a Univariate Process with Bad Data Points, via
Maximum Entropy and Linear Predictive Techniques, N U SC Technical Report TR-5303,
New London, Conn., March.

N u tta ll,

H. 1928. Certain Topics in Telegraph Transmission Theory, Trans. A IE E , Vol. 47,

pp. 617-644, April.

N y q u is t,

A. V. 1978. Applications o f Digital Signal Processing, Prentice Hall, Englewood

Cliffs, N J.

O p p e n h e im ,

References and Bibliography

R11

A. V., and S c h a f e r , R. W. 1989. Discrete-Time Signal Processing, Prentice Hall,

Englewood Cliffs, N J.

O p p e n h e im ,

O p p e n h e im , A. V., and W e in s te in , C. W . 1972. Effects of Finite Register Length in Digital

Filters and the Fast Fourier Transform, Proc. IE E E , Vol. 60, pp. 957-976. August.
O p p e n h e im , A. V ., and W i l l s k y , A. S. 1983. Signals and Systems, Prentice Hall, Englewood

Cliffs, N J.
P a p o u lis ,

A. 1962 The Fourier Integral and Its Applications, McGraw-Hall, New York.

A. 1984. Probability, R andom Variables, and Stochastic Processes, 2nd ed.,

McGraw-Hill, New York.

P a p o u lis ,

P a r k e r , S. R., and H ess, S. F. 1971. Limit-Cycle Oscillations in Digital Filters, IE E E Trans.

Circuit Theory, Vol. CT-18, pp. 687-696, November.

P a r k s , T . W., and M c C l e l l a n , J. H. 1972a. Chebyshev-Approximation for Nonrecursive
Digital Filters with Linear P h a s e , IE E E Trans. Circuit Theory, Vol. C T -1 9 , pp. 1 8 9 -1 9 4 ,
M a rc h .

T. W., and M c C l e l l a n , J. H. 1972b. A Program for the Design of Linear Phase Finite
Impulse Response Digital Filters, IE E E Trans. A udio and Electroacoustics, Vol. AU-20,
pp. 195-199, August.

P a rk s,

E. 1957. On Consistent Estimates of the Spectrum of a Stationary Time Series,

A m . Math. Stat., Vol. 28, pp. 329-348.

P a rze n ,

E. 1974. Some Recent Advances in Time Series Modeling, IE E E Trans. A u to

matic Control, Vol. AC-19, pp. 723-730, December.

P a rze n ,

K. L, and T r e i t e l , S. 1969. Predictive Deconvolution-Theory and Practice, Geo

physics, Vol. 34, pp. 155-169.

P eacock,

Z., J r . 1987. Probability, Random Variables, and Random Signal Principles, 2nd
ed., McGraw-Hill, New York.

P e e b le s , P.

V. F. 1973. The Retrieval of Harmonics from a Convariance Function, Geophys. J. R. Astron. Soc., Vol. 33, pp. 347-366.

P isa re n k o ,

1981. Autoregressive M ethods o f Spectral Analysis, E.E. degree thesis, Depart

ment of Electrical and Computer Engineering, Northeastern University, Boston, May.

P o o le , M . A.

P r ic e , R. 1990. Mirror FFT and Phase FFT Algorithms, unpublished work, Raytheon

Research Division, May.

P r o a k is ,

J. G. 1995. Digital Communications, 3rd ed., McGraw-Hill, New York.

R a b i n e r , L. R., and S c h a e f e r , R. W. 1974a. On the Behavior of Minimax Relative Error

FIR Digital Differentiators, Bell Syst. Tech. J., Vol. 53, pp. 333-362, February.
R a b i n e r , L. R ., and S c h a f e r , R . W . 1974b. On th e Behavior of Minimax F I R Digital Hilbert

Transformers, Bell SysL Tech. J., Vol. 53, pp. 363-394, February.
R a b i n e r , L. R ., and S c h a f e r , R. W. 1978. Digital Processing o f Speech Signals, Prentice Hall,

Englewood Cliffs, N J.
R a b i n e r , L. R., S c h a f e r , R . W ., and R a d e r , C. M. 1969. The Chirp ^-Transform Algorithm

and Its Applications, Bell S yst Tech. J., Vol. 48, pp. 1249-1292, May-June.
R a b i n e r , L . R ., G o l d , B., a n d M c G o n e g a l, C . A . 1970. An Approach to the Approxima

tion Problem for Nonrecursive Digital Filters, IE E E Trans. A u d io and Electroacoustics,

Vol. AU-18, pp. 83-106, June.

R12

References and Bibliography

W. 1975. F I R Digital Filter Design Tech

niques Using Weighted Chebyshev Approximation, Proc. IE E E , Vol. 63, pp. 595-610,
April.

R a b i n e r , L . R ., M c C l e l l a n , J . H ., a n d P a r k s , T .

1970. An Improved Algorithm f o r High-Speed Auto-correlation with Appli

cations to Spectral Estimation, IE E E Trans. A udio and Electroacoustics, Vol. AU-18,
pp. 439-441, December.

R a d e r, C. M .

C. M., and B r e n n e r , N. M. 1976. A New Principle for Fast Fourier Transformation,

IE E E Trans. Acoustics, Speech and Signal Processing, Vol. ASSP-24, pp. 264-266, June.

R a d e r,

C. M., and G o l d , B. 1967a. Digital Filter Design Techniques in the Frequency

Domain, Proc. IE E E , Vol. 55, pp. 149-171, February.

R ad er,

C. M., and G o l d , B. 1967b. Effects of Parameter Quantization on the Poles of a

Digital Filter, Proc. IE E E , Vol. 55, pp. 688-689, May.

R ad er,

1984. Digital Methods for Conversion Between Arbitrary Sampling Fre

quencies, IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-32, pp. 577591, June.

R a m s ta d , T. A.

1957. General Computational M ethods o f Chebyshev A pproxim ation, Atomic

Energy Translation 4491, Kiev, USSR.

R e m e z, E. Y a.

J. 1983. A Universal Prior for the Integers and Estimation by Minimum D e

scription Length, Ann. Stat., Vol. 11, pp. 417-431.

R js s a n e n .

R o b e r t s , R . A .,

and

M u llis ,

C.

T,

1987. Digital Signal Processing, Addison-Wesley, Reading,

Mass.
R o b in s o n ,

E. A. 1962. R andom Wavelets and Cybernetic Systems, Charles Griffin, London.

E. A. 1982. A Historical Perspective of Spectrum Estimation, Proc. IE E E ,

Vol. 70, pp. 885-907, September.

R o b in so n ,

E. A., and T r e i t e l , S. 1978. Digital Signal Processing in Geophysics, in A p

plications o f Digital Signal Processing, A. V. Oppenheim, ed., Prentice Hall, Englewood
Cliffs, N.J.

R o b in s o n ,

and
wood Cliffs, N.J.

R o b i n s o n , E . A .,

T r e i t e l , S.

1980. Geophysical Signal Analysis, Prentice Hall, Engle

A., and K a i l a t h , T. 1986. ESPRIT: A Subspace Rotation Approach to

Estimation of Parameters of Cisoids in Noise, IE E E Trans. Acoustics, Speech, and Signal
Processing, Vol. ASSP-34, pp. 1340-1342, October.

R o y , R ., P a u l r a j ,

R. J ., M a c K a y , K . J a y a n t , N. W., and K im , T. 1988. Image Coding Based on

Selective Quantization of the Reconstruction Noise in the Dominant Sub-band, Proc.
1988 IE E E International Conference on Acoustics, Speech, and Signal Processing, pp. 765768, April.

S a fra n e k ,

H. 1979. Statistical Properties of AR Spectral Analysis, IE E E Trans. Acoustics,

Speech, and Signal Processing, Vol. ASSP-27, pp. 402-409, August.

S a k a i,

I. W ., and K a i s e r , J. F. 1972. A Bound on Limit Cycles in Fixed-Point Im

plementations of Digital Filters, IE E E Trans. A udio and Electroacoustics, Vol. AU-20,
pp. 110-112, June.

S a n d b e rg ,

E. H,, and A l e x a n d e r J. T. 1978. High Resolution Spectral Analysis of Sinusoids

in Correlated Noise, Proc. 1978 IC A SSP, pp. 349-351, Tulsa, Okla., April 10-12.

S a to riu s ,

References and Bibliography

R13

W., and R a b i n e r , L. R . 1973. A Digital Signal Processing Approach to Inter

polation, Proc. IE E E , Vol. 61, pp. 692-702, June.

S c h a f e r , R.

H., and G o c k l e r , H. 1981. A Comprehensive Survey of Digital Transmul

tiplexing Methods, Proc. IE E E , Vol. 69, pp. 1419-1450.

S c h e u e rm a n n ,

R. D. 1981. A Signal Subspace Approach to Multiple Emitter Location and

Spectral Estimation, Ph.D. dissertation, Department of Electrical Engineering, Stanford
University, Stanford, Calif., November.

S c h m id t,

R. D. 1986. Multiple Emitter Location and Signal Parameter Estimation, IE E E

Trans. A ntennas and Propagation, Vol. AP 34, pp. 276-280, March.

S c h m id t,

J. P., and M c C l e l l a n , J. H. 1984. Maximum Entropy Power Spectrum Estimation

with Uncertainty in Correlation Measurements, IE E E Trans. Acoustics, Speech, and
Signal Processing, Vol. ASSP-32, pp. 410-418, April.

S c h o tt,

I. 1917. On Power Series Which Are Bounded in the Interior of the Unit Circle,
J. Reine Angew. Math., Vol. 147, pp. 205-232, Berlin. For an English translation of the
paper, see Gohberg 1986.

S ch u r,

1898. On the Investigation of Hidden Periodicities with Application

to a Supposed Twenty-Six-Day Period of Meteorological Phenomena, Terr. Mag.. Vol. 3,
pp. 13-41, March.

S c h u s te r , S ir A r t h u r .

and F e t t w e i s . A . 1973. Digital Filters with True Ladder Configuration.

Int. J. Circuit Theory A ppL, Vol. 1, pp. 5-10. March.

S e d l m e y e r , A .,

S h a n k s , J. L. 1967. Recursion Filters for Digital Processing," Geophysics, Vol. 32, p p . 3 3 5 1 , February.
Shannon,

C. E. 1949. Communication in the Presence of Noise, Proc. IRE , pp. 10-21,

January.
D. H., ed. 1986. Analog-Digital Conversion H andbook, Prentice Hall, Englewood
Cliffs, N.J.

S h e in g o ld ,

S ie b e rt,

W. M. 1986. Circuits, Signals and Systems, McGraw-Hill, New York.

R. C. 1967. A Method for Computing the Fast Fourier Transform with Auxil
iary Memory and Limit High speed Storage. IE E E Trans. A udio and Electroacoustics,
Vol. AU-15, pp. 91-98, June.

S in g le to n ,

R. C. 1969. An Algorithm for Computing the Mixed Radix Fast Fourier Trans
form, IE E E Trans. A u dio and Electroacoustics, Vol. AU-17, pp. 93-103, June.

S in g le to n ,

M. J. T., and B a r w e l l , T. P. 1984. A Procedure for Designing Exact Reconstruc

tion Filter Banks for Tree Structured Subband Coders, Proc. 1984 IE E E International
Conference on Acoustics, Speech, and Signal Processing, pp. 27.1.1-27.1.4, San Diego,
March.

S m ith ,

M. J. T., and E d d i n s , S. L. 1988. Subband Coding of Images with Octave Band Tree
Structures, Proc. 1987 IE E E International Conference on Acoustics, Speech, and Signal
Processing, pp. 1382-1385, Dallas, April.

S m ith ,

K. 1965. The Equivalence of Digital and Analog Signal Processing, Inf. Control,
Vol. 8 , pp. 455-467, October.

S te ig litz ,

K. 1970. Computer-Aided Design of Recursive Digital Filters, IE E E Trans.

A u d io and Electroacoustics, Vol. AU-18, pp. 123-129, June.

S te ig litz ,

R14

References and Bibliography

T. G. 1966. High Speed Convolution and Correlation, 1966 Spring Joint Com puter Conference, A F IP S Proc., Vol. 28, pp. 229-233.

S to c k h a m ,

S to re r.

J. E. 1957. Passive Network Synthesis, McGraw-Hill, New York.

S w a r z tr a u b e r.

P. 1986. Symmetric FFTs, Mathematics o f Computation, Vol. 47, pp. 323-

346. July.
D. N. 1979a. A Comparison Between Burgs Maximum Entropy Method and a
Nonrecursive Technique for the Spectral Analysis of Deterministic Signals, J. Geophys.
Res.. Vol. 84, pp. 679-685, February.

S w in g le r,

D. N. 1979b. A Modified Burg Algorithm for Maximum Entropy Spectral Anal

ysis, Proc. IE E E , Vol. 67, pp. 1368-1369, September.

S w in g le r,

D. N. 1980. Frequency Errors in MEM Processing, IE E E Tram. Acoustics,

Speech, and Signal Processing, Vol. ASSP-28, pp. 257-259, April.

S w in g le r,

G. 1967. Orthogonal Polynomials, 3rd ed., Colloquium Publishers, no. 23, American
Mathematical Society, Providence, R.l.

Szego,

1981. A Comparison of the Least-Squares Method and the Burg Method

for Autoregressive Spectral Analysis, IE E E Trans. A ntennas and Propagation. Vol. AP29, pp. 675-679, July.

T h o rv a ld s e n , T.

H. 1975. Autoregressive Model Fitting with Noisy Data by Akaike's Information

Criterion," IE E E Trans. Information Theory, Vol. IT-21, pp. 4 7 6 - 4 8 0 , July.

Tong,

H. 1977. More on Autoregressive Model Fitting with Noise Data b y Akaike's Infor
mation Criterion," IE E E Trans. Information Theory, Vol. IT-23, pp. 409-410. May.

Tong,

T re tte r.

S. A. 1976. Introduction to Discrete-Time Signal Processing. Wiley, New York.

D. W.. and K u m a r e s a n , R. 1982. Estimation of Frequencies of Multiple Sinusoids:

Making Linear Prediction Perform Like Maximum Likelihood, Proc. IE E E , Vol. 70,
pp. 975-989. September.

T u fts ,

T. J., and B i s h o p , T. N. 1975. Maximum Entropy Spectral Analysis and Autore

gressive Decomposition,7' Rev. Geophys. Space Phys., Vol. 13, pp. 183-200, February.

U lr y c h .

T. J and C l a y t o n , R. W. 1976. Time Series Modeling and Maximum Entropy,

Phys. Earth Planet. Inter., Vol. 12, pp. 188-200, August.

U lry c h ,

P. P. 1990. Multirate Digital Filters, Filter Banks, Polyphase Networks, and

Applications: A Tutorial, Proc. IE E E , Vol. 78, pp. 56-93, January.

V a id y a n a th a n ,

V a id y a n a th a n ,

P. P. 1993. Multirate Systems and Filter B anks, Prentice Hall, Englewood

Cliffs, N.J.
J. 1984. Multi-dimensional Sub-band Coding: Some Theory and Algorithms,
Signal Processing, Vol. 6 , pp. 97-112, April.

V e tte r li,

J. 1987. A Theory of Multirate Filter Banks, IE E E Trans. Acoustics, Speech,

and Signal Processing, Vol. ASSP-35, pp. 356-372, March.

V e tte rli,

A. C. G. 1977. Matrix Orthogonal Polynomials with Applications to Autoregressive

Modeling and Ladder Forms, Ph.D. dissertation, Department of Electrical Engineering,
Stanford University, Stanford, Calif., December.

V ie ir a ,

G. 1931. On Periodicity in Series of Related Terms, Proc. R. Soc., Ser. A.

Vol. 313, pp. 518-532.

W a lk e r ,

M., and K a i l a t h , T. 1985. Detection of Signals by Information Theoretic Criteria.

IE E E Trans. Acoustics, Speech, and Signal Processing, Vol. ASSP-32, pp. 387-392, April.

W ax,

References and Bibliography

W e i n b e r g , L. 1962.

R15

Network Analysis and Synthesis, M cG raw-H ill, N ew York.

P. D. 1967. The Use of Fast Fourier Transform for the Estimation of Power Spectra:
A Method Based on Time Averaging over Short Modified Periodograms, IE E E Trans.
A u d io and Electroacoustics, Vol. AU-15, pp. 70-73, June.
i e n e r , N. 1949. Extrapolation, Interpolation and Sm oothing o f Stationary Time Series with
Engineering Applications, Wiley, New York.

W e lc h ,

N., and P a l e y , R. E. A. C. 1934. Fourier Transforms in the Complex Domain.

American Mathematical Society, Providence, R.I.
W i n o g r a d , S. 1976. "On Computing the Direte Fourier Transform, Proc. Natl. Acad. S e t,
Vol. 73, pp. 105-106.
W i n o g r a d , S. 1978. On Computing the Discrete Fourier Transform, Math. Comp., Vol. 32,
pp. 1 7 7 -1 9 9 .
W o l d , H. 1938. A Study in the Analysis o f Stationary Time Series, reprinted by Almqvist &
Wiksell, Stockholm, 1954.
W o o d , L. C., and T r e i t e l , S. 1975. Seismic Signal Processing, Proc. IE E E , Vol. 63, pp. 649661, April.
W o o d s , J. W ., and O N e i l , S. D. 1986. Subband Coding of Images, IE E E Trans. Acoustics,
Speech, and Signal Processing, Vol. ASSP-34, pp. 1278-1288, October.
Y o u l a , D and K a z a n j i a n , N. 1978. Bauer-Type Factorization of Positive Matrices and the
Theory of Matrices Orthogonal on the Unit Circle, IE E E Trans. Circuits and Systems,
Vol. CAS-25, pp. 57-69, January.
Y ule, G. U. 1927. On a Method of Investigating Periodicities in Disturbed Series with
Special References to Wolfers Sunspot Numbers, Philos. Trans. R. Soc. London, Ser. A.
Vol. 226, pp. 267-298, July.
Z a d e h , L. A., and D e s o e r , C. A. 1963. Linear System Theory: The State-Space Approach,
McGraw-Hill, New York.
Z v e r e v , A. I. 1967. H andbook o f Filter Synthesis, Wiley, New York.
W ie n e r ,

Accumulator, 58
Akaike information criterion (AIC), 932
Algorithms, 4
Chiip-z, 482-486
FFT, 448-475
Goertzel, 4 8 0 -4 8 1
Remez, 645-647
Aliasing
frequency-domain, 22, 276-279
time-domain. 276
Alternation theorem, 643
Amplitude, 16
Analog-to-digital (A/D) converter, 5, 21,
748-762
oversampling, 756-762
Analog signals (see Signals)
Antialiasing filter, 746-747
Autocorrelation
of deterministic signals, 118-133
of random signals, 327-329, A3
Autocovariance, A4
Autoregressive-moving average (ARMA)
process, 855, 922
autocorrelation of, 856
Autoregressive (AR) process, 855, 922
autocorrelation of, 856
Averages
autocorrelation, A3
autocovariance. A4
ensemble, A3-A5
for discrete-time signals, A6-A7
expected value, A3
moments, A3
power, A3
time-averages, A8-A10
Backward predictor, 515. 860
Bandlimited signals, 281
Bandpass filter, 331. 337-338
Bandpass signal, 280, 738-742
Bandwidth, 279-282. 619
Bartlett's method (see Power spectrum
estimation)
Bessel filter, 690-691
Bibliography, R1-R15
Bilinear transformation. 676-680
Binary codes, 752

11

Blackman-Tukey method (see Power

spectrum estimation)
Burg algorithm (see Power spectrum
estimation)
Butterworth filters, 682-683
Canonic form. 114
Capon method ( s e e Power spectrum
estimation)
Carrier frequency. 739-740
Cauchy integral theorem. 160
Causal signals, 87
Causal systems, 68-69, 86-87, 208-209
Causality, implications of, 615-618
Cepstral coefficients. 854
Cepstrum, 265-266
Characteristic polynomial, 101. 547
Chebyshev filters, 683-689
Chirp signal, 484
Chirp-z transform algorithm. 482-486
Circular convolution, 415-420
Coding, 22-38
Comb filter, 345-349
Complex envelope, 740
Constant-coefficient difference equations,
95-111
solution of, 100-111
Continuous-time signals
exponentials. 19-20
sampling of, 2 1-24, 269-279
sampling theorem for, 29-31, 269-279
Convolution (linear), 75-82
circular, 415-420
properties, 82-85
sum, 75
Correlation, 1J 8133. A3
autocorrelation, 122, 325-330, A3
computation, 130-131
cross-correlation, 120, 325-327
of periodic signals. 124, 130
properties, 122124
Coupled-form oscillator, 352-354
Cross-power density spectrum, A6
Dead band, 584
Decimation, 784-787
See also Sampling rate conversion

Deconvolution. 266, 355-359, 363-365

homomorphic. 266. 365-367
Delta modulation, 758
Difference equations. 9 1 - 1 11
constant coefficient, 100-111
solution, 100-111
homogeneous solution. 100-103
particular solution. 103-104
for recursive systems. 95. I ll
total solution. 105-111
from one-sided ;-translomi. 201-202
Differentiator, 652
design of, 652-657
Digital resonator. 340
Digital sinusoidal oscillator. 353-354
Digital-to-analog (D/A) converter. 5, 22,
3 8 .7 6 3 -7 7 4
oversampling. 774
Dirichlet conditions
for Fourier series, 234
for Fourier transform. 243
Discrete Fourier transform (DFT). 399-402
computation, 4 49-473
butterfly, 460, 464, 466
decimination-in-frequency FFT
algorithm, 461464
decimination-in-time FFT algorithm,
456-461
direct, 449450
divide-and-conquer method, 450-473
in-place computations, 461
radix-2 FFT algorithms, 456-464
radix-4 FFT algorithms, 465469
shuffling of data, 461
split radix. 4 70 -47 3
via linear filtering. 479-486
definition, 401
IDFT, 401
implementation o f FFT algorithm,
473-479
properties, 4 09 -42 5
circular convolution, 415-420
circular correlation, 423
circular frequency shift, 422
circular time shift, 421-422
complex conjugate, 423
linearity, 410
multiplication, 4 15 -42 0

12

Index
Parseval's theorem, 424
periodicity, 410
symmetry, 413415
table. 415
time reversal. 421
relationship to Fourier series. 407. 409
relationship to Fourier transform, 407
relationship to -transform, 408
use in frequency analysis, 433 -44 0
use in linear filtering, 425-433
Discrete-time signals, 9, 43-55
antisymmetric (odd), 5 1
correlation, 1 IS133
definition, 9, 43
exponential, 46-47
frequency analysis of. 247-264
nonperiodic, 50
periodic. 14-19
random, 12
representation of, 44
sinusoidal, 16-18
symmetric (even), 51
unit ramp, 45
unit sample, 45
unit step, 45
Discrete-time systems, 56-71
causal, 6 8-6 9, 86-87
dynamic, 62
finite-duration impulse response. 90-91
finite memory, 62, 90-91
implementation of, 500-556
infinite-duration impulse response. 90-91
infinite memory, 62, 90-91
linear, 65
memoryless, 60
noncausal, 6 8-69
nonlinear, 67
nonrecursive, 94-95
recursive, 92-93
relaxed, 59
representation, 44
shift-invariant, 63-65
stability test for, 213-219
stability triangle, 216
stable (BIBO), 69-70, 87-90
static, 62
time-invariant, 63-65
unit sample (impulse) response, 76-82
unstable, 6 9-70, 87-90
Distortion
amplitude, 317
delay. 332
harmonic, 378
phase, 317
Down sampling, 54, 55
See also Sampling rate conversion
Dynamic range, 35, 561, 751
Eigenfunction, 307
Eigenvalue. 307, 547
Eigenvector, 547
Elliptic fitters, 689-690
Emigy
definition, 47
density spectrum, 243-246, 260-264

partial. 391
signal. 47-49
Energy density spectrum, 243-246,
260-264
computation. 897-902
Ensemble, AI
averages, A3-A3
Envelope, 740-741
complex, 740
Envelope delay, 332
Ergodic, A8
correlation-ergodic, A9-A10
mean-ergodic, A8-A9
Estimate (properties)
asymptotic bias, 904
asymptotic variance, 904
bias, 904
consistent, 904
variance, 904
See also Power spectrum estimation
Fast Fourier transform (FFT) algorithms,
448-475
application to
correlation, 4 77-479
efficient computation of DFT, 448-475
linear filtering, 477-479
implementation. 473-475
minor FFT, 473
phase FFT, 473
radix-2 algorithm. 456-464
decimation-in-frequcncy. 461-464
decimation-in-time. 456-461
radix-4 algorithm, 465-469
split-radix, 470-473
Fibonacci sequence, 201, 548-549, 553
difference equation, 201-202
state-space form, 548-549, 553
Filter
bandpass, 331, 337-338
definition, 317, 330-332
design of linear-phase FIR, 620-665
transition coefficient for, C1-C5
design of HR filters, 330-354, 666-692
all pass, 350-352
comb, 345-349
notch, 343-344
by pole-zero placement, 333-354
resonators (digital), 340-343
distortion. 317
distortionless, 332
frequency-selective, 331
highpass, 331. 333-334
ideal. 331-332
least squares inverse, 711-718
lowpass, 331, 333-334
nonideal, 332-333
passband ripple, 619
stopband ripie, 619
transition band, 619
prediction error filter, 512. 858
smoothing, 39
structures, 500-556
Wiener filter, 715, 880-890
n h e r banks, 825-831

critically sampled, 829

quadrature minor, 833-841
uniform DFT, 826-829
Filter transformations, 338-340, 692-700
analog domain. 693-698
digital domain, 698 -70 0
lowpass-lo-highpass, 338-340
Filtering
of long data sequences, 430-433
overlap-add method for, 430-432
overlap-save method for, 430-431
via DFT, 430-433
Final prediction error (FPE) criterion,
931-932
Final value theorem, 200
FIR filters
antisymmetric, 620-622
design, 620-665
comparison of methods, 662-665
differentiators, 652-657
equiripple (Chebyshev) approximation,
637-661
frequency sampling method, 630-637
Hilbert transformers, 657-662
window method, 623-630
linear phase property, 621-623
symmetric, 62(W>22
FIR filter structures, 502-519
cascade form, 504-505
direct form, 503-504
conversion to lattice form, 518-519
frequency sampling form, 506-510
lattice form, 511-561, 877
conversion to direct form, 516-517
transposed form, 5 25-526
FIR systems, 90, 94, 115
First-order hold, 768-769
Fixed-point representation, 557-560
Floating-point representation, 561-564
Flowgraphs, 521-524
Folding frequency, 28, 274-275
Forced response, 96-97
Forward predictor, 515, 857-858
Fourier series, 20, 232-240, 247-250
coefficients of, 234-235, 247-248
for continuous-time periodic signals,
232-240
for discrete-time periodic signals,
247-250
Fourier transform, 240-243, 253-256
of continuous-time aperiodic signals,
240-243
convergence of, 256-259
of discrete-time aperiodic signals,
253-256
inverse, 242
properties
convolution, 297-298
correlation, 298
differentiation, 303-304
frequency shifting, 300
linearity, 294-295
modulation, 300-302
multiplication, 302-303
Paraevals theorem, 302

Index

13
Fourier transform (continued)
symmetry, 287-294
table, 304
time-reversal, 297
time-shifting, 296
relationship to ;-transform, 264-265
of signals with poles on unit circle,
267-268
Frequency, 14-18
alias, 18, 26
content, 29
folding, 28, 274-275
fundamental range, 19
highest. 19
negative. 15
normalized, 24
positive, 15
relative, 24
Frequency analysis
continuous-time aperiodic signals,
240-243
continuous-time periodic signals,
232-240
discrete-time aperiodic signals, 253-256
discrete-time periodic signals, 247-250
dualities, 282-286
for LTI systems, 305-330
table of formulas for, 285
Frequency response. 311
computation. 321325
to exponentials, 306-314
geometric interpretation of, 321-325
magnitude of, 311
phase of. 311
relation to system function, 319-321
lo sinusoids, 311-314
Frequency transformations (see Filter
transformations)
Fundamental period, 17
Gibbs phenomenon, 259, 629
Goertzel algorithm, 480-481
Granular noise, 753
Group (envelope) delay, 332
Harmonic distortion. 378, 779-780
High-frequency signal. 280
Hilbert transform, 618
Hilbert transformer, 657-662, 739
Homomorphic
deconvolution, 365-367
system, 366
HR filters
design from analog filters, 666-692
by approximation of derivatives,
667-671
by bilinear transformation, 676-680,
692
by impulse invariance, 671-676
by maiched-z transformation, 681
least-squares design methods, 706-724
frequency domain optimization,
719-724
least squares inverse, 711
718

Pronys (least squares), 706-708

Shanks' (least squares), 709-710
Padi approximation. 701-705
pole-zero placement, 333-354
IIR filter structures, 519-556
cascade form, 526-528
direct form, 519-521
lattice-ladder, 531-539, 8 78-880
parallel fonn, 529-531
second-order modules, 527
state-space forms, 539-556
transposed forms. 521-526
Impulse response. 108-110
Initial value theorems, 172
Innovations process, 852-854
Interpolation, 30-31, 273, 784, 787-790
first-order hold, 768-771
function, 30, 763
ideal, 3 0-31. 273
linear, 38, 768-774
See also Sampling-rate conversion
Inverse filter, 355-356
Inverse Fourier transform, 242, 256
Inverse system, 355, 357
Inverse -transform, 160-172, 184-197
by contour integration, 160-161,
184-186
integral formula, 160
partial fraction expansion, 188-197
power series, 186-188
Lattice filters, 511-516, 531-539, 859.
876-880
AR structures, 877-878
ARMA structure, 878-880
MA structure, 857-859
Leakage, 434, 899
Least squares
filter design, 706-724
inverse filter, 711-718
Levinson-Durbin algorithm, 716, 865-868
generalized, 868. 893
split Levinson. 891
Limit cycle oscillations, 583-587
Linear filtering
based on DFT, 425-433
overlap-add method, 430-432
overlap-save method, 430-431
Linear interpolation, 768-774
Linear prediction. 512, 857-876
backward, 860-863
forward. 857-860
lattice filter for, 859-860
normal equations for, 864
properties of, 873-876
maximum-phase, 843-844
minimum-phase, 842-843
orthogonality. 844
whitening, 844
Linear prediction filter (see Linear
prediction)
LTI systems
moving average, 115, 117
second order, 115-116
structures. 111-118

canonic form, 114

direct form I. 111-112
direct form II, 113-114
nonrecursive, 116-118
recursive, 116-118
weighted moving average, 115
Low-frequency signal, 279
Lowpass filter, 331
Maximal ripple filters. 644
Maximum entropy method. 928
Maximum-phase system, 361-362
Mean square estimation, 882
orthogonality principle, 884-886
Minimum description length (MDL), 932
Minimum-phase system, 359-362
Minimum variance estimate, 942-946
Mixed-phase system, 361-362
Moving-average filter, 309
Moving-average (M A) process, 855, 922
autocorrelation of, 857
Moving-average signal, 115
Multichannel signal. 7
Multidimensional signal, 7 -8
Narrowband signal. 281
Natural response. 97
Natural signals. 282-283
Noise subspace, 951
Noise whitening filter, 854
Normal equations, 864
solution of, 864-873
Levinson-Durbin algorithm, 865-868
Schur algorithm. 868-872
Number representation, 556-564
fixed-point. 557-561
floating point, 5 61-564
Nyquist rate, 30
Ones complement. 558
One-sided z-transform, 197-202
Orthogonality principle, 884-885
Oscillators (sinusoidal generators)
CORDIC algorithm for, 354
coupled-form, 353-354
digital. 352
Overflow, 588-589
Overtap-add method, 430 -43 2
Overlap-save method, 430-431
Overload noise, 418
Oversampling A/D, 756-762
Oversampling D/A, 774
Paley-Wiener theorem, 616
Parseval's relations
aperiodic (energy) signals, 244, 260. 302
DFT. 424
periodic (power) signals, 236, 251
Partial energy, 363
Partial fraction expansion (tee Inverse
r-transform)
Periodogram, 9 02 -90 6
estimation of, 902 -90 6
mean value. 903

14

Index
variance, 903
Phase. 14, 741
maximum. 359-363
minimum, 359-363
mixed, 359-363
response, 311
Pisarenko method, 948-950
Poles, 172
complex conjugate, 193-194, 218-219
distinct, 178-179, 189-191. 217
location, 178-181
multiple-order, 179, 191-192
Polyphase filters, 797-800
for decimation, 800
for interpolation, 797
Power
definition, 49
signal. 50
Power density spectrum, 235-240
definition. 236
estimation of (see Power spectrum
estimation)
periodic signals, 235-240, 250-253
random signals, A5-A7
rectangular pulse train, 237-240
Power spectrum estimation
Capon (minimum variance) method.
942-945
direct method. 899
eigenanalysis algorithms, 950-959
ESPRIT. 953-955
MUSIC, 952
order selection, 955-956
Pisarenko, 948-950
experimental results, 936-942
from finite data, 902-908
indirect method, 899
leakage, 899
nonparametric methods, 908 -92 0
Bartlett, 910-911, 917
Blackman-Tukey, 913-916, 918-919
computational requirements, 919-920
performance characteristics, 916-919
Welch, 911-913, 917-918
parametric (model-based) methods,
920-942
AR model, 924
AR model order selection, 9 3 1-932
ARMA model, 924, 9 34-936
Burg method, 925-928
least-squares, 929-930
MA model, 924, 933-934
maximum entropy method. 928
model parameters, 923-924
modified Burg, 928
relation to linear prediction. 923-924
sequential least squares, 930-931
Yule-Walker. 925
use of DFT, 906-908
Prediction coefficients, 857
Prediction-error filter, 512, 858
properties of, 873-876
Principal eigenvalues, 951
Probability density function, A1-A3
Probability distribution function. B1-B2

Prony's method, 706-708

Pseudorandom sequences
Barker sequence. 148
maximal-length shift register sequences,
148-149
Quadrature components, 740
Quadrature mirror filters
for perfect reconstruction, 833-841
for subband coding, 832
Quality, 916-919
of Bartlett estimate, 917
of Blackman-Tukey estimate, 918-919
of Welch estimate, 917-918
Quantization, 21-22, 33-38, 750-753
in A/D conversion. 750-753
differential, 756
differential predictive. 757
dynamic range, 35, 561, 751
error. 37. 42, 582-598
in filler coefficients, 569-582
rounding. 35, 565-567
truncation. 35. 564-565
level, 35. 750
resolution. 35. 561
step size, 35. 5 6 1
Quantization effects
in A/D conversion. 37-38, 753-756
in computation of DFT, 486-493
direct computation. 487-489
FFT algorithms. 489-493
in filter coefficients, 569-582
fixed-point numbers, 557-560
ones complement, 558-559
sign-magnitude. 558
table of bipolar codes, 752
Iwo's complement. 559-560
floating-point numbers, 561-564
limit cycles, 583-587
dead band, 584
overflow, 5 88-589
zero-input, 584
scaling to prevent overflow, 588 -58 9
statistical characterization, 590-598
Quantizer
midrise, 750
midtread. 750
resolution. 750-752
uniform. 750
Random number generators. B I-B 6
Gaussian random variable. B4-B6
subroutine for. B6
Random processes. 327-330. A I-A 10
averages, A3-A8
autocorrelation. A3
autocovariance, A4
for discrete-time signals, A6-A7
expected value, A3
moments. A3
power. A3
cofielation-crgodic, A 9-A 10
discrete-time. A6-A7
ergodic, A8
jointly stationary, A2-A3

mean-ergodic, A8-A9
power density spectrum, A5-A6
response of linear systems, 327-330
autocorrelation, 327-329
expected value, 328
power density spectrum, 329-330
sample function, A 1
stationary, A3
wide-sense, A3
time-averages, A8-A9
Random signals (see Random processes)
Rational z-transforms, 188-196
poles, 172-174
zeros, 172-174
Recursive systems, 116-118
References, R1-R15
Reflection coefficients, 512, 536. 863-864
Resonator (see Digital resonator)
Reverse (reciprocal) polynomial. 515, 861
backward system function. 515, 861
Round-off error, 565-567, 590-598
Sample function. Al
Sample-and-hold, 748-749, 765
Sampling, 9, 21. 23, 269-279, 742-746
aliasing effects. 2 7-28, 271-279
of analog signals, 23-33, 269-279.
742-746
of bandpass signals. 742-746
of discrete-time signals. 782-845
frequency, 23
frequency domain, 394-399
interval, 23
Nyquist rate, 30
period, 23
periodic, 23
rate, 23
of sinusoidal signals, 24-28
theorem, 29-30
time-domain, 2 4-2 8, 269-279
uniform, 23
Sampling-rate conversion, 782-845
applications of, 821-845
for DFT filter banks, 825-831
for interfacing, 823
for lowpass filters, 824
for oversampling A/D and D/A.
843-844
for phase shifters. 821-822
for subband coding, 831-832
for transmultipiexing, 841-843
by arbitrary factor. 815-821
of bandpass signals, 810-815
decimation, 784-787
filter design for, 792-806
interpolation, 784. 787-790
multistage, 806-810
polyphase filters for, 797-800
by rational factor, 790-792
Sampling theorem, 29-30, 269-279
Schur algorithm, 868-872
pipelined architecture for, 872-873
split-Schiir algorithm. 892

15
Schur-Cohn stability test. 213-215
conversion to lattice coefficients.
213-214
Shanks' method, 709-710
Sigma-delta modulation. 758
Sign magnitude representation, 558
Signal flowgraphs, 521-526
Signals, 2-3
analog, g
antisymmetric, 51
aperiodic. 50
bandpass, 280. 738-742
complex envelope. 740
envelope, 741
quadrature components. 740
continuous-time. 8
deterministic. 11
digital. 11
discrete-time, 9, 43-55
electrocardiogram (ECG), 7
equivalent lowpass. 740
harmonically related. 19
multichannel. 7
multidimensional, 7
natural. 282
frequency ranges. 2K2-283
periodic, 15
random, 12, AI-A 10
correlation-ergodic. A9-A10
ergodic. A9
expected value of, A4
mean-ergodic. A9-A10
moments of. A4-A7
statistically independent. A4
strict-sense stationary'. A3
time-averages, A 8-A I0
wide-sense stationary. A3
unbiased. A8
unconelated. A4
seismic, 283
sinusoidal. 14
speech. 2-3
symmetric, 51
Signal subspace, 951
Sinusoidal generators {.tee Oscillators)
Spectrum, 230-232
analysis. 232
estimation of. 232. 896-959
line, 237
Set also Power spectrum estimation
Split-radix algorithms. 470-473
Stability of LTI systems, 208-217
of second-order systems, 215-217
Stability triangle. 216
State-space analysis, 539-566
definition of stale, 540
for difference equations. 540-542
LTI state-space model. 542
output equation. 542
relation to impulse response, 551-553

Index
solution of state-space equations,
543-544
state equations, 542
state space, 541
state-space realizations
cascade form, 555
coupled form, 556
minimal, 546
normal (diagonal) form, 555
parallel form, 555
state transition matrix, 544
state variables, 539
z-domain, 550-554
zero-input response, 544
zero-state response. 544
Steady-state response. 206-207, 314-316
Structures. 111-118
direct form I, 111-112
direct form II, 113-114
Subband coding, 831-833
Superposition principle, 65
Superposition summation, 76
System, 3, 56-59
dynamic, 62
finite memory, 62
infinite memory, 62
inverse, 356
invertible, 356
relaxed, 59
Syslem function 181-184, 319-321
of all-pole system, 183
of all-zero system, 182-183
of LTI systems, 182-183
relation to frequency response, 319-321
System identification, 355, 363-364
System modeling, 855
System responses
forced, 96-97
impulse, 108-110
natural (free), 97, 204
o f relaxed pole-zero systems, 172-184
steady-stale, 206-207
of systems with initial conditions.
204-206
transient, 107, 206-207
zero-input, 97
zero-state, 96
Toeplitz matrix, 865, 883
Time averages, A8-A10
Time-limited signals, 281
Transient response. 107, 206-207. 314-315
Transition band, 619
Transposed structures, 521-526
Truncation error, 35, 564-565
Twos complement representation, 559
Uniform distribution, 487-488, 565-568,
755
Unit circle, 265, 267
Unit sample (impulse) response. 108-110

Unit sample sequence, 45

Variability. 916
Variance, 487-lXX, 591-593
Welch method. 911-913, 917-918, 919-920
Wideband signal, 281
Wiener filters, 715, 880-890
for filtering, 8 8 1
FIR structure, 715, 881-884
IIR structure, 885-889
noncausal. 8 89-890
for prediction. 881
for smoothing. 881
Wiener-Hopf equation, 882
Wiener-Khintchine theorem. 299
Window functions, 626
Wold representation, 854
Wolfer sunspot numbers, 10
autocorrelation, 127-128
graph. 128
table, 127
Yule-Walker equations, 857
modified. 935
Yule-Walker method, 925
Zero-input linear, 98
Zero-input response, 97
Zero-order hold. 38. 765
Zero padding. 400
Zero-state linear, 98
Zeros, 172
Zoom frequency analysis, 850-851
Z - Irans forms
definition, 151-152
bilateral (two-sided), 151-152
unilateral (one-sided), 197-202
inverse, 160-172. 184-197
by contour integration, 160-161,
184-186
by partial fraction-expansion, 188-197
by power series, 186-188
properties, 161-172
convolution. 168-169
correlation, 169-170
differentiation, 166-167
initial value theorem, 172
linearity. 161-163
multiplication, 170-171
Parsevals relation, 171-172
scaling. 164-165
table of, 173
time reversal, 166
time shifting, 163-164
rational, 172-184
region of convergence (ROC), 152-160
relationship of Fourier transform,
264-265
table of, 174