Algorithms for Communications Systems

and their Applications

Algorithms for Communications Systems
and their Applications

Second Edition

Nevio Benvenuto
University of Padua
Italy

Giovanni Cherubini
IBM Research Zurich
Switzerland

Stefano Tomasin
University of Padua
Italy
This edition first published 2021
© 2021 John Wiley & Sons Ltd
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form
or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on
how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

The right of Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin to be identified as the authors of this work has
been asserted in accordance with law.

Registered Offices
John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial Office
The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products visit us at
www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in
standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no representations or
warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all
warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No
warranty may be created or extended by sales representatives, written sales materials or promotional statements for this
work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of
further information does not mean that the publisher and authors endorse the information or services the organization,
website, or product may provide or recommendations it may make. This work is sold with the understanding that the
publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable
for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites
listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the
publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to
special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data

Names: Benvenuto, Nevio, author. | Cherubini, Giovanni, 1957- author. |

Tomasin, Stefano, 1975- author.
Title: Algorithms for communications systems and their applications / Nevio
Benvenuto, University of Padua, Italy, Giovanni Cherubini,
IBM Research Zurich, Switzerland, Stefano Tomasin, University of Padua,
Italy.
Description: Second edition. | Hoboken, NJ, USA : Wiley, 2021. | Includes
bibliographical references and index.
Identifiers: LCCN 2020004346 (print) | LCCN 2020004347 (ebook) | ISBN
9781119567967 (cloth) | ISBN 9781119567974 (adobe pdf) | ISBN
9781119567981 (epub)
Subjects: LCSH: Signal processing–Mathematics. | Telecommunication
systems–Mathematics. | Algorithms.
Classification: LCC TK5102.9 .B46 2020 (print) | LCC TK5102.9 (ebook) |
DDC 621.382/2–dc23
LC record available at https://lccn.loc.gov/2020004346
LC ebook record available at https://lccn.loc.gov/2020004347

Cover Design: Wiley

Cover Image: © betibup33/Shutterstock

Set in 9.5/12.5pt STIXGeneral by SPi Global, Chennai, India

10 9 8 7 6 5 4 3 2 1
To Adriana, to Antonio, Claudia, and Mariuccia, and in memory of Alberto
 Contents

Preface xxv
Acknowledgements xxvii

1 Elements of signal theory 1

1.1 Continuous-time linear systems 1
1.2 Discrete-time linear systems 2
Discrete Fourier transform 7
The DFT operator 7
Circular and linear convolution via DFT 8
Convolution by the overlap-save method 10
IIR and FIR filters 11
1.3 Signal bandwidth 14
The sampling theorem 17
Heaviside conditions for the absence of signal distortion 17
1.4 Passband signals and systems 18
Complex representation 18
Relation between a signal and its complex representation 21
Baseband equivalent of a transformation 26
Envelope and instantaneous phase and frequency 28
1.5 Second-order analysis of random processes 29
1.5.1 Correlation 29
Properties of the autocorrelation function 30
1.5.2 Power spectral density 30
Spectral lines in the PSD 30
Cross power spectral density 31
Properties of the PSD 32
PSD through filtering 32
1.5.3 PSD of discrete-time random processes 32
Spectral lines in the PSD 33
PSD through filtering 34
Minimum-phase spectral factorization 35
1.5.4 PSD of passband processes 36
PSD of in-phase and quadrature components 36
Cyclostationary processes 38
1.6 The autocorrelation matrix 43
viii Contents

1.7 Examples of random processes 46

1.8 Matched filter 52
White noise case 53
1.9 Ergodic random processes 55
1.9.1 Mean value estimators 57
Rectangular window 58
Exponential filter 59
General window 59
1.9.2 Correlation estimators 60
Unbiased estimate 60
Biased estimate 60
1.9.3 Power spectral density estimators 61
Periodogram or instantaneous spectrum 61
Welch periodogram 62
Blackman and Tukey correlogram 63
Windowing and window closing 63
1.10 Parametric models of random processes 65
ARMA 65
MA 67
AR 67
Spectral factorization of AR models 69
Whitening filter 70
Relation between ARMA, MA, and AR models 70
1.10.1 Autocorrelation of AR processes 72
1.10.2 Spectral estimation of an AR process 74
Some useful relations 75
AR model of sinusoidal processes 77
1.11 Guide to the bibliography 78
Bibliography 78
Appendix 1.A Multirate systems 79
1.A.1 Fundamentals 79
1.A.2 Decimation 81
1.A.3 Interpolation 83
1.A.4 Decimator filter 84
1.A.5 Interpolator filter 86
1.A.6 Rate conversion 88
1.A.7 Time interpolation 90
Linear interpolation 90
Quadratic interpolation 91
1.A.8 The noble identities 91
1.A.9 The polyphase representation 92
Efficient implementations 93
Appendix 1.B Generation of a complex Gaussian noise 98
Appendix 1.C Pseudo-noise sequences 99
Maximal-length 99
CAZAC 101
Gold 102
Contents ix

2 The Wiener filter 105

2.1 The Wiener filter 105
Matrix formulation 106
Optimum filter design 107
The principle of orthogonality 109
Expression of the minimum mean-square error 110
Characterization of the cost function surface 110
The Wiener filter in the z-domain 111
2.2 Linear prediction 114
Forward linear predictor 115
Optimum predictor coefficients 115
Forward prediction error filter 116
Relation between linear prediction and AR
models 117
First- and second-order solutions 117
2.3 The least squares method 118
Data windowing 119
Matrix formulation 119
Correlation matrix 120
Determination of the optimum filter coefficients 120
2.3.1 The principle of orthogonality 121
Minimum cost function 121
The normal equation using the data matrix 122
Geometric interpretation: the projection operator 122
2.3.2 Solutions to the LS problem 123
Singular value decomposition 124
Minimum norm solution 125
2.4 The estimation problem 126
Estimation of a random variable 126
MMSE estimation 127
Extension to multiple observations 128
Linear MMSE estimation of a random variable 129
Linear MMSE estimation of a random vector 129
2.4.1 The Cramér–Rao lower bound 131
Extension to vector parameter 132
2.5 Examples of application 134
2.5.1 Identification of a linear discrete-time system 134
2.5.2 Identification of a continuous-time system 135
2.5.3 Cancellation of an interfering signal 138
2.5.4 Cancellation of a sinusoidal interferer with known
frequency 139
2.5.5 Echo cancellation in digital subscriber loops 140
2.5.6 Cancellation of a periodic interferer 141
Bibliography 142
Appendix 2.A The Levinson–Durbin algorithm 142
Lattice filters 144
The Delsarte–Genin algorithm 145
x Contents

3 Adaptive transversal filters 147

3.1 The MSE design criterion 148
3.1.1 The steepest descent or gradient algorithm 148
Stability 149
Conditions for convergence 150
Adaptation gain 151
Transient behaviour of the MSE 152
3.1.2 The least mean square algorithm 153
Implementation 154
Computational complexity 155
Conditions for convergence 155
3.1.3 Convergence analysis of the LMS algorithm 156
Convergence of the mean 157
Convergence in the mean-square sense: real scalar case 157
Convergence in the mean-square sense: general case 159
Fundamental results 161
Observations 162
Final remarks 163
3.1.4 Other versions of the LMS algorithm 163
Leaky LMS 164
Sign algorithm 164
Normalized LMS 164
Variable adaptation gain 165
3.1.5 Example of application: the predictor 166
3.2 The recursive least squares algorithm 171
Normal equation 172
Derivation 173
Initialization 174
Recursive form of the minimum cost function 175
Convergence 176
Computational complexity 176
Example of application: the predictor 177
3.3 Fast recursive algorithms 177
3.3.1 Comparison of the various algorithms 177
3.4 Examples of application 178
3.4.1 Identification of a linear discrete-time system 178
Finite alphabet case 179
3.4.2 Cancellation of a sinusoidal interferer with known frequency 181
Bibliography 181

4 Transmission channels 183

4.1 Radio channel 183
4.1.1 Propagation and used frequencies in radio transmission 183
Basic propagation mechanisms 184
Frequency ranges 184
4.1.2 Analog front-end architectures 185
Radiation masks 185
Conventional superheterodyne receiver 186
Contents xi

Alternative architectures 187

Direct conversion receiver 187
Single conversion to low-IF 188
Double conversion and wideband IF 188
4.1.3 General channel model 189
High power amplifier 189
Transmission medium 191
Additive noise 191
Phase noise 191
4.1.4 Narrowband radio channel model 193
Equivalent circuit at the receiver 195
Multipath 196
Path loss as a function of distance 197
4.1.5 Fading effects in propagation models 200
Macroscopic fading or shadowing 200
Microscopic fading 201
4.1.6 Doppler shift 202
4.1.7 Wideband channel model 204
Multipath channel parameters 205
Statistical description of fading channels 206
4.1.8 Channel statistics 208
Power delay profile 208
Coherence bandwidth 209
Doppler spectrum 210
Coherence time 211
Doppler spectrum models 211
Power angular spectrum 211
Coherence distance 212
On fading 212
4.1.9 Discrete-time model for fading channels 213
Generation of a process with a pre-assigned spectrum 214
4.1.10 Discrete-space model of shadowing 216
4.1.11 Multiantenna systems 218
Line of sight 218
Discrete-time model 219
Small number of scatterers 220
Large number of scatterers 220
Blockage effect 222
4.2 Telephone channel 222
4.2.1 Distortion 222
4.2.2 Noise sources 222
Quantization noise: 222
Thermal noise: 224
4.2.3 Echo 224
Bibliography 225
Appendix 4.A Discrete-time NB model for mmWave channels 226
4.A.1 Angular domain representation 226
xii Contents

5 Vector quantization 229

5.1 Basic concept 229
5.2 Characterization of VQ 230
Parameters determining VQ performance 231
Comparison between VQ and scalar quantization 232
5.3 Optimum quantization 233
Generalized Lloyd algorithm 233
5.4 The Linde, Buzo, and Gray algorithm 235
Choice of the initial codebook 236
Splitting procedure 236
Selection of the training sequence 238
5.4.1 k-means clustering 239
5.5 Variants of VQ 239
Tree search VQ 239
Multistage VQ 240
Product code VQ 240
5.6 VQ of channel state information 242
MISO channel quantization 242
Channel feedback with feedforward information 244
5.7 Principal component analysis 244
5.7.1 PCA and k-means clustering 246
Bibliography 248

6 Digital transmission model and channel capacity 249

6.1 Digital transmission model 249
6.2 Detection 253
6.2.1 Optimum detection 253
ML 254
MAP 254
6.2.2 Soft detection 256
LLRs associated to bits of BMAP 256
Simplified expressions 258
6.2.3 Receiver strategies 260
6.3 Relevant parameters of the digital transmission model 260
Relations among parameters 261
6.4 Error probability 262
6.5 Capacity 265
6.5.1 Discrete-time AWGN channel 266
6.5.2 SISO narrowband AWGN channel 266
Channel gain 267
6.5.3 SISO dispersive AGN channel 267
6.5.4 MIMO discrete-time NB AWGN channel 269
Continuous-time model 270
MIMO dispersive channel 270
6.6 Achievable rates of modulations in AWGN channels 270
6.6.1 Rate as a function of the SNR per dimension 271
6.6.2 Coding strategies depending on the signal-to-noise ratio 272
Coding gain 274
Contents xiii

6.6.3 Achievable rate of an AWGN channel using PAM 275

Bibliography 276
Appendix 6.A Gray labelling 277
Appendix 6.B The Gaussian distribution and Marcum functions 278
6.B.1 The Q function 278
6.B.2 Marcum function 279

7 Single-carrier modulation 281

7.1 Signals and systems 281
7.1.1 Baseband digital transmission (PAM) 281
Modulator 281
Transmission channel 283
Receiver 283
Power spectral density 284
7.1.2 Passband digital transmission (QAM) 285
Modulator 285
Power spectral density 286
Three equivalent representations of the modulator 287
Coherent receiver 288
7.1.3 Baseband equivalent model of a QAM system 288
Signal analysis 288
7.1.4 Characterization of system elements 291
Transmitter 291
Transmission channel 291
Receiver 293
7.2 Intersymbol interference 294
Discrete-time equivalent system 294
Nyquist pulses 295
Eye diagram 298
7.3 Performance analysis 302
Signal-to-noise ratio 302
Symbol error probability in the absence of ISI 303
Matched filter receiver 303
7.4 Channel equalization 304
7.4.1 Zero-forcing equalizer 304
7.4.2 Linear equalizer 305
Optimum receiver in the presence of noise and ISI 305
Alternative derivation of the IIR equalizer 306
Signal-to-noise ratio at detector 310
7.4.3 LE with a finite number of coefficients 310
Adaptive LE 311
Fractionally spaced equalizer 313
7.4.4 Decision feedback equalizer 315
Design of a DFE with a finite number of coefficients 318
Design of a fractionally spaced DFE 320
Signal-to-noise ratio at the decision point 322
Remarks 322
7.4.5 Frequency domain equalization 323
xiv Contents

DFE with data frame using a unique word 323

7.4.6 LE-ZF 326
7.4.7 DFE-ZF with IIR filters 327
DFE-ZF as noise predictor 331
DFE as ISI and noise predictor 331
7.4.8 Benchmark performance of LE-ZF and DFE-ZF 333
Comparison 333
Performance for two channel models 334
7.4.9 Passband equalizers 335
Passband receiver structure 335
Optimization of equalizer coefficients and carrier phase offset 337
Adaptive method 338
7.5 Optimum methods for data detection 340
Maximum a posteriori probability (MAP) criterion 341
7.5.1 Maximum-likelihood sequence detection 341
Lower bound to error probability using MLSD 342
The Viterbi algorithm 343
Computational complexity of the VA 346
7.5.2 Maximum a posteriori probability detector 347
Statistical description of a sequential machine 347
The forward–backward algorithm 348
Scaling 351
The log likelihood function and the Max-Log-MAP criterion 352
LLRs associated to bits of BMAP 353
Relation between Max-Log–MAP and Log–MAP 354
7.5.3 Optimum receivers 354
7.5.4 The Ungerboeck’s formulation of MLSD 356
7.5.5 Error probability achieved by MLSD 358
Computation of the minimum distance 361
7.5.6 The reduced-state sequence detection 365
Trellis diagram 365
The RSSE algorithm 367
Further simplification: DFSE 369
7.6 Numerical results obtained by simulations 370
QPSK over a minimum-phase channel 370
QPSK over a non-minimum phase channel 370
8-PSK over a minimum phase channel 372
8-PSK over a non-minimum phase channel 372
7.7 Precoding for dispersive channels 373
7.7.1 Tomlinson–Harashima precoding 374
7.7.2 Flexible precoding 376
7.8 Channel estimation 378
7.8.1 The correlation method 378
7.8.2 The LS method 379
Formulation using the data matrix 380
7.8.3 Signal-to-estimation error ratio 380
Computation of the signal-to-estimation error ratio 381
On the selection of the channel length 384
Contents xv

7.8.4 Channel estimation for multirate systems 384

7.8.5 The LMMSE method 385
7.9 Faster-than-Nyquist Signalling 386
Bibliography 387
Appendix 7.A Simulation of a QAM system 389
Appendix 7.B Description of a finite-state machine 393
Appendix 7.C Line codes for PAM systems 394
7.C.1 Line codes 394
Non-return-to-zero format 395
Return-to-zero format 396
Biphase format 397
Delay modulation or Miller code 398
Block line codes 398
Alternate mark inversion 398
7.C.2 Partial response systems 399
The choice of the PR polynomial 401
Symbol detection and error probability 404
Precoding 406
Error probability with precoding 407
Alternative interpretation of PR systems 408
7.D Implementation of a QAM transmitter 410

8 Multicarrier modulation 413

8.1 MC systems 413
8.2 Orthogonality conditions 414
Time domain 415
Frequency domain 415
z-Transform domain 415
8.3 Efficient implementation of MC systems 416
MC implementation employing matched filters 416
Orthogonality conditions in terms of the polyphase components 418
MC implementation employing a prototype filter 419
8.4 Non-critically sampled filter banks 422
8.5 Examples of MC systems 426
OFDM or DMT 426
Filtered multitone 427
8.6 Analog signal processing requirements in MC systems 429
8.6.1 Analog filter requirements 429
Interpolator filter and virtual subchannels 429
Modulator filter 430
8.6.2 Power amplifier requirements 431
8.7 Equalization 432
8.7.1 OFDM equalization 432
8.7.2 FMT equalization 434
Per-subchannel fractionally spaced equalization 434
Per-subchannel T-spaced equalization 435
Alternative per-subchannel T-spaced equalization 436
8.8 Orthogonal time frequency space modulation 437
xvi Contents

OTFS equalization 437

8.9 Channel estimation in OFDM 437
Instantaneous estimate or LS method 438
LMMSE 440
The LS estimate with truncated impulse response 440
8.9.1 Channel estimate and pilot symbols 441
8.10 Multiuser access schemes 442
8.10.1 OFDMA 442
8.10.2 SC-FDMA or DFT-spread OFDM 443
8.11 Comparison between MC and SC systems 444
8.12 Other MC waveforms 445
Bibliography 446

9 Transmission over multiple input multiple output channels 447

9.1 The MIMO NB channel 447
Spatial multiplexing and spatial diversity 451
Interference in MIMO channels 452
9.2 CSI only at the receiver 452
9.2.1 SIMO combiner 452
Equalization and diversity 455
9.2.2 MIMO combiner 455
Zero-forcing 456
MMSE 456
9.2.3 MIMO non-linear detection and decoding 457
V-BLAST system 457
Spatial modulation 458
9.2.4 Space-time coding 459
The Alamouti code 459
The Golden code 461
9.2.5 MIMO channel estimation 461
The least squares method 462
The LMMSE method 463
9.3 CSI only at the transmitter 463
9.3.1 MISO linear precoding 463
MISO antenna selection 464
9.3.2 MIMO linear precoding 465
ZF precoding 465
9.3.3 MIMO non-linear precoding 466
Dirty paper coding 467
TH precoding 468
9.3.4 Channel estimation for CSIT 469
9.4 CSI at both the transmitter and the receiver 469
9.5 Hybrid beamforming 470
Hybrid beamforming and angular domain representation 472
9.6 Multiuser MIMO: broadcast channel 472
CSI only at the receivers 473
CSI only at the transmitter 473
9.6.1 CSI at both the transmitter and the receivers 473
Block diagonalization 473
Contents xvii

User selection 474

Joint spatial division and multiplexing 475
9.6.2 Broadcast channel estimation 476
9.7 Multiuser MIMO: multiple-access channel 476
CSI only at the transmitters 477
CSI only at the receiver 477
9.7.1 CSI at both the transmitters and the receiver 477
Block diagonalization 477
9.7.2 Multiple-access channel estimation 478
9.8 Massive MIMO 478
9.8.1 Channel hardening 478
9.8.2 Multiuser channel orthogonality 479
Bibliography 479

10 Spread-spectrum systems 483

10.1 Spread-spectrum techniques 483
10.1.1 Direct sequence systems 483
Classification of CDMA systems 490
Synchronization 490
10.1.2 Frequency hopping systems 491
Classification of FH systems 491
10.2 Applications of spread-spectrum systems 493
10.2.1 Anti-jamming 494
10.2.2 Multiple access 496
10.2.3 Interference rejection 496
10.3 Chip matched filter and rake receiver 496
Number of resolvable rays in a multipath channel 497
Chip matched filter 498
10.4 Interference 500
Detection strategies for multiple-access systems 502
10.5 Single-user detection 502
Chip equalizer 502
Symbol equalizer 503
10.6 Multiuser detection 504
10.6.1 Block equalizer 504
10.6.2 Interference cancellation detector 506
Successive interference cancellation 506
Parallel interference cancellation 507
10.6.3 ML multiuser detector 508
Correlation matrix 508
Whitening filter 508
10.7 Multicarrier CDMA systems 509
Bibliography 510
Appendix 10.A Walsh Codes 511

11 Channel codes 515

11.1 System model 516
11.2 Block codes 517
11.2.1 Theory of binary codes with group structure 518
xviii Contents

Properties 518
Parity check matrix 520
Code generator matrix 522
Decoding of binary parity check codes 523
Cosets 523
Two conceptually simple decoding methods 524
Syndrome decoding 525
11.2.2 Fundamentals of algebra 527
modulo-q arithmetic 528
Polynomials with coefficients from a field 530
Modular arithmetic for polynomials 531
Devices to sum and multiply elements in a finite field 534
Remarks on finite fields 535
Roots of a polynomial 538
Minimum function 541
Methods to determine the minimum function 542
Properties of the minimum function 544
11.2.3 Cyclic codes 545
The algebra of cyclic codes 545
Properties of cyclic codes 546
Encoding by a shift register of length r 551
Encoding by a shift register of length k 552
Hard decoding of cyclic codes 552
Hamming codes 554
Burst error detection 556
11.2.4 Simplex cyclic codes 556
Property 557
Relation to PN sequences 558
11.2.5 BCH codes 558
An alternative method to specify the code polynomials 558
Bose-Chaudhuri–Hocquenhem codes 560
Binary BCH codes 562
Reed–Solomon codes 564
Decoding of BCH codes 566
Efficient decoding of BCH codes 568
11.2.6 Performance of block codes 575
11.3 Convolutional codes 576
11.3.1 General description of convolutional codes 579
Parity check matrix 581
Generator matrix 581
Transfer function 582
Catastrophic error propagation 585
11.3.2 Decoding of convolutional codes 586
Interleaving 587
Two decoding models 587
Decoding by the Viterbi algorithm 588
Decoding by the forward-backward algorithm 589
Sequential decoding 590
11.3.3 Performance of convolutional codes 592
Contents xix

11.4 Puncturing 593

11.5 Concatenated codes 593
The soft-output Viterbi algorithm 593
11.6 Turbo codes 597
Encoding 597
The basic principle of iterative decoding 600
FBA revisited 601
Iterative decoding 608
Performance evaluation 610
11.7 Iterative detection and decoding 611
11.8 Low-density parity check codes 614
11.8.1 Representation of LDPC codes 614
Matrix representation 614
Graphical representation 615
11.8.2 Encoding 616
Encoding procedure 616
11.8.3 Decoding 617
Hard decision decoder 617
The sum-product algorithm decoder 619
The LR-SPA decoder 622
The LLR-SPA or log-domain SPA decoder 623
The min-sum decoder 625
Other decoding algorithms 625
11.8.4 Example of application 625
Performance and coding gain 625
11.8.5 Comparison with turbo codes 627
11.9 Polar codes 627
11.9.1 Encoding 628
Internal CRC 630
LLRs associated to code bits 631
11.9.2 Tanner graph 631
11.9.3 Decoding algorithms 633
Successive cancellation decoding – the principle 634
Successive cancellation decoding – the algorithm 635
Successive cancellation list decoding 638
Other decoding algorithms 639
11.9.4 Frozen set design 640
Genie-aided SC decoding 640
Design based on density evolution 641
Channel polarization 643
11.9.5 Puncturing and shortening 644
Puncturing 644
Shortening 645
Frozen set design 647
11.9.6 Performance 647
11.10 Milestones in channel coding 648
Bibliography 649
Appendix 11.A Non-binary parity check codes 652
Linear codes 653
xx Contents

Parity check matrix 654

Code generator matrix 655
Decoding of non-binary parity check codes 656
Coset 656
Two conceptually simple decoding methods 656
Syndrome decoding 657

12 Trellis coded modulation 659

12.1 Linear TCM for one- and two-dimensional signal sets 660
12.1.1 Fundamental elements 660
Basic TCM scheme 661
Example 662
12.1.2 Set partitioning 664
12.1.3 Lattices 666
12.1.4 Assignment of symbols to the transitions in the trellis 671
12.1.5 General structure of the encoder/bit-mapper 675
Computation of dfree 677
12.2 Multidimensional TCM 679
Encoding 680
Decoding 682
12.3 Rotationally invariant TCM schemes 684
Bibliography 685

13 Techniques to achieve capacity 687

13.1 Capacity achieving solutions for multicarrier systems 687
13.1.1 Achievable bit rate of OFDM 687
13.1.2 Waterfilling solution 688
Iterative solution 689
13.1.3 Achievable rate under practical constraints 689
Effective SNR and system margin in MC systems 690
Uniform power allocation and minimum rate per subchannel 690
13.1.4 The bit and power loading problem revisited 691
Problem formulation 692
Some simplifying assumptions 692
On loading algorithms 693
The Hughes-Hartogs algorithm 694
The Krongold–Ramchandran–Jones algorithm 694
The Chow–Cioffi–Bingham algorithm 696
Comparison 698
13.2 Capacity achieving solutions for single carrier systems 698
Achieving capacity 702
Bibliography 703

14 Synchronization 705
14.1 The problem of synchronization for QAM systems 705
14.2 The phase-locked loop 707
14.2.1 PLL baseband model 708
Linear approximation 709
Contents xxi

14.2.2 Analysis of the PLL in the presence of additive noise 711

Noise analysis using the linearity assumption 711
14.2.3 Analysis of a second-order PLL 713
14.3 Costas loop 716
14.3.1 PAM signals 716
14.3.2 QAM signals 719
14.4 The optimum receiver 720
Timing recovery 721
Carrier phase recovery 725
14.5 Algorithms for timing and carrier phase recovery 725
14.5.1 ML criterion 726
Assumption of slow time varying channel 726
14.5.2 Taxonomy of algorithms using the ML criterion 726
Feedback estimators 727
Early-late estimators 728
14.5.3 Timing estimators 729
Non-data aided 729
NDA synchronization via spectral estimation 732
Data aided and data directed 733
Data and phase directed with feedback: differentiator scheme 735
Data and phase directed with feedback: Mueller and Muller scheme 735
Non-data aided with feedback 738
14.5.4 Phasor estimators 738
Data and timing directed 738
Non-data aided for M-PSK signals 738
Data and timing directed with feedback 739
14.6 Algorithms for carrier frequency recovery 740
14.6.1 Frequency offset estimators 741
Non-data aided 741
Non-data aided and timing independent with feedback 742
Non-data aided and timing directed with feedback 743
14.6.2 Estimators operating at the modulation rate 743
Data aided and data directed 744
Non-data aided for M-PSK 744
14.7 Second-order digital PLL 744
14.8 Synchronization in spread-spectrum systems 745
14.8.1 The transmission system 745
Transmitter 745
Optimum receiver 745
14.8.2 Timing estimators with feedback 746
Non-data aided: non-coherent DLL 747
Non-data aided modified code tracking loop 747
Data and phase directed: coherent DLL 747
14.9 Synchronization in OFDM 751
14.9.1 Frame synchronization 751
Effects of STO 751
Schmidl and Cox algorithm 752
14.9.2 Carrier frequency synchronization 754
xxii Contents

Estimator performance 755

Other synchronization solutions 755
14.10 Synchronization in SC-FDMA 756
Bibliography 756

15 Self-training equalization 759

15.1 Problem definition and fundamentals 759
Minimization of a special function 762
15.2 Three algorithms for PAM systems 765
The Sato algorithm 765
Benveniste–Goursat algorithm 766
Stop-and-go algorithm 766
Remarks 767
15.3 The contour algorithm for PAM systems 767
Simplified realization of the contour algorithm 769
15.4 Self-training equalization for partial response systems 770
The Sato algorithm 770
The contour algorithm 772
15.5 Self-training equalization for QAM systems 773
The Sato algorithm 773
15.5.1 Constant-modulus algorithm 775
The contour algorithm 776
Joint contour algorithm and carrier phase tracking 777
15.6 Examples of applications 779
Bibliography 783
Appendix 15.A On the convergence of the contour algorithm 784

16 Low-complexity demodulators 787

16.1 Phase-shift keying 787
16.1.1 Differential PSK 787
Error probability of M-DPSK 789
16.1.2 Differential encoding and coherent demodulation 791
Differentially encoded BPSK 791
Multilevel case 791
16.2 (D)PSK non-coherent receivers 793
16.2.1 Baseband differential detector 793
16.2.2 IF-band (1 bit) differential detector 794
Signal at detection point 796
16.2.3 FM discriminator with integrate and dump filter 797
16.3 Optimum receivers for signals with random phase 798
ML criterion 799
Implementation of a non-coherent ML receiver 800
Error probability for a non-coherent binary FSK system 804
Performance comparison of binary systems 806
16.4 Frequency-based modulations 807
16.4.1 Frequency shift keying 807
Coherent demodulator 808
Non-coherent demodulator 808
Contents xxiii

Limiter–discriminator FM demodulator 809

16.4.2 Minimum-shift keying 810
Power spectral density of CPFSK 812
Performance 814
MSK with differential precoding 815
16.4.3 Remarks on spectral containment 816
16.5 Gaussian MSK 816
16.5.1 Implementation of a GMSK scheme 819
Configuration I 821
Configuration II 821
Configuration III 822
16.5.2 Linear approximation of a GMSK signal 824
Performance of GMSK 824
Performance in the presence of multipath 829
Bibliography 830
Appendix 16.A Continuous phase modulation 831
Alternative definition of CPM 831
Advantages of CPM 832

17 Applications of interference cancellation 833

17.1 Echo and near-end crosstalk cancellation for PAM systems 834
Crosstalk cancellation and full-duplex transmission 835
Polyphase structure of the canceller 836
Canceller at symbol rate 836
Adaptive canceller 837
Canceller structure with distributed arithmetic 838
17.2 Echo cancellation for QAM systems 842
17.3 Echo cancellation for OFDM systems 844
17.4 Multiuser detection for VDSL 846
17.4.1 Upstream power back-off 850
17.4.2 Comparison of PBO methods 851
Bibliography 855

18 Examples of communication systems 857

18.1 The 5G cellular system 857
18.1.1 Cells in a wireless system 857
18.1.2 The release 15 of the 3GPP standard 858
18.1.3 Radio access network 859
Time-frequency plan 859
NR data transmission chain 861
OFDM numerology 861
Channel estimation 862
18.1.4 Downlink 862
Synchronization 863
Initial access or beam sweeping 864
Channel estimation 865
Channel state information reporting 865
18.1.5 Uplink 865
xxiv Contents

Transform precoding numerology 866

Channel estimation 866
Synchronization 866
Timing advance 867
18.1.6 Network slicing 867
18.2 GSM 868
Radio subsystem 870
18.3 Wireless local area networks 872
Medium access control protocols 872
18.4 DECT 873
18.5 Bluetooth 875
18.6 Transmission over unshielded twisted pairs 875
18.6.1 Transmission over UTP in the customer service area 876
18.6.2 High-speed transmission over UTP in local area networks 880
18.7 Hybrid fibre/coaxial cable networks 881
Ranging and power adjustment in OFDMA systems 885
Ranging and power adjustment for uplink transmission 886
Bibliography 889
Appendix 18.A Duplexing 890
Three methods 890
Appendix 18.B Deterministic access methods 890

19 High-speed communications over twisted-pair cables 893

19.1 Quaternary partial response class-IV system 893
Analog filter design 893
Received signal and adaptive gain control 894
Near-end crosstalk cancellation 895
Decorrelation filter 895
Adaptive equalizer 895
Compensation of the timing phase drift 896
Adaptive equalizer coefficient adaptation 896
Convergence behaviour of the various algorithms 897
19.1.1 VLSI implementation 897
Adaptive digital NEXT canceller 897
Adaptive digital equalizer 900
Timing control 904
Viterbi detector 906
19.2 Dual-duplex system 906
Dual-duplex transmission 906
Physical layer control 908
Coding and decoding 909
19.2.1 Signal processing functions 912
The 100BASE-T2 transmitter 912
The 100BASE-T2 receiver 913
Computational complexity of digital receive filters 914
Bibliography 915
Appendix 19.A Interference suppression 915

Index 917
Preface

The motivation for writing this book is twofold. On the one hand, we provide a teaching tool for advanced
courses in communications systems. On the other hand, we present a collection of fundamental algo-
rithms and structures useful as an in-depth reference for researchers and engineers. The contents reflect
our experience in teaching university courses on algorithms for telecommunications, as well as our
professional experience acquired in industrial research laboratories.
The text illustrates the steps required for solving problems posed by the design of systems for reli-
able communications over wired or wireless channels. In particular, we have focused on fundamental
developments in the field in order to provide the reader with the necessary insight to design practical
systems.
The second edition of this book has been enriched by new solutions in fields of application and stan-
dards that have emerged since the first edition of 2002. To name one, the adoption of multiple antennas
in wireless communication systems has received a tremendous impulse in recent years, and an entire
chapter is now dedicated to this topic. About error correction, polar codes have been invented and are
considered for future standards. Therefore, they also have been included in this new book edition. On
the standards side, cellular networks have evolved significantly, thus we decided to dedicate a large part
of a chapter to the new fifth-generation (5G) of cellular networks, which is being finalized at the time of
writing. Moreover, a number of transmission techniques that have been designed and studied for appli-
cation to 5G systems, with special regard to multi-carrier transmission, have been treated in this book.
Lastly, many parts have been extensively integrated with new material, rewritten, and improved, with
the purpose of illustrating to the reader their connection with current research trends, such as advances
in machine learning.
Acknowledgements

We gratefully acknowledge all who have made the realization of this book possible. In particular, the
editing of the various chapters would never have been completed without the contributions of numerous
students in our courses on Algorithms for Telecommunications. Although space limitations preclude
mentioning them all by name, we nevertheless express our sincere gratitude. We also thank Christian
Bolis and Chiara Paci for their support in developing the software for the book, Charlotte Bolliger and
Lilli M. Pavka for their assistance in administering the project, and Urs Bitterli and Darja Kropaci for
their help with the graphics editing. For text processing, also for the Italian version, the contribution of
Barbara Sicoli and Edoardo Casarin was indispensable; our thanks also go to Jane Frankenfield Zanin
for her help in translating the text into English. We are pleased to thank the following colleagues for
their invaluable assistance throughout the revision of the book: Antonio Assalini, Leonardo Bazzaco,
Paola Bisaglia, Matthieu Bloch, Alberto Bononi, Alessandro Brighente, Giancarlo Calvagno, Giulio
Colavolpe, Roberto Corvaja, Elena Costa, Daniele Forner, Andrea Galtarossa, Antonio Mian, Carlo
Monti, Ezio Obetti, Riccardo Rahely, Roberto Rinaldo, Antonio Salloum, Fortunato Santucci, Andrea
Scaggiante, Giovanna Sostrato, and Luciano Tomba. We gratefully acknowledge our colleague and
mentor Jack Wolf for letting us include his lecture notes in the chapter on channel codes. We also
acknowledge the important contribution of Ingmar Land on writing the section on polar codes. An
acknowledgement goes also to our colleagues Werner Bux and Evangelos Eleftheriou of the IBM
Zurich Research Laboratory, and Silvano Pupolin of the University of Padova, for their continuing
support. Finally, special thanks go to Hideki Ochiai of Yokohama National University and Jinhong
Yuan of University of New South Wales for hosting Nevio Benvenuto in the Fall 2018 and Spring 2019,
respectively: both colleagues provided an ideal setting for developing the new book edition.
xxviii Acknowledgements

To make the reading of the adopted symbols easier, the Greek alphabet is reported below.

The Greek alphabet

𝛼 A alpha 𝜈 N nu
𝛽 B beta 𝜉 Ξ xi
𝛾 Γ gamma o O omicron
𝛿 Δ delta 𝜋 Π pi
𝜖, 𝜀 E epsilon 𝜌, 𝜚 P rho
∑
𝜁 Z zeta 𝜎, 𝜍 sigma
𝜂 H eta 𝜏 T tau
𝜃, 𝜗 Θ theta 𝜐 Y upsilon
𝜄 I iota 𝜙, 𝜑 Φ phi
𝜅 K kappa 𝜒 X chi
𝜆 Λ lambda 𝜓 Ψ psi
𝜇 M mu 𝜔 Ω omega
Chapter 1

Elements of signal theory

In this chapter, we recall some concepts on signal theory and random processes. For an in-depth study,
we recommend the companion book [1]. First, we introduce various forms of the Fourier transform.
Next, we provide the complex representation of passband signals and their baseband equivalent. We will
conclude with the study of random processes, with emphasis on the statistical estimation of first- and
second-order ergodic processes, i.e. periodogram, correlogram, auto-regressive (AR), moving-average
(MA), and auto-regressive moving average (ARMA) models.

1.1 Continuous-time linear systems

A time-invariant continuous-time continuous-amplitude linear system, also called analog filter, is rep-
resented in Figure 1.1, where x and y are the input and output signals, respectively, and h denotes the
filter impulse response.

x(t) y(t)
h

Figure 1.1 Analog filter as a time-invariant linear system with continuous domain.

The output at a certain instant t ∈ ℝ, where ℝ denotes the set of real numbers, is given by the convo-
lution integral
∞ ∞
y(t) = h(t − 𝜏) x(𝜏) d𝜏 = h(𝜏) x(t − 𝜏) d𝜏 (1.1)
∫−∞ ∫−∞
denoted in short
y(t) = x ∗ h(t) (1.2)
We also introduce the Fourier transform of the signal x(t), t ∈ ℝ,
+∞
( f ) =  [x(t)] = x(t) e−j2𝜋 ft dt f ∈ℝ (1.3)
∫−∞
√
where j = −1. The inverse Fourier transform is given by
∞
x(t) = ( f ) e j2𝜋ft df (1.4)
∫−∞
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
2 Chapter 1. Elements of signal theory

In the frequency domain, (1.2) becomes

(f ) = (f ) (f ), f ∈ℝ (1.5)
where  is the filter frequency response. The magnitude of the frequency response, |(f )|, is usually
called magnitude response or amplitude response.
General properties of the Fourier transform are given in Table 1.1,1 where we use two important
functions {
1 t>0
step function: 1(t) = (1.6)
0 t<0
{
1 t>0
sign function: sgn(t) = (1.7)
−1 t < 0
Moreover, we denote by 𝛿(t) the Dirac impulse or delta function,
d1(t)
𝛿(t) = (1.8)
dt
where the derivative is taken in the generalized sense.

Definition 1.1
We introduce two functions that will be extensively used:
{
1
1 |f| <
rect(f ) = 2 (1.9)
0 elsewhere
sin(𝜋t)
sinc(t) = (1.10)
𝜋t
The following relation holds ( )
1 f
 [sinc(Ft)] = rect (1.11)
F F
as illustrated in Figure 1.2. ◽

Further examples of signals and relative Fourier transforms are given in Table 1.2.
We reserve the notation H(s) to indicate the Laplace transform of h(t), t ∈ ℝ:
+∞
H(s) = h(t)e−st dt (1.12)
∫−∞
with s complex variable; H(s) is also called the transfer function of the filter. A class of functions H(s)
often used in practice is characterized by the ratio of two polynomials in s, each with a finite number of
coefficients.
It is easy to observe that if the curve s = j2𝜋f in the s-plane belongs to the convergence region of the
integral in (1.12), then (f ) is related to H(s) by
(f ) = H(s)|s=j2𝜋f (1.13)

1.2 Discrete-time linear systems

A discrete-time time-invariant linear system, with sampling period Tc , is shown in Figure 1.3, where x(k)
and y(k) are, respectively, the input and output signals at the time instant kTc , k ∈ ℤ, where ℤ denotes

1
x∗ denotes the complex conjugate of x, while Re(x) and Im(x) denote, respectively, the real and imaginary part of x.
1.2. Discrete-time linear systems 3

Table 1.1: Some general properties of the Fourier transform.

property signal Fourier transform

x(t) (f )
linearity a x(t) + b y(t) a (f ) + b (f )
duality (t) x(−f )
time inverse x(−t) (−f )
complex conjugate x∗ (t)  ∗ (−f )
x(t) + x∗ (t) 1
real part Re[x(t)] = [(f ) +  ∗ (−f )]
2 2
x(t) − x∗ (t) 1
imaginary part Im[x(t)] = [(f ) −  ∗ (−f )]
2j 2j ( )
1 f
time scaling x(at), a ≠ 0 
|a| a
time shift x(t − t0 ) e−j2𝜋f t0 (f )
frequency shift x(t) ej2𝜋f0 t (f − f0 )
1 j𝜑
modulation x(t) cos(2𝜋f0 t + 𝜑) [e (f − f0 ) + e−j𝜑 (f + f0 )]
2
1 j𝜑
x(t) sin(2𝜋f0 t + 𝜑) [e (f − f0 ) − e−j𝜑 (f + f0 )]
2j
1 j𝜑
Re[x(t) ej(2𝜋f0 t+𝜑) ] [e (f − f0 ) + e−j𝜑  ∗ (−f − f0 )]
2
d
differentiation x(t) j2𝜋f (f )
dt
t
1 (0)
integration x(𝜏) d𝜏 = 1 ∗ x(t) (f ) + 𝛿(f )
∫−∞ j2𝜋f 2
convolution x ∗ y(t) (f ) (f )
correlation [x(𝜏) ∗ y∗ (−𝜏)](t) (f )  ∗ (f )
product x(t) y(t)  ∗ (f )
real signal x(t) = x (t) ∗
(f ) =  (−f ),  Hermitian,
∗

Re[(f )] even, Im[(f )] odd,

|(f )|2 even
imaginary signal x(t) = −x∗ (t) (f ) = − ∗ (−f )
real and even signal x(t) = x∗ (t) = x(−t) (f ) =  ∗ (f ) = (−f ),
 real and even
real and odd signal ∗
x(t) = x (t) = −x(−t) (f ) = − ∗ (f ) = −(−f ),
 imaginary and odd
+∞ +∞
Parseval theorem Ex = |x(t)|2 dt = |(f )|2 df = E
∫−∞ ∫−∞
( )
∑
+∞
1 ∑
+∞
𝓁
Poisson sum formula x(kTc ) = 
k=−∞
Tc 𝓁=−∞
Tc
4 Chapter 1. Elements of signal theory

sinc(tF)

−4/F −3/F −2/F −1/F 0 1/F 2/F 3/F 4/F t

1/F·rect(f/F)

1/F

−F/2 0 F/2 f

Figure 1.2 Example of signal and Fourier transform pair.

x(k) y(k)
h
Tc Tc

Figure 1.3 Discrete-time linear system (filter).

the set of integers. We denote by {x(k)} or {xk } the entire discrete-time signal, also called sequence. The
impulse response of the system is denoted by {h(k)}, k ∈ ℤ, or more simply by h.
The relation between the input sequence {x(k)} and the output sequence {y(k)} is given by the con-
volution operation:
∑
+∞
y(k) = h(k − n)x(n) (1.14)
n=−∞

denoted as y(k) = x ∗ h(k). In the discrete time, the delta function is simply the Kronecker impulse
{
1 n=0
𝛿n = 𝛿(n) = (1.15)
0 n≠0

Here are some definitions holding for time-invariant linear systems.

The system is causal (anticausal) if h(k) = 0, k < 0 (if h(k) = 0, k > 0).
1.2. Discrete-time linear systems 5

Table 1.2: Examples of Fourier transform signal pairs.

signal Fourier transform

x(t) (f )
𝛿(t) 1
1 (constant) 𝛿(f )
ej2𝜋f0 t 𝛿(f − f0 )
1
cos(2𝜋f0 t) [𝛿(f − f0 ) + 𝛿(f + f0 )]
2
1
sin(2𝜋f0 t) [𝛿(f − f0 ) − 𝛿(f + f0 )]
2j
1 1
1(t) 𝛿(f ) +
2 j2𝜋f
1
sgn(t)
( ) j𝜋f
t
rect T sinc(f T)
(T )
t
sinc T rect(fT)
( ) T
|t| ( )
t
1− rect Tsinc2 (fT)
T 2T
1
e−at 1(t), a > 0
a + j2𝜋f
1
t e−at 1(t), a > 0
(a + j2𝜋f )2
2a
e−a|t| , a > 0
a2 + (2𝜋f )2
√
𝜋 −𝜋 𝜋a f 2
e−at , a > 0
2
e
a

The transfer function of the filter is defined as the z-transform2 of the impulse response h, given by
∑
+∞
H(z) = h(k)z−k (1.16)
k=−∞

Let the frequency response of the filter be defined as

∑
+∞
(f ) =  [h(k)] = h(k)e−j2𝜋fkTc = H(z)z=ej2𝜋f Tc (1.17)
k=−∞

The inverse Fourier transform of the frequency response yields

+ 2T1
c
h(k) = Tc (f )ej2𝜋fkTc df (1.18)
∫− 1
2Tc

We note the property that, for x(k) = bk , where b is a complex constant, the output is given by
y(k) = H(b) bk . In Table 1.3, some further properties of the z-transform are summarized.
For discrete-time linear systems, in the frequency domain (1.14) becomes

(f ) = (f )(f ) (1.19)

where all functions are periodic of period 1∕Tc .

2
Sometimes the D transform is used instead of the z-transform, where D = z−1 , and H(z) is replaced by
∑+∞
h(D) = k=−∞ h(k)Dk .
6 Chapter 1. Elements of signal theory

Table 1.3: Properties of the z-transform.

property sequence z transform

x(k) X(z)
linearity ax(k) + by(k) aX(z) + bY(z)
time shift x(k − m) z−m X(z)
∗
complex conjugate x (k) X ∗ (z∗ )
( )
1
time inverse x(−k) X
z
( )
1
x∗ (−k) X∗
z∗
z-domain scaling a−k x(k) X(az)
convolution x ∗ y(k) X(z)Y(z)
( )
1
correlation x ∗ (y∗ (−m))(k) X(z)Y ∗
z∗
real sequence x(k) = x∗ (k) X(z) = X ∗ (z∗ )

Example 1.2.1
A fundamental example of z-transform is that of the sequence:
{ k
a k≥0
h(k) = , |a| < 1 (1.20)
0 k<0
Applying the transform (1.16), we find
1
H(z) = (1.21)
1 − az−1
defined for |az−1 | < 1 or |z| > |a|.

Example 1.2.2
Let q(t), t ∈ ℝ, be a continuous-time signal with Fourier transform (f ), f ∈ ℝ. We now consider the
sequence obtained by sampling q, that is

hk = q(kTc ), k∈ℤ (1.22)

Using the Poisson formula of Table 1.1, we have that the Fourier transform of the sequence {hk } is
related to (f ) by
( )
1 ∑
∞
1
(f ) =  [hk ] = H (ej2𝜋f Tc ) =  f −𝓁 (1.23)
Tc 𝓁=−∞ Tc

Definition 1.2
Let us introduce the useful pulse with parameter N, a positive integer number,
1 sin(𝜋a)
sincN (a) = ( ) (1.24)
N sin 𝜋 a
N

and sincN (0) = 1. The pulse is periodic with period N (2N) if N is odd (even). For N, very large sincN (a)
approximates sinc(a) in the range |a| ≪ N∕2. ◽
1.2. Discrete-time linear systems 7

Example 1.2.3
For the signal {
1 k = 0, 1, … , N − 1
hk = (1.25)
0 otherwise
with sampling period Tc , it is
N−1
(f ) = e−j2𝜋f 2
Tc
N sincN (f NTc ) (1.26)

Discrete Fourier transform

For a sequence with a finite number of samples, {gk }, k = 0, 1, … , N − 1, the Fourier transform becomes
∑
N−1
(f ) = gk e−j2𝜋fkTc (1.27)
k=0
Evaluating (f ) at the points f = m∕(NTc ), m = 0, 1, … , N − 1, and setting m = (m∕(NTc )), we
obtain:
∑
N−1
2𝜋
m = gk WNkm , WN = e−j N (1.28)
k=0
The sequence {m }, m = 0, 1, … , N − 1, is called the discrete Fourier transform (DFT) of {gk }, k =
0, 1, … , N − 1. The inverse of (1.28) is given by

1 ∑
N−1
gk =  W −km , k = 0, 1, … , N − 1 (1.29)
N m=0 m N
We note that, besides the factor 1∕N, the expression of the inverse DFT (IDFT) coincides with that of
the DFT, provided WN−1 is substituted with WN .
We also observe that the direct computation of (1.28) requires N(N − 1) complex additions and N 2
complex multiplications; however, the algorithm
( known)as fast Fourier transform (FFT) computes the
DFT by N log2 N complex additions and N2 log2 N − N complex multiplications.3
A simple implementation is also available when the DFT size is an integer power of some numbers
(e.g. 2, 3, and 5). The efficient implementation of a DFT with length power of n (2, 3, and 5) is denoted as
radix-n FFT. Moreover, if the DFT size is the product of integer powers of these numbers, the DFT can
be implemented as a cascade of FFTs. In particular, by letting M = 2𝛼2 , L = 3𝛼3 ⋅ 5𝛼5 , the DFT of size
N = LM can be implemented as the cascade of L M-size DFTs, the multiplication by twiddle factors
(operating only on the phase of the signal) and an L-size DFT. Applying again the same approach to
the inner M-size DFT, we obtain that the N-size DFT is the cascade of 2𝛼2 FFTs of size 3𝛼3 5𝛼5 , each
implemented by 3𝛼3 FFTs of size 5𝛼5 .

The DFT operator

The DFT operator can be expressed in matrix form as
⎡1 1 1 … 1 ⎤
⎢ ⎥
⎢ 1 WN WN2 … WN(N−1) ⎥
⎢ ⎥
F = ⎢ 1 WN2 WN4 … WN2(N−1) ⎥ (1.30)
⎢ ⎥
⎢⋮ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ (N−1) (N−1)2 ⎥
⎣ 1 WN WN … WN(N−1)(N−1) ⎦

3
The computational complexity of the FFT is often expressed as N log2 N.
8 Chapter 1. Elements of signal theory

with elements [F]i,n = WNin , i, n = 0, 1, … , N − 1. The inverse operator (IDFT) is given by4
1 ∗
F−1 = F (1.31)
N
√
We note that F = FT and (1∕ N)F is a unitary matrix.5
The following property holds: if C is a right circulant square matrix, i.e. its rows are obtained by
successive shifts to the right of the first row, then FCF−1 is a diagonal matrix whose elements are given
by the DFT of the first row of C. This property is exploited in the most common modulation scheme
(see Chapter 8).
Introducing the vector formed by the samples of the sequence {gk }, k = 0, 1, … , N − 1,
gT = [g0 , g1 , … , gN−1 ] (1.32)
and the vector of its transform coefficients
T = [0 , 1 , … , N−1 ] = DFT[g] (1.33)
from (1.28) we have
 = Fg (1.34)
Moreover, based on (1.31), we obtain
1 ∗
g= F  (1.35)
N

Circular and linear convolution via DFT

Let the two sequences x and h have a finite support of Lx and N samples, respectively, (see Figure 1.4)
with Lx > N:
x(k) = 0 k < 0 k > Lx − 1 (1.36)
and
h(k) = 0 k<0 k >N−1 (1.37)
We define the periodic signals of period L,
∑
+∞
∑
+∞
xrepL (k) = x(k − 𝓁L), hrepL (k) = h(k − 𝓁L) (1.38)
𝓁=−∞ 𝓁=−∞

x(k) h(k)

0 Lx – 1 k 0 1 N–1 k

Figure 1.4 Time-limited signals: {x(k)}, k = 0, 1, … , Lx − 1, and {h(k)}, k = 0, 1, … , N − 1.

4 T
stands for transpose and H for transpose complex conjugate or Hermitian.
5
A square matrix A is unitary if AH A = I, where I is the identity matrix, i.e. a matrix for which all elements are zero
except the elements on the main diagonal that are all equal to one.
1.2. Discrete-time linear systems 9

where in order to avoid time aliasing, it must be

L ≥ max{Lx , N} (1.39)

Definition 1.3
The circular convolution between x and h is a periodic sequence of period L defined as
L ∑
L−1
y(circ) (k) = h ⊗ x(k) = hrepL (i) xrepL (k − i) (1.40)
i=0

with main period corresponding to k = 0, 1, … , L − 1. ◽

Then, if we indicate with {m }, {m }, and {m(circ) }, m = 0, 1, … , L − 1, the L-point DFT of sequences
x, h, and y(circ) , respectively, we obtain
m(circ) = m m , m = 0, 1, … , L − 1 (1.41)
In vector notation (1.33), (1.41) becomes6
[ ]T
 (circ) = 0(circ) , 1(circ) , … , L−1
(circ)
= diag {DFT[x]}  (1.42)

where  is the column vector given by the L-point DFT of the sequence h, completed with L − N zeros.
We are often interested in the linear convolution between x and h given by (1.14):
∑
N−1
y(k) = x ∗ h(k) = h(i)x(k − i) (1.43)
i=0

whose support is k = 0, 1, … , Lx + N − 2.
We give below two relations between the circular convolution y(circ) and the linear convolution y.

Relation 1. For
L ≥ Lx + N − 1 (1.44)
by comparing (1.43) with (1.40), the two convolutions y(circ) and y coincide only for the instants
k = 0, 1, … , L − 1, i.e.
y(k) = y(circ) (k), k = 0, 1, … , L − 1 (1.45)
To compute the convolution between the two finite-length sequences x and h, (1.44) and (1.45) require
that both sequences be completed with zeros (zero padding) to get a length of L = Lx + N − 1 samples.
Then, taking the L-point DFT of the two sequences, performing the product (1.41), and taking the inverse
transform of the result, one obtains the desired linear convolution.

Relation 2. For L = Lx > N, the two convolutions y(circ) and y coincide only for the instants k = N − 1,
N, … , L − 1, i.e.
y(circ) (k) = y(k) only for k = N − 1, N, … , L − 1 (1.46)
An example of circular convolution is provided in Figure 1.5. Indeed, the result of circular convolution
coincides with {y(k)}, output of the linear convolution, only for a delay k such that it is avoided the
product between non-zero samples of the two periodic sequences hrepL and xrepL , indicated by • and ⚬,
respectively. This is achieved only for k ≥ N − 1 and k ≤ L − 1.

6
The notation diag{𝒗} denotes a diagonal matrix whose elements on the diagonal are equal to the elements of the vector
𝒗.
10 Chapter 1. Elements of signal theory

hrep L(i) xrep L(k– i)

0 N–1 L–1 i k– (L–1) k i

Figure 1.5 Illustration of the circular convolution operation between {x(k)}, k = 0, 1, … , L − 1, and
{h(k)}, k = 0, 1, … , N − 1.

Relation 3. A relevant case wherein the cyclic convolution is equivalent to the linear convolution
requires a special structure of the sequence x. Consider x(cp) , the extended sequence of x, obtained by
partially repeating x with a cyclic prefix of Ncp samples:
{
x(k) k = 0, 1, … , Lx − 1
x(cp) (k) = (1.47)
x(Lx + k) k = −Ncp , … , −2, −1

Let y(cp) be the linear convolution between x(cp) and h, with support {−Ncp , … , Lx + N − 2}. If
Ncp ≥ N − 1, we have
y(cp) (k) = y(circ) (k), k = 0, 1, … , Lx − 1 (1.48)

Let us define {
y(cp) (k) k = 0, 1, … , Lx − 1
z(k) = (1.49)
0 elsewhere
then from (1.48) and (1.41) the following relation between the corresponding Lx –point DFTs is obtained:

 m =  m m , m = 0, 1, … , Lx − 1 (1.50)

Convolution by the overlap-save method

For a very long sequence x, the application of (1.46) leads to the overlap-save method to determine
the linear convolution between x and h (with L = Lx > N). It is not restrictive to assume that the first
(N − 1) samples of the sequence {x(k)} are zero. If this were not True, it would be sufficient to shift
the input by (N − 1) samples. A fast procedure to compute the linear convolution {y(k)} for instants
k = N − 1, N, … , L − 1, operates iteratively and processes blocks of L samples, where adjacent blocks
are overlapping by (N − 1) samples. The procedure operates the following first iteration:7

1. Loading
L−N zeros
⏞⏞⏞
h′T = [h(0), h(1), … , h(N − 1), 0, … , 0] (1.51)
x ′T
= [x(0), x(1), … , x(N − 1), x(N), … , x(L − 1)] (1.52)

in which we have assumed x(k) = 0, k = 0, 1, … , N − 2.

7
In this section, the superscript ′ indicates a vector of L elements.
1.2. Discrete-time linear systems 11

2. Transform
 ′ = DFT[h′ ] vector (1.53)
 = diag{DFT[x ]}
′ ′
matrix (1.54)

3. Matrix product
 ′ =  ′′ vector (1.55)

4. Inverse transform
N−1 terms
⏞⏞⏞
y′T = DFT−1 [ ′ T ] = [ ♯, … , ♯ , y(N − 1), y(N), … , y(L − 1)] (1.56)

where the symbol ♯ denotes a component that is neglected.

The second iteration operates on load

x′T = [x((L − 1) − (N − 2)), … , x(2(L − 1) − (N − 2))] (1.57)

and the desired output samples will be

y(k) k = L, … , 2(L − 1) − (N − 2) (1.58)

The third iteration operates on load

x′T = [x(2(L − 1) − 2(N − 2)), … , x(3(L − 1) − 2(N − 2))] (1.59)

and will yield the desired output samples

y(k) k = 2(L − 1) − (N − 2) + 1, … , 3(L − 1) − 2(N − 2) (1.60)

The algorithm proceeds iteratively until the entire input sequence is processed.

IIR and FIR filters

An important class of linear systems is identified by the input–output relation
∑
p
∑
q
𝚊n y(k − n) = 𝚋n x(k − n) (1.61)
n=0 n=0

where we will set 𝚊0 = 1 without loss of generality.

If the system is causal, (1.61) becomes
∑
p
∑
q
y(k) = − 𝚊n y(k − n) + 𝚋n x(k − n) k≥0 (1.62)
n=1 n=0

and the transfer function for such system is

∑q ∏q
Y(z) 𝚋 z−n
n=0 n
𝚋0 n=1 (1 − zn z−1 )
H(z) = = ∑ = ∏p (1.63)
X(z) 1 + pn=1 𝚊n z−n n=1
(1 − pn z−1 )
where {zn } and {pn } are, respectively, the zeros and poles of H(z). Equation (1.63) generally defines an
infinite impulse response (IIR) filter. In the case in which 𝚊n = 0, n = 1, 2, … , p, (1.63) reduces to
∑
q
H(z) = 𝚋n z−n (1.64)
n=0
12 Chapter 1. Elements of signal theory

Table 1.4: Impulse responses of systems having the same magnitude of

the frequency response.

h(0) h(1) h(2) h(3) h(4)

−j1.57 −j0.31
h1 (minimum phase) 0.9e 0 0 0.4e 0.3e−j0.63
j0.63 j0.31
h2 (maximum phase) 0.3e 0.4e 0 0 0.9ej1.57
−j1.57 j2.34 −j1.66 −j0.51
h3 (general case) 0.7e 0.24e 0.15e 0.58e 0.4e−j0.63

and we obtain a finite impulse response (FIR) filter, with h(n) = 𝚋n , n = 0, 1, … , q. To get the impulse
response coefficients, assuming that the z-transform H(z) is known, we can expand H(z) in partial frac-
tions and apply the linear property of the z-transform (see Table 1.3, page 6). If q < p, and assuming
that all poles are distinct, we obtain

⎧∑ p
∑
p
rn ⎪ rn pkn k≥0
H(z) = =⇒ h(k) = ⎨ n=1 (1.65)
1 − pn z−1 ⎪0
n=1
⎩ k<0
where
[ ]|
rn = H(z) 1 − pn z−1 | (1.66)
|z=pn
We give now two definitions.

Definition 1.4
A causal system is stable (bounded input-bounded output stability) if |pn | < 1, ∀n. ◽

Definition 1.5
The system is minimum phase (maximum phase) if |pn | < 1 and |zn | ≤ 1 (|pn | > 1 and |zn | > 1),
∀n. ◽

Among all systems having the same magnitude response |(ej2𝜋f Tc )|, the minimum (maximum) phase
system presents a phase8 response, arg (ej2𝜋f Tc ), which is below (above) the phase response of all other
systems.

Example 1.2.4
It is interesting to determine the phase of a system for a given impulse response. Let us consider the
system with transfer function H1 (z) and impulse response h1 (k) shown in Figure 1.6a. After determining
the zeros of the transfer function, we factorize H1 (z) as follows:
∏
4
H1 (z) = 𝚋0 (1 − zn z−1 ) (1.67)
n=1

As shown in Figure 1.6a, H1 (z) is minimum phase. We now observe that the magnitude of the
frequency response does not change if 1∕z∗n is replaced with zn in (1.67). If we move all the zeros
outside the unit circle, we get a maximum-phase system H2 (z) whose impulse response is shown
in Figure 1.6b. A general case, that is a transfer function with some zeros inside and others outside
the unit circle, is given in Figure 1.6c. The coefficients of the impulse responses h1 , h2 , and h3 are
given in Table 1.4. The coefficients are normalized so that the three impulse responses have equal
energy.

8
For a complex number c, arg c denotes the phase of c.
1.2. Discrete-time linear systems 13

1 2
0.9

0.8
1

Imaginary part
0.6
∣h1(k)∣

0
0.4
0.4
0.3
–1
0.2

0 0
0 –2
–1 0 1 2 3 4 5 –2 –1 0 1 2
k Real part
(a)

1 2
0.9

0.8
1

Imaginary part
0.6
∣h2(k)∣

0
0.4
0.4
0.3
–1
0.2

0 0
0 –2
–1 0 1 2 3 4 5 –2 –1 0 1 2
k Real part
(b)

1 2

0.8
0.7 1
Imaginary part

0.6 0.58
∣h3(k)∣

0
0.4
0.4
0.24
–1
0.2 0.15

0 –2
–1 0 1 2 3 4 5 –2 –1 0 1 2
k Real part
(c)

Figure 1.6 Impulse response magnitudes and zero locations for three systems having the same
frequency response magnitude. (a) Minimum-phase system, (b) maximum-phase system, and (c)
general system.
We define the partial energy of a causal impulse response as
∑
k
E(k) = |h(i)|2 (1.68)
i=0

Comparing the partial-energy sequences for the three impulse responses of Figure 1.6, one finds that the
minimum (maximum) phase system yields the largest (smallest) {E(k)}. In other words, the magnitude
14 Chapter 1. Elements of signal theory

│ ( f )│ │ ( f )│

0 f2 f 0 f1 f

(a) (b)

│  ( f )│ │ (f)│

0 f1 f2 f 0 f1 f2 f

(c) (d)

│ ( f )│

0 f

(e)

Figure 1.7 Classification of real valued analog filters on the basis of the support of |(f )|. (a)
f1 = 0, f2 < ∞: lowpass filter (LPF). (b) f1 > 0, f2 = ∞: highpass filter (HPF). (c) f1 > 0, f2 < ∞:
passband filter (PBF). (d) B = f2 − f1 ≪ (f2 + f1 )∕2: narrowband filter (NBF). (e) f1 = 0, f2 = ∞:
allpass filter (APF).

of the frequency responses being equal, a minimum (maximum) phase system concentrates all its energy
on the first (last) samples of the impulse response.
Extending our previous considerations also to IIR filters, if h1 is a causal minimum-phase filter, i.e.
H1 (z) =(Hmin −1
) (z) is a ratio of polynomials in z with poles and zeros inside the unit circle, then Hmax (z) =
∗ 1
K Hmin z∗ , where K is a constant, is an anticausal maximum-phase filter, i.e. Hmax (z) is a ratio of
polynomials in z with poles and zeros outside the unit circle.
In the case of( a) minimum-phase FIR filter with impulse response hmin (n), n = 0, 1, … , q,
∗ 1
H2 (z) = z−q Hmin z∗
is a causal maximum-phase filter. Moreover, the relation {h2 (n)} = {h∗1 (q − n)},
n = 0, 1, … , q, is satisfied. In this text, we use the notation {h2 (n)} = {hB∗
1
(n)}, where B is the backward
operator that orders the elements of a sequence from the last to the first.
In Appendix 1.A multirate transformations for systems are described, in which the time domain of the
input is different from that of the output. In particular, decimator and interpolator filters are introduced,
together with their efficient implementations.

1.3 Signal bandwidth

Definition 1.6
The support of a signal x(𝜉), 𝜉 ∈ ℝ, is the set of values 𝜉 ∈ ℝ for which |x(𝜉)| ≠ 0. ◽

Let us consider a filter with impulse response h and frequency response . If h assumes real val-
ues, then  is Hermitian, (−f ) =  ∗ (f ), and |(f )| is an even function. Depending on the support
1.3. Signal bandwidth 15

│ (f)│ │ (f)│

f1 0 f2 f 0 f1 f2 f
(a) (b)

│ (f)│

f1 f2 0 f3 f4 f
(c)

Figure 1.8 Classification of complex-valued analog filters on the basis of support of |(f )|. (a)
−∞ < f1 ≤ 0, 0 < f2 < ∞: lowpass filter. (b) f1 > 0, f2 < ∞: passband filter. (c) f1 > −∞, f2 < 0,
f3 > 0, f4 < ∞: passband filter.

of |(f )|, the classification of Figure 1.7 is usually done. If h assumes complex values, the terminology
is less standard. We adopt the classification of Figure 1.8, in which the filter is a lowpass filter (LPF) if
the support |(f )| includes the origin; otherwise, it is a passband filter (PBF).
Analogously, for a signal x, we will use the same denomination and we will say that x is a base-
band (BB) or passband (PB) signal depending on whether the support of |(f )|, f ∈ ℝ, includes or not
the origin.

Definition 1.7
In general, for a real-valued signal x, the set of positive frequencies such that |(f )| ≠ 0 is called pass-
band or simply band :
 = {f ≥ 0 ∶ |(f )| ≠ 0} (1.69)
As |(f )| is an even function, we have |(−f )| ≠ 0, f ∈ . We note that  is equivalent to the support
of  limited to positive frequencies. The bandwidth of x is given by the measure of :

B= df (1.70)
∫
In the case of a complex-valued signal x,  is equivalent to the support of , and B is thus given by the
measure of the entire support. ◽

Observation 1.1
The signal bandwidth may also be given different practical definitions. Let us consider an LPF having
frequency response (f ). The filter gain 0 is usually defined as 0 = |(0)|; other definitions of gain
refer to the average gain of the filter in the passband , or as maxf |(f )|. We give the following four
definitions for the bandwidth B of h:
(a) First zero:
B = min{f > 0 ∶ (f ) = 0} (1.71)
(b) Based on amplitude, bandwidth at A dB:
{ }
|(f )| A
B = max f > 0 ∶ = 10 − 20
(1.72)
0
Typically, A = 3, 40, or 60.
16 Chapter 1. Elements of signal theory

(c) Based on energy, bandwidth at p%:

B
|(f )|2 df
∫0 p
∞ = (1.73)
100
|(f )|2 df
∫0
Typically, p = 90 or 99.
(d) Equivalent noise bandwidth:
∞
|(f )|2 df
∫0
B= (1.74)
02

Figure 1.9 illustrates the various definitions for a particular |(f )|. For example, with regard to the
signals of Figure 1.7, we have that for an LPF B = f2 , whereas for a PBF B = f2 − f1 .
For discrete-time filters, for which  is periodic of period 1∕Tc , the same definitions hold, with the
caution of considering the support of |(f )| within a period, let us say between −1∕(2Tc ) and 1∕(2Tc ).
In the case of discrete-time highpass filters (HPFs), the passband will extend from a certain frequency
f1 to 1∕(2Tc ).

0
B3dB
−10
Breq
−20

−30

−40
|H(f )| (dB)

−50 Bz

−60
B40dB
−70
BE (p = 90)
−80
B50dB

−90 BE (p = 99)

−100
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
f (Hz)

Figure 1.9 The real signal bandwidth following the definitions of (1) bandwidth at first zero:
Bz = 0.652 Hz; (2) amplitude-based bandwidth: B3 dB = 0.5 Hz, B40 dB = 0.87 Hz, B50 dB = 1.62 Hz;
(3) energy-based bandwidth: BE(p=90) = 1.362 Hz, BE(p=99) = 1.723 Hz; (4) equivalent noise
bandwidth: Breq = 0.5 Hz.
1.3. Signal bandwidth 17

The sampling theorem

As discrete-time signals are often obtained by sampling continuous-time signals, we will state the fol-
lowing fundamental theorem.

Theorem 1.1 (Sampling theorem)

Let q(t), t ∈ ℝ be a continuous-time signal, in general complex-valued, whose Fourier transform (f )
has support within an interval  of finite measure B0 . The samples of the signal q, taken with period Tc
as represented in Figure 1.10a,
hk = q(kTc ) (1.75)
univocally represent the signal q(t), t ∈ ℝ, under the condition that the sampling frequency 1∕Tc satisfies
the relation
1
≥ B0 (1.76)
Tc

q(t) kTc hk hk q(t)

gI

Tc Tc
(a) (b)

Figure 1.10 Operation of (a) sampling and (b) interpolation.

◽

For the proof, which is based on the relation (1.23) between a signal and its samples, we refer the
reader to [2].
B0 is often referred to as the minimum sampling frequency. If 1∕Tc < B0 the signal cannot be perfectly
reconstructed from its samples, originating the so-called aliasing phenomenon in the frequency-domain
signal representation.
In turn, the signal q(t), t ∈ ℝ, can be reconstructed from its samples {hk } according to the scheme of
Figure 1.10b, where it is employed an interpolation filter having an ideal frequency response given by
{
1 f ∈
I (f ) = (1.77)
0 elsewhere
We note that for real-valued baseband signals B0 = 2B. For passband signals, care must be taken in
the choice of B0 ≥ 2B to avoid aliasing between the positive and negative frequency components of
(f ).

Heaviside conditions for the absence of signal distortion

Let us consider a filter having frequency response (f ) (see Figures 1.1 or 1.3) given by
(f ) = 0 e−j2𝜋f t0 , f ∈ (1.78)
where 0 and t0 are two non-negative constants, and  is the passband of the filter input signal x. Then
the output is given by
(f ) = (f )(f ) = 0 (f ) e−j2𝜋f t0 (1.79)
or, in the time domain,
y(t) = 0 x(t − t0 ) (1.80)
18 Chapter 1. Elements of signal theory

│ (f)│
arg  (f )
0

f1 f2

–f2 –f1 0 f1 f2 f –f2 –f1 0 f

(a) (b)

Figure 1.11 Characteristics of a filter satisfying the conditions for the absence of signal distortion in
the frequency interval (f1 , f2 ). (a) Magnitude and (b) phase.

In other words, for a filter of the type (1.78), the signal at the input is reproduced at the output with a
gain factor 0 and a delay t0 .
A filter of the type (1.78) satisfies the Heaviside conditions for the absence of signal distortion and is
characterized by

1. Constant magnitude
|(f )| = 0 , f ∈ (1.81)

2. Linear phase
arg (f ) = −2𝜋f t0 , f ∈ (1.82)

3. Constant group delay, also called envelope delay

1 d
𝜏(f ) = − arg (f ) = t0 , f ∈ (1.83)
2𝜋 df

We underline that it is sufficient that the Heaviside conditions are verified within the support of ; as
|(f )| = 0 outside the support, the filter frequency response may be arbitrary.
We show in Figure 1.11 the frequency response of a PBF, with bandwidth B = f2 − f1 , that satisfies
the conditions stated by Heaviside.

1.4 Passband signals and systems

We now provide a compact representation of passband signals and describe their transformation by linear
systems.

Complex representation
For a passband signal x, it is convenient to introduce an equivalent representation in terms of a baseband
signal x(bb) .
Let x be a PB real-valued signal with Fourier transform as illustrated in Figure 1.12. The following
two procedures can be adopted to obtain x(bb) .
1.4. Passband signals and systems 19

( f)

–f2 –f1 0 f1 f0 f2 f
 (a)( f ) Ideal phase splitter
2

–f2 –f1 0 f1 f0 f2 f
 (a)( f )

0 f1 f0 f2 f
 (bb)(f )

f1 – f0 0 f2 – f0 f

Figure 1.12 Transformations to obtain the baseband equivalent signal x(bb) around the carrier
frequency f0 using a phase splitter.

Phase splitter
x(t) x(a)(t) x(bb)(t)
h(a)

e–j2 πf0t

Figure 1.13 Transformations to obtain the baseband equivalent signal x(bb) around the carrier
frequency f0 using a phase splitter.

PB filter. Referring to Figure 1.12 and to the transformations illustrated in Figure 1.13, given x we
extract its positive frequency components using an analytic filter or phase splitter, h(a) , having the fol-
lowing ideal frequency response
{
2 f >0
 (a) (f ) = 2 ⋅ 1(f ) = (1.84)
0 f <0

In practice, it is sufficient that h(a) is a complex PB filter, with  (a) (f ) ≃ 2 in the passband that extends
from f1 to f2 , as (f ), and stopband, in which | (a) (f )| ≃ 0, that extends from −f2 to −f1 . The signal x(a)
is called the analytic signal or pre-envelope of x.
It is now convenient to introduce a suitable frequency f0 , called reference carrier frequency, which
belongs to the passband (f1 , f2 ) of x. The filter output, x(a) , is frequency shifted by f0 to obtain a BB
signal, x(bb) . The signal x(bb) is the baseband equivalent of x, also named complex envelope of x around
the carrier frequency f0 .
Analytically, we have

x(a) (t) = x ∗ h(a) (t) ←−→  (a) (f ) = (f ) (a) (f ) (1.85)

x(bb) (t) = x(a) (t) e−j2𝜋f0 t ←−→  (bb) (f ) =  (a) (f + f0 ) (1.86)
20 Chapter 1. Elements of signal theory

and in the frequency domain

{
2(f + f0 ) for f > −f0
 (bb)
(f ) = (1.87)
0 for f < −f0

In other words, x(bb) is given by the components of x at positive frequencies, scaled by 2 and frequency
shifted by f0 .

BB filter. We obtain the same result using a frequency shift of x followed by a lowpass filter (see
Figures 1.14 and 1.15). It is immediate to determine the relation between the frequency responses of the
filters of Figures 1.12 and 1.14:
(f ) =  (a) (f + f0 ) (1.88)

From (1.88) one can derive the relation between the impulse response of the analytic filter and the
impulse response of the lowpass filter:

h(a) (t) = h(t) ej2𝜋f0 t (1.89)

X( f)

–f2 –f1 0 f1 f0 f2 f
X(f + f0(t)

–2f0 f1 – f 0 0 f2 – f0 f
2 (f)

f1 – f 0 0 f2 – f 0 f
(bb)
X (f)

f1 – f 0 0 f2 – f 0 f

Figure 1.14 Illustration of transformations to obtain the baseband equivalent signal x(bb) around the
carrier frequency f0 using a lowpass filter.

LPF
x(t) x(bb)(t)
h

e–j2 πf0t

Figure 1.15 Transformations to obtain the baseband equivalent signal x(bb) around the carrier
frequency f0 using a lowpass filter.
1.4. Passband signals and systems 21

x(bb)(t) x(a)(t) x(t)

Re[.]

ej2πf0 t

Figure 1.16 Relation between a signal, its complex envelope and the analytic signal.

Relation between a signal and its complex representation

A simple analytical relation exists between a real signal x and its complex envelope. In fact, making use
of the property (−f ) =  ∗ (f ), it follows
(f ) = (f )1(f ) + (f )1(−f ) = (f )1(f ) +  ∗ (−f )1(−f ) (1.90)
or equivalently,
x(a) (t) + x(a)∗ (t)
x(t) = = Re [x(a) (t)] (1.91)
2
Using (1.86) it also follows
x(t) = Re [x(bb) (t)ej2𝜋f0 t ] (1.92)
as illustrated in Figure 1.16.

Baseband components of a PB signal. We introduce the notation

x(bb) (t) = xI(bb) (t) + jxQ
(bb)
(t) (1.93)
where
xI(bb) (t) = Re [x(bb) (t)] (1.94)
and
(bb)
xQ (t) = Im [x(bb) (t)] (1.95)
are real-valued baseband signals, named in-phase and quadrature components of x, respectively. Sub-
stituting (1.93) in (1.92), we obtain
x(t) = xI(bb) (t) cos(2𝜋f0 t) − xQ
(bb)
(t) sin(2𝜋f0 t) (1.96)
as illustrated in Figure 1.17.
Conversely, given x, one can use the scheme of Figure 1.15 and the relations (1.94) and (1.95) to get
the baseband components. If the frequency response (f ) has Hermitian-symmetric characteristics with
respect to the origin, h is real and the scheme of Figure 1.18 holds. The scheme of Figure 1.18 employs
instead an ideal Hilbert filter with frequency response given by
𝜋
 (h) (f ) = −j sgn(f ) = e−j 2 sgn(f ) (1.97)
Magnitude and phase of  (h) (f ) are shown in Figure 1.19. We note that h(h) phase-shifts by −𝜋∕2 the
positive-frequency components of the input and by 𝜋∕2 the negative-frequency components. In practice,
these filter specifications are imposed only on the passband of the input signal.9 To simplify the notation,
in block diagrams a Hilbert filter is indicated as −𝜋∕2.

9
We note that the ideal Hilbert filter in Figure 1.19 has an impulse response given by (see Table 1.2 on page 5):
1
h(h) (t) = (1.98)
𝜋t
22 Chapter 1. Elements of signal theory

cos(2 πf0t)

xI(bb)(t)

x(t)

(bb)(t)
xQ

–sin(2 πf0t)

Figure 1.17 Relation between a signal and its baseband components.

cos(2 πf0 t)
LPF
x(bb)
I (t)
h
x(t)
LPF
x(bb)
Q (t)
h

–sin(2 πf0 t)

(a)

cos(2 πf0 t)

x(t)
x(bb)
I (t)

x(t) –sin(2 πf0 t)

sin(2 πf0 t)
–jsgn( f ) x(bb)
Q (t)
x(h)(t)
–π
2

cos(2 πf0 t)

(b)

Figure 1.18 Relations to derive the baseband signal components. (a) Implementation using LPF
and (b) Implementation using Hilbert filter.
1.4. Passband signals and systems 23

│ (h) ( f)│
1

0 f

arg  (h) ( f)
π
2

0 f
π
−
2

Figure 1.19 Magnitude and phase responses of the ideal Hilbert filter.

Comparing the frequency responses of the analytic filter (1.84) and of the Hilbert filter (1.97), we
obtain the relation
 (a) (f ) = 1 + j (h) (f ) (1.101)
Then, letting
x(h) (t) = x ∗ h(h) (t) (1.102)
the analytic signal can be expressed as
x(a) (t) = x(t) + jx(h) (t) (1.103)
Consequently, from (1.86), (1.94), and (1.95), we have
xI(bb) (t) = x(t) cos(2𝜋f0 t) + x(h) (t) sin(2𝜋f0 t) (1.104)
(bb)
xQ (t) = x(h) (t) cos(2𝜋f0 t) − x(t) sin(2𝜋f0 t) (1.105)

as illustrated in Figure 1.18.10

Consequently, if x is the input signal, the output of the Hilbert filter (also denoted as Hilbert transform of x) is
+∞
1 x(𝜏)
x(h) (t) = d𝜏 (1.99)
𝜋 ∫−∞ t − 𝜏
Moreover, noting that from (1.97) (−j sgn f )(−j sgn f ) = −1, taking the Hilbert transform of the Hilbert transform of a
signal, we get the initial signal with the sign changed. Then it results as
+∞ (h)
1 x (𝜏)
x(t) = − d𝜏 (1.100)
𝜋 ∫−∞ t − 𝜏

10
We recall that the design of a filter, and in particular of a Hilbert filter, requires the introduction of a suitable delay. In
other words, we are only able to produce an output with a delay tD , x(h) (t − tD ). Consequently, in the block diagram of
Figure 1.18, also x and the various sinusoidal waveforms must be delayed.
24 Chapter 1. Elements of signal theory

We note that in practical systems, transformations to obtain, e.g. the analytic signal, the complex
envelope, or the Hilbert transform of a given signal, are implemented by filters. However, it is usually
more convenient to perform signal analysis in the frequency domain by the Fourier transform. In the
following two examples, we use frequency-domain techniques to obtain the complex envelope of a PB
signal.

Example 1.4.1
Consider the sinusoidal signal
x(t) = A cos(2𝜋f0 t + 𝜑0 ) (1.106)

with
A j𝜑0 A
(f ) = e 𝛿(f − f0 ) + e−j𝜑0 𝛿(f + f0 ) (1.107)
2 2
The analytic signal is given by
 −1
 (a) (f ) = Aej𝜑0 𝛿(f − f0 ) ←−→ x(a) (t) = Aej𝜑0 ej2𝜋f0 t (1.108)

and
 −1
 (bb) (f ) = Aej𝜑0 𝛿(f ) ←−→ x(bb) (t) = Aej𝜑0 (1.109)

We note that we have chosen as reference carrier frequency of the complex envelope the same carrier
frequency as in (1.106).

Example 1.4.2
Let
x(t) = A sinc(Bt) cos(2𝜋f0 t) (1.110)

with the Fourier transform given by

[ ( ) ( )]
A f − f0 f + f0
(f ) = rect + rect (1.111)
2B B B
as illustrated in Figure 1.20. Then, using f0 as reference carrier frequency,
( )
A f
 (f ) = rect
(bb)
(1.112)
B B
and
x(bb) (t) = Asinc(Bt) (1.113)

Another analytical technique to get the expression of the signal after the various transformations is
obtained by applying the following theorem.

Theorem 1.2
Let the product of two real signals be
x(t) = a(t) c(t) (1.114)

where a is a BB signal with a = [0, B) and c is a PB signal with c = [f0 , +∞). If f0 > B, then the
analytic signal of x is related to that of c by

x(a) (t) = a(t) c(a) (t) (1.115)

◽
1.4. Passband signals and systems 25

Proof. We consider the general relation (1.91), valid for every real signal
1 (a) 1
c(t) = c (t) + c(a)∗ (t) (1.116)
2 2
Substituting (1.116) in (1.114) yields
1 1
x(t) = a(t) c(a) (t) + a(t) c(a)∗ (t) (1.117)
2 2
In the frequency domain, the support of the first term in (1.117) is given by the interval [f0 − B, +∞),
while that of the second is equal to (−∞, −f0 + B]. Under the hypothesis that f0 ≥ B, the two terms in
(1.117) have disjoint supports in the frequency domain and (1.115) is immediately obtained. ◽

Corollary 1.1
From (1.115), we obtain
x(h) (t) = a(t)c(h) (t) (1.118)
and
x(bb) (t) = a(t)c(bb) (t) (1.119)
In fact, from (1.103) we get
x(h) (t) = Im [x(a) (t)] (1.120)
which substituted in (1.115) yields (1.118). Finally, (1.119) is obtained by substituting (1.86),
x(bb) (t) = x(a) (t)e−j2𝜋f0 t (1.121)
in (1.115). ◽

An interesting application of (1.120) is in the design of a Hilbert filter h(h) starting from a lowpass
filter h. In fact, from (1.89) and (1.120), we get
h(h) (t) = h(t) sin(2𝜋f0 t) (1.122)

Example 1.4.3
Let a modulated double sideband (DSB) signal be expressed as
x(t) = a(t) cos(2𝜋f0 t + 𝜑0 ) (1.123)

X( f)
A
2B

B − f0 B 0 B f0 B f
−f0 − −f0 + f0 − f0 +
2 2 X(bb)( f) 2 2
A
B

B 0 B f
−
2 2

Figure 1.20 Frequency response of a PB signal and corresponding complex envelope.

26 Chapter 1. Elements of signal theory

Table 1.5: Some properties of the Hilbert transform.

property (real) signal (real) Hilbert transform

x(t) x(h) (t)
duality x(h) (t) −x(t)
time inverse x(−t) −x(h) (−t)
even signal x(t) = x(−t) x (t) = −x(h) (−t), odd
(h)

odd signal x(t) = −x(−t) x(h) (t) = x(h) (−t), even

product (see Theorem 1.2) a(t) c(t) a(t) c(h) (t)
cosinusoidal signal cos(2𝜋f0 t + 𝜑0 ) sin(2𝜋f0 t + 𝜑0 )
+∞ +∞
energy Ex = |x(t)|2 dt = |x(h) (t)|2 dt = Ex(h)
∫−∞ ∫−∞
+∞
orthogonality x(t) x(h) (t) dt = 0
∫−∞

where a is a BB signal with bandwidth B. Then, if f0 > B, from the above theorem we have the following
relations:

x(a) (t) = a(t)ej(2𝜋f0 t+𝜑0 ) (1.124)

x (t) = a(t) sin(2𝜋f0 t + 𝜑0 )
(h)
(1.125)
(bb) j𝜑0
x (t) = a(t)e (1.126)

We list in Table 1.5 some properties of the Hilbert transformation (1.102) that are easily obtained by
using the Fourier transform and the properties of Table 1.1.

Baseband equivalent of a transformation

Given a transformation involving also passband signals, it is often useful to determine an equivalent
relation between baseband complex representations of input and output signals. Three transfor-
mations are given in Figure 1.21, together with their baseband equivalent. Note that schemes in
Figure 1.21a,b produce very different output signals, although both use a mixer with the same
carrier.
We will prove the relation illustrated in Figure 1.21b. Assuming that h is the real-valued impulse
response of an LPF and using (1.92),
{ ( )}
y(t) = h ∗ Re [x(bb) (𝜏)ej2𝜋f0 𝜏 cos(2𝜋f0 𝜏 + 𝜑1 ) ] (t)
[( −j𝜑1 +j(2𝜋2f0 𝜏+𝜑1 )
) ]
(bb) e (bb) e
= Re h ∗ x +h∗x (t) (1.127)
2 2
[( ) ]
e−j𝜑1
= Re h ∗ x(bb) (t)
2

where the last equality follows because the term with frequency components around 2f0 is filtered by
the LPF.
1.4. Passband signals and systems 27

Baseband equivalent system

(BB)

x(t) y(t) x(bb)(t) = x(t) y(bb)(t) = e jφ0x(t)

e jφ0
cos(2 πf0t + φ0)

(a)

Baseband equivalent system

(PB)
LPF
x(t) y(t) x(bb)(t) y(bb)(t) y(t)
h h Re[.]

1 e–j φ1
cos(2 πf0t + φ1) 2

(b)

Baseband equivalent system

(PB)
PBF
x(t) y(t) x(bb)(t) y(bb)(t) = x(bb) * 1 h(bb) (t)
h 1 h(bb) 2
2

(c)

Figure 1.21 Passband transformations and their baseband equivalent. (a) Modulator, (b)
demodulator, and (c) passband filtering.

We note, moreover, that the filter h(bb) in Figure 1.21 has in-phase component h(bb)
I
and quadrature
(bb)
component hQ that are related to  by (see (1.94) and (1.95))
(a)

1 (bb)
I(bb) (f ) = [ (f ) +  (bb)∗ (−f )]
2
1 (a)
= [ (f + f0 ) +  (a)∗ (−f + f0 )] (1.128)
2
and
1 (bb)
Q(bb) (f ) = [ (f ) −  (bb)∗ (−f )]
2j
1
= [ (a) (f + f0 ) −  (a)∗ (−f + f0 )] (1.129)
2j
Consequently, if  (a) has Hermitian symmetry around f0 , then

I(bb) (f ) = a(a) (f + f0 )

and
Q(bb) (f ) = 0
28 Chapter 1. Elements of signal theory

In other words, h(bb) (t) = h(bb)

I
(t) is real and the realization of the filter 12 h(bb) is simplified. In prac-
tice, this condition is verified by imposing that the filter h(a) has symmetrical frequency specifications
around f0 .

Envelope and instantaneous phase and frequency

We will conclude this section with a few definitions. Given a PB signal x, with reference to the analytic
signal we define

1. Envelope
Mx (t) = |x(a) (t)| (1.130)

2. Instantaneous phase
𝜑x (t) = arg x(a) (t) (1.131)

3. Instantaneous frequency
1 d
fx (t) = 𝜑 (t) (1.132)
2𝜋 dt x

In terms of the complex envelope signal x(bb) , from (1.86) the equivalent relations follow:
Mx (t) = |x(bb) (t)| (1.133)
𝜑x (t) = arg x (t) + 2𝜋f0 t
(bb)
(1.134)
1 d
fx (t) = [arg x(bb) (t)] + f0 (1.135)
2𝜋 dt
Then, from the polar representation, x(a) (t) = Mx (t) ej𝜑x (t) and from (1.91), a PB signal x can be written
as
x(t) = Re [x(a) (t)] = Mx (t) cos(𝜑x (t)) (1.136)

For example if x(t) = A cos(2𝜋f0 t + 𝜑0 ), it follows that

Mx (t) = A (1.137)
𝜑x (t) = 2𝜋f0 t + 𝜑0 (1.138)
fx (t) = f0 (1.139)

With reference to the above relations, three other definitions follow.

1. Envelope deviation
ΔMx (t) = |x(a) (t)| − A = |x(bb) (t)| − A (1.140)

2. Phase deviation
Δ𝜑x (t) = 𝜑x (t) − (2𝜋f0 t + 𝜑0 ) = arg x(bb) (t) − 𝜑0 (1.141)

3. Frequency deviation
1 d
Δfx (t) = fx (t) − f0 = Δ𝜑x (t) (1.142)
2𝜋 dt

Then (1.136) becomes

x(t) = [A + ΔMx (t)] cos(2𝜋f0 t + 𝜑0 + Δ𝜑x (t)) (1.143)

1.5. Second-order analysis of random processes 29

1.5 Second-order analysis of random processes

We recall the functions related to the statistical description of random processes, especially those func-
tions concerning second-order analysis.

1.5.1 Correlation
Let x(t) and y(t), t ∈ ℝ, be two continuous-time complex-valued random processes. We indicate the
expectation operator with E.
1. Mean value
𝚖x (t) = E[x(t)] (1.144)
2. Statistical power
𝙼x (t) = E[|x(t)|2 ] (1.145)
3. Autocorrelation
𝚛x (t, t − 𝜏) = E[x(t)x∗ (t − 𝜏)] (1.146)
4. Crosscorrelation
𝚛xy (t, t − 𝜏) = E[x(t)y∗ (t − 𝜏)] (1.147)
5. Autocovariance
𝚌x (t, t − 𝜏) = E[(x(t) − 𝚖x (t))(x(t − 𝜏) − 𝚖x (t − 𝜏))∗ ]

= 𝚛x (t, t − 𝜏) − 𝚖x (t)𝚖∗x (t − 𝜏) (1.148)

6. Crosscovariance
𝚌xy (t, t − 𝜏) = E[(x(t) − 𝚖x (t))(y(t − 𝜏) − 𝚖y (t − 𝜏))∗ ]

= 𝚛xy (t, t − 𝜏) − 𝚖x (t)𝚖∗y (t − 𝜏) (1.149)

Observation 1.2
● x and y are orthogonal if 𝚛xy (t, t − 𝜏) = 0, ∀t, 𝜏. In this case, we write x ⟂ y.11
● x and y are uncorrelated if 𝚌xy (t, t − 𝜏) = 0, ∀t, 𝜏.
● if at least one of the two random processes has zero mean, orthogonality is equivalent to uncorre-
lation.
● x is wide-sense stationary (WSS) if
1. 𝚖x (t) = 𝚖x , ∀t,
2. 𝚛x (t, t − 𝜏) = 𝚛x (𝜏), ∀t.
In this case, 𝚛x (0) = E[|x(t)|2 ] = 𝙼x is the statistical power, whereas 𝚌x (0) = 𝜎x2 = 𝙼x − |𝚖x |2 is the
variance of x.

11
We observe that the notion of orthogonality between two random processes is quite different from that of orthogonality
∞
between two deterministic signals. In fact, while in the deterministic case, it is sufficient that ∫−∞ x(t)y∗ (t)dt = 0, in the
random case, the crosscorrelation must be zero for all the delays and not only for the zero delay. In particular, we note
that the two random variables 𝑣1 and 𝑣2 are orthogonal if E[𝑣1 𝑣∗2 ] = 0.
30 Chapter 1. Elements of signal theory

● x and y are jointly wide-sense stationary if

1. 𝚖x (t) = 𝚖x , 𝚖y (t) = 𝚖y , ∀t,
2. 𝚛xy (t, t − 𝜏) = 𝚛xy (𝜏), ∀t.

Properties of the autocorrelation function

1. 𝚛x (−𝜏) = 𝚛∗x (𝜏), 𝚛x (𝜏) is a function with Hermitian symmetry.
2. 𝚛x (0) ≥ |𝚛x (𝜏)|.
3. 𝚛x (0)𝚛y (0) ≥ |𝚛xy (𝜏)|2 .
4. 𝚛xy (−𝜏) = 𝚛∗yx (𝜏).
5. 𝚛x∗ (𝜏) = 𝚛∗x (𝜏).

1.5.2 Power spectral density

Given the WSS random process x(t), t ∈ ℝ, its power spectral density (PSD) is defined as the Fourier
transform of its autocorrelation function
+∞
x (f ) =  [𝚛x (𝜏)] = 𝚛x (𝜏)e−j2𝜋f 𝜏 d𝜏 (1.150)
∫−∞
The inverse transformation is given by the following formula:
+∞
𝚛x (𝜏) = x (f )ej2𝜋f 𝜏 df (1.151)
∫−∞
In particular from (1.151), we obtain the statistical power
+∞
𝙼x = 𝚛x (0) = x (f )df (1.152)
∫−∞
Hence, the name PSD for the function x (f ): it represents the distribution of the statistical power in the
frequency domain.
The pair of equations (1.150) and (1.151) are obtained from the Wiener–Khintchine theorem [3].

Definition 1.8
The passband  of a random process x is defined with reference to its PSD function. ◽

Spectral lines in the PSD

In many applications, it is important to detect the presence of sinusoidal components in a random process.
With this intent we give the following theorem.

Theorem 1.3
The PSD of a WSS process, x (f ), can be uniquely decomposed into a component x(c) (f ) without delta
functions and a discrete component consisting of delta functions (spectral lines) x(d) (f ), so that
x (f ) = x(c) (f ) + x(d) (f ) (1.153)
where x(c) (f ) is an ordinary (piecewise linear) function and
∑
x(d) (f ) = 𝙼i 𝛿(f − fi ) (1.154)
i∈

where  identifies a discrete set of frequencies {fi }, i ∈ . ◽

1.5. Second-order analysis of random processes 31

The inverse Fourier transform of (1.153) yields the relation

𝚛x (𝜏) = 𝚛(c) (d)

x (𝜏) + 𝚛x (𝜏) (1.155)

with
∑
𝚛(d)
x (𝜏) = 𝙼i ej2𝜋fi 𝜏 (1.156)
i∈

The most interesting consideration is that the following random process decomposition corresponds to
the decomposition (1.153) of the PSD:

x(t) = x(c) (t) + x(d) (t) (1.157)

where x(c) and x(d) are orthogonal processes having PSD functions

x(c) (f ) = x(c) (f ) and x(d) (f ) = x(d) (f ) (1.158)

Moreover, x(d) is given by

∑
x(d) (t) = xi ej2𝜋fi t (1.159)
i∈

where {xi } are orthogonal random variables (r.v.s.) having statistical power

E[|xi |2 ] = 𝙼i , i∈ (1.160)

where 𝙼i is defined in (1.154).

Observation 1.3
The spectral lines of the PSD identify the periodic components in the process.

Definition 1.9
A WSS random process is said to be asymptotically uncorrelated if the following two properties hold:

(1) lim 𝚛x (𝜏) = |𝚖x |2 (1.161)

𝜏→∞
(2) 𝚌x (𝜏) = 𝚛x (𝜏) − |𝚖x |2 is absolutely integrable (1.162)

The property (1) shows that x(t) and x(t − 𝜏) become uncorrelated for 𝜏 → ∞. ◽

For such processes, one can prove that

𝚛(c)
x (𝜏) = 𝚌x (𝜏) and 𝚛(d)
x (𝜏) = |𝚖x |
2
(1.163)

Hence, x(d) (f ) = |𝚖x |2 𝛿(f ), and the process exhibits at most a spectral line at the origin.

Cross power spectral density

One can extend the definition of PSD to two jointly WSS random processes:

xy (f ) =  [𝚛xy (𝜏)] (1.164)

Since 𝚛xy (−𝜏) ≠ 𝚛∗xy (𝜏), xy (f ) is in general a complex function.

32 Chapter 1. Elements of signal theory

Properties of the PSD

1. x (f ) is a real-valued function. This follows from property 1 of the autocorrelation.
2. x (f ) is generally not an even function. However, if the process x is real valued, then both 𝚛x (𝜏)
and x (f ) are even functions.
3. x (f ) is a non-negative function.
4. yx (f ) = xy∗ (f ).

5. x∗ (f ) = x (−f ).
Moreover, the following inequality holds:

0 ≤ |xy (f )|2 ≤ x (f )y (f ) (1.165)

Definition 1.10 (White random process)

The zero-mean random process x(t), t ∈ ℝ, is called white if

𝚛x (𝜏) = K𝛿(𝜏) (1.166)

with K a positive real number. In this case, x (f ) is a constant, i.e.

x (f ) = K (1.167)
◽

PSD through filtering

With reference to Figure 1.22, by taking the Fourier transform of the various crosscorrelations, the
following relations are easily obtained:

yx (f ) = x (f )(f ) (1.168)

y (f ) = x (f )|(f )|2 (1.169)
yz (f ) = x (f )(f )∗ (f ) (1.170)

The relation (1.169) is of particular interest since it relates the PSDs of the output process of a filter
to the PSD of the input process, through the frequency response of the filter. In the particular case in
which y and z have disjoint passbands, i.e. y (f )z (f ) = 0, then from (1.165) 𝚛yz (𝜏) = 0, and y ⟂ z.

1.5.3 PSD of discrete-time random processes

Let {x(k)} and {y(k)} be two discrete-time random processes. Definitions and properties of Section 1.5.1
remain valid also for discrete-time processes: the only difference is that the correlation is now defined

y
h

z
g

Figure 1.22 Reference scheme of PSD computations.

1.5. Second-order analysis of random processes 33

on discrete time and is called autocorrelation sequence (ACS). It is however interesting to review the
properties of PSDs. Given a discrete-time WSS random process x, the PSD is obtained as
∑
+∞
x (f ) = Tc  [𝚛x (n)] = Tc 𝚛x (n)e−j2𝜋fnTc (1.171)
n=−∞

We note a further property: x (f ) is a periodic function of period 1∕Tc . The inverse transformation
yields:
1
2Tc
𝚛x (n) = x (f )ej2𝜋fnTc df (1.172)
∫− 1
2Tc

In particular, the statistical power is given by

1
2Tc
𝙼x = 𝚛x (0) = x (f )df (1.173)
∫− 1
2Tc

Definition 1.11 (White random process)

A discrete-time random process {x(k)} is white if

𝚛x (n) = 𝜎x2 𝛿n (1.174)

In this case, the PSD is a constant:

x (f ) = 𝜎x2 Tc (1.175)
◽

Definition 1.12
If the samples of the random process {x(k)} are statistically independent and identically Distributed, we
say that {x(k)} has i.i.d. samples. ◽

Generating an i.i.d. sequence is not simple; however, it is easily provided by many random number
generators [4]. However, generating, storing, and processing a finite length, i.i.d. sequence requires a
complex processor and a lot of memory. Furthermore, the deterministic correlation properties of such a
subsequence may not be very good. Hence, in Appendix 1.C we introduce a class of pseudonoise (PN)
sequences, which are deterministic and periodic, with very good correlation properties. Moreover, the
symbol alphabet can be just binary.

Spectral lines in the PSD

Also the PSD of a discrete time random process can be decomposed into ordinary components and
spectral lines on a period of the PSD. In particular for a discrete-time WSS asymptotically uncorrelated
random process, the relation (1.163) and the following are true
∑
+∞
x(c) (f ) = Tc 𝚌x (n) e−j2𝜋fnTc (1.176)
n=−∞
( )
∑
+∞
𝓁
x(d) (f ) = |𝚖x | 2
𝛿 f− (1.177)
𝓁=−∞
Tc

We note that, if the process has non-zero mean value, the PSD exhibits lines at multiples of 1∕Tc .
34 Chapter 1. Elements of signal theory

Example 1.5.1
We calculate the PSD of an i.i.d. sequence {x(k)}. From
{
𝙼x n=0
𝚛x (n) = (1.178)
|𝚖x |2 n≠0
it follows that {
𝜎x2 n=0
𝚌x (n) = (1.179)
0 n≠0
Then
𝚛(c)
x (n) = 𝜎x 𝛿n ,
2
𝚛(d)
x (n) = |𝚖x |
2
(1.180)
( )
∑ +∞
𝓁
x(c) (f ) = 𝜎x2 Tc , x(d) (f ) = |𝚖x |2 𝛿 f− (1.181)
𝓁=−∞
Tc

PSD through filtering

Given the system illustrated in Figure 1.3, we want to find the relation between the PSDs of the input
and output signals, assuming these processes are individually as well as jointly WSS. We introduce the
z-transform of the correlation sequence:
∑
+∞
Px (z) = 𝚛x (n)z−n (1.182)
n=−∞

From the comparison of (1.182) with (1.171), the PSD of x is related to Px (z) by

x (f ) = Tc Px (ej2𝜋f Tc ) (1.183)

Using Table 1.3 in page 6, we obtain the relations between ACS and PSD listed in Table 1.6. Let the
deterministic autocorrelation of h be defined as12
∑
+∞
𝚛h (n) = h(k)h∗ (k − n) = [h(m) ∗ h∗ (−m)](n) (1.184)
k=−∞

whose z-transform is given by

( )
∑
+∞
1
Ph (z) = 𝚛h (n) z−n = H(z) H ∗ (1.185)
n=−∞
z∗

Table 1.6: Relations between ACS and PSD for

discrete-time processes through a linear filter.

ACS PSD
𝚛yx (n) = 𝚛x ∗ h(n) Pyx (z) = Px (z)H(z)
𝚛xy (n) = [𝚛x (m) ∗ h∗ (−m)](n) Pxy (z) = Px (z)H ∗ (1∕z∗ )
𝚛y (n) = 𝚛xy ∗ h(n) Py (z) = Pxy (z)H(z)
= 𝚛x ∗ 𝚛h (n) = Px (z)H(z)H ∗ (1∕z∗ )

12
We use the same symbol to indicate the correlation between random processes and the correlation between
deterministic signals.
1.5. Second-order analysis of random processes 35

In case Ph (z) is a rational function, from (1.185) one deduces that, if Ph (z) has a pole (zero) of the type
ej𝜑 |a|, it also has a corresponding pole (zero) of the type ej𝜑 ∕|a|. Consequently, the poles (and zeros) of
Ph (z) come in pairs of the type ej𝜑 |a|, ej𝜑 ∕|a|.
From the last relation in Table 1.6, we obtain the relation between the PSDs of input and output signals,
i.e.
| |2
y (f ) = x (f )|H(ej2𝜋f Tc )| (1.186)
| |
In the case of white noise input ( )
1
Py (z) = 𝜎x2 H(z)H ∗ ∗ (1.187)
z
and
| |2
y (f ) = Tc 𝜎x2 |H(ej2𝜋f Tc )| (1.188)
| |
In other words, y (f ) has the same shape as the filter frequency response.
In the case of real filters ( )
∗ 1
H = H(z−1 ) (1.189)
z∗
Among the various applications of (1.188), it is worth mentioning the process synthesis, which
deals with the generation of a random process having a pre-assigned PSD. Two methods are shown in
Section 4.1.9.

Minimum-phase spectral factorization

In the previous section, we introduced the relation between an impulse response {h(k)} and its ACS
{𝚛h (n)} in terms of the z-transform. In many practical applications, it is interesting to determine the
minimum-phase impulse response for a given autocorrelation function: with this intent we state the
following theorem [5].

Theorem 1.4 (Spectral factorization for discrete-time processes)

Consider the process y with ACS {𝚛y (n)} having z-transform Py (z), which satisfies the Paley–Wiener
condition for discrete-time systems, i.e.
| |
|ln Py (ej2𝜋f Tc )| df < ∞ (1.190)
∫1∕Tc | |
where the integration is over an arbitrarily chosen interval 1∕Tc . Then the function Py (z) can be factorized
as follows: ( )
̃ 1
Py (z) = f02 F(z) F̃ ∗ ∗ (1.191)
z
where
̃ = 1 + f̃1 z−1 + f̃2 z−2 + · · ·
F(z) (1.192)
is monic, minimum phase, and associated with a causal sequence {1, f̃1 , f̃2 , …}. The factor f0 in (1.191)
is the geometric mean of Py (ej2𝜋f Tc ):

ln f02 = Tc ln Py (ej2𝜋f Tc ) df (1.193)

∫1∕Tc
The logarithms in (1.190) and (1.193) may have any common base.
The Paley–Wiener criterion implies that Py (z) may have only a discrete set of zeros on the unit circle,
̃ is causal, monic and minimum
and that the spectral factorization (1.191) (with the constraint that F(z)
36 Chapter 1. Elements of signal theory

̃ is obtained by extracting the poles and zeros of

phase) is unique. For rational Py (z), the function f0 F(z)
Py (z) that lie inside the unit circle (see (1.453) and the considerations relative to (1.185)). Moreover, in
(1.191) f0 F̃ ∗ (1∕z∗ ) is the z-transform of an anticausal sequence f0 {… , f̃2∗ , f̃1∗ , 1}, associated with poles
and zeros of Py (z) that lie outside the unit circle. ◽

1.5.4 PSD of passband processes

Definition 1.13
A WSS random process x is said to be PB (BB) if its PSD is of PB (BB) type. ◽

PSD of in-phase and quadrature components

Let x be a real PB WSS process. Our aim is to derive the PSD of the in-phase and quadrature compo-
nents of the process. We assume that x does not have direct current (DC) components, i.e. a frequency
component at f = 0, hence, its mean is zero and consequently also x(a) and x(bb) have zero mean.
We introduce the two (ideal) filters with frequency response
 (+) (f ) = 1(f ) and  (−) (f ) = 1(−f ) (1.194)
Note that they have non-overlapping passbands. For the same input x, the output of the two filters is,
respectively, x(+) and x(−) . We find that
x(t) = x(+) (t) + x(−) (t) (1.195)
with x(−) (t) = x(+)∗ (t). The following relations hold
x(+) (f ) = | (+) (f )|2 x (f ) = x (f )1(f ) (1.196)
x(−) (f ) = | (−)
(f )| x (f ) = x (f )1(−f )
2
(1.197)
and
x(+) x(−) (f ) = 0 (1.198)
as x(+) and x(−) have non-overlapping passbands. Then x(+) ⟂ x(−) , and (1.195) yields
x (f ) = x(+) (f ) + x(−) (f ) (1.199)
where x(−) (f ) = x(+)∗ (f ) = x(+) (−f ), using Property 5 of the PSD. The analytic signal x(a) is equal to
2x(+) , hence,
𝚛x(a) (𝜏) = 4𝚛x(+) (𝜏) (1.200)
and
x(a) (f ) = 4x(+) (f ) (1.201)
Moreover, being x(a)∗ = 2x(−) , it follows that x(a) ⟂ x(a)∗ and
𝚛x(a) x(a)∗ (𝜏) = 0 (1.202)
The complex envelope x(bb) is related to x(a) by (1.86) and
𝚛x(bb) (𝜏) = 𝚛x(a) (𝜏)e−j2𝜋f0 𝜏 (1.203)
Hence,
x(bb) (f ) = x(a) (f + f0 ) = 4x(+) (f + f0 ) (1.204)
Moreover, from (1.202), it follows that x(bb) ⟂ x(bb)∗ .
1.5. Second-order analysis of random processes 37

Using (1.204), (1.199) can be written as

1
x (f ) = [ (bb) (f − f0 ) + x(bb) (−f − f0 )] (1.205)
4 x
Finally, from
x(bb) (t) + x(bb)∗ (t)
xI(bb) (t) = Re [x(bb) (t)] = (1.206)
2
and
(bb) x(bb) (t) − x(bb)∗ (t)
xQ (t) = Im [x(bb) (t)] = (1.207)
2j
we obtain the following relations:
1
𝚛x(bb) (𝜏) = Re [𝚛x(bb) (𝜏)] (1.208)
I 2
1
x(bb) (f ) = [x(bb) (f ) + x(bb) (−f )] (1.209)
I 4
𝚛x(bb) (𝜏) = 𝚛x(bb) (𝜏) (1.210)
Q I

1
𝚛x(bb) x(bb) (𝜏) = Im [𝚛x(bb) (𝜏)] (1.211)
Q I 2
1
x(bb) x(bb) (f ) = [x(bb) (f ) − x(bb) (−f )] (1.212)
Q I 4j
𝚛x(bb) x(bb) (𝜏) = −𝚛x(bb) x(bb) (𝜏) = −𝚛x(bb) x(bb) (−𝜏) (1.213)
I Q Q I I Q

The second equality in (1.213) follows from Property 4 of ACS.

From (1.213), we note that 𝚛x(bb) x(bb) (𝜏) is an odd function. Moreover, from (1.212), we obtain xI(bb) ⟂
I Q
(bb)
xQ only if x(bb) is an even function; in any case, the random variables xI(bb) (t) and xQ
(bb)
(t) are always
orthogonal since 𝚛x(bb) x(bb) (0) = 0. Referring to the block diagram in Figure 1.18b, as
I Q

x(h) (f ) = x (f ) and x(h) x (f ) = −j sgn(f ) x (f ) (1.214)

we obtain
𝚛x(h) (𝜏) = 𝚛x (𝜏) and 𝚛x(h) x (𝜏) = 𝚛(h)
x (𝜏) (1.215)
Then
𝚛x(bb) (𝜏) = 𝚛x(bb) (𝜏) = 𝚛x (𝜏) cos(2𝜋f0 𝜏) + 𝚛(h)
x (𝜏) sin(2𝜋f0 𝜏) (1.216)
I Q

and
𝚛x(bb) x(bb) (𝜏) = −𝚛(h)
x (𝜏) cos(2𝜋f0 𝜏) + 𝚛x (𝜏) sin(2𝜋f0 𝜏) (1.217)
I Q

In terms of statistical power, the following relations hold:

1
𝚛x(+) (0) = 𝚛x(−) (0) =𝚛 (0) (1.218)
2 x
𝚛x(bb) (0) = 𝚛x(a) (0) = 4𝚛x(+) (0) = 2𝚛x (0) (1.219)
𝚛x(bb) (0) = 𝚛x(bb) (0) = 𝚛x (0) (1.220)
I Q

𝚛x(h) (0) = 𝚛x (0) (1.221)

Example 1.5.2
Let x be a WSS process with PSD
[ ( ) ( )]
N0 f − f0 f + f0
x (f ) = rect + rect (1.222)
2 B B
38 Chapter 1. Elements of signal theory

x( f)
N0
2
B –f0 –f + B 0 B f0 B f
–f0 – 0 f0 – f0 +
2 2 2 2
x(a) ( f )
2N0

0 B f0 B f
f0 – f0 +
2 2
2N0 x(bb) ( f )

B 0 B f
–
2 2
N0 x (bb) ( f ), x (bb) ( f)
I Q

B 0 B f
–
2 2

Figure 1.23 Spectral representation of a PB process and its BB components.

depicted in Figure 1.23. It is immediate to get

( )
f − f0
x(a) (f ) = 2N0 rect (1.223)
B
and ( )
f
x(bb) (f ) = 2N0 rect (1.224)
B
Then ( )
1 f
x(bb) (f ) = x(bb) (f ) =  (bb) (f ) = N0 rect (1.225)
I Q 2 x B
Moreover, being x(bb) x(bb) (f ) = 0, we have that xI(bb) ⟂ xQ
(bb)
.
I Q

Cyclostationary processes
We have seen that, if x is a real passband WSS process, then its complex envelope is WSS, and x(bb) ⟂
x(bb)∗ . The converse is also true: if x(bb) is a WSS process and x(bb) ⟂ x(bb)∗ , then
x(t) = Re [x(bb) (t) ej2𝜋f0 t ] (1.226)
is WSS with PSD given by (1.205). If x(bb) is WSS, however, with
𝚛x(bb) x(bb)∗ (𝜏) ≠ 0 (1.227)
1.5. Second-order analysis of random processes 39

observing (1.226) we find that the autocorrelation of x is a periodic function in t of period 1∕f0 :
1[ ]
𝚛x (t, t − 𝜏) = 𝚛x(bb) (𝜏)ej2𝜋f0 𝜏 + 𝚛∗x(bb) (𝜏)e−j2𝜋f0 𝜏 + 𝚛x(bb) x(bb)∗ (𝜏)e−j2𝜋f0 𝜏 ej4𝜋f0 t + 𝚛∗x(bb) x(bb)∗ (𝜏)ej2𝜋f0 𝜏 e−j4𝜋f0 t
4
(1.228)
In other words, x is a cyclostationary process of period T0 = 1∕f0 .13
In this case, it is convenient to introduce the average correlation
T0
1
𝚛x (𝜏) = 𝚛x (t, t − 𝜏)dt (1.229)
T0 ∫ 0
whose Fourier transform is the average PSD
T0
1
 x (f ) =  [𝚛x (𝜏)] = x (f , t)dt (1.230)
T0 ∫0
where
x (f , t) = 𝜏 [𝚛x (t, t − 𝜏)] (1.231)
In (1.231), 𝜏 denotes the Fourier transform with respect to the variable 𝜏. In our case, it is
1
 x (f ) = [ (bb) (f − f0 ) + x(bb) (−f − f0 )] (1.232)
4 x
as in the stationary case (1.205).

Example 1.5.3
Let x be a modulated DSB signal (see (1.123)), i.e.
x(t) = a(t) cos(2𝜋f0 t + 𝜑0 ) (1.233)
with a real random BB WSS process with bandwidth Ba < f0 and autocorrelation 𝚛a (𝜏). From (1.126) it
results x(bb) (t) = a(t) ej𝜑0 . Hence, we have
𝚛x(bb) (𝜏) = 𝚛a (𝜏), 𝚛x(bb) x(bb)∗ (𝜏) = 𝚛a (𝜏) ej2𝜑0 (1.234)
Because 𝚛a (𝜏) is not identically zero, observing (1.227) we find that x is cyclostationary with period
1∕f0 . From (1.232), the average PSD of x is given by
1
 x (f ) =
[ (f − f0 ) + a (f + f0 )] (1.235)
4 a
Therefore, x has a bandwidth equal to 2Ba and an average statistical power
1
𝙼 𝙼x = (1.236)
2 a
We note that one finds the same result (1.235) assuming that 𝜑0 is a uniform r.v. in [0, 2𝜋); in this case
x turns out to be WSS.

Example 1.5.4
Let x be a modulated single sideband (SSB) with an upper sideband, i.e.
[ ( ) ]
1
x(t) = Re a(t) + ja(h) (t) ej(2𝜋f0 t+𝜑0 )
2
1 1
= a(t) cos(2𝜋f0 t + 𝜑0 ) − a(h) (t) sin(2𝜋f0 t + 𝜑0 ) (1.237)
2 2
13
To be precise, x is cyclostationary in mean value with period T0 = 1∕f0 , while it is cyclostationary in correlation with
period T0 ∕2.
40 Chapter 1. Elements of signal theory

PBF LPF
r(t) ro (t) r (bb)(t) 1 (bb) ro (t)
hPB h h h Re[.]
2 PB *

cos(2πf0t + φ0) 1 e–j φ0

2
(a) (b)

Figure 1.24 Coherent DSB demodulator and baseband-equivalent scheme. (a) Coherent DSB
demodulator and (b) baseband-equivalent scheme.

where a(h) is the Hilbert transform of a, a real WSS random process with autocorrelation 𝚛a (𝜏) and
bandwidth Ba .
We note that the modulating signal (a(t) + ja(h) (t)) coincides with the analytic signal a(a) and its spec-
tral support contains only positive frequencies.
Being
1
x(bb) (t) = (a(t) + ja(h) (t))ej𝜑0
2
it results that x(bb) and x(bb)∗ have non-overlapping passbands and
𝚛x(bb) x(bb)∗ (𝜏) = 0 (1.238)
The process (1.237) is then stationary with
1
x (f ) = [ (+) (f − f0 ) + a(+) (−f − f0 )] (1.239)
4 a
where a(+) is defined in (1.195). In this case, x has bandwidth equal to Ba and statistical power given by
1
𝙼x = 𝙼 (1.240)
4 a

Example 1.5.5 (DSB and SSB demodulators)

Let the signal r be the sum of a desired part x and additive white noise 𝑤 with PSD equal to 𝑤 (f ) =
N0 ∕2,
r(t) = x(t) + 𝑤(t) (1.241)
where the signal x is modulated DSB (1.233). To obtain the signal a from r, one can use the coherent
demodulation scheme illustrated in Figure 1.24 (see Figure 1.21b), where h is an ideal lowpass filter,
having a frequency response ( )
f
(f ) = 0 rect (1.242)
2Ba
Let ro be the output signal of the demodulator, given by the sum of the desired part xo and noise 𝑤o :
ro (t) = xo (t) + 𝑤o (t) (1.243)
We evaluate now the ratio between the powers of the signals in (1.243),
𝙼xo
Λo = (1.244)
𝙼𝑤o
in terms of the reference signal-to-noise ratio
𝙼x
Γ= (1.245)
(N0 ∕2) 2Ba
1.5. Second-order analysis of random processes 41

Using the equivalent block scheme of Figure 1.24 and (1.126), we have
r(bb) (t) = a(t) ej𝜑0 + 𝑤(bb) (t) (1.246)
with 𝑤(bb) (f ) = 2N0 1(f + f0 ). Being
h ∗ a(t) = 0 a(t) (1.247)
it results [ ]
1
xo (t) = Re h ∗ e−j𝜑1 a ej𝜑0 (t)
2
(1.248)
0
= a(t) cos(𝜑0 − 𝜑1 )
2
Hence, we get
02
𝙼 cos2 (𝜑0 − 𝜑1 )
𝙼xo = (1.249)
4 a
In the same baseband equivalent scheme, we consider the noise 𝑤eq at the output of filter h; we find
1
𝑤eq (f ) = |(f )|2 2N0 1(f + f0 )
4
( )
2 f
= 0 N0 rect (1.250)
2 2Ba
Being now 𝑤 WSS, 𝑤(bb) is uncorrelated with 𝑤(bb)∗ and thus 𝑤eq with 𝑤∗eq . Then, from
𝑤o (t) = 𝑤eq,I (t) (1.251)
and using (1.209) it follows
( )
02 f
𝑤0 (f ) = N0 rect (1.252)
4 2Ba
and
02
𝙼𝑤0 = N0 2Ba (1.253)
4
In conclusion, using (1.236), we have
(02 ∕4) 𝙼a cos2 (𝜑0 − 𝜑1 )
Λo = = Γcos2 (𝜑0 − 𝜑1 ) (1.254)
(02 ∕4) N0 2Ba
For 𝜑1 = 𝜑0 (1.254) becomes
Λo = Γ (1.255)
It is interesting to observe that, at the demodulator input, the ratio between the power of the desired
signal and the power of the noise in the passband of x is given by
𝙼x Γ
Λi = = (1.256)
(N0 ∕2) 4Ba 2
For 𝜑1 = 𝜑0 then
Λo = 2Λi (1.257)
We will now analyse the case of a SSB signal x (see (1.237)), coherently demodulated, following the
scheme of Figure 1.25, where hPB is a filter used to eliminate the noise that otherwise, after the mixer,
would have fallen within the passband of the desired signal. The ideal frequency response of hPB is
given by ( ) ( )
f − f0 − Ba ∕2 −f − f0 − Ba ∕2
PB (f ) = rect + rect (1.258)
Ba Ba
42 Chapter 1. Elements of signal theory

PBF LPF
r(t) ro (t) r(bb)(t) 1 (bb) ro (t)
hPB h h h Re[.]
2 PB *
1 e–j φ1
cos(2 πf0t + φ0) 2
(a) (b)

Figure 1.25 (a) Coherent SSB demodulator and (b) baseband-equivalent scheme.

Note that in this scheme, we have assumed the phase of the receiver carrier equal to that of the transmitter,
to avoid distortion of the desired signal.
Being ( )
(bb) f − Ba ∕2
PB (f ) = 2 rect (1.259)
Ba
the filter of the baseband-equivalent scheme is given by
1 (bb)
heq (t) = h ∗ h(t) (1.260)
2 PB
with frequency response ( )
f − Ba ∕2
eq (f ) = 0 rect (1.261)
Ba
We now evaluate the desired component xo . Using the fact x(bb) ∗ heq (t) = 0 x(bb) (t), it results
[ ]
1 1
xo (t) = Re heq ∗ e−j𝜑0 (a + j a(h) ) ej𝜑0 (t)
2 2
0 [ ] 
= Re a(t) + j a (t) = 0 a(t)
(h)
(1.262)
4 4
In the baseband-equivalent scheme, the noise 𝑤eq at the output of heq has a PSD given by
( )
1 N f − Ba ∕2
𝑤eq (f ) = |eq (f )|2 2N0 1(f + f0 ) = 0 02 rect (1.263)
4 2 Ba
From the relation 𝑤o = 𝑤eq,I and using (1.209), which is valid because 𝑤eq ⟂ 𝑤∗eq , we have
( )
1 2 f
𝑤o (f ) = [𝑤eq (f ) + 𝑤eq (−f )] = 0 N0 rect (1.264)
4 8 2Ba
and
02
𝙼𝑤o = N0 2Ba (1.265)
8
Then we obtain
(02 ∕16) 𝙼a
Λo = (1.266)
(02 ∕8) N0 2Ba
which using (1.240) and (1.245) can be written as
Λo = Γ (1.267)
We note that the SSB system yields the same performance (for 𝜑1 = 𝜑0 ) of a DSB system, even though
half of the bandwidth is required. Finally, it results
𝙼x
Λi = = Λo (1.268)
(N0 ∕2) 2Ba
1.6. The autocorrelation matrix 43

Observation 1.4
We note that also for the simple examples considered in this section, the desired signal is analysed
via the various transformations, whereas the noise is analysed via the PSD. As a matter of fact, we are
typically interested only in the statistical power of the noise at the system output. The demodulated signal
xo , on the other hand, must be expressed as the sum of a desired component proportional to a and an
orthogonal component that represents the distortion, which is, typically, small and has the same effects
as noise.

In the previous example, the considered systems do not introduce any distortion since xo is proportional
to a.

1.6 The autocorrelation matrix

Definition 1.14
Given the discrete-time wide-sense stationary random process {x(k)}, we introduce the random vector
with N components
xT (k) = [x(k), x(k − 1), … , x(k − N + 1)] (1.269)
The N × N autocorrelation matrix of x∗ (k) is given by
⎡ 𝚛x (0) 𝚛x (−1) … 𝚛x (−N + 1) ⎤
⎢ ⎥
⎢ 𝚛x (1) 𝚛x (0) … 𝚛x (−N + 2) ⎥
R = E[x∗ (k)xT (k)] = ⎢ ⎥. (1.270)
⎢ ⋮ ⋮ ⋱ ... ⎥
⎢ ⎥
⎢ ⎥
⎣ 𝚛x (N − 1) 𝚛x (N − 2) … 𝚛x (0) ⎦ ◽

Properties
1. R is Hermitian: R H = R. For real random processes R is symmetric: R T = R.
2. R is a Toeplitz matrix, i.e. all elements along any diagonal are equal.
3. R is positive semi-definite and almost always positive definite. Indeed, taking an arbitrary vector
𝒗T = [𝑣0 , … , 𝑣N−1 ], and letting yk = xT (k)𝒗, we have
∑∑
N−1 N−1
E[|yk |2 ] = E[𝒗H x∗ (k)xT (k)𝒗] = 𝒗H R𝒗 = 𝑣∗i 𝚛x (i − j)𝑣j ≥ 0 (1.271)
i=0 j=0

If 𝒗H R𝒗 > 0, ∀𝒗 ≠ 𝟎, then R is said to be positive definite and all its principal minor determinants
are positive; in particular R is non-singular.

Eigenvalues
We indicate with det R the determinant of a matrix R. The eigenvalues of R are the solutions 𝜆i , i =
1, … , N, of the characteristic equation of order N
det[R − 𝜆I] = 0 (1.272)
and the corresponding column eigenvectors ui , i = 1, … , N, satisfy the equation
Rui = 𝜆i ui (1.273)
44 Chapter 1. Elements of signal theory

Table 1.7: Correspondence between

eigenvalues and eigenvectors of four
matrices.

R Rm R−1 I − 𝜇R
Eigenvalue 𝜆i 𝜆mi 𝜆−1
i
(1 − 𝜇𝜆i )
Eigenvector ui ui ui ui

Example 1.6.1
Let {𝑤(k)} be a white noise process. Its autocorrelation matrix R assumes the form
⎡ 𝜎𝑤2 0 … 0 ⎤
⎢ ⎥
0 𝜎𝑤2 … 0 ⎥
R=⎢ (1.274)
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ 0 0 … 𝜎𝑤2 ⎥⎦
⎣
from which it follows that
𝜆1 = 𝜆2 = · · · = 𝜆N = 𝜎𝑤2 (1.275)
and
ui can be any arbitrary vector 1≤i≤N (1.276)

Example 1.6.2
We define a complex-valued sinusoid as
x(k) = ej(𝜔k+𝜑) , 𝜔 = 2𝜋f Tc (1.277)
with 𝜑 a uniform r.v. in [0, 2𝜋). The autocorrelation matrix R is given by
⎡ 1 e−j𝜔 … e−j(N−1)𝜔 ⎤
⎢ j𝜔 ⎥
e 1 … e−j(N−2)𝜔 ⎥
R=⎢ (1.278)
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ ej(N−1)𝜔 ej(N−2)𝜔 … 1 ⎥
⎣ ⎦
One can see that the rank of R is 1 and it will therefore have only one eigenvalue. The solution is given
by
𝜆1 = N (1.279)
and the relative eigenvector is
uT1 = [1, ej𝜔 , … , ej(N−1)𝜔 ] (1.280)

Other properties
1. From Rm u = 𝜆m u, we obtain the relations of Table 1.7.
2. If the eigenvalues are distinct, then the eigenvectors are linearly independent:
∑
N
ci ui ≠ 𝟎 (1.281)
i=1

for all combinations of {ci }, i = 1, 2, … , N, not all equal to zero. Therefore, in this case, the eigen-
vectors form a basis in ℝN .
1.6. The autocorrelation matrix 45

3. The trace of a matrix R is defined as the sum of the elements of the main diagonal, and we indicate
it with tr R. It holds
∑N
tr R = 𝜆i (1.282)
i=1

Eigenvalue analysis for Hermitian matrices

As previously seen, the autocorrelation matrix R is Hermitian, thus enjoys the following properties:
1. The eigenvalues of a Hermitian matrix are real.
By left multiplying both sides of (1.273) by uH
i
, it follows

i Rui = 𝜆i ui ui
uH H
(1.283)
from which, by the definition of norm, we obtain
uH
i
Rui uH
i
Rui
𝜆i = = (1.284)
uH u
i i
‖ui ‖2
The ratio (1.284) is defined as Rayleigh quotient. As R is positive semi-definite, uH i
Rui ≥ 0, from
which 𝜆i ≥ 0.
2. If the eigenvalues of R are distinct, then the eigenvectors are orthogonal. In fact, from (1.273), we
obtain:
uHi Ruj = 𝜆j ui uj
H
(1.285)

i Ruj = 𝜆i ui uj
uH H
(1.286)
Subtracting the second equation from the first:
0 = (𝜆j − 𝜆i )uH
i uj (1.287)
and since 𝜆j − 𝜆i ≠ 0 by hypothesis, it follows uH u = 0.
i j
3. If the eigenvalues of R are distinct and their corresponding eigenvectors are normalized, i.e.
{
1 i=j
‖ui ‖ = ui ui =
2 H
(1.288)
0 i≠j
then the matrix U = [u1 , u2 , … , uN ], whose columns are the eigenvectors of R, is a unitary matrix,
that is
U−1 = UH (1.289)
This property is an immediate consequence of the orthogonality of the eigenvectors {ui }. More-
over, if we define the matrix
⎡ 𝜆1 0 … 0 ⎤
⎢ ⎥
0 𝜆2 … 0 ⎥
𝚲=⎢ (1.290)
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ 0 0 … 𝜆 ⎥
⎣ N ⎦

we get
UH RU = 𝚲 (1.291)
From (1.291), we obtain the following important relations:
∑
N
R = U𝚲UH = 𝜆i ui uH
i (1.292)
i=1
46 Chapter 1. Elements of signal theory

and
∑
N
I − 𝜇R = U(I − 𝜇𝚲)UH = (1 − 𝜇𝜆i )ui uH
i (1.293)
i=1

4. The eigenvalues of a positive semi-definite autocorrelation matrix R and the PSD of x are related
by the inequalities,
min{x (f )} ≤ 𝜆i ≤ max{x (f )}, i = 1, … , N (1.294)
f f

In fact, let Ui (f ) be the Fourier transform of the sequence represented by the elements of ui , i.e.
∑
N
Ui (f ) = ui,n e−j2𝜋fnTc (1.295)
n=1

where ui,n is the n-th element of the eigenvector ui . Observing that

∑ ∑
N N
uH
i Rui = u∗i,n 𝚛x (n − m)ui,m (1.296)
n=1 m=1

and using (1.172) and (1.295), we have

∑ ∑
1 N N
2Tc
uH
i Rui = x (f ) u∗i,n ej2𝜋fnTc ui,m e−j2𝜋fmTc df
∫− 1
n=1 m=1
2Tc
1
2Tc
= x (f ) |Ui (f )|2 df (1.297)
∫− 1
2Tc

Substituting the latter result in (1.284) one finds

1
2Tc
x (f ) |Ui (f )|2 df
∫− 1
2Tc
𝜆i = 1
(1.298)
2Tc
|Ui (f )|2 df
∫− 1
2Tc

from which (1.294) follows.

If we indicate with 𝜆min and 𝜆max , respectively, the minimum and maximum eigenvalue of R, in view
of the latter point, we can define the eigenvalue spread as:
𝜆max maxf {x (f )}
𝜒(R) = ≤ (1.299)
𝜆min minf {x (f )}
From (1.299), we observe that 𝜒(R) may assume large values in the case x (f ) exhibits large variations.
Moreover, 𝜒(R) assumes the minimum value of 1 for a white process.

1.7 Examples of random processes

Before reviewing some important random processes, we recall the definition of Gaussian complex-
valued random vector.
1.7. Examples of random processes 47

Example 1.7.1
A complex r.v. with a Gaussian distribution can be generated from two r.v.s. with uniform distribution
(see Appendix 1.B for an illustration of the method).

Example 1.7.2
Let xT = [x1 , … , xN ] be a real Gaussian random vector, each component has mean 𝚖xi and variance 𝜎x2i ,
denoted as xi ∼  (𝚖xi , 𝜎x2i ). The joint probability density function (pdf) is
1 1 T C−1 (𝝃−m )
px (𝝃) = [(2𝜋)N det CN ]− 2 e− 2 (𝝃−mx ) N x
(1.300)
where = [𝜉1 , … , 𝜉N ], mx = E[x] is the vector of its components’ mean values and
𝝃T
CN = E[(x − mx )(x − mx )T ] is its covariance matrix.

Example 1.7.3
Let xT = [x1,I + jx1,Q , … , xN,I + jxN,Q ] be a complex-valued Gaussian random vector. If the in-phase
component xi,I and the quadrature component xi,Q are uncorrelated,
E[(xi,I − 𝚖xi,I )(xi,Q − 𝚖xi,Q )] = 0, i = 1, 2, … , N (1.301)
Moreover, we have
1 2
𝜎x2i,I = 𝜎x2i,Q = 𝜎 (1.302)
2 xi
then the joint pdf is
H C−1 (𝝃−m )
px (𝝃) = [𝜋 N det CN ]−1 e−(𝝃−mx ) N x (1.303)
with the vector of mean values and the covariance matrix given by
mx = E[x] = E[xI ] + jE[xQ ] (1.304)
H
CN = E[(x − mx )(x − mx ) ] (1.305)
Vector x is called circularly symmetric Gaussian random vector. For the generic component, we write
xi ∼  (𝚖xi , 𝜎x2i ) and
|𝜉i,I −𝚖x |2 |𝜉i,Q −𝚖x |2
i,I i,Q
− −
1 2𝜎x2 1 2𝜎x2
pxi (𝜉i ) = √ e i,I
√ e i,Q (1.306)
2𝜋𝜎x2i,I 2𝜋𝜎x2i,Q
|𝜉i −𝚖x |2
i
1 − 𝜎x2
= e i (1.307)
𝜋𝜎x2i
with 𝜉i = 𝜉i,I + j𝜉i,Q complex valued.

Example 1.7.4
Let xT = [x1 , … , xN ] = [x1 (t1 ), … , xN (tN )] be a complex-valued Gaussian (vector) process, with each
element xi (ti ) having real and imaginary components that are uncorrelated Gaussian r.v.s. whose pdf
is with zero mean and equal variance for all values of ti . The vector x is called circularly symmetric
Gaussian random process. The joint pdf in this case results
H C−1 𝝃
px (𝝃) = [𝜋 N det C]−1 e−𝝃 (1.308)
where C is the covariance matrix of [x1 (t1 ), x2 (t2 ), … , xN (tN )].
48 Chapter 1. Elements of signal theory

Example 1.7.5
Let x(t) = A sin(2𝜋ft + 𝜑) be a real-valued sinusoidal signal with 𝜑 r.v. uniform in [0, 2𝜋), for which we
will use the notation 𝜑 ∼  [0, 2𝜋). The mean of x is
𝚖x (t) = E[x(t)]
2𝜋
1
= A sin(2𝜋ft + a)da (1.309)
∫0 2𝜋
=0
and the autocorrelation function is given by
2𝜋
1
𝚛x (𝜏) = A sin(2𝜋ft + a)A sin[2𝜋f (t − 𝜏) + a]da
∫0 2𝜋
A2
= cos(2𝜋f 𝜏) (1.310)
2

Example 1.7.6
Consider the sum of N real-valued sinusoidal signals, i.e.
∑
N
x(t) = Ai sin(2𝜋fi t + 𝜑i ) (1.311)
i=1

with 𝜑i ∼  [0, 2𝜋) statistically independent, from Example 1.7.5 it is immediate to obtain the mean
∑
N
𝚖x (t) = 𝚖xi (t) = 0 (1.312)
i=1

and the autocorrelation function

∑
N
A2i
𝚛x (𝜏) = cos(2𝜋fi 𝜏) (1.313)
i=1
2

We note that, according to the Definition 1.9, page 31, the process (1.311) is not asymptotically uncor-
related.

Example 1.7.7
Consider the sum of N complex-valued sinusoidal signals, i.e.
∑
N
x(t) = Ai ej(2𝜋fi t+𝜑i ) (1.314)
i=1

with 𝜑i ∼  [0, 2𝜋) statistically independent. Following a similar procedure to that used in Examples
1.7.5 and 1.7.6, we find
∑
N
𝚛x (𝜏) = |Ai |2 ej2𝜋fi 𝜏 (1.315)
i=1

We note that the process (1.315) is not asymptotically uncorrelated.

1.7. Examples of random processes 49

x(k) y(t)
hTx
T

Figure 1.26 Modulator of a PAM system as interpolator filter.

Example 1.7.8
Let the discrete-time random process y(k) = x(k) + 𝑤(k) be given by the sum of the random process x
of Example 1.7.7 and white noise 𝑤 with variance 𝜎𝑤2 . Moreover, we assume x and 𝑤 uncorrelated. In
this case,
∑
N
𝚛y (n) = |Ai |2 ej2𝜋fi nTc + 𝜎𝑤2 𝛿n (1.316)
i=1

Example 1.7.9
We consider a signal obtained by pulse-amplitude modulation (PAM), expressed as
∑
+∞
y(t) = x(k) hTx (t − kT) (1.317)
k=−∞

The signal y is the output of the system shown in Figure 1.26, where hTx is a finite-energy pulse and
{x(k)} is a discrete-time (with T-spaced samples) WSS sequence, having PSD x (f ). We note that x (f )
is a periodic function of period 1∕T.
Let the deterministic autocorrelation of the signal hTx be
+∞
𝚛hTx (𝜏) = ∗
hTx (t)hTx (t − 𝜏)dt = [hTx (t) ∗ hTx
∗
(−t)](𝜏) (1.318)
∫−∞
with Fourier transform |Tx (f )|2 . In general, y is a cyclostationary process of period T. In fact, we have
1. Mean
∑
+∞
𝚖y (t) = 𝚖x hTx (t − kT) (1.319)
k=−∞

2. Correlation
∑
+∞
∑
+∞
𝚛y (t, t − 𝜏) = 𝚛x (i) hTx (t − (i + m) T)h∗Tx (t − 𝜏 − mT) (1.320)
i=−∞ m=−∞

If we introduce the average spectral analysis

T
− 1
𝚖y = 𝚖 (t) dt = 𝚖x Tx (0) (1.321)
T ∫0 y
1 ∑
T +∞
1
𝚛y (𝜏) = 𝚛y (t, t − 𝜏)dt = 𝚛 (i)𝚛hTx (𝜏 − iT) (1.322)
T ∫0 T i=−∞ x
and
|1 |2
 y (f ) =  [𝚛y (𝜏)] = || Tx (f )|| x (f ) (1.323)
|T |
we observe that the modulator of a PAM system may be regarded as an interpolator filter with
frequency response Tx ∕T.
50 Chapter 1. Elements of signal theory

3. Average power for a white noise input For a white noise input with power 𝙼x , from (1.322), the
average statistical power of the output signal is given by
Eh
𝙼y = 𝙼x (1.324)
T
+∞
where Eh = ∫−∞ |hTx (t)|2 dt is the energy of hTx .
4. Moments of y for a circularly symmetric i.i.d. input
Let {x(k)} be a complex-valued random circularly symmetric sequence with zero mean (see (1.301)
and (1.302)), i.e. letting
xI (k) = Re[x(k)], xQ (k) = Im[x(k)] (1.325)
we have
E[|x(k)|2 ]
E[xI2 (k)] = E[xQ
2
(k)] = (1.326)
2
and
E[xI (k) xQ (k)] = 0 (1.327)
These two relations can be merged into the single expression
E[x2 (k)] = E[xI2 (k)] − E[xQ
2
(k)] + 2j E[xI (k) xQ (k)] = 0 (1.328)
Filtering the i.i.d. input signal {x(k)} by using the system depicted in Figure 1.26, and from the
relation
∑
+∞
∑
+∞
𝚛yy∗ (t, t − 𝜏) = 𝚛xx∗ (i) hTx (t − (i + m)T)hTx (t − 𝜏 − mT) (1.329)
i=−∞ m=−∞

we have
𝚛xx∗ (i) = E[x2 (k)]𝛿(i) = 0 (1.330)
and
𝚛yy∗ (t, t − 𝜏) = 0 (1.331)
that is y ⟂ y∗ . In particular, we have that y is circularly symmetric, i.e.
E[y2 (t)] = 0 (1.332)
We note that the condition (1.331) can be obtained assuming the less stringent condition that x ⟂ x∗ ;
on the other hand, this requires that the following two conditions are verified
𝚛xI (i) = 𝚛xQ (i) (1.333)
and
𝚛xI xQ (i) = −𝚛xI xQ (−i) (1.334)

Observation 1.5
It can be shown that if the filter hTx has a bandwidth smaller than 1∕(2T) and {x(k)} is a WSS sequence,
then {y(k)} is WSS with PSD (1.323).

Example 1.7.10
Let us consider a PAM signal sampled with period TQ = T∕Q0 , where Q0 is a positive integer number.
Let
yq = y(q TQ ), hp = hTx (p TQ ) (1.335)
1.7. Examples of random processes 51

from (1.317) it follows

∑
+∞
yq = x(k) hq−kQ0 (1.336)
k=−∞

If Q0 ≠ 1, (1.336) describes the input–output relation of an interpolator filter (see (1.536)). We recall the
statistical analysis given in Table 1.6, page 34. We denote with (f ) the Fourier transform (see (1.17))
and with 𝚛h (n) the deterministic autocorrelation (see (1.184)) of the sequence {hp }. We also assume
that {x(k)} is a WSS random sequence with mean 𝚖x and autocorrelation 𝚛x (n). In general, {yq } is a
cyclostationary random sequence of period Q0 with

1. Mean
∑
+∞
𝚖y (q) = 𝚖x hq−kQ0 (1.337)
k=−∞

2. Correlation
∑
+∞
∑
+∞
𝚛y (q, q − n) = 𝚛x (i) hq−(i+m)Q0 h∗q−n−m Q0 (1.338)
i=−∞ m=−∞

By the average spectral analysis, we obtain

1 ∑
Q0 −1
(0)
𝚖y = 𝚖y (q) = 𝚖x (1.339)
Q0 q=0 Q0

where
∑
+∞
(0) = hp (1.340)
p=−∞

and
1 ∑ 1 ∑
Q0 −1 +∞
𝚛y (n) = 𝚛y (q, q − n) = 𝚛 (i) 𝚛h (n − iQ0 ) (1.341)
Q0 q=0 Q0 i=−∞ x

Consequently, the average PSD is given by

| 1 |2
| |
 y (f ) = TQ  [𝚛y (n)] = | (f )| x (f ) (1.342)
| Q0 |
| |
If {x(k)} is white noise with power 𝙼x , from (1.341) it results

𝚛h (n)
𝚛y (n) = 𝙼x (1.343)
Q0

In particular, the average power of the filter output signal is given by

Eh
𝙼y = 𝙼x (1.344)
Q0
∑+∞
where Eh = p=−∞ |hp |2 is the energy of {hp }. We point out that the condition 𝙼y = 𝙼x is satisfied if
the energy of the filter impulse response is equal to the interpolation factor Q0 .
52 Chapter 1. Elements of signal theory

t0
x(t) = g(t) + w(t) y(t) y (t0) = gu (t0) + wu (t0)
gM

*( f )
M ( f ) = K e–j2π ft0
w(f)

Figure 1.27 Reference scheme for the matched filter.

1.8 Matched filter

Referring to Figure 1.27, we consider a finite-energy signal pulse g in the presence of additive noise 𝑤
having zero mean and PSD 𝑤 . The signal
x(t) = g(t) + 𝑤(t) (1.345)
is filtered with a filter having impulse response gM . We indicate with gu and 𝑤u , respectively, the desired
signal and the noise component at the output:
gu (t) = gM ∗ g(t) (1.346)
𝑤u (t) = gM ∗ 𝑤(t) (1.347)
The output is
y(t) = gu (t) + 𝑤u (t) (1.348)
We now suppose that y is observed at a given instant t0 . The problem is to determine gM so that the ratio
between the square amplitude of gu (t0 ) and the power of the noise component 𝑤u (t0 ) is maximum, i.e.
|gu (t0 )|2
gM ∶ max (1.349)
gM E[|𝑤u (t0 )|2 ]
The optimum filter has frequency response
∗ (f ) −j2𝜋f t0
M (f ) = K e (1.350)
𝑤 (f )
where K is a constant. In other words, the best filter selects the frequency components of the desired
input signal and weights them with weights that are inversely proportional to the noise level.

Proof. gu (t0 ) coincides with the inverse Fourier transform of M (f )(f ) evaluated in t = t0 , while the
power of 𝑤u (t0 ) is equal to
+∞
𝚛𝑤u (0) = 𝑤 (f )|M (f )|2 df (1.351)
∫−∞
Then we have
| +∞ |2
| |
| M (f )(f )ej2𝜋f t0 df |
|gu (t0 )|2 |∫−∞ |
= | |
𝚛𝑤u (0) +∞
 (f )|M (f )|2 df
∫−∞ 𝑤
| +∞ √ |2
| (f ) j2𝜋f t0 ||
| M (f ) 𝑤 (f ) √ e df |
|∫
| −∞ 𝑤 (f ) |
= | | (1.352)
+∞
𝑤 (f )|M (f )| df
2
∫−∞
1.8. Matched filter 53

√
where the integrand at the numerator was divided and multiplied by 𝑤 (f ). Implicitly, it is assumed
that 𝑤 (f ) ≠ 0. Applying the Schwarz inequality14 to the functions
√
M (f ) 𝑤 (f ) (1.355)
and
∗ (f ) −j2𝜋f t0
√ e (1.356)
𝑤 (f )
it turns out
+∞ || |2 +∞ || |2
|gu (t0 )|2
| (f ) j2𝜋f t0 || | (f ) ||
≤ √ e | df = ∫ |√ | df (1.357)
𝚛𝑤u (0) ∫−∞ ||  (f ) | −∞ | |
| 𝑤 | | 𝑤 (f ) |
Therefore, the maximum value is equal to the right-hand side of (1.357) and is achieved for
√ ∗ (f ) −j2𝜋f t0
M (f ) 𝑤 (f ) = K √ e (1.358)
𝑤 (f )
where K is a constant. From (1.358), the solution (1.350) follows immediately. ◽

White noise case

If 𝑤 is white, then 𝑤 (f ) = 𝑤 is a constant and the optimum solution (1.350) becomes
M (f ) = K∗ (f )e−j2𝜋f t0 (1.359)
Correspondingly, the filter has impulse response
gM (t) = Kg∗ (t0 − t) (1.360)
from which the name of matched filter (MF), i.e. matched to the input signal pulse. The desired signal
pulse at the filter output has the frequency response
u (f ) = K|(f )|2 e−j2𝜋f t0 (1.361)
From the definition of the autocorrelation function of pulse g,
+∞
𝚛g (𝜏) = g(a)g∗ (a − 𝜏)da (1.362)
∫−∞
then, as depicted in Figure 1.28,
gu (t) = K𝚛g (t − t0 ) (1.363)
i.e. the pulse at the filter output coincides with the autocorrelation function of the pulse g. If Eg is the
energy of g, using the relation Eg = 𝚛g (0) the maximum of the functional (1.349) becomes

|gu (t0 )|2 |K|2 𝚛2g (0) Eg

= = (1.364)
𝚛𝑤u (0) 𝑤 |K| 𝚛g (0) 𝑤
2

14
Given two signals x and y it holds
| ∞ |2 ∞ ∞
| |
|∫ x(t)y (t)dt| ≤ ∫ |x(t)| dt ∫ |y(t)| dt
∗ 2 2
(1.353)
| −∞ | −∞ −∞

where equality holds if and only if

y(t) = Kx(t) (1.354)
with K a complex constant.
54 Chapter 1. Elements of signal theory

t0
x(t) = g(t) + w(t) y(t) = Krg (t – t0) + wu (t) y(t0)
gM

gM (t) = Kg*(t0 – t)

Figure 1.28 Matched filter for an input pulse in the presence of white noise.

g(t)

0 tg t

gM (t)
t0 = 0

–tg 0 t

gM (t)
t0 = tg

0 tg t

rg (t)

–tg 0 tg t

Figure 1.29 Various pulse shapes related to a matched filter.

In Figure 1.29, the different pulse shapes are illustrated for a signal pulse g with limited duration tg .
Note that in this case, the matched filter has also limited duration, and it is causal if t0 ≥ tg .

Example 1.8.1 (MF for a rectangular pulse)

Let ( )
t − T∕2
g(t) = 𝚠T (t) = rect (1.365)
T
with ( ) ( )
|𝜏| 𝜏
𝚛g (𝜏) = T 1 − rect (1.366)
T 2T
1.9. Ergodic random processes 55

For t0 = T, the matched filter is proportional to g

gM (t) = K𝚠T (t) (1.367)

and the output pulse in the absence of noise is equal to

( ) ( )
|t − T |
gu (t) = KT 1 − || | rect t − T (1.368)
|
| T | 2T

1.9 Ergodic random processes

The functions that have been introduced in the previous sections for the analysis of random processes
give a valid statistical description of an ensemble of realizations of a random process. We investigate now
the possibility of moving from ensemble averaging to time averaging, that is we consider the problem
of estimating a statistical descriptor of a random process from the observation of a single realization.
Let x be a discrete-time WSS random process having mean 𝚖x . If in the limit it holds15

1 ∑
K−1
lim x(k) = E[x(k)] = 𝚖x (1.369)
K→∞ K
k=0

then x is said to be ergodic in the mean. In other words, for when the above limit holds, the time-average
of samples tends to the statistical mean as the number of samples increases. We note that the existence
of the limit (1.369) implies the condition
⎡|| K−1 |2 ⎤
⎢ | 1 ∑ |
lim E x(k) − 𝚖x || ⎥ = 0 (1.370)
K→∞ ⎢|| K | ⎥⎦
⎣| k=0 |
or equivalently
[ ]
1 ∑
K−1
|n|
lim 1− 𝚌x (n) = 0 (1.371)
K→∞ K K
n=−(K−1)

From (1.371), we see that for a random process to be ergodic in the mean, some conditions on the
second-order statistics must be verified. Analogously to definition (1.369), we say that x is ergodic in
correlation if in the limits it holds:

1 ∑
K−1
lim x(k)x∗ (k − n) = E[x(k)x∗ (k − n)] = 𝚛x (n) (1.372)
K→∞ K
k=0

Also for processes that are ergodic in correlation, one could get a condition of ergodicity similar to
that expressed by the limit (1.371). Let y(k) = x(k)x∗ (k − n). Observing (1.372) and (1.369), we find that
the ergodicity in correlation of the process x is equivalent to the ergodicity in the mean of the process y.
Therefore, it is easy to deduce that the condition (1.371) for y translates into a condition on the statistical
moments of the fourth order for x.
In practice, we will assume all stationary processes to be ergodic; ergodicity is however difficult
to prove for non-Gaussian random processes. We will not consider particular processes that are not
ergodic such as x(k) = A, where A is a random variable, or x(k) equal to the sum of sinusoidal signals
(see (1.311)).
( ∑ )
1 K−1
15
The limit is meant in the mean square sense, that is the variance of the r.v. K k=0 x(k) − 𝚖x vanishes for K → ∞.
56 Chapter 1. Elements of signal theory

Signal
{x(k)}

K–1 K–1 2
1∑
x(n) = lim K x(k)x* (k–n) x(f ) = lim E 1 Tc ∑ x(k)e–j2 πfk Tc
K +∞ k= 0 K +∞ KTc k= 0

ACS  PSD
x(n) Fourier transform
x( f )
+∞
x ( f) = Tcn =∑–∞x(n)e–j2 πfn Tc

Figure 1.30 Relation between ergodic processes and their statistical description.

The property of ergodicity assumes a fundamental importance if we observe that from a single Real-
ization, it is possible to obtain an estimate of the autocorrelation function, and from this, the PSD.
Alternatively, one could prove that under the hypothesis16

∑
+∞
|n| 𝚛x (n) < ∞ (1.373)
n=−∞

the following limit holds:

⎡ | K−1 |2 ⎤
1 | ∑ |
lim E ⎢ |T
| x(k) e −j2𝜋fkTc | ⎥ =  (f )
| ⎥ (1.374)
K→∞ ⎢ KTc |
c x
⎣ | ⎦
| k=0 |
Then, exploiting the ergodicity of a WSS random process, one obtains the relations among the process
itself, its autocorrelation function, and PSD shown in Figure 1.30. We note how the direct computation of
the PSD, given by (1.374), makes use of a statistical ensemble of the Fourier transform of the process x,
while the indirect method via ACS makes use of a single realization.
If we let
̃KTc (f ) = Tc  [x(k) 𝚠K (k)] (1.375)

where 𝚠K is the rectangular window of length K (see (1.401)) and Td = KTc , (1.374) becomes

E[|̃Td (f )|2 ]
x (f ) = lim (1.376)
Td →∞ Td

The relation (1.376) holds also for continuous-time ergodic random processes, where ̃Td (f ) denotes
the Fourier transform of the windowed realization of the process, with a rectangular window of
duration Td .

16
We note that for random processes with non-zero mean and/or sinusoidal components this property is not verified.
Therefore, it is usually recommended that the deterministic components of the process be removed before the spectral
estimation is performed.
1.9. Ergodic random processes 57

1.9.1 Mean value estimators

Given the random process {x(k)}, we wish to estimate the mean value of a related process {y(k)}: for
example to estimate the statistical power of x we set y(k) = |x(k)|2 , while for the estimation of the cor-
relation of x with lag n, we set y(k) = x(k)x∗ (k − n). Based on a realization of {y(k)}, from (1.369) an
estimate of the mean value of y is given by the expression

1 ∑
K−1
𝚖̂ y = y(k) (1.377)
K k=0
In fact, (1.377) attempts to determine the average component of the signal {y(k)}. As illustrated in
Figure 1.31a, in general, we can think of extracting the average component of {y(k)} using an LPF
filter h having unit gain, i.e. (0) = 1, and suitable bandwidth B. Let K be the length of the impulse
response with support from k = 0 to k = K − 1. Note that for a unit step input signal, the transient part
of the output signal will last K − 1 time instants. Therefore, we assume
𝚖̂ y = z(k) = h ∗ y(k) for k ≥ K − 1 (1.378)
We now compute the mean and variance of the estimate. From (1.378), the mean value is given by
E[𝚖̂ y ] = 𝚖y (0) = 𝚖y (1.379)
as (0) = 1. Using the expression in Table 1.6 of the correlation of a filter output signal given the input,
the variance of the estimate is given by
∑
+∞
var[𝚖̂ y ] = 𝜎y2 = 𝚛h (−n)𝚌y (n) (1.380)
n=−∞
Assuming
∑
+∞
∑
+∞
|𝚌y (n)|
S= |𝚌y (n)| = 𝜎y2 <∞ (1.381)
n=−∞ n=−∞ 𝜎y2
and being |𝚛h (n)| ≤ 𝚛h (0), the variance in (1.380) is bounded by
var[𝚖̂ y ] ≤ Eh S (1.382)
where Eh = 𝚛h (0).
For an ideal lowpass filter, ( )
f 1
(f ) = rect , |f | < (1.383)
2B 2Tc
assuming as filter length K that of the principal lobe of {h(k)}, and neglecting a delay factor, it results
as Eh = 2B and K ≃ 1∕B. Introducing the criterion that for a good estimate, it must be
var [𝚖̂ y ] ≤ 𝜀 (1.384)
with 𝜀 ≪ |𝚖y |2 , from (1.382) it follows
𝜀
B≤ (1.385)
2S
and
2S
K≥ (1.386)
𝜀
In other words, from (1.381) and (1.386), for a fixed 𝜀, the length K of the filter impulse response
must be larger, or equivalently the bandwidth B must be smaller, to obtain estimates for those processes
{y(k)} that exhibit larger variance and/or larger correlation among samples. Because of their simple
implementation, two commonly used filters are the rectangular window and the exponential filter, whose
impulse responses are shown in Figure 1.31.
58 Chapter 1. Elements of signal theory

0.035

0.03

0.025

LPF 0.02
y(k) z(k) = m˄ y

h(k)
h 0.015
Tc Tc
0.01

0.005

0
0 5 10 15 20 25 30 35 40
k
(a) (b)
1.2

0.8

0.6
|H( f )|

0.4

0.2

−0.2
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
fTc
(c)

Figure 1.31 (a) Time average as output of a narrow band lowpass filter. (b) Typical impulse
responses: exponential filter with parameter a = 1 − 2−5 and rectangular window with K = 33.
(c) Corresponding frequency responses.

Rectangular window
For a rectangular window,
⎧ 1
⎪ k = 0, 1, … , K − 1
h(k) = ⎨ K (1.387)
⎪0 elsewhere
⎩
the frequency response is given by (see (1.24))
( )
K−1
−j2𝜋f Tc
(f ) = e 2 sincK (fKTc ) (1.388)
1.9. Ergodic random processes 59

We have Eh = 1∕K and, adopting as bandwidth the frequency of the first zero of |(f )|, B = 1∕(KTc ).
The filter output is given by
∑ 1
K−1
z(k) = y(k − n) (1.389)
n=0
K
that can be expressed as
y(k) − y(k − K)
z(k) = z(k − 1) + (1.390)
K

Exponential filter
For an exponential filter {
(1 − a)ak k≥0
h(k) = (1.391)
0 elsewhere
with |a| < 1, the frequency response is given by
1−a
(f ) = (1.392)
1 − ae−j2𝜋f Tc
Moreover, Eh = (1 − a)∕(1 + a) and, adopting as length of h the time constant of the filter, i.e. the inter-
val it takes for the amplitude of the impulse response to decrease of a factor e,
1 1
K−1= ≃ (1.393)
ln 1∕a 1 − a
where the approximation holds for a ≃ 1. The 3 dB filter bandwidth is equal to
1−a 1
B= for a > 0.9 (1.394)
2𝜋 Tc
The filter output has a simple expression given by the recursive equation
z(k) = az(k − 1) + (1 − a) y(k) (1.395)
We note that choosing a as
a = 1 − 2−l (1.396)
then (1.395) becomes
z(k) = z(k − 1) + 2−l (y(k) − z(k − 1)) (1.397)
whose computation requires only two additions and one shift of l bits. Moreover, from (1.393), the filter
time constant is given by
K − 1 = 2l (1.398)

General window
In addition to the two filters described above, a general window can be defined as
h(k) = A𝚠(k) (1.399)
with {𝚠(k)} window17 of length K. Factor A in (1.399) is introduced to normalize the area of h to 1.
We note that, for random processes with slowly time-varying statistics, (1.390) and (1.397) give an
expression to update the estimates.
17
We define the continuous-time rectangular window with duration Td as
( ) {
t − Td ∕2 1 0 < t < Td
𝚠Td (t) = rect = (1.400)
Td 0 elsewhere
Commonly used discrete-time windows are:
60 Chapter 1. Elements of signal theory

1.9.2 Correlation estimators

Let {x(k)}, k = 0, 1, … , K − 1, be a realization of a random process with K samples. We examine two
estimates.

Unbiased estimate
The unbiased estimate
1 ∑
K−1
𝚛̂ x (n) = x(k)x∗ (k − n) n = 0, 1, … , K − 1 (1.405)
K − n k=n
has mean
1 ∑
K−1
E[𝚛̂ x (n)] = E[x(k)x∗ (k − n)] = 𝚛x (n) (1.406)
K − n k=n
If the process is Gaussian, one can show that the variance of the estimate is approximately given by

K ∑
+∞
var[𝚛̂ x (n)] ≃ [𝚛2 (m) + 𝚛x (m + n)𝚛x (m − n)] (1.407)
(K − n)2 m=−∞ x
from which it follows
var [𝚛̂ x (n)] −−−−→ 0 (1.408)
K→∞

The above limit holds for n ≪ K. Note that the variance of the estimate increases with the correlation
lag n.

Biased estimate
The biased estimate
( )
1 ∑
K−1
|n|
𝚛̌ x (n) = x(k)x∗ (k − n) = 1− 𝚛̂ x (n) (1.409)
K k=n K

1. Rectangular window {
1 k = 0, 1, … , D − 1
𝚠(k) = 𝚠D (k) = (1.401)
0 elsewhere
where D denotes the length of the rectangular window expressed in number of samples.
2. Raised cosine or Hamming window
( )
⎧ k − D−1
⎪ 0.54 + 0.46 cos 2𝜋 2
k = 0, 1, … , D − 1
𝚠(k) = ⎨ D−1 (1.402)
⎪0
⎩ elsewhere

3. Hann window ( )
⎧ k − D−1
⎪ 0.50 + 0.50 cos 2𝜋 2
k = 0, 1, … , D − 1
𝚠(k) = ⎨ D−1 (1.403)
⎪0
⎩ elsewhere
4. Triangular or Bartlett window

⎧ | k − D−1 |
| 2 ||
⎪ 1 − 2 || | k = 0, 1, … , D − 1
𝚠(k) = ⎨ | D−1 | (1.404)
⎪ | |
⎩0 elsewhere
1.9. Ergodic random processes 61

has mean satisfying the following relations:

( )
|n|
E[𝚛̌ x (n)] = 1 − 𝚛x (n) −−−−→ 𝚛x (n) (1.410)
K K→∞

Unlike the unbiased estimate, the mean of the biased estimate is not equal to the autocorrelation function,
but approaches it as K increases. Note that the biased estimate differs from the autocorrelation function
by one additive constant, denoted as bias:

𝜇bias = E[𝚛̌ x (n)] − 𝚛x (n) (1.411)

For a Gaussian process, the variance of the biased estimate is

( )2
1 ∑ 2
+∞
K − |n|
var[𝚛̌ x (n)] = var[𝚛̂ x (n)] ≃ [𝚛 (m) + 𝚛x (m + n)𝚛x (m − n)] (1.412)
K K m=−∞ x

In general, the biased estimate of the ACS exhibits a mean-square error18 larger than the unbiased,
especially for large values of n. It should also be noted that the estimate does not necessarily yield
sequences that satisfy the properties of autocorrelation functions: for example the following property
may not be verified:
𝚛̂ x (0) ≥ |𝚛̂ x (n)|, n≠0 (1.414)

1.9.3 Power spectral density estimators

After examining ACS estimators, we review some spectral density estimation methods.

Periodogram or instantaneous spectrum

̃ ) = Tc (f ), where (f ) is the Fourier transform of {x(k)}, k = 0, … , K − 1; an estimate of the
Let (f
statistical power of {x(k)} is given by

1 ∑
K−1 1
1 ̃ )|2 df
𝙼̂ x =
2Tc
|x(k)|2 = |(f (1.415)
K k=0 KTc ∫− 1
2Tc

using the properties of the Fourier transform (Parseval theorem). Based on (1.415), a PSD estimator
called periodogram is given by
1 ̃
PER (f ) = |(f )|2 (1.416)
KTc
We can write (1.416) as
∑
K−1
PER (f ) = Tc 𝚛̌ x (n) e−j2𝜋fnTc (1.417)
n=−(K−1)

18
For example, for the estimator (1.405) the mean-square error is defined as
[ ]
E |𝚛̂ x (n) − 𝚛x (n)|2 = var[𝚛̂ x (n)] + |𝜇bias |2 (1.413)
62 Chapter 1. Elements of signal theory

and, consequently,
∑
K−1
E[PER (f )] = Tc E[𝚛̌ x (n)]e−j2𝜋fnTc
n=−(K−1)
( )
∑
K−1
|n|
= Tc 1− 𝚛x (n)e−j2𝜋fnTc (1.418)
n=−(K−1)
K
= Tc B ∗ x (f )
where B (f ) is the Fourier transform of the symmetric Bartlett window
{ |n|
1− |n| ≤ K − 1
𝚠B (n) = K (1.419)
0 |n| > K − 1
and
B (f ) = K [sincK (fkTc )]2 (1.420)
We note the periodogram estimate is affected by bias for finite K. Moreover, it also exhibits a large
variance, as PER (f ) is computed using the samples of 𝚛̌ x (n) even for lags up to K − 1, whose variance
is very large.

Welch periodogram
This method is based on applying (1.374) for finite K. Given a sequence of K samples, different subse-
quences of consecutive D samples are extracted. Subsequences may partially overlap. Let x(s) be the s-th
subsequence, characterized by S samples in common with the preceding subsequence x(s−1) and with
the following one x(s+1) . In general, 0 ≤ S ≤ D∕2, with the choice S = 0 yielding subsequences with no
overlap and therefore with less correlation. The number of subsequences Ns is19
⌊ ⌋
K−D
Ns = +1 (1.421)
D−S
Let 𝚠 be a window (see footnote 17 on page 59) of D samples: then
x(s) (k) = 𝚠(k) x(k + s(D − S)), k = 0, 1, … , D − 1 s = 0, 1, … , Ns − 1 (1.422)
For each s, compute the Fourier transform
∑
D−1
̃ (s) (f ) = Tc x(s) (k)e−j2𝜋fkTc (1.423)
k=0

and obtain
1
(s)
PER (f ) = |̃ (s) (f )|2 (1.424)
DTc 𝙼𝚠
where
1 ∑ 2
D−1
𝙼𝚠 = 𝚠 (k) (1.425)
D k=0
is the normalized energy of the window. As a last step, for each frequency, average the periodograms:

1 ∑ (s)
Ns −1
WE (f ) =  (f ) (1.426)
Ns s=0 PER

19
The symbol ⌊a⌋ denotes the function floor, that is the largest integer smaller than or equal to a. The symbol ⌈a⌉
denotes the function ceiling, that is the smallest integer larger than or equal to a.
1.9. Ergodic random processes 63

The mean of the estimate is given by

E[WE (f )] = Tc [||2 ∗ x ](f ) (1.427)
where
∑
D−1
(f ) = 𝚠(k)e−j2𝜋fkTc (1.428)
k=0

Assuming the process Gaussian and the different subsequences statistically independent, we get20
1 2
var[WE (f )] ∝  (f ) (1.429)
Ns x
Note that the partial overlap introduces correlation between subsequences. From (1.429), we see that
the variance of the estimate is reduced by increasing the number of subsequences. In general, D must
be large enough so that the generic subsequence represents the process21 and also Ns must be large to
obtain a reliable estimate (see (1.429)); therefore, the application of the Welch method requires many
samples.

Blackman and Tukey correlogram

For an unbiased estimate of the ACS, {𝚛̂ x (n)}, n = −L, … , L, consider the Fourier transform
∑
L
BT (f ) = Tc 𝚠(n)𝚛̂ x (n) e−j2𝜋fnTc (1.430)
n=−L

where 𝚠 is a window22 of length 2L + 1, with 𝚠(0) = 1. If K is the number of samples of the realization
sequence, we require that L ≤ K∕5 to reduce the variance of the estimate. Then if the Bartlett window
(1.420) is chosen, one finds that BT (f ) ≥ 0.
In terms of the mean value of the estimate, we find
E[BT (f )] = Tc ( ∗ x )(f ) (1.431)
For a Gaussian process, if the Bartlett window is chosen, the variance of the estimate is given by
1 2 2L 2
var[BT (f )] =  (f )E𝚠 =  (f ) (1.432)
K x 3K x

Windowing and window closing

The windowing operation of time sequence in the periodogram, and of the ACS in the correlogram, has
a strong effect on the performance of the estimate. In fact, any truncation of a sequence is equivalent to a
windowing operation, carried out via the rect function. The choice of the window type in the frequency
domain depends on the compromise between a narrow central lobe (to reduce smearing) and a fast decay
of secondary lobes (to reduce leakage). Smearing yields a lower spectral resolution, that is the capability
to distinguish two spectral lines that are close. On the other hand, leakage can mask spectral components
that are further apart and have different amplitudes.
The choice of the window length is based on the compromise between spectral resolution and the vari-
ance of the estimate. An example has already been seen in the correlogram, where the condition L ≤ K∕5
must be satisfied. Another example is the Welch periodogram. For a given observation of K samples, it is

20
Notation a ∝ b means that a is proportional to b.
21
For example, if x is a sinusoidal process, DTc must at least be greater than 5 or 10 periods of x.
22
The windows used in (1.430) are the same introduced in footnote 17: the only difference is that they are now centered
around zero instead of (D − 1)∕2. To simplify the notation, we will use the same symbol in both cases.
64 Chapter 1. Elements of signal theory

initially better to choose a small number of samples over which to perform the DFT, and therefore a large
number of windows (subsequences) over which to average the estimate. The estimate is then repeated
by increasing the number of samples per window, thus decreasing the number of windows. In this way,
we get estimates with not only a higher resolution but also characterized by an increasing variance. The
procedure is terminated once it is found that the increase in variance is no longer compensated by an
increase in the spectral resolution. The aforementioned method is called window closing.

Example 1.9.1
Consider a realization of K = 10 000 samples of the signal:

1 ∑
16
y(kTc ) = h(nTc )𝑤((k − n)Tc ) + A1 cos(2𝜋f1 kTc + 𝜑1 ) + A2 cos(2𝜋f2 kTc + 𝜑2 ) (1.433)
Ah n=−16
where 𝜑1 , 𝜑2 ∼  [0, 2𝜋), 𝑤(nTc ) is a white random process with zero mean and variance 𝜎𝑤2 = 5, Tc =
0.2, A1 = 1∕20, f1 = 1.5, A2 = 1∕40, f2 = 1.75, and
∑
16
Ah = h(kTc ) (1.434)
−16

Moreover
( ) ( )
kTc kTc kTc
sin 𝜋(1 − 𝜌) + 4𝜌 cos 𝜋(1 + 𝜌) ( )
T T T kTc
h(kTc ) = [ ( ) ] rect (1.435)
kT 2 kTc 8T + Tc
𝜋 1 − 4𝜌 c
T T
with T = 4Tc and 𝜌 = 0.32.
Actually y is the sum of two sinusoidal signals and filtered white noise through h. Consequently,
observing (1.188) and (1.313),
2
|(f )|2 A1
y (f ) = 𝜎𝑤2 Tc + (𝛿(f − f1 ) + 𝛿(f + f1 ))
A2h 4
A22
+ (𝛿(f − f2 ) + 𝛿(f + f2 )) (1.436)
4
where (f ) is the Fourier transform of {h(kTc )}.
The shape of the PSD in (1.436) is shown in Figures 1.32–1.34 as a solid line. A Dirac impulse is
represented by an isosceles triangle having a base equal to twice the desired frequency resolution Fq .
Consequently, a Dirac impulse, for example of area A21 ∕4 will have a height equal to A21 ∕(4Fq ), thus,
maintaining the equivalence in statistical power between different representations.
We now compare several spectral estimates, obtained using the previously described methods; in
particular, we will emphasize the effect on the resolution of the type of window used and the number of
samples for each window.
We state beforehand the following important result. Windowing a complex sinusoidal signal {ej2𝜋f1 kTc }
with {w(k)} produces a signal having Fourier transform equal to (f − f1 ), where (f ) is the Fourier
transform of 𝑤. Therefore, in the frequency domain, the spectral line of a sinusoidal signal becomes a
signal with shape (f ) centred around f1 .
In general, from (1.424), the periodogram of a real sinusoidal signal with amplitude A1 and frequency
f1 is
( )2
T A1
PER (f ) = c |(f − f1 ) + (f + f1 )|2 (1.437)
D𝙼𝚠 2
1.10. Parametric models of random processes 65

10

–10

–20
Amplitude (dB)

–30
rect
–40

–50
Hamming

–60

–70

–80
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
fTc

Figure 1.32 Comparison between spectral estimates obtained with Welch periodogram method,
using the Hamming or the rectangular window, and the analytical PSD given by (1.436).

Figure 1.32 shows, in addition to the analytical PSD (1.436), the estimate obtained by the Welch
periodogram method using the Hamming or the rectangular windows. Parameters used in (1.423) and
(1.426) are: D = 1000, Ns = 19, and 50% overlap between windows. We observe that the use of the
Hamming window yields an improvement of the estimate due to less leakage. Likewise Figure 1.33
shows how the Hamming window also improves the estimate carried out with the correlogram; in par-
ticular, the estimates of Figure 1.33 were obtained using in (1.430) L = 500. Lastly, Figure 1.34 shows
how the resolution and the variance of the estimate obtained by the Welch periodogram vary with the
parameters D and Ns , using the Hamming window. Note that by increasing D, and hence decreasing Ns ,
both resolution and variance of the estimate increase.

1.10 Parametric models of random processes

ARMA
Let us consider the realization of a random process x according to the auto-regressive moving average
(ARMA) model illustrated in Figure 1.35. In other words, the process x, also called observed sequence,
is the output of an IIR filter having as input white noise with variance 𝜎𝑤2 , and is given by the recursive
66 Chapter 1. Elements of signal theory

10

–10

–20 rect
Amplitude (dB)

–30
Hamming

–40

–50

–60

–70

–80
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
f Tc

Figure 1.33 Comparison between spectral estimates obtained with the correlogram using the
Hamming or the rectangular window, and the analytical PSD given by (1.436).

equation23
∑
p
∑
q
x(k) = − 𝚊n x(k − n) + 𝚋n 𝑤(k − n) (1.438)
n=1 n=0
and the model is denoted as ARMA(p, q).
Rewriting (1.438) in terms of the filter impulse response hARMA , we find in general
∑
+∞
x(k) = hARMA (n)𝑤(k − n) (1.439)
n=0
which indicates that the filter used to realize the ARMA model is causal. From (1.63), one finds that
the filter transfer function is given by
⎧ ∑q
⎪ B(z) = 𝚋n z−n
B(z) ⎪ n=0
HARMA (z) = where ⎨ (1.440)
A(z) ∑p
⎪ A(z) = 𝚊n z−n assuming 𝚊0 = 1
⎪
⎩ n=0

Using (1.188), the PSD of the process x is given by

{
| (f ) |2 (f ) = B(ej2𝜋f Tc )
x (f ) = Tc 𝜎𝑤2 || |
| where (1.441)
| (f ) | (f ) = A(ej2𝜋f Tc )
23
In a simulation of the process, the first samples x(k) generated by (1.438) should be neglected because they depend
on the initial conditions. Specifically, if NARMA is the length of the filter impulse response hARMA , the minimum number
of samples to be ignored is NARMA − 1, equal to the filter transient.
1.10. Parametric models of random processes 67

10

–10

–20
Amplitude (dB)

–30

–40

–50 D = 500, Ns = 39
D = 2000, Ns = 9
–60

–70

–80
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
f Tc

Figure 1.34 Comparison of spectral estimates obtained with the Welch periodogram method, using
the Hamming window, by varying parameters D ed Ns .

MA
If we particularize the ARMA model, assuming

𝚊i = 0, i = 1, 2, … , p (1.442)

or A(z) = 1, we get the moving average (MA) model of order q, also denoted MA(q). The equations of
the ARMA model therefore are reduced to

HMA (z) = B(z) (1.443)

and
x (f ) = Tc 𝜎𝑤2 |(f )|2 (1.444)

If we represent the function x (f ) of a process obtained by the MA model, we see that its behaviour
is generally characterized by wide peaks and narrow valleys, as illustrated in Figure 1.36.

AR
The auto-regressive (AR) model of order N, also denoted as AR(N) is shown in Figure 1.37.
The output process is described in this case by the recursive equation

∑
N
x(k) = − 𝚊n x(k − n) + 𝑤(k) (1.445)
n=1
68 Chapter 1. Elements of signal theory

w(k) w(k – 1) w(k – q)

Tc Tc Tc

b0 b1 bq

+
x(k)

ap a2 a1

Tc Tc Tc
x(k – p) x(k – 2) x(k–1)

Figure 1.35 ARMA model of a process x.

where 𝑤 is white noise with variance 𝜎𝑤2 . The transfer function is given by
1
HAR (z) = (1.446)
A(z)
with
∑
N
A(z) = 1 + 𝚊n z−n (1.447)
n=1

We observe that (1.446) describes a filter having N poles. Therefore, HAR (z) can be expressed as
1
HAR (z) = (1.448)
(1 − p1 z−1 )(1 − p2 z−1 ) … (1 − pN z−1 )
For a causal filter, the stability condition is |pi | < 1, i = 1, 2, … , N, i.e. all poles must be inside the unit
circle of the z plane.
In the case of the AR model, from Table 1.6 the z-transform of the ACS of x is given by

1 𝜎𝑤2
Px (z) = P𝑤 (z) ( ) = ( ) (1.449)
1 1
A(z)A∗ A(z)A∗ ∗
z∗ z
Hence, the function Px (z) has poles of the type
1 j𝜑i
|pi |ej𝜑i and e (1.450)
|pi |
Letting
(f ) = A(ej2𝜋f Tc ) (1.451)
one obtains the PSD of x, given by
Tc 𝜎𝑤2
x (f ) = (1.452)
|(f )|2
Typically, the function x (f ) of an AR process will have narrow peaks and wide valleys (see Figure 1.38),
opposite to the MA model.
1.10. Parametric models of random processes 69

20

15

10

5
Amplitude (dB)

–5

–10

–15

–10

–25
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
fTc

Figure 1.36 PSD of a MA process with q = 4.

w(k) + x(k)
–

aN a2 a1

Tc Tc Tc

Figure 1.37 AR model of a process x.

Spectral factorization of AR models

Consider the AR process described by (1.449). Observing (1.450), we have the following decomposition:
𝜎𝑤2
Px (z) = ( ) ( ) (1.453)
ej𝜑1 z−1 ej𝜑N z−1
(1 − |p1 |e z )...(1 − |pN |e z ) 1 −
j𝜑1 −1 j𝜑N −1 ... 1 −
|p1 | |pN |

For a given Px (z), it is clear that the N zeros of A(z) in (1.449) can be chosen in 2N different ways. The
selection of the zeros of A(z) is called spectral factorization. Two examples are illustrated in Figure 1.39.
As stated by the spectral factorization theorem (see page 35), there exists a unique spectral factor-
ization that yields a minimum-phase A(z), which is obtained by associating with A(z) only the poles of
Px (z) that lie inside the unit circle of the z-plane.
70 Chapter 1. Elements of signal theory

25

20

15

10
Amplitude (dB)

–5

–10

–15

–20
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
fTc

Figure 1.38 PSD of an AR process with N = 4.

Whitening filter
We observe an important property illustrated in Figure 1.40. Suppose x is modelled as an AR process
of order N and has PSD given by (1.449). If x is input to a filter having transfer function A(z), the output
of this latter filter would be white noise. In this case, the filter A(z) is called whitening filter (WF).
If A(z) is minimum phase, the white process 𝑤 is also called innovation of the process x, in the sense
that the new information associated with the sample x(k) is carried only by 𝑤(k).

Relation between ARMA, MA, and AR models

The relations among the three parametric models are expressed through the following propositions.

Wold decomposition. Every WSS random process y can be decomposed into:

y(k) = s(k) + x(k) (1.454)
where s and x uncorrelated processes. The process s, completely predictable, is described by the recursive
equation
∑
+∞
s(k) = − 𝛼n s(k − n) (1.455)
n=1

while x is obtained as filtered white noise:

∑
+∞
x(k) = h(n)𝑤(k − n) (1.456)
n=0
1.10. Parametric models of random processes 71

0.5

–0.5

–1

–1.5
–1.5 –1 –0.5 0 0.5 1 1.5
Generic function A(z)

0.5

–0.5

–1

–1.5
–1.5 –1 –0.5 0 0.5 1 1.5
Minimum-phase function A(z)

Figure 1.39 Two examples of possible choices of the zeros (×) of A(z), among the poles of Px (z).
72 Chapter 1. Elements of signal theory

AR(N ) x(k) x(k – 1) x(k – 2) x(k – N)

T T T
process

0 1 2 N–1 N

w(k)

Figure 1.40 Whitening filter for an AR process of order N.

Theorem 1.5 (Kolmogorov theorem)

Any ARMA or MA process can be represented by an AR process of infinite order. ◽

Therefore, any one of the three descriptions (ARMA, MA, or AR) can be adopted to approximate the
spectrum of a process, provided that the order is sufficiently high. However, in practice, determining the
coefficients associated with a model when the order is high is not a simple task from a numerical point
of view. It is better to start from a quite low order value, evaluating the fitting quality (usually by the
mean square error) between the given process and the output model and then increasing the order value
up to when the fitting quality does not significantly change, i.e. it reaches approximately its asymptotic
value.

1.10.1 Autocorrelation of AR processes

It is interesting to evaluate the autocorrelation function of a process x obtained by the AR model. Mul-
tiplying both members of (1.445) by x∗ (k − n), we find
∑
N
x(k)x∗ (k − n) = − 𝚊m x(k − m)x∗ (k − n) + 𝑤(k)x∗ (k − n) (1.457)
m=1

Taking expectations, and observing that 𝑤(k) is uncorrelated with all past values of x, for n ≥ 0 we
obtain
∑
N
E[x(k)x∗ (k − n)] = − 𝚊m E[x(k − m) x∗ (k − n)] + 𝜎𝑤2 𝛿n (1.458)
m=1

From (1.458), it follows

∑
N
𝚛x (n) = − 𝚊m 𝚛x (n − m) + 𝜎𝑤2 𝛿n (1.459)
m=1

In particular, we have
⎧ ∑ N
⎪− 𝚊m 𝚛x (n − m) n>0
⎪
⎪ m=1
𝚛x (n) = ⎨ ∑ N (1.460)
⎪ − 𝚊m 𝚛x (−m) + 𝜎𝑤2 n = 0
⎪ m=1
⎪ 𝚛∗x (−n) n<0
⎩
1.10. Parametric models of random processes 73

We observe that, for n > 0, 𝚛x (n) satisfies an equation analogous to the (1.445), with the exception of
the component 𝑤(k). This implies that, if {pi } are zeros of A(z), 𝚛x (n) can be written as
∑
N
𝚛x (n) = 𝚌i pni n>0 (1.461)
i=1

Assuming an AR process with |pi | < 1, for i = 1, 2, … , N, we get:

𝚛x (n) −−−−→ 0 (1.462)
n→∞

Simplifying the notation 𝚛x (n) with 𝚛(n), and observing (1.460), for n = 1, 2, … , N, we obtain a set of
equations that in matrix notation are expressed as
⎡ r(0) r(−1) … r(−N + 1) ⎤ ⎡ 𝚊1 ⎤ ⎡ 𝚛(1) ⎤
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ r(1) r(0) … r(−N + 2) ⎥ ⎢ 𝚊2 ⎥ ⎢ 𝚛(2) ⎥
⎢ ⎥⎢ ⎥ = −⎢ ⎥ (1.463)
⎢ ⋮ ⋮ ⋱ ... ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ r(N − 1) ⎥⎢𝚊 ⎥ ⎢ 𝚛(N) ⎥
⎣ r(N − 2) … r(0) ⎦⎣ N ⎦ ⎣ ⎦
that is
Ra = −r (1.464)
with obvious definition of the vectors. In the hypothesis that the matrix R has an inverse, the solution
for the coefficients {𝚊i } is given by
a = −R−1 r (1.465)
Equations (1.464) and (1.465), called Yule-Walker equations, provide the coefficients of an AR model
for a process having autocorrelation function 𝚛x . The variance 𝜎𝑤2 of white noise at the input can be
obtained from (1.460) for n = 0, which yields
∑
N
𝜎𝑤2 = 𝚛x (0) + 𝚊m 𝚛x (−m)
m=1
= 𝚛x (0) + r a H
(1.466)

Observation 1.6

● From (1.464) one finds that 𝚊 does not depend on 𝚛x (0), but only on the correlation coefficients
𝚛x (n)
𝜌x (n) = , n = 1, … , N (1.467)
𝚛x (0)
● Exploiting the fact that R is Toeplitz and Hermitian,24 the set of equations (1.465) and (1.466) can be
numerically solved by the Levinson–Durbin or by Delsarte–Genin algorithms, with a computational
complexity proportional to N 2 (see Appendix 2.A).
● We note that the knowledge of 𝚛x (0), 𝚛x (1), … , 𝚛x (N) univocally determines the ACS of an AR(N)
process; for n > N, from (1.460), we get
∑
N
𝚛x (n) = − 𝚊m 𝚛x (n − m) (1.468)
m=1

24
A matrix is Toeplitz if all elements of any diagonal are equal within the diagonal. Matrix A is Hermitian if A = AH .
74 Chapter 1. Elements of signal theory

10

–10

–20
Amplitude (dB)

–30

–40

–50

–60

–70

–80
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
f Tc

Figure 1.41 Comparison between the spectral estimate obtained by an AR(12) process model and
the analytical PSD given by (1.436).

1.10.2 Spectral estimation of an AR process

Assuming an AR(N) model for a process x, (1.465) yields the coefficient vector 𝚊, which implies an
estimate of the ACS up to lag N is available. From (1.452), we define as spectral estimate
Tc 𝜎𝑤2
AR (f ) = (1.469)
|(f )|2
Usually the estimate (1.469) yields a better resolution than estimates obtained by other methods, such
as BT (f ), because it does not show the effects of ACS truncation. In fact, the AR model yields
∑
+∞
AR (f ) = Tc 𝚛̂ x (n) e−j2𝜋fnTc (1.470)
n=−∞

where 𝚛̂ x (n) is estimated for |n| ≤ N with one of the two methods of Section 1.9.2, while for |n| > N the
recursive equation (1.468) is used. The AR model accurately estimates processes with a spectrum similar
to that given in Figure 1.38. For example, a spectral estimate for the process of Example 1.9.1 on page 64
obtained by an AR(12) model is depicted in Figure 1.41. The correlation coefficients were obtained by
a biased estimate on 10 000 samples. Note that the continuous part of the spectrum is estimated only
approximately; on the other hand, the choice of a larger order N would result in an estimate with larger
variance.
Also note that the presence of spectral lines in the original process leads to zeros of the polynomial A(z)
near the unit circle (see page 77). In practice, the correlation estimation method and a choice of a large N
may result in an ill-conditioned matrix R. In this case, the solution may have poles outside the unit circle,
1.10. Parametric models of random processes 75

3.5

3 1<0 1>0

2.5
Amplitude

1.5

0.5

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
f Tc

Figure 1.42 PSD of an AR(1) process.

and hence, the system would be unstable. Hence, two observations are in order. If possible, by filtering,
we should split the input process into its continuous and discrete parts and estimate the two components
separately. To avoid using an ill-conditioned matrix R, yielding unstable numerical solutions, we should
use an iterative approach where for increasing values of N, we evaluate 𝜎𝑤2 by (1.466), and where the
correlation 𝚛̂ x (n) is replaced by its estimate. A suitable value of N is reached when the estimate of 𝜎𝑤2
does not significantly change, i.e. it reaches approximately its asymptotic value.

Some useful relations

We will illustrate some examples of AR models. In particular, we will focus on the Yule–Walker
equations and the relation (1.466) for N = 1 and N = 2.

AR(1). From {
𝚛x (n) = −𝚊1 𝚛x (n − 1) n > 0
(1.471)
𝜎𝑤2 = 𝚛x (0) + 𝚊1 𝚛x (−1)
we obtain
𝜎𝑤2
𝚛AR(1) (n) = (−𝚊1 )|n| (1.472)
1 − |𝚊1 |2
from which the PSD is
Tc 𝜎𝑤2
AR(1) (f ) = (1.473)
|1 + 𝚊1 e−j2𝜋f Tc |2
The behaviour of the PSD of an AR(1) process is illustrated in Figure 1.42.
76 Chapter 1. Elements of signal theory

15

10
Amplitude

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
fTc

Figure 1.43 PSD of an AR(2) process.

AR(2). Let p1,2 = 𝜚e±j𝜑0 , where 𝜑0 = 2𝜋f0 Tc , be the two complex roots of A(z) = 1 + 𝚊1 z−1 + 𝚊2 z−2 .
We consider a real process:
{
𝚊1 = −2𝜚 cos(2𝜋f0 Tc )
(1.474)
𝚊2 = 𝜚2

Letting
[ ]
1 − 𝜚2 −1
𝜗 = tan−1 tan (2𝜋f T
0 c ) (1.475)
1 + 𝜚2

we find
√ ( )2
1+𝜚2 1−𝜚2
1−𝜚2
1+ 1+𝜚2
[tan−1 (2𝜋f0 Tc )]2
𝚛AR(2) (n) = 𝜎𝑤2 𝜚|n| cos(2𝜋f0 |n|Tc − 𝜗) (1.476)
1 − 𝜚2 cos2 (4𝜋f0 Tc ) + 𝜚4

The PSD is thus given by

Tc 𝜎𝑤2
AR(2) (f ) = (1.477)
|1 − 𝜚e−j2𝜋(f −f0 )Tc |2 |1 − 𝜚e−j2𝜋(f +f0 )Tc |2

We observe that, as 𝜚 → 1, AR(2) (f ) has a peak that becomes more pronounced, as illustrated in
Figure 1.43, and 𝚛x (k) tends to exhibit a sinusoidal behaviour.
1.10. Parametric models of random processes 77

Solutions of the Yule–Walker equations are

⎧ 𝚛x (1)𝚛x (0) − 𝚛x (1)𝚛x (2)
⎪ 𝚊1 = −
⎪ 𝚛2x (0) − 𝚛2x (1)
⎪
⎨ 𝚊 = − 𝚛x (0)𝚛x (2) − 𝚛x (1)
2
(1.478)
⎪ 2 𝚛x (0) − 𝚛x (1)
2 2
⎪
⎪ 𝜎𝑤 = 𝚛x (0) + 𝚊1 𝚛x (1) + 𝚊2 𝚛x (2)
2
⎩
Solving the previous set of equations with respect to 𝚛x (0), 𝚛x (1), and 𝚛x (2), one obtains
⎧ 1 + 𝚊2 𝜎𝑤2
⎪ 𝚛x (0) =
⎪ 1 − 𝚊2 (1 + 𝚊2 )2 − 𝚊21
⎪ 𝚊1
⎨ 𝚛x (1) = − 1 + 𝚊 𝚛x (0) (1.479)
⎪ ( 2 )
⎪ 𝚊21
⎪ 𝚛x (2) = −𝚊2 + 1 + 𝚊 𝚛x (0)
⎩ 2

In general, for n > 0, we have

[ ]
p1 (p22 − 1) p2 (p21 − 1)
𝚛x (n) = 𝚛x (0) pn1 − pn2 (1.480)
(p2 − p1 )(p1 p2 + 1) (p2 − p1 )(p1 p2 + 1)

AR model of sinusoidal processes

The general formulation of a sinusoidal process is
x(k) = A cos(2𝜋f0 kTc + 𝜑) (1.481)
with 𝜑 ∼  [0, 2𝜋). We observe that the process described by (1.481) satisfies the following difference
equation for k ≥ 0:
x(k) = 2 cos(2𝜋f0 Tc )x(k − 1) − x(k − 2) + 𝛿k A cos 𝜑 − 𝛿k−1 A cos(2𝜋f0 Tc − 𝜑) (1.482)
with x(−2) = x(−1) = 0. We note that the Kronecker impulses determine only the amplitude and phase
of x.
In the z-domain, we get the homogeneous equation
A(z) = 1 − 2 cos(2𝜋f0 Tc )z−1 + z−2 (1.483)
The zeros of A(z) are
p1,2 = e±j2𝜋f0 Tc (1.484)
It is immediate to verify that these zeros belong to the unit circle of the z plane. Consequently, the rep-
resentation of a sinusoidal process via the AR model is not possible, as the stability condition, |pi | < 1,
is not satisfied. Moreover, the input (1.482) is not white noise. In any case, we can try to find an approx-
imation. In the hypothesis of uniform 𝜑, from Example 1.7.5,
A2
𝚛x (n) =
cos(2𝜋f0 nTc ) (1.485)
2
This autocorrelation function can be approximated by the autocorrelation of an AR(2) process for 𝜚 → 1
and 𝜎𝑤2 → 0. Using (1.476), for 𝜚 ≃ 1 we find
⎡ 2
𝜎𝑤2 1−𝜚 ⎤
𝚛AR(2) (n) ≃ ⎢ ⎥ cos(2𝜋f nT )
2
(1.486)
⎢ 2 − 𝜚2 cos2 (4𝜋f0 Tc ) ⎥ 0 c
⎣ ⎦
78 Chapter 1. Elements of signal theory

and impose the condition

2
𝜎𝑤2 1−𝜚 2 A2
lim2 = (1.487)
𝜚→1,𝜎𝑤 →0 2− 𝜚2 cos2 (4𝜋f0 Tc ) 2

Observation 1.7
We can observe the following facts about the order of an AR model approximating a sinusoidal process.
● From (1.315), we find that an AR(N) process is required to model N complex sinusoids; on the other
hand, from (1.313), we see that an AR process of order 2N is required to model N real sinusoids.
● An ARMA(2N, 2N) process is required to model N real sinusoids plus white noise having variance
𝜎b2 . Observing (1.440), it results 𝜎𝑤2 → 𝜎b2 and B(z) → A(z).

1.11 Guide to the bibliography

The basics of signal theory and random processes can be found in the companion book [1]. Many of
the topics surveyed in this chapter are treated in general in several texts on digital communications, in
particular [6–8].
In-depth studies on deterministic systems and signals are found in [5, 9–13]. For a statistical analysis
of random processes, we refer to [2, 14, 15]. Finally, the subject of spectral estimation is discussed in
detail in [3, 16–18].

Bibliography

[1] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[2] Papoulis, A. (1991). Probability, Random Variables and Stochastic Processes, 3e. New York, NY:
McGraw-Hill.
[3] Priestley, M.B. (1981). Spectral Analysis and Time Series. New York, NY: Academic Press.
[4] Gentle, J. (2006). Random Number Generation and Monte Carlo Methods, Statistics and Comput-
ing. New York, NY: Springer.
[5] Papoulis, A. (1984). Signal Analysis. New York, NY: McGraw-Hill.
[6] Benedetto, S. and Biglieri, E. (1999). Principles of Digital Transmission with Wireless Applications.
New York, NY: Kluwer Academic Publishers.
[7] Messerschmitt, D.G. and Lee, E.A. (1994). Digital Communication, 2e. Boston, MA: Kluwer
Academic Publishers.
[8] Proakis, J.G. (1995). Digital Communications, 3e. New York, NY: McGraw-Hill.
[9] Cariolaro, G. (1996). La teoria unificata dei segnali. Torino: UTET.
[10] Papoulis, A. (1962). The Fourier Integral and Its Applications. New York, NY: McGraw-Hill.
[11] Oppenheim, A.V. and Schafer, R.W. (1989). Discrete-Time Signal Processing. Englewood Cliffs,
NJ: Prentice-Hall.
[12] Vaidyanathan, P.P. (1993). Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall.
[13] Crochiere, R.E. and Rabiner, L.R. (1983). Multirate Digital Signal Processing. Englewood Cliffs,
NJ: Prentice-Hall.
[14] Davenport, J.W.B. and Root, W.L. (1987). An Introduction to the Theory of Random Signals and
Noise. New York, NY: IEEE Press.
[15] Shiryayev, A.N. (1984). Probability. New York, NY: Springer-Verlang.
1.A. Multirate systems 79

[16] Kay, S.M. (1988). Modern Spectral Estimation-Theory and Applications. Englewood Cliffs, NJ:
Prentice-Hall.
[17] Marple, L.S. Jr. (1987). Digital Spectral Analysis with Applications. Englewood Cliffs, NJ:
Prentice-Hall.
[18] Stoica, P. and Moses, R. (1997). Introduction to Spectral Analysis. Englewood Cliffs, NJ:
Prentice-Hall.
[19] Erup, L., Gardner, F.M., and Harris, R.A. (1993). Interpolation in digital modems - Part II: imple-
mentation and performance. IEEE Transactions on Communications 41: 998–1008.
[20] Meyr, H., Moeneclaey, M., and Fechtel, S.A. (1998). Digital Communication Receivers. New York,
NY: Wiley.
[21] Golomb, S.W. (1967). Shift Register Sequences. San Francisco, CA: Holden-Day.
[22] Fan, P. and Darnell, M. (1996). Sequence Design for Communications Applications. Taunton:
Research Studies Press.
[23] Chu, D.C. (1972). Polyphase codes with good periodic correlation properties. IEEE Transactions on
Information Theory 18: 531–532.
[24] Frank, R.L. and Zadoff, S.A. (1962). Phase shift pulse codes with good periodic correlation proper-
ties. IRE Transactions on Information Theory 8: 381–382.
[25] Milewsky, A. (1983). Periodic sequences with optimal properties for channel estimation and fast
start-up equalization. IBM Journal of Research and Development 27: 426–431.
[26] Peterson, R.L., Ziemer, R.E., and Borth, D.E. (1995). Introduction to Spread Spectrum Communica-
tions. Englewood Cliffs, NJ: Prentice-Hall.
[27] Gold, R. (1967). Optimal binary sequences for spread spectrum multiplexing. IEEE Transactions on
Information Theory 13: 619–621.
[28] Gold, R. (1968). Maximal recursive sequences with 3-valued recursive cross correlation functions.
IEEE Transactions on Information Theory 14: 154–155.

Appendix 1.A Multirate systems

The first part of this appendix is a synthesis from [12, 13].

1.A.1 Fundamentals

We consider the discrete-time linear transformation of Figure 1.44, with impulse response h(t), t ∈ ℝ;
the sampling period of the input signal is Tc , whereas that of the output signal is Tc′ .
The input–output relation is given by the equation
∑
+∞
y(kTc′ ) = h(kTc′ − nTc )x(nTc ) (1.488)
n=−∞

We will use the following simplified notation:

xn = x(nTc ) (1.489)
yk = y(kTc′ ) (1.490)

If we assume that h has a finite support, say between t1 and t2 , that is h(kTc′ − nTc ) ≠ 0 for

kTc′ − nTc < t2 , kTc′ − nTc > t1 (1.491)

80 Chapter 1. Elements of signal theory

xn yk
h
Tc Tʹc

Figure 1.44 Discrete-time linear transformation.

or equivalently for
kTc′ − t2 kTc′ − t1
n> , n< (1.492)
Tc Tc
then, letting ⌈ ⌉
kTc′ − t2
n1 = (1.493)
T
⌊ ′c ⌋
kTc − t1
n2 = (1.494)
Tc
(1.488) can be written as
∑
n2
yk = h(kTc′ − nTc )xn = xn1 h(kTc′ − n1 Tc ) + · · · + xn2 h(kTc′ − n2 Tc ) (1.495)
n=n1

One observes from (1.488) that

● the values of h that contribute to yk are equally spaced by Tc ;
● the limits of the summation (1.495) are a complicated function of Tc , Tc′ , t1 , and t2 .
Introducing the change of variable ⌊ ⌋
kTc′
i= −n (1.496)
Tc
and setting
⌊ ′⌋
kTc′ kTc
Δk = − (1.497)
Tc Tc
⌈ ⌉ ⌈ ⌉ ⌊ ′⌋
t t kT ′ kTc
I1 = 1 − Δk = 1 − c + (1.498)
Tc Tc Tc T
⌈ ⌉ ⌈ ⌉ ⌊ c′ ⌋
t2 t2 kTc′ kTc
I2 = − Δk = − + (1.499)
Tc Tc Tc Tc
(1.495) becomes
∑
I2
yk = h((i + Δk )Tc )x⌊kT ′ ∕Tc ⌋−i (1.500)
c
i=I1

From the definition (1.497), it is clear that Δk represents the truncation error of kTc′ ∕Tc and that
0 ≤ Δk < 1. In the special case,
Tc′ M
= (1.501)
Tc L
with M and L integers, we get25
⌊ ⌋
M M
Δk = k − k
L L
( ⌊ ⌋ )
1 M
= kM − k L (1.502)
L L
1
= (kM)mod L
L
25
We denote by (k)mod n or k mod n the reminder of the division of integer k by integer n.
1.A. Multirate systems 81

We observe that Δk can assume L values {0, 1∕L, 2∕L, … , (L − 1)∕L} for any value of k. Hence, there
are only L univocally determined sets of values of h that are used in the computation of {yk }; in particular,
if L = 1 only one set of coefficients exists, while if M = 1 the sets are L. Summarizing, the output of a
filter with impulse response h and with different input and output time domains can be expressed as
∑
+∞
yk = gk,i x⌊ kM ⌋−i (1.503)
L
i=−∞

where
gk,i = h((i + Δk )Tc ) (1.504)
We note that the system is linear and periodically time varying. For Tc′ = Tc , that is for L = M = 1, we
get Δk = 0, and the input–output relation is the usual convolution
∑
+∞
yk = g0,i xk−i (1.505)
i=−∞

We will now analyse a few elementary multirate transformations.

1.A.2 Decimation

Figure 1.45 represents a decimator or downsampler, with the output sequence related to the input
sequence {xn } by
yk = xkM (1.506)
where M, the decimation factor, is an integer number.
We now obtain an expression for the z-transform of the output Y(z) in terms of X(z). We will show
that
M−1 ( )
1 ∑ 1
m
Y(z) = X z M WM (1.507)
M m=0
2𝜋
where WM = e−j M is defined in (1.28). Equivalently, in terms of the radian frequency normalized by the
sampling frequency, 𝜔′ = 2𝜋f ∕Fc′ , (1.507) can be written as
M−1 ( )
′ 1 ∑ 𝜔′ −2𝜋m
Y(ej𝜔 ) = X ej M (1.508)
M m=0
A graphical interpretation of (1.508) is shown in Figure 1.46:
′
● expand X(ej𝜔 ) by a factor M, obtaining X(ej𝜔 ∕M );
● create M − 1 replicas of the expanded version, and frequency-shift them uniformly with increments
of 2𝜋 for each replica;
● sum all the replicas and divide the result by M.

xn yk
M
Tc T′c =MTc
1 Fc
Fc = F′c =
Tc M

Figure 1.45 Decimation or downsampling transformation by a factor M.

82 Chapter 1. Elements of signal theory

X(e j ω)

{xn}

–3 –1 2 3

–2 0 1 n –4π –2π –π 0 π 2π 4π ω
X(e ωʹ)
j

{yk}

–1 1

0 k –4π –2π –π 0 π 2π 4π ωʹ

(a) (b)

Figure 1.46 Decimation by a factor M = 3: (a) in the time domain, and (b) in the normalized
radian frequency domain.

X(f )

–2 –1 1 2 f
0
Tc Tc Tc Tc
Y(f )

–2 –1 1 1 2
0
Tc Tc MTc Tc Tc
f

Figure 1.47 Effect of decimation in the frequency domain.

We observe that, after summation, the result is periodic in 𝜔′ with period 2𝜋, as we would expect
from a discrete-time Fourier transform.
It is also useful to give the expression of the output sequence in the frequency domain; we get
( )
1 ∑
M−1
m
(f ) =  f− (1.509)
M m=0 MTc

where
(f ) = X(ej2𝜋f Tc ) (1.510)

(f ) = Y(ej2𝜋fMTc ) (1.511)

The relation (1.509) for the signal of Figure 1.46 is represented in Figure 1.47. Note that the only dif-
ference with respect to the previous representation is that all frequency responses are now functions of
the frequency f .
1.A. Multirate systems 83

Proof of (1.507). The z-transform of {yk } can be written as

∑
+∞
∑
+∞
Y(z) = yk z−k = xMk z−k (1.512)
k=−∞ k=−∞

We define the intermediate sequence

{
xk k = 0, ±M, ±2M, …
xk′ = (1.513)
0 otherwise
′
so that yk = xMk = xMk . With this position, we get
∑
+∞
′
∑
+∞
Y(z) = xk′ ′ M z−k = xk′ z−k∕M = X ′ (z1∕M ) (1.514)
k′ =−∞ k=−∞

This relation is valid, because x′ is non-zero only at multiples of M. It only remains to express X ′ (z) in
terms of X(z); to do this, we note that (1.513) can be expressed as
xk′ = ck xk (1.515)
where ck is defined as {
1 k = 0, ±M, ±2M, …
ck = (1.516)
0 otherwise
Note that the (1.516) can be written as

1 ∑ −km
M−1
ck = W (1.517)
M m=0 M
Hence, we obtain
1 ∑ ∑ 1 ∑ ∑
M−1 +∞ M−1 +∞
X ′ (z) = −km −k
x k WM z = m −k
x (zWM ) (1.518)
M m=0 k=−∞ M m=0 k=−∞ k
m
The inner summation yields X(zWM ): hence, using (1.514) we get (1.507).

1.A.3 Interpolation

Figure 1.48 represents an interpolator or upsampler, with the input sequence {xn } related to the output
sequence by { ( )
k
x k = 0, ±L, ±2L, …
yk = L (1.519)
0 otherwise
where L, the interpolation factor, is an integer number.
We will show that the input–output relation in terms of the z-transforms Y(z) and X(z) is given by
Y(z) = X(zL ) (1.520)
Equivalently, in terms of radian frequency normalized by the sampling frequency, 𝜔′ = 2𝜋f ∕Fc′ , the
(1.520) can be expressed as
′ ′
Y(ej𝜔 ) = X(ej𝜔 L ) (1.521)
The graphical interpretation of (1.521) is illustrated in Figure 1.49: Y(ej𝜔 ) is the compressed version by
a factor L of X(ej𝜔 ); moreover, there are L − 1 replicas of the compressed spectrum, called images. The
creation of images implies that a lowpass signal does not remain lowpass after interpolation.
84 Chapter 1. Elements of signal theory

xn yk
L Tc
Tc T′c =
L
1
Fc = F′c =LFc
Tc

Figure 1.48 Interpolation or upsampling transformation by a factor L.

X(ejω)

{xn}

–1 1
0 n –2π –π 0 π 2π ω
Y(ejω')

{yk}

–3 –2 –1 1 2 3
0 k –2π –π 0 π 2π ω'

(a) (b)

Figure 1.49 Interpolation by a factor L = 3: (a) in the time domain, (b) in the normalized radian
frequency domain.

It is also useful to give the expression of the output sequence in the frequency domain; we get
(f ) = (f ) (1.522)
where
(f ) = X(ej2𝜋f Tc ) (1.523)
Tc
(f ) = Y(ej2𝜋f L ) (1.524)
Relation (1.522) for the signal of Figure 1.49 is illustrated in Figure 1.50. We note that the only effect
of the interpolation is that the signal  must be regarded as periodic with period Fc′ rather than Fc .

Proof of (1.520). Observing (1.519), we get

∑
+∞
∑
+∞
∑
+∞
Y(z) = yk z−k = ynL z−nL = xn z−nL = X(zL ) (1.525)
k=−∞ n=−∞ n=−∞

1.A.4 Decimator filter

In most applications, a downsampler is preceded by a lowpass digital filter, to form a decimator filter
as illustrated in Figure 1.51. The filter ensures that the signal 𝑣n is bandlimited, to avoid aliasing in the
downsampling process.
Let hn = h(nTc ). Then we have
yk = 𝑣kM (1.526)
1.A. Multirate systems 85

(f)

1 1 2 3
– Tc 0 Tc Tc Tc f

(f)

1 1 2 3
– Tc 0 Tc Tc Tc f

Figure 1.50 Effect of interpolation in the frequency domain.

xn yk xn υn yk
h h M
Tc T'c = MTc Tc Tc T'c

Figure 1.51 Decimator filter.

and
∑
+∞
𝑣n = hi xn−i (1.527)
i=−∞

The output can be expressed as

∑
+∞
∑
+∞
yk = hi xkM−i = hkM−n xn (1.528)
i=−∞ n=−∞

Using definition (1.504), we get

gk,i = hi ∀k, i (1.529)
Note that the overall system is not time invariant, unless the delay applied to the input is constrained to
be a multiple of M.
From V(z) = X(z)H(z), it follows that

1 ∑
M−1
Y(z) = H(z1∕M WM
m
)X(z1∕M WM
m
) (1.530)
M m=0
or, equivalently, recalling that 𝜔′ = 2𝜋fMTc ,
( 𝜔′ −2𝜋m ) ( 𝜔′ −2𝜋m )
1 ∑
M−1
′
Y(ej𝜔 ) = H ej M X ej M (1.531)
M m=0
If { 𝜋
j𝜔 1 |𝜔| ≤
H(e ) = M (1.532)
0 otherwise
we obtain ( j𝜔′ )
′1
Y(ej𝜔 ) = X eM |𝜔′ | ≤ 𝜋 (1.533)
M
In this case, h is a lowpass filter that avoids aliasing caused by sampling; if x is bandlimited, the speci-
fications of h can be made less stringent.
86 Chapter 1. Elements of signal theory

|  (f )|

0 Fc /2 Fc f
|  (f )|

0 Fc /2 Fc f
| v (f )|

0 Fc / 2 Fc f
| y (f )|

0 F′c /2 F′c Fc /2 Fc f

Figure 1.52 Frequency responses related to the transformations in a decimator filter for M = 4.

The decimator filter transformations are illustrated in Figure 1.52 for M = 4.

1.A.5 Interpolator filter

An interpolator filter is given by the cascade of an upsampler and a digital filter, as illustrated in
Figure 1.53; the task of the digital filter is to suppress images created by upsampling [19].
Let hn = h(nTc′ ). Then we have the following input–output relations:
∑
+∞
yk = hk−j 𝑤j (1.534)
j=−∞
{ ( )
k
x k = 0, ±L, …
𝑤k = L (1.535)
0 otherwise
Therefore,
∑
+∞
yk = hk−rL xr (1.536)
r=−∞

Let i = ⌊k∕L⌋ − r and gk,i = hiL+(k)mod L . From (1.536), we get

∑
+∞
yk = gk,i x⌊ k ⌋−i (1.537)
L
i=−∞

We note that gk,i is periodic in k of period L.

1.A. Multirate systems 87

xn yk xn wk yk
h Tc
L h
Tc Tʹc = Tc Tʹc Tʹc
L

Figure 1.53 Interpolator filter.

{xn} |  (f )|
1

0 1 n 0 Fc F′c /2 F′c f

|  (f )|
{wk}
1

0123 k 0 F′c /2 F′c f

|  (f )|

0 F′c /2 F′c f

{yk} |  (f )|

0123 k 0 F′c /2 F′c f

Figure 1.54 Time and frequency responses related to the transformations in an interpolator filter
for L = 3.

In the z-transform domain, we find

W(z) = X(zL ) (1.538)

L
Y(z) = H(z)W(z) = H(z)X(z ) (1.539)

or, equivalently,
′ ′ ′
Y(ej𝜔 ) = H(ej𝜔 )X(ej𝜔 L ) (1.540)

where 𝜔′ = 2𝜋fT∕L = 𝜔∕L.

The interpolator filter transformations in the time and frequency domains are illustrated in Figure 1.54
for L = 3.
88 Chapter 1. Elements of signal theory

If { 𝜋
j𝜔′ 1 |𝜔′ | ≤
H(e ) = L (1.541)
0 elsewhere
we find { 𝜋
|𝜔′ | ≤
′
j𝜔′ X(ej𝜔 )
Y(e ) = L (1.542)
0 elsewhere
The relation between the input and output signal power for an interpolator filter is expressed by (1.344).

1.A.6 Rate conversion

Decimator and interpolator filters can be employed to vary the sampling frequency of a signal by an
integer factor; in some applications, however, it is necessary to change the sampling frequency by
a rational factor L∕M. A possible procedure consists of first converting a discrete-time signal into a
continuous-time signal by a digital-to-analog converter (DAC), then re-sampling it at the new fre-
quency. It is however easier and more convenient to change the sampling frequency by discrete-time
transformations, for example using the structure of Figure 1.55.
This system can be thought of as the cascade of an interpolator and decimator filter, as illustrated in
Figure 1.56, where h(t) = h1 ∗ h2 (t). We obtain that
{ ( )
𝜋 𝜋
j𝜔′ 1 |𝜔′ | ≤ min ,
H(e ) = L M (1.543)
0 elsewhere
In the time domain, the following relation holds:
∑
+∞
yk = gk,i x⌊ kM ⌋−i (1.544)
L
i=−∞

where gk,i = h((iL + (kM)mod L )Tc′ ) is the time-varying impulse response.

In the frequency domain, we get
M−1 ( )
′′ 1 ∑ 𝜔′′ −2𝜋l
Y(ej𝜔 ) = V ej M (1.545)
M l=0
As
′ ′ ′
V(ej𝜔 ) = H(ej𝜔 )X(ej𝜔 L ) (1.546)

we obtain
( 𝜔′′ −2𝜋l ) ( 𝜔′′ L−2𝜋l )
1 ∑
M−1
′′
Y(ej𝜔 ) = H ej M X ej M (1.547)
M l=0
From (1.543), we have
{ ( j𝜔′′ L ) ( )
1 𝜋M
j𝜔′′ X e M |𝜔′′ | ≤ min 𝜋,
Y(e )= M L (1.548)
0 elsewhere
or ( )
1 1 L
(f ) = (f ) for |f | ≤ min , (1.549)
M 2Tc 2MTc
1.A. Multirate systems 89

xn yk xn wl υl yk
h M
L h M
Tc Tʺc = T Tc Tʹc
L c
Tʹc = Tc Tʺc = MTʹc
L

Figure 1.55 Sampling frequency conversion by a rational factor.

Interpolator Decimator
xn
L h1 h2 M
Tc Tʹc Tʹc = Tc Tʹc Tʺc = MTʹc
L

Figure 1.56 Decomposition of the system of Figure 1.55.

X(e j ω)

.
0 4 2π ω = 2π f T
4Fc f
W(e j ω′)

0 π /L 2π ω′ = ω L
Fc /2 4Fc f
H(e j ω′)

0 π/M 2π ω′
LFc LFc f
V(e j ω′) 2M

0 π/M 2π ω′
LFc LFc f
Y(e j ω″) 2M
M=5

0 π 5.2π ω″ = ω′ M
LFc LFc f
2M

Figure 1.57 Rate conversion by a rational factor L∕M, where M > L.

Example 1.A.1 (M > L: M = 5, L = 4)

′
Transformations for M = 5 and L = 4 are illustrated in Figure 1.57. Observing the fact that W(ej𝜔 ) is
zero for M𝜋 ≤ 𝜔′ ≤ 2 𝜋L − M𝜋 , the desired result is obtained by a response H(ej𝜔 ) that has the stopband
′

cutoff frequency within this interval.

90 Chapter 1. Elements of signal theory

X(e j ω)

0 π 2π 5.2π ω = 2π f Tc
Fc /2 Fc 5Fc f
W(e j ω′) L=5

0 π /5 2π ω′ = ω L
Fc /2 5Fc f
H(e j ω′)

0 π/5 2π ω′
Fc /2 LFc f
V(e j ω′)

0 π/5 2π ω′
Fc /2 LFc f
Y(e j ω″)

0 M π π 2π 4.2π ω″ = ω′ M
L
MFc Fc MFc LFc f
2L 2 L

Figure 1.58 Rate conversion by a rational factor L∕M, where M < L.

Example 1.A.2 (M < L: M = 4, L = 5)

The inverse transformation of the above example is obtained by a transformation with M = 4 and L = 5,
as depicted in Figure 1.58.

1.A.7 Time interpolation

Referring to the interpolator filter h of Figure 1.53, one finds that if L is large the filter implementa-
tion may require non-negligible complexity; in fact, the number of coefficients required for a FIR filter
implementation can be very large. Consequently, in the case of a very large interpolation factor L, after a
first interpolator filter with a moderate value of the interpolation factor, the samples {yk = y(kTc′ )} may
be further time interpolated until the desired sampling accuracy is reached [19].
As shown in Figure 1.59, let {yk } be the sequence that we need to interpolate to produce the signal
z(t), t ∈ ℝ; we describe two time interpolation methods: linear and quadratic.

Linear interpolation
Given two samples yk−1 and yk , the continuous signal z defined on the interval [(k − 1) Tc′ , kTc′ ], is
obtained by the linear interpolation
t − (k − 1)Tc′
z(t) = yk−1 + (yk − yk−1 ) (1.550)
Tc′
1.A. Multirate systems 91

yk

zn z(t)

yk – 1

yk + 1

(k – 2) T ′c (k – 1) T ′c kT ′c (k + 1) T ′c t

Figure 1.59 Linear interpolation in time by a factor P = 4.

For an interpolation factor P of yk , we need to consider the sampling instants

Tc′
nTc′′ = n (1.551)
P
and the values of zn = z(nTc′′ ) are given by
n − (k − 1)P
zn = yk−1 + (yk − yk−1 ) (1.552)
P
where n = (k − 1)P, (k − 1)P + 1, … , kP − 1. The case k = 1 is of particular interest:
n
zn = y0 + (y1 − y0 ), n = 0, 1, … , P − 1 (1.553)
P
In fact, regarding y0 and y1 as the two most recent input samples, their linear interpolation originates the
sequence of P values given by (1.553).

Quadratic interpolation
In many applications linear interpolation does not always yield satisfactory results. Therefore, instead of
connecting two points with a straight line, one resorts to a polynomial of degree Q − 1 passing through
Q points that are determined by the samples of the input sequence. For this purpose, the Lagrange
interpolation is widely used. As an example, we report here the case of quadratic interpolation. In this
case, we consider a polynomial of degree 2 that passes through three points that are determined by
the input samples. Let yk−1 , yk , and yk+1 be the samples to interpolate by a factor P in the interval
[(k − 1)Tc′ , (k + 1)Tc′ ]. The quadratic interpolation yields the values
( ) ( )( ) ( )
n′ n′ n′ n′ n′ n′
zn = − 1 yk−1 + 1 − 1+ yk + + 1 yk+1 (1.554)
2P P P P 2P P
with n′ = 0, 1, … , P − 1 and n = (k − 1) P + n′ .

1.A.8 The noble identities

We recall some important properties of decimator and interpolator filters, known as noble identities;
they will be used extensively in the Appendix 1.A.9 on polyphase decomposition.
Let G(z) be a rational transfer function, i.e. a function expressed as the ratio of two polynomials in z
or in z−1 ; it is possible to exchange the order of downsampling and filtering, or the order of upsampling
92 Chapter 1. Elements of signal theory

xn y1,k xn y1,k
M G(z) G(zM) M

(a) (b)

xn y3,k xn x3,k y3,k

G(z) L L G(zL)

(c) (d)

Figure 1.60 Noble identities.

and filtering as illustrated in Figure 1.60; in other words, the system of Figure 1.60a is equivalent to that
of Figure 1.60b, and the system of Figure 1.60c is equivalent to that of Figure 1.60d.
−mM
The proof of the noble identities is simple. For the first identity, it is sufficient to note that WM = 1,
hence,

1 ∑
M−1
Y2 (z) = X (z1∕M WM
−m
)G((z1∕M WM
−m M
) )
M m=0

1 ∑
M−1
= X(z1∕M WM
−m
)G(z) = Y1 (z) (1.555)
M m=0
For the second identity, it is sufficient to observe that
Y4 (z) = G(zL )X4 (z) = G(zL )X(zL ) = Y3 (z) (1.556)

1.A.9 The polyphase representation

The polyphase representation allows considerable simplifications in the analysis of transformations via
interpolator and decimator filters, as well as the efficient implementation of such filters. To explain
∑∞
the basic concept, let us consider a filter having transfer function H(z) = n=0 hn z−n . Separating the
coefficients with even and odd time indices, we get
∑
∞
∑
∞
H(z) = h2m z−2m + z−1 h2m+1 z−2m (1.557)
m=0 m=0

Defining
∑
∞
∑
∞
E(0) (z) = h2m z−m E(1) (z) = h2m+1 z−m (1.558)
m=0 m=0

we can write H(z) as

H(z) = E(0) (z2 ) + z−1 E(1) (z2 ) (1.559)
To expand this idea, let M be an integer; we can always decompose H(z) as
∑
∞
H(z) = hmM z−mM
m=0
∑
∞
+ z−1 hmM+1 z−mM + · · · (1.560)
m=0
∑
∞
+ z−(M−1) hmM+M−1 z−mM
m=0
1.A. Multirate systems 93

e(0)
m

0 1 2 m

hn e(1)
m

1 5 0
0 2 3 4 6 n 1 m

e(2)
m

1
0 m

Figure 1.61 Polyphase representation of the impulse response {hn }, n = 0, … , 6, for M = 3.

Letting
e(𝓁)
m = hmM+𝓁 0≤𝓁 ≤M−1 (1.561)

we can express compactly the previous equation as

∑
M−1
H(z) = z−𝓁 E(𝓁) (zM ) (1.562)
𝓁=0

where
∑
∞
E(𝓁) (z) = e(𝓁)
i
z−i (1.563)
i=0

The expression (1.562) is called the type-1 polyphase representation (with respect to M), and E(𝓁) (z),
where 𝓁 = 0, 1, … , M − 1, the polyphase components of H(z).
The polyphase representation of an impulse response {hn } with seven coefficients is illustrated in
Figure 1.61 for M = 3.
A variation of (1.562), named type-2 polyphase representation, is given by

∑
M−1
H(z) = z−(M−1−𝓁) R(𝓁) (zM ) (1.564)
𝓁=0

where the components R(𝓁) (z) are permutations of E(𝓁) (z), that is R(𝓁) (z) = E(M−1−𝓁) (z).

Efficient implementations
The polyphase representation is key to obtain efficient implementation of decimator and interpolator
filters. In the following, we will first consider the efficient implementations for M = 2 and L = 2, then
we will extend the results to the general case.
94 Chapter 1. Elements of signal theory

Decimator filter. Referring to Figure 1.51, we consider a decimator filter with M = 2. By the (1.562),
we can represent H(z) as illustrated in Figure 1.62; by the noble identities, the filter representation can be
drawn as in Figure 1.63a. The structure can be also drawn as in Figure 1.63b, where input samples {xn }
are alternately presented at the input to the two filters e(0) and e(1) ; this latter operation is generally called
serial-to-parallel (S/P) conversion. Note that the system output is now given by the sum of the outputs
of two filters, each operating at half the input frequency and having half the number of coefficients as the
original filter.
To formalize the above ideas, let N be the number of coefficients of h, and N (0) and N (1) be the number
of coefficients of e(0) and e(1) , respectively, so that N = N (0) + N (1) . In this implementation, e(𝓁) requires
N (𝓁) multiplications and N (𝓁) − 1 additions; the total cost is still N multiplications and N − 1 additions,
but, as e(𝓁) operates at half the input rate, the computational complexity in terms of multiplications per
second (MPS) is
NFc
MPS = (1.565)
2
while the number of additions per second (APS) is
(N − 1)Fc
APS = (1.566)
2
Therefore, the complexity is about one-half the complexity of the original filter. The efficient imple-
mentation for the general case is obtained as an extension of the case for M = 2 and is shown in
Figure 1.64.

Interpolator filter. With reference to Figure 1.53, we consider an interpolator filter with L = 2. By
the (1.562), we can represent H(z) as illustrated in Figure 1.65; by the noble identities, the filter repre-
sentation can be drawn as in Figure 1.66a. The structure can be also drawn as in Figure 1.66b, where
output samples are alternately taken from the output of the two filters e(0) and e(1) ; this latter operation
is generally called parallel-to-serial (P/S) conversion. Remarks on the computational complexity are
analogous to those of the decimator filter case.

xn
E(0)(z2)
Tc
z–1
vn yk
E(1)(z2) 2
T ′c = 2Tc

Figure 1.62 Implementation of a decimator filter using the type-1 polyphase representation
for M = 2.

S/P
xn
2 E(0)(z) n=0
E(0)(z)
xn
z–1 Tc
yk yk
2 E(1)(z) n = –1 E(1)(z)
T ′c = 2Tc
(a) (b)

Figure 1.63 Optimized implementation of a decimator filter using the type-1 polyphase
representation for M = 2.
1.A. Multirate systems 95

S/P
xn
M E(0)(z) E(0)(z)
n=0
z–1
n = –1
M E(1)(z) E(1)(z)

z–1 xn

yk n = –(M–1) yk
M E(M–1)(z) E(M–1)(z)

(a) (b)

Figure 1.64 Implementation of a decimator filter using the type-1 polyphase representation.

xn
2 E(0)(z2)
Tc

yk
E(1)(z2) z–1
Tc
T ′c = 2

Figure 1.65 Implementation of an interpolator filter using the type-1 polyphase representation for
L = 2.

In the general case, efficient implementations are easily obtainable as extensions of the case for L = 2
and are shown in Figure 1.67. The type-2 polyphase implementations of interpolator filters are depicted
in Figure 1.68.

Interpolator-decimator filter. As illustrated in Figure 1.69, at the receiver of a transmission system,

it is often useful to interpolate the signal {r(nTQ )} from TQ to TQ′ to get the signal {x(qTQ′ )}.
Let
rn = r(nTQ ), xq = x(qTQ′ ) (1.567)

The sequence {x(qTQ′ )} is then downsampled with timing phase t0 . Let yk be the output with sampling
period Tc ,
yk = x(kTc + t0 ) (1.568)

To simplify the notation, we assume the following relations:

TQ Tc
L= , M= (1.569)
TQ′ TQ
96 Chapter 1. Elements of signal theory

P/S
xn xn
E(0) (z) 2 E(0) (z)
Tc
yk
Tc
T ′c =
E(1) (z) 2 z–1 E(1) (z) 2
yk
(a) (b)

Figure 1.66 Optimized implementation of an interpolator filter using the type-1 polyphase
representation for L = 2.

P/S
xn yk xn
E(0)(z) L E(0)(z)
k=0
z–1
k=1
E(1)(z) L E(1)(z)
yk
z–1

k = L–1
E(L –1)(z) L E(L – 1)(z)

(a) (b)

Figure 1.67 Implementation of an interpolator filter using the type-1 polyphase representation.

P/S
xn xn k = L–1
R(0)(z) L R(0)(z)

z–1

k = L–2
R(1)(z) L R(1)(z)
yk

z–1

yk k=0
R(L–1)(z) L R(L– 1)(z)

(a) (b)

Figure 1.68 Implementation of an interpolator filter using the type-2 polyphase representation.
1.A. Multirate systems 97

Timing phase t0

rn xq yk
h ML
TQ TQ Tc = MLT ′Q = MTQ
TQ
′ = L

Figure 1.69 Interpolator-decimator filter.

with L and M positive integer numbers. Moreover, we assume that t0 is a multiple of TQ′ , i.e.
t0
= 𝓁0 +  0 L (1.570)
TQ′
where 𝓁0 ∈ {0, 1, … , L − 1}, and 0 is a non-negative integer number. For the general case of an
interpolator-decimator filter, where t0 and the ratio Tc ∕TQ′ are not constrained, we refer to [20] (see
also Chapter 14).
Based on the above equations we have

yk = xkML+𝓁0 +0 L (1.571)

We now recall the polyphase representation of {h(nTQ′ )} with L phases

{E(𝓁) (z)}, 𝓁 = 0, 1, … , L − 1 (1.572)

The interpolator filter structure from TQ to TQ′ is illustrated in Figure 1.67. For the special case M = 1,
that is for Tc = TQ , the implementation of the interpolator-decimator filter is given in Figure 1.70, where

yk = 𝑣k+0 (1.573)

In other words, {yk } coincides with the signal {𝑣n } at the output of branch 𝓁0 of the polyphase struc-
ture. In practice, we need to ignore the first 0 samples of {𝑣n }, as the relation between {𝑣n } and {yk }
must take into account a lead, z0 , of 0 samples. With reference to Figure 1.70, the output {xq } at
instants that are multiples of TQ′ is given by the outputs of the various polyphase branches in sequence.

rn x(nTQ + 0)
E(0)(z)
TQ

x(nTQ + l0T ′Q) = vn yk

E(l0)(z) z 0
TQ Tc = TQ

x(nTQ + (L –1)T ′Q)

E(L–1)(z)

Figure 1.70 Polyphase implementation of an interpolator-decimator filter with timing phase

t0 = (𝓁0 + 0 L)TQ′ .
98 Chapter 1. Elements of signal theory

rn r′p x′p yk
z–0 E(l0)(z) z ℳ0M M
TQ Tc = MTQ

(a)

E(l0,0)(z)

r′p E(l0,1)(z)

TQ

x′mM = vm yk
E(l0,M–1)(z) zℳ 0
Tc = MTQ Tc = MTQ

(b)

Figure 1.71 Implementation of an interpolator-decimator filter with timing phase

t0 = (𝓁0 + 0 L)TQ′ . (a) Basic scheme and (b) efficient implementation.

In fact, let q = 𝓁 + nL, 𝓁 = 0, 1, … , L − 1, and n integer, we have

x𝓁+nL = x(nLTQ′ + 𝓁TQ′ ) = x(nTQ + 𝓁TQ′ ) (1.574)

We now consider the general case M ≠ 1. First, to downsample the signal interpolated at TQ′ one can
still use the polyphase structure of Figure 1.70. In any case, once t0 is chosen, the branch is identified
(say 𝓁0 ) and its output must be downsampled by a factor ML. Notice that there is the timing lead 0 L in
(1.570) to be considered. Given 0 , we determine a positive integer 0 so that 0 + 0 is a multiple
of M, that is
0 + 0 = 0 M (1.575)

The structure of Figure 1.70, considering only branch 𝓁0 , is equivalent to that given in Figure 1.71a, in
which we have introduced a lag of 0 samples on the sequence {rn } and a further lead of 0 samples
before the downsampler. In particular, we have

rp′ = rp−0 and xp′ = xp−0 (1.576)

As a result, the signal is not modified before the downsampler.

Using now the representation of E(𝓁0 ) (zL ) in M phases:

E(𝓁0 ,m) (zLM ), m = 0, 1, … , M − 1 (1.577)

an efficient implementation of the interpolator-decimator filter is given in Figure 1.71b.

Appendix 1.B Generation of a complex Gaussian noise

Let 𝑤 = 𝑤I + j𝑤Q be a complex Gaussian r.v. with zero mean and unit variance; note that 𝑤I = Re [𝑤]
and 𝑤Q = Im [𝑤]. In polar notation,
𝑤 = A ej𝜑 (1.578)
1.C. Pseudo-noise sequences 99

It can be shown that 𝜑 is a uniform r.v. in [0, 2𝜋), and A is a Rayleigh r.v. with probability distribution
{
a>0
2
1 − e−a
P[A ≤ a] = (1.579)
0 a<0
Observing (1.579) and (1.578), if u1 , u2 ∼  (0, 1) independent, then
√
A = − ln(1 − u1 ) (1.580)

and
𝜑 = 2𝜋 u2 (1.581)

In terms of real components, it results that

𝑤I = A cos 𝜑 and 𝑤Q = A sin 𝜑 (1.582)

are two statistically independent Gaussian r.v.s., each with zero mean and variance equal to 0.5.
The r.v. 𝑤 is also called circularly symmetric Gaussian r.v., as the real and imaginary components,
being statistically independent with equal variance, have a circularly symmetric Gaussian joint proba-
bility density function.

Appendix 1.C Pseudo-noise sequences

In this Appendix, we introduce three classes of deterministic periodic sequences having spectral char-
acteristics similar to those of a white noise, hence, the name pseudo-noise (PN) sequences.

Maximal-length
A maximal-length sequence (MLS), also denoted m-sequence or r-sequence, is a binary PN sequence
generated recursively, e.g. using a shift-register, and has period L = 2r − 1. Let {p(𝓁)}, 𝓁 = 0, 1, … ,
L − 1, p(𝓁) ∈ {0, 1}, be the values assumed by the sequence in a period. It enjoys the following proper-
ties [7, 21].

● Every non-zero sequence of r bits appears exactly once in each period; Therefore, all binary
sequences of r bits are generated, except the all zero sequence.
● The number of bits equal to 1 in a period is 2r−1 , and the number of bits equal to 0 is 2r−1 − 1.
● A subsequence is intended here as a set of consecutive bits of the MLS. The relative frequency of
any non-zero subsequence of length i ≤ r is
2r−i
≃ 2−i (1.583)
2r − 1
and the relative frequency of a subsequence of length i < r with all bits equal to zero is
2r−i − 1
≃ 2−i (1.584)
2r − 1
In both formulas the approximation is valid for a sufficiently large r.
● The sum of two MLSs, which are generated by the same shift-register, but with different initial
conditions, is still an MLS.
● The linear span, that determines the predictability of a sequence, is equal to r [22]. In other words,
the elements of a sequence can be determined by any 2r consecutive elements of the sequence itself,
while the remaining elements can be produced by a recursive algorithm.
100 Chapter 1. Elements of signal theory

p(l) p(l – 1) p(l – 2) p(l – 3) p(l – 4)

Tc Tc Tc Tc

MOD 2

Figure 1.72 Generation of an MLS with period L = 15.

Table 1.8: Recursive equations to generate PN sequences of

length L = 2r − 1, for different values of r. p(𝓁) ∈ {0, 1} and ⊕
denotes the modulus-2 sum.

r period L = 2r − 1
1 p(𝓁) = p(𝓁 − 1)
2 p(𝓁) = p(𝓁 − 1) ⊕ p(𝓁 − 2)
3 p(𝓁) = p(𝓁 − 2) ⊕ p(𝓁 − 3)
4 p(𝓁) = p(𝓁 − 3) ⊕ p(𝓁 − 4)
5 p(𝓁) = p(𝓁 − 3) ⊕ p(𝓁 − 5)
6 p(𝓁) = p(𝓁 − 5) ⊕ p(𝓁 − 6)
7 p(𝓁) = p(𝓁 − 6) ⊕ p(𝓁 − 7)
8 p(𝓁) = p(𝓁 − 2) ⊕ p(𝓁 − 3) ⊕ p(𝓁 − 4) ⊕ p(𝓁 − 8)
9 p(𝓁) = p(𝓁 − 5) ⊕ p(𝓁 − 9)
10 p(𝓁) = p(𝓁 − 7) ⊕ p(𝓁 − 10)
11 p(𝓁) = p(𝓁 − 9) ⊕ p(𝓁 − 11)
12 p(𝓁) = p(𝓁 − 2) ⊕ p(𝓁 − 10) ⊕ p(𝓁 − 11) ⊕ p(𝓁 − 12)
13 p(𝓁) = p(𝓁 − 1) ⊕ p(𝓁 − 11) ⊕ p(𝓁 − 12) ⊕ p(𝓁 − 13)
14 p(𝓁) = p(𝓁 − 2) ⊕ p(𝓁 − 12) ⊕ p(𝓁 − 13) ⊕ p(𝓁 − 14)
15 p(𝓁) = p(𝓁 − 14) ⊕ p(𝓁 − 15)
16 p(𝓁) = p(𝓁 − 11) ⊕ p(𝓁 − 13) ⊕ p(𝓁 − 14) ⊕ p(𝓁 − 16)
17 p(𝓁) = p(𝓁 − 14) ⊕ p(𝓁 − 17)
18 p(𝓁) = p(𝓁 − 11) ⊕ p(𝓁 − 18)
19 p(𝓁) = p(𝓁 − 14) ⊕ p(𝓁 − 17) ⊕ p(𝓁 − 18) ⊕ p(𝓁 − 19)
20 p(𝓁) = p(𝓁 − 17) ⊕ p(𝓁 − 20)

A practical example is given in Figure 1.72 for a sequence with L = 15 (r = 4), which is generated by
the recursive equation:
p(𝓁) = p(𝓁 − 3) ⊕ p(𝓁 − 4) (1.585)

where ⊕ denotes modulo 2 sum. Assuming initial conditions p(−1) = p(−2) = p(−3) = p(−4) = 1,
applying (1.585) we obtain the sequence

0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 … (1.586)
⏟⏟⏟ ⏟⏟⏟ ⏟⏟⏟
p(0) p(1) p(L−1)
1.C. Pseudo-noise sequences 101

Obviously, the all zero initial condition must be avoided. To generate MLSs with a larger period L,
we refer to Table 1.8. The above properties make an MLS, even if deterministic and periodic, appear
as a random i.i.d. sequence from the point of view of the relative frequency of subsequences of bits.
It turns out that an MLS appears as random i.i.d. also from the point of view of the autocorrelation
function. In fact, mapping ‘0’ to ‘−1’ and ‘1’ to ‘+1’, we get the following correlation properties (see
also Theorem 11.1).26
1. Mean
1∑
L−1
1
p(𝓁) = (1.587)
L 𝓁=0 L
2. Correlation (periodic of period L, see (11.159))
{
1∑
L−1 1 for (n)mod L = 0
𝚛p (n) = p(𝓁)p∗ (𝓁 − n)mod L = 1 (1.588)
L 𝓁=0 − otherwise
L
3. PSD (periodic of period L)

( ) ⎧ 1
1 ∑
L−1
−j2𝜋m LT1 nTc ⎪Tc for (m)mod L =0
p m = Tc 𝚛p (n)e c =⎨ L ( ) (1.589)
LTc 1
n=0 ⎪Tc 1 + otherwise
⎩ L
We note that, with the exception of the values assumed for (m)mod L = 0, the PSD of an MLS is
constant.

CAZAC
The constant amplitude zero autocorrelation (CAZAC) sequences, also denoted as Zadoff–Chu
sequences, are complex-valued PN sequences with constant amplitude (assuming values on the
unit circle) and autocorrelation function 𝚛p (n), with 𝚛p (n) = 0 for (n)mod L ≠ 0. Because of these
characteristics they are also called polyphase sequences [23–25]. Let L and M be two integer numbers
that are relatively prime. The CAZAC sequences are defined as
M𝜋𝓁 2
for L even p(𝓁) = ej L , 𝓁 = 0, 1, … , L − 1 (1.590)
M𝜋𝓁(𝓁+1)
for L odd p(𝓁) = ej L , 𝓁 = 0, 1, … , L − 1 (1.591)
It can be shown that these sequences have the following properties:
1. Mean The mean of a CAZAC sequence is usually zero, although this property does not hold in
general.
2. Correlation {
1 for (n) mod L =0
𝚛p (n) = (1.592)
0 otherwise
3. PSD ( )
1
p m = Tc (1.593)
LTc

We note the complexity of generating, storing and processing a CAZAC sequence w.r.t. an MLS.

26
Note that p(𝓁) ∈ {0, 1} when generated, while p(𝓁) ∈ {−1, 1} when used as a signal. The context will clarify the
alphabet of p(𝓁).
102 Chapter 1. Elements of signal theory

Gold
In a large number of applications, as for example in spread-spectrum systems with code-division mul-
tiple access, sets of sequences having one or both of the following properties [26] are required.

● Each sequence of the set must be easily distinguishable from its own time shifted versions.
● Each sequence of the set must be easily distinguishable from any other sequence of the set and from
its time-shifted versions.

An important class of periodic binary sequences that satisfy these properties, or, in other words,
that have good autocorrelation and crosscorrelation characteristics, is the set of Gold sequences
[27, 28].

Construction of pairs of preferred MLSs. In general, the crosscorrelation sequence (CCS) between
two MLSs may assume three, four or a larger number of values. We show now the construction of a
pair of MLSs, called preferred MLSs [22], whose CCS assumes only three values. Let a = {a(𝓁)} be
an MLS with period L = 2r − 1. We define now another MLS of length L = 2r − 1 obtained from the
sequence a by decimation by a factor M, that is

b = {b(𝓁)} = {a(M𝓁)mod L } (1.594)

We make the following assumptions:

● rmod 4 ≠ 0, that is r must be odd or equal to odd multiples of 2, i.e. rmod 4 = 2.

● The factor M satisfies one of the following properties:

M = 2k + 1 or M = 22k − 2k + 1 k integer (1.595)

● For k determined as in the (1.595), defining g.c.d.(r, k) as the greatest common divisor of r and k,
let {
1 r odd
e = g.c.d.(r, k) = (1.596)
2 rmod 4 = 2

Then the CCS between the two MLS a and b assumes only three values [21, 22]:

1∑
L−1
𝚛ab (n) = a(𝓁)b∗ (𝓁 − n)mod L
L 𝓁=0
⎧−1 + 2 r+e
2
r+e−2
(value assumed 2r−e−1 + 2 2 times
1 ⎪
= ⎨ −1 (value assumed 2r − 2r−e − 1 times (1.597)
L⎪ r+e
r−e−1
r+e−2

⎩−1 − 2 2 (value assumed 2 +2 2 times

Example 1.C.1 (Construction of a pair of preferred MLSs)

Let the following MLS of period L = 25 − 1 = 31 be given:

{a(𝓁)} = (0000100101100111110001101110101) (1.598)

As r = 5 and rmod 4 = 1, we take k = 1. Therefore, e = g.c.d.(r, k) = g.c.d.(5, 1) = 1 and M = 2k + 1 =

21 + 1 = 3. The sequence {b(𝓁)} obtained by decimation of the sequence {a(𝓁)} is then given by

{b(𝓁)} = {a(3𝓁)mod L } = (0001010110100001100100111110111) (1.599)

1.C. Pseudo-noise sequences 103

The CCS between the two sequences, assuming 0 is mapped to −1, is

1
{𝚛ab (n)} = (7, 7, −1, −1, −1, −9, 7, −9, 7, 7, −1, −1, 7, 7, −1, 7, −1, −1, −9, −1, −1, −1, −1, −9,
31
− 1, 7, −1, −9, −9, 7, −1) (1.600)
We note that, if we had chosen k = 2, then e = g.c.d.(5, 2) = 1 and M = 22 + 1 = 5, or else M =
2 − 22 + 1 = 13.
2⋅2

Construction of a set of Gold sequences. A set of Gold sequences can be constructed from any pair
{a(𝓁)} and {b(𝓁)} of preferred MLSs of period L = 2r − 1. We define the set of sequences:
G(a, b) = {a, b, a ⊕ b, a ⊕ Zb, a ⊕ Z 2 b, … , a ⊕ Z L−1 b} (1.601)
where Z is the shift operator that cyclically shifts a sequence to the left by a position. The set (1.601)
contains L + 2 = 2r + 1 sequences of length L = 2r − 1 and is called the set of Gold sequences. It can
be proven [27, 28] that, for the two sequences {a′ (𝓁)} and {b′ (𝓁)} belonging to the set G(a, b), the CCS
as well as the ACS, with the exception of zero lag, assume only three values:
{ r+1 r+1
1 −1, −1 − 2 2 , −1 + 2 2 r odd
𝚛a′ b′ (n) = r+2 r+2 (1.602)
L −1, −1 − 2 2 , −1 + 2 2 rmod 4 = 2
Clearly, the ACS of a Gold sequence no longer has the characteristics of an MLS, as is seen in the next
example.

Example 1.C.2 (Gold sequence properties)

Let r = 5, hence, L = 25 − 1 = 31. From Example 1.C.1, the two sequences (1.598) and (1.599) are a pair
of preferred MLSs, from which it is possible to generate the whole set of Gold sequences. For example we
calculate the ACS of {a(𝓁)}and {b′ (𝓁)} = {a(𝓁) ⊕ b(𝓁 − 2)} = a ⊕ Z 2 b, and the CCS between {a(𝓁)}
and {b′ (𝓁)}:
{a(𝓁)} = (−1, −1, −1, −1, 1, −1, −1, 1, −1, 1, 1, −1, −1, 1, 1, 1, 1, 1, −1,
−1, −1, 1, 1, −1, 1, 1, 1, −1, 1, −1, 1) (1.603)

{b′ (𝓁)} = (−1, −1, 1, −1, −1, −1, 1, −1, −1, −1, 1, −1, −1, 1, −1, −1, 1,
1, 1, −1, −1, −1, −1, 1, −1, −1, 1, 1, −1, 1, 1) (1.604)

1
{𝚛a (n)} = (31, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1,
31
−1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1, −1)
(1.605)

1
{𝚛b′ (n)} = (31, −1, −9, 7, 7, −9, −1, 7, −1, −9, 7, 7, −1 − 1, 7, −1, −1, 7,
31
−1, −1, 7, 7, −9, −1, 7, −1, 9, 7, 7, −9, −1) (1.606)

1
{𝚛ab′ (n)} = (−1, 7, 7, 7, −1, −1, −1, −1, −1, 7, −9, −1, −1, 7, −1, −9, 7,
31
−1 − 9, 7, 7, −9, −1, 7, −1, −9, −1, −1, −1, −9, −1) (1.607)
Chapter 2

The Wiener filter

The theory of the Wiener filter [1, 2] that will be presented in this chapter is fundamental for the
comprehension of several important applications. The development of this theory assumes the knowl-
edge of the correlation functions of the relevant processes. An approximation of the Wiener filter can
be obtained by the least squares (LS) method, through realizations of the processes involved.
The section on estimation extends the Wiener filter to a structure with multiple inputs and outputs.
Next, some examples of application of the developed theory are illustrated.

2.1 The Wiener filter

With reference to Figure 2.1, let x and d be two individually and jointly wide sense stationary (WSS)
random processes with zero mean; the problem is to determine the finite impulse system (FIR) filter so
that, if the filter input is x(k), the output y(k) replicates as closely as possible d(k), for each k. The Wiener
theory provides the means to design the required filter.

x(k) x(k – 1) x(k – 2) x(k – N + 1)

Tc Tc Tc

c0 c1 c2 cN – 2 cN – 1

y(k) –
e(k)

+
d(k)

Figure 2.1 The Wiener filter with N coefficients.

The FIR filter in Figure 2.1 is called transversal filter, as the output is formed by summing the products
of the delayed input samples by suitable coefficients. If we indicate with {cn }, n = 0, 1, … , N − 1, the
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
106 Chapter 2. The Wiener filter

N coefficients of the filter, we have

∑
N−1
y(k) = cn x(k − n) (2.1)
n=0

If d(k) is the desired sample at the filter output at instant k, we define the estimation error as

e(k) = d(k) − y(k) (2.2)

In the Wiener theory, in order that the filter output y(k) replicates as closely as possible d(k), the coeffi-
cients of the filter are determined using the minimum mean-square error (MMSE) criterion. Therefore,
the cost function is defined as the mean square error (MSE)

J = E[|e(k)|2 ] (2.3)

and the coefficients of the optimum filter minimize J, i.e.1

copt = arg min J (2.4)

{cn },n=0,1,…,N−1

The Wiener filter problem can be formulated as the problem of estimating d(k) by a linear combination
of x(k), … , x(k − N + 1). A brief introduction to estimation theory is given in Section 2.4, where the
formulation of the Wiener theory is further extended to the case of vector signals.

Matrix formulation
The problem introduced in the previous section is now formulated using matrix notation. We define2 :

1. Coefficient vector
c T = [c0 , c1 , … , cN−1 ] (2.5)

2. Filter input vector at instant k3

xT (k) = [x(k), x(k − 1), … , x(k − N + 1)] (2.6)

The filter output at instant k is expressed as

y(k) = c T x(k) = xT (k) c (2.7)

and the estimation error as

e(k) = d(k) − c T x(k) (2.8)

Moreover,
y∗ (k) = c H x∗ (k) = xH (k)c ∗ , e∗ (k) = d∗ (k) − c H x∗ (k) (2.9)

We express now the cost function J as a function of the vector c. We will then seek vector copt that
minimizes J. Recalling the definition

J = E[e(k)e∗ (k)] = E[(d(k) − xT (k)c)(d∗ (k) − c H x∗ (k))] (2.10)

1
Here arg mina f (a) provides the value of a that minimizes f (a). A similar definition holds for arg maxa f (a).
2
The components of an N-dimensional vector are usually identified by an index varying either from 1 to N or from 0 to
N − 1.
3
Indeed, the theory is more general than the one developed here, and the various inputs to the transversal filter taps need
not to be shifted versions of a given process x(k). In fact, we can replace xT (k) in (2.6) by xT (k) = [x1 (k), x2 (k), … ,
xN (k)], where {xn (k)}, n = 1, … , N, are random processes, and carry out a more general derivation than the one
developed here. Although this general approach is seldom useful, two examples of application are given in (7.193) and
Section 2.5.4.
2.1. The Wiener filter 107

and computing the products, it follows

J = E[d∗ (k)d(k)] − c H E[x∗ (k)d(k)] +
(2.11)
− E[d∗ (k)xT (k)]c + c H E[x∗ (k)xT (k)]c
Assuming that x and d are individually and jointly WSS, we introduce the following quantities:
1. Variance of the desired signal
𝜎d2 = E[d(k)d∗ (k)] (2.12)
2. Correlation between the desired output at instant k and the filter input vector at the same instant
⎡ E [d(k)x∗ (k)] ⎤
⎢ ⎥
[ ∗
] ⎢ E [d(k)x∗ (k − 1)] ⎥
rdx = E d(k)x (k) = ⎢ ⎥=p (2.13)
⎢ ⋮ ⎥
⎢ ⎥
⎣E [d(k)x (k − N + 1)]⎦
∗

The components of p are given by

[ ]
[p]n = E d(k)x∗ (k − n) = 𝚛dx (n), n = 0, 1, … , N − 1 (2.14)
Moreover, it holds
[ ]
p H = E d∗ (k)xT (k) (2.15)
3. N × N autocorrelation matrix of the complex-conjugate filter input vector x∗ (k), as defined in
(1.270),
[ ]
R = E x∗ (k)xT (k) (2.16)

Then
J = 𝜎d2 − c H p − p H c + c H Rc (2.17)
The cost function J, considered as a function of c, is a quadratic function. Then, if R is positive definite,
J admits one and only one minimum value. J as a function of c is shown in Figure 2.2, for the particular
cases N = 1 and N = 2.

Optimum filter design

It is now a matter of finding the minimum of (2.17) with respect to c. Recognizing that, as c = cI + jcQ ,
the real independent variables are 2N, to accomplish this task, we define the derivative of J with respect
to c as the gradient vector
⎡ 𝜕J 𝜕J ⎤
⎢ 𝜕c + j 𝜕c ⎥
⎢ 0,I 0,Q
⎥
⎢ 𝜕J 𝜕J ⎥
𝜕J ⎢ +j
∇c J = = 𝜕c1,I 𝜕c1,Q ⎥ (2.18)
𝜕c ⎢ ⎥
⎢ ⋮ ⎥
⎢ 𝜕J 𝜕J ⎥
⎢ 𝜕c +j
⎣ N−1,I 𝜕cN−1,Q ⎥⎦
and also
∇c J = ∇cI J + j∇cQ J (2.19)
In general, because the vector p and the autocorrelation matrix R are complex, we also write
p = pI + jpQ (2.20)
108 Chapter 2. The Wiener filter

J
J

N=1 c1

Jmin
Jmin copt,1

copt c 0 copt,0 c0
(a) (b)

c1

J1 J2
copt,1 Jmin J0

0 copt,0 c0
(c)

Figure 2.2 J as a function of c for the cases N = 1 and N = 2. (a) J versus c for N = 1, (b) J versus
c for N = 2, and (c) equal-J contours for N = 2.

and
R = RI + jRQ (2.21)

If now we take the derivative of the terms of (2.17) by the real and imaginary part of c, we find

∇cI p H c = ∇cI p H (cI + jcQ ) = p H (2.22a)

H H H
∇cQ p c = ∇cQ p (cI + jcQ ) = jp (2.22b)
∇cI c H p = ∇cI (cTI − jcQ T )p = p (2.22c)
∇cQ c H p = ∇cQ (cTI − jcTQ )p = −jp (2.22d)
H
∇cI (c Rc) = 2RI cI − 2RQ cQ (2.22e)
H
∇cQ (c Rc) = 2RI cQ + 2RQ cI (2.22f)

From (2.22) and (2.19), we obtain

∇c p H c = 𝟎 (2.23)

∇c c H p = 2p (2.24)

∇c c H Rc = 2Rc (2.25)

Substituting the above results into (2.17), it turns out

∇c J = −2p + 2Rc = 2(Rc − p) (2.26)

2.1. The Wiener filter 109

For the optimum coefficient vector copt , it must be ∇c J = 𝟎, where 𝟎 is a vector of all zeros, hence, we
get
Rcopt = p (2.27)
Equation (2.27) is called the Wiener–Hopf (W–H) equation.

Observation 2.1
The computation of the optimum coefficients copt requires the knowledge only of the input correlation
matrix R and of the crosscorrelation vector p between the desired output and the input vector.

In scalar form, the W–H equation is a linear system of N equations in N unknowns:

∑
N−1
copt,i 𝚛x (n − i) = 𝚛dx (n), n = 0, 1, … , N − 1 (2.28)
i=0

If R−1 exists, the solution of (2.27) is

copt = R−1 p (2.29)

The principle of orthogonality

It is interesting to observe the relation
[ ] [ ]
E e(k)x∗ (k) = E x∗ (k)(d(k) − xT (k)c) = p − Rc (2.30)
which for c = copt yields
[ ]
E e(k)x∗ (k) = 𝟎 (2.31)
We now state the following important result.

Theorem 2.1 (Principle of orthogonality)

The condition of optimality for c is satisfied if e(k) and x(k) are orthogonal.4 In scalar form, the filter is
optimum if e(k) is orthogonal to {x(k − n)}, n = 0, 1, … , N − 1, that is
[ ]
E e(k)x∗ (k − n) = 0, n = 0, 1, … , N − 1 (2.32)
◽
This is equivalent to say that the optimum-filter output y(k) reproduces in some statistical sense all
components of d(k) proportional to x(k), x(k − 1), … , x(k − N + 1).

Corollary 2.1
For c = copt , e(k) and y(k) are orthogonal, i.e.
[ ]
E e(k)y∗ (k) = 0 for c = copt (2.33)
◽
In fact, using the orthogonality principle,
[ ] [ ]
E e(k)y∗ (k) = E e(k)c H x∗ (k)
= c H E [e(k)x∗ (k)]
(2.34)
= cH 𝟎
=0
For an optimum filter, Figure 2.3 depicts the relation between the three signals d, e, and y.
4
Note that orthogonality holds only if e and x are considered at the same instant. Indeed, there could be statistical
correlation between e(k) and x(k − M) with M ≥ N.
110 Chapter 2. The Wiener filter

d(k)
e(k)

y(k)

Figure 2.3 Orthogonality of signals for an optimum filter.

Expression of the minimum mean-square error

We now determine the value of the cost function J in correspondence of copt . Substituting the expression
(2.27) of copt in (2.17), we get

Jmin = 𝜎d2 − cH H H
opt p − p copt + copt p
(2.35)
= 𝜎d − p copt
2 H

We note that R in (2.27) can be ill-conditioned if N is very large. Hence, as for the order of the
auto-regressive moving-average (ARMA) model (see Section 1.10), also here N should be determined
in an iterative way by evaluating Jmin (2.35) for increasing values of N. We stop when Jmin does not
significantly change, i.e. it reaches approximately its asymptotic value.
Another useful expression of Jmin is obtained from (2.2):

d(k) = e(k) + y(k) (2.36)

As e(k) and y(k) are orthogonal for c = copt , then

𝜎d2 = Jmin + 𝜎y2 (2.37)

whereby it follows
Jmin = 𝜎d2 − 𝜎y2 (2.38)

Using (2.35), we can find an alternative expression to (2.17) for the cost function J:

J = Jmin + (c − copt )H R(c − copt ) (2.39)

Recalling that the autocorrelation matrix is positive semi-definite, it follows that the quantity
(c − copt )H R(c − copt ) is non-negative and in particular it vanishes for c = copt .

Characterization of the cost function surface

The result (2.39) calls for further observations on J. In fact, using the decomposition (1.292), we get

J = Jmin + (c − copt )H U𝚲UH (c − copt ) (2.40)

Let us now define

⎡ 𝜈1 ⎤
𝝂 = ⎢ ⋮ ⎥ = UH (c − copt ) (2.41)
⎢ ⎥
⎣𝜈N ⎦
2.1. The Wiener filter 111

where 𝜈i = uH
i
(c − copt ). Vector 𝝂 may be interpreted as a translation and a rotation of vector c. Then J
assumes the form:
J = Jmin + 𝝂 H 𝚲𝝂
∑N
= Jmin + 𝜆i |𝜈i |2
i=1 (2.42)
∑N
| | 2
= Jmin + 𝜆i |uH (c − copt )|
| i |
i=1

The result (2.42) expresses the excess mean-square error J − Jmin as the sum of N components in the
direction of each eigenvector of R. Note that each component is proportional to the corresponding eigen-
value.
From the above observation, we conclude that J increases more rapidly in the direction of the eigen-
vector corresponding to the maximum eigenvalue 𝜆max . Likewise, the increase is slower in the direction
of the eigenvector corresponding to the minimum eigenvalue 𝜆min . Let u𝜆max and u𝜆min denote the eigen-
vectors of R in correspondence of 𝜆max and 𝜆min , respectively; it follows that ∇c J is largest along u𝜆max .
This is also observed in Figure 2.4, where sets (loci) of points c for which a constant value of J is
obtained are graphically represented. In the two-dimensional case, they trace ellipses with axes parallel
to the direction of the eigenvectors and the ratio of axes length is related to the eigenvalues.

c1 c1
v1 v1

u1 u1
v2 v2
u2 c0 c0
u2
(a) (b)

Figure 2.4 Loci of points with constant J (contour plots). (a) Case 𝜆1 ≃ 𝜆2 and (b) case 𝜆1 < 𝜆2 .

The Wiener filter in the z-domain

For a filter with an infinite number of coefficients, not necessarily causal, (2.28) of the optimum filter
becomes
∑
+∞
copt,i 𝚛x (n − i) = 𝚛dx (n) ∀n (2.43)
i=−∞

Taking the z-transform of both sides of the equation yields

Copt (z)Px (z) = Pdx (z) (2.44)

Then the transfer function of the optimum filter is

Pdx (z)
Copt (z) = (2.45)
Px (z)
We note that while (2.29) is useful to evaluate the coefficients of the optimum FIR filter, (2.45) is
employed to analyse the system in the general case of an infinite impulse response (IIR) filter.
112 Chapter 2. The Wiener filter

d(k)
h
+
–
c
x(k) y(k) e(k)

Figure 2.5 An application of the Wiener filter theory.

Example 2.1.1
Let d(k) = h ∗ x(k), as shown in Figure 2.5. In this case, from Table 1.3,
Pdx (z) = Px (z)H(z) (2.46)
and the optimum filter is given by
Copt (z) = H(z) (2.47)

From (2.35) in scalar notation, applying Fourier transform properties, we get

∑
N−1
Jmin = 𝜎d2 − copt,i 𝚛∗dx (i)
i=0
1 (2.48)
2Tc
= 𝜎d2 − dx
∗
(f )Copt (ej2𝜋f Tc ) df
∫− 1
2Tc

where dx
∗
(f ) is the Fourier transform of 𝚛∗dx (i).
Using (2.45), we have
1
2Tc dx (f )
Jmin = 𝜎d2 − dx
∗
(f ) df
∫− 1 x (f )
2Tc
1
2Tc |dx (f )|2
= 𝜎d2 − df (2.49)
∫− 1 x (f )
2Tc
1
2Tc |Pdx (ej2𝜋f Tc )|2
= 𝜎d2 − Tc df
∫− 1 Px (ej2𝜋f Tc )
2Tc

where x (f ) is the Fourier transform of 𝚛x (n).

Example 2.1.2
We desire to filter the noise from a signal given by one complex sinusoid (tone) plus noise, i.e.
x(k) = Aej(𝜔0 k+𝜑) + 𝑤(k) (2.50)
In (2.50) 𝜔0 = 2𝜋f0 Tc is the tone radian frequency normalized to the sampling period, in radians. We
assume the desired signal is given by
d(k) = B ej[𝜔0 (k−D)+𝜑] (2.51)
where D is a known delay. We also assume that 𝜑 ∼  (0, 2𝜋), and 𝑤 is white noise with zero mean and
variance 𝜎𝑤2 , uncorrelated with 𝜑. The autocorrelation function of x and the crosscorrelation between d
and x are given by
𝚛x (n) = A2 ej𝜔0 n + 𝜎𝑤2 𝛿n (2.52)
𝚛dx (n) = AB e j𝜔0 (n−D)
(2.53)
2.1. The Wiener filter 113

For a Wiener filter with N coefficients, the autocorrelation matrix R and the vector p have the following
structure:
⎡ A2 + 𝜎𝑤2 A2 e−j𝜔0 … A2 e−j𝜔0 (N−1) ⎤
⎢ A2 ej𝜔0 A2 + 𝜎𝑤2 … A2 e−j𝜔0 (N−2) ⎥⎥
R=⎢ (2.54)
⎢ ⋮ ⋮ ⋱ … ⎥
⎢A2 ej𝜔0 (N−1) A2 ej𝜔0 (N−2) … A2 + 𝜎𝑤2 ⎥⎦
⎣
⎡ 1 ⎤
⎢ j𝜔0 ⎥
e
p=⎢ ⎥ AB e−j𝜔0 D (2.55)
⎢ ⋮ ⎥
⎢ej𝜔0 (N−1) ⎥
⎣ ⎦
Defining
[ ]
ET (𝜔) = 1, ej𝜔 , … , ej𝜔(N−1) (2.56)
we can express R and p as
R = 𝜎𝑤2 I + A2 E(𝜔0 )EH (𝜔0 ) (2.57)

p = ABe−j𝜔0 D E(𝜔0 ) (2.58)

Observing that EH (𝜔)E(𝜔) = N, the inverse of R is given by
[ ]
−1 1 A2 H
R = 2 I− 2 E(𝜔0 )E (𝜔0 ) (2.59)
𝜎𝑤 𝜎𝑤 + NA2
Hence, using (2.29):
−j𝜔 D
ABe−j𝜔0 D B Λe 0
copt = E(𝜔0 ) = E(𝜔0 ) (2.60)
𝜎𝑤 + NA
2 2 A 1 + NΛ

where Λ = A2 ∕𝜎𝑤2 is the signal-to-noise ratio. From (2.35) the minimum value of the cost function J is
given by
ABe−j𝜔0 D B2
Jmin = B2 − ABej𝜔0 D EH (𝜔0 )E(𝜔0 ) 2 = (2.61)
𝜎𝑤 + NA 2 1 + NΛ
Defining 𝜔 = 2𝜋f Tc , the optimum filter frequency response is given by
∑
N−1
Copt (ej𝜔 ) = copt,i e−j𝜔i
i=0

= EH (𝜔)copt (2.62)

B Λe 0 ∑ −j(𝜔−𝜔0 )i
−j𝜔 D N−1
= e
A 1 + NΛ i=0
that is,
⎧ B NΛe−j𝜔0 D
⎪ 𝜔 = 𝜔0
j𝜔 ⎪ A 1 + NΛ
Copt (e ) = ⎨ (2.63)
−j𝜔 D
⎪ B Λe 0 1 − e−j(𝜔−𝜔0 )N
⎪ A 1 + NΛ 1 − e−j(𝜔−𝜔0 ) 𝜔 ≠ 𝜔0
⎩
We observe that, for Λ ≫ 1,
1. Jmin becomes negligible;
114 Chapter 2. The Wiener filter

B −j𝜔0 D
2. copt = e E(𝜔0 );
AN
B
3. |Copt (ej𝜔0 )| = .
A
Conversely, for Λ → 0, i.e. when the power of the useful signal is negligible with respect to the power
of the additive noise, it results in
1. Jmin = B2 ;
2. copt = 𝟎;
3. |Copt (ej𝜔0 )| = 0.
Indeed, as the signal-to-noise ratio vanishes, the best choice is to set the output y to zero. The plot of
|Copt (ej2𝜋f Tc )| is given in Figure 2.6 for some values of the parameters.

–5

–10
Copt(ej2πf Tc)(dB)

–15

–20

–25

–30

0 f0 = 1/(2Tc) 1/Tc
f

Figure 2.6 Magnitude of Copt (ej2𝜋f Tc ) given by (2.63) for f0 Tc = 1∕2, B = A, Λ = 30 dB, and N = 35.

2.2 Linear prediction

The Wiener theory considered in the Section 2.1 has an important application to the solution of the
following problem. Let x be a discrete-time WSS random process with zero mean; prediction (or regres-
sion) consists in estimating a future value of the process starting from a set of known past values. In
particular, let us define the vector
xT (k − 1) = [x(k − 1), x(k − 2), … , x(k − N)] (2.64)
The one-step forward predictor of order N, given xT (k − 1), attempts to estimate the value of x(k). There
exists also the problem of predicting x(k − N), given the values of x(k − N + 1), … , x(k). In this case,
the system is called the one step backward predictor of order N.
2.2. Linear prediction 115

x(k) x(k – 1) x(k – 2) x(k – N )

Tc Tc Tc

c1 c2 cN – 1 cN

x^(k| x (k – 1))

Figure 2.7 Linear predictor of order N.

Forward linear predictor

The estimate of x(k) is expressed as a linear combination of the preceding N samples:
∑
N
x̂ (k|x(k − 1)) = ci x(k − i) (2.65)
i=1
The block diagram of the linear predictor is represented in Figure 2.7.
This estimate will be subject to a forward prediction error given by
fN (k) = x(k) − x̂ (k|x(k − 1))
∑ N
(2.66)
= x(k) − ci x(k − i)
i=1

Optimum predictor coefficients

If we adopt the criterion of minimizing the mean-square prediction error,
[ ]
J = E |fN (k)|2 (2.67)
to determine the predictor coefficients, we can use the optimization results according to Wiener. We
recall the following definitions.
1. Desired signal
d(k) = x(k) (2.68)
2. Filter input vector (defined by (2.64))
xT (k − 1) (2.69)
3. Cost function J given by (2.67).
Then it turns out:
[ ]
𝜎d2 = E x(k)x∗ (k) = 𝜎x2 = 𝚛x (0) (2.70)
[ ]
E x∗ (k − 1)xT (k − 1) = RN (2.71)
with RN the N × N correlation matrix of x∗ , and
⎡ 𝚛x (1) ⎤
[ ] [ ] ⎢ ⎥
𝚛 (2)
p = E d(k)x∗ (k − 1) = E x(k)x∗ (k − 1) = ⎢ x ⎥ = rN (2.72)
⎢ ⋮ ⎥
⎢𝚛 (N)⎥
⎣ x ⎦
116 Chapter 2. The Wiener filter

Applying (2.27) the optimum coefficients satisfy

RN copt = rN (2.73)
Moreover, from (2.35), we get the minimum value of the cost function
Jmin = JN = 𝚛x (0) − rH
N copt (2.74)
We can combine the latter two equations to get an augmented form of the W–H equation (2.27) for the
linear predictor: [ ][ ] [ ]
𝚛x (0) rH 1 J
N = N (2.75)
rN RN −copt 𝟎N
where 𝟎N is the column vector of N zeros.

Forward prediction error filter

We determine the filter that gives the forward linear prediction error fN . For an optimum predictor,
∑
N
fN (k) = x(k) − copt,i x(k − i) (2.76)
i=1

We introduce the vector {

1 i=0
𝚊′i,N = (2.77)
−copt,i i = 1, 2, … , N
which can be rewritten as [ ]
1
a′N = (2.78)
a
where a = −copt . Substituting (2.77) in (2.76) and taking care of extending the summation also to i = 0,
we obtain
∑
N
fN (k) = 𝚊′i,N x(k − i) (2.79)
i=0

as shown in Figure 2.8.

T
The coefficients aN′ = [𝚊′0,N , 𝚊′1,N , … , 𝚊′N,N ] are directly obtained by substituting (2.78) in (2.75):
[ ]
J
RN+1 a′N = N (2.80)
𝟎N

x(k) x(k – 1) x(k – 2) x(k – N )

Tc Tc Tc

a′0,N a′1,N a′2,N a′N–1,N a′N,N

fN (k)

Figure 2.8 Forward prediction error filter.

2.2. Linear prediction 117

With a similar procedure, we can derive the filter that gives the backward linear prediction error,
∑
N
bN (k) = x(k − N) − gi x(k − i + 1) (2.81)
i=1

It can be shown that the optimum coefficients are given by

gopt = cB∗
opt (2.82)
where B is the backward operator that orders the elements of a vector backward, from the last to the first
(see page 14).
Efficient algorithms to solve (2.80) are given in Appendix 2.A.

Relation between linear prediction and AR models

The similarity of (2.73) with the Yule–Walker equation (1.464) allows us to state that, given an
auto-regressive (AR) process x of order N, the optimum prediction coefficients copt coincide with the
parameters −a of the process and JN = 𝜎𝑤2 . Actually, for copt = −a, comparing (2.76) with (1.445)
we have fN (k) = 𝑤(k), that is, the prediction error fN coincides with the white noise 𝑤(k) at the input
of the AR system. In general, if the prediction error filter has a large enough order, it has whitening
properties, as it is capable of removing the correlated signal component at the input, producing at the
output only the uncorrelated or white component. Moreover, while prediction can be interpreted as the
analysis of an AR process, the AR model may be regarded as the synthesis of the process. As illustrated
in Figure 2.9, given a realization of the process {x(k)}, by estimating the autocorrelation sequence
over a suitable observation window, the parameters copt and JN can be determined. Using the predictor
then we determine the prediction error {fN (k)}. To reproduce {x(k)}, an all-pole filter with coefficients
a′T
N
= [1, −copt ], having white noise {𝑤(k) = fN (k)} of power 𝜎𝑤2 = JN as input, can be used.
An interesting application of this scheme is in the voice-coding [3–5], where the generic subsequence
of input voice samples {x(k)}, say of length Lx , yields a given {fN (k)} which is compared against a set of
2b predetermined vectors of length Lx and the closest one is selected. This information of b bits together
with quantized values of copt and JN is sent to the receiver where the synthesis scheme is applied to
reproduce an approximation of the input voice signal.

First- and second-order solutions

We give now formulas to compute the predictor filter coefficients and prediction error filter coefficients
for orders N = 1 and N = 2. In the next equations, 𝜌(n) is the correlation coefficient of x, introduced in
(1.467).

a = –copt

Analysis w(k) = fN (k) x(k)

1
A(z)
copt, JN
(σ2w= JN)
x(k) fN (k)
Predictor A(z) = a′0, N + a′1, N z –1 + ... + a′N, N z –N
=1 + a1 z –1 + ... + aN z –N

(a) (b)

Figure 2.9 (a) Analysis and (b) synthesis of AR(N) processes.

118 Chapter 2. The Wiener filter

These results extend to the complex case the formulas obtained in Section 1.10.2.
● N = 1. From [ ][ ] [ ]
𝚛x (0) 𝚛∗x (1) 𝚊′0,1 J
= 1 (2.83)
𝚛x (1) 𝚛x (0) 𝚊′1,1 0
it results
⎧ J
⎪ 𝚊′ = 1 𝚛x (0)
⎪ 0,1 Δ𝚛
⎨ (2.84)
⎪𝚊′ = − J1 𝚛 (1)
⎪ 1,1 Δ𝚛 x
⎩
where
|𝚛 (0) 𝚛∗ (1)|
| |
Δ𝚛 = | x x
| = 𝚛2x (0) − |𝚛x (1)|2 (2.85)
|𝚛x (1) 𝚛x (0) |
| |
As 𝚊′0,1 = 1, it turns out

⎧ Δ𝚛
⎪J1 = ⎧copt,1 = −𝚊′ = 𝜌(1)
⎪ 𝚛x (0) ⎪ 1,1
⎨ ⇒ ⎨ J1 (2.86)
⎪𝚊′ = − 𝚛x (1) ⎪ = 1 − |𝜌(1)|2
⎪ 1,1 𝚛x (0) ⎩ 𝚛x (0)
⎩
● N = 2.
⎧ 𝚛x (1)𝚛x (0) − 𝚛∗x (1)𝚛x (2) ⎧ 𝜌(1) − 𝜌∗ (1)𝜌(2)
⎪𝚊′1,2 = − ⎪copt,1 =
⎪ 𝚛2x (0) − |𝚛x (1)|2 ⎪ 1 − |𝜌(1)|2
⎨ ⇒ ⎨ (2.87)
⎪𝚊′ = − 𝚛x (0)𝚛x (2) − 𝚛x (1) 𝜌(2) − 𝜌2 (1)
2
⎪c =
⎪ 2,2
𝚛2x (0) − |𝚛x (1)|2 ⎪ opt,2
1 − |𝜌(1)|2
⎩ ⎩
and
J2 1 − 2|𝜌(1)|2 + 𝜌∗2 (1)𝜌(2) + |𝜌(1)|2 𝜌∗ (2) − |𝜌(2)|2
= (2.88)
𝚛x (0) 1 − |𝜌(1)|2

2.3 The least squares method

The Wiener filter will prove to be a powerful analytical tool in various applications, one of which is
indeed prediction. However, from a practical point of view, often only realizations of the processes
{x(k)} and {d(k)} are available. Therefore, to get the solution, it is necessary to determine estimates of
𝚛x and 𝚛dx , and various alternatives emerge. Two possible methods are (1) the autocorrelation method,
in which from the estimate of 𝚛x , we construct R as a Toeplitz correlation matrix, and (2) the covariance
method, in which we estimate each element of R by (2.94). In this case, the matrix is only Hermitian
and the solution that we are going to illustrate is of the LS type [1, 2].
We reconsider the problem of Section 2.1, introducing a new cost function. Based on the observation
of the sequences
{x(k)} and {d(k)}, k = 0, … , K − 1 (2.89)
and of the error
e(k) = d(k) − y(k) (2.90)
2.3. The least squares method 119

where y is given by (2.1), according to the LS method the optimum filter coefficients yield the minimum
of the sum of the square errors:
cLS = arg min  (2.91)
{cn },n=0,1,…,N−1

where
∑
K−1
= |e(k)|2 (2.92)
k=N−1

Note that in the LS method, a time average is substituted for the expectation (2.3), which gives
the MSE.

Data windowing
In matrix notation, the output {y(k)}, k = N − 1, … , K − 1, given by (2.1), can be expressed as
⎡y(N − 1)⎤ ⎡x(N − 1) x(N − 2) … x(0) ⎤ ⎡ c0 ⎤
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ y(N) ⎥ = ⎢ x(N) x(N − 1) … x(1) ⎥ ⎢ c1 ⎥ (2.93)
⎢ ⋮ ⎥ ⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥
⎢y(K − 1)⎥ ⎢x(K − 1) x(K − 2) … x(K − N)⎥ ⎢c ⎥
⎣ ⎦ ⎣ ⎦ ⎣ N−1 ⎦
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟
input data matrix 

In (2.93), we note that the input data sequence goes from x(0) to x(K − 1). Other choices are possible for
the input data window. The examined case is called the covariance method, and the input data matrix
, defined by (2.93), is L × N, where L = K − N + 1.

Matrix formulation
We define
∑
K−1
Φ(i, n) = x∗ (k − i) x(k − n), i, n = 0, 1, … , N − 1 (2.94)
k=N−1

∑
K−1
𝜗(n) = d(k) x∗ (k − n), n = 0, 1, … , N − 1 (2.95)
k=N−1

Using (1.405) for an unbiased estimate of the correlation, the following identities hold:

Φ(i, n) = (K − N + 1)𝚛̂ x (i − n) (2.96)

𝜗(n) = (K − N + 1)𝚛̂ dx (n) (2.97)

in which the values of Φ(i, n) depend on both indices (i, n) and not only upon their difference, especially
if K is not very large. We give some definitions:

1. Energy of {d(k)}
∑
K−1
d = |d(k)|2 (2.98)
k=N−1

2. Crosscorrelation vector between d and x

[ ]
𝝑 T = 𝜗(0), 𝜗(1), … , 𝜗(N − 1) (2.99)
120 Chapter 2. The Wiener filter

3. Input autocorrelation matrix

⎡ Φ(0, 0) Φ(0, 1) … Φ(0, N − 1) ⎤
⎢ ⎥
Φ(1, 0) Φ(1, 1) … Φ(1, N − 1) ⎥
𝚽=⎢ (2.100)
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢Φ(N − 1, 0) Φ(N − 1, 1) … Φ(N − 1, N − 1)⎥⎦
⎣

Then the cost function can be written as

 = d − c H 𝝑 − 𝝑 H c + c H 𝚽c (2.101)

Correlation matrix
𝚽 is the time average of x∗ (k)xT (k), i.e.
∑
K−1
𝚽= x∗ (k)xT (k) (2.102)
k=N−1

Properties of 𝚽
1. 𝚽 is Hermitian.
2. 𝚽 is positive semi-definite.
3. Eigenvalues of 𝚽 are real and non-negative.
4. 𝚽 can be written as
𝚽 = H  (2.103)
with  the input data matrix defined by (2.93). We note that matrix  is Toeplitz.

Determination of the optimum filter coefficients

By analogy of (2.101) with (2.17), the gradient of (2.101) is given by
∇c  = 2(𝚽c − 𝝑) (2.104)
Then the vector of optimum coefficients based on the LS method, cLS , satisfies the normal equation:
𝚽cLS = 𝝑 (2.105)
If 𝚽 −1
exists, then the solution to (2.105) is given by
cLS = 𝚽−1 𝝑 (2.106)
In the solution of the LS problem, (2.105) corresponds to the W–H equation (2.27). For an ergodic
process, (2.96) yields:
1
𝚽 −−−−→ R (2.107)
K−N+1 k→∞

and
1
𝝑 −−−−→ p (2.108)
K − N + 1 k→∞
Hence, we have that the LS solution converges to the Wiener solution as K goes to infinity, that is
cLS −−−−→ copt (2.109)
k→∞

In other words, for K → ∞, the covariance method gives the same solution as the autocorrelation
method.
2.3. The least squares method 121

In scalar notation, (2.105) becomes a system of N equations in N unknowns:

∑
N−1
Φ(n, i)cLS,i = 𝜗(n), n = 0, 1, … , N − 1 (2.110)
i=0

2.3.1 The principle of orthogonality

From (2.92), taking the gradient with respect to cn , we have
∇cn  = ∇cn ,I  + j∇cn ,Q 
∑ [
K−1
= −x∗ (k − n)e(k) − x(k − n)e∗ (k)
k=N−1
] (2.111)
+ j (jx∗ (k − n)e(k) − jx(k − n)e∗ (k))
∑
K−1
= −2 x∗ (k − n)e(k)
k=N−1

If we denote with {emin (k)}, the estimation error found with the optimum coefficient cLS , they must
satisfy the conditions:
∑
K−1
emin (k)x∗ (k − n) = 0, n = 0, 1, … , N − 1 (2.112)
k=N−1

which represent the time-average version of the statistical orthogonality principle (2.31). Moreover,
being y(k) a linear combination of {x(k − n)}, n = 0, 1, … , N − 1, we have
∑
K−1
emin (k)y∗ (k) = 0 (2.113)
k=N−1

which expresses the fundamental result: the optimum filter output sequence is orthogonal to the minimum
estimation error sequence.

Minimum cost function

Substituting (2.105) in (2.101), the minimum cost function can be written as
min = d − 𝝑 H cLS (2.114)
An alternative expression to min uses the energy of the output sequence:
∑
K−1
y = |y(k)|2 = c H 𝚽c (2.115)
k=N−1

Note that for c = cLS , we have

d(k) = y(k) + emin (k) (2.116)
then, because of the orthogonality (2.113) between y and emin , it follows that
d = y + min (2.117)
from which, substituting (2.105) in (2.115), we get
min = d − y (2.118)
where y = cH
LS
𝝑.
122 Chapter 2. The Wiener filter

The normal equation using the data matrix

Defining the vector of desired samples
dT = [d(N − 1), d(N), … , d(K − 1)] (2.119)
from the definition (2.95) of 𝜗(n) we get
⎡ 𝜗(0) ⎤ ⎡x∗ (N − 1) x∗ (N) … x∗ (K − 1) ⎤ ⎡d(N − 1)⎤
⎢ 𝜗(1) ⎥ ⎢x∗ (N − 2) x∗ (N − 1) … x∗ (K − 2) ⎥ ⎢ d(N) ⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎢ ⋮ ⎥=⎢ ⋮ ⋮ ⋱ ⋮ ⎥⎢ ⋮ ⎥ (2.120)
⎢𝜗(N − 1)⎥ ⎢ x∗ (0) x∗ (2) … x∗ (K − N)⎥ ⎢d(K − 1)⎥
⎢ ⎥ ⎢ ⎥⎢ ⎥
⎣ ⎦ ⎣ ⎦⎣ ⎦
that is
𝝑 = H d (2.121)
Thus, using (2.103) and (2.121), the normal equation (2.105) becomes
 H cLS =  H d (2.122)
Associated with system (2.122), it is useful to introduce the system of equations for the minimization
of ,
c = d (2.123)
From (2.122), if ( H )−1 exists, the solution is
cLS = ( H )−1  H d (2.124)
and correspondingly (2.114) becomes
min = dH d − dH ( H )−1  H d (2.125)
We note how both formulas (2.124) and (2.125) depend only on the desired signal samples and input
samples. Moreover, the solution c is unique only if the columns of  are linearly independent, that is the
case of non-singular  H . This requires at least K − N + 1 > N, that is the system of equations (2.123)
must be overdetermined (more equations than unknowns).

Geometric interpretation: the projection operator

In general, from (2.93) the vector of filter output samples
[ ]
yT = y(N − 1), y(N), … , y(K − 1) (2.126)
can be related to the input data matrix  as
y = c (2.127)
This relation still holds for c = cLS , and from (2.124), we get
y = cLS = ( H )−1  H d (2.128)
Correspondingly, the estimation vector error is given by
emin = d − y (2.129)
The matrix  = ( H )−1  H can be thought of as a projection operator defined on the space gener-
ated by the columns of . Let I be the identity matrix. The difference
 ⟂ = I −  = I − ( H )−1  H (2.130)
2.3. The least squares method 123

d emin

Figure 2.10 Relations among vectors in the LS minimization.

is the complementary projection operator, orthogonal to . In fact, from (2.128)

y = d (2.131)

and from (2.129)

emin = d − y =  ⟂ d (2.132)

where emin ⟂ y (see (2.113)). Moreover, (2.125) can be written as

min emin = d emin = d ⟂ d

min = eH H H
(2.133)

Figure 2.10 shows an example illustrating the relation among d, y, and emin .

2.3.2 Solutions to the LS problem

If the inverse of ( H ) does not exist, the solution of the LS problem (2.124) must be re-examined. This
is what we will do in this section after taking a closer look at the associated system of equations (2.123).
In general, let us consider the solutions to a linear system of equations

c = d (2.134)

with  on N × N square matrix. If  −1 exists, the solution c =  −1 d is unique and can be obtained in
various ways [6]:

1. If  is triangular and non-singular, a solution to the system (2.134) can be found by the successive
substitutions method with O(N 2 ) operations.
2. In general, if  is non-singular, one can use the Gauss method, which involves three steps:
a. Factorization of 
 = LU (2.135)

with L lower triangular having all ones along the diagonal and U upper triangular;
b. Solution of the system in z
Lz = d (2.136)

through the successive substitutions method;

c. Solution of the system in c
Uc = z (2.137)

through the successive substitutions method.

This method requires O(N 3 ) operations and O(N 2 ) memory locations.
124 Chapter 2. The Wiener filter

3. If  is Hermitian and non-singular, the factorization (2.135) becomes the Cholesky decomposition:

 = LL H (2.138)

with L lower triangular having non-zero elements on the diagonal. This method requires O(N 3 )
operations, about half as many as the Gauss method.
4. If  is Toeplitz and non-singular, the generalized Schur algorithm can be used, with a complexity
of O(N 2 ): generally, it is applicable to all  structured matrices [7]. We also recall the Kumar fast
algorithm [8].

However, if  −1 does not exist, e.g. because  is not a square matrix, it is necessary to use alternative
methods to solve the system (2.134) [6]: in particular, we will consider the method of the pseudoinverse.
First, we will state the following result:

Singular value decomposition

We have seen in (1.292) how the N × N Hermitian matrix R can be decomposed in terms of a matrix
U of eigenvectors and a diagonal matrix 𝚲 of eigenvalues. Now, we extend this concept to an arbitrary
complex matrix . Given an L × N matrix  of rank R, two unitary matrices V and U exist, the singular
value decomposition (SVD) of  is
 = U𝚺V H
∑R
(2.139)
= 𝜎i ui 𝒗H
i
i=1

with [ ]
D 𝟎
𝚺= (2.140)
𝟎 𝟎 L×N

D = diag(𝜎1 , 𝜎2 , … , 𝜎R ), 𝜎1 ≥ 𝜎2 ≥ · · · ≥ 𝜎R > 0 (2.141)

[ ]
U = u1 , u2 , … , uL L×L , H
U U = IL×L (2.142)
[ ]
V = 𝒗1 , 𝒗2 , … , 𝒗N N×N , H
V V = IN×N (2.143)

In (2.141) {𝜎i }, i = 1, … , R, are singular values of . Being U and V unitary, it follows

UH V = 𝚺 (2.144)

as illustrated in Figure 2.11.

D 0 R
N V
L UH L  L
0 0
N

L N N

Figure 2.11 Singular value decomposition of matrix .

2.3. The least squares method 125

The following facts hold

● Matrices  H  and  H have the same non-zero eigenvalues, 𝜆i , i = 1, … , R, and
√
𝜎i = 𝜆i i = 1, … , R (2.145)
● ui , i = 1, … , R, can be normalized (to unit norm) eigenvectors of  H . Moreover, uj , j = R +
1, … , L, are obtained by including successive vectors of length L orthonormal to previous selected
vectors, i.e. uH u = 0, i = 1, … , j − 1, j = R + 1, … , L. {ui } are denoted left singular vectors of .
i j
● 𝒗i , i = 1, … , R, can be normalized eigenvectors of  H . Moreover, 𝒗j , j = R + 1, … , N, are
obtained by including successive vectors of length N orthonormal to previous selected vectors, i.e.
𝒗H 𝒗 = 0, i = 1, … , j − 1, j = R + 1, … , N. {𝒗i } are denoted right singular vectors of . Moreover,
i j
from (2.139)
u
𝒗i =  H i (2.146)
𝜎i

Definition 2.1
The pseudoinverse of , L × N, of rank R, is given by the matrix
∑
R
 # = V𝚺# UH = 𝜎i−1 𝒗i uH
i (2.147)
i=1
where [ ]
D−1 𝟎 ( )
𝚺# = , D−1 = diag 𝜎1−1 , 𝜎2−1 , … , 𝜎R−1 (2.148)
𝟎 𝟎 ◽
We find an expression of  # for the two cases in which  has full rank,5 that is R = min(L, N).

Case of an overdetermined system (L > N) and R = N. Note that in this case, the system (2.134) has
more equations than unknowns. Using the above relations, it can be shown that
 # = ( H )−1  H (2.149)
In this case,  # d coincides with the solution of system (2.105).

Case of an underdetermined system (L < N) and R = L. Note that in this case, there are fewer
equations than unknowns, hence, there are infinite solutions to the system (2.134). Again, it can be
shown that
 # =  H ( H )−1 (2.150)

Minimum norm solution

Definition 2.2
The solution of a least squares problem is given by the vector
cLS =  # d (2.151)
where # is the pseudoinverse of . ◽

By applying (2.151), the pseudoinverse matrix  # gives the LS solution of minimum norm; in other
words, it provides the vector c that minimizes the square error (2.92),  = ‖e‖2 = ‖y − d‖2 =
‖c − d‖2 , and simultaneously minimizes the norm of the solution, ‖c ‖2 . The constraint on ‖c ‖2 is
‖ ‖ ‖ ‖ ‖ ‖
needed in those cases wherein there is more than one vector that minimizes ‖ ‖2
‖c − d‖ .
5
We will denote the rank of  by rank().
126 Chapter 2. The Wiener filter

We summarize the different cases:

1. If L = N and rank() = N, i.e.  is non-singular,
 # =  −1 (2.152)
2. If L > N and
a. rank() = N, then
 # = ( H )−1  H (2.153)
and cLS is the LS solution of an overdetermined system of equations (2.134).
b. rank() = R (also < N), from (2.151)
∑
R
𝒗H
i
H d
cLS = 𝒗i (2.154)
i=1 𝜎i2
3. If L < N and
a. rank() = L, then
 # =  H ( H )−1 (2.155)
and cLS is the minimum norm solution of an underdetermined system of equations.
b. rank() = R (also < L),
∑R
uH
i
d H
cLS =  ui (2.156)
i=1 𝜎i
2

Only solutions (2.151) in the cases (2.152) and (2.153) coincide with the solution (2.106).
The expansion of c in terms of {ui }, {𝒗i }, and {𝜎i2 } have two advantages with respect to applying
(2.151) by the computation of  # (in the form (2.153), for L > N and rank() = N, or in the form (2.155),
for L < N and rank() = L).
1. The SVD also gives the rank of  through the number of non-zero singular values.
2. The required accuracy in computing  # via SVD is almost halved with respect to the computation
of ( H )−1 or ( H )−1 .
There are two algorithms to determine the SVD of : the Jacobi algorithm and the Householder
transformation [9].
We conclude this section citing two texts [10, 11], which report examples of realizations of the algo-
rithms described in this section.

2.4 The estimation problem

This section extends the formulation of the Wiener problem to a more general multiple-input multiple-
output framework.

Estimation of a random variable

Let d and x be two r.v., somehow related. Two examples are x = f (d) with f (⋅) an unknown function, or
the relation between d and x is explicit, as in x = d + 𝑤, with 𝑤 another random variable. On the basis
of an observation, given x = 𝛽, the estimation problem is to determine what is the corresponding value
of d, given that we know only the joint probability density function (pdf) of the two r.v.s., pdx (𝛼, 𝛽). In
any case, using as estimate of d the function
d̂ = h(x) (2.157)
2.4. The estimation problem 127

the estimation error is given by

e = d − d̂ (2.158)
The estimator is called unbiased if
̂ = E[d]
E[d] (2.159)
Otherwise, the bias is
̂ − E[d]
𝜇bias = E[d] (2.160)

MMSE estimation
Let pd (𝛼) and px (𝛽) be the pdfs of d and x, respectively, and pd∣x (𝛼 ∣ 𝛽) the conditional probability den-
sity function of d given x = 𝛽; moreover, let px (𝛽) ≠ 0, ∀𝛽. We wish to determine the function h that
minimizes the mean-square error, that is
+∞ +∞
J = E[e2 ] = [𝛼 − h(𝛽)]2 pdx (𝛼, 𝛽) d𝛼 d𝛽
∫−∞ ∫−∞
+∞ +∞
(2.161)
= px (𝛽) [𝛼 − h(𝛽)]2 pd∣x (𝛼 ∣ 𝛽) d𝛼 d𝛽
∫−∞ ∫−∞

where the relation pdx (𝛼, 𝛽) = pd∣x (𝛼 ∣ 𝛽)px (𝛽) is used.

Theorem 2.2
The estimator h(𝛽) that minimizes J is given by the expected value of d given x = 𝛽,
h(𝛽) = E[d ∣ x = 𝛽] (2.162)

◽
Proof. The integral (2.161) is minimum when the function
+∞
[𝛼 − h(𝛽)]2 pd∣x (𝛼 ∣ 𝛽) d𝛼 (2.163)
∫−∞
is minimized for every value of 𝛽. Using the variational method (see Ref. [12]), we find that this occurs
if +∞
2 [𝛼 − h(𝛽)] pd∣x (𝛼 ∣ 𝛽) d𝛼 = 0 ∀𝛽 (2.164)
∫−∞
that is for +∞ +∞
pdx (𝛼, 𝛽)
h(𝛽) = 𝛼 pd∣x (𝛼 ∣ 𝛽) d𝛼 = 𝛼 d𝛼 (2.165)
∫−∞ ∫−∞ px (𝛽)
from which (2.162) follows. ◽

An alternative to the MMSE criterion for determining d̂ is given by the maximum a posteriori prob-
ability (MAP) criterion, which yields
d̂ = arg max pd∣x (𝛼 ∣ 𝛽) (2.166)
𝛼

If the distribution of d is uniform, the MAP criterion becomes the maximum-likelihood (ML) criterion,
where
d̂ = arg max px∣d (𝛽 ∣ 𝛼) (2.167)
𝛼

Examples of both MAP and ML criteria are given in Chapters 6 and 14.
128 Chapter 2. The Wiener filter

Example 2.4.1
Let d and x be two jointly Gaussian r.v.s. with mean values 𝚖d and 𝚖x , respectively, and covariance
𝚌 = E[(d − 𝚖d )(x − 𝚖x )]. After several steps, it can be shown that [13]
𝚌
h(𝛽) = 𝚖d + 2 (𝛽 − 𝚖x ) (2.168)
𝜎x
The corresponding mean-square error is equal to
( )2
𝚌
Jmin = 𝜎d2 − (2.169)
𝜎x

Example 2.4.2
Let x = d + 𝑤, where d and 𝑤 are two statistically independent r.v.s. For 𝑤 ∼  (0, 1) and d ∈ {−1, 1}
with P[d = −1] = P[d = 1] = 1∕2, it can be shown that
e𝛽 − e−𝛽
h(𝛽) = tanh 𝛽 = (2.170)
e𝛽 + e−𝛽

Extension to multiple observations

Let x = [x1 , … , xN ]T and 𝜷 = [𝛽1 , … , 𝛽N ]T . In case of multiple observations,
x=𝜷 (2.171)
the estimation of d is obtained by applying the following theorem, whose proof is similar to that given
in the case of a single observation.

Theorem 2.3
The estimator of d, d̂ = h(x), that minimizes J = E[(d − d)
̂ 2 ] is given by
h(𝜷) = E[d ∣ x = 𝜷]
+∞
= 𝛼 pd∣x (𝛼 ∣ 𝜷) d𝛼
∫−∞
(2.172)
pd,x (𝛼, 𝜷)
= 𝛼 d𝛼
∫ px (𝜷)
◽
In the following, to simplify the formulation, we will refer to r.v.s. with zero mean.

Example 2.4.3
Let d, x = [x1 , … , xN ]T , be real-valued jointly Gaussian r.v.s. with zero mean and the following second-
order description:
● Correlation matrix of observations
R = E[x xT ] (2.173)
● Crosscorrelation vector
p = E[dx] (2.174)

For x = 𝜷, it can be shown that

h(𝜷) = pT R−1 𝜷 (2.175)
and
Jmin = 𝜎d2 − pT R−1 p (2.176)
2.4. The estimation problem 129

Linear MMSE estimation of a random variable

For a low-complexity implementation, it is often convenient to consider a linear function h. Letting c =
[c1 , … , cN ]T , in the case of multiple observations, the estimate is a linear combination of observations,
and
d̂ = cT x + b (2.177)
where b is a constant.
In the case of real-valued r.v.s., using the definitions (2.173) and (2.174), it is easy to prove the fol-
lowing theorem:

Theorem 2.4
Given the vector of observations x, the linear minimum mean-square error (LMMSE) estimator of d has
the following expression:
d̂ = pT R−1 x (2.178)
In other words, b = 0 and
copt = R−1 p (2.179)
The corresponding mean-square error is
Jmin = 𝜎d2 − pT R−1 p (2.180)
Note that the r.v.s. are assumed to have zero mean. ◽

Observation 2.2
Comparing (2.178) and (2.180) with (2.175) and (2.176), respectively, we note that, in the case of jointly
Gaussian r.v.s., the linear estimator coincides with the optimum MMSE estimator.

Linear MMSE estimation of a random vector

We extend the results of the previous section to the case of complex-valued r.v.s., and for a desired vector
signal. Let x be an observation, modelled as a vector of N r.v.s.,
xT = [x1 , x2 , … , xN ] (2.181)
Moreover, let d be the desired vector, modelled as a vector of M r.v.s.,
dT = [d1 , d2 , … , dM ] (2.182)
We introduce the following correlation matrices:
rdi x = E[di x∗ ] (2.183)
Rxd = E[x d ] = [rd1 x , rd2 x , … , rdM x ]
∗ T
(2.184)
∗ T
Rdx = E[d x ] = RH
xd (2.185)
∗ T
Rx = E[x x ] (2.186)
∗ T
Rd = E[d d ] (2.187)
The problem is to determine a linear transformation of x, given by
d̂ = CT x + b (2.188)
such that d̂ is a close replica of d in the mean-square error sense.
130 Chapter 2. The Wiener filter

Definition 2.3
The LMMSE estimator of a random vector, consisting of the N × M matrix C, and of the M × 1 vector
b, coincides with the linear function of the observations (2.188) that minimizes the cost function:
[ ] ∑ M [ ]
̂ 2 = ‖ ‖2
J = E ∥ d − d∥ E ‖dm − d̂ m ‖ (2.189)
‖ ‖
m=1

In other words, the optimum coefficients C and b are the solution of the following problem:
min J (2.190)
C,b ◽

We note that in the formulation of Section 2.1, we have

xT = [x(k), … , x(k − N + 1)] (2.191)
T
d = [d(k)] (2.192)
that is M = 1, and the matrix C becomes a column vector.
We determine now the expression of C and b in terms of the correlation matrices introduced above.
First of all, we observe that if d and x have zero mean, then b = 𝟎, since the choice of b = b̃ ≠ 𝟎 implies
an estimator C̃ x + b̃ with a suboptimal value of the cost function. In fact,
T

[ ]
J = E ‖d − C̃ x − b‖ ̃ 2
T

[ ] { [ ]}
= E ‖d − C̃ x‖2 − 2Re E (d − C̃ x)H b̃ + ‖b‖ ̃ 2
T T
(2.193)
[ ]
= E ‖d − C̃ x‖2 + ‖b‖ ̃ 2
T

being E[d] = 𝟎 and E[x] = 𝟎. Now, (2.193) implies that the choice b̃ = 𝟎 yields the minimum value of
J. Without loss of generality, we will assume that both x and d are zero mean random vectors.

Scalar case For M = 1, d = d1 , we have

d̂ = d̂ 1 = cT1 x = xT c1 (2.194)
with c1 column vector with N coefficients. In this case, the problem (2.190) leads again to the Wiener
filter; the solution is given by
Rx c1 = rd1 x (2.195)
where rd1 x is defined by (2.183).

Vector case For M > 1, d, and d̂ are M-dimensional vectors. Nevertheless, since function (2.189) oper-
ates on single components, the vector problem (2.190) leads to M scalar problems, each with input x
and output d̂ 1 , d̂ 2 , … , d̂ M , respectively. Therefore, the columns of matrix C, {cm }, m = 1, … , M, satisfy
equations of the type (2.195), namely
Rx cm = rdm x , m = 1, … , M (2.196)
hence, based on the definition (2.184), we have
C = R−1
x Rxd (2.197)
Thus, the optimum estimator in the LMMSE sense is given by
( )T
d̂ = R−1
x Rxd x (2.198)
2.4. The estimation problem 131

Value of the cost function On the basis of the estimation error

e = d − d̂ (2.199)
with correlation matrix
Re = E[e∗ eT ] = Rd − Rdx C − CH Rxd + CH Rx C (2.200)
substituting (2.197) in (2.200), yields
Re,min = Rd − Rdx R−1
x Rxd (2.201)
The cost function (2.189) is given by the trace of Re , and
Jmin = tr Re,min (2.202)

2.4.1 The Cramér–Rao lower bound

Until now we have considered the estimation of a random variable or vector, under the assumption of
knowing the joint distribution of d and x. However, in some cases, we only know the distribution of x,
and its relation to d, without specific knowledge of the distribution of d. Hence, d is modelled as an
unknown parameter vector. For example, we may have multiple noisy observations of a scalar d by
xn = d + 𝑤n , n = 1, … , N (2.203)
where we know the distribution of 𝑤n , e.g. 𝑤n ∼  (0, 1). An example of estimator of d is (see (1.377))

1 ∑
N
d̂ = h(x) = x (2.204)
N n=1 n

In this section, we focus on the estimation of a scalar parameter d by x, as d̂ = h(x). In particular,

given px|d (𝜷|𝛼), we focus on the best performance of any estimator, when the metric is the variance of
the estimator, i.e. the design objective is
̂
min var(d) (2.205)
h

Note that (2.205) coincides with the MMSE only if

̂ =d
E[d] (2.206)
which, from (2.159), is equivalent to consider an unbiased estimator of parameter d. Note that in order
to find the estimator, we must design the function h(x), which is in general a challenging task.
The Cramér–Rao lower bound (CRLB) theorem provides a limit on the minimum variance that any
estimator can achieve, given the number of observations N and the relation between observations and the
parameter. Moreover, when the unbiased estimator exists, under mild conditions, the theorem provides
the expression of the minimum-variance estimator.

Theorem 2.5 (Cramér–Rao lower bound)

Consider a conditional pdf px|d (𝜷|𝛼) satisfying
[ ]
𝜕 ln px|d (x|𝛼)
E =0 ∀𝛼 (2.207)
𝜕𝛼
where the expectation is taken with respect to x, given d = 𝛼. Let
[ 2 ]
𝜕 ln px|d (x|𝛼)
I(𝛼) = −E (2.208)
𝜕𝛼 2
132 Chapter 2. The Wiener filter

Then, any unbiased estimator satisfies

̂ ≥ 1
var(d) ∀𝛼 (2.209)
I(𝛼)
Moreover, if and only if there exists a function hopt (x) such that
𝜕 ln px|d (𝜷|𝛼)
= I(𝛼)[hopt (𝜷) − 𝛼] ∀𝛼 (2.210)
𝜕𝛼
the estimator d̂ = hopt (x) is unbiased and reaches the minimum variance (2.209). ◽

Example 2.4.4
( )
Consider xn = d + 𝑤n , n = 1, … , N, with 𝑤n ∼  0, 𝜎𝑤2 , statistically independent, for a known 𝜎𝑤2 .
First, we observe that
1 − 12 (𝜷−𝛼𝟏)T (𝜷−𝛼𝟏)
px|d (𝜷|𝛼) = (√ )N e 𝑤
2𝜎
(2.211)
2
2𝜋𝜎𝑤

where 𝟏 is the N-size column vector with all entries equal to 1. We have
𝜕 ln px|d (𝜷|𝛼) 1
= (𝜷 − d𝟏)T 𝟏 (2.212)
𝜕𝛼 𝜎𝑤2
and clearly (2.207) is satisfied. Then we can apply Theorem 2.5 and
N 1
I(𝛼) = , d̂ = h(x) = xT 𝟏 (2.213)
𝜎𝑤2 N
Note that h(𝜷) corresponds to (2.204), which is therefore the minimum-variance estimator, and the error
variance is 1∕I(𝛼) = 𝜎𝑤2 ∕N for all values of 𝛼.

Extension to vector parameter

We now focus on the estimate of M real parameters, collected into the vector parameter d (2.182) from
a vector of observations x, i.e. we look for the estimator
d̂ = h(x) (2.214)
A simple example is the estimation of mean and variance (d = [𝚖x , 𝜎x2 ]T ) of a Gaussian random variable
from N observations collected into vector x, as in (2.181). Theorem 2.5 can be extended to the case of
the estimate of a vector parameter, as follows:

Theorem 2.6 (Cramér–Rao lower bound for vector parameter)

Consider a conditional pdf px|d (𝜷|𝜶) satisfying
[ ]
𝜕 ln px|d (x|𝜶)
E = 0 ∀𝛼m , m = 1, … , M (2.215)
𝜕𝛼m
where the expectation is taken with respect to x, given d = 𝜶. Let us define the M × M Fisher information
matrix, with entry m1 , m2 ∈ {1, … , M}, given by
[ 2 ]
𝜕 ln px|d (x|𝜶)
[I(𝜶)]m1 ,m2 = −E (2.216)
d𝛼m1 d𝛼m2

Then for any unbiased estimator d̂ with covariance matrix Cd̂ = E[d̂ d̂ ], we have that matrix
T

Cd̂ − [I(𝜶)]−1 is positive semidefinite.

2.4. The estimation problem 133

Moreover, if and only if (here [⋅]m denotes entry m of the vector argument)
𝜕 ln px|d (𝜷|𝜶)
= [I(𝜶)(h(𝜷) − 𝜶)]m ∀𝛼m , m = 1, … , M (2.217)
𝜕𝛼m
for some function h, the estimator d̂ = h(x) is unbiased and its covariance matrix is [I(𝜶)]−1 . ◽

Before proceeding, we observe that in some cases, we can directly compute the Fisher information
matrix. Then, if (2.217) and the semi-definitive condition on Cd̂ are verified, and (2.215) holds, then
h(x) in (2.217) is the desired estimator. Otherwise, the theorem does not apply.

Example 2.4.5
( )
Consider xn , n = 1, … , N, uncorrelated with xn ∼  𝚖x , 𝜎x2 , where both 𝚖x and 𝜎x2 are unknown
parameters; hence, d = [𝚖x , 𝜎x2 ]T . We have
1
1 − (𝜷−𝚖x 𝟏)T (𝜷−𝚖x 𝟏)
2𝜎x2
px|d (𝜷|𝜶) = (√ )N e (2.218)
2𝜋𝜎x2

First, we observe that

𝜕 ln px|d (𝜷|𝜶) 1 ∑
N
= [𝛽n − 𝚖x ] (2.219)
𝜕𝛼1 𝜎x2 n=1
𝜕 ln px|d (𝜷|𝜶) 1 ∑
N
N
= 4 − 6 [𝛽n − 𝚖x ]2 (2.220)
𝜕𝛼2 2𝜎x 𝜎x n=1
and it can easily concluded that (2.215) holds. The Fisher information matrix in this case is
[ 2 ] [ 2 ]
⎡ 𝜕 ln px|d (x|𝜶) 𝜕 ln px|d (x|𝜶) ⎤
⎡N ⎤
⎢E d𝛼1 d𝛼1
E
d𝛼1 d𝛼2 ⎥ ⎢ 2 0 ⎥
⎢ ⎥ ⎢ 𝜎x ⎥
I(𝜶) = − ⎢ [ ] [ 2 ]⎥ = (2.221)
⎢ 𝜕 2 ln px|d (x|𝜶) 𝜕 ln px|d (x|𝜶) ⎥ ⎢0 N ⎥
⎢E E ⎥ ⎢⎣ ⎥
2𝜎x4 ⎦
⎣ d𝛼2 d𝛼1 d𝛼2 d𝛼2 ⎦
with inverse
⎡ 𝜎x2 ⎤
−1 ⎢N 0 ⎥
[I(𝜶)] = ⎢
2𝜎x4 ⎥
(2.222)
⎢0 ⎥
⎣ N ⎦
which provides on the diagonal the minimum variances of any unbiased estimator.
Inserting (2.221), (2.219), and (2.220) into (2.217), we conclude that (2.217) holds for m = 1, while
for m = 2 for any function h2 (x), we have

1 ∑
N
N 2𝜎 4
− [xn − 𝚖x ]2 ≠ (h (x) − 𝜎x2 ) (2.223)
2𝜎x4
𝜎x n=1
6 N 2
Therefore, we can conclude that in this case, no unbiased estimator can have covariance matrix [I(𝜶)]−1 ,
2𝜎 4
and any estimator of 𝜎x2 will have a variance strictly larger than Nx .

Example 2.4.6
Consider a known complex sequence sn , with |sn | = 1, n = 1, … , N, and two observation vectors
x1 = [x1,1 , … , x1,N ]T and x2 = [x2,1 , … , x2,N ]T with
x1,n = Asn + 𝑤1,n , x2,n = Asn ej𝜖 + 𝑤2,n (2.224)
134 Chapter 2. The Wiener filter

( )
where A it a complex number, 𝜖 is a real number, and 𝑤1,n and 𝑤2,n ∼  0, 𝜎 2 , with 𝑤1,n and 𝑤2,n
statistically independent. Assume that 𝜎 2 is known, while both parameters A and 𝜖 are to be estimated,
thus dT = [A, 𝜖]. Let sT = [s1 , … , sN ], m1 = As, and m2 = Aej𝜖 s, we have
1 1
ln px1 ,x2 |d (x1 , x2 |𝜶) = −N ln(𝜋𝜎 2 ) − (x − m1 )H (x1 − m1 ) − 2 (x2 − m2 )H (x2 − m2 ) (2.225)
2𝜎 2 1 2𝜎
First observe that
𝜕 ln px1 ,x2 |d (x1 , x2 |𝜶) 1 ej𝜖
= (x1 − m1 )H s + 2 (x2 − m2 )H s (2.226)
𝜕A 𝜎 2 𝜎
𝜕 ln px1 ,x2 |d (x1 , x2 |𝜶) jAej𝜖
= 2 (x2 − m2 )H s (2.227)
𝜕𝜖 𝜎
and it can be easily verified that (2.215) is satisfied. Moreover, using also |sn | = 1, we have
⎡ 2N AN ⎤ ⎡ 2 A3 N ⎤
⎢ 𝜎2 𝜎2 ⎥ −1 𝜎2 ⎢ A − 𝜎2 ⎥
I(𝜶) = ⎢ 2 ⎥
and [I(𝜶)] = (2.228)
⎢ AN A N ⎥ NA2 ⎢ A3 N ⎥
⎣ 𝜎2 𝜎2 ⎦ ⎣− 𝜎 2 2 ⎦

2.5 Examples of application

We give now some examples of application of the Wiener filter theory [14–17].

2.5.1 Identification of a linear discrete-time system

We want to determine the relation between the input x and the output z of the system illustrated in
Figure 2.12. We note that the observation d is affected by additive noise 𝑤, having zero mean and
variance 𝜎𝑤2 , assumed statistically independent of x.
Assuming the system between z and x can be modelled as a FIR filter, the configuration experiment
illustrated in Figure 2.12 can be adopted to estimate the filter impulse response.

w(k) noise
x(k) unknown z(k) d(k)
system
h +
input
(PN sequence) +
e(k)
+
–
y(k)
c

Figure 2.12 Scheme to estimate the impulse response of an unknown system.

Using an input x, known to both systems, we determine the output of the transversal filter c with N
coefficients
∑
N−1
y(k) = ci x(k − i) = c T x(k) (2.229)
i=0
2.5. Examples of application 135

and the estimation error

e(k) = d(k) − y(k) (2.230)
We analyse the specific case of an unknown linear FIR system whose impulse response has Nh coeffi-
cients. Assuming N ≥ Nh , we introduce the vector h with N components,
hT = [h0 , h1 , … , hNh −1 , 0, … , 0] (2.231)
In this case, ( )
d(k) = h0 x(k) + h1 x(k − 1) + · · · + hNh −1 x k − (Nh − 1) + 𝑤(k)
= hT x(k) + 𝑤(k) (2.232)

For N ≥ Nh , and assuming the input x is white noise with statistical power 𝚛x (0), we get
[ ]
R = E x∗ (k)xT (k) = 𝚛x (0)I (2.233)
and
[ ]
p = E d(k)x∗ (k) = 𝚛x (0)h (2.234)
Then the W–H equation (2.29) yields the desired solution as
copt = R−1 p = h (2.235)
and
Jmin = 𝜎𝑤2 (2.236)
On the other hand, if N < Nh , then copt in (2.235) coincides with the first N coefficients of h, and
Jmin = 𝜎𝑤2 + 𝚛x (0) ‖𝚫h(∞)‖2 , (2.237)
where 𝚫h(∞) represents the residual error vector,
𝚫h(∞) = [0, … , 0, −hN , −hN+1 , … , −hNh −1 ]T (2.238)

2.5.2 Identification of a continuous-time system

We now consider the identification of a continuous-time system.
With reference to Figure 2.13 [18], a pseudo-noise (PN) sequence p of period L, repeated several
times, is used to modulate in amplitude the pulse
( )
t − Tc ∕2
g(t) = wTc (t) = rect (2.239)
Tc

p(k) x(t) z(t) u(t) v(t)

g h gRc
Tc
x*(t – τ)
delay
τ

Figure 2.13 Basic scheme to measure the impulse response of an unknown system.
136 Chapter 2. The Wiener filter

The modulated output signal is therefore given by

∑
+∞
x(t) = p(i)mod L g(t − iTc ) (2.240)
i=0

The autocorrelation of x, periodic function of period LTc , is

LTc
+
1 2
𝚛x (t) = x(𝜂)x∗ (𝜂 − t) d𝜂 (2.241)
∫
LTc − LTc
2

As g has finite support of length Tc , we have

1 ∑
L−1
𝚛x (t) = 𝚛 (𝓁)𝚛g (t − 𝓁Tc ) 0 ≤ t ≤ LTc (2.242)
Tc 𝓁=0 p

where, in the case of g given by (2.239), we have

( ) ( )
Tc
|t| t
𝚛g (t) = g(𝜂)g(𝜂 − t) d𝜂 = Tc 1 − rect (2.243)
∫0 Tc 2Tc

Substituting (2.243) in (2.242) and assuming that p is a maximal-length sequence (MLS) (see
Appendix 1.C), with 𝚛p (0) = 1 and 𝚛p (𝓁) = −1∕L, for 𝓁 = 1, … , L − 1, we obtain
( )( |t|
) ( )
LTc
1 1 t
𝚛x (t) = − + 1+ 1− rect , |t| ≤ (2.244)
L L Tc 2Tc 2

as shown in Figure 2.14 for L = 8. If the output z of the unknown system to be identified is multiplied
by a delayed version of the input, x∗ (t − 𝜏), and the result is filtered by an ideal integrator between 0 and
LTc with impulse response
( )
1 1 t − LTc ∕2
gRc (t) = wLTc (t) = rect (2.245)
LTc LTc LTc

x(t)

–4Tc –3Tc –2Tc –Tc Tc 2Tc 3Tc 4Tc

0 t
–1/L

Figure 2.14 Autocorrelation function of x.

2.5. Examples of application 137

we obtain t
1
𝑣(t) = u(𝜂) d𝜂
LTc ∫t−LTc
t +∞
1
= [h(𝜉)x(𝜂 − 𝜉) d𝜉] x∗ (𝜂 − 𝜏) d𝜂 (2.246)
LTc ∫t−LTc ∫0
+∞
= h(𝜉)𝚛x (𝜏 − 𝜉) d𝜉 = h ∗ 𝚛x (𝜏) = 𝑣𝜏
∫0
Therefore, the output assumes a constant value 𝑣𝜏 equal to the convolution between the unknown system
h and the autocorrelation of x evaluated in 𝜏. Assuming 1∕Tc is larger than the maximum frequency of the
spectral components of h, and L is sufficiently large, the output 𝑣𝜏 is approximately proportional to h(𝜏).
The scheme represented in Figure 2.15 is alternative to that of Figure 2.13, of simpler implementation
because it does not require synchronization of the two PN sequences at transmitter and receiver. In
this latter scheme, the output z of the unknown system is multiplied by a PN sequence having the same
characteristics of the transmitted sequence, but a different clock frequency f0′ = 1∕Tc′ , related to the clock
frequency f0 = 1∕Tc of the transmitter by the relation
( )
1
f0′ = f0 1 − (2.247)
K

p(k) x(t) z(t) v(t)

g h gRc
Tc

[x′(t)]*
p(k)
g
T′c

Figure 2.15 Sliding window method to measure the impulse response of an unknown system.

where K is a parameter of the system. We consider the function

LTc
1
𝚛x′ x (𝜏) = [x′ (𝜂)]∗ x(𝜂 − 𝜏) d𝜂 (2.248)
LTc ∫0
where 𝜏 is the delay at time t = 0 between the two sequences. As time elapses, the delay between the
two sequence diminishes by (t∕Tc′ )(Tc′ − Tc ) = t∕K, so that
t ( )
1 t − LTc
[x′ (𝜂)]∗ x(𝜂 − 𝜏) d𝜂 ≃ 𝚛x′ x 𝜏 − for t ≥ LTc (2.249)
LTc ∫t−LTc K
If K is sufficiently large, we can assume that
𝚛x′ x (𝜏) ≃ 𝚛x (𝜏) (2.250)
At the output of the filter gRc , given by (2.245), therefore, we have
t +∞
1
𝑣(t) = [h(𝜉)x(𝜂 − 𝜉) d𝜉] [x′ (𝜂)]∗ d𝜂
LTc ∫t−LTc ∫0
+∞ ( )
t − LTc
= h(𝜉)𝚛x′ x 𝜉 − d𝜉
∫0 ( K ) (2.251)
+∞
t − LTc
≃ h(𝜉)𝚛x − 𝜉 d𝜉
∫0 K
138 Chapter 2. The Wiener filter

or with the substitution t′ = (t − LTc )∕K,

+∞
y(Kt′ + LTc ) ≃ h(𝜉)𝚛x (t′ − 𝜉) d𝜉 (2.252)
∫0
where the integral in (2.252) coincides with the integral in (2.246). If K is sufficiently large (an increase
of K clearly requires a greater precision and hence a greater cost of the frequency synthesizer to generate
f0′ ), it can be shown that the approximations in (2.249) and (2.250) are valid. Therefore, the systems in
Figures 2.15 and 2.13 are equivalent.

2.5.3 Cancellation of an interfering signal

With reference to Figure 2.16, we consider two sensors:

1. Primary input, consisting of the desired signal s corrupted by additive noise 𝑤0 ,

d(k) = s(k) + 𝑤0 (k) with s ⟂ 𝑤0 (2.253)

2. Reference input, consisting of the noise signal 𝑤1 , with s ⟂ 𝑤1 . We assume that 𝑤0 and 𝑤1 are in
general correlated.

𝑤1 is filtered by filter c with N coefficients, so that the filter output y, given by

∑
N−1
y(k) = ci 𝑤1 (k − i) (2.254)
i=0

is the most accurate replica of 𝑤0 . Defining the error

e(k) = d(k) − y(k) = s(k) + 𝑤0 (k) − y(k) (2.255)

the cost function, assuming real-valued signals and recalling that s is orthogonal to the noise signals, is
given by
[ ]
J = E[e2 (k)] = E[s2 (k)] + E (𝑤0 (k) − y(k))2 (2.256)

We have two cases.

Primary
input +
s(k) d(k) e(k) output
Signal source + +
– Estimate
w0 (k) of desired
estimate of
interference signal
∧
s (k)
w1(k) x(k) y(k)
interference c
source
reference

interference canceller

Figure 2.16 General configuration of an interference canceller.

2.5. Examples of application 139

d(k) desired
signal

+ e(k)
+
– error signal

x(k) y(k)
c
input output

Figure 2.17 Block diagram of an interference canceller.

1. 𝑤1 and 𝑤0 are correlated:

[ ]
min J = 𝚛s (0) + min E (𝑤0 (k) − y(k))2 = 𝚛s (0) (2.257)
c c

for y(k) = 𝑤0 (k). In this case, e(k) = s(k).

2. 𝑤1 and 𝑤0 are uncorrelated:
[ ] [ ]
min J = E (s(k) + 𝑤0 (k))2 + min E y2 (k)
c
[ ] c
= E (s(k) + 𝑤0 (k))2 (2.258)

for y(k) = 0. In this case, e(k) = d(k), and the noise 𝑤0 is not cancelled.
With reference to Figure 2.17, for a general input x to the filter c, the W–H solution in the z-transform
domain is given by (see (2.45))
P (z)
Copt (z) = dx (2.259)
Px (z)
Adopting for d and x, the model of Figure 2.18, in which 𝑤′0 and 𝑤′1 are additive noise signals uncor-
related with 𝑤 and s, and using Table 1.3, (2.259) becomes
P𝑤 (z)H ∗ (1∕z∗ )
Copt (z) = (2.260)
P𝑤′ (z) + P𝑤 (z)H(z)H ∗ (1∕z∗ )
1

If 𝑤′1 (k) = 0, ∀k, (2.260) becomes

1
Copt (z) = (2.261)
H(z)

2.5.4 Cancellation of a sinusoidal interferer with known frequency

Let
d(k) = s(k) + A cos(2𝜋f0 kTc + 𝜑0 ) (2.262)
where s is the desired signal, and the sinusoidal term is the interferer. We take as reference signals
x1 (k) = B cos(2𝜋f0 kTc + 𝜑) (2.263)
and
x2 (k) = B sin(2𝜋f0 kTc + 𝜑) (2.264)
140 Chapter 2. The Wiener filter

w′0 (k)

+ primary
s(k) + input d(k) + e(k) output
+ + +
signal
–

w(k) reference y(k)

h + c
interference +
+

interference canceller
w′1(k)

Figure 2.18 Specific configuration of an interference canceller.

The two coefficients c1 and c2 , which respectively multiply x1 and x2 , change the amplitude and phase
of the reference signal to cancel the interfering tone.
It is easy to see that x2 is obtained from x1 via a Hilbert filter (see Figure 1.19). We note that in this
case, x2 can be obtained as a delayed version of x1 .

2.5.5 Echo cancellation in digital subscriber loops

With reference to the simplified scheme of Figure 2.19, the signal of user A is transmitted over a trans-
mission line consisting of a pair of wires (local loop) [19] to the central office A, where the signals
in the two directions of transmission, i.e. the signal transmitted by user A and the signal received from
user B, are separated by a device called hybrid. A similar situation takes place at the central office B,
with the roles of the signals A and B reversed. Because of impedance mismatch, the hybrids give ori-
gin to echo signals that are added to the desired signals, a problem that will be addressed in details
in Chapter 17. A method to remove echo signals is illustrated in Figure 2.20, where y is a replica of the
echo. At convergence, e will consist of the signal B only.
We note that this problem is a case of system identification, however, in echo cancellation, we are
interested in the filter output y rather than the impulse response.

Figure 2.19 Transmission between two users in the public network.

2.5. Examples of application 141

signal A

signal B
adaptive filter hybrid
B

y(k) signal B

echo A r(k)
–
e(k) + d(k) = r(k) + u(k) u(k)
+

Figure 2.20 Configuration to remove the echo of signal A caused by the hybrid B.

2.5.6 Cancellation of a periodic interferer

For the cancellation of a periodic interfering signal, here a sinusoidal signal, we can use the scheme of
Figure 2.21, where

● we note the absence of an external reference signal; the reference signal is generated by delaying
the primary input;
● a delay Δ = DTc , where D is an integer, is needed to decorrelate the desired component of the
primary signal from that of the reference signal, otherwise, part of the desired signal would also be
cancelled.

On the other hand, to cancel a wideband interferer from a periodic signal it is sufficient to take the
output of the filter.
Note that in both schemes, the filter acts as predictor of the periodic signal.
Exploiting the general concept described above, an alternative approach to that of Section 2.5.4 is
illustrated in Figure 2.21, where the knowledge of the frequency of the interfering signal is not required.
In general, the scheme of Figure 2.21 requires many more than two coefficients, therefore, it has a higher
implementation complexity than that of Section 2.5.4. If the wideband signal can be modelled as white
noise, then D = 1.

d(k) = s(k)+A cos(2πf0kTc+φ0) + e(k)

+
–
y(k)

Δ c

delay
element

Figure 2.21 Scheme to remove a sinusoidal interferer from a wideband signal.

142 Chapter 2. The Wiener filter

Bibliography

[1] Haykin, S. (1996). Adaptive Filter ThEory, 3e. Englewood Cliffs, NJ: Prentice-Hall.
[2] Honig, M.L. and Messerschmitt, D.G. (1984). Adaptive Filters: Structures, Algorithms and Applica-
tions. Boston, MA: Kluwer Academic Publishers.
[3] Atal, B.S. and Remde, J.R. (1982). A new model of LPC excitation for producing natural-sounding
speech at low bit rates. Proceedings of ICASSP, pp. 614–617.
[4] Atal, B.S., Cuperman, V., and Gersho, A. (eds.) (1991). Advances in Speech Coding. Boston, MA:
Kluwer Academic Publishers.
[5] Jayant, N.S. and Noll, P. (1984). Digital Coding of Waveforms. Englewood Cliffs, NJ: Prentice-Hall.
[6] Golub, G.H. and van Loan, C.F. (1989). Matrix Computations, 2e. Baltimore, MD and London: The
Johns Hopkins University Press.
[7] Al-Dhahir, N. and Cioffi, J.M. (1995). Fast computation of channel-estimate based equalizers in
packet data transmission. IEEE Transactions on Signal Processing 43: 2462–2473.
[8] Kumar, R. (1985). A fast algorithm for solving a Toeplitz system of equations. IEEE Transactions on
Acoustics, Speech, and Signal Processing 33: 254–267.
[9] Press, S.A.T.W.H., Flannery, B.P., and Vetterling, W.T. (1988). Numerical Recipes, 3e. New York,
NY: Cambridge University Press.
[10] Marple, L.S. Jr. (1987). Digital Spectral Analysis with Applications. Englewood Cliffs, NJ:
Prentice-Hall.
[11] Kay, S.M. (1988). Modern Spectral Estimation-Theory and Applications. Englewood Cliffs, NJ:
Prentice-Hall.
[12] Franks, L.E. (1981). Signal Theory, revised ed. Stroudsburg, PA: Dowden and Culver.
[13] Kay, S.M. (1993). Fundamentals of Statistical Signal Processing: Estimation Theory. Henceforth,
NJ: Prentice-Hall.
[14] Treichler, J.R., Johnson, C.R. Jr., and Larimore, M.G. (1987). Theory and Design of Adaptive
Filters. New York, NY: Wiley.
[15] Cowan, C.F.N. and Grant, P.M. (1985). Adaptive Filters. Englewood Cliffs, NJ: Prentice-Hall.
[16] Widrow, B. and Stearns, S.D. (1985). Adaptive Signal Processing. Englewood Cliffs, NJ:
Prentice-Hall.
[17] Macchi, O. (1995). Adaptive Processing: The LMS Approach with Applications in Transmission.
New York, NY: Wiley.
[18] Benvenuto, N. (1984). Distortion analysis on measuring the impulse response of a system using a
cross correlation method. AT&T Bell Laboratories Technical Journal 63: 2171–2192.
[19] Messerschmitt, D.G. (1984). Echo cancellation in speech and data transmission. IEEE Journal on
Selected Areas in Communications 2: 283–297.
[20] Delsarte, P. and Genin, Y.V. (1986). The split Levinson algorithm. IEEE Transactions on Acoustics,
Speech, and Signal Processing 34: 470–478.

Appendix 2.A The Levinson–Durbin algorithm

The Levinson–Durbin algorithm (LDA) yields the solution of matrix equations like (2.80), in which
RN+1 is positive definite, Hermitian, and Toeplitz, with a computational complexity proportional to
N 2 , instead of N 3 as it happens with algorithms that make use of the inverse matrix. In the case of real
signals, RN+1 is symmetric and the computational complexity of the Delsarte–Genin algorithm (DGA)
is halved with respect to that of LDA.
2.A. The Levinson–Durbin algorithm 143

Here, we start by a step-by-step description of the LDA:

1. Initialization: We set
J0 = rx (0) (2.265)
Δ0 = rx (1) (2.266)
2. n-th iteration, n = 1, 2, … , N: We calculate
Δn−1
n = − (2.267)
Jn−1
[ ′ ] [ ]
an−1 0
′
an = + n ′ B∗ (2.268)
0 an−1
The (2.268) corresponds to the scalar equations:
𝚊′k,n = 𝚊′k,n−1 + n 𝚊′∗
n−k, n−1 k = 0, 1, … , n (2.269)
with 𝚊′0,n−1 = 1 and 𝚊′n,n−1 = 0. Moreover,

Δn = (rBn+1 )T a′n (2.270)

( )
Jn = Jn−1 1 − |n |2 (2.271)
We now interpret the physical meaning of the parameters in the algorithm. Jn represents the statistical
power of the forward prediction error at the n-th iteration:
[ ]
Jn = E |fn (k)|2 (2.272)
It results
0 ≤ Jn ≤ Jn−1 n≥1 (2.273)
with
J0 = rx (0) (2.274)
and
∏
N
( )
J N = J0 1 − |n |2 (2.275)
n=1

The following relation holds for Δn :

[ ]
Δn−1 = E fn−1 (k)b∗n−1 (k − 1) (2.276)
In other words, Δn can be interpreted as the crosscorrelation between the forward linear prediction error
and the backward linear prediction error delayed by one sample. n satisfies the following property:
n = 𝚊′n,n (2.277)
Last, by substitution, from (2.267), along with (2.272) and (2.276), we get
E[fn−1 (k)b∗n−1 (k − 1)]
n = − [ ] (2.278)
E |fn−1 (k)|2
and noting that E[|fn−1 (k)|2 ] = E[|bn−1 (k)|2 ] = E[|bn−1 (k − 1)|2 ], from (2.278), we have
|n | ≤ 1 (2.279)
The coefficients {n } are named reflection coefficients or partial correlation coefficients (PARCOR).
144 Chapter 2. The Wiener filter

Lattice filters
We have just described the LDA. Its analysis permits to implement the prediction error filter via a mod-
ular structure. Defining
xn+1 (k) = [x(k), … , x(k − n)]T (2.280)
we can write [ ] [ ]
xn (k) x(k)
xn+1 (k) = = (2.281)
x(k − n) xn (k − 1)
We recall the relation for forward and backward linear prediction error filters of order n:
{
fn (k) = 𝚊′0,n x(k) + · · · + 𝚊′n,n x(k − n) = a′T
n xn+1 (k)
(2.282)
bn (k) = 𝚊′∗n,n x(k) + · · · + 𝚊′∗
0,n
x(k − n) = a′BH
n xn+1 (k)
From (2.268), we obtain
[ ]T [ ]T
a′n−1 0
fn (k) = xn+1 (k) + n ′B∗ xn+1 (k)
0 an−1
(2.283)
= fn−1 (k) + n bn−1 (k − 1)

By a similar procedure, we also find

bn (k) = bn−1 (k − 1) + n∗ fn−1 (k) (2.284)
Finally, taking into account the initial conditions,
f0 (k) = b0 (k) = x(k) and 𝚊′0,0 = 1 (2.285)
the block diagram of Figure 2.22 is obtained, in which the output is given by fN .
We list the following fundamental properties:
1. The optimum coefficients n , n = 1, … , N, are independent of the order of the filter; therefore, one
can change N without having to re-calculate all the coefficients. This property is useful if the filter
length is unknown and must be estimated.
2. If conditions |n | ≤ 1, n = 1, … , N, are verified, the filter is minimum phase.
3. The lattice filters are quite insensitive to coefficient quantization.

Observation 2.3
From the above property 2 and (2.279), we have that all predictor error filters are minimum phase.

f0 (k) f1(k) fm – 1(k) fm(k) fN – 1(k) fN(k)

C1 Cm CN

x(k)

C*1 C*m C*N

Tc Tc Tc
b0(k) b1(k) bm – 1(k) bm(k) bN –1(k) bN(k)

Figure 2.22 Lattice filter.

2.A. The Levinson–Durbin algorithm 145

The Delsarte–Genin algorithm

In the case of real signals, the DGA, also known as the split Levinson algorithm [20], further reduces the
number of operations with respect to the LDA, at least for N ≥ 10.6 Here is the step-by-step description.
1. Initialization: We set
𝒗0 = 1 𝛽0 = rx (0) 𝛾0 = rx (1) (2.286)
𝒗1 = [1, 1] 𝛽1 = rx (0) + rx (1) 𝛾1 = rx (1) + rx (2)
T
(2.287)
2. n-th iteration, n = 2, … , N: We compute
(𝛽n−1 − 𝛾n−1 )
𝛼n = (2.288)
(𝛽n−2 − 𝛾n−2 )
𝛽n = 2𝛽n−1 − 𝛼n 𝛽n−2 (2.289)
[ ] [ ] ⎡ 0 ⎤
𝒗n−1 0
𝒗n = + − 𝛼n ⎢𝒗n−2 ⎥ (2.290)
0 𝒗n−1 ⎢ ⎥
⎣ 0 ⎦
( ) ( )
𝛾n = rTn+1 𝒗n = rx (1) + rx (n + 1) + [𝒗n ]2 rx (2) + rx (n) + · · · (2.291)
𝛽
𝜆n = n (2.292)
𝛽n−1
[ ]
0
a′n = 𝒗n − 𝜆n (2.293)
𝒗n−1
Jn = 𝛽n − 𝜆n 𝛾n−1 (2.294)
n = 1 − 𝜆n (2.295)

We note that (2.291) exploits the symmetry of the vector 𝒗n ; in particular it is [𝒗n ]1 = [𝒗n ]n+1 = 1.

6
Faster algorithms, with a complexity proportional to N(log N)2 , have been proposed by Kumar [8].
Chapter 3

Adaptive transversal filters

We reconsider the Wiener filter introduced in Section 2.1. Given two random processes x and d, we want
to determine the coefficients of a finite impulse response (FIR) filter having input x, so that its output y is
a replica as accurate as possible of the process d. Adopting, for example, the mean-square error (MSE)
criterion, the knowledge of the autocorrelation matrix R of the filter input vector and the crosscorrelation
p between the desired output and the input vector are required. Next, the filter coefficients are determined
by solving the system of equations (2.27). When the autocorrelation and crosscorrelation are not known
a priori, but only (long) realizations of x and d are available, two approaches are possible. The direct
approach provides that both autocorrelation and crosscorrelation are estimated,1 and then the system of
equations is solved. Clearly, estimation errors are propagated to the designed filter coefficients. An alter-
native approach discussed in this chapter provides that the filter coefficients are directly obtained from x
and d by a learning and adaptive procedure. In this case, neither autocorrelation nor crosscorrelation are
explicitly estimated, and no linear system of equations is solved. It turns out that in most cases adaptive
methods are more robust to estimate errors than the direct approach. Note that the solutions presented
in this chapter can be seen as machine learning approaches, where we have a machine (i.e. a parametric
function, in this case an FIR filter) that learns by given input and desired output (x and d).
We will consider transversal FIR filters2 with N coefficients. In general, the coefficients may vary with
time. The filter structure at instant k is illustrated in Figure 3.1. We define:

1. Coefficient vector at instant k:

cT (k) = [c0 (k), c1 (k), … , cN−1 (k)] (3.1)

2. Input vector at instant k:

xT (k) = [x(k), x(k − 1), … , x(k − N + 1)] (3.2)

The output signal is given by

∑
N−1
y(k) = ci (k)x(k − i) = xT (k)c(k) (3.3)
i=0

1
Two estimation methods are presented in Section 1.9.2.
2
For the analysis of infinite impulse response (IIR) adaptive filters, we refer the reader to [1, 2].

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
148 Chapter 3. Adaptive transversal filters

x(k) x(k – 1) x(k – 2) x(k – N + 1)

Tc Tc Tc

c0(k) c1(k) c2(k) cN – 1(k)

– y(k)
e(k)
+ d(k)

Figure 3.1 Structure of an adaptive transversal filter at instant k.

Comparing y(k) with the desired response d(k), we obtain the estimation error3

e(k) = d(k) − y(k) (3.4)

Depending on the cost function associated with {e(k)}, in Chapter 2 two classes of algorithms have been
developed:

1. mean-square error (MSE),

2. least squares (LS).

In the following sections, we will present iterative algorithms for each of the two classes.

3.1 The MSE design criterion

The cost function, or functional, to minimize is

J(k) = E[|e(k)|2 ] (3.5)

Assuming that x and d are individually and jointly wide-sense stationary (WSS), analogously to (2.17),
J(k) can be written as
J(k) = 𝜎d2 − cH (k)p − pH c(k) + cH (k)Rc(k) (3.6)

where R and p are defined respectively in (2.16) and (2.13). The optimum Wiener–Hopf solution is
c(k) = copt , where copt is given by (2.29). The corresponding minimum value of J(k) is Jmin , given
by (2.35).

3.1.1 The steepest descent or gradient algorithm

Our first step is to realize a deterministic iterative procedure to compute copt . We will see that this method
avoids the computation of the inverse R−1 ; however, it requires that R and p be known.

3
In this chapter, the estimation error is defined as as the difference between the desired signal and the filter output.
Depending on the application, the estimation error may be defined using the opposite sign. Some caution is therefore
necessary in using the equations of an adaptive filter.
3.1. The MSE design criterion 149

The steepest descent or gradient algorithm is defined as:

1
c(k + 1) = c(k) − 𝜇∇c(k) J(k) (3.7)
2
where ∇c(k) J(k) denotes the gradient of J(k) with respect to c (see (2.18)), 𝜇 is the adaptation gain, a
real-valued positive constant, and k is the iteration index, in general not necessarily coinciding with time
instants.
As J(k) is a quadratic function of the vector of coefficients, from (2.26) we get
∇c(k) J(k) = 2[Rc(k) − p] (3.8)
hence
c(k + 1) = c(k) − 𝜇[Rc(k) − p] (3.9)
In the scalar case (N = 1), for real-valued signals the above relations become:
J(k) = Jmin + 𝚛x (0)[c0 (k) − copt,0 ]2 (3.10)
and
𝜕
∇c0 J(k) = J(k) = 2𝚛x (0)[c0 (k) − copt,0 ] (3.11)
𝜕c0
The iterative algorithm is given by:
c0 (k + 1) = c0 (k) − 𝜇𝚛x (0)[c0 (k) − copt,0 ] (3.12)
The behaviour of J and the sign of ∇c0 J(k) as a function of c0 is illustrated in Figure 3.2. In the two
dimensional case (N = 2), the trajectory of ∇c J(k) is illustrated in Figure 3.3. We recall that in general
the gradient vector for c = c(k) is orthogonal to the locus of points with constant J that includes c(k).

∇<0
∇>0
∇=0
Jmin

0 copt, 0 c0(1) c0(0) c0

Figure 3.2 Behaviour of J and sign of the gradient vector ∇c in the scalar case (N = 1).

Note that the steepest descent algorithm (3.7) is commonly used to update the parameters of other
parametric functions (such as neural networks) in machine learning solutions.

Stability
Substituting for p the expression given by (2.27), the iterative algorithm (3.9) can be written as
c(k + 1) = c(k) − 𝜇[Rc(k) − Rcopt ]
(3.13)
= [I − 𝜇R]c(k) + 𝜇Rcopt
Defining the coefficient error vector as
Δc(k) = c(k) − copt (3.14)
150 Chapter 3. Adaptive transversal filters

c1
J3
J2
J1
copt, 1 Jmin

–∆c J(k) copt, 0 c0

(for small values of μ)

Figure 3.3 Loci of points with constant J and trajectory of ∇c J in the two dimensional case (N = 2).

from (3.13) we obtain

Δc(k + 1) = [I − 𝜇R]c(k) + [𝜇R − I]copt
(3.15)
= [I − 𝜇R]Δc(k)
Starting at k = 0 with arbitrary c(0), that is with Δc(0) = c(0) − copt , we determine now the conditions
for the convergence of c(k) to copt or, equivalently, for the convergence of Δc(k + 1) to 𝟎.
Using the decomposition (1.292), R = U𝚲UH , where U is the unitary matrix formed of eigenvectors
of R, and 𝚲 is the diagonal matrix of eigenvalues {𝜆i }, i = 1, … , N, and setting (see (2.41))

𝝂(k) = UH Δc(k) (3.16)

Eq. (3.15) becomes

𝝂(k + 1) = [I − 𝜇𝚲]𝝂(k) (3.17)

Conditions for convergence

As 𝚲 is diagonal, the i-th component of the vector 𝝂(k) in (3.17) satisfies the difference equation:

𝜈i (k + 1) = (1 − 𝜇𝜆i )𝜈i (k), i = 1, 2, … , N (3.18)

Hence, 𝜈i (k) as a function of k is given by (see Figure 3.4):

𝜈i (k) = (1 − 𝜇𝜆i )k 𝜈i (0) (3.19)

νi (k)

0 1 2 3 4 5 6 k

Figure 3.4 𝜈i (k) as a function of k for 𝜇𝜆i < 1 and |1 − 𝜇𝜆i | < 1. In the case 𝜇𝜆i > 1 and
|1 − 𝜇𝜆i | < 1, 𝜈i (k) is still decreasing in magnitude, but it assumes alternating positive and negative
values.
3.1. The MSE design criterion 151

The i-th component of the vector 𝝂(k) converges, that is

𝜈i (k) −−−−→ 0 ∀𝜈i (0) (3.20)
k→∞
if and only if
|1 − 𝜇𝜆i | < 1 (3.21)
or, equivalently,
−1 < 1 − 𝜇𝜆i < 1 (3.22)
As 𝜆i is positive (see (1.284)), we have that the algorithm converges if and only if
2
0<𝜇< , i = 1, 2, … , N (3.23)
𝜆i
If 𝜆max (𝜆min ) is the largest (smallest) eigenvalue of R, observing (3.23) the convergence condition can
be expressed as
2
0<𝜇< (3.24)
𝜆max
Correspondingly, observing (3.16) and (3.19) we obtain the expression of the vector of coefficients,
given by
c(k) = copt + U𝝂(k) = copt + [u1 , u2 , … , uN ]𝝂(k)
∑N
∑
N (3.25)
= copt + ui 𝜈i (k) = copt + ui (1 − 𝜇𝜆i )k 𝜈i (0)
i=1 i=1
Therefore, for each coefficient it results
∑
N
cn (k) = copt,n + ui,n (1 − 𝜇𝜆i )k 𝜈i (0), n = 0, … , N − 1 (3.26)
i=1

where ui,n is the n-th component of ui . In (3.26), the term ui,n (1 − 𝜇𝜆i )k 𝜈i (0) characterizes the i-th
mode of convergence. Note that, if the convergence condition (3.23) is satisfied, each coefficient cn ,
n = 0, … , N − 1, converges to the optimum solution as a weighted sum of N exponentials, each with the
time constant4
1 1
𝜏i = − ≃ , i = 1, 2, … , N (3.27)
ln |1 − 𝜇𝜆i | 𝜇𝜆i
where the approximation is valid if 𝜇𝜆i ≪ 1.

Adaptation gain
The speed of convergence, which is related to the inverse of the convergence time, depends on the choice
of 𝜇. We define as 𝜇opt the value of 𝜇 that minimizes the time constant of the slowest mode. If we let
𝜉(𝜇) = max |1 − 𝜇𝜆i | (3.28)
i
then we need to determine
min 𝜉(𝜇) (3.29)
𝜇

As illustrated in Figure 3.5, the solution is obtained for

1 − 𝜇𝜆min = −(1 − 𝜇𝜆max ) (3.30)
from which we get
2
𝜇opt = (3.31)
𝜆max + 𝜆min
4
The time constant is the number of iterations needed to reduce the signal associated with the i-th mode by a factor e.
152 Chapter 3. Adaptive transversal filters

|1– μλ i|
1 ξ(μ)

0
0 1 μopt 1 1 μ
λmax λi λmin

Figure 3.5 𝜉 and |1 − 𝜇𝜆i | as a function of 𝜇 for different values of 𝜆i : 𝜆min < 𝜆i < 𝜆max .

and
2 𝜆 − 𝜆min 𝜒(R) − 1
𝜉(𝜇opt ) = 1 − 𝜆min = max = (3.32)
𝜆max + 𝜆min 𝜆max + 𝜆min 𝜒(R) + 1
where 𝜒(R) = 𝜆max ∕𝜆min is the eigenvalue spread (1.299). We note that other values of 𝜇 (associated
with 𝜆max or 𝜆min ) cause a slower mode.
We emphasize that 𝜉(𝜇opt ) is a monotonic function of the eigenvalue spread (see Figure 3.6), and
consequently, the larger the eigenvalue spread, the slower the convergence.

0.8

0.6
ξ(μopt)

0.4

0.2

0
0 1 2 3 4 5 6 7 8 9 10
χ(R)

Figure 3.6 𝜉(𝜇opt ) as a function of the eigenvalue spread 𝜒(R) = 𝜆max ∕𝜆min .

Transient behaviour of the MSE

From (2.42) the general relation holds
∑
N
J(k) = Jmin + 𝜆i |𝜈i (k)|2 (3.33)
i=1
3.1. The MSE design criterion 153

Now using (3.19) we have

∑
N
J(k) = Jmin + 𝜆i (1 − 𝜇𝜆i )2k 𝜈i2 (0) (3.34)
i=1

Consequently, if the condition for convergence is verified, for k → ∞, J(k) → Jmin as a weighted sum of
exponentials. The i-th mode will have a time constant given by
1 1
𝜏MSE,i = − ≃ (3.35)
2 ln |1 − 𝜇𝜆i | 2𝜇𝜆i
assuming 𝜇𝜆i ≪ 1. We note that (3.34) is different from (3.26) because of the presence of 𝜆i as weight of
the i-th mode: consequently, modes associated with small eigenvalues tend to weigh less in the conver-
gence of J(k). In particular, let us examine the two dimensional case (N = 2). Recalling the observation
that J(k) increases more rapidly (slowly) in the direction of the eigenvector corresponding to 𝜆 = 𝜆max
(𝜆 = 𝜆min ) (see Figure 2.4), we have the following two cases.

Case 1 for 𝜆1 ≪ 𝜆2 . Choosing c(0) on the 𝜈2 axis (in correspondence of 𝜆max ), the iterative algorithm
has following behaviour
⎧ 1
⎪< 𝜆 non-oscillatory
⎪ max
⎪ 1
if 𝜇 ⎨ = convergent in one iteration (3.36)
𝜆max
⎪
⎪ 1
⎪> 𝜆 oscillatory around the minimum
⎩ max

Let u𝜆min and u𝜆max be the eigenvectors corresponding to 𝜆min and 𝜆max , respectively. If no further infor-
mation is given regarding the initial condition c(0), choosing 𝜇 = 𝜇opt the algorithm exhibits monotonic
convergence along u𝜆min , and an oscillatory behaviour around the minimum along u𝜆max .

Case 2 for 𝜆2 = 𝜆1 . Choosing 𝜇 = 1∕𝜆max , the algorithm converges in one iteration, independently of
the initial condition c(0).

3.1.2 The least mean square algorithm

The least mean square (LMS) or stochastic gradient algorithm is an algorithm with low computational
complexity that provides an approximation to the optimum Wiener–Hopf solution without requiring the
knowledge of R and p. Actually, the following instantaneous estimates are used:
̂
R(k) = x∗ (k)xT (k) (3.37)

and
p̂ (k) = d(k)x∗ (k) (3.38)

The gradient vector (3.8) is thus estimated as5

∇̂ ∗ ∗ T
c(k) J(k) = −2d(k)x (k) + 2x (k)x (k)c(k)
= −2x∗ (k)[d(k) − xT (k)c(k)] (3.39)
= −2x∗ (k)e(k)

5
We note that (3.39) represents an unbiased estimate of the gradient (3.8), but in general it also exhibits a large variance.
154 Chapter 3. Adaptive transversal filters

The equation for the adaptation of the filter coefficients, where k now denotes a given instant, becomes
1
c(k + 1) = c(k) − 𝜇 ∇̂ J(k) (3.40)
2 c(k)
that is
c(k + 1) = c(k) + 𝜇e(k)x∗ (k) (3.41)

Observation 3.1
The same equation is obtained for a cost function equal to |e(k)|2 , whose gradient is given by

∇̂ ∗
c(k) J(k) = −2e(k)x (k) (3.42)

Implementation
The block diagram for the implementation of the LMS algorithm is shown in Figure 3.7, with reference
to the following parameters and equations.

x(k) x(k – 1) x(k – 2) x(k – N + 1)

Tc Tc Tc

* c0(k) * c1(k) * c2(k) * cN – 1(k)

ACC ACC ACC ACC

y(k)
e(k) –
+
μ d(k)

Figure 3.7 Block diagram of an adaptive transversal filter adapted according to the LMS algorithm.

Parameters. Required parameters are:

1. N, number of coefficients of the filter.

2
2. 0 < 𝜇 < .
statistical power of input vector

Filter. The filter output is given by

y(k) = xT (k)c(k) (3.43)

Adaptation.
1. Estimation error
e(k) = d(k) − y(k) (3.44)

2. Coefficient vector adaptation

c(k + 1) = c(k) + 𝜇e(k)x∗ (k) (3.45)
3.1. The MSE design criterion 155

Initialization. If no a priori information is available, we set

c(0) = 0 (3.46)
The accumulators (ACCs) in Figure 3.7 are used to memorize the coefficients, which are updated by the
current value of 𝜇e(k)x∗ (k).

Computational complexity
For every iteration, we have 2N + 1 complex multiplication (N due to filtering and N + 1 to adaptation)
and 2N complex additions. Therefore, the LMS algorithm has a complexity of O(N).

Conditions for convergence

Recalling that the objective of the LMS algorithm is to approximate the Wiener–Hopf solution, we
introduce two criteria for convergence.

Convergence of the mean.

E[c(k)] −−−−→ copt (3.47)
k→∞
E[e(k)] −−−−→ 0 (3.48)
k→∞

In other words, it is required that the mean of the iterative solution converges to the Wiener–Hopf
solution and the mean of the estimation error approaches zero. To show the weakness of this criterion,
in Figure 3.8 we illustrate the results of a simple experiment for an input x given by a real-valued AR(1)
random process:
x(k) = −a x(k − 1) + 𝑤(k), 𝑤(k) ∼  (0, 𝜎𝑤2 ) (3.49)
where a = 0.95, and 𝜎x2 = 1 (i.e. 𝜎𝑤2 = 0.097).
For a first-order predictor, we adapt the coefficient, c(k), according to the LMS algorithm with 𝜇 = 0.1.
For c(0) = 0 and k ≥ 0, we compute
1. Predictor output
y(k) = c(k)x(k − 1) (3.50)
2. Prediction error
e(k) = d(k) − y(k) = x(k) − y(k) (3.51)
3. Coefficient update
c(k + 1) = c(k) + 𝜇e(k)x(k − 1) (3.52)

In Figure 3.8, realizations of the processes {x(k)}, {c(k)}, and {|e(k)|2 } are illustrated, as well as
mean values E[c(k)] and J(k) = E[|e(k)|2 ], estimated by averaging over 500 realizations; copt and Jmin
represent the Wiener–Hopf solution. From the plots in Figure 3.8, we observe two facts:
1. The random processes x and c exhibit a completely different behaviour, for which they may be
considered uncorrelated. It is interesting to observe that this hypothesis corresponds to assuming
the filter input vectors, {x(k)}, statistically independent. Actually, for small values of 𝜇, c tracks
mean statistical parameters associated with the process x and not the process itself.
2. Convergence of the mean is an easily reachable objective. By itself, however, it does not yield
the desired results, because the iterative solution c may exhibit very large oscillations around the
optimum solution. A constraint on the amplitude of the oscillations must be introduced.
156 Chapter 3. Adaptive transversal filters

2
x(k) 0
−2
0 50 100 150
k
(a)
0
E[c(k)]
−0.5
c(k)

−1
−a
0 50 100 150
k
(b)
2
0
J(k) = E[|e(k)|2]
J(k), |e(k)|2 (dB)

−2
−4
−6
−8
−10
Jmin
−12
0 50 100 150
k
(c)

Figure 3.8 Realizations of (a) input {x(k)}, (b) coefficient {c(k)}, and (c) square error {|e(k)|2 } for
a one-coefficient predictor (N = 1), adapted according to the LMS algorithm.
Convergence in the mean-square sense.
E[‖c(k) − copt ‖2 ] −−−−→ constant (3.53)
k→∞
J(k) = E[|e(k)|2 ] −−−−→ J(∞) constant (3.54)
k→∞
In other words, at convergence, both the mean of the coefficient error vector norm and the output MSE
must be finite. The quantity J(∞) − Jmin = Jex (∞) is the MSE in excess and represents the price paid
for using a random adaptation algorithm for the coefficients rather than a deterministic one, such as the
steepest-descent algorithm. In any case, we will see that the ratio Jex (∞)∕Jmin can be made small by
choosing a small adaptation gain 𝜇. We note that the coefficients are obtained by averaging in time the
quantity 𝜇e(k)x∗ (k). Choosing a small 𝜇 the adaptation will be slow and the effect of the gradient noise
on the coefficients will be strongly attenuated.

3.1.3 Convergence analysis of the LMS algorithm

We recall the following definitions.
1. Coefficient error vector
Δc(k) = c(k) − copt (3.55)
2. Optimum filter output error
emin (k) = d(k) − xT (k)copt (3.56)
3.1. The MSE design criterion 157

We also make the following assumptions.

1. c(k) is statistically independent of x(k).
2. The components of the coefficient error vector, transformed according to UH , 𝝂(k) = [𝜈1 (k), … ,
𝜈N (k)] = UH Δc(k) (see (3.16)), are orthogonal:
E[𝜈i (k)𝜈n ∗ (k)] = 0, i ≠ n i, n = 1, … , N (3.57)
This assumption is justified by the observation that the linear transformation that orthogonalizes
both x(k) (see (1.291)) and Δc(k) (see (3.16)) in the gradient algorithm is given by UH .
3. Fourth-order moments can be expressed as products of second-order moments (see (3.86)).
The adaptation equation of the LMS algorithm (3.41) can thus be written as
Δc(k + 1) = Δc(k) + 𝜇x∗ (k)[d(k) − xT (k)c(k)] (3.58)
T
Adding and subtracting x (k)copt to the terms within parentheses, we obtain
Δc(k + 1) = Δc(k) + 𝜇x∗ (k)[emin (k) − xT (k)Δc(k)]
(3.59)
= [I − 𝜇x∗ (k)xT (k)]Δc(k) + 𝜇emin (k)x∗ (k)
We note that Δc(k) depends only on the terms x(k − 1), x(k − 2), …
Moreover, with the change of variables (3.16), observing (3.33), the cost function6
J(k) = E [|e(k)|2 ] (3.60)
x,c

can be written as
∑
N
J(k) = Jmin + 𝜆i E[|𝜈i (k)|2 ] (3.61)
i=1

Convergence of the mean

Taking the expectation of (3.59) and exploiting the statistical independence between x(k) and Δc(k),
we get
E[Δc(k + 1)] = [I − 𝜇E[x∗ (k)xT (k)]]E[Δc(k)] + 𝜇E[emin (k)x∗ (k)] (3.62)
As E[x∗ (k)xT (k)] = R and the second term on the right-hand side of (3.62) vanishes for the orthogonality
property (2.31) of the optimum filter, we obtain the same equation as in the case of the steepest descent
algorithm:
E[Δc(k + 1)] = [I − 𝜇R]E[Δc(k)] (3.63)
Consequently, for the LMS algorithm, the convergence of the mean is obtained if
2
0<𝜇< (3.64)
𝜆max
Observing (3.25) and (3.32), and choosing the value of 𝜇 = 2∕(𝜆max + 𝜆min ), the vector E[Δc(k)] is
reduced at each iteration at least by the factor (𝜆max − 𝜆min )∕(𝜆max + 𝜆min ). We can therefore assume that
E[Δc(k)] becomes rapidly negligible with respect to the MSE during the process of convergence.

Convergence in the mean-square sense: real scalar case

The assumption of a filter with real-valued input and only one coefficient c(k) provides by a simple
analysis important properties of the convergence in the mean-square sense. From
Δc(k + 1) = (1 − 𝜇x2 (k))Δc(k) + 𝜇emin (k)x(k) (3.65)

6
E denotes the expectation with respect to both x and c.
x, c
158 Chapter 3. Adaptive transversal filters

because x(k) and Δc(k) are assumed to be statistically independent and x(k) is orthogonal to emin (k), and
assuming furthermore that x(k) and emin (k) are statistically independent, we get
E[Δc2 (k + 1)] = E[1 + 𝜇 2 x4 (k) − 2𝜇x2 (k)]E[Δc2 (k)] + 𝜇 2 E[x2 (k)]Jmin
(3.66)
+ 2E[(1 − 𝜇x2 (k))Δc(k)𝜇x(k)emin (k)]
where the last term vanishes, as Δc(k) has zero mean and is statistically independent of all other terms.
Assuming7 moreover E[x4 (k)] = E[x2 (k)x2 (k)] = 𝚛2x (0), (3.66) becomes

E[Δc2 (k + 1)] = (1 + 𝜇 2 𝚛2x (0) − 2𝜇𝚛x (0))E[Δc2 (k)] + 𝜇 2 𝚛x (0)Jmin (3.67)

Let
𝛾 = 1 + 𝜇 2 𝚛2x (0) − 2𝜇𝚛x (0) (3.68)

whose behaviour as a function of 𝜇 is given in Figure 3.9. Then for the convergence of the difference
equation (3.67), it must be |𝛾| < 1. Consequently, 𝜇 must satisfy the condition
2
0<𝜇< (3.69)
𝚛x (0)
Moreover, assuming 𝜇𝚛x (0) ≪ 1, we get
𝜇 2 𝚛x (0)
E[Δc2 (∞)] = J
𝜇𝚛x (0)(2 − 𝜇𝚛x (0)) min
𝜇 (3.70)
= J
(2 − 𝜇𝚛x (0)) min
𝜇
≃ Jmin
2

0.8

0.6
γ

0.4

0.2

0
0 1/rx(0) 2/rx(0)
μ

Figure 3.9 𝛾 as a function of 𝜇.

7
In other books the Gaussian assumption is made, whereby E[x4 (k)] = 3𝚛2x (0). The conclusions of the analysis are
similar.
3.1. The MSE design criterion 159

Likewise, from
e(k) = d(k) − x(k)c(k) = emin (k) − Δc(k)x(k) (3.71)
it turns out
E[e2 (k)] = E[e2min (k)] + E[x2 (k)]E[Δc2 (k)] (3.72)
that is
J(k) = Jmin + 𝚛x (0)E[Δc2 (k)] (3.73)
In particular, for k → ∞, we have
𝜇
J(∞) ≃ Jmin + 𝚛x (0) Jmin (3.74)
2
The relative MSE deviation, or misadjustment, is:
J(∞) − Jmin J (∞) 𝜇
MSD = = ex = 𝚛x (0) (3.75)
Jmin Jmin 2

Convergence in the mean-square sense: general case

The convergence theory given here follows the method developed in [3]. With the change of variables
(3.16), (3.59) becomes
𝝂(k + 1) = [I − 𝜇UH x∗ (k)xT (k)U]𝝂(k) + 𝜇emin (k)UH x∗ (k) (3.76)
Let us define
x̃ (k) = [̃x1 (k), … , x̃ N (k)]T = UT x(k) (3.77)
and
𝛀 = x̃ ∗ (k)̃xT (k) = UH x∗ (k)xT (k)U (3.78)
N × N matrix with elements
𝛀(i, n) = x̃ i ∗ (k)̃xn (k), i, n = 1, … , N (3.79)
From (1.291) we get
E[𝛀] = 𝚲 (3.80)
hence the components {̃xi (k)} are mutually orthogonal. Then (3.76) becomes
𝝂(k + 1) = [I − 𝜇𝛀]𝝂(k) + 𝜇emin (k)̃x∗ (k) (3.81)
Recalling Assumption 1 of the convergence analysis, and assuming emin (k) and x(k) are not only orthog-
onal but also statistically independent, and consequently emin (k) independent of x̃ (k), the correlation
matrix of 𝝂 ∗ at the instant k + 1 can be expressed as
E[𝝂 ∗ (k + 1)𝝂 T (k + 1)] = E[(I − 𝜇𝛀∗ )𝝂 ∗ (k)𝝂 T (k)(I − 𝜇𝛀T )] + 𝜇 2 Jmin E[𝛀∗ ] (3.82)
Observing (3.80), the second term at the right-hand side of (3.82) is equal to 𝜇2 Jmin 𝚲. Moreover, con-
sidering that 𝝂(k) is statistically independent of x(k) and x̃ (k), the first term can be written as
E [(I − 𝜇𝛀∗ )𝝂 ∗ (k)𝝂 T (k)(I − 𝜇𝛀T )] = E{(I − 𝜇𝛀∗ )E[𝝂 ∗ (k)𝝂 T (k)](I − 𝜇𝛀T )}
x,𝝂 x 𝝂 (3.83)
= E[(I − 𝜇𝛀∗ )(I − 𝜇𝛀T )]E[𝝂 ∗ (k)𝝂 T (k)]
Recalling Assumption 2 of the convergence analysis, we get that the matrix E[𝝂 ∗ (k) 𝝂 T (k)] is diagonal,
with elements on the main diagonal given by the vector
𝜼T (k) = [𝜂1 (k), … , 𝜂N (k)] = [E[|𝜈1 (k)|2 ], … , E[|𝜈N (k)|2 ]] (3.84)
160 Chapter 3. Adaptive transversal filters

Observing (3.79), the elements with indices (i, i) of the matrix expressed by (3.83) are given by
[ ]
∑
N
E 𝜂i (k) + 𝜇 2 𝛀 (i, n)𝛀(n, i)𝜂n (k) − 2𝜇𝛀(i, i)𝜂i (k)
∗
n=1
∑
N
(3.85)
= 𝜂i (k) + 𝜇 2 E[|̃xi (k)|2 |̃xn (k)|2 ]𝜂n (k) − 2𝜇𝜆i 𝜂i (k)
n=1
∑N
= 𝜂i (k) + 𝜇 2 𝜆i 𝜆n 𝜂n (k) − 2𝜇𝜆i 𝜂i (k)
n=1

where, recalling Assumption 3 of the convergence analysis,

E[|̃xi (k)|4 ] = E[|̃xi (k)|2 ]E[|̃xi (k)|2 ] = 𝜆2i (3.86)

Let

𝝀T = [𝜆1 , … , 𝜆N ] (3.87)

be the vector of eigenvalues of R, and

⎡(1 − 𝜇𝜆1 )2 𝜇 2 𝜆1 𝜆2 … 𝜇 2 𝜆1 𝜆N ⎤
⎢ 2 ⎥
𝜇 𝜆2 𝜆1 (1 − 𝜇𝜆2 )2 … 𝜇 2 𝜆2 𝜆N ⎥
B=⎢ (3.88)
⎢ ⋮ ⋮ ⋱ ⋮ ⎥
⎢ 𝜇2 𝜆 𝜆 𝜇 2𝜆 𝜆 … (1 − 𝜇𝜆N )2 ⎥⎦
⎣ N 1 N 2

N × N symmetric positive definite matrix with positive elements. From (3.82) and (3.85), we obtain the
relation
𝜼(k + 1) = B𝜼(k) + 𝜇 2 Jmin 𝝀 (3.89)

Using the properties of B, the general decomposition (2.139) becomes

B = V diag(𝜎1 , … , 𝜎N )V H (3.90)

where {𝜎i } denote the eigenvalues of B, and V is the unitary matrix formed by the eigenvectors {𝒗i } of
B. After simple steps, similar to those applied to get (3.25) from (3.13), and using the relation
∑
N
N𝚛x (0) = tr R = 𝜆i (3.91)
i=1

the solution of the vector difference equation (3.89) is given by:

∑
N
𝜇Jmin
𝜼(k) = i 𝜎ik 𝒗i + 1, k≥0 (3.92)
i=1
2 − 𝜇N𝚛x (0)

where 1 = [1, 1, … , 1]T . In (3.92), the constants {i } are determined by the initial conditions
( )
𝜇Jmin
i = 𝒗i H 𝜼(0) − 1 , i = 1, … , N (3.93)
2 − 𝜇N𝚛x (0)
where the components of 𝜼(0) depend on the choice of c(0) according to (3.84) and (3.16):

𝜂n (0) = E[|𝜈n (0)|2 ] = E[|un H Δc(0)|2 ], n = 1, … , N (3.94)

Using (3.84) and (3.87), the cost function J(k) given by (3.61) becomes

J(k) = Jmin + 𝝀T 𝜼(k) (3.95)

3.1. The MSE design criterion 161

Substituting the result (3.92) in (3.95), we get

∑
N
2
J(k) = i 𝜎ik + J (3.96)
i=1
2 − 𝜇N𝚛x (0) min
where
i = i 𝝀T 𝒗i (3.97)
The first term on the right-hand side of (3.96) describes the convergence behaviour of the mean-square
error, whereas the second term gives the steady-state value. Therefore, the further investigation of the
properties of the matrix B will allow us to characterize the transient behaviour of J.

Fundamental results
From the above convergence analysis, we will obtain some fundamental properties of the LMS algo-
rithm.
1. The transient behaviour of J does not exhibit oscillations; this result is obtained by observing the
properties of the eigenvalues of B.
2. The LMS algorithm converges if the adaptation gain 𝜇 satisfies the condition
2
0<𝜇< (3.98)
statistical power of input vector
In fact the adaptive system is stable and J converges to a constant steady-state value under the
conditions |𝜎i | < 1, i = 1, … , N. This happens if
2
0<𝜇< (3.99)
N𝚛x (0)
Conversely, if 𝜇 satisfies (3.99), from (3.88) the sum of the elements of the i-th row of B satisfies
∑
N
[B]i,n = 1 − 𝜇𝜆i (2 − 𝜇N𝚛x (0)) < 1 (3.100)
n=1

A matrix with these properties and whose elements are all positive has eigenvalues with absolute
value less than one. In particular, being
∑
N
𝜆i = tr [R]
i=1
= N𝚛x (0)
(3.101)
∑
N−1
= E[|x(k − i)|2 ]
i=0

the statistical power of input vector, Eq. (3.99) becomes (3.98).

We recall that, for convergence of the mean, it must be
2
0<𝜇< (3.102)
𝜆max
but since
∑
N
𝜆i > 𝜆max (3.103)
i=1

the condition for convergence in the mean square implies convergence of the mean. In other words,
convergence in the mean square imposes a tighter bound to allowable values of the adaptation gain
162 Chapter 3. Adaptive transversal filters

𝜇 than that imposed by convergence of the mean (3.102). The new bound depends on the number
of coefficients, rather than on the eigenvalue distribution of the matrix R. The relation (3.99) can
be intuitively explained noting that, for a given value of 𝜇, an increase in the number of coefficients
causes an increase in the excess MSE due to fluctuations of the coefficients around the mean value.
Increasing the number of coefficients without reducing the value of 𝜇 eventually leads to instability
of the adaptive system.
3. Equation (3.96) reveals a simple relation between the adaptation gain 𝜇 and the value J(k) in the
steady state (k → ∞):
2
J(∞) = J (3.104)
2 − 𝜇N𝚛x (0) min
from which the excess MSE is given by
𝜇N𝚛x (0)
Jex (∞) = J(∞) − Jmin = J (3.105)
2 − 𝜇N𝚛x (0) min
and the misadjustment is
Jex (∞) 𝜇N𝚛x (0) 𝜇
MSD = = ≃ N𝚛x (0) (3.106)
Jmin 2 − 𝜇N𝚛x (0) 2
for 𝜇 ≪ 2∕(N𝚛x (0)).

Observations
1. For 𝜇 → 0 all eigenvalues of B tend towards 1.
2. As shown below, a small eigenvalue of the matrix R (𝜆i → 0) determines a large time constant for
one of the convergence modes of J, as 𝜎i → 1. However, a large time constant of one of the modes
implies a low probability that the corresponding term contributes significantly to the MSE.

Proof. If 𝜆i = 0, the i-th row of B becomes (0, … , 0, [B]i,i = 1, 0, … , 0). Consequently, 𝜎i = 1 and
𝒗i T = (0, … , 0, 𝑣i,i = 1, 0, … , 0). As 𝝀T 𝒗i = 0, from (3.97) we get i = 0. ◽

It is generally correct to state that a large eigenvalue spread of R determines a slow convergence of
J to the steady state. However, the fact that modes with a large time constant usually contribute to
J less than the modes that converge more rapidly, mitigates this effect. Therefore, the convergence
of J is less influenced by the eigenvalue spread of R than would be the convergence of Δc(k).
3. If all eigenvalues of the matrix R are equal,8 𝜆i = 𝚛x (0), i = 1, … , N, the maximum eigenvalue of
the matrix B is given by
𝜎imax = 1 − 𝜇𝚛x (0)(2 − 𝜇N𝚛x (0)) (3.107)

The remaining eigenvalues of B do not influence the transient behaviour of J, since i = 0, i ≠ imax .

Proof. It is easily verified that 𝜎imax is an eigenvalue of B and 𝒗Ti = N −1∕2 [1, 1, … , 1] is the
max
corresponding eigenvector. Moreover, the Perron–Frobenius theorem affirms that the maximum
eigenvalue of a positive matrix B is a positive real number and that the elements of the correspond-
ing eigenvector are positive real numbers [4]. Since all elements of 𝒗imax are positive, it follows
that 𝜎imax is the maximum eigenvalue of B. Moreover, because 𝒗imax is parallel to 𝝀T , the other
eigenvectors of B are orthogonal to 𝝀. Hence, i = 0, i ≠ imax . ◽

8
This occurs, for example, if the input x is white noise.
3.1. The MSE design criterion 163

4. If all eigenvalues of the matrix R are equal, 𝜆i = 𝚛x (0), i = 1, … , N, combining (3.96) with the
(3.107) and considering a time varying adaptation gain 𝜇(k), we obtain
J(k + 1) ≃ [1 − 𝜇(k)𝚛x (0)(2 − 𝜇(k)N𝚛x (0))]J(k) + 2𝜇(k)𝚛x (0)Jmin (3.108)
The maximum convergence rate of J is obtained for the adaptation gain
1 J(k) − Jmin
𝜇opt (k) = (3.109)
N𝚛x (0) J(k)
As the condition J(k) ≫ Jmin is normally verified at the beginning of the iteration process, it results
1
𝜇opt (k) ≃ (3.110)
N𝚛x (0)
and ( )
1
J(k + 1) ≃ 1 − J(k) (3.111)
N
We note that the number of iterations required to reduce the value of J(k) by one order of magnitude
is approximately 2.3N.
5. Equation (3.92) indicates that at steady state, all elements of 𝜼 become equal. Consequently, recall-
ing Assumption 2 of the convergence analysis, in steady state the filter coefficients are uncorrelated
random variables with equal variance. The mean corresponds to the optimum vector copt .
6. In case the LMS algorithm is used to estimate the coefficients of a system that slowly changes in
time, the adaptation gain 𝜇 must be strictly larger than 0. In this case, the value of J in steady state
varies with time and is given by the sum of three terms:
Jtot (k) = Jmin (k) + Jex (∞) + J𝓁 (3.112)
where Jmin (k) corresponds to the Wiener–Hopf solution, Jex (∞) depends instead on the LMS algo-
rithm and is directly proportional to 𝜇, and J𝓁 depends on the ability of the LMS algorithm to
track the system variations and expresses the lag error in the estimate of the coefficients. It turns
out that J𝓁 is inversely proportional to 𝜇. Therefore, for time-varying systems 𝜇 must be chosen as
a compromise between Jex and J𝓁 and cannot be arbitrarily small [5–7].

Final remarks

1. The LMS algorithm is easy to implement.

2. The relatively slow convergence is influenced by 𝜇, the number of coefficients and the eigenvalues
of R. In particular it must be
2 2
0<𝜇< = (3.113)
N𝚛x (0) statistical power of input vector
3. Choosing a small 𝜇 results in a slow adaptation, and in a small excess MSE at convergence Jex (∞).
For a large 𝜇, instead, the adaptation is fast at the expense of a large Jex (∞).
4. Jex (∞) is determined by the large eigenvalues of R, whereas the speed of convergence of E[c(k)]
is imposed by 𝜆min . If the eigenvalue spread of R increases, the convergence of E[c(k)] becomes
slower; on the other hand, the convergence of J(k) is less sensitive to this parameter. Note, however,
that the convergence behaviour depends on the initial condition c(0) [8].

3.1.4 Other versions of the LMS algorithm

In the Section 3.1.3, the basic version of the LMS algorithm has been presented. We now give a brief
introduction to other versions that can be used for various applications.
164 Chapter 3. Adaptive transversal filters

Leaky LMS
The leaky LMS algorithm is a variant of the LMS algorithm that uses the following adaptation equation:
c(k + 1) = (1 − 𝜇𝛼)c(k) + 𝜇e(k)x∗ (k) (3.114)
with 0 < 𝛼 ≪ 𝚛x (0). This equation corresponds to the following cost function:
J(k) = E[|e(k)|2 ] + 𝛼E[||c(k)||2 ] (3.115)
where, as usual,
e(k) = d(k) − cT (k)x(k) (3.116)
In other words, the cost function includes an additional term proportional to the norm of the vector of
coefficients. In steady state, we get
lim E[c(k)] = (R + 𝛼I)−1 p (3.117)
k→∞
It is interesting to give another interpretation to what has been stated. Observing (3.117), the applica-
tion of the leaky LMS algorithm results in the addition of a small constant 𝛼 to the terms on the main
diagonal of the correlation matrix of the input process; one obtains the same result by summing white
noise with statistical power 𝛼 to the input process. Both approaches are useful to make invertible an
ill-conditioned matrix R, or to accelerate the convergence of the LMS algorithm. It is usually sufficient
to choose 𝛼 two or three orders of magnitude smaller than 𝚛x (0), in order not to modify substantially the
original Wiener–Hopf solution. Therefore, the leaky LMS algorithm is used when the Wiener problem
is ill-conditioned, and multiple solutions exist.

Sign algorithm
There are adaptation equations that are simpler to implement, at the expense of a lower speed of con-
vergence, for the same J(∞). Three versions are9
⎧sgn(e(k))x∗ (k)
⎪
c(k + 1) = c(k) + 𝜇 ⎨e(k)sgn (x∗ (k)) (3.119)
⎪sgn(e(k))sgn (x∗ (k))
⎩
Note that the function sgn(e(k)) never goes to zero, hence 𝜇 must be very small to have good convergence
properties. The first version has as objective the minimization of the cost function
J(k) = E[|e(k)|] (3.120)

Normalized LMS
In the LMS algorithm, if some x(k) assume large values, the adaptation algorithm is affected by strong
noise in the gradient. This problem can be overcome by choosing an adaptation gain 𝜇 of the type:
𝜇̃
𝜇= (3.121)
p + 𝙼̂ x (k)
where 0 < 𝜇̃ < 2, and
∑
N−1
𝙼̂ x (k) = ||x||2 = |x(k − i)|2 (3.122)
i=0

9
The sign of a complex-valued vector is defined as
sgn (x(k)) = [sgn (xI (k)) + j sgn (xQ (k)), … , sgn (xI (k − N + 1)) + j sgn (xQ (k − N + 1))] (3.118)
3.1. The MSE design criterion 165

or, alternatively,
𝙼̂ x (k) = N 𝙼̂ x (k) (3.123)
where 𝙼̂ x (k) is the estimate of the statistical power of x(k). A simple estimate is obtained by the iterative
equation (see (1.395)):
𝙼̂ x (k) = a𝙼̂ x (k − 1) + (1 − a)|x(k)|2 (3.124)
where 0 < a < 1, with time constant given by
−1 1
𝜏= ≃ (3.125)
ln a 1 − a
for a ≃ 1. In (3.121), p is a positive parameter that is introduced to avoid that the denominator becomes
too small; typically
1
p≃ 𝙼 (3.126)
10 x
The normalized LMS algorithm has a speed of convergence that is potentially higher than the standard
algorithm, for uncorrelated as well as correlated input signals [9]. To be able to apply the normalized
algorithm, however, some knowledge of the input process is necessary, in order to assign the values of
𝙼x and p so that the adaptation process does not become unstable.

Variable adaptation gain

In the following variants of the LMS algorithms, the coefficient 𝜇 varies with time.
1. Two values of 𝜇:
a. Initially, a large value of 𝜇 is chosen for fast convergence, for example 𝜇 = 1∕(N𝚛x (0)).
b. Subsequently, 𝜇 is reduced to achieve a smaller J(∞).
For a choice of 𝜇 of the type {
𝜇1 0 ≤ k ≤ K1
𝜇= (3.127)
𝜇2 k ≥ K1
the behaviour of J is illustrated in Figure 3.10.

μ1
J(k)

μ2 = μ1/2

Jmin

0 K1 k

Figure 3.10 Behaviour of J(k) obtained by using two values of 𝜇.

166 Chapter 3. Adaptive transversal filters

2. Decreasing 𝜇: For a time-invariant system, the adaptation gain usually selected for application with
the sign algorithm (3.119) is given by
𝜇1
𝜇(k) = , k≥0 (3.128)
𝜇2 + k
3. 𝜇 proportional to e(k): The following expression of 𝜇 is used:
𝜇(k + 1) = 𝛼1 𝜇(k) + 𝛼2 |e(k)|2 (3.129)
with 𝜇 limited to the range [𝜇min , 𝜇max ]. Typical values are 𝛼1 ≃ 1 and 𝛼2 ≪ 1.
4. Vector of values of 𝜇: Let 𝝁T = [𝜇0 , … , 𝜇N−1 ]; two approaches are possible.
a. Initially, larger values 𝜇i are chosen in correspondence of those coefficients ci that have larger
amplitude.
b. 𝜇i changes with time following the rule
⎧ 1
⎪ 𝜇i (k) 𝛼 if the i-th component of the gradient has al𝑤ays changed sign in the
⎪ last m0 iterations,
𝜇i (k + 1) = ⎨
⎪ 𝜇i (k)𝛼 if the i-th component of the gradient has ne𝑣er changed sign in the
⎪ last m1 iterations,
⎩
(3.130)
with 𝜇 limited to the range [𝜇min , 𝜇max ]. Typical values are m0 , m1 ∈ {1, 3} and 𝛼 = 2.

3.1.5 Example of application: the predictor

We consider a real AR(2) process of unit power, described by the equation
x(k) = −𝚊1 x(k − 1) − 𝚊2 x(k − 2) + 𝑤(k) (3.131)
with 𝑤 additive white Gaussian noise (AWGN), and
𝚊1 = −1.3, 𝚊2 = 0.995 (3.132)
From (1.474), the roots of A(z) are given by 𝜚e±𝜑0 , where
√
𝜚 = 𝚊2 = 0.997 (3.133)
and ( )
𝚊
𝜑0 = cos−1 − 1 = 2.28 rad (3.134)
2𝜚
Being 𝚛x (0) = 𝜎x2 = 1, from the (1.479) we have that the statistical power of 𝑤 is given by
1 − 𝚊2
𝜎𝑤2 = [(1 + 𝚊2 )2 − 𝚊21 ] = 0.0057 = −22.4 dB (3.135)
1 + 𝚊2
We construct a predictor for x of order N = 2 with coefficients cT = [c1 , c2 ], as illustrated in Figure 3.11,
using the LMS algorithm and some of its variants [10]. From (2.78), we expect to find in steady state
c ≃ −a (3.136)
that is c1 ≃ −𝚊1 , c2 ≃ −𝚊2 , and 𝜎e2 ≃ 𝜎𝑤2 . In any case, the predictor output is given by
y(k) = cT (k)x(k − 1)
(3.137)
= c1 (k)x(k − 1) + c2 (k) x(k − 2)
with prediction error
e(k) = x(k) − y(k) (3.138)
3.1. The MSE design criterion 167

x(k – 1) x(k – 2)
Tc

c1(k) c2(k)

y(k)
–
e(k) = d(k) – y(k)
+
d(k) = x(k)

Figure 3.11 Predictor of order N = 2.

For the predictor of Figure 3.11, we consider now various versions of the adaptive LMS algorithm and
their relative performance.

Example 3.1.1 (Standard LMS)

The equation for updating the coefficient vector is
c(k + 1) = c(k) + 𝜇e(k)x(k − 1) (3.139)
Convergence curves are plotted in Figure 3.12 for a single realization and for the mean (estimated over
500 realizations) of the coefficients and of the square prediction error, for 𝜇 = 0.04.

−a1 0.2
1.2 0
1
−0.2 average
0.8
average
c1(k)

−0.4
c2(k)

0.6
0.4 −0.6
0.2 −0.8
0
−a2 −1
−0.2
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)

0
J(k), │e(k)│2 (dB)

−5
J(k)
−10
−15
−20

σw2 −250 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.12 Convergence curves for the predictor of order N = 2, obtained by the standard LMS
algorithm. (a) c1 (k), (b) c2 (k), (c) J(k) and |e(k)|2 , all versus k.
168 Chapter 3. Adaptive transversal filters

−a1 0.2
1.2 0
1
μ = 0.01 −0.2 μ = 0.1
0.8
μ = 0.04 −0.4
c1(k)

c2(k)
0.6 μ = 0.04
0.4 μ = 0.1 −0.6
μ = 0.01
0.2 −0.8
0 −a2 −1
−0.2
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)
0
μ = 0.1
−5
μ = 0.04
J(k) (dB)

−10
μ = 0.01
−15
−20
σw2
−25
0 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.13 Comparison among curves of convergence of the mean obtained by the standard LMS
algorithm for three values of 𝜇. (a) c1 (k), (b) c2 (k), (c) J(k), all versus k.

In Figure 3.13, a comparison is made among the curves of convergence of the mean for three values
of 𝜇. We observe that, by decreasing 𝜇, the excess error decreases, thus giving a more accurate solution,
but the convergence time increases.

Example 3.1.2 (Leaky LMS)

The equation for updating the coefficient vector is
c(k + 1) = (1 − 𝜇𝛼)c(k) + 𝜇e(k)x(k − 1) (3.140)
Convergence curves are plotted in Figure 3.14 for a single realization and for the mean (estimated
over 500 realizations) of the coefficients and of the square prediction error, for 𝜇 = 0.04 and 𝛼 = 0.01.
We note that the steady-state values are worse than in the previous case.

Example 3.1.3 (Normalized LMS)

The equation for updating the coefficient vector is (3.139) with adaptation gain 𝜇 is of the type
𝜇̃
𝜇(k) = (3.141)
p + N 𝜎̂ x2 (k)
where
𝜎̂ x2 (k) = a𝜎̂ x2 (k − 1) + (1 − a)|x(k)|2 , s k ≥ 0 (3.142)
1
𝜎̂ x2 (−1) = [|x(−1)|2 + |x(−2)|2 ] (3.143)
2
with a = 1 − 2−5 = 0.97 and
1
p= E[||x||2 ] = 0.2 (3.144)
10
3.1. The MSE design criterion 169

−a1 0.2
1.2 0
1
−0.2 average
0.8 average
c1(k)

c2(k)
0.6 −0.4
0.4 −0.6
0.2 −0.8
0 −a2 −1
−0.2
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)
0
J(k)
J(k), │e(k)│2 (dB)

−5
−10
−15
−20

σw2 −250 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.14 Convergence curves for the predictor of order N = 2, obtained by the leaky LMS. (a)
c1 (k), (b) c2 (k), (c) J(k) and |e(k)|2 , all versus k.

Convergence curves are plotted in Figure 3.15 for a single realization and for the mean (estimated over
500 realizations) of the coefficients and of the square prediction error, for 𝜇̃ = 0.08. We note that, with
respect to the standard LMS algorithm, the convergence is considerably faster.
A direct comparison of the convergence curves obtained in the previous examples is given in
Figure 3.16.

Example 3.1.4 (Sign LMS algorithm)

We consider the three versions of the sign LMS algorithm:
(1) c(k + 1) = c(k) + 𝜇 sgn(e(k))x(k − 1),
(2) c(k + 1) = c(k) + 𝜇e(k) sgn(x(k − 1)),
(3) c(k + 1) = c(k) + 𝜇 sgn(e(k)) sgn(x(k − 1)).
A comparison of convergence curves is given in Figure 3.17 for the three versions of the sign LMS
algorithm, for 𝜇 = 0.04.
It turns out that version (2), where the estimation error in the adaptation equation is not quantized,
yields the best performance in steady state. Version (3), however, yields fastest convergence. To decrease
the prediction error in steady state for versions (1) and (3), the value of 𝜇 could be further lowered, at
the expense of reducing the speed of convergence.

Observation 3.2
As observed on page 74, for an AR process x, if the order of the predictor is greater than the required
minimum, the correlation matrix results ill-conditioned with a large eigenvalue spread. Thus, the con-
vergence of the LMS prediction algorithm may turn out to be extremely slow and can lead to a solution
170 Chapter 3. Adaptive transversal filters

−a1 0.2
1.2
0
0.8 −0.2
average average
c1(k)

−0.4

c2(k)
0.6
0.4 −0.6
0.2 −0.8
0
−0.2 −a2 −1
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)
0
J(k), │e(k)│2 (dB)

−5
J(k)
−10
−15
−20

σw2 −250 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.15 Convergence curves for the predictor of order N = 2, obtained by the normalized LMS
algorithm. (a) c1 (k), (b) c2 (k), (c) J(k) and |e(k)|2 , all versus k.

0.2
−a1
1.2 leaky 0
1 −0.2
0.8 standard normalized
c1(k)

−0.4
c2(k)

0.6 standard
0.4 −0.6
normalized leaky
0.2 −0.8
0 −a2 −1
−0.2
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)
0
normalized
−5
standard
J(k) (dB)

−10
leaky
−15
−20
σw2
−25
0 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.16 Comparison of convergence curves for the predictor of order N = 2, obtained by three
versions of the LMS algorithm. (a) c1 (k), (b) c2 (k), (c) J(k), all versus k.
3.2. The recursive least squares algorithm 171

0.2
−a1 1.2
0
1 ver.3 −0.2
0.8 ver.2
ver.1 −0.4
c1(k)

c2(k)
0.6 ver.1
0.4 ver.2 −0.6
ver.3
0.2 −0.8
0
−a2 −1
−0.2
0 200 400 600 800 1000 0 200 400 600 800 1000
k k
(a) (b)
0
ver.3
−5
ver.1
J(k) (dB)

−10
ver.2
−15
−20
σw2
−25
0 100 200 300 400 500 600 700 800 900 1000
k
(c)

Figure 3.17 Comparison of convergence curves obtained by three versions of the sign LMS
algorithm. (a) c1 (k), (b) c2 (k), (c) J(k), all versus k.

quite different from the Yule–Walker solution. In this case, it is necessary to adopt a method that ensures
the stability of the error prediction filter, such as the leaky LMS.

3.2 The recursive least squares algorithm

We now consider a recursive algorithm to estimate the vector of coefficients c by an LS method, named
recursive least squares (RLS) algorithm. Note that this algorithm is an implementation of the Kalman
filter algorithm, for a non-evolving state. The RLS algorithm is characterized by a speed of convergence
that can be one order of magnitude faster than the LMS algorithm, obtained at the expense of a larger
computational complexity.
With reference to the system illustrated in Figure 3.18, we introduce the following quantities:

1. Input vector at instant i

xT (i) = [x(i), x(i − 1), … , x(i − N + 1)] (3.145)

2. Coefficient vector at instant k

cT (k) = [c0 (k), c1 (k), … , cN−1 (k)] (3.146)

3. Filter output signal at instant i, obtained for the vector of coefficients c(k)

y(i) = cT (k)x(i) = xT (i)c(k) (3.147)

4. Desired output at instant i

d(i) (3.148)
172 Chapter 3. Adaptive transversal filters

x(i) x(i – 1) x(i – 2) x(i – N + 1)

Tc Tc Tc

c0(k) c1(k) c2(k) cN – 1(k)

y(i)
–
e(i)
+ d(i)

Figure 3.18 Reference system for a RLS adaptive algorithm.

At instant k, based on the observation of the sequences

{x(i)}{d(i)}, i = 1, 2, … , k (3.149)

the criterion for the optimization of the vector of coefficients c(k) is the minimum sum of square errors
up to instant k. Defining
∑ k
(k) = 𝜆k−i |e(i)|2 (3.150)
i=1

we want to find
min (k) (3.151)
c(k)

where the error signal is e(i) = d(i) − xT (i)c(k).

Two observations arise:

● 𝜆 is a forgetting factor, that enables proper filtering operations even with non-stationary signals or
slowly time-varying systems. The memory of the algorithm is approximately 1∕(1 − 𝜆).
● This problem is the classical LS problem (2.92), applied to a sequence of pre-windowed samples
with the exponential weighting factor 𝜆k .

Normal equation
Using the gradient method, the optimum value of c(k) satisfies the normal equation

𝚽(k)c(k) = 𝝑(k) (3.152)

where
∑
k
𝚽(k) = 𝜆k−i x∗ (i)xT (i) (3.153)
i=1
∑ k
𝝑(k) = 𝜆k−i d(i)x∗ (i) (3.154)
i=1

From (3.152), if 𝚽−1 (k) exists, the solution is given by

c(k) = 𝚽−1 (k)𝝑(k) (3.155)

3.2. The recursive least squares algorithm 173

Derivation
To solve the normal equation by the inversion of 𝚽(k) may be too hard, especially if N is large. Therefore,
we seek a recursive algorithm for k = 1, 2, ….
Both expressions of 𝚽(k) and 𝝑(k) can be written recursively. From
∑
k−1
𝚽(k) = 𝜆k−i x∗ (i)xT (i) + x∗ (k)xT (k) (3.156)
i=1

it follows that
𝚽(k) = 𝜆𝚽(k − 1) + x∗ (k)xT (k) (3.157)
and similarly
𝝑(k) = 𝜆𝝑(k − 1) + d(k)x∗ (k) (3.158)
We now recall the following identity known as matrix inversion lemma [11]. Let
A = B−1 + CD−1 CH (3.159)
where A, B, and D are positive definite matrices. Then
A−1 = B − BC (D + CH BC)−1 CH B (3.160)
For
A = 𝚽(k), B−1 = 𝜆𝚽(k − 1), C = x∗ (k), D = [1] (3.161)
Eq. (3.160) becomes
𝜆−1 𝚽−1 (k − 1)x∗ (k)xT (k)𝜆−1 𝚽−1 (k − 1)
𝚽−1 (k) = 𝜆−1 𝚽−1 (k − 1) − (3.162)
1 + xT (k)𝜆−1 𝚽−1 (k − 1)x∗ (k)
We introduce two quantities:
P(k) = 𝚽−1 (k) (3.163)
and
𝜆−1 𝚽−1 (k − 1)x∗ (k)
k∗ (k) = (3.164)
1 + 𝜆−1 xT (k)𝚽−1 (k − 1)x∗ (k)
also called the Kalman vector gain. From (3.162), we have the recursive relation
P(k) = 𝜆−1 P(k − 1) − 𝜆−1 k∗ (k)xT (k)P(k − 1) (3.165)
We derive now a simpler expression for k∗ (k). From (3.164), we obtain
k∗ (k)[1 + 𝜆−1 xT (k)𝚽−1 (k − 1)x∗ (k)] = 𝜆−1 𝚽−1 (k − 1)x∗ (k) (3.166)
from which we get
k∗ (k) = 𝜆−1 P(k − 1)x∗ (k) − 𝜆−1 k∗ (k)xT (k)P(k − 1)x∗ (k)
(3.167)
= [𝜆−1 P(k − 1) − 𝜆−1 k∗ (k)xT (k)P(k − 1)]x∗ (k)
Using (3.165), it follows
k∗ (k) = P(k)x∗ (k) (3.168)
Using (3.158), the recursive equation to update the estimate of c is given by
c(k) = 𝚽−1 (k)𝝑(k)
= P(k)𝝑(k) (3.169)
= 𝜆P(k)𝝑(k − 1) + P(k)x∗ (k)d(k)
174 Chapter 3. Adaptive transversal filters

Substituting the recursive expression for P(k) in the first term, we get
c(k) = 𝜆[𝜆−1 P(k − 1) − 𝜆−1 k∗ (k)xT (k)P(k − 1)]𝝑(k − 1) + P(k)x∗ (k)d(k)
= P(k − 1)𝝑(k − 1) − k∗ (k)xT (k)P(k − 1)𝝑(k − 1) + P(k)x∗ (k)d(k) (3.170)
= c(k − 1) + k∗ (k)[d(k) − xT (k)c(k − 1)]
where in the last step (3.168) has been used. Defining the a priori estimation error,
𝜖(k) = d(k) − xT (k)c(k − 1) (3.171)
we note that xT (k)c(k − 1) is the filter output at instant k obtained by using the old coefficient estimate.
In other words, from the a posteriori estimation error
e(k) = d(k) − xT (k)c(k) (3.172)
we could say that 𝜖(k) is an approximated value of e(k), that is computed before updating c. In any case,
the following relation holds
c(k) = c(k − 1) + k∗ (k)𝜖(k) (3.173)
In summary, the RLS algorithm consists of four equations:
P(k − 1)x∗ (k)
k∗ (k) = (3.174)
𝜆 + xT (k)P(k − 1)x∗ (k)

𝜖(k) = d(k) − xT (k)c(k − 1) (3.175)

c(k) = c(k − 1) + 𝜖(k)k∗ (k) (3.176)

P(k) = 𝜆−1 P(k − 1) − 𝜆−1 k∗ (k)xT (k)P(k − 1) (3.177)

In (3.174), k(k) is the input vector filtered by P(k − 1) and normalized by the 𝜆 + xT (k)P(k − 1)x∗ (k).
The term xT (k)P(k − 1)x∗ (k) may be interpreted as the energy of the filtered input.

Initialization
We need to assign a value to P(0). We modify the definition of 𝚽(k) as
∑
k
𝚽(k) = 𝜆k−i x∗ (i)xT (i) + 𝛿𝜆k I with 𝛿 ≪ 1 (3.178)
i=1

so that
𝚽(0) = 𝛿I (3.179)
This is equivalent to having for k ≤ 0 an all-zero input with the exception of x(−N + 1) = (𝜆−N+1 𝛿)1∕2 ,
therefore
P(0) = 𝛿 −1 I 𝛿 ≪ 𝚛x (0) (3.180)
Typically, we have
100
𝛿 −1 = (3.181)
𝚛x (0)
where 𝚛x (0) is the statistical power of the input signal. Table 3.1 shows a version of the RLS algorithm
that exploits the fact that P(k) (inverse of the Hermitian matrix 𝚽(k)) is Hermitian, hence
xT (k)P(k − 1) = [P(k − 1)x∗ (k)]H = 𝝅 T (k) (3.182)
3.2. The recursive least squares algorithm 175

Table 3.1: The RLS algorithm.

Initialization
c(0) = 0
P(0) = 𝛿 −1 I
For k = 1, 2, …
𝝅 ∗ (k) = P(k − 1)x∗ (k)
1
r(k) =
𝜆 + xT (k)𝝅 ∗ (k)
k∗ (k) = r(k)𝝅 ∗ (k)
𝜖(k) = d(k) − xT (k)c(k − 1)
c(k) = c(k − 1) + 𝜖(k)k∗ (k)
P(k) = 𝜆−1 (P(k − 1) − k∗ (k)𝝅 T (k))

Recursive form of the minimum cost function

We set
∑
k
d (k) = 𝜆k−i |d(i)|2 = 𝜆d (k − 1) + |d(k)|2 (3.183)
i=1

From the general LS expression (2.114),

min (k) = d (k) − 𝝑 H (k)c(k) (3.184)
and from (3.158) and (3.176) we get
min (k) = 𝜆d (k − 1) + |d(k)|2
− [𝜆𝝑 H (k − 1) + xT (k)d∗ (k)][c(k − 1) + 𝜖(k)k∗ (k)]
= 𝜆d (k − 1) − 𝜆𝝑 H (k − 1)c(k − 1) (3.185)
+ d(k)d (k) − d (k)x (k)c(k − 1) − 𝝑 (k)k (k)𝜖(k)
∗ ∗ T H ∗

= 𝜆min (k − 1) + d∗ (k)𝜖(k) − 𝝑H (k)k∗ (k)𝜖(k)

Using (3.163), and recalling that 𝚽(k) is Hermitian, from (3.168) we obtain
𝝑 H (k)k∗ (k) = 𝝑 H (k)𝚽−1 (k)x∗ (k)
(3.186)
= [𝚽−1 (k)𝝑(k)]H x∗ (k)
Moreover from (3.168) and (3.155), it follows that
𝝑 H (k)k∗ (k) = cH (k)x∗ (k) = xH (k)c∗ (k) (3.187)
Then, (3.185) becomes
min (k) = 𝜆min (k − 1) + d∗ (k)𝜖(k) − cH (k)x∗ (k)𝜖(k)
(3.188)
= 𝜆min (k − 1) + 𝜖(k)[d∗ (k) − (xT (k)c(k))∗ ]
Finally, the recursive relation is given by
min (k) = 𝜆min (k − 1) + 𝜖(k)e∗ (k) (3.189)
We note that, as min (k) is real, we get
𝜖(k)e∗ (k) = 𝜖 ∗ (k)e(k) (3.190)
that is 𝜖(k)e∗ (k) is a real scalar value.
176 Chapter 3. Adaptive transversal filters

Convergence
We make some remarks on the convergence of the RLS algorithm.
● The RLS algorithm converges in the mean-square sense in about 2N iterations, independently of
the eigenvalue spread of R.
● For k → ∞, there is no excess error and the misadjustment MSD is zero. This is true for 𝜆 = 1.
● In any case, when 𝜆 < 1 the memory of the algorithm is approximately 1∕(1 − 𝜆) and
1−𝜆
N MSD = (3.191)
1+𝜆
● From the above observation, it follows that the RLS algorithm for 𝜆 < 1 gives origin to noisy esti-
mates.
● On the other hand, the RLS algorithm for 𝜆 < 1 can be used for tracking slowly time-varying sys-
tems.

Computational complexity
Exploiting the symmetry of P(k), the computational complexity of the RLS algorithm, expressed as the
number of complex multiplications per output sample, is given by
CCRLS = 2N 2 + 4N (3.192)
For a number of (K − N + 1) output samples, the direct method (3.155) requires instead
N3
CCDIR = N 2 + N + K −N+1
(3.193)

−a1
0.2
1.2 0
1
–0.2 average
0.8
average
c1(k)

c2(k)

0.6 –0.4
0.4 –0.6
0.2 –0.8
0 −a2
–1
–0.2
0 10 20 30 40 0 10 20 30 40
k k
(a) (b)
–15
σw2 J(k)
J(k), │e(k)│2 (dB)

–20

–25
–30
–35
–40
0 5 10 15 20 25 30 35 40
k
(c)

Figure 3.19 Convergence curves for the predictor of order N = 2, obtained by the RLS algorithm.
(a) c1 (k), (b) c2 (k), (c) J(k) and |e(k)|2 , all versus k.
3.3. Fast recursive algorithms 177

We note that, if K ≫ N, the direct method is more convenient. In any case, the RLS solution has other
advantages:
1. it can be numerically more stable than the direct method;
2. it provides an estimate of the coefficients at each step and not only at the end of the data sequence;
3. for 𝜆 < 1 and 1∕(1 − 𝜆) much smaller than the time interval it takes for the input samples to change
statistics, the algorithm is capable of tracking the changes.

Example of application: the predictor

With reference to the AR(2) process considered in Section 3.1.5, convergence curves for the RLS algo-
rithm are plotted in Figure 3.19 for a single realization and for the mean (estimated over 500 realizations)
of the coefficients and of the square estimation error, for 𝜆 = 1. We note that a different scale is used
for the abscissa as compared to the LMS method; in fact the RLS algorithm converges in a number of
iterations of the order of N.

3.3 Fast recursive algorithms

As observed in the Section 3.2, the RLS algorithm has the disadvantage of requiring (2N 2 + 4N) mul-
tiplications per iteration. Therefore, we will list a few fast algorithms, whose computational complexity
increases linearly with N, the length of the coefficient vector c.
1. Algorithms for transversal filters: The fast Kalman algorithm has the same speed of convergence as
the RLS, but with a computational complexity comparable to that of the LMS algorithm. Exploit-
ing some properties of the correlation matrix 𝚽(k), Falconer and Ljung [12] have shown that the
recursive equation (3.177) requires only 10(2N + 1) multiplications. Cioffi and Kailath [13], with
their fast transversal filter (FTF), have further reduced the number of multiplications to 7(2N + 1).
The implementation of these algorithms still remains relatively simple; their weak point resides
in the sensitivity of the operations to round off errors in the various coefficients and signals. As a
consequence, the fast algorithms may become numerically unstable.
2. Algorithms for transversal filters based on systolic structures: A particular structure is the QR-
decomposition-based LSL. The name comes from the use of an orthogonal triangularization pro-
cess, usually known as QR decomposition, that leads to a systolic-type structure with the following
characteristics:
● high speed of convergence;

● numerical stability, owing to the QR decomposition and lattice structure;

● a very efficient and modular structure, which does not require the a priori knowledge of the filter

order and is suitable for implementation in very large-scale integration (VLSI) technology.
For further study on the subject, we refer the reader to [14–19].

3.3.1 Comparison of the various algorithms

In practice, the choice of an algorithm must be made bearing in mind some fundamental aspects:
● computational complexity;
● performance in terms of speed of convergence, error in steady state, and tracking capabilities under
non-stationary conditions;
● robustness, that is good performance achieved in the presence of a large eigenvalue spread and
finite-precision arithmetic [5, 20].
178 Chapter 3. Adaptive transversal filters

Table 3.2: Comparison of three adaptive algorithms in terms of

computational complexity.
additions
cost function algorithm multiplications divisions subtractions
MSE LMS 2N + 1 0 2N
RLS 2N 2 + 7N + 5 N 2 + 4N + 3 2N 2 + 6N + 4
LS
FTF 7(2N + 1) 4 6(2N + 1)

Regarding the computational complexity per output sample, a brief comparison among LMS, RLS,
and FTF is given in Table 3.2. Although the FTF method exhibits a lower computational complexity
than the RLS method, its implementation is rather laborious, therefore it is rarely used.

3.4 Examples of application

This section re-examines two examples of Section 2.5 from an adaptive (LMS) point of view.

3.4.1 Identification of a linear discrete-time system

From Figure 2.12 and definition of d(k), e(k), and x(k), the LMS adaptation equation follows,
c(k + 1) = c(k) + 𝜇e(k)x∗ (k) (3.194)
From (2.235), we see that the noise 𝑤 does not affect the solution copt , consequently the expectation
of (3.194) for k → ∞ (equal to copt ) is also not affected by 𝑤. Anyway, as seen in Section 3.1.3, the noise
influences the convergence process and the solution obtained by the adaptive LMS algorithm. The larger
the power of the noise, the smaller 𝜇 must be so that c(k) approaches E[c(k)]. In any case, J(∞) ≠ 0.
As the input x is white, the convergence behaviour of the LMS algorithm (3.194) is easily determined.
Let 𝛾 be defined as in (3.68):
𝛾 = 1 + 𝚛x (0)(𝜇 2 N𝚛x (0) − 2𝜇)
Let 𝚫c(k) = c(k) − copt ; then we get
J(k) = E[|e(k)|2 ] = Jmin + 𝚛x (0)E[||𝚫c(k)||2 ] (3.195)
where
1 − 𝛾k
E[||𝚫c(k)||2 ] = 𝛾 k E[||𝚫c(0)||2 ] + 𝜇 2 N 𝚛x (0) Jmin k≥0 (3.196)
1−𝛾
The result (3.196) is obtained by (3.59) and the following assumptions:
1. 𝚫c(k) is statistically independent of x(k);
2. emin (k) is orthogonal to x(k);
3. the approximation xT (k) x∗ (k) ≃ N 𝚛x (0) holds.
Indeed, (3.196) is an extension of (3.67). At convergence, for 𝜇 𝚛x (0) ≪ 1, it results
N
E[||𝚫c(∞)||2 ] = 𝜇 J (3.197)
2 min
and ( )
N
J(∞) = Jmin 1 + 𝜇 𝚛x (0) (3.198)
2
A faster convergence and a more accurate estimate, for fixed 𝜇, is obtained by choosing a smaller value
of N; this, however, may increase the residual estimation error (2.237).
3.4. Examples of application 179

Example 3.4.1
Consider an unknown system whose impulse response, given in Table 1.4 on page 12 as h1 , has energy
equal to 1.06. AWGN is present with statistical power 𝜎𝑤2 = 0.01. Identification via standard LMS and
RLS adaptive algorithms is obtained using as input a maximal-length pseudo-noise (PN) sequence of
length L = 31 and unit power, 𝙼x = 1. For a filter with N = 5 coefficients, the convergence curves of
the MSE (estimated over 500 realizations) are shown in Figure 3.20. For the LMS algorithm, 𝜇 = 0.1 is
chosen, which leads to a misadjustment equal to MSD = 0.26.

0
LMS
–5
J(k) (dB)

–10
–15
σw2
–20
0 5 10 15 20 25 30
k
(a)
–15
σw2
–20
J(k) (dB)

–25
RLS
–30
–35
–40
0 5 10 15 20 25 30
k
(b)

Figure 3.20 Convergence curves of the MSE for system identification using (a) LMS and (b) RLS.

As discussed in Section 7.8.3, as metric of the estimate quality we adopt the ratio:
𝜎𝑤2
Λn = (3.199)
E[‖𝚫h‖2 ]
where 𝚫h = c − h is the estimate error vector. At convergence, that is for k = 30 in our example, it
results: {
3.9 for LMS
Λn = (3.200)
7.8 for RLS
We note that, even if the input signal is white, the RLS algorithm usually yields a better estimate than
LMS. However, for systems with a large noise power and/or slow time-varying impulse responses,
the two methods tend to give the same performance in terms of speed of convergence and error in
steady state. As a result, it is usually preferable to adopt the LMS algorithm, as it leads to an easier
implementation.

Finite alphabet case

Assume a more general, non-linear relation between the unknown system output z and input x, given by
z(k) = g[x(k), x(k − 1), x(k − 2)] = g(x(k)) (3.201)
180 Chapter 3. Adaptive transversal filters

w(k) noise

x(k) unknown z(k) d(k)

system
+
e(k)

–
x(k) loc.
1
Tc
2
address

x(k – 1) y(k) = g(x(k))

g(x(k))
Tc

x(k – 2) M3

Figure 3.21 Adaptive scheme to estimate the input–output relation of a system.

where x(i) ∈ , finite alphabet with M elements. Then, z assumes values in an alphabet with at most
M 3 values, which can be identified by a table or random-access memory (RAM) method, as illustrated
in Figure 3.21. The cost function to be minimized is expressed as

E[|e(k)|2 ] = E[|d(k) − ĝ (x(k))|2 ] (3.202)

and the gradient estimate is given by

∇ĝ |e(k)|2 = −2e(k) (3.203)

Therefore, the LMS adaptation equation becomes

ĝ (x(k)) = ĝ (x(k)) + 𝜇e(k) (3.204)

In other words, the input vector x identifies a particular RAM location whose content is updated
by adding a term proportional to the error. In the absence of noise, if the RAM is initialized to zero,
the content of a memory location can be immediately identified by looking at the output. In practice,
however, it is necessary to access each memory location several times to average out the noise. We note
that, if the sequence {x(k)} is i.i.d., x selects in the average each RAM location the same number of
times.
An alternative method consists in setting y(k) = 0 during the entire time interval devoted to system
identification, and to update the RAM as

ĝ (x(k)) = ĝ (x(k)) + d(k) k = 0, 1, … (3.205)

To complete the identification process, the value at each RAM location is scaled by the number of
updates that have taken place for that location. This is equivalent to considering

ĝ (x) = E[g(x) + 𝑤] (3.206)

We note that this method is a block version of the LMS algorithm with block length equal to the
input sequence, where the RAM is initialized to zero, so that e(k) = d(k), and 𝜇 is given by the relative
frequency of each address.
Bibliography 181

2
│(1 – μB2) ( f )│

0.7
2μB2

0 f0Tc 0.5
normalized frequency f Tc

Figure 3.22 Frequency response of a notch filter.

3.4.2 Cancellation of a sinusoidal interferer with known frequency

With reference to Figure 3.23, the adaptation equations of the LMS algorithm are
c1 (k + 1) = c1 (k) + 𝜇e(k)x1 (k) (3.207)

c2 (k + 1) = c2 (k) + 𝜇e(k)x2 (k) (3.208)

d(k) + e(k)
primary output
input –

x1(k)
+
Bcos (2πf0kTc + φ)
c1(k)
reference x2(k) y(k)
–π/2 +
c2(k)
Hilbert
filter
adaptive e(k)
algorithm

Figure 3.23 Configuration to cancel a sinusoidal interferer of known frequency.

The relation between d and output e corresponds to a notch filter as illustrated in Figure 3.22.

Bibliography

[1] Treichler, J.R., Johnson, C.R. Jr., and Larimore, M.G. (1987). Theory and Design of Adaptive
Filters. New York, NY: Wiley.
[2] Shynk, J.J. (1989). Adaptive IIR filtering. IEEE ASSP Magazine 6: 4–21.
[3] Ungerboeck, G. (1972). Theory on the speed of convergence in adaptive equalizers for digital com-
munication. IBM Journal of Research and Development 16: 546–555.
182 Chapter 3. Adaptive transversal filters

[4] Golub, G.H. and van Loan, C.F. (1989). Matrix Computations, 2e. Baltimore, MD and London: The
Johns Hopkins University Press.
[5] Ardalan, S.H. and Alexander, S.T. (1987). Fixed-point round-off error analysis of the exponentially
windowed RLS algorithm for time varying systems. IEEE Transactions on Acoustics, Speech, and
Signal Processing 35: 770–783.
[6] Eweda, E. (1994). Comparison of RLS, LMS and sign algorithms for tracking randomly time vary-
ing channels. IEEE Transactions on Signal Processing 42: 2937–2944.
[7] Gardner, W.A. (1987). Nonstationary learning characteristics of the LMS algorithm. IEEE Transac-
tions on Circuits and Systems 34: 1199–1207.
[8] Solo, V. (1989). The limiting behavior of LMS. IEEE Transactions on Acoustics, Speech, and
Signal Processing 37: 1909–1922.
[9] Douglas, S.C. (1994). A family of normalized LMS algorithms. IEEE Signal Processing Letters 1:
49–51.
[10] Zeidler, J.R. (1990). Performance analysis of LMS adaptive prediction filters. IEEE Proceedings 78:
1781–1806.
[11] Haykin, S. (1996). Adaptive Filter Theory, 3e. Englewood Cliffs, NJ: Prentice-Hall.
[12] Falconer, D. and Ljung, L. (1978). Application of fast Kalman estimation to adaptive equalization.
IEEE Transactions on Communications 26: 1439–1446.
[13] Cioffi, J.M. and Kailath, T. (1984). Fast, recursive-least-squares transversal filter for adaptive filter-
ing. IEEE Transactions on Acoustics, Speech, and Signal Processing 32: 304–337.
[14] Cioffi, J.M. (1988). High speed systolic implementation of fast QR adaptive filters. Proceedings of
ICASSP, pp. 1584–1588.
[15] Ling, F., Manolakis, D., and Proakis, J.G. (1986). Numerically robust least-squares lattice-ladder
algorithm with direct updating of the reflection coefficients. IEEE Transactions on Acoustics,
Speech, and Signal Processing 34: 837–845.
[16] Regalia, P.A. (1993). Numerical stability properties of a QR-based fast least squares algorithm.
IEEE Transactions on Signal Processing 41: 2096–2109.
[17] Alexander, S.T. and Ghirnikar, A.L. (1993). A method for recursive least-squares filtering based
upon an inverse QR decomposition. IEEE Transactions on Signal Processing 41: 20–30.
[18] Cioffi, J.M. (1990). The fast adaptive rotor’s RLS algorithm. IEEE Transactions on Acoustics,
Speech, and Signal Processing 38: 631–653.
[19] Liu, Z.-S. (1995). QR methods of O(N) complexity in adaptive parameter estimation. IEEE Trans-
actions on Signal Processing 43: 720–729.
[20] Bucklew, J.A., Kurtz, T.G., and Sethares, W.A. (1993). Weak convergence and local stability
properties of fixed step size recursive algorithms. IEEE Transactions on Information Theory 39:
966–978.
Chapter 4

Transmission channels

This chapter introduces the radio channel with its statistical description and associated numerous
parameters, including the use of multiple antennas at the transmitter and/or receiver. Briefly, we also
mention the telephone channel, which however is gradually replaced by optical fibres.

4.1 Radio channel

The term radio is used to indicate the transmission of an electromagnetic field that propagates in free
space. Some examples of radio transmission systems are as follows:
● point-to-point terrestrial links [1];
● mobile terrestrial communication systems [2–6];
● earth–satellite links (with satellites employed as signal repeaters) [7];
● deep-space communication systems (with space probes at a large distance from earth).
A radio channel model is illustrated in Figure 4.1, where we assume that the transmit antenna input
impedance and the receive antenna output impedance are matched for maximum transfer of power.

Tx antenna Rc antenna

transmitter receiver

model: 2-port network

Figure 4.1 Radio channel model.

4.1.1 Propagation and used frequencies in radio transmission

Propagation over the various layers of the atmosphere is denoted respectively as ionospheric and tropo-
spheric propagation, while propagation occurring very close to the earth’s surface is denoted as ground
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
184 Chapter 4. Transmission channels

propagation. The importance of each of propagation methods depends on the length of the path and the
carrier frequency [8].

Basic propagation mechanisms

The mechanisms behind electromagnetic wave propagation are diverse, but can generally be attributed
to reflection, diffraction, and scattering [3].

Reflection. This phenomenon occurs when an electromagnetic wave impinges on an interface between
two media having different properties and the incident wave is partially reflected and partially transmit-
ted. In general, the dimensions of the object where the wave impinges must be large in comparison to
the wavelength for reflection to occur. In radio propagation, reflections typically occur from the surface
of the earth and from buildings and walls.

Diffraction. It occurs when the path between transmitter and receiver is obstructed by a surface that
has sharp irregularities. The waves originating from the obstructing surface are present throughout the
space and behind the obstacle, even when a line of sight (LOS) does not exist between transmitter and
receiver.

Scattering. It occurs when the medium through which the wave travels consists of objects whose
dimensions are small compared to the wavelength. When a wave is scattered, it is spread out in many
different directions. In mobile communication systems, scattering is due for example to foliage.

Frequency ranges
Frequencies used for radio transmission are in the range from about 100 kHz to hundreds of GHz. To
a frequency f0 corresponds a wavelength 𝜆 = c∕f0 , where c is the speed of light in free space (that is,
in a vacuum). The choice of the carrier frequency depends on various factors, among which the size
of the transmit antenna plays an important role. Indeed, for a dipole antenna both the efficiency and its
resistance grows with the dipole length up to the wavelength 𝜆, and typically a length of 𝜆∕2 is chosen,
as it provides both high efficiency and a good resistance (to be matched by the amplifier). For example,
smartphones operating at a carrier frequency f0 = 900 MHz and wavelength 𝜆 = 0.3 m, usually have
dipole antennas of 0.15 m.
Recall that, if the atmosphere is not homogeneous (in terms of temperature, pressure, humidity, …),
the radio propagation depends on the changes of its refraction index: this gives origin to reflections. We
speak of diffusion or scattering phenomena if molecules that are present in the atmosphere absorb part
of the power of the incident wave and then re-emit it in all directions. Obstacles such as mountains,
buildings, etc., give also origin to signal reflection and/or scattering. In any case, these phenomena
permit transmission between two points that are not in LOS.
We will now consider the types of propagation associated with major frequency bands, where f0 is the
signal carrier frequency.

Low frequency (LF), 0.03 < f0 < 0.3 MHz: The earth and the ionosphere form a waveguide for the
electromagnetic waves. At these frequencies, the signals propagate around the earth.

Medium frequency (MF), 0.3 < f0 < 3 MHz: The signal propagates as a ground wave up to a dis-
tance of 160 km. Low and medium frequency bands are utilized for long-distance communications in
navigation, aircraft beacons, and weather monitoring systems.
4.1. Radio channel 185

High frequency (HF), 3 < f0 < 30 MHz: The signal is reflected by the ionosphere at an alti-
tude that may vary between 50 and 400 km. However, propagation depends on the time of the
day/sunlight/darkness, season of the year, and geographical location. They are used for shortwave radio
broadcasts.

Very high frequency (VHF), 30 < f0 < 300 MHz: For f0 > 30 MHz, the signal propagates through
the ionosphere with small attenuation. Therefore, these frequencies are adopted for frequency modula-
tion (FM) radio transmissions. They are also employed for LOS transmissions, using high towers where
the antennas are positioned to cover a wide area. The limit to the coverage is set by the√earth curvature.
If h is the height of the tower in meters, the range covered expressed in km is d = 4.12 h: for example,
if h = 100 m, coverage is up to about d = 41.2 km. However, ionospheric and tropospheric scattering
(at an altitude of 16 km or less) are present at frequencies in the range 30–60 MHz and 40–300 MHz,
respectively, which cause the signal to propagate over long distances, however with a large attenuation.

Ultra high frequency (UHF), 300 MHz < f0 < 3 GHz: These frequencies are used for video
broadcasting and unlicensed transmissions in the industrial, scientific, and medical (ISM) band with
2.4 < f0 < 2.483 GHz, such as cordless phones, Bluetooth, wireless local area network (LAN) or WiFi,
door openers, key-less vehicle locks, baby monitors, and many other applications.

Super high frequency (SHF), 3 < f0 < 30 GHz: At frequencies of about 10 GHz, atmospheric condi-
tions play an important role in signal propagation. We note the following absorption phenomena, which
cause additional signal attenuation:

1. Due to water vapour: For f0 > 20 GHz, with peak attenuation at around 20 GHz;
2. Due to rain: For f0 > 10 GHz, assuming the diameter of the rain drops is of the order of the signal
wavelength.

We note that, if the antennas are not positioned high enough above the ground, the electromagnetic
field propagates not only into the free space but also through ground waves. These frequencies are used
for satellite communications, point-to-point terrestrial links, radars, and short-range communications.

Extremely high frequency (EHF), 30 < f0 < 300 GHz: These frequencies are used for radio
astronomy, wireless LAN, and millimetre waves (mmWave) cellular transmissions. They are affected
by absorption due to oxygen for f0 > 30 GHz, with peak attenuation at 60 GHz.

4.1.2 Analog front-end architectures

In this section, an overview of a general front-end architecture for radio receivers is given.

Radiation masks
A radio channel by itself does not set constraints on the frequency band that can be used for transmission.
In any case, to prevent interference among radio transmissions, regulatory bodies specify power radiation
masks: a typical example is given in Figure 4.2, where the plot represents the limit on the power spectrum
of the transmitted signal with reference to the power of a unmodulated carrier for the global system for
mobile communications (GSM). The shown frequency offset is with respect to the carrier frequency. To
comply with these limits, a filter is usually employed at the transmitter front-end.
186 Chapter 4. Transmission channels

20
10
0
–10
Amplitude (dB) –20
–30
–40
–50
–60
–70
–80
0 100 200 300 400 500 600 700 800 900 1000
Carrier frequency offset (kHz)

Figure 4.2 Radiation mask of the GSM system with a bandwidth of 200 kHz around the carrier
frequency.

Conventional superheterodyne receiver

Many receivers for radio frequency (RF) communications use the conventional superheterodyne scheme
illustrated in Figure 4.3 [9]. A first mixer employs a local oscillator (LO), whose frequency f1 is vari-
able, to shift the signal to around a fixed intermediate frequency (IF) fIF . The output signal of the first
mixer is filtered by a passband filter centred around the frequency fIF to eliminate out of band com-
ponents. The cascade of RF filter, linear amplifier (LNA), and image rejection (IR) filter performs the
tasks of amplifying the desired signal as well as eliminating the noise outside of the desired band and
rejecting the spectral image components introduced by the LO. The IF filter output signal is shifted into
baseband through a second mixer, which makes use of an oscillator with a fixed frequency f2 , whose
output is filtered by a lowpass filter to eliminate high frequency components. The signal is then sent to
the analog-to-digital converter (ADC).
A drawback of the superheterodyne receiver is the high selectivity in frequency (high Q-factor) that
the various elements must exhibit, which makes its integrated implementation at high frequency prob-
lematic.
Issues related to a full integration can be divided into two categories.

1. Integration of the acquisition structure and signal processing requires the elimination of both IR
and IF filters.
2. For channel selection, a synthesizer can be integrated using an on-chip voltage-controlled oscillator
(VCO) with low Q-factor, which, however, yields poor performance in terms of phase noise (see
Section 4.1.3).

LNA
RF IR IF LPF ADC
filter filter

A1 cos(2πf1 t+φ1) A2 cos(2πf2 t+φ2)

Figure 4.3 Conventional superheterodyne receiver.

4.1. Radio channel 187

Alternative architectures
In this section, three receiver architectures are considered that attempt to integrate the largest number of
receiver elements. To reach this objective, the analog-to-digital conversion of the scheme of Figure 4.3
must be shifted from baseband to IF or RF. In this case, implementation of the channel selection by an
analog filter bank is not efficient; indeed, it is more convenient to use a wideband analog filter followed by
a digital filter bank. Reference is also made to multi-standard software-defined radio (SDR) receivers,
or to the possibility of receiving signals that have different bandwidths and carriers, that are defined
according to different standard systems.
We can identify two approaches to the analog-to-digital conversion.
1. Full bandwidth digital conversion: By using a very high sampling frequency, the whole bandwidth
of the SDR system is available in the digital domain, i.e. all desired channels are considered. Con-
sidering that this bandwidth can easily achieve 100 MHz, and taking into account the characteristics
of interference signals, the dynamic of the ADC should exceed 100 dB. This solution, even though
is the most elegant, cannot be easily implemented as it leads to high complexity and high power
consumption.
2. Partial bandwidth digital conversion: In this approach, the sampling frequency is determined by
the radio channel with the most extended bandwidth within the different systems we want to imple-
ment.
The second approach will be considered, as its implementation has a moderate complexity, and it is
used in SDR systems.

Direct conversion receiver

This architecture eliminates many off-chip components. As illustrated in Figure 4.4, all desired chan-
nels are shifted to baseband through a mixer using an oscillator with a varying frequency. Undesired
spectral components are removed by an on-chip baseband filter. Clearly, this structure provides a higher
integration level with respect to the superheterodyne scheme with two important advantages:
1. the problem of image components is bypassed and the IR filter is not needed;
2. the IF filter and following operations are replaced by a lowpass filter and a baseband amplifier,
which are suitable for monolithic integration.
This structure has still numerous issues; in fact, the oscillator operates at the RF carrier and it may
leak signal towards both the mixer input as well as towards the antenna, with consequent radiation.
The interference signal generated by the oscillator could be reflected on surrounding objects and
re-received: this spurious signal would produce a time variant direct current (DC) offset [9] at the mixer
output.

LNA
RF
LPF ADC
filter

Figure 4.4 Direct conversion receiver.

188 Chapter 4. Transmission channels

To understand the origin and consequences of this offset, we can make the following two observations.
1. The isolation between the oscillator input and the inputs of the mixer and linear amplifier is not
ideal; in fact an LO leakage is determined by both capacitive coupling and device substrate. The
spurious signal appears at the linear amplifier and mixer inputs and is then mixed with the signal
generated by the oscillator, creating a DC component; this phenomenon is called self-mixing. A
similar effect is present when there is a strong signal coming from the linear amplifier or from the
mixer that couples with the LO input: this signal would then be multiplied with itself.
2. In order to amplify the input signal of Figure 4.4 of the order of microvolts, to a level such that it can
be digitized by a low cost and low power ADC, the total gain, from the antenna to the LPF output,
is about 100 dB. 30 dB of this gain are usually provided by the linear amplifier/mixer combination.
With these data, we can make a first computation of the offset due to self-mixing. We assume that
the oscillator generates a signal with a peak-to-peak value of 0.63 V and undergoes an attenuation
of 60 dB when it couples with the LNA input. If the gain of the linear amplifier/mixer combination
is 30 dB, then the offset produced at the mixer output is of the order of 10 mV; if directly amplified
with a gain of 70 dB, the voltage offset would saturate the following circuits.
The problem is even worse if self-mixing is time variant. This event, as previously mentioned, occurs
if the oscillator signal leaks to the antenna, thus being irradiated and reflected back to the receiver from
moving objects.
Finally, the direct conversion receiver needs a tuner for the channel frequency selection that works at
high frequency and with low phase noise; this is hardly obtainable with an integrated VCO with low Q.

Single conversion to low-IF

The low-IF architecture reduces the problem of DC offset of the direct conversion receiver. This system
has the structure of Figure 4.4 and, similarly to direct conversion, utilizes a single mixer stage; the main
difference is that the frequency shift is not made to DC but to a small IF. The main advantage of the
low-IF system is that the desired channel has no DC components; therefore, the usual problems due to
the DC offset in the direct conversion are avoided.

Double conversion and wideband IF

This architecture, illustrated in Figure 4.5, takes all desired channels and shifts them from RF to IF using
the first mixer by a fixed frequency oscillator; a simple lowpass filter is used to eliminate high frequency
image components. All channels are then shifted down to baseband by a second mixer, this time with a
variable frequency oscillator. The following baseband filter has a variable gain and eliminates spectral
components outside the desired signal band. Channel tuning is obtained by the second low frequency
LO; in this case, the first RF oscillator can be implemented by a quartz oscillator with a fixed frequency,

LNA
RF
LPF LPF ADC
filter

Figure 4.5 Double conversion with wideband IF.

4.1. Radio channel 189

w(bb)(t)
high power transmission additive noise and
amplifier medium interference

s(bb)(t) s(bb)
Tx
(t) 1 g(bb)
(bb)(t)
sCh r(bb)(t)
HPA
2 Ch

ejφj(t) phase noise

Figure 4.6 Baseband equivalent model of a transmission channel including a non-linear device.

while the second can be implemented by on-chip techniques that provide low-frequency oscillators with
low phase noise and low Q.
It is important to emphasize the absence of oscillators that operate at the same frequency of the RF
carrier; this eliminates the potential problem of re-radiation of the oscillator. We can conclude that the
double conversion architecture is the most attractive for the analog front-end.

4.1.3 General channel model

In this section, we will describe a transmission channel model that takes into account the non-linear
effects due to the transmitter and the disturbance introduced by the receiver and by the radio channel.
We will now analyse the various blocks of the baseband equivalent model illustrated in Figure 4.6 (see
Section 1.4).

High power amplifier

The final transmitter stage in a communication system usually consists of a high power amplifier (HPA).
The HPA is a non-linear device with saturation i.e. in addition to not amplifying the input signal above
a certain value, it introduces non-linear distortion of the signal itself. The non-linearity of a HPA can be
described by a memoryless envelope model. Let the input signal of the HPA be
s(t) = A(t) cos[2𝜋f0 t + 𝜑(t)] (4.1)
where A(t) ≥ 0 is the signal envelope and 𝜑 is the instantaneous phase deviation.
The envelope and the phase of the output signal, sTx , depend on the instantaneous, i.e. without memory,
transformation of the input, i.e.
sTx (t) = G[A(t)] cos(2𝜋f0 t + 𝜑(t) + Φ[A(t)]) (4.2)
It is usually more convenient to refer to baseband equivalent signals
s(bb) (t) = A(t)ej𝜑(t) (4.3)
and
s(bb)
Tx
(t) = G[A(t)]ej(𝜑(t)+Φ[A(t)]) (4.4)
The functions G[A] and Φ[A], called envelope transfer functions, represent respectively the ampli-
tude/amplitude (AM/AM) conversion and the amplitude/phase (AM/PM) conversion of the amplifier.
In practice, the HPAs are of two types. For each type, we give the AM/AM and AM/PM functions
commonly adopted for the analysis. First, however, we need to introduce some normalizations. As a
190 Chapter 4. Transmission channels

rule, the point at which the amplifier operates is identified by the back-off . We adopt here the following
definitions for the input back-off (IBO) and the output back-off (OBO):
( )
S
IBO = 20 log10 √ (dB) (4.5)
𝙼s
⎛ ⎞
⎜ STx ⎟
OBO = 20 log10 ⎜ √ ⎟ (dB) (4.6)
⎜ 𝙼sTx ⎟
⎝ ⎠
where 𝙼s is the statistical power of the input signal s, 𝙼sTx is the statistical power of the output signal sTx ,
and S and STx are the amplitudes of the input and output signals, respectively, that lead to saturation of
the amplifier. Here, we assume S = 1 and STx = G[1].
A posteriori, these definitions measure the trade-off between transmitting at the highest possible
power, for a longer coverage of the system, and avoiding distortion (due to clamping) of the trans-
mit signal. Hence, we neglect distortion due to the fact that in practice G[A(t)] is not a straight line for
0 ≤ A ≤ 1, and consider only the problem of saturation of G[A(t)] for A(t) > 1.
Indeed, the above trade-off depends on the amplitude distribution of s. If for example s has a constant
envelope, say A(t) = A, we can transmit sTx at the highest power without clamping of the signal and
OBO = 0 dB. If, however, s has a Gaussian amplitude distribution, we have to transmit sTx at a much
lower average power than the maximum value in order to control distortion due to clamping and OBO
could be even 3 or 4 dB. In general, we want to select a working point of the HPA average output power
level, where OBO is minimized and at the same time keep distortion under control. However, this
needs a measure of the effect of distortion of the system, e.g. the level of out-of-band leakage of power
spectral density (PSD) of sTx .

Travelling wave tube The travelling wave tube (TWT) is a device characterized by a strong AM/PM
conversion. The conversion functions are
𝛼A A
G[A] = (4.7)
1 + 𝛽A A2
𝛼Φ A2
Φ[A] = (4.8)
1 + 𝛽Φ A2
where 𝛼A , 𝛽A , 𝛼Φ , and 𝛽Φ are suitable parameters.
Functions (4.7) and (4.8) are illustrated in Figure 4.7, for 𝛼A = 1, 𝛽A = 0.25, 𝛼Φ = 0.26, and 𝛽Φ = 0.25.

Solid state power amplifier A solid state power amplifier (SSPA) has a more linear behaviour in
the region of small signals as compared to the TWT. The AM/PM conversion is usually negligible.
Therefore, the conversion functions are
A
G[A] = (4.9)
(1 + A2p )1∕2p
Φ[A] = 0 (4.10)
where p is a suitable parameter.
In Figure 4.8, the function G[A] is plotted for three values of p; the superimposed dashed line is an
ideal curve given by {
A, 0<A<1
G[A] = (4.11)
1, A≥1
It is interesting to compare the above analytical models with the behaviour of a practical HPA. Figure 4.9
illustrates the AM/AM characteristics of two waveguide HPA operating at frequencies of 38 and 40 GHz.
4.1. Radio channel 191

0
G[A] (dB)
−5

−10

−15
−14 −12 −10 −8 −6 −4 −2 0 2
A (dB)
(a)
25

20

15
Φ[A] (°)

10

0
−14 −12 −10 −8 −6 −4 −2 0 2
A (dB)
(b)

Figure 4.7 AM/AM and AM/PM characteristics of a TWT for 𝛼A = 1, 𝛽A = 0.25, 𝛼Φ = 0.26, and
𝛽Φ = 0.25.

Transmission medium
The transmission medium is typically modelled as a filter, as detailed in Section 4.1.

Additive noise
Several noise sources that cause a degradation of the received signal may be present in a transmission
system. Consider for example the noise introduced by a receive antenna or the thermal noise and shot
noise generated by the pre-amplifier stage of a receiver. At the receiver input, all these noise signals are
modelled as an effective additive white Gaussian noise (AWGN), statistically independent of the desired
signal. The power spectral density of the AWGN noise can be obtained by the analysis of the system
devices, or by experimental measurements.

Phase noise
The demodulators used at the receivers are classified as coherent or non-coherent, depending on whether
they use or not a carrier signal, which ideally should have the same phase and frequency as the carrier
192 Chapter 4. Transmission channels

0
p=3
p=2
p=1
G[A] (dB)

−5

−10

−15
−14 −12 −10 −8 −6 −4 −2 0 2
A (dB)

Figure 4.8 AM/AM characteristic of an SSPA.

5
HPA 38 GHz
HPA 40 GHz

0
G[A] (dB)

−5

−10

−15
−14 −12 −10 −8 −6 −4 −2 0 2
A (dB)

Figure 4.9 AM/AM experimental characteristic of two amplifiers operating at 38 and 40 GHz.

at the transmitter, to demodulate the received signal. Typically, both phase and frequency are recovered
from the received signal by a phase locked loop (PLL) system, which employs a local oscillator. The
recovered carrier may differ from the transmitted carrier frequency f0 because of the phase noise,1 due

1
Sometimes also called phase jitter.
4.1. Radio channel 193

to short-term stability, i.e. frequency drift, of the oscillator, and because of the dynamics and transient
behaviour of the PLL.
The recovered carrier is
( )
𝓁D t 2
𝑣(t) = Vo [1 + a(t)] cos 2𝜋f0 t + 𝜑j (t) + (4.12)
2
where 𝓁D (long term drift) represents the effect of the oscillator ageing, a is the amplitude noise, and 𝜑j
denotes the phase noise. Often both the amplitude noise and the ageing effect can be neglected.
The phase noise 𝜑j , represented in Figure 4.6, consists of deterministic components and random noise.
For example, among deterministic components, we have effects of temperature, supply voltage, and
output impedance of the oscillator.
Ignoring the deterministic effects, with the exception of the frequency drift, the PSD of 𝜑j comprises
five terms:
f02 f02 f02 f02
𝜑j (f ) = k−4 + k−3 + k−2 + k−1 + k0 f02 (4.13)
f4 f3 f2 f ⏟⏟⏟
⏟⏟⏟ ⏟⏟⏟ ⏟⏟⏟ ⏟⏟⏟
white
random flicker random phase flicker
phase noise
frequency frequency walk or white phase noise
walk noise frequency noise

for f𝓁 ≤ f ≤ fh .
A simplified model, often used, is given by
⎧
⎪ a, |f | ≤ f1
𝜑j (f ) = c + ⎨ 1 (4.14)
b , f1 ≤ |f | < f2
⎪ f2
⎩
where parameters a and c are typically of the order of −65 and −125 dBc/Hz, respectively, and b is a
scaling factor that depends on f1 and f2 and ensures continuity of the PSD. dBc means dB carrier, i.e. the
statistical power of the phase noise, expressed in dB with respect to the statistical power of the desired
signal received in the passband.
Typical values of the statistical power of 𝜑j are in the range from 10−2 to 10−4 . Figure 4.10 shows the
PSD (4.14) for f1 = 0.1 MHz, f2 = 2 MHz, a = −65 dBc/Hz, and c = −125 dBc/Hz.

4.1.4 Narrowband radio channel model

The propagation of electromagnetic waves should be studied using the Maxwell equations with appro-
priate boundary conditions. Nevertheless, for our purposes a very simple model, approximating the
electromagnetic wave as a ray (in the optical sense), is often adequate.
The deterministic model is used to evaluate the power of the received signal in the absence of obsta-
cles between the transmitter and receiver, that is in LOS: in this case we can think of only one wave
that propagates from the transmitter to the receiver. This situation is typical of transmissions between
satellites and terrestrial radio stations in the SHF band.
Let 𝙿Tx be the power of the signal transmitted by an ideal isotropic antenna, which uniformly radiates
in all directions in the free space. At a distance d from the antenna, the power density (per area) is
𝙿Tx
Φ0 = (4.15)
4𝜋d2
194 Chapter 4. Transmission channels

−60

−70

−80
Pϕ(f ) (dBc/Hz)

−90

−100

−110

(≃ −20 dB/decade)
−120

−130
0 5 10 15
f (MHz)

Figure 4.10 Simplified model of the phase-noise power spectral density.

where 4𝜋d2 is the surface of a sphere of radius d that is uniformly illuminated by the antenna. We
observe that the power density decreases with the square of the distance. On a logarithmic scale (dB),
this is equivalent to a decrease of 20 dB-per-decade with the distance.
In the case of a directional antenna, the power density is concentrated within a cone and is given by
𝙶Tx 𝙿Tx
Φ = 𝙶Tx Φ0 = (4.16)
4𝜋d2
where 𝙶Tx is the transmit antenna gain. Obviously, 𝙶Tx = 1 for an isotropic antenna; usually, 𝙶Tx ≫ 1 for
a directional antenna.
At the receive antenna, the available power in conditions of matched impedance is given by

𝙿Rc = Φ𝙰Rc 𝜂Rc (4.17)

where 𝙿Rc is the received power, 𝙰Rc is the effective area of the receive antenna, and 𝜂Rc is the efficiency
of the receive antenna. The factor 𝜂Rc < 1 takes into account the fact that the antenna does not capture all
the incident radiation, because a part is reflected or lost. To conclude, the power of the received signal
is given by
𝙰
𝙿Rc = 𝙿Tx Rc2 𝙶Tx 𝜂Rc (4.18)
4𝜋d
The antenna gain can be expressed as [10]
4𝜋𝙰
𝜂 𝙶= (4.19)
𝜆2
where 𝙰 is the effective area of the antenna, 𝜆 = c∕f0 is the wavelength of the transmitted signal, f0 is
the carrier frequency, and 𝜂 is the efficiency factor. Equation (4.19) holds for both transmit and receive
antennas.
We note that, because of the factor 𝙰∕𝜆2 , at higher frequencies we can use smaller antennas, for a
given 𝙶. Usually, 𝜂 ∈ [0.5, 0.6] for parabolic antennas, while 𝜂 ≃ 0.8 for horn antennas.
4.1. Radio channel 195

Observing (4.19), we get

( )2
𝜆
𝙿Rc = 𝙿Tx 𝙶Tx 𝙶Rc (4.20)
4𝜋d
known as the Friis transmission equation and valid in conditions of maximum transfer of power. The
term (𝜆∕4𝜋d)2 is called free-space path loss. Later, we will use the following definition:
( )2
𝜆
𝙿0 = 𝙿Tx 𝙶Tx 𝙶Rc (4.21)
4𝜋
which represents the power of a signal received at the distance d = 1 from the transmitter. In any case,
(4.20) does not take into account the attenuation due to rain or other environmental factors, nor the
possibility of wrong positioning of the antennas.
The (available) attenuation of the medium, expressed in dB, is
𝙿Tx
(𝚊)dB = 10 log10 = 32.4 + 20 log10 d|km + 20 log10 f0 |MHz − (𝙶Tx )dB − (𝙶Rc )dB (4.22)
𝙿Rc
where 32.4 = 10 log10 (4𝜋∕c)2 , d is expressed in km, f0 in MHz, and 𝙶Tx and 𝙶Rc in dB.
It is worthwhile to make the following observations on the attenuation as from (4.22): (i) it increases
with distance as log10 d (whereas for metallic transmission lines the dependency is linear); (ii) it increases
with frequency as log10 f0 .
For 𝙶Tx = 𝙶Rc = 1, (𝚊)dB is the free space path loss due to distance, i.e.

(𝚊PL,fs )dB = 32.4 + 20 log10 d|km + 20 log10 f0 |MHz (4.23)

In many cases, this is also the attenuation in the presence of LOS. In practice, there is a minimum
value of the path loss due to distance, defined as the minimum coupling loss (MCL). For example,
(𝚊MCL )dB = 70 dB (80 dB) for a macro cell urban (rural) area [11]. In general, if we denote by (𝚊PL )dB
the path loss of the effective link (in dB), whose expression will be given in (4.53), the global attenuation
is
(𝚊)dB = max{(𝚊PL )dB − (𝙶Tx )dB − (𝙶Rc )dB , (𝚊MCL )dB } (4.24)

and
(𝙿Rc )dBm = (𝙿Tx )dBm − (𝚊)dB (4.25)

where, if (𝙿)mW is the power in mW, (𝙿)dBm = 10 log10 (𝙿)mW .

Equivalent circuit at the receiver

Figure 4.11 shows the electrical equivalent circuit at the receiver. The antenna produces the desired
signal s, and 𝑤 represents the total noise due to both the antenna and the amplifier. The amplifier has
a nominal bandwidth B around the carrier frequency f0 . The spectral density of the open circuit noise
voltage is 𝑤 (f ) = 2𝚔 𝚃𝑤 Ri , and the available noise power per unit of frequency is 𝚙𝑤 (f ) = (𝚔∕2)𝚃𝑤 ,
where 𝚔 = 1.3806 10−23 J/K is the Boltzmann constant. The effective noise temperature at the input
is 𝚃𝑤 = 𝚃S + (𝙵 − 1)𝚃0 , where 𝚃S is the effective noise temperature of the antenna, and 𝚃A = (𝙵 − 1)𝚃0
is the noise temperature of the amplifier; 𝚃0 is the room temperature and 𝙵 is the noise figure of the
amplifier.
For matched input and output circuits, the signal-to-noise ratio at the amplifier output is
𝙿Rc available power of received desired signal
Γ= = (4.26)
𝚔𝚃𝑤 B nominal available power of effective input noise
196 Chapter 4. Transmission channels

amplifier
Rc antenna

RL
Ri
so(t)
s(t) Ri RL

wo(t)
w(t)

PRc

Figure 4.11 Electrical equivalent circuit at the receiver.

Multipath
It is useful to study the propagation of a sinusoidal signal over the one-ray channel model, which implies
using a directional antenna. Let the narrowband (NB) transmitted signal be

sTx (t) = Re [ATx ej2𝜋f0 t ] (4.27)

The received signal at a distance d from the transmitter is

sRc (t) = Re[ARc ej2𝜋f0 (t−𝜏0 ) ] = Re[ARc ej𝜑Rc ej2𝜋f0 t ] (4.28)

where 𝜏0 = d∕c denotes the propagation delay, ARc is the amplitude of the received signal, and
d
𝜑Rc = −2𝜋f0 𝜏0 = −2𝜋 (4.29)
𝜆
is the phase of the received signal.
Using the definition (1.84) of h(a) , the radio channel associated with (4.28) has impulse response
[ ]
A
gCh,o𝑣erall (𝜏) = Re Rc h(a) (𝜏 − 𝜏0 ) ANY INPUT (4.30)
ATx
that is the channel attenuates the signal and introduces a delay equal to 𝜏0 . Indeed, in the above ideal case,
A
it is simply gCh,o𝑣erall (𝜏) = ARc 𝛿(𝜏 − 𝜏0 ). However, (4.30) yields a more general mathematical model,
Tx
better analysed in the frequency domain, where h(a) (h(a)∗ ) accounts for the channel behaviour for positive
(negative) frequencies.
Choosing f0 as the carrier frequency, the baseband equivalent of gCh,o𝑣erall is given by2
2ARc −j2𝜋f 𝜏 2ARc −j2𝜋f 𝜏
g(bb)
Ch,o𝑣erall
(𝜏) = e 0 𝛿(𝜏 − 𝜏0 ) = e 0 0 𝛿(𝜏 − 𝜏 )
0 (4.31)
ATx ATx
Limited to signals sTx of the type (4.27), (4.30) can be rewritten as
[ ]
A
gCh,o𝑣erall (𝜏) = Re Rc ej𝜑Rc h(a) (𝜏) NB INPUT (4.32)
ATx

2
The constraint (bb)
Ch,o𝑣erall
(bb)
(f ) = 0 for f < −f0 was removed because the input already satisfies the condition Tx (f ) = 0
for f < −f0 .
4.1. Radio channel 197

Equation (4.32) indicates that the received signal exhibits a phase shift of 𝜑Rc = −2𝜋f0 𝜏0 with respect
to the transmitted signal, because of the propagation delay. In general, as the propagation delay is given
by d∕c, the delay per unit of distance is 3.3 ns/m.
Moreover, from (4.20) in free space, as the power decreases with the square of the distance between
transmitter and receiver, the amplitude of the received signal decreases with the distance, hence ARc ∝
ATx ∕d; in particular, if A0 is the amplitude of the received signal at the distance d = 1 from the trans-
mitter, then
ARc = A0 ∕d (4.33)

and the power of the received signal is given by

2
1 2 1 A0 𝙿
𝙿Rc = ARc = 2
= 02 (4.34)
2 2d d
where 𝙿0 is given by (4.21).
Reflection and scattering phenomena imply that the one-ray model is applicable only to propagation in
free space, and is not adequate to characterize radio channels, such as for example the channel between
a fixed radio station and a mobile receiver.
We will now consider the propagation of a signal in the presence of reflections. In particular, a sig-
nal propagates from the transmitter to the receiver through Nc paths (hence the multipath effect). Path
i, i = 0, … , Nc − 1, of length di , is characterized by Ki reflections caused by surfaces, where at each
reflection part of its power is absorbed by the surface, while the rest is re-transmitted in another direc-
tion. If aij is a complex number denoting the reflection coefficient of the j-th reflection of the i-th ray, the
total reflection factor is defined as
∏
Ki
ai = aij (4.35)
j=1

Path i attenuates the signal proportionally to |ai | < 1 besides 1∕di (see (4.33)). Moreover, the total phase
shift associated with path i is the sum of the phase shift due to the path length di (see (4.29)) and an
additional phase shift arg ai , due to reflections. Since signals over the Nc paths add at the receiver, the
resulting channel is modelled as
[ ]
A0 ∑
Nc −1
ai (a)
gCh,o𝑣erall (𝜏) = Re h (𝜏 − 𝜏i ) ANY INPUT (4.36)
ATx i=0 di

where 𝜏i = di ∕c is the delay of the i-th ray. The complex envelope of the channel impulse response (4.36)
around f0 is equal to

2A0 ∑
Nc −1
ai −j2𝜋f 𝜏
g(bb)
Ch,o𝑣erall
(𝜏) = e 0 i 𝛿(𝜏 − 𝜏 )
i ANY INPUT (4.37)
ATx i=0 di

We note that the only difference between the passband model and its baseband equivalent is constituted
by the additional phase term e−j2𝜋f0 𝜏i for the i-th ray.

Path loss as a function of distance

Limited to NB signals, extending the channel model (4.32) to the case of multipath, the received signal
can still be written as
sRc (t) = Re [ARc ej𝜑Rc ej2𝜋f0 t ] (4.38)
198 Chapter 4. Transmission channels

ψ2

ARc
ψ1
φRc
ψ0

Figure 4.12 Representation of ARc ej𝜑Rc in the complex plane.

where from (4.36) amplitude and phase are given by

∑ ai
Nc −1
∑
Nc −1
ARc ej𝜑Rc = A0 ej𝜑i = Ai ej𝜓i (4.39)
i=0
d i i=0

with 𝜑i = −2𝜋f0 𝜏i , and Ai and 𝜓i are the amplitude and phase, respectively, of the term A0 (ai ∕di )ej𝜑i .
Equation (4.39) is represented in Figure 4.12. As 𝙿0 = A20 ∕2, the received power is
|N∑ −1 |2
| c ai j𝜑 |
𝙿Rc = 𝙿0 || e i || (4.40)
| i=0 di |
| |
and is independent of the total phase of the first ray.
The simplest relation between the transmitted power and average (within a short time/distance)
received power due to distance is
𝙿
𝙿Rc = 𝛼0 (4.41)
d
or equivalently by
(𝚊PL )dB = 32.4 + 𝛼10 log10 d|km + 20 log10 f0 |MHz (4.42)
In the free space model, 𝛼 = 2.

Ground reflection model A slightly more complex model is the ground reflection model, a two-ray
model shown in Figure 4.13. It is based on geometric optics and considers both the direct path and
a ground reflected propagation path. This model yields the average received power over distances of
several kilometres for mobile radio systems that use tall towers with heights exceeding 50 m, as well
as for LOS micro-cell channels in urban environments [3]. h1 and h2 denote the heights of transmit and
receive antennas, respectively, that are placed at distance d, with
d ≫ h1 , d ≫ h2 (4.43)

LOS

h1 h2

Figure 4.13 Ground reflection model.

4.1. Radio channel 199

At the receiver, the signal is given by the sum of the direct LOS and the ground reflected component.
We suppose that the earth acts as an ideal reflecting surface and does not absorb power, so reflection
coefficient in this case is a1 = −1. Observing (4.40), and considering that for the above assumptions the
lengths of both paths are approximately equal to d, the received power is given by
𝙿0
𝙿Rc ≃|1 − ejΔ𝜑 |2 (4.44)
d2
where Δ𝜑 = 2𝜋f0 Δd∕c = 2𝜋Δd∕𝜆 is the phase shift between the two paths, and Δd = 2h1 h2 ∕d is the
difference between the lengths of the two paths. For small values of Δ𝜑, we obtain
h21 h22
|1 − ejΔ𝜑 |2 ≃ |Δ𝜑|2 = 16𝜋 2 (4.45)
𝜆2 d 2
from which, by substituting (4.21) in (4.44), we get
𝙿0 h21 h22
𝙿Rc = |Δ𝜑| 2
= 𝙿Tx 𝙶Tx 𝙶Rc (4.46)
d2 d4
We note that the received power decreases as the fourth-power of the distance d, that is 40 dB/decade
instead of 20 dB/decade as in the case of free space. Therefore, the law of power attenuation as a function
of distance changes in the presence of multipath with respect to the case of propagation in free space.

Example of radio propagation in indoor environments We now consider an indoor environment,

where both transmitter and receiver are positioned in a room, so that (4.43) does not hold any more. It is
assumed, moreover, that the rays that reach the receive antenna are due, respectively, to LOS, reflection
from the floor, and reflection from the ceiling, as shown in Figure 4.14.
As a result, the instantaneous received power is
|∑ |2
| 2 ai ej𝜑i |
|
𝙿Rc = 𝙿0 | | (4.47)
|
| i=0 di |
| |
where the reflection coefficients are a0 = 1 for the LOS path, and a1 = a2 = −0.7. With these assump-
tions, the power decreases with the distance in an erratic way, in the sense that by varying the position
of the antennas the received power presents fluctuations of about 20÷30 dB. In fact, depending on the

Tx antenna

LOS

Rc antenna

h1
h2

Figure 4.14 Example of propagation in indoor environment.

200 Chapter 4. Transmission channels

position, the phases of the various rays change and the sum in (4.39) also varies: in some positions all
rays are aligned in phase and the received power is high, whereas in others the rays cancel each other and
the received power is low. In the previous example, this phenomenon is not observed because the dis-
tance d is much larger than the antenna heights, and the phase difference between the two rays remains
always small.
To summarize, relation (4.41), that describes the average received power as a function of the path
loss exponent 𝛼, yields 𝛼 = 2 in the free space model and 𝛼 = 4 in the ground reflection model, so the
path loss is, respectively, 20 and 40 dB/decade. In general, we can also see the path loss as the attenua-
tion at a given distance and frequency, averaged over all other phenomena (such as fading described in
Section 4.1.5). For example, in real cellular environments, where more complicated multipath phenom-
ena occur, (4.41) still holds with exponent 𝛼 ∈ [2.5, 6].
Moreover, according to the 3GPP channel model for 5G cellular systems, in a urban area if hBS is the
base station antenna elevation measured from the average rooftop level [11], the path loss, as the average
attenuation at a given distance, is given by
(𝚊PL )dB = 40(1 − 4 × 10−3 hBS |m ) log10 d|km
−18 log10 hBS |m + 21 log10 f0 |MHz + 80 (4.48)
For a carrier frequency f0 = 900 MHz and hBS = 15 m, (𝚊PL )dB = 120.9 + 37.6 log10 d|km . This expres-
sion should be compared with (𝚊PL )dB in LOS
(𝚊PL )dB = 31.95 + 20 log10 d|km (4.49)
In rural areas, the Hata model is used and
(𝚊PL )dB = 69.55 + 26.16 log10 f0 |MHz − 13.82 log10 hBS |m (4.50)
+ (44.9 − 6.55 log10 hBS |m )log10 d|km − 4.78 (log10 f0 |MHz )
2

+ 18.33 log10 f0 |MHz − 40.94s

For f0 = 900 MHz and hBS = 95 m above ground, (𝚊PL )dB = 95.4 + 31.94 log10 d|km . In any case, it must
be (𝚊PL )dB ≥ (𝚊PL,fs )dB .

4.1.5 Fading effects in propagation models

In addition to path loss due to distance, the short-term received power exhibits fluctuations called fading;
these fluctuations are typically composed of two multiplicative components, macroscopic and micro-
scopic fading.
Macroscopic fading represents the long-term variation (within a distance of many wavelengths) of the
received power level, while microscopic fading represents short-term variation (within a distance of a
fraction of wavelength), as described next.

Macroscopic fading or shadowing

Macroscopic fading is generally attributed to shadowing effects of buildings or natural features, and an
example is shown in Figure 4.15. Indeed, it is determined by the local mean of fast fading and its sta-
tistical distribution has been derived experimentally. Its distribution depends on the antenna height, the
carrier frequency, and the specific environment. Experimentally, it is seen that the attenuation compo-
nent in dB due to macroscopic fading follows a normal distribution, hence it is also called log-normal
fading. In terms of attenuation, we have
(𝚊PL )dB = (𝚊PL )dB + (𝜉)dB (4.51)
4.1. Radio channel 201

65

60

55

50
(αPL)dB

45

40

35

30
0 0.2 0.4 0.6 0.8 1
log(d/dc)

Figure 4.15 Attenuation fluctuation as function of distance in wireless channels. Mean propagation
loss increases monotonically with distance (solid line), while fluctuations are due to macroscopic
fading (dashed line).

where (𝜉)dB ∼  (0, 𝜎(𝜉)

2
) models the shadowing component, while (𝚊PL )dB is the path loss attenua-
dB
tion due to distance. The shadowing components for two different receiver locations at distance Δ are
correlated with correlation given by the Gudmundson model [12]
𝚛(𝜉)dB (Δ) = 𝜎(𝜉)
2
e−|Δ|∕Dcoh (4.52)
dB

where Dcoh is the coherence distance.3 The standard deviation of (𝜉)dB depends on the particular envi-
ronment, usually being 𝜎(𝜉)dB ∈ [4, 12].
In practice, shadowing provides a measure of the adequacy of the adopted deterministic model. A
propagation model that completely ignores any information on land configuration, and therefore is based
only on the distance between transmitter and receiver, has shadowing with 𝜎(𝜉)dB = 12. Improving the
accuracy of the propagation model, for example by using more details regarding the environmental
configuration, shadowing can be reduced; in case we had an enormous amount of topographic data
and the means to elaborate them, we would have a model with 𝜎(𝜉)dB = 0. Hence, shadowing should be
considered in the performance evaluation of mobile radio systems, whereas for the correct design of a
network it is good practice to make use of the largest possible quantity of topographic data.

Microscopic fading
Microscopic fading refers to rapid fluctuations of the received signal power in space, time, and frequency
domain, and is caused by the signal scattering off objects along the propagation path and by changes in
the relative position of the transmitter and/or receiver and environment (e.g. foliage).
3
For a receiver moving along a straight line departing from the transmitter, if we denote the distance as d = id0 , with d0 a
suitable step size, we can model (𝜉)dB by an AR(1) process (see Section 1.10) and
(𝜉)dB [id0 ] = −𝚊1 (𝜉)dB [(i − 1)d0 ] + 𝑤[id0 ] (4.53)
with 𝑤(id0 ) white noise with variance 𝜎𝑤2 = (1 − |a1 |2 )𝜎(𝜉)
2
and 𝚊1 = −e−d0 ∕Dcoh . For a suitable representation of (𝜉)dB
dB
versus d, it must be d0 < Dcoh ∕5.
202 Chapter 4. Transmission channels

For a NB signal, assuming a large number of statistically independent components in the propagation
paths, the central limit theorem can be used to model ARc ej𝜑Rc in (4.39) as complex Gaussian. Hence,
phase 𝜑Rc ∼  ((0, 2𝜋]), and amplitude ARc has a Rayleigh probability density function (pdf)
a2
2a −
𝙿Rc
pARc (a) = e 1(a) (4.54)
𝙿Rc

where 𝙿Rc = 𝙿Tx ∕𝚊 is the short-term average received power.

If in addition to the scattering components, there is an LOS component, the signal envelope ARc will
have a Ricean distribution. Defining the Rice factor K as to the ratio between the power of the direct
component and the power of the reflected and/or scattered components, the pdf of ARc is
( √ )
2
2a(K + 1) −K− (K+1)a
𝙿Rc
K(K + 1)
pARc (a) = e I0 2a 1(a) (4.55)
𝙿Rc 𝙿Rc
where I0 (a) is the modified Bessel function of the first type and order zero defined as
𝜋
1
I0 (x) = ex cos 𝜃 d𝜃 (4.56)
𝜋 ∫0
Note that in the absence of a direct path (K = 0), the Ricean pdf in (4.55) reduces to the Rayleigh pdf
in (4.54), since I0 (0) = 1.
We note that both (4.54) and (4.55) refer to the NB channel model for a sinusoidal transmitted signal,
and a more general wideband channel model will be given in Section 4.1.7.

4.1.6 Doppler shift

In the presence of relative motion between transmitter and receiver, the frequency of the received signal
undergoes a shift with respect to the frequency of the transmitted signal, known as Doppler shift.
We now analyze in detail the Doppler shift. With reference to Figure 4.16, we consider a transmitter
Tx and a receiver that moves with speed 𝑣p from a point P to a point Q. The variation in distance between
the transmitter and the receiver is Δ𝓁 = 𝑣p Δt cos 𝜃, where 𝑣p is the speed of the receiver relative to the
transmitter, Δt is the time required for the receiver to go from P to Q, and 𝜃 is the angle of incidence of
the signal with respect to the direction of motion. 𝜃 is assumed to be approximately the same in P and
in Q. The phase variation of the received signal because of the different path length in P and Q is
2𝜋Δ𝓁 2𝜋𝑣p Δt
Δ𝜑 = = cos 𝜃 (4.57)
𝜆 𝜆
and hence the apparent change in frequency or Doppler shift is
1 Δ𝜑 𝑣p
fs = ± = ± cos 𝜃 (4.58)
2𝜋 Δt 𝜆

Tx

θ ∆l
Rc
P Q

Figure 4.16 Illustration of the Doppler shift.

4.1. Radio channel 203

where the sign is positive if Q is a further away from the transmitter than P, and negative otherwise. This
implies that if the NB signal (4.27) is transmitted, the received signal is
sRc (t) = Re[ARc ej2𝜋(f0 −fs )t ] (4.59)
With typically used unit measures, (4.58) becomes
fs = ±9.259 10−4 𝑣p |km∕h f0 |MHz cos 𝜃 (Hz) (4.60)
where 𝑣p |km/h is the speed of the mobile in km/h, and f0 |MHz is the carrier frequency in MHz. For example,
if 𝑣p = 100 km/h, f0 = 900 MHz, and 𝜃 = 0, with the receiver moving away from the transmitter, we
have fs = 83 Hz.
Indeed, from (4.59), the Doppler can also be interpreted as transmitting the NB signal at frequency
f0 , through a time varying channel with
2ARc −j2𝜋f t
g(bb)
Ch,o𝑣erall
(t, 𝜏) = e s 𝛿(𝜏) (4.61)
ATx
We now consider a NB signal transmitted in an indoor environment,4 where the signal received by the
antenna is given by the contribution of many rays, each with a different length. In the presence of a
single ray, the received signal would undergo only one Doppler shift. However, according to (4.58) the
frequency shift fs depends on the angle 𝜃. Therefore, because of the different paths, the received signal is
no longer monochromatic, and we speak of a Doppler spectrum to indicate the spectrum of the received
signal around f0 . This phenomenon occurs even when if both transmitter and receiver are static, but a
person or an object is moving around them, modifying the signal propagation.
The Doppler spectrum is characterized by the Doppler spread, which measures the dispersion in the
frequency domain that is experienced by a transmitted sinusoidal signal. For example, from (4.28), for
one ray channel model affected by Doppler, the PSD of the received amplitude is 𝛿(f − fs ) up to a constant
factor. It is intuitive that the more the characteristics of the radio channel are varying with time, the larger
the Doppler spread will be. An important consequence of this observation is that the convergence time
of channel-dependent algorithms used in receivers (e.g. adaptive equalization) must be much smaller
than the inverse of the Doppler spread of the channel, in order to follow the channel variations.

Example 4.1.1 (Doppler shift)

Consider a transmitter that radiates a sinusoidal carrier at frequency f0 = 1850 MHz. For a vehicle trav-
elling at 96.55 km/h (26.82 m/s), we want to evaluate the frequency of the received carrier if the vehicle
is moving: (a) approaching the transmitter, (b) going away from the transmitter, and (c) perpendicular
to the direction of arrival of the transmitted signal.
(a) The Doppler shift is positive and 𝜃 = 0; the received frequency is
fRc = f0 + fs = 1850 ⋅ 106 + 9.259 ⋅ 10−4 ⋅ 96.55 ⋅ 1850 = 1850.000 166 MHz (4.62)
(b) The Doppler shift is negative and 𝜃 = 0; the received frequency is
fRc = f0 − fs = 1850 × 106 − 9.259 ⋅ 10−4 ⋅ 96.55 ⋅ 1850 = 1849.999 834 MHz (4.63)
(c) In this case, cos 𝜃 = 0; therefore, there is no Doppler shift.

4
The term indoor is usually referred to areas inside buildings, possibly separated by walls of various thickness, material,
and height. The term outdoor, instead, is usually referred to areas outside buildings: these environments can be of
various types, for example, urban, suburban, rural, etc.
204 Chapter 4. Transmission channels

4.1.7 Wideband channel model

For a wideband signal with spectrum centred around the carrier frequency f0 , the channel model
(4.37) still holds. If 𝚊 is the attenuation due to both path loss and shadowing, we now write the overall
baseband-equivalent channel impulse response as
1
g(bb)
Ch,o𝑣erall
(t, 𝜏) = √ g(bb)
Ch
(t, 𝜏) (4.64)
𝚊
where g(bb)
Ch
(t, 𝜏) is normalized to a unit statistical power. Moreover,
∑
Nc −1
g(bb)
Ch
(t, 𝜏) = gi (t)𝛿(𝜏 − 𝜏i (t)) (4.65)
i=0

and gi represents the normalized complex-valued gain of the i-th ray that arrives with delay 𝜏i . Power
normalization yields
∑
Nc −1
E[|gi |2 ] = 1 (4.66)
i=0

For a given receiver location, (4.65) models the channel as a linear filter having time-varying impulse
response, where the channel variability is due to the motion of transmitter and/or receiver, or to changes
in the surrounding environment, or to both factors.
If the channel is time-invariant, or at least it is time-invariant within a short time interval, in this time
interval, the impulse response is only a function of 𝜏.
The transmitted signal undergoes three phenomena: (i) fading of some gains gi due to multipath, which
implies rapid changes of the received signal power over short distances (of the order of the carrier
wavelength) and brief time intervals, (ii) time dispersion of the impulse response caused by diverse
propagation delays of multipath rays, and (iii) Doppler shift, which introduces a random frequency
modulation that is in general different for each ray.
In a digital transmission system, the effect of multipath depends on both the relative duration of the
symbol period and the channel impulse response. If the duration of the channel impulse response is very
small with respect to the duration of the symbol period, i.e. the transmitted signal is NB with respect
to the channel, then the one-ray model is appropriate, and if the gain of the single ray varies in time we
have a flat fading channel. Otherwise, an adequate model must include several rays, and if the gains vary
in time, we have of a frequency selective fading channel.
Neglecting the group delay 𝜏0 , and letting 𝜏̃1 = 𝜏1 − 𝜏0 , a simple two-ray radio channel model has
impulse response
g(bb)
Ch
(t, 𝜏) = g0 (t)𝛿(𝜏) + g1 (t)𝛿(𝜏 − 𝜏̃1 (t)) (4.67)
At a given instant t, the channel is equivalent to a filter with impulse response shown in Figure 4.17 and
frequency response given by5 :
C (t, f ) = g0 (t) + g1 (t)e−j2𝜋f 𝜏̃1 (t) (4.69)
It is evident that the channel has a selective frequency behaviour, as the attenuation depends on fre-
quency. For g0 and g1 real-valued and time-invariant, from (4.69) the following frequency response is
obtained
|C (f )|2 = g20 + g21 + 2g0 g1 cos(2𝜋f 𝜏̃1 ) (4.70)

5
If we normalize the coefficients with respect to g0 , (4.69) becomes
C (f ) = 1 + b e−j2𝜋f 𝜏̃1 (4.68)
where b is a complex number. In the literature (4.68) is called Rummler model of the radio channel.
4.1. Radio channel 205

Tx antenna Rc antenna
(a)
(bb)
gCh (t,τ)
2

g0 + g1 2
g0(t) g1(t)
2
g0 – g1

0 τ0(t) τ1(t) τ 0 1/ τ1 f
(b) (c)

Figure 4.17 Physical representation (a) and model in time (b) and frequency (c) of a two-ray radio
channel, where g0 and g1 are assumed to be positive.

shown in Figure 4.17c. In any case, the signal distortion depends on the transmitted signal bandwidth.
In particular, a non-negligible signal distortion is observed if the signal bandwidth is of the order
of 1∕𝜏̃1 .
Going back to the general case, for wideband communications, rays with different delays are assumed
to be independent, i.e. they do not interact with each other. In this case, by using a suitable receiver,
from (4.65) the received power is

𝙿Tx ∑
Nc −1
𝙿Rc = |g |2 (4.71)
𝚊 i=0 i

From (4.71), we note that the received power is given by the sum of the square amplitude of all the rays.
Conversely, in the transmission of NB signals, the received power is the square of the vector amplitude
resulting from the vector sum of all the received rays. Therefore, for a given transmitted power, the
received power will be lower for a NB signal as compared to a wideband signal.

Multipath channel parameters

To study the performance of mobile radio systems, it is convenient to introduce a measure of the channel
dispersion in the time domain known as multipath delay spread (MDS). In order to simplify notation,
we consider here time-invariant channels, dropping dependency on time t. The MDS is the measure of
the time interval that elapses between the arrival of the first and the last ray. Another measure is the
delay time that it takes for the amplitude of the ray to decrease by x dB below the maximum value; this
time is also called excess delay spread (EDS).
However, the most common measure is the -mean square (rms) delay spread
√
2
𝜏rms = 𝜏2 − 𝜏1 (4.72)
206 Chapter 4. Transmission channels

Table 4.1: Values of E[|gi |2 ] (in dB) and 𝜏i (in ns) for three typical channels. In order to satisfy constraint
(4.75) it is necessary to scale all entries by a suitable constant

Standard GSM Indoor offices Indoor business

𝜏i E[|gi |2 ] 𝜏i E[|gi |2 ] 𝜏i E[|gi |2 ]
0 −3.0 0 0.0 0 −4.6
200 0 50 −1.6 50 0
500 −2.0 150 −4.7 150 −4.3
1600 −6.0 325 −10.1 225 −6.5
2300 −8.0 550 −17.1 400 −3.0
5000 −10.0 700 −21.7 525 −15.2
750 −21.7

where
∑
Nc −1
|gi |2 𝜏in
i=0
𝜏n = , n = 1, 2 (4.73)
∑
Nc −1
|gi | 2

i=0

Typical values of (average) rms delay spread are of the order of microseconds in outdoor mobile radio
channels (precisely on the order of 0.05/0.2/2.5 μs in rural/urban/hilly areas), and of the order of some
tenths of nanoseconds in indoor channels.
We define as power delay profile (PDP) 𝙼(𝜏), also called delay power spectrum or multipath intensity
profile, the expectation of the square amplitude of the channel gain as a function of given delays 𝜏i ,
𝙼(𝜏i ) = E(|gi |2 ), i = 0, … , Nc − 1 (4.74)
Associated to the function 𝙼(𝜏i ), we define the rms delay spread 𝜏 rms obtained replacing |gi |2 by 𝙼(𝜏i ) in
(4.73). From (4.66), it is
∑
Nc −1
𝙼(𝜏i ) = 1 (4.75)
i=0

Table 4.1 shows PDPs for some typical channels.

Statistical description of fading channels

Typically, gains {gi } are given by the sum of a large number of random components, that can be approx-
imated by independent Gaussian random variables. The most widely used statistical description is
g0 = C0 + g̃ 0 i=0
(4.76)
gi = g̃ i i = 1, … , Nc − 1
where C0 is a real-valued constant and g̃ i , i = 0, … , Nc − 1, are independent complex circular
Gaussian-distributed random variables, with zero mean and variance 𝜎i2 (see Example 1.7.3 on page
47). Moreover, in order to ensure the power normalization (4.77), we must have
∑
Nc −1
C02 + 𝜎i2 = 1 (4.77)
i=0

Therefore, the phase of g̃ i is uniformly distributed in [0, 2𝜋), the distribution of |gi | varies for each index
i. Since the first ray contains a direct (deterministic) component in addition to a random component as
4.1. Radio channel 207

other rays, |gi | will have a Rice distribution for i = 0, and a Rayleigh distribution for i ≠ 0. In particular,
introducing the normalized random variables
g
gi = √ i (4.78)
𝙼(𝜏i )
we have (see also (4.54) and (4.55))
2
√
p|g0 | (a) = 2(1 + K) ae−K−(1+K)a I0 (2a K(1 + K))1(a)
2 (4.79)
p|gi | (a) = 2a e−a 1(a)
where K is the Rice factor, with
C02 ∑
Nc −1
K= 𝙼d = 𝜎i2 (4.80)
𝙼d i=0

In turn, from (4.80) and (4.77), given K it is

√
K
C0 = (4.81)
K+1
and
𝙼d = 1 − C02 (4.82)
The pdf (4.79) is shown in Figure 4.18 for various values of K (where K = 0 provides the Rayleigh
distribution).
It may be useful to recall that if |gi | is Rayleigh, then
√
𝜋√ √
E[|gi |] = 𝙼(𝜏i ) = 0.8862 𝙼(𝜏i ) (4.83)
4
( )
𝜋
E[(|gi | − E[|gi |])2 ] = 1 − 𝙼(𝜏i ) = 0.2146 𝙼(𝜏i ) (4.84)
4

2
K = 10
1.8

1.6
K=5
1.4

1.2
K=2
p|g0| (a)

1
K=0
0.8

0.6

0.4

0.2

0
0 0.5 1 1.5 2 2.5 3
a

Figure 4.18 The Rice probability density function for various values of K. The Rayleigh density
function is obtained for K = 0.
208 Chapter 4. Transmission channels

Typical reference values for K are 2 and 10. If C0 = 0, i.e. no direct component exists, it is K = 0, and
the Rayleigh distribution is obtained for all the gains {gi }. For K → ∞, i.e. with no reflected and/or
scattered components and, hence, C0 = 1, we find the model having only the deterministic component.
In general, K falls off exponentially with distance, with K = 20 near the transmitter and K = 0 at large
distances.

4.1.8 Channel statistics

We evaluate the autocorrelation function of gi in (4.65) at two different instants and delays

𝚛g (t, t − Δt, 𝜏i , 𝜏j ) = E[gi (t)g∗j (t − Δt)] (4.85)

According to the model known as the wide-sense stationary uncorrelated scattering (WSSUS), the val-
ues of gi for rays that arrive with different delays are uncorrelated, and gi is stationary in t. Therefore,
we have {
𝚛g (Δt; 𝜏i ), 𝜏i = 𝜏j
𝚛g (t, t − Δt, 𝜏i , 𝜏j ) = (4.86)
0, 𝜏i ≠ 𝜏 j

In other words, the correlation is not zero only for gains at the same delay time and autocorrelation of
gi only depends on the difference of the times at which the gain is considered.

Power delay profile

In (4.86) for Δt = 0, we find the PDP (4.74). A typical channel model is the two rays model (see (4.68))
with two equal-power rays, i.e.
⎧1
⎪ , 𝜏=0
𝙼(𝜏) = ⎨ 2 (4.87)
1
⎪ 2 , 𝜏 = 2𝜏 rms
⎩
Sometimes the discrete-time function 𝙼(𝜏i ), rather than experimentally, is assigned analytically by sam-
pling a continuous-time function 𝙼cnt (𝜏) = E[|g𝜏 (t)|2 ] defined as in (4.74), where gi , with some abuse
of notation, is replaced by g𝜏 , the channel gain at lag 𝜏, with 𝜏 now a continuous variable.
As in the case of the previously studied channel model, we define the rms delay spread associated to
𝙼cnt (𝜏) as
√
√ ∞
√ ∫ (𝜏 − 𝜏 cnt )2 𝙼cnt (𝜏)d𝜏
𝜏 rms,cnt = √ −∞ ∞ (4.88)
∫−∞ 𝙼cnt (𝜏)d𝜏

where
∞
∫−∞ 𝜏𝙼cnt (𝜏)d𝜏
𝜏 cnt = ∞ (4.89)
∫−∞ 𝙼cnt (𝜏)d𝜏
For a Rayleigh channel model, two typical curves are now given for 𝙼cnt (𝜏).

1. Gaussian, unilateral
⎧√ 2 1 −𝜏 2 ∕(2𝜏 2 )
⎪ e rms,cnt , 𝜏≥0
𝙼cnt (𝜏) = ⎨ 𝜋 𝜏 rms,cnt (4.90)
⎪0, otherwise
⎩
4.1. Radio channel 209

2. Exponential, unilateral
{ 1
𝜏 rms,cnt
e−𝜏∕𝜏 rms,cnt , 𝜏≥0
𝙼cnt (𝜏) = (4.91)
0, otherwise

As 𝙼cnt (𝜏) does not model LOS, once we select a suitable sampling period TQ and a corresponding
number of gains Nc , for 𝜏i = iTQ , we set
{
s𝙼cnt (0) + C02 i = 0
𝙼(iTQ ) = (4.92)
s𝙼cnt (iTQ ) i = 1, … , Nc − 1

where s is a scaling factor to satisfy constraint (4.82).

Coherence bandwidth
The dispersion of 𝙼(𝜏) is often measured by an equivalent description of the channel in the frequency
domain using a parameter called coherence bandwidth, Bcoh , which is inversely proportional to 𝜏 rms .
Bcoh gives the range of frequencies over which the channel can be assumed to be flat, with approximately
constant gain and linear phase. If Bcoh is defined as the bandwidth over which the frequency correlation
function is above 0.5, then approximately it is [3]
1
Bcoh = (4.93)
5𝜏 rms
For example, in rural and hilly areas, Bcoh is on the order of MHz and hundreds of kHz, respectively.
For digital transmissions over such channels, we observe that if 𝜏 rms is about 20% of the symbol period
or larger, then signal distortion is not negligible. Equivalently, if Bcoh defined as (4.93) is lower than the
modulation rate, then we speak of a frequency selective fading channel. Otherwise the channel is denoted
as flat fading. Note that in the presence of flat fading, the received signal may vanish completely, whereas
frequency selective fading produces several replicas of the transmitted signal at the receiver, so that a
suitably designed receiver can recover the transmitted information.

Example 4.1.2 (Power delay profile)

We compute the average rms delay spread for the multipath delay profile 𝙼(0) = −20 dB,
𝙼(TQ ) = −10 dB, 𝙼(2TQ ) = −10 dB, 𝙼(5TQ ) = 0 dB, with TQ = 1 μs, and determine the coherence
1
bandwidth, defined as Bcoh = .
5𝜏 rms
From (4.73), we have
(1)(5) + (0.1)(1) + (0.1)(2) + (0.01)(0)
𝜏= = 4.38 μs (4.94)
0.01 + 0.1 + 0.1 + 1
and
(1)(5)2 + (0.1)(1)2 + (0.1)(2)2 + (0.01)(0)
𝜏2 = = 21.07 (μs)2 (4.95)
0.01 + 0.1 + 0.1 + 1
Therefore, we get
√
𝜏 rms = 21.07 − (4.38)2 = 1.37 μs (4.96)

Consequently, the coherence bandwidth of the channel is equal to Bcoh = 146 kHz.
210 Chapter 4. Transmission channels

Doppler spectrum
We now analyse the WSSUS channel model with reference to time variations due to the motion of ter-
minals and/or surrounding objects. First, we introduce the correlation function of the channel frequency
response taken at instants t and t − Δt, and, respectively, at frequencies f and f − Δf as

𝚛C (t, t − Δt; f , f − Δf ) = E[C (t, f )∗C (t − Δt, f − Δf )] (4.97)

with
+∞
C (t, f ) = g(bb) (t, 𝜏) e−j2𝜋f 𝜏 d𝜏 (4.98)
∫−∞ Ch

By inserting (4.98) into (4.97) we find that 𝚛C depends only on Δt and Δf . Moreover, it holds

∑
Nc−1
𝚛C (Δt; Δf ) = 𝚛g (Δt; 𝜏i )e−j2𝜋Δf 𝜏i (4.99)
i=0

i.e. 𝚛C (Δt; Δf ) is the Fourier transform of 𝚛g (Δt; 𝜏i ) as a function of 𝜏i . The Fourier transform of 𝚛C as
a function of Δt, for a given Δf , is given by
∞
C (𝛼; Δf ) = 𝚛C (Δt; Δf )e−j2𝜋𝛼(Δt) d(Δt) (4.100)
∫−∞
The time variation of the frequency response is measured by C (𝛼; 0).
Now, we define the Doppler spectrum (𝛼), which represents the power of the Doppler shift for
different values of the frequency 𝛼 as the Fourier transform of the autocorrelation function of the impulse
response in correspondence of a given delay 𝜏i , evaluated at two different instants, separated by Δt, i.e.
+∞ 𝚛g (Δt; 𝜏i )
(𝛼) = e−j2𝜋𝛼Δt d(Δt) (4.101)
∫−∞ 𝚛g (0; 𝜏i )

Note that we omitted the delay 𝜏i in (𝛼), as we assume the same Doppler spectrum for all delays.6
The term 𝚛g (0; 𝜏i ) in (4.101) represents a normalization factor such that
+∞
(𝛼) d𝛼 = 1 (4.102)
∫−∞

By letting 𝚛d (Δt) =  −1 [(𝛼)], from (4.101) we note that 𝚛g (Δt; 𝜏i ) is a separable function,

𝚛g (Δt; 𝜏i ) = 𝚛d (Δt) ⋅ 𝚛g (0; 𝜏i ) = 𝚛d (Δt) 𝙼(𝜏i ) (4.103)

with (see (4.102))

𝚛d (0) = 1 (4.104)

and 𝙼(𝜏) such that (4.75) holds. With the above assumptions, the following equality holds

(𝛼) = C (𝛼; 0) (4.105)

Therefore, from (4.100) (𝛼) can also be obtained as the Fourier transform of 𝚛C (Δt; 0), which in turn
can be determined by transmitting a sinusoidal signal (hence Δf = 0) and estimating the autocorrelation
function of the amplitude of the received signal.

6
In very general terms, we could have a different Doppler spectrum for each path, or gain gi , of the channel.
4.1. Radio channel 211

Coherence time
The maximum frequency of a Doppler spectrum support is called the Doppler spread of a given channel
as from (4.58) for 𝜃 = 0, i.e.
𝑣p 𝑣p f0
fd = = (4.106)
𝜆 c
The Doppler spread gives a measure of the channel fading rate.
Variations in time of the channel are often measured by the coherence time Tcoh , as the time interval
within which a channel can be assumed to be time invariant or static. Hence, Tcoh is inversely proportional
to fd .
For a digital modulation system with symbol period T, we usually say that the channel is slow fading
if fd T < 10−3 and fast fading if fd T > 10−2 .

Doppler spectrum models

A widely used model, known as the Jakes model or classical Doppler spectrum, to represent the Doppler
spectrum is due to Clarke. It is given in Figure 4.19 with
⎧ 1 1
⎪ √ , |f | ≤ fd
(f ) = ⎨ 𝜋fd 1 − (f ∕fd )2 (4.107)
⎪ 0, otherwise
⎩
with inverse Fourier transform (see also (4.103))
𝚛d (Δt) = J0 (2𝜋fd Δt) (4.108)
where 𝜋
1
J0 (x) = ejx cos 𝜃 d𝜃 (4.109)
𝜋 ∫0
is the Bessel function of the first type and order zero. The model of the Doppler spectrum described
above agrees with the experimental results obtained for mobile radio channels.
For indoor radio channels, the Doppler spectrum can be modelled as
⎧ 1
⎪ , |f | ≤ fd
(f ) = ⎨ 2fd (4.110)
⎪ 0, elsewhere
⎩
with a corresponding
𝚛d (Δt) = sinc(2fd Δt) (4.111)
A further model assumes that the Doppler spectrum is described by a second- or third-order Butterworth
filter with the 3 dB cutoff frequency equal to fd .

Power angular spectrum

A receive antenna sees waves impinging at different angles; so for each angle, there is a certain amount
of the overall power that the antenna receives.
If we indicate with 𝜗 the horizontal angle of aperture at the receive antenna (or array antenna), we
introduce the density function (𝜗), called angular spectrum, which indicates the percentage of power
received at angle 𝜗. As in previous analogous definitions, we also define the rms angle spread associated
to (𝜗). The rms angular spread depends on the specific environment, and is high in urban centres and
low in rural scenarios.
212 Chapter 4. Transmission channels

20

15

10

5
(D( f )/π fd)dB

–5

–10

–15

–20
–1.5 –1 –0.5 0 0.5 1 1.5
f f̸ d

Figure 4.19 Classical Doppler spectrum.

Coherence distance
The phenomena behind the angular spread have also an impact on space-selective fading, characterized
by the coherence distance Dcoh defined as the distance in space within which received signals are strongly
correlated, i.e. fading effects are similar. Dcoh is inversely proportional to the rms angle spread. Hence,
antenna array elements that are spaced above Dcoh tend to experience uncorrelated fading. Note that the
coherence distance is used for example to model space correlation of shadowing (see (4.52)).
For example, in Section 4.1.11, we will see that in a cellular scenario with uniformly distributed
scatterers around a terminal, Dcoh = 𝜆∕2; hence, for f0 = 1.8 GHz, we have Dcoh = 0.08 m. In general,
typical values of Dcoh at the base station are from 3𝜆 to 20𝜆, respectively, in hilly/dense urban areas and
flat rural areas, while at terminals Dcoh ∈ [𝜆∕4, 5𝜆]. Overall, Dcoh strongly depends on both the terrain
and the antenna height, and in any case the angle spread does not exceed the antenna beamwidth.

On fading
A signal that propagates in a radio channel for mobile communications undergoes a fading that depends
on the channel characteristics. In particular, whereas the delay spread due to multipath leads to dispersion
in the time domain and therefore frequency-selective fading, the Doppler spread causes time-selective
fading.
Fading can be divided into flat fading and frequency selective fading. In the first case, the channel has
a constant gain; in other words, the inverse of the transmitted signal bandwidth is much larger than the
delay spread of the channel and g(t, 𝜏) can be approximated by a delta function, with random amplitude
and phase, centred at 𝜏 = 0. In the second case, instead the channel has a varying frequency response
within the passband of the transmitted signal and consequently the signal undergoes frequency selective
fading; these conditions occur when the inverse of the transmitted signal bandwidth is of the same order
4.1. Radio channel 213

or larger than the coherence bandwidth of the channel. The received signal consists of several attenuated
and delayed versions of the transmitted signal.
A channel can be fast fading or slow fading. In a fast fading channel, the impulse response of the
channel changes very fast, with coherence time of the channel on the order of the symbol period; this
condition leads to signal distortion, which increases with increasing Doppler spread. Usually, there are
no remedies to compensate for such distortion unless the symbol period is decreased; on the other hand,
this choice leads to larger intersymbol interference. In a slow fading channel, the impulse response
changes much more slowly with respect to the symbol period. In general, the channel can be assumed
as time invariant for a time interval that is proportional to the inverse of the rms Doppler spread.

4.1.9 Discrete-time model for fading channels

Our aim is to model a transmission channel defined in the continuous-time domain by a channel in
the discrete-time domain characterized by sampling period TQ . We immediately notice that the various
delays in (4.65) must be a multiple of TQ and consequently we need to approximate the delays of the
power delay profile (see, e.g. Table 4.1). The discrete-time model of the radio channel is represented, as
illustrated in Figure 4.20, by a time-varying linear filter whose coefficient gi corresponds to the complex
gain of the ray i with delay iTQ , i = 0, 1, … , Nc − 1. In the case of flat fading, Nc = 1.
If the channel is time invariant (fd = 0), all coefficients {gi }, i = 0, … , Nc − 1, are constant, and are
obtained as realizations of Nc random variables. In general, however, the coefficients {gi } are random
processes. To generate each process gi (kTQ ), the scheme of Figure 4.21 is used, where 𝑤i (𝓁TP ) is a
complex-valued Gaussian white noise with zero mean and unit variance, hds is a narrowband filter that
produces a signal g′i with the desired Doppler spectrum, and hint is an interpolator filter. We choose TP
such that and 1∕10 ≤ fd TP ≤ 1∕5. Note that, as fd TQ ≪ 1, a direct realization of g′i (kTQ ) by filtering
𝑤i (lTp ) is not feasible.
The interpolator output signal is then multiplied by 𝜎i , which imposes the desired power delay profile.
If the channel model includes a deterministic component for ray i, a constant Ci must be added to the
random component g̃ i . Here we assume that only C0 may be different from zero. Furthermore, if the
channel model includes a Doppler shift fsi for the i-th branch, then we need to multiply the term Ci + g̃ i
by the exponential function exp(j2𝜋fsi kTQ ). In order to ensure average unit power of the channel (see
(4.66)), g′i has unit statistical power7 and {𝜎i } satisfy (4.77). If we start from a continuous-time model

x(kTQ)
TQ TQ TQ

g0(kTQ) g1(kTQ) gNc–1(kTQ)

y(kTQ)

Figure 4.20 Discrete time model of a radio channel.

∑∞
7
It is 𝙼g′ = 1 if 𝙼𝑤i = 1, and filter has unit energy ( i=0 |hds (iTP )|2 = 1). If TP ∕TQ is very large, interpolation by hint does
i
not change the short-time signal power.
214 Chapter 4. Transmission channels

narrowband interpolator Ci
filter filter
wi(lTP) gʹi(lTP) gʹi(kTQ) g~i(kTQ) gi(kTQ)
hds hint
TP TP TQ σi

exp(2jπfsi kTQ)

Figure 4.21 Model to generate the i-th coefficient of a time-varying channel.

of the power delay profile, we need to obtain a sampled version as shown in (4.92). Hence, from the
assigned value of the Rice factor K, we determine C0 by (4.81). Next, by imposing the power delay
profile 𝙼(𝜏i ), we have
𝙼cnt (iTQ )
𝜎i2 = ∑N −1 [1 − C02 ], i = 0, 1 … , Nc − 1 (4.112)
c
j=0
𝙼 cnt (jT Q )
The same equation (4.112) can be used if 𝙼cnt (iTQ ) is replaced by a table of values as in Table 4.1.

Generation of a process with a pre-assigned spectrum

The signal g′i with the desired Doppler spectrum can be obtained in two ways: (1) use the scheme of
Figure 4.21 with |ds (f )|2 = (f ) and (2) generate a set of Nf (at least 10) complex sinusoids with
frequencies {±fm }, m = 1, … , Nf , in the range from −fd to fd .
We analyse the two methods.

(1) Using a filter: We give the description of hds for two cases:

(1.a) IIR filter with classical Doppler spectrum: Now hds is implemented as the cascade of two filters.
The first, Hds1 (z), is a finite-impulse-response (FIR) shaping filter with amplitude characteris-
tic of the frequency response given by the square root of (4.107). The second, Hds2 (z), is an
infinite-impulse-response (IIR) Chebychev lowpass filter, with cutoff frequency fd . Table 4.2
reports values of the overall filter parameters for fd TP = 0.1 [13].
(1.b) Doppler spectrum defined by a second-order Butterworth filter: Given 𝜔d = 2𝜋fd , where fd is
the Doppler spread, the transfer function of the discrete-time filter is
c0 (1 + z−1 )2
Hds (z) = ( ) (4.113)
∑
2
1+ 𝚊n z−n
n=1

where, defining
𝜔0 = tan(𝜔d TP ∕2)
we have [14]
2(1 − 𝜔20 )
𝚊1 = − √ (4.114)
1 + 𝜔20 + 2 𝜔0
1 + 𝜔40
𝚊2 = √ (4.115)
(1 + 𝜔20 + 2 𝜔0 )2
1
c0 = (1 + 𝚊1 + 𝚊2 ) (4.116)
4
4.1. Radio channel 215

Table 4.2: Parameters of an IIR filter which implements a

classical Doppler spectrum

Hds (z) = B(z)∕A(z) fd TP = 0.1

{𝚊n }, n = 0, … , 11 ∶
1.0000 e + 0 −4.4153 e + 0 8.6283 e + 0 −9.4592 e + 0
6.1051 e + 0 −1.3542 e + 0 −3.3622 e + 0 7.2390 e + 0
−7.9361 e + 0 5.1221 e + 0 −1.8401 e + 0 2.8706 e − 1
{𝚋n }, n = 0, … , 21 ∶
1.3651 e − 4 8.1905 e − 4 2.0476 e − 3 2.7302 e − 3
2.0476 e − 3 9.0939 e − 4 6.7852 e − 4 1.3550 e − 3
1.8067 e − 3 1.3550 e − 3 5.3726 e − 4 6.1818 e − 5
−7.1294 e − 5 −9.5058 e − 5 −7.1294 e − 5 −2.5505 e − 5
1.3321 e − 5 4.5186 e − 5 6.0248 e − 5 4.5186 e − 5
1.8074 e − 5 3.0124 e − 6

The filter output gives

g′i (𝓁TP ) = −𝚊1 g′i ((𝓁 − 1)TP ) − 𝚊2 g′i ((𝓁 − 2)TP )
+ c0 (𝑤̃ i (𝓁TP ) + 2𝑤̃ i ((𝓁 − 1)TP ) + 𝑤̃ i ((𝓁 − 2)TP )) (4.117)

(2) Using sinusoidal signals: Let

Nf
∑
g′i (𝓁TP ) = Ai,m [ej(2𝜋fm 𝓁TP +𝛼i,m ) + e−j(2𝜋fm 𝓁TP +𝛽i,m ) ] (4.118)
m=1

The spacing between the different frequencies is Δfm ; letting f1 = Δf1 ∕2, for m > 1 we have fm = fm−1 +
Δfm . Each Δfm can be chosen as a constant,
fd
Δfm = (4.119)
Nf
fd
or, defining Kd = 1∕3 (f )df , as
∫0
Kd
Δfm = , m = 1, … , Nf (4.120)
Nf 1∕3 (fm )

Suppose f 0 = 0 and f m = f m−1 + Δfm , m = 1, … , Nf , the choice (4.120) corresponds to minimizing the
error
Nf
∑ fm
(fm − f )2 (f )df (4.121)
∫f
m=1 m−1

The phases 𝛼i,m , 𝛽i,m are uniformly distributed in [0, 2𝜋) and statistically independent to ensure that the
real and imaginary parts of g′i are statistically independent. The amplitude is given by
√
Ai,m = (fm )Δfm (4.122)
If (f ) is flat, by the central limit theorem (for Nf → ∞), we can claim that g′i is a Gaussian process;
if instead (f ) presents some frequencies with large amplitude, Ai,m must be generated as a Gaussian
random variable with zero mean and variance (fm )Δfm .
216 Chapter 4. Transmission channels

0.8

0.6
Amplitude

0.4

0.2

0
0
0.5
1
1.5 25
τ/T 20
2 15
2.5 10 t
5
3
0

Figure 4.22 Nine realizations of |g(bb)

Ch
(t, 𝜏)| for a Rayleigh channel with an exponential power delay
profile having 𝜏 rms,cnt = 0.5 T.

Figure 4.22 shows nine realizations of the amplitude of the impulse response of a Rayleigh channel
obtained by the simulation model of Figure 4.20, for an exponential power delay profile with 𝜏 rms,cnt =
0.5 T. The Doppler frequency fd was assumed zero. We point out that the parameter 𝜏 rms provides scarce
information on the actual behaviour of g(bb)
Ch
, which can scatter for a duration equal to 4–5 times 𝜏 rms .

4.1.10 Discrete-space model of shadowing

The aim is to simulate (𝜉)dB [n] in a two-dimensional (2D) space with discrete coordinates
c = (c1 , c2 ) = (n1 A, n2 A) = (n1 , n2 )A, (n1 , n2 ) ∈  ⊂ ℤ2 (4.123)
with A a suitable (space) step size, given the autocorrelation function 𝚛(𝜉)dB (Δ), a circularly symmetric
function, where √ √
Δ= c21 + c22 = A n21 + n22 (4.124)
We need to introduce the 2D discrete Fourier transform (DFT). Given a pulse hsh (n), its 2D DFT on the
domain { } { }
N N N N
 = − 1 , … , 0, … , 1 − 1 × − 2 , … , 0, … , 2 − 1 (4.125)
2 2 2 2
is denoted as sh (k), k = (k1 , k2 ) ∈ , and defined as [15]
N2 N2
−1 −1
∑2 ∑2
−k2𝜋
k1 n1
−j2𝜋
k2 n2

sh (k) = DFT2 [hsh (n)] = hsh (n1 , n2 )e N1

e N2
(4.126)
N1 N2
n1 =− 2
n2 =− 2

while in turn, we have

N1 N2
−1 −1
1 ∑2 ∑2
j2𝜋
k1 n1
j2𝜋
k2 n2

hsh (n) = DFT−1

2 [sh (k)] = sh (k1 , k2 )e N1
e N2
(4.127)
N1 N2 N1 N2
k1 =− 2
k2 =− 2
4.1. Radio channel 217

We note that sh (k) is periodic in k1 and k2 , of period N1 and N2 , respectively. We note that a similar
property holds for hsh (n) in (4.127). In fact, (4.126) leads to

∑
+∞
∑
+∞
hsh (n1 − 𝓁1 N1 , n1 − 𝓁2 N2 ) = hsh (n1 , n2 ) (4.128)
𝓁1 =−∞ 𝓁2 =−∞
( √ )
Given 𝚛(𝜉)dB A n21 + n22 , we evaluate its PSD

√
(k) = DFT2 [𝚛(𝜉)dB (A n21 + n22 )] k∈ (4.129)

It is interesting to recall that the Fourier transform of a circularly symmetric function is still circularly
symmetric [15].
Now, we consider
√
sh (k) =  (k) and hsh (n) = DFT−1 2 [sh (k)] (4.130)

where  is such that hsh has unit energy, i.e.

∑ 1 ∑
|hsh (n)|2 = | (k)|2 = 1 (4.131)
n∈
N1 N2 k∈ sh

Coming back to the problem of simulating (𝜉)dB [n], we consider two solutions.

1. Filtering approach: If by setting to zero small values of hsh it turns out that its support, h , is much
smaller that , the following derivation of (𝜉)dB [n] is very efficient from a computational point
of view.
We generate an i.i.d. Gaussian noise across the space ,

𝑤(n) ∼  (0, 𝜎(𝜉)

2
), n∈ (4.132)
dB

Then, filtering 𝑤(n) by hsh (n) in (4.130) yields the desired result, i.e.
∑
(𝜉)dB [n] = hsh (p)𝑤(n − p), n ∈  (4.133)
p∈h

A word of caution: on the boundaries of  the convolution (4.133) is affected by the transient of
hsh , hence the corresponding values should be dropped.
The filtering approach to generate (𝜉)dB [n] is also very efficient if we are interested in evaluating
it only on a small subset of  rather than over the whole region, e.g. along a given path inside .
2. Frequency domain approach: If the above conditions are not verified, the general approach is
preferable [16].
Let  be an i.i.d. Gaussian noise in the frequency domain,

(k) ∼  (0, N1 N2 𝜎(𝜉)

2
), k∈ (4.134)
dB

Then, we multiply (k) by sh (k) in (4.130). By taking the inverse DFT, we obtain the desired
results, i.e.
(𝜉)dB [n] = DFT−1
2 [sh (k)(k)], n∈ (4.135)

As (4.135) is implementing a circular convolution, now (4.135) is not affected by the boundary
effect.
218 Chapter 4. Transmission channels

4.1.11 Multiantenna systems

A way to overcome fading consists in using more antennas (namely an array of omnidirectional antennas)
at the either the transmitter or the receiver, or both devices. Due to its importance, the entire Chapter 9
is devoted to this topic. Indeed, the model described until now holds for any couple of transmit-receive
antennas of the arrays. Here we just provide the geometrical or statistical relations between the rays for
various antenna couples.

Line of sight
For a single source (that can be either a transmit antenna or a scatterer), we consider the scenario shown
in Figure 4.23, with an array of receive antennas, and where 𝛼 represents the angle of arrival (AoA), i.e.
the angle between the direction of arrival (DoA) of the ray and the line of the antenna array. Note that we
consider a direct transmission between the source and the receive antenna, thus this can be considered
as an LOS scenario. Let D(r1 ,r2 ) be the distance between receive antennas r1 and r2 , and d(r) the distance
between the source and receive antenna r. Neglecting attenuation, signals at the antennas are affected
by phase shifts due to their distance from the source. For the NB model of (4.28), the signal received by
antenna r is affected by the phase offset (see (4.29))
d(r)
(r)
C
= e−j2𝜋 𝜆 (4.136)
Hence, the correlation function of the channel at antenna r1 and r2 , for a given source is
{
D(r1 ,r2 ) sin 𝛼
e−j2𝜋 𝜆 , r1 ≠ r2
(r1 ) (r2 )
(r1 ,r2 ) (r1 ) (r2 )∗ −j2𝜋 d −d
𝚛 = C C = e 𝜆 = (4.137)
C
1, r1 = r2
where we used the geometric approximation (see Figure 4.23)
d(r1 ) − d(r2 ) ≃ D(r1 ,r2 ) sin 𝛼 (4.138)
Let us now compute the correlation of the channel at antennas r1 and r2 , by averaging also on the AoA
𝛼, over its angular spectrum (𝛼), with support (−𝜗Rc ∕2, 𝜗Rc ∕2), thus obtaining
{ D(r1 ,r2 ) sin 𝛼
+𝜗 ∕2
(r1 ,r2 ) ∫−𝜗 Rc∕2 e−j2𝜋 𝜆 (𝛼)d𝛼, r1 ≠ r2
𝚛 = Rc (4.139)
C
1, r1 = r2
When the probability density function of direction of arrival is uniform and 𝜗Rc = 2𝜋, we have 𝚛(r1 ,r2 ) =
J0 (2𝜋D(r1 ,r2 ) ∕𝜆), where J0 (⋅) is defined in (4.109). As J0 (𝜋) ≃ 0, to achieve zero correlation, the antenna

Dmin sin α
Dmin

Figure 4.23 Propagation model for an LOS scenario.

4.1. Radio channel 219

θRc = π
20
0.8
θ = π
Rc 12
θRc = π
0.6 8
θ = π
Rc 4

0.4
r (r,r+1)

0.2

–0.2

–0.4
–30 –20 –10 0 10 20 30
Dmin/λ

Figure 4.24 Channel correlation (4.139) at adjacent antennas in a fading MIMO channel with a
uniformly distributed direction of arrival.

elements should be spaced by 𝜆∕2. Figure 4.24 shows the correlation between adjacent antennas, where
D(r,r+1) = Dmin , for small values of 𝜗Rc .
As it has already been observed, antennas at a large enough distance will receive essentially uncor-
related signals. However, in handsets or terminals, large separations among the antennas may not be
feasible, whereas, when either the transmitter or receiver are not surrounded by scatterers, the antenna
signals are correlated. Overall, small antenna spacing and lack of scattering cause the antenna signals
to be correlated [4].
Similar considerations hold in the presence of multiple antennas transmitting signals that hit a source,
before been reflected to a receive antenna. The geometric observations now hold for the phase shifts
introduced by the path from the various transmit antennas to the source, where 𝛼 now becomes the
angle of departure (AoD) between the transmit antenna array and the source.

Discrete-time model
We aim at modelling a transmission channel between NTx transmit antennas and NRc receive antennas.
For the sake of simplicity, we assume that all antennas are on the same plane, and scatterers are at a far
distance from both the transmit and the receive antennas, thus a single AoD (AoA) is enough to char-
acterize the path between the transmit (receiver) antennas and each scatterer. In general, a discrete-time
model for multiple-input-multiple-output (MIMO) channel includes all the effects of channels between
two antennas, i.e. path loss, fading, shadowing, and Doppler. However, it must also take into account
the correlation of channels among different antennas, as described above.
For each couple of transmit-receive antennas (r, t), we have a channel with taps gi(r,t) (kTQ ), with
r = 1, … , NRc , and t = 1, … , NTx . Two approaches are possible for the generation of the channel, accord-
ing to the number of scatterers of the channel.
220 Chapter 4. Transmission channels

Small number of scatterers

When the number of scatterers is small, they can be grouped into Nc clusters, each including the same
number of scatterers Nsct . Rays of scatterers of cluster i yield the same delay 𝜏i and the same power 𝜎i2 .
(Tx) (Rc)
Ray n = 0, … , Nsct − 1, within cluster i has AoD 𝛼i,n and AoA 𝛼i,n . In general, they are correlated
variables: see [17] for a detailed description.
Let us define
(Tx) (Tx) T (Rc) (Rc) T
cTx,i,n = [cos 𝛼i,n , sin 𝛼i,n ] , cRc,i,n = [cos 𝛼i,n , sin 𝛼i,n ] (4.140)
and the position of antennas t and r on the plane at time kTQ as

p(t)
Tx
(kTQ ) = [p(t)
Tx,1
(kTQ ), p(t)
Tx,2
(kTQ )]T , p(r)
Rc
(kTQ ) = [p(r)
Rc,1
(kTQ ), p(r)
Rc,2
(kTQ )]T (4.141)
where in any case we assume that antennas are close to each other (within a few 𝜆). We indicate the
velocities of the transmitter and receiver (we assume that scatterers are static) as
𝒗(t)
Tx
(kTQ ) = [𝑣(t)
Tx,1
(kTQ ), 𝑣(t)
Tx,2
(kTQ )]T , 𝒗(r)
Rc
(kTQ ) = [𝑣(r)
Rc,1
(kTQ ), 𝑣(r)
Rc,2
(kTQ )]T (4.142)
Up to an irrelevant phase shift and neglecting scaling by the path loss due to distance and shadowing
components (see (4.64)), the model for channel tap i at time kTQ between antennas t and r is (see also
(4.136))

1 ∑
Nsct −1 cT
(t )
p A (kTQ )
Tx,i,n Tx
cT
(r )
p A (kTQ )
Rc,i,n Rc
gi(r,t) (kTQ ) = √ e−j2𝜋 𝜆 e−j2𝜋 𝜆

Nsct n=0
(t ) (r )
cT 𝒗 A (kTQ ) cT 𝒗 A (kTQ )
Tx,i,n Tx Rc,i,n Rc
× e−j2𝜋 𝜆
kTQ
e−j2𝜋 𝜆
kTQ
(4.143)
which includes both the antenna correlation and the Doppler effect. This model still assumes a planar
wave hitting the receive antennas (see Figure 4.23), i.e. the signal source (cluster i) is at far distance from
the antennas. Therefore, (4.143) describes the resulting geometrical model, including the movement of
the antennas through their velocity. A similar observation holds from the transmit antennas.
The model with a small number of scatters is particularly suitable for transmissions at mmWave,
i.e. with 𝜆 ∈ [1, 10] mm, or f0 = c∕𝜆 ∈ [30,300] GHz. At these frequencies, the free-space attenuation
per unit length is very high, therefore multiple reflections give negligible contributions and only very
few (Nc = 1 and Nsct = 2 or 3) scatterers characterize the channel. Hence at these frequencies, multiple
antennas are required, as will be seen in more details in Chapter 9.
In this case, (4.143) yields the channel at time kTQ , which in Chapter 9 will be denoted as

C(r,t) (kTQ ) = g0(r,t) (kTQ ) (4.144)

This notation reflects the fact that C(r,t) (kTQ )

is the channel frequency response between the
transmit-receive antenna couple. The corresponding flat fading MIMO channel matrix at time kTQ is
NRc × NTx with entries
[C (kTQ )]r,t = C(r,t) (kTQ ), r = 1, … , NRc , t = 1, … , NTx (4.145)

Large number of scatterers

Here to simplify notation, we drop the Doppler effect, which in any case can be introduced for each
channel coefficient next, as described in Section 4.1.9.
For a large number scatterers, we extend the general model of Section 4.1.9; however, we only give the
model of the MIMO channel matrix in correspondence of cluster i = 0 with lag 𝜏0 . The channel model
4.1. Radio channel 221

for other clusters (at different lags) will be the same, with a proper power assignment among various
clusters. Let us define the following matrices.
Spatially white noise matrix 𝒘, NRc × NTx , with entries

𝑤(r,t) ∼  (0, 1) i.i.d. (4.146)

Receive antennas correlation matrix R(Rc) , NRc × NRc , with entries given by (4.139).
Transmit antennas correlation matrix R(Tx ) , NTx × NTx , with entries given by (4.139) and receive anten-
nas replaced by transmit antennas.
Spatial channel matrix g0 , NRc × NTx , with entries g0(r,t) .
We have
g0 = (R(Rc) )1∕2 𝒘 (R(Tx)T )1∕2 (4.147)

We analyse the correlation matrix of entries of g0 . Introducing the notation

(1,NTx ) (NRc ,NTx ) T
vec(g0 ) = [g(1,1)
0
, … , g0 , g(2,1)
0
, … , g0 ] (4.148)

the correlation among entries of g0 is simply the NRc NTx × NRc NTx matrix

Rvec(g0 ) = E[vec(g0 )(vec(g0 ))H ] (4.149)

If ⊗ denotes the Kronecker matrix product8

Rvec(g0 ) = R(Rc) ⊗ R(Tx) (4.152)

In short notation, it is common to write

g0 ∼  (𝟎, R(Rc) ⊗ R(Tx) ) (4.153)

This model approximately holds at frequencies up to 1 GHz.

(r ,t )
It is interesting to note that in (4.143) for Nsct → ∞, gi A A tends to become Gaussian with correlation
matrix (4.152). In this case, the two approaches (4.143) and (4.147) yield the same statistical model for
the channel matrix.
When the antennas at both the transmitter and the receiver are spaced enough far apart and in the
presence of numerous scatters, R(Rc) = I (of size NRc × NRc ), R(Tx) = I (of size NTx × NTx ), and

g0 ∼  (𝟎, I) (4.154)

This is a spatially rich MIMO channel model, often used as mathematical model, but seldom observed in
(r,t)
practice. Lastly, gi(r,t) obtained as in (4.153) can be used as input 𝑤i (lTP ) of NTx NRc signal generators
in correspondence of the i-th tap of the MIMO dispersive channel, for the extended model including
power delay profile and Doppler as detailed in Section 4.1.9.

8
The symbol ⊗ denotes the Kronecker matrix product. If

⎡ a1,1 … a1,M ⎤
A=⎢ ⋮ ⋱ ⋮ ⎥ (4.150)
⎢ ⎥
⎣aN,1 … aN,M ⎦

and B is any matrix, it is

⎡ a1,1 B … a1,M B ⎤
A⊗B=⎢ ⋮ ⋱ ⋮ ⎥ (4.151)
⎢ ⎥
⎣aN,1 B … aN,M B⎦
222 Chapter 4. Transmission channels

Blockage effect
An important effect typically observed when operating at mmWave, is the blockage effect, i.e. some
paths are obstructed by objects. In the discrete-time model, this is taken into account by removing
some clusters having AoD and/or AoA within ranges that are generated randomly [17]. An alternative
simpler blockage model [18] provides that the cluster is in blockage with probability e−𝛽bl d (and not
in blockage with probability 1 − e−𝛽bl d ), where 𝛽bl is a suitable parameter (e.g. 𝛽bl = 0.006), and d is
the transmitter–receiver distance. Then, clusters in blockage have different path loss from those not in
blockage.

4.2 Telephone channel

A telephone channel is the channel between two end users of the telephone service. It was originally
conceived for the transmission of voice, and today is extensively used also for data transmission. The
telephone channel utilizes several transmission media, such as symmetrical transmission lines, coaxial
cables, optical fibres, radio, and satellite links. Therefore, channel characteristics depend on the partic-
ular connection established. As a statistical analysis made in 1983 indicated [19], a telephone channel
is characterized by the following disturbances and distortions.

4.2.1 Distortion
The telephone channel has a linear distortion, where the frequency response Ch (f ) of a telephone chan-
nel can be approximated by a passband filter with band in the range of frequencies from 300 to 3400 Hz.
The attenuation
(𝚊(f ))dB = −20 log10 |Ch (f )| (4.155)
and of the group delay or envelope delay (see (1.83))
1 d
𝜏(f ) = − arg Ch (f ) (4.156)
2𝜋 df
as a function of f is shown in Figure 4.25 for two typical channels. The attenuation and envelope delay
distortion are normalized by the values obtained for f = 1004 Hz and f = 1704 Hz, respectively.
Moreover, the channel is also affected by a non-linear distortion caused by amplifiers and by non-linear
A-law and μ-law converters (see Ref. [20]).
The channel is also affected by frequency offset caused by the use of carriers for frequency up and
downconversion. The relation between the channel input x and output y signals is given in the frequency
domain by {
X(f − foff ), f >0
Y(f ) = (4.157)
X(f + foff ), f <0
Usually, foff ≤ 5 Hz.
Lastly, phase jitter is present, which is a generalization of the frequency offset (see (4.12)).

4.2.2 Noise sources

Quantization noise:
It is introduced by the digital representation of voice signals and is the dominant noise in telephone chan-
nels (see Ref. [20]). For a single quantizer, the signal-to-quantization noise ratio Λq has the behaviour
illustrated in Figure 4.26.
4.2. Telephone channel 223

25

20
Short (< 290 km)
Long (> 1150 km)
15
a( f ) (dB)

10

–5
0 0.5 1 1.5 2 2.5 3 3.5
f (kHz)
(a)

7
Short (< 290 km)
6 Long (> 1150 km)
5
τ(f) (ms)

–1
0 0.5 1 1.5 2 2.5 3 3.5 4
f (kHz)
(b)

Figure 4.25 (a) Attenuation and (b) envelope delay distortion for two typical telephone channels.
224 Chapter 4. Transmission channels

50

45

40

35

30
Λq (dB)

25

20
Input signal
15 Sinusoidal
Laplacian (voice)
10
White noise
5
0
–70 –60 –50 –40 –30 –20 –10 0 10
Input signal power (dBm)

Figure 4.26 Signal to quantization noise ratio as a function of the input signal power for three
different inputs.

Thermal noise:
It is present at a level of 20 ÷ 30 dB below the desired signal.

4.2.3 Echo
It is caused by the mismatched impedances of the hybrid. As illustrated in Figure 4.27, there are two
types of echoes:
1. Talker echo: Part of the signal is reflected and is received at the transmit side. If the echo is not
significantly delayed, then it is practically indistinguishable from the original voice signal, however,
it cannot be neglected for data transmission;
2. Listener echo: If the echo is reflected a second time, it returns to the listener and disturbs the original
signal.
On terrestrial channels, the round-trip delay of echoes is of the order of 10–60 ms, whereas on satellite
links, it may be as large as 600 ms. We note that the effect of echo is similar to multipath fading in radio

talker talker listener

speech path echo echo

Figure 4.27 Three of the many signal paths in a simplified telephone channel with a single
two-to-four wire conversion at each end.
Bibliography 225

systems. However in the telephone channel, the echo signal can be derived directly from the transmitted
signal, while in the radio channel, it can be obtained at the receiver only after detection. Therefore,
while in radio system, equalization solutions must be used (with noise propagation effect), specific
solutions using the noise-less signal available at the transmitter can be used, in the telephone channel
namely:
● echo suppressors that attenuate the unused connection of a four-wire transmission line;
● echo cancellers that cancel the echo at the source (see Section 2.5.5).

Bibliography

[1] Messerschmitt, D.G. and Lee, E.A. (1994). Digital Communication, 2e. Boston, MA: Kluwer
Academic Publishers.
[2] Feher, K. (1995). Wireless Digital Communications. Upper Saddle River, NJ: Prentice-Hall.
[3] Rappaport, T.S. (1996). Wireless Communications: Principles and Practice. Englewood Cliffs, NJ:
Prentice-Hall.
[4] Pahlavan, K. and Levesque, A.H. (1995). Wireless Information Networks. New York, NY: Wiley.
[5] Jakes, W.C. (1993). Microwave Mobile Communications. New York, NY: IEEE Press.
[6] Stuber, G.L. (1996). Principles of Mobile Communication. Norwell, MA: Kluwer Academic Pub-
lishers.
[7] Spilker, J.J. (1977). Digital Communications by Satellite. Englewood Cliffs, NJ: Prentice-Hall.
[8] Parsons, J.D. (2000). The Mobile Radio Propagation Channel. Chichester: Wiley.
[9] Razavi, B. (1997). RF Microelectronics. Englewood Cliffs, NJ: Prentice-Hall.
[10] Someda, C.G. (1998). Electromagnetic Waves. London: Chapman & Hall.
[11] 3GPP (2017). E-UTRA; radio frequency (RF) system scenarios.
[12] Gudmundson, M. (1991). Correlation model for shadow fading in mobile radio systems. Electronics
Letters 27 (1): 2145–2146.
[13] Anastasopoulos, A. and Chugg, K. (1997). An efficient method for simulation of frequency selec-
tive isotropic Rayleigh fading. Proceedings of 1997 IEEE Vehicular Technology Conference, pp.
2084–2088.
[14] Oppenheim, A.V. and Schafer, R.W. (1989). Discrete-Time Signal Processing. Englewood Cliffs,
NJ: Prentice-Hall.
[15] Cariolaro, G. (2011). Unified Signal Theory. Springer.
[16] Zordan, D., Quer, G., Zorzi, M., and Rossi, M. (2011). Modeling and generation of space-time
correlated signals for sensor network fields. Proceedings of IEEE Global Telecommunications
Conference (GLOBECOM), pp. 1–6.
[17] 3GPP. (2018) Study on channel model for frequencies from 0.5 to 100 GHz. 3GPP TR 38.901
version 15.0.0 Release 15.
[18] Bai, T., Desai, V., and Heath, R.W. (2014). Millimeter wave cellular channel models for system
evaluation. Proceedings International Conference on Computing, Networking and Communications
(ICNC), pp. 178–182.
[19] Laboratories, B.T. (1982). Transmission Systems for Communications, 5e. Winston, NC: Bell Tele-
phone Laboratories.
[20] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems.
Wiley.
226 Chapter 4. Transmission channels

Appendix 4.A Discrete-time NB model for mmWave channels

For a transmitter and receiver, both equipped with linear antenna arrays, as in Figure 4.23, let D(Tx)min
(D(Rc)
min
) be the distance between adjacent antennas at the transmitter (receiver).
Let us define the suitable phase shift between adjacent transmit (receiver) antennas for ray i with AoD
(AoA) 𝛼i(Tx) (𝛼i(Rc) ) as
D(Tx)
min
D(Rc)
min
𝜂i(Tx) = sin 𝛼i(Tx) , 𝜂i(Rc) = sin 𝛼i(Rc) (4.158)
𝜆 𝜆
Entry (r, t) of matrix C is
∑
Nc −1
(Rc) (Tx)
C(r,t) = ǧ i e−j2𝜋(r−1)𝜂i e−j2𝜋(t−1)𝜂i (4.159)
i=0

r = 1, … , NRc , t = 1, … , NTx , where ǧ i is a complex gain. When comparing (4.143) and (4.159), we
observe that (4.143) describes fading due to Doppler using a geometric model (through velocities),
while (4.159) describes it using the gain ǧ i . We remark once more that, while (4.159) holds for radio
transmission at any frequency (with Nc very large), it becomes of practical use at mmWave frequencies,
where Nc = 2 or 3.
Other antenna geometries, such as rectangular planar arrays require multiple angles to describe the
departure (arrival) directions of the signals; still, C(r,t) is the sum of Nc exponential products, one for
each angle.

4.A.1 Angular domain representation

The mmWave channel model finds a simple representation in the angular domain (also denoted virtual
channel), i.e. by taking the 2D DFT of the channel matrix. For example, with reference to the case of

1
[Θ]φ1,φ2

0.5

0
120
100 120
80 100
60 80
40 60
20 40
φ1 20 φ2
0

Figure 4.28 Example of mmWave channel in the angular domain.

4.A. Discrete-time NB model for mmWave channels 227

transmit and receive antennas implemented by linear arrays, let us introduce matrix W(𝛀), 𝛀 = (Ω1 , Ω2 ),
with entries
[ ( ) ]
NRc (𝜑1 − Ω1 ) −j𝜋(𝜑1 −Ω1 ) NRcM −1
[W(𝛀)]𝜑1 ,𝜑2 = NRc sincNRc e 1 × (4.160)
M1
[ ( ) ]
NTx (𝜑2 − Ω2 ) −j𝜋(𝜑2 −Ω2 ) NTxM −1
NTx sincNTx e 2
M2
where the periodic function sincN (x) is defined in (1.24). By taking the M1 × M2 DFT2 of (4.159), the
M1 × M2 matrix 𝚯 is obtained, with entries9
∑ ∑
NRc NTx
−
j2𝜋𝜑1 (r−1)
−
j2𝜋𝜑2 (t−1) ∑
Nc −1
[𝚯]𝜑1 ,𝜑2 = C(r,t) e M1
e M2
= ǧ i [W(𝛀(i) )]𝜑1 ,𝜑2 (4.161)
r=1 t=1 i=0

where 𝜑1 = 0, 1, … , M1 − 1 and 𝜑2 = 0, 1, … , M2 − 1, define the angular domain, and 𝛀(i) =

(−M1 𝜂i(Rc) , −M2 𝜂i(Tx) ). For a large number of antennas, the periodic sincN functions that compose 𝚯
become simple sinc functions. In this asymptotic regime, 𝚯 presents Nc peaks at 𝛀(i) , i = 0, … , Nc − 1,
i.e. it is a sparse matrix. This representation can be particularly useful to design efficient channel
estimation and resource allocation techniques. An example of mmWave channel in the angular domain
is shown in Figure 4.28, wherein we can clearly identify a small number of peaks.

9
Rather than using the definition of DFT, it is simpler to first evaluate the discrete-time Fourier transform of (4.159), and
then sampling it at angular frequencies (𝜑1 ∕M1 , 𝜑2 ∕M2 ).
Chapter 5

Vector quantization

Vector quantization (VQ) is introduced as a natural extension of the scalar quantization (SQ) [1] concept.
However, using multidimensional signals opens the way to many techniques and applications that are
not found in the scalar case [2, 3].

5.1 Basic concept

The basic concept is that of associating with an input vector s = [s1 , … , sN ]T , generic sample of a
vector random process s(k), a reproduction vector sq = Q[s] chosen from a finite set of L elements
(code vectors),  = {Q1 , … , QL }, called codebook, so that a given distortion measure d(s, Q[s]) is
minimized.
Figure 5.1 exemplifies the encoder and decoder functions of a VQ scheme. The encoder computes
the distortion associated with the representation of the input vector s by each reproduction vector of 
and decides for the vector Qi of the codebook  that minimizes it; the decoder associates the vector Qi
to the index i received. We note that the information transmitted over the digital channel identifies the
code vector Qi ; therefore, it depends only on the codebook size L and not on N, dimension of the code
vectors.

encoder decoder
1 codebook 1 codebook
. . A . . A
. . . .
. . . .

L L

Ql l ϵ {1,2,...,L}
sq(k) = Ql
S=S (k) i i
compute binary
d(S, Ql) channel

Figure 5.1 Block diagram of a vector quantizer.

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
230 Chapter 5. Vector quantization

An example of input vector s is obtained by considering N samples at a time of a signal,

s(k) = [s(kNTc ), … , s((kN − N + 1)Tc )]T , or the N linear predictive coding (LPC) coefficients (see
Section 2.2), s(k) = [c1 (k), … , cN (k)]T , associated with an observation window of a signal.

5.2 Characterization of VQ

Considering the general case of complex-valued signals, a vector quantizer is characterized by

● Source or input vector s = [s1 , s2 , … , sN ]T ∈ ℂN .

● Codebook  = {Qi }, i = 1, … , L, where Qi ∈ ℂN is a code vector.
● Distortion measure d(s, Qi ).
● Quantization rule (minimum distortion)

Q ∶ ℂN →  with Qi = Q[s] if i = arg min d(s, Q𝓁 ) (5.1)

𝓁
Definition 5.1 (Partition of the source space)
The equivalence relation
Q = {(s1 , s2 ) ∶ Q[s1 ] = Q[s2 ]} (5.2)

which associates input vector pairs having the same reproduction vector, identifies a partition  =
{1 , … , L } of the source space ℂN , whose elements are the sets

𝓁 = {s ∈ ℂN ∶ Q[s] = Q𝓁 } 𝓁 = 1, … , L (5.3)

The sets {𝓁 }, 𝓁 = 1, … , L, are called Voronoi regions.

It can be easily demonstrated that the sets {𝓁 } are non-overlapping and cover the entire space ℂN :

⋃
L
𝓁 = ℂN i ∩ j = ∅ i ≠ j (5.4)
𝓁=1

In other words, as indicated by (5.3) every subset 𝓁 contains all input vectors associated by the quan-
tization rule with the code vector Q𝓁 . An example of partition for N = 2 and L = 4 is illustrated in
Figure 5.2. ◽

R4 R1
C2
Q4
Q1

Q3 Q2

R2

R3

Figure 5.2 Partition of the source space ℂ2 in four subsets or Voronoi regions.
5.2. Characterization of VQ 231

Parameters determining VQ performance

We define the following parameters.

● Quantizer rate
Rq = log2 L (bit/vector) or (bit/symbol) (5.5)

● Rate per dimension

Rq log2 L
RI = = (bit/sample) (5.6)
N N
● Rate in bit/s
RI log2 L
Rb = = (bit/s) (5.7)
Tc NTc
where Tc denotes the time interval between two consecutive samples of a vector. In other words, in
(5.7) NTc is the sampling period of the vector sequence {s(k)}.
● Distortion
d(s, Qi ) (5.8)

The distortion is a non-negative scalar function of a vector variable,

d ∶ ℂN ×  → ℜ+ (5.9)

If the input process s(k) is stationary and the probability density function ps (a) is known, we can
compute the mean distortion as

D(, ) = E[d(s, Q[s])] (5.10)

∑
L
= E[d(s, Qi ), s ∈ i ] (5.11)
i=1

∑L
= di (5.12)
i=1

where
di = d(a, Qi )ps (a) da (5.13)
∫i
If the source is also ergodic, we obtain

1 ∑
K
D(, ) = lim d(s(k), Q[s(k)]) (5.14)
K→∞ K
k=1

In practice, we always assume that the process {s} is stationary and ergodic, and we use the average
distortion (5.14) as an estimate of the expectation (5.10).
Defining
Qi = [Qi,1 , Qi,2 , … , Qi,N ]T (5.15)

we give below two measures of distortion of particular interest.

1. Distortion as the 𝓁𝜈 norm to the 𝜇-th power:

[N ]𝜇∕𝜈
∑
d(s, Qi ) = ‖s − Qi ‖𝜇𝜈 = |sn − Qi,n | 𝜈
(5.16)
n=1
232 Chapter 5. Vector quantization

The most common version is the square distortion1 :

∑
N
d(s, Qi ) = ‖s − Qi ‖22 = |sn − Qi,n |2 (5.17)
n=1

2. Itakura–Saito distortion:
∑ ∑
N N
d(s, Qi ) = (s − Qi )H Rs (s − Qi ) = (sn − Qi,n )∗ [Rs ]n,m (sm − Qi,m ) (5.18)
n=1 m=1

where Rs is the autocorrelation matrix of the vector s∗ (k), defined in (1.270), with entries [Rs ]n,m ,
n, m = 1, 2, … , N.

Comparison between VQ and scalar quantization

̃ = D∕N, for a given rate RI we find (see [4] and refer-
Defining the mean distortion per dimension as D
ences therein)
̃ SQ
D
= F(N) S(N) M(N) (5.19)
̃ VQ
D
where

● F(N) is the space filling gain. In the scalar case, the partition regions must necessarily be intervals.
In an N-dimensional space, Ri can be shaped very closely to a sphere. The asymptotic value for
N → ∞ equals F(∞) = 2𝜋e∕12 = 1.4233 = 1.53 dB.
● S(N) is the gain related to the shape of ps (a), defined as2
∥ p̃ s (a)∥1∕3
S(N) = (5.21)
∥ p̃ s (a)∥N∕(N+2)
where p̃ s (a) is the probability density function of the input s considered with uncorrelated compo-
nents. S(N) does not depend on the variance of the random variables of s, but only on the norm
order N∕(N + 2) and shape p̃ s (a). For N → ∞, we obtain
∥ p̃ s (a)∥1∕3
S(∞) = (5.22)
∥ p̃ s (a)∥1
● M(N) is the memory gain, defined as
∥ p̃ s (a)∥N∕(N+2)
M(N) = (5.23)
∥ ps (a)∥N∕(N+2)
where ps (a) is the probability density function of the input s. The expression of M(N) depends on the
two functions p̃ s (a) and ps (a), which differ for the correlation among the various vector components;
obviously if the components of s are statistically independent, we have M(N) = 1; otherwise M(N)
increases as the correlation increases.

1
Although the same symbol is used, the metric defined by (5.17) is the square of the Euclidean distance between the two
vectors.
2
Extending (5.16) to the continuous case, we obtain
[ ]𝜇∕𝜈
∥ p̃ s (a)∥𝜇∕𝜈 = · · · p̃ 𝜈s (a1 , … , aN ) da1 · · · daN (5.20)
∫ ∫
5.3. Optimum quantization 233

5.3 Optimum quantization

Our objective is to design a vector quantizer, choosing the codebook  and the partition  so that the
mean distortion given by (5.10) is minimized.
Two necessary conditions arise.
Rule A (Optimum partition): Assuming the codebook,  = {Q1 , … , QL } is given, we want to find
the optimum partition  that minimizes D(, ). Observing (5.10), the solution is given by

i = {s ∶ d(s, Qi ) = min d(s, Q𝓁 )}, i = 1, … , L (5.24)

Q𝓁 ∈

As illustrated in Figure 5.3, i contains all the points s nearest to Qi .

Q8
Q1

Q7

Q6 Q2

Q4
Q5

Q3

Figure 5.3 Example of partition for K = 2 and N = 8.

Rule B (Optimum codebook): Assuming the partition  is given, we want to find the optimum code-
book . By minimizing (5.13), we obtain the solution

Qi = arg minN E[d(s, Q), s ∈ i ] (5.25)

Q∈ℂ

In other words, Qi coincides with the centroid of the region i .

As a particular case, choosing the square distortion (5.17), (5.13) becomes

di = ‖s − Qi ‖22 ps (a) da (5.26)

∫i
and (5.25) yields
a ps (a) da
∫ i
Qi = (5.27)
ps (a) da
∫i

Generalized Lloyd algorithm

The generalized Lloyd algorithm, given in Figure 5.4, generates a sequence of suboptimum quantizers
specified by {i } and {Qi } using the previous two rules.
234 Chapter 5. Vector quantization

initialization
given: ps (a), ε > 0, j = 0
A0, D−1 = + ∞

given A = {Q , ... , Q }
j 1 L
j=j+1 determine
R[Aj] = {R1, ..., RL}
via rule A

determine
compute
Aj = {Q1, ... , QL} Dj = D(R [Aj], Aj)
via rule B

Dj−1 − Dj
<ε
Dj
No

Yes

stop

Figure 5.4 Generalized Lloyd algorithm for designing a vector quantizer.

1. Initialization. We choose an initial codebook 0 and a termination criterion based on a relative

error 𝜖 between two successive iterations. The iteration index is denoted by j.
2. Using the rule A, we determine the optimum partition [j ] using the codebook j .
3. We evaluate the distortion associated with the choice of j and [j ] using (5.12).
4. If
Dj−1 − Dj
<𝜖 (5.28)
Dj

we stop the procedure, otherwise we update the value of j.

5. Using rule B, we evaluate the optimum codebook associated to the partition [j−1 ].
6. We go back to step 2.

The solution found is at least locally optimum; nevertheless, given that the number of locally optimum
codes can be rather large, and some of the locally optimum codes may give rather poor performance, it is
often advantageous to provide a good codebook to the algorithm to start with, as well as trying different
initial codebooks.
5.4. The Linde, Buzo, and Gray algorithm 235

The algorithm is clearly a generalization of the Lloyd algorithm: the only difference is that the vec-
tor version begins with a codebook (alphabet) rather than with an initial partition of the input space.
However, the implementation of this algorithm is difficult for the following reasons.

● The algorithm assumes that ps (a) is known. In the scalar quantization, it is possible, in many appli-
cations, to develop an appropriate model of ps (a), but this becomes a more difficult problem with
the increase of the number of dimensions N: in fact the identification of the distribution type, for
example Gaussian or Laplacian, is no longer sufficient, as we also need to characterize the statistical
dependence among the elements of the source vector.
● The computation of the input space partition is much harder for the VQ. In fact, whereas in
the scalar quantization the partition of the real axis is completely specified by a set of (L − 1)
points, in the two-dimensional case the partition is specified by a set of straight lines, and for the
multi-dimensional case to find the optimum solution becomes very hard. For VQ with a large
number of dimensions, the partition becomes also harder to describe geometrically.
● Also in the particular case (5.27), the calculation of the centroid is difficult for the VQ, because it
requires evaluating a multiple integral on the region i .

5.4 The Linde, Buzo, and Gray algorithm

An alternative approach led Linde, Buzo, and Gray (LBG) [5] to consider some very long realizations
of the input signal and to substitute (5.10) with (5.14) for K sufficiently large. The sequence used to
design the VQ is called training sequence (TS) and is composed of K vectors

{s(m)} m = 1, … , K (5.29)

The average distortion is now given by

1 ∑
K
D= d(s(k), Q[s(k)]) (5.30)
K k=1
and the two rules to minimize D become:
Rule A i = {s(k) ∶ d(s(k), Qi ) = min d(s(k), Q𝓁 )}, i = 1, … , L (5.31)
Q𝓁 ∈

that is i is given by all the elements of the TS nearest to Qi .

Rule B 1 ∑
Qi = arg minN d(s(k), Q) (5.32)
Q∈ℂ mi s(k)∈
i

where mi is the number of elements of the TS that are inside i

Using the structure of the Lloyd algorithm with the new cost function (5.30) and the two new rules
(5.31) and (5.32), we arrive at the LBG algorithm. Before discussing the details, it is worthwhile pointing
out some aspects of this new algorithm.

● It converges to a minimum, which is not guaranteed to be a global minimum, and generally depends
on the choice of the TS.
● It does not require any stationarity assumption.
● The partition is determined without requiring the computation of expectations over ℂN .
● The computation of Qi in (5.32) is still burdensome.
236 Chapter 5. Vector quantization

However, for the square distortion (5.17), we have

∑
N
d(s(k), Qi ) = |sn (k) − Qi,n |2 (5.33)
n=1

and (5.32) simply becomes

Rule B
1 ∑
Qi = s(k) (5.34)
mi s(k)∈
i

that is Qi coincides with the arithmetic mean of the TS vectors that are inside i .

Choice of the initial codebook

With respect to the choice of the initial codebook, the first L vectors of the TS can be used; however, if
the data are highly correlated, it is necessary to use L vectors that are sufficiently spaced in time from
each other.
A more effective alternative is that of taking as initial value the centroid of the TS and start with
a codebook with a number of elements L = 1. Slightly changing the components of this code vector
(splitting procedure), we derive two code vectors and an initial alphabet with L = 2; at this point, using
the LBG algorithm, we determine the optimum VQ for L = 2. At convergence, each optimum code
vector is changed to obtain two code vectors and the LBG algorithm is used for L = 4. Iteratively the
splitting procedure and optimization is repeated until the desired number of elements for the codebook
is obtained.
Let j = {Q1 , … , QL } be the codebook at iteration j-th. The splitting procedure generates 2L
N-dimensional vectors yielding the new codebook

j+1 = {−j } ∪ {+j } (5.35)

where
−j = {Qi − 𝜺− }, i = 1, … , L , (5.36)

+j = {Qi − 𝜺+ }, i = 1, … , L (5.37)

Typically, 𝜺− is the zero vector,

𝜺− = 𝟎 (5.38)

and √
1 𝙼s
𝜺+ = ⋅𝟏 (5.39)
10 N
where 𝟏 is an all-one vector, so that
∥ 𝜺+ ∥22 ≤ 0.01 𝙼s (5.40)

where 𝙼s is the power of the TS.

Splitting procedure
Choosing 𝜖 > 0 (typically 𝜖 = 10−3 ) and an initial alphabet given by the splitting procedure applied
to the average of the TS, we obtain the LBG algorithm, whose block diagram is shown in Figure 5.5,
whereas its operations are depicted in Figure 5.6.
5.4. The Linde, Buzo, and Gray algorithm 237

2 0 −1

1 { 1 }

[ ] } 1 }

( [ ] )

{ 1 } 1

0 { } { }
2 0

Figure 5.5 LBG algorithm with splitting procedure.

1
0
( 2)
2

{ ( )} 2
1

Figure 5.6 Operations of the LBG algorithm with splitting procedure.

238 Chapter 5. Vector quantization

Selection of the training sequence

A rather important problem associated with the use of a TS is that of empty cells. It is in fact possible
that some regions i contain few or no elements of the TS: in this case, the code vectors associated with
these regions contribute little or nothing at all to the reduction of the total distortion.
Possible causes of this phenomenon are:

● TS too short: the training sequence must be sufficiently long, so that every region i contains at
least 30–40 vectors;
● Poor choice of the initial alphabet: in this case, in addition to the obvious solution of modifying
this choice, we can limit the problem through the following splitting procedure.

Let i be a region that contains mi < mmin elements. We eliminate the code vector Qi from the code-
book and apply the splitting procedure limited only to the region that gives the largest contribution to
the distortion; then we compute the new partition and proceed in the usual way.
We give some practical rules, taken from [6] for LPC applications,3 that can be useful in the design
of a vector quantizer.

● If K∕L ≤ 30, in practice there is a possibility of empty regions, where we recall K is the number of
vectors of the TS, and L is the number of code vectors.
● If K∕L ≤ 600, an appreciable difference between the distortion calculated with the TS and that
calculated with a new sequence may exist.

In the latter situation, it may in fact happen that, for a very short TS, the distortion computed for
vectors of the TS is very small; the extreme case is obtained by setting K = L, hence D = 0. In this
situation, for a sequence different from the TS (outside TS), the distortion is in general very high. As
illustrated in Figure 5.7, only if K is large enough the TS adequately represents the input process and no
substantial difference appears between the distortion measured with vectors inside or outside TS [5].4

1.5

1 outside training sequence

D

0.5

inside training sequence

0
1 1.5 2 2.5 3 3.5 4 4.5 5
log10 (K )

Figure 5.7 Values of the distortion as a function of the number of vectors K in the inside and
outside training sequences.

3
These rules were derived in the VQ of LPC vectors. They can be considered valid in the case of strongly correlated
vectors.
4
This situation is similar to that obtained by the least squares (LS) method (see Section 3.2).
5.5. Variants of VQ 239

Finally, we find that the LBG algorithm, even though very simple, requires numerous computa-
tions. We consider for example as vector source the LPC coefficients with N = 10, computed over
windows of duration equal to 20 ms of a speech signal sampled at 8 kHz. Taking L = 256, we have a rate
Rb = 8 bit/20 ms equal to 400 bit/s. As a matter of fact, the LBG algorithm requires a minimum K =
600 ⋅ 256 ≃ 155 000 vectors for the TS, which roughly corresponds to three minutes of speech.

5.4.1 k-means clustering

k-means clustering is used in pattern recognition [7] and aggregates a vector sequence into L (in the
literature L is often denoted as k) clusters. This is equivalent to the LBG algorithm of Section 5.4.

5.5 Variants of VQ

Tree search VQ
A random VQ, determined according to the LBG algorithm, requires:
● a large memory to store the codebook;
● a large computational complexity to evaluate the L distances for encoding.
A variant of VQ that requires a lower computational complexity, at the expense of a larger memory,
is the tree search VQ. As illustrated in Figure 5.8, whereas in the memoryless VQ case the comparison
of the input vector s must occur with all the elements of the codebook, thus determining a full search, in
the tree search VQ we proceed by levels: first we compare s with QA1 and QA2 , then we proceed along
the branch whose node has a representative vector closest to s.

( 3 3
8 ) ( 3)

1 1
1
2 2
1
3 3

4 2 4

5 5
3
6 6
2
7 7

8 4
8

1st level 2nd level 3rd level

Figure 5.8 Comparison between full search and tree search.

To determine the code vectors at different nodes, for a binary tree the procedure consists of the fol-
lowing steps.
1. Calculate the optimum quantizer for the first level by the LBG algorithm; the codebook contains
two code vectors.
2. Divide the training sequence into subsequences relative to every node of level n (n = 2, 3, … , NLEV ,
NLEV = log2 L); in other words, collect all vectors that are associated with the same code vector.
3. Apply the LGB algorithm to every subsequence to calculate the codebook of level n.
240 Chapter 5. Vector quantization

Table 5.1: Comparison between full search and tree search.

Computation of d(⋅, ⋅) No. of vectors to memorize

Full search 2 Rq 2 Rq
∑R q
Tree search 2Rq i=1
2i ≃ 2Rq +1

For Rq = 10 (bit/vector) Computation of d(⋅, ⋅) No. of vectors to memorize

Full search 1024 1024
Tree search 20 2046

As an example, the memory requirements and the number of computations of d(⋅, ⋅) are shown in
Table 5.1 for a given value of Rq (bit/vector) in the cases of full search and tree search. Although the
performance is slightly lower, the computational complexity of the encoding scheme for a tree search is
considerably reduced.

Multistage VQ
The multistage VQ technique presents the advantage of reducing both the encoder computational com-
plexity and the memory required. The idea consists in dividing the encoding procedure into successive
stages, where the first stage performs quantization with a codebook with a reduced number of elements.
Successively, the second stage performs quantization of the error vector e = s − Q[s]: the quantized
error gives a more accurate representation of the input vector. A third stage could be used to quantize
the error of the second stage and so on.
We compare the complexity of a one-stage scheme with that of a two-stage scheme, illustrated in
Figure 5.9. Let Rq = log2 L be the rate in bit/vector for both systems and assume that all the code vectors
have the same dimension N = N1 = N2 .

● Two-stage:
Rq = log2 L1 + log2 L2 , hence L1 L2 = L.
Computations of d(⋅, ⋅) for encoding: L1 + L2 .
Memory: L1 + L2 locations.
● One-stage:
Rq = log2 L.
Computations of d(⋅, ⋅) for encoding: L.
Memory: L locations.

The advantage of a multistage approach in terms of cost of implementation is evident; however, it has
lower performance than a one-stage VQ.

Product code VQ
The input vector is split into subvectors that are quantized independently, as illustrated in Figure 5.10.
This technique is useful if (i) there are input vector components that can be encoded separately because
of their different effects, e.g. prediction gain and LPC coefficients, or (ii) the input vector has too large
a dimension to be encoded directly. It presents the disadvantage that it does not consider the correlation
that may exist between the various subvectors, that could bring about a greater coding efficiency.
5.5. Variants of VQ 241

1 1

1 2

1 2

1
2

Figure 5.9 Multistage (two-stage) VQ.

1
codebook
1

(1) (1)
1 (1)

(1) (2) (1) (2)

( ) ( )
(2)
(2)
2
(2)

Figure 5.10 Product code VQ.

A more general approach is the sequential search product code [2], where the quantization of the
subvector n depends also on the quantization of previous subvectors. With reference to Figure 5.10,
assuming L = L1 L2 and N = N1 + N2 , we note that the rate per dimension for the VQ is given by

log2 L log2 L1 log2 L2

Rq = = + (5.41)
N N N
242 Chapter 5. Vector quantization

whereas for the product code VQ, it is given by

log2 L1 log2 N2
Rq = + (5.42)
N1 N2

5.6 VQ of channel state information

We present now two examples of use of VQ, where the vectors to be quantized represent sampled fre-
quency responses of the channel between two devices, also denoted as channel state information (CSI).
In particular, we consider a cellular system, with a base station (BS) aiming at transmitting data to user
equipments (UEs) using multicarrier (MC) transmission (see Chapter 8), where the discrete frequency
response of the channel is a vector of  complex numbers, one for each subcarrier.
As described in details in Section 13.1.2, power and bit adaptation on a per-subcarrier basis greatly
increases the achievable data rate, but it requires the knowledge of CSI before transmission. In time
division multiplexing (TDD) systems (see Appendix 18.A), where uplink and downlink transmissions
occur in the same band, although at different times, the BS estimates the uplink channel, and the down-
link channel is the same (by reciprocity), within a short time interval. However, when frequency division
duplexing (FDD) (see Appendix 18.A) is used, uplink and downlink transmissions occur in two separate
bands, thus the BS cannot exploit the channel reciprocity. Indeed, the UE must (i) estimate the downlink
CSI, (ii) quantize the CSI, and (iii) feedback the quantization index to the BS. Therefore, VQ is needed.
When the BS is equipped with NTx > 1 antennas (obtaining a multiple-input single-output, MISO
system, see also Chapter 9), the UE must estimate the NTx channel vectors and stack them into a single
vector, to be quantized. Clearly, this problem is even more challenging, as the vector size grows with
NTx , and a larger codebook is needed. In this case, CSI availability at the transmitter is relevant also to
exploit space diversity by precoding (see Section 9.3.2), or to implement spatial multiuser transmission
(see Section 9.6).
We will first consider the problem of MISO channel quantization, investigating the more appropriate
VQ distortion measure. Then, we consider a more elaborate CSI reporting scheme, where we exploit the
correlation between uplink and downlink channels (although being still in two separate bands).

MISO channel quantization

First consider the quantization of channel estimates in MISO orthogonal frequency division multiplexing
(OFDM) systems with  subcarriers, modelling the downlink of a cellular system with a BS equipped
with NTx > 1 antennas and single-antenna UEs. Let C [m] be the 1 × NTx (space) channel vector at
subcarrier m, m = 0, … ,  − 1, and let

C = [C [0], … , C [ − 1]]T (5.43)

be the overall NTx channel vector.

For CSI feedback, the solution adopted for example in the long-term evolution (LTE) of the 3GPP
cellular systems [8] provides that the UE feeds back: the channel quality indicator (CQI), and the channel
direction indicator (CDI). The CQI is a quantized version of ||C [m]||, m = 0, … ,  − 1, i.e. the norm
of the space channel vector at subcarrier m. Next, we define the normalized channel vectors and their
collection as
C [m]
C [m] = , s = [C [0], … , C [ − 1]]T (5.44)
||C [m]||
5.6. VQ of channel state information 243

The CDI is a quantized version Qi = [Qi [0], … , Qi [ − 1]] of s, where Qi [m], m = 0, … ,  − 1,

are 1 × NTx -size vectors. Using the MISO linear precoding B[m] = QH i [m] the signal-to-noise-plus-
interference ratio (SNIR) at the detector input is [9]
𝛾MF ||C [m]||2 |C [m]QH
i [m]|
2
𝛾[m] = (5.45)
1 + 𝛾MF ||C [m]||2 (1 − |C [m]QH 2
i [m]| )
where 𝛾MF is the signal-to-noise-ratio (SNR) at the detection point for an ideal additive white Gaussian
noise (AWGN) channel. Differently from (9.109), note that the denominator of 𝛾[m] accounts also for
the contribution of multiuser interference.
The novelty here is that on quantizing the channel we aim at maximizing the sum-rate of data trans-
mission, i.e.
−1
∑
R(s, Qi ) = log2 (1 + 𝛾[m]) (5.46)
m=0
Thus, for VQ we use the rate-based distortion measure
d(s, Qi ) = −R(s, Qi ) (5.47)
However, using optimum quantization with this distortion measure is rather complicated, therefore an
approximate simplified expression is derived [9].
Now, we can exploit the Jensen inequality5 on log2 (1 + x) to obtain the following upper bound on
R(s, Qi ) ( )
−1
1 ∑
R(s, Qi ) ≤  log2 1 + 𝛾[m] (5.48)
 m=0
Then, by exploiting the fact that log2 (1 + x) is strictly monotonically increasing, minimizing d(s, Qi ) is
equivalent to minimizing
−1
∑
d′ (s, Qi ) = − 𝛾[m] (5.49)
m=0
When the multiuser interference is negligible with respect to the channel noise, the expectation of (5.49)
yields [−1 ] −1
∑ ∑
′
E[d (s, Qi )] = −E 𝛾[m] = −𝛾MF E [||C [m]||2 ] E[|C [m]QH 2
i [m]| ] (5.50)
m=0 m=0

where we have exploited the independence between the norm and the direction of the MISO channel.
Therefore, the maximization of the expectation in (5.50) is achieved by the new distortion measure
−1
∑
d′′ (s, Qi ) = − |C [m]QH
i [m]|
2
(5.51)
m=0
When multiuser interference becomes predominant, it can be shown [9] that again using (5.51) as dis-
tortion measure is reasonable.
A further simplified approach is obtained by ignoring the correlation among the channels at different
frequencies and applying a vector quantizer on each C [m], separately, using the following inner product
distortion measure
d(C [m], Qi [m]) = −|C [m]QH i [m]|
2
(5.52)
Note that since C [m] has unit norm, Qi [m] will have unit norm, and minimizing (5.52) is equivalent to
minimizing the square distortion (5.17) between C [m] and Qi [m].
5
For a strictly concave function h, E[h(x)] < h(E[x]).
244 Chapter 5. Vector quantization

Channel feedback with feedforward information

We consider another scenario, wherein both the BS and the UE are equipped with a single antenna (NTx =
NRc = 1), but in order to reduce the feedback rate (thus, the number of bits used to describe the down-
link channel), we exploit the correlation between the uplink and downlink channels. Consider two MC
systems6 for both uplink and downlink, with  subcarriers each, and let (U)
C
= [(U)
C
[0], … , (U)
C
[ −
(D) (D) (D)
1]]T and C = [C [0], … , C [ − 1]]T , be the uplink and downlink channel vectors, respectively.
The standard procedure provides the feedback of CQI and CDI, as described in the previous part of
this section (for NTx = 1 in this case). We now consider a modified procedure [10], wherein both BS and
UE obtain a quantized version of the channel, and then share their knowledge. The procedure works as
follows:

1. The BS transmits bF feedforward bits to the UE representing the index c(F) of vector-quantized
uplink channel.
2. The UE, making use of the quantized uplink channel, transmits bB feedback bits to the BS repre-
senting the index c(B) of vector-quantized downlink channel.
3. The BS uses both the quantized uplink channel and the feedback bits of step 2 to reconstruct the
downlink channel as ̂ C .
(D)

Note that when bF = 0, i.e. no feedforward bits are transmitted, we find the CDI feedback described
in the previous part of this section.
For step 1, the codebook of size 2bF for the uplink channel is constructed over the statistics of (U) C
,
using as distortion measure, for example (5.52). Let (U) = {(U) 𝓁
} be the partition of the uplink channel
vector quantizer.
For step 2 instead we need 2bF different codebooks 𝓁(D) , 𝓁 = 1, … , 2bF , each designed on the statistics
of the downlink channel, conditioned to the fact that the uplink channel belongs to region (U) 𝓁
. Note that
we are exploiting the correlation of uplink and downlink channels, and also in this case we can use the
inner-product distortion. Let (D)c(F)
= {(D)
c(F) ,i
}, i = 1, … , 2bB , be the partition of the downlink channel
vector quantizer conditioned to the fact that the uplink channel belongs to the region (U)
c(F)
.
Finally, in step 3, the BS will put together the dependent and independent components to obtain a
complete estimate of the downlink channel, i.e.

̂ C = E[(D)
(D)
C
|(D)
C
∈ (D)
c(F) ,c(B)
, (U)
C
∈ (U)
c(F)
] (5.53)

An enhancement of the procure is obtained by observing the BS perfectly knows the uplink channel
(U)
C
, therefore (5.53) can be replaced by

̂ C = E[(D)
(D)
C
|(D)
C
∈ (D)
c(F) ,c(B)
, (U)
C
] (5.54)

5.7 Principal component analysis

Given a set of K vectors, {s(k)}, k = 1, … , K, with s(k) ∈  N , we encode s(k) into the corresponding
codevector r(k), i.e.
r(k) = enc(s(k)) (5.55)

6
In LTE systems, in the downlink orthogonal frequency division multiple access (OFDMA) is used, while in uplink
single-carrier frequency division multiple access (SC-FDMA) is used.
5.7. Principal component analysis 245

If r(k) ∈  M with M < N, a sort of compression of the sequence {s(k)} is established, in the sense
that we can represent {r(k)} by fewer elements than the original sequence {s(k)}. Decoding implies a
reconstruction
ŝ (k) = dec(r(k)) (5.56)
with ŝ (k) ≃ s(k). The simplest decoding function is given by matrix B, such that BH has orthonormal
columns, and
ŝ (k) = BH r(k) with BH N × M (5.57)
under constraint
BBH = I (5.58)
where I is the M × M identity matrix. The encoder is
r(k) = Bs(k) (5.59)
Hence,
ŝ (k) = BH Bs(k)) (5.60)
H
Selection of columns of B is by the square of the Euclidean distance (5.17)
∑
K
Bopt = arg min ||s(k) − BH Bs(k)||2 (5.61)
B
k=1

under constraint (5.58).

Before giving the solution, we introduce some matrices. We define the data matrix
S = [s(1), … , s(K)], N×K (5.62)
and its covariance matrix (see also (2.102))
∑
K
Φs = s(k)sH (k) = SSH , N×N (5.63)
k=1

In (5.63), we assume {s(k)} has zero mean, otherwise we subtract from s(k) the estimated mean
1 ∑K
K k=1 s(k). Next, we perform the decomposition of Φs by determining eigenvalues {𝜆1 , 𝜆2 , …}, and
corresponding eigenvectors {u1 , u2 , …}, ordered by decreasing values. It is (see (1.292))
Φs = U𝚲UH (5.64)
with
UH U = I (5.65)
where I is the N × N identity matrix and
𝚲 = diag [[𝜆1 , … , 𝜆N ]T ], U = [u1 , … , uN ], N×N (5.66)
Finally, the optimum solution of BH is given by the first M eigenvectors {ui }, i = 1, … , M,7

opt = [u1 , … , uM ],
BH N×M (5.67)
or
1 ⎤
⎡ uH
Bopt =⎢ ⋮ ⎥ (5.68)
⎢ H⎥
⎣ uM ⎦

7
The solution is determined iteratively for increasing values of M, starting with M = 1.
246 Chapter 5. Vector quantization

An important property of the principal component analysis (PCA) is that the covariance matrix of r(k)
is diagonal, the components of r(k) for B = Bopt are uncorrelated. In fact, let

r(ext) (k) = UH s(k), N×N (5.69)

where r(ext) (k) = [rT (k) 𝟎T ]T , i.e. we are interested only on the first M components of r(ext) (k), and set
to zero the remaining (N − M) components. The covariance matrix of

R(ext) = [r(ext) (1), … , r(ext) (K)] = UH S (5.70)

is
Φr(ext) = R(ext) R(ext)H = UH SSH U (5.71)

Using (5.64) and (5.65), it is

Φr(ext) = UH U𝚲UH U (5.72)

which from constraint (5.65) yields

Φr(ext) = 𝚲 (5.73)

a diagonal matrix. Entries of r(ext) (k) are called principal components and are obtained by projecting the
data vector s(k) onto matrix UH . Using the singular value decomposition (SVD) of S (see (2.139))
∑
N
√
S = U𝚲1∕2 V H = 𝜆i ui 𝒗H
i (5.74)
i=1

where U is given in from (5.63). From (5.70), it is also

√
⎡ 𝜆1 𝒗 H ⎤
⎢ 1
⎥
⎢ ⋮ ⎥
R(ext)
= √ (5.75)
⎢ 𝜆M 𝒗 HM
⎥
⎢ 𝟎 ⎥
⎣ ⎦

5.7.1 PCA and k-means clustering

The PCA unsupervised dimension reduction is used in very broad areas of science such as meteorology,
image processing, genomic analysis, and information retrieval. It is also common that PCA is used to
project data to a lower-dimensional subspace and k-means is then applied in the subspace. However,
this space reduction property alone is inadequate to explain the effectiveness of PCA. It turns out that
principal components are actually the continuous solution of the cluster membership indicators in the
k-means clustering method, i.e. the PCA dimension reduction automatically performs data clustering
according to the k-means objective function [11]. This provides an important justification of PCA-based
data reduction.
The notation here is that of Section 5.4. We recall the average distortion (5.30) for a number of clusters
L that we write as
∑L
1 ∑
DL = ||s − Qi ||2 (5.76)
i=1
mi s∈
i

If Qi is the centroid of cluster i (see (5.32))

1 ∑
Qi = s (5.77)
mi s∈
i
5.7. Principal component analysis 247

it minimizes the general term in (5.76)

1 ∑ 1 ∑
||s − Qi ||2 = ||s||2 + ||Qi ||2 − sH Qi − sQH
i (5.78)
mi s∈ mi s∈
i i

which can be written as

1 ∑ ∑ 1 ∑ ∑
2
||si ||2 − sH
i sj = ||si − sj ||2 (5.79)
mi si ∈i sj ∈j 2m2i si ∈i sj ∈j

We now consider the case for L = 2 and introduce the average distortion (square distance) between
elements of clusters 1 and 2
1 ∑ ∑
D(1 , 2 ) = ||s − s2 ||2 (5.80)
m1 m2 s ∈ s ∈ 1
1 1 2 2

Moreover, let
m1 m2
JD = [2D(1 , 2 ) − D(1 , 1 ) − D(2 , 2 )] (5.81)
2K
and
1 ∑
K
𝙼s = ||s(k)||2 (5.82)
K k=1
it turns out that by substituting (5.79) into (5.76), we can write

DL = 𝙼s − JD (5.83)

Therefore, for L = 2, finding the minimum of the k-means cluster objective function DL is equivalent
to finding the maximum of the distance objective JD , which can be shown to be always positive. In
JD , the first term represents average distances between clusters that are maximized, forcing the resulting
clusters as separated as possible. Furthermore, the second and third terms represent the average distances
within clusters that will be minimized, forcing the resulting clusters as compact as possible. Finally, the
presence of the factor m1 m2 favours a balanced number of elements in the clusters. These results lead
to a solution of k-means clustering via PCA, as stated by the following theorem, which we give without
proof.

Theorem 5.1
We consider real-values signals and M = L − 1. For k-means clustering with L = 2, the continuous solu-
tion of the cluster indicator vector is the principal component 𝒗1 , i.e. the clusters are given by

1 = {s(k)|(𝒗1 )k ≤ 0} (5.84)
2 = {s(k)|(𝒗1 )k > 0} (5.85)

Further, the optimal value of the k-means objective satisfies the bounds

𝙼s − 𝜆1 < DL=2 < 𝙼s (5.86)

◽
The result for L = 2 can be generalized to the case L > 2, using L − 1 cluster indicator vectors by a
suitable representation [11].

Theorem 5.2
When optimizing the k-means objective function, the continuous solutions for the transformed discrete
cluster membership indicator vectors are the L − 1 principal components {𝒗1 , … , 𝒗L−1 }. Furthermore,
248 Chapter 5. Vector quantization

the optimal value of the k-means objective satisfies the bounds

∑
L−1
𝙼s − 𝜆i < DL < 𝙼s (5.87)
i=1 ◽

Principal components can be viewed as approximate cluster membership indicators in k-means clus-
tering. The two views of PCA are consistent, as data clustering is also a form of data reduction. Standard
data reduction is obtained in Euclidean space by applying SVD, whereas clustering is obtained as data
reduction in classification space. In fact, in Section 5.4, it is shown how the high-dimensional space
of signal feature vectors is divided into Voronoi cells by vector quantization. Signal feature vectors are
approximated by the cluster centroids represented by the code-vectors. PCA plays a crucial role in both
types of data reduction.

Bibliography

[1] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[2] Gersho, A. and Gray, R.M. (1992). Vector Quantization and Signal Compression. Boston, MA:
Kluwer Academic Publishers.
[3] Gray, R.M. (1984). Vector quantization. IEEE ASSP Magazine 1: 4–29.
[4] Lookabaugh, T.D. and Gray, R.M. (1989). High–resolution quantization theory and the vector quan-
tizer advantage. IEEE Transactions on Information Theory 35: 1020–1033.
[5] Linde, Y., Buzo, A., and Gray, R.M. (1980). An algorithm for vector quantizer design. IEEE Trans-
actions on Communications 28: 84–95.
[6] Sereno, D. and Valocchi, P. (1996). Codifica numerica del segnale audio. Scuola Superiore G.
Reiss Romoli: L’Aquila.
[7] MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations. In:
Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability: Statistics,
vol. 1, 281–297. Berkeley, CA: University of California Press.
[8] 3GPP Technical Specification Group Radio Access Network (2010). Physical layer procedures
(FDD) (release 8).
[9] Trivellato, M., Tomasin, S., and Benvenuto, N. (2009). On channel quantization and feedback
strategies for multiuser MIMO-OFDM downlink systems. IEEE Transactions on Communications
57: 2645–2654.
[10] Zhang, X., Centenaro, M., Tomasin, S. et al. (2019). A study on CSI feedback schemes exploiting
feedforward information in FDD cellular systems. Transactions on Emerging Telecommunications
Technologies. e3628. https://doi.org/10.1002/ett.3628.
[11] Ding, C. and He, X. (2004). K-means clustering via principal component analysis. Proceedings 21st
International Conference on Machine Learning, ICML ’04, 29-2-. New York, NY: ACM.
Chapter 6

Digital transmission model and

channel capacity

In this chapter, we define a general model of a digital transmission system, where data must be
transmitted reliably to a receiver using an analog transmission medium. We provide a description of
the main system parameters. We then focus on detection, a key operation performed at the receiver to
properly recover the data. Relevant performance metrics (error probability, capacity, and achievable
rates) are introduced and discussed for various channel models and modulation systems.

6.1 Digital transmission model

With reference to Figure 6.1, we discuss some common aspects of a communication systems.

Source. We assume that a source generates a message {b𝓁 } composed of a sequence of binary symbols
b𝓁 ∈ {0, 1}, that are emitted every Tb seconds:
{b𝓁 } = {… , b−1 , b0 , b1 , b2 , …} (6.1)
Usually, {b𝓁 } is a sequence of i.i.d. symbols.
The system bit rate, which is associated with the message {b𝓁 }, is equal to
1
Rb = [bit/s] (6.2)
Tb

Channel encoder. In order to detect and/or correct errors introduced by the transmission over the
channel, a mapping of information message {b𝓁 } into a coded message1 {cm } is introduced, according to
rules investigated in Chapter 11. Here the code bit is cm ∈ {0, 1}, and the period between two consecutive
code bits is Tcod , with Tcod < Tb .

Bit interleaver (or scrambler). In order to randomize the positions of the errors introduced by the
channel, sometimes a permutation of bits {cm } is performed. The simplest scheme provides that, given
an M1 × M2 matrix and writing in code bits {cm } row-wise, they are read out column-wise. We denote
1
We distinguish three types of coding: (i) source or entropy coding, (ii) channel coding, and (iii) line coding. Their
objectives are respectively: (i) compress the digital message by lowering the bit rate without losing the original signal
information (see Ref. [1]), (ii) increase the reliability of the transmission by inserting redundancy in the transmitted
message, so that errors can be detected and/or corrected at the receiver (see Chapter 11), and (iii) shape the spectrum of
the transmitted signal (see Appendix 7.C).

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
250 Chapter 6. Digital transmission model and channel capacity

binary channel
channel
encoder bit-mapper modulator
bl cm c′m ak s(t)
source COD interleaver BMAP MOD
Tb Tcod T

transmission
binary channel channel
inverse
decoder bit-mapper ^ demodulator
b^ l c~m ~
c′m ak r(t)
user DEC deinterleaver IBMAP DEMOD
Tb Tcod T

Figure 6.1 Simplified model of a transmission system.

the new sequence {c′m }. At the receiver, a deinteleaver writes in the detected bits {̃c′m } column-wise and
reads them out row-wise, to yield c̃ m . As a result, an error burst of length iM1 is broken into shorten
bursts of length i. In the following analysis, to simplify notation, the interleaver is omitted. Note that
originally the term scrambler indicated an interleaver designed to avoid long sequences of bits with the
same value.

Bit mapper. The bit mapper (BMAP) uses a one-to-one map to associate a multilevel symbol ak , emit-
ted every T seconds (symbol period or modulation interval), to an input bit pattern, i.e. a collection of
bits {cm }. 1∕T is the modulation rate or symbol rate of the system and is measured in Baud: it indicates
the number of symbols per second that are transmitted. In the following, we describe the BMAP for
baseband (PAM) and passband (QAM and PSK) modulations, while further examples will be provided
in Chapter 16.

Bit mapper (PAM). Let us consider for example symbols {ak } from a quaternary pulse amplitude
modulation (PAM) alphabet, ak ∈  = {−3, −1, 1, 3}. To select the values of ak , we consider pairs of
input bits and map them into quaternary symbols as indicated in Table 6.1. Note that this correspondence
is one-to-one. Usually, Gray labelling or mapping is used (see Appendix 6.A), where binary labelling
associated to adjacent symbol values differ only in one bit.
For quaternary transmission, the symbol period T is given by T = 2Tcod . In general, if the values of
ak belong to an alphabet  with M elements, then

T = Tcod log2 M (6.3)

Table 6.1: Example

of quaternary bit map
(Gray labelling).

c2k c2k−1 ak
0 0 −3
0 1 −1
1 1 1
1 0 3
6.1. Digital transmission model 251

Moreover, the alphabet is

 = {−(M − 1), … , −1, 1, … , (M − 1)} (6.4)
i.e. symbol values are equally spaced around zero, with distance dmin = 2 between adjacent values.

Bit mapper (QAM). For quadrature amplitude modulation (QAM) systems, values of ak are complex,
and the bit mapper associates a complex-valued symbol to an input bit pattern. Figure 6.2 shows two
examples of constellations and corresponding binary labelling, where
ak = ak,I + jak,Q (6.5)
and ak,I = Re [ak ] and ak,Q = I m [ak ]. Values of ak,I and ak,Q are from an alphabet similar to (6.4).
For the 4-phase shift keying (4-PSK) (or quadrature phase shift keying, QPSK), symbols are taken
from an alphabet with four elements, each identified by 2 bits. Similarly each element in a 16-QAM
constellation is uniquely identified by 4 bits. Gray labelling of values of ak is obtained by Gray labelling
values of ak,I and ak,Q , independently. In Figure 6.2, the first (second) 2 bits refer to values of ak,I (ak,Q ).

ak,Q
(0010) (0110) (1110) (1010)
3

(0011) (0111) (1111) (1011)

ak,Q 1
–3 –1 1 3
1 ak,I
(01) (11) (0001) (0101) (1101) (1001)
–1 1 –1
ak,I
–1 (0000) (0100) (1100) (1000)
(00) (10)
–3

(a) (b)

Figure 6.2 Two rectangular QAM constellations and corresponding bit map (Gray labelling).
(a) QPSK, (b) 16-QAM.

In general, for a rectangular constellation, each value of ak is identified by 2𝚖 = log2 M bits by

BMAP
ak ←−→(ck2𝚖 , … , ck2𝚖−2𝚖+1 ) (6.6)
or by
BMAP
(ak,I ; ak,Q ) ←−→(ck𝚖,I , … , ck𝚖−𝚖+1,I ; ck𝚖,Q , … , ck𝚖−𝚖+1,Q ) (6.7)

Bit mapper (PSK). Phase-shift keying is a particular example of QAM with

ak = ej𝜗k (6.8)
where 𝜗k ∈ {𝜋∕M, 3𝜋∕M, … , (2M − 1)𝜋∕M} or equivalently if 𝛼n , n = 1, … M, are the values of ak ,
and 𝜑n are the values of 𝜗k , it is 2
𝛼n = ej𝜑n (6.9)
𝜋
2
A more general definition of 𝜑n is given by 𝜑n = M
(2n − 1) + 𝜑0 , where 𝜑0 is a constant phase.
252 Chapter 6. Digital transmission model and channel capacity

and
𝜋
𝜑n = (2n − 1), n = 1, … , M (6.10)
M
i.e. symbol values lie equispaced on a circle.
The minimum distance between symbol values is
𝜋
dmin = 2 sin (6.11)
M
PSK is rarely used for M > 8, as dmin gets quite small.

ak,Q
010 1 110

011 111

ak,I
1

001 101

000 100

Figure 6.3 8-PSK constellation and decision regions.

An example of M-PSK constellation with M = 8 is illustrated in Figure 6.3, where Gray labelling
is used.

Modulator. The basic principle of a digital modulator is to map each symbol ak , generated every T
seconds, to a property of a given pulse, e.g. its amplitude. The digital modulated signal requires an
up-frequency conversion for PSK and QAM, where the signal is allocated in the pass band around a
certain carrier frequency. The corresponding analog modulated signal s is transmitted over the channel.
Two relevant examples of modulation systems are discussed in Chapters 7 and 8.

Transmission channel. The channel may (i) distort the transmitted signal s to yield the received useful
signal sCh and (ii) introduce noise 𝑤 modelled as (real value) additive white Gaussian noise (AWGN)
with power spectral density (PSD) N0 ∕2. Hence, the received signal is
r(t) = sCh (t) + 𝑤(t) (6.12)
For a more detailed description of the transmission channel, see Chapter 4. In particular, for an ideal
AWGN channel sCh (t) = s(t).

Receiver. Given an observation of r, the final task of the receiver is to recover the information bits {b𝓁 }
with as few errors as possible. In order to accomplish this task with a feasible complexity, the receiver
performs demodulation (DEMOD) (including down-frequency conversion for PSK and QAM, filtering
6.2. Detection 253

to remove noise and/or shape the received signal, equalization to shape the received signal and remove
inter-symbol interference (ISI), mainly in the digital domain) to recover the transmitted data, which will
be denoted as {̂ak }. Inverse bit-mapping (IBMAP) yields bits {̃cm } (or {̃c′m } if a de-interleaver is used).
Next, the decoder (DEC) for an input {̃cm } will generate the code sequence {̂cm } with corresponding
decoded (or detected) information message {b̂ 𝓁 } (see Chapter 11). Sometimes, some of these operations
are performed jointly, and sometimes they are iterated.

6.2 Detection

At the receiver, the demodulator block includes operations in the analog and digital domain that strongly
depend on the specific channel and modulation technique. In general, after analog filtering, a sampler
working at T or T∕2 is usually employed. Next, an equalizer tries to compensate the distortion introduced
by the channel, taking into account also the noise. The output of these operations is signal zk , with
sampling period T, whose simplest model is given by ak plus noise and/or interference, i.e.
zk = ak + 𝑤k , ak ∈  (6.13)
If 𝜎I2 is the noise variance per dimension, it is
{
 (0, 𝜎I2 ) PAM
𝑤k ∼ (6.14)
 (0, 2𝜎I2 ) QAM, PSK
Note that in (6.13) the equivalent channel gain between the modulator input and the equalizer output
is equal to 1. Furthermore, we assume {𝑤k } to be a sequence of independent random variables. The
following derivation will be for complex-valued (i.e. QAM and PSK) symbols, where also zk is complex.
Let the received observed sample corresponding to zk be 𝜌k , i.e.
zk = 𝜌k , 𝜌k ∈ ℂ (6.15)
For PAM, we just consider the real part of (6.15).
By reflecting the operations done at the transmitter, the detector should output the hard symbols
â k ∈ , corresponding to ak . Next, IBMAP yields bits c̃ m , and the decoder aims at removing errors
introduced by the channel, to yield b̂ 𝓁 . This approach is denoted as hard detection, where the decoder is
fed with hard detected bits c̃ m .
Another detection/decoding strategy provides a first elaboration of zk to obtain a statistical a posteriori
description of the encoded bits, avoiding the two steps of hard detection and IBMAP. It turns out that
suitably designed decoders yield better performance (i.e. typically correct more errors) by exploiting
this statistical description. This approach is denoted as soft detection, where the decoder is fed with soft
detected bits (i.e. the statistics).

6.2.1 Optimum detection

Let M be the cardinality of alphabet . We divide the space ℂ of the received sample zk , into M
non-overlapping regions ⋃
𝛼 𝛼 ∈  with 𝛼 = ℂ (6.16)
𝛼∈

such that, if 𝜌k belongs to 𝛼 , then the symbol 𝛼 is detected:

if 𝜌k ∈ 𝛼 =⇒ â k = 𝛼 (6.17)
254 Chapter 6. Digital transmission model and channel capacity

The probability of correct decision is computed as follows:

P[𝙲] = P[̂ak = ak ]
∑
= P[̂ak = 𝛼 ∣ ak = 𝛼]P[ak = 𝛼]
𝛼∈
∑ (6.18)
= P[zk ∈ 𝛼 ∣ ak = 𝛼]P[ak = 𝛼]
𝛼∈
∑
= ∫ pz∣a (𝜌k ∣ 𝛼)d𝜌k P[ak = 𝛼]
𝛼
𝛼∈
Two criteria are typically adopted to maximize (6.18) with respect to {𝛼 }:
1. Maximum a posteriori probability (MAP) detection:
𝜌k ∈ 𝛼opt (and â k = 𝛼opt ) if 𝛼opt = arg max pz∣a (𝜌k ∣ 𝛼)P[ak = 𝛼] (6.19)
𝛼∈
where argmax is defined in page 106. Using the identity
P[ak = 𝛼 ∣ zk = 𝜌k ]
pz∣a (𝜌k ∣ 𝛼) = pz (𝜌k ) (6.20)
P[ak = 𝛼]
the decision criterion becomes:
â k = arg max P[ak = 𝛼 ∣ zk = 𝜌k ] (6.21)
𝛼∈
2. Maximum-likelihood (ML) detection: If all symbol values are equally likely, the MAP criterion
becomes the ML criterion
â k = arg max pz∣a (𝜌k ∣ 𝛼) (6.22)
𝛼∈
In other words, the symbol 𝛼 is chosen, for which the probability to observe zk = 𝜌k is maximum.
We now present computationally efficient methods for both ML and MAP criteria.

ML
As 𝑤k is a complex-valued Gaussian r.v. with zero mean and variance 2𝜎I2 , it follows (see (1.307))
1
1 − 2 |𝜌k −𝛼|2
pz∣a (𝜌k ∣ 𝛼) = e 2𝜎
I (6.23)
𝜋2𝜎I2
Taking the logarithm, which is a monotonic increasing function, of both members we get
− ln pz∣a (𝜌k ∣ 𝛼) ∝ |𝜌k − 𝛼|2 (6.24)
where non-essential constant terms have been neglected.
Then the ML criterion (6.22) is formulated as
â k = arg min |𝜌k − 𝛼|2 (6.25)
𝛼∈
which corresponds to the minimum-distance detector, as it suggests to detect symbol 𝛼 of the constella-
tion that is closest to the observed sample 𝜌k . If M is very large, it is simpler to use the decision regions,
where to each region a symbol value corresponds. Two examples are reported in Figure 6.4. Detection
(6.25) is also denoted hard detection.

MAP
We introduce the likelihood function, associated to ak as
𝙻ak (𝛼) = P [ak = 𝛼 ∣ zk = 𝜌k ], 𝛼∈ (6.26)
6.2. Detection 255

zk,Q

zk,Q 1
–3 –1 1 3
1 zk,I
–1 1 –1
zk,I
–1
–3

(a) (b)

Figure 6.4 Decision regions for (a) QPSK and (b) 16-QAM constellations.

Then, (6.21) becomes

â k = arg max 𝙻ak (𝛼) (6.27)
𝛼∈

Here for 𝛼 ∈ ,
𝙻ak (𝛼) = P[ak = 𝛼, zk = 𝜌k ]∕P[zk = 𝜌k ]
−
|𝜌k −𝛼|2 (6.28)
2𝜎 2
= Kk e I

where Kk does not depend on 𝛼. Note that (6.28) holds for both real-valued and complex-valued signals
if 𝜎I2 is the noise variance per dimension. As 𝙻ak (𝛼) is a probability, (6.28) can be written as
− 1 |𝜌k −𝛼|2
2𝜎 2
I
e
𝙻ak (𝛼) = , 𝛼∈ (6.29)
∑ − 1
2𝜎 2
|𝜌k −𝜂|2
𝜂∈ e
I

It is easier working with the log-likelihood (LL) function by taking the logarithm of 𝙻ak (𝛼)
|𝜌k − 𝛼|2
𝓁ak (𝛼) = ln 𝙻ak (𝛼) = − (6.30)
2𝜎I2
and (6.21) becomes
â k = arg min |𝜌k − 𝛼|2 (6.31)
𝛼∈

which corresponds again to the minimum-distance detector.

Hard detection (decoder with hard input). In summary, by using symbol decisions, for example based
on the minimum-distance detector (6.25), once â k has been chosen, the corresponding inverse bit-map
yields the bits c̃ m ∈ {0, 1}, as from (6.6). The sequence {̃cm } will form the input to the decoder, which
in turn provides the decoded (or detected) information bits {b̂ 𝓁 }.
256 Chapter 6. Digital transmission model and channel capacity

6.2.2 Soft detection

In the binary case, where 𝛼 ∈ {−1, 1}, it is convenient to introduce the likelihood ratio by (see (6.26))
P [ak = 1 | zk = 𝜌k ] 𝙻ak (1)
𝙻ak = = (6.32)
P [ak = −1 | zk = 𝜌k ] 𝙻ak (−1)
Then, the decision rule (6.27) becomes
{
1 if 𝙻ak ≥ 1
â k = (6.33)
−1 if 𝙻ak < 1
The analysis is simplified by the introduction of the log-likelihood ratio (LLR),

𝓁ak = ln 𝙻ak = 𝓁ak (1) − 𝓁ak (−1) (6.34)

where 𝓁ak (𝛼) is given by (6.30). Then (6.33) becomes

â k = sgn(𝓁ak ) (6.35)

Observing (6.35), we can write

𝓁ak = â k |𝓁ak | (6.36)

In other words, the sign of the LLR yields the hard-decision, while its magnitude indicates the decision
reliability.
Indeed, most of the time we are not interested on {̂ak } but on extracting from {zk } statistics of cm to be
used by a decoder with soft input. This procedure is called soft detection. In particular, the considered
statistics are summarized by the LLR
P[cm = 1 ∣ zk = 𝜌k ]
𝓁cm = ln (6.37)
P[cm = 0 ∣ zk = 𝜌k ]
Next, we derive techniques to compute (6.37).
One word of caution: on labelling each code bit assumes {0, 1} values. However, LLRs refer to
{−1, 1}, where the original 0 has become −1.

LLRs associated to bits of BMAP

We give correct and simplified LLR expressions of bits of BMAP (6.7) for square constellations. We
start with the simplest case.

1. Binary: For the mapping ck = 1 → ak = 1 and ck = 0 → ak = −1, and 𝜌k real, evaluation of (6.30)
for 𝛼 = −1, 1 yields
(𝜌 − 1)2 (𝜌 + 1)2
𝓁ck (1) = − k 2 , 𝓁ck (−1) = − k 2 (6.38)
2𝜎I 2𝜎I
Hence from (6.34)
2𝜌k
𝓁ck = (6.39)
𝜎I2
2. QPSK: Using the BMAP (6.7) with 𝚖 = 𝟷, it is like having a binary constellation in each dimension.
By letting 𝜌k = 𝜌k,I + j𝜌k,Q , it is
2𝜌k,I 2𝜌k,Q
𝓁ck,I = , 𝓁ck,Q = (6.40)
𝜎I2 𝜎I2
6.2. Detection 257

3. 16-QAM and above: For simplicity, ck𝚖−m,I and ck𝚖−m,Q in (6.7) will be denoted here as cm,I and
cm,Q , respectively. Moreover, 𝜌 will replace 𝜌k . For each bit cm,I in (6.7), we split the constellation
symbols into two sets (0)
m,I
and (1)
m,I
, which, respectively, collect symbols whose BMAP labelling
has cm,I = 0 and cm,I = 1. Similarly for cm,Q . For a 16-QAM constellation, where the BMAP is
BMAP
(ak,I ; ak,Q ) ←−→(c0,I , c1,I ; c0,Q , c1,Q ) (6.41)
as reported in Figure 6.2, the four partitions {(0)
0,I
(1)
, 0,I }, {(0)
1,I
, (1)
1,I
}, {(0)
0,Q
, (1)
0,Q
}, and
{(0)
1,Q
, (1)
1,Q
}
are shown in Figure 6.5.
We introduce the following approximation: for the generic bit cm,I , the LLs are
⎡ ∑ ⎤ ⎡ ∑ ⎤
𝓁cm,I (1) = ln ⎢ 𝙻ak (𝛼)⎥ , 𝓁cm,I (−1) = ln ⎢ 𝙻ak (𝛼)⎥ (6.42)
⎢ (1) ⎥ ⎢ (0) ⎥
⎣𝛼∈m,I ⎦ ⎣𝛼∈m,I ⎦
By the Max-Log-MAP approximation, we replace the sums in (6.42) by their largest terms, and
𝓁̃cm,I (1) = max 𝓁ak (𝛼), 𝓁̃cm,I (−1) = max 𝓁ak (𝛼) (6.43)
𝛼∈(1)
m,I
𝛼∈(0)
m,I

ZQ ZQ

00 10 01 10 11 10 10 10
3 3

2dI,2
00 11 01 11 11 11 10 11
1 1

–3 –1 1 3 ZI –3 –1 1 3 ZI
–1 –1
00 01 01 01 11 01 10 01
(0) (1) (0)
(0) (1) R R R
R R 1,I 1,I 1,I
0,I 0,I
–3 –3
00 00 01 00 11 00 10 00

ZQ ZQ

(0)
3 R 3
1,Q
(1)
R
0,Q
(1)
1 R 1
1,Q
2dQ,2
–3 –1 1 3 ZI –3 –1 1 3 ZI
–1 –1

(0)
R
0,Q
(0)
–3 R –3
1,Q

Figure 6.5 Partitions of the 16-QAM constellation.

258 Chapter 6. Digital transmission model and channel capacity

Finally, the approximated LLR is

𝓁̃cm,I = 𝓁̃cm,I (1) − 𝓁̃cm,I (−1) (6.44)
To simplify notation, we remove the symbol time index k. Substituting (6.30) in (6.44) yields, for
m = 0, 1, … , 𝚖 − 1, { }
1
𝓁̃cm,I = 2 min |𝜌 − 𝛼|2 − min(0) |𝜌 − 𝛼|2 (6.45)
2𝜎I 𝛼∈(1)
m,I
𝛼∈m,I

and { }
1
𝓁̃cm,Q = 2 min |𝜌 − 𝛼| − min(0) |𝜌 − 𝛼|
2 2
(6.46)
2𝜎I 𝛼∈(1)
m,Q
𝛼∈m,Q

For square constellations, boundaries of partitions are vertical lines for bits cm,I , while are hori-
zontal lines for bits cm,Q . Let 𝛼 = 𝛼I + j𝛼Q , 𝜌 = 𝜌I + j𝜌Q , and Re [(i) m,I
] (Im [(i)
m,I
]) containing the
real (imaginary) parts of symbols in (i) m,I
. Then, (6.45) and (6.46) can be written as
{ }
1
𝓁̃cm,I = 2 min |𝜌I − 𝛼I |2 − min (0) |𝜌I − 𝛼I |2 (6.47)
2𝜎I 𝛼I ∈Re[(1)
m,I
] 𝛼I ∈Re[m,I ]

and { }
1
𝓁̃cm,Q = 2 min |𝜌Q − 𝛼Q | −2
min |𝜌Q − 𝛼Q | 2
(6.48)
2𝜎I 𝛼Q ∈Im[(1)
m,Q
] 𝛼Q ∈Im[(0)
m,Q
]

Simplified expressions
Now, we concentrate on simplifying (6.47), keeping in mind that problem (6.48) is similar with index I
replaced by Q. Introduced the metric
{ }
1
dm,I = min |𝜌I − 𝛼I | − min (0) |𝜌I − 𝛼I |
2 2
(6.49)
4 𝛼I ∈Re[(1)
m,I
] 𝛼I ∈Re[m,I ]

it is
2dm,I
𝓁̃cm,I = (6.50)
𝜎I2
For 16-QAM, evaluation of (6.49) yields [2]
⎧𝜌 I |𝜌I | ≤ 2
⎪
d0,I = ⎨2(𝜌I − 1) 𝜌I > 2 , d1,I = −|𝜌I | + 2 (6.51)
⎪
⎩2(𝜌I + 1) 𝜌I < −2
Similarly, for 64-QAM
⎧
⎪𝜌I |𝜌I | ≤ 2
⎪2(𝜌 − 1) 2 < 𝜌I ≤ 4
⎪ I
⎪3(𝜌I − 2) 4 < 𝜌I ≤ 6
⎪
d0,I = ⎨4(𝜌I − 3) 𝜌I > 6 (6.52)
⎪
⎪2(𝜌I + 1) −4 < 𝜌I ≤ −2
⎪3(𝜌 + 2) −6 < 𝜌I ≤ −4
⎪ I
⎪4(𝜌I + 3) 𝜌I < −6
⎩
6.2. Detection 259

⎧2(−|𝜌I | + 3) |𝜌I | ≤ 2 {
⎪ |𝜌I | − 2 |𝜌I | ≤ 4
d1,I = ⎨4 − |𝜌I | 2 < |𝜌I | ≤ 4 , d2,I = (6.53)
⎪ −|𝜌I | + 6 |𝜌I | > 4
⎩2(−|𝜌I | + 5) |𝜌I | > 6
Above expressions are still cumbersome to evaluate, so we introduce a further simplification [3]. First,
for 16-QAM we have the following approximation
d0,I ≃ 𝜌I , d1,I = −|𝜌I | + 2 (6.54)
while for 64-QAM it is
d0,I ≃ 𝜌I , d1,I ≃ −|𝜌I | + 4, d2,I = −||𝜌I | − 4| + 2 (6.55)

10 10 10
Theoretical
Approximation
8 8 8

6 6 6

4 4 4

2 2 2
d0,I

d1,I

d2,I

0 0 0

–2 –2 –2

–4 –4 –4

–6 –6 –6

–8 –8 –8

–10 –10 –10

–5 0 5 –5 0 5 –5 0 5
ρI ρI ρI

Figure 6.6 Approximate versus exact metrics.

These approximations correspond to evaluating |dm,I | as the distance of 𝜌I from the nearest parti-
tion boundary and assigning to dm,I the sign according to the partition wherein 𝜌I falls. For 16-QAM,
Figure 6.6 shows the exact and approximated functions. These simplifications can be generalized to any
square QAM constellation. For m > 0, let Dm,I denote half the distance between the partition boundary
relative to bit cm,I . For example, for 16-QAM D1,I = 2, see Figure 6.5, while for 64-QAM D1,I = 4 and
D2,I = 2. Then, dm,I , m = 0, 1, … , 𝚖 − 1, can be written in a recursive form as
{
𝜌I m=0
dm,I ≃ (6.56)
−|dm−1,I | + Dm,I m > 1
260 Chapter 6. Digital transmission model and channel capacity

Comparing (6.56) with the Max-LogMAP approach (6.45), we note that the amount of complexity of
the proposed scheme, in terms of real multiplications, is independent of the number of bits per symbol,
as well as the dimension of the constellation. From simulation results [3], it is seen that for both 16- and
64-QAM there is almost no performance degradation using the approximate metric (6.56).
We also mention alternative approximation methods for evaluating LLRs. Some make use of a differ-
ent bit mapper [4]. Moreover, LLRs for PSK and differential PSK (DPSK) can be found in [5–7].

6.2.3 Receiver strategies

The receiver may adopt one of the following strategies to obtain b̂ 𝓁 :
1. Equalization of the received signal, hard detection (to yield â k ), IBMAP (to yield ĉ m ), and
hard-input decoding. In general, this approach, although rather simple, yields poor performance.
2. Equalization of the digital signal, soft detection of code bits (to yield their LLRs), and soft-input
decoding. The two operations of equalization and detection/decoding are, sometimes, iterated (see
Section 11.7).
3. Soft sequence detection (to yield the LLRs starting from the received signal, without equalization)
and soft-input decoding. This strategy significantly improves performance and is feasible for a
small constellation size and simple transmission channel (see Section 7.5).
4. Combined detection and decoding. This yields best performance (see Section 11.3.2); however,
this approach is not modular in the constellation, and is usually quite complex to implement.

6.3 Relevant parameters of the digital transmission model

We report below several general definitions widely used to describe a modulation system.
● Tb : bit period (s): time interval between two consecutive information bits. We assume that the mes-
sage {b𝓁 } is composed of binary i.i.d. symbols.
● Rb = 1∕Tb : bit rate of the system (bit/s).
● T: modulation interval or symbol period (s).
● 1∕T: modulation rate or symbol rate (Baud).
● Lb : number of information message bits b𝓁 per modulation interval T.
● M: cardinality of the alphabet of symbols at the transmitter.
● I: number of dimensions of the symbol constellation , or, equivalently, of the modulator space
(e.g. for PAM I = 1, for QAM I = 2).
● RI : rate of the encoder-modulator (bit/dim).
● 𝙼sCh : statistical power of the desired signal at the receiver input (V2 ).
● EsCh : average pulse energy (V2 s).
● EI : average energy per dimension (V2 s/dim).
● Eb : average energy per information bit (V2 s/bit).
● N0 ∕2: PSD of additive white noise introduced by the channel (V2 /Hz).
● Bmin : conventional minimum bandwidth of the modulated signal (Hz).
● 𝜈: spectral efficiency of the system (bit/s/Hz).
● Γ: conventional signal-to-noise ratio (SNR) at the receiver input.
● ΓI : SNR per dimension.
● 𝙿sCh : available power of the desired signal at the receiver input (W).
● 𝚃𝑤i : effective receiver noise temperature (K).
● : sensitivity (W): the minimum value of 𝙿sCh (see (4.26)) such that the system achieves a given
performance in terms of bit error probability.
6.3. Relevant parameters of the digital transmission model 261

Relations among parameters

1. Rate of the encoder-modulator:
Lb
RI = (6.57)
I
2. Number of information bits per modulation interval: via the channel encoder and the bit-mapper,
Lb information bits of the message {b𝓁 } are mapped in an M-ary symbol, ak . In general we have
Lb ≤ log2 M (6.58)
where the equality holds for a system without coding, or, with abuse of language, for an uncoded
system. In this case, we also have
log2 M
RI = (uncoded case) (6.59)
I
3. Symbol period:
T = Tb Lb (6.60)
4. Statistical power of the desired signal at the receiver input:
Es
𝙼sCh = Ch (6.61)
T
For a continuous transmission, 𝙼sCh is finite and we define EsCh = 𝙼sCh T.
5. Average energy per dimension:
Es
EI = Ch (6.62)
I
6. Average energy per information bit:
E Es
Eb = I = Ch (6.63)
RI Lb
For an uncoded system, (6.63) becomes
EsCh
Eb = (uncoded case) (6.64)
log2 M
7. Conventional minimum bandwidth of the modulated signal:
1
Bmin = for baseband signals (PAM), (6.65)
2T
1
Bmin = for passband signals (QAM, PSK) (6.66)
T
8. Spectral efficiency:
R Lb
𝜈= b = (6.67)
Bmin Bmin T
In practice, 𝜈 measures how many information bits per unit of time are sent over a channel with
the conventional bandwidth Bmin . In terms of RI , from (6.57), we have
RI I
𝜈= (6.68)
Bmin T
For most modulation systems, RI = 𝜈∕2.
9. Conventional SNR at the receiver input:
𝙼sCh EsCh
Γ= = (6.69)
(N0 ∕2)2Bmin N0 Bmin T
In general, Γ expresses the ratio between the statistical power of the desired signal at the receiver
input and the statistical power of the noise measured with respect to the conventional bandwidth
Bmin . We stress that the noise power is infinite, from a theoretical point of view.
262 Chapter 6. Digital transmission model and channel capacity

Note that, for the same value of N0 ∕2, if Bmin doubles the statistical power must also double to
maintain a given ratio Γ.
10. SNR per dimension:
EI 2EsCh
ΓI = = (6.70)
N0 ∕2 N0 I
is the ratio between the energy per dimension EI and the channel noise PSD (per dimension)
N0 ∕2. Using (6.63), the general relation becomes
Eb
ΓI = 2RI (6.71)
N0
It is interesting to observe that in most modulation systems it turns out ΓI = Γ.

6.4 Error probability

We now consider a metric to assess the performance of uncoded communication systems with hard
detection.

Definition 6.1
The transformation that maps cm into c̃ m is called binary channel. It is characterized by the bit rate
1∕Tcod , which is the transmission rate of the bits of the sequence {cm }, and by the bit error probability
Pbit = PBC = P[̃cm ≠ cm ], c̃ m , cm ∈ {0, 1} (6.72)
In the case of a binary symmetric channel (BSC), it is assumed that P[̃cm ≠ cm ∣ cm = 0] = P[̃cm ≠ cm ∣
cm = 1]. We say that the BSC is memoryless if, for every choice of N distinct instants m1 , m2 , … , mN ,
the following relation holds:
P[̃cm1 ≠ cm1 , c̃ m2 ≠ cm2 , … , c̃ mN ≠ cmN ]
(6.73)
= P[̃cm1 ≠ cm1 ] P[̃cm2 ≠ cm2 ] … P[̃cmN ≠ cmN ] ◽

In a memoryless binary symmetric channel, the probability distribution of {̃cm } is obtained from that
of {cm } and PBC according to the statistical model shown in Figure 6.7.

1–PBC
1 1
PBC

~
cm cm
PBC

0 0
1–PBC

Figure 6.7 Memoryless binary symmetric channel.

We note that the aim of the channel encoder is to introduce redundancy in the sequence {cm }, which
is exploited by the decoder to detect and/or correct errors introduced by the binary channel.
6.4. Error probability 263

The overall objective of the transmission system is to reproduce the sequence of information bits {b𝓁 }
with a high degree of reliability, measured by the bit error probability

P(dec)
bit
= P[b̂ 𝓁 ≠ b𝓁 ] (6.74)

An approximated expression of Pbit is derived, while the relation between P(dec) bit
and Pbit is detailed in
Chapter 11 for some error correcting code classes. We assume the signal at the detection point is given
by (6.13). If dmin is the distance between adjacent constellation points, we define the SNR at the detection
point as
( )
dmin ∕2 2
𝛾= (6.75)
𝜎I
We recall that here dmin = 2 for both PAM and QAM. The symbol error probability

Pe = 1 − P[𝙲] = P[̂ak ≠ ak ] (6.76)

has the following expressions for PAM and for rectangular QAM constellations (see Ref. [1])
( ) √
1
Pe = 2 1 − Q( 𝛾), M − PAM (6.77)
( M )
1 √
Pe ≃ 4 1 − √ Q( 𝛾), QAM (M = 4, 16, 64, …) (6.78)
M
where Q(⋅) is the complementary Gaussian cumulative distribution function (see Appendix 6.B).
In narrowband transmission systems, it is (see the observation after (7.107))
Γ
𝛾= , PAM (6.79)
𝙼a
Γ
𝛾= , QAM, PSK (6.80)
𝙼a ∕2
where 𝙼a ∕2 is the statistical power per-dimension of the symbol sequence {ak }. In this case,
( ) (√ )
1 3
Pe = 2 1 − Q Γ , PAM (6.81)
M (M 2 − 1)
( ) (√ )
1 3
Pe ≃ 4 1 − √ Q Γ , QAM (6.82)
M (M − 1)

while for PSK (see Ref. [1]) [√ )] (

𝜋
Pe ≃ 2Q 2Γ sin (6.83)
M
is a good approximation of Pe for M ≥ 4 and Γ >> 1. For M = 2 (binary PSK, BPSK), we have
√
Pe = Pbit = Q( 2Γ) (6.84)

If an error occurs on detecting ak , â k is most probably one of the adjacent symbols of ak , and for Gray
labelling only one bit will be in error, therefore
Pe
Pbit ≃ (6.85)
log2 M
Curves of Pbit for PAM, QAM, and PSK, are reported, respectively, in Figures 6.8–6.10.
In order to achieve a given Pbit with QAM, if M is increased by a factor 4, we need to increase Γ by
6 dB: in other words, if we increment by one the number of bits per symbol, Γ should be increased on
average by 3 dB.
264 Chapter 6. Digital transmission model and channel capacity

10–1

M = 16
10–2

M=8
10–3
Pbit

M=4
10–4

M=2
10–5

10–6
0 5 10 15 20 25 30 35
Γ(dB)

Figure 6.8 Bit error probability as a function of Γ for M-PAM.

10–1

M = 256
10–2

M = 64
10–3
Pbit

M = 16
10–4

M=4
10–5

10–6
0 5 10 15 20 25 30 35
Γ(dB)

Figure 6.9 Bit error probability as a function of Γ for M-QAM with a rectangular constellation.
6.5. Capacity 265

10–1

M = 32
10–2

M = 16
10–3
Pbit

M=8
10–4

M=4
10–5

M=2
10–6
0 5 10 15 20 25 30 35
Γ(dB)

Figure 6.10 Bit error probability as a function of Γ for M-PSK.

6.5 Capacity

The channel capacity is defined as the maximum of the mutual information between the input and output
signals of the channel [8, 9], where the maximum is taken with respect to the statistical distribution of the
channel input. The channel capacity will be denoted as C when measured in bit/symbol/dim or simply
in bit/dim, or as C[bit∕s] when measured in bit/s.
Let RI be the rate of the encoder–modulator (in bit/dim) (see (6.57)) of a communication system over
a channel with capacity C. We give, without proof, the following fundamental theorem [8, 10].

Theorem 6.1 (Shannon’s theorem)

For any rate RI < C, with transmitted symbols having a Gaussian distribution, there exists a channel
coding that provides an arbitrarily small probability of error; such coding does not exist if RI > C. ◽

We note that Shannon’s theorem indicates the bounds, in terms of encoder–modulator rate or, equiva-
lently, in terms of transmission bit rate (see (6.96)), within which we can develop systems for the reliable
transmission of information, but it does not give any indication about the practical realization of channel
coding.
The bound RI < C for a reliable transmission corresponds to Rb < C[bit∕s] .
We now report expressions on the channel capacity for AWGN channels. In particular, we consider (i)
the discrete-time AWGN channel, (ii) the single-input-single-output (SISO) narrowband (NB) AWGN
channel, (iii) the SISO dispersive AGN channel, and (iv) the discrete-time MIMO NB AWGN channel,
and its extension to continuous-time channels.
266 Chapter 6. Digital transmission model and channel capacity

6.5.1 Discrete-time AWGN channel

With reference to a message {ak } composed of a sequence of i.i.d. symbols, with zero mean and power
𝙼a , which belongs to a I-dimensional constellation, let the received signal be as in (6.13), where the noise
sequence 𝑤k is i.i.d. and each 𝑤k is Gaussian distributed with power per dimension 𝜎I2 , i.e. 𝜎𝑤2 = I𝜎I2 .
It is ( )
1 𝙼a
C = log2 1 + 2 [bit/dim] (6.86)
2 𝜎𝑤
The capacity is reached if ak is Gaussian distributed. From (6.69) and (6.70), assuming 𝙼sCh = 𝙼a and
N0
𝜎𝑤2 = 2
2Bmin , it is also
𝙼a 𝙼a ∕I
Γ= = = ΓI (6.87)
𝜎𝑤2 𝜎I2
and
1
log (1 + ΓI )
C= (6.88)
2 2
In particular, for PAM and QAM constellations, we have
( )
⎧1 𝙼a
⎪ log2 1 + 2 PAM
⎪2 𝜎I
C=⎨ ( ) (6.89)
⎪1 𝙼a ∕2
⎪ 2 log2 1 + 2 QAM
⎩ 𝜎I
Moreover, if the symbol rate is 1∕T
⎧1 1
⎪ T C = 2T log2 (1 + Γ) PAM (I = 1)
C[bit∕s] =⎨ (6.90)
⎪ 1 2C = 1 log2 (1 + Γ) QAM (I = 2)
⎩T T
From (6.88), C can be upper bounded and approximated for small values of ΓI by a linear function and
also lower bounded and approximated for large values of ΓI by a logarithmic function as follows:
1
ΓI ≪ 1 ∶ C ≤ log2 (e) ΓI (6.91)
2
1
ΓI ≫ 1 ∶ C ≥ log2 ΓI (6.92)
2

6.5.2 SISO narrowband AWGN channel

Let us consider an ideal SISO NB AWGN channel for sCh (t) = s(t) with, see (6.12),
r(t) = s(t) + 𝑤(t) (6.93)
where s is the real-valued transmitted signal, and 𝑤 is AWGN with PSD N0 ∕2. We further assume that
the transmitted signal has PSD s (f ) and passband , collecting only positive sequences (see (1.69)),
with bandwidth given by (1.70), i.e. B = ∫ df . We further assume a power constraint on the signal s, i.e.
1 2
E[s (t)] = s ( f ) df ≤ VP (6.94)
2 ∫
6.5. Capacity 267

The capacity of this channel is achieved when s is Gaussian distributed with zero-mean and constant
PSD over the band. Thus, recalling the power constraint (6.94), we have for f > 0,
⎧ VP
⎪ f ∈
s,opt ( f ) = ⎨ B (6.95)
⎪0 otherwise
⎩
and for f < 0 we have s,opt ( f ) = s,opt (−f ).
The resulting capacity of the SISO NB AWGN channel (6.93), under constraint (6.94) is
C[bit∕s] = B log2 (1 + Γ) [bit/s] (6.96)
where Γ is the SNR at the receiver input (6.69) for Bmin = B, i.e.
VP
Γ= N0
(6.97)
2
B

Channel gain
A first generalization of the ideal channel (6.93) is obtained by considering a channel with gain Ch , i.e.
r(t) = Ch s(t) + 𝑤(t) (6.98)
Under power constraint (6.94), the capacity of channel (6.98) is given by (6.96), still achieved for a
Gaussian signal s with constant PSD over , where
2Ch VP
Γ= N0
(6.99)
2
B

6.5.3 SISO dispersive AGN channel

In general, a linear dispersive channel with additive Gaussian noise (AGN) not necessarily white, is
characterized by a filter with impulse response gCh and input–output relation
r(t) = s ∗ gCh (t) + 𝑤(t) (6.100)
where the noise 𝑤 has a PSD 𝑤 ( f ). Let Tx be the transmit signal band, and consider the transmit
power constraint
s ( f ) df ≤ VP (6.101)
∫Tx
The channel-gain-to-noise-PSD ratio as a function of the frequency is defined as
|Ch ( f )|2
ΘCh ( f ) = (6.102)
𝑤 ( f )
where Ch ( f ) is the Fourier transform of gCh .
In order to find the capacity of a dispersive AGN channel, we divide the passband Tx , having measure
B, into N subbands i , i = 1, … , N, of width Δf = B∕N, where Δf is chosen sufficiently small so that
s ( f ) and ΘCh ( f ) are, to a first approximation, constant within i , i.e. we assume ∃fi ∈ i such that
s ( f ) ≃ s ( fi ) and ΘCh ( f ) ≃ ΘCh ( fi ), ∀f ∈ i .
268 Chapter 6. Digital transmission model and channel capacity

Since on each subchannel we have a SISO NB AGN channel, from (6.96) and (6.98) subchannel i has
an (approximated) capacity
[ ]
Δf s,opt (fi ) |Ch (fi )|2
C[bit∕s] [i] = Δf log2 1 + (6.103)
Δf 𝑤 (fi )
with s Gaussian on each subchannel and the s,opt (f ) is optimized to satisfy the power constraint (as
detailed in the following). Since the noise components are independent on each subchannel, we have
a parallel AGN channel and the capacity of the overall dispersive AGN channel is obtained by adding
terms C[bit∕s] [i], i = 1, 2, … , N, that is
[ ]
∑N
∑N
Δf s,opt (fi ) |Ch (fi )|2
C[bit∕s] [i] = Δf log2 1 + (6.104)
i=1 i=1
Δf 𝑤 (fi )
By letting Δf tend to zero, we obtain
[ ]
| (f )|2
C[bit∕s] = log2 1 + s,opt (f ) Ch df = log [1 + s,opt (f ) ΘCh (f )]df (6.105)
∫Tx 𝑤 (f ) ∫Tx 2
We now find the PSD s (f ) that maximizes (6.105), under the constraint (6.101). By applying the method
of Lagrange multipliers, the optimum s,opt (f ) maximizes the integral

{log2 [1 + s,opt (f ) ΘCh (f )] + 𝜆 s,opt (f )}df (6.106)

∫Tx
where 𝜆 is a Lagrange multiplier. Using the calculus of variations [11], we find the following condition:
ΘCh (f )
+ 𝜆 ln 2 = 0 (6.107)
1 + s,opt (f ) ΘCh (f )
Therefore, the PSD providing the capacity is given for f ≥ 0 by3
[ ]+
1
s,opt (f ) = L − (6.109)
ΘCh (f )
where L is a constant such that (6.101) is satisfied with the equal sign. As the channel impulse response
is assumed real valued, for f < 0 we get s,opt (f ) = s,opt (−f ). The function s,opt (f ) is illustrated in
Figure 6.11 for a typical behaviour of the function ΘCh (f ).

1
Ps,opt ( f ) = L –
ΘCh ( f )
L
1
ΘCh ( f )

0 f1 f2 f

Figure 6.11 Illustration of s,opt (f ) for a typical behaviour of the function ΘCh (f ).

3
We use the notation
{
x if x > 0
(x)+ = (6.108)
0 otherwise
6.5. Capacity 269

It results that s,opt (f ) should assume large values (small values) at frequencies for which ΘCh (f )
assumes large values (small values). From the figure we note that if 1∕ΘCh (f ) is the profile of a cup in
which if we pour a quantity of water equivalent to VP , the distribution of the water in the cup takes place
according to the behaviour predicted by (6.109); this observation leads to the water pouring interpreta-
tion of the optimum distribution of s,opt (f ) as a function of frequency.
Note that from (6.109) s,opt (f ) > 0 only for a subset of frequencies f ∈  ⊂ Tx ; therefore, the effec-
tive transmission passband of s will be further restricted to .
In analogy with the case of an ideal (non-dispersive) AWGN channel with limited bandwidth, a linear
dispersive channel can be roughly characterized by two parameters: the bandwidth B = ∫ df and the
effective SNR Γeff , implicitly defined so that
C[bit∕s] = B log2 (1 + Γeff ) (6.110)
The comparison with (6.105) yields
{ }
1
Γeff = exp ln [1 + s,opt (f )ΘCh (f )]df −1 (6.111)
B ∫
Note that 1 + Γeff corresponds to the geometric mean of the function 1 + s,opt (f )ΘCh (f ) in the band .
The passband  represents the most important parameter of the spectrum obtained by water pouring.
For many channels utilized in practice, the passband  is composed of only one frequency interval
[f1 , f2 ], as illustrated in Figure 6.11; in other cases, for example in the presence of high-power narrowband
interference signals,  may be formed by the union of disjoint frequency intervals. In practice, we find
that the dependence of the capacity on s,opt (f ) is not so critical as the dependence on ; a constant
power spectral density in the band  usually allows a system to closely approach capacity. Therefore,
the application of the water pouring criterion may be limited to the computation of the passband.
Expression (6.105) for any s (f ) ≠ s,opt (f ) under constraint (6.101) is called achievable bit rate.
More in general, the achievable bit rate refers to the performance obtained with suboptimal coding
and/or modulation schemes.

6.5.4 MIMO discrete-time NB AWGN channel

We now refer to the discrete-time (with symbol period T) multiple-input-multiple-output (MIMO) NB
(i.e. non-dispersive) AWGN channel, between NTx inputs and NRc outputs, that can be modelled as (see
(9.17))
xk = C sk + 𝒘k (6.112)
where xk is the NRc × 1 vector of received signals, C is the NRc × NTx channel matrix, sk =
(N )
[sk(1) , … , sk Tx ]T is the NTx × 1 transmitted signal vector, and 𝒘k is the NRc × 1 noise vector. We consider
the transmit power constraint

1∑
NTx
1
E [||sk ||2 ] = E [|s(i)
k
|2 ] ≤ VP (6.113)
2 2 i=1
Let NS,max = rank C , hence NS,max ≤ min(NRc , NTx ). The singular value decomposition (SVD) of the
channel matrix yields (see Section 2.3.2)
C = Uext 𝚺 ext V H
ext (6.114)
with Uext NRc × NRc , and V ext NTx × NTx unitary matrices, and 𝚺 ext NRc × NTx diagonal matrix such that
[ ]
𝚺 𝟎
𝚺 ext = (6.115)
𝟎 𝟎
270 Chapter 6. Digital transmission model and channel capacity

with 𝚺 = diag([𝜍1 , … , 𝜍NS,max ]T ) and 𝟎 an all-zero matrix of suitable size. Let us also define V as the
sub-matrix of V ext defined by its first NS,max columns. If ak = [ak,1 , … , ak,NS,max ]T is the vector of data
streams, we set sk = Vak . Then, the capacity of the MIMO NB AWGN channel is achieved when entries
of sk are zero-mean Gaussian distributed with constant PSD and (spatial) correlation matrix

k ] = Vdiag([𝙼1 , … , 𝙼NS,max ] )V
H
Rs = E[sk sH T
(6.116)
∑NS,max
with 𝙼n = E[|ak,n |2 ] ≥ 0. Note that E[||sk ||2 ] = tr Rs = n=1
𝙼n . Define the SNR on stream n as
|𝜍n |2
Γn = 𝙼n Θn , Θn = N0
(6.117)
2
2Bmin
with Bmin = 1∕T, the capacity is [12, 13]

1 ∑
NS,max
C[bit∕s] = log2 (1 + Γn ) (6.118)
T n=1
where values of 𝙼n = 𝙼opt,n , n = 1, … , NS,max , maximize (6.118) under the power constraint (6.113),
which can be written as
1 ∑
NS,max
𝙼 ≤ VP (6.119)
2 n=1 n
From (6.118), we observe that the achievable bit rate of the MIMO NB AWGN channel is that of a
parallel AWGN channel with NS,max subchannels.
By maximizing (6.118) under constraint (6.119), we have the following solution
[ ]+
1
𝙼opt,n = L − (6.120)
Θn
where the constant L is such that (6.119) is satisfied with the equal sign.
With NTx inputs and NRc outputs we can transmit up to NS,max independent data streams, thus exploiting
the multiplexing capabilities of the MIMO channel. Since NS,max ≤ min(NRc , NTx ), we observe that a
larger number of streams can be transmitted only when both NTx and NRc increase. In Section 9.4, it will
be seen that the MIMO capacity provides also a specific transmitter and receiver architecture.

Continuous-time model
For the continuous-time NB AWGN MIMO channel model, the derivation of the capacity follows the
same line of the discrete-time case.

MIMO dispersive channel

For a MIMO dispersive channel, we can still consider its frequency-domain representation, and extend
above capacity formulas. In particular, capacity is achieved by transmitting a Gaussian signal, with
correlation given by (6.116), where V is replaced by V(f ), the right singular vector matrix of C (f ), the
matrix frequency response of the channel.

6.6 Achievable rates of modulations in AWGN channels

We now provide some derivations on the achievable rate (in bit/dim) of communication systems using
specific modulation techniques over the NB AWGN channel.
6.6. Achievable rates of modulations in AWGN channels 271

6.6.1 Rate as a function of the SNR per dimension

For a given value of the symbol error probability, we now derive ΓI as a function of RI for some multilevel
modulations in an ideal AWGN channel. The result will be compared with the Shannon limit (6.88)
ΓI,bound = 22RI − 1 (6.121)
that represents the minimum theoretical value of ΓI , in correspondence of a given RI , for which
Pbit can be made arbitrarily small by using channel coding without constraints in complexity and
latency.
A first comparison is√made by assuming the same symbol error probability, Pe = 10−6 , for all
systems. Since Pe =√Q( 𝜁), where in turn 𝜁 is a function of ΓI , according to the considered modu-
lation, imposing Q( 𝜁 ) = 10−6 provides 𝜁 ≃ 22. We now consider M-PAM, M-QAM, and M-PSK
modulations.
1. M-PAM: From (6.81), neglecting terms in from of the Q function, we have
3
Γ=𝜁 (6.122)
M2 − 1
and
ΓI = Γ, RI = log2 M (6.123)
we obtain the following relation
𝜁 2RI
ΓI = (2 − 1) (6.124)
3
2. M-QAM: From (6.82) we have
3
Γ=𝜁 (6.125)
M−1
and
1
ΓI = Γ, RI = log M (6.126)
2 2
we obtain
𝜁 2RI
ΓI = (2 − 1) (6.127)
3
We note that for QAM, a certain RI is obtained with a number of symbols equal to MQAM = 22RI ,
√
whereas for PAM the same efficiency is reached for MPAM = 2RI = MQAM .
3. M-PSK: From (6.83) it turns out
𝜁 4RI
ΓI = 2 (6.128)
20
Equation (6.128) is obtained by approximating sin(𝜋∕M) with 𝜋∕M, and 𝜋 2 with 10.
We note that, for a given value of RI , PAM, and QAM require the same value of ΓI , whereas PSK
requires a much larger value of ΓI .
An exact comparison is now made for a given bit error probability. Using the Pbit curves previously
obtained, the behaviour of RI as a function of ΓI for Pbit = 10−6 is illustrated in Figure 6.12. We observe
that the required ΓI is much larger than the minimum value obtained by the Shannon limit. As will be
discussed next, the gap can be reduced by channel coding.
We also note that, for large RI , PAM and QAM allow a lower ΓI with respect to PSK.
PAM, QAM, and PSK are bandwidth-efficient modulation methods as they cover the region for RI > 1,
as illustrated in Figure 6.12. The bandwidth is traded off with the power, that is Γ, by increasing the
number of levels: we note that, in this region, higher values of Γ are required to increase RI .
272 Chapter 6. Digital transmission model and channel capacity

5 32

4.5

4 16 256

PAM
3.5 Shannon
bound
3 8 64
64
RI (bit/dim)

1/2log2 (1+ ΓI) QAM

PSK
2.5 32

2 4 16
16

1.5 8
2 4
1

0.5

0
–10 0 10 20 30 40
ΓI (dB)

Figure 6.12 ΓI required for a given rate RI , for different uncoded modulation methods and bit error
probability Pbit = 10−6 . The parameter in the figure denotes the number of symbols M of the
constellation.

6.6.2 Coding strategies depending on the signal-to-noise ratio

The formula of the NB AWGN channel capacity (6.88) can be expressed as ΓI ∕(22C − 1) = 1. This
relation suggests the definition of the normalized SNR
ΓI
ΓI = (6.129)
22RI − 1
for a given RI given by (6.57). For a scheme that achieves the capacity, RI is equal to the capacity of the
channel C and ΓI = 1 (0 dB); if RI < C, as it must be in practice, then ΓI > 1. Therefore, the value of
ΓI indicates how far from the Shannon limit a system operates, or, in other words, the gap that separates
the system from capacity. We now consider two cases.

High signal-to-noise ratios. We note from Figure 6.12 that for high values of ΓI , it is possible to find
coding methods for the reliable transmission of several bits per dimension. For an uncoded M-PAM
system,
RI = log2 M or 2RI = M (6.130)

bits of information are mapped into each transmitted symbol.

We note that Pe is function only of M and ΓI . Moreover, using (6.130) and (6.129) we obtain
ΓI
ΓI = (uncoded case) (6.131)
M2 − 1
6.6. Achievable rates of modulations in AWGN channels 273

For large M, Pe can therefore be expressed as

( ) (√ ) (√ )
1
Pe = 2 1 − Q 3ΓI ≃ 2Q 3ΓI (uncoded case) (6.132)
M
We note that the relation between Pe and ΓI is almost independent of M, if M is large. This relation is
used in the comparison illustrated in Figure 6.13 between uncoded systems and the Shannon limit given
by ΓI = 1.

100

10–1
Shannon limits
uncoded M–PAM, M large
10–2 RI 0 RI = 1/2
Eb/N0 Eb/N0 ΓI
–1.59 dB 0 dB 0 dB
10–3
uncoded 2–PAM
10–4
Pe

9 dB, M large
10.8 dB, M = 2
10–5
12.5 dB, M = 2

10–6

10–7

10–8
–4 –2 0 2 4 6 8 10 12
ΓI (dB)

–2 0 2 4 6 8 10 12
Eb /N0 (dB)

Figure 6.13 Bit error probability as a function of Eb ∕N0 for an uncoded 2-PAM system, and symbol
error probability as a function of ΓI for an uncoded M-PAM system. Source: Reproduced
with permission from Forney and Ungerboeck [14]. 1998, IEEE.

Low SNRs. For low values of ΓI , the capacity is less than 1 and can be almost reached by binary
transmission systems: consequently we refer to coding methods that employ more binary symbols to
obtain the reliable transmission of 1 bit. For low values of ΓI , it is customary to introduce the following
ratio (see (6.71)):
Eb 22RI − 1
= ΓI (6.133)
N0 2RI
We note the following particular cases:
● if RI ≪ 1, then Eb ∕N0 ≃ (ln 2) ΓI ;
● if RI = 1∕2, then Eb ∕N0 = ΓI ;
● if RI = 1, then Eb ∕N0 = (3∕2) ΓI .
For low ΓI , if the bandwidth can be extended without limit for a given power, for example by using
an orthogonal modulation (e.g. see Chapter 10) with T → 0, then by increasing the bandwidth, or
274 Chapter 6. Digital transmission model and channel capacity

equivalently the number of dimensions I of input signals, both ΓI and RI tend to zero. For systems
with limited power and unlimited bandwidth, usually Eb ∕N0 is adopted as a figure of merit.
From (6.133) and the Shannon limit ΓI > 1, we obtain the Shannon limit in terms of Eb ∕N0 for a given
rate RI as
Eb 22RI − 1
> (6.134)
N0 2RI
This lower bound monotonically decreases with RI .
In particular, we examine again the three cases:
● if RI tends to zero, the ultimate Shannon limit is given by
Eb
> ln 2 (−1.59 dB) (6.135)
N0
in other words, (6.135) states that even though an infinitely large bandwidth is used, reliable trans-
mission can be achieved only if Eb ∕N0 > −1.59 dB;
● if the bandwidth is limited, from (6.134) we find that the Shannon limit in terms of Eb ∕N0 is higher;
for example, if RI = 1∕2, the limit becomes Eb ∕N0 > 1 (0 dB);
● if RI = 1, as Eb ∕N0 = (3∕2) ΓI , the symbol error probability or bit error probability for an uncoded
2-PAM system can be expressed in two equivalent ways:
(√ ) (√ )
2Eb
Pbit ≃ Q 3ΓI = Q (6.136)
N0

Coding gain
Definition 6.2
The coding gain of a coded modulation scheme is equal to the reduction in the value of Eb ∕N0 , or in the
value of Γ or ΓI (see (11.84)), that is required to obtain a given probability of error relative to a reference
uncoded system. If the modulation rate of the coded system remains unchanged, we typically refer to Γ
or ΓI . ◽

Let us consider as reference systems a 2-PAM system and an M-PAM system with M ≫ 1, for small
and large values of ΓI , respectively. Figure 6.13 illustrates the bit error probability for an uncoded 2-PAM
system as a function of both Eb ∕N0 and ΓI . For Pbit = 10−6 , the reference uncoded 2-PAM system oper-
ates at about 12.5 dB from the ultimate Shannon limit. Thus, a coding gain up to 12.5 dB is possible,
in principle, at this probability of error, if the bandwidth can be sufficiently extended to allow the use
of binary codes with RI ≪ 1; if, instead, the bandwidth can be extended only by a factor 2 with respect
to an uncoded system, then a binary code with rate RI = 1∕2 can yield a coding gain up to about
10.8 dB.
Figure 6.13 also shows the symbol error probability for an uncoded M-PAM system as a function of ΓI
for large M. For Pe = 10−6 , a reference uncoded M-PAM system operates at about 9 dB from the Shannon
limit: in other words, assuming a limited bandwidth system, the Shannon limit can be achieved by a code
having a gain of about 9 dB.
For a dispersive channel, in analogy with the case of an ideal AWGN channel, it is useful to define a
normalized SNR as (see (6.111))
Γeff
Γeff = 2R (6.137)
2 I −1
where Γeff > 1 measures the gap that separates the considered system from capacity.
6.6. Achievable rates of modulations in AWGN channels 275

6.6.3 Achievable rate of an AWGN channel using PAM

We now consider the achievable rate of a AWGN channel, when the input is forced to be a PAM signal,
thus not being Gaussian as required by a capacity-achieving solution. The maximum achievable rate of
a real-valued ideal AWGN channel having as input an M-PAM signal is given in bit/dim by [15]
[ ]
∑M +∞ pz∣a0 (𝜂 ∣ 𝛼n )
C = max pn p (𝜂 ∣ 𝛼n ) log2 ∑M d𝜂 (6.138)
p1 ,…,pM ∫−∞ z∣a0
n=1 i=1 pi pz∣a0 (𝜂 ∣ 𝛼i )

where pn indicates the probability of transmission of the symbol a0 = 𝛼n . By the hypothesis of white
Gaussian noise, we have { }
(𝜂 − 𝛼n )2
pz∣a0 (𝜂 ∣ 𝛼n ) ∝ exp − (6.139)
2𝜎I2
With the further hypothesis that only codes with equally likely symbols (els) are of practical interest,
the computation of the maximum of C with respect to the probability distribution of the input signal can
be omitted. The channel achievable rate is therefore given by
[M ( )]
1 ∑ ∑
M
(𝛼n + 𝜉 − 𝛼i )2 − 𝜉 2
2
+∞
1 − 𝜉2
√
2𝜎
Cels = log2 M − e I log2 exp − d𝜉 (6.140)
M n=1 ∫−∞ 2𝜋𝜎 i=1 2𝜎 2 I
I
The capacity Cels is illustrated in Figure 6.14, where the Shannon limit given by (6.88), as well as
the SNR given by (6.124) for which a symbol error probability equal to 10−6 is obtained for uncoded
transmission, are also shown [14]. We note that the curves saturate as information cannot be transmitted
with a rate larger than RI = log2 M.
Let us consider for example the uncoded transmission of 1 bit of information per modulation inter-
val by a 2-PAM system, where we have a symbol error probability equal to 10−6 for ΓI = 13.5 dB. If

5
32-PAM
1.53 dB
4
16-PAM
Capacity (bit/dim)

3
8-PAM

2 1
2
log2 (1+ ΓI ) 4-PAM

1 1
2
log2 (e)ΓI Pe = 10–6 2-PAM

1 log ΓI
2 2
0
–10 –5 0 5 10 15 20 25 30 35 40
ΓI (dB)

Figure 6.14 Capacity Cels of an ideal AWGN channel for Gaussian for M-PAM input signals. Source:
Reproduced with permission from Forney and Ungerboeck [14]. © 1998, IEEE.
276 Chapter 6. Digital transmission model and channel capacity

the number of symbols in the alphabet  doubles, choosing 4-PAM modulation, we see that the coded
transmission of 1 bit of information per modulation interval with rate RI = 1 is possible, and an arbitrar-
ily small error probability can be obtained for ΓI = 5 dB. This indicates that a coded 4-PAM system may
achieve a gain of about 8.5 dB in SNR over an uncoded 2-PAM system, at an error probability of 10−6 .
If the number of symbols is further increased, the additional achievable gain is negligible. Therefore
we conclude that, by doubling the number of symbols with respect to an uncoded system, we obtain in
practice the entire gain that would be expected from the expansion of the input alphabet.
We see from Figure 6.14 that for small values of ΓI the choice of a binary alphabet is almost optimum:
in fact for ΓI < 1 (0 dB), the capacity given by (6.88) is essentially equivalent to the capacity given by
(6.140) with a binary alphabet of input symbols.
For large values of ΓI , the capacity of multilevel systems asymptotically approximates a straight line
that is parallel to the capacity of the AWGN channel. The asymptotic loss of 𝜋e∕6 (1.53 dB) is due
to the choice of a uniform distribution rather than Gaussian for the set of input symbols. To achieve
the Shannon limit, it is not sufficient to use coding techniques with equally likely input symbols, no
matter how sophisticated they are: to bridge the gap of 1.53 dB, shaping techniques are required [16]
that produce a distribution of the input symbols similar to a Gaussian distribution.
Coding techniques for small ΓI and large ΓI are therefore quite different: for low ΓI , the binary codes
are almost optimum and the shaping of the constellation is not necessary; for high ΓI instead constel-
lations with more than two elements must be used. To reach capacity, coding must be extended with
shaping techniques; moreover, to reach the capacity in channels with limited bandwidth, techniques are
required that combine coding, shaping, and equalization.

Bibliography

[1] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[2] Pyndiah, R., Glavieux, A., Picart, A., and Jacq, S. (1994). Near optimum decoding of product
codes. 1994 IEEE GLOBECOM. Communications: The Global Bridge, volume 1, pp. 339–343.
[3] Tosato, F. and Bisaglia, P. (2002). Simplified soft-output demapper for binary interleaved COFDM
with application to HIPERLAN/2. 2002 IEEE International Conference on Communications. Con-
ference Proceedings. ICC 2002 (Cat. No. 02CH37333), Volume 2, pp. 664–668.
[4] Alhabsi, A.H. (2005). Spectrally efficient modulation and turbo coding for communication systems.
[5] Pursley, M.B. and Shea, J.M. (1997). Bit-by-bit soft-decision decoding of trellis-coded M-DPSK
modulation. IEEE Communications Letters 1: 133–135.
[6] Blasiak, D. (1995). A soft decision metric for coherently detected, differentially encoded MPSK
systems. Proceedings 1995 IEEE 45th Vehicular Technology Conference. Countdown to the Wire-
less 21st Century, Volume 2, pp. 664–668.
[7] Stuber, G.L. (1988). Soft decision direct-sequence DPSK receivers. IEEE Transactions on Vehicular
Technology 37: 151–157.
[8] Gallager, R.G. (1968). Information Theory and Reliable Communication. New York, NY: Wiley.
[9] Cover, T.M. and Thomas, J. (1991). Elements of Information Theory. New York, NY: Wiley.
[10] Shannon, C.E. (1948). A mathematical theory of communication. Bell System Technical Journal 27
(Part I): 379–427 and (Part II) 623–656.
[11] Franks, L.E. (1981). Signal Theory, revised ed. Stroudsburg, PA: Dowden and Culver.
[12] Foschini, G.J. and Gans, M.J. (1998). On limits of wireless communications in a fading environ-
ment when using multiple antennas. Wireless Personal Communications 6: 311–335.
[13] Telatar, E. (1999). Capacity of multi–antenna Gaussian channels. European Transactions on
Telecommunications 10: 585–595.
6.A. Gray labelling 277

[14] Forney, G.D. Jr. and Ungerboeck, G. (1998). Modulation and coding for linear Gaussian channels.
IEEE Transactions on Information Theory 44: 2384–2415.
[15] Ungerboeck, G. (1982). Channel coding with multilevel/phase signals. IEEE Transactions on Infor-
mation Theory 28: 55–67.
[16] Forney, G.D. Jr. (1992). Trellis shaping. IEEE Transactions on Information Theory 38: 281–300.
[17] Simon, M.K. and Alouini, M.-S. (2000). Exponential-type bounds on the generalized Marcum
Q-function with application to error probability analysis over fading channels. IEEE Transactions
on Communications 48: 359–366.
[18] McGee, W.F. (1970). Another recursive method of computing the Q–function. IEEE Transactions
on Information Theory 16: 500–501.

Appendix 6.A Gray labelling

In this appendix, we give the procedure to construct a list of 2n binary words of n bits, where adjacent
words differ in only one bit.
The case for n = 1 is immediate. We have two words with two possible values
0
(6.141)
1

The list for n = 2 is constructed by considering first the list of (1∕2)22 = 2 words that are obtained by
appending a 0 in front of the words of the list (6.141):
0 0
(6.142)
0 1
The remaining two words are obtained by inverting the order of the words in (6.141) and appending a 1
in front:
1 1
(6.143)
1 0
The final result is the following list of words:
0 0
0 1
(6.144)
1 1
1 0
Iterating the procedure for n = 3, the first four words are obtained by repeating the list (6.144) and
appending a 0 in front of the words of the list.
Inverting then the order of the list (6.144) and appending a 1 in front, the final result is the list of eight
words
0 0 0
0 0 1
0 1 1
0 1 0
(6.145)
1 1 0
1 1 1
1 0 1
1 0 0
278 Chapter 6. Digital transmission model and channel capacity

It is easy to extend this procedure to any value of n. By induction, it is just as easy to prove that two
adjacent words in each list differ by 1 bit at most.

Appendix 6.B The Gaussian distribution and Marcum functions

6.B.1 The Q function

The probability density function of a Gaussian variable 𝑤 with mean 𝚖 and variance 𝜎 2 is given by
2
1 − (b−𝚖)2
p𝑤 (b) = √ e 2𝜎
(6.146)
2𝜋𝜎
We define normalized Gaussian distribution (𝚖 = 0 and 𝜎 2 = 1) the function
a a
1 −b2 ∕2
Φ(a) = p𝑤 (b)db = √ e db (6.147)
∫−∞ ∫−∞ 2𝜋
It is often convenient to use the complementary Gaussian distribution function, defined as
+∞
1 −b2 ∕2
Q(a) = 1 − Φ(a) = √ e db (6.148)
∫a 2𝜋
Two other functions that are widely used are the error function
√
a 2
erf(a) = −1 + 2 ∫−∞ p𝑤 (b)db
+∞ (6.149)
= 1 − 2 ∫ √ √1 e−b db
2

a 2 2𝜋

and the complementary error function

erfc(a) = 1 − erf(a) (6.150)

which are related to Φ and Q by the following equations

[ ( )]
1 a
Φ(a) = 1 + erf √ (6.151)
2 2
( )
1 a
Q(a) = erfc √ (6.152)
2 2
In Table 6.2, the values assumed by the complementary Gaussian distribution are given for values of
the argument between 0 and 8. We present below some bounds of the Q function.
( ) ( 2)
1 1 a
bound1 ∶ Q1 (a) = √ 1 − 2 exp − (6.153)
2𝜋a a 2
( 2)
1 a
bound2 ∶ Q2 (a) = √ exp − (6.154)
2𝜋a 2
( 2)
1 a
bound3 ∶ Q3 (a) = exp − (6.155)
2 2
The Q function and the above bounds are illustrated in Figure 6.15.
6.B. The Gaussian distribution and Marcum functions 279

Table 6.2: Complementary Gaussian distribution.

a Q(a) a Q(a) a Q(a)

0.0 5.0000(−01)a 2.7 3.4670(−03) 5.4 3.3320(−08)
0.1 4.6017(−01) 2.8 2.5551(−03) 5.5 1.8990(−08)
0.2 4.2074(−01) 2.9 1.8658(−03) 5.6 1.0718(−08)
0.3 3.8209(−01) 3.0 1.3499(−03) 5.7 5.9904(−09)
0.4 3.4458(−01) 3.1 9.6760(−04) 5.8 3.3157(−09)
0.5 3.0854(−01) 3.2 6.8714(−04) 5.9 1.8175(−09)
0.6 2.7425(−01) 3.3 4.8342(−04) 6.0 9.8659(−10)
0.7 2.4196(−01) 3.4 3.3693(−04) 6.1 5.3034(−10)
0.8 2.1186(−01) 3.5 2.3263(−04) 6.2 2.8232(−10)
0.9 1.8406(−01) 3.6 1.5911(−04) 6.3 1.4882(−10)
1.0 1.5866(−01) 3.7 1.0780(−04) 6.4 7.7688(−11)
1.1 1.3567(−01) 3.8 7.2348(−05) 6.5 4.0160(−11)
1.2 1.1507(−01) 3.9 4.8096(−05) 6.6 2.0558(−11)
1.3 9.6800(−02) 4.0 3.1671(−05) 6.7 1.0421(−11)
1.4 8.0757(−02) 4.1 2.0658(−05) 6.8 5.2310(−12)
1.5 6.6807(−02) 4.2 1.3346(−05) 6.9 2.6001(−12)
1.6 5.4799(−02) 4.3 8.5399(−06) 7.0 1.2798(−12)
1.7 4.4565(−02) 4.4 5.4125(−06) 7.1 6.2378(−13)
1.8 3.5930(−02) 4.5 3.3977(−06) 7.2 3.0106(−13)
1.9 2.8717(−02) 4.6 2.1125(−06) 7.3 1.4388(−13)
2.0 2.2750(−02) 4.7 1.3008(−06) 7.4 6.8092(−14)
2.1 1.7864(−02) 4.8 7.9333(−07) 7.5 3.1909(−14)
2.2 1.3903(−02) 4.9 4.7918(−07) 7.6 1.4807(−14)
2.3 1.0724(−02) 5.0 2.8665(−07) 7.7 6.8033(−15)
2.4 8.1975(−03) 5.1 1.6983(−07) 7.8 3.0954(−15)
2.5 6.2097(−03) 5.2 9.9644(−08) 7.9 1.3945(−15)
2.6 4.6612(−03) 5.3 5.7901(−08) 8.0 6.2210(−16)
a
Writing 5.0000(−01) means 5.0000 × 10−1 .

6.B.2 Marcum function

We define the first order Marcum function as

+∞ x2 +a2
Q1 (a, b) = x e− 2 I0 (ax)dx (6.156)
∫b
where I0 is the modified Bessel function of the first type and order zero, defined in (4.56).
From (6.156), two particular cases follow:
b2
Q1 (0, b) = e− 2 (6.157)
Q1 (a, 0) = 1 (6.158)
Moreover, for b ≫ 1 and b ≫ b − a the following approximation holds
Q1 (a, b) ≃ Q(b − a) (6.159)
where the Q function is given by (6.148).
280 Chapter 6. Digital transmission model and channel capacity

100 Q2(a)
Q(a)
10–1

10–2 Q1(a) Q3(a)

10–3
Q(a)

10–4

10–5

10–6

10–7
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
a

Figure 6.15 The Q function and relative bounds.

A useful approximation valid for b ≫ 1, a ≫ 1, b ≫ b − a > 0, is given by

1 + Q1 (a, b) − Q1 (b, a) ≃ 2Q(b − a) (6.160)
We also give the Simon bound [17]
(b+a)2 (b−a)2
e− 2 ≤ Q1 (a, b) ≤ e− 2 , b>a>0 (6.161)
and [ 2
]
1 − (a−b) (a+b)2
1− e 2 − e− 2 ≤ Q1 (a, b), a>b≥0 (6.162)
2
We observe that in (6.161), the upper bound is very tight, and the lower bound for a given value of
b becomes looser as a increases. In (6.162), the lower bound is very tight. A recursive method for
computing the Marcum function is given in [18].
Chapter 7

Single-carrier modulation

This chapter is split into two parts. The first part considers digital pulse amplitude modulation (PAM)
and quadrature amplitude modulation (QAM) for continuous transmission by using a single carrier,
taking into account the possibility that the channel may distort the transmitted signal [1–3]. The Nyquist
criterion will be given, which states conditions on the pulse at the receiver for the absence of intersymbol
interference. The second part instead first presents several equalization methods to compensate for linear
distortion introduced by the channel. Then, we analyse detection methods that operate on sequences of
samples. Last, we present techniques to estimate the discrete-time channel impulse response, a starting
point of receiver design.

7.1 Signals and systems

With reference to the general scheme of Figure 6.1, here we omit the channel encoder to simplify the
analysis, hence, the information message {b𝓁 } is input to the bit mapper to yield the data sequence {ak }.

7.1.1 Baseband digital transmission (PAM)

The fundamental blocks of a baseband digital transmission system are illustrated in Figure 7.1, while
signals at the various points of a quaternary PAM system are shown in Figure 7.2.

Modulator
For a PAM system, the modulator associates the symbol ak , having alphabet (6.4), with the amplitude
of a given pulse hTx :
ak → ak hTx (t − kT) (7.1)
Therefore, the modulated signal s that is input to the transmission channel is given by
∑
+∞
s(t) = ak hTx (t − kT) (7.2)
k=−∞

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
282 Chapter 7. Single-carrier modulation

data inverse
bit mapper t0 + kT detector bit mapper ^
bl ak s (t) sCh (t) r(t) rR(t) yk a^k bl
BMAP hTx gCh gRc IBMAP
Tb T T T Tb
[ w (f)]

transmitter channel w(t) receiver

Figure 7.1 Block diagram of a baseband digital transmission system.

ak 3
transmitted
symbols 1 −1 −3 −1

0 T 2T 3T 4T t
transmitted
signal s(t) hTx(t)

T 2T 3T
0 4T t 0 t

channel output
sCh(t) qCh (t)
signal

T 2T 3T
0 4T t 0 t

equalized
sR(t) qR(t)
signal
h0

T 2T 3T
0 4T t 0 t0 t

received
rR(t) 2 h0
signal

detected
a^k 3
− 2 h0
symbols
1 −1 −3 −1

t0 t0 +T t0 +2T t0 + 3T t0 +4T t

Figure 7.2 Signals at various points of a quaternary PAM transmission system with alphabet
 = {−3, −1, 1, 3}.
7.1. Signals and systems 283

Transmission channel
The transmission channel is assumed to be linear and time invariant, with impulse response gCh . There-
fore, the desired signal at the output of the transmission channel still has a PAM structure. From the
relation,
sCh (t) = gCh ∗ s(t) (7.3)
we define
qCh (t) = hTx ∗ gCh (t) (7.4)
then, we have
∑
+∞
sCh (t) = ak qCh (t − kT) (7.5)
k=−∞

The transmission channel introduces an effective noise 𝑤. Therefore, the signal at the input of the receive
filter is given by
r(t) = sCh (t) + 𝑤(t) (7.6)

Receiver
The receiver consists of three functional blocks:
1. Amplifier-equalizer filter: This block is assumed linear and time invariant with impulse response
gRc . Then, the desired signal is given by
sR (t) = gRc ∗ sCh (t) (7.7)
Let the overall impulse response of the system be
qR (t) = qCh ∗ gRc (t) = hTx ∗ gCh ∗ gRc (t) (7.8)
then
∑
+∞
sR (t) = ak qR (t − kT) (7.9)
k=−∞

In the presence of noise,

rR (t) = sR (t) + 𝑤R (t) (7.10)
where 𝑤R (t) = 𝑤 ∗ gRc (t).
2. Sampler: Sampling rR at instants t0 + kT yields1 :
∑
∞
yk = rR (t0 + kT) = ai qR (t0 + (k − i)T) + 𝑤R (t0 + kT) (7.11)
i=−∞

Sometimes, yk can be simplified as

yk = h0 ak + 𝑤R,k (7.12)
where h0 = qR (t0 ) is the amplitude of the overall impulse response at the sampling instant t0 and
𝑤R,k = 𝑤R (t0 + kT). The parameter t0 is called timing phase, and its choice is fundamental for
system performance. We scale now yk by h0 to yield
y 𝑤R,k
yk = k = ak + (7.13)
h0 h0

1
To simplify the notation, the sample index k, associated with the instant t0 + kT, here appears as a subscript.
284 Chapter 7. Single-carrier modulation

3. Threshold detector: From the sequence {yk }, we detect the transmitted sequence {ak }. The simplest
structure is the instantaneous non-linear threshold detector:
â k =  [yk ] (7.14)
where  [yk ] is the quantizer characteristic with yk ∈ ℝ and â k ∈ , alphabet of ak . An example
of quantizer characteristic for  = {−3, −1, 1, 3} is given in Figure 7.3.
Indeed, under some assumptions examined in Section 6.2.1, the threshold detector, implementing the
minimum distance detection criterion, is optimum.

â k

−2 2 y–k

−2

Figure 7.3 Characteristic of a threshold detector for quaternary symbols with alphabet
 = {−3, −1, 1, 3}.

From the sequence {̂ak }, using an inverse bit mapper (IBMAP), the detected binary information mes-
sage {b̂ 𝓁 } is obtained. Incidentally, as yk is in general affected by interference from other symbols than
ak and noise, it may be â k ≠ ak .

Power spectral density

The PAM signals found in various points of the system, sR , sCh , and s, have the following structure:
∑
+∞
s(t) = ak q(t − kT) (7.15)
k=−∞

where q is the impulse response of a suitable filter. In other words, a PAM signal s may be regarded as
a signal generated by an interpolator filter with impulse response q(t), t ∈ ℝ, as shown in Figure 7.4.
From the spectral analysis (see Example 1.7.9 on page 49), we know that s is a cyclostationary process
with average power spectral density (PSD) given by (see (1.323)):
|1 |2
 s (f ) = || (f )|| a (f ) (7.16)
|T |
where a is the PSD of the message and  is the Fourier transform of q.

ak s(t)
q
T

Figure 7.4 The PAM signal as output of an interpolator filter.

7.1. Signals and systems 285

From Figure 7.4, it is immediate to verify that by filtering s we obtain a signal that is still PAM, with
a pulse given by the convolution of the filter impulse responses. The PSD of a filtered PAM signal is
obtained by multiplying a (f ) in (7.16) by the square magnitude of the filter frequency response. As
a (f ) is periodic of period 1∕T, then the bandwidth B of the transmitted signal is equal to that of hTx .

Example 7.1.1 (PSD of an i.i.d. symbol sequence)

Let {ak } be a sequence of i.i.d. symbols with values from the alphabet
 = {𝛼1 , 𝛼2 , … , 𝛼M } (7.17)
and p(𝛼), 𝛼 ∈ , be the probability distribution of each symbol.
The mean value and the statistical power of the sequence are given by
∑ ∑
𝚖a = 𝛼p(𝛼), 𝙼a = |𝛼|2 p(𝛼) (7.18)
𝛼∈ 𝛼∈

Consequently, 𝜎a2 = 𝙼a − |𝚖a |2 .

Following Example 1.5.1 on page 34, that describes the decomposition of the PSD of a message into
ordinary and impulse functions, the decomposition of the PSD of s is given by
(c) |1 |2 𝜎2
 s (f ) = || (f )|| 𝜎a2 T = |(f )|2 a (7.19)
|T | T
and
( )
(d) |1 |2 ∑
+∞
𝓁
 s (f ) = || (f )|| |𝚖a |2 𝛿 f−
|T | 𝓁=−∞
T
( ) ( )
| 𝚖 |2 ∑ || 𝓁 ||
+∞ 2
𝓁
= || a || | | 𝛿 f− (7.20)
| T | 𝓁=−∞ || T || T
We note that spectral lines occur in the PSD of s if 𝚖a ≠ 0 and (f ) is non-zero at frequencies multiple
of 1∕T.
Typically, the presence of spectral lines is not desirable for a transmission scheme as it implies a higher
transmit power without improving system performance. We obtain 𝚖a = 0 by choosing an alphabet with
symmetric values with respect to zero, and by assigning equal probabilities to antipodal values.
In some applications, the PSD of the transmitted signal is shaped by introducing correlation among
transmitted symbols by a line encoder, see (7.16): in this case, the sequence has memory. We refer the
reader to Appendix 7.12 for a description of the more common line codes.

7.1.2 Passband digital transmission (QAM)

We consider the QAM transmission system illustrated in Figure 7.5. Now, values of ak are complex, and
two examples of constellations with corresponding bit map are shown in Figure 6.2.

Modulator
Typically, the pulse hTx is real-valued, however, the baseband modulated signal is complex-valued:
∑
+∞
s(bb) (t) = ak hTx (t − kT)
k=−∞
(7.21)
∑
+∞
∑
+∞
= ak,I hTx (t − kT) + j ak,Q hTx (t − kT)
k=−∞ k=−∞
286 Chapter 7. Single-carrier modulation

cos (ω0t + φ0) cos (ω0t + φ1)

–sin (ω0t + φ0) –sin (ω0t + φ1)

Figure 7.5 Block diagram of a passband digital transmission system.

Let f0 (𝜔0 = 2𝜋f0 ) and 𝜑0 be, respectively, the carrier frequency (radian frequency) and phase. We define
the following signal and its corresponding Fourier transform
1  1
s(+) (t) = s(bb) (t)ej(𝜔0 t+𝜑0 ) ←−→  (+) (f ) =  (bb) (f − f0 )ej𝜑0 (7.22)
2 2
then the real-valued transmitted signal is given by

s(t) = 2Re{s(+) (t)} ←−→ (f ) =  (+) (f ) +  (+)∗ (−f ) (7.23)
The transformation in the frequency domain from s(bb) to s is illustrated in Figure 7.6.

S(bb)(f )

−B 0 B f
S (+)( f)

0 f0 − B f0 f0 + B f
S (f )

−f0 − B −f0 −f0 + B 0 f0 − B f0 f0 +B f

Figure 7.6 Fourier transforms of baseband signal and modulated signal.

Power spectral density

From the analysis leading to (1.320), s(bb) is a cyclostationary process of period T with an average PSD
(see (1.323)):
|1 |2
 s(bb) (f ) = || Tx (f )|| a (f ) (7.24)
|T |
7.1. Signals and systems 287

Moreover, starting from a relation similar to (1.228), we get that s is a cyclostationary random process
with an average PSD given by2
[ ]
1
 s (f ) =  s(bb) (f − f0 ) +  s(bb) (−f − f0 ) (7.25)
4
We note that the bandwidth of the transmitted signal is equal to twice the bandwidth of hTx .

Three equivalent representations of the modulator

1. From (7.22) and (7.23), using (7.21), it turns out

[ ]
s(t) = Re s(bb) (t)ej(𝜔0 t+𝜑0 )
[ ]
∑
+∞
(7.26)
= Re ej(𝜔0 t+𝜑0 ) ak hTx (t − kT)
k=−∞

The block-diagram representation of (7.26) is shown in Figure 7.7. As s(bb) is in general a

complex-valued signal, an implementation based on this representation requires a processor
capable of complex arithmetic.
2. As
ej(𝜔0 t+𝜑0 ) = cos(𝜔0 t + 𝜑0 ) + j sin(𝜔0 t + 𝜑0 ) (7.27)
(7.26) becomes:
∑
+∞
∑
+∞
s(t) = cos(𝜔0 t + 𝜑0 ) ak,I hTx (t − kT) − sin(𝜔0 t + 𝜑0 ) ak,Q hTx (t − kT) (7.28)
k=−∞ k=−∞

The block-diagram representation of (7.28) is shown in Figure 7.5. The implementation of a QAM
transmitter based on (7.28) is discussed in Appendix 7.D.
3. Using the polar notation ak = |ak |ej𝜃k , (7.26) becomes
[ ]
∑
+∞
s(t) = Re ej(𝜔0 t+𝜑0 ) |ak |ej𝜃k hTx (t − kT)
k=−∞
[ ]
∑
+∞
= Re |ak |ej(𝜔0 t+𝜑0 +𝜃k ) hTx (t − kT) (7.29)
k=−∞

∑
+∞
= |ak | cos(𝜔0 t + 𝜑0 + 𝜃k )hTx (t − kT)
k=−∞

If |ak | is a constant, we obtain the phase-shift keying (PSK) signal, where the information bits
select only the value of the carrier phase.

LPF
bl ak s(bb) (t) s (t)
BMAP hTx Re [ ]
Tb T

e j(ω0 t + φ0)

Figure 7.7 QAM transmitter: complex-valued representation.

2
The result (7.25) needs a clarification. We assume that 𝚛s(bb) s(bb)∗ (t, t − 𝜏) = 0 is satisfied, as for example in the case of
QAM with i.i.d. circularly symmetric symbols (see (1.332)). From the equation (similar to (1.228)) that relates 𝚛s to
𝚛s(bb) and 𝚛s(bb) s(bb)∗ , as the crosscorrelations are zero, we find that the process s is cyclostationary in t of period T. Taking
the average correlation in a period T, the results (7.24) and (7.25) follow.
288 Chapter 7. Single-carrier modulation

Coherent receiver
In the absence of noise, the general scheme of a coherent receiver is shown in Figure 7.8.

data
detector
LPF t0 +kT ^
sCh (t) sMo (t) sR (t) yk a^k bl
gRc IBMAP
T T Tb

e−j(ω0 t + φ1)

Figure 7.8 QAM receiver: complex-valued representation.

The received signal is given by


sCh (t) = s ∗ gCh (t) ←−→ Ch (f ) = (f )Ch (f ) (7.30)

First the received signal is translated to baseband by a frequency shift,


sM0 (t) = sCh (t)e−j(𝜔0 t+𝜑1 ) ←−→ M0 (f ) = Ch (f + f0 )e−j𝜑1 (7.31)

then it is filtered by a lowpass filter (LPF), gRc ,


sR (t) = sM0 ∗ gRc (t) ←−→ R (f ) = M0 (f )Rc (f ) (7.32)

We note that, if gRc is a non-distorting ideal filter with unit gain, then sR (t) = (1∕2)s(bb)
Ch
(t). In the par-
ticular case where gRc is a real-valued filter, then the receiver in Figure 7.8 is simplified into that of
Figure 7.5. Figure 7.9 illustrates these transformations.
We note that in the above analysis, as the channel may introduce a phase offset, the receive carrier
phase 𝜑1 may be different from the transmit carrier phase 𝜑0 .

7.1.3 Baseband equivalent model of a QAM system

Recalling the relations of Figure 1.21, we illustrate the baseband equivalent scheme with reference to
Figure 7.10:3 by assuming that the transmit and receive carriers have the same frequency, we can study
QAM systems by the same method that we have developed for PAM systems.

Signal analysis
We refer to Section 1.5.4 for an analysis of passband signals; we recall here that if for f > 0 the PSD of
𝑤, 𝑤 (f ), is an even function around the frequency f0 , then the real and imaginary parts of 𝑤(bb) (t) =
𝑤I (t) + j𝑤Q (t) the PSD of 𝑤I and 𝑤Q are
{
1 2𝑤 (f + f0 ), f ≥ −f0
𝑤I (f ) = 𝑤Q (f ) = 𝑤(bb) (f ) = (7.33)
2 0, elsewhere

3
We note that the term ej𝜑0 has been moved to the receiver; Therefore, the signals s(bb) and r(bb) in Figure 7.10 are defined
apart from the term ej𝜑0 . This is the same as assuming as reference carrier ej(2𝜋f0 t+𝜑0 ) .
7.1. Signals and systems 289

Figure 7.9 Frequency responses of channel and signals at various points of the receiver.

( ) ( )( 0
( )( ) 1 ( )
() ) ()
2
1 (φ1 φ )
2

( ) [
() ( ) 4
( ) ( 0) ( 0) ]

Figure 7.10 Baseband equivalent model of a QAM transmission system.

Moreover, 𝑤I 𝑤Q (f ) = 0, that is 𝑤I and 𝑤Q are uncorrelated, hence,

1 2
𝜎𝑤2 I = 𝜎𝑤2 Q = 𝜎 = 𝜎𝑤2 (7.34)
2 𝑤(bb)
To simplify the analysis, for the study of a QAM system, we will adopt the PAM model of Figure 7.1,
assuming that all signals and filters are in general complex. −j(𝜑1 −𝜑0 ) appears
√ We note that the factor (1∕2)e
in Figure 7.10. We will include −j(𝜑1 −𝜑0 ) ∕ 2 in the impulse response of the transmission
√ the factor e
channel gCh , and the factor 1∕ 2 in the impulse response gRc . Consequently, the additive noise has a
PSD equal to (1∕2)𝑤(bb) (f ) = 2𝑤 (f + f0 ) for f ≥ −f0 . Therefore, the scheme of Figure 7.1 holds also
for QAM in the presence of additive noise: the only difference is that in the case of a QAM system
290 Chapter 7. Single-carrier modulation

the noise is complex-valued with uncorrelated in-phase and quadrature components, each having PSD
𝑤 (f + f0 ) for f ≥ −f0 .
Hence, the scheme of Figure 7.11 is a reference scheme for both PAM and QAM, where
⎧  (f ) for PAM
⎪ Ch
C (f ) = ⎨ e−j(𝜑1 −𝜑0 )  (f + f )1(f + f ) for QAM (7.35)
√
⎪ 2
Ch 0 0
⎩
We note that for QAM, we have
e−j(𝜑1 −𝜑0 ) (bb)
gC (t) = √ gCh (t) (7.36)
2 2

0
s(t) (t) (t) (t)

(t) ʹ (t) ʹ (t) [2 ( 0) 1 ( 0) ]

Figure 7.11 Baseband equivalent model of PAM and QAM transmission systems.

With reference to the scheme of Figure 7.8, the relation between the impulse responses of the receive
filters is
1
gRc (t) = √ gRc (t) (7.37)
2
We summarize the definitions of the various signals in QAM systems.
1. Sequence of input symbols, {ak }, sequence of symbols with values from a complex-valued alphabet
. In PAM systems, the symbols of the sequence {ak } assume real values.
2. Modulated signal,4
∑
+∞
s(t) = ak hTx (t − kT) (7.38)
k=−∞
3. Signal at the channel output,
∑
+∞
sC (t) = ak qC (t − kT) qC (t) = hTx ∗ gC (t) (7.39)
k=−∞

4. Circularly-symmetric, complex-valued, additive Gaussian noise, 𝑤C (t) = 𝑤′I (t) + j𝑤′Q (t), with
PSD 𝑤C . In the case of white noise, it is5 :
N0
𝑤′ (f ) = 𝑤′ (f ) = (V2 /Hz) (7.40)
I Q 2
and
𝑤C (f ) = N0 (V2 /Hz) (7.41)
In the model of PAM systems, only the component 𝑤′I is considered.

4
We point out that for QAM s is in fact s(bb) .
5
In fact (7.40) should include the condition f > −f0 . Because the bandwidth of gRc is smaller than f0 , this condition can
be omitted.
7.1. Signals and systems 291

5. Received or observed signal,

rC (t) = sC (t) + 𝑤C (t) (7.42)
6. Signal at the output of the complex-valued amplifier-equalizer filter gRc ,
rR (t) = sR (t) + 𝑤R (t) (7.43)
where
∑
+∞
sR (t) = ak qR (t − kT) (7.44)
k=−∞

with
qR (t) = qC ∗ gRc (t) and 𝑤R (t) = 𝑤C ∗ gRc (t) (7.45)
In PAM systems, gRc is a real-valued filter.
7. Signal at the decision point at instant t0 + kT,
yk = rR (t0 + kT) (7.46)
8. Data detector: Based on the sequence yk , we examine various methods for data detection. Here we
recall the simplest one, which is based on the fact that yk can be approximated as (see Section 7.3)
yk = h0 ak + 𝑤R,k (7.47)
where
𝑤R,k = 𝑤R (t0 + kT) (7.48)
Scaling yk by h0 yields 𝑤R,k
yk = ak + (7.49)
h0
If 𝑤R,k is i.i.d., decision is based on reporting yk on the data constellation and deciding for the
closest symbol value. Two examples of decision region are reported in Figure 6.4.
9. Sequence of detected symbols,
{̂ak } (7.50)

7.1.4 Characterization of system elements

We consider some characteristics of the signals in the scheme of Figure 7.11.

Transmitter
The choice of the transmit pulse is quite important because it determines the bandwidth of the system
(see (7.16) and (7.25)). Two choices are shown in Figure 7.12, where
( T)
t−
1. hTx (t) = 𝚠T (t) = rect T 2 , with wide spectrum;
2. hTx with longer duration and smaller bandwidth.

Transmission channel
The transmission channel is modelled as a time invariant linear system. Therefore, it is represented
by a filter having impulse response gCh . As described the majority of channels are characterized by a
292 Chapter 7. Single-carrier modulation

1 1
0.8 0.8

HTx( f )
hTx(t)

0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 2T 4T 6T 0 2/T 4/T 6/T
t f
(a)

1 1
0.8
0.8
0.6 HTx( f )
hTx(t)

0.6
0.4
0.4
0.2
0 0.2
−0.2 0
0 2T 4T 6T 0 2/T 4/T 6/T
t f
(b)

Figure 7.12 Two examples of transmit pulse hTx . (a) Wideband. (b) Narrowband.

frequency response having a null at DC, the shape of Ch (f ) is as represented in Figure 7.13, where the
passband goes from f1 to f2 . For transmission over cables f1 may be of the order of a few hundred Hz,
whereas for radio links f1 may be in the range of MHz or GHz. Consequently, PAM (possibly using a
line code) as well as QAM transmission systems may be considered over cables; for transmission over
radio, instead a QAM system may be used, assuming as carrier frequency f0 the centre frequency of the
passband (f1 , f2 ). In any case, the channel is bandlimited with a finite bandwidth f2 − f1 .
With reference to the general model of Figure 7.11, we adopt the polar notation for C :
C (f ) = |C (f )|ej arg C (f ) (7.51)
Let B be the bandwidth of s. According to (1.78), a channel presents ideal characteristics, known as
Heaviside conditions for the absence of distortion, if the following two properties are satisfied:
1. the magnitude response is a constant for |f | < B,
|C (f )| = 0 for |f | < B (7.52)
2. the phase response is proportional to f for |f | < B,
arg C (f ) = −2𝜋f t0 for |f | < B (7.53)
Under these conditions, s is reproduced at the output of the channel without distortion, that is:
sC (t) = 0 s(t − t0 ) (7.54)
7.1. Signals and systems 293

(f)

−f2 −f 0 −f1 f1 f0 f2 f

(bb)
( f)

f1− f0 f2− f0 f

Figure 7.13 Frequency response of the transmission channel. (a) Channel. (b) Baseband equivalent
channel.

In practice, channels introduce both amplitude distortion and phase distortion. An example of frequency
response of a radio channel is given in Figure 4.17: the overall effect is that the signal sC may be very
different from s.
In short, for channels encountered in practice conditions (7.52) and (7.53) are too stringent; for PAM
and QAM transmission systems, we will refer instead to the Nyquist criterion (7.72).

Receiver
We return to the receiver structure of Figure 7.11, consisting of a filter gRc followed by a sampler with
sampling rate 1∕T, and a data detector.
In general, if the frequency response of the receive filter Rc (f ) contains a factor C(ej2𝜋fT ), periodic of
period 1∕T, such that the following factorization holds:
Rc (f ) = M (f )C(ej2𝜋fT ) (7.55)
where M (f ) is a generic function, then the filter-sampler block before the data detector of Figure 7.11
can be represented as in Figure 7.14, where the sampler is followed by a discrete-time filter and the
impulse response c is the inverse discrete-time Fourier transform of C(ej2𝜋fT ). It is easy to prove that in
the two systems, the relation between rC and yk is the same.
294 Chapter 7. Single-carrier modulation

t0 + kT
rC (t) x(t) xk yk a^k
gM c
T T T

Figure 7.14 Receiver structure with analog and discrete-time filters.

Ideally, in the system of Figure 7.14, yk should be equal to ak . In practice, linear distortion and additive
noise, the only disturbances considered here, may determine a significant deviation of yk from the desired
symbol ak .
The last element in the receiver is the data detector. The simplest scheme is the threshold detector, as
described earlier.

7.2 Intersymbol interference

Discrete-time equivalent system

From (7.43), we define
∑
+∞
sR,k = sR (t0 + kT) = ai qR (t0 + (k − i)T) (7.56)
i=−∞

and
𝑤R,k = 𝑤R (t0 + kT) (7.57)
Then, from (7.56), at the decision point, the generic sample is expressed as
yk = sR,k + 𝑤R,k (7.58)
Introducing the version of qR , time-shifted by t0 , as
h(t) = qR (t0 + t) (7.59)
and defining
hi = h(iT) = qR (t0 + iT) (7.60)
it follows that
∑
+∞
sR,k = ai hk−i = ak h0 + 𝚒k (7.61)
i=−∞

where
∑
+∞
𝚒k = ai hk−i = · · · + h−1 ak+1 + h1 ak−1 + h2 ak−2 + · · · (7.62)
i=−∞, i≠k

represents the intersymbol interference (ISI). The coefficients {hi }i≠0 are called interferers. Moreover,
(7.58) becomes
yk = ak h0 + 𝚒k + 𝑤R,k (7.63)
We observe that, even in the absence of noise, the detection of ak from yk by a threshold detector takes
place in the presence of the term 𝚒k , which behaves as a disturbance with respect to the desired term
ak h0 .
7.2. Intersymbol interference 295

For the analysis, it is often convenient to approximate 𝚒k as noise with a Gaussian distribution: the
more numerous and similar in amplitude are the interferers, the more valid is this approximation. In the
case of i.i.d. symbols, the first two moments of 𝚒k are easily determined.
∑
+∞
Mean value of 𝚒k ∶ 𝚖𝚒 = 𝚖a hi (7.64)
i=−∞, i≠0

∑
+∞
Variance of 𝚒k ∶ 𝜎𝚒2 = 𝜎a2 |hi |2 (7.65)
i=−∞, i≠0

From (7.58), with {sR,k } given by (7.61), we derive the discrete-time equivalent scheme, with period T
(see Figure 7.15), that relates the signal at the decision point to the data transmitted over a discrete-time
channel with impulse response given by the sequence {hi }, called overall discrete-time equivalent
impulse response of the system.

Figure 7.15 Discrete-time equivalent scheme, with period T, of a QAM system.

Concerning the additive noise {𝑤R,k },6 being

𝑤R (f ) = 𝑤C (f )|Rc (f )|2 (7.66)
the PSD of {𝑤R,k } is given by
∑
+∞ ( )
1
𝑤R,k (f ) = 𝑤R f − 𝓁 (7.67)
𝓁=−∞
T
In any case, the variance of 𝑤R,k is equal to that of 𝑤R and is given by
+∞
𝜎𝑤2 R,k = 𝜎𝑤2 R = 𝑤C (f )|Rc (f )|2 df (7.68)
∫−∞
In particular, the variance per dimension of the noise is given by
PAM 𝜎I2 = E[𝑤2R,k ] = 𝜎𝑤2 R (7.69)
[ ] [ ] 1
QAM 𝜎I2 = E (Re[𝑤R,k ])2 = E (Im[𝑤R,k ])2 = 𝜎𝑤2 R (7.70)
2
In the case of PAM (QAM) transmission over a channel with white noise, where 𝑤C (f ) = N0 ∕2 (N0 ),
(7.68) yields a variance per dimension equal to
N
𝜎I2 = 0 EgRc (7.71)
2
where EgRc is the energy of the receive filter. We observe that (7.71) holds for PAM as well as for QAM.

Nyquist pulses
The problem we wish to address consists in finding the conditions on the various filters of the system,
so that, in the absence of noise, yk is a replica of ak . The solution is the Nyquist criterion for the absence
of distortion in digital transmission.
6
See Observation 1.4 on page 43.
296 Chapter 7. Single-carrier modulation

From (7.61), to obtain yk = ak , it must be

Nyquist criterion in the time domain

{
h0 = 1
(7.72)
hi = 0, i≠0
and ISI vanishes. A pulse h that satisfies the conditions (7.72) is said to be a Nyquist pulse with modu-
lation interval T
The conditions (7.72) have their equivalent in the frequency domain. They can be derived using the
Fourier transform of the sequence {hi } (1.23),
( )
∑
+∞
1 ∑
+∞
𝓁
hi e−j2𝜋fiT =  f− (7.73)
i=−∞
T 𝓁=−∞ T
where (f ) is the Fourier transform of h. From the conditions, (7.72) the left-hand side of (7.73) is equal
to 1, hence, the condition for the absence of ISI is formulated in the frequency domain for the generic
pulse h as

Nyquist criterion in the frequency domain

( )
∑
+∞
𝓁
 f− =T (7.74)
𝓁=−∞
T
From (7.74), we deduce an important fact: the Nyquist pulse with minimum bandwidth is given by
t  f
h(t) = h0 sinc ←−→ (f ) = Th0 rect (7.75)
T 1∕T

Definition 7.1
The frequency 1∕(2T), which coincides with half of the modulation frequency, is called Nyquist fre-
quency. ◽

A family of Nyquist pulses widely used in telecommunications is composed of the raised cosine pulses
whose time and frequency plots, for three values of the parameter 𝜌, are illustrated in Figure 7.16a.
We define
⎧ 1−𝜌
⎪ 1, 0 ≤ |x| ≤
⎪ ( ) 2
⎪ 𝜋 |x| − 1−𝜌 1 − 𝜌 1+𝜌
rcos(x, 𝜌) = ⎨ cos2 2
, < |x| ≤ (7.76)
⎪ 2 𝜌 2 2
⎪ 1+𝜌
⎪ 0, |x| >
⎩ 2
then ( )
f
(f ) = T r cos ,𝜌 (7.77)
1∕T
with inverse Fourier transform
( ) [ ( ) ( )]
t 𝜋 t 1 t 1
h(t) = sinc sinc 𝜌 + + sinc 𝜌 −
T 4 T 2 T 2
( ) ( )
t t 1
= sinc cos 𝜋𝜌 ( ) (7.78)
T T t 2
1 − 2𝜌
T
7.2. Intersymbol interference 297

1.2
1.6
roll-off =0 roll-off=0
1.4 roll-off =0.5 roll-off=0.5
roll-off =1 1 roll-off=1
1.2
1 0.8

0.8

1 H(f )
h(t)

0.6
0.6

T
0.4 0.4
0.2
0 0.2
−0.2
0
4T 3T −2T −T 0 T 2T 3T 4T 0 1/2T 1/T 3/2T
t f
(a)

1.2
1.6
roll-off =0 roll-off=0
1.4 roll-off =0.5 roll-off=0.5
roll-off =1 1 roll-off=1
1.2
1 0.8

0.8
1 H( f)
h(t)

0.6
0.6
T

0.4 0.4
0.2
0 0.2
−0.2
0
4T 3T −2T −T 0 T 2T 3T 4T 0 1/2T 1/T 3/2T
t f
(b)

Figure 7.16 Time and frequency plots of (a) raised cosine and (b) square root raised cosine pulses
for three values of the roll-off factor 𝜌.

It is easily proven that, from (7.76), the area of (f ) in (7.77) is equal to 1, that is
+∞ ( )
f
h(0) = T r cos , 𝜌 df = 1 (7.79)
∫−∞ 1∕T
and the energy is
( ) (
+∞
f 𝜌) ( 𝜌)
Eh = T 2 r cos2 , 𝜌 df = T 1 − = (0) h(0) 1 − (7.80)
∫−∞ 1∕T 4 4
We note that, from (7.77), the bandwidth of the baseband equivalent system is equal to (1 + 𝜌)∕(2T).
Consequently, for a QAM system, the required bandwidth is (1 + 𝜌)∕T.
298 Chapter 7. Single-carrier modulation

Later, we will also refer to square root raised cosine pulses, with frequency response given by
√ ( )
f
(f ) = T rcos ,𝜌 (7.81)
1∕T
and inverse Fourier transform
[ ] [ ( )] ( )
t t 1 t 1
h(t) = (1 − 𝜌)sinc (1 − 𝜌) + 𝜌 cos 𝜋 + sinc 𝜌 +
T T 4 T 4
[ ( )] ( )
t 1 t 1
+ 𝜌 cos 𝜋 − sinc 𝜌 −
[ T ] 4 [ T 4 ]
sin 𝜋(1 − 𝜌) T + 4𝜌 T cos 𝜋(1 + 𝜌) Tt
t t

= [ ( )2 ] (7.82)
𝜋 1 − 4𝜌 Tt t
T

In this case, √
+∞ ( ) ( )
f 4
h(0) = T rcos , 𝜌 df = 1 − 𝜌 1 − (7.83)
∫−∞ 1∕T 𝜋
and the pulse energy is given by
+∞ ( )
f
Eh = 2
T r cos , 𝜌 df = T (7.84)
∫−∞ 1∕T
We note that (f ) in (7.81) is not the frequency response of a Nyquist pulse. Plots of h and (f ), given
respectively by (7.82) and (7.81), for various values of 𝜌 are shown in Figure 7.16b.
The parameter 𝜌, called excess bandwidth parameter or roll-off factor, is in the range between 0 and 1.
We note that 𝜌 determines how fast the pulse decays in time.

Observation 7.1
From the Nyquist conditions, we deduce that
1. a data sequence can be transmitted with modulation rate 1∕T without errors if (f ) satisfies the
Nyquist criterion, and there is no noise;
2. the channel, with frequency response C , must have a bandwidth equal to at least 1∕(2T), otherwise,
intersymbol interference cannot be avoided.

Eye diagram
From (7.61), we observe that if the samples {hi }, for i ≠ 0, are not sufficiently small with respect to h0 ,
the ISI may result a dominant disturbance with respect to noise and impair the system performance. On
the other hand, from (7.59) and (7.60), the discrete-time impulse response {hi } depends on the choice
of the timing phase t0 (see Chapter 14) and on the pulse shape qR .
In the absence of noise, at the decision point, the sample y0 , as a function of t0 , is given by
∑
+∞
y0 = y(t0 ) = ai qR (t0 − iT)
i=−∞
= a0 qR (t0 ) + 𝚒0 (t0 ) (7.85)
where
∑
+∞
𝚒0 (t0 ) = ai qR (t0 − iT)
i=−∞, i≠0 (7.86)
= · · · + a−1 qR (t0 + T) + a1 qR (t0 − T) + a2 qR (t0 − 2T) + · · ·
is the ISI.
7.2. Intersymbol interference 299

We now illustrate, through an example, a graphic method to represent the effect of the choice of t0 for
a given pulse qR . We consider a PAM transmission system, where y(t0 ) is real: for a QAM system, both
Re [y(t0 )] and Im [y(t0 )] need to be represented. We consider a quaternary transmission with
ak = 𝛼n ∈  = {−3, −1, 1, 3} (7.87)
and pulse qR as shown in Figure 7.17.

1.5

1
qR(t)

0.5

−0.5
−T 0 T 2T 3T 4T
t

Figure 7.17 Pulse shape for the computation of the eye diagram.

In the absence of ISI, 𝚒0 (t0 ) = 0 and y0 = a0 qR (t0 ). In correspondence to each possible value 𝛼n of
a0 , the pattern of y0 as a function of t0 is shown in Figure 7.18: it is seen that the possible values of
y0 , for 𝛼n ∈ , are farther apart, therefore, they offer a greater margin against noise in correspondence
of the peak of qR , which in this example occurs at instant t0 = 1.5T. In fact, for a given t0 and for a
given message … , a−1 , a1 , a2 , …, it may result 𝚒0 (t0 ) ≠ 0, and this value is added to the desired sample
a0 qR (t0 ).
The range of variations of y0 (t0 ) around the desired sample 𝛼n qR (t0 ) is determined by the values
𝚒max (t0 ; 𝛼n ) = max 𝚒0 (t0 ) (7.88)
{ak }, a0 =𝛼n

𝚒min (t0 ; 𝛼n ) = min 𝚒0 (t0 ) (7.89)

{ak }, a0 =𝛼n

The eye diagram is characterized by the 2M profiles

{
𝚒max (t0 ; 𝛼n )
𝛼n qR (t0 ) + , 𝛼n ∈  (7.90)
𝚒min (t0 ; 𝛼n )
If the symbols {ak } are statistically independent with balanced values, that is both 𝛼n and −𝛼n belong
to , defining
𝛼max = max 𝛼n (7.91)
n
300 Chapter 7. Single-carrier modulation

3
αn = 3
2

1 αn = 1
αn qR(t0)

0
αn = − 1
−1

−2
αn =− 3
−3

−T 0 T 2T 3T 4T
t0

Figure 7.18 Desired component 𝛼n qR (t0 ) as a function of t0 , 𝛼n ∈ {−3, −1, 1, 3}.

and
∑
+∞
𝚒abs (t0 ) = 𝛼max |qR (t0 − iT)| (7.92)
i=−∞, i≠0

we have that
𝚒max (t0 ) = 𝚒abs (t0 ) (7.93)
𝚒min (t0 ) = −𝚒abs (t0 ) (7.94)
We note that both functions do not depend on a0 = 𝛼n .

3 αnqR(t0) + imax(t0;αn)

2 αnqR(t0) + imin(t0;αn)

1
y0(t0)

−1

−2

−3

−T 0 T 2T 3T 4T
t0

Figure 7.19 Eye diagram for quaternary transmission and pulse qR of Figure 7.17.
7.2. Intersymbol interference 301

For the considered pulse, the eye diagram is given in Figure 7.19. We observe that as a result of the
presence of ISI, the values of y0 may be very close to each other, and therefore, reduce considerably
the margin against noise. We also note that, in general, the timing phase that offers the largest margin
against noise is not necessarily found in correspondence of the peak of qR . In this example, however, the
choice t0 = 1.5T guarantees the largest margin against noise.
In the general case, where there exists correlation between the symbols of the sequence {ak }, it is easy
to show that 𝚒max (t0 ; 𝛼n ) ≤ 𝚒abs (t0 ) and 𝚒min (t0 ; 𝛼n ) ≥ −𝚒abs (t0 ). Consequently, the eye may be wider as
compared to the case of i.i.d. symbols.
For quaternary transmission, we show in Figure 7.20 the eye diagram obtained with a raised cosine
pulse qR , for two values of the roll-off factor.
In general, the M − 1 pupils of the eye diagram have a shape as illustrated in Figure 7.21, where two
parameters are identified: the height a and the width b. The height a is an indicator of the noise immunity
of the system. The width b indicates the immunity with respect to deviations from the optimum timing
phase. For example a raised cosine pulse with 𝜌 = 1 offers greater immunity against errors in the choice
of t0 as compared to 𝜌 = 0.125. The price we pay is a larger bandwidth of the transmission channel.
We now illustrate an alternative method to obtain the eye diagram. A long random sequence of symbols
{ak } is transmitted over the channel, and the portions of the curve y(t) = sR (t) relative to the various inter-
vals [t1 , t1 + T), [t1 + T, t1 + 2T), [t1 + 2T, t1 + 3T), …], are mapped on the same interval, for example

5
4
3
2
1
0
−1
−2
−3
−4
−5
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5
t/T
(a)

5
4
3
2
1
0
−1
−2
−3
−4
−5
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.5
t/T
(b)

Figure 7.20 Eye diagram for quaternary transmission and raised cosine pulse qR with roll-off factor:
(a) 𝜌 = 0.125 and (b) 𝜌 = 1.
302 Chapter 7. Single-carrier modulation

a
b
t0

Figure 7.21 Height a and width b of the pupil of an eye diagram.

on [t1 , t1 + T). Typically, we select t1 so that the centre of the eye falls in the centre of the interval
[t1 , t1 + T). Then the contours of the obtained eye diagram correspond to the different profiles (7.90). If
the contours of the eye do not appear, it means that for all values of t0 the worst-case ISI is larger than
the desired component and the eye is shut. We note that, if the pulse qR (t), t ∈ ℝ, has a duration equal to
th , and define Nh = ⌈th ∕T⌉, we must omit plotting the values of y for the first and last Nh − 1 modulation
intervals, as they would be affected by the transient behaviour of the system. Moreover, for transmission
with i.i.d. symbols, at every instant t ∈ ℝ the number of symbols {ak } that contribute to y(t) is at most
equal to Nh . To plot the eye diagram, we need in principle to generate all the M-ary symbol sequences of
length Nh : in this manner, we will reproduce the values of y(t) in correspondence of the various profiles.

7.3 Performance analysis

Signal-to-noise ratio
The performance of a system is expressed by Pe or Pbit (see definitions (6.76) and (6.85), respectively) as
a function of the signal-to-noise ratio Γ (6.69). We express now EsCh = T𝙼sCh in PAM and QAM systems.

PAM systems. For an i.i.d. input symbol sequence, using (1.324) we have
EsCh = 𝙼a EqCh (7.95)
and, for baseband signals Bmin = 1∕(2T), we obtain
𝙼a EqCh
Γ= (7.96)
N0 ∕2
where 𝙼a is the statistical power of the data and EqCh is the energy of the pulse qCh (t) = hTx ∗ gCh (t).
Because for PAM, observing (7.35), we get qCh (t) = qC (t), then (7.96) can be expressed as
𝙼a EqC
Γ= (7.97)
N0 ∕2

QAM systems. From (1.219), using (7.35), we obtain

[ ] 1 [ ] [ ]
E s2Ch (t) = E |s(bb)
Ch
(t)| 2
= E |sC (t)|2 (7.98)
2
and
EsC
Γ= (7.99)
N0 (Bmin T)
7.3. Performance analysis 303

As for passband signals Bmin = 1∕T, (7.99) becomes

𝙼a EqC
Γ= (7.100)
N0
We note that (7.100), expressed as7
𝙼a EqC ∕2
Γ= (7.101)
N0 ∕2
represents the ratio between the energy per component of sC , given by T E [(Re[sC (t)])2 ] and
T E [(Im[sC (t)])2 ], and the PSD of the noise components, equal to N0 ∕2. Then (7.100), observing also
(7.35), coincides with (7.96) of PAM systems.

Symbol error probability in the absence of ISI

If the Nyquist conditions (7.72) are verified, from (7.58), the samples of the received signal at the deci-
sion point are given by (see also (6.13))

yk = ak + 𝑤R,k , ak ∈  (7.102)

We consider a memoryless decision rule on yk , i.e. regarding yk as an isolated sample. If 𝑤R,k has a
circularly symmetric Gaussian probability density function, as is the case if equation (7.40) holds, and
the values assumed by ak are equally likely, then given the observation yk , the minimum-distance detector
leads to choosing the value of 𝛼n ∈  that is closest to yk .8 Moreover, the error probability depends on
the distance dmin between adjacent values 𝛼n ∈ , and on the variance per dimension of the noise 𝑤R,k ,
𝜎I2 . Hence, from the definition (6.75) of the signal-to-noise ratio (SNR) at detection point 𝛾, the bit error
probability for M-PAM and M-QAM are given in (6.77) and (6.78), respectively. We note that, for the
purpose of computing Pe , only the variance of the noise 𝑤R,k is needed and not its PSD.
With reference to the constellations of Table 6.1 and Figure 6.2, we consider dmin = 2h0 = 2. Now,
for a channel with white noise, 𝜎I2 is given by (7.71), hence, from (6.75) it follows that
2
𝛾= (7.103)
N0 EgRc
Apparently, the above equation could lead to choosing a filter gRc with very low energy so that 𝛾 ≫ 1.
However, here gRc is not arbitrary, but it must be chosen such that the condition (7.72) for the absence
of ISI is satisfied. We will see later a criterion to design the filter gRc .

Matched filter receiver

Assuming absence of ISI, (6.77) and (6.78) imply that the best performance, that is the minimum value
of Pe , is obtained when the ratio 𝛾 is maximum.
Assuming that the pulse qC that determines the signal at the channel output is given, the solution (see
Section 1.8) is provided by the receive filter gRc matched to qC : hence, the name matched filter (MF). In
particular, with reference to the scheme of Figure 7.11 and for white noise 𝑤C , we have

Rc (f ) = KQ∗C (f ) e−j2𝜋f t0 (7.104)

7
The term 𝙼a EqC ∕2 represents the energy of both Re [sC (t)] and Im [sC (t)], assuming that sC is circularly symmetric (see
(1.332)).
8
We observe that this memoryless decision criterion is optimum only if the noise samples {𝑤R,k } are statistically
independent.
304 Chapter 7. Single-carrier modulation

where K is a constant. In this case, from the condition

h0 =  −1 [Rc (f )QC (f )]|t=t0 = K 𝚛qC (0) = 1 (7.105)

we obtain
1
K= (7.106)
EqC
Substitution of (7.106) in (7.104) yields EgRc = 1∕EqC . Therefore, (7.103) assumes the form
2EqC
𝛾 = 𝛾MF = (7.107)
N0
The matched filter receiver is of interest also for another reason. Using (7.97) and (7.100), it is possible to
determine the relations (6.79) and (6.80) between the SNRs 𝛾 at the decision point and Γ at the receiver
input.
We stress that, for a given modulation with pulse qC and Γ, it is not possible by varying the filter gRc
to obtain a higher 𝛾 at the decision point than (6.80), and consequently a lower Pe . The equation (6.80)
is often used as an upper bound of the system performance. However, we note that we have ignored the
possible presence of ISI at the decision point that the choice of (7.104) might imply.

7.4 Channel equalization

We analyse methods which approximate the ISI-free condition (7.72) at detection point.

7.4.1 Zero-forcing equalizer

From (7.59), assuming that Tx (f ) and C (f ) are known, and imposing (f ) as in (7.77), then the
equation
(f ) = R (f )ej2𝜋f t0 = Tx (f )C (f )Rc (f )ej2𝜋f t0 (7.108)

can be solved with respect to the receive filter, yielding

( )
f
T r cos 1∕T ,𝜌
Rc (f ) = e−j2𝜋f t0 (7.109)
Tx (f )C (f )
From (7.109), the magnitude and phase responses of Rc can be obtained. In practice, however, although
condition (7.109) leads to the suppression of the ISI, hence, the filter gRc is called linear equalizer
zero-forcing (LE-ZF)), it may also lead to the enhancement of the noise power at the decision point, as
expressed by (7.68).
In fact, if the frequency response C (f ) exhibits strong attenuation at certain frequencies in the range
[−(1 + 𝜌)∕(2T), (1 + 𝜌)∕(2T)], then Rc (f ) presents peaks that determine a large value of 𝜎𝑤2 R . In any
event, the choice (7.109) guarantees the absence of ISI at the decision point, and from (7.103), we get
2
𝛾LE−ZF = (7.110)
N0 EgRc
Obviously, based on the considerations of Section 7.1.3, it is

𝛾LE−ZF ≤ 𝛾MF (7.111)

where 𝛾MF is defined in (7.107).

7.4. Channel equalization 305

In the particular case of an ideal NB channel, with Ch (f ) = 0 in the passband of the system, and
assuming hTx √ ( )
f
Tx (f ) = T rcos ,𝜌 (7.112)
1∕T
from (7.39)
C (f ) = Tx (f ) C (f ) = k1 Tx (f ) (7.113)
√
where from (7.35), k1 = 0 for a PAM system, whereas k1 = (0 ∕ 2)e 1 0 for a QAM system.
−j(𝜑 −𝜑 )

Moreover, from (7.109), neglecting a constant delay, i.e. for t0 = 0, it results that
√ ( )
1 f
Rc (f ) = rcos ,𝜌 (7.114)
k1 1∕T
In other words, gRc is matched to qC (t) = k1 hTx (t), and
𝛾LE−ZF = 𝛾MF (7.115)
Methods for the design of a LE-ZF filter with a finite number of coefficients are given in Section 7.4.6.

7.4.2 Linear equalizer

We introduce an optimization criterion for Rc that takes into account the ISI as well as the statistical
power of the noise at the decision point.

Optimum receiver in the presence of noise and ISI

With reference to the scheme of Figure 7.11 for a QAM system, the criterion of choosing the receive
filter such that the signal yk is as close as possible to ak in the mean-square sense is widely used.9
Let hTx and gC be known. Defining the error
ek = a k − yk (7.116)
the receive filter gRc is chosen such that the mean-square error
J = E[|ek |2 ] = E[|ak − yk |2 ] (7.117)
is minimized.
The following assumptions are made:
1. the sequence {ak } is wide sense stationary (WSS) with spectral density a (f );
2. the noise 𝑤C is complex-valued and WSS. In particular, we assume it is white with PSD
𝑤C (f ) = N0 ;
3. the sequence {ak } and the noise 𝑤C are statistically independent.
The minimization of J in this situation differs from the classical problem of the optimum Wiener filter
because hTx and gC are continuous-time pulses. By resorting to the calculus of variations [4], we obtain
the general solution
∗ (f ) a (f )
Rc (f ) = C e−j2𝜋f t0 ( )2 (7.118)
∑ 1 || 𝓁 ||
N0 1 +∞
T + a (f ) T 𝓁=−∞ N |C f − T |
0| |
where C (f ) = Tx (f )C (f ).
9
It would be desirable to find the filter such that P[̂ak ≠ ak ] is minimum. This problem, however, is usually very difficult
to solve. Therefore, we resort to the criterion of minimizing E[|yk − ak |2 ] instead.
306 Chapter 7. Single-carrier modulation

If the symbols are statistically independent and have zero mean, then a (f ) = T𝜎a2 , and (7.118)
becomes:
𝜎a2
Rc (f ) = ∗C (f )e−j2𝜋f t0 (7.119)
∑+∞ | ( )|2
N0 + 𝜎a2 T1 𝓁=−∞ ||C f − T𝓁 ||
| |
The expression of the cost function J in correspondence of the optimum filter (7.119) is given in (7.147).
From the decomposition (7.55) of Rc (f ), in (7.119), we have the following correspondences:
M (f ) = ∗C (f )e−j2𝜋f t0 (7.120)
and
𝜎a2
C(ej2𝜋fT ) = (7.121)
| ( )|2
∑+∞
| 𝜎a2 T1 𝓁 |
N0 + 𝓁=−∞ |C f − T |
| |
The optimum receiver thus assumes the structure of Figure 7.22. We note that gM is the matched filter
to the impulse response of the QAM system at the receiver input.10

(t) (t0 −t)

(t) (t) 0

Figure 7.22 Optimum receiver structure for a channel with additive white noise.

The filter c is called linear equalizer (LE). It attempts to find the optimum trade-off between removing
the ISI and enhancing the noise at the decision point.
We analyse two particular cases of the solution (7.119).
1. In the absence of noise, 𝑤C (t) ≃ 0, and
1
C(ej2𝜋fT ) = ( )|2 (7.122)
|∑+∞
1 | f − 𝓁 |
𝓁=−∞ | C T |
T
| |
Note that the system is perfectly equalized, i.e. there is no ISI. In this case, the filter (7.122) is the
LE-ZF, as it completely eliminates the ISI.
2. In the absence of ISI at the ouput of gM , that is if |C (f )|2 is a Nyquist pulse, then C(ej2𝜋fT ) is
constant and the equalizer can be removed.

Alternative derivation of the IIR equalizer

Starting from the receiver of Figure 7.22 and for any type of filter gM , not necessarily matched, it is
possible to determine the coefficients of the finite-impulse-response (FIR) equalizer filter c using the
Wiener formulation, with the following definitions:
● filter input signal, xk ;
● filter output signal, yk ;

10
As derived later in the text (see Observation 7.14 on page 357), the output signal of the matched filter, sampled at the
modulation rate 1∕T, forms a sufficient statistic if all the channel parameters are known.
7.4. Channel equalization 307

● desired output signal, dk = ak−D ;

● estimation error, ek = dk − yk .
Parameter D that denotes the lag of the desired signal, i.e. the delay introduced by the equalizer in
numbers of symbol intervals. The overall delay from the emission of ak to the generation of the detected
symbol â k is equal to t0 + DT.
However, the particular case of a matched filter, for which gM (t) = q∗C (−(t − t0 )), is very interesting
from a theoretical point of view. We assume that the filter c may have an infinite number of coefficients,
i.e. it may be with infinite impulse response (IIR). With reference to the scheme of Figure 7.23a, q is the
overall impulse response of the system at the sampler input:
q(t) = hTx ∗ gC ∗ gM (t) = qC ∗ gM (t) = 𝚛qC (t − t0 ) (7.123)
where the autocorrelation of the deterministic pulse qC is
𝚛qC (t) = [qC (t′ ) ∗ q∗C (−t′ )](t) (7.124)

dk = ak–D
t0 + kT
+
ak x(t) xk yk − ek
q + c +
T T T T

wR (t)

(a)

dk = ak–D
{hi = q(t0 + iT)}
+
ak xk yk − ek
h + c +
T T T T

~
wk

(b)

Figure 7.23 Linear equalizer as a Wiener filter. (a) Continuous-time model. (b) Discrete-time
equivalent.

The Fourier transform of 𝚛qC (t) is given by

qC (f ) = |C (f )|2 (7.125)
We note that if qC has a finite support (0, tqC ), then gM (t) = − t) has support (t0 − tqC , t0 ). Hence, to
q∗C (t0
obtain a causal filter gM the minimum value of t0 is tqC . In any case, from (7.123), as 𝚛qC is a correlation
function, the desired sample q(t0 ) is taken in correspondence of the maximum value of |q(t)|.
Assuming 𝑤C is white, we have (see Figure 7.23a)
𝑤R (t) = 𝑤C ∗ gM (t) (7.126)
with autocorrelation function given by
𝚛𝑤R (𝜏) = 𝚛𝑤C ∗ 𝚛qC (𝜏)
(7.127)
= N0 𝚛qC (𝜏)
308 Chapter 7. Single-carrier modulation

and PSD
𝑤R (f ) = N0 qC (f ) = N0 |C (f )|2 (7.128)
In Figure 7.23a, sampling at instants tk = t0 + kT yields the sampled QAM signal
∑
+∞
xk = ai hk−i + 𝑤̃ k (7.129)
i=−∞

The discrete-time equivalent model is illustrated in Figure 7.23b. The discrete-time overall impulse
response is given by
hn = q(t0 + nT) = 𝚛qC (nT) (7.130)
In particular, it results
h0 = 𝚛qC (0) = EqC (7.131)
The sequence {hn } has z-transform
Φ(z) = [hn ] = PqC (z) (7.132)
which, by the Hermitian symmetry of an autocorrelation sequence, 𝚛qC (nT) = 𝚛∗qC (−nT), satisfies the
relation: ( )
1
Φ(z) = Φ∗ ∗ (7.133)
z
On the other hand, from (1.23), the Fourier transform of (7.130) is given by
( )2
1 ∑ || 𝓁 ||
+∞
Φ(ej2𝜋fT ) =  [hn ] = |C f − | (7.134)
T 𝓁=−∞ || T ||
Moreover, the correlation sequence of {hn } has z-transform equal to
( )
1
[𝚛h (m)] = Φ(z)Φ∗ ∗ (7.135)
z
Also, from (7.127), the z-transform of the autocorrelation of the noise samples 𝑤̃ k = 𝑤R (t0 + kT) is
given by
[𝚛𝑤̃ (n)] = [𝚛𝑤R (nT)] = N0 Φ(z) (7.136)
The Wiener solution that gives the optimum coefficients is given in the z-transform domain by (see
(2.259))
P (z)
Copt (z) = [cn ] = dx (7.137)
Px (z)
where
Pdx (z) = [𝚛dx (n)] and Px (z) = [𝚛x (n)] (7.138)
We assume
1. The sequence {ak } is WSS, with symbols that are statistically independent and with zero mean,
𝚛a (n) = 𝜎a2 𝛿n and a (f ) = T𝜎a2 (7.139)
2. {ak } and {𝑤̃ k } are statistically independent and hence uncorrelated.
7.4. Channel equalization 309

Using assumption 2, the crosscorrelation between {dk } and {xk } is given by

𝚛dx (n) = E[dk xk−n
∗
]
[ ( )∗ ]
∑
+∞
= E ak−D ai hk−n−i + 𝑤̃ k−n
i=−∞

∑
+∞
= h∗k−n−i E[ak−D a∗i ] (7.140)
i=−∞

Lastly, from assumption 1,

𝚛dx (n) = 𝜎a2 h∗D−n (7.141)
Under the same assumptions 1 and 2, the computation of the autocorrelation of the process {xk } yields
𝚛x (n) = E[xk xk−n
∗
] = 𝜎a2 𝚛h (n) + 𝚛𝑤̃ (n) (7.142)
Thus, using (7.135), we obtain
( )
1
Pdx (z) = 𝜎a2 Φ∗ z−D
z∗ ( ) (7.143)
1
Px (z) = 𝜎a2 Φ(z)Φ∗ z∗
+ N0 Φ(z)

Therefore, from (7.137), ( )

𝜎a2 Φ∗ z1∗ z−D
Copt (z) = ( ) (7.144)
𝜎a2 Φ(z)Φ∗ z1∗ + N0 Φ(z)

Taking into account the property (7.133), (7.144) is simplified as

𝜎a2 z−D
Copt (z) = (7.145)
N0 + 𝜎a2 Φ(z)
It can be observed that, for z = ej2𝜋fT , (7.145) corresponds to (7.121), apart from the term z−D , which
accounts for a possible delay introduced by the equalizer.
In correspondence of the optimum filter Copt (z), we determine the minimum value of the cost function.
We recall the general expression for the Wiener filter (2.48)
∑
N−1
Jmin = 𝜎d2 − copt,i 𝚛∗dx (i)
i=0
1
(7.146)
2T
= 𝜎d2 − dx
∗
(f ) Copt (e j2𝜋fT
) df
∫− 1
2T

Finally, substitution of the relations (7.143) in (7.146) yields

1
2T
Jmin = 𝜎d2 − T P∗dx (ej2𝜋fT )Copt (ej2𝜋fT )df
∫− 1
2T
1
(7.147)
2T N0
= 𝜎a2 T df
∫− 1 N0 + 𝜎a2 Φ(ej2𝜋fT )
2T

If Φ(z) is a rational function of z, the integral (7.147) may be computed by evaluating the coefficient
of the term z0 of the function 𝜎a2 N0 ∕(N0 + 𝜎a2 Φ(z)), which can be obtained by series expansion of the
integrand, or by using the partial fraction expansion method (see (1.65)).
310 Chapter 7. Single-carrier modulation

We note that in the absence of ISI, at the output of the MF, we get Φ(z) = h0 = EqC , and
𝜎a2 N0
Jmin = (7.148)
N0 + 𝜎a2 EqC

Signal-to-noise ratio at detector

We define the overall impulse response at the equalizer output, sampled with a sampling rate equal to
the modulation rate 1∕T, as
𝜓i = (hn ∗ copt,n )i (7.149)
where {hn } is given by (7.130) and copt,n is the impulse response of the optimum filter (7.145).
At the decision point, we have
∑
+∞
yk = 𝜓D ak−D + 𝜓i ak−i + (𝑤̃ n ∗ copt,n )k (7.150)
i=−∞
i≠D

We assume that in (7.150), the total disturbance 𝑤eq,k , given by ISI plus noise, is modelled as Gaussian
noise with variance 2𝜎I2 . As ek = ak−D − yk = ak−D − 𝜓D ak−D − 𝑤eq,k , with 𝑤eq,k uncorrelated to ak−D ,
it follows
2𝜎I2 = Jmin − 𝜎a2 |1 − 𝜓D |2 (7.151)
where Jmin is given by (7.147) or in general (7.196). Hence, for a minimum distance among symbol
values of the constellation equal to 2, (6.75) yields the SNR at the detector
( )
|𝜓D | 2 2|𝜓D |2
𝛾LE = = (7.152)
𝜎I Jmin − 𝜎a2 |1 − 𝜓D |2
In case the approximation 𝜓D ≃ 1 holds, the total disturbance in (7.150) coincides with ek , hence, 2𝜎I2 ≃
Jmin , and (7.152) becomes
2
𝛾LE ≃ (7.153)
Jmin

7.4.3 LE with a finite number of coefficients

In practice, if the channel is either unknown a priori or it is time variant, it is necessary to design a
receiver that tries to identify the channel characteristics and at the same time equalize it through suitable
adaptive algorithms.
Two alternative approaches are usually considered.

First solution. The classical block diagram of an adaptive receiver is shown in Figure 7.24.

rC (t) x(t) x (t0 + kT) yk â k–D

gM c
t0 + kT T T
matched data
filter detector

Figure 7.24 Receiver implementation by an analog matched filter followed by a sampler and a
discrete-time linear equalizer.

The matched filter gM is designed assuming an ideal channel. Therefore, the equalization task is left
to the filter c; otherwise, if it is possible to rely on some a priori knowledge of the channel, the filter gM
7.4. Channel equalization 311

may be designed according to the average characteristics of the channel. The filter c is then an adaptive
transversal filter that attempts, in real time, to equalize the channel by adapting its coefficients to the
channel variations.

Second solution. The receiver is represented in Figure 7.25.

antialiasing data
filter decimator filter detector
rC (t) x (t) x (t0 + nTc) yk a^k–D
gAA c F0
Tc T
t0 + nTc Tc = T T
F0

Figure 7.25 Receiver implementation by discrete-time filters.

The anti-aliasing filter gAA is designed according to specifications imposed by the sampling theorem.
In particular if the desired signal sC has a bandwidth B and x is sampled with period Tc = T∕F0 , where F0
is the oversampling index, with F0 ≥ 2, then the passband of gAA should extend at least up to frequency
B. Moreover, because the noise 𝑤C is considered as a wideband signal, gAA should also attenuate the
noise components outside the passband of the desired signal sC , hence the cut-off frequency of gAA is
between B and F0 ∕(2T). In practice, to simplify the implementation of the filter gAA , it is convenient to
consider a wide transition band.
Thus, the discrete-time filter c needs to accomplish the following tasks:

1. filter the residual noise outside the passband of the desired signal sC ;
2. act as a matched filter;
3. equalize the channel.

Note that the filter c of Figure 7.25 is implemented as a decimator filter (see Appendix 1.A), where
the input signal xn = x(t0 + nTc ) is defined over a discrete-time domain with period Tc = T∕F0 , and the
output signal yk is defined over a discrete-time domain with period T.
In turn two strategies may be used to determine an equalizer filter c with N coefficients:

1. direct method, which employs the Wiener formulation and requires the computation of the matrix
R and the vector p. The description of the direct method is postponed to Section 7.4.4 (see Obser-
vation 7.3 on page 319);
2. adaptive method, which we will describe next (see Chapter 3).

Adaptive LE
We analyse now the solution illustrated in Figure 7.24: the discrete-time equivalent scheme is illustrated
in Figure 7.26, where {hn } is the discrete-time impulse response of the overall system, given by

hn = q(t0 + nT) = hTx ∗ gC ∗ gM (t)|t=t0 +nT (7.154)

and
𝑤̃ k = 𝑤R (t0 + kT) (7.155)

with 𝑤R (t) = 𝑤C ∗ gM (t).

312 Chapter 7. Single-carrier modulation

~
w k
{hi = q(t0 + iT)}
+ xk yk
ak a^ k–D
h + c
T T +

−+ +
ak–D
ek

Figure 7.26 Discrete-time equivalent channel model and linear equalizer.

The design strategy consists of the following steps.

1. Define the performance measure of the system. The mean square error (MSE) criterion is typically
adopted:
J(k) = E [|ek |2 ] (7.156)
2. Select the law of coefficient update. For example, for an FIR filter c with N coefficients using the
least mean square (LMS) algorithm (see Section 3.1.2), we have
ck+1 = ck + 𝜇ek x∗k (7.157)
where
(a) input vector
xTk = [xk , xk−1 , … , xk−N+1 ] (7.158)
(b) coefficient vector
cTk = [c0,k , c1,k , … , cN−1,k ] (7.159)
(c) adaptation gain
2
0<𝜇< (7.160)
N𝚛x (0)
3. To evaluate the signal error ek to be used in the adaptive algorithm, we distinguish two modes.
(a) Training mode:
ek = ak−D − yk , k = D, … , LTS + D − 1 (7.161)
Evaluation of the error in training mode is possible if a sufficiently long sequence of LTS
symbols known at the receiver, called training sequence (TS), {ak }, k = 0, 1, … , LTS −
1,
is transmitted. The duration of the transmission of TS is equal to LTS T. During this
time interval, the automatic identification of the channel characteristics takes place,
allowing the computation of the optimum coefficients of the equalizer filter c, and
consequently channel equalization. As the PSD of the training sequence must be wide,
typically a pseudo-noise (PN) sequence is used (see Appendix 1.C). We note that even
the direct method requires a training sequence to determine the vector p and the matrix
R (see Observation 7.4 on page 319).
(b) Decision directed mode:
ek = â k−D − yk , k ≥ LTS + D (7.162)
7.4. Channel equalization 313

Once the transmission of the TS is completed, we assume that the equalizer has reached
convergence. Therefore, â k ≃ ak , and the transmission of information symbols may start.
In (7.161), we then substitute the known transmitted symbol with the detected symbol
to obtain (7.162).
Unfortunately, in the presence of a very dispersive channel, the signal xk is far from being white and
the LMS algorithm will have a very slow convergence. Hence, it is seldom used.

Fractionally spaced equalizer

We consider the receiver structure with oversampling illustrated in Figure 7.25, denoted fractionally
spaced equalizer (FSE). The discrete-time overall system, shown in Figure 7.27, has impulse response
given by
hi = q(t0 + iTc ) (7.163)
where
q(t) = hTx ∗ gC ∗ gAA (t) (7.164)
The noise is given by
𝑤̃ n = 𝑤R (t0 + nTc ) = 𝑤C ∗ gAA (t)|t=t0 +nTc (7.165)
Only for analysis purposes, filter c is decomposed into a discrete-time filter with sampling period Tc that
is cascaded with a downsampler.

{hi = q(t0 + iTc)}

ak xn yʹn yk a^k–D
h c F0
T Tc Tc T T

~
w n

Figure 7.27 Fractionally spaced linear equalizer.

The input signal to the filter c is the sequence {xn } with sampling period Tc = T∕F0 ; the n-th sample
of the sequence is given by
∑
+∞
xn = hn−kF0 ak + 𝑤̃ n (7.166)
k=−∞
The output of the filter c is given by
∑
N−1
y′n = ci xn−i (7.167)
i=0

The overall impulse response at the filter output, defined on the discrete-time domain with sampling
period Tc , is given by
𝜓i = h ∗ ci (7.168)
If we denote by {y′n }, the sequence of samples at the filter output, and by {yk } the downsampled sequence,
we have
yk = y′kF (7.169)
0

As mentioned earlier, in a practical implementation of the filter the sequence {y′n } is not explicitly gen-
erated; only the sequence {yk } is produced (see Appendix 1.A). However, introducing the downsampler
helps to illustrate the advantages of operating with an oversampling index F0 > 1.
314 Chapter 7. Single-carrier modulation

Before analysing this system, we recall the Nyquist problem. Let us consider a QAM system with
pulse h:
∑
+∞
sR (t) = an h(t − nT), t∈ℝ (7.170)
n=−∞
In Section 7.1.3, we considered continuous-time Nyquist pulses h(t), t ∈ ℝ. Let h be defined now on
a discrete-time domain {nTc }, n integer, where Tc = T∕F0 . If F0 is an integer, the discrete-time pulse
satisfies the Nyquist criterion if h(0) ≠ 0, and h(𝓁F0 Tc ) = 0, for all integers 𝓁 ≠ 0. In the particular
case F0 = 1, we have Tc = T, and the Nyquist conditions impose that h(0) ≠ 0, and h(nT) = 0, for
n ≠ 0. Recalling the input–output downsampler relations in the frequency domain, it is easy to deduce
the behaviour of a discrete-time Nyquist pulse in the frequency domain: two examples are given in
Figure 7.28, for F0 = 2 and F0 = 1. We note that, for F0 = 1, in the frequency domain a discrete-time
Nyquist pulse is equal to a constant.

h(t) H(e j2πf Tc)

0 T 2T t = nTc 1 0 1 1 f
(a) 2T 2T T

h(t) H(e j2πfTc)

0 T 2T t = nT 1 0 1 1 f
(b) 2T 2T T

Figure 7.28 Discrete-time Nyquist pulses and relative Fourier transforms. (a) F0 = 2. (b) F0 = 1.

With reference to the scheme of Figure 7.27, the QAM pulse defined on the discrete-time domain with
period Tc is given by (7.163), where q is defined in (7.164). From (7.168), using (7.163), the pulse {𝜓i }
at the equalizer output before the downsampler has the following Fourier transform:
Ψ(ej2𝜋f Tc ) = C(ej2𝜋f Tc )H(ej2𝜋f Tc )
( ) ( )
1 ∑
+∞
F F
j2𝜋 f −𝓁 T0 t0
= C(ej2𝜋f Tc )  f −𝓁 0 e (7.171)
T 𝓁=−∞ T
The task of the equalizer is to yield a pulse {𝜓i } that approximates a Nyquist pulse, i.e. a pulse of the type
shown in Figure 7.28. We see that choosing F0 = 1, i.e. sampling the equalizer input signal with period
equal to T, it may happen that H(ej2𝜋f Tc ) assumes very small values at frequencies near f = 1∕(2T),
because of an incorrect choice of the timing phase ( t) 0 . In fact, let us assume q real with a bandwidth
1
smaller than 1∕T. Using the polar notation for  2T , we have
( ) ( )
1 1
 = Aej𝜑 and  − = Ae−j𝜑 (7.172)
2T 2T
as (f ) is Hermitian. Therefore, from (7.171),
( ) ( ) ( )
| 1 t0
1 1 j2𝜋 2T1 − T1 t0
H(ej2𝜋fT )| 1 =  ej2𝜋 2T +  − e
|f = 2T 2T 2T T
( )
t0
= 2A cos 𝜑 + 𝜋 (7.173)
T
7.4. Channel equalization 315

If t0 is such that
t0 2i + 1
𝜑+𝜋 = 𝜋, i integer (7.174)
T 2
then ( 1
)
H ej2𝜋 2T T = 0 (7.175)

In this situation, the equalizer will enhance the noise around f = 1∕(2T), or converge with difficulty in
the adaptive case. If F0 ≥ 2 is chosen, this problem is avoided because aliasing between replicas of (f )
does not occur. Therefore, the choice of t0 may be less accurate. In fact, as we will see in Chapter 14,
if c has an input signal sampled with sampling period T∕2 it also acts as an interpolator filter, whose
output can be used to determine the optimum timing phase.
In conclusion, the FSE receiver presents two advantages over T-spaced equalizers:
1. it is an optimum structure according to the MSE criterion, in the sense that it carries out the task
of both matched filter (better rejection of the noise) and equalizer (reduction of ISI);
2. it is less sensitive to the choice of t0 . In fact, the correlation method (7.602) with accuracy TQ′ = T∕2
is usually sufficient to determine the timing phase.
The direct method to compute the coefficients of FSE is described in Section 7.4.4. Here we just
mention that the design of FSE may incur a difficulty in the presence of noise with variance that is small
with respect to the level of the desired signal: in this case some eigenvalues of the autocorrelation matrix
of x∗2k may assume a value that is almost zero and consequently, the problem of finding the optimum
coefficients become ill-conditioned, with numerous solutions that present the same minimum value of
the cost function. This effect can be illustrated also in the frequency domain: outside the passband of the
input signal the filter c may assume arbitrary values, in the limit case of absence of noise. In general,
simulations show that FSE outperform equalizers operating at the symbol rate [5].

7.4.4 Decision feedback equalizer

We consider the sampled signal at the output of the analog receive filter (see Figure 7.26 or Figure 7.27).
For example, in the scheme of Figure 7.26, the desired signal is
∑
+∞
sk = sR (t0 + kT) = ai hk−i (7.176)
i=−∞

where the sampled pulse {hn } is defined in (7.154). In the presence of noise, we have
xk = sk + 𝑤̃ k (7.177)
where 𝑤̃ k is the noise, given by (7.155).
We assume, as illustrated in Figure 7.29, that {hn } has finite duration and support [−N1 , −N1 +
1, … , N2 − 1, N2 ]. The samples with positive time indices are called postcursors, and those with neg-
ative time indices precursors. Explicitly writing terms that include precursors and postcursors, (7.177)
becomes
xk = h0 ak + (h−N1 ak+N1 + · · · + h−1 ak+1 ) + (h1 ak−1 + · · · + hN2 ak−N2 ) + 𝑤̃ k (7.178)
In addition to the actual symbol ak that we desire to detect from the observation of xk , in (7.178) two
terms are identified in parentheses: one that depends only on past symbols ak−1 , … , ak−N2 , and another
that depends only on future symbols, ak+1 , … , ak+N1 .
If the past symbols and the impulse response {hn } were perfectly known, we could use an ISI can-
cellation scheme limited only to postcursors. Substituting the past symbols with their detected versions
316 Chapter 7. Single-carrier modulation

hn = q(t0 + nT)

N1 = 2 N2 = 3

−2 −1 0 1 2 3 n

precursors postcursors

D=3
ψn = h*cn M1 = 4

−2 −1 0 1 2 3 4 5 6 n

precursors postcursors

Figure 7.29 Discrete-time pulses in a DFE. (a) Before the FF filter. (b) After the FF filter.

Figure 7.30 Simplified scheme of a DFE, where only the feedback filter is included.

{̂ak−1 , … , â k−N2 }, we obtain a scheme to cancel in part ISI, as illustrated in Figure 7.30, where, in general,
the feedback filter has impulse response {bn }, n = 1, · · · , M2 , and output given by
xFB,k = b1 â k−1 + … + bM2 â k−M2 (7.179)
If M2 ≥ N2 , bn = −hn , for n = 1, … , N2 , bn = 0, for n = N2 + 1, … , M2 , and â k−i = ak−i , for
i = 1, … , N2 , then the decision feedback equalizer (DFE) cancels the ISI due to postcursors. We note
that this is done without changing the noise 𝑤̃ k that is present in xk .
The general structure of a DFE is shown in Figure 7.31, where two filters and the detection delay are
outlined:
7.4. Channel equalization 317

Figure 7.31 General structure of the DFE.

1. Feedforward (FF) filter c, with M1 coefficients,

∑
M1 −1
zk = xFF,k = ci xk−i (7.180)
i=0

2. Feedback (FB) filter b, with M2 coefficients,

∑
M2
xFB,k = bi ak−i−D (7.181)
i=1

Moreover,
yk = xFF,k + xFB,k (7.182)
which from (7.180) and (7.181) can also be written as
∑
M1 −1
∑
M2
yk = ci xk−i + bj ak−D−j (7.183)
i=0 j=1

We recall that for a LE, the goal is to obtain a pulse {𝜓n } free of ISI, with respect to the desired
sample 𝜓D . Now (see Figure 7.29) ideally the task of the feedforward filter is to obtain an overall impulse
response {𝜓n = h ∗ cn } with very small precursors and a transfer function [𝜓n ] that is minimum phase
(see Example 1.2.4). In this manner, almost all the ISI is cancelled by the FB filter. We note that the FF
filter may be implemented as an FSE, whereas the feedback filter operates with sampling period equal
to T.
The choice of the various parameters depends on {hn }. The following guidelines, however, are usually
observed.
1. M1 T∕F0 (time-span of the FF filter) at least equal to (N1 + N2 + 1)T∕F0 (time-span of h), so the
FF filter can effectively equalize;
2. M2 T (time-span of the FB filter) equal to or less than (M1 − 1)T∕F0 (time-span of the FF filter
minus one); M2 depends also on the delay D, which determines the number of postcursors.
M T
3. For very dispersive channels, for which we have N1 + N2 ≫ M1 , it results M2 T ≫ F1 .
0
4. The choice of DT, equal to the detection delay, is obtained by initially choosing a large delay, D ≥
(M1 − 1)∕F0 , to simplify the FB filter. If the precursors are not negligible, to reduce the constraints
on the coefficients of the FF filter the value of D is lowered and the system is iteratively designed.
In practice, DT is equal to or smaller than (M1 − 1)T∕F0 .
For a LE, instead, DT is approximately equal to N−1 T
2 F0
; the criterion is that the centre of gravity
of the coefficients of the filter c is approximately equal to (N − 1)∕2.
318 Chapter 7. Single-carrier modulation

The detection delays discussed above are referred to a pulse {hn } centered in the origin, that does not
introduce any delay.

Design of a DFE with a finite number of coefficients

If the channel impulse response {hi } and the autocorrelation function of the noise 𝚛𝑤̃ (n) are known, for
the MSE criterion with
J = E[|ak−D − yk |2 ] (7.184)
the Wiener filter theory may be applied to determine the optimum coefficients of the DFE filter in the
case â k = ak , with the usual assumptions of symbols that are i.d.d. and statistically independent of the
noise.
For a generic sequence {hi } in (7.176), we recall the following results:
1. cross correlation between ak and xk
𝚛ax (n) = 𝜎a2 h∗−n (7.185)
2. auto correlation of xk
𝚛x (n) = 𝜎a2 𝚛h (n) + 𝚛𝑤̃ (n) (7.186)
where
∑
N2
𝚛h (n) = hj h∗j−n , 𝚛𝑤̃ (n) = N0 𝚛gM (nT) (7.187)
j=−N1

Defining
∑
M1 −1
𝜓p = h ∗ cp = c𝓁 hp−𝓁 (7.188)
𝓁=0

Equation (7.183) becomes

∑
N2 +M1 −1
∑
M1 −1
∑
M2
yk = 𝜓p ak−p + ci 𝑤̃ k−i + bj ak−D−j (7.189)
p=−N1 i=0 j=1

From (7.189), the optimum choice of the feedback filter coefficients is given by
bi = −𝜓i+D , i = 1, … , M2 (7.190)
Substitution of (7.190) in (7.183) yields
( )
∑
M1 −1
∑
M2
yk = ci xk−i − hj+D−i ak−j−D (7.191)
i=0 j=1

To obtain the Wiener–Hopf solution, the following correlations are needed:

[ ( )∗ ]
∑
M2
[p]p = E ak−D xk−p − hj+D−p ak−D−j = 𝜎a2 h∗D−p ,
j=1

p = 0, 1, … , M1 − 1 (7.192)
[( )
∑
M2
[R]p,q = E xk−q − hj1 +D−q ak−D−j1
j1=1
( )∗ ]
∑
M2
xk−p − hj2 +D−p ak−D−j2
j2=1
7.4. Channel equalization 319

( )
∑
N2
∑
M2
= 𝜎a2 hj h∗j−(p−q) − hj+D−q h∗j+D−p + 𝚛𝑤̃ (p − q),
j=−N1 j=1

p, q = 0, 1, … , M1 − 1 (7.193)
Therefore, the optimum FF filter coefficients are given by
copt = R−1 p (7.194)
and, from (7.190), the optimum feedback filter coefficients are given by
∑
M1 −1
bi = − copt,𝓁 hi+D−𝓁 i = 1, 2, … , M2 (7.195)
𝓁=0

Moreover, using (7.192), we get

∑
M1 −1
Jmin = 𝜎a2 − copt,𝓁 [p]∗𝓁
𝓁=0
( )
∑
M1 −1
= 𝜎a2 1− copt,𝓁 hD−𝓁 (7.196)
𝓁=0

Observation 7.2
In the particular case, wherein all the postcursors are cancelled by the feedback filter, that is for
M2 + D = N2 + M1 − 1 (7.197)
(7.193) is simplified into
∑
D
[R]p,q = 𝜎a2 hj−q h∗j−p + 𝚛𝑤̃ (p − q) (7.198)
j=−N1

For determining optimum values of M1 , M2 , and D, rather than following the generic guidelines of
Section 7.4.4, a better approach is repeating the design method for increasing values of M1 and M2
(and determining the corresponding optimum value of D) until Jmin does not significantly change, i.e. it
reaches approximately its asymptotic value.

Observation 7.3
The equations to determine copt for a LE are identical to (7.192)–(7.196), with M2 = 0. In particular, the
vector p in (7.192) is not modified, while the expression of the elements of the matrix R in (7.193) are
modified by the terms including the detected symbols.

Observation 7.4
For white noise 𝑤C , the autocorrelation of 𝑤̃ k is proportional to the autocorrelation of the receive filter
impulse response: consequently, if the statistical power of 𝑤̃ k is known, the autocorrelation 𝚛𝑤̃ (n) is
easily determined. Lastly, the coefficients of the channel impulse response {hn } and the statistical power
of 𝑤̃ k used in R and p can be determined by the methods given in Section 7.8.

Observation 7.5
For a LE, the matrix R is Hermitian and Toeplitz, while for a DFE, it is only Hermitian; in any case
it is (semi)definite positive. Efficient methods to determine the inverse of the matrix are described in
Section 2.3.2.
320 Chapter 7. Single-carrier modulation

Observation 7.6
The definition of {hn } depends on the value of t0 , which is determined by methods described in
Chapter 14. A particularly useful method to determine the impulse response {hn } in wireless systems
resorts to a short training sequence to achieve fast synchronization. We recall that a fine estimate
of t0,MF is needed if the sampling period of the signal at the MF output is equal to T. The overall
discrete-time system impulse response obtained by sampling the output signal of the anti-aliasing filter
gAA (see Figure 7.25) is assumed to be known, e.g. by estimation. The sampling period, for example
T∕8, is in principle determined by the accuracy with which we desire to estimate the timing phase
t0,MF at the MF output. To reduce implementation complexity, however, a larger sampling period of the
signal at the MF input is considered, for example T∕2. We then implement the MF gM by choosing,
among the four polyphase components (see Section 1.A.9 on page 92) of the impulse response, the
component with the largest energy, thus realizing the MF criterion (see also (7.123)). This is equivalent
to selecting the component with largest statistical power among the four possible components with
sampling period T∕2 of the sampled output signal of the filter gAA . This method is similar to the timing
estimator (14.117).
The timing phase t0,AA for the signal at the input of gM is determined during the estimation of the
channel impulse response. It is usually chosen either as the time at which the first useful sample of the
overall impulse response occurs, or the time at which the peak of the impulse response occurs, shifted
by a number of modulation intervals corresponding to a given number of precursors. Note that, if tMF
denotes the duration of gM , then the timing phase at the output of gM is given by t0,MF = t0,AA + tMF .
The criterion (7.602), according to which t0 is chosen in correspondence of the correlation peak, is a
particular case of this procedure.

Observation 7.7
In systems where the training sequence is placed at the end of a block of data (see the Global System for
Mobile Communications, GSM, frame in Section 18.2), it is convenient to process the observed signal
{xk } starting from the end of the block, let us say from k = K − 1 to 0, thus exploiting the knowledge
of the training sequence. Now, if {𝜓n = h ∗ cn } and {bn } are the optimum impulse responses if the
signal is processed in the forward mode, i.e. for k = 0, 1, … , K − 1, it is easy to verify that {𝜓nB∗ } and
{bB∗
n }, where B is the backward operator defined on page 14, are the optimum impulse responses in the
backward mode for k = K − 1, … , 1, 0, apart from a constant delay. In fact, if {𝜓n } is ideally minimum
phase and causal with respect to the timing phase, now {𝜓nB∗ } is maximum phase and anticausal
with respect to the new instant of optimum sampling. Also the FB filter will result anticausal. In the
particular case {hn } is a correlation sequence, then {𝜓nB∗ } can be obtained using as FF filter the filter
having impulse response {cB∗ n }.

Design of a fractionally spaced DFE

We briefly describe the equations to determine the coefficients of a DFE comprising a fractionally spaced
FF filter with M1 coefficients and sampling period of the input signal equal to T∕2, and an FB filter with
M2 coefficients; the extension to an FSE with an oversampling factor F0 > 2 is straightforward. The
resulting structure is denoted as fractionally spaced DFE (FS-DFE).
We consider the scheme of Figure 7.31, where the FF filter c is now a fractionally spaced filter, as
illustrated in Figure 7.27. The overall receiver structure is shown in Figure 7.32, where the filter gAA
may be more sophisticated than a simple anti-aliasing filter and partly perform the function of the MF.
Otherwise, the function of the MF may be performed by a discrete-time filter placed in front of the filter
c. We recall that the MF, besides reducing the complexity of c (see task 2 on page 311), facilitates the
optimum choice of t0 (see Chapter 14).
7.4. Channel equalization 321

0
rC (t ) x (t)
0

Figure 7.32 FS-DFE structure.

As usual, the symbols are assumed i.i.d. and statistically independent of the noise signal. If {xn } is
the input signal of the FF filter, we have
∑
+∞
xn = hn−2k ak + 𝑤̃ n (7.199)
k=−∞

The signal {yk } at the DFE output is given by

∑
M1 −1
∑
M2
yk = ci x2k−i + bj ak−D−j (7.200)
i=0 j=1

Let
∑
M1 −1
𝜓p = h ∗ cp = c𝓁 hp−𝓁 (7.201)
𝓁=0

then the optimum choice of the coefficients {bi } is given by

∑
M1 −1
bi = −𝜓2(i+D) = − c𝓁 h2(i+D)−𝓁 , i = 1, … , M2 (7.202)
𝓁=0

With this choice (7.200) becomes

( )
∑
M1 −1
∑
M2
yk = ci x2k−i − h2(j+D)−i ak−(j+D) (7.203)
i=0 j=1

Using the following relations (see also Example 1.7.10 on page 51)
∗
E [ak−K x2k−i ] = 𝜎a2 h∗2K−i (7.204)
∑+∞
∗
E [x2k−q x2k−p ] = 𝜎a2 h2n−q h∗2n−p + 𝚛𝑤̃ (p − q) (7.205)
n=−∞

where, from (7.165), ( )

T
𝚛𝑤̃ (m) = N0 𝚛gAA m (7.206)
2
the components of the vector p and the matrix R of the Wiener problem associated with (7.203) are
given by

[p]p = 𝜎a2 h∗2D−p , p = 0, 1, … , M1 − 1 (7.207)

( +∞ )
∑ ∑
M2
[R]p,q = 𝜎a
2 ∗
h2n−q h2n−p − h2(j+D)−q h2(j+D)−p + 𝚛𝑤̃ (p − q),
∗

n=−∞ j=1

p, q = 0, 1, … , M1 − 1 (7.208)
322 Chapter 7. Single-carrier modulation

The feedforward filter is obtained by solving the system of equations

R copt = p (7.209)
and the feedback filter is determined from (7.202). The minimum value of the cost function is given by
∑
M1 −1
Jmin = 𝜎a2 − copt,𝓁 [p]∗𝓁
𝓁=0
( )
∑
M1 −1
= 𝜎a2 1− copt,𝓁 h2D−𝓁 (7.210)
𝓁=0

A problem encountered with this method is the inversion of the matrix R in (7.209), because it may be
ill-conditioned. Similarly to the procedure outlined on page 164, a solution consists in adding a positive
constant to the elements on the diagonal of R, so that R becomes invertible; obviously, the value of
this constant must be rather small, so that the performance of the optimum solution does not change
significantly.

Observation 7.8
Observations 7.2–7.4 on page 319 hold also for a FS-DFE, with appropriate changes. In this case, the
timing phase t0 after the filter gAA can be determined with accuracy T∕2, for example by the correlation
method (7.603). For an FSE, or FS-LE, the equations to determine copt are given by (7.207)–(7.209)
with M2 = 0. Note that the matrix R is Hermitian, but in general, it is no longer Toeplitz.

Signal-to-noise ratio at the decision point

Using FF and FB filters with an infinite number of coefficients, it is possible to achieve the minimum
value of Jmin . Salz derived the expression of Jmin for this case, given by [6]
( 1
)
2T N0
Jmin = 𝜎a exp T
2
ln df (7.211)
∫− 1 N0 + 𝜎a2 Φ(ej2𝜋fT )
2T

where Φ is defined in (7.134).

Applying the Jensen’s inequality,
e∫ f (a)da
≤ ef (a) da (7.212)
∫
to (7.211), we can compare the performance of a linear equalizer given by (7.147) with that of a DFE
given by (7.211): the result is that, assuming Φ(ej2𝜋fT ) not constant and the absence of detection errors
in the DFE, for infinite-order filters the value Jmin of a DFE is always smaller than Jmin of a LE.
If FF and FB filters with a finite number of coefficients are employed, in analogy with (7.152), also
for a DFE, we have
2|𝜓D |2
𝛾DFE = (7.213)
Jmin − 𝜎a2 |1 − 𝜓D |2
where Jmin is given by (7.196). An analogous relation holds for an FS-DFE, with Jmin given by (7.210).

Remarks

1. In the absence of errors of the data detector, the DFE has better asymptotic (M1 , M2 → ∞) perfor-
mance than the linear equalizer. However, for a given finite number of coefficients M1 + M2 , the
performance is a function of the channel and of the choice of M1 .
7.4. Channel equalization 323

2. The DFE is definitely superior to the linear equalizer for channels that exhibit large variations of
the attenuation in the passband, as in such cases the linear equalizer tends to enhance the noise.
3. Detection errors tend to propagate, because they produce incorrect cancellations. Error propaga-
tion leads to an increase of the error probability. However, simulations indicate that for typical
channels and symbol error probability smaller than 5 × 10−2 , error propagation is not catastrophic.
4. For channels with impulse response {hi }, such that detection errors may propagate catastrophically,
instead of the DFE structure, it is better to implement the linear FF equalizer at the receiver, and the
FB filter at the transmitter as a precoder, using the precoding method discussed in Appendix 7.C
and Chapter 13.

7.4.5 Frequency domain equalization

We consider now an equalization method, which is attractive for its low complexity for both determining
the filter coefficients and performing the filtering operations.

DFE with data frame using a unique word

In this section, a DFE with the FF filter operating in the frequency domain (FD) is given [7, 8]. The
first step concerns the format and size of signals that must be mapped in the FD by the discrete Fourier
transform (DFT) (1.28). With reference to Figure 7.23, we take as new timing phase at the receiver the
instant t00 , corresponding to the first significant sample of the equivalent channel impulse response, and
define the new discrete time channel impulse response as h0i . It is
t00 = t0 − N1 and h0i = hi−N1 , i = 0, 1, … , Nh − 1 (7.214)
with Nh = N1 + N2 + 1. Consequently,
xk = x(t00 + kT), k≥0 (7.215)
Next, we select a suitable information block size . Then, we collect P =  + Nu𝑤 , with
Nu𝑤 ≥ Nh − 1 (7.216)
samples of xk . In the absence of noise, from (7.176)
∑
Nh −1
(aug)
xk = sk = h0i ak−i , k = Nu𝑤 , Nu𝑤 + 1, … , Nu𝑤 + P − 1 (7.217)
i=0
(aug)
where {ak } is the augmented transmitted data sequence. Convolution (7.217) is circular on a block
size P if (see (1.46))
(aug) (aug)
ak = ak+P for k = 0, 1, … , Nu𝑤 − 1 (7.218)
Defined the following DFTs, all of size P,
∑
P−1
ip
p = DFT{h0i } = h0i e−j2𝜋 P (7.219)
i=0
(aug) (aug)
p = DFT{ai , i = Nu𝑤 , Nu𝑤 + 1, … , Nu𝑤 + P − 1} (7.220)

p = DFT{xk , k = Nu𝑤 , Nu𝑤 + 1, … , Nu𝑤 + P − 1} (7.221)

it is
(aug)
 p =  p p , p = 0, 1, … , P − 1 (7.222)
324 Chapter 7. Single-carrier modulation

and
xk = DFT−1 {p } k = Nu𝑤 , Nu𝑤 + 1, … , Nu𝑤 + P − 1 (7.223)
To satisfy property (7.218), we augment each data block of size  with a unique word given by a PN
sequence of length Nu𝑤 , {pk }, k = 0, 1, … , Nu𝑤 − 1. The augmented transmit block of P =  + Nu𝑤
data samples is
a(aug) (𝓁) = [a𝓁 , a𝓁+1 , … , a𝓁+−1 , p0 , p1 , … , pNu𝑤 −1 ]T (7.224)
Moreover, the PN sequence is transmitted before the first data block. In other words, block a(aug) (0) is to
be sent at time k = Nu𝑤 . Consequently, samples xk start to be collected at k = Nu𝑤 up to k = Nu𝑤 + P − 1.
In fact, the first PN sequence absorbs the transient due to the channel impulse response {h0i }.
Let
x(𝓁) = [xNu𝑤 +𝓁P , … , xNu𝑤 +𝓁P+P−1 ]T (7.225)
with
 (𝓁) = DFT[x(𝓁)] (7.226)
and
 = [0 , 1 , … , P−1 ]T (7.227)
the FF filter in the FD. We define the DFT of the FF output
(𝓁) = diag[] (𝓁) (7.228)
whose inverse yields
z(𝓁) = [zNu𝑤 +𝓁P , … , zNu𝑤 +𝓁P+P−1 ]T = DFT−1 [(𝓁)] (7.229)
Out of the P samples of z(𝓁), we only retain the first , i.e. zk for k = Nu𝑤 + 𝓁P, … , Nu𝑤 +
 − 1 + 𝓁P, as the remaining Nu𝑤 are not relevant for detection. In fact, they would correspond to the
PN sequence. A block diagram of the whole receiver is shown in Figure 7.33. Note how the use of the
PN sequence is very useful at the beginning of each detection block where interference is due to the PN
sequence transmitted in the previous block.
Surely transmitting the PN sequence reduces the spectral efficiency by ∕( + Nu𝑤 ). There is also
a power inefficiency as the transmitted data corresponding to the PN sequence do not carry information.
The choice of the PN sequence may be done according to different criteria, including the reduction of
the peak to average power ration of the transmitted signal.
In Figure 7.33 â k−Nu𝑤 , for k = Nu𝑤 , … , Nu𝑤 +  − 1, is the detected value of the information sequence
a0 , a1 , … , a−1 . Moreover, for k = 0, 1, … , Nu𝑤 − 1, the PN sequence is filling up the shift register of
the FB filter and zFB,k can be evaluated starting at k = Nu𝑤 . Next values of zFB,k use also detected data.
Concerning the design derivation of {p }, p = 0, 1, … , P − 1, and {bi }, i = 1, … , Nu𝑤 , the interested
reader is referenced to [7]. Here we report the final equations. We introduce the DFT of correlation of
noise 𝑤̃ k (see (7.187) and (1.23)), for p = 0, 1, … , P − 1, as

1 ∑
∞ | ( ) 2|
𝓁 || ||
|
𝑤,p
̃ = DFT[𝚛𝑤̃ (n)] = N | f− | (7.230)
T 𝓁=−∞ 0 || M T || || p
|f = PT
For an MSE criterion,
J = E[|yk − ak |2 ] (7.231)
defined the matrix R Nu𝑤 × Nu𝑤 and the vector p Nu𝑤 × 1 by
∑
P−1 p(𝓁−m)
e−j2𝜋 P
[R]m,𝓁 = , m, 𝓁 = 1, 2, … , Nu𝑤 (7.232)
p=0 |p |2 + 𝑤,p 2
̃ ∕𝜎a
7.4. Channel equalization 325

C0
xNu𝑤 0 0 zNu𝑤

xNu𝑤 + 1 zNu𝑤 + 1
xk C1
1 1 .. zk
S/P P/S
.
.. DF T .. IDF T z
. . Nu 𝑤 +  − 1

CP – 1
xNu𝑤 + P − 1 P – 1 P – 1
zNu𝑤 + P − 1

zk yk âk–Nu𝑤
detector

zF B, k
(aug)
âk–Nu𝑤
bi P N sequence

Figure 7.33 Block diagram of a DFE with FF in the FD. To simplify notation the block index 𝓁 is
assumed 0.
∑
P−1
ej2𝜋 P
pm

[p]m = , m = 1, 2, … , Nu𝑤 (7.233)

p=0 |p |2 + 𝑤,p 2
̃ ∕𝜎a

the vector with FB coefficients

b = [b1 , … , bNu𝑤 ]T (7.234)
with DFT  = [0 , 1 … , P−1 ], satisfies the following system of Nu𝑤 equations with Nu𝑤 unknowns
Rb = p (7.235)
Moreover,
p∗ (1 − p )
p = , p = 0, 1, … , P − 1 (7.236)
|p |2 + 𝑤,p 2
̃ ∕𝜎a

and
1 ∑ |1 − p | 𝑤,p
P−1 2
̃
Jmin = (7.237)
P p=0 |p | + 𝑤,p
2
̃ ∕𝜎a
2

We conclude this section by mentioning a DFE where also the FB operates in the FD [9]. Now, filters
design and detection are iterated (for a few times) up to reaching convergence. Computational complexity
is further reduced w.r.t. previous DFE structures, and more importantly, there are no systems of equations
to be solved. Moreover, this new iterative DFE yields better performance than the above DFEs. A version
with the FF working at T∕2 was also proposed [10].

Observation 7.9
For a DFE with the FF operating in the FD, the channel estimate of p to be used in filter design and
processing can be FD based. Starting from (7.222), methods to estimate p , p = 0, 1, … , P − 1, are
326 Chapter 7. Single-carrier modulation

given in Section 8.9. A simple and effective approach determines a first rough estimate by sending a
block of known data symbols and estimating the channel by ̂ p = p ∕p . Let the inverse discrete
(aug)

Fourier transform (IDFT) of {̂ p } be {ĥ 0i }, i = 0, 1, … , P − 1. Next, we truncate ĥ 0i to a length Nu𝑤 + 1,

and set to zero the remaining taps. The DFT of size P of the truncated ĥ 0i usually yields a very good
estimate of {p }.

Observation 7.10
In [11, 12], a LE working in the FD was proposed whose structure is similar to the above DFE, obviously,
for bi = 0, i = 1, … , Nu𝑤 . However, as for all LEs, its performance may be much worse than that of
DFE [7]. We recall this LE structure, as there is no cancellation of ISI using known/detected data, the
augmented data structure may use a cyclic prefix of length Ncp , with Ncp ≥ Nh − 1, where the information
data block of size  is partially repeated up front,
a(aug) (𝓁) = [a𝓁+−1−Ncp , … , a𝓁+−1 , a𝓁 , … , a𝓁+−1 ]T (7.238)
As data is circular of period , the advantages is that all DFTs are of size . In fact, (7.222) holds for
P = . If (7.238) is transmitted starting at time 𝓁( + Ncp )T, the receiver drops the first Ncp samples,
due to the channel transient, and stores the remaining  samples for the LE in the FD. This approach
is actually used also in orthogonal frequency division multiplexing (OFDM) (see Section 8.7.1).

7.4.6 LE-ZF
Neglecting the noise, the signal at the output of a LE with N coefficients (see (7.189)) is given by
∑
N−1
∑
N2 +N−1
yk = ci xk−i = 𝜓p ak−p (7.239)
i=0 p=−N1

where from (7.188),

∑
N−1
𝜓p = c𝓁 hp−𝓁 = c0 hp + c1 hp−1 + … + cN−1 hp−(N−1)
𝓁=0 (7.240)
p = −N1 , … , 0, … , N2 + N − 1
For a LE-ZF, it must be
𝜓p = 𝛿p−D (7.241)
where D is a suitable delay.
If the overall impulse response {hn }, n = −N1 , … , N2 , is known, a method to determine the coeffi-
cients of the LE-ZF consists in considering the system (7.240) with Nt = N1 + N2 + N equations and
N unknowns, that can be solved by the method of the pseudoinverse (see (2.151)); alternatively, the
solution can be found in the frequency domain by taking the Nt -point DFT of the various signals (see
(1.43)), and the result windowed in the time domain so that the filter coefficients are given by the N
consecutive coefficients that maximize the energy of the filter impulse response.
An approximate solution is obtained by forcing the condition (7.241) only for N values of p in (7.240)
centred around D; then the matrix of the system (7.240) is square and, if the determinant is different
from zero, it can be inverted.
Note that all these methods require an accurate estimate of the overall impulse response, otherwise,
the equalizer coefficients may deviate considerably from the desired values.
7.4. Channel equalization 327

7.4.7 DFE-ZF with IIR filters

We determine the expressions of FF and FB filters of a DFE in the case of IIR filter structure.
We consider a receiver with a matched filter gM (see Figure 7.22) followed by the DFE illustrated in
Figure 7.34, where, to simplify the notation, we assume t0 = 0 and D = 0. The z-transform of the QAM
system impulse response is given by Φ(z), as defined in (7.132).

Figure 7.34 DFE zero-forcing.

With reference to Figure 7.34, the matched filter output xk is input to a linear equalizer zero forcing
(LE-ZF) with transfer function 1∕Φ(z) to remove ISI: therefore, the LE-ZF output is given by
xE,k = ak + 𝑤E,k (7.242)
From (7.136), using the property (7.133), we obtain that the PSD P𝑤E (z) of 𝑤E,k is given by
1
P𝑤E (z) = N0 Φ(z) ( )
1
Φ(z)Φ∗ z∗
1
= N0 (7.243)
Φ(z)
As Φ(z) is the z-transform of a correlation sequence, it can be factorized as (see page 35)
( )
1
Φ(z) = F(z) F ∗ ∗ (7.244)
z
where
∑
+∞
F(z) = 𝚏n z−n (7.245)
n=0

is a minimum-phase function, that is with poles and zeros inside the unit circle, associated with a causal
sequence {𝚏n }.

Observation 7.11
A useful method to determine the filter F(z) in (7.244), with a computational complexity that
is proportional to the square of the number of filter coefficients, is obtained by considering a
minimum-phase prediction error filter A(z) = 1 + 𝚊′1,N z−1 + · · · + 𝚊′N,N z−N (see page 116), designed
using the auto-correlation sequence {𝚛qC (nT)} defined by (7.130). The equation to determine the
coefficients of A(z) is given by (2.80), where {𝚛x (n)} is now substituted by {𝚛qC (nT)}. The final result
is F(z) = 𝚏0 ∕A(z).
328 Chapter 7. Single-carrier modulation

On the other hand, F ∗ (1∕z∗ ) is a function with zeros and poles outside the unit circle, associated with
an anticausal sequence {𝚏∗−n }.
We choose as transfer function of the filter 𝚠 in Figure 7.34 the function
1
W(z) = F(z) (7.246)
𝚏0
The ISI term in zk is determined by W(z) − 1, hence there are no precursors; the noise is white with
statistical power(N0 ∕𝚏20 ). Therefore, the filter 𝚠 is called whitening filter (WF). In any case, the filter
composed of the cascade of LE-ZF and 𝚠 is also a WF.
If â k = ak then, for
B(z) = 1 − W(z) (7.247)

the FB filter removes the ISI present in zk and leaves the white noise unchanged. As yk is not affected by
ISI, this structure is called DFE-ZF, for which we obtain
2|𝚏0 |2
𝛾DFE−ZF = (7.248)
N0
Summarizing, the relation between xk and zk is given by
1 1 1
F(z) = ( ) (7.249)
Φ(z) 𝚏 0 𝚏 F∗ 1
0 z∗

With this filter, the noise in zk is white. The relation between ak and the desired signal in zk is instead gov-
erned by Ψ(z) = F(z)∕𝚏0 and B(z) = 1 − Ψ(z). In other words, the overall discrete-time system is causal
and minimum phase, that is the energy of the impulse response is mostly concentrated at the beginning
of the pulse. The overall receiver structure is illustrated in Figure 7.35, where the block including the
matched filter, sampler, and whitening filter, is called whitened matched Filter (WMF). Note that the
impulse response at the WF output has no precursors.

Figure 7.35 DFE-ZF as whitened matched filter followed by a canceller of ISI.

In principle, the WF of Figure 7.35 is non-realizable, because it is anticausal. In practice, we can

implement it in two ways:

(a) introducing an appropriate delay in the impulse response of an FIR WF, and processing the output
samples in the forward mode for k = 0, 1, …;
(b) processing the output samples of the IIR WF in the backward mode, for k = K − 1, K − 2, … , 0,
starting from the end of the block of samples.

We observe that the choice F(z) = 𝚏0 ∕A(z), where A(z) is discussed in Observation 7.11 of page 327,
leads to an FIR WF with transfer function 𝚏12 A∗ (1∕z∗ ).
0
7.4. Channel equalization 329

Observation 7.12
With reference to the scheme of Figure 7.35, using a LE-ZF instead of a DFE structure means that a
filter with transfer function 𝚏0 ∕F(z) is placed after the WF to produce the signal xE,k given by (7.242).
For a data detector based on xE,k , the SNR at the detector input 𝛾 is
2
𝛾LE−ZF = (7.250)
𝚛𝑤E (0)
where 𝚛𝑤E (0) is determined as the coefficient of z0 in N0 ∕Φ(z). This expression is alternative to (7.110).

Example 7.4.1 (WF for a channel with exponential impulse response)

A method to determine the WF in the scheme of Figure 7.35 is illustrated by an example. Let
√
qC (t) = EqC 2𝛽e−𝛽t 1(t) (7.251)

be the overall system impulse response at the MF input; in (7.251) EqC is the energy of qC .
The autocorrelation of qC , sampled at instant nT, is given by
𝚛qC (nT) = EqC a|n| a = e−𝛽T < 1 (7.252)
Then
Φ(z) = [𝚛qC (nT)]
(1 − a2 )
= EqC (7.253)
−az + (1 + a2 ) − az
−1
(1 − a2 )
= EqC
(1 − az−1 )(1 − az)
We note that the frequency response of (7.252) is
(1 − a2 )
Φ(ej2𝜋fT ) = EqC (7.254)
1 + a2 − 2a cos(2𝜋fT)
and presents a minimum for f = 1∕(2T).
With reference to the factorization (7.244), it is easy to identify the poles and zeros of Φ(z) inside the
unit circle, hence √
1
F(z) = EqC (1 − a2 )
1 − az−1
√ ∑ n −n
+∞ (7.255)
= EqC (1 − a2 ) a z
n=0

In particular, the coefficient of z0 is given by

√
𝚏0 = EqC (1 − a2 ) (7.256)

The WF of Figure 7.35 is expressed as

1 1
( ) = (1 − az)
1 EqC (1 − a2 )
𝚏0 F ∗
z∗ (7.257)
1
= z(−a + z−1 )
EqC (1 − a2 )
In this case the WF, apart from a delay of one sample (D = 1), can be implemented by a simple FIR
with two coefficients, whose values are equal to −a∕(EqC (1 − a2 )) and 1∕(EqC (1 − a2 )).
330 Chapter 7. Single-carrier modulation

The FB filter is a first-order IIR filter with transfer function

1 ∑ n −n
+∞
1 − F(z) = − a z
𝚏0 n=1
1 (7.258)
=1−
1 − az−1
−az−1
=
1 − az−1

Example 7.4.2 (WF for a two-ray channel)

In this case, we directly specify the autocorrelation sequence at the matched filter output:
( )
1
Φ(z) = QCC (z)Q∗CC (7.259)
z∗
where √
QCC (z) = EqC (q0 + q1 z−1 ) (7.260)
with q0 and q1 such that
|q0 |2 + |q1 |2 = 1 (7.261)
|q0 | > |q1 | (7.262)
In this way, EqC is the energy of {qCC (nT)} and QCC (z) is minimum phase.
Equation (7.260) represents the discrete-time model of a wireless system with a two-ray channel. The
impulse response is given by √
qCC (nT) = EqC (q0 𝛿n + q1 𝛿n−1 ) (7.263)
The frequency response is given by
√
CC (f ) = EqC (q0 + q1 e−j2𝜋fT ) (7.264)
We note that if q0 = q1 , the frequency response has a zero for f = 1∕(2T).
From (7.259) and (7.260), we get
Φ(z) = EqC (q0 + q1 z−1 )(q∗0 + q∗1 z) (7.265)
hence, recalling assumption (7.262),
√
F(z) = EqC (q0 + q1 z−1 ) = QCC (z) (7.266)
and √
𝚏0 = EqC q0 (7.267)
The WF is given by
1 1
( ) = (7.268)
𝚏0 F ∗ 1 EqC |q0 |2 (1 − bz)
z∗
where ( )∗
q
b= − 1
q0
We note that the WF has a pole for z = b−1 , which, recalling (7.262), lies outside the unit circle. In
this case, in order to have a stable filter, it is convenient to associate the z-transform 1∕(1 − bz) with an
anticausal sequence,
1 ∑0
∑∞
= (bz)−i = (bz)n (7.269)
1 − bz i=−∞ n=0
7.4. Channel equalization 331

On the other hand, as |b| < 1, we can approximate the series by considering only the first (N − 1) terms,
obtaining
1 1 ∑N
( ) ≃ (bz)n
1 EqC |q0 |2 n=0
𝚏0 F ∗
(7.270)
z∗
1 N N −(N−1) −N
= z [b + · · · + bz +z ]
EqC |q0 |2
Consequently, the WF, apart from a delay D = N, can be implemented by an FIR filter with N + 1
coefficients.
The FB filter in this case is a simple FIR filter with one coefficient
1 q
1 − F(z) = − 1 z−1 (7.271)
𝚏0 q0

DFE-ZF as noise predictor

Let A(z) = [ak ]. From the identity
Y(z) = XE (z)W(z) + (1 − W(z))A(z)
(7.272)
= XE (z) + (1 − W(z))(A(z) − XE (z))
the scheme of Figure 7.34 is redrawn as in Figure 7.36, where the FB filter acts as a noise predictor. In
fact, for â k = ak , the FB filter input is coloured noise. By removing the correlated noise from xE,k , we
obtain yk that is composed of white noise, with minimum variance, plus the desired symbol ak .

xk zero-forcing
xE,k + yk a^k
equalizer +
+
B(Z)=1–W(Z)
+
b +
−
FB filter as noise predictor

Figure 7.36 Predictive DFE: the FF filter is a linear equalizer zero forcing.

DFE as ISI and noise predictor

A variation of the scheme of Figure 7.36 consists in using as FF filter, a minimum-MSE linear equalizer.
We refer to the scheme of Figure 7.37, where the filter c is given by (7.145). The z-transform of the overall
impulse response at the FF filter output is given by
𝜎a2 Φ(z)
Φ(z)C(z) = (7.273)
N0 + 𝜎a2 Φ(z)
As t0 = 0, the ISI in zk is given by
N0
Φ(z)C(z) − 1 = − (7.274)
N0 + 𝜎a2 Φ(z)
332 Chapter 7. Single-carrier modulation

Figure 7.37 Predictive DFE with the FF filter as a minimum-MSE linear equalizer.

Hence, the PSD of the ISI has the following expression:

N0 N0
PISI (z) = Pa (z) ( )
N0 + 𝜎a2 Φ(z) N0 + 𝜎a2 Φ∗ 1
z∗
𝜎a2 (N0 )2
= (7.275)
(N0 + 𝜎a2 Φ(z))2
using (7.133) and the fact that the symbols are uncorrelated with Pa (z) = 𝜎a2 .
The PSD of the noise in zk is given by
( )
1
Pnoise (z) = N0 Φ(z)C(z)C∗ ∗
z
Φ(z)𝜎a4 (7.276)
= N0
(N0 + 𝜎a2 Φ(z))2
Therefore, the PSD of the disturbance 𝑣k in zk , composed of ISI and noise, is given by
N0 𝜎a2
P𝑣 (z) = (7.277)
N0 + 𝜎a2 Φ(z)
We note that the FF filter could be an FSE and the result (7.277) would not change.
To minimize the power of the disturbance in yk , the FB filter, with input â k − zk = ak − zk = −𝑣k
(assuming â k = ak ), needs to remove the predictable components of zk . For a predictor of infinite length,
we set
∑
+∞
B(z) = 𝚋n z−n (7.278)
n=1

An alternative form is
B(z) = 1 − A(z) (7.279)
where
∑
+∞
A(z) = 𝚊′n z−n , 𝚊′0 = 1 (7.280)
n=0
7.4. Channel equalization 333

is the forward prediction error filter defined in (2.78). To determine B(z), we use the spectral factorization
in (1.453):
𝜎y2
P𝑣 (z) = ( )
∗ 1
A(z)A
z∗
(7.281)
𝜎y2
= [ ( )]
1
[1 − B(z)] 1 − B∗ ∗
z
with P𝑣 (z) given by (7.277).
In conclusion, it results that the prediction error signal yk is a white noise process with statistical
power 𝜎y2 .
An adaptive version of the basic scheme of Figure 7.37 suggests that the two filters, c and b, are
separately adapted through the error signals {eF,k } and {eB,k }, respectively. This configuration, although
sub-optimum with respect to the DFE, is used in conjunction with trellis-coded modulation [13].

7.4.8 Benchmark performance of LE-ZF and DFE-ZF

We compare bounds on the performance of LE-ZF and DFE-ZF, in terms of the SNR at the decision
point, 𝛾.

Comparison
From (7.243), the noise sequence {𝑤E,k } can be modelled as the output of a filter having transfer function
1
CF (z) = (7.282)
F(z)
and input given by white noise with PSD N0 . Because F(z) is causal, also CF (z) is causal:
∑
∞
CF (z) = cF,n z−n (7.283)
n=0

where {cF,n }, n ≥ 0 is the filter impulse response.

Then, we can express the statistical power of 𝑤E,k as
∑
∞
𝚛𝑤E (0) = N0 |cF,n |2 (7.284)
n=0

where, from (7.282),

1 1
cF,0 = C(∞) = = (7.285)
F(∞) 𝚏0
Using the inequality
∑
∞
1
|cF,n |2 ≥ |cF,0 |2 = (7.286)
n=0
|𝚏0 |2
the comparison between (7.248) and (7.250) yields
𝛾LE−ZF ≤ 𝛾DFE−ZF (7.287)
334 Chapter 7. Single-carrier modulation

Performance for two channel models

We analyse now the value of 𝛾 for the two simple systems introduced in Examples 7.4.1 and 7.4.2.

LE-ZF
Channel with exponential power delay profile. From (7.253), the coefficient of z0 in N0 ∕Φ(z) is equal to
N0 1 + a2
𝚛𝑤E (0) = (7.288)
EqC 1 − a2
and consequently, from (7.250)
EqC 1 − a2
𝛾LE−ZF = 2 (7.289)
N0 1 + a2
Using the expression of 𝛾MF (7.107), obtained for a MF receiver in the absence of ISI, we get
1 − a2
𝛾LE−ZF = 𝛾MF (7.290)
1 + a2
Therefore, the loss due to the ISI, given by the factor (1 − a2 )∕(1 + a2 ), can be very large if a is close to
1. In this case, the minimum value of the frequency response in (7.254) is close to 0.

Two-ray channel. From (7.265), we have

1 1
= (7.291)
Φ(z) EqC (q0 + q1 z−1 )(q∗0 + q∗1 z)
By a partial fraction expansion, we find only the pole for z = −q1 ∕q0 lying inside the unit circle, hence,
N0 N0
𝚛𝑤E (0) = ( ) = (7.292)
0 − |q1 | )
|
∗ ∗ q1 E (|q 2 2
EqC q0 q0 − q1 q qC
0

Then
𝛾LE−ZF = 𝛾MF (|q0 |2 − |q1 |2 ) (7.293)
Also in this case, we find that the LE is unable to equalize channels with a spectral zero.

DFE-ZF
Channel with exponential power delay profile. Substituting the expression of 𝚏0 given by (7.256) in
(7.248), we get
2
𝛾DFE−ZF = E (1 − a2 )
N0 qC
= 𝛾MF (1 − a2 ) (7.294)
We note that 𝛾DFE−ZF is better than 𝛾LE−ZF by the factor (1 + a2 ).

Two-ray channel. Substitution of (7.267) in (7.248) yields

2
𝛾DFE−ZF = E |q |2 = 𝛾MF |q0 |2 (7.295)
N0 qC 0
In this case, the advantage with respect to LE-ZF is given by the factor |q0 |2 ∕(|q0 |2 − |q1 |2 ), which may
be substantial if |q1 | ≃ |q0 |.
We recall that in case EqC ≫ N0 , that is for low noise levels, LE and DFE yields similar performance
to LE-ZF and DFE-ZF, respectively. Anyway, for the two systems of Examples 7.4.1 and 7.4.2 the values
of 𝛾 in terms of Jmin are given in [13], for both LE and DFE.
7.4. Channel equalization 335

7.4.9 Passband equalizers

For QAM signals, we analyse alternatives to the baseband equalizers considered at the beginning of this
chapter. We refer to the QAM transmitter scheme of Figure 7.5 and we consider the model of Figure 7.38,
where the transmission channel has impulse response gCh and introduces additive white noise 𝑤. The
received passband signal is given by
[ +∞ ]
∑
r(t) = Re ak qCh (t − kT)e j(2𝜋f0 t+𝜑)
+ 𝑤(t) (7.296)
k=−∞

where, from (7.39) or equivalently from Figure 7.10, qCh is a baseband equivalent pulse with frequency
response
1
Ch (f ) = Tx (f ) (bb) (f ) = Tx (f ) Ch (f + f0 ) 1(f + f0 ) (7.297)
2 Ch

s (t) r (t)
[[

2π 0
φ w (t)

Figure 7.38 Passband modulation scheme with phase offset introduced by the channel.

In this model, the phase 𝜑 in (7.296) implies that arg Ch (0) = 0, Hence, it includes also the phase off-
set introduced by the channel frequency response at f = f0 , in addition to the carrier phase offset between
the transmit and receive carriers. Because of this impairment, which as a first approximation is equivalent
to a rotation of the symbol constellation, a suitable receiver structure needs to be developed [14].
As 𝜑 may be time varying, it can be decomposed as the sum of three terms
𝜑(t) = Δ𝜑 + 2𝜋Δft + 𝜓(t) (7.298)
where Δ𝜑 is a fixed phase offset, Δf is a fixed frequency offset, and 𝜓 is a random or quasi-periodic
term (see the definition of phase noise in (4.13)).
For example, over telephone channels typically |𝜓(t)| ≤ 𝜋∕20. Moreover, the highest frequency of
the spectral components of 𝜓 is usually lower than 0.1∕T: in other words, if 𝜓 were a sinusoidal signal,
it would have a period larger than 10T.
Therefore, 𝜑 may be regarded as a constant, or at least as slowly time varying, at least for a time
interval equal to the duration of the overall system impulse response.

Passband receiver structure

Filtering and equalization of the signal r are performed in the passband. First, a filter extracts the positive
frequency components of r, acting also as a matched filter. It is a complex-valued passband filter, as
illustrated in Figure 7.39.
In particular, from the theory of the optimum receiver, we have
(pb)
gM (t) = gM (t)ej2𝜋f0 t (7.299)
where
gM (t) = q∗Ch (t0 − t) (7.300)
with qCh defined in (7.297).
336 Chapter 7. Single-carrier modulation

(pb)
(f)

Figure 7.39 Frequency response of a passband matched filter.

Then, with reference to Figure 7.40, we have

(pb) (pb) (pb) (pb)
gM,I (t) = Re{gM (t)} and gM,Q (t) = Im{gM (t)} (7.301)

(2 π φ(
(pb) 0

(pb)(t) (pb) (pb)

r (t)
(pb) 0
(pb)
(t) 0
(pb) 0

(pb)

(2 π 0
φ)

Figure 7.40 QAM passband receiver.

(pb) (pb)
If gM in (7.300) is real-valued, then gM,I and gM,Q are related by the Hilbert transform (1.97), that is
(pb) (pb)
gM,Q (t) =  (h) [gM,I (t)] (7.302)
After the passband matched filter, the signal is oversampled with sampling period T∕F0 ; oversampling
is suggested by the following two reasons:
(pb)
1. if qCh is unknown and gM is a simple passband filter, matched filtering is carried out by the filter
c(pb) , hence the need for oversampling;
2. if the timing phase t0 is not accurate, it is convenient to use an FSE.
Let the overall baseband equivalent impulse response of the system at the sampler input be
q(t) =  −1 [(f )] (7.303)
where
(f ) = Ch (f ) M (f ) = Tx (f ) Ch (f + f0 ) M (f ) 1(f + f0 ) (7.304)
The sampled passband signal is given by
( ) ( )
(pb) (pb) T (pb) T
x n = xI t0 + n + jxQ t0 + n
F0 F0
( )
∑
+∞
j 2𝜋f0 n FT +𝜑
= ak hn−kF0 e 0 + 𝑤̃ n (7.305)
k=−∞
7.4. Channel equalization 337

where11 ( )
T
hn = q t0 + n (7.306)
F0
In (7.305), 𝑤̃ n denotes the noise component
( )
T
𝑤̃ n = 𝑤R t0 + n (7.307)
F0
(pb)
where 𝑤R (t) = 𝑤 ∗ gM (t).
For a passband equalizer with N coefficients, the output signal with sampling period equal to T is
given by
(pb)
∑ (pb) (pb)
N−1
(pb)T
yk = ci xkF −i = xkF c (pb) (7.308)
0 0
i=0
(pb)
with the usual meaning of the two vectors xn and c (pb) .
Ideally, it should result
(pb)
yk = ak−D ej(2𝜋f0 kT+𝜑) (7.309)
(pb)
where the phase offset 𝜑 needs to be estimated as 𝜑. ̂ In Figure 7.40, the signal yk is shifted to baseband
by multiplication with the function e−j(2𝜋f0 kT+𝜑)
̂ . Then the data detector follows.

At this point, some observations can be made: as seen in Figure 7.41, by demodulating the received
signal, that is by multiplying it by e−j2𝜋f0 t , before the equalizer or the receive filter we obtain a scheme
equivalent to that of Figure 7.24, with a baseband equalizer. As we will see at the end of this section, the
only advantage of a passband equalizer is that the computational complexity of the receiver is reduced;
in any case, it is desirable to compensate for the presence of the phase offset, that is to multiply the
received signal by e−j𝜑̂ , as near as possible to the decision point, so that the delay in the loop for the
update of the phase offset estimate is small.

Optimization of equalizer coefficients and carrier phase offset

To simplify the notation, the analysis is carried out for F0 = 1. As usual, for an error signal defined as
ek = ak−D − yk (7.310)
where yk is given by (7.308), we desire to minimize the following cost function:
J = E[|ek |2 ]
[ ]
| ̂ |
2
= E |ak−D − xk c (pb) e−j(2𝜋f0 kT+𝜑)
(pb)T
|
| |
[ ]
| |2
= E |ak−D ej(2𝜋f0 kT+𝜑)
̂ (pb)T
− xk c (pb) | (7.311)
| |
Equation (7.311) expresses the classical Wiener problem for a desired signal expressed as
ak−D ej(2𝜋f0 kT+𝜑)
̂
(7.312)
(pb)
and input xk . Assuming 𝜑 known, the Wiener–Hopf solution is given by
(pb)
copt = R−1 p (7.313)
where [ ]
(pb)∗ (pb)T
R = E xk xk (7.314)

11
In (7.305), 𝜑 takes also into account the phase 2𝜋f0 t0 .
338 Chapter 7. Single-carrier modulation

has elements for 𝓁, m = 0, 1, … , N − 1, given by

[ ]
(pb) (pb)∗
[R]𝓁,m = E xk−m xk−𝓁 = 𝚛x(pb) ((𝓁 − m)T)
∑
+∞
= 𝜎a2 hi h∗i−(𝓁−m) ej2𝜋f0 (𝓁−m)T + 𝚛𝑤R ((𝓁 − m)T) (7.315)
i=−∞

and [ ]
p = E ak−D ej(2𝜋f0 kT+𝜑)
̂ (pb)∗
xk (7.316)

has elements for 𝓁 = 0, 1, … , N − 1, given by

[p]𝓁 = [p′ ]𝓁 e−j(𝜑−𝜑)
̂
(7.317)
where
[p′ ]𝓁 = 𝜎a2 h∗D−𝓁 ej2𝜋f0 𝓁T (7.318)
From (7.317), the optimum solution (7.313) is expressed as
copt = R−1 p′ e−j(𝜑−𝜑)
̂
(pb)
(7.319)
where R−1 and p′ do not depend on the phase offset 𝜑.
From (7.319), it can be verified that if 𝜑 is a constant the equalizer automatically compensates for the
(pb)
phase offset introduced by the channel, and the output signal yk remains unchanged.
A difficulty appears if 𝜑 varies (slowly) in time and the equalizer attempts to track it. In fact, to avoid
that the output signal is affected by convergence errors, typically an equalizer in the steady state must not
vary its coefficients by more than 1% within a symbol interval. Therefore, another algorithm is needed
to estimate 𝜑.

Adaptive method
The adaptive LMS algorithm is used for an instantaneous square error defined as
(pb)T (pb) −j(2𝜋f0 kT+𝜑)
̂ 2
|ek |2 = |ak−D − xk c e | (7.320)
The gradient of the function in (7.320) with respect to c (pb) is equal to
∇c (pb) |ek |2 = −2ek (xk e−j(2𝜋f0 kT+𝜑̂ k ) )∗
(pb)

= −2ek ej(2𝜋f0 kT+𝜑̂ k ) xk

(pb)∗

(pb) (pb)∗
= −2ek xk (7.321)
where
= ek ej(2𝜋f0 kT+𝜑̂ k )
(pb)
ek (7.322)
The law for coefficient adaptation is given by
(pb) (pb) (pb) (pb)∗
ck+1 = ck + 𝜇ek xk (7.323)
We now compute the gradient with respect to 𝜑̂ k . Let
𝜃 = 2𝜋f0 kT + 𝜑̂ k (7.324)
then
|ek |2 = (ak−D − yk )(ak−D − yk )∗
(pb) (pb)∗ j𝜃
= (ak−D − yk e−j𝜃 )(a∗k−D − yk e ) (7.325)
7.4. Channel equalization 339

Therefore, we obtain
𝜕 (pb) (pb)∗
∇𝜑̂ |ek |2 = |e |2 = jyk e−j𝜃 e∗k − ek jyk ej𝜃
𝜕𝜃 [ k ( ) ]
(pb)
= 2Im ek yk e−j𝜃 ∗
= 2Im[ek yk ∗ ] (7.326)
As ek = ak−D − yk , (7.326) may be rewritten as
∇𝜑̂ |ek |2 = 2Im[ak−D y∗k ] (7.327)
We note that Im[ak−D y∗k ] is related to the sine of the phase difference between ak−D and yk , therefore, the
algorithm has reached convergence only if the phase of yk coincides (on average) with that of ak−D .
The law for updating the phase offset estimate is given by
𝜑̂ k+1 = 𝜑̂ k − 𝜇𝜑 Im[ek y∗k ] (7.328)
or
𝜑̂ k+1 = 𝜑̂ k − 𝜇𝜑 Im[ak−D y∗k ] (7.329)
The adaptation gain is typically normalized as
𝜇̃ 𝜑
𝜇𝜑 = (7.330)
|ak−D | |yk |
In general, 𝜇𝜑 is chosen larger than 𝜇, so that the variations of 𝜑 are tracked by the carrier phase offset
estimator, and not by the equalizer.
In the ideal case, we have
(pb)
yk = ak−D ej(2𝜋f0 kT+𝜑) (7.331)
and
yk = ak−D ej(𝜑−𝜑̂ k ) (7.332)
Therefore, the adaptive algorithm becomes:
𝜑̂ k+1 = 𝜑̂ k − 𝜇̃ 𝜑 Im[e−j(𝜑−𝜑̂ k ) ]
= 𝜑̂ k + 𝜇̃ 𝜑 sin(𝜑 − 𝜑̂ k ) (7.333)
which is the equation of a first order phase-locked-loop (PLL) (see Section 14.7). For a constant phase
offset, at convergence we get 𝜑̂ = 𝜑.

(pb) 0 4 (pb) (pb)

r(t)
(pb)
(t) (pb) 2k
2 (pb) 2

4 2

(pb)

Figure 7.41 QAM passband receiver for transmission over telephone channels.
340 Chapter 7. Single-carrier modulation

7.5 Optimum methods for data detection

Adopting an MSE criterion at the decision point, we have derived the configuration of Figure 7.23 for
an LE, and that of Figure 7.31 for a DFE. In both cases, the decision on a transmitted symbol ak−D is
based only on yk through a memoryless threshold detector.
Actually, the decision criterion that minimizes the probability that an error occurs in the detection
of a symbol of the sequence {ak } requires in general that the entire sequence of received samples is
considered for symbol detection.
We assume a sampled signal having the following structure:
zk = uk + 𝑤k k = 0, 1, … , K − 1 (7.334)
where
● uk is the desired signal that carries the information,
∑
L2
uk = 𝜂n ak−n (7.335)
n=−L1

where 𝜂0 is the sample of the overall system impulse response, obtained in correspondence of
the optimum timing phase; {𝜂−L1 , … , 𝜂−1 } are the precursors, which are typically negligible with
respect to 𝜂0 . Recalling the expression of the pulse {𝜓p } given by (7.188), we have 𝜂n = 𝜓n+D .
We assume the coefficients {𝜂n } are known; in practice, however, they are estimated by the methods
discussed in Section 7.8;
● 𝑤k is a circularly symmetric white Gaussian noise, with equal statistical power in the two I and Q
components given by 𝜎I2 = 𝜎𝑤2 ∕2; hence, the samples {𝑤k } are uncorrelated and therefore, statisti-
cally independent.
In this section, a general derivation of optimum detection methods is considered; possible applications
span, e.g. decoding of convolutional codes (see Chapter 11), coherent demodulation of constant-phase
modulation (CPM) signals (see Appendix 16.A), and obviously detection of sequences transmitted over
channels with ISI.
We introduce the following vectors with K components (K may be very large):
1. Sequence of transmitted symbols, or information message, modelled as a sequence of r.v.s. from a
finite alphabet
a = [aL1 , aL1 +1 , … , aL1 +K−1 ]T , ai ∈  (7.336)
2. Sequence of detected symbols, modelled as a sequence of r.v.s. from a finite alphabet
â = [̂aL1 , â L1 +1 , … , â L1 +K−1 ]T , â i ∈  (7.337)
3. Sequence of detected symbol values
𝜶 = [𝛼L1 , 𝛼L1 +1 , … , 𝛼L1 +K−1 ]T , 𝛼i ∈  (7.338)
4. Sequence of received samples, modelled as a sequence of complex r.v.s.
z = [z0 , z1 , … , zK−1 ]T (7.339)
5. Sequence of received sample values, or observed sequence
𝝆 = [𝜌0 , 𝜌1 , … , 𝜌K−1 ]T , 𝜌i ∈ , 𝝆 ∈ K (7.340)
7.5. Optimum methods for data detection 341

Let M be the cardinality of the alphabet . We divide the vector space of the received samples,  K ,
into M K non-overlapping regions
𝜶 , 𝜶 ∈ K (7.341)

such that, if 𝝆 belongs to 𝜶 , then the sequence 𝜶 is detected:

if 𝝆 ∈ 𝜶 =⇒ â = 𝜶 (7.342)

The probability of a correct decision is computed as follows:

P[𝙲] = P[̂a = a]
∑
= P[̂a = 𝜶 ∣ a = 𝜶]P[a = 𝜶]
𝜶∈K
∑
= P[z ∈ 𝜶 ∣ a = 𝜶]P[a = 𝜶]
𝜶∈K
∑
= pz∣a (𝝆 ∣ 𝜶)d𝝆P[a = 𝜶] (7.343)
∫𝜶
𝜶∈K

The following criteria may be adopted to maximize (7.343).

Maximum a posteriori probability (MAP) criterion

𝝆 ∈ 𝜶opt (and â = 𝜶 opt ) if 𝜶 opt = arg maxK pz∣a (𝝆 ∣ 𝜶)P[a = 𝜶] (7.344)

𝜶∈

Using the identity

P[a = 𝜶 ∣ z = 𝝆]
pz∣a (𝝆 ∣ 𝜶) = pz (𝝆) (7.345)
P[a = 𝜶]
the decision criterion becomes
â = arg maxK P[a = 𝜶 ∣ z = 𝝆] (7.346)
𝜶∈

An efficient realization of the MAP criterion will be developed in Section 7.5.2.

If all data sequences are equally likely, the MAP criterion coincides with the maximum-likelihood
sequence detection (MLSD) criterion

â = arg maxK pz∣a (𝝆 ∣ 𝜶) (7.347)

𝜶∈

In other words, the sequence 𝜶 is chosen, for which the probability to observe z = 𝝆 is maximum.
Note that the above derivation is simply a vector extension of Section 6.2.1.

7.5.1 Maximum-likelihood sequence detection

We now discuss a computationally efficient method to find the solution indicated by (7.347). As the
vector a of transmitted symbols is hypothesized to assume the value 𝜶, both a and u = [u0 , … , uK−1 ]T
are fixed. Recalling that the noise samples are statistically independent, from (7.334), we get12
∏
K−1
pz∣a (𝝆 ∣ 𝜶) = pzk ∣a (𝜌k ∣ 𝜶) (7.348)
k=0

12
We note that (7.348) formally requires that the vector a is extended to include the symbols a−L2 , … , aL1 −1 . In fact, the
symbols may be known (see (7.395) or unknown (see (7.391)).
342 Chapter 7. Single-carrier modulation

As 𝑤k is a complex-valued Gaussian r.v. with zero mean and variance 𝜎𝑤2 , it follows
∏
K−1
1 − 𝜎1𝑤2 |𝜌k −uk |2
pz∣a (𝝆 ∣ 𝜶) = e (7.349)
k=0 𝜋𝜎𝑤2
Taking the logarithm, which is a monotonic increasing function, of both members, we get
∑
K−1
− ln pz∣a (𝝆 ∣ 𝜶) ∝ |𝜌k − uk |2 (7.350)
k=0

where non-essential constant terms have been neglected.

Then the MLSD criterion is formulated as
∑
K−1
â = arg min |𝜌k − uk |2 (7.351)
𝜶
k=0

where uk , defined by (7.335), is a function of the transmitted symbols expressed by the general relation
uk = f̃ (ak+L1 , … , ak , … , ak−L2 ) (7.352)
We note that (7.351) is a particular case of the minimum distance criterion, and it suggests to detect the
vector u that is closest to the observed vector 𝝆. However, we are interested in detecting the symbols
{ak } and not the components {uk }.
A direct computation method requires that, given the sequence of observed samples, for each possible
data sequence 𝜶 of length K, the corresponding K output samples, elements of the vector u, should be
determined, and the relative distance, or metric, should be computed as
∑
K−1
ΓK−1 = |𝜌k − uk |2 (7.353)
k=0

The detected sequence is the sequence that yields the smallest value of Γ(K − 1); as in the case of i.i.d.
symbols, there are M K possible sequences, this method has a complexity O(M K ).

Lower bound to error probability using MLSD

We interpret the vector u as a function of the sequence a, that is u = u(a). In the signal space spanned
by u, we compute the distances
d2 (u(𝜶), u(𝜷)) = ‖u(𝜶) − u(𝜷)‖2 (7.354)
for each possible pair of distinct 𝜶 and 𝜷 in K . As the noise is white, we define
Δ2min = min d2 (u(𝜶), u(𝜷)) (7.355)
𝜶,𝜷

Then the minimum-distance lower bound on the error probability [3, (5.116)] can be used, and we get
( )
N Δmin
Pe ≥ min Q (7.356)
MK 2𝜎I
where M K is the number of vectors u(a), and Nmin is the number of vectors u(a) whose distance from
another vector is Δmin .
In practice, the exhaustive method for the computation of the expressions in (7.353) and (7.355) is not
used; the Viterbi algorithm, that will be discussed in the next section, is utilized instead.
7.5. Optimum methods for data detection 343

The Viterbi algorithm

The Viterbi algorithm (VA) efficiently implements the maximum likelihood (ML) criterion. With refer-
ence to (7.352), it is convenient to describe {uk } as the output sequence of a finite-state machine (FSM),
as discussed in Appendix 7.B. In this case the input is ak+L1 , , the state is
sk = (ak+L1 , ak+L1 −1 , … , ak , … , ak−L2 +1 ) (7.357)
and the output is given in general by (7.352).
We denote by  the set of the states, that is the set of possible values of sk :
sk ∈  = {𝝈 1 , 𝝈 2 , … , 𝝈 Ns } (7.358)
With the assumption of i.i.d. symbols, the number of states is equal to Ns = M L1 +L2 .

Observation 7.13
We denote by s′k−1 the vector that is obtained by removing from sk−1 the oldest symbol, ak−L2 . Then
sk = (ak+L1 , s′k−1 ) (7.359)
From (7.352) and (7.357), we may define uk as a function of sk and sk−1 as
uk = f (sk , sk−1 ) (7.360)

Defining the metric

∑
k
Γk = |𝜌i − ui |2 (7.361)
i=0

the following recursive equation holds:

Γk = Γk−1 + |𝜌k − uk |2 (7.362)
or, using (7.360),
Γk = Γk−1 + |𝜌k − f (sk , sk−1 )|2 (7.363)
Thus, we have interpreted {uk } as the output sequence of a finite state machine, and we have expressed
recursively the metric (7.353). Note that the metric is a function of the sequence of states s0 , s1 , … , sk ,
associated with the sequence of output samples u0 , u1 , … , uk . The following example illustrates how to
describe the transitions between states of the FSM.

Example 7.5.1
Let us consider a transmission system with symbols taken from a binary alphabet, that is M = 2, ak ∈
{−1, 1}, and overall impulse response characterized by L1 = L2 = 1.
In this case, we have
sk = (ak+1 , ak ) (7.364)
and the set of states contains Ns = 22 = 4 elements:
 = {𝝈 1 = (−1, −1), 𝝈 2 = (−1, 1), 𝝈 3 = (1, −1), 𝝈 4 = (1, 1)} (7.365)
The possible transitions
sk−1 = 𝝈 i −−−−→ sk = 𝝈 j (7.366)
344 Chapter 7. Single-carrier modulation

are represented in Figure 7.42, where a dot indicates a possible value of the state at a certain instant, and
a branch indicates a possible transition between two states at consecutive instants. According to (7.359),
the variable that determines a transition is ak+L1 . The diagram of Figure 7.42, extended for all instants
k, is called trellis diagram. We note that in this case there are exactly M transitions that leave each state
sk−1 ; likewise, there are M transitions that arrive to each state sk .

sk−1 = (ak, ak−1) sk = (ak+1, ak)

σ1 = (−1,−1) −1 (−1,−1)
1

σ2 = (−1,1) −1 (−1,1)
1

σ3 = (1,−1) −1 (1,−1)
1

σ4 =(1,1) −1 (1,1)
1

a k + L1 ∈

Figure 7.42 Portion of the trellis diagram showing the possible transitions from state sk−1 to state
sk , as a function of the symbol ak+L1 ∈ .

With each state 𝝈 j , j = 1, … , Ns , at instant k we associate two quantities:

1. the path metric, or cost function, defined as:
Γ(sk = 𝝈 j ) = min Γk ; (7.367)
s0 ,s1 ,…,sk =𝝈 j

2. the survivor sequence, defined as the sequence of symbols that ends in that state and determines
Γ(sk = 𝝈 j ):
(sk = 𝝈 j ) = (s0 , s1 , … , sk = 𝝈 j ) = (aL1 , … , ak+L1 ) (7.368)
Note that the notion of survivor sequence can be equivalently applied to a sequence of symbols or
to a sequence of states.
These two quantities are determined recursively. In fact, it is easy to verify that if, at instant k, a
survivor sequence of states includes sk = 𝝈 j then, at instant k − 1, the same sequence includes sk−1 =
𝝈 iopt , which is determined as follows:

𝝈 iopt = arg min Γ(sk−1 = 𝝈 i ) + |𝜌k − f (𝝈 j , 𝝈 i )|2 (7.369)

sk−1 =𝝈 i ∈→sk =𝝈 j

The term |𝜌k − f (𝝈 j , 𝝈 i )|2 is called branch metric. Therefore, we obtain

Γ(sk = 𝝈 j ) = Γ(sk−1 = 𝝈 iopt ) + |𝜌k − f (𝝈 j , 𝝈 iopt )|2 (7.370)
and the survivor sequence is augmented as follows:
(sk = 𝝈 j ) = ((sk−1 = 𝝈 iopt ), 𝝈 j ) (7.371)
7.5. Optimum methods for data detection 345

Starting from k = 0, with initial state s−1 , which may be known or arbitrary, the procedure is repeated
until k = K − 1. The optimum sequence of states is given by the survivor sequence (sK−1 = 𝝈 jopt ) asso-
ciated with sK−1 = 𝝈 jopt having minimum cost.
If the state s−1 is known and equal to 𝝈 i0 , it is convenient to assign to the states s−1 the following costs:
{
0, for 𝝈 i = 𝝈 i0
Γ(s−1 = 𝝈 i ) = (7.372)
∞, otherwise
Analogously, if the final state sK is equal to 𝝈 f0 , the optimum sequence of states coincides with the
survivor sequence associated with the state sK−1 = 𝝈 j having minimum cost among those that admit a
transition into sK = 𝝈 f0 .

Example 7.5.2
Let us consider a system with the following characteristics: ak ∈ {−1, 1}, L1 = 0, L2 = 2, K = 4,
and s−1 = (−1, −1). The development of the survivor sequences on the trellis diagram from k = 0 to
k = K − 1 = 3 is represented in Figure 7.43a. The branch metric, |𝜌k − f (𝝈 j , 𝝈 i )|2 , associated with each
transition is given in this example and is written above each branch. The survivor paths associated with
each state are represented in bold; we note that some paths are abruptly interrupted and not extended at
the following instant: for example the path ending at state 𝝈 3 at instant k = 1.

k = −1 k=0 k=1 k=2 k=3

STATE
1 1 0 3
σ1 = (−1,−1)
2 4

2 1 0
σ2 = (−1,1)
1 2
1 1 2 5
σ3 = (1,−1)
0 4

0 1 3
σ4 = (1,1)
1 4

(a)

Γ Γ (a0) Γ (a0 a1) Γ (a0 a1 a2) Γ (a0 a1 a2 a3)

1 0 3
0 1 2 2 5
(−1) (−1,−1) 2 (–1,–1,–1) 4 (−1,−1,−1,−1)

1 0
2 3 2 3
(1,−1) 1 (1, 1,–1) 2 (1,−1,1,−1)
1
2 5
1 2 3 6
(1) (−1,1) 0 (1,–1,1) 4 (1,1,−1,1)
0
1 3
1 2 6
(1,1) 1 (1,1,1) 4 (1,1,1,1)

(b)

Figure 7.43 Trellis diagram and determination of the survivor sequences. (a) Trellis diagram from
k = −1 to k = K − 1 = 3. (b) Determination of the survivors.
346 Chapter 7. Single-carrier modulation

Figure 7.43b illustrates how the survivor sequences of Figure 7.43a are determined. Starting with
s−1 = (−1, −1), we have
Γ(s−1 = 𝝈 1 ) = 0
Γ(s−1 = 𝝈 2 ) = ∞
(7.373)
Γ(s−1 = 𝝈 3 ) = ∞
Γ(s−1 = 𝝈 4 ) = ∞
We apply (7.369) for k = 0; starting with s0 = 𝝈 1 , we obtain
Γ(s0 = 𝝈 1 ) = min{Γ(s−1 = 𝝈 1 ) + 1, Γ(s−1 = 𝝈 2 ) + 3}
= min{1, ∞}
=1 (7.374)
We observe that the result (7.374) is obtained for s−1 = 𝝈 1 . Then the survivor sequence associated with
s0 = 𝝈 1 is (s0 = 𝝈 1 ) = (−1), expressed as a sequence of symbols rather than states.
Considering now s0 = 𝝈 2 , 𝝈 3 , and 𝝈 4 , in sequence, and applying (7.369), the first iteration is com-
pleted. Obviously, there is no interest in determining the survivor sequence for states with metric equal
to ∞, because the corresponding path will not be extended.
Next, for k = 1, the final metrics and the survivor sequences are shown in the second diagram of
Figure 7.43b, where we have
Γ(s1 = 𝝈 1 ) = min{1 + 1, ∞} = 2 (7.375)
Γ(s1 = 𝝈 2 ) = min{1 + 2, ∞} = 3 (7.376)
Γ(s1 = 𝝈 3 ) = min{1 + 1, ∞} = 2 (7.377)
Γ(s1 = 𝝈 4 ) = min{1 + 0, ∞} = 1 (7.378)
The same procedure is repeated for k = 2, 3, and the trellis diagram is completed. The minimum among
the values assumed by Γ(s3 ) is
min Γ(s3 = 𝝈 i ) = Γ(s3 = 𝝈 2 ) = 3 (7.379)
i∈{1,…,Ns }

The associated optimum survivor sequence is

a0 = 1, a1 = −1, a2 = 1, a3 = −1 (7.380)
In practice, if the parameter K is large, the length of the survivor sequences is limited to a value Kd ,
called transient length or trellis depth or path memory depth, that typically is between 3 and 10 times
the length of the channel impulse response: this means that at every instant k, we decide on ak−Kd +L1 , a
value that is then removed from the diagram. The decision is based on the survivor sequence associated
with the minimum among the values of Γ(sk ).
Indeed, in most cases, we should find that survivor sequences associated with all states at time k,
contain the same symbol value in correspondence of ak−Kd +L1 , hence, this symbol can be detected.
In practice, k and Γ(sk ) may become very large; then the latter value is usually normalized by sub-
tracting the same amount from all the metrics, for example the smallest of the metrics Γ(sk = 𝝈 i ),
i = 1, … , Ns .

Computational complexity of the VA

Memory. The memory to store the metrics Γ(sk ), Γ(sk−1 ) and the survivor sequences is proportional to
the number of states Ns .
7.5. Optimum methods for data detection 347

Computational complexity. The number of additions and comparisons is proportional to the number
of transitions, MNs = M (L1 +L2 +1) . In any case, the complexity is linear in K.
We note that the values of f (𝝈 j , 𝝈 i ), that determine uk in (7.360) can be memorized in a table, limited
to the possible transitions sk−1 = 𝝈 i → sk = 𝝈 j , for 𝝈 i , 𝝈 j ∈ . However, at every instant k and for every
transition 𝝈 i → 𝝈 j , the branch metric |𝜌k − f (𝝈 j , 𝝈 i )|2 needs to be evaluated: this computation, however,
can be done outside the recursive algorithm.

7.5.2 Maximum a posteriori probability detector

We now discuss an efficient method to compute the a posteriori probability (APP) in (7.346) for an
isolated symbol. Given a vector z = [z0 , z1 , … , zK−1 ]T , the notation zm
𝓁
indicates the vector formed only
by the components [z𝓁 , z𝓁+1 , … , zm ]T .
We introduce the likelihood function,

𝙻ak+L (𝛼) = P[ak+L1 = 𝛼 ∣ zK−1

0 = 𝝆K−1
0 ], 𝛼∈ (7.381)
1

Then, we have
â k+L1 = arg max 𝙻ak+L (𝛼), k = 0, 1, … , K − 1 (7.382)
𝛼∈ 1

As for the VA, it is convenient to define the state

sk = (ak+L1 , ak+L1 −1 , … , ak−L2 +1 ) (7.383)

so that the desired signal at instant k, uk , given by (7.335), turns out to be only a function of sk and sk−1
(see (7.360)).
The formulation that we give for the solution of the problem (7.382) is called forward–backward
algorithm (FBA) and it follows the work by Rabiner [15]: it is seen that it coincides with the
Bahl-Cocke–Jelinek–Raviv (BCJR) algorithm [16] for the decoding of convolutional codes.
We observe that, unlike in (7.383), in the two formulations [15, 16] the definition of state also includes
the symbol ak−L2 and, consequently, uk is only a function of the state at the instant k.

Statistical description of a sequential machine

Briefly, we give a statistical description of the sequential machine associated with the state sk .

1. Let Ns be the number of values that sk can assume. For a sequence of i.i.d. symbols {ak }, we have
Ns = M L1 +L2 ; as in (7.359), the values assumed by the state are denoted by 𝝈 j , j = 1, … , Ns .
2. The sequence {sk } is obtained by a time invariant sequential machine with transition probabilities

Π(j ∣ i) = P[sk = 𝝈 j ∣ sk−1 = 𝝈 i ] (7.384)

Now, if there is a transition from sk−1 = 𝝈 i to sk = 𝝈 j , determined by the symbol ak+L1 = 𝛼, 𝛼 ∈ ,

then
Π(j ∣ i) = P[ak+L1 = 𝛼] (7.385)

which represents the a priori probability of the generic symbol. There are algorithms to iteratively
estimate this probability from the output of another decoder or equalizer (see Section 11.6). Here,
for the time being, we assume there is no a priori knowledge on the symbols. Consequently, for
i.i.d. symbols, we have that every state sk = 𝝈 j can be reached through M states, and
1
P[ak+L1 = 𝛼] = , 𝛼∈ (7.386)
M
348 Chapter 7. Single-carrier modulation

If there is no transition from sk−1 = 𝝈 i to sk = 𝝈 j , we set

Π(j ∣ i) = 0 (7.387)
3. The channel transition probabilities are given by
pzk (𝜌k ∣ j, i) = P[zk = 𝜌k ∣ sk = 𝝈 j , sk−1 = 𝝈 i ] (7.388)
assuming that there is a transition from sk−1 = 𝝈 i to sk = 𝝈 j . For a channel with complex-valued
additive Gaussian noise, (6.23) holds, and
1 − 𝜎1𝑤2 |𝜌k −uk |2
pzk (𝜌k ∣ j, i) = e (7.389)
𝜋𝜎𝑤2
where uk = f (𝝈 j , 𝝈 i ).
4. We merge (7.384) and (7.388) by defining the variable
Ck (j ∣ i) = P[zk = 𝜌k , sk = 𝝈 j ∣ sk−1 = 𝝈 i ]
= P[zk = 𝜌k ∣ sk = 𝝈 j , sk−1 = 𝝈 i ] P[sk = 𝝈 j ∣ sk−1 = 𝝈 i ]
= pzk (𝜌k ∣ j, i) Π(j ∣ i) (7.390)
5. Initial and final conditions are given by
1
pj = P[s−1 = 𝝈 j ] = , j = 1, … , Ns (7.391)
Ns
1
qj = P[sK = 𝝈 j ] = , j = 1, … , Ns (7.392)
Ns
CK (j ∣ i) = Π(j ∣ i), i, j = 1, … , Ns (7.393)
If the initial and/or final state are known, for example
s−1 = 𝝈 i0 , sK = 𝝈 f0 (7.394)
we set {
1 for 𝝈 j = 𝝈 i0
pj = (7.395)
0 otherwise
and {
1 for 𝝈 j = 𝝈 f0
qj = (7.396)
0 otherwise

The forward–backward algorithm

We consider the following four metrics.

(A) Forward metric:

Fk (j) = P[zk0 = 𝝆k0 , sk = 𝝈 j ] (7.397)
Equation (7.397) gives the probability of observing the sequence 𝜌0 , 𝜌1 , … , 𝜌k , up to instant k, and the
state 𝝈 j at instant k.

Theorem 7.1
Fk (j) can be defined recursively as follows:
1. Initialization
F−1 (j) = pj , j = 1, … , Ns (7.398)
7.5. Optimum methods for data detection 349

2. Updating for k = 0, 1, … , K − 1,
∑
Ns
Fk (j) = Ck (j ∣ 𝓁) Fk−1 (𝓁), j = 1, … , Ns (7.399)
𝓁=1 ◽

Proof. Using the total probability theorem, and conditioning the event on the possible values of sk−1 ,
we express the probability in (7.397) as
∑
Ns
Fk (j) = P[zk−1
0 = 𝝆k−1
0 , zk = 𝜌k , sk = 𝝈 j , sk−1 = 𝝈 𝓁 ]
𝓁=1
(7.400)
∑
Ns
= P[zk−1
0 = 𝝆k−1
0 , zk = 𝜌k ∣ sk = 𝝈 j , sk−1 = 𝝈 𝓁 ] P[sk = 𝝈 j , sk−1 = 𝝈 𝓁 ]
𝓁=1

Because the noise samples are i.i.d., once the values of sk and sk−1 are assigned, the event [zk−1
0
= 𝝆k−1
0
]
is independent of the event [zk = 𝜌k ], and it results
∑
Ns
Fk (j) = P[zk−1
0 = 𝝆k−1
0 ∣ sk = 𝝈 j , sk−1 = 𝝈 𝓁 ]
𝓁=1

P[zk = 𝜌k ∣ sk = 𝝈 j , sk−1 = 𝝈 𝓁 ] P[sk = 𝝈 j , sk−1 = 𝝈 𝓁 ] (7.401)

Moreover, given sk−1 , the event [zk−1
0
= 𝝆k−1
0
] is independent of sk , and we have
∑
Ns
Fk (j) = P[zk−1
0 = 𝝆k−1
0 ∣ sk−1 = 𝝈 𝓁 ]
𝓁=1 (7.402)
P[zk = 𝜌k ∣ sk = 𝝈 j , sk−1 = 𝝈 𝓁 ] P[sk = 𝝈 j , sk−1 = 𝝈 𝓁 ]
By applying Bayes’ rule, (7.402) becomes
∑
Ns
1
Fk (j) = P[zk−1 = 𝝆k−1
0 , sk−1 = 𝝈 𝓁 ] P[z = 𝜌k , sk = 𝝈 j , sk−1 = 𝝈 𝓁 ]
𝓁=1
0
P[sk−1 = 𝝈 𝓁 ] k
(7.403)
∑
Ns
= P[zk−1
0 = 𝝆k−1
0 , sk−1 = 𝝈 𝓁 ]P[zk = 𝜌k , sk = 𝝈 j ∣ sk−1 = 𝝈 𝓁 ]
𝓁=1
Replacing (7.390) in (7.403) yields (7.399). ◽

(B) Backward metric:

k+1 = 𝝆k+1 ∣ sk = 𝝈 i ]
Bk (i) = P[zK−1 K−1
(7.404)
Equation (7.404) is the probability of observing the sequence 𝜌k+1 , … , 𝜌K−1 , from instant k + 1 onwards,
given the state 𝝈 i at instant k.

Theorem 7.2
A recursive expression also exists for Bk (i).
1. Initialization
BK (i) = qi , i = 1, … , Ns (7.405)

2. Updating for k = K − 1, K − 2, … , 0,
∑
Ns
Bk (i) = Bk+1 (m) Ck+1 (m ∣ i), i = 1, … , Ns (7.406)
m=1 ◽
350 Chapter 7. Single-carrier modulation

Proof. Using the total probability theorem, and conditioning the event on the possible values of sk+1 ,
we express the probability in (7.404) as
∑
Ns

k+1 = 𝝆k+1 , sk+1 = 𝝈 m ∣ sk = 𝝈 i ]

P[zK−1 K−1
Bk (i) =
m=1

∑Ns

k+1 = 𝝆k+1 ∣ sk+1 = 𝝈 m , sk = 𝝈 i ] P[sk+1 = 𝝈 m ∣ sk = 𝝈 i ]

P[zK−1 K−1
= (7.407)
m=1

Now, given the values of sk+1 and sk , the event [zK−1

k+2
= 𝝆K−1
k+2
] is independent of [zk+1 = 𝜌k+1 ]. In turn,
assigned the value of sk+1 , the event [zk+2 = 𝝆k+2 ] is independent of sk .
K−1 K−1

Then (7.407) becomes

∑
Ns

k+2 = 𝝆k+2 ∣ sk+1 = 𝝈 m , sk = 𝝈 i ]

P[zK−1 K−1
Bk (i) =
m=1
P[zk+1 = 𝜌k+1 ∣ sk+1 = 𝝈 m , sk = 𝝈 i ] P[sk+1 = 𝝈 m ∣ sk = 𝝈 i ]
∑
Ns

k+2 = 𝝆k+2 ∣ sk+1 = 𝝈 m ]

P[zK−1 K−1
=
m=1
P[zk+1 = 𝜌k+1 ∣ sk+1 = 𝝈 m , sk = 𝝈 i ] P[sk+1 = 𝝈 m ∣ sk = 𝝈 i ] (7.408)
Observing (7.404) and (7.390), (7.406) follows. ◽

(C) State metric:

Vk (i) = P[sk = 𝝈 i ∣ zK−1
0 = 𝝆K−1
0 ] (7.409)
Equation (7.409) expresses the probability of being in the state 𝝈 i at instant k, given the whole observa-
tion 𝝆K−1
0
.

Theorem 7.3
Equation (7.409) can be expressed as a function of the forward and backward metrics,
F (i) Bk (i)
Vk (i) = ∑N k , i = 1, … , Ns (7.410)
◽
s
n=1
F k (n) B k (n)

Proof. Using the fact that, given the value of sk , the r.v.s. {zt } with t > k are statistically independent
of {zt } with t ≤ k, from (7.409) it follows
1
Vk (i) = P[zk0 = 𝝆k0 , zK−1
k+1 = 𝝆k+1 , sk = 𝝈 i ]
K−1
P[zK−1
0
= 𝝆K−10
]
1
= P[zk0 = 𝝆k0 , sk = 𝝈 i ] P[zK−1
k+1 = 𝝆k+1 ∣ sk = 𝝈 i ]
K−1
(7.411)
P[zK−1
0
= 𝝆K−1
0
]
Observing the definitions of forward and backward metrics, (7.410) follows. ◽

We note that the normalization factor

∑
Ns
P[zK−1
0 = 𝝆K−1
0 ]= Fk (n) Bk (n) (7.412)
n=1
makes Vk (i) a probability, so that
∑
Ns
Vk (i) = 1 (7.413)
i=1
7.5. Optimum methods for data detection 351

(D) Likelihood function of the generic symbol: Applying the total probability theorem to (7.381), we
obtain the relation
∑
Ns
𝙻ak+L (𝛼) = P[ak+L1 = 𝛼, sk = 𝝈 i ∣ zK−1
0 = 𝝆K−1
0 ] (7.414)
1
i=1

From the comparison of (7.414) with (7.409), indicating with [𝝈 i ]m , m = 0, … , L1 + L2 − 1, the mth
component of the state 𝝈 i (see (7.483)), we have

∑
Ns
𝙻ak+L (𝛼) = Vk (i), 𝛼∈ (7.415)
1
i=1
condition
[𝝈 i ]0 = 𝛼

In other words, at instant k, the likelihood function coincides with the sum of the metrics Vk (i) associated
with the states whose first component is equal to the symbol of value 𝛼. Note that 𝙻ak+L (𝛼) can also be
1
obtained using the state metrics evaluated at different instants, that is

∑
Ns
𝙻ak+L (𝛼) = Vk+m (i), 𝛼∈ (7.416)
1
i=1
[𝝈 i ]m = 𝛼

for m ∈ {0, … , L1 + L2 − 1}.

Scaling
We see that, due to the exponential form of pzk (𝜌k ∣ j, i) in Ck (j ∣ i), in a few iterations, the forward and
backward metrics may assume very small values; this leads to numerical problems in the computation of
the metrics: therefore, we need to substitute equations (7.399) and (7.406) with analogous expressions
that are scaled by a suitable coefficient.
We note that the state metric (7.410) does not change if we multiply Fk (i) and Bk (i), i = 1, … , Ns , by
the same coefficient k . The idea [15] is to choose

1
k = ∑N (7.417)
s
n=1
Fk (n)

Indicating with F k (i) and Bk (i) the normalized metrics, for

∑
Ns
Fk (j) = Ck (j ∣ 𝓁) F k−1 (𝓁) (7.418)
𝓁=1

Equation (7.399) becomes

∑
Ns
Ck (j ∣ 𝓁) F k−1 (𝓁)
𝓁=1 j = 1, … , Ns
F k (j) = , (7.419)
∑ ∑ Ns Ns k = 0, 1, … , K − 1
Ck (n ∣ 𝓁) F k−1 (𝓁)
n=1 𝓁=1
352 Chapter 7. Single-carrier modulation

Correspondingly, (7.406) becomes

∑
Ns
Bk+1 (m) Ck+1 (m ∣ i)
m=1 i = 1, … , Ns
Bk (i) = , (7.420)
∑ ∑
Ns Ns k = K − 1, K − 2, … , 0
Ck (n ∣ 𝓁) F k−1 (𝓁)
n=1 𝓁=1

Hence,
F (i) Bk (i) i = 1, … , Ns
Vk (i) = ∑N k , (7.421)
s
F k (n) Bk (n) k = 0, 1, … , K − 1
n=1

The log likelihood function and the Max-Log-MAP criterion

We introduce the Log-MAP criterion, which employs the logarithm of the variables Fk (i), Bk (i), Ck (j ∣ i),
Vk (i), and 𝙻ak+L (𝛼). The logarithmic variables are indicated with the corresponding lower-case letters;
1
in particular, from (7.410), we have
𝑣k (i) = ln Vk (i) (7.422)
and (7.415) becomes
⎛ ⎞
⎜ ⎟
⎜ ∑ Ns ⎟
𝓁ak+L (𝛼) = ln 𝙻ak+L (𝛼) = ln ⎜ e𝑣k (i) ⎟ , 𝛼∈ (7.423)
1 1 ⎜ ⎟
⎜ i=1 ⎟
⎜ [𝝈 ] = 𝛼 ⎟
⎝ i0 ⎠
The function 𝓁k (𝛼) is called log-likelihood (LL).
The exponential emphasize the difference between the metrics 𝑣k (i): typically a term dominates within
each sum; this suggests the approximation
∑
Ns
ln e𝑣k (i) ≃ max 𝑣k (i) (7.424)
i∈{1,…,Ns }
i=1
Consequently, (7.423) is approximated as
𝓁̃ak+L (𝛼) = max 𝑣k (i) (7.425)
1
i ∈ {1, … , Ns }
[𝝈 i ]0 = 𝛼
and the Log-MAP criterion is replaced by the Max-Log–MAP criterion
â k+L1 = arg max 𝓁̃ak+L (𝛼) (7.426)
𝛼∈ 1

Apart from non-essential constants, the Max-Log–MAP algorithm in the case of transmission of i.i.d.
symbols over a dispersive channel with additive white Gaussian noise is formulated as follows:
1. Computation of channel transition metrics: For k = 0, 1, … , K − 1,
ck (j ∣ i) = −|𝜌k − uk |2 , i, j = 1, … , Ns (7.427)
where uk = f (𝝈 j , 𝝈 i ), assuming there is a transition between sk−1 = 𝝈 i and sk = 𝝈 j . For k = K, we
let
cK (j ∣ i) = 0 i, j = 1, … , Ns (7.428)
again, assuming there is a transition between 𝝈 i and 𝝈 j .
7.5. Optimum methods for data detection 353

2. Backward procedure: For k = K − 1, K − 2, … , 0,

b̃ k (i) = max [b̃ k+1 (m) + ck+1 (m ∣ i)], i = 1, … , Ns (7.429)
m∈{1,…,Ns }

If the final state is known, then

{
0, 𝝈 i = 𝝈 f0
b̃ K (i) = (7.430)
−∞, otherwise
If the final state is unknown, we set b̃ K (i) = 0, i = 1, … , Ns .
3. Forward procedure: For k = 0, 1, … , K − 1,
f̃k (j) = max [f̃k−1 (𝓁) + ck (j ∣ 𝓁)], j = 1, … , Ns (7.431)
𝓁∈{1,…,Ns }

If the initial state is known, then

{
0, 𝝈 j = 𝝈 i0
f̃−1 (j) = (7.432)
−∞, otherwise
If the initial state is unknown, we set f̃−1 (j) = 0, j = 1, … , Ns .
4. State metric: For k = 0, 1, … , K − 1,
𝑣̃k (i) = f̃k (i) + b̃ k (i), i = 1, … , Ns (7.433)
5. LL function of an isolated symbol: For k = 0, 1, … , K − 1, the LL function is given by (7.425),
with 𝑣̃k (i) in substitution of 𝑣k (i); the decision rule is given by (7.427).
In practical implementations of the algorithm, steps 3–5 can be carried out in sequence for each value
of k: this saves memory locations. To avoid overflow, for each value of k, a common value can be added
to all variables f̃k (i) and b̃ k (i), i = 1, … , Ns .
We observe that the two procedures, backward and forward, can be efficiently implemented by a VA
trellis diagram run both in backward and forward directions. The simplified MAP algorithm requires
about twice the complexity of the VA implementing the MLSD criterion. Memory requirements are
considerably increased with respect to the VA, because the backward metrics must be stored before
evaluating the state metrics. However, methods for an efficient use of the memory are proposed in [17].
The most common approach is splitting the very long receiver sequence into shorter subsequences over
which the algorithm is run. If Kd is the transient length, now at the beginning and at the end of each
detected subsequence, the various receiver subsequences must overlap by 2Kd samples.

LLRs associated to bits of BMAP

In the multilevel case, we recall from (6.6) and (6.7) that each value of ak is identified by 2𝚖 = log2 M
bits by
BMAP
ak ←−→ (bk2𝚖 , … , bk2𝚖−2𝚖+1 ) (7.434)
or by
BMAP
(ak,I ; ak,Q ) ←−→ (bk𝚖,I , … , bk𝚖−𝚖+1,I ; bk𝚖,Q , … , bk𝚖−𝚖+1,Q ) (7.435)
If we denote the generic state value by a bit pattern 𝝈 bit
i
rather than a symbol pattern 𝝈 i , with regards to
(7.434) the log-likelihood-ratio (LLR) associated with bits of bit mapper (BMAP) b(k+L1 )2𝚖−m is given by

𝓁̃b(k+L = max 𝑣k (i) − max 𝑣k (i), m = 0, 1, … , 2𝚖 − 1 (7.436)

1 )2𝚖−m
i ∈ {1, … , Ns } i ∈ {1, … , Ns }
[𝝈 bit
i m
] =1 [𝝈 bit
i m
] =0
354 Chapter 7. Single-carrier modulation

One word of caution: on labelling a symbol value each bit assumes {0, 1} values. However, LLRs refer
to {−1, 1}, where the original 0 has become −1. Moreover, 𝑣k (i) in (7.436), as given by (7.422), could be
replaced by its approximation 𝑣̃k (i) as given by (7.433). Note that (7.436), when {b𝓁 } is replaced by {cm },
i.e. in the presence of coding, extends expressions of LLRs of Section 6.2.2 to the more general case
of a channel with ISI. Indeed, their computational complexity is much higher, due to the computation
of {𝑣k (i)}.

Relation between Max-Log–MAP and Log–MAP

We define the following function of two variables [17]
max∗ (x, y) = ln(ex + ey ) (7.437)
it can be verified that the following relation holds:
max∗ (x, y) = max(x, y) + ln(1 + e−|x−y| ) (7.438)
We now extend the above definition to the case of three variables,
max∗ (x, y, z) = ln(ex + ey + ez ) ; (7.439)
then we have
max∗ (x, y, z) = max∗ (max∗ (x, y), z) (7.440)
The extension to more variables is readily obtained by induction. So, if in the backward and forward
procedures of page (7.429), we substitute the max function with the max∗ function, we obtain the
exact Log–MAP formulation that relates 𝑣k (i) = ln Vk (i) to bk (i) = ln Bk (i) and fk (i) = ln Fk (i), using
the branch metric ck (j ∣ i).

7.5.3 Optimum receivers

Definition 7.2
Given a desired signal corrupted by noise, in general the notion of sufficient statistics applies to any
signal, or sequence of samples, that allows the optimum detection of the desired signal. In other words,
no information is lost in considering a set of sufficient statistics instead of the received signal. ◽

It is possible to identify two different receiver structures that supply the signal zk given by (7.336).
1. The first, illustrated in Figure 7.44a, is considered for the low implementation complexity; it refers
to the receiver of Figure 7.11, where
√ ( )
f
Rc (f ) = rcos ,𝜌 (7.441)
1∕T
and 𝑤C is white Gaussian noise with PSD N0 . Recalling that rR (t) = sR (t) + 𝑤R , with
( )
f
𝑤R (f ) = 𝑤 (f ) |Rc (f )|2 = N0 r cos ,𝜌 (7.442)
1∕T
it is immediate to verify that the noise sequence {𝑤k = 𝑤R (t0 + kT)} has a constant PSD equal
to N0 , and the variance of the noise samples is 𝜎𝑤2 = N0 ∕T.
Although the filter defined by (7.441) does not necessarily yield a sufficient statistic (see Obser-
vation 7.14 on page 357), it considerably reduces the noise and this may be useful in estimating
the channel impulse response. Another problem concerns the optimum timing phase, which may
be difficult to determine for non-minimum phase channels.
7.5. Optimum methods for data detection 355

f ,ρ
RC ( f ) = rcos
1/T t0 + kT
ak sC (t ) rC (t ) rR (t ) zk
qC gRc
T T
wC (t )
(AWGN)
(a)

gM (t ) = q*C(t0 –t)
t0 + kT WF
ak sC (t) rC (t ) x(t) xk zk
qC gM W
T T T
wC (t)
(AWGN)
(b)

Figure 7.44 Two receiver structures with i.i.d. noise samples at the decision point.

2. An alternative, also known as the Forney receiver, is represented in Figure 7.35 and repeated
in Figure 7.44b (see also Section 13.2). To construct the WF, however, it is necessary to deter-
mine poles and zeros of the function Φ(z); this can be rather complicated in real-time applications.
A practical method is based on Observation 7.11 of page 327. From the knowledge of the autocor-
relation sequence of the channel impulse response, the prediction error filter A(z) is determined.
The WF of Figure 7.44 is given by W(z) = 𝚏12 A∗ (1∕z∗ ); therefore, it is an FIR filter. The impulse
0

response {𝜂n } is given by the inverse z-transform of 𝚏1 F(z) = 1∕A(z). For the realization of symbol
0
detection algorithms, a windowed version of the impulse response is considered.
A further method, which usually requires a filter 𝚠 with a smaller number of coefficients than
the previous method, is based on the observation that, for channels with low noise level, the DFE
solution determined by the MSE method coincides with the DFE-ZF solution; in this case, the FF
filter plays the role of the filter 𝚠 in Figure 7.44b. We consider two cases.
a. At the output of the MF gM , let {hn } be the system impulse response with sampling period T,
determined, for example through the method described in Observation 7.6 on page 320. Using
(7.194), a DFE is designed with filter parameters (M1 , M2 ), and consequently 𝚠(t) = copt (t)
in Figure 7.44b. At the output of the filter 𝚠, the ideally minimum phase impulse response {𝜂n }
corresponds to the translated, by D sampling intervals, and windowed, with L1 = 0 and L2 = M2 ,
version of 𝜓(t) = h ∗ copt (t).
b. If the impulse response of the system is unknown, we can resort to the FS-DFE structure
of Figure 7.32 with
{
1, n=0
𝜂n = 𝜓n+D ≃ (7.443)
−bn , n = 1, … , M2

Actually, unless the length of {𝜂n } is shorter than that of the impulse response qC , it is convenient
to use Ungerboeck’s formulation of the MLSD [18] that utilizes only samples {xk } at the MF output; now,
however, the metric is no longer Euclidean. The derivation of the non-Euclidean metric is the subject
of the next section.
356 Chapter 7. Single-carrier modulation

We note that, as it is not immediate to obtain the likelihood of an isolated symbol from
the non-Euclidean metric, there are cases in which this method is not adequate. We refer in par-
ticular to the case in which decoding with soft input is performed separately from symbol detection
in the presence of ISI (see Section 11.3.2). However, joint decoding and detection is always possible
using a suitable trellis (see Section 11.3.2).

7.5.4 The Ungerboeck’s formulation of MLSD

We refer to the transmission of K symbols and to an observation interval TK = KT sufficiently large, so
that the transient of filters at the beginning and at the end of the transmission has a negligible effect.
The derivation of the likelihood is based on the received signal (7.42),

rC (t) = sC (t) + 𝑤C (t) (7.444)

where 𝑤C is white noise with PSD N0 , and

∑
K−1
sC (t) = ak qC (t − kT) (7.445)
k=0

For a suitable basis, we consider for (7.444) the following vector representation:

r=s+𝒘

Assuming that the transmitted symbol sequence a = [a0 , a1 , … , aK−1 ] is equal to 𝜶 = [𝛼0 , 𝛼1 , … , 𝛼K−1 ],
the probability density function of r is given by
( )
1
pr∣a (𝝆 ∣ 𝜶) = K exp − ‖𝝆 − s‖2
(7.446)
N0
Using the equivalence between a signal x and its vector representation x,
+∞
Ex = |x(t)|2 dt = ‖x‖2 = Ex (7.447)
∫−∞
and observing rC (t) = 𝜌(t), we get
( )
1
pr∣a (𝝆 ∣ 𝜶) = K exp − |𝜌(t) − sC (t)|2 dt (7.448)
N0 ∫TK
Taking the logarithm in (7.448), the log-likelihood (to be maximized) is
| ∑
K−1 |2
| |
𝓁(𝜶) = − ||𝜌(t) − 𝛼k qC (t − kT)|| dt (7.449)
∫TK | |
| k=0 |
Correspondingly, the detected sequence is given by

â = arg max 𝓁(𝜶) (7.450)

𝜶

Expanding the square term in (7.449), we obtain

{ [ ]
∑
K−1
𝓁(𝜶) = − |𝜌(t)| dt − 2Re
2
𝜌(t) 𝛼k∗ q∗C (t − kT) dt
∫TK ∫TK k=0
}
∑ ∑
K−1 K−1
+ 𝛼k1 𝛼k∗ qC (t − k1 T) q∗C (t − k2 T) dt (7.451)
∫TK k 2
1 =0 k2 =0
7.5. Optimum methods for data detection 357

We now introduce the MF

gM (t) = q∗C (t0 − t) (7.452)

and the overall impulse response at the MF output

q(t) = (qC ∗ gM )(t) = 𝚛qC (t − t0 ) (7.453)

where 𝚛qC is the autocorrelation of qC , whose samples are given by (see (7.130))

hn = q(t0 + nT) = 𝚛qC (nT) (7.454)

The MF output signal is expressed as

x(t) = (rC ∗ gM )(t) (7.455)

with samples given by

xk = x(t0 + kT) (7.456)

In (7.451), the first term can be neglected since it does not depend on 𝜶, while the other two terms are
rewritten in the following form:
{ [K−1 ] K−1 K−1 }
∑ ∑∑
𝓁(𝜶) = − −2Re 𝛼k xk +
∗
𝛼k1 𝛼k hk2 −k1
∗
(7.457)
2
k=0 k1 =0 k2 =0

Observing (7.457), we obtain the following important result.

Observation 7.14
The sequence of samples {xk }, taken by sampling the MF output signal with sampling period equal to
the symbol period T, forms a sufficient statistic to detect the message {ak } associated with the signal rC
defined in (7.444).

We express the double summation in (7.457) as the sum of three terms, the first for k1 = k2 , the second
for k1 < k2 , and the third for k1 > k2 :
∑∑
K−1 K−1
A= 𝛼k1 𝛼k∗ hk2 −k1
2
k1 =0 k2 =0

∑
K−1
∑∑
K−1 k1 −1
∑∑
K−1 k2 −1
= 𝛼k1 𝛼k∗ h0 + 𝛼k1 𝛼k∗ hk2 −k1 + 𝛼k1 𝛼k∗ hk2 −k1 (7.458)
1 2 2
k1 =0 k1 =1 k2 =0 k2 =1 k1 =0

Because the sequence {hn } is an autocorrelation, it enjoys the Hermitian property, i.e. h−n = h∗n ; conse-
quently, the third term in (7.458) is the complex conjugate of the second, and
[K−1 k−1 ]
∑
K−1
∑∑
A= 𝛼k 𝛼k h0 + 2Re
∗
𝛼k 𝛼k2 hk−k2
∗
(7.459)
k=0 k=1 k2 =0

By the change of indices n = k − k2 , assuming 𝛼k = 0 for k < 0, we get

{K−1 [K−1 k ]}
∑ ∑∑
A = Re 𝛼k 𝛼k h0 + 2
∗
𝛼k 𝛼k−n hn
∗

k=0 k=1 n=1

{K−1 [ ]}
∑ ∑
k
= Re 𝛼k∗ h0 𝛼k + 2 hn 𝛼k−n (7.460)
k=0 n=1
358 Chapter 7. Single-carrier modulation

In particular, if
|hn | ≃ 0 for |n| > Nh (7.461)
(7.460) is simplified in the following expression
{K−1 [ ]}
∑ ∑
Nh
A = Re 𝛼k h0 𝛼k + 2
∗
hn 𝛼k−n (7.462)
k=0 n=1

Then the log-likelihood (7.457) becomes

{ [ ]}
∑
K−1
∑
Nh
𝓁(𝜶) = − Re 𝛼k −2xk + h0 𝛼k + 2
∗
hn 𝛼k−n (7.463)
k=0 n=1

To maximize 𝓁(𝜶) or, equivalently, to minimize −𝓁(𝜶) with respect to 𝜶 we apply the VA (see page 343)
with the state vector defined as
sk = (ak , ak−1 , … , ak−Nh +1 ) (7.464)
and branch metric given by
{ [ ]}
∑
Nh
Re a∗k −2xk + h0 ak + 2 hn ak−n (7.465)
n=1

Rather than detecting the data sequences by (7.450), the extraction of LLRs associated to bits of BMAP
follows the same lines of (7.436).

7.5.5 Error probability achieved by MLSD

In the VA, we have an error if a wrong sequence of states is chosen in the trellis diagram; it is of interest
the probability that one or more states of the detected ML sequence are in error. The error probability
is dominated by the probability that a sequence at the minimum Euclidean distance from the correct
sequence is chosen as ML sequence. We note, however, that by increasing the sequence length K the
number of different paths in the trellis diagram associated with sequences that are at the minimum
distance also increases. Therefore, by increasing K, the probability that the chosen sequence is in error
usually tends to 1.
The probability that the whole sequence of states is not received correctly is rarely of interest; instead,
we consider the probability that the detection of a generic symbol is in error. For the purpose of deter-
mining the symbol error probability, the concept of error event is introduced. Let {𝝈} = (𝝈 i0 , … , 𝝈 iK−1 )
be the realization of the state sequence associated with the information sequence, and let {𝝈} ̂ be the
sequence chosen by the VA. In a sufficiently long time interval, the paths in the trellis diagram associ-
ated with {𝝈} and {𝝈} ̂ diverge and converge several times: every distinct separation from the correct
path is called an error event.

Definition 7.3
An error event e is defined as a path in the trellis diagram that has only the initial and final states in
common with the correct path; the length of an error event is equal to the number of nodes visited in the
trellis before rejoining with the correct path. ◽

Error events of length two and three are illustrated in a trellis diagram with two states, where the
correct path is represented by a continuous line, in Figure 7.45a and Figure 7.45b, respectively.
Let 𝙴 be the set of all error events beginning at instant i. Each element e of 𝙴 is characterized by a
̂ which diverges from {𝝈} at instant i and converges at {𝝈} after
correct path {𝝈} and a wrong path {𝝈},
7.5. Optimum methods for data detection 359

k k +1 k +2 k k +1 k +2 k +3

(a) (b)

Figure 7.45 Error events of length (a) two and (b) three in a trellis diagram with two states.

a certain number of steps in the trellis diagram. We assume that the probability P[e] is independent of
instant i: this hypothesis is verified with good approximation if the trellis diagram is much longer than
the length of the significant error events. An error event produces one or more errors in the detection of
symbols of the input sequence. We have a detection error at instant k if the detection of the input at the
k-th stage of the trellis diagram is not correct. We define the function [14]
{
1, if e causes a detection error at the instant i + m , with m ≥ 0
cm (e) = (7.466)
0, otherwise
The probability of a particular error event that starts at instant i and causes a detection error at instant k
is given by ck−i (e)P[e]. Because the error events in 𝙴 are disjoint, we have
∑
k
∑
Pe = P[̂ak ≠ ak ] = ck−i (e) P[e] (7.467)
i=−∞ e∈𝙴

Assuming that the two summations can be exchanged, we obtain

∑ ∑
k
Pe = P[e] ck−i (e) (7.468)
e∈𝙴 i=−∞

With a change of variables, it turns out

∑
k
∑
∞
ck−i (e) = cm (e) = N(e) (7.469)
i=−∞ m=0

which indicates the total number of detection errors caused by the error event e. Therefore,
∑
Pe = N(e)P[e] (7.470)
e∈𝙴

where the dependence on the time index k vanishes. We therefore have that the detection error probability
is equal to the average number of errors caused by all the possible error events initiating at a given instant
i; this result is expected, because the detection error probability at a particular instant k must take into
consideration all error events that initiate at previous instants and are not yet terminated.
If {s} = (s0 , … , sK−1 ) denotes the random variable sequence of states at the transmitter and
{̂s} = (̂s0 , … , ŝ K−1 ) denotes the random variable sequence of states selected by the ML receiver, the
probability of an error event e beginning at a given instant i depends on the joint probability of the
correct and incorrect path, and it can be written as
̂ ∣ {s} = {𝝈}]P[{s} = {𝝈}]
P[e] = P[{̂s} = {𝝈} (7.471)
̂ ∣ {s} = {𝝈}], we resort to
Because it is usually difficult to find the exact expression for P[{̂s} = {𝝈}
upper and lower bounds.
360 Chapter 7. Single-carrier modulation

Upper bound. Because detection of the sequence of states {s} is obtained by observing the sequence
{u}, for the signal in (7.334) with zero mean additive white Gaussian noise having variance 𝜎I2 per
dimension, we have the upper bound
( )
̂
d[u({𝝈}), u({𝝈})]
̂ ∣ {s} = {𝝈}] ≤ Q
P[{̂s} = {𝝈} (7.472)
2𝜎I
̂ is the Euclidean distance between signals u({𝝈}) and u({𝝈}),
where d[u({𝝈}), u({𝝈})] ̂ given by (7.354).
Substitution of the upper bound in (7.470) yields
( )
∑ ̂
d[u({𝝈}), u({𝝈})]
Pe ≤ N(e) P[{s} = {𝝈}]Q (7.473)
e∈𝙴
2𝜎I
which can be rewritten as follows, by giving prominence to the more significant terms,
( )
∑ Δmin
Pe ≤ N(e) P[{s} = {𝝈}]Q + other terms (7.474)
e∈𝙴
2𝜎I
min

where 𝙴min is the set of error events at minimum distance Δmin defined in (7.355), and the remaining
terms are characterized by arguments of the Q function larger than Δmin ∕(2𝜎I ). For higher values of the
signal-to-noise ratio, these terms are negligible and the following approximation holds
( )
Δmin
P e ≤ 1 Q (7.475)
2𝜎I
where ∑
1 = N(e) P[{s} = {𝝈}] (7.476)
e∈𝙴min

Lower bound. A lower bound to the error probability is obtained by considering the probability that
any error event may occur rather than the probability of a particular error event. Since N(e) ≥ 1 for all
the error events e, from (7.470) we have
∑
Pe ≥ P[e] (7.477)
e∈𝙴

Let us consider a particular path in the trellis diagram determined by the sequence of states {𝝈}. We set

̃
Δmin ({𝝈}) = min d[u({𝝈}), u({𝝈})] (7.478)
̃
{𝝈}

i.e. for this path, Δmin ({𝝈}) is the Euclidean distance of the minimum distance error event. We have
Δmin ({𝝈}) ≥ Δmin , where Δmin is the minimum distance obtained considering all the possible state
sequences. If {𝝈} is the correct state sequence, the probability of an error event is lower bounded by
( )
Δmin ({𝝈})
P[e ∣ {s} = {𝝈}] ≥ Q (7.479)
2𝜎I
Consequently, ( )
∑ Δmin ({𝝈})
Pe ≥ P[{s} = {𝝈}]Q (7.480)
{𝝈}
2𝜎I

If some terms are omitted in the summation, the lower bound is still valid, because the terms are
non-negative. Therefore, taking into consideration only those state sequences {𝝈} for which Δmin ({𝝈}) =
Δmin , we obtain ( )
∑ Δmin
Pe ≥ P[{s} = {𝝈}]Q (7.481)
{𝝈}∈A
2𝜎I
7.5. Optimum methods for data detection 361

where A is the set of state sequences that admit an error event with minimum distance Δmin , for an
arbitrarily chosen initial instant of the given error event. Defining
∑
2 = P[{s} = {𝝈}] (7.482)
{𝝈}∈A

as the probability that a path {𝝈} admits an error event with minimum distance, it is
( )
Δmin
P e ≥ 2 Q (7.483)
2𝜎I
Combining upper and lower bounds, we obtain
( ) ( )
Δmin Δmin
2 Q ≤ P e ≤ 1 Q (7.484)
2𝜎I 2𝜎I
For large values of the signal-to-noise ratio, therefore, we have
( )
Δmin
Pe ≃  Q (7.485)
2𝜎I
for some value of the constant  between 1 and 2 .
We stress that the error probability, expressed by (7.485) and (7.356), is determined by the ratio
between the minimum distance Δmin and the standard deviation of the noise 𝜎I .
Here the expressions of the constants 1 and 2 are obtained by resorting to various approximations.
An accurate method to calculate upper and lower bounds of the error probability is proposed in [19].

Computation of the minimum distance

The application of the VA to MLSD in transmission systems with ISI requires that the overall impulse
response is FIR, otherwise the number of states, and hence also the complexity of the detector, becomes
infinite. From (7.334), the samples at the detector input, conditioned on the event that the sequence of
symbols {ak } is transmitted, are statistically independent Gaussian random variables with mean
∑
L2
𝜂n ak−n (7.486)
n=−L1

and variance 𝜎I2 per dimension. The metric that the VA attributes to the sequence of states corresponding
to the sequence of input symbols {ak } is given by the square Euclidean distance between the sequence
of samples {zk } at the detector input and its mean value, which is known given the sequence of symbols
(see (6.24)),
∞ | |2
∑ | ∑L2
|
|z − 𝜂 a | (7.487)
|k n k−n |
|
k=0 | |
n=−L1 |
In the previous section, it was demonstrated that the symbol error probability is given by (7.485). In
particularly simple cases, the minimum distance can be determined by direct inspection of the trellis
diagram; in practice, however, this situation is rarely verified in channels with ISI. To evaluate the min-
imum distance, it is necessary to resort to simulations. To find the minimum distance error event with
initial instant k = 0, we consider the desired signal uk under the condition that the sequence {ak } is
transmitted, and we compute the square Euclidean distance between this signal and the signal obtained
for another sequence {̃ak },
∞ | L2 |2
∑ |∑ ∑L2
|
2
d [u({ak }), u({̃ak })] = | 𝜂n ak−n − 𝜂n ã k−n || (7.488)
|
k=0 ||n=−L1 n=−L1 |
|
where it is assumed that the two paths identifying the state sequences are identical for k < 0.
362 Chapter 7. Single-carrier modulation

It is possible to avoid computing the minimum distance for each sequence {ak } if we exploit the
linearity of the ISI. Defining
𝜖k = ak − ã k (7.489)

we have
∞ | L2 |2
∑ |∑ |
2 2
d ({𝜖k }) = d [u({ak }), u({̃ak })] = | 𝜂n 𝜖k−n || (7.490)
|
k=0 ||n=−L1 |
|
The minimum among the square Euclidean distances relative to all error events that initiate at k = 0 is

Δ2min = min d2 ({𝜖k }) (7.491)

{𝜖k }∶ 𝜖k =0, k<L1 ,𝜖L1 ≠0

It is convenient to solve this minimization without referring to the symbol sequences. In particular, we
define the state sk = (𝜖k+L1 , 𝜖k+L1 −1 , … , 𝜖k−L2 +1 ), and a trellis diagram that describes the development of
this state. Adopting the branch metric
|∑ L |2
| 2 |
| 𝜂n 𝜖k−n || (7.492)
|
|n=−L1 |
| |
the minimization problem is equivalent to determining the path in a trellis diagram that has minimum
metric (7.490) and differs from the path that joins states corresponding to correct decisions: the resulting
metric is Δ2min . We note, however, that the cardinality of 𝜖k is larger than M, and this implies that the
complexity of this trellis diagram can be much larger than that of the original trellis diagram. In the
PAM case, the cardinality of 𝜖k is equal to 2M − 1.
In practice, as the terms of the series in (7.490) are non-negative, if we truncate the series after a finite
number of terms, we obtain a result that is smaller than or equal to the effective value. Therefore, a lower
bound to the minimum distance is given by
K−1 | L2 |2
∑| ∑ |
Δ2min ≥ min | 𝜂n 𝜖k−n || (7.493)
|
k=0 ||n=−L1 |
{𝜖k }∶ 𝜖k =0, k<L1 ,𝜖L1 ≠0
|

wk
BMAP
bk 0 → −1 ak uk zk a^k
η (D) MLSD
1 → +1 T

Figure 7.46 PR-IV (modified duobinary) transmission system.

Example 7.5.3
We consider the partial response system class IV (PR-IV), also known as modified duobinary (see
Appendix 7.12), illustrated in Figure 7.46. The transfer function of the discrete-time overall system is
given by 𝜂(D) = 1 − D2 . For an ideal noiseless system, the input sequence {uk } to the detector is formed
by random variables taking values in the set {−2, 0, +2}, as shown in Figure 7.47.
Assuming that the sequence of noise samples {𝑤k } is composed of real-valued, statistically indepen-
dent, Gaussian random variables with mean zero and variance 𝜎I2 , and observing that uk for k even (odd)
depends only on symbols with even (odd) indices, the MLSD receiver for a PR-IV system is usually
implemented by considering two interlaced dicode independent channels, each having a transfer func-
tion given by 1 − D. As seen in Figure 7.48, for detection of the two interlaced input symbol sequences,
7.5. Optimum methods for data detection 363

bk ... 0 1 1 0 1 0 0 ...

ak ... −1 +1 +1 −1 +1 −1 −1 ...

uk = ak – ak−2 ... +2 −2 0 0 −2 ...

Figure 7.47 Input and output sequences for an ideal PR-IV system.
ak 2 bk uk bk 2 uk 2 bk 4 uk 4 bk 6 uk 6

1 0 1 0 1 0 1 0
+1
+2 +2 +2 +2
0 0 0 0
1 –2 1 –2 1 –2 1 –2
–1
0 0 0 0 0 0 0 0

ak 1 bk 1 uk 1 bk 3 uk 3 bk 5 uk 5 bk 7 uk 7

1 0 1 0 1 0 1 0
+1
+2 +2 +2 +2
0 0 0 0
1 –2 1 –2 1 –2 1 –2
–1
0 0 0 0 0 0 0 0

Figure 7.48 Trellis diagrams for detection of interlaced sequences.

two trellis diagrams are used. The state at instant k is given by the symbol sk = ak , where k is even-valued
in one of the two diagrams and odd-valued in the other. Each branch of the diagram is marked with a
label that represents the binary input symbol bk or, equivalently, the value of the dicode signal uk . For a
particular realization of the output signal of a dicode channel, the two survivor sequences at successive
iterations of the VA are represented in Figure 7.49.
It is seen that the minimum square Euclidean distance between two separate paths in the trellis diagram
is given by Δ2min = 22 + 22 = 8. However, we note that for the same initial instant, there are an infinite
number of error events with minimum distance from the effective path: this fact is evident in the trellis
diagram of Figure 7.50a, where the state sk = (𝜖k ) ∈ {−2, 0, +2} characterizes the development of the
error event and the labels indicate the branch metrics associated with an error event. It is seen that at
every instant an error event may be extended along a path, for which the metric is equal to zero, parallel
to the path corresponding to the zero sequence. Paths of this type correspond to a sequence of errors
having the same polarity. Four error events
( √ with
) minimum distance are shown in Figure 7.50b.
The error probability is given by Q 2𝜎8 , where 2 ≤  ≤ 1 . The constant 2 can be immedi-
I
ately determined by noting that every effective path admits at every instant at least one error event with
minimum distance: consequently, 2 = 1. To find 1 , we consider the contribution of an error event
with m consecutive errors. For this event to occur, it is required that m consecutive input symbols have
the same polarity, which happens with probability 2−m . Since such an error event determines m symbol
errors and two error events with identical characteristics can be identified in the trellis diagram, we have
∑
∞
1 = 2 m 2−m = 4 (7.494)
m=1
364 Chapter 7. Single-carrier modulation

k= 0 1 2 3 4 5

σ1 =1 Γ (s1 = σ1)= Γ(s0 = σ2) + (u1 –2)2

σ2 =−1 Γ (s1 = σ2) = Γ(s0 = σ2) + (u1)2

b0 = 0

b0 = 0 b1 = 0

b0 = 0 b1 = 0

b2 = 1 b3 = 1

b0 = 0 b1 = 0

b2 = 1 b3 = 1

b0 = 0 b1 = 0

Figure 7.49 Survivor sequences at successive iterations of the VA for a dicode channel.

sk
0 0
–2
4 4
4 4 4
16 16

16 16 4 4 4
4 4
+2
0 0

(a) (b)

Figure 7.50 Examples of (a) trellis diagram to compute the minimum distance for a dicode channel
and (b) four error events with minimum distance.
7.5. Optimum methods for data detection 365

√
Besides the error events associated with the minimum distance Δmin = 8, the occurrence of long
sequences of identical output symbols from a dicode channel raises two problems.
● The system becomes catastrophic in the sense that in the trellis diagram used by the detector, valid
sequences of arbitrary length are found having square Euclidean distance from the effective path
equal to 4; therefore, a MLSD
√ receiver with finite memory will make additional errors with prob-
ability proportional to Q( 4∕(2𝜎I )) if the channel produces sequences {uk = 0} of length larger
than the memory of the detector.
● The occurrence of these sequences is detrimental for the control of receive filter gain and sampling
instants: the problem is solved by suitable coding of the input binary sequence, that sets a limit on
the number of consecutive identical symbols that are allowed at the channel input.

7.5.6 The reduced-state sequence detection

For transmissions over channels characterized by strong ISI, the maximum-likelihood detection imple-
mented by the VA typically gives better performance than the decision-feedback equalizer. On the other
hand, if we indicate with N = L1 + L2 the length of the channel memory and with M the cardinality of
the alphabet of transmitted symbols, the implementation complexity of the VA is of the order of M N ,
which can be too large for practical applications if N and/or M assume large values.
To find receiver structures that are characterized by performance similar to that of a MLSD receiver,
but lower complexity, two directions may be taken:
● decrease N;
● decrease the number of paths considered in the trellis.
The first direction leads to the application of pre-processing techniques, e.g. LE or DFE [20], to
reduce the length of the channel impulse response. However, they only partially solve the problem for
bandwidth-efficient modulation systems that utilize a large set of signals, the minimization of N typically
does not provide a solution with manageable complexity.
The second direction leads to a further study of the MLSD method. An interesting algorithm is the
-algorithm [21, 22]: it operates in the same way as the MLSD, i.e. using the full trellis diagram, but
at every step it takes into account only  ≤ M N states, that is, those associated with paths with smaller
metrics. The performance of the -algorithm is close to that of the MLSD even for small values of ,
however, it works only if the channel impulse response is minimum phase: otherwise, it is likely that
among the paths that are eliminated in the trellis diagram, the optimum path is also included.
The reduced-state sequence estimator (RSSE)13 [23, 24] yields performance very close to that of
MLSD, with significantly reduced complexity even though it retains the fundamental structure of MLSD.
It can be used for modulations with a large symbol alphabet  and/or for channels with a long impulse
response. The basic idea consists in using a trellis diagram with a reduced number of states, obtained by
combining the states of the ML trellis diagram in a manner suggested by the principle of partitioning the
set , and possibly including the decision-feedback method in the computation of the branch metrics. In
this way, the RSSE guarantees a performance/complexity trade-off that can vary from that characterizing
a DFE zero-forcing to that characterizing MLSD.

Trellis diagram
We consider the transmission system depicted in Figure 7.44b. Contrary to an MLSD receiver, the
performance of an RSSE receiver may be poor if the overall channel impulse response is not minimum
13
We will maintain the name RSSE, although the algorithm is applied to perform a detection rather than an estimation.
366 Chapter 7. Single-carrier modulation

phase: to underline this fact, we slightly change the notation adopted in Section 7.5. We indicate with
f T = [f1 , f2 , … , fN ] the coefficients of the impulse response that determine the ISI and assume, without
loss of generality, the desired sample is f0 = 1. Hence, the observed signal is
∑
N
zk = fn ak−n + 𝑤k = ak + sTk−1 f + 𝑤k (7.495)
n=0

where the state at instant k − 1 is

sTk−1 = [ak−1 , ak−2 , … , ak−N ] (7.496)

Observation 7.15
In the transmission of sequences of blocks of data, the RSSE yields its best performance by imposing
the final state, for example using the knowledge of a training sequence. Therefore, the formulation of
this section is suited for the case of a training sequence placed at the end of the data block. In the case
the training sequence is placed at the beginning of a data block, it is better to process the signals in
backward mode as described in Observation 7.7 on page 320.

The RSSE maintains the fundamental structure of MLSD unaltered, corresponding to the search in
the trellis diagram of the path with minimum cost.
To reduce the number of states to be considered, we introduce for every component ak−n , n = 1, … , N,
of the vector sk−1 defined in (7.496) a suitable partition Ω(n) of the two-dimensional set  of possible
values of ak−n : a partition is composed of Jn subsets, with Jn an integer between 1 and M. The index cn
indicates of the subset of the partition Ω(n) to which the symbol ak−n belongs, an integer value between 0

Jn = 1

0 1

Jn = 2

0 2 1 3

Jn = 4

0 4 2 6 1 5 3 7

Jn = 8

0 8 4 12 2 10 6 14 1 9 5 13 3 11 7 15

Jn = 16

Figure 7.51 Ungerboeck’s partitioning of the symbol set associated with a 16-QAM system. The
various subsets are identified by the value of cn ∈ {0, 1, … , Jn − 1}.
7.5. Optimum methods for data detection 367

and Jn − 1. Ungerboeck’s partitioning of the symbol set associated with a 16-QAM system is illustrated
in Figure 7.51. The partitions must satisfy the following two conditions [25]:
1. the numbers Jn are non-increasing, that is J1 ≥ J2 ≥ · · · ≥ JN ;
2. the partition Ω(n) is obtained by subdividing the subsets that make up the partition Ω(n + 1), for
every n between 1 and N − 1.
Therefore, we define as reduced-state at instant k − 1 the vector tk−1 that has as n-th element the index
cn of the subset of the partition Ω(n) to which the n-th element of sk−1 belongs, for n = 1, 2, … , N, that is
tTk−1 = [c1 , c2 , … , cN ], cn ∈ {0, 1, … , Jn − 1} (7.497)
and we write
sk−1 = s(tk−1 ) (7.498)
It is useful to stress that the reduced state tk−1 does not uniquely identify a state sk−1 , but all the states
sk−1 that include as n-th element one of the symbols belonging to the subset cn of partition Ω(n).
The conditions imposed on the partitions guarantee that, given a reduced state at instant k − 1, tk−1 ,
and the subset j of partition Ω(1) to which the symbol ak belongs, the reduced state at instant k, tk , can
be uniquely determined. In fact, observing (7.497), we have
tTk = [c′1 , c′2 , … , c′N ] (7.499)
c′1 c′2
where = j, is the index of the subset of the partition Ω(2) to which belongs the subset with index
c1 of the partition Ω(1), c′3 is the index of the subset of the partition Ω(3) to which belongs the subset
with index c2 of the partition Ω(2), and so forth. In this way, the reduced states tk−1 define a proper
reduced-state trellis diagram, that represents all the possible sequences {ak }.
As the symbol cn can only assume one of the integer values between 0 and Jn − 1, the total number of
possible reduced states of the trellis diagram of the RSSE is given by the product Ns = J1 J2 … JN , with
Jn ≤ M, for n = 1, 2, … , N.
We know that in the VA, for uncoded transmission of i.i.d. symbols, there are M possible transitions
from a state, one for each of the values that ak can assume. In the reduced-state trellis diagram, M
transitions are still possible from a state, however, to only J1 distinct states, thus giving origin to parallel
transitions.14 In fact, if J1 < M, J1 sets of branches depart from every state tk−1 , each set consisting of
as many parallel transitions as there are symbols belonging to the subset of Ω(1) is associated with the
reduced state.
Therefore, partitions must be obtained such that two effects are guaranteed: (i) minimum performance
degradation with respect to MLSD, and (ii) easy search of the optimum path among the various parallel
transitions. Usually, the Ungerboeck’s set partitioning method is adopted, which, for every partition
Ω, maximizes the minimum distance Δ among the symbols belonging to the same subset. For QAM
systems and Jn a power of 2, the maximum distance Δn relative to partition Ω(n) is obtained through a
tree diagram with binary partitioning. An example of partitioning of the symbol set associated with a
16-QAM system is illustrated in Figure 7.51.
In Figure 7.52, two examples of reduced-state trellis diagram are shown, both referring to the partition
of Figure 7.51.

The RSSE algorithm

As in MLSD, for each transition that originates from a state tk−1 , the RSSE computes the branch metric
according to the expression
|zk − ak − sTk−1 f |2 = |zk − ak − sT (tk−1 )f |2 (7.500)
14
Parallel transitions are present when two or more branches connect the same pair of states in the trellis diagram.
368 Chapter 7. Single-carrier modulation

tk–1 tk
[0, 0] [0, 0]

tk−1 tk [0, 1] [0, 1]

[0] [0]
[1, 0] [1, 0]

[1] [1]
[1, 1] [1, 1]

[2, 0] [2, 0]
[2] [2]

[2, 1] [2, 1]
[3] [3]

[3, 0] [3, 0]

[3, 1] [3, 1]
(a) N = 1, J1 = 4. (b) N = 2, J1 = 4, J2 = 2.

Figure 7.52 Reduced-state trellis diagrams.

However, whereas in MLSD, a survivor sequence may be described in terms of the sequence of states
that led to a certain state, in the RSSE, if J1 < M, it is convenient to memorize a survivor sequence as a
sequence of symbols. As a matter of fact, there is no one-to-one correspondence between state sequences
and symbol sequences. Therefore, backward tracing the optimum path in the trellis diagram in terms of
the sequence of states does not univocally establish the optimum sequence of symbols.
Moreover, if J1 < M, and therefore, there are parallel transitions in the trellis diagram, for every
branch, the RSSE selects the symbol ak in the subset of the partition Ω(1) that yields the minimum
metric.15 Thus, the RSSE already makes a decision in selecting one of the parallel transitions, using
also the past decisions memorized in the survivor sequence associated with the considered state.
At every iteration, these decisions reduce the number of possible extensions of the Ns states from Ns M
to Ns J1 . This number is further reduced to Ns by selecting, for each state tk , the path with the minimum
metric among the J1 entering paths, operation which requires Ns (J1 − 1) comparisons. As in the case
of the VA, final decisions are taken with a certain delay by tracing the history of the path with lowest
metric.
We note that if Jn = 1, n = 1, … , N, then the RSSE becomes a DFE-ZF, and if Jn = M, n = 1, … , N,
the RSSE performs full MLSD. Therefore, the choice of {Jn } determines a trade-off between perfor-
mance and computational complexity. The error probability for an RSSE is rather difficult to evaluate

15
We note that if the points of the subsets of the partition Ω(1) are on a rectangular grid, as in the example of
Figure 7.51, the value of the symbol ak that minimizes the metric is determined through simple quantization rules,
without explicitly evaluating the branch metric for every symbol of the subset. Hence, for every state, only J1 explicit
computations of the branch metric are needed.
7.5. Optimum methods for data detection 369

because of the presence of the decision-feedback mechanism. For the analysis of RSSE performance,
we refer the reader to [26].

Further simplification: DFSE

There are many applications in which the complexity of MLSD is mainly due to the length of the channel
impulse response. In these cases, a method to reduce the number of states consists of cancelling part of
the ISI by a DFE; better results are obtained by incorporating the decision-feedback mechanism in the
VA, that is using for each state a different feedback symbol sequence given by the survivor sequence.
The idea of cancelling the residual ISI based on the survivor sequence is a particular application of
a general principle that is applied when the branch metric is affected by some uncertainty that can be
eliminated or reduced by (possibly adaptive) data-aided estimation techniques. Typical examples are
the non-perfect knowledge of some channel characteristics, as the carrier phase, the timing phase, or the
impulse response: all these cases can be solved in part by a general approach called per-survivor process-
ing (PSP) [27]. It represents an effective alternative to classical methods to estimate channel parameters,
as the effects of error propagation are significantly reduced. Other interesting aspects characterizing this
class of algorithms are the following:
a. The estimator associated with the survivor sequence uses symbols that can be considered decisions
with no delay and high reliability, making the PSP a suitable approach for channels that are fast
time-varying.
b. Blind techniques, i.e. without knowledge of the training sequence, may be adopted as part of the PSP
to estimate the various parameters.
A variant of the RSSE, belonging to the class of PSP algorithms, is the decision feedback sequence
estimator (DFSE) [23] which considers as reduced state the vector s′k−1 formed simply by truncating the
ML state vector at a length N1 ≤ N

k−1 = [ak−1 , ak−2 , … , ak−N1 ]

s′T (7.501)
This is the same as considering Jn = M, for n = 1, … , N1 , and Jn = 1, for n = N1 + 1, … , N.
We discuss the main points of this algorithm. We express the received sequence {zk }, always under
the hypothesis of a minimum-phase overall impulse response with f0 = 1, as

k−1 f + sk−1 f + 𝑤k
′ ′′T ′′
zk = ak + s′T (7.502)
where f ′ , f ′′ , s′k−1 , and s′′k−1 are defined as follows:
f T = [f ′ T ∣ f ′′ T ] = [f1 , … , fN1 ∣ fN1 +1 , … , fN ] (7.503)
′T ′′ T
sTk−1 = [sk−1 ∣ sk−1 ] = [ak−1 , … , ak−N1 ∣ ak−(N1 +1) , … , ak−N ] (7.504)
The trellis diagram is built by assuming the reduced state s′k−1 . The term s′′T
k−1
f ′′ represents the residual
′′
ISI that is estimated by considering as sk−1 the symbols that are memorized in the survivor sequence
associated with each state. We write
ŝ ′′k−1 = s′′ (s′k−1 ) (7.505)
In fact, with respect to zk , it is as if we would have cancelled the term s′′T
k−1
f ′′ by a FB filter associated
with each state sk−1 . With respect to the optimum path, the feedback sequence s′′k−1 is expected to be
′

very reliable.
The branch metric of the DFSE is computed as follows:
|zk − ak − s′T
k−1 f − s (sk−1 ), f |
′ ′′T ′ ′′ 2
(7.506)
We note that the reduced state s′k−1 may be further reduced by adopting the RSSE technique (7.498).
370 Chapter 7. Single-carrier modulation

The primary difference between an MLSD receiver and the DFSE is that in the trellis diagram used
by the DFSE two paths may merge earlier, as it is sufficient that they share the more recent N1 symbols,
rather than N as in MLSD. This increases the error probability; however, in general DFSE outperforms
a classical DFE.

7.6 Numerical results obtained by simulations

Using Monte Carlo simulations (see Appendix 7.A) we give a comparison, in terms of Pbit as a function
of the signal-to-noise Γ, of the various equalization and data detection methods described in the previous
sections. We refer to the system model of Figure 7.26 with an overall impulse response {hn } having five
coefficients, as given in Table 1.4 on page 12, and additive white Gaussian noise (AWGN) 𝑤. ̃ Recalling
the definition of the signal-to-noise Γ = 𝜎a2 Eh ∕𝜎𝑤2̃ , we examine four cases.

QPSK over a minimum-phase channel

We examine the following receivers, where the delay D is optimized by applying the MSE criterion.
We anticipate that the ZF equalizer is designed by the DFT method (see Section 7.4.6). We also
introduce the abbreviation DFE-VA to indicate the method consisting of the FF filter of a DFE, followed
by MLSD implemented by the VA, with M M2 states determined by the impulse response 𝜂n = 𝜓n+D for
n = 0, … , M2 : this technique is commonly used to shorten the overall channel impulse response and
thus simplify the VA (see case (2a) on page 355).
1. ZF with N = 7 and D = 0;
2. LE with N = 7 and D = 0;
3. DFE with M1 = 7, M2 = 4, and D = 6;
4. VA with 44 = 256 states and path memory depth equal to 15, i.e. approximately three times the
length of {hn };
5. DFSE with J1 = J2 = 4, J3 = J4 = 1 (16 states);
6. DFE-VA with M1 = 7, M2 = 2, and D = 0 (VA with 4M2 = 16 states).
The performance of the various receivers is illustrated in Figure 7.53; for comparison, the performance
achieved by transmission over an ideal AWGN channel is also given.

QPSK over a non-minimum phase channel

We examine the following receivers.
1. ZF with N = 7 and D = 4;
2. LE with N = 7 and D = 4;
3. DFE with M1 = 7, M2 = 4, and D = 6;
4. VA with 44 = 256 states and path memory depth equal to 15;
5. DFSE with J1 = J2 = 4, J3 = J4 = 1 (16 states);
6. DFE-VA with M1 = 7, M2 = 2, and D = 4 (VA with 16 states).
The performance of the various receivers is illustrated in Figure 7.54. Comparing the error probability
curves shown in Figures 7.53 and 7.54, we note that VA outperforms the other methods; moreover,
performance of the VA is almost independent of the phase of the overall channel impulse response;
however, if the channel is minimum phase even a simple DFE or DFSE can give performance close to
the optimum. We also note that in these simulations, the DFE-VA gives poor performance because the
value of M2 is too small, hence, the DFE is unable to equalize the channel.
7.6. Numerical results obtained by simulations 371

10–1

10–2

10–3
Pbit

10–4

ZF
LE
DFE−VA
10–5 DFSE
DFE
VA
AWGN

10–6
6 7 8 9 10 11 12 13 14 15 16
Γ (dB)

Figure 7.53 Bit error probability, Pbit , as a function of Γ for quadrature phase-shift-keying (QPSK)
transmission over a minimum phase channel, using various equalization and data detection methods.

10–1

10–2

10–3
Pbit

10–4

ZF
LE
DFE−VA
10–5 DFSE
DFE
VA
AWGN

10–6
6 7 8 9 10 11 12 13 14 15 16
Γ (dB)

Figure 7.54 Bit error probability, Pbit , as a function of Γ for QPSK transmission over a
non-minimum phase channel, using various equalization and data detection methods.
372 Chapter 7. Single-carrier modulation

8-PSK over a minimum phase channel

We examine the following receivers.
1. DFE with M1 = 6, M2 = 4, and D = 0;
2. DFSE with J1 = J2 = 8, J3 = J4 = 1 (64 states);
3. DFE-VA with M1 = 7, M2 = 2, and D = 0 (VA with 8M2 = 64 states);
4. RSSE with J1 = 8, J2 = 4, J3 = 2, J4 = 1 (64 states).
The performance of the various receivers is illustrated in Figure 7.55.

8-PSK over a non-minimum phase channel

We examine the following receivers:
1. DFSE with J1 = J2 = 8, J3 = J4 = 1 (64 states);
2. DFE with M1 = 12, M2 = 4, and D = 11;
3. RSSE with J1 = 8, J2 = 4, J3 = 2, J4 = 1 (64 states).
In these simulations, the error probability for a DFE-VA is in the range between 0.08 and 0.2, and it
is not shown; the performance of the various receivers is illustrated in Figure 7.56.
By comparison of the results of Figures 7.55 and 7.56, we observe that the RSSE and DFSE may be
regarded as valid approximations of the VA as long as the overall channel impulse response is minimum
phase.

10–1
DFE
DFE−VA
DFSE
RSSE
AWGN
10–2

10–3
Pbit

10–4

10–5

10–6
11 12 13 14 15 16 17 18 19 20
Γ (dB)

Figure 7.55 Bit probability error, Pbit , as a function of Γ for 8-PSK transmission over a minimum
phase channel, using various equalization and data detection methods.
7.7. Precoding for dispersive channels 373

10–1
DFE
DFSE
RSSE
AWGN
10–2

10–3
Pbit

10–4

10–5

10–6
11 12 13 14 15 16 17 18 19 20
Γ (dB)

Figure 7.56 Bit probability error rate, Pbit , as a function of Γ for 8-PSK transmission over a
non-minimum phase channel, using various equalization and data detection methods.

7.7 Precoding for dispersive channels

From Section 7.4.7 (see also Section 13.2), the discrete-time equivalent channel model at the receiver
after a whitened matched filter can be written in the D-transform domain as
z(D) = a(D)𝚏̃ (D) + 𝑤(D) (7.507)
where {ai } is the data sequence, {𝚏̃ i } is causal and minimum-phase, with 𝚏̃ 0 = 1, and {𝑤i } is a white
Gaussian noise.
The convolution u(D) = a(D) 𝚏̃ (D) that determines the output sequence in the absence of noise can
be viewed as the transform of a vector by the channel matrix, that is u = Fa,̃ where a and u are vectors
whose components are given by the transmitted symbols and channel output samples, respectively. As
the canonical response 𝚏̃ (D) is causal and monic, the matrix F̃ is triangular with all elements equal to 1
on the diagonal and therefore, has determinant equal to 1; thus, it follows that the matrix F̃ identifies a
linear transformation that preserves the volume between the input and output spaces. In other words, the
channel matrix F̃ transforms a hypersphere into a hyperellipsoid having the same volume and containing
the same number of constellation points.
Coding methods for linear dispersive channels [28–34] that yield high values of coding gain can be
obtained by requiring that the channel output vectors in the absence of noise u are points of a set Λ′
with good properties in terms of Euclidean distance, for example a lattice identified by integers. From
the model of Figure 13.6c, an intuitive explanation of the objectives of coding for linear dispersive
channels is obtained by considering the signal sequences a, u, 𝒘, and z as vectors with a finite number
374 Chapter 7. Single-carrier modulation

of components. If the matrix F̃ is known at the transmitter, the input vector a can be predistorted so
that the points of the vector a correspond to points of a signal set F̃ Λ; the volumes V(Λ) in the output
−1

signal space, and V(F̃ Λ) in the input signal space are equal. In any case, the output channel vectors
−1

are observed in the presence of additive white Gaussian noise vectors 𝒘. If a detector chooses the point
of the lattice Λ′ with minimum distance from the output vector we obtain a coding gain, relative to Λ,
as in the case of an ideal AWGN channel (see Section 6.5).
Recall that to achieve capacity, it is necessary that the distribution of the transmitted signal approxi-
mates a Gaussian distribution. We mention that commonly used methods of shaping, which also mini-
mize the transmitted power, require that the points of the input constellation are uniformly distributed
within a hypersphere [31]. Coding and shaping thus occur in two Euclidean spaces related by a known
linear transformation that preserves the volume. Coding and shaping can be separately optimized, by
choosing a method for predistorting the signal set in conjunction with a coding scheme that leads to a
large coding gain for an ideal AWGN channel in the signal space where coding takes place, and a method
that leads to a large shaping gain in the signal space where shaping takes place. In the remaining part of
this chapter, we focus on precoding and coding methods to achieve large coding gains for transmission
over channels with ISI, assuming the channel impulse response is known.

7.7.1 Tomlinson–Harashima precoding

Precoding is a method of pre-equalization that may achieve also for transmission over linear dispersive
channels the objectives of coding illustrated in Section 6.5 for transmission over ideal AWGN channels.
A special case of precoding is illustrated in Appendix 7.C.
We now consider the Tomlinson–Harashima precoding (THP) scheme. Let 𝚏̃ (D) be the response of a
discrete-time equivalent channel, with ISI and additive noise, and assume that the canonical response
of the channel 𝚏̃ (D) is known at the transmitter. Let 𝚏̃ (D) = 1 + D𝚏̃ ′ (D), and furthermore, assume
that for every pair of symbols of the input constellation 𝛼i , 𝛼j ∈  the following relation holds:16
𝛼i = 𝛼j mod Λ0 , where  ⊂ Λ0 + 𝜆 is a finite set of symbols (constellation), Λ0 represents the lattice
associated with , and 𝜆 is a given offset value, possibly non-zero.
The objective of all precoding methods, with and without channel coding, consists in transmitting a
pre-equalized sequence
u(p) (D)
a(p) (D) = (7.508)
𝚏̃ (D)
so that, in the absence of noise, the channel output sequence u(D) = a(p) (D) 𝚏̃ (D) = u(p) (D) represents
the output of an ideal channel apparently ISI-free, with input sequence u(p) (D); u(p) (D) is a sequence of
symbols belonging to a set (p) ⊂ Λ0 + 𝜆. To achieve this objective with channel input signal samples
that belong to a given finite region, the cardinality of the set (p) must be larger than that of the set .
The redundancy in u(p) (D) can therefore be used to minimize the average power of the sequence a(p) (D)
given by (7.508), or to obtain other desirable characteristics of a(p) (D), for example a low peak-to-average
ratio. In the case of systems that adopt trellis coding (see Chapter 12), the channel output sequence in
the absence of noise u(D) must be a valid code sequence and can then be decoded by a decoder designed
for an ideal channel. Note that, in a system with precoding, the elements of the transmitted sequence
a(p) (D) are not in general symbols with discrete values.
The first precoding method was independently proposed for uncoded systems by Tomlinson [35] and
Harashima and Miyakawa [36] (TH precoding). Initially, TH precoding was not used in practice because
in an uncoded transmission system, the preferred method to cancel ISI employs a DFE, as it does not

16
The expression 𝛼i = 𝛼j mod Λ0 denotes that the two symbols 𝛼i and 𝛼j differ by a quantity that belongs to Λ0 .
7.7. Precoding for dispersive channels 375

require to send information on the channel impulse response to the transmitter. However, if trellis coding
is adopted, decision-feedback equalization is no longer a very attractive solution, as reliable decisions
are made available by the Viterbi decoder only with a certain delay.

discrete−time inverse
bit−mapper precoder
channel bit−mapper
b(D) b a a(D) + + a(p)(D) + u(D) + u^(D) + a^(D) b^ (D)
Σ Σ Σ Σ detector Σ a b
a A − + + + −

c(D) w(D) c^(D)

~ ~
Dfʹ(D) Dfʹ(D)

Figure 7.57 Block diagram of a system with TH precoding.

TH precoding, illustrated in Figure 7.57, uses memoryless operations at the transmitter and at the
receiver to obtain samples of both the transmitted sequence a(p) (D) and the detected sequence â (D)
within a finite region that contains . In principle, TH precoding can be applied to arbitrary symbol
sets ; however, unless it is possible to define an efficient extension of the region containing , the
advantages of TH precoding are reduced by the increase of the transmit signal power (transmit power
penalty). An efficient extension exists only if the signal space of a(p) (D) can be tiled, that is completely
covered without overlapping with translated versions of a finite region containing , given by the union
of the Voronoi regions of symbols of , and defined as R(). Figure 7.58 illustrates the efficient exten-
sion of a two-dimensional 16-QAM constellation, where ΛT denotes the sublattice of Λ0 that identifies
the efficient extension.

point of lattice
ΛT = 2L Z2

L Δ0
Δ0

Voronoi region (Λ0)

point of lattice Λ0

signal region a , L × L signal set

R = R()

Figure
√7.58 Illustration of the efficient extension of a two-dimensional 16-QAM constellation
(L = M = 4).

With reference to Figure 7.57, the precoder computes the sequence of channel input signal samples
a(p) (D) as
a(p) (D) = a(D) − p(D) + c(D) (7.509)
where the sequence
p(D) = [𝚏̃ (D) − 1] a(p) (D) = D𝚏̃ ′ (D) a(p) (D) (7.510)
376 Chapter 7. Single-carrier modulation

represents the ISI at the channel output that must be compensated at the transmitter. The elements of
the sequence c(D) are points of the sublattice ΛT used for the efficient extension of the region R() that
contains .17 The k-th element ck ∈ ΛT of the sequence c(D) is chosen so that the statistical power of
(p) (p)
the channel input sample ak is minimum; in other words, the element ck is chosen so that ak belongs
to the region R = R(), as illustrated in Figure 7.58. From (7.508), (7.509), and (7.510), the channel
output sequence in the absence of noise is given by
u(D) = a(p) (D) 𝚏̃ (D) = a(p) [1 + D 𝚏̃ ′ (D)]
= a(p) (D) + p(D) = a(D) + c(D) (7.511)
Note that from (7.511), we get the relation uk = ak mod ΛT , which is equivalent to (7.651).
The samples of the sequence a(p) (D) can be considered, with a good approximation, uniformly dis-
tributed in the region R. Assuming a constellation with M = L × L points for a QAM system, the power
of the transmitted sequence is equal to that of a complex-valued signal with both real and imaginary
parts that are uniformly distributed in [−(L∕2) Δ0 , (L∕2) Δ0 ], where Δ0 denotes the minimum distance
between points of the lattice Λ0 . Therefore, using the fact the quantization error power can be approxi-
mated as 1/12 of the square of the quantization step, it follows that
(p) L2 2
E[|ak |2 ] ≃ 2 Δ (7.512)
12 0
2
Recalling that the statistical power of a transmitted symbol in a QAM system is given by 2 L12−1 Δ20 (see
[[3], (5.197)] for Δ0 = 2), we find that the transmit power penalty in a system that applies TH precoding
is equal to Δ20 ∕12 per dimension.
From (7.507), the channel output signal is given by
z(D) = u(D) + 𝑤(D) (7.513)
In the case of TH precoding for an uncoded system, the detector yields a sequence û(D) of symbols
belonging to the constellation (p) ; from (7.511) the detected sequence â (D) of transmitted symbols is
therefore, given by the memoryless operation
â (D) = û(D) − ĉ (D) (7.514)
The k-th element ĉ k ∈ ΛT of the sequence ĉ (D) is chosen so that the symbol â k = ûk − ĉ k belongs to the
constellation . As the inverse operation of precoding is memoryless, error propagation at the receiver is
completely avoided. Moreover, as the inversion of 𝚏̃ (D) is not required at the receiver, 𝚏̃ (D) may exhibit
spectral nulls, that is it can contain factors of the form (1 ± D).

7.7.2 Flexible precoding

Two versions are introduced [32, 33].
First version [32]. In the first version, illustrated in Figure 7.59, the transmitted signal a(p) (D) can be
expressed as
a(p) (D) = a(D) + d(D) (7.515)
where d(D) is called the dither signal and is given by
d(D) = c(D) − p(D) (7.516)

17
Equation (7.509) represents the extension of (7.648) to the general case, in which the operation mod M is substituted
by the addition of the sequence c(D).
7.7. Precoding for dispersive channels 377

trellis coding & discrete−time inverse precoder

bit−mapper precoder channel inverse
(p) bit−mapper
b(D) b a a(D) + u(D) + a (D) + u(D) + Viterbi u^(D) + a^ (p)(D) + a^(D)
^
b(D)
Σ Σ Σ Σ Σ a
a A + − + + decoder − Σ b
−
~ ~
Df′(D) Df′(D)
w(D)
p(D) p^(D)
~
Df′(D)
QΛ ~ Q
m +1 Λ m~ +1
c(D) c^(D)

Figure 7.59 Block diagram of a system with flexible precoding.

where p(D) is given by (7.510) and c(D) is obtained from the quantization of p(D) with quantizer QΛm+1 ̃
.
The quantizer QΛm+1 ̃
yields the k-th element of the sequence c(D) by quantizing the sample pk to the
closest point of the lattice Λm+1
̃ , which corresponds to the (m ̃ + 1)-th level of partitioning of the signal
set; in the case of an uncoded sequence, it is Λm+1
̃ = Λ0 . Note that the dither signal can be interpreted
as the signal with minimum amplitude that must be added to the sequence a(D) to obtain a valid code
sequence at the channel output in the absence of noise. In fact, at the channel output, we get the sequence
z(D) = u(D) + 𝑤(D), where u(D) is obtained by adding a sequence of points taken from the lattice Λm+1 ̃
to the code sequence a(D), and therefore, it represents a valid code sequence.
The sequence z(D) is input to a Viterbi decoder, which yields the detected sequence û(D). To obtain a
detection of the sequence a(D), it is first necessary to detect the sequence ĉ (D) of the lattice points Λm+1
̃
added to the sequence a(D). Observing
û(D)
â (p) (D) = (7.517)
𝚏̃ (D)
and
p̂ (D) = D 𝚏̃ ′ (D) â (p) (D) (7.518)

the sequence ĉ (D) is obtained by quantizing with a quantizer QΛm+1

̃
the sequence p̂ (D), as illustrated in
Figure 7.59. Then subtracting ĉ (D) from û(D), we obtain the sequence â (D), that is used to detect the
sequence of information bits.
At this point, we can make the following observations with respect to the first version of flexible
precoding:

1. an efficient extension of the region R() is not necessary;

2. it is indispensable that the implementation of the blocks that perform similar functions in the
precoder and in the inverse precoder (see Figure 7.59) is identical with regard to the binary repre-
sentation of input and output signals;
3. as the dither signal can be assumed uniformly distributed in the Voronoi region V(Λm+1 ̃ ) of the point
of the lattice Λm+1
̃ corresponding to the origin, the transmit power penalty is equal to Δ2m+1
̃
∕12 per
dimension; this can significantly reduce the coding gain if the cardinality of the constellation  is
small;
4. to perform the inverse of the precoding operation, the inversion of the channel transfer function is
required (see (7.517)); if 𝚏̃ (D) is minimum phase, then 1∕𝚏̃ (D) is stable and the effect of an error
event at the Viterbi decoder output vanishes after a certain number of iterations; on the other hand,
if 𝚏̃ (D) has zeros on the unit circle (spectral nulls), error events at the Viterbi decoder output can
result in an unlimited propagation of errors in the detection of the sequence a(D).

Second version [33]. The second version of flexible precoding includes trellis coding with feedback.
378 Chapter 7. Single-carrier modulation

7.8 Channel estimation

Equalization, sequence detection, and precoding techniques for systems with ISI require the knowledge
of the channel impulse response. Here techniques to obtain an estimate of the channel are described.

w(k)
x(k) z(k) d(k)
h

Figure 7.60 System model (see also the front-end of Figure 7.26).

7.8.1 The correlation method

With reference to Figure 7.60, where nomenclature is from Chapter 2, and (2.232), which describes
the relation between input and output of an unknown linear system with impulse response {hi },
i = 0, 1, … , N − 1, we take as an input signal white noise with statistical power 𝚛x (0). As anticipated in
(2.235), to estimate the impulse response of such a system, we evaluate the crosscorrelation between d
and x. In fact, as 𝑤 and x are uncorrelated, we have
𝚛dx (n) = 𝚛zx (n) = 𝚛x ∗ h(n) = 𝚛x (0)hn (7.519)
In practice, instead of noise a PN sequence {p(i)}, with period L, is used as input. Moreover, the statistical
correlation is replaced by the time average. We recall that the autocorrelation of a PN sequence is given
by (see Appendix 1.C): {
= 1, n = 0, L, 2L, …
𝚛p (n) (7.520)
≃ 0, n = 1, … , L − 1, L + 1, …
As input we assume
x(k) = p(k), k = 0, 1, … , L − 1, … , L − 1 + N − 1 (7.521)
i.e. just a full period plus (N − 1) samples of {p(i)} are used. At the output, we collect L samples
d(k), k = N − 1, … , N − 1 + L − 1 (7.522)
The first (N − 1) samples of x are used to fill up the buffer of the unknown FIR filter. In fact, at time
k = N − 1, the filter transient is over. The correlation method exploits of the following estimate

1 ∑
(N−1)+(L−1)
ĥ COR,n = 𝚛̂ dx (n) = d(k) p∗ (k − n)mod L ≃ hn , n = 0, 1, … , N − 1 (7.523)
L k=(N−1)

We denote the output of the estimated system as

∑
N−1
̂
d(k) = ĥ i x(k − i) (7.524)
i=0
which is an estimate of the unknown system output z(k).
Mean and variance of the estimate (7.523) are obtained as follows. First, we replace d(k) with (2.232)
to yield
1 ∑ ∑
N−1+L−1 N−1
1 ∑
N−1+L−1
ĥ COR,n = hi p(k − i)mod L p∗ (k − n)mod L + 𝑤(k)p∗ (k − n)mod L (7.525)
L k=N−1 i=0
L k=N−1

∑
N−1
1 ∑
N−1+L−1
= hi 𝚛p (n − i)mod L + 𝑤(k)p∗ (k − n)mod L
i=0
L k=N−1
7.8. Channel estimation 379

If L ≫ 1, the second term on the right-hand side of (7.525) can be neglected, hence, observing (7.520)
we get
ĥ COR,n ≃ hn (7.526)
Then, we obtain:
1. Mean:
∑
N−1
E[ĥ COR,n ] = hi 𝚛p (n − i)mod L (7.527)
i=0

assuming 𝑤 has zero mean.

2. Variance: [ ]
1 ∑
N−1+L−1
𝜎𝑤2
var[ĥ COR,n ] = var 𝑤(k)p (k − n)mod
∗
L = (7.528)
L k=N−1
L
assuming 𝑤 white and |p(i)| = 1.
If ĥ COR,n ≃ hn , it is d(k) − d(k)
̂ ≃ 𝑤(k), the noise term. This approximation can be used to estimate
the variance of 𝑤(k), e.g. as
1 ∑
N−1+L−1
𝜎̂ 𝑤2 = ̂
|d(k) − d(k)| 2
(7.529)
L k=N−1

7.8.2 The LS method

With reference to the system of Figure 7.60, letting xT (k) = [x(k), x(k − 1), … , x(k − (N − 1))], the noisy
output of the unknown system can be written as
d(k) = hT x(k) + 𝑤(k), k = (N − 1), … , N − 1 + L − 1 (7.530)
The unknown system can be identified using the least squares (LS) method of Section 2.3 [37–39]. For
a certain estimate ĥ of the unknown system, the sum of square output errors is given by
∑
N−1+L−1
= ̂
|d(k) − d(k)| 2
(7.531)
k=N−1

where, from (7.524)

̂
d(k) = ĥ T x(k) (7.532)
As for the analysis of Section 2.3, we introduce the following quantities.
1. Energy of the desired signal:
∑
N−1+L−1
d = |d(k)|2 (7.533)
k=N−1

2. Correlation matrix of the input signal:

𝚽 = [Φ(i, n)], i, n = 0, … , N − 1 (7.534)
where
∑
N−1+L−1
Φ(i, n) = x∗ (k − i) x(k − n) (7.535)
k=N−1

3. Crosscorrelation vector:
𝝑 T = [𝜗(0), … , 𝜗(N − 1)] (7.536)
380 Chapter 7. Single-carrier modulation

where
∑
N−1+L−1
𝜗(n) = d(k) x∗ (k − n) (7.537)
k=N−1

A comparison with (7.523) shows that L1 𝜗(n) coincides with ĥ COR,n . Then the cost function (7.531)
becomes
 = d − ĥ H 𝝑 − 𝝑 H ĥ + ĥ H 𝚽 ĥ (7.538)
As the matrix 𝚽 is determined by a suitably chosen training sequence, we can assume that 𝚽 is positive
definite and therefore, the inverse exists. The solution to the LS problem yields
ĥ LS = 𝚽−1 𝝑 (7.539)
with a corresponding error energy equal to
min = d − 𝝑 H ĥ LS (7.540)
We observe that the matrix 𝚽−1 in the (7.540) can be pre-computed and memorized, because it depends
only on the known input sequence. For an estimate of the variance of the system noise 𝑤, observing
(7.531), for ĥ LS ≃ h we can assume
1
𝜎̂ 𝑤2 = min (7.541)
L
which coincides with (7.529).

Formulation using the data matrix

From the general analysis given on page 122, we recall the following definitions:
1. L × N input data matrix:
⎡ x(N − 1) … x(0) ⎤
=⎢ ⋮ ⋱ ⋮ ⎥ (7.542)
⎢ ⎥
⎣ x(N − 1 + L − 1) … x(L − 1) ⎦
2. Desired sample vector:
oT = [d(N − 1), … , d((N − 1) + (L − 1))] (7.543)
where d(k) is given by (7.530).
Observing (2.103), (2.95), and (2.124), we have
𝚽 = H  𝝑 = H o (7.544)
and
ĥ LS = ( H )−1  H o (7.545)
which coincides with (7.539).
We note the introduction of the new symbol o, in relation to an alternative linear minimium mean
square error (LMMSE) estimation method, which will be given in Section 7.8.5.

7.8.3 Signal-to-estimation error ratio

Let hT = [h0 , h1 , … , hN−1 ] be the filter coefficients and ĥ = [ĥ 0 , ĥ 1 , … , ĥ N−1 ] their estimates. Let 𝚫h
T

be the estimation error vector

𝚫h = ĥ − h
7.8. Channel estimation 381

The quality of the estimate is measured by the signal-to-estimation error ratio

‖h‖2
Λe = (7.546)
E[‖𝚫h‖2 ]
The noise present in the observed system is measured by (see Figure 7.60)
𝙼x ‖h‖2
Λ= (7.547)
𝜎𝑤2
where 𝙼x is the statistical power of the input signal. In our case, 𝙼x = 1. Lastly, we refer to the normalized
ratio
Λ 𝜎𝑤2
Λn = e = (7.548)
Λ 𝙼x E[‖𝚫h‖2 ]
̂ the output of the identified system,
We note that if we indicate with d(k)
∑
N−1
̂
d(k) = ĥ i x(k − i) (7.549)
i=0

the fact that ĥ ≠ h causes d(k)

̂ ≠ z(k), with an error given by
∑
N−1
̂
z(k) − d(k) = (hi − ĥ i ) x(k − i) (7.550)
i=0

having variance 𝙼x E[‖𝚫h‖2 ] for a white noise input. As a consequence, (7.548) measures the ratio
between the variance of the additive noise of the observed system and the variance of the error at the
output of the identified system. From (7.550), we note that the difference
̂
d(k) − d(k) ̂
= (z(k) − d(k)) + 𝑤(k) (7.551)

consists of two terms, one due to the estimation error and one due to the system noise.

Computation of the signal-to-estimation error ratio

We now evaluate the LS method performance for the estimate of h. From (7.537), (7.536) can be rewrit-
ten as
∑
N−1+L−1
𝝑= d(k) x∗ (k) (7.552)
k=N−1

Substituting (7.530) in (7.552), and letting

∑
N−1+L−1
𝝃= 𝑤(k) x∗ (k) (7.553)
k=N−1

observing (7.534), we obtain the relation

𝝑 = 𝚽h + 𝝃 (7.554)

Consequently, substituting (7.554) in (7.539), the estimation error vector can be expressed as

𝚫h = 𝚽−1 𝝃 (7.555)

If 𝑤 is zero-mean white noise with variance 𝜎𝑤2 , 𝝃 ∗ is a zero-mean random vector with correlation matrix

R𝝃 = E[𝝃 ∗ 𝝃 T ] = 𝜎𝑤2 𝚽∗ (7.556)

382 Chapter 7. Single-carrier modulation

Therefore, 𝚫h has zero mean and correlation matrix

R𝚫h = 𝜎𝑤2 (𝚽∗ )−1 (7.557)
In particular,
E[‖𝚫h‖2 ] = 𝜎𝑤2 tr[(𝚽∗ )−1 ] (7.558)
and, from (7.548), we get
Λn = (tr[𝚽−1 ])−1 LS (7.559)
Using as input sequence a CAZAC sequence, the matrix 𝚽 is diagonal,
𝚽=LI (7.560)
where I is the N × N identity matrix. The elements on the diagonal of 𝚽−1
are equal to 1∕L, and (7.559)
yields
L
Λn = LS–CAZAC (7.561)
N
The (7.561) gives a good indication of the relation between the number of observations L, the number of
system coefficients N, and Λn . For example, doubling the length of the PN sequence, Λn also doubles.
Now, using as PN sequence a maximum-length sequence (MLS) of periodicity L, and indicating with
𝟏N×N the matrix with all elements equal to 1, the correlation matrix 𝚽 can be written as
𝚽 = (L + 1)I − 𝟏N×N (7.562)
whose inverse is given by ( )
1 𝟏N×N
𝚽−1 = I+ (7.563)
L+1 L+1−N
which, substituted in (7.563), yields
(L + 1)(L + 1 − N)
Λn = LS-MLS (7.564)
N(L + 2 − N)

4
L = 63
3

2
Λn (dB)

L = 31
L = 15
1

0
L=7
–1 L=3

–2

–3
0 5 10 15 20 25 30
N

Figure 7.61 LS channel estimate. Λn vs. N for CAZAC sequences (solid line) and MLSs
(dotted-dashed line), for various values of L.
7.8. Channel estimation 383

In Figure 7.61, the behaviour of Λn is represented as a function of N, for CAZAC sequences (solid line)
and for MLSs (dotted-dashed line), with parameter L. We make the following observations.
1. For a given N, choosing L ≫ N, the two sequences yield approximately the same Λn . The worst
case is obtained for L = N; for example, for L = 15 the MLS yields a value of Λn that is about
3 dB lower than the upper bound (7.561). However, in order to get a good estimation performance,
usually L is from 5 to 10 times N.
2. For a given value of L, because of the presence of the noise 𝑤, the estimate of the coefficients
becomes worse if the number of coefficients N is larger than the number of coefficients of the
system Nh . On the other hand, if N is smaller than Nh , the estimation error may assume large
values (see (2.238)).
3. For sparse systems, where the number of coefficients may be large, but only a few of them are
non-zero, the estimate is usually very noisy. Therefore, after obtaining the estimate, it is necessary
to set to zero all coefficients whose amplitude is below a certain threshold.
The correlation method (7.523) is now analysed. From
1
ĥ COR = 𝝑 (7.565)
L
where 𝝑 is given by (7.536) and observing (7.554), we get
( )
1 1
𝚫h = 𝚽−I h+ 𝝃 (7.566)
L L
Consequently, the estimate is affected by a bias term equal to ((1∕L) 𝚽 − I)h, and has a covariance
matrix equal to (1∕L2 ) R𝝃 . In particular, using (7.556), it turns out
‖( 1 ) ‖2 𝜎 2
E[‖𝚫h‖2 ] = ‖ ‖ 𝑤
‖ L 𝚽 − I h‖ + L2 tr[𝚽] (7.567)
‖ ‖
and
1
Λn = COR (7.568)
1 ‖( 1 ) ‖2
1
tr[𝚽] + ‖ 𝚽 − I h‖
‖ ‖ 2
L2 ‖ L ‖ 𝜎𝑤
Using a CAZAC sequence, from (7.560) the second term of the denominator in (7.568) vanishes, and
L
Λn = COR–CAZAC (7.569)
N
as in (7.561). In fact, for a CAZAC sequence, as (7.520) is strictly true and 𝚽−1 is diagonal, the LS
method coincides with the correlation method.
Using instead an MLS, from (7.562), we get
‖( 1 ) ‖2 ∑
N−1
‖ 𝚽 − I h ‖ = 1 ‖(𝟏 − I) h‖2 = 1 |h − (0)|2
‖ L ‖ L i=0 | i |
‖ ‖ L 2 2

1
= 2 (‖h‖2 + (N − 2) |(0)|2 ) (7.570)
L
∑N−1
where (0) = i=0 hi . Moreover, we have
tr[𝚽] = NL (7.571)
hence
L
Λn = [ ] COR–MLS (7.572)
1 |(0)|2
N + L Λ + (N − 2)
𝜎𝑤2
384 Chapter 7. Single-carrier modulation

where Λ is defined in (7.547). We observe that using the correlation method, we obtain the same values
Λn (7.564) as the LS method, if L is large enough to satisfy the condition
|(0)|2
Λ + (N − 2) <L
𝜎𝑤2

On the selection of the channel length

Indeed, both COR and LS methods provide an estimate of the channel coefficients once the channel
length has been selected. For a suitable choice of the channel length Nh , we can start from a small value
of N, increase it gradually and stop when ∕L does not significantly change, i.e. it reaches approximately
its asymptotic value. As from Observation 2 at page 383, we assume that for N < Nh ∕L is affected
by a bias due to the truncation of the channel impulse response (see (2.238)), while for N > Nh some
coefficients of the estimate are just noise.
Note that by a conservative choice of N = Nmax > Nh , with corresponding PN input sequence length L,
we can evaluate the channel estimate for all values of N up to Nmax and select Nh by the above procedure.

7.8.4 Channel estimation for multirate systems

Let us assume the input to the channel is at T while its output is at T∕2 as illustrated in Figure 7.62 with
d(n) = z(n) + 𝑤(n) (7.573)
∑
∞
z(n) = x(k)h(n − 2k) (7.574)
k=−∞

Extension to outputs at T∕M, with M integer, is straightforward.

w(n)

x(k) z(n) d(n)

h
T T/2

Figure 7.62 Multirate system.

By the polyphase representation (see Section 1.A.9) of d, 𝑤, and h, with i = 0, 1,

d(i) (k) = d(2k + i) (7.575)
𝑤 (k) = 𝑤(2k + i)
(i)
(7.576)
(i)
z (k) = z(2k + i) (7.577)
(i)
h (m) = h(2m + i) (7.578)
the system of Figure 7.62 can be redrawn as in Figure 7.63.
For an input PN-sequence x(k), the output d(n) at T∕2 is split into its even (d(0) ) and odd (d(1) ) number
samples at T. Next, for each subsequence d(i) by using one of the above methods (LS or COR) we can
obtain an estimate of the corresponding polyphase channel component h(i) . Note that in Figure 7.62 for a
WSS noise 𝑤 it is
𝜎𝑤2 (0) = 𝜎𝑤2 (1) = 𝜎𝑤2 (7.579)
Moreover, as
||h||2 = ||h(0) ||2 + ||h(1) ||2 (7.580)
7.8. Channel estimation 385

w(0)(k)

z(0)(k) d (0)(k)
h(0)
T
x(k) d(n)
w(1)(k) P/S
T T/2
z(1)(k) d (1)(k)
h(1)
T

Figure 7.63 Polyphase decomposition of a multirate system.

it is easy to show that

𝜎𝑤2 1
Λn = = (7.581)
𝙼x ||h||2 1 1
+
Λ(0)
n Λ(1)
n
where
𝜎𝑤2 (i)
Λ(i)
n = (7.582)
𝙼x ||h(i) ||2

7.8.5 The LMMSE method

We refer to the system model of Figure 7.60. Let us assume that 𝑤 and h are statistically independent
random processes, whose second-order statistic is known.
For a known input sequence x, we estimate h using the LMMSE method given in Section 2.4, from
the observation of the noisy output sequence d, which now will be denoted as o. Let C be a L × N matrix
and ĥ = CT o with 𝑤 and h both random. The problem is the following:
ĥ LMMSE = arg min E[||h − h||
̂ 2] (7.583)
C

We note that in the LS method, the observation was the transmitted signal x, while the desired signal
was given by the system noisy output d. The observation is now given by d and the desired signal is the
system impulse response h. Consequently, some caution is needed to apply (2.198) to the problem under
investigation. Recalling the definition (7.543) of the observation vector o, and (2.198), the LMMSE
estimator is given by
ĥ LMMSE = (R−1 T
o Roh ) o (7.584)
where we have assumed E[o] = 𝟎 and E[h] = 𝟎. Consequently, (7.584) provides an estimate only of the
random (i.e. non-deterministic) component of the channel impulse response.
Now, from (7.530), defining the random vector with noise components
𝒘T = [𝑤(N − 1), … , 𝑤((N − 1) + (L − 1))] (7.585)
we can write
o = h + 𝒘 (7.586)
Assuming that the sequence {x(k)} is known, we have
Ro = E[o∗ oT ] =  ∗ Rh  T + R𝒘 (7.587)
386 Chapter 7. Single-carrier modulation

and
Roh = E[o∗ hT ] =  ∗ Rh (7.588)
Then (7.584) becomes
ĥ LMMSE = [( ∗ Rh  T + R𝒘 )−1  ∗ Rh ]T o (7.589)
Using the matrix inversion lemma (3.160), (7.589) can be rewritten as
ĥ LMMSE = [(R∗h )−1 +  H (R∗𝒘 )−1 ]−1  H (R∗𝒘 )−1 o (7.590)
If R𝒘 = 𝜎𝑤2 I, we have
ĥ LMMSE = [𝜎𝑤2 (R∗h )−1 +  H ]−1  H o (7.591)
We note that with respect to the LS method (7.545), the LMMSE method (7.591) introduces a weighting
of the components given by 𝝑 =  H o, which depends on the ratio between the noise variance and the
variance of h. If the variance of the components of h is large, then Rh is also large and likely R−1
h can
be neglected in (7.591).
We conclude recalling that Rh is diagonal for a wide-sense stationary uncorrelated scattering
(WSSUS) radio channel model (see (4.86)), whose components are derived by the power delay profile.
For an analysis of the estimation error, we can refer to (2.201), which uses the error vector
𝚫h = ĥ LMMSE − h having a correlation matrix
R𝚫h = {[(R∗h )−1 +  H (R∗𝒘 )−1 ]∗ }−1 (7.592)
If R𝒘 = 𝜎𝑤2 I, we get
R𝚫h = 𝜎𝑤2 {[𝜎𝑤2 (R∗h )−1 +  H ]∗ }−1 (7.593)
Moreover, in general
E[‖𝚫h‖2 ] = tr[R𝚫h ] (7.594)
For a comparison with other estimators, from (7.548),
𝜎𝑤2
Λn = (7.595)
𝙼x tr[R𝚫h ]
Given the required information about the statistical description of the channel, this method is not exten-
sively used in the single carrier system of Figure 7.60.

7.9 Faster-than-Nyquist Signalling

Introduced in 1975 by Mazo [40], faster-than-Nyquist (FTN) signalling, for an ideal non-dispersive chan-
nel, utilizes as transmit and receive filter a square-root Nyquist pulse with Nyquist frequency 1∕(2T ′ ),
while the transmit symbol rate is 1∕T, with T = 𝜏T ′ , where 𝜏 is a system parameter such that 0 < 𝜏 ≤ 1.
Note that for 𝜏 = 1 we obtain the traditional ISI free system. The system transmission bandwidth is
(1 + 𝜌)1∕T ′ , with 𝜌 being the roll-off factor of the transmit and receive filters. At the output of the
receiver matched filter the sampled signal at rate 1∕(𝜏T ′ ) is affected by substantial ISI, especially for
a small 𝜏. However, the system spectral efficiency is increased by a factor 1∕𝜏. Similarly to other pro-
posed modulation schemes (e.g. Section 8.8), a trade-off is obtained between increased performance
(here spectral efficiency) and receiver complexity [41, 42]. We note that the choice of coding and of
codeword length play a fundamental role in these systems.
Bibliography 387

Bibliography

[1] Couch, L.W. (1997). Digital and Analog Communication Systems. Upper Saddle River, NJ:
Prentice-Hall.
[2] Proakis, J.G. and Salehi, M. (1994). Communication System Engineering. Englewood Cliffs, NJ:
Prentice-Hall.
[3] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[4] Franks, L.E. (1981). Signal Theory, revised ed. Stroudsburg, PA: Dowden and Culver.
[5] Gitlin, R.D. and Weinstein, S.B. (1981). Fractionally-spaced equalization: an improved digital
transversal equalizer. Bell System Technical Journal 60. 275–296.
[6] Mazo, J.E. and Salz, J. (1965). Probability of error quadratic detector. Bell System Technical Jour-
nal 44. 2165–2186.
[7] Benvenuto, N. and Tomasin, S. (2002). On the comparison between OFDM and single carrier mod-
ulation with a DFE using a frequency domain feedforward filter. IEEE Transactions on Communica-
tions. 50: (6) 947–955.
[8] Benvenuto, N., Dinis, R., Falconer, D., and Tomasin, S. (2010). Single carrier modulation with
nonlinear frequency domain equalization: an idea whose time has come–again. Proceedings of the
IEEE 98: 69–96.
[9] Benvenuto, N. and Tomasin, S. (2002). Block iterative DFE for single carrier modulation. Electron-
ics Letters 38: 1144–1145.
[10] Benvenuto, N., Ciccotosto, S., and Tomasin, S. (2015). Iterative block fractionally spaced nonlinear
equalization for wideband channels. IEEE Wireless Communications Letters 4: 489–492.
[11] Sari, H., Karam, G., and Jeanclaude, I. (1995). Transmission techniques for digital terrestrial TV
broadcasting. IEEE Communications Magazine 33: 100–109.
[12] Huemer, M., Reindl, L., Springer, A., and Weigel, R. (2000). Frequency domain equalization of lin-
ear polyphase channels. Proceedings of 2000 IEEE Vehicular Technology Conference, Tokyo, Japan.
[13] Proakis, J.G. (1995). Digital Communications, 3e. New York, NY: McGraw-Hill.
[14] Messerschmitt, D.G. and Lee, E.A. (1994). Digital Communication, 2e. Boston, MA: Kluwer
Academic Publishers.
[15] Rabiner, L.R. (1989). A tutorial on hidden Markov models and selected applications in speech
recognition. IEEE Proceedings 77: 257–285.
[16] Bahl, L.R., Cocke, J., Jelinek, F., and Raviv, J. (1974). Optimal decoding of linear codes for mini-
mizing symbol error rate. IEEE Transactions on Information Theory 20: 284–287.
[17] Viterbi, A.J. (1998). An intuitive justification and simplified implementation of the MAP decoder
for convolutional codes. IEEE Journal on Selected Areas in Communications 16: 260–264.
[18] Ungerboeck, G. (1974). Adaptive maximum likelihood receiver for carrier modulated data transmis-
sion systems. IEEE Transactions on Communications 22: 624–635.
[19] Chugg, K.M. and Anastasopoulos, A. (2001). On symbol error probability bounds for ISI-like chan-
nels. IEEE Transactions on Communications 49: 1704–1709.
[20] Lee, W.U. and Hill, F.S. (1977). A maximum-likelihood sequence estimator with decision feedback
equalization. IEEE Transactions on Communications 25: 971–979.
[21] Foschini, G.J. (1977). A reduced-state variant of maximum-likelihood sequence detection attaining
optimum performance for high signal-to-noise ratio performance. IEEE Transactions on Information
Theory 24: 505–509.
[22] Anderson, J.B. and Mohan, S. (1984). Sequential coding algorithms: a survey and cost analysis.
IEEE Transactions on Communications 32: 169–176.
388 Chapter 7. Single-carrier modulation

[23] Eyuboglu, M.V. and Qureshi, S.U.H. (1988). Reduced-state sequence estimator with set partitioning
and decision feedback. IEEE Transactions on Communications 36: 13–20.
[24] Eyuboglu, M.V. (1989). Reduced-state sequence estimator for coded modulation on intersymbol
interference channels. IEEE Transactions on Communications 7: 989–995.
[25] Kamel, R.E. and Bar-Ness, Y. (1996). Reduced-complexity sequence estimation using state parti-
tioning. IEEE Transactions on Communications 44: 1057–1063.
[26] Sheen, W. and Stuber, G.L. (1992). Error probability for reduced-state sequence estimation. IEEE
Journal on Selected Areas in Communications 10: 571–578.
[27] Raheli, R., Polydoros, A., and Tzou, C.-K. (1995). Per-survivor processing: a general approach to
MLSE in uncertain environments. IEEE Transactions on Communications 43: 354–364.
[28] Cioffi, J.M., Dudevoir, G.P., Eyuboglu, M.V., and Forney, G.D. Jr. (1995). MMSE
decision-feedback equalizers and coding. IEEE Transactions on Communications 43: 2582–2604.
[29] Ungerboeck, G. (1982). Channel coding with multilevel/phase signals. IEEE Transactions on Infor-
mation Theory 28: 55–67.
[30] Forney, G.D. Jr. and Ungerboeck, G. (1998). Modulation and coding for linear Gaussian channels.
IEEE Transactions on Information Theory 44: 2384–2415.
[31] Eyuboglu, M.V. and Forney, G.D. Jr. (1992). Trellis precoding: combined coding, precoding and
shaping for intersymbol interference channels. IEEE Transactions on Information Theory 38:
301–314.
[32] Laroia, R., Tretter, S.A., and Farvardin, N. (1993). A simple and effective precoding scheme for
noise whitening on intersymbol interference channels. IEEE Transactions on Communications 41:
1460–1463.
[33] Laroia, R. (1996). Coding for intersymbol interference channels - combined coding and precoding.
IEEE Transactions on Information Theory 42: 1053–1061.
[34] Cherubini, G., Ölcer, S., and Ungerboeck, G. (1997). Trellis precoding for channels with spectral
nulls. Proceedings 1997 IEEE Internaitonal Symposium on Information Theory, Ulm, Germany, p.
464.
[35] Tomlinson, M. (1971). New automatic equalizer employing modulo arithmetic. Electronics Letters
7: 138–139.
[36] Harashima, H. and Miyakawa, H. (1972). Matched transmission technique for channels with inter-
symbol interference. IEEE Transactions on Communications 20: 774–780.
[37] Marple, S.L. Jr. (1981). Efficient least squares FIR system identification. IEEE Transactions on
Acoustics, Speech, and Signal Processing 29: 62–73.
[38] Nagumo, J.I. and Noda, A. (1967). A learning method for system identification. IEEE Transactions
on Automatic Control 12: 282–287.
[39] Crozier, S.N., Falconer, D.D., and Mahmoud, S.A. (1991). Least sum of squared errors (LSSE)
channel estimation. IEE Proceedings-F 138: 371–378.
[40] Mazo, J.E. (1975). Faster-than-Nyquist signaling. Bell System Technical Journal 54: 1451–1462.
[41] Anderson, J.B. (2016). Faster-than-Nyquist signaling for 5G communications. In: Signal Processing
for 5G: Algorithms and Implementations, (ed. Fa-Long Luo, Charlie (Jianzhong) Zhang) 24–26.
Wiley-IEEE Press.
[42] Li, S., Bai, B., Zhou, J. et al. (2018). Reduced-complexity equalization for faster-than Nyquist
signaling: new methods based on Ungerboeck observation model. IEEE Transactions on Communi-
cations 66: 1190–1204.
[43] Jeruchim, M.C., Balaban, P., and Shanmugan, K.S. (1992). Simulation of Communication Systems.
New York, NY: Plenum Press.
7.A. Simulation of a QAM system 389

[44] Benedetto, S. and Biglieri, E. (1999). Principles of Digital Transmission with Wireless Applications.
New York, NY: Kluwer Academic Publishers.
[45] Kabal, P. and Pasupathy, P. (1975). Partial-response signaling. IEEE Transactions on Communica-
tions 23: 921–934.
[46] Duttweiler, D.L., Mazo, J.E., and Messerschmitt, D.G. (1974). An upper bound on the error proba-
bility in decision-feedback equalization. IEEE Transactions on Information Theory 20: 490–497.
[47] Birkoff, G. and MacLane, S. (1965). A Survey of Modern Algebra, 3e. New York, NY: Macmillan
Publishing Company.

Appendix 7.A Simulation of a QAM system

In Figure 7.11, we consider the baseband equivalent scheme of a QAM system. The aim is to simulate
the various transformations in the discrete-time domain and to estimate the bit error probability.
This simulation method, also called Monte Carlo, is simple and general because it does not require
any special assumption on the processes involved; however, it is intensive from the computational point
of view. For alternative methods, for example semi-analytical, to estimate the error probability, we refer
to specific texts on the subject [43].
We describe the various transformations in the overall discrete-time system depicted in Figure 7.64,
where the only difference with respect to the scheme of Figure 7.11 is that the filters are discrete-time
with quantum TQ = T∕Q0 , which is chosen to accurately represent the various signals.

(a)
bit mapper
bl ak s(qTQ) sC(qTQ) rC(qTQ)
BMAP hTx gC
Tb T TQ = T TQ TQ
Q0

wC (qTQ) = wʹI (qTQ) + jwʹQ (qTQ)

(b)
t0 + nTc
rC(qTQ) x(pTʹQ) yk
gM c F0
TQ TʹQ
Tc = T Tc T
F0

data inverse
t0 + kT detector bit mapper
rC(qTQ) rR(qTQ) yk a^k b^ l
gRc IBMAP
TQ TQ T T Tb

Figure 7.64 Baseband equivalent model of a QAM system with discrete-time filters and sampling
period TQ = T∕Q0 . At the receiver, in addition to the general scheme, a multirate structure to obtain
samples of the received signal at the timing phase t0 is also shown. (a) Transmitter and channel
block diagram. (b) Receiver block diagram.
390 Chapter 7. Single-carrier modulation

0 Nh = 17
Nh = 25
Nh = 33
−10

−20
│HTx ( f )│ (dB)

−30

−40

−50

−60
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
fT

Figure 7.65 Magnitude of the transmit filter frequency response, for a windowed square root raised
cosine pulse with roll-off factor 𝜌 = 0.3, for three values of Nh (TQ = T∕4).

Binary sequence {b𝓁 }: The sequence {b𝓁 } is generated as a random sequence or as a PN sequence
(see Appendix 1.C), and has length K.

Bit mapper: The bit mapper maps patterns of information bits to symbols; the symbol constellation
depends on the modulator (see Figure 6.2 for two constellations).

Interpolator filter hTx from period T to TQ : The interpolator filter is efficiently implemented by using
the polyphase representation (see Appendix 1.A). For a bandlimited pulse of the raised cosine or square
root raised cosine type the maximum value of TQ , submultiple of T, is T∕2. In any case, the imple-
mentation of filters, for example the filter representing the channel, and non-linear transformations,
for example the transformation due to a power amplifier operating near saturation (not considered in
Figure 7.64), typically require a larger bandwidth, leading, for example to the choice TQ = T∕4 or T∕8.
In the following examples we choose TQ = T∕4.
For the design of hTx , the window method can be used (Nh odd):
[( ) ]
N −1
hTx (qTQ ) = hid q − h TQ 𝚠Nh (q), q = 0, 1, … , Nh − 1 (7.596)
2
where typically 𝚠Nh is the discrete-time rectangular window or the Hamming window, and hid is the ideal
impulse response.
Frequency responses of hTx are illustrated in Figure 7.65 for hid square root raised cosine pulse with
roll-off factor 𝜌 = 0.3, and 𝚠Nh rectangular window of length Nh , for various values of Nh (TQ = T∕4).
The corresponding impulse responses are shown in Figure 7.66.

Transmission channel. For a radio channel, the discrete-time model of Figure 4.20 can be used, where
in the case of channel affected by fading, the coefficients of the FIR filter that model the channel impulse
response are random variables with a given power delay profile.
7.A. Simulation of a QAM system 391

0.3 Nh = 17
hTx (qTQ)

0.2
0.1
0
–0.1
–5 0 5 10 15 20
q

0.3 Nh = 25
hTx (qTQ)

0.2
0.1
0
–0.1
0 5 10 15 20 25
q

0.3 Nh = 33
hTx (qTQ)

0.2
0.1
0
–0.1
0 5 10 15 20 25 30
q

Figure 7.66 Transmit filter impulse response, {hTx (qTQ )}, q = 0, … , Nh − 1, for a windowed square
root raised cosine pulse with roll-off factor 𝜌 = 0.3, for three values of Nh (TQ = T∕4).

We assume the statistical power of the signal at output of the transmission channel is given
by 𝙼sCh = 𝙼sC .

Additive white Gaussian noise. Let 𝑤I (qTQ ) and 𝑤Q (qTQ ) be two Gaussian statistically indepen-
dent r.v.s., each with zero mean and variance 1∕2, generated according to (1.582). To generate the
complex-valued noise signal {𝑤C (qTQ )} with spectrum N0 , it is sufficient to use the relation
𝑤C (qTQ ) = 𝜎𝑤C [𝑤I (qTQ ) + j𝑤Q (qTQ )] (7.597)
where
1
𝜎𝑤2 C = N0 (7.598)
TQ
Usually, the signal-to-noise ratio Γ of (6.69) is given. For a QAM system, from (7.98), we have
𝙼sC 𝙼s
Γ= = 2 C (7.599)
N0 (1∕T) 𝜎𝑤 (TQ ∕T)
C

The standard deviation of the noise to be inserted in (7.597) is given by

√
𝙼sC Q0
𝜎𝑤C = (7.600)
Γ
We note that 𝜎𝑤C is a function of 𝙼sC , of the oversampling ratio Q0 = T∕TQ , and of the given ratio Γ. In
place of Γ, the ratio Eb ∕N0 (equal to Γ∕log2 M for a uncoded system, see also (6.63)) may be assigned.
392 Chapter 7. Single-carrier modulation

Receive filter. As discussed in this chapter, there are several possible solutions for the receive filter.
The most common choice is a matched filter gM , matched to hTx , of the square root raised cosine type.
Alternatively, the receive filter may be a simple anti-aliasing FIR filter gAA , with passband at least equal
to that of the desired signal. The filter attenuation in the stopband must be such that the statistical power
of the noise evaluated in the passband is larger by a factor of 5–10 with respect to the power of the noise
evaluated in the stopband, so that we can neglect the contribution of the noise in the stopband at the
output of the filter gAA .
If we adopt as bandwidth of gAA the Nyquist frequency 1∕(2T), the stopband of an ideal filter with unit
gain goes from 1∕(2T) to 1∕(2TQ ): therefore, the ripple 𝛿s in the stopband must satisfy the constraint
1
N0 2T
( ) > 10 (7.601)
1 1
𝛿s2 N0 2TQ
− 2T

from which we get the condition

10−1
𝛿s2 < (7.602)
Q0 − 1
Usually, the presence of other interfering signals forces the selection of a value of 𝛿s that is smaller than
that obtained in (7.602).

Interpolator filter. The interpolator filter is used to increase the sampling rate from 1∕TQ to 1∕TQ′ : this
is useful when TQ is insufficient to obtain the accuracy needed to represent the timing phase t0 . This filter
can be part of gM or gAA . From Appendix 1.A, the efficient implementation of {gM (pTQ′ )} is obtained by
the polyphase representation with TQ ∕TQ′ branches.
To improve the accuracy of the desired timing phase, further interpolation, for example linear, may
be employed.

Timing phase. Assuming a training sequence is available, for example of the PN type {a0 = p(0),
a1 = p(1), … , aL−1 = p(L − 1), aL = p(0), aL+Nh −1 = p(Nh − 1)}, with Nh T duration of the overall sys-
tem impulse response, a simple method to determine t0 is to choose the timing phase in correspondence
of the peak of the overall impulse response. Let {x(pTQ′ )} be the signal before downsampling. If we
evaluate
mopt = arg max |𝚛xa (mTQ′ )|
m
| ∑ |
| 1 L−1 |
= arg max || x(𝓁T + mTQ′ )p∗ (𝓁)|| , mmin TQ′ < mTQ′ < mmax TQ′ (7.603)
m |L |
| 𝓁=0 |
then
t0 = mopt TQ′ (7.604)
In (7.603) mmin TQ′ and mmax TQ′ are estimates of minimum and maximum system delay, respectively.
Moreover, we note that the accuracy of t0 is equal to TQ′ and that the amplitude of the desired signal is
h0 = 𝚛xa (mopt TQ′ )∕𝚛a (0). Note that (7.603) is similar to the channel estimate by the correlation method
in Section 7.8, where now the received signal rather than the PN sequence is shifted in time.

Downsampler. The sampling period after downsampling is usually T or Tc = T∕2, with timing phase
t0 . The interpolator filter and the downsampler can be jointly implemented, according to the scheme of
Figure 1.71. For example, for TQ = T∕4, TQ′ = T∕8, and Tc = T∕2 the polyphase representation of the
interpolator filter with output {x(pTQ′ )} requires two branches. Also the polyphase representation of the
interpolator–decimator requires two branches.
7.B. Description of a finite-state machine 393

Equalizer. After downsampling, the signal is usually input to an equalizer (LE, FSE, or DFE). The
output signal of the equalizer has always sampling period equal to T. As observed several times, to
decimate simply means to evaluate the output at the desired instants.

Data detection. The simplest method resorts to a threshold detector, which operates on a sample by
sample basis.

Viterbi algorithm. An alternative to the threshold detector is represented by maximum likelihood

sequence detection by the Viterbi algorithm.

Inverse bit mapper. The inverse bit mapper performs the inverse function of the bit mapper. It translates
the detected symbols into bits that represent the recovered information bits.
Simulations are typically used to estimate the bit error probability of the system, for a certain set of
values of Γ. We recall that caution must be taken at the beginning and at the end of a simulation to
consider transients of the system. Let K be the number of recovered bits. The estimate of the bit error
probability Pbit is given by
number of bits received with errors
P̂ bit = (7.605)
number of received bits, K
It is known that as K → ∞ the estimate P̂ bit has a Gaussian probability distribution with mean Pbit
and variance Pbit (1 − Pbit )∕K. From this we can deduce, by varying K, the confidence interval [P− , P+ ]
within which the estimate P̂ bit approximates Pbit with an assigned probability, that is
P[P− ≤ P̂ bit ≤ P+ ] = Pconf (7.606)
For example, we have that with Pbit = 10−𝓁 and K = 10𝓁+1 , we get a confidence interval of about a
factor 2 with a probability of 95%, that is P[1∕2Pbit ≤ P̂ bit ≤ 2Pbit ] ≃ 0.95. This is in good agreement
with the experimental rule of selecting
K = 3 ⋅ 10𝓁+1 (7.607)
For a channel affected by fading, the average Pbit is not very significant: in this case, it is meaningful
to compute the distribution of Pbit for various channel realizations. In practice, we assume the
transmission of a sequence of Np packets, each one with K p information bits to be recovered: typically
K p = 1000-10 000 bits and Np = 100–1000 packets. Moreover, the channel realization changes at every
packet. For a given average signal-to-noise ratio Γ, the probability P̂ bit (np ), np = 1, … , Np , is computed
for each packet. As a performance measure, we use the percentage of packets with P̂ bit (np ) < Pbit ,
also called bit error probability cumulative distribution function (cdf), where Pbit assumes values in a
certain set.
This performance measure is more significant than the average Pbit evaluated for a very long, contin-
uous transmission of Np K p information bits. In fact, the average Pbit does not show that, in the presence
of fading, the system may occasionally have a very large Pbit , and consequently an outage.

Appendix 7.B Description of a finite-state machine

We consider a discrete-time system with input sequence {ik } and output sequence {ok } with sampling
time T. We say that the output sequence is generated by a finite-state machine (FSM) if there are a
sequence of states {sk } and two functions fo and fs , such that
ok = fo (ik , sk−1 ) (7.608)
sk = fs (ik , sk−1 ) (7.609)
394 Chapter 7. Single-carrier modulation

as illustrated in Figure 7.67. The first equation describes the fact that the output sample depends on the
current input and the state of the system. The second equation represents the memory part of the FSM
and describes the state evolution.

ik ok
fo(ik,sk – 1)
sk– 1 fs(ik,sk – 1) sk

Figure 7.67 General block diagram of a finite state sequential machine.

We note that if fs is a one-to-one function of ik , that is if every transition between two states is deter-
mined by a unique input value, then (7.608) can be written as
ok = f (sk , sk−1 ) (7.610)

Appendix 7.C Line codes for PAM systems

The objectives of line codes are
1. to shape the spectrum of the transmitted signals, and match it to the characteristics of the channel
(see (7.16)); this task may be performed also by the transmit filter;
2. to facilitate synchronization at the receiver, especially in case the information message contains
long sequences of ones or zeros;
3. to improve system performance in terms of Pe .
This appendix is divided in two parts: in the first several representations of binary symbols are listed; in
the second, partial response systems are introduced. For in-depth study and analysis of spectral properties
of line codes we refer to the bibliography, in particular [1, 44].

7.C.1 Line codes

With reference to Figure 7.68, the binary sequence {b𝓁 }, b𝓁 ∈ {0, 1}, could be directly generated by a
source, or be the output of a channel encoder. The sequence {ak } is produced by a line encoder. The
channel input is the PAM signal s, obtained by modulating a pulse hTx , here a rectangular pulse.

bl ak s(t)
encoder hTx
Tb T

Figure 7.68 PAM transmitter with line encoder.

7.C. Line codes for PAM systems 395

Non-return-to-zero format
The non-return-to-zero (NRZ) signals are antipodal signals: therefore, NRZ line codes are characterized
by the lowest error probability, for transmission over AWGN channels in the absence of ISI. Four formats
are illustrated in Figure 7.69.
1. NRZ level (NRZ-L) or, simply, NRZ:
1 and 0 are represented by two different levels.
2. NRZ mark (NRZ-M):
1 is represented by a level transition, 0 by no level transition.
3. NRZ space (NRZ-S):
1 is represented by no level transition, 0 by a level transition.
4. Dicode NRZ:
A change of polarity in the sequence {b𝓁 }, 1–0 or 0–1, is represented by a level transition; every
other case is represented by the zero level.

NRZ−L NRZ−M
2 2

1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1
1 1

0 0

−1 −1

−2 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

NRZ−S Dicode NRZ

2 2

1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1
1 1

0 0

−1 −1

−2 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

Figure 7.69 NRZ line codes.

396 Chapter 7. Single-carrier modulation

Return-to-zero format

1. Unipolar return-to-zero (RZ):

1 is represented by a pulse having duration equal to half a bit interval, 0 by a zero pulse; we observe
that the signal does not have zero mean. This property is usually not desirable, as, for example
for transmission over coaxial cables.
2. Polar RZ:
1 and 0 are represented by opposite pulses with duration equal to half a bit interval.
3. Bipolar RZ or alternate mark inversion (AMI):
Bits equal to 1 are represented by rectangular pulses having duration equal to half a bit interval,
sequentially alternating in sign, bits equal to 0 by the zero level.
4. Dicode RZ:
A change of polarity in the sequence {b𝓁 }, 1–0 or 0–1, is represented by a level transition, using a
pulse having duration equal to half a bit interval; every other case is represented by the zero level.

Unipolar RZ Polar RZ
2 2

1.5 1 0 1 1 0 0 0 1 1 0 1
1 0 1 1 0 0 0 1 1 0 1 1
1

0.5 0

0
−1
−0.5

−1 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

Bipolar RZ Dicode RZ
2 2

1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1
1 1

0 0

−1 −1

−2 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

Figure 7.70 RZ line codes.

RZ line codes are illustrated in Figure 7.70.

7.C. Line codes for PAM systems 397

Biphase format

1. Biphase level (B–𝜙–L) or Manchester NRZ:

1 is represented by a transition from high level to low level, 0 by a transition from low level to high
level. Long sequences of ones or zeros in the sequence {b𝓁 } do not create synchronization prob-
lems. It is easy to see, however, that this line code leads to a doubling of the transmission bandwidth.
2. Biphase mark (B–𝜙–M) or Manchester 1:
A transition occurs at the beginning of every bit interval; 1 is represented by a second transition
within the bit interval, 0 is represented by a constant level.
3. Biphase space (B–𝜙–S):
A transition occurs at the beginning of every bit interval; 0 is represented by a second transition
within the bit interval, 1 is represented by a constant level.

Biphase−L Biphase−M
2 2

1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1
1 1

0 0

−1 −1

−2 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

Biphase−S Delay modulation

2 2

1 0 1 1 0 0 0 1 1 0 1 1 0 1 1 0 0 0 1 1 0 1
1 1

0 0

−1 −1

−2 −2
0 2 4 6 8 10 0 2 4 6 8 10
t/T t/T

Figure 7.71 B–𝜙 and delay modulation line codes.

Biphase line codes are illustrated in Figure 7.71.

398 Chapter 7. Single-carrier modulation

Delay modulation or Miller code

1 is represented by a transition at midpoint of the bit interval, 0 is represented by a constant level; if 0 is
followed by another 0, a transition occurs at the end of the bit interval. This code shapes the spectrum
similar to the Manchester code, but requires a lower bandwidth. The delay modulation line code is
illustrated in Figure 7.71.

Block line codes

The input sequence {b𝓁 } is divided into blocks of K bits. Each block of K bits is then mapped into a
block of N symbols belonging to an alphabet of cardinality M, with the constraint

2K ≤ M N (7.611)

The KBNT codes are an example of block line codes where the output symbol alphabet is ternary
{−1, 0, 1}.

Alternate mark inversion

We consider the alternate mark inversion (AMI) differential binary encoder, that is

ak = bk − bk−1 with bk ∈ {0, 1} (7.612)

At the decoder, the bits of the information sequence may be recovered by

b̂ k = â k + b̂ k−1 (7.613)

Note that ak ∈ {−1, 0, 1}; in particular

{
±1, if bk ≠ bk−1
ak = (7.614)
0, if bk = bk−1
From (7.612), the relation between the PSDs of the sequences {ak } and {bk } is given by
a (f ) = b (f ) |1 − e−j2𝜋fT |2
= b (f ) 4 sin2 (𝜋fT)
Therefore, a (f ) exhibits zeros at frequencies that are integer multiples of 1∕T, in particular at f = 0.
Moreover, from (7.612), we have 𝚖a = 0, independently of the distribution of {bk }.
If the power of the transmitted signals is constrained, a disadvantage of the encoding method (7.612)
is a reduced noise immunity with respect to antipodal transmission, that is for ak ∈ {−1, 1}, because a
detector at the receiver must now decide among three levels. Moreover, long sequences of information
bits {b̂ k } that are all equal to 1 or 0 generate sequences of symbols {ak } that are all equal: this is not
desirable for synchronization.
In any case, the biggest problem is the error propagation at the decoder, which, observing (7.613),
given that an error occurs in {̂ak }, generates a sequence of bits {b̂ k } that are in error until another error
occurs in {̂ak }. This problem can be solved by precoding: from the sequence of bits {bk }, we first generate
the sequence of bits {ck }, with ck ∈ {0, 1}, by

ck = bk ⊕ ck−1 (7.615)

where ⊕ denotes the modulo 2 sum. Next,

ak = ck − ck−1 (7.616)
7.C. Line codes for PAM systems 399

with ak ∈ {−1, 0, 1}. Hence, it results

{
±1, if bk = 1
ak = (7.617)
0, if bk = 0
In other words, a bit bk = 0 is mapped into the symbol ak = 0, and a bit bk = 1 is mapped alternately
in ak = +1 or ak = −1. Consequently, from (7.617) decoding may be performed simply by taking the
magnitude of the detected symbol:
b̂ k = |̂ak | (7.618)

It is easy to prove that for a message {bk } with statistically independent symbols, and p = P[bk = 1], we
have
sin2 (𝜋fT)
a (f ) = 2p(1 − p) (7.619)
p2 + (1 − 2p)sin2 (𝜋fT)
The plot of a (f ) is shown in Figure 7.72 for different values of p. Note that the PSD presents a zero at
f = 0. Also in this case, 𝚖a = 0.

p = 3/4
4
Amplitude

2
p = 2/4

1
p = 1/4
0
0 0.1 0.2 0.3 0.4 0.5
fT

Figure 7.72 Power spectral density of an AMI encoded message.

We observe that the AMI line code is a particular case of the partial response system named
dicode [45].

7.C.2 Partial response systems

From Figure 7.1, we recall the block diagram of a baseband transmission system, where the symbols {ak }
belong to the alphabet (6.4)18 of cardinality M and 𝑤 is an additive white Gaussian noise (Figure 7.73).

18
In the present analysis, only M-PAM systems are considered; for M-QAM systems, the results can be extended to the
signals on the I and Q branches.
400 Chapter 7. Single-carrier modulation

ak rR(t) yk a^k ak sR,k yk a^k

q h
T T
t0 + kT
wR(t) wR,k

(a) (b)

Figure 7.73 Schemes equivalent to Figure 7.1. (a) Analog equivalent. (b) Digital equivalent.

We assume that the transmission channel is ideal: the overall system can then be represented as an
interpolator filter having impulse response
q(t) = hTx ∗ gRc (t) (7.620)
A noise 𝑤R , obtained by filtering 𝑤 by the receive filter, is added to the desired signal. Sampling
the received signal at instants t0 + kT yields the sequence {yk }, as illustrated in Figure 7.73a. The
discrete-time equivalent of the system is shown in Figure 7.73b, where {hi = q(t0 + iT)}, and 𝑤R,k =
𝑤R (t0 + kT). We assume that {hi } is equal to zero for i < 0 and i ≥ N.
The partial response (PR) polynomial of the system is defined as
∑
N−1
l(D) = 𝚕i Di (7.621)
i=0

where the coefficients {𝚕i } are equal to the samples {hi }, and D is the unit delay operator.

ak ak(t) rR(t) yk a^k

l(D) g
T
t0 + kT

wR(t)

Figure 7.74 Schemes equivalent to Figure 7.1. (a) Analog equivalent. (b) Digital equivalent.

A PR system is illustrated in Figure 7.74, where l(D) is defined in (7.621), and g is an analog filter
satisfying the Nyquist criterion for the absence of ISI (see (7.74))
∑
+∞ ( )
m
 f− =T (7.622)
m=−∞
T
The symbols at the output of the filter l(D) in Figure 7.74 are given by
∑
N−1
a(t)
k
= 𝚕i ak−i (7.623)
i=0

Note that the overall scheme of Figure 7.74 is equivalent to that of Figure 7.73a with
∑
N−1
q(t) = 𝚕i g(t − iT) (7.624)
i=0

Also, observing (7.74), the equivalent discrete-time model is obtained for hi = 𝚕i .

In other words, from (7.624), the system is decomposed into two parts:
● a filter with frequency response l(e−j2𝜋fT ), periodic of period 1∕T, that forces the system to have an
overall discrete-time impulse response equal to {hi };
7.C. Line codes for PAM systems 401

● an analog filter g that does not modify the overall filter h(D) and limits the system bandwidth.
As it will be clear from the analysis, the decomposition of Figure 7.74 on the one hand simplifies
the study of the properties of the filter h(D), and on the other hand allows us to to design an efficient
receiver.
The scheme of Figure 7.74 suggests two possible ways to implement the system of Figure 7.1:
1. Analog: the system is implemented in analog form; therefore, the transmit filter hTx and the receive
filter gRc must satisfy the relation
Tx (f ) Rc (f ) = (f ) = l(e−j2𝜋fT ) (f ) (7.625)
2. Digital: the filter l(D) is implemented as a component of the transmitter by a digital filter; then the
transmit filter h(PR)
Tx
and receive filter g(PR)
Rc
must satisfy the relation
(PR)
Tx (f ) (PR)
Rc
(f ) = (f ) (7.626)
The implementation of a PR system using a digital filter is shown in Figure 7.75.

ak ak(t) (PR) s(t) (PR)

rR(t) yk a^k
l(D) hTx gCh gRc
T t0 + kT
w(t)

Figure 7.75 Implementation of a PR system using a digital filter.

Note from (7.622) that in both relations (7.625) and (7.626) g is a Nyquist filter.

The choice of the PR polynomial

Several considerations lead to the selection of the polynomial l(D).

(A) System bandwidth: With the aim of maximizing the transmission bit rate, many PR systems are
designed for minimum bandwidth, i.e. from (7.625) it must be
1
l(e−j2𝜋fT ) (f ) = 0, |f | > (7.627)
2T
Substitution of (7.627) into (7.622) yields the following conditions on the filter g
{
T, |f | ≤
1  −1
( )
t
(f ) = 2T ←−→ g(t) = sinc (7.628)
0, elsewhere T
Correspondingly, observing (7.624) the filter q assumes the expression
∑
N−1 ( )
t − iT
q(t) = 𝚕i sinc (7.629)
i=0
T

(B) Spectral zeros at f = 1∕(2T): From signal theory, it is known that if (f ) and its first (n − 1)
derivatives are continuous and the n-th derivative is discontinuous, then |q(t)| asymptotically decays as
1∕|t|n+1 . The continuity of (f ) and of its derivatives helps to reduce the portion of energy contained in
the tails of q.
It is easily proven that in a minimum bandwidth system, the (n − 1)-th derivative of (f ) is continuous
if and only if l(D) has (1 + D)n as a factor. On the other hand, if l(D) has a 0 of multiplicity greater than
1 in D = −1, then the transition band of (f ) around f = 1∕(2T) can be widened, thus simplifying the
design of the analog filters.
402 Chapter 7. Single-carrier modulation

(C) Spectral zeros at f = 0: A transmitted signal with attenuated spectral components at low frequen-
cies is desirable in many cases, e.g. for transmission over channels with frequency responses that exhibit
a spectral null at the frequency f = 0. Note that a zero of l(D) in D = 1 corresponds to a zero of l(e−j2𝜋fT )
at f = 0.

(D) Number of output levels: From (7.623), the symbols at the output of the filter l(D) have an alphabet
(t) of cardinality M (t) . If we indicate with n𝚕 the number of coefficients of l(D) different from zero, then
the following inequality for M (t) holds

n𝚕 (M − 1) + 1 ≤ M (t) ≤ M n𝚕 (7.630)

In particular, if the coefficients {𝚕i } are all equal, then M (t) = n𝚕 (M − 1) + 1.

We note that, if l(D) contains more than one factor (1 ± D), then n𝚕 increases and, observing (7.630),
also the number of output levels increases. If the power of the transmitted signal is constrained, detection
of the sequence {a(t)k
} by a threshold detector will cause a loss in system performance.

(E) Some examples of minimum bandwidth systems: In the case of minimum bandwidth systems, it
is possible to evaluate the expression of (f ) and q once the polynomial l(D) has been selected.
As the coefficients {𝚕i } are generally symmetric or antisymmetric around i = (N − 1)∕2, it is conve-
nient to consider the time-shifted pulse
( )
q̃ (t) = q t + (N−1)T
2 (7.631)
̃ ) = ej𝜋f (N−1)T (f )
(f
In Table 7.1, the more common polynomials l(D) are described, as well as the corresponding expres-
̃ ) and q̃ , and the cardinality M (t) of the output alphabet (t) . In the next three examples,
sions of (f
polynomials l(D) that are often found in practical applications of PR systems are considered.

Table 7.1: Properties of several minimum bandwidth systems

l(D) ̃ ) for |f | ≤ 1∕(2T)

(f q̃ (t) M (t)
4T 2 cos(𝜋t∕T)
1+D 2T cos(𝜋fT) 2M − 1
𝜋 T 2 − 4t2
8Tt cos(𝜋t∕T)
1−D j2T sin(𝜋fT) 2M − 1
𝜋 4t2 − T 2
2T 2 sin(𝜋t∕T)
1 − D2 j2T sin(2𝜋fT) 2M − 1
𝜋 t2 − T 2
3 sin(𝜋t∕T)
2T
1 + 2D + D2 4Tcos2 (𝜋fT) 4M − 3
𝜋t T 2 − t2
64T 3 t cos(𝜋t∕T)
1 + D − D2 − D3 j4T cos(𝜋fT) sin(2𝜋fT) − 4M − 3
𝜋 (4t2 − 9T 2 )(4t2 − T 2 )
16T 2 cos(𝜋t∕T)(4t2 − 3T 2 )
1 − D − D2 + D3 −4T sin(𝜋fT) sin(2𝜋fT) 4M − 3
𝜋 (4t2 − 9T 2 )(4t2 − T 2 )
3 sin(𝜋t∕T)
8T
1 − 2D2 + D4 −4Tsin2 (2𝜋fT) 4M − 3
𝜋t t2 − 4T 2
( )
T2 3t − T
2 + D − D2 T + T cos(2𝜋fT) + j3T sin(2𝜋fT) sin(𝜋t∕T) 2 4M − 3
𝜋t 2
2
(t − T )
2T 2T − 3t
2 − D2 − D4 −T + T cos(4𝜋fT) + j3T sin(4𝜋fT) sin(𝜋t∕T) 2 4M − 3
𝜋t t − 4T 2
7.C. Line codes for PAM systems 403

Example 7.C.1 (Dicode filter)

The dicode filter introduces a zero at frequency f = 0 and has the following expression:
l(D) = 1 − D (7.632)
The frequency response, obtained by setting D = e−j2𝜋fT , is given by
l(e−j2𝜋fT ) = 2j e−j𝜋fT sin(𝜋fT) (7.633)

Example 7.C.2 (Duobinary filter)

The duobinary filter introduces a zero at frequency f = 1∕(2T) and has the following expression:
l(D) = 1 + D (7.634)
The frequency response is given by
l(e−j2𝜋fT ) = 2e−j𝜋fT cos(𝜋fT) (7.635)
Observing (7.629), we have ( ) ( )
t t−T
q(t) = sinc + sinc (7.636)
T T
The plot of the impulse response of a duobinary filter is shown in Figure 7.76 with a continuous line. We
notice that the tails of the two sinc functions cancel each other, in line with what was stated at point B)
regarding the aymptotical decay of the pulse of a PR system with a zero in D = −1.

1.5

0.5
q(t)

–0.5

–1

–3 –2 –1 0 1 2 3 4 5
t/T

Figure 7.76 Plot of q for duobinary (−) and modified duobinary (- -) filters.

Example 7.C.3 (Modified duobinary filter)

The modified duobinary filter combines the characteristics of duobinary and dicode filters and has the
following expression
l(D) = (1 − D) (1 + D) = 1 − D2 (7.637)
404 Chapter 7. Single-carrier modulation

The frequency response becomes

l(e−j2𝜋fT ) = 1 − e−j4𝜋fT = 2j e−j2𝜋fT sin(2𝜋fT) (7.638)
Using (7.629) it results ( ) ( )
t t − 2T
− sinc
q(t) = sinc (7.639)
T T
The plot of the impulse response of a modified duobinary filter is shown in Figure 7.76 with a dashed line.
(F) Transmitted signal spectrum: With reference to the PR system of Figure 7.75, the spectrum of
the transmitted signal is given by (see (7.16))
|1 |2
s (f ) = || l(e−j2𝜋fT ) Tx
(PR)
(f )|| a (f ) (7.640)
|T |
(PR)
For a minimum bandwidth system, with Tx (f ) given by (7.628), (7.640) simplifies into
⎧ |l(e−j2𝜋fT )|2  (f ), 1
|f | ≤
⎪ a
2T
s (f ) = ⎨ (7.641)
⎪ 0, 1
⎩ |f | >
2T
In Figure 7.77, the PSD of a minimum bandwidth PR system is compared with that of a PAM sys-
tem. The spectrum of the sequence of symbols {ak } is assumed white. For the PR system, a mod-
ified duobinary filter is considered, so that the spectrum is obtained as the product of the functions
(PR)
|l(−ej2𝜋fT )|2 = |2 sin(2𝜋fT)|2 and |Tx (f )|2 = T 2 rect(fT), plotted with continuous lines. For the PAM
system, the transmit filter hTx is a square root raised cosine with roll-off factor 𝜌 = 0.5, and the spectrum
is plotted with a dashed line.

2
1.8 PAM
PR
1.6
1.4
Amplitude

1.2
1
0.8
0.6
0.4
0.2
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
fT

Figure 7.77 PSD of a modified duobinary PR system and of a PAM system.

Symbol detection and error probability

We consider the discrete-time equivalent scheme of Figure 7.73b; the signal sR,k can be expressed as a
function of symbols {ak } and coefficients {𝚕i } of the filter l(D) in the following form:
∑
N−1
sR,k = a(t)
k
= 𝚕0 ak + 𝚕i ak−i (7.642)
i=1
7.C. Line codes for PAM systems 405

The term 𝚕0 ak is the desired part of the signal sR,k , whereas the summation represents the ISI term that
is often designated as controlled ISI, as it is deliberately introduced.

yk yk yk ~y
1 a^k k a^k
l(D)

1 l(D)
1–
l0 l0

(a) (b)

yk a^ k(t) 1 a^k yk a^k

MLSD
l(D)

(c) (d)

Figure 7.78 Four possible solutions to the detection problem in the presence of controlled ISI. (a)
LE-ZF followed by M-level threshold detector. (b) DFE. (c) M(t) -level threshold detector followed by
LE-ZF. (d) Viterbi algorithm.

The receiver detects the symbols {ak } using the sequence of samples {yk = a(t)
k
+ 𝑤R,k }. We discuss
four possible solutions.19

1. LE-ZF: A zero-forcing linear equalizer (LE-ZF) having D transform equal to 1∕l(D) is used. At the
equalizer output, at instant k the symbol ak plus a noise term is obtained; the detected symbols {̂ak }
are obtained by an M-level threshold detector, as illustrated in Figure 7.78a. We note however that
the amplification of noise by the filter 1∕l(D) is infinite at frequencies f such that l(e−j2𝜋fT ) = 0.
2. DFE: A second solution resorts to a decision-feedback equalizer (DFE), as shown in Figure 7.78b.
An M-level threshold detector is also employed by the DFE, but there is no noise amplification as
the ISI is removed by the feedback filter, having D transform equal to 1 − l(D)∕𝚕0 . We observe that
at the decision point, the signal yk has the expression
( )
1 (t)
∑
N−1
ỹ k = ak + 𝑤R,k − 𝚕i â k−i (7.643)
𝚕0 i=1

If we indicate with ek = ak − â k a detection error, then substituting (7.642) in (7.643), we obtain

( )
1 ∑
N−1
ỹ k = ak + 𝑤R,k + 𝚕i ek−i (7.644)
𝚕0 i=1

Equation (7.644) shows that a wrong decision negatively influence successive decisions: this phe-
nomenon is known as error propagation.

19
For a first reading, it is suggested that only solution 3 is considered. The study of the other solutions falls within the
equalization methods.
406 Chapter 7. Single-carrier modulation

3. Threshold detector with M (t) levels: This solution, shown in Figure 7.78c, exploits the M (t) -ary
nature of the symbols a(t)
k
, and makes use of a threshold detector with M (t) levels followed by
a LE-ZF. This structure does not lead to noise amplification as solution 1, because the noise is
eliminated by the threshold detector; however, there is still the problem of error propagation.
4. Viterbi algorithm: This solution, shown in Figure 7.78d, corresponds to maximum-likelihood
sequence detection (MLSD) of {ak }. It yields the best performance.

Solution 2 using the DFE is often adopted in practice: in fact it avoids noise amplification and is
simpler to implement than the Viterbi algorithm. However, the problem of error propagation remains.
In this case, using (7.644), the error probability can be written as
[| | ]
( ) | ∑
N−1
|
1 | |
Pe = 1 − P |𝑤R,k + 𝚕i ek−i | > 𝚕0 (7.645)
M | |
| i=1 |
A lower bound Pe,L can be computed for Pe by assuming the error propagation is absent, or setting
{ek } = 0, ∀k, in (7.645). If we denote by 𝜎𝑤R , the standard deviation of the noise 𝑤R,k , we obtain
( )
( ) 𝚕0
1
Pe,L = 2 1 − Q (7.646)
M 𝜎𝑤R

Assuming 𝑤R,k white noise, an upper bound Pe,U is given in [46] in terms of Pe,L :

M N−1 Pe,L
Pe,U = (7.647)
(M∕(M − 1)) Pe,L (M N−1 − 1) + 1
From (7.647), we observe that the effect of the error propagation is that of increasing the error probability
by a factor M N−1 with respect to Pe,L .
A solution to the problem of error propagation is represented by precoding, which will be investigated
in depth in Section 7.7.

Precoding
We make use here of the following two simplifications:

1. the coefficients {𝚕i } are integer numbers;

2. the symbols {ak } belong to the alphabet  = {0, 1, … , M − 1}; this choice is made because arith-
metic modulo M is employed.
(p)
We define the sequence of precoded symbols {ak } as
( )
(p)
∑
N−1
(p)
ak 𝚕0 = ak − 𝚕i ak−i mod M (7.648)
i=1

We note that (7.648) has only one solution if and only if 𝚕0 and M are relatively prime [47]. In case
𝚕0 = · · · = 𝚕j−1 = 0 mod M, and 𝚕j and M are relatively prime, (7.648) becomes
( )
(p)
∑
N−1
(p)
ak−j 𝚕j = ak − 𝚕i ak−i mod M (7.649)
i=j+1

For example, if l(D) = 2 + D − D2 and M = 2, (7.648) is not applicable as 𝚕0 mod M = 0. Therefore,

(7.649) is used.
7.C. Line codes for PAM systems 407

(p)
Applying the PR filter to {ak }, we obtain the sequence
∑
N−1
(p)
a(t)
k
= 𝚕i ak−i (7.650)
i=0

From the comparison between (7.648) and (7.650), or in general (7.649), we have the fundamental
relation
a(t)
k
mod M = ak (7.651)

Equation (7.651) shows that, as in the absence of noise we have yk = a(t) k

, the symbol ak can be detected
by considering the received signal yk modulo M; this operation is memoryless, therefore, the detection
of â k is independent of the previous detections {̂ak−i }, i = 1, … , N − 1. Therefore, the problem of error
propagation is solved. Moreover, the desired signal is not affected by ISI.
If the instantaneous transformation
(p) (p)
ak = 2ak − (M − 1) (7.652)
(p)
is applied to the symbols {ak }, then we obtain a sequence of symbols that belong to the alphabet (p)
(p)
in (6.4). The sequence {ak } is then input to the filter l(D). Precoding consists of the operation (7.648)
followed by the transformation (7.652).
However, we note that (7.651) is no longer valid. From (7.652), (7.650), and (7.648), we obtain the
new decoding operation, given by
( (t) )
ak
ak = +  mod M (7.653)
2
where
∑
N−1
 = (M − 1) 𝚕i (7.654)
i=0

A PR system with precoding is illustrated in Figure 7.79. The receiver is constituted by a threshold
detector with M (t) levels that provides the symbols {̂a(t)
k
}, followed by a block that realizes (7.653) and
yields the detected data {̂ak }.

ak ak(p) ak(t) yk a^ k(t) a^k

precoder l(D) decoder

Figure 7.79 PR system with precoding .

Error probability with precoding

To evaluate the error probability of a system with precoding, the statistics of the symbols {a(t)k
} must be
(t)
known; it is easy to prove that if the symbols {ak } are i.i.d., the symbols {ak } are also i.i.d.
If we assume that the cardinality of the set (t) is maximum, i.e. M (t) = M n𝚕 , then the output levels
are equally spaced and the symbols a(t) k
result equally likely with probability
1
P[a(t)
k
= 𝛼] = , 𝛼 ∈ (t) (7.655)
M n𝚕
In general, however, the symbols {a(t)
k
} are not equiprobable, because several output levels are redundant,
as can be deduced from the following example:
408 Chapter 7. Single-carrier modulation

Example 7.C.4 (Dicode filter)

We assume M = 2, therefore, ak = {0, 1}; the precoding law (7.648) is simply an exclusive or and
(p) (p)
ak = ak ⊕ ak−1 (7.656)
(p)
The symbols {ak } are obtained from (7.652),
(p) (p)
ak = 2ak − 1 (7.657)
(p)
they are antipodal as ak = {−1, +1}. Finally, the symbols at the output of the filter l(D) are given by
(p) (p) (p) (p)
a(t)
k
= ak − ak−1 = 2(ak − ak−1 ) (7.658)

Table 7.2: Precoding for

the dicode filter

(p) (p)
ak−1 ak ak a(t)
k
0 0 0 0
0 1 1 +2
1 0 1 0
1 1 0 −2

(p) (p)
The values of ak−1 , ak , ak and a(t)
k
are given in Table 7.2. We observe that both output levels ±2
correspond to the symbol ak = 1 and therefore, are redundant; the three levels are not equally likely. The
symbol probabilities are given by
1
P[a(t)
k
= ±2] =
4
1
P[a(t)k
= 0] = (7.659)
2

(p)
ak ak(p) 0
← –1 ak
←
1 +1

D
(a)
←
a^ k(t) 0 0 a^k
←
±2 1
(b)
Figure 7.80 Precoder and decoder for a dicode filter l(D) with M = 2. (a) Precoder. (b) Decoder.

Figure 7.80a shows the precoder that realizes equations (7.656) and (7.657). The decoder, realized as
a map that associates the symbol â k = 1 to ±2, and the symbol â k = 0 to 0, is illustrated in Figure 7.80b.

Alternative interpretation of PR systems

Up to now we have considered a general transmission system, and looked for an efficient design method.
We now assume that the system is given, i.e. that the transmit filter as well as the receive filter are
assigned. The scheme of Figure 7.74 can be regarded as a tool for the optimization of a given system,
where l(D) includes the characteristics of the transmit and receive filters: as a result, the symbols {a(t)
k
}
7.C. Line codes for PAM systems 409

are no longer the transmitted symbols, but must be interpreted as the symbols that are ideally received.
In the light of these considerations, the assumption of an ideal channel can also be removed. In this case,
the filter l(D) will also include the ISI introduced by the channel.
We observe that the precoding/decoding technique is an alternative equalization method to the DFE
that presents the advantage of eliminating error propagation, which can considerably deteriorate system
performance.
In the following two examples [14], additive white Gaussian noise 𝑤R,k = 𝑤̃ k is assumed, and various
systems are studied for the same signal-to-noise ratio at the receiver.

Example 7.C.5 (Ideal channel g)

(A) Antipodal signals: We transmit a sequence of symbols from a binary alphabet, ak ∈ {−1, 1}. The
received signal is
yk = ak + 𝑤̃ A,k (7.660)
where the variance of the noise is given by 𝜎𝑤2̃ = 𝜎I2 .
A
At the receiver, using a threshold detector with threshold set to zero, we obtain
( )
1
Pbit = Q (7.661)
𝜎I
(p) (p)
(B) Duobinary signal with precoding: The transmitted signal is now given by a(t)
k
= ak + ak−1 ∈
(p)
{−2, 0, 2}, where ak ∈ {−1, 1} is given by (7.652) and (7.648).
The received signal is given by
yk = a(t)
k
+ 𝑤̃ B,k (7.662)
where the variance of the noise is 𝜎𝑤2̃ = 2𝜎I2 , as 𝜎 2(t) = 2.
B ak
At the receiver, using a threshold detector with thresholds set at ±1, we have the following conditional
error probabilities: ( )
(t) 1
P[E ∣ ak = 0] = 2Q
𝜎𝑤̃ B
( )
(t) (t) 1
P[E ∣ ak = −2] = P[E ∣ ak = 2] = Q
𝜎𝑤̃ B
Consequently, at the detector output, we have
Pbit = P[̂ak ≠ ak ]
1 1
= P [𝙴 ∣ a(t)
k
= 0] + P [𝙴 ∣ a(t)
k
= ±2]
( ) 2 2
1
= 2Q √
2 𝜎I
We observe a worsening of about 3 dB in terms of the signal-to-noise ratio with respect to case A).
(C) Duobinary signal. The transmitted signal is a(t)
k
= ak + ak−1 . The received signal is given by
yk = ak + ak−1 + 𝑤̃ C,k (7.663)
where 𝜎𝑤2̃
= 2𝜎I2 .
We consider using a receiver that applies MLSD to recover the data; from
C
Example 7.5.3 on page 365 it results
( √ ) ( )
8 1
Pbit = K Q =KQ (7.664)
2𝜎𝑤̃ C 𝜎I
where K is a constant.
410 Chapter 7. Single-carrier modulation

We note that the PR system employing MLSD at the receiver achieves a performance similar to that
of a system transmitting antipodal signals, as MLSD exploits the correlation between symbols of the
sequence {a(t)
k
}.

Example 7.C.6 (Equivalent channel g of the type 1 + D)

In this example, it is the channel itself that forms a duobinary signal.
(D) Antipodal signals: Transmitting ak ∈ {−1, 1}, the received signal is given by

yk = ak + ak−1 + 𝑤̃ D,k (7.665)

where 𝜎𝑤2̃ = 2𝜎I2 .

D
An attempt of pre-equalizing the signal at the transmitter by inserting a filter l(D) = 1∕(1 + D) =
1 − D + D2 + · · · would yield symbols a(t)
k
with unbounded amplitude; therefore, such configuration
cannot be used. Equalization at the receiver using the scheme of Figure 7.78a would require a filter of
the type 1∕(1 + D), which would lead to unbounded noise enhancement.
Therefore, we resort to the scheme of Figure 7.78c, whose detector thresholds is set at ±1. To avoid
(p)
error propagation, we precode the message and transmit the sequence {ak } instead of {ak }. At the
receiver, we have
(p) (p)
yk = ak + ak−1 + 𝑤̃ D,k (7.666)

We are therefore in the same conditions as in case (B), and

( )
1
Pbit = 2Q √ (7.667)
2 𝜎I
(E) MLSD receiver: To detect the sequence of information bits from the received signal (7.665), MLSD
can be adopted. Pbit is in this case given by (7.664).

Appendix 7.D Implementation of a QAM transmitter

Three structures, which differ by the position of the digital-to-analog converter (DAC), may be con-
sidered for the implementation of a QAM transmitter. In Figure 7.81, the modulator employs for both
in-phase and quadrature signals a DAC after the interpolator filter hTx , followed by an analog mixer that
shifts the signal to passband. This scheme works if the sampling frequency 1∕Tc is much greater than
twice the bandwidth B of hTx .

ak,I hTx DAC

Tc
ak ~ cos(2π f0t)
bl s(t)
BMAP π/2 +
T PB
–sin(2πf0t)
ak,Q hTx DAC
Tc
frequency upconversion

Figure 7.81 QAM with analog mixer.

7.D. Implementation of a QAM transmitter 411

ak,I hTx
Tc
ak ~ cos(2π f1nTc)
bl frequency s(t)
BMAP + DAC
π/2 upconversion
PB
Tb T Tc
–sin(2πf1nTc)

ak,Q hTx
Tc
IF digital mixer

Figure 7.82 QAM with digital and analog mixers.

h(0)
T

ak,I h(1)
T
T P/S
T
Tc =
8

h(7)
T

Figure 7.83 Polyphase implementation of the filter hTx for Tc = T∕8.

For applications where the symbol rate is very high, the DAC is placed right after the bit mapper and
filters are implemented in the analog domain (see Chapter 19).
In the implementation illustrated in Figure 7.82, the DAC is placed instead at an intermediate stage
with respect to the case of Figure 7.81. Samples are premodulated by a digital mixer to an intermediate
frequency f1 , interpolated by the DAC and subsequently remodulated by a second analog mixer that
shifts the signal to the desired band. The intermediate frequency f1 must be greater than the bandwidth
B and smaller than 1∕(2Tc ) − B, thus avoiding overlap among spectral components. We observe that this
scheme requires only one DAC, but the sampling frequency must be at least double as compared to the
previous scheme.
For the first implementation, as the system is typically oversampled with a sampling interval
Tc = T∕4 or Tc = T∕8, the frequency response of the DAC, I (f ), may be considered as a constant
in the passband of both the in-phase and quadrature signals. For the second implementation, unless
f1 ≪ 1∕Tc , the distortion introduced by the DAC should be considered and equalized by one of these
methods:
● including the compensation for I (f ) in the frequency response of the filter hTx ,
● inserting a digital filter before the DAC,
● inserting an analog filter after the DAC.
We recall that an efficient implementation of interpolator filters hTx is obtained by the polyphase
representation, as shown in Figure 7.83 for Tc = T∕8, where
( )
T
h(𝓁) (m) = hTx mT + 𝓁 , 𝓁 = 0, 1, … , 7, m = −∞, … , +∞ (7.668)
8
412 Chapter 7. Single-carrier modulation

To implement the scheme of Figure 7.83, once the impulse response is known, it may be convenient
to pre-compute the possible values of the filter output and store them in a table or RAM. The symbols
{ak,I } are then used as pointers for the table itself. The same approach may be followed to generate the
values of the signals cos(2𝜋f1 nTc ) and sin(2𝜋f1 nTc ) in Figure 7.82, using an additional table and the
index n as a cyclic pointer.
Chapter 8

Multicarrier modulation

For channels that exhibit high signal attenuation at frequencies within the passband, a valid alternative to
single carrier (SC) is represented by a modulation technique based on filter banks, known as multicarrier
(MC) modulation. As the term implies, MC modulation is obtained in principle by modulating several
carriers in parallel using blocks of symbols, therefore using a symbol period that is typically much longer
than the symbol period of an SC system transmitting at the same bit rate. The resulting narrowband
signals around the frequencies of the carriers are then added and transmitted over the channel. The
narrowband signals are usually referred to as subchannel (or subcarrier) signals.
An advantage of MC modulation with respect to SC is represented by the lower complexity required
for equalization, that under certain conditions can be performed by a single coefficient per subchannel.
A long symbol period also yields a greater immunity of an MC system to impulse noise; however the
symbol duration, and hence the number of subchannels, is limited for transmissions over time-variant
channels. As we will see in this chapter, another important advantage is the efficient implementation of
modulator and demodulator, obtained by sophisticated signal processing algorithms.

8.1 MC systems

In MC systems, blocks of  symbols are transmitted in parallel over  subchannels, using  modula-
tion filters [1] with frequency responses i (f ), i = 0, … ,  − 1. We consider the discrete-time baseband
equivalent MC system illustrated in Figure 8.1, where for now the channel has been omitted and the
received signal rn coincides with the transmitted signal sn . The  input symbols at the k-th modulation
interval are represented by the vector
[ ]T
ak = ak [0], … , ak [ − 1] (8.1)
where ak [i] ∈ [i], i = 0, … ,  − 1. The alphabets [i], i = 0, … ,  − 1, correspond to two dimen-
sional constellations, which are not necessarily identical.
The symbol rate of each subchannel is equal to 1∕T and corresponds to the rate of the input symbol
vectors {ak }: 1∕T is also called MC modulation rate. The sampling period of each input sequence is
changed from T to T∕ by upsampling, that is by inserting  − 1 zeros between consecutive sym-
bols (see Appendix 1.A); each sequence thus obtained is then filtered by a filter properly allocated in

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
414 Chapter 8. Multicarrier modulation

ak[0] yk[0]
M H0(z) G0(z) M
T T T T
M M

ak[1] yk[1]
M H1(z) G1(z) M
T T T T
M sn rn M
T •
• M •
• •
•
ak[M–1] yk[M–1]
M HM–1(z) GM–1(z) M
T T T T
M M

Figure 8.1 Block diagram of an MC system, where the channel has been omitted.

frequency. The filter on the i-th subchannel is characterized by the impulse response {hn [i]}, with transfer
function
∑+∞
Hi (z) = hn [i] z−n (8.2)
n=−∞

and frequency response i ( f ) = Hi (z)|z=ej2𝜋fT∕ , i = 0, … ,  − 1. The transmit filters, also called inter-
polator filters, work in parallel at the rate ∕T, called transmission rate. The transmitted signal sn is
given by the sum of the filter output signals, that is
−1 +∞
∑ ∑
sn = ak [i] hn−k [i] (8.3)
i=0 k=−∞

The received signal rn = sn is filtered by  filters in parallel at the transmission rate. The receive filters
∑
have impulse responses {gn [i]}, i = 0, … ,  − 1, and transfer functions given by Gi (z) = n gn [i]z−n ,
i = 0, … ,  − 1. By downsampling the  output signals of the receive filters at the modulation rate
1∕T, we obtain the vector sequence {yk = [yk [0], … , yk [ − 1]]T }, which is used to detect the vector
sequence {ak }; receive filters are also known as decimator filters.
We consider as transmit filters finite-impulse-response (FIR) causal filters having length 𝛾, and
support {0, 1, … , 𝛾 − 1}, that is hn [i] may be non-zero for n = 0, … , 𝛾 − 1. We assume matched
receive filters; as we are dealing with causal filters, we consider the expression of the matched filters
{gn [i]}, for i = 0, … ,  − 1, given by

gn [i] = h∗𝛾−n [i], ∀n (8.4)

We observe that the support of {gn [i]} is {1, 2, … , 𝛾}.

8.2 Orthogonality conditions

If the transmit and receive filter banks are designed such that certain orthogonality conditions are sat-
isfied, the subchannel output signals are delayed versions of the transmitted symbol sequences at the
corresponding subchannel inputs. The orthogonality conditions, also called perfect reconstruction con-
ditions, can be interpreted as a more general form of the Nyquist criterion.
8.2. Orthogonality conditions 415

Time domain
With reference to the general scheme of Figure 8.1, at the output of the j-th receive subchannel, before
downsampling, the impulse response relative to the i-th input is given by
𝛾−1 𝛾−1
∑ ∑
hp [i] gn−p [ j] = hp [i] h∗𝛾+p−n [j], ∀n (8.5)
p=0 p=0

We note that, for j = i, the peak of the sequence given by (8.5) is obtained for n = 𝛾. Observing (8.5),
transmission in the absence of intersymbol interference (ISI) over a subchannel, as well as absence of
interchannel interference (ICI) between subchannels, is achieved if orthogonality conditions are satis-
fied, that in the time domain are expressed as
𝛾−1
∑
hp [i] h∗p+(𝛾−k) [j] = 𝛿i−j 𝛿k−𝛾 , i, j = 0, … ,  − 1 (8.6)
p=0

Hence, in the ideal channel case considered here, the vector sequence at the output of the decimator
filter bank is a replica of the transmitted vector sequence with a delay of 𝛾 modulation intervals, that is
{yk } = {ak−𝛾 }. Sometimes the elements of a set of orthogonal impulse responses that satisfy (8.6) are
called wavelets.

Frequency domain
In the frequency domain, the conditions (8.6) are expressed as
−1 ( ) ( )
∑ 𝓁 𝓁
i f − j∗ f − = 𝛿i−j , i, j = 0, … ,  − 1 (8.7)
𝓁=0
T T

z-Transform domain
Setting
[ ]T
∑ ∑
a(z) = ak [0]z , … ,
−k
ak [ − 1]z −k
(8.8)
k k

and [ ]T
∑ ∑
y(z) = yk [0]z , … ,
−k
yk [ − 1]z −k
(8.9)
k k

the input–output relation is given by (see Figure 8.2),

y(z) = S(z) a(z) (8.10)
where the element [S(z)]p,q of the matrix S(z) is the transfer function between the q-th input and the
p-th output. We note that the orthogonality conditions, expressed in the time domain by (8.6), are
satisfied if
S(z) = z−𝛾 I (8.11)
where I is the  ×  identity matrix.
416 Chapter 8. Multicarrier modulation

ak[0] yk[0]
T T

ak[1] yk[1]
T T
S(z)

ak[ – 1] yk[ – 1]
T T

Figure 8.2 Equivalent MC system with input–output relation expressed in terms of the matrix S(z).

8.3 Efficient implementation of MC systems

For large values of , a direct implementation of an MC system as shown in Figure 8.1 would require an
exceedingly large computational complexity, as all filtering operations are performed at the high ∕T
rate. A reduction in the number of operations per unit of time is obtained by resorting to the polyphase
representation of the various filters [2].

MC implementation employing matched filters

Using the polyphase representation of both transmit and receive filter banks introduced in Appendix 1.A,
we have (see (1.562))
−1
∑
Hi (z) = z−𝓁 Ei(𝓁) (z ), i = 0, … ,  − 1 (8.12)
𝓁=0

where Ei(𝓁) (z)

is the transfer function of the 𝓁-th polyphase component of Hi (z). Defined the vector of
the transfer functions of transmit filters as

h(z) = [H0 (z), … , H−1 (z)]T (8.13)

and E(z) the matrix of the transfer functions of the polyphase components of the transmit filters, given by
⎡ E0(0) (z) E1(0) (z) · · · E−1(0)
(z) ⎤
⎢ (1) (1) (1) ⎥
E (z) E (z) · · · E (z) ⎥
E(z) = ⎢ 0 1 −1 (8.14)
⎢ ⋮ ⋮ ⋮ ⎥
⎢ (−1) (−1) (−1) ⎥
⎣ E0 (z) E1 (z) · · · E−1 (z) ⎦

hT (z) can be expressed as

hT (z) = [1, z−1 , … , z−(−1) ]E(z ) (8.15)

Observing (8.15), the  interpolated input sequences are filtered by filters whose transfer functions
are represented in terms of the polyphase components, expressed as columns of the matrix E(z ).
Because the transmitted signal {sn } is given by the sum of the filter outputs, the operations of the transmit
filter bank are illustrated in Figure 8.3, where the signal {sn } is obtained from a delay line that collects
the outputs of the  vectors of filters with transfer functions given by the rows of E(z ), and input
given by the vector of interpolated input symbols.
8.3. Efficient implementation of MC systems 417

ak[0] sn yk[0]
 T T

T T
 
z–1 z–1
ak[1] yk[1]
 T T

T T
 –1 –1 
z z
•
E(z) z–(γ–1)EH 1

z* •
• • •
•
• • •
•
• •
ak[ – 1] rn yk[ – 1]
 
T T T T
 

Figure 8.3 Implementation of an MC system using the polyphase representation.

At the receiver, using a representation equivalent to (1.564), that is obtained by a permutation of the
polyphase components, we get
−1
∑
Gi (z) = z−(−1−𝓁) R(𝓁)
i
(z ), i = 0, … ,  − 1 (8.16)
𝓁=0

where R(𝓁)
i
(z) is the transfer function of the ( − 1 − 𝓁)-th polyphase component of Gi (z). Then, the
vector of the transfer functions of the receive filters g(z) = [G0 (z), … , G−1 (z)]T can be expressed as
g(z) = R(z )[z−(−1) , … , z−1 , 1]T (8.17)
where R(z) is the matrix of the transfer functions of the polyphase components of the receive filters, and
is given by
(0) (1) (−1)
⎡ R0 (z) R0 (z) · · · R0 (z) ⎤
⎢ (0) (1) (−1) ⎥
⎢ R1 (z) R1 (z) · · · R1 (z) ⎥
R(z) = ⎢ ⎥ (8.18)
⎢ ⋮ ⋮ ⋮ ⎥
⎢ (0) (1) (−1) ⎥
⎣ R−1 (z) R−1 (z) · · · R−1 (z) ⎦
Observing (8.17), the  signals at the output of the receive filter bank before downsampling are
obtained by filtering in parallel the received signal by filters whose transfer functions are represented in
terms of the polyphase components, expressed as rows of the matrix R(z ). Therefore, the  output
signals are equivalently obtained by filtering the vector of received signal samples [rn−(−1) , rn−+2 ,
… , rn ]T by the  vectors of filters with transfer functions given by the rows of the matrix R(z ).
In particular, recalling that the receive filters have impulse responses given by (8.4), we obtain
( )
1
Gi (z) = z−𝛾 Hi∗ ∗ , i = 0, 1, … ,  − 1 (8.19)
z
Substituting (8.19) in (8.17), and using (8.15), the expression of the vector g(z) of the transfer functions
of the receive filters becomes
( )
g(z) = z−𝛾 hH z1∗
( ) (8.20)
1
= z−𝛾+−1 EH z∗ [z−(−1) , … , z−1 , 1]T

Apart from a delay term z−1 , the operations of the receive filter bank are illustrated by the scheme of
Figure 8.3.
418 Chapter 8. Multicarrier modulation

ak[0] yk[0]
T T

ak[1] yk[1]
T T
sn = rn
E(z) P/S S/P z–(γ–1)EH 1
• T
z* •
•  •
• •

ak[ – 1] yk[ – 1]
T T

Figure 8.4 Equivalent MC system implementation.

(a) (b)
 
T T T T
 z–1 –1
z 

 
T T T T
 z–1 –1
z 

 T T 
T T T
  

Figure 8.5 Block diagrams of: (a) parallel to serial converter, (b) serial to parallel converter.

From Figure 8.3 using (8.20), and applying the noble identities given in Figure 1.60, we obtain the
system represented in Figure 8.4, where parallel to serial (P/S) and serial to parallel (S/P) converters are
illustrated in Figure 8.5a,b, respectively.

Orthogonality conditions in terms of the polyphase components

We observe that the input–output relation of the cascade of the P/S converter and the S/P converter is
expressed by the matrix z−1 I, that means that any input is obtained at the output on the same branch with
a delay equal to T: therefore, the system of Figure 8.4 is equivalent to that of Figure 8.2, with matrix
transfer function given by
( ) ( )
−(𝛾−1) H 1 −1 −𝛾 H 1
S(z) = z E z I E(z) = z E E(z) (8.21)
z∗ z∗
From (8.11), in terms of the polyphase components of the transmit filters in (8.14), the orthogonality
conditions in the z-transform domain are therefore expressed as
( )
1
EH E(z) = I (8.22)
z∗
Equivalently, using the polyphase representation of the receive filters in (8.18), we find the condition
( )
1
R(z) RH =I (8.23)
z∗
8.3. Efficient implementation of MC systems 419

MC implementation employing a prototype filter

The complexity of an MC system can be further reduced by resorting to uniform filter banks [1, 2]. In
this case, the frequency responses of the various filters are obtained by shifting the frequency response
of a prototype filter around the carrier frequencies, given by
i
, i = 0, … ,  − 1
fi = (8.24)
T
In other words, the spacing in frequency between the subchannels is 1∕T.
To derive an efficient implementation of uniform filter banks we consider the scheme represented in
Figure 8.6a, where H(z) and G(z) are the transfer functions of the transmit and receive prototype filters,
respectively. We consider as transmit prototype filter a causal FIR filter with impulse response {hn },
having length 𝛾 and support {0, 1, … , 𝛾 − 1}; the receive prototype filter is a causal FIR filter
matched to the transmit prototype filter, with impulse response given by

gn = h∗𝛾−n , ∀n (8.25)

With reference to Figure 8.6a, the i-th subchannel signal at the channel input is given by
in ∑
∞
sn [i] = ej2𝜋  ak [i] hn−k (8.26)
k=−∞

ak[0] sn[0] yk[0]

T
 T
H(z) G(z) 
T T
 nT
ej2πf0  nT
e–j2πf0  
ak[1] yk[1]
 H(z) G(z) 
T T T T
 nT sn rn
ej2πf1  nT 
• T e–j2πf1 
• 
•
• •
•
ak[–1] yk[–1]
 H(z) G(z) 
T T sn[–1] T T
 nT nT 
e j2πf–1  e–j2πf–1 
(a)

ak[0] yk[0]
 H(z) G(z) 
T T T T
 
ak[1] yk[1]
 H(zW) G(zW) 
T T T T
 sn rn

T
• •

• •
• •
ak[–1] yk[–1]
 H(zW–1) G(zW–1) 
T T T T
 
(b)

Figure 8.6 Block diagram of an MC system with uniform filter banks: (a) general scheme,
(b) equivalent scheme for fi = i∕T, i = 0, … ,  − 1.
420 Chapter 8. Multicarrier modulation

As
in i(n−k)
ej2𝜋  = ej2𝜋  (8.27)
we obtain
∑
∞
i(n−k) ∑
∞
sn [i] = ak [i] hn−k ej2𝜋  = ak [i] hn−k [i] (8.28)
k=−∞ k=−∞

where
in
hn [i] = hn ej2𝜋  (8.29)
2𝜋
−j 
Recalling the definition W = e , the z-transform of {hn [i]} is expressed as
i
Hi (z) = H(zW ), i = 0, … ,  − 1 (8.30)
Observing (8.28) and (8.30), we obtain the equivalent scheme of Figure 8.6b.
The scheme of Figure 8.6b may be considered as a particular case of the general MC scheme rep-
resented in Figure 8.1; in particular, the transfer functions of the filters can be expressed using the
polyphase representations, which are however simplified with respect to the general case expressed by
(8.14) and (8.18), as we show in the following. Observing (8.26), we express the overall signal sn as
−1
∑ in ∑
+∞
sn = ej2𝜋  hn−k ak [i] (8.31)
i=0 k=−∞

With the change of variables n = m + 𝓁, for m = −∞, … , +∞, and 𝓁 = 0, 1, … ,  − 1, we get

−1
∑ i ∑
+∞
sm+𝓁 = ej2𝜋  (m+𝓁) h(m−k)+𝓁 ak [i] (8.32)
i=0 k=−∞

Observing that ej2𝜋im = 1, setting s(𝓁) (𝓁)

m = sm+𝓁 and hm = hm+𝓁 , and interchanging the order of
summations, we express the 𝓁-th polyphase component of {sn } as
−1
∑
+∞
∑
s(𝓁)
m = h(𝓁)
m−k
−i𝓁
W ak [i] (8.33)
k=−∞ i=0

The sequences {h(𝓁)

m }, 𝓁 = 0, … ,  − 1, denote the polyphase components of the prototype filter
impulse response, with transfer functions given by
𝛾−1
∑
H (𝓁) (z) = h(𝓁)
m z ,
−m
𝓁 = 0, … ,  − 1 (8.34)
m=0

Recalling the definition (1.30) of the discrete Fourier transform (DFT) operator as  ×  matrix,
the inverse discrete Fourier transform (IDFT) of the vector ak is expressed as

 ak = Ak = [Ak [0], … , Ak [ − 1]]

F−1 T
(8.35)
We find that the inner summation in (8.33) yields
−1
1 ∑ −i𝓁
W ak [i] = Ak [𝓁], 𝓁 = 0, 1, … ,  − 1 (8.36)
 i=0
and
∑
∞
∑
∞
s(𝓁)
m =  h(𝓁) A [𝓁] =
m−k k
 h(𝓁)
p Am−p [𝓁] (8.37)
k=−∞ p=−∞

Including the factor  in the definition of the prototype filter impulse response, or in  gain factors that
establish the statistical power levels to be assigned to each subchannel signal, an efficient implementation
8.3. Efficient implementation of MC systems 421

ak[0] Ak[0] s(0)

k
H(0)(z)
T T

ak[1] Ak[1] s(1)

k
H(1)(z)
T T
• IDFT • P/S
sn
• • T
• • 

• •
ak[–1] Ak[–1] s(–1)
k
T
H(–1)(z)
T

r(0)
k 1 yk[0] ak[0]
T z –(γ–1) H(0)* z* T T

r(1)
k 1 yk[1] ak[1]
rn T
z –(γ–1) H(1)* z* T T

S/P • DFT • •
• • •
• • •
•
rk[–1] 1 yk[–1] ak[–1]
T z –(γ–1) H(–1)* z* T T

Figure 8.7 Block diagram of an MC system with efficient implementation.

of a uniform transmit filter bank is given by an IDFT, a polyphase network with  branches, and a P/S
converter, as illustrated in Figure 8.7.
Observing (8.33), the transfer functions of the polyphase components of the transmit filters are
given by
E(z) = diag {H (0) (z), … , H (−1) (z)}F−1
 (8.38)

Therefore, the vector of the transfer functions of the transmit filters is expressed as

hT (z) = [1, z−1 , … , z−(−1) ] diag {H (0) (z ), … , H (−1) (z )}F−1
 (8.39)

We note that we would arrive at the same result by applying the notion of prototype filter with the
condition (8.24) to (8.14).
The vector of the transfer functions of a receive filter bank, which employs a prototype filter with
impulse response given by (8.25), is immediately obtained by applying (8.20), with the matrix of the
transfer functions of the polyphase components given by (8.38). Therefore, we get
{ ( )
∗ 1
g(z) = z−𝛾+−1 F diag H (0) ,…
z∗ 
( )}
∗ 1
… , H (−1) [z−(−1) , … , z−1 , 1]T (8.40)
z∗ 
Hence, an efficient implementation of a uniform receive filter bank, also illustrated in Figure 8.7, is given
by a S/P converter, a polyphase network with  branches, and a DFT; we note that the filter of the i-th
branch at the receiver is matched to the filter of the corresponding branch at the transmitter.
422 Chapter 8. Multicarrier modulation

With reference to Figure 8.6a, it is interesting to derive (8.40) by observing the relation between the
received sequence rn and the output of the i-th subchannel yk [i], given by
∑
+∞
i T
yk [i] = gk−n e−j2𝜋 T n  rn (8.41)
n=−∞

With the change of variables n = m + 𝓁, for m = −∞, … , +∞, and 𝓁 = 0, … ,  − 1, and recalling
the expression of the prototype filter impulse response given by (8.25), we obtain
−1 +∞
∑ ∑ i
yk [i] = h∗(𝛾−k+m)+𝓁 e−j2𝜋  (m+𝓁) rm+𝓁 (8.42)
𝓁=0 m=−∞
i
Observing that e−j2𝜋  m = 1, setting rm(𝓁) = rm+𝓁 , and h(𝓁)∗
m = h∗m+𝓁 , and interchanging the order of
summations, we get
−1
∑ −j 2𝜋 i𝓁 ∑∞
∗
yk [i] = e  h(𝓁)
𝛾+m−k m
r(𝓁) (8.43)
𝓁=0 m=−∞
2𝜋
−j  i𝓁 i𝓁
Using the relation e = W , we finally find the expression
−1
∑ ∑
∞
∗
yk [i] = i𝓁
W h(𝓁) r(𝓁)
𝛾+m−k m
(8.44)
𝓁=0 m=−∞

Provided the orthogonality conditions are satisfied, from the output samples yk [i], i = 0, … ,  − 1,
threshold detectors may be employed to yield the detected symbols â k [i], i = 0, … ,  − 1, with a suit-
able delay.
As illustrated in Figure 8.7, all filtering operations are carried out at the low rate 1∕T. Also note that
in practice the fast Fourier transform (FFT) and the inverse FFT (IFFT) are used in place of the DFT
and the IDFT, respectively, thus further reducing the computational complexity.

8.4 Non-critically sampled filter banks

For the above discussion on the efficient realization of MC systems, we referred to Figure 8.7, where the
number of subchannels, , coincides with the interpolation factor of the modulation filters in the trans-
mit filter bank. These systems are called filter-bank systems with critical sampling or in short critically
sampled filter banks.
We now examine the general case of  modulators where each uses an interpolation filter by a factor
 > : this system is called non-critically sampled. In principle, the schemes of transmit and receive
non-critically sampled filter banks are illustrated in Figure 8.8. As in critically sampled systems, also in
non-critically sampled systems it is advantageous to choose each transmit filter as the frequency-shifted
version of a prototype filter with impulse response {hn }, defined over a discrete-time domain with sam-
pling period Tc = T∕. At the receiver, each filter is the frequency-shifted version of a prototype filter
with impulse response {gn }, also defined over a discrete-time domain with sampling period Tc .
As depicted in Figure 8.9, each subchannel filter has a bandwidth equal to ∕(T), larger than
1∕T. Maintaining a spacing between subchannels of ∕(T), it is easier to avoid spectral overlapping
between subchannels and consequently to avoid ICI. It is also possible to choose {hn }, e.g. as the impulse
response of a square root raised cosine filter, such that, at least for an ideal channel, the orthogonality
conditions are satisfied and ISI is also avoided. We note that this advantage is obtained at the expense of
a larger bandwidth required for the transmission channel, that changes from ∕T for critically sampled
8.4. Non-critically sampled filter banks 423

(a)
ak[0] sn[0]
 H(z)
T TC

e j2 π f0 nTC
ak[1] sn[1]
 H(z)
T TC
sn
e j2 πf1 nTC
TC
•
•
•
ak[–1]
 H(z)
T TC sn[–1]

e j2 πf–1 nTC

(b)
 yk[0]
G(z)
TC 2 T
2
e–j2 πf0 nTC
yk[1]
G(z) 
TC 2 T
rn 2
e–j2 πf1 nTC
TC •
•
•

yk[–1]
G(z) 
TC 2 T
2
e–j2 πf–1 nTC

Figure 8.8 Block diagram of (a) transmitter and (b) receiver in a transmission system employing
non-critically sampled filter banks, with  >  and fi = (i)∕(T) = i∕(Tc ).


T
1
T

  2  f
2T T T T

Figure 8.9 Filter frequency responses in a non-critically sampled system.

systems to ∕T for non-critically sampled systems. Therefore, the system requires an excess bandwidth
given by ( − )∕.
Also for non-critically sampled filter banks, it is possible to obtain an efficient implementation using
the discrete Fourier transform [3, 4]. The transmitted signal is expressed as a function of the input symbol
sequences as
−1
∑ j2𝜋 i n T ∑ ∞
sn = e T  ak [i] hn−k (8.45)
i=0 k=−∞
424 Chapter 8. Multicarrier modulation

or, equivalently,
−1
∑
∞
∑
−in
sn = hn−k ak [i] W (8.46)
k=−∞ i=0
With the change of indices
n = m + 𝓁, m ∈ , 𝓁 = 0, 1, … ,  − 1 (8.47)
(8.46) becomes
−1
∑
∞
∑
−i𝓁
sm+𝓁 = hm−k+𝓁 W ak [i] (8.48)
k=−∞ i=0
Using the definition of the IDFT (8.36), apart from a factor  that can be included in the impulse
response of the filter, and introducing the following polyphase representation of the transmitted signal
s(𝓁)
m = sm+𝓁 (8.49)
we obtain
∑
∞
s(𝓁)
m = hm−k+𝓁 Ak [𝓁] (8.50)
k=−∞

In analogy with (1.488), (8.50) is obtained by interpolation of the sequence {Ak [𝓁]} by a factor ,
followed by decimation by a factor . From (1.496) and (1.497), we introduce the change of indices
⌊ ⌋
m
p= −k (8.51)

and ⌊ ⌋ (m)
m m mod 
Δm = − = (8.52)
  
Using (1.503), it results
∑
+∞
s(𝓁)
m = h(p+Δm )+𝓁 A⌊ m ⌋−p [𝓁]

p=−∞

∑
+∞
= hp+𝓁+(m)mod 
A⌊ m ⌋−p [𝓁]

p=−∞

Letting
h(𝓁)
p,m = hp+𝓁+(m)mod  , p, m ∈ , 𝓁 = 0, 1, … ,  − 1 (8.53)
we get
∑
∞
s(𝓁) h(𝓁) ⌊ ⌋
m = p,m A m −p [𝓁] (8.54)

p=0

The efficient implementation of the transmit filter bank is illustrated in Figure 8.10. We note that the
system is now periodically time-varying, i.e. the impulse response of the filter components cyclically
changes. The  elements of an IDFT output vector are input to  delay lines. Also note that within
a modulation interval of duration T, the samples stored in some of the delay lines are used to produce
more than one sample of the transmitted signal. Therefore, the P/S element used for the realization of
critically sampled filter banks needs to be replaced by a commutator. At instant nTc , the commutator
is linked to the 𝓁 = nmod  -th filtering element. The transmit signal sn is then computed by convolv-
ing the signal samples stored in the 𝓁-th delay line with the nmod  -th polyphase component of the
Tc -spaced-coefficients prototype filter. In other terms, each element of the IDFT output frame is fil-
tered by a periodically time-varying filter with period equal to [l.c.m.(, )]T∕, where l.c.m.(, )
denotes the least common multiple of  and .
8.4. Non-critically sampled filter banks 425

(ℓ)
hm : ℓ-th polyphase component {h(0)
m }
(with respect to ) of prototype filter

{h(–1)
m }
(n mod ) ()
{hm } {hm }

Inverse discrete
Fourier transform {h(–1)
m }

ak[0] 0

ak[1] sn
n mod 
IDFT
T

ak[–1] –1
T

Figure 8.10 Efficient implementation of the transmitter of a system employing non-critically

sampled filter banks; the filter components are periodically time-varying.

Likewise, the non-critically sampled filter bank at the receiver can also be efficiently implemented
using the DFT. In particular, we consider the case of downsampling of the subchannel output signals by
a factor ∕2, which yields samples at each subchannel output at an (over)sampling rate equal to 2∕T.
With reference to Figure 8.8b, we observe that the output sequence of the i-th subchannel is given by
∑
∞
in
yn′ [i] = gn′  −n e−j2𝜋  rn (8.55)
2
n=−∞

where gn = h∗𝛾−n .
With the change of indices

n = m + 𝓁, m ∈ , 𝓁 = 0, 1, … ,  − 1 (8.56)

and letting rm(𝓁) = rm+𝓁 , from (8.55) we get

−1
( ∞ )
∑ ∑ (𝓁) i𝓁
yn′ [i] = gn′  −m−𝓁 rm W (8.57)
2
𝓁=0 m=−∞

We note that in (8.57) the term within parenthesis may be viewed as an interpolation by a factor 
followed by a decimation by a factor ∕2.
Letting
⌊ ′ ⌋
n
q= −m (8.58)
2
and ⌊ ′ ⌋
n′  n (n′ ∕2)mod 
Δn′ = − = (8.59)
2 2 
the terms within parenthesis in (8.57) can be written as
∑
∞
gq+(n′  )mod  −𝓁
r⌊(𝓁)n′ ∕2 ⌋ (8.60)
2

−q
q=−∞

Introducing the  periodically time-varying filters,

g(𝓁)
q,n′
= gq+(n′  )mod  −𝓁
, q, n′ ∈ ℤ, 𝓁 = 0, 1, … ,  − 1 (8.61)
2
426 Chapter 8. Multicarrier modulation

and defining the DFT input samples

∑
∞
u(𝓁)
n′
= g(𝓁)
q,n′
r⌊(𝓁)n′  ⌋ (8.62)
−q
q=−∞ 2

(8.57) becomes
−1
∑
yn′ [i] = u(𝓁)
n′
i𝓁
W (8.63)
𝓁=0
The efficient implementation of the receive filter bank is illustrated in Figure 8.11, where we assume
for the received signal the same sampling rate of ∕T as for the transmitted signal, and a downsampling
factor ∕2, so that the samples at each subchannel output are obtained at a sampling rate equal to 2∕T.
Note that the optimum timing phase for each subchannel can be recovered by using per-subchannel
fractionally spaced equalization, as discussed in Section 7.4.3 for SC modulation. Also note that within
a modulation interval of duration T, more than one sample is stored in some of the delay lines to produce
the DFT input vectors. Therefore, the S/P element used for the realization of critically sampled filter
banks needs to be replaced by a commutator. After the  elements of a DFT input vector are produced,
the commutator is circularly rotated ∕2 steps clockwise from its current position, allowing a set of
∕2 consecutive received samples to be input into the delay lines. The content of each delay line is then
convolved with one of the  polyphase components of the T∕-spaced coefficients receive prototype
filter. A similar structure is obtained if in general a downsampling factor ′ ≤  is considered.

{g(0)
m }
()
gm : -th polyphase component
{g(1)
m }
(with respect to ) of prototype filter

(–1)
{gm }
K –n)mod 
(nʹ—
{gm 2 }
Discrete
Fourier transform
0 ynʹ[0]

ynʹ[1]
rn n mod 
T
DFT

–1 ynʹ[–1]
T T
2 2

Figure 8.11 Efficient implementation of the receiver of a system employing non-critically sampled
filter banks; the filter components are periodically time-varying (see (8.61)).

8.5 Examples of MC systems

We consider two simple examples of critically sampled filter bank modulation systems. For practical
applications, equalization techniques and possibly non-critically sampled filter bank realizations are
required, as will be discussed in the following sections.

OFDM or DMT
In this text, we refer to orthogonal frequency division multiplexing (OFDM) as a particular technique of
MC transmission, which applies to both wired and wireless systems. Other authors use the term discrete
8.5. Examples of MC systems 427

ak[0] Ak[0] r(0)

k yk[0]
T T

ak[1] Ak[1] r(1) yk[1]

k
T T
sn rn
IDFT P/S S/P DFT
• T •

• •
• •

ak[ – 1] Ak[ – 1] r(

k
– 1)
yk[ – 1]
T T

Figure 8.12 Block diagram of an OFDM system with impulse response of the prototype filter given
by a rectangular window of length .

multitone (DMT) modulation for wired transmission systems, whereas they reserve the term OFDM for
wireless systems. The transmit and receive filter banks use a prototype filter with impulse response given
by [5–8] {
1 if 0 ≤ n ≤  − 1
hn = (8.64)
0 otherwise
The impulse responses of the polyphase components of the prototype filter are given by
{h(𝓁)
n } = {𝛿n }, 𝓁 = 0, … ,  − 1 (8.65)
and we can easily verify that the orthogonality conditions (8.6) are satisfied.
As shown in Figure 8.12, because the frequency responses of the polyphase components are constant,
we obtain directly the transmit signal by applying a P/S conversion at the output of the IDFT. Assuming
an ideal channel, at the receiver a S/P converter forms blocks of  samples, with boundaries between
blocks placed so that each block at the output of the IDFT at the transmitter is presented unchanged
at the input of the DFT. At the DFT output, the input blocks of  symbols are reproduced without
distortion with a delay equal to T. We note however that the orthogonality conditions are satisfied only
if the channel is ideal.
From the frequency response of the prototype filter (see (1.26)):
( T
) −1 T
H ej2𝜋f  = e−j2𝜋f 2  sinc (fT) (8.66)

using (8.30) the frequency responses of the individual subchannel filters Hi (z) are obtained.
Figure 8.13 shows the amplitude of the frequency responses of adjacent subchannel filters, obtained for
f ∈ (0, 0.06 
T
) and  = 64. We note that the choice of a rectangular window of length  as impulse
response of the baseband prototype filter leads to a significant overlapping of spectral components of
transmitted signals in adjacent subchannels.

Filtered multitone
In filtered multitone (FMT) systems, the transmit and receive filter banks use a prototype filter with
frequency response given by [3, 4]
{
( ) | 1+e−j2𝜋fT | 1
T | 1+𝜌e−j2𝜋fT | if |f | ≤
H e j2𝜋f 
= | | 2T (8.67)
0 otherwise
428 Chapter 8. Multicarrier modulation

10

Amplitude characteristics (dB)

–10

–20

–30

–40

–50

–60

–70

–80
0 0.01 0.02 0.03 0.04 0.05 0.06
fT/M

Figure 8.13 Amplitude of the frequency responses of adjacent subchannel filters in OFDM for
f ∈ (0, 0.06 
T
) and  = 64. Source: Reproduced with permission from Cherubini et al. [4]. ©2002,
IEEE.

where the parameter 0 ≤ 𝜌 ≤ 1 controls the spectral roll-off of the filter. The frequency response exhibits
spectral nulls at the band edges and, when used as the prototype filter characteristic, leads to trans-
mission free of ICI but with ISI within a subchannel. For 𝜌 → 1, the frequency characteristic of each
subchannel exhibits steep roll-off towards the band edge frequencies. On the other hand, for 𝜌 = 0 the
partial-response class I characteristic is obtained (see (7.634)). In general, it is required that at the output
of each subchannel the ICI is negligible with respect to the noise.

10

0
Amplitude characteristics (dB)

–10

–20

–30

–40

–50

–60

–70

–80
0 0.01 0.02 0.03 0.04 0.05 0.06
fT/M

Figure 8.14 Amplitude of the frequency responses of adjacent subchannel filters in an FMT system
for f ∈ (0, 0.06 
T
), and design parameters  = 64, 𝛾 = 10, and 𝜌 = 0.1. Source: Reproduced with
permission from Cherubini et al. [4]. ©2002, IEEE.
8.6. Analog signal processing requirements in MC systems 429

The amplitude of the frequency responses of subchannel filters obtained with a minimum-phase
prototype FIR filter for f ∈ (0, 0.06 
T
), and design parameters  = 64, 𝛾 = 10, and 𝜌 = 0.1 are illus-
trated in Figure 8.14.

8.6 Analog signal processing requirements in MC systems

In this section, we consider the case of a modulated signal {sn } with a sampling frequency of ∕T that
we obtain at the output of the transmit filter bank and must be converted into an analog signal before
being sent to a power amplifier and then over the transmission channel with impulse response gCh (t),
t ∈ ℝ. First, we consider the design criteria of the transmit filters and then we further elaborate on the
power amplifier requirements.

8.6.1 Analog filter requirements

Two interpretations of {sn } lead to different filter requirements.

Interpolator filter and virtual subchannels

The first interpretation considers {sn } as a sampled signal.
Assuming {sn } is a real signal, we refer to the scheme of Figure 8.15, where the task of the transmit
analog filter gTx , together with the digital-to-analog-converter (DAC) interpolator filter gI , is to attenuate
the spectral components of {sn } at frequencies higher than ∕(2T). Therefore, aliasing is avoided at the
receiver by sampling the signal with rate ∕T at the output of the anti-aliasing filter gRc . We note that the
filter gRc does not equalize the channel, which would be a difficult task, whenever the channel frequency
response presents large attenuations at frequencies within the passband, and is therefore entirely carried
out in the digital domain.

w(t)

sn rn
DAC gTx gCh gRc ADC
T T
 

Figure 8.15 Baseband analog transmission of an MC signal.

With reference to Figure 8.15, to simplify the specifications of the transmit analog filter, it is conve-
nient to interpolate the signal {sn } by a digital filter before the DAC.
To further simplify the transmit filters gI and gTx , it is desirable to introduce a transition band of
{sn } by avoiding transmission on subchannels near the frequency ∕(2T); in other words, we trans-
mit sequences of all zeros, ak [i] = 0, ∀k, for i = (∕2) − NCV , … , (∕2), … , (∕2) + NCV . These
subchannels are generally called virtual channels and their number, 2NCV + 1, may be a non-negligible
percentage of ; this is usually the case in OFDM systems because of the large support of the proto-
type filter in the frequency domain. In FMT systems, choosing NCV = 1 or 2 is sufficient. Typically, a
square root raised cosine filter, with Nyquist frequency equal to ∕(2T), is selected to implement the
interpolator filter and the anti-aliasing filter.
We recall now the conditions to obtain a real-valued transmitted signal {sn }. For OFDM systems with
the efficient implementation illustrated in Figure 8.7, it is sufficient that the coefficients of the prototype
430 Chapter 8. Multicarrier modulation

filter and the samples Ak [i], ∀k, i = 0, … ,  − 1, are real-valued. Observing (8.36), the latter condition
implies that the following Hermitian symmetry conditions must be satisfied
[ ]

ak [0], ak ∈ℝ
2 (8.68)

ak [i] = a∗k [ − i], i = 1, … , −1
2
In this case, the symmetry conditions (8.68) also further reduce the implementation complexity of the
IDFT and DFT.
When {sn } is a complex signal, the scheme of Figure 8.16 is adopted, where the filters gTx and gRc
have the characteristics described above and f0 is the carrier frequency.

(a)
Re[.] DAC gTx gRc ADC
w(t)
T
cos(2πf0t) cos(2π f0t)  rn
sn
π /2 π /2
T T
 –sin(2πf0t) –sin(2π f0t) 
Im[.] DAC gTx gRc ADC
T
 j
(b)
sn sC, n rn
gC
T T
 
wn

Figure 8.16 (a) Passband analog MC transmission scheme; (b) baseband equivalent scheme.

We note that this scheme is analogous to that of a QAM system, with the difference that the transmit
and receive lowpass filters, with impulse responses gTx and gRc , respectively, have different requirements.
The baseband equivalent scheme shown in Figure 8.16b is obtained from the passband scheme by the
method discussed in Chapter 7. Note that gC is the discrete-time equivalent channel obtained by the cas-
cade of analog transmit filter, channel and receive filter. Since transmit and receive filters are in general
designed according to different criteria for SC and MC systems, we use different notations for baseband
equivalent channels, namely, gC for MC systems and h for SC systems. With some approximations, the
two pulses may be assumed identical to simplify simulations. As the receive filters approximate an ideal
lowpass filter with bandwidth ∕(2T), the signal-to-noise ratio Γ at the channel output is assumed equal
to that obtained at the output of the baseband equivalent discrete-time channel, given by
𝙼sC
Γ= (8.69)
N0 ∕T
assuming the noise {𝑤n } complex-valued with PSD N0 .

Modulator filter
The second interpretation considers {sn } as a data signal which must be modulated before transmission
on the channel. Both schemes of Figures 8.15 and 8.16 hold with different requirements for filters:
8.6. Analog signal processing requirements in MC systems 431

gTx should be a square root Nyquist pulse and typically also gRc , although this choice is not optimum.
On the positive side, no virtual subchannel is needed; however, filters with a higher implementation
complexity are required. Moreover, the required channel bandwidth is slightly higher than in the previous
interpretation. Overall, this interpretation is only used in discrete-time system simulations but seldom
in hardware implementations.

8.6.2 Power amplifier requirements

As seen in Section 4.1.3, the high-power amplifier (HPA) used to amplify the analog signal s before
transmission, has a linear envelope transfer function for a limited input range. When the envelope A of
the input signal is outside that range, the output is not a scaled version with a constant factor of the HPA
input, resulting into a signal distortion.
For example, an SC QPSK transmission, and rectangular transmit filter hTx has at the HPA input the
signal
s(t) = ASC cos(2𝜋f0 t + arg ak ) (8.70)
for t ∈ [kT, (k + 1)T], having constant envelope A(t) = A and we can select to work at saturation point
with S = ASC and corresponding output back-off (OBO) OBO = 0 dB. Incidentally, this choice of hTx
requires a very large channel bandwidth.
For an OFDM signal, instead, the envelope changes not only according to the constellation of the
data symbols {ak [i]} but also to the number of subchannel. For example, an OFDM transmission with
data symbols ak [i] still taken from a QPSK constellation, the data block ak = [1 + j, 1 + j, … , 1 + j] has
IDFT Ak = [(1 + j), 0, … , 0], and the transmitted signal (still with a rectangular transmit filter) is
⎧ ( )
𝜋 T
⎪AOFDM cos 2𝜋f0 t + 4 0≤t< 
s(t) = ⎨ T
(8.71)
⎪0 
≤t≤T
⎩
and the first part of s has envelope A(t) = AOFDM , while the remaining OFDM symbol has envelope
A(t) = 0. For the same maximum value of s, the average power of the two configurations yields
1
𝙼Tx,OFDM = 𝙼 (8.72)
 Tx,SC
Hence, the OBO of OFDM would be much higher than that of SC.
In general, in this set up with rectangular interpolator filters, we may scale AM [i] to A′M [i] = AM [i]
with  such that
max|A′k [i]| ≤ S (8.73)
The corresponding transmitted power would be
𝙼Tx,OFDM = E(|A′k [i]|2 ) (8.74)
which, in general, yields a very high OBO. On the other hand, we may wonder what is the probability
of that particular data sequence determining the (minimum) value of  such that (8.73) holds. If this
probability is very low, the corresponding high value of OBO may not be justifiable.
In the literature, the peak to average power ratio (PAPR), defined as
max |Ak [i]|2
PAPR = (8.75)
E(|Ak [i]|2 )
is used as an estimate of the OBO of the HPA and various methods to limit the PAPR have been proposed
in the literature [9]. However, a single parameter is not enough to fully characterize the impact of the
432 Chapter 8. Multicarrier modulation

HPA on OFDM performance. For example, more useful performance measures would be to report the
probability cumulative distribution of s or sTx for a given 𝙼Tx .
We may realize that most analytical approaches on reducing the OBO are indeed based on reduc-
ing the input back-off (IBO) as in (8.75). Evaluation of the OBO requires to introduce the non-linear
transformations of the HPA whose output most of the time can be determined only by simulation.

8.7 Equalization

8.7.1 OFDM equalization

We consider the baseband equivalent channel model shown in Figure 8.16.b, where the channel impulse
response (CIR) has support {0, 1, … , Nc − 1}, with Nc > 1. In this case, the orthogonality conditions
for the OFDM system described in Section 8.2 are no longer satisfied; indeed, the transfer matrix S(z),
defined by (8.10) and evaluated for a non-ideal channel, has in general elements different from a delay
factor along the main diagonal, meaning the presence of ISI for transmission over the individual sub-
channels, and non-zero elements off the main diagonal, meaning the presence of ICI. A simple method
to avoid equalization is based on the concept of circular convolution introduced in Section 1.2, that
expresses a convolution in the time domain as product of finite length vectors in the frequency domain
(see (1.42)). Using the method indicated as Relation 2 on page 10, we extend the block of samples Ak
by repeating Ncp elements with
Ncp ≥ Nc − 1 (8.76)
where the repeated part is denoted as cyclic prefix (CP) and Ncp is called the CP length. In this way, we
obtain the OFDM system illustrated in Figure 8.17.

cyclic prefix • • cyclic prefix

• •
• •

ak[0] Ak[0] rk(0) xk[0]

yk[0]
T’ T’
P/S S/P 0
ak[1] Ak[1] rk(1)
T’ •
sn rn •
IDFT GC(z) DFT •
• T
•  wn
•

ak[–1] Ak[–1] rk(–1) xk[–1]

yk[–1]
T’ T’
–1

Figure 8.17 Block diagram of an OFDM system with CP and frequency-domain scaling on a per
subchannel basis.
For the same channel bandwidth and hence for a given transmission rate ∕T, the IDFT (modula-

tion) must be carried out at the rate T1′ = (+N )T
< T1 . After the modulation, each block of samples
cp
8.7. Equalization 433

is cyclically extended by copying the Ncp samples Ak [ − Ncp ], … , Ak [ − 1] in front of the block,
as shown in Figure 8.17. After the P/S conversion, where the Ncp samples of the CP are the first to be
sent, the Ncp +  samples are transmitted over the channel. At the receiver, blocks of samples of length
Ncp +  are taken; the boundaries between blocks are set so that the last  samples depend only on
the elements of only one cyclically extended block of samples. The first Ncp samples of a block are
discarded.
We now recall the result (1.50). The vector rk of the last  samples of the block received at the k-th
modulation interval is
rk = 𝚵k gC + 𝒘k (8.77)
where gC = [gC,0 , … , gC,Nc −1 , 0, … , 0]T is the -component vector of the CIR extended with  − Nc
zeros, 𝒘k = [𝑤(0)
k
, … , 𝑤(−1)
k
]T is a vector of additive white Gaussian noise (AWGN) samples, and 𝚵k
is an  ×  circulant matrix
⎡ Ak [0] Ak [ − 1] · · · Ak [1] ⎤
⎢ ⎥
Ak [1] Ak [0] · · · Ak [2] ⎥
𝚵k = ⎢ (8.78)
⎢ ⋮ ⋮ ⋮ ⎥
⎢ A [ − 1] A [ − 2] · · · A [0] ⎥
⎣ k k k ⎦
Equation (8.77) is obtained by observing that only the elements of the first Nc columns of the matrix
𝚵k contribute to the convolution that determines the vector rk , as the last  − Nc elements of gC are
equal to zero. The elements of the last  − Nc columns of the matrix 𝚵k are chosen so that the matrix
is circulant, even though they might have been chosen arbitrarily. Moreover, we observe that the matrix
𝚵k , being circulant, satisfies the relation
⎡ ak [0] 0 · · · 0 ⎤
⎢ ⎥
0 a [1] · · · 0
F 𝚵k F−1
 =⎢ k ⎥ = diag{a } (8.79)
⎢ ⋮ ⋮ ⋮ ⎥ k
⎢ 0 ⎥
0 · · · ak [ − 1] ⎦
⎣
Defining the DFT of vector gC as
GC = [C,0 , C,1 , … , C,−1 ]T = DFT{gC } = F gC (8.80)
and using (8.79), we find that the demodulator output is given by
xk = F rk = diag{ak } GC + W k (8.81)
where W k = [Wk [0], … , Wk [ − 1]]T = F 𝒘k is given by the DFT of the vector 𝒘k . Relation (8.81)
is illustrated in Figure 8.18 with
xk [i] = ak [i]C,i + Wk [i], i = 0, 1, … ,  − 1 (8.82)
Recalling the properties of 𝒘k , W k is a vector of independent Gaussian r.v.s. In particular, if 𝑤(𝓁)
k
∼
 (0, 𝜎𝑤2 ) with 𝜎𝑤2 = N0 
T
, then W k [i] ∼  (0, 𝜎 2
W
) with 𝜎 2
W
= 𝜎 2.
𝑤
Before detection, the signal xk (8.81) is multiplied by the diagonal matrix K, whose elements on the
diagonal are given by1
1
i = [K]i,i = , i = 0, 1, … ,  − 1 (8.83)
C,i
From (8.81), the input to the data detectors is given by (see Figure 8.19)
yk = Kxk = ak + W ′k , W ′k = KW k (8.84)
1
To be precise, the operation indicated by (8.83), rather than equalizing the signal, that is received in the absence of ISI,
normalizes the amplitude and adjusts the phase of the desired signal.
434 Chapter 8. Multicarrier modulation

C,0 Wk[0]
ak[0]
xk[0]

•
•
•
C,–1 Wk[–1]
ak[–1]
xk[–1]

Figure 8.18 OFDM discrete-time model.

Wʹk [0]
ak[0] yk[0]

•
•
•
Wʹk [–1]

ak[–1] yk[–1]

Figure 8.19 OFDM relation between transmitted data and signal at detection point.

i.e. ( )
𝜎W
2
yk [i] = ak [i] + Wk′ [i] with Wk′ [i] ∼  0, (8.85)
|C,i |2
We assume that the sequence of input symbol vectors {ak } is a sequence of i.i.d. random vectors.
Equation (8.84) shows that the sequence {ak } can be detected by assuming transmission over  inde-
pendent and orthogonal subchannels in the presence of AWGN.
A drawback of this simple detection scheme is the reduction in the modulation rate by a factor ( +
Ncp )∕. Therefore, it is essential that the length of the CIR is much smaller than the number of sub-
channels, so that the reduction of the modulation rate due to the CP can be considered negligible.
To reduce the length of the CIR one approach is to equalize the channel before demodulation [8, 10,
11]. With reference to Figure 8.16b, a linear equalizer with input rn is used; it is usually chosen as the
FF filter of a decision-feedback equalizer (DFE) that is determined by imposing a prefixed length of the
feedback filter, smaller than the CP length.

8.7.2 FMT equalization

We analyse three schemes.

Per-subchannel fractionally spaced equalization

We consider an FMT system with non-critically sampled transmit and receive filter banks, so that trans-
mission within individual subchannels with non-zero excess bandwidth is achieved, and subchannel
output signals obtained at a sampling rate equal to 2∕T, as discussed in Section 8.4. We recall that
the frequency responses of FMT subchannels are characterized by steep roll-off towards the band-edge
frequencies, where they exhibit near spectral nulls. This suggests that a per-subchannel DFE is used
8.7. Equalization 435

xnʹ[0] ak[0]
c[0]

b[0]

xnʹ[1] ak[1]
c[1]
rn
FMT
T demodulator b[1]


xnʹ[–1] ak[–1]
c[–1]
T T
2
b[–1]

Figure 8.20 Per-subchannel equalization (DFE) for an FMT system with non-critically sampled
filter banks.

to remove ISI. The block diagram of an FMT receiver employing per-subchannel fractionally spaced
equalization is illustrated in Figure 8.20. Over the i-th subchannel, the DFE is designed for an overall
impulse response given by
∑
+∞
∑
Nc −1
ho𝑣erall,n [i] = gn−n1 [i] hn1 −n2 [i] gC,n2 , n∈ℤ (8.86)
n1 =−∞ n2 =0

In the given scheme, the  DFEs depend on the transmission channel. If the transmission channel
is time variant, each DFE must be able to track the channel variations, or it must be recomputed peri-
odically. Error propagation inherent to decision-feedback equalization can be avoided by resorting to
precoding techniques, as discussed in Section 7.7.1.

Per-subchannel T -spaced equalization

We consider now an FMT system with critically sampled filter banks, and subchannel output signals
obtained at the sampling rate of 1∕T. The high level of spectral containment of the transmit filters sug-
gests that, if the number of subchannels is sufficiently high, and the group delay in the passband of the
transmission channel is approximately constant, the frequency response of every subchannel becomes
approximately a constant. In this case, the effect of the transmission channel is that of multiplying every
subchannel signal by a complex value. Therefore, as for OFDM systems with CP, equalization of the
transmission channel can be performed by choosing a suitable constant for every subchannel. We note
however that, whereas for an OFDM system with CP, the model of the transmission channel as a multi-
plicative constant for each subchannel is exact, if CP is longer than CIR, for an FMT system such model
is valid only as an approximation.2 In FMT, the degree of the approximation depends on the dispersion
of the transmission channel and on , the number of subchannels.
Assuming a constant frequency response for transmission over each subchannel, the equalization
scheme is given in Figure 8.21a, where Ki is defined in (8.83), and the DFE is designed to equalize
only the cascade of the transmit and receive filters. Using (8.25) and (8.29), we find that the convolution

2
In fact, while in FMT the channel frequency response is the discrete-time Fourier transform of gC,n , n = 0, 1, … , Nc − 1,
say C (f ), in OFDM we simply use the DFT of {gC,n }, say C,i , i = 0, 1, … ,  − 1, and from (1.28) C,i = C (f )|f = i .
T
436 Chapter 8. Multicarrier modulation

of transmit and receive subchannel filters is independent of the subchannel index. In fact, we get
∑
∞
∑
∞
hn1 [i] gn−n1 [i] = hn1 gn−n1 = heq,n (8.87)
n1 =0 n1 =0

In this case, all DFEs are equal.

Alternative per-subchannel T -spaced equalization

An further simplification is obtained by using the scheme of Figure 8.21b. The idea is that, in the presence
of a transmission channel with flat frequency response for each subchannel, a reconstruction of the signal
is achieved by designing the 𝓁-th polyphase component of the receive prototype filter, g(𝓁) , to equalize
the corresponding polyphase component h(𝓁) of the transmit prototype filter. In general, a DFE scheme
can be used, where the 𝓁-th polyphase components of the receive filters, g(𝓁) FF
and g(𝓁)
FB
, equalize the
corresponding 𝓁-th polyphase component h(𝓁) of the overall subCIR.

r(0) xk[0] ak [0]

(z*1 )
γ–1) (0)*
k
z–( H c
T T T
r(1) 0
rn
T
k
z–(γ–1) (1)*
H ( ) 1
z* b

S/P DFT

xk [ – 1]
r(–1) ak [ – 1]
z–( )H(–1)* 1 ( )
k γ–1
z* c
T T T
–1
b

(a)

(0)
GFB (z)

(0)
GFB (z)

(M–1)
GFB (z)
0 1/0
r(0) xk[0] ak [0]
k (0)
GFF (z)
T T T 1/
1 1
r(1)
k (1)
xk[1] ak [1]
rn GFF (z)
T T T
S/P DFT IDFT

1/–1
xk[ – 1]
r(–1)
k ak [ – 1]
(–1)
GFF (z)
T T T
–1

(b)

Figure 8.21 (a) Equalization scheme for FMT in the case of approximately constant frequency
response for transmission over each subchannel; (b) alternative scheme.
8.9. Channel estimation in OFDM 437

8.8 Orthogonal time frequency space modulation

In OFDM, data symbols are allocated in frequency (subchannel index i), and time (block index k) and
equalization is performed on a block-by-block basis. To capture the time diversity of the channel, the
principle of orthogonal time frequency space (OTFS) modulation [12] is to collect the received signal
across K blocks, and only then recover data. In fact, it may be simpler to work in the DFT-domain of
time (index k), i.e. the Doppler domain with index 𝓁, where the channel is less dispersive. Similarly, the
channel is less dispersive in the DFT domain of frequency (index i), i.e. the delay domain described by
the power delay profile with index m. Here, we should remark that equalization may not be simpler for
a more compact channel.
In OTFS, modulation based on OFDM [13] data symbols are spread by a 2D-DFT in time and fre-
quency before being transmitted by OFDM. In particular, let us collect K data blocks of size  into the
 × K matrix a′ . Data are pre-processed (spread) by the symplectic discrete Fourier transform (SDFT)
to obtain the signals to be transmitted on frequency i at time k
−1 K−1 ( )
∑∑ −j2𝜋 im
− k𝓁
ak [i] = a′m,𝓁 e  K , i = 0, … ,  − 1, k = 0, … , K − 1 (8.88)
m=0 𝓁=0

Note that, apart from an irrelevant sign difference in front of k𝓁, this is simply a 2D-DFT.

OTFS equalization
If {sn } is the OFDM transmitted signal corresponding to data ak [i] given by (8.88), over a linear
time-variant dispersive AWGN channel, the received signal is
∑
Nc −1
rn = gC (n, m)sn−m + 𝑤n (8.89)
m=0

where gC (n, m), m = 0, … , Nc − 1, is the CIR at time n.

At the OFDM receiver, after the S/P converter and the DFT, matrix X = [x0 , … , xK−1 ] is formed.
Note that in OTFS systems the CP may not be used for a higher spectral efficiency. Next, matrix X goes
through the 2D-DFT−1 as the inverse of (8.88). Let {𝓁,m } be the equivalent CIR in the Doppler-delay
domain, as the DFT of {gC (n, m)} with respect to time index n, after a suitable windowing. The recovered
signal can be written as [12]
−1 K−1
∑ ∑
ym,𝓁 = 𝓁′ ,m′ a′m−m′ ,𝓁−𝓁′ + noise (8.90)
m′ =0 𝓁 ′ =0

Although the channel matrix  is banded, detection of data symbols from (8.90) is a formidable task,
as each data symbol is affected by interference from neighbouring (both along Doppler and delay axes)
symbols. Iterative cancellation techniques may be used to recover data.
The advantage of OTFS is that the receiver, besides capturing the time diversity, shows a reduced
variability of the channel across data symbols, i.e. in the Doppler-delay domain. Hence, in OTFS coding
is less important.

8.9 Channel estimation in OFDM

From the system model of Figure 8.18, given the vector xk at the output of the DFT, in order to scale each
component of xk by the DFT of the CIR, we need an estimate of GC . Various methods to estimate GC are
438 Chapter 8. Multicarrier modulation

now given. At first, we assume that the first OFDM symbol for k = 0 is a known vector of  symbols,
a0 [i] = 𝛼[i], i = 0, 1, … ,  − 1, or a0 = 𝜶. In the OFDM literature, a known subchannel symbol is also
denoted as pilot symbol.
For a simplified notation, we introduce the  ×  input data matrix (see also (2.93)):
 = diag[𝛼[0], … , 𝛼[ − 1]] (8.91)
In correspondence to the transmission of 𝜶, the received block x0 is denoted as
o = [o[0], … , o[ − 1]]T (8.92)
̂ is an estimate of G , the channel estimation error vector is defined as
If GC C

̂ −G
ΔGC = G (8.93)
C C

From the decomposition (8.81), we have

o = x0 = GC + W 0
̂ − ΔG + W (8.94)
= GC C 0

and defined
RΔGC = E [ΔG∗C ΔGTC ] (8.95)
we measure the estimate quality by the signal-to-noise-ratio (SNR) (see also (7.548))
E [||W 0 ||2 ] 𝜎W
2
Λn = = (8.96)
𝜎a2 E [||ΔGC ||2 ] 𝜎a2 tr RΔGC
assuming components of W 0 are i.i.d., each with variance 𝜎W 2
.
It will be useful to introduce a parameter related to the data constellation
−1
1 ∑ 1 [ ]
E 1 [ ]
 i=0 |𝛼[i]| 2
|a0 [i]|2 1 (8.97)
𝛽 = = = 𝜎a E
2
1 1 |a0 [i]|2
𝜎a2
𝜎a2

Note that for a constant envelope constellation (PSK) 𝛽 = 1, while for a 16-QAM constellation 𝛽 = 17∕9.
We will also make use of the ratio
𝜎2
𝜂 = 2a (8.98)
𝜎W

Instantaneous estimate or LS method

From (8.94), an estimate of GC is simply given by
o[i]
̂C,LS,i = , i = 0, 1, … ,  − 1 (8.99)
𝛼[i]
or
̂
C,LS =  o
−1
G (8.100)
This estimate is also denoted as least squares (LS) because of the following correspondence. Introduced
the functional
−1
∑
 = ||o − ô ||2 = |o[i] − ô[i]|2 (8.101)
i=0

with
̂
ô =  G (8.102)
C
8.9. Channel estimation in OFDM 439

and defined matrix 𝚽 and vector 𝝑 as

𝚽 =  H  = [|𝛼[0]|2 , … , |𝛼[ − 1]|2 ] (8.103)

𝝑 =  H o = [𝛼 ∗ [0]o[0], … , 𝛼 ∗ [ − 1]o[ − 1]]T (8.104)

the solution which minimizes (8.101), given by 𝚽−1 𝝑, coincides with (8.100).
̂ C,LS are easily derived. From (8.104) and (8.94)
The statistical parameters associated to the estimate G
we have that
𝝑 = 𝚽GC +  H W0 (8.105)
By left multiplying by 𝚽−1 yields
̂
C,LS = GC + 𝚽  W 0 = GC +  W 0
−1 H −1
G (8.106)
and
ΔGC =  −1 W 0 (8.107)
hence
RΔGC = 𝜎W
2
𝚽−1 (8.108)
As
−1
∑ 1
tr [𝚽−1 ] = (8.109)
i=0
|𝛼[i]|2
it is
1
Λn = (8.110)
𝛽
Note that from (8.108)
𝜎W
2
E [|ΔC,LS,i |2 ] = (8.111)
|𝛼[i]|2
In other words, the channel estimate is affected by an error equal to the channel noise scaled by the
symbol value. Note that for a constant envelope constellation Λn = 1.
Two variants of the LS methods are as follows:
1. Transmit pilots at a power level higher than the information symbols. However, this would increase
the transmit signal power spectral density which, on the other hand, must be confined within certain
bounds;
2. Use more pilots in time, say 𝜶 k , k = 0, 1, … , Np − 1, and
Np −1
̂ 1 ∑ ok [i]
C,LS,i = , i = 0, 1, … ,  − 1 (8.112)
Np k=0 𝛼k [i]
It is seen that the variance of estimate is reduced by a factor Np and
Np
Λn = (8.113)
𝛽
However, this solution is seldom used due to its large redundancy.
For an estimate of the channel noise variance, from (8.101) it is

𝜎̂ W
2
= (8.114)

̂
Indeed, this estimate is good only if G C,LS ≃ GC .
440 Chapter 8. Multicarrier modulation

LMMSE
This solution makes use of the channel correlation matrix RGC . From (7.584)
( )T
̂
G = R−1
R o (8.115)
C,LMMSE o oGC

which, from (7.591) becomes

[ ( )−1 ]−1
̂
G = 𝜎 2
R∗
+  H
 H o (8.116)
C,LMMSE W GC

Also, from (8.103) and (8.104)

[ ( )−1 ]−1
̂
GC,LMMSE = 𝜎W RGC
2 ∗
+𝚽 𝝑
(8.117)
= R∗GC [𝜎W
2 ̂
𝚽−1 + R∗GC ]−1 G C,LS

Note that the linear minimum-mean-square-error (LMMSE) solution introduces a weighting of the LS
̂
C,LS which depends on the ratio between 𝜎W and the correlation matrix RGC . In particular
2
components G
if 𝜎W
2 ̂ C,LMMSE = G
(R∗GC )−1 is negligible w.r.t. 𝚽, it turns out that G ̂
C,LS . Moreover, from (7.593)
{[ ( )−1 ]∗ } −1
RΔG = 𝜎W 2
𝜎W2
R∗GC +𝚽 (8.118)

and 1
Λn = {[ ( ]∗ }−1 (8.119)
)−1
2 1
𝜎a  tr 𝜎W RGC
2 ∗
+𝚽
[ ]
1 1
An approximated version of (8.117) replaces in 𝚽−1 with its expectation E
|𝛼[i]|2 |𝛼[i]|2
, then it is
[ ]−1
̂ 𝛽 ̂
G = R∗GC I + RGC G (8.120)
C,LMMSE
𝜂 C,LS

where the  ×  matrix left multiplying G ̂

C,LS is supposed to be known. To simplify the algorithm,
RGC is sometimes evaluated for a uniform power-delay profile for a duration equal to the CP length
(Ncp + 1) and a high value of 𝜂 (e.g. 20 dB). In fact, 𝜎W2
is a parameter to be estimated once the channel
has been estimated.
Overall, we can say that the LMMSE method exploits the correlation in the frequency domain.

The LS estimate with truncated impulse response

In many simulations, it is seen that what makes the LMMSE method significantly outperforms the LS
̂
method is the use of the power delay profile length. So the idea is to take the IDFT of the estimate G C,LS ,
as given by (8.99), to obtain:
ĝ C,LS = IDFT {Ĝ C,LS } (8.121)
If ĝ C,LS = [̂gC,LS,0 , … , ĝ C,LS,−1 ]T , the next step is to keep just the first Ncp + 1 samples to yield an
enhanced channel estimate:
{
ĝ C,LS,n n = 0, 1, … , Ncp
ĝ C,LS-enh,n = (8.122)
0 n = Ncp + 1, … ,  − 1
with DFT
̂
G C,LS-enh = DFT[gC,LS-enh ] (8.123)
Moreover, as gC,LS,n , n = Ncp + 1, … ,  − 1, is just noise, we can estimate the noise variance in time as
−1
∑
𝜎̂𝑤2 =
1
|̂g |2 (8.124)
 − Ncp − 1 n=N +1 C,LS,n
cp
8.9. Channel estimation in OFDM 441

and
𝜎̂W
2
= 𝜎̂ 𝑤2 (8.125)

̂
It is seen that the noise in estimate G
Ncp +1
C,LS-enh is reduced by a fraction 
w.r.t. the LS method (8.99)
and (see (8.110))
 1
Λn = (8.126)
Ncp + 1 𝛽
with (see (8.100))
Ncp + 1 𝜎W
2
E[|ΔC,LS-enh,i |2 ] = (8.127)
 |𝛼[i]|2
For many channel models, it is seen that the enhanced version of the LS method performs as good as
the LMMSE method with a much reduced complexity.

8.9.1 Channel estimate and pilot symbols

We recall that pilots are transmitted symbols which are known at the receiver. They are used for channel
estimation and synchronization (tracking). As they introduce overhead, their number should be kept to
a minimum value. They are typically arranged in fixed patterns over the time-frequency plane.
Pilot patterns depend on the coherence bandwidth and coherence time of the channel. As a practical
rule, the pilot frequency spacing should be smaller than the coherence bandwidth, while the pilot time
spacing should be smaller than the coherence time.
Some pilots arrangements are shown in Figure 8.22. In the block-type (left of Figure 8.22), an entire
OFDM symbol contains pilots. This configuration is repeated after many OFDM symbol periods. This
arrangement is particularly recommended for slow-fading channels.
In the comb-type (right of Figure 8.22) for each OFDM symbol, some channels are dedicated to pilots.
This arrangement is recommended for fast-fading channels. Other arrangements are given in Figure 8.23.
Once the pilot arrangement is chosen, (polynomial) interpolation, usually first in time next in fre-
quency, is carried out to acquire the channel for every subchannel. Two-dimensional (time and fre-
quency) interpolation techniques may also be used.

1 OFDM Symbol 1 OFDM Symbol

T

T
i

i
frequency

frequency

time (kT) time (kT)

Figure 8.22 Pilot symbols arrangements in the time-frequency plane: the black circles are the pilot
symbols, while the white ones are the information.
442 Chapter 8. Multicarrier modulation

1 OFDM Symbol 1 OFDM Symbol

T
i

i
frequency

frequency
time (kT ) time (kT )
1 OFDM Symbol
T
i
frequency

time (kT )

Figure 8.23 Other pilot symbols arrangements in the time-frequency plane.

8.10 Multiuser access schemes

In this section, we introduce two frequency division multiple access (FDMA) schemes that are closely
related to OFDM. One solution directly uses OFDM by allocating subchannels to different users, while
the other scheme, denoted SC-FDMA (or DFT-spread OFDM), merges advantages of SC and MC mod-
ulations.

8.10.1 OFDMA
The orthogonal frequency division multiple access (OFDMA) scheme is a generalization of OFDM to
a multiuser context. With OFDMA, each user is assigned a subset of all subchannels. For example, for
a transmission by U users, let
u ⊂ {0, 1, … ,  − 1}, u = 1, 2, … , U (8.128)
be the subset of subchannels assigned to user u. User u will transmit data symbol on subchannels u ,
while not transmitting anything on other subchannels, i.e. indicating with au,k [i] the data symbol of user
u at OFDM symbol k on subchannel i, we have au,k [i] = 0 for i ∉ u .
Typically, we have u1 ∩ u2 = ∅, for u1 ≠ u2 , i.e. users transmit on different subchannels in order
to avoid interference. Note that the destination of each user transmission may be a single device, such
8.10. Multiuser access schemes 443

as the base station in the uplink of a cellular system, or different devices, including other (typically
non-transmitting) users. A receiver interested in data coming from user u will implement a regular
OFDM receiver, and extract only the signals of subchannels u . Also, a single device (such as a base
station in a downlink transmission to mobile terminals) can transmit to multiple users by OFDMA.
Note that the subchannel allocation can be flexibly modified, even for each OFDM block, thus pro-
viding a very easy resource allocation among the users. Indeed, OFDMA is for example adopted in the
downlink of the fourth and fifth generations of the 3GPP cellular standard.

8.10.2 SC-FDMA or DFT-spread OFDM

The power inefficiency of HPAs due to the large dynamic of Ak [i], as discussed in Section 8.6.2 has lead
to a number of solutions. Among those, we focus here on the SC-FDMA, which aims at reducing the
PAPR, while at the same time reaping the benefits of MC modulation, in particular the flexible resource
allocation of OFDMA.
The scheme of SC-FDMA for a single user is shown in Figure 8.24. As for OFDMA, the  subchan-
nels are assigned to U users according to (8.128). Let 𝜅u = |u |, each block of data for user u, au,k [l],
l = 0, … , 𝜅u − 1, are first pre-processed with a DFT of size 𝜅u , to obtain
𝜅u −1
∑ −j2𝜋 𝜅𝓁i
a′u,k [i] = au,k [𝓁]e u , i = 0, … , 𝜅u − 1 (8.129)
𝓁=0

Then, {a′u,k [i]} are transmitted by an OFDMA scheme using subchannels u out of .
At the receiver, we apply equalization according to (8.84) to obtain
yk [i]
′
xu,k [i] = = a′u,k [i] + Wk′ [i], i = 0, … , 𝜅u − 1 (8.130)
GC,i
where Wk′ [i] is defined in (8.85).

au,k[0] aʹu,k[0] Aʹu,k[0]

DFT
au,k[κu − 1] aʹu,k[κu − 1] sn rn
IDFT .. P/S gC
.
-SIZE +CP
wn
Aʹu,k[κu − 1]

(a)

−1
C,0
yk[0] xʹu,k[0] xu,k[0]

IDFT
rn yk[κu − 1] xʹu,k[κu − 1] xu,k[κu − 1]
–CP DFT
S/P -SIZE
−1
C,κ u −1

(b)

Figure 8.24 SC-FDMA scheme, for user 1: (a) transmitter, (b) receiver.
444 Chapter 8. Multicarrier modulation

To simplify notation in Figure 8.24 and (8.130), we assume u given by the first 𝜅u subchannels of
OFDM. Finally, we select the subchannels u on which an IDFT of size 𝜅u is performed to obtain
𝜅u −1
∑ j2𝜋 𝜅𝓁i
xu,k [𝓁] = ′
xu,k [i]e u = 𝜅u au,k [𝓁] + 𝑤̃ k [𝓁], 𝓁 = 0, … , 𝜅u − 1 (8.131)
i=0

Detection of au,k [𝓁] is based on xu,k [𝓁] after a suitable scaling. Note that while Wk′ [i] is statistically
independent across subchannels, even if with a different variance 𝜎W 2
∕|C,i |2 , 𝑤̃ k [𝓁] turns out to be
correlated across index 𝓁. Its correlation matrix R𝑤̃ has entry (𝓁1 , 𝓁2 ) given by
𝜅u −1
∑ 𝜎W
2
j2𝜋
(𝓁1 −𝓁2 )i
[R𝑤̃ ]𝓁1 ,𝓁2 = E[𝑤̃ k [𝓁1 ]𝑤̃ ∗k [𝓁2 ]] = e 𝜅u (8.132)
i=0
|C,i | 2

As R𝑤̃ is not diagonal, the above receiver is not optimum. In any case, to summarize
𝑤̃ k [𝓁] ∼  (0, 𝜎𝑤2̃ k ) (8.133)
with
𝜅u −1
∑ 1
𝜎𝑤2̃ k = 𝜎W
2
(8.134)
i=0
|C,i |2
Note that for a single user using all the subchannels, i.e. 1 = {0, … ,  − 1}, the DFT precoding
cancels out with the transmit IDFT, and we have an SC transmission with the CP, while at the receiver
we obtain a ZF linear equalizer (see Observation 7.10) implemented in the frequency domain, exploiting
the CP. Hence, performance of this scheme is typically worse than that of OFDM for a frequency
selective channel. With multiple users, we can still interpret DFT precoding as the SC transmission of
each user on part of the whole spectrum. On one hand, SC-FDMA reduces the PAPR, shifting from
an MC to a quasi-SC transmission. On the other hand, clearly SC-FDMA retains the easy allocation of
resources of OFDMA, as assigning subchannels to users simply corresponds to selecting the connection
between the precoding DFT output and the transmit IDFT (and similarly at the receiver), i.e. choosing
sets u . SC-FDMA is a particularly useful solution when power efficiency is a concern, and when cheap
HPAs are used, in order to reduce the transmitter cost. This is the case for example of smartphones in
cellular systems, and indeed both the 4th and the 5th generations of the 3GPP cellular standard adopt
SC-FDMA for the uplink.

8.11 Comparison between MC and SC systems

It can be shown that MC and SC systems achieve the same theoretical performance for transmission
over ideal AWGN channels [14]. In practice, however, MC systems offer some considerable advantages
with respect to SC systems.
● OFDM systems achieve higher spectral efficiency if the channel frequency response exhibits large
attenuations at frequencies within the passband. In fact, the band used for transmission can be varied
by increments equal to the modulation rate 1∕T, and optimized for each subchannel. Moreover,
if the noise exhibits strong components in certain regions of the spectrum, the total band can be
subdivided in two or more subbands.
8.12. Other MC waveforms 445

● OFDM systems guarantee a higher robustness with respect to impulse noise. If the average arrival
rate of the pulses is lower than the modulation rate, the margin against the impulse noise is of the
order of 10 log10  dB.
● For typical values of , OFDM systems achieve the same performance as SC systems with a
complexity that can be lower.
● In multiple-access systems, the finer granularity of OFDM systems allows a greater flexibility in
the spectrum allocation. For example, it is possible to allocate subsets of subchannels to different
users, and update the allocation at each OFDM block. The same operations in SC systems would
require, for each user spectrum allocation, different transmit filter, whose design and switch (in
correspondence of the time fractions of blocks in OFDM) is problematic.
On the other hand, OFDM systems present also a few drawbacks with respect to SC systems.
● In OFDM systems, the transmitted signals exhibit a higher PAPR, that contributes to increase the
susceptibility of these systems to non-linear distortion.
● A coding scheme is needed to overcome channels with nulls in the frequency response.
● High sensitivity to carrier frequency offset.
● Because of the block processing of samples, a higher latency is introduced by OFDM systems in
the transmission of information.
However, FMT is significantly more robust than OFDM against frequency errors, because of the small
spectral overlap between adjacent subchannels. Variants of the basic OFDM scheme are numerous [15],
but rarely accepted in recent standards. As an exception, FMT has been adopted by the European Terres-
trial Trunked Radio (TETRA), and is being considered for the VHF Data Exchange System (VDES), a
basic method of data exchange in e-Navigation, an application of Internet of things (IoT) in the maritime
environment [16].
With significant limitations of the HPA in consumer products, SC-FDMA represents an attractive
alternative to OFDM, as it limits PAPR, while providing a simple resource allocation, although with
suboptimal equalization.

8.12 Other MC waveforms

We conclude this section by mentioning some MC waveforms that have attracted attention recently. They
can see as variants of the OFDM system, with some filtering operation operated either in the frequency
or time domain. These solutions have been recently considered for the fifth generation (5G) of mobile
communication systems, but up to release 15 only OFDM and SC-FDMA have been included in the
3GPP standard.
The filterbank multicarrier (FBMC) modulation [17] is obtained as the combination of a filter bank
with offset QAM (OQAM), resulting into 2 sub-carriers, each carrying a real symbol, having inter-
ference limited to two adjacent subchannels.
The generalized frequency division multiplexing (GFDM) [15] can be seen as a FMT system wherein
each subchannel signal is filtered by a circular filter, and transmission includes a CP. It can be also seen
as a linear-precoded OFDM system, where precoding is performed by a Toeplitz matrix.
446 Chapter 8. Multicarrier modulation

The universal filtered MC (UFMC) scheme [18] generalizes the OFDM system by replacing the CP
with the ramp-up and ramp-down of suitably designed filters. Therefore, UFMC can be obtained by
cascading an OFDM system with a transmit filter operating in time and on each OFDM block.

Bibliography

[1] Vaidyanathan, P.P. (1993). Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall.
[2] Bellanger, M.G., Bonnerot, G., and Coudreuse, M. (1976). Digital filtering by polyphase network:
application to sample-rate alteration and filter banks. IEEE Transactions on Acoustics, Speech, and
Signal Processing ASSP-24: 109–114.
[3] Cherubini, G., Eleftheriou, E., Ölçer, S., and Cioffi, J.M. (2000). Filter bank modulation techniques
for very high-speed digital subscriber lines. IEEE Communications Magazine 38: 98–104.
[4] Cherubini, G., Eleftheriou, E., and Ölcer, S. (2002). Filtered multitone modulation for
very-high-speed digital subscriber lines. IEEE Journal on Selected Areas in Communications.
20: (5) 1016–1028.
[5] Weinstein, S.B. and Ebert, P.M. (1971). Data transmission by frequency-division multiplexing using
the discrete Fourier transform. IEEE Transactions on Communications 19: 628–634.
[6] Bingham, J.A.C. (1990). Multicarrier modulation for data transmission: an idea whose time has
come. IEEE Communications Magazine 28: 5–14.
[7] Sari, H., Karam, G., and Jeanclaude, I. (1995). Transmission techniques for digital terrestrial TV
broadcasting. IEEE Communications Magazine 33: 100–109.
[8] Chow, J.S., Tu, J.C., and Cioffi, J.M. (1991). A discrete multitone transceiver system for HDSL
applications. IEEE Journal on Selected Areas in Communications 9: 895–908.
[9] Rahmatallah, Y. and Mohan, S. (2013). Peak-to-average power ratio reduction in OFDM systems: a
survey and taxonomy. IEEE Communications Surveys Tutorials 15: 1567–1592.
[10] Melsa, P.J.W., Younce, R.C., and Rohrs, C.E. (1996). Impulse response shortening for discrete mul-
titone transceivers. IEEE Transactions on Communications 44: 1662–1672.
[11] Baldemair, R. and Frenger, P. (2001). A time-domain equalizer minimizing intersymbol and inter-
carrier interference in DMT systems. Proceedings of GLOBECOM ’01, San Antonio, TX, USA.
[12] Hadani, R., Rakib, S., Tsatsanis, M. et al. (2017). Orthogonal time frequency space modulation.
Proceedings IEEE Wireless Communications and Networking Conference (WCNC), pp. 16.
[13] Farhang, A., RezazadehReyhani, A., Doyle, L.E., and Farhang-Boroujeny, B. (2018). Low complex-
ity modem structure for OFDM-based orthogonal time frequency space modulation. IEEE Wireless
Communications Letters 7: 344–347.
[14] Cioffi, J.M., Dudevoir, G.P., Eyuboglu, M.V., and Forney, G.D. Jr. (1995). MMSE
decision-feedback equalizers and coding. Part I and Part II. IEEE Transactions on Communications
43: 2582–2604.
[15] Michailow, N., Matthé, M., Gaspar, I.S. et al. (2014). Generalized frequency division multiplexing
for 5th generation cellular networks. IEEE Transactions on Communications 62: 3045–3061.
[16] Hu, Q., Jing, X., Xu, L. et al. (2018). Performance specifications of filtered multitone modula-
tion for the roll-off factor and filter order in the maritime VHF data exchange system. 2018 IEEE
International Conference on Smart Internet of Things (SmartIoT), pp. 87–92.
[17] Foch, B.L., Alard, M., and Berrou, C. (1995). Coded orthogonal frequency division multiplex. Pro-
ceedings of the IEEE 83: 982–996.
[18] Vakilian, V., Wild, T., Schaich, F. et al. (2013). Universal-filtered multi-carrier technique for wire-
less systems beyond LTE. 2013 IEEE Globecom Workshops (GC Wkshps), pp. 223–228.
Chapter 9

Transmission over multiple input multiple

output channels

In this chapter, we discuss several transmit/receive techniques over multiple input multiple output
(MIMO) channels. The chapter will focus on single carrier (SC) systems over narrowband (NB) MIMO
channels. More interesting, the same models hold for a subchannel of a multicarrier (MC) system over
a dispersive MIMO channel. As a reference scenario, we will consider wireless links (for which the
MIMO channel model was also introduced in Section 4.1.11), where inputs are given by transmitting
antennas and outputs are given by receiving antennas. Still, MIMO techniques find applications also
in other communication contexts, for example optical fibres and acoustic communications, where
antennas are replaced by optical and acoustic devices, respectively. Moreover, a MIMO channel can
also model a (wireless) MC system affected by inter-carrier interference, wherein inputs and outputs
are subchannels, rather than physical devices.
In the first part of the chapter, we will consider a single transmitter communicating with a single
receiver, where each device is equipped with multiple antennas. Therefore, transmitted (received) signals
can be jointly processed at the transmitter (receiver). In the second part of the chapter, we will consider
instead multiple transmitters (each possibly equipped with multiple antennas) transmitting independent
data signals to multiple receivers (again, each possibly equipped with multiple antennas), wherein trans-
mitted (received) signals can be processed only locally, i.e. at a single transmitter (receiver).

9.1 The MIMO NB channel

For a SC transmit signal, we start by developing a discrete-time model for the MIMO NB channel.
We recall the baseband equivalent model of an SC single input single output (SISO) system given in
Figure 7.10, where we assume that hTx is a square-root Nyquist pulse and gRc is matched to hTx . Moreover,
the channel is non-dispersive, i.e. flat fading, with channel impulse response

gC (𝜏) = (1,1)
C
𝛿(𝜏) (9.1)

where C(1,1) is the channel gain.

If we denote by s(1)
k
the transmitted signal, i.e. the digital signal at rate 1∕T modulated by hTx , and by
(1)
xk the signal at the receiver sampler, their relation is illustrated in Figure 9.1.
Let {ak } be the generated data at the output of the bit mapper, in general a quadrature amplitude
modulation (QAM) data stream. While in Chapter 7 it was s(1) k
= ak , here the relation between transmitted
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
448 Chapter 9. Transmission over multiple input multiple output channels

(1,1) (1)
C 𝑤C (t), 𝑤 ( f )
C
= N0
(1)
t0 + kT
(1) (1) (1)
sk s(1) (t) s C (t) r C (t) xk
hTx gRc
T T

(a)

(1)
C
(1,1)
𝑤k   ( 0, σ 2w = N0/T )
(1) (1)
sk xk
T
(b)

Figure 9.1 (a) Relation between the transmitted digital signal and the discrete-time received signal
for a SISO system and flat fading channel, (b) equivalent discrete-time model of a SISO flat fading
channel.

samples and generated data may be more complex, as it will be seen. For a SISO system, we define the
channel signal-to-noise-ratio (SNR) as (see (6.69))
E [|s(1)
C
(t)|2 ] E [|s(1) (t)|2 ]
Γ= = E [|(1,1)
C
|2 ] (9.2)
N0 T1 N0 T1

where s(1) is the signal at the channel input, and s(1)

C
(t) = (1,1)
C
s(1) (t). As the convolution between hTx and
gRc yields a Nyquist pulse, for a suitable timing phase t0 and assuming filters hTx and gRc with energy T
and 1∕T, respectively (see also (7.106)), it is (see Figure 9.1b)

xk(1) = (1,1)
C
s(1)
k
+ 𝑤(1)
k
(9.3)

with ( )
N
𝑤(1)
k
∼  0, 𝜎𝑤2 = 0 (9.4)
T
We recall from (1.324) the relation
EhTx
𝙼Tx = E [|s(1) (t)|2 ] = E [|s(1)
k
|2 ] (9.5)
T
Hence, for EhTx = T, the transmit statistical power is equal to the statistical power of s(1)
k
,

𝙼Tx = E [|s(1)
k
|2 ] (9.6)

and
𝙼Tx
Γ= E [|(1,1) |2 ] (9.7)
𝜎𝑤2 C

The ratio between 𝙼Tx and 𝜎𝑤2

𝙼Tx
𝜌= with Γ = 𝜌E [|(1,1) |2 ] (9.8)
𝜎𝑤2 C

will be useful. Often it is assumed

E [|(1,1)
C
|2 ] = 1 (9.9)
9.1. The MIMO NB channel 449

i.e. (1,1)
C
models the channel fast fading component, while path loss due to distance and shadowing are
included in 𝜌. Moreover, for a Rayleigh fading channel
(1,1)
C
∼  (0, 1) (9.10)
The extension of Figure 9.1 to a MIMO channel with NTx transmit antennas and NRc receive antennas
is straightforward, as illustrated in Figure 9.2. Here the channel impulse response between the transmit
antenna t and the receive antenna r is given by C(r,t) 𝛿(𝜏). Assuming hTx and gRc as in Figure 9.1, the
discrete-time model of a MIMO system is reported in Figure 9.2b, where
1
𝑤(r)
k
∼  (0, 𝜎𝑤2 ), 𝜎𝑤2 = N0 (9.11)
T

(1)
𝑤C (t), 𝑤 (1) ( f ) = N0
c t0 + kT
(1) (1) (1)
(1,1) sC (t) rC (t) xk
C gRc
T

(1)
sk s(1) (t) (1, NTx)
hTx C
T

(N ,1)
(NTx)
C Rc
(NTx)
sk s (t)
hTx
T
(N )
𝑤C Rc (t), 𝑤 (N ) ( f ) = N0
c Rc
t0 + kT
(N , N ) (N )
s C Rc (t)
(N )
rC Rc (t) (N )
C Rc Tx xk Rc
gRc
T
(a)
(1)
𝑤k
(1)
(1,1) xk
C

(1)
sk (1, NTx)
C
T

(N ,1)
(N )
C Rc
sk Tx
T
(NRc)
𝑤k
(N , NTx) (NRc)
C Rc xk

(b)

Figure 9.2 (a) A MIMO system and flat fading channel, (b) equivalent discrete-time model.
450 Chapter 9. Transmission over multiple input multiple output channels

are uncorrelated across receive antennas, i.e.

(r ) (r )
E[𝑤k 1 (𝑤k 2 )∗ ] = 0 if r2 ≠ r1 (9.12)
Hence, the correlation matrix of
(NRc ) T
𝒘k = [𝑤k(1) , … , 𝑤k ] (9.13)
is
R𝒘 = E[𝒘k 𝒘H
k ] = 𝜎𝑤 I
2
(9.14)
Let
(NTx ) T
sk = [s(1)
k
, … , sk ] , NTx × 1 (9.15)
(1,1) (1,N )
⎡ C (1,2)
C
… C Tx ⎤
⎢ (2,1) (2,2) …
(2,N )
C Tx ⎥
⎢ C C ⎥
C = ⎢ ⋮ ⋮ ⋱ ⋮ ⎥, NRc × NTx (9.16)
⎢ (NRc ,1) (NRc ,2) (NRc ,NTx ) ⎥
… C
⎢ C C ⎥
⎣ ⎦
(NRc ) T
xk = [xk(1) , … , xk ] , NRc × 1 (9.17)
it is
xk = C sk + 𝒘k (9.18)
The transmitted signals across antennas are assumed zero mean and with correlation matrix
Rs = E[sk sH
k
]. The total transmit power is given by
𝙼Tx = tr[Rs ] (9.19)
Definition of 𝜌 is sill given by (9.8).
Concerning the statistical model of channel matrix C , see Chapter 4.
For a MC transmit signal, namely orthogonal frequency division multiplexing (OFDM), and a SISO
configuration, we recall the discrete-time model of Figure 8.17. Its extension to a MIMO configuration
needs some notation update. Here i is the subchannel index, and, for example, s(t) k
[i] is the transmitted
digital signal on antenna t, subchannel i. Similarly to (9.15), (9.16), (9.13), and (9.17), we define sk [i],
C [i], W k [i], and xk [i], respectively. It holds (see (8.80))
xk [i] = C [i]sk [i] + W k [i], i = 0, … ,  − 1 (9.20)
(r,t)
where C [i] is the NRc × NTx MIMO channel matrix for subchannel i. In particular, if n = 0, 1, … , gC,n ,
(r,t)
Nc − 1, is the impulse response between the transmit antenna t and the receive antenna r, it is
{C(r,t) [i], i = 0, … ,  − 1} = DFT{gC,n
(r,t)
} (9.21)
One word of caution is due concerning the noise
(NRc )
W k [i] = [Wk(1) [i], … , Wk [i]]T (9.22)
with
Wk(r) [i] ∼  (0, 𝜎W
2
), 𝜎W
2
= 𝜎𝑤2 (9.23)
where 𝜎𝑤2 is the channel noise variance in the time domain as in (9.3).
Indicating the statistical power across all transmit antennas, subchannel i, as
∑
NTx
𝙼Tx [i] = 𝜎s2(t) [i] , i = 0, 1 … ,  − 1 (9.24)
t=1
9.1. The MIMO NB channel 451

the total transmit power is

−1
∑
𝙼Tx = 𝙼Tx [i] (9.25)
i=0

Depending upon the application, we may use either 𝙼Tx or 𝙼Tx [i], hence (9.8) becomes
𝙼Tx [i] 𝙼Tx
𝜌[i] = or 𝜌= (9.26)
𝜎W
2
𝜎W
2

To simplify the notation, often we drop both the subchannel index i and the time index k, hence (9.18)
and (9.20) merge into the general equation
x = C s + 𝒘 (9.27)
We stress the fact that in OFDM the noise in (9.27) is in the frequency domain. For the rest, all consid-
erations about (9.18) hold also about (9.20).
Figure 9.3 shows a block diagram of (9.27), where the double lines describe vector signals. In this
figure, we also indicate the size of vector signals, a notation that is drop in the following.

s x
C
NTx NRc
𝑤

Figure 9.3 NB MIMO channel model.

Spatial multiplexing and spatial diversity

The transmission of sk is typically associated to NS data streams, a1,k , … , aNS ,k , and this feature goes
under the name of spatial multiplexing. The generic symbol an,k takes values in a QAM constellation
with minimum distance 2. Hence, the power of an,k , 𝜎a2 , depends on the constellation size. Moreover, it
is NS ≤ NS,max , with
NS,max = min{NTx , NRc } (9.28)
This fact will be further analysed in Section 9.2.2. Note that the NS streams can also be received by
multiple users, that turn out to be separated in space (rather than in time or frequency), providing a
space-division multiple access (SDMA) scheme. This use of MIMO will be further discussed starting
from Section 9.6.
We also need to set a bound on the transmit power 𝙼Tx . Here we choose
𝙼Tx ≤ NS 𝜎a2 (9.29)
However, other bounds can be used. This basically corresponds to assigning power 𝜎a2 to each stream or
equivalently
tr[Rs ] ≤ NS 𝜎a2 (9.30)
In a transmission with a single transmit antenna and multiple receive antennas, the signal travels over
NRc channels, and if the distance between receive antennas is large enough and scatterers are numerous,
channels are significantly different, if not statistically independent (see Section 4.1.11). Therefore, by
using more receive antennas, there is a higher probability of having channels with high gains, thus
potentially providing a high SNR (9.8) for some of the receive antennas. Proper operations at the receiver
(such as antenna selection, combining of signals from multiple antennas) yield the benefits of multiple
452 Chapter 9. Transmission over multiple input multiple output channels

channels by providing a high SNR at the detection point. This feature of the MIMO channel goes under
the name of spatial diversity. In particular, with one transmit antenna and multiple receive antennas
(single input multiple output, SIMO), we have receive diversity. Similar observations hold when the same
data symbol is transmitted by multiple antennas, while being received with a single antenna (multiple
input single output, MISO) and we have transmit diversity.
The concepts of multiplexing and diversity find also further explanation in Section 6.5.4, where the
capacity of a MIMO system is discussed.

Interference in MIMO channels

Transmission over the MIMO channel in general requires suitable signal processing techniques at either
or both the transmitter and the receiver. A linear combination of the transmitted samples with complex
coefficients given by matrix C is received by each receive antenna. Therefore, if we simply transmit a
different data symbol over each transmit antenna, the NRc received samples will exhibit severe interfer-
ence. The operations required to avoid interference are tailored to C ; therefore, it is essential to know
the channel state information (CSI) at either the transmitter or the receiver, or at both communication
ends. In the following, we consider various MIMO communication techniques, classified according to
the device where CSI is available.

9.2 CSI only at the receiver

Various approaches can be adopted when CSI is available at the receiver only. Linear operations, which
will be written as multiplications of x by suitably designed matrices, are denoted as combining tech-
niques. More in general, non-linear solutions will be denoted as non-linear detection and decoding. Note
that in the literature combining is also denoted as receive beamforming or simply beamforming, which
however may also refer to transmitter processing (more appropriately denoted as transmit beamform-
ing). In order to avoid ambiguities, we will only use combining referring to linear operations performed
at the receiver. Operations at the transmitter are instead denoted as precoding.

9.2.1 SIMO combiner

The simplest MIMO scheme using combining is SIMO, with only one transmit antenna (NTx = 1) and
multiple receive antennas (NRc > 1). In this case, we will transmit a single data symbol at each time,
thus s = a. Moreover, C becomes an NRc -size column vector and (9.27) is

x = C a + 𝒘 (9.31)

Receive antenna r (r = 1, … , NRc ), sees a scaled and phase-shifted version (by (r,1)
C
) of s.
Applying a combiner means multiplying x by vector CT = [C(1,1) , … , C(1,NRc ) ], obtaining

y = C T x = C T C a + C T 𝒘 (9.32)

Based on y, detection of a follows. Since the transmit signal is in this case a scalar, the combiner output
is again a scalar signal. From (9.32), note that the combiner vector C has effects also on the noise. The
resulting scheme is shown in Figure 9.4.
9.2. CSI only at the receiver 453

s C x y
CT

Figure 9.4 SIMO combiner.

From (6.75), the SNR at detection point (which we recall yields the probability of symbol error) can
be immediately written as1
|CT C |2
𝛾= 1 (9.35)
2
||C|| 2𝜎2
𝑤

Let
2
𝛾MF = (9.36)
𝜎𝑤2
be the SNR at the detection point for an ideal additive white Gaussian noise (AWGN) channel. It is
|CT C |2
𝛾 = 𝛾MF (9.37)
||C||2
As expected, multiplying C by any non-zero constant does not change 𝛾. On the choice of C various
methods are now listed.

1. Selective combining: Only one of the received signals is selected. Let rSC be the antenna corre-
sponding to the received signal with highest power,
rSC = arg max 𝙼x(r) (9.38)
r∈{1,…,NRc }

where the different powers may be estimated using a training sequence. Based on the decision (9.38),
the receiver selects the antenna rSC and consequently extracts the signal x(rSC ) aligned in phase.
With reference to (9.32), this method is equivalent to setting
{ (r,1)∗
(1,r) C r = rSC
C = (9.39)
0 r ≠ rSC
and from (9.32)
(r ,1) (r ,1)∗
y = |C SC |2 a + C SC 𝑤(rSC ) (9.40)
At the detection point, we have
(r ,1)
𝛾SC = 𝛾MF |C SC |2 (9.41)
A variant of this technique provides to select a subset of the signals {x(r) }; next, their combination takes
place.

2. Switched combining: Another antenna is selected only when the statistical power of the received
signal drops below a given threshold. Once a new antenna is selected, the signal is processed as in the
previous case.
1
If A is an N × M matrix, ||A|| denotes its Frobenius norm,
[N M ]1∕2
∑∑
||A|| = |[A]m,n |2
(9.33)
n=1 m=1

which extends the classical Euclidean norm of a vector. Also

||A||2 = tr[AAH ] = tr[A∗ AT ] (9.34)
454 Chapter 9. Transmission over multiple input multiple output channels

3. Equal gain combining (EGC): In this case, the signals are only aligned in phase; therefore, we have
(r,1)
C(1,r) = e−j arg(C ) , r = 1, … , NRc (9.42)

It results ( )
∑
NRc
∑
NRc
(r,1)
y=a |(r,1)
C
| + e−j arg(C )
𝑤(r) (9.43)
r=1 r=1

which yields
(∑ )2
NRc
r=1
|(r,1)
C
|
𝛾EGC = 𝛾MF (9.44)
NRc
This technique is often used in receivers of differential phase-shift-keying (DPSK) signals (see
Section 16.1.1); in this case, the combining is obtained by summing the differentially demodulated
signals on the various receive antennas.

4. Maximal ratio combining (MRC): The maximal ratio combining (MRC) criterion consists in max-
imizing the SNR at the detection point. From (9.35),
|∑N |2
| Rc (1,r) (r,1) |
| C C ||
|
| r=1 |
𝛾 = 𝛾MF | | (9.45)
∑
NRc
|C(1,r) |2
r=1

Using the Schwarz inequality (see (1.353)), (9.45) is maximized for

C(1,r) =  (r,1)∗
C
, r = 1, … , NRc (9.46)

where  is a constant and

∑
NRc
𝛾MRC = 𝛾MF |(r,1)
C
|2 (9.47)
r=1

Introducing the SNR of the r-th branch

𝛾 (r) = 𝛾MF |(r,1)

C
|2 (9.48)

(9.47) can be written as

∑
NRc
𝛾MRC = 𝛾 (r) (9.49)
r=1

that is, the total SNR is the sum of the SNR’s of the individual branches.

5. MMSE optimum combining (OC): The system model (9.31) can be extended with 𝒘 modelling
besides white noise also interference due to other users. If a statistical model of interference is known,
the coefficients {C(1,r) }, r = 1, … , NRc , can be determined by the Wiener formulation, which yields the
minimum mean square error (MMSE) between y, the sample at the decision point, and a, the desired
symbol. Recalling the expression of the autocorrelation matrix

R = E [x∗ xT ] (9.50)

and the crosscorrelation vector

p = E [a x∗ ] (9.51)
9.2. CSI only at the receiver 455

the optimum solution is given by

CTopt = R−1 p (9.52)
In the literature, the OC criterion is known as maximizing the ratio between the power 𝜎a2 of the desired
signal and the power of the residual interference plus noise (signal to interference plus noise ratio, SINR)
at detection point. In fact SINR ∝ 1∕MMSE.
As an alternative, or next to a first solution through the direct method (9.52), the least-mean-square
(LMS) or recursive-least-square (RLS) iterative methods may be used (see Chapter 3), that present the
advantage of tracking the system variations.

Equalization and diversity

Techniques of the previous section can be extended to a transmission over multipath channels using,
instead of a single coefficient C(1,r) per antenna, a filter with N coefficients. Introducing the vector whose
elements are the coefficients of the NRc filters, that is
(1,NRc ) (1,N )
CT = [C0(1,1) , … , CN−1
(1,1)
, C0(1,2) , … , CN−1
(1,2)
, … , C0 , … , CN−1Rc ] (9.53)
and the input vector
(NRc ) (N )
xk = [xk(1) , … , xk−N+1
(1)
, xk(2) , … , xk−N+1
(2)
, … , xk , … , xk−N+1
Rc
]T (9.54)
the solution is of the type (9.52).
Also in this case, a decision feedback equalizer (DFE) structure, or better a maximum-likelihood
sequence detection (MLSD) receiver, yields improved performance at the cost of a greater complexity.
For an in-depth study, we refer to the bibliography [1–18].
We point out that the optimum structure of a combiner/linear equalizer is given by a bank of filters,
each matched to the overall receive pulse at the antenna output, whose outputs are then summed; the
signal thus obtained is equalized by a linear equalizer (LE) or a DFE [19].

9.2.2 MIMO combiner

In the presence of multiple antennas at both the transmitter and the receiver (NTx > 1 and NRc > 1), we
obtain a fully MIMO system. We here assume NRc ≥ NTx and NS = NTx , with s(t) = at , t = 1, … , NTx ,
and
s = a = [a1 , … , aNS ]T (9.55)
For a MIMO system, (9.27) cannot be simplified and a receiver combiner must both collect as much
power as possible for each stream and combat the interference due to the multiple streams transmitted
over the channel. The vector signal obtained after combining is now
y = [y1 , … , yNS ]T = CT x = CT C a + CT 𝒘 (9.56)

where CT is an NS × NRc matrix and each entry of y now corresponds to a different data stream.
We observe that y is affected by noise CT 𝒘, which in general is (i) correlated over the various streams
and (ii) with different power on each stream. Moreover, even after combining, some residual interference
among streams may be present, unless CT C is diagonal. However, to simplify implementation, typically
the correlation among samples in y is neglected and a separate detector is applied on each element of the
combiner output yr , r = 1, … , NS . In other words, the resulting system is a set of NS parallel channels.
The block diagram of the MIMO combiner is shown in Figure 9.5. Note that hard detectors can be
replaced by soft detectors, followed by decoders.
456 Chapter 9. Transmission over multiple input multiple output channels

y1 â1
detector
s=a x
C CT
yNs âNs
𝑤 detector

Figure 9.5 MIMO combiner.

Observation 9.1
Since the cascade of the MIMO channel and combiner can be modelled as a set of parallel channels, we
can allocate the power to each stream in an optimum way, e.g. in order to maximize the achievable rate
of the system. Once we set
s = diag ([1 , … , NS ]T )a (9.57)

values of n , n = 1, … , NS , under the constraint (9.29) are obtained by a applying the waterfilling algo-
rithm (see Section 13.1.2). Note however that this operation requires CSI at the transmitter.

Zero-forcing
A first solution for the choice of combiner CT is denoted zero-forcing (ZF) combiner and aims at nulling
the interference among streams, i.e.
CTZF C = I (9.58)

where I is the NS × NS -size identity matrix. From (9.56), we obtain in this case y = a + CTZF 𝒘.
First note that CTZF is readily obtained as the pseudo-inverse of C (see (2.149)), i.e.

C C ] C
CTZF = [H −1 H
(9.59)

assuming here that the inverse exists, i.e. C is full rank. When the inverse of H
C C does not exist, we
cannot decode NS = NTx streams.
Second, the resulting SNR at the detection point for stream n = 1, … , NS , is
1
𝛾ZF,n = 𝛾MF (9.60)
||[CTZF ]n,⋅ ||2

where [CTZF ]n,⋅ denotes row n of CTZF .

MMSE
The ZF combiner may enhance the noise, by only imposing the strict absence of interference among
streams, with a resulting potentially high SNR at the detection point. A better trade-off between noise and
interference is instead obtained with the MMSE combiner aiming at minimizing the average mean-square
error (MSE) over the streams, i.e. (see (2.124))

J = E[||y − a||2 ] (9.61)

Assuming independent data streams and noise entries, from (9.27) we obtain the following correlation
matrices (see (2.184) and (2.186))
Rx = E[x∗ xT ] = 𝜎a2 ∗C TC + 𝜎𝑤2 I (9.62)
Rxa = E[x a ] =∗ T
𝜎a2 ∗C (9.63)
9.2. CSI only at the receiver 457

The MMSE combiner is (see (2.198))

x Rxa ) = (𝜎a C C + 𝜎𝑤 I) C 𝜎a
2 H 2 −1 H 2
CTMMSE = (R−1 T
(9.64)
where we note that (𝜎a2 H
C C + 𝜎𝑤 I) is always invertible when 𝜎𝑤 ≠ 0. The resulting SNR at detection
2 2

point for stream n = 1, … , NS , is

|[ T ] |2
| CMMSE C n,n |
𝛾MMSE,n = 𝛾MF | [ ] ||2
| (9.65)
|| T
|| CMMSE n,⋅ ||
|| ||
where [X]m,n is the entry (m, n) of matrix X. Note also that as 𝜎𝑤2 → 0, the MMSE combiner converges
to the ZF combiner, i.e. CMMSE → CZF .

9.2.3 MIMO non-linear detection and decoding

Non-linear processing can be applied on x before stream detection. In general, these schemes are more
complex than the combiner, while having the potential to provide better performance, i.e. higher data
rates or lower bit error rate (BER). Here we focus on (i) the vertical Bell labs layered space-time
(V-BLAST) solution [20] and (ii) the spatial modulation.

V-BLAST system
With V-BLAST, we still transmit an independent data symbol on each transmit antenna, according to
(9.55), with NRc ≥ NTx . The basic idea of V-BLAST is to combine the received signals, detect a single
data stream, remove its contribution from x, and then repeat combining and detection for the next data
stream. Combining, detection, and cancellation are iterated NS times, until all data streams are detected.
Let x(n), n = 1, … , NS , be the receive vector of size NS = NTx at iteration n, where for the first iter-
ation we have x(1) = x. A combiner of size 1 × NRc is applied on x(n) to obtain stream n, for example
designed according to the MMSE criterion (9.64), where C is replaced by C (n), the NRc × (NS − n + 1)
sub-matrix of C comprising columns from n to NTx , i.e.
C(r,t) (n) = C(r,n+t−1) , t = 1, … , NTx − n + 1, r = 1, … , NRc (9.66)
T T
and C is replaced by the (NS − n + 1) × NRc matrix C (n). At iteration n, the combiner output is
yn = [CT (n)]1,⋅ x(n) (9.67)
and detection follows, providing ân . Then, the contribution of this and all previously detected data sym-
bols are removed from x, to obtain
x(n + 1) = x(n) − C(⋅,1) (n)ân (9.68)
with
(N ,n)
C(⋅,1) (n) = [C(1,n) , … , C Rc ]T (9.69)
We observe that if â𝓁 = a𝓁 , 𝓁 = 1, … , n, we obtain

⎡an+1 ⎤
x(n + 1) = C (n + 1) ⎢ ⋮ ⎥ + 𝒘 (9.70)
⎢ ⎥
⎣ a NS ⎦
Note that at each iteration the combiner is applied on a channel with smaller dimension and fewer
streams, as previously detected streams are removed. The resulting scheme is shown in Figure 9.6.
458 Chapter 9. Transmission over multiple input multiple output channels

s=a x â1
C [CT (1)]1,· detector

(·,1)
 C (1)

â2
[CT (2)]1,· detector

(·,1)
 C (2)

âNs
[CT (Ns)]1,· detector

Figure 9.6 V-BLAST system.

The order of detection can be modified, as the best performance is obtained by detecting (and remov-
ing) streams with higher SINR first, taking into account the interference of unremoved streams. Note
that the detection of data streams is prone to errors that are propagated in forthcoming iterations by
the cancellation process. In order to mitigate this problem, each data stream, before the bit mapper, can
be encoded with forward-error-correcion (FEC) at the transmitter. FEC decoding is performed on each
detected stream. Next, bit mapping follows, then cancellation. This solution however comes at the cost
of both increased complexity and higher latency, as a block of data must be decoded before the next
stream is detected.

Spatial modulation
Another solution for using the spatial dimension made available by multiple transmit (and receive) anten-
nas is the spatial modulation [21]. In this approach, a data symbol is transmitted with a single antenna
out of the NTx , thus NS = 1. Besides this information carried by the symbol, also the index of the selected
antenna carries an information of log2 NTx [bit]. As NS = 1 in any case, we can also consider the case
NTx ≥ NRc .
For example, assume NTx = 2, NRc = 1, and quadrature phase-shift-keying (QPSK) symbols are trans-
mitted. Given 3 data bits b1 , b2 , and b3 , to be transmitted: b1 is used to select the antenna used for
transmitting the QPSK symbol a, obtained by mapping b2 and b3 . The received signal will be
{
(1,1)
C
a + 𝑤 b1 = 1
x= (9.71)
(1,2)
C
a + 𝑤 b1 = 0
The receiver needs to know the channel with respect to the various antennas (which must be different,
i.e. (1,1)
C
≠ C(1,2) ). The receiver can apply two minimum-distance decisions on x, one assuming channel
9.2. CSI only at the receiver 459

C(1,1) , the other assuming channel (1,2)

C
. Let â|b1 =1 and â|b1 =0 be the (hard) detected symbols for the two
channel cases. Then, the hard decision on bit b1 is obtained as
{
1 |x − (1,1) â|b1 =1 | < |x − (1,2) â|b1 =0 |
̂b1 = C C (9.72)
(1,1) (1,2)
0 |x − C â|b1 =1 | ≥ |x − C â|b1 =0 |

Moreover, if b̂ 1 = 1 the QPSK detected symbol is â = â|b1 =1 , otherwise â = â|b1 =0 .

Spatial modulation has the advantage of avoiding spatial interference, since only one transmit antenna
is used at any given time; this also reduces the complexity of the transmitter, where only one radio
frequency chain is needed. Moreover, the transmission of each QPSK symbol is characterized by a
different SNR, as channel changes according to the selected transmit antenna; on the other hand, a
higher difference between the gain of the two channels |C(1,1) | and |(1,2)
C
| reduces the detection errors
on b1 .
A further extreme version of spatial modulation is space shift keying, where information bits are
carried only by the index of the selected transmit antenna. In this case, with reference to (9.71), a is
a constant and the receiver only needs to detect from which antenna the transmission occurred. While
further simplifying the receiver, this solution further reduces the transmit bit rate.

9.2.4 Space-time coding

The solutions presented until now operate on a single symbol period. We now consider a space modu-
lation scheme operating over multiple symbol periods. Hence, this approach is denoted as space-time
coding, and it still requires CSI at the receiver only. In particular, we focus here on a MISO transmission,
with NTx = 2 and NRc = 1.
First, we extend the channel model (9.27) to include the time index k. If sk = [sk(1) , s(2)
k
]T is the trans-
mitted signal at time k,
xk = C sk + 𝑤k = (1,1)
C
sk(1) + (1,2)
C
s(2)
k
+ 𝑤k (9.73)
at the receiver. Note that we assume that channel vector C is time-invariant.

The Alamouti code

We consider the so-called Alamouti code [22], by the name of its inventor, where
s(1)
2k
= a2k , s(2)
2k
= a2k+1 , s(1)
2k+1
= −a∗2k+1 , s(2)
2k+1
= a∗2k (9.74)
with  a suitable scaling factor such that the power constraint (9.29) is satisfied as we transmit a single
data stream ak with two antennas (note that as NRc = 1 it must be NS ≤ 1). In particular, as sk carries two
independent symbols in two time periods, or, equivalently, NS = 1 symbol per time period, on average
the power constraint (9.29) yields  = √1 and
2

𝙼Tx
𝙼s(1) = 𝙼s(2) = (9.75)
2
with 𝙼Tx = 𝜎a2 .
Note that symbol a2k+1 must be already available for transmission at time k. Note also that the trans-
mitted signal sk is a non-linear transformation of the data signal ak due to the transmission of its complex
conjugate at time 2k + 1. From (9.73) and (9.74), at the receiver we obtain
x2k = (1,1)
C
s(1)
2k
+ C(1,2) s(2)
2k
+ 𝑤2k = (1,1)
C
a2k + C(1,2) a2k+1 + 𝑤2k (9.76a)
x2k+1 = (1,1)
C
s(1)
2k+1
+ (1,2)
C
(2)
s2k+1 + 𝑤2k+1 = −(1,1)
C
a∗2k+1 + (1,2)
C
a∗2k + 𝑤2k+1 (9.76b)
460 Chapter 9. Transmission over multiple input multiple output channels

Now, by performing the following operations on the two received samples

y2k = (1,1)∗
C
x2k + (1,2)
C
∗
x2k+1
(9.77a)
= (|(1,1)
C
|2 + |(1,2)
C
|2 )a2k + (1,1)∗
C
𝑤2k + (1,2)
C
𝑤∗2k+1
y2k+1 = (1,2)∗
C
x2k − (1,1)
C
∗
x2k+1
(9.77b)
= (|(1,1)
C
|2 + |(1,2)
C
|2 )a2k+1 + (1,2)∗
C
𝑤2k − (1,1)
C
𝑤∗2k+1
we obtain equivalent AWGN channels for both symbols a2k and a2k+1 with (a) the same channel gain
(|(1,1)
C
|2 + |(1,2)
C
|2 ) and the same noise power (|C(1,1) |2 + |C(1,2) |2 )𝜎𝑤2 . Moreover, the noise compo-
nents of the two samples in (9.77) are uncorrelated, hence statistically independent, being Gaussian.
As the two samples carry independent data symbols, and noise samples are independent, two minimum
distance detectors can be applied separately on y2k and y2k+1 . The resulting SNR at the detector input is
(from (9.75))
1
𝛾AL = 𝛾MF (|(1,1) |2 + |(1,2) |2 ) (9.78)
2 C C

Note the loss of 3 dB in 𝛾 with respect to the MISO linear precoder (9.109) to be seen in Section 9.3.1,
due to the fact that the channel is known only at the receiver. The Alamouti scheme works only if the
channel is time-invariant for the duration of two consecutive symbols.

Observation 9.2
The time axis can be replaced by the frequency axis of a MC systems; in other words, a2k is transmitted
on a subchannel, and a2k+1 on an adjacent subchannel. In order to obtain expressions similar to those
derived here, the two adjacent subchannels should have the same gain and phase, a condition typically
verified with a small subchannel spacing with respect to the channel coherence bandwidth. The Alam-
outi code is adopted in various standards, including video data for the European Telecommunications
Standards Institute (ETSI) digital video broadcasting standard for terrestrial applications (DVB-T2) [23],
and for the master information block (i.e. the broadcast transmission on cell information used for ini-
tial connection to the network) of the long-term evolution (LTE) of the 3GPP cellular communication
standard [24]. In both cases, the frequency axis is used instead of the time axis.

Observation 9.3
The Alamouti scheme is implementing the maximum-likelihood (ML) detection only for phase-shift-
keying (PSK) constellations, where symbol values have the same amplitude [22].

Observation 9.4
The Alamouti scheme can be easily extended to the MIMO case with multiple receive antennas
(NRc > 1), and still NTx = 2. Indeed, the MIMO channel can be seen as NRc parallel MISO (2 input, 1
output) channels. By applying the Alamouti scheme and decoding each of these channels, we obtain
NRc replicas of the transmitted data without interference. In particular, focusing on a2k and receive
antenna r = 1, … , NRc , (9.77a) can be rewritten as
′
y(r)
2k
= (|(r,1)
C
|2 + |(r,2)
C
|2 )a2k + 𝑤2k(r) (9.79)
′
where 𝑤2k(r) ∼  (0, (|(r,1)
C
|2 + |(r,2)
C
|2 )𝜎𝑤2 ). By normalizing the noise power in y(r)
2k
to 𝜎𝑤2 , we have

(r)
y(r)
2k
y2k = √ , r = 1, … , NRc (9.80)
|(r,1)
C
| 2 + |(r,2) |2
C
9.2. CSI only at the receiver 461

(r)
We thus obtain an equivalent SIMO channel with NRc outputs {y2k }, each with noise power 𝜎𝑤2 , and
channel gain √
 |(r,1)
C
|2 + |(r,2)
C
|2 , r = 1, … , NRc (9.81)
We then apply combiner (9.46) to merge the output of these channels, here a simple sum as the channel
gains (9.81) are real non-negative, providing the following SNR at the detector input
[ 2 N ]
1 ∑ ∑ (r,t) 2
Rc

𝛾AL = 𝛾MF | | (9.82)

2 t=1 r=1 C

The Golden code

The Golden code map two data streams (NS = 2) a1,k and a2,k to the transmitted samples
(1) 𝛼 𝛼
s2k = √ (a1,2k + 𝜃a2,2k ), s(2)
2k
= √ (a1,2k+1 + 𝜃a2,2k+1 )
5 5
(9.83)
(1) j𝛼 (2) 𝛼
s2k+1 = √ (a1,2k+1 + 𝜃a2,2k+1 ), s2k+1 = √ (a1,2k + 𝜃a2,2k )
5 5
√
where 𝜃 = 1+2 5 is the Golden ratio, 𝜃 = 1 − 𝜃, 𝛼 = 1 + j(1 − 𝜃), and 𝛼 = 1 + j(1 − 𝜃). With this code
four data symbols a1,2k , a2,2k , a1,2k+1 , and a2,2k+1 , are transmitted over two symbol periods, using two
antennas. Therefore the number of symbols per transmit time interval T achieved by this code is twice
that of the Alamouti code. The detection of this code requires a more complex ML approach, with respect
to the simple operations of the Alamouti receiver (9.77). The Golden code also provides a smaller error
probability, in particular for symbol constellations with many points.
We conclude this section mentioning diversity techniques with more than one transmit and receive
antenna, called space-time coding techniques, whereby data symbols are coded in both space and time
[25–27].

9.2.5 MIMO channel estimation

Channel estimation for MIMO refers to the problem of estimating C at the receiver, by exploiting the
transmission of symbols already known at the receiver, denoted training or pilot symbols. Since the
channel to be estimated is a NRc × NTx matrix, and at each symbol time we only receive NRc samples,
many symbols must be transmitted in order to estimate the full matrix. Moreover, we will assume that
the channel does not change for the duration of the channel estimation.
(N )
Let sk = [s(1)
k
, … , sk Tx ]T be vector of transmitted symbols at time k, and
(NRc ) T
xk = [xk(1) , … , xk ] = C sk + 𝒘k (9.84)
(NRc ) T
be the vector of received samples at time k, with 𝒘k = [𝑤(1)
k
, … , 𝑤k ] the AWGN vector. Let us also
collect L training symbol vectors into matrix
S = [s0 , … , sL−1 ], NTx × L (9.85)
Collecting the corresponding L received samples and noise vectors into matrices
X = [x0 , … , xL−1 ], W = [𝒘0 , … , 𝒘L−1 ] (9.86)
the input–output relation (9.84) can be rewritten as
X = C S + W, NRc × L (9.87)
462 Chapter 9. Transmission over multiple input multiple output channels

The least squares method

We denote by ̂ C the channel matrix estimate with corresponding output

X̂ = ̂ C S (9.88)

Defined the error matrix

e = X − X̂ (9.89)

the least squares (LS) estimator aims at minimizing the sum of square output errors

∑ ∑
NRc L−1
 = ||e||2 = |xk(r) − x̂ k(r) |2 (9.90)
r=1 k=0

with respect to ̂ C . As {xk(r) } ({̂xk(r) }), k = 0, … , L − 1, depends only on row r of C (̂ C ), the above
problem can be split into NRc problems, where the generic function to be minimized with respect to
[̂ C ]r,⋅ is
∑
L−1
 (r) = |xk(r) − x̂ k(r) |2 (9.91)
k=0

From (9.88), it is
̂ T = ST [̂ ]T
[X] (9.92)
r,⋅ C r,⋅

hence, using the results of Section 7.8.2, the solution of the problem is (7.545)

[̂ C,LS ]Tr,⋅ = (S∗ ST )−1 S∗ [X]Tr,⋅ (9.93)

or
[̂ C,LS ]r,⋅ = [X]r,⋅ SH (SSH )−1 (9.94)

Hence, the whole channel matrix estimate is

̂ C,LS = XSH (SSH )−1 (9.95)

assuming L ≥ NTx and S′ full rank so that the above inverse exists.
By inserting (9.87) into (9.95), it is

̂ C,LS = C + WSH (SSH )−1 (9.96)

Hence,
ΔC = ̂ C,LS − C,LS = WSH (SSH )−1 (9.97)

with corresponding normalized SNR (see (7.548))

𝜎𝑤2
Λn = (9.98)
𝙼s(1) E [‖𝚫C ‖2 ]

For S orthonormal, SSH = I and white noise across antennas and time, it turns out
L
Λn = (9.99)
NTx NRc
9.3. CSI only at the transmitter 463

The LMMSE method

By recalling the notation vec(⋅), (see (4.148)) as
(1,NTx ) (N ,NTx ) T
gC = vec(C ) = [C(1,1) , … , C , (2,1)
C
, … , C Rc ] (9.100)
and similarly x = vec(X), 𝒘 = vec(W), and the (LNRc ) × (NTx NRc ) matrix
S = ST ⊗ I (9.101)
with I the NRc × NRc identity matrix, we have that (9.84) can be written as
x = SgC + 𝒘 (9.102)
The channel estimation problem can be seen as the estimation of channel vector gC , a random vector
with correlation matrix RgC , starting from an observation vector x, a problem discussed in detail
in Section 7.8.2. In particular, assuming 𝒘 ∼  (𝟎, 𝜎𝑤2 I), the linear minimum-mean-square-error
(LMMSE) estimator is given by (7.591) as
ĝ C,LMMSE = (R−1 T
x RxgC ) x
H H (9.103)
[𝜎𝑤2 (R∗gC )−1 + S S]−1 S x

Example 9.2.1
For a MIMO channel with (Gaussian) zero-mean uncorrelated entries RgC = I, identity matrix of size
(NTx NRc ) × (NTx NRc ), it is
H H
ĝ C,LMMSE = [𝜎𝑤2 I + S S]−1 S x (9.104)

Moreover, if S has orthonormal columns, we have

H
S S=I (9.105)
and
1 H
ĝ C,LMMSE = S x (9.106)
1 + 𝜎𝑤
2

Example 9.2.2
Another example of channel correlation matrix is given by (4.152).

9.3 CSI only at the transmitter

Until now we have considered CSI available only at the receiver, and all the burden of handling the
interference introduced by the MIMO channel has been upon the receiver. In this section, we consider
instead that CSI is available at the transmitter only, thus the transmitter must pre-elaborate the data sym-
bols before transmission over the MIMO channel. Also in this case, we will start with linear approaches
and then consider non-linear solutions. In this section, we will assume NTx ≥ NRc and NS = NS,max = NRc .

9.3.1 MISO linear precoding

In a MISO system, multiple transmit antennas (NTx > 1) are employed, while the receiver is equipped
with a single antenna (NRc = 1), and therefore, (9.27) becomes x = C s + 𝑤. At the receiver, a
464 Chapter 9. Transmission over multiple input multiple output channels

a s GC x
B

Figure 9.7 MISO linear precoder.

conventional symbol-by-symbol detector is employed. Now, since a single sample x is received for
each transmit vector s, we can easily conclude that even with NTx > 1 antennas we can transmit a single
stream at any given time, NS = NRc = 1, and the transmit vector can be written as
s = Ba (9.107)
where a is the data symbol of the single stream, while B is an NTx vector of linear precoding, with entries
B(t,1) , t = 1, … , NTx . The resulting scheme is shown in Figure 9.7.
As Rs = 𝜎a2 BBH and NS = 1, from the power constraint (9.30), we obtain a constraint on the precoder,
namely
∑
NTx
||B||2 = |B(t,1) |2 = 1 (9.108)
t=1
The SNR at detection point turns out to be
𝛾 = 𝛾MF |C B|2 (9.109)
The precoder maximizing 𝛾 under the constraint (9.108) can be obtained by the Lagrange multipliers
method, i.e. finding the maximum of
J = ||C B||2 𝜎a2 + 𝜆(||B||2 − 1) (9.110)
where 𝜆 is the Lagrange multiplier to be optimized to satisfy (9.108). By taking the gradient of J with
respect to B and 𝜆, and enforcing (9.108), we obtain2
H
C
B= (9.111)
||C ||
Note that, similar to MRC (9.46), the MISO linear precoder aligns the signals at the receiver by compen-
sating the phase shift introduced by the channel, and transmits more power on channels having a higher
gain |C(1,t) |. Correspondingly, we have
(N )
∑ Tx
(1,t) 2
𝛾LP = 𝛾MF |C | (9.112)
t=1

MISO antenna selection

As illustrated in Figure 9.8, a simple configuration for MISO system is antenna selection, wherein one
antenna is selected for transmission among the NTx available, and all others are switched off. As selection
criterion, we can choose the antenna having the highest channel gain to the receive antenna, i.e.
tAS = arg max |C(1,t) |2 (9.113)
t
At detection point, the SNR is
(1,tAS ) 2
𝛾AS = 𝛾MF |C | (9.114)

2
Similarly to (9.46), the solution (9.111) can be derived from (9.109) by the Schwarz inequality up to a constant which in
turn is determined by the constraint (9.108).
9.3. CSI only at the transmitter 465

(1,1)
C
(1)
s

(1,2)
C 𝑤
(2) (1)
a s x

(1, NTx)
C
(NTx)
s

Figure 9.8 MISO antenna selection or switched transmit diversity.

9.3.2 MIMO linear precoding

For a MIMO system (NTx > 1 and NRc > 1), multiple streams can potentially be transmitted, which
however interfere with each other at the receiver. Similarly to the combining solutions, two approaches
are typically employed in the choice of the linear precoder matrix B, namely ZF and MMSE precoding.
As we have already seen, a MIMO system can transmit at most NS independent data streams (see (9.28)).
With linear precoding, a = [a1 , … , aNS ]T multiplies the precoder NTx × NS matrix B, with entries B(t,n) ,
t = 1, … , NTx , n = 1, … , NS , to yield the transmitted vector

s = Ba (9.115)

The resulting input–output relation is

x = C Ba + 𝒘 (9.116)

As the channel is not known at the receiver, detection of the NS symbols is based independently on
each component of x. Therefore, also in this case suitable power allocation techniques (e.g. using to the
waterfilling algorithm) can be applied across the components of s and under the constraint (see (9.30))
∑ ∑
NS NTx
tr[BBH ] = |B(t,n) |2 = NS (9.117)
n=1 t=1

Note that with precoding, the channel matrix C must be already available at the transmitter for the
design of B. The resulting scheme is similar to Figure 9.7, where now a, 𝒘, and x are vectors.

ZF precoding
Similarly to ZF combining, the ZF precoder aims at removing the interference introduced by the channel,
operating now only at the transmitter. Let us define the pseudo-inverse matrix

B̃ = H
C (C C )
H −1
(9.118)

assuming that matrix C H ̃ ̃

C is invertible. Note that we have C B = I thus B could be the ZF precoder.
However, with this choice constraint, (9.117) may not be satisfied.
466 Chapter 9. Transmission over multiple input multiple output channels

We now investigate a proper power splitting among components of s. To this end, we normalize B̃
such that (9.117) holds, and also 𝙼Tx is equally split among the components of s. We leave the power
allocation among the components of s in the presence of the channel C as a separate problem to be
̃ is
solved next. From (9.115), the transmit power gain of an , due to the precoder B,
𝛽̃n = ||B̃ ⋅,n ||2 (9.119)
Therefore, the normalized ZF beamformer
[√ √ ]T
⎛ NS NS ⎞
BZF = B̃ diag ⎜ ,…, ⎟ (9.120)
⎜ 𝛽̃1 𝛽̃NS ⎟
⎝ ⎠
satisfies (9.117) and at the same time forces the same transmit power for all data symbols. This precoder,
also denoted as diagonal precoder, provides at detection point the set of parallel channels
√
NS
xn = an + 𝑤n , n = 1, … , NS (9.121)
𝛽̃n
Once we replace an with n an , with n , n = 1, … , NS , suitable scaling factors such that
∑
NS
2n = 1 (9.122)
n=1

waterfilling algorithm can be applied (see Section 13.1.2).

A simpler ZF solution is obtained by enforcing the same channel gain for all streams at the receiver.
In this case, we have the regularized zero-forcing (RZF) precoder
√
NS
BRZF = 𝜅 B̃ with 𝜅 = (9.123)
̃
tr(B̃ B)
H

providing xn = 𝜅an + 𝑤n . In this case, however, we use a different transmit power per component of s,
which however has not been optimized, e.g. with waterfilling. The constraint (9.117) is still satisfied.

9.3.3 MIMO non-linear precoding

We may wonder if non-linear precoding is more effective than linear precoding for MIMO systems. First,
consider the LQ factorization of channel matrix C , a variant of the LU factorization [28], wherein
C = LQ (9.124)
where the NTx × NTx matrix Q is unitary (i.e. QH Q = I), and L is a NRc × NTx lower triangular matrix,
i.e. having its upper triangular part with only zeros. Note that the diagonal of L has NS non-zero entries.
Then, by transmitting s = QH s̃ (where s̃ will be detailed next), we obtain
x = L̃s + 𝒘 (9.125)
We now provide details on how to design s̃ = [̃s(1) , … , s̃(NTx ) ]T .
From (9.125), we observe that at the first
receive antenna we have x(1) = L1,1 s̃(1) + 𝑤(1) , without interference from other entries of s̃, and we can
set
s̃(1) = a1 (9.126)
The second entry of x instead is affected by interference from u(2) = L2,1 a1 , as from (9.125)
x(2) = L2,2 s̃(2) + u(2) + 𝑤(2) = L2,2 s̃(2) + L2,1 a1 + 𝑤(2) (9.127)
9.3. CSI only at the transmitter 467

Therefore, by setting
L2,1
s̃(2) = a2 − a1 (9.128)
L2,2
we obtain x(2) = L2,2 a2 + 𝑤(2) , which again is without interference. This approach can be iterated for
the other entries of s̃. Two observations are in place here. First, by a closer look at (9.128), we note
that we are transmitting a linear combination of the data symbols a1 and a2 , thus proceeding with this
approach we would re-discover the linear precoding approaches of the previous section. Second, the
transmit strategy (9.128) does not satisfy the power constraint (9.117).
We now consider instead non-linear approaches where s̃(2) is a non-linear function of both a1 and a2 .
Two solutions are investigated, namely Tomlinson Harashima (TH) precoding, that operates on a symbol
level, which is an extension of the approach encountered for transmission over inter-symbol-interference
(ISI) channels (see Section 7.7.1), and dirty paper precoding, which provides the maximum data rate.
We would like first to underline a key difference between MIMO narrowband channels and SISO
dispersive channels; in MIMO, the interference occurs among different streams, that are transmitted at
the same time, while in dispersive channels interference occurs among symbols transmitted at different
times. Therefore, in MIMO, we can use space-time coding techniques that operate on blocks of symbols
within each stream in order to avoid interference: per-stream precoding will be smartly exploited in the
dirty paper coding (DPC) approach.

Dirty paper coding

DPC has been introduced by Costa [29], who focused on a transmission channel affected by interference,
which is known at the transmitter but not at the receiver. The dirt is present on the paper when an author
is writing, as the interference in the considered channel model.
The theoretical paper by Costa only established that DPC is a technique to achieve the MIMO channel
capacity, without providing its implementation details. Over the years, it turned out that the implemen-
tation of DPC is computationally demanding and the performance promised by theory is reachable at
the cost of encoding long data streams, similarly to what happens for a capacity-achieving transmission
on the SISO AWGN channel.
An implementation of DPC has been proposed in [30] using linear block codes. In particular, consider
the LQ precoded channel model (9.125), with NTx = NRc = NS = 2. Let {b1,𝓁 } and {b2,𝓁 } be the two bit
streams. We now provide details on how to design s̃k .
The first bit stream b1,𝓁 is encoded with any error correcting code with codeword length n0 , and then
bit mapped into K symbols a1 = [a1,1 , … , a1,K ]T . Then, we set

s̃(1)
k
= 1 a1,k (9.129)

where 1 is a suitable power scaling constant. From (9.127), the resulting interference on
s̃ (2) = [̃s(2)
1
, … , s̃(2)
K
]T is
u(2) = [L2,1 1 a1,1 , … , L2,1 1 a1,K ]T (9.130)

Consider now a (k0 , n0 ) block code (see Chapter 11), and for a generic bit stream z of length n0 , let 𝝇(z)
be its (n0 − k0 )-size syndrome (see Section 11.2.1). Data bits of the second stream are not associated to
a codeword, but rather to a syndrome; therefore, we transmit one of the 2n0 −k0 possible messages, each
associated to a different syndrome. Remember that bit sequences of length n0 having the same syndrome
𝝇(z) belong to the same coset (see Section 11.2.1) (𝝇(z)).
The (n0 − k0 ) bits of the second bit stream b2 = [b2,1 , … , b2,n0 −k0 ]T are considered a syndrome, and
from its coset (b2 ), we extract the n0 bit sequence c2 to be mapped into a2 and sent as s̃(2) . In particular,
468 Chapter 9. Transmission over multiple input multiple output channels

the selected sequence c2 is such that its bit-mapped and scaled symbol sequence a2 (c2 ) is closest (in
MSE sense) to the interference u(2) , i.e.
c2 = arg min ||2 a2 (z) − u(2) ||2 (9.131)
z∈(b2 )

where 2 is a suitable power scaling constant. In particular, c2 is obtained by applying a decoder of the
error correcting code on u(2) , constrained to select the decoded codewords from the reduced set (b2 ),
thus denoted reduced set code (RSC) decoder. Lastly,
2 a2,k (c2 ) − u(2)
k
s̃(2)
k
= , k = 1, … , K (9.132)
L2,2

The block diagram of the generation of s̃(2)

k
is shown in Figure 9.9.

b2
Coset map
1/L2,2
(b2) 2
(2)
–uk
(2)
u(2) c2 a2 a2,k (c2) s̃k
RSC Dec. BMAP P/S

Figure 9.9 Example of generation of the second stream for dirty paper coding.

At the receiver, we have

xk(2) = L2,2 s̃(2)
k
+ u(2)
k
+ 𝑤(2)
k
= 2 a2,k + 𝑤k(2) (9.133)
Next, detection and inverse bit mapping follow. Last, we compute the syndrome of the detected bit
sequence, that in absence of noise is b2 .
Note that with this choice, we do not explicitly enforce the power constraint (9.117), which however
can be enforced on average by properly choosing 1 and 2 .
The coding-based solution for DPC can be applied also to other codes than block codes, such as
convolutional, turbo, and low-density-parity-check (LDPC) codes [30].

TH precoding
TH precoding removes interference among streams. This approach is in general suboptimal with respect
to DPC, as it splits the encoding process into two parts: the error correcting coding and the TH precoding.
Still, the TH precoding yields a simple implementation and is effective in reducing interference.
With reference to the channel model (9.125) obtained with LQ precoding, the TH precoder operates
as follows. The first stream is transmitted directly, i.e. s̃(1) = a1 . For the second stream, we can apply
the TH precoder of Section 7.7.1. Recall that, with reference to Figure 7.58, the constellation alphabet
 is extended to cover the entire plane, and ΛT denotes the sublattice of Λ0 . Therefore, in the MIMO
context, a2 is transmitted as
L2,1
s̃(2) = a2 − a +c (9.134)
L2,2 1
where c is a point of the sublattice ΛT chosen so that s̃(2) belongs to the region R() of Figure 7.58.
When comparing with dirty paper, both techniques exploit knowledge of the interference at the trans-
mitter in order to strengthen the message rather than being seen as an obstacle. However, notice that
in TH precoding, the interference is generated by the transmitter itself, while in dirty paper coding, the
known interference is related to a separate stream.
9.4. CSI at both the transmitter and the receiver 469

9.3.4 Channel estimation for CSIT

CSI at the transmitter (CSIT) can be obtained in various ways. First, if the two communicating devices
are operating in half-duplex alternating transmission and reception in time division duplexing (TDD),
the channel reciprocity can be exploited. In other words, if C is the channel from the transmitter to the
receiver, the channel from the receiver (operating as a transmitter) and the transmitter (operating as a
receiver) is TC . Therefore, we first estimate the channel from the receiver to the transmitter, ̂ C and then
T

the transmitter simply takes its transpose to estimate ̂ C .

In frequency division duplexing (FDD) systems, where the transmissions in the two directions occur
in two different bands, channel reciprocity cannot be exploited to obtain CSIT. In Section 5.6, a solution
is described, where the channel matrix C is first estimated at the receiver, it is quantized by vector
quantization, and the quantization index is fed back to the transmitter by a feedback signalling channel.

9.4 CSI at both the transmitter and the receiver

We now consider specific techniques when CSI is available at both the transmitter and the receiver.
In this case, we can apply a linear processing at both the transmitter and the receiver. In particular, let
NS,max = rank C (9.135)
hence NS,max ≤ min(NRc , NTx ), the singular value decomposition (SVD) of the channel matrix yields
(6.114), with 𝚺 = diag([𝜉1 , … , 𝜉NS,max ]T ) the diagonal matrix of non-zero singular values. We introduce
the extended generated data vector
aext = [aT , 𝟎]T (9.136)
of size NTx × 1. We apply the precoder
s = Baext (9.137)
and the combiner CT to yield
̃ ext
yext = CT [C s + 𝒘] = CT C s + 𝒘 (9.138)
In particular, by choosing as combiner and precoder
CT = U H , B=V (9.139)
H
we note that we satisfy the transmit power constraint (9.117), as VV = I. This choice clearly requires
CSI at both the transmitter and the receiver. If 𝒘 is an AWGN vector with i.i.d. entries, 𝒘 ̃ ext has the
same statistics of 𝒘. Denoting by y (𝒘)̃ the first NS,max entries of yext (𝒘ext ) and removing NRc − NS,max
entries, thanks to the unitary properties of matrices U and V, at the detection point we have
̃
y = 𝚺a + 𝒘 (9.140)
Therefore, with these precoder and combiner, we have converted the MIMO channel into a set of parallel
channels (with independent noise terms)
yn = 𝜉n an + 𝑤̃ n (9.141)
𝑤̃ n ∼  (0, 𝜎𝑤2 ), n = 1, … , NS,max . The waterfilling algorithm can be applied to allocate power to each
data stream, considering the power budget.
By comparing this choice of the precoder and combiner with the results obtained on the capacity of
a MIMO system (see Section 6.5.4), we will conclude that this solution is capacity-achieving, as long
470 Chapter 9. Transmission over multiple input multiple output channels

as waterfilling is applied and ideal constellation and coding schemes are used (even separately) for each
stream. Therefore, non-linear solutions wherein C is known at both the transmitter and receiver cannot
outperform this linear approach.

9.5 Hybrid beamforming

Even simple linear precoding and combining solutions require matrix operations, whose complexity
grows with the square of the number of antennas. In particular, for mmWave transmissions (see
Section 4.1.11), the number of antennas quickly becomes large, in order to overcome huge atten-
uations. In this context, a number of solutions have been proposed to reduce the implementation
complexity. Two directions have been mostly considered: the use of precoding/combining solutions
partially implemented with analog circuits (the so-called hybrid beamforming), and the use of
analog-to-digital-converters/digital-to-analog-converters (ADCs/DACs) with a small number of bits
per sample.
The reduction of the number of bits used in ADC/DAC increases the quantization error at each antenna.
Indeed, using a large number of antennas we can average out the noise, so we can use more antennas
and ADCs, each with a smaller number of quantization levels, and still achieve good performance.
Let us now focus on the second approach, i.e. the hybrid beamforming. It generically refers to linear
operations performed at either the transmitter or the receiver, where some operations are implemented
with analog circuits. With reference to a hybrid beamformer used at the receiver, thus operating as a
combiner, we obtain the scheme of Figure 9.10. The received vector is first processed by linear operations
to provide the vector
z = [z1 , … , zNADC ]T = CTA x (9.142)
where NADC is the number of ADCs (typically NADC ≤ NRc ), and combiner CA is implemented by an
analog circuit. After digital conversion, yielding vector z̃ = [̃z1 , … , z̃NADC ]T , a further NS × NADC digital
combiner CTD follows, to obtain
y = CTD z (9.143)
Therefore, ignoring the imperfections of ADC conversion, this scheme implements the overall combiner
CT = CTD CTA .
First note that having fewer ADCs than antennas already limits the resulting combiner C, which in
general will not be equivalent to one of the solutions studied earlier in this chapter, which require a fully
digital implementation.
Then, a second constraint is introduced by the nature of CTA . In principle, the analog combiner should
be designed according to the specific MIMO channel, thus the analog circuit must be reconfigurable. In

x1
z1
ADC
y1

CTA CTD yNS

zNADC
ADC
xNRc

Figure 9.10 Hybrid beamformer operating as a combiner.

9.5. Hybrid beamforming 471

60
AC (SNR = –20 dB)
AC (SNR = –0 dB)
50 DC (SNR = –20 dB)
DC (SNR = 0 dB)
HC (SNR = –20 dB)
40 HC (SNR = 0 dB)
SE (bits/s/Hz)

30

20

10

0
1 2 3 4 5 6 7 8
bits

Figure 9.11 Spectral efficiency for analog, digital, and hybrid beamforming structures. Source:
Reproduced with permission from Abbas et al. [31]. ©2017, IEEE.

practice, the reconfigurability typically consists in either switches connecting ADCs to different circuits
or reconfigurable phase shifters with a finite set of shifts, as entries of matrix CA . In other words, entries
of CA are constrained to be either zero or ej𝜙 , with 𝜙 ∈ {𝜙0 , 𝜙1 , … , 𝜙Q−1 }. The choice of these values is
typically further constrained by the fact that each ADC may be connected (even through switches) only
to a sub-set of antennas, in order to reduce the complexity of the analog circuit.
These solutions provide a trade-off between the complexity of implementing a parametric analog
circuits and the flexibility requested by adapting the combiner to the channel. Both simulations and
experiments show that even with a small number of ADCs and switches or phase shifts, the performance
of hybrid solutions can be very close to the fully digital solution, especially when the number of receive
antennas is much larger than the number transmitted data streams (or transmit antennas).
An example of spectral efficiency (SE) (as given by (6.118)) obtained with hybrid beamforming and a
fix number of quantization bits per ADC is given in Figure 9.11. AC refers to a purely analog combiner,
DC refers to a purely digital combiner, and HC refers to a hybrid combiner. In all cases, ADCs have a
finite number of bits per sample. The x-axis indicates the number of bits used for each sample conversion
in the ADC. Here NTx = 64, NRc = 16, and NRF = 4. The hybrid beamforming architecture comprises
analog phase shifters with infinite precision, thus CTA is full, with entries [CTA ]m,p = ej𝜙m,p , 𝜙m,p ∈ [0, 2𝜋).
According to the small-number-of-paths model (see Section 4.1.11), the mmWave channel is described
by a number of clusters from which paths are generated, and the number of clusters is a Poisson random
variable with parameter 1.8, while the number of paths per cluster3 is 20. The two values of channel
SNR Γ in the figure refers to non-line-of-sight (LOS) (Γ = −20 dB) and LOS (Γ = 0 dB) scenarios.
We note that hybrid beamforming structures provide an interesting trade-off between complexity and
performance, with respect to the fully digital solution, while significantly outperforming the fully analog
solution with phase shifters.

3
Note that even if the number of paths is large, it turns out that only a few of them are actually relevant (see
Appendix 4.A).
472 Chapter 9. Transmission over multiple input multiple output channels

Hybrid beamforming and angular domain representation

A special solution is obtained by applying the angular domain representation of mmWave channels (see
Appendix 4.A) with hybrid beamforming at both the transmitter and the receiver. Consider that the
virtual channel, as the 2D discrete Fourier transform (DFT) (4.162) of C , can be written as

𝚯 = FM1 C FM2 (9.144)

with FMi the DFT matrix of size Mi × Mi (see (1.30)). As 𝚯 in mmWave channels is sparse (see
Appendix 4.A), we can use the following precoder and combiner:

● BA = FM2 with all columns set to zero, except those corresponding to the non-zero columns of 𝚯
● CA = FM1 with all rows set to zero, except those corresponding to the non-zero rows of 𝚯.

Both matrices can be implemented in analog by phase shifters and switches to active selective ele-
ments.

9.6 Multiuser MIMO: broadcast channel

We have already mentioned in Section 9.1 that spatial multiplexing may enable simultaneous transmis-
sions (by SDMA) among multiple devices or users, in what is called multiuser MIMO (MU-MIMO),
in contrast to single-user MIMO (SU-MIMO) systems considered until now in this chapter. In
particular, we focus on two relevant MU-MIMO scenarios: the broadcast or downlink channel and the
multiuser access or uplink channel. The former scenario (treated in this section) occurs for example
in a cellular system when a base station transmits to multiple users at the same time on the same
frequency, and either or both the transmitter and receivers are equipped with multiple antennas. The
latter scenario (treated in Section 9.7) still occurs in a cellular system when multiple users transmit
simultaneously on the same frequency to the same base station. Another example of MU-MIMO
is the communication between stations and the access point (AP) of a wireless local area network
(WLAN).
With reference to the broadcast channel, let the transmitter be equipped with NTx antennas, and the
U users be equipped with NRc antennas each. 4 Let {C,u } be the NRc × NTx channel matrices for users
u = 1, 2, … , U, and let
C = [TC,1 , … , TC,U ]T (9.145)

be the UNRc × NTx overall channel matrix. The input–output relation for user u can be written as

xu = C,u s + 𝒘u (9.146)

where 𝒘u is the noise vector, and s is the transmitted signal containing data for all users. In general,
the transmission of data symbols for different users will generate multiuser interference at the receiver,
i.e. the reception of a user signal will be interfered by the reception of other users’ signals. This can be
seen as a generalization of the MIMO interference of SU-MIMO. However, in the broadcast MU-MIMO
channel, signals received by different users can only be processed locally, rather than jointly among all
receiving antennas as occurs in SU-MIMO systems.

4
For the sake of a simpler presentation, we focus on users with the same number of antennas. Its extension to more
general cases is straightforward.
9.6. Multiuser MIMO: broadcast channel 473

CSI only at the receivers

Consider now a broadcast MU-MIMO channel where CSI is available only at the receivers, i.e. the
transmitter does not know the channel. We can apply at each user the MIMO combiners of Section 9.2.2
or the non-linear MIMO detection and decoding schemes of Section 9.2.3. Each user receives (and
potentially decodes, e.g. in V-BLAST) data streams of all users and is able to decode its own signal
only if NRc ≥ NTx . Note that this condition rarely occurs in wireless systems, such as cellular systems or
WLANs, where users have been equipped with a single antenna for a long time, while base stations or
access points are equipped with multiple antennas. Only recently the use of multiple antennas at end-user
mobile power-supplied devices is spreading.

CSI only at the transmitter

When CSI is available only at the transmitter of a broadcast MU-MIMO channel, all techniques
for SU-MIMO systems developed under the hypothesis of having CSI only at the transmitter (see
Section 9.3) are immediately applicable, as we still have a single transmitter. As a result of precoding,
each user will see an equivalent SISO channel on each receive antenna.

9.6.1 CSI at both the transmitter and the receivers

When CSI is available at both the transmitter and the receivers, we can adopt specific solutions for
a broadcast MU-MIMO channel. We assume here that NTx ≥ UNRc , so that we can transmit up to
NS,u = NRc streams per user u, for a total number of NS = UNRc streams.
We will consider two linear precoding and combining techniques suited for the broadcast MU-MIMO
channel. In particular, we generate the transmitted vector as
∑
U
s= Bu au (9.147)
u=1

where {Bu } are NTx × NS,u precoding matrices, one for each user, and au = [au,1 , … , au,NS,u ]T is the data
vector for user u. We also define the overall precoding matrix
B = [B1 , … , BU ] (9.148)

Block diagonalization
Block diagonalization is a linear precoding technique such that
{
𝟎 p≠u
C,u Bp = (9.149)
eq,u p = u
Hence, the cascade of precoding and MU-MIMO channel results in a block-diagonal matrix, i.e.
⎡eq,1 𝟎 ··· 𝟎 ⎤
⎢ ⎥
𝟎 ⋱ ⋱ ⋮ ⎥
C B = ⎢ (9.150)
⎢ ⋮ ⋱ ⋱ 𝟎 ⎥
⎢ 𝟎 ··· 𝟎 eq,U ⎥⎦
⎣
and eq,u , u = 1, … , U, is the NRc × NS,u resulting channel matrix for user u. From (9.146) and (9.147),
the vector signal received at user u is
(U )
∑
xu = C,u Bp ap + 𝒘u = eq,u au + 𝒘u (9.151)
p=1
474 Chapter 9. Transmission over multiple input multiple output channels

i.e. we have an equivalent SU-MIMO channel, with input au and output xu . For this channel, we apply
precoding the techniques (possibly using also further combining) described in the previous sections of
this chapter.
In order to obtain (9.149), Bp must be in the right null space of C,p for p ≠ u, or still equivalently,
defining the NRc (U − 1) × NTx matrix
 u = [TC,1 , … , TC,u−1 , TC,u+1 , … , TC,U ]T (9.152)
Bu must be in the right null space of  u , i.e.
 u Bu = 𝟎 (9.153)
By taking the SVD of  u , we obtain Nnz = rank( u ) non-zero singular values and
 u = UHu diag([𝜈Hu ,1 , … , 𝜈Hu ,Nnz , 0, … , 0]) [V Hu ,nz V Hu ,z ]H (9.154)
where V Hu ,z is the NRc × NTx matrix containing the right singular vectors corresponding to the right null
space of  u . Therefore, by setting
Bu = V Hu ,z (9.155)
we satisfy (9.149), and
eq,u = C,u V Hu ,z (9.156)
Note that V Hu ,z has unit norm, thus the power constraint (9.117) is satisfied. Indeed, block diagonalization
can be seen as a generalization of ZF, where instead of imposing a diagonal equivalent MIMO channel
we impose a block-diagonal equivalent multiuser MIMO channel.
For example, if we use the capacity-achieving linear precoder and combiner of Section 9.4 for each
user, and denote with Beq,u and Ceq,u the precoder and combiner matrices obtained from the SVD of eq,u ,
the overall precoder for user u (cascade of block diagonalization and user precoding) is BBC,u = Bu Beq,u .
By collecting all precoders into matrix BBC = [BBC,1 , … , BBC,U ] and all combiners into matrix CBC ,
block-diagonal with blocks Ceq,u , u = 1, … , U, we obtain the overall equivalent channel
CTBC C BBC = 𝚺 BC (9.157)
where 𝚺BC is a diagonal matrix containing on the diagonal the singular values of matrices eq,u , u =
1, … , U.

User selection
Multiuser interference can be avoided by transmitting only to a subset of users that do not interfere with
each other. In particular, out of the U users, we select U ′ ≤ U users with indices  ′ = {u′1 , … , u′U′ },
such that their channel matrices are semi-orthogonal, i.e.
C,p H
C,q ≃ 𝟎 for p, q ∈  ′ and p ≠ q (9.158)
Let ′C = [TC,u1 , … , TC,uU′ ]T the new overall channel. By choosing the user precoder
H
C,u′j
Bu′ = (9.159)
j ||C,u′ ||
j
′ ′ ′
for j = 1, … , U , and setting B = [Bu′ , … , Bu′ ], from (9.158) we have that C B is block-diagonal with
′

1 ′
U
blocks of size NRc × NS,u′ , thus avoiding multiuser interference. The resulting equivalent SU-MIMO
i
channel for user u is
C,u HC,u
eq,u = (9.160)
||C,u ||
9.6. Multiuser MIMO: broadcast channel 475

for which we can apply precoding/combining SU-MIMO techniques. Note that for single-antenna users
(NRc = 1), the precoder is simply the MRC MISO linear precoder (9.111).
Various approaches have been proposed in the literature to select users such that condition (9.158)
holds (see [32] for a survey). For example, in [33] a greedy iterative semi-orthogonal user selection
algorithm has been proposed, where set  is built by adding one user at each iteration. In particular,
′

for single-antenna users (NRc = 1), we first select the user having the largest channel gain (set 1 =
{1, … , U}), with C,u 1 × NTx row vector,
u′1 = arg max ||C,u ||2 (9.161)
u∈1

Then, we select the subset of users whose channels are almost orthogonal to C,u′ , according to the
1
correlation coefficient
|C,p C,u′ |
H

𝚛p,u′ = 1
(9.162)
1 ||C,p || ||C,u′ ||
1

In particular, let 2 = {p ∈ 1 ∶ 𝚛p,u′ < 𝜖}, with 𝜖 a small-enough positive number, be the set of users
1
whose channels have a small correlation with that of user u′1 . Among these users, we select the user with
the largest channel gain, i.e.
u′2 = arg max ||C,u ||2 (9.163)
u∈2

The process is iterated, where at iteration i > 1, we extract the user subset
i+1 = {p ∈ i ∶ 𝚛p,u′ < 𝜖} (9.164)
i
′
and within i+1 , we select the user ui+1 with the largest channel gain.
Clearly, user selection can also be applied in conjunction with block diagonalization: first, we select
almost-orthogonal users and then we apply block diagonalization on the set of selected users to com-
pletely remove interference. With this choice, we avoid having to remove interference among users with
highly correlated channels, thus saving in transmit power.

Joint spatial division and multiplexing

Both block diagonalization and user selection require the knowledge of CSI C at the transmitter. As
discussed in Section 9.3.4, using CSI feedback techniques, the number of feedback bits is proportional
to the number of users and increases for time-varying channels, as CSI must be refreshed whenever the
channel changes.
The joint spatial division and multiplexing (JSDM) technique [34] provides an alternative solution,
reducing the need to have updated CSI from all users at the transmitter, by exploiting the fact that users’
channels are typically spatially correlated and their autocorrelation matrices change much more slowly
than the instantaneous channel realizations. Also in this case, for the sake of a simpler explanation, let us
consider single-antenna users, i.e. NRc = 1. We consider the SVD of the channel autocorrelation matrix
of user u (see also the transmit correlation matrix in (4.147))
Ru = E[C,u H
C,u ] = U u 𝚲u U u
H
(9.165)
where 𝚲u is diagonal with real non-negative entries. For zero-mean Gaussian channel matrices, we can
decompose the channel vector C,u as (see (4.147) for MIMO channel matrix with correlated Gaussian
entries)
1∕2
C,u = gTu 𝚲u UH u (9.166)
with gu a column vector with  (0, 1) independent entries.
476 Chapter 9. Transmission over multiple input multiple output channels

Let us now group users, where users in the same group have the same autocorrelation matrix. Let G be
the number of groups, and Ri , i = 1, … , G, the corresponding autocorrelation matrices, and i is the set
of user indices in group i. We can select a subset of groups {i1 , … , iG′ } such that they have orthogonal
crosscorrelation matrices, i.e. it is

i U j = 𝟎,
UH i ≠ j, i, j ∈ {i1 , … , iG′ } (9.167)
Clearly, from (2.142), we also have = I for i ∈ {i1 , … , iG′ }. Now, the precoding matrix for user
UH
i Ui
u ∈ i is the cascade of a suitable per-user precoding matrix B′u , to be suitably designed, and the
pre-beamforming matrix Ui , i.e.
Bu = Ui B′u (9.168)
With this precoding, the received vector signal can be written as
∑
U
∑ 1∕2
xu = C Bp s + 𝒘u = gTp 𝚲p B′p ap + 𝒘u (9.169)
p=1 p∈i

Note that the interference of users belonging to other groups has been eliminated by Ui , and only
the interference of users of the same group remains. For the design of B′u , we can resort to block
diagonalization for users in group i. Note that pre-beamforming simplifies the design of the per-user
precoder, as it significantly reduces the dimensionality of the precoder from U to |i |. Furthermore, the
pre-beamformer is designed on slowly changing channel characteristics, thus can be kept constant over
multiple transmissions and for all users of the group. Moreover, pre-beamforming is amenable for an
analog implementation in a hybrid beamforming scheme (see Section 9.5).

9.6.2 Broadcast channel estimation

The channel estimation at the receivers of a MIMO broadcast channel does not poses major challenges.
Indeed, the transmitter can transmit a single sequence of symbols (known to all receivers), from which
each receiver estimates its own channel.
When CSIT is needed, we can either estimate the channel from the receiver to the transmitter con-
sidering the resulting multiple-access channel (see Section 9.7.2), or (if possible) we can exploit the
channel reciprocity, estimating the downlink channel at the receiver and then feeding back a quantized
version to the transmitter.

9.7 Multiuser MIMO: multiple-access channel

With reference to the multiple-access MU-MIMO channel, U users are equipped with NTx antennas each,
and the receiver has NRc antennas.5 Let C,u be the NRc × NTx channel matrix for user u = 1, 2, … , U,
and let
C = [C,1 , … , C,U ] (9.170)
be the NRc × UNTx overall channel matrix. The signal vector at the receiver can be written as
∑
U
x= C,u su + 𝒘 (9.171)
u=1
where su is the transmitted vector by user i and 𝒘 is the noise vector. Clearly, also in this case we have
multiuser interference. Let also au = [au,1 , … , au,NS,u ]T be the data vector to be transmitted to user u, and
a = [aT1 , … , aTU ]T .
5
Also in this case, more general solutions with a different number of transmit antennas per user can be easily obtained.
9.7. Multiuser MIMO: multiple-access channel 477

CSI only at the transmitters

We consider here the case wherein CSI is known only at the U transmitters. We also assume that
NRc = U, and each user transmits a single data stream to the receiver.
We may linearly precoding the data signal at each user in order to completely eliminate the multiuser
interference at the receiver. This can be achieved by ensuring that data signal of user u is received only
by antenna u. 6 To this end, let us define the NRc × (NRc − 1) matrix
(1,⋅)T (N ,⋅)T T
C,u = [C,u , … , (u−1,⋅)T
C,u
(u+1,⋅)T
, C,u , … , C,uRc ] (9.172)

i.e. the user channel matrix C,u without the row corresponding to antenna u. We can then apply a
procedure similar to block diagonalization, where user u utilizes as precoder Bu the vector of the right
null space of C,u . Then, from (9.171), we have

∑
U
(r,⋅)
x(r) = C,u Bu au + 𝒘 = (r)
eq ar + 𝑤
(r)
(9.173)
u=1

where
(u) (u,⋅)
eq = C,u Bu (9.174)

Note from (9.173) that we have eliminated the multiuser interference.

CSI only at the receiver

Consider now a multiple-access MU-MIMO channel, where CSI is available only at the receiver. We
can apply at the receiver the MIMO combiner of Section 9.2.2 or the non-linear MIMO detection and
decoding of Section 9.2.3, seeing antennas of different users as antennas of a single user, wherein each
antenna transmits an independent data signal. In this case, it must be NRc ≥ UNTx for proper detection.

9.7.1 CSI at both the transmitters and the receiver

We now consider the knowledge of CSI at both the transmitters and the receiver for the multiuser access
MU-MIMO channel. First note that if the users are equipped with a single antenna each (NTx = 1), CSI
at the transmitter is of no use and we can only resort to solutions using CSI at the receiver. In this latter
case, we can also apply user selection techniques, as those described for the broadcast channel, in order
to select users with semi-orthogonal channels and ease the processing at the receiver. An example is
given by cellular systems where the mobile user’s terminal has been traditionally equipped with a single
antenna.
We now consider the case wherein each user is equipped with multiple antennas, i.e. NRc > 1.

Block diagonalization
For the multiuser access channel with NS,u streams for user u, we aim at designing a block-diagonal
precoding matrix BMA (with blocks of size NTx × NS,u ) and a full combining matrix CMA such that

CTMA C BMA = 𝚺 (9.175)

6
Note that the association between the users and the receive antennas can be optimized according to instantaneous
channel conditions, although it then requires some coordination among users. Here, we consider the simple association
of user u with receive antenna u.
478 Chapter 9. Transmission over multiple input multiple output channels

with 𝚺 a block-diagonal matrix. This is similar to (9.157), obtained by block diagonalization of the
broadcast MU-MIMO channel, where however there the combining matrix was block-diagonal while
the precoding matrix was full. Now, let us define the equivalent broadcast channel matrix as
C,BC = TC (9.176)
Note that for uplink and downlink transmissions occurring on the same channel, (9.176) is given by the
channel reciprocity. In any case, we will take (9.176) as the definition of matrix C,BC without further
physical meanings. Then, we can design matrices CBC and BBC according to (9.157), where we assume
a broadcast channel for a transmitter with NRc antennas and receivers with NTx antennas each. Lastly, if
we set
BMA = CBC , CMA = BBC (9.177)
we note that we have a block-diagonal precoder, suited for uplink transmission, and a full combining
matrix. Moreover, observing that
𝚺 = 𝚺 T = BTBC TC,BC CBC = CTMA C BMA (9.178)
we conclude that with definitions (9.177), condition (9.175) holds. Thus, we have obtained an
interference-free MIMO channel with linear precoding and combining, where the pre-coding is
performed on a per-user basis. Clearly, this approach requires CSI at both the transmitter and the
receiver.

9.7.2 Multiple-access channel estimation

Channel estimation at the receiver in multiple-access channel requires the estimation of the NRc × UNTx
channel matrix (9.170) between a transmitter with UNTx antennas and NRc antennas. If each user is
transmitting different pilot sequences, the receiver jointly estimates all the users’ channels. Note that
this solution requires at least the transmission of UNTx symbol vectors by each user, with a sequence
length proportional to the number of users. Moreover, we need orthogonal training blocks (see (9.105)).
For CSIT, if we can exploit channel reciprocity, the transmitter can estimate the receiver-transmitters
channel at the transmitters by exploiting the broadcast channel estimation techniques of Section 9.6.2.
Otherwise, we can estimate the channel at the receiver and then feed back (quantized) estimated channels
to the transmitters.

9.8 Massive MIMO

The term massive MIMO refers to a MIMO system wherein the number of either or both the transmit
and the receive antennas grows to infinity, i.e.
NTx → ∞ and/or NRc → ∞ (9.179)
Massive MIMO systems were first studied in [35].
Operating under such asymptotic conditions has a number of consequences. Here we indicate two main
benefits of massive MIMO systems, namely the channel hardening and the favourable propagation.

9.8.1 Channel hardening

Typically, massive MIMO is referred to channels with uncorrelated Gaussian entries (see (4.154))
R𝑣ec( ) = I (9.180)
C
Bibliography 479

This scenario occurs when a large number of scatterers is available. Note that this may not always be
the case, such as in the relevant case of mmWave channels, wherein the number of scatterers is low,
although a very large number of antennas may be employed.
As a consequence of conditions (9.179) and (9.180), we obtain the channel hardening phenomenon.
Consider a MISO system, then from (9.180) and the law of large numbers we have
||C ||2
lim =1 (9.181)
NTx →∞ NTx

Therefore, the channel gain of any link tends to a constant. For example, using the MISO linear precoding
from (9.112), we obtain
𝛾LP → 𝛾MF NTx (9.182)
As a consequence of channel hardening, the effects of fading disappears and any receiver experiences
always the same SNR at the decoder input. Similar results hold for a SIMO system using a linear com-
biner matched to the channel, and also for a general massive MIMO system.
Avoiding fading phenomenon significantly simplifies also the resource allocation (power allocation
but also subcarrier allocation to multiple users in OFDM systems), since channels conditions are
time-invariant and the same for all receivers.

9.8.2 Multiuser channel orthogonality

Consider now a broadcast massive MISO channel, where the transmitter has NTx antennas and each of
the U user has a single (NRc = 1) antenna. Assume that users’ channel C,u , u = 1, … , U, is zero-mean
statistically independent, then we have
lim C,u1 H
C,u2 = 0, u1 ≠ u2 , u1 , u2 = 1, … , U (9.183)
NTx →∞

In this case, users have orthogonal channels and condition (9.158) holds with equality. This condition
is also denoted as favourable propagation. Therefore, a simple linear precoder matched to each user
separately
Bu = HC,u (9.184)
maximizes the SNR at the detector input, and avoids any multiuser interference.

Bibliography

[1] Jakes, W.C. (1993). Microwave Mobile Communications. New York, NY: IEEE Press.
[2] Glance, B. and Greenstein, L.J. (1983). Frequency-selective fading in digital mobile radio with
diversity combining. IEEE Transactions on Communications 31: 1085–1094.
[3] Kennedy, W.K., Greenstein, L.J., and Clark, M.V. (1994). Optimum linear diversity receivers for
mobile communications. IEEE Transactions on Vehicular Technology 43: 47–56.
[4] Clark, M.V., Greenstein, L.J., Kennedy, W.K., and Shafi, M. (1992). MMSE diversity combining for
wide-band digital cellular radio. IEEE Transactions on Communications 40: 1128–1135.
[5] Compton, R.T. (1988). Adaptive Antennas: Concepts and Performance. Englewood Cliffs, NJ:
Prentice-Hall.
[6] Falconer, D.D., Abdulrahman, M., Lo, N.W.K. et al. (1993). Advances in equalization and diversity
for portable wireless systems. Digital Signal Processing 3: 148–162.
[7] Foschini, G.J. (1996). Layered space-time architecture for wireless communication in a fading envi-
ronment when using multi-element antennas. Bell System Technical Journal 1: 41–59.
480 Chapter 9. Transmission over multiple input multiple output channels

[8] Godara, L.C. (1997). Application of antenna arrays to mobile communications - Part I: Performance
improvement, feasibility and system considerations. IEEE Proceedings 85: 1031–1060.
[9] Godara, L.C. (1997). Application of antenna arrays to mobile communications - Part II:
Beam-forming and direction-of-arrival considerations. IEEE Proceedings 85: 1195–1245.
[10] Kirsteins, I.P. and Tufts, D.W. (1994). Adaptive detection using low rank approximation to a data
matrix. IEEE Transactions on Aerospace and Electronic Systems 30: 55–67.
[11] Kohno, R., Imai, H., Hatori, M., and Pasupathy, S. (1990). Combination of an adaptive array
antenna and a canceller of interference for direct-sequence spread-spectrum multiple-access system.
IEEE Journal on Selected Areas in Communications 8: 675–682.
[12] Lee, W.C.Y. (1973). Effects on correlation between two mobile radio base-station antennas. IEEE
Transactions on Communications 21: 1214–1224.
[13] Monzingo, R.A. and Miller, T.W. (1980). Introduction to Adaptive Arrays. New York, NY: Wiley.
[14] Paulraj, A.J. and Papadias, C.B. (1997). Space-time processing for wireless communications. IEEE
Signal Processing Magazine 14: 49–83.
[15] Salz, J. and Winters, J.H. (1994). Effect of fading correlation on adaptive arrays in digital wireless
communications. IEEE Transactions on Vehicular Technology 43: 1049–1057.
[16] Tsoulos, G., Beach, M., and McGeehan, J. (1997). Wireless personal communications for the 21st
century: European technological advances in adaptive antennas. IEEE Communications Magazine
35: 102–109.
[17] Widrow, B., Mantey, P.E., Griffiths, L.J., and Goode, B.B. (1967). Adaptive antenna systems. IEEE
Proceedings 55: 2143–2159.
[18] Winters, J.H. (1998). Smart antennas for wireless systems. IEEE Personal Communications Maga-
zine 5: 23–27.
[19] Balaban, P. and Salz, J. (1991). Dual diversity combining and equalization in digital cellular mobile
radio. IEEE Transactions on Vehicular Technology 40: 342–354.
[20] Foschini, G.J. (1977). A reduced-state variant of maximum-likelihood sequence detection attaining
optimum performance for high signal-to-noise ratio performance. IEEE Transactions on Information
Theory 24: 505–509.
[21] Chau, Y.A. and Yu, S.-H. (2001). Space modulation on wireless fading channels. IEEE 54th Vehic-
ular Technology Conference. VTC Fall 2001. Proceedings (Cat. No.01CH37211), Volume 3, pp.
1668–1671.
[22] Alamouti, S.A. (1998). A simple transmit diversity technique for wireless communications. IEEE
Journal on Selected Areas in Communications 16: 1451–1458.
[23] DVB (2009). Frame structure channel coding and modulation for a second generation digital terres-
trial television broadcasting system (DVB-T2), A122.
[24] 3GPP (2008). Evolved universal terrestrial radio access (E-UTRA); physical layer procedures, TS
36.213, v. 8.5.0.
[25] Tarokh, V., Seshadri, N., and Calderbank, A.R. (1998). Space-time codes for high data rate wireless
communication: performance criterion and code construction. IEEE Transactions on Information
Theory 44: 744–765.
[26] Naguib, A., Tarokh, V., Seshadri, N., and Calderbank, A.R. (1998). A space-time coding modem
for high data rate wireless communications. IEEE Journal on Selected Areas in Communications
16: 1459–1478.
[27] Tarokh, V., Naguib, A., Seshadri, N., and Calderbank, A.R. (1999). Combined array processing and
space-time coding. IEEE Transactions on Information Theory 45: 1121–1128.
[28] Golub, G.H. and van Loan, C.F. (1989). Matrix Computations, 2e. Baltimore, MD and London: The
Johns Hopkins University Press.
Bibliography 481

[29] Costa, M. (1983). Writing on dirty paper (corresp.). IEEE Transactions on Information Theory 29:
439–441.
[30] Kim, T., Kwon, K., and Heo, J. (2017). Practical dirty paper coding schemes using one error
correction code with syndrome. IEEE Communications Letters 21: 1257–1260.
[31] Abbas, W.B., Gomez-Cuba, F., and Zorzi, M. (2017). Millimeter wave receiver efficiency: A com-
prehensive comparison of beamforming schemes with low resolution ADCs. IEEE Transactions on
Wireless Communications 16: 8131–8146.
[32] Casta neda, E., Silva, A., Gameiro, A., and Kountouris, M. (2017). An overview on resource allo-
cation techniques for multi-user MIMO systems. IEEE Communications Surveys Tutorials 19:
239–284.
[33] Yoo, T. and Goldsmith, A. (2006). On the optimality of multiantenna broadcast scheduling using
zero-forcing beamforming. IEEE Journal on Selected Areas in Communications 24: 528–541.
[34] Adhikary, A., Nam, J., Ahn, J., and Caire, G. (2013). Joint spatial division and multiplexing–the
large-scale array regime. IEEE Transactions on Information Theory 59: 6441–6463.
[35] Marzetta, T.L. (2010). Noncooperative cellular wireless with unlimited numbers of base station
antennas. IEEE Transactions on Wireless Communications 9: 3590–3600.
Chapter 10

Spread-spectrum systems

The term spread-spectrum systems [1–7] was coined to indicate communication systems in which the
bandwidth of the signal obtained by a standard modulation method (see Chapter 6) is spread by a certain
factor before transmission over the channel, and then despread, by the same factor, at the receiver. The
operations of spreading and despreading are the inverse of each other, i.e. for an ideal and noiseless
channel the received signal after despreading is equivalent to the transmitted signal before spreading.
For transmission over an ideal additive white Gaussian noise (AWGN) channel, these operations do not
offer any improvement with respect to a system that does not use spread-spectrum techniques. However,
the practical applications of spread-spectrum systems are numerous, for example in multiple-access
systems, narrowband interference rejection, and transmission over channels with fading, as discussed in
Section 10.2.

10.1 Spread-spectrum techniques

We consider the two most common spread-spectrum techniques: direct sequence (DS) and frequency
hopping (FH).

10.1.1 Direct sequence systems

The baseband equivalent model of a DS system is illustrated in Figure 10.1. We consider the possibility
that U users in a multiple-access system simultaneously transmit, using the same frequency band, by
code division multiple access (CDMA) (see Appendix 18.B).
The sequence of information bits {b(u)
𝓁
} of user u undergoes the following transformations.

Bit-mapper From the sequence of information bits, a sequence of i.i.d. symbols {a(u)
k
} with statistical
power 𝙼a is generated. The symbols assume values in an M-ary constellation with symbol period T.

Spreading We indicate by the integer NSF the spreading factor, and by Tchip the chip period. These
two parameters are related to the symbol period T by the relation
T
Tchip = (10.1)
NSF
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
484 Chapter 10. Spread-spectrum systems

spreader pulse generator

{bl(u)} bit ak(u) dm(u) A(u) s(u)(t)

gsp(u) hTx
Tb mapping T
Tchip = T
NSF

{cm(u)}
(a)

(2)
sC (t) wC (t)
(1)
s(1)(t) sC (t) rC (t)
(1)
gC

(U)
sC (t)
(b)

Figure 10.1 Baseband equivalent model of a DS system: (a) transmitter, (b) multiuser channel.

We consider the channelization code {c(u)Ch,m

}, u ∈ {1, … , U}, where c(u)
Ch,m
∈ {−1, 1} and |c(u)
Ch,m
| = 1.
We assume that the sequences of the channelization code are periodic, that is c(u)
Ch,m
= c (u)
Ch,mmod NSF
. The
channelization codes are orthogonal among users, if
{
1 ∑ (u1 ) (u2 )∗
NSF −1
1 if u1 = u2
cCh,m cCh,m = (10.2)
NSF m=0 0 if u1 ≠ u2
An example of orthogonal channelization code sequence is given by the Walsh–Hadamard sequences
(see Appendix 10.A).
We also introduce the user code {c(u)
m }, also called signature sequence or spreading sequence, that we
initially assume equal to the channelization code, i.e.

c(u) (u)
m = cCh,m (10.3)

Consequently, {c(u) m } is also a periodic sequence of period NSF .

The operation of spreading consists of associating to each symbol a(u) k
a sequence of NSF symbols of
period Tchip , that is obtained as follows. First, each symbol a(u)
k
is repeated NSF times with period Tchip .
As illustrated in Figure 10.2, this operation is equivalent to upsampling {a(u) k
}, so that (NSF − 1) zeros
are inserted between two consecutive symbols, and using a holder of NSF values. The obtained sequence
is then multiplied by the user code. Formally we have
(u)
am = a(u)
k
, m = kNSF , … , kNSF + NSF − 1
(10.4)
(u)
dm(u) = am c(u)
m

If we introduce the filter

g(u) (u)
sp (i Tchip ) = ci , i = 0, … , NSF − 1 (10.5)

the correlation of Figure 10.2a can be replaced by the interpolator filter g(u)
sp , as illustrated in Figure 10.2b.
10.1. Spread-spectrum techniques 485

a (u)
k
am(u) (u)
dm
NSF holder
T Tchip
(u)
cm
(a)

a (u)
k
(u)
dm
g (u)
sp
T Tchip
(b)

Figure 10.2 Spreading operation: (a) correlator, (b) interpolator filter.

Recalling that |c(u)

m | = 1, from (10.4) we get

𝙼d = 𝙼a (10.6)

Pulse-shaping Let hTx be the modulation pulse, typically a square root raised cosine function or rect-
angular window. The baseband equivalent of the transmitted signal of user u is expressed as
∑
+∞
s(u) (t) = A(u) dm(u) hTx (t − m Tchip ) (10.7)
m=−∞

where A(u) accounts for the transmit signal power. In fact, if Eh is the energy of hTx and {dm(u) } is assumed
i.i.d., the average statistical power of s(u) is given by (see (1.324))
Eh
𝙼s(u) = (A(u) )2 𝙼d (10.8)
Tchip
Using (10.4), an alternative expression for (10.7) is given by

∑
+∞
∑
NSF −1
s(u) (t) = A(u) a(u)
k
c(u)
𝓁+kN
hTx (t − (𝓁 + kNSF ) Tchip ) (10.9)
SF
k=−∞ 𝓁=0

In the scheme of Figure 10.1a we note that, if condition (10.3) holds, then g(u)
sp is invariant with respect
(u)
to the symbol period, and the two filters gsp and hTx can be combined into one filter (see also (10.9))

∑
NSF −1
h(u)
T
(t) = c(u)
𝓁
hTx (t − 𝓁 Tchip ) (10.10)
𝓁=0

thus providing
∑
+∞
s(u) (t) = A(u) a(u)
k
h(u)
T
(t − kT) (10.11)
k=−∞

As shown in Figure 10.3, the cascade of spreader and pulse-shaping filter is still equivalent to the modu-
lator of a quadrature amplitude modulation (QAM). The peculiarity is that the filter h(u)
T
has a bandwidth
much larger than the Nyquist frequency 1∕(2T). Therefore, a DS system can be interpreted as a QAM
system either with input symbols {dm(u) } and transmit pulse hTx , or with input symbols {a(u)
k
} and pulse
h(u)
T
; later both interpretations will be used.
486 Chapter 10. Spread-spectrum systems

(u)
a (u) A s (u)(t)
k
h (u)
T
T

Figure 10.3 Equivalent scheme of spreader and pulse-shaping filter in a DS system.

Transmission channel Modelling the transmission channel as a filter having impulse response g(u)
C
, the
output signal is given by
s(u)
C
(t) = (s(u) ∗ g(u)
C
)(t) (10.12)

Multiuser interference At the receiver, say of user 1, we have also the signal of other users (rather
than user 1), namely signals sC(2) (t), … , s(U)
C
(t).
The possibility that many users transmit simultaneously over the same frequency band leads to a total
signal
∑
U
sC (t) = s(u)
C
(t) (10.13)
u=1

We now aim at reconstructing the message {a(1)

k
} of user u = 1, identified as desired user. If 𝙼s(u) is the
C

statistical power of s(u)

C
, the following signal-to-interference ratios (SIRs) define the relative powers of
the user signals:
𝙼s(1)
Γ(u) = C
, u = 2, 3, … , U (10.14)
i 𝙼s(u)
C

Noise In Figure 10.1b, the term 𝑤C includes both the noise of the receiver and other interference
sources, and it is modelled as white noise with power spectral density (PSD) equal to N0 .
Two signal-to-noise ratios (SNRs) are of interest. To measure the performance of the system in terms
of Pbit , it is convenient to refer to the SNR defined in Chapter 6 for passband transmission, i.e.
𝙼s(1)
C
Γs = (10.15)
N0 ∕T

We recall that for an uncoded sequence of symbols {a(1)

k
}, the following relation holds:
Eb Γs
= (10.16)
N0 log2 M
However, there are cases, as for example in the evaluation of the performance of the channel impulse
response estimation algorithm, wherein it is useful to measure the power of the noise over the whole
transmission bandwidth. Hence, we define
𝙼s(1) Γs
C
Γc = = (10.17)
N0 ∕Tchip NSF

Receiver The receiver structure varies according to the channel model and number of users. Deferring
until Section 10.3, the analysis of more complicated system configurations, here we limit ourselves to
an ideal AWGN channel with g(u) C
(t) = 𝛿(t) and synchronous users. The latter assumption implies that
the transmitters of the various users are synchronized and transmit at the same instant. For an ideal
AWGN channel, this means that at the receiver the optimum timing phase of signals of different users
is the same.
10.1. Spread-spectrum techniques 487

With these assumptions, we verify that the optimum receiver is simply given by the matched filter to
h(1)
T
. According to the analog or discrete-time implementation of the matched filters, we get the schemes
of Figure 10.4 or Figure 10.5, respectively; note that in Figure 10.5 the receiver front-end comprises
an anti-aliasing filter followed by a sampler with sampling period Tc = Tchip ∕2. Let t0 be the optimum
timing phase at the matched filter output (see Section 14.7).

g (1)(t) =h (1)*(−t)
M T
a^ k
rC (t) (1)
(1) yk
g
M
T

(a)

g (t) =h * (−t) t 0 +m Tchip

M Tx
a^ k
rC (t) (1)
xm kNSF +NSF −1 yk
g ∑ xm c (1)*
m
M
Tchip m=kN SF T
(1)*
cm

(b)

Figure 10.4 Optimum receiver with analog filters for a DS system with ideal AWGN channel and
synchronous users. Two equivalent structures: (a) overall matched filter, (b) matched filter to hTx
and despreading correlator.

For an ideal AWGN channel, it results

∑
U
rC (t) = s(u) (t) + 𝑤C (t)
u=1
(10.18)
∑
U +∞
∑ ∑
NSF −1
= A(u) a(u)
i
c(u)
𝓁
hTx (t − (𝓁 + iNSF ) Tchip ) + 𝑤C (t)
u=1 i=−∞ 𝓁=0

In the presence only of the desired user, that is for U = 1, it is clear that in the absence of inter-symbol
interference (ISI), the structure with the matched filter to h(1) T
is optimum. We verify that the presence
of other users is cancelled at the receiver, given that the various user codes are orthogonal.
We assume that the overall analog impulse response of the system is a Nyquist pulse, hence
( ) |
hTx ∗ g(u)
C
∗ g AA ∗ g M (t)|| = (hTx ∗ gM )(t)|t=t0 +j Tchip = Eh 𝛿j (10.19)
|t=t0 +j Tchip
We note that, if t0 is the instant at which the peak of the overall pulse at the output of gM is observed,
then t0′ in Figure 10.5 is given by t0′ = t0 − tgM , where tgM is the duration of gM . Moreover, from (10.19),
we get that the noise at the output of gM , sampled with sampling rate 1∕Tchip ,
𝑤̃ m = (𝑤C ∗ gM )(t)|t=t0 +m Tchip (10.20)
is an i.i.d. sequence with variance N0 Eh .
Hence, from (10.9) and (10.19), the signal at the output of gM , sampled with sampling rate 1∕Tchip , is

∑
U
∑
+∞
∑
NSF −1
xm = E h A(u) a(u)
i
c(u)
𝓁+iN
𝛿m−𝓁−iNSF + 𝑤̃ m (10.21)
SF
u=1 i=−∞ 𝓁=0
488 Chapter 10. Spread-spectrum systems

g (t) = h (1)*(−t)
(1)
t 0ʹ +nTc M T
a^ k
rC (t) (1)
rAA (t) (1) yk
g g 2NSF
AA M
Tchip T T
Tc =
2 (a)

g (t) =h * (−t)
M Tx
xm yk
g
M 2 g (1) N SF
ds
Tchip Tchip (b)
2
kNSF +NSF −1
xm yk
g 2 ∑ xm c m(1)*
M
Tchip Tchip m=kN SF (c)
2 (1)*
cm

despreading

Figure 10.5 Optimum receiver with discrete-time filters for a DS system with ideal AWGN channel
and synchronous users. Three equivalent structures: (a) overall matched filter, (b) matched filter to
hTx and despreading filter, and (c) matched filter to hTx and despreading correlator.

With the change of indices m = j + kNSF , j = 0, 1, … , NSF − 1, k integer, we get

∑
U
xj+kNSF = Eh A(u) a(u)
k
c(u)
j+kN
+ 𝑤̃ j+kNSF (10.22)
SF
u=1

Despreading We now correlate the sequence of samples {xm }, suitably synchronized, with the code
sequence of the desired user, and we form the signal
∑
NSF −1
yk = xj+kNSF c(1)∗
j+kN
(10.23)
SF
j=0

As usual, introducing the filter g(u)

ds
given by

g(u)
ds
(i Tchip ) = c(1)∗
N −1−i
, i = 0, 1, … , NSF − 1 (10.24)
SF

the correlation (10.23) is implemented through the filter (10.24), followed by a downsampler, as illus-
trated in Figure 10.5b.
Replacing (10.22) in (10.23) yields
∑
U
yk = NSF Eh A(u) a(u)
k
𝚛c(u) c(1) (0) + 𝑤k (10.25)
u=1

where in general
NSF −1−|nD |
1 ∑ (u ) (u )∗
𝚛c(u1 ) c(u2 ) (nD ) = NSF −|nD |
1
cj+kN 2
cj+kN
SF +nD
j=0 SF
(10.26)
nD = −(NSF − 1), … , −1, 0, 1, … , NSF − 1
is the cross-correlation at lag nD between the user codes u1 and u2 .
10.1. Spread-spectrum techniques 489

In the considered case, from (10.3) and (10.2) we get 𝚛c(u1 ) c(u2 ) (0) = 𝛿u1 −u2 . Therefore, (10.25) simply
becomes
yk = NSF Eh A(1) a(1)
k
+ 𝑤k (10.27)

where NSF Eh is the energy of the pulse associated with a(1)

k
(see (10.10)).
In (10.27), the noise is given by
∑
NSF −1
𝑤k = 𝑤̃ j+kNSF c(1)∗
j+kN
(10.28)
SF
j=0

therefore assuming {𝑤̃ m } i.i.d., the variance of 𝑤k is given by

𝜎𝑤2 = NSF 𝜎𝑤2̃ = NSF N0 Eh (10.29)

Data detector Using a threshold detector, from (10.27) the SNR at the decision point is given by (see
(6.75))
( )
dmin 2 (NSF Eh A(1) )2 N E (A(1) )2
𝛾= = = SF h (10.30)
2𝜎I NSF N0 Eh ∕2 N0 ∕2
On the other hand, from (10.8) and (10.15) we get
(A(1) )2 𝙼a Eh ∕Tchip NSF Eh (A(1) )2 𝙼a E
Γs = = = s (10.31)
N0 ∕(NSF Tchip ) N0 N0
where Es is the average energy per symbol of the transmitted signal.
In other words, the relation between 𝛾 and Γs is optimum, as given by (6.80). Therefore, with regard
to user u = 1, at the decision point the system is equivalent to an M-QAM system. However, as observed
before, the transmit pulse h(1)
T
has a bandwidth much larger than 1∕(2T).

Multiuser receiver The derivation of the optimum receiver carried out for user 1 can be repeated for
each user. Therefore, we obtain the multiuser receiver of Figure 10.6, composed of a matched filter to
the transmit pulse and a despreader bank, where each branch employs a distinct user code.

a^ k
(1)
g (1)
ds
T

a^ k
(2)
(2)
g
g (t) =h * (−t) ds
T
M Tx
r (t)
C
g
M
Tchip
a^ k
(U )
g (U)
ds
T

Figure 10.6 Multiuser receiver for a CDMA synchronous system with an ideal AWGN channel.

We observe that for the ideal AWGN channel case, spreading the bandwidth of the transmit signal by
a factor U allows the simultaneous transmission of U messages using the same frequency band.
490 Chapter 10. Spread-spectrum systems

Classification of CDMA systems

Synchronous systems This is the case just examined, in which the user codes are orthogonal and the user
signals are time-aligned. In a wireless cellular system, this situation occurs in the forward or downlink
transmission from the base station to the mobile stations, e.g. in the 3rd generation of the 3GPP mobile
communication standard. From the point of view of each mobile station, all U users share the same
channel. Therefore, although the channel impulse response depends on the site of the mobile, we have
g(u)
C
(t) = gC (t), u = 1, … , U (10.32)
and the residual interference is due to signals originating from adjacent cells in addition to the multipath
interference introduced by the channel. In general, interference due to the other users within the same
cell is called multiuser interference (MUI) or co-channel interference (CCI).

Asynchronous systems In this case, the various user signals are not time-aligned. In a wireless cellular
system, this situation typically occurs in the reverse or uplink transmission from the mobile stations to
the base station.
Because the Walsh–Hadamard codes do not exhibit good cross-correlation properties for lags different
from zero, pseudo-noise (PN) scrambling sequences are used (see Appendix 1.C). The user code is then
given by
c(u) (u)
m = cCh,m cscr,m (10.33)
where {cscr,m } may be the same for all users in a cell.
It is necessary now to make an important observation. In some systems, the period of {cscr,m } is equal
to the length of {cCh,m }, that is NSF , whereas in other systems, it is much larger than NSF .1 In the latter
case, spreading and despreading operations remain unchanged, even if they are symbol time varying, as
{c(u)
m } changes from symbol to symbol; note that consequently the receiver is also symbol time varying.
Asynchronous systems are characterized by codes with low cross-correlation for non-zero lags; how-
ever there is always a residual non-zero correlation among the various user signals. Especially in the
presence of multipath channels, the residual correlation is the major cause of interference in the system,
which now originates from signals within the cell: for this reason the MUI is usually characterized as
intracell MUI.

Synchronization
Despreading requires the reproduction at the receiver of a user code sequence synchronous with that
used for spreading. Therefore, the receiver must first perform acquisition, that is the code sequence
{c(u)
m } produced by the local generator must be synchronized with the code sequence of the desired user,
so that the error in the time alignment between the two sequences is less than one chip interval.
As described in Section 14.7, acquisition of the desired user code sequence is generally obtained by a
sequential searching algorithm that, at each step, delays the local code generator by a fraction of a chip,
typically half a chip, and determines the correlation between the signals {xm } and {c(u) m }; the search
terminates when the correlation level exceeds a certain threshold value, indicating that the desired time
alignment is attained. Following the acquisition process, a tracking algorithm is used to achieve, in
the steady state, a time alignment between the signals {xm } and {c(u) m } that has the desired accuracy;
the more commonly used tracking algorithms are the delay-locked loop and the tau-dither loop. The

1
This observation must not be confused with the distinction between the use of short (of period ≃ 215 ) or long (of period
≃ 242 ) PN scrambling sequences, which are employed to identify the base stations or the users and to synchronize the
system [8].
10.1. Spread-spectrum techniques 491

synchronization method also suggests the use of PN sequences as user code sequences. In practice,
the chip frequency is limited to values of the order of hundreds of Mchip/s because of the difficulty
in obtaining an accuracy of the order of a fraction of a nanosecond in the synchronization of the code
generator. In turn, this determines the limit in the bandwidth of a DS signal.

10.1.2 Frequency hopping systems

The FH spread-spectrum technique is typically used for the spreading of multiple frequency shifting
keying (M-FSK) signals. We consider an M-FSK signal [9], with carrier frequency f0 expressed in com-
∑+∞
plex form, i.e. we consider the analytic signal, as A ej2𝜋(f0 +Δf (t))t , where Δf (t) = k=−∞ ak 𝚠T (t − kT),
with {ak } sequence of i.i.d. symbols taken from the alphabet  = {−(M − 1), … , −1, +1, … , M − 1},
at the symbol rate 1∕T. An FH/M-FSK signal is obtained by multiplying the M-FSK signal by
∑
+∞
cFH (t) = ej(2𝜋f0,i t+𝜑0,i ) 𝚠Thop (t − iThop ) (10.34)
i=−∞

where {f0,i } is a pseudorandom sequence that determines shifts in frequency of the FH/M-FSK signal,
{𝜑0,i } is a sequence of random phases associated with the sequence of frequency shifts, and 𝚠Thop is a
rectangular window of duration equal to a hop interval Thop . In an FH/M-FSK system, the transmitted
signal is then given by
s(t) = Re [cFH (t) ej2𝜋(f0 +Δf (t))t ] (10.35)
In practice, the signal cFH is not generated at the transmitter; the transmitted signal s is obtained by
applying the sequence of pseudorandom frequency shifts {f0,i } directly to the frequency synthesizer that
generates the carrier at frequency f0 . With reference to the implementation illustrated in Figure 10.7,
segments of L consecutive chips from a PN sequence, not necessarily disjoint, are applied to a frequency
synthesizer that makes the carrier frequency hop over a set of 2L frequencies. As the band over which
the synthesizer must operate is large, it is difficult to maintain the carrier phase coherent between two
consecutive hops [10]; if the synthesizer is not equipped with any device to maintain a coherent phase,
it is necessary to include a random phase 𝜑0,i as in (10.34). In a time interval that is long with respect
to Thop , the bandwidth of the signal s, BSS , can be in practice of the order of several GHz. However, in
a short time interval during which no FH occurs, the bandwidth of an FH/M-FSK signal is the same as
the bandwidth of the M-FSK signal that carries the information, usually much lower than BSS .
Despreading, in this case also called dehopping, is ideally carried out by multiplying the received
signal r by a signal ĉ FH equal to that used for spreading, apart from the sequence of random phases
associated with the frequency shifts. For non-coherent demodulation, the sequence of random phases
can be modelled as a sequence of i.i.d. random variables with uniform probability density in [0, 2𝜋).
The operation of despreading yields the signal x, given by the sum of the M-FSK signal, the noise and
possibly interference. The signal x is then filtered by a lowpass filter and presented to the input of the
receive section comprising a non-coherent demodulator for M-FSK signals. As in the case of DS systems,
the receiver must perform acquisition and tracking of the FH signal, so that the waveform generated by
the synthesizer for dehopping reproduces as accurately as possible the signal cFH .

Classification of FH systems
FH systems are traditionally classified according to the relation between Thop and T. Fast frequency-
hopped (FFH) systems are characterized by one or more frequency hops per symbol interval, that is
T = NThop , N integer, and slow frequency-hopped (SFH) systems are characterized by the transmission
of several symbols per hop interval, that is Thop = NT.
492 Chapter 10. Spread-spectrum systems

{ak} {âk}
mod. frequency frequency demod.
channel
M-FSK synthesizer dehopper M-FSK

PN code
PN code PN code
acquisition
generator generator
and tracking

(a)

frequency

time
(b)

Figure 10.7 (a) Block diagram of an FH/M-FSK system. (b) Time-frequency allocation.

Moreover, a chip frequency Fchip is defined also for FH systems, and is given by the largest value
among Fhop = 1∕Thop and F = 1∕T. Therefore, the chip frequency Fchip corresponds to the highest
among the clock frequencies used by the system. The frequency spacing between tones of an FH/M-FSK
signal is related to the chip frequency and is therefore determined differently for FFH and SFH systems.

SFH systems For SFH systems, Fchip = F, and the spacing between FH/M-FSK tones is equal to the
spacing between the M-FSK tones themselves. In a system that uses a non-coherent receiver for M-FSK
signals, orthogonality of tones corresponding to M-FSK symbols is obtained if the frequency spacing
is an integer multiple of 1∕T. Assuming the minimum spacing is equal to F, the bandwidth BSS of an
FH/M-FSK signal is partitioned into Nf = BSS ∕F = BSS ∕Fchip subbands with equally spaced centre fre-
quencies; in the most commonly used FH scheme, the Nf tones are grouped into Nb = Nf ∕M adjacent
bands without overlap in frequency, each one having a bandwidth equal to MF = MFchip , as illustrated
in Figure 10.8. Assuming M-FSK modulation symmetric around the carrier frequency, the centre fre-
quencies of the Nb = 2L bands represent the set of carrier frequencies generated by the synthesizer, each
associated with an L-uple of binary symbols. According to this scheme, each of the Nf tones of the
FH/M-FSK signal corresponds to a unique combination of carrier frequency and M-FSK symbol.
In a different scheme, that yields a better protection against an intentional jammer using a sophis-
ticated disturbance strategy, adjacent bands exhibit an overlap in frequency equal to (M − 1)Fchip as
illustrated in Figure 10.9. Assuming that the centre frequency of each band corresponds to a possible
carrier frequency, as all Nf tones except (M − 1) are available as centre frequencies, the number of car-
rier frequencies increases from Nf ∕M to Nf − (M − 1), which for Nf ≫ M represents an increase by a
factor M of the randomness in the choice of the carrier frequency.

FFH systems For FFH systems, where Fchip = Fhop , the spacing between tones of an FH/M-FSK
signal is equal to the hop frequency. Therefore, the bandwidth of the spread-spectrum signal is parti-
tioned into a total of Nf = BSS ∕Fhop = BSS ∕Fchip subbands with equally spaced centre frequencies, each
10.2. Applications of spread-spectrum systems 493

BSS

MF MF MF MF MF

1 2 3 4 5 6 7 8 4i − 3 4i − 2 4i − 1 4i Nf – 3 Nf – 2 Nf – 1 Nf = 4Nb
F

1 2 i Nb

frequency

Figure 10.8 Frequency distribution for an FH/4-FSK system with bands non-overlapping in
frequency; the dashed lines indicate the carrier frequencies.

BSS

MF
MF
MF MF
MF MF MF

1 2 3 4 5 6 7 8 4i − 3 4i − 2 4i − 1 4i Nf – 3 Nf – 2 Nf – 1 Nf = 4Nb
F

frequency

Figure 10.9 Frequency distribution for an FH/4-FSK system with bands overlapping in frequency.

corresponding to a unique L-uple of binary symbols. Because there are Fhop ∕F hops per symbol, the
metric used to decide upon the symbol with a non-coherent receiver is suitably obtained by summing
Fhop ∕F components of the received signal.

10.2 Applications of spread-spectrum systems

The most common applications of spread-spectrum systems, that will be discussed in the next sections,
may be enumerated as follows.
494 Chapter 10. Spread-spectrum systems

1. Multiple access. In alternative to frequency division multiple access (FDMA) and time division
multiple access (TDMA) systems, introduced in Appendix 18.B, spread-spectrum systems allow
the simultaneous transmission of messages by several users over the channel, as discussed in
Section 10.1.1.
2. Narrowband interference rejection. We consider the DS case. Because interference is introduced
in the channel after signal spreading, at the receiver the despreading operation compresses the
bandwidth of the desired signal to the original value, and at the same time it expands by the same
factor the bandwidth of the interference, thus reducing the level of the interference PSD. After
demodulation, the ratio between the desired signal power and the interference power is therefore
larger than that obtained without spreading the signal spectrum.
3. Robustness against fading. Widening the signal bandwidth allows the exploitation of multipath
diversity of a radio channel affected by fading. Applying a DS spread-spectrum technique, intu-
itively, has the effect of modifying a channel model that is adequate for transmission of narrowband
signals in the presence of flat fading or multipath fading with a few rays, to a channel model with
many rays. Using a receiver that combines the desired signal from the different propagation rays,
the power of the desired signal at the decision point increases. In an FH system, on the other hand,
we obtain diversity in the time domain, as the channel changes from one hop interval to the next.
The probability that the signal is affected by strong fading during two consecutive hop intervals
is usually low. To recover the transmitted message in a hop interval during which strong fading is
experienced, error correction codes with very long interleaver and automatic repeat request (ARQ)
schemes are used (see Chapter 11).

10.2.1 Anti-jamming

Narrowband interference We consider the baseband equivalent signals of an M-QAM passband com-
munication system with symbol rate F = 1∕T, transmitted signal power equal to 𝙼s , and PSD with
minimum bandwidth, i.e. s (f ) = Es rect(f ∕F), where Es F = 𝙼s .
We consider now the application of a DS spread-spectrum modulation system. Due to spreading, the
bandwidth of the transmitted signal is expanded from F to BSS = NSF F. Therefore, for the same trans-
mitted signal power, the PSD of the transmitted signal becomes s′ (f ) = (Es ∕NSF )rect(f ∕BSS ), where
Es ∕NSF = 𝙼s ∕BSS . We note that spreading has decreased the amplitude of the PSD by the factor NSF , as
illustrated in Figure 10.10.

Es M-QAM

Es
NSF M-QAM + spread-spectrum

b 1 frequency
T

Bss = NSF 1
T

Figure 10.10 Power spectral density of an M-QAM signal with minimum bandwidth and of a
spread-spectrum M-QAM signal with spreading factor NSF = 4.
10.2. Applications of spread-spectrum systems 495

In the band of the spread-spectrum signal, in addition to AWGN with PSD N0 , we assume the channel
introduces an additive interference signal or jammer with power 𝙼𝚓 , uniformly distributed on a bandwidth
B𝚓 , with B𝚓 < 1∕T.
With regard to the operation of despreading, we consider the signals after the multiplication by the
user code sequence. The interference signal spectrum is expanded and has a PSD equal to 𝚓′ (f ) =
N𝚓 rect(f ∕BSS ), with N𝚓 = 𝙼𝚓 ∕BSS . The noise, that originally has a uniformly distributed power over all
the frequencies, still has PSD equal to N0 , i.e. spreading has not changed the PSD of the noise.
At the output of the despreader, the desired signal exhibits the original PSD equal to Es rect(f ∕F).
Modelling the despreader filter as an ideal lowpass filter with bandwidth 1∕(2T), for the SNR 𝛾 at the
decision point the following relation holds:
( ) Es 𝙼s ∕F
1
𝙼a 𝛾 = = (10.36)
2 N0 + N𝚓 (N0 + 𝙼𝚓 ∕BSS )
In practice, performance is usually limited by interference and the presence of white noise can be
ignored. Therefore, assuming N𝚓 ≫ N0 , (10.36) becomes
( ) E 𝙼 ∕F 𝙼 B
1
𝙼a 𝛾 ≃ s = s = s SS (10.37)
2 N𝚓 𝙼𝚓 ∕BSS 𝙼𝚓 F
where 𝙼s ∕𝙼𝚓 is the ratio between the power of the desired signal and the power of the jammer, and BSS ∕F
is the spreading ratio NSF also defined as the processing gain of the system.
The above considerations are now defined more precisely in the following case.

Sinusoidal interference We assume that the baseband equivalent received signal is expressed as
rC (t) = s(t) + 𝚓(t) + 𝑤C (t) (10.38)
where s is a DS signal given by (10.9) with amplitude A(u) = 1, 𝑤C is AWGN with spectral density N0 ,
and the interferer is given by
𝚓(t) = A𝚓 ej𝜑 (10.39)
√
In (10.39), A𝚓 = 𝙼𝚓 is the amplitude of the jammer and 𝜑 a random phase with uniform distribution in
√
[0, 2𝜋). We also assume a minimum bandwidth transmit pulse, hTx (t) = Eh ∕Tchip sinc(t∕Tchip ), hence
√
gM (t) = hTx (t), and M (0) = Eh Tchip .
For the coherent receiver of Figure 10.4, at the detection point the sample at instant kT is given by
∑
NSF −1
yk = NSF Eh ak + 𝑤k + A𝚓 ej𝜑 M (0) c∗j+kN (10.40)
SF
j=0

Modelling the sequence {c∗kN , c∗kN +1 , … , c∗kN +N −1 } as a sequence of i.i.d. random variables, the
SF SF SF SF
variance of the summation in (10.40) is equal to NSF , and the ratio 𝛾 is given by
(NSF Eh )2
𝛾= (10.41)
(NSF N0 Eh + 𝙼𝚓 Eh Tchip NSF )∕2
Using (10.8) and the relation Es = 𝙼s T, we get
( )
1 1
𝙼a 𝛾 = (10.42)
2 N0 ∕Es + 𝙼𝚓 ∕(NSF 𝙼s )
We note that in the denominator of (10.42) the ratio 𝙼𝚓 ∕𝙼s is divided by NSF . Recognizing that 𝙼𝚓 ∕𝙼s is
the ratio between the power of the jammer and the power of the desired signal before the despreading
operation, and that 𝙼𝚓 ∕(NSF 𝙼s ) is the same ratio after the despreading, we find that, in analogy with the
496 Chapter 10. Spread-spectrum systems

previous case of narrowband interference, also in the case of a sinusoidal jammer, the use of the DS
technique reduces the effect of the jammer by a factor equal to the processing gain.

10.2.2 Multiple access

Spread-spectrum multiple-access communication systems represent an alternative to TDMA or FDMA
systems and are normally referred to as CDMA systems (see Sections 18.B and 10.1.1). With CDMA, a
particular spreading sequence is assigned to each user for accessing the channel; unlike FDMA, where
users transmit simultaneously over non-overlapping frequency bands, or TDMA, where users transmit
over the same band but in disjoint time intervals, users in a CDMA system transmit simultaneously over
the same frequency band.
Because in CDMA systems correlation receivers are usually employed, it is important that the spread-
ing sequences are characterized by low cross-correlation values. We have already observed that CDMA
systems may be classified as synchronous or asynchronous. In the first case, the symbol transition instants
of all users are aligned; this allows the use of orthogonal sequences as spreading sequences and con-
sequently the elimination of interference caused by one user signal to another; in the second case, the
interference caused by multiple access limits the channel capacity, but the system design is simplified.
CDMA has received particular interest for applications in wireless communications systems, for
example cellular radio systems, personal communications services (PCS), and wireless local-area
networks; this interest is mainly due to performance that spread-spectrum systems achieve over channels
characterized by multipath fading.
Other properties make CDMA interesting for application to cellular radio systems, for example the
possibility of applying the concept of frequency reuse (see Section 18.1.1). In cellular radio systems
based on FDMA or TDMA, to avoid excessive levels of interference from one cell onto neighbouring
cells, the frequencies used in one cell are not used in neighbouring cells. In other words, the system is
designed so that there is a certain spatial separation between cells that use the same frequencies. For
CDMA, this spatial separation is not necessary, making it possible, in principle, to reuse all frequencies.

10.2.3 Interference rejection

Besides the above described properties, that are relative to the application in multiple-access systems,
the robustness of spread-spectrum systems in the presence of narrowband interferers is key in other
applications, for example in systems where interference is unintentionally generated by other users that
transmit over the same channel. We have CCI when a certain number of services are simultaneously
offered to users transmitting over the same frequency band. Although in these cases some form of spatial
separation among signals interfering with each other is usually provided, for example by using directional
antennas, it is often desirable to use spread-spectrum systems for their inherent interference suppression
capability. In particular, we consider a scenario in which a frequency band is only partially occupied
by a set of narrowband conventional signals; to increase the spectral efficiency of the system, a set of
spread-spectrum signals can simultaneously be transmitted over the same band, thus two sets of users can
access the transmission channel. Clearly, this scheme can be implemented only if the mutual interference,
which a signal set imposes on the other, remains within tolerable limits.

10.3 Chip matched filter and rake receiver

Before introducing a structure that is often employed in receivers for DS spread-spectrum signals, we
make the following considerations on the radio channel model introduced in Section 4.1.
10.3. Chip matched filter and rake receiver 497

Number of resolvable rays in a multipath channel

We want to represent a multipath radio channel with a number of rays having gains modelled as complex
valued, Gaussian uncorrelated random processes. From (4.64) and (4.65), with some simplification in
the notation, a time-invariant channel impulse response is approximated as
Nc,∞ −1
∑
gC (𝜏) = gi 𝛿(𝜏 − iTQ ) (10.43)
i=0

where delays 𝜏i = iTQ are approximated as multiples of a sufficiently small period TQ .

Hence, from (10.43), the channel output signal sC is related to the input signal s by
Nc,∞ −1
∑
sC (t) = gi s(t − iTQ ) (10.44)
i=0

Now the number of resolvable or uncorrelated rays in (10.44) is generally less than Nc,∞ and is related
to the bandwidth B of s as the uncorrelated rays are spaced by a delay of the order of 1∕B. Consequently,
for a channel with a delay spread 𝜏rms and bandwidth B ∝ 1∕Tchip , the number of resolvable rays is
𝜏rms
Nc,res ∝ (10.45)
Tchip

Using the notion of channel coherence bandwidth, Bcoh ∝ 1∕𝜏rms , (10.45) may be rewritten as
B
Nc,res ∝ (10.46)
Bcoh
We now give an example that illustrates the above considerations. Let {gi } be a realization of the chan-
nel impulse response with uncorrelated coefficients having a given power delay profile; the infinite
bandwidth of the channel will be equal to B = 1∕(2TQ ). We now filter {gi } with two filters having,
respectively, bandwidth B = 0.1∕(2TQ ) and B = 0.01∕(2TQ ), and we compare the three pulse shapes
given by the input sequence and the two output sequences. We note that the output obtained in corre-
spondence of the filter with the narrower bandwidth has fewer resolvable rays. In fact, in the limit for
B → 0 the output is modelled as a single random variable.
Another way to derive (10.45) is to observe that, for t within an interval of duration 1∕B, s does not
vary much. Therefore, letting
Nc,∞
Ncor = (10.47)
Nc,res

(10.44) can be written as

Nc,res −1
∑
sC (t) = gres,j s(t − jNcor TQ ) (10.48)
j=0

and the gains of the resolvable rays are

∑
Ncor −1
gres,j ≃ gi+jNcor (10.49)
i=0

In summary, assuming the symbol period T is given and DS spread-spectrum modulation is adopted,
the larger NSF the greater the resolution of the radio channel, that is, the channel can be modelled with
a larger number of uncorrelated rays, with delays of the order of Tchip .
498 Chapter 10. Spread-spectrum systems

Chip matched filter

We consider the transmission of a DS signal (10.9) for U = 1 on a dispersive channel as described by
(10.48). The receiver that maximizes the ratio between the amplitude of the pulse associated with the
desired signal sampled with sampling rate 1∕Tchip and the standard deviation of the noise is obtained by
the filter matched to the received pulse, named chip matched filter (CMF). We define
qC (t) = (hTx ∗ gC ∗ gAA )(t) (10.50)
and let gM (t) = q∗C (t0 − t) be the corresponding matched filter. In practice, at the output of the filter
gAA an estimate of qC with sampling period Tc = Tchip ∕2 is evaluated,2 which yields the corresponding
discrete-time matched filter with sampling period of the input signal equal to Tc and sampling period of
the output signal equal to Tchip (see Figure 10.11).

t 0ʹ +nTc g (t)=q * (t −t) despreader

M C 0
a^k
rC (t) (1)
rAA (t) rAA,n xm yk
g
AA
g
M 2 g (1)
ds
Tchip Tchip T T
Tc =
2

Figure 10.11 Chip matched filter receiver for a dispersive channel.

If qC is sparse, that is, it has a large support but only a few non-zero coefficients, for the realization of
gM , we retain only the coefficients of qC with larger amplitude; it is better to set to zero the remaining
coefficients because their estimate is usually very noisy (see observation 3 at page 383).
Figure 10.12a illustrates in detail the receiver of Figure 10.11 for a filter gM with at most NMF coeffi-
cients spaced of Tc = Tchip ∕2. If we now implement the despreader on every branch of the filter gM , we
obtain the structure of Figure 10.12b. We observe that typically only 3 or 4 branches are active, that is
they have a coefficient gM,i different from zero.
Ideally, for an overall channel with Nres resolvable paths, we assume
∑
Nres
qC (t) = qC,i 𝛿(t − 𝜏i ) (10.51)
i=1
hence
∑
Nres
gM (t) = q∗C,j 𝛿(t0 − t − 𝜏j ) (10.52)
j=1

Defining
tM,j = t0 − 𝜏j , j = 1, … , Nres (10.53)
the receiver scheme, analogous to that of Figure 10.12b, is illustrated in Figure 10.13.
To simplify the analysis, we assume that the spreading sequence is a PN sequence with NSF sufficiently
large, such that the following approximations hold: (1) the autocorrelation of the spreading sequence is
a Kronecker delta and (2) the delays {𝜏i } are multiples of Tchip .
From (10.51), in the absence of noise, the signal rAA is given by
∑
Nres
∑
+∞
∑
NSF −1
rAA (t) = qC,n a(1)
i
c(1)
𝓁+iN
𝛿(t − 𝜏n − (𝓁 + iNSF ) Tchip ) (10.54)
SF
n=1 i=−∞ 𝓁=0

2
To determine the optimum sampling phase t0 , usually rAA is oversampled with a period TQ such that Tc ∕TQ = 2 or 4 for
Tc = Tchip ∕2; among the 2 or 4 estimates of gC obtained with sampling period Tc , the one with the largest energy is
selected (see Observation 7.6 on page 320).
10.3. Chip matched filter and rake receiver 499

r
AA,2m
Tc Tc ... Tc
Tchip
Tc =
2 g g g g
M,0 M,1 M,NMF−2 M,N −1
MF

xm yk
2 g (1) ^a (1)
ds k
Tchip T

(a)

r
AA,2N SF k
Tc ... Tc
Tchip
Tc =
2
2 2

Tchip Tchip

g (1) ... g (1)

despreader ds ds

T T
g g
M,0 M,NMF −1

yk
^a (1)
k
T
(b)

Figure 10.12 Two receiver structures: (a) chip matched filter with despreader, (b) rake.

and the output of the sampler on branch j is given by3

∑
Nres
∑
+∞
∑
NSF −1
xj,m = qC,n a(1) c(1)
𝓁+iN
𝛿m+ 𝜏j −𝜏n −(𝓁+iN (10.55)
i SF Tchip SF )
n=1 i=−∞ 𝓁=0

Correspondingly the despreader output, assuming 𝚛c(1) (nD ) = 𝛿nD and the absence of noise, yields the
signal NSF a(1)
k
qC,j . The contributions from the various branches are then combined according to the
maximum-ratio-combining (MRC) technique (see Section 9.2.1) to yield the sample
( )
∑
Nres
yk = NSF |qC,n | a(1)
2
k
(10.56)
n=1

3
Instead of using the Dirac delta in (10.51), a similar analysis assumes that (1) gAA (t) = h∗Tx (−t) and (2) 𝚛hTx is a Nyquist
pulse. The result is the same as (10.55).
500 Chapter 10. Spread-spectrum systems

rAA (t)

delay tM,Nres tM,1

t0 + m Tchip
Tchip xNres, m Tchip x1,m

despreader g (1) g (1)

ds ... ds

T T
* res
qC,N q*,1
C

yk
a^k
(1)
T

Figure 10.13 Rake receiver for a channel with Nres resolvable paths.

∑Nres
where EqC = n=1 |qC,n |2 is the energy per chip of the overall channel impulse response.
The name rake originates from the structure of the receiver that is similar to a rake with Nres fingers.
In practice, near the rake receiver a correlator estimates the delays, with precision Tchip ∕2, and the gains
of the various channel rays. The rake is initialized with the coefficients of rays with larger gain. The
delays and the coefficients are updated whenever a change in the channel impulse response is observed.
However, after the initialization has taken place, on each finger of the rake the estimates of the amplitude
and of the delay of the corresponding ray may be refined by using the correlator of the despreader,
as indicated by the dotted line in Figure 10.13. We note that if the channel is static, the structure of
Figure 10.12a with Tc = Tchip ∕2 yields a sufficient statistic.

10.4 Interference

For a dispersive channel and in the case of U users, we evaluate the expression of the signal yk at the
decision point using the matched filter receiver of Figure 10.11.
Similarly to (10.50), we define

q(u)
C
(t) = (hTx ∗ g(u)
C
∗ gAA )(t), u = 1, … , U (10.57)

and let
g(𝑣)
M
(t) = q(𝑣)∗
C
(t0 − t), 𝑣 = 1, … , U (10.58)

be the corresponding matched filter. Moreover, we introduce the correlation between q(u)
C
and q(𝑣)
C
,
expressed by
𝚛q(u) q(𝑣) (𝜏) = (q(u)
C
(t) ∗ q(𝑣)∗
C
(−t))(𝜏) (10.59)
C C
10.4. Interference 501

Assuming without loss of generality that the desired user signal has the index u = 1, we have
∑
U
∑
+∞
∑
NSF −1
xm = A(u) a(u)
i
c(u)
𝓁+iN
𝚛q(u) q(1) ((m − 𝓁 − iNSF ) Tchip ) + 𝑤̃ m (10.60)
SF C C
u=1 i=−∞ 𝓁=0

where 𝑤̃ m is given by (10.20).

At the despreader output, we obtain
∑
NSF −1
yk = xj+kNSF c(1)∗
j+kN
+ 𝑤k
SF
j=0

∑
U
∑
+∞
∑
NSF −1 NSF −1
∑ (10.61)
= A(u) a(u)
i
c(u)
𝓁+iN SF
u=1 i=−∞ 𝓁=0 j=0

𝚛q(u) q(1) ((j − 𝓁 + (k − i) NSF ) Tchip )c(1)∗

j+kN
+ 𝑤k
C C SF

where 𝑤k is defined in (10.28).

Introducing the change of index n = 𝓁 − j and recalling the definition of cross-correlation between
two code sequences (10.26), the double summation in 𝓁 and j in (10.61) can be written as
∑
−1
(NSF − |n|)𝚛q(u) q(1) ((−n + (k − i) NSF )Tchip )𝚛∗c(1) c(u)
(−n)
C C
n=−(NSF −1)
(10.62)
∑
NSF −1
+ (NSF − |n|)𝚛q(u) q(1) ((−n + (k − i) NSF )Tchip )𝚛c(u) c(1) (n)
C C
n=0

where, to simplify the notation, we have assumed that the user code sequences are periodic of period
NSF .
The desired term in (10.61) is obtained for u = 1; as 𝚛∗c(1) (−n) = 𝚛c(1) (n), it has the following expression:

∑
+∞
∑
NSF −1
A(1) a(1)
i
(NSF − |n|) 𝚛c(1) (n) 𝚛q(1) ((−n + (k − i) NSF ) Tchip ) (10.63)
C
i=−∞ n=−(NSF −1)

Consequently, if the code sequences are orthogonal, that is

𝚛c(1) (n) = 𝛿n (10.64)
and in the absence of ISI, that is
𝚛q(1) (iNSF Tchip ) = 𝛿i Eq(1) (10.65)
C C

where Eq(1) is the energy per chip of the overall pulse at the output of the filter gAA , then the desired term
C
(10.63) becomes
A(1) NSF EqC a(1)k
(10.66)
which coincides with the case of an ideal AWGN channel (see (10.27)). Note that using the same assump-
tions we find the rake receiver behaves as an MRC (see (10.56)).
If (10.64) is not verified, as it happens in practice, and if
𝚛q(1) (nTchip ) ≠ 𝛿n EqC (10.67)
C

the terms for n ≠ 0 in (10.63) give rise to ISI, in this context also called inter-path interference (IPI).
Usually, the smaller NSF the larger the IPI. We note however that if the overall pulse at the output of the
CMF is a Nyquist pulse, that is
𝚛q(1) (nTchip ) = 𝛿n EqC (10.68)
C

then there is no IPI, even if (10.64) is not verified.

502 Chapter 10. Spread-spectrum systems

With reference to (10.62) we observe that, in the multiuser case, if 𝚛c(u) c(1) (n) ≠ 0 then yk is affected
by MUI, whose value increases as the cross-correlation between the pulses q(u) C
and q(1)
C
increases.

Detection strategies for multiple-access systems

For detection of the user messages in CDMA systems, we make a distinction between two classes of
receivers: single-user and multiuser. In the first class, the receivers focus on detecting the data from a sin-
gle user, and the other user signals are considered as additional noise. In the second class, the receivers
seek to simultaneously detect all U messages. The performance of the multiuser receivers is substantially
better than that of the single-user receivers, achieved at the expense of a higher computational complex-
ity. Using as front-end a filter bank, where the filters are matched to the channel impulse responses of
the U users, the structures of single-user and multiuser receivers are exemplified in Figure 10.14.

(a) x m(1)
g (1) detector ^a (1)
M Tchip k

rC (t)
...

...
x m(U)
g (U) detector ^a (U )
M k
Tchip

g (i) (t) = q (i)* (t (i) −t)

M C 0

(b) x m(1)
g (1) ^a (1)
M Tchip multi k
user
rC (t)
...

...

detector
x m(U) ^a (U )
g (U)
M k
Tchip

Figure 10.14 (a) Single-user receiver, and (b) multiuser receiver.

10.5 Single-user detection

We consider two equalizers for single-user detection.

Chip equalizer
To mitigate the interference in the signal sampled at the chip rate, after the CMF (see (10.68)) a zero
forcing (ZF) or an mean square error (MSE) equalizer (denoted as chip equalizer, CE) can be used
[11–14]. As illustrated in Figure 10.15, let gCE be the equalizer filter with output {d̃ m }. For an MSE
criterion, the cost function is given by
J = E [|d̃ m − dm |2 ] (10.69)
10.5. Single-user detection 503

wC (t)
t0ʹ +nTc
(1)
dm A
(1) ~
dm yk a^ k−D
(1)
hTx *g (1) g g 2 g (1)
C AA CE ds
Tchip Tchip Tchip T T
Tc =
2
...

(U) (U)
dm A hTx *g (U)
C
Tchip

Figure 10.15 Receiver as a fractionally-spaced chip equalizer.

where {dm } is assumed i.i.d.. We distinguish the two following cases.

All code sequences are known This is the case that may occur for downlink transmission in wireless
networks. Then, g(u)
C
(t) = gC (t), u = 1, … , U, and we assume

∑
U
dm = dm(u) (10.70)
u=1

that is, for the equalizer design, all user signals are considered as desired signals.

Only the code sequence of the desired user signal is known In this case, we need to assume
dm = dm(1) (10.71)

The other user signals are considered as white noise, with overall PSD Ni , that is added to 𝑤C .
From the knowledge of q(1) C
and the overall noise PSD, the minimum of the cost function defined
in (10.69) is obtained by following the same steps developed in Chapter 7. Obviously, if the level of
interference is high, the solution corresponding to (10.71) yields a simple CMF, with low performance
whenever the residual interference (MUI and IPI) at the decision point is high.
A better structure for single-user detection is obtained by the following approach.

Symbol equalizer
Recalling that we adopt the transmitter model of Figure 10.3, and that we are interested in the message
{a(1)
k
}, the optimum receiver with linear filter, symbol equalizer (SE), g(1)
SE
is illustrated in Figure 10.16.

wC (t)
t0ʹ +nTc
a (1) A (1) rAA,n yk a^ k−D
(1)
k hT(1) *g(1) g g (1) 2NSF
C AA SE
T Tchip T T
Tc =
2
...

a (U)
k A(U)
h(U
T
) (U )
*g
C
T

Figure 10.16 Receiver as a fractionally-spaced symbol equalizer.

504 Chapter 10. Spread-spectrum systems

The cost function is now given by [15–17]

J = E[|yk − a(1)
k−D
|2 ] (10.72)

Note that g(1)

SE
, that includes also the function of despreading, depends on the code sequence of the desired
user. Therefore, the length of the code sequence is usually not larger than NSF , otherwise we would find
a different solution for every symbol period, even if g(1) C
is time invariant. Moreover, in this formulation,
the other user signals are seen as interference, and one of the tasks of gSE is to mitigate the MUI.
In an adaptive approach, for example using the least mean square (LMS) algorithm, the solution is
simple to determine and does not require any particular a priori knowledge, except the training sequence
in {a(1)
k
} for initial convergence. On the other hand, using a direct approach, we need to identify the
autocorrelation of rAA,n and the cross-correlation between rAA,n and a(1) k−D
. As usual these correlations
are estimated directly or, assuming the messages {a(u) k
}, u = 1, … , U, are i.i.d. and independent of each
other, they can be determined using the knowledge of the various pulses {hT } and {g(u) (u)
C
}, that is the
channel impulse responses and code sequences of all users; for the special case of downlink transmission,
the knowledge of the code sequences is sufficient, as the channel is common to all user signals.

10.6 Multiuser detection

Multiuser detection techniques are essential for achieving near-optimum performance in communica-
tion systems where signals conveying the desired information are received in the presence of ambient
noise plus MUI. The leitmotiv of developments in multiuser detection is represented by the reduction in
complexity of practical receivers with respect to that of optimal receivers, which is known to increase
exponentially with the number of active users and with the delay spread of the channel, while achiev-
ing near-optimum performance. A further element that is being recognized as essential to reap the full
benefits of interference suppression is the joint application of multiuser detection with other techniques
such as spatial-temporal processing and iterative decoding.

10.6.1 Block equalizer

Here we first consider the simplest among multiuser receivers. It comprises a bank of U filters g(u)
T
,
u = 1, … , U, matched to the impulse responses4
∑
NSF −1
q(u)
T
(t) = c(u)
𝓁
q(u)
C
(t − 𝓁 Tchip ), u = 1…,U (10.73)
𝓁=0

where the functions {q(u)C

(t)} are defined in (10.57). Decisions taken by threshold detectors on the U
output signals, sampled at the symbol rate, yield the detected user symbol sequences. It is useful to intro-
duce this receiver, that we denote as MF, as, substituting the threshold detectors with more sophisticated
detection devices, it represents the first stage of several multiuser receivers, as illustrated in general in
Figure 10.17.
We introduce the following vector notation. The vector of symbols transmitted by U users in a symbol
period T is expressed as
ak = [a(1)
k
, … , a(U)
k
]T (10.74)

4
We assume that the information on the power of the user signals is included in the impulse responses g(u)
C
, u = 1, … , U,
so that A(u) = 1, u = 1, … , U.
10.6. Multiuser detection 505

(1)
t 0 + m Tchip
y (1)
k ^a (1)
g (1) g (1)
M ds multi k
Tchip T
user
rC (t)

...

...

...
(U)
t 0 + m Tchip detector
y (U )
k ^a (U)
g (U ) (U)
g ds
M k
Tchip T

Figure 10.17 Receiver as MF and multiuser detector.

and the vector that carries the information on the codes and the channel impulse responses of the U
users is expressed as
qT (t) = [q(1)
T
(t), … , q(U)
T
(t)]T (10.75)
Joint detectors constitute an important class of multiuser receivers. They effectively mitigate both
ISI and MUI, exploiting the knowledge of the vector qT . In particular, we consider now block linear
receivers: as the name suggests, a block linear receiver is a joint detector that recovers the information
contained in a window of K symbol periods. Let
a = [aT0 , … , aTK−1 ]T
(10.76)
= [a(1)
0
, … , a(U)
0
, … , a(1)
K−1
, … , a(U)
K−1
]T
be the information transmitted by U users and let y be the corresponding vector of KU elements at the
MF output. We define the following correlations:
( ) |
𝚛q(u,𝑣) (k) = q(u) T
(t) ∗ q (𝑣)∗
T
(−t) (𝜏)|| (10.77)
|𝜏=kT
Assuming
𝚛q(u,𝑣) (k) = 0 for |k| > 𝜈 (10.78)
with 𝜈 < K, and following the approach in [18–21], we introduce the KU × KU block banded matrix
⎡R(0) … R(−𝜈) 𝟎 … 𝟎 ⎤
⎢ ⋮ ⋱ ⋱ ⋱ ⋮ ⎥
⎢ ⎥
⎢ ⋮ ⋱ 𝟎 ⎥
⎢R(𝜈) R(−𝜈)⎥
T=⎢
⋮ ⎥⎥
(10.79)
⎢ 𝟎 ⋱
⎢ ⋮ ⋱ ⋱ ⋱ ⋮ ⎥
⎢ 𝟎 ··· 𝟎 R(𝜈) · · · R(0) ⎥
⎢ ⎥
⎣ ⎦
with
⎡ 𝚛q(1,1) (n) … 𝚛q(1,U) (n) ⎤
R(n) = ⎢ ⋮ ⋱ ⋮ ⎥ (10.80)
⎢ ⎥
⎣𝚛q(U,1) (n) … 𝚛q(U,U) (n)⎦
Let 𝒘 be the vector of noise samples at the MF output. It is immediate to verify that its covariance
matrix is N0 T. Then the matrix T is Hermitian and, assuming that it is definite positive, the Cholesky
decomposition (2.138) can be applied
T = LH L (10.81)
where LH is a lower triangular matrix with positive real elements on the main diagonal.
506 Chapter 10. Spread-spectrum systems

Using (10.76) and (10.79), we find that the vector y satisfies the linear relation
y=T a+𝒘 (10.82)
Once the expression (10.82) is obtained, the vector a can be detected by well-known techniques [21].
Applying the zero-forcing criterion, at the decision point we get the vector
z = T −1 y
(10.83)
= a + T −1 𝒘
Equation (10.83) shows that the zero-forcing criterion completely eliminates both ISI and MUI, but it
may enhance the noise.
Applying instead the MSE criterion to the signal y leads to the solution (see (2.198))
z = (T + N0 I)−1 y (10.84)
Both approaches require the inversion of a KU × KU Hermitian matrix and therefore a large com-
putational complexity. A scheme that is computationally efficient while maintaining comparable per-
formance is described in [22]. A minimum mean square error (MMSE) method with further reduced
complexity operates on single output samples, that is for K = 1. However, the performance is lower
because it does not exploit the correlation among the different observations.

10.6.2 Interference cancellation detector

The block equalizer operates linearly on the received signal. Non-linear techniques, based on detec-
tion and interference cancellation can actually provide better results, at the cost of a higher complexity.
For the case K = 1, an example of non-linear detection is represented by a decision feedback equal-
izer (DFE) structure (see Section 7.4.4) that yields performance near the optimum maximum-likelihood
(ML) receiver. However, it comes at a very higher complexity of both design and implementation. We
now introduce two simpler non-linear techniques based on the principle of interference cancellation.

Successive interference cancellation

The first approach, denoted as successive interference cancellation (SIC) works at follows. Users’ sig-
nals are detected in U steps (according to a pre-determined order): suppose here for simplicity that the
sequence of user detection is simply 1, … , U, i.e. they are detected according to their index. At the first
step, user u = 1 is detected, using any single-user detection strategy. Then, its contribution to the received
signal is removed from the output of the despreader of user 2, i.e. we compute from (10.61)
∑
+∞
∑ ∑
NSF −1 NSF −1
y(2)
k
= yk − A(1) â (1)
i
c(1)
𝓁+iN SF
i=−∞ 𝓁=0 j=0

𝚛q(1) q(2) ((j − 𝓁 + (k − i) NSF ) Tchip )c(1)∗

j+kN
C C SF
(10.85)
∑
U
∑
+∞
∑ ∑
NSF −1 NSF −1
= A(u) a(u)
i
c(u)
𝓁+iNSF
u=2 i=−∞ 𝓁=0 j=0

𝚛q(u) q(2) ((j − 𝓁 + (k − i) NSF ) Tchip )c(2)∗

j+kN
+ 𝑤(2)
k
C C SF

where the second equation is obtained if detection of the signal of the first user was without errors, thus
â k(1) = a(1)
k
. Moreover, we also have

(p)
∑
NSF −1
(p)∗
𝑤k = 𝑤̃ j+kNSF cj+kN (10.86)
SF
j=0
10.6. Multiuser detection 507

Then, single user detection is applied to detect the signal of the second user. At the generic step
p = 2, … , U of SIC, we remove from the output of despreader of user p the contribution of signals of
all previously detected users, i.e.
(p) (p−1)
+∞
∑ (p−1) ∑ ∑
NSF −1 NSF −1
(p−1)
y k = yk − A(p−1) â i c𝓁+iN
SF
i=−∞ 𝓁=0 j=0
(p−1)∗
𝚛q(p−1) qC
(p) ((j − 𝓁 + (k − i) NSF ) Tchip )cj+kN
C SF
(10.87)
∑U
∑
+∞
∑
NSF −1 NSF −1
∑
= A(u) a(u)
i
c(u)
𝓁+iNSF
u=p i=−∞ 𝓁=0 j=0
(p)∗ (p)
𝚛q(u) qC
(p) ((j − 𝓁 + (k − i) NSF ) Tchip )cj+kN + 𝑤k
C SF

and single-user detection is applied.

Note that the order of detection of the users can be optimized, typically starting with the detection
users with the highest signal-to-interference-plus-noise ratio (SINR) at the detector input.
The SIC receiver can also be applied on a single symbol. In this case, each single-user detector operates
on a single symbol and before detection of symbol k of user u, the interference generated by symbols at
time k′ < k of all users and of symbols at time k for users 𝓁 < u is removed. With this option we reduce
latency, while being in general less effective in cancelling the interference, as in this case
(p) ∑
U
∑
k
∑
NSF −1 NSF −1
∑
yk = A(u) a(u)
i
c(u)
𝓁+iN SF
u=p i=−∞ 𝓁=0 j=0
(p)∗
𝚛q(u) qC
(p) ((j − 𝓁 + (k − i) NSF ) Tchip )cj+kN +
C SF
(10.88)
∑
U
∑
+∞
∑
NSF −1 NSF −1
∑
+ A(u) a(u)
i
c(u)
𝓁+iNSF
u=1 i=k 𝓁=0 j=0
(p)∗ (p)
𝚛q(u) qC
(p) ((j − 𝓁 + (k − i) NSF ) Tchip )cj+kN + 𝑤k
C SF

Note that SIC follows a similar principle to that of the V-BLAST receiver for multiple-input-multiple-
output (MIMO) systems (see Section 9.2.3).

Parallel interference cancellation

The second approach, denoted as parallel interference cancellation (PIC) works as follows.
PIC operates iteratively on a single symbol for all users. In the first iteration, a single-user detector is
applied for all users, providing â (1,u)
k
, u = 1, … , U. At iteration i > 1, MUI is partially removed exploiting
detected symbols at iteration i − 1, to obtain
′ ∑
U
(i−1,p) ∑ ∑
NSF −1 NSF −1
(p)
y(i,u)
k
= y(u)
k
− A(p) â k c𝓁+kN
SF
p=1,p≠k 𝓁=0 j=0
(10.89)
𝚛q(p) q(u) ((j − 𝓁) Tchip )c(u)∗
j+kN
C C SF

′
where y(u)
k
is the signal at the output of the despreader of user k, where we have removed the interfer-
(p)
ence of symbols a𝓁 , with p = 1, … , U, and 𝓁 < k. Then, single-user detection is applied for each user,
to obtain â (i,u)
k
, u = 1, … , U. Note that errors in the detection process may lead to imperfect interfer-
ence cancellation at first iterations. However, in a well-designed system, typically the process converges
within few iterations to yield good detection performance.
508 Chapter 10. Spread-spectrum systems

10.6.3 ML multiuser detector

We now present two implementations of the ML multiuser detector.

Correlation matrix
Using the notation introduced in the previous section, the multiuser signal is
∑
U
rC (t) = s(u)
C
(t) + 𝑤C (t) (10.90)
u=1
∑
K−1
= aTi qT (t − iT) + 𝑤C (t) (10.91)
i=0

The log-likelihood associated with (10.90) is [23]

| ∑U |2
| (u) ||
|
𝓁C = − |rC (t) − sC (t)| dt (10.92)
∫ | |
| u=1 |
Defining the matrices
Qk1 −k2 = q∗T (t − k1 T) qTT (t − k2 T) dt (10.93)
∫
after several steps, (10.92) can be written as
{ [ 𝜈
]}
∑
K−1
∑
𝓁C = H
Re ak Q0 ak + 2Qm ak−m − 2yk (10.94)
k=0 m=1

where the generic term is the branch metric, having assumed that

Qm = 𝟎, |m| > 𝜈 (10.95)

We note that the first two terms within the brackets in (10.94) can be computed off-line.
The sequence {̂ak } that maximizes (10.94) can be obtained using the Viterbi algorithm (see
Section 7.5.1); the complexity of this scheme is however exceedingly large, because it requires O(42U𝜈 )
branch metric computations per detected symbol, assuming quadrature phase shift keying (QPSK)
modulation.

Whitening filter
We now derive an alternative formulation of the ML multiuser detector; for this reason, it is convenient to
express the MF output using the D transform, defined as the z-transform where D = z−1 (see Section 1.2)
[23]. Defining
𝜈
∑
Q(D) = Qk Dk (10.96)
k=−𝜈

the MF output can be written as

y(D) = Q(D) a(D) + 𝒘(D) (10.97)

where 𝒘(D) is the noisy term with matrix spectral density N0 Q(D). Assuming that it does not have poles
on the unit circle, Q(D) can be factorized as

Q(D) = FH (D−1 ) F(D) (10.98)

10.7. Multicarrier CDMA systems 509

where F(D) is minimum phase; in particular F(D) has the form

𝜈
∑
F(D) = Fk Dk (10.99)
k=0

where F0 is a lower triangular matrix. Now let 𝚪(D) = [FH (D−1 )]−1 , an anti-causal filter by construction.
Applying 𝚪(D) to y(D) in (10.97), we get
z(D) = 𝚪(D) y(D)
(10.100)
= F(D) a(D) + 𝒘′ (D)
where the noisy term 𝒘′ (D) is a white Gaussian process. Consequently, in the time domain (10.100)
becomes
𝜈
∑
zk = Fm ak−n + 𝒘′k (10.101)
m=0

With reference to [24], the expression (10.101) is an extension to the multidimensional case of Forney’s
maximum-likelihood sequence detector (MLSD) approach. In fact, the log-likelihood can be expressed
as the sum of branch metrics defined as
‖ 𝜈
∑ ‖2
‖ ‖
‖zk − Fm ak−m ‖
‖ ‖
‖ m=0 ‖
(10.102)
U | U ( )|2
∑ | (u) ∑ (u,i) (i) |
= |zk − F0 ak + · · · + F𝜈(u,i) a(i)
k−𝜈 ||
u=1 || i=1 |
We note that, as F0 is a lower triangular matrix, the metric has a causal dependence also with regard to
the ordering of the users.
For further study on multiuser detection techniques, we refer the reader to [25–27].

10.7 Multicarrier CDMA systems

CDMA can be combined with orthogonal frequency division multiplexing (OFDM) systems in various
fashions.
For a downlink transmission where a single base station transmits simultaneously to many users, data
symbols are spread by CDMA with different spreading sequence per user, and then chips are allocated
over the subchannels. At each user receiver, one-tap OFDM equalization (see (8.83) in Section 8.7.1)
is performed, and then despreading follows, which completely removes MUI. While in orthogonal fre-
quency division multiple access (OFDMA) schemes (see Section 8.10.1), each user is allocated a subset
of subchannels, in OFDM-CDMA each user transmits over all subchannels, thus increasing the fre-
quency diversity. However, for a channel that is highly selective in frequency and may exhibit a very
low gain for some subchannels, the one-tap per subchannel zero-forcing technique considered above
may substantially increase the noise level at the detection point, thus deteriorating the system perfor-
mance. Alternatives, which are obtained for an increased receiver complexity, have been proposed in
the literature and include (1) a single-tap MSE per subchannel equalizer, (2) an MSE per user, or even
(3) an ML approach in the version MRC [28]. Note that both approaches (1) and (2) introduce MUI.
Using a CDMA-OFDM approach in uplink yields that the spread signal has undergone different
channels for the various users generating MUI at the despreader output. Suitable multiuser detection
techniques are required [29].
510 Chapter 10. Spread-spectrum systems

A recent approach [28, 30] denoted index-modulated OFDM-CDMA allocates to each user multiple
spreading codes. Data bits are still QAM-mapped, spread, and mapped to OFDM subcarriers. However,
the code use for spreading and subcarrier selection is determined again by the data bits. At the receiver, a
detector is first used to identify the selected spreading code and subcarriers, before CDMA despreading
follows. A particular solution provides that some data bits are used only for code or subcarrier selection,
thus their detection at the receiver is only associated with the identification of used codes and subcar-
riers (and not to despreading). Lastly, observe that this approach is similar to spatial modulation (see
Section 9.2.3), where information is carried also by the selected antenna by a multi-antenna transmitter.

Bibliography

[1] Simon, M.K., Omura, J.K., Scholtz, R.A., and Levitt, B.K. (1994). Spread Spectrum Communica-
tions Handbook. New York, NY: McGraw-Hill.
[2] Dixon, R.C. (1994). Spread Spectrum Systems, 3e. New York, NY: Wiley.
[3] Milstein, L.B. and Simon, M.K. (1996). Spread spectrum communications. In: The Mobile Commu-
nications Handbook, Chapter 11 (ed. J.D. Gibson), 152–165. New York, NY: CRC/IEEE Press.
[4] Proakis, J.G. (1995). Digital Communications, 3e. New York, NY: McGraw-Hill.
[5] Price, R. and Green, P.E. (1958). A communication technique for multipath channels. IRE Proceed-
ings 46: 555–570.
[6] Viterbi, A.J. (1995). CDMA: Principles of Spread-Spectrum Communication. Reading, MA:
Addison-Wesley.
[7] Peterson, R.L., Ziemer, R.E., and Borth, D.E. (1995). Introduction to Spread Spectrum Communica-
tions. Englewood Cliffs, NJ: Prentice-Hall.
[8] Holma, H. and Toskala, A. (2010). WCDMA for UMTS: HSPA Evolution and LTE. Chichester, UK:
Wiley.
[9] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[10] Cherubini, G. and Milstein, L.B. (1989). Performance analysis of both hybrid and
frequency–hopped phase–coherent spread–spectrum system. Part I and Part II. IEEE Transactions
on Communications 37: 600–622.
[11] Klein, A. (1997). Data detection algorithms specially designed for the downlink of CDMA mobile
radio systems. Proceedings of 1997 IEEE Vehicular Technology Conference, Phoenix, USA (4–7
May 1997), pp. 203–207.
[12] Li, K. and Liu, H. (1999). A new blind receiver for downlink DS-CDMA communications. IEEE
Communications Letters 3: 193–195.
[13] Werner, S. and Lilleberg, J. (1999). Downlink channel decorrelation in CDMA systems with long
codes. Proceedings of 1999 IEEE Vehicular Technology Conference, Houston, USA (16–20 May
1999), pp. 1614–1617.
[14] Hooli, K., Latva-aho, M., and Juntti, M. (1999). Multiple access interference suppression with
linear chip equalizers in WCDMA downlink receivers. Proceedings of 1999 IEEE Global Telecom-
munications Conference, Rio de Janeiro, Brazil (5–9 December 1999), pp. 467–471.
[15] Madhow, U. and Honig, M.L. (1994). MMSE interference suppression for direct-sequence
spread-spectrum CDMA. IEEE Transactions on Communications 42: 3178–3188.
[16] Miller, S.L. (1995). An adaptive direct-sequence code-division multiple-access receiver for mul-
tiuser interference rejection. IEEE Transactions on Communications 43: 1746–1755.
[17] Rapajic, P.B. and Vucetic, B.S. (1994). Adaptive receiver structures for asynchronous CDMA
systems. IEEE Journal on Selected Areas in Communications 12: 685–697.
10.A. Walsh Codes 511

[18] Klein, A. and Baier, P.W. (1993). Linear unbiased data estimation in mobile radio systems applying
CDMA. IEEE Journal on Selected Areas in Communications 11: 1058–1066.
[19] Blanz, J., Klein, A., Naıhan, M., and Steil, A. (1994). Performance of a cellular hybrid C/TDMA
mobile radio system applying joint detection and coherent receiver antenna diversity. IEEE Journal
on Selected Areas in Communications 12: 568–579.
[20] Kaleh, G.K. (1995). Channel equalization for block transmission systems. IEEE Journal on Selected
Areas in Communications 13: 110–120.
[21] Klein, A., Kaleh, G.K., and Baier, P.W. (1996). Zero forcing and minimum mean-square-error
equalization for multiuser detection in code-division multiple-access channels. IEEE Transactions
on Vehicular Technology 45: 276–287.
[22] Benvenuto, N. and Sostrato, G. (2001). Joint detection with low computational complexity for
hybrid TD-CDMA systems. IEEE Journal on Selected Areas in Communications 19: 245–253.
[23] Bottomley, G.E. and Chennakeshu, S. (1998). Unification of MLSE receivers and extension to
time-varying channels. IEEE Transactions on Communications 46: 464–472.
[24] Duel-Hallen, A. (1995). A family of multiuser decision feedback detectors for asynchronous
code-division multiple access channels. IEEE Transactions on Communications 43: 421–434.
[25] Verdù, S. (1998). Multiuser Detection. Cambridge: Cambridge University Press.
[26] (2001). Multiuser detection techniques with application to wired and wireless communications
systems I. IEEE Journal on Selected Areas in Communications 19.
[27] (2002). Multiuser detection techniques with application to wired and wireless communications
systems II. IEEE Journal on Selected Areas in Communications 20.
[28] Li, Q., Wen, M., Basar, E., and Chen, F. (2018). Index modulated OFDM spread spectrum. IEEE
Transactions on Wireless Communications 17: 2360–2374.
[29] Fazel, K. and Kaiser, S. (2008). Multi-carrier and spread spectrum systems: from OFDM and
MC-CDMA to LTE and WiMAX. Chichester: Wiley.
[30] Basar, E., Aygölü, U., Panayirci, E., and Poor, H.V. (2013). Orthogonal frequency division multi-
plexing with index modulation. IEEE Transactions on Signal Processing 61: 5536–5549.

Appendix 10.A Walsh Codes

We illustrate a procedure to obtain orthogonal binary sequences, with values {−1, 1}, of length 2m .
We consider 2m × 2m Hadamard matrices Am , with binary elements from the set {0, 1}.
For the first orders, we have

A0 = [0] (10.103)

[ ]
0 0
A1 = (10.104)
0 1

⎡0 0 0 0⎤
⎢ ⎥
0 1 0 1⎥
A2 = ⎢ (10.105)
⎢0 0 1 1⎥
⎢0 1 1 0 ⎥⎦
⎣
512 Chapter 10. Spread-spectrum systems

1 1

0 0

–1 –1

0 8Tc 0 8Tc

1 1

0 0

–1 –1

0 8Tc 0 8Tc

1 1

0 0

–1 –1

0 8Tc 0 8Tc

1 1

0 0

–1 –1

0 8Tc 0 8Tc

Figure 10.18 Eight orthogonal signals obtained from the Walsh code of length 8.

⎡0 0 0 0 0 0 0 0⎤
⎢0 1 0 1 0 1 0 1⎥
⎢ ⎥
⎢0 0 1 1 0 0 1 1⎥
⎢0 1 1 0 0 1 1 0⎥
A3 = ⎢
1 ⎥⎥
(10.106)
⎢0 0 0 0 1 1 1
⎢0 1 0 1 1 0 1 0⎥
⎢0 0 1 1 1 1 0 0⎥
⎢ ⎥
⎣0 1 1 0 1 0 0 1⎦
In general, the construction is recursive
[ ]
Am Am
Am+1 = (10.107)
Am Am

where Am denotes the matrix that is obtained by taking the 1’s complement of the elements of Am .
10.A. Walsh Codes 513

A Walsh code of length 2m is obtained by taking the rows (or columns) of the Hadamard matrix Am
and by mapping 0 into −1. From the construction of Hadamard matrices, it is easily seen that two words
of a Walsh code are orthogonal.
Figure 10.18 shows the eight signals obtained with the Walsh code of length 8: the signals are obtained
by interpolating the Walsh code sequences by a filter having impulse response
t − Tc ∕2
𝚠Tc (t) = rect (10.108)
Tc
Chapter 11

Channel codes

Forward error correction (FEC) is a widely used technique to achieve reliable data transmission. The
redundancy introduced by an encoder for the transmission of data in coded form allows the decoder at the
receiver to detect and partially correct errors. An alternative transmission technique, known as automatic
repeat query or request (ARQ), consists in detecting the errors (usually by a check-sum transmitted with
the data, see page 556) and requesting the retransmission of a data packet whenever it is received with
errors.
The FEC technique presents two advantages with respect to the ARQ technique.

1. In systems that make use of the ARQ technique, the data packets do not necessarily have to be
retransmitted until they are received without errors; however, for large values of the error proba-
bility, the aggregate traffic of the link is higher.
2. In systems that make use of the FEC technique, the receiver does not have to request the retrans-
mission of data packets, thus making possible the use of a simplex link; this feature represents a
strong point in many applications like time division multiple access (TDMA) and video satellite
links, where a central transmitter broadcasts to receive-only terminals, which are unable to make a
possible retransmission request. The FEC technique is also particularly useful in various satellite
communication applications, in which the long round-trip delay of the link would cause serious
traffic problems whenever the ARQ technique would be used.

We distinguish two broad classes of FEC techniques, each with numerous subclasses, employing block
codes or convolutional codes.
All error correction techniques add redundancy, in the form of additional bits, to the information
bits that must be transmitted. Redundancy makes the correction of errors possible and for the classes of
codes considered in this chapter represents the coding overhead. The effectiveness of a coding technique
is expressed in terms of the coding gain, Gcode , given by the difference between the signal-to-noise ratios
(SNRs), in dB, that are required to achieve a certain bit error probability for transmission without and
with coding (see Definition 6.2 on page 274). The overhead is expressed in terms of the code rate,
Rc , given by the ratio between the number of information bits and the number of code bits that are
transmitted. The transmission bit rate is inversely proportional to Rc , and is larger than that necessary
for uncoded data. If one of the modulation techniques of Chapter 6 is employed, the modulation rate is
also larger. In Chapter 12, methods to transmit coded sequences of symbols without an increase in the
modulation rate will be discussed.
For further study on the topic of error correcting codes we refer to [1–3].
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
516 Chapter 11. Channel codes

11.1 System model

With reference to the model of a transmission system with coding, illustrated in Figure 6.1, we introduce
some fundamental parameters.
A block code is composed of a set of vectors of given length called code words; the length of a code
word is defined as the number of vector elements, indicated by n0 . The elements of a code word are
chosen from an alphabet of q elements: if the alphabet consists of two elements, for example 0 and 1, the
code is a binary code, and we refer to the elements of each code word as bits; if, on the other hand, the
elements of a code word are chosen from an alphabet having q elements (q > 2), the code is non-binary.
It is interesting to note that if q is a power of two, that is q = 2b , where b is a positive integer, each q-ary
element has an equivalent binary representation of b bits and therefore a non-binary code word of length
N can be mapped to a binary code word of length n0 = bN.
There are 2n0 possible code words in a binary code of length n0 . From these 2n0 possible code words,
we choose 2k0 words (k0 < n0 ) to form a code. Thus, a block of k0 information bits is mapped to a code
word of length n0 chosen from a set of 2k0 code words; the resulting block code is indicated as (n0 , k0 )
code, and the ratio Rc = k0 ∕n0 is the code rate. 1

Observation 11.1
The code rate Rc is related to the encoder-modulator rate RI (6.57) by the following relation:
k0 log2 M log M
RI = = Rc 2 (11.1)
n0 I I
where M is the number of symbols of the I-dimensional constellation adopted by the bit-mapper.

Because the number of bits per unit of time produced by the encoder is larger than that produced by
the source, two transmission strategies are possible.

Transmission for a given bit rate of the information message. With reference to Figure 6.1, from the
relation
k0 Tb = n0 Tcod (11.2)
we get
1 1 1
= (11.3)
Tcod R c Tb
note that the bit rate at the modulator input is increased in the presence of the encoder. For a given
modulator with M symbols, that is using the same bit mapper, this implies an increase of the modulation
rate given by
1 1 1 1
= = (11.4)
T′ Tcod log2 M T Rc
and therefore an increase of the bandwidth of the transmission channel by a factor 1∕Rc . Moreover, for
the same transmitted power, from (6.97) in the presence of the encoder the SNR becomes
Γ′ = Γ Rc (11.5)
i.e. it decreases by a factor Rc with respect to the case of transmission of an uncoded message. Therefore,
for a given information message bit rate 1∕Tb , the system operates with a lower Γ′ : consequently, the
receiver is prone to introduce more bit errors at the decoder input. In spite of this, for a suitable choice

1
In this chapter, a block code will be sometimes indicated also with the notation (n, k).
11.2. Block codes 517

of the code, in many cases, the decoder produces a detected message {b̂ 𝓁 } affected by fewer errors with
respect to the case of transmission of an uncoded message. We note that the energy per information bit
of the encoded message {cm } is equal to that of {b𝓁 }. In fact, for a given bit mapper,
k0
Lb′ = log M = Rc Lb (11.6)
n0 2
Assuming the same transmitted power, from (6.61) and (11.4), we get
Es′ = Rc EsCh (11.7)
Ch

Therefore, (6.63) yields for the encoded message {cm } an energy per bit of information equal to
Es′ EsCh
Ch
= = Eb (11.8)
Lb′ Lb
Since Γ′ ≠ Γ, a comparison between the performance of the two systems, with and without coding, is
made for the same Eb ∕N0 . In this case the coding gain, in dB, is given by
10 (log10 Γ − log10 Γ′ + log10 Rc ) (11.9)

Transmission for a given modulation rate. For given transmitted power and given transmission chan-
nel bandwidth, Γ remains unchanged in the presence of the encoder. Therefore, there are three possibil-
ities.
1. The bit rate of the information message decreases by a factor Rc and becomes
1 1
= R (11.10)
Tb′ Tb c
2. The source emits information bits in packets, and each packet is followed by additional bits gener-
ated by the encoder, forming a code word; the resulting bits are transmitted at the rate
1 1
= (11.11)
Tcod Tb
3. A block of m information bits is mapped to a transmitted symbol using a constellation with cardi-
nality M > 2m . In this case, transmission occurs without decreasing the bit rate of the information
message.
In the first two cases, for the same number of bits of the information message, we have an increase in
the duration of the transmission by a factor 1∕Rc .
For a given bit error probability in the sequence {b̂ 𝓁 }, we expect that in the presence of coding, a
smaller Γ is required to achieve a certain error probability as compared to the case of transmission of an
uncoded message; this reduction corresponds to the coding gain.

11.2 Block codes

We give the following general definition.2

Definition 11.1
The Hamming distance between two vectors 𝒗1 and 𝒗2 , dH (𝒗1 , 𝒗2 ), is given by the number of elements
in which the two vectors differ. ◽
2
The material presented in Sections 11.2 and 11.3 is largely based on lectures given at the University of California, San
Diego, by Prof. Jack K. Wolf [4], whom the authors gratefully acknowledge.
518 Chapter 11. Channel codes

11.2.1 Theory of binary codes with group structure

Properties
A binary block code of length n is a subset containing Mc of the 2n possible binary sequences of length
n, also called code words. The only requirement on the code words is that they are all of the same length.

Definition 11.2
The minimum Hamming distance of a block code, to which we will refer in this chapter simply as the
H
minimum distance, is denoted by dmin and coincides with the smallest number of positions in which any
two code words differ. ◽

H
An example of a block code with n = 4, Mc = 4, and dmin = 2 is given by (11.22).
For the binary symmetric channel model (6.72), assuming that the binary code word c of length n is
transmitted, we observe at the receiver3
z=c⊕e (11.12)
where ⊕ denotes the modulo-2 sum of respective vector components; for example (0111) ⊕ (0010) =
(0101). In (11.12), e is the binary error vector whose generic component is equal to 1 if the channel has
introduced an error in the corresponding bit of c, and 0 otherwise. We note that z can assume all the 2n
possible combinations of n bits.
With reference to Figure 6.1, the function of the decoder consists in associating with each possible
value z of a code word. A commonly adopted criterion is to associate z with the code word ĉ that is
closest according to the Hamming distance. From this code word, the k0 information bits, which form
the sequence {b̂ l }, are recovered by inverse mapping.
Interpreting the code words as points in an n-dimensional space, where the distance between points
is given by the Hamming distance, we obtain the following properties:
H
1. A binary block code with minimum distance dmin can correct all patterns of
⌊ H ⌋
dmin − 1
t= (11.13)
2
or fewer errors, where ⌊x⌋ denotes the integer value of x.
H H
2. A binary block code with minimum distance dmin can detect all patterns of (dmin − 1) or fewer
errors.
3. In a binary erasure channel, the transmitted binary symbols are detected using a ternary alphabet
{0, 1, erasure}; a symbol is detected as erasure if the reliability of a binary decision is low. In the
H H
absence of errors, a binary block code with minimum distance dmin can fill in (dmin − 1) erasures.
H H
4. Seeking a relation among n, Mc , and dmin , we find that, for fixed n and odd dmin , Mc is upper bounded
by4
⎢ /⎧ ⎛ ⎞⎫ ⎥⎥
⎢ n ⎪ ( ) ( )
⎟⎪
n n n
Mc ≤ MUB = ⎢2 ⎨1 + 1 + 2 + · · · + ⎜ dmin ⎜ ⌊ ⌋ ⎥
⎟⎬
H
−1 (11.15)
⎢ ⎪ ⎝ ⎠⎪⎥
⎣ ⎩ 2
⎭⎦

3
In Figure 6.1, z is indicated as c̃ .
4
We recall that the number of binary sequences of length n with m ‘ones’ is equal to
( )
n n!
= (11.14)
m m!(n − m)!
where n! = n(n − 1) · · · 1.
11.2. Block codes 519

H
5. For fixed n and dmin , it is always possible to find a code with Mc∗ words, where
⌈ /{ ( ) ( ) ( )} ⌉
n n n
Mc∗ = 2n 1+ + +···+ H
(11.16)
1 2 dmin −1
where ⌈x⌉ denotes the smallest integer greater than or equal to x. We will now consider a procedure
for finding such a code.
Step 1: choose any code word of length n and exclude from future choices that word and all words
H
that differ from it in (dmin − 1) or fewer positions. The total number of words excluded from future
choices is ( ) ( ) ( )
H n n n
Nc (n, dmin − 1) = 1 + + +···+ H
(11.17)
1 2 dmin −1
Step i: choose a word not previously excluded and exclude from future choices all words previ-
H
ously excluded plus the chosen word and those that differ from it in (dmin − 1) or fewer positions.
Continue this procedure until there are no more words available to choose from. At each step, if
H
still not excluded, at most Nc (n, dmin − 1) additional words are excluded; therefore, after step i,
H
when i code words have been chosen, at most i Nc (n, dmin − 1) words have been excluded. Then, if
n H
2 ∕Nc (n, dmin − 1) is an integer, we can choose at least that number of code words; if it is not an
integer, we can choose at least a number of code words equal to the next largest integer.

Definition 11.3
A binary code with group structure is a binary block code for which the following conditions are verified:
1. the all zero word is a code word (zero code word);
2. the modulo-2 sum of any two code words is also a code word.
◽

Definition 11.4
The weight of any binary vector x, denoted as 𝑤(x), is the number of ones in the vector. ◽

H
Property 1 of a group code. The minimum distance of the code dmin is given by
H
dmin = min 𝑤(c) (11.18)
where c can be any non-zero code word.

Proof. The sum of any two distinct code words is a non-zero word. The weight of the resulting word
is equal to the number of positions in which the two original words differ. Because two words at the
H H
minimum distance differ in dmin positions, there is a word of weight dmin . If there were a non-zero word
H
of weight less than dmin , it would be different from the zero word in less than dmin H
positions. ◽

Property 2 of a group code. If all code words in a group code are written as rows of an Mc × n matrix,
then every column is either zero or consists of half zeros and half ones.

Proof. An all zero column is possible if all code words have a zero in that column. Suppose in column
i, there are m 1s and (Mc − m) 0s. Choose one of the words with a 1 in that column and add it to all
words that have a 1 in that column, including the word itself: this operation produces m words with a 0
in that column, hence, (Mc − m) ≥ m. Now, we add that word to each word that has a 0 in that column:
this produces (Mc − m) words with a 1 in that column, hence, (Mc − m) ≤ m. Therefore, Mc − m = m or
m = Mc ∕2. ◽
520 Chapter 11. Channel codes

Corollary 11.1
From Property 2, it turns out that the number of code words Mc must be even for a binary
group code. ◽

Corollary 11.2
Excluding codes of no interest from the transmission point of view, for which all code words have a 0
in a given position, from Property 2 the average weight of a code word is equal to n∕2. ◽

Parity check matrix

Let H be a binary r × n matrix, which is called parity check matrix, of the form
H = [A B] (11.19)
where B is an r × r matrix with det[B] ≠ 0, i.e. the columns of B are linearly independent.
A binary parity check code is a code consisting of all binary vectors c that are solutions of the equation
Hc = 0 (11.20)
The matrix product in (11.20) is computed using the modulo-2 arithmetic.

Example 11.2.1
Let the matrix H be given by [ ]
1 0 1 1
H= (11.21)
0 1 0 1
There are four code words in the binary parity check code corresponding to the matrix H; they are
⎡0⎤ ⎡1⎤ ⎡0⎤ ⎡1⎤
⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥
0 0 1 1
c0 = ⎢ ⎥ c1 = ⎢ ⎥ c2 = ⎢ ⎥ c3 = ⎢ ⎥ (11.22)
⎢0⎥ ⎢1⎥ ⎢1⎥ ⎢0⎥
⎢0⎥ ⎢0⎥ ⎢1⎥ ⎢1⎥
⎣ ⎦ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦

Property 1 of a parity check code. A parity check code is a group code.

Proof. The all zero word is always a code word, as

H0 = 0 (11.23)
Suppose that c1 and c2 are code words; then Hc1 = 0 and Hc2 = 0. It follows that
H(c1 ⊕ c2 ) = Hc1 ⊕ Hc2 = 0 ⊕ 0 = 0 (11.24)
Therefore, c1 ⊕ c2 is also a code word. ◽

Property 2 of a parity check code. The code words corresponding to the parity check matrix H =
̃ = [B−1 A, I] = [Ã I],
[A B] are identical to the code words corresponding to the parity check matrix H
where I is the r × r identity matrix.
[ ]
cn−r
Proof. Let c = 1 be a code word corresponding to the matrix H = [A B], where cn−r are the first
cnn−r+1 1

(n − r) components of the vector and cnn−r+1 are the last r components of the vector. Then

1 ⊕ Bcn−r+1 = 0
Hc = Acn−r n
(11.25)
11.2. Block codes 521

Multiplying by B−1 , we get

1 ⊕ Icn−r+1 = 0
B−1 Acn−r n
(11.26)
̃ = 0.
or Hc ◽

From Property 2, we see that parity check matrices of the form H ̃ = [Ã I] are not less general than
parity check matrices of the form H = [A B], where det[B] ≠ 0. In general, we can consider any r × n
matrix as a parity check matrix, provided that some set of r columns has a non-zero determinant. If we
are not concerned with the order by which the elements of a code word are transmitted, then such a code
would be equivalent to a code formed by a parity check matrix of the form

H = [A I] (11.27)

The form of the matrix (11.27) is called canonical or systematic form. We assume that the last r columns
of H have a non-zero determinant and Therefore, that the parity check matrix can be expressed in canon-
ical form.

Property 3 of a parity check code. There are exactly 2n−r = 2k code words in a parity check code.

Proof. Referring to the proof of Property 2, we find that

cnn−r+1 = Acn−r
1 (11.28)

For each of the 2n−r = 2k possible binary vectors cn−r

1
, it is possible to compute the corresponding vector
cnn−r+1 . Each of these code words is unique as all of them differ in the first (n − r) = k positions. Assume
that there are more than 2k code words; then at least two will agree in the first (n − r) = k positions. But
from (11.28), we find that these two code words also agree in the last r positions and therefore, they are
identical. ◽

The code words have the following structure:

c = [m0 … mk−1 , p0 … pr−1 ]T (11.29)

where the first k = (n − r) bits are called information bits and the last r bits are called parity check bits.
As mentioned in Section 11.1, a parity check code that has code words of length n that are obtained by
encoding k information bits is an (n, k) code.

Property 4 of a parity check code. A code word of weight 𝑤 exists if and only if the modulo-2 sum
of 𝑤 columns of H equals 0.

Proof. c is a code word if and only if Hc = 0. Let hi be the ith column of H and let cj be the jth
component of c. Therefore, if c is a code word, then
∑
n
h j cj = 0 (11.30)
j=1

If c is a code word of weight 𝑤, then there are exactly 𝑤 non-zero components of c, for example
cj1 , cj2 , … , cj𝑤 . Consequently, hj1 ⊕ hj2 ⊕ … ⊕ hj𝑤 = 0, thus a code word of weight 𝑤 implies that the
sum of 𝑤 columns of H equals 0. Conversely, if hj1 ⊕ hj2 ⊕ … ⊕ hj𝑤 = 0, then Hc = 0, where c is a
binary vector with elements equal to 1 in positions j1 , j2 , … , j𝑤 . ◽
522 Chapter 11. Channel codes

H
Property 5 of a parity check code. A parity check code has minimum distance dmin if some modulo-2
H H
sum of dmin columns of H is equal to 0, but no modulo-2 sum of fewer than dmin columns of H is equal
to 0.

Proof. Property 5 follows from Property 1 of a group code and also from Properties 1 and 4 of a parity
check code. ◽

Property 5 may be considered as the fundamental property of parity check codes, as it forms the basis
for the design of almost all such codes. An important exception is constituted by low-density parity check
codes, which will be discussed in Section 11.8. A bound on the number of parity check bits required for
H
a given block length n and given dmin derives directly from this property.

Property 6 of a parity check code. A binary parity check code exists of block length n and minimum
H
distance dmin , having no more than r∗ parity check bits, where
⎢ ⎛d∑
H
min
−2 ( )⎞⎥
∗ ⎢ ⎜ n − 1 ⎟⎥
r = log2 +1 (11.31)
⎢ ⎜ i=0 i ⎟⎥
⎣ ⎝ ⎠⎦

Proof. The proof derives from the following exhaustive construction procedure of the parity check
matrix of the code.
Step 1: choose as the first column of H any non-zero vector with r∗ components.
Step 2: choose as the second column of H any non-zero vector different from the first.
Step 3: choose as the ith column of H any vector distinct from all vectors obtained by modulo-2 sum
H
of (dmin − 2) or fewer previously chosen columns.
H
Clearly, such a procedure will result in a matrix H, where no set of (dmin − 1) or fewer columns of
H sum to 0. However, we must show that we can indeed continue this process for n columns. After
applying this procedure for (n − 1) columns, there will be at most
( ) ( ) ( )
H n−1 n−1 n−1
Nc (n − 1, dmin − 2) = 1 + + +···+ H
(11.32)
1 2 dmin −2
∗
distinct vectors that are forbidden for the choice of the last column, but there are 2r vectors to choose
∗
from; observing (11.31) and (11.32) we get 2r > Nc (n − 1, dmin H
− 2). Thus, n columns can always be
H
chosen where no set of (dmin − 1) or fewer columns sums to zero. From Property 5, the code therefore
has minimum distance at least dminH
. ◽

Code generator matrix

Using (11.28), we can write
[ ] [ ]
cn−r I n−r
c= 1 = c = GT cn−r (11.33)
cnn−r+1 A 1 1

where G = [I AT ] is a k × n binary matrix, and I is the k × k identity matrix. Taking the transpose of
(11.33), we obtain
cT = (cn−r T
1 ) G (11.34)
thus the code words, considered now as row vectors, are given as all linear combinations of the rows of
G, which is called the generator matrix of the code. A parity check code can be specified by giving its
parity check matrix H or its generator matrix G.
11.2. Block codes 523

Example 11.2.2
Consider the parity check code (7,4) with the parity check matrix
⎡1 1 0 1 1 0 0⎤
H = ⎢ 1 1 1 0 0 1 0 ⎥ = [A I] (11.35)
⎢ ⎥
⎣1 0 1 1 0 0 1⎦
Expressing a general code word according to (11.29), to every 4 information bits 3 parity check bits are
added, related to the information bits by the equations (see (11.28))
p0 = m0 ⊕ m1 ⊕ m3
p1 = m0 ⊕ m1 ⊕ m2 (11.36)
p2 = m0 ⊕ m2 ⊕ m3
The generator matrix of this code is given by
⎡1 0 0 0 1 1 1⎤
⎢ ⎥
0 1 0 0 1 1 0⎥
G=⎢ = [I AT ] (11.37)
⎢0 0 1 0 0 1 1⎥
⎢0 0 0 1 1 0 1 ⎥⎦
⎣
There are 16 code words consisting of all linear combinations of the rows of G. By inspection, we find
H
that the minimum weight of a non-zero code word is 3; hence, from (11.18) the code has dmin = 3 and
therefore is a single error correcting code.

Decoding of binary parity check codes

Conceptually, the simplest method for decoding a block code is to compare the received block of n bits
with each code word and choose that code word that differs from the received word in the minimum
number of positions; in case several code words satisfy this condition, choose amongst them at random.
Although the simplest conceptually, the described method is out of the question practically because we
usually employ codes with very many code words. It is however instructive to consider the application
of this method, suitably modified, to decode group codes.

Cosets
The 2n possible binary sequences of length n are partitioned into 2r sets, called cosets, by a group code
with 2k = 2n−r code words; this partitioning is done as follows:
Step 1: choose the first set as the set of code words c1 , c2 , … , c2k .
Step 2: choose any vector, say 𝜼2 , that is not a code word; then choose the second set as c1 ⊕ 𝜼2 , c2 ⊕
𝜼2 , … , c2k ⊕ 𝜼2 .
Step i: choose any vector, say 𝜼i , not included in any previous set; choose the ith set, i.e. coset, as
c1 ⊕ 𝜼i , c2 ⊕ 𝜼i , … , c2k ⊕ 𝜼i . The partitioning continues until all 2n vectors are used.
Note that each coset contains 2k vectors; if we show that no vector can appear in more than one coset,
we will have demonstrated that there are 2r = 2n−k cosets.

Property 1 of cosets. Every binary vector of length n appears in one and only one coset.

Proof. Every vector appears in at least one coset as the partitioning stops only when all vectors are used.
Suppose that a vector appeared twice in one coset; then for some value of the index i, we have cj1 ⊕ 𝜼i =
524 Chapter 11. Channel codes

cj2 ⊕ 𝜼i , or cj1 = cj2 , that is a contradiction as all code words are unique. Suppose that a vector appears in
two cosets; then cj1 ⊕ 𝜼i1 = cj2 ⊕ 𝜼i2 , where we assume i2 > i1 . Then 𝜼i2 = cj1 ⊕ cj2 ⊕ 𝜼i1 = cj3 ⊕ 𝜼i1 ,
that is a contradiction as 𝜼i2 would have appeared in a previous coset, against the hypothesis. ◽

Example 11.2.3
Consider partitioning the 24 binary vectors of length 4 into cosets using the group code with code words
0000, 0011, 1100, 1111, as follows:
0000 0011 1100 1111
𝜼2 = 0 0 0 1 0010 1101 1110
(11.38)
𝜼3 = 0 1 1 1 0100 1011 1000
𝜼4 = 1 0 1 0 1001 0110 0101
The vectors 𝜼1 = 0, 𝜼2 , 𝜼3 , … , 𝜼2r , are called coset leaders; the partitioning (11.38) is called coset table
or decoding table.

Property 2 of cosets. Suppose that instead of choosing 𝜼i as the coset leader of the ith coset, we choose
another element of that coset; the new coset formed by using this new coset leader contains exactly the
same vectors as the old coset.

Proof. Assume that the new coset leader is 𝜼i ⊕ cj1 , and that z is an element of the new coset; then
z = 𝜼i ⊕ cj1 ⊕ cj2 = 𝜼i ⊕ cj3 , so z is an element of the old coset. As the new and the old cosets both
contain 2k vectors, and all vectors in a coset are unique, every element of the new coset belongs to the
old coset, and vice versa. ◽

Example 11.2.4
Suppose that in the previous example, we had chosen the third coset leader as 0100; then the table (11.38)
would be
0000 0011 1100 1111
𝜼2 = 0 0 0 1 0 0 1 0 1 1 0 1 1 1 1 0
(11.39)
𝜼3 = 0 1 0 0 0 1 1 1 1 0 0 0 1 0 1 1
𝜼4 = 1 0 1 0 1 0 0 1 0 1 1 0 0 1 0 1

Two conceptually simple decoding methods

Assume that each coset leader is chosen as the minimum weight vector in its coset; in case several vectors
in a coset have the same minimum weight, choose any one of them as the coset leader.
Then a second method of decoding, using the decoding table, is as follows:

Step 1: locate the received vector in the coset table.

Step 2: choose the code word that appears as the first vector in the column containing the received
vector.

Proposition 11.1
Decoding using the decoding table decodes to the closest code word to the received word; in case several
code words are at the same smallest distance from the received word, it decodes to one of these closest
words. ◽
11.2. Block codes 525

Proof. Assume that the received word is the jth vector in the ith coset. The received word, given by
z = cj ⊕ 𝜼i , is corrected to the code word cj and the distance between the received word and the jth code
word is 𝑤(𝜼i ). Suppose that another code word, say ck , is closer to the received vector: then

𝑤(ck ⊕ cj ⊕ 𝜼i ) < 𝑤(𝜼i ) (11.40)

or
𝑤(c𝓁 ⊕ 𝜼i ) < 𝑤(𝜼i ) (11.41)

but this cannot be as 𝑤(𝜼i ) is assumed to be the minimum weight vector in its coset and c𝓁 ⊕ 𝜼i is in
that coset. ◽

We note that the coset leaders determine the only error patterns that can be corrected by the code.
Coset leaders, moreover, have many other interesting properties: ⌊ dH for ⌋example if a code has minimum
H −1
distance dmin , all binary n-tuple of weight less than or equal to min2 are coset leaders.

Definition 11.5
A code for which coset leaders are all vectors of weight t or less, and no others, is called a perfect t-error
correcting code. A code for which coset leaders are all vectors of weight t or less, and some vectors of
weight t + 1 but not all, and no others, is called quasi-perfect t-error correcting code. ◽

The perfect binary codes are:

1. codes given by the repetition of n bits, with n odd: these codes contain only two code words, 000…0
H
(all zeros) and 111…1 (all ones), and correct t = (n − 1)∕2 errors (dmin = n);
2. Hamming codes: these codes correct t = 1 errors (dmin = 3) and have n = 2r − 1, k = n − r, r > 1;
H

the columns of the matrix H are given by all non-zero vectors of length r;
H
3. Golay code: t = 3 (dmin = 7), n = 23, k = 12, r = 11.

The following modification of the decoding method dealt with in this section will be useful later on:

Step 1′ : locate the received vector in the coset table and identify the coset leader of the coset contain-
ing that vector.
Step 2′ : add the coset leader to the received vector to find the decoded code word.

Syndrome decoding
A third method of decoding is based on the concept of syndrome. Among the methods described in this
section, syndrome decoding is the only method of practical value for a code with a large number of code
words.

Definition 11.6
For any parity check matrix H, we define the syndrome s(z) of a binary vector z of length n as

s(z) = Hz (11.42)
◽

We note that the syndrome is a vector of length r, whereas z is a vector of length n. Therefore, many
vectors will have the same syndrome. All code words have an all zero Syndrome, and these are the only
vectors with this property. This property of the code words is a special case of the following:
526 Chapter 11. Channel codes

Property 3 of cosets. All vectors in the same coset have the same syndrome; vectors in different cosets
have distinct syndromes.

Proof. Assume that z1 and z2 are in the same coset, say the ith: then z1 = 𝜼i ⊕ cj1 and z2 = 𝜼i ⊕ cj2 .
Moreover, s(z1 ) = Hz1 = H(𝜼i ⊕ cj1 ) = H𝜼i ⊕ Hcj1 = H𝜼i ⊕ 0 = s(𝜼i ). Similarly s(z2 ) = s(𝜼i ), so
s(z1 ) = s(z2 ) = s(𝜼i ): this proves the first part of the property.
Now, assume that z1 and z2 are in different cosets, say the i1 th and i2 th: then z1 = 𝜼i1 ⊕ cj1 and z2 =
𝜼i2 ⊕ cj2 , so s(z1 ) = s(𝜼i1 ) and s(z2 ) = s(𝜼i2 ). If s(z1 ) = s(z2 ) then s(𝜼i1 ) = s(𝜼i2 ), which implies H𝜼i1 =
H𝜼i2 . Consequently, H(𝜼i1 ⊕ 𝜼i2 ) = 0, or 𝜼i1 ⊕ 𝜼i2 is a code word, say cj3 . Then 𝜼i2 = 𝜼i1 ⊕ cj3 , which
implies that 𝜼i1 and 𝜼i2 are in the same coset, that is a contradiction. Thus, the assumption that s(z1 ) =
s(z2 ) is incorrect. ◽

From Property 3, we see that there is a one-to-one relation between cosets and syndromes; this leads
to the third method of decoding, which proceeds as follows:

Step 1′′ : compute the syndrome of the received vector; this syndrome identifies the coset in which
the received vector is. Identify then the leader of that coset.
Step 2′′ : add the coset leader to the received vector to find the decoded code word.

Example 11.2.5
Consider the parity check matrix
⎡1 1 0 1 0 0⎤
H =⎢1 0 1 0 1 0⎥ (11.43)
⎢ ⎥
⎣1 1 1 0 0 1⎦
The coset leaders and their respective syndromes obtained using (11.42) are reported in Table 11.1.
⎡1⎤
Suppose that the vector z = 000111 is received. To decode, we first compute the syndrome Hz = ⎢1⎥,
⎢ ⎥
⎣1⎦
⎡1⎤
⎢ ⎥
⎢0⎥
⎢0⎥
then by Table 11.1 we identify the coset leader as ⎢ ⎥, and obtain the decoded code word ( 1 0 0 1 1 1 ).
⎢0⎥
⎢0⎥
⎢0⎥
⎣ ⎦

The advantage of syndrome decoding over the other decoding methods previously described is that
there is no need to memorize the entire decoding table at the receiver. The first part of Step 1′′ , namely
computing the syndrome, is trivial. The second part of Step 1′′ , namely identifying the coset leader
corresponding to that syndrome, is the difficult part of the procedure; in general, it requires a RAM with
2r memory locations, addressed by the syndrome of r bits and containing the coset leaders of n bits.
Overall, the memory bits are n2r .
There is also an algebraic method to identify the coset leader. In fact, this problem is equivalent to
finding the minimum set of columns of the parity check matrix which sum to the syndrome. In other
words, we must find the vector z of minimum weight such that Hz = s.
For a single error correcting Hamming code, all coset leaders are of weight 1 or 0, so a non-zero
syndrome corresponds to a single column of H and the correspondence between syndrome and coset
11.2. Block codes 527

Table 11.1: Coset leaders and

respective syndromes for
Example 11.2.5.

Coset leader Syndrome

000000 000
000001 001
000010 010
000100 100
001000 011
010000 101
100000 111
100001 110

leader is simple. For a code with coset leaders of weight 0, 1, or 2, the syndrome is either 0, a single
column of H, or the sum of two columns, etc.
For a particular class of codes that will be considered later, the structure of the construction of H
identifies the coset leader starting from the syndrome by using algebraic procedures. In general, each
class of codes leads to a different technique to perform this task.

Property 7 of parity check codes. There are exactly 2r correctable error vectors for a parity check
code with r parity check bits.

Proof. Correctable error vectors are given by the coset leaders and there are 2n−k = 2r of them, all
of which are distinct. On the other hand, there are 2r distinct syndromes and each corresponds to a
correctable error vector. ◽

For a binary symmetric channel (see Definition 6.1) we should correct all error vectors of weight i,
i = 0, 1, 2, …, until we exhaust the capability of the code. Specifically, we should try to use a perfect
code or a quasi-perfect code. For a quasi-perfect t-error correcting code, the coset leaders consist of all
error vectors of weight i = 0, 1, 2, … , t, and some vectors of weight t + 1.
Non-binary parity check codes are discussed in Appendix 11.A.

11.2.2 Fundamentals of algebra

The calculation of parity check bits from information bits involves solving linear equations. This pro-
cedure is particularly easy for binary codes since we use modulo-2 arithmetic. An obvious question is
whether or not the concepts of the previous section generalize to codes with symbols taken from alpha-
bets with a larger cardinality, say alphabets with q symbols. We will see that the answer can be yes or no
according to the value of q; furthermore, even if the answer is yes, we might not be able to use modulo-q
arithmetic.
Consider the equation for the unknown x
ax = b (11.44)
528 Chapter 11. Channel codes

Table 11.2:
Multiplication table for 3
elements (modulo-3
arithmetic).

⋅ 0 1 2
0 0 0 0
1 0 1 2
2 0 2 1

Table 11.3: Multiplication

table for an alphabet with four
elements (modulo-4
arithmetic).

⋅ 0 1 2 3
0 0 0 0 0
1 0 1 2 3
2 0 2 0 2
3 0 3 2 1

Table 11.4: Multiplication

table for an alphabet with four
elements.

⋅ 0 1 2 3
0 0 0 0 0
1 0 1 2 3
2 0 2 3 1
3 0 3 1 2

where a and b are known coefficients, and all values are from the finite alphabet {0, 1, 2, … , q − 1}.
First, we need to introduce the concept of multiplication, which is normally given in the form of a
multiplication table, as the one given in Table 11.2 for the three elements {0, 1, 2}.
Table 11.2 allows us to solve (11.44) for any values of a and b, except a = 0. For example, the solution
to equation 2x = 1 is x = 2, as from the multiplication table, we find 2 ⋅ 2 = 1.
Let us now consider the case of an alphabet with four elements. A multiplication table for the four
elements {0, 1, 2, 3}, resulting from the modulo-4 arithmetic, is given in Table 11.3.
Note that the equation 2x = 2 has two solutions, x = 1 and x = 3, and equation 2x = 1 has no solution.
It is possible to construct a multiplication table that allows the equation (11.44) to be solved uniquely
for x, provided that a ≠ 0, as shown in Table 11.4.
Note that Table 11.4 is not obtained using modulo 4 arithmetic. For example 2x = 3 has the solution
x = 2, and 2x = 1 has the solution x = 3.

modulo-q arithmetic
Consider the elements {0, 1, 2, … , q − 1}, where q is a positive integer larger than or equal to 2. We
define two operations for combining pairs of elements from this set. The first, denoted by ⊕, is called
modulo-q addition and is defined as
{ }
a+b if 0 ≤ a + b < q
c=a⊕b= . (11.45)
a+b−q if a + b ≥ q
11.2. Block codes 529

Here a + b is the ordinary addition operation for integers that may produce an integer not in the set. In
this case, q is subtracted from a + b, and a + b − q is always an element in the set {0, 1, 2, … , q − 1}.
The second operation, denoted by ⊗, is called modulo-q multiplication and is defined as

⎧ ab if 0 ≤ ab < q
⎪ ⌊ ⌋
d =a⊗b=⎨ ab (11.46)
⎪ ab − q if ab ≥ q
⎩ q
⌊ ⌋
Note that ab − ab q
q, is the remainder or residue of the division of ab by q, and is always an integer in
the set {0, 1, 2, … , q − 1}. Often, we will omit the notation ⊗ and write a ⊗ b simply as ab.
We recall that special names are given to sets which possess certain properties with respect to oper-
ations. Consider the general set G that contains the elements {𝛼, 𝛽, 𝛾, 𝛿, …}, and two operations for
combining elements from the set. We denote the first operation △ (addition), and the second operation
◊ (multiplication). Often, we will omit the notation ◊ and write a◊b simply as ab. The properties we
are interested in are the followig:

1. Existence of additive identity: For every 𝛼 ∈ G, there exists an element ∅ ∈ G, called additive
identity, such that 𝛼 △ ∅ = ∅ △ 𝛼 = 𝛼.
2. Existence of additive inverse: For every 𝛼 ∈ G, there exists an element 𝛽 ∈ G, called additive
inverse of 𝛼, and indicated with −𝛼, such that 𝛼 △ 𝛽 = 𝛽 △ 𝛼 = ∅.
3. Additive closure: For every 𝛼, 𝛽 ∈ G, not necessarily distinct, 𝛼 △ 𝛽 ∈ G.
4. Additive associative law: For every 𝛼, 𝛽, 𝛾 ∈ G, 𝛼 △ (𝛽 △ 𝛾) = (𝛼 △ 𝛽) △ 𝛾.
5. Additive commutative law: For every 𝛼, 𝛽 ∈ G, 𝛼 △ 𝛽 = 𝛽 △ 𝛼.
6. Multiplicative closure: For every 𝛼, 𝛽 ∈ G, not necessarily distinct, 𝛼◊𝛽 ∈ G.
7. Multiplicative associative law: For every 𝛼, 𝛽, 𝛾 ∈ G, 𝛼◊(𝛽◊𝛾) = (𝛼◊𝛽)◊𝛾.
8. Distributive law: For every 𝛼, 𝛽, 𝛾 ∈ G, 𝛼◊(𝛽 △ 𝛾) = (𝛼◊𝛽) △ (𝛼◊𝛾) and (𝛼 △ 𝛽)◊𝛾
= (𝛼◊𝛾) △ (𝛽◊𝛾).
9. Multiplicative commutative law: For every 𝛼, 𝛽 ∈ G, 𝛼◊𝛽 = 𝛽◊𝛼.
10. Existence of multiplicative identity: For every 𝛼 ∈ G, there exists an element I ∈ G, called mul-
tiplicative identity, such that 𝛼◊I = I◊𝛼 = 𝛼.
11. Existence of multiplicative inverse: For every 𝛼 ∈ G, except the element ∅, there exists an element
𝛿 ∈ G, called multiplicative inverse of 𝛼, and indicated with 𝛼 −1 , such that 𝛼◊𝛿 = 𝛿◊𝛼 = I.

Any set G for which Properties 1−4 hold is called a group with respect to △. If G has a finite number
of elements, then G is called finite group, and the number of elements of G is called the order of G.
Any set G for which Properties 1−5 hold is called an Abelian group with respect to △.
Any set G for which Properties 1−8 hold is called a ring with respect to the operations △ and ◊.
Any set G for which Properties 1−9 hold is called a commutative ring with respect to the operations
△ and ◊.
Any set G for which Properties 1−10 hold is called a commutative ring with identity.
Any set G for which Properties 1−11 hold is called a field.
It can be seen that the set {0, 1, 2, … , q − 1} is a commutative ring with identity with respect to the
operations of addition ⊕ defined in (11.45) and multiplication ⊗ defined in (11.46). We will show by the
next three properties that this set satisfies also Property 11 if and only if q is a prime: in other words, we
will show that the set {0, 1, 2, … , q − 1} is a field with respect to the modulo-q addition and modulo-q
multiplication if and only if q is a prime.
Finite fields are called Galois fields; a field of q elements is usually denoted as GF(q).
530 Chapter 11. Channel codes

Property 11a of modulo-q arithmetic. If q is not a prime, each factor of q (less than q and greater
than 1) does not have a multiplicative inverse.

Proof. Let q = ab, where 1 < a, b < q; then, observing (11.46), a ⊗ b = 0. Assume that a has a multi-
plicative inverse a−1 ; then a−1 ⊗ (a ⊗ b) = a−1 ⊗ 0 = 0. Now, from a−1 ⊗ (a ⊗ b) = 0 it is 1 ⊗ b = 0;
this implies b = 0, which is a contradiction as b > 1. Similarly, we show that b does not have a multi-
plicative inverse. ◽

Property 11b of modulo-q arithmetic. If q is a prime and a ⊗ b = 0, then a = 0, or b = 0, or

a = b = 0.

Proof. Assume a ⊗ b = 0 and a, b > 0; then ab = Kq, where K < min(a, b). If 1 < a ≤ q − 1 and a has
no factors in common with q, then it must divide K; but this is impossible as K < min(a, b). The only
other possibility is that a = 1, but then a ⊗ b ≠ 0 as ab < q. ◽

Property 11c of modulo-q arithmetic. If q is a prime, all non-zero elements of the set {0, 1, 2, … , q −
1} have multiplicative inverse.

Proof. Assume the converse, that is the element j, with 1 ≤ j ≤ q − 1, does not have a multiplicative
inverse; then there must be two distinct elements a, b ∈ {0, 1, 2, … , q − 1} such that a ⊗ j = b ⊗ j. This
is a consequence of the fact that the product i ⊗ j can only assume values in the set {0, 2, 3, … , q − 1},
as by assumption i ⊗ j ≠ 1; then
(a ⊗ j) ⊕ (q − (b ⊗ j)) = 0 (11.47)
On the other hand, q − (b ⊗ j) = (q − b) ⊗ j, and
(a ⊕ (q − b)) ⊗ j = 0 (11.48)
But j ≠ 0 and consequently, by Property 11b, we have a ⊕ (q − b) = 0. This implies a = b, which is a
contradiction. ◽

Definition 11.7
An ideal I is a subset of elements of a ring R such that:
1. I is a subgroup of the additive group R, that is the elements of I form a group with respect to the
addition defined in R;
2. for any element a of I and any element r of R, ar and ra are in I.
◽

Polynomials with coefficients from a field

We consider the set of polynomials in one variable; as this set is interesting in two distinct applications,
to avoid confusion, we will use a different notation for the two cases.
The first application permits to extend our knowledge of finite fields. We have seen in Section 11.2.2
how to construct a field with a prime number of elements. Polynomials allow us to construct fields in
which the number of elements is given by a power of a prime; for this purpose, we will use polynomials
in the variable y.
The second application introduces an alternative method to describe code words. We will consider
cyclic codes, a subclass of parity check codes, and in this context will use polynomials in the single
variable x.
11.2. Block codes 531

Consider any two polynomials with coefficients from the set {0, 1, 2, , p − 1}, where p is a prime:
g(y) = g0 + g1 y + g2 y2 + · · · + gm ym
(11.49)
f (y) = f0 + f1 y + f2 y2 + · · · + fn yn
We assume that gm ≠ 0 and fn ≠ 0. We define m as degree of the polynomial g(y), and we write m =
deg(g(y)); in particular, if g(y) = a, a ∈ {0, 1, 2, … , p − 1}, we say that deg(g(y)) = 0. Similarly, it is
n = deg(f (y)). If gm = 1, we say that g(y) is a monic polynomial.
Assume m ≤ n: then the addition among polynomials is defined as
f (y) + g(y) = (f0 ⊕ g0 ) + (f1 ⊕ g1 ) y + (f2 ⊕ g2 ) y2 + · · · + (fm ⊕ gm ) ym + · · · + fn yn (11.50)

Example 11.2.6
Let p = 5, f (y) = 1 + 3y + 2y4 , and g(y) = 4 + 3y + 3y2 ; then f (y) + g(y) = y + 3y2 + 2y4 .

Note that
deg (f (y) + g(y)) ≤ max (deg (f (y)), deg(g(y)))
Multiplication among polynomials is defined as usual
f (y) g(y) = d0 + d1 y + · · · + dm+n ym+n (11.51)
where the arithmetic to perform operations with the various coefficients is modulo p,
di = (f0 ⊗ gi ) ⊕ (f1 ⊗ gi−1 ) ⊕ … ⊕ (fi−1 ⊗ g1 ) ⊕ (fi ⊗ g0 ) (11.52)

Example 11.2.7
Let p = 2, f (y) = 1 + y + y3 , and g(y) = 1 + y2 + y3 ; then f (y) g(y) = 1 + y + y2 + y3 + y4 + y5 + y6 .

Note that
deg (f (y) g(y)) = deg(f (y)) + deg(g(y))

Definition 11.8
If f (y) g(y) = d(y), we say that f (y) divides d(y), and g(y) divides d(y). We say that p(y) is an irreducible
polynomial if and only if, assuming another polynomial a(y) divides p(y), then a(y) = a ∈ {0, 1, … , p −
1} or a(y) = k p(y), with k ∈ {0, 1, … , p − 1}. ◽

The concept of an irreducible polynomial plays the same role in the theory of polynomials as does the
concept of a prime number in the number theory.

Modular arithmetic for polynomials

We define a modulo arithmetic for polynomials, analogously to the modulo q arithmetic for integers.
We choose a polynomial q(y) = q0 + q1 y + … + qm ym with coefficients that are elements of the field
{0, 1, 2, … , p − 1}. We consider the set  of all polynomials of degree less than m with coefficients
from the field {0, 1, 2, … , p − 1}; this set consists of pm polynomials.

Example 11.2.8
Let p = 2 and q(y) = 1 + y + y3 ; then the set  consists of 23 polynomials, {0, 1, y, y + 1, y2 , y2 + 1, y2 +
y, y2 + y + 1}.
532 Chapter 11. Channel codes

Example 11.2.9
Let p = 3 and q(y) = 2y2 ; then the set  consists of 32 polynomials, {0, 1, 2, y, y + 1, y + 2, 2y, 2y +
1, 2y + 2}.

We now define two operations among polynomials of the set , namely modulo q(y) addition, denoted
by △, and modulo q(y) multiplication, denoted by ◊. Modulo q(y) addition is defined for every pair of
polynomials a(y) and b(y) from the set  as
a(y) △ b(y) = a(y) + b(y) (11.53)
where a(y) + b(y) is defined in (11.50).
The definition of modulo q(y) multiplication requires the knowledge of the Euclidean division algo-
rithm.

Euclidean division algorithm. For every pair of polynomials 𝛼(y) and 𝛽(y) with coefficients from
some field, and deg(𝛽(y)) ≥ deg(𝛼(y)) > 0, there exists a unique pair of polynomials q(y) and r(y) such
that
𝛽(y) = q(y) 𝛼(y) + r(y) (11.54)
where 0 ≤ deg(r(y)) < deg(𝛼(y)); polynomials q(y) and r(y) are called, respectively, quotient polynomial
and remainder or residue polynomial. In a notation analogous to that used for integers, we can write
⌊ ⌋
𝛽(y)
q(y) = (11.55)
𝛼(y)
and ⌊ ⌋
𝛽(y)
r(y) = 𝛽(y) − 𝛼(y) (11.56)
𝛼(y)

Example 11.2.10
Let p = 2, 𝛽(y) = y4 + 1, and 𝛼(y) = y3 + y + 1; then y4 + 1 = y(y3 + y + 1) + y2 + y + 1, so q(y) = y
and r(y) = y2 + y + 1.

We define modulo q(y) multiplication, denoted by ◊, for polynomials a(y) and b(y) in the set  as
⎧ a(y) b(y) if deg(a(y) b(y)) < deg(q(y))
⎪ ⌊ ⌋
a(y)◊b(y) = ⎨ a(y) b(y) (11.57)
⎪ a(y) b(y) − q(y) otherwise
⎩ q(y)

It is easier to think of (11.57) as a typical multiplication operation for polynomials whose coefficients
are given according to (11.52). If in this multiplication, there are terms of degree greater than or equal
to deg(q(y)), then we use the relation q(y) = 0 to lower the degree.

Example 11.2.11
Let p = 2 and q(y) = 1 + y + y3 ; then (y2 + 1)◊(y + 1) = y3 + y2 + y + 1 = (−1 − y) + y2 + y + 1 =
(1 + y) + y2 + y + 1 = y2 .

It can be shown that the set of polynomials with coefficients from some field and degree less than
deg(q(y)) is a commutative ring with identity with respect to the operations modulo q(y) addition and
modulo q(y) multiplication. We now find under what conditions this set of polynomials and operations
forms a field.
11.2. Block codes 533

Property 11a of modular polynomial arithmetic. If q(y) is not irreducible, then the factors of q(y),
of degree greater than zero and less than deg(q(y)), do not have multiplicative inverses.

Proof. Let q(y) = a(y) b(y), where 0 < deg(a(y)), deg(b(y)) < deg(q(y)); then a(y)◊b(y) = 0. Assume
a(y) has a multiplicative inverse, a−1 (y); then, from a−1 (y) ◊(a(y)◊b(y)) = a−1 (y)◊0 = 0 it is
(a−1 (y)◊a(y))◊b(y) = 0, then 1◊b(y) = 0, or b(y) = 0. The last equation is a contradiction as by
assumption deg(b(y)) > 0. Similarly, we show that b(y) does not have a multiplicative inverse. ◽

We give without proof the following properties:

Property 11b of modular polynomial arithmetic. If q(y) is irreducible and a(y)◊b(y) = 0, then a(y) =
0, or b(y) = 0, or a(y) = b(y) = 0.

Property 11c of modular polynomial arithmetic. If q(y) is irreducible, all non-zero elements of the
set of polynomials  of degree less than deg(q(y)) have multiplicative inverses.
We now have that the set of polynomials with coefficients from some field and degree less than
deg(q(y)) forms a field, with respect to the operations of modulo q(y) addition and modulo q(y) multi-
plication, if and only if q(y) is irreducible.
Furthermore, it can be shown that there exists at least one irreducible polynomial of degree m, for every
m ≥ 1, with coefficients from a generic field {0, 1, 2, … , p − 1}. We now have a method of generating
a field with pm elements.

Example 11.2.12
Let p = 2 and q(y) = y2 + y + 1; we have that q(y) is irreducible. Consider the set  with elements
{0, 1, y, y + 1}. The addition and multiplication tables for these elements modulo y2 + y + 1 are given
in Table 11.5 and Table 11.6, respectively.

Table 11.5: Modulo y2 + y + 1 addition table for p = 2.

△ 0 1 y y+1
0 0 1 y y+1
1 1 0 y+1 y
y y y+1 0 1
y+1 y+1 y 1 0

Table 11.6: Modulo y2 + y + 1 multiplication table for p = 2.

◊ 0 1 y y+1
0 0 0 0 0
1 0 1 y y+1
y 0 y y+1 1
y+1 0 y+1 1 y
534 Chapter 11. Channel codes

Devices to sum and multiply elements in a finite field

For the GF(pm ) obtained by an irreducible polynomial of degree m,
∑
m
q(y) = q i yi , qi ∈ GF(p) (11.58)
i=0

let a(y) and b(y) be two elements of :

∑
m−1
a(y) = a i yi , ai ∈ GF(p) (11.59)
i=0

and
∑
m−1
b(y) = b i yi , bi ∈ GF(p) (11.60)
i=0

The device to perform the addition(11.53),

∑
m−1
s(y) = si yi = (a(y) + b(y)) mod q(y) (11.61)
i=0

is illustrated in Figure 11.1. The implementation of a device to perform the multiplication is slightly more
complicated, as illustrated in Figure 11.2, where Tc is the period of the clock applied to the shift-register
(SR) with m elements, and all operations are modulo p.
Let us define
∑
m−1
d(y) = di yi = (a(y) b(y)) mod q(y) (11.62)
i=0

The device is based on the following decomposition:

∑
m−1
a(y) b(y) mod q(y) = ai yi b(y) mod q(y)
i=0
= a0 b(y) (11.63)
+ a1 (y b(y)) mod q(y)
⋮
+ am−1 (ym−1 b(y)) mod q(y)
∑m
where additions and multiplications are modulo p. Now, using the identity i=0 qi yi = 0 mod q(y),
note that the following relation holds:
y b(y) = b0 y + b1 y2 + · · · + bm−2 ym−1 + bm−1 ym
(11.64)
= (−bm−1 q−1 −1 −1
m q0 ) + (b0 − bm−1 qm q1 ) y + · · · + (bm−2 − bm−1 qm qm−1 ) y
m−1

a0 b0 am−1 bm−1

...
mod p mod p

s0 sm−1

Figure 11.1 Device for the sum of two elements (a0 , … , am−1 ) and (b0 , … , bm−1 ) of GF(pm ).
11.2. Block codes 535

q0 q1 qm–1 – qm–1

Tc Tc Tc
b0 b1 bm–1

ai ai ai

ACC ACC ACC

d0 d1 dm–1

Figure 11.2 Device for the multiplication of two elements (a0 , … , am−1 ) and (b0 , … , bm−1 ) of
GF(pm ). Tc is the clock period, and ACC denotes an accumulator. All additions and multiplications
are modulo p.

The term (yi b(y)) mod q(y) is thus obtained by initializing the SR of Figure 11.2 to the sequence
(b0 , … , bm−1 ), and by applying i clock pulses; the desired result is then contained in the shift
register.
Observing (11.63), we find that it is necessary to multiply each element of the SR by ai and accumulate
the result; after multiplications by all coefficients {ai } have been performed, the final result is given by
the content of the accumulators. Note that in the binary case, for p = 2, the operations of addition and
multiplication are carried out by XOR and AND functions, respectively.

Remarks on finite fields

1. We have seen how to obtain finite fields with p (p a prime) elements, given by {0, 1, … , p − 1},
or pm elements, using the Property 11c. These fields are also known as Galois fields and
are usually denoted by GF(p) or GF(pm ). It can be shown that there are no other fields with
a finite number of elements. Moreover, all fields with the same number of elements are
identical, that is all finite fields are generated by the procedures discussed in the previous
sections.
2. The field, from which the coefficients of the irreducible polynomial are chosen, is called ground
field; the field generated using the arithmetic of polynomials is called extension field.
3. Every row of the addition table contains each field element once and only once; the same is true
for the columns.
4. Every row of the multiplication table, except the row corresponding to the element 0, contains each
field element once and only once; the same is true for the columns.
5. If we multiply any non-zero element by itself, we get a non-zero element of the field (perhaps
itself). As there are only (q − 1) non-zero elements, we must eventually find a situation for
536 Chapter 11. Channel codes

j > i such that an element 𝛼 multiplied by itself j times will equal 𝛼 multiplied by itself i times,
that is
j times i times
(11.65)
𝛼⊗𝛼⊗𝛼⊗···⊗𝛼 = 𝛼⊗𝛼⊗···⊗𝛼 = 𝛽
We observe that
j − i times i times j times
(11.66)
𝛼⊗𝛼⊗···⊗𝛼 ⊗ 𝛼⊗𝛼⊗···⊗𝛼 = 𝛼⊗𝛼⊗···⊗𝛼
Substituting (11.65) in (11.66), and observing that 𝛽 has a multiplicative inverse, we can multiply
from the right by this inverse to obtain
j − i times
(11.67)
𝛼 j−i = 𝛼 ⊗ 𝛼 ⊗ · · · ⊗ 𝛼 = 1

Definition 11.9
For every non-zero field element, 𝛼, the order of 𝛼 is the smallest integer 𝓁 such that 𝛼 𝓁 = 1. ◽

Example 11.2.13
Consider the field with elements {0, 1, 2, 3, 4} and modulo 5 arithmetic. Then
element order
1 1
2 4 (11.68)
3 4
4 2

Example 11.2.14
Consider the field GF(22 ) with four elements, {0, 1, y, y + 1}, and addition and multiplication modulo
y2 + y + 1. Then
element order
1 1
(11.69)
y 3
y+1 3

6. An element from the field GF(q) is said to be primitive if it has order q − 1. For fields generated
by arithmetic modulo a polynomial q(y), if the field element y is primitive, we say that q(y) is a
primitive irreducible polynomial.
A property of finite fields that we give without proof is that every finite field has at least one primitive
element; we note that once a primitive element has been identified, every other non-zero field element
can be obtained by multiplying the primitive element by itself an appropriate number of times. A list of
primitive polynomials for the ground field GF(2) is given in Table 11.7.

Example 11.2.15
For the field GF(4) generated by the polynomial arithmetic modulo q(y) = y2 + y + 1, for the ground
field GF(2), y is a primitive element (see (11.69)); thus y2 + y + 1 is a primitive polynomial.
11.2. Block codes 537

Table 11.7: List of primitive polynomials q(y) of degree m for the ground field GF(2).

m m

2 1 + y + y2 14 1 + y + y6 + y10 + y14

3 1 + y + y3 15 1 + y + y15

4 1 + y + y4 16 1 + y + y3 + y12 + y16

5 1 + y2 + y 5 17 1 + y3 + y17

6 1 + y + y6 18 1 + y7 + y18

7 1 + y3 + y 7 19 1 + y + y2 + y5 + y19

8 1 + y2 + y 3 + y 4 + y 8 20 1 + y3 + y20

9 1 + y4 + y 9 21 1 + y2 + y21

10 1 + y3 + y10 22 1 + y + y22

11 1 + y2 + y11 23 1 + y5 + y23

12 1 + y + y4 + y6 + y12 24 1 + y + y2 + y7 + y24

13 1 + y + y3 + y4 + y13

7. The order of every non-zero element of GF(q) must divide (q − 1).

Proof. Every non-zero element 𝛽 can be written as the power of a primitive element 𝛼p ; this implies
that there is some i < (q − 1) such that
i times
(11.70)
𝛽 = 𝛼p ⊗ 𝛼p ⊗ · · · ⊗ 𝛼p = 𝛼pi
q−1 j
Note that from the definition of a primitive element, we get 𝛼p = 1, but 𝛼p ≠ 1 for j < (q − 1); further-
more, there exists an integer 𝓁 such that 𝛽 𝓁 = 𝛼pi𝓁 = 1. Consequently, (𝓁)(i) is a multiple of (q − 1), and
it is exactly the smallest multiple of i that is a multiple of (q − 1), thus (i)(𝓁) = l.c.m.(i, q − 1), i.e. the
least common multiple of i and (q − 1). We recall that

ab
l.c.m.(a, b) = (11.71)
g.c.d.(a, b)

where g.c.d.(a, b) is the greatest common divisor of a and b. Thus,

(i)(q − 1)
(i)(𝓁) = (11.72)
g.c.d.(i, q − 1)

and
q−1
=𝓁 (11.73)
g.c.d.(i, q − 1) ◽
538 Chapter 11. Channel codes

Example 11.2.16
Let 𝛼p be a primitive element of GF(16); from (11.73), the orders of the non-zero field elements are

field element g.c.d.(i, q − 1) order of field element

q−1
𝛽 = 𝛼i
g.c.d.(i, q − 1)
𝛼p 1 15
𝛼p2 1 15
𝛼p3 3 5
𝛼p4 1 15
𝛼p5 5 3
𝛼p6 3 5
𝛼p7 1 15 (11.74)
𝛼p8 1 15
𝛼p9 3 5
𝛼p10 5 3
𝛼p11 1 15
𝛼p12 3 5
𝛼p13 1 15
𝛼p14 1 15
𝛼p15 15 1

8. A ground field can itself be generated as an extension field. For example GF(16) can be generated by
taking an irreducible polynomial of degree 4 with coefficients from GF(2), which we would call GF(24 ),
or by taking an irreducible polynomial of degree 2 with coefficients from GF(4), which we would call
GF(42 ). In either case, we would have the same field, except for the names of the elements.

Example 11.2.17
Consider the field GF(23 ) generated by the primitive polynomial q(y) = 1 + y + y3 , with ground field
GF(2). As q(y) is a primitive polynomial, each element of GF(23 ), except the zero element, can be
expressed as a power of y. Recalling the polynomial representation , we may associate to each poly-
nomial a vector representation, with m components on GF(p) given by the coefficients of the powers of
the variable y. The three representations are reported in Table 11.8.

Roots of a polynomial
Consider a polynomial of degree m with coefficients that are elements of some field. We will use the
variable x, as the polynomials are now considered for a purpose that is not that of generating a finite
field. In fact, the field of the coefficients may itself have a polynomial representation.
Consider, for example a polynomial in x with coefficients from GF(4). We immediately see that it
is not worth using the notation {0, 1, y, y + 1} to identify the four elements of GF(4), as the notation
{0, 1, 𝛼, 𝛽} would be much simpler. For example a polynomial of degree three with coefficients from
GF(4) is given by f (x) = 𝛼x3 + 𝛽x2 + 1.
11.2. Block codes 539

Table 11.8: Three equivalent representations of the

elements of GF(23 ).

exponential polynomial binary

(y0 y1 y2 )
0 0 000
1 1 100
y y 010
y2 y2 001
y3 1+y 110
4 2
y y+y 011
y5 1 + y + y2 111
y6 1 + y2 101

Given any polynomial f (x), we say that 𝛾 is a root of the equation f (x) = 0 or, more simply, that it is a
root of f (x), if and only if f (𝛾) = 0. The definition is more complicated than it appears, as we must know
the meaning of the two members of the equation f (𝛾) = 0. For example, we recall that the fundamental
theorem of algebra states that every polynomial of degree m has exactly m roots, not necessarily distinct.
If we take the polynomial f (x) = x2 + x + 1 with coefficients from {0, 1}, what are its roots? As f (0) =
f (1) = 1, we have that neither 0 nor 1 are roots.
Before proceeding, we recall a similar situation that we encounter in ordinary algebra. The polynomial
2
x + 3, with coefficients in the field of real numbers, has two roots in the field of complex numbers;
however, no roots exist in the field of real numbers; therefore, the polynomial does not have factors
whose coefficients are real numbers. Thus, we would say that the polynomial is irreducible, yet even the
irreducible polynomial has complex-valued roots and can be factorized.
This situation is due to the fact that, if we have a polynomial f (x) with coefficients from some field, the
roots of the polynomial are either from that field or from an extension field of that field. For example, take
the polynomial f (x) = x2 + x + 1 with coefficients from GF(2), and consider the extension field GF(4)
with elements {0, 1, 𝛼, 𝛽} that obey the addition and the multiplication rules given in Tables 11.9 and
11.10, respectively. Then f (𝛼) = f (x)|x=𝛼 = 𝛼 2 △ 𝛼 △ 1 = (𝛽 △ 𝛼) △ 1 = 1 △ 1 = 0, thus 𝛼 is a root.

Table 11.9: Addition table for the elements of GF(4).

△ 0 1 𝛼 𝛽
0 0 1 𝛼 𝛽
1 1 0 𝛽 𝛼
𝛼 𝛼 𝛽 0 1
𝛽 𝛽 𝛼 1 0

Table 11.10: Multiplication table for the elements of GF(4).

◊ 0 1 𝛼 𝛽
0 0 0 0 0
1 0 1 𝛼 𝛽
𝛼 0 𝛼 𝛽 1
𝛽 0 𝛽 1 𝛼
540 Chapter 11. Channel codes

Similarly, we find f (𝛽) = f (x)|x=𝛽 = 𝛽 2 △ 𝛽 △ 1 = (𝛼 △ 𝛽) △ 1 = 1 △ 1 = 0, thus, the two roots of

f (x) are 𝛼 and 𝛽. We can factor f (x) into two factors, each of which is a polynomial in x with coefficients
from GF(4). For this purpose, we consider (x △ −𝛼)◊(x △ −𝛽) = (x △ 𝛼)◊(x △ 𝛽); leaving out the
notations △ and ◊ for + and ×, we get
(x △ 𝛼)◊(x △ 𝛽) = x2 + (𝛼 + 𝛽) x + 𝛼𝛽 = x2 + x + 1 (11.75)
Thus, if we use the operations defined in GF(4), (x + 𝛼) and (x + 𝛽) are factors of x2 + x + 1; it remains
that x2 + x + 1 is irreducible as it has no factors with coefficients from GF(2).

Property 1 of the roots of a polynomial. If 𝛾 is a root of f (x) = 0, then (x − 𝛾), that is (x + (−𝛾)), is a
factor of f (x).

Proof. Using the Euclidean division algorithm, we divide f (x) by (x − 𝛾) to get

f (x) = Q(x) (x − 𝛾) + r(x) (11.76)
where deg(r(x)) < deg(x − 𝛾) = 1. Therefore,
f (x) = Q(x) (x − 𝛾) + r0 (11.77)
But f (𝛾) = 0, so
f (𝛾) = 0 = Q(𝛾) (𝛾 − 𝛾) + r0 = r0 (11.78)
therefore,
f (x) = Q(x) (x − 𝛾) (11.79)
◽

Property 2 of the roots of a polynomial. If f (x) is an arbitrary polynomial with coefficients from
GF(p), p a prime, and 𝛽 is a root of f (x), then 𝛽 p is also a root of f (x).

Proof. We consider the polynomial f (x) = f0 + f1 x + f2 x2 + · · · + fm xm , where fi ∈ GF(p), and form the
power (f (x))p . It results
p p p
(f (x))p = (f0 + f1 x + f2 x2 + · · · + fm xm )p = f0 + f1 xp + · · · + fm xmp (11.80)
as the cross terms contain a factor p, which is the same as 0 in GF(p). On the other hand, for fi ≠ 0,
p
fi = fi , as from Property 7 on page 537 the order of any non-zero element divides p − 1; the equation is
true also if fi is the zero element. Therefore,
(f (x))p = f (xp ) (11.81)
If 𝛽 is a root of f (x) = 0, then f (𝛽) = 0, and f p (𝛽) = 0. But f p (𝛽) = f (𝛽 p ), so that f (𝛽 p ) = 0; therefore,
𝛽 p is also a root of f (x). ◽

A more general form of the property just introduced, that we will give without proof, is expressed by
the following property.

Property 2a of the roots of a polynomial. If f (x) is an arbitrary polynomial having coefficients from
GF(q), with q a prime or a power of a prime, and 𝛽 is a root of f (x) = 0, then 𝛽 q is also a root of f (x) = 0,

Example 11.2.18
Consider the polynomial x2 + x + 1 with coefficients from GF(2). We already have seen that 𝛼, element
of GF(4), is a root of x2 + x + 1 = 0. Therefore, 𝛼 2 is also a root, but 𝛼 2 = 𝛽, so 𝛽 is a second root. The
polynomial has degree 2, thus it has 2 roots, and they are 𝛼 and 𝛽, as previously seen. Note also that 𝛽 2
is also a root, but 𝛽 2 = 𝛼.
11.2. Block codes 541

Minimum function

Definition 11.10
Let 𝛽 be an element of an extension field of GF(q); the minimum function of 𝛽, m𝛽 (x), is the monic
polynomial of least degree with coefficients from GF(q) such that m𝛽 (x)|x=𝛽 = 0. ◽

We now list some properties of the minimum function.

1. The minimum function is unique.

Proof. Assume there were two minimum functions, of the same degree and monic, m𝛽 (x) and m′𝛽 (x).
Form the new polynomial (m𝛽 (x) − m′𝛽 (x)) whose degree is less than the degree of m𝛽 (x) and m′𝛽 (x); but
(m𝛽 (x) − m′𝛽 (x))|x=𝛽 = 0, so we have a new polynomial, whose degree is less than that of the minimum
function, that admits 𝛽 as root. Multiplying by a constant, we can thus find a monic polynomial with
this property, but this cannot be since the minimum function is the monic polynomial of least degree for
which 𝛽 is a root. ◽

2. The minimum function is irreducible.

Proof. Assume the converse were true, that is m𝛽 (x) = a(x) b(x); then m𝛽 (x)|x=𝛽 = a(𝛽) b(𝛽) = 0. Then
either a(𝛽) = 0 or b(𝛽) = 0, so that 𝛽 is a root of a polynomial of degree less than the degree of m𝛽 (x).
By making this polynomial monic, we arrive at a contradiction. ◽

3. Let f (x) be any polynomial with coefficients from GF(q), and let f (x)|x=𝛽 = 0; then f (x) is divisible
by m𝛽 (x).

Proof. Use the Euclidean division algorithm to yield

f (x) = Q(x) m𝛽 (x) + r(x) (11.82)

where deg(r(x)) < deg(m𝛽 (x)). Then we have that

f (𝛽) = Q(𝛽) m𝛽 (𝛽) + r(𝛽) (11.83)

but as f (𝛽) = 0 and m𝛽 (𝛽) = 0, then r(𝛽) = 0. As deg(r(x)) < deg(m𝛽 (x)), the only possibility is r(x) = 0;
thus f (x) = Q(x) m𝛽 (x). ◽

4. Let f (x) be any irreducible monic polynomial with coefficients from GF(q) for which f (𝛽) = 0, where
𝛽 is an element of some extension field of GF(q); then f (x) = m𝛽 (x).

Proof. From Property 3 f (x) must be divisible by m𝛽 (x), but f (x) is irreducible, so it is only trivially
divisible by m𝛽 (x), that is f (x) = K m𝛽 (x): but f (x) and m𝛽 (x) are both monic polynomials, therefore,
K = 1. ◽

We now introduce some interesting propositions.

1. Let 𝛽 be an element of GF(qm ), with q prime; then the polynomial F(x), defined as
∏
m−1
i 2 m−1
F(x) = (x − 𝛽 q ) = (x − 𝛽) (x − 𝛽 q ) (x − 𝛽 q ) · · · (x − 𝛽 q ) (11.84)
i=0

has all its coefficients from GF(q).

542 Chapter 11. Channel codes

Proof. Observing Property 7 on page 537, we have that the order of 𝛽 divides qm − 1, therefore, 𝛽 q = 𝛽.
m

Thus, we can express F(x) as

∏m
i
F(x) = (x − 𝛽 q ) (11.85)
i=1

Therefore,
∏
m
i
∏
m
i−1
∏
m−1
j
F(xq ) = (xq − 𝛽 q ) = (x − 𝛽 q )q = (x − 𝛽 q )q = (F(x))q (11.86)
i=1 i=1 j=0
∑m
Consider now the expression F(x) = i=0 fi xi ; then
∑
m
q
F(xq ) = fi xi (11.87)
i=0

and ( )q
∑
m
∑
m
q q
q i
(F(x)) = fi x = fi xi (11.88)
i=0 i=0
q
Equating like coefficients in (11.87) and (11.88), we get fi = fi ; hence, fi is a root of the equation
xq − x = 0. But on the basis of Property 7 on page 537, the q elements from GF(q) all satisfy the equation
xq − x = 0, and this equation only has q roots; therefore, the coefficients fi are elements from GF(q). ◽

2. If g(x) is an irreducible polynomial of degree m with coefficients from GF(q), and g(𝛽) = 0, where 𝛽
is an element of some extension field of GF(q), then 𝛽, 𝛽 q , 𝛽 q , … , 𝛽 q are all the roots of g(x).
2 m−1

Proof. At least one root of g(x) is in GF(qm ); this follows by observing that, if we form GF(qm ) using
the arithmetic modulo g(y), then y will be a root of g(x) = 0. From Proposition 11.1, if 𝛽 is an element
∏m−1
from GF(qm ), then F(x) = i=0 (x − 𝛽 q ) has all coefficients from GF(q); thus, F(x) has degree m, and
i

F(𝛽) = 0. As g(x) is irreducible, we know that g(x) = K m𝛽 (x), but as F(𝛽) = 0, and F(x) and g(x) have
the same degree, then F(x) = K1 m𝛽 (x), and therefore, g(x) = K2 F(x). As 𝛽, 𝛽 q , 𝛽 q , … , 𝛽 q , are all roots
2 m−1

of F(x), then they must also be all the roots of g(x). ◽

3. Let g(x) be a polynomial with coefficients from GF(q) which is also irreducible in this field. Moreover,
let g(𝛽) = 0, where 𝛽 is an element of some extension field of GF(q); then the degree of g(x) equals the
smallest integer k such that
k
𝛽q = 𝛽 (11.89)

Proof. We have that deg(g(x)) ≥ k as 𝛽, 𝛽 q , 𝛽 q , … , 𝛽 q , are all roots of g(x) and by assumption are
2 k−1

distinct. Assume that deg(g(x)) > k; from Proposition 11.2, we know that 𝛽 must be at least a double
root of g(x) = 0, and therefore, g′ (x) = (d∕dx)g(x) = 0 must also have 𝛽 as a root. As g(x) is irreducible,
we have that g(x) = K m𝛽 (x), but m𝛽 (x) must divide g′ (x); we get a contradiction because deg(g′ (x)) <
deg(g(x)). ◽

Methods to determine the minimum function

1. Direct calculation.
Example 11.2.19
Consider the field GF(23 ) obtained by taking the polynomial arithmetic modulo the irreducible poly-
nomial y3 + y + 1 with coefficients from GF(2); the field elements are {0, 1, y, y + 1, y2 , y2 + 1, y2 +
11.2. Block codes 543

y, y2 + y + 1}. Assume we want to find the minimum function of 𝛽 = (y + 1). If (y + 1) is a root, also
(y + 1)2 = y2 + 1 and (y + 1)4 = y2 + y + 1 are roots. Note that (y + 1)8 = (y + 1) = 𝛽, thus the mini-
mum function is
my+1 (x) = (x − 𝛽) (x − 𝛽 2 ) (x − 𝛽 4 )
= (x + (y + 1))(x + (y2 + 1))(x + (y2 + y + 1)) (11.90)
= x3 + x2 + 1

2. Solution of the system of the coefficient equations.

Example 11.2.20
Consider the field GF(23 ) of the previous example; as (y + 1), (y + 1)2 = y2 + 1, (y + 1)4 = y2 + y +
1, (y + 1)8 = y + 1, the minimum function has degree three; as the minimum function is monic and
irreducible, we have
my+1 (x) = m3 x3 + m2 x2 + m1 x + m0 = x3 + m2 x2 + m1 x + 1 (11.91)
As my+1 (y + 1) = 0, then
(y + 1)3 + m2 (y + 1)2 + m1 (y + 1) + 1 = 0 (11.92)
that can be written as
y2 (1 + m2 ) + ym1 + (m2 + m1 + 1) = 0 (11.93)
As all coefficients of the powers of y must be zero, we get a system of equations in the unknown m1 and
m2 , whose solution is given by m1 = 0 and m2 = 1. Substitution of this solution in (11.91) yields
my+1 (x) = x3 + x2 + 1 (11.94)

3. Using the minimum function of the multiplicative inverse.

Definition 11.11
The reciprocal polynomial of any polynomial m𝛼 (x) = m0 + m1 x + m2 x2 + · · · + mK xK is defined by
m𝛼 (x) = m0 xK + m1 xK−1 + · · · + mK−1 x + mK . ◽

We use the following proposition that we give without proof.

The minimum function of the multiplicative inverse of a given element is equal to the reciprocal of
the minimum function of the given element. In formulas: let 𝛼𝛽 = 1, then m𝛽 (x) = m𝛼 (x).

Example 11.2.21
Consider the field GF(26 ) obtained by taking the polynomial arithmetic modulo the irreducible polyno-
mial y6 + y + 1 with coefficients from GF(2); the polynomial y6 + y + 1 is primitive, thus from Prop-
erty 7 on page 537 any non-zero field element can be written as a power of the primitive element
y. From Proposition 11.2, we have that the minimum function of y is also the minimum function of
y2 , y4 , y8 , y16 , y32 , the minimum function of y3 is also the minimum function of y6 , y12 , y24 , y48 , y33 , and
so forth. We list in Table 11.11 the powers of y that have the same minimum function.
Given the minimum function of y11 , my11 = x6 + x5 + x3 + x2 + 1, we want to find the minimum func-
tion of y13 . From Table 11.11, we note that y13 has the same minimum function as y52 ; furthermore,
we note that y52 is the multiplicative inverse of y11 , as (y11 )(y52 ) = y63 = 1. Therefore, the minimum
function of y13 is the reciprocal polynomial of my11 , given by my13 = x6 + x4 + x3 + x + 1.
544 Chapter 11. Channel codes

Table 11.11: Powers of a primitive element in GF(26 ) with the same

minimum function.
1 2 4 8 16 32
3 6 12 24 48 33
5 10 20 40 17 34
7 14 28 56 49 35
9 18 36
11 22 44 25 50 37
13 26 52 41 19 38
15 30 60 57 51 39
21 42
23 46 29 58 53 43
27 54 45
31 62 61 59 55 47

Properties of the minimum function

1. Let 𝛽 be an element of order n in an extension field of GF(q), and let m𝛽 (x) be the minimum function
of 𝛽 with coefficients from GF(q); then xn − 1 = m𝛽 (x) b(x), but xi − 1 ≠ m𝛽 (x) b(x) for i < n.

Proof. We show that 𝛽 is a root of xn − 1, as 𝛽 n − 1 = 0, but from Property 3 of the minimum function
(see page 541) we know that m𝛽 (x) divides any polynomial f (x) such that f (𝛽) = 0; this proves the
first part.
Assume that xi − 1 = m𝛽 (x) b(x) for some i < n: then

xi − 1|x=𝛽 = m𝛽 (x) b(x)|x=𝛽 = 0 (11.95)

so 𝛽 i − 1 = 0 for i < n. But from Definition 11.2.9 of the order of 𝛽 (see page 536), n is the smallest
integer such that 𝛽 n = 1, hence, we get a contradiction. ◽

2. Let 𝛽1 , 𝛽2 , … , 𝛽L be elements of some extension field of GF(q), and let 𝓁1 , 𝓁2 , … , 𝓁L be the orders of
these elements, respectively. Moreover, let m𝛽1 (x), m𝛽2 (x), … , m𝛽L (x) be the minimum functions of these
elements with coefficients from GF(q), and let g(x) be the smallest monic polynomial with coefficients
from GF(q) that has 𝛽1 , 𝛽2 , … , 𝛽L as roots: then

(a) g(x) = l.c.m.(m𝛽1 (x), m𝛽2 (x), … , m𝛽L (x));

(b) if the minimum functions are all distinct, that is they do not have factor polynomials in common,
then g(x) = m𝛽1 (x) m𝛽2 (x) · · · m𝛽L (x);
(c) if n = l.c.m.(𝓁1 , 𝓁2 , … , 𝓁L ), then xn − 1 = h(x) g(x), and xi − 1 ≠ h(x) g(x) for i < n.

Proof.
(a) Noting that g(x) must be divisible by each of the minimum functions, it must be the smallest
degree of monic polynomial divisible by m𝛽1 (x), m𝛽2 (x), …, m𝛽L (x), but this is just the definition
of the least common multiple.
(b) If all the minimum functions are distinct, as each is irreducible, the least common multiple is
given by the product of the polynomials.
11.2. Block codes 545

(c) As n is a multiple of the order of each element, 𝛽jn − 1 = 0, for j = 1, 2, … , L; then xn − 1 must
be divisible by m𝛽j (x), for j = 1, 2, … , L, and therefore, it must be divisible by the least common
multiple of these polynomials. Assume now that g(x) divides xi − 1 for i < n; then 𝛽ji − 1 = 0 for
each j = 1, 2, … , L, and thus i is a multiple of 𝓁1 , 𝓁2 , … , 𝓁L . But n is the smallest integer multiple
of 𝓁1 , 𝓁2 , … , 𝓁L , hence, we get a contradiction. ◽

We note that if the extension field is GF(qk ) and L = qk − 1 = n, then g(x) = xn − 1 and h(x) = 1.

11.2.3 Cyclic codes

In Section 11.2.1, we dealt with the theory of binary group codes. We now discuss a special class of
linear codes. These codes, called cyclic codes, are based upon polynomial algebra and lead to particularly
efficient implementations for encoding and decoding.

The algebra of cyclic codes

We consider polynomials with coefficients from some field GF(q); in Particular, we consider the poly-
nomial xn − 1, and assume it can be factorized as

xn − 1 = g(x) h(x) (11.96)

Many such factorizations are possible for a given polynomial xn − 1; we will consider any one of them.
We denote the degrees of g(x) and h(x) as r and k, respectively; thus, n = k + r. The choice of the symbols
n, k and r is intentional, as they assume the same meaning as in the previous sections.
The polynomial arithmetic modulo q(x) = xn − 1 is particularly important in the discussion of cyclic
codes.

Proposition 11.2
Consider the set of all polynomials of the form c(x) = a(x) g(x) modulo q(x), as a(x) ranges over all
polynomials of all degrees with coefficients from GF(q). This set must be finite as there are at most qn
remainder polynomials that can be obtained by dividing a polynomial by xn − 1. Now, we show that
there are exactly qk distinct polynomials. ◽

Proof. There are at least qk distinct polynomials a(x) of degree less than or equal to k − 1, and each
such polynomial leads to a distinct polynomial a(x) g(x). In fact, as the degree of a(x) g(x) is less than
r + k = n, no reduction modulo xn − 1 is necessary for these polynomials.
Now, let a(x) be a polynomial of degree greater than or equal to k. To reduce the polynomial a(x) g(x)
modulo xn − 1, we divide by xn − 1 and keep the remainder; thus

a(x) g(x) = Q(x) (xn − 1) + r(x) (11.97)

where 0 ≤ deg(r(x)) < n. By using (11.96), we can express r(x) as

r(x) = (a(x) − h(x) Q(x))g(x) = a′ (x) g(x) (11.98)

As r(x) is of degree less than n, a′ (x) is of degree less than k, but we have already considered all poly-
nomials of this form; therefore, r(x) is one of the qk polynomials determined in the first part of the
proof. ◽
546 Chapter 11. Channel codes

Example 11.2.22
Let g(x) = x + 1, GF(q) = GF(2), and n = 4; then all polynomials a(x) g(x) modulo x4 − 1 = x4 + 1 are
given by
a(x) a(x) g(x) mod (x4 − 1) code word
0 0 0000
1 x+1 1100
x x2 + x 0110
x+1 x2 + 1 1010 (11.99)
x2 x3 + x2 0011
x2 + 1 x3 + x2 + x + 1 1111
x2 + x x3 + x 0101
x2 + x + 1 x3 + 1 1001

We associate with any polynomial of degree less than n and coefficients from GF(q) a vector of length n
with components equal to the coefficients of the polynomial, that is
f (x) = f0 + f1 x + f2 x2 + · · · + fn−1 xn−1 ↔ f = (f0 , f1 , f2 , … , fn−1 ) (11.100)
Note that in the definition, fn−1 does not need to be non-zero.
We can now define cyclic codes. The code words will be the vectors associated with a set of polyno-
mials; alternatively, we speak of the polynomials themselves as being code words or code polynomials
(see (11.99)).

Definition 11.12
Choose a field GF(q), a positive integer n, and a polynomial g(x) with coefficients from GF(q) such
that xn − 1 = g(x) h(x); furthermore, let deg(g(x)) = r = n − k. Words of a cyclic code are the vectors
of length n that are associated with all multiples of g(x) reduced modulo xn − 1. In formulas: c(x) =
a(x) g(x) mod (xn − 1), for a(x) polynomial with coefficients from GF(q).
The polynomial g(x) is called generator polynomial. ◽

Properties of cyclic codes

1. In a cyclic code, there are qk code words, as shown in the previous section.
2. A cyclic code is a linear code.

Proof. The all zero word is a code word as 0 g(x) = 0; any multiple of a code word is a code word, as
if a1 (x) g(x) is a code word so is 𝛼a1 (x) g(x). Let a1 (x) g(x) and a2 (x) g(x) be two code words; then
𝛼1 a1 (x) g(x) + 𝛼2 a2 (x) g(x) = (𝛼1 a1 (x) + 𝛼2 a2 (x))g(x) = a3 (x) g(x) (11.101)
is a code word. ◽

3. Every cyclic permutation of a code word is a code word.

Proof. It is enough to show that if c(x) = c0 + c1 x + · · · + cn−2 xn−2 + cn−1 xn−1 corresponds to
a code word, then also cn−1 + c0 x + · · · + cn−3 xn−2 + cn−2 xn−1 corresponds to a code word. But
if c(x) = a(x) g(x) = c0 + c1 x + · · · + cn−2 xn−2 + cn−1 xn−1 mod(xn − 1), then xc(x) = xa(x) g(x) =
cn−1 + c0 x + · · · + cn−3 xn−2 + cn−2 xn−1 mod(xn − 1). ◽
11.2. Block codes 547

Example 11.2.23
Let GF(q) = GF(2), g(x) = x + 1, and n = 4. From the previous example, we obtain the code words,
which can be grouped by the number of cyclic shifts.
code polynomials code words cyclic shifts
0 0000 } 1
1+x 1100 ⎫
x + x2 0110 ⎪
x2 + x3 0011 ⎬ 4
(11.102)
⎪
1 + x3 1001 ⎭
1+x+x +x 2 3 1111 1
}
}
1 + x2 1010
2
x + x3 0101

4. c(x) is a code polynomial if and only if c(x) h(x) = 0 mod(xn − 1).

Proof. If c(x) is a code polynomial, then c(x) = a(x)g (x) mod(xn − 1), but h(x) c(x) = h(x) a(x) g(x) =
a(x) (g(x)h(x)) = a(x)(xn − 1) = 0 mod(xn − 1).
Assume now h(x) c(x) = 0 mod(xn − 1); then h(x) c(x) = Q(x)(xn − 1) = Q(x) h(x) g(x), or c(x) =
Q(x) g(x); therefore, c(x) is a code polynomial. ◽

5. Let xn − 1 = g(x) h(x), where g(x) = g0 + g1 x + · · · + gr xr and h(x) = h0 + h1 x + · · · + hk xk ; then the

code corresponding to all multiples of g(x) modulo xn − 1 has the generator matrix
⎡ g 0 g1 g2 … gr 0 0 … 0 ⎤
⎢ ⎥
0 g0 g1 … gr−1 gr 0 … 0 ⎥
G=⎢ (11.103)
⎢ ⎥
⎢ 0 0 0 … … gr ⎥⎦
⎣
and parity check matrix
⎡ 0 0 … 0 hk hk−1 … h1 h0 ⎤
⎢ ⎥
0 0 … hk hk−1 hk−2 … h0 0 ⎥
H=⎢ (11.104)
⎢ ⎥
⎢h h … … … 0 0 ⎥
⎣ k k−1 ⎦

Proof. We show that G is the generator matrix. The first row of G corresponds to the polynomial g(x),
the second to xg(x) and the last row to xk−1 g(x), but the code words are all words of the form
(a0 + a1 x + · · · + ak−1 xk−1 )g(x) = a0 g(x) + a1 (xg(x)) + · · · + ak−1 (xk−1 g(x)) (11.105)
But (11.105) expresses all code words as linear combinations of the rows of G; therefore, G is the
generator matrix of the code.
To show that H is the parity check matrix, we consider the product c(x) h(x). If we write
c(x) = c0 + c1 x + · · · + cn−1 xn−1 (11.106)
and
h(x) = h0 + h1 x + · · · + hk−1 xk−1 + hk xk + · · · + hn−1 xn−1 (11.107)
where hk+1 = hk+2 = · · · = hn−1 = 0, we get
d(x) = c(x) h(x) = d0 + d1 x + · · · + d2n−2 x2n−2 (11.108)
548 Chapter 11. Channel codes

where
⎧∑ i ⎫
⎪ cj hi−j if 0 ≤ i ≤ n − 1 ⎪
⎪ j=0 ⎪
di = ⎨ n−1 ⎬. (11.109)
⎪ ∑ c h if n ≤ i ≤ 2n − 2 ⎪
⎪ j i−j ⎪
⎩ j=i−(n−1) ⎭

We consider reducing d(x) modulo xn − 1, and denote the result as d(x) ̂ = d̂ 0 + d̂ 1 x + · · · + d̂ n−1 xn−1 ;
̂
then di = di + dn+i , i = 0, 1, 2, … , n − 1. If c(x) h(x) = 0 mod(x − 1), then d̂ i = 0, i = 0, 1, 2, … , n − 1,
n

therefore, we get
∑ i
∑
n−1
cj hi−j + cj hn+i−j = 0, i = 0, 1, 2, … , n − 1 (11.110)
j=0 j=i+1

For i = n − 1, (11.110) becomes

∑
n−1
cj hn−1−j = 0 (11.111)
j=0

or [hn−1 hn−2 … h1 h0 ] [c0 c1 … cn−1 ]T = 0.

For i = n − 2, (11.110) becomes

∑
n−2
cj hn−2−j + cn−1 hn−1 = 0 (11.112)
j=0

or [hn−2 hn−3 … h0 hn−1 ] [c0 c1 … cn−1 ]T = 0.

After r steps, for i = n − r, (11.110) becomes

∑
n−r
∑
n−1
cj hn−r−j + cj h2n−r−j = 0 (11.113)
j=0 j=n−r+1

or [hn−r hn−r−1 · · · hn−r+2 hn−r+1 ] [c0 c1 · · · cn−1 ]T = 0. The r equations can be written in matrix form
as
⎡ hn−1 hn−2 … h1 h 0 ⎤ ⎡ c0 ⎤ ⎡ 0 ⎤
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ hn−2 hn−3 … h0 hn−1 ⎥ ⎢ c1 ⎥ ⎢ 0 ⎥
= (11.114)
⎢ ⋮ ⋮ ⎥⎢ ⋮ ⎥ ⎢⋮⎥
⎢h hn−r+1 ⎥⎦ ⎢⎣ cn−1 ⎥⎦ ⎢⎣ 0 ⎥⎦
⎣ n−r hn−r−1 … hn−r+2

Therefore, all code words are solutions of the equation Hc = 0, where H is given by (11.104). It still
remains to be shown that all solutions of the equation Hc = 0 are code words. As hn−1 = hn−2 = · · · =
hn−r+1 = 0, and h0 ≠ 0, from (11.104) H has rank r, and can be written as H = [A B], where B is an
r × r matrix with non-zero determinant; therefore,
⎡ ck ⎤ ⎡ c0 ⎤
⎢ ⎥ ⎢ ⎥
⎢ k+1 ⎥ = −B−1 A ⎢ c1 ⎥
c
(11.115)
⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎢c ⎥ ⎢c ⎥
⎣ n−1 ⎦ ⎣ k−1 ⎦
so there are qk = qn−r solutions of the equation Hc = 0. As there are qk code words, all solutions of the
equation Hc = 0 are the code words in the cyclic code. ◽
11.2. Block codes 549

Example 11.2.24
Let q = 2 and n = 7. As x7 − 1 = x7 + 1 = (x3 + x + 1)(x3 + x2 + 1)(x + 1), we can choose g(x) = x3 +
x + 1 and h(x) = (x3 + x2 + 1)(x + 1) = x4 + x2 + x + 1; thus, the matrices G and H of this code are
given by
⎡1 1 0 1 0 0 0⎤
⎢ ⎥
0 1 1 0 1 0 0⎥
G=⎢ (11.116)
⎢0 0 1 1 0 1 0⎥
⎢0 0 0 1 1 0 1⎥
⎣ ⎦
⎡0 0 1 0 1 1 1⎤
H =⎢0 1 0 1 1 1 0⎥ (11.117)
⎢ ⎥
⎣1 0 1 1 1 0 0⎦
Note that the columns of H are all possible non-zero vectors of length 3, so the code is a Hamming
single error correcting (7,4) code.

6. In a code word, any string of r consecutive symbols, even taken cyclically, can identify the check
positions.

Proof. From (11.115), it follows that the last r positions can be check positions. Now, if we cyclically
permute every code word of m positions, the resultant words are themselves code words; thus the r check
positions can be cyclically permuted anywhere in the code words. ◽

7. As the r check positions can be the first r positions, a simple encoding method in canonical form is
given by the following steps:

Step 1: represents the k information bits by the coefficients of the polynomial m(x) = m0 + m1 x +
· · · + mk−1 xk−1 .
Step 2: multiplies m(x) by xr to obtain xr m(x).
Step 3: divides xr m(x) by g(x) to obtain the remainder r(x) = r0 + r1 x + … + rr−1 xr−1 .
Step 4: forms the code word c(x) = (xr m(x) − r(x)); note that the coefficients of (−r(x)) are the parity
check bits.

Proof. To show that (xr m(x) − r(x)) is a code word, we must prove that it is a multiple of g(x): from
step 3, we obtain
xr m(x) = Q(x) g(x) + r(x) (11.118)

so that
(xr m(x) − r(x)) = Q(x) g(x) (11.119)

Example 11.2.25
Let g(x) = 1 + x + x3 , for q = 2 and n = 7. We report in Table 11.12 the message words (m0 , … , m3 )
and the corresponding code words (c0 , … , c6 ) obtained by the generator polynomial according to Def-
inition 11.12 on page 546 for a(x) = m(x); the same code in canonical form, obtained by (11.119), is
reported in Table 11.13.
550 Chapter 11. Channel codes

Table 11.12: (7,4) binary cyclic code, generated by g(x) = 1 + x + x3 .

message code polynomial code

(m0 m1 m2 m3 ) c(x) = m(x) g(x) mod x7 − 1 (c0 c1 c2 c3 c4 c5 c6 )

0000 0g(x) = 0 0000000

1000 1g(x) = 1 + x + x3 1101000
0100 xg(x) = x + x2 + x4 0110100
1100 (1 + x)g(x) = 1 + x2 + x3 + x4 1011100
2 2 3 5
0010 x g(x) = x + x + x 0011010
1010 (1 + x2 )g(x) = 1 + x + x2 + x5 1110010
2 3 4 5
0110 (x + x )g(x) = x + x + x + x 0101110
1110 (1 + x + x2 )g(x) = 1 + x4 + x5 1000110
3 3 4 6
0001 x g(x) = x + x + x 0001101
1001 (1 + x3 )g(x) = 1 + x + x4 + x6 1100101
3 2 3 6
0101 (x + x )g(x) = x + x + x + x 0111001
1101 (1 + x + x3 )g(x) = 1 + x2 + x6 1010001
2 3 2 4 5 6
0011 (x + x )g(x) = x + x + x + x 0010111
1011 (1 + x2 + x3 )g(x) = 1 + x + x2 + x3 + x4 + x5 + x6 1111111
2 3 5 6
0111 (x + x + x )g(x) = x + x + x 0100011
1111 (1 + x + x2 + x3 )g(x) = 1 + x3 + x5 + x6 1001011

Table 11.13: (7,4) binary cyclic code in canonical form, generated by g(x) = 1 + x + x3 .

message code polynomial code

(m0 m1 m2 m3 ) r(x) = xr m(x) mod g(x) (c0 c1 c2 c3 c4 c5 c6 )
c(x) = xr m(x) − r(x)

0000 0 0000000
3
1000 1+x+x 1101000
0100 x + x2 + x4 0110100
2 3 4
1100 1+x +x +x 1011100
0010 1 + x + x2 + x5 1110010
2 3 5
1010 x +x +x 0011010
0110 1 + x4 + x5 1000110
3 4 5
1110 x+x +x +x 0101110
0001 1 + x2 + x6 1010001
2 3 6
1001 x+x +x +x 0111001
0101 1 + x + x4 + x6 1100101
3 4 6
1101 x +x +x 0001101
0011 x + x5 + x6 0100011
3 5 6
1011 1+x +x +x 1001011
0111 x2 + x4 + x5 + x6 0010111
2 3 4 5 6
1111 1+x+x +x +x +x +x 1111111
11.2. Block codes 551

Encoding by a shift register of length r

We show that the steps of the encoding procedure can be accomplished by a linear shift register with
r stages. We begin by showing how to divide m0 xr by the generator polynomial g(x) and obtain the
remainder. As
g(x) = gr xr + gr−1 xr−1 + · · · + g1 x + g0 (11.120)
then
xr = −g−1
r (gr−1 x
r−1
+ gr−2 xr−2 + · · · + g1 x + g0 ) mod g(x) (11.121)
and
m0 xr = −m0 g−1
r (gr−1 x
r−1
+ gr−2 xr−2 + · · · + g1 x + g0 ) mod g(x) (11.122)
is the remainder after dividing m0 xr by g(x).
We now consider the scheme illustrated in Figure 11.3, where multiplications and additions are in
GF(q), and Tc denotes the clock period with which the message symbols {mi }, i = k − 1, … , 1, 0, are
input to the shift register. In the binary case, the storage elements are flip flops, the addition is the
modulo 2 addition, and multiplication by gi is performed by a switch that is open or closed depending
upon whether gi = 0 or 1, respectively. Note that if m0 is input, the storage elements of the shift register
will contain the coefficients of the remainder upon dividing m0 xr by g(x).
Let us suppose we want to compute the remainder upon dividing m1 xr+1 by g(x). We could first
compute the remainder of the division of m1 xr by g(x), by presenting m1 at the input, then multiplying
the remainder by x, and again reduce the result modulo g(x). But once the remainder of the first division
is stored in the shift register, multiplication by x and division by g(x) are obtained simply by clocking
the register once with no input. In fact, if the shift register contains the polynomial
b(x) = b0 + b1 x + · · · + br−1 xr−1 (11.123)
and we multiply by x and divide by g(x), we obtain
x b(x) = b0 x + b1 x2 + · · · + br−1 xr
= b0 x + b1 x2 + · · · + br−2 xr−1
+ br−1 (−g−1r (gr−1 x
r−1 + · · · + g x + g ))mod g(x)
1 0 (11.124)
= − br−1 gr g0 + (b0 − br−1 g−1
−1
r g1 ) x + · · ·
+ (br−2 − br−1 g−1 r gr−1 ) x
r−1 mod g(x)

that is just the result obtained by clocking the register once.

Finally, we note that superposition holds in computing remainders; in other words, if m0 xr =
r1 (x) mod g(x) and m1 xr+1 = r2 (x) mod g(x), then m0 xr + m1 xr+1 = r1 (x) + r2 (x) mod g(x). There-
fore, to compute the remainder upon dividing m0 xr + m1 xr+1 by g(x) using the scheme of Figure 11.3,
we would first input m1 and then next input m0 to the shift register.

g0 g1 gr – 1 – gr–1

Tc Tc Tc
IN

mi
OUT

Figure 11.3 Scheme of an encoder for cyclic codes using a shift register with r elements.
552 Chapter 11. Channel codes

Hence, to compute the remainder upon dividing xr m(x) = m0 xr + m1 xr+1 + · · · + mk−1 xn−1 by g(x)
we input the symbols mk−1 , mk−2 , … , m1 , m0 to the device of Figure 11.3; after the last symbol, m0 ,
enters, the coefficients of the desired remainder will be contained in the storage elements. From (11.119),
we note that the parity check bits are the inverse elements (with respect to addition) of the values con-
tained in the register.
In general for an input z(x), polynomial with n coefficients, after n clock pulses the device of
Figure 11.3 yields xr z(x) mod g(x).

Encoding by a shift register of length k

It is also possible to accomplish the encoding procedure for cyclic codes by using a shift register with k
stages. Again, we consider the first r positions of the code word as the parity check bits, p0 , p1 , … , pr−1 ;
utilizing the first row of the parity check matrix, we obtain

hk pr−1 + hk−1 m0 + · · · + h1 mk−2 + h0 mk−1 = 0 (11.125)

or
pr−1 = −h−1
k (hk−1 m0 + hk−2 m1 + · · · + h1 mk−2 + h0 mk−1 ) (11.126)

Similarly, using the second row, we obtain

pr−2 = −h−1
k (hk−1 pr−1 + hk−2 m0 + · · · + h1 mk−3 + h0 mk−2 ) (11.127)

and so forth.

– hk–1 hk – 1 hk – 2 hk – 3 h1 h0

Tc Tc Tc Tc
OUT

IN

Figure 11.4 Scheme of an encoder for cyclic codes using a shift register with k elements.

Let us consider the scheme of Figure 11.4 and assume that the register initially contains the symbols
m0 , m1 , … , mk−1 . After one clock pulse mk−1 will appear at the output, all information symbols will
have moved by one place to the right and the parity check symbol pr−1 will appear in the first left-most
storage element; after the second clock pulse, mk−2 will appear at the output, all symbols contained in
the storage elements will move one place to the right and the parity check symbol pr−2 will appear in the
left-most storage element. It is easy to verify that, if we apply n clock pulses to the device, the output
will be given by the k message symbols followed by the r parity check bits.

Hard decoding of cyclic codes

We recall (see page 526) that all vectors in the same coset of the decoding table have the same syndrome
and that vectors in different cosets have different syndromes.
11.2. Block codes 553

Proposition 11.3
All polynomials corresponding to vectors in the same coset have the same remainder if they are divided
by g(x); polynomials corresponding to vectors in different cosets have different remainders if they are
divided by g(x). ◽

Proof. Let aj (x) g(x), j = 0, 1, 2, … , qk − 1, be the code words, and 𝜂i (x), i = 0, 1, 2, … , qr − 1, be the
coset leaders. Assume z1 (x) and z2 (x) are two arbitrary polynomials of degree n − 1: if they are in the
same coset, say the ith, then
z1 (x) = 𝜂i (x) + aj1 (x) g(x) (11.128)

and
z2 (x) = 𝜂i (x) + aj2 (x) g(x) (11.129)

As upon dividing aj1 g(x) and aj2 g(x) by g(x), we get 0 as a remainder, the division of z1 (x) and z2 (x) by
g(x) gives the same remainder, namely the polynomial ri (x), where

𝜂i (x) = Q(x) g(x) + ri (x), deg(ri (x)) < deg(g(x)) = r (11.130)

Now, assume z1 (x) and z2 (x) are in different cosets, say the i1 th and i2 th cosets, but have the same
remainder, say r0 (x), if they are divided by g(x); then the coset leaders 𝜂i1 (x) and 𝜂i2 (x) of these cosets
must give the same remainder r0 (x) if they are divided by g(x), i.e.
𝜂i1 (x) = Q1 (x) g(x) + r0 (x) (11.131)

and
𝜂i2 (x) = Q2 (x) g(x) + r0 (x) (11.132)

Therefore, we get

𝜂i2 (x) = 𝜂i1 (x) + (Q2 (x) − Q1 (x))g(x) = 𝜂i1 (x) + Q3 (x) g(x) (11.133)

This implies that 𝜂i1 (x) and 𝜂i2 (x) are in the same coset, which is a contradiction. ◽

This result leads to the following decoding method for cyclic codes.

Step 1: compute the remainder upon dividing the received polynomial z(x) of degree n − 1 by g(x), for
example by the device of Figure 11.5 (see (11.124)), by presenting at the input the sequence
of received symbols, and applying n clock pulses. The remainder identifies the coset leader
of the coset where the received polynomial is located.
Step 2: subtract the coset leader from the received polynomial to obtain the decoded code word.

g0 g1 gr – 1 – gr–1

IN Tc Tc Tc

z0 z1 ... zn– 2 zn– 1

Figure 11.5 Device to compute the division of the polynomial z(x) = z0 + z1 x + · · · + zn−1 xn−1 by
g(x). After n clock pulses the r storage elements contain the remainder r0 , r1 , … , rr−1 .
554 Chapter 11. Channel codes

Hamming codes
Hamming codes are binary cyclic single error correcting codes.
We consider cyclic codes over GF(2), where g(x) is an irreducible polynomial of degree r such that
g(x) divides x2 −1 − 1, but not x𝓁 − 1 for 𝓁 < 2r − 1.
r

To show that g(x) is a primitive irreducible polynomial, we choose n = 2r − 1, thus xn − 1 = g(x) h(x)
and the corresponding cyclic code has parameters n = 2r − 1, r, and k = 2r − 1 − r.

Proposition 11.4
H
This code has minimum distance dmin = 3 and therefore, is a single error correcting code. ◽

H
Proof. We first prove that dmin ≥ 3 by showing that all single error polynomials have distinct, non-zero
remainders if they are divided by g(x).
Assume that xi = 0 mod g(x), for some 0 ≤ i ≤ n − 1; then xi = Q(x) g(x), which is impossible since
g(x) is not divisible by x.
Now, assume that xi and xj give the same remainder upon division by g(x) and that 0 ≤ i < j ≤ n − 1;
then
xj − xi = xi (xj−i − 1) = Q(x) g(x) (11.134)
but g(x) does not divide xi , so it must divide (xj−i − 1). But 0 < j − i ≤ n − 1 and by assumption g(x)
does not divide this polynomial. Hence, dminH
≥ 3.
By the bound (11.15), we know that for a code with fixed n and k the following inequality holds:
⎡ ( ) ( ) ⎛ n ⎞⎤
n n
2k ⎢1 + + + · · · + ⎜ ⌊ dH −1 ⌋ ⎟⎥ ≤ 2n (11.135)
⎢ 1 2 ⎜ min ⎟⎥
⎣ ⎝ 2 ⎠⎦
As n = 2r − 1 and k = n − r, we have
⎡ ( r ) ( r ) ⎛ 2r − 1 ⎞⎤
⎢1 + 2 − 1 + 2 − 1 + · · · + ⎜ ⌊ H ⌋ ⎟⎥ ≤ 2r (11.136)
⎢ 1 2 ⎜ dmin −1 ⎟⎥
⎣ ⎝ 2 ⎠⎦
but ( )
2r − 1
1+ = 2r (11.137)
1
H
and therefore, dmin ≤ 3. ◽

We have seen in the previous section how to implement an encoder for a cyclic code. We consider
now the decoder device of Figure 11.6, whose operations are described as follows:
1. Initially, all storage elements of the register contain zeros and the switch SW is in position 0. The
received n-bit word z = (z0 , … , zn−1 ) is sequentially clocked into the lower register, with n storage
elements, and into the feedback register, with r storage elements, whose content is denoted by
r0 , r1 , … , rr−1 .
2. After n clock pulses, the behaviour of the decoder depends on the value of 𝑣: if 𝑣 = 0, the switch
SW remains in the position 0 and both registers are clocked once. This procedure is repeated until
𝑣 = 1, which occurs for r0 = r1 = · · · = rr−2 = 0; then SW moves to position 1 and the content of
the last stage of the feedback shift register is added modulo 2 to the content of the last stage of the
lower register; both registers are then clocked until the n bits of the entire word are obtained at the
output of the decoder. Overall, 2n clock pulses are needed.
11.2. Block codes 555

0 (v = 0)
SW
1 (v = 1)
g1 g2 gr – 1
x0 x1 x r– 2 x r– 1
Tc Tc Tc Tc

NOR

v
z
IN
x0 x1 x n –1
Tc Tc Tc c^
OUT

Figure 11.6 Scheme of a decoder for binary cyclic single error correcting codes (Hamming codes).
All operations are in GF(2).

We now illustrate the procedure of the scheme of Figure 11.6. First of all we note that for the first n
clocks the device coincides with that of Figure 11.3, hence, the content of the shift register is given by
r(x) = xr z(x) mod g(x) (11.138)
We consider two cases.
1. The received word is correct, z(x) = c(x). After the first n clock pulses, from (11.138), we have
r(x) = xr c(x) = xr a(x) g(x) = 0 mod g(x) (11.139)
and thus
𝑣=1 and rr−1 = 0 (11.140)
In the successive n clock pulses, we have
ĉ i = zi + 0, i = 0, … , n − 1 (11.141)
therefore, ĉ = c.
2. The received word is affected by one error, z(x) = c(x) + xi . In other words, we assume that there
is a single error in the ith bit, 0 ≤ i ≤ n − 1.
After the first n clock pulses, it is
r(x) = xr xi mod g(x) (11.142)
If i = n − 1, we have
r(x) = xr xn−1
= xn xr−1
= (xn − 1) xr−1 + xr−1 (11.143)
= h(x) g(x) xr−1 + xr−1 mod g(x)
= xr−1
and consequently,
rr−1 = 1 and rr−2 = · · · = r0 = 0 (𝑣 = 1) (11.144)
556 Chapter 11. Channel codes

This leads to switching SW, therefore, during the last n clock pulses, we have
ĉ n−1 = zn−1 +1 ĉ i = zi + 0, i = n − 2, … , 0 (11.145)
Therefore, the bit in the last stage of the buffer is corrected.
If i = n − j, then we have
r(x) = xr−j (11.146)
thus only at the (n + j − 1)th clock pulse the condition (11.144) that forces to switch SW from 0 to
1 occurs; therefore, at the next clock pulse, the received bit in error will be corrected.

Burst error detection

We assume that a burst error occurs in the received word and that this burst affects 𝓁 ≤ n − k consecutive
bits, that is the error pattern is
bit j bit (j+𝓁−1)
e = (0, 0, 0, … , 0, 1 , … , … , 1 , 0, … , 0) (11.147)
where within the two ‘1’s the values can be either ‘0’ or ‘1’.
Then we can write the vector e in polynomial form,
e(x) = xj B(x) (11.148)
where B(x) is a polynomial of degree 𝓁 − 1 ≤ n − k − 1. Thus, e(x) is divisible by the generator poly-
nomial g(x) if B(x) is divisible by g(x), as x is not a factor of g(x), but B(x) has a degree at most equal
to (n − k − 1), lower than the degree of g(x), equal to n − k; therefore, e(x) cannot be a code word. We
have then that all burst errors of length 𝓁 less than or equal to r = n − k are detectable by (n, k) cyclic
codes. This result leads to the introduction of the cyclic redundancy check (CRC) codes.

11.2.4 Simplex cyclic codes

We consider a class of cyclic codes over GF(q) such that the Hamming distance between every pair of
distinct code words is a constant; this is equivalent to stating that the weight of all non-zero code words
is equal to the same constant. We show that in the binary case, for these codes the non-zero code words
are related to the PN sequences of Appendix 1.C.
Let n = qk − 1, and xn − 1 = g(x) h(x), where we choose h(x) as a primitive polynomial of degree k;
then the resultant code has minimum distance
H
dmin = (q − 1) qk−1 (11.149)
The parameters of some binary codes in this class are listed in Table 11.14.
To show that these codes have minimum distance given by (11.149), first we prove the following:

Table 11.14: Parameters of some simplex binary codes.

n k r dmin

7 3 4 4
15 4 11 8
31 5 26 16
63 6 57 32
127 7 120 64
11.2. Block codes 557

Property
All non-zero code words have the same weight.

Proof. We begin by showing that

xi g(x) ≠ xj g(x), mod(xn − 1), 0≤i<j≤n−1 (11.150)

Assume the converse is true, that is xi g(x) = xj g(x) mod(xn − 1); then
xi (xj−i − 1) g(x) = Q(x) g(x) h(x) (11.151)

or
xi (xj−i − 1) = Q(x) h(x) (11.152)

But this is impossible since h(x) is a primitive polynomial of degree k and cannot divide (xj−i − 1), as
(j − i) < n = (qk − 1).
Relation (11.150) implies that all cyclic shifts of the code polynomial g(x) are unique, but there are
n = (qk − 1) cyclic shifts. Furthermore, we know that there are only qk code words and one is the all-zero
word; therefore, all cyclic shifts of g(x) are all the non-zero code words, and they all have the same
weight. ◽

Recall Property 2 of a group code (see page 519), that is if all code words of a linear code are written as
rows of a matrix, every column is either formed by all zeros, or it consists of each field element repeated
an equal number of times. If we apply this result to a simplex code, we find that no column can be all
zero as the code is cyclic, so the sum of the weights of all code words is given by
qk
sum of weights = n(q − 1) = (qk − 1) (q − 1) qk−1 (11.153)
q
But there are (qk − 1) non-zero code words, all of the same weight; the weight of each word is then given
by
weight of non-zero code words = (q − 1) qk−1 (11.154)

Therefore, the minimum weight of the non-zero code words is given by

H
dmin = (q − 1) qk−1 (11.155)

Example 11.2.26
H
Let q = 2, n = 15, and k = 4; hence, r = 11, and dmin = 8. Choose h(x) as a primitive irreducible poly-
nomial of degree 4 over GF(2), h(x) = x4 + x + 1.
The generator polynomial g(x) is obtained by dividing x15 − 1 by h(x) = x4 + x + 1 in GF(2), obtaining
g(x) = x11 + x8 + x7 + x5 + x3 + x2 + x + 1 (11.156)

Given an extension field GF(2k ) and n = 2k − 1, from Property 2 on page 544, xn − 1 is given by the
l.c.m. of the minimum functions of the elements of the extension field. As h(x) is a primitive polyno-
mial, g(x) is, therefore, given by the l.c.m. of the minimum functions of the elements 1, 𝛼 3 , 𝛼 5 , 𝛼 7 , from
GF(24 ). By a table similar to Table 11.11, obtained for GF(26 ), and using one of the three methods to
determine the minimum function (see page 542), it turns out that the generator polynomial for this code is
given by
g(x) = (x + 1)(x4 + x3 + x2 + x + 1)(x2 + x + 1)(x4 + x3 + 1) (11.157)
558 Chapter 11. Channel codes

Relation to PN sequences
We consider a periodic binary sequence of period L, given by … , p(−1), p(0), p(1), …, with p(𝓁) ∈
{0, 1}. We define the normalized autocorrelation function of this sequence as
[ ]
1 ∑
L−1
𝚛p (m) = L−2 (p(𝓁) ⊕ p(𝓁 − m)) (11.158)
L 𝓁=0

Note that with respect to (1.588), now p(𝓁) ∈ {0, 1} rather than p(𝓁) ∈ {−1, 1}.

Theorem 11.1
If the periodic binary sequence {p(𝓁)} is formed by repeating any non-zero code word of a simplex
binary code of length L = n = 2k − 1, then
{
1 m = 0, ±L, ±2L …
𝚛p (m) = 1 (11.159)
− otherwise
L ◽

Proof. We recall that for a simplex binary code all non-zero code words

(a) have weight 2k−1 ,

(b) are cyclic permutations of the same code word.

As the code is linear, the Hamming distance between any code word and a cyclic permutation of this
word is 2k−1 ; this means that for the periodic sequence formed by repeating any non-zero code word, we
get
{
∑
L−1
0 m = 0, ±L, ±2L, …
(p(𝓁) ⊕ p(𝓁 − m)) = k−1 (11.160)
𝓁=0
2 otherwise

Substitution of (11.160) in (11.158) yields

⎧
⎪1 m = 0, ±L, ±2L, …
𝚛p (m) = ⎨ 2k − 1 − 2k 1 (11.161)
⎪ 2k − 1 = − 2k − 1 otherwise
⎩ ◽

If we recall the implementation of Figure 11.4, we find that the generation of such sequences is easy.
We just need to determine the shift register associated with h(x), load it with anything except all zeros,
and let it run. For example choosing h(x) = x4 + x + 1, we get the PN sequence of Figure 1.72, as illus-
trated in Figure 11.7, where L = n = 24 − 1 = 15.

11.2.5 BCH codes

An alternative method to specify the code polynomials

Definition 11.13
Suppose we arbitrarily choose L elements from GF(qm ) that we denote as 𝛼1 , 𝛼2 , … , 𝛼L (we will discuss
later how to select these elements), and we consider polynomials, of degree n − 1 or less, with coefficients
from GF(q). A polynomial is a code polynomial if each of the elements 𝛼1 , 𝛼2 , … , 𝛼L is a root of the
polynomial. The code then consists of the set of all the code polynomials. ◽
11.2. Block codes 559

p(l ) p(l – 1) p(l – 2) p(l – 3) p(l – 4)

l p(l – 1) p(l – 2) p(l – 3) p(l – 4)

0 0 0 0 1
1 1 0 0 0
2 0 1 0 0
3 0 0 1 0
4 1 0 0 1
5 1 1 0 0
6 0 1 1 0
7 1 0 1 1
8 0 1 0 1
9 1 0 1 0
10 1 1 0 1
11 1 1 1 0
12 1 1 1 1
13 0 1 1 1
14 0 0 1 1
15 0 0 0 1

Figure 11.7 Generation of a PN sequence as a repetition of a code word of a simplex code with
L = n = 15.

Using this method we see that c(x) = c0 + cx + · · · + cn−1 xn−1 is a code polynomial if and only if
c(𝛼1 ) = c(𝛼2 ) = · · · = c(𝛼L ) = 0; thus

⎡ (𝛼1 )0 (𝛼1 )1 (𝛼1 )2 … (𝛼1 )n−1 ⎤ ⎡ 0 ⎤ ⎡ 0 ⎤

c
⎢ ⎥ ⎢ c ⎥ ⎢ ⎥
⎢ (𝛼2 ) (𝛼2 ) (𝛼2 ) … (𝛼2 ) ⎥ ⎢⎢ c ⎥⎥ = ⎢ 0 ⎥
0 1 2 n−1 1
(11.162)
⎢ ⋮ ⋮ ⎥ 2
⎢⋮⎥
⎢ (𝛼 )0 (𝛼 )1 (𝛼 )2 … (𝛼 )n−1 ⎥ ⎢⎢ ⋮ ⎥⎥ ⎢ 0 ⎥
⎣ L L L L ⎦⎣c ⎦ ⎣ ⎦
n−1

All vectors c = [c0 , c1 , … , cn−1 ]T with elements from GF(q) that are solutions of this set of equations,
where operations are performed according to the rules of GF(qm ), are code words. The form of (11.162)
resembles equation (11.20), where H is the generalized parity check matrix. One obvious difference is
that in (11.20) H and c have elements from the same field, whereas this does not occur for the vector
equation (11.162). However, this difference is not crucial as each element from GF(qm ) can be written
as a vector of length m with elements from GF(q). Thus, each element (𝛼i )j in the matrix is replaced
by a column vector with m components. The resultant matrix, with Lm rows and n columns, consists
of elements from GF(q) and is, therefore, just a generalized parity check matrix for the considered code.
From the above discussion it appears that, if L roots are specified, the resultant linear code has r = Lm
parity check symbols, as the parity check matrix has r = Lm rows. However, not all rows of the matrix
are necessarily independent; therefore, the actual number of parity check symbols may be less than Lm.
We now show that if n is properly chosen, the resultant codes are cyclic codes. Let mj (x) be the min-
imum function of 𝛼j , j = 1, 2, … , L, where 𝛼j ∈ GF(qm ) and mj (x) has coefficients from GF(q). For
Property 3 on page 541, every code polynomial c(x) must be divisible by m1 (x), m2 (x), … , mL (x), and is
thus divisible by the least common multiple of such minimum functions, l.c.m.(m1 (x), m2 (x), … , mL (x)).
560 Chapter 11. Channel codes

If we define
g(x) = l.c.m.(m1 (x), m2 (x), … , mL (x)) (11.163)
then all multiples of g(x) are code words. In particular from Definition 11.2.12, the code is cyclic if
xn − 1 = g(x) h(x) (11.164)
Let 𝓁i be the order of 𝛼i , i = 1, 2, … , L, and furthermore, let
n = l.c.m.(𝓁1 , 𝓁2 , … , 𝓁L ) (11.165)
From the properties of the minimum function (see page 544, Property 2), we know that g(x) divides
xn − 1; thus, the code is cyclic if n is chosen as indicated by (11.165). We note that
r = deg(g(x)) ≤ mL (11.166)
as deg(mi (x)) ≤ m. We see that r is equal to mL if all minimum functions are distinct and are of degree
m; conversely, r < mL if any minimum function has degree less than m or if two or more minimum
functions are identical.

Example 11.2.27
Choose q = 2 and let 𝛼 be a primitive element of GF(24 ); furthermore, let the code polynomials have
as roots the elements 𝛼, 𝛼 2 , 𝛼 3 , 𝛼 4 . To derive the minimum functions of the chosen elements, we look
up for example the Appendix C of [3], where such functions are listed. Minimum functions and orders
of elements chosen for this example are given in Table 11.15.
Then
g(x) = (x4 + x + 1) (x4 + x3 + x2 + x + 1)
(11.167)
n = l.c.m.(15, 15, 5, 15) = 15
H
The resultant code is, therefore, a (15,7) code; later, we will show that dmin = 5.

Table 11.15: Minimum functions and orders of elements

𝛼 , 𝛼 2 , 𝛼 3 , 𝛼 4 , in GF(24 ).

roots minimum function order

𝛼 x4 + x + 1 15
𝛼2 x4 + x + 1 15
𝛼3 x4 + x3 + x2 + x + 1 5
𝛼 4 4
x +x+1 15

Bose-Chaudhuri–Hocquenhem codes
The Bose–Chaudhuri–Hocquenhem (BCH) codes are error correcting codes with symbols from GF(q)
and roots of code polynomials from GF(qm ).
The basic mathematical fact required to prove the error correcting capability of BCH codes is that if
𝛼1 , 𝛼2 , … , 𝛼r are elements from any field, the determinant of the Vandermonde matrix, given by
| 1 1 … 1 ||
|
| |
| 𝛼1 𝛼2 … 𝛼r |
| |
| |
det | 𝛼12 𝛼22 … 𝛼r2 | (11.168)
| |
| ⋮ ⋮ |
| |
| r−1 r−1 |
|𝛼 𝛼 … 𝛼 r−1 |
| 1 2 r |
is non-zero if and only if 𝛼i ≠ 𝛼j , for all indices i ≠ j. In particular, we prove the following result:
11.2. Block codes 561

Lemma. The determinant (11.168) is given by

| 1 1 … 1 |
| |
| 𝛼 |
| 1 𝛼2 … 𝛼r |
| | ∏ r
| 2 … 𝛼r || = (−1) 2
D = det | 𝛼1 𝛼2
2 2 r(r+1)
(𝛼i − 𝛼j ) (11.169)
| |
| ⋮ ⋮ || i, j = 1
|
| r−1 r−1 | i<j
| 𝛼1 𝛼2 … 𝛼rr−1 ||
|

Proof. Consider the polynomial P(x) defined as

| 1 … 1 ||
| 1
| |
| x 𝛼2 … 𝛼r |
| |
P(x) = det || x2 𝛼22 … 𝛼r2 || (11.170)
| ⋮ ⋮ ||
|
| r−1 r−1 |
|x 𝛼2 … 𝛼rr−1 ||
|
so that D = P(𝛼1 ). Now, P(x) is a polynomial of degree at most r − 1 whose zeros are x = 𝛼2 , x =
𝛼3 , … , x = 𝛼r , because if x = 𝛼i , i = 2, 3, … , r, the determinant D is equal to zero as two columns of the
matrix are identical. Thus,
P(x) = k1 (x − 𝛼2 )(x − 𝛼3 ) · · · (x − 𝛼r ) (11.171)
and
D = P(𝛼1 ) = k1 (𝛼1 − 𝛼2 )(𝛼1 − 𝛼3 ) · · · (𝛼1 − 𝛼r ) (11.172)
It remains to calculate k1 . The constant k1 is the coefficient of xr−1 ; therefore, from (11.170) we get
| 1 1 … 1 ||
|
| 𝛼 𝛼 |
| 3 … 𝛼r |
(−1)r k1 = det | 2 | = k2 (𝛼2 − 𝛼3 )(𝛼2 − 𝛼4 ) … (𝛼2 − 𝛼r ) (11.173)
| ⋮ ⋮ ||
|
| 𝛼 r−2 𝛼 r−2 … 𝛼 r−1 |
| 2 3 r |
using a result similar to (11.172).
Proceeding we find
(−1)r−1 k2 = k3 (𝛼3 − 𝛼4 )(𝛼3 − 𝛼5 ) · · · (𝛼3 − 𝛼r )
(−1)r−2 k3 = k4 (𝛼4 − 𝛼5 )(𝛼4 − 𝛼6 ) · · · (𝛼4 − 𝛼r )
(11.174)
⋮
(−1)2 kr−1 = (−1)(𝛼r−1 − 𝛼r )
and therefore,
∏
r
r(r+1) ∏
r
D = (−1)r+(r−1)+···+2+1 (𝛼i − 𝛼j ) = (−1) 2 (𝛼i − 𝛼j ) (11.175)
i, j = 1 i, j = 1
i<j i<j ◽

We now prove the important Bose–Chaudhuri–Hocquenhem theorem.

Theorem 11.2
Consider a code with symbols from GF(q), whose code polynomials have as zeros the elements
𝛼 m0 , 𝛼 m0 +1 , … , 𝛼 m0 +d−2 , where 𝛼 is any element from GF(qm ) and m0 is any integer. Then the resultant
(n, k) cyclic code has the following properties:
H
a) it has minimum distance dmin ≥ d if the elements 𝛼 m0 , 𝛼 m0 +1 , … , 𝛼 m0 +d−2 , are distinct;
562 Chapter 11. Channel codes

⌈ ⌉
b) n − k ≤ (d − 1)m; if q = 2 and m0 = 1, then n − k ≤ d−1 2
m;
c) n is equal to the order of 𝛼, unless d = 2, in which case n is equal to the order of 𝛼 m0 ;
d) g(x) is equal to the least common multiple of the minimum functions of 𝛼 m0 , 𝛼 m0 +1 , …, 𝛼 m0 +d−2 .
◽

Proof. The proof of part (d) has already been given (see (11.163)); the proof of part (b) then follows by
noting that each minimum function is at most of degree m, and there are at most (d − 1) distinct minimum
functions. If q = 2 and m0 = 1, the minimum function of 𝛼 raised to an even power, for example 𝛼 2i ,
⌉ )as the minimum function of 𝛼 (see Property 2 on page 542), therefore, there are at most
is i
(⌈the same
d−1
2
m distinct minimum functions.
To prove part (c) note that, if d = 2, we have only the root 𝛼 m0 , so that n is equal to the order of 𝛼 m0 . If
there are more than one root, then n must be the least common multiple of the order of the roots. If 𝛼 m0
and 𝛼 m0 +1 are both roots, then (𝛼 m0 )n = 1 and (𝛼 m0 +1 )n = 1, so that 𝛼 n = 1; thus n is a multiple of the
order of 𝛼. On the other hand, if 𝓁 is the order of 𝛼, (𝛼 m0 +i )𝓁 = (𝛼 𝓁 )m0 +i = 1m0 +i = 1; therefore, 𝓁 is a
multiple of the order of every root. Then n is the least common multiple of numbers all of which divide
𝓁, and therefore, n ≤ 𝓁; thus n = 𝓁.
Finally, we prove part (a). We note that the code words must satisfy the condition

⎡ 1 𝛼 m0 ⎡ c ⎤
(𝛼 m0 )2 … (𝛼 m0 )n−1 ⎤ ⎢ 0 ⎥ ⎡ 0 ⎤
⎢ ⎥ c ⎢ ⎥
⎢1 𝛼
m0 +1 (𝛼 m0 +1 )2 … (𝛼 m0 +1 )n−1 ⎥ ⎢ 1 ⎥ ⎢ 0 ⎥
⎢⋮ ⋮ ⎥ ⎢ c2 ⎥ =
⎢⋮⎥
(11.176)
⎢ 1 𝛼 m0 +d−2 (𝛼 m0 +d−2 )2 … (𝛼 m0 +d−2 )n−1 ⎥ ⎢⎢ ⋮ ⎥⎥ ⎢ 0 ⎥
⎣ ⎦⎣c ⎦ ⎣ ⎦
n−1

We now show that no linear combination of (d − 1) or fewer columns is equal to 0. We do this by showing
that the determinant of any set of (d − 1) columns is non-zero. Choose columns j1 , j2 , … , jd−1 ; then
| (𝛼 m0 ) j1 (𝛼 m0 ) j2 … (𝛼 m0 ) jd−1 ||
|
| (𝛼 m0 +1 ) j1 (𝛼 m0 +1 ) j2 … (𝛼 m0 +1 ) jd−1 |
| |
det | | (11.177)
| ⋮ ⋮ |
| |
| (𝛼 m0 +d−2 ) j1 (𝛼 m0 +d−2 ) j2 … (𝛼 m0 +d−2 ) jd−1 |
| |
| 1 1 … 1 |
| |
| 𝛼 j1 𝛼 j2 𝛼 jd−1 |
| … |
=𝛼 m0 ( j1 +j2 +···+jd−1 )
det | | (11.178)
| ⋮ ⋮ |
| |
| (𝛼 j1 )d−2 (𝛼 j2 )d−2 … (𝛼 jd−1 )d−2 |
| |
(d−1)d ∏
d−1
= 𝛼 m0 (j1 +j2 +···+jd−1 ) (−1) 2 (𝛼 ji − 𝛼 jk ) ≠ 0 (11.179)
i, k = 1
i<k

Note that we have proven that (d − 1) columns of H are linearly independent even if they are multiplied
by elements from GF(qm ). All that would have been required was to show linear independence if the
multipliers are from GF(q). ◽

Binary BCH codes

In this section, we consider binary BCH codes. Choose m0 = 1; then from Property (c) of Theorem 11.2,
we get
{ m
2 −1 if 𝛼 is a primitive element of GF(2m )
n= 2m − 1 (11.180)
if 𝛼 = 𝛽 c , where 𝛽 is a primitive element of GF(2m )
c
11.2. Block codes 563

and r = n − k satisfies the relation (see Property (b))

⌈ ⌉
d−1
r≤m (11.181)
2
Moreover for Property (d)
g(x) = l.c.m.(minimum functions of 𝛼, 𝛼 3 , 𝛼 5 , … , 𝛼 d−2 ) with d odd number . (11.182)

Example 11.2.28
Consider binary BCH codes of length 63, that is q = 2 and m = 6. To get a code with design distance d,
we choose as roots 𝛼, 𝛼 2 , 𝛼 3 , … , 𝛼 d−1 , where 𝛼 is a primitive element from GF(26 ). Using Table 11.11
on page 544, we get the minimum functions of the elements from GF(26 ) given in Table 11.16.
Then the roots and generator polynomials for different values of d are given in Table 11.17; the param-
eters of the relative codes are given in Table 11.18.
Table 11.16: Minimum functions of the elements of GF(26 ).

roots minimum function

𝛼 1 𝛼 2 𝛼 4 𝛼 8 𝛼 16 𝛼 32 x6 + x + 1
𝛼 𝛼 𝛼
3 6 12
𝛼 24
𝛼 48
𝛼 33
x + x4 + x2 + x + 1
6

𝛼 5 𝛼 10 𝛼 20 𝛼 40 𝛼 17 𝛼 34 x6 + x5 + x2 + x + 1
𝛼 𝛼 7 14
𝛼 28
𝛼 56
𝛼 49
𝛼 35
x6 + x3 + 1
𝛼 9 𝛼 18 𝛼 36 x3 + x2 + 1
𝛼 11
𝛼 22
𝛼 44
𝛼 25
𝛼 50
𝛼 37
x + x5 + x3 + x2 + 1
6

𝛼 13 𝛼 26 𝛼 52 𝛼 41 𝛼 19 𝛼 38 x6 + x4 + x3 + x + 1
𝛼 15
𝛼 30
𝛼 60
𝛼 57
𝛼 51
𝛼 39
x6 + x5 + x4 + x2 + 1
𝛼 21 𝛼 42 x2 + x + 1
𝛼 23
𝛼 46
𝛼 29
𝛼 58
𝛼 53
𝛼 43
x + x5 + x4 + x + 1
6

𝛼 27
𝛼 54
𝛼 45
x3 + x + 1
𝛼 31 𝛼 62 𝛼 61 𝛼 59 𝛼 55 𝛼 47 x6 + x5 + 1

Example 11.2.29
Let q = 2, m = 6, and choose as roots 𝛼, 𝛼 2 , 𝛼 3 , 𝛼 4 , with 𝛼 = 𝛽 3 , where 𝛽 is a primitive element of
m
GF(26 ); then n = 2 c−1 = 63
3
H
= 21, dmin ≥ d = 5, and
g(x) = l.c.m(m𝛽 3 (x), m𝛽 6 (x), m𝛽 9 (x), m𝛽 12 (x))
= m𝛽 3 (x)m𝛽 9 (x) (11.183)
= (x6 + x4 + x2 + x + 1)(x3 + x2 + 1)
As r = deg(g(x)) = 9, then k = n − r = 12; thus, we obtain a (21,12) code.

Example 11.2.30
Let q = 2, m = 4, and choose as roots 𝛼, 𝛼 2 , 𝛼 3 , 𝛼 4 , with 𝛼 primitive element of GF(24 ); then a (15,7)
H
code is obtained having dmin ≥ 5, and g(x) = (x4 + x + 1)(x4 + x3 + x2 + x + 1).

Example 11.2.31
Let q = 2, m = 4, and choose as roots 𝛼, 𝛼 2 , 𝛼 3 , 𝛼 4 , 𝛼 5 , 𝛼 6 , with 𝛼 primitive element of GF(24 ); then a
H
(15,5) code is obtained having dmin ≥ 7, and g(x) = (x4 + x + 1)(x4 + x3 + x2 + x + 1)(x2 + x + 1).
564 Chapter 11. Channel codes

Table 11.17: Roots and generator polynomials of BCH codes of length

n = 63 = 26 − 1 for different values of d. 𝛼 is a primitive element of
GF(26 ) (see (11.180)).

d roots generator polynomial

3 𝛼 𝛼2 (x6 + x + 1) = g3 (x)
5 𝛼 𝛼2 𝛼3 𝛼4 (x6 + x + 1)(x6 + x4 + x2 + x + 1) = g5 (x)
7 𝛼𝛼 … 𝛼2 6
(x6 + x5 + x2 + x + 1) g5 (x) = g7 (x)
9 𝛼 𝛼2 … 𝛼8 (x6 + x3 + 1) g7 (x) = g9 (x)
11 𝛼𝛼 … 𝛼2 10
(x3 + x2 + 1) g9 (x) = g11 (x)
13 𝛼 𝛼 2 … 𝛼 12 (x6 + x5 + x3 + x2 + 1) g11 (x) = g13 (x)
15 𝛼𝛼 … 𝛼2 14
(x6 + x4 + x3 + x + 1) g13 (x) = g15 (x)
21 𝛼 𝛼 2 … 𝛼 20 (x6 + x5 + x4 + x2 + 1) g15 (x) = g21 (x)
23 𝛼𝛼 … 𝛼2 22
(x2 + x + 1) g21 (x) = g23 (x)
27 𝛼 𝛼 2 … 𝛼 26 (x6 + x5 + x4 + x + 1) g23 (x) = g27 (x)
31 𝛼𝛼 … 𝛼2 30
(x3 + x + 1) g27 (x) = g31 (x)

Table 11.18: Parameters of BCH codes of length n = 63.

k 57 51 45 39 36 30 24 18 16 10 7
d 3 5 7 9 11 13 15 21 23 27 31
t 1 2 3 4 5 6 7 10 11 13 15

Reed–Solomon codes
Reed–Solomon codes represent a particular case of BCH codes obtained by choosing m = 1; in other
words, the field GF(q) and the extension field GF(qm ) coincide. Choosing 𝛼 as a primitive element of
(11.180) we get
n = qm − 1 = q − 1 (11.184)
Note that the minimum function with coefficients in GF(q) of an element 𝛼 i from GF(q) is
m𝛼i (x) = (x − 𝛼 i ) (11.185)
For m0 = 1, if we choose the roots 𝛼, 𝛼 2 , 𝛼 3 , … , 𝛼 d−1 , then
g(x) = (x − 𝛼)(x − 𝛼 2 ) … (x − 𝛼 d−1 ) (11.186)
so that r = (d − 1); the block length n is given by the order of 𝛼. In this case, we show that dminH
= d;
in fact, for any code, we have dmin ≤ r + 1, as we can always choose a code word with every message
H

symbol but one equal to zero and therefore, its weight is at most equal to r + 1; from this it follows that
H
dmin ≤ d, but from the BCH theorem, we know that dmin H
≥ d.

Example 11.2.32
Choose 𝛼 as a primitive element of GF(25 ), and choose the roots 𝛼, 𝛼 2 , 𝛼 3 , 𝛼 4 , 𝛼 5 , and 𝛼 6 ; then the resul-
H
tant (31,25) code has dmin = 7, g(x) = (x − 𝛼)(x − 𝛼 2 )(x − 𝛼 3 )(x − 𝛼 4 )(x − 𝛼 5 )(x − 𝛼 6 ), and the symbols
of the code words are from GF(25 ).
11.2. Block codes 565

Table 11.19: Addition table for the elements of GF(24 ).

+ 0 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14
0 0 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14
𝛼 0
𝛼 0
0 𝛼 4
𝛼 8
𝛼 14
𝛼 1
𝛼 10
𝛼 13
𝛼 9
𝛼 2
𝛼 7
𝛼 5
𝛼 12
𝛼 11
𝛼 6
𝛼3
𝛼 1
𝛼 1
𝛼 4
0 𝛼 5
𝛼 9
𝛼 0
𝛼 2
𝛼 11
𝛼 14
𝛼 10
𝛼 3
𝛼 8
𝛼 6
𝛼 13
𝛼 12
𝛼7
𝛼 2
𝛼 2
𝛼 8
𝛼 5
0 𝛼 6
𝛼 10
𝛼 1
𝛼 3
𝛼 12
𝛼 0
𝛼 11
𝛼 4
𝛼 9
𝛼 7
𝛼 14
𝛼 13
𝛼 3
𝛼 3
𝛼 14
𝛼 9
𝛼 6
0 𝛼 7
𝛼 11
𝛼 2
𝛼 4
𝛼 13
𝛼 1
𝛼 12
𝛼 5
𝛼 10
𝛼 8
𝛼0
𝛼4 𝛼4 𝛼1 𝛼0 𝛼 10 𝛼7 0 𝛼8 𝛼 12 𝛼3 𝛼5 𝛼 14 𝛼2 𝛼 13 𝛼6 𝛼 11 𝛼9
𝛼5 𝛼5 𝛼 10 𝛼2 𝛼1 𝛼 11 𝛼8 0 𝛼9 𝛼 13 𝛼4 𝛼6 𝛼0 𝛼3 𝛼 14 𝛼7 𝛼 12
𝛼6 𝛼6 𝛼 13 𝛼 11 𝛼3 𝛼2 𝛼 12 𝛼9 0 𝛼 10 𝛼 14 𝛼5 𝛼7 𝛼1 𝛼4 𝛼0 𝛼8
𝛼 7
𝛼 7
𝛼 9
𝛼 14
𝛼 12
𝛼 4
𝛼 3
𝛼 13
𝛼 10
0 𝛼 11
𝛼 0
𝛼 6
𝛼 8
𝛼 2
𝛼 5
𝛼1
𝛼 8
𝛼 8
𝛼 2
𝛼 10
𝛼 0
𝛼 13
𝛼 5
𝛼 4
𝛼 14
𝛼 11
0 𝛼 12
𝛼 1
𝛼 7
𝛼 9
𝛼 3
𝛼6
𝛼 9
𝛼 9
𝛼 7
𝛼 3
𝛼 11
𝛼 1
𝛼 14
𝛼 6
𝛼 5
𝛼 0
𝛼 12
0 𝛼 13
𝛼 2
𝛼 8
𝛼 10
𝛼4
𝛼 10 𝛼 10 𝛼5 𝛼8 𝛼4 𝛼 12 𝛼2 𝛼0 𝛼7 𝛼6 𝛼1 𝛼 13 0 𝛼 14 𝛼3 𝛼9 𝛼 11
𝛼 11
𝛼 11
𝛼 12
𝛼 6
𝛼 9
𝛼 5
𝛼 13
𝛼 3
𝛼 1
𝛼 8
𝛼 7
𝛼 2
𝛼 14
0 𝛼 0
𝛼 4
𝛼 10
𝛼 12
𝛼 12
𝛼 11
𝛼 13
𝛼 7
𝛼 10
𝛼 6
𝛼 14
𝛼 4
𝛼 2
𝛼 9
𝛼 8
𝛼 3
𝛼 0
0 𝛼 1
𝛼5
𝛼 13
𝛼 13
𝛼 6
𝛼 12
𝛼 14
𝛼 8
𝛼 11
𝛼 7
𝛼 0
𝛼 5
𝛼 3
𝛼 10
𝛼 9
𝛼 4
𝛼 1
0 𝛼2
𝛼 14
𝛼 14
𝛼 3
𝛼 7
𝛼 13
𝛼 0
𝛼 9
𝛼 12
𝛼 8
𝛼 1
𝛼 6
𝛼 4
𝛼 11
𝛼 10
𝛼 5
𝛼 2
0

Observation 11.2
The encoding of Reed–Solomon codes can be done by the devices of Figure 11.3 or Figure 11.4, where
the operations are in GF(q). In Tables 11.19 and 11.20 we give, respectively, the tables of additions and
multiplications between elements of GF(q) for q = 24 ; the conversion of the symbol representation from
binary to exponential is implemented using Table 11.21. We note that the encoding operations can be

Table 11.20: Multiplication table for the elements of GF(24 ).

⋅ 0 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
𝛼0 0 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14
𝛼1 0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14 𝛼0
𝛼 2
0 𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼1
𝛼3 0 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14 𝛼0 𝛼1 𝛼2
𝛼4 0 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14 𝛼0 𝛼1 𝛼2 𝛼3
𝛼 5
0 𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼4
𝛼 6
0 𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼5
𝛼 7
0 𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼6
𝛼 8
0 𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼7
𝛼9 0 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13 𝛼 14 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8
𝛼 10
0 𝛼 10
𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼9
𝛼 11
0 𝛼 11
𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 12
0 𝛼 12
𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 13
0 𝛼 13
𝛼 14
𝛼 0
𝛼 1
𝛼 2
𝛼 3
𝛼 4
𝛼 5
𝛼 6
𝛼 7
𝛼 8
𝛼 9
𝛼 10
𝛼 11
𝛼 12
𝛼 14 0 𝛼 14 𝛼0 𝛼1 𝛼2 𝛼3 𝛼4 𝛼5 𝛼6 𝛼7 𝛼8 𝛼9 𝛼 10 𝛼 11 𝛼 12 𝛼 13
566 Chapter 11. Channel codes

Table 11.21: Three equivalent representations of the elements

of GF(24 ), obtained applying the polynomial arithmetic modulo
x4 + x + 1.

exponential polynomial binary

(x0 x1 x2 x3 )
0 0 0000
𝛼0 1 1000
𝛼1 x 0100
𝛼2 x2 0010
𝛼3 x3 0001
𝛼 4
1+x 1100
𝛼5 x + x2 0110
𝛼6 x2 + x3 0011
𝛼 7
1+x+x 3
1101
𝛼8 1 + x2 1010
𝛼9 x + x3 0101
𝛼 10 1 + x + x2 1110
𝛼 11 x + x2 + x3 0111
𝛼 12
1+x+x +x 2 3
1111
𝛼 13 1 + x2 + x3 1011
𝛼 14 1 + x3 1001

performed by interpreting the field elements as polynomials with coefficients from GF(2), and applying
the polynomial arithmetic mod x4 + x + 1 (see Figures 11.1 and 11.2).

Decoding of BCH codes

Suppose we transmit the code polynomial c0 + c1 x + · · · + cn−1 xn−1 , and we receive z0 + z1 x + · · · +
zn−1 xn−1 . We define the polynomial e(x) = (z0 − c0 ) + (z1 − c1 )x + … + (zn−1 − cn−1 )xn−1 = e0 + e1 x +
· · · + en−1 xn−1 , where ei are elements from GF(qm ), the field in which parity control is performed. If we
express polynomials as vectors, we obtain z = c + e; furthermore, we recall that, defining the matrix
⎡ 1 𝛼 m0 (𝛼 m0 )2 … (𝛼 m0 )n−1 ⎤
⎢ ⎥
1 𝛼 m0 +1
(𝛼 m0 +1 )2 … (𝛼 m0 +1 )n−1 ⎥
H=⎢ (11.187)
⎢⋮ ⋮ ⎥
⎢ 1 𝛼 m0 +d−2 (𝛼 m0 +d−2 )2 … (𝛼 m0 +d−2 )n−1 ⎥
⎣ ⎦
we obtain
⎡ sm0 ⎤
⎢ s ⎥
⎢ m0 +1 ⎥
Hz = He = s = ⎢ sm0 +2 ⎥ (11.188)
⎢ ⋮ ⎥
⎢ ⎥
⎣ sm0 +d−2 ⎦
where
∑
n−1
sj = e𝓁 (𝛼 j )𝓁 , j = m0 , m0 + 1, … , m0 + d − 2 (11.189)
𝓁=0
11.2. Block codes 567

Assuming there are 𝜈 errors, that is 𝑤(e) = 𝜈, we can write

𝜈
∑
sj = 𝜀i (𝜉i )j , j = m0 , m0 + 1, … , m0 + d − 2 (11.190)
i=1

where the coefficients 𝜀i are elements from GF(qm ) that represent the values of the errors, and the coef-
ficients 𝜉i are elements from GF(qm ) that give the positions of the errors. In other words, if 𝜉i = 𝛼 𝓁 , then
an error has occurred in position 𝓁, where 𝓁 ∈ {0, 1, 2, … , n − 1}. The idea of a decoding algorithm
is to solve the set of non-linear equations for the unknowns 𝜀i and 𝜉i ; then there are 2𝜈 unknowns and
d − 1 equations.
We show that it is possible to solve this set of equations if 2𝜈 ≤ d − 1, assuming m0 = 1; in the case
m0 ≠ 1, the decoding procedure does not change.
Consider the polynomial in x, also called error indicator polynomial,

𝜆(x) = 𝜆𝜈 x𝜈 + 𝜆𝜈−1 x𝜈−1 + · · · + 𝜆1 x + 1 (11.191)

defined as the polynomial that has as zeros the inverse of the elements that locate the positions of the
errors, that is 𝜉i−1 , i = 1, … , 𝜈. Then

𝜆(x) = (1 − x𝜉1 )(1 − x𝜉2 ) · · · (1 − x𝜉𝜈 ) (11.192)

If the coefficients of 𝜆(x) are known, it is possible to find the zeros of 𝜆(x), and thus determine the
positions of the errors.
The first step of the decoding procedure consists in evaluating the coefficients 𝜆1 , … , 𝜆𝜈 using the
j+𝜈
syndromes (11.190). We multiply both sides of (11.191) by 𝜀i 𝜉i and evaluate the expression found for
x = 𝜉i , obtaining
−1

j+𝜈
0 = 𝜀i 𝜉i (1 + 𝜆1 𝜉i−1 + 𝜆2 𝜉i−2 + · · · + 𝜆𝜈 𝜉i−𝜈 ) (11.193)

which can be written as

j+𝜈 j+𝜈−1 j+𝜈−2 j
𝜀i (𝜉i + 𝜆1 𝜉i + 𝜆2 𝜉i + · · · + 𝜆𝜈 𝜉i ) = 0 (11.194)

(11.194) holds for i = 1, … , 𝜈, and for every value of j. Adding these equations for i = 1, … , 𝜈, we
get
𝜈
∑ j+𝜈 j+𝜈−1 j+𝜈−2 j
𝜀i (𝜉i + 𝜆1 𝜉i + 𝜆2 𝜉i + … + 𝜆𝜈 𝜉i ) = 0 for every j (11.195)
i=1

or equivalently
𝜈 𝜈 𝜈 𝜈
∑ j+𝜈
∑ j+𝜈−1
∑ j+𝜈−2
∑ j
𝜀i 𝜉i + 𝜆1 𝜀i 𝜉i + 𝜆2 𝜀i 𝜉i + · · · + 𝜆𝜈 𝜀i 𝜉i = 0 for every j (11.196)
i=1 i=1 i=1 i=1

As 𝜈 ≤ (d − 1)∕2, if 1 ≤ j ≤ 𝜈 the summations in (11.196) are equal to the syndromes (11.190); there-
fore, we obtain
𝜆1 sj+𝜈−1 + 𝜆2 sj+𝜈−2 + · · · + 𝜆𝜈 sj = −sj+𝜈 , j = 1, … , 𝜈 (11.197)

(11.197) is a system of linear equations that can be written in the form

⎡ s1 s2 s3 … s𝜈−1 s𝜈 ⎤ ⎡ 𝜆𝜈 ⎤ ⎡ −s𝜈+1 ⎤
⎢ ⎥⎢ ⎥ ⎢ ⎥
⎢ s2 s3 s4 … s𝜈 s𝜈+1 ⎥ ⎢ 𝜆𝜈−1 ⎥ = ⎢ −s𝜈+2 ⎥ (11.198)
⎢⋮ ⋮ ⋮ ⎥⎢ ⋮ ⎥ ⎢ ⋮ ⎥
⎢s s ⎥⎢ ⎥ ⎢ −s ⎥
s
⎣ 𝜈 𝜈+1 𝜈+2 … s2𝜈−2 2𝜈−1 ⎦ ⎣ 𝜆1 ⎦
s ⎣ 2𝜈 ⎦
568 Chapter 11. Channel codes

Thus, we have obtained 𝜈 equations in the unknowns 𝜆1 , 𝜆2 , … , 𝜆𝜈 . To show that these equations are
linearly independent we see that, from (11.190), the matrix of the coefficients can be factorized as
⎡ s1 s2 s3 … s𝜈−1 s𝜈 ⎤
⎢ ⎥
⎢ s2 s3 s4 … s𝜈 s𝜈+1 ⎥ =
⎢⋮ ⋮ ⋮ ⎥
⎢s s ⎥
⎣ 𝜈 𝜈+1 s𝜈+2 … s2𝜈−2 s2𝜈−1 ⎦
(11.199)
⎡ 1 1 1 … 1 ⎤ ⎡ 𝜀1 𝜉1 ⎤ ⎡ 1 𝜉1 … 𝜉 𝜈−1 ⎤
⎢ ⎥⎢ ⎥⎢ 1
𝜈−1 ⎥
⎢ 𝜉1 𝜉2 𝜉3 … 𝜉𝜈 ⎥ ⎢ 𝜀2 𝜉2 0 ⎥ ⎢ 1 𝜉2 … 𝜉2 ⎥
⎢ ⋮ ⋮ ⎥⎢ ⋱ ⎥⎢⋮ ⋮ ⎥
⎢ 𝜉 𝜈−1 𝜉 𝜈−1 𝜉 𝜈−1 … 𝜉 𝜈−1 ⎥ ⎢ 0 𝜀𝜈 𝜉𝜈 ⎥ ⎢ 1 𝜉 … 𝜉 𝜈−1 ⎥
⎣ 1 2 3 𝜈 ⎦⎣ ⎦⎣ 𝜈 𝜈 ⎦
The matrix of the coefficients has a non-zero determinant if each of the matrices on the right-hand
side of (11.199) has a non-zero determinant. We note that the first and third matrix are Vandermonde
matrices, hence, they are non-singular if 𝜉i ≠ 𝜉m , i ≠ m; the second matrix is also non-singular, as it is
a diagonal matrix with non-zero terms on the diagonal. This assumes we have at most 𝜈 errors, with
𝜈 ≤ (d − 1)∕2. As 𝜈 is arbitrary, we initially choose 𝜈 = (d − 1)∕2 and compute the determinant of the
matrix (11.199). If this is non-zero, then we have the correct value of 𝜈; otherwise, if the determinant is
zero, we reduce 𝜈 by one and repeat the computation. We proceed until we obtain a non-zero determinant,
thus finding the number of errors that occurred.
After finding the solution for 𝜆1 , … , 𝜆𝜈 (see (11.198)), we obtain the positions of the errors by finding
the zeros of the polynomial 𝜆(x) (see (11.192)). Note that an exhaustive method to search for the 𝜈 zeros
of the polynomial 𝜆(x) requires that at most n possible roots of the type 𝛼 𝓁 , 𝓁 ∈ {0, 1, … , n − 1}, are
taken into consideration. We now compute the value of the non-zero elements 𝜀i , i = 1, … , 𝜈, of the
vector e. If the code is binary, the components of the vector e are immediately known, otherwise, we
solve the system of linear equations (11.190) for the 𝜈 unknowns 𝜀1 , 𝜀2 , … , 𝜀𝜈 . The determinant of the
matrix of the system of linear equations is given by
⎡ 𝜉1 𝜉2 … 𝜉𝜈 ⎤ ⎡ 1 1 … 1 ⎤
⎢ 2 ⎥ ⎢ ⎥
𝜉 𝜉2 … 𝜉𝜈 ⎥
2 2 𝜉 𝜉2 … 𝜉𝜈 ⎥
det ⎢ 1 = 𝜉1 𝜉2 … 𝜉𝜈 det ⎢ 1 (11.200)
⎢ ⋮ ⋮ ⋮ ⎥ ⎢ ⋮ ⋮ ⎥
⎢ 𝜉𝜈 𝜉2𝜈 … 𝜉𝜈𝜈 ⎥⎦ ⎢ 𝜉 𝜈−1 𝜉 𝜈−1 … 𝜉 𝜈−1 ⎥
⎣ 1 ⎣ 1 2 𝜈 ⎦
The determinant of the Vandermonde matrix in (11.200) is non-zero if 𝜈 errors occurred, as the ele-
ments 𝜉1 , … , 𝜉𝜈 are non-zero and distinct.
In summary, the original system of non-linear equations (11.190) is solved in the following three steps:
Step 1: find the coefficients 𝜆1 , 𝜆2 , … , 𝜆𝜈 of the error indicator polynomial by solving a system of
linear equations.
Step 2: find the 𝜈 roots 𝜉1 , 𝜉2 , … , 𝜉𝜈 of a polynomial of degree 𝜈.
Step 3: find the values of the errors 𝜀1 , 𝜀2 , … , 𝜀𝜈 by solving a system of linear equations.
For binary codes, the last step is omitted.

Efficient decoding of BCH codes

The computational complexity for the decoding of BCH codes illustrated in the previous Section, lies
mainly in the solution of the systems of linear equations (11.198) and (11.190). For small values of 𝜈, the
direct solution of these systems of equations by inverting matrices does not require a high computational
complexity; we recall that the number of operations necessary to invert a 𝜈 × 𝜈 matrix is of the order of
11.2. Block codes 569

𝜈 3 . However, in many applications, it is necessary to resort to codes that are capable of correcting several
errors, and it is thus desirable to find more efficient methods for the solution. The method developed by
Berlekamp is based on the observation that the matrix of the coefficients and the known data vector
in (11.198) have a particular structure. Assuming that the vector 𝝀 = [𝜆𝜈 , 𝜆𝜈−1 , … , 𝜆1 ]T is known, then
from (11.197) for the sequence of syndromes s1 , s2 , … , s2𝜈 , the recursive relation holds
𝜈
∑
sj = − 𝜆i sj−i , j = 𝜈 + 1, … , 2𝜈 (11.201)
i=1

For a given 𝝀, (11.201) is the equation of a recursive filter, which can be implemented by a shift
register with feedback, whose coefficients are given by 𝝀, as illustrated in Figure 11.8. The solution of
(11.198) is thus equivalent to the problem of finding the shift register with feedback of minimum length
that, if suitably initialized, yields the sequence of syndromes. This will identify the polynomial 𝜆(x) of
minimum degree 𝜈, that we recall exists and is unique, as the 𝜈 × 𝜈 matrix of the original problem admits
the inverse.

−λ1 −λ2 −λ υ

Tc Tc Tc
sj sj −1 sj −υ

Figure 11.8 Recursive filter to compute syndromes (see (11.201)).

The Berlekamp–Massey algorithm to find the recursive filter can be applied in any field and does not
make use of the particular properties of the sequence of syndromes s1 , s2 , … , sd−1 . To determine the
recursive filter we must find two quantities, that we denote as (L, 𝜆(x)), where L is the length of the shift
register and 𝜆(x) is the polynomial whose degree 𝜈 must satisfy the condition 𝜈 ≤ L. The algorithm is
inductive, that is for each r, starting from r = 1, we determine a shift register that generates the first r
syndromes. The shift register identified by (Lr , 𝜆(r) (x)) will then be a shift register of minimum length
that generates the sequence s1 , … , sr .

Berlekamp–Massey algorithm. Let s1 , … , sd−1 be a sequence of elements from any field. Assuming
the initial conditions 𝜆(0) (x) = 1, 𝛽 (0) (x) = 1, and L0 = 0, we use the following set of recursive equations
to determine 𝜆(d−1) (x):
for r = 1, … , d − 1,
∑
n−1
Δr = 𝜆j(r−1) sr−j (11.202)
j=0
{
1 if Δr ≠ 0 and 2Lr−1 ≤ r − 1
𝛿r = (11.203)
0 otherwise ,
Lr = 𝛿r (r − Lr−1 ) + (1 − 𝛿r ) Lr−1 (11.204)
[ ] [ ] [ (r−1) ]
𝜆(r) (x) 1 −Δr x 𝜆 (x)
= (11.205)
𝛽 (r) (x) Δ−1r 𝛿 r (1 − 𝛿 r ) x 𝛽 (r−1) (x)

Then 𝜆(d−1) (x) is the polynomial of minimum degree such that

𝜆(d−1)
0
=1 (11.206)
570 Chapter 11. Channel codes

and
∑
n−1
sr = − 𝜆(d−1)
j
sr−j , r = Ld−1 + 1, … , d − 1 (11.207)
j=1

Note that Δr can be zero only if 𝛿r = 0; in this case, we assign to Δ−1 r ⋅ 𝛿r the value zero. Moreover,
we see that the algorithm requires a complexity of the order of d2 operations, against a complexity
of the order of d3 operations needed by the matrix inversion in (11.198). To prove that the polyno-
mial 𝜆(d−1) (x) given by the algorithm is indeed the polynomial of minimum degree with 𝜆(d−1) 0
= 1 that
satisfies (11.207), we use the following two lemmas [2.
In Lemma 11.1, we find the relation between the lengths of the shift registers of minimum length
obtained in two consecutive iterations, Lr and Lr−1 . In Lemma 11.2, we use the algorithm to construct
a shift register that generates s1 , … , sr starting from the shift register of minimum length that generates
s1 , … , sr−1 . We will conclude that the construction yields the shift register of minimum length since it
satisfies Lemma 11.1.

Lemma 11.1
We assume that (Lr−1 , 𝜆(r−1) (x)) is the shift register of minimum length that generates s1 , … , sr−1 , while
(Lr , 𝜆(r) (x)) is the shift register of minimum length that generates s1 , … , sr−1 , sr , and 𝜆(r) (x) ≠ 𝜆(r−1) (x);
then
Lr ≥ max(Lr−1 , r − Lr−1 ) (11.208)

Proof. The inequality (11.208) is the combination of the two inequalities Lr ≥ Lr−1 and Lr ≥ r − Lr−1 .
The first inequality is obvious, because if a shift register generates a certain sequence it must also
generate any initial part of this sequence; the second inequality is obvious if Lr−1 ≥ r because Lr is a
non-negative quantity. Thus, we assume Lr−1 < r, and suppose that the second inequality is not satisfied;
then Lr ≤ r − 1 − Lr−1 , or r ≥ Lr−1 + Lr + 1. By assumption we have
⎧ ∑
Lr−1
⎪ sj = − 𝜆i(r−1) sj−i j = Lr−1 + 1, … , r − 1
⎪ i=1
⎨ (11.209)
⎪ ∑
Lr−1

⎪ sr ≠ − 𝜆i(r−1) sr−i
⎩ i=1

and
∑
Lr
sj = − 𝜆(r)
k
sj−k , j = Lr + 1, … , r (11.210)
k=1

We observe that
∑
Lr
∑
Lr
∑
Lr−1
sr = − 𝜆(r)
k
sr−k = 𝜆(r)
k
𝜆i(r−1) sr−k−i (11.211)
k=1 k=1 i=1

where the expression of sr−k is valid, as (r − k) goes from r − 1 to r − Lr . Hence, it belongs to the set
Lr−1 + 1, … , r − 1, as it is assumed r ≥ Lr−1 + Lr + 1.
Furthermore,
∑
Lr−1
∑
Lr−1
∑
Lr
sr ≠ − 𝜆(r−1)
i
sr−i = 𝜆 (r−1)
i
𝜆(r)
k
sr−i−k (11.212)
i=1 i=1 k=1

where the expression of sr−i is valid as (r − i) goes from r − 1 to r − Lr−1 . Hence, it belongs to the set
Lr + 1, … , r − 1, as it is assumed r ≥ Lr−1 + Lr + 1. The summations on the right-hand side of (11.212)
can be exchanged, thus obtaining the right-hand side of (11.211). But this yields sr of (11.211) different
from sr of (11.212), thus we get a contradiction. ◽
11.2. Block codes 571

Lemma 11.2
We assume that (Li , 𝜆(i) (x)), i = 1, … , r, identifies a sequence of shift registers of minimum length such
that 𝜆(i) (x) generates s1 , … , si . If 𝜆(r) (x) ≠ 𝜆(r−1) (x), then

Lr = max(Lr−1 , r − Lr−1 ) (11.213)

and every shift register that generates s1 , … , sr , and has a length that satisfies (11.213), is a shift register
of minimum length. The Berlekamp–Massey algorithm yields this shift register.

Proof. From Lemma 11.1, Lr cannot be smaller than the right-hand side of (11.213); thus, if we con-
struct a shift register that yields the given sequence and whose length satisfies (11.213), then it must be
a shift register of minimum length. The proof is obtained by induction.
We construct a shift register that satisfies the Lemma at the rth iteration, assuming that shift registers
were iteratively constructed for each value of the index k, with k ≤ r − 1. For each k, k = 1, … , r − 1,
let (Lk , 𝜆(k) (x)) be the shift register of minimum length that generates s1 , … , sk . We assume that

Lk = max(Lk−1 , k − Lk−1 ), k = 1, … , r − 1 (11.214)

Equation (11.214) is verified for k = 0, as L0 = 0 and L1 = 1. Let m be the index k at the most recent
iteration that required a variation in the length of the shift register. In other words, at the end of the
(r − 1)th iteration, m is the integer such that

Lr−1 = Lm > Lm−1 (11.215)

From (11.209) and (11.202), we have that

{
∑
Lr−1
∑
Lr−1
0 j = Lr−1 , … , r − 1
sj + 𝜆(r−1)
i
sj−i = 𝜆(r−1)
i
sj−i = (11.216)
i=1 i=0
Δr j=r

If Δr = 0, then the shift register (Lr−1 , 𝜆(r−1) (x)) also generates the first r symbols of the sequence,
hence, {
Lr = Lr−1
(11.217)
𝜆(r) (x) = 𝜆(r−1) (x)
If Δr ≠ 0, then it is necessary to find a new shift register. Recall from (11.215) that there was a variation
in the length of the shift register for k = m; therefore,
{
∑ (m−1)
Lm−1
0 j = Lm−1 , … , m − 1
sj + 𝜆i sj−i = (11.218)
i=1
Δ m ≠ 0 j=m

and by induction,
Lr−1 = Lm = max(Lm−1 , m − Lm−1 ) = m − Lm−1 (11.219)

as Lm > Lm−1 . We choose now the new polynomial

𝜆(r) (x) = 𝜆(r−1) (x) − Δr Δ−1

m x 𝜆
r−m (m−1)
(x) (11.220)

and let Lr = deg(𝜆(r) (x)). Then, as deg(𝜆(r−1) (x)) ≤ Lr−1 , and deg[xr−m 𝜆(m−1) (x)] ≤ r − m + Lm−1 , we get

Lr ≤ max(Lr−1 , r − m + Lm−1 ) ≤ max(Lr−1 , r − Lr−1 ) (11.221)

Thus, recalling Lemma 11.1, if 𝜆(r) (x) generates s1 , … , sr , then

Lr = max(Lr−1 , r − Lr−1 ) (11.222)

572 Chapter 11. Channel codes

It remains to prove that the shift register (Lr , 𝜆(r) (x)) generates the given sequence. By direct compu-
tation, we obtain
( L )
∑ r
(r)
sj − − 𝜆i sj−i
i=1 [ ]
∑Lr−1
∑ (L )
Lm−1
=s + 𝜆 (r−1)
s − Δ Δ−1 s + 𝜆 m−1 s (11.223)
j i j−i r m j−r+m i j−r+m−i
{ i=1 i=1
0 j = Lr , Lr + 1, … , r − 1
=
Δr − Δr Δ−1
m Δm = 0 j=r
Therefore, the shift register (Lr , 𝜆(r) (x)) generates s1 , … , sr . In particular, (Ld−1 , 𝜆(d−1) (x)) generates
s1 , … , sd−1 . This completes the proof of Lemma 2. ◽

We have seen that the computational complexity for the solution of the system of equations (11.198),
that yields the error indicator polynomial, can be reduced by the Berlekamp–Massey algorithm. We now
consider the system of equations (11.190) that yields the values of the errors. The computation of the
inverse matrix can be avoided in the solution of (11.190) by applying the Forney algorithm [2]. We recall
the expression (11.191) of the error indicator polynomial 𝜆(x), that has zeros for x = 𝜉i−1 , i = 1, … , 𝜈,
given by
𝜈
∏
𝜆(x) = (1 − x𝜉𝓁 ) (11.224)
𝓁=1
Define the syndrome polynomial as
d−1 𝜈
∑
d−1
∑ ∑ j
s(x) = sj xj = 𝜀i 𝜉i xj (11.225)
j=1 j=1 j=1

and furthermore, define the error evaluator polynomial 𝜔(x) as

𝜔(x) = s(x) 𝜆(x), mod xd−1 (11.226)

Proposition 11.5
The error evaluator polynomial can be expressed as
𝜈 𝜈
∑ ∏
𝜔(x) = x 𝜀i 𝜉i (1 − 𝜉j x) (11.227)
i=1 j=1
j≠i ◽

Proof. From the definition (11.226) of 𝜔(x), we obtain

[ d−1 𝜈 ][ 𝜈 ]
∑∑ j j
∏
𝜔(x) = 𝜀i 𝜉i x (1 − 𝜉𝓁 x) , mod xd−1
𝓁=1
𝜈
j=1 i=1
[ ] 𝜈
∑ ∑
d−1
∏ (11.228)
= 𝜀i 𝜉i x (1 − 𝜉i x) (𝜉i x)j−1
(1 − 𝜉𝓁 x), mod xd−1
i=1 j=1 𝓁=1
𝓁≠i

By inspection, we see that the term within brackets is equal to (1 − 𝜉id−1 xd−1 ); thus,
𝜈 𝜈
∑ ∏
𝜔(x) = 𝜀i 𝜉i x(1 − 𝜉id−1 xd−1 ) (1 − 𝜉𝓁 x), mod xd−1 (11.229)
i=1 𝓁=1
𝓁≠i

We now observe that (11.229), evaluated modulo xd−1 , is identical to (11.227). ◽

11.2. Block codes 573

Forney algorithm. We introduce the derivative of 𝜆(x) given by

𝜈 𝜈
∑ ∏
𝜆′ (x) = − 𝜉i (1 − x𝜉j ) (11.230)
i=1 j=1
j≠i

The values of the errors are given by

𝜔(𝜉𝓁−1 ) 𝜔(𝜉𝓁−1 )
𝜀𝓁 = 𝜈
=− (11.231)
∏ 𝜉𝓁−1 𝜆′ (𝜉𝓁−1 )
(1 − 𝜉j 𝜉𝓁−1 )
j=1
j≠𝓁

Proof. We evaluate (11.227) for x = 𝜉𝓁−1 , obtaining

𝜈
∏
𝜔(𝜉𝓁−1 ) = 𝜀𝓁 (1 − 𝜉j 𝜉𝓁−1 ) (11.232)
j=1
j≠𝓁

which proves the first part of equality (11.231). Moreover, from (11.230), we have
𝜈
∏
𝜆′ (𝜉𝓁−1 ) = −𝜉𝓁 (1 − 𝜉j 𝜉𝓁−1 ) (11.233)
j=1
j≠𝓁

which proves the second part of equality (11.231). ◽

Example 11.2.33 (Reed–Solomon (15,9) code with d = 7 (t = 3), and elements from GF(24 ))
From (11.186), using Tables 11.19 and 11.20, the generator polynomial is given by
g(x) = (x − 𝛼)(x − 𝛼 2 )(x − 𝛼 3 )(x − 𝛼 4 )(x − 𝛼 5 )(x − 𝛼 6 )
(11.234)
= x6 + 𝛼 10 x5 + 𝛼 14 x4 + 𝛼 4 x3 + 𝛼 6 x2 + 𝛼 9 x + 𝛼 6
Suppose that the code polynomial c(x) = 0 is transmitted and that the received polynomial is

z(x) = 𝛼x7 + 𝛼 5 x5 + 𝛼 11 x2 (11.235)

In this case e(x) = z(x). From (11.189), using Tables 11.19 and 11.20, the syndromes are
s1 = 𝛼𝛼 7 + 𝛼 5 𝛼 5 + 𝛼 11 𝛼 2 = 𝛼 12
s2 = 𝛼𝛼 14 + 𝛼 5 𝛼 10 + 𝛼 11 𝛼 4 = 1
s3 = 𝛼𝛼 21 + 𝛼 5 𝛼 15 + 𝛼 11 𝛼 6 = 𝛼 14
(11.236)
s4 = 𝛼𝛼 28 + 𝛼 5 𝛼 20 + 𝛼 11 𝛼 8 = 𝛼 13
s5 = 𝛼𝛼 35 + 𝛼 5 𝛼 25 + 𝛼 11 𝛼 10 = 1
s6 = 𝛼𝛼 42 + 𝛼 5 𝛼 30 + 𝛼 11 𝛼 12 = 𝛼 11
From (11.202) to (11.205), the algorithm develops according to the following d − 1 = 6 steps, starting
from the initial conditions
𝜆(0) (x) = 1
𝛽 (0) (x) = 1 (11.237)
L0 = 0
574 Chapter 11. Channel codes

Step 1 (r = 1): Δ1 = 𝛼 12 , and 2L0 = 0 = r − 1, 𝛿1 = 1, L1 = 1.

[ (1) ] [ ][ ]
𝜆 (x) 1 −𝛼 12 x 1 1 + 𝛼 12 x
= =
𝛽 (1) (x) 𝛼3 0 1 𝛼3

Step 2 (r = 2): Δ2 = 1 + 𝛼 9 = 𝛼 7 , and 2L1 = 2 > r − 1, 𝛿2 = 0, L2 = L1 = 1.

[ (2) ] [ ][ ]
𝜆 (x) 1 −𝛼 7 x 1 + 𝛼 12 x
=
𝛽 (2) (x) 0 x 𝛼3
[ ] [ ]
1 + 𝛼 12 x + 𝛼 10 x 1 + 𝛼3 x
= =
𝛼3 x 𝛼3 x

Step 3 (r = 3): Δ3 = 𝛼 14 + 𝛼 3 = 1, and 2L2 = 2 = r − 1, 𝛿3 = 1, L3 = 3 − 1 = 2.

[ (3) ] [ ][ ] [ ]
𝜆 (x) 1 −x 1 + 𝛼3 x 1 + 𝛼 3 x + 𝛼 3 x2
= =
𝛽 (3) (x) 1 0 𝛼3 x 1 + 𝛼3 x

Step 4 (r = 4): Δ4 = 𝛼 13 + 𝛼 3 𝛼 14 + 𝛼 3 = 1, and 2L3 = 4 > r − 1, 𝛿4 = 0, L4 = L3 = 2.

[ (4) ] [ ][ ]
𝜆 (x) 1 −x 1 + 𝛼 3 x + 𝛼 3 x2
=
𝛽 (4) (x) 0 x 1 + 𝛼3 x
[ ] [ ]
1 + 𝛼 x + 𝛼 x2 + x + 𝛼 3 x2
3 3 1 + 𝛼 14 x
= =
x + 𝛼 3 x2 x + 𝛼 3 x2

Step 5 (r = 5): Δ5 = 1 + 𝛼 14 𝛼 13 = 𝛼 11 , and 2L4 = 4 = r − 1, 𝛿5 = 1, L5 = 5 − 2 = 3.

[ (5) ] [ ][ ] [ ]
𝜆 (x) 1 −𝛼 11 x 1 + 𝛼 14 x 1 + 𝛼 14 x + 𝛼 11 x2 + 𝛼 14 x3
= =
𝛽 (5) (x) 𝛼4 0 x + 𝛼 3 x2 𝛼4 + 𝛼3 x

Step 6 (r = 6): Δ6 = 𝛼 11 + 𝛼 14 + 𝛼 11 𝛼 13 + 𝛼 14 𝛼 14 = 0, 𝛿6 = 0, L6 = L5 = 3.
[ (6) ] [ ][ ]
𝜆 (x) 1 0 1 + 𝛼 14 x + 𝛼 11 x2 + 𝛼 14 x3
=
𝛽 (6) (x) 0 x 𝛼4 + 𝛼3 x
[ ]
1 + 𝛼 14 x + 𝛼 11 x2 + 𝛼 14 x3
=
𝛼 4 x + 𝛼 3 x2

The error indicator polynomial is 𝜆(x) = 𝜆(6) (x). By using the exhaustive method to find the three
roots, we obtain

𝜆(x) = 1 + 𝛼 14 x + 𝛼 11 x2 + 𝛼 14 x3 = (1 − 𝛼 7 x)(1 − 𝛼 5 x)(1 − 𝛼 2 x) (11.238)

Consequently, the three errors are at positions 𝓁 = 2, 5, and 7.

To determine the values of the errors, we use the Forney algorithm. The derivative of 𝜆(x) is given by

𝜆′ (x) = 𝛼 14 + 𝛼 14 x2 (11.239)

The error evaluator polynomial is given by

𝜔(x) = (𝛼 12 x + x2 + 𝛼 14 x3 + 𝛼 13 x4 + x5 + 𝛼 11 x6 )
(1 + 𝛼 14 x + 𝛼 11 x2 + 𝛼 14 x3 ) mod x6 (11.240)
= 𝛼 12 x + 𝛼 12 x2 + 𝛼 8 x3
11.2. Block codes 575

Thus, the values of the errors are

𝜔(𝛼 −2 )
𝜀2 = − = 𝛼 11
𝛼 −2 𝜆′ (𝛼 −2 )
𝜔(𝛼 −5 )
𝜀5 = − = 𝛼5 (11.241)
𝛼 −5 𝜆′ (𝛼 −5 )
𝜔(𝛼 −7 )
𝜀7 = − −7 ′ −7 = 𝛼
𝛼 𝜆 (𝛼 )

An alternative approach for the encoding and decoding of Reed–Solomon codes utilizes the con-
cept of Fourier transform on a Galois field [2, 5]. Let 𝛼 be a primitive element of the field GF(q). The
Fourier transform on the field GF(q) (GFFT) of a vector c = (c0 , c1 , … , cn−1 ) of n bits is defined as
(C0 , C1 , … , Cn−1 ), where
∑
n−1
Cj = ci 𝛼 ij , j = 0, … , n − 1 (11.242)
i=0

Let us consider a code word c of n bits in the time domain from a Reed–Solomon cyclic code that
corrects up to t errors; then c corresponds to a code polynomial that has as roots 2t = d − 1 consecutive
powers of 𝛼. If we take the GFFT of this word, we find that in the ‘frequency domain’, the transform has
2t consecutive components equal to zero. Indeed, from (11.176), specialized to Reed–Solomon codes,
and from (11.242), we can show that the two conditions are equivalent, that is a polynomial has 2t
consecutive powers of 𝛼 as roots if and only if the transform has 2t consecutive components equal
to zero. The approach that resorts to the GFFT is, therefore, dual of the approach that uses the gen-
erator polynomial. This observation leads to the development of efficient methods for encoding and
decoding.
Reed–Solomon codes have found many applications since their original design. Even nowadays, they
are used in many fields, including magnetic storage [6–8].

11.2.6 Performance of block codes

In this section, we consider the probability of error in the decoding of block codes, in the case of decoding
with hard or soft input (see Section 6.6 on page 270). For an in-depth study of the subject, we refer the
reader, for example, to [9].
With reference to Figure 6.1, let Pbit be the bit error probability for the detection of the bits of the
binary sequence {̃cm }, or bit error probability of the channel, P𝑤 the error probability for a code word,
and P(dec)
bit
the error probability for a bit of the binary sequence {b̂ l } obtained after decoding. For a (n, k)
H
block code with t = (dmin − 1)∕2 and hard input decoding, the following inequality holds:
n ( )
∑ n
P𝑤 ≤ Pibit (1 − Pbit )n−i (11.243)
i=t+1
i

which, under the condition nPbit ≪ 1, can be approximated as

( )
n t+1
P𝑤 ≃ Pbit (1 − Pbit )n−t−1 (11.244)
t+1
The inequality (11.243) follows from the channel model (11.12) assuming errors that are i.i.d., and
from the consideration that the code may not be perfect (see page 525), and therefore, it could correct
also some received words with more than t errors.
576 Chapter 11. Channel codes

If a word error occurs, the most probable event is that the decoder decides for a code word with distance
H H
dmin = 2t + 1 from the transmitted code word, thus making dmin bit errors in the sequence {̂cm }. As c is
formed of n bits, we have that at the decoder output, the bit error probability is
2t + 1
P(dec)
bit
≃ P𝑤 (11.245)
n

Example 11.2.34
H
For a (5,1) repetition code with dmin = 5 (see page 525), decoding with hard input yields
( ) ( )
5 5
P𝑤 = P3bit (1 − Pbit )2 + P4bit (1 − Pbit ) + P5bit (11.246)
3 4

Example 11.2.35
H
For an (n, k) Hamming code with dmin = 3 (see page 525), (11.243) yields
n ( )
∑ n
P𝑤 ≤ Pibit (1 − Pbit )n−i
i=2 i (11.247)
= 1 − [(1 − Pbit )n + nPbit (1 − Pbit )n−1 ]

For example, for a (15,11) code, if Pbit = 10−3 , then P𝑤 ≃ 10−4 , and from (11.245), we get P(dec)
bit
≃
2 10−5 .

The decoders that have been considered so far are classified as hard input decoders, as the demodulator
output is quantized to the values of the coded symbols before decoding. In general, other decoding algo-
rithms with soft input may be considered, that directly process the demodulated signal, and consequently,
the decoder input is real valued (see Section 11.3.2).
In the case of antipodal binary signals and soft input decoding, we get
√
⎛ R dH 2E ⎞
P𝑤 ≃ (2k − 1) Q ⎜
c min b⎟
(11.248)
⎜ N0 ⎟
⎝ ⎠

11.3 Convolutional codes

Convolutional codes are a subclass of the class of tree codes, so named because their code words are
conveniently represented as sequences of nodes in a tree. Tree codes are of great interest because decod-
ing algorithms have been found that are easy to implement and can be applied to the entire class of tree
codes, in contrast to decoding algorithms for block codes, each designed for a specific class of codes, as
for example BCH codes.
Several approaches have been used in the literature for describing convolutional codes; here we will
illustrate these approaches by first considering a specific example.

Example 11.3.1
Consider a rate 1∕2 binary convolutional code, obtained by the encoder illustrated in Figure 11.9a.
For each bit bk that enters the encoder, two output bits, c(1)
k
and c(2)
k
, are transmitted. The first out-
put ck is obtained if the switch at the output is in the upper position, and the second output c(2)
(1)
k
is
obtained if the switch is in the lower position; the two previous input bits, bk−1 and bk−2 , are stored
in the memory of the encoder. As the information bit is not presented directly to one of the outputs,
we say that the code is non-systematic. The two coded bits are generated as linear combinations of the
11.3. Convolutional codes 577

ck(1)

bk
D D

(2)
ck
(a)
10
(a)
10
(a)
01
01 (b)
(a) 00
(c)
01
(b)
11
11 (d )
(c) 01
(a)
00
(c)
10
10 (b)
(b) 11
1 (c)
11
(d)
00
(d )
(d) 10
(a)
01
(a)
0 01
11 (b)
(c) 00
(c)
10
(b)
11
00 (d )
(d) 01
(a)
11
(c)
10
00 (b)
(d) 11
(c)
00
(d)
00
(d )
(b)

Figure 11.9 (a) Encoder and (b) tree diagram for the convolutional code of Example 11.3.1.

bits of the message; denoting the input sequence as … , b0 , b1 , b2 , b3 , …, and the output sequence as
… , c0(1) , c(2)
0
, c(1)
1
, c(2)
1
, c2(1) , c(2)
2
, c(1)
3
, c(2)
3
, …, then the following relations hold:

c(1)
k
= bk ⊕ bk−1 ⊕ bk−2
(11.249)
c(2)
k
= bk ⊕ bk−2
A convolutional code may be described in terms of a tree, trellis, or state diagram; for the code defined
by (11.249), these descriptions are illustrated in Figures 11.9b and 11.10a,b, respectively.
With reference to the tree diagram of Figure 11.9b, we begin at the left (root) node and proceed to the
right by choosing an upper path if the input bit is equal to 1 and a lower path if the input bit is 0. We
578 Chapter 11. Channel codes

00 00 00 00 00
(d ) = (0,0)
11 11 11 11 11
11 11 11
(c) = (1,0)
10 10 10 10
(b) = (0,1)
01 01 01 01 01 01 01
(a) = (1,1)
10 10 10
(a)

1/10

1/01 a
0/01

0/11
0/00 d b

1/00
1/11
0/10
c
(b)

Figure 11.10 (a) Trellis diagram and (b) state diagram for the convolutional code of
Example 11.3.1.

output the two bits represented by the label on the branch that takes us to the next node and then repeat
this process at the next node. The nodes or states of the encoder are labeled with the letters a, b, c, and d,
which indicate the relation with the four possible values assumed by the two bits stored in the encoder,
according to the table:
bk−1 bk−2 label
0 0 d
1 0 c (11.250)
0 1 b
1 1 a

If for example we input the sequence b0 , b1 , b2 , b3 , … = 1 1 0 1 …, we would, then output the

sequence c(1)0
, c(2)
0
, c(1)
1
, c(2)
1
, c(1)
2
, c(2)
2
, c(1)
3
, c(2)
3
, … = 1 1 0 1 0 1 0 0 … . As, at any depth in the tree,
nodes with the same label will have the same tree growing from them, we can superimpose these nodes
on a single node. This results in the trellis diagram represented in Figure 11.10a, where solid and
dashed lines correspond to transitions determined by input bits equal to 1 and 0, respectively.
The state diagram for the encoder is illustrated in Figure 11.10b. The four states (a, b, c, d) correspond
to the four possible combinations of bits stored in the encoder. If the encoder is in a certain state, a
transition to one of two possible states occurs, depending on the value of the input bit. Possible transitions
between states are represented as arcs, on which an arrow indicates the direction of the transition; with
each arc is associated a label that indicates the value assumed by the input bit and also the value of the
resulting output bits. The description of the encoder by the state diagram is convenient for analysing the
properties of the code, as we will see later.
11.3. Convolutional codes 579

It is also convenient to represent code sequences in terms of the D transform, as

b(D) = b0 + b1 D + b2 D2 + b3 D3 + · · ·
= c(1)
c(1) (D) 0
+ c(1)
1
D + c(1)
2
D2 + c(1)
3
D3 + · · · = g(1,1) (D) b(D) (11.251)
(2) (2) (2) (2)
c (D) = c0 + c1 D + c2 D + c3 D3 + · · · = g(2,1) (D) b(D)
(2) 2

where g(1,1) (D) = 1 + D + D2 , and g(2,1) (D) = 1 + D2 .

11.3.1 General description of convolutional codes

In general, we consider convolutional codes with symbols from GF(q); assuming the encoder produces
n0 output code symbols for every k0 input message symbols, the code rate is equal to k0 ∕n0 . It is conve-
nient to think of the message sequence as being the interlaced version of k0 different message sequences,
and to think of the code sequence as the interlaced version of n0 different code sequences. In other words,
given the information sequence {b𝓁 }, we form the k0 subsequences
b(i)
k
= bkk0 +i−1 , i = 1, … , k0 (11.252)
that have D transform defined as
b(1) b(1) b(1) · · · ⇐==⇒ b(1) (D) = b(1)
0 1 2 0
+ b(1)
1
D + b(1)
2
D2 + · · ·
(2) (2) (2) (2) (2) (2) 2
b0 b1 b2 · · · ⇐==⇒ b (D) = b0 + b1 D + b2 D + · · ·
(2)
(11.253)
⋮ ⋮
(k ) (k ) (k ) (k ) (k ) (k )
b0 0 b1 0 b2 0 · · · ⇐==⇒ b(k0 ) (D) = b0 0 + b1 0 D + b2 0 D2 + · · ·
Let c(1) (D), c(2) (D), … , c(n0 ) (D) be the D transforms of the n0 output sequences; then
∑
k0
c(j) (D) = g(j,i) (D) b(i) (D), j = 1, 2, … , n0 (11.254)
i=1

An (n0 , k0 ) convolutional code is then specified by giving the coefficients of all the polynomials
g(j,i) (D), i = 1, 2, … , k0 , j = 1, 2, … , n0 .
If for all j = 1, 2, … , k0 , we have {
1 j=i
g(j,i) (D) = (11.255)
0 j≠i
then the code is systematic and k0 of the n0 output sequences are just the message sequences.
An encoder for a convolutional code needs storage elements. Let 𝜈 be the constraint length of the
code,5
𝜈 = max (deg g(j,i) (D)) (11.256)
j,i

Therefore, the encoder of a convolutional code must store 𝜈 previous blocks of k0 message symbols to
form a block of n0 output symbols.
The general structure of an encoder for a code with k0 = 1 and n0 = 2 is illustrated in Figure 11.11;
for such an encoder, 𝜈k0 storage elements are necessary. If the code is systematic, then the encoder can
be implemented with 𝜈(n0 − k0 ) storage elements, as illustrated in Figure 11.12 for k0 = 2 and n0 = 3.
If we interpret the sequence {ck } as the output of a sequential finite-state machine (see Appendix 7.B),
at instant k the trellis of a non-systematic code is defined by the three signals:
1. Input
(k )
[b(1)
k
, b(2)
k
, … , bk 0 ] (11.257)

5
Many authors define the constraint length as 𝜈 + 1, where 𝜈 if given by (11.256).
580 Chapter 11. Channel codes

c(1)(D)

(1,1) (1,1) (1,1) (1,1) (1,1)

g0 g1 g2 g3 gν

b(1)(D) D D D D

(2,1) (2,1) (2,1) (2,1) (2,1)

g0 g1 g2 g3 gν

c(2)(D)

Figure 11.11 Block diagram of an encoder for a convolutional code with k0 = 1, n0 = 2, and
constraint length 𝜈.

(1)
b (D) (1)
c (D)
(2)
b (D) (2)
c (D)

(3,1) (3,1)
gν g(3,2)
ν
g (3,1) (3,2)
gν–1 g0 g(3,2)
ν–1 0

D D D (3)
c (D)

Figure 11.12 Block diagram of an encoder for a systematic convolutional code with k0 = 2, n0 = 3,
and constraint length 𝜈.

2. State
(k ) (k )
[b(1)
k
, … , bk 0 , … , b(1)
k−(𝜈−1)
, … , bk−(𝜈−1)
0
] (11.258)

3. Output
(n )
[c(1)
k
, c(2)
k
, … , ck 0 ] (11.259)

where ck , j = 1, … , n0 , is given by (11.254). Then there are qk0 𝜈 states in the trellis. There are qk0
(j)

branches departing from each state and qk0 branches merging into a state. The output vector consists
of n0 q-ary symbols.
11.3. Convolutional codes 581

Parity check matrix

A semi-infinite parity check matrix can be defined in general for convolutional codes; however, we note
that it is only in the case of systematic codes that we can easily express the elements of this matrix in
terms of the coefficients of the generator polynomials g(j,i) (D). We write the coefficients of the generator
polynomials in the form
(n ,1) (n ,1) (n ,1)
g(1,1)
0
g(2,1)
0
· · · g0 0 g(1,1)
1
· · · g1 0 · · · g(1,1)
𝜈 g(2,1)
𝜈 · · · g𝜈 0
(1,2) (2,2) (n0 ,2) (1,2) (n0 ,2) (1,2) (2,2) (n ,2)
g0 g0 · · · g0 g 1 · · · g1 · · · g𝜈 g 𝜈 · · · g𝜈 0
(11.260)
⋮ ⋮
(1,k ) (2,k ) (n ,k ) (1,k ) (n ,k ) (1,k ) (2,k ) (n ,k )
g0 0 g 0 0 · · · g 0 0 0 g 1 0 · · · g1 0 0 · · · g 𝜈 0 g 𝜈 0 · · · g𝜈 0 0
If the code is systematic, the parity matrix of the generator polynomials can be written as
I P0 0 P1 · · · 0 P 𝜈 (11.261)
where I and 0 are k0 × k0 matrices and Pi , i = 0, … , 𝜈, are k0 × (n0 − k0 ) matrices.
The semi-infinite parity check matrix is then
⎡ −PT I 0 0 0 0 …⎤
⎢ 0
T ⎥
⎢ −P1 0 −PT0 I 0 0 …⎥
⎢ … … … …⎥
=⎢
0 … ⎥⎥
H∞ T (11.262)
⎢ −P𝜈 0 −PT𝜈−1 0 −PT𝜈−2
⎢ 0 0 −PT𝜈 0 −PT𝜈−1 0 …⎥
⎢ … … … … ⎥⎦
⎣
Thus, for any code word c of infinite length, H∞ c = 0. Often, rather than considering the semi-infinite
matrix H∞ , we consider the finite matrix H defined as
⎡ −PT0 I 0 0 … 0 0⎤
⎢ T T ⎥
H=⎢
−P 1 0 −P 0 I … 0 0⎥
(11.263)
⎢ … … … … ⎥
⎢ −PT 0 −PT 0 … −PT I ⎥
⎣ 𝜈 𝜈−1 0 ⎦
The bottom row of matrices of the matrix H is called the basic parity check matrix. From it, we can
see that the parity symbols in a block are given by the linear combination of information bits in that
block, corresponding to non-zero terms in −PT0 , in the immediately preceding block, corresponding to
non-zero terms in −PT1 , and so on until the 𝜈th preceding block, corresponding to non-zero terms in −PT𝜈 .

Generator matrix
From (11.260), we introduce the matrices
⎡ g(1,1) g(2,1) … g(n0 ,1) ⎤
⎢ i i i ⎥
gi = ⎢ ⋮ ⋮ ⎥, i = 0, … , 𝜈 (11.264)
⎢ g(1,k0 ) g(2,k0 ) … g(n0 ,k0 ) ⎥
⎣ i i i ⎦
Hence, the generator matrix is of the form
⎡ g0 g1 … g𝜈 0 … ⎤
G∞ = ⎢ 0 g0 … g𝜈−1 g𝜈 … ⎥ (11.265)
⎢ ⎥
⎣… … … … … …⎦
Some examples of convolutional codes with the corresponding encoders and generator matrices are
illustrated in Figure 11.13.
582 Chapter 11. Channel codes

Transfer function
H
An important parameter of a convolutional code is dfree , that determines the performance of the code
(see Section 11.3.3)

Definition 11.14
(n )
Let e(D) = [e(n0 ) (D), … , e(1) (D)] be any error sequence between two code words c1 (D) = [c1 0 (D), … ,
(n0 ) (n0 )
c1 (D)] and c2 (D) = [c2 (D), … , c2 (D)], that is c1 (D) = c2 (D) + e(D), and ek = [ek , … , e(1)
(1) (1)
k
]
denotes the kth element of the sequence. We define the free Hamming distance of the code as
∑
∞
H
dfree = min 𝑤(ek ) (11.266)
e(D)
k=0

where 𝑤 is introduced in Definition 11.4 on page 519. As the code is linear, dfree
H
corresponds to the
minimum number of symbols different from zero in a non-zero code word. ◽

Next, we consider a method to compute the weights of all code words in a convolutional code; to
illustrate the method, we examine the simple binary encoder of Figure 11.9a.
We begin by reproducing the trellis diagram of the code in Figure 11.14, where each path is now
labeled with the weight of the output bits corresponding to that path. We consider all paths that diverge
from state (d) and return to state (d) for the first time after a number of steps j. By inspection, we find one
such path of weight 5 returns to state (d) after 3 steps; moreover, we find two distinct paths of weight 6,
one that returns to state (d) after 4 steps and another after 5 steps. Hence, we find that this code has
H
dfree = 5.
We now look for a method that enables us to find the weights of all code words as well as the lengths of
the paths that give origin to the code words with these weights. Consider the state diagram for this code,
redrawn in Figure 11.15 with branches labeled as D2 , D, or D0 = 1, where the exponent corresponds to
the weight of the output bits corresponding to that branch. Next, we split node (0,0) to obtain the state
diagram of Figure 11.16, and we compute a generating function for the weights. The generating function
is the transfer function of a signal flow graph with unit input. From Figure 11.16, we obtain this transfer
function by solving the system of equations

𝛽 = D2 𝛼 + 1𝛾
𝛾 = D𝛽 + D𝛿
(11.267)
𝛿 = D𝛽 + D𝛿
𝜂 = D2 𝛾
Then, we get
𝜂 D5
t(D) = = = D5 + 2D6 + 4D7 + · · · + 2i Di+5 + … (11.268)
𝛼 1 − 2D
From inspection of t(D), we find there is one code word of weight 5, two of weight 6, four of weight 7,
. . . . Equation (11.268) holds for code words of infinite length.
If we want to find code words that return to state (d) after j steps, we refer to the state diagram of
Figure 11.17. The term L introduced in the label on each branch keeps track of the length of the sequence,
as the exponent of L is augmented by 1 every time a transition occurs. Furthermore, we introduce the
term I in the label on a branch if the corresponding transition is due to an information bit equal to 1; for
each path on the trellis diagram, we can compute the corresponding number of information bits equal
11.3. Convolutional codes 583

c (1) (D)
g0 = (1,1)
(1)
b (D)
D D g1 = (1,0)

g2 = (1,1)
c (2) (D)
k0 = 1, n0 = 2, ν = 2
(a)

(1)
b (D) D
(1)
c (D)

g0 = 111
(2) 010
c (D)

101
(3) g1 =
c (D) 110

(2)
b (D) D

k0 = 2, n0 = 3, ν = 1
(b)

(1)
c (D)
(1)
b (D)
D

c (2) (D) 1001

g0 = 0101
(2)
b (D) 0011
D

c (3) (D)
0001
b (3) (D) g1 = 0000
D 0001

c (4) (D)

k0 = 3, n0 = 4, ν = 1
(c)

Figure 11.13 Examples of encoders for three convolutional codes.

584 Chapter 11. Channel codes

0 0 0 0 0
(d) = (0,0)
2 2 2 2 2
2 2 2
0 0 0 (c) =(1,0)

1 1 1 1
(b) =(0,1)
1 1 1
(a) =(1,1)
1 1 1

Figure 11.14 Trellis diagram of the code of Example 11.3.1; the labels represent the Hamming
weight of the output bits.

D
(1,1)

D D D
(1,0) (0,1)
1
D2 D2
(0,0)

Figure 11.15 State diagram of the code of Example 11.3.1; the labels represent the Hamming
weight of the generated bits.

D
(1,1)

δ
D D

α D2 β D γ D2 η
(0,0) (0,0)
(1,0) (0,1)
1

Figure 11.16 State diagram of the code of Example 11.3.1; node (0,0) is split to compute the
transfer function of the code.

to 1. The augmented transfer function is given by

D5 L3 I
t(D, L, I) =
1 − DL(1 + L)I
= D5 L3 I + D6 L4 (1 + L)I 2 + D7 L5 (1 + L)2 I 3 + · · · (11.269)
+ D5+i L3+i (1 + L)i I 1+i + · · ·
Thus, we see that the code word of weight 5 is of length 3 and is originated by a sequence of informa-
tion bits that contains one bit equal to 1, there are two code words of weight 6, one of length 4 and the
11.3. Convolutional codes 585

DLI
(1,1)

δ
DLI DL

α D2LI β DL γ D2L η
(0,0) (0,0)
(1,0) (0,1)
D0LI

Figure 11.17 State diagram of the code of Example 36; node (0,0) is split to compute the
augmented transfer function.

other of length 5, both of which are originated by a sequence of information bits that contain two bits
equal to 1, . . . .

Catastrophic error propagation

For certain codes, a finite number of channel errors may lead to an infinite number of errors in the
sequence of decoded bits. For example, consider the code with encoder and state diagram illustrated
in Figure 11.18a,b, respectively. Note that in the state diagram, the self loop at state (1,1) does not
increment the weight of the code word so that a code word corresponding to a path passing through the
states (0,0),(1,0),(1,1),(1,1), …,(1,1),(0,1),(0,0) is of weight 6, independently of the number of times it
passes through the self-loop at state (1,1). In other words, long sequences of coded bits equal to zero
may be obtained by remaining in the state (0, 0) with a sequence of information bits equal to zero, or by

g(1,1) (D) = 1+ D
D D
g(2,1) (D) = 1+ D2

(a)

1/00
(1,1)

δ
1/01 0/11

α 1/11 β 0/10 γ 0/01 η

(0,0) (0,0)
(1,0) (0,1)
1/10
(b)

Figure 11.18 (a) Encoder and (b) state diagram for a catastrophic convolutional code.
586 Chapter 11. Channel codes

remaining in the state (1, 1) with a sequence of information bits equal to one. Therefore, a limited number
of channel errors, in this case 6, can cause a large number of errors in the sequence of decoded bits.

Definition 11.15
A convolutional code is catastrophic if there exists a closed loop in the state diagram that has all branches
with zero weight. ◽

For codes with rate 1∕n0 , it has been shown that a code is catastrophic if and only if all generator
polynomials have a common polynomial factor. In the above example, the common factor is 1 + D. This
can be proven using the following argument: suppose that g(1,1) (D), g(2,1) (D), … , g(k0 ,1) (D) all have the
common factor gc (D), so that
g(i,1) (D) = gc (D) g̃ (i,1) (D) (11.270)
Suppose the all zero sequence is sent, b(1) (D) = 0 and that the finite error sequence g̃ (i,1) (D), equal
to that defined in (11.270), occurs in the ith subsequence output, for i = 1, 2, … , n0 , as illustrated in
Figure 11.19a. The same output sequence is obtained if the sequence of information bits with infinite
length b(1) (D) = 1∕gc (D) is sent, and no channel errors occur, as illustrated in Figure 11.19b. Thus, a
finite number of errors yields a decoded sequence of information bits that differ from the transmitted
sequence in an infinite number of positions.

~g (1,1) (D)

(1,1) ~g (1,1) (D)

g (D)

b (1)(D) = 0 ~g ( n0 ,1)(D)

( n0 ,1) ~g ( n0 ,1)(D)
g (D)

(a)

(1,1) ~g (1,1) (D)

g (D)

1
b (1)(D) =
gc (D)
( n0 ,1) ~ ( n0 ,1) (D)
g (D) g

(b)

Figure 11.19 Two distinct infinite sequences of information bits that produce the same output
sequence with a finite number of errors.

11.3.2 Decoding of convolutional codes

Various algorithms have been developed for the decoding of convolutional codes. One of the first decod-
ing methods was algebraic decoding, which is similar to the methods developed for the decoding of
block codes. However, this method has the disadvantages that it is applicable only to a limited number
11.3. Convolutional codes 587

of codes having particular characteristics and exhibits performance that is lower as compared to decod-
ing methods based on the observation of the whole received sequence. The latter methods, also called
probabilistic decoding methods, include the Viterbi algorithm (VA), the sequential decoding algorithm
by Fano [9], and the forward–backward algorithm (FBA) by Bahl–Cocke–Jelinek–Raviv (BCJR).
Before illustrating the various decoding methods, we consider an important function.

Interleaving
The majority of block codes as well as convolutional codes is designed by assuming that the errors
introduced by the noisy channel are statistically independent. This assumption is not always true in
practice. To make the channel errors, at least approximately, statistically independent it is customary to
resort to an interleaver, as described in Section 6.1.

Two decoding models

We consider a binary convolutional code with k0 = 1, n0 = 2, and constraint length 𝜈. In general, from
(11.254), we write the code sequence as a function of the message sequence as
c(1)
k
= g(1,1) (bk , … , bk−𝜈 )
(11.271)
c(2)
k
= g(2,1) (bk , … , bk−𝜈 )
At the receiver, two models may be adopted.

Model with hard input. With reference to the transmission system of Figure 6.1, we consider the
sequence at the output of the binary channel. In this case, the demodulator has already detected the
transmitted symbols, for example by a threshold detector, and the inverse bit mapper provides the binary
sequence {zm = c̃ m } to the decoder, from which we obtain the interlaced binary sequences

z(1)
k
= z2k = c(1)
k
⊕ e(1)
k
(11.272)
z(2)
k
= z2k+1 = c(2)
k
⊕ e(2)
k

where the errors e(i)

k
∈ {0, 1}, for a memoryless binary symmetric channel, are i.i.d. (see (6.73)).
From the description on page 579, introducing the state of the encoder at instant k as the vector with
𝜈 elements
sk = [bk , … , bk−(𝜈−1) ] (11.273)

the desired sequence in (11.272), that coincides with the encoder output, can be written as (see (11.271)
and (11.273))
c(1)
k
= f (1) (sk , sk−1 )
(11.274)
c(2)
k
= f (2) (sk , sk−1 )

Model with soft input. Again with reference to Figure 6.1, at the decision point of the receiver, the
signal can be written as (see (7.334))
zk = uk + 𝑤k (11.275)

where we assume 𝑤k is white Gaussian noise with variance 𝜎𝑤2 = 2𝜎I2 , and uk is given by (7.335)

∑
L2
uk = 𝜂n ak−n (11.276)
n=−L1
588 Chapter 11. Channel codes

where {ak } is the sequence of symbols at the output of the bit mapper. Note that in (11.276), the symbols
{ak } are in general not independent, as the input of the bit mapper is a code sequence according to the
law (11.274).
The relation between uk and the bits {b𝓁 } depends on the intersymbol interference in (11.276), the type
of bit mapper and the encoder (11.271). We consider the case of absence of inter-symbol interference
(ISI), that is (see also (6.13))
u k = ak (11.277)

and a 16-pulse-amplitude-modulation (PAM) system where, without interleaving, four consecutive code
bits c(1)
2k
, c(2)
2k
, c(1) , c(2) are mapped into a symbol of the constellation. For an encoder with constraint
2k−1 2k−1
length 𝜈, we have

uk = f̃ (ak ) = f̃ [BMAP {[c(1)

2k
, c(2)
2k
, c(1) , c(2) ]}]
2k−1 2k−1
= f̃ [BMAP {g(1,1) (b2k , … , b2k−𝜈 ), g(2,1) (b2k , … , b2k−𝜈 ), (11.278)
g(1,1) (b2k−1 , … , b2k−1−𝜈 ), g(2,1) (b2k−1 , … , b2k−1−𝜈 )}]

In other words, let

sk = [b2k , … , b2k−𝜈+1 ] (11.279)

we can write
uk = f (sk , sk−1 ) (11.280)

We observe that in this example each state of the trellis admits four possible transitions. As we
will see in Chapter 12, better performance is obtained by jointly optimizing the encoder and the
bit mapper.

Decoding by the Viterbi algorithm

The VA, described in Section 7.5.1, is a probabilistic decoding method that implements the
maximum-likelihood (ML) criterion, which minimizes the probability of detecting a sequence that is
different from the transmitted sequence.

VA with hard input. The trellis diagram is obtained by using the definition (11.273), and the branch
metric is the Hamming distance between zk = [z(1)
k
, z(2)
k
]T and ck = [c(1)
k
, ck(2) ]T , (see Definition 11.1),

dH (zk , ck ) = number of positions where zk differs from ck , (11.281)

where ck is generated according to the rule (11.274).

VA with soft input. The trellis diagram is now obtained by using the definition (11.279), and the branch
metric is the Euclidean distance between zk and uk ,

|zk − uk |2 (11.282)

where uk , in the case of the previous example of absence of ISI and 16-PAM transmission, is given by
(11.280).
As an alternative to the VA, we can use the forward–backward algorithm (FBA) of Section 7.5.2.
11.3. Convolutional codes 589

Decoding by the forward-backward algorithm

The previous approach, which considers joint detection in the presence of ISI and convolutional decod-
ing, requires a computational complexity that in many applications may turn out to be exceedingly large.
In fact the state (11.279), that takes into account both encoding and the presence of ISI, usually is diffi-
cult to define and is composed of several bits of the sequence {b𝓁 }. An approximate solution is obtained
by considering the detection and the decoding problems separately, however, assuming that the detector
passes the soft information on the detected bits to the decoder. This is also the principle of bit-interleaved
coded modulation [10].

Soft output detection by FBA. For a channel model at detection point with ISI, as by (11.276), extrac-
tion of soft information (through log-likelihood ratios, LLRs) associated to code bits follows the deriva-
tion of (7.436) where we use the FBA of page 348 to extract the state metrics Vk (i), i = 1, … , Ns . In
the particular case of no ISI, extraction of LLRs associated to code bits is simplified, as outlined in
Section 6.2.2.
An alternative to the FBA is obtained by modifying the VA to yield a soft output Viterbi algorithm
(SOVA), as discussed in the next section.

Convolutional decoding with soft input The decoder for the convolutional code typically uses the VA
with branch metric (associated with a cost function to be minimized) given by

∥ 𝓵 (in)
k
− c k ∥2 (11.283)

where ck is given by (11.274) for a code with n0 = 2, and 𝓵 (in)

k
= [𝓁k(in,1) , 𝓁k(in,2) ] are the LLRs associated,
(1) (2) (j) 2
respectively, with ck and ck . As |ck | = 1, (11.283) can be rewritten as

(𝓁k(in,1) − c(1)
k
)2 +(𝓁k(in,2) − c(2)
k
)2
(11.284)
= (𝓁k ) + (𝓁k(in,2) )2 + 2 − 2c(1)
(in,1) 2
k
𝓁k(in,1) − 2c(2)
k
𝓁k(in,2)
Leaving out the terms that do not depend on ck and extending the formulation to a convolutional code
with rate k0 ∕n0 , the branch metric (associated with a cost function to be maximized) is expressed as (see
also Ref. [11])
∑
n0
(j) (in,j)
2 ck 𝓁k (11.285)
j=1

where the factor 2 can be omitted.

Observation 11.3
As we have previously stated, best system performance is obtained by jointly designing the encoder and
the bit mapper. However, in some systems, typically radio, an interleaver is used between the encoder
and the bit mapper. In this case, joint detection and decoding are impossible to implement in practice.
Detection with soft output followed by decoding with soft input remains a valid approach, obviously
after re-ordering the LLR as determined by the deinterleaver.

In applications that require a soft output (see Section 11.7), the decoder, that is called in this case
soft-input soft-output, can use one of the versions of the FBA or the SOVA. 6

6
An extension of soft-input soft-output decoders for the decoding of block codes is found in [12, 13].
590 Chapter 11. Channel codes

Sequential decoding
Sequential decoding of convolutional codes represented the first practical algorithm for ML decoding.
It has been employed, for example, for the decoding of signals transmitted by deep-space probes, such
as the Pioneer, 1968 [14]. There exist several variants of sequential decoding algorithms that are charac-
terized by the search of the optimum path in a tree diagram (see Figure 11.9b), instead of along a trellis
diagram, as considered, e.g. by the VA.
Sequential decoding is an attractive technique for the decoding of convolutional codes and trellis
codes if the number of states of the encoder is large [15]. In fact, as the implementation complexity of
ML decoders such as the Viterbi decoder grows exponentially with the constraint length of the code,
𝜈, the complexity of sequential decoding algorithms is essentially independent of 𝜈. On the other hand,
sequential decoding presents the drawback that the number of computations Nop required for the decod-
ing process to advance by one branch in the decoder tree is a random variable with a Pareto distribution,
i.e.
P[Nop > N] = AN −𝜌 (11.286)
where A and 𝜌 are constants that depend on the channel characteristics and on the specific code and
the specific version of sequential decoding used. Real-time applications of sequential decoders require
buffering of the received samples. As practical sequential decoders can perform only a finite num-
ber of operations in a given time interval, re-synchronization of the decoder must take place if the
maximum number of operations that is allowed for decoding without incurring buffer saturation is
exceeded.
Sequential decoders exhibit very good performance, with a reduced complexity as compared to the
VA, if the constraint length of the code and the SNR are sufficiently large. Otherwise, the average number
of operations required to produce one symbol at the decoder output is very large.

The Fano Algorithm In this section, we consider sequential decoding of trellis codes, a class of codes
that will be studied in detail in Chapter 12. However, the algorithm can be readily extended to convo-
(k )
lutional codes. At instant k, the k0 information bits bk = [b(1)
k
, … , bk 0 ] are input to a rate k0 ∕(k0 + 1)
(k )
convolutional encoder with constraint length 𝜈 that outputs the coded bits ck = [c(0)k
, … , ck 0 ]. The k0 + 1
coded bits select from a constellation with M = 2k0 +1 elements a symbol ak , which is transmitted over
an additive white Gaussian noise (AWGN) channel. Note that the encoder tree diagram has 2k0 branches
that correspond to the values of bk stemming from each node. Each branch is labelled by the symbol ak
selected by the vector ck . The received signal is given by (see (11.275))
zk = ak + 𝑤k (11.287)
The received signal sequence is input to a sequential decoder. Using the notation of Section 7.5.1, in
the absence of ISI and assuming a = 𝜶, the ML metric to be maximized can be written as [9]

⎡ ⎤
∑⎢K−1
Pzk |ak (𝜌k |𝛼k ) ⎥
Γ(𝜶) = ⎢log2 ∑ − B⎥ (11.288)
k=0 ⎢
P (𝜌 |𝛼 )P
zk |ak k i ak i (𝛼 ) ⎥
⎣ 𝛼i ∈ ⎦
where B is a suitable constant that determines a trade-off between computational complexity and per-
formance and is related to the denominator in (11.288). Choosing B = k0 and Pak (𝛼i ) = M1 = 2−(k0 +1) ,
11.3. Convolutional codes 591

𝛼i ∈ , we obtain
⎡ ⎤
K−1 ⎢ ⎥
|𝜌k −𝛼k |2
−
∑⎢ e 2𝜎 2
I ⎥
Γ(𝜶) = ⎢log2 |𝜌 −𝛼 |2
+ 1⎥ (11.289)
k=0 ⎢ ∑ − k2𝜎2i ⎥
⎢ e I
⎥
⎣ 𝛼i ∈ ⎦
Various algorithms have been proposed for sequential decoding [16–18]. We will restrict our attention
here to the Fano algorithm [9, 15].
The Fano algorithm examines only one path of the decoder tree at any time using the metric in
(11.288). The considered path extends to a certain node in the tree and corresponds to a segment of
the entire code sequence 𝜶, up to symbol 𝛼k .
Three types of moves between consecutive nodes on the decoder tree are allowed: forward, lateral,
and backward. On a forward move, the decoder goes one branch to the right in the decoder tree from the
previously hypothesized node. This corresponds to the insertion of a new symbol 𝛼k+1 in (11.288). On
a lateral move, the decoder goes from a path on the tree to another path differing only in the last branch.
This corresponds to the selection of a different symbol 𝛼k in (11.288). The ordering among the nodes
is arbitrary, and a lateral move takes place to the next node in order after the current one. A backward
move is a move one branch to the left on the tree. This corresponds to the removal of the symbol 𝛼k from
(11.288).
To determine which move needs to be made after reaching a certain node, it is necessary to compute
the metric Γk of the current node being hypothesized and consider the value of the metric Γk−1 of the
node one branch to the left of the current node, as well as the current value of a threshold Th, which
can assume values that are multiple of a given constant Δ. The transition diagram describing the Fano
algorithm is illustrated in Figure 11.20. Typically, Δ assumes values that are of the order of the minimum
distance between symbols.
As already mentioned, real-time applications of sequential decoding require buffering of the input
samples with a buffer of size S. Furthermore, the depth of backward search is also finite and is usually
chosen at least five times the constraint length of the code. To avoid erasures of output symbols in case
of buffer saturation, in [19] a buffer looking algorithm (BLA) is proposed. The buffer is divided into L
sections, each with size Sj , j = 1, … , L. A conventional sequential decoder (primary decoder) and L − 1
secondary decoders are used. The secondary decoders employ fast algorithms, such as the M-algorithm
[20], or variations of the Fano algorithm that are obtained by changing the value of the bias B in the
metric (11.288).

Example 11.3.2 (Sequential decoding of a 512-state 16-PAM trellis code)

We illustrate sequential decoding with reference to a 512-state 16-PAM trellis code specified for
single-line high-bit-rate digital subscriber line (SHDSL) transmission (see Section 18.6). The encoder
for this trellis code comprises a rate 1∕2 non-systematic non-recursive convolutional encoder with
constraint length 𝜈 = 9 and a bit mapper as specified in Figure 11.21. The symbol error probabilities
versus SNR Γ obtained by sequential decoding with infinite buffer size and depth of backward search
of 64 and 128 symbols, and by a 512-state VA decoding with length of the path memory of 64 and 128
symbols are shown in Figure 11.22. Also shown for comparison are the error probabilities obtained for
uncoded 8-PAM and 16-PAM transmission.
592 Chapter 11. Channel codes

start
Th = 0
Γ0 = 0, Γ–1 = –∞

forward move

Γk ≥ Th ? no
lateral move

yes
lateral yes
move
yes possible?
end of stop
tree? no
no
backward move

no Γk–1 <
Th +Δ?
yes Γk–1 ≥
yes Th ?
Th ← Th + j Δ, j: no
Γk – Δ < Th + j Δ ≤ Γk
Th ← Th – Δ

Figure 11.20 Transition diagram of the Fano algorithm.

11.3.3 Performance of convolutional codes

H
For binary convolutional codes with free distance dfree and bit error probability of the channel equal to
Pbit , decoding with hard input yields
H
P(dec)
bit
≃ A 2−dfree Pbit (11.290)

and decoding with soft input, for a system with antipodal signals, yields
√
⎛ Rc dH 2Eb ⎞
≃ A Q⎜ ⎟
free
P(dec) (11.291)
bit ⎜ N0 ⎟
⎝ ⎠
where A is a constant [21].
In particular, we consider BPSK transmission over an ideal AWGN channel. Assuming an encoder
with rate Rc = 1∕2 and constraint length 𝜈 = 6, the coding gain for a soft Viterbi decoder is about 3.5 dB
for Pbit = 10−3 ; it becomes about 4.6 dB for Pbit = 10−5 . Note that a soft decoder provides a gain of about
2.4 dB with respect to a hard decoder, for Pbit < 10−3 .
11.5. Concatenated codes 593

(3) (3)
bk ck
(2) (2)
bk ck
ak
(1)
ck bit mapper
(1)
bk rate 1/2
(0)
convolutional ck
encoder

(a)

(0)
ck

(1)
bk
T T T T T T T T T

(1)
ck

(b)

–15 –13 –11 –9 –7 –5 –3 –1 1 3 5 7 9 11 13 15 ak

(0)
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 ck
(1)
0 0 1 1 0 0 1 1 0 0 1 1 0 0 1 1 ck
(2)
0 0 0 0 1 1 1 1 1 1 1 1 0 0 0 0 ck
0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 (3)
ck
(c)

Figure 11.21 (a) Block diagram of the encoder and bit mapper for a trellis code for 16-PAM
transmission, (b) structure of the rate-1/2 convolutional encoder, and (c) bit mapping for the
16-PAM format.

11.4 Puncturing

This operation takes place after the encoder, before the interleaver. By puncturing we omit to transmit
some code bits, generated by (11.254). Obviously, these positions are known at the receiver, which
inserts a LLR equal to zero before decoding. Hence, puncturing provides a higher code rate without
modifying substantially the encoder and the decoder.

11.5 Concatenated codes

Concatenated coding is usually introduced to achieve an improved error correction capability [22]. Inter-
leaving is also commonly used in concatenated coding schemes, as illustrated in Figure 11.23 so that the
decoding processes of the two codes (inner and outer) can be considered approximately independent.
The first decoding stage is generally utilized to produce soft decisions on the information bits that are
passed to the second decoding stage. While the FBA directly provides a soft output, the VA must be
slightly modified.

The soft-output Viterbi algorithm

We have seen that the FBA in the original maximum-a-posteriori (MAP) version directly yields a soft
output, at the expense of a large computational complexity. The Max-Log-MAP criterion has a reduced
594 Chapter 11. Channel codes

100

10−1 16−PAM, uncoded

10−2

10−3
8−PAM, uncoded
Pe

10−4

10−5

10−6 SD, depth=64

SD, depth=128
VA, path mem.= 64
VA, path mem.= 128
10−7
16 18 20 22 24 26 28
Г (dB)

Figure 11.22 Symbol error probabilities for the 512-state 16-PAM trellis code with sequential
decoding (depth of search limited to 64 or 128) and 512-state Viterbi decoding (length of path
memory limited to 64 or 128). Symbol error probabilities for uncoded 8-PAM and 16-PAM
transmission are also shown.

outer code inner code

encoder encoder
interleaver channel
1 2

decoder decoder
deinterleaver
1 2
outer code inner code

Figure 11.23 Transmission scheme with concatenated codes and interleaver.

11.5. Concatenated codes 595

complexity, but there remains the problem of performing the two procedures, forward and backward.
We now illustrate how to modify the VA to obtain a soft output equivalent to that produced by the
Max-Log-MAP.
In this section, we consider the different methods to generate the soft output.

The difference metric algorithm (DMA). Figure 11.24 shows a section of a trellis diagram with four
states, where we assume sk = (bk , bk−1 ). We consider two states at instant k − 1 that differ in the least
significant bit bk−2 of the binary representation, that is s(0)
k−1
= (00) and s(1)
k−1
= (01). A transition from
each of these two states to state sk = (00) at instant k is allowed. According to the VA, we choose the
survivor sequence minimizing the metric
s(i) →sk
{Γk−1 (s(i)
k−1
) + 𝛾k k−1 } (11.292)

where Γk−1 (s(i)

k−1
) is the path metric associated with the survivor sequence up to state s(i)
k−1
at instant
(i)
sk−1 →sk (i)
k − 1, and 𝛾k denotes the branch metric associated with the transition from state sk−1 to state sk .
Let Ak = Γk−1 (00) + 𝛾k00→00 and Bk = Γk−1 (01) + 𝛾k01→00 , then we choose the upper or the lower transi-
tion according to whether Δk = Ak − Bk is smaller or larger than zero, respectively. Note that |Δk | is a
reliability measure of the selection of a certain sequence as survivor sequence.
In other words, if |Δk | is small, there is a non-negligible probability that the bit b′k−2 associated with
the transition from state s(i)
k−1
to sk on the survivor sequence is in error. The difference |Δk | = 𝜆k yields
the value of the soft decision for bk−2 , in case the final sequence chosen by the VA includes the state
sk ; conversely, this information is disregarded. Thus, the difference metric algorithm (DMA) can be
formulated as follows:
For each state sk of the trellis diagram at instant k, the metric Γk (sk ) and the most recent (Kd + 1) bits
of the survivor sequence b′ (sk ) = {b′k , … , b′k−K } are stored, where Kd denotes the path memory depth of
d
the VA. Furthermore, the reliability measures 𝝀(sk ) = {𝜆k , … , 𝜆k−Kd } associated with the bits b′ (sk ) are
also stored. Interpreting bk and b̂ k as binary symbols in the alphabet {−1, 1}, the soft output associated
with bk is given by
𝓁̃k = b̂ k 𝜆k+2 (11.293)
where {b̂ k } is the sequence of information bits associated with the ML sequence.

bk –2 = 0
sk– 4
0 0 0 0
(00) λk = |Δk|
(0) (1)
λk–4
(0) (0)
λk– 2 path 0 (0)
λk –1
λk–4 = λk– 3 1
bk– 2 = 1
(01) 0
(1)
λk– 1

1
(10)
(1)
0 path 1
λk –3

(11)
(1)
λk–2

k–4 k–3 k–2 k–1 k

Figure 11.24 Soft-output VA.

596 Chapter 11. Channel codes

The soft-output VA (SOVA). As for the DMA, the SOVA determines the difference between the met-
rics of the survivor sequences on the paths that converge to each state of the trellis and updates at
every instant k the reliability information 𝝀(sk ) for each state of the trellis. To perform this update, the
sequences on the paths that converge to a certain state are compared to identify the positions at which
the information bits of the two sequences differ. With reference to Figure 11.24, we denote the two paths
that converge to the state (00) at instant k as path 0 and path 1. Without loss of generality, we assume
that the sequence associated with path 0 is the survivor sequence, and thus the sequence with the smaller
cost; furthermore, we define 𝝀(s(0) k
) = {𝜆(0)
k
, … , 𝜆(0)
k−Kd
} and 𝝀(s(1)
k
) = {𝜆(1)
k
, … , 𝜆(1)
k−Kd
} as the two relia-
bility vectors associated with the information bits of two sequences. If one information bit along path 0
differs from the corresponding information bit along path 1, then its reliability is updated according to
the rule, for i = k − Kd , … , k − 1,

𝜆i = min (|Δk |, 𝜆(0)

i
) if b(0)
i−2
≠ b(1)
i−2
(11.294)

With reference to Figure 11.24, the two sequences on path 0 and on path 1 diverge from state sk = (00)
at instant k − 4. The two sequences differ in the associated information bits at the instants k and k − 1;
therefore, only 𝜆k−1 will be updated.

Modified SOVA. In the modified version of the SOVA, the reliability of an information bit along
the survivor path is also updated if the information bit is the same, according to the rule, for i = k −
Kd , … , k − 1
{
min (|Δk |, 𝜆(0) ) if b(0) ≠ b(1)
𝜆i = i i−2 i−2
(11.295)
min (|Δk | + 𝜆i , 𝜆i ) if bi−2 = b(1)
(1) (0) (0)
i−2

Note that (11.294) is still used to update the reliability if the information bits differ; this version of the
SOVA gives a better estimate of 𝜆i . As proven in [23], if the VA is used as decoder, the modified SOVA
is equivalent to Max–Log–MAP decoding. An example of how the modified SOVA works is illustrated
in Figure 11.25.

(1) (0)
λk–3 = min(|Δk| + λk –3 , λk – 3)
(1) (0)
λk –2 = min(|Δk| + λk – 2 , λk –2)
(0)
λk –1 = min(|Δk| , λk – 1 )
sk– 4
0 0 0 0
(00) λk = |Δk|
(0) (1)
λk–4
(0) (0)
λk– 2 path 0 (0)
λk– 1
λk– 4 = λk –3 1

(01) 0
(1)
λk –1

1
(10)
(1)
0 path 1
λk –3

(11)
(1)
λk– 2

k–4 k–3 k–2 k–1 k

Figure 11.25 Modified soft-output VA.

11.6. Turbo codes 597

11.6 Turbo codes

Turbo codes, proposed in 1993 by Berrou et al. [24, 25], are an evolution of concatenated codes, in the
form of parallel concatenated convolutional codes (PCCC), and allow reliable transmission of informa-
tion at rates near the Shannon limit [24–26]. As will be discussed in this section, the term turbo, even
though it is used to qualify these codes, is rather tied to the decoder, whose principle is reminiscent of
that of turbo engines.

Encoding
For the description of turbo codes, we refer to the first code of this class that appeared in the scientific
literature [24, 25]. A sequence of information bits is encoded by a simple recursive systematic convolu-
tional (RSC) binary encoder with code rate 1∕2, to produce a first sequence of parity bits, as illustrated
in Figure 11.26. The same sequence of information bits is permuted by a long interleaver and then
encoded by a second recursive systematic convolutional encoder with code rate 1∕2 to produce a second
sequence of parity bits. Then the sequence of information bits and the two sequences of parity bits are
transmitted. Note that the resulting code has rate Rc = 1∕3. Higher code rates Rc are obtained by trans-
mitting only some of the parity bits (puncturing). For example, for the turbo code in [24, 25], a code
rate equal to 1/2 is obtained by transmitting only the bits of the parity sequence 1 with odd indices
and the bits of the parity sequence 2 with even indices. A specific example of turbo encoder is reported
in Figure 11.27.
The exceptional performance of turbo codes is due to one particular characteristic. We can think of a
turbo code as being a block code for which an input word has a length equal to the interleaver length,
and a code word is generated by initializing to zero the memory elements of the convolutional encoders
before the arrival of each input word. This block code has a group structure. As for the usual block codes,
the asymptotic performance, for large values of the SNR, is determined by the code words of minimum
weight and by their number. For low values of the SNR, also the code words of non-minimum weight
and their multiplicity need to be taken into account. Before the introduction of turbo codes, the focus
on designing codes was mainly on asymptotic performance, and thus on maximizing the minimum

sequence of information bits

systematic sequence 1
recursive of parity bits
convolutional
encoder 1, rate 1/2

interleaver

systematic sequence 2
recursive of parity bits
convolutional
encoder 2, rate 1/2

Figure 11.26 Encoder of a turbo code with code rate Rc = 13 .

598 Chapter 11. Channel codes

bk
ck(1)

ck(2)

D D D

interleaver

ck(3)

D D D

Figure 11.27 Turbo encoder adopted by the UMTS standard.

distance. With turbo codes, the approach is different. Because of the large ensemble of code words,
the performance curve, in terms of bit error probability as a function of the SNR, rapidly decreases
for low values of the SNR. For Pbit lower than 10−5 , where performance is determined by the minimum
distance between code words, the bit error probability curve usually exhibits a reduction in the value
of slope.
The two encoders that compose the scheme of Figure 11.26 are called component encoders, and they
are usually identical. As mentioned above, Berrou and Glavieux proposed two recursive systematic con-
volutional encoders as component encoders. Later, it was shown that it is not necessary to use systematic
encoders [21, 27].
Recursive convolutional codes are characterized by the property that the code bits at a given instant
do not depend only on the information bits at the present instant and the previous 𝜈 instants, where 𝜈
is the constraint length of the code, but on all the previous information bits, as the encoder exhibits a
structure with feedback.
Starting from a non-recursive, non-systematic convolutional encoder for a code with rate 1∕n0 , it is
possible to obtain in a very simple way a recursive systematic encoder for a code with the same rate
H
and the same code words, and hence, with the same free distance dfree . Obviously, for a given input
word, the output code words will be different in the two cases. Consider, for example a non-recursive,
non-systematic convolutional encoder for a code with code rate 1∕2. The code bits can be expressed
in terms of the information bits as (see (11.254))
′
c (1) (D) = g(1,1) (D) b(D)
′ (11.296)
c (2) (D) = g(2,1) (D) b(D)

The corresponding recursive systematic encoder is obtained by dividing the polynomials in (11.296)
by g(1,1) (D) and implementing the functions

c(1) (D) = b(D) (11.297)

g(2,1) (D)
c(2) (D) = b(D) (11.298)
g(1,1) (D)
11.6. Turbo codes 599

Let us define
c(2) (D) b(D)
d(D) = = (11.299)
g(2,1) (D) g(1,1) (D)
then the code bits can be expressed as a function of the information bits and the bits of the sequence
{dk } as
c(1)
k
= bk (11.300)
𝜈
∑
c(2)
k
= g(2,1)
i
dk−i (11.301)
i=0

where, using the fact that g(1,1)

0
(D) = 1, from (11.299), we get
𝜈
∑
d k = bk + g(1,1)
i
dk−i (11.302)
i=1

We recall that the operations in the above equations are in GF(2). Another recursive systematic encoder
that generates a code with the same free distance is obtained by exchanging the role of the polynomials
g(1,1) (D) and g(2,1) (D) in the above equations.
One recursive systematic encoder corresponding to the non-recursive, non-systematic encoder
of Figure 11.9a is illustrated in Figure 11.28.
The 16-state component encoder for a code with code rate 1∕2 used in the turbo code of Berrou
and Glavieux [24, 25], is shown in Figure 11.29. The encoder in Figure 11.27, with an 8-state com-
ponent encoder for a code with code rate 1/2, is adopted in the standard for third-generation universal
mobile telecommunications systems (UMTS) [28]. Turbo codes are also used in digital video broadcast-
ing (DVB) [29] standards and in space telemetry applications as defined by the Consultative Committee
for Space Data Systems (CCSDS) [30]. In [31] are listed generator polynomials of recursive systematic
convolutional encoders for codes with rates 1/2, 1/3, 1/4, 2/3, 3/4, 4/5, and 2/4, that can be used for the
construction of turbo codes.
Another fundamental component in the structure of turbo codes is represented by a non-uniform inter-
leaver. We recall that a uniform7 interleaver, as that described in Section 11.3.2, operates by writing input
bits in a matrix by rows and reading them by columns. In practice, a non-uniform interleaver determines
a permutation of the sequence of input bits so that adjacent bits in the input sequence are separated, after
the permutation, by a number of bits that varies with the position of the bits in the input sequence. The

ck(1)

bk dk dk−1 dk−2
D D

ck(2)

Figure 11.28 Recursive systematic encoder that generates a code with the same free distance
as the non-recursive, non-systematic encoder of Figure 11.9a.

7
The adjective uniform, referred to an interleaver, is used with a different meaning in [27].
600 Chapter 11. Channel codes

c (1)
k

bk
D D D D

c k(2)

Figure 11.29 A 16-state component encoder for the turbo code of Berrou and Glavieux.

interleaver determines directly the minimum distance of the code and therefore, performance for high
values of the SNR. Nevertheless, the choice of the interleaver is not critical for low values of the SNR.
Beginning with the interleaver originally proposed in [24, 25], various interleavers have since been
proposed (see Ref. [32] and references contained therein).
One of the interleavers that yields better performance is the so-called spread interleaver [33]. Consider
a block of M1 input bits. The integer numbers that indicate the position of these bits after the permuta-
tion are randomly generated with the following constraint: each integer randomly generated is compared
with the S1 integers previously generated; if the distance from them is shorter than a prefixed thresh-
old S2 , the generated integer is discarded and another one is generated until the condition is satisfied.
The two parameters S1 and S2 must be larger than the memory of the two-component encoders. If
the two-component encoders are equal, it is convenient to choose S1 = S2 . The computation time needed
to generate the interleaver increases with S1 and S2 , and there is no guarantee that the procedure termi-
nates successfully.
√ Empirically, it has been verified that, choosing both S1 and S2 equal to the closest
integer to M1 ∕2, it is possible to generate the interleaver in a finite number of steps.
Many variations of the basic idea of turbo codes have been proposed. For example, codes generated
by serial concatenation of two convolutional encoders, connected by means of a non-uniform inter-
leaver [34]. Parallel and serial concatenation schemes were then extended to the case of multi-level
constellations to obtain coded modulation schemes with high spectral efficiency (see [35] and references
therein).

The basic principle of iterative decoding

The presence of the interleaver in the scheme of Figure 11.26 makes an encoder for a turbo code
have a very large memory even if very simple component encoders are used. Therefore, the optimum
maximum-likelihood sequence detection (MLSD) decoder would require a Viterbi decoder with an
exceedingly large number of states and it would not be realizable in practice. For this reason, we resort to
a suboptimum iterative decoding scheme with a much lower complexity, that however, as it was verified
empirically, exhibits near optimum performance [27]. The decoder for the turbo encoder of Figure 11.26
is illustrated in Figure 11.30. The received sequences corresponding to the sequence of information bits
and the first sequence of parity bits are decoded using a soft input soft output decoder, corresponding to
the first convolutional encoder. Thus, this decoder provides a soft decision for each information bit; these
soft decisions are then used by a second decoder corresponding to the second convolutional encoder,
together with the received sequence corresponding to the second sequence of parity bits. Soft decisions
so obtained are taken back to the input of the first decoder for a new iteration, where the additional
11.6. Turbo codes 601

(a)
ℓk,1 (ext) (a)
ℓk,1 ℓn,2
(in,s) interleaver (in,s) (ext) (ext)
ℓk DEC1 ℓn ℓn,2 ℓk,2
(in,p) interleaver (in,p)
DEC2 deinterleaver
ℓk,1 ℓn,2

(in,p)
ℓk,2

ℓk,overall

b^ k

last iteration

Figure 11.30 Principle of the decoder for a turbo code with rate 1∕3.

information obtained by the second decoder is used to produce more reliable soft decisions. The proce-
dure continues iteratively for about 10–20 cycles until final decisions are made on the information bits.
The two component decoders of the scheme in Figure 11.30 are soft input, soft output decoders that
produce estimates on the reliability of the decisions. Therefore, they implement the SOVA or the FBA
(or one of its simplified realizations). The basic principle of iterative decoding is the following: each
component decoder uses the hints of the other to produce more reliable decisions. In the next sections, we
will see in detail how this is achieved, and in particular how the reliability of the decisions is determined.
The algorithms for iterative decoding introduced with the turbo codes were also immediately applied
in wider contexts. In fact, this iterative procedure may be used every time the transmission system
includes multiple processing elements with memory interconnected by an interleaver. Iterative decod-
ing procedures may be used, for example for detection in the presence of intersymbol interference, also
called turbo equalization or turbo detection [36] (see Section 11.7), for non-coherent decoding [37, 38],
and for joint detection and decoding in the case of transmission over channels with fading [39].
Before discussing in detail iterative decoding, it is useful to revisit the FBA.

FBA revisited
The formulation of the FBA presented here is useful for the decoding of recursive systematic convolu-
tional codes [40].
We consider a binary recursive systematic convolutional encoder for a code with rate k0 ∕n0 and
constraint length 𝜈. Let the encoder input be given by a sequence of K vectors, each composed of k0
binary components. As described on page 579, each information vector to be encoded is denoted by (see
(11.252))
(k )
bk = [bk(1) , b(2)
k
, … , bk 0 ], b(i)
k
∈ {−1, 1}, k = 0, 1, … K − 1 (11.303)
where k0 can be seen either as the number of encoder inputs or as the length of an information vector.
As the convolutional encoder is systematic, at instant k the state of the convolutional encoder is given
602 Chapter 11. Channel codes

by the vector (see extension of (11.302))

(k +1) (k +1) (k +2) (k +2) (n ) (n )
sk = [dk 0 , … , dk−𝜈+1
0
, dk 0 , … , dk−𝜈+1
0
, … , dk 0 , … , dk−𝜈+1
0
] (11.304)

which has a number of components N2 = 𝜈 ⋅ (n0 − k0 ), equal to the number of the encoder memory
elements. The set of states , that is the possible values assumed by sk , is given by

sk ∈  = {σ1 , σ2 , … , σNs } (11.305)

where Ns = 2N2 is the number of encoder states. It is important to observe that the convolutional encoder
can be seen as a sequential finite-state machine with i.i.d. input bk , and state transition function sk =
fs (bk , sk−1 ). Hence, for a given information vector bk , the transition from state sk−1 = σi to state sk = σj
is unique, in correspondence of which a code vector is generated, that is expressed as
(k ) (k +1) (n ) (p)
ck = [c(1)
k
, c(2)
k
, … , ck 0 , c k 0 , … , ck 0 ] = [ c(s)
k
, ck ] (11.306)

where the superscript(s) denotes systematic bits, and(p) denotes parity check bits. Then c(s)
k
= bk , and
(p)
from (11.301), we can express ck as a function of sk and sk−1 as
(p)
ck = f (p) (sk , sk−1 ) (11.307)

The values assumed by the code vectors are indicated by

𝜷 = [𝛽 (1) , 𝛽 (2) , … , 𝛽 (k0 ) , 𝛽 (k0 +1) , … , 𝛽 (n0 ) ] = [ 𝜷 (s) , 𝜷 (p) ] (11.308)

The actual input to the decoder is the LLR 𝓁k(in) associated with ck and whose expression is recalled in
page 589. Moreover, we model the decoder input as if the code binary symbols were transmitted by a
binary modulation scheme over an AWGN channel. In this case, at the decoder input, we get the signal
(see (11.275)),
zk = ck + 𝑤k (11.309)

where ck ∈ {−1, 1} denotes a code bit, and {𝑤k } is a sequence of real-valued i.i.d. Gaussian noise
samples with (unknown) variance 𝜎dec 2
. It is useful to organize the samples {zk } into subsequences that
follow the structure of the code vectors (11.306). Then we introduce the vectors
(k ) (k +1) (n ) (p)
zk = [zk(1) , zk(2) , … , zk 0 , zk 0 , … , zk 0 ] = [ z(s)
k
, zk ] (11.310)

As usual, we denote as 𝝆k an observation of zk ,

(k ) (k +1) (n ) (p)
𝝆k = [𝜌(1)
k
, 𝜌(2)
k
, … , 𝜌k 0 , 𝜌k 0 , … , 𝜌k 0 ] = [𝝆(s)
k
, 𝝆k ] (11.311)

We recall from Section 7.5 that the FBA yields the detection of the single information vector bk , k =
0, 1, … , K − 1, expressed as
b̂ k = [ b̂ (1) , b̂ (2) , … , b̂ k 0 ]
(k )
k k
(11.312)

through the computation of the a posteriori probability. We also recall that in general, for a sequence
a = [a0 , … , ak , … , aK−1 ], with the notation am
l
, we indicate the subsequence formed by the components
[al , al+1 , … , am ].
Defining the likelihood of the generic information vector (see (6.26)),

𝙻k (𝜷 (s) ) = P [bk = 𝜷 (s) ∣ zK−1

0 = 𝝆K−1
0 ] (11.313)

detection by the MAP criterion is expressed as

b̂ k = arg max 𝙻k (𝜷 (s) ), k = 0, 1, … , K − 1 (11.314)

𝜷 (s)
11.6. Turbo codes 603

We note that the likelihood associated with the individual bits of the information vector bk are obtained
by suitably adding (11.313), as
𝙻k,i (𝛼) = P [b(i) = 𝛼 ∣ zK−1 = 𝝆K−1 ]
∑
k 0 0
= 𝙻k (𝜷 ), 𝛼 ∈ {−1, 1}
(s)
(11.315)
𝜷 (s) ∈{−1,1}k0
𝛽 (i) =𝛼

In a manner similar to the analysis of page 347, we introduce the following quantities:
1. The state transition probability Π(j ∣ i) = P [sk = 𝝈 j ∣ sk−1 = 𝝈 i ], that assumes non-zero values
only if there is a transition from the state sk−1 = 𝝈 i to the state sk = 𝝈 j for a certain input 𝜷 (s) ,
and we write
Π(j ∣ i) = P [bk = 𝜷 (s) ] = 𝙻k(a) (𝜷 (s) ) (11.316)
𝙻k(a) (𝜷 (s) ) is called the a priori information on the information vector bk = 𝜷 (s) , and is one of the
soft inputs.
2. For an AWGN channel, the channel transition probability pzk (𝝆k ∣ j, i) can be separated into two
contributions, one due to the systematic bits and the other to the parity check bits,
pzk (𝝆k ∣ j, i) = P [zk = 𝝆k ∣ sk = 𝝈 j , sk−1 = 𝝈 i ]
= P [z(s)
k
= 𝝆(s)
k
∣ sk = 𝝈 j , sk−1 = 𝝈 i ]
(p) (p)
P [zk = 𝝆k ∣ sk = 𝝈 j , sk−1 = 𝝈 i ]
(p) (p) (p)
= P [z(s)
k
= 𝝆(s)
k
∣ c(s)
k
= 𝜷 (s) ] P [zk = 𝝆k ∣ ck = 𝜷 (p) ]
k
⎛⎛ ⎞0 ⎞
⎜⎜ 1 ⎟ − 2𝜎12 ∥𝝆(s) k
−𝜷 (s) ∥2 ⎟
(11.317)
= ⎜⎜ √ ⎟ e I
⎟
⎜⎜ 2𝜋𝜎 2 ⎟ ⎟
⎝⎝ I ⎠ ⎠
n0 −k0
⎛⎛ ⎞ ⎞
⎜⎜ 1 ⎟ − 12 ∥𝝆k −𝜷 (p) ∥2 ⎟
(p)

⎜⎜ √ ⎟
2𝜎
e I ⎟
⎜⎜ 2𝜋𝜎 2 ⎟ ⎟
⎝⎝ I ⎠ ⎠
3. We merge (11.316) and (11.317) into one variable (see(7.390)),
Ck (j ∣ i) = P [zk = 𝝆k , sk = 𝝈 j ∣ sk−1 = 𝝈 i ] = pzk (𝝆k ∣ j, i) Π(j ∣ i)
n
⎛ ⎞0
⎜ 1 ⎟ (s) (p) (11.318)
= ⎜√ ⎟ Ck (j ∣ i) Ck (j ∣ i)
⎜ 2𝜋𝜎 ⎟ 2
⎝ I ⎠

where
− 1
∥𝝆(s) −𝜷 (s) ∥2
Ck(s) (j ∣ i) = e 2𝜎 2
𝙻k(a) (𝜷 (s) )
k
I (11.319)
(p)
(p) − 12 ∥𝝆k −𝜷 (p) ∥2
2𝜎
Ck (j ∣ i) = e I (11.320)
The two previous quantities are related, respectively, to the systematic bits and the parity check
bits of a code vector. Observe that the exponential term in (11.319) represents the reliability of a
certain a priori information 𝙻k(a) (𝜷 (s) ) associated with 𝜷 (s) .
4. The computation of the forward and backward metrics is carried out as in the general case.
– Forward metric, for k = 0, 1, … , K − 1:
∑
Ns
Fk (j) = Ck (j ∣ 𝓁) Fk−1 (𝓁), j = 1, … , Ns (11.321)
𝓁=1
604 Chapter 11. Channel codes

– Backward metric, for k = K − 1, K − 2, … , 0:

∑
Ns
Bk (i) = Bk+1 (m) Ck+1 (m ∣ i), i = 1, … , Ns (11.322)
m=1

Suitable initializations are obtained, respectively, through (7.398) and (7.405).

5. By using the total probability theorem, the likelihood (11.313) can be written as

∑
Ns
𝙻k (𝜷 (s) ) = A P [sk−1 = 𝛔i , sk = σj , zK−1
0 = 𝝆K−1
0 ] (11.323)
i=1
𝛔j = fs (𝜷 (s) , 𝛔i )

where fs is the state transition function, and the multiplicative constant A = 1∕P[zK−1 0
= 𝝆K−1
0
] is
irrelevant for vector detection as can be seen from (11.314). We note that the summation in (11.323)
is over all transitions from the general state sk−1 = 𝝈 i to the state sk = 𝝈 j = fs (𝜷 (s) , σi ) generated by
the information vector bk = 𝜷 (s) . On the other hand, the probability in (11.323) can be written as

P [sk−1 = σi , sk = σj , zK−1
0
= 𝝆K−1
0
]
= P [zK−1
k+1
= 𝝆K−1
k+1
| sk−1 = σi , sk = σj , zk0 = 𝝆k0 ]
P [sk = σj , zk = 𝝆k | sk−1 = σi , zk−1
0
= 𝝆k−1
0
]
P [sk−1 = σi , zk−1
0
= 𝝆k−1
0
] (11.324)
= P [zK−1
k+1
= 𝝆K−1
k+1
| sk = σj ]
P [sk = σj , zk = 𝝆k | sk−1 = σi ] P [sk−1 = σi , zk−1
0
= 𝝆0k−1 ]
= Bk (j) Ck (j|i) Fk−1 (i)
Substituting for Ck (j|i) the expression in (11.318), the likelihood becomes

𝙻k (𝜷 (s) ) = B 𝙻(a)
k
(𝜷 (s) ) 𝙻(int)
k
(𝜷 (s) ) 𝙻(ext)
k
(𝜷 (s) ) (11.325)

where B = A∕(2𝜋𝜎I2 ) is an irrelevant constant,

− 1
∥𝝆(s) −𝜷 (s) ∥2
𝙻(int) (𝜷 (s) ) = e 2𝜎 2 k
k
I (11.326)

and
∑
Ns
(p)
𝙻(ext)
k
(𝜷 (s) ) = Bk (j) Ck (j|i) Fk−1 (i) (11.327)
i=1
σj = fs (𝜷 (s) , σi )

Observing each term in (11.325), we make the following considerations.

i. 𝙻k(a) (𝜷 (s) ) represents the a priori information on the information vector bk = 𝜷 (s) .
ii. 𝙻(int)
k
(𝜷 (s) ) depends on the received samples associated with the information vector and on the
channel characteristics.
iii. 𝙻(ext)
k
(𝜷 (s) ) represents the extrinsic information extracted from the received samples associ-
ated with the parity check bits. This is the incremental information on the information vector
obtained by the decoding process.
6. Typically, it is easier to work with the logarithm of the various likelihoods.
We associate with each bit of the code vector ck a LLR
(in,n0 ) (in,p)
𝓵 (in)
k
= [𝓁k(in,1) , … , 𝓁k ] = [𝓵 (in,s)
k
, 𝓵k ] (11.328)
11.6. Turbo codes 605

For binary modulation, from (11.309), 𝜌k and 𝓁k(in) are related through (6.39)
2
𝓵 (in) = 𝝆k (11.329)
k
𝜎dec
2

where 𝝆k is the observation at the instant k. Moreover, we can see that 𝜎dec
2
is not relevant since it
is a common scaling factor not relevant for final detection.
We define now two quantities that are related, respectively, to the systematic bits and the parity
check bits of the code vector, as

1 ∑ (in,m) (m)
k0
𝓁k(s) (𝜷 (s) ) = 𝓁 𝛽 (11.330)
2 m=1 k
and
1 ∑ (in,m) (m)
n0
(p)
𝓁k (j, i) = 𝓁 𝛽 (11.331)
2 m=k +1 k
0

where by (11.307) and (11.308), we have

𝜷 (p) = [𝛽 (k0 +1) , … , 𝛽 (n0 ) ] = f (p) (𝝈 j , 𝝈 i ) (11.332)

Expressing (11.319) and (11.320) as a function of the likelihoods (11.330) and (11.331), apart from
factors that do not depend on {𝛽 (m) }, m = 1, … , n0 , we get
(s)
′
(𝜷 (s) ) 𝓁k(a) (𝜷 (s) )
Ck(s) (j ∣ i) = e𝓁k e (11.333)

and
′ (p) (p)
Ck (j ∣ i) = e𝓁k (j,i)
(11.334)

To compute the forward and backward metrics, we use, respectively, (11.321) and (11.322), where
′ ′ (p) (p)
the variable Ck (j|i) is replaced by Ck (j ∣ i) = Ck(s) (j ∣ i) Ck (j ∣ i). Similarly in (11.327) Ck (j ∣ i)
′

′ (p)
is replaced by Ck (j ∣ i). Taking the logarithm of (11.327), we obtain the extrinsic component
𝓁k(ext) (𝜷 (s) ).
Finally, from (11.325), by neglecting non-essential terms, the log-likelihood associated with the
information vector bk = 𝜷 (s) is given by

𝓁k (𝜷 (s) ) = 𝓁k(a) (𝜷 (s) ) + 𝓁k(int) (𝜷 (s) ) + 𝓁k(ext) (𝜷 (s) ) (11.335)

where 𝓁k(int) (𝜷 (s) )

= 𝓁k(s) (𝜷 (s) )
is usually called the intrinsic component.
Expression (11.335) suggests an alternative method to (11.327) to obtain 𝓁k(ext) (𝜷 (s) ), which uses
′
the direct computation of 𝓁k (𝜷 (s) ) by (11.323) and (11.324), where Ck (j ∣ i) is replaced by Ck (j ∣ i),
whose factors are given in (11.333) and (11.334). From the known a priori information 𝓁k(a) (𝜷 (s) )
and the intrinsic information (11.330), from (11.335), we get

𝓁k(ext) (𝜷 (s) ) = 𝓁k (𝜷 (s) ) − 𝓁k(a) (𝜷 (s) ) − 𝓁k(int) (𝜷 (s) ) (11.336)

Going back to the expression (11.335), detection of the vector bk is performed according to the
rule
b̂ k = arg max 𝓁k (𝜷 (s) ) (11.337)
𝜷 (s)

Note that to compute 𝓁k(ext) (𝜷 (s) ) (𝓁k (𝜷 (s) )) by the logarithm of (11.327) (or (11.323) and (11.324)), we
can use the Max-Log-MAP method introduced in (6.43) and page 352.
606 Chapter 11. Channel codes

Example 11.6.1 (Systematic convolutional code with rate 1∕2)

For a convolutional code with rate Rc = 1∕2, the information vector bk = [bk ] is composed of only one
(p)
bit (k0 = 1), like the systematic part and the parity check part of ck = [c(s)
k
, ck ].
In this case, it is sufficient to determine the log-likelihoods 𝓁k (−1) and 𝓁k (1), or better the LLR
𝙻k (1)
𝓁k = ln = 𝓁k (1) − 𝓁k (−1) (11.338)
𝙻k (−1)
Detection of the information bit is performed according to the rule
b̂ k = sgn(𝓁k ) (11.339)
The a priori information at the decoder input is given by
P[bk = 1]
𝓁k(a) = ln (11.340)
P[bk = −1]
from which we derive the a priori probabilities
1 (a)
e− 2 𝓁k
(s) 𝓁 (a)
e𝛽 1 (s) 𝓁 (a)
e2𝛽
(a)
P[bk = 𝛽 (s) ] = e𝓁k
k
(𝛽 (s) )
= = k , 𝛽 (s) ∈ {−1, 1} (11.341)
𝛽 (s) 𝓁k(a) −𝓁k(a)
1+e 1+e
By using LLRs, (11.330) yields
𝓁k(int) = 𝓁k(int) (1) − 𝓁k(int) (−1) = 𝓁k(in,1) = 𝓁k(in,s) (11.342)
In turn (11.333) and (11.334) for k0 = 1 and n0 = 2 simplify into
′ 1
𝛽 (s) (𝓁k(in,s) +𝓁k(a) )
Ck(s) (j ∣ i) = e 2 (11.343)
1 (in,p)
′ (p)
𝛽 (p) 𝓁k
Ck (j ∣ i) = e 2 (11.344)
The extrinsic component is obtained starting from (11.327) and using the above variables
𝙻(ext)
k
(1)
𝓁k(ext) = ln = 𝓁k(ext) (1) − 𝓁k(ext) (−1) (11.345)
𝙻(ext)
k
(−1)
From (11.335), apart from irrelevant terms, the LLR associated with the information bit bk can be written
as
𝓁k = 𝓁k(a) + 𝓁k(int) + 𝓁k(ext) (11.346)
where the meaning of each of the three contributions is as follows:
– A priori information 𝓁k(a) : It is an a priori reliability measure on the bit bk . This value can be extracted
either from the known statistic of the information sequence or, in the case of iterative decoding of
turbo codes, from the previous analysis.
– Channel information 𝓁k(int) = 𝓁k(in,s) : As it is evident from the case of binary modulation, where
𝓁k(in,s) = 𝜎22 𝜌(s)
k
, if the noise variance is low, the contribute of 𝓁k(int) usually dominates with respect
dec

to the other two terms; in this case bit detection is simply obtained by the sign of 𝜌(s)
k
.
(ext)
– Extrinsic information 𝓁k : It is a reliability measure that is determined by the redundancy in the
transmitted sequence. This contribute improves the reliability of transmission over a noisy channel
using the parity check bits.
The decomposition (11.346) forms the basis for the iterative decoding of turbo codes.
11.6. Turbo codes 607

Example 11.6.2 (Non-systematic code and LLR associated with the code bits)
Consider the case of a non-systematic code. If the code is also non- recursive, we need to use in place
of (11.304) the state definition (11.273).
Now, all bits are parity check bits and (11.306) and (11.310) become, respectively,
(p) (n )
ck = ck = [c(1)
k
, … , ck 0 ] (11.347)
(p) (n )
zk = zk = [z(1)
k
, … , zk 0 ] (11.348)
(k )
However, the information vector is still given by bk = [bk(1) , … , bk 0 ] with values 𝜶 = [𝛼 (1) , … , 𝛼 (k0 ) ],
𝛼 (i) ∈ {−1, 1}. The likelihood (11.313) is given by
𝙻k (𝜶) = P [bk = 𝜶 ∣ zK−1
0 = 𝝆K−1
0 ] (11.349)
The various terms with superscript (s) of the previous analysis vanish by setting k0 = 0. Therefore,
(11.330) and (11.331) become
𝓁k(int) (𝜷 (s) ) = 𝓁k(s) (𝜷 (s) ) = 0 (11.350)
and
1 ∑ (in,m) (m)
n0
(p)
𝓁k (j, i) = 𝓁 𝛽 (11.351)
2 m=1 k

where 𝜷 = 𝜷 (p) = [𝛽 (1) , … , 𝛽 (n0 ) ] = f (𝝈 j , 𝝈 i ) is the code vector associated with the transition from state
𝝈 i to state 𝝈 j . Note that, apart from irrelevant factors, (11.351) coincides with (11.285).
For k0 = 1, it is convenient to use LLRs; in particular, (11.346) yields a LLR associated with the
information bit bk that is given by
𝓁k = 𝓁k(a) + 𝓁k(ext) (11.352)

where 𝓁k(ext) can be obtained directly using (11.345), (11.334), and (11.327).
(q)
In some applications, it is useful to associate a LLR with the encoded bit ck , q = 1, … , n0 , rather
than to the information bit bk . We define
(q)
P [ck = 1 ∣ zK−1
0
= 𝝆K−1
0
]
𝓁 k,q = ln (q)
(11.353)
P [ck = −1 ∣ zK−1
0
= 𝝆K−1
0
]
(a)
Let 𝓁 k,q be the a priori information on the code bits. The analysis is similar to the previous case, but
(q)
now, with respect to the encoder output, ck is regarded as an information bit, while the remaining bits
(m)
ck , m = 1, … , n0 , m ≠ q, are regarded as parity check bits. Equations (11.330), (11.331), (11.343), and
(11.344), are modified as follows:
(s) 1 (in,q)
𝓁 k,q (𝛽 (q) ) = 𝓁k 𝛽 (q) (11.354)
2
1 ∑ (in,m) (m)
n0
(p)
𝓁 k,q (j, i) = 𝓁k 𝛽 (11.355)
2
m=1
m≠q

and
1 (in,q) (a)
′
(s) 𝛽 (q) (𝓁k +𝓁 k,q )
Ck,q (j ∣ i) = e 2 (11.356)
(p)
Ck,q (j ∣ i) = e𝓁k,q (j,i)
′ (p)
(11.357)
608 Chapter 11. Channel codes

(ext)
Associated with (11.357), we obtain 𝓁 k,q by using (11.345) and (11.327). The overall result is given by
(a) (in,q) (ext)
𝓁 k,q = 𝓁 k,q + 𝓁k + 𝓁 k,q , q = 1, … , n0 (11.358)

Example 11.6.3 (Systematic code and LLR associated with the code bits)
With reference to the previous example, if the code is systematic, whereas (11.346) holds for the sys-
(q)
tematic bit c(1)
k
, for the parity check bits ck , the following relations hold [41]. For k0 = 1, let 𝛼 be the
value of the information bit bk , bk = 𝛼, with 𝛼 ∈ {−1, 1}, associated with the code vector

ck = 𝜷 = [𝛼, 𝛽 (2) , … , 𝛽 (n0 ) ] (11.359)

where we assume 𝛽 (1) = 𝛼. For q = 2, … , n0 , we get

(s) 1 (in,q) (q)
𝓁 k,q (𝛽 (q) ) = 𝓁 𝛽 (11.360)
2 k
1 ∑ (in,m) (m) 1 (a)
n0
(p)
𝓁 k,q (j, i) = 𝓁 k 𝛽 + 𝓁k 𝛼 (11.361)
2 2
m=1
m≠q

where 𝓁k(a) is the a priori information of bk . Furthermore,

1 (in,q)
(s)′
𝛽 (q) 𝓁k
Ck,q (j ∣ i) = e 2 (11.362)
(p)
Ck,q (j ∣ i) = e𝓁k,q (j,i)
′ (p)
(11.363)
(ext)
From (11.363), we get 𝓁 k,q using (11.345) and (11.327). The overall result is given by
(in,q) (ext)
𝓁 k,q = 𝓁k + 𝓁 k,q (11.364)

Iterative decoding
In this section, we consider the iterative decoding of turbo codes with k0 = 1. In this case, as seen in
Example 11.6.1, using the LLRs simplifies the procedure. In general, for k0 > 1 we should refer to the
formulation (11.335).
We now give a step-by-step description of the decoding procedure of a turbo code with rate 1∕3, of the
type shown in Figure 11.27, where each of the two component decoders DEC1 and DEC2 implements
the FBA for recursive systematic convolutional codes with rate 1∕2. The decoder scheme is shown in
Figure 11.30, where the subscript in LLR corresponds to the component decoder. In correspondence to
the information bit bk , the turbo code generates the vector

ck = [c(1)
k
, c(2)
k
, c(3)
k
] (11.365)

where c(1)
k
= bk . We now introduce the following notation for the observation vector 𝓵 (in)
k
that relates to
the considered decoder:
(in,p) (in,p)
𝓵 (in)
k
= [𝓁k(in,s) , 𝓁k,1 , 𝓁k,2 ] (11.366)
(in,p) (in,p)
where 𝓁k(in,s) corresponds to the systematic part, and 𝓁k,1 and 𝓁k,2 correspond to the parity check parts
generated by the first and second convolutional encoder, respectively.
If some parity check bits are punctured to increase the code rate, at the receiver, the corresponding
LLRs 𝓁k(in,m) are set to zero.
11.6. Turbo codes 609

1. First iteration:
1.1 Decoder DEC1 : If the statistic of the information bits is unknown, then the bits of the infor-
mation sequence are considered i.i.d. and the a priori information is zero,
(a) P[bk = 1]
𝓁k,1 = ln =0 (11.367)
P[bk = −1]
(in,p)
For k = 0, 1, 2, … , K − 1, observed 𝓁k(in,s) and 𝓁k,1 , we compute according to (11.343) and
′ (s) ′ (p)
(11.344) the variables Ck and Ck , and from these the corresponding forward metric Fk (j)
(11.321). After the entire sequence has been received, we compute the backward metric Bk (i)
(11.322) and, using (11.327), we find 𝙻(ext)
k,1
(1) and 𝙻(ext)
k,1
(−1). The decoder soft output is the
extrinsic information obtained by the LLR

(ext)
𝙻(ext)
k,1
(1)
𝓁k,1 = ln (11.368)
𝙻(ext)
k,1
(−1)

1.2 Interleaver: Because of the presence of the interleaver, the parity check bit c(3)n is obtained in
correspondence to a transition of the convolutional encoder state determined by the informa-
tion bit bn , where n depends on the interleaver pattern. In decoding, the extrinsic information
(ext)
𝓁k,1 , extracted from DEC1 , and the systematic observation 𝓁k(in,s) are scrambled by the turbo
(in,p)
code interleaver and associated with the corresponding observation 𝓁n,2 to form the input of
the second component decoder.
1.3 Decoder DEC2 : The extrinsic information generated by DEC1 is set as the a priori information
(a)
𝓁n,2 to the component decoder DEC2 ,

(a) P[bn = 1] (ext)

𝓁n,2 = ln = 𝓁n,1 (11.369)
P[bn = −1]
(ext)
The basic idea consists in supplying DEC2 only with the extrinsic part 𝓁n,1 of 𝓁n,1 , in order to
minimize the correlation between the a priori information and the observations used by DEC2 .
Ideally, the a priori information should be an independent estimate.
(ext)
As done for DEC1 , we extract the extrinsic information 𝓁n,2 .
1.4 Deinterleaver: The deinterleaver realizes the inverse function of the interleaver, so that the
(ext)
extrinsic information extracted from DEC2 , 𝓁n,2 , is synchronized with the systematic part
(in,p)
𝓁k(in,s) and the parity check part 𝓁k,1 of the observation of DEC1 . By a feedback loop, the a
(ext) (a)
posteriori information 𝓁k,2 is placed at the input of DEC1 as a priori information 𝓁k,1 , giving
origin to an iterative structure.
2. Successive iterations: Starting from the second iteration, each component decoder has at its input an
a priori information. The information on the bits become more reliable as the a priori information
stabilizes in sign and increases in amplitude.
3. Last iteration: When the decoder achieves convergence, the iterative process can stop and form the
overall LLR (11.346),

𝓁k,o𝑣erall = 𝓁k(in,s) + 𝓁k,1

(ext) (ext)
+ 𝓁k,2 , k = 0, 1, … , K − 1 (11.370)

and detection of the information bits bk is obtained by

b̂ k = sgn(𝓁k,o𝑣erall ) (11.371)
610 Chapter 11. Channel codes

(1)
ck
(2)
ck
bk dk dk–1 dk–2
D D D

Figure 11.31 Termination of trellis.

To make decoding more reliable, the final state of each component decoder is set to zero, thus enabling
an initialization of the backward metric as in (7.405). As illustrated in Figure 11.31, at the instant fol-
lowing the input of the last information bit, that is for k = K, the commutator is switched to the lower
position, and therefore, we have dk = 0; after 𝜈 clock intervals the zero state is reached. The bits c(1)
k
and
ck(2) , for k = K, K + 1, … , K + 𝜈 − 1, are appended at the end of the code sequence to be transmitted.

Performance evaluation
Performance of the turbo code with the encoder of Figure 11.27 is evaluated in terms of error probability
and convergence of the iterative decoder implemented by the FBA. For the memoryless AWGN channel,
error probability curves versus Eb ∕N0 are plotted in Figure 11.32 for a sequence of information bits of
length K = 640, and various numbers of iterations of the iterative decoding process. Note that perfor-
mance improves as the number of iterations increases; however, the gain between consecutive iterations
becomes smaller as the number of iterations increases.

100

10–1

10–2

10–3
(dec)
Pbit

10–4

10–5
1
2
10–6 3
4
6
8
10–7
−0.25 0 0.25 0.5 0.75 1 1.25 1.5 1.75
Eb /N0 (dB)

Figure 11.32 Performance of the turbo code defined by the UMTS standard, with length of the
information sequence K=640, and various numbers of iterations of the iterative decoding process.
11.7. Iterative detection and decoding 611

100
Eb /N0 = 0 dB
Eb /N0 = 0.5 dB
Eb /N0 = 1 dB
Eb /N0 = 1.5 dB
10–1

10–2
(dec)
Pbit

10–3

10–4

10–5
1 2 3 4 5 6 7 8 9 10 11 12
Number of iterations

Figure 11.33 Curves of convergence of the decoder for the turbo code defined by the UMTS
standard, for K = 320 and various values of Eb ∕N0 .

In Figure 11.33, the error probability P(dec)

bit
is given as a function of the number of iterations, for fixed
values of Eb ∕N0 , and K = 320. From the behaviour of the error probability, we deduce possible criteria
for stopping the iterative decoding process at convergence [40]. A timely stop of the iterative decoding
process leads to a reduction of the decoding delay and of the overall computational complexity of the
system. Note, however, that convergence is not always guaranteed.
The performance of the code depends on the length K of the information sequence. Figure 11.34
illustrates how the bit error probability decreases by increasing K, for a constant Eb ∕N0 . A higher value
of K corresponds to an interleaver on longer sequences and thus the assumption of independence among
the inputs of each component decoder is better satisfied. Moreover, the burst errors introduced by the
channel are distributed over all the original sequence, increasing the correction capability of the decoder.
As the length of the interleaver grows also the latency of the system increases.

11.7 Iterative detection and decoding

We consider the transmitter of Figure 11.35 composed of a convolutional encoder, interleaver, bit map-
per, and modulator for 16-PAM. Interpreting the channel as a finite-state machine, the overall structure
may be interpreted as a serial concatenated convolutional code (SCCC). The procedure of soft-input,
soft-output detection and soft input (SI) decoding of page 589 can be made iterative by applying the prin-
ciples of the previous section, by including a soft-input, soft-output decoding stage. With reference
to Figure 11.36, a step-by-step description follows:
0. Initialization: Suppose we have no information on the a priori probability of the code bits, therefore,
we associate with cn a zero LLR,
(a)
𝓁n,det =0 (11.372)
612 Chapter 11. Channel codes

100
K = 40
K = 320
K = 640
10–1

10–2

10–3
(dec)
Pbit

10–4

10–5

10–6

10–7
−0.75 −0.5 −0.25 0 0.25 0.5 0.75 1 1.25 1.5 1.75
Eb /N0 (dB)

Figure 11.34 Performance of the turbo code defined by the UMTS standard achieved after 12
iterations, for K=40, 320, and 640.

bl convolutional cm cn ck ak
interleaver S/P BMAP modulator
code

Figure 11.35 Encoder structure, bit mapper, and modulator; for 16-PAM:
ck = [c4k , c4k−1 , c4k−2 , c4k−3 ].

(a) interleaver
ℓn,det

bit
likelihood
(a,SYM) (in)
ℓk,det (𝛾) ℓm,dec = 0
SISO (ext) (in) SISO (ext)
ℓn,det ℓm,dec ℓm,dec
rk detector deinterleaver decoder

SI
decoder

^
bl

Figure 11.36 Iterative detection and decoding.

11.7. Iterative detection and decoding 613

1. Detector: First, we associate a log-likelihood with the two possible values of cn = 𝜂, 𝜂 ∈ {−1, 1},
according to the rule
(a) (a) (a)
𝓁n,det (1) = 𝓁n,det , 𝓁n,det (−1) = 0 (11.373)

Then we express the symbol ak as a function of the bits {cn } according to the bit mapper,
for example for 16-PAM,

ak = BMAP{ck = [c4k , c4k−1 , c4k−2 , c4k−3 ]} (11.374)

Assuming the sequence {cn } is a sequence of i.i.d. binary symbols, we associate with each value
of the symbol ak the a priori information expressed by the log-likelihood

(a,SYM)
∑
3
(a)
𝓁k,det (𝛾) = 𝓁4k−t,det (𝜂t ), 𝛾∈ (11.375)
t=0

where 𝛾 = BMAP{[𝜂0 , … , 𝜂3 ]}, 𝜂t ∈ {−1, 1}.

For multilevel transmission over a channel with ISI, the FBA of Section 7.5.2 provides a
log-likelihood for each value of ak . The new feature is that now in (7.385), we take into account
the a priori information on the various values of ak+L1 ; then (7.390) becomes
1
1 − 2 |𝜌k −uk |2 𝓁 (a,SYM) (𝛾)
Ck (j ∣ i) = √ e 2𝜎I e k+L1 ,det (11.376)
2
2𝜋𝜎I

where 𝛾 = f (𝝈 j , 𝝈 i ) ∈  is the symbol associated with the transition from the state 𝝈 i to the state
𝝈 j on the trellis determined by the ISI.
If Vk (i), i = 1, … , Ns , denotes the metric corresponding to the various states of the trellis, we asso-
ciate with each value of the code bits {cn } the following likelihood:

∑
Ns
𝙻4(k+L1 )−t,det (𝛼) = Vk (i), 𝛼 ∈ {−1, 1}, t = 0, … , 3 (11.377)
i=1
𝝈 i such that
c4(k+L )−t = 𝛼
1

Taking the logarithm of (11.377), we obtain the LLR

𝓁n,det = 𝓁n,det (1) − 𝓁n,det (−1) (11.378)

To determine the extrinsic information associated with {cn }, we subtract the a priori information
from (11.378),
(ext) (a)
𝓁n,det = 𝓁n,det − 𝓁n,det (11.379)

Note that in this application, the detector considers the bits {cn } as information bits and the log-
likelihood associated with cn at the detector output is due to the channel information8 in addition
to the a priori information.
(a)
In [42], the quantity 𝓁n,det in (11.379) is weighted by a coefficient, which is initially chosen small,
when the a priori information is not reliable, and is increased after each iteration.
(ext)
2. Deinterleaver: The metrics 𝓁n,det are re-ordered according to the deinterleaver to provide
(ext)
the sequence 𝓁m,det .

8
For the iterative decoding of turbo codes, this information is defined as intrinsic.
614 Chapter 11. Channel codes

3. Soft-input, soft-output decoder: As input LLR, we use

(in) (ext)
𝓁m,dec = 𝓁m,det (11.380)

and we set
(a)
𝓁m,dec =0 (11.381)

in the lack of an a priori information on the code bits {cm }. Indeed, we note that in the various
(a) (in)
formulas, the roles of 𝓁m,dec and 𝓁m,dec can be interchanged.
Depending on whether the code is systematic or not, we use the soft-input, soft-output decoding
procedure reported in Examples 11.6.1 and 11.6.2, respectively. In both cases, we associate with the
encoded bits cm the quantity
(ext) (in)
𝓁m,dec = 𝓁m,dec − 𝓁m,dec (11.382)

that is passed to the soft-input soft-output detector as a priori information, after reordering by the
interleaver.
4. Last iteration: After a suitable number of iterations, the various metrics stabilize, and from
(in)
the LLRs {𝓁m,dec } associated with {cm }, the SI decoding of bits {bl } is performed, using
the procedure of Example 11.6.1.

11.8 Low-density parity check codes

Low-density parity check (LDPC) codes were introduced by Gallager [9] as a family of linear block
codes with parity check matrices containing mostly zeros and only a small number of ones. The sparsity
of the parity check matrices defining LDPC codes is the key for the efficient decoding of these codes
by a message-passing procedure also known as the sum-product algorithm. LDPC codes and their effi-
cient decoding were reinvented by MacKay and Neal [43, 44] in the mid 1990s, shortly after Berrou
and Glavieux introduced the turbo-codes. Subsequently, LDPC codes have generated interest from a
theoretical as well as from a practical viewpoint and many new developments have taken place.
It is today well acknowledged that LDPC codes are as good as turbo codes, or even better. Also the
decoding techniques used for both methods can be viewed as different realizations of the same funda-
mental decoding process. However, the soft input soft output FBA, or suboptimal versions of it, used for
turbo decoding is rather complex, whereas the sum-product algorithm used for LDPC decoding lends
itself to parallel implementation and is computationally simpler. LDPC codes, on the other hand, may
lead to more stringent requirements in terms of storage.

11.8.1 Representation of LDPC codes

Matrix representation
From Section 11.2.1, we recall that a linear (n0 , k0 ) block code, where n0 and k0 denote the transmitted
block length and the source block length, respectively, can be described in terms of a parity check matrix
H, such that the equation
Hc = 0 (11.383)

is satisfied for all code words c. Each row of the r0 × n0 parity check matrix, where r0 = n0 − k0 is the
number of parity check bits, defines a parity check equation that is satisfied by each code word c. For
11.8. Low-density parity check codes 615

example, the (7,4) Hamming code is defined by the following parity check equations:

⎡ c1 ⎤
⎢c ⎥
⎢ 2⎥
⎡ 1 1 1 0 1 0 0 ⎤ ⎢ c3 ⎥ c5 = c1 ⊕ c2 ⊕ c3 (check 1)
⎢1 1 0 1 0 1 0⎥⎢c ⎥ = 0 ↔ c = c ⊕ c ⊕ c (check 2) (11.384)
⎢ ⎥⎢ ⎥ 4 6 1 2 4
⎣ 1 0 1 1 0 0 1 ⎦ ⎢ c5 ⎥ c 7 = c1 ⊕ c3 ⊕ c4 (check 3)
⎢ c6 ⎥
⎢ ⎥
⎣ c7 ⎦
LDPC codes differ in major ways with respect to the above simple example; they usually have long
block lengths n0 in order to achieve near Shannon-limit performance, their parity check matrices are
defined in non-systematic form and exhibit a number of ones that is much less than r0 ⋅ n0 . Traditionally,
the parity-check matrices of LDPC codes have been defined pseudo-randomly subject to the requirement
that H be sparse, and code construction of binary LDPC codes involves randomly assigning a small
number of the values in all-zero matrix to be 1. So the lack of any obvious algebraic structure in randomly
constructed LDPC codes is another reason that sets them apart from traditionally parity-check codes.
Moreover, they have a decoding algorithm whose complexity is linear in the block length of the code,
which allows the decoding of large codes.
A parity check matrix for a (J, K)-regular LDPC code has exactly J ones in each of its columns and K
ones in each of its rows. If H is low density, but the number of 1s in each column or row is not constant,
then the code is an irregular LDPC code.
Importantly, an error correction code can be described by more than one parity check matrix. It is not
required that two parity-check matrices, for the same code, have the same number of rows, but they need
the rank over GF(2) of both be the same, since the number of information bits, k0 , in a binary code it is
given by
k0 = n0 − rank2 (H) (11.385)

where rank2 (H) is the number of rows H which are linearly independent over GF(2). The pair (J, K)
together with the code length n0 specifies an ensemble of codes, rather than any particular code.
Gallager showed that the minimum distance of a typical regular LDPC code increases linearly with
n0 , provided J ≥ 3. Therefore, regular LDPC codes are constructed with J on the order of 3 or 4, subject
to the above constraint. For large block lengths, the random placement of 1s in H such that each row has
exactly K 1s and each column has exactly J 1s requires some effort and systematic methods for doing
this have been developed.
We emphasize that the performance of an LDPC code depends on the random realization of the parity
check matrix H. Hence, these codes form a constrained random code ensemble.

Graphical representation
A parity check matrix can generally be represented by a bipartite graph, also called Tanner graph, with
two types of nodes: the bit nodes and the parity check nodes (or check nodes) [45]. A bit node n, rep-
resenting the code bit cn , is connected by an edge to the check node m only if the element (m, n) of the
parity check matrix is equal to 1. No bit (check) node is connected to a bit (check) node. In a graph, the
number of edges incident upon a node is called the degree of the node. For example, the (7,4) Hamming
code can be represented by the graph shown in Figure 11.37.
We note in this specific case that because the parity check matrix is given in systematic form, bit nodes
c5 , c6 , and c7 in the associated graph are connected to single distinct check nodes. The parity check matrix
616 Chapter 11. Channel codes

check 1 check 2 check 3

x1 x2 x3 x4 x5 x6 x7

Figure 11.37 Tanner graph corresponding to the parity check matrix of the (7,4) Hamming code.

of a (J, K)-regular LDPC code leads to a graph where every bit node is connected to precisely J check
nodes, and every check node is connected to precisely K bit nodes.
A cycle in a Tanner graph is a sequence of connected vertices which starts and ends at the same vertex
in the graph, and which contains other vertices no more than once. The length of a cycle is the number
of edges it contains, and the girth of a graph is the size of its smallest cycle. The shortest possible cycle
in a bipartite graph is clearly a length-4 cycle, and such cycles manifest themselves in the H matrix as
four 1s that lie on the corners of a submatrix of H. We are interested in cycles, particularly short cycles,
because they degrade the performance of the iterative decoding algorithm used for LDPC codes.
Graphical representations of LDPC codes are useful for deriving and implementing the iterative
decoding procedure introduced in [9]. Gallager decoder is a message-passing decoder, in a sense to be
made clear below, based on the so-called sum-product algorithm, which is a general decoding algorithm
for codes defined on graphs.9

11.8.2 Encoding

Encoding procedure
From (11.34), encoding is performed by multiplying the vector of k0 information bits b by the generator
matrix G of the LDPC code:
cT = b T G (11.386)

where the operations are in GF(2). Hence, the generator and parity check matrices satisfy the relation

HGT = 0 (11.387)

The parity check matrix in systematic form is H ̃ I], where I is the r × r identity matrix and à is
̃ = [A,
0 0
a binary matrix. Recall also that any other r0 × n0 matrix H whose rows span the same space as H ̃ is a
valid parity check matrix.
Given the block length n0 of the transmitted sequence and the block length k0 of the information
sequence, we select a column weight J, greater than or equal to 3. To define the code, we generate a
rectangular r0 × n0 matrix H = [A B] at random with exactly J ones per column and, assuming a proper
choice of n0 and k0 , exactly K ones per row. The r0 × k0 matrix A and the square r0 × r0 matrix B are
very sparse. If the rows of H are independent, which is usually true with high probability if J is odd [44],

9
A wide variety of other algorithms (e.g. the VA, the FBA, the iterative turbo decoding algorithm, the fast Fourier
transform,…) can also be derived as specific instances of the sum-product algorithm [46].
11.8. Low-density parity check codes 617

̃
by Gaussian elimination and reordering of columns we determine an equivalent parity check matrix H
in systematic form. We obtain the generator matrix in systematic form as
[ ] [ ]
T I I
G = ̃ = (11.388)
A B−1 A
where I is the k0 × k0 identity matrix.
One of the most relevant problem of this encoding procedure is its quadratic complexity in the block
length. In order to reduce this complexity, either linear or quadratic, Richardson and Urbanke (RU) took
advantage of the sparsity of the H matrix. The RU algorithm consists of two steps: a preprocessing and
the actual encoding [47].

11.8.3 Decoding
The key to extracting maximal benefit from LDPC codes is soft decision decoding. In general, they are
iterative decoding algorithms which are based on the code’s Thanner graph, where message passing is
an essential part of them. In particular, we imagine that bit nodes are a sort of processors of one type,
check nodes processors of another type, and the edges represent message paths.

Hard decision decoder

To illustrate the process of iterative decoding, we first show an intuitive algorithm which uses an hard
decision (0 or 1) assessment of each received bit. For this algorithm, the messages are simple: each bit
node n sends a message to each of the check nodes to which it is connected, declaring if it is 1 or 0, and
each check node i sends a message to each of the bit nodes to which it is connected declaring what value
the bit node n should have in order to satisfy the check equation i, given the value of the bit nodes (other
than n) involved in it. This algorithm operates as follows:
Initialization: Each bit node is assigned the bit value received, by hard decision, from the channel and
send messages to check nodes to which it is connected indicating its value.
First step: Using the messages from the bit nodes, each check node determines which value should have
each bit node, in order to satisfy the parity check equation. If the sum (modulo 2) of the bit nodes
(other than the message receiver) is 1, the left bit node will receive the message 1, otherwise, if the
sum is 0, the message sent will be 0.
Second step: If the majority of the messages received by each bit node is different from the value with it
associated, the bit node flips its current value, in order to satisfies the greater part of check equations,
otherwise, the value is retained.
Third step: If Ĥc = 0, where ĉ is the updated received code word, decoding is stopped; if the maximum
number of allowed iterations is reached, the algorithm terminates and a failure to converge is reported;
otherwise, each bit node sends new messages to the check nodes to which it is connected, indicating
its value, and the algorithm returns to the First step.

Example 11.8.1
Consider the following (6,2) code with parity check matrix
⎡1 1 0 1 0 0⎤
⎢ ⎥
0 1 1 0 1 0⎥
H=⎢ (11.389)
⎢1 0 0 0 1 1⎥
⎢0 0 1 1 0 1 ⎥⎦
⎣
with J = 2 and K = 3. The corresponding Thanner graph is shown in Figure 11.38:
618 Chapter 11. Channel codes

Figure 11.38 Tanner graph corresponding to the parity check matrix of code (11.389).

(1,0,0) (0,1,1) (1,1,1) (1,0,1)

1 0 1 0 1 1 1 0 1 0 1 1
(a) (b)

1 0 1 0 1 1 0 0 1 0 1 1
(0,0) (1,0) (1,1) (1,0) (1,0) (0,1) (0,0) (1,0) (1,1) (1,0) (1,0) (0,1)
(c) (d)

Figure 11.39 Illustrative example of iterative decoding: each sub-figure show the messages flow of
the algorithm at each step. The arrow indicates the messages direction.

Assume that the code word sent is c = [0 0 1 0 1 1] and that the word received by hard decision
from the channel is c̃ = [1 0 1 0 1 1]. The steps required to decode this received word are shown in
Figure 11.39.
First of all the bit values are initialized to be 1, 0, 1, 0, 1, and 1, respectively (Figure 11.39a), and mes-
sages are sent to the check nodes indicating these values (Figure 11.39b). Then each check node sends the
desired value, in order to satisfy the related equation, to the bit nodes connected with it (Figure 11.39c).
Finally, we can see that the first bit has the majority of its messages indicating a value different from the
initial one and so it flips from 1 to 0. Now, the word received is a code word, so the algorithm halts and
returns c̃ = [0 0 1 0 1 1].
The existence of cycles in the Thanner graph of a code reduces the effectiveness of the iterative decod-
ing process. To illustrate the effect of a 4-cycle, we can take into consideration the code which leads to
the representations on Figure 11.40, where the sent code word is c = [0 0 1 0 0 1], while the word
received from the channel is c̃ = [1 0 1 0 0 1].
As a result of the 4-cycle each of the first two code word bits are involved in the same two parity-check
equations, and so when neither of the parity check equations is satisfied, it is not possible to determine
which bit is causing the error.
11.8. Low-density parity check codes 619

(1,0,0) (1,0,0) (1,0,1) (1,0,1)

1 0 1 0 0 1 1 0 1 0 0 1
(a) (b)

(0,1,0) (0,1,0) (1,0,1) (1,0,1)

1 0 1 0 0 1 0 1 1 0 0 1
(0,0) (1,1) (1,1) (1,0) (1,0) (1,1) (0,0) (1,1) (1,1) (1,0) (1,0) (1,1)
(c) (d)

0 1 1 0 0 1 1 0 1 0 0 1
(1,1) (0,0) (1,1) (1,0) (1,0) (1,1) (1,1) (0,0) (1,1) (1,0) (1,0) (1,1)
(e) (f)

Figure 11.40 Illustrative example of a 4-cycle Thanner graph. The cycle is shown in solid lines in
the first subfigure.

The sum-product algorithm decoder

The above description of hard input decoding is mainly to get an idea about the decoder procedure. Now,
we introduce the sum-product algorithm (SPA), also denoted as belief propagation algorithm (BPA) or
message passing algorithm: the underlying idea is exactly the same as in hard input decoding, but with
messages representing now probabilities.
Given the vector of input LLRs 𝓵 (in) = [𝓁1(in) , … , 𝓁n(in)
0
]T associated to code words c = [c1 , … cn0 ]T ,
the decoder problem is to determine the most likely binary vector x such that (see (11.383))

s = Hx = 0 (11.390)

A word on notation: when we use parity check matrices or equations, we assume the {0, 1} alpha-
bet, while we use the {−1, 1} alphabet on detected bits of decoder output. First, we give some general
definitions:

● checks si , i = 1, … , r0 : elements of the vector s. They are represented by the check nodes in the
corresponding Tanner graph.
● L(i) = {n ∶ Hi,n = 1}, i = 1, … , r0 : set of the bit nodes that participate in the check with index i.
● L(i)∖ñ: set L (i) from which the element with index ñ has been removed.
620 Chapter 11. Channel codes

● M (n) = {i ∶ Hi,n = 1}, n = 1, … , n0 : set of the check nodes in which the bit with index n partici-
pates.
● M (n)∖̃i: set M (n) from which the element with index ̃i has been removed.
Then the aim is to compute the a posteriori probability (APP) that a given bit in code word c equals
𝛽, 𝛽 ∈ {0, 1}, given input vector 𝓵 (in) . Without loss of generality, let us focus on the decoding of bit xn ,
so that we are interested in computing the APP
q𝛽n = P [xn = 𝛽 ∣ s = 0] (11.391)
For the sake of a simpler notation in the above and following expressions, we drop the condition on the
input LLRs 𝓵 (in) .
The algorithm consists of two alternating steps, illustrated in Figure 11.41, in which two quantities q𝛽i,n
𝛽
and ri,n , associated with each non-zero element of the matrix H, are iteratively updated. The quantity q𝛽i,n
denotes the probability that xn = 𝛽, 𝛽 ∈ {0, 1}, given the check equations (other than check i) involving
xn are satisfied:
q𝛽i,n = P [xn = 𝛽 ∣ {si′ = 0, i′ ∈ M(n)∖i}] (11.392)
𝛽
On the other hand, the quantity ri,n denotes the probability of check i being satisfied and the bits (other
than xn ) having a known distribution (given by the probabilities {q𝛽i,n′ ∶ n′ ∈ L(i)∖n, 𝛽 ∈ {0, 1}}), given
xn = 𝛽, 𝛽 ∈ {0, 1}:
𝛽
ri,n = P[si = 0, {xn′ , n′ ∈ L(i)∖n} ∣ xn = 𝛽] (11.393)

update r0 check nodes

i i

ri,n qi,n

n n
update n0 bit nodes

Figure 11.41 Message-passing decoding.

𝛽
In the first step, the quantities ri,n
associated with check node i are updated and passed as messages to
the bit nodes checked by check node i. This operation is performed for all check nodes. In the second
step, quantities q𝛽i,n associated with bit node n are updated and passed as messages to the check nodes
that involve bit node n. This operation is performed for all bit nodes.
From (11.398) and (11.401), we have that
0
ri,n = 1 − P [si = 1, {xn′ , n′ ∈ L(i)∖n} ∣ xn = 0]
= 1 − P [si = 0, {xn′ , n′ ∈ L(i)∖n} ∣ xn = 1] (11.394)
1
= 1 − ri,n
The algorithm is described as follows:
Initialization: Let p0n = P [xn = 0] denote the probability that xn = 0 given the observation 𝓵 (in) , and
p1n = P [xn = 1] = 1 − p0n . From the definition (6.37) of LLR 𝓁n(in)
1 1
p0n = (in)
, p1n = (in)
(11.395)
1+ e𝓁n 1 + e−𝓁n
Let q𝛽i,n = p𝛽n , n ∈ L(i), i = 1, … , r0 , 𝛽 ∈ {0, 1}. Then all bit nodes sends their q𝛽i,n messages to the
connected check nodes.
11.8. Low-density parity check codes 621

First step: We run through the checks, and for the ith check we compute for each n ∈ L(i) the probability
0
ri,n that, given xn = 0, si = 0 and the other bits {xn′ ∶ n′ ≠ n} have a distribution {q0i,n′ , q1i,n′ }.
0
In order to evaluate ri,n , we introduce the important result (due to Gallager, [48]): consider the
sequence of K independent bits {dn′ }, n′ = 1, … , K, for which P [dn′ = 1] = pn . Then the probability
that the sequence {dn′ }, n′ = 1, … , K, contains an even number of 1s is

1 1∏
K
+ (1 − 2pn′ ) (11.396)
2 2 n′ =1
The proof is by induction on K. In view of this result, from (11.394), we obtain
0 1 1 ∏
ri,n = + (1 − 2q1i,n′ ) (11.397)
2 2 n′ ∈L(i)∖n
since, given xn = 0, check equation i must contain an even number of 1s in order to be satisfied.
1 0
Moreover, from (11.402), ri,n = 1 − ri,n .
0
Second step: After computing ri,n and ri,n1
, we update the values of the probabilities q0i,n and q1i,n . From
(11.400) we obtain
P [xn = 𝛽, {si′ = 0, i′ ∈ M(n)∖i}]
q𝛽i,n = (11.398)
P [{si′ = 0, i′ ∈ M(n)∖i}]
Lumping in 𝛼i,n , the contribution of the terms that do not depend on 𝛽, we get
q𝛽i,n = 𝛼i,n P [si′ = 0, i′ ∈ M(n)∖i ∣ xn = 𝛽] p𝛽n
∏ 𝛽
= 𝛼i,n p𝛽n ri′ ,n (11.399)
i′ ∈M(n)∖i

where we used Bayes rule twice to obtain the first line and the independence assumption on code bits
to obtain the second line. The constant 𝛼i,n is chosen such that q0i,n + q1i,n = 1. Taking into account
the information from all check nodes, from (11.399), we can also compute the pseudo a posteriori
probabilities q0n and q1n at this iteration,
∏
q0n = 𝛼n p0n 0
ri,n (11.400)
i∈M(n)
∏
q1n = 𝛼n p1n 1
ri,n (11.401)
i∈M(n)

where 𝛼n is chosen such that q0n + q1n = 1.

At this point, it is possible to detect a code word ĉ by the log-MAP criterion; for n = 1, … , n0
( )
q1n
ĉ n = sgn ln 0 (11.402)
qn
Third step: If Ĥc = 0, where ĉ is the updated received code word, decoding is stopped; if the maximum
number of allowed iterations is reached, the algorithm terminates and a failure to converge is reported;
otherwise, each bit node sends their new q𝛽i,n messages to the check nodes to which it is connected,
and the algorithm returns to the First step.
The algorithm would produce the exact a posteriori probabilities of all the bits if the bipartite graph
defined by H contained no cycles [49].
We remark that the sum-product algorithm for the decoding of LDPC codes has been derived under
the assumption that the check nodes si , i = 1, … , r0 , are statistically independent given the bit nodes
xn , n = 1, … , n0 , and vice versa, i.e. the variables of the vectors s and x form a Markov field [46].
Although this assumption is not strictly true, it turns out that the algorithm yields very good performance
with low computational complexity. However, 4-cycle Tanner graphs would introduce non-negligible
622 Chapter 11. Channel codes

statistical dependence between nodes. A general method for constructing Tanner graphs with large girth
is described in [50].

The LR-SPA decoder

Actually, the SPA requires many multiplications which are more costly to implement than additions;
beside multiplications of probabilities could become numerically unstable. The result will come very
close to zero for large block lengths. To prevent this problem, likelihood ratios (LRs) or LLRs can be
used. First, we introduce the SPA using LRs
p1n
𝙻n(a) =
p0n
1 0
ri,n 1 − ri,n
𝙻(r)
i,n
= 0
= 0
ri,n ri,n
(11.403)
(q)
q1i,n
𝙻i,n =
q0i,n
q1n
𝙻n =
q0n
Using these ratios, the SPA can be rewritten as follows:
(in)
Initialization: Given the a priori LR Ln(a) = e𝓁n , we assign 𝙻i,n = 𝙻n(a) , n ∈ L(i), i = 1, … , r0 . Next, all
(q)

(q)
bit nodes send their 𝙻i,nmessages to the check nodes to which they are connected.
First step: Using the information from their bit nodes, each check node computes
∏
1 − 12 − 12 (1 − 2q1i,n′ )
n′ ∈L(i)∖n
𝙻(r)
i,n
= 1 1 ∏
2
+ 2
(1 − 2q1i,n′ )
n′ ∈L(i)∖n
∏ (11.404)
1− (1 − 2q1i,n′ )
n′ ∈L(i)∖n
= ∏
1+ (1 − 2q1i,n′ )
n′ ∈L(i)∖n
(q)
Now, from definition of 𝙻i,n′ it is
(q)
1 − 𝙻i,n′
1 − 2q1i,n′ = (q)
(11.405)
1 + 𝙻i,n′
and we can write ( )
∏ 1−𝙻i,n′
(q)

1− (q)
n′ ∈L(i)∖n 1+𝙻i,n′
𝙻(r) = ( ) (11.406)
i,n
∏ 1−𝙻i,n′
(q)

1+ (q)
n′ ∈L(i)∖n 1+𝙻i,n′

(q)
Second step: Using (11.407), now each bit node updates 𝙻i,n by simply dividing q1i,n by q0i,n , obtaining
∏ 1
𝛼i,n p1n ri′ ,n
(q) i′ ∈M(n)∖i
𝙻i,n = ∏
𝛼i,n p0n ri0′ ,n
i′ ∈M(n)∖i
(11.407)
∏
= 𝙻(a)
n 𝙻(r)
i′ ,n
i′ ∈M(n)∖i
11.8. Low-density parity check codes 623

while from (11.408) and (11.409)

∏
𝙻n = 𝙻(a)
n 𝙻(r)
i,n
(11.408)
i∈M(n)

and detection becomes, for n = 1, … , n0

ĉ n = sgn (ln(𝙻n )) (11.409)

Third step: If Ĥc = 0, where ĉ is the updated received code word, decoding is stopped; if the maximum
number of allowed iterations is reached, the algorithm terminates and a failure to converge is reported;
(q)
otherwise, each bit node sends their new 𝙻i,n messages to the check nodes to which it is connected,
and the algorithm returns to the First step.

The LLR-SPA or log-domain SPA decoder

The log-domain SPA is now presented. We first define
p1n
𝓁n(a) = ln = 𝓁n(in)
p0n
1
(r)
ri,n
𝓁i,n = ln 0
ri,n
(11.410)
(q)
q1i,n
𝓁i,n = ln
q0i,n
q1n
𝓁n = ln
q0n
(q) (q)
Initialization: Assign 𝓁i,n = 𝓁n(a) , n ∈ L(i), i = 1, … , r0 . Now, all bit nodes send their 𝓁i,n message to
the connected check nodes.
0 1
First step: We replace ri,n with 1 − ri,n and from (11.405) we obtain
∏
1
1 − 2ri,n = (1 − 2q1i,n′ ) (11.411)
n′ ∈L(i)∖n
[ ]
1
Using the fact that if 𝑣1 and 𝑣0 are probabilities with 𝑣0 + 𝑣1 = 1, tanh 2
ln(𝑣1 ∕𝑣0 ) = 𝑣1 − 𝑣0 =
1 − 2𝑣0 , we can rewrite (11.419) as
( ) ∏ [ ]
1 (r) 1 (q)
tanh 𝓁i,n = − tanh (−𝓁i,n′ ) (11.412)
2 n′ ∈L(i)∖n
2

However, we are still left with products and the tanh function. We can remedy as follows: First, we
(q)
factorize −𝓁i,n′ into its sign and amplitude. Let
(q) (q)
si,n′ = sgn (−𝓁i,n′ ), Mi,n′ = |𝓁i,n′ | (11.413)

it is
(q)
−𝓁i,n′ = si,n′ Mi,n′ (11.414)

Hence, (11.420) can be rewritten as

( )
( ) ∏ ∏ ( )
1 (r) 1
tanh 𝓁i,n = − si,n′ tanh M ′ (11.415)
2 n′ ∈L(i)∖n n′ ∈L(i)∖n
2 i,n
624 Chapter 11. Channel codes

from which we derive

( ) ( )
∏ ∏ ( )
(r) 1
𝓁i,n = − si,n′ 2 tanh−1
tanh M ′
n′ ∈L(i)∖n n′ ∈L(i)∖n
2 i,n
( ) ( )
∏ ∏ ( )
−1 −1 1
= − si,n′ 2 tanh ln ln tanh M ′
n′ ∈L(i)∖n n′ ∈L(i)∖n
2 i,n
( ) ( )
∏ ∑ ( ( ))
−1 −1 1
= − si,n′ 2 tanh ln ln tanh M ′
n′ ∈L(i)∖n n′ ∈L(i)∖n
2 i,n
(11.416)

We now define the function

[ ( )] ( x )
x e +1
𝜙(x) = − ln tanh = ln x (11.417)
2 e −1
As shown in Figure 11.42, the function is fairly well behaved and so it may be implemented by a
look-up table. Using the fact that 𝜙−1 (x) = 𝜙(x) for x > 0, (11.416) becomes
( ) ( )
(r)
∏ ∑
𝓁i,n = − si,n′ 𝜙 𝜙(Mi,n′ ) (11.418)
n′ ∈L(i)∖n n′ ∈L(i)∖n

Second step: From (11.407) it is

(q)
∑
𝓁i,n = 𝓁n(a) + 𝓁i(r)
′ ,n (11.419)
i′ ∈M(n)∖i

while (11.408) yields

∑
𝓁n = 𝓁n(a) + (r)
𝓁i,n (11.420)
i∈M(n)

Detection is performed as, for n = 1, … , n0 ,

ĉ n = sgn (𝓁n ) (11.421)

5
ϕ(x)

0
0 1 2 3 4 5 6
x

Figure 11.42 𝜙(x) function.

11.8. Low-density parity check codes 625

Third step: If Ĥc = 0, where ĉ is the updated received code word, decoding is stopped; if the maximum
number of allowed iterations is reached the algorithm terminates and a failure to converge is reported;
(q)
otherwise, each bit node sends their new 𝓁i,n messages to the check nodes to which it is connected,
and the algorithm returns to the First step, or, better, equation (11.414).

The min-sum decoder

Decoders of even lower complexity than LLR-SPA can be devised at the cost of a small performance
degradation, typically on the order of 0.5 dB which, however, is a function of the code and the channel.
Considering the shape of the 𝜙(x) function, it issues that the term corresponding to the smallest Mi,n
in the summation of (11.418) dominates, so that we approximate
( )
∑
𝜙 𝜙 (Mi,n′ ) ≃ 𝜙(𝜙 ( ′ min Mi,n′ ))
n ∈L(i)∖n (11.422)
n′ ∈L(i)∖n
= ′ min Mi,n′
n ∈L(i)∖n

Thus, the min-sum algorithm is simply the LLR-SPA where the First step is replaced by
( )
(r)
∏
𝓁i,n = − si,n′ ′ min Mi,n′ (11.423)
n ∈L(i)∖n
n′ ∈L(i)∖n

Other decoding algorithms

Despite of being the simplest method to avoid calculating the hyper tangent function, the min-sum algo-
rithm cannot approach the performance of the LLR-SPA decoder.
In [51], two simplified versions of the SPA, devised to achieve a faster iterative decoding, are proposed.
Both algorithms are implemented with real additions only, and this simplifies the decoding complexity.
To reduce decoding complexity, also the concept of forced convergence has been introduced.
Restricting the message passing to the nodes that still significantly contribute to the decoding result,
this approach substantially reduces the complexity with negligible performance loss. If the channel is
(q)
good (high SNR), some 𝓁i,n tend to have high magnitude, hence, their corresponding 𝜙 values is close
to zero. In this case, their magnitude contribution in (11.418) can be neglected. However, their sign
must be considered. The min-sum algorithm is the extreme case where the bit node with minimum
magnitude among the connected bit nodes is used.
Other simplified approaches are given in [52–54], while for a flexible encoder hardware implementa-
tion we refer to [55].

11.8.4 Example of application

We study in this section the application of LDPC codes to two-dimensional QAM transmission over an
AWGN channel [56].

Performance and coding gain

Recall from (6.82), the expression of the error probability for uncoded M-QAM transmission,
(√ )
3
Pe ≃ 4Q Γ (11.424)
M−1
626 Chapter 11. Channel codes

where Γ is the SNR given by (6.97). In general, the relation between M and the rate of the
encoder-modulator from (11.1) is given by
k log2 M
RI = 0 (11.425)
n0 2
Recall also, from (6.71), that the SNR per dimension is given by
E
ΓI = Γ = 2RI b (11.426)
N0
Using (6.129), we introduce the normalized SNR
ΓI 2RI Eb
ΓI = 2RI
= 2RI
(11.427)
2 −1 2 − 1 N0
Then for an uncoded M-QAM system we express (11.424) as
√
Pe ≃ 4Q ( 3ΓI ) (11.428)

As illustrated in Figure 6.13, the curve of Pe versus ΓI indicates that the gap to capacity for uncoded
QAM with M ≫ 1 is equal to Γgap,dB ≃ 9.8 dB at a symbol error probability of 10−7 . We, therefore,
c
determine the value of the normalized SNR ΓI needed for the coded system to achieve a symbol error
probability of 10−7 , and compute the coding gain at that symbol error probability as
c
Gcode,dB = 9.8 − 10 log10 ΓI (11.429)
From Figure 6.13, as for large SNRs the Shannon limit cannot be approached to within less than 1.53 dB
without shaping, we note that an upper limit to the coding gain measured in this manner is about 8.27 dB.
Simulation results for three high-rate (n0 , k0 ) binary LDPC codes are specified in Table 11.22 in terms of
the coding gains obtained at a symbol error probability of 10−7 for transmission over an AWGN channel
for 16, 64, and 4096-QAM modulation formats. Transmitted QAM symbols are obtained from coded bits
via Gray labelling. To measure error probabilities, one code word is decoded using the message-passing
(sum-product) algorithm for given maximum number of iterations. Figure 11.43 shows the effect on
performance of the maximum number of iterations allowed in the decoding process for code 2 specified
in Table 11.22 and 16-QAM transmission.

Table 11.22: LDPC codes considered for the simulation and coding gains achieved at a symbol error
probability of 10−7 for different QAM constellations. The spectral efficiencies 𝜈 are also indicated.

k0 n0 code ratek0 ∕n0 16-QAM 64-QAM 4096-QAM

Code 1 433 495 0.8747 4.9 dB 4.6 dB 3.5 dB
(3.49 bit/s/Hz) (5.24 bit/s/Hz) (10.46 bit/s/Hz)
Code 2 1777 1998 0.8894 6.1 dB 5.9 dB 4.8 dB
(3.55 bit/s/Hz) (5.33 bit/s/Hz) (10.62 bit/s/Hz)
Code 3 4095 4376 0.9358 6.2 dB 6.1 dB 5.6 dB
(3.74 bit/s/Hz) (5.61 bit/s/Hz) (11.22 bit/s/Hz)

The codes given in Table 11.22 are due to MacKay and have been obtained by a random construction
method. The results of Table 11.22 indicate that LDPC codes offer net coding gains that are similar
to those that have been reported for turbo codes. Furthermore, LDPC codes achieve asymptotically an
excellent performance without exhibiting error floors and admit a wide range of trade-offs between
performance and decoding complexity.
11.9. Polar codes 627

10–1

1: Maximum number of iterations = 5

10–2 2: Maximum number of iterations = 10
3: Maximum number of iterations = 20
4: Maximum number of iterations = 50
10–3
Bit-error rate

10–4

10–5
1

10–6 4 3 2

10–7
6 7 8 9 10 11 12
Eb/N0 (dB)

Figure 11.43 Performance of LDPC decoding with Code 2 and 16-QAM for various values of the
maximum number of iterations.

11.8.5 Comparison with turbo codes

LDPC codes are capable of near capacity performance while admitting a simple decoder. The advantages
that the LDPC codes have over turbo codes are the following: (i) they allow parallelizable decoder; (ii)
they are more amenable to high code rates; (iii) they generally posses a lower error-rate floor (for the
same length and rate); (iv) they possess superior performance in the presence of burst errors (due to
interference, fading, and so on); (v) they require no interleavers in the encoder and decoder; and (vi) a
single LDPC code can be universally good aver a collection of channels. Among their disadvantages are
the following (i) most LDPC codes have somewhat complex encoders; (ii) the connectivity among the
decoder component processors can be large and unwieldy; and (iii) turbo codes can often perform better
when the code length is short.
In Figure 11.44 a comparison between (i) regular LDPC, (ii) irregular LDPC, (iii) turbo codes, and
(iv) Shannon bound, is represented. The LDPC curves have been obtained by letting the decoder run
for enough iterations to get the best possible performance. We should note the great performance of
irregular LDPC codes, which are within a fraction of dB, in terms of Eb ∕N0 , from the Shannon bound,
and outperform turbo codes of the same length.

11.9 Polar codes

Polar codes are a new class of channel codes, introduced in 2008 by Arıkan in his seminal work [58].10 By
specific encoding and decoding, the unreliable communication channel is turned into virtual bit-channels
of which one portion is highly reliable and the other is highly unreliable. When the code length tends
to infinity, the highly reliable channels become error free, and the unreliable channels become fully
random – this phenomenon is referred to as channel polarization. Furthermore, the proportion of the

10
The material presented in this section was written by Dr Ingmar Land, whom the authors gratefully acknowledge.
628 Chapter 11. Channel codes

10–1
Bit Error Probability (BER)

10–2

10–3
Shannon Limit 0.19 dB

10–4

(3,6) Regular LDPC

Irregular LDPC

Turbo Code

10–5

Eb/N0
10–6
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 (dB)

Figure 11.44 Simulation results for a regular (3,6) LDPC, an optimized irregular LDPC and a turbo
code. All three codes are of rate Rc = 1∕2 and have block length 106 . Source: Reproduced with
permission from Richardson et al. [57]. ©2001, IEEE.

reliable channels tends to the capacity of the communication channel, and thus polar coding achieves
capacity. Remarkably, the encoding and decoding complexity of this capacity-achieving coding scheme
has only order O(n0 ln n0 ) in the code length n0 .
As opposed to other modern codes like turbo codes or LDPC codes, which are iteratively decoded,
polar coding is based on successive cancellation (SC) decoding. While in theory, plain SC decoding is
sufficient to achieve capacity, successive cancellation list (SCL) decoding is applied in the short-length to
medium-length regime [59–61]. The performance of polar codes is comparable to that of turbo codes or
LDPC codes; depending on the application, code length and code rate, and implementation constraints,
polar codes may be superior [60, 62]. Polar codes (under SCL decoding) have been adopted for 5G in
2016, namely as channel coding for uplink and downlink control information for the enhanced mobile
broadband (eMBB) communication service [63, 64].

11.9.1 Encoding
The structure of a polar code encoder is shown in Figure 11.45. In the following, we will go through the
blocks and their functionalities.

b υ c
I TN
k0 bits n0 bits n0 bits

Figure 11.45 Polar code encoding.

11.9. Polar codes 629

A polar code P(n0 , k0 ) of information length k0 , code length n0 = 2𝜖0 , 𝜖0 ∈ ℕ, and code rate
Rc = k0 ∕n0 is defined by the transformation matrix
⊗𝜖0
T n0 = T 2 (11.430)

given by the 𝜖0 -fold Kronecker product (see (4.152)) of the kernel matrix
[ ]
1 0
T2 = (11.431)
1 1
and by an information set  ⊂  of size k0 , where  = {0, 1, … , n0 − 1}. The set  =  ∖  of size
(n0 − k0 ) is the complementary set of  and is called the frozen set. Note that the information set and
the frozen set are index sets.
Consider the encoding of an information word b of length k0 bits into a codeword c of length n0 bits.
b is first mapped to input vector 𝒗 of length n0 , as
𝒗 = b
𝒗 = 0 (11.432)

i.e. the elements of 𝒗 indexed by the information set are set to the values of the information bits, and
the remaining elements, indexed by the frozen set, are set (frozen) to zero.11 This input vector is then
multiplied by the transformation matrix to obtain the codeword, (see also (11.34))

cT = 𝒗T T n0 (11.433)

This construction12 yields polar codes of lengths n0 that are powers of 2. Polar codes of other lengths
may be obtained by puncturing or shortening, see Section 11.9.5. The choice of the indices in the infor-
mation and frozen sets is a part of the polar code design, see Section 11.9.4.

Example 11.9.1
Consider a polar code P(8, 5) of length n0 = 8 and information length k0 = 5. With 𝜖0 = log2 n0 = 3,
the transformation matrix is given by
T 8 = T ⊗3
2
= T2 ⊗ T2 ⊗ T2
⎡1 0 0 0 0 0 0 0⎤
⎢1 1 0 0 0 0 0 0⎥
⎢ ⎥
1 0 1 0 0 0 0 0⎥
[ ] [ ] [ ] ⎢
1 0 1 0 1 0 ⎢1 1 1 1 0 0 0 0⎥
⊗ ⊗ =⎢
0⎥⎥
= (11.434)
1 1 1 1 1 1 ⎢1 0 0 0 1 0 0
⎢1 1 0 0 1 1 0 0⎥
⎢1 0 1 0 1 0 1 0⎥
⎢ ⎥
⎣1 1 1 1 1 1 1 1⎦
The information set  has size k0 = 5 and the frozen set  has size (n0 − k0 ) = 3; we assume the sets
 = {2, 3, 5, 6, 7}
 = {0, 1, 4}

11
Typically, the frozen bits are chosen to be zero and then the polar code is linear; in general, however, the frozen bits
may be set to arbitrary known values without affecting the performance.
12
Note that Arıkan used a bit-reversal permutation in the encoding of the polar code of his original work. Doing so leads
to an equivalent code and has no further effect on code properties, decoding algorithms, or performance.
630 Chapter 11. Channel codes

For encoding the information word bT = [b0 … b4 ] of length k0 = 5, its elements are first mapped into
the input vector,
𝒗T = [0 0 b0 b1 0 b2 b3 b4 ] (11.435)
i.e. the information bits {bn } are placed on the positions of 𝒗 indexed by the information set, and the
remaining positions of 𝒗 are set (frozen) to the known values zero. The codeword cT = [c0 … c7 ] is
obtained by multiplying 𝒗 by the transformation matrix,

⎡1 0 0 0 0 0 0 0⎤
⎢1 1 0 0 0 0 0 0⎥
⎢ ⎥
⎢1 0 1 0 0 0 0 0⎥
⎢1 1 1 1 0 0 0 0⎥
[c0 … c7 ]T = [0 0 b0 b1 0 b2 b3 b4 ]T ⎢
0⎥⎥
(11.436)
⎢1 0 0 0 1 0 0
⎢1 1 0 0 1 1 0 0⎥
⎢1 0 1 0 1 0 1 0⎥
⎢ ⎥
⎣1 1 1 1 1 1 1 1⎦
Note that the frozen positions effectively eliminate the rows of T 8 indexed by the frozen set  . Thus,
the remaining rows represent the generator matrix of the polar code.

Internal CRC
A CRC is typically used to detect decoding failures. In polar coding, an internal CRC may be used for
different purpose, namely to significantly increase the error-rate performance. The encoder structure is
shown in Figure 11.46.

b b′ v c
CRC I TN
k0 bits k0′ bits n0 bits n0 bits

Figure 11.46 Polar code encoding with CRC.

The information word b of length k0 is first extended by a CRC bCRC of length k0,CRC to the vector
[ ]
b
b′ = (11.437)
bCRC
of length k0′ = k0 + k0,CRC . The information set  of size k0′ and the frozen set  of size (n0 − k0′ ) are
then used to map the elements of b′ to the input vector 𝒗 of length n0 ,
𝒗 = b′
𝒗 = 0 (11.438)
The input vector 𝒗 is finally encoded into the codeword c as before, according to (11.441). Note that the
rate13 of the polar code is still Rc = k0 ∕n0 , but there are fewer frozen bits.
The internal CRC typically improves the distance spectrum of the polar code. Starting from the code
P(n0 , k0′ ) without CRC, the CRC constraints remove some of the codewords, and thus in general also
reduce the number of minimum distance codewords. If the CRC is long enough (and well chosen), it

13
Some authors refer to k0 ∕n0 as the overall rate (CRC and polar code), and to k0′ ∕n0 as the rate of the polar code
(excluding the CRC).
11.9. Polar codes 631

may even remove all minimum-distance codewords, such that the code P(n0 , k0 ) obtains an increased
minimum distance. Under SC decoding, these effects cannot be exploited; under SCL decoding, however,
this internal CRC leads to a significant performance improvement and is of paramount importance in
the short-length and medium-length regime [59, 60].

LLRs associated to code bits

The receiver, by soft detection methods (see page 589), provides to the decoder the LLR 𝓁n in correspon-
dence of each code bit cn . Moreover, as in previous codes, for analysis purposes, we model the decoder
input as (see (11.309))
zn = an + 𝑤n (11.439)
where codebit cn is mapped into the binary symbol an : 0 → +1, 1 → −1. Note that this mapping is
different from that used in LDPC codes. In (11.439), 𝑤n is AWGN with variance 𝜎dec
2
. Hence, set z =
[z0 , … , zn0 −1 ], if the observation is z = 𝝆, it is
P[cn = 0|z = 𝝆]
𝓁n = ln (11.440)
P[cn = 1|z = 𝝆]
For the above model (11.439), from (6.39), it is
2
𝓁n = 𝜌n (11.441)
𝜎dec
2

11.9.2 Tanner graph

The relationship between the input bits {𝑣n } and the output bits (code bits) {cn } is defined by the trans-
formation matrix T n0 , and it may be represented by a Tanner graph, similar to the graph of LDPC codes.
The decoding algorithms for polar codes, as discussed in Section 11.9.3, operate on this graph by pass-
ing messages along the edges, however, in a way different from LDPC message passing algorithms.
This section describes the construction of the Tanner graph. We start with the example of T 8 and then
generalize the procedure.

Example 11.9.2
Consider T 8 with 𝜖0 = log2 8 = 3, as depicted in Figure 11.47. The graph consists of n0 = 8 variable
nodes at the left, corresponding to the 8 input bits {𝑣n }, and n0 = 8 variable nodes at the right, cor-
responding to the 8 code bits {cn }, n = 0, 1, … , 7. In between there are 𝜖0 = 3 stages, connected by
intermediate variable nodes {𝑣(s) (0) (3)
n }, s = 1, 2. For convenience, we define {𝑣n = 𝑣n } and {𝑣n = cn };
(s−1) (s)
this way for stage s, the input bits are {𝑣n } and the output bits are 𝑣n , see also Figure 11.48. This
gives an overall of n0 (𝜖0 + 1) = 8 ⋅ 4 variables nodes in the graph. We now have a closer look at the
relations defined by each stage.
In stage 1, pairs of input bits are processed as
[ ]
1 0
[𝑣(0)
2i
𝑣(0)
2i+1
] ⋅ = [𝑣(1)
2i
𝑣(1)
2i+1
]
1 1
for i = 0, 1, 2, and 3: the first output bit is equal to the sum of the two input bits, and the second output
bit is equal to the second input bit.
In stage 2, pairs of blocks of two input bits are processed. Consider first the two input-blocks [𝑣(1)
0
𝑣(1)
1
]
(1) (1) (2) (2) (2) (2)
and [𝑣2 𝑣3 ], and the two output-blocks [𝑣0 𝑣1 ] and [𝑣2 𝑣3 ]. Their relation is given by
[𝑣0(2) 𝑣1(2) ] = [𝑣(1)
0
𝑣(1)
1
] ⊕ [𝑣(1)
2
𝑣(1)
3
]
[𝑣2(2) 𝑣3(2) ] = [𝑣(1)
2
𝑣(1)
3
]
632 Chapter 11. Channel codes

υ0(1) υ0(2)
υ0 = υ0(0) υ0(3) = c0

υ1 = υ1(0) υ1(3) = c1

υ2 = υ2(0) υ2(3) = c2

υ7 = υ7(0) υ7(3) = c7
υ7(1) υ7(2)

stage 1 stage 2 stage 3

Figure 11.47 Tanner graph for transformation matrix T 8 .

(s–1) (s)
υi υi

(s–1) (s)
υj υj

Figure 11.48 One T 2 -block at stage s of the Tanner graph.

where the addition is element-wise. This can equivalently be formulated as the application of T 2 to the
first bits of the two input blocks,
[ ]
1 0
[𝑣(1)
0
𝑣(1)
2
] ⋅ = [𝑣(2)
0
𝑣(2)
2
]
1 1
and its application to the second bits of the two input blocks,
[ ]
(1) (1) 1 0
[𝑣1 𝑣3 ] ⋅ = [𝑣(2)
1
𝑣(2)
3
]
1 1

In the same way, the four input bits 𝑣(1)

4
, 𝑣(1)
5
, 𝑣6(1) , and 𝑣(1)
7
, are transformed to obtain the four output bits
(2) (2) (2) (2)
𝑣4 , 𝑣5 , 𝑣6 , and 𝑣7 .
In stage 3, two blocks of four input bits each is processed. Similarly to stage 2, the operation may be
written using vectors,
[𝑣0(3) 𝑣1(3) 𝑣2(3) 𝑣3(3) ] = [𝑣(2)
0
𝑣(2)
1
𝑣(2)
2
𝑣(2)
3
] ⊕ [𝑣(2)
4
𝑣(2)
5
𝑣(2)
6
𝑣(2)
7
]
[𝑣4(3) 𝑣5(3) 𝑣6(3) 𝑣7(3) ] = [𝑣(2)
4
𝑣(2)
5
𝑣(2)
6
𝑣(2)
7
]
or equivalently using T 2 -blocks,
[ ]
1 0
[𝑣(2)
i
𝑣(2)
i+4
] ⋅ = [𝑣(3)
i
𝑣(3)
i+4
] (11.442)
1 1
11.9. Polar codes 633

for i = 0, 1, 2, and 3: the first bits of the two input blocks are combined, the second bits of the two input
blocks are combined, etc.
Going through the graph from right to left reveals the recursive nature of the overall transform. At
stage 3, the output vector of length 8 is obtained by combining two input vectors of length 4. At stage 2,
each output vector of length 4 is obtained by combining two input vectors of length 2. And at stage 1,
each output vector of length 2 is obtained by combining single input bits (vectors of length 1). This
recursive structure of the Tanner graph reflects the Kronecker product in the transformation matrix T 8 ,
as given in (11.442), each stage of the graph corresponding to one of the kernels T 2 .
The example motivates the general construction.14 Assume that we want to construct the graph of
T n0 : this graph has 𝜖0 = log2 n0 stages and an overall of n0 (𝜖0 + 1) variable nodes, denoted by {𝑣(s) n },
n = 0, 1, … , n0 − 1, s = 0, 1, … , 𝜖0 .
At stage s, there are 2𝜖0 −s output blocks of length 2s bits {𝑣(s)
n }, and each output block is formed by
combining two input blocks of length 2s−1 bits {𝑣n(s−1) }. For each output block, the two respective input
blocks are combined as given in the example, namely the first bits of the two input blocks are combined
according to T 2 , the second bits of the two input blocks are combined according to T 2 , etc.
This property is used to formulate the algorithm to build the Tanner graph. The relation between input
bits and output bits is given by
[ ]
1 0
[𝑣(s−1)
i
𝑣(s−1)
j
] ⋅ = [𝑣(s)
i
𝑣(s)
j
] (11.443)
1 1
for
i = l + t ⋅ 2s
j = l + t ⋅ 2s + 2s−1 (11.444)
𝜖0 −s
where l = 0, 1, … , 2 − 1 denotes the index within the input block, and t = 0, 1, … , 2
s−1
− 1 denotes
the index of the output block. This T 2 -block of the Tanner graph is shown in Figure 11.48.
The Tanner graph, according to (11.443), may be used to efficiently compute the transformation T n0 ,
for a given input vector, as the computations can be highly parallelized. Further, the Tanner graph pro-
vides the basis for the decoding algorithms of polar codes.

11.9.3 Decoding algorithms

The following sections explain the principle and algorithm of SC decoding and how to perform the more
powerful SCL decoding.
Consider a polar code P(n0 , k0 ) with information set  and frozen set  . The decoding algorithms aim
at maximum-likelihood (ML) decoding, i.e. finding among all codewords in the polar code P(n0 , k0 ) the
one with the largest likelihood:
ĉ ML = argmax P[z = 𝝆|c = C] (11.445)
C∈P

Equivalently, we may search over all valid input sequences 𝒗, i.e. the input vectors with the frozen bits
being zero, and minimize the negative log-likelihood:
𝒗̂ ML = argmin − ln P[z = 𝝆|𝒗 = V] (11.446)
V∶V  =𝟎

To simplify the following derivations, we assume that all input vectors 𝒗 are equiprobable (including
the ones with frozen bits not being zero), and thus that all input bits {𝑣n } are equiprobable (even the

14
The proof is direct from the recursive structure and the details are omitted.
634 Chapter 11. Channel codes

frozen bits). As the decoder searches only over the valid input vectors, i.e. 𝒗 with 𝒗 = 0, this assumption
does not change the decoding outcome. Using this assumption, we can write the decoding rule above as
𝒗̂ ML = argmin − ln P[𝒗 = V|z = 𝝆]
V∶V  =𝟎
∑
n0 −1 (11.447)
= argmin − ln P[𝑣n = Vn |z = 𝝆; 𝑣0 = V0 , 𝑣1 = V1 , … , 𝑣n−1 = Vn−1 ]
V∶V  =𝟎 n=0

Bayes’ rule was applied in the last step to expand the original expression into a chain. Note that (11.447)
coincides with the MAP criterion.
Practical decoding algorithms approximate this optimal decoding rule in order to reduce the decoding
complexity. In SC decoding, the individual terms of the sum are minimized successively; in SC list
decoding, the full expression is used, but the search space is reduced.

Successive cancellation decoding – the principle

The schematic of SC decoding [58] is depicted in Figure 11.49. The principle is as follows: To detect the
first bit, 𝑣0 , given the channel LLRs (the LLRs on the code bits), 𝓵 = [𝓁0 , 𝓁1 , … , 𝓁n0 −1 ], we compute
the a posteriori LLR associated to 𝑣0 ,
𝜆0 = 𝓁𝑣0 (𝓵) (11.448)
and a hard-decision is made to obtain the detected bit 𝑣̂ 0 . If the index 0 is in the frozen set, i.e. if 0 ∈  ,
𝑣̂ 0 is set to the known value zero. Then 𝑣̂ 0 is fed back into the decoding process. To detect the second
bit, 𝑣1 , again, all channel LLRs and the decoded bit 𝑣̂ 0 are used to compute the a posteriori LLR
𝜆1 = 𝓁𝑣1 (𝓵; 𝑣̂ 0 ) (11.449)
and a hard-decision is made to obtain the detected bit 𝑣̂ 1 . Again, if 1 ∈  , 𝑣̂ 1 is set to zero; then 𝑣̂ 1 is
fed back into the decoding process. Similarly, decoding continues for all other input bits.
In general, successively the a posteriori LLRs are computed for all input bits {𝑣n }, n = 0, 1, … , n0 − 1,
𝜆n = 𝓁𝑣n (𝓵; 𝑣̂ 0 , … , 𝑣̂ n−1 ) (11.450)

λ0 υ^0
HD
ℓ0
λ1 υ^1
ℓ1 HD
computation

b^
LLR

ℓn0–1
λn0–1 υ^n0–1
HD

SC decoder

n0 LLRs n0 LLRs n0 bits k0 bits

Figure 11.49 Structure of the polar code SC decoder.

11.9. Polar codes 635

and the detected bits are determined by a hard decision, subject to the frozen set, as
⎧0 if 𝜆n ≥ 0 and n ∈ 
⎪
𝑣̂ n = ⎨1 if 𝜆n < 0 and n ∈  (11.451)
⎪
⎩0 if n ∈  ;
and then the detected bits are fed back into the decoding process.
From the detected input bits, the detections of the information bits {b̂ n } are extracted via the informa-
tion set ,
b̂ = 𝒗̂  (11.452)

This operation corresponds to (11.440) on the encoder side. If an internal CRC is present, then in addi-
tion, the k0,CRC CRC bits have to be removed, corresponding to (11.447) and (11.446) on the encoder
side. The specific use of the CRC bits in SCL decoding will be discussed later on.
During the internal computations of the a posteriori LLRs, the feedback of detected input bits is
equivalent to cancelling the contribution of this input bit to the channel LLRs. Therefore, this decoding
principle has been termed successive cancellation decoding.
SC decoding is prone to error propagation, as with a single wrongly detected bit 𝑣̂ n fed back into
the decoding process, the whole codeword will be in error, and thus decoding cannot recover from any
wrong decision on input bits. There are two ways to deal with this problem: (i) Positions that are likely
to be in error should be in the frozen set such that their values are known by the decoder; this idea is
considered in polar code design, see Section 11.9.4. (ii) Instead of feeding back the detected bit into
the decoding process, both hypotheses may be used for a while and the actual decision may be delayed,
which allows for recovery from some wrong decisions; this idea is employed in SC list decoding, as
addressed later on.

Successive cancellation decoding – the algorithm

SC decoding requires the efficient computation of the a posteriori LLRs given detection of the previous
input bits. This can be performed on the Tanner graph of the polar code. We will use the Tanner graph
as determined in Section 11.9.2, with input bits at the left and code bits at the right. Thus, the channel
LLRs {𝓁n }, corresponding to the code bits {cn }, go into the graph at the right, and the a posteriori LLRs
{𝜆n }, corresponding to the input bits {𝑣n } leave the graph at the left. We will first describe the general
algorithm and then specify it to the example of T 8 .
The algorithm on the graph is based on message passing: LLRs travel through the graph from right
to left; detected bit and their sums, termed partial sums, travel through the graph from left to right. To
each intermediate bit value {𝑣(s) (s) (s)
n } in the graph, we associate its detected value {𝑣̂ n } and its LLR {𝓁n },
(𝜖0 )
see Figure 11.50. The LLRs for the last stage 𝜖0 are initialized as 𝓁n = 𝓁n , n = 0, 1, … , n0 − 1. The
a posteriori LLRs are given by the LLRs of the first stage, 𝜆n = 𝓁n(0) , n = 0, 1, … , n0 − 1. The core of
the algorithm is the operation on a single T 2 -block at stage s, s = 0, 1, … , 𝜖0 , as shown in Figure 11.48,
with indices i and j as specified in (11.458).

(s–1) ^ (s–1) (s) (s)

ℓi , υi υ^i , ℓi

(s–1) ^ (s–1) (s) (s)

ℓj , υj υ^j , ℓj

Figure 11.50 T 2 -block at stage s with LLRs and detected bits.

636 Chapter 11. Channel codes

For application to decoding, we rewrite the input–output constraint (11.457) as

𝑣i(s−1) = 𝑣(s)
i
⊕ 𝑣(s)
j
(11.453)
𝑣j(s−1) = 𝑣i(s−1) ⊕ 𝑣(s)
i
= 𝑣(s)
j
(11.454)
There are three kinds of operations on the values of this T 2 -block [58, 61]:
Computation of first LLR/f-function (right-to-left): Assume that LLRs 𝓁i(s) and 𝓁j(s) , corresponding to
the output bits 𝑣(s)
i
and 𝑣(s)
j
are available. Given the check constraint (11.467), the LLR 𝓁i(s−1) , corre-
sponding to the input bit 𝑣i(s−1) , can be computed as
⎛ 𝓁 (s) 𝓁j(s) ⎞
𝓁i(s−1) = 2 tanh−1 ⎜tanh i ⋅ tanh ⎟
⎜ 2 2 ⎟
⎝ ⎠
≃ sgn𝓁i(s) ⋅ sgn𝓁j(s) ⋅ min{|𝓁i(s) |, |𝓁j(s) |} (11.455)

= f (𝓁i(s) , 𝓁j(s) ) (11.456)

Replacing the exact operation by the f -function has no significant impact on performance but heavily
reduces the complexity. This operation on LLRs is the same as decoding a degree-3 check node of
LDPC codes.
Computation of second LLR/g-function (right-to-left): Assume that the LLRs 𝓁i(s) and 𝓁j(s) are avail-
able, as well as the detected input bit 𝑣̂ (s−1)
i
. Given the equality constraint (11.468), the LLR 𝓁j(s−1) ,
corresponding to the input bit 𝑣(s−1)
j
can be computed as
(s−1)
𝓁j(s−1) = (−1)𝑣̂i ⋅ 𝓁i(s) + 𝓁j(s) = g(𝓁i(s) , 𝓁j(s) ; 𝑣̂ (s−1)
i
) (11.457)
This operation corresponds to decoding a degree-3 variable node of LDPC codes, with the difference
that the sign of 𝓁i(s) needs to be corrected according to the value of the bit 𝑣̂ (s−1)
i
.
Computation of partial sums (left-to-right): Assume that the detected input bits 𝑣̂ (s−1) i
and 𝑣̂ (s−1)
j
are
available. Then the two detected output bits can be computed as
[ ]
(s−1) (s−1) 1 0
[𝑣̂ i 𝑣̂ j ] ⋅ = [𝑣̂ (s)
i
𝑣̂ (s)
j
] (11.458)
1 1
corresponding to (11.457). These values are called partial sums.
Given these three kinds of operations for all (n0 𝜖0 ∕2) T 2 -blocks in the Tanner graph, there is only
one possible scheduling. We will explain this schedule based on an example for T 8 and generalize it
afterwards.

Example 11.9.3
Continuing the previous example with T 8 , we label the Tanner graph from Figure 11.47 with the inter-
mediate LLRs, as shown in Figure 11.51. Note that we keep information set  = {2, 3, 5, 6, 7} and frozen
set  = {0, 1, 4} as before.
The LLRs of the code bits at the right are initialized as {𝓁n(3) = 𝓁n }, n = 0, 1, … , 7. Then the
f -functions and g-functions are applied with the schedule as indicated by the numbers with the hashes;
the computations of the partial sums are not indicated in the graph.
Input bit 0: First, the four f -functions of stage 3 (#1, #2, #3, #4) produce the LLRs 𝓁0(2) , 𝓁1(2) , 𝓁2(2) ,
𝓁3(2) . Then the two upper f -functions of stage 2 (#5, #6) are called to obtain 𝓁0(1) , 𝓁1(1) . And, last, the
uppermost f -function of stage 1 (#7) produces the a posteriori LLR 𝜆0 = 𝓁0(0) . Position 0 is a frozen
position, therefore, we set 𝑣̂ 0 = 0, independently of 𝜆0 .
11.9. Polar codes 637

(1) (2)
ℓ0 ℓ0 (3)
#7 #5 #1
λ0 = ℓ0(0) ℓ0 = ℓ0

#8 #6 #2
λ1 = ℓ1(0) (3)
ℓ1 = ℓ1

#11 #9 #3 (3)
λ2 = ℓ2(0) ℓ2 = ℓ2

#12 #10 #4

#19 #17 #13

#20 #18 #14

#23 #21 #15

#24 #22 #16 (3)

λ7 = ℓ7(0) ℓ7 = ℓ7
(1) (2)
ℓ7 ℓ7

Figure 11.51 Tanner graph for T 8 , as used for decoding.

Input bit 1: The uppermost g-function (#8) uses 𝑣̂ 0 = 0 to compute 𝜆1 and 𝑣̂ 1 . Position 1 is also a frozen
position, therefore, we set 𝑣̂ 1 = 0. From 𝑣̂ (0)0
and 𝑣̂ (0)
1
, the partial sums 𝑣̂ 0(1) and 𝑣̂ 1(1) are computed.
Input bit 2: The two upper g-functions of stage 2 (#9, #10) take 𝑣̂ (1) 0
and 𝑣̂ 1(1) and produce the LLRs 𝓁2(1) ,
(1)
𝓁3 . The second f -function of stage 1 (#11) computes then 𝜆2 . As this is an information position, the
detected bit 𝑣̂ 2 is set to the hard decision of 𝜆2 .
Input bit 3: The second g-function of stage 2 (#12) uses 𝑣̂ 2 to compute 𝜆3 . This is again an information
position, so 𝑣̂ 3 is set to the hard decision of 𝜆3 . Now the partial sums 𝑣̂ (1) 2
and 𝑣̂ (1)
3
can be computed,
and further 𝑣̂ (2)
0
, 𝑣
̂ (2)
1
, 𝑣
̂ (2)
2
, and 𝑣
̂ (2)
3
.
Input bits 4–7: The four g-functions of stage 3 (#13, #14, #15, #16) produce the LLRs 𝓁4(2) , 𝓁5(2) , 𝓁6(2) ,
and 𝓁7(2) . From here on, the procedure is the same as for the upper half of the graph in stages 2 and 1.

Note that all three operations, i.e. f -function, g-function, and partial-sum computation, need to wait
until all required input values are available and then each of them is activated exactly once. This defines
a unique scheduling.

The example shows in detail the scheduling for T 8 . For the general case, the schedule for the LLR
computation (and computation of partial sums) can be determined in a similar way. For fast decoding,
the schedule may be pre-computed offline and stored, rather than determined during decoding.
The algorithm for SC decoding may be improved in terms of complexity or speed. Here are a few
examples: (i) Instead of starting with propagating LLRs from right to left, the algorithm may start with
propagating partial sums due to frozen bits from left to right; this way, a posteriori LLRs of frozen bits
are not computed. (ii) Instead of making decisions on input bits individually, decisions may be made on
blocks of input bits where all bits are frozen or all bits are not frozen [65]. (iii) Extending this idea, small
codes may be identified that can efficiently be decoded in a single step, e.g. [66]. Further enhancements
can be found in literature.
638 Chapter 11. Channel codes

Successive cancellation list decoding

The SC decoder cannot recover from a wrong decision on an input bit. To overcome this deficiency and
improve performance, SC list decoding may be used [59–61].
The principle is as follows: Whenever a detected input-bit is to be fed back into the decoding process,
two instances of the decoder continue from this point onwards, each using one of the bit values. Further
a path metric is associated with each of these paths. The number of paths grows exponentially in the
information positions, and thus needs to be limited to a certain list length L. When the number of paths in
the list exceeds the list length, only the L paths with the best metric are kept and the others are discarded.
Once decoding has reached the end and the paths have length n0 bits, the path with the best metric is
selected. If a CRC is present, we first discard all paths that do not fulfil1 the CRC; then among the
remaining paths we select the one with the best metric.
We look now into the path metric in detail. For convenient notation, we denote a partial path as
𝒗[n] = [𝑣0 , 𝑣1 , … , 𝑣n ] (11.459)
with value V [n] = [V0 , V1 , … , Vn ]. The metric to be used for path selection is as given in the decoding
rule (11.461). For a complete path 𝒗[n0 −1] = V [n0 −1] , the path metric is given by

∑
n0 −1
Γ(V [n0 −1] ) = − ln P[𝑣n = Vn |z = 𝝆; 𝒗[n−1] = V [n−1] ]
n=0
(11.460)
∑
n0 −1
−𝜆n (−1)Vn
= ln(1 + e )
n=0

The first line is taken from (11.461). The second line uses the definition of 𝜆n in (11.464) and the way
to express a probability by the corresponding LLR, as given by (11.395).
To simplify notation, the value of 𝑣n , Vn , will still be denoted as 𝑣n .
The individual terms of the sum in (11.474) are called the path metric increments. For those, we can
distinguish the case where the sign of the LLR corresponds to the bit value, i.e. sgn 𝜆n = (−1)𝑣n , and the
case where it does not, i.e. sgn 𝜆n ≠ (−1)𝑣n . For each case, we then use only the absolute value of the
LLR to determine the value of the path metric increment,
{
−𝜆n (−1)𝑣n ln(1 + e|𝜆n | ) = |𝜆n | + ln(1 + e−|𝜆n | ) for sgn 𝜆n ≠ (−1)𝑣n
ln(1 + e )= (11.461)
ln(1 + e−|𝜆n | ) for sgn 𝜆n = (−1)𝑣n
or with some approximations
{
−𝜆n (−1)𝑣n |𝜆n | for sgn 𝜆n ≠ (−1)𝑣n
ln(1 + e )≃ (11.462)
0 for sgn 𝜆n = (−1)𝑣n
which provides for a simple low-complexity method to compute the path metric.
Assume that the partial path 𝒗[n−1] has path metric Γ(𝒗[n−1] ), the LLR of the current bit position is 𝜆n .
Then the metric of the extended path 𝒗[i] = [𝒗[n−1] , 𝑣n ] can be written as
{
|𝜆n | for sgn 𝜆n ≠ (−1)𝑣n
Γ(𝒗[n] ) = Γ(𝒗[n−1] ) + (11.463)
0 for sgn 𝜆n = (−1)𝑣n
This means that not following the path as suggested by the sign of the LLR leads to a penalization, and
the penalty corresponds to the absolute value of the LLR; both make sense intuitively.
Following the principle of SC list decoding as outlined above and using the path metric and its com-
putation, we can formally describe the SCL decoding algorithm. Assume a polar code P(n0 , k0 ). Assume
11.9. Polar codes 639

further an SCL decoder with list length L; each list entry consists of a partial path 𝒗(l)
[n]
and its metric
(l)
Γ(𝒗[n] ), l = 0, 1, … , L − 1. The SCL decoder operates as follows:

Initialization: Take as input the LLRs {𝓁n } from the channel, n = 0, 1, … , n0 − 1. Initialize the path list
 of size L to the empty set,  = ∅.
Iteration: For input bit position n = 0, 1, … , n0 − 1, perform the following:
LLR calculation and list expansion: If the path list is empty (if and only if n = 0), run the SC decoder
to compute 𝜆0 . Otherwise, for each path 𝒗(l)
[n−1]
in the path list, l = 0, 1, … , ||, take the path out
of the list, run the SC decoder, and compute 𝜆n (𝒗(l)
[n−1]
). If position n is a frozen bit, i.e. if n ∈  ,
extend the path to [𝒗(l)
[n−1]
, 0], compute the path metric Γ ([𝒗(l)
[n−1]
, 0]), and put the new path including
its metric in the list . If position n is an information bit, n ∈ , extend the path to [𝒗(l)
[n−1]
, 0] and
[𝒗(l)
[n−1]
, 1], compute the path metrics Γ([𝒗(l)
[n−1]
, 0]) and Γ([𝒗(l)
[n−1]
, 1]), and put the two new paths
including their metrics in the list . The new path metrics are computed according to (11.477).
List pruning: If the list contains more than L paths, keep the L paths with the smallest path metrics
and discard the others.
Decision: If a CRC is present, discard all paths in  that do not fulfill the CRC. Then select the path
with the smallest path metric. Finally, extract the information bits from the selected input vector and
remove the CRC bits if a CRC is present.

A few comments on this decoding algorithm are appropriate:

1. At the iteration for input–bit position n, the list size remains constant if n ∈  , and it is doubled
if n ∈ . The doubling happens as each existing path l is extended by both possible hypotheses
𝑣(l) (l)
n = 0 and 𝑣n = 1. The step with the list pruning keeps the list at maximum size L.
2. Every path in the list is associated with a decoder memory storing all intermediate LLRs and all
partial sums. After the list-pruning step, it may be that both extensions of a previous path are
kept; in this case, the memory needs to be duplicated. More efficient memory managements and
implementations have been addressed in literature.
3. In the decision step, the CRC acts as a kind of genie, i.e. the CRC is likely to pick the transmitted
codeword from the list, if the transmitted codeword is in the list and has not been discarded before.
4. If a CRC is present and none of the paths in the list fulfill the CRC at the end, there are two options:
(i) the decoder may output the information bits corresponding to the best path, accepting decoding
errors; (ii) the decoder may declare a decoding failure. The choice depends on the application.

The complexity of SCL decoding is about L times larger than that of SC decoding, i.e. of order
O(L n0 ln n0 ), as an SC decoder has to be run for each of the L paths in the list. The error-rate per-
formance, however, is largely improved.

Other decoding algorithms

Various other decoding algorithms have been proposed for polar codes, most based on SC decoding,
among which are the following three: SC stack decoding is similar to SC list decoding, however, not all
paths in the list are extended in every step, and thus the list in general contains paths of different lengths
[67]. In SC flip decoding, the CRC is used to check after SC decoding if decoding was successful; if not,
SC decoding is run again, however, with a different decision on the least reliable position, thus leading to
a new decoding outcome [68]. Belief-propagation decoding, as for LDPC codes, may be directly applied
to the Tanner graph of the polar code, allowing for a high degree of parallelism [69].
640 Chapter 11. Channel codes

11.9.4 Frozen set design

A polar code of length n0 = 2𝜖0 and information length k0 is defined by its information set  of size k0 and
its frozen set  of size (n0 − k0 ). Polar code design is thus concerned with the choice of which n0 − k0 of
the n0 input-bit positions to put into the frozen set and which k0 positions to put into the information set.
The SC decoder cannot make wrong decisions on frozen bits, as they are known to be zero, and we want
that wrong decisions on information bits are unlikely; therefore, unreliable positions should be frozen
and reliable positions should be used to transmit information. This section deals with an algorithm to
determine the reliability of the input-bit positions. A received signal model as in (11.439) is assumed.

Genie-aided SC decoding
The frozen-set design is based on genie-aided SC decoding [58], as follows: we denote the transmitted
input vector by 𝒗̌ and the detected value (after decoding) by 𝒗,
̂ as before. Arbitrary input vectors 𝑣,
̌ with-
out any frozen bits, are assumed to be transmitted over an AWGN channel with a chosen design SNR. For
each input-bit position n, n = 0, 1, … , n0 − 1, decoding is performed and its reliability is determined:
1. The SC decoder runs as shown in Figure 11.49, and the a posteriori LLRs {𝜆n } are computed, as
explained above.
2. The hard-decisions {𝑣̂ n } of {𝜆n } are used to count errors and determine the bit-error rate pn for
position n. Note that no bits are frozen.
3. The correct bit values 𝑣̌ n , i.e. the actually transmitted ones, are fed back into the decoding process.
This is the genie-aided part.
According to this decoding model, the a posteriori LLRs are now
𝜆n = 𝓁𝑣n (𝓵; 𝑣̌ 0 , … , 𝑣̌ n−1 ) (11.464)
with perfect feedback (genie-aided and thus error-free). The hard-decisions are
⎧0 if 𝜆n > 0
⎪
𝑣̂ i = ⎨1 if 𝜆n < 0 (11.465)
⎪
⎩randomly chosen if 𝜆n = 0
and bit errors are counted to determine the position-wise bit-error rates (BERs),
pn = P[𝑣̂ n ≠ 𝑣̌ n ] (11.466)
n = 0, 1, … , n0 − 1.
We aim at minimizing the union bound for the word error rate under SC decoding,
∑
Pe = P[𝒗̂ ≠ 𝒗]
̌ ≤ pn (11.467)
n∈

(The inequality is due to possibly correlated error events.) To do so, we collect the (n0 − k0 ) positions
with the largest BER pn into the frozen set and the k0 positions with the smallest BER pn into the
information set:
 = {n ∈ {0, 1, … , n0 − 1} ∶ {pn } are the (n0 − k0 ) largest values} (11.468)
 = {n ∈ {0, 1, … , n0 − 1} ∶ {pn } are the k0 smallest values} (11.469)
The required position-wise BERs {pn } may be determined by Monte Carlo simulation. Note that due
to binary modulation model and the symmetry of the AWGN channel, it is sufficient to transmit the
all-zero input codevector. Though precise, this is method is time-consuming, as the values {pn } need to
be about as accurate as the overall target BER of the decoder.
11.9. Polar codes 641

Design based on density evolution

A faster method with low complexity is based on density evolution using a Gaussian approximation
(DE/GA). Density evolution and its variant with the Gaussian approximation were originally suggested
for LDPC code design [70] and later proposed to polar code design [71]; further refinements for polar
codes have been proposed in literature.
The concept of DE/GA is as follows. We assume that the all-zero input vector is transmitted, without
loss of generality as explained above; thus all partial sums and all code bits are zero as well. Then on
the decoder side, the LLRs {𝓁n(s) } are considered as random variables with a conditional probability
density given that the corresponding bit value is zero. The goal is to start with the Gaussian pdf of {𝓁n }
to compute the intermediate densities of {𝓁n(s) }, and finally the densities of the a posteriori LLRs {𝜆n };
this process is referred to as density evolution. To simplify the computations, we approximate the LLRs
as Gaussian random variables. As Gaussian LLRs have the property that their variance is twice their
mean value [70], we can fully characterize each distribution by its mean value. Thus, for DE/GA, it is
sufficient to track the mean values of the LLRs.

(s–1) (s)
mi mi

(s–1) (s)
mj mj

Figure 11.52 T 2 -block at stage s with mean values of LLRs.

For the initialization, we need the mean values of the channel LLRs. Based on the channel model
(11.439), the channel LLRs are computed according to (11.450). Under the assumption of the
all-zero-codeword, from the bit-mapping (11.439), the channel output zn has mean +1, and thus the
channel LLR 𝓁n has mean 𝚖n = 2∕𝜎dec 2
for all positions n. Thus, we initialize the DE/GA with
(𝜖 ) 2
𝚖n 0 = 𝚖n = (11.470)
𝜎dec
2

for n = 0, 1, … , n0 − 1.
The computation of the LLRs in the Tanner graph is based on T 2 -blocks, and so is the density evolu-
tion. With reference to Figure 11.50, we have the corresponding mean values of the LLRs as shown in
Figure 11.52. Assume a T 2 -block at stage s with the mean values 𝚖(s)
i
and 𝚖(s)
j
(at the right) are available.
Our goal is to compute the mean values 𝚖i(s−1) and 𝚖j(s−1) (at the left).
The LLR 𝓁n(s−1) is computed by the f -function, corresponding to a check-node in LDPC coding, see
(11.470). Correspondingly, the DE/GA for a check node may be applied,

𝚖i(s−1) = 𝜑−1 (1 − (1 − 𝜑(𝚖(s)

i
)) ⋅ (1 − 𝜑(𝚖(s)
j
))) (11.471)

The function 𝜑 relates the mean value m = E[𝓁] of a Gaussian LLR to the expectation E[tanh(𝓁∕2)],
𝜑 ∶ E[𝓁] → E[tanh(𝓁∕2)], and it is formally defined as
⎧ 1 ∞ (a−𝚖)2
⎪1 − √ ∫−∞ tanh( a2 ) e− 2𝚖 da for m > 0
𝜑(m) = ⎨ 4𝜋m
⎪1 for m = 0
⎩
the function 𝜑−1 (m) is its inverse.
642 Chapter 11. Channel codes

To simplify the computations, the function 𝜑 may be approximated by

{ 2
ea𝚖 +b𝚖 for 0 ≤ m < 𝚖0
𝜑(m) ≃ 𝛾 +𝛽
(11.472)
e 𝛼𝚖 for 𝚖0 ≤ m
with 𝛼 = −0.4527, 𝛽 = 0.0218, 𝛾 = 0.86, a = 0.0564, b = −0.48560, 𝚖0 = 0.867861 [70, 72], where
the parameters have been determined by curve fitting; note that both approximations can be analytically
inverted. Refined approximations of 𝜑(m) have been proposed in literature, particularly for improved per-
formance of long codes under SC decoding, which is very sensitive to the choice of the frozen positions.
SCL decoding is typically more robust to the frozen set design.
The LLR 𝓁j(s−1) is computed by the g-function, corresponding to a variable-node in LDPC coding,
see (11.471). As all feedback bits are zero, all partial sums are zero, and therefore the detected bit is
𝑣̂ i(s−1) = 0. As for decoding, we then simply add two Gaussian LLRs, the mean of 𝓁j(s−1) is given by the
sum of the two mean values on the right,

𝚖j(s−1) = 𝚖(s)
i
+ 𝚖(s)
j
(11.473)

Equations (11.485) and (11.488) are referred to as the density-evolution equations. Applying them to
the Tanner graph from right to left, we can perform the full DE/GA.
When this DE is finished, we set the mean values 𝜇n of the a posteriori LLRs 𝜆n to 𝜇n = 𝚖(0)n . From
the means of the a posteriori LLRs and the Gaussian approximation, we can determine the position-wise
BERs as
√
pn ≃ Q( 𝜇n ∕2) (11.474)

As the BER is continuous in the mean, we may directly use the mean values to determine the frozen set
and the information set:

 = {n ∈ {0, 1, … , n0 − 1} ∶ {𝜇n } are the (n0 − k0 ) smallest values} (11.475)

 = {n ∈ {0, 1, … , n0 − 1} ∶ {𝜇n } are the k0 largest values} (11.476)

This provides a fast and convenient method for the frozen set design.
We will illustrate the approach for the example of the transformation T 8 .

Example 11.9.4
The LLRs involved in the DE are shown in Figure 11.53. Assume the initial channel LLRs are 𝚖n = 2.1.
Then we set 𝚖n(3) = 𝚖n = 2.1, n = 0, 1, … , 7.
Consider stage 3 and first the T 2 -block with the index pair (i, j) = (0, 4). We have 𝚖(3)
0
= 𝚖(3)
4
= 2.1,
and we compute the DE as
𝚖0(2) = 𝜑−1 (1 − (1 − 𝜑(𝚖0(3) )) ⋅ (1 − 𝜑(𝚖4(3) )))
= 𝜑−1 (1 − (1 − 𝜑(2.1)) ⋅ (1 − 𝜑(2.1))) = 0.89 (11.477)
𝚖4 = 𝚖(3)
(2)
0
+ 𝚖4(3) = 2.1 + 2.1 = 4.2
The calculations for the other three T 2 -blocks with index pairs (i, j) being (1, 5), (2, 6), and (3, 7) are
exactly the same, as the input values are the same.
At stage 2 for the T 2 -block with indices (i, j) = (0, 2), we have 𝚖0(2) = 𝚖(2)
2
= 0.89. The DE is com-
puted as
𝚖0(1) = 𝜑−1 (1 − (1 − 𝜑(𝚖0(2) )) ⋅ (1 − 𝜑(𝚖(2)
2
)))
= 𝜑−1 (1 − (1 − 𝜑(0.89)) ⋅ (1 − 𝜑(0.89))) = 0.23 (11.478)
𝚖2(1) = 𝚖(2)
0
+ 𝚖(2)
2
= 0.89 + 0.89 = 1.77
11.9. Polar codes 643

(1) (2)
m0 m0
(3)
μ0 = m 0(0) m0 = m0

(3)
μ1 = m 1(0) m1 = m1

(3)
μ2 = m 2(0) m2 = m2

(3)
μ7 = m 7(0) m7 = m7
(1) (2)
m7 m7

Figure 11.53 Tanner graph for T 8 , as used for DE/GA.

For the T 2 -block with indices (i, j) = (1, 3), we obtain the same result as for the one with (i, j) = (0, 2),
as the input values are the same. Similarly, we continue for the remaining two T 2 -blocks at stage 2.
For stage 1, the DE has to be computed for all four T 2 -blocks individually, as their input values are
different; for each T 2 -block, however, the two input values are identical. Note that input bit 𝑣7 has the
highest reliability, with 𝚖(7)
0
= 8 ⋅ 2.1 = 16.8.
Using 𝜇n = 𝚖n(1) , we obtain the mean values of the a-posteriori LLRs as

[𝜇0 , … , 𝜇7 ] = [0.04, 0.47, 0.69, 3.55, 1.11, 4.89, 6.15, 16.8] (11.479)

Ranking the indices in increasing order of the mean values, and thus in increasing reliability, we get the
ordered index sequence
[0, 1, 2, 4, 3, 5, 6, 7] (11.480)

If we want a polar code with information length k0 = 4, we collect the n0 − k0 = 8 − 4 = 4 first indices
of this sequence into the frozen set and the last k0 = 4 indices into the information set:

 = {0, 1, 2, 4}
 = {3, 5, 6, 7}

This completes the frozen and information set design.

For a general polar code of length n0 = 2𝜖0 and information length k0 , the procedure is similar to this
example, only comprising more stages.

Channel polarization
The channels from an input bit 𝑣n to its corresponding a posteriori LLR 𝜆n are termed bit-channels,
n = 0, 1, … , n0 . The method for the frozen set design gives rise to two theoretical questions: When the
644 Chapter 11. Channel codes

code length n0 tends to infinity, (i) how good and how bad are the bit-channels, and (ii) how many of
the bit-channels are good? The answer was given by Arıkan in his seminal work [58].
For n0 → ∞, the bit-channels tend to become fully reliable (BER pn being zero), or fully unreliable
(BER pn being 1∕2); this phenomenon is referred to as channel polarization. Further, denoting the
capacity of the communication channel with C, the number of reliable channels tend to Cn0 .
Thus, asymptotically, k0 = Cn0 positions can be used for the information bits and the probability of
making wrong decisions on these positions during SC decoding tends to zero; the other positions are
frozen. The resulting code rate tends to R = k0 ∕n0 = Cn0 ∕n0 = C, i.e. to the channel capacity. This gives
the fundamental result that polar codes under SC decoding are capacity-achieving.

11.9.5 Puncturing and shortening

Puncturing and shortening are methods to change the length and rate of a given code, called the mother
code, while using the encoder and decoder of the original code. As from Section 11.4, in puncturing,
certain code bits of the mother code are not transmitted; at the receiver side, the corresponding LLRs
are set to zero, as the probabilities for 0 and 1 are both 1∕2. In shortening, encoding of the mother code
is done such that certain code bits are always zero, and these are not transmitted; at the receiver side, the
corresponding LLRs are set to plus infinity, as the bits are known to be zero.
Polar codes based on the kernel T 2 can be constructed for any information word length k0 but only
for code lengths n0 = 2𝜖0 that are powers of two. Therefore, puncturing and shortening are important
methods in polar coding to obtain code lengths different from powers of two. Further, due to the way
polar codes are encoded and decoded, the frozen set design requires special attention when puncturing
or shortening are applied. This section explains the underlying principles. Further details may be found
in [64, 73, 74].

Puncturing
Consider a polar code P(n0 , k0 ) of length n0 = 2𝜖0 , information length k0 , and rate Rc = k0 ∕n0 to be used
as the mother polar code. By puncturing, we reduce the code length to n′0 < n0 and keep the informa-
tion length k0 ; thus, we obtain a code of rate R′c = k0 ∕n′0 > Rc , which is higher than the rate Rc of the
mother code.
The encoding is performed for the mother polar code and then we puncture p0 = n0 − n′0 code bits,
selected by the index set p ⊂ {0, 1, … , n0 − 1} of size p0 , called the puncturing pattern; i.e. the code
bits, {cn }, n ∈ p , are not transmitted. At the receiver side, the LLRs {𝓁n } for the unpunctured positions,
i.e. for n ∉ p , are computed as usual from the observations {yn }. For the punctured positions, the LLRs
are set to zero, {𝓁n = 0} for n ∈ p . SC-decoding or SC-based decoding is then performed as usual for
the mother polar code.
SC decoding reacts in a special way to puncturing: every channel LLR with value zero produces one
a posteriori LLR with value zero. Instead of a proof, we motivate this by an example. Figures 11.54 and
11.55 show the Tanner graph of the transform T 4 = T 2 ⊗ T 2 , consisting of two stages, the former with
the bits for encoding, the latter with the LLRs for decoding. Assume that c2 was punctured, such that
𝓁2(2) = 𝓁2 = 0. When running the decoder, 𝓁0(1) will become zero since 𝓁2(2) = 0; on the other hand, 𝓁2(1)
will be non-zero according to 𝓁0(2) (unless this LLR is zero as well); see (11.470) and (11.471). For the
same reason, we obtain 𝓁0(0) = 0. Thus, puncturing position 2 leads to the a posteriori LLR 𝜆0 = 0 in
position 0.
Therefore, the positions with zero a posteriori LLR, induced by the puncturing patterns p , need to
be in the frozen set  . Two special structures of puncturing patterns have the property that the index
11.9. Polar codes 645

υ0(1)
υ0 = υ0(0) υ0(2) = c0

υ1(1)
υ1 = υ1(0) υ1(2) = c2

υ2(1)
υ2 = υ2(0) υ2(2) = c2

υ3(1)
υ3 = υ3(0) υ3(2) = c3

Figure 11.54 Tanner graph of T 4 for encoding.

ℓ0(1) = 0
λ0 = ℓ0(0) = 0 ℓ0(2) = ℓ0

ℓ1(1)
λ1 = ℓ1(0) ℓ1(2) = ℓ1

ℓ2(1)
λ2 = ℓ2(0) ℓ2(2) = ℓ2 = 0

ℓ3(1)
λ3 = ℓ3(0) ℓ3(2) = ℓ3

Figure 11.55 Tanner graph of T 4 for decoding with punctured position.

pattern of the zero a posterior LLRs is identical to the puncturing pattern, which is a property convenient
for polar code design. The first structure is called block puncturing, where the first p0 positions are
punctured,
p = {0, 1, … , p0 − 1} (11.481)
The second structure is called bit-reversal puncturing. The bit-reversal permutation is a permutation
of length n0 = 2𝜖0 , where the index n with binary representation (d0 , d1 , … , d𝜖0 −1 ) is mapped to index
𝜋n with binary representation (d𝜖0 −1 , d𝜖0 −2 , … , d0 ) [58]. The bit-reversal puncturing uses the puncturing
pattern
p = {𝜋0 , 𝜋1 , … , 𝜋p0 −1 } (11.482)
For those two puncturing schemes, we have then the simple requirement that p ⊂  .

Shortening
Consider, as before, a polar code P(n0 , k0 ) of length n0 = 2𝜖0 , information length k0 and rate Rc = k0 ∕n0 ,
to be used as the mother polar code. By shortening, we reduce the code length to n′0 = n0 − nS , and we
reduce the information length to k0′ = k0 − nS , shortening by nS positions; thus, we obtain a code of rate
R′c = k0′ ∕n′0 < Rc , which is lower than the rate Rc of the mother code.
For shortening a systematically encoded code, k0 − nS information bits are used to actually transmit
message bits, and the other nS information bits are set to zero; the nS zero information bits are
not transmitted, and only the k0 − nS message-carrying information bits and the parity bits are
646 Chapter 11. Channel codes

transmitted. For systematically encoded polar codes (not discussed here), this method may directly
be applied.
Non-systematically encoded polar codes (as discussed above and often used) require a different
approach. The encoding is performed for the mother polar code, however, nS additional input bits
are frozen in a way such that after encoding, nS code bits are always zero. The index set of these
zero code bits, denoted by s ⊂ {0, 1, … , n0 − 1}, is called the shortening pattern of size nS = |s |.
These code bits are not transmitted. At the receiver side, the LLRs 𝓁n for the positions that are not
shortened, i.e. for n ∉ s , are computed as usual. For the shortened positions, the LLRs are set to be
plus infinity, 𝓁n = +∞ for n ∈ s ; the reason is that the values of these bits are known to be zero, thus
the probabilities for 0 and 1 are 1 and 0, respectively, and correspondingly the LLR is plus infinity.
SC-decoding or SC-based decoding is then performed as usual for the mother polar code.
Similarly to the case of puncturing, shortening has a special effect on SC decoding: every channel
LLR with value infinity produces one a posteriori LLR with value infinity. Again, we provide an example
instead of a proof, and we use Figures 11.54 and 11.56 for illustration. Assume that c3 was shortened,
such that 𝓁3(2) = 𝓁3 = +∞. When running the decoder, 𝓁3(1) will become infinity due to 𝓁3(2) , independent
of 𝓁1(2) ; on the other hand, 𝓁1(1) will be non-zero due to 𝓁1(2) ; see (11.470) and (11.471). For similar reasons,
we obtain 𝓁3(0) = +∞. Thus, shortening position 3 leads to the a posteriori LLR 𝜆3 = +∞ in position 3.
Note that this is the input bit position used to make sure the code bit c3 is zero, such that it can be
shortened.

ℓ0(1)
λ0 = ℓ0(0) ℓ0(2) = ℓ0

ℓ1(1)
λ1 = ℓ1(0) ℓ1(2) = ℓ2

ℓ2(1)
λ2 = ℓ2(0) ℓ2(2) = ℓ2

ℓ3(1) = +∞
λ3 = ℓ3(0) = +∞ ℓ3(2) = ℓ3 = +∞

Figure 11.56 Tanner graph of T 4 for decoding with shortened position.

Similar to puncturing, there are two special structures of shortening patterns s with the property that
the index set of the additionally frozen input bits is identical to the shortening pattern. The structure,
called block shortening, is obtained by the shortening the last nS bits, i.e.

s = {n0 − nS , n0 − nS − 1, … , n0 − 2, n0 − 1} (11.483)

The second, called bit-reversal shortening, is obtained by using the last nS positions of the bit-reversal
permutation, i.e.
s = {𝜋n0 −nS , 𝜋n0 −nS −1 , … , 𝜋n0 −2 , 𝜋n0 −1 } (11.484)

As for puncturing, this gives the simple requirement that s ⊂  .

Note that there are other options for puncturing and shortening patterns with the desired property, e.g.
patterns composed recursively of blocked and bit-reversal patterns.
11.9. Polar codes 647

Frozen set design

We have seen above that for puncturing and shortening of polar codes, we have the constraints that the
puncturing and the shortening pattern for the block and bit-reversal schemes have to be within the frozen
sets,
p ⊂  , s ⊂  (11.485)
Considering these properties, there are two ways to determine the overall frozen set. First, the puncturing
or shortening pattern may be fixed, and then DE/GA may be run, using mean value zero for the punctured
positions and mean value infinity for the shortened positions. This leads to the best performance but
needs to be redone for every puncturing or shortening pattern. Second, the frozen set  determined for
the mother code may be used, and then the union of this frozen set with the puncturing or shortening
pattern be used as the overall frozen set, i.e. p =  ∪ p or s =  ∪ s , respectively. This requires
DE/GA only to be performed for the mother code but typically leads to a performance loss.

11.9.6 Performance
We consider a polar code P(1024, 512) of length n0 = 1024 and rate Rc = 512∕1024 = 1∕2, first without
inner CRC. The information set and frozen set  and  are designed with DE/GA, as described in
Section 11.9.4. The code is transmitted over a 2-PAM (−1, 1) AWGN channel with noise variance 𝜎dec 2
Eb 1 1
(see (11.439)); the SNR is given in N = R 2𝜎 2 . The LLRs of the code bits are computed as given in
0 c dec
(11.450). For decoding, we apply plain SC decoding and SCL decoding with list lengths L = 2, 4, and 8.
Figure 11.57 shows the BER. Going from SC decoding to SCL decoding with list length L = 2 sig-
nificantly improves the performance. Further increase of the list length lowers the error rates at low
SNR. In the high SNR regime, the BER is dominated by the minimum distance of the code, which is
not particularly good, and thus increasing the list length is not beneficial.
Next, we consider a polar code with internal CRC. Code length and information length are again n0 =
1024 and k0 = 512, the CRC has length k0,CRC = 11 bits. As discussed in Section 11.9.1, given k0 = 512
information bits and k0,CRC = 11 CRC bits, the information set  is now of size k0′ = 512 + 11 = 523. As

100
SC
SCL L = 2
10–1 SCL L = 4
SCL L = 8
SCL CRC L = 2
10–2 SCL CRC L = 4
SCL CRC L = 8
10–3
BER

10–4

10–5

10–6

10–7 0 1 2 3 4
Eb/N0 (dB)

Figure 11.57 Performance of a polar code of length n0 = 1024 and rate R = 1∕2 under SC and SCL
decoding, without inner CRC (solid lines) and with inner CRC (dashed lines).
648 Chapter 11. Channel codes

before, the information set and frozen set  and  are designed with DE/GA. We apply SCL decoding
with list lengths L = 2, 4, and 8, and the CRC is exploited in the decoder to select the detected codeword
from the list, as discussed in Section 11.9.3.
Figure 11.57 shows also the BER for this CRC-aided SCL decoding algorithm. For list length L = 2,
the CRC leads to a worse performance than without CRC. This is due to the fact, that with CRC the
number of frozen bits is smaller, and the short list length does not allow to exploit the benefits of the
CRC. For larger list lengths, however, the error rates are clearly lower. In particular, the curves with
CRC show a steeper slope than the curves without CRC, caused by the increased minimum distance of
the overall code, see Section 11.9.1. To take full advantage of this improved distance properties, the list
length has to be long enough. The joint selection of CRC length and list length for SCL decoding is an
important part of the polar code design.
In general, polar codes under SCL decoding achieve a performance comparable to LDPC codes and
turbo codes in the short-length and medium-length regime. Depending on the particular application
and constraints on decoding (like complexity, area, power consumption, latency), polar codes may be
preferable to other codes [62].

11.10 Milestones in channel coding

Figure 11.58 shows the spectral efficiency versus Eb ∕N0 for various codes proposed over time, showing
the slow but inexorable shift towards the Shannon bound.

Uncoded
2.0
spectral efficiency v (bit/s/Hz)

ng
mit

1.9
e codi RS

ng
n li

td
(255,223)
1.8
sof di
Hamming (31,26)
nno

1.7 c o RS
1.6 de (255,223)
Sha

r d Hamming (15,11)
1.5 ha
1.4
1.3 LDPC TC CC Hamming (7,4)
1.2 (8096,3,6) (210,18) (2,1,32) RS
1.1 LDPC (32,64) BCH (31,16)
1.0
TC LDPC
0.9 (216,18) (504,3,6) BCH
Golay (24,12)

0.8 18
Voyager (123,255)
PC(2 )
0.7
0.6 Galileo CC(3,1,6)
0.5
TC CC
0.4 14
(2 ,18) (4,1,15)
0.3 RM (32,6) RM (32,6)
0.2 Mariner

0.1

–1 0 1 2 3 4 5 6 7 8 9 10 11
Eb/N0 (dB)
1948: Uncoded (binary) 1977: Concatenated codes
1950: Hamming codes 1988: CC with big Viterbi decoder
1949/1954: Golay and Reed–Muller (RM) codes 1989: Soft decoding (for RS)
1960: BCH and Reed–Solomon (RS) codes 1993: Turbo codes
1969: Soft decoding (for RM) 1996: LDPC codes
1955/1960: Convolutional and LDPC codes 2008: Polar codes
1967: convolutional code (CC) with Viterbi decoder

Figure 11.58 Spectral efficiency versus Eb ∕N0 for various codes proposed over time. Source:
Courtesy of Prof. Matthieu Bloch.
Bibliography 649

Bibliography

[1] Lin, S. and Costello, D.J. Jr. (1983). Error Control Coding. Englewood Cliffs, NJ: Prentice-Hall.
[2] Blahut, R.E. (1983). Theory and Practice of Error Control Codes. Reading, MA: Addison-Wesley.
[3] Peterson, W.W. and Weldon, E.J. Jr. (1972). Error-Correcting Codes, 2e. Cambridge, MA: MIT
Press.
[4] Wolf, J.K. (1980). Lecture Notes. University of California: San Diego, CA.
[5] Wicker, S.B. and Bhargava, V.K. (eds.) (1994). Reed-Solomon Codes and Their Applications.
Piscataway, NJ: IEEE Press.
[6] Oh, J., Ha, J., Park, H., and Moon, J. (2015). RS-LDPC concatenated coding for the modern tape
storage channel. IEEE Transactions on Communications 64: 59–69.
[7] Cideciyan, R., Furrer, S., and Lantz, M. (2016). Product codes for data storage on magnetic tape.
IEEE Transactions on Magnetics 53: 1–10.
[8] Furrer, S., Lantz, M., Reininger, P. et al. (2017). 201 Gb/in2 recording areal density on sputtered
magnetic tape. IEEE Transactions on Magnetics 54: 1–8.
[9] Gallager, R. (1968). Information Theory and Reliable Communication. New York, NY: Wiley.
[10] Caire, G., Taricco, G., and Biglieri, E. (1998). Bit-interleaved coded modulation. IEEE Transactions
on Information Theory 44: 927–946.
[11] Hagenauer, J. and Hoeher, P. (1989). A Viterbi algorithm with soft-decision output and its applica-
tions. Proceedings of GLOBECOM ’89, Dallas, Texas, USA, pp. 2828–2833.
[12] Fossorier, M.P.C. and Lin, S. (1995). Soft-decision decoding of linear block codes based on ordered
statistics. IEEE Transactions on Information Theory 41: 1379–1396.
[13] Fossorier, M.P.C. and Lin, S. (1998). Soft-input soft-output decoding of linear block codes based
on ordered statistics. Proceedings of GLOBECOM ’98, Sidney, Australia, pp. 2828–2833.
[14] Costello, D.J. Jr., Hagenauer, J., Imai, H., and Wicker, S.B. (1998). Applications of error-control
coding. IEEE Transactions on Information Theory 44: 2531–2560.
[15] Fano, R.M. (1963). A heuristic discussion on probabilistic decoding. IEEE Transactions on Infor-
mation Theory 9: 64–74.
[16] Zigangirov, K. (1966). Some sequential decoding procedures. Problemy Peredachi Informatsii 2:
13–25.
[17] Jelinek, F. (1969). An upper bound on moments of sequential decoding effort. IEEE Transactions
on Information Theory 15: 140–149.
[18] Jelinek, F. (1969). Fast sequential decoding algorithm using a stack. IBM Journal of Research
and Development 13: 675–685.
[19] Wang, F.Q. and Costello, D.J. (1994). Erasure-free sequential decoding of trellis codes. IEEE Trans-
actions on Information Theory 40: 1803–1817.
[20] Lin, C.F. and Anderson, J.B. (1986). M-algorithm decoding of channel convolutional codes. Proc.,
Princeton Conf. Inform. Sci. Syst., Princeton, NJ, USA, pp. 362–365.
[21] Benedetto, S. and Biglieri, E. (1999). Principles of Digital Transmission with Wireless Applications.
New York, NY: Kluwer Academic Publishers.
[22] Forney, G.D. Jr. (1966). Concatenated Codes. Cambridge, MA: MIT Press.
[23] Fossorier, M.P.C., Burkert, F., Lin, S., and Hagenauer, J. (1998). On the equivalence between
SOVA and max-log-MAP decodings. IEEE Communications Letters 2: 137–139.
[24] Berrou, C., Glavieux, A., and Thitimajshima, P. (1993). Near Shannon limit error-correcting coding
and decoding: turbo codes. Proceedings of IEEE International Conference on Communications,
Geneva, Switzerland (23–26 May 1993), pp. 1064–1070.
[25] Berrou, C. and Glavieux, A. (1996). Near optimum error-correcting coding and decoding:
turbo-codes. IEEE Transactions on Communications 44: 1261–1271.
[26] Sklar, B. (1997). A primer on turbo code concepts. IEEE Communications Magazine 35: 94–101.
650 Chapter 11. Channel codes

[27] Benedetto, S. and Montorsi, G. (1996). Unveiling turbo codes: some results on parallel concate-
nated coding schemes. IEEE Transactions on Information Theory 42: 409–428.
[28] 3-rd Generation Partnership Project (3GPP) (2000). Technical Specification Group (TSG), Radio
Access Network (RAN), Working Group 1 (WG1), Multiplexing and channel coding (TDD).
[29] International Telecommunication Union (ITU), Radiocommunication Study Groups (1998). A guide
to digital terrestrial television broadcasting in the VHF/UHF bands.
[30] Consultative Committee for Space Data Systems (CCSDS) (1999). Telemetry Systems (Panel 1),
Telemetry channel coding.
[31] Benedetto, S., Garello, R., and Montorsi, G. (1998). A search for good convolutional codes to be
used in the construction of turbo codes. IEEE Transactions on Communications 46: 1101–1105.
[32] Takeshita, O.Y. and Costello, D.J. Jr. (2000). New deterministic interleaver designs for turbo codes.
IEEE Transactions on Information Theory 46: 1988–2006.
[33] Divsalar, D. and Pollara, F. (1995). Turbo codes for PCS applications. Proceedings of 1995 IEEE
International Conference on Communications, Seattle, Washington, USA, pp. 54–59.
[34] Benedetto, S., Divsalar, D., Montorsi, G., and Pollara, F. (1998). Serial concatenation of interleaved
codes: performance analysis, design, and iterative decoding. IEEE Transactions on Information The-
ory 45: 909–926.
[35] Benedetto, S. and Montorsi, G. (2000). Versatile bandwidth-efficient parallel and serial
turbo-trellis-coded modulation. Proceedings of 2000 International Symposium on Turbo Codes
& Related Topics, Brest, France, pp. 201–208.
[36] Douillard, C., Jezequel, M., Berrou, C. et al. (1995). Iterative correction of intersymbol interference
turbo-equalization. European Transactions on Telecommunications (ETT) 6: 507–511.
[37] Colavolpe, G., Ferrari, G., and Raheli, R. (2000). Noncoherent iterative (turbo) decoding. IEEE
Transactions on Communications 48: 1488–1498.
[38] Bauch, G., Khorram, H., and Hagenauer, J. (1997). Iterative equalization and decoding in mobile
communications system. Proceedings of European Personal Mobile Communications Conference,
Bristol, UK, pp. 307–312.
[39] Hoeher, P. and Lodge, J. (1999). ŞTurbo-DPSKŤ: iterative differential PSK demodulation. IEEE
Transactions on Communications 47: 837–843.
[40] Hagenauer, J., Offer, E., and Papke, L. (1996). Iterative decoding of binary block and convolutional
codes. IEEE Transactions on Information Theory 42: 429–445.
[41] Bauch, G., Khorram, H., and Hagenauer, J. (1997). Iterative equalization and decoding in mobile
communications systems. Proceedings of EPMCC, pp. 307–312.
[42] Picart, A., Didier, P., and Glavieux, G. (1997). Turbo-detection: a new approach to combat chan-
nel frequency selectivity. Proceedings of IEEE International Conference on Communications, pp.
1498–1502.
[43] MacKay, D. and Neal, R. (1996). Near Shannon limit performance of low density parity check
codes. Electronics Letters 32: 1645–1646.
[44] MacKay, D. (1999). Good error-correcting codes based on very sparse matrices. IEEE Transactions
on Information Theory 45: 399–431.
[45] Tanner, R.M. (1981). A recursive approach to low complexity codes. IEEE Transactions on Infor-
mation Theory 27: 533–547.
[46] Frey, B.J. (1998). Graphical Models for Machine Learning and Digital Communications. Cam-
bridge, MA: MIT Press.
[47] Richardson, T.J. and Urbanke, R.L. (2001). Efficient encoding of low-density parity-check codes.
IEEE Transactions on Information Theory 47: 638–656.
[48] Gallager, R. (1963). Low Density Parity Check Codes. Cambridge, MA: MIT Press.
Bibliography 651

[49] Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Interference.
Morgan Kaufmann.
[50] Hu, X.-Y., Eleftheriou, E., and Arnold, D.M. (2001). Progressive edge-growth Tanner graphs.
Proceedings of GLOBECOM ’01, San Antonio, Texas, USA.
[51] Fossorier, M.P.C., Mahaljevic, M., and Imai, H. (1999). Reduced complexity iterative decoding
of low-density parity check codes based on belief propagation. IEEE Transactions on Computers
47: 673–680.
[52] Zimmermann, E., Rave, W., and Fettweis, G. (2005). Forced convergence decoding of LDPC codes
- EXIT chart analysis and combination with node complexity reduction techniques. 11th European
Wireless Conference (EW’2005), Nicosia, Cyprus.
[53] Zimmermann, E., Pattisapu, P., and Fettweis, G. (2005). Bit-flipping post-processing for forced con-
vergence decoding of LDPC codes. 13th European Signal Processing Conference, Antalya, Turkey.
[54] Shin, K. and Lee, J. (2007). Low complexity LDPC decoding techniques with adaptive selection
of edges. 2007 IEEE 65th Vehicular Technology Conference - VTC2007-Spring, pp. 2205–2209.
[55] Lee, D.-U., Luk, W., Wang, C., and Jones, C. (2004). A flexible hardware encoder for low-density
parity-check codes. 12th Annual IEEE Symposium on Field-Programmable Custom Computing
Machines, pp. 101–111.
̈
[56] Cherubini, G., Eleftheriou, E., and Olcer, S. (2000). On advanced signal processing and coding
techniques for digital subscriber lines. Records of the Workshop ŞWhat is next in xDSL?Ť, Vienna,
Austria.
[57] Richardson, T.J., Shokrollahi, M.A., and Urbanke, R.L. (2001). Design of capacity-approaching
irregular low-density parity-check codes. IEEE Transactions on Information Theory 47 2: 619–637.
[58] Arıkan, E. (2009). Channel polarization: a method for constructing capacity-achieving codes
for symmetric binary-input memoryless channels. IEEE Transactions on Information Theory 55:
3051–3073.
[59] Tal, I. and Vardy, A. (2015). List decoding of polar codes. IEEE Transactions on Information
Theory 61: 2213–2226.
[60] Niu, K., Chen, K., Lin, J., and Zhang, Q.T. (2014). Polar codes: primary concepts and practical
decoding algorithms. IEEE Communications Magazine 52: 60–69.
[61] Balatsoukas-Stimming, A. and Parizi, M.B. (2015). LLR-based successive cancellation list decoding
of polar codes. IEEE Transactions on Signal Processing 63: 5165–5179.
[62] Balatsoukas-Stimming, A., Giard, P., and Burg, A. (2017). Comparison of polar decoders
with existing low-density parity-check and turbo decoders. Proceedings of IEEE Wireless Com-
munications and Networking Conference, San Francisco, CA, USA.
[63] Hui, D., Sandberg, S., Blankenship, Y. et al. (2018). Channel coding in 5G new radio: a tutorial
overview and performance comparison with 4G LTE. IEEE Vehicular Technology Magazine 13:
60–69.
[64] Bioglio, V., Condo, C., and Land, I. (2019). Design of polar codes in 5G new radio, arXiv.
[65] Alamdar-Yazdi, A. and Kschischang, F.R. (2011). A simplified successive- cancellation decoder
for polar codes. IEEE Communications Letters 15: 1378–1380.
[66] Sarkis, G., Giard, P., Vardy, A. et al. (2014). Fast polar decoders: algorithm and implementation.
IEEE Journal on Selected Areas in Communications 32: 946–957.
[67] Chen, K., Niu, K., and Lin, J. (2013). Improved successive cancellation decoding of polar codes.
IEEE Transactions on Communications 61: 3100–3107.
[68] Afisiadis, O., Balatsoukas-Stimming, A., and Burg, A. (2014). A low-complexity improved succes-
sive cancellation decoder for polar codes. Proceedings of Asilomar Conference on Signals, Systems,
and Computers, Pacific Grove, California, USA.
652 Chapter 11. Channel codes

[69] Cammerer, S., Ebada, M., Elkelesh, A., and ten Brink, S. (2018). Sparse graphs for belief propaga-
tion decoding of polar codes. Proceedings of IEEE International Symposium on Information Theory
(ISIT), Vail, Colorado, USA.
[70] Chung, S.-Y., Richardson, T.J., and Urbanke, R.L. (2001). Analysis of sum-product decoding
of low-density parity-check codes using a Gaussian approximation. IEEE Transactions on Infor-
mation Theory 47: 657–670.
[71] Trifonov, P. (2012). Efficient design and decoding of polar codes. IEEE Transactions on Communi-
cations 60: 3221–3227.
[72] Ha, J., Kim, J., and McLaughlin, S.W. (2004). Rate-compatible puncturing of low-density
parity-check codes. IEEE Transactions on Information Theory 50: 2824–2836.
[73] Zhang, L., Zhang, Z., Wang, X. et al. (2014). On the puncturing patterns for punctured polar codes.
Proceedings of IEEE International Symposium on Information (ISIT), Honolulu, Hawaii, USA.
[74] Miloslavskaya, V. (2015). Shortened polar codes. IEEE Transactions on Information Theory 61:
4852–4865.

Appendix 11.A Non-binary parity check codes

Assume that code words are sequences of symbols from the finite field GF(q) (see Section 11.2.2), all of
length n. As there are qn possible sequences, the introduction of redundancy in the transmitted sequences
is possible if the number of code words Mc is less than qn .
We denote by c a transmitted sequence of n symbols taken from GF(q). We also assume that the
symbols of the received sequence z are from the same alphabet. We define the error sequence e by the
equation (see (11.12) for the binary case)
z=c+e (11.486)

where + denotes a component by component addition of the vectors in accordance with the rules of
addition in the field GF(q).

Definition 11.16
The number of non-zero components of a vector x is defined as the weight of the vector, denoted by
𝑤(x). ◽

Then 𝑤(e) is equal to the number of errors occurred in transmitting the code word.

Definition 11.17
H
The minimum distance of a code, denoted dmin , is equal to the minimum Hamming distance between all
pairs of code words; i.e. it is the same as for binary codes. ◽

We will give without proof the following propositions, similar to those for binary codes on page 518.
H
1. A non-binary block code with minimum distance dmin can correct all error sequences of weight
⌊ dH −1 ⌋
min
2
or less.
H
2. A non-binary block code with minimum distance dmin can detect all error sequences of weight
H
(dmin − 1) or less.
11.A. Non-binary parity check codes 653

H
As in the binary case, we ask for a relation among the parameters of a code: n, Mc , dmin , and q. It can
H
be proven that for a block code with length n and minimum distance dmin , Mc must satisfy the inequality

⎢ ⎧ ⎛ ⎞
⌊ H ⌋ ⎫⎥
⎢ n ⎪ ( ) ( ) n
d
min
−1
⎪⎥
n n ⎜ ⌊ ⌋ ⎟
Mc ≤ ⎢q ∕ ⎨1 + (q − 1) + 2
(q − 1) + · · · + dH −1 (q − 1) 2
⎬⎥ (11.487)
⎢ ⎪ 1 2 ⎜ min ⎟ ⎪⎥
⎣ ⎩ ⎝ 2 ⎠ ⎭⎦
H
Furthermore, for n and dmin given, it is always possible to find a code with Mc∗ words, where
⌈ { ( ) ( ) ( ) }⌉
∗ n n n 2 n H
dmin −1
Mc = q ∕ 1 + (q − 1) + (q − 1) + · · · + H
(q − 1) (11.488)
1 2 dmin −1

Linear codes

Definition 11.18
A linear code is a block code with symbols from GF(q) for which:

(a) the all zero word is a code word;

(b) any multiple of a code word is a code word;
(c) any linear combination of any two code words is a code word.
◽

Example 11.A.1
A binary group code is a linear code with symbols from GF(2).

Example 11.A.2
Consider a block code of length 5 having symbols from GF(3) with code words
0 0 0 0 0
1 0 0 2 1
0 1 1 2 2
2 0 0 1 2
1 1 1 1 0 (11.489)
2 1 1 0 1
0 2 2 1 1
1 2 2 0 2
2 2 2 2 0
It is easily verified that this code is a linear code.

We give the following two properties of a linear code.

H
1. The minimum distance of the code, dmin , is given as
H
dmin = min 𝑤(c) (11.490)

where c can be any non-zero code word.

Proof. By definition of the Hamming distance between two code words, we get

dH (c1 , c2 ) = 𝑤(c1 + (−c2 )) (11.491)

654 Chapter 11. Channel codes

By Property (b), (−c2 ) is a code word if c2 is a code word; by Property (c), c1 + (−c2 ) must also
H
be a code word. As two code words differ in at least dmin positions, there is a code word of weight
H H
dmin ; if there were a code word of weight less than dmin , this word would be different from the zero
word in fewer than dmin H
positions. ◽

2. If all code words in a linear code are written as rows of an Mc × n matrix, every column is composed
of all zeros, or contains all elements of the field, each repeated Mc ∕q times.

Parity check matrix

Let H be an r × n matrix with coefficients from GF(q), expressed as
H = [A B] (11.492)
where the r × r matrix B is such that det[B] ≠ 0.
A generalized non-binary parity check code is a code composed of all vectors c of length n, with
elements from GF(q), that are the solutions of the equation
Hc = 0 (11.493)
The matrix H is called the generalized parity check matrix.

Propriety 1 of non-binary generalized parity check codes. A non-binary generalized parity check
code is a linear code.

Proof.
(a) The all zero word is a code word, as H0 = 0.
(b) Any multiple of a code word is a code word, because if c is a code word, then Hc = 0. But H(𝛼c) =
𝛼Hc = 0, and therefore 𝛼c is a code word; here 𝛼 is any element from GF(q).
(c) Any linear combination of any two code words is a code word, because if c1 and c2 are two
code words, then H(𝛼c1 + 𝛽c2 ) = 𝛼Hc1 + 𝛽Hc2 = 𝛼0 + 𝛽0 = 0, and therefore 𝛼c1 + 𝛽c2 is a
code word. ◽

Property 2 of non-binary generalized parity check codes. The code words corresponding to
the matrix H = [A B] are identical to the code words corresponding to the parity check matrix
H̃ = [B−1 A, I].

Proof. Same as for the binary case. ◽

The matrices in the form [A I] are said to be in canonical or systematic form.

Property 3 of non-binary generalized parity check codes. A code consists of exactly qn−r = qk code
words.

Proof. Same as for the binary case (see Property 3 on page 521). ◽

The first k = n − r symbols are called information symbols, and the last r symbols are called general-
ized parity check symbols.
11.A. Non-binary parity check codes 655

Property 4 of non-binary generalized parity check codes. A code word of weight 𝑤 exists if and
only if some linear combination of 𝑤 columns of the matrix H is equal to 0.

Proof. c is a code word if and only if Hc = 0. Let cj be the jth component of c and let hi be the ith
column of H; then if c is a code word we have
∑
n
h j cj = 0 (11.494)
j=1

If c is a code word of weight 𝑤, there are exactly 𝑤 non-zero components of c, say cj1 , cj2 , … , cj𝑤 ; then
cj1 hj1 + cj2 hj2 + · · · + cj𝑤 hj𝑤 = 0 (11.495)
thus a linear combination of 𝑤 columns of H is equal to 0. Conversely, if (11.495) is true, then Hc = 0,
where c is a vector of weight 𝑤 with non-zero components cj1 , cj2 , … , cj𝑤 . ◽

H
Property 5 of non-binary generalized parity check codes. A code has minimum distance dmin if some
H H
linear combination of dmin columns of H is equal to 0, but no linear combination of fewer than dmin of
columns of H is equal to 0.

Proof. This property is obtained by combining Property 1 of a linear code and Properties 1 and 4 of a
nonbinary generalized parity check code. ◽

Property 5 is fundamental for the design of non-binary codes.

Example 11.A.3
Consider the field GF (4), and let 𝛼 be a primitive element of this field; Moreover, consider the general-
ized parity check matrix [ ]
1 1 1 1 0
H= (11.496)
1 𝛼 𝛼2 0 1
We find that no linear combination of two columns is equal to 0. However, there are many linear combi-
nations of three columns that are equal to 0, for example h1 + h4 + h5 = 0, 𝛼h2 + 𝛼h4 + 𝛼 2 h5 = 0, …;
H
hence, the minimum distance of this code is dmin = 3.

Code generator matrix

We assume that the parity check matrix is in canonical form; then
cnn−r+1 = −Acn−r
1 (11.497)
and [ ] [ ]
cn−r I
c= 1 = cn−r = GT cn−r (11.498)
cnn−r+1 −A 1 1

The matrix G is called the generator matrix of the code and is expressed as
G = [I, −AT ] (11.499)
so that
cT = (cn−r T
1 ) G (11.500)
Thus, the code words, considered as row vectors, are given as all linear combinations of the rows of the
matrix G. A non-binary generalized parity check code can be specified by giving its generalized parity
check matrix or its generator matrix.
656 Chapter 11. Channel codes

Example 11.A.4
Consider the field GF (4) and let 𝛼 be a primitive element of this field; Moreover, consider the generalized
parity check matrix (11.496). The generator matrix of this code is given by

⎡1 0 0 1 1 ⎤
G=⎢0 1 0 1 𝛼 ⎥ (11.501)
⎢ ⎥
⎣ 0 0 1 1 𝛼2 ⎦
There are 64 code words corresponding to all linear combinations of the rows of the matrix G.

Decoding of non-binary parity check codes

Methods for the decoding of non-binary generalized parity check codes are similar to those for the binary
case. Conceptually, the simplest method consists in comparing the received block of n symbols with each
code word and choosing that code word that differs from the received word in the fewest positions. An
equivalent method for a linear code consists in partitioning the qn possible sequences into qr sets.
The partitioning is done as follows:

Step 1: choose the first set as the set of qn−r = qk code words, c1 , c2 , … , cqk .
Step 2: choose any vector, say 𝜼2 , that is not a code word; then choose the second set as c1 + 𝜼2 , c2 +
𝜼2 , … , cqk + 𝜼2 .
Step i: choose any vector, say 𝜼i , not included in any previous set; choose the ith set as c1 + 𝜼i , c2 +
𝜼i , … , cqk + 𝜼i .

The partitioning continues until all qn vectors are used; each set is called a coset, and the vectors 𝜼i
are called coset leaders. The all zero vector is the coset leader for the first set.

Coset
We give the following properties of the cosets omitting the proofs.

1. Every one of the qn vectors occurs in one and only one coset.
2. Suppose that, instead of choosing 𝜼i as coset leader of the ith coset, we choose another element of
that coset as the coset leader; then the coset formed by using the new coset leader contains exactly
the same vectors as the old coset.
3. There are qr cosets.

Two conceptually simple decoding methods

We now form a coset table by choosing as coset leader for each coset the vector of minimum weight
in that coset. The table consists of an array of vectors, with the ith row in the array being the ith coset;
the coset leaders make up the first column, and the jth column consists of the vectors cj , cj + 𝜼2 , cj +
𝜼3 , … , cj + 𝜼qr .
A method for decoding consists of the following steps:

Step 1: locate the received vector in the coset table.

Step 2: choose the code word that appears as the first vector in the column containing the received
vector.

This decoding method decodes to the closest code word to the received word and the coset leaders are
the correctable error patterns.
11.A. Non-binary parity check codes 657

A modified version of the described decoding method is

Step 1′ : locate the received vector in the coset table and then identify the coset leader of the coset
containing this vector.
Step 2′ : subtract the coset leader from the received vector to find the decoded code word.

Syndrome decoding
Another method of decoding is the syndrome decoding. For any generalized parity, check matrix H and
all vectors z of length n, we define the syndrome of z, s(z), as
s(z) = Hz (11.502)
We can show that all vectors in the same coset have the same syndrome and vectors in different cosets
have different syndromes. This leads to the following decoding method:
Step 1′′ : compute the syndrome of the received vector, as this syndrome identifies the coset in which
the received vector is in, and so identifies the leader of that coset.
Step 2′′ : subtract the coset leader from the received vector to find the decoded code word.
The difficulty with this decoding method is in the second part of step 1′′ , that is identifying the coset
leader that corresponds to the computed syndrome; this step is equivalent to finding a linear combination
of the columns of H which is equal to that syndrome, using the smallest number of columns. Exploiting
the algebraic structure of the generalized parity check matrix for certain classes of codes, the syndrome
provides the coset leader by algebraic means.
Chapter 12

Trellis coded modulation

During the 1980s, an evolution in the methods to transmit data over channels with limited bandwidth
took place, giving origin to techniques for joint coding and modulation that are generally known with
the name of trellis coded modulation (TCM). The main characteristic of TCM lies in joint design of
modulation mapping and coding. The first article on TCM appeared in 1976 by Ungerboeck; later, a
more detailed publication by the same author on the principles of TCM [1] spurred considerable interest
in this topic [2–8], leading to a full development of the theory of TCM.
TCM techniques use multilevel modulation with a set of signals from a one, two, or multi-dimensional
space. The choice of the signals that generate a code sequence is determined by a finite-state encoder.
In TCM, the set of modulation signals is expanded with respect to the set used by an uncoded, i.e.
without redundancy, system; in this manner, it is possible to introduce redundancy in the transmitted
signal without widening the bandwidth. At the receiver, the signals in the presence of additive noise and
channel distortion are decoded by a maximum-likelihood sequence decoder. By simple TCM techniques
using a four-state encoder, it is possible to obtain a coding gain of 3 dB with respect to conventional
uncoded modulation; with more sophisticated TCM techniques, coding gains of 6 dB or more can be
achieved (see Chapter 6).
Errors in the decoding of the received signal sequence are less likely to occur if the waveforms, which
represent the code sequences, are easily distinguishable from each other; in mathematical terms, the
signal sequences, represented in the Euclidean multidimensional space, need to be separated by large
distances. The novelty of TCM is in postulating the expansion of the set of symbols1 in order to provide
the redundancy necessary for the encoding process. The construction of modulation code sequences
that are characterized by a free distance, i.e. the minimum Euclidean distance between code sequences,
that is much larger than the minimum distance between uncoded modulation symbols, with the same
information bit rate, and the same bandwidth and power of the modulated signal, is obtained by the joint
design of encoder and bit mapper. The term trellis derives from the similarity between state transition
diagrams of a TCM encoder and trellis diagrams of binary convolutional codes; the difference lies in
the fact that, in TCM schemes, the branches of the trellis are labelled with modulation symbols rather
than binary symbols.

1
In the first part of this chapter, we mainly use the notion of symbols of an alphabet with cardinality M, although the
analysis could be conducted by referring to vectors in the signal space as modulation signals. We will use the term
signals instead of symbols only in the multidimensional case.

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
660 Chapter 12. Trellis coded modulation

The IEEE 802.3ab standard for 1 Gbit/s Ethernet [9] adopts four-dimensional (4-D) 5-pulse-amplitude
-modulation (PAM) trellis-coded modulation (TCM), which uses signal-constellation expansion in con-
junction with set partitioning to perform modulation and coding jointly, thus achieving coding gains
for improved system robustness. The emerging IEEE P802.3bs standard for 400 Gbit/s Ethernet over
eight lanes enables both 2-PAM and 4-PAM operation, while targeting a data rate of 50 Gbit/s per lane
for chip-to-chip and chip-to-module applications. Design techniques for high-speed low-complexity
eight-state 4-D 5-PAM TCM decoding, which exhibit the potential for application to future Ethernet
standards achieving the required reliability by concatenated Reed Solomon (RS) coding and 4-D 5-PAM
TCM, have been presented in [10].

12.1 Linear TCM for one- and two-dimensional signal sets

12.1.1 Fundamental elements

Consider the transmission system illustrated in Figure 6.1, that consists of the modulator, transmission
channel, demodulator, and data detector. Errors occasionally occur in the symbol detection, and âk ≠ ak .
Usually, the simplest data detector is a threshold detector that takes an instantaneous hard decision on the
value âk of the transmitted symbol, based on the observation of the sample zk at the demodulator output.
Detection is of the nearest-neighbour type, i.e. the detector decides for the symbol of the constellation
that is at the minimum Euclidean distance from the received sample zk . The objective of traditional chan-
nel coding techniques consists in detecting and/or correcting the errors present in the detected sequence
of bits {̃cm }.
In the approach followed in Chapter 11, a binary encoder was used to map k0 information binary
symbols {b𝓁 } in n0 code binary symbols {cm }. As mentioned in Section 11.1, we note that, if we want
to maintain the effective rate 1∕Tb of the information message and at the same time the modulation rate
1∕T of the system, we need to increase the cardinality of the modulation alphabet. If we do not consider
the joint design of encoder and bit mapper, however, a reduction of the bit error probability cannot be
efficiently achieved, as we see from the following example.

Example 12.1.1
Consider an uncoded 4-phase-shift-keying (PSK) system and an 8-PSK system that uses a binary error
correcting code with rate 2/3; both systems transmit two information bits per modulation interval, which
corresponds to a spectral efficiency of 2 bit/s/Hz. If the 4-PSK system works with an error probability of
10−5 , for a given signal-to-noise ratio Γ, the 8-PSK system works with an error probability larger than
10−2 , due to the smaller Euclidean distance between signals of the 8-PSK system. We must use an error
correcting code with minimum Hamming distance dmin H
≥ 7 to reduce the error probability to the same
value of the uncoded 4-PSK system. A binary convolutional code with rate 2/3 and constraint length
H
6 has the required value of dfree = 7. Decoding requires a decoder with 64 states that implements the
Viterbi algorithm (VA). However, even after increasing the complexity of the 8-PSK system, we have
obtained an error probability only equal to that of the uncoded 4-PSK system.

Two problems determine the unsatisfactory result obtained with the traditional approach. The first
is originated by the use of independent hard decisions taken by the detector before decoding; hard
input decoding leads to an irreversible loss of information; the remedy is the use of soft decoding (see
page 586), whereby the decoder directly operates on the samples at the demodulator output. The second
derives from the independent design of encoder and bit mapper.
12.1. Linear TCM for one- and two-dimensional signal sets 661

bl encoder ak s(t)
source + MOD channel
Tb BMAP T

bl decoder zk r(t)
user soft
+ DEMOD
Tb IBMAP T

Figure 12.1 Block diagram of a transmission system with trellis coded modulation.

We now consider the transmission system of Figure 12.1, where the transmitted symbol sequence
{ak } is produced by a finite-state machine (FSM) having the information bit sequence {b𝓁 } as input,
possibly with a number of information bits per modulation interval larger than one. We denote by 1∕T
the modulation rate and by  the alphabet of ak . For an additive white Gaussian noise (AWGN) channel,
at the decision point the received samples in the absence of inter-symbol-interference (ISI) are given by
(see (7.334))
zk = ak + 𝑤k (12.1)

where {𝑤k } is a sequence of white Gaussian noise samples. Maximum-likelihood sequence detection
(MLSD) represents the optimum strategy for decoding a sequence transmitted over a dispersive noisy
channel. The decision rule consists in determining the sequence {âk } closest to the received sequence
in terms of Euclidean distance (see (6.25)) in the set  of all possible code symbol sequences. MLSD is
efficiently implemented by the VA, provided that the generation of the code symbol sequences follows
the rules of a FSM.
In relation to (7.355), we define as free distance, dfree , the minimum Euclidean distance between two
code symbol sequences {𝛼k } and {𝛽k }, that belong to the set , given by
∑
2
dfree = min |𝛼k − 𝛽k |2 , {𝛼k }, {𝛽k } ∈  (12.2)
{𝛼k }≠{𝛽k }
k

The most probable error event is determined by two code symbol sequences of the set  at the minimum
distance. The assignment of symbol sequences using a code that is optimized for Hamming distance
does not guarantee an acceptable structure in terms of Euclidean distance, as in general the relation
between the Hamming distance and the Euclidean distance is not monotonic.
Encoder and modulator must then be jointly designed for the purpose of assigning to symbol sequences
waveforms that are separated in the Euclidean signal space by a distance equal to at least dfree , where
dfree is greater than the minimum distance between the symbols of an uncoded system.
At the receiver, the demodulator–decoder does not make errors if the received signal in the Euclidean
signal space is at a distance smaller than dfree ∕2 from the transmitted sequence.

Basic TCM scheme

The objective of TCM is to obtain an error probability lower than that achievable with uncoded modula-
tion, for the same bit rate of the system, channel bandwidth, transmitted signal power, and noise power
spectral density.
The generation of code symbol sequences by a sequential FSM sets some constraints on the symbols of
a sequence, thus introducing interdependence among them (see Appendix 7.B). The transmitted symbol
662 Chapter 12. Trellis coded modulation

at instant kT depends not only on the information bits generated by the source at the same instant, as in
the case of memoryless modulation, but also on the previous symbols.
We define as bk the vector of log2 M information bits at instant kT. We recall that for M-ary uncoded
transmission there exists a one to one correspondence between bk and the symbol ak ∈ . We also
introduce the state sk at the instant kT. According to the model of Appendix 7.B, the generation of a
sequence of encoded symbols is obtained by the two functions
ak = f (bk , sk−1 )
(12.3)
sk = g(bk , sk−1 )
For an input vector bk and a state sk−1 , the first equation describes the choice of the transmitted symbol
ak from a certain constellation, the second the choice of the next state sk .
Interdependence between the symbols {ak } is introduced without a reduction of the bit rate by increas-
ing the cardinality of the alphabet. For example, for a length K of the sequence of input vectors, if we
change  of cardinality M with ′ ⊃  of cardinality M ′ > M, and we select M K sequences as a subset
of (′ )K , a better separation of the code sequences in the Euclidean space may be obtained. Hence,
we can obtain a minimum distance dfree between any two sequences larger than the minimum distance
between signals in K .
Note that this operation may cause an increase in the average symbol energy from Es,u for uncoded
transmission to Es,c for coded transmission, and hence a loss in efficiency given by Es,c ∕Es,u .
Furthermore, we define as Nfree the number of sequences that a code sequence has, on average, at the
distance dfree in the Euclidean multidimensional space.

Example
Suppose we want to transmit two bits of information per symbol. Instead of using quadrature
phase-shift-keying (QPSK) modulation, we can use the scheme illustrated in Figure 12.2.

b(2)
k

bit ak
b(1)
k mapper
(2)
sk–1 (1)
sk–1 sk(0) f
–1
T T T

finite-state-machine

Figure 12.2 Eight-state trellis encoder and bit mapper for the transmission of 2 bits per modulation
interval by 8-PSK.

The scheme has two parts. The first is a finite-state sequential machine with 8 states, where the state sk
is defined by the content of the memory cells sk = [s(2) k
, s(1)
k
, s(0)
k
]. The two bits bk = [b(2)
k
, b(1)
k
] are input
to the FSM, which undergoes a transition from state sk−1 to one of four next possible states, sk , according
to the function g. The second part is the bit mapper, which maps the two information bits and one bit
that depends on the state, i.e. the three bits [b(2)
k
, b(1)
k
, s(0)
k−1
], in one of the symbols of an eight-ary con-
stellation according to the function f , for example an 8-PSK constellation using the map of Figure 12.5.
Note that the transmission of two information bits per modulation interval is achieved. Therefore, the
constellation of the system is expanded by a factor 2 with respect to uncoded QPSK transmission. Recall
from the discussion in Section 6.5 that most of the achievable coding gain for transmission over an ideal
AWGN channel of two bits per modulation interval can be obtained by doubling the cardinality of the
12.1. Linear TCM for one- and two-dimensional signal sets 663

constellation from four to eight symbols. We will see that TCM using the simple scheme of Figure 12.2
achieves a coding gain of 3.6 dB.
For the graphical representation of the functions f and g, it is convenient to use a trellis diagram;
the nodes of the trellis represent the FSM states, and the branches represent the possible transitions
between states. For a given state sk−1 , a branch is associated with each possible vector bk by the function
g, that reaches a next state sk . Each branch is labelled with the corresponding value of the transmitted
symbol ak . For the encoder of Figure 12.2 and the map of Figure 12.5, the corresponding trellis is shown
in Figure 12.3, where the trellis is terminated by forcing the state of the FSM to zero at the instant k = 4.
For a general representation of the trellis, see Figure 12.13.

k =0 1 2 3 4 5
s0 = 0 0
4
1 2 40
2 6 6
2
3 5
1 1
4 7 5
3
5 3
6 7
7

Figure 12.3 Trellis diagram for the encoder of Figure 12.2 and the map of Figure 12.5. Each
branch is labelled by the corresponding value of ak .

Each path of the trellis corresponds to only one message sequence {b𝓁 } and is associated with only
one sequence of code symbols {ak }. The optimum decoder searches the trellis for the most probable
path, given the received sequence {zk } is observed at the output of the demodulator. This search is
usually realized by the VA (see Section 7.5.1). Because of the presence of noise, the chosen path may
not coincide with the correct one, but diverge from it at the instant k = i and rejoin it at the instant
k = i + L; in this case, we say that an error event of length L has occurred, as illustrated in the example
in Figure 12.4 for an error event of length three (see Definition 7.3 on page 358).
Note that in a trellis diagram more branches may connect the same pair of nodes. In this case we speak
of parallel transitions, and by the term free distance of the code we denote the minimum among the
distances between symbols on parallel transitions and the distances between code sequences associated
with pairs of paths in the trellis that originate from a common node and merge into a common node after
L transitions, L > 1.
By utilizing the sequence of samples {zk }, the decoding of a TCM signal is done in two phases. In the
first phase, called subset decoding, within each subset of symbols assigned to the parallel transitions in
the trellis diagram, the receiver determines the symbol closest to the received sample; these symbols are
then memorized together with their square distances from the received sample. In the second phase, we
apply the VA to find the code sequence {âk } along the trellis such that the sum of the square distances
between the code sequence and the sequence {zk } is minimum. Recalling that the signal is obtained at
the output of the demodulator in the presence of AWGN with variance 𝜎I2 per dimension, the probability
of an error event for large values of the signal-to-noise ratio is approximated by (see (7.356))
( )
dfree
Pe ≃ Nfree Q (12.4)
2𝜎I
664 Chapter 12. Trellis coded modulation

ak = 4 1 6 6 4
sk = 0

1
2

7
^a = 4 7 7 0 4
k

Figure 12.4 Section of the trellis for the decoder of an eight-state trellis code. The two continuous
lines indicate two possible paths relative to two 8-PSK signal sequences, {ak } and {̂ak }.

where dfree is defined in (12.2).

From the Definition 6.2 on page 274 and the relation (12.4) between Euclidean distance and error
probability, we give the definition of asymptotic coding gain, Gcode ,2 as the ratio between the minimum
distance, dfree , between code sequences and the minimum Euclidean distance for uncoded sequences,
equal to the minimum distance between symbols of the constellation of an uncoded system, Δ ̃ 0 , normal-
ized by the ratio between the average energy of the coded sequence, Es,c , and the average energy of the
uncoded sequence, Es,u . The coding gain is then expressed in dB as
2
dfree ̃2
∕Δ0
Gcode = 10 log10 (12.5)
Es,c ∕Es,u

12.1.2 Set partitioning

The design of trellis codes is based on a method called mapping by set partitioning. This method requires
that the bit mapper assigns symbol values to the input binary vectors so that the minimum Euclidean
distance between possible code sequences {ak } is maximum. For a given encoder, the search of the
optimum assignment is made by taking into consideration subsets of the symbol set . These subsets are
obtained by successive partitioning of the set , and are characterized by the property that the minimum
Euclidean distance between symbols in a subset corresponding to a certain level of partitioning is larger
than or equal to the minimum distance obtained at the previous level.
Consider the symbol alphabet  = 0 with 2n elements, that corresponds to level zero of partitioning.
At the first level of partitioning, that is characterized by the index q = 1, the set 0 is subdivided into
two disjoint subsets 1 (0) and 1 (1) with 2n−1 elements each. Let Δ1 (0) and Δ1 (1) be the minimum
Euclidean distances between elements of the subsets 1 (0) and 1 (1), respectively; define Δ1 as the
minimum between the two Euclidean distances Δ1 (0) and Δ1 (1); we choose a partition for which Δ1
is maximum. At the level of partitioning characterized by the index q > 1, each of the 2q−1 subsets

2
To emphasize the dependence of the asymptotic coding gain on the choice of the symbol constellations of the coded and
uncoded systems, sometimes the information on the considered modulation schemes is included as a subscript in the
symbol used to denote the coding gain, e.g. G8PSK∕4PSK for the introductory example.
12.1. Linear TCM for one- and two-dimensional signal sets 665

q−1 (𝓁), 𝓁 = 0, 1, … , 2q−1 − 1, is subdivided into two subsets, thus originating 2q subsets. During the
procedure, it is required that the minimum Euclidean distance at the q-th level of partitioning,
Δq = min Δq (𝓁) with Δq (𝓁) = min |𝛼i − 𝛼m | (12.6)
𝓁∈{0,1,…,2q −1}
𝛼i , 𝛼m ∈ q (𝓁)
𝛼i ≠ 𝛼m
is maximum. At the n-th level of partitioning, the subsets n (𝓁) consist of only one element each; to
subsets with only one element we assign the minimum distance Δn = ∞; at the end of the procedure,
we obtain a tree diagram of binary partitioning for the symbol set. At the q-th level of partitioning,
to the two subsets obtained by a subset at the (q − 1)-th level we assign the binary symbols y(q−1) = 0
and y(q−1) = 1, respectively; in this manner, an n-tuple of binary symbols yi = (y(n−1)
i
, … , y(1)
i
, y(0)
i
) is
associated to each element 𝛼i found at an end node of the tree diagram. 3

Therefore, the Euclidean distance between two elements of , 𝛼i and 𝛼m , indicated by the binary
vectors yi and ym that are equal in the first q components, satisfies the relation
(p) (p)
|𝛼i − 𝛼m | ≥ Δq for yi = ym , p = 0, … , q − 1, i≠m (12.7)
In fact, because of the equality of the components in the positions from (0) up to (q − 1), we have that the
two elements are in the same subset q (𝓁) at the q-th level of partitioning. Therefore, their Euclidean
distance is at least equal to Δq .

Example 12.1.2
The partitioning of the set 0 of symbols with statistical power E[|ak |2 ] = 1 for an 8-PSK system is
illustrated in Figure 12.5. The minimum Euclidean distance between elements of the set 0 is given by
Δ0 = 2 sin(𝜋∕8) = 0.765. At the first level of partitioning the two subsets 0 = {(y(2) , y(1) , 0); y(i) = 0, 1}

0 = 8-PSK

Δ0
y(0) =0
1
0 1

Δ1

y(1) = 0 1 0 1
0 2 1 3

Δ2

y(2) = 0 1 0 1 0 1 0 1

000 100 010 110 001 101 011 111

Figure 12.5 Partitioning of the symbol set for an 8-PSK system. Source: Reproduced with
permission from Ungerboeck [1]. ©1982, IEEE.

3
For TCM encoders, the n-tuples of binary code symbols will be indicated by y = (y(n−1) , … , y(0) ) rather than by the
notation c employed in Chapter 11.
666 Chapter 12. Trellis coded modulation

and 1 = {(y √ , y , 1); y = 0, 1} are found, with four elements each and minimum Euclidean dis-
(2) (1) (i)

tance Δ1 = 2. At the second level of partitioning, four subsets 0 = {(y(2) , 0, 0); y(2) = 0, 1}, 2 =
{(y(2) , 1, 0); y(2) = 0, 1}, 1 = {(y(2) , 0, 1); y(2) = 0, 1}, and 3 = {(y(2) , 1, 1); y(2) = 0, 1} are found with
two elements each and minimum Euclidean distance Δ2 = 2. Finally, at the last level eight subsets
D0 , … , D7 , are found, with one element each and minimum Euclidean distance Δ3 = ∞.

Example 12.1.3
The partitioning of the set 0 of symbols with statistical power E[|ak |2 ] = 1 for a 16-quadrature ampli-
tude modulation (QAM) system is illustrated √ in Figure 12.6. The minimum Euclidean distance between
the elements of 0 is given by Δ0 = 2∕ 10 = 0.632. Note that at each successive partitioning√ level,
the minimum Euclidean distance among the elements of a subset increases by a factor 2. Therefore
at the third level of partitioning, the minimum
√ Euclidean distance between the elements of each of the
subsets Di , i = 0, 1, … , 7, is given by Δ3 = 8Δ0 .

A0 = 16-QAM

Δ0
y(0) =0 1
B0 B1

Δ1
y(1) = 0
1 0 1
C0 C2 C1 C3

Δ2

y(2) = 0 1 0 1 0 1 0 1

Δ3

000 100 010 011 001 101 011 111

0000 1000 0100 1100 0010 1010 0110 1110 0001 1001 0101 1101 0011 1011 0111 1111

Figure 12.6 Partitioning of the symbol set for a 16-QAM system. Source: Reproduced with
permission from Ungerboeck [1]. ©1982, IEEE.

12.1.3 Lattices
Several constellations and the relative partitioning can be effectively described by lattices; furthermore,
as we will see in the following sections, the formulation based on lattices is particularly convenient in
the discussion on multidimensional trellis codes.
In general, let ZD = ZD , where Z = ℤ denotes the set of integers; a lattice Λ in ℝD is defined by the
relation { }
Λ= (i1 , … , iD ) G ∣ (i1 , … , iD ) ∈ ZD (12.8)
12.1. Linear TCM for one- and two-dimensional signal sets 667

where G is a non-singular D × D matrix, called lattice generator matrix, by means of which we obtain a
correspondence ZD → Λ. The vectors given by the rows of G form a basis for the lattice Λ; the vectors
of the basis define a parallelepiped whose volume V0 = | det(G)| represents the characteristic volume
of the lattice. The volume V0 is equivalent to the volume of a Voronoi cell associated with an element
or point of lattice Λ and defined as the set of points in ℝD whose distance from a given point of Λ is
smaller than the distance from any other point of Λ. The set of Voronoi cells associated with the points
of Λ is equivalent to the space ℝD . A lattice is characterized by two parameters:
1. dmin , defined as the minimum distance between points of the lattice;
2. the kissing number, defined as the number of lattice points at minimum distance from a given point.
We obtain a subgroup Λq (0) if points of the lattice Λ are chosen as basis vectors in a matrix Gq , such
that they give rise to a characteristic volume Vq = | det(Gq )| > V0 .

Example 12.1.4 (Zp lattice)

In general, as already mentioned, the notation Zp is used to define a lattice with an infinite number of
points in the p-dimensional Euclidean space with coordinates given by integers. The generator matrix G
for the lattice Zp is the p × p identity matrix; the minimum distance is dmin = 1 and the kissing number
is equal to 2p. The Z2 type constellations (see Figure 12.7a) for QAM systems are finite subsets of Z2 ,
with centre at the origin and minimum Euclidean distance equal to Δ0 .

3
2
1

0 1 2 3 4
−1

(a)

2
1

0 1 2 3 4

(b)
Figure 12.7 (a) Z2 lattice; (b) D2 lattice.

Example 12.1.5 (Dn lattice)

Dn is the set of all n-dimensional points whose coordinates are integers that sum to an even number; it
may be regarded as a version of the Zn lattice from which the points whose
√coordinates are integers that
sum to an odd number were removed. The minimum distance is dmin = 2 and the kissing number is
2n(n − 1). The lattice D2 is represented in Figure 12.7b.
668 Chapter 12. Trellis coded modulation

D4 , called the Schläfli lattice, constitutes the densest lattice in ℝ4 ; this means that if four-dimensional
spheres with centres given by the points of the lattice are used to fill ℝ4 , then D4 is the lattice having
the largest number of spheres per unit of volume.

Example 12.1.6 (E8 lattice)

E8 is given by points
{ }
1∑
8
(x1 , … , x8 ) ∣ ∀i ∶ xi ∈ Z or ∀i ∶ xi ∈ Z + x = 0 mod 2 (12.9)
2 i=1 i

In other words, E8 is the set of eight-dimensional points whose components are all integers, or all halves
of odd integers, that sum to an even number. E8 is called the Gosset lattice.

We now discuss set partitioning with the aid of lattices. First we recall the properties of subsets
obtained by partitioning.
If the set  has a group structure with respect to a certain operation (see page 529), the partitioning
can be done so that the sequence of subsets 0 , 1 (0), … , n (0), with q (0) ⊂ q−1 (0), form a chain of
subgroups of 0 ; in this case, the subsets q (𝓁), 𝓁 ∈ {1, … , 2q − 1}, are called cosets of the subgroup
q (0) with respect to 0 (see page 523), and are obtained from the subgroup q (0) by translations. The
distribution of Euclidean distances between elements of a coset q (𝓁) is equal to the distribution of the
Euclidean distances between elements of the subgroup q (0), as the difference between two elements of
a coset yields an element of the subgroup; in particular, for the minimum Euclidean distance in subsets
at a certain level of partitioning it holds

Δq (𝓁) = Δq ∀𝓁 ∈ {0, … , 2q − 1} (12.10)

The lattice Λ in ℝD defined as in (12.8) has a group structure with respect to the addition. With a suitable
translation and normalization, we obtain that the set 0 for PAM or QAM is represented by a subset
of Z or Z2 . To get a QAM constellation from Z2 , we define for example the translated and normalized
lattice Q = c(Z2 + {1∕2, 1∕2}), where c is an arbitrary scaling factor, generally chosen to normalize
the statistical power of the symbols to 1. Figure 12.8 illustrates how QAM constellations are obtained
from Z2 .
If we apply binary partitioning to the set Z or Z2 , we still get infinite lattices in ℝ or ℝ2 , in which
the minimum Euclidean distance increases with respect to the original lattice. Formally, we can assign
the binary representations of the tree diagram obtained by partitioning to the lattices; for transmission,
a symbol is chosen as representative for each lattice at an end node of the tree diagram.

Definition 12.1
The notation X∕X′ denotes the set of subsets obtained from the decomposition of the group X in the
subgroup X′ and its cosets. The set X∕X′ forms in turn a group, called the quotient group of X with
respect to X′ . It is called binary if the number of elements is a power of two. ◽

PAM In this case, the subgroups of the lattice Z are expressed by

q (0) = {2q i ∣ i ∈ Z} = 2q Z (12.11)

Let
t(𝓁) = 𝓁 ∈ {0, … , 2q − 1} (12.12)
12.1. Linear TCM for one- and two-dimensional signal sets 669

256QAM

128QAM

64QAM
32cross
16QAM
QPSK

Figure 12.8 The integer lattice Z2 as template for QAM constellations.

then the cosets of a subgroup, expressed as

q (𝓁) = {a + t(𝓁) ∣ a ∈ q (0)} (12.13)

are obtained by translations of the subgroup. The sequence

Z ∕ 2Z ∕ 22 Z ∕ … ∕ 2q Z ∕ … (12.14)

forms a binary partitioning chain of the lattice Z, with increasing minimum distances given by

Δq = 2q (12.15)

QAM The first subgroup in the binary partitioning chain of the two-dimensional
√ lattice Z2 is a lat-
tice that is obtained from Z2 by rotation of 𝜋∕4 and multiplication by 2. The matrix of this linear
transformation is ( )
1 1
R= (12.16)
−1 1
Successive subgroups in the binary partitioning chain are obtained by repeated application of the linear
transformation R,
q (0) = Z2 Rq (12.17)

Let { }
iZ2 Rq = (ik, im)Rq ∣ k, m ∈ Z , i∈ℕ (12.18)

then the sequence

Z2 ∕ Z2 R ∕ Z2 R2 ∕ Z2 R3 ∕ … = Z2 ∕ Z2 R ∕ 2Z2 ∕ 2Z2 R ∕ … (12.19)

670 Chapter 12. Trellis coded modulation

forms a binary partitioning chain of the lattice Z2 , with increasing minimum distances given by

Δq = 2q∕2 (12.20)

This binary partitioning chain is illustrated in Figure 12.9.

Z2 Z2 R 2Z2 2Z2 R

Figure 12.9 Binary partitioning chain of the lattice Z2 .

The cosets q (𝓁) are obtained by translations of the subgroup q (0) as in the one-dimensional case
(12.13), with
t(𝓁) = (i, m) (12.21)

where, as it can be observed in Figure 12.10 for the case q = 2 with 2 (0) = 2Z2 ,
{ q
} { q
} q
i ∈ 0, … , 2 2 − 1 , m ∈ 0, … , 2 2 − 1 , 𝓁 = 2 2 i + m q even
{ q+1
} { q−1
} q−1 (12.22)
i ∈ 0, … , 2 2 − 1 , m ∈ 0, … , 2 2 − 1 , 𝓁 = 2 2 i + m q odd

2Z2 2Z2 + (0,1) 2Z2 + (1,1) 2Z2 + (1,0)

Figure 12.10 The four cosets of 2Z2 in the partition Z2 ∕2Z2 .

M-PSK In this case, the set of symbols 0 = {ej2𝜋(k∕M) ∣ k ∈ Z} on the unit circle of the complex plane
forms a group with respect to the multiplication. If the number of elements M is a power of two, the
sequence
{ 2q
}
0 ∕ 1 (0) ∕ 2 (0) ∕ … ∕ log2 M (0) with q (0) = ej2𝜋 M k ∣ k ∈ Z (12.23)

forms a binary partitioning chain of the set 0 , with increasing minimum distances
⎧ ( )
2q
⎪ 2 sin 𝜋 for 0 ≤ q < log2 M
Δq = ⎨ M (12.24)
⎪ ∞ for q = log2 M
⎩
The cosets of the subgroups q (0) are given by
{ 𝓁
}
q (𝓁) = a ej2𝜋 M ∣ a ∈ q (0) , 𝓁 ∈ {0, … , 2q − 1} (12.25)
12.1. Linear TCM for one- and two-dimensional signal sets 671

12.1.4 Assignment of symbols to the transitions in the trellis

As discussed in the previous sections, an encoder can be modelled as a finite-state machine with a given
number of states and well-defined state transitions. If the encoder input consists of m binary symbols per
modulation interval,4 then there are 2m possible transitions from a state to the next state; there may be
parallel transitions between pairs of states; furthermore, for reasons of symmetry, we take into consider-
ation only encoders with a uniform structure. After selecting a diagram having the desired characteristics
in terms of state transitions, the design of a code is completed by assigning symbols to the state transi-
tions, such that dfree is maximum. Following the indications of information theory (see Section 6.5), the
symbols are chosen from a redundant set  of 2m+1 elements.

Example 12.1.7 (Uncoded transmission of 2 bits per modulation interval by 8-PSK)

Consider an uncoded 4-PSK system as reference system. The uncoded transmission of 2 bits per modu-
lation interval by 4-PSK can be viewed as the result of the application of a trivial encoder with only one
state, and trellis diagram with four parallel transitions, as illustrated in Figure 12.11. A distinct symbol
of the alphabet of a 4-PSK system is assigned to each parallel transition. Note from Figure 12.5 that
the alphabet of a 4-PSK system is obtained choosing the subset √ 0 (or 1 ) obtained by partitioning the
alphabet 0 of an 8-PSK system. Therefore, dmin = Δ1 = 2. In the trellis diagram, any sequence on
the trellis represents a possible symbol sequence. The optimum receiver decides for the symbol of the
subset 0 (or 1 ) that is found at the minimum distance from the received signal.

2 1
0 0 0
Δ0 = 2
1 1 1
2 2 2
3 3 3 3 0

(a) (b)

Figure 12.11 Uncoded transmission of 2 bits per modulation interval by 4-PSK. (a) Trellis diagram
with a single state and (b) 4-PSK constellation.

Now consider the two-state trellis diagram of Figure 12.12. The symbols of the subsets 0 = 0 ∪ 2
and 1 = 1 ∪ 3 are assigned to the transitions that originate from the first and second state, respec-
tively; this guarantees that the minimum dfree between the code symbol sequences is at least equal to
that obtained for uncoded 4-PSK system. With a trellis diagram with only two states, it is impossi-
ble to have only signals of 0 or 1 assigned to the transitions that originate from a certain state,
and also to all transitions that lead to the same state; therefore, we find that
√ the minimum Euclidean
distance between code symbol sequences is in this case equal to dfree = Δ21 + Δ20 = 1.608, greater
than that obtained for(uncoded 4-PSK transmission. As Es,c = Es,u and Δ̃ 0 = Δ1 , the coding gain is
√ √ )
G8PSK∕4PSK = 20log10 Δ21 + Δ20 ∕ Δ21 = 1.1 dB. Furthermore, Nfree = 2.

4
In this chapter, encoding and bit mapping are jointly optimized and m represents the number of information bits Lb per
modulation interval (see (6.57)). For example, for QAM the rate of the encoder–modulator is RI = m2 .
672 Chapter 12. Trellis coded modulation

C0 C2 0 0
3 2
4
0 42 6
2 dfree = Δ12 +Δ02 4 1
6 Δ0 = 0.765
1 1 = 1.608
C1 C3 5
5 0
1 53 7 3
6 7
7

(a) (b)

Figure 12.12 Transmission of 2 bits per modulation interval using (a) a two-state trellis code and
(b) 8-PSK constellation. For each state, the values of the symbols assigned to the transitions that
originate from that state are indicated.

High coding gains for the transmission of 2 bits per modulation interval by 8-PSK are obtained by
the codes represented in Figure 12.13, with trellis diagrams having 4, 8, and 16 states. For the heuristic
design of these codes with moderate complexity, we resort to the following rules proposed by Unger-
boeck:
1. All symbols of the set 0 must be assigned equally likely to the state transitions in the trellis
diagram, using criteria of regularity and symmetry;
2. To transitions that originate from the same state are assigned symbols of the subset 0 , or symbols
of the subset 1 ;
3. To transitions that merge to the same state are assigned symbols of the subset 0 , or symbols of
the subset 1 ;
4. To parallel transitions between two states we assign the symbols of one of the subsets 0 , 1 , 2 ,
or 3 .
Rule 1 intuitively points to the fact that good trellis codes exhibit a regular structure. Rules 2, 3, and
4 guarantee that the minimum Euclidean distance between code symbol sequences that differ in one or
more elements is at least twice the minimum Euclidean distance between uncoded 4-PSK symbols, so
that the coding gain is greater than or equal to 3 dB, as we will see in the next examples.

Example 12.1.8 (Four-state trellis code for the transmission of 2 bit/s/Hz by 8-PSK)
Consider the code with four states represented in Figure 12.13. Between each pair of code symbol
sequences in the trellis diagram that diverge at a certain
√ state and merge after more than one transition,
the Euclidean distance is greater than or equal to Δ21 + Δ20 + Δ21 = 2.141. For example, this distance
exists between sequences in the trellis diagram labelled by the symbols 0–0–0 and 2–1–2; on the other
hand, the Euclidean distance between symbols assigned to parallel transitions is equal to Δ2 = 2. There-
fore, the minimum Euclidean distance between code symbol sequences is equal to 2; hence, with a
four-state trellis code, we obtain a gain equal to 3 dB over uncoded 4-PSK transmission. Note that, as
the minimum distance dfree is determined by parallel transitions, the sequence at minimum distance from
a transmitted sequence differs only by one element that corresponds to the transmitted symbol rotated
by 180∘ .
A possible implementation of the encoder/bit-mapper for a four-state trellis code is illustrated in
Figure 12.14.
12.1. Linear TCM for one- and two-dimensional signal sets 673

C0 C2 0 0 0
0426 4
C1 C3 6
dfree = Δ2 = 2
1537 2
1
2604 G8PSK/4PSK = 3 dB

3715 Pe ≳ 1 Q (dfree/2σ)

Trellis diagram with four states

(a)
0 0 0
0426
1537 6 dfree = Δ12 + Δ20 + Δ21
4062 = 2.141
5173
6 G8PSK/4PSK = 3.6 dB
2604
3715 7
Pe ≳ 2 Q (dfree/2σ)
6240
7351
Trellis diagram with eight states
(b)
0 0 0 0
0426
1537 6
4062 2
5173 dfree = Δ12 + Δ20 + Δ20 + Δ12
2604 = 2.274
3715
G8PSK/4PSK = 4.1 dB
6240
7351 Pe ≳ 3 Q (dfree/2σ)
4062
5173 1 7

0426
1537
6240
7351
2604
3715
Trellis diagram with sixteen states
(c)

Figure 12.13 Trellis codes with (a) 4, (b) 8, and (c) 16 states for transmission of 2 bits per
modulation interval by 8-PSK. Source: Reproduced with permission from Ungerboeck [1]. ©1982,
IEEE.
674 Chapter 12. Trellis coded modulation

bit-mapper
yk(2)
yk(2) yk(1) yk(0) ak
b(1)
k
0 0 0 0
yk(1) ak
0 0 1 1
b(2)
k
0 1 0 2
(1)
sk– 1
(0)
sk– 1 yk(0)
T T
1 1 1 7

Figure 12.14 Encoder/bit-mapper for a 4-state trellis code for the transmission of 2 bits per
modulation interval by 8-PSK.

The 4-state trellis code for transmission with spectral efficiency of 2 bit/s/Hz by 8-PSK was described
in detail as an introductory example. In Figure 12.13, the values of dfree and Nfree for codes with 4, 8,
and 16 states are reported.
Consider now a 16-QAM system for the transmission of 3 bits per modulation interval; in this case,
the reference system uses uncoded 8-PSK or 8-amplitude modulation (AM)-phase modulation (PM), as
illustrated in Figure 12.15.

Example 12.1.9 (Eight-state trellis code for the transmission of 3 bit/s/Hz by 16-QAM)
The partitioning of a symbol set with unit statistical power for a 16-QAM system is shown in Figure 12.6.
For the assignment of symbols to the transitions on the trellis consider the subsets of the symbol set 0
denoted by D0 , D1 , … , D7 , that
√contain two elements each. The minimum Euclidean distance between
the signals in 0 is Δ0 = 2∕ 10√= 0.632; the minimum Euclidean distance between elements of a
subset Di , i = 0, 1, … , 7, is Δ3 = 8Δ0 .

Δ0 2Δ0

16-QAM constellation 8-AMPM constellation

D0 D2 D4 D6
D1 D7 D5 D3
D6 D4 D2 D0 dfree = 5Δ0
D7 D1 D3 D5
G16QAM 8AMPM = 4 dB
D4 D6 D0 D2
D5 D3 D1 D7
Pe ≳ 3.75 Q (dfree/2σ)
D2 D0 D6 D4
D3 D5 D7 D1

Figure 12.15 Eight-state trellis code for the transmission of 3 bits per modulation interval by
16-QAM.
12.1. Linear TCM for one- and two-dimensional signal sets 675

In the 8-state trellis code illustrated in Figure 12.15, four transitions diverge from each state and four
merge to each state. To each transition one of the subsets Di , i = 0, 1, … , 7, is assigned; therefore, a
transition in the trellis corresponds to a pair of parallel transitions. The assignment of subsets to the
transitions satisfies Ungerboeck rules. The subsets D0 , D4 , D2 , D6 , or D1 , D5 , D3 , D7 are assigned to the
four transitions from or to the same state. In evaluating dfree , this choice guarantees a square Euclidean
distance equal to at least 2Δ20 between sequences that diverge from a state and merge after L transitions,
L > 1. The square distance between sequences that diverge from a state and merge after two transitions
is equal to 6Δ20 . If two sequences diverge and merge again after three or more transitions, at least one
intermediate transition contributes to an incremental square Euclidean distance equal to Δ20 ; thus, the
minimum√Euclidean distance between code symbol sequences that do not differ only for one symbol is
given
√ by 5Δ0 . As the Euclidean distance between √ symbols assigned to parallel transitions is equal to
8Δ0 , the free distance of the code is dfree = 5Δ0 . Because the minimum Euclidean distance for an
√
uncoded 8-AM-PM reference system with the same average symbol energy is ̃ 0 = 2Δ0 , the coding
Δ
√
gain is G16QAM∕8AM−PM = 20 log10 { 5∕2} = 4 dB.
In the trellis diagram of Figure 12.15, four paths are shown that represent error events at minimum dis-
tance from the code sequence, having symbols taken from the subsets D0 − D0 − D3 − D6 ; the sequences
in error diverge from the same state and merge after three or four transitions. It can be shown that for each
code sequence and for each state, there are two paths leading to error events of length three and two of
length four. The number of code sequences at the minimum distance depends on the code sequence being
considered, hence Nfree represents a mean value. The proof is simple in the case of uncoded 16-QAM,
where the number of symbols at the minimum distance from a symbol that is found at the centre of the
constellation is larger than the number of symbols at the minimum distance from a symbol found at the
edge of the constellation; in this case, we obtain Nfree = 3. For the eight-state trellis code of Figure 12.15,
we get Nfree = 3.75. For constellations of type Z2 with a number of signals that tends to infinity, the limit
is Nfree = 4 for uncoded modulation, and Nfree = 16 for coded modulation with an eight-state code.

12.1.5 General structure of the encoder/bit-mapper

(m)
(m) ~
m–m m yk m+1
bk
bk(m–1) yk(m–1)
select signal
within a
~ subset
yk(m + 1) ak
–
bk(m + 1) ~ ~ + 1 y (m~)
– m m k
bk(m) convolutional
encoder (0)
yk select
(1) ~/(m
rate m ~ + 1) subset
bk

bit mapper

Figure 12.16 General structure of the encoder/bit-mapper for TCM.

The structure of trellis codes for TCM can be described in general by the combination of a convolu-
tional encoder and a special bit mapper, as illustrated in Figure 12.16. A code symbol sequence is gen-
erated as follows. Of the m information bits [b(m)
k
, … , b(1)
k
] that must be transmitted during a cycle of the
676 Chapter 12. Trellis coded modulation

encoder/bit-mapper operations, m ̃ ≤ m are input to a convolutional encoder with rate Rc = m∕( ̃ m̃ + 1);
̃ ̃
the m ̃ + 1 bits at the encoder output, ỹ k = [y(km) , … , yk(0) ]T , are used to select one of the 2m+1 subsets
of the symbol set  with 2m+1 elements, and to determine the next state of the encoder. The remaining
(m − m) ̃ uncoded bits determine which of the 2m−m̃ symbols of the selected subset must be transmitted.
For example in the encoder/bit-mapper for a four-state code illustrated in Figure 12.14, the two bits y(0) k
and yk(1) select one of the four subsets 0 , 2 , 1 , 3 of the set 0 with eight elements. The uncoded bit
yk(2) determines which of the two symbols in the subset i is transmitted.
Let yk = [y(m)
k
, … , y(1)
k
, yk(0) ]T be the (m + 1)-dimensional binary vector at the input of the bit-mapper
at the k-th instant, then the selected symbol is expressed as ak = a[yk ]. Note that in the trellis the symbols
of each subset are associated with 2m−m̃ parallel transitions.
The free Euclidean distance of a trellis code, given by (12.2), can be expressed as
{ }
dfree = min Δm+1̃ , dfree (m) ̃ (12.26)

where Δm+1̃ is the minimum distance between symbols assigned to parallel transitions and dfree (m) ̃
denotes the minimum distance between code sequences that differ in more than one symbol. In the par-
̃ = m, each subset has only one element and therefore there are no parallel transitions; this
ticular case m
occurs, for example, for the encoder/ bit-mapper for an 8-state code illustrated in Figure 12.2.
From Figure 12.16, observe that the vector sequence {̃yk } is the output sequence of a convolutional
encoder. Recalling (11.263), for a convolutional code with rate m∕(̃ m̃ + 1) and constraint length 𝜈, we
have the following constraints on the bits of the sequence {̃yk }:
̃
∑
m
h(i) (i) (i) (i) (i) (i)
𝜈 yk−𝜈 ⊗ h𝜈−1 yk−𝜈+1 ⊗ · · · ⊗ h0 yk = 0 ∀k (12.27)
i=0

where {h(i)
j
}, 0 ≤ j ≤ 𝜈, 0 ≤ i ≤ m,̃ are the parity check binary coefficients of the encoder. For an encoder
having 𝜈 binary memory cells, a trellis diagram is generated with 2𝜈 states.
Note that (12.27) defines only the constraints on the code bits, but not the input/output relation of the
encoder.
Using polynomial notation, for the binary vector sequence y(D) (12.27) becomes
[ ] [ ]T
y(m) (D), … , y(1) (D), y(0) (D) h(m) (D), … , h(1) (D), h(0) (D) = 0 (12.28)

where h(i) (D) = h(i) 𝜈 (i) 𝜈−1 + · · · + h(i) D + h(i) , for i = 0, 1, … , m,

̃ and h(i) (D) = 0 for m
̃ <i≤
𝜈 D + h𝜈−1 D 1 0
m. From (12.28), we observe that the code sequences y(D) can be obtained by a systematic encoder
with feedback as5
T
⎡ 0 ⎤
⎡ y(m) (D) ⎤ ⎢ ⋮
⎥
⎢ ⎥ ⎢⎢ ⎥ ⎡ b(m) (D) ⎤
⎥ ⎢
⎢ ⋮ ⎥=⎢I 0
⋮ ⎥
⎢ y(1) (D) ⎥ ⎢ m h(m) ̃ (D)∕h(0) (D) ⎥ ⎢ ⎥
(12.29)
⎢ y(0) (D) ⎥ ⎢ ⎥ ⎣ b (D) ⎦
(1)
⎣ ⎦ ⋮ ⎥
⎢ h (1)
(D)∕h (0)
(D) ⎥
⎣ ⎦
The rational functions h(i) (D)∕h(0) (D), i = 1, … , m,
̃ are realizable if the following condition is satisfied
[11]: { }
0 i≠0
h(i)
0
= h (i)
𝜈 = , 𝜈≥2 (12.30)
1 i=0

5
Note that (12.29) is analogous to (11.298); here the parity check coefficients are used.
12.1. Linear TCM for one- and two-dimensional signal sets 677

The implementation of a systematic encoder with feedback having 𝜈 binary memory elements is illus-
trated in Figure 12.17.

(m) (m)
bk yn

bk(m +1) yk(m +1)

(m) (m)
bk yk

(1) (1)
bk yk

h (1)
ν−1 h (m)
ν−1 h (1)
2 h(m) (1)
2 h1 h (m)
1

(0)
T T T T yk

h (0)
ν−1 h (0)
2 h (0)
1

Figure 12.17 Block diagram of a systematic convolutional encoder with feedback. Source:
Reproduced with permission from Ungerboeck [1]. ©1982, IEEE.

Computation of dfree
Consider the two code sequences
[ ]T
y1 (D) = y(m)
1
(D), … , y1(0) (D) (12.31)
[ ]T
y2 (D) = y(m)
2
(D), … , y2
(0)
(D) (12.32)

related by y2 (D) = y1 (D) ⊗ e(D), as the code is linear. Let ek = [e(m)

k
, … , e(0)
k
], then the error sequence
e(D) is given by
e(D) = ek Dk + ek+1 Dk+1 + · · · + ek+L Dk+L (12.33)
where ei , ei+L ≠ 0, and L > 0. Note that e(D) is also a valid code sequence as the code is linear. To
find a lower bound on the Euclidean distance between the code symbol sequences a1 (D) = a[y1 (D)] and
a2 (D) = a[y2 (D)] obtained from y1 (D) and y2 (D), we define the function d[ek ] = minzk d(a[zk ], a[zk ⊗
ek ]), where minimization takes place in the space of the binary vectors zk = [z(m)
k
, … , zk(1) , zk(0) ]T , and
d(⋅, ⋅) denotes the Euclidean distance between specified symbols. For the square distance between the
sequences a1 (D) and a2 (D) then the relation holds
∑
i+L ( ) ∑i+L
d2 a[yk ], a[yk ⊗ ek ] ≥ d2 [ek ] = d2 [e(D)] (12.34)
k=i k=i

We give the following fundamental theorem [1].

Theorem 12.1
For each sequence e(D) there exists a pair of symbol sequences a1 (D) and a2 (D) for which relation
(12.34) is satisfied with the equal sign. ◽
678 Chapter 12. Trellis coded modulation

Proof. Due to the symmetry in the subsets of symbols obtained by partitioning, d[ek ] = minzk d(a[zk ], a[zk
⊗ ek ]) can arbitrarily be obtained by letting the component z(0)
k
of the vector zk equal to 0 or to 1
and performing the minimization only with respect to components [z(m) k
, … , zk(1) ]. As encoding does
not impose any constraint on the component sequence [y(m) k
, … , y(1)
k
],6 for each sequence e(D) a
code sequence y(D) exists such that the relation (12.34) is satisfied as equality for every value of the
index k. ◽

The free Euclidean distance between code symbol sequences can therefore be determined by a
method similar to that used to find the free Hamming distance between binary code sequences y(D) (see
(11.266)). We need to find an efficient algorithm to examine all possible error sequences e(D) (12.33)
and to substitute the square Euclidean distance d2 [ek ] to the Hamming weight of ek ; thus
∑
i+L
2
dfree ̃ =
(m) min d2 [ek ] (12.35)
e(D)=y2 (D)−y1 (D)≠𝟎
k=i

Let q(ek ) be the number of consecutive components equal to zero in the vector ek , starting with compo-
nent ek(0) . For example, if ek = [e(m)
k
, … , e(3)
k
, 1, 0, 0]T , then q(ek ) = 2. From the definition of the indices
assigned to symbols by partitioning, we obtain that d[ek ] ≥ Δq(ek ) ; moreover, this relation is satisfied as
equality for almost all vectors ek . Note that d[𝟎] = Δq(𝟎) = 0. Therefore,

∑
i+L
2
dfree ̃ ≥ min
(m) ̃
Δ2q(e ) = Δ2free (m) (12.36)
e(D)≠𝟎 k
k=i

If we assume dfree (m)̃ = Δfree (m), ̃ the risk of committing an error in evaluating the free distance of the
code is low, as the minimum is usually reached by more than one error sequence. By the definition of
free distance in terms of the minimum distances between elements of the subsets of the symbol set, the
computation of dfree (m) ̃ will be independent of the particular assignment of the symbols to the binary
vectors with (m + 1) components, provided that the values of the minimum distances among elements
of the subsets are not changed.
At this point, it is possible to identify a further important consequence of the constraint (12.30) on the
binary coefficients of the systematic convolutional encoder. We can show that an error sequence e(D)
begins with ei = (e(m) i
, … , e(1)
i
, 0) and ends with ei+L = (e(m)
i+L
(1)
, … , ei+L , 0). It is therefore guaranteed that
all transitions that originate from the same state and to the transitions that merge at the same state are
assigned signals of the subset 0 or those of the subset 1 . The square Euclidean distance associated
with an error sequence is therefore greater than or equal to 2Δ21 . The constraint on the parity check
coefficients allows however to determine only a lower bound for dfree (m). ̃ For a given sequence Δ0 ≤
Δ1 ≤ · · · ≤ Δm+1̃ of minimum distances between elements of subsets and a code with constraint length
𝜈, a convolutional code that yields the maximum value of dfree (m) ̃ is usually found by a computer program
for code search. The search of the (𝜈 − 1)(m ̃ + 1) parity check binary coefficients is performed by means
such that the explicit computation of dfree (m) ̃ is often avoided.
Tables 12.1 and 12.2 report the optimum codes for TCM with symbols of the type Z1 and Z2 , respec-
tively [2]. For 8-PSK, the optimum codes are given in Table 12.3 [2]. Parity check coefficients are
specified in octal notation; for example, the binary vector [h(0) 6
, … , h(0)
0
] = [1, 0, 0, 0, 1, 0, 1] is repre-
(0)
sented by h = 1058 . In the tables, an asterisk next to the value dfree indicates that the free distance is
determined by the parallel transitions, that is dfree (m) ̃ > Δm+1
̃ .

6
From the parity equation (12.29), we observe that a code sequence {zk } can have arbitrary values for each m-tuple
[z(m)
k
, … , z(1)
k
].
12.2. Multidimensional TCM 679

Table 12.1: Codes for one-dimensional modulation.

G4AM∕2AM G8AM∕4AM Gcode Nfree
2𝜈 ̃
m h1 h0 2
dfree ∕Δ20 (m = 1) (m = 2) (m → ∞) (m → ∞)
4 1 2 5 9.0 2.55 3.31 3.52 4
8 1 04 13 10.0 3.01 3.77 3.97 4
16 1 04 23 11.0 3.42 4.18 4.39 8
32 1 10 45 13.0 4.15 4.91 5.11 12
64 1 024 103 14.0 4.47 5.23 5.44 36
128 1 126 235 16.0 5.05 5.81 6.02 66
Source: Reproduced with permission from Ungerboeck [1]. ©1982, IEEE.

Table 12.2: Codes for two-dimensional modulation.

G16QAM∕8PSK G32QAM∕16QAM G64QAM∕32QAM Gcode Nfree
2𝜈 m
̃ h2 h1 h0 2
dfree ∕Δ20 (m = 3) (m = 4) (m = 5) (m → ∞) (m → ∞)
4 1 — 2 5 4.0∗ 4.36 3.01 2.80 3.01 4
8 2 04 02 11 5.0 5.33 3.98 3.77 3.98 16
16 2 16 04 23 6.0 6.12 4.77 4.56 4.77 56
32 2 10 06 41 6.0 6.12 4.77 4.56 4.77 16
64 2 064 016 101 7.0 6.79 5.44 5.23 5.44 56
128 2 042 014 203 8.0 7.37 6.02 5.81 6.02 344
256 2 304 056 401 8.0 7.37 6.02 5.81 6.02 44
512 2 0510 0346 1001 8.0∗ 7.37 6.02 5.81 6.02 4
Source: Reproduced with permission from Ungerboeck [2]. ©1987, IEEE.

Table 12.3: Codes for 8-PSK.

G8PSK∕4PSK
2𝜈 ̃
m h2 h1 h0 2
dfree ∕Δ20 (m = 2) Nfree
4 1 — 2 5 4.0∗ 3.01 1
8 2 04 02 11 4.586 3.60 2
16 2 16 04 23 5.172 4.13 ≃2.3
32 2 34 16 45 5.758 4.59 4
64 2 066 030 103 6.343 5.01 ≃5.3
128 2 122 054 277 6.586 5.17 ≃0.5
Source: Reproduced with permission from Ungerboeck
[2]. ©1987, IEEE.

12.2 Multidimensional TCM

So far we have dealt with TCM schemes that use two-dimensional (2D) constellations, that is to send m
information bits per modulation interval we employ a 2D constellation of 2(m+1) points; the intrinsic cost
is represented by doubling the cardinality of the 2D constellation with respect to uncoded schemes, as a
bit of redundancy is generated at each modulation interval. Consequently, the minimum distance within
points of the constellation is reduced, for the same average power of the transmitted signal; without this
cost, the coding gain would be 3 dB higher.
An advantage of using a multidimensional constellation to generate code symbol sequences is that
doubling the cardinality of a constellation does not lead to a 3 dB loss; in other words, the signals are
spaced by a larger Euclidean distance dmin and therefore the margin against noise is increased.
680 Chapter 12. Trellis coded modulation

A simple way to generate a multidimensional constellation is obtained by time division. If for example
𝓁 two-dimensional symbols are transmitted over a time interval of duration Ts , and each of them has a
duration Ts ∕𝓁, we may regard the 𝓁 2D symbols as an element of a 2𝓁-dimensional constellation. There-
fore in practice, multidimensional signals can be transmitted as sequences of one or two-dimensional
symbols.
In this section, we describe the construction of 2𝓁-dimensional TCM schemes for the transmission
of m bits per 2D symbol, and thus m𝓁 bits per 2𝓁-dimensional signal. We maintain the principle of
using a redundant signal set, with a number of elements doubled with respect to that used for uncoded
modulation; therefore, the 2𝓁-dimensional TCM schemes use sets of 2m𝓁+1 2𝓁-dimensional signals.
With respect to two-dimensional TCM schemes, this implies a lower redundancy in the two-dimensional
component sets. For example, in the 4D case √ doubling the number of elements causes an expansion of
the 2D component constellations by a factor 2; this corresponds to a half bit of redundancy per 2D
component constellation. The cost of the expansion of the signal set is reduced by 1.5 dB in the 4D case
and by 0.75 dB in the 8D case. A further advantage of multidimensional constellations is that the design
of schemes invariant to phase rotation is simplified (see Section 12.3).

Encoding
Starting with a constellation 0 of the type Z1 or Z2 in the one or two-dimensional signal space, we
consider multidimensional trellis codes where the signals to be assigned to the transitions in the trellis
diagram come from a constellation 0I , I > 2, in the multidimensional space ℝI . In practice, if (m + 1)
binary output symbols of the finite state encoder determine the assignment of modulation signals, it
is possible to associate with these binary symbols a sequence of 𝓁 modulation signals, each in the
constellation 0 , transmitted during 𝓁 modulation intervals, each of duration T; this sequence can be
considered as an element of the space ℝI , as signals transmitted in different modulation intervals are
assumed orthogonal. If we have a constellation 0 with M elements, the relation M 𝓁 ≥ 2m+1 holds.
Possible sequences of 𝓁 symbols of a constellation 0 give origin to a block code B of length 𝓁 in
the space ℝI . Hence, we can consider the multidimensional TCM as an encoding method in which the
binary output symbols of the finite state encoder represent the information symbols of the block code.
Part of the whole coding gain is obtained by choosing a multidimensional constellation 0I with a large
minimum Euclidean distance, that is equivalent to the choice of an adequate block code; therefore, it is
necessary to identify block codes in the Euclidean space ℝI that admit a partitioning of code sequences
in subsets such that the minimum Euclidean distance among elements of a subset is the largest possible.
For a linear block code, it can be shown that this partitioning yields as subsets a subgroup of the code and
̃
its cosets. Consider the trellis diagram of a code for the multidimensional TCM where each state has 2m+1
m−m̃
transitions to adjacent states, and between pairs of adjacent states there exist 2 parallel transitions.
̃
Then, the construction of the trellis code requires that the constellation 0I is partitioned into 2m+1 block
j ̃
codes Bm+1̃
, j ∈ {0, … , 2 m+1
− 1}, each of length 𝓁 and rate equal to (m − m)∕I
̃ bits/dimension. In any
period equivalent to 𝓁 modulation intervals, (m ̃ + 1) binary output symbols of the finite state encoder
select one of the code blocks, and the remaining (m − m) ̃ binary symbols determine the sequence to be
transmitted among those belonging to the selected code. From (6.67), for a constellation 0 of type Z2 ,
Bmin Ts = 𝓁, and M 𝓁 = 2𝓁m+1 , the spectral efficiency of a system with multidimensional TCM is equal
to
m 𝓁 log2 M − 1 1
𝜈= = = log2 M − bit/s/Hz (12.37)
𝓁 𝓁 𝓁
The optimum constellation in the space ℝI is obtained by solving the problem of finding a lattice
such that, given the minimum Euclidean distance dmin between two points, the number of points per
12.2. Multidimensional TCM 681

unit of volume is maximum; for I = 2, the solution is given by the hexagonal lattice. For a number of
constellation points M ≫ 1, the ratio between the statistical power of signals of a constellation of the
type Z2 and that of signals of a constellation chosen as a subset of the hexagonal lattice is equal to
0.62 dB; hence, a constellation subset of the hexagonal lattice yields a coding gain equal to 0.62 dB with
respect to a constellation of the type Z2 .
For I = 4 and I = 8, the solutions are given by the Schläfli lattice D4 and the Gosset lattice E8 , respec-
tively, defined in Examples 12.1.5 and 12.1.6 of page 668. Note that to an increase in the number of
dimensions and in the density of the optimum lattice also corresponds an increase of the number of
lattice points with minimum distance from a given point.
To design codes with a set of modulation signals whose elements are represented by sym-
bol sequences, we have to address the problem of partitioning a lattice in a multidimensional
space.
In the multidimensional TCM, if m ̃ + 1 binary output symbols of the finite state encoder determine the
̃ ̃
next state to a given state, the lattice Λ must be partitioned into 2m+1 ̃ (j), j = 0, … , 2
m+1
subsets Λm+1 − 1,
such that the minimum Euclidean distance Δm+1 ̃ between the elements of each subset is maximum;
hence, the problem consists in determining the subsets of Λ so that the density of the points of Λm+1 ̃ (j),
̃
j = 0, … , 2 m+1
− 1, is maximum. In the case of an I-dimensional lattice Λ that can be expressed as the
Cartesian product of 𝓁 terms all equal to a lattice in the space ℝI∕𝓁 , the partitioning of Λ can be derived
from the partitioning of the I∕𝓁-dimensional lattice.

Example 12.2.1 (Partitioning of the lattice Z4 )

The lattice 04 = Z4 can be expressed in terms of the Cartesian product of the lattice Z2 with itself,
Z4 = Z2 ⊗ Z2 ; the subsets of the lattice 04 are therefore characterized by two sets of signals belonging
to a Z2 lattice and by their subsets. The partitioning of a signal set belonging to a Z2 lattice is represented
in Figure 12.6. Thus,
Z4 = 04 = 0 ⊗ 0 = (0 ∪ 1 ) ⊗ (0 ∪ 1 )
(12.38)
= (0 ⊗ 0 ) ∪ (0 ⊗ 1 ) ∪ (1 ⊗ 0 ) ∪ (1 ⊗ 1 )
At the first level of partitioning, the two optimum subsets are

04 = (0 ⊗ 0 ) ∪ (1 ⊗ 1 ) (12.39)

14 = (0 ⊗ 1 ) ∪ (1 ⊗ 0 ) (12.40)

In terms of the four-dimensional TCM, the assignment of pairs of two-dimensional modulation signals
to the transitions in the trellis diagram is such that in the first modulation interval all the points of the
component QAM constellation are admissible. If two pairs assigned to adjacent transitions are such that
the signals in the first modulation interval are separated by the minimum Euclidean distance Δ0 , the
signals in the second modulation interval will also have Euclidean distance at least √ equal to Δ0 ; in this
way the minimum Euclidean distance among points of 4 or 4 is equal to Δ1 = 2Δ0 . The subset 04
0 1

is the Schläfli lattice D4 , that is the densest lattice in the space ℝ4 ; the subset 14 is instead the coset of
D4 with respect to Z4 . The next partitioning in the four subsets

0 ⊗ 0 , 1 ⊗ 1 , 0 ⊗ 1 , 0 ⊗ 1 (12.41)

does not yield any increase in the minimum Euclidean distance. Note that the four subsets at the second
level of partitioning differ from the Z4 lattice only by the position with respect to the origin, direction,
and scale; therefore, the subsets at successive levels of partitioning are obtained by iterating the same
682 Chapter 12. Trellis coded modulation

procedure described for the first two. Thus, we have the following partitioning chain
Z4 ∕ D4 ∕ (Z2 R)2 ∕ 4D4 ∕ … (12.42)
where R is the 2 × 2 matrix given by (12.16). The partitioning of the Z4 lattice is illustrated in
Figure 12.18 [2].

 04 0 0 Δ0

y0 0 1

 04 ( 0  0) ( 1  1)  14 ( 0 1 ) ( 1 0) 2Δ0

y1 0 1 0 1

0 0 1 1 0 1 1 0 2Δ0

2 1 0
y 0 1 0 1 0 1

44 (0  2) ( 2 C0)  64 ( 1 3) (3 1)  54 ( 0 3 ) (2 1 ) 74 (1 2) (3 0)
 04 (0 0) (2 2)  24 (1 1 ) (3 3) 14 (0  1) (2 3 ) 34 (1 0 ) (3 2) 4Δ0

Figure 12.18 Partitioning of the lattice 04 = Z4 . Source: Reproduced with permission from
Ungerboeck [2]. ©1987, IEEE.

Optimum codes for the multidimensional TCM are found in a similar manner to that described
for the one and two-dimensional TCM codes. Codes and relative asymptotic coding gains for the
four-dimensional TCM, with respect to uncoded modulation with signals of the type Z2 , are reported
in Table 12.4 [2]. These gains are obtained for signal sets with a large number of elements that, in the
signal space, take up the same volume of elements of the signal set used for uncoded modulation; then,
the comparison is made for the same statistical power and the same peak power of the two-dimensional
signals utilized for uncoded modulation.

Decoding
The decoding of signals generated by multidimensional TCM is achieved by a sequence of operations
that is the inverse with respect to the encoding procedure described in the previous section. The first stage
of decoding consists in determining, for each modulation interval, the Euclidean distance between the
received sample and all M signals of the constellation 0 of symbols and also, within each subset q (i)

Table 12.4: Codes for four-dimensional modulation.

Gcode Nfree
2𝜈 m
̃ h4 h3 h2 h1 h0 2
dfree ∕Δ20 (m → ∞) (m → ∞)
8 2 — — 04 02 11 4.0 4.52 88
16 2 — — 14 02 21 4.0 4.52 24
32 3 — 30 14 02 41 4.0 4.52 8
64 4 050 030 014 002 101 5.0 5.48 144
128 4 120 050 022 006 203 6.0 6.28
Source: Reproduced with permission from Ungerboeck [2]. ©1987,
IEEE.
12.2. Multidimensional TCM 683

of the constellation 0 , the signal âk (i) that has the minimum Euclidean distance from the received sam-
̃
ple. The second stage consists in the decoding, by a maximum-likelihood decoder, of 2m+1 block codes
𝓁
̃ (𝓁 ). Due to the large number M of signals in the multidimensional space ℝ , in general the block
Bm+1 ′ I

codes have a number of elements such that the complexity of a maximum-likelihood decoder that should
compute the metric for each element would result excessive. The task of the decoder can be greatly sim-
plified thanks to the method followed for the construction of block codes Bm+1 ′
̃ (𝓁 ). Block codes are
identified by the subsets of the multidimensional constellation I , which are expressed in terms of the
0

Cartesian product of subsets q (i) of the one- or two-dimensional constellation 0 . Decoding of the
block codes is jointly carried out by a trellis diagram defined on a finite number 𝓁 of modulation inter-
vals, where Cartesian products of subsets of the constellation 0 in different modulation intervals are
represented as sequences of branches, and the union of subsets as the union of branches. Figure 12.19
illustrates the trellis diagram for the decoding of block codes obtained by the partitioning into two, four,
and eight subsets of a constellation 04 ⊂ Z4 .

B0 C0 C 04
B04
B0 C2 C2
B1
C 44
C0
B1 C1
B1 C0 C 14
B14
B0 C2 C3 C3
C1 C 54

B0 C1 C 24
C1
C3 C3
B0
B1 C 64
C3
C1
C2
B0 C 34
B1
C0 C0
B1 C2 C 74

Figure 12.19 Trellis diagram for the decoding of block codes obtained by the partitioning the
lattice 04 = Z4 into two, four, and eight subsets.

As the decisions taken in different modulation intervals are independent, the branch metrics of the
trellis diagram is the square Euclidean distance d2 (zk , âk (i)) between the received sample during the k-th
modulation interval and the signal âk (i) of the subset q (i) at the minimum Euclidean distance from zk .
The maximum-likelihood decoder for block codes can then be implemented by the VA, which is applied
̃
for 𝓁 iterations from the initial node of the trellis diagram to the 2m+1 terminal nodes; the procedure to
decode block codes by a trellis diagram is due to Wolf [12]. The third and last decoding stage consists
̃
in using the metrics, obtained at the 2m+1 terminal nodes of the trellis diagram for the decoding of the
block codes, in the Viterbi decoder for the trellis code, that yields the sequence of detected binary vectors
(b̂ (m)
k
, … , b̂ (1)
k
).
To evaluate the complexity of each iteration in multidimensional TCM decoding, it is necessary to
consider the overall number of branches in the trellis diagrams considered at the different decoding
684 Chapter 12. Trellis coded modulation

stages; this number includes the 𝓁M branches for decisions on signals within the subsets q (i) in 𝓁
modulation intervals, the NR branches in the trellis diagram for the decoding of the block codes, and the
2𝜈+m̃ branches in the trellis diagram for the decoding of the trellis code. For a code with efficiency equal
to (log2 M − 1∕𝓁) bits per modulation interval, the complexity expressed as the number of branches in
the trellis diagram per transmitted information bit is thus given by
𝓁M + NR + 2𝜈+m̃
(12.43)
𝓁log2 M − 1
The number NR can be computed on the basis of the particular choice of the multidimensional constella-
tion partitioning chain; for example, in the case of partitioning of the lattice Z4 into four or eight subsets,
for the four-dimensional TCM we obtain NR = 20. Whereas in the case of two-dimensional TCM, the
decoding complexity essentially lies in the implementation of the VA to decode the trellis code, in the
four-dimensional TCM most of the complexity is due to the decoding of the block codes.
In multidimensional TCM, it is common to find codes with a large number Nfree of error events at
the minimum Euclidean distance; this characteristic is due to the fact that in dense multidimensional
lattices, the number of points at the minimum Euclidean distance rapidly increases with the increase in
the number of dimensions. In this case, the minimum Euclidean distance is not sufficient to completely
characterize code performance, as the difference between asymptotic coding gain and effective coding
gain, for values of interest of the error probability, cannot be neglected.

12.3 Rotationally invariant TCM schemes

In PSK or QAM transmission schemes with coherent demodulation, an ambiguity occurs at the receiver
whenever we ignore the value of the carrier phase used at the transmitter. In binary phase-shift-keying
(BPSK), for example, there are two equilibrium values of the carrier phase synchronization system at
the receiver, corresponding to a rotation of the phase equal to 0∘ and 180∘ , respectively; if the rotation
of the phase is equal to 180∘ , the binary symbols at the demodulator output result inverted. In QAM,
instead, because of the symmetry of the Z2 lattice points, there are four possible equilibrium values.
The solution to the problem is represented by the insertion in the transmitted sequence of symbols for
the synchronization, or by differential encoding (see Section 16.1.2). We recall that with differential
encoding the information symbols are assigned to the phase difference between consecutive elements in
the symbol sequence; in this way, the absolute value of phase in the signal space becomes irrelevant to
the receiver.
With TCM, we obtain code signal sequences that, in general, do not present symmetries with respect to
the phase rotation in the signal space. This means that, for a code symbol sequence, after a phase rotation
determined by the phase synchronization system, we may obtain a sequence that does not belong to the
code; therefore, trellis coding not invariant to phase rotation can be seen as a method for the construction
of signal sequences that allow the recovery of the absolute value of the carrier phase. Then, for the
demodulation, we choose the value of the carrier phase corresponding to an equilibrium value of the
synchronization system, for which the Euclidean distance between the received sequence and the code
sequence is minimum.
For fast carrier phase synchronization, various rotationally invariant trellis codes have been developed;
these codes are characterized by the property that code signal sequences continue to belong to the code
even after the largest possible number of phase rotations. In this case, with differential encoding of the
information symbols, the independence of demodulation and decoding from the carrier phase recovered
by the synchronization system is guaranteed. A differential decoder is then applied to the sequence of
Bibliography 685

binary symbols {b̃ 𝓁 } at the output of the decoder for the trellis code to obtain the desired detection of the
sequence of information symbols {b̂ 𝓁 }. Rotationally invariant trellis codes were initially proposed by
Wei [13, 14]. In general, the invariance to phase rotation is more easily obtained with multidimensional
TCM; in the case of two-dimensional TCM for PSK or QAM systems, it is necessary to use non-linear
codes in GF(2).
In the case of TCM for PSK systems, the invariance to phase rotation can be directly obtained using
PSK signals with differential encoding. The elements of the symbol sets are not assigned to the binary
vectors of the convolutional code but rather to the phase differences relative to the previous symbols.

Bibliography

[1] Ungerboeck, G. (1982). Channel coding with multilevel/phase signals. IEEE Transactions on Infor-
mation Theory 28: 55–67.
[2] Ungerboeck, G. (1987). Trellis coded modulation with redundant signal sets. Part I and Part II.
IEEE Communications Magazine 25: 6–21.
[3] Pietrobon, S.S., Deng, R.H., Lafanechere, A. et al. (1990). Trellis-coded multidimensional phase
modulation. IEEE Transactions on Information Theory 36: 63–89.
[4] Biglieri, E., Divsalar, D., McLane, P.J., and Simon, M.K. (1991). Introduction to Trellis-Coded
Modulation with Applications. New York, NY: Macmillan Publishing Company.
[5] Pietrobon, S.S. and Costello, D.J. Jr. (1993). Trellis coding with multidimensional QAM signal sets.
IEEE Transactions on Information Theory 39: 325–336.
[6] Huber, J. (1992). Trelliscodierung. Heidelberg: Springer-Verlag.
[7] Schlegel, C. (1997). Trellis Coding. New York, NY: IEEE Press.
[8] Forney, G.D. Jr. and Ungerboeck, G. (1998). Modulation and coding for linear Gaussian channels.
IEEE Transactions on Information Theory 44: 2384–2415.
[9] IEEE Standard 802.3ab (2015). Physical layer parameters and specifications for 1000 Mb/s opera-
tion over 4-pair of category 5 balanced copper cabling, type 1000BASE-T. IEEE.
[10] Yueksel, H., Braendli, M., Burg, A. et al. (2018). Design techniques for high-speed multi-level
Viterbi detectors and trellis-coded-modulation decoders. IEEE Transactions on Circuits and Systems
I: Regular Papers 65: 3529–3542.
[11] Forney, G.D. Jr. (1970). Convolutional codes I: algebraic structure. IEEE Transactions on Informa-
tion Theory IT–16: 720–738.
[12] Wolf, J.K. (1978). Efficient maximum likelihood decoding of linear block codes using a trellis.
IEEE Transactions on Information Theory IT–24: 76–80.
[13] Wei, L.F. (1987). Trellis-coded modulation with multidimensional constellations. IEEE Transactions
on Information Theory 33: 483–501.
[14] Wei, L.F. (1989). Rotationally invariant trellis-coded modulations with multidimensional M-PSK.
IEEE Journal on Selected Areas in Communications 7: 1281–1295.
Chapter 13

Techniques to achieve capacity

In this chapter, we introduce practical solutions that provide bit rates close to capacity. First, we
consider the parallel channel, comprising a set of independent narrowband (NB) additive white
Gaussian noise (AWGN) subchannels: such a channel is found when considering multicarrier (MC)
single-input-single-output (SISO) transmissions over a dispersive AWGN channel, or when applying a
suitable precoder and combiner to a multiple-input-multiple-output (MIMO) NB AWGN channel. Then,
we provide solutions tailored to single-carrier SISO transmissions, combined with joint precoding and
coding techniques.

13.1 Capacity achieving solutions for multicarrier systems

In the following, we discuss techniques to achieve capacity, with reference to the orthogonal frequency
division multiplexing (OFDM) system modelled in (8.83), comprising  subchannels, with AWGN
with power 𝜎W 2
and gain C,i on subchannel i, i = 0, 1, … ,  − 1. The considered techniques can also
be applied in other scenarios, e.g. MIMO transmissions over NB AWGN channels, providing NS,max
equivalent subchannels with gains {𝜉n }, n = 1, 2, … , NS,max (see (9.141)).

13.1.1 Achievable bit rate of OFDM

For an OFDM system (see Section 8.7.1), the signal-to-noise-ratio (SNR), normalized to unit transmit
power, over a given subchannel i is (see (8.83)))
|C,i |2
Θ[i] = , i = 0, 1, … ,  − 1 (13.1)
𝜎W
2

while the SNR is (𝙼[i] = E[|ak [i]|2 ])

Γ[i] = 𝙼[i]Θ[i], i = 0, 1, … ,  − 1 (13.2)

For a quadrature amplitude modulation (QAM) transmission, from (6.90), the capacity of the SISO
NB AWGN subchannel i is (see also (6.103))
1
C[bit∕s] [i] = log (1 + 𝙼[i]Θ[i]) (13.3)
T′ 2
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
688 Chapter 13. Techniques to achieve capacity

where powers {𝙼[i]} are chosen in order to maximize the overall capacity
−1
∑
C[bit∕s] = C[bit∕s] [i] (13.4)
i=0

under the total power constraint

−1
∑
𝙼[i] ≤ 𝙼tot (13.5)
i=0

By applying the Lagrange multiplier method, the optimal power allocation is given by
( )+
1
𝙼[i] = L − (13.6)
Θ[i]
where L is chosen in order to satisfy (13.5) with equality.

13.1.2 Waterfilling solution

From (13.6), the optimum solution is to allocate a higher power to subchannels with a higher gain, where
we assume that subchannels have decreasing normalized SNRs, i.e.

Θ[i + 1] < Θ[i], i = 1, … ,  − 2 (13.7)

1 1
With reference to the illustrative example of Figure 13.1, we can see that Θ[−1] > L and Θ[−2] > L.
Hence, in this case 𝙼opt [i] = 0, for i =  − 1,  − 2, i.e. the subchannels  − 1 and  − 2 are not
active for transmission. Instead, for the other subchannels, the gap between the constant L and the factor
1
Θ[i]
is the power assigned for the subchannel. This illustrates the concept of water filling, where for a
lake with depth 1∕Θ[i] at location i, i = 0, … ,  − 1, the water (power) has the same level L in the
whole lake.

10

L 5 WF

0
0 1 2 M–3 M–2 M–1

1
Figure 13.1 Increasing order of factors Θ[i]
.
13.1. Capacity achieving solutions for multicarrier systems 689

Iterative solution
We now show an iterative solution for the computation of the water level L, and consequently the optimal
power allocation. First, we assume that (13.7) holds.
1
Let N denote the number of active subchannels, with Θ[i] below level L. In general, for a given N,
from
1
𝙼[i] = L − i = 0, … , N − 1 (13.8)
Θ[i]
and the power constraint
∑
N−1
∑
N−1
1
𝙼tot = 𝙼[i] = N L − (13.9)
i=0 i=0
Θ[i]
we obtain the following iterative algorithm.

0. Initialize N to the number of available subchannels .

1. From (13.9) calculate the level
[ ]
1 ∑ 1
N−1
L= 𝙼tot + (13.10)
N i=0
Θ[i]

2. From (13.8) calculate the transmit power on subchannel i

1
𝙼[i] = L − i = 0, … , N − 1 (13.11)
Θ[i]
4. If 𝙼[i] > 0, for i = 0, … , N − 1, (i.e. check if 𝙼[N − 1] > 0) we are done. Otherwise decrease N by
one and go to step 1.

13.1.3 Achievable rate under practical constraints

In practice, due to constraints imposed by both signal constellations and available codes, the achievable
bit rate on subchannel i is smaller than (13.3), and is determined starting from the desired bit error
probability. As capacity and achievable bit rate are proportional to 1∕T ′ , we will drop this factor in the
following, referring instead to the achievable rate, or throughput, in bit/MCsymbol, as given by
∑
N−1
𝚋= 𝚋[i] [bit∕MCsymbol] (13.12)
i=0

where N is the number of active subchannels, and 𝚋[i] is the achievable rate for subchannel i.
The following derivation takes into account the fact that practical codes do not achieve the capacity.
In particular, we refer to M-QAM for subchannel i, and to a bit error rate for which the symbol error
probability in the uncoded case is given by (6.82), neglecting the factor term (1 − 1∕M) for simplicity,
(√ ) √
3
Pe [i] = 4Q Γ[i] = 4Q( 3Γ[i]) (13.13)
M−1

where (see (6.131))

Γ[i]
Γ[i] = (13.14)
2𝚋[i] − 1
with 𝚋[i] ≤ log2 M (equality holds in the uncoded case). For a given Pe [i], we have a corresponding
value of
Γgap [i] = Γ[i] (13.15)
690 Chapter 13. Techniques to achieve capacity

which provides 𝚋[i] = 𝚋. Therefore, by inverting (13.14) we have

( ) ( )
Γ[i] Γgap,dB [i]
𝚋[i] = log2 1 + = log2 1 + Γ[i] ⋅ 10− 10 (13.16)
Γgap [i]

For example, for Pe [i] = 2 × 10−7 , in the uncoded case the SNR gap from capacity is Γgap,dB [i] = 9.8 dB.
Introducing a system margin Γmar,dB for unforeseen channel impairments (Γmar,d B = 6 dB for subscriber
loop applications) and a code with gain Gcode,d B [i], we define

Γgap,d B [i] = Γgap,d B [i] + Γmar,d B [i] − Gcode,d B [i] (13.17)

and 𝚋[i] is computed from (13.16) replacing Γgap,d B [i] with Γgap,d B [i]. When Γgap = 1 (i.e. Γgap,d B = 0),
(13.16) is equal to the Shannon capacity (13.3) (apart from the scaling factor 1∕T ′ ). Any reliable and
implementable system must transmit at a rate below capacity, and hence, Γgap is a measure of loss with
respect to theoretically optimum performance.

Effective SNR and system margin in MC systems

Analogously to (6.111) for a linear dispersive channel, we define an effective SNR for OFDM. Assuming
the same Γgap,d B across subchannels, since from (13.12) and (13.16), it is
( ) [N−1 ( )]
∑
N−1
Γ[i] ∏ Γ[i]
𝚋= log2 1 + = log2 1+ (13.18)
i=0
Γgap i=0
Γgap

we define the effective SNR as

√
√N−1 ( )
Γeff √ ∏
= √
N
1+
Γ[i]
−1 (13.19)
Γgap i=0
Γgap

Then, from (13.18) we have

( )
Γeff
𝚋 = Nlog2 1 + [bit∕MCsymbol] (13.20)
Γgap
If +1 and -1 are negligible in (13.19), we can approximate Γeff as the geometric mean of the subchannel
SNRs, i.e. √
√N−1
√ ∏
Γ =√
N
eff Γ[i] (13.21)
i=0

Moreover, from (13.17) and (13.20), we have the relation as

Γeff
Γmar,d B = 10log10 𝚋
+ Gcode,d B − Γgap,d B (13.22)
2 −1
N

Uniform power allocation and minimum rate per subchannel

In order to maximize the achievable rate (13.18), we can apply the waterfilling algorithm using Θ[i]∕Γgap
as normalized SNR. Here we consider a simpler solution used in some applications, to simplify compu-
tation. We will enforce a uniform power allocation among active subchannels, and we will assume that
a subchannel must transmit at least 𝜌 bits of information when active: more on this assumption can be
found in Section 13.1.4 on transmission modes.
13.1. Capacity achieving solutions for multicarrier systems 691

First, once the N active subchannels have been selected, for a uniform power allocation strategy we
set {
𝙼tot
i = 0, … , N − 1
𝙼[i] = N (13.23)
0 i = N, … ,  − 1
It is seen that the optimum (waterfilling) and uniform power distributions yield a similar achievable rate.
In other words, it is more important to select the active subchannels over which transmission takes place
rather than allocate optimum powers.
A simple algorithm to determine the active subchannels is the following:
0. Set N to the number of subchannels .
1. Evaluate the power per subchannel
𝙼tot
𝙼[i] = , i = 0, … , N − 1 (13.24)
N
We recall that Γ[i] = 𝙼[i]Θ[i]
2. For i = 0, … , N − 1, evaluate ( )
Γ[i]
𝚋[i] = log2 1 + (13.25)
Γgap
If 𝚋[i] < 𝜌 drop subchannel i, as it cannot transmit the minimum number of information bits 𝜌.
3. Evaluate the number of active subchannels N ′ .
4. If N = N ′ , we are done and N is the number of active subchannels. Otherwise we reduce the number
of active subchannels, i.e. set N = N ′ , and repeat from step 1.

13.1.4 The bit and power loading problem revisited

Unfortunately, the above derivations yield 𝚋[i], the number of information bits per subchannel, as a real
number, while in practice 𝚋[i] may assume only a finite number of values depending upon the used code
and the constellation. Two problems can be approached:
● Maximize the achievable rate with a given total transmit power.
● Minimize the total transmit power for a given achievable rate.

Transmission modes
A transmission mode consists of a given QAM constellation and a coding scheme, and in general, only
a finite set of K transmission modes is available for subchannel i, and we write 𝚖[i] ∈ {1, … , K}. To
our concern, mode k is characterized by the couple
(𝚋k , Γk ), k = 1, … , K (13.26)
where 𝚋k is number of information bits/QAM-symbol carried by mode k and Γk is the minimum SNR Γ
required to transmit with mode k while ensuring a given Pbit .
Without loss of generality, we assume
𝚋k < 𝚋k+1 (number of information bits/QAM symbol ordered in increasing order)
𝚋0 = 0 (subchannel can be turned off)
A transmission mode 𝚖[i] and a transmit power
Γ𝚖[i]
𝙼[i] = (13.27)
Θ[i]
have been assigned to each subchannel, i = 0, … ,  − 1.
692 Chapter 13. Techniques to achieve capacity

A possible allocation of modes m = [𝚖[0], … , 𝚖[ − 1]], with corresponding bits b = [𝚋𝚖[0] , … ,
𝚋𝚖[−1] ] and powers M = [𝙼[0], … , 𝙼[ − 1]] given by (13.27) is called bit and power loading.

Example 13.1.1
Table 13.1 reports an example of transmission modes. SNRs Γk , k = 1, … , 7, correspond to a bit error
probability Pbit = 10−3 and have been obtained through system simulation (soft decoding has been used).

Table 13.1: Example of transmission modes.

Code bits per QAM symbol Code rate Info. bits per QAM symbol Γk (dB)
Mode 0 0 0 0 —
Mode 1 1 (BPSK) 1/2 0.5 −0.39
Mode 2 1 (BPSK) 3/4 0.75 2.55
Mode 3 2 (QPSK) 1/2 1 2.68
Mode 4 2 (QPSK) 3/4 1.5 5.50
Mode 5 4 (16-QAM) 9/16 2.25 8.87
Mode 6 4 (16-QAM) 3/4 3 11.65
Mode 7 6 (64-QAM) 3/4 4.5 17

Problem formulation
● Throughput maximization problem (TMP)
In this problem, we aim at distributing the total available power among the  subchannels, such
that the achievable rate is maximized. The TMP is stated as follows.
−1
∑
arg max 𝚋= 𝚋𝚖[i] (13.28)
𝚖[0],𝚖[1],…,𝚖[−1]
i=0
subject to (13.5)
−1
∑
𝙼[i] ≤ 𝙼tot (13.29)
i=0
Note that we consider again all  subchannels here, as we aim at determining which will be active
(whose total number will be N).
● Margin maximization problem (MMP)
The goal is to determine the bit allocation that requires the least amount of power. The MMP is
stated as follows.
−1
∑
arg min 𝙼tot = 𝙼[i] (13.30)
𝚖[0],𝚖[1],…,𝚖[−1]
i=0
subject to
−1
∑
𝚋𝚖[i] = 𝚋tot (13.31)
i=0
where 𝚋tot is the target achievable rate.

Some simplifying assumptions

Most algorithms developed to solve above problems work under one or more of the following hypothe-
ses: Convex hypothesis
𝚋k+1 − 𝚋k 𝚋 − 𝚋k−1
< k (13.32)
Γk+1 − Γk Γk − Γk−1
i.e. going from 𝚋k to 𝚋k+1 more power is required than going from 𝚋k−1 to 𝚋k .
13.1. Capacity achieving solutions for multicarrier systems 693

Constant granularity hypothesis

𝚋k = k𝜌 (13.33)
where 𝜌 is a constant called granularity. Usually 𝜌 = 1; however, 𝜌 may not be an integer, as some coding
schemes can yield a fractional number of information bits/symbol.

The constant gap approximation

The constant gap approximation assumes that Γgap [i] of (13.15) is only function of Pe , i.e. it is a
constant with respect to 𝚋[i]. Indeed many coding schemes verify the gap approximation quite well, but
only for very small values of Pe . Thus, under the constant gap approximation, all transmission modes
have the same gap Γgap .

On loading algorithms
∑−1
We can represent all possible loadings on the (𝙼, 𝚋) plane where 𝙼 = 𝙼tot = i=0 𝙼[i] and 𝚋 = 𝚋tot =
∑−1
i=0 𝚋[i].
From Figure 13.2, we can see that loadings are arranged in rows. For a given row, the same achievable
rate can be obtained with many values of the total power. The left-most loadings of each rows are called
efficient loadings, because they demand the minimum power. In the convex hypothesis, efficient loadings
draw a convex curve
The optimum loading is obviously an efficient loading. An efficient loading has the following
properties:
1. No other loading exists with the same achievable rate and a lower total power (or a higher achievable
rate with the same power).
2. Under convexity and constant granularity hypotheses, if we take 𝜌 bits away from any subchan-
nel and add them to any other subchannel, the new loading (with the same achievable rate) has
increased total power.

Optimal solution

Efficient solutions

M Mtot

Figure 13.2 Representation of all possible loadings.

694 Chapter 13. Techniques to achieve capacity

Some bit and power loading algorithms are greedy, i.e. they allocate one bit at a time to the subchannel
that will do the most good for the current partial allocation. In other words, the algorithm maximizes
the achievable rate increase at each step, regardless to the global effects of this choice.

The Hughes-Hartogs algorithm

For MMP, the Hughes-Hartogs (HH) [1] algorithm successively assigns the bits to the subchannels
until the target achievable rate 𝚋tot is reached. Thereby, at each steep, the selected subchannel is such
that the transmission of an additional bit can be done with the smallest additional transmit power. The
HH algorithm is optimal when the hypotheses of convexity and constant granularity hold, and it has a
complexity O(𝚋tot ∕𝜌). As shown in Figure 13.3, it visits all efficient loadings in increasing order until
the optimum one is found.

M Mtot

Figure 13.3 The Hughes-Hartogs algorithm.

The Krongold–Ramchandran–Jones algorithm

The Krongold–Ramchandran–Jones (KRJ) algorithm [2] makes use of the convexity hypothesis and
exploits a function which, given a parameter 𝜆 > 0, returns an efficient loading L𝜆 = (𝙼𝜆 , b𝜆 ). It has the
following property: the total power 𝙼𝜆 and achievable rate 𝚋𝜆 of loading L𝜆 are non-increasing function
of 𝜆. Let a plane wave of slope 𝜆 propagate down-rightwards on the (𝙼, 𝚋) plane. The first loading point
L𝜆 hit by the plane wave is efficient (see Figure 13.4). Moreover, hit points have equal or smaller values
of 𝙼 and 𝚋 for increasing values of 𝜆.
L𝜆 is built by applying the principle of the plane wave to each subchannel. For each subchannel i, let us
represent the set of modes on the (𝙼[i], 𝚋[i]) plane; each mode corresponds to the point (𝙼[i], 𝚋𝚖[i] ), with
Γ𝚖[i]
𝙼[i] = (13.34)
Θ[i]
13.1. Capacity achieving solutions for multicarrier systems 695

M Mtot

Figure 13.4 Description of the plane wave used by the KRJ algorithm.

Each representation includes K points. Indeed from (13.34) the set of points for subchannel i is obtained
by the horizontal scaling of the points (𝚋k , Γk ), k = 1, … , K, by a factor Θ[i].
From Figure 13.4, if we connect all adjacent efficient modes with segments, we obtain a collection of
slopes, where slope 𝜆 is associated to the segment edge point with lower power 𝚖𝜆 [i] = (𝙼𝜆 [i], 𝚋𝜆 [i]), for
an overall loading L𝜆 . Since 𝚋𝜆 and 𝙼𝜆 are non-increasing functions of 𝜆, the algorithm can be summa-
rized as follows.
The TMP and MMP algorithms:
We will give the algorithm for TMP, as shown in Figure 13.5, and, when different, in square brackets
the MMP.
1. Start with two slopes 𝜆lo = +∞ and 𝜆hi = 0; build the plane wave solutions Llo = L𝜆lo and
Lhi = L𝜆hi and compute their corresponding total power and achievable rate 𝙼lo = 𝙼(Llo ),
𝙼hi = 𝙼(Lhi ), 𝚋lo = 𝚋(Llo ), and 𝚋hi = 𝚋(Lhi ).
2. Compute the new slope
𝚋 − 𝚋lo
𝜆ne𝑤 = hi (13.35)
𝙼hi − 𝙼lo
3. Build the plane wave solution Lne𝑤 = L𝜆ne𝑤 and compute the achievable rate 𝚋ne𝑤 and power 𝙼ne𝑤 .
4. If 𝙼ne𝑤 = 𝙼lo , or 𝙼ne𝑤 = 𝙼hi , or 𝙼ne𝑤 = 𝙼tot , [If 𝚋ne𝑤 = 𝚋tot ]
then go to step 5;
else if 𝙼ne𝑤 < 𝙼tot , [else if 𝚋ne𝑤 < 𝚋tot ]
then 𝜆lo𝑤 ← 𝜆ne𝑤 , 𝙼lo𝑤 ← 𝙼ne𝑤 , 𝚋lo𝑤 ← 𝚋ne𝑤 ; go to step 2.
else if 𝙼ne𝑤 > 𝙼tot , [else if 𝚋ne𝑤 > 𝚋tot ]
then 𝜆hi ← 𝜆ne𝑤 , 𝙼hi ← 𝙼ne𝑤 , 𝚋hi ← 𝚋ne𝑤 ; go to step 2.
5. The optimal solution has been found. If 𝙼ne𝑤 = 𝙼tot (which is really very unlikely), then the optimal
solution is Lne𝑤 , otherwise the optimal solution is Llo . (The optimal solution is simply Lne𝑤 .)
696 Chapter 13. Techniques to achieve capacity

b slope = λn

slope = λh
(Mn, bn)

slope = λℓ (Mh, bh)

(Mℓ, bℓ)

Mtot
M

Figure 13.5 Bisection search algorithm.

This algorithm is in practice a bisection search among efficient solutions, which are known to yield
a power and number of bits/symbol that are a decreasing function of 𝜆. We recall that, when building a
plane wave solution L𝜆 , 𝜆 can be thought of as the slope of the curve that connects the efficient solutions,
at the point corresponding to L𝜆 . This is why step 2 yields a solution that is midway between Llo𝑤 and Lhi .
Because the total number of efficient solutions is K, the bisection search finds the optimal solution
in approximately O(log2 (K)) iterations. For each subchannel, i = 0, … ,  − 1, finding 𝚖𝜆 [i] would
require O(K) comparisons with a pre-computed table which stores the map between slopes and efficient
modes. The overall complexity is O(Klog2 (K)).
The KRJ algorithm is optimal.

The Chow–Cioffi–Bingham algorithm

The Chow–Cioffi–Bingham (CCB) algorithm [3] uses the hypotheses of convexity, constant granularity,
and constant gap approximation. It would be suboptimal even if the constant gap hypothesis were satis-
fied exactly. It estimates the subchannels to be turned off, and distributes power almost equally among
remaining subchannels. The key assumption is that performance is mostly determined by the choice of
subchannels to be turned off. We provide here its description for the MMP.

1. We start by assigning the same power to all subchannels:

1
𝙼[i] = 𝙼 , i = 0, … ,  − 1 (13.36)
 tot
We also calculate Γ[i] = 𝙼[i]Θ[i], i = 0, … ,  − 1. Let N be the number of active subchannels,
initialized to N = . Then we assume a starting margin 𝛾 = 1.
2. For each subchannel we compute:
( )
Γ[i]
𝚋 [i] = log2 1 +
′
(13.37)
Γgap 𝛾
13.1. Capacity achieving solutions for multicarrier systems 697

If we assign 𝚋′ [i] bits/symbol to subchannel i, we would actually obtain the same margin on all
subchannels, with the total power equal to 𝙼tot . However, 𝚋′ [i]∕𝜌 in general is not an integer, so we
have to choose the nearest available mode. Hence we assign1 :
[ ( ′ ) ]
𝚋 [i]
𝚖[i] = max round ,0 (13.38)
𝜌
For future use, we also define
d[i] = 𝚋′ [i] − 𝚋𝚖[i] (13.39)

We also perform the following operation for i = 0, … ,  − 1

if 𝚖[i] = 0, then N ← N − 1 (13.40)

∑N
3. We compute the number of bits/symbol obtained with the current loading: 𝚋 = i=1 𝚋𝚖[i] . The
obtained solution yields approximately the same margin 𝛾 for all subchannels, although 𝛾 may not
be optimal, and 𝚋 may not be the maximum achievable rate, since we have arbitrarily assigned
uniformly distributed power at the beginning.
4. The new margin 𝛾 obtained when lowering the achievable rate by 𝚋 − 𝚋tot is estimated. To this end,
we consider an equivalent average subchannel with achievable rate 𝚋∕N, margin 𝛾, and unknown
SNR Γ. For such an equivalent subchannel, it would be

ΓΓgap
𝛾= (13.41)
2𝚋∕N
Setting the achievable rate of the equivalent subchannel to 𝚋tot ∕N yields the new margin
𝛾ne𝑤 2𝚋∕N − 1 𝚋−𝚋tot
= 𝚋 ∕N ≃2 N (13.42)
𝛾 2 tot −1
Then, we update 𝛾 as
𝚋−𝚋tot
𝛾 ←𝛾 ⋅2 N (13.43)

5. If 𝚋 = 𝚋tot , we stop iterating and we go to the last step.

6. If this is the first iteration, then we go back to step 2, because we have to recompute the combination
of modes according to the new margin. Otherwise, if it is not the first iteration, we check if N has
changed since the last iteration. If it has not, it is not useful to go on iterating, because the difference
between 𝚋 and 𝚋tot is only due to rounding, and not to subchannel exclusions. Thus, if N has changed
we go back to step 2, otherwise we go on to the next step.
7. Now we fine-tune the bit distribution to get exactly 𝚋 = 𝚋tot . We have to add (or subtract) 𝚋 − 𝚋tot
bits. We add (subtract) 𝜌 bits at a time to the subchannels that have received less (more) bits than
they should, due to rounding. Thus we do
● While 𝚋 < 𝚋
tot
i∗ = argmaxi d[i]
𝚖[i∗ ] ← 𝚖[i∗ ] + 1, d[i∗ ] ← d[i∗ ] − 𝜌, 𝚋 ← 𝚋 + 1
● While 𝚋 > 𝚋
tot
i∗ = argmaxi∶𝚖[i]>0 d[i]
𝚖[i∗ ] ← 𝚖[i∗ ] − 1, d[i∗ ] ← d[i∗ ] + 𝜌, 𝚋 ← 𝚋 − 1
Actually, we can do this more efficiently with a complexity proportional to the number of active
channels [3].

1
round(x) denotes the nearest integer to x.
698 Chapter 13. Techniques to achieve capacity

8. Lastly, we compute the correct powers according to the combination of selected modes:
Γgap
𝙼[i] = (2𝚋𝚖[i] − 1) , i = 0, … , N − 1 (13.44)
Θ[i]
and we scale by
∑
N−1
𝙼′tot = 𝙼[i] (13.45)
i=0

so that the total power is 𝙼tot as:

𝙼tot
𝙼[i] ← 𝙼[i] , i = 0, … , N − 1 (13.46)
𝙼′tot

The solution we have obtained ensures that all subchannels have exactly the same margin, and hence
if 𝛾 > 1 it guarantees that the error probability constraint is satisfied; it also uses a total power 𝙼tot and
yields an achievable rate 𝚋tot . However, it is not guaranteed to be the optimal solution, that is the margin
may be slightly lower than it could be.
In general, the CCB algorithm may be slightly suboptimal with respect to the HH algorithm, while
having lower complexity O().

Comparison
A comparison between the various algorithms examined is reported in Table 13.2. For the CCB algo-
rithm, Niter is the number of iterations. Note that the HH algorithm is very complex in some applications
like asymmetric digital subscribers line (ADSL), where a large number of modes are available and 
is large.
Table 13.2: Comparison between four loading algorithms.

Algorithm optimal Hypothesis Complexity

𝜌 convex Γgap Other
HH Yes x x O(𝚋tot ∕𝜌)
KRJ Yes x O(Klog2 (K))
CCB No x x x O(Niter )

13.2 Capacity achieving solutions for single carrier systems

If the passband  consists of only one frequency interval, as an alternative to OFDM transmission the
capacity of a linear dispersive channel can be achieved by single carrier transmission. For a comparison
of the characteristics of the two systems, we refer to page 444.
First we examine the equivalence (in terms of capacity) between a continuous-time channel and a
discrete-time channel with inter-symbol interference (ISI); this equivalence is obtained by referring to
a transmit filter that shapes the spectrum of the transmit signal as indicated by water pouring, and a
receiver that implements a matched filter (MF) or a whitened matched filter (WMF), as illustrated in
Section 7.4.6 (Figure 7.35). By the WMF, we obtain a canonical form of the discrete-time channel
with trailing ISI, that is ISI due only to postcursors, and AWGN (see Figure 13.6). As will be shown
in this section, the contribution of the ISI of the resulting discrete-time channel to the capacity of the
 (a) whitening
e j2πf0t e j2π f0t whitening matched filter
filter filter
ak s(t) sCh(t) r(t) xk 1 zk
hTx(t) Re[.] gCh(t) gw(t) gM(t) ~
f 20 f * 1*
T kT D

w(t) [w( f )]

(b)
Ch( f + f 0)1( f + f 0)
C ( f ) = whitening
2 whitening matched filter
filter filter
sC(t) rC(t) 1
ak s(bb)(t) xk zk
hTx(t) gC(t) gw(t) gM(t) ~
f 20 f * 1*
T kT D
wC(t)

(c) [wC ( f ) = 2 w( f + f 0)1( f + f 0)]

ak ~ uk zk
f
T

wk

1
Figure 13.6 Equivalence between a continuous time system, (a) passband model, (b) baseband equivalent model with g𝚠 (t) = √ g (t), and (c) a
2 𝚠
discrete-time system, for transmission over a linear dispersive channel.
700 Chapter 13. Techniques to achieve capacity

channel becomes negligible for high SNRs. This suggests that capacity can be achieved by combining
ISI cancellation techniques in the discrete-time domain with channel coding and shaping.
Assume that the passband  that allows achieving capacity consists of only one frequency interval
[f1 , f2 ], with 0 < f1 < f2 and bandwidth B = f2 − f1 . If  consists of several intervals, then the same
procedure can be separately applied to each interval, although in this case MC transmission is preferable.
Consider passband transmission with modulation interval T and with minimum bandwidth of a
complex-valued symbol sequence, that is we choose T = 1∕B.
We recall the signal analysis that led to the scheme of Figure 7.35, extending it to the case of noise
𝑤(t) with power spectral density (PSD) 𝑤 (f ) not necessarily constant. With reference to Figure 13.6,
from (7.26) the transmitted signal is given by
[ ]
∑
s(t) = Re ak hTx (t − kT) ej2𝜋f0 t (13.47)
k

where {ak } is modelled as a sequence of complex-valued i.i.d. symbols with Gaussian distribution and
variance 𝜎a2 , and hTx (t) denotes the transmit filter impulse response, with Fourier transform Tx (f ).
The transmit filter is chosen such that the PSD of s(t), given by (7.24) and (7.25), is equal to s,opt (f ),
that is
𝜎a2
|Tx (f )|2 = s,opt (f + f0 ) 1(f + f0 ) (13.48)
4T
Note that this requires the adaptation of the transmit filter to the channel impulse response, a solution
rarely adopted in practice, where the transmit filter is typically fixed, and implemented in part by analog
circuitry.
At the receiver, the signal r(t) is first demodulated and filtered by a filter with impulse response g𝚠 that
suppresses the signal components around −2f0 and whitens the noise. As an alternative we could use a
passband whitening filter phase splitter that suppresses the signal components with negative frequency
and whitens the noise in the passband, see Section 7.4.9.
Consider the baseband equivalent model of Figure 13.6b, where C (f ) = √1 Ch (f + f0 ) 1(f + f0 ) and
2
𝑤C (f ) = 2𝑤 (f + f0 ) 1(f + f0 ); we define
0 = [f1 − f0 , f2 − f0 ] (13.49)
1
as the new passband of the desired signal at the receiver. The whitening filter g𝚠 (t) = √ g𝚠 (t) has fre-
2
quency response given by
⎧ 1
⎪ f ∈ 0
|𝚠 (f )| = ⎨ 𝑤C (f )
2
(13.50)
⎪0 elsewhere
⎩
From the scheme of Figure 7.35, the whitening filter is then followed by a MF gM with frequency
response
M (f ) = [Tx (f ) C (f ) 𝚠 (f )]∗ (13.51)
The cascade of whitening filter g𝚠 and MF gM yields the composite filter gMF
Rc
(f ) with frequency response
MF
Rc
(f ) = Tx
∗
(f ) ∗C (f ) |𝚠 (f )|2
Tx
∗
(f ) ∗C (f ) (13.52)
=
𝑤C (f )
The overall QAM pulse q at the output of the MF has frequency response given by
|Tx (f ) C (f )|2
(f ) = Tx (f ) C (f ) MF
Rc (f ) = (13.53)
𝑤C (f )
13.2. Capacity achieving solutions for single carrier systems 701

Note that (f ) has the properties of a PSD with passband 0 ; in particular, Q(f ) is equal to the noise
PSD 𝑤R (f ) at the MF output, as

|Tx (f ) C (f )|2
𝑤R (f ) = 𝑤C (f ) |MF 2
Rc (f )| = = (f ) (13.54)
𝑤C (f )
Therefore, the sequence of samples at the MF output can be expressed as
∑
+∞
xk = ai hk−i + 𝑤̃ k (13.55)
i=−∞

where the coefficients hi = q(iT) are given by the samples of the overall impulse response q(t), and

𝑤̃ k = 𝑤R (kT) = [𝑤C (t′ ) ∗ gMF ′

Rc (t )](t)|t=kT (13.56)

In general, the Fourier transform of the discrete-time response {hi } is given by

( )
1 ∑
+∞
𝓁
 (dis) (f ) =  f− (13.57)
T 𝓁=−∞ T
In this case, because (f ) is limited to the passband 0 with bandwidth B = 1∕T, there is no aliasing; the
function  (dis) (f ), periodic of period 1∕T, is therefore equal to (1∕T)(f ) in the band 0 . As 𝑤R (f ) =
(f ), {𝑤̃ k } is a sequence of Gaussian noise samples with autocorrelation sequence {𝚛𝑤̃ k (n) = hn }. Note
moreover that {hi } satisfies the Hermitian property, as  (dis) is real valued.
We have thus obtained a discrete-time equivalent channel that can be described using the D trans-
form as
̃
x(D) = a(D)h(D) + 𝑤(D) (13.58)

where h(D) has Hermitian symmetry.

We now proceed to develop an alternative model of the discrete-time equivalent channel with causal,
monic, and minimum-phase response f̃ (D), and AWGN 𝑤(D). With this regard, we recall the theorem
of spectral factorization for discrete time systems (see page 36). If  (dis) (f ) satisfies the Paley–Wiener
condition, then the function h(D) can be factorized as follows:
( )
1
h(D) = f̃ ∗ f 2 f̃ (D) (13.59)
D∗ 0
 (dis) (f ) = ̃ ∗ (f ) f02 ̃ (f ) (13.60)

where the function f̃ (D) = 1 + f̃1 D + · · · is associated with a causal (f̃i = 0 for i < 0), monic and
minimum-phase sequence f̃i , and ̃ (f ) = f̃ (e−j2𝜋fT ) is the Fourier transform of the sequence {f̃i }. The
factor f02 is the geometric mean of  (dis) (f ) over an interval of measure 1∕T, that is

log f02 = T log  (dis) (f ) d f (13.61)

∫1∕T
where logarithms may have any common base.
Then, (13.58) can be written as
( ) ( )
1 1
x(D) = a(D) f02 f̃ (D) f̃ ∗ + 𝑤′ (D) f0 f̃ ∗ (13.62)
D∗ D∗
where 𝑤′ (D) is a sequence of i.i.d. Gaussian noise samples with unit variance. Filtering x(D) by a filter
having transfer function 1∕[f02 f̃ ∗ (1∕D∗ )], we obtain the discrete-time equivalent canonical model of the
dispersive channel
z(D) = a(D) f̃ (D) + 𝑤(D) (13.63)
702 Chapter 13. Techniques to achieve capacity

where 𝑤(D) is a sequence of i.i.d. Gaussian noise samples with variance 1∕f02 . Equation (13.63) is
obtained under the assumption that f̃ (D) has a stable reciprocal function, and hence, f̃ ∗ (1∕D∗ ) has an
anti-causal stable reciprocal function; this condition is verified if h(D) has no spectral zeros.
However, to obtain the reciprocal of f̃ (D) does not represent a problem, as z(D) can be indirectly
obtained from the sequence of samples at the output of the WMF. The transfer function of the composite
filter that consists of the whitening filter g𝚠 and the WMF has a transfer function given by
MF
Rc
(f ) Tx
∗
(f ) ∗C (f ) ̃ (f )
WMF
Rc (f ) = = (13.64)
f02 ̃ ∗ (f ) 𝑤C (f )  (dis) (f )
The only condition for the stability of the filter (13.64) is given by the Paley–Wiener criterion.

Achieving capacity
Note that the model (13.63) expresses the output sequence as the sum of the noiseless sequence
a(D) f̃ (D) and AWGN 𝑤(D).
From (13.63), if a(D) is an uncoded sequence with symbols taken from a finite constellation, and f̃ (D)
has finite length, then the received sequence in the absence of noise a(D) f̃ (D) can be viewed as the output
of a finite state machine, and the sequence a(D) can be optimally detected by the Viterbi algorithm, as
discussed in Chapter 7. As an alternative, maximum-likelihood sequence detection (MLSD) can be
directly performed by considering the MF output sequence x(D), using a trellis of the same complexity
but with a different metric (see Section 7.5.3).
In fact the MF output sequence x(D) can be obtained from the WMF output sequence z(D) by filtering
z(D) with a stable filter having transfer function f02 f̃ ∗ (1∕D∗ ). As x(D) is a sufficient statistic2 (see Defi-
nition 7.2 on page 354) for the detection of a(D), also z(D) is a sufficient statistic; therefore, the capacity
of the overall channel including the MF or the WMF is equal to the capacity C[bit∕s] given by (6.105).
Therefore, capacity can be achieved by coding in combination with the cancellation of ISI.
We now evaluate the capacity. Using (13.53), (13.48), and (6.102) with the definitions of C and
𝑤C , yields
|Tx (f ) C (f )|2 4T 1
(f ) = = 2 s,opt (f + f0 ) Θ(f + f0 ) 1(f + f0 ) (13.65)
𝑤C (f ) 𝜎a 4
and capacity can be expressed as
C[bit∕s] = ∫ log2 [1 + s,opt (f ) Θ(f )] df
= ∫ log2 [1 + s,opt (f + f0 ) Θ(f + f0 )] df (13.66)
[ ]
0

𝜎2
= ∫ log2 1 + Ta (f ) df
0

Recall from (13.57) that  (f ), periodic of period 1∕T, is equal to (1∕T)(f ) in the band 0 ;
(dis)

therefore, using (13.61) for B = 1∕T, the capacity C[bit∕s] and its approximation for large values of the
SNR Γ can be expressed as
( ) ( )
C[bit∕s] = log2 1 + 𝜎a2  (dis) (f ) df ≃ log2 𝜎a2  (dis) (f ) df = Blog2 (𝜎a2 f02 ) (13.67)
∫1∕T ∫1∕T

2
In particular, we have a sufficient statistic when projecting the received signal on a basis of the desired (useful) signals.
For example, considering the basis {e±j2𝜋ft , t ∈ ℝ, f ∈ } to represent real-valued signals with passband  in the
presence of additive noise, that is the Fourier transform of the noisy signal filtered by an ideal filter with passband ,
we are able to reconstruct the noisy signals within the passband of the desired signals; therefore, the noisy signal filtered
by a filter with passband  is a sufficient statistic.
Bibliography 703

Assume that the tail of the impulse response that causes ISI can be in some way eliminated, so that
at the receiver we observe the sequence a(D) + 𝑤(D) rather than the sequence (13.63). The SNR of the
resultant ideal AWGN channel becomes ΓISI-free = 𝜎a2 f02 ; thus, from (6.96) the capacity of the ISI-free
channel and its approximation for large values of Γ become
CISI-free [bit∕s] = B log2 (1 + 𝜎a2 f02 ) ≃ B log2 (𝜎a2 f02 ) (13.68)
Comparing (13.67) and (13.68), we finally obtain
C[bit∕s] ≃ CISI-free [bit∕s] (13.69)
Price was the first to observe that for large values of Γ we obtain (13.69), that is for high SNRs
the capacity C[bit∕s] of the linear dispersive channel is approximately equal to the capacity of the ideal
ISI-free channel obtained assuming that the residual ISI in the discrete-time channel can be in some
way eliminated from the sequence x(D); in other words, the residual ISI of the discrete channel does not
significantly contribute to the capacity [4]. Note however again that this results leverages the adaptation
of the transmit filter to channel conditions, in order to obtain (13.48), a mostly theoretical setting rarely
adopted in practice.
It is interesting to observe that in faster-than-Nyquist systems, where ISI is predominant, the capacity
is indeed higher than (13.69) (see references in Section 7.9).

Bibliography

[1] Hughes-Hartogs, D. (1987). Ensemble modem structure for imperfect transmission media. US Patent
No. 4, 679, 227, July 1987; 4, 731, 816, March 1988, and 4, 833, 796, May 1989.
[2] Krongold, B.S., Ramchandran, K., and Jones, D.L. (2000). Computationally efficient optimal power
allocation algorithms for multicarrier communication systems. IEEE Transactions on Communica-
tions 48: 23–27.
[3] Chow, J.S., Tu, J.C., and Cioffi, J.M. (1991). A discrete multitone transceiver system for HDSL
applications. IEEE Journal on Selected Areas in Communications 9: 895–908.
[4] Price, R. (1972). Nonlinearly feedback-equalized PAM versus capacity for noisy filter channels. In:
Proceedings of 1972 International Conference on Communications, pp. 22.12–22.17.
Chapter 14

Synchronization

In quadrature amplitude modulation (QAM) systems, coherent detection requires the generation of a
carrier signal at the frequency of the transmitted modulated signal. Although the nominal frequency of
the transmitter and the receiver are the same, imperfections of the components or Doppler shift effects
determine an unknown (significant) difference between the receiver oscillator frequency and the carrier
frequency of the received signal.
In this chapter, we will discuss methods for carrier phase and frequency recovery, as well as algo-
rithms to estimate the timing phase. To avoid ambiguity, we refer to the latter as timing recovery algo-
rithms, dropping the term phase. These algorithms are developed for application in the pulse amplitude
modulation (PAM) and QAM transmission systems of Chapter 7, and the spread spectrum systems of
Chapter 10. The problem of carrier frequency synchronization for orthogonal frequency division multi-
plexing (OFDM) systems will be addressed in Section 14.9.
Carrier recovery must be performed using one of the following two strategies.
1. The first consists in multiplexing, usually in the frequency domain, a special signal, called pilot sig-
nal, at the transmitter. This allows extracting the carrier at the receiver and therefore synchronizing
the receive oscillator in phase and frequency with the transmit oscillator. If the pilot signal consists
of a non-modulated carrier, carrier recovery is obtained by the phase-locked loop (PLL) described
in Section 14.2.
2. The second consists in getting the carrier directly from the modulated signal; this approach presents
the advantage that all the transmitted power is allocated for the transmission of the signal carrying
the desired information. Some structures that implement this strategy are reported in Section 14.3.

14.1 The problem of synchronization for QAM systems

As a generalization of the receiver block diagram shown in Figure 7.5, the representation of the analog
front end for a passband QAM system is illustrated in Figure 14.1. The received signal r is multiplied
by a complex-valued carrier generated by a local oscillator, then filtered by an anti-imaging filter, gAI ,
that extracts the complex representation of r, rC (see Section 1.4). Often the function of the anti-imaging
filter is performed by other filters, however, here gAI is considered only as a model for the analysis and
is assumed to be non-distorting.
In Figure 14.1, the reconstructed carrier, in complex form, has the expression
𝑣̂ C (t) = exp{−j(2𝜋f1 t + 𝜑1 )} (14.1)
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
706 Chapter 14. Synchronization

s(t) sCh(t) r (t) rC (t)

gCh(t) gAI (t)
exp{–j(2πf 1t + φ1)}
w(t)
carrier
recovery

Figure 14.1 Analog front end for passband QAM systems.

Therefore, if the carrier generated at the transmitter, as observed at the receiver (received carrier) is given
by exp{j(2𝜋 f0 t + 𝜑0 )},1 in general we have a reconstruction error of the carrier phase 𝜑PA given by
𝜑PA (t) = (2𝜋f1 t + 𝜑1 ) − (2𝜋f0 t + 𝜑0 ) (14.2)
By defining
Ω = 2𝜋(f0 − f1 )
(14.3)
𝜃 = (𝜑0 − 𝜑1 )
(14.2) can be rewritten as
𝜑PA (t) = −Ωt − 𝜃 (14.4)

s(bb)(t) sC (t) rC (t)

gC (t)

wC (t) e–j φPA(t)

Figure 14.2 Baseband equivalent model of the channel and analog front end for a QAM system.

With reference to the notation of Figure 7.11, observing that now the phase offset is not included in
the baseband equivalent channel impulse response (CIR), we have
1
gC (t) = √ g(bb)
Ch
(t) e−j arg Ch (f0 ) (14.5)
2 2
The resulting baseband equivalent scheme of a QAM system is given in Figure 14.2. We assume that
the anti-imaging filter frequency response is flat within the frequency interval
Ωmax
|f | ≤ B + (14.6)
2𝜋
where B is the bandwidth of the signal sC and Ωmax is the maximum value of |Ω|.
The received signal rC is affected by a frequency offset Ω and a phase offset 𝜃; moreover, both the
transmit filter hTx and the channel filter gC introduce a transmission delay t0 . To simplify the analysis,
this delay is assumed known with an error in the range (−T∕2, T∕2). This coarse timing estimate can
be obtained, for example, by a correlation method with known input (see (7.603)). This corresponds to
assuming the overall pulse qC non-causal, with peak at the origin.
Once set t0 = 𝜀T, with |𝜀| ≤ 1∕2, the signal rC can be written as
∑
+∞
rC (t) = ej(Ωt+𝜃) ak qC (t − kT − 𝜀T) + 𝑤C𝜑 (t) (14.7)
k=−∞

1
In this chapter, 𝜑0 is given by the sum of the phase of the transmitted carrier and the channel phase at f = f0 , equal
to arg Ch (f0 ).
14.2. The phase-locked loop 707

where
qC (t) = (hTx ∗ gC )(t + 𝜀T)
(14.8)
𝑤C𝜑 (t) = 𝑤C (t) e−j𝜑PA (t)
Furthermore, the receiver clock is independent of the transmitter clock, consequently the receiver clock
period, that we denote as Tc , is different from the symbol period T at the transmitter: we assume that the
ratio F0 = T∕Tc is not necessary a rational number.
The synchronization process consists in recovering the carrier, in the presence of phase offset 𝜃 and
frequency offset Ω, and the timing phase or time shift 𝜀T.

14.2 The phase-locked loop

We assume now that at the transmitter an unmodulated carrier is superimposed to the data signal, in
order to ease synchronization at the receiver.
We also assume that the transmitted sinusoidal component 𝑣 has been isolated from the signal at the
receiver by a suitable narrowband filter. Now the problem consists in generating by a local oscillator, for
example using a PLL, a signal 𝑣VCO with the same frequency and phase of 𝑣, apart from a known offset.

loop filter
v(t) e(t) u(t)
PD F(s)

vVCO(t)

π
— VCO
carrier to 2
coherent
demodulator

Figure 14.3 Block diagram of a PLL.

A PLL, whose block diagram is shown in Figure 14.3, is a control system used to automatically
regulate the phase of a locally generated signal 𝑣VCO so that it coincides with that of an input signal
𝑣. We assume that the two signals are given by
𝑣(t) = A1 sin[𝜔0 t + 𝜑0 (t)], 𝑣VCO (t) = A2 cos[𝜔1 t + 𝜑1 (t)] (14.9)
where the phase 𝜑0 is a slowly time varying function with respect to 𝜔0 , or
| d𝜑0 (t) |
| |
| dt | ≪ 𝜔0 (14.10)
| |
We write the instantaneous phase of 𝑣VCO as follows:
𝜔1 t + 𝜑1 (t) = 𝜔0 t + (𝜔1 − 𝜔0 ) t + 𝜑1 (t) (14.11)
and we let
𝜑̂ 0 (t) = (𝜔1 − 𝜔0 ) t + 𝜑1 (t) (14.12)
thus
𝑣VCO (t) = A2 cos[𝜔0 t + 𝜑̂ 0 (t)] (14.13)
and 𝜑̂ 0 then represents the estimate of the phase 𝜑0 obtained by the PLL.
708 Chapter 14. Synchronization

We define the phase error as the difference between the instantaneous phases of the signals 𝑣 and
𝑣VCO , that is
𝜙(t) = (𝜔0 t + 𝜑0 (t)) − (𝜔1 t + 𝜑1 (t)) = 𝜑0 (t) − 𝜑̂ 0 (t) (14.14)
As illustrated in Figure 14.3, a PLL comprises:
● a phase detector (PD), that yields an output signal e given by the sine of the difference between the
instantaneous phases of the two input signals, that is
e(t) = KD sin[𝜙(t)] (14.15)
where KD denotes the phase detector gain; we observe that e is an odd function of 𝜙; therefore, the
PD produces a signal having the same sign as the phase error, at least for values of 𝜙 between −𝜋
and +𝜋;
● a lowpass filter F(s), called loop filter, whose output u is equal to
u(t) = f ∗ e(t) (14.16)
● a voltage controlled oscillator (VCO), which provides a periodic output signal 𝑣VCO whose phase
𝜑̂ 0 satisfies the relation
d𝜑̂ 0 (t)
= K0 u(t) (14.17)
dt
called VCO control law, where K0 denotes the VCO gain.
In practice, the PD is often implemented by a simple multiplier; then the signal e is proportional to
the product of 𝑣 and 𝑣VCO . If Km denotes the multiplier gain and we define
1
KD = A A K (14.18)
2 1 2 m
then we obtain
e(t) = Km 𝑣(t) 𝑣VCO (t)
= Km A1 sin[𝜔0 t + 𝜑0 (t)] A2 cos[𝜔1 t + 𝜑1 (t)] (14.19)
= KD sin[𝜙(t)] + KD sin[2𝜔0 t + 𝜑0 (t) + 𝜑̂ 0 (t)]
Note that, with respect to the signal e defined in (14.15), there is now an additional term with radian
frequency 2𝜔0 . However, as from (14.10), 𝜑0 is slowly varying in comparison with the term at frequency
2𝜔0 , the high frequency components are eliminated by the lowpass filter F(s) or, in case the lowpass filter
is not implemented, by the VCO that has a lowpass frequency response. Therefore, the two schemes,
with a PD or with a multiplier, may be viewed as equivalent; because of its simplicity, the latter will be
considered in the following analysis.

14.2.1 PLL baseband model

We now derive a baseband equivalent model of the PLL. From (14.16), we have
t
u(t) = f (t − 𝜉) e(𝜉) d𝜉 (14.20)
∫0
Substitution of (14.20) in (14.17) yields
d𝜑̂ 0 (t) t
= K0 f (t − 𝜉) e(𝜉) d𝜉 (14.21)
dt ∫0
By this relation, we derive the baseband scheme of Figure 14.4. Subtraction of the phase estimate 𝜑̂ 0
from the phase 𝜑0 yields the phase error 𝜙 that, transformed by the non-linear block KD sin(⋅), in turn
14.2. The phase-locked loop 709

φ0(t) + ϕ(t) e(t)

KD sin(.) F(s)
–
φ0(t)

K0 u(t)
s

Figure 14.4 Baseband model of a PLL.

gives the signal e. The signal e is input to the loop filter F(s), which outputs the control signal u. The
integration block, with gain K0 , integrates the signal u and yields the estimate 𝜑̂ 0 , thus closing the loop.
Replacing 𝜑̂ 0 by 𝜑0 (t) − 𝜙 in (14.21), and e by (14.15) we have
t
d𝜙(t) d𝜑0 (t)
= − KD K0 f (t − 𝜉) sin[𝜙(𝜉)] d𝜉 (14.22)
dt dt ∫0
This equation represents the integro-differential equation that governs the dynamics of the PLL. Later,
we will study this equation for particular expressions of the phase 𝜑0 , and only for the case 𝜙(t) ≃ 0, i.e.
assuming the PLL is in the steady state or in the so-called lock condition; the transient behaviour, that
is for the case 𝜙(t) ≠ 0, is difficult to analyse and we refer to [1] for further study.

Linear approximation
Assume that the phase error 𝜙 is small, or 𝜙(t) ≃ 0; then, the following approximation holds
sin[𝜙(t)] ≃ 𝜙(t) (14.23)
and (14.22) simplifies into
t
d𝜙(t) d𝜑0 (t)
= − KD K0 f (t − 𝜉) 𝜙(𝜉) d𝜉 (14.24)
dt dt ∫0
In this way, the non-linear block KD sin(⋅) of Figure 14.4 becomes a multiplier by the constant KD , and
the whole structure is linear, as illustrated in the simplified block diagram of Figure 14.5.

φ0(t) + ϕ(t) e(t)

KD F(s)
–

φ0(t)
K0 u(t)
s

Figure 14.5 Linearized baseband model of the PLL.

We denote by P𝜙 (s) the Laplace transform of 𝜙; by taking the Laplace transform of (14.24) and assum-
ing 𝜑̂ 0 (0) = 0, we obtain
sP𝜙 (s) = sΦ0 (s) − KD K0 F(s) P𝜙 (s) (14.25)
̂ 0 (s), we derive the loop transfer function as
Substituting P𝜙 (s) with Φ0 (s) − Φ
̂ 0 (s)
Φ KF(s)
H(s) = = , K = KD K0 (14.26)
Φ0 (s) s + KF(s)
710 Chapter 14. Synchronization

Table 14.1: Three

expressions of 𝜑0 and
corresponding Laplace
transforms.

𝜑0 (t) Φ0 (s)
𝜑s
𝜑s 1(t)
s
𝜔s
𝜔s t 1(t)
s2
t2 𝜔r
𝜔r 1(t)
2 s3

Then from (14.26), we get the following two relations:

̂ 0 (s) = [1 − H(s)] Φ0 (s)
P𝜙 (s) = Φ0 (s) − Φ (14.27)
P𝜙 (s) 1
= (14.28)
Φ0 (s) 1 + [KF(s)∕s]
We define as steady state error 𝜙∞ the limit for t → ∞ of 𝜙; recalling the final value theorem, and
using (14.28), 𝜙∞ can be computed as follows:
1
𝜙∞ = lim 𝜙(t) = lim sP𝜙 (s) = lim sΦ0 (s) (14.29)
t→∞ s→0 s→0 1 + KF(s)∕s
We compute now the value of 𝜙∞ for the three expressions of 𝜑0 given in Table 14.1 along with the
corresponding Laplace transforms.
● Phase step: 𝜑0 (t) = 𝜑s 1(t);
[ ]
𝜑 1
𝜙∞ = lim s s
s→0 s 1 + [KF(s)∕s]
[ ] (14.30)
𝜑s s
= lim
s→0 s + KF(s)

thus we obtain
𝜙∞ = 0 ⇔ F(0) ≠ 0 (14.31)
Observe that (14.31) holds even if F(s) = 1, i.e. in case the loop filter is absent.
● Frequency step: 𝜑0 (t) = 𝜔s t 1(t);
[ ] [ ]
𝜔s 1 𝜔s
𝜙∞ = lim s 2 = lim (14.32)
s→0 s 1 + [KF(s)∕s] s→0 s + KF(s)

If we choose
F(s) = s−k F1 (s), with k ≥ 1 and 0 < |F1 (0)| < ∞ (14.33)
then 𝜙∞ = 0.
● Frequency ramp: 𝜑0 (t) = (𝜔r t2 ∕2) 1(t)
[ ] [ ]
𝜔r 1 𝜔r
𝜙∞ = lim s 3 = lim 2 (14.34)
s→0 s 1 + [KF(s)∕s] s→0 s + KF(s)s

If we use a loop filter of the type (14.33) with k = 1, i.e. with one pole at the origin, then we obtain
a steady state error 𝜙∞ given by
𝜔r
𝜙∞ = ≠0 (14.35)
KF1 (0)
14.2. The phase-locked loop 711

As a general rule we can state that, in the presence of an input signal having Laplace transform of
the type s−k with k ≥ 1, to get a steady state error 𝜙∞ = 0, a filter with at least (k − 1) poles at the
origin is needed.
The choice of the above elementary expressions of the phase 𝜑0 for the analysis is justified by the
fact that an arbitrary phase 𝜑0 can always be approximated by a Taylor series expansion truncated to the
second order, and therefore as a linear combination of the considered functions.

14.2.2 Analysis of the PLL in the presence of additive noise

We now extend the PLL baseband model and relative analysis to the case in which white noise 𝑤 with
spectral density N0 ∕2 is added to the signal 𝑣. Introducing the in-phase and quadrature components of
𝑤, from (1.96) we get the relation
𝑤(t) = 𝑤I (t) cos(𝜔0 t) − 𝑤Q (t) sin(𝜔0 t) (14.36)
where 𝑤I and 𝑤Q are two uncorrelated random processes having spectral density in the desired signal
band given by
𝑤I (f ) = 𝑤Q (f ) = N0 (14.37)
Letting
1
A KK𝑤 = (14.38)
2 2 m
the multiplier output signal e assumes the expression
e(t) = Km [𝑣(t) + 𝑤(t)] 𝑣VCO (t)
= KD sin [𝜙(t)] + K𝑤 𝑤Q (t) sin [𝜑̂ 0 (t)] + K𝑤 𝑤I (t) cos [𝜑̂ 0 (t)]
(14.39)
+ KD sin [2𝜔0 t + 𝜑0 (t) + 𝜑̂ 0 (t)] − K𝑤 𝑤Q (t) sin [2𝜔0 t + 𝜑̂ 0 (t)]
+ K𝑤 𝑤I (t) cos [2𝜔0 t + 𝜑̂ 0 (t)]
Neglecting the high-frequency components in (14.39), (14.20) becomes
t
u(t) = f (t − 𝜉){KD sin [𝜙(𝜉)] + K𝑤 𝑤Q (𝜉) sin[𝜑̂ 0 (𝜉)] + K𝑤 𝑤I (𝜉) cos[𝜑̂ 0 (𝜉)]} d𝜉 (14.40)
∫0
Defining the noise signal
𝑤e (t) = K𝑤 [𝑤I (t) sin 𝜑̂ 0 (t) + 𝑤Q (t) cos 𝜑̂ 0 (t)] (14.41)
from (14.21) we get the integro-differential equation that describes the dynamics of the PLL in the
presence of noise, expressed as
t
d𝜙(t) d𝜑0 (t)
= − K0 f (t − 𝜉){KD sin[𝜙(𝜉)] + 𝑤e (𝜉)} d𝜉 (14.42)
dt dt ∫0
From (14.42), we obtain the PLL baseband model illustrated in Figure 14.6.

Noise analysis using the linearity assumption

In the case 𝜙(t) ≃ 0, we obtain the linearized PLL baseband model shown in Figure 14.7.
We now determine the contribution of the noise 𝑤 to the phase error 𝜙 in terms of variance of the
phase error, 𝜎𝜙2 , assuming that the phase of the desired input signal is zero, or 𝜑0 (t) = 0. From (14.14),
we obtain
𝜑̂ 0 (t) = −𝜙(t) (14.43)
712 Chapter 14. Synchronization

we(t)

φ0(t) + ϕ(t) e(t)

KD sin(.) F(s)
–

φ0(t)
K0 u(t)
s

Figure 14.6 PLL baseband model in the presence of noise.

we(t)

φ0(t) + ϕ(t) e(t)

KD F(s)
–

φ0(t)
K0 u(t)
s

Figure 14.7 Linearized PLL baseband model in the presence of additive noise.

Recalling that the transfer function of a filter that has 𝑤e as input and 𝜑̂ 0 as output is given by
(1∕KD ) H(s) (see (14.26)), the spectral density of 𝜙 is given by
1
𝜙 (f ) = 𝜑̂ 0 (f ) = |(f )|2 𝑤e (f ) (14.44)
KD2
where
(f ) = H(j2𝜋f ) (14.45)
To obtain 𝑤e (f ), we use (14.41). Assuming 𝑤I and 𝑤Q are uncorrelated white random processes with
autocorrelation
𝚛𝑤I (𝜏) = 𝚛𝑤Q (𝜏) = N0 𝛿(𝜏) (14.46)
and using the property of the Dirac function 𝛿(𝜏)f (t + 𝜏) = 𝛿(𝜏)f (t), the autocorrelation of 𝑤e turns out
to be
𝚛𝑤e (t, t − 𝜏) = K𝑤2 𝚛𝑤I (𝜏){sin[𝜑̂ 0 (t)] sin[𝜑̂ 0 (t − 𝜏)] + cos[𝜑̂ 0 (t)] cos[𝜑̂ 0 (t − 𝜏)]}
= K𝑤2 N0 𝛿(𝜏){sin2 [𝜑̂ 0 (t)] + cos2 [𝜑̂ 0 (t)]} (14.47)
= K𝑤2 N0 𝛿(𝜏)
Taking the Fourier transform of (14.47), we get
𝑤e (f ) = K𝑤2 N0 (14.48)
Therefore using (14.18) and (14.38), from (14.44) we get the variance of the phase error, given by
+∞
1 N0 +∞
𝜎𝜙2 = |(f )| 2
𝑤 (f ) d f = |(f )|2 d f (14.49)
∫−∞ KD2 e
A21 ∫−∞
From (1.74) we now define the equivalent noise bandwidth of the loop filter as
+∞
∫0 |(f )|2 d f
BL = (14.50)
|(0)|2
14.2. The phase-locked loop 713

Then (14.49) can be written as

2N0 BL
𝜎𝜙2 = (14.51)
A21
where A21 ∕2 is the statistical power of the desired input signal, and N0 BL = (N0 ∕2)2BL is the input noise
power evaluated over a bandwidth BL .
In Table 14.2, the expressions of BL for different choices of the loop filter F(s) are given.

Table 14.2: Expressions of BL for different choices of the loop filter F(s).

Loop order F(s) H(s) BL

K K
First 1
s+K 4
s+a K(s + a) K+a
Second
s s2 + Ks + Ka 4
s+a K(s + a) K(K + a)
Sec.-imperfect
s+b s2 + (K + b)s + Ka 4(K + b)
s2 + as + b K(s2 + as + b) K(aK + a2 − b)
Third
s2 s3 + Ks2 + aKs + bK 4(aK − b)

14.2.3 Analysis of a second-order PLL

In this section, we analyse the behaviour of a second-order PLL, using the linearity assumption. In
particular, we find the expression of the phase error 𝜙 for the input signals given in Table 14.1, and we
evaluate the variance of the phase error 𝜎𝜙2 .
From Table 14.2, the transfer function of a second-order loop is given by
K(s + a)
H(s) = (14.52)
s2 + Ks + Ka
We define the natural radian frequency, 𝜔n , and the damping factor of the loop, 𝜁, as
√ 1 √
𝜔n = Ka, 𝜁= Ka (14.53)
2a
As K = 2𝜁𝜔n and a = 𝜔n ∕(2𝜁), (14.52) can be expressed as
2𝜁 𝜔n s + 𝜔2n
H(s) = (14.54)
s2 + 2𝜁 𝜔n s + 𝜔2n
Once the expression of 𝜑0 is known, 𝜙 can be obtained by (14.27) and finding the inverse transform
of P𝜙 (s). The relation is simplified if in place of s we introduce the normalized variable s̃ = s∕𝜔n ; in
this case, we obtain
s̃2
P𝜙 (̃s) = 2 Φ (̃s) (14.55)
s̃ + 2𝜁 s̃ + 1 0
which depends only on the parameter 𝜁.
In Figures 14.8–14.10, we show the plots of 𝜙, with 𝜁 as a parameter, for the three inputs of Table 14.1,
respectively. Note that for the first two inputs 𝜙 converges to zero, while for the third input it converges
to a non-zero value (see (14.35)), because F(s) has only one pole at the origin, as can be seen from
Table 14.2. We note that if 𝜙 is a phase step the speed of convergence increases with increasing 𝜁,
whereas if 𝜙 is a frequency step, the speed of convergence is maximum for 𝜁 = 1.
714 Chapter 14. Synchronization

0.8

0.6

0.4
ϕ(τ)
φs
0.2

0 ζ=5
ζ=2
−0.2 ζ=1
ζ = 0.7
ζ = 0.5
−0.4
0 2 4 6 8 10 12
τ = ω nt

Figure 14.8 Plots of 𝜙(𝜏) as a function of 𝜏 = 𝜔n t, for a second-order loop filter with a phase step
input signal: 𝜑0 (t) = 𝜑s 1(t).

0.6
ζ = 0.5
0.5
ζ = 0.7

0.4 ζ=1

0.3
ϕ(τ)
ωs/ωn ζ=2
0.2

ζ=5
0.1

−0.1
0 2 4 6 8 10 12
τ = ω nt

Figure 14.9 Plots of 𝜙(𝜏) as a function of 𝜏 = 𝜔n t, for a second-order loop filter with a frequency
step input signal: 𝜑0 (t) = 𝜔s t 1(t).
14.2. The phase-locked loop 715

1.4

1.2
ζ = 0.5
1 ζ = 0.7
ζ=1
0.8
ϕ(τ)
ωr /ω2n
0.6
ζ=2

0.4

ζ=5
0.2

0
0 2 4 6 8 10 12
τ = ωnt

Figure 14.10 Plots of 𝜙(𝜏) as a function of 𝜏 = 𝜔n t, for a second-order loop filter with a frequency
ramp input signal: 𝜑0 (t) = 𝜔r (t2 ∕2) 1(t).

3.5

2.5

2
BL
ωn
1.5

0.5

0
0 1 2 3 4 5 6
ζ

Figure 14.11 Plot of BL as function of 𝜁 for a second-order loop.

716 Chapter 14. Synchronization

In Figure 14.11, the plot of BL is shown as a function of 𝜁. As

( )
𝜁 1
BL = 𝜔n + (14.56)
2 8𝜁
we note that BL has a minimum for 𝜁 = 0.5, and that for 𝜁 > 0.5 it increases as (1∕2) 𝜁; the choice of 𝜁
is therefore critical and represents a trade-off between the variance of the phase error and the speed of
convergence.
For a detailed analysis of the second- and third-order loops, we refer to [1].

14.3 Costas loop

In the previous section, the PLL was presented as a structure capable of performing carrier recovery for
signals of the type
sCh (t) = sin(2𝜋f0 t + 𝜑0 ) (14.57)
and, in general, for signals that contain periodic components of period n∕f0 , with n positive integer.
We now discuss carrier recovery schemes for both PAM-double-side-band (DSB) (see (1.123) where
a is a PAM signal as given by (7.2)) and QAM (see Section 7.1.3) signals; these signals do not contain
periodic components, but are cyclostationary and hence have periodic statistical moments.
We express the generic received signal sCh , of PAM-DSB or QAM type, in terms of the complex
envelope s(bb)
Ch
, that for simplicity we denote by a, as
sCh (t) = Re[a(t) ej(2𝜋f0 t+𝜑0 ) ] (14.58)
If in the reference carrier of the complex envelope we include also the phase 𝜑0 , (7.36) becomes equal
to (14.5), and
1 a(t)
sC (t) = √ s(bb)
Ch
(t) = √ (14.59)
2 2
The expression of sC , apart from the delay 𝜀T and the phase offset e−j arg Ch (f0 ) , is given by (7.39).
The autocorrelation of sCh is given by (see also (1.228))
1
Re[𝚛a (t, t − 𝜏) ej2𝜋f0 𝜏 ]
𝚛sCh (t, t − 𝜏) =
2 (14.60)
1
+ Re[𝚛aa∗ (t, t − 𝜏) ej[2𝜋f0 (2t−𝜏)+2𝜑0 ] ]
2
from which the statistical power is obtained as
1 1
𝙼sCh (t) = 𝚛sCh (t, t) = E[ |a(t)|2 ] + Re{E[a2 (t)] ej(4𝜋f0 t+2𝜑0 ) } (14.61)
2 2

14.3.1 PAM signals

We assume that the channel frequency response C (f ), obtained from (14.5), is Hermitian2 ; then gC and
a are real valued, hence
|a(t)|2 = [a(t)]2 (14.62)
and (14.61) becomes
1
𝙼sCh (t) = 𝙼 (t)[1 + cos(4𝜋f0 t + 2𝜑0 )] (14.63)
2 a
2
In practice, it is sufficient that C (f ) is Hermitian in a small interval around f = 0 (see page 17).
14.3. Costas loop 717

As a is a cyclostationary random process with period T (see Example 1.7.9 on page 49), 𝙼a is periodic
of period T; therefore, 𝙼sCh is also periodic and, assuming 1∕T ≪ f0 , which is often verified in practice,
its period is equal to 1∕(2f0 ).
Suppose the signal sCh is input to a device (squarer) that computes the square of the signal. The
output of the squarer has a mean value (deterministic component) equal to the statistical power of sCh ,
given by (14.63); if the squarer is cascaded with a narrow passband filter N (f ) (see Figure 14.12), with
N (2f0 ) = 1, then the mean value of the filter output is a sinusoidal signal with frequency 2f0 , phase
2𝜑0 (in practice we need to sum also the phase introduced by the filter), and amplitude (1∕2) 𝙼a .

frequency
divider
carrier to
r(t) r2(t) y(t)
(.)2 HN(s) ÷2 coherent
demodulator

Figure 14.12 Carrier recovery in PAM-DSB systems.

Assuming that N (f ) completely suppresses the components at low frequencies, the output filter signal
is given by
1
y(t) = Re[(h(bb)
N
∗ a2 )(t) ej[4𝜋f0 t+2𝜑0 ] ] (14.64)
2
This expression is obtained from the scheme of Figure 14.13 in which the product of two generic pass-
band signals, y1 and y2 , is expressed as a function of their complex envelopes, and decomposed into a
baseband component z1 and a passband component z2 centred at ±2f0 .

y1(t) = Re y(bb)
1
(t)e j (2π f0t + φ0) 1
LPF z1(t) = — Re y(bb)
1
(t)y(bb)*
2
(t)
2
z(t)

y2(t) = Re y(bb)
2
(t)e j(2πf0t + φ0) 1
PBF z2(t) = — Re y(bb)
1
(t)y(bb)
2
(t)e j(4π f0t + 2φ0)
2

Figure 14.13 Baseband and passband components of the product of two generic passband signals,
y1 and y2 , as a function of their complex envelopes.

If N (f ) is Hermitian around the frequency 2f0 , then (h(bb)

N
∗ a2 ) is real valued,3 and y can be written as

(h(bb)
N
∗ a2 )(t)
y(t) = cos(4𝜋f0 t + 2𝜑0 ) (14.65)
2
Thus, we have obtained a sinusoidal signal with frequency 2f0 , phase 2𝜑0 , and slowly varying amplitude,
function of the bandwidth of N (f ).
The carrier can be reconstructed by passing the signal y through a limiter, which eliminates the depen-
dence on the amplitude, and then to a frequency divider that returns a sinusoidal signal with frequency
and phase equal to half those of the square wave.

3
In PAM-single-side-band (SSB) and PAM-vestigial-side-band (VSB) systems, (h(bb)
N
∗ a2 )(t) will also contain a
quadrature component.
718 Chapter 14. Synchronization

In the case of a time-varying phase 𝜑0 , the signal y can be sent to a PLL with a VCO that operates at
frequency 2f0 , and generates a reference signal equal to
𝑣VCO (t) = −A sin(4𝜋f1 t + 2𝜑1 (t)) = −A sin(4𝜋f0 t + 2𝜑̂ 0 (t)) (14.66)
The signal 𝑣VCO must then be sent to a frequency divider to obtain the desired carrier. The block diagram
of this structure is illustrated in Figure 14.14. Observe that the passband filter N (f ) is substituted by the
lowpass filter LPF (f ), with LPF (0) = 1, inserted in the feedback (FB) loop of the PLL; this structure
is called the squarer/PLL.

r(t) e(t)
(.)2 HLPF(s)

vVCO(t)
÷2
carrier to
coherent u(t)
frequency VCO F(s)
demodulator divider
loop
filter

Figure 14.14 Squarer/PLL for carrier recovery in PAM-DSB systems.

An alternative tracking structure to the squarer/PLL, called Costas loop, is shown in Figure 14.15. In
a Costas loop, the signal e is obtained by multiplying the I and Q components of the signal r; the VCO
directly operates at frequency f0 , thus eliminating the frequency divider, and generates the reconstructed
carrier √
𝑣VCO (t) = 2A cos(2𝜋f0 t + 𝜑̂ 0 (t)) (14.67)
By the equivalences of Figure 14.13, we find that the input of the loop filter is identical to that of the
squarer/PLL and is given by
A
e(t) = a2 (t) sin[2𝜙(t)] (14.68)
4
where 𝜙 is the phase error (14.14).

demodulated signal
LPF rC(t)

vVCO(t)

r(t) u(t) e(t)

VCO F(s)

loop
π filter
—
2

LPF

Figure 14.15 Costas loop for PAM-DSB systems.

14.3. Costas loop 719

14.3.2 QAM signals

The schemes of Figures 14.12 and 14.14 cannot be directly applied to QAM systems. Indeed, the sym-
metry of the constellation of a QAM system usually leads to E[a2 (t)] = 0 (see (1.328)); therefore, the
periodic component in (14.61) is suppressed.
We compute now the fourth power of sCh ; after a few steps, we obtain
1
s4Ch (t) = Re[a4 (t) exp(j8𝜋f0 t + j4𝜑0 )]
8
+ Re[|a(t)|2 a2 (t) exp(j4𝜋f0 t + j2𝜑0 )] (14.69)
3
+ |a(t)|4
8

frequency
divider
r 4(t) carrier to
r(t) y(t)
(.)4 HN(s) ÷4 coherent
demodulator

Figure 14.16 Carrier recovery in QAM systems.

Filtering s4Ch by a passband filter centred at ±4f0 (see Figure 14.16), eventually followed by a PLL in
the case of a time-varying phase, we obtain a signal y having a mean value given by
1
E[y(t)] = {E[a4I (t)] − 𝙼2aI } cos(8𝜋f0 t + 4𝜑0 ), aI (t) = Re[a(t)] = s(bb)
Ch,I
(14.70)
4
which is a periodic signal with period 1∕(4f0 ) and phase 4𝜑0 .
In the case of M-phase-shift-keying (PSK) signals, there exists a variant of the Costas loop called
extended Costas loop; the scheme for a quadrature phase-shift-keying (QPSK) system is illustrated in
Figure 14.17.
In the presence of additive noise, a passband filter centred at ±f0 is placed in front of all schemes to
limit the noise without distorting the desired signal. For the performance analysis, we refer to [2, 3], in
which similar conclusions to those described in Section 14.2 for the PLL are reached.

LPF (.)2

vVCO(t)
demodulated
r(t) u(t) e(t) + signal
VCO F(s) +
– rC(t)
loop
π filter
—
2

LPF (.)2

Figure 14.17 Extended Costas loop for QPSK systems.

720 Chapter 14. Synchronization

14.4 The optimum receiver

With reference to the signal model (14.7), once the carrier has been recovered by one of the two meth-
ods described in the previous sections, after demodulation, the three parameters (𝜃, 𝜀, Ω) need to be
estimated. In this section, the optimum receiver is obtained using the maximum-likelihood (ML) cri-
terion discussed in Chapter 6 (see also (2.167)). The next two sections synthetically describe various
estimation methods given in [4, 5].
From (14.7), the received signal rC can be expressed as
rC (t) = ej(Ωt+𝜃) sC (t) + 𝑤C𝜑 (t)
(14.71)
= sC (t; 𝜃, 𝜀, Ω) + 𝑤C𝜑 (t)
where sC is given by4
∑
+∞
sC (t) = ak qC (t − kT − 𝜀T) (14.72)
k=−∞
We express (14.71) using the vector notation, that is
r=s+𝒘 (14.73)
with the following assumptions:
1. the phase offset 𝜃 is equal to z,
2. the time shift 𝜀T is equal to eT,
3. the frequency offset Ω is equal to o,
4. the transmitted data sequence a is equal to 𝜶;
then the probability density function of r is given by (see (1.303))
( )
1
pr∣𝜃,𝜀,Ω,a (𝝆 ∣ z, e, o, 𝜶) =  exp − ∥ 𝝆 − s∥2 (14.74)
N0
The quantities ‖𝝆‖2 and ‖s‖2 are constants5 [3]; therefore, (14.74) is proportional to the likelihood
{ }
2
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp Re[𝝆T s∗ ] (14.75)
N0
Referring to the transmission of K symbols or to a sufficiently large observation interval TK = KT,
(14.75) can be written as
{ [ ]}
2
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp Re 𝜌(t) s∗C (t) e−j(ot+z) dt (14.76)
N0 ∫TK
Inserting the expression (14.72) of sC , limited to the transmission of K symbols, in (14.76) and inter-
changing the operations of summation and integration, we obtain
{ K−1 [ ]}
2 ∑
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp Re 𝛼k e
∗ −jz
𝜌(t) e −jot ∗
qC (t − kT − eT) dt (14.77)
N0 k=0 ∫TK

We introduce the matched filter6

gM (t) = q∗C (−t) (14.78)

4
The phasor e−j arg Ch (f0 ) is included in qC .
5
Here we are not interested in the detection of a, as in the formulation of Section 7.5, but rather in the estimate of the
parameters 𝜃, 𝜀, and Ω; in this case, if the observation is sufficiently long, we can assume that ‖s‖2 is invariant with
respect to the different parameters.
6
In this formulation, the filter gM is anticausal; in practice, a delay equal to the duration of qC must be taken into account.
14.4. The optimum receiver 721

and assume that the pulse (qC ∗ gM ) is a Nyquist pulse; therefore, there exists a suitable sampling phase
for which inter-symbol-interference (ISI) is avoided.
Last, if we denote the integral in (14.77) by x(kT + eT, o), that is
x(kT + eT, o) = (rC (𝜏) e−jo𝜏 ∗ gM (𝜏))(t)|t=kT+eT (14.79)
(14.77) becomes
{ }
2 ∑
K−1
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp ∗ −jz
Re[𝛼k x(kT + eT, o) e ] (14.80)
N0 k=0
Let us now suppose that the optimum values of z, e, o that maximize (14.80), i.e. the estimates of z, e, o
have been determined in some manner.

rC (t) rD(t) x(t) x(kT+εT,Ω)

gM(t) ak
εT+kT
e–j Ω t e–j θ

Figure 14.18 Analog receiver for QAM systems.

The structure of the optimum receiver derived from (14.80) is illustrated in Figure 14.18. The signal
̂ where Ω
rC is multiplied by exp{−jΩt}, ̂ is an estimate of Ω, to remove the frequency offset, then filtered
by the matched filter gM and sampled at the sampling instants kT + 𝜀T, ̂ where 𝜀̂ is an estimate of 𝜀.
The samples x(kT + 𝜀T,̂ Ω)̂ are then multiplied by exp{−j𝜃} ̂ to remove the phase offset. Last, the data
detector decides on the symbol â k that, in the absence of ISI, maximizes the k-th term of the summation
̂ 𝜀,
in (14.80) evaluated for (z, e, o) = (𝜃, ̂
̂ Ω):
â k = arg max Re[𝛼k∗ x(kT + 𝜀T,
̂ Ω)̂ e−j𝜃̂ ] (14.81)
𝛼k

The digital version of the scheme of Figure 14.18 is illustrated in Figure 14.19; it uses an anti-aliasing
filter and a sampler with period Tc such that (recall the sampling theorem on page 17)
1 Ω
≥ B + max (14.82)
2Tc 2𝜋

Observation 14.1
To simplify the implementation of the digital receiver, the ratio F0 = T∕Tc is chosen as an integer; in
this case, for F0 = 4 or 8, the interpolator filter may be omitted and the timing after the matched filter
has a precision of Tc = T∕F0 . This approach is usually adopted in radio systems for the transmission of
packet data.

To conclude this section, we briefly discuss the algorithms for timing and carrier phase recovery.

Timing recovery
Ideally, at the output of the anti-aliasing filter, the received signal should be sampled at the instants
t = kT + 𝜀T; however, there are two problems:
1. the value of 𝜀 is not known;
2. the clock at the receiver allows sampling at multiples of Tc , not at multiples of T, and the ratio
T∕Tc is not necessarily a rational number.
Therefore, time synchronization methods are usually composed of two basic functions.
 μk

timing
estimator

rC (t) rAA(t) rAA(nTc) rD(nTc) x(kT + εT,Ω) ak

gAA(t) hI (iTc, μk) gM (iTc)
nTc x(nTc + μkTc,Ω)
(lead of μkTc) matched
interpolator filter
filter
rD(nTc + μkTc) ε–j θ (kT)
ε–j ΩnTc frequency
error
detection
frequency phase
integrator estimator

real signals
complex signals

Figure 14.19 Digital receiver for QAM systems. Source: Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.
14.4. The optimum receiver 723

Timing estimate The first function gives an estimate 𝜀̂ of 𝜀.

Interpolation and decimation The sampling instants t = kT + 𝜀T

̂ can be written as
[ ]
T T
kT + 𝜀T
̂ = k + 𝜀̂ T (14.83)
Tc Tc c
The expression within brackets admits the following decomposition:
[ ] ( )⌋
T T T T
k + 𝜀̂ = k + 𝜀̂ + 𝜇k
Tc Tc Tc Tc (14.84)
= 𝚖k + 𝜇k
Given a real number a, ⌊a⌋ denotes the largest integer smaller than or equal to a (floor), and 𝜇 = a − ⌊a⌋
is the fractional part that we denote as [a]F . Suppose now that the estimate of 𝜀 is time varying; we denote
it by 𝜀̂ k . Consider the (k + 1)-th sampling instant, expressed as
(k + 1) T + 𝜀̂ k+1 T (14.85)
By summing and subtracting 𝜀̂ k T, (14.85) can be rewritten as follows:
(k + 1) T + 𝜀̂ k+1 T = kT + 𝜀̂ k T + T + (𝜀̂ k+1 − 𝜀̂ k ) T (14.86)
Substituting kT + 𝜀̂ k T with 𝚖k Tc + 𝜇k Tc , we obtain
(k + 1) T + 𝜀̂ k+1 T = 𝚖k Tc + 𝜇k Tc + T + (𝜀̂ k+1 − 𝜀̂ k ) T
[ ]
T (14.87)
= 𝚖k + 𝜇k + (1 + (𝜀̂ k+1 − 𝜀̂ k )) Tc
Tc
Recalling that 𝚖k is a positive integer, and 𝜇k is real valued and belongs to the interval [0,1), from (14.87)
the following recursive expressions for 𝚖k and 𝜇k are obtained:
⌊ ⌋
T
𝚖k+1 = 𝚖k + 𝜇k + (1 + (𝜀̂ k+1 − 𝜀̂ k ))
Tc
[ ] (14.88)
T
𝜇k+1 = 𝜇k + (1 + (𝜀̂ k+1 − 𝜀̂ k ))
Tc F

saw (x)

1/2

1/2 1 x

Figure 14.20 Plot of the saw-tooth function saw(x).

The quantity saw(𝜀̂ k+1 − 𝜀̂ k ) is often substituted for (𝜀̂ k+1 − 𝜀̂ k ), where saw(x) is the saw-tooth function
illustrated in Figure 14.20. Thus, the difference between two successive estimates belongs to the interval
[−1∕2, 1∕2]; this choice reduces the effects that a wrong estimate of 𝜀 would have on the value of the
pair (𝚖k , 𝜇k ).
724 Chapter 14. Synchronization

(a)

(k –1)T + εT kT + εT (k + 1)T + εT (k + 2)T + εT (k + 3)T + εT (k + 4)T + εT

(b)
μk – 1 Tc μk Ts μk+ 1 Tc μk + 2 Tc μk +3 Tc μk +4 Tc

mk – 1 Tc mk Tc mk+ 1 Ts mk +2 Tc mk +3 Tc mk + 4 Tc lTc

Figure 14.21 (a) Transmitter time scale; (b) receiver time scale with Tc < T, for the ideal case
𝜀̂ = 𝜀. Source: Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.

Figure 14.21 illustrates the graphic representation of (14.84) in the ideal case 𝜀̂ = 𝜀. The transmitter
time scale, defined by multiples of T, is shifted by a constant quantity equal to 𝜀T. The receiver time
scale is defined by multiples of Tc . The fact that the ratio T/Tc may be a non-rational number has two
consequences: first, the time shift 𝜇k Tc is time varying even if 𝜀T is a constant; second, the instants 𝚖k Tc
form a non-uniform subset of the receiver time axis, such that on average the considered samples are
separated by an interval T.
With reference to Figure 14.19, to obtain the samples of the signal x at the instants (𝚖k + 𝜇k ) Tc , we
can proceed as follows:
(a) implement a digital interpolator filter that provides samples of the received signal at the instants
(n + 𝜇k ) Tc , starting from samples at the instants nTc (see Section 1.A.5);
(b) implement a downsampler that yields samples at the instants (𝚖k + 𝜇k ) Tc = kT + 𝜀T.̂
With regard to the digital interpolator filter, consider a signal rD with bandwidth BrD ≤ 1∕(2Tc ). From
the sampling theorem, the signal rD can be reconstructed from its samples rD (iTc ) using the relation
( )
∑
+∞
t − iTc
rD (t) = rD (iTc )sinc (14.89)
i=−∞
Tc
This expression is valid for all t; in particular, it is valid for t = t1 + 𝜇k Tc , thus yielding the signal rD (t1 +
𝜇k Tc ). Sampling this signal at t1 = nTc , we obtain
∑
+∞
rD (nTc + 𝜇k Tc ) = rD (iTc )sinc (n + 𝜇k − i) (14.90)
i=−∞

Observe that the second member of (14.90) is a discrete-time convolution; in fact, introducing the
interpolator filter with impulse response hI and parameter 𝜇k ,
hI (iTc ; 𝜇k ) = sinc(i + 𝜇k ), i = −∞, … , +∞ (14.91)
(14.90) can be rewritten as
rD (nTc + 𝜇k Tc ) = [rD (iTc ) ∗ hI (iTc ; 𝜇k )] (nTc ) (14.92)
In other words, to obtain from samples of rD at instants nTc the samples at nTc + 𝜇k Tc , we can use a
filter with impulse response hI (iTc ; 𝜇k ).7

7
In practice, the filter impulse response hI (⋅; 𝜇k ) must have a finite number N of coefficients. The choice of N depends on
the ratio T∕Tc and on the desired precision; for example for T∕Tc = 2 and a normalized mean-square error (MSE),
14.5. Algorithms for timing and carrier phase recovery 725

With regard to the cascade of the matched filter gM (iTc ) and the decimator at instants 𝚖k Tc , we point
out that a more efficient solution is to implement a filter with input at instants nTc that generates output
samples only at instants 𝚖k Tc .
We conclude this section by recalling that, if after the matched filter gM , or directly in place of gM ,
there is an equalizer filter c with input signal having sampling period equal to Tc , the function of the
filter hI is performed by the filter c itself (see Section 7.4.3).

Carrier phase recovery

An offset of the carrier phase equal to 𝜃 has the effect of rotating the complex symbols by exp(j𝜃); this
̂ where 𝜃̂ is an estimate of 𝜃.
error can be corrected by multiplying the matched filter output by exp(−j𝜃),
Carrier phase recovery consists of three basic functions:

Phase estimate In the scheme of Figure 14.19, phase estimation is performed after the matched filter,
using samples with sampling period equal to the symbol period T. In this scheme, timing recovery is
implemented before phase recovery, and must operate in one of the following modes:
(a) with an arbitrary phase offset;
̂
(b) with a phase estimate anticipating the multiplication by e−j𝜃 after the decimator;
(c) jointly recovering phase and timing.

Phase rotation
(a) The samples x(kT + 𝜀T,̂ Ω)̂ are multiplied by the complex signal exp(−j𝜃(kT))
̂ (see Figure 14.19);
̂
a possible residual frequency offset ΔΩ can be corrected by a time-varying phase given by 𝜃(kT) =
̂
̂𝜃 + kT ΔΩ.
(b) The samples x(kT + 𝜀T, ̂ e−j𝜃̂ are input to the data detector, assuming (𝜃,
̂ Ω) ̂ 𝜀)
̂ are the true values
of (𝜃, 𝜀).

Frequency synchronization A first coarse estimate of the frequency offset needs to be performed in
the analog domain. In fact, algorithms for timing and phase recovery only work in the presence of a
small residual frequency offset. A second block provides a fine estimate of this residual offset that is
used for frequency offset compensation.

14.5 Algorithms for timing and carrier phase recovery

In this section, we discuss digital algorithms to estimate the time shift and the carrier phase offset under
the assumption of absence of frequency offset, or Ω = 0. Thus, the output samples of the decimator of
Figure 14.19 are expressed as x(kT + 𝜀T, ̂ 0); they will be simply denoted as x(kT + 𝜀T),̂ or in compact
notation as xk (𝜀).
̂

given by
1 1∕(2T)
J = 2T |ej2𝜋f 𝜇 − I (f ; 𝜇)|2 df d𝜇 (14.93)
∫0 ∫−1∕(2T)
where
∑
N∕2
I (f ; 𝜇) = hI (iTc ; 𝜇) e−j2𝜋fiTc (14.94)
i=−(N∕2)+1
equal to −50 dB, it turns out N ≃ 5. Of course, more efficient interpolator filters than that defined by (14.91) can be
utilized.
726 Chapter 14. Synchronization

14.5.1 ML criterion
The expression of the likelihood is obtained from (14.80) assuming o = 0, that is
{ K−1 }
2 ∑
𝙻𝜃,𝜀,a (z, e, 𝜶) = exp ∗ −jz
Re[𝛼k xk (e) e ]
N0 k=0
{ } (14.95)
∏
K−1
2
= exp Re[𝛼k∗ xk (e) e−jz ]
k=0
N0

Assumption of slow time varying channel

In general, both the time shift and the phase offset are time varying; from now on, we assume that
the rate at which these parameters vary is much lower than the symbol rate 1∕T. Thus, it is useful to
consider two time scales: one that refers to the symbol period T, for symbol detection and estimation of
synchronization parameters, and the other that refers to a period much larger than T, for the variation of
the synchronization parameters.

14.5.2 Taxonomy of algorithms using the ML criterion

Synchronization algorithms are obtained by the ML criterion (see (14.77) or equivalently (14.74)) aver-
aging the probability density function pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶) with respect to parameters that do not need to
be estimated. Then, we obtain the following likelihood functions:
● for the joint estimate of (𝜃, 𝜀),
∑
pr∣𝜃,𝜀 (𝝆 ∣ z, e) = pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶) P[a = 𝜶] (14.96)
𝜶

● for the estimate of the phase,

[ ]
∑
pr∣𝜃 (𝝆 ∣ z) = pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶) P[a = 𝜶] p𝜀 (e) de (14.97)
∫ 𝜶

● for the estimate of timing,

[ ]
∑
pr∣𝜀 (𝝆 ∣ e) = pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶) P[a = 𝜶] p𝜃 (z) dz (14.98)
∫ 𝜶

With the exception of some special cases, the above functions cannot be computed in close form. Con-
sequently, we need to develop appropriate approximation techniques.
A first classification of synchronization algorithms is based on whether knowledge of the data
sequence is available or not; in this case, we distinguish two classes:
1. decision-directed (DD) or data-aided (DA);
2. non-data-aided (NDA).
If the data sequence is known, for example by sending a training sequence a = 𝜶 0 during the acquisition
phase, we speak of DA algorithms. As the sequence a is known, in the sum in the expression of the
likelihood function, only the term for 𝜶 = 𝜶 0 remains. For example, the joint estimate of (𝜃, 𝜀) reduces
to the maximization of the likelihood pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶 0 ), and we get
̂ 𝜀)
(𝜃, ̂ DA = arg max pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶 0 ) (14.99)
z,e
14.5. Algorithms for timing and carrier phase recovery 727

On the other hand, whenever we use the detected sequence â as if it were the true sequence a, we speak
of data-directed algorithms. If there is a high probability that â = a, again in the sum in the expression
of the likelihood function only one term remains. Taking again the joint estimate of (𝜃, 𝜀) as an example,
in (14.96) the sum reduces to
∑
pr∣𝜃,𝜀,a (𝝆 ∣ z, e, 𝜶) P[a = 𝜶] ≃ pr∣𝜃,𝜀,a (𝝆 ∣ z, e, â ) (14.100)
𝜶

as P[a = â ] ≃ 1. The joint estimate (𝜃, 𝜀) is thus given by

̂ 𝜀)
(𝜃, ̂ DD = arg max pr∣𝜃,𝜀,a (𝝆 ∣ z, e, â ) (14.101)
z,e

NDA algorithms apply instead, in an exact or approximate fashion, the averaging operation with
respect to the data sequence.
A second classification of synchronization algorithms is made based on the synchronization parame-
ters that must be eliminated; then, we have four cases
1. DD&D𝜀: data and timing directed,
̂ â )
pr∣𝜃 (𝝆 ∣ z) = pr∣𝜃,𝜀,a (𝝆 ∣ z, 𝜀, (14.102)
2. DD, timing independent,

pr∣𝜃 (𝝆 ∣ z) = pr∣𝜃,𝜀,a (𝝆 ∣ z, e, â ) p𝜀 (e) de (14.103)

∫
3. DD&D𝜃: data and phase directed,
̂ e, â )
pr∣𝜀 (𝝆 ∣ e) = pr∣𝜃,𝜀,a (𝝆 ∣ 𝜃, (14.104)
4. DD, phase independent or non-coherent (NC),

pr∣𝜀 (𝝆 ∣ e) = pr∣𝜃,𝜀,a (𝝆 ∣ z, e, â ) p𝜃 (z) dz (14.105)

∫
A further classification is based on the method for obtaining the timing and phase estimates:
1. Feedforward (FF): The FF algorithms directly estimate the parameters (𝜃, 𝜀) without using signals
that are modified by the estimates; this implies using signals before the interpolator filter for timing
recovery and before the phase rotator (see Figure 14.19) for carrier phase recovery.
2. Feedback (FB): The FB algorithms estimate the parameters (𝜃, 𝜀) using also signals that are mod-
ified by the estimates; in particular, they yield an estimate of the errors e𝜃 = 𝜃̂ − 𝜃 and e𝜀 = 𝜀̂ − 𝜀,
which are then used to control the interpolator filter and the phase rotator, respectively. In general,
FB structures are able to track slow changes of parameters.
Next, we give a brief description of FB estimators, with emphasis on the fundamental blocks and on
input–output relations.

Feedback estimators
In Figure 14.22, the block diagrams of a FB phase (FB𝜃) estimator and of a FB timing (FB𝜀) estimator
are illustrated. These schemes can be easily extended to the case of a FB frequency offset estimator. The
two schemes only differ in the first block. In the case of the FB𝜃 estimator, the first block is a phase
rotator that, given the input signal s(kT) (x(kT + 𝜀T) ̂ in Figure 14.19), yields s(kT) exp(−j𝜃̂k ), where 𝜃̂k
is the estimate of 𝜃 at instant kT; in the case of the FB𝜀 estimator, it is an interpolator filter that, given the
input signal s(kT) (rD (nTc ) in Figure 14.19), returns s(kT + 𝜀̂ k T) (rD (kT + 𝜀T)
̂ in Figure 14.19), where
𝜀̂ k is the estimate of 𝜀 at instant kT.
728 Chapter 14. Synchronization

(a) s(kT ) phase s(kT )e–j θk error e(kT)

rotator detector

θk
loop filter
u(kT)
NCO F(z)

(b) s(kT)
s(kT + εkT) error e(kT)
interpolator
detector

εk loop filter
u(kT)
NCO F(z)

Figure 14.22 (a) Feedback estimator of 𝜃; (b) feedback estimator of 𝜀.

We analyse only the FB𝜃 estimator, as the FB𝜀 estimator is similar. The error detector block is the
fundamental block and has the function of generating a signal e(kT), called error signal, whose mean
value is written as
E[e(kT)] = g𝜃 (𝜃 − 𝜃̂k ) (14.106)

where g𝜃 (⋅) is an odd function. Then, the error signal e(kT) admits the following decomposition

e(kT) = g𝜃 (𝜃 − 𝜃̂k ) + 𝜂𝜃 (kT; 𝜃; 𝜃̂k ) (14.107)

where 𝜂𝜃 (⋅) is a disturbance term called loop noise; the loop filter F(z) is a lowpass filter that has two
tasks: it regulates the speed of convergence of the estimator and mitigates the effects of the loop noise.
The loop filter output, u(kT), is input to the numerically controlled oscillator (NCO), that updates the
phase estimate 𝜃 according to the following recursive relation:

𝜃̂k+1 = 𝜃̂k + 𝜇𝜃 u(kT) (14.108)

where 𝜇𝜃 denotes the NCO gain.

In most FB estimators, the error signal e(kT) is obtained as the derivative of the likelihood, or of its
logarithm, with respect to the parameter to be estimated, evaluated using the most recent estimates 𝜃̂k ,
𝜀̂ k . In particular, we have the two cases
𝜕 |
e(kT) ∝ 𝙻 (𝜃̂ , e, 𝜶)||
𝜕e 𝜃,𝜀,a k |e=𝜀̂k
| (14.109)
𝜕
e(kT) ∝ 𝙻 (z, 𝜀̂ k , 𝜶)||
𝜕z 𝜃,𝜀,a |z=𝜃̂k
We note that in FB estimators, the vector a represents the transmitted symbols from instant 0 up to the
instant corresponding to the estimates 𝜃̂k , 𝜀̂ k .

Early-late estimators
Early-late estimators constitute a subclass of FB estimators, where the error signal is computed according
to (14.109), and the derivative operation is approximated by a finite difference [3], i.e. given a signal p,
14.5. Algorithms for timing and carrier phase recovery 729

its derivative is computed as follows:

d p(t) p(t + 𝛿) − p(t − 𝛿)
≃ (14.110)
dt 2𝛿
where 𝛿 is a positive real number.
Consider, for example, a DD&D𝜃 timing estimator and denote by G𝙻 [⋅] the function that, given the
input signal xk (𝜀̂ k ), yields the likelihood 𝙻𝜀 (e) evaluated at e = 𝜀̂ k . From (14.110), the error signal is
given by
e(kT) ∝ G𝙻 [xk (𝜀̂ k + 𝛿)] − G𝙻 [xk (𝜀̂ k − 𝛿)] (14.111)

The block diagram of Figure 14.22b is modified into that of Figure 14.23. Observe that in the upper
branch the signal xk (𝜀̂ k ) is anticipated of 𝛿T, while in the lower branch, it is delayed of −𝛿T. Hence, the
name of early-late estimator.

+δT GL[.]
x(t) +
+
kT + εkT –
–δT GL[.]

εk loop filter
u(kT ) e(kT)
NCO F(z)

Figure 14.23 FB early-late timing estimator.

14.5.3 Timing estimators

Non-data aided
The estimate of 𝜀 can be obtained from (14.95) by eliminating the dependence on the parameters 𝜶 and
z that are not estimated. To remove this dependence, we take the expectation with respect to a; assuming
i.i.d. symbols, we obtain the likelihood
[ { }]
∏
K−1
2
𝙻𝜃,𝜀 (z, e) = Eak exp ∗ −jz
Re[ak xk (e) e ] (14.112)
k=0
N0
where Eak denotes the expectation with respect to ak .
For an M-PSK signal, with M > 2, we approximate ak as ej𝜑k , where 𝜑k is a uniform random variable
(r.v.) on (−𝜋; 𝜋]; then, (14.112) becomes
{ }
∏
K−1 +𝜋
2 d𝑣
𝙻𝜃,𝜀 (z, e) = exp Re[e−j𝑣k xk (e) e−jz ] k (14.113)
∫
k=0 −𝜋
N0 2𝜋
If we use the definition of the Bessel function (4.58), (14.113) is independent of the phase 𝜃 and we get
( )
∏
K−1
|xk (e)|
𝙻𝜀 (e) = I0 (14.114)
k=0
N0 ∕2
730 Chapter 14. Synchronization

On the other hand, if we take the expectation of (14.95) only with respect to the phase 𝜃, we get
( )
∏
K−1
|xk (e) 𝛼k∗ |
𝙻𝜀,a (e, 𝜶) = I0 (14.115)
k=0
N0 ∕2
We observe that, for M-PSK, 𝙻𝜀,a (e, 𝜶) = 𝙻𝜀 (e), as |𝛼k | is a constant, while this does not occur for
M-QAM.
To obtain estimates from the two likelihood functions just obtained, if the signal-to-noise ratio (SNR)
Γ is sufficiently high, we utilize the fact that I0 (⋅) can be approximated as
𝜁2
I0 (𝜁) ≃ 1 + for |𝜁| ≪ 1 (14.116)
2
Taking the logarithm of the likelihood and eliminating non-relevant terms, we obtain the following NDA
estimator and DA estimator.
NDA ∶ 𝜀̂ = arg max ln{𝙻𝜀 (e)}
e

∑
K−1
≃ arg max |xk (e)|2 (14.117)
e
k=0

DA ∶ 𝜀̂ = arg max ln{𝙻𝜀,a (e, 𝜶)}

e

∑
K−1
≃ arg max |xk (e)|2 |𝛼k |2 (14.118)
e
k=0

On the other hand, if the SNR Γ ≪ 1, (14.112) can be approximated using a power series expansion of
the exponential function. Taking the logarithm of (14.112), using the hypothesis of i.i.d. symbols, with
E[an ] = 0, and eliminating non-relevant terms, we obtain the following log-likelihood:
[ ]
∑
K−1
∑
K−1
𝓁𝜃,𝜀 (z, e) = E[|an | ]
2
|xk (e)| + Re E[an ]
2 2 ∗ 2 j2z
(xk (e)) e (14.119)
k=0 k=0

Averaging with respect to 𝜃, we obtain the following phase independent log-likelihood:

∑
K−1
𝓁𝜀 (e) = |xk (e)|2 (14.120)
k=0

which yields the same NDA estimator as (14.117).

For a modulation technique characterized by E[a2n ] ≠ 0, (14.119) may be used to obtain an NDA joint
estimate of phase and timing. In fact for a phase estimate given by
{ }
1 ∑
K−1
𝜃̂ = − arg E[a2n ] (xk∗ (e))2 (14.121)
2 k=0

the second term of (14.119) is maximized. Substitution of (14.121) in (14.119) yields a new estimate 𝜀̂
given by
∑
K−1 | ∑
K−1 |
| |
𝜀̂ = arg max E[|an |2 ] |xk (e)|2 + ||E[a2n ] xk2 (e)|| (14.122)
e | |
k=0 | k=0 |
The block diagram of the joint estimator is shown in Figure 14.24, where P values of the time shift
𝜀, 𝜀(m) , m = 1, … , P, equally spaced in [−1∕2, 1∕2] are considered; usually, the resolution obtained
with P = 8 or 10 is sufficient. For each time shift 𝜀(m) , the log-likelihood (14.122) is computed and the
value of 𝜀(m) associated with the largest value of the log-likelihood is selected as the timing estimate.
Furthermore, we observe that in the generic branch m, filtering by the matched filter gM (iTc + 𝜀(m) T)
 |xk(ε (1))|2 K–1
|.|2 Σ (.)
k= 0

xk(ε (1)) (xk(ε (1)))2 K–1

gM(iTc + ε (1)T ) (.)2 Σ (.) |.|
kT k= 0

E[a2n]
(xk(ε (1)))2 E[a2n]
arg max

rAA(t) rAA(nTc) ε 1 K–1

– — arg E[a2n] Σ (xk*(ε))2 θ
2 k=0
nTc
|xk(ε (P))|2 K–1
|.|2 Σ (.)
k=0

xk(ε (P)) (xk (ε (P)))2 K–1

gM(iTc + ε (P)T ) (.)2 Σ (.) |.|
k= 0
kT
E[a2n]
(xk(ε (P)))2 E[a2n]

Figure 14.24 NDA joint timing and phase (for E[a2n ] ≠ 0) estimator. Source: Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.
732 Chapter 14. Synchronization

and sampling at the instants kT can be implemented by the cascade of an interpolator filter hI (iTc ; 𝜇 (m) )
(where 𝜇 (m) depends on 𝜀(m) ) and a filter gM (iTc ), followed by a decimator that provides samples at the
instants 𝚖k Tc , as illustrated in Figure 14.19 and described in Section 14.4.

NDA synchronization via spectral estimation

Let us consider the log-likelihood (14.120) limited to a symmetric observation time interval [−LT, LT].
Thus, we get
∑L
𝓁𝜀 (e) = |x(kT + eT)|2 (14.123)
k=−L

Now, as x is a QAM signal, the process |x(kT + eT)|2 is approximately cyclostationary in e of period 1
(see Section 7.1.2). We introduce the following Fourier series representation
∑
+∞
|x(kT + eT)|2 = c(k)
i
ej2𝜋ie (14.124)
i=−∞

where the coefficients {c(k)

i
} are r.v.s. given by
1
c(k) = |x(kT + eT)|2 e−j2𝜋ie de (14.125)
i ∫0
Now (14.123) is equal to the average of the cyclostationary process |x(kT + eT)|2 in the interval [−L; L].
Defining
∑L
ci = c(k)
i
(14.126)
k=−L
it results [4] that only c0 and c1 have non-zero mean, and (14.123) can be written as
∑
𝓁𝜀 (e) = c0 + 2Re[c1 ej2𝜋e ] + 2Re[ci ej2𝜋ie ] (14.127)
|i|≥2
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟
disturbance with zero mean
for each value of e
As c0 and |c1 | are independent of e, the maximum of 𝓁𝜀 (e) yields
1
𝜀̂ = −
arg c1 (14.128)
2𝜋
However, the coefficient c1 is obtained by integration, which in general is hard to implement in the
digital domain; on the other hand, if the bandwidth of |x(lT)|2 satisfies the relation
1 1
B|x|2 = (1 + 𝜌) < (14.129)
T 2Tc
where 𝜌 is the roll-off factor of the matched filter, then c1 can be computed by discrete Fourier transform
(DFT). Let F0 = T∕Tc , then we get
[ ]
∑L
1 ∑
F0 −1
c1 = |x([kF0 + l] Tc )|2 e−j(2𝜋∕F0 )l (14.130)
k=−L
F0 l=0
A simple implementation of the estimator is possible for F0 = 4; in fact, in this case no multiplications
are needed. As e−j(2𝜋∕4)l = (−j)l , (14.130) simplifies into
[ ]
∑L
1 ∑
3
c1 = |x([4k + l] Tc )| (−j)
2 l
(14.131)
k=−L
4 l=0
Figure 14.25 illustrates the implementation of the estimator for F0 = 4.
14.5. Algorithms for timing and carrier phase recovery 733

L Im(c1)
–
+ Σ (.)
+ k=–L

|x[(4k – 1)Tc]|2 |x[(4k – 3)Tc]|2

(4kTc) ...

– — arg(c1)
|x(nTc)|2 ε
Tc Tc Tc

2π
1
(4kTc) ...

|x(4kTc)|2 |x[(4k – 2)Tc]|2

– L Re(c1)
+
+ Σ (.)
k=–L

Figure 14.25 NDA timing estimator via spectral estimation for the case F0 = 4. Source:
Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.

Data aided and data directed

If in (14.95) we substitute the parameters 𝛼k and z with their estimates, we obtain the phase independent
DA (DD) log-likelihood { [K−1 ]}
2 ∑
−j𝜃̂
𝙻𝜀 (e) = exp Re â k xk (e) e
∗
(14.132)
N0 k=0

from which we immediately derive the estimate

𝜀̂ = arg max 𝙻𝜀 (e) (14.133)
e
The block diagram of the estimator is shown in Figure 14.26. Note that this algorithm can only be used
in the case phase recovery is carried out before timing recovery.

a*k

xk (ε (1))
gM(iTc + ε (1)) Re[Σ(.)]
k
kT
arg max

rAA(t) ε
...

...

nTc
xk (ε (P))
gM(iTc + ε (P)) Re[Σ(.)]
k
ε–j θ kT

Figure 14.26 Phase independent DA (DD) timing estimator. Source: Reproduced with permission
from Meyr et al. [4]. ©1998, IEEE.

For a joint phase and timing estimator, from (14.95) we get

{ [K−1 ]}
2 ∑
𝙻𝜃,𝜀 (z, e) = exp Re â k xk (e) e
∗ −jz
(14.134)
N0 k=0
734 Chapter 14. Synchronization

Defining
∑
K−1
𝚛(e) = â ∗k xk (e) (14.135)
k=0

the estimation algorithm becomes

̂ 𝜀)
(𝜃, ̂ = arg max Re[𝚛(e) e−jz ]
z,e
(14.136)
= arg max |𝚛(e)| Re[e−j(z−arg(𝚛(e))) ]
z,e

The two-variable search of the maximum reduces to a single-variable search; as a matter of fact, once
the value of e that maximizes |𝚛(e)| is obtained, which is independent of z, the second term
Re[e−j(z−arg(𝚛(e))) ] (14.137)
is maximized by z = arg(𝚛(e)). Therefore, the joint estimation algorithm is given by
|K−1 |
|∑ ∗ |
𝜀̂ = arg max |𝚛(e)| = arg max || â k xk (e)||
e e | |
| k=0 | (14.138)
∑
K−1
𝜃̂ = arg 𝚛(𝜀)
̂ = arg â ∗k xk (𝜀)
̂
k=0

Figure 14.27 illustrates the implementation of this second estimator. Note that this scheme is a particular
case of (7.603).

a*k

xk (ε (1)) Σ(.)
gM(iTc + ε (1))
r(ε1) ε
arg max |r(ε)|

kT k
rAA(t)
...

...

nTc θ
xk (ε (P)) arg r(ε)
gM(iTc + ε (P)) Σ(.)
kT k r(εP)

Figure 14.27 DA (DD) joint phase and timing estimator. Source: Reproduced with permission from
Meyr et al. [4]. ©1998, IEEE.

For both estimators, estimation of the synchronization parameters is carried out every K samples,
according to the assumption of slow parameter variations made at the beginning of the section.

Observation 14.2
When the channel is not known, to implement the matched filter gM , we need to estimate the over-
all impulse response qC . Then, the estimation of qC , for example by one of the methods presented in
Appendix 1.C, and of timing can be performed jointly.
Let F0 = T∕Tc and Q0 = T∕TQ be integers, with Q0 ≥ F0 . From the signal {rAA (qTQ )}, obtained by
oversampling rAA or by interpolation of {rAA (nTc )}, and the knowledge of the training sequence {ak },
k = 0, … , LTS − 1, we estimate either qC with sampling period TQ or equivalently its Q0 ∕F0 polyphase
components with sampling period Tc (see Observation 7.6). Limiting the estimate to the more significant
consecutive samples around the peak, the estimating of the timing phase with precision TQ coincides with
14.5. Algorithms for timing and carrier phase recovery 735

selecting the polyphase component with the largest energy among the Q0 ∕F0 polyphase components.
This determines the optimum filter gM with sampling period Tc . Typically, for radio systems F0 = 2, and
Q0 = 4 or 8.

Data and phase directed with feedback: differentiator scheme

Differentiating the log-likelihood (14.95) with respect to e, neglecting non-relevant terms, and evaluating
̂ e, â ), we obtain
the result at (𝜃,
[K−1 ]
𝜕 ∑
∗ 𝜕 −j𝜃̂
ln{𝙻𝜀 (e)} ∝ Re â k x(kT + eT) e (14.139)
𝜕e k=0
𝜕e
With reference to the scheme of Figure 14.22, if we suppose that the sum in (14.139) is approximated
by the filtering operation by the loop filter F(z), the error signal e(kT) results
[ ]
𝜕 ̂
e(kT) = Re â ∗k x(kT + eT)|e=𝜀̂k e−j𝜃 (14.140)
𝜕e
The partial derivative of x(kT + eT) with respect to e can be carried out in the digital domain by a
differentiator filter with an ideal frequency response given by
1
d (f ) = j2𝜋f , |f | ≤ (14.141)
2Tc
In practice, if T∕Tc ≥ 2, it is simpler to implement a differentiator filter by a finite difference filter having
a symmetric impulse response given by
1
hd (iTc ) = (𝛿 − 𝛿i−1 ) (14.142)
2Tc i+1
Figure 14.28 illustrates the block diagram of the estimator, where the compact notation ẋ is used in
place of (dx(t)∕dt); moreover, based on the analysis of Section 14.5.2, if u(kT) is the loop filter output,
the estimate of 𝜀 is given by
𝜀̂ k+1 = 𝜀̂ k + 𝜇𝜀 u(kT) (14.143)
where 𝜇𝜀 is a suitable constant. Applying (14.88) to the value of 𝜀̂ k+1 , we obtain the values of 𝜇k+1 and
𝚖k+1 .

Data and phase directed with feedback: Mueller and Muller scheme
The present algorithm gets its name from Mueller and Muller, who first proposed it in 1976 [6]. Consider
the estimation error e𝜀 = 𝜀̂ − 𝜀 and the pulse qR (t) = qC ∗ gM (t). The basic idea consists in generating
an error signal whose mean value assumes one of the following two expressions:
{ }
1
Type A: E[e(kT)] = Re [qR (e𝜀 T + T) − qR (e𝜀 T − T)] (14.144)
2
Type B: E[e(kT)] = Re{qR (e𝜀 T + T)} (14.145)
Observe that, under the assumptions of Section 14.4, qR is a Nyquist pulse; moreover, we assume that
in the absence of channel distortion, qR is an even function.
Note that the signal (14.144) is an odd function of the estimation error e𝜀 for e𝜀 ∈ (−∞, ∞), whereas
the signal (14.145) is an odd function of e𝜀 only around e𝜀 = 0. Under lock conditions, i.e. for e𝜀 → 0, the
two versions of the algorithm exhibit a similar behaviour. However, the type A algorithm outperforms
the type B algorithm in transient conditions because the mean value of the error signal for the type B
algorithm is not symmetric. Moreover, the type A algorithm turns out to be effective also in the presence
of signal distortion.
 mkTc
x(nTc + μkTc)
to data detector
x(kT + εT)

mathched (lead of μk Tc) loop

filter interpolator differentiator filter mk
rAA(t) interpolator
hI(iTc,μk) e(kT) u(kT )
gM (iTc) hd (iTc) Re{.} F(z) decimator μk
nTc control

ak*e–jθk
x(nTc + μkTc)

Figure 14.28 DD&D𝜃-FB timing estimator. Source: Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.
14.5. Algorithms for timing and carrier phase recovery 737

The error signal for the type A algorithm is chosen equal to

̂
e(kT) = 𝜅 Re[[̂a∗k−1 xk (𝜀) ̂ e−j𝜃 ]
̂ − â ∗k xk−1 (𝜀)] (14.146)
where 𝜅 is a suitable constant whose value is discussed below. Assuming that â k−1 = ak−1 , â k = ak , and
𝜃̂ = 𝜃, from (14.71) and (14.79) for o = Ω = 0 (14.146) can be written as
{[
∑
+∞
e(kT) = 𝜅 Re a∗k−1 ai qR (kT + e𝜀 T − iT)+
i=−∞
]
∑
+∞
(14.147)
−a∗k ai qR ((k − 1)T + e𝜀 T − iT)
i=−∞

+ a∗k−1 𝑤̃ k − a∗k 𝑤̃ k−1

where 𝑤̃ k is the decimated noise signal at the matched filter output. We define
qm (e𝜀 ) = qR (mT + e𝜀 T) (14.148)
then with a suitable change of variables, (14.147) becomes
{[ ] }
∑
+∞
∑
+∞
e(kT) = 𝜅Re ∗
ak−1 ∗
ak−m qm (e𝜀 ) − ak ak−1−m qm (e𝜀 ) + ak 𝑤̃ k − ak 𝑤̃ k−1
∗ ∗
(14.149)
m=−∞ m=−∞

Taking the mean value of e(kT), we obtain

E[e(kT)] = 𝜅 Re{(E[|ak |2 ] − |𝚖a |2 )[q1 (e𝜀 ) − q−1 (e𝜀 )]}
(14.150)
= 𝜅 Re{(E[|ak |2 ] − |𝚖a |2 )[qR (e𝜀 T + T) − qR (e𝜀 T − T)]}
For
1
𝜅= (14.151)
2(E[|ak |2 ] − |𝚖a |2 )
we obtain (14.144). Similarly, in the case of the type B algorithm, the error signal assumes the expression
1 ̂
e(kT) = ̂ e−j𝜃
[̂a − 𝚖a ]∗ xk (𝜀) (14.152)
(E[|ak |2 ] − |𝚖a |2 ) k−1
Figure 14.29 illustrates the block diagram of the direct section of the type A estimator. The constant
𝜅 is included in the loop filter and is not explicitly shown.

xk(ε) ak

(.)*
e–j θ
a*k

+ e(kT)
+
– Re[.]

Figure 14.29 Mueller and Muller type A timing estimator.

738 Chapter 14. Synchronization

Non-data aided with feedback

We consider the log-likelihood (14.120), obtained for a NDA estimator, and differentiate it with respect
to e to get
𝜕 ∑
K−1
𝜕𝓁𝜀 (e)
= |x(kT + eT)|2
𝜕e 𝜕e k=0
(14.153)
∑
K−1
= 2Re[x(kT + eT) ẋ (kT + eT)]
∗

k=0

If we assume that the sum is carried out by the loop filter, the error signal is given by
e(kT) = Re[x(kT + 𝜀̂ k T) ẋ ∗ (kT + 𝜀̂ k T)] (14.154)

14.5.4 Phasor estimators

Data and timing directed

We discuss an algorithm that directly yields the phasor exp(j𝜃) ̂ in place of the phase 𝜃.
̂ Assuming that
an estimate of â and 𝜀̂ is available, the likelihood (14.95) becomes
{ [K−1 ]}
2 ∑
𝙻𝜃 (z) = exp Re â k xk (𝜀)
∗
̂ e −jz
(14.155)
N0 k=0

and is maximized by
∑
K−1
j arg â ∗k xk (𝜀)
̂
j𝜃̂
e =e k=0 (14.156)
Figure 14.30 shows the implementation of the estimator (14.156).

a*k
xk(ε) K–1
ej θ
Σ a*k xk(ε)
k=0

Figure 14.30 DD&D𝜀 estimator of the phasor ej𝜃 .

Non-data aided for M-PSK signals

In an M-PSK system, to remove the data dependence from the decimator output signal in the scheme of
Figure 14.19, we raise the samples xk (𝜀)
̂ to the M-th power. Assuming absence of ISI, we get
̂ = [ak ej𝜃 + 𝑤̃ k ]M = aM
xkM (𝜀) k e
jM𝜃
+ 𝑤M,k (14.157)
where 𝑤̃ k represents the decimator output noise, and 𝑤M,k denotes the overall disturbance.
As aMk
= (ej2𝜋l∕M )M = 1, (14.157) becomes
xkM (𝜀)
̂ = ejM𝜃 + 𝑤M,k (14.158)
From (14.95), we substitute (xk (𝜀))
̂ M for 𝛼k∗ xk (𝜀)
̂ obtaining the likelihood
{ [K−1 ]}
2 ∑
𝙻𝜃 (z) = exp Re (xk (𝜀))
̂ eM −jzM
(14.159)
N0 k=0
14.5. Algorithms for timing and carrier phase recovery 739

which is maximized by the phasor

[ ]
∑
K−1
̂
exp(j𝜃M) = exp j arg (xk (𝜀))
̂ M
(14.160)
k=0

We note that raising xk (𝜀) ̂ to the M-th power causes a phase ambiguity equal to a multiple of (2𝜋)∕M; in
fact, if 𝜃̂ is a solution to (14.160), also (𝜃̂ + 2𝜋l∕M) for l = 0, … , M − 1, are solutions. This ambiguity
can be removed, for example, by differential encoding (see Section 16.1.2). The estimator block diagram
is illustrated in Figure 14.31.

xk(ε) K–1
(.)M arg Σ (xk(ε))M e–j θM
k=0

Figure 14.31 NDA estimator of the phasor ej𝜃 for M-PSK.

Data and timing directed with feedback

Consider the likelihood (14.155) obtained for the DD&D𝜀 estimator of the phasor ej𝜃 . Taking the loga-
rithm, differentiating it with respect to z, and neglecting non-relevant terms, we obtain the error signal
̂
̂ e−j𝜃k ]
e(kT) = Im[̂a∗k xk (𝜀) (14.161)

Observe that, in the absence of noise, xk (𝜀)

̂ = ak ej𝜃 , and (14.161) becomes

e(kT) = |̂ak |2 sin(𝜃 − 𝜃̂k ) (14.162)

(a) pk ek uk pk+1 pk
F(z) z–1
–
loop filter NCO

(b) ~
xk (ε) = ak ejθ + w k ak

(.)* (.)*
a*k

1/|.|

pk pk

PHLL

Figure 14.32 (a) PHLL; (b) DD&D𝜀-FB phasor estimator. Source: Reproduced with permission
from Meyr et al. [4]. ©1998, IEEE.
740 Chapter 14. Synchronization

Hence, we can use a digital version of the PLL to implement the estimator. However, the error signal
(14.161) introduces a phase ambiguity; in fact it assumes the same value if we substitute (𝜃̂k − 𝜋) for 𝜃̂k .
An alternative to the digital PLL is given by the phasor-locked loop (PHLL), that provides an estimate
of the phasor ej𝜃 , rather than the estimate of 𝜃, thus eliminating the ambiguity.
The block diagram of the PHLL is illustrated in Figure 14.32a; it is a FB structure with the phasor
̂
pk = ej𝜃k as input and the estimate p̂ k = ej𝜃k as output. The error signal ek is obtained by subtracting the
estimate p̂ k from pk ; then ek is input to the loop filter F(z) that yields the signal uk , which is used to
update the phasor estimate according to the recursive relation
p̂ k+1 = p̂ k + (f ∗ u)(k) (14.163)
Figure 14.32b illustrates the block diagram of a DD&D𝜀 phasor estimator that implements the PHLL.
Observe that the input phasor pk is obtained by multiplying xk (𝜀)
̂ by â ∗k to remove the dependence on
the data. The dashed block normalizes the estimate p̂ k in the QAM case.

14.6 Algorithms for carrier frequency recovery

As mentioned in Section 14.4, phase and timing estimation algorithms work correctly only if the fre-
quency offset is small. Therefore, the frequency offset must be compensated before the estimate of the
other two synchronization parameters takes place. Hence, the algorithms that we will present are mainly
NDA and non-clock aided (NCA); timing-directed algorithms are possible only in the case the frequency
offset has a magnitude much smaller than 1∕T.

interpolator
frequency matched filter phase
translator filter (lead of μkTc) decimator rotator
rAA(t) rD(nTc) x(nTc,Ω)
gM(iTc) hI (iTc,μk)
nTc
e–j θ
e–jΩnTc x(nTc + μkTc,Ω) x(kT + εT,Ω)

Figure 14.33 Receiver of Figure 14.19 with interpolator and matched filter interchanged.
Source: Reproduced with permission from Meyr et al. [4]. ©1998, IEEE.

In Figure 14.33, we redraw part of the digital receiver scheme of Figure 14.19. Observe that the
position of the matched filter is interchanged with that of the interpolator filter. In this scheme, the
̂
samples rAA (nTc ) are multiplied by exp(−jΩnT c ) to remove the frequency offset.
In [4] it is shown that, whenever Ω satisfies the condition
| ΩT |
| |
| 2𝜋 | ≤ 0.15 (14.164)
| |
the following approximation holds:
x(kT + eT, o) ≃ e−jokT xk (e) (14.165)
Then, the likelihood (14.80) can be written as
{ }
2 ∑
K−1
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp ∗
Re[𝛼k xk (e) e −jokT −jz
e ] (14.166)
N0 k=0
14.6. Algorithms for carrier frequency recovery 741

Therefore, in the schemes of Figures 14.19 and 14.33, the frequency translator may be moved after the
decimator, together with the phase rotator.

14.6.1 Frequency offset estimators

Non-data aided
Suppose the receiver operates with a low SNR Γ. Similarly to (14.123) the log-likelihood for the joint
estimate of (𝜀, Ω) in the observation interval [−LT, LT] is given by
∑
L
𝓁𝜀,Ω (e, o) = |x(kT + eT, o)|2 (14.167)
k=−L

By expanding 𝓁𝜀,Ω (e, o) in Fourier series and using the notation introduced in the previous section, we
obtain ∑
𝓁𝜀,Ω (e, o) = c0 + 2Re[c1 ej2𝜋e ] + 2Re[ci ej2𝜋ie ] (14.168)
|i|≥2
⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏟
disturbance

Now the mean value of c0 , E[c0 ], depends on o, but is independent of e and furthermore is maximized
̂ Hence
for o = Ω.
̂ = arg max c
Ω (14.169)
0 o

As we did for the derivation of (14.131), starting with (14.169) and assuming the ratio F0 = T∕Tc is an
integer, we obtain the following joint estimate of (Ω, 𝜀) [4]:
∑
LF0 −1
̂ = arg max
Ω |x(nTc , o)|2
o
n=−LF0
(14.170)
∑
LF0 −1
𝜀̂ = arg |x(nTc , o)|2 e−j2𝜋n∕F0
n=−LF0

The implementation of the estimator is illustrated in Figure 14.34. Observe that the signal x(nTc , o)
can be rewritten as
∑
x(nTc , o) = rAA (iTc ) e−joiTc gM (nTc − iTc )
i
∑
= e−jonTc rAA (iTc ) e−jo(i−n)Tc gM (nTc − iTc ) (14.171)
i
∑ (pb)
= e−jonTc rAA (iTc ) gM (nTc − iTc ; o)
i

where the expression of the filter

(pb)
gM (iTc ; o) = gM (iTc ) ejoiTc (14.172)
depends on the offset o. Defining
∑ (pb)
xo (nTc ) = rAA (iTc ) gM (nTc − iTc ; o) (14.173)
i

we note that |x(nTc , o)| = |xo (nTc )|, and hence in the m-th branch of Figure 14.34, the cascade of
the frequency translator and the filter can be substituted with a simple filter with impulse response
(pb)
gM (iTc ; Ω(m) ).
742 Chapter 14. Synchronization

(1)
e–j Ω nTc

gM(iTc) |(.)|2 Σn

...
rAA(t) x(nTc,Ω)

arg max
...
nTc to timing
and/or phase
(M ) estimator
e–j Ω nTc

gM(iTc) |(.)|2 Σn

Figure 14.34 NDA frequency offset estimator. Source: Reproduced with permission from Meyr
et al. [4]. ©1998, IEEE.

Non-data aided and timing independent with feedback

Differentiating the log-likelihood defined by (14.170), equal to c0 (Ω), with respect to Ω, we obtain the
error signal
[ ]|
𝜕 ∗
e(nTc ) = 2Re x(nTc , o) x (nTc , o) || (14.174)
𝜕o |o=Ω̂ n
Observe that, as c0 (Ω) is independent of e, then also e(nTc ) is independent of e. From the first of (14.171),
the partial derivative of x(nTc , o) with respect to o is given by

𝜕 ∑
+∞
x(nTc , o) = (−jiTc ) rAA (iTc ) e−joiTc gM (nTc − iTc ) (14.175)
𝜕o i=−∞
We define the frequency matched filter as
gFM (iTc ) = (jiTc ) ⋅ gM (iTc ) (14.176)
Observe now that, if the signal rD (nTc ) = rAA (nTc ) e−jonTc is input to the filter gFM (iTc ), then from
(14.175), the output is given by
𝜕
xFM (nTc ) = gFM ∗ rD (nTc ) = x(nTc , o) + jnTc x(nTc , o) (14.177)
𝜕o
from which we obtain
𝜕
x(nTc , o) = xFM (nTc ) − jnTc x(nTc , o) (14.178)
𝜕o
Therefore, the expression of the error signal (14.174) becomes
̂ ) x∗ (nT , Ω
e(nTc ) = 2Re[x(nTc , Ω ̂ )] (14.179)
n FM c n

The block diagram of the resultant estimator is shown in Figure 14.35. The loop filter output u(kTc )
is sent to the NCO that yields the frequency offset estimate according to the recursive equation
̂ ̂
Ω n+1 Tc = Ωn Tc + 𝜇Ω u(nTc ) (14.180)
where 𝜇Ω is the NCO gain.
14.6. Algorithms for carrier frequency recovery 743

matched filter
rAA(nTc) x(nTc,Ωn)
gM(iTc)

e–j ΩnnTc frequency

matched filter
xFM(nTc,Ωn)
gFM(iTc) (.)* 2Re{.}

loop filter
u(nTc) e(nTc)
NCO F(z)

Figure 14.35 NDA-ND𝜀-FB frequency offset estimator. Source: Reproduced with permission from
Meyr et al. [4]. ©1998, IEEE.

Non-data aided and timing directed with feedback

Consider the log-likelihood (14.167). To get the D𝜀 estimator, we substitute e with the estimate 𝜀̂ obtain-
ing the log-likelihood
∑
K−1
𝓁Ω (o) = |x(kT + 𝜀T,
̂ o)|2 (14.181)
k=0
Proceeding as in the previous section, we get the block diagram illustrated in Figure 14.36.

μk mkTs
matched filter interpolator decimator
rAA(nTc) x(nTc, Ωn)
gM(iTc) hI (iTc, μk)
e–j ΩnnTc
frequency μk mkTc
matched filter interpolator decimator
xFM(nTc,Ωn)
gFM(iTc) hI (iTc, μk) (.)*

loop filter
NCO u(kT ) e(kT)
Tc F(z) 2Re{.}
T

Figure 14.36 NDA-D𝜀-FB frequency offset estimator.

14.6.2 Estimators operating at the modulation rate

As mentioned at the beginning of Section 14.6, whenever the condition (14.164) is satisfied, the fre-
quency translator can be moved after the decimator and consequently frequency offset estimation may
take place after timing estimation, thus obtaining D𝜀 algorithms. The likelihood (14.166) for an obser-
vation interval [−(K − 1)∕2, (K − 1)∕2], with K odd, evaluated at e = 𝜀̂ becomes
{ }
2 ∑
(K−1)∕2
𝙻𝜃,Ω,a (z, o, 𝜶) = exp Re[𝛼k∗ xk (𝜀)
̂ e−jokT e−jz ] (14.182)
N0 k=−(K−1)∕2
744 Chapter 14. Synchronization

Data aided and data directed

We assume a known training sequence a = 𝜶 0 is transmitted during the acquisition phase, and denote
as 𝛼0,k the k-th symbol of the sequence. The log-likelihood yields the joint estimate of (Ω, 𝜃) as

∑
(K−1)∕2
̂ 𝜃)
(Ω, ̂ = arg max ∗
Re[𝛼0,k xk (𝜀)
̂ e−jokT e−jz ] (14.183)
z,o
k=−(K−1)∕2

Note that the joint estimate is computed by finding the maximum of a function of one variable; in fact,
defining
∑
(K−1)∕2
𝚛(o) = 𝛼0,k
∗
xk (𝜀)
̂ e−jokT (14.184)
k=−(K−1)∕2

(14.183) can be rewritten as

̂ 𝜃)
(Ω, ̂ = arg max |𝚛(o)| Re[e−j(z−arg{𝚛(o)}) ] (14.185)
z,o

The maximum (14.183) is obtained by first finding the value of o that maximizes |𝚛(o)|,
̂ = arg max |𝚛(o)|
Ω (14.186)
o

and then finding the value of z for which the term within brackets in (14.185) becomes real valued,
̂
𝜃̂ = arg{𝚛(Ω)} (14.187)
We now want to solve (14.186) in close form; a necessary condition to get a maximum is that the
̂ Defining
derivative of |𝚛(o)|2 = 𝚛(o) 𝚛∗ (o) with respect to o is equal to zero for o = Ω.
[ 2 ]
K −1
bk = − k(k + 1) (14.188)
4
we get { }
∑
(K−1)∕2 𝛼0,k+1
∗
̂ = arg
ΩT bk [xk+1 (𝜀)
̂ xk∗ (𝜀)]
̂ (14.189)
k=−(K−1)∕2
𝛼0,k
∗

The DD estimator is obtained by substituting 𝜶 0 with the estimate â .

Non-data aided for M-PSK

∗
By raising (𝛼0,k+1 ∗
∕𝛼0,k )(xk+1 (𝜀)
̂ xk∗ (𝜀))
̂ to the M-th power, we obtain the NDA version of the DA (DD)
algorithm for M-PSK signals as
{ (K−1)∕2 }
∑
̂
ΩT = arg b [x (𝜀)̂ x (𝜀)]
∗
̂ M
(14.190)
k k+1 k
k=−(K−1)∕2

14.7 Second-order digital PLL

In FB systems, the recovery of the phase 𝜃 and the frequency Ω can be jointly performed by a
second-order digital phase-locked loop (DPLL), given by
̂
𝜃̂k+1 = 𝜃̂k + 𝜇𝜃 e𝜃 (kT) + Ω (14.191)
k
̂
Ω = Ω̂ + 𝜇 e (kT) + 𝜇 e (kT) (14.192)
k+1 k Ω,1 𝜃 Ω,2 Ω
14.8. Synchronization in spread-spectrum systems 745

where e𝜃 and eΩ are estimates of the phase error and of the frequency error, respectively, and 𝜇𝜃 , 𝜇Ω,1 ,
√
and 𝜇Ω,2 are suitable constants. Typically, (see Example 15.6.4 on page 783) 𝜇Ω,1 ≃ 𝜇Ω,2 ≃ 𝜇𝜃 .
Observe that (14.191) and (14.192) form a digital version of the second-order analog PLL illustrated
in Figure 14.7.

14.8 Synchronization in spread-spectrum systems

In this section, we discuss early-late FB schemes, named delay-locked loops (DLLs) [7], for the timing
recovery in spread-spectrum systems (see Chapter 10).

14.8.1 The transmission system

Transmitter
In Figure 14.37 (see Figures 10.2 and 10.3), the (baseband equivalent) scheme of a spread-spectrum
transmitter is illustrated. The symbols {ak }, generated at the modulation rate 1∕T, are input to a holder,
which outputs the symbols am at the chip rate 1∕Tchip . The chip period Tchip is given by
T
Tchip = (14.193)
NSF
where NSF denotes the spreading factor. Then, the symbols {ak } are multiplied by the spreading code
{cm } and input to a filter with impulse response hTx that includes the digital-to-analog-converter (DAC).
The (baseband equivalent) transmitted signal s is expressed as
∑
+∞
s(t) = am cm hTx (t − mTchip ) (14.194)
m=−∞
In terms of the symbols {ak }, we obtain the following alternative representation that will be used next:
∑
+∞
∑
kNSF +NSF −1
s(t) = ak cm hTx (t − mTchip ) (14.195)
k=−∞ m=kNSF

holder includes DAC

ak am s(t)
NSF hTx(t)
T Tchip

cm

Figure 14.37 Baseband transmitter for spread-spectrum systems.

Optimum receiver
Under the same assumptions of Section 14.1 and for the transmission of K symbols, the received signal
rC is expressed as
∑
K−1
∑
kNSF +NSF −1
rC (t) = ak cm qC (t − mTchip − 𝜀Tchip ) ej(Ωt+𝜃) + 𝑤C𝜑 (t) (14.196)
k=0 m=kNSF
746 Chapter 14. Synchronization

The likelihood 𝙻ss = 𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) can be computed as in (14.77); after a few steps, we obtain
{ K−1 [
2 ∑ ∑
kNSF +NSF −1
𝙻ss = exp Re 𝛼k∗ e−jz c∗m
N0 k=0 m=kNSF
]} (14.197)
𝜌(t) e −jot
gM (mTchip + eTchip − t) dt
∫TK
Defining the two signals

x(t, o) = 𝜌(𝜏) e−jo𝜏 gM (t − 𝜏) d𝜏

∫TK
(14.198)
∑
kNSF +NSF −1
y(kT, e, o) = c∗m x(mTchip + eTchip , o)
m=kNSF

the likelihood becomes

{ }
2 ∑
K−1
𝙻𝜃,𝜀,Ω,a (z, e, o, 𝜶) = exp ∗ −jz
Re[𝛼k y(kT, e, o) e ] (14.199)
N0 k=0
To obtain the samples y(kT, e, o), we can proceed as follows:
1. obtain the samples
∑
l
y(lTchip , e, o) = c∗m x(mTchip + eTchip , o) (14.200)
m=l−NSF +1

2. decimate y(lTchip , e, o) at l = (k + 1)NSF − 1, i.e. evaluate y(lTchip , e, o) for l = NSF − 1, 2NSF −

1, … , K ⋅ NSF − 1.
By (14.199), it is possible to derive the optimum digital receiver. In particular, up to the decimator that
outputs samples at instants {𝚖m Tc }, the receiver is identical to that of Figure 14.198 ; then in Figure 14.38,
only part of the receiver is shown.

m y(kT, ε, Ω) ak
x(mTchip + εTchip, Ω) Σ NSF
i = m−NSF +1
e–j θ(kT)
cm*
y(mTchip, ε, Ω)

phase
estimator

Figure 14.38 Digital receiver for spread-spectrum systems.

14.8.2 Timing estimators with feedback

Assume there is no frequency offset, i.e. Ω = 0. The likelihood is obtained by letting o = 0 in (14.199)
to yield { K−1 }
2 ∑
𝙻𝜃,𝜀,a (z, e, 𝜶) = exp Re[𝛼k∗ yk (e) e−jz ] (14.201)
N0 k=0

8
Note that now Tc ≃ Tchip ∕2, and the sampling instants at the decimator are such that mTchip + 𝜀̂ Tchip = 𝚖m Tc + 𝜇m Tc .
The estimate 𝜀̂ is updated at every symbol period T = Tchip ⋅ NSF .
14.8. Synchronization in spread-spectrum systems 747

where we use the compact notation

yk (e) = y(kT, e, 0) (14.202)
The early-late FB estimators are obtained by approximating the derivative of (14.201) with respect to e
with a finite difference, and evaluating it for e = 𝜀̂ k .

Non-data aided: non-coherent DLL

In the NDA case, the log-likelihood is obtained from (14.120). Using yk (e) instead of xk (e) yields
∑
K−1
𝓁𝜀 (e) = |yk (e)|2 (14.203)
k=0

From (14.110), the derivative of 𝓁𝜀 (e) is approximated as

1 ∑
K−1
𝜕𝓁𝜀 (e)
≃ [|y (e + 𝛿)|2 − |yk (e − 𝛿)|2 ] (14.204)
𝜕e 2𝛿 k=0 k
By including the constant 1∕(2𝛿) in the loop filter and also assuming that the sum is performed by the
loop filter, we obtain the error signal
e(kT) = |yk (𝜀̂ k + 𝛿)|2 − |yk (𝜀̂ k − 𝛿)|2 (14.205)
The block diagram of the estimator is shown in Figure 14.39. Note that the lag and the lead equal
to 𝛿Tchip are implemented by interpolator filters operating at the sampling period Tc (see (14.91))
with parameter 𝜇 equal, respectively, to −𝛿̃ and +𝛿,
̃ where 𝛿̃ = 𝛿(Tchip ∕Tc ). The estimator is called
non-coherent digital DLL [8, 9], as the dependence of the error signal on the pair (𝜃, a) is eliminated
without computing the estimates.

Non-data aided modified code tracking loop

We compute now the derivative of (14.203) in exact form,
[K−1 ]
𝜕𝓁𝜀 (e) ∑ d ∗
= 2Re yk (e) y (e) (14.206)
𝜕e k=0
de k

Approximating the derivative of y∗k (e) as in (14.110), we obtain

𝜕𝓁𝜀 (e) 1 ∑
K−1
≃ Re[yk (e)(yk (e + 𝛿) − yk (e − 𝛿))∗ ] (14.207)
𝜕e 𝛿 k=0
Assuming the loop filter performs the multiplication by 1∕𝛿 and the sum, the error signal is given by
e(kT) = Re[yk (𝜀̂ k )(yk (𝜀̂ k + 𝛿) − y(𝜀̂ k − 𝛿))∗ ] (14.208)
The block diagram of the estimator is shown in Figure 14.40 and is called modified code tracking loop
(MCTL) [10]; also in this case, the estimator is non-coherent.

Data and phase directed: coherent DLL

In case the estimates 𝜃̂ and â are given, the log-likelihood is expressed as
∑
K−1
̂
𝓁𝜀 (e) = Re[̂a∗k yk (e) e−j𝜃 ] (14.209)
k=0
 x(nTc + μm Tc) m to data detector
Σ NSF
i=m−NSF +1
mmTc

cm*

x(mTchip+(εk + δ)Tchip)

m
~ Σ NSF
hI(iTc, + δ) |.|2
i=m−NSF +1
mmTc
interpolator
cm* |yk(εk + δ)|2
+
x(nTc) hI(iTc,μm) Σ
cm* – |yk(εk + δ)|2

m
~ Σ NSF
hI(iTc, – δ) |.|2
i= m− NSF +1
mmTc

mm
x(mTchip+(εk – δ)Tchip)
loop
filter
μm interpolator εk u(kT) e(kT)
decimator NCO F(z)
control

Figure 14.39 Non-coherent DLL.

 x(nTc + μm Tc) m to data detector
Σ NSF
i=m−NSF +1
mmTc

cm* yk(εk)
m
~ Σ NSF
hI(iTc, + δ) yk(εk + δ)
mmTc i=m−NSF +1

interpolator
x(mTchip+(εk – δ)Tchip) cm*
+ e(kT)
x(nTc) hI(iTc,μm) Σ (.)* Re[.]
cm* –

m yk(εk –δ)
~ Σ NSF
hI(iTc, – δ)
mmTc i=m−NSF +1

x(mTchip+(εk – δ)Tchip)

Figure 14.40 Direct section of the non-coherent MCTL.

 x(nTc + μm Tc) m to data detector
Σ NSF
i =m−NSF + 1
mmTc

cm*

m yk(εk + δ)
~ Σ NSF
hI(iTc, + δ) Re[.]
mmTc i =m−NSF + 1

interpolator
x(mTchip+(εk – δ)Tchip) cm* ak*e–j θ
+ e(kT)
x(nTc) hI(iTc,μm) Σ
–
cm* ak*e–j θ

~
m yk(εk – δ)
hI(iTc, – δ) Σ NSF
Re[.]
mmTc i =m−NSF +1

x(mTchip+(εk – δ)Tchip)

Figure 14.41 Direct section of the coherent DLL.

14.9. Synchronization in OFDM 751

Approximating the derivative as in (14.110) and including both the multiplicative constant and the sum-
mation in the loop filter, the error signal is given by
̂
e(kT) = Re[̂a∗k e−j𝜃 [yk (𝜀̂ k + 𝛿) − yk (𝜀̂ k − 𝛿)]] (14.210)
Figure 14.41 illustrates the block diagram of the estimator, which is called coherent DLL [9–11], as
the error signal is obtained by the estimates 𝜃̂ and â .
In the three schemes of Figures 14.39–14.41, the direct section of the DLL gives estimates of 𝚖m and
𝜇m at every symbol period T, whereas the FB loop may operate at the chip period Tchip . Observe that by
removing the decimation blocks the DLL is able to provide timing estimates at every chip period.

14.9 Synchronization in OFDM

Various algorithms may be applied to achieve synchronization of multicarrier systems for transmission
over dispersive channels, depending on the system and on the type of equalization adopted.
For multicarrier systems, two synchronization processes are identified: synchronization of the clock of
the analog-to-digital converter (ADC) at the receiver front-end, or clock synchronization, and synchro-
nization of the vector rk of (8.77) at the output of the serial-to-parallel (S/P) element, or frame synchro-
nization. Clock synchronization guarantees alignment of the timing phase at the receiver with that at the
transmitter; frame synchronization, on the other hand, extracts from the sequence of received samples
blocks of  samples that form the sequence of vectors rk that are presented at the input of the DFT.
For filtered multitone (FMT) systems with non-critically sampled filter banks and fractionally-spaced
equalization, the synchronization is limited to clock recovery (see Section 7.4.3). For OFDM systems,
we provide now a more in depth description of a relevant synchronization method.

14.9.1 Frame synchronization

With regards to frame synchronization, in this first section we will discuss the effects of OFDM symbol
time offset (STO) and how it may be resolved or attenuated, in particular through the Schmidl and Cox
method [12]. Later, we will focus on the effects of carrier frequency offset (CFO) and its estimate.

Effects of STO
The effect of STO depends on the starting position of the block over which vector rk is defined and DFT
is taken. Four different cases of timing offset (TO) must be considered, as shown in Figure 14.42, where
each case is identified by a delay 𝛿 in the start of rk , with respect to the end of the cyclic prefix (CP).
For notation simplicity, we ignore the noise. Note that 𝛿 = 0 corresponds to (8.77).
● Case I: The begin of rk is before the end of the CIR, i.e. −Ncp ≤ 𝛿 ≤ −Ncp + Nc − 1. It is
⎧∑Nc −1 A [ − 1 − (m−
⎪ m=Nc −(Nc −Ncp −1−𝛿)+𝓁 k−1
⎪
⎪ (Nc − (Nc − Ncp − 1 − 𝛿) + 𝓁))]gC,m +
⎪ ∑Nc −1−(Nc −Ncp −1−𝛿)+𝓁
⎪ + m=0 Ak [(𝓁 + 𝛿 − m) mod ]gC,m
rk(𝓁) = [rk ]𝓁 = ⎨ (14.211)
⎪ 𝓁 = 0, … , (Nc − Ncp − 1 − 𝛿) − 1
⎪∑
⎪ Nc −1 A [(𝓁 + 𝛿 − m) mod ]g
⎪ m=0 k C,m
⎪ 𝓁 = (Nc − Ncp − 1 − 𝛿), … ,  − 1
⎩
hence we have interference from the previous block k − 1.
752 Chapter 14. Synchronization

Ncp  Ncp 
CP kth symbol CP (k + 1)th symbol
Nc Case III
Case II
Case I
Case IV

time

Figure 14.42 The four different cases on the location of the timing offset.

● Case II: The begin of rk is after the end of the CIR and before the end of the CP, −Ncp + Nc − 1 ≤
𝛿 ≤ −1. It is
∑
Nc −1
rk(𝓁) = Ak [(𝓁 + 𝛿 − m) mod ]gC,m , 𝓁 = 0, 1, … ,  − 1 (14.212)
m=0

thus we do not have interference from other blocks, and the delay will become an additional phase
shift of i𝛿T on the channel frequency response, i.e. In particular, we have
xk [i] = ej2𝜋i𝛿∕ C [i]ak [i], i = 0, 1, …  − 1 (14.213)
Note that for 𝛿 = −⌈(Ncp − Nc )∕2⌉, the block rk begins in the middle point between the end of the
CIR and the end of the CP. In this case, possible uncontrolled variations in synchronization still keep
the system in Case II, which we have already seen that does not introduce inter-block interference.
Therefore, this particular choice of 𝛿 is particularly robust.
● Case III: The begin of rk is at the end of the CIR, i.e. 𝛿 = 0. This has been the synchronization case
of (8.77). This case is critical, since a small synchronization error lead to Case IV, which we will
see to be affected by interference.
● Case IV: The begin of rk is after the end of the CP, i.e. 0 < 𝛿 < Nc . It is
⎧∑Nc −1
⎪ m=0 Ak [𝓁 + 𝛿 − m mod ]gC,m 𝓁 = 0, … ,  − 1 − 𝛿
⎪ ∑Nc −1
rk(𝓁) =⎨ A [(𝓁 + 𝛿 − m) mod ]gC,m
m=𝓁−(−𝛿)+1 k
(14.214)
⎪ ∑𝓁−(−𝛿)
⎪ + m=0 Ak+1 [𝓁 + 𝛿 − Ncp − m]gC,m 𝓁 =  − 𝛿, … ,  − 1
⎩
Therefore, rk is affected by interference from the next block k + 1.

Schmidl and Cox algorithm

We consider now the Schmidl and Cox algorithm for the estimation of 𝛿, i.e. for OFDM symbol syn-
chronization. It is based on the transmission of an OFDM symbol, whose time domain block Ak (see
Section 8.7.1) has the second half identical to the first half (see Figure 14.43). This can be obtained by
transmitting a pseudonoise (PN) sequence on the even subchannels and zeros on the odd subchannels of
OFDM symbol k = 0 (see {a0 [i]} in Table 14.3).
Therefore, in the absence of noise and for an ideal channel, we have from Figure 8.17
rn = rn+∕2 , n = Ncp , … , Ncp + ∕2 − 1 (14.215)
14.9. Synchronization in OFDM 753

Training symbol Data symbol

Ncp  Ncp

CP A A CP

Window 1 Window 2
Shift
/2

Figure 14.43 Example of payload using repetitive training symbol for STO estimation.

Table 14.3: Illustration of use of PN

sequences for training [72].

Subchannel i a0 [i] a1 [i] 𝑣[i]

0 7 + 7j 5 − 5j −j
1 0 −5 − 5j
2 −7 + 7j −5 − 5j j
3 0 −5 + 5j
4 7 + 7j −5 − 5j −1
5 0 5 + 5j
6 7 − 7j −5 + 5j −1
7 0 5 − 5j
8 7 + 7j 5 + 5j 1

With this in mind, two metrics can be defined:

● The correlation of r at lead n, evaluated on ∕2 samples, i.e.

∕2−1
∑
∗
P(n) = rn+i+Ncp rn+i+N (14.216)
cp +∕2
i=0

● The energy of r at lead n, evaluated on ∕2 samples (see Figure 14.43, window 2)
∕2−1
∑
E(n) = |rn+i+Ncp +∕2 |2 (14.217)
i=0

With reference to a channel introducing only a delay D0 (see Figure 8.17), we observe that P(n) reaches
the maximum value for n = D0 , since the two windows will contain the same values hence the correlation
is maximized. These two quantities can then be used to define the timing metric
|P(n)|
c(n) = (14.218)
E(n)
whose maximization yields the estimate of the STO [12] as

𝛿̂ = arg max c(𝛿) (14.219)

𝛿
754 Chapter 14. Synchronization

1.2

0.8
c(n)

0.6

0.4

0.2

0
0 50 100 150 200 250 300
n

Figure 14.44 Example of timing metric for an AWGN channel.

When studying c(n) as a function of n, (see Figure 14.44), we observe a plateau, or flat interval,
of extension given by the difference between the guard interval duration and the CIR length, i.e.
Ncp + 1 − Nc . Precisely, the nature of the channel determines the length of this plateau:

● for an ideal additive white Gaussian noise (AWGN) channel with Nc = 1, the plateau extension is
Ncp + 1, since for c(n) is maximum for n = −Ncp , … , 0; indeed, in (14.216) we find the same values
of the CP in the first half of the OFDM symbol.
● for a frequency selective channel, instead, the plateau has duration Ncp + 2 − Nc , since, in this case,
only after the end of the CIR we find the same values of the CP in the first half of the OFDM symbol.

In order to achieve the synchronization of Case II in Figure 14.42, we can first find the value of the
plateau as
cmax = max c(n) (14.220)
n

then the edges of the plateau can be found by setting a level slightly below the maximum (e.g. 𝜂 = 0.9,
i.e. 90% of the maximum) and finding 𝛿1 and 𝛿2 such that
c(𝛿1 ) = 𝜂 cmax (14.221)
= c(𝛿2 )

and c(n) < c(𝛿1 ) for n < 𝛿1 , c(n) < c(𝛿2 ) for n > 𝛿2 Then, synchronization according to Case II is
obtained by choosing the middle point of the plateau support
𝛿 1 + 𝛿2
𝛿̂ = (14.222)
2

14.9.2 Carrier frequency synchronization

The CFO Ω∕(2𝜋) = f0 − f1 is originated either by a difference between the transmit (f0 ) and receive (f1 )
oscillator frequencies or by a Doppler shift (see Section 4.1.6). In a way symmetric to STO, the CFO
yields a phase offset of the time-domain signal.
14.9. Synchronization in OFDM 755

In ideal channels with gC,0 ≠ 0, and gC,n = 0, for n ≠ 0, it results

rn = gC,0 sn ej2𝜋(f0 −f1 )nT∕ + 𝑤n (14.223)
where 𝑤n is AWGN with power 𝜎𝑤2 .
Introducing the normalized CFO to twice the inter-carrier spacing
f −f T
Δ = 0 1 = (f0 − f1 ) (14.224)
2∕T 2
from (14.216) and (14.223), we have
∕2−1 ∕2−1
∑ ∑
̂ = e−j2𝜋Δ
P(𝛿) |r𝛿+i+N
̂ |2 = e−j2𝜋(ΔI +ΔF ) |r𝛿+i+N
̂ |2 (14.225)
cp cp
i=0 i=0
where ΔI = ⌊Δ⌋ and ΔF = Δ − ΔF . An estimate of the fractional part of Δ, ΔF , can be obtained as
̂ ̂
Δ̂ F = − arg P(𝛿) (14.226)
2𝜋
On the other hand, from (14.223) the integer part of Δ, ΔI , yields a shift of data on subchannels,9 i.e.
xk [i] = ak [i − 2ΔI ] (14.227)
The Schmidl and Cox algorithm, after sending the first OFDM symbol (k = 0) for frame synchronization,
sends a second OFDM block (k = 1) that contains a different PN sequence (with respect to that of the
first symbol) on the odd frequencies, see for example Table 14.3. Let us define
√ a [2i]
𝑣[2i] = 2 1 (14.228)
a0 [2i]
for i = 0, … , ∕2 − 1. Then, we estimate the number of even shifted positions by exploiting the fact
that the phase shift is the same for each pair of frequencies in both OFDM symbols, [12], thus
|∑∕2−1 ∗ |2
| i=0 x0 [2i + 2𝜉]𝑣∗ [2i]x1 [2i + 2𝜉]|
̂ I = arg max |
Δ (∑ )2
| (14.229)
𝜉 ∕2−1
i=0
|x 1 [2i]| 2

Estimator performance
We now assess the performance of the ΔF estimator (14.226).
Following the derivations of [12], we can show that the estimator is unbiased and, assuming that
|sn | = 1, the variance of the estimate is
2𝜎𝑤2
̂ =
var(Δ) (14.230)
(2𝜋)2 |gC,0 |2
When we compare this variance with the minimum variance of an unbiased estimator provided by the
Cramér-Rao lower bound (see Example 2.4.6), we conclude that the estimate (14.226) achieves the
minimum error variance among all possible estimators.

Other synchronization solutions

Various frequency synchronization methods have been proposed for OFDM systems that include both
DA or training-assisted schemes [13–16], and NDA or blind schemes [17–20]. NDA methods use the
inherent structure of the OFDM block for synchronization. However, these algorithms usually require
9
It should be kept in mind that the cyclic shifting of the subcarrier frequencies actually holds only in the discrete-time
case, where the DFT of a finite-time signal provides a periodic signal. This does not hold in the continuous-time case,
where the subcarrier signals at margins is lost.
756 Chapter 14. Synchronization

a large number of OFDM blocks to achieve satisfactory performance. DA synchronization schemes

employ either the auto-correlation of received training blocks or its cross-correlation with a local copy
at the receiver. Methods resorting to the auto-correlation for synchronization use training symbols
with repetitions that offer robustness to large CFO and multipath channel. To facilitate synchronization
and cell search procedures in OFDM systems specified by the long-term evolution (LTE) standard,
constant-amplitude zero auto-correlation (CAZAC) sequences, for which circularly shifted copies of a
sequence are uncorrelated, have been proposed as training symbols. CAZAC sequences, also known
as Zadoff–Chu sequences, have been introduced in Appendix 1.C and are ideal candidates for uplink
synchronization because different users can transmit CAZAC sequences with different shifts, and
timing offset (TO) estimation can be performed efficiently using cross-correlation with the primary
CAZAC sequence. Joint timing and frequency synchronization using CAZAC sequences have been
proposed for downlink transmission as well [21].

14.10 Synchronization in SC-FDMA

A word of caution is needed on synchronization of single-carrier frequency division multiple access

(SC-FDMA), see Section 8.10.2: in order to avoid interblock and intersymbol interference at the receiver,
signals coming from all users must satisfy the orthogonality principle, which for SC-FDMA (as for
OFDM) corresponds to the fact that the users’ channels fall within the CP.
Consider first a single-user transmission over two CIRs, gC , and g′C that coincide apart from a delay
D0 > 0, i.e. {
′ gC,n−D0 n = D0 , … , D0 + Nc
gC,n = (14.231)
0 otherwise
Clearly, a single-user SC-FDMA receiver can use in both cases a CP of length Ncp = Nc − 1, where for
channel g′C the begin of block rk will be delayed by D0 with respect to the signal going through channel
gC . In a multiuser system instead, where a user is transmitting over gC and another over g′C , we must use
a CP of length Nc − 1 + D0 in order to avoid interference.
When signals from many users are collected at a single receiver (such as in the uplink of a cellular
system), an alternative solution provides that transmission by different users is suitably delayed in order
to have the impulse responses of the users to be overlapping at the receiver. In our example, we delay
by D0 the transmission of user going through channel gC , so that the correct block synchronization for
each signal individually is the same, and then a CP of size Ncp = Nc − 1 ensures orthogonality for both
signals. This solution goes under the name of time advance in 3GPP cellular systems.
With multiple receiving devices in different positions, thus experiencing different delays, avoiding the
use of longer CPs may be problematic.

Bibliography

[1] Meyr, H. and Ascheid, G. (1990). Synchronization in Digital Communications, vol. 1. New York,
NY: Wiley.
[2] Franks, L.E. (1980). Carrier and bit synchronization in data communication - a tutorial review.
IEEE Transactions on Communications 28: 1107–1120.
[3] Proakis, J.G. (1995). Digital Communications, 3e. New York, NY: McGraw-Hill.
[4] Meyr, H., Moeneclaey, M., and Fechtel, S.A. (1998). Digital Communication Receivers. New York,
NY: Wiley.
Bibliography 757

[5] Mengali, U. and D’Andrea, A.N. (1997). Synchronization Techniques for Digital Receivers. New
York, NY: Plenum Press.
[6] Mueller, K.H. and Muller, M.S. (1976). Timing recovery in digital synchronous data receivers.
IEEE Transactions on Communications 24: 516–531.
[7] Simon, M.K., Omura, J.K., Scholtz, R.A., and Levitt, B.K. (1994). Spread Spectrum Communica-
tions Handbook. New York, NY: McGraw-Hill.
[8] De Gaudenzi, R., Luise, M., and Viola, R. (1993). A digital chip timing recovery loop for
band-limited direct-sequence spread-spectrum signals. IEEE Transactions on Communications
41: 1760–1769.
[9] De Gaudenzi, R. (1999). Direct-sequence spread-spectrum chip tracking loop in the presence of
unresolvable multipath components. IEEE Transactions on Vehicular Technology 48: 1573–1583.
[10] Yost, R.A. and Boyd, R.W. (1982). A modified PN code tracking loop: its performance analysis and
comparative evaluation. IEEE Transactions on Communications 30: 1027–1036.
[11] De Gaudenzi, R. and Luise, M. (1991). Decision-directed coherent delay-lock tracking loop for
DS-spread spectrum signals. IEEE Transactions on Communications 39: 758–765.
[12] Schmidl, T.M. and Cox, D.C. (1997). Robust frequency and timing synchronization for OFDM.
IEEE Transactions on Communications 45: 1613–1621.
[13] Speth, M., Fechtel, S.A., Fock, G., and Meyr, H. (1999). Optimum receiver design for wireless
broad-band systems using OFDM. I. IEEE Transactions on Communications 47: 1668–1677.
[14] Minn, H., Bhargava, V.K., and Letaief, K.B. (2003). A robust timing and frequency synchronization
for OFDM systems. IEEE Transactions on Wireless Communications 2: 822–839.
[15] Ren, G., Chang, Y., Zhang, H., and Zhang, H. (2005). Synchronization method based on a new
constant envelop preamble for OFDM systems. IEEE Transactions on Broadcasting 51: 139–143.
[16] Wang, C. and Wang, H. (2009). On joint fine time adjustment and channel estimation for OFDM
systems. IEEE Transactions on Wireless Communications 8: 4940–4944.
[17] Bolcskei, H. (2001). Blind estimation of symbol timing and carrier frequency offset in wireless
OFDM systems. IEEE Transactions on Communications 49: 988–999.
[18] Luise, M., Marselli, M., and Reggiannini, R. (2002). Low-complexity blind carrier frequency
recovery for OFDM signals over frequency-selective radio channels. IEEE Transactions on Commu-
nications 50: 1182–1188.
[19] Park, B., Cheon, H., Ko, E. et al. (2004). A blind OFDM synchronization algorithm based on cyclic
correlation. IEEE Signal Processing Letters 11: 83–85.
[20] Yao, Y. and Giannakis, G.B. (2005). Blind carrier frequency offset estimation in SISO, MIMO, and
multiuser OFDM systems. IEEE Transactions on Communications 53: 173–183.
[21] Gul, M.M.U., Ma, X., and Lee, S. (2015). Timing and frequency synchronization for OFDM down-
link transmissions using Zadoff–Chu sequences. IEEE Transactions on Wireless Communications
14: 1716–1729.
Chapter 15

Self-training equalization

By the expression self-training equalization we refer to channel equalization techniques by which we

obtain the initial convergence of the parameters of an adaptive equalizer without resorting to the trans-
mission of a training sequence. Although these techniques generally achieve suboptimum performance at
convergence, they are applied in many cases of practical interest, where the transmission of known train-
ing sequences is not viable, as for example in broadcast systems, or may result in an undesirable increase
of system complexity [1–6]. In these cases, it is necessary to consider self-training of the adaptive equal-
izer, where the terms for adjusting the coefficient parameters are obtained by processing the received
signal. Usually, the receiver performs self-training of the equalizer by referring to the knowledge of the
input signal characteristics, for example the probability distribution of the input symbols.
The subject of self-training equalization in communications systems has received considerable atten-
tion since the publication of the article by Sato [7] in 1975; the proposed algorithms have been alterna-
tively called self-recovering, self-adaptive, or blind.

15.1 Problem definition and fundamentals

With reference to the discrete-time equivalent scheme of Figure 7.15 shown in Figure 15.1,
we consider a real-valued discrete-time system given by the cascade of a linear channel {hi },
in general with non-minimum phase transfer function1 H, and an equalizer C ↔ {ci }; in general,
the impulse responses {hi } and {ci } are assumed with unlimited duration. The overall system is given
∑∞
by Ψ = H ⋅ C ↔ {𝜓i }, where 𝜓i = 𝓁=−∞ c𝓁 hi−𝓁 . The channel input symbol sequence {ak } is modelled
as a sequence of i.i.d. random variables with symmetric probability density function pa (𝛼). The channel
output sequence is {xk }. In this section, additive noise introduced by the channel is ignored.
We note that a non-minimum phase rational transfer function (see Definition 1.5 on page 12) can be
expressed as
P (z) P2 (z)
H(z) = H0 1 (15.1)
P3 (z)
where H0 denotes the gain, P3 (z) and P1 (z) are monic polynomials with zeros inside the unit circle,
and P2 (z) is a monic polynomial with zeros outside the unit circle. We introduce the inverse func-
tions with respect to the polynomials P1 (z) and P2 (z) given by P−1
1
(z) = 1∕P1 (z) and P−1
2
(z) = 1∕P2 (z),

1
In this chapter, to simplify notation, often the argument of the z-transform is not explicitly indicated.

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
760 Chapter 15. Self-training equalization

discrete-time data
equivalent channel equalizer detector
ak xk yk a^k
h c

Figure 15.1 Discrete-time equivalent channel and equalizer filter.

respectively. By expanding P−1

1
(z) and P−1
2
(z) in Laurent series we obtain, apart from lag factors,
∑
+∞
P−1
1
(z) = c1,n z−n
n=0
(15.2)
∑ 0
−n
P−1
2
(z) = c2,n z
n=−∞

both converging in a ring that includes the unit circle. Therefore, we have
( +∞ )( 0 )
1 ∑ ∑
−1 −n −n
H (z) = P (z) c1,n z c2,n z (15.3)
H0 3 n=0 n=−∞

In general, from (15.3) we note that, as the system is non-minimum phase, the inverse system cannot be
described by a finite number of parameters; in other words, the reconstruction of a transmitted symbol
at the equalizer output at a certain instant requires the knowledge of the entire received sequence.
In practice, to obtain an implementable system, the series in (15.3) are truncated to N terms, and an
approximation of H −1 with a lag equal to (N1 − 1) modulation intervals is given by
(N−1 )( 0 )
1 −(N1 −1) ∑ ∑
−1 −n −n
H (z) ≃ z P3 (z) c1,n z c2,n z (15.4)
H0 n=0 n=−(N−1)

Therefore, the inverse system can only be defined apart from a lag factor.
The problem of self-training equalization is formulated as follows: from the knowledge of the prob-
ability distribution of the channel input symbols {ak } and from the observation of the channel out-
put sequence {xk }, we want to find an equalizer C such that the overall system impulse response is
inter-symbol interference (ISI) free.
Observe that, if the channel H is minimum phase,2 both H and H −1 are causal and stable; in this
case, the problem of channel identification, and hence the determination of the sequence {ak }, can be
solved by whitening (see page 117) the observation {xk } using known procedures that are based on the
second order statistical description of signals. If, as it happens in the general case, the channel H is
non-minimum phase, by the second order statistical description (1.187), it is possible to identify only
the amplitude characteristic of the channel transfer function, but not its phase characteristic.
In particular, if the probability density function of the input symbols is Gaussian, the output {xk } is
also Gaussian and the process is completely described by a second order analysis; therefore, the above
observations are valid and in general the problem of self-training equalization cannot be solved for Gaus-
sian processes. Note that we have referred to a system model in which the sampling frequency is equal
to the symbol rate; a solution to the problem using a second order description with reference to an
oversampled model is obtained in [8].
Furthermore we observe that, as the probability density function of the input symbols is symmetric,
the sequence {−ak } has the same statistical description as the sequence {ak }; consequently, it is not

2
We recall that all auto-regressive (AR) models (1.445) are minimum phase.
15.1. Problem definition and fundamentals 761

possible to distinguish the desired equalizer H −1 from the equalizer −H −1 . Hence, the inverse system
can only be determined apart from the sign and a lag factor. Therefore, the solution to the problem
of self-training equalization is given by C = ±H −1 , which yields the overall system Ψ = ±I, where I
denotes the identity, with the exception of a possible lag.
In this chapter, we will refer to the following theorem, demonstrated in [9].

Theorem 15.1
Assuming that the probability density function of the input symbols is non-Gaussian, then Ψ = ±I if
the output sample
∑
+∞
yk = cn xk−n (15.5)
n=−∞

has a probability density function pyk (b) equal to the probability density function of the symbols
{ak }. ◽

Therefore, using this theorem to obtain the solution, it is necessary to determine an algorithm for the
adaptation of the coefficients of the equalizer C such that the probability distribution of yk converges
to the distribution of ak .
We introduce the cost function
J = E[Φ(yk )] (15.6)

where yk is given by (15.5) and Φ is an even, real-valued function that must be chosen so that the optimum
solution, determined by
Copt (z) = arg min J (15.7)
C(z)

is found at the points ±H −1 , apart from a lag factor.

In most applications, minimization is done by equalizers of finite length N having coefficients at instant
k given by
ck = [c0,k , c1,k , … , cN−1,k ]T (15.8)

Let
xk = [xk , xk−1 , … , xk−(N−1) ]T (15.9)

then (15.5) becomes

∑
N−1
yk = cn,k xk−n = cTk xk (15.10)
n=0

Then also (15.7) simplifies into

copt = arg min J (15.11)
c

If Θ is the derivative of Φ, the gradient of J with respect to c is given by (see (2.18))

∇c J = E[xk Θ(yk )] (15.12)

For the minimization of J, we use a stochastic gradient algorithm for which (see (3.40))

ck+1 = ck − 𝜇 Θ(yk ) xk (15.13)

where 𝜇 is the adaptation gain. Note that the convergence of C to H −1 or to −H −1 depends on the initial
choice of the coefficients.
Before tackling the problem of choosing the function Φ, we consider the following problem.
762 Chapter 15. Self-training equalization

Minimization of a special function

We consider an overall system 𝝍 with impulse response having unlimited duration. We want to determine
the values of the sequence {𝜓i } that minimize the following cost function:
∑
+∞
= |𝜓i | (15.14)
i=−∞

subject to the constraint

∑
+∞
𝜓i2 = 1 (15.15)
i=−∞

The function  characterizes the peak amplitude of the system output signal {yk } and the constraint can
be interpreted as a requirement that the solution {𝜓i } belong to the sphere with centre the origin and
radius r = 1 in the parameter space {𝜓i }.
Letting 𝝍 = [… , 𝜓0 , 𝜓1 , …]T , it results ∇𝝍  = [… , sgn(𝜓0 ), sgn(𝜓1 ), …]T ; then, if 𝜓max is the max-
imum value assumed by 𝜓i , the cost function (15.14) presents stationary points for 𝜓i = ±𝜓max . Now,
taking into account the constraint (15.15), it is easily seen that the minimum of (15.14) is reached only
when one element of the sequence {𝜓i } is different from zero (with a value equal to 𝜓max ) and the others
are all zeros. In other words, the only points of minimum are given by Ψ = ±I, and the other stationary
points correspond to saddle points. Figure 15.2 illustrates the cost function  along the unit circle for a
system with two parameters.

1.5

0.5

+I
+I
0
1
0.5 −I 1
−I
0 0.5
ψ1 −0.5 0
−0.5 ψ0
−1 −1

Figure 15.2 Illustration of the cost function  for the system Ψ ↔ {𝜓0 , 𝜓1 } and of the gradient
−∇ projected onto the straight line tangent to the unit circle.

The minimization can also be obtained recursively, updating the parameters {𝜓i } as indicated by the
direction of steepest descent3 of the cost function .

3
The direction is defined by the gradient vector.
15.1. Problem definition and fundamentals 763

The projection of the gradient  onto the plane tangent to the unit sphere at the point 𝝍 is given by
∇𝝍  − [(∇𝝍 )T 𝝍]𝝍 (15.16)
then the recursive equation yields
[ ]
∑
+∞
𝜓i,k+1
′
= 𝜓i,k − 𝜇 sgn(𝜓i,k ) − 𝜓i,k |𝜓𝓁,k | (15.17)
𝓁=−∞
𝜓i,k+1
′
𝜓i,k+1 = √ (15.18)
∑+∞ ′ 2
𝓁=−∞ (𝜓𝓁,k+1 )
∑
Note that, if the term 𝜓i,k 𝓁 |𝜓𝓁,k | is omitted in (15.17), with good approximation the direction of the
steepest descent is still followed, provided that the adaptation gain 𝜇 is sufficiently small.
From (15.17), for each parameter 𝜓i the updating consists of a correction towards zero of a fixed value
and a correction in the opposite direction of a value proportional to the parameter amplitude. Assuming
that the initial point is not a saddle point, by repeated iterations of the algorithm, one of the parameters
𝜓i approaches the value one, while all others converge to zero, as shown in Figure 15.3.

Reduce by ±∆
−∆ ψi,0

0 1 2 3 4 5 6
+∆
Rescale by ε ψ'i,1
(1+ε)ψʹi,1 = ψ i,1

ψ ʹi,1

0 1 2 3 4 5 6

ψi,1 Reduce by ±∆

0 1 2 3 4 5 6
Rescale by ε ψ'i,2

0 1 2 3 4 5 6
ψi,k

0 1 2 3 4 5 6

Figure 15.3 Update of the parameters {𝜓i }, i = 0, … , 6.

764 Chapter 15. Self-training equalization

We now want to obtain the same adaptation rule for the parameters {𝜓i } using the output signal yk .
We define 𝝍 k as the vector of the parameters of 𝝍 at instant k,
𝝍 k = [… , 𝜓0,k , 𝜓1,k , …]T (15.19)

and
ak = [… , ak , ak−1 , …]T (15.20)

Therefore,
yk = 𝝍 Tk ak (15.21)

Assume that at the beginning of the adaptation process the overall system Ψ satisfies the condition
∥ 𝝍∥2 = 1, but it deviates significantly from the system identity; then the equalizer output signal yk will
occasionally assume positive or negative values which are much larger in magnitude than 𝛼max = max ak .
∑
The peak value of yk is given by ±𝛼max i |𝜓i,k |, obtained with symbols {ak−i } equal to ±𝛼max , and
indicates that the distortion is too large and must be reduced. In this case, a correction of a fixed value
towards zero is obtained using the error signal

ek = yk − 𝛼max sgn(yk ) (15.22)

and applying the stochastic gradient algorithm

𝝍 k+1 = 𝝍 k − 𝜇 ek ak (15.23)

If the coefficients are scaled so that the condition ∥ 𝝍 k ∥2 = 1 is satisfied at every k, we obtain a coeffi-
cient updating algorithm that approximates algorithm (15.17)–(15.18).
Obviously, algorithm (15.23) cannot be directly applied, as the parameters of the overall system Ψ
are not available. However, observe that if the linear transformation H is non-singular, then formally
C = H −1 Ψ. Therefore, the overall minima of  at the points Ψ = ±I are mapped into overall minima
at the points C = ±H −1 of a cost function J that is the image of  under the transformation given by
H −1 , as illustrated in Figure 15.4. Furthermore, it is seen that the direction of steepest descent of  has
not been modified by this transformation. Thus, the updating terms for the equalizer coefficients are still
given by (15.22)–(15.23), if symbols ak are replaced by the channel output samples xk .

Global minima of  (ψ) Global minima of J(C)

H−1

ψ C

Figure 15.4 Illustration of the transformation C = H −1 Ψ.

Then, a general algorithm that converges to the desired solution C = H −1 can be formulated as follows:

● observe the equalizer output signal {yk } and determine its peak value;
15.2. Three algorithms for PAM systems 765

● whenever a peak value occurs, update the coefficients according to the algorithm
ek = yk − 𝛼max sgn(yk )
(15.24)
ck+1 = ck − 𝜇 ek xk
● scale the coefficients so that the statistical power of the equalizer output samples is equal to the
statistical power of the channel input symbols.

We observe that it is not practical to implement an algorithm that requires computing the peak value
of the equalizer output signal and updating the coefficients only when a peak value is observed. In the
next sections, we describe algorithms that allow the updating of the coefficients at every modulation
interval, thus avoiding the need of computing the peak value of the signal at the equalizer output and of
scaling the coefficients.

15.2 Three algorithms for PAM systems

The Sato algorithm

The Sato cost function is defined as
[ ]
1 2
J=E y − 𝛾S |yk | (15.25)
2 k
where
E[a2k ]
𝛾S = (15.26)
E[|ak |]
The gradient of J is given by
∇c J = E[xk (yk − 𝛾S sgn(yk ))] (15.27)

We introduce the signal

𝜖S,k = yk − 𝛾S sgn(yk ) (15.28)

which assumes the meaning of pseudo error that during self-training replaces the error used in the
least-mean-square (LMS) decision-directed algorithm. We recall that for the LMS algorithm the error
signal is4
ek = yk − â k (15.29)

where âk is the detection of the symbol ak , obtained by a threshold detector from the sample yk .
Figure 15.5 shows the pseudo error 𝜖S,k as a function of the value of the equalizer output sample yk .
Therefore, the Sato algorithm for the coefficient updating of an adaptive equalizer assumes the expres-
sion
ck+1 = ck − 𝜇 𝜖S,k xk (15.30)

It was proven [9] that, if the probability density function of the output symbols {ak } is sub-Gaussian,5
then the Sato cost function (15.25) admits as unique points of minimum the systems C = ±H −1 , apart
from a possible lag; however, note that the uniqueness of the points of minimum of the Sato cost function
is obtained by assuming a continuous probability distribution of input symbols. In the case of a discrete

4
Note that in this chapter, the error signal is defined with opposite sign with respect to the previous chapters.
5
A probability density function pak (𝛼) is sub-Gaussian if it is uniform or if pak (𝛼) = K exp{−g(𝛼)}, where g(𝛼) is an even
1 dg
function such that both g(𝛼) and 𝛼 d𝛼
are strictly increasing in the domain [0, +∞).
766 Chapter 15. Self-training equalization

εS,k

–γS γS yk

Figure 15.5 Characteristic of the pseudo error 𝜖S,k as a function of the equalizer output.

probability distribution with an alphabet  = {±1, ±3, … , ±(M − 1)}, the convergence properties of
the Sato algorithm are not always satisfactory.
Another undesirable characteristic of the Sato algorithm is that the pseudo error 𝜖S,k is not equal to
zero for C = ±H −1 , unless we consider binary transmission. In fact only the gradient of the cost function
given by (15.27) is equal to zero for C = ±H −1 ; moreover, we find that the variance of the pseudo error
may assume non-negligible values in the neighbourhood of the desired solution.

Benveniste–Goursat algorithm
To mitigate the above mentioned inconvenience, we observe that the error ek used in the LMS algorithm
in the absence of noise becomes zero for C = ±H −1 . It is possible to combine the two error signals
obtaining the pseudo error proposed by Benveniste and Goursat [10], given by

𝜖G,k = 𝜅1 ek + 𝜅2 |ek | 𝜖S,k (15.31)

where 𝜅1 and 𝜅2 are constants. If the distortion level is high, |ek | assumes high values and the second
term of (15.31) allows convergence of the algorithm during the self-training. Near convergence, for
C ≃ ±H −1 , the second term has the same order of magnitude as the first and the pseudo error assumes
small values in the neighbourhood of C = ±H −1 .
Note that an algorithm that uses the pseudo error (15.31) allows a smooth transition of the equalizer
from the self-training mode to the decision-directed mode. In the case of a sudden change in channel
characteristics, the equalizer is found working again in self-training mode. Thus, the transitions between
the two modes occur without control on the level of distortion of the signal {yk } at the equalizer output.

Stop-and-go algorithm
The stop-and-go algorithm [11] can be seen as a variant of the Sato algorithm that achieves the same
objectives of the Benveniste–Goursat algorithm with better convergence properties. The pseudo error
for the stop-and-go algorithm is formulated as
{
ek if sgn(ek ) = sgn(𝜖S,k )
𝜖P,k = (15.32)
0 otherwise
where 𝜖S,k is the Sato pseudo error given by (15.28), and ek is the error used in the decision-directed
algorithm given by (15.29). The basic idea is that the algorithm converges if updating of the equalizer
15.3. The contour algorithm for PAM systems 767

coefficients is turned off with sufficiently high probability every time the sign of error (15.29) differs
from the sign of error eid,k = yk − ak , that is sgn(ek ) ≠ sgn(eid,k ). As eid,k is not available in a self-training
equalizer, with the stop-and-go algorithm coefficient updating is turned off whenever the sign of error ek
is different from the sign of Sato error 𝜖S,k . Obviously, in this way we also get a non-zero probability that
coefficient updating is inactive when the condition sgn(ek ) = sgn(eid,k ) occurs, but this does not usually
bias the convergence of the algorithm.

Remarks
At this point, we can make the following observations.

● Self-training algorithms based on the minimization of a cost function that includes the term E[|yk |p ],
p ≥ 2, can be explained referring to the algorithm (15.24), because the effect of raising to the p-th
power the amplitude of the equalizer output sample is that of emphasizing the contribution of sam-
ples with large amplitude.
● Extension of the Sato cost function (15.25) to quadrature amplitude modulation (QAM) systems,
which we discuss in Section 15.5, is given by
[ ]
1
J=E |yk |2 − 𝛾S |yk | (15.33)
2
where 𝛾S = E[|ak |2 ]∕E[|ak |]. In general this term guarantees that, at convergence, the statistical
power of the equalizer output samples is equal to the statistical power of the input symbols.
● In the algorithm (15.24), the equalizer coefficients are updated only when we observe a peak value
of the equalizer output signal. As the peak value decreases with the progress of the equalization
process, updating of the equalizer coefficients ideally depends on a threshold that varies depending
on the level of distortion in the overall system impulse response.

15.3 The contour algorithm for PAM systems

The algorithm (15.24) suggests that the equalizer coefficients are updated when the equalizer output
sample reaches a threshold value, which in turn depends on the level of distortion in the overall system
impulse response. In practice, we define a threshold at instant k as Tk = 𝛼max + 𝛾CA,k , where the term
𝛾CA,k ≥ 0 represents a suitable measure of distortion. Each time the absolute value of the equalizer
output sample reaches or exceeds the threshold Tk , the coefficients are updated so that the peak value of
the equalizer output is driven towards the constellation boundary ±𝛼max ; at convergence of the equalizer
coefficients, 𝛾CA,k vanishes. Figure 15.6 illustrates the evolution of the contour Tk for a two dimensional
constellation.
The updating of coefficients described above can be obtained by a stochastic gradient algorithm that
is based on a cost function E[Φ(yk )] defined on the parameter space {𝜓i }. We assume that the overall
system Ψ initially corresponds to a point on a sphere of arbitrary radius r. With updating terms that on
the average exhibit the same sign as the terms found in the general algorithm (15.24), the point on the
sphere of radius r moves in such a way to reduce distortion. Moreover, if the radial component of the
gradient, i.e. the component that is orthogonal to the surface of the sphere, is positive for r > 1, negative
for r < 1 and vanishes on the sphere of radius r = 1, it is not necessary to scale the coefficients and the
convergence will take place to the point of global minimum Ψ = ±I. Clearly, the derivative function
of Φ(yk ) with respect to yk that defines the pseudo error can be determined in various ways. A suitable
768 Chapter 15. Self-training equalization

Im[yk]
T0

Tk
αmax

αmax Re[yk]

Figure 15.6 Evolution in time of the contour Tk for a two-dimensional constellation.

definition is given by
{
yk − [𝛼max + 𝛾CA,k ]sgn(yk ) |yk | ≥ 𝛼max
Θ(yk ) = (15.34)
−𝛾CA,k sgn(yk ) otherwise
For a self-training equalizer, the updating of coefficients as indicated by (15.8) is obtained by
ck+1 = ck − 𝜇 Θ(yk ) xk (15.35)
In (15.35), to avoid the computation of the threshold Tk at each iteration, the amplitude of the equalizer
output signal is compared with 𝛼max rather than with Tk ; note that the computation of Θ(yk ) depends on
the event that yk falls inside or outside the constellation boundary. In the next section, for a two dimen-
sional constellation, we define in general the constellation boundary as the contour line that connects the
outer points of the constellation; for this reason, we refer to this algorithm as the contour algorithm [12].
To derive the algorithm (15.34)–(15.35) from the algorithm (15.24), several approximations are intro-
duced; consequently, the convergence properties cannot be directly derived from those of the algorithm
(15.24). In the Appendix 15.A, we show how 𝛾CA should be defined to obtain the desired behaviour of
the algorithm (15.34)–(15.35) in the case of systems with input symbols having a uniform continuous
distribution.
An advantage of the functional introduced in this section with respect to the Sato cost function is that
the variance of the pseudo error vanishes at the points of minimum Ψ = ±I; this means that it is possi-
ble to obtain the convergence of the mean square error (MSE) to a steady state value that is close to the
achievable minimum value. Furthermore, the radial component of the gradient of E[Φ(yk )] vanishes at
15.3. The contour algorithm for PAM systems 769

every point on the unit sphere, whereas the radial component of the gradient in the Sato cost function
vanishes on the unit sphere only at the points Ψ = ±I. As the direction of steepest descent does not inter-
sect the unit sphere, the contour algorithm avoids overshooting of the convergence trajectories observed
using the Sato algorithm; in other words, the stochastic gradient yields a coefficient updating that is
made in the correct direction more often than in the case of the Sato algorithm. Therefore, substantially
better convergence properties are expected for the contour algorithm even in systems with a discrete
probability distribution of input symbols.
The complexity of the algorithm (15.34)–(15.35) can be deemed prohibitive for practical implemen-
tations, especially for self-training equalization in high-speed communication systems, as the parameter
𝛾CA,k must be estimated at each iteration. In the next section, we discuss a simplified algorithm that allows
implementation with low complexity; we will see later how the simplified formulation of the contour
algorithm can be extended to self-training equalization of partial response (PR) and QAM systems.

Simplified realization of the contour algorithm

We assume that the input symbols {ak } form a sequence of i.i.d. random variables with a uniform discrete
probability density function. From the scheme of Figure 15.1, in the presence of noise, the channel output
signal is given by
∑
∞
xk = hi ak−i + 𝑤k (15.36)
i=−∞

where {𝑤k } denotes additive white Gaussian noise. The equalizer output is given by yk = cTk xk . To obtain
an algorithm that does not require the knowledge of the parameter 𝛾CA , the definition (15.34) suggests
introducing the pseudo error
{
yk − 𝛼max sgn(yk ) if |yk | ≥ 𝛼max
𝜖CA,k = (15.37)
−𝛿k sgn(yk ) otherwise
where 𝛿k is a non-negative parameter that is updated at every iteration as follows:

⎧ M−1
⎪ 𝛿k − M Δ if |yk | ≥ 𝛼max
𝛿k+1 =⎨ 1 (15.38)
⎪ 𝛿k + Δ otherwise
⎩ M
and Δ is a positive constant. The initial value 𝛿0 is not a critical system parameter and can be, for example,
chosen equal to zero; the coefficient updating algorithm is thus given by

ck+1 = ck − 𝜇 𝜖CA,k xk (15.39)

In comparison to (15.34), now 𝛿k does not provide a measure of distortion as 𝛾CA . The definition
(15.37) is justified by the fact that the term yk − [𝛼max + 𝛾CA ]sgn(yk ) in (15.34) can be approximated as
yk − 𝛼max sgn(yk ), because if the event |yk | ≥ 𝛼max occurs the pseudo error yk − 𝛼max sgn(yk ) can be used
for coefficient updating. Therefore, 𝛿k should increase in the presence of distortion only in the case the
event |yk | < 𝛼max occurs more frequently than expected. This behaviour of the parameter 𝛿k is obtained
by applying (15.38). Moreover, (15.38) guarantees that 𝛿k assumes values that approach zero at the
convergence of the equalization process; in fact, in this case the probabilities of the events {|yk | < 𝛼max }
and {|yk | ≥ 𝛼max } assume approximately the values (M − 1)∕M and 1∕M, respectively, that correspond
to the probabilities of such events for a noisy PAM signal correctly equalized. Figure 15.7 shows the
pseudo error 𝜖CA,k as a function of the value of the equalizer output sample yk .
770 Chapter 15. Self-training equalization

εCA,k

+δk
+αmax
–αmax yk
–δk

Figure 15.7 Characteristic of the pseudo error 𝜖CA,k as a function of the equalizer output.

The contour algorithm has been described for the case of uniformly distributed input symbols; how-
ever, this assumption is not necessary. In general, if {yk } represents an equalized signal, the terms
(M − 1)∕M and 1∕M in (15.38) are, respectively, substituted with
1
p0 = P[|ak | < 𝛼max ] + P[|ak | = 𝛼max ] ≃ P[|yk | < 𝛼max ] (15.40)
2
and
1
p1 = P[|ak | = 𝛼max ] ≃ P[|yk | ≥ 𝛼max ] (15.41)
2
We note that the considered receiver makes use of signal samples at the symbol rate; the algorithm
can also be applied for the initial convergence of a fractionally spaced equalizer (see Section 7.4.3) in
case a sampling rate higher than the symbol rate is adopted.

15.4 Self-training equalization for partial response systems

Self-training adaptive equalization has been mainly studied for full response systems; however,
self-training equalization methods for partial response systems have been proposed for linear and
non-linear equalizers [7, 13, 14]. In general, a self-training equalizer is more difficult to implement for
partial response systems (see Appendix 7.C), especially if the input symbol alphabet has more than
two elements. Moreover, as self-training is a slow process, the accuracy achieved in the recovery of the
timing of the received signal before equalization plays an important role. In fact, if timing recovery is
not accomplished, because of the difference between the sampling rate and the modulation rate we have
that the sampling phase varies with respect to the timing phase of the remote transmitter clock; in this
case, we speak of phase drift of the received signal; self-training algorithms fail if the phase drift of
the received signal is not sufficiently small. In this section, we discuss the extension to partial response
systems of the algorithms for PAM systems presented in the previous section.

The Sato algorithm

We consider a multilevel partial response class IV (PR-IV) system, also called modified duobinary
system. Using the D transform, the desired transfer function of the overall system is given by
15.4. Self-training equalization for partial response systems 771

𝜓(D) = (1 − D2 ). The objective of an adaptive equalizer for a PR-IV system consists in obtaining an
equalized signal of the form
yk = (ak − ak−2 ) + 𝑤y,k = uk + 𝑤y,k (15.42)
where 𝑤y,k is a disturbance due to noise and residual distortion. We consider the case of quaternary
modulation. Then, the input symbols ak are from the set {−3, −1, +1, +3}, and the output signal
uk = ak − ak−1 , for an ideal channel in the absence of noise, can assume one of the seven values
{−6, −4, −2, 0, +2, +4, +6}.6

xk xk–N+1
T T T

c0,k cN–1,k

coefficient
adaptation

μ
yk
εS,k
+
+ –

+ vS,k 𝛾S +
+ +
+ –𝛾S –
1 βS
vS(D) = y(D)
1–βSD2
2T 2T

Figure 15.8 Block diagram of a self-training equalizer for a PR-IV system using the Sato algorithm.

As illustrated in Figure 15.8, to obtain a pseudo error to be employed in the equalizer coefficient
updating algorithm, the equalizer output signal {yk } is transformed into a full-response signal 𝑣S,k by
the linear transformation
𝑣S,k = yk + 𝛽S 𝑣S,k−2 (15.43)
where 𝛽S is a constant that satisfies the condition 0 < 𝛽S < 1. Then, the signal 𝑣S,k is quantized by a
quantizer with two levels corresponding to ±𝛾S , where 𝛾S is given by (15.26). The obtained signal is
again transformed into a partial response signal that is subtracted from the equalizer output to generate
the pseudo error
𝜖S,k = yk − 𝛾S [sgn(𝑣S,k ) − sgn(𝑣S,k−2 )] (15.44)
Then, the Sato algorithm for partial response systems is expressed as
ck+1 = ck − 𝜇 𝜖S,k xk (15.45)
6
In general, for an ideal PR-IV channel in the absence of noise, if the alphabet of the input symbols is
 = {±1, ±3, … , ±(M − 1)}, the output symbols assume one of the (2M − 1) values {0, ±2, … , ±2(M − 1)}.
772 Chapter 15. Self-training equalization

The contour algorithm

In this case, first the equalizer output is transformed into a full response signal and a pseudo error is
computed. Then, the error to compute the terms for coefficient updating is formed.
The method differs in two ways from the Sato algorithm described above. First, the channel equal-
ization, carried out to obtain the full response signals, is obtained by combining linear and non-linear
feedback, whereas in the case of the Sato algorithm, it is carried out by a linear filter. Second, the knowl-
edge of the statistical properties of the input symbols is used to determine the pseudo error, as suggested
by the contour algorithm.
As mentioned above, to apply the contour algorithm the equalizer output signal is transformed into a
full response signal using a combination of linear feedback and decision feedback, that is we form the
signal
𝑣k = yk + 𝛽CA 𝑣k−2 + (1 − 𝛽CA ) âk−2 (15.46)

Equation (15.46) and the choice of the parameter 𝛽CA are justified in the following way. If 𝛽CA = 0
is selected, we obtain an equalization system with decision feedback, that presents the possibility of
significant error propagation. The effect of the choice 𝛽CA = 1 is easily seen using the D transform.
From (15.42) and assuming correct decisions, (15.46) can be expressed as
𝑤y (D)
𝑣(D) = a(D) + (15.47)
1 − 𝛽CA D2
Therefore with 𝛽CA = 1, we get the linear inversion of the PR-IV channel with infinite noise enhancement
at frequencies f = 0 and ±1∕(2T) Hz. The value of 𝛽CA is chosen in the interval 0 < 𝛽CA < 1, to obtain
the best trade-off between linear feedback and decision feedback.
We now apply the contour algorithm (15.37)–(15.39) using the signal 𝑣k rather than the equalizer
output signal yk . The pseudo error is defined as
{
𝑣 𝑣k − 𝛼max sgn(𝑣k ) if |𝑣k | ≥ 𝛼max
𝜖CA,k = (15.48)
−𝛿k𝑣 sgn(𝑣k ) otherwise
where 𝛿k𝑣 is a non-negative parameter that is updated at each iteration as

⎧ 𝑣 M−1
𝑣 ⎪ 𝛿k − M Δ if |𝑣k | ≥ 𝛼max
𝛿k+1 =⎨ 1 (15.49)
⎪ 𝛿k𝑣 + Δ otherwise
⎩ M
The stochastic gradient must be derived taking into consideration the channel equalization performed
with linear feedback and decision feedback. We define the error on the M-ary symbol ak after channel
equalization as
e𝑣k = 𝑣k − ak (15.50)

Assuming correct decisions, by (15.46) it is possible to express the equalizer output as

yk = (ak − ak−2 ) + e𝑣k − 𝛽CA e𝑣k−2 (15.51)

Equation (15.51) shows that an estimate of the term e𝑣k − 𝛽CA e𝑣k−2 must be included as error signal in the
expression of the stochastic gradient. After initial convergence of the equalizer coefficients, the estimate
ê𝑣k = 𝑣k − âk is reliable. Therefore, decision directed coefficient updating can be performed according to
the algorithm
ck+1 = ck − 𝜇dd (ê𝑣k − 𝛽CA ê𝑣k−2 ) xk (15.52)
15.5. Self-training equalization for QAM systems 773

xk xk–N+1
T T T

c0,k cN–1,k mode selection

– self training
– decision directed
coefficient adaptation

yk
μdd

+ e^ kv
Σ
–
+
+
εCA,k –
+ vk
+ a^k
+ 2T
μ
βCA 1–βCA βCA
Control
+ +
+ of δvk

2T

Figure 15.9 Block diagram of a self-training equalizer with the contour algorithm for a QPR-IV
system.

The contour algorithm for coefficient updating during self-training is obtained by substituting the deci-
sion directed error signal e𝑣k with the pseudo error 𝜖CA,k
𝑣
(15.48),
𝑣 𝑣
ck+1 = ck − 𝜇(𝜖CA,k − 𝛽CA 𝜖k−2 ) xk (15.53)
During self-training, satisfactory convergence behaviour is usually obtained for 𝛽CA ≃ 1∕2.
Figure 15.9 shows the block diagram of an equalizer for a PR-IV system with a quaternary alphabet
(QPR-IV), with the generation of the error signals to be used in decision directed and self-training mode.
In the described scheme, the samples of the received signal can be initially filtered by a filter with
transfer function 1∕(1 − aD2 ), 0 < a < 1, to reduce the correlation among samples. The obtained signal
is then input to the equalizer delay line.

15.5 Self-training equalization for QAM systems

We now describe various self-training algorithms for passband transmission systems that employ a
two-dimensional constellation.

The Sato algorithm

Consider a QAM transmission system with constellation  and a sequence {ak } of i.i.d. symbols from 
such that ak,I = Re[ak ] and ak,Q = Im[ak ] are independent and have the same probability distribution. We
774 Chapter 15. Self-training equalization

data
equalizer detector
xk y~k yk a^k
c

^
e–jφk

carrier
coefficient
phase
adaptation
tracking

Figure 15.10 Block diagram of a self-training equalizer with the Sato algorithm for a QAM system.

assume a receiver in which sampling of the received signal occurs at the symbol rate and tracking of the
carrier phase is carried out at the equalizer output, as shown in Figure 15.10 (see scheme of Figure 7.41
with a baseband equalizer filter where {xk } is already demodulated and 𝜃k = 𝜑̂ k ). If ỹ k = cT xk is the
equalizer filter output, the sample at the decision point is then given by
yk = ỹ k e−j𝜑̂ k (15.54)
We let
yk,I = Re[yk ] and yk,Q = Im[yk ] (15.55)
and introduce the Sato cost function for QAM systems,
J = E[Φ(yk,I ) + Φ(yk,Q )] (15.56)
where
1 2
Φ(𝑣) = 𝑣 − 𝛾S |𝑣| (15.57)
2
and
E[a2k,I ] E[a2k,Q ]
𝛾S = = (15.58)
E[|ak,I |] E[|ak,Q |]
The gradient of (15.56) with respect to c yields (see also (7.321))
∇c J = ∇Re[c] J + j ∇Im[c] J = E[ej𝜑̂ k x∗k (Θ(yk,I ) + j Θ(yk,Q ))] (15.59)
where
d
Θ(𝑣) = Φ(𝑣) = 𝑣 − 𝛾S sgn(𝑣) (15.60)
d𝑣
The partial derivative with respect to the carrier phase estimate is given by (see also (7.326))
𝜕
J = E[Im(yk (Θ(yk,I ) + j Θ(yk,Q ))∗ )] (15.61)
𝜕 𝜑̂
Defining the Sato pseudo error for QAM systems as
𝜖S,k = yk − 𝛾S sgn(yk ) (15.62)
and observing that
Im(yk 𝜖S,k
∗
) = −Im(𝛾S yk sgn(y∗k )) (15.63)
the equalizer coefficient updating and carrier phase estimate are given by
ck+1 = ck − 𝜇 𝜖S,k ej𝜑̂ k x∗k (15.64)
15.5. Self-training equalization for QAM systems 775

𝜑̂ k+1 = 𝜑̂ k + 𝜇𝜑 Im[𝛾S yk sgn(y∗k )] (15.65)

where 𝜇𝜑 is the adaptation gain of the carrier phase tracking loop. Equations (15.64) and (15.65) are
analogous to (7.323) and (7.328) for the decision directed case.
The same observations made for self-training equalization of PAM systems using the Sato algorithm
hold for QAM systems. In particular, assuming that the algorithm converges to a point of global
minimum of the cost function, we recall that the variance of the pseudo error assumes high values in
the neighbourhood of the point of convergence. Therefore, in the steady state it is necessary to adopt
a decision directed algorithm. To obtain smooth transitions between the self-training mode and the
decision directed mode without the need of a further control of the distortion level, in QAM systems we
can use extensions of the Benveniste–Goursat and stop-and-go algorithms considered for self-training
of PAM systems.

15.5.1 Constant-modulus algorithm

In the constant-modulus algorithm (CMA) for QAM systems proposed by Godard [15], self-training
equalization is based on the cost function
J = E[(|̃yk |p − Rp )2 ] = E[(|yk |p − Rp )2 ] (15.66)
where p is a parameter that usually assumes the value p = 1 or p = 2. We note that, as J depends on the
absolute value of the equalizer output raised to the p-th power, the CMA does not require the knowledge
of the carrier phase estimate. Figure 15.11 shows the block diagram of a receiver using the CMA.

data
equalizer detector
xk y~k yk a^k
c

^
e–jφk

carrier
coefficient
phase
adaptation
tracking

Figure 15.11 Block diagram of a self-training equalizer using the CMA.

The gradient of (15.66) is given by

∇c J = 2p E[(|̃yk |p − Rp )|̃yk |p−2 ỹ k x∗k ] (15.67)
The constant Rp is chosen so that the gradient is equal to zero for a perfectly equalized system; therefore,
we have
E[|ak |2p ]
Rp = (15.68)
E[|ak |p ]
For example, for a 64-QAM constellation, we obtain R1 = 6.9 and R2 = 58. The 64-QAM constellation
and the circle of radius R1 = 6.9 are illustrated in Figure 15.12.
By using (15.67), we obtain the equalizer coefficient updating law
ck+1 = ck − 𝜇(|̃yk |p − Rp )|̃yk |p−2 ỹ k x∗k (15.69)
776 Chapter 15. Self-training equalization

Im[ak]

R1 = 6.9

R1

Re[ak]
1 3

Figure 15.12 The 64-QAM constellation and the circle of radius R1 = 6.9.

For p = 1, (15.69) becomes

ỹ k ∗
ck+1 = ck − 𝜇(|̃yk | − R1 ) x (15.70)
|̃yk | k
We note that the Sato algorithm, introduced in Section 15.2, can then be viewed as a particular case of
the CMA.

The contour algorithm

Let us consider a receiver in which the received signal is sampled at the symbol rate and the carrier
phase recovery is ideally carried out before the equalizer. The scheme of Figure 15.1 is still valid, and
the complex-valued baseband equivalent channel output is given by
∑
+∞
xk = hi ak−i + 𝑤k (15.71)
i=−∞

The equalizer output is expressed as yk = cT xk . To generalize the notion of pseudo error of the contour
algorithm introduced in Section 15.3 for PAM systems, we define a contour line  that connects the
outer points of the constellation. For simplicity, we assume a square constellation with L × L points, as
illustrated in Figure 15.13 for the case L = 8.
Let  be the region of the complex plane enclosed by the contour line  and let  ∉  by definition.
We denote by yk the closest point to yk on  every time that yk ∉ , that is every time the point yk is
found outside the region enclosed by . The pseudo error (15.37) is now extended as follows:
⎧ yk − yk
⎪ } if yk ∉ 
𝜖CA,k = ⎨ −𝛿k sgn(yk,I ) if |yk,I | ≥ |yk,Q | (15.72)
⎪ −j 𝛿k sgn(yk,Q ) if yk ∈ 
⎩ if |yk,I | < |yk,Q |

Also in this case, 𝛿k is a non-negative parameter, updated at each iteration as

{
𝛿k − p Δ if yk ∉ 
𝛿k+1 = (15.73)
𝛿k + (1 − p ) Δ if yk ∈ 
15.5. Self-training equalization for QAM systems 777

Im[yk]

y~kC y~k

Re[yk]

Figure 15.13 Illustration of the contour line and surface  for a 64-QAM constellation.

where, in analogy with (15.40), the probability p ≃ P[yk ∈ ] is computed assuming that yk is an
equalized signal in the presence of additive noise.
Let 𝛼max be the maximum absolute value of the real and imaginary parts of the square L × L symbol
constellation. If |yk,I | ≥ 𝛼max or |yk,Q | ≥ 𝛼max , but not both, the projection of the sample yk on the contour
line  yields a non-zero pseudo error along one dimension and a zero error in the other dimension. If
both |yk,I | and |yk,Q | are larger than 𝛼max , yk is chosen as the corner point of the constellation closest to
yk ; in this case, we obtain a non-zero pseudo error in both dimensions.
Thus, the equalizer coefficients are updated according to the algorithm

ck+1 = ck − 𝜇 𝜖CA,k x∗k (15.74)

Clearly, the contour algorithm can also be applied to systems that use non-square constellations. In
any case, the robust algorithm for carrier phase tracking that is described in the next section requires
that the shape of the constellation is non-circular.

Joint contour algorithm and carrier phase tracking

We now apply the idea of generating an error signal with respect to the contour line of a constellation
to the problem of carrier phase recovery and frequency offset compensation [12]. With reference to
the scheme of Figure 15.10, we denote as 𝜑̂ k the carrier phase estimate used for the received signal
demodulation at instant k. If carrier recovery follows equalization, the complex equalizer output signal
is given by
yk = ỹ k e−j𝜑̂ k (15.75)

As for equalizer coefficient updating, reliable information for updating the carrier phase estimate 𝜑̂ k
is only available if yk falls outside of the region . As illustrated in Figure 15.14, the phase estimation
error can then be computed as (see also (15.65))

Δ𝜑k ≃ Im(yk y∗ ∗ ∗

k ) = −Im[yk (yk − yk )] (15.76)

If yk falls within , the phase error is set to zero, that is Δ𝜑k = 0.

778 Chapter 15. Self-training equalization

Im[yk]
(–αmax , +αmax) (+αmax , +αmax)

ykC
yk
Δφk

Re[yk]

region 
region 

(–αmax , –αmax) (+αmax , –αmax)

Figure 15.14 Illustration of the rotation of the symbol constellation in the presence of a phase
error, and definition of Δ𝜑k .

From Figure 15.14, we note that the phase error Δ𝜑k is invariant with respect to a rotation of yk equal
to an integer multiple of 𝜋∕2. Then to determine Δ𝜑k , we can first rotate yk as
y′k = yk ej𝓁𝜋∕2 (15.77)
where 𝓁 is chosen such that Re[y′n ] > |Im[y′n ]| (shaded region in Figure 15.14). Furthermore, we observe
that the information on the phase error obtained by samples of the sequence {yk } that fall in the corner
regions, where |yk,I | > 𝛼max and |yk,Q | > 𝛼max , is not important. Thus we calculate a phase error only if yk
is outside of , but not in the corner regions, that is if y′k ∈ , with  = {y′k ∶ Re[y′k ] > 𝛼max , |Im[y′k ]| <
𝛼max }. Then, (15.76) becomes
{
−Im[y′k ][Re[y′k ] − 𝛼max ] if y′k ∈ 
Δ𝜑k = (15.78)
0 otherwise
In the presence of a frequency offset equal to Ω∕(2𝜋), the probability distribution of the samples {yk }
rotates at a rate of Ω∕(2𝜋) revolutions per second. For large values of Ω∕(2𝜋), the phase error Δ𝜑k
does not provide sufficient information for the carrier phase tracking system to achieve a lock condition;
therefore, the update of 𝜑̂ k must be made by a second-order phase-locked loop (PLL), where in the
update of the second-order term a factor that is related to the value of the frequency offset must be
included (see Section 14.7).
The needed information is obtained observing the statistical behaviour of the term Im[y′k ], conditioned
by the event y′k ∈ . At instants in which the sampling distribution of yk is aligned with , the distri-
bution of Im[y′k ] is uniform in the range [−𝛼max , 𝛼max ]. Between these instants, the distribution of Im[y′k ]
exhibits a time varying behaviour with a downward or upward trend depending on the sign of the fre-
quency offset, with a minimum variance when the corners of the rotating probability distribution of yk ,
which we recall rotates at a rate of Ω∕(2𝜋) revolutions per second, cross the coordinate axes. Defining
{
𝑣 if |𝑣| < 𝛼max
Q(𝑣) = (15.79)
0 otherwise
15.6. Examples of applications 779

from the observation of Figure 15.14, a simple method to extract information on Ω∕(2𝜋) consists in
evaluating
Δ Im[y′k ] = Q{Im[y′k ] − Im[y′m ]}, y′k ∈  (15.80)
where m < k denotes the last time index for which y′k ∈ . In the mean, Δ Im[y′k ] exhibits the sign of
the frequency offset.
The equations for the updating of the parameters of a second-order PLL for the carrier phase recovery
then become {
𝜑̂ k+1 = 𝜑̂ k + 𝜇𝜑 Δ𝜑k + Δ𝜑̂ c,k
if y′k ∈ 
Δ𝜑̂ c,k+1 = Δ𝜑̂ c,k + 𝜇f1 Δ𝜑k + 𝜇f2 Δ Im[y′k ]
{ (15.81)
𝜑̂ k+1 = 𝜑̂ k
otherwise
Δ𝜑̂ c,k+1 = Δ𝜑̂ c,k
where 𝜇𝜑 , 𝜇f1 , and 𝜇f2 are suitable adaptation gains; typically, 𝜇𝜑 is in the range 10−4 to 10−3 , 𝜇f1 =
(1∕4)𝜇𝜑2 , and 𝜇f2 ≃ 𝜇f1 .
The rotation of yk given by (15.77) to obtain y′k also has the advantage of simplifying the error com-
putation for self-training equalizer coefficient adaptation with the contour algorithm.
With no significant effect on performance, we can introduce a simplification similar to that adopted
to update the carrier phase, and let the pseudo error equal zero if yk is found in the corner regions, that
is 𝜖CA,k = 0 if Im[y′k ] > 𝛼max . By using (15.72) and (15.73) to compute the pseudo error, the coefficient
updating equation (15.74) becomes (see (7.323))
ck+1 = ck − 𝜇 𝜖CA,k ej𝜑̂ k x∗k (15.82)

15.6 Examples of applications

In this section, we give examples of applications that illustrate the convergence behaviour and steady
state performance of self-training equalizers, with particular regard to the contour algorithm.
We initially consider self-training equalization for PAM transmission systems over unshielded
twisted-pair (UTP) cables with frequency response given by
1.2 √
Ch (f ) = 10− 20 e−(0.00385 jf +0.00028 f )L
(15.83)
where f is expressed in MHz and L in meters.

Example 15.6.1
As a first example, we consider a 16-PAM system (M = 16) with a uniform probability distribution of the
input symbols and symbol rate equal to 25 MBaud; the transmit and receive filters are designed to yield
an overall raised cosine channel characteristic for a cable length of 50 m. In the simulations, the cable
length is chosen equal to 100 m, and the received signal is disturbed by additive white Gaussian noise.
The signal-to-noise ratio at the receiver input is equal to Γ = 36 dB. Self-training equalization is achieved
by a fractionally spaced equalizer having N = 32 coefficients, and input signal sampled with sampling
period equal to T∕2. Figure 15.15 shows the convergence of the contour algorithm (15.37)–(15.38) for
𝛿0 = 0 and c0 chosen equal to the zero vector. The results are obtained for a cable with attenuation
𝛼(f )|f =1 = 3.85 × 10−6 [m−1 Hz−1∕2 ], parameters of the self-training equalizer given by 𝜇 = 10−5 and
Δ = 2.5 × 10−4 , and ideal timing recovery.
780 Chapter 15. Self-training equalization

(a) 0

–10

–20
δn (dB)

–30

–40

–50
0 1 2 3 4 5 × 105
Number of iterations

(b)

0
Mean–square error (dB)

–10

–20
0 1 2 3 4 5 × 105
Number of iterations
(c)
0.025

0.02
Relative frequency

0.015

0.01

0.005

0
–30 –15 0 15 30 –30 –15 0 15 30
yk yk

Figure 15.15 Illustration of the convergence of the contour algorithm for a 16-PAM system: (a)
behaviour of the parameter 𝛿n , (b) MSE convergence, and (c) relative frequency of equalizer output
samples at the beginning and the end of the convergence process.
15.6. Examples of applications 781

Example 15.6.2
We consider self-training equalization for a baseband quaternary partial response class IV system
(M = 4) for transmission at 125 Mbit/s over UTP cables; a very-large-scale integration (VLSI)
transceiver implementation for this system will be described in Chapter 19. We compare the per-
formance of the contour algorithm, described in Section 15.3, with the Sato algorithm for partial
response systems. Various realizations of the MSE convergence of a self-training equalizer with
N = 16 coefficients are shown in Figures 15.16 and 15.17 for the Sato algorithm and the contour
algorithm, respectively. The curves are parameterized by t = ΔT∕T, where T = 16 ns, and ΔT denotes
the difference between the sampling phase of the channel output signal and the optimum sampling
phase that yields the minimum MSE; we note that the contour algorithm has a faster convergence with
respect to the Sato algorithm and yields significantly lower values of MSE in the steady state. The
Sato algorithm can be applied only if timing recovery is achieved prior to equalization; note that the
convergence characteristics of the contour algorithm makes self-training equalization possible even in
the presence of considerable distortion and phase drift of the received signal.

–5
t = 0.15
Mean–square error (dB)

–10 t = 0.1

–15 t = 0.05

t=0
–20

–25

–30
0 8000 16 000 24 000
Number of iterations

Figure 15.16 MSE convergence with the Sato algorithm for a QPR-IV system.

0
t = 0.15
–5
Mean–square error (dB)

t = 0.1
–10

–15

–20
t = 0.05
–25
t=0
–30
0 8000 16 000 24 000
Number of iterations

Figure 15.17 MSE convergence with the contour algorithm for a QPR-IV system.
782 Chapter 15. Self-training equalization

Example 15.6.3
We now examine self-training equalization for a 256-QAM transmission system having a square con-
stellation with L = 16 (M = 256), and symbol rate equal to 6 MBaud. Along each dimension, symbols
±3, ±1 have probability 2∕20, and symbols ±15, ±13, ±11, ±9, ±7, and ±5 have probability 1∕20.
The overall baseband equivalent channel impulse response is illustrated in Figure 15.18. The received
signal is disturbed by additive white Gaussian noise. The signal-to-noise ratio at the receiver input is
equal to Γ = 39 dB. Signal equalization is obtained by a fractionally spaced equalizer having N = 32
coefficients, and input signal sampled with sampling period equal to T∕2.

0.4
Real part
Imaginary part
0.2

–0.2

–0.4

0 1 2 3 4 5 6
t/T

Figure 15.18 Overall baseband equivalent channel impulse response for simulations of a 256-QAM
system. Source: Reproduced with permission from Cherubini et al. [12]. ©1998, IEEE.
0
δ0 = 0.3
δ0= 0.1 Mean-square error (dB)
–10

–20

–30

20 log10(δk)
–40

0 2 4 6 8 10
Number of iterations × 104

Figure 15.19 Convergence behaviour of MSE and parameter 𝛿k using the contour algorithm for a
256-QAM system with non-uniform distribution of input symbols. Source: Reproduced with
permission from Cherubini et al. [12]. ©1998, IEEE.
Bibliography 783

Figure 15.19 shows the convergence of the contour algorithm and the behaviour of the parameter 𝛿k
for p = 361∕400, various initial values of 𝛿0 , and c0 given by a vector with all elements equal to zero
except for one element. Results are obtained for 𝜇 = 10−4 , Δ = 10−4 , and ideal timing and carrier phase
recovery.

Example 15.6.4
With reference to the previous example, we examine the behaviour of the carrier phase recovery algo-
rithm, assuming ideal timing recovery. Figure 15.20 illustrates the behaviour of the MSE and of the
second order term Δ𝜑̂ c,k for an initial frequency offset of +2.5 kHz, 𝜇𝜑 = 4 × 10−4 , 𝜇f1 = 8 × 10−8 , and
𝜇f2 = 2 × 10−8 .

0
Mean-square error (dB)

–10

Δϕ^ c,k × 104

(rad)
–20

–30
0 0.4 0.8 1.2 1.6 2
Number of iterations × 105

Figure 15.20 Illustration of the convergence behaviour of MSE and second order term Δ𝜑̂ c,k using
the contour algorithm in the presence of an initial frequency offset equal to 500 ppm for a 256-QAM
system with non-uniform distribution of input symbols. Source: Reproduced with permission from
Cherubini et al. [12]. ©1998, IEEE.

Bibliography

[1] Ichikawa, H., Sango, J., and Murase, T. (1987). 256 QAM multicarrier 400 Mb/s microwave
radio system field tests. Proceedings of 1987 IEEE International Conference on Communications,
Philadelphia, PA, pp. 1803–1808.
[2] Ross, F.J. and Taylor, D.P. (1991). An enhancement to blind equalization algorithms. IEEE Trans-
actions on Communications 39: 636–639.
[3] Proakis, J.G. and Nikias, C.L. (1991). Blind equalization. Proceedings of SPIE Adaptive Signal
Processing, San Diego, CA, Volume 1565 (22–24 July 1991), pp. 76–87.
[4] Bellini, S. (1991). Blind equalization and deconvolution. Proc. SPIE Adaptive Signal Processing,
San Diego, CA, Volume 1565 (22–24 July 1991), pp. 88–101.
[5] Benvenuto, N. and Goeddel, T.W. (1995). Classification of voiceband data signals using the constel-
lation magnitude. IEEE Transactions on Communications 43: 2759–2770.
[6] Liu, R. and Tong, L. (eds.) (1998). Special issue on blind systems identification and estimation.
IEEE Proceedings 86.
784 Chapter 15. Self-training equalization

[7] Sato, Y. (1975). A method of self-recovering equalization for multilevel amplitude-modulation

systems. IEEE Transactions on Communications 23: 679–682.
[8] Tong, L., Xu, G., Hassibi, B., and Kailath, T. (1995). Blind channel identification based on
second-order statistics: a frequency-domain approach. IEEE Transactions on Information Theory
41: 329–334.
[9] Benveniste, A., Goursat, M., and Ruget, G. (1980). Robust identification of a nonminimum phase
system: blind adjustment of a linear equalizer in data communications. IEEE Transactions on
Automatic Control 25: 385–399.
[10] Benveniste, A. and Goursat, M. (1984). Blind equalizers. IEEE Transactions on Communications
32: 871–883.
[11] Picchi, G. and Prati, G. (1987). Blind equalization and carrier recovery using a ‘Stop-and-Go’ deci-
sion directed algorithm. IEEE Transactions on Communications 35: 877–887.
[12] Cherubini, G., Ölçer, S., and Ungerboeck, G. (1998). The contour algorithm for self-training equal-
ization. In: Proceedings of Broadband Wireless Communications, 9th Tyrrhenian International
Workshop on Digital Communications, Lerici, Italy (7–10 September 1997) (ed. M. Luise and S.
Pupolin), 58–69. Berlin: Springer-Verlag.
[13] Cherubini, G., Ölçer, S., and Ungerboeck, G. (1995). Self-training adaptive equalization for multi-
level partial-response transmission systems. Proceedings of 1995 IEEE International Symposium on
Information Theory, Whistler, Canada (17–22 September 1995), p. 401.
[14] Cherubini, G. (1994). Nonlinear self-training adaptive equalization for partial-response systems.
IEEE Transactions on Communications 42: 367–376.
[15] Godard, D.N. (1980). Self recovering equalization and carrier tracking in two-dimensional data
communication systems. IEEE Transactions on Communications 28: 1867–1875.

Appendix 15.A On the convergence of the contour algorithm

∑+∞
Given y = y0 = i=−∞ ci x−i , we show that the only minima of the cost function J = E[Φ(y)] correspond
to the equalizer settings C = ±H −1 , for which we obtain Ψ = ±I, except for a possible delay. If the
systems H and C have finite energy, then also Ψ has finite energy and E[Φ(y)] may be regarded as
∑+∞
a functional J(Ψ) = E[Φ(y)], where y is expressed as y = i=−∞ 𝜓i a−i ; thus, we must prove that the
only minima of (Ψ) are found at points Ψ = ±I. We consider input symbols with a probability density
function pak (𝛼) uniform in the interval [−𝛼max , 𝛼max ].
∑ 2
For the analysis, we express the system Ψ as Ψ = rΨ, where r ≥ 0, and Ψ ↔ {𝜓 i }, with i 𝜓 i = 1,
denotes the normalized overall system. We consider the cost function J as a functional (Ψ) = E[Φ(ry)],
∑+∞
where y = y0 = i=−∞ 𝜓 i a−i denotes the output of the system Ψ, and Φ has derivative Θ given by
(15.34). Let {
̃ x − 𝛼max sgn(x) if |x| ≥ 𝛼max
Θ(x) = (15.84)
0 otherwise
and pyk (x) denote the probability density function of y.
We express the parameter 𝛾CA as
𝛾CA = 𝛾Ψ + 𝛾r
̃
∫ b Θ(b) pyk (b) db
𝛾Ψ =
∫ |b| pyk (b) db (15.85)
{
1−r if r ≤ 1
𝛾r =
0 otherwise
15.A. On the convergence of the contour algorithm 785

Examine the function (Ψ) on the unit sphere . To claim that the only minima of (Ψ) are found at
points Ψ = ±I, we apply Theorem 3.5 of [9]. Consider a pair of indices (i, j), i ≠ j, and a fixed system
∑ 2
with coefficients {𝜓 𝓁 }𝓁≠i,j , such that R2 = 1 − 𝓁≠i,j 𝜓 𝓁 > 0. For 𝜑 ∈ [0, 2𝜋), let Ψ𝜑 ∈  be the system
with coefficients {𝜓 𝓁 }𝓁≠i,j , 𝜓 i = R cos 𝜑, and 𝜓 j = R sin 𝜑; moreover let (𝜕∕𝜕𝜑)(Ψ𝜑 ) be the derivative
of (Ψ𝜑 ) with respect to 𝜑 at point Ψ = Ψ𝜑 . As pak (𝛼) is sub-Gaussian, it can be shown that
𝜕 𝜋
(Ψ𝜑 ) = 0 for 𝜑 = k k∈ℤ (15.86)
𝜕𝜑 4
and
𝜕 𝜋
(Ψ𝜑 ) > 0 for 0 < 𝜑 < (15.87)
𝜕𝜑 4
From the above equations we have that the stationary points of (Ψ𝜑 ) correspond to systems charac-
terized by the property that all non-zero coefficients have the same absolute value. Furthermore, using
symmetries of the problem, we find the only minima are at ±I, except for a possible delay, and the other
stationary points of (Ψ𝜑 ) are saddle points.
The study of the functional  is then extended to the entire parameter space. As the results obtained
for the restriction of  to  are also valid on a sphere of arbitrary radius r, we need to study only the
̃ = (rΨ), whose first and second
radial derivatives of . For this reason, we consider the function (r)
derivatives are

̃ ′ (r) = ̃
b Θ(rb) pyk (b) d b − (𝛾Ψ + 𝛾r ) |b| pyk (b) db (15.88)
∫ ∫
and
̃ ′′ (r) = ̃ ′ (rb) p (b) db − 𝛾 ′
b2 Θ |b| pyk (b) db (15.89)
∫ yk r ∫
where Θ̃ ′ and 𝛾 ′ denote derivatives.
r
Recalling the expressions of 𝛾Ψ and 𝛾r given by (15.85), we obtain ̃ ′ (0) < 0 and ̃ ′′ (r) > 0. Therefore,
there exists a radius r0 such that the radial component of the gradient is negative for r < r0 and positive
for r > r0 . For a fixed point Ψ ∈ , r0 is given by the solution of the equation

̃
b Θ(rb) pyk (b) db − (𝛾Ψ + 𝛾r ) |b| pyk (b) db = 0 (15.90)
∫ ∫
Substituting the expressions of 𝛾Ψ and 𝛾r in (15.90), we obtain r0 = 1, ∀Ψ ∈ . Therefore the only
minima of  are at ±I. Furthermore, as the radial component of the gradient vanishes on , the steepest
descent lines of  do not cross the unit sphere. Using the same argument given in [9], we conclude that
the points ±I are the only stable attractors of the steepest descent lines of the function , and that the
unique stable attractors of the steepest descent lines of J are ±H −1 .
Note that the parameter 𝛾Ψ is related to the distortion of the distribution of the input sequence filtered
by a normalized system; this parameter varies along the trajectories of the stochastic gradient algorithm
and vanishes as a point of minimum is reached. Moreover, note that the parameter 𝛾r indicates the devi-
ation of the overall system gain from the desired unit value; if the gain is too small, the gradient is
augmented with an additional driving term.
Chapter 16

Low-complexity demodulators

In this chapter, we discuss modulation and demodulation schemes that are well suited for applications
to mobile radio systems because of their simplicity, low-power consumption, and robustness against
disturbances introduced by the transmission channel.

16.1 Phase-shift keying

In phase-shift keying (PSK), values of ak are given by (6.8). A PSK transmitter for M = 8 is shown in
Figure 16.1. The bit mapper (BMAP) maps a sequence of log2 M bits to a constellation point represented
by ak . The quadrature components ak,I and 𝛼k,Q are input to interpolator filters hTx . The filter output
signals are multiplied by the carrier signal, cos(2𝜋f0 t), and by the carrier signal phase-shifted by 𝜋∕2,
for example by a Hilbert filter, sin(2𝜋f0 t), respectively. The transmitted signal is obtained by adding the
two components.
An implementation of the receiver is illustrated in Figure 16.2, which is similar to that of Figure 7.5.
We note that the decision regions are angular sectors with phase 2𝜋∕M. For M = 2, 4, and 8, simple
decision rules can be defined. For M > 8, detection can be made by observing the phase 𝑣k of the received
sample yk .
Curves of Pbit as a function of Γ are shown in Figure 6.10.
For M = 2 (binary phase-shift keying, BPSK), we get 𝜑1 = 𝜑0 and 𝜑2 = 𝜋 + 𝜑0 , where 𝜑0 is an
arbitrary phase. Pe is given in (6.84).
The transmitter and the receiver for a BPSK system are shown in Figure 16.3 and have a very simple
implementation. The BMAP of the transmitter maps 0 in −1 and 1 in +1. At the receiver, the decision
element implements the sign function to detect binary data. The inverse bit-mapper (IBMAP) to recover
the bits of the information message is straightforward.

16.1.1 Differential PSK

We assume now that the receiver recovers the carrier signal, except for a phase offset 𝜑a . In particular,
with reference to the scheme of Figure 16.2, the reconstructed carrier is cos(2𝜋f0 t − 𝜑a ). Consequently,
it is as if the constellation at the receiver were rotated by 𝜑a . To prevent this problem, there are two
strategies. By the coherent (CO) method, a receiver estimates 𝜑a from the received signal, and considers
the original constellation for detection, using the received signal multiplied by a phase offset, i.e. yk e−j𝜑̂ a ,

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
788 Chapter 16. Low-complexity demodulators

αk,I
BMAP hTx

b0
cos(2πf0t)
binary +
b2 s(t)
data
S/P
NOT π/2 +

b1 b2 –sin(2πf0t)

BMAP hTx
αk,Q
bit-mapper

Figure 16.1 Transmitter of an 8-PSK system.

2
gM(t) = h*Tx(t0 – t)
Eh
t0 + kT
gM
yk,I
cos(2πf0 t)
yk,Q
carrier υk = tan–1
r(t) recovery
yk,I
binary
compare vk IBMAP data
with
π/2 M thresholds

–sin(2πf0 t)
t0 + kT
gM
yk,Q

Figure 16.2 Receiver of an M-PSK system. Thresholds are set at (2𝜋∕M)n, n = 1, … , M.

hTx s(t)
binary data

cos(2πf0t)

(a)

t0 + kT
r(t) yk
1 binary
gM data
0
carrier
recovery cos(2πf0t)

(b)

Figure 16.3 Schemes of (a) transmitter and (b) receiver for a BPSK system with 𝜑0 = 0.
16.1. Phase-shift keying 789

where 𝜑̂ a is the estimate of 𝜑a . By the differential non-coherent (NC) method, a receiver detects the data
using the difference between the phases of signals at successive sampling instants. In other words
● for M-PSK, the phase of the transmitted signal at instant kT is given by (6.10), with
{ }
𝜋 3𝜋 (2M − 1)𝜋
𝜃k ∈ , ,…, (16.1)
M M M
● for M-differential phase-shift keying (M-DPSK),1 the transmitted phase as instant kT is given by
{ }
2𝜋 2𝜋
𝜓k′ = 𝜓k−1
′
+ 𝜃k , 𝜃k ∈ 0, ,…, (M − 1) (16.2)
M M
that is, the phase associated with the transmitted signal at instant kT is equal to that transmitted
at the previous instant (k − 1)T plus the increment 𝜃k , which can assume one of M values. We note
that the decision thresholds for 𝜃k are now placed at (𝜋∕M)(2n − 1), n = 1, … , M.
For a phase offset equal to 𝜑a introduced by the channel, the phase of the signal at the detection point
becomes
𝜓k = 𝜓k′ + 𝜑a (16.3)
In any case,
𝜓k − 𝜓k−1 = 𝜃k (16.4)
and the ambiguity of 𝜑a is removed. For phase-modulated signals, three differential non-coherent
receivers that determine an estimate of (16.4) are discussed in Section 16.2.

Error probability of M-DPSK

For Γ ≫ 1, using the definition of the Marcum function Q1 (⋅, ⋅) (see Appendix 6.B), it can be shown
that the error probability is approximated by the following bound [1, 2]
(√ ( ) √ ( ))
𝜋 𝜋
Pe ⪅ 1 + Q1 Γ 1 − sin , Γ 1 + sin
M M
(√ ( ) √ ( ))
(16.5)
𝜋 𝜋
− Q1 Γ 1 + sin , Γ 1 − sin
M M
Moreover, if M is large, the approximation (6.160) can be used and we get
[√ (√ √ )]
𝜋 𝜋
Pe ≃ 2Q Γ 1 + sin − 1 − sin
M M
(√ ) (16.6)
𝜋
≃ 2Q Γ sin
M
For Gray labelling of the values of 𝜃k in (16.2), the bit error probability is given by
Pe
Pbit = (16.7)
log2 M
For M = 2, the exact formula of the error probability is [1, 2]
1 −Γ
Pbit = Pe = e (16.8)
2
For M = 4, the exact formula is [1, 2]
2 +b2 )
Pe = 2Q1 (a, b) − I0 (ab) e−0.5(a (16.9)

1
Note that we consider a differential non-coherent receiver to which is associated a differential symbol encoder at the
transmitter (see (16.2) or (16.14)). However, as we will see in the next section, a differential encoder and a coherent
receiver can be used.
790 Chapter 16. Low-complexity demodulators

10−1
PSK
DPSK

10−2

10−3
Pbit

10−4 M=2 M=4 M=8 M = 16 M = 32

10−5

10−6
5 10 15 20 25 30 35
Γ (dB)

Figure 16.4 Comparison between PSK and DPSK.

where √ √ √ √
a= Γ(1 − 1∕2), b= Γ(1 + 1∕2) (16.10)
and where the function I0 is defined in (4.56).
Using the previous results, a comparison in terms of Pbit between DPSK (16.6) and PSK (6.83) is
given in Figure 16.4. We note that, for Pbit = 10−3 , DPSK presents a loss of only 1.2 dB in Γ for M = 2,
that increases to 2.3 dB for M = 4, and to 3 dB for M > 4.
As a DPSK receiver is simpler as compared to a coherent PSK receiver, in that it does not require
recovery of the carrier phase, for M = 2 DPSK is usually preferred to PSK.
Note that, if the previously received sample is used as a reference, DPSK gives lower performance
with respect to PSK, especially for M ≥ 4, because both the current sample and the reference sample are
corrupted by noise. This drawback can be mitigated if the reference sample is constructed by using more
than one previously received sample [3]. In this way, we establish a gradual transition between differen-
tial phase demodulation and coherent demodulation. In particular, if the reference sample is constructed
using the samples received in the two previous modulation intervals, DPSK and PSK yield similar perfor-
mance [3]. PSK, especially in its differential form, is used in satellite communications. This is due to the
fact that satellite transmission undergoes very sudden changes in signal attenuation, which do not affect
the performance of a DPSK receiver as would be the case for a quadrature-amplitude-modulation (QAM)
receiver. As an application example, we recall that 8-PSK is used in satellite digital video broadcasting
(DVB-S) [4].
16.1. Phase-shift keying 791

16.1.2 Differential encoding and coherent demodulation

If 𝜑a is a multiple of 2𝜋∕M, at the receiver the phase difference can be formed between the phases of two
consecutive coherently detected symbols, instead of between the phases of two consecutive samples. In
this case, symbols are differentially encoded before modulation.

Differentially encoded BPSK

Let bk be the value of the information bit at instant kT, bk ∈ {0, 1}.

BPSK system without differential encoding The phase 𝜃k ∈ {0, 𝜋} is associated with bk by the bit
map of Table 16.1.

Differential encoder For any c−1 ∈ {0, 1}, we encode the information bits as
ck = ck−1 ⊕ bk , bk ∈ {0, 1} , k≥0 (16.11)

where ⊕ denotes the modulo 2 sum; therefore, ck = ck−1 if bk = 0, and2 ck = ck−1 if bk = 1. For the bit
map of Table 16.2, we have that bk = 1 causes a phase transition, and bk = 0 causes a phase repetition.
{ }
Decoder If ĉ k are the detected coded bits at the receiver, the information bits are recovered by
b̂ k = ĉ k ⊕ (−̂ck−1 ) = ĉ k ⊕ ĉ k−1 (16.12)
{ }
We note that a phase ambiguity 𝜑a = 𝜋 does not alter the recovered sequence b̂ k : in fact. In this case,
{ }
ĉ k becomes {̂c′k = ĉ k ⊕ 1} and we have

(̂ck ⊕ 1) ⊕ (̂ck−1 ⊕ 1) = ĉ k ⊕ ĉ k−1 = b̂ k (16.13)

Multilevel case
{ }
Let dk be a multilevel information sequence, with dk ∈ {0, 1, … , M − 1}. In this case, we have
ck = ck−1 ⊕ dk (16.14)
M

Table 16.1: Bit map for a BPSK system

bk Transmitted phase 𝜃k (rad)

0 0
1 𝜋

Table 16.2: Bit map for a differentially

encoded BPSK system

ck Transmitted phase 𝜓k (rad)

0 0
1 𝜋

2
c denotes the one’s complement of c: 1 = 0 and 0 = 1.
792 Chapter 16. Low-complexity demodulators

where ⊕ denotes the modulo M sum. Because ck ∈ {0, 1, … , M − 1}, the phase associated with the bit
M
map is 𝜓k ∈ {𝜋∕M, 3𝜋∕M, … , (2M − 1)𝜋∕M}. This encoding and bit-mapping scheme are equivalent
to (16.2).
At the receiver, the information sequence is recovered by
d̂ k = ĉ k ⊕ (−̂ck−1 ) (16.15)
M
{ }
It is easy to see that an offset equal to j ∈ {0, 1, … , (M − 1)} in the sequence ĉ k , corresponding to a
{ } { }
phase offset equal to {0, 2𝜋∕M, … , (M − 1)2𝜋∕M} in 𝜓k , does not cause errors in d̂ k . In fact,
( ) [ ( )]
ĉ k ⊕ j ⊕ − ĉ k−1 ⊕ j = ĉ k ⊕ (−̂ck−1 ) = d̂ k (16.16)
M M M M

Performance of a PSK system with differential encoding and coherent demodulation by the scheme of
Figure 16.2 is worse as compared to a system with absolute phase encoding. However, for small Pe , up
{ } { }
to values of the order of 0.1, we observe that an error in ĉ k causes two errors in d̂ k . Approximately,
Pe increases by a factor 2,3 which causes a negligible loss in terms of Γ.
To combine Gray labelling of values of ck with the differential encoding (16.14), a two-step procedure
is adopted:
1. represent the values of dk with a Gray labelling using a combinatorial table, as illustrated for
example in Table 16.3 for M = 8;
2. determine the differentially encoded symbols according to (16.14).

Example 16.1.1 (Differential encoding 2B1Q)

We consider a differential encoding scheme for a four-level system that makes the reception insensitive
to a possible change of sign of the transmitted sequence. For M = 4, this implies insensitivity to a phase
rotation equal to 𝜋 in a 4-PSK signal or to a change of sign in a 4-pulse-amplitude-modulation (PAM)
signal.
For M = 4, we give the law between the binary representation of dk = (dk(1) , dk(0) ), dk(i) ∈ {0, 1}, and
the binary representation of ck = (ck(1) , c(0)
k
), c(i)
k
∈ {0, 1}:

c(1)
k
= dk(1) ⊕ c(1)
k−1
(16.19)
c(0)
k
= dk(0) ⊕ c(1)
k

Table 16.3: Gray coding for M = 8

Three information bits values of dk

0 0 0 0
0 0 1 1
0 1 1 2
0 1 0 3
1 1 0 4
1 1 1 5
1 0 1 6
1 0 0 7

3
If we indicate with Pe,Ch the channel error probability, then the error probability after decoding is given by [1]

Binary case Pbit = 2Pbit,Ch [1 − Pbit,Ch ] (16.17)

Quaternary case Pe = 4Pe,Ch − 8P2e,Ch + 8P3e,Ch − 4P4e,Ch (16.18)
16.2. (D)PSK non-coherent receivers 793

Table 16.4: Bit map for the differential

encoder 2B1Q

c(1)
k
c(0)
k
transmitted symbol ak
0 0 −3
0 1 −1
1 0 1
1 1 3

The bit map is given in Table 16.4.

The equations of the differential decoder are
d̂ k(1) = ĉ (1)
k
⊕ ĉ (1)
k−1
(16.20)
d̂ k(0) = ĉ (0)
k
⊕ ĉ (1)
k

16.2 (D)PSK non-coherent receivers

We introduce three non-coherent receivers to demodulate phase modulated signals.

16.2.1 Baseband differential detector

For a continuous M-DPSK transmission with symbol period T, the transmitted signal is given by
∑
+∞
[ ]
s(t) = Re ej𝜓k hTx (t − kT)ej2𝜋f0 t (16.21)
k=−∞

where 𝜓k is the phase associated with the transmitted symbol at instant kT given by the recursive
equation (16.2).
At the receiver, the signal r is a version of s, filtered by the transmission channel and corrupted by
additive noise. We denote as gA the cascade of passband filters used to amplify the desired signal and
partially remove noise. As shown in Figure 16.5, let x be the passband received signal, centred around

LPF t0 + kT
gRc
yk,I
cos(2πf0 t)
yk = yk,I + jyk,Q data
PBF detector
carrier
r(t) x(t) recovery zk bl
gA *
yk yk–1 vk = arg zk

π/2

–sin(2πf0 t)
yk,Q
gRc
t0 + kT

Figure 16.5 Non-coherent baseband differential receiver. Thresholds are set at (2n − 1)𝜋∕M,
n = 1, 2, … , M.
794 Chapter 16. Low-complexity demodulators

the frequency f0 , equal to the carrier of the transmitted signal

[ ]
x(t) = Re x(bb) (t)ej2𝜋f0 t (16.22)
where x(bb) is the complex envelope of x with respect to f0 . Using the polar notation x(bb) (t) = Mx (t)
ejΔ𝜑x (t) , (16.22) can be written as
( )
x(t) = Mx (t) cos 2𝜋f0 t + Δ𝜑x (t) (16.23)
where Δ𝜑x is the instantaneous phase deviation of x with respect to the carrier phase (see (1.141)).
In the ideal case of absence of distortion and noise, sampling at suitable instants yields Δ𝜑x (t0 + kT) =
𝜓k . Then, for the recovery of the phase 𝜃k we can use the receiver scheme of Figure 16.5, which, based
on signal x, determines the baseband component y as
1
y(t) = yI (t) + jyQ (t) = x(bb) ∗ gRc (t) (16.24)
2
The phase variation of the sampled signal yk = y(t0 + kT) between two consecutive symbol instants
is obtained by means of the signal
zk = yk y∗k−1 (16.25)
Always in the ideal case and assuming that gRc does not distort the phase of x(bb) , zk turns out to be
proportional to ej𝜃k . The simplest data detector is the threshold detector (TD) based on the value of
( )
𝑣k = arg zk = Δ𝜑x (t0 + kT) − Δ𝜑x t0 + (k − 1)T (16.26)
Note that a possible phase offset Δ𝜑0 and a frequency offset Δf0 , introduced by the receive mixer,
yields a signal y given by [ ]
1
y(t) = x(bb) (t)e−j(Δ𝜑0 +2𝜋Δf0 t) ∗ gRc (t) (16.27)
2
Assuming that gRc does not distort the phase of x(bb) , the signal 𝑣k becomes
𝑣k = Δ𝜑x (t0 + kT) − Δ𝜑x (t0 + (k − 1)T) − 2𝜋Δ f0 T (16.28)
which shows that the phase offset Δ𝜑0 does not influence 𝑣k , while a frequency offset must be compen-
sated by the data detector, summing the constant phase 2𝜋Δf0 T.
The baseband equivalent scheme of the baseband differential receiver is given in Figure 16.6.
The choice of hTx , g(bb)
A
, and gRc is governed by the same considerations as in the case of a QAM
system; for an ideal channel, the convolution of these elements must be a Nyquist pulse.

16.2.2 IF-band (1 bit) differential detector

The scheme of Figure 16.5 is introduced only to illustrate the basic principle, as its implementation
complexity is similar to that of a coherent scheme. Indeed, this scheme has a reduced complexity because
specifications on the carrier recovery can be less stringent; moreover, it does not need phase recovery.
An alternative scheme, denoted as 1-bit differential detector (1BDD), that does not use carrier recov-
ery is illustrated in Figure 16.7. In this case, the signal is first delayed of a symbol period T and then

data
t0 + kT detector
r(bb)(t) 1 g(bb) x(bb)(t) y(t) yk zk bl
gRc vk = arg zk
2 A 1
2 *
yk–1
T (.)*
delay

Figure 16.6 Baseband equivalent scheme of Figure 16.5.

16.2. (D)PSK non-coherent receivers 795

LPF
x(t) t0 + kT
gRc
zk,I
w(t) [N0 /2] data
PBF detector
delay
r(t) x(t) bl
gA T vk = arg zk

–π
2
zk,Q
gRc
t0 + kT
x(h)(t)

Figure 16.7 Non-coherent 1 bit differential detector.

multiplied with itself (I branch) and with its −𝜋∕2 phase shifted version (Q branch) by a Hilbert filter.
On the two branch, the signals are given by
I ∶ x(t)x(t − T) [ ] [ ]
= Mx (t) cos 2𝜋f0 t + Δ𝜑x (t) Mx (t − T) cos 2𝜋f0 (t − T) + Δ𝜑x (t − T)
(16.29)
Q ∶ x(h) (t)x(t − [T) ] [ ]
= Mx (t) sin 2𝜋f0 t + Δ𝜑x (t) Mx (t − T) cos 2𝜋f0 (t − T) + Δ𝜑x (t − T)
The filter gRc removes the components around 2f0 ; the sampled filter outputs are then given by
( )
I ∶ zk,I = Mx (t0 + kT)Mx t0 + (k − 1)T
1 [ ]
cos 2𝜋f0 T + Δ𝜑x (t0 + kT) − Δ𝜑x (t0 + (k − 1)T)
2
( ) (16.30)
Q ∶ zk,Q = Mx (t0 + kT)Mx t0 + (k − 1)T
1 [ ]
sin 2𝜋f0 T + Δ𝜑x (t0 + kT) − Δ𝜑x (t0 + (k − 1)T)
2
If f0 T = n, n an integer, or by removing this phase offset by phase shifting x or zk , it results
zk,Q ( )
𝑣k = tan−1 = Δ𝜑x (t0 + kT) − Δ𝜑x t0 + (k − 1)T (16.31)
zk,I
as in (16.26).
The baseband equivalent scheme is shown in Figure 16.8, where, assuming that gRc does not distort
the desired signal, ( )
1
zk = x(bb) (t0 + kT)x(bb)∗ t0 + (k − 1)T (16.32)
2
Typically, in this modulation system, the transmit filter hTx is a rectangular pulse or a Nyquist pulse;
instead, gA is a narrow band filter to eliminate out of band noise.

w(bb)(t) [2N0]
data
t0 + kT detector
r(bb)(t) x(bb)(t) zk bl
1 g(bb) vk = arg zk
2 A
1
2
T (.)*
delay

Figure 16.8 Baseband equivalent scheme of Figure 16.7.

796 Chapter 16. Low-complexity demodulators

For a simple differential binary phase-shift-keying (DBPSK) modulation, with 𝜃k ∈ {0, 𝜋}, we only
consider the I branch, and 𝑣k = zk,I is compared with a threshold set to 0.

Signal at detection point

With reference to the transmitted signal (16.21), we consider the isolated pulse k

s(t) = Re[ej𝜓k hTx (t − kT)ej2𝜋f0 t ] (16.33)

where { }
2𝜋 2𝜋(M − 1)
𝜓k = 𝜓k−1 + 𝜃k , 𝜃k ∈ 0, ,…, (16.34)
M M
In general, referring to the scheme of Figure 16.8, the filtered signal is given by

x(bb) (t) = u(bb) (t) + 𝑤(bb)

R
(t) (16.35)

where u(bb) is the desired signal at the demodulator input (isolated pulse k)
1 (bb) 1 (bb)
u(bb) (t) = ej𝜓k hTx ∗ g ∗ gA (t − kT) (16.36)
2 Ch 2
as (1∕2)g(bb)
Ch
is the complex envelope of the impulse response of the transmission channel, and 𝑤(bb)
R
is
zero mean additive complex Gaussian noise with variance 𝜎 2 = 2N0 Brn , where
+∞ +∞
Brn = | (f )|2 df = |g (t)|2 dt
∫−∞ | A | ∫−∞ | A |
+∞ +∞
(16.37)
1 | (bb) |2 1 | (bb) |2
= | (f )| df = |g (t)| dt
∫−∞ 4| A | ∫−∞ 4 | A |
is the equivalent bilateral noise bandwidth.
We assume the following configuration: the transmit filter impulse response is given by
√
2Es t − T∕2
hTx (t) = rect (16.38)
T T
the channel introduces only a phase offset

g(bb)
Ch
(t) = 2ej𝜑a 𝛿(t) (16.39)

and the receive filter is matched to the transmit filter,

√
(bb) 2 t − T∕2
gA (t) = rect (16.40)
T T
At the sampling intervals t0 + kT = T + kT, let

𝑤k = 𝑤(bb)
R
(t0 + kT) (16.41)

and √ √
2Es 1 2 √
A= T = Es (16.42)
T 2 T
then
x(bb) (t0 + kT) = Aej(𝜓k +𝜑a ) + 𝑤k (16.43)

with
[ ]
E |𝑤k |2 = 𝜎 2 = 2N0 Brn = N0 (16.44)
16.2. (D)PSK non-coherent receivers 797

since Brn = 1∕2. Moreover, it results

1 (bb)
zk = x (T + kT)x(bb)∗ (T + (k − 1)T)
2
1 [ j(𝜓k +𝜑a ) ][ ]
= Ae + 𝑤k Ae−j(𝜓k−1 +𝜑a ) + 𝑤∗k−1
2
⎡ ⎤ (16.45)
⎢ 𝑤 𝑤 ∗ ⎥
1
= A ⎢ Aej𝜃k + ej(𝜓k +𝜑a ) 𝑤∗k−1 + e−j(𝜓k−1 +𝜑a ) 𝑤k + ⎥
k k−1
2 ⎢ ⏟⏟⏟ A ⎥
⎢desired term ⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏞⏟⎥
⎣ disturbance ⎦

√The desired term is similar to that obtained2in the coherent case, M-phases on a circle of radius A=
Es . The variance of 𝑤k is equal to N0 , and 𝜎I = N0 ∕2. However, even neglecting the term 𝑤k 𝑤∗k−1 ∕A,
if 𝑤k and 𝑤k−1 are statistically independent we have
[ ]
| |2
E |ej(𝜓k +𝜑a ) 𝑤∗k−1 + e−j(𝜓k−1 +𝜑a ) 𝑤k | = 2𝜎 2 = 2N0 (16.46)
| |
There is an asymptotic penalty, that is for Es ∕N0 → ∞, of 3 dB with respect to the coherent receiver
case. Indeed, for a 4-DPSK, a more accurate analysis demonstrates that the penalty is only 2.3 dB for
higher values of Es ∕N0 .

16.2.3 FM discriminator with integrate and dump filter

The scheme of Figure 16.9 makes use of a limiter discriminator (LD), as for frequency modulated (FM)
signals, followed by an integrator filter over a symbol period, or integrate and dump (I&D) filter. This
is limiter discriminator integrator (LDI) scheme. Ideally, the discriminator output provides the instanta-
neous frequency deviation of x, i.e.
d
Δf (t) = Δ𝜑x (t) (16.47)
dt
Then, integrating (16.47) over a symbol period, we have
t0 +kT ( )
Δf (𝜏) d𝜏 = Δ𝜑x (t0 + kT) − Δ𝜑x t0 + (k − 1)T + 2n𝜋 (16.48)
∫t0 +(k−1)T
that coincides with (16.31) taking mod 2𝜋. An implementation of the limiter–discriminator is given in
Figure 16.10, while the baseband equivalent scheme is given in Figure 16.11, which employs the general
relation [ ]
Im ẋ (bb) (t)x(bb)∗ (t)
Δf (t) = (16.49)
2𝜋|x(bb) (t)|2

t t0 + kT
r(t) x(t) FM Δf(t) modulo θk
gA(t) limiter discriminator (.) dτ 2π
t–T

Figure 16.9 FM discriminator and integrate and dump filter.

r(t) x(t) Δf(t)

gA(t) limiter
d envelope
dt detector

Figure 16.10 Implementation of a limiter–discriminator.

798 Chapter 16. Low-complexity demodulators

t0 + kT
r(bb)(t) 1 (bb) x(bb)(t) d ∆f(t) modulo vk
g Im[.] I&D
2 A dt 2π

1
2π |.|2

Figure 16.11 Baseband equivalent scheme of a FM discriminator followed by an integrate and

dump filter.
In conclusion, all three schemes (baseband differential detector, differential detector, and FM discrim-
inator) yield the same output.

16.3 Optimum receivers for signals with random phase

In this section, we ignore inter-symbol interference (ISI) and consider transmit and receiver structures
for some modulation formats.
Let us consider transmission over an ideal additive white Gaussian noise (AWGN) channel of one of
the signals [ ]
sn (t) = Re s(bb)
n (t) e j2𝜋f0 t
, n = 1, 2, … , M, 0 < t < T (16.50)

where s(bb)
n is the complex envelope of sn , relative to the carrier frequency f0 , with support (0, t0 ), t0 ≤ T.
Hence, the received signal is
r(t) = sn (t) + 𝑤(t) (16.51)
with 𝑤 AWGN with power spectral density (PSD) N0 ∕2.
If in (16.50) every signal s(bb)
n has a bandwidth smaller than f0 , then the energy of sn is given by
t0 t0
1 | (bb) |2
En = s2n (t) dt = |s (t)| dt (16.52)
∫0 ∫0 2 |n |
At the receiver, we observe the signal
r(t) = sn (t; 𝜑) + 𝑤(t) (16.53)
where [ ]
sn (t; 𝜑) = Re s(bb)
n (t)e e
j𝜑 j2𝜋f0 t

[ ] (16.54)
= Re s(bb)∗
n (t)e−j𝜑 e−j2𝜋f0 t , n = 1, 2, … , M
In other words, at the receiver we assume the carrier is known, except for a phase 𝜑 that we assume to be
a uniform r.v. in [−𝜋, 𝜋). Receivers, which do not rely on the knowledge of the carrier phase, are called
non-coherent receivers.
We give three examples of signalling schemes that employ non-coherent receivers.

Example 16.3.1 (Non-coherent binary FSK)

For a non coherent binary frequency shift keying (FSK) transmission, the useful part of the received
signals are expressed as (see also Ref. [5]):
s1 (t; 𝜑1 ) = A cos(2𝜋f1 t + 𝜑1 ), 0<t<T
(16.55)
s2 (t; 𝜑2 ) = A cos(2𝜋f2 t + 𝜑2 ), 0<t<T
16.3. Optimum receivers for signals with random phase 799

where, if T is the symbol period and Es the average signalling energy,

√
2Es
A= , 𝜑1 , 𝜑2 ∼ U[−𝜋, 𝜋) (16.56)
T
and
f1 = f0 − fd , f2 = f0 + fd (16.57)

where fd is the frequency deviation with respect to the carrier f0 . We recall that if
1 1
f1 + f2 = k1 (k integer) or else f0 ≫ (16.58)
T 1 T
and if
2 fd T = k (k integer) (16.59)

then s1 (t, 𝜑1 ) and s2 (t, 𝜑2 ) are orthogonal.

The minimum value of fd is given by
1
(fd )min = (16.60)
2T
which is twice the value we find for the coherent demodulation case [5].

Example 16.3.2 (On–off keying)

On–off keying (OOK) is a binary modulation scheme where, for example,

s1 (t; 𝜑) = A cos(2𝜋f0 t + 𝜑), 0<t<T (16.61)

and
s2 (t; 𝜑) = 0 (16.62)
√
where A = 4Es ∕T.

Example 16.3.3 (Double side band (DSB) modulated signalling with random phase)
We consider an M-ary baseband signalling scheme, {s(bb) n (t)}, n = 1, … , M, that is modulated in the
passband by the double sideband technique (see Example 1.5.3 on page 39).
The useful part of the received signals are expressed as

sn (t; 𝜑) = s(bb)
n (t) cos(2𝜋f0 t + 𝜑), n = 1, … , M (16.63)

ML criterion
Given 𝜑 = p, that is for known 𝜑, the maximum-likelihood (ML) criterion to detect the transmitted
signal starts from the following likelihood function [5]
( t0 t0 )
2 1
𝙻n [p] = exp r(t) sn (t; p) dt − s2n (t; p) dt (16.64)
N0 ∫0 N0 ∫0
Using the result
t0
s2n (t; 𝜑) dt = En (16.65)
∫0
we have ( ) ( t0 )
En 2
𝙻n [p] = exp − exp r(t) sn (t; p) dt , n = 1, … , M (16.66)
N0 N0 ∫0
800 Chapter 16. Low-complexity demodulators

Given 𝜑 = p, the ML criterion yields the decision rule

â0 = arg max 𝙻n [p] (16.67)

n

The dependency on the r.v. 𝜑 is removed by taking the expectation of 𝙻n [p] with respect to 𝜑4 :
𝜋
𝙻n = 𝙻n [p] p𝜑 (p) dp
∫−𝜋
E 𝜋 ( [ t0 ]) (16.68)
− Nn 1 2 (bb)∗ −j(p+2𝜋f0 t)
=e 0 exp Re r(t) sn (t) e dt dp
2𝜋 ∫−𝜋 N0 ∫0
using (16.54). We define
0 t
1
Ln = √ r(t)[s(bb)
n (t) e
j2𝜋f0 t ∗
] dt (16.69)
∫
En 0
Introducing the polar notation Ln = |Ln |ej arg Ln , (16.68) becomes
√
E 𝜋
2 En
− Nn 1 Re[Ln e−jp ]
𝙻n = e 0 e N0 dp
2𝜋 ∫−𝜋
𝜋 √ (16.70)
E 2 En
− Nn 1 |L | cos(p−arg Ln )
=e 0 e N0 n dp
2𝜋 ∫−𝜋
We recall that the Bessel functions (4.56) I0 (x) is monotonic increasing for x > 0. Then (16.70)
becomes ( √ )
E
− Nn 2 En
𝙻n = e I0 0 |Ln | , n = 1, … , M (16.71)
N0

Taking the logarithm, we obtain the log-likelihood function

( √ )
2 En E
𝓁n = ln I0 |Ln | − n (16.72)
N0 N0

If the signals have all the same energy, and considering that both ln and I0 are monotonic functions, the
ML decision criterion can be expressed as

â0 = arg max |Ln | (16.73)

n

Implementation of a non-coherent ML receiver

The scheme that implements the criterion (16.73) is illustrated in Figure 16.12 for the case of all En
equal. Polar notation is adopted for the complex envelope:

s(bb) (bb)
n (t) = |sn (t)|e
jΦn (t)
(16.74)

From (16.69), the scheme first determines the real and the imaginary parts of Ln starting from s(bb) n ,
and then determines the square magnitude. Note that the available signal is s(bb)
n (t) e j𝜑0 , where 𝜑 is a
0
constant, rather than s(bb)
n . This however does not modify the magnitude of Ln . As shown in Figure 16.13,
the generic branch of the scheme in Figure 16.12, composed of the I branch and the Q branch, can be
implemented by a complex-valued passband filter (see (16.69)); the bold line denotes a complex-valued
signal.

4
Averaging with respect to the phase 𝜑 cannot be considered for PSK and QAM systems, where information is also
carried by the phase of the signal.
16.3. Optimum receivers for signals with random phase 801

t t0
(.) dτ (.)2
t–t0
s1(t;φ0) =│s1(bb)(t)│cos(2πf0 t + φ0 + Φ1(t)) │L1│
2

π/2
(h)
–s1 (t;φ0) = –│s1(bb)(t)│sin(2πf0 t + φ0 + Φ1(t))
t t0
(.) dτ (.)2
t– t0

r(t) choose a0
the
largest
t t0
(.) dτ (.)2
t –t0
sM(t;φ0) =│sM(bb)(t)│cos(2πf0 t + φ0 + ΦM(t)) │L │2
M

π/2
(h)
–sM (t;φ0) = –│sM(bb)(t)│sin(2πf0 t + φ0 + ΦM(t))
t t0
(.) dτ (.)2
t– t0

Figure 16.12 Non-coherent ML receiver of the type square-law detector .

t0 2
│Ln│
r(t) 2
(bb)*
sn (t0 – t)e j(2πf0 t + φ0) │.│

Figure 16.13 Implementation of a branch of the scheme of Figure 16.12 by a complex-valued

passband matched filter.

Alternatively, the matched filter (MF) can be real valued if it is followed by a phase-splitter. In this
case, the receiver is illustrated in Figure 16.14. For the generic branch, the desired value |Ln | coincides
with the absolute value of the output signal of the phase-splitter at instant t0 ,
| |
|Ln | = |yn(a) (t)| (16.75)
| |t=t0
The cascade of the phase-splitter and the modulo transformation is called envelope detector of the signal
yn (see (1.130) and (1.136)).
A simplification arises if the various signals s(bb) n have a bandwidth B much lower than f0 . In this case,
recalling (1.136), if y(bb)
n is the complex envelope of yn , at the matched filter output the following relation
holds [ ]
yn (t) = Re y(bb)
n (t)e j2𝜋f0 t

( ) (16.76)
| (bb) |
= |yn (t)| cos 2𝜋f0 t + arg y(bb)
n (t)
| |
802 Chapter 16. Low-complexity demodulators

t0
y1(t) │L1│
Re (bb)*
s1 (t0 – t)e j(2πf0 t + φ0) envelope
detector

r(t) ... choose a0

...

...

...

...
the
largest
t0
yM(t) │LM│
(bb)* envelope
Re sM (t0 – t)e j(2πf0 t + φ0)
detector

Figure 16.14 Non-coherent ML receiver of the type envelope detector , using passband matched
filters.

y(t) │y(bb)(t)│
phase splitter │.│

(a)

1. │y(bb)(t)│
y(t)
peak detector LPF

2.
y(t) OUT │y(bb)(t)│
LPF
IN

(b)

Figure 16.15 (a) Ideal implementation of an envelope detector, and (b) two simpler approximate
implementations.

Moreover, from (16.75) and (1.133) we have

| |
|Ln | = |y(bb) (t)| (16.77)
| n |t=t0
Now, if f0 ≫ B, to determine the amplitude |y(bb)
n (t)|, we can use one of the schemes of Figure 16.15.

Example 16.3.4 (Non-coherent binary FSK)

We show in Figure 16.16 two alternative schemes of the ML receiver, where 𝚠T is a rectangular window
(see (1.400)) of length T, for the modulation system considered in Example 16.3.1.

Example 16.3.5 (On–off keying)

We illustrate in Figure 16.17 the receiver for the modulation system of Example 16.3.2 where, recalling
(16.71), we have
N
UTh = √0 I0−1 (eE1 ∕N0 ) (16.78)
2 E1
where
A2 T
E1 = (16.79)
2
16.3. Optimum receivers for signals with random phase 803

T
t
(.) dτ (.)2
t–T
cos(2πf1 t + φ0)

π/2 U1

–sin(2πf1 t + φ0)
T
t
(.) dτ (.)2
t–T + a
r(t) U U > 0 a0 = 1 0
T – U < 0 a0 = 2
t
(.) dτ (.)2
t–T
cos(2πf2 t + φ0)

U2
π/2
–sin(2πf2 t + φ0)
T
t
(.) dτ (.)2
t–T

(a)

T
wT (t)cos(2πf1 t + φ0) envelope
detector
r(t) + U U > 0 a0 = 1 a0
– U < 0 a0 = 2
T
wT (t)cos(2πf2 t + φ0) envelope
detector

(b)

Figure 16.16 Two ML receivers for a non-coherent 2-FSK system. (a) Square-law detector and
(b) envelope detector.

T ^a
r(t) ^
wT (t)cos(2πf0 t + φ0) envelope U U > UTh , a0 = 1 0
detector U < UTh , a^0 = 2

Figure 16.17 Envelope detector receiver for an on–off keying system.

Example 16.3.6 (DSB modulated signalling with random phase)

With reference to the Example 16.3.3, we show in Figure 16.18 the receiver for a baseband M-ary sig-
nalling scheme that is DSB modulated with random phase. Depending upon the signalling type, further
simplifications arise by extracting functions that are common to the different branches.
804 Chapter 16. Low-complexity demodulators

t t0
(.) dτ (.)2
t – t0
│L1│2
(bb)

cos(2πf0 t + φ0)
s1 (t)
t t0
r(t)
(.) dτ (.)2
t – t0

s(bb)
M (t)
choose a0
π/2 the
t0 largest
t
(.) dτ (.)2
t – t0
│LM│2
(bb)
s1 (t)
t0
t
(.) dτ (.)2
t – t0

s(bb)
M (t)

(a)

y1(t) t0 │L1│
(bb)
s1 (t0 – t)cos(2πf0 t + φ0) envelope
detector

r(t) choose a0
the
largest
yM(t) t0 │LM│
s(bb) envelope
M (t0 – t)cos(2πf0 t + φ0)
detector

(b)

Figure 16.18 Two receivers for a DSB modulation system with M-ary signalling and random phase.
(a) Square-law detector and (b) envelope detector.

Error probability for a non-coherent binary FSK system

We now derive the error probability of the system of Example 16.3.4. We assume that s1 is transmitted.
Then,
r(t) = s1 (t) + 𝑤(t) (16.80)

and
Pbit = P[U1 < U2 ∣ s1 ] (16.81)

Equivalently, if we define
√ √
V1 = U1 and V2 = U2 (16.82)
16.3. Optimum receivers for signals with random phase 805

we have
Pbit = P[V1 < V2 ∣ s1 ] (16.83)

Now, recalling assumption (16.59), that is s1 (t, 𝜑1 ) ⟂ s2 (t, 𝜑2 ), we have

( T )2 ( T )2
U2 = r(t) cos(2𝜋f2 t + 𝜑0 ) dt + r(t) sin(2𝜋f2 t + 𝜑0 ) dt
∫0 ∫0 (16.84)
= 𝑤22,c + 𝑤22,s

where
T
𝑤2,c = 𝑤(t) cos(2𝜋f2 t + 𝜑0 ) dt
∫0
T
(16.85)
𝑤2,s = 𝑤(t) sin(2𝜋f2 t + 𝜑0 ) dt
∫0
If we define
T
𝑤1,c = 𝑤(t) cos(2𝜋f1 t + 𝜑0 ) dt
∫0
T
(16.86)
𝑤1,s = 𝑤(t) sin(2𝜋f1 t + 𝜑0 ) dt
∫0
we have
( T )2 ( T )2
U1 = r(t) cos(2𝜋f1 t + 𝜑0 ) dt + r(t) sin(2𝜋f1 t + 𝜑0 ) dt
∫0 ∫0
(16.87)
( )2 ( )2
AT AT
= cos(𝜑0 − 𝜑1 ) + 𝑤1,c + sin(𝜑0 − 𝜑1 ) + 𝑤1,s
2 2
where from (16.56) we also have
√
AT Es T
= (16.88)
2 2
As 𝑤 is a white Gaussian random process with zero mean, 𝑤2,c and 𝑤2,s are two jointly Gaussian r.v.s.
with
E[𝑤2,c ] = E[𝑤2,s ] = 0
N0 T
E[𝑤22,c ] = E[𝑤22,s ] = (16.89)
2 2
T T
N0
E[𝑤2,c 𝑤2,s ] = 𝛿(t1 − t2 ) cos(2𝜋f2 t1 + 𝜑0 ) sin(2𝜋f2 t2 + 𝜑0 ) dt1 dt2 = 0
∫0 ∫0 2
Similar considerations hold for 𝑤1,c and 𝑤1,s .
Therefore V2 , with statistical power 2(N0 T∕4), has a Rayleigh probability density

𝑣2 𝑣 2
− 2
pV2 (𝑣2 ) = e 2(N0 T∕4) 1(𝑣2 ) (16.90)
N0 T∕4
whereas V1 has a Rice probability density function
( )
𝑣1 𝑣1 (AT∕2)
2
(𝑣 +(AT∕2) ) 2
− 1
pV1 (𝑣1 ) = e 2(N0 T∕4) I0 1(𝑣1 ) (16.91)
N0 T∕4 N0 T∕4
806 Chapter 16. Low-complexity demodulators

Consequently, (16.83) becomes5

+∞
Pbit = P[V1 < 𝑣2 ∣ V2 = 𝑣2 ] pV2 (𝑣2 ) d𝑣2
∫0
+∞ ( 𝑣2 )
= pV1 (𝑣1 ) d𝑣1 pV2 (𝑣2 ) d𝑣2 (16.93)
∫0 ∫0
1 − 12 Γ
= e
2
It can be shown that this result is not limited to frequency shift keying (FSK) systems and is valid for
any pair of non-coherent orthogonal signals with energy Es .

Performance comparison of binary systems

The received signals are given by (16.55), where fd satisfies the constraint (16.59). The two signals are
orthogonal (𝜌 = 0); hence from (16.93), we have (Γ = Es ∕N0 )
1 1
FSK(NC): Pbit = e− 2 Γ (16.94)
2
A comparison with a non-coherent binary system with differentially encoded bits, such as DBPSK, is
illustrated in Figure 16.19. The differential receiver for DBPSK directly gives the original uncoded bits,

10−1

10−2

10−3
FSK (ρ = 0) FSK
Pbit

CO NC

10−4
(d.e.)BPSK DBPSK
(ρ = –1)

10−5

10−6
5 6 7 8 9 10 11 12 13 14 15
Γ (dB)

Figure 16.19 Bit error probability as a function of Γ for BPSK and binary FSK systems, with
coherent (CO) and non-coherent (NC) detection. Here 𝜌 denotes the correlation coefficient between
transmit signals.
5
To compute the following integrals, we recall the Weber–Sonine formula:
+∞
2 ∕2) 1 𝛽 2 ∕2𝛼
x e−𝛼(x I0 (𝛽x) dx = e (16.92)
∫0 𝛼
16.4. Frequency-based modulations 807

thus from (16.8) we have

1 −Γ
DBPSK: e Pbit = (16.95)
2
A comparison between (16.94) and (16.95) indicates that DBPSK is better than FSK by about 3 dB in Γ,
for the same Pbit .
The performance of a binary frequency shift keying (BFSK) system and that of a differentially encoded
BPSK system with coherent detection are compared in Figure 16.19. In particular, from Ref. [5] it follows
(√ )
FSK(CO): Pbit = Q Γ (16.96)

Taking into account differential decoding, from (6.84) we have6

(√ )
(d.e.)BPSK: Pbit ≃ 2Q 2Γ (16.97)
We observe that the difference between coherent FSK and non-coherent FSK is less than 2 dB for Pe ≤
10−3 , and becomes less than 1 dB for Pe ≤ 10−5 . We also note that because of the large bandwidth
required, FSK systems with M > 2 are not widely used.

16.4 Frequency-based modulations

16.4.1 Frequency shift keying

The main advantage of FSK modulators consists in generating a signal having a constant envelope,
therefore the distortion introduced by a high-power amplifier (HPA) is usually negligible. However,
they present two drawbacks:
● wider spectrum of the modulated signal as compared to amplitude and/or phase modulated systems,
● complexity of the optimum receiver in the presence of non-ideal channels.
As already expressed in (16.55), a BFSK modulator maps the information bits in frequency deviations
(±fd ) around a carrier with frequency f0 ; the possible transmitted waveforms are then given by
( )
s1 (t) = A cos 2𝜋(f0 − fd )t + 𝜑1
( ) (16.98)
s2 (t) = A cos 2𝜋(f0 + fd )t + 𝜑2 , kT < t < (k + 1)T
√
where, if we denote by Es the average energy of an isolated pulse, A = 2Es ∕T.
Figure 16.20 illustrates the generation of the above signals by two oscillators, with frequency f1 =
f0 − fd and f2 = f0 + fd , selected at instants kT by the variable ak ∈ {1, 2} related to the information bits.
A particular realization of s1 and s2 is shown in Figure 16.21a. The resultant signal is given by
∑
+∞
s(t) = sak (t − kT), ak ∈ {1, 2} (16.99)
k=−∞
A realization of s is shown in Figure 16.21b.
We consider both coherent and non-coherent receivers.
Acos(2πf2 t + φ2)
~
ak
s(t)

~
Acos(2πf1 t + φ1)

Figure 16.20 Generation of a binary (non-coherent) FSK signal by two oscillators.

6
For a more accurate evaluation of the probability of error see footnote 3 on page 792.
808 Chapter 16. Low-complexity demodulators

A A
s1(t)

s2(t)
o o

–A –A
o t T o t T

(a)

ak A 1 2 1 2 2

o
s(t)

–A
o T 2T 3T 4T 5T
t
(b)

Figure 16.21 (a) Binary FSK waveforms for f1 = 2∕T, f2 = 3∕T, 𝜑1 = 𝜋∕2, and 𝜑2 = 𝜋∕4 and
(b) transmitted signal for a particular sequence {ak }.

Coherent demodulator
A coherent demodulator is used when the transmitter provides all M signals with a well-defined phase;
at the receiver, there must be a circuit for the recovery of the phase of the various carriers. In practice,
this coherent receiver is rarely used because of its implementation complexity.
In any case, for orthogonal signals we can adopt the scheme of Figure 16.22 for the binary case and
the bit error probability is given by (16.96).
We note that the signals are orthogonal if
1
(2fd )min = (16.100)
2T

Non-coherent demodulator
The transmitted waveforms can now have a different phase, and in any case unknown to the receiver
(see Example 16.3.1 on page 799). For orthogonal signals, from the general scheme of Figure 16.16, we
16.4. Frequency-based modulations 809

t0 + kT
wT

cos(2πf1 t)
U1
r(t) choose the ak
largest
U2
cos(2πf2 t)
t0 + kT
wT

Figure 16.22 Coherent demodulator for orthogonal binary FSK.

t0 + kT
wT envelope
detection

cos(2πf1 t)
U1
r(t) choose the ak
largest
U2
cos(2πf2 t)

t0 + kT
wT envelope
detection

Figure 16.23 Non-coherent demodulator for orthogonal binary FSK.

give in Figure 16.23 a possible non-coherent demodulator. In this case, the bit error probability is given
by (16.94),
From (16.60), we note that, in the non-coherent case, signals are orthogonal if
1
(2fd )min = (16.101)
T
Therefore, a non-coherent FSK system needs a double frequency deviation and has slightly lower per-
formance (see Figure 16.19) as compared to coherent FSK; however, it does not need to acquire the
carrier phase.

Limiter–discriminator FM demodulator
A simple non-coherent receiver, with good performance, for f1 T ≫ 1, f2 T ≫ 1 and sufficiently high Γ,
is depicted in Figure 16.24.

data
t0 + kT detector
r(t) d 1 ak
gA(t) envelope
limiter
dt detector
–1

Figure 16.24 Limiter–discriminator FSK demodulator.

810 Chapter 16. Low-complexity demodulators

After a passband filter to partially eliminate noise, a classical FM demodulator is used to extract the
instantaneous frequency deviation of r, equal to ±fd . Sampling at instants t0 + kT and using a threshold
detector, we get the detected symbol âk ∈ {−1, 1}.
In general, for 2fd T = 1, the performance of this scheme is very similar to that of a non-coherent
orthogonal FSK.

16.4.2 Minimum-shift keying

We recall the following characteristics of an ideal FSK signal.

● In order to avoid high-frequency components in the PSD of s, the phase of an FSK signal should be
continuous and not as represented in Figure 16.21. For signals (16.98), with 𝜑1 = 𝜑2 , this implies
that it must be fn T = In , n = 1, 2, In an integer.
● To minimize the error probability, it must be sn ⟂ sm , m ≠ n.
● To minimize the required bandwidth, the separation between the various frequencies must be
minimum.

For signals as given by (16.98), choosing |I1 − I2 | = 1 we have f2 = f1 + 1∕T. In this case, s1 and s2
are as shown in Figure 16.25.
Moreover from (16.101) it is easy to prove that the signals are orthogonal. The result is that the
frequency deviation is such that 2fd = |f2 − f1 | = 1∕T, and the carrier frequency is f0 = (f1 + f2 )∕2 =
f1 + 1∕(2T).

0.8

0.6

0.4

0.2
s1(t), s2(t)

−0.2

−0.4

−0.6

−0.8

−1
0 T/2 T
t
Figure 16.25 Waveforms as given by (16.98) with A = 1, f1 = 2∕T, f2 = 3∕T, and 𝜑1 = 𝜑2 = 0.
16.4. Frequency-based modulations 811

In a digital implementation of an FSK system, the following observation is very useful. Independent
of f1 and f2 , a method to obtain a phase continuous FSK signal given by

s(t) = A cos 𝜑(t) (16.102)

where
𝜑(t) = 2𝜋f0 t + ak 2𝜋fd t + 𝜂k , ak ∈ {−1, 1} kT ≤ t < (k + 1)T (16.103)

is to employ a single oscillator, whose phase satisfies the constraint 𝜑((k + 1)T (−) ) = 𝜑((k + 1)T (+) );
thus, it is sufficient that at the beginning of each symbol interval 𝜂k+1 is set equal to

𝜂k+1 = ((ak − ak+1 )2𝜋fd (k + 1)T + 𝜂k ) mod 2𝜋 (16.104)

An alternative method is given by the scheme of Figure 16.26, in which the sequence {ak }, with binary
elements in {−1, 1}, is filtered by g to produce a PAM signal
∑
+∞
xf (t) = ak g(t − kT), ak ∈ {−1, 1} (16.105)
k=−∞

The signal xf is input to a voltage controlled oscillator (VCO), whose output is given by
( t )
s(t) = A cos 2𝜋f0 t + 2𝜋h x (𝜏) d𝜏 (16.106)
∫−∞ f
In (16.106), f0 is the carrier, h = 2fd T is the modulation index and
t
𝜑(t) = 2𝜋f0 t + 2𝜋h xf (𝜏) d𝜏 (16.107)
∫−∞
represents the phase of the modulated signal.
Choosing for g a rectangular pulse
1
g(t) = 𝚠 (t) (16.108)
2T T
we obtain a modulation scheme called continuous phase frequency shift keying (CPFSK). In turn,
CPFSK is a particular case of the continuous phase modulation described in Appendix 16.A.
We note that the area of g is equal to 0.5; in this case, the information is in the instantaneous frequency
of s, in fact
1 d𝜑(t) ha
fs (t) = = f0 + hxf (t) = f0 + k , kT < t < (k + 1)T (16.109)
2𝜋 dt 2T

frequency
modulator
bk ak xf (t) s(t)
BMAP g VCO
Tb T
(a)

phase
modulator
bk ak xf (t) t Δφ(t) s(bb)(t)
BMAP g 2πh AejΔφ(t)
Tb T –∞

(b)

Figure 16.26 CPFSK modulator. (a) Passband model and (b) baseband equivalent model.
812 Chapter 16. Low-complexity demodulators

s1(t) A A

–A
0 T 2T 3T 4T

s(t)
0
t
A
s2(t)

–A –A
0 T 2T 3T 4T 0 T 2T 3T 4T
t t
(a) (b)

A
1/2T
xf (t)

0 0
s(t)

–1/2T
–A
0 T 2T 3T 4T 0 T 2T 3T 4T
t t
(c) (d)

Figure 16.27 Comparison between FSK and MSK signals. (a) FSK waveforms, (b) FSK signal,
(c) signal xf (t), and (d) MSK signal.

The minimum shift keying (MSK) modulation is equivalent to CPFSK with modulation index h = 0.5,
that is for fd = 1∕(4T). Summarizing: an MSK scheme is a BFSK scheme (Tb = T) in which, besides
having fd = 1∕(4T), the modulated signal has continuous phase.
Figure 16.27 illustrates a comparison between an FSK signal with fd = 1∕(4T) as given by (16.98),
{ }
and an MSK signal for a binary sequence ak equal to {−1, 1, 1, −1}. Note that in the MSK case, it is
like having four waveforms s1 , −s1 , s2 , −s2 , each pair related to ak = 1 and ak = −1, respectively. Signal
selection within the pair is done in a way to guarantee the phase continuity of s.

Power spectral density of CPFSK

From the expressions given in [1], the behaviour of the continuous part of the PSD is represented in
Figure 16.28. Note that for higher values of h, a peak will tend to emerge around fd = h∕(2T), showing
the instantaneous frequency of waveforms.
It is seen that phase continuity implies a lower bandwidth, at least if we use the definition of bandwidth
based on a given percentage of signal power.
16.4. Frequency-based modulations 813

0
h
0.5
–10 0.7
Normalized spectral density (dB)

0.8

–20

–30

–40

–50

–60
0 0.5 1 1.5 2 2.5
fT
(a)

0
h
1.2
–10 1.4
Normalized spectral density (dB)

–20

–30

–40

–50

–60
0 0.5 1 1.5 2 2.5
fT
(b)

Figure 16.28 Continuous part of the PSD of a CPFSK signal for five values of the modulation
index. (a) h < 1 and (b) h > 1.
814 Chapter 16. Low-complexity demodulators

Performance
Among the different demodulation schemes listed in Figure 16.29, we show in Figure 16.30 a differential
1BDD non-coherent demodulator. This scheme is based on the fact that the phase deviation of an MSK
signal can vary of ±𝜋∕2 between two suitably instants spaced of Tb . In any case, the performance for an
ideal AWGN channel is that of a DBPSK scheme, but with half the phase variation; hence from (16.8),
with Es that becomes Es ∕2, we get
1 1
Pbit = e− 2 Γ (16.110)
2
Note that this is also the performance of a non-coherent orthogonal BFSK scheme (see (16.94)).
In Figure 16.32, we illustrate a coherent demodulator that at alternate instants on the I branch and on
the Q branch is of the BPSK type. In this case, from (6.84), the error probability for decisions on the
{ }
symbols ck is given by
√
Pbit,Ch = Q( 2Γ) (16.111)
{ }
As it is as if the bits ck were differentially encoded, to obtain the bit error probability for decisions
{ }
on the symbols ak , we use (16.18):
Pbit = 4Pbit,Ch − 8P2bit,Ch + 8P3bit,Ch − 4P4bit,Ch (16.112)
In Figure 16.32, we show error probability curves for various receiver types. From the graph we note
that to obtain an error probability of 10−3 , going from a coherent system to a non-coherent one, it is
necessary to increase the value of Γ as indicated in Table 16.5.

MSK
demodulators

coherent noncoherent
demodulators demodulators

OQPSK CPFSK CPM serial differential limiter and direct

type type type demodulator demodulator discriminator conversion

Figure 16.29 MSK demodulator classification.

data
delay LPF t0 + kTb detector
r(t) π ak
gA Tb – gRc
2

(a)
data
(bb)
t0 + kTb detector
r (t) 1 (bb) 1 Im[ ]
ak
2 gA
Tb
2

(b)

Figure 16.30 Differential (1BDD) non-coherent MSK demodulator. (a) Passband model and
(b) baseband equivalent model.
16.4. Frequency-based modulations 815

data
t0 + k2Tb detector
1
gRc
−1

cos(2πf0t) Tb delay

r(t) P/S a^ k
CR STR +
decoder
π/2
−sin(2π f0t)
1
g Rc
−1
t0 + k2Tb + Tb

Figure 16.31 Coherent MSK demodulator.

100

10–1

10–2

noncoherent
Pbit

10–3 BPSK coherent MSK

MSK

10–4

10–5

10–6
0 2 4 6 8 10 12 14
Γ = Es /N0 (dB)

Figure 16.32 Comparison among various error probabilities.

Table 16.5: Increment of Γ (in dB) for an MSK scheme with

respect to a coherent BPSK demodulator for Pbit = 10−3

coherent BPSK coherent MSK non-coherent MSK

Γ 6.79 +1.1 +4.2

MSK with differential precoding

In the previously considered coherent scheme, the transmitted symbols {ak } are obtained from non-run
zero (NRZ) mapping of bits {bk }. Note that the transformation that maps the bits of {ak } into {ck }
816 Chapter 16. Low-complexity demodulators

corresponds to a differential encoder given by7

ck = ck−1 ak , ak ∈ {−1, 1} (16.113)
with ck ∈ {−1, 1}. If we now use a differential (pre)coding, where ãk ∈ {0, 1}, ãk = bk ⊕ bk−1 and ak =
1 − 2ãk = 1 − 2(bk ⊕ bk−1 ), then from (16.113) with c̃ k ∈ {0, 1} and ck = 1 − 2̃ck , we get
c̃ k = c̃ k−1 ⊕ ãk
= ã0 ⊕ ã1 ⊕ · · · ⊕ ãk (16.114)
= b−1 ⊕ bk
In other words, the symbol ck is directly related to the information bit bk ; the performance loss due to
(16.112) is thus avoided and we obtain
Pbit = Pbit,Ch (16.115)
We emphasize that this differential (pre)coding scheme should be avoided if differential non-coherent
{ } { }
ãk ⊕ b̂ k−1 .
receivers are employed, because one error in âk generates a long error sequence in b̂ k = ̂

16.4.3 Remarks on spectral containment

Modulating both BPSK and quadrature phase-shift keying (QPSK) with hTx given by a rectangular
pulse with duration equal to the symbol period, a comparison between the various PSDs is illustrated in
Figure 16.33. We note that for limited bandwidth channels, it is convenient to choose hTx of the raised
cosine or square root raised cosine type. However, in some radio applications, the choice of a rectangular
pulse may be appropriate, as it generates a signal with lower a peak/average power ratio and therefore
more suitable for being amplified with a power amplifier that operates near saturation.
Two observations on Figure 16.33 follow.
● For the same Tb , the main lobe of QPSK extends up to 1∕T = 0.5∕Tb , whereas that of MSK extends
up to 1∕T = 0.75∕Tb ; thus, the lobe of MSK is 50% wider than that of QPSK, consequently requiring
larger bandwidth.
● At high frequencies, the PSD of MSK decays as 1∕f 4 , whereas the PSD of QPSK decays as 1∕f 2 .

16.5 Gaussian MSK

The Gaussian minimum shift keying (GMSK) is a variation of MSK in which, to reduce the bandwidth
of the modulated signal s, the PAM signal xf is filtered by a Gaussian filter.
Consider the scheme illustrated in Figure 16.34, in which we have the following filters:
● interpolator filter
1
gI (t) = 𝚠 (t) (16.116)
2T T
● shaping filter
K 2 2 2𝜋B
gG (t) = √ e−K t ∕2 with K= √ t (Bt is the 3 dB bandwidth) (16.117)
2𝜋 ln(2)
● overall filter
g(t) = gI ∗ gG (t) (16.118)

7
It corresponds to the exclusive OR (16.11) if ak ∈ {0, 1}.
16.5. Gaussian MSK 817

0
MSK
BPSK
QPSK
–10

–20
PSD (dB)

–30

–40

–50

–60
0 0.5 1 1.5 2 2.5 3
fTb

Figure 16.33 Normalized PSD of the complex envelope of signals obtained by three modulation
schemes.

interpolator filter Gaussian filter FM

ak xI(t) xf (t) s(t)
gI gG VCO
Tb = T

ak xf (t)
g

Figure 16.34 GMSK modulator.

Considering the signals we have:

● transmitted binary symbols

ak ∈ {−1, 1} (16.119)

● interpolated signal
∑
+∞
∑
+∞
1
xI (t) = ak gI (t − kT) = ak 𝚠 (t − kT) (16.120)
k=−∞ k=−∞
2T T
818 Chapter 16. Low-complexity demodulators

● PAM signal
∑
+∞
xf (t) = ak g(t − kT) (16.121)
k=−∞

● modulated signal
( t ) ( )
s(t) = A cos 2𝜋f0 t + 2𝜋h xf (𝜏) d𝜏 = A cos 2𝜋f0 t + Δ𝜑(t) (16.122)
∫−∞
√
where h is the modulation index, nominally equal to 0.5, and A = 2Es ∕T.

From the above expressions, it is clear that the GMSK signal is a FM signal with phase deviation
t
Δ𝜑(t) = 𝜋 ∫−∞ xf (𝜏) d𝜏.
An important parameter is the 3dB bandwidth, Bt , of the Gaussian filter. However, a reduction in
Bt , useful in making prefiltering more selective, corresponds to a broadening of the PAM pulse with a
consequent increase in the intersymbol interference, as can be noted in the plots of Figure 16.35. Thus,
a trade-off between the two requirements is necessary.
The product Bt T was chosen equal to 0.3 in the Global System for Mobile Communications (GSM)
standard, and equal to 0.5 in the Digital Enhanced Cordless Telecommunications (DECT) standard (see
Chapter 18). The case Bt T = ∞, i.e. without the Gaussian filter, corresponds to MSK.
Analysing g in the frequency domain, we have

g(t) = gI ∗ gG (t) ←−→ G(f ) = GI (f ) ⋅ GG (f ) (16.123)

0.6

0.5

0.4
T g(t)

0.3
Bt T = ∞
Bt T = 0.5
0.2
Bt T = 0.3
Bt T = 0.1
0.1

0
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3
t/T

Figure 16.35 Overall pulse g(t) = gI ∗ gG (t), with amplitude normalized to 1∕T, for various values of
the product Bt T.
16.5. Gaussian MSK 819

Bt T = 0.3
(a) 4

Δ φ (t) 2

−2

−4
2 4 6 8 10 12 14 16

Bt T = ∞
(b) 4

2
Δ φ (t)

−2

−4
2 4 6 8 10 12 14 16
t/T

Figure 16.36 Phase deviation Δ𝜑 of (a) GMSK signal for Bt T = 0.3, and (b) MSK signal.

As
1
GI (f ) = sinc(fT)e−j𝜋fT
2 (16.124)
2 2
GG (f ) = e−2𝜋 (f ∕K)
it follows that
1 2 2
sinc(fT)e−2𝜋 (f ∕K) e−j𝜋fT
G(f ) = (16.125)
2
In Figure 16.36, the behaviour of the phase deviation of a GMSK signal with Bt T = 0.3 is compared with
the phase deviation of an MSK signal; note that in both cases the phase is continuous, but for GMSK
we get a smoother curve, without discontinuities in the slope.
Possible trajectories of the phase deviation for Bt T = 0.3 and Bt T = ∞ are illustrated in Figure 16.37.

PSD of GMSK
GMSK is a particular case of phase modulation which performs a non-linear transformation on the
message, thus, it is very difficult to evaluate analytically the PSD of a GMSK signal. Hence, we resort to
a discrete time spectral estimate, for example the Welch method (see Section 1.9). The estimate is made
with reference to the baseband equivalent of a GMSK signal, that is using the complex envelope of the
modulated signal given by s(bb) (t) = AejΔ𝜑(t) .
The result of the spectral estimate is illustrated in Figure 16.38. Note that the central lobe has an
extension up to 0.75∕T; therefore, the sampling frequency, FQ , to simulate the baseband equivalent
scheme of Figure 16.34 may be chosen equal to FQ = 4∕T or FQ = 8∕T.

16.5.1 Implementation of a GMSK scheme

For the scheme of Figure 16.34, three possible configurations are given, depending on the position of
the digital-to-analog converter (DAC).
820 Chapter 16. Low-complexity demodulators

0.8

0.6

0.4

0.2
Δ φ (t)/π

–0.2

–0.4

–0.6

–0.8

–1
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
t/T

Figure 16.37 Trajectories of the phase deviation of a GMSK signal for Bt T = 0.3 (solid line) and
Bt T = ∞ (dotted line).

10
BtT = 0.3
0
BtT = 0.5
−10 BtT = 1.0
BtT = ∞
−20
PSD (dB)

−30

−40

−50

−60

−70

−80
0 0.5 1 1.5 2 2.5 3 3.5 4
fT

Figure 16.38 Estimate of the PSD of a GMSK signal for various values of Bt T.
16.5. Gaussian MSK 821

Configuration I
The first configuration, illustrated in Figure 16.39, is in the analog domain; in particular, an analog
low-pass Gaussian filter is employed. As shown in [6], it is possible to implement a good approximation
of the Gaussian filter by resorting to simple devices, such as lattice filters. However, the weak point of this
scheme is represented by the VCO, because in an open loop, the voltage/frequency VCO characteristic
is non-linear, and as a consequence, the modulation index can vary even of a factor 10 in the considered
frequency range.

Configuration II
The second configuration is represented in Figure 16.40. The digital filter g(nTQ ) that approximates
the analog filter g is designed by the window method [[7], p. 444]. For an oversampling factor of Q0 = 8,
letting TQ = T∕Q0 , we consider four filters that are obtained by windowing the pulse g to intervals (0, T),
(−T∕2, 3T∕2), (−T, 2T), and (−3T∕2, 5T∕2), respectively; the coefficients of the last filter are listed
in Table 16.6, using the fact that g(nTQ ) has even symmetry with respect to the peak at 4TQ = T∕2.

ak xI (t) xf (t) s(t)

DAC gG VCO
T

Figure 16.39 GMSK modulator: configuration I.

ak xn s(t)
g DAC VCO
T TQ

Figure 16.40 GMSK modulator: configuration II.

Table 16.6: Digital filter

coefficients obtained
by windowing g; TQ = T∕8

g(nTQ ) value
g(4TQ ) 0.371 19
g(5TQ ) 0.361 77
g(6TQ ) 0.334 78
g(7TQ ) 0.293 81
g(8TQ ) 0.244 11
g(9TQ ) 0.191 58
g(10TQ ) 0.141 68
g(11TQ ) 0.098 50
g(12TQ ) 0.064 23
g(13TQ ) 0.039 21
g(14TQ ) 0.022 36
g(15TQ ) 0.011 89
g(16TQ ) 0.005 89
g(17TQ ) 0.002 71
g(18TQ ) 0.001 16
g(19TQ ) 0.000 46
822 Chapter 16. Low-complexity demodulators

fT

Figure 16.41 Frequency responses of g and g(nTQ ), for TQ = T∕8, and various lengths of the FIR
filters.

A comparison among the frequency responses of the four discrete-time filters and the continuous-time
filter is illustrated in Figure 16.41. For a good approximation to the analog filter, the possible choices
are limited to the two finite-impulse-response (FIR) filters with 23 and 31 coefficients, with support
(−T, 2T) and (−3T∕2, 5T∕2), respectively. From now on, we will refer to the filter with 31 coefficients.
We note from Figure 16.35 that, for Bt T ≥ 0.3, most of the pulse energy is contained within the
interval (−T, 2T); therefore, the effect of interference does not extend over more than three symbol peri-
ods. With reference to Figure 16.40, the filter g is an interpolator filter from T to T∕Q0 , that can be effi-
ciently implemented by using the polyphase representation of the impulse response (see Appendix 1.A).
Then, recognizing that xkQ0 +𝓁 = f (𝓁) (ak+2 , ak+1 , ak , ak−1 ), the input–output relation can be memorized
in a RAM or look up table, and let the input vector [ak+2 , ak+1 , ak , ak−1 ] address one of the possible
24 = 16 output values; this must be repeated for every phase 𝓁 ∈ {0, 1, … , Q0 − 1}. Therefore, there
are Q0 RAMs, each with 16 memory locations.

Configuration III
The weak point of the previous scheme is again represented by the analog VCO; thus, it is convenient
to partially implement in the digital domain also the frequency modulation stage.
{ }
Real-valued scheme The real-valued scheme is illustrated in Figure 16.42. The samples un are
given by ( )
un = u(nTQ ) = A cos 2𝜋f1 nTQ + Δ𝜑(nTQ ) (16.126)

ak xn digital un u(t) RF s(t)

g DAC
T TQ VCO TQ stage

Figure 16.42 GMSK modulator: configuration III.

16.5. Gaussian MSK 823

where f1 is an intermediate frequency smaller than the sampling rate,

N1 1
f1 = (16.127)
N2 TQ
where N1 and N2 are relatively prime numbers, with N1 < N2 . Thus f1 TQ = N1 ∕N2 , and
( )
N
un = A cos 2𝜋 1 n + Δ𝜑n = A cos 𝜑n (16.128)
N2
where
nTQ ∑
n
Δ𝜑n = Δ𝜑(nTQ ) = 𝜋 x(𝜏) d𝜏 ≃ 𝜋 TQ x(iTQ ) (16.129)
∫−∞ i=−∞
More simply, let xn = xf (nTQ ) and Xn = Xn−1 + xn ; then it follows that Δ𝜑n = 𝜋TQ Xn . Therefore in
(16.128), 𝜑n , with carrier frequency f1 , becomes
(f ) N1 (f1 ) N
𝜑n 1 = 2𝜋 n + 𝜋TQ Xn = 𝜑n−1 + 2𝜋 1 + 𝜋TQ xn (16.130)
N2 N2
(f )
that is the value 𝜑n 1 is obtained by suitably scaling the accumulated values of xn . To obtain un , we map
(f )
the value of 𝜑n 1 into the memory address of a RAM, which contains values of the cosine function (see
Figure 16.43).
Obviously, the size of the RAM depends on the accuracy with which un and 𝜑n are quantized.8 We
note that u(nTQ ) is a real-valued passband signal, with PSD centred around the frequency f1 ; the choice
of f1 is constrained by the bandwidth of the signal u(nTQ ), equal to about 1.5∕T, and also by the sampling
period, chosen in this example equal to T∕8; then, it must be
31 31 4
− + f1 > 0, + f1 < (16.131)
4T 4T T
or 3∕(4T) < f1 < 13∕(4T). A possible choice is f1 = 1∕(4TQ ) assuming N1 = 1 and N2 = 4. With this
choice, we have a image spacing/signal bandwidth ratio equal to 4/3. Moreover, cos 𝜑n = cos(2(𝜋∕4)n +
Δ𝜑n ) becomes cos((𝜋∕2)n + Δ𝜑n ), which in turn is equal to ± cos(Δ𝜑n ) for n even and ± sin(Δ𝜑n ) for
n odd. Therefore, the scheme of Figure 16.42 can be further simplified.

Complex-valued scheme Instead of digitally shifting the signal to an intermediate frequency, it is

possible to process it at baseband, thus simplifying the implementation of the DAC equalizer filter.
Consider the scheme of Figure 16.44, where at the output of the exponential block we have the sam-
pled signal s(bb) (nTQ ) = AejΔ𝜑(nTQ ) = s(bb)

(nTQ ) + js(bb)

(nTQ ), where s(bb)

and s(bb)

are the in phase and
quadrature components. Then, we get
s(bb)

(nTQ ) = A cos(Δ𝜑n )
( ) (16.132)
𝜋
s(bb)

(nTQ ) = A sin(Δ𝜑n ) = A cos 2
− Δ𝜑n

(f )
xn φn1 un
ACC RAM
πTQ
N1
2π
N2

Figure 16.43 Digital implementation of the VCO.

8
To avoid quantization effects, the number of bits used to represent the accumulated values is usually much larger than
the number of bits used to represent 𝜑n . In practice, 𝜑n coincides with the most significant bits of the accumulated
values.
824 Chapter 16. Low-complexity demodulators

(bb) (bb)
sI (nTQ) sI (t)
DAC
cos(2πf0t)

xn Δφn s(t)
ACC Ae
jΔφn
π
πTQ
2
–sin(2πf0t)

DAC
(bb) (bb)
sQ (nTQ) sQ (t)

Figure 16.44 GMSK modulator with a complex-valued digital VCO.

Once the two components have been interpolated by the two DACs, the signal s can be reconstructed as
s(t) = s(bb)

(t) cos(2𝜋f0 t) − s(bb)

(t) sin(2𝜋f0 t) (16.133)
With respect to the real-valued scheme, we still have a RAM which stores the values of the cosine
function, but two DACs are now required.

16.5.2 Linear approximation of a GMSK signal

According to Laurent [8], if Bt T ≥ 0.3, a GMSK signal can be approximated by a QAM signal given by
∑
+∞ 𝜋 ∑k
s(bb) (t) = ej 2 i=−∞ ai
hTx (t − kTb )
k=−∞
(16.134)
∑
+∞
= jk+1 ck hTx (t − kTb ), ck = ck−1 ak
k=−∞

where hTx is a suitable real-valued pulse that depends on the parameter Bt T and has a support equal to
(L + 1)Tb , if LTb is the support of g. For example, we show in Figure 16.45 the plot of hTx for a GMSK
signal with Bt T = 0.3.
The linearization of s(bb) , that leads to interpreting GMSK as a QAM extension of MSK with a different
transmit pulse, is very useful for the design of the optimum receiver, which is the same as for QAM
systems. Figure 16.46 illustrates the linear approximation of the GMSK model.
{ }
As for MSK, also for GMSK it is useful to differentially (pre)code the data ak if a coherent demod-
ulator is employed.

Performance of GMSK
We consider the performance of a GMSK system for an ideal AWGN channel, and compare it with the
that of 𝜋∕4-DQPSK, where the transmitted symbol is ak = ej𝜓k , with
𝜓k = 𝜓k−1 + 𝜃k (16.135)
where now 𝜃k ∈ {𝜋∕4, 3𝜋∕4, 5𝜋∕4, 7𝜋∕4}. Note that for k even, 𝜓k assumes values in {0, 𝜋∕2, 𝜋, 3𝜋∕2},
and for k odd 𝜓k ∈ {𝜋∕4, 3𝜋∕4, 5𝜋∕4, 7𝜋∕4}. Phase variations between two consecutive instants are
±𝜋∕4 and ±3𝜋∕4, with a good peak/average power ratio of the modulated signal.
16.5. Gaussian MSK 825

1
0.9
0.8
0.7
hTx(t) 0.6
0.5
0.4
0.3
0.2
0.1
0
−1 0 1 2 3 4 5
t/Tb

Figure 16.45 Pulse hTx for a GMSK signal with Bt T = 0.3.

j k+1

ak ck ck j k+1 s(bb)(t)
ck = ck–1 ak hTx

Figure 16.46 Linear approximation of a GMSK signal. hTx is a suitable pulse which depends on the
value of Bt T.

Coherent demodulator Assuming a coherent receiver, as illustrated in Figure 16.31, performance of

the optimum receiver according to the maximum-a posteriori (MAP) criterion, evaluated on the basis
of the minimum distance of the received signals, is approximated by the following relation [9]
√
Pbit,Ch = Q( cΓ) (16.136)

where the coefficient c assumes the values given in Table 16.7 for four modulation systems. The plots
of Pbit,Ch for the various cases are illustrated in Figure 16.48.
{ }
As usual, if the data ak are differentially (pre)coded, Pbit = Pbit,Ch holds, otherwise the relation
(16.112) holds. From now on we assume that a differential precoder is employed in the presence of
coherent demodulation.

Table 16.7: Values of coefficient c as a

function of the modulation system

Modulation system c
MSK 2.0
GMSK,Bt T = 0.5 1.93
GMSK,Bt T = 0.3 1.78
𝜋∕4-DQPSK, ∀𝜌 1.0
826 Chapter 16. Low-complexity demodulators

100
MSK BtT = +∞
GMSK BtT = 0.5
10−1
GMSK BtT = 0.3
π/4 − DQPSK
10−2
Pbit,Ch

10−3

10−4

10−5

10−6
0 2 4 6 8 10 12 14
Γ (dB)

Figure 16.47 Pbit,Ch as a function of Γ for the four modulation systems of Table 16.7.

10–1
TD
VA

10–2
Pbit

10–3

10–4

0 1 2 3 4 5 6 7 8 9
Γ (dB)

Figure 16.48 Pbit as a function of Γ for a coherently demodulated GMSK (Bt T = 0.3), for an ideal
channel and Gaussian receive filter with Br T = 0.3. Two detectors are compared: (1) four-state VA
and (2) threshold detector.
16.5. Gaussian MSK 827

For the case Bt T = 0.3, and for an ideal AWGN channel, in Figure 16.48 we also give the per-
formance obtained for a receive filter gRc of the Gaussian type [10, 11], whose impulse response is
given in (16.117), where the 3 dB bandwidth is now denoted by Br . Clearly, the optimum value of
Br T depends on the modulator type and in particular on Bt T. System performance is evaluated using
a 4-state Viterbi algorithm (VA) or a threshold detector (TD). The VA uses an estimated overall
system impulse response obtained by the linear approximation of GMSK. The Gaussian receive
filter is characterized by Br T = 0.3, chosen for best performance. We observe that the VA gains
a fraction of dB as compared to the TD; furthermore, the performance is slightly better than the
approximation (16.136).

Non-coherent demodulator (1BDD) For a non-coherent receiver, as illustrated in Figure 16.30, with-
out including the receive filter gA , we illustrate in Figure 16.49 the eye diagram at the decision point of
a GMSK system, for three values of Bt T.
We note that, for decreasing values of Bt T, the system exhibits an increasing level of ISI, and the eye
tends to shut; including the receive filter gA with finite bandwidth, this phenomenon is further empha-
sized.
Simulation results obtained by considering a receive Gaussian filter gA , and a Gaussian base-
band equivalent system, whose bandwidth is optimized for each different modulation, are shown in
Figure 16.50.

(a) (b) BtT (c) BtT

t/T t/T t/T

Figure 16.49 Eye diagram at the decision point of the 1BDD for a GMSK system, for an ideal
channel and without the filter gA : (a) Bt T = +∞, (b) Bt T = 0.5, (c) Bt T = 0.3.
828 Chapter 16. Low-complexity demodulators

100

10–1

10–2
Pbit

10–3

10–4

MSK; BrT = 0.5

10–5 GMSK BtT = 0.5; BrT = 0.625
GMSK BtT = 0.3; BrT = 0.625
π/4-DQPSK ρ = 0.475; BrT = 0.5

10–6
0 5 10 15 20
Γ (dB)

Figure 16.50 Pbit as a function of Γ obtained with the 1BDD for GMSK, for an ideal channel and
Gaussian receive filter having a normalized bandwidth Br T.

The sensitivity of the 1BDD to the parameter Bt T of GMSK is higher than that of a coherent demod-
ulator. With decreasing Bt T, we observe a considerable worsening of performance; in fact, to achieve
a Pbit = 10−3 the GMSK with Bt T = 0.5 requires a signal-to-noise ratio Γ that is 2.5 dB higher with
respect to MSK, whereas GMSK with Bt T = 0.3 requires an increment of 7.8 dB. In Figure 16.50, we
also show for comparison purposes the performance of 𝜋∕4-DQPSK with a receive Gaussian filter.
The performance of another widely used demodulator, LDI (see Section 16.2.3),is quite similar to that
of 1BDD, showing a substantial equivalence between the two non-coherent demodulation techniques
applied to GMSK and 𝜋∕4-DQPSK.

Comparison Always for an ideal AWGN channel, a comparison among the various modulators and
demodulators is given in Table 16.8. As a first observation, we note that a coherent receiver with
an optimized Gaussian receive filter provides the same performance, evaluated by the approximate
relation (16.136), of the MAP criterion. Furthermore, we note that for GMSK with Bt T ≤ 0.5,
because of strong ISI, a differential receiver undergoes a substantial penalty in terms of Γ to achieve
a given Pbit with respect to a coherent receiver; this effect is mitigated by cancelling ISI by suitable
equalizers [12].
Another method is to detect the signal in the presence of ISI, in part due to the channel and in part
to the differential receiver, by the VA. Substantial improvements with respect to the simple threshold
detector are obtained, as shown in [13].
In the previous comparison between amplitude modulation (𝜋∕4-DQPSK) and phase modulation
(GMSK) schemes, we did not take into account the non-linearity introduced by the power amplifier
(see Chapter 4), which leads to (1) signal distortion and (2) spectral spreading that creates interference
in adjacent channels. Usually, the latter effect is dominant and is controlled using a HPA with a back-off
that can be even of several dB. In some cases, signal predistortion before the HPA allows a decrease of
the output back-off (OBO).
16.5. Gaussian MSK 829

100

90 VA(32)

80 MF + DFE(13,7)

70
Pbit cdf

60

50

VA(32)
40 MF + DFE(13,7)

30 VA(32): Γ = 15 dB
MF + DFE(13,7): Γ = 15 dB
VA(32): Γ = 10 dB
20
MF + DFE(13,7): Γ = 10 dB

10–3 10–2 10–1

Pbit

Figure 16.51 Comparison between Viterbi algorithm and DFE preceded by a MF for the multipath
channel EQ6, in terms of bit error rate (BER) cdf.

Table 16.8:Required values of Γ, in dB, to achieve a Pbit = 10−3 for various modulation
and demodulation schemes

Demodulation
Modulation coherent coherent differential
(MAP) (gA Gauss. + TD) (gA Gauss. + 1BDD)
𝜋∕4-DQPSK or QPSK (𝜌 = 0.3) 9.8 9.8 (𝜌 = 0.3) 12.5 (Br T = 0.5)
MSK 6.8 6.8 (Br T = 0.25) 10.3 (Br T = 0.5)
GMSK (Bt T = 0.5) 6.9 6.9 (Br T = 0.3) 12.8 (Br T = 0.625)
GMSK (Bt T = 0.3) 7.3 7.3 (Br T = 0.3) 18.1 (Br T = 0.625)

Overall, the best system is the one that achieves, for the same Pbit , the smaller value of
(Γ)dB + (OBO)dB (16.137)
In other words (16.137), for the same Pbit , additive channel noise and transmit HPA, selects the system
for which the transmitted signal has the lowest power. Obviously in (16.137), Γ depends on the OBO;
at high frequencies, where the HPA usually introduces large levels of distortion, the OBO for a linear
modulation scheme may be so large that a phase modulation scheme may be the best solution in terms
of (16.137).

Performance in the presence of multipath

We conclude this section giving the performance in terms of Pbit cumulative distribution function
(cdf) (see Appendix 7.A) of a GMSK scheme with Bt T = 0.3 for transmission over channels in the
830 Chapter 16. Low-complexity demodulators

Table 16.9: Power delay profiles for the analysed channels

Coefficient Relative Power delay profile

delay EQ HT UA RA
0 0 1/6 0.02 0.005 66 0.000 86
1 T 1/6 0.75 0.017 25 0.997 62
2 2T 1/6 0.08 0.802 56 0.001 09
3 3T 1/6 0.02 0.152 64 0.000 26
4 4T 1/6 0.01 0.016 68 0.000 11
5 5T 1/6 0.12 0.005 21 0.000 06

presence of frequency selective Rayleigh fading (see Chapter 4). The receive filter is Gaussian with
Br T = 0.3, implemented as a FIR filter with 24 T∕8-spaced coefficients; the detector is a 32-state VA
or a decision-feedback equalizer (DFE) with 13 coefficients of the T-spaced feedforward (FF) filter and
seven coefficients of the feedback (FB) filter. For the DFE, a substantial performance improvement is
observed by placing before the FF filter a matched filter (MF) that, to save computational complexity,
operates at T; the improvement is mainly due to a better acquisition of the timing phase.
The channel model is also obtained by considering a T-spaced impulse response; in particular, the
performance is evaluated for the four models given in Table 16.9: equal gain (EQ), hilly terrain (HT),
urban area (UA), and rural area (RA). The difference in performance between the two receivers is
higher for the EQ channel; this is the case shown in Figure 16.51 for two values of the signal-to-noise
ratio Γ at the receiver.

Bibliography

[1] Benedetto, S. and Biglieri, E. (1999). Principles of Digital Transmission with Wireless Applications.
New York, NY: Kluwer Academic Publishers.
[2] Proakis, J.G. (1995). Digital Communications, 3e. New York, NY: McGraw-Hill.
[3] Divsalar, D., Simon, M.K., and Shahshahani, M. (1990). The performance of trellis-coded MDPSK
with multiple symbol detection. IEEE Transactions on Communications 38: 1391–1403.
[4] European DVB Project (1993). Digital broadcasting system for television, sound and data services;
framing structure, channel coding and modulation for 11/12 GHz satellite services.
[5] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[6] Razavi, B. (1997). RF Microelectronics. Englewood Cliffs, NJ: Prentice-Hall.
[7] Oppenheim, A.V. and Schafer, R.W. (1989). Discrete-Time Signal Processing. Englewood Cliffs,
NJ: Prentice-Hall.
[8] Laurent, P. (1986). Exact and approximate construction of digital phase modulations by superposi-
tion of amplitude modulated pulses (AMP). IEEE Transactions on Communications 34: 150–160.
[9] Ohmori, S., Wakana, H., and Kawase, S. (1998). Digital Communications Technologies. Boston,
MA: Artech House.
[10] Murota, K. and Hirade, K.H. (1991). GMSK modulation for digital mobile radio telephony. IEEE
Transactions on Communications 29: 1045.
[11] Simon, M.K. and Wang, C.C. (1984). Differential detection of Gaussian MSK in a mobile radio
environment. IEEE Transactions on Vehicular Technology 33: 311–312.
[12] Benvenuto, N., Bisaglia, P., Salloum, A., and Tomba, L. (2000). Worst case equalizer for
non-coherent HIPERLAN receivers. IEEE Transactions on Communications 48: 28–36.
[13] Benvenuto, N., Bisaglia, P., and Jones, A.E. (1999). Complex noncoherent receivers for GMSK sig-
nals. IEEE Journal on Selected Areas in Communications 17: 1876–1885.
16.A. Continuous phase modulation 831

Appendix 16.A Continuous phase modulation

A signal with constant envelope can be defined by its passband version as
√
2Es ( )
s(t) = cos 2𝜋f0 t + Δ𝜑(t, a) (16.138)
T
where Es is the energy per symbol, T the symbol period, f0 the carrier frequency, and a denotes the
{ }
symbol message ak at the modulator input. For a continuous phase modulation (CPM) scheme, the
phase deviation Δ𝜑(t, a) can be expressed as
t
Δ𝜑(t, a) = 2𝜋h xf (𝜏) d𝜏 (16.139)
∫−∞
with
∑
+∞
xf (t) = ak g(t − kT) (16.140)
k=−∞

where g is called instantaneous frequency pulse. In general, the pulse g satisfies the following properties:

limited duration g(t) = 0 for t < 0 and t > LT (16.141)

symmetry g(t) = g(LT − t) (16.142)
LT
1
normalization g(𝜏) d𝜏 = (16.143)
∫0 2

Alternative definition of CPM

From (16.140) and (16.139), we can redefine the phase deviation as follows

t ∑
k
∑
k
t−iT
Δ𝜑(t, a) = 2𝜋h ∫−∞ ai g(𝜏 − iT) d𝜏 = 2𝜋h ai ∫−∞ g(𝜏) d𝜏,
i=−∞ i=−∞ (16.144)
kT ≤ t < (k + 1)T
or
∑
k
Δ𝜑(t, a) = 2𝜋h q(t − iT), kT ≤ t < (k + 1)T (16.145)
i=−∞

with
t
q(t) = g(𝜏) d𝜏 (16.146)
∫−∞
The pulse q is called phase response pulse and represents the most important part of the CPM signal
because it indicates to what extent each information symbol contributes to the overall phase deviation. In
Figure 16.52, the phase response pulse q is plotted for PSK, CPFSK, and BFSK. In general, the maximum
value of the slope of the pulse q is related to the width of the main lobe of the PSD of the modulated
signal s, and the number of continuous derivatives of q influences the shape of the secondary lobes.
{ }
In general, information symbols ak belong to an M-ary alphabet , that for M even is given
by {±1, ±3, … , ±(M − 1)}. The constant h is called modulation index and determines, together with
the dimension of the alphabet, the maximum phase variation in a symbol period, equal to (M − 1)h𝜋.
By changing q (or g), h, and M, we can generate several continuous phase modulation schemes. The
modulation index h is always given by the ratio of two integers, h = 𝓁∕p, because this implies that the
phase deviation, evaluated modulo 2𝜋, assumes values in a finite alphabet. In fact, we can write
∑
k
Δ𝜑(t, a) = 2𝜋h ai q(t − iT) + 𝜓k−L , kT ≤ t < (k + 1)T (16.147)
i=k−L+1
832 Chapter 16. Low-complexity demodulators

(a)
1/2 PSK

t
T 2T
(b)
1/2 CPFSK

t
T 2T
(c)
1/2 BFSK

t
T 2T

Figure 16.52 Three examples of phase response pulses for CPM: (a) PSK, (b) CPFSK, (c) BFSK.

with phase state [ ]

𝓁 ∑
k−L
𝜓k−L = 𝜋 a (16.148)
p i=−∞ i
mod 2𝜋
The phase state represents the overall contribution given by the symbols … , a−1 , a0 , a1 , … , ak−L to the
phase duration in the interval [kT, (k + 1)T), and can only assume 2p distinct values. The first term
in (16.145) is called corrective state and, because it depends on L symbols ak−L+1 , … , ak , at a certain
instant t it can only assume M L−1 distinct values. The phase deviation is therefore characterized by a
total number of values equal to 2pM L−1 .
CPM schemes with L = 1, i.e. CPFSK, are called full response schemes and have a reduced complex-
ity with a number of states equal to 2p; schemes with L ≠ 1 are instead called partial response schemes.
Because of the memory in the modulation process a partial response scheme allows a trade-off between
the error probability at the receiver and the shaping of the modulated-signal PSD. However, this advan-
tage is obtained at the expense of a greater complexity of the receiver. For this reason, the modulation
index is usually a simple rational number as 1, 1∕2, 1∕4, and 1∕8.

Advantages of CPM
The advantage of the continuous phase modulation technique derives first of all from the constant enve-
lope property of the CPM signal; in fact a signal with a constant envelope allows very efficient power
amplifiers. In the case of linear modulation techniques, as QAM or orthogonal frequency division mul-
tiplexing (OFDM), it is necessary to compensate for the non-linearity of the amplifier by predistortion
or to decrease the average power in order to work in linear conditions.
Before the introduction of trellis code modulation (TCM), it was believed that source and/or channel
coding would improve performance only at the expense of a loss in transmission efficiency, and hence
would require a larger bandwidth; CPM provides both good performance and highly efficient transmis-
sion. However, one of the drawbacks of the CPM is the implementation complexity of the optimum
receiver, especially in the presence of dispersive channels.
Chapter 17

Applications of interference cancellation

The algorithms and structures discussed in this chapter can be applied to both wired and wireless
systems, even though transmission systems over twisted-pair cables will be considered to describe
examples of applications. Full-duplex data transmission over a single twisted-pair cable permits the
simultaneous flow of information in two directions using the same frequency band. Examples of appli-
cations of this technique are found in digital communications systems that operate over the telephone
network. In a digital subscriber loop, at each end of the full-duplex link, a circuit called hybrid separates
the two directions of transmission. To avoid signal reflections at the near and far-end hybrid, a precise
knowledge of the line impedance would be required. As the line impedance depends on line parameters
that, in general, are not exactly known, an attenuated and distorted replica of the transmit signal leaks
to the receiver input as an echo signal. Data-driven adaptive echo cancellation mitigates the effects of
impedance mismatch.
A similar problem is caused by crosstalk in transmission systems over voice-grade unshielded
twisted-pair (UTP) cables for local-area network applications, where multipair cables are used to physi-
cally separate the two directions of transmission. Crosstalk is a statistical phenomenon due to randomly
varying differential capacitive and inductive coupling between adjacent two-wire transmission lines
(see Ref. [1]). At the rates of several megabit-per-second that are usually considered for local-area
network applications, near-end crosstalk (NEXT) represents the dominant disturbance; hence adaptive
NEXT cancellation must be performed to ensure reliable communications.
A different problem shows up in frequency-division duplexing (FDD) transmission, where different
frequency bands are used in the two directions of transmission. In such a case, far-end crosstalk (FEXT)
interference is dominant. This situation occurs, for instance, in very high speed digital subscriber line
(VDSL) systems, where multiple users are connected to a central station via UTPs located in the same
cable binder.
In voiceband data modems, the model for the echo channel is considerably different from the echo
channel model adopted for baseband transmission. The transmitted signal is a passband quadrature
amplitude modulation (QAM) signal, and the far-end echo may exhibit significant carrier-phase jitter
and carrier-frequency shift, which are caused by signal processing at intermediate points in the tele-
phone network. Therefore, a digital adaptive echo canceller for voiceband modems needs to embody
algorithms that account for the presence of such additional impairments.
In the first three sections of this chapter,1 we describe the echo channel models and adaptive echo
canceller structures for various digital communications systems, which are classified according to
1
The material presented in Sections 17.1–17.3 is reproduced with permission from Cherubini [2].

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
834 Chapter 17. Applications of interference cancellation

the employed modulation techniques. We also address the trade-offs between complexity, speed of
adaptation, and accuracy of cancellation in adaptive echo cancellers. In Section 17.4, we address the
problem of FEXT interference cancellation for upstream transmission in a VDSL system. The system is
modelled as a multi-input multi-output (MIMO) system, to which detection techniques of Section 9.2.3
can be applied.

17.1 Echo and near-end crosstalk cancellation for PAM systems

The model of a full-duplex baseband pulse amplitude modulation (PAM) data transmission system
employing adaptive echo cancellation is shown in Figure 17.1. To describe system operations, we
consider one end of the full-duplex link. The configuration of an echo canceller for a PAM transmission
system (see Section 2.5.5) is shown in Figure 17.2. The transmitted data consist of a sequence {ak } of
i.i.d. real-valued symbols from the M-ary alphabet  = {±1,±3, … , ±(M − 1)}. The sequence {ak } is
filtered by a digital transmit filter, whose output is converted into an analog signal by a digital-to-analog
converter (DAC) with frequency response HD∕A (f ), and T denotes the modulation interval. The DAC
output is filtered by the analog transmit filter and is input to the channel through the hybrid (see also
Appendix 7.D).

{ak} transmit receive + {a^k}

DAC sampler + detector
filter filter –

echo echo
canceller hybrid hybrid
canceller

{a^kR} – {akR}
receive transmit
detector + sampler DAC
+ filter filter

Figure 17.1 Model of a full-duplex PAM transmission system.

ak
T

echo t = kT
hE
canceller
u(t)
zk x(t) + r(t)
– + sampler +
+ + +

wR(t)

Figure 17.2 Configuration of an echo canceller for a PAM transmission system.

The signal x at the output of the low-pass analog receive filter has three components, namely, the signal
from the far-end transmitter r, the echo u, and additive Gaussian noise 𝑤R . The signal x is given by
∑
+∞
∑
+∞
x(t) = r(t) + u(t) + 𝑤R (t) = aRk h(t − kT) + ak hE (t − kT) + 𝑤R (t) (17.1)
k=−∞ k=−∞
17.1. Echo and near-end crosstalk cancellation for PAM systems 835

where {aRk } is the sequence of symbols from the remote transmitter, and h and hE (t) = {hD∕A ∗ gE }(t)
are the impulse responses of the overall channel and of the echo channel, respectively. In the expression
of hE (t), the function hD∕A is the inverse Fourier transform of HD∕A (f ). The signal obtained after echo
cancellation is processed by a detector that outputs the sequence of detected symbols {̂aRk }.

Crosstalk cancellation and full-duplex transmission

In the case of full-duplex PAM data transmission over multi-pair cables for local-area network applica-
tions, where NEXT represents the main disturbance, the configuration of a digital NEXT canceller is
also obtained as shown in Figure 17.2, with the echo channel replaced by the crosstalk channel. For these
applications, however, instead of mono-duplex transmission, where one pair is used to transmit only in
one direction and the other pair to transmit only in the reverse direction, dual-duplex transmission may
be adopted. Bi-directional transmission at rate Rb over two pairs is then accomplished by full-duplex
transmission of data streams at Rb /2 over each of the two pairs. The lower modulation rate and/or spec-
tral efficiency required per pair for achieving an aggregate rate equal to Rb represents an advantage of
dual-duplex over mono-duplex transmission. Dual-duplex transmission requires two transmitters and
two receivers at each end of a link, as well as separation of the simultaneously transmitted and received
signals on each pair, as illustrated in Figure 17.3. In dual-duplex transceivers, it is therefore necessary
to suppress echoes originated by reflections at the hybrids and at impedance discontinuities in the cable,
as well as self NEXT, by adaptive digital echo and NEXT cancellation. Although a dual-duplex scheme
might appear to require higher implementation complexity than a mono-duplex scheme, it turns out that
the two schemes are equivalent in terms of the number of multiply-and-add operations per second that
are needed to perform the various filtering operations.
One of the transceivers in a full-duplex link will usually employ an externally provided reference
clock for its transmit and receive operations. The other transceiver will extract timing from the receive
signal, and use this timing for its transmitter operations. This is known as loop timing, also illustrated in
Figure 17.3. If signals were transmitted in opposite directions with independent clocks, signals received
from the remote transmitter would generally shift in phase relative to the also received echo signals. To
cope with this effect, some form of interpolation (see Chapter 14) would be required.

Tx rate Rb /2 Tx
rate Rb rate Rb
+ H H +

Rc Rc
echo
external loop
self-NEXT
timing timing
Tx Tx

+ H H +
rate Rb rate Rb
Rc rate Rb /2 Rc
echo

Figure 17.3 Model of a dual-duplex transmission system.

836 Chapter 17. Applications of interference cancellation

Polyphase structure of the canceller

In general, we consider baseband signalling techniques such that the signal at the output of the overall
channel has non-negligible excess bandwidth, i.e. non-negligible spectral components at frequencies
larger than half of the modulation rate, |f | ≥ 1∕(2T). Therefore, to avoid aliasing, the signal x is sampled
at twice the modulation rate or at a higher sampling rate. Assuming a sampling rate equal to F0 ∕T,
F0 > 1, the i-th sample during the k-th modulation interval is given by
( )
T
x (kF0 + i) = xkF0 +i = rkF0 +i + ukF0 +i + 𝑤kF0 +i , i = 0, … , F0 − 1
F0
∑
+∞
∑
+∞
= hnF0 +i aRk−n + hE,nF0 +i ak−n + 𝑤kF0 +i (17.2)
n=−∞ n=−∞

where {hkF0 +i , i = 0, … , F0 − 1} and {hE,kF0 +i , i = 0, … , F0 − 1} are the discrete-time impulse

responses of the overall channel and the echo channel, respectively, and {𝑤kF0 +i , i = 0, … , F0 − 1} is
a sequence of Gaussian noise samples with zero mean and variance 𝜎𝑤2 . Equation (17.2) suggests that
the sequence of samples {xkF0 +i , i = 0, … , F0 − 1} be regarded as a set of F0 interlaced sequences,
each with a sampling rate equal to the modulation rate. Similarly, the sequence of echo samples
{ukF0 +i , i = 0, … , F0 − 1} can be regarded as a set of F0 interlaced sequences that are output by F0
independent echo channels with discrete-time impulse responses {hE,kF0 +i }, i = 0, … , F0 − 1, and an
identical sequence {ak } of input symbols (see also Figures 7.83 and 7.63). Hence, echo cancellation
can be performed by F0 interlaced echo cancellers, as shown in Figure 17.4. As the performance of
each canceller is independent of the other F0 − 1 units, in the remaining part of this section we will
consider the operations of a single echo canceller.

{ak}

EC (0) EC (1) EC (F0 – 1)

{u^k F0} {u^k F0 + 1} {u^k F0 +F0 – 1}

t=l T
F0

{u^k F0 + i , i = 0,..., F0 – 1}

{zk F0+i , i = 0,..., F0–1} {xk F0+i , i = 0,..., F0–1}

– x(t)
+ sampler
+

Figure 17.4 A set of F0 interlaced echo cancellers (EC).

Canceller at symbol rate

The echo canceller generates an estimate û k of the echo signal. If we consider a transversal filter
implementation, û k is obtained as the vector product of the vector of filter coefficients at time t = kT,
17.1. Echo and near-end crosstalk cancellation for PAM systems 837

ck = [c0,k , … , cN−1,k ]T , and the vector of signals stored in the echo canceller delay line at the same
instant, ak = [ak , … , ak−N+1 ]T , expressed by
∑
N−1
û k = cTk ak = cn,k ak−n (17.3)
n=0

The estimate of the echo is subtracted from the received signal. The result is defined as the cancellation
error signal
zk = xk − û k = xk − cTk ak (17.4)
The echo attenuation that must be provided by the echo canceller to achieve proper system operation
depends on the application. For example, for the integrated services digital network (ISDN) U-Interface
transceiver, the echo attenuation must be larger than 55 dB [3]. It is then required that the echo sig-
nals outside of the time span of the echo canceller delay line be negligible, i.e. hE,n ≈ 0 for n < 0 and
n > N − 1. As a measure of system performance, we consider the mean-square error Jk at the output of
the echo canceller at time t = kT, defined by
Jk = E[z2k ] (17.5)
For a particular coefficient vector ck , substitution of (17.4) into (17.5) yields (see (2.17))
Jk = E[xk2 ] − 2cTk p + cTk Rck (17.6)
where p = E[xk ak ] and R = E[ak aTk ]. With the assumption of i.i.d. transmitted symbols, the correlation
matrix R is diagonal. The elements on the diagonal are equal to the variance of the transmitted symbols,
𝜎a2 = (M 2 − 1)∕3. From (2.35), the minimum mean-square error is given by
Jmin = E[xk2 ] − cTopt Rcopt (17.7)

where the optimum coefficient vector is copt = R−1 p.

We note that proper system operation is achieved only if the transmitted symbols are uncorrelated with
the symbols from the remote transmitter. If this condition is satisfied, the optimum filter coefficients are
given by the values of the discrete-time echo channel impulse response, i.e. copt,n = hE,n , n = 0, … , N − 1
(see Section 2.5.1).

Adaptive canceller
By the least mean square (LMS) algorithm, the coefficients of the echo canceller converge in the mean to
copt . The LMS algorithm (see Section 3.1.2) for an N-tap adaptive linear transversal filter is formulated
as follows:
ck+1 = ck + 𝜇 zk ak (17.8)
where 𝜇 is the adaptation gain.
The block diagram of an adaptive transversal filter echo canceller is shown in Figure 17.5.
If we define the vector 𝚫ck = ck − copt , the mean-square error can be expressed by (2.35)
Jk = Jmin + 𝚫cTk R 𝚫ck (17.9)
where the term 𝚫cTk R 𝚫 ck represents an excess mean-square error due to the misadjustment of the filter
settings. Under the assumption that the vectors 𝚫ck and ak are statistically independent, the dynamics
of the mean-square error are given by (see (3.196))
[ ]k 2Jmin
Jk = 𝜎02 1 − 𝜇𝜎a2 (2 − 𝜇N𝜎a2 ) + (17.10)
2 − 𝜇N𝜎a2
838 Chapter 17. Applications of interference cancellation

ak

ak ak−1 ak−N+2 ak−N+1

T T T

Σ Σ Σ Σ

c0,k c1,k cN−2,k cN−1,k

μ
+

u^k
zk xk
−
+
+

Figure 17.5 Block diagram of an adaptive transversal filter echo canceller.

where 𝜎02 is determined by the initial conditions. The mean-square error converges to a finite steady-state
value J∞ if the stability condition 0 < 𝜇 < 2∕(N𝜎a2 ) is satisfied. The optimum adaptation gain that yields
fastest convergence at the beginning of the adaptation process is 𝜇opt = 1∕(N𝜎a2 ). The corresponding time
constant and asymptotic mean-square error are 𝜏opt = N and J∞ = 2Jmin , respectively.
We note that a fixed adaptation gain equal to 𝜇opt could not be adopted in practice, as after echo cancel-
lation the signal from the remote transmitter would be embedded in a residual echo having approximately
the same power. If the time constant of the convergence mode is not a critical system parameter, an adap-
tation gain smaller than 𝜇opt will be adopted to achieve an asymptotic mean-square error close to Jmin .
On the other hand, if fast convergence is required, a variable adaptation gain will be chosen.
Several techniques have been proposed to increase the speed of convergence of the LMS algorithm. In
particular, for echo cancellation in data transmission, the speed of adaptation is reduced by the presence
of the signal from the remote transmitter in the cancellation error. To mitigate this problem, the data
signal can be adaptively removed from the cancellation error by a decision-directed algorithm [4].
Modified versions of the LMS algorithm have been also proposed to reduce system complexity. For
example, the sign algorithm suggests that only the sign of the error signal be used to compute an approx-
imation of the gradient [5] (see Section 3.1.4). An alternative method to reduce the implementation
complexity of an adaptive echo canceller consists in the choice of a filter structure with a lower compu-
tational complexity than the transversal filter.

Canceller structure with distributed arithmetic

At high rates, very large scale integration (VLSI) technology is needed for the implementation of
transceivers for full-duplex data transmission. High-speed echo cancellers and NEXT cancellers that
17.1. Echo and near-end crosstalk cancellation for PAM systems 839

do not require multiplications represent an attractive solution because of their low complexity. As
an example of an architecture suitable for VLSI implementation, we consider echo cancellation by
a distributed-arithmetic filter, where multiplications are replaced by table look-up and shift-and-add
operations [6]. By segmenting the echo canceller into filter sections of shorter lengths, various
trade-off concerning the number of operations per modulation interval and the number of memory
locations needed to store the look-up tables are possible. Adaptivity is achieved by updating the
values stored in the look-up tables by the LMS algorithm. To describe the principles of operations
of a distributed-arithmetic echo canceller, we assume that the number of elements in the alphabet of
the input symbols is a power of two, M = 2W . Therefore, each symbol is represented by the vector
[ak(0) , … , a(W−1)
k
], where a(i)
k
∈ {0, 1}, i = 0, … , W − 1, are independent binary random variables, i.e.

∑
W−1
∑
W−1
ak = (2a(𝑤)
k
− 1) 2𝑤 = b(𝑤)
k
2𝑤 (17.11)
𝑤=0 𝑤=0

where b(𝑤)
k
= (2a(𝑤)
k
− 1) ∈ {−1, +1}.
By substituting (17.11) into (17.3) and segmenting the delay line of the echo canceller into L sections
with K = N∕L delay elements each, we obtain
[ ]
∑ ∑ 𝑤 K−1
L−1 W−1
∑ (𝑤)
û k = 2 bk−𝓁K−m c𝓁K+m,k (17.12)
𝓁=0 𝑤=0 m=0

Note that the summation within parenthesis in (17.12) may assume at most 2K distinct real values, one
for each binary sequence {a(𝑤) k−𝓁K−m
}, m = 0, … , K − 1. If we pre-compute these 2K values and store
them in a look-up table addressed by the binary sequence, we can substitute the real time summation by
a simple reading from the table.
Equation (17.12) suggests that the filter output can be computed using a set of L2K values that are
stored in L tables with 2K memory locations each. The binary vectors a(𝑤) k,𝓁
= [a(𝑤)
k−𝓁K
, … , a(𝑤)
k−𝓁K−K+1
],
𝑤 = 0, … , W − 1, 𝓁 = 0, … , L − 1, determine the addresses of the memory locations where the values
that are needed to compute the filter output are stored. The filter output is obtained by WL table look-up
and shift-and-add operations.
(𝑤)
We observe that a(𝑤)
k,𝓁
and its binary complement ak,𝓁 select two values that differ only in their sign.
This symmetry is exploited to halve the number of values to be stored. To determine the output of a
distributed-arithmetic filter with reduced memory size, we reformulate (17.12) as
[ ]
∑ ∑ 𝑤 (𝑤)
L−1 W−1
(𝑤)
∑ (𝑤)
K−1
û k = 2 bk−𝓁K c𝓁K,k + bk−𝓁K bk−𝓁K−m c𝓁K+m,k (17.13)
𝓁=0 𝑤=0 m=1

Then the binary symbol b(𝑤)

k−𝓁K
determines whether a selected value is to be added or subtracted. Each
K−1
table has now 2 memory locations, and the filter output is given by

∑ ∑
L−1 W−1
û k = 2𝑤 b(𝑤) d (i(𝑤) , 𝓁)
k−𝓁K k k,𝓁
(17.14)
𝓁=0 𝑤=0

where
K−1
∑
dk (n, 𝓁) = c𝓁K,k + b(𝑤)
k−𝓁K
b(𝑤) c
k−𝓁K−m 𝓁K+m,k
, n = i(𝑤)
k,𝓁
m=1
840 Chapter 17. Applications of interference cancellation

and
⎧ K−1
∑ (𝑤)
⎪ a 2m−1 a(𝑤) =1
⎪ m=1 k−𝓁K−m k−𝓁K
i(𝑤) = ⎨ K−1 (17.15)
⎪ ∑ a(𝑤)
k,𝓁

⎪ k−𝓁K−m
2m−1 a(𝑤)
k−𝓁K
=0
⎩ m=1
Both dk (n, 𝓁) and i(𝑤)
k,𝓁
values can be stored in look-up tables.
We note that, as long as (17.12) and (17.13) hold for some coefficient vector [c0,k , …, cN−1,k ], a
distributed-arithmetic filter emulates the operation of a linear transversal filter. For arbitrary values
dk (n, 𝓁), however, a non-linear filtering operation results.
The expression of the LMS algorithm to update the values of a distributed-arithmetic echo canceller
is derived as in (3.204). To simplify the notation, we set
∑
W−1
û k (𝓁) = 2𝑤 b(𝑤) d (i(𝑤) , 𝓁)
k−𝓁K k k,𝓁
(17.16)
𝑤=0

hence (17.14) can be written as

∑
L−1
û k = û k (𝓁) (17.17)
𝓁=0

We also define the vector of the values in the 𝓁-th look-up table as
dk (𝓁) = [dk (0, 𝓁), … , dk (2K−1 − 1, 𝓁)]T (17.18)
indexed by the variable (17.15).
The values dk (𝓁) are updated according to the LMS criterion, i.e.
1
dk+1 (𝓁) = dk (𝓁) − 𝜇 ∇dk (𝓁) z2k (17.19)
2
where
∇dk (𝓁) z2k = 2zk ∇dk (𝓁) zk = −2zk ∇dk (𝓁) û k = −2zk ∇dk (𝓁) û k (𝓁) (17.20)
The last expression has been obtained using (17.17) and the fact that only û k (𝓁) depends on dk (𝓁).
Defining
yk (𝓁) = [yk (0, 𝓁), … , yk (2K−1 − 1, 𝓁)]T = ∇dk (𝓁) û k (𝓁)
(17.19) becomes
dk+1 (𝓁) = dk (𝓁) + 𝜇 zk yk (𝓁) (17.21)
For a given value of k and 𝓁, we assign the following values to the W addresses (17.15):
I (𝑤) = i(𝑤)
k,𝓁
, 𝑤 = 0, 1, … , W − 1
From (17.16) we get
∑
W−1
yk (n, 𝓁) = 2𝑤 b(𝑤) 𝛿
k−𝓁K n−I (𝑤)
(17.22)
𝑤=0

In conclusion, in (17.21) for every instant k and for each value of the index 𝑤 = 0, 1, … , W − 1, the
product 2𝑤 b(𝑤)
k−𝓁K
𝜇zk is added to the memory location indexed by I (𝑤) . The complexity of the imple-
mentation can be reduced by updating, at every iteration k, only the values corresponding to the addresses
given by the most significant bits of the symbols in the filter delay line. In this case, (17.22) simplifies
into { W−1 (W−1)
2 bk−𝓁K n = I (W−1)
yk (n, 𝓁) = (17.23)
0 n ≠ I (W−1)
17.1. Echo and near-end crosstalk cancellation for PAM systems 841

(0) (1)
ak(0) ak(1) ak−(L−1)K ak−(L−1)K
address address
computation computation
(0) (0)
i (1) ik,0
k,0 i (1) ik,L−1
k,L−1

μ zk μ zk
table table
(1)
dk+1(ik,0 ,0)
+
0 +
L−1
+ +
+ (1) (0) + (1)
dk(ik,0 ,0) dk(ik,0 ,0) dk(ik,L−1,L−1) (0)
dk(ik,L−1,L−1)

(1) (1)
1
←
+1 bk 1
←
+1 bk−(L−1)K
0 −1 0 −1
← ←
(0) (0)
1
←
+1 bk 1
←
+1 bk−(L−1)K
0 −1 0 −1
← ←

2 2

zk u^k xk
+
+

Figure 17.6 Block diagram of an adaptive distributed-arithmetic echo canceller.

The block diagram of an adaptive distributed-arithmetic echo canceller with input symbols from a
quaternary alphabet is shown in Figure 17.6.
The analysis of the mean-square error convergence behaviour and steady-state performance can be
extended to adaptive distributed-arithmetic echo cancellers [7]. The dynamics of the mean-square error
are in this case given by
[ ]k
𝜇𝜎a2 2Jmin
Jk = 𝜎02 1 − K−1 (2 − 𝜇L𝜎a2 ) + (17.24)
2 2 − 𝜇L𝜎a2
The stability condition for the echo canceller is 0 < 𝜇 < 2∕(L𝜎a2 ). For a given adaptation gain, echo can-
celler stability depends on the number of tables and on the variance of the transmitted symbols. There-
fore, the time span of the echo canceller can be increased without affecting system stability, provided
that the number L of tables is kept constant. In that case, however, mean-square error convergence will
be slower. From (17.24), we find that the optimum adaptation gain that provides the fastest mean-square
error convergence at the beginning of the adaptation process is 𝜇opt = 1∕(L𝜎a2 ). The time constant of
842 Chapter 17. Applications of interference cancellation

the convergence mode is 𝜏opt = L2K−1 . The smallest achievable time constant is therefore proportional
to the total number of values. As mentioned above, the implementation of a distributed-arithmetic echo
canceller can be simplified by updating at each iteration only the values that are addressed by the most
significant bits of the symbols stored in the delay line. The complexity required for adaptation can thus
be reduced at the price of a slower rate of convergence.

17.2 Echo cancellation for QAM systems

Although most of the concepts presented in the preceding section can be readily extended to echo cancel-
lation for communications systems employing QAM, the case of full-duplex transmission over a voice-
band data channel requires a specific discussion. We consider the system model shown in Figure 17.7.
The transmitter generates a sequence {ak } of i.i.d. complex-valued symbols from a two-dimensional
constellation , that are modulated by the carrier ej2𝜋f0 kT , where T and f0 denote the modulation interval
and the carrier frequency, respectively. The discrete-time signal at the output of the transmit passband
phase splitter filter may be regarded as an analytic signal, which is generated at the rate of Q0 ∕T sam-
ples/s, Q0 > 1. The real part of the analytic signal is converted into an analog signal by a DAC and
input to the channel. We note that by transmitting the real part of a complex-valued signal, positive and
negative-frequency components become folded. The attenuation in the image band of the transmit filter
thus determines the achievable echo suppression. In fact, the receiver cannot extract aliasing image-band
components from desired passband frequency components, and the echo canceller is able to suppress
only echo arising from transmitted passband components.

ak
(pb)
hTx Re[.] DAC
T T
delay
Q0
ej2π f0kT gFE
far-end near-end
echo echo
t=nT ejφ(t)
Q0
canceller canceller gNE
ejφ^ℓ
PLL uNE(t) Re[.]
u^ℓFE u^ℓNE e–j2π f0ℓT/F0
uFE(t)
zℓ – – xℓ r(t)
+ + gR sampler
+ + T T
F0 Q0 wR(t)

Figure 17.7 Configuration of an echo canceller for a QAM transmission system.

The output of the echo channel is represented as the sum of two contributions. The near-end echo
uNE arises from the impedance mismatch between the hybrid and the transmission line, as in the case
of baseband transmission. The far-end echo uFE represents the contribution due to echoes that are gen-
erated at intermediate points in the telephone network. These echoes are characterized by additional
impairments, such as jitter and frequency shift, which are accounted for by introducing a carrier-phase
rotation equal to 𝜑 in the model of the far-end echo.
At the receiver, samples of the signal at the channel output are obtained synchronously with the
transmitter timing, at the sampling rate of Q0 ∕T samples/s. The discrete-time received signal is con-
verted to a complex-valued baseband signal {xkF0 +i }, i = 0, … , F0 − 1, at the rate of F0 ∕T samples/s,
17.2. Echo cancellation for QAM systems 843

1 < F0 < Q0 , through filtering by the receive phase splitter filter, decimation, and demodulation. From
delayed transmit symbols, estimates of the near and far-end echo signals after demodulation, {̂uNE kF0 +i
},
i = 0, … , F0 − 1, and {̂uFE
kF0 +i
}, i = 0, … , F 0 − 1, respectively, are generated using F 0 interlaced near
and far-end echo cancellers. The cancellation error is given by
z𝓁 = x𝓁 − (̂uNE
𝓁
+ û FE
𝓁
) (17.25)
A different model is obtained if echo cancellation is accomplished before demodulation. In this case,
two equivalent configurations for the echo canceller may be considered. In one configuration, the mod-
ulated symbols are input to the transversal filter, which approximates the passband echo response.
Alternatively, the modulator can be placed after the transversal filter, which is then called a baseband
transversal filter [8].
In the considered implementation, the estimates of the echo signals after demodulation are given by
∑
NNE −1
û NE
kF +i = nF +i,k ak−n ,
cNE i = 0, … , F0 − 1 (17.26)
0 0
n=0
(N −1 )
∑
FE

û FE
kF0 +i = cFE
nF0 +i,k ak−n−DE ej𝜑̂ kF0 +1 , i = 0, … , F0 − 1 (17.27)
n=0

where [cNE
0,k
, … , cNE
F0 NNE −1,k
] and [cFE
0,k
, … , cFE
F0 NFE −1,k
] are the coefficients of the F0 interlaced near and
far-end echo cancellers, respectively, {𝜑̂ kF0 +i }, i = 0, … , F0 − 1, is the sequence of far-end echo phase
estimates, and DFE denotes the bulk delay accounting for the round-trip delay from the transmitter to
the point of echo generation. To prevent overlap of the time span of the near-end echo canceller with the
time span of the far-end echo canceller, the condition DFE > NNE must be satisfied. We also note that,
because of the different nature of near and far-end echo generation, the time span of the far-end echo
canceller needs to be larger than the time span of the near-end canceller, i.e. NFE > NNE .
Adaptation of the filter coefficients in the near and far-end echo cancellers by the LMS algorithm
leads to

nF +i,k+1 = cnF +i,k + 𝜇 zkF0 +i (ak−n ) ,

cNE NE ∗
0 0

n = 0, … , NNE − 1, i = 0, … , F0 − 1 (17.28)
and
∗ −j𝜑̂ kF0 +i
nF +i,k+1 = cnF +i,k + 𝜇 zkF0 +i (ak−n−DFE ) e
cFE ,
FE
0 0

n = 0, … , NFE − 1, i = 0, … , F0 − 1 (17.29)
respectively.
The far-end echo phase estimate is computed by a second-order phase-lock loop algorithm (see
Section 14.7), where the following gradient approach is adopted:
⎧ 1
⎪ 𝜑̂ 𝓁+1 = 𝜑̂ 𝓁 − 2 𝜇𝜑 ∇𝜑̂ |z𝓁 | + Δ𝜑𝓁
2
(mod 2𝜋)
⎨ 1 (17.30)
⎪ Δ𝜑𝓁+1 = Δ𝜑𝓁 − 𝜇f ∇𝜑̂ |z𝓁 |
2
⎩ 2
where 𝓁 = kF0 + i, i = 0, … , F0 − 1, 𝜇𝜑 and 𝜇f are parameters of the loop, and
𝜕|z𝓁 |2
∇𝜑̂ |z𝓁 |2 = = −2 Im{z𝓁 (̂uFE
𝓁
)∗ } (17.31)
𝜕 𝜑̂ 𝓁
We note that the algorithm (17.30) requires F0 iterations per modulation interval, i.e. we cannot resort
to interlacing to reduce the complexity of the computation of the far-end echo phase estimate.
844 Chapter 17. Applications of interference cancellation

17.3 Echo cancellation for OFDM systems

We discuss echo cancellation for orthogonal frequency division multiplexing (OFDM) with reference
to a discrete-multitone (DMT) system (see Chapter 8), as shown in Figure 17.8. Let {hi }, i = 0, … ,
Nc − 1, be the channel impulse response, sampled with period T∕, with length Nc ≪  and {hE,i },
i = 0, … , N − 1, be the discrete-time echo impulse response whose length is N < , where  denotes
the number of subchannels of the DMT system. To simplify the notation, the length of the cyclic prefix
will be set to L = Nc − 1. Recall that in a DMT transmitter the block of  samples at the inverse
discrete Fourier transform (IDFT) output in the k-th modulation interval, [Ak [0], … , Ak [ − 1]], is
cyclically extended by copying the last L samples at the beginning of the block. After a P/S conversion,
wherein the L samples of the cyclic extension are the first to be output, the L +  samples of the block
are transmitted into the channel. At the receiver, the sequence is split into blocks of length L + ,
[xk(+L) , … , x(k+1)(+L)−1 ]. These blocks are separated in such a way that the last  samples depend
only on a single cyclically extended block, then the first L samples are discarded.

{ak} {Ak} add

IDFT cyclic P/S DAC
T′ T′ extension T

delay

{Ak–1}
t=n T gE
frequency time echo 
domain domain canceller
u(t)
{diag (ak) Ck} {Θk,k–1 ck}
{Zk} – – remove x(t) r(t)
+ DFT + cyclic S/P sampler
T′ + T′ + extension T
 wR(t)

Figure 17.8 Configuration of an echo canceller for a DMT transmission system.

Case N ≤ L + 1 We initially assume that the length of the echo channel impulse response is N ≤ L + 1.
Furthermore, we assume that the boundaries of the received blocks are placed such that the last 
samples of the k-th received block are expressed by the vector (see (8.77))

xk = 𝚵Rk h + 𝚵k hE + 𝒘k (17.32)

where h = [h0 , … , hL , 0, … , 0]T is the vector of the overall channel impulse response extended with
 − L − 1 zeros, hE = [hE,0 , … , hE,N−1 , 0, … , 0]T is the vector of the overall echo impulse response
extended with  − N zeros, 𝒘k is a vector of additive white Gaussian noise (AWGN) samples, 𝚵Rk is
the circulant matrix with elements given by the signals from the remote transmitter
⎡ ARk [0] ARk [ − 1] … ARk [1] ⎤
⎢ R
⎥
⎢ Ak [1] ARk [0] … ARk [2] ⎥
𝚵k = ⎢
R
⎥ (17.33)
⎢ ⋮ ⋮ ⋮ ⎥
⎢ R ⎥
⎣ Ak [ − 1] ARk [ − 2] … ARk [0] ⎦
17.3. Echo cancellation for OFDM systems 845

and 𝚵k is the circulant matrix with elements generated by the local transmitter
⎡ Ak [0] Ak [ − 1] … Ak [1] ⎤
⎢ ⎥
⎢ Ak [1] Ak [0] … Ak [2] ⎥
𝚵k = ⎢ ⎥ (17.34)
⎢ ⋮ ⋮ ⋮ ⎥
⎢ ⎥
⎣ Ak [ − 1] Ak [ − 2] … Ak [0] ⎦
In the frequency domain, the echo is expressed as
Uk = diag(ak ) HE (17.35)
where HE denotes the discrete Fourier transform (DFT) of the vector hE . In this case, the echo canceller
provides an echo estimate that is given by
̂ = diag(a ) C
U (17.36)
k k k

where Ck denotes the DFT of the vector ck of the N coefficients of the echo canceller filter extended
with  − N zeros.
In the time domain, (17.36) corresponds to the estimate
û k = 𝚵k ck (17.37)

Case N > L + 1 In practice, however, we need to consider the case N > L + 1. The expression of the
cancellation error is then given by
xk = 𝚵Rk h + 𝚿k,k−1 hE + 𝒘k (17.38)
where 𝚿k,k−1 is a circulant matrix given by
𝚿k,k−1 =
⎡ Ak [0] Ak [ − 1] … Ak [ − L] Ak−1 [ − 1] … Ak−1 [L + 1] ⎤
⎢ ⎥
⎢ Ak [1] Ak [0] … Ak [ − L + 1] Ak [ − L] … Ak−1 [L + 2] ⎥ (17.39)
⎢ ⎥.
⎢ ⋮ ⎥
⎢ ⎥
⎣ Ak [ − 1] Ak [ − 2] … Ak [ − L − 1] Ak [ − L − 2] … Ak [0] ⎦
From (17.38), the expression of the cancellation error in the time domain is then given by
zk = xk − 𝚿k,k−1 ck (17.40)
We now introduce the Toeplitz triangular matrix
𝚯k,k−1 = 𝚿k,k−1 − 𝚵k (17.41)
Substitution of (17.41) into (17.40) yields
zk = xk − 𝚯k,k−1 ck − 𝚵k ck (17.42)
In the frequency domain, (17.42) can be expressed as
Zk = F (xk − 𝚯k,k−1 ck ) − diag(ak ) Ck (17.43)
Equation (17.43) suggests a computationally efficient, two-part echo cancellation technique. First, in
the time domain, a short convolution is performed and the result subtracted from the received signals to
compensate for the insufficient length of the cyclic extension. Second, in the frequency domain, cancel-
lation of the residual echo is performed over a set of  independent echo subchannels. Observing that
(17.43) is equivalent to
Zk = X k − 𝚿 ̃
k,k−1 Ck (17.44)
846 Chapter 17. Applications of interference cancellation

̃
where 𝚿 k,k−1 = F 𝚿k,k−1 F , the echo canceller adaptation by the LMS algorithm in the frequency
−1

domain takes the form

Ck+1 = Ck + 𝜇 𝚿̃H Z (17.45)
k,k−1 k

where 𝜇 is the adaptation gain.

We note that, alternatively, echo canceller adaptation may also be performed by the simplified algo-
rithm [9]
Ck+1 = Ck + 𝜇 diag(a∗k ) Zk (17.46)
which entails a substantially lower computational complexity than the LMS algorithm, at the price of a
slower rate of convergence.
In DMT systems, it is essential that the length of the channel impulse response be much smaller than
the number of subchannels, so that the reduction in rate due to the cyclic extension may be considered
negligible. Therefore, time-domain equalization is adopted in practice to shorten the length of the chan-
nel impulse response. From (17.43), however, we observe that transceiver complexity depends on the
relative lengths of the echo and of the channel impulse responses. To reduce the length of the cyclic
extension as well as the computational complexity of the echo canceller, various methods have been
proposed to shorten both the channel and the echo impulse responses jointly [10].

17.4 Multiuser detection for VDSL

In this section, we address the problem of multiuser detection for upstream VDSL transmission (see
Chapter 18), where FEXT signals at the input of a VDSL receiver are viewed as interferers that share
the same channel as the remote user signal [11].
We assume knowledge of the FEXT responses at the central office and consider a decision-feedback
equalizer (DFE) structure with cross-coupled linear feedforward (FF) equalizers and feedback (FB) fil-
ters for crosstalk suppression. DFE structures with cross-coupled filters have also been considered for
interference suppression in wireless code-division multiple-access (CDMA) communications [12], see
also Appendix 19.A. Here we determine the optimum DFE coefficients in a minimum mean-square error
(MMSE) sense assuming that each user adopts OFDM modulation for upstream transmission. A system
with reduced complexity may be considered for practical applications, in which for each user and each
subchannel only the most significant interferers are suppressed.
To obtain a receiver structure for multiuser detection that exhibits moderate complexity, we assume
that each user adopts filtered multitone (FMT) modulation with  subchannels for upstream trans-
mission (see Chapter 8). Hence, the subchannel signals exhibit non-zero excess bandwidth as well
as negligible spectral overlap. Assuming upstream transmission by U users, the system illustrated in
Figure 17.9 is considered. In general, the sequences of subchannel signal samples at the multicarrier
demodulator output are obtained at a sampling rate equal to a rational multiple F0 of the modulation rate
1∕T. To simplify the analysis, here an integer F0 ≥ 2 is assumed.
We introduce the following definitions:
1. {a(u)
k
[i]}, sequence of i.i.d. complex-valued symbols from a QAM constellation (u) [i] transmitted
by user u over subchannel i, with variance 𝜎a2(u) [i] ;
2. {̂a(u)
k
[i]}, sequence of detected symbols of user u at the output of the decision element of subchan-
nel i;
3. h(u)
n [i], overall impulse response of subchannel i of user u;
4. h(u,𝑣)
𝙵𝙴𝚇𝚃,n
[i], overall FEXT response of subchannel i, from user 𝑣 to user u;
17.4. Multiuser detection for VDSL 847

5. G(u) [i], gain that determines the power of the signal of user u transmitted over subchannel i;
6. {𝑤̃ (u)
n [i]}, sequence of additive Gaussian noise samples with correlation function 𝚛𝑤(u) [i] (m).

At the output of subchannel i of the user-u demodulator, the complex baseband signal is given by
∑
∞
xn(u) [i] = G(u) [i] h(u)
n−kF
[i] a(u)
k
[i]
0
k=−∞

∑ U
∑∞
+ G(𝑣) [i] h(u,𝑣)
𝙵𝙴𝚇𝚃,n−kF
[i]a(𝑣)
k
[i] + 𝑤̃ (u)
n [i] (17.47)
0
𝑣=1 k=−∞
𝑣≠u

For user u, symbol detection at the output of subchannel i is achieved by a DFE structure such that the
input to the decision element is obtained by combining the output signals of U linear filters and U FB
filters from all users, as illustrated in Figure 17.9.

Additional terms from other U–1

users’ subchannel outputs

Forward a^k(v)[0]
ak(v)[0] filter
+
Multicarrier demodulator,

Far-end crosstalk from

Multicarrier modulator,

U–1 interferers Feedback

filter
user v

user v

gC(v) + Additional terms from other U–1

users’ subchannel outputs
ak(v)[M– 1] Additive noise

Forward +
a^k(v)[M– 1]
filter

Feedback
filter
Forward Feedback
filter filter

Forward a^k(u)[0]
ak(u)[0] filter +
Multicarrier demodulator,

Far-end crosstalk from

Multicarrier modulator,

U–1 interferers Feedback

filter
user u

user u

gC(u) +

ak(u)[M– 1] Additive noise

T Forward +
a^k(u)[M– 1]
filter
T
Feedback
filter

Figure 17.9 Block diagram of transmission channel and DFE structure. Source: Reproduced with
permission from Cherubini [11]. ©2001, IEEE.

In practice, to reduce system complexity, for each user only a subset of all other user signals (interfer-
ers) is considered as an input to the DFE structure [11]. The selection of the subset signals is based on
the power and the number of the interferers. This strategy, however, results in a loss of performance, as
some strong interferers may not be considered. This effect is similar to the near–far problem in CDMA
systems. To alleviate this problem, it is necessary to introduce power control of the transmitted signals;
a suitable method will be described in the next section.
848 Chapter 17. Applications of interference cancellation

To determine the DFE filter coefficients, we assume M1 and M2 coefficients for each FF and FB filter,
respectively. We define the following vectors:

1. x(u)
kF0
(u)
[i] = [xkF (u)
[i], … , xkF [i]]T , signal samples stored at instant kF0 in the delay lines of the
0 0 −M1 +1
FF filters with input given by the demodulator output of user u at subchannel i;
2. c(u,𝑣) [i] = [c(u,𝑣)
0
[i], … , c(u,𝑣)
M1 −1
[i]]T , coefficients of the FF filter from the demodulator output of user
𝑣 to the decision element input of user u at subchannel i;
3. b(u,𝑣) [i] = [b(u,𝑣)
1
[i], … , b(u,𝑣)
M2
[i]]T , coefficients of the FB filter from the decision element output of
user 𝑣 to the decision element input of user u at subchannel i;
[ ]T
4. â (u)
k
[i] = â (u) ̂ (u) ⌈ D(u)[i] ⌉ [i]
⌈ (u)[i] ⌉ [i], … , a , symbol decisions stored at instant k in the delay
k−1− D F k−M2 − F
0 0
lines of the FB filters with input given by the decision element output of user u at subchannel i,
where D(u) [i] is a suitable integer delay related to the DFE; we assume no decision errors, that is
â (u)
k
[i] = a(u)
k
[i].

The input to the decision element of user u at subchannel i at instant k is then expressed by (see
Section 7.4.4)
T T
y(u)
k
[i] = c(u,u) [i] x(u)
kF
[i] + b(u,u) [i] a(u)
k
[i]
0

∑
U
T (17.48)
(u,𝑣)T
+ {c [i] x(𝑣)
kF
[i] + b(u,𝑣) [i] a(𝑣)
k
[i]}
0
𝑣=1
𝑣≠u

and the error signal is given by

e(u)
k
[i] = y(u)
k
[i] − a(u)⌈ D(u) [i] ⌉ (17.49)
k− F0

Without loss of generality, we extend the technique developed in Section 7.4.4 for the single-user
fractionally spaced DFE to determine the optimum coefficients of the DFE structure for user u = 1. We
introduce the following vectors and matrices:
[ ]T
1. h(u) (u) (u)
m [i] = G [i] hmF +M −1+D(u) [i] [i], … , hmF +D(u) [i] [i] , vector of M1 samples of the impulse
(u)
0 1 0
response of subchannel i of user u;
[ ]T
2. h(u,𝑣)
𝙵𝙴𝚇𝚃,m
[i] = G(𝑣) [i] h(u,𝑣)
𝙵𝙴𝚇𝚃,mF0 +M1 −1+D [i]
(u,𝑣)
[i], … , h (u,𝑣)
𝙵𝙴𝚇𝚃,mF0 +D [i]
(u,𝑣)
[i] , vector of M1 samples of the
FEXT impulse response of subchannel i from user 𝑣 to user u; assuming the differences between
the propagation delays of the signal of user u and of the crosstalk signals originated by the other
users are negligible, we have D(u,𝑣) [i] = D(u) [i];
3.
(
(1)∗ (1)T
∑M2
(1)∗ T
(1,1)
R [i] = E[xk [i] xk [i]] − 𝜎a2(1) [i] hm [i] h(1)
m [i]
m=1
)
∑
V
∗ T
+ 𝜎a2(𝑣) [i] h(1,𝑣)
𝙵𝙴𝚇𝚃,m
[i] h(1,𝑣)
𝙵𝙴𝚇𝚃,m
[i] (17.50)
𝑣=2

∗ T
R(l,l) [i] = E[x(l)
k
[i] x(l)
k
[i]]
∑
M2
∗ T ∗ T
− (𝜎a2(l) [i] h(l) (l) 2 (l,1) (l,1)
m [i] hm [i] + 𝜎a(1) [i] h𝙵𝙴𝚇𝚃,m [i] h𝙵𝙴𝚇𝚃,m [i]),
m=1
l = 2, … , U (17.51)
17.4. Multiuser detection for VDSL 849

∗ T
R(l,1) [i] = E[x(l)
k
[i] x(1)
k
[i]]
⎛
∑
M2
⎜ 2 (l,1)∗ (1)T (l)∗ (l,1)T
⎜𝜎a(1) [i] h𝙵𝙴𝚇𝚃,m [i] hm [i] + 𝜎a(l) [i] h𝙵𝙴𝚇𝚃,m [i] h𝙵𝙴𝚇𝚃,m [i]
2
−
m=1 ⎜
⎝
⎞
∑
U
(l,p)∗ (1,p)T ⎟
+ 𝜎a2(p) [i] h𝙵𝙴𝚇𝚃,m [i] h𝙵𝙴𝚇𝚃,m [i]⎟ ,
p=2 ⎟
p≠l ⎠
l = 2, … , U (17.52)
∗ (j)T
R(l,j) [i] = E[x(l)
k
[i] xk [i]]
∑
M2
∗ (j,1)T (l,j)∗ (j)T
− (𝜎a2(1) [i] h(l,1)
𝙵𝙴𝚇𝚃,m
[i] h𝙵𝙴𝚇𝚃,m [i] + 𝜎a2(j) [i] h𝙵𝙴𝚇𝚃,m [i] hm [i]),
m=1
1<j<l≤U (17.53)
where all above matrices are M1 × M1 square matrices;
4.
⎡ R(1,1) [i] R(1,2) [i] … R(1,U) [i] ⎤
⎢ (2,1) ⎥
R [i] R(2,2) [i] … R(2,U) [i] ⎥
R(1) [i] = ⎢ (17.54)
⎢⋮ ⋮ ⋱ ⋮ ⎥
⎢ R(U,1) [i] R(U,2) [i] … R(U,U) [i] ⎥⎦
⎣
where R(1) [i] in general is a positive semi-definite Hermitian matrix, for which we assume here the
inverse exists;
5.
p(1) [i] = 𝜎a2(1) [i] [h0(1) [i], h(2,1)
𝙵𝙴𝚇𝚃,0
[i], … , h(U,1)
𝙵𝙴𝚇𝚃,0
[i]]T (17.55)
(1,1)T T
Defining the vectors c(1) [i] = [c [i], c (1,1)T (1,2)T [i], … , c (1,U)T (1)
[i]]T , and b [i] = [b [i], … , b(1,U)
T
[i]] , the optimum coefficients are given by
c(1) (1) −1 (1)
opt [i] = [R [i]] p [i] (17.56)
and
⎡ h(1)T [i], h(2,1)T [i], … , h(U,1)T [i] … ⎤
⎢ 1(1)T 𝙵𝙴𝚇𝚃,1 𝙵𝙴𝚇𝚃,1 ⎥
(2,1)T (U,1)T
⎢ hM1 [i], h𝙵𝙴𝚇𝚃,M1 [i], … , h𝙵𝙴𝚇𝚃,M1 [i] ⎥
⎢ (1,2)T (2)T (U,2)T ⎥
h
⎢ 𝙵𝙴𝚇𝚃,1 [i], h 1
[i], … , h 1
[i] … ⎥
b(1) ⎢ (1,2) T (2)T (U,2)T
⎥ c(1) [i]
opt [i] = h𝙵𝙴𝚇𝚃,M [i], h𝙵𝙴𝚇𝚃,M [i], … , h𝙵𝙴𝚇𝚃,M [i] … (17.57)
⎢ 1 1 1 ⎥ opt
⎢ h(1,U) [i], h(2,U) [i], … , h(U) [i] … ⎥
T T T

⎢ 𝙵𝙴𝚇𝚃,1
(1,U)T
𝙵𝙴𝚇𝚃,1
(2,U)T
1
(U)T ⎥
⎢ h𝙵𝙴𝚇𝚃,M1 [i], h𝙵𝙴𝚇𝚃,M1 [i], … , h𝙵𝙴𝚇𝚃,M1 [i] ⎥
⎢ ⎥
⎣ ⎦
The MMSE value at the decision point of user 1 on subchannel i is thus given by
(1) H
Jmin [i] = 𝜎a2(1) [i] − p(1) [i]c(1)
opt [i] (17.58)
The performance of an OFDM system is usually measured in terms of achievable bit rate for given
channel and crosstalk characteristics (see Chapter 13). The number of bits per modulation interval that
can be loaded with a bit-error probability of 10−7 on subchannel i is given by (see (13.16))
( )
𝜎a2(1) [i]
𝚋 [i] = log2 1 + (1)
(1)
10(Gcode,dB −Γgap,dB )∕10
(17.59)
Jmin [i]
850 Chapter 17. Applications of interference cancellation

where Gcode,dB is the coding gain assumed to be the same for all users and all subchannels. The achievable
bit rate for user 1 is therefore given by
−1
1 ∑ (1)
R(1)
b
= 𝚋 [i] bit∕s (17.60)
T i=0

17.4.1 Upstream power back-off

Upstream power back-off (PBO) methods are devised to allow remote users in a VDSL system to achieve
a fair distribution of the available capacity in the presence of FEXT [13]. The upstream VDSL trans-
mission rates, which are achievable with PBO methods, usually depend on parameters, for example
a reference length or the integral of the algorithm of the received signal power spectral density, that
are obtained as the result of various trade-offs between services to be offered and allowed maximum
line length. However, the application of such PBO methods results in a suboptimum allocation of the
signal power for upstream transmission. It is desirable to devise a PBO algorithm with the following
characteristics:
● for each individual user, the transmit signal power spectral density (PSD) is determined by taking
into account the distribution of known target rates and estimated line lengths of users in the network,
and
● the total power for upstream signals is kept to a minimum to reduce interference with other services
in the same cable binder.
In the preceding section, we found the expression (17.60) of the achievable upstream rate for a user in
a VDSL system with U users, assuming perfect knowledge of FEXT impulse responses and multiuser
detection. However, PBO may be applied by assuming only the knowledge of the statistical behaviour
of FEXT coupling functions with no attempt to cancel interference. In this case, the achievable bit rate
of user u is given by (see also Section 6.105)
⎡ ⎤
⎢  (u) (f )| (u) (f )|2 (Gcode,dB −Γgap,dB )∕10 ⎥
R(u) = log2 ⎢1 + ∑U 10 ⎥ df (17.61)
b ∫ (u,𝑣)
⎢ 𝑣=1  (f )|𝙵𝙴𝚇𝚃 (f )| + N0 ⎥
(𝑣) 2
⎣ 𝑣≠u ⎦
where  (u) (f ) denote the PSD of the signal transmitted by user u,  (u) (f ) is the frequency response of
(u,𝑣)
the channel for user u, 𝙵𝙴𝚇𝚃 (f ) is the FEXT frequency response from user 𝑣 to user u, and N0 is the
PSD of AWGN. The expression of the average FEXT power coupling function is given by
(u,𝑣)
|𝙵𝙴𝚇𝚃 (f )|2 = kt f 2 min(Lu , L𝑣 )| (𝑣) (f )|2 (17.62)
where Lu and L𝑣 denote the lengths of the lines of user u and 𝑣, respectively, and kt is a constant.
Assuming that the various functions are constant within each subchannel band for OFDM modulation,
we approximate (17.61) as

−1
⎡ ⎤
∑ 1 ⎢  (u) (fi )| (u) (fi )|2 (Gcode,dB −Γgap,dB )∕10 ⎥
R(u)
T 2 ⎢⎢ ⎥
=b
log 1 + ∑U (u,𝑣)
10 (17.63)
𝑣=1  (fi )|𝙵𝙴𝚇𝚃 (fi )| + N0 ⎥
(𝑣) 2
i=0
⎣ 𝑣≠u ⎦
where fi denotes the centre frequency of subchannel i.
(u)
Let  (fi ) denote the PSD of the signal transmitted by user u on subchannel i with gain G(u) [i] = 1.
Then, the PBO problem can be formulated as follows: find the minimum of the function
U −1
∑
U
∑ ∑ 1 (u)
P(u) (f ) df ≃ (G(u) [i])2  (fi ) (17.64)
u=1
∫ u=1 i=0
T
17.4. Multiuser detection for VDSL 851

subject to the constraints:

1.
(u)
0 ≤ (G(u) [i])2  (fi ) ≤ max , u = 1, … , U, i = 0, … ,  − 1 (17.65)
and
2.
R(u)
b
≥ R(u)
b,target
, u = 1, … , U (17.66)

where max is a constant maximum PSD value and R(u)

b,target
is the target rate for user u.

In (17.66), R(u)
b
is given by (17.60) or (17.63), depending on the receiver implementation.
Finding the optimum upstream transmit power distribution for each user is therefore equiva-
lent to solving a non-linear programming problem in the U parameters G(u) [i], u = 1, … , U,
i = 0, … ,  − 1. The optimum values of these parameters that minimize (17.64) can be found by
simulated annealing [14, 15].

17.4.2 Comparison of PBO methods

The European Telecommunication Standard Institute (ETSI) has defined two modes of operation, named
A and B, for PBO [13]. For a scenario using upstream VDSL transmission of two adjacent links with
unequal lengths, mode A states that the signal-to-noise ratio degradation to either link shall not exceed
3 dB relative to the equal-length FEXT case. Mode B requires that the signal-to-noise ratio on the
longer line shall not be degraded relative to the equal-length FEXT case; furthermore, degradation to
the signal-to-noise ratio on the shorter line shall be bounded such that the shorter line can support at
least the upstream rate supported on the longer line. Several methods compliant with either mode A or B
have been proposed. PBO methods are also classified into methods that allow shaping of the PSD of the
transmitted upstream VDSL signal, e.g. the equalized FEXT method, and methods that lead to an essen-
tially flat PSD of the transmitted signal over each individual upstream band, e.g. the average log method.
Both the equalized FEXT and the average log method, which are described below, comply with mode B.
The equalized FEXT method requires that the PSD of user u be computed as [13]
[ ]
L ref | ref (f )| 2
 (u) (f ) = min  , (17.67)
L𝑣 | (u) (f )|2 max max
where Lref and ref denote a reference length and a reference channel frequency response, respectively.
The average log method requires that, for an upstream channel in the frequency band (f1 , f2 ), user u
adopt a constant PSD given by [16]
 (u) (f ) = (f(u),f ) , f ∈ (f1 , f2 ) (17.68)
1 2

where (f(u),f ) is a constant PSD level chosen such that it satisfies the condition
1 2
f2
log2 [(f(u),f ) | (u) (f )2 |] df = K(f1 ,f2 ) (17.69)
∫f1 1 2

where K(f1 ,f2 ) is a constant.

In this section, the achievable rates of VDSL upstream transmission using the optimum algorithm
(17.64) and the average log method are compared for various distances and services. The numerical
results presented in this section are derived assuming a 26-gauge telephone twisted-pair cable. The
noise models for the alien-crosstalk disturbers at the line termination and at the network termination
are taken as specified in [17] for the fibre-to-the-exchange case. AWGN with a PSD of −140 dBm/Hz
is assumed. We consider upstream VDSL transmission of U = 40 users over the frequency band given
852 Chapter 17. Applications of interference cancellation

by the union of 1 = (2.9, 5.1 MHz) and 2 = (7.05, 12.0 MHz), similar to those specified in [13].
The maximum PSD value is max = −60 dBm/Hz. FEXT power coupling functions are determined
according to (17.62), where kt = 6.65 × 10−21 . Upstream transmission is assumed to be based on FMT
modulation with bandwidth of the individual subchannels equal to 276 kHz and excess bandwidth of
12.5%; for an efficient implementation, a frequency band of (0, 17.664 MHz) is assumed, with  = 64
subchannels, of which only 26 are used. For the computation of the achievable rates, for an error prob-
′
ability of 10−7 a signal-to-noise ratio gap to capacity equal to Γgap,dB = Γgap,dB + 6 = 15.8 dB, which
includes a 6 dB margin against additional noise sources that may be found in the digital-subscriber-line
(DSL) environment [18], and Gcode = 5.5 dB are assumed.
For each of the methods and for given target rates, we consider two scenarios: the users are (i) all
the same distance L from the central office and (ii) uniformly distributed at 10 different nodes, having
distances jLmax ∕10, j = 1, … , 10, from the central office. To assess the performance of each method, the
maximum line length Lmax is found, such that all users can reliably achieve a given target rate Rb,target =
13 MBit/s. The achievable rates are also computed for the case that all users are at the same distance
from the central office and no PBO is applied.
For the optimum algorithm, the achievable rates are computed using (17.63). Furthermore, different
subchannel gains may be chosen for the two bands, but transmission gains within each band are equal.
Figure 17.10 shows the achievable rates for each group of four users with the optimum algorithm for the
given target rate. The maximum line length Lmax for scenario (ii) turns out to be 950 m. For application
to scenario (ii), the optimum algorithm requires the computation of 20 parameters. Note that for all
users at the same distance from the central office, i.e. scenario (i), the optimum algorithm requires the
computation of two gains equal for all users. For scenario (i), the achievable rate is equal to the target
rate up to a certain characteristic length Lmax , which corresponds to the length for which the target rate is
achieved without applying any PBO. Also note that Lmax for scenario (ii) is larger than the characteristics
length found for scenario (i).

50
Opt. PBO, 40 users @ various distances
No PBO, 40 users @ equal distance L
40
Achievable rate (Mbit/s)

30

20

10

0
0 500 1000 1500
L (m)

Figure 17.10 Achievable rates of individual users versus cable length using the optimum upstream
PBO algorithm for a target rate of 13 Mbit/s. Source: Reproduced with permission from Cherubini
[11]. ©2001, IEEE.
17.4. Multiuser detection for VDSL 853

50
Ave. log PBO, 40 users @ various distances
Ave. log PBO, 40 users @ equal distance L
Achievable rate (Mbit/s) 40 No PBO, 40 users @ equal distance L

30

20

10

0
0 500 1000 1500
L (m)

Figure 17.11 Achievable rates of individual users versus cable length using the average log
upstream PBO method for a target rate of 13 Mbit/s. Source: Reproduced with permission from
Cherubini [11]. ©2001, IEEE.

Figure 17.11 illustrates the achievable rates with the average log algorithm (17.69). Joint optimization
of the two parameters K1 and K2 for maximum reach under scenario (ii) yields K1 = 0.02 mW, K2 =
0.05 mW, and Lmax = 780 m. By comparison with Figure 17.10, we note that for the VDSL transmission
spectrum plan considered, optimum upstream PBO leads to an increase in the maximum reach of up to
20%. This increase depends on the distribution of target rates and line lengths of the users in the network.
At this point, further observations can be made on the application of PBO.
● Equal upstream services have been assumed for all users. The optimum algorithm described is even
better suited for mixed-service scenarios.
● The application of PBO requires the transmit PSDs of the individual user signals to be recomputed
at the central office whenever one or more users join the network or drop from the network.
To illustrate system performance achievable with multiuser detection, we consider U = 20 users,
uniformly distributed at 10 different nodes having distances jLmax ∕10, j = 1, … , 10, from the central
office, where identification of FEXT impulse responses is performed. For the computation of the achiev-
able rates of individual users, the FEXT impulse responses are generated by a statistical model, and
Lmax = 500 m is assumed. Furthermore, to assess the relative merits of multiuser detection and cod-
ing, the achievable rates are computed for the two cases of uncoded and coded transmission. For coded
transmission, a powerful coding technique yielding 8.5 dB coding gain for an error probability of 10−7
is assumed. Figure 17.12 illustrates the achievable rates for perfect suppression of all interferers, which
corresponds to the single-user bound. For comparison, the achievable rates for the case that all users are
the same distance away from the central office and neither multiuser detection nor coding are applied are
also given. Figure 17.13 illustrates the achievable rates for perfect suppression of the 10 worst interferers
and no application of PBO. We observe that, without PBO, the partial application of multiuser detec-
tion does not lead to a significant increase of achievable rates for all users, even assuming large coding
gains. Finally, Figure 17.14 depicts the achievable rates obtained for perfect suppression of the 10 worst
interferers and application of the optimum PBO algorithm with target rate of Rb,target = 75 Mbit/s for all
854 Chapter 17. Applications of interference cancellation

150
20 users: opt. PBO, 8.5 dB cod. gain, id. MD
20 users: opt. PBO, no coding, ideal MD
Achievable rate (Mbit/s) 20 users: no PBO, no coding, no MD

100

50

0
0 500 1000 1500
L (m)

Figure 17.12 Achievable rates of individual users versus cable length with all interferers suppressed.
Source: Reproduced with permission from Cherubini [11]. ©2001, IEEE.

150
20 users: no PBO, 8.5 dB coding gain, partial MD
20 users: no PBO, no coding, partial MD
20 users: no PBO, no coding, no MD
Achievable rate (Mbit/s)

100

50

0
0 500 1000 1500
L (m)

Figure 17.13 Achievable rates of individual users versus cable length with 10 interferers suppressed
and no PBO applied. Source: Reproduced with permission from Cherubini 11. ©2001, IEEE.
Bibliography 855

150
20 users: opt. PBO, 8.5 dB coding gain, partial MD
20 users: opt. PBO, no coding, partial MD
Achievable rate (Mbit/s) 20 users: no PBO, no coding, no MD

100

50

0
0 500 1000 1500
L (m)

Figure 17.14 Achievable rates of individual users versus cable length with 10 interferers suppressed
and optimum PBO applied for a target rate of 75 Mbit/s. Source: Reproduced with permission from
Cherubini [11]. ©2001, IEEE.

users. The target rate is achieved by all users with the joint application of multiuser detection, coding,
and power back-off.
To summarize the results of this section, a substantial increase in performance with respect to methods
that do not require the identification of FEXT responses is achieved by resorting to reduced-complexity
multiuser detection (MD) in conjunction with power back-off. This approach yields a performance close
to the single-user bound shown in Figure 17.12.

Bibliography

[1] Benvenuto, N. and Zorzi, M. (2011). Principles of Communications Networks and Systems. Wiley.
[2] Cherubini, G. (1999). Echo cancellation. In: The Mobile Communications Handbook, Chapter 7, 2e
(ed. J.D. Gibson), 7.1–7.15. Boca Raton, FL: CRC Press.
[3] Messerschmitt, D.G. (1986). Design issues for the ISDN U-Interface transceiver. IEEE Journal on
Selected Areas in Communications 4: 1281–1293.
[4] Falconer, D.D. (1982). Adaptive reference echo-cancellation. IEEE Transactions on Communica-
tions 30: 2083–2094.
[5] Duttweiler, D.L. (1982). Adaptive filter performance with nonlinearities in the correlation multi-
plier. IEEE Transactions on Acoustics, Speech, and Signal Processing 30: 578–586.
[6] Smith, M.J., Cowan, C.F.N., and Adams, P.F. (1988). Nonlinear echo cancelers based on transposed
distributed arithmetic. IEEE Transactions on Circuits and Systems 35: 6–18.
[7] Cherubini, G. (1993). Analysis of the convergence behavior of adaptive distributed-arithmetic echo
cancelers. IEEE Transactions on Communications 41: 1703–1714.
[8] Weinstein, S.B. (1977). A passband data-driven echo-canceler for full-duplex transmission on
two-wire circuits. IEEE Transactions on Communications 25: 654–666.
856 Chapter 17. Applications of interference cancellation

[9] Ho, M., Cioffi, J.M., and Bingham, J.A.C. (1996). Discrete multitone echo cancellation. IEEE
Transactions on Communications 44: 817–825.
[10] Melsa, P.J.W., Younce, R.C., and Rohrs, C.E. (1996). Impulse response shortening for discrete mul-
titone transceivers. IEEE Transactions on Communications 44: 1662–1672.
[11] Cherubini, G. (2001). Optimum upstream power back-off and multiuser detection for VDSL. Pro-
ceedings of GLOBECOM ’01, San Antonio, TX, USA.
[12] Duel-Hallen, A. (1993). Decorrelating decision-feedback multiuser detector for synchronous
code-division multiple-access channel. IEEE Transactions on Communications 41: 285–290.
[13] (2000). Access transmission systems on metallic access cables; Very high speed Digital Subscriber
Line (VDSL); Part 2: Transceiver specification. ETSI Technical Specification 101 270-2 V1.1.1, May
2000.
[14] Kirkpatrik, S., Gelatt, C.D. Jr., and Vecchi, M.P. (1983). Optimization by simulated annealing
approach. Science 220: 671–680.
[15] Vanderbilt, D. and Louie, S. (1984). A Monte Carlo simulated annealing approach to optimization
over continuous variables. Journal of Computational Physics 56: 259–271.
[16] (1999). Constant average log: robust new power back-off method. Contribution D.815 (WP1/15),
ITU-T SG 15, Question 4/15, April 1999.
[17] ETSI VDSL specifications (Part 1) functional requirements. Contribution D.535 (WP1/15), ITU-T
SG 15, Question 4/15. June 21 V- July 2, 1999.
[18] Starr, T., Cioffi, J.M., and Silverman, P.J. (1999). Digital Subscriber Line Technology. Upper Saddle
River, NJ: Prentice-Hall.
Chapter 18

Examples of communication systems

In this chapter, we provide relevant examples of communication systems, where many of the consid-
ered techniques and algorithms presented in previous chapters are used. First, the fifth generation of
cellular wireless systems, as defined by release 15 of the 3rd Generation Partnership Project (3GPP) is
introduced. Then, an overview of the GSM standard is provided. Although this belongs to the second
generation of cellular standards, it is still used for its ability to efficiently cover large area and pro-
vide basic data services suitable for the Internet of things (IoT). We conclude the overview of wireless
systems with the wireless local area network (WLAN) and the digital enhanced cordless telecommunica-
tions (or digital European cordless telecommunications, DECT) standards. As example of transmission
over unshielded twisted pairs, we provide an overview of digital subscriber lines (DSL) and Ethernet
technologies. Lastly, an example of hybrid fibre and coax communication system is discussed.

18.1 The 5G cellular system

We now provide an overview of the physical layer of the cellular communication standard defined by
3GPP. In particular, we refer to the release 15, which aims at defining the 5th generation (5G) of cellular
systems.

18.1.1 Cells in a wireless system

A fundamental concept when we speak of wireless systems is that of a cell. The coverage area of a
certain system is not served by a single transmitter but by numerous transmitters, called base stations
(BSs) or base station transceivers (BTSs): each one of them can cover only a small part of this area,
called cell. To each cell, a set of carrier frequencies completely different from those of the neighbouring
cells is assigned, so that co-channel interference is as low as possible.
The traditional method of frequency reuse, illustrated in Figure 18.1, is related to the original con-
cept of cell. To two cells separated by an adequate distance, the same set of carrier frequencies can
be assigned with minimum co-channel interference; then, there is a periodic repetition in assigning the
frequencies, as seen in Figure 18.1. Cluster is a set of cells that subdivides the available bandwidth of
the system. Obviously, each BS will interfere with the other BSs, especially with those in the same cell
and in adjacent cells, as there is always a certain percentage of irradiated power outside of the nominal
bandwidth of the assigned channel; hence, the strategy in the choice of carrier frequencies assigned to
the various cells is of fundamental importance.
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
858 Chapter 18. Examples of communication systems

B
G C B
A G C
F D A
B E F D
G C B E
A G C
F D A
E F D
E

Cells with the same set of assigned carriers.

Figure 18.1 Illustration of cell and frequency reuse concepts; cells with the same letter are assigned
the same carrier frequencies.

Related to the concept of cell is also the concept of hand-off or hand-over. When a user moves from
one cell to another, the communication cannot be interrupted; hence, a mechanism must be present that,
at every instant, keeps track of which BTS sends the strongest signal and possibly is capable of changing
the BTS to which the mobile terminal is linked.
Most recent cellular systems overcome the concept of frequency reuse, for a more flexible cell concept
(also exploiting spatial diversity), assigning to each cell the whole spectrum.

18.1.2 The release 15 of the 3GPP standard

The 3GPP activity of 5G started in 2015, with the definition of 74 use cases that would characterize the
new standard. With respect to the previous generation, the envisioned 5G scenario included a signifi-
cantly higher data rate (in accordance with a rapidly growing use of mobile data over the past years),
but also a much larger number of connected devices, including those of the IoT.
Among the applications, virtual reality, autonomous driving, heath solutions, emergency communi-
cations, and tactile Internet were the most promising and still challenging. For some applications such
as virtual reality and tactile Internet, a key factor is a low latency, i.e. a short time for a message to go
from end-to-end at the application level. A typical example is provided by a person interacting with a
moving object (e.g. playing with a ball by remote through a robot). If the visual feedback on the effects
of the actions is not fast enough, the actions on the ball will not be effective, e.g. the player will believe
to hit the ball, but the ball will be in another position in the meantime. Another important factor is
reliability, in particular for critical applications where connection interruptions may significantly dis-
rupt the application. Therefore, low end-to-end latency and high reliability were also included in the
requirements.
The 74 initial cases have been later summarized into four categories:
● Enhanced mobile broadband: This category targets the spectral efficiency enhancement, by provid-
ing higher data rates (up to Gbit/s) to the end users. It follows the evolution over the past years of
mobile service usage.
18.1. The 5G cellular system 859

● Critical communications: This category addresses new applications where timely and reliable
communications are important. It includes self-driving cars, mission critical broadband applica-
tions, augmented reality, industrial and vehicular automation.
● Massive machine-type communications: This category collects a new group of envisioned applica-
tions, related to the IoT in various realms.
● Network operations: This category does not focus on performance requirements of specific services,
but more on operational requirements that 5G networks have to fulfil.
The first three categories are somehow related to similar categories defined by the international telecom-
munication union (ITU) that set the rules for the 5G of cellular systems.
The 3GPP upgrades the cellular standard by releases that are provided every 6–12 months. The stan-
dardization of 5G has started with release 15. The release defines protocols and interfaces at the various
communication layers, and here we focus on the physical layer (layer 1), as described by the technical
specification (TS) documents of the series 38.200 (see Ref. [1] for an overview). Not all the require-
ments and features of 5G are implemented in release 15, as some have been left for next releases.1 The
evolution of the standard, with access to all technical specifications, can be followed on the 3GPP web
site https://www.3gpp.org.

18.1.3 Radio access network

The radio access network (RAN) comprises multiple cells, where each cell is a region in the space
where mobile users are served by a single fixed base station. According to the 5G 3GPP nomenclature,
the base station is denoted as new generation node base (gNB), while the mobile user is denoted as
user equipment (UE). Uplink transmissions refer to communications from the UEs to the gNB, while
downlink transmissions refer to the communications from the gNB to the UE. gNB is typically equipped
with multiple antennas, while UE devices typically have a single antenna, although the standard pro-
vides the possibility of using more antennas. Therefore, the resulting network is in general a multiuser
multiple-input-multiple-output (MIMO) system.
The new radio (NR) is the air interface developed by 3GPP for the physical and medium access control
(MAC) layers of the communications between gNB and UE. This will be the focus of the rest of this
section.

Time-frequency plan
NR has been designed to work with two frequency bands: the sub-6 GHz, from 450 MHz to 6 GHz,
and millimetre wave (mmWave), from 24.25 GHz to 52.6 GHz.
Each cell is assigned a specific band, that is used by the cell continuously over time. In release 15 of the
3GPP standard, time is divided in radio frames of the duration of 10 ms, and within each radio frame,
both uplink and downlink transmissions occur. Each radio frame is split into 10 subframes of the same
duration, each in turn split into slots. The adopted modulation format is orthogonal frequency division
multiplexing (OFDM), possibly with discrete Fourier transform (DFT) spreading (see Section 8.10.2)
in uplink, denoted as transform precoding in the 3GPP standard. Therefore, each slot is finally split
into OFDM symbols. Therefore, we have the multiple-access schemes orthogonal frequency division
multiple access (OFDMA) and single-carrier frequency division multiple access (SC-FDMA).
Five configurations, denoted with index 𝜇 = 0, … , 4, are provided by 3GPP [2], with subframes
subframe,𝜇 frame,𝜇 slot
of Nslot slots each, for a total of Nslot slots per frame. In any case, we have Nsymb = 14 symbols per

1
Note that in the bibliography of this chapter, we provide references to release documents as of April, 2019.
860 Chapter 18. Examples of communication systems

slot. Moreover, it is possible to transmit data blocks smaller than a slot, to facilitate low-rate low-latency
communications. Table 18.1 shows the parameters for the five configurations, while Figure 18.2 shows
an example of radio frame configuration for 𝜇 = 1.

Table 18.1: Radio frame configurations.

frame,𝜇 subframe,𝜇
𝜇 Nslot Nslot slot
Nsymb
0 10 1 14
1 20 2 14
2 40 4 14
3 80 8 14
4 160 16 14

1 radio frame = 10 subframes = 20 slots = 10 ms

1 sub carrier = 30 kHz

1 subframe = 2 slots = 1 ms

1 slot = 14 symbols = 0.5 ms

Figure 18.2 Example of radio frame configuration with index 𝜇 = 1.

In time-division duplexing (TDD) mode, within each sub-frame, some slots are used for uplink
and others for downlink, according to predetermined patterns. In frequency-division duplexing (FDD)
mode, the cell band is split into two sub-bands, each exclusively and continuously used for either uplink
or downlink.
Users are served over different tiles of the time-frequency plan defined by OFDMA (SC-FDMA).
The basic tile assigned to a user is a resource block (RB) spanning 12 subcarriers and a duration of 1 ms
(sub-frame duration), as shown in Figure 18.3. The mapping to RB is done first in frequency then in time
to enable early decoding at the receiver, thus reducing latency.
The NR includes also the possibility of performing carrier aggregation, i.e. assign different portions
of the spectrum to the same user. For example, it is possible to assign to a user subcarriers in both
the sub-6 GHz and the mmWave bands. Indeed, working at mmWave with a huge bandwidth is well
suited for small cells, and having also a communication link at lower frequencies is of great help to assist
handover (i.e. the mobility of an UE) from a small cell to another.
The synchronization of transmission is established by the gNB, which sets the time-frequency bound-
aries of the radio frame and all its parts. UEs are required to align their transmission to this general
timing. In the following, we will also describe the techniques used to achieve alignment.
The channel coding includes both low-density parity check (LDPC) codes (for the data channel)
and polar codes (for the control channel). Moreover, dedicated coding solutions are defined for data
blocks of size between 1 and 11 bits. Further details on coding in NR standard are provided in [3].
18.1. The 5G cellular system 861

Tframe = 10 ms

Tsubframe = 1 ms
0 1 2 3 4 5 6 7 8 9

1 slot (= 14 OFDM symbols)

Resource grid

15 kHz
1 slot

(up to 3300 subcarriers, i.e.275 RBs)

30 kHz

(with 15 kHz subcarrier spacing)

1 RB = 12 subcarriers
1 slot

60 kHz

A NR carrier

A BWP
1 slot

120 kHz

Note: some symbols have longer CP, and thus depending on the slot positions the slot length can vary slightly

Optional cell-specific slot configuration with a period of 4 slots 1 RE

DL DL Flexible UL

Optional UE-specific configuration

1 subcarrier
can overwrite flexible resource 1 symbol
DL symbols Flexible symbols UL symbols

Figure 18.3 5G NR frame structure and basic terminologies. Source: Lin et al. 2019 [4].
Reproduced with permission from IEEE.

NR data transmission chain

Data bits are coded and scrambled (see Chapter 6) before modulation. In downlink, the complex symbols
are assigned (by the layer map block) to streams (denoted layers by 3GPP), where for details on mapping
the reader is referred to [2, Table 7.3.1.3-1].
In order to exploit possibly available 1, 2, or 4 antennas, the streams are precoded in space (see
Section 9.3.2) before OFDM modulation.
In uplink, it is possible to use spreading by DFT and mapping its output to a subset of the OFDM
subcarriers (see Section 18.1.5).

OFDM numerology
The parameters of the OFDM transmission are common for both uplink and downlink of release 15
[2]. The sample period is Tc = 1∕(Δfmax Nf ), where Δfmax = 480 ⋅ 103 Hz and Nf = 4096 resulting in
Tc = 0.5 ns.
Various configurations of the number of subcarriers and length of the cyclic prefix (CP) are available,
in what is denoted as the modulation numerology. A key parameter is the subcarrier spacing configura-
tion (depending on the configuration index 𝜇), that defines the subcarrier spacing as Δf = 2𝜇 ⋅ 15 kHz.
The number of subcarriers is then defined as  = 2048 𝜅 2−𝜇 , where 𝜅 = 64, thus providing a number
of subcarriers between 213 = 8192 and 217 = 131072. Note that multiplying the number of subcarriers
by their spacing provides the same bandwidth of about 2 GHz for all modes. Moreover, a smaller
subcarrier spacing is more suitable for lower frequencies (sub-6 GHz band), where larger cells and
longer delay spreads are expected. On the other hand, a larger subcarrier spacing is adequate for higher
frequencies (as in mmWave).
The CP length is defined by the configuration index 𝜇 and by the CP mode, that can be normal or
extended. For the normal mode, the CP length (in samples) is 144 𝜅 ⋅ 2−𝜇 + 16 𝜅 for OFDM symbols
0 and 7 ⋅ 2𝜇 , within each sub-frame, and 144 𝜅 ⋅ 2−𝜇 for other OFDM symbols. For the extended mode
instead the CP length is 512 𝜅 ⋅ 2−𝜇 for all OFDM symbols. Also in this case, the dependency of the CP
862 Chapter 18. Examples of communication systems

length by the configuration index 𝜇 (thus the subcarrier spacing) ensures a fixed minimum duration (in
time) of the CP: for example, the normal CP has a minimum duration of (144 ⋅ 64∕16)Tc = 0.29 μs.

Channel estimation
In NR, the training signals used for channel estimation are denoted as reference signals. Some reference
signals are present both in uplink and downlink transmissions. In particular, these are as follows:
● Demodulation reference signal (DMRS),
● Phase tracking reference signal (PTRS),
and they will be described in more details in the following.
In most cases, the reference signals in NR are transmitted on demand, rather than on a regular basis,
following a lean design principle. Their position in the time-frequency plan is extremely flexible, provid-
ing an energy reduction with respect to previous generations, wherein reference signals were transmitted
also when not needed.

18.1.4 Downlink
The downlink signal comprises different logical signals, which can either carry data and control bits
coming from higher communication layers, and are denoted as physical channels, or carry signalling
information generated and used only at the physical layer, and are denoted as physical signals.
The physical channels defined in the downlink are as follows:
● Physical downlink shared channel (PDSCH),
● Physical downlink control channel (PDCCH), and
● Physical broadcast channel (PBCH).
The PDSCH channel carries the data, and it is encoded by LDPC codes, scrambled and modulated
as described in Section 18.1.3. It can be transmitted on up to 4 MIMO streams (layers for 3GPP nomen-
clature), where the precoder is left to the operator and is not specified by the standard.
The PDCCH contains control information such as downlink and uplink scheduling assignments.
PDCCH includes the DMRS for UE-specific precoding of the control channel. PDCCH is transmitted
in specific configurable control resource set (CORESET), spanning from 1 to 3 consecutive OFDM
symbols, and 12 resource elements (REs) of one OFDM symbol in one RB are organized into a RE
group (REG). For the structure of PDCCH, see Figure 18.4a. Codewords are bit mapped into complex
symbols, as described in Section 18.1.3, with quadrature phase shift keying (QPSK), 16 quadrature
amplitude modulation (QAM), 64 QAM, and 256 QAM.
The PBCH carries the synchronization signal.
The physical signals defined in the downlink are as follows:
● Demodulation reference signals (DMRSs),
● Phase tracking reference signals (PTRSs),
● Channel-state information reference signal (CSIRS),
● Primary synchronization signal (PSS),
● Secondary synchronization signal (SSS).
The primary and secondary synchronization signals are named together synchronization signals (SS)
and are used for synchronization purposes. CSIRS, DMRS, and PTRS are used for channel estimation
and tracking purposes.
18.1. The 5G cellular system 863

(a) PDCCH monitoring occasion PDCCH monitoring occasion 1 REG = 12 REs of 1 symbol
on a 1-symbol CORESET on a 3-symbol CORESET
... ... ...

6 7 8
1 CCE = 6 REGs
BWP DMRS
3 4 5

0 1 2

Slot

(b) Long PUCCH Short PUCCH

PUCCH

PUCCH
PDCCH

PDCCH
BWP BWP
PDSCH

PUCCH

Slot Slot

Figure 18.4 5g NR (a) PDCCH and (b) PUCCH. Source: Lin et al. 2019 [4]. Reproduced
with permission from IEEE.

Synchronization
The synchronization of the UEs to the gNB adjusts the reference time at the UE, by exploiting the
synchronization signals SSS and PSS transmitted in downlink by gNB. In order to access the network,
a UE first detects the synchronization signal block (SSB), which comprises 1 PSS, 1 SSS, and 2 PBCH
symbols.

48 subcarriers 144 subcarriers 48 subcarriers

127 subcarriers

PSS
4 OFDM Symbols

PBCH

PBCH SSS PBCH

PBCH

240 subcarriers

Figure 18.5 The NR SSB.

The SSB is transmitted over 4 OFDM symbols and 240 contiguous subcarriers (20 RBs), as shown in
Figure 18.5. Each SSB comprises a primary and a secondary synchronization signal, and by receiving
this signal, the UE obtains the cell identity, gets synchronous with downlink transmissions in both time
and frequency domains, and acquires the frame timing of the PBCH. The PSS uses binary phase shift
keying (BPSK)-modulated m-sequence of length 127, while SSS is obtained as a BPSK-modulated Gold
sequence of length 127.
864 Chapter 18. Examples of communication systems

Initial access or beam sweeping

In mmWaves, an adequate signal-to-noise ratio (SNR) at the receiver can be achieved only by precoding
and multiple transmit antennas, in order to overcome the strong attenuation incurred at those frequencies
(see also Sections 4.1.11 and 9.5). A special issue occurs when a UE enters a cell. In this case, the gNB
is not aware of the UE, therefore cannot precode any control signal to it (to start data transmission),
and similarly, the UE does not know the spatial direction of the MIMO channel to the gNB and may
even have problems in receiving control signals from the gNB. To this end, an initial access or beam
sweeping procedure is provided.
In particular, the SSB can be precoded to specific spatial directions, and multiple SSBs can be trans-
mitted consequently in a SS burst set with a maximum duration of 5 ms. The burst is used to transmit
SSBs in multiple directions, as illustrated in Figure 18.6. Since each SSB is identified by a unique index,
the UE infers the suitable direction for good reception, and then uses it for uplink transmission to the
gNB or even feedback the index to gNB, where it is going to be used for downlink precoding. In particu-
lar, the feedback is obtained by selecting specific resources of physical random access channel (PRACH)
for random access message transmission (see Section 18.1.5), corresponding to the selected SSB index.
Release 15 defines various numbers of SSBs per burst, depending on the operating frequency: up to 4
when operating below 3 GHz, 8 when operating between 3 GHz and 6 GHz, and 64 when operating
between 6 GHz and 52.6 GHz.

10 ms
Frequency

NR

5 ms

Beams in time

0
7
1 6
SSB index 2 5
(mapped to each beam) 3 4

UE-A UE-B

Figure 18.6 The beam sweeping procedure in NR.

18.1. The 5G cellular system 865

Channel estimation
Channel estimation is performed at the UE by exploiting specific training signals transmitted in the
downlink by the gNB, and known at the UE, namely the DMRS and the CSIRS.
DMRS is specific for each physical channel, and for each UE. It is transmitted on demand. The
density of DMRS in the time-frequency plane is flexible, and is organized in a comb-type struc-
ture (see Section 8.9.1). It can be integrated with MIMO supporting 12 different precoding and is
QPSK-modulated and based on Gold sequences. DMRS is used in idle mode, i.e. when the UE is not
connected.
CSIRS is transmitted by the gNB so that the UE can estimate the downlink channel. Moreover, it
also provides ways to measure the received signal power, to track precoding and time-frequency offsets.
CSIRS is specific for each UE, but multiple users can share the same resource. The allocation of CSIRS in
the time-frequency plan is very flexible. When used for time frequency tracking, CSIRS can be periodic
or aperiodic. CSIRS is used when the UE is connected.
The PTRS is used to track the phase of the local oscillators (see phase noise in Section 4.1.3). It has a
low density in the frequency domain (as the phase noise is white), but high density in the time domain
for proper tracking. PTRS is always transmitted together with DMRS.

Channel state information reporting

The channel estimate obtained by CSIRS can be reported back to gNB. The feedback includes

1. the index of a quantized version of the precoding matrix (in the transmit precoding matrix indicator,
TPMI) (see Section 5.6),
2. the number of suggested streams, denoted as rank indicator (RI) of the channel matrix (see (9.135)),
and
3. the SNR (channel quality indicator, CQI).

Various fixed TPMI codebooks are defined. For example, for a single-layer transmission using two
antenna ports2 the precoding codebook is
{ [ ] [ ] [ ] [ ] [ ] [ ]}
1 1 1 0 1 1 1 1 1 1 1
√ ,√ ,√ , ,√ ,√ (18.1)
2 0 2 1 2 1 −1 2 j 2 −j
while for two two-layers over two antennas is
{ [ ] [ ] [ ]}
1 1 0 1 1 1 1 1 1
√ ,√ ,√ (18.2)
2 0 1 2 1 −1 2 j −j
The complete set of codebooks is provided in [2].

18.1.5 Uplink
The physical channels defined in the uplink are as follows:

● Physical random access channel (PRACH),

● Physical uplink shared channel (PUSCH), and
● Physical uplink control channel (PUCCH).

2
NR standard refers to antenna ports rather than to antennas.
866 Chapter 18. Examples of communication systems

The PRACH is used to let a UE initiate a communication with the gNB, and carries control signals.
The PUSCH carries the UE data, and its transmit chain is described in Section 18.1.3, where transform
precoding can be optionally used. The PUCCH carries control information such as the feedback hybrid
automatic repeat request (HARQ), channel state information, and scheduling requests. PUCCH includes
both control information and DMRS. For a structure of the PUCCH, see Figure 18.4b.
The physical signals defined in the uplink are as follows:
● DMRSs,
● PTRSs, and
● Sounding reference signal (SRS).
DMRS and PTRS are similar to those available in downlink, and they provide information only for the
RBs used by the UE for data transmission. The SRS provides information on the whole bandwidth and is
used by the gNB to schedule uplink (and downlink when TDD is used) transmissions in the forthcoming
frames.

Transform precoding numerology

OFDM with DFT spreading (see Section 8.10.2) is used on QPSK-modulated symbols in the uplink
PUSCH. Spreading (transform precoding) is obtained by a DFT of size N = 12 ⋅ N, with N = 2𝛼2 ⋅ 3𝛼3 ⋅
5𝛼5 , where 𝛼2 , 𝛼3 , and 𝛼5 are non-negative integers. The resulting possible values of the DFT size are
reported in Table 18.2. Since DFT sizes are product of powers of 2, 3, and 5, an efficient fast Fourier
transform (FFT) implementation is available (see Section 1.2).

Table 18.2: DFT size for transform-precoding.

N 1 2 3 4 5 6 8 9 10 12 15 16
N 12 24 36 48 60 72 96 108 120 144 180 192
N 18 20 24 25 27 30 32 36 40 45 48 50
N 216 240 288 300 324 360 384 432 480 540 576 600
N 54 60 64 72 75 80 81 90 96 100
N 648 720 768 864 900 960 972 1080 1152 1200

Channel estimation
The DMRS in uplink is similar to that of downlink. However, in this case, it can be generated either
starting from a Gold sequence or from a constant-amplitude zero autocorrelation (CAZAC) (also known
as Zadoff-Chu) sequence.
SRS provides uplink channel estimation at the gNB, when in connected mode. The estimated channel
is used for equalization in uplink but also, when channel reciprocity can be exploited, to design downlink
precoding. The SRS is specific for each UE, with a variety of sequence design. Moreover, since it must
cover the whole communication band (not only the RB used for data transmission), frequency hopping
mechanisms are defined to obtain a channel estimate on different frequencies over various slots.

Synchronization
The PRACH is used to access the network by the UE. In particular, to start the transmission the UE
first sends, within the PRACH, a random-access preamble containing CAZAC sequences, of either 139
or 839 bits, for large and small cells, respectively. For the small-cell configuration, it is also possible
18.1. The 5G cellular system 867

3 ms

1 ms
Format 0
CP SEQ GP
1.25 kHz

Format 1
1.25 kHz CP SEQ SEQ GP

Format 3
5 kHz CP SEQ SEQ SEQ SEQ GP

Figure 18.7 PRACH formats (long sequence).

to transmit special OFDM symbols whose last part operates as CP of the next OFDM symbol, and
the length of the OFDM preamble equals the length of OFDM data. With this arrangement, the gNB
uses the same DFT for data and random-access preamble detection. Moreover, by adding the various
replicas of the preamble, synchronization is more robust to time varying channels and frequency errors.
An example of PRACH format is shown in Figure 18.7. Note that the structure is similar to that defined
for the Schmidl and Cox algorithm (see Section 14.9.1) for time synchronization.
Upon transmission of the preamble, the UE can apply different antenna precoders, thus the same
preamble can be received from different angular direction (beams) by the gNB. For mmWave with beam
sweeping, within PRACH the preamble is an indication of the precoding index to be used by the gNB
in downlink.

Timing advance
Once downlink synchronization is achieved, the UE knows the timing of each slot. However, a message
transmitted at time t by the UE will reach gNB at time t + t0 , with t0 being the propagation time from
the UE to gNB. Note that each UE will have its own propagation time, therefore the scheduled time for
uplink transmission will be completely modified.
In order to solve this issue, the timing advance procedure is used, by which the gNB measures the
effective time t + t0 of reception of the message, and compares it with the expected time t. Then, it
sends a signalling message to the UE with a request to anticipate transmission by t0 in the future. For
next uplink transmissions scheduled at time t′ , the UE will start transmitting at time t′ − t0 so that it is
received by gNB at the correct time t′ .
The timing advance value is measured upon reception of messages on the PRACH channel, wherein
the UE indicates the transmission time.

18.1.6 Network slicing

Network slicing is a key concept of 3GPP 5G system. It refers to the capability of the cellular network
(including both the RAN and the core network) to support simultaneous services with different qualities.
Each service is associated to a slice, and the 3GPP currently defines three slice types: enhanced mobile
broadband (eMBB), massive machine type communications (mMTC), and ultra reliable low latency
communications (URLLC). They differ in terms of data rate, number of users per unit area that can be
868 Chapter 18. Examples of communication systems

served, latency, and other metrics. A slice describes whatever is needed in terms of network resources
and configuration to provide the requested performance metrics.

User equipment (UE) 5G core network (5GC)

New generation
PLMN A
radio access network
(NG RAN) Slice ID = X
ce specific core
m
ce# network functions
sli
CN
Slice specific core
Smartphone NG RAN node Slic
netw network functions
Slice ID = X (gNB or eLTE eNB) N2 Common func
Slice specific core
network network functions
(PLMN A) Radio
Sl
Electronic (CP itf) Slic
ne
fun

news resources functions netw

func Slice CP Slice CP
network network

gathering i
function l function n

RAN slice#
Sl
ne
fun
device Air N3 Slice UP Slice UP
network network
itf (UP itf)
Layer 1
Layer 2
Layer 3
function a function x

PS terminal

N2
RAN slic Slice specific core Slice ID = Y
e#j N3
IoT device etwork functions
Slice ID = X n
ce#
(PLMN B) sli Sl
Slice specific core
CN
n
fu
network functions
Xn Common
Slice ID=Z Sl

(PLMN B) (To other NG network n

fu
Slice CP
network
function l
Slice CP
network
function n

RAN nodes) functions

Connected car
CN slic Slice UP Slice UP
vendor A e#function
pnetworka network
function x

Connected car Slice ID = Z

vendor B 5G core network (5GC)
PLMN B

Figure 18.8 Illustration of the network slicing architecture. Source: Ferrus et al. 2018 [5].
Reproduced with permission from IEEE.

A single end-user device may run multiple services, thus being associated to different slices. For
example, a smartphone may be equipped with sensors that are performing machine-to-machine com-
munication, thus being connected by an mMTC slice to the cellular network, while at the same time, a
video streaming on the same smartphone will be connected by an eMBB slice.
Figure 18.8 shows an example of network slicing architecture. We note that a slice affects both the
RAN and the core network, as interventions at all layers and end-to-end are needed in general to ensure
the requested quality of service. The figure shows a NR RAN node serving two public land mobile
networks (PLMNs), thus two (virtual) operators. Network slicing is enabled by an evolution toward
software control of the network (software defined networking) and by the possibility to make available
specific network functions to the operator, regardless of the specific hardware/software implementation,
in what is called network function virtualization (NFV).
At the time of writing, 3GPP has not yet defined the network slicing functionalities for the RAN. They
will include a list of capabilities of gNBs, and lists of offered services, together with entities that manage
the activation of slices at gNBs and their mobility functions.

18.2 GSM

The global system for mobile communications (GSM) was started in the early 1990s with the objective
of providing one standard within Europe, and belongs to the so-called second generation of cellular stan-
dards. Although various generations followed, GSM has been widely in use for its ability to efficiently
cover large area and provide basic data services suitable for the IoT.
18.2. GSM 869

The services that the GSM system offers are as follows [6]:
● Telephony service: Digital telephone service with guaranty of service to users that move at a speed
of up to 250 km/h;
● Data service: Can realize the transfer of data packets with bit rates in the range from 300 to 9600
bit/s.
● ISDN service: Some services, as the identification of a user that sends a call and the possibility of
sending short messages (SMSs), are realized by taking advantage of the integrated services digital
network (ISDN), whose description can be found in [7].
A characteristic of the GSM system is the use of the subscriber identity module (SIM) card together
with a four-digit number (ID); inserting the card in any mobile terminal, it identifies the subscriber who
wants to use the service. Important is also the protection of privacy offered to the subscribers of the
system.
Figure 18.9 represents the structure of a GSM system, that can be subdivided into three subsystems.

BTS

MS BTS BSC

BTS HLR VLR AUC

PSTN

MS BTS
ISDN
BTS BSC MSC

BTS Data
network

OMC

Radio subsystem Operation support Public networks

subsystem

Figure 18.9 GSM system structure.

The first subsystem, composed of the set of BTSs and mobile terminals or mobile stations (MSs), is
called radio subsystem; it allows the communication between the MSs and the mobile switching centre
(MSC), that coordinates the calls and also other system control operations. To an MSC are linked many
base station controllers (BSCs); each BSC is linked up to several hundreds BTSs, each of which iden-
tifies a cell and directly realizes the link with the mobile terminal. The hand-off procedure between two
BTSs is assisted by the mobile terminal in the sense that it is a task of the MS to establish at any instant
which BTS is sending the strongest signal. In the case of the hand-over between two BTS linked to the
same BSC, the entire procedure is handled by the BSC itself and not by the MSC; in this way, the MSC
can save many operations.
The second subsystem is the network switching subsystem (NSS), that in addition to the MSC includes:
● Home location register (HLR): It is a database that contains information regarding subscribers that
reside in the same geographical area as the MSC;
● Visitor location register (VLR): It is a database that contains information regarding subscribers that
are temporarily under the control of the MSC, but do not reside in the same geographical area;
870 Chapter 18. Examples of communication systems

● Authentication centre (AUC): It controls codes and other information for correct communications
management;
● Operation maintenance centres (OMCs): They take care of the proper functioning of the various
blocks of the structure.
Lastly, the MSC is directly linked to the public networks: public switched telephone network (PSTN)
for telephone services, ISDN for particular services as SMS, and data network for the transmission of
data packets. Note that GSM offers a rigid network structure, while 5G system has converted all its
subsystems into flexible software network functions to be used dynamically.

Radio subsystem
We now give some details with regard to the radio subsystem. The total bandwidth allocated for the
system is 50 MHz; frequencies that go from 890 to 915 MHz are reserved for MS-BTS communi-
cations, whereas the bandwidth 935–960 MHz is for communications in the opposite direction.3 In
this way, a full-duplex communication by FDD is realized. Within the total bandwidth, there are 248
carriers allocated, that identify as many frequency channels called absolute radio frequency channel
number (ARFCN); of these, 124 are for uplink communications and 124 for downlink communica-
tions. The separation between two adjacent carriers is 200 kHz; the bandwidth subdivision is illustrated
in Figure 18.10. Full-duplex communication is achieved by assigning two carriers to the user, one for
transmission and one for reception, such that they are about 45 MHz apart.

f1 f2 f3 f124

fi+1 – fi = 200 kHz

f

1 2 3 124
subchannel number

25 MHz bandwidth

Figure 18.10 Bandwidth allocation of the GSM system.

Each carrier is used for the transmission of an overall bit rate Rb of 270.833 kbit/s, corresponding to a
bit period Tb = 3.692 μs. The system employs Gaussian minimum shift-keying (GMSK) with parameter
Bt T equal to 0.3; the aim is to have a power efficient system. However, the spectral efficiency is not very
high; in fact we have
270.833
𝜈= ≃ 1.354 bit/s/Hz (18.3)
200
which is smaller than that of other systems.
Besides this frequency division multiplexing (FDM) structure, there is also a time division multiple
access (TDMA) structure; each transmission is divided into eight time intervals, or time slots, that iden-
tify the time division multiplexing (TDM) frame. Figure 18.11 shows the structure of a frame as well as
that of a single time slot.
As a slot is composed of 156.25 bits (not an integer number because of the guard time equal to 8.25
bits), its duration is about 576.92 μs; therefore, the frame duration is about 4.615 ms. In the figure, it is

3
There also exists a version of the same system that operates at around the frequency of 1.8 GHz (in the United States of
America the frequency is 1.9 GHz).
18.2. GSM 871

4.6155 ms

frame
1 2 3 4 5 6 7 8

slot
T F train F data bits T guard
3 1 26 1 57 3 time

0.577 ms

T: head bits
F: flag bits
train: training sequence

Figure 18.11 TDM frame structure and slot structure of the GSM system.

important to note the training sequence of 26 bits, used to analyse the channel by the MS or BS. The
flag bits signal if the 114 information bits are for voice transmission or for control of the system. Lastly,
the tail bits indicate the beginning and the end of the frame bits.
Although transmissions in the two directions occur over different carriers, to each communication is
dedicated a pair of time slots spaced 4 slots apart (one for the transmit station and one for the receive
station); for example the first and fifth or the second and sixth etc. Considering the sequence of 26
consecutive frames, that have a duration of about 120 ms, the 13th and 26th frames are used for control;
then in 120 ms, a subscriber can transmit (or receive) 114 ⋅ 24 = 2736 bits, which corresponds to a bit
rate of 22.8 kbit/s. Indeed the net bit rate of the message can be 2.4, 4.8, 9.6, or 13 kbit/s. Redundancy
bits are introduced by the channel encoder for protection against errors, so that we get a bit rate of 22.8
kbit/s in any case.
The original speech encoder chosen for the system was a residual-excited predictor, improved by a
long-term predictor, with a rate of 13 kbit/s. The use of a voice activity detector reduces the rate to a
minimum value during silence intervals. For channel coding, a convolutional encoder with code rate 1/2
is used. In [7], the most widely used speech and channel encoders are described, together with the data
interleavers.
In Figure 18.12, a scheme is given that summarizes the protection mechanism against errors used
by the GSM system. The speech encoder generates, in 20 ms, 260 bits; as the bit rate per subscriber is
22.8 kbit/s, in 20 ms 456 bits must then be generated, introducing a few redundancy bits.4
To achieve reliable communications in the presence of multipath channels with delay spread up to
16 μs, at the receiver equalization by a decision feedback equalizer (DFE) and/or detection by the Viterbi
algorithm are implemented.
In [7, 8], the calling procedure followed by the GSM system and other specific details are fully
described.

4
The described slot structure and coding scheme refer to the transmission of user information, namely speech. Other
types of communications, as for the control and management of the system, use different coding schemes and different
time slot structures (see [8]).
872 Chapter 18. Examples of communication systems

78 least-significant bits

voice 132 significant bits

encoder

convolu- 456
compute bits
53 bits tional inter-
50 essential 3 parity
encoder 378 bits leaving
bits bits
rate 1/2

78 + 132 + 50 = 260 bits in 20 ms ‘0000’

from the voice encoder

Figure 18.12 Channel coding for the GSM system.

18.3 Wireless local area networks

For the transmission at bit rates of the order of 1–10 Mbit/s, WLANs normally use the industrial,
scientific, and medical (ISM) frequency bands defined by the United States Federal Communications
Commission (FCC), that is 902–928 MHz, 2400–2483.5 MHz, and 5725–5850 MHz. Specifications for
physical and MAC layers of WLANs are developed by various standardization organizations, among
which we cite the IEEE 802.11 Working Group and the European telecommunications standard institute
(ETSI).
In the United States, WLANs are allowed to operate in the industrial, scientific, and medical (IMS)
frequency bands without the need of a license from the authorities, which, however, sets restrictions on
the power of the radio signal that must be less than 1 W and specifies that spread-spectrum technology
(see Chapter 10) must be used whenever the signal power is larger than 50 mW. Most WLANs employ
direct sequence or frequency hopping spread-spectrum systems; WLANs that use narrowband modu-
lation systems usually operate in the band around 5.8 GHz, with a transmitted signal power lower than
50 mW, in compliance with FCC regulations. The coverage radius of WLANs is typically of the order
of a few hundred metres.

Medium access control protocols

Unlike cabled LANs, WLANs operate over channels with multipath fading, and channel characteris-
tics typically vary over short distances. Channel monitoring to determine whether other stations are
transmitting requires a larger time interval than that required by a similar operation in cabled LANs;
this translates into an efficiency loss of the carrier sense multiple access (CSMA) protocols, whenever
they are used without modifications. The MAC layer specified by the IEEE 802.11 Working Group is
based on the CSMA protocol with collision avoidance (CSMA/CA), in which four successive stages are
foreseen for the transmission of a data packet, as illustrated in Figure 18.13 [9].
The CSMA/CA principle is simple. All mobile stations that have packets to transmit compete for
channel access by sending request to send (RTS) messages by a CSMA protocol. If the base stations
recognize a RTS message sent by a mobile station, it sends a clear to send (CTS) message to the same
mobile station and this one transmits the packet; if reception of this packet occurs correctly, then the
base station sends an acknowledgement (ACK) message to the mobile station. With CSMA/CA, the
only possibility of collision will occur during the RTS stage of the protocol; however, we note that also
18.4. DECT 873

Base station Mobile station

RTS

CTS

DATA

ACK

Figure 18.13 Illustration of the CSMA/CA protocol.

the efficiency of this protocol is reduced with respect to that of the simple CSMA/CD because of the
presence of the RTS and CTS stages.
As mobility is allowed, the number of mobile stations that are found inside a cell can change at any
instant. Therefore, it is necessary that each station informs the others of its presence as it moves around.
A protocol used to solve this problem is the so-called hand-off, or hand-over, protocol, which can be
described as follows:
● a switching station, or all base stations with a coordinated operation, registers the information rel-
ative to the signal levels of all mobile stations inside each cell;
● if a mobile station M is serviced by base station B1, but the signal level of station M becomes larger
if received by another base station B2, the switching station proceeds to a hard hand-off operation
whose final result is that mobile station M is considered part of the cell covered by base station B2.

18.4 DECT

The digital European cordless telecommunications (DECT) system is a cordless digital system used in
Europe for speech transmission. Unlike cellular systems that use cells with radius of the order of a few
kilometres, the DECT system is mainly employed indoor and the cell radius is at most of a few tens of
meters (typically 100 m). The main characteristics of DECT are summarized in Table 18.3.

Table 18.3: Table summarizing the main characteristics of DECT.

Frequency range 1880–1900 MHz

RF channel spacing 1728 kHz
Modulation GMSK
Transmission bit rate 1152 kbit/s
Voice encoding method 32 kbit/s ADPCM
Access method FDMA/TDMA/TDD
Frame duration 10 ms (24 time slots)
Subscriber TX peak power 250 mW
Radius of service 100–150 m
Frequency planning Dynamic channel allocation
874 Chapter 18. Examples of communication systems

While originally the hand-off problem was not considered, as each MS corresponded to one BS only,
now even for DECT we speak of hand-off assisted by the mobile terminal, such that the system config-
uration is similar to that of a cellular system (see Figure 18.9). An interesting characteristic is the use
of the dynamic channel selection (DCS) algorithm, that allows the portable to know at every moment
which channel (frequency) is the best (with the lowest level of interference) for communication and
select it. We briefly illustrate the calling procedure followed by the system:
1. when a mobile terminal wishes to make a call, it first measures the received signals from the various
BS5 and selects the one which yields the best signal level;
2. by the DCS algorithm, the mobile terminal selects the best free channels of the selected BS;
3. MS sends a message, called access request, over the least interfered channel;
4. BS sends (or not) an answer: access granted;
5. if the MS receives this message, in turn it transmits the access confirm message and the commu-
nication starts;
6. if the MS does not receive the access granted signal on the selected channel, it abandons this
channel and selects the second least interfered channel, repeating the procedure; after failing on
five channels, the MS selects another BS and repeats all operations.
The total band allocated to the system goes from 1880 to 1900 MHz and is subdivided into ten subbands,
each with a width of 1.728 MHz; this is the frequency division multiple access (FDMA) structure of
DECT represented in Figure 18.14. Each channel has an overall bit rate of 1.152 Mbit/s, that corresponds
to a bit period of about 868 ns.

f1 f2 f3 f4 f5 f6 f7 f8 f9 f10

1.728 MHz

1880 MHz fi = 1881.792 + (i–1)x1.728 MHz 1900 MHz

Figure 18.14 FDMA structure of the DECT system.

Similarly to the other systems, TDMA is used. The TDM frame, given in Figure 18.15, is composed of
24 slots. The first 12 are used for the communication BS-MS, the other 12 for the reverse communication;
thus, we realize a full-duplex communication by TDD. In this DECT differs from all above considered
wireless systems which use FDD; DECT allocates half of the frame for transmission and the other half
for reception.
In Figure 18.15, the slot structure is also shown; it is composed of 480 bits of which the first 32
are fixed and correspond to a synchronization word, the successive 388 are information bits, and the
remaining 60 constitute the guard time. The frame has a duration of 480 ⋅ 24∕1152 = 10 ms. The field
of 388 bits reserved for information bits is subdivided into two subfields A and B. The first (64 bits)

5
For DECT, the acronyms portable handset (PH) in place of MS and radio fixed part (RFP) in place of BS, are
often used.
18.6. Transmission over unshielded twisted pairs 875

10 ms

base station-mobile mobile-base station

1 12 13 24 frame

guard
sinc subfield A subfield B
time slot
12 bits 64 bits 324 bits
60 bits

data field: 388 bits

0.417 ms

Figure 18.15 TDM frame structure and slot structure for the DECT system.

contains information for signalling and control of the system, the second (324 bits) contains user data.
If the signal is speech, 4 of these 324 bits are parity bits, which translates into a net bit rate of 320 bits
in a frame interval of 10 ms, and therefore in a bit rate of 32 kbit/s; an adaptive differential pulse code
modulation voice encoder at 32 kbit/s is used and no channel coding is provided.
For transmission, GMSK with parameter Bt T of 0.5 is adopted. At the MS receiver, a pair of antennas
are often used to realize switched antenna diversity. With this mechanism, fading and interference are
mitigated. We summarize in Table 18.4 the characteristics of the various standards so far discussed.

18.5 Bluetooth

The Bluetooth standard [10, 11] is designed for data transmission. It operates around the frequency of
2.45 GHz, in the unlicensed and open ISM band. Bluetooth uses a frequency hopping (FH) /TDD scheme
with Gaussian shaped frequency-shift-keying (FSK) (h ≤ 0.25). The modulation rate is 1 MBaud with a
slot of 625 μs; a different hop channel is used for each slot. This gives a nominal hop rate of 1600 hops/s.
There are 79 hop carriers spaced by 1 MHz. The maximum bit rate of Bluetooth 1.0 is of 723.2 kbit/s in
one direction and 57.6 kbit/s in the reverse direction. Evolved versions of Bluetooth achieve data rates
up to 25 Mbit/s.

18.6 Transmission over unshielded twisted pairs

We now provide two examples of transmission systems over unshielded twisted pairs (UTPs), namely
the DSL, for communication in the customer service area, and the Ethernet, for communications in local
area networks, technologies.
876 Chapter 18. Examples of communication systems

Table 18.4: Summary of characteristics of the standards GSM and DECT.

System GSM DECT

Multiple access TDMA/FDMA TDMA/FDMA
Band (MHz)
Downlink 935–960 1880–1900
1805–1880
Uplink 890–915 1880–1900
1710–1785
Duplexing FDD TDD
Spacing between carriers 200 kHz 1728 kHz
Modulation GMSK 0.3 GMSK 0.5
Bit rate per carrier (kbit/s) 270.833 1152
Speech encoder RPE-LTP ADPCM bit rate (kbit/s)
13 32
Convolutional
encodera) 1/2 none
code rate 1/3 (R)
Frame duration (ms) 4.615 10
a) All these standards use a code redundancy check (CRC) code, possibly
together with a convolutional code.

18.6.1 Transmission over UTP in the customer service area

In customer service areas, UTP cables also represent a low-cost alternative to optical fibres for links
that allow data transmission at a considerably higher bit rates than those achievable by modems for
transmission over the telephone channel, over distances that can reach 6 km; in fact, although optical
fibres have substantially better transmission characteristics (see Chapter 4), a reliable link over a local
loop is preferable in many cases given the large number of already installed cables [12–14].
The various DSL technologies (in short xDSL) allow full-duplex transmission between user and cen-
tral office at bit rates that may be different in the two directions [15, 16]. For example, the high bit
rate digital subscriber line (HDSL) offers a solution for full-duplex transmission at a bit rate of 1.544
Mbit/s, also called T1 rate (see Section 18.8), over two twisted pairs and up to distances of 4500 m; the
single-line high-speed DSL (SHDSL) provides full duplex transmission at rates up to 2.32 Mbit/s over
a single twisted pair, up to distances of 2000 m.
A third example is given by the asymmetric digital subscriber line (ADSL) technology (see
Figure 18.16); originally, ADSL was proposed for the transmission of video-on-demand signals; later
it emerged as a technology capable of providing a large number of services. For example, ADSL-3
is designed for downstream transmission of four compressed video signals, each having a bit rate of
1.5 Mbit/s, in addition to the full-duplex transmission of a signal with a bit rate of 384 kbit/s, a control
signal with a bit rate of 16 kbit/s, and an analog telephone signal, up to distances of 3600 m.
A further example is given by the very high-speed DSL (VDSL) technology, mainly designed for
the fibre-to-the-curb (FTTC) architecture. The considered data rates are up to 26 Mbit/s downstream
i.e. from the central office or optical network unit to the remote terminal, and 4.8 Mbit/s upstream for
asymmetric transmission, and up to 14 Mbit/s for symmetric transmission, up to distances not exceeding
a few hundred meters. Figure 18.17 illustrates the FTTC architecture, where the link between the user
18.6. Transmission over unshielded twisted pairs 877

Local office

Asymmetric digital subscriber lines (ADSL)

1.544 / 2.048 Mbit/s, 16 kbit/s

ADSL–1
modem

18 kfeet = 5.4 km
ADSL–1
modem
Optical network unit
3.152 Mbit/s,
ONU n × 64 kbit/s
Fibre ADSL–2
ADSL–2
modem modem

12 kfeet = 3.6 km

Fibre 6.321 Mbit/s,

ONU N × 64 kbit/s
ADSL–3 ADSL–3
modem modem

6 kfeet = 1.8 km

Figure 18.16 Illustration of links between ADSL modems over the subscriber line.

and an optical network unit (ONU) is obtained by a UTP cable with maximum length of 300 m, and the
link between the ONU and a local central office is realized by optical fibre; in the figure, links between
the user and the ONU that are realized by coaxial cable or optical fibre are also indicated, as well as
the direct link between the user and the local central office using optical fibre with a fibre-to-the-home
(FTTH) architecture.
Different baseband and passband modulation techniques are considered for high-speed transmission
over UTP cables in the customer service area. For example, the Study Group T1E1.4 of Committee T1
chose 2B1Q quaternary pulse amplitude modulation (PAM), see Example 16.1.1 on page 793, for HDSL,
and DMT modulation (see Chapter 8) for ADSL. Among the organizations that deal with the standard-
ization of DSL technologies we also mention the Study Group TM6 of ETSI and the Study Group 15 of
the ITU-T. A table summarizing the characteristics of DSL technologies is given in Table 18.5; spectral
allocations of the various signals are illustrated in Figure 18.18.
We now discuss more in detail the VDSL technology. Reliable and cost effective VDSL transmission
at a few tens of Megabit per second is made possible by the use of FDD (see Appendix 18.A), which
avoids signal disturbance by near-end crosstalk (NEXT), a particularly harmful form of interference at
VDSL transmission frequencies. Ideally, using FDD, transmissions on neighbouring pairs within a cable
binder couple only through far-end crosstalk (FEXT) (see also Section 17.4), the level of which is signif-
icantly below that of NEXT. In practice, however, other forms of signal coupling come into play because
upstream and downstream transmissions are placed spectrally as close as possible to each other in order
to avoid wasting useful spectrum. Closely packed transmission bands exacerbate interband interference
by echo and NEXT from similar systems (self-NEXT), possibly leading to severe performance degra-
dation. Fortunately, it is possible to design modulation schemes that make efficient use of the available
878 Chapter 18. Examples of communication systems

Local office

51.84 Mbit/s,
Very high speed DSL (VDSL)
N × 64 kbit/s
VDSL
modem
ONU
VDSL
modem
FTTC 300 m

Fibre Cable Coax Cable

modem modem

Fibre
modem
Fibre

Fibre
modem

FTTH
Fibre Fibre
modem

Figure 18.17 Illustration of FTTC and FTTH architectures.

Table 18.5: Characteristics of DSL technologies.

acronym standard modulation bit rate (Mbit/s) distance (m)

basic rate ISDN 2B1Q 0.144 ≤ 6000
HDSL G.991.1 2B1Q 1.544, 2.048 ≤ 4000
SHDSL G.shdsl TC-PAM 0.192–2.32 ≤ 2000
ADSL G.992.1 DMT downstream 6.144 ≤ 3600
upstream 0.640
ADSL lite G.992.1 DMT downstream 1.5 best effort service
upstream 0.512
VDSL downstream ≤ 26 ≤ 1500
upstream ≤ 14

spectrum and simultaneously achieve a sufficient degree of separation between transmissions in oppo-
site directions by relying solely on digital signal processing techniques. This form of FDD is sometimes
referred to as digital duplexing.
The concept of divide and conquer has been used many times to facilitate the solution of very com-
plex problems; therefore, it appears unavoidable that digital duplexing for VDSL will be realized by the
18.6. Transmission over unshielded twisted pairs 879

PSD (dBm/Hz)
BR-ISDN
–30

–40 HDSL
ADSL UPSTREAM

ADSL DOWNSTREAM
–50

–60

0.1 0.5 1 frequency

(a) (MHz)

PSD (dBm/Hz)
VDSL
–60

DOWN UP DOWN UP

20 frequency
(b) (MHz)

Figure 18.18 Spectral allocation of signals for (a) HDSL and ADSL, and (b) VDSL.

sophisticated version of this concept represented by multicarrier transmission, although single carrier
methods have been also proposed. As discussed in Section 8.5, various variants of multicarrier transmis-
sion exist. The digital duplexing method for VDSL known as Zipper [17] is based on discrete multitone
(DMT) modulation; here, we consider filtered multitone (FMT) modulation (see Section 8.2), which
involves a different set of trade-offs for achieving digital duplexing in VDSL and offers system as well
as performance advantages over DMT [18, 19].
The key advantages of FMT modulation for VDSL can be summarized as follows. First, there is
flexibility to adapt to a variety of spectrum plans for allocating bandwidth for upstream and downstream
transmission by proper assignment of the subchannels. This feature is also provided by DMT modulation,
but not as easily by single-carrier modulation systems. Second, FMT modulation allows a high-level of
subchannel spectral containment and thereby avoids disturbance by echo and self-NEXT. Furthermore,
disturbance by a narrowband interferer, e.g. from amplitude modulation (AM) or amateur radio sources,
does not affect neighbouring subchannels as the side lobe filter characteristics are significantly attenu-
ated. Third, FMT modulation does not require synchronization of the transmissions at both ends of a
link or at the binder level, as is sometimes needed for DMT modulation.
As an example of system performance, we consider the bit rate achievable for different values of the
length of a twisted pair, assuming symmetrical transmission at 22.08 MBaud and full-duplex transmis-
sion by FDD based on FMT modulation. The channel model is obtained by considering a line with
880 Chapter 18. Examples of communication systems

attenuation equal to 11.1 dB/100 m at 11.04 MHz, with 49 near-end crosstalk interference signals, 49
far-end crosstalk interference signals, and additive white Gaussian noise with a power spectral density
(PSD) of −140 dBm/Hz. The transmitted signal power is assumed equal to 10 dBm. The FMT system
considered here employs the same linear-phase prototype filter for the realization of transmit and receive
polyphase filter banks, designed for  = 256, K = 288, and polyphase filter length 𝛾 = 10; we recall
that with these values of  and K the excess bandwidth within each subchannel is equal to 12.5%.
Per-subchannel equalization is obtained by a Tomlinson–Harashima precoder (THP) (see Section 7.7.1)
with 9 coefficients at the transmitter and a fractionally spaced linear equalizer with 26 T∕2 spaced coef-
ficients at the receiver. With these parameter values, and using the bit loading technique of Chapter 13,
the system achieves bit rates of 24.9, 10.3, and 6.5 Mbit/s for the three lengths of 300, 1000, and 1400 m,
respectively [20]. We refer to Section 17.4 for a description of the general case where users are connected
at different distances from the central office and power control is applied.

18.6.2 High-speed transmission over UTP in local area networks

High-speed transmission over UTP cables installed in buildings is studied by different standardization
organizations. For example, the asynchronous transfer mode (ATM) Forum and the Study Group 13 of
the ITU-T consider transmission at bit rates of 155.52 Mbit/s and above for the definition of the ATM
interface between the user and the network. The IEEE 802.3 Working Group investigates the transmis-
sion at 1 Gbit/s over four twisted pairs, type UTP-5, for Ethernet (1000BASE-T) networks.
UTP cables were classified by the electronic industries association and telecommunications indus-
tries association (EIA/TIA) according to the transmission characteristics (see Chapter 4). We recall that
UTP-3, or voice-grade, cables exhibit a signal attenuation and a crosstalk coupling much greater than
that of UTP-5, or data-grade, cables. For local area network (LAN) applications, the maximum cable
length for a link between stations is 100 m. Existing cabling systems use bundles of twisted pairs, usually
4 or 25 pairs, and signals may cross line discontinuities, represented by connectors. For transmission
over UTP-3 cables, in order to meet limits on emitted radiation, the signal band must be confined to fre-
quencies below 30 MHz and sophisticated signal processing techniques are required to obtain reliable
transmission.
Standards for Ethernet networks that use the carrier sense multiple access with collision detection
(CSMA/CD) protocol are specified by the IEEE 802.3 Working Group for different transmission media
and bit rates. With the CSMA/CD protocol, a station can transmit a data packet only if no signal from
other stations is being transmitted on the transmission medium. As the probability of collision between
messages cannot be equal to zero because of the signal propagation delay, a transmitting station must
continuously monitor the channel; in the case of a collision, it transmits a special signal called jam signal
to inform the other stations of the event, and then stops transmission. Retransmission takes place after
a random delay. The 10BASE-T standard for operations at 10 Mbit/s over two unshielded twisted pairs
of category 3 or higher defines one of the most widely used implementations of Ethernet networks; this
standard considers conventional mono duplex (see Section 17.1) transmission, where each pair is utilized
to transmit only in one direction using simple Manchester line coding (see Appendix 7.C) to transmit
data packets, as shown in Figure 18.19. Transmitters are not active outside of the packets transmission
intervals, except for transmission of a signal called link beat that is occasionally sent to assure the link
connection.
The request for transmission speeds higher than 10 Mbit/s motivated the IEEE 802.3 Working Group
to define standards for fast Ethernet that maintain the CSMA/CD protocol and allow transmission at
100 Mbit/s and above. For example, the 100BASE-FX standard defines a physical layer (PHY) for Eth-
ernet networks over optical fibres. The 100BASE-TX standard instead considers conventional mono
18.7. Hybrid fibre/coaxial cable networks 881

Manchester−coded binary modulation or idle (no signal)

1 0 0 1 1 0 1
Idle Idle

Tb = 100 ns (bit interval)

Spectrum

0 1/Tb = 10 MHz f

Figure 18.19 Illustration of 10BASE-T signal characteristics.

duplex transmission over two twisted pairs of category 5; the bit rate of 100 Mbit/s is obtained by
transmission with a modulation rate of 125 MBaud and multilevel transmission (MLT-3) line coding
combined with a channel code with rate 4/5 and scrambling, as illustrated in Figure 18.20. We also
mention the 100BASE-T4 standard, which considers transmission over four twisted pairs of category 3;
the bit rate of 100 Mbit/s is obtained in the following way by using an 8B6T code with a modulation rate
equal to 25 MBaud. On the first two pairs, data are transmitted at a bit rate of 33.3 Mbit/s in half duplex
fashion, while on the two remaining pairs, data are transmitted at 33.3 Mbit/s in mono duplex fashion.
A further version of fast Ethernet is represented by the 100BASE-T2 standard, which allows users of
the 10BASE-T technology to increase the bit rate from 10 to 100 Mbit/s without modifying the cabling
from category 3 to 5, or using four pairs for a link over UTP-3 cables. The bit rate of 100 Mbit/s is
achieved with dual duplex transmission over two twisted pairs of category 3, where each pair is used
to transmit in the two directions (see Figure 18.21). The 100BASE-T2 standard represents the most
advanced technology for high-speed transmission over UTP-3 cables in LANs; the transceiver design
for 100BASE-T2 will be illustrated in Chapter 19.
Other examples of LANs are the token ring and the fibre distributed data interface (FDDI) networks;
standards for token ring networks were specified by the IEEE 802.5 Working Group, and standards for
FDDI networks were specified by the American National Standards Institute (ANSI). In these networks,
the access protocol is based on the circulation of a token. A station is allowed to transmit a data packet
only after having received the token; once the transmission has been completed, the token is passed to
the next station. The IEEE 802.5 standard specifies operations at 16 Mbit/s over two UTPs of category 3
or higher, with mono duplex transmission. For the transmission of data packets, Manchester line coding
is adopted; the token is indicated by coding violations, as illustrated in Figure 18.22. The ANSI standard
for FDDI networks specifies a physical layer for mono duplex transmission at 100 Mbit/s over two UTPs
of category 5 identical to that adopted for the Ethernet 100BASE-TX standard.
Table 18.6 summarizes the characteristics of existing standards for high-speed transmission over UTP
cables.

18.7 Hybrid fibre/coaxial cable networks

A hybrid fibre/coax (HFC) network is a multiple-access network, in which a head-end controller (HC)
broadcasts data and information for MAC over a set of channels in the downstream direction to a cer-
tain number of stations, and these stations send information to the HC over a set of shared channels
882 Chapter 18. Examples of communication systems

MLT–3 coding

0 +2
0
1 0 1
–2
0
T = 8 ns
–2 +2
0 Spectrum
1 0 1
0

0 62.5 MHz f
(a)

Medium
independent Physical Physical
interface medium medium
(MII) dependent dependent
(PMI) (PMD)
100 Mbit/s data
125 MBaud
or control info. 125 Mbit/s
MLT–3
Symbol
MLT–3
encoder
encoder
(rate–4/5)

Scrambling
sequence 2–pair UTP Cat. 5
generator

Symbol
MLT–3
decoder
decoder
(rate–4/5)

Scrambling
sequence
Clock
generator
recovery

(b)

Figure 18.20 Illustration of (a) 100BASE-TX signal characteristics and (b) 100BASE-TX
transceiver block diagram.

Tx Tx

pair 1
Tx Tx

Rc Rc
pair 2

Rc Rc

Figure 18.21 Illustration of a dual duplex transmission system.

18.7. Hybrid fibre/coaxial cable networks 883

Manchester−coded binary modulation with code violations (J,K)

0 1 1 1 0 J K 1 1

Tb = 62.5 ns
Spectrum
The code violations (J,K)
are used to mark the beginning
and end of 802.5 frames.
0 1/Tb = 16 MHz f

Figure 18.22 Illustration of signal characteristics for transmission over token ring networks.

Table 18.6: Scheme summarizing characteristics of standards for

high-speed transmission over UTP cables.

acronym standard bit rate (Mbit/s) cable type

Legacy LANs
10BASE-T Ethernet IEEE 802.3 10 2-pair UTP-3
16TR Token Ring IEEE 802.5 16 2-pair UTP-3
Fibre Distributed Data Interface
FDDI ANSI X3T9.5 100 2-pair UTP-5
Fast Ethernet
100BASE-TX IEEE 802.3 100 2-pair UTP-5
100BASE-T4 IEEE 802.3 100 4-pair UTP-3
100BASE-T2 IEEE 802.3 100 2-pair UTP-3
1000BASE-T4 IEEE 802.3 1000 4-pair UTP-5
AnyLAN
100 VG IEEE 802.12 100 4-pair UTP-3
ATM User-Network Interfaces
ATM-25 ATM Forum 25.6 2-pair UTP-3
ATM-51 ATM Forum 51.84 2-pair UTP-3
ATM-100 ATM Forum 100 2-pair UTP-5
ATM-155a ATM Forum 155.52 2-pair UTP-3
a
Cannot operate over UTP-3 in the presence of alien-NEXT interference

in the upstream direction [21, 22]. The topology of an HFC network is illustrated in Figure 18.23: an
HFC network is a point-to-multipoint, tree and branch access network in the downlink, with down-
stream frequencies in the 50–860 MHz band, and a multipoint-to-point, bus access network in the uplink,
with upstream frequencies in the 5–42 MHz band. Examples of frequency allocations are shown in
Figure 18.24. The maximum round-trip delay between the HC and a station is of the order of 1 ms.
The IEEE 802.14 Working Group is one of the standardization bodies working on the specifications for
the PHY and MAC layers of HFC networks. A set of PHY and MAC layer specifications adopted by
North American cable operators is described in [23].
In the downstream direction, transmission takes place in broadcast made over channels with band-
width equal to 6 or 8 MHz, characterized by low distortion and high SNR, typically ≥ 42 dB. The J.83
884 Chapter 18. Examples of communication systems

Fibre node
Head–end
controller FN

HC FN
Trunk = Fibre Feeder = Coaxial cable

FN
Termination
Splitter Bidirectional
Tap split-band
amplifier Drop = S
≤ 70 km coaxial cable
Station

≤ 10 km

Max. round trip delay: RTDmax ≈ 0.8 ms (= 2 × 80 km/200 000 km/s)

Figure 18.23 Illustration of the HFC network topology.

Upstream frequencies
Downstream frequencies
Analog and digital HDTV channels Switched digital video Data and
delivered to all of network services, delivered telephony
individually to each user
5 40 54 450 650 750 MHz
One down-stream
One up-stream 6 MHz channels system for all
system for all
digital services for analog TV digital services
5 40 54 450 750 MHz
Expected second generation HFC spectrum allocation

Figure 18.24 Examples of frequency allocations in HFC networks.

document of the ITU-T defines two QAM transmission schemes for transmission in the downstream
direction, with a bit rate in the range from 30 to 45 Mbit/s [24]; by these schemes, spectral efficiencies
of 5–8 bit/s/Hz are therefore obtained.
In the upstream direction, the implementation of PHY and MAC layers is considerably more difficult
than in the downstream. In fact, we can make the following observations:

● signals are transmitted in bursts from the stations to the HC; therefore, it is necessary for the HC
receiver to implement fast synchronization algorithms;
● signals from individual stations must be received by the HC at well-defined instants of arrival
and power levels; therefore, procedures are required for the determination of the round-trip delay
between the HC and each station, as well as for the control of the power of the signal transmitted by
each station, as channel attenuation in the upstream direction may present considerable variations,
of the order of 60 dB;
● the upstream channel is usually disturbed by impulse noise and narrowband interference signals;
moreover, the distortion level is much higher than in the downstream channel.
18.7. Hybrid fibre/coaxial cable networks 885

Interference signals in the upstream channel are caused by domestic appliances and high frequency
(HF) radio stations; these signals accumulate along the paths from the stations to the HC and exhibit time
varying characteristics; they are usually called ingress noise. Because of the high level of disturbance
signals, the spectral efficiency of the upstream transmission is limited to 2–4 bit/s/Hz.
The noise PSD suggests that the upstream transmission is characterized by the possibility of changing
the frequency band of the transmitted signal (frequency agility), and of selecting different modulation
rates and spectral efficiencies. In [23], a QAM scheme for upstream transmission that uses a 4 or 16
point constellation and a maximum modulation rate of 2.560 MBaud is defined; the carrier frequency,
modulation rate, and spectral efficiency are selected by the HC and transmitted to the stations as MAC
information.

Ranging and power adjustment in OFDMA systems

In some multiple-access systems, a HC broadcasts data and MAC information over a set of downlink
channels to several stations. These stations send information to the HC over a set of shared uplink
channels. Examples of systems exhibiting these characteristics are the two-way HFC systems. This
technology has been adopted the data over cable service interface specification (DOCSIS), originally
developed for cable modems. DOCSIS most recent version specifies OFDM downstream transmission
and OFDMA upstream transmission to achieve efficient use of the spectrum in both uplink and down-
link. In the uplink, implementation of physical (PHY) layer transmission and MAC layer functions pose
considerable technical challenges. In fact, individual station signals must be received at the HC at defined
arrival times and power levels. Therefore, it is important to determine the round-trip delay between the
HC and each individual station (ranging) as well as the individual transmit power control for each station
to compensate for widely varying attenuations in the uplink direction. In the following, we illustrate an
algorithm for initial ranging and power adjustment in OFDMA systems [25], which however may be
adapted also for SC systems.
At the beginning of operations, for the realization of the ranging and power adjustment process, a sta-
tion first tunes its receiver to a downlink channel that conveys MAC information and acquires the global
timing reference provided by the HC. Thereafter, the station sends a ranging-request message in the spec-
ified subchannel, which results in negligible interference in adjacent subchannels. If a ranging-response
message is not received, subsequent ranging-request messages are sent with increasing transmit power,
e.g. incremented by 1 dB steps. For ranging-request messages, one may consider a transmission format
with a preamble containing a CAZAC sequence [26] of length L, e.g. L = 16, which is repeated R times,
e.g. R = 8, followed by a special start-of-message (SOM) sequence.
Upon detection of the CAZAC sequence, the HC performs subchannel identification using, for
example, a least-squares algorithm, see Section 7.8. From the amplitude and phase characteristics of
the identified subchannel response, a transmit power level adjustment and a timing phase in [0, T] are
derived. The timing phase is needed to compute the round-trip delay compensation so that the MC
signal is received in proper synchronism. The detection of the signature sequence provides the HC with
further timing information that is used to determine the total round-trip delay from the head-end node
to the station, and hence the round-trip delay compensation.
After determining the transmit power level adjustment and the round-trip delay compensation, the
HC sends this information to the station as part of a ranging-response message. The station then waits
for a message from the HC with an individual station maintenance information and sends at the spec-
ified time a ranging-request message using the power level and timing corrections. The HC receives
the ranging-request message in proper synchronism at the filter bank output. The HC returns another
ranging-response message to the station with information about any additional fine-tuning required. The
ranging request/response steps are repeated until the response contains a Ranging Successful notification.
886 Chapter 18. Examples of communication systems

Ranging and power adjustment for uplink transmission

We describe now the registration procedure of a cable modem at the HC. The time base relative to the
upstream transmission is divided into intervals by the mechanism adopted by the HC for the allocation of
resources to the stations. Each interval is constituted by an integer number of subintervals usually called
mini-slots; a mini-slot then represents the smallest time interval for the definition of an opportunity for
upstream transmission.
In [23], a TDMA scheme is considered where uplink transmission is divided into a stream of
mini-slots. Each mini-slot is numbered relative to a master reference clock maintained by the HC. The
HC distributes timing information to the cable modems by means of time synchronization messages,
which include time stamps. From these time stamps, the stations establish a local time base locked to
the time base of the HC. For uplink transmission, access to the mini-slots is controlled by allocation
map (MAP) messages, which describe transmission opportunities on available uplink channels. A MAP
message includes a variable number of information elements (IE), each of which defines the modality
of access to a range of mini-slots in an uplink channel, as illustrated in Figure 18.25. Each station has a
unique address of 48 bits; with each active station is also associated, for service request, a 14-bit service
identifier (SID).

MAP transmitted on downstream channel by the HC

mini slots

CM tx opportunity request contention CM tx opportunity maintenance CM tx opp.

previous map current map future map

Figure 18.25 Example of a MAP message. Source: Reproduced with permission of Cable Television
Laboratories, Inc.

At the beginning of the registration procedures, a station tunes its receiver to the downstream channel
on which it receives SYNC messages from the HC; the acquired local timing is delayed with respect to
the HC timing due to the signal propagation delay. The station monitors the downstream channel until
it receives a MAP message with an IE of initial maintenance, which specifies the time interval during
which new stations may send a ranging request (RNG-REQ) message to join the network. The duration
of the initial maintenance interval is equivalent to the maximum round-trip delay plus the transmission
time of a RNG-REQ message. At the instant specified in the MAP message, the station sends a first
RNG-REQ message using the lowest power level of the transmitted signal, and is identified by a SID
equal to zero as a station that requests to join the network.
If the station does not receive a response within a pre-established time, it means that a collision
occurred between RNG-REQ messages sent by more than one station, or that the power level of the
transmitted signal was too low; to reduce the probability of repeated collisions, a collision resolution
protocol is used with random back-off. After the back-off time interval, of random duration, is elapsed,
the station waits for a new MAP message containing an IE of initial maintenance and at the specified
instant re-transmits a RNG-REQ message with a higher power level of the transmitted signal. These steps
18.7. Hybrid fibre/coaxial cable networks 887

are repeated until the HC detects a RNG-REQ message, from which it can determine the round-trip delay
and the correction of the power level that the station must apply for future transmissions. In particular,
the compensation for the round-trip delay is computed so that, once applied, the transmitted signals
from the station arrive at the HC at well-defined time instants. Then, the HC sends to the station, in a
ranging response (RNG-RSP) message, the information on round-trip delay compensation and power
level correction to be used for future transmissions; this message also includes a temporary SID.
The station awaits for a MAP message containing an IE of station maintenance, individually addressed
to it by its temporary SID, and in turn responds through a RNG-REQ message, signing it with its tempo-
rary SID and using the specified round-trip delay compensation and power level correction; next the HC

Wait for broadcast

maintenance
opportunity

Time out T1 Map with initial

maintenance
opportunity
Error
re-initialize MAC
RNG-REQ

Wait for
RNG-RSP

Time out T2 RNG-RSP

Adjust local parameters

Yes Retries
exausted
? Wait for unicast
Error maintenance
re-initialize MAC No
opportunity
Adjust transmit power
Time out T3 Map with initial
maintenance
Random backoff opportunity
Error
re-initialize MAC
Wait for broadcast RNG-REQ
maintenance
opportunity
Wait for
RNG-RSP

Time out T4 RNG-RSP

Adjust local parameters

Yes Retries
exausted
?
Error Abort
No ranging No Success Yes
re-initialize MAC set from
No set from
HC? HC
?
Yes Enable data transfer
Wait for unicast Error
maintenance re-initialize MAC Establish
opportunity
IP layer

Figure 18.26 Finite state machine used by the cable modem for the registration procedure. Source:
Reproduced with permission of Cable Television Laboratories, Inc.
888 Chapter 18. Examples of communication systems

Wait for
detectable
RNG-REQ

RNG-REQ

SID
already
Yes No
assigned
to this CM
? Assign temporary SID
Increment retry counter
in poll list for this CM Add CM to poll list
for future maps

RNG-RSP

Map with station

maintenance
opportunity

Wait for
polled RNG-REQ

RNG-REQ RNG-REQ
not received

No Retries Yes Retries No Parameters

exausted exausted within
limits
? ? ?
Yes No Yes
RNG-RSP RNG-RSP RNG-RSP
(abort ranging) (continue) (success)

Remove CM from Remove CM from

poll list poll list

Wait for Wait for

detectable Done
polled RNG-REQ
RNG-REQ

Figure 18.27 Finite state machine used by the HC for the registration procedure. Source:
Reproduced with permission of Cable Television Laboratories, Inc.

sends another RNG-REQ message to the station with information for a further refinement of round-trip
delay compensation and power level correction. The steps of ranging request/response are repeated until
the HC sends a ranging successful message; at this point, the station can send a registration request
(REG-REQ) message, to which the HC responds with a registration response (REG-RSP) message con-
firming the registration and specifying one or more SID that the cable modem must use during the
following transmissions. Finite state machines for the ranging procedure and the regulation of the power
level for the cable modem and for the HC are shown in Figures 18.26 and 18.27, respectively.
Bibliography 889

Bibliography

[1] 3GPP Technical Specification Group Radio Access Network (2017). NR; physical layer; general
description 12 2017. TS 38.201 V15.0.0.
[2] 3GPP Technical Specification Group Radio Access Network (2018). NR; physical channels and
modulation 6 2018. TS 38.211 V15.2.0.
[3] 3GPP Technical Specification Group Radio Access Network (2018). NR; multiplexing and channel
coding 6 2018. TS 38.212 V15.2.0.
[4] Lin, X., Li, J., Baldemair, R. et al. (2019). 5G new radio: unveiling the essentials of the next gener-
ation wireless access technology. IEEE Communications Standards Magazine 3(3): 30–37.
[5] Ferrus, R., Sallent, O., Perez-Romero, J., and Agusti, R. (2018). On 5G radio access network
slicing: radio interface protocol features and configuration. IEEE Communications Magazine 56:
184–192.
[6] Rahnema, M. (1993). Overview of the GSM system and protocol architecture. IEEE Communica-
tions Magazine 31: 92–100.
[7] Rappaport, T.S. (1996). Wireless Communications: Principles and Practice. Englewood Cliffs, NJ:
Prentice-Hall.
[8] Goodman, D.J. (1997). Wireless Personal Communication Systems. Reading, MA: Addison-Wesley.
[9] Singh, S. (1996). Wireless LANs. In: The Mobile Communications Handbook, Chapter 4 (ed. J.D.
Gibson), 540–552. New York, NY: CRC/IEEE Press.
[10] Miller, B.A. and Bisdikian, C. (2001). Bluetooth Revealed. Upper Saddle River, NJ: Prentice-Hall.
[11] Haartsen, J.C. and Mattison, S. (2000). Bluetooth — a new low–power radio interface providing
short–range connectivity. Proceedings of the IEEE 88: 1651–1661.
[12] Ahamed, S.V., Gruber, P.L., and Werner, J.-J. (1995). Digital subscriber line (HDSL and ADSL)
capacity of the outside loop plant. IEEE Journal on Selected Areas in Communications 13:
1540–1549.
[13] Chen, W.Y. and Waring, D.L. (1994). Applicability of ADSL to support video dial tone in the cop-
per loop. IEEE Communications Magazine 32: 102–109.
[14] Hawley, G.T. (1997). Systems considerations for the use of xDSL technology for data access. IEEE
Communications Magazine 35: 56–60.
[15] Chen, W.Y. and Waring, D.L. (1994). Applicability of ADSL to support video dial tone in the cop-
per loop. IEEE Communications Magazine 32: 102–109.
[16] Starr, T., Cioffi, J.M., and Silverman, P.J. (2000). Understanding Digital Subscriber Line Technol-
ogy. Englewood Cliffs, NJ: Prentice-Hall.
[17] Sjoberg, F., Isaksson, M., Nilsson, R. et al. (1999). Zipper: a duplex method for WDSL based on
DMT. IEEE Transactions on Communications 47: 1245–1252.
[18] Cherubini, G., Eleftheriou, E., Ölçer, S., and Cioffi, J.M. (2000). Filter bank modulation techniques
for very high-speed digital subscriber lines. IEEE Communications Magazine 38: 98–104.
[19] Cherubini, G., Eleftheriou, E., and Ölcer, S. (2002). Filtered multitone modulation for very-high-
speed digital subscriber lines. IEEE Journal on Selected Areas in Communications 20 5:
1016–1028.
[20] Cherubini, G., Eleftheriou, E., and Ölcer, S. (1999). Filtered mutitone modulation for VDSL.
Proceedings of IEEE GLOBECOM ’99, Rio de Janeiro, Brazil, pp. 1139–1144.
[21] Eldering, C.A., Himayat, N., and Gardner, F.M. (1995). CATV return path characterization for reli-
able communications. IEEE Communications Magazine 33: 62–69.
890 Chapter 18. Examples of communication systems

[22] Bisdikian, C., Maruyama, K., Seidman, D.I., and Serpanos, D.N. (1996). Cable access beyond the
hype: on residential broadband data services over HFC networks. IEEE Communications Magazine
34: 128–135.
[23] Fellows, D. and Jones, D. (2001). DOCSIS(tm) . IEEE Communications Magazine 39: 202–209.
[24] ITU-T Study Group 9 (1995). Digital multi-programme systems for television sound and data
services for cable distribution. ITU-T Recommendation J.83, 24 October 1995.
[25] Cherubini, G. (2003). Hybrid TDMA/CDMA based on filtered multitone modulation for uplink
transmission in HFC networks. IEEE Communications Magazine 41: 108–115.
[26] Chu, D.C. (1972). Polyphase codes with good periodic correlation properties. IEEE Transactions on
Information Theory 18: 531–532.
[27] Abramson, N. (ed.) (1993). Multiple Access Communications: Foundations for Emerging Technolo-
gies. Piscataway, NJ: IEEE Press.

Appendix 18.A Duplexing

A transmission link between two users of a communication network may be classified as
(a) Full duplex, when two users A and B can send information to each other simultaneously, not
necessarily by using the same transmission channels in the two directions.
(b) Half duplex, when two users A and B can send information in only one direction at a time, from
A to B or from B to A, alternatively.
(c) Simplex, when only A can send information to B, that is the link is unidirectional.

Three methods
In the following, we give three examples of transmission methods which are used in practice.
(a) Frequency-division duplexing (FDD): In this case, the two users are assigned different trans-
mission bands using the same transmission medium, thus allowing full-duplex transmission.
Examples of FDD systems are the GSM, which uses a radio channel (see Section 18.2), and the
VDSL, which uses a twisted pair cable (see Section 18.6.1).
(b) Time-division duplexing (TDD): In this case, the two users are assigned different slots in a time
frame (see Appendix 18.B). If the duration of one slot is small with respect to that of the message,
we speak of full-duplex TDD systems. Examples of TDD systems are the DECT, which uses a
radio channel (see Section 18.4), and the ping-pong BR-ISDN, which uses a twisted pair cable.
(c) Full-duplex systems over a single band: In this case, the two users transmit simultaneously in
two directions using the same transmission band; examples are the HDSL (see Section 18.6.1),
and in general high-speed transmission systems over twisted-pair cables for LAN applications
(see Section 18.6.2). The two directions of transmission are separated by a hybrid; the receiver
eliminates echo signals by echo cancellation techniques. We note that full-duplex transmission
over a single band is possible also over radio channels, but in practice, alternative methods are
still preferred because of the complexity required by echo cancellation.

Appendix 18.B Deterministic access methods

We distinguish three cases for channel access by N users:
1. Subdivision of the channel passband into NB separate subbands that may be used for transmission
(see Figure 18.28a).
18.B. Deterministic access methods 891

2. Subdivision of a sequence of modulation intervals into adjacent subsets called frames, each in turn
subdivided into NS adjacent subsets called slots. Within a frame, each slot is identified by an index
i, i = 0, … , NS − 1 (see Figure 18.28b).
3. Signalling by N0 orthogonal signals (see for example Figure 10.18).

(a)

f1 f2 frequency

(b) frame m frame (m +1)

slot 0 slot 1 slot 2 slot 0 slot 1 slot 2 slot 0

time

Figure 18.28 Illustration of (a) FDMA, and (b) TDMA.

N users may share the channel using one of the following methods.6
1. Frequency division multiple access (FDMA): To each user is assigned one of the NB subbands.
2. Time division multiple access (TDMA): To each user is assigned one of the NS time sequences
(slots), whose elements identify the modulation intervals.
3. Code division multiple access (CDMA): To each user is assigned a modulation scheme that
employs one of the N0 orthogonal signals, preserving the orthogonality between modulated
signals of the various users. For example, for the case N0 = 8, to each user may be assigned one
orthogonal signal of those given in Figure 10.18; for binary modulation, within a modulation
interval each user then transmits the assigned orthogonal signal or its antipodal version.
We give an example of implementation of the TDMA principle.

Example 18.B.1 (Time-division multiplexing)

Time-division multiplexing (TDM) is the interlacing of several digital messages into one digital message
with a higher bit rate; as an example we illustrate the generation of the European base group, called E1,
at 2.048Mbit/s, that is obtained by multiplexing 30 pulse-code-modulation (PCM) coded speech signals
(or channels) at 64kbit/s. As shown in Figure 18.29, each 8-bit sample of each channel is inserted into a
pre-assigned slot of a frame composed of 32 slots, equivalent to 32 ⋅ 8 = 256 bits. The frame structure
must contain information bits to identify the beginning of a frame (channel ch0) by 8 known framing
bits; 8bits are employed for signalling between central offices (channel ch16). The remaining 30 channels
are for the transmission of signals. As the duration of a frame is of 125 μs, equal to the interval between
two PCM samples of the same channel, the overall digital message has a bit rate of 256 bit/125 μs =
2.048 Mbit/s; we note however that of the 256 bits of the frame only 30 ⋅ 8 = 240 bits carry information
related to signals.

6
The access methods discussed in this section are deterministic, as each user knows exactly at which point in time the
channel resources are reserved for transmission; an alternative approach is represented by random access techniques,
e.g. ALOHA, CSMA/CD, collision resolution protocols [27].
892 Chapter 18. Examples of communication systems

Ch 16 Ch 0

Ch 1
Ch 15
P/S
Ch 17
Ch 31

(a)

Ch 0 Ch 1 Ch 16 Ch 31Ch 0 Ch 1 Ch 16 Ch 31
8 bit 8 bit 8 bit 8 bit
frame signaling
alignment
256 bit
frame interval
1
= 125 μs
8kHz
(b)

Figure 18.29 TDM in the European base group at 2.048 Mbit/s. (a) Basic scheme. (b) Frame
structure.

In the United States, Canada, and Japan, the base group, analog to E1, is called T1 carrier system and
has a bit rate of 1.544 Mbit/s, obtained by multiplexing 24 PCM speech coded signals at 64 kbit/s. In this
case, the frame is such that 1bit per channel is employed for signalling. This bit is robbed from the least
important bit of the 8-bit PCM sample, thus making it a 7-bit code word per sample; there is then only
1bit for the synchronization of the whole frame. The entire frame is formed of 24 ⋅ 8 + 1 = 193 bits.
Chapter 19

High-speed communications over

twisted-pair cables

In this chapter, we describe the design of two high-speed data transmission systems over unshielded
twisted-pair (UTP) cables [1, 2].

19.1 Quaternary partial response class-IV system

Figure 19.1 shows the block diagram of a transceiver for a quaternary partial response class-IV
(QPR-IV), or quaternary modified duobinary (see Appendix 7.C), system for data transmission at 125
Mbit/s over UTP cables [1]. In the transmitter, information bits are first scrambled and then input
to a 2B1Q differential encoder that yields output symbols belonging to the quaternary alphabet
 = {−3, −1, +1, +3} (see Example 16.1.1 on page 793); differential encoding makes the transmission
of information insensitive to the polarity of the received signals. Signal shaping into partial response
class-IV (PR-IV) form is accomplished by the cascade of the following elements: the digital-to-analog
converter (DAC), the analog transmit filter (ATF), the cable, the analog receive filter (ARF) with auto-
matic gain control (AGC), the analog-to-digital converter (ADC), and, in the digital domain, the fixed
decorrelation filter (DCF) and the adaptive equalizer. After equalization, the sequence of transmitted
quaternary symbols is detected by a Viterbi algorithm. Since the Viterbi algorithm introduces a delay
in the detection of the transmitted sequence, as an alternative it is possible to use a threshold detector
that individually detects transmitted symbols with a negligible delay. If the mean-square error (MSE)
is below a fixed value, threshold detection, or symbol-by-symbol detection (SSD), is selected as the
corresponding symbol decisions are then sufficiently reliable. If, instead, the MSE exceeds the fixed
value, the Viterbi algorithm is employed. Finally, differential 1Q2B decoding and descrambling are
performed.

Analog filter design

Figure 19.2 depicts the overall analog channel considered for the design of ATF and ARF. The impulse
response is denoted by h(t, L, uc ), where L is the cable length and uc is the control signal used for AGC.
To determine ATF and ARF with low implementation complexity, the transfer functions of these filters
are first expressed in terms of poles and zeros; the pole-zero configurations are then jointly optimized
by simulated annealing as described in [3]. The cost function that is used for the optimization reflects
two criteria: (i) the MSE between the impulse response h(t, L, uc ) for a cable length equal to 50 m, and
the ideal PR-IV response must be below a certain value and (ii) spectral components of the transmitted
Algorithms for Communications Systems and their Applications, Second Edition.
Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
894 Chapter 19. High-speed communications over twisted-pair cables

125
Mbit/s diff.
scrambler DAC
2B/1Q ATF
& 2-bit S/P 2-bit
mapper

digital
adaptive
NEXT
canceller

125 threshold
Mbit/s diff.dic. 1Q2B detector digital D – ARF
elastic ADC
2-bitP/S or adaptive C + &
buffer + 8-bit
& descrambler viterbi equalizer F AGC
detector

62.5 Msample/s ± ε 62.5 Msample/s

Figure 19.1 Block diagram of a QPR-IV transceiver. Source: Reproduced with permission from
Cherubini et al. [1]. ©1995, IEEE.

overall analog channel

a0 = +1 h[t,L,uc(L)]| L = 50 m
UTP-3
DAC ATF ARF AGC
L = 50 m

uc(L)|L = 50 m
T = 16 ns

Figure 19.2 Overall analog channel considered for the joint optimization of the analog transmit
(ATF) and receive (ARF) filters. Source: Reproduced with permission from Cherubini et al. [1].
©1995, IEEE.

signal above 30 MHz should be well suppressed for achieving compliance with regulations on radiation
limits. Investigations with various orders of the respective transfer functions have shown that a good
approximation of an ideal PR-IV response is obtained with 5 poles and 3 zeros for the ATF, and 3 poles
for the ARF.

Received signal and adaptive gain control

The received signal at the input of the ADC is expressed as

∑
+∞
∑
+∞
x(t) = ak h[t − kT, L, uc (L)] + aNk hN [t − kT, L, uc (L)] + 𝑤R (t) (19.1)
k=−∞ k=−∞

where {ak } and {aNk } are the sequences of quaternary symbols generated by the local transmitter and
remote transmitter, respectively, hN [t, L, uc (L)] is the near-end crosstalk (NEXT) channel response and
𝑤R is additive Gaussian noise. The signal x is sampled by the ADC that operates synchronously with
the DAC at the modulation rate of 1∕T = 62.5 MBaud. The adjustment of the AGC circuit is computed
digitally, using the sampled signal xk = x(kT), such that the ADC output signal achieves a constant
19.1. Quaternary partial response class-IV system 895

average statistical power 𝙼R , i.e.

T |
1 |
E[xk2 ]|uc =uc (L) = E[x2 (t)] dt| = 𝙼R (19.2)
T ∫0 |
|uc =uc (L)
This ensures that the received signal is converted with optimal precision independently of the cable
length; moreover, a controlled level of the signal at the adaptive digital equalizer input is required for
achieving optimal convergence properties.

Near-end crosstalk cancellation

As illustrated in Chapter 17, NEXT cancellation is achieved by storing the transmit symbols aNk in a
delay line and computing an estimated NEXT signal û Nk , which is then subtracted from the received
signal xk , that is
∑
NN −1
x̃ k = xk − û Nk = xk − cNi,k aNk−i (19.3)
i=0

where {cNi,k }, i = 0, … , NN − 1, are the coefficients of the adaptive NEXT canceller. Using the mini-
mization of E[̃xk2 ] as a criterion for updating the NEXT canceller coefficients (see Section 17.1), leads
to the least mean square (LMS) algorithm

cNi,k+1 = cNi,k + 𝜇N x̃ k aNk−i , 0 ≤ i ≤ NN − 1 (19.4)

where 𝜇N is the adaptation gain.

As discussed in Chapter 17, high-speed full-duplex transmission over two separate wire pairs with
NEXT cancellation and full-duplex transmission over a single pair with echo cancellation pose simi-
lar challenges. In the latter case, a hybrid is included to separate the two directions of transmission; a
QPR-IV transceiver with NEXT cancellation can then be used also for full-duplex transmission over a
single pair, as in this case the NEXT canceller acts as an echo canceller.

Decorrelation filter
After NEXT cancellation, the signal is filtered by a DCF, which is used to improve the convergence prop-
erties of the adaptive digital equalizer by reducing the correlation between the samples of the sequence
{̃xk }. The filtering operation performed by the DCF represents an approximate inversion of the PR-IV
frequency response. The DCF has frequency response 1∕(1 − 𝛽 e−j4𝜋fT ), with 0 < 𝛽 < 1, and provides
at its output the signal
zk = x̃ k + 𝛽 zk−2 (19.5)

Adaptive equalizer
The samples {zk } are stored in an elastic buffer, from which they are transferred into the equalizer delay
line. Before describing this operation in more detail, we make some observations about the adaptive
equalizer. As mentioned above, the received signal x is sampled in synchronism with the timing of the
local transmitter. Due to the frequency offset between local and remote transmitter clocks, the phase of
the remote transmitter clock will drift in time relative to the sampling phase. As the received signal is
bandlimited to one half of the modulation rate, signal samples taken at the symbol rate are not affected
by aliasing; hence, a fractionally spaced equalizer is not required for a QPR-IV system. Furthermore, as
896 Chapter 19. High-speed communications over twisted-pair cables

the signal value x can be reconstructed from the T-spaced samples {zk }, an equalizer of sufficient length
acts also as an interpolator. The adaptive equalizer output signal is given by

∑
NE −1
yk = cEi,k zk−i (19.6)
i=0

where {cEi,k }, i = 0, … , NE − 1, denote the filter coefficients.

Compensation of the timing phase drift

The effect of timing phase drift can be compensated by continuously adjusting the equalizer coefficients.
As a result of these adjustments, for a positive frequency offset of +𝛿∕T Hz, i.e. for a frequency of
the local transmitter clock larger than the frequency of the remote transmitter clock, the value of the
i-th coefficient at time k is approximately equal to the value assumed by the (i + 1)-th coefficient 1∕𝛿
modulation intervals earlier. In other words, the coefficients move to the left by one position relative to
the equalizer delay line after 1∕𝛿 modulation intervals; conversely, for a negative frequency offset of
−𝛿∕T Hz, the coefficients move to the right by one position relative to the equalizer delay line after 1∕𝛿
modulation intervals. Hence, the centre of gravity of the filter coefficients drifts. Proper operation of a
finite-length equalizer requires that the coefficients be re-centred; this is accomplished as follows:

● Normally, for each equalizer output yk , one new signal zk is transferred from the buffer into the
equalizer delay line;
● Periodically, after a given number of modulation intervals, the sums of the magnitudes of N ′ first
and last coefficients are compared; if the first coefficients are too large, the coefficients are shifted
by one position towards the end of the delay line (right shift) and two new signals zk and zk+1 are
transferred from the buffer into the delay line; if the last coefficients are too large, the coefficients
are shifted by one position towards the beginning of the delay line (left shift) and no new signal is
retrieved from the buffer.

To prevent buffer overflow or underflow, the rate of the remaining receiver operations is controlled so
that the elastic buffer is kept half full on average, thus performing indirect timing recovery. The control
algorithm used to adjust the voltage-controlled oscillator (VCO) providing the timing signal for the
remaining receiver operations is described in Chapter 14.

Adaptive equalizer coefficient adaptation

Overall system complexity is reduced if self-training adaptive equalization is employed, as in this case a
start-up procedure with the transmission of a known training sequence is not needed. We describe here
the operation of the adaptive equalizer and omit the presentation of the self-training algorithm, discussed
in Section 15.4.
The MSE at the equalizer output is continuously monitored using seven-level tentative decisions. If the
MSE is too large, self-training adaptive equalization is performed during a fixed time period TST . At the
end of the self-training period, if the MSE is sufficiently small, equalizer operation is continued with the
decision directed LMS algorithm

cEi,k+1 = cEi,k − 𝜇E zk−i ê k , 0 ≤ i ≤ NE − 1 (19.7)

where ê k = yk − (̂ak − â k−2 ) is the error obtained using tentative decisions â k on the transmitted quater-
nary symbols, and 𝜇E is the adaptation gain.
19.1. Quaternary partial response class-IV system 897

Convergence behaviour of the various algorithms

We resort to computer simulations to study the convergence behaviour of the adaptive digital
NEXT canceller and of the adaptive equalizer. The length of the NEXT canceller is chosen equal to
NN = 48 to ensure that, in the worst case signal attenuation, the power of the residual NEXT is
with high probability more than 30 dB below the power of the signal from the remote transmitter. In
Figure 19.3, the residual NEXT statistical power at the canceller output is plotted versus the number of
iterations for a worst-case cable length of L = 100 m and an adaptation gain 𝜇N = 2−18 . In the same
figure, the statistical power in dB of the portion of the NEXT signal that cannot be cancelled due to
finite NEXT-canceller length is also indicated. For the simulations, the NEXT canceller was assumed
to be realized in the distributed-arithmetic form (see Section 17.1).
The convergence of the MSE at the output of the equalizer in the absence of timing phase drift is shown
in Figure 19.4a,b for best and worst-case sampling phase, respectively, and a value ΓTx = 2ETx ∕N0 of
43 dB, where ETx is the average energy per modulation interval of the transmitted signal. The NEXT
canceller is assumed to have converged to the optimum setting. An equalizer length of NE = 24 is chosen,
which guarantees that the mean-square interpolation error with respect to an ideal QPR-IV signal is less
than −25 dB for the worst-case sampling phase. The self-training period is TST ≈ 400 μs, corresponding
to approximately 25000T. The adaptation gains for self-training and decision directed adjustment have
the same value 𝜇E = 2−9 , that is chosen for best performance in the presence of a worst-case timing
phase drift 𝛿 = 10−4 . Figure 19.5 shows the MSE convergence curves obtained for 𝛿 = 10−4 .

19.1.1 VLSI implementation

Adaptive digital NEXT canceller

As shown in Chapter 17, for the very large-scale integration (VLSI) implementation of the adaptive
digital NEXT canceller, a distributed-arithmetic filter presents significant advantages over a transversal

–5
Residual NEXT power (dB)

–10

–15

–20

–25 Uncancelable NEXT

–30
0 100 000 200 000 300 000 400 000 500 000 600 000 700 000
t/T

Figure 19.3 Convergence of the adaptive NEXT canceller. Source: Reproduced with permission
from Cherubini et al. [1]. ©1995, IEEE.
898 Chapter 19. High-speed communications over twisted-pair cables

Mean-square error (dB)

–5

–10

–15
400 μs

–20
0 15 000 30 000 45 000 60 000 75 000
t/T
(a)

0
Mean-square error (dB)

–5

–10

–15
400 μs

–20
0 15 000 30 000 45 000 60 000 75 000
t/ T
(b)

Figure 19.4 Convergence of the adaptive equalizer for (a) best-case sampling phase and
(b) worst-case sampling phase. Source: Reproduced with permission from Cherubini et al. [1].
©1995, IEEE.

filter in terms of implementation complexity. In a NEXT canceller distributed-arithmetic filter, the par-
tial products that appear in the expression of a transversal filter output are not individually computed;
evaluation of partial products is replaced by table look-up and shift-and-add operations of binary words.
To compute the estimate of the NEXT signal to be subtracted from the received signal, look-up values
are selected by the bits in the NEXT canceller delay line and added by a carry-save adder. By segmenting
the delay line of the NEXT canceller into sections of shorter lengths, a trade-off concerning the number
of operations per modulation interval and the number of memory locations that are needed to store the
look-up values is possible. The convergence of the look-up values to the optimum setting is achieved by
an LMS algorithm.
19.1. Quaternary partial response class-IV system 899

Mean-square error (dB) –5

–10

–15
400 μs

–20
0 15 000 30 000 45 000 60 000 75 000
t/T

Figure 19.5 Convergence of the adaptive equalizer for worst-case timing phase drift. Source:
Reproduced with permission from Cherubini et al. [1]. ©1995, IEEE.

If the delay line of the NEXT canceller is segmented into L sections with K = NN ∕L delay elements
each, the NEXT canceller output signal is given by
∑
NN −1
∑ ∑
L−1 K−1
û Nk = aNk−i cNi,k = aNk−𝓁K−m cN𝓁K+m,k (19.8)
i=0 𝓁=0 m=0

In a distributed-arithmetic implementation, the quaternary symbol aNk is represented by the binary

vector [a(0)
k
, a(1)
k
], that is
∑
1
𝑤
∑
1
aNk = (2a(𝑤)
k
− 1)2 = b(𝑤)
k
2𝑤 (19.9)
𝑤=0 𝑤=0

where b(𝑤)
k
= (2a(𝑤)
k
− 1) ∈ {−1, +1}. Introducing (19.9) into (19.8), we obtain (see (17.12))
[ ]
∑ ∑ 𝑤 K−1
L−1 1
∑ (𝑤)
û k =
N
2 N
bk−𝓁K−m c𝓁K+m,k (19.10)
𝓁=0 𝑤=0 m=0

Equation (19.10) suggests that the filter output can be computed using a set of L2K look-up values that
are stored in L look-up tables with 2K memory locations each. Extracting the term b(𝑤)
k−𝓁K
out of the square
bracket in (19.10), to determine the output of a distributed-arithmetic filter with reduced memory size
L2K−1 , we rewrite (19.10) as (see (17.13))
∑ ∑
L−1 1
û Nk = 2𝑤 b(𝑤) dN (i(𝑤) , 𝓁)
k−𝓁K k k,𝓁
(19.11)
𝓁=0 𝑤=0

where {dkN (n, 𝓁)}, n = 0, … , 2K−1 − 1, 𝓁 = 0, … , L − 1, are the look-up values, and i(𝑤)
k,𝓁
denotes the
selected look-up address that is computed as follows:
900 Chapter 19. High-speed communications over twisted-pair cables

⎧ ∑ (𝑤)
K−1

⎪ ak−𝓁K−m 2m−1 if a(𝑤)

k−𝓁K
=1
⎪ m=1
i(𝑤)
k,𝓁
=⎨ (19.12)
∑ (𝑤)
⎪ K−1
⎪ a 2m−1 if a(𝑤) =0
⎩ m=1 k−𝓁K−m k−𝓁K

where an is the one’s complement of an . The expression of the LMS algorithm to update the look-up
values of a distributed-arithmetic NEXT canceller takes the form
∑
1
N
dk+1 (n, 𝓁) = dkN (n, 𝓁) + 𝜇N x̃ k 2𝑤 b(𝑤) 𝛿 (𝑤) ,
k−𝓁K n−i
𝑤=0 k,𝓁 (19.13)
n = 0, … , 2K−1 − 1, 𝓁 = 0, … , L − 1
where 𝛿n is the Kronecker delta. We note that at each iteration only those look-up values that are selected
to generate the filter output are updated. The implementation of the NEXT canceller is further simplified
by updating at each iteration only the look-up values that are addressed by the most significant bits of
the symbols, i.e. those with index 𝑤 = 1, stored in the delay line (see (17.23)). The block diagram of an
adaptive distributed-arithmetic NEXT canceller is shown in Figure 19.6. In the QPR-IV transceiver, for
the implementation of a NEXT canceller with a time span of 48T, L = 16 segments with K = 3 delay
elements each are employed. The look-up values are stored in 16 tables with four 16-bit registers each.

Adaptive digital equalizer

As discussed A in the previous section, the effect of the drift in time of the phase of the remote trans-
mitter clock relative to the sampling phase is compensated by continuously updating and occasionally
re-centring the coefficients of the digital equalizer. The need for fast equalizer adaptation excludes a
distributed-arithmetic approach for the implementation of the digital equalizer. An efficient solution for
occasional re-centring of the NE equalizer coefficients is obtained by the following structure, which is
based on an approach where NE multiply-accumulate (MAC) units are employed.
According to (19.6), assuming that the coefficients are not time varying, at a given time instant
k, each MAC unit is presented with a different tap coefficient and carries out the multiplication of
this tap coefficient with the signal sample zk . The result is a partial product that is added to a value
stored in the MAC unit, which represents the sum of partial products up until time instant k. The
MAC unit that has accumulated NE partial products provides the equalizer output signal at time instant
∑NE −1 E
k, yk = i=0 ci zk−i , and its memory is cleared to allow for the accumulation of the next NE partial
products. At this time instant, the MAC unit that has accumulated the result of (NE − 1) partial products
∑NE −1 E
has stored the term i=1 ci zk−(i−1) . At time instant (k + 1), this unit computes the term cE0 zk+1 and
provides the next equalizer output signal
∑
NE −1
yk+1 = cE0 zk+1 + cEi zk−(i−1) (19.14)
i=1

This MAC unit is then reset and its output will be considered again NE time instants later.
Figure 19.7 depicts the implementation of the digital equalizer. The NE coefficients {cEi },
i = 0, … , NE − 1, normally circulate in the delay line shown at the top of the figure. Except when
re-centring of the equalizer coefficients is needed, NE coefficients in the delay line are presented each
to a different MAC unit, and the signal sample zk is input to all MAC units. At the next time instant,
the coefficients are cyclically shifted by one position, and the new signal sample zk+1 is input to all the
units. The multiplexer shown at the bottom of the figure selects in turn the MAC unit that provides the
equalizer output signal.
19.1. Quaternary partial response class-IV system 901

(0) (1)
ak(0) ak(1) a k−(L−1)K a k−(L−1)K
address address
computation computation
(1) (0) (1) (0)
i k,0 ik,0 i k,L−1 i k,L−1

µ~
xk µ~
xk
table table
N
d k+1 (i k(1)
,0
,0) 0 L−1
+ +
+ (1) (0)
+
d kN (i k,0 ,0) d kN (i k,0 ,0) (1)
d kN (i k,L−1 ,L−1) d kN (i k(0) ,L− 1)
,L−1

(1)
1
←
+1 bk(1) 1
←
+1 b k−(L − 1)K

0 −1 −1
←
0
←
(0)
1
←
+1 bk(0) 1
←
+1 b k−(L − 1)K

0 −1 −1
←
0
←

2 2

~x u^ kN
k xk

Figure 19.6 Adaptive distributed-arithmetic NEXT canceller. Source: Reproduced with permission
from Cherubini et al. [1]. ©1995, IEEE.

To explain the operations for re-centring of the equalizer coefficients, we consider as an example
a simple equalizer with NE = 4 coefficients; in Figure 19.8, where for simplicity the coefficients are
denoted by {c0 , c1 , c2 , c3 }, the coefficients and signal samples at the input of the 4 MAC units are given
as a function of the time instant. At time instant k, the output of the MAC unit 0 is selected, at time instant
k + 1, the output of the MAC unit 1 is selected, and so on. For a negative frequency offset between the
local and remote transmitter clocks, a re-centring operation corresponding to a left shift of the equalizer
coefficients occasionally occurs as illustrated in the upper part of Figure 19.8. We note that as a result
of this operation, a new coefficient c4 , initially set equal to zero, is introduced. We also note that signal
samples with proper delay need to be input to the MAC units. A similar operation occurs for a right shift
of the equalizer coefficients, as illustrated in the lower part of the figure; in this case, a new coefficient
c−1 , initially set equal to zero, is introduced. In the equalizer implementation shown in Figure 19.7,
the control operations to select the filter coefficients and the signal samples are implemented by the
multiplexer MUXC at the input of the delay line and by the multiplexers MUXS(0), …, MUXS(NE − 1)
at the input of the MAC units, respectively. A left or right shift of the equalizer coefficients is completed
902 Chapter 19. High-speed communications over twisted-pair cables

10
cNE c0E c1E cNE −3 cNE −2
11 E −1 E E
01 + + + + A C B
00
MUXC

Nu coefficient
updating terms

z k−1

zk

10 10 10 1 0 1 0
z k(0)
MUXS0 MUXS1 MUXS2 MUXS N MUXS NE −1
E −2
z k(1)

MAC MAC MAC MAC MAC

0 1 2 NE −2 NE −1

MUX

y
k

Figure 19.7 Digital adaptive equalizer: coefficient circulation and updating, and computation of
output signal. Source: Reproduced with permission from Cherubini et al. [1]. ©1995, IEEE.

in NE cycles. To perform a left shift, in the first cycle the multiplexer MUXC is controlled so that a
new coefficient cEN = 0 is inserted into the delay line. During the following (NE − 1) cycles, the input
E
of the delay line is connected to point B. After inserting the coefficient cE1 at the NE -th cycle, the input
of the delay line is connected to point C and normal equalizer operations are restored. For a right shift,
the multiplexer MUXC is controlled so that during the first NE − 1 cycles the input of the delay line
is connected to point A. A new coefficient cE−1 = 0 is inserted into the delay line at the NE -th cycle
and normal equalizer operations are thereafter restored. At the beginning of the equalizer operations,
the equalizer coefficients are initialized by inserting the sequence {0, … , 0, +1, 0, −1, 0, … , 0} into the
delay line.
The adaptation of the equalizer coefficients in decision-directed mode is performed according to the
LMS algorithm (19.7). However, to reduce implementation complexity, equalizer coefficients are not
updated at every cycle; during normal equalizer operations, each coefficient is updated every NE ∕NU
cycles by adding correction terms at NU equally spaced fixed positions in the delay line, as shown
in Figure 19.9. The architecture adopted for the computation of the correction terms is similar to the
architecture for the computation of the equalizer output signal. The gradient components −𝜇E zk−i ê k ,
i = 0, … , NE − 1, are accumulated in NE MAC units, as illustrated in the figure. The delay line stores
the signal samples z(0)
k
and z(1)
k
. The inputs to each MAC unit are given by the error signal and a signal
from the delay line. The multiplexers at the input of each register in the delay line allow selecting the
appropriate inputs to the MAC units in connection with the re-centring of the equalizer coefficients. At
19.1. Quaternary partial response class-IV system 903

MAC 0 MAC 1 MAC 2 MAC 3

c3 z k−3 c0 zk−3 c1 z k−3 c2 zk−3 k−3
c2 z k−2 c3 z k−2 c0 z k−2 c1 zk−2 k−2
c1 z k−1 c2 z k−1 c3 zk−1 c0 zk−1 k−1
c0 zk c1 zk c2 zk c3 zk k
c3 zk + 1 c0 zk + 1 c1 zk + 1 c2 zk + 1 k+1
c2 z k+2 c3 zk+2 c0 zk+2 c1 zk+2 k+2
c1 z k+3 c2 zk+3 c3 zk+3 c0 zk+3 k+3
c0 z k+4 c1 zk+4 c2 zk+4 c3 zk+4 k+4
shift left c4 zk+4 c0 zk+5 c1 zk+5 c2 zk+5 k+5
c3 zk+5 c4 zk+5 c0 zk+6 c1 zk+6 k+6
c2 zk+6 c3 zk+6 c4 zk+6 c0 zk+7 k+7
c1 zk+7 c2 zk+7 c3 zk+7 c4 zk+7 k+8
c4 zk+8 c1 zk+8 c2 zk+8 c3 zk+8 k+9
c3 zk+9 c4 zk+9 c1 zk+9 c2 zk+9 k + 10
c2 z k + 10 c3 z k + 10 c4 z k + 10 c1 z k + 10 k + 11
c1 z k + 11 c2 z k + 11 c3 z k + 11 c4 z k + 11 k + 12
c1 z k + 12 c2 z k + 12 c3 z k + 12 k + 13
c1 z k + 13 c2 z k + 13 k + 14
c1 z k + 14 k + 15

c3 zk−3 c0 zk−3 c1 zk−3 c2 z k−3 k−3

c2 zk−2 c3 zk−2 c0 zk−2 c1 zk − 2 k−2
c1 zk−1 c2 zk−1 c3 zk−1 c0 zk−1 k−1
c0 zk c1 zk c2 zk c3 zk k
c3 zk + 1 c0 zk + 1 c1 zk + 1 c2 zk + 1 k+1
c2 zk+2 c3 zk+2 c0 zk+2 c1 z k+2 k+2
c1 zk+3 c2 zk+3 c3 zk+3 c0 z k+3 k+3
c0 zk+4 c1 zk+4 c2 zk+4 c3 z k+4 k+4
shift right c0 zk+5
c2 zk + 6 c1 z c2 z k+5 k+5
k+5
c1 zk+7 c2 zk+7 c0 zk+6 c1 z k+6 k+6
c0 zk+8 c1 zk+8 c2 zk+8 c0 z k+7 k+7
c−1 zk+9 c0 z k + 9 c1 zk+9 c2 zk+9 k+8
c2 z k + 10 c−1 z k + 10 c0 z k + 10 c1 z k + 10 k+9
c1 z k + 11 c z k + 11 c−1 z k + 11 c0 z k + 11 k + 10
2
c0 z k + 12 c1 z k + 12 c2 z k + 12 c−1 z k + 12 k + 11
c−1 z k + 13 c0 z k + 13 c1 z k + 13 c2 z k + 13 k + 12
c−1 z k + 14 c0 z k + 14 c1 z k + 14 k + 13
c−1 z k + 15 c0 z k + 15 k + 14
c−1 zk + 16 k + 15

Figure 19.8 Coefficients and signals at the input of the multiply-accumulate (MAC) units during
coefficient shifting. Source: Reproduced with permission from Cherubini et al. [1]. ©1995, IEEE.
904 Chapter 19. High-speed communications over twisted-pair cables

z (0)
k
10 10 10 10 10
00 00 00 00 00
01 01 01 01 01
z (1)
k

e^k

MAC MAC MAC MAC MAC

0 1 2 NE − 2 NE − 1

MUX

NU coefficient updating terms

Figure 19.9 Adaptive digital equalizer: computation of coefficient adaptation. Source: Reproduced
with permission from Cherubini et al. [1]. ©1995, IEEE.

each cycle, the output of NU MAC units are selected and input to the NU adders in the delay line where
the equalizer coefficients are circulating, as illustrated in Figure 19.7.

Timing control
An elastic buffer is provided at the boundary between the transceiver sections that operate at the transmit
and receive timings. The signal samples at the output of the DCF are obtained at a rate that is given by
the transmit timing and stored at the same rate into the elastic buffer. Signal samples from the elastic
buffer are read at the same rate that is given by the receive timing. The VCO that generates the receive
timing signal is controlled in order to prevent buffer underflow or overflow.
Let WPk and RPk denote the values of the two pointers that specify the write and read addresses,
respectively, for the elastic buffer at the k-th cycle of the receiver clock. We consider a buffer with eight
memory locations, so that WPk , RPk ∈ {0, 1, 2, … , 7}. The write pointer is incremented by one unit at
every cycle of the transmitter clock, while the read pointer is also incremented by one unit at every cycle
of the receiver clock. The difference pointer,
DPk = WPk − RPk (mod 8) (19.15)
is used to generate a binary control signal Δk ∈ {±1} that indicates whether the frequency of the VCO
must be increased or decreased:
⎧
⎪ +1 if DPk = 4, 5
⎪
Δk = ⎨ −1 if DPk = 2, 3 (19.16)
⎪
⎪ Δk−1 otherwise
⎩
The signal Δk is input to a digital loop filter which provides the control signal to adjust the VCO. If the
loop filter comprises both a proportional and an integral term, with corresponding gains of 𝜇𝜏 and 𝜇Δ𝜏 ,
19.1. Quaternary partial response class-IV system 905

respectively, the resulting second order phase-locked loop is described by (see Section 14.7)
𝜏k+1 = 𝜏k + 𝜇𝜏 Δk + Δ𝜏k
(19.17)
Δ𝜏k+1 = Δ𝜏k + 𝜇Δ𝜏 Δk
where 𝜏k denotes the difference between the phases of the transmit and receive timing signals. With a
proper setting of the gains 𝜇𝜏 and 𝜇Δ𝜏 , the algorithm (19.17) allows for correct initial frequency acqui-
sition of the VCO and guarantees that the write and read pointers do not overrun each other during
steady-state operations.
For every time instant k, the two consecutive signal samples stored in the memory locations with
the address values RPk and (RPk − 1) are read and transferred to the equalizer. These signal samples are
denoted by zk and zk−1 in Figure 19.10. When a re-centring of the equalizer coefficients has to take place,

Normal operation

WPk WPk + 1
WP k + 2

RPk + 2

zk − 1 zk+2
RPk + 1
RPk

zk zk + 1 zk zk + 1

Shift left

WPk WPk + 1

WP k + 2

zk − 1 zk − 1 RPk + 2
RPk
RPk + 1
zk zk zk + 1 zk

Shift right

WPk + 1
WPk RPk + 2
zk + 3 WP k + 2

RPk + 1

zk − 1 z k + 2 zk+2
RPk

zk zk + 1

Figure 19.10 Elastic buffer: control of the read pointer.

906 Chapter 19. High-speed communications over twisted-pair cables

for one cycle of the receiver clock the read pointer is either not incremented (left shift), or incremented
by two units (right shift). These operations are illustrated in the figure, where the elastic buffer is rep-
resented as a circular memory. We note that by the combined effect of the timing control scheme and
the re-centring of the adaptive equalizer coefficients the frequency of the receive timing signal equals
on average the modulation rate at the remote transceiver.

Viterbi detector
For the efficient implementation of near maximum likelihood sequence detection (MLSD) of QPR-IV
signals, we consider the reduced-state Viterbi detector of Example 7.5.3 on page 365. In other words,
the signal samples at the output the of (1 − D2 ) partial response channel are viewed as being generated
by two interlaced (1 − D′ ) dicode channels, where D′ = D2 corresponds to a delay of two modula-
tion intervals, 2T. The received signal samples are hence deinterlaced into even and odd time-indexed
sequences. The Viterbi algorithm using a 2-state trellis is performed independently for each sequence.
This reduced-state Viterbi algorithm retains at any time instant k only the two states with the smallest
and second smallest metrics and their survivor sequences, and propagates the difference between these
metrics instead of two metrics. Because the minimum distance error events in the partial-response trellis
lead to quasi-catastrophic error propagation, a sufficiently long path memory depth is needed. A path
memory depth of 64T has been found to be appropriate for this application.

19.2 Dual-duplex system

We now describe the 100BASE-T2 system for fast Ethernet mentioned in Section 18.6.2 [2], [4].

Dual-duplex transmission
Figure 19.11 indicates that near-end crosstalk (NEXT) represents the main disturbance for transmission
at high data rates over UTP-3 cables. As illustrated in Figure 19.12, showing the principle of dual-duplex
transmission (see also Section 17.1), self NEXT is defined as NEXT from the transmitter output to the
receiver input of the same transceiver, and can be cancelled by adaptive filters as discussed in Chapter 17.
Alien NEXT is defined instead as NEXT from the transmitter output to the receiver input of another
transceiver; this is generated in the case of simultaneous transmission over multiple links within one
multi-pair cable, typically with 4 or 25 pairs. Suppression of alien NEXT from other transmissions in
multi-pair cables and far-end crosstalk (FEXT), although normally not very significant, requires specific
structures (see for example Section 17.4).
To achieve best performance for data transmission over UTP-3 cables, signal bandwidth must be con-
fined to frequencies not exceeding 30 MHz. As shown in Chapter 18, this restriction is further mandated
by the requirement to meet Federal Communications Commission (FCC) and Comité européen de nor-
malisation en électronique et en électrotechnique (CENELEC) class B limits on emitted radiation from
communication systems. These limits are defined for frequencies above 30 MHz. Twisted pairs used in
UTP-3 cables have fewer twists per unit of length and generally exhibit a lower degree of homogene-
ity than pairs in UTP-5 cables; therefore, transmission over UTP-3 cables produces a higher level of
radiation than over UTP-5 cables. Thus, it is very difficult to comply with the class B limits if signals
containing spectral components above 30 MHz are transmitted over UTP-3 cables.
As illustrated in Figure 19.12, for 100BASE-T2 a dual-duplex baseband transmission concept was
adopted. Bidirectional 100 Mbit/s transmission over two pairs is accomplished by full-duplex trans-
mission of 50 Mbit/s streams over each of two wire pairs. The lower modulation rate and/or spectral
19.2. Dual-duplex system 907

0
Amplitude characteristic
for 100 m cable length 16 MHz

–10 NEXT coupling envelope curve

–21 + 15 log10 ( f/16 ), f in MHz
–14.6 dB

–20 –21.0 dB
Amplitude (dB)

–30

–40

Four NEXT coupling functions

–50

–60
0 5 10 15 20 25 30 35 40
f (MHz)

Figure 19.11 Amplitude of the frequency response for a voice-grade twisted-pair cable with length
equal to 100 m, and four realizations of NEXT coupling function. Source: Reproduced with
permission from Cherubini et al. [2]. ©1997, IEEE.

master slave

Tx 50 Mbit/s Tx
100 Mbit/s 100 Mbit/s
H H
pair A
Rc Rc
echo
external loop
self-NEXT timing
timing
Tx Tx
pair B
H H
100 Mbit/s 100 Mbit/s
Rc 50 Mbit/s Rc
echo

Figure 19.12 Dual-duplex transmission over two wire pairs. Source: Reproduced with permission
from Cherubini et al. [2]. ©1997, IEEE.

modulation efficiency required per pair for achieving the 100 Mbit/s aggregate rate represents an obvi-
ous advantage over mono-duplex transmission, where one pair would be used to transmit only in one
direction and the other to transmit only in the reverse direction. Dual-duplex transmission requires two
transmitters and two receivers at each end of a link, as well as separation of the simultaneously trans-
mitted and received signals on each wire pair. Sufficient separation cannot be accomplished by analog
908 Chapter 19. High-speed communications over twisted-pair cables

hybrid circuits only. In 100BASE-T2 transceivers, it is necessary to suppress residual echoes returning
from the hybrids and impedance discontinuities in the cable as well as self NEXT by adaptive digital
echo and NEXT cancellation. Furthermore, by sending transmit signals with nearly 100% excess band-
width, received 100BASE-T2 signals exhibit spectral redundancy that can be exploited to mitigate the
effect of alien NEXT by adaptive digital equalization. It will be shown later in this chapter that, for digital
NEXT cancellation and equalization as well as echo cancellation in the case of dual-duplex transmis-
sion, dual-duplex and mono-duplex schemes require a comparable number of multiply-add operations
per second.
The dual transmitters and receivers of a 100BASE-T2 transceiver will henceforth be referred to simply
as transmitter and receiver. Signal transmission in 100BASE-T2 systems takes place in an uninterrupted
fashion over both wire pairs in order to maintain timing synchronization and the settings of adaptive
filters at all times. Quinary pulse-amplitude baseband modulation at the rate of 25 MBaud is employed
for transmission over each wire pair. The transmitted quinary symbols are randomized by side-stream
scrambling. The redundancy of the quinary symbol sets is needed to encode 4-bit data nibbles, to send
between data packets an idle sequence that also conveys information about the status of the local receiver,
and to insert special delimiters marking the beginning and end of data packets.

Physical layer control

The diagram in Figure 19.13 shows in a simplified form the operational states defined for the
100BASE-T2 physical layer. Upon power-up or following a request to re-establish a link, an
auto-negotiation process is executed during which two stations connected to a link segment advertise
their transmission capabilities by a simple pulse-transmission technique. While auto-negotiation is in

Auto–Negotiation Process
100BASE–T2 available
TIMEOUT on both sides of the link

TRAINING STATE:
blind receiver training, followed
loc_rcvr_status = OK
by decision–directed training;
rem_rcvr_status = OK
send idle

loc_rcvr_status = NOT_OK loc_rcvr_status = OK

rem_rcvr_status = NOT_OK

IDLE STATE:
decision–directed receiver
operation; send idle

rem_rcvr_status = NOT_OK rem_rcvr_status = OK

loc_rcvr_status = NOT_OK
NORMAL STATE:
decision–directed receiver
operation; send idle or data

Figure 19.13 State diagram of 100BASE-T2 physical layer control. Source: Reproduced with
permission from Cherubini et al. [2]. ©1997, IEEE.
19.2. Dual-duplex system 909

progress, the 100BASE-T2 transmitters remain silent. If the physical layers of both stations are capable
of 100BASE-T2 operation, the auto-negotiation process further determines a master/slave relation
between the two 100BASE-T2 transceivers: the master transceiver will employ an externally provided
reference clock for its transmit and receive operations. The slave transceiver will extract timing from
the received signal and use this timing for its transmitter operations. This operation is usually referred
to as loop timing. If signals were transmitted in opposite directions with independent clocks, signals
received from the remote transmitter would generally shift in phase relative to the also-received echo
and self-NEXT signals, as discussed in the previous section. To cope with this effect, some form of
interpolation would be required, which can significantly increase the transceiver complexity.
After auto-negotiation is completed, both 100BASE-T2 transceivers enter the TRAINING state. In
this state, a transceiver expects to receive an idle sequence and also sends an idle sequence, which indi-
cates that its local receiver is not yet trained (loc_rcvr_status = NOT_OK). When proper local
receiver operation has been achieved by blind training and then by further decision-directed training,
a transition to the IDLE state occurs. In the IDLE state, a transceiver sends an idle sequence express-
ing normal operation at its receiver (loc_rcvr_status = OK) and waits until the received idle
sequence indicates correct operation of the remote receiver (rem_rcvr_status = OK). At this
time, a transceiver enters the NORMAL state, during which data nibbles or idle sequences are sent
and received as demanded by the higher protocol layers. The remaining transitions shown in the state
diagram of Figure 19.13 mainly define recovery functions.
The medium independent interface (MII) between the 100BASE-T2 physical layer and higher protocol
layers is the same as for the other 10/100 Mbit/s IEEE 802.3 physical layers. If the control line TX_EN
is inactive, the transceiver sends an idle sequence. If TX_EN is asserted, 4-bit data nibbles TXD(3:0) are
transferred from the MII to the transmitter at the transmit clock rate of 25 MHz. Similarly, reception of
data results in transferring 4-bit data nibbles RXD(3:0) from the receiver to the MII at the receive clock of
25 MHz. Control line RX_DV is asserted to indicate valid data reception. Other control lines, such as CRS
(carrier sense) and COL (collision), are required for carrier-sense-multiple-access/collision-avoidance
(CSMA/CD) specific functions.

Coding and decoding

The encoding and decoding rules for 100BASE-T2 are now described. During the k-th modulation
interval, symbols aAk and aBk from the quinary set {−2, −1, 0, +1, +2} are sent over pair A and pair B,
respectively. The encoding functions are designed to meet the following objectives:

● the symbols −2, −1, 0, +1, +2 occur with probabilities 1/8, 1/4, 1/4, 1/4, 1/8, respectively;
● idle sequences and data sequences exhibit identical power spectral densities;
● reception of an idle sequence can rapidly be distinguished from reception of data;
● scrambler state, pair A and pair B assignment, and temporal alignment and polarities of signals
received on these pairs can easily be recovered from a received idle sequence.

At the core of idle sequence generation and side-stream scrambling is a binary maximum-length
shift-register (MLSR) sequence {pk } (see Appendix 1.C) of period 233 − 1. One new bit of this sequence
is produced at every modulation interval. The transmitters in the master and slave transceivers gen-
erate the sequence {pk } using feedback polynomials gM (x) = 1 + x13 + x33 and gS (x) = 1 + x20 + x33 ,
respectively. The encoding operations are otherwise identical for the master and slave transceivers. From
910 Chapter 19. High-speed communications over twisted-pair cables

delayed elements {pk }, four derived bits are obtained at each modulation interval as follows:
xk = pk−3 ⊕ pk−8
yk = pk−4 ⊕ pk−6
(19.18)
ak = pk−1 ⊕ pk−5
bk = pk−2 ⊕ pk−12
where ⊕ denotes modulo 2 addition. The sequences {xk }, {yk }, {ak }, and {bk } represent shifted versions
of {pk }, that differ from {pk } and from each other only by large delays. When observed in a constrained
time window, the five sequences appear as mutually uncorrelated sequences. Figures 19.14 and 19.15
illustrate the encoding process for the idle mode and data mode, respectively. Encoding is based in
both cases on the generation of pairs of two-bit vectors (Xk , Y k ), (Sak , Sbk ), and (T ak , T bk ), and Gray-code
labelling of (T ak , T bk ) into symbol pairs (Dak , Dbk ), where D𝛾k ∈ {−2, −1, 0, +1}, 𝛾 = a, b. The generation
of these quantities is determined by the sequences {xk }, {yk } and {pk }, the even/odd state of the time
index k (equal to 2n or 2n + 1), and the local receiver status. Finally, pairs of transmit symbols (aAk , aBk )
are obtained by scrambling the signs of (Dak , Dbk ) with the sequences {ak } and {bk }. In Figure 19.16, the
symbol pairs transmitted in the idle and data modes are depicted as two-dimensional signal points.
We note that, in idle mode, if pk = 1 then symbols aAk ∈ x = {−1, +1} and aBk ∈ y = {−2, 0, +2}
are transmitted; if pk = 0 then aAk ∈ y and aBk ∈ x are transmitted. This property enables a receiver
to recover a local replica of {pk } from the two received quinary symbol sequences.
The associations of the two sequences with pair A and pair B can be checked, and a possible temporal
shift between these sequences can be corrected. Idle sequences have the further property that in every
two-symbol interval with even and odd time indices k, two symbols ±1, one symbol 0, and one symbol
±2 occur. The signs depend on the receiver status of the transmitting transceiver and on the elements

Master/Slave

bk ± 1
MLSR sequence S
ak ± 1 S: 0 +1
generator S 1 –1

x *2n + 1 = x 2n + 1 ⊕
(loc_rcvr_status = OK)
pk
n:even n:odd
x2n X2n x2n
*
+ 1 X2n + 1 a
xk Xk Sk = Tka Dk
a A
pk = 0: ak ϵ {–2, 0, +2}
1
0 [1,0] 0 [0,1] M
A
1 [0,1] 1 [1,0] pk = 1: ak ϵ {–1, + 1}
0
y2n Y2n Y2n + 1 b
yk Yk Sk = Tkb Dk
b B
pk = 0: ak ϵ {–1, +1}
0 [1,0] [1,1] M
1 B
1 [0,1] [0,0] pk = 1: ak ϵ {–2, 0, +2}
n:even even odd

[01] +1
M (Xk) ϵ {–1, +1} [00] 0
M: [10] (Gray labelling)
–1
M (Yk) ϵ {–2, 0}
[11] –2

Figure 19.14 Signal encoding during idle mode.

19.2. Dual-duplex system 911

Master/Slave

bk 1
MLSR sequence S
−1
S: 0
←
ak 1
generator S 1
←
+1

pk
[TDXk (3), TDXk (2)]

a a akA {−2, −1, 0, +1, +2}

a
∋
xk Sk Tk Dk
M

(same as for idle mode)

b b akB {−2, −1, 0, +1, +2}
∋
yk Skb Tk Dk
M

[TDXk (1), TDXk (0)]

Figure 19.15 Signal encoding during data mode.

idle mode 1 – D symbol data mode

probabilities
aB aB
k k
2 1/8 2

1 1/4 1
aA aA
k k
1/4
–2 –1 0 1 2 –2 –1 0 1 2

–1 1/4 –1

–2 1/8 –2

2 – D symbol probabilities: 1/8 1/16 2 – D symbol probabilities: 1/16 1/32 1/64

Figure 19.16 Two-dimensional symbols sent during idle and data transmission.

of the sequences {xk }, {yk }, {ak }, and {bk }. Once a receiver has recovered {pk }, these sequences are
known, and correct signal polarities, the even/odd state of the time index k, and remote receiver status
can be determined.
In data mode, the two-bit vectors Sak and Sbk are employed to side-stream scramble the data nibble bits.
Compared to the idle mode, the signs of the transmitted symbols are scrambled with opposite polarity.
In the event that detection of the delimiter marking transitions between idle mode and data mode fails
due to noise, a receiver can nevertheless rapidly distinguish an idle sequence from a data sequence by
912 Chapter 19. High-speed communications over twisted-pair cables

inspecting the signs of the two received ±2 symbols. As mentioned above, during the transmission of
idle sequences one symbol ±2, i.e. with absolute value equal to 2, occurs in every two-symbol interval.
The previous description does not yet explain the generation of delimiters. A start-of-stream delim-
iter (SSD) indicates a transition from idle-sequence transmission to sending packet data. Similarly, an
end-of-stream delimiter (ESD) marks a transition from sending packet data to idle sequence trans-
mission. These delimiters consist of two consecutive symbol pairs (aAk = ±2, aBk = ±2) and (aAk+1 =
±2, aBk+1 = 0). The signs of symbols ±2 in a SSD and in an ESD are selected opposite to the signs nor-
mally used in the idle mode and data mode, respectively. The choice of these delimiters allows detection
of mode transitions with increased robustness against noise.

19.2.1 Signal processing functions

The principal signal processing functions performed in a 100BASE-T2 transmitter and receiver are illus-
trated in Figure 19.17. The digital-to-analog and analog-to-digital converters operate synchronously,
although possibly at different multiples of 25 MBaud symbol rate. Timing recovery from the received
signals, as required in slave transceivers, is not shown. This function can be achieved, for example,
by exploiting the strongly cyclostationary nature of the received signals (see Chapter 14).

The 100BASE-T2 transmitter

A B
At the transmitter, pairs of quinary symbols (ak T , ak T ) are generated at the modulation rate of 1∕T = 25
MBaud. The power spectral density (PSD) of transmit signals sA and sB has to conform to the spectral
template given in Figure 19.18. An excess bandwidth of about 100%, beyond the Nyquist frequency
of 12.5 MHz is specified to allow for alien NEXT suppression in the receiver by adaptive fractionally

A SA (t)
ak T T
DTF DAC ATF
TXD (3:0) bit- 1 = 25 MBaud 1 F0
mapper T T T
SB (t)
T
DTF DAC ATF
BT
ak

TRANSMIT SECTION
adaptive FIR filters for pair A
RECEIVE SECTION
echo and NEXT cancellation
H
common
echo FIR timing pair B
next FIR H
symbol decisions and
adaptive FIR filters for echo FIR
feedback equalization
next FIR
fb FIR
A
a^k R symbol
ARF
ff FIR ADC &
VGA rA (t)
decision
inverse ff FIR*
RXD (3:0) 1 fb FIR* 1 F0
bit-
mapper T fb FIR* T ff FIR* T
ARF
symbol ff FIR ADC &
B decision rB (t)
a^k R
VGA
fb FIR adaptive FIR filters for
forward equalization;
*optional fractional-T spaced taps

Figure 19.17 Principal signal processing functions performed in a 100BASE-T2 transceiver. Source:
Reproduced with permission from Cherubini et al. [2]. ©1997, IEEE.
19.2. Dual-duplex system 913

–10

Power spectral density (dB)

–20

–30

–40

–50

–60

–70
0 10 20 25 30 40 50 60 70 80 90 100
f (MHz)

Figure 19.18 Spectral template specified by the 100BASE-T2 standard for the power spectral
density of transmit signals and achieved power spectral density for a particular transmitter
implementation comprising a 5-tap digital transmit filter, 100 MHz DAC, and a 3∘ order Butterworth
ATF. Source: Reproduced with permission from Cherubini et al. [2]. ©1997, IEEE.

spaced equalization, as explained below. The transmit signals are produced by digital pulse-shaping
and interpolation filters or digital transmit filters (DTFs), conversion from digital to analog signals
at a multiple rate of 25 Msamples/s, and ATFs. The PSD obtained with a particular implementation
in which the digital-to-analog conversion occurs at a sampling rate of 100 Msamples/s (oversampling
factor F0 = 4) is included in Figure 19.18.

The 100BASE-T2 receiver

The receive signals rA and rB are bandlimited by ARFs to approximately 25 MHz, adjusted in amplitude
by variable gain amplifiers (VGAs) and converted from analog-to-digital at a multiple rate of 25 Msam-
ples/s. For the following discussion, a sampling rate of 50 Msamples/s will be assumed (oversampling
factor F0 = 2, or equivalently sampling at the rate 2∕T).
The remaining receiver operations are performed digitally. Before detection of the pairs of quinary
A B
symbols (ak R , ak R ) transmitted by the remote transceiver, an adaptive decision-feedback equalizer (DFE)
structure together with adaptive echo and self NEXT cancellers is employed, as shown in Figure 19.17.
The forward equalizer sections of the DFE operate on T∕2-spaced input signals. The estimated echo and
self NEXT signals are subtracted from the T-spaced equalizer output signals. When signal attenuation
and disturbances increase with frequency, as is the case for 100BASE-T2 transmission, a DFE receiver
provides noticeably higher noise immunity compared to that achieved by a receiver with linear forward
equalization only. In Figure 19.17, additional feedback filters for direct-current (DC) restoration are not
shown. In a complete receiver implementation, these filters are needed to compensate for a spectral null
at DC introduced by linear transform coupling. This spectral notch may be broadened in a well-defined
manner by the ARFs and compensated for by non-adaptive IIR filters.
The use of forward equalizers with T∕2-spaced coefficients serves two purposes. First, as illustrated
in Section 7.4.3, equalization becomes essentially independent of the sampling phase. Second, when
914 Chapter 19. High-speed communications over twisted-pair cables

the received signals exhibit excess bandwidth, the superposition of spectral input-signal components
at frequencies f and f − 1∕T, for 0 < f < 1∕(2T), in the T-sampled equalizer output signals, can
mitigate the effects of synchronous interference and asynchronous disturbances, as shown in the
Appendix 19.4. Interference suppression achieved in this manner can be interpreted as a frequency
diversity technique [5]. Inclusion of the optional cross-coupling feedforward and backward filters shown
in Figure 19.17 significantly enhances the capability of suppressing alien NEXT. This corresponds to
adding space diversity at the expense of higher implementation complexity. Mathematical explanations
for the ability to suppress synchronous and asynchronous interference with the cross-coupled forward
equalizer structure are given in the Appendix 19.4. This structure permits the complete suppression
of the alien NEXT interferences stemming from another 100BASE-T2 transceiver operating in the
same multi-pair cable at identical clock rate. Alternatively, the interference from a single asynchronous
source, e.g. alien NEXT from 10BASE-T2 transmission over an adjacent pair, can also be eliminated.
The 100BASE-T2 standard does not provide the transmission of specific training sequences. Hence, for
initial receiver-filter adjustments, blind adaptation algorithms must be employed. When the MSEs at the
symbol-decision points reach sufficiently low values, filter adaptation is continued in decision directed
mode based on quinary symbol decisions. The filter coefficients can henceforth be continuously updated
by the LMS algorithm to track slow variations of channel and interference characteristics.
The 100BASE-T2 Task Force adopted a symbol-error probability target value of 10−10 that must
not be exceeded under the worst-case channel attenuation and NEXT coupling conditions when two
100BASE-T2 links operate in a four-pair UTP-3 cable, as illustrated in Figure 19.11. During the devel-
opment of the standard, the performance of candidate 100BASE-T2 systems has been extensively inves-
tigated by computer simulation. For the scheme ultimately adopted, it was shown that by adopting time
spans of 32T for the echo and self NEXT cancellers, 12T for the forward filters, and 10T for the feedback
filters, the MSEs at the symbol-decision points remain consistently below a value corresponding to a
symbol-error probability of 10−12 .

Computational complexity of digital receive filters

The digital receive filters account for most of the transceiver implementation cost. It is worthwhile to
compare the filter complexities for a dual-duplex and a mono-duplex scheme. Intuitively, the dual-duplex
scheme may appear to be more complex, because it requires two transceivers. We define the complexity
of a finite-impulse response finite impulse response (FIR) as
Filter complexity = time span × input sampling rate × output sampling rate
= number of coefficients × output sampling rate (19.19)
= number of multiply-and-adds per second.
Note that the time span of an FIR filter is given in seconds by the product of the number of filter coef-
ficients times the sampling period of the input signal. Transmission in a four-pair cable environment
with suppression of alien NEXT from a similar transceiver is considered. Only the echo and self NEXT
cancellers and forward equalizers will be compared. Updating of filter coefficients will be ignored.
For dual-duplex transmission, the modulation rate is 25 MBaud and signals are transmitted with about
100% excess bandwidth. Echo and self NEXT cancellation requires four FIR filters with time spans TC
and input/output rates of 25 Msamples/s. For equalization and alien NEXT suppression, four forward
FIR filters with time spans TE , an input rate of 50 Msamples/s and an output rate of 25 Msamples/s are
needed.
The modulation rate for mono duplex transmission is 1∕T = 50 MBaud and, signals are transmitted
with no significant excess bandwidth. Hence, both schemes transmit within a comparable bandwidth
19.A. Interference suppression 915

Table 19.1: Complexities of filtering for two transmission schemes.

Dual duplex Mono duplex

Echo and self NEXT cancellers 4 × TC × 25 × 1012 1 × TC × 50 × 50 × 1012
Forward equalizers 4 × TE × 50 × 25 × 1012 2 × TE × 50 × 50 × 1012

(≤ 25 MHz). For a receiver structure that does not allow alien NEXT suppression, one self NEXT can-
celler with time span TC and input/output rates of 50 Msamples/s, and one equalizer with time span TE
and input/output rates of 50 Msamples/s will be needed. However, for a fair comparison, a mono duplex
receiver must have the capability to suppress alien NEXT from another mono duplex transmission. This
can be achieved by receiving signals not only from the receive pair but also in the reverse direction of
the transmit pair, and combining this signal via a second equalizer with the output of the first equalizer.
The additionally required equalizer exhibits the same complexity as the first equalizer.
The filter complexities for the two schemes are summarized in Table 19.1. As the required time spans
of the echo and self NEXT cancellers and the forward equalizers are similar for the two schemes, it
can be concluded that the two schemes have the same implementation complexity. The same arguments
can be extended to the feedback filters. Finally, we note that with the filter time spans considered in the
preceding section (TC = 32T, TE = 12T and TFb = 10T), in a 100BASE-T2 receiver on the order of 1010
multiply-and-add operations/s need to be executed.

Bibliography

[1] Cherubini, G., Ölçer, S., and Ungerboeck, G. (1995). A quaternary partial response class-IV
transceiver for 125 Mbit/s data transmission over unshielded twisted-pair cables: principles of
operation and VLSI realization. IEEE Journal on Selected Areas in Communications 13: 1656–1669.
[2] Cherubini, G., Ölçer, S., Ungerboeck, G. et al. (1997). 100BASE-T2: a new standard for 100Mb/s
ethernet transmission over voice-grade cables. IEEE Communications Magazine 35: 115–122.
[3] Cherubini, G., Ölçer, S., and Ungerboeck, G. (1996). Adaptive analog equalization and receiver
front-end control for multilevel partial-response transmission over metallic cables. IEEE Transactions
on Communications 44: 675–685.
[4] IEEE Standard 802.3y (1997). Supplement to carrier sense multiple access with collision detec-
tion (CSMA/CD) access method and physical layer specifications: physical layer specification for
100 Mb/s operation on two pairs of Category 3 or better balanced twisted pair cable (100BASE-T2,
Clause 32).
[5] Petersen, B.R. and Falconer, D.D. (1991). Minimum mean-square equalization in cyclostationary and
stationary interference - analysis and subscriber line calculations. IEEE Journal on Selected Areas in
Communications 9: 931–940.

Appendix 19.A Interference suppression

Figure 19.19 illustrates the interference situations considered here. Equalization by linear forward filters
only is assumed. Reception of 100BASE-T2 signals is disturbed either by alien NEXT from another syn-
chronous 100BASE-T2 transmitter or by crosstalk from a single asynchronous source. Only one of these
A B
disturbances may be present. The symbol sequences {ak R } and {ak R } denote the sequences transmitted
A′ B′
by the remote 100BASE-T2 transceiver, whereas {ak T } and {ak T } denote the sequences transmitted
916 Chapter 19. High-speed communications over twisted-pair cables

Bʹ Aʹ
(a) {ak T } {ak T } (b) S( f )

decimation
nxt nxt xt
HBʹA ( f ) HAʹA (f) HSA ( f )
1 2
T T
* CAA ( f ) HAc ( f ) AR
ak
DPA
A
≃ ak R

CBA ( f ) nxt nxt xt

HBʹB ( f ) HAʹB ( f ) HSB (f)

c BR
DPB HB ( f ) ak

Figure 19.19 Crosstalk disturbance by: (a) alien NEXT from another synchronous 100BASE-T2
transmitter, (b) an asynchronous single source, for example a 10BASE-T transmitter. Source:
Reproduced with permission from Cherubini et al. [2]. ©1997, IEEE.

by an adjacent synchronous 100BASE-T2 transmitter. The spectrum S(f ) of the asynchronous source
may be aperiodic or exhibit a period different from 1∕T. The functions H(f ) represent the frequency
responses of the signal or crosstalk paths from the respective sources to the inputs of the forward equal-
izer filters with transfer functions CAA (f ) and CBA (f ). Because of 2∕T sampling rate, these functions
exhibit 2∕T-periodicity. All signals and filter coefficients are real-valued. It is therefore sufficient to con-
sider only frequencies f and f − 1∕T, for 0 < f < 1∕(2T). We will concentrate on the signals arriving at
decision point A (DPA); the analysis for signals at decision point B (DPB) is similar.
A
Intersymbol-interference free reception of the symbol sequence {ak R } and the suppression of signal
BR
components stemming from {ak } at DPA requires
( ) ( )
1 1
HAc (f ) CAA
c
(f ) + HAc f − CAA f − =1
T T
( ) ( ) (19.20)
1 1
HBc (f ) CBA
c
(f ) + HBc f − CBA f − =0
T T
To suppress alien NEXT from a 100BASE-T2 transmitter, two additional conditions must be met:
[ ( ) ( ) ( ) ( )]
∑ 𝓁 𝓁 𝓁 𝓁
nxt nxt
HA′ A f − CAA f − + HA′ B f − CBA f − =0
𝓁=0,1
T T T T
[ ( ) ( ) ( ) ( )] (19.21)
∑ 𝓁 𝓁 𝓁 𝓁
HBnxt′ A f − CAA f − + HBnxt′ B f − CBA f − =0
𝓁=0,1
T T T T
Alternatively, the additional conditions for the suppression of crosstalk caused by a single asynchronous
source become
xt xt
HSA (f ) CAA (f ) + HSB (f ) CBA (f ) = 0
( ) ( ) ( ) ( ) (19.22)
xt 1 1 xt 1 1
HSA f− CAA f − + HSB f− CBA f − =0
T T T T
Therefore, in each case the interference is completely suppressed if for every frequency in the interval
0 < f < 1∕(2T) the transfer function values CAA (f ), CAA (f − (1∕T)), CBA (f ) and CBA (f − (1∕T)) satisfy
four linear equations. It will be highly unlikely that the crosstalk responses are such that the coefficient
matrix of these equations becomes singular. Hence a solution will exist with high probability. In the
absence of filter-length constraints, the T∕2-spaced coefficients of these filters can be adjusted to achieve
these transfer functions. For a practical implementation, a trade-off between filter lengths and achieved
interference suppression has to be made.
Index

M-algorithm, 591 acknowledgement, see ACK

k-means clustering, 239, 246 ACS, 33, 68
UTP, 893 ADC, 186, 470, 894
3GPP, 443–445, 460, 859 additive Gaussian noise, see AGN
channel model, 200 ADSL, 876, 877
new radio, 861 AGC, 893
release 15, 857–859 AGN, 267
3rd generation partnership project, Alamouti code, 459
see 3GPP aliasing, 17, 84, 315
5G, 857 ALOHA, 891
DFT size, 866 alternate mark inversion, 398
initial access, 864 AM/AM, 189
layer, 861 AM/PM, 189
MIMO, 862, 864 amplifier
precoding, 867 high power, see HPA
radio access network, 859 MC modulation, 431
radio frame, 859 solid state power, 190
analog front-end, 185
slot, 859
analog-to-digital converter, see ADC
subcarrier spacing, 861
analytic signal, 21
subframe, 859
angle of arrival, 218, 220
angle of departure, 219, 220
configurable control resource set, angular domain, 226, 472
see CORESET antenna
Cramér-Rao lower bound, 755 effective area, 194
fast Fourier transform, see FFT efficiency, 194
global system for mobile communications, gain, 194
see GSM horn, 194
medium-access control, see MAC isotropic, 193
Walsh-Hadamard sequence, 484 parabolic, 194
selection, 464
access AR, 67, 69, 70, 74, 117, 760
code division multiple, see CDMA argument, 106
frequency division multiple, see FDMA arithmetic
time division multiple, see TDMA modulo-q, 528
access methods, 890 ARMA, 65, 67, 70, 110
achievable bit rate, 269 ARQ, 515
achievable rate, 687, 689 asymmetric DSL, see ADSL
maximization, 692, 695 asynchronous transfer mode, see ATM
ACK, 872 ATM, 880

Algorithms for Communications Systems and their Applications, Second Edition.

Nevio Benvenuto, Giovanni Cherubini, and Stefano Tomasin.
© 2021 John Wiley & Sons Ltd. Published 2021 by John Wiley & Sons Ltd.
918 Index

authentication centre, 870 Bose-Chaudhuri-Hocquenhem

auto-regressive, see AR code, see BCH
auto-regressive moving average, see ARMA theorem, 561
autocorrelation, 29, 30, 35, 39, 72 BPSK, 263, 787, 806, 816
autocorrelation sequence, see ACS differentially encoded, 791
autocovariance, 29 BS, 242, 857
automatic gain control, see AGC BSC, 262
automatic repeat request, see ARQ memoryless, 262
AWGN, 191, 252, 687, 754 buffer looking algorithm, 591
Butterworth filter, 211, 214
back-off, 189
input, see IBO capacity, 249, 265, 687
output, see OBO discrete-time AWGN channel, 266
backward metric, 349 equally likely symbols, 275
Bahl-Cocke-Jelinek-Raviv, see BCJR MIMO discrete-time NB AWGN channel,
band, 15 269
industrial, scientific, and medical, see ISM MIMO dispersive channel, 270
bandwidth, 14, 15 PAM, 266
3 dB, 818 QAM, 266
coherence, 209 SISO dispersive AGN channel, 267
minimum, 260 SISO narrowband AWGN channel, 266
base station, see BS carrier
base-band, see BB frequency offset, see CFO
baseband equivalent, 26 frequency recovery, 740
Baud, 250 phase recovery, 725
BB, 20 recovery, 716, 719
BCH, 558, 560, 564 catastrophic error propagation, 585
binary, 562 CAZAC, 382, 756, 866, 885
decoding, 566 CDI, 242
BCJR, 347 CDMA, 483, 489, 490, 502, 891
beam sweeping, 864, 867 asynchronous, 490
beamforming, 452 synchronous, 490
belief propagation algorithm, see sum-product cell, 857
algorithm CFO, 754
Benveniste-Goursat algorithm, 766 channel, 183, 252
Berlekamp algorithm, 569 3GPP model, 200
Berlekamp-Massey algorithm, 569 binary erasure, 518
Bessel function, 211 binary symmetric, 518
modified, 202 capacity, see capacity
bias, 60, 383 coding, 249
binary phase-shift keying, see BPSK direction indicator, see CDI
binary symmetric channel, see BSC discrete time model, 213, 216, 219
bit dispersive, 196, 354
interleaver, 249 encoder, 249
period, 260 equal gain model, 830
and power loading, 691 estimation, 441
rate, 260 fading, 200
Blackman and Tukey correlogram, 63 flat fading, 209
block diagonalization, 473, 477 frequency selective fading, 209
blockage, 222 hardening, 478
Bluetooth, 875 hilly terrain model, 830
BMAP, 250, 256, 353, 390 impulse response, see CIR
Boltzmann constant, 195 indoor wireless, 199, 206
Index 919

memoryless, 262 polar, 627

model, one ray, 196 quasi-perfect, 527
multiantenna, 218 return-to-zero, 396
multipath, 206 serial concatenated convolutional,
narrowband radio, 193
611
orthogonality, 479
simplex cyclic, 556
outdoor wireless, 206
parallel, 687 space-time, 459
polarization, 627, 643 systematic form, 521
quality indicator, see CQI, see CQI tree, 576
radio, see channel, wireless turbo, 468, 597, 627
Rayleigh model, 208 vector, 229
rural area model, 830 codebook, 229
shadowing, 200 coding
state information reference signal, see CSIRS Golay, 525
telephone, 222
coherence distance, 212
two-ray, 204, 208
urban area model, 830 coherence time, 211
virtual, 429 combining
wideband, 204 equal gain, 454
wirelss, 183 maximal ratio, see MRC
channel estimation, 378, 476, 478 MIMO, 455
5G, 862 MMSE, 456
5G downlink, 865 MMSE optimum, 454
5G uplink, 866 selective, 453
MIMO, 461
SIMO, 452
characteristic equation, 43
switched, 453
Cholesky decomposition, 124
Chow-Cioffi Bingham algorithm, 696 zero-forcing, see ZF
CIR, 204, 432, 751 complementary Gaussian cumulative distribution
clamping, 190 function, see Q function
Clarke model, 211 constant amplitude zero autocorrelation, see
clear to send, see CTS sequence, CAZAC
cluster, 857 constant-modulus algorithm, 775
code, 272 constraint length, 579
block, 468, 517, 575 continuous
canonical form, 521
phase FSK, see CPFSK
channel, 515
concatenated, 593 phase modulation, see CPM
convolutional, 576, 659, 871 contour algorithm, 767, 769, 772, 776
cyclic, 530, 545 convergence, 784
dirty paper, see DPC convolution, 8
division multiple access, see CDMA circular, 8, 9, 433
gain, 274, 515 linear, 9
group, 519 CORESET, 862
line, 394 correlation, 29, 107
linear, 653
antenna, 219
low-density parity check, see LDPC
average, 39
mother, 644
non-return-to-zero, 395 matrix antenna, 221
nonbinary parity check, 652 coset, 467, 523, 656
parity check, 520 leader, 524
perfect, 527 table, 524
920 Index

Costas loop, 716, 718 DECT, 873, 875, 890

extended, 719 dehopping, 491
covariance method, 119 delay
CP, 10, 444, 751, 861 envelope, 18
extended mode, 861 group, 18
normal mode, 861 power spectrum, 206
CPFSK, 811, 812, 831 delay spread
CPM, 811, 831 excess, 205
CQI, 242, 865 multipath, 205
Cramér-Rao lower bound, 131, 755 root mean square, 205
vector parameter, 132 delay-locked loops, see DLL
CRC, 556 Delsarte-Genin algorithm, see DGA
internal, 630 demodulation, 252
critical communications, 859 coherent, 791, 808
cross power spectral density, 31 limiter-discriminator FM, 809
crosscorrelation, 29 low-complexity, 787
crosscovariance, 29 non-coherent, 808
crosstalk, 833, 835, 880 reference signal, see DMRS
CSI, 242, 452, 463, 469, 473, 477 density evolution, 641
at the transmitter, see CSIT despreading, 488, 504
reporting, 865 detection, 253, 340, 502
CSIRS, 862, 865 baseband differential, 793
CSIT, 469, 473, 476, 477 envelope, 801
CSMA, 872 hard, 255
CSMA with collision avoidance, see CSMA/CA IF-band (1 Bit) differential, 794
CSMA/CA, 872 interference cancellation, 506
CSMA/CD, 880, 891 joint, 505
CTS, 872 maximum likelihood, 254
cyclic MIMO, 457
prefix, see CP ML multiuser, 508
redundancy check, see CRC multiuser, 504, 846
optimum, 253
DAC, 88, 410, 470, 894 reduced-state sequence, 365, 369
damping factor, 713 single-user, 502
data reduction, 248 soft, 256
DBPSK, 806, 814 detector
DCF, 893, 895 threshold, 294
decimation, 81, 84, 94 DFE, 315, 317, 318, 331, 406, 434, 455, 506, 846
factor, 81 fractionally spaced, 320
decision feedback equalizer, see DFE frequency domain, 323
decoding, 523 DFSE, 369
convolutional code, 586 DFT, 7, 9, 425
hard, 552 inverse, 7
iterative, 600 spreading, 859
MIMO, 457 DFT-spread OFDM, see SC-FDMA
min-sum, 625 DGA, 73, 142, 145
SC flip, 639 difference metric algorithm, 595
SC stack, 639 differential phase shift keying, see DPSK
sequential, 590 differentiator, 735
successive cancellation, 635 diffraction, 184
successive cancellation list, see SCL digital European cordless telecommunications, see
syndrome, 525 DECT
decorrelation filter, see DCF digital transmission model, 249
Index 921

digital video broadcasting, see DVB EHF, 185

digital-to-analog converter, see DAC eMBB, 628, 858, 867
dimensions of the symbol constellation, 260 encoding, 552
direct sequence, see DS component, 598
direction of arrival, 218 differential, 791
discrete Fourier transform, see DFT recursive systematic convolutional, 597
discrete multitone modulation, see DMT energy
discrete time linear system, 2 per dimension, 260
discriminator per information bit, 260
limiter, 797 pulse, 260
distance enhanced mobile broadband, see eMBB
minimum, 652 envelope, 28
distortion, 222, 231 transfer function, 189, 431
Itakura-Saito, see Itakura-Saito distortion equalization, 260, 304, 455
measure, 230 adaptive, 895, 900
phase, 293 adaptive linear, 311
square, 232 blind, 759
diversity, 455 block, 504
receive, 452 chip, 502
spatial, 451 fractionally spaced, see FSE
transmit, 452 frequency domain, 323
DLL, 745 IIR, 306
coherent, 747 linear, see LE
DMRS, 862, 865, 866 linear zero-forcing, 326
DMT, see OFDM, 879 passband, 335
Doppler per subchannel, 434
shift, 202–204 self-adaptive, 759
spectrum, 210, 211 self-recovering, 759
spectrum classical, 211 self-training, 759
spread, 203 symbol, 503
double zero forcing DFE, 327
conversion and wideband IF, 188 zero-forcing, 304
sideband, see DSB erasure, 518
downlink, 443, 490, 504, 870, 885 erf, 278
downsampling, see decimation error
DPC, 467 backward linear prediction, 117
DPLL, 744 probability, 262
DPSK, 260, 787 event, 358
error probability, 789 function, see erf
DS, 483 indicator polynomial, 567
DSB, 25, 40 estimation
DSL, 875, 876, 878 bias, 127
duplex, 890 correlation, 60
dual, 835, 882, 906 frequency offset, 741
frequency-division, see FDD linear MMSE, 129
full, 835, 838 mean, 57
full over a single band, 890 MMSE, 127
mono, 835, 907 multiple observations, 128
time-division, see TDD phase, 725
DVB, 460, 599 phase rotation, 725
phasor, 738
echo, 224, 833, 842 PSD, 61
efficiency random variable, 126
spectral, 260, 261 timing, 723, 727, 729, 746
922 Index

Ethernet, 875 exponential, 59

ETSI, 872 feedback, see FB
Euclidean feedforward, see FF
distance, 232 finite impulse response, see FIR
division algorithm, 532 frequency matched, 742
European frequency response, 5
telecommunications standard institute, highpass, see HPF
see ETSI Hilbert, 21, 23
terrestrial trunked radio, see TETRA impulse response, 1, 14
excess bandwidth parameter, see roll-off infinite impulse response, see IIR
eye diagram, 298 integrate and dump, 797
interpolator, 14, 49
fading, see channel, fading, 204, 211, 213 Kalman, 171
coherence distance, 201 lattce, 144
fast, 211 lowpass, see LPF
flat, 204 matched, see MF
frequency selective, 204 minimum phase, 14, 35
macroscopic, 200 passband, see PBF
microscopic, 201 prototype, 419
slow, 211 square root raised cosine, 429
Fano algorithm, 590 transfer function, 5
far-end crosstalk, see FEXT transversal, 105
fast recursive algorithm, 177 adaptive, 147
fast transversal filter, see FTF whitening, 70, 328, 508
faster-then-Nyquist signalling, see FTN Wiener, 105, 147, 307
favourable propagation, 479 Wiener z-domain, 111
FB, 317 filter bank, 422
FBA, 588, 589, 601 multicarrier, see FBMC
soft output, 589 non critically sampled, 422
FBMC, 445 filtered multitone modulation, see FMT
FDD, 242, 469, 860, 870, 877, 890 finite-state machine, 393, 661
FDMA, 442, 874, 891 FIR, 11, 14, 105, 147
FEC, 515 FM, 797
FEXT, 833, 850, 851, 877 discriminator, 797
FF, 317, 319 FMT, 427, 434, 445, 879
FFH, 492 foliage, 201
FFT, 7 Forney
radix-2, 7 algorithm, 572, 573
radix-3, 7 receiver, 355
radix-5, 7 forward
FH, 483, 491 backward algorithm, 348, see FBA
fast frequency-hopped, see FFH error correction, see FEC
slow frequency-hopped, see SFH metric, 348
fibre-to-the-home, 877 Fourier
field, 529 symplectic discrete transform, see SDFT
finite, 535 transform, 5, 17
Galois, 575 properties, 2
filter transform on the field GF(q), see GFFT
analytic, 19, 23 free distance, 659, 661
bank, uniform, 419 frequency
chip matched, 498 carrier, 19, 20, 24, 42
decimator, 14, 84 division duplexing, see FDD
dicode, 408 division multiple access, see FDMA
Index 923

drift, 193 Hadamard matrix, 511

extremely high, see EHF Hamming
high, see HF code, 554
hopping, see FH distance, 517, 518
instantaneous, 28 distance minimum, 518
intermediate, 186, 411 hand-off, 858
low, see LF hand-over, 858
medium, see MF Hata model, 200
modulation, see FM HDSL, 876, 890
super high, see SHF Heaviside conditions, 17, 292
ultra high, see UHF HF, 185
very high, see VHF HFC, 881
Friis transmission equation, 195 high bit rate digital subscriber line, see HDSL
Frobenius norm, 453 home location register, 869
frozen set, 640 Householder transformation, 126
FSE, 313, 317 HPA, 189, 431, 443, 807, 828
DFE, 320 HPF, 16
FSK, 491, 802, 804, 806, 807, 875 Hughes-Hartogs algorithm, 694
non-coherent binary, 798 hybrid, 224
FSM, 343 beamforming, 470
FTF, 177 fibre/coaxial, see HFC
FTN, 386
full response scheme, 832 IBMAP, 253
function minimum, 541 IBO, 189, 432
ICI, 415, 427, 432
gain identification
asymptotic coding, 664 continuous-time system, 135
Gauss method, 123 linear discrete-time system,
Gaussian MSK, see GMSK 134, 178
Gaussian variable, 700 IDFT, see DFT, 424
generalized frequency division multiplexing , IIR, 11, 14, 65, 147, 214
see GFDM image rejection, 186
GFDM, 445 impulse response
GFFT, 575 partial energy, 13
girth, 616 information message bits, 260
GMSK, 816, 819, 875 initial access, 864
1BDD demodulator, 827 integrated services digital network, see ISDN, see
linear approximation, 824 ISDN
gNB, 859, 864, 866 interference
Gold sequence, 103, 863 adaptive cancellation, 837
Gold sequence sequence, 102 cancellation, 138, 833, 836
Golden code, 461 co-channel, 490
gradient algorithm, 148 inter-path, 501
Gray labelling, 250, 251, 263, 277, 789, 792 intracell multiuser, 490
group multiuser, 243, 486, 490
Abelian, 529 parallel cancellation, see PIC
finite, 529 rejection, 496
order, 529 successive cancellation, see SIC
GSM, 868, 890 interleaver
bandwidth allocation, 870 spread, 600
channel delay profile, 206 interleaving, 587
Gudmundson model, 201 Internet of things, see IoT
924 Index

interpolation, 17, 83, 86, 90, 94, 441 line coding, 249
factor, 83 line of sight, see LOS
linear, 90 linear
quadratic, 91 amplifier, 186
inverse Fourier transform, see Fourier transform, see minimum mean-square error, see LMMSE,
DFT 130
IoT, 445, 858, 868 predictive coding, see LPC
ISDN, 837, 869 system, 1
ISI, 253, 294, 361, 415, 427, 432, 589, 698, 760, 798 LL, 255, 509, 800
ISM, 872, 875 Lloyd algorithm, 233, 235
Itakura-Saito distortion, 232 generalized, 233
LLR, 256, 260, 353
Jacobi algorithm, 126 simplified expression, 258
Jakes model, 211 LMMSE, 129, 130, 385, 440, 463
jamming, 494 LMS, 153, 163, 177
Jensen inequality, 243 convergence, 155, 156
joint spatial division and multiplexing, see JSDM leaky, 164
JSDM, 475 normalized, 164
sign algorithm, 164
Kalman LO, 192
filter fast, 177 local loop, 140
vector gain, 173 lock condition, 709
kissing number, 667 log likelihood
Kolmogorov theorem, 72 function, 352
Kronecker ratio, see LLR
impulse, 4 Log-MAP, 352, 354
product, 221 loop timing, 835
Krongold-Ramchandran Jones algorithm, LOS, 184, 193, 218
694 LPC, 230
Kumar fast algorithm, 124 LPF, 15
LQ factorization, 466
Laplace transform, 2 LS, 118, 119, 123, 379, 438, 462
lattice, 666 LTE, 242, 460, 756
characteristic volume, 667
Gosset, 668 M-algorithm, 365
Schläfli, 668, 681 m-sequence, see MLS
LBG, 235 MA, 67, 70
LDA, 142 MAC, 872, 881, 885
split, 145 machine learning, 147, 149
LDPC, 468, 614, 627, 860 MAP, 127, 254, 341, 602
decoding, 617 mapper
graphical representation, 615 bit, see BMAP
irregular, 615 Marcum function, 279
matrix representation, 614 margin, 690
regular, 615 maximization, 695
LE, 305, 306, 310, 455 maximization problem, 692
least mean square, see LMS massive
least squares, see LS machine type communications, see mMTC
Levinson-Durbin algorithm, 73, see LDA MIMO, 478
LF, 184 matrix
likelihood, 254, 255 autocorrelation, 43, 107, 110
function, 347, 351 circulant, 8
ratio, 256 crosscorrelation, 128
Index 925

determinant, 43 alphabet, 260

diagonal, 9 interval, 260
Fisher information, 132 multicarrier, see MC
generalized parity check, 654 rate, 260
generator, 522, 581, 655 single carrier, see SC
Hermitian, 45, 73, 124, 142 spatial, 458, 510
identity, 8 trellis coded, see TCM
parity check, 520 moving average, see MA
Toeplitz, 73, 124, 142 MRC, 454
trace, 45 MS, 869
transformation, 629 MSE, 893
unitary, 8 MSK, 810, 812, 814, 815
Vandermonde, 560, 568 demodulation, 810
Max-Log-MAP, 257, 352, 354, 605 Mueller & Muller algorithm, 735
maximum multiantenna, 218
likelihood, see ML multipath, 196, 204
likelihood sequence detection, intensity profile, 206
see MLSD multiple-access, 496
a posteriori probability, see MAP channel, 476
MC, 242, 426, 429, 444, 460 multiplexing
capacity, 687 spatial, 451
efficient implementation, 419 time-division, see TDM
synchronization, 751 multirate system system, 79
MCTL, 747
multiuser
MD, 460
access, 442, 472
mean, 29
MIMO, 472, 476
message passing algorithm, see sum-product
receiver, 489
algorithm
mutual information, 265
MF, 52, 53, 184, 303, 698
Miller code, 398
millimeter wave, see mmWave narrowband, see NB
MIMO, 219, 269, 270, 447, 472, 687, 834, 859 NB, 193, 197, 447
channel, 450 near-end crosstalk, see NEXT
massive, 478 network function virtualization, see NFV
mini-slot, 886 network
minimum slicing, 867
mean square error, see MMSE switching subsystem, 869
shift keying, see MSK neural network, 149
MISO, 242, 452, 459, 479 new generation node base, see gNB
ML, 127, 254, 341, 461, 590, 800 NEXT, 833, 877, 895, 897
receiver, 799 NFV, 868
synchronization, 726 noble identities identities, 91
MLS, 99, 102 noise
MLSD, 341, 342, 356, 358, 410, 509, 906 additive, 191
mMMT, 859 additive white Gaussian, see AWGN
MMSE, 454, 456 figure, 195
mMTC, 867 ingress, 885
mmWave, 226, 470, 479, 859, 860, 864 predictor, 331
mobile station, see MS quantization, 222, 224
mod, 80 temperature, 195, 260
modified code tracking loop, see MCTL thermal, 224
modified duobinary, 362 non-run zero, see NRZ
modulation, 252 NRZ, 815
926 Index

Nyquist partial response, 362, 399, 400, 893

criterion, 293, 296 scheme, 832
frequency, 296 self-training equalization, 770
pulse, 306, 314, 735 partition, 230
passband, see PB
OBO, 189, 431, 828 path loss, 197
OFDM, 242, 326, 426, 432, 435, 437, 450, 479, 687, free space, 195
859, 862 path memory depth, 346
channel estimation, 441 PB, 18
echo cancellation, 844 PBCH, 862, 863
release 15, 861 PBF, 15, 18
synchronization, 751 PBO, 850, 851
OFDMA, 244, 442 PCA, 244, 246
offset QAM, see OQAM PDCCH, 862, 863
on–off keying, see OOK PDP, 206, 208, 209
OOK, 799, 802 PDSCH, 862
operation maintenance centre, 870 peak to average power ratio, see PAPR
operator per-survivor processing, 369
Hermitian, 8 periodogram, 61
transpose, 8 Perron–Frobenius theorem, 162
optical network unit, 877 phase
OQAM, 445 instantaneous, 28
orthogonal
jitter, 192, 222
frequency division multiple access,
noise, 191
see OFDMA
shift keying, see PSK
frequency division multiplexing,
splitter, 19
see OFDM
timing, 95
time frequency space, see OTFS
tracking reference signal, see PTRS
orthogonality
phase-locked loop, see PLL
multicarrier modulation, 414
phasor-locked loop, 740
principle, 109, 121
oscillator, 705 physical
numerically controlled, 728 broadcast channel, see PBCH
local, 186 downlink control channel, see PDCCH
OTFS, 437 downlink shared channel, see PDSCH
overlap-save method, 10 random access channel, see PRACH
uplink control channel, see PUCCH
P/S, 94 uplink shared channel, see PUSCH
Paley-Wiener criterion, 35 PIC, 507
PAM, 250, 271, 282, 302, 394, 660, 668, pilot symbol, 438, 441, 461, 705
834 block-type, 441
achievable rate, 275 comb-type, 441
capacity, 266 PLL, 192, 705, 707, 718
equalization, 765 baseband model, 708
modulator, 281 digital, see DPLL
PSD of the signal, 284 linear approximation, 709
symbol error rate, 263 second order, 713
synchronization, 716 second order digital, 744
PAPR, 432, 443–445 PLMN, 868
parallel interference cancellation, see PIC PN, 99, 312, 324, 378, 556, 752
parallel–to–serial, see P/S Poisson formula, 6
Parseval theorem, 2, 61 polynomial
partial correlation coefficient, 143 root, 538
Index 927

polyphase, 412, 418 PSS, 862, 863

component, 93 PSTN, 870
decomposition, 384 PTRS, 862, 866
representation, 92, 93 public land mobile network, see PLMN
structure, 836 public switched telephone network, see PSTN
postcursor, 315 PUCCH, 863, 865, 866
power pulse
allocation, 688, 690 Nyquist, see Nyquist, pulse
back-off, see PBO raised cosine, 296
delay profile, see PDP square root raised cosine, 298
spectral density, see PSD puncturing, 644
statistical, 29, 30 PUSCH, 865
PRACH, 865, 866
precoding, 323, 373, 406, 435, 452, 477, Q function, 263, 278
867 QAM, 251, 257, 259, 271, 285, 302, 410, 669, 689,
diagonal, 466 691, 705, 775, 776, 824, 842
differential, 815 baseband equivalent model, 288
flexible, 376 capacity, 266
linear, 243 coherent receiver, 288
linear MISO, 463 equalization, 773
MIMO, 465 modulator, 285
MIMO non-linear, 466 PSD of the signal, 286
regularized ZF, see RZF simulation, 389
Tomlinson-Harashima, see THP symbol error rate, 263
ZF, 465 synchronization, 705, 719, 732
precursor, 315
QPSK, 251, 256, 370, 508, 816
prediction, 166, 177
synchronization, 719
forward linear, 115
quadrature phase shift keying, see QPSK
linear, 114
quantization
primary synchronization signal, see PSS
optimum, 233
principal component analysis, see PCA
rate, 231
probability
scalar, 229, 232
a posteriori, 620
vector, see VQ
a priori, 606
quantizer
process
noise, 222
analysis, 117
quotient group, 668
baseband, 36
circularly symmetric Gaussian, 47
cyclostationary, 38, 49, 51 r-sequence, see MLS
ergodic, 55 radiation mask, 185
Gaussian, 46 radio propagation, 183
orthogonal, 29 rake receiver, 496
passband, 36 RAN, 867
synthesis, 35, 117 random variable
wide-sense stationary, 30 Gaussian, 46, 98
projection operator, 122 Pareto, 590
PSD, 30, 32, 33, 43 Rayleigh, 45, 202, 207
average, 39 Rice, 202, 207
discrete-time random processes, 32 ranging request message, see RNG-REQ
PSK, 251, 260, 271, 287, 372, 438, 460, 660, 670, rank indicator, see RI
738, 744, 792, 831 rate
demodulation, 787 achievable, 270
symbol error rate, 263 code, 515
synchronization, 719, 729 conversion, 88
928 Index

rate (contd.) SDR, 187

encoder-modulator, 260, 261, 516 secondary synchronization signal , see SSS
per dimension, 231 self-mixing, 188
Rayleigh quotient, 45 semi-orthogonal user selection, 475
RB, 860, 862, 863 sensitivity, 260
RE, 862 sequence
receiver, 252, 293 anticausal, 36
coherent, 787 CAZAC, 101, 866
differential non-coherent, 789 maximal-length, see MLS
direct conversion, 187 PN, 33, 137
non-coherent, 798 pseudo-noise, see PN
optimum, 305, 798 signature, 484
recursive least squares algorithm, see RLS training, 235, 238, 312
Reed-Solomon code, 564 Zadoff-Chu, 101, 756
reflection, 184, 197, 198 serial–to–parallel, see S/P
reflection coefficient, 197 set
regression, 114 frozen, 629
request to send, see RTS SFH, 492
resource shadowing, 200
block, see RB Shannon
element, see RE limit, 274
I, 865 theorem, 265
Rice factor, 202 shaping, 276, 374
ring, 529 pulse, 485
commutative, 529 SHDSL, 876
RLS, 171, 177 SHF, 185
convergence, 176 shift register, 99, 551
RNG-REQ, 886 short message, see SMS
roll-off, 298 shortening, 645
root mean square delay spread, 208 SIC, 506
RTS, 872 signal
Rummler model, 204 analysis, 24
RZF, 466 analytic, 19
baseband, 15, 282
S/P, 94 baseband equivalent, 21
sampling, 17 complex envelope, 19, 21, 36
minimum frequency, 17 complex representation, 18
theorem, 17 distortion, 17
Sato algorithm, 765, 769, 770 dither, 376
QAM, 773 in-phase component, 21
SC, 413, 431, 444, 447 passband, 15, 21, 288
capacity, 698 quadrature component, 21
SC-FDMA, 442, 443 theory, 1
scatterer, 219 signal-to-interference-plus-noise ratio, see SINR
scattering, 184 SIMO, 452
Schmidl and Cox algorithm, 752, 755, 867 single
Schur algorithm, 124 carrier FDMA, see SC-FDMA
Schwarz inequality, 53 conversion to low-IF, 188
SCL, 628, 634, 638 line high-bit-rate digital subscriber line, 591
scrambler, 249 line high-speed DSL, see SHDSL
SDFT, 437 sideband, see SSB
SDMA, 451 user MIMO, 472
SDN, 868 singular value decomposition, see SVD
Index 929

SINR, 507 synchronization, 705, 751, 756

sinusoidal interferer, 181 5G, 863
SISO, 266, 450, 467, 687 5G uplink, 866
SMS, 869 carrier frequency recovery, 740
SNR, 195, 260, 302, 310, 438 carrier phase recovery, 725
conventional, 261 carrier recovery, 716
gap, 690 CDMA, 490
per dimension, 260, 262 data aided, 726, 733, 744
soft-input soft-output, 589 data directed, 733, 735, 738, 739, 744
software decision directed, 726
defined networking, see SDN early-late, 728
defined radio, see SDR frequency, 725
sounding reference signal, see SRS frequency offset estimation, 741
source, 249 non-data aided, 726, 729, 732, 738, 742–744,
coding, 249 747
SOVA, 589, 593, 596 phasor estimation, 738
modified, 596 timing, 721
space shift keying, 459 syndrome, 467, 525, 657
space-division multiple access, see SDMA system
spectrum anticausal, 4
angular, 211 causal, 4
classification, 184 identification, 140
spread-spectrum, 102, 493 maximum phase, 12
synchronization, 745 minimum phase, 12
system, 483 multirate, 79, 384
timing estimation, 746 stable, 12
spreading, 483
Tanner graph, 631
squarer, 717, 718
TCM, 333, 659, 832
SRS, 866
linear, 660
SSB, 39, 40, 863, 864
rotationally invariant, 684
SSS, 862
TDD, 242, 469, 860
state metric, 350 TDM, 891
steepest descent algorithm, see gradient TDMA, 515, 870, 886, 891
algorithm TETRA, 445
STO, 751 THP, 374, 376, 467, 468, 880
stochastic gradient algorithm, see LMS time advance, 756, 867
stop-and-go algorithm, 766 time division duplexing, see TDD
subchannel, 413, 442 timing, 904
active, 689, 691 recovery, 721
per s. equalization, 435 Tomlinson-Harashima precoder, see THP
successive interference cancellation, TPMI, 865
see SIC transform
sufficient statistics, 354 D-transform, 5
sum-product algorithm, 614, 619 Fourier, 1
log-domain decoder, 623 Hilbert, 23
superheterodyne, 186 precoding, 866
support, 14 z-transform, 5, 6, 36, 68
SVD, 124, 246 transient length, 346
symbol transmission mode, 691
error probability, 303 transmit
period, 260, 261 CSI, see CSIT
rate, 260 power penalty, 375
time offset, see STO precoding matrix indicator, see TPMI
930 Index

transmitter, 291 VLSI, 838, 897

travelling wave tube, 190 voltage controlled oscillator, see VCO
trellis, 659, 671 Voronoi
depth, 346 cell, 248, 667
diagram, 344, 365, 663 region, 230, 375
two-way hybrid fibre/coax, 885 VQ, 229, 232, 239
multistage, 240
UE, 859, 866, 867 product code, 240
UFMC, 446
UHF, 185 Walsh code, 511
ultra reliable low-latency communications, see water pouring, 269
URLLC waterfilling algorithm, 466, 469, 688
UMTS, 598, 599 Weber-Sonine formula, 806
unbias estimation random variable, 127 Welch periodogram, 62
Ungerboeck’s set partitioning method, 367 wide-sense stationary
unique word, 323 see WSS
universal filtered MC, see UFMC uncorrelated scattering, see WSSUS
universal mobile telecommunications systems, see Wiener
UMTS filter, see filter, Wiener
uplink, 443, 444, 472, 490, 870, 886 Hopf equation, 109, 120, 148, 153, 164,
5G, 865 318, 337
upsampling, see interpolation Khintchine theorem, 30
URLLC, 867 window
user equipment, see UE Bartlett, 59
user selection, 474 closing, 63
UTP, 779, 833, 875–877, 880 Hann, 60
rectangular, 58
V-BLAST, 457, 473 triangular, 59
Vandermonde matrix, 568 wireless local area network, see WLAN
VCO, 708 WLAN, 472, 872
VDES, 445 Wold decomposition, 70
VDSL, 833, 850, 876 WSS, 29–31, 107
vector weight, 519 WSSUS, 208, 210
vertical Bell labs layered space-time, see V-BLAST
very high-speed DSL, see VDSL xDSL, 876, 879
very large scale integration, see VLSI
VHF, 185
Yule–Walker equation, 73, 117
VHF data exchange system, see VDES
visitor location register, 869
Viterbi algorithm, 343, 346, 358, 588, 660, 893, 906 zero-forcing, see ZF
hard input, 588 ZF, 456
soft input, 588 Zipper duplexing method, 879
soft output, see SOVA
 WILEY END USER LICENSE AGREEMENT
Go to www.wiley.com/go/eula to access Wiley’s ebook EULA.