TURBO CODES

Principles and Applications

 THE KLUWER INTERNATIONAL SERIES
IN ENGINEERING AND COMPUTER SCIENCE
 TURBOCODES
Principles and Applications

Branka Vucetic
The University ofSydney
Sydney, Australia

Jinhong Yuan
The University ofNew South Wales
Sydney, Australia

"
~.

Springer Science+Business Media, LLC

Library of Congress Cataloging-in-Publication

Vucetic, Branka.
Turbo codes : principles and applications / Branka Vucetic, Jinhong Yuan.
p. cm. -- (The Kluwer international series in engineering and computer science
; SECS 559.)
lncludes bibliographical references and index.
ISBN 978-1-4613-7013-0 ISBN 978-1-4615-4469-2 (eBook)
DOI 10.1007/978-1-4615-4469-2
1. Coding theory. 2. Signal theory (Telecommunication) I. Yuan, Jinhong, 1969- II.
Title. III. Series.

TK5102.92. V83 2000

003'.54--dc21
OO-Q33104

Copyright@ 2000 by Springer Science+Business Media New York

Originally published by Kluwer Academic Publishers, New York in 2000
Softcover reprint of the hardcover 1st edition 2000
AII rights reserved. No part of this publicatlOn may be reproduced, stored in a retrievat
system or transmitted in any form or by any means, mechanical, photo-copying, recording,
or otherwise, without the prior written permission of the publisher,
Springer Science+Business Media, LLC.

Printed an acid-free paper.

Contents

List of Acronyms xi

List of Figures xiii

List of Tables xxiii

Preface xxv

1 Introduction 1
1.1 Digital Communication System Structure. 2
1.2 Fundamental Limits . . . . . . . . . . . . 5

2 Block Codes 13
2.1 Block Codes 13
2.2 Linear Systematic Block Codes 15
2.3 Parity Check Matrix . . . . . . 16
2.4 The Minimum Distance of a Block Code 17
2.5 Maximum Likelihood Decoding of Block Codes for a
BSC Channel . . . . . . . . . . . . . . . . . . . .. 18
2.6 Maximum Likelihood Decoding of Block Codes for a
Gaussian Channel. . . . . . . . . . 19
2.7 Weight Distribution of Block Codes . . . . . . 20
2.8 Performance Upper Bounds . . . . . . . . . . 23
2.8.1 Word Error Probability Upper Bounds 23
2.8.2 Bit Error Probability Upper Bounds 26
2.9 Coding Gain . . . . . . . . . . . . . . . . . . 28
2.10 Soft Decision Decoding of Block Codes . . . . 30
2.11 Trellis Structure of Linear Binary Block Codes. 30
vi CONTENTS

3 Convolutional Codes 37
3.1 Introduction . . . . .............. 37
3.2 The Structure of (n,l) Convolutional Codes 38
3.3 The Structure of (n, k) Convolutional Codes 43
3.4 Systematic Form .. 45
3.5 Parity Check Matrix 50
3.6 Catastrophic Codes . 51
3.7 Systematic Encoders 53
3.8 State Diagram. . . . 58
3.9 Trellis Diagram . . . 60
3.10 Distance Properties of Convolutional Codes 62
3.11 Weight Distribution of Convolutional Codes 63
3.12 Punctured Convolutional Codes . . . . . . . 66

4 Turbo Coding Performance Analysis and Code De-

sign 73
4.1 Introduction . . . . . . . 73
4.2 Turbo Coding . . . . . . 74
4.2.1 A Turbo Encoder 74
4.2.2 Interleaving ... 76
4.2.3 Trellis Termination 77
4.2.4 High Rate Turbo Codes 78
4.3 Performance Upper Bounds of Turbo Codes 80
4.3.1 Conditional WEF's of Average Turbo Codes 80
4.3.2 Conditional WEF's of Component Codes . . 82
4.3.3 Average Upper Bounds on Bit Error Proba-
bility . . . . . . . . . . . . . . . 84
4.3.4 Interleaving Performance Gain . 87
4.3.5 Effective Free Distance . . . . 90
4.4 Turbo Code Performance Evaluation 92
4.5 Turbo Code Design . . . . . . . . . . . 98
4.5.1 Turbo Code Design at High SNR's 100
4.5.2 Turbo Code Design at Low SNR's . 103
4.5.3 Simulation Results ......... 105
4.6 Serial Concatenated Convolutional Codes. 107
4.6.1 A Serial Concatenated Encoder .. 107
4.6.2 Performance Analysis and Code Design . 108
CONTENTS vii

5 Trellis Based Decoding of Linear Codes 117

5.1 Introduction . . . . . . 117
5.2 System Model . . . . . 118
5.3 Optimization Criteria. 120
5.4 The Viterbi Algorithm 122
5.5 The Bidirectional Soft Output Viterbi Algorithm 126
5.6 Sliding Window SOYA . . . . . 135
5.7 The MAP Algorithm. . . . . . 138
5.8 The Max-Log-MAP Algorithm. 149
5.9 The Log-MAP Algorithm. . . . 151
5.10 Comparison of Decoding Algorithms 152

6 Iterative Decoding 157

6.1 Optimum Decoding of Thrbo Codes. . . . . . . .. 157
6.2 Iterative Decoding of Turbo Codes Based on the
MAP Algorithm. . . . . . . . . . . . . . . . 159
6.3 The Effect of the Number of Iterations on Turbo
Code Performance . . . . . . . . . . . . . . . . . . 164
6.4 The Effect of Interleaver Size on Turbo Code Perfor-
mance . . . . . . . . . . . . . . . . . . . . . . . .. 166
6.5 The Effect of Puncturing Component Codes on Turbo
Code Performance . . . . . . . . . . . . . . . . . . 167
6.6 Comparison Between Analytical Upper Bounds and
Simulation Results . . . . . . . . . . . . . 169
6.7 Asymptotic Behavior of Turbo Codes . . . . . . .. 170
6.8 Iterative SOYA Decoding of Turbo Codes. . . . .. 171
6.9 Comparison of MAP and SOYA Iterative Decoding
Algorithms. . . . . . . . . . . . . . . . . . . . . .. 177
6.10 Iterative MAP Decoding of Serial Concatenated Con-
volutional Codes . . . . . . . . . . . . . . . . . . . 178
6.11 Iterative SOYA Decoding of Serial Concatenated Con-
volutional Codes . . . . . . . . . . . . . . . . . . . 180
6.12 Serial Concatenated Convolutional Codes with Iter-
ative Decoding .. . . . . . . . . . . . . . . . . .. 182
6.12.1 The Effect ofInterleaver Size and the Number
of Iterations on AWGN Channels . . . . .. 182
viii CONTENTS

6.12.2 The Effect of Memory Order on AWGN chan-

nels. . . . . . . . . . . . . . . . . . . . . .. 184
6.12.3 Comparison of MAP and SOYA Decoding Al-
gorithms on AWGN Channels . . . . . . .. 186

7 Interleavers 193
7.1 Interleaving . . . . . . . . . . . . . . . . 193
7.2 Interleaving with Error Control Coding. 195
7.3 Interleaving in Turbo Coding . . . . . . 196
7.3.1 The Effect of Interleaver Size on Code Per-
formance. . . . . . . . . . . . . . . . . . .. 197
7.3.2 The Effect of Interleaver Structure on Code
Performance . . . . . . . 198
7.3.3 Interleaving Techniques. 200
7.4 Block Type Interleavers 200
7.4.1 Block Interleavers . . . . 200
7.4.2 Odd-Even Block Interleavers . 202
7.4.3 Block Helical Simile Interleavers . 204
7.5 Convolutional Type Interleavers . 206
7.5.1 Convolutional Interleavers 206
7.5.2 Cyclic Shift Interleavers 208
7.6 Random Type Interleavers . . . 209
7.6.1 Random Interleavers . . 209
7.6.2 Non-uniform Interleavers 210
7.6.3 S-random Interleavers . 211
7.7 Code Matched Interleavers . . . 213
7.8 Design of Code Matched Interleavers 214
7.9 Performance of Turbo Codes with Code Matched In-
terleavers . . . . . . . . . . . . . . . . . . . . . . . 220
7.10 Performance of Turbo Codes with Cyclic Shift Inter-
leavers . . . . . . . . . . . . . . . . . . . . . . . .. 222

8 Turbo Coding for Fading Channels 231

8.1 Introduction.......... 231
8.2 Fading Channels . . . . . . . 232
8.2.1 Multipath Propagation 232
8.2.2 Doppler Shift . . . . . 232
8.3 Statistical Models for Fading Channels 233
CONTENTS ix

8.3.1 Rayleigh Fading. . . . 233

8.3.2 Rician Fading . . . . . 235
8.4 Capacity of Fading Channels. 236
8.5 Performance Upper Bounds on Fading Channels 240
8.5.1 Upper Bounds on the Pairwise Error Proba-
bility . . . . . . . . . . . . . . . . . . . . .. 241
8.5.2 Average Upper Bounds on the Bit Error Prob-
ability . . . . . . . . . . . . . . . . 246
8.6 Iterative Decoding on Fading Channels . . 249
8.6.1 Modified MAP Decoding with CSI 252
8.6.2 Modified SOYA Decoding with CSI 254
8.7 Performance Simulation Results on Fading Channels 256
8.7.1 Performance Comparison Between MAP
and SOYA Algorithms on Independent
Fading Channels . . . . . . . . . . . . . 256
8.7.2 Performance Comparison Between Turbo and
Serial Concatenated Codes on Independent
Fading Channels . . . . . . . . . . . . . . . 256
8.7.3 Performance Comparison Between MAP and
SOYA Algorithms on Correlated Fading Chan-
nels. . . . . . . . . . . . . . . . . . . . . .. 258
8.7.4 Performance Comparison Between Turbo and
Serial Concatenated Codes on Correlated Fad-
ing Channels . . . . . . . . . . . . . . . . . 260

9 Turbo Trellis Coded Modulation Schemes 263

9.1 Introduction...................... 263
9.2 Binary Thrbo Coded Modulation . . . . . . . . .. 264
9.2.1 Pragmatic Binary Thrbo Coded Modulation 264
9.2.2 Multilevel Thrbo Coding . . . . . . . . 267
9.3 Thrbo Trellis Coded Modulation. . . . . . . . 270
9.3.1 Schemes with Alternate Puncturing of
Parity Digits . . . . . . . . . . . . . . 270
9.3.2 Log-MAP Decoding Algorithm for Thrbo Trel-
lis Coded Modulation with Punctured Parity
Digits . . . . . . . . . . . . . . . . . . . .. 274
x CONTENTS

9.3.3 SOYA Decoding Algorithm for Turbo

Trellis Coded Modulation with
Punctured Parity Digits . . . . . . . . 278
9.3.4 Performance of Turbo Trellis Coded Modula-
tion with Punctured Parity Digits . . . . .. 279
9.3.5 Schemes with Puncturing of Systematic Bits 282
9.4 I-Q Thrbo Coded Modulation for Fading Channels 288
9.4.1 I-Q Coded Modulation Structure . . . . .. 290
9.4.2 The Decoder . . . . . . . . . . . . . . . . . 292
9.4.3 Performance of I-Q Turbo Coded Modulation
on Fading Channels. . . . . . . . . . . . .. 292

10 Applications of Turbo Codes 297

10.1 Thrbo Codes for Deep Space Communications 297
10.2 Turbo Codes for CDMA2000 . . . . . . . . 299
10.3 Turbo Codes for 3GPP . . . . . . . . . . . 300
10.4 Thrbo Codes for Satellite Communications 302

Index 307
List of Acronyms

3GPP 3rd Generation Partnership Project

APP a posteriori probability

AWGN additive white Gaussian noise

BER bit error rate

BPSK binary phase shift keying

BSC binary symmetric channel

bps bits per second

CCSOS Consultative Committee for Space Data Systems

COMA code division multiple access

CRC cyclic redundancy check

CSI channel state information

GCO greatest common divisor

IOWEF input-output weight enumerating function

IRWEF input-redundancy weight enumerating function

151 intersymbol interference

LLR log-likelihood ratio

xii LIST OF ACRONYMS

MAP maximum a posteriori

ML maximum likelihood

NRC nonrecursi ve convolutional

ODS optimal distance spectrum

PCCC parallel concatenated convolutional code

PSK phase shift keying

QAM quadrature amplitude modulation

RSC recursive systematic convolutional

SCCC serial concatenated convolutional code

SER symbol error rate

SISO soft-input soft-output

SNR signal-to-noise ratio

SOVA soft-output Viterbi algorithm

TCM trellis coded modulation

TTCM turbo trellis coded modulation

UEP unequal error protection

VA Viterbi algorithm
WEF weight enumerating function

WER word error rate

List of Figures

1.1 Model of a digital communication system. . . . .. 2

1.2 Spectral efficiency of various modulation and coding
schemes computed for the bit error rate of 10-5 8

2.1 Coded system model . . . . . . . . . . . 24

2.2 Performance upper bounds for the (7,4) Hamming
code . . . . . . . . . . . . . . . . . . . . 28
2.3 Trellis for the binary (5,3) code . . . . . 33
2.4 Expurgated trellis for the binary (5,3) code. 33

3.1 A rate 1/2 convolutional encoder . . . . . . 38

3.2 A general (n, 1, v) convolutional code feedforward en-
coder. . . . . . . . . . . . . . . . . . . . . . . . .. 41
3.3 Encoder for a (3,2,1) code. . . . . . . . . . . . .. 46
3.4 The controller canonical form of a rational transfer
function a(D)/q(D) . . . . . . . . . . . . . . . . . 48
3.5 The observer canonical form of a rational transfer
function a(D)/q(D) . . . . . . . . . . . . . . . . . 49
3.6 The controller canonical form ofthe systematic (2,1)
encoder with the generator matrix G1(D) . . . . . 49
3.7 The observer canonical form of the systematic (2,1)
encoder with the generator matrix G 1 (D) 50
3.8 Nonsystematic encoder in Example 3.9 . . . . . . , 55
3.9 Systematic encoder in Example 3.9 . . . . . . . .. 56
3.10 A systematic encoder with the generator matrix in
(3.73) . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.11 Observer canonical form of an (n, n - 1) systematic
encoder . . . . . . . . . . . . . . . . . . . . . . . . 58
xiv LIST OF FIGURES

3.12 State diagram for the (2,1) nonsystematic convolu-

tional encoder from Fig. 3.1 . . . . . . . . . . . .. 59
3.13 State diagram for the (2,1) systematic encoder in
Fig. 3.7 .. . . . . . . . . . . . . . . . . . . . . .. 60
3.14 Trellis diagram for the (2,1) nonsystematic encoder
in Fig. 3.1 . . . . . . . . . . . . . . . . . . . . . .. 61
3.15 Augmented state diagram of Fig. 3.12 . . . . . . . 64
3.16 Trellis diagram of a rate 2/3 punctured code pro-
duced by periodically deleting symbols from a rate
1/2 code . . . . . . . . . . . . . . 67
3.17 Encoder for a rate 2/3 code . . . 68
3.18 Trellis diagram of a rate 2/3 code 68

4.1 A turbo encoder. . . . . . 75

4.2 A rate 1/3 turbo encoder. 77
4.3 Trellis termination . . . . 78
4.4 A rate 1/2 turbo encoder. 80
4.5 A compound error path. . 83
4.6 Bit error probability upper bounds for a turbo code
with interleaver size 500 .. . . . . . . . . . . . .. 86
4.7 Bit error probability upper bounds for a turbo code
with various interleaver sizes 91
4.8 Turbo encoder TC1 . . . . . . . . . . . . . . . . .. 94
4.9 Turbo encoder TC2 . . . . . . . . . . . . . . . . .. 94
4.10 Distance spectra for component code of TC1 and
turbo code TC1 with interleaver sizes of 20 and 50 . 95
4.11 Bit error probability upper bounds for component
code of TC1 and turbo code TC1 with interleaver
sizes of 20 and 50 . . . . . . . . . . . . . . . . . .. 96
4.12 Relative contributions of various distance spectral
lines to overall bit error probability for turbo code
TC1 with interleaver size 20 . . . . . . . . . . . .. 97
4.13 Relative contributions of various distance spectral
lines to overall bit error probability for turbo code
TC1 with interleaver size 50 . . . . . . . . . . . .. 98
4.14 Distance spectra for turbo codes TC1 and TC2 with
interleaver sizes of 20 and 50 . . . . . . . . . . . .. 99
LIST OF FIGURES xv

4.15 Bit error probability upper bounds for turbo codes

TC1 and TC2 with interleaver sizes of 20 and 50.. 100
4.16 Distance spectra for ODS turbo codes with inter-
leaver size 40 . . . . . . . . . . . . . . . . . . . .. 105
4.17 Bit error probability upper bounds for ODS turbo
codes with interleaver size 40 . . . . . . . . . . . . 106
4.18 Performance of ODS and BM turbo codes with rate
1/3 and memory order 4 on AWGN channels. 107
4.19 A serial concatenated encoder 108

5.1 System model . . . . . . . . . 118

5.2 A convolutional encoder and its graphical represen-
tation . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.3 The branch metrics in Example 5.1 . . . . . . . .. 128
5.4 The survivors and their path metrics in Example 5.1 128
5.5 The branch metrics in Example 5.2 . . . . . . . .. 133
5.6 The forward recursion in Example 5.2, the ML path
is shown by the thick line . . . . . . . . . . . . . . 133
5.7 The backward recursion in Example 5.2, the ML
path is shown by the thick line. . . . . . . . . . .. 133
5.8 Forward and Backward processing for the simplified
SOYA . . . . . . . . . . . . . . . . . . . . . . .. 137
5.9 A rate 1/2 memory order 2 RSC encoder. . . .. 139
5.10 State transition diagram for the (2,1,2) RSC code 140
5.11 Trellis diagram for the (2,1,2) RSC code . . . .. 141
5.12 Graphical representation of the forward recursion 146
5.13 Graphical representation of the backward recursion 147
5.14 Trellis diagram for the encoder in Example 5.3 150
5.15 Performance comparison of MAP and SOYA 154

6.1 Basic turbo encoder. . . . . . . . . . . . . . 158

6.2 An iterative turbo code decoder based on the MAP
algorithm . . . . . . . . . . . . . . . . . . . . . .. 160
6.3 BER performance of a 16 state, rate 1/3 turbo code
with MAP algorithm on an AWGN channel, inter-
leaver size 4096 bits, variable number of iterations. 164
xvi LIST OF FIGURES

6.4 BER performance of a 16 state, rate 1/3 turbo code

with MAP algorithm on an AWGN channel, inter-
leaver size 16384 bits, variable number of iterations 165
6.5 BER performance of a 16 state, rate 1/3 turbo code
with MAP algorithm on an AWGN channel, inter-
leaver size N, the number of iterations 18. . . . .. 167
6.6 BER performance of a 16 state, rate 1/2 turbo code
with MAP algorithm on an AWGN channel, inter-
leaver size N, the number of iterations 18. . . . .. 168
6.7 BER performance of a 16 state, rate 2/3 turbo code
with MAP algorithm on an AWGN channel, inter-
leaver size N, the number of iterations 18 . . . . . 169
6.8 Simulation result of a 16 state, rate 1/3 turbo code
with MAP, interleaver size 1024 bits, variable num-
ber of iterations I and the theoretical bound on an
AWGN channel. . . . . . . . . . . . . . . . . . . .. 170
6.9 Simulation result of a 16 state, rate 1/3 turbo code
with MAP, interleaver size 1024 bits, the number of
iterations 10 and the theoretical bound on an AWGN
channel. . . . . . . . . . . . . . . . . . . . . . . .. 172
6.10 An iterative turbo code decoder based on the SOYA
algorithm . . . . . . . . . . . . . . . . . . . . . .. 174
6.11 BER performance of a 16 state, rate 1/3 turbo code
with MAP, Log-MAP and SOYA algorithm on an
AWGN channel, interleaver size 4096 bits, the num-
ber of iterations 18 . . . . . . . . . . . . . . . . .. 178
6.12 Iterative MAP decoder for serial concatenated codes 179
6.13 Iterative SOYA decoder for serial concatenated codes 181
6.14 Performance of a rate 1/3 serial concatenated code,
with a rate 1/2, 4 state nonrecursive convolutional
code as the outer code, a rate 2/3, 4 state recursive
convolutional code as the inner code, AWGN chan-
nel, SOYA decoding algorithm, various interleaver
size N, and the number of iterations 20 . . . . . . 184
LIST OF FIGURES xvii

6.15 Comparison of a rate 1/3, memory order 2 turbo

code with interleaver size 4096 bits and a rate 1/3
serial concatenated code with memory order 2 outer
code, interleaver size 4096 bits on an AWGN channel,
SOYA decoding algorithm, the number of iterations
18. .......................... 185
6.16 BER performance of a rate 1/3 serial concatenated
code with rate 1/2, 4 state outer code and rate 2/3, 4
state inner code with SOYA algorithm on an AWGN
channel, interleaver size 4096 bits, variable number
of iterations . . . . . . . . . . . . . . . . . . . . .. 186
6.17 Comparison of a rate 1/3 turbo code for different
memory order with SOYA algorithm on an AWGN
channel, interleaver size 1024 bits, the number of it-
erations 12 . . . . . . . . . . . . . . . . . . . . . .. 187
6.18 Comparison of a rate 1/3 turbo code for different
memory order with SOYA algorithm on an AWGN
channel, interleaver size 4096 bits, the number of it-
erations 18 . . . . . . . . . . . . . . . . . . . . . .. 188
6.19 Comparison of a rate 1/3 serial concatenated code
for different outer code memory order with SOYA al-
gorithm on an AWGN channel, interleaver size 1024
bits, the number of iterations 12. . . . . . . . . . . 188
6.20 Comparison of a rate 1/3 serial concatenated code
for different outer code memory order with SOYA al-
gorithm on an AWGN channel, interleaver size 4096
bits, the number of iterations 18 . . . . . . . . . .. 189
6.21 Performance comparison of MAP and SOYA for a
rate 1/3 serial concatenated convolutional code 189

7.1 An interleaver device . . . . . . . . . . . . . . . 194

7.2 An interleaver mapping. . . . . . . . . . . . . . 195
7.3 Distance spectra for a turbo code with various inter-
leaver sizes. . . . . . . . . . . . . . . . . . . . . .. 198
7.4 Bit error probability upper bounds for a turbo code
with various interleaver sizes 199
7.5 A block interleaver . . . . . . . . . . . . . . . . .. 201
xviii LIST OF FIGURES

7.6 A weight-4 square input pattern of block interleavers 202

7.7 A convolutional interleaver and deinterleaver . .. 207
7.8 A convolutional interleaver with L = 3 and B = 2. 208
7.9 A cyclic shift interleaver . . . . . . . . . . . . . .. 209
7.10 A general m-stage shift register with linear feedback 210
7.11 A weight-2 input sequence pattern. . . . . . . . .. 217
7.12 A weight-4 input sequence pattern. . . . . . . . .. 218
7.13 BER performance of the 4-state, rate 1/3, (1, 5/7)
turbo code with random, S-random and code matched
interleavers on an AWGN channel . . . . . . . . .. 221
7.14 BER performance of the 8-state, rate 1/3, (1, 17/15)
turbo code with random, S-random and code matched
interleavers on an AWGN channel. . . . . . . . .. 222
7.15 BER performance of the 16-state, rate 1/3, (1,33/31)
turbo code with random, S-random and code matched
interleavers on an AWGN channel. . . . . . . . .. 223
7.16 BER performance ofthe 16-state, rate 1/3, (1,33/31)
turbo code with S-random and cyclic shift inter-
leavers on an AWG N channel .. 224

8.1 The pdf of Rayleigh distribution. 234

8.2 The pdf of Rician distributions with various K . 237
8.3 Capacity of independent Rayleigh fading channels
with coherent BPSK signalling. . . . . . . . . . .. 239
8.4 Coded system block diagram . . . . . . . . . . . .. 241
8.5 Bit error probability upper bound for the 4 state,
rate 1/3 turbo code with interleaver size 100 on in-
dependent Rician fading channels with ideal channel
state information. The curves are for Rician chan-
nels with K =0, 2, 5, 50, starting from the top, with
the bottom one referring to an AWGN channel. .. 249
8.6 Bit error probability upper bound for the 4 state,
rate 1/3 turbo code with interleaver size 100 on in-
dependent Rician fading channels without channel
state information. The curves are for Rician chan-
nels with K =0, 2, 5, 50, starting from the top, with
the bottom one referring to an AWGN channel. .. 250
LIST OF FIGURES xix

8.7 Bit error probability upper bound for the 4 state,

rate 1/3 serial code with information size 100 on in-
dependent Rician fading channels with ideal channel
state information. The curves are for Rician chan-
nels with K =0, 2, 5, 50, starting from the top, with
the bottom one referring to an AWGN channel. .. 251
8.8 Bit error probability upper bound for the 4 state,
rate 1/3 serial code with information size 100 on in-
dependent Rician fading channels without channel
state information. The curves are for Rician chan-
nels with K =0, 2, 5, 50, starting from the top, with
the bottom one referring to an AWGN channel. .. 252
8.9 Distance spectrum comparison of the 4 state, rate
1/3 turbo and serial concatenated codes with infor-
mation size 100 . . . . . . . . . . . . . . . . . .. 253
8.10 Bit error probability upper bound comparison of the
4 state, rate 1/3 turbo and serial concatenated codes
with information size 100 on independent Rayleigh
fading channels . . . . . . . . . . . . . . . . . . " 254
8.11 Performance comparison of MAP and SOYA, with
and without CSI, for the 16 state, rate 1/3 turbo
code on an independent Rayleigh fading channel, in-
formation size 1024, the number of iterations 8 . .. 257
8.12 Performance comparison for the 4 state, rate 1/3
turbo and serial codes on an independent Rayleigh
fading channel . . . . . . . . . . . . . . . . . . .. 258
8.13 Performance comparison of MAP and SOYA, with
and without CSI, for the 16 state, rate 1/3 turbo
code on a correlated Rayleigh fading channel, the
fading rate normalized by the symbol rate is 10- 2 ,
information size 1024, the number of iterations 8.. 259
8.14 Performance comparison for the turbo and serial codes
on a correlated Rayleigh fading channel, the fading
rate normalized by the symbol rate is 10- 2 , informa-
tion size N, the number of iterations I 260

9.1 Pragmatic turbo TCM encoder .... 265

xx LIST OF FIGURES

9.2 Pragmatic turbo TCM decoder 265

9.3 16-QAM with Gray mapping. 266
9.4 Multilevel turbo encoder . . . . 268
9.5 Multilevel turbo decoder . . .. 269
9.6 Turbo TCM encoder with parity symbol puncturing 271
9.7 Example of a turbo trellis coded 8-PSK with parity
symbol puncturing . . . . . . . . . . . . . . . . .. 273
9.8 Turbo TCM decoder with parity symbol puncturing 275
9.9 Performance of the rate 2/3, 4-state turbo trellis
coded 8-PSK with various interleaver sizes on an
AWGN channel, SOYA decoding algorithm, the num-
ber of iterations I, bandwidth efficiency 2 bits/s/Hz 280
9.10 Performance of the rate 2/3, 8-state turbo trellis
coded 8-PSK with various interleaver sizes on an
AWGN channel, SOYA decoding algorithm, the num-
ber of iterations I, bandwidth efficiency 2 bits/s/Hz 281
9.11 Performance of the rate 2/3, 16-state turbo trellis
coded 8-PSK with various interleaver sizes on an
AWGN channel, SOYA decoding algorithm, the num-
ber of iterations I, bandwidth efficiency 2 bits/s/Hz 282
9.12 Performance comparison of the Log-MAP and SOYA
for the rate 2/3, 8-state turbo trellis coded 8-PSK
with interleaver size 1024 on an AWGN channel,
bandwidth efficiency 2 bits/s/Hz . . . . . . . . . . 283
9.13 Performance comparison of the Log-MAP and SOYA
for the rate 3/4, 4-state turbo trellis coded 16-QAM
with various interleaver sizes on an AWGN channel,
bandwidth efficiency 3 bits/s/Hz . . . . . . . . . . 284
9.14 Performance comparison of the Log-MAP and SOYA
for the rate 3/4, 8-state turbo trellis coded 16-QAM
with various interleaver sizes on an AWGN channel,
bandwidth efficiency 3 bits/s/Hz . . . . . . . . . . 285
9.15 Performance comparison of the Log-MAP and SOYA
for the rate 3/4, 16-state turbo trellis coded 16-QAM
with various interleaver sizes on an AWGN channel,
bandwidth efficiency 3 bits/s/Hz . . . . . . . . . . 286
LIST OF FIGURES xxi

9.16 Turbo trellis coded 16-QAM with systematic symbol

puncturing. . . . . . . . . . . . . . . . . . . . . .. 287
9.17 Performance comparison of the turbo trellis coded
16-QAM with systematic symbol puncturing and the
pragmatic turbo coded 16-QAM with bandwidth ef-
ficiency 2 bits/s/Hz and interleaver size 32768 on an
AWGN channel, the number of iterations 8 . . . .. 288
9.18 Turbo trellis coded 8-PSK with systematic symbol
puncturing. . . . . . . . . . . . . . . . . . . . . .. 289
9.19 Performance of the turbo trellis coded 8-PSK with
systematic symbol puncturing with bandwidth effi-
ciency 2 bits/s/Hz and interleaver size 16384 on an
AWGN channel, the number of iterations I . . . .. 290
9.20 I-Q turbo trellis coded 16-QAM . . . . . . . . . .. 291
9.21 Performance of the I-Q turbo coded 16-QAM with
bandwidth efficiency 2 bits/s/Hz and various inter-
Ie aver sizes on a Rayleigh fading channel . . . . .. 293
9.22 Performance comparison of the I-Q turbo coded 16-
QAM and the pragmatic turbo coded 16-QAM with
bandwidth efficiency 2 bits/s/Hz and interleaver size
4096 on a Rayleigh fading channel. . . . . . . . 294

10.1 CCSDS turbo encoder block diagram . . . . . . 298

10.2 The reverse link turbo encoder for CDMA2000 . 300
10.3 The turbo encoder for 3GPP . . . . . . . . . . . 301
10.4 The serial concatenated convolutional encoder for
3GPP . . . . . . . . . . . . . . . . . . . . . . . . . 303
List of Tables

2.1 A (6, 3) linear block code ... 15

3.1 Punctured convolutional codes. 70

4.1 Best rate 1/3 turbo codes at high SNR's [14] 103
4.2 Rate 1/3 ODS turbo codes at Low SNR's . 104

5.1 Decoder complexity comparison . . . . . . 153

8.1 Channel capacity limits for coherent BPSK . 240

9.1 Rate 2/3 turbo trellis coded 8-PSK schemes 279

9.2 Rate 3/4 turbo trellis coded 16-QAM schemes 281
Preface

This book grew out of our research, industry consulting and con-
tinuing education courses.
Turbo coding initially seemed to belong to a restricted research
area, while now has become a part of the mainstream telecommu-
nication theory and practice. The turbo decoding principles have
found widespread applications not only in error control, but in de-
tection, interference suppression and equalization.
Intended for use by advanced students and professional engi-
neers, involved in coding and telecommunication research, the book
includes both basic and advanced material. The chapters are se-
quenced so that the knowledge is acquired in a logical and progres-
sive way. The algorithm descriptions and analysis are supported
by examples throughout the book. Performance evaluations of the
presented algorithms are carried out both analytically and by sim-
ulations.
Basic material included in the book has been taught to students
and practicing professionals over the last four years in the form of
senior undergraduate or graduate courses, lecture series and short
continuing education courses.
Most of the presented material is a compilation of the various
publications from the well established literature. There are, how-
ever, original contributions, related to decoding algorithms, inter-
Ie aver design, turbo coded modulation design for fading channels
and performance of turbo codes on fading channels. The bidirec-
tional SOYA decoding algorithm, presented in the book, had been
developed for soft output detection and originally applied to cellu-
lar mobile receivers, but was subsequently modified for decoding of
turbo codes. We have published various versions of the algorithm
xx.vi PREFACE

at a number of conferences, but never as a journal paper, so it has

not been widely known. Its distinguishing features are excellent
performance, which is only slightly worse than the optimum MAP
algorithm, and very low complexity, making it attractive for prac-
tical implementation. It should be of particular value to telecom-
munication system designers.
A great deal of effort has been put into ensuring consistency
and continuity between chapters.

Special Thanks
We would like to thank everyone who has been involved in the
process of writing, proof reading and publishing this book. In par-
ticular we would like to thank Dr Lei Wei, Dr Steven Pietrobon,
Dr Adrian Barbulescu, Dr Miroslar Despotovic, Prof Shu Lin, and
Prof Dusan Drajic for reading the manuscript and providing valu-
able feedback. We would also like to thank Dr Akihisa Ushirokawa
for constructive discussions and Enrico Vassallo for providing the
details on the CCSDS standard.
We are pleased to acknowledge the students' contribution to
advancing the understanding of turbo coding. In particular, we
thank Wen Feng for her work reported in Chapters 6 and 7, Jade
Kim for her work reported in Chapter 6, Mark Tan for his work
reported in Chapter 7 and Lei Wan for her comments on Chapters
5 and 6.
We express our appreciation to Wen Feng for providing simula-
tion results as well as to Maree Belleli and Zhuo Chen for typing
the manuscript and preparing illustrations for the book.
We owe special thanks to the Australian Research Council, NEC,
DSTO, Motorola and other companies, whose support enables grad-
uate students and the staff of Sydney University to pursue contin-
uing research in this important field.
Alex Greene, senior editor, of Kluwer, helped and motivated us
during all phases of the preparation of the book.
Finally, we would like to thank our families for providing the
most meaningful content in our lives.
Chapter 1

Introduction

Novel communication and information services are being introduced

almost daily and the demands for higher data rates and commu-
nication capacity continue to grow. This spectacular progress of
communication is to a great extent due to consistent performance
increase and cost reduction of devices and circuit technology. Such
advancements have also been fostered by major theoretical devel-
opments. The synergy between components and signal processing
techniques is considered to be the main cornerstone of modern com-
munication equipment.
This book is based on two theoretical discoveries that have had
a considerable impact on communication systems.
In the first one, Shannon established the fundamental limits on
the transmission rates in digital communication systems and mo-
tivated the search for coding techniques to approach the capacity
limit [3]. In another landmark development, Berrou, Glavieux and
Thitimajshima proposed turbo error control coding by which the
gap between the capacity limit and practically feasible channel uti-
lization is almost closed [4].
The emphasis in the book is on various aspects of encoding and
decoding algorithms, interleaver and code design and performance
analysis.
We start by outlining the structure of digital communication
systems and proceed by reviewing the channel capacity theorem
and its application to various error control coding schemes. Fur-

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
2 Introduction

thermore, basic principles of mathematical and trellis representa-

tion, encoding/decoding and performance analysis of block codes
are presented.

1.1 Digital Communication System

Structure
In order to understand the role of error control coding we present
a model of a general communication system, as shown in Fig. 1.1.

Inform-
ation r-- Source
r-- Encoder
Channel ---+- Modu-
Source Encoder lator
!
Channel

Data
Sink
+--
Source Channel Demodu-
Decoder f+- Decoder f+- lator
I
Fig. 1.1: Model of a digital communication system

The task of the transmitter in such a system is to transform the

information generated by a source into a form that can withstand
the effects of noise over the transmission medium.
An information source generates messages bearing information
to be transmitted. The messages can be words, code symbols etc.
The output of the information source is converted to a sequence of
symbols from a certain alphabet. Most often binary symbols are
transmitted.
The output of the information source is in general not suitable
for transmission as it might contain too much redundancy. For effi-
ciency reasons, the source encoder is designed to convert the source
output sequence into a sequence of binary digits with minimum re-
dundancy. If the source encoder generates rb bits per second (bps),
rb is called the data rate.
Digital Communication System Structure 3

The channel impairments cause errors in the received signal.

The channel encoder is incorporated in the system to add redun-
dancy to the information sequence. This redundancy is used to
minimize transmission errors. The channel encoder assigns to each
message of k digits a longer message of n digits called a codeword.
A good error control code generates codewords which are as differ-
ent as possible from one another. This makes the communication
system less vulnerable to channel errors. Each code is characterized
by the ratio R = kin < 1 called the code rate. The data rate at
the output of the channel encoder is rc = rbl R bps. The primary
goal of error control coding is to maximize the reliability of trans-
mission within the constraints of signal power, system bandwidth
and complexity of the circuitry. It is achieved by introducing struc-
tured redundancy into transmitted signals. This usually results in
a lowered data transmission rate or increased channel bandwidth,
relative to an uncoded system.
The output signal of the channel encoder is not normally suit-
able for transmission. The modulator enables signal transmission
over a channel. The main goals of the modulation operation are to
match the signal to the channel, enable simultaneous transmission
of a number of signals over the same physical channel and increase
the speed of information transmission.
The modulator, maps the encoded digital sequences into a train
of short analog waveforms suitable for propagation. An M-ary mod-
ulator maps a block of l binary digits from the channel encoder into
one of the M possible waveforms, where M = 21. The duration of
the modulator output waveform is T sec, and it is referred as the
signalling interval, while rs = liT is called the symbol rate. The
minimum signal bandwidth is equal to r s Hz, where r s can be ex-
pressed as
rb
rs = Rl (1.1)
Modulation can be performed by varying the amplitude, the phase
or the frequency of a sinusoidal waveform called the carrier.
Channels are transmission media used to carry or store informa-
tion. Channel examples are wire lines, microwave radio links over
free space, satellite links, fiber optic channels, magnetic recording
media etc. Very often the term channel is referred to the frequency
4 Introduction

range allocated to a particular service such as television or a tele-

phone channel. Two major limitations of real channels are thermal
noise and finite bandwidth. In addition, mobile radio channels suf-
fer from multipath propagation, fiber-optic cables from signal dis-
persion, magnetic recording media are exposed to dust and physical
damage.
In the receiver, the demodulator typically generates a binary or
analog sequence at its output as the best estimates of the transmit-
ted codeword or the modulated sequence respectively. The channel
decoder makes estimates of the actually transmitted message. The
decoder process is based on the encoding rule and the character-
istics of the channel. The goal of the decoder is to minimize the
effect of channel noise.
Based on the source encoding rule the source decoder transforms
the sequence at its input into an estimate of the source output
sequence and delivers it to the user.
By proper design of the transmitter-receiver system it would be
possible to remove or reduce the effects of attenuation and distor-
tion and to minimize the noise effects. The impact of noise cannot
be totally removed since we do not have the complete knowledge of
the noise.
If the demodulator makes hard decisions, its output is a binary
sequence. The subsequent channel decoding process is called hard
decision decoding. The three blocks consisting of the modulator,
the waveform channel and the demodulator can be represented by a
discrete channel. The input and the output of the discrete channel
are binary sequences at r c bps. If the demodulator output in a
given symbol interval depends only on the signal transmitted in
that interval and not on any previous transmission, it is said that
this channel is memoryless. Such a channel is described as by the set
of transition probabilities p( i I j) where i denotes a binary input
symbol and j is a binary output symbol. The simplest channel
model is obtained when the probability of error for binary symbols
o and 1 are the same and the channel is memoryless. This channel
model is known as the binary symmetric channel (BSC). The BSC
channel is completely described by the transition probability p.
Hard decisions in the demodulator result in some inreversible
Fundamental Limits 5

information loss. An alternative is to quantize the demodulator

output to more than two levels or take samples of the analog re-
ceived baseband signal and pass it to the channel decoder. The
subsequent decoding process is called soft decision decoding.
It is important to note that in both above systems coding is
performed separately from modulation/demodulation and a perfor-
mance gain is obtained at the expense of increased signal band-
width. If bandwidth efficiency is essential a more effective signal
design is obtained by combining coding and modulation into a sin-
gle entity. In this approach binary messages are encoded into signal
sequences over a certain modulation signal set. This results in in-
creased noise immunity of the signal without increasing the channel
bandwidth. The combined coding and modulation is called coded
modulation.
The two most frequently used types of codes are block and con-
volutional codes. The main difference between the two of them
is memory of the encoder. In block codes each encoding operation
depends on the current input message and is independent on pre-
vious encodings. That is, the encoder has no memory of history of
past encodings. In contrast, for a convolutional code, each encoder
output sequence depends not only on the current input message,
but also on a number of past message blocks.

1.2 Fundamental Limits

The signal bandwidth is a measure of its speed. The signals that
change quickly in time have large bandwidth. On the other hand,
every communication system has a limited bandwidth due to ca-
pacitances and inductances which prevent instantaneous change of
signals. The system bandwidth B limits the speed of signal varia-
tions. Bandwidth limitation of a system is quantified by the spectral
efficiency, denoted by 'Tj, defined as

rb
'Tj = - bits/sec/Hz (1.2)
B
6 Introduction

It can be expressed as
rslR
ry=- (1.3)
B
where rs is the symbol rate. As the minimum required bandwidth
for a modulated signal is r s Hz, the maximum spectral efficiency,
denoted by rymax, is given by
rymax = lR (1.4)
Another important parameter used to measure the reliability of
information transmission in digital communication systems is the
bit error probability. Power efficiency is captured by the required
bit energy to one sided noise power spectral density ratio, Ebl No, to
achieve a specified bit error probability. The signal-to-noise ratio
(SNR), denoted by SIN, is related to Eb/No as

(1.5)

For a given channel, there is an upper limit on the data rate related
to the signal-to-noise ratio and the system bandwidth. Shannon
introduced the concept of channel capacity, C, as the maximum
rate at which information can be transmitted over a noisy channel.
This rate is referred to as the channel capacity and for an additive
white Gaussian noise (AWGN) channel it is given by the Shannon-
Hartley formula

C= Blog2 (1 + ~) bits/sec (1.6)

Shannon's channel coding theorem guarantees the existence of codes

that can achieve arbitrary small probability of error if the data
transmission rate rb is smaller than the channel capacity. Con-
versely, for a data rate rb > C it is not possible to design a code
that can achieve an arbitrarily small error probability. This fun-
damental result shows that noise sets a limit on the data rate but
not on the error probability as widely believed before. Though the
theorem does not indicate how to design specific codes achieving
maximum possible data rate at arbitrarily small error probabili-
ties, it motivated the development of a number of error control
techniques.
Fundamental Limits 7

Assuming that the data rate takes its maximum possible value
for error-free transmission, equal to the channel capacity C, the
maximum spectral efficiency, TJrnax = ~, can be expressed as

(1.7)

or substituting for TJrnax from (1.4) we get

TJrnax = log2 (1 + TJrnax ~~) (1.8)

The minimum required ~ for error-free transmission is then

given by
(1.9)
TJrnax
If the bandwidth is not limited, it could be expanded to in-
crease the channel capacity. In the limiting case when B ---t 00,
or TJrnax ---t 0, we get for the minimum ~

lim
1Jmax ---+ 0
~b = In 2 = -1.59
1" 0
dB (1.10)

The minimum required ~ for error-free transmission is -1.59 dB.

Fig. 1.2 shows the bandwidth versus power efficiency of modu-
lation and coding for a number schemes for the bit error probability
of 10- 5 on an AWG N channel. They are shown against the Shannon
spectral efficiency limit expressed by Eq. (1.9).
The uncoded BPSK modulation achieves bit error probability of
10- 5 at ~ of 9.5 dB and spectral efficiency of 1, while the Shannon
limit for this case is ~ = 10 IOglO 1 = 0 dB. Thus, this scheme is
about 9.5 dB worse than the Shannon limit. In other words, the
required signal power could be reduced by about 9.5 dB.
The code used in satellite communications as well as in the
Voyager mission is the (2,1,6) Odenwalder convolutional code [14].
It achieves the bit error probability of 10- 5 at 4.5 dB with BPSK
modulation and soft decision Viterbi decoder giving the spectral
efficiency of 0.5 bits/sec/Hz.
8 Introduction

N 4
~
U!
'"
.0

"3

256state8PSKTCM
2 x QPSK
64state8PSKTCM

x BPSK

Tuwo (2, 1) Voe ger "" (2,1,6) CC

"Galh~(4,1,14) CC
O~----~-===~----~----~----~----~----~
-2 0 2 4 6 8 10 12
Eb/NO (dB)

Fig. 1.2: Spectral efficiency of various modulation and coding schemes

computed for the bit error rate of 10- 5

To improve the performance, the (2,1,6) convolutional code was

concatenated with a (255,223) Reed-Solomon code in the space mis-
sion of Voyager to Uranus in 1986. The inner convolutional code
was decoded by a soft decision Viterbi algorithm and the Reed-
Solomon code was decoded by a Massey-Berlekamp hard decision
decoder. The concatenated code can achieve a bit error rate of 10- 5
with an E b / No ratio of 2.6 dB, with the spectral efficiency of 0.4370.

A more powerful concatenated coding scheme was chosen for the

Gallieo spacecraft launched in 1989 [13]. The inner (4,1,14) con-
volutional code [15] was serially concatenated with the (255,233)
Reed-Solomon code. The inner code has the bit error rate of 10- 5 at
Eb/No ratio of 1.75 dB with the spectral efficiency of 0.25 bits/sec/Hz
while the concatenated coding scheme for the same error rate needs
at least an ~ of 0.8 dB with the spectral efficiency of 0.219.
Trellis coded modulation (TCM) [6] has a desirable property
that coding gains are achieved without requiring more bandwidth
Fundamental Limits 9

than the reference uncoded system with the same spectral efficiency.
The asymptotic coding gains for two-dimensional TCM schemes
vary form 3 to 6 dB.
For example, a 64-state 8-PSK TCM has a bit error rate of
10- 5 at an ~ ratio of 6.05 dB with the spectral efficiency of 2
bits/sec/Hz, gaining 3.5 dB relative to the reference uncoded QPSK,
while a 256-state 8PSK TCM gains 4 dB.
Turbo codes with iterative decoding [4] have almost closed the
gap between the capacity limit and real code performance. They
get a bit error rate of 10- 5 at an ~ of 0.7 dB with the spectral
efficiency of 0.5 bits/sec/Hz.
Bibliography

[1] J. M. Wozencraft, and I. M. Jacobs, "Principles of Communi-

cation Engineering," John Wiley, 1965.

[2] R. G. Gallager, "Information Theory and Reliable Communi-

cations," John Wiley, 1968.

[3] C. E. Shannon, "A Mathematical Theory of Communication" ,

Bell System Technical Journal, Vol. 27, pp. 379-423 (Part
One), pp. 623-656 (Part Two), Oct. 1948, reprinted in book
form, University of Illinois Press, Urbana, 1949.

[4] C. Berrou, A. Glavieux and P. Thitimajshima, "Near Shannon

Limit Error-Correcting Coding and Decoding: Turbo Codes" ,
in Proc. 1993 Inter. Conf. Commun., 1993, pp. 1064-1070.

[5] J. G. Proakis, "Digital Communications", 2nd Ed. McGraw-

Hill, New York, 1989.

[6] G. Ungerboeck, "Channel Coding with Multilevel Phase Sig-

nals", IEEE Trans. Inform. Theory, vol. 17-28, Jan. 1982, pp.
55-67.

[7] S. Lin, and D. J. Costello, Jr., "Error Control Coding: Funda-

mentals and Applications", Prentice-Hall, 1983.

[8] E. R. Berlekamp, "Algebraic Coding Theor'!J', McGraw-Hill,

1968.

[9] R. Blahut, "Theory and Practice of Error Control Codes",

Adison-Wesley Publishing Company, 1983.
12 BIBLIOGRAPHY

[10] A. J. Viterbi, and J. K. Omura, "Principles of Digital Com-

munication and Coding', McGraw-Hill, New York, 1979.

[11] E. Biglieri, D. Divsalar, P. McLane and M. Simon, "Introduc-

tion to Trellis-Coded Modulation", Macmilan Publishing Com-
pany, 1991.

[12] C. Heegard and S. Wicker, "Turbo Coding', Kluwer Academic

Publishers, 1999.

[13] L. Swanson, "A New Code for Galileo" , Abstracts 1988, Inter-
national Symposium on Information Theory, p. 94.
[14] J. P. Odenwalder, "Optimal Decoding of Convolutional
Codes", PhD Thesis, Systems Science Department, University
of California, Los Angeles, 1970.

[15] O. M. Collins, "The Subtleties and Intricacies of Building a

Constraint Length 15 Convolutional Encoder", IEEE Trans.
Commun., vol. COM-40, 1992, pp. 1810-1819.
Chapter 2
Block Codes

2.1 Block Codes

The encoder of an (n, k) block code accepts a message of k symbols
and transforms it into a longer sequence of n symbols called a code-
word. The important feature of a block code is that a codeword
depends only on the current input message and not on the past
messages. That is, the encoder is a memoryless device. In general,
both messages and codewords can consist of nonbinary symbols.
However, block codes with binary symbols are most often used due
to implementation complexity constraints.
In an (n, k) block code there are 2k distinct messages. Since
there is one-to-one correspondence between a message and a code-
word, there are 2k distinct codewords. The code rate R = kin
determines the amount of redundancy.
An (n, k) block code is linear if

(1) component-wise modulo-2 sum of two codewords is another

codeword and

(2) the code contains the all-zero codeword.

In the language of linear algebra an (n, k) linear block code is a

k-dimensional subspace of the vector space Vn of all the binary n-

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
14 Block Codes

tuples. An (n, k) linear block code can be generated by a set of

k linearly independent binary n-tuples go, gl, ... , gk-l. The code-
words are obtained as linear combinations of these k n-tuples.
Consequently, the codeword for a message c = (co, CI, ... , ck-d
can be represented as

I[
The k vectors generating the code go, gll ... , gk-l can be arranged

I
as rows of a k x n matrix as follows

~~::=~
gOI

G =
go
gl =
goo
glO gl1
[

~k-l ~k-I'O gk-l,l ~k-l,n-l

The array G is called the generator matrix of the code. Then, the
codeword v for message c can be written as

v = c . G = Co . go + Cl . gl + ... + Ck-l . gk-l

Example 2.1 An (6,3) linear block code can be generated by

the generator matrix

1 1 1 0
o 1 0 1
1 000 ~1
The message c = (0 1 1) is encoded as follows:

v e·G
o· (011100) + 1 . (101010) + 1 . (110001)
(000000) + (101010) + (110001)
(011011)

Table 2.1 presents the list of messages and codewords for the (6, 3)
linear block code.
Linear Systematic Block Codes 15

Table 2.1: A (6, 3) linear block code

Messages Codewords
(co, CI, C2) (VO,VI,V2,V3,V4,V5)

(0 0 0) (0 0 0 0 0 0)
(1 0 0) (0 1 1 1 0 0)
(0 1 0) (1 0 1 0 1 0)
(1 1 0) (1 1 0 1 1 0)
(0 0 1) (1 1 0 0 0 1)
(1 0 1) (1 0 1 1 0 1)
(0 1 1) (0 1 1 0 1 1)
(1 1 1) (0 0 0 1 1 1)

2.2 Linear Systematic Block Codes

A linear systematic block code has the additional structure that the
message sequence is itself part of the codeword. In addition to the
k-digit message sequence the codeword contains the (n - k )-digit
parity check sequence. This format allows direct extraction of the
message from the codeword. The systematic format is shown below
mes,:'age codeword

'(co, CI, ... , Ck-;) ----+ VO, VI, ... ,Vn-k~l' CO, CI, ... Ck-l'
, y ' , y "
parity-check message

The generator matrix for a systematic block code has the following
form
G = [P IkJ
Ik is the k x k identity matrix and P is a k x (n - k) matrix of the
form
Poo POl PO,n-k-1
PlO Pll PI,n-k-1
P= P20 P21 P2,n-k-1

Pk-I,O Pk-l,l Pk-l,n-k-l

where Pij = 0 or 1.
16 Block Codes

Example 2.2 The (6,3) code given by Table 2.1 is a systematic

code since its generator matrix can be represented in the form

G = [ :~ 1= [~1 1~ 0~ 0~ 0~ ~1 1= [P
~
13 ]

2.3 Parity Check Matrix

Let us consider a systematic linear block code specified by its gen-
erator matrix in the form

Given a message sequence c the codeword is obtained as

v=c·G
Since the code is systematic the codeword can be written as
v = [a c]
where
a = c· P (2.1)
is the parity check consisting of (n - k) digits. Note that a +a = 0
and combining with (2.1),
a+c·P=O (2.2)
The above equation can be rewritten as

[a c]· [ ~-k 1= 0 (2.3)

where I n - k is the (n - k) identity matrix. (2.3) can be written in a

more compact form as
v· HT = 0 (2.4)
where
H = [In _ k p T]
The (n - k) x n matrix H is called the parity check matrix. It is
used as an another way to specify an (n - k) linear block code.
The parity check matrix is used in error detection to test whether
a received vector is a codeword by checking (2.4).
The Minimum Distance of a Block Code 17

2.4 The Minimum Distance of a Block

Code
Let us consider an (n, k) linear block code. The Hamming weight (or
weight) of a codeword v is defined as the total number of nonzero
components in the codeword. The Hamming distance, d(v, u), be-
tween two codewords v and u is defined as the number of places
in which these two codewords differ. For example, if the two code
words are v = (100111) and u = (010001) the Hamming distance
between them is 4. The Hamming distance between two codewords
is identical to the weight of their modulo-2 sum. The minimum
Hamming distance (or minimum distance) of a code, dmin, is defined
as the smallest Hamming distance between two different codewords
in the code. The linearity property of block codes requires that the
modulo-2 sum of two codewords is another codeword. This implies
that the minimum distance of a linear block code is the smallest
weight of the nonzero codewords in the code. The importance of
the minimum distance parameter is in the fact that it determines
error correcting and detecting capability of a code.

Example 2.3 Determine the minimum distance of the (6,3)

code given in Example 2.1. A linear block code is defined by the
set of codewords that satisfy

v· HT = 0

Let us express the parity check matrix in the form

where hi represents the ith column of H. Then the code can be

specified as

Va . ha + VI . hI + ... + Vi . hi + ... + Vn-I • hn- I = 0

where Vi is the ith component of the codeword v. To satisfy this

equation codeword v must have nonzero components in such posi-
tions that corresponding columns of H sum to the zero vector O.
18 Block Codes

On the other hand, the minimum distance of the code is the small-
est number of nonzero components in a codeword. Therefore, the
minimum distance of a linear block code is equal to the smallest
number of columns of the matrix H that sum to O. The parity
check matrix H for the (6,3) code is given by

1 000 1 1
H= [ 0 1 0 1 0 1
1
001 1 1 0

All columns in H are nonzero and no two of them are the same.
Hence, no two or less columns sum to O. Hence, the minimum
distance of the code is 3.

2.5 Maximum Likelihood Decoding of

Block Codes for a BSC Channel
For a BSC channel the demodulator makes hard decisions and the
received sequence r is a binary sequence. The maximum likelihood
decoding (MLD) rule for a hard decision decoder over a BSC channel
can be stated as follows. For each codeword v in an (n, k) block
code compute the conditional probability

P(r I v)

The codeword with the maximum conditional probability is chosen

as the estimate of the actually transmitted codeword v. For a BSC
channel with the transition probability p the conditional probability
is given by
P(r I v) = pd(r,v)(l _ p)n-d(r,v)
where d( r, v) is the Hamming distance between the vectors rand
v.
Since for p <1/2, P(r I v) is a monotonically decreasing function
of d(r, v), we will have

P(r I v) > P(r I w)

Maximum Likelihood Decoding of Block Codes for a Gaussian Channel 19

if and only if
d(r, v) < d(r, w)
Hence the MLD rule for a BSC channel can be obtained by com-
paring the received vector r with all codewords and selecting the
codeword that has the smallest Hamming distance from r as the
estimate of the actually transmitted codeword. This decoding rule
is known as the minimum distance decoding. The MLD rule results
in the minimum block error probability.

2.6 Maximum Likelihood Decoding of

Block Codes for a Gaussian Chan-
nel
Let us consider a communication system with an (n, k) block code
and binary antipodal modulation. That is, the modulator maps
binary symbols Vi to the modulated signals Xi according to the rule

_{+1
Xi -
-1
Vi
Vi
=1
= 0

where Vi is the ith component in codeword v. We assume that the

signal in the channel is corrupted by additive white Gaussian noise.
The ith component of the received sequence ri is given by

where ni is a sample of a Gaussian noise with zero mean and vari-

ance a 2 • The demodulator takes samples of the received sequence

and presents it to the decoder. Since the demodulator does not

produce binary outputs the decoder will make soft decision decod-
ing. The maximum likelihood soft decision decoder computes the
conditional probability
P(r I v)
20 Block Codes

for each codeword v and selects the codeword with the largest
conditional probability as the estimate of the actually transmitted
codeword.
The conditional probability P(r I v) for a channel with noise
affecting each sample being independent Gaussian random variable
with zero mean and variance a 2 is given by

P{r I v) = rr n- 1

i=O
1
--===e
J27ra2
_(r._~)2
20-

(
1 )n e _ " n - l (r._x;)2
L.J.=o ~
J27ra 2
where Xi is the ith component of the sequence x obtained by mod-
ulating codeword v. The above expression is a monotonically de-
creasing function of Er~l (ri - Xi)2 and it is maximized when
n-l
d~ =L (ri - Xi)2
i=O

is minimized. The sum represents the Euclidean distance between

the received sample sequence and the modulated codeword. Hence,
the MLD rule for a Gaussian channel can be accomplished as fol-
lows. The decoder computes the Euclidean distance between the
received sample sequence and all modulated codewords and selects
a codeword with the minimum Euclidean distance as the estimate of
the actually transmitted codeword. However, this method becomes
impractical for long codes.

2.7 Weight Distribution of Block Codes

The weight distribution of a block code is useful in computing prob-
abilities of undetected or uncorrected errors. In this section we will
consider the weight distribution of a block code. Subsequently, we
will apply it in the calculation of word and bit error probability
bounds for a block code on an additive white Gaussian noise chan-
nel.
Given an (n, k) linear systematic block code, its weight distri-
bution can be expressed by the code weight enumerating function
Weight Distribution of Block Codes 21

(WEF) [14]. The WEF of a code is given by

n
A(X) = LAiX i (2.5)
i=O

where Ai is the number of codewords of Hamming weight i and X

is a dummy variable. The set

is called the weight distribution or the weight spectrum of the code.

There is a special class of codes with a binomial weight distri-
bution. These codes are known as perfect codes.
The WEF describes the code weight distribution which only
depends on the codewords. It does not give the contributions of
the input information bits to the total Hamming codeword weight.
A more detailed weight profile of a code is described by the input-
output weight enumerating function (IOWEF) defined as

A(W,X) = LAw,xWwxx (2.6)

w,x

where Aw,x is the number of codewords of weight x generated by

input information of weight w. For systematic block codes, we
can separate codeword weights into input information weight and
parity check information weight and define the input-redundancy
weight enumerating function (IRWEF) of a code as

A(W, Z) = L Aw,z WW ZZ (2.7)

w,z

where Aw ,z is the number of the codewords of the block code with

input information weight wand parity check information weight z.
The overall Hamming weight of a codeword is d = w + z. The rela-
tionship between the WEF coefficients Ai, the IOWEF coefficients
Aw,x and the IRWEF coefficients Aw,z is given by

Ai = L Aw,z, Ai = LAw,x, Aw,z = Aw,xl

z=x-w
(2.8)
w+z=i x=i
22 Block Codes

Furthermore, the IRWEF can be decomposed according to the

contributions of distinct input information weights w, as

A(W, Z) =L ww Aw(Z) (2.9)

w

where Aw(Z) is called the conditional weight enumerating function.

The conditional WEF describes the parity check weights of the
codewords generated by input information of weight w. It is given
by
(2.10)
z
The relationship between the IRWEF and the conditional WEF can
be rewritten as

(2.11)

Example 2.4 Weight enumerating functions for a (7,4) Ham-

ming code
Let us consider a systematic (7,4) Hamming code with generator
matrix
000 1

G(D) = [ ~100 1
o 1 0 1 1 0 ~ i1 (2.12)
o 0 1 0 1 1
The WEF of the code is

A(X) = 1 + 7X 3 + 7X4 + X7 (2.13)

The IOWEF of the code is

A(W, X) = 1 + 3WX 3 + WX4 + 3W 2X 3 + 3W 2X 4

+ W 3X 3 + 3W 3X 4 + W4X7 (2.14)
Separating the codeword weights into the input information weight
and the parity check information weight, we get for the IRWEF

A(W, Z) = 1 + W (3Z 2 + Z3) + W2 (3Z + 3Z 2)

+ W 3(1 + 3Z) + W 4 Z 3 (2.15)
Performance Upper Bounds 23

The conditional WEF's of the code are

Ao(Z) 1
AI(Z) 3Z 2 + Z3
A 2 (Z) 3Z + 3Z 2
A3(Z) 1 +3Z
A 4 (Z) Z3

2.8 Performance Upper Bounds

Weight enumerating functions can be used to calculate code er-
ror performance. In particular, the WEF specifies the weight dis-
tribution of codewords which determines the word error probabil-
ity, while both IRWEF and conditional WEF's characterize the
input/output code weight profile, which can provide information
on bit error probability.

2.8.1 Word Error Probability Upper Bounds

In this performance analysis, we assume binary phase shift keying
(BPSK) modulation is used to map each binary symbol into a signal
from the { -1, 1} modulation signal set. The modulated signals are
transmitted over an AWGN channel. The system model is shown
in Fig. 2.1.
Let a coded sequence be v = (vo, VI,"', Vn-l), where n is the
code length. The modulated sequence is x = (Xo, Xl,' . " Xn-l). At
the receiver, the received sequence r = (ro, rl,"', rn-d is observed.
The received signal at time i can be represented by

(2.16)

where ni is a noise sample with a zero mean and variance (]"2. The
noise sequence is n = (no, nl, .. " nn-d. The decoder performs a
maximum likelihood sequence decoding which maximizes the prob-
ability P(r I v).
24 Block Codes

c Channel v x
Modulator
Encoder

+ n

Channel r
Demodulator
Decoder

Fig. 2.1: Coded system model

If errors occur in transmission the decoder might choose another

code sequence as the decoded sequence. That is, an error event oc-
curs whenever the received sequence is closer in Euclidean distance
to another code sequence v. The sequence v is called an error se-
quence. The probability that the decoder makes a wrong decision
by selecting an error sequence is called the error event probability
or pairwise error probability.
Let the transmitted sequence associated with the code sequence
v be x. The pairwise error probability, denoted by Pd , can be given
by

(2.17)

where d is the Hamming distance between the pair codeword se-

quences. In this analysis, we will introduce a random variable

n-l
A = L ni (Xi - Xi) (2.18)
i=O

Since ni is a zero mean Gaussian noise sample with variance (J2,

the random variable A is a Gaussian variable with zero mean and
variance (J~ = 4d(J2. Thus the pairwise error probability can be
Performance Upper Bounds 25

expressed as [3]

(2.19)

where R = kin is the code rate, Eb is the signal energy per bit, No
is the single sided power spectral density of the Gaussian noise and
Q(x) is the complementary error function defined by

(2.20)

The word error probability of a code on AWGN channels can

be upper-bounded by a union bound that sums contributions from
all error events with various Hamming distances. The word error
probability union bound is given by

(2.21)

where dmin is the code minimum Hamming distance, Ad is the num-

ber of error events with Hamming distance d, called the error coef-
ficient.
For linear codes, we may assume the all-zero code sequence is
transmitted. The error coefficient Ad will be the number of code-
words with weight d. It can be obtained from the code WEF A(X).
Considering the Q-function bound

(2.22)

in (2.21), we obtain an upper bound for the word error probability

"L -1 A de -dR5
No

d=dmin
2

~[A(X)
2 - III X=e
E
-R~
NO
(2.23)
26 Block Codes

As an alternative to (2.22), making use of the inequality

(2.24)

a tighter upper bound can be obtained from (2.21) as

2d . R-
mIll
Eb)
No e
~
d minR N ""'
o.!-'.
A
de
-dR ~
N
0
d-dmm

2d min R ~b) edminR~ [A(X) - 111 -R~ (2.25)

o X=e ~

2.8.2 Bit Error Probability Upper Bounds

The bit error probability of an (n, k) block code decoded by a max-
imum likelihood algorithm over an AWGN channel can be upper-
bounded by a union bound as

d=dmin

(2.26)

where Bd is the error coefficient, i.e., the average number of bit er-
rors caused by transitions between the all-zero codeword and code-
words of weight d (d 2: d min ). The error coefficient Bd can be
obtained from the code IRWEF. It is given by

(2.27)

Bd determines the contribution of the codewords with the same

weight d to the bit error probability. The set of all pairs of (d, B d ),
denoted by

(2.28)

is called the code distance spectrum.

Performance Upper Bounds 27

Considering the Q-function bound in (2.22), the bit error prob-

ability upper bound can be given by

"~ -IBde -dR~

NO

d=dmin
2

E k W
2k Ww Aw(Z)
/
W=z=e-R~
W8A(W,Z)/ (2.29)
2k 8W w=z=e-R~
In addition, a tighter bit error probability upper bound can be
expressed as

Eb) edminR~
2dmm. R M "
~
B de -dR~ (2.30)

t
o d=dmin

r-2d-m-inR-E-b') edminR~ WWWAw(Z)IE

No w=l k W=Z=e -R~
I
r------=::-.
2d . R Eb) dminR~ W 8A(W, Z)
mm M e k 8W _RIEiL
o W=Z=e NO

Example 2.5 Error performance upper bounds for the (7,4)

Hamming code from Example 2.4
The word error probability upper bound of the code is

"~ -lA d e -dR~

NO

d=dmin
2
7 -3R~ 7 -4R~ 1 -7R~
-e NO + -e No + -e NO (2.31)
222
By referring to (2.27) and (2.14), the code error coefficients Bd
can be computed as
123
d=3 Bd = -4
x 3+ - x 3+ -
44
x 1=3
123
d=4 Bd = - x 1 + - x 3 + - x 3 = 4
444
4
d=7 Bd = -4 x 1=1 (2.32)
28 Block Codes

The bit error probability upper bound is

"
L.,; -1 B d e -dR.§z.
NO
d=dmin
2
3
-e
-3R.§z.
NO + 2e-4R.§z. 1 -7R.§z.
No + -e No (2.33)
2 2
The word error probability and bit error probability upper bounds
are shown in Fig. 2.2.

Word Error Rate

Bit Error Rate

'" ... ....

,.
,..
...
" ",
"... ...
10-6
...
"

10-6L-_L-_-'--_~_~_...L-_--L-_--:':-_-L-_---:'-_....:..J
o 23456789
Ell/No (dB)

Fig. 2.2: Performance upper bounds for the (7,4) Hamming code

2.9 Coding Gain

Coded systems typically require a less SNR than uncoded systems
to achieve the same output bit error probability. This reduction,
expressed in decibels, in the required SNR to achieve a specific bit
error probability, of a coded system over an uncoded system, is
called coding gain. The asymptotic coding gain of a coded system
Coding Gain 29

with soft decision decoding over an uncoded reference system, if the

two systems have the same bandwidth, is defined as

d~,min
G = 101OgIO-;P- dB (2.34)
u

where du is the Euclidean distance of the uncoded system and dE,min

is the minimum Euclidean distance between all modulated code-
words. At high SNR's most likely errors that the decoder will make
will be coming from the modulated codewords at the Euclidean
distance dE ,min from the correct modulated codeword.
However, coded systems often require a higher bandwidth than
uncoded systems resulting in different noise variances. The asymp-
totic coding gain in this case will be

(2.35)

where a~ and a~ are the variances of Gaussian noise for the coded
and uncoded systems, respectively.

Example 2.6 Compute the asymptotic coding gain for the

(7,4) block code with the minimum Hamming distance of 3.
The minimum Euclidean distance of the uncoded system with bi-
nary antipodal modulation is du = 2. The minimum Euclidean
distance of the code combined with binary antipodal modulation
is dE,min = 2V3. The coded system with the (7,4) code requires
a bandwidth 7/4 times higher than the bandwidth of the uncoded
system. Therefore, the variance of the white additive Gaussian
noise of the coded system a~ is related to the variance of the same
noise in the uncoded system a; by

The asymptotic coding gain is then

124
G = 101ogiO 47 = 2.34dB
30 Block Codes

2.10 Soft Decision Decoding of Block

Codes
We will present here a look-up table algorithm that performs soft
decision maximum likelihood decoding for block codes.
Let us consider an (n, k) binary block code. Assume that the
transmitted codeword is v. In the modulator each binary symbol is
mapped into one of two antipodal signals {-I, I}. At the receiver,
the demodulator only samples the received signal, presenting a se-
quence of sampled symbols r to the decoder, where

This algorithm is based on the assumption that for a Gaussian

channel most likely error patterns will be closest in the Euclidean
distance sense to the received sequence.
The decoding operations can be summarized as follows.
1. Compute Euclidean distances between the received sample
sequence r and each codeword from the code set {VI, V2, ... , V2k},
which is stored in a look-up table.
2. Choose a codeword Vj which is closest to the received se-
quence r in Euclidean distance as the estimate of the transmitted
codeword.
This algorithm can be used for relatively short block codes only,
as its complexity increases with the message length.

2.11 Trellis Structure of Linear Binary

Block Codes
In 1974 Bahl, Cocke, Jelinek and Raviv [8] presented a method
for representing codewords of a linear block code by a trellis. Later,
Wolf [9] introduced an identical trellis and applied the Viterbi al-
gorithm for maximum likelihood decoding of block codes. In 1988
Forney [10] generated a trellis which minimizes the number of nodes
at each depth. For this reason this trellis is known as the minimal
trellis.
Thellis Structure of Linear Binary Block Codes 31

Consider a linear (n, k) code C over a binary field GF(2), with

a parity check matrix H

(2.36)

where hi is an (n - k) x 1 column vector.

Note that the analysis can be easily generalized to include non-
binary codes, defined over a finite field GF(q).
The parity check matrix H characterizes the code C. The code
C then consists of all vectors
(2.37)

such that

(2.38)

The code C can also be described by a trellis. A trellis is a set

of nodes interconnected by unidirectional branches. Every node is
assigned an integer 1, which is referred to as the depth of the trellis,
and an (n - k)-tuple, called a state, denoted by 8 i (1), for a certain
integer i. So there will be at most 2n - k states. They are all ordered
from 0 to 2n - k - 1, with 0 referring to the all-zero (n - k)-tuple.
There is only one node at depth 0, denoted by 8 0 (0) and only
one node at depth n, denoted by 8 0 (n). A path in the trellis of
length L is a sequence of L branches. Let us denote by II the
set of node subscripts at depth 1, which represents a subset of the
integers {O, 1,2,· .. , 2n - k - 1}. The trellis for the linear block code
C is constructed as follows.
At depth 1 = 0, the trellis contains only one node, which is the
all-zero (n - k)-tuple 8 0 (0).
For each 1 = 0,1, ... , (n - 1) the set of states at depth (1 + 1) is
obtained from the set of nodes at depth 1 by the formula

(2.39)

for all i E /z, m E I I +1, and j E {O, 1}, where at are binary inputs
. {1
ai = 0
j=1
j=O
(2.40)
32 Block Codes

Therefore, for each i in It, connecting branches are formed be-

tween node Si (I) and two nodes at depth (I + 1), for two different
binary values a1 E {O, I}, according to the above formula. Each
branch is labelled by the corresponding value a1.
At depth 1 = n, the final node, So(n), is given by
n-l
So(n) = So(O) + :L a1hi = 0 (2.41)
i=O

As both So(n) and So(O) are the all-zero states, it implies

n-l

:La1h = 0
i (2.42)
i=O

As any codeword satisfies Eq. (2.38), which is identical to Eq.

(2.42), it implies that only those a1 which constitute a codeword
will form paths in the trellis ending in the final all-zero state. Thus
every valid path in the trellis corresponds to a codeword.
By the construction method in Eq. (2.39), all possible binary
n-tuples, are formed in the trellis. There are total 2n of them.
They include 2k valid codewords. Thus, every codeword in C is
represented in the trellis and it corresponds to a path in the trellis.
Finally, we remove any nodes that do not have a path to the
final all-zero state at depth n and all branches leading to these
expurgated nodes.

Example 2.7 We illustrate the trellis construction for the bi-

nary (5,3) code with the parity check matrix

(2.43)

Following the described procedure we obtain the trellises before

and after expurgation, as shown in Fig. 2.3 and 2.4.
Trellis Structure of Linear Binary Block Codes 33

(00)

(01)

(10)

(11)

Fig. 2.3: Trellis for the binary (5,3) code

l=O l=1 l=2 l=3 l=4 l=5

(00)

(01)

(10)

(11)

Fig. 2.4: Expurgated trellis for the binary (5,3) code

Bibliography

[IJ J. M. Wozencraft, and I. M. Jacobs, "Principles of Communi-

cation Engineering," John Wiley, 1965.
[2J R. G. Gallager, "Information Theory and Reliable Communi-
cations," John Wiley, 1968.
[3J J. G. Proakis, "Digital Communications", 2nd Ed. McGraw-
Hill, New York, 1989.
[4J S. Lin, and D. J. Costello, Jr., "Error Control Coding: Funda-
mentals and Applications", Prentice-Hall, 1983.
[5J E. R. Berlekamp, "Algebraic Coding Theory', McGraw-Hill,
1968.
[6J R. Blahut, "Theory and Practice of Error Control Codes",
Adison-Wesley Publishing Company, 1983.
[7J A. J. Viterbi, and J. K. Omura, "Principles of Digital Com-
munication and Coding', McGraw-Hill, New York, 1979.
[8J L. R. Bahl, J. Cocke, F. Jelinek and J. Raviv, "Optimum De-
coding of Linear Codes for Minimizing Symbol Error Rate",
IEEE Trans. Inform. Theory, vol. IT-13, 1974, pp. 284-287.
[9] J. K. Wolf, "Efficient Maximum Likelihood Decoding of Linear
Block Codes", IEEE Trans. Inform. Theory, vol. IT-24, Jan.
1978, pp. 76-80.
[10] G. D. Forney, Jr., "Coset Codes-Part II: Binary Lattices and
Related Codes", IEEE Trans. Inform. Theory, vol. 34, Sep.
1988, pp. 1152-1187.
36 BIBLIOGRAPHY

[11] R. J. McEiece, "On the BCJR Trellis for Linear Block Codes",
IEEE Trans. Inform. Theory, vol. 42, No.4, July 1996.
[12] H. Imai, "Essentials of Error-Control Coding Techniques",
Academic Press, 1990.

[13] D. Chase, "A Class of Algorithms for Decoding Block Codes

with Channel Measurement Information", IEEE Trans. In-
form. Theory, vol. IT-18, Jan. 1972, pp. 170-182.

[14] S. Benedetto and G. Montorsi, "Unveiling turbo-codes: Some

results on parallel concatenated coding schemes," IEEE
Trans. Inform. Theory, vol. 42, no. 2, Mar. 1996, pp. 409-428.
Chapter 3

Convolutional Codes

3.1 Introduction
Convolutional codes have been widely used in applications such as
space and satellite communications, cellular mobile, digital video
broadcasting etc. Its popularity stems from simple structure and
availability of easily implement able maximum likelihood soft deci-
sion decoding methods.
Convolutional codes were first introduced by Elias [5]. The
ground work on algebraic theory of convolutional codes was per-
formed by Forney [6]. In this chapter we present the main results
on the algebraic structure of convolutional codes needed in the de-
sign of turbo codes. Of particular importance are code structure,
encoder realization and trellis representation. We first discuss the
structure of convolutional codes and then show how to implement
feedforward and feedback convolutional encoders.
The equivalence of encoders is discussed and it is shown how to
get equivalent systematic encoders with rational generator matrices
from nonsystematic encoders with polynomial generator matrices.
Next, we present the finite state machine description of convolution
codes and derive the state and trellis diagrams.
In the interest of brevity many details of the theory had to be
omitted. The chapter is concluded by the discussion of punctured
convolutional codes.

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
38 Convolutional Codes

Fig. 3.1: A rate 1/2 convolutional encoder

3.2 The Structure of (n,l)

Convolutional Codes
Let us consider a simple rate 1/2 convolutional code generated by
the encoder shown in Fig. 3.1. The encoder is a linear feedforward
shift register. The input data stream is a binary sequence, given by

c = (... , C-,1 Co , C1 , ••• " Cl ••• ) (3.1)

where 1 is a time instant.

The outputs are two sequences V(l) and V(2)

(... , v~li, V~l), vP), ... ,v?), ... )

(... , v~i, v~2), vi2), ••• , v?), ... ) (3.2)

where we assume that vii), i = 1,2 are binary symbols.

At time 1 the input to the encoder is Cl and the output is a code
block VI
(3.3)
The elements of the output code block are computed as
(1)
VI Cl + CI-2
Cl + CI-1 + CI-2
(2)
VI (3.4)
The Structure of (n,l) Convolutional Codes 39

where + denotes modulo-2 addition. Hence, the encoder must store

past two information bits. We say that the memory of the encoder is
v = 2. The two output sequences are multiplexed by a commutator
to form a single code sequence.

(3.5)

The set of all possible code sequences v produced by the encoder

forms the (2,1) convolutional code. The connections between the
shift register elements and the modulo 2 adders can be conveniently
described by the following two generator sequences

g(1) = (g~l)gP)g~l)) = (101)

g(2) = (g~2) g~2) g~2)) = (111)

where g(l) represents the upper and g(2) the lower connections with
the leftmost entry being the connection to the leftmost stage. The
term convolutional codes comes from the observation that the ith
output sequence, i = 1,2, given by Eq. (3.2), represents the convo-
lution of the input sequence and the ith generator sequence.

(3.6)

where * denotes the convolution operator.

Example 3.1 Let us assume that the input sequence is

c = (1011100···) (3.7)

Then the two output sequences are

v(1) = (1001011 ... ) (3.8)

v(2) = (1100101 ... ) (3.9)

The transmitted code sequence is

v = (11,01,00,10,01,10,11,··0 (3.10)
40 Convolutional Codes

If the generator sequences g(l) and g(2) are interleaved and ar-
ranged in the matrix
9~1) 9~2) 9P) 9i2) 92(1) 92(2) o 0 o 0
o 0 9~1) 9~2) 91 91 92(1) 92(2) o 0
(1) (2)

G= o 0 0 0 90(1) 90(2) 91(1) 91(2) 92(1) 92(2)

o 0 o 0
the encoding operation can be represented in a matrix form as
V= c· G (3.11)
where all operations are modulo 2.
Observe that each row of G is obtained by shifting the preceding
row by n = 2 places to the right. The generator matrix is semi-
infinite corresponding to the fact that the input sequence may be
infinite.
Similarly, we can generate an (n, 1) convolutional code by a
feedforward linear shift register shown in Fig. 3.2, which has one
input and n output sequences. The code rate is lin.
The encoder is specified by a set of n generator sequences of
length (lJ + 1), where lJ is the encoder memory, given by
g(l) (9~1) 9P) .. ·9£1))
g(2) (9~2) 9i2) ... 9£2)) (3.12)

g(n) = (9~n) 9in) .. . 9£n))

with coefficients in the binary field GF(2). The ith output sequence
is obtained by convolving the input message sequence c and the ith
generator sequence g(i)
V(i) -
-
c * g(i)
""
i = 1 2 ... n (3.13)
At the time l, the ith output symbol is
(i)
VI

(3.14)
The Structure of (n,l) Convolutional Codes 41

Input c
... --Bt
Cl Cl-I Cl- v Cl Cl-I Cl- v Cl Cl-I Cl- v

Fig. 3.2: A general (n, 1, v) convolutional code feedforward encoder

Let us assume that the input message c consists of L bits. As

it takes v time instants to move out of the memory, each output
sequence consists of L + v binary symbols.
We can associate with input and output streams, c(D) , v(i)(D),
respectively, power series sequences in the form

c(D) ... + c_1D- 1 + Co + clD + C2 D2 + ...

v(i)(D) (i) D-1 + Vo(i) + VI(i)D + V2(i)D2 + .••
• . • + V-I

where the symbol D is an indeterminate. The indeterminate D

can be considered as a unit delay operator. Its power denotes the
number of time units the binary symbol is delayed with respect to
the initial binary symbol in the sequence I .
A feedforward shift register of length v can be represented by
a polynomial of degree at most v in the field of the register. The
coefficients of the polynomial are the shift register taps. These
polynomials are called the generator polynomials.
IThe delay operator D corresponds to Z-l in sampled data theory, where
Z is a complex variable. Operators involving D are called transform domain
operators. c(D) and vieD) can be regarded as modulo-2 transforms of c and
v(i), respectively.
42 Convolutional Codes

For the encoder in Fig. 3.1 the generator polynomials are

(3.15)

where all the operations are modulo-2.

The generator polynomials of a convolutional code are typically
arranged into a matrix of polynomials called the generator matrix.
For the encoder in Fig. 3.1 the generator matrix is

(3.16)

The encoding equations for the encoder in Fig. 3.1 can be now
written as

c(D)g(1)(D)
C(D)g(2)(D) (3.17)

After multiplexing the entire code sequence becomes

(3.18)

Example 3.2 For the information sequence

(3.19)

the encoding equation becomes

(1 + D2 + D3 + D4)(1 + D2) = 1 + D3 + D 5 + D6
(1 + D2 + D3 + D4)(1 + D + D2) = 1 + D + D4 + D6

The transmitted code sequence is

v(D) v(1)(D2) + DV(2) (D2)

1+D + D3 + D6 + D9 + DlO + D12 + D13
which agrees with the result in Example 3.1.
The Structure of (n, k) Convolutional Codes 43

For an (n, 1) convolutional code there are n generator polynomi-

als in the generator matrix. They are arranged into a 1 x n matrix
of polynomials

(3.20)

The encoding operation can be represented as a vector matrix prod-

uct
v(D) = c(D)G(D) (3.21)
where
(3.22)

3.3 The Structure of (n, k)

Convolutional Codes
In general, the encoder has k input and n output sequences. Each
input information sequence is encoded into a code sequence. The
set of all possible code sequences generated by the encoder forms
an (n, k) convolutional code. As with block codes, the code rate is
defined as the ratio R = kin.
Any (n, k) convolutional code is specified by k x n generator
polynomials, which form the generator matrix, G(D)

gll gl2 ... gIn [

G(D) = [ g~l g22 ... g~n (3.23)
gkl gk2 ... gkn

Consider k input sequences CI, C2,· .. , Ci,· .. , Ck. Each of them

can be represented by a power series

(3.24)

for i = 1,2,···, k. They can be arranged as a row vector of se-

quences
(3.25)
44 Convolutional Codes

Alternatively, we can write for c(D) as

c(D) = [C~l), C~2), ... ,C~k)] + [Cll) ,Cl2) , ••• , clk)] D

+ ... + [CP) ,C~2) , ... , c~k)] Dl + ...
Each bracketed block represents an input block. Similarly, the en-
tire output sequence consists of n code polynomials

(3.26)

The encoding operation can now be represented by a vector-

matrix product
v(D) = c(D)G(D) (3.27)
The jth code sequence is computed as
k
v(j)(D) = L C(i) (D)gij(D) (3.28)
i=l

After multiplexing the transmitted code sequence is given by

The memory of the ith encoder input is given by

(3.30)

The memory of the encoder is defined as the maximum memory of

the encoder inputs
(3.31)

and the overall encoder memory as the sum of the memories of the
encoder inputs
(3.32)
i=l

The encoder for an (n, k) convolutional code can be realized by

a linear sequential circuit consisting of k feedforward shift registers,
the ith of length Vi, with the outputs formed as modulo-2 sums of
the corresponding shift register contents.
Systematic Form 45

Example 3.3 Consider the (3,2) convolutional encoder shown

in Fig. 3.3. The generator matrix for this encoder is

(3.33)

If the input sequences are

(3.34)

the encoding operation can be represented as

v(D) [1 + D2 ~ ~ 1~ D 1
1 + D] [ 1 D

[1 + D3 1 + D3 D2 + D 3] (3.35)

After multiplexing the entire code sequence is given by

v(D) + DV(2) (D 3) + D 2v(3) (D 3)

v(1)(D3)
1 + D + DB + D9 + DlO + Dll (3.36)

The overall memory for the encoder in Fig. 3.3 is 2 and the
memory for each encoder input is 1. The number of memory el-
ements required for this type of implementation is equal to the
overall memory.

3.4 Systematic Form

In a systematic (n, k) convolutional code the first k output se-
quences are exact replicas of the input information sequences. The
generator matrix of a systematic convolutional code is of the form

G(D) = [I P(D)] (3.37)

where 1 is a k x k identity matrix, and P(D) is a k x (n-k) matrix

of polynomials with elements in GF(2).
46 Convolutional Codes

(1)
Cl

(2) (2)
+ VI VI V

(2)
Cl

Fig. 3.3: Encoder for a (3,2,1) code

For a block code a generator matrix can be transformed into a

systematic form by elementary row operations. This is not always
possible for convolutional codes.
In order to obtain a systematic form for a given generator matrix
of a convolutional code we need first to define equivalent generator
matrices. Two generator matrices are equivalent if they generate
the same convolutional code.
Example 3.4 Consider the generator matrix for the encoder in
Fig 3.1.
(3.38)
The code sequence of this encoder is given by
v(D) c(D) [1 + D2 1 + D + D2]

c(D)(l + D2) [ 1 1+ D
1 + D2
+ D2] (3.39)

c'(D) [1 1~~;2D2]
c' (D)G 1 (D)
Systematic Form 47

where
c' (D) = c(D)T(D) (3.40)
and
(3.41)
Clearly, multiplication of c(D) by T(D) generates a scrambled ver-
sion of the input sequence. Thus, the set of scrambled input se-
quences, c' (D), multiplied by the generator matrix GI(D) produces
the same set of output sequences as the original generator matrix
G(D), where
G(D) = T(D)G I (D) (3.42)
We say that these two matrices are equivalent. The set of sequences
c(D) is identical to the set of sequences c' (D) if and only if T(D)
is invertible. The inverse of T(D) in this example is given by

(3.43)

In general, two encoding matrices G( D) T(D)G I (D) and

GI(D) are equivalent if the matrix T(D) is invertible.
Matrix G I (D) in the previous example is in systematic form.
However, its entries are not polynomials, but rational functions.
The parity output sequence in Eq. 3.39 is obtained by multiplying
the input sequence by the polynomial (1 + D + D2) and dividing it
by the polynomial (1 + D2). The multiplication operations can be
performed by a feed forward shift register and division by a feedback
shift register.
In general, the multiplication and division of the input sequence
c(D) by the polynomials a(D) and q(D), respectively, as shown
below
a(D)
v(D) = c(D) q(D) (3.44)

can be performed by the circuits illustrated in Figs. 3.4 and 3.5.

Note that a(D) and q(D) are the polynomials with binary coeffi-
cients
a(D) ao + aID + ... + avDv
q(D) 1 + qlD + . .. + qvDV (3.45)
48 Convolutional Codes

It is assumed that q(D) is a delay free polynomial, i.e. qo = 1. In

other words, it can always be written in this form by pulling out
the common form Di, where i is the delay.
v

Fig. 3.4: The controller canonical form of a rational transfer function

a(D)jq(D)

The ratio
T(D) = a(D) (3.46)
q(D)
represents a rational transfer function. In general, the generator
matrices of convolutional codes have as entries rational transfer
functions. The outputs are obtained by multiplying the input se-
quence with the generator matrix containing rational transfer func-
tions. Given a rational transfer function and the input sequence the
multiplication can be performed by linear circuits in many different
ways. Fig. 3.4 shows the controller canonical form of the rational
function in Eq. (3.46).
Another so-called observer canonical form of the rational trans-
fer function in Eq. (3.46) is illustrated in Fig. 3.5.
As the circuit in Fig. 3.5 is linear, we have
v(D) = c(D)(ao + aID + '" + avDV) + V(D)(qID + ... + qvDV)
which is the same as (3.44). In this implementation the delay ele-
ments, in general, do not form a shift register, as they are separated
Systematic Form 49

Fig. 3.5: The observer canonical form of a rational transfer function

a(D)/q(D)

by adders.
Systematic encoders in the controller and observer canonical
form, based on the generator matrix G1(D), are shown in Figs. 3.6
and 3.7, respectively.
c

Fig. 3.6: The controller canonical form of the systematic (2,1) encoder
with the generator matrix G 1 (D)

In some cases, the elements of the systematic code generator

matrix are rational fraction of the unit delay operator D. For such
generator matrices it is not possible to construct a systematic en-
coder with feedforward registers. It is, however, always possible to
50 Convolutional Codes

Fig. 3.7: The observer canonical form of the systematic (2,1) encoder
with the generator matrix G 1 (D)

construct a systematic encoder with a feedback register.

3.5 Parity Check Matrix

We can describe a convolutional code by a parity check matrix in
a similar way as for a block code. A parity check matrix H(D) is
an (n - k) x n matrix of rational functions, that satisfies

(3.47)

The encoding operation can be written as

(3.48)

for all code sequences v(D).

If G(D) is a systematic generator matrix, the parity check ma-
trix can be as
(3.49)

where I is an (n - k) x (n - k) identity matrix and pT(D) is the

transpose of matrix P(D) in (3.37).
We can always find a parity check matrix corresponding to a
generator matrix G(D) if G(D) is transformed into a systematic
form, by referring to (3.49) and (3.37).
For the 1/2 rate encoder shown in Fig. 3.1 the parity check
matrix obtained from the systematic generator matrix G1(D) is
Catastrophic Codes 51

given by
H(D) = [1 +1 D+D2+ D2 1] (3.50)

3.6 Catastrophic Codes

Systematic encoders have a significant advantage because the mes-
sage is displayed in the encoded sequence and can be read out di-
rectly from the received sequence. On the other hand, with nonsys-
tematic encoders the message is not visible in the coded sequence.
Therefore, an inverter is required to recover the information se-
quence from the decoded sequence. The information sequence can
be recovered by passing the decoded sequence, v(D), through an
inverter, specified by the generator matrix G -1 (D)

v(D)G- 1 (D) = c(D)G(D)G- 1 (D) = c(D)DI (3.51)

In order to obtain the original input sequence c(D) or its delayed

version c(D)Dl, where l is the time delay, the following relationship
must be satisfied
(3.52)
where I is a k x k identity matrix. That is, the generator matrix
must be invertible.
To see under what condition G- 1 (D) exists, let us consider an
(n, k) convolutional code, defined by the generator matrix G(D).
Let Ai(D) , i = 1,2,··· ( ~) be the determinants of the

( ~) distinct k x k submatrices of the generator matrix G(D).

Massey and Sain [13] have shown that the inverse polynomial ma-
trix G -1 (D) exists if and only if

GCD [Ai(D), i = 1,2,··· ( ~ )] = Dl, l 2:: 0 (3.53)

where GCD denotes the greatest common divisor.

52 Convolutional Codes

Example 3.5 For the encoder in Fig. 3.1

(3.54)

and the generator matrix inverse is

(3.55)

If the generator matrix inverse G -1 (D), does not exist, the code
is called catastrophic. For catastrophic codes a finite number of
channel errors causes an infinite number of decoding errors.

Example 3.6 The generator matrix

(3.56)

generates a catastrophic code.

As
GCD [1 + D 1 + D2] = 1 + D (3.57)
G- 1 (D) does not exist. If the information sequence is

(3.58)

the code sequences are

1
l+D (3.59)

The weight of the code sequence is 3, while the weight of the in-
put sequence is infinite. If this code sequence is transmitted over a
binary symmetric channel (BSC) in which three errors occur, chang-
ing the three non zero symbols to zero, the received sequence will
contain all zeros. Since this is a valid code sequence, the decoder
will deliver it to the user. Thus, the decoded sequence will have an
infinite number of errors, though there were only three errors in the
channel.
Catastrophic codes should be avoided for obvious reasons.
Systematic Encoders 53

Example 3.7 Consider the generator matrix for a (3,2) convo-

lutional code
1 D2
G(D) = [ 0 1 D + D2
1 1 (3.60)

The determinants of the 2 x 2 submatrices are

The GCD of these three polynomials is 1. Therefore, there exists

the inverse polynomial matrix G-1(D). It is given by

(3.61)

Hence, G(D) generates a noncatastrophic code.

It can be seen that systematic convolutional codes always satisfy
(3.53). That is due to the fact that one ofthe submatrices is the k x
k identity matrix with unity determinant. Therefore all systematic
convolutional encoders possess a polynomial right inverse matrix
G-1(D), which makes them noncatastrophic.

3.7 Systematic Encoders

Systematic encoders are simpler to implement, do not require an
inverter and are never catastrophic.
However, unless feedback is allowed in the encoder, systematic
generator matrices generate codes with worse error performance
than nonsystematic codes of the same code rate and overall con-
straint length [15].
It has been shown [4] that every convolutional encoder generator
matrix is equivalent to a systematic rational encoding matrix.
Let us assume that a given generator matrix G(D) is polynomial
and that it has the polynomial right inverse G-l(D). A polynomial
generator matrix with the right inverse has the GCD of all k x k
subdeterminants equal to Dl. Premultiplication by the inverse of
54 Convolutional Codes

such a submatrix gives an equivalent systematic generator matrix,

possibly rational. If the generator matrix is represented as

G(D) = [T(D) S(D)] (3.62)

the equivalent systematic generator matrix G 1(D) is given by

(3.63)

Example 3.8 Consider the generator matrix

G(D) = [1 OlD
D 11 (3.64)

Let T(D) be the 2 x 2 submatrix consisting of the first two

columns of G(D)

T(D) = [~ ~ 1 (3.65)

Its determinant is given by

det(T(D)) = 1

and its inverse is

T-1(D) = [~ ~ 1 (3.66)

Multiplying G(D) by T-1(D) yields a systematic generator matrix

G 1 (D) equivalent to G(D).

T-1(D) . G(D)

[ 1 0 1 + D2 ] (3.67)
OlD

The elements of G 1 (D) are polynomials over GF(2). Encoders with

generator matrices G(D) and G1(D) generates the same code. The
obtained generator matrix is polynomial and can be implemented
as a feedforward register.
Systematic Encoders 55

Fig. 3.8: Nonsystematic encoder in Example 3.9

Example 3.9 A rate 2/3 nonsystematic encoder with the basic

generator matrix
l+D D
G(D) = [ D2
1
1 1 + D + D2
1 (3.68)

is shown in Fig 3.8.

Let T(D) be the matrix formed of the first two columns ofG(D)

T(D) = [1 ~2D ~1 (3.69)

Its determinant det(T(D)) is

det(T(D)) = 1 + D + D3 (3.70)
and its inverse T- 1 (D) is given by

T
-1( )
D
1 [1
= 1 + D + D3 D
D2 1 + D
1 (3.71)

The equivalent systematic generator matrix is obtained as

(3.72)
 56 Convolutional Codes

A systematic encoder in its observer canonical form for this gener-

ator matrix is shown in Fig. 3.9.

C(l) ---...------+------+--------.--- v(1)

cOO v OO

Fig. 3.9: Systematic encoder in Example 3.9

When the systematic encoder contains feedback, the entries of

the generator matrix are rational functions of the form aF)(D)/q(D),
where
j,lV)(D)
""'l
= a~j)
%,0
+ a~j)
%,1
D + ... + a~j)
S,'" D'" , 1 -< i -< k , (k + 1) -
< J.-
<n
and
qeD) = 1 + qlD + ... + q",D'"
A rational generator matrix is realizable if qo = 1 and qeD) is
delay free. A rational generator matrix can be written as
a~k+l)(D) a~n)(D)
1 0 0 q(D) q~D)
a~k+l)(D) a~n (D)
0 1 0
G(D) = q(D) q(D)
(3.73)

0 0 ... 1
a~k+l)(D)
q(D)
akn)(D)
q(D)

The encoder circuit in its canonical observer form for this system-
atic generator matrix is depicted in Fig. 3.10.
An (n, n - 1) systematic rational generator matrix is given by
1 0
o 1 ... 0
0 ai(D)/q(D)
a~(D)/q(D)
1
G(D) = [ : (3.74)

o 0 1 ~-1 (D)/q(D)
Systematic Encoders 57

Fig. 3.10: A systematic encoder with the generator matrix in (3.73)

58 Convolutional Codes

where
a~n)
z
(D) = a~n)
z,O
+ a~n)
z,l
D + ... + a~n)
Z,II DII (3.75)
for 1 :::; i :::; n - 1. The parity check matrix for this encoder is given
by

H(D) = [aln ) (D)/q(D) a~n)(D)/q(D) ... ~~l(D)/q(D)]

or

The encoder is shown in Fig. 3.1l.

C(l)

c(n-l) v(n-I)

Fig. 3.11: Observer canonical form of an (n, n - 1) systematic encoder

3.8 State Diagram

As a finite state linear circuit, a convolutional encoder can be de-
scribed by a state diagram. The encoder state is defined as its
State Diagram 59

memory content. If K denotes the overall encoder memory, the

total number of states is 2K. The current state and the output
of the encoder are uniquely determined by the previous state and
current input. The encoder undergoes a state transition when a
message block is shifted into the encoder. The state diagram is a
graph that consists of nodes, representing the encoder states, and
directed lines, representing state transitions. Each directed line is
labelled with the input/output pair. Given a current encoder state,
the information sequence at the input determines the path through
the state diagram and the output sequence. For example, consider
the nonsystematic encoder shown in Fig. 3.1. The state diagram
for this encoder is shown in Fig. 3.12. The encoder has four states,
labelled as 80, 8 1 , 8 2 and 8 3 , where the subscript corresponds to
the integer representations of the contents of the shift register el-
ements. There are two paths leaving each state, corresponding to
two possible values of the input message bit.

Fig. 3.12: State diagram for the (2,1) nonsystematic convolutional en-
coder from Fig. 3.1

The equivalent systematic encoder for this code in the observer

canonical form is shown in Fig. 3.7. The state diagram for this
60 Convolutional Codes

encoder is illustrated in Fig. 3.13.

0/00

0/01

Fig. 3.13: State diagram for the (2,1) systematic encoder in Fig. 3.7

Note that the output sequences in both state diagrams in Figs.

3.12 and 3.13 are identical, as the encoders are equivalent. However,
the mapping between the input message bits and the output code
block is different for the two encoders.

3.9 Trellis Diagram

A trellis diagram is derived from the state diagrams by tracing all
possible input/output sequences and state transitions. As an ex-
ample, consider the trellis diagram for the (2,1) encoder described
by the state diagram in Fig. 3.12. It is obtained by expanding in
time every state transition and every path starting from all zero
state (00). The resulting trellis is illustrated in Fig. 3.14.
When the first message is shifted into the register, the encoder
can move either to state (10) if the input symbol is 1 or to state (00)
if the input symbol is O. When the next symbol enters the register
'Trellis Diagram 61

11
10
01
00
o 1 2 3 4
Fig. 3.14: Trellis diagram for the (2,1) nonsystematic encoder in Fig.
3.1

the encoder can be in one offour possible states (00), (01), (10) and
(11). The number of states is doubled upon every new shift. The
encoder reaches the maximum possible number of states, 2K after
K time units. For t > K, the structure of trellis becomes repetitive.
In the example, the encoder reaches the maximum number of states
after two time units.
There are two branches emanating from each state correspond-
ing to two different symbols. There are also two branches merging
into each state. The transitions caused by an input symbol 0 are in-
dicated by lower branches while the transitions caused by an input
symbol 1 are indicated by upper branches. Each branch is labelled
by the output block.
The output code sequence can be obtained by tracing a path in
the trellis specified by the input information sequence. For example,
if the input sequence is c = (11001), the output code sequence can
be read out from the trellis

V= (1110101111) (3.76)

In general, the trellis of an (n, k) code has 2k branches leaving each

state and 2k branches entering each state.
62 Convolutional Codes

3.10 Distance Properties of

Convolutional Codes

The error probability performance of convolutional codes depends

on their distance properties. We consider here two types of distance.
For hard decision decoding, the decoder operates with binary sym-
bols and the code performance is measured by Hamming distance.
A soft decision decoder receives quantized or analog signals from
the demodulator and the decoding operation is based on Euclidean
distance.
The minimum free distance, denoted by dfree , of a convolutional
code is defined as the minimum Hamming distance between any
two code sequences in the code. Since convolutional codes are lin-
ear, the Hamming distance between two code sequences is equal to
the weight of their modulo-2 sum, which is another code sequence.
Therefore, the minimum free distance is the minimum weight of all
non-zero code sequences. In other words, the all-zero code sequence
can be used as the reference sequence in determining the minimum
free distance.
For example, for the code (2,1), represented by the trellis in
Fig. 3.14, it is clear that the path v = (110111) is at the minimum
Hamming distance from the all-zero path o. Thus, the minimum
free distance is d free = 5.
The minimum free Euclidean distance, denoted by dE ,free , is de-
fined as the minimum Euclidean distance between any two code
sequences. The minimum free Euclidean distance depends both on
the convolutional code trellis structure and the modulation type.
For example if the BPSK modulation is used to map each binary
symbol into a signal from the {-1,1} modulation signal set, the min-
imum free Euclidean distance for the (2,1) code with the trellis in
Fig. 3.14 is dE,free = 2V5.
Weight Distribution of Convolutional Codes 63

3.11 Weight Distribution of Convolu-

tional Codes
The weight distribution of a convolutional code is important for
computing its error performance. We define Ai as the number of
code sequences of weight i in the trellis that diverge from the all-
zero path at a node and then remerge for the first time at a later
node. The set

is called the weight distribution of a convolutional code.

The weight distribution can be computed by modifying the code
state diagram. The modified state diagram is obtained by splitting
the all-zero state So into an initial state, Sin, and a final state,
Sout, with the self-loop around So being removed. Each path in the
state diagram connecting the initial state Sin and the final state
Sout represents a code sequence which diverges from and remerges
with the all-zero state exactly once. The code sequence weight Ai
represents the number of code sequences which diverge from the
all-zero path at the same node and remerge for the first time at
later nodes. In fact, Ai is equal to the number of paths of weight
i in the modified state diagram which connect the initial and final
state.
Let X be the indeterminate associated with the Hamming weight
of the encoded output sequence i, Y the indeterminate associated
with the Hamming weight of the information sequence j, and Z
the indeterminate associated with every branch. Each branch in
the modified state diagram is labelled with a branch gain Xiyi Z.
The modified state diagram with gain labels is called an aug-
mented state diagram.
The weight distribution can be obtained from the generating
function of the augmented state diagram. The augmented state
diagram can be regarded as a signal flow graph and the generat-
ing function can be computed by Mason's rule [12]. Alternatively,
the generating function can be obtained from a set of equations
describing the state transitions in the augmented state diagram.
64 Convolutional Codes

Example 3.10 Consider the (2,1) convolutional code with the

state diagram shown in Fig. 3.12. The augmented state diagram is
illustrated in Fig. 3.15.
XYZ

ou

YZ
Fig. 3.15: Augmented state diagram of Fig. 3.12

The set of equations describing the state transitions in the aug-

mented state diagram are

82 YZ8 1 + X 2YZ8in (3.77)

81 XZS2 +XZ83 (3.78)
83 XYZ82 + XYZ83 (3.79)
8 0ut X 2Z81 (3.80)

Solving for 8 3 from (3.79) we get

8 _ XYZ82
(3.81)
3 - 1-XYZ

Substituting (3.81) into (3.78) we get

XZ82
(3.82)
8 = 1- XYZ
1

Substituting (3.82) into (3.77) we get

8 _ (1- XYZ)X 2YZ8in

(3.83)
2 - 1 - XY Z - XY Z2
Weight Distribution of Convolutional Codes 65

Furthermore, substituting (3.82) into (3.80) we get for Sout

X3Z 2S 2
Sout = 1- XYZ (3.84)

Now, substituting (3.83) into (3.84) we get for the generating func-
tion T(X, Y, Z)

Sout
T ( X,Y,Z ) = (3.85)
Sin 1-XYZ(1+Z)

we can write

T(X, Y,Z) X 5YZ 3 + X6y2(Z4 + Z5)

+X7y3(Z5 + 2Z6 + Z7)
+XBy4(Z6 + 3Z7 + 3ZB+ Z9) + ...
Expanding T(X, Y, Z) in series, we can write

T(X, Y, Z) = X5y Z3 + X6y2 Z4 + (X6y2 + X7y3)Z5

+(2X7y3 + XBy4)Z6
+(X 7y3 + 3XBy4 + X 9y5)Z7
+(3X By4 + 4X9y5 + X 10 y6)ZB + ...

Note that the minimum free distance for the code is 5 and that the
number of code sequences at this distance is A5 = 1. The infor-
mation sequence generating this code sequence has the Hamming
weight of 1 and the code sequence contains three branches. An-
other information sequence of weight 2 produces a code sequence
of weight 6 with four branches and so on.
If we ignore the branch lengths, by setting Z to 1, the generator
function becomes

(3.86)

Furthermore, the weight distribution Ai can be obtained by set-

ting Y = 1.

T(X) = X 5 + 2X 6 + 4X7 + 8X B+ ... (3.87)

66 Convolutional Codes

with A5 = 1, A6 = 2, A7 = 4 and so on.

The total number of nonzero information bits on all paths of
Hamming weight j is given by the partial derivative of T(X, Y, Z)
with respect to Y

BT(X, Y,Z) I (3.88)

BY Y=l,Z=l

That is, on the path with a weight of 5 there is one information bit,
on the path with a weight of 6, there are 4 information bits etc.

3.12 Punctured Convolutional Codes

In certain situations it is required to vary the code rate of a con-
volutional code and keep its trellis structure unchanged. Examples
are unequal error protection (UEP) schemes in which one encoder
and decoder with variable code rates are used. The code rate is
varied by not transmitting certain code symbols, or puncturing the
original code. Another reason for constructing punctured codes is
that their trellises have simpler structure than those for the corre-
sponding rate non-punctured codes.
Punctured codes are (n, k) convolutional codes derived from an
(n, 1) mother convolutional code. We will demonstrate the con-
struction method on an example.
Let us consider the (2,1) encoder shown in Fig. 3.1 with the
generator matrix G(D) = [1 + D2 1 + D + D2] and the trellis
diagram illustrated in Fig. 3.14. If one out of four encoder output
symbols is punctured, the encoder produces three coded symbols
for every two information bits. That is, every leading symbol of the
rate 1/2 code is punctured from every other branch. The new code
is time varying and has a code rate of 2/3.
The trellis diagram of the punctured (3,2) code is shown in Fig.
3.16. Note that X indicates the deleted symbols.
The set of code sequences in this punctured code is identical to
the set of code sequences generated by the 2/3 rate code with the
Punctured Convolutional Codes 67

1/01 1!X1
11

10

01

00
Fig. 3.16: Trellis diagram of a rate 2/3 punctured code produced by
periodically deleting symbols from a rate 1/2 code

generator matrix

G(D) = [ 1+D
o
1+D
D
1
l+D
1 (3.89)

The encoder and the trellis diagram for this code are given in Figs
3.17 and 3.18, respectively. The trellis is more complex than the
trellis for the punctured rate 2/3 code shown in Fig. 3.16 since there
are four paths entering each state rather than two. This leads to
more complex encoding and decoding operation. Thus, punctured
codes have an advantage, particularly for high rate applications.

The erasure of code symbols in the trellis in Fig. 3.16 can be

described by a puncturing table

(3.90)

A zero in the puncturing table means that the code symbol is not
transmitted. In the above example the first symbol in the second
branch is not transmitted.
In general, a rate pi q punctured convolutional code can be con-
structed from an (n, 1) convolutional code by deleting np - q code
68 Convolutional Codes

Fig. 3.17: Encoder for a rate 2/3 code

000
00 111::---------------,,,,.

11
Fig. 3.18: Trellis diagram of a rate 2/3 code

symbols from every np code symbols corresponding to p input in-

formation symbols. The (n,l) code is called the mother code. It is
specified by the generator matrix

(3.91)
Punctured Convolutional Codes 69

where
g(i)(D) = g~i) + g~i) D + ... + g~i) D V
(3.92)
where 1::; i::; nand g1 E {a, I}, l = a···v.
The code rate of the punctured code, Rp, is defined by

R = P
p np - np+ q

The code symbol erasures are represented by the puncturing table

PH PI2 ... PIp

P2I P22 •.. P2
p=[ . .p (3.93)
.. ..
Pnl Pn2 ••• Pnp

with Pij E {a, I} , 1 ::; i ::; nand 1 ::; j ::; p.

As the puncturing is performed periodically every np code sym-
bols, P is called the puncturing period.
The puncturing tables and corresponding free distances for the
punctured codes generated from the (2,1) mother code with the
encoder memory of 6 are shown in Table 3.1. The generator matrix
is
70 Convolutional Codes

Table 3.1: Punctured convolutional codes

punctured code rate puncturing table P dfree
11
2/3 6
10
110
3/4 101
5
1111
4/5 4
1000
11010
5/6 4
10101
111010
6/7 3
100101
1111010
7/8 3
1000101
Bibliography

[1] G. Begin and D. Haccoun, "High Rate Punctured Convo-

lutional Code Structure Properties and Construction Tech-
niques", IEEE Trans. Commun., vol. COM-37, no. 12, Dec.
1989, pp 1813-1825.

[2] Y. Bian, A. Popplewell and J. J. O'Reilly, "New Very High

Rate Punctured Convolutional Codes", Elect. Lett., vol. 30,
no. 14, July 1994, pp. 1119-1120.

[3] J. B. Cain, G. C. Clark and J. M. Geist, "Punctured Con-

volutional Codes of Rate (n - l)/n and Simplified Maximum
Likelihood Decoding", IEEE Trans. Inform. Theory, vol. IT-
25, no. 1, Jan. 1979, pp. 97-100.

[4] D. J. Costello, Jr., "Constructions of Convolutional Codes for

Sequential Decoding", Rep. EE-962, Univ. of Norte Dame,
Norte Dame, IN, Aug 1969.

[5] P. Elias, "Error-free Coding", IRE Trans. Inform. Theory, vol.

IT-4, 1954, pp.29-37.

[6] G. D. Forney, Jr., "Convolutional Codes I: Algebraic Struc-

ture", IEEE Trans. Inform Theory, vol. IT-16, no. 6, Nov.
1970, pp. 720-738, and vol. IT-17, no. 3, May 1971, p. 360.

[7] G. D. Forney, Jr., "Structural Analysis of Convolutional Codes

via Dual Codes", IEEE Trans. Inform. Theory, vol. IT-19, no.
4, July 1973, pp. 512-518.
72 BIBLIOGRAPHY

[8] J. Hagenauer, "Rate-compatible Punctured Convolutional

Codes (RCPC codes) and Their Applications", IEEE Trans.
Commun., vol. COM-36, no. 4, April 1988, pp.389-400.

[9] R. Johannesson, and Z. X. Wan, "A Linear Algebraic Ap-

proach to Minimal Convolutional Encoder", IEEE Trans. In-
form. Theory, vol. IT-39, no. 4, July 1993, pp. 1219-1233.
[10] L. H. C. Lee, "Convolutional Coding: Fundamentals and Ap-
plications", Artech House, 1997.

[11] S. Lin, and D.J. Costello, Jr., "Error Control Coding: Funda-
mentals and Applications", Prentice-Hall, 1983.

[12] S. Mason and Zimmermann H., Electronic Circuits, Signals

and Systems, New York, Wiley, 1960.

[13] J. L. Massey, and M. K. Sain, "Inverses of Linear Sequential

Circuits", IEEE Trans. Camp., vol. 17, no. 4, Apr. 1968, pp.
330-337.
[14] J. E. Porath, "Algorithms for Converting Convolutional Codes
from Feedback to Feedforward and Vice Versa", Elect. Lett.,
vol. 25, no. 15, July 1989, pp. 1008-1009.

[15] A. J. Viterbi and J. K. Omura, Principles of Digital Commu-

nications and Coding, McGraw Hill, 1979.

[16] Y. Yasuda, Y. Kashiki and Y. Hirata, "High Rate Punc-

tured Convolutional Codes for Soft-Decision Viterbi Decod-
ing", IEEE Trans. Commun., vol. COM-32, no. 3, March 1984,
pp. 315-319.
Chapter 4

Turbo Coding Performance

Analysis and Code Design

4.1 Introduction
It is well known that a good trade-off between coding gain and
complexity can be achieved by serial concatenated codes proposed
by Forney [1]. A serial concatenated code is one that applies two
levels of coding, an inner and an outer code linked by an inter-
leaver. This approach has been used in space communications, with
convolutional codes as the inner code and low redundancy Reed-
Solomon codes as the outer code. The primary reason for using
a concatenated code is to achieve a low error rate with an overall
decoding complexity lower than that required for a single code of
the corresponding performance. The low complexity is attained by
decoding each component code separately. As the inner decoder
generates burst errors an interleaver is typically incorporated be-
tween the two codes to decorrelate the received symbols affected by
burst errors. Another application of concatenation is using a band-
width efficient trellis code as an inner code [2] or concatenating two
convolutional codes [3]. In decoding these concatenated codes, the
inner decoder may use a soft-input/soft-output decoding algorithm
to produce soft decisions for the outer decoder.
Turbo codes exploit a similar idea of connecting two codes and
separating them by an interleaver [4]. The difference between

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
74 Turbo Coding Performance Analysis and Code Design

turbo and serial concatenated codes is that in turbo codes two

identical systematic component codes are connected in parallel.
The information bits for the second code are not transmitted thus
increasing the code rate relative to a corresponding serial concate-
nated code. The primary reason for using a long interleaver in turbo
coding is to generate a concatenated code with large block length
which leads to a large coding gain. The turbo decoder consists of
two concatenated decoders of the component codes separated by
the same interleaver. The component decoders are based on a max-
imum a posteriori (MAP) probability algorithm or a soft output
Viterbi algorithm (SOVA) generating a weighted soft estimate of
the input sequence. The iterative process performs information ex-
change between the two component decoders. By increasing the
number of iterations in turbo decoding, a bit error probability as
low as 10- 5 _10- 7 can be achieved at a signal-to-noise ratio close to
the Shannon capacity limit.
In this chapter, we consider turbo and serial concatenated con-
volutional codes (SCCC). We present analytical upper bounds of
code performance and propose design criteria for turbo and serial
concatenated convolutional codes.

4.2 Turbo Coding

4.2.1 A Turbo Encoder

A turbo encoder is formed by parallel concatenation of two recursive
systematic convolutional (RSC) encoders separated by a random
interleaver [4].
The encoder structure is called parallel concatenation because
the two encoders operate on the same set of input bits, rather than
one encoding the output of the other. Thus turbo codes are also re-
ferred to as parallel concatenated convolutional codes (PCCC) [13].
A block diagram of a rate 1/3 turbo encoder is shown in Fig. 4.l.
The generator matrix of a rate 1/2 component RSC code can
Thrbo Coding 75

c Vo

RSC
Encoder 1

IInterleaver I
c RSC
Encoder 2

Fig. 4.1: A turbo encoder

be represented as
G(D) = [1 (4.1)

where go(D) and gl (D) are a feedback and feedforward polynomial

with degree v, respectively. In the encoder, the same information
sequence is encoded twice but in a different order. The first encoder
operates directly on the input sequence, denoted by c, of length N.
The first component encoder has two outputs. The first output,
denoted by vo, is equal to the input sequence since the encoder is
systematic. The other output is the parity check sequence, denoted
by VI. The interleaved information sequence at the input of the
second encoder is denoted by c. Only the parity check sequence of
the second encoder, denoted by V2, is transmitted. The information
sequence Vo and the parity check sequences of the two encoders, VI
and V2, are multiplexed to generate the turbo code sequence. The
overall code rate is 1/3.

Example 4.1 A rate 1/3 turbo code encoder

A rate 1/3 turbo code encoder, based on a (2,1,4) RSC code, is
shown in Fig. 4.2. The component codes are identical (2, 1,4) RSC
76 Turbo Coding Performance Analysis and Code Design

codes with code rate 1/2 and memory order v = 4 (the number of
states is M s =16). The generator matrix of the RSC code is given
by
G(D) = [1, 1 + D + D2 + D3 + D4 (4.2)

Let us assume that the input sequence is

c = (1011001) (4.3)
Then the two output sequences of the first component encoder are

Vo (1011001)
VI (1110001) (4.4)
We assume that the interleaver permutes the information sequence
to
c = (1101010) (4.5)
The parity check sequence of the second component encoder is

V2 = (1000000) (4.6)
The turbo code sequence is given by

V= (111,010,110,100,000,000,110) (4.7)

4.2.2 Interleaving
The interleaver in turbo coding is a pseudo-random block scrambler
defined by a permutation of N elements with no repetitions.
The first role of the interleaver is to generate a long block code
from small memory convolutional codes. Secondly, it decorrelates
the inputs to the two decoders so that an iterative suboptimum
decoding algorithm based on information exchange between the two
component decoders can be applied. If the input sequences to the
two component decoders are decorrelated there is a high probability
that after correction of some of the errors in one decoder some of the
remaining errors should become correctable in the second decoder.
Turbo Coding 77
c ~~------------------------------------------~Vo

1...-_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ VI

~-----------------V2

Fig. 4.2: A rate 1/3 turbo encoder

In a pseudo-random interleaver a block of N input bits is read

into the interleaver and read out pseudo-randomly. The pseudo-
random interleaving pattern must be available at the decoder as
well.

4.2.3 Trellis Termination

A turbo encoder operating on each block of information bits can
be regarded as a block encoder. Since the component codes are
recursive, it is not possible to terminate the trellis by transmitting
v zero tail bits. Trellis termination means driving the encoder to
the all-zero state. This is required at the end of each block to make
sure that the initial state for the next block is the all-zero state.
78 Turbo Coding Performance Analysis and Code Design

The tail bits depend on the state of the component encoder

after N information bits. A simple solution to this problem is shown
in Fig. 4.3 [6]. A switch in each parallel component encoder is in
position "A" for the first N clock cycles and in position "B" for 11
additional cycles. This will drive the encoder to the all-zero state.
Trellis termination is based on setting the input "at" to the shift
register to zero. This will flush the register with zeros after 11 shifts.
r--~I------{+: +: +:r+-------,

~--------------------~+:r--.

Fig. 4.3: Trellis termination

With a pseudo-random interleaver it is highly unlikely that this

method will terminate both encoders simultaneously with only 11
tail bits since the codes are recursive. Usually, only the first com-
ponent encoder is forced to return to the all-zero state. The per-
formance degradation produced by an unknown final state of the
second encoder is negligible for a large interleaver size N [7].
A rate 1/3 turbo code with trellis termination is equivalent to a
(3(N + 11), N) linear systematic block code.
In the remainder of the book we will assume that the first en-
coder is forced to the all-zero state. Special interleaver structures
that can drive both encoders to the all-zero state are discussed in
Chapter 7 and in [8]- [11].

4.2.4 High Rate Turbo Codes

In general, the two component codes and their rates are not nec-
essarily the same. If the code rates of two component codes are
denoted by Rl and R 2, respectively, the overall turbo code rate R
will satisfy
1 1 1
-=-+--1 (4.8)
R Rl R2
Thrbo Coding 79

Various overall code rates such as 1/2, 2/3, 3/4, 5/6 and so on,
can be obtained by puncturing the rate 1/3 turbo encoder shown
in Fig. 4.l.
When puncturing is considered, some output bits of vo, VI and
V2 are deleted according to a chosen pattern defined by a puncturing
matrix P. For instance, a rate 1/2 turbo code can be obtained by
puncturing a rate 1/3 turbo code. The commonly used puncturing

[i n
matrix is given by

(4.9)
p=

where the puncturing period is 2. According to the puncturing

matrix, the parity check digits from the two component encoders
are alternately deleted. The punctured turbo code symbol at a
given time consists of an information digit followed by a parity
check digit which is alternately obtained from the first and the
second component encoders.
An alternative approach in high rate turbo code design is to
employ high rate component codes [12] [15].

Example 4.2 A rate 1/2 turbo code encoder

A rate 1/2 turbo code based on a (3,2,4) RSC code is shown in
Fig. 4.4. The generator matrix of the RSC code is given by

(4.10)

The component codes are two identical (3,2,4) RSC codes with
code rate 2/3 and memory order v = 4. The information sequences
Co and CI are encoded by the first encoder to generate the first
parity check sequence V2. The interleaved sequences Co and CI are
encoded by the second encoder to produce the second parity check
sequence V3. Then the information sequences Vo and Vi, and the
parity check sequences of the two component encoders V2 and Va,
are multiplexed to generate the code sequence for a rate 1/2 turbo
encoder.
80 Turbo Coding Performance Analysis and Code Design

COI~~--~------~~------~----------------~----VO

Va

Fig. 4.4: A rate 1/2 turbo encoder

4.3 Performance Upper Bounds of

Turbo Codes
The weight distribution of a turbo code can be used to compute
error performance bounds for maximum likelihood decoding. As a
turbo code can be represented by an equivalent block code if the
component encoders are forced to the all-zero state at the end of
each block, we first consider the weight distribution of its equivalent
block code in this section. Subsequently, we will apply it in the
calculation of bit error probability bounds on an additive white
Gaussian noise channel.

4.3.1 Conditional WEF's of Average Turbo

Codes
In order to evaluate turbo code error performance upper bounds,
we need to calculate the conditional WEF's of the equivalent block
Performance Upper Bounds of Turbo Codes 81

code.
A turbo codeword consists of the input information sequence
and the parity check sequences from the first and second component
encoders. Let w be the weight of an input information sequence,
and Zl and Z2 be the weights of the first and second parity check
sequences, respectively. The weight of the corresponding codeword
will be d = w + Zl + Z2. If the conditional WEF's of the com-
ponent code are known, it can be used to determine the weight
of the first parity check sequence. However the weight of the sec-
ond parity check sequence will not only depend on the weight of
the input information sequence, but also on how the information
bits have been permuted by the interleaver. The turbo code con-
ditional WEF's cannot be uniquely determined because of various
interleaver structures.
A simple solution to this problem is obtained by using a uni-
form interleaver, which is based on a probabilistic analysis of the
ensemble of all interleavers [13].
A uniform interleaver is a probabilistic device which maps a
given input sequence of length N and weight w into its all distinct
( ~) permutations with equal probability of 1/ ( ~ ).
For a uniform interleaver of size N, there are N! possible per-
mutations of binary sequences with length N, each of which has a
probability 1/ N!. If the weight of an input sequence is w, there
will be w!(N - w)! permutations which generate the same output
sequence. Thus the probability for each distinct permutation is

w!(N-w)! 1
(4.11)
N!

The uniform interleaver makes independent weight distributions

of the parity check sequences generated by the first and second en-
coders. It enables the computation of the weight distribution of an
average turbo code from the weight distribution of the component
codes.
The conditional WEF's of the equivalent block code of a turbo
82 Turbo Coding Performance Analysis and Code Design

code with a uniform interleaver is given by

(4.12)

where A~(Z) and A~(Z) are the conditional WEF's of the equiv-
alent block codes of the first and second component codes, respec-
tively. Since the conditional WEF's of the turbo code are averaged
over the ensemble of interleavers of length N, they are also called
the conditional WEF's of the "average" turbo code. If two iden-
tical component encoders are employed in a turbo encoder, it is
appropriate to let

A~ (Z) = A~ (Z) = A~(Z) (4.13)

4.3.2 Conditional WEF's of Component Codes

The conditional WEF's of the equivalent block code of a turbo
component code can be obtained from the code transfer function.
For a component recursive systematic convolutional code, the code
transfer function generated from the code augmented state diagram
is given by [13J

T(W,Z,L) = L:Tw,z,lwwzzL I (4.14)

w,z,l
where Tw,z,l is the number of error paths in the code trellis with
an input sequence of weight w, parity check sequence of weight z
and error path length l. A single error path is a path that diverges
from the all-zero path in the code trellis and then remerges with the
all-zero path within a finite number of branches. For the equivalent
block code, an error path associated with an error event consists of
all possible combinations of all single error paths. We call such an
error path a compound error path.
Let us consider a compound error path as shown in Fig. 4.5.
The compound error path concatenates n single error paths with a
Performance Upper Bounds of Turbo Codes 83

Error Event 1 2 n
.. Interleaver Size N

Fig. 4.5: A compound error path

total input weight of w, parity check weight of z, and error path

length 1 within the block length N, equal to the interleaver size.
It is clear that suitable sequences of zeros will connect these single
error events so that the total length of the path equals N. Since the
positions of the zero sequences do not affect the information and
parity weight profile of the compound error path, for a combination
of n error paths with total length l, there are

(4.15)

ways to obtain the same conditional WEF's.

Let T(W, Z, L, E) be the modified transfer function of the equiv-
alent block code of the component code which enumerates all pos-
sible combinations of single error paths in the trellis. The modified
transfer function can be represented as
T(W, Z, L, E) = L Tw,z,l,n WW ZZ Ll En (4.16)
w,z,l,n
where Tw,z,l,n is the number of error paths in the trellis produced by
an information sequence of weight w, with parity check weight z,
path length 1 and consisting of n concatenated single error paths.
The conditional WEF A~(Z) of the equivalent block code of the
component convolutional code is given by
A~(Z) = L Aw,z ZZ (4.17)
z
where
" ( N -nl + n ) T,w,z,l,n
ACw,z -_L...... (4.18)
l,n
84 Turbo Coding Performance Analysis and Code Design

For N much larger than the memory of the convolutional code,

the conditional WEF of the equivalent block code can be approxi-
mated by

(4.19)

where n max is the largest number of single error paths concatenated

to produce a compound error path with an input sequence of weight
w. Tw,z,n is the number of error paths with information weight w,
parity check weight z and number of concatenated error paths n.
It is given by
T:w,z,n -- ""
L...J T:w,z,n,l· (4.20)
I

Let
A(w,Z,n) = LTw,z,nZz (4.21)
z

be the redundancy weight enumerating function of the component

code generated by concatenating n single error paths with total
information weight w. The approximate conditional WEF of the
equivalent block code is thus

nmax ( N ) (4.22)
A~(Z) :;:::j ~ n A(w, Z, n)

4.3.3 Average Upper Bounds on Bit Error Prob-

ability
Inserting (4.22) into (4.12) we get for the conditional WEF of the
turbo code

A~(Z) . A~(Z)

(:)
Aw(Z)
Performance Upper Bounds of Turbo Codes 85

where

(4.23)

is the contribution of each compound error path to the conditional

WEF. Using the approximation of the binomial coefficient

( ~ ) ~ ~~ (4.24)

the conditional WEF of the turbo code can be rewritten as

L L
nmax nmax ,
Aw(Z) = ,w.,. N n l+ n 2- w
nl=l n2=1 nl· . n2·
. A(w, Z,nl)A(w, Z, n2) (4.25)
It is clear that, for large N, the conditional WEF is dominated by
the terms which have the highest power of N. This can be achieved
if nl = n2 = n max • Thus (4.25) is further approximated by
A w (Z) _·_N
rv
rv
w'
,2 2nmax- W [A(w ' Z ,n max )]2 (4.26)
n max •
Considering the bit error probability upper bounds (2.29) and
(2.30), we get for the union upper bound

LN w.w'. N2nmax-W-l
W=Wmin 2· n max- '2
. WW[A(w, Z, n max )]2Iw=z=e -R~ (4.27)
and the tighter union upper bound

Pb(e) _< '"'

L...J Q
N ( 2d·
mm
Eb)
RJ\T- edmin R~
NO
W •
_ _•
w',2 N 2nmax-w - 1
w=wmin 1 V0 n max •

. WW[A(w, Z, nmax)]2Iw=z=e -R~ (4.28)

86 Thrbo Coding Performance Analysis and Code Design

where Wmin is the minimum information weight in error paths of

the component codes. Since the uniform interleaver is used in the
performance analysis, these upper bounds are also called average
upper bounds.

Example 4.3 Thrbo code bit error probability average upper

bound.
Consider a rate 1/3 turbo code of memory order 2. The gener-
ator matrix of the turbo component code is given by

G(D} = [1, 1 +D2

1 +D+D2
1 (4.29)

The average upper bound on the bit error probability of the turbo
code with interleaver size 500 is evaluated. The result is shown in
Fig. 4.6.

t·
I
I Duman&Salehi Bound
I Union Upper Bound
I
I
'I
\
I
10-2 \
: \

I
\
10-4 \
\

10-<lL---'---'-:c_--,J::--~---:":------:'-:---:::'=---:'-=--~
12 1.4 1.6 18 2 2.2 2.4 2.6 2.8 3
EbiNo (dB)

Fig. 4.6: Bit error probability upper bounds for a turbo code with in-
terleaver size 500

Note that the average upper bound is based on union bound

techniques. It diverges when the ratio of information bit energy
Performance Upper Bounds of Turbo Codes 87

to noise power spectral density E b/ No drops below the threshold

determined by the computation cutoff rate R o, especially for large
interleaver sizes. For a turbo code with rate R = 1/3, this threshold
for E b / No is given by [17]

1
--In(2 1 R
- - 1)
R
2.03dB (4.30)

For E b / No above the computation cutoff rate threshold, the aver-

age upper bound can be used to estimate the code performance. In
this region, evaluation of the bound requires only the first several
terms of the distance spectrum in the summation. For E b / No below
the threshold, the upper bounds will be useless for accurate per-
formance evaluations for turbo codes with large interleaver sizes.
To overcome the divergence problem of the average upper bound,
several improved bounding techniques have been proposed. Tighter
bounds for a larger range of E b / No extending below the cutoff rate
threshold have been derived by Duman and Salehi in [18] and by
Viterbi in [19] based on variants of Gallager's exponential bound.
Sason and Shamai proposed ensemble upper bounds based on the
tangential-sphere bound, where the ensemble is generated by a uni-
form choice of both the interleaver and the component codes [20].
In Fig. 4.6 Duman and Salehi's upper bound is also shown.

4.3.4 Interleaving Performance Gain

Using the analysis tools developed in the previous sections, we now
investigate the role played by recursive systematic convolutional
codes as component codes in turbo coding.
We begin with comparing error performance of recursive sys-
tematic convolutional codes and nonrecursive convolutional (NRC)
or feed forward convolutional codes. Then, through an analytical
upper-bounding technique, we will show that only a turbo code
with RSC component codes can produce an interleaving perfor-
mance gain, which is crucial for overall performance gain.
88 TUrbo Coding Performance Analysis and Code Design

In order to compare the performance of RSC and NRC codes,

we define two sets of error coefficients Ad and Wd for convolu-
tional codes which can be obtained from the code transfer function
T(X, Y) [21J. Let
00

T(X,y)IY=l L
d=dfree
Ad Xd

BT(X, Y) I
00

- L Wd Xd (4.31 )
BY Y=l d=dfree

where dfree is the code free distance, Ad is the number of code se-
quence with weight d and Wd is the total weight of information
sequences which generate code sequences of weight d. The two sets
of error coefficients Ad and Wd can be used to derive the code word
error probability and bit error probability upper bounds, respec-
tively.
In [22], various RSC and NRC codes have been investigated and
their error coefficients compared. It is shown that the set of error
coefficients Ad is the same for RSC and NRC codes, since both RSC
and NRC encoders generate the same set of code sequences. This
results in the same word error probability for both codes. However,
the set of Wd for RSC codes is different from that of NRC codes.
This is due to a different input-output weight correspondence be-
tween RSC and NRC encoders, which leads to different bit error
probability performance. In general, for code rates R ::; 2/3, the
first two coefficients wdfree and wdfree+1 of RSC codes are larger than
those of NRC codes. Therefore, at high SNR's, the bit error proba-
bility performance of NSC codes is a little better than that of RSC
codes. On the other hand, with increasing weight d, the value of Wd
for RSC codes grows more slowly than that for NRC codes. Thus,
at low SNR's, the bit error probability performance of RSC codes
is superior to that of NRC codes.
The difference in the input-output weight correspondence be-
tween RSC and NRC encoders can also be explained by the differ-
ence in the minimum weight of information sequences generating a
finite weight code sequence.
For NRC codes any information sequence of weight one will
generate a finite weight code sequence. On the other hand RSC
Performance Upper Bounds of Turbo Codes 89

codes may map finite weight information sequences into infinite

weight code sequences. The generator matrix of RSC codes contains
at least one entry which is a rational fraction of the unit delay
operator D, such as gl (D)/go(D). The encoder thus generates a
sequence of infinite weight in response to a finite weight information
sequence, except if the polynomial representation of the information
sequence is a multiple of the rational function denominator go(D).
Since the rational function denominator is of the form 1 + ... + DV,
where v is the encoder memory, it cannot divide a polynomial of
the form DJ for any j. Therefore there are no finite weight code
sequences generated by weight one information sequences in these
codes. The minimum weight information sequence generating a
finite weight code sequence is two.
If NRC encoders are employed in turbo coding, the minimum
information weight in error paths is Wmin = 1. The largest number
of single error paths in a compound error path with an information
weight W is n max = w. In this case the conditional WEF is given
by
A(w, Z, w) = [A(l, Z, 1)t (4.32)
From (4.27), the code bit error probability is upper-bounded by

(4.33)

The above upper bound shows that, for NRC component codes,
error paths with information weight w = 1 and their compound er-
ror paths have a dominant effect on the turbo code bit error prob-
ability. At high SNR's the single error paths with w = 1 have the
dominant contributions to the error performance. In this case, the
bit error probability is independent of N, as the factor Nw-l = l.
Thus no performance gain can be achieved by interleaving for these
codes. The same conclusion applies to turbo codes with component
nonsystematic block codes [13].
In an RSC encoder, the minimum information weight for error
paths is Wmin = 2. The largest number of single error paths in
a compound error path with an information weight W is n max =
lw /2 J, where lx J means "integer part of x". We now separate the
90 Turbo Coding Performance Analysis and Code Design

analysis of terms in the sum in (4.27) into odd and even weight w.
For odd values of w, i.e., w = 2i + 1, a term in the sum in (4.27)
can be expressed as

(2i + 1~(i + 1) ( 2i t 1 ) N- 2W 2i +1 [A(2i + 1, Z, i)]2 (4.34)

whereas, for even values of w = 2i, this term becomes

i ( ~i ) N- 1 W 2i [A(2i, Z, i)]2

= i ( ~i ) N- 1 W 2i [A(2, Z, 1)]2i (4.35)

Comparing (4.34) and (4.35), it is clear that, for large N, the terms
with odd ware negligible, since they depend on N- 2 while the
terms with even w depend on N- 1 . Hence, for turbo codes with
RSC component codes, the bit error probability is upper-bounded
by

P,(e) :S ~ i(:i ) N- 1W" [A(2, Z, 1) j'l=z=, _.;: (4.36)

From the upper bound it can be seen that the performance of a

turbo code with RSC component codes is affected by the interleaver
size N. By increasing the interleaver size N times, the bit error
probability is reduced by a factor N. This is called the interleaving
performance gain [13].
To illustrate the interleaving performance gain, consider the
turbo code in Example 4.3. The bit error probability average upper
bounds for the turbo code with various interleaver sizes of 128, 256
and 512 are shown in Fig. 4.7. It is clear that the asymptotic bit
error probability goes down with increasing the interleaver size.

4.3.5 Effective Free Distance

For turbo codes with RSC component encoders, the bit error proba-
bility upper bound shows that the error paths with minimum infor-
mation weight Wmin = 2 and their compound error paths dominate
Performance Upper Bounds of'IUrbo Codes 91

10·
\
I
, '

,, ,
N=128
I N=256
10-'
:\ N=512
I 'j
,
10-2
I
I ,
,I,
I-

"
I
ffi 10-"
III
,I ,
,
\ \
\
\ \
10-' \
I
\
\
, ...
\ ...

- -, --- --- ---

\
10--

10-6
1,5 2 25 3
EblNo (dB)

Fig. 4.7: Bit error probability upper bounds for a turbo code with various
interleaver sizes

the turbo code bit error probability performance. Let Zmin denote
the lowest weight of the parity check sequence in error paths of
RSC component encoders generated by an information sequence
with weight 2. The weight enumerating function of single error
paths with information weight 2 for RSC component codes is given
by [13]
A(2, Z, 1) ZZmin + Z2Z min- 2 + Z3Zmin- 4 + ...
ZZmin
(4.37)

Substituting (4.37) into (4.36) and setting W = Z = H, the turbo

code bit error probability upper bound can be rewritten as

R (e) <Llf j i
'" (2·) N-
't
(H2+2Zmin)i
1 • ---.:.---..:......-;:-:-
I (4.38)
b - ~ i (1 _ HZmin-2)2i H=e -R~

The above bound clearly shows that the most important param-
eter which has a significant influence upon turbo code performance
92 Turbo Coding Performance Analysis and Code Design

is Zmin. The effective free distance of a turbo code is defined as [23]

[13]
dfree,eff = 2 + 2Zmin (4.39)

It is the lowest weight of the turbo code sequence generated by an

information sequence of weight 2 with a uniform interleaver. The
effective free distance plays a role similar to that of free distance
for convolutional codes.
For a turbo code with a uniform interleaver, the free distance
dominates the error performance at high SNR's. The free distance
may be smaller than the effective free distance dfree,eff' In this case
the code sequence associated with the free distance is generated by
an information sequence of weight w ~ 3. However, since a random
interleaver is more likely to break information sequences with a
high weight than a low weight, it can make the error coefficient
shown in (4.41) of the free distance negligibly small. Therefore, for
a turbo code with a random interleaver, the bit error probability at
high SNR's is mainly determined by the code effective free distance.
Effectively, this means that a random interleaver will break error
paths corresponding to the free distance of uniform interleavers, so
that the free distance becomes equal to the effective free distance.
From the above discussion we can conclude that the component
encoders for turbo codes must be recursive, and that they should be
chosen to maximize Zmin and thus the effective free distance dfree,eff'
especially when the turbo code works in the region of high SNR's.

4.4 Turbo Code Performance Evalua-

tion
In previous sections, turbo code performance is discussed based on
the asymptotic bounds of the bit error probability. We now evaluate
turbo code performance based on distance spectrum and its average
upper bounds.
As in (2.26) the turbo code bit error probability upper bound
Thrbo Code Performance Evaluation 93

can be expressed based on the code distance spectrum by

(4.40)

where the set of all pairs of (d, Ed) is the code distance spectrum.
For turbo codes, the error coefficients Ed can be represented by
w
L N . Aw,z
d=w+z

(4.41)

where A~ z is the IRWEF coefficient for the component code and is

given by (4.18).
For performance evaluation and comparison, we consider two
turbo codes, denoted by TC1 and TC2, with generator matrices
G 1 (D) and G 2 (D), respectively. The generator matrices are given
by

G1(D) [1,
= 1 + D2 ]
1 + D+D2

G (D) = [1,
1 +D +D2]
2 (4.42)
1 +D2

The turbo encoders are shown in Figs. 4.8 and 4.9, respectively.
In the performance evaluation, we consider interleavers with
small sizes. This is due to the performance upper bound diverging
severely from the actual bit error probability obtained by simulation
in the region of small SNR's for turbo codes with large interleaver
sizes. Using the transfer function method, the distance spectra of
the component code of TC1 and the turbo code TC1 are calculated.
The results for interleaver sizes of 20 and 50 are shown in Fig.
4.10. The bit error probability upper bounds obtained from (4.40)
are plotted in Fig. 4.11. It is apparent from Fig. 4.10 that the
shape of the distance spectrum of the turbo code is very similar
to the component code. However, for small distance values, the
94 Turbo Coding Performance Analysis and Code Design

c ------~------------------------------------~Vo

L..-_ _ _ _ _ _ +-I+ Vl

L - -_ _ _ _ _ _~+ V2

Fig. 4.8: Turbo encoder TCl

c ------~----------------------------------~Vo

I----Vl

Fig. 4.9: Turbo encoder TC2

error coefficients of the turbo code are reduced by a factor between

10- 3 to 10- 5 for interleaver size 50 and between 10- 2 to 10- 3 for
interleaver size 20, relative to the component code. This results in
significant performance improvements, as shown in Fig. 4.11. This
is consistent with the observation in the previous section that the
Turbo Code Performance Evaluation 95

interleaver gain is proportional to the 1/N.

1012i---.------,----.------;:==::::I======:c=====::;l
Component Code. N=50
Component Code. N=20
10'· Turbo Code, N=50
Turbo Code, N=20
10·
,
,
10· ,
,
/

dl
5iU 10' ... ,
~
/
I
~ 10' I
g 1 '1\
W I 1\ I
10· \/'I',
10-2 " ,
10-<

10'"
0 20 40 60 80 100 120
Hamming Distance d

Fig. 4.10: Distance spectra for component code of TCI and turbo code
TCI with interleaver sizes of 20 and 50

One may notice also that the distance spectrum is approxi-

mately symmetrical with respect to the middle distance spectral
line. Since the error function decreases with increasing Hamming
distance, only the distance spectral lines on the left side are impor-
tant for the error performance.
In computing the union upper bound, each term in the sum on
the right-hand side of (4.40) depends on the corresponding distance
spectral line. The contribution of the ith term to the overall bit
error probability, denoted by Pi(e) , can be written as

Pi(e) = Bd,Q (/2diR~:) (4.43)

Its relative contribution to the total bit error probability can be

represented as
(4.44)
96 Turbo Coding Performance Analysis and Code Design

10-' E--~----''------r---,:::r====:::I=====r;====:::;-l

\
--.-
---ir-

-9-
ComponentCode,N=50
Component Code, N=20
Turbo Code, N=50
Turbo Code, N=20
10-' \"
\ ,,
\
\
,,
\
\
6l ,,
10-3
\
\ . ,
....
\
a:
'!\ , "' ....
"0,
~
....
,, ,,
, ,
10'"
" ",

10'"
" ....
",
",
'Q.
,,
" ",
, " 'e
'-- ",
", ,
10'"
1 2 3 4 5 6 7 8
EbINo (dB)

Fig. 4.11: Bit error probability upper bounds for component code of
TCl and turbo code TCl with interleaver sizes of 20 and 50

For turbo code TC1, the relative contribution of each distance

spectral line to the overall bit error probability is shown in Fig.
4.12 for interleaver size 20 and in Fig. 4.13 for interleaver size 50.
From Figs. 4.12 and 4.13, it is clear that the bit error probability
at low SNR's is dominated by a number of medium weight spectral
line error coefficients B d . The Hamming distances of these spectral
lines are between 10 to 20 for interleaver size 20 and between 15
to 35 for interleaver size 50. At medium to high SNR's, the first
several spectral lines have a significant effect on bit error probabil-
ity. At very high SNR's, the free distance dfree determines the error
performance. So the turbo code error performance is determined
by a number of the medium distance spectral lines for low SNR's
and the first several spectral lines for high SNR's.
The effect of distance spectrum on error probability can be ex-
plained by comparing the distance spectra of TCI with interleaver
sizes of 20 and 50, shown in Fig. 4.10. In the low to medium region
of distances, up to 30, error coefficients for the code with N = 50
Turbo Code Performance Evaluation 97

are lower than those for the code with N = 20. However, for large
distances, the error coefficients for the code with N = 50 are higher.
In spite of this, the code with N = 50 has better performance than
the code with N = 20 in the medium to high SNR region as shown
in Fig. 4.11. This property remains invariant for various codes.
Consequently, we can conclude that only error coefficients at low to
medium distances are significant for error performance. We call the
part of the spectrum that gives a considerable contribution to the
error probability the significant spectral lines. Significant spectral
lines are shown by bold thick lines in Fig. 4.10.

.
0.45
/
- - d=7 Bd=O.04 (
/
0.4 - -€)- d=8 Bd=O.01
--€)- d=9 Bd=O.22
/
- -8- d=10 Bd=O.26 /.
0.35
-+-- d=16 Bd=8.30 /
--<3-- d=17 Bd=12.35 /
/
ffi 0.3
--e-- d=18 Bd=25.04 I
III ----8-- d=19 Bd=37.02
g
'"i5 0.25
~
E
8 02

.!
~ 0.15

0.1

3.5 4 4.5 5

Fig. 4.12: Relative contributions of various distance spectral lines to

overall bit error probability for turbo code Tel with interleaver size 20

For performance comparison, we show the distance spectra and

the bit error probability upper bounds of turbo codes TC1 and
TC2 in Figs. 4.14 and 4.15. From these figures, it can be seen
that the error coefficients of TC1 at the low to medium Hamming
distances are much smaller than those of TC2. Therefore, TC1
will have a superior error performance. With the increase of the
98 Turbo Coding Performance Analysis and Code Design

045

-*- d=7 Bd=7.05e-3

0.4 -9-
-+-
d=8 Bd=7.35e-4
d=9Bd=O.04
/ •
-,,- d=10 Bd=O.08
/
/
/

035 -8- d=11 Bd=O.12 /

-~- d=12 Bd=O.18

-
/

D:
-A- d=13 Bd=O.27 /
/

w 0.3 -'11-
ea d=14 Bd=O.39 /

..!5
9
0.25 -e--
d=28 Bd=3.18e+2
d=29 Bd=6.11 e+2
/
/

- -+
-e--
E
·c ~
d=30 Bd=1.05e+3
d=31 Bd=1.72e+3
C
0
0 0.2 ~ d=32 Bd=2.93e+3

j
£015

0.1

0.05

0 0.5 1.5 2 2.5 3 3.5 4 4.5 5

EbiNo (dB)

Fig. 4.13: Relative contributions of various distance spectral lines to

overall bit error probability for turbo code Tel with interleaver size 50

interleaver size, the error performance improvement achieved by

TC1 over TC2 is increased. Fig. 4.14 also shows that the relative
distance spectrum properties of various codes of a given memory
order remain invariant with increasing interleaver size.
From these figures it can be observed that the impact of error
coefficient Bd on the code error performance is much higher than
that of free distance dfree at low and medium SNR's. If a code works
at medium and low SNR's, it is desirable to have Bd minimized
rather than d free maximized.

4.5 Turbo Code Design

In previous sections we have shown that the distance spectrum
and the effective free distance affect the turbo code error perfor-
mance considerably. In particular, the average bit error probability
'llirbo Code Design 99

10'°,.----.,----,----,----.-------,-----.-----,

TCl N=20
/
\.
TC2 N=20
\ TCl N=50
\ TC2 N=50

/
- I
... \.
·1
I I

I',"
I I
I I
I
I / I
I /
/. I I"
I"
,
~ /
~,

"

10-''----.1...----'-----'-----'-----'------'------'
o 20 40 60 80 100 120 140
HammIng DIstance d

Fig. 4.14: Distance spectra for turbo codes TC1 and TC2 with inter-
leaver sizes of 20 and 50

of turbo codes at low SNR's is dominated by error coefficients of

medium weight distance spectral lines. On the other hand, turbo
code performance at high SNR's is determined by the distances of
the first several spectral lines. When turbo code interleaver size
N is much larger than the code memory v, the code effective free
distance mainly dominates the code performance at high SNR's.
In this section we consider design of turbo codes with good error
performance. We use the code distance spectrum and the code
effective free distance as the design criteria to find the optimum
component codes for a given memory. Obviously the code design
criteria depend on the region of SNR's where the code is used. A
turbo code performing well in the region of high SNR's may not
perform well in the region of low SNR's, or vice versa. Thus, good
turbo codes should be designed according to the range of SNR's
where the code is used. The proper threshold values for SNR's
depend on the code rate, the memory order and the interleaver
size. The bit error probability corresponding to high SNR's is below
100 Turbo Coding Performance Analysis and Code Design

,,
... --- TC1 N=20
...~ - .. - TC2 N=20
" " ,'.,. --e-- TC1 N=50
- -e - TC2 N=50
, ' ...
~

10-4

10~

1 0 - 6 ' - - - - - - - ' - - - ' - - - - - - - ' - - - - '_ _- - L - _ - - '_ _-'-_---II>

1 15 2 2.5 3 35 4 4.5 5
EblNo(dB)

Fig. 4.15: Bit error probability upper bounds for turbo codes TC1 and
TC2 with interleaver sizes of 20 and 50

10- 5 , and the bit error probability corresponding to low SNR's is

in the range of 10- 1 10- 5 . f'.J

4.5.1 Turbo Code Design at High SNR's

Maximizing the code effective free distance can be used as the de-
sign criterion for constructing good turbo codes at high SNR's when
the interleaver size N becomes large [14]. The design objective is
equivalent to maximizing Zmin of the component recursive convolu-
tional codes.
It is shown by Benedetto and Montorsi [14] and Divsalar and
Pollara [12] that the design objective can be achieved if a primitive
feedback polynomial is used in the RSC component encoders.
For a rate lin RSC code with memory 1/ and generator matrix

(4.45)
Turbo Code Design 101

the following upper bound for Zmin holds

Zmin ::; (n - 1) (2 V
- 1 + 2) (4.46)

When go(D) is a primitive polynomial of degree v and gi(D), 1 ::;

i ::; n - 1, is any monic polynomial of degree v, gi (D) =1= go (D),
Zmin can achieve its upper bound [14] [12].
To illustrate why we choose go(D) as a primitive polynomial,
for simplicity, we limit our discussion to the case of a component
RSC code of rate 1/2 with generator matrix

G(D) = [1, (4.47)

The extension to general codes is straightforward.

Let 1 + Dr be the shortest input sequence of weight 2 that gener-
ates a finite-length code sequence. The code parity check sequence
is given by
(4.48)

Since gl(D) and go(D) are relatively prime, the input sequence
1 + Dr must be a multiple of go(D) and periodic with a period of
I. Increasing the period I will increase the length of the shortest
code sequence with input weight 2. Intuitively, this will result in
increasing weight of the code sequence. For polynomial go(D) with
degree v, any polynomial divisible by go(D) is periodic with period
I ::; 2V - 1. The maximum period is 2V - 1, which is obtained
when go(D) is a primitive polynomial. The corresponding encoder
is generated by a maximal length linear feedback shift register with
degree v [24]. In this case, the parity check sequence weight depends
only on the primitive feedback polynomial and is independent of the
polynomial gl(D).
According to this design criterion, Benedetto and Montorsi found
the best RSC component codes with rate 1/2 and memory order
from one to five by computer search. The code search procedure
can be summarized as follows [14]

1. Choose go(D) to be a primitive polynomial of degree v.

102 Turbo Coding Performance Analysis and Code Design

2. Choose gl(D) to be a polynomial of degree lJ, where go(D)

and gl (D) are relatively prime.

3. Evaluate the average bit error probability bound of the can-

didate turbo code for a given interleaver size.

4. From all of the candidate codes, choose the one with the low-
est bit error probability in the desired range of SNR's.
In the design procedure, Steps 1 and 2 make sure the candidate
code has a maximum Zmin = 211 - 1 + 2, and thus the maximum
drree,eff = 211 + 6. Then the best code is chosen from all the candidate

codes. Since the code performance at high SNR's is determined

by the code effective free distance, the bit error probability can be
expressed by the dominant term

(4.49)

where B rree eff is the error coefficient related to the code effective
free distance. For a fixed effective free distance, optimizing the
bit error probability implies minimization of the error coefficient.
Thus, Steps 3 and 4 in the procedure can be replaced by

3. Evaluate the error coefficient Brree,eff of the candidate code.

4. From all the candidate codes, choose the one with the mini-
mum Brree,eff.
The above procedure was applied to find good rate 1/3 turbo
codes using rate 1/2 RSC component codes [14]. A uniform inter-
leaver with a size of 100 is employed in performance evaluation.
The results are reported in Table 4.1, where generator polynomials
go(D) and gl (D), the effective free distance d rree ,eff, the free dis-
tance d rree and its corresponding input weight W rree are shown. In
the table, the generator polynomials are given in octal form. For
example, the generator polynomials of the RSC code with memory
lJ = 4, go(D) = 1 + D + D4 and gl(D) = 1 + D + D3 + D4, are
represented by go(D) = (31) and gl(D) = (33), respectively.
Turbo Code Design 103

Table 4.1: Best rate 1/3 turbo codes at high SNR's [14]
v go(D) gl(D) dfree,eff dfree Wfree

1 3 2 4 4 2
2 7 5 10 7 3
3 15 17 14 8 4
4 31 33 22 9 5
31 27 22 9 5
5 51 77 38 10 6
51 67 38 12 4

Turbo codes using the best rate 1/2 RSC component codes can
achieve near optimum bit error probability performance at high
SNR's. These codes have applications in many real systems oper-
ating at relatively low BER's, such as data transmission in satellite
and space communications.

4.5.2 Turbo Code Design at Low SNR's

At low SNR's, the error probability of a turbo code is determined
by its distance spectrum. So the distance spectrum can be used as
a criterion to design good turbo codes in this region. In this sec-
tion, we will present design criteria and construct optimal distance
spectrum (ODS) turbo codes. These codes have minimum bit error
probability at low SNR's [25].
A turbo code is said to be an optimal distance spectrum turbo
code when its significant distance spectral lines are below those of
any other code with the same rate, memory order and interleaver
size. Assuming that the uniform interleaver is used, code design can
be formulated as: (1) determine the distance spectrum of the turbo
code, and (2) find good component codes by computer search which
produce an optimal distance spectrum. The search procedure can
be summarized as follows

1. Choose go(D) and gl (D) to be relatively prime polynomials

of degree v.
104 TUrbo Coding Performance Analysis and Code Design

2. Evaluate the distance spectrum of the candidate turbo code

for a given interleaver size.

3. From all of the candidate codes, choose the one with the small-
est error coefficients for low to medium Hamming distances.

Using the above method good component RSC codes with rate
1/2 and memory order v from two to five were found [25]. A uniform
interleaver with size 40 is used to evaluate the distance spectrum. In
Table 4.2, the code parameters of the ODS turbo codes are shown.

Table 4.2: Rate 1/3 ODS turbo codes at Low SNR's

v go(D) gl(D) dfree,eff dfree
2 7 5 10 7
3 15(13) 17 14 8
4 37 21 10 9
5 43 55 30 11

Comparing the ODS turbo code with the Benedetto-Montorsi

(BM) codes shown in Table 4.1, we can observe that for memory
orders 2 and 3, the BM codes are the same as the ODS turbo codes.
However, the BM codes do not have the optimal distance spectrum
for memory orders larger than 3. For memory orders 4 and 5, a
turbo code with a nonmaximum effective free distance can have an
optimal distance spectrum and better performance at low SNR's.
The distance spectra and bit error probability upper bounds for
the ODS turbo codes are given in Figs. 4.16 and 4.17, respectively.
The ODS turbo codes can achieve optimum bit error probabil-
ity performance at low SNR's. The codes can have applications in
many real systems operating at relatively high BER's such as voice
transmission in mobile communications, personal mobile communi-
cations, satellite mobile communications, military communications,
etc.
Turbo Code Design 105

10' , - - - - - . - - - - , - - - , - - - - - - - - , - - - . , - - - - - ,

10'

, • I
I I
I I
I
I I
I I
I I
I I
/
/ I
I I
I
I

I: ,"
\

10-<

10-"'------'-------'-----'-----'----""-----'
o 20

Fig. 4.16: Distance spectra for ODS turbo codes with interleaver size 40

4.5.3 Simulation Results

In order to compare the error performance of the ODS and BM
turbo codes with rate 1/3, simulation results for both codes with
memory order 4 are shown in Fig. 4.18. A pseudo-random inter-
leaver with size 4096 is employed for both codes.
The turbo codes are decoded with an iterative maximum a pos-
teriori probability algorithm. The number of iterations is 18. It
is obvious that the ODS turbo code performs better than the BM
turbo code at low SNR's corresponding to BER's from 10- 1 to 10- 5 .
However, the BM code outperforms the ODS turbo code at high
SNR's corresponding to BER's below 10- 5 • This figure shows that
code design based on performance analysis can also be verified by
simulation.
The code design discussed in this chapter is based on uniform
interleaving. We did not take into consideration the influence of
interleaving structure on code performance. Since the interleaving
structure affects turbo code error performance significantly at high
106 Turbo Coding Performance Analysis and Code Design

- v=2 N=40
10 -e- v=3 N=40
--a-- v=4 N=40
-b- v=5 N=40

10"

10-<1

10-11 L - - _ - - L_ _-L-_----'-_ _...l...-_----'_ _--L-_ _L - - _ - . J

1 1.5 2 2.5 3 35 4 4.5 5
EblNo (dB)

Fig. 4.17: Bit error probability upper bounds for ODS turbo codes with
interleaver size 40

SNR's, interleaver design can be combined with code design to fur-

ther improve turbo code error performance at high SNR's [25]. This
is achieved by devising an interleaver which is matched to the opti-
mum code found in the previous phase. The interleaver eliminates
the first several spectral lines in the original distance spectrum. The
bit error probability at high SNR's is then reduced and the asymp-
totic floor in the bit error probability curve is lowered. Interleaver
design will be discussed in Chapter 7.
Serial Concatenated Convolutional Codes 107

---+- ODS turbo code

~ 8M turbo code

10-6

1O-7 L -_ _L -_ _L -_ _L - - _ - - '_ _----.l_ _----'_ __

-0.4 -02 0 0.2 0.4 0.6 0.8
EbiNo (dB)

Fig. 4.18: Performance of ODS and BM turbo codes with rate 1/3 and
memory order 4 on AWGN channels

4.6 Serial Concatenated Convolutional

Codes
4.6.1 A Serial Concatenated Encoder
Let us consider a serial concatenated coding system consisting of
an outer code Co with rate Ro = kip and an inner code Ci with
rate ~ = pin. The two component codes are connected by an
interleaver of size N. For simplicity we will assume that N is divis-
ible by p. The overall concatenated code rate is R = kin. A block
diagram of a serial concatenated encoder is shown in Fig. 4.19.
For example, a rate 1/3 serial concatenated convolutional code is
formed by an outer (2,1) convolutional code and an inner (3,2) con-
volutional code, joined by an interleaver. In this section we assume
that the component codes are linear so that the serial concatenated
code is linear as well. In addition, the interleaver is assumed to
108 Turbo Coding Performance Analysis and Code Design

Outer Code Inner Code

Interleaver
Encoder Encoder

Fig. 4.19: A serial concatenated encoder

be uniform which enables the weight distribution of a serial con-

catenated code to be computed from the weight distribution of the
component codes.

4.6.2 Performance Analysis and Code Design

As a serial concatenated convolutional code is connected by an in-
terleaver of size N, for N much larger than the memory of the con-
volutional codes, the overall concatenated code can be considered
to be equivalent to an (N/ ~, N Ro) block code. The codewords are
sequences of the serial concatenated code that start and end at the
zero states of both convolutional codes.
The upper bound of the bit error probability Pb(e) for the serial
concatenated convolutional code is given by [26]

P.(e) ~ .I.I: N~ Aw,.Q (J2dR. !:) (4.50)

where R is the overall turbo code rate, Eb is the received bit en-
ergy, No is the one sided Gaussian noise spectral density, d is the
Hamming distance between the codewords, N is the interleaver size,
and dmin is the concatenated code minimum distance. Aw,d is the
average number of concatenated codewords of the equivalent block
code whose Hamming weight is d produced by a weight w input
sequence. Using the uniform interleaver the coefficients Aw,d can
be expressed as
N AGo. AG;
Aw,d = L
1=0
(W'IN )l'd (4.51)

1
where A~71 and Afd are the corresponding input-output weight dis-
tribution coefficients for equivalent block codes for the outer and
Serial Concatenated Convolutional Codes 109

inner code, respectively. These coefficients for the equivalent block

codes can be computed from the corresponding convolutional codes.
By definition, the codewords of the equivalent block code are con-
catenation of error events of the convolutional codes.
Let
A(w,X,n) = L:Aw,d,nXd (4.52)
d
be the weight enumerating function of sequences of the convolu-
tional code that concatenate n error events with total input weight
w, where Aw,d,n is the number of sequences of weight d, input weight
wand number of concatenated error events n in a codeword. For
interleaver size N much larger than the memory of the convolu-
tional codes, the input-output weight distribution coefficients of
the equivalent block code A~ ,d can be approximated by

A~,d ~ E(N/
nM
nP
)Aw,d,n (4.53)

where nM is the largest number of error events concatenated in a

codeword of weight d and generated by a weight w input sequence.
To compute these coefficients for the outer and inner equivalent
block code, we will denote by nO and ni the number of concatenated
error events in the outer and inner code, respectively. Also, we let
Aw,d,no and Aw,d,ni be the number of sequences of weight d, input
weight wand the number of concatenated error events nO or ni in a
codeword for the outer and inner code, respectively. By substituting
the coefficients A~~d and A~:d computed from (4.53) into (4.51), we
can approximate the coefficient Aw,d for the equivalent block code
of the serially concatenated code as

(4.54)

where dj is the free distance of the outer code.

For large N the binomial coefficients can be approximated by

(~)~:~
110 Turbo Coding Performance Analysis and Code Design

By substituting this approximation in (4.54) and in the expression

for the bit error probability bound (4.50) we obtain [26]

(4.55)

The bound (4.55) can be represented in the form

(4.56)

where
a(d) = max{nO(d)
w,l
+ ni(d) - i - I } (4.57)

and Cd is a coefficient which does not depend on N. At high SNR's

the bit error probability is dominated by the first term in the sum-
mation of (4.56). First we are going to compute a(dmin ), where
dmin corresponds to the first term. For the first term it is possible
to prove that [26]
(4.58)
This result shows that the exponent of N corresponding to the min-
imum weight codewords of the serial concatenated code is always
negative for dj ~ 2. That is, at high SNR's there is always an
interleaver gain if dj ~ 2.
Next we are going to compute the largest exponent of N, defined
as
aM = maxa(d)
d

The term with the largest exponent of N in the summation of (4.56)

dominates the contribution to the bit error probability for N --+ 00.
Let nk and n~ be the largest number of concatenated error
events in the codewords of the inner and outer code of weights d
and i, respectively. For block and nonrecursive convolutional inner
codes we have nk = l. Thus

aM = n~ -1 ~ 0 (4.59)
Serial Concatenated Convolutional Codes 111

This result indicates that for serial concatenated codes with block
or nonrecursive convolutional inner codes there are always some
terms of Hamming distance d, whose error coefficients in (4.55)
increase with interleaver size of N. That means no interleaver gain
is achieved for these terms.
For recursive convolutional inner codes the largest exponent of
N is given by [26]

aM = - ldO + I'
f 2 J< 0 (4.60)

As this result shows, if the inner code is a recursive convolutional

code there is always an interleaver gain. The interleaver gain is
N- d j/2 for even values of the outer code free distance and N-(dj+l)/2
for odd values of the outer code free distance.
The codeword weight associated with the highest exponent N,
denoted by d(aM), is given by

for dj even
(4.61 )
for dj odd

where d~,ef f is the effective free distance of the inner code. This
is the minimum weight of sequences of the inner code generated
by a weight-2 input sequence. Also, d~?n is the minimum weight of
sequences of the inner code generated by a weight-3 input sequence.
The asymptotic expressions, for N very large, for the bit error
probability are given by

(4.62)

for even values of dj, and

(4.63)

for odd values of dj, where Ceven and Codd are the coefficients for
di even and odd, respectively, which do not depend on N.
112 TUrbo Coding Performance Analysis and Code Design

Equations (4.62) and (4.63) show that the bit error probability
for large interleaver size N is dominated by the inner code effective
free distance and the outer code free distance. On the basis of these
results it is possible to formulate serial concatenated code design
rules. They are summarized as follows.
Design Rules for Serial Concatenated Codes
1. The inner code should be chosen to be a recursive convolu-
tional code. The outer code could be either nonrecursive or
recursive.

2. The effective free distance of the inner code d},elf should be

maximized.

3. The free distance of the outer code d, should be maximized.

4. The number of input sequences generating the free distance

error events of the outer code and their input weights should
be minimized.
As nonrecursive codes have error events with w=1 and, in gen-
eral, less input errors associated with error events at free distance,
it is appropriate to choose nonrecursive convolutional codes as outer
codes.
Bibliography

[1] G. D. Forney, Jr., Concatenated Codes. Cambridge, MA: MIT

Press, 1966.

[2] R. H. Deng and D. J. Costello, "High rate concatenated coding

systems using bandwidth efficient trellis inner codes," IEEE
Trans. Commun., vol. 37, no. 5, May 1989, pp. 420-427.

[3] J. Hagenauer and P. Hoeher, "Concatenated Viterbi decod-

ing," in Pmc. 4th Joint Swedish-Soviet Int. Workshop on In-
form. Theory, Gotland, Sweden, Studenlitteratur, Lund, Aug.
1989, pp. 29-33.

[4] C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon

limit error-correcting coding and decoding: Turbo-codes(I),"
in Pmc. ICC'93, Geneva, Switzerland, May 1993, pp. 1064-
1070.

[5] C. Berrou and A. Glavieux, "Near optimum error correcting

coding and decoding: Turbo-codes," IEEE Trans. Commun.,
vol. 44, no. 10, Oct. 1996, pp. 1261-1271.

[6] D. Divsalar and F. Pollara, "Turbo codes for PCS applica-

tions," in Pmc. ICC'95, Seattle, WA, June 1995, pp. 54-59.

[7] P. Robertson, "Illuminating the structure of parallel con-

catenated recursive systematic (TURBO) codes," in Pmc.
GLOBECOM'94, San Francisco, CA, Nov. 1994, pp. 1298-
1303.
114 BIBLIOGRAPHY

[8] A. S. Barbulescu and S. S. Pietrobon, "Interleaver design for

turbo codes," Electron. Lett., vol. 30, no. 25, Dec. 1994, p.
2107.

[9] A. S. Barbulescu and S. S. Pietrobon, "Terminating the trellis

of turbo-codes in the same state," Electron. Lett., vol. 31, no.
1, Jan. 1995, pp. 22-23.

[10] A. S. Barbulescu and S. S. Pietrobon, "Interleaver design for

three dimensional turbo codes," in Proc. 1995 IEEE I8IT,
Whistler, BC, Canada, Sep. 1995, p. 37.

[11] O. Joerssen and H. Meyr, "Terminating the trellis of turbo-

codes," Electron. Lett., vol. 30, no. 16, Aug. 1994, pp. 1285-
1286.

[12] D. Divsalar and F. Pollara, "On the design of turbo codes,"

TDA Progress Report 42-123, Jet Propulsion Lab., Nov. 1995,
pp. 99-121.

[13] S. Benedetto and G. Montorsi, "Unveiling turbo-codes: Some

results on parallel concatenated coding schemes," IEEE
Trans. Inform. Theory, vol. 42, no. 2, Mar. 1996, pp. 409-428.

[14] S. Benedetto and G. Montorsi, "Design of parallel concate-

nated convolutional codes," IEEE Trans. Commun., vol. 44,
no. 5, May 1996, pp. 591-600.

[15] S. Benedetto, R. Garello, and Guido Montorsi, "A search for

good convolutional codes to be used in the construction of
turbo codes," IEEE Trans. Commun., vol. 46, no. 9, Sep. 1998,
pp. 1101-1105.

[16] J. G. Proakis, Digital Communications, 2nd Ed., McGraw-Hill,

New York, 1989.

[17] D. Divsalar, S. Dolinar, R. J. McEliece, and F. Pollara, "Trans-

fer function bounds on the performance of turbo codes," TDA
Progress Report 42-123, Jet Propulsion Lab., Aug. 1995, pp.
44-55.
BIBLIOGRAPHY 115

[18] T. M. Duman and M. Salehi, "New performance bounds for

turbo codes," IEEE Trans. Commun., vol. 46, no. 6, June 1998,
pp. 717-723.

[19] A. M. Viterbi and A. J. Viterbi, "Improved union bound on

linear codes for the input-binary AWGN channel with appli-
cations to turbo-codes," in Proc. IEEE 1998 I8IT, MIT, MA,
Aug. 1998, p. 29.

[20] I. Sason and S. Shamai(shitz), "Improved upper bounds on the

performance of parallel and serial concatenated turbo codes,"
in Proc. IEEE 1998 I8IT, MIT, MA, Aug. 1998, p. 30.

[21] A. J. Viterbi, "Convolutional codes and their performance in

communications systems," IEEE Trans. Commun., vol. 19, no.
5, Oct. 1971, pp. 751-772.

[22] P. Thitimajshima, "Les codes convolutifs recursifs systema-

tiques et leur application a la concatenation parallele," (in
French), Ph.D. no. 284, University de Bretagne Occidentale,
Brest, France, Dec. 1993.

[23] D. Divsalar and R. J. McEliece, "The effective free distance

of turbo codes," Electron. lett., vol. 32, no. 5, Feb. 1996, pp.
445-446.

[24] L. C. Perez, J. Seghers, and D. J. Costello, Jr., "A distance

spectrum interpretation of turbo codes," IEEE Trans. Inform.
Theory, vol. 42, no. 6, Nov. 1996, pp. 1698-1709.
[25] J. Yuan, B. Vucetic, and W. Feng, "Combined turbo codes
and interleaver design," IEEE Trans. Commun., vol. 47, no. 4,
Apr. 1999, pp. 484-487.

[26] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Se-

rial concatenation of interleaved codes: Performance analysis,
design, and iterative decoding," IEEE Trans. Inform. Theory,
vol. 44, no. 3, May 1998, pp. 909-926.
Chapter 5

Trellis Based Decoding of

Linear Codes

5 .1 Introduction
In Chapters 2 and 3 we presented methods of representing linear
block and convolutional codes by trellises. In this chapter we first
examine various trellis based strategies for decoding of linear codes.
Each of these can be used as a basic building block for decoding of
concatenated and turbo codes.
Trellis based decoding algorithms are recursive methods for es-
timation of the state sequence of a discrete-time finite-state Markov
process observed in memoryless noise. The Viterbi algorithm (VA)
minimizes the sequence error probability. Its output is in the form
of hard-quantized estimation of symbols in the most likely transmit-
ted code sequence. In concatenated systems, with multiple signal
processing stages, the overall receiver performance is improved if the
inner stages produce soft output estimation. We show how the VA
can be modified to generate soft-output information. The related
algorithm is known as the soft output Viterbi algorithm (SOVA).
The SOYA produces in addition to the maximum likelihood hard
estimates of a code sequence, a reliability measure for each received
symbol. However, these reliability estimates are suboptimum.
In another class of decoders, the decoding criterion is minimiza-
tion of the symbol or bit error probability. The decoder generates

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
118 Trellis Based Decoding of Linear Codes

optimum reliability estimates in the form of a posteriori symbol

probabilities. The algorithm is known as the maximum a posteriori
probability (MAP) decoding. We further examine various versions
of the MAP algorithm, such as Max-Log-MAP and Log-MAP, in
search for a good performance-complexity trade-off.

5.2 System Model

In this section we introduce the underlying system model for the
decoding methods to be considered. The system block diagram is
shown in Fig. 5.1.

Memoryless
noise

c v x r
Encoder 1-----+1 Modulatorl----+l ........--+1 Decoder

Fig. 5.1: System model

In order to simplify the analysis, the following description of

the VA is specific to (n, 1, v) binary convolutional codes, though it
could easily be generalized to include kin rate convolutional codes,
as well as decoding of block codes. A binary message sequence,
denoted by c and given by

(5.1)

where Ct is the message symbol at time t and N is the sequence

length, is encoded by a linear code. In general the message sym-
bols Ct can be nonbinary but for simplicity we assume that they
are independently generated binary symbols and have equal a pri-
ori probabilities. The encoding operation is modelled as a discrete
time finite-state Markov process. This process can be graphically
System Model 119

represented by state and trellis diagrams. In respect to the input Ct,

the finite-state Markov process generates an output Vt and changes
its state from St to StH, where t + 1 is the next time instant. The
process can be completely specified by the following two relation-
ships

(5.2)
The functions f (.) and g(.) are generally time varying.
The state sequence from time 0 to t is denoted by S& and is
written as
(5.3)
The state sequence is a Markov process, so that the probability
P(StH I So, Sl,"', St) of being in state St+1, at time (t + 1), given
all states up to time t, depends only on the state St, at time t,

(5.4)
The encoder output sequence from time 1 to t is represented as

(5.5)
where
Vt = (Vt,O, Vt,l, ... , Vt,n-1) (5.6)
is the code block of length n.
The code sequence vi is modulated by a BPSK modulator. The
modulated sequence is denoted by xi and is given by

(5.7)
where
(5.8)
and
Xt,i = 2Vt,i - 1, i = 0 , 1 , ... , n - 1 (5.9)
As there is one-to-one correspondence between the code and
modulated sequence, the encoder/modulator pair can be repre-
sented by a discrete-time finite-state Markov process and can be
graphically described by state or trellis diagrams.
120 Trellis Based Decoding of Linear Codes

The modulated sequence xt is corrupted by additive white Gaus-

sian noise, resulting in the received sequence
(5.10)
where
(5.11)
and
rt,i = Xt,i + nt,i, i = 0,1" . " n - 1 (5.12)
where nt,i is a zero-mean Gaussian noise random variable with vari-
ance (72. Each noise sample is assumed to be independent from each
other.
The decoder gives an estimate of the input to the discrete finite-
state Markov source, by examining the received sequence rt.
The
decoding problem can be alternatively formulated as finding the
modulated sequence xt or the coded sequence vi. As there is one-
to-one correspondence between the sequences vi and xt, if one of
them has been estimated, the other can be obtained by simple map-
ping.
The discrete-time finite-state Markov source model is applicable
to a number of systems in communications, such as linear convolu-
tional and block coding, continuous phase modulation and channels
with intersymbol interference.

5.3 Optimization Criteria

In the system model shown in Fig. 5.1 we assumed BPSK modula-
tion. In general, we can consider M -ary modulation where the code
sequence v is divided into groups of log2 M binary symbols and each
of them is mapped into an M -ary symbol in the modulator.
Assume that the modulated sequence x contains N' M-ary sym-
bols
(5.13)
The corresponding code sequence v is N' . log2 M binary symbols
long
Optimization Criteria 121

The message sequence is given by

(5.14)

where N = RN' log2 M and R is the code rate.

The receiver estimates the unknown message sequence c, which
can be written as
(5.15)
Due to one-to-one correspondence between c, v and x, the estimates
of the code and modulated sequence can be obtained by simple
mapping operations from c. They can be written as

v (VI,I, ... ,VI,log2 M, V2,1, ... ,V2,log2 M, ... ,VN ',1' ... ,vN ' ,log2 M)
X (XI,X2,·· ·,xN ')
An optimum receiver is designed to minimize one of the follow-
ing error probabilities:

• Word error rate (WER) which is defined as the probability

that v f v, where v and v are the transmitted and estimated
code sequences, respectively.

• Symbol error rate (SER) which is defined as the probability of

Xt f Xt, t = 1,2, ... , N', where Xt and It are the transmitted
and estimated modulated symbols at time t, respectively.

• Bit error rate (BER) which is defined as the probability of

Ct f Ct, where Ct and Ct are the transmitted and corresponding
estimated bits at time t, respectively.

If a word error occurs, there must be at least one symbol error.

If a symbol is in error, there is at least one bit in error. The above
probabilities are defined for hard output estimates only.
It is possible to design three classes of optimum receivers, which
correspond to minimization of WER, SER and BER. While these
receivers have different performance at low signal-to-noise ratios
(SNR), their performances are almost identical at high SNR's. That
is, minimization of WER approximately produces minimum SER
and BER.
122 Trellis Based Decoding of Linear Codes

In a binary transmission system, the SER is identical to BER.

Even in non-binary systems, receivers minimizing SER have iden-
tical structure as those minimizing BER.

5.4 The Viterbi Algorithm

The Viterbi algorithm (VA) was originally proposed for decoding
of convolutional codes [1]. Essentially, the algorithm performs esti-
mation of the input sequence of a discrete time finite-state Markov
process observed in memoryless noise. Hence it is applicable as a
solution to various communication estimation problems, as long as
the system can be modelled as a finite state machine and repre-
sented by a time invariant or time varying trellis diagram.
We consider the system model shown in Fig. 5.1. The pair en-
coder /decoder represents a discrete time finite-state Markov source
described by a trellis diagram with Ms states.
Initially we assume that the process runs from state 0 at time
instant 0 to state 0 at time instant T, where T is the terminal time
instant.
The assumption of known start and final states is trivial and is
not an essential requirement for the VA to work. The state sequence
can be represented as

(5.16)

The VA finds an information sequence c = cI that corresponds to

the modulated sequence x = xI in the trellis diagram such that
the word error probability, denoted by Pw , is minimized. The word
error probability can be expressed as

Pw = 1 - JPr(c Ir)Pr(r) dr
r
(5.17)

where r is the received sequence r = rI.

Minimizing the word error probability is equivalent to maximiz-
ing the second term in Eq. (5.17), which is the probability that a
word is correct.
The Viterbi Algorithm 123

As Pr(r) is positive and independent of c, this is further equiv-

alent to maximizing the a posteriori probability Pr (c I r). The re-
ceiver which maximizes this probability is known as a maximum a
posteriori probability (MAP) receiver.
Using Bayes' rule we get

D ( I ) = Pr ( c) . Pr (r I c) (5.18)
rr C r Pr{r)
Assuming that the signals are equally likely, it suffices for the re-
ceiver to maximize the likelihood function Pr(r I c). A decoder that
selects its estimate by maximizing Pr{r Ic) is called maximum like-
lihood (ML) decoder. The above expression shows that if the se-
quences are equally likely, MAP and ML decoders are equivalent in
terms of word error probability.
The probability Pr{r Ic) for the received sequence of length T,
can be expressed as

Pr{rI IcD
Pr(rI IxI)

t=l
n-l 1
II II - - e-
T (rt .-Xt .)2
"2.,.2 " (5.19)
t= 1 i=O V2ii a
In order to simplify the operations we introduce the log function
log Pr{r I c), given by
T

log Pr(r Ic) = L log Pr{rt IXt) (5.20)

t=l

For the Gaussian channel log Pr{r I c) becomes

~ nII-l 1 _ (rt,,-xt,i)2
logPr{r I c) ~log --e 2.,.2
t= 1 i=O V2ii a
nT
-2Iog{27r) - nTlogO'
)2
-L L T n-l {
rt,i - ~t,i (5.21)
t=l i=O 20'
124 Trellis Based Decoding of Linear Codes

Expression (5.21) shows that maximizing P(r I c) is equivalent

to minimizing the Euclidean difference
T n-1
L L (rt,i - Xt,i)2
t=l i=O

between the received sequence rl and the modulated sequence xl

in the trellis diagram.
We assign to each branch at time t on the path x in the trellis,
the Euclidean distance, called branch metric, denoted by lIf, as
follows
n-l
1I~ = L (rt,i - Xt,i)2 (5.22)
i=O

Then the path metric corresponding to the path x, denoted by fJ,~x)

is given by
t
(x) ~ (x)
fJ,t L...J lit'
t'=l
(x)
fJ,t-1
+ lit(x) (5.23)
Note that minimization of the Euclidean distance is equivalent
to maximization of the expression 2:;=1 2:~ol rt,iXt,i, which is an-
other definition of the path metric.
The VA is an efficient way of finding a path in the trellis with
the minimum path metric. It is based on the idea that among the
paths merging into a state in the code trellis only the most probable
paths need to be saved for future processing, while all other paths
can be discarded with no consequence to decoding optimality.
The computation is based on keeping only one path per node
with the minimum path metric at each time instant. This path is
called the survivor.
The decision on the message estimate c is made at the final time
instant T. The maximum likelihood path is chosen as the survivor
in the final node. If the minimum path metric corresponds to a
path X, the decoder will select the binary sequence c on this path
as the hard estimate of the transmitted sequence c.
The VA can be summarized as follows.
The Viterbi Algorithm 125

Summary of the VA

1. Set initial values

t = 0; So = 0; Jl~x) (So = 0) = 0; Jl~x) (So =1= 0) = 00
2. Increase time t by 1
- Compute the branch metrics for all branches entering a
node at time t
- Compute the path metrics for all paths entering a node
at time t, by adding the branch metric of the branch
entering the node to the path metric of the connecting
survivor at a previous node at time t-I, as in Eq. (5.23)
- Compare the path metrics for all the paths entering a
node and find the survivor for each node. For each node
store the survivor path and its metric.
- Repeat 2 until t = T

3. The survivor at node S7 is the maximum likelihood path.

When the sequence length is long or infinite, it is necessary to trun-

cate the survivors to some manageable length, called the decision
depth, denoted by D.
The decoder makes the decision on the transmitted symbol Ct
at time t' = D + t. In general, if D is large enough, there is a high
probability that all survivors will stem from the same branch at
time t. The symbol on the maximum likelihood path at time t can
be delivered as the algorithm's firm decision. The decision on the
maximum likelihood path at time t' = t + D can be to choose either
any survivor or the survivor with the minimum path metric. When
D is large enough (about six times the encoder memory order), the
effect of choosing any survivor on the performance is negligible.
If time t becomes large it is necessary to normalize the survivor
path metrics from time to time by subtracting a constant from
all of them. For quantized branch metrics, the normalization of
the survivor path metrics can be avoided by using modulo-sum
algorithms.
126 Trellis Based Decoding of Linear Codes

The VA may be required to start with no knowledge of the initial

state. In this case, it might be initialized by any reasonable value
such as Ji.o(l) = 0, 1 = 1,2,,'" M s , for all states l. Usually, after
an initial transient there is a high probability that all survivors will
merge with the correct path. That is, the VA synchronizes itself
with no special procedure.
The VA requires Ms . D memory units, to store the survivors
and their path metrics. For an (n, k) convolutional code, there
are 2k branches entering a node. Thus, the algorithm computes
Ms' 2k branch metrics and performs Ms ·2k additions and Ms' 2k
comparisons at each time.
Therefore, the memory requirement is proportional to the num-
ber of states, and the computational complexity is proportional to
the number of branches and the number of states.

Example 5.1 Consider a 1/2 rate RSC code with the encoder
shown in Fig. 5.2 (a). Its state and trellis diagrams are shown in
Figs.5.2 (b) and (c), respectively. If the received sequence is

r = [(1, -1), (0.8,1), (-1,1), (-1,1), (1, -1)]

the branch metrics are shown in Fig. 5.3. As there are four message
symbols, the transmitted sequence is extended by adding one input
symbol equal to the feedback symbol, to terminate the trellis to the
zero state.

The survivors and their path metrics at times t = 1 to t = 5

are shown in Figs. 5.4, (a) to (e). The transmitted information
sequence estimate is c = (0,1,0,0,1).

5.5 The Bidirectional Soft Output

Viterbi Algorithm
The drawback of the VA is that it generates hard symbol estimates
resulting in performance loss in multistage decoding. It can be mod-
ified to produce soft outputs [6], [14], [12], [2]. Here we will present
a bidirectional SOVA algorithm which does not require knowledge
The Bidirectional Soft Output Viterbi Algorithm 127

Ct ---------~~------. Vt,O

J
I
ll-----+( + 1----..----- Vt,l

(a) Encoder

1/11

0/00 0/01

1/10
(b) State diagram
01 01
1

o
(c) Trellis diagram
input bit Ct = 1
- - - - - - input bit Ct = 0

Fig. 5.2: A convolutional encoder and its graphical representation

of the noise variance and is simple to implement particularly for

systems with block structured data [2] [12] [13].

The SOYA estimates the soft output information for each trans-
mitted binary symbol in the form of the log-likelihood function
128 Trellis Based Decoding of Linear Codes

Time 0 1 2 3 4 5
3.24 o o
1

o 4 7.24 4 4 4
---input bit Ct = 1
- - - - - input bit Ct = 0

Fig. 5.3: The branch metrics in Example 5.1

a
L t=t
4

b) ~4.04

/--~~.04
c) - .. 4.04

- - - - "12.04
t=3
d)

.. - -- -
e)
/.----~----~~04
.. - - - -
Fig. 5.4: The survivors and their path metrics in Example 5.1

A( Ct), as follows

(5.24)
The Bidirectional Soft Output Viterbi Algorithm 129

where rr is the received sequence and Pr { Ct = i 1 rl} , i = 0, 1, is

the a posteriori probability (APP) of the transmitted symbol.
The SOVA decoder makes a hard decision by comparing A( Ct)
to a threshold equal to zero

if A(Ct) ~ 0
(5.25)
otherwise
The decoder selects the path x with the minimum path metric
J-tr,min as the maximum likelihood (ML) path in the same way as

the standard VA. The probability of selecting this path, from Eqs.
(5.18), (5.19), (5.21) and (5.23), is proportional to

(5.26)

Let us denote by J-tt,c the minimum path metric of the paths

with the complementary symbol to the ML symbol at time t. If
the ML symbol at time t is 1, then its complementary symbol is O.
Therefore we can write

Pr(ct= 11 rD f"Ve-JtT,min

Pr(Ct = 01 rD f"Ve- Jtt •c (5.27)

The logarithm of the ratio of the above two probabilities is given

by
e-JtT,min
log---
e-Jtt,c

J-tt,c - J-tr ,min (5.28)

Let us denote by J..l~ the minimum path metric for all paths for
which Ct is 1 and J-t~ the minimum path metric for all paths for
which Ct is O. If the ML estimate at time t is 1, its complementary
symbol at time t is O. Then J..l~ = J..lr,min and J..l~ = J..lt,c and the
log-likelihood in Eq. (5.28) becomes

I Pr { Ct = 11 rl} 0 1
(5.29)
og P r { Ct = 0 IrI} f"V J-tt - J-tt
130 Trellis Based Decoding of Linear Codes

On the other hand if the ML hard estimates at time t is 0, its

complementary symbol is 1, giving J-l~ = J-lr,min and J-li = J-lt,c' The
log-likelihood ratio for this case is given by
e-/.lt,c
log---
e-/.lT,min

J-lr,min - J-lt,c
J-l~ - J-li (5.30)
As Eqs. (5.29) and (5.30) indicate, regardless of the value of the
ML hard estimate, the log-likelihood ratio can be expressed as

I og Pr{ Ct = 11 rl}
Pr{Ct = 0 I rI}
rv J-l~ - J-li (5.31)

That is, the soft output of the decoder can be obtained as the
difference of the minimum path metric among all the paths with
symbol 0 at time t and the minimum path metric among all the
paths with symbol 1 at time t. The sign of A(Ct) determines the
hard estimate at time t and its absolute value represents the soft
output information that can be used for decoding in the next stage.
If the decision is made on a finite length block, as in block
codes, turbo codes or convolutional codes in TDMA systems, the
SOYA can be implemented as a bidirectional recursive method with
forward and backward recursions.
The SOYA can be summarized as follows.

Summary of the SOYA

I Forward recursion
1. Set initial values
t = 0; So = 0; I/(;')(So = 0) = 0; J-l~x)(So =1= 0)
00; Sr = 0
2. Increase time t by 1
- Compute the branch metrics for all branches enter-
ing a node at time t
The Bidirectional Soft Output Viterbi Algorithm 131

- Compute the path metrics for all paths entering a

node at time t
- Compare the path metrics and find the survivor for
each node
- Store the survivor and its path metric for each node
- Repeat 2 until t =r
3. The survivor at the node Sr is the maximum likelihood
path and its metric is /-lr,min.

II Backward recursion

1. Set the initial values

t = r; Sr = 0; /-l}x) (Sr = 0) = 0; /-l}x) (Sr =f. 0) =
00; So = 0
2. Decrease time t by 1
- Compute the branch metrics for all branches enter-
ing a node at time t
- Compute the path metrics for all paths entering a
node at time t
- Compare the path metrics and find the survivor for
each node
- Store the survivor path metric for each node
- Repeat 2 until t = 0

III Soft decision

1. Set t =0
2. Increase time t by 1
- At time t, identify the maximum likelihood estimate
ct=i,i=O,l
- Determine /-l~ as
i _ .
/-It - /-lr,mm
132 'Trellis Based Decoding of Linear Codes

- Find the path metric of its best competitor /l-~ , C=

i EB 1, where EB is modulo 2 sum, as

/l-~ = min{/l-Ll(l')
I,e
+ v~(l', i) + /l-~(i)} (5.32)

where i', i = 0,1"", Ms - 1, /l-Ll (i') is the path

metric of the forward survivor at time t - 1 and
node i', v% (i', i) is the branch metric at time t for a
complement symbols c from node i' to i, /l-Z(l) is the
backward survivor path metric at time t and node i.
- Compute A(ct) as

(5.33)

Note that in order to reduce the decoding complexity, the back-

ward recursion and soft decision steps can be performed simultane-
ously.

Example 5.2 Consider the encoder shown in Fig. (5.2) . The

received sequence is given by

rt = [(11), (-11), (0.8 -1), (-1 - 1)] (5.34)

and the encoder starts and ends at state O. The branch metrics for
this received sequence are shown in Fig. 5.5. They are computed
as in the standard VA, Eq. (5.22). The result of applying the VA
in the forward recursion is shown in Fig. 5.6. The survivor path
metrics are shown above each node and the ML path is represented
by the thick line. The final survivor is the ML path with the metric
/l-4,min = 0.04.
The results of the backward recursion are shown in Fig. 5.7.
The first number above each node shows the forward survivor path
metric, the second number shows the backward survivor path met-
ric.
At time t = 1, the ML hard estimate is Cl = 1 and thus 14 =
/l-4,min = 0.04. /l-~ is the minimum path metric of the paths that
have 0 at time 1. There is only one path with zero at time 1, and
The Bidirectional Soft Output Viterbi Algorithm 133

Time 0 1 2 3 4
7.24
1

o
- - - - input bit Ct = 1
- input bit Ct = 0

Fig. 5.5: The branch metrics in Example 5.2

o o 7.24

..
0.04

Fig. 5.6: The forward recursion in Example 5.2, the ML path is shown
by the thick line

0/0.04 0/0.04 7.24/4

..
0.04/0

Fig. 5.7: The backward recursion in Example 5.2, the ML path is shown
by the thick line

its path metric is calculated from Fig. 5.7 as the sum of the forward
and backward survivor path metrics at node 0,

J1~ = 8 + 4.04 = 12.04 (5.35)

134 Trellis Based Decoding of Linear Codes

The log-likelihood ratio at time 1 is given by

A(cd J-l~ - J-l~
12.04 - 0.04
12 (5.36)
The ML symbol at time 2 is C2 = 0, and the minimum path
metric is J-lg = J-l4,min = 0.04. The best competitor path at time 2,
denoted by J-l~ is obtained as
J-l~ = min{J-l{ (l')
1,1
+ vi(lf, l) + J-l~(l)} (5.37)

where J-l{ (If) is the forward survivor path metric at time 1 and node
t', If = 1, 0, vi( If, l) is the branch metric at time 2 for the input
symbol 1 and J-l~(l) is the backward survivor path metric at time 2
and node l, l = 1, O.
J-l~ = min{(O + 8 + 3.24), (8 + 4 + 0.04)}
= 11.24 (5.38)
The log-likelihood ratio at time 2 is given by
A(C2) = J-lg - J-l~ = 0.04 - 11.24 = -11.2 (5.39)
By applying this procedure to other time instants we obtain for
other soft outputs
A(C3) = 11.2 (5.40)
The computational complexity of the forward recursion is equiv-
alent to that of the Viterbi algorithm. The computational com-
plexity of the backward recursion is usually less than that of the
Viterbi algorithm, since there is no need to store backward sur-
vivors. Therefore, the computational complexity of the SOVA is
upper-bounded by two times that of the VA. With binary inputs,
the computational complexity is about 1.5 times of the VA.
The algorithm can be directly extended to the case k > 1, by
considering a1l2 k - 1 complement symbols to the ML symbol for each
node in (5.32).
It can be similarly generalized to handle non-binary modulation
schemes.
Sliding Window SOYA 135

5.6 Sliding Window SOYA

The standard Viterbi decoder selects a path in the trellis with the
minimum path metric. The decision making process is based on
keeping only one path per node with the minimum metric at each
time instant. The decisions are made with a delay, which depends
on the code memory order. The minimum path metric is computed
as the minimum among Ms survivor paths, where Ms is the number
of states in the trellis.
In the standard SOYA algorithm the decision delay is equal to
the received sequence length. For large sequence lengths, the mem-
ory required for decoder implementation is excessive. In this algo-
rithm, the decisions are based on forward and backward recursions.
The results of both recursions must be stored. The forward and
backward recursions are performed on the whole sequence length,
so that the decoder needs to store T x Ms survivors and survivor
path metrics in each direction.
The SOYA algorithm can be simplified by reducing the required
memory.
The decoder memory can be reduced as reliable decoding de-
cisions for a convolutional code decoded by VA can be made after
the decision depth which is about six times the encoder memory.
Another important point is that the Viterbi decoder can start at
any time from any state. In the beginning, the path metrics that are
generated are not reliable, but after several encoder memory lengths
the results approach those achieved as if the decoder started from
the initial node.
In the SOYA algorithm in both forward and backward recursions
the decision on a symbol can be made with a delay at about six
times the memory length. Let us assume that the required decision
depth for a single component convolutional code is denoted by D.
The decoder does not need to start from the initial node. It can
start at any node and after the decision depth the decision can be
made as reliably as starting from the initial node.
The simplified decoding operations can be summarized as fol-
lows.
Let the received sequence be divided into a few subblocks, with
 136 Trellis Based Decoding of Linear Codes

subblock length of D. For the decoder output of each subblock, the

forward and backward recursions are extended to the next subblock
to generate the reliable metrics.
The forward algorithm processing starts at the initial time in-
stant 0, computing path metrics for each node at each time instant,
and storing these metrics in memory. At the 2Dth time instant, it
selects the path with the minimum path metric as the ML path,
and makes the hard decisions for the first subblock. The backward
recursion starts from the 2Dth time instant and goes backward,
setting each initial path metric to the same value and not storing
anything until time instant D. By this time it has built up reliable
path metrics. The decoding process is shown graphically in Fig.
5.8. (In this Figure, unreliable path metrics are shown by dashed
lines.) The backward recursion continues from the Dth time instant
until it reaches the initial time instant o. As soon as the backward
recursion reaches time instant D, the decoder starts making soft de-
cisions for the first subblock, based on both forward and backward
recursions.
Note that since the soft decisions are computed at the same
time as the backward recursions, there is no need to store all D
backward path metrics, but only one for each node.
When the soft outputs are generated, all path metrics for the
first subblock will be discarded and the emptied storage can be filled
with the second subblock's metrics computed by the last forward
recursion. The forward recursion computes the path metrics from
the (2D + l)th to the 3Dth time instant and selects the minimum
path metric for the second subblock. The backward recursion be-
gins from 3Dth time instant and builds up reliable state metrics at
2Dth time instant. Then simultaneously with the backward recur-
sions, soft decisions from the 2Dth to the Dth time instant will be
generated from the results of the forward and backward recursions.
The forward recursion only needs to store 2D x Ms path met-
rics because this memory can be used repeatedly. The backward
recursion just needs to store 1 x Ms path metrics.
Note that for this algorithm the computation time is increased
since the backward recursions are computed twice. However, the
comparison of the standard SOYA and simplified SOYA shows that
Sliding Window SOYA 137

Sub-block 1 Sub-block 2 Sub-block 3 Sub-block 4

Nodes
I0 ID 12D 13D 14D

Forward
Processing 10

Backward
Processing 10 In - - - 12D

Sub-block 1
Decoder
Output I I
o D

Forward
Processing

Backward
Processing

Sub-block 2
Decoder
Output I I
D 2D

Forward
Processing

Backward
Processing

Decoder Sub-block 3
Output I I
2D 3D

Fig. 5.8: Forward and Backward processing for the simplified SOYA

the increase in running time is marginal.

138 Trellis Based Decoding of Linear Codes

5.7 The MAP Algorithm

The MAP algorithm minimizes the symbol (or bit) error probabil-
ity. For each transmitted symbol it generates its hard estimate and
soft output in the form of the a posteriori probability on the basis
of the received sequence r. It computes the log-likelihood ratio

A( ) - 1 Pr { Ct = 11 r} (5.41)
Ct - og -P,-r{=-c-t-O---'I'---r-7-}
for 1 ~ t ~ T, where T is the received sequence length, and compares
this value to a zero threshold to determine the hard estimate Ct as
if A(ct) ~ 0
(5.42)
otherwise
The value A(ct) represents the soft information associated with the
hard estimate Ct. It might be used in a next decoding stage.
Consider again a system model shown in Fig. 5.1. For simplicity,
we assume that a binary sequence c of length N is encoded by a
systematic convolutional code of rate lin. The encoding process is
modelled by a discrete-time finite-state Markov process described
by a state and a trellis diagram with the number of states Ms. We
assume that the initial state So = 0 and the final state S7 = O. The
received sequence r is corrupted by a zero-mean Gaussian noise
with variance (J'2.
As an example a rate 1/2 memory order 2 RSC encoder is shown
in Fig. 5.9, and its state and trellis diagrams are illustrated in Figs.
5.10 and 5.11, respectively.
The content of the shift register in the encoder at time t repre-
sents St and it transits into St+! in response to the input Ct+! giving
as output the coded block Vt+!. The state transition of the encoder
is shown in the state diagram.
The state transitions of the encoder are governed by the transi-
tion probabilities
Pt (lil') = Pr {St = l I St-1 = l'}; O:S; l , l' :s; Ms - 1
The encoder output is determined by the probabilities
qt (Xt Il', l) = Pr {Xt I St-1 = l', St = l}; 0 ~ l , l' ~ Ms - 1
The MAP Algorithm 139

Fig. 5.9: A rate 1/2 memory order 2 RSC encoder

Because of one-to-one correspondence between Xt and Vt we have

qdXt Il',l) = Pr{Vt I St-l = l',St = l}; 0::; l, l'::; Ms-1

For the encoder in Fig. 5.9, Pt (l Il') is either 0.5, when there
°
is a connection from St-l = l' to St = l or when there is no
connection. qt (x Il', l) is either 1 or 0. For example, from Figs.
5.10 and 5.11 we have
pt(210) = pt(l) = 0.5; pt(112) = Pt(1) = 0.5
(5.43)
pt(310) = 0; pt(113) = Pt(O) = 0.5
and

qt( -1, -110,0) = 1 qt( -1, +110, 0) =°

°
qt( +1, -110, 1) = qt( +1, +110, 2) = 1 (5.44)

For a given input sequence

°
the encoding process starts at the initial state So = and produces
an output sequence xI ending in the terminal state S7 = 0, where
T = N + 1/. The input to the channel is xI and the output is
rI = (rl,r2,···,r7 ).
140 'Ifellis Based Decoding of Linear Codes

Fig. 5.10: State transition diagram for the (2,1,2) RSC code

The transition probabilities of the Gaussian channel are defined

by
T

Pr {rIlxD = II R (rjlxj) (5.45)

j=l

where
n-l
R (rjlxj) = II Pr (rj,ilxj,i) (5.46)
i=O

and
1 _(r]'i+l)2
Pr{rj,ilxj,i = -I} = V2if(je 2u 2 (5.47)

1 h,i-l)2
Pr {r·),Z·Ix··
),Z
= +l} = - -e-
V2if(j 2u 2 (5.48)

where (j2 is the noise variance.

Let Ct be the information bit associated with the transition St-l
to St, producing as output Vt. The decoder gives an estimate of
The MAP Algorithm 141

0/00 0/00 0/00 0/00

8=00

8=01

8=10

8=11
t=O t=1 t=2 t=3 t=4

Fig. 5.11: 'Trellis diagram for the (2,1,2) R8C code

the input to the Markov source, by examining r[. The MAP al-
gorithm provides the log likelihood ratio, denoted by A(Ct), given
the received sequence rI, as indicated in Eq. (5.41). The decoder
makes a decision by comparing A (Ct) to a threshold equal to zero.
We can compute the APP's in (5.41) as

Pr{Ct °
= I rI} = L
(I' ,1)fB~
Pr{St-l = l',St = llrI}
~ Pr{St-l=l',St=l,rI} (
L.J 5.49)
(I' ,1)EB~ Pr {rI}
where B2 is the set of transitions St-l = l' ~ St = l that are caused
by the input bit Ct = o. For example, B2 for the diagram in Fig.
5.11 are (3,1), (0,0), (1,2) and (2,3).
Also
Pr{Ct = 11 rI} = L Pr{St-l = l',St = llrI}
(l' ,1)EBI

L Pr {St-l = l', St = l, rI} (5.50)

(I' ,1)EBI Pr {rI}
142 Trellis Based Decoding of Linear Codes

where BI is the set of transitions St-l = l' -+ St = 1 that are caused

by the input bit Ct = 1. For the diagram in Fig. 5.11, BI consists
of (0,2), (2,1), (3,3) and (1,0).
The APP's of the decoded data bit Ct can be derived from the
joint probability defined as

(ft(l',l) = Pr{St-l = l',St = l,rD, l',l = O,I,···,Ms - l

Eqs. (5.49) and (5.50) can be written as

P r {Ct -_ 01 r T}
1 -
_ '"
L...i
(ft(l" l) (5.51)
(1',I)EBr Pr {rI}

Pr{Ct = llrD = L (ft(l',!) (5.52)

(1',I)EBl Pr {rd
The log-likelihood ratio A (Ct) is then

(5.53)

The log-likelihood A (Ct) represents the soft output of the MAP

decoder. It can be used as an input to another decoder in a con-
catenated scheme or in the next iteration in an iterative decoder. In
the final operation, the decoder makes a hard decision by comparing
A (ct) to a threshold equal to zero.

Definitions of 0:', f3 and 'Y

In order to compute the joint probability (ft (l', l) required for cal-
culation of A (Ct) in (5.53), we define the following probabilities

(}t(l) = Pr {St = l, rO (5.54)

f3t(l) = Pr {rT+IISt = l} (5.55)

'"t: (l', l) = Pr {Ct = i, St = l, rtlSt-l = l'}; i = 0,1 (5.56)

The MAP Algorithm 143

Now we can express at (i', i) as

at (i', l) Pr {St-I = i', St = i, rI}

Pr {r;+I' rt, ri- I , St = i, St-I = i'}
Pr {r[+llrt, ri- I , St = i, St-I = i'}
. Pr {rt, ri- I , St = I, St-I = I'}
Pr{r[+IISt = I}Pr{rt,ri-I,St = I,St-1 = I'}
(3t(l) . Pr {rt, ri-l, St = I, St-I = I'}
(3t(l) . Pr {St = i, rtlSt-1 = I', ri- I }
. Pr {St-I = I', ri- I }
Pr {St-I = i', ri- I } . (3t(l) . Pr {St = I, rtlSt-1 = i'}
CYt-1 (I') . (3t(l) . LiE(O,I) '"yt (i', i)
(5.57)
The log-likelihood ratio A (Ct) can be written as

A (c) = iog
L(l' I)EBl
, t
,i (i', I) (3t(l)
CYt-1 (i')
(5.58)
t L(l',I)EB? CYt-1 (i') ,2 (i', i) (3t(l)

Derivation of a
We can obtain CY defined in (5.54) as

CYt(l) = Pr {St = I, rU
L~OI Pr {St-I = I', St = I, rU
L~OI Pr iSt-1 = I', St = i, ri- I , rt}
"Ms-I PS
L..I'=O .. r t-I i= ' ,r t-I}
l

. Pr {St = i, rtlSt-1 = I', ri- I }

L~Ol CYt-1 (I') . Pr {St = I, rtlSt-1 = I'}
L~OI CYt-1 (I') . LiE(O,I) Pr {St = I, Ct = i, rtlSt-1 = I'}
L~OI CYt-1 (I') . LiE(O,I) ,i (I', l)
(5.59)
for t = 1,2,· .. T.
For t = 0 we have the boundary conditions CYo(O) = 1 and CYo(i)
= 0 for I =I- o.
144 Trellis Based Decoding of Linear Codes

Derivation of f3
We can express 13t(l) defined in (5.55) as

13t(l) Pr {rf+lISt = l} (5.60)

E~OI Pr {StH = l', rfHISt = l}
"Ms -l Pr{ SH1=1',r[+1,St=l}
- wl'=O Pr{St=l}
"Ms -1 Pr{ r[+2,rt+l.St+l =1' ,St=l}
L..Jl'=O Pr{St=l}
"Ms-l Pr{ r[+2I r t+l,St+l=I' ,St=l }Pr{rHl,St+l=I' ,St=l}
L..JI'=O Pr{St=l}
"M.-l Pr{ r[+2ISt+l=I'}.Pr{rt+l,SH1=1' ,St=l}
L..Jl'=O Pr{St=l}
"M.-l (3t+l(I')Pr{SH1=1',rt+lISt=I}Pr{St=l}
L..Jl'=O Pr{St=l}
E~OI 13tH (l') Pr {StH = l',rtHISt = l}
E~OI 13tH (l')
. E iE (O,I) Pr {CHI = i, StH = l', rtHISt = l}
E~OI 13tH (l') E iE (O,I) 1';H (l, l')

for t = 'T - 1, ... , 1, O.

The boundary conditions are 137(0) = 1 and 137(l) = 0 for l =1= o.

Derivation of I
We can write for 1': (l', l) defined in (5.56)
The MAP Algorithm 145

We can further express '"yt ([', [) as

"n-l( i _ • (1))2)
. P (i)exp ( - for (1, 1') E Bti
1'; (1', 1) = { t
L.Jj=O Tt,J Xt,j
20- 2
o otherwise
(5.61)
where pt(i) is the a priori probability of Ct = i and xtj(l) is the en-
coder output associated with the transition St-l = [' to St = [ and
input Ct = i. Note that the expression for R (rtIXt) is normalized
by multiplying (5.46) (y"Fia) n.

Summary of the MAP Algorithm

1. Forward recursion

• Initialize D!o(l), [= 0,1, ... , Ms - 1

D!0(0) = 1 and D!o(l) = 0 for [ =I 0
• For t = 1,2,· .. T, [ = 0,1,· .. Ms - 1 and all branches in
the trellis calculate

for i = 0,1 (5.62)

where pt(i) is the a priori probability of each information

bit, lf2 (rt, Xt) is the squared Euclidean distance between
rt and the modulated symbol in the trellis Xt.
• For i = 0,1 store 1'1(1', l).
• For t = 1,2,· .. , T, and [ = 0, 1,· .. , Ms -1 calculate and
store D!t(l)
M.-l
D!t(l) = ~ ~ O!t-l(z'h;(l; l) (5.63)
/' =0 iE(O,l)

The graphical representation of the forward recursion is given

in Fig. 5.12.

2. Backward recursion
146 Trellis Based Decoding of Linear Codes

Fig. 5.12: Graphical representation of the forward recursion

• Initialize (37(l), l = 0, 1, ... , Ms - 1

(37(0) = 1 and (37(l) = 0 for l =I 0
• For t = T - 1, .. ·1,0 and l = 0,1, ... Ms - 1 calculate
(3t(l) as
Ms-l
(3t(l) = L L (3t+l(l,)--'d+l(l,z') (5.64)
l' =0 iE(O,l)

where "'d+1 (l, z') was computed in the forward recursion.

• For t < T calculate the log-likelihood A(Ct) as

A( ) -1 Et!O-l O!t-l(l}Yi(l; l)(3t(l) (5.65)

Ct - og El=o-
M 1
O!t-l ( l') '"Yt0 (l,' l) (3t ()
l
The graphical representation of the backward recursion is shown in
Fig. 5.13.
Note that because Eq. (5.65) is a ratio, the values for O!t(l')
and (3t(l) can be normalized at any node which keeps them from
overflowing.
The MAP algorithm can be applied only if the sequence length
T is finite, while the VA can be used for any sequence length, by
truncating the survivors.
However, it is possible to apply the sliding window MAP de-
coding as for the SOVA algorithm, in the same way as shown in
Section 5.6 to achieve memory savings.
The MAP Algorithm 147

Fig. 5.13: Graphical representation of the backward recursion

The MAP requires the storage of Ms x 7 memory units to store

the results of the forward recursion. The memory site grows lin-
early with the sequence length 7. If the number of branches leaving
and entering each node is Nb = 2k, where k is the number of infor-
mation bits at the encoder input, then for each time unit Ms . Nb
multiplications and Ms(Nb - 1) additions are needed for both the
forward and backward recursions. The computational requirement
per time unit is constant and independent of the sequence length
7. The branch metric for the MAP is given by Eq. (5.62).
Comparing this expression with Eq. (5.22), we can see that the
branch metric for the MAP is much more complex to calculate than
for the VA. For this reason the branch metric calculation can be
implemented as a look-up table.
It is worth noting that Chang and Hancock [11 J developed a
MAP algorithm similar to the one in [9J for removal of intersym-
bol interference (lSI). For the same application, Abend and Fritch-
man [8J proposed a different MAP algorithm, which requires only
a forward recursion and can be used in continuous mode decoding.
Raviv [7J used this MAP algorithm for character recognition. How-
ever, the memory and computational complexity of this algorithm
grow exponentially with the decoding delay. A MAP symbol-by-
symbol decoding algorithm with a fixed decoding delay and linear
memory and complexity growth with decoding delay was proposed
in [14J. A soft-input soft-output (SISO) MAP algorithm for de-
coding of turbo and serial concatenated codes has been reported in
148 Trellis Based Decoding of Linear Codes

[16].
It is also worth mentioning that the branch metric is dependent
on the noise variance a 2 , while the branch metric for the VA and
SOVA is not. However, it has been shown that, at high SNR's, the
accurate estimation of the noise variance is not very important [17].
If the final state of the trellis is not known, the probability 137(/),
can be initialized as

(5.66)

Example 5.3 In a system encoded by the (2,1,1) recursive

convolutional code with the generator matrix (1, l~D) with BPSK
modulation and ~ = 2dB, the transmitted sequence is given by
c = (11001) (5.67)
After time t = 5 the encoder was driven to the zero state. The
received sequence is given by
(5.68)
and the values for Tt,j, t = 1,2, .. ',6, j = 0,1, are shown in Table
5.1. Determine the soft and hard outputs of a MAP decoder.

Solution
The trellis diagram for this code is shown in Fig. 5.14.
The branch metrics 'Y~i)(l,z'), 1,1' = 0,1, t = 1,2"",6 are
computed by Eq. (5.61).
Starting from the initial condition ao(O) = 1, ao(l) = 0 and
using the forward recursion formula in Eq. (5.59) at(l), I = 0,1,
t = 1, ... ,6, are computed.
Similarly, using the initial conditions 136(0) = 1 and 136(1) =
0, 13t(l) , I = 0,1, t = 1,2"",6, are computed by the backward
recursion formula in Eq. (5.60).
The soft-outputs are calculated by Eq. (5.65).
The hard estimates Ct are obtained by comparing the soft output
At(Ct) to the threshold of O.
The values of Ct, 'Yii) (1,1'), at(l), 13t(l), At(Ct) and Ct are shown
in Table 5.1.
The Max-Log-MAP Algorithm 149

Table 5.1
t 1 2 3 4 5 6
Ct 1 1 0 0 1
rl
t 0.030041 -0.570849 -0.38405 -0.744790 0.525812 0.507154
r; 0.726249 -0.753015 -1.107597 -0.495092 1.904994 -1.591323
1'P(O,O) 0.034678 0.425261 0.386226 0.404741 0.000408 0.088558
1'P(I,I) 0.236152 0.058185 0.020712 0.109450 0.062573 0.001323
1'{(0,1) 0.255655 0.012881 0.007510 0.015309 0.250953 0.005052
1'{(1,0) 0.037542 0.094143 0.140044 0.056593 0.001638 0.338085
G.t(O) 0.034678 0.038815 0.017137 0.006971 0.000003 0.000599
G.t(1) 0.255655 0.015322 0.000609 0.000329 0.001770 0.000002
/h (0) 0.005783 0.013448 0.034680 0.089880 0.088558 1
!1t (1) 0.001557 0.005005 0.007135 0.021300 0.338085 0
At (Ct) 0.685647 0.177998 -1.920772 -4.239018 4.407100 7.598074
Ct 1 1 0 0 1

5.8 The Max-Log-MAP Algorithm

The MAP decoding requires large memory and a large number of
operations involving exponentiations and multiplications. The al-
gorithm is likely to be considered too complex for implementation
in many communication systems.
One way of simplifying computations is to work with the log-
arithms of 'Y:(l', l), (Xt(t) and f3t(l), denoted by ~(z', l), at(l) and
7Jt(l), respectively. ~(z', l) is given by
'Y~(z',l) = log'Y;(l',l) (5.69)
Referring to Eq. (5.59), at(l) can be expressed as
(5.70)
log L L
M.-l
eOt - 1 (1
"
)+'n(/,/)
I' =0 iE(O,l)
150 Trellis Based Decoding of Linear Codes

1
0/01
Fig. 5.14: Trellis diagram for the encoder in Example 5.3

with the initial conditions

ao(O) = 0 and ao(l) = -00 for 1 i= 0

Similarly, f3t(l) can be represented as (Eq. (5.60))

f3t(l) logf3t(l)
L L
M.-l
log e13t +1 (1')+'Yt+1 (1,1')
I' =0 iE(O,l)

with the initial conditions

f3r = 0 and f3r (l) = -00 and 1 i= 0

By substituting values for at(a), f3t(l) and 1{l) in Eq. (5.65),
the log-likelihood A{Ct) can be expressed as

(5.71)

This expression can be simplified by using the approximation

(5.72)

where maxiE{1,2, ... ,n} 6i can be computed by successively calculating

(n - 1) maximum function over only two values.
The Log-MAP Algorithm 151

The log-likelihood A(Ct) can now be approximated by

A(Ct) ~ mrx [1';(l; l) + at-1(l') + :Bt(l)]

- mrx [~(l;l) +at-1(l') + :Bt(l)] (5.73)

The computations ofat(l') and :Bt(l) in Eq. (5.73) is equivalent

to the computation of path metrics in the forward and backward
recursions, respectively, in the VA, with the branch metric 11(l: l),
since at(l) and :Bt(l) can be written as
O!t(l) = max{at-1(l') + 1'~(Z: In for 0::; z' ::; Ms - 1, i = 0,1
:Bt(l) = max {:Bt+ 1(z') + 1'~+1 (l, z'n for 0 ::; z' ::; Ms - 1, i = 0,1
The operations involved in computing at (l) and :Bt (l) are the
same as the add-com pare-select operations in the VA. Thus multi-
plications in the MAP are replaced by additions in the Max-Log-
MAP.
For each bit, the Max-Log-MAP algorithm calculates two Viterbi
type metrics and takes the largest one.

5.9 The Log-MAP Algorithm

As the approximation in Eq. (5.72) is used for the computation of
the log-likelihood function A(Ct) the performance of the Max-Log-
MAP is suboptimal. It could be improved by using the Jacobian
algorithm [5]
log(e ll1 + e1l2 ) = max(81 , ( 2 ) + log(1 + e-11l2-(111)
= max(81 , ( 2 ) + fc(18 2 - ( 1 1) (5.74)
where fc(·) is a correction function, which can be implemented using
a look-up table.
The expression log( ell1 + ... + e lln ) can be computed exactly by
a recursive algorithm in Eq. (5.74), as follows
log(e ll1 + ... + elln ) log(6 + elln ) , 6 = e ll1 + ... + elln - 1 = ell
max(log 6, 8n ) + fc(llog 6 - 8n I)
max(8, 8n ) + fc(18 - 8n l) (5.75)
152 Trellis Based Decoding of Linear Codes

The recursive procedure in (5.75) can be applied to evaluating

A(Ct) in (5.71), where

(5.76)

for n = 0,1,···, Ms - 1, and i = 0,1.

The performance of the Log-MAP is identical to the perfor-
mance ofthe MAP algorithm. However, by computing feO at each
step, there is a sacrifice in the low complexity of the Max-Log-MAP.
To simplify the computations, feO is stored in a pre-computed ta-
ble. Since the correction depends only on 16 - 6n I, this table is
one-dimensional.

5.10 Comparison of Decoding

Algorithms
In computation of the log-likelihood for each received symbol, the
MAP algorithm considers all paths in the trellis, but divides them
into two sets, one that has a bit one at time t and the other that
has a bit zero. It then calculates the log-likelihood function for all
of these two sets.
The Max-Log-MAP considers only two paths per step: the best
path with bit zero and the best path with bit one at time t. It
computes the log-likelihood for each of the paths and returns its
difference. From step to step one of these paths might change but
one of them will always be the ML path.
The SOYA also takes two paths. One is the ML path and the
other is the best path with the complementary symbol at time t to
the ML path. Therefore, these two paths are identical to the two
paths considered by the Max-Log-MAP algorithm.
A complexity comparison between MAP, SOYA, Log-MAP and
Max-Log-MAP, per one time unit is illustrated in Table 5.1. Each
algorithm is represented by the number of computation operations
for an (n, k) convolutional code of memory order v. The Log-MAP

°
can be implemented by a look-up table. It has been shown that it
is enough to store 8 values of 162 - 61 1, ranging between and 5
Comparison of Decoding Algorithms 153

[10], with no improvement with a finer representation. Assuming

that one look-up operation costs as much as one addition, we can
conclude that the Log-MAP is 3 times more complex than SOYA.
On the other hand Max-Log-MAP is about twice as complex as
SOYA.

Table 5.1: Decoder complexity comparison

MAP Log-MAP Max-Log-MAP SOYA
add. 2·2"·2" + 6 6·2"·2" +6 4·2"·2" + 8 2·2"·2" +9
multipl. 5· 2k ·2" + 8 2k .2" 2· 2k . 2" 2k .2"
max ops 4·2" - 2 4·2" - 2 2·2" -1
look-ups 4·2" - 2
expo 2· 2k ·2"

The performance of the soft output algorithms cannot be ade-

quately compared in only one stage of decoding by the BER results,
because the BER is defined only for hard decision estimates. Thus
the comparison on the basis of BER refers only to hard decision es-
timates. Fig. 5.15 shows the BER results obtained by simulations
for a 4-state convolutional code with the MAP and SOYA decod-
ing algorithms. The MAP and the Log-MAP are identical, as well
as the Max-Log-MAP and the SOYA. The difference between the
MAP and the SOYA becomes noticeable only at very high BER's,
such as BER is above 10- 2 .
154 Trellis Based Decoding of Linear Codes

1if.-----.------.-----,------,------.-----.------.

MAP
--6-- SOVA

a:
w
<Xl

10-<

1~~----~-----L------~----~----~----~----~
-2 -1 0 2 3 4 5
Eb/No (dB)

Fig. 5.15: Performance comparison of MAP and SOYA

Bibliography

[1] A.J. Viterbi, "Error bounds for convolutional codes and an

asymptotically optimum decoding algorithm", IEEE Trans. In-
form Theory, VoI.lT-13, No.2, pp.260-269, April 1967
[2] B. Vucetic, "Iterative Decoding Algorithms", Invited paper,
The International Symposium on Personal, Indoor and Mo-
bile Radio Communications, PIMRC'97, Sept. 1-4, Helsinki,
Finland.
[3] B. Vucetic, Iterative Decoding Algorithms, Wireless Commu-
nications, S. Glisic and P. Leppnen, Edt., Kluwer, 1997.
[4] G.D. Forney, Jr. "The Viterbi algorithm", Proc. IEEE, Vo1.61,
No.3, pp.268-278, March 1973
[5] J.A. Erfanian, S.Pasupathy and G.Gulot, "Reduced com-
plexity symbol detectors with parallel structures for lSI
channels", IEEE Trans. Commun., Vol. 42, pp. 1661-1671,
Feb./March/ April 1994.
[6] J. Hagenauer and P. Hoeher, "A Viterbi algorithm with soft-
decision outputs and its applications", Proc. Globecom'89,
pp.1680-1686, Nov. 1989.
[7] J. Raviv, "Decision making in Markov chains applied to the
problem of pattern recognition", IEEE Trans. Inform Theory,
VoI.lT-13, pp. 536-551, Oct. 1967.
[8] K. Abend and B.D. Fritchman, "Statistical defection for com-
munication channels with intersymbol interference", Proc.
IEEE, Vol. 58, No.5, pp.779-785, May 1970.
156 BIBLIOGRAPHY

[9J L. Bahl, J.Cocke, F. Jelinek and J. Raviv, "Optimal decoding

of linear codes for minimizing symbol error rate" , IEEE Trans.
Inform Theory, Vol.lT-20, pp.284-287, March 1979
[10] P. Robertson, E. Villebrun and P. Hoeher, "A comparison of
optimal and sub-optimal MAP decoding algorithms operating
in the log domain", Proc. ICC'95, Seattle, June 1995.

[11] R.W. Chang and J.C. Hancock, "On receiver structures for
channels having memory", IEEE Trans. Inform Theory, Vol.
IT-12, pp. 463-468, Oct. 1966.

[12] Y. Li and B. Vucetic, "A Generalized MLSE Algorithm",

INNSP '95, China, December 10-13, 1995.

[13] Y. Li and B. Vucetic, "A Low-complexity Soft-output TDMA

Receiver", TELFOR '95, Belgrade, 3-8 Dec., Yugoslavia.
[14] Y. Li, B. Vucetic and Y. Sato, "Optimum soft-output detection
for channels with inter-symbol inference", IEEE Trans. Inform.
Theory, Vol-41, No.3, May 1995, pp. 704-713.

[15] Y. Li and B. Vucetic, "A low complexity soft output receiver

for cellular mobile applications" , Technique Report,
[16] S. Benedetto, G. Montorsi, D. Divsalar and F. Pollara, "A
soft-input soft-output maximum a posteriori (MAP) module
to decode parallel and serial concatenated codes," JPL TDA
Progress Report 42-127, Nov. 15, 1996. Sep. 1995.

[17J M. Reed and J. Asenstorfer, "A novel variance estimator for

turbo code decoding," in Proc. Int. Conr. on Telecomm., Mel-
bourne, Australia, Apr. 1997.
Chapter 6

Iterative Decoding

Turbo and serial concatenated codes can be decoded by MAP or

ML decoding methods based on the overall code trellis presented
in the previous chapter. These decoders could be implemented for
small interleavers only as they are too complex for medium and
large interleaver sizes. The practical importance of turbo and se-
rial concatenated codes lies in the availability of a simple subopti-
mum decoding algorithm [1 J. In this chapter we present a heuristic
explanation of iterative decoding algorithms for turbo and serial
concatenated codes. There are no proofs of convergence of the iter-
ative decoding methods to the optimum MAP or ML decoding. We
present the performance results of the iterative decoding relative to
the ML performance bounds and some heuristic evidence that the
suboptimum iterative methods can come very close to the optimum
algorithms.

6.1 Optimum Decoding of Turbo Codes

Consider a turbo encoder consisting of two component RSC en-
coders of code rate 1/2, so that the overall code rate is 1/3 (Fig.
6.1). As before, we assume that the information sequence c is a
binary vector with components being independent identically dis-
tributed random variables with common distribution
(6.1)

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
158 Iterative Decoding

c Vo }----rO

Fig. 6.1: Basic turbo encoder

The received sequences ro and rl form the input to the first

decoder, denoted by r'
(6.2)
The deinterleaved received information sequence ro and sequence
r2 form the input to the second decoder, denoted by r"
(6.3)
The optimum bit decision algorithm based on the observations
r' and r" computes the log-likelihood ratio A(Ct) from the overall
turbo code trellis as
Pr (Ct = 1 I r', r")
log Pr (Ct = 0 Ir,
' ' r' ) (6.4)

I:c:Ct=l Pr (r',r" I c) Pr(c)

log I:c:Ct=o r, r "1 c ) Pr ()
Pr (' c
The decision rule is then

(6.5)
Iterative Decoding of Turbo Codes Based on the MAP Algorithm 159

If the random variables r' and r" are uncorrelated we can assume

(6.6)

The log-likelihood ratio A( Ct) then becomes

Ec:ct=IPr (r' Ie) Pr (r" Ic) Pr(c)

A( Ct) = log ( 'I)
Ec:ct=o Pr r c Pr ("r I)c Pr()
c (6.7)

Calculating the log-likelihood ratio A(Ct) from (6.7) requires to

evaluate the sums over all possible transmitted sequences. There-
fore, if we apply the MAP algorithm presented in the previous chap-
ter, its complexity will grow linearly with the number of states of
the code trellis and the information sequence length. However, the
overall trellis for a turbo code is time-varying and has a number of
states that grows exponentially with the interleaver length. MAP
decoding is thus possible for very short interleavers only.

6.2 Iterative Decoding of Turbo Codes

Based on the MAP Algorithm
The iterative turbo decoding [1] consists of two component decoders
serially concatenated via an interleaver, identical to the one in the
encoder, as shown in Fig. 6.2.
The first MAP decoder takes as input the received information
sequence ro and the received parity sequence generated by the first
encoder ri. The decoder then produces a soft output, which is
interleaved and used to produce an improved estimate of the a priori
probabilities of the information sequence for the second decoder.
The other two inputs to the second MAP decoder are the in-
terleaved received information sequence fo and the received parity
sequence produced by the second encoder r2. The second MAP
decoder also produces a soft output which is used to improve the
estimate of the a priori probabilities for the information sequence at
the input of the first MAP decoder. The decoder performance can
be improved by this iterative operation relative to a single operation
160 Iterative Decoding

: Deinterleaver :

MAP
ro Ale . 11
I Interleaver I
Decoder 1

i...-.+ A2e
MAP
. : Interleaver :
ro
Decoder 2 A2

IDeinterleaverl

1
!
Fig. 6.2: An iterative turbo code decoder based on the MAP algorithm

serial concatenated decoder. The feedback loop is a distinguishing

feature of this decoder and the name turbo code is given with ref-
erence to the principle of the turbo engine.
After a certain number of iterations the soft outputs of both
Iterative Decoding of Turbo Codes Based on the MAP Algorithm 161

MAP decoders stop to produce further performance improvements.

Then the last stage of decoding makes a hard decision after dein-
terleaving.
Note that for simplicity in Fig. 6.2 the delays introduced by
two component decoders are not taken into account.
Let us examine the first MAP decoder in the first iteration for
rate ~ component codes. The log-likelihood ratio of the first MAP
decoder is given by

"M.-I a (l')pl(l) exp (_ E:.=;(rt'J- x!,;(I»)2) . R (l)

L..J 1'1=0 t-l t 20-2 fJt

A1(ct) = log "M.-I

,
a
( n-l 0 2)
(l')pl(O) exp _ Ej-o (rt.J-Xt,j(l}) . R (l)
L..J(1=0 t-l t 20-2 fJt

(6.8)
where we introduce the notation p~(l) and pHD) for the a priori
probabilities for 1 and 0 at the input of the first decoder, respec-
tively, since they will differ from the corresponding a priori prob-
abilities at the input of the second decoder denoted by p;(l) and
p;(O).
In the initial decoding operation in the first decoder we assume
that
p~(l) =p~(O) = 1/2 (6.9)

pHI)
Al (ct) = log pHD)

"M.-I ([') (_ (rt,0-x:'o)2+E;~:(rt,j-x!,;(I»)2) . R (l)

L..JI'I=O at-l exp 20-2 fJt

+ log "M.-I
, (l') 0)2 "n-l(
( _ rt,o-xt,o o (»)2)
+L..Jj=o rt,J-xt,j 1 . R (l)
L..J(1=0 at-l exp 20-2 fJt

where (n - 1) is the number of parity digits in the encoded block.

Since the code is systematic x~ 0' i = 0,1, are independent of
the trellis and state t. That is x~,o ' 1 and x~,o = -1. A1(Ct) could
be further decomposed into

(6.10)
162 Iterative Decoding

where
",Ms-l a (z') exp (_ L;~11(rt')-XL(I»)2) . j3 (l)
~I'I=O t-l 20-2 t
A1e(Ct) = log ,
",Ms-l a
(
(l') exp _
n-l 0 2) . j3 (l)
L)=o (rt,j-x t ,) (I»)
~(I=l t-l 20-2 t

(6.11)
Ale(Ct) is called the extrinsic information. It is a function of the
redundant information introduced by the encoder. It does not con-
tain the information decoder input rt,o. This quantity may be used
to improve the a priori probability estimate for the next decoding
stage.
Let us observe the input to the second MAP decoder. Since the
input to the second decoder includes the interleaved version of ro,
°
the received information signal ft correlates with the interleaved
soft output from the first decode~, Al(Ct). Therefore, the contri-
bution due to rt,O must be taken out from Al (et), to eliminate this
correlation.
However, A1e(Ct) does not contain rt,O and it can be used as the
a priori probability for decoding in the second stage. That is, the
interleaved extrinsic information of the first decoder Ale is the a
priori probability estimate for the second decoder
- p;(l)
Ale(Ct) = log p~(O) (6.12)

From Eq. (6.12) and the relationship

p;(l) = 1 - p;(O) (6.13)
we can write for the a priori probabilities in the second decoder
eA1e(Ct)
p; (1) = -----=c,----
1+ eA1e(Ct)
(6.14)

and
(6.15)

In the second decoding stage the MAP decoder estimates the

log-likelihood ratio A2(Ct). Similarly to (6.10) the log-likelihood
ratio for the second MAP decoder can be decomposed into
Iterative Decoding of Thrbo Codes Based on the MAP Algorithm 163

(6.16)

By substituting the a priori probabilities from Eq. (6.12) in Eq.

(6.16), we get

(6.17)

A2e (Ct) is the extrinsic information for the second decoder, which
depends on the redundant information supplied by the second en-
coder, as in Eq. (6.11). The second decoder extrinsic information
can be used as the estimates of the a priori probabilities for the first
decoder as in (6.12). The log-likelihood ratio for the first decoder
can be written as

(6.18)

Summary of the Iterative MAP Decoding Method

1. lni tialize A~~) (Ct) = o.
2. For iterations r = 1,2, ... , I, where I is the total number of
iterations
- Compute A~r)(Ct) and A~r)(Ct) by using Eq. (6.8).
- Compute A~:) (Ct) as

ACr)()
Ie Ct = ACr)()
I Ct -
2
-2 rt 0 -
A-Cr-I)(
2e Ct
) (6.19)
a '

- Compute At) (Ct) as

Cr)()
A2e ACr)()
Ct = 2 Ct -
2-
-2 rt 0 -
A- Cr )()
Ie Ct (6.20)
a '

3. After I iterations make a hard decision on Ct based on XY)(Ct).

164 Iterative Decoding

6.3 The Effect of the Number of Itera-

tions on Turbo Code Performance
Let us consider the performance of turbo codes with random in-
terleavers when the number of iterations in the iterative decoder is
variable. As an example, a rate 1/3, 16 state turbo code with the
generator polynomials go = (37) and gl = (21) and the interleaver
sizes of 4096 and 16384 is simulated. The simulation results based
on the iterative MAP decoding algorithm are shown in Figs. 6.3
and 6.4 for the interleaver sizes of 4096 and 16384, respectively.

• 1=1
-01=3
- x 1=5
- + 1=8
- - • 1=12
- - 0 1=15
- - x 1=18

10-6 L -_ _-'--_ _---'--_ _---'-_ _----"'--_ _-'--_'--=':.......J'

o 01 0.2 03 04 05 0.6
EblNO (dB)

Fig. 6.3: BER performance of a 16 state, rate 1/3 turbo code with
MAP algorithm on an AWGN channel, interleaver size 4096 bits, variable
number of iterations.

Initially, increasing the number of iterations has a considerable

effect on the performance. However, there are diminishing returns
when the number of iterations becomes high. For interleaver size
4096, there is no significant improvement if the number of iterations
increases above 12, while for interleaver size 16384 this threshold is
about 15.
The Effect of the Number of Iterations on Turbo Code Performance 165

10° , - - - - - - , - - - - - - - , - - - - - - - , - - - - , - - - - - - , - - - - - - ,

-01=1
- 01=3
- x 1=5
- + 1=8 ,,
- - 01=12
- - 0 1=15
,,
- - x 1=18 "
"- ,
"
10-''------'--------'--------'----'------'------''
o 0.05 0.1 015 0.2 025 0.3
EblNO (dB)

Fig. 6.4: BER performance of a 16 state, rate 1/3 turbo code with MAP
algorithm on an AWGN channel, interleaver size 16384 bits, variable
number of iterations

As the decoding approaches the performance limit of a given

turbo code, any further iteration results in very little improvement.
Therefore, it is important to devise an efficient criterion to stop
the iteration process and prevent unnecessary computations and
decoding delay. One such stopping criterion has been devised based
on the cross entropy between the distributions of the estimates at
the outputs of the decoders at each iteration [5J [15J [16J. This
criterion is known as cross entropy criterion. It effectively stops the
iteration process with very little performance degradation.
Another method of terminating the iterative decoding process is
to use systematic cyclic redundancy checks (eRe). In the encoder,
the input information is encoded by a eRe code, and the output
of the eRe encoder is passed to the turbo encoder. In the receiver,
the turbo decoder output at each iteration is fed to the eRe error
detector. If the eRe code detects no errors in the decoder output,
the iterative decoding is terminated [17J.
166 Iterative Decoding

6.4 The Effect of Interleaver Size on

Turbo Code Performance
In the original turbo code a pseudo-random block interleaver is used
where information is written row by row and read out following a
non-uniform rule, based on randomly generated numbers. The in-
terleaver length is critical for the code performance, particularly at
low SNR's. The interleaver structure is important for the perfor-
mance at high SNR's, as it affects the distance properties of the
overall turbo code. It determines the code free distance which has
a dominant effect on the asymptotic performance.
As the interleaver enables the information exchange between
the two component decoders, increasing the interleaver size has the
effect of randomizing the information sequence at the input of the
second decoder. Consequently, the inputs to the two component
decoders become less correlated, with respect to noise, improving
the decoding performance.
For feedback component codes interleaving modifies the code
weight distribution relative to the uninterleaved input. This is most
significant for weight 2 information sequence which has the domi-
nant effect on the performance. If an input weight 2 sequence to the
first encoder generates a low weight path in the first code trellis, the
interleaved sequence at the input of the second encoder will produce
a higher weight path, thus improving the code performance.
The simulation results for a rate 1/3, 16-state turbo code with
generator polynomials go = (37) and gl = (21) and a pseudo-
random interleaver of various lengths on an AWGN channel are
shown in Fig. 6.5. The number of iterations for the interleaver sizes
above 2048 was 18. For the interleaver sizes 420 to 2048 it was 8,
as there is no improvement in performance for these interleavers if
the number of iterations is increased above 8.
Clearly, the performance is consistently improved by increasing
the interleaver size. This is consistent with the expression for the
bit error probability, shown in Chapter 4, Eq. (4.38), where the bit
error probability is inversely proportional to the interleaver size.
The Effect of Puncturing Component Codes on 'Thrbo Code Performance167

10· r-----,-----,--T----.--~~~::;:;;:;:;I:;-"l
- • N=420. 1=8
- 0 N=1024.1=8
- x N=2048. 1=8
- - • N=4096.1=18
- - 0 N=8192.1=18
,, - - x N=16384.1=18

, lit

\,
10-2 ~, '
\ '>.. \
~ '< "
\
\
\'
ffi 10-3 k \ \
<II \ III \
~ \
\
~
'\ \"
"\
\ \ \
\ \ "\

\ ~ \:
\ \ '1\
't \ -,
\ - \
\ - \
\ \: \"
~ \- "\
~ \
10-6 L..-_-'-_'2I...--L!.-\_ - - ' -_ _- ' - - - - ' - - - - ' - _ - - ' ' ' ' - _ - - - ' - - _ - - - '
o 02 0.4 06 08 1.4 1.6
EtlINO (dB)

Fig. 6.5: BER performance of a 16 state, rate 1/3 turbo code with
MAP algorithm on an AWGN channel, interleaver size N, the number
of iterations 18

6.5 The Effect of Puncturing Compo-

nent Codes on Turbo Code Perfor-
mance
One way of increasing the rate of a turbo code is by puncturing the
encoder outputs of the basic 1/3 rate turbo code. In this way it is
possible to design rate 1/2, 2/3, 3/4, 5/6 etc. codes.
The performance results for the 16-state, rate 1/2 punctured
turbo code, for the number of iterations of 18 and various interleaver
sizes are shown in Fig. 6.6. They are obtained by simulation with
an iterative MAP decoding method on an AWGN channel. The
puncturing matrix for this code is

p=[~ ~l
168 Iterative Decoding

The code is obtained from the 1/3 rate turbo code with the gener-
ator polynomials go = (37) and gl = (21). The performance loss
relative to the 1/3 rate code for interleaver size 4096 is about 0.70
dB at the bit error rate of 10- 4 .

10' 1-----,-----,~-,---,---,.---__;:::::J;::;::::;:;:::~:;_"l
- • N=16384,1=18
- 0 N=8192, 1=18
- x N=4096, 1=18

0.5 0.6 07 0.8 0.9 1.1 1.2

EblNO (dB)

Fig. 6.6: BER performance of a 16 state, rate 1/2 turbo code with
MAP algorithm on an AWGN channel, interleaver size N, the number
of iterations 18

The simulation results for a rate 2/3, 16-state turbo code, ob-
tained by puncturing the rate 1/2 turbo code in the previous ex-
ample are shown in Fig. 6.7. The puncturing matrix is

p=[ll 10]
101 1
Comparison Between Analytical Upper Bounds and Simulation Results 169

10-'L:,.---"T--.-----r--.---,--~;r::=:;:;::r;::;;~;:,_1
- • N=16384,1=18
- 0 N=8192, 1=18
- x N=4096,1=18

10-4

Fig. 6.7: BER performance of a 16 state, rate 2/3 turbo code with
MAP algorithm on an AWGN channel, interleaver size N, the number
of iterations 18

6.6 Comparison Between Analytical

Upper Bounds and Iterative
Decoding Simulation Results
The upper bound (4.38) derived in Chapter 4 is tight for the bit
error probabilities lower than 10- 4 • This bound is a good estimate
of the turbo codes optimum decoding and as such can be used to
determine how far are the results of the iterative decoding method
from the optimum one. The comparison is shown in Fig. 6.8. They
are obtained for interleaver size 1024.
For the iterative decoding a random interleaver with no con-
straints is chosen and the system is simulated on an AWGN channel,
with MAP component decoders and various numbers of iterations.
Clearly, the results show the convergence of the iterative decod-
ing curves to the optimum decoding bound for increasing numbers
170 Iterative Decoding

10' ,,
\
\
\
\

10-2

~ 10-3

10-4

10-'
--
--&--

-+--
Average bound
it=1
it=3
it=5
it=8
-+- it=10

10-"
0 0.2 0.4 0.6 0.8
EblNo(dB)

Fig. 6.8: Simulation result of a 16 state, rate 1/3 turbo code with MAP,
interleaver size 1024 bits, variable number of iterations I and the theo-
retical bound on an AWGN channel.

of iterations.
Approaching the optimum performance requires a higher num-
ber of iterations for low SNR's. For example, 5 iterations are needed
at the BER of 10-5 , while at BER 10- 3 at least 8 iterations are re-
quired.
It is worth noting that, for particular interleaver designs, it is
possible to achieve better results than for random and uniform in-
terleavers, as discussed in Chapter 7.

6.7 Asymptotic Behavior of Turbo

Codes
At high SNR's the bit error probability of turbo codes is dominated
by the code free distance and it can be expressed as
Iterative SOYA Decoding of Thrbo Codes 171

(6.21)

where B free is the average number of ones on the minimum free

distance path in the overall turbo code trellis, R is the code rate,
Eb is the received bit energy, No is the Gaussian noise one sided
power spectral density, and dfree is the code minimum free distance.
Eq. (6.21) defines the asymptotic behavior of turbo codes.
The value of dfree depends on the generator polynomials and the
interleaver structure. For a uniform interleaver and the 16-state rate
1/3 code with the generator polynomials go = (37) and gl = (21) it
is 9. As this value of d free is relatively low, the curve defined by Eq.
(6.21) has a very low slope at high SNR's, which is often referred
to as the "error floor" .
The simulation results for this code with iterative MAP decod-
ing on an AWGN channel, interleaver size 1024 and 10 iterations
along the theoretical bound in Eq. (6.21) are shown in Fig. 6.9. A
random interleaver is chosen in the simulation. It is clear that the
bound in (6.21) closely determines the code performance at high
SNR's.

6.8 Iterative SOYA Decoding of Turbo

Codes
Let us consider a turbo encoder with rate 1/2 component RSC bi-
nary codes shown in Fig. 6.1. For iterative decoding ofturbo codes
it is possible to use the SOYA algorithm for component decoders
[13] [14]. The block diagram ofthe iterative SOYA decoder is shown
in Fig. 6.10.
The first SOYA decoder generates the soft output and subse-
quently the extrinsic information as in the iterative MAP decoder.
The extrinsic information is interleaved and used by the second
SOYA decoder as the estimate of the a priori probability. The
second SOYA decoder also produces the extrinsic information and
172 Iterative Decoding

Simulation
Lower bound

a:
w
'"

---

10-<lL-_----'-_ _--'-_ _---'---_ _.L-_--'-_ _--'-_----'

o 0.2 04 0.6 O.B 12 1.4
EblNo (dB)

Fig. 6.9: Simulation result of a 16 state, rate 1/3 turbo code with MAP,
interleaver size 1024 bits, the number of iterations 10 and the theoretical
bound on an AWGN channel.

passes it after deinterleaving to the first SOYA decoder to be used

in the next decoding operation.
The expression for the path metric of the SOYA was derived by
maximizing the a posteriori probability, Pr { c, r}

(6.22)

or its logarithm

logPr{c,r} = logPr{c} + logPr{r I c} (6.23)

By assuming that the channel is subject to independent Gaus-

sian noise, we get for the logarithm of Pr { c, r}
nT
L
T

log Pr { c, r} = logpt(Ct) - -log(27r)

t=l 2
n-l ( r t ' - Xt' )2
LL
T

- nTlogcy - ,z 2 ,z (6.24)
t=l i=O 2cy
Iterative SOYA Decoding of Turbo Codes 173

where the notation is the same as in the previous chapter. Assuming

that the a priori probabilities Pt(Ct) might change with the iteration
number in an iterative decoding procedure, maximizing log Pr ( c, r)
is equivalent to maximizing
T T n-l
2: logpt(Ct) - 2: 2: (rt,i - Xt,i)2 (6.25)
t=l t=l i=O

Consequently, we define the branch metric, assigned to a trellis

branch at time t, as
n-l
vft = 2: (rt,i - Xt,i)2 - logpt (Ct) (6.26)
i=O

Similarly, we define the path metric, for a path x in the trellis,

as
(6.27)

The SOYA decoder selects a path with the minimum path met-
ric. A SOYA decoder in the iterative scheme shown in Fig. 6.10
computes the soft output as

(6.28)

where J-L~ and J-L} are the minimum path metrics with the symbol
o and symbol 1, respectively, at time t, J-LT,min is the maximum
likelihood path metric and J-Lt,c is the best competitor path metric,
as described in Section 5.5. They can be computed by the following
procedure.
At time t, the maximum likelihood estimate Ct is obtained from
the maximum likelihood path J.lt,cp computed by using the Viterbi
algorithm, as

J-LT,min
t-l T

2: Vt' + vft + 2: Vt' (6.29)

t'=O t'=t+l

J.l~ + v~t (6.30)

174 Iterative Decoding

: Deinterleaver 1

SOYA Ale I ,
rO -, Interleaver ,
Decoder 1

: Interleaver :
ro
- SOYA

Decoder 2
A 2e

A2

IDeinterleaverl

1
l
Fig. 6.10: An iterative turbo code decoder based on the SOYA algorithm

where

t-l T

M~ = L lit' + L lit' (6.31)

t'=O t'=t+l
Iterative SOYA Decoding of Turbo Codes 175

and vft is given by

n-l
v~t = L (rt,i - X~:i)2 - logpt(ct) (6.32)
i=O
where x~ti, is the ith modulated coded signal in the trellis, generated
by input Ct.
Next, the path metric of the best competitor J.1t,e, where C =
Ct EB 1, can be computed as

J.1t,e = min {J.1L 1 ([') + v~ ([' , l) + J.1~ (l) } (6.33)

where l', l = 0,1, ... , Ms - 1, J.1Ll (l') is the path metric of the
forward survivor at time t - 1 and node l', J.1~(l) is the path metric
of the backward survivor at time t, Ms is the number of nodes in the
trellis and vf(l', l) is the branch metric of the complement branch
from node l' to l. J.1t,e can be represented as
e
+ Vt (6.34)
/I
J.1t,e = J.1t
where J.1~ is a positive number, similar to (6.31), and vf is given by
n-l
v~ = L (rt,i - X~,i)2 - logpt(c) (6.35)
i=O

where X~,i is the ith modulated coded signal in the trellis, generated
by input c.
By substituting the path metrics from Eqs. (6.30) and (6.34)
into Eq. (6.28), we can compute the soft output for the first de-
coder, Al(ct), as

(6.36)
h
were J.1t,l' et J.1t,l an de
' vt,l, /I
Vt,l reler
C
to t h e quantItIes
.. J.1t,
' vtet , J.1t an d
/I

vi, for the first decoder, respectively.

By substituting vft and vi from Eqs. (6.32) and (6.35), respec-
tively, into (6.36) and observing that x~o = -1 and xto = 1, we
obtain for AI(ct) "
pt(l)
Al(cd = log pHO) + 4rt,O + Ale(ct) (6.37)
176 Iterative Decoding

where A1e(Ct) is the extrinsic information for the first decoder given
by

A1e(Ct) = (_1)c t {jt~'l - jt~,l - 2 ~ (X~:i - X~,i) } (6.38)

and pi(l) and pi(O) are the a priori probabilities for the binary 1
and 0, respectively, at the input of the first decoder.
The interleaved extrinsic information of the first decoder im-
proves the a priori probabilities at the input of the second SOYA
decoder, as in the MAP iterative decoder

(6.39)

where pt(l) and pt(O) are a priori probabilities for the binary 1 and
o at the input of the second decoder, at time t, respectively, which
can be estimated by Eqs. (6.14) and (6.15).
The second decoder computes the soft output A 2(Ct) which can
be written as

(6.40)

where A 2e (Ct) is the extrinsic information from the first decoder,

given

A2e (Ct) = (-1)c t {jt~'2 - jt~,2 - 2 ~ (X~:i - X~,i) } (6.41)

where jt~ ,2 and jt~,2 refer to the quantities jt~ and jt~ , for the second
decoder, respectively.
Substituting the extrinsic information from the first decoder,
A1e(Ct) from Eq. (6.39) into Eq. (6.40) we get for A 2 (Ct)
(6.42)
The interleaved extrinsic information from the second decoder,
A2e (Ct), is used as the estimate of the logarithm of the encoder input
probabilities in the next iteration

(6.43)
Comparison of MAP and SOYA Iterative Decoding Algorithms 177

so that the log-likelihood ratio A 1(Ct) in the next iteration can be

written as
(6.44)

Summary of the Iterative SOYA Decoding Method

1. Ini tialize A~~ (Ct) = O.

2. For iterations r = 1,2, ... ,I,
- Compute A~r)(Ct) and A~r)(Ct) by using Eq. (6.28).
- Compute Al:)(Ct) as

A(r)()
Ie Ct =
A(r)()
1 Ct - 4rt,O - A-(r-I)()
2e Ct (6.45)

(r)()
A 2e Ct = A(r)()
2 Ct - r t,O - A-(r)()
4- Ie Ct (6.46)

3. After I iterations, make a hard decision on Ct based on A2 (Ct).

6.9 Comparison of MAP and SOYA It-

erative Decoding Algorithms
The bit error probabilities for the rate 1/3, 16-state turbo code
decoded by the iterative MAP, Log-MAP and SOYA methods are
shown in Fig. 6.11, for interleaver size 4096 and 18 iterations. The
SOYA has a worse performance of 0.55 dB at the BER of lD- 5 .
The complexity of the iterative SOYA is lower by the same factor
compared to the iterative MAP as the complexity of the SOYA
relative to the MAP, and these two algorithms are compared in
Section 5.lD.
178 Iterative Decoding

I--MAP
-
-
0 Logmap
+ SOVA

a:
w
III

Fig. 6.11: BER performance of a 16 state, rate 1/3 turbo code with MAP,
Log-MAP and SOYA algorithm on an AWGN channel, interleaver size
4096 bits, the number of iterations 18

Component decoders can be implemented also as Log-MAP or

Max-Log-MAP decoders, with the iterative procedure similar to the
iterative MAP.
As discussed in Section 5.9, the performance of the Log-MAP
is almost identical to the MAP, while the performance of Max-Log-
MAP is identical to the performance of the SOYA.

6.10 Iterative MAP Decoding of Serial

Concatenated Convolutional
Codes
The block diagram of an iterative MAP decoder for serial concate-
nated codes is shown in Fig. 6.12.
Following the design rules, as described in Chapter 4, we assume
that the inner code is an (ni' k i ) RSC code and that the outer code
Iterative MAP Decoding of Serial Concatenated Convolutional Codes 179

r r r c
MAP MAP
Deinter- outer
Aoe inner A,e A,e Aoe
decoder leaver decoder

Interleaver

Fig. 6.12: Iterative MAP decoder for serial concatenated codes

is in general an (no, k o) nonsystematic feedforward convolutional

code.
In the first iteration the MAP inner decoder computes the log-
likelihood ratio Ai(Ct,j), assuming that the binary message compo-
nents are equally likely, in the form
- 2
Ai( Ct,j) = Aoe( Ct,j) + a 2 rt,j + Aie( Ct,j) (6.47)

where j = 0,1,"', k i - 1, Aoe(Ct,j) is the interleaved extrinsic in-

formation of output bits conditioned on the constraint of the outer
decoder_ and Aie (Ct,j) is the extrinsic information for the inner de-
coder. Aoe(Ct,j) is assumed to be zero in the first iteration.
The extrinsic information and decoded soft output information
sequence from the first decoder are deinterleaved and passed to the
second decoder.
The deinterleaved information sequence of the inner code de-
noted by r', is the received coded sequence for the outer code. The
extrinsic information sequence from the inner decoder is the dein-
terleaved logarithm of the a priori probability ratio of the input to
the outer decoder. The log-likelihood ratio of the outer decoder can
be written as
Ao(Ct,j) = Aie(Ct,j) + Aoe(Ct,j) (6.48)
where j = 0,1"", no - 1. Note that as the outer code is non-
systematic, it is not possible to extract the information bit at the
180 Iterative Decoding

output of the outer decoder, so the soft output does not contain
the contribution of the information bit.
The extrinsic information Aoe (Ct,j) is interleaved and passed to
the first encoder as the a priori probability for the next iteration.

Summary of the Iterative MAP Algorithm for Serial

Concatenated Codes

1. Initialize Aoe (Ct,j) = O.

2. For iterations r = 1,2, ... ,I,
- Compute A~r)(Ct,j) and A~r)(Ct,j) by using Eqs. (6.47)
and (6.48).
- Compute A~:)(Ct,j) as

(r) -
A ie -
A(r)(
i Ct,).) _ ~
a 2 r t,). _ A-(r-l)(
oe Ct,).) (6.49)

where j = 0,1, .. " k i - 1.

- Compute A~~) (Ct,j) as

oe ,) = A(r)(Ct')
A(r)(Ct') 0 ,)
- x(r)(Ct')
ze ,) (6.50)
where j = 0,1, .. " no - 1.

3. After I iterations, make a hard decision on Ct,j based on

A~(Ct,j).

6.11 Iterative SOYA Decoding of Serial

Concatenated Convolutional
Codes
The block diagram of a SOYA iterative decoder for serial concate-
nated convolutional codes is shown in Fig. 6.13.
Iterative SOYA Decoding of Serial Concatenated Convolutional Codes 181

r r r c
SOVA SOVA
Deinter-
Aoe inner
decoder
A ..
leaver A. e outer
decoder
Aoe

Interieaver

Fig. 6.13: Iterative SOYA decoder for serial concatenated codes

The soft output of the inner decoder can be expressed as

J. = 0" 1 ... k· - 1 (6 .51)
,~

where Aoe(Ct,j) is the interleaved extrinsic information from the

outer decoder, which is assumed to be zero in the first iteration.
The received coded sequence for the outer code, r', and the
extrinsic information Aie(Ct,j) are passed to the outer decoder. The
soft output of the outer decoder can be written by
(6.52)

Summary of the Iterative SOYA for Serial Concatenated

Codes

2. For iterations r = 1,2, ... ,I,

- Compute A}r)(Ct,j) and A~r)(Ct,j) by using Eqs. (6.51)
and (6.52).
- Compute A}:)(Ct,j) as
Aie(r)( Ct,j ) -- A(r)(
i Ct,j ) - 4r t,j - A-(r-l)()
oe Ct,j (6.53)
where j = 0, 1, .. " ki - 1.
182 Iterative Decoding

- Compute A(r)(Ct
oe ,).) as

(r)( Ct,).) -- A(r)(

Aoe 0 Ct,)
.) _ A-(r)(
ie Ct,)
.) (6.54)

where j = 0,1, ... , no - 1.

3. After I iterations, make a hard decision on Ct,j based on

A~( Ct,j).

6.12 Serial Concatenated Convolutional

Codes with Iterative Decoding
In this section we present the performance of serial concatenated
codes with variable interleaver sizes, memory order and number of
iterations for MAP and SOYA decoding.
We consider serial concatenated codes with two component codes,
an inner and an outer code.
The inner code was selected to be a recursive convolutional code,
which is for a given memory order designed to have a maximum ef-
fective free distance. The choice of a recursive code enables the
benefit of the interleaving gain, and the effective free distance cri-
terion gives the optimum performance in the BER range of interest
in practical applications. The outer code was chosen to have a
maximum free distance for a given memory order, resulting in the
optimum performance at very large SNR's.

6.12.1 The Effect of Interleaver Size and the

N umber of Iterations on AWG N Chan-
nels
If the inner code in a serial concatenated scheme is a recursive con-
volutional code the bit error probability is proportional to N- dj/2
for even values of the outer code minimum free distance d, and
Serial Concatenated Convolutional Codes with Iterative Decoding 183

to N-(dj+1)/2 for odd values of df . That is, by increasing the in-

terleaver size, the bit error probability is reduced for this class of
codes. For turbo codes there is a virtual error floor at high SNR's
and by increasing the interleaver size it is possible to reduce the
error floor. Serial concatenated codes due to much higher inter-
leaver gain and minimum free distance do not exhibit an error floor
in the BER range of interest in data communications. From the
expressions for the BER in Eq. (6.21) it can be concluded that the
asymptotic behavior of the error probability is determined by the
overall code free distance and the interleaver size.
The impact of the interleaver size on serial concatenated codes
on AWGN channels is shown in Fig. 6.14. The serial concatenated
code consists of a rate 2/3, 4-state recursive systematic convolu-
tional code as the inner code and a rate 1/2, 4-state feedforward
convolutional code as the outer code, so the overall code rate is 1/3.
The generator polynomial for the outer code, denoted by Go(D), is
given by

(6.55)
The generator polynomial for the inner code, denoted by Gi(D), is
given by
Gi(D) = [ 1
0 ltD2
ltft1p2
1 (6.56)
o 1 ltD+D2
As the simulation results indicate, the bit error probability can
be reduced by increasing the interleaver size.
Comparison of turbo and serial concatenated codes is illustrated
in Fig. 6.15. This figure shows the turbo code outperforms the
corresponding serial concatenated code for the BER above 10- 5 .
At lower BER's, the serial concatenated code is superior, since the
BER of turbo code levels out in this region due to the small free
distance, while the BER of serial concatenated code has no error
floor even at the BER of lO- lO •
The effect of the variable number of iterations on the BER per-
formance on AWGN channels is shown in Fig. 6.16. As with the
turbo codes, increasing the number of iterations reduces the BER,
up to a certain threshold, which increases with the interleaver size.
184 Iterative Decoding

10-'k---.-----.------,r----;=~~:==,___,
_ . N=512
- x N=1024
- 0 N=2048
- - • N=4096
- - x N=8192
- - 0 N=16384
,
I '
10-3 \ ''\
\ 1io
\ '\
\ "

~ 10-4
'-
,
,
I
\.
~
..,
\ ,\
\
if

10-' I
I
I
,
I
<i1 I
I
'\ I
I
if

10'" \
~

,
I
,
\ \

,
I
\ \ \
\
I
10-7 \
15 2 2.5 3
EblNO (dB)

Fig. 6.14: Performance of a rate 1/3 serial concatenated code, with a

rate 1/2, 4 state nonrecursive convolutional code as the outer code, a
rate 2/3, 4 state recursive convolutional code as the inner code, AWGN
channel, SOYA decoding algorithm, various interleaver size N, and the
number of iterations 20

For interleaver size 1024, there is no further improvement above

the number of iterations of 12, while for interleaver size 4096 and
higher the threshold is 18.

6.12.2 The Effect of Memory Order on AWGN

channels
Increasing the memory order of component codes generally results
in increased free distance and effective free distance. As these two
parameters determine the turbo code performance at high SNR's
increasing the memory order will give the improved asymptotic er-
ror performance.
At low SNR's, the performance of turbo codes is dominated by
error coefficients, which are generally higher for codes with higher
Serial Concatenated Convolutional Codes with Iterative Decoding 185

10'
- • Turbo l=lB
- 0 Seriall=lB

10-'

10-'

ffiIII 10-3

10-'

10- 5

10-'
0.6 0.7 O.B 09 1 11 1.2 1.3 1.4
EblNO (dB)

Fig. 6.15: Comparison of a rate 1/3, memory order 2 turbo code with
interleaver size 4096 bits and a rate 1/3 serial concatenated code with
memory order 2 outer code, interleaver size 4096 bits on an AWGN
channel, SOYA decoding algorithm, the number of iterations 18.

memory orders. The effect of memory order on the turbo code

performance is illustrated in Figs. 6.17 and 6.18, for two different
interleaver sizes. For interleaver size 1024 bits, the memory order
4 code outperforms the memory order 3 and memory order 2 codes
for E b / No values above 1.5 dB. However increasing the interleaver
size from 1024 to 4096, reduces the error coefficients in the same
proportion, and thus the effective free distance becomes the domi-
nant parameter at lower E b / No, than in the previous case, so that
the memory order 4 code has the best performance above E b / No of
1 dB.
In iterative decoding of serial concatenated codes there is an
additional effect of error propagation between the inner and outer
decoder, which in general is more prominent with higher memory
order codes. The error performance of rate 1/3 serial concatenated
codes with variable memory order of the outer code is shown in Fig.
6.19, for interleaver size 1024. Clearly, in the Eb/No region up to
186 Iterative Decoding

10·

10-'

::::::2~~__ :
10-'
................ -
....... '"'Q.... .... ,.. •
.... ~.:- ... .:-~
.... ... -Sc.... :: .... ...
. ................ . .
: . . )C.: . . . . . .
....... ""0... .... ...

-"
10"
~
a: ............ ~ ........
w
III , ....
.....................
10-' , , '

.."
........ '$.... ......... .
'1<...' , .
..........

10-'
.... '
. . .:-e. , . .
.....
-·1=1 ,,.,
- 0 1=3
- x 1=5
.. " "
- + 1=8 '~
10"" - - • 1=12
- - 0 1=15
- - x 1=18
10-7':--_-::'::-_---:':--_-:":-_-'-_ _-'-_-'-_ _~_...J
0.6 07 08 09 1 1.1 1.2 1.3 1.4
EblNO{dB)

Fig. 6.16: BER performance of a rate 1/3 serial concatenated code with
rate 1/2, 4 state outer code and rate 2/3, 4 state inner code with SOYA
algorithm on an AWGN channel, interleaver size 4096 bits, variable num-
ber of iterations

1.8 dB, increasing the memory order results in worse performance.

Increasing the interleaver size to 4096, diminishes the effect of er-
ror coefficients, so that for E b / No above 1.5 dB the code with the
highest memory order has the best performance, as shown in Fig.
6.20.

6.12.3 Comparison of MAP and SOYA Decod-

ing Algorithms on AWGN Channels
The description of the MAP and SOYA iterative decoding algo-
rithms for serial concatenated codes is presented in Sections 6.10
and 6.11.
The comparison of the MAP and SOYA decoding algorithms for
a rate 1/3 serial concatenated code on an AWGN channel, obtained
Serial Concatenated Convolutional Codes with Iterative Decoding 187

10-7'-----'L-----'-----'----'-----'----'--~--'--_'_--'
0.6 0.7 0.8 0.9 1.1 1.2 1.3 1.4 1.5 1.6
EIlINO (dB)

Fig. 6.17: Comparison of a rate 1/3 turbo code for different memory
order with SOYA algorithm on an AWGN channel, interleaver size 1024
bits, the number of iterations 12

by simulation, is illustrated in Fig. 6.21. The serial concatenated

code consists of a rate 2/3, 4 state RSC code as the inner code
and a rate 1/2 16 state feedforward convolutional code as the outer
code. The interleaver size is 1024. The figure shows that the MAP
algorithm outperforms SOYA at the bit error probability of 10-4 by
about 0.9 dB. Since ideal estimation of the Gaussian noise variance
is assumed, this difference in reality might be smaller, as the MAP
algorithm requires knowledge of the noise variance, while the SOYA
does not.
188 Iterative Decoding

10-'

10'"

10~L6--0:-':7----:"08::------::-0,':-9---'---1-'-1--'':-----,L,-~1.J...4--'15---.J
16
EbiNO (dB)

Fig. 6.18: Comparison of a rate 1/3 turbo code for different memory
order with SOYA algorithm on an AWGN channel, interleaver size 4096
bits, the number of iterations 18

1O-5~_.l....-_-'-_--'--_--'-_--'-_---'_ _'--_.l....-_-'----',-"
080,9 1,1 1,2131.4 15 1.6
EbiNO (dB)

Fig. 6.19: Comparison of a rate 1/3 serial concatenated code for different
outer code memory order with SOYA algorithm on an AWGN channel,
interleaver size 1024 bits, the number of iterations 12.
Serial Concatenated Convolutional Codes with Iterative Decoding 189

9 ---1---,"'":.,---,':,2:----,".3::----:-'1.-._ _':-..>...0.-"'.
'O-~L..8------::0.L..
EbiNO (dB)

Fig. 6.20: Comparison of a rate 1/3 serial concatenated code for different
outer code memory order with SOVA algorithm on an AWGN channel,
interleaver size 4096 bits, the number of iterations 18

- - "MAP, 1=12
- - "SOVA, 1=12

II:
~
10-3

o 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

EblNO (dB)

Fig. 6.21: Performance comparison of MAP and SOVA for a rate 1/3
serial concatenated convolutional code
Bibliography

[1] C. Berrou, A. Glavieux and P. Thitimajshima, "Near Shannon

limit error-correcting coding and decoding: turbo-codes (1),"
in Proc. ICC'93, May 1993.
[2] G. Ungerboeck, "Channel coding with multilevel/phase signal-
ing", IEEE Trans. Information Theory, Vol. 25, pp. 55-67, Jan.
1982.
[3] J. Hagenauer, P. Robertson and L. Papke, "Iterative ('Turbo')
decoding of systematic convolutional codes with the MAP and
SOVA algorithms", in Proc. of the 1994 ITG Conference on
Source and Channel Coding, Munich, October 1994.
[4] B. Vucetic, "Iterative decoding algorithms", PIMRC'97, pp
99-120, Sept. 1997, Helsinki, Finland.
[5] J. Hagenauer, E. Offer and L. Papke, "Iterative decoding of
binary block and convolutional codes", IEEE Trans. Inform.
Theory, Vol. 42, No.2, March 1996, pp. 429-445.
[6] G. Battail, M. C. Decouvelaere, and P. Godlewski, "Replica-
tion decoding", IEEE Trans. Inform. Theory, Vol. IT-25, pp.
332-345, May 1979.
[7] J. Lodge, R. Young, P. Hoeher and J. Hagenauer, "Separable
MAP "filters" for the decoding of product and concatenated
codes", Proc. IEEE ICC'93, Geneva, Switzerland, may 1993,
pp. 1740-1745.
[8] P. Robertson, "Illumunating the structure of decoders of par-
allel concatenated recursive systematic (turbo) codes, Proc.
192 BIBLIOGRAPHY

IEEE Globecom Conf, San Francisco, CA, Dec. 1994, pp. 1298-
1303.

[9] P. Robertson, E. Villebrun and P. Hoeher, " A comparison of

optimal and sub-optimal MAP decoding algorithms operating
in the log domain", Proc. IEEE ICC'95, Seattle, WA, June
1995, pp. 1009-1013.

[10] S. Benedetto, G. Montorsi, D. Divsalar and F. Pollara, A soft-

input soft-output maximum a posteriori (MAP) module to de-
code parallel and serial concatenated codes, TDA Progress Re-
port 42-127.

[11] P. Jung, "Novel low complexity decoder for turbo codes", Elec-
tronics Letters, 19th Jan. 1995, Vol. 31, No.2, pp. 86-87.

[12] A. Ushirokawa, T. Okamura, N. Kamiya and B. Vucetic, "Prin-

ciples of turbo codes and their application to mobile commu-
nications," IEICE, Trans. Fundamentals, Vol. E81-A, No.7,
July 1998, pp. 1320-1329.

[13] B. Vucetic, Coding for fading channels, Lecture Notes, The

University of Sydney, Sydney.

[14] W. Feng and B. Vucetic, "A list bidirectional soft output de-
coder of turbo codes," in Proc. Int. Symp. on Turbo Codes
and Related Topics, Brest, France, Sep. 1997, pp. 288-292.

[15] M. Moher, "Decoding via cross entropy minimization," in

Proc. IEEE Globecom. Conf., Houston, TX, Dec. 1993, pp.
809-813.
[16] R. Shao, S. Lin, and M. Fossorier, "Two simple stopping cri-
teria for turbo decoding," IEEE Trans. Commun., vol. 47, no.
8, Aug. 1999, pp. 1117-1120.
[17] B. Vucetic, W. Feng, and J. Yuan, "Performance of turbo
and serial concatenated convolutional codes," Technical Re-
port, The University of Sydney, Aug. 1997.
Chapter 7

Interleavers

In the previous chapters we have presented the design and perfor-

mance analysis of turbo codes. Interleaving plays an important role
in achieving good performance of these schemes.
In this chapter, we consider interleaving techniques for turbo
coding, the effects of interleaving size and structure on code per-
formance and interleaver design for turbo codes. In particular, we
discuss four types of interleaving structures: block interleavers, con-
volutional interleavers, random interleavers and code matched in-
terleavers. A code matched interleaver design for turbo codes is
proposed. We show that, with code matched interleavers, the turbo
code error performance at moderate to high SNR's is improved and
the "error floor" is reduced significantly compared to random inter-
leavers. A new class of convolutional interleavers, called cyclic shift
interleavers, is discussed. Their distinguishing features are low de-
sign complexity and memory, while the performance is comparable
to random interleavers.

7 .1 Interleaving
Interleaving is a process of rearranging the ordering of a data se-
quence in a one-to-one deterministic format. The inverse of this
process is deinterleaving which restores the received sequence to its
original order.

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
194 Interleavers

An interleaver device with size N is shown in Fig. 7.1. For

simplicity, we assume that the data sequence at the input of the
interleaver I is binary, given by

(7.1)
where Ci E {O, I}, 1 ~ i ~ N. The interleaver permutes the
sequence c to a binary sequence

(7.2)

where Cj E {O, I}, 1 ~ j ~ N. The sequence c has all the elements

of c but in a different order. If we consider the input sequence c
and the output sequence c as a pair of sets with N elements, there
is one-to-one correspondence Ci ~ Cj between each element of c
and each element of c.
Let us define a set A as

A = {I " 2 ... , N} (7.3)

The interleaver can then be defined by a one-to-one index mapping

function
7I"(A -t A) : j = 71" (i) , i, j E A (7.4)
where i and j are the index of an element in the original sequence c
and the interleaved sequence c, respectively. The mapping function
can be represented by an interleaving vector

7I"N = (71"(1),71"(2),71"(3),···, 7I"(N)) (7.5)

Fig. 7.1: An interleaver device

For example, we consider a pseudo-random interleaver with size

N = 8. The input sequence is represented by
Interleaving with Error Control Coding 195

The interleaved sequence is given by

~ (~,~,~,~,~,~,~,~)
(C2, C4, Cl, C6, C3, Cs, C5, C7)

The mapping function is illustrated in Fig. 7.2. The interleaving

vector can be expressed as

7fs (7f(1), 7f(2), 7f(3), 7f(4), 7f(5), 7f(5), 7f(7), 7f(8))

(3,1,5,2,7,4,8,6)

Fig. 7.2: An interleaver mapping

An interleaver or a deinterleaver is characterized by its delay

and storage capacity. The interleaving or deinterleaving delay is
the maximum delay encountered by any symbol before it is inserted
into the output sequence. The storage capacity is the number of
symbols stored by the interleaver or deinterleaver.

7.2 Interleaving with Error Control

Coding
Interleaving is a practical technique to enhance the error correcting
capability of coding. It has been widely used in conjunction with
error control coding for channels that exhibit bursty error charac-
teristics [1]. One example is a multipath fading channel in which
signal variations due to multipath propagation often causes the sig-
nal to fall below the noise level, thus resulting in a large number of
errors. A second example is a magnetic recording channel (tape or
196 Interleavers

disk) in which defects in the recorcing media result in clusters of

errors.
An effective method to cope with burst errors is to insert an in-
terleaver between the channel encoder and the channel. The coded
data are reordered by the interleaver and then transmitted over the
channel. At the receiver, the deinterleaver performs the reverse op-
eration to restore the data to its original order. As a result of the
interleaving/ deinterleaving operation, bursty errors are spread out
in time so that errors within a codeword appear independent. Thus,
the burst error channel is transformed into a random error channel
at the input of the decoder and a code designed for independent
error channels could be used for burst error channels.
Another application of interleaving is for generation of product
codes. Interleavers/deinterleavers are placed between successive en-
coders/decoders with the objective of decorrelating the component
decoder input signals. Long product codes can be easily decoded
in multiple stages. They provide a substantial improvement in per-
formance when compared with one-stage codes.

7.3 Interleaving in Turbo Coding

In turbo coding, interleaving is employed before the information
data is encoded by the second component encoder. In general, the
interleaver size N is significantly larger than the code memory v
and the interleaver vector elements are chosen randomly.
The basic role of the interleaver is to construct a long block
code from small memory convolutional codes, as long codes can ap-
proach the Shannon capacity limit. Secondly, it spreads out burst
errors. The interleaver provides "scrambled" information data to
the second component encoder and decorrelates the inputs to the
two component decoders so that an iterative suboptimum decoding
algorithm based on "un correlated" information exchange between
the two component decoders can be applied. For example, after cor-
rection of some of the errors in the first component decoder, some of
the remaining errors can be spread by the interleaver such that they
become correctable in the other decoder. By increasing the number
of iterations in the decoding process the bit error probability ap-
Interleaving in Turbo Coding 197

proaches the channel capacity. The final role of the interleaver is to

break low weight input sequences, and hence increase the code free
Hamming distance or reduce the number of codewords with small
distances in the code distance spectrum.

7.3.1 The Effect of Interleaver Size on Code

Performance
The turbo code error performance is determined by the code dis-
tance spectrum. The interleaver in a turbo encoder can reduce the
error coefficients of low weight codewords through a process called
"spectral thinning" [2J. This distance spectrum results in a reduced
bit error probability by a factor liN, which is called the interleav-
ing gain [8J. The turbo code error performance at low SNR's is
dominated by the interleaver size.
To illustrate the effect of the interleaver size on the code er-
ror performance, we consider a memory order 2 turbo code with
generator matrix (1, 5/7)(oct).
As shown in Chapter 4, the bit error probability of a turbo code
over an additive white Gaussian noise channel is upper-bounded by

(7.6)

where Ed is the error coefficient and the set of all pairs of (d, Ed)
represents the code distance spectrum.
The distance spectra of the turbo code with various interleaver
sizes are calculated and substituted in (7.6) to calculate the bit
error probability. The results are shown in Figs. 7.3 and Fig. 7.4
for the interleave sizes of 128, 256 and 512.
It is apparent from Fig. 7.3 that the shapes of the distance
spectra for the turbo code with various interleaver sizes are quite
similar. However, for significant spectral lines which dominate the
turbo code performance, as discussed in Section 4.4, the error co-
efficients decrease with increasing interleaver size. The spectral
thinning manifests in significant performance improvements pro-
portional to the interleaver gain. As Fig. 7.4 shows the interleaver
gain grows linearly with the interleaver size.
198 Interleavers

10200 ,---,---,------,,------,,----,----,----,

N=128
N=256
N=512

10"JO
\
/ \
.,t- .....
E // '
" /,.... " ,
~ 10
/
50 //

8 // \
\
g \ \
w \ \
10' \ \
\

10- '00 " - -_ _" - -_ _' - - - _ - - " ' - - - _ - '_ _- '_ _- - - '_ _-------'
o 200 400 600 800 1000 1200 1400
Hamming distance

Fig. 7.3: Distance spectra for a turbo code with various interleaver sizes

7.3.2 The Effect of Interleaver Structure on

Code Performance
As we discussed in the previous chapter, the turbo code perfor-
mance at high SNR's is dominated by the code first several distance
spectral lines which are produced by low weight input sequences.
The interleaver structure affects the mapping of low weight input
sequences to the interleaver output, and hence the first several dis-
tance spectral lines of the turbo code distance spectrum. It plays an
important role in determining the code performance at high SNR's.
For example, we consider an input sequence to the first compo-
nent encoder generating a low weight parity check sequence. It is
desirable that the interleaver is capable of breaking this input pat-
tern. That means that the interleaver does not produce the same
input pattern to the second encoder, or an input sequence which
generates a finite weight code sequence. In such a case, the input
sequence to the second component encoder will most likely produce
a high weight parity check sequence. This will result in an increase
Interleaving in Turbo Coding 199

10· ~-~-----r--------,----------,
\ :

N=128
N=256
:,
\

'-
\ N=512
:\
\ \
\ \
:11
. ';1
\
I'
\
\ \
\ \

--
'. \ ,
\

\
\
\
"- "-
... -
---
10-6 L -_ _ _ _ _ _...l.-_ _ _ _ _ _- ' -_ _ _ _ _ __
15 2 2.5 3
Eb/No (dB)

Fig. 7.4: Bit error probability upper bounds for a turbo code with various
interleaver sizes

in the turbo codeword weight. If an interleaver is designed to break

the low weight input sequences so that the resulting turbo code
has a large minimum free distance, the error performance at high
SNR's can be improved.

The previous analysis and discussion dearly show that the in-
terleaver size and structure affect the turbo code error performance
considerably. At low SNR's, the interleaver size is the only impor-
tant factor, as the code performance is dominated by the interleaver
gain. The effects induced by changing the interleaver structure at
low SNR region are not significant. However, both the interleaver
size and structure affect the turbo code minimum free distance and
first several distance spectral lines. They play an important role
in determining the code performance at high SNR's, and conse-
quently, the asymptotic performance of the turbo code. It is possi-
ble to design particular interleavers which can result in good code
performance at high SNR's. This is achieved by breaking several
200 Interleavers

low weight input patterns that produce low weight codewords in

the overall code.
In the interleaver design, we first consider weight-2 input se-
quences as the most likely to generate low weight codewords. For
example, the weight-2 input patterns for a turbo code with gener-
ator matrix (1,5/7)(oct) are 1001, 100001, etc. A good interleaver
ought to break as many of these input patterns as possible.

7.3.3 Interleaving Techniques

The use of pseudo-random interleavers does playa fundamental role
in turbo coding schemes. As the pseudo-random structure is a ma-
jor obstacle in the interleaver performance analysis and design, the
use of the uniform interleavers, which are based on the probabilis-
tic analysis of the ensemble of all interleavers, has been effective.
However, in practice, one has to choose a particular interleaver
structure which might perform better than the uniform interleaver.
In the following sections, we will consider four types of interleaving
techniques. They are block interleavers, convolutional interleavers,
random interleavers, and code matched interleavers. The design of
code matched interleavers will be described and turbo code perfor-
mance with various interleavers will be presented.

7.4 Block Type Interleavers

7.4.1 Block Interleavers
A block interleaver formats the input sequence in a matrix of m
rows and n columns, such that N = m x n. The input sequence
is written into the matrix row-wise and read out column-wise as
illustrated in Fig. 7.5.
The deinterleaver stores the data into a matrix with the same
format as the interleaver. The data are read out and delivered to
the decoder row-wise.
The interleaving function of the block interleavers can be rep-
resented by
7r(i) = [(i - 1) mod n] x m + L(i - l)jnJ + 1, i E A (7.7)
Block Type Interleavers 201

l·eM

I1 T
1 2 n

~ite [,
n+l n+2 2n
m rows

(m - l)n + 1 (m - l)n + 2 mn
I' n columns '1

Fig. 7.5: A block interleaver

where l x J means the integer of x. The end-to-end interleaver-

deinterleaver delay is 2mn. The memory requirement is mn for
both the interleaver and deinterleaver.
In error control coding, the number of rows in the interleaver
matrix is also called the interleaver degree (or depth) and the num-
ber of columns is called the interleaver span. The block interleavers
and deinterleavers are easy to implement. However, they may fail
to break certain low weight input patterns, such as weight 4,6,9,
etc, squai'e or rectangular input patterns [3J. For example, we con-
sider a weight-2 input pattern 1001 generating a low weight parity
check sequence. A weight-4 square input pattern shown in Fig. 7.6
cannot be broken by block interleavers. The weight-4 square input
pattern concatenates two weight-2 input patterns. When the input
sequence is written into the interleaving matrix, the two weight-2
input patterns are located in two rows with the four "l"s at the four
corners of a square. When the input sequence is read out from the
matrix, the two weight-2 input patterns remain in the interleaved
sequence. That means block interleavers cannot break the weight-4
square input pattern.
Block interleavers are effective if the error patterns to be bro-
ken are confined to one row [12J. If the error patterns are con-
fined to several consecutive rows, as for example concatenated error
patterns, which are often long and spread over a number of rows,
then the interleaving technique should be modified, such that the n
202 Interleavers

Iread
0 0 0 0
0 0
1 0 0 1
write. 0 0 0 0
0 0 0 0
1 0 0 1
0 0
0 0 0 0

Fig. 7.6: A weight-4 square input pattern of block interleavers

columns of the interleaving matrix should be read out in a specified

order to spread as many error patterns as possible. A method to
reorder the columns is given in [11].

7.4.2 Odd-Even Block Interleavers

An odd-even interleaver is a particular type of interleaver which
maps even position elements to even positions and odd position
elements to odd positions. It can be expressed as

7r(A -+ A): (7r(i) + i) mod 2 = 0, i E A (7.8)

For a rate 1/2 punctured turbo code, an odd-even interleaver can

provide uniform error protection across the information sequence
[4], as explained below.
For example, let us consider a binary data sequence

The data sequence is encoded by a rate 1/3 turbo encoder. A rate

1/2 turbo code can be obtained by puncturing the even parity check
digits of the first component encoder and the odd parity check digits
Block Type Interleavers 203

of the second component encoder. The odd information bits Ci, i =

1,3,5, ... , generate the encoded parity digits Vi,1 at the output of
the first component encoder, as shown in Table 7.1.

Table 7.1 Odd coded parity digits of the first encoder

CI C2 C3 C4 C5 C6 C7 Cs Cg ClO Cll CI2 CI3 CI4 CI5

VI,I - V3,1 - V5,1 - V7,1 - V9,1 - fUll,1 - rv13,1 - fVI5,1

Let us first assume that the data sequence is interleaved by a

pseudo-random interleaver. The interleaved data sequence is given
by

The second component encoder generates the even coded parity

digits Vi,2, as shown in Table 7.2. The overall code parity sequence
is obtained by multiplexing both the odd and even coded digits and
it is shown in Table 7.3.
Table 7.2 Even coded parity digits of the second encoder
C3 C5 Cll Cs CI4 CI2 Cg C6 C2 CI5 CI C4 C7 ClO CI3

- V5,2 - VS,2 - V12,2 - V6,2 - [v15,2 - V4,2 - [v1O,2 -

Table 7.3 Multiplexed coded sequence

'.-
CI C2 C3 C4 C5 C6 C7 Cs Cg ClO Cll CI2 CI3 CI4 CI5

VI,I V5,2 V3,1 VS,2 V5,1 [v12,2 V7,1 V6,2 V9,1 [v15,2 Vll,1 V4,2 fU13,1 rvlO,2 V15,1

It is obvious from Table 7.1 that each odd information bit has
a coded parity digit associated with it. However, the even coded
204 Interleavers

digits, shown in Table 7.2, can be generated by either odd or even

information bits. This means that some of the odd information bits
have two, such as V5,1 and V5,2, but some of the even information
bits have no parity digits associated with them, such as C2. Thus,
error protection is not uniformly distributed across the information
sequence. If some information bits with no associated parity digits
are in error, they cannot be corrected by either of the two decoders.
An odd-even interleaver can overcome this problem. Here we
present a block type odd-even interleaver. In this case the numbers
of rows and columns for the interleaver matrix must be odd [4]. For
the previous example, the interleaver matrix can be represented as

C4 C5]
Cg CIO
C14 C15

The interleaved data sequence can be expressed as

(7.10)
The interleaved data sequence is encoded by the second component
encoder, which generates the even parity digits shown in Table 7.4.
Table 7.4 Even coded parity digits after interleaving

CI C6 Cu C2 C7 C12 C3 Cs C13 C4 Cg C14 C5 CIO C15

- V6,2 - V2,2 - tv12,2 - VS,2 - V4,2 - tv14,2 - tvlO,2 -

Now each odd and even information bit has a coded digit asso-
ciated with it. The coded sequence is obtained by multiplexing the
coded sequences from Tables 7.1 and 7.4, as shown in Table 7.5.
With this sequence the error protection is uniformly distributed,
resulting in a better decoder performance, compared to the scheme
with a non odd-even interleaver.

7.4.3 Block Helical Simile Interleavers

Block helical simile interleavers are based on the conventional block
interleaver. The basic principle of the block helical simile interleaver
Block Type Interleavers 205

Table 7.5 Multiplexed coded sequence for an odd-even interleaver

CI C2 C3 C4 C5 C6 C7 Cs Cg ClO Cll Cl2 Cl3 Cl4 Cl5

VI,1 V6,2 V3,1 V2,2 V5,1 V12,2 V7,1 VS,2 V9,1 V4,2 Vll,l V14,2 'lJr3,1 ~10,2 tu15,1

is to read data diagonally instead of column-wise, with certain lim-

itations on the number of columns in the interleaver matrix. The
resulting interleaver terminates both component encoders in a turbo
encoder to the all-zero state [5]. This is achieved by appending v
tail bits at the end of the information sequence, where v is the
memory of the component convolutional code. The interleaver is
thus more efficient than the pseudo-random interleavers in which
only one encoder is forced to go to the all-zero state.
To generate a block helical simile interleaver, the number of the
columns for the interleaver matrix must be a multiple of (v + 1)
[5]. The information sequence is written into the matrix row-wise
and read out diagonally from the left to the right and from the
bottom to the top to prevent two adjacent data bits written in the
same column or row from remaining neighbours in the interleaved
sequence. This interleaver structure can avoid consecutive bits be-
ing output from the same column or row. An example of a block
helical interleaver with size 15 for a memory-2 turbo code is given
by the matrix

CI C2 C3
C4 C5 C6
C7 Cs Cg (7.11)
ClO Cll Cl2
Cl3 Cl4 C15

The interleaved sequence generated from this matrix is given by

As we can see this scheme does not produce an odd-even inter-

leaver. A necessary condition to generate a simile odd-even block
206 Interleavers

helical interleaver is to choose the number of columns to be an even

number and a multiple of (v+ 1). Also, the number of columns and
the number of rows have to be relatively prime [5].

7.5 Convolutional Type Interleavers

7.5.1 Convolutional Interleavers
Convolutional interleavers have been proposed by Ramsey [14] and
Forney [15]. The structure proposed by Forney is illustrated in Fig.
7.7.
Both the interleaver and deinterleaver consist of an input and
output commutator and a bank of L shift registers. The information
sequence to be interleaved is arranged in blocks of L bits. The input
commutator cyclically inserts each block of L bits into the bank of
L registers. The ith bit in each block is delayed by the ith shift
register and the delay of the shift register is (i - l)B. The output
commutator cyclically samples the bank of L registers in the same
order as the input one.
The deinterleaver performs the inverse operation. That is, the
ith bit in each block is delayed by the ith shift register, where the
delay of the shift register is (L - i)B.
The convolutional interleaving function, after the initial states
of the shift registers have been cleared out, can be expressed as
n(i) = i + [(i - 1) mod L] . LB, iEA (7.13)
This interleaver is also called an LB x L interleaver. The end-
to-end interleaver-deinterleaver delay is (L - l)LB. The memory
requirement is (L-1)LB/2 for either the interleaver or the deinter-
leaver. Therefore, there is a reduction of one-half in the end-to-end
delay and the memory of a convolutional interleaver compared to
the block interleaver to achieve a similar level of scrambling.
An example of convolutional interleaver with L = 3 and B = 2
is shown in Fig. 7.8. The interleaver size is 21 and the interleaved
sequence is given by
Convolutional Type Interleavers 207

0------1 (L-l)8 ~
(a) Interleaver

o o

(b) Deinterleaver

Fig. 7.7: A convolutional interleaver and deinterleaver

where zeros are produced by the initial zero states of the shift reg-
isters.
Ramsey's Type III (nI' n2) interleaver can be generated by using
the same function as for the convolutional interleaver, given by
(7.13), with ni = LB - 1 and n2 = L. The interleaver has the
following property [14]:

(7.14)
208 Interleavers

I 19 I 16 I 13 I 10 I 7 4 1

I 20 I 17 I 14 I 11 I 8 5 2

21 18 15 12 9 6 3

Fig. 7.8: A convolutional interleaver with L = 3 and B = 2

whenever
17r(i) -7r(j)I:::; n2 -1, i, j E A (7.15)
If the parameters Band L of the convolutional interleaver are
chosen properly, it can break some rectangular low weight input
patterns which appear in the block interleavers, and it can give
very good performance [16]. However, from Eq. (7.13), we can see
that for any value of i, the corresponding value of 7r(i) is always
larger than or equal to i. That is, the output sequence is expanded
by (L - I)LB symbols relative to the input sequence. One way to
overcome the sequence expansion is to not include the zeros within
the interleaved sequence. However, this will change the interleaving
function and worsen the interleaver performance.

7.5.2 Cyclic Shift Interleavers

An alternative way of generating convolutional type interleavers is
based on cyclic shifts. In the interleaver, the information sequence
is written into a matrix column-wise. The matrix has m rows and n
columns such that N = m x nand m :::; n. Then m row sequences of
size n are applied to the bank of m registers, where the ith register
shifts the ith row sequence in the matrix cyclically to the left by
(i -1)B, where B is a small integer whose value is chosen such that
B :::; n/m. The shifted sequences form a new matrix and the data
are read out from the matrix column-wise.
The deinterleaver performs a similar operation, whereby, the
ith row sequence in the matrix is cyclically shifted to the right by
(i - I)B.
For example, we consider a cyclic shift interleaver with size 21
Random Type Interleavers 209

as shown in Fig. 7.9. The parameters m, nand B are set to 3, 7

and 2, respectively. The interleaved sequence is given by

C= (Cl' C17, C12, C4, C20, C15, C7, C2, C18, ClO, C5, C2l,

C13, C8, C3, C16, Cn, C6, C19, C14, Cg)

[~~
21
~~ ~~
18 15
t! i i ~ 1 cyclic shift, [ ~g~

Fig. 7.9: A cyclic shift interleaver

Comparing the convolutional interleaver in Fig. 7.8 and the

cyclic shift interleaver in Fig. 7.9, it is obvious that the cyclic shift
interleaver retains the property of the convolutional interleaver, ex-
pressed by (7.14) and (7.15) with nl = mB - 1 and n2 = m, but it
overcomes its disadvantage of expanding the output sequence.

7.6 Random Type Interleavers

7.6.1 Random Interleavers
In the random interleaver a block of N input bits is read into the
interleaver and read out randomly. The interleaver vector 7r(i), i E
1,2, ... , N, can be generated according to the following algorithm,
which requires N steps:

Step 1. Choose randomly an integer i l from the set A = {I, 2, ... , N},
according to a uniform distribution between 1 and N, with
the probability of p( id = 1:i. The chosen integer i l is set to
be 7r(1).

Step k. (k > 1) Choose randomly an integer i k from the set Ak =

{i E A, i =I iI, i 2 , · · · , i k - l }, according to a uniform distri-
bution, with the probability of P(ik) = N-~+1. The chosen
integer i k is set to be 7r( k).
210 Interleavers

When k = N, the last integer iN is set to be 7r(N).

When the interleaver size is N = 2m - 1, a pseudo-random in-
terleaver can be generated by an m-stage shift register with linear
feedback as illustrated in Fig. 7.10. The feedback polynomial rep-
resented by
(7.16)
must be primitive with degree m. If the initial state of the shift
register is not the all-zero state, the shift register will go through
all 2m - 1 states cyclically. Therefore, the state of the m-stage shift
register can represent the interleaving function.

Fig. 7.10: A general m-stage shift register with linear feedback

For example, for interleaver size N = 7, a pseudo-random inter-

leaver can be generated by a 3-stage shift register with the poly-
nomial of 1 + D + D3. Assume the initial state is 010 and input
sequence is
(7.17)
The interleaved sequence generated by the pseudo-random inter-
leaver is
(7.18)
where the interleaver outputs are the octal form representative of
state numbers.

7.6.2 Non-uniform Interleavers

The interleaver used in the original turbo codes is a non-uniform
interleaver [3]. It is based on the ordinary square block interleaver,
Random Type Interleavers 211

but the data are read out diagonally with certain row and column
jumps between each reading.
Let i and j be the addresses of the row and column for writing,
and ir and jr the addresses of the row and column for reading. For
N = M x M, where M is a power of 2, the non-uniform interleaving
may be described by [3J

M
Zr (2 + 1) . (i + j) mod M
k (i + j) mod L
Jr {[P(k)· (j + l)J -I} mod M

where L is a small integer whose value is chosen empirically as a

function of M, P(k), k = 0,1,2"", L - 1, are functions of the
row address ir, and the values of P(k) should be relatively prime
with respect to M. Note that, for reading, the column index jr
is a function of the row index ir' A multiplying factor (~ + 1)
is used to prevent two input data written in two consecutive rows
from remaining neighbours in the interleaved sequence. In addi-
tion, reading is performed diagonally with respect to writing to
break rectangular low weight input patterns which appear in the
conventional block interleavers.
Typically, for interleaver size N = 256 x 256, L is chosen to be
8. The numbers of P(k), k = 0,1,2"",7, are given by [3J

P(o) = 17, P(l) = 37, P(2) = 19, P(3) = 29

(7.19)
P(4) = 41, P(5) = 23, P(6) = 13, P(7) = 7

7.6.3 S-random Interleavers

5-random interleavers proposed by Divsalar and Pollara are pseudo-
random interleavers [13J. They are based on the random generation
of N integers from 1 to N with an 5-constraint, where 5 is defined
as the minimum interleaving distance. 5-random interleavers are
also called spread interleavers.
An 5-random interleaver is defined as follows. Each randomly
selected integer is compared to the 51 previously selected integers.
If the absolute value of the difference between the current selected
 212 Interleavers

integer and any of the S1 previous selected integers is smaller than

S2, then the current selected integer is rejected. This process is
repeated until all N integers are selected.
In general an S-random interleaver can be described as

(7.20)

whenever
(7.21)
where S1 and S2 are two integers smaller than N. In a turbo en-
coder, these two parameters should, in general, be chosen to cor-
respond to the maximum input pattern lengths to be broken by
the interleaver. Thus, they should be chosen as large as possible.
The length of an input pattern is defined as the length of the input
sequence, starting from the first binary "I" to the last binary "1".
However, as the search time for this algorithm becomes pro-
hibitively large for large values of S1 and S2, a good trade-off be-
tween interleaver performance and search time is obtained for S1,
S2 < IN/2.
When two identical component encoders are employed in a turbo
encoder, it is appropriate to set S1 = S2. In the following analysis,
we assume the two component codes are identical and S1 = S2 = S.
Note that, for S = 1, the S-random interleaver becomes a random
interleaver.
For a turbo encoder, an S-random interleaver can break the
input patterns with lengths up to S + 1 and generate high weight
parity check sequences, as explained below.
Let {c} be the set of all the input patterns generating an error
event of the component code. The length of an input pattern c is
denoted by 1(c), and the weight of the input pattern is denoted by
w(c). If the length of an input pattern is small, it will likely produce
a low weight codeword. Therefore, the interleaver should break this
kind of input patterns. With an S-random interleaver, the input
pattern will be mapped to another sequence c. If c is not an error
pattern, we say that the input pattern for the component encoder is
broken. The second encoder will produce a parity check sequence
of infinite weight (if no termination is performed). Otherwise, if
Code Matched Interleavers 213

l(c)::; 8+1, because of the 8-constraint, l(c) > (w(c) -1) (8+1).
As the path length increases, c will likely produce a high weight
parity check sequence. Thus, in both cases, the overall codeword
weight will be high.
Based on the previous discussion, we can conclude that an 8-
random interleaver can either break the input patterns with length
up to 8 + 1 or expand these input patterns to longer error patterns
with length more than (w-1)(8+1), where w is the input sequence
weight, no matter what the component code is. Thus 8-random
interleavers can achieve better performance compared to pseudo-
random interleavers.
It is worth noting that the 8-random interleaver functions (7.20)
and (7.21) agree with the property of the convolutional interleaver
as shown in (7.14) and (7.15), if the parameters n1 and n2 of the
convolutional interleaver are chosen in a particular way, such as
n1 = 8 2 , n2 = 8 1 + 1. That is to say these two types of interleavers
are equivalent in the sense of breaking low weight input patterns.
In addition, dithered golden interleavers have been reported in
[23]. It is shown that they perform quite similar to 8-random inter-
leavers for low rate turbo codes, but better for high rate punctured
turbo codes.

7.7 Code Matched Interleavers

A code matched interleaver is a particular pseudo-random inter-
leaver which can break several low weight input sequences in such
a manner that the first several spectral lines of the turbo code dis-
tance spectrum are eliminated. It is obvious that these low weight
input patterns to be broken depend on the component codes, and
therefore, the interleaver should match the distance spectrum of
the component codes. We call such an interleaver a code matched
interleaver [9].
To break the low weight input patterns, we must compute the
distance spectrum of the low weight codewords of the component
codes first. Then on the basis of the performance analysis, we de-
termine the most significant input patterns which give large contri-
214 Interleavers

but ions to the error probability for the overall code at high SNR's.
In the interleaver design, we make sure that these significant input
patterns are broken so that they do not appear after interleaving.
This interleaver will eliminate first several spectral lines of the origi-
nal distance spectrum and increase the overall turbo code Hamming
distance. Consequently, the code performance at high SNR's is im-
proved and the error floor is lowered.

7.8 Design of Code Matched

Inter leavers
For a given component code, several attempts have been taken to
optimize the interleaver structure of a given length [17]-[22]. At
present, there is no systematic method of designing code matched
interleavers for turbo codes. This is partly due to a high complexity
of the problem. An effective way is to derive interleaver design
guidelines based on the code performance analysis. This is followed
by construction of interleaver search algorithms and actual search
for good interleaver structures. The guidelines proposed in [9] [10]
are summarized as follows:
1) Keep the interleaver random.
Note that in iterative soft output decoding algorithms, the
information exchange between the two component decoders
is possible because of the interleaving/deinterleaving oper-
ations. The input and output sequences of the interleaver
should be uncorrelated. The more "scrambled" the interleaver
is, the more "uncorrelated" the information exchange is.
2) Break as many of the short length input patterns as possible.
The input patterns with short lengths will likely produce low
weight codewords. The interleaver should break these input
patterns or expand these input patterns to longer paths so
that the resulting overall codeword weight will increase. Ob-
viously, there are quite a large number of these input patterns
and we need to include as many of these patterns as possible
in the interleaver design.
Design of Code Matched Interleavers 215

3) Determine the most significant low weight input patterns to

be broken.
The most significant input patterns are those giving large
contributions to the code error probability at high SNR's.
These input patterns produce low weight codewords corre-
sponding to the first several distance spectral lines in turbo
code distance spectrum. The most significant input patterns
are determined on the basis of the performance analysis. The
interleaver design makes sure that these input patterns are
broken so that the first several distance spectral lines of the
original distance spectrum are eliminated.
Recall that the S-random interleaver functions (7.20) and (7.21)
agree with the first two interleaver design guidelines. Because of the
S-constraint, a large number of short input patterns with length up
to S + 1 are broken. The main remaining task is to determine the
most significant low weight input patterns to be broken.
Let us denote by w the input pattern weight, by Zl (w) the parity
check weight of the first component encoder and by Z2(W) the parity
check weight of the second component encoder. The overall weight
of the error pattern is given by

(7.22)
Note that, for recursive systematic convolutional component codes,
the minimum input weight resulting in a finite weight error pattern
is two. Throughout the section we use an example of a turbo code
with generator matrix (1, 21/37)(oct) for both component codes.

(a) Input patterns with weight w = 2

The weight-2 input pattern generating the lowest weight of the
parity check sequence, denoted by C2, can be represented by
C2 = (00···0010000100···00) (7.23)
where the distance between the two "1"s is 5. The parity check
sequence of the component code generated by this weight-2 input
pattern is given by

v = (00···0011001100···00) (7.24)
216 Interleavers

The weight of this parity check sequence, denoted by Zmin, is 4. If

an interleaver maps the input sequence to a sequence with the same
pattern, the resulting overall codeword weight will be equal to the
code effective free distance dfree,eff given by

dfree,eff = 2 + 2Zmin (7.25)

In general, a weight-2 input sequence generating a finite weight

codeword can be represented by a polynomial

(7.26)

where 5 is the fundamental period of the recursive systematic con-

volutional encoder, t and T are integers such that t > 0, 5t is the
distance between the two "l"s in the weight-2 input sequence and
T is a time delay. The corresponding parity check weight of the
component code is

Z (2) = t· (Zmin - 2) + 2 (7.27)

Let us denote by i and j the positions of "l"s in a weight-2 input

sequence. If an interleaver function meets the following conditions

Ii - jl
mod 5 o
17r(i)-7r(j)1 mod 5 o (7.28)

this interleaver will map the input sequence C2 to another same

pattern sequence C2' Thus the weight-2 input sequence with this
pattern will produce a finite weight codeword as illustrated in Fig.
7.11, where t l , t2 = 1,2,3" . " and 5t 1 and 5t2 are distances between
the two "l"s in the input sequence C2 and its interleaved sequence
C2, respectively. The overall codeword weight is

(7.29)

A code matched interleaver should break this kind of weight-2

input patterns such that

17r(i) - 7r(j)1 mod 5 # 0 (7.30)

Design of Code Matched Interleavers 217

5t 1
z··---------------·j
C2 ••• 00100 00100···

\
C2 ... ···00100···00100
I
7r(i) • • 7r(j)
5t 2

Fig. 7.11: A weight-2 input sequence pattern

whenever
Ii - j I mod 5 = 0 (7.31)
In fact, it is not possible to break all these patterns. For an un-
broken input pattern, in order to maximize the overall codeword
weight, we require maximizing the value of

min(tl + t2 ) = min (Ii - jl + 17r(i) - 7r(j)I) /5 (7.32)

(b) Input patterns with weight w = 3

The procedure for breaking weight-3 input patterns is similar
to the one described for weight-2 input patterns. The mapping
from one weight-3 input pattern to another which satisfies the two
distances between the three "l"s is not easy to make. Nevertheless,
an interleaver with the 8-constraint can either break a short weight-
3 input pattern with length up to 8 + 1 or expand it to a longer one
with length more than 2(8 + 1). Thus, weight-3 input patterns will
likely produce high weight codewords. We do not typically consider
them in the interleaver design.

( c) Input patterns with weight w = 4

There are two types of weight-4 input patterns. The first type
is a single error pattern with input weight-4. Similar to the case
of input weight-3, an interleaver with S-constraint can either break
a short pattern or expand it to length 3(8 + 1). Thus we will not
consider this case in the interleaver design. The other type is a
218 Interleavers

combination of two single weight-2 input patterns. This type of

input sequences has the form

(7.33)

where 7"1 and 7"2 are integers with 7"2 > 7"1 + 5t l .

C4 0 0 10 o 100 0010···0100

C4 0 0 1 0 ... 0 1 0 0 0010 ... 0 1 00

n(il) - n(i2) n(jd - n(h)
5t3 5t 4

Fig. 7.12: A weight-4 input sequence pattern

Let us denote by iI, jl, i 2, and j2 the positions of data "l"s in a

weight-4 input sequence, where i l < jl < i2 < 12. A weight-4 input
pattern is shown in Fig. 7.12. If an interleaver mapping function
satisfies the following conditions

IiI - jll mod 5 0

li2 -121 mod 5 0 (7.34)
In(il) - n(i 2)1 mod 5 0
In(jl) - n(j2)1 mod 5 0

the input sequence will generate a low weight codeword. The overall
codeword weight is
(7.35)

where
Design of Code Matched Interleavers 219

t2 li2 - hi /5
t3 11T(id-1T(i2)1/5
t4 11T(jl)-1T(h)I/5
A code matched interleaver should avoid the crossing mapping
of two single weight-2 input patterns as illustrated in Fig. 7.12 or
maximizing the value of min( tl + t2 + t3 + t4) for the unbroken input
patterns ..

(d) Input patterns with weight w > 4

The effect of error patterns with input weight w > 4 on code
performance is small. This is partly due to the fact that they usu-
ally produce high weight codewords as a result of the S-constraint.
Therefore, we will not consider them in the interleaver design.

From the above discussion we can conclude that the most sig-
nificant low weight input patterns are those of weight-2 and com-
binations of weight-2. In the interleaver design, we only need to
break these particular input patterns. This makes the interleaver
design computationally efficient.
The code matched interleaver proposed in [9] is a modified S-
random interleaver. The modification consists in breaking the most
significant low weight error patterns. The weight of the input se-
quence to be broken is denoted by w. The design procedure is
summarized as follows:
1. Select an integer randomly from the set A = {1, 2, 3, ... , N}.
The selected integer represents the position of the interleaver
output.

2. Each randomly selected integer is compared to the S previous

selected integers. If the absolute value of the difference be-
tween the current selected integer and any of the S previous
selected in smaller than S, then reject the current selected
integer and go to Step 1.
3. Check whether the current interleaver output at time t and
w -1 previous interleaver outputs within time (1,2,3, ... , t -
1) form an error pattern (or a compound error pattern) of
220 Interleavers

weight w, which we want to break. If these w selected integers

form this error pattern, reject the current selected integer and
go to Step 1.

4. If no integer from the set A can satisfy both Step 2 and Step
3 simultaneously, for a specified number of iterations, then
reduce S by 1 and start the search again.

5. Save the current integer as the interleaver output. The pro-

cess is repeated until all N integers are selected.

In addition to the interleaver design guidelines, it is highly de-

sirable to develop a good search strategy such that the interleaver
design algorithms are flexible and computationally efficient. One
example is to use a "bidirectional process, boundary reflection"
method [10]. It is composed of recursive forward and backward
processing of selecting suitable integers to satisfy the design guide-
lines. The search method is faster and more efficient than other
random number generation methods.

7.9 Performance of Turbo Codes with

Code Matched Interleavers
The code matched interleavers are designed for a number of turbo
codes following the proposed design method. The interleaver size
was 1024 bits and S was chosen to be 10. The code bit error
rate performance on an AWGN channel is estimated by simulation.
In the simulation, iterative decoding with the SOYA algorithm is
employed. The maximum number of iterations is 8.
The code performance is compared to random interleavers and
S-random interleavers for the same interleaver size. S was chosen
to be 15 for S-random interleavers. The code rate is 1/3. The
simulation results are shown in Figs. 7.13-7.15.
From the figures it is clear that the code matched interleavers
improve the code performance at moderate to high SNR's and lower
the bit error rate "error floor" significantly.
Performance of 'IUrbo Codes with Code Matched Interleavers 221

10-'.--------,----,---------,----,---------.

----- Random
- S-Random
-+- Code Matched

10~

10-"

10-7'----L_ _ _ _--'--_ _ _ --'~ _ _ _--'--_ _ _---'

1.5 2 2.5 3
EblNo (dB)

Fig. 7.13: BER performance of the 4-state, rate 1/3, (1, 5/7) turbo code
with random, S-random and code matched interleavers on an AWGN
channel

Comparing the simulation results of the 4-state code in Fig.

7.13, we observe that the code matched interleaver achieves an im-
provement of 0.4 dB relative to the S-random interleaver and 1.1
dB relative to the random interleaver at a BER of 10- 5 • For the 8-
state turbo code in Fig. 7.14, the performance gains achieved by the
code matched interleaver are 0.1 dB and 0.6 dB relative to the S-
random interleaver and random interleaver, respectively, at a BER
of 10-5 . By comparing the simulation results of the 16-state turbo
code in Fig. 7.15, the code matched interleaver outperforms the
S-random interleaver by about 0.1 dB and the random interleaver
by about 0.25 dB at the same BER. An interesting observation in
Figs. 7.13-7.15 is that the BER for turbo codes with code matched
interleavers decrease rapidly with the increasing Eb/No. There is no
"error-floor" observed in the simulations, especially for the turbo
codes with large numbers of states.
222 Interleavers

10·

10-' -- Random
-e- S-Random
-+-- Code Matched
10-2

10-"

W10-4

10-5

10-5

10-7 .!

10'"
0.6 0.8 1.2 1.4 1.8 2
EblNo (dB)

Fig. 7.14: BER performance of the 8-state, rate 1/3, (1, 17/15) turbo
code with random, S-random and code matched interleavers on an
AWGN channel

7.10 Performance of Turbo Codes with

Cyclic Shift Interleavers
In Section 7.6.3, it is pointed out that the convolutional interleavers
are equivalent to the S-random interleavers in the sense of their
ability to break low weight input patterns. In this section, we con-
sider cyclic shift type convolutional interleavers. Turbo code per-
formance with cyclic shift interleavers is estimated by simulation
and compared to the S-random interleavers.
Let a cyclic shift interleaver size be N = m x n, where m and
n are the number of rows and columns of the interleaving matrix,
respectively, such that m :::; n. To generate a cyclic shift interleaver
based on (7.14) and (7.15) comparable to an S-random interleaver
based on (7.20) and (7.21), we should choose

(7.36)
Performance of Turbo Codes with Cyclic Shift Interleavers 223

10-'

---
--
-+--
Random
S-Random
Code Matched
10-3

10--4

~ 10--

10-6

10-7

10-8

10-·
08 1.2 1.4 1.6 1.8 2 2.2
EbINo(dB)

Fig. 7.15: BER performance of the 16-state, rate 1/3, (1, 33/31) turbo
code with random, S-random and code matched interleavers on an
AWGN channel

where 8 1 and 8 2 are the parameters of the 8-random interleaver.

For turbo codes with identical component codes, it is appropriate
to set 8 1 = 8 2 = 8. Therefore the parameters m and B from (7.36)
become
m = 8+ 1, B = 1 (7.37)
In a cyclic shift interleaver design, B is an integer such that
B:::; n/m = N/m 2 (7.38)
Increasing the value of B will result in breaking more low weight
input patterns, which on the other hand improves the turbo code
performance.
Cyclic shift interleavers are designed for interleaver size 1024.
Turbo code performance with the cyclic shift interleavers is evalu-
ated by simulations on AWGN channels. In the simulation, itera-
tive decoding with the SOYA algorithm is employed. The number
of iterations is 8.
224 Interleavers

The performance of a cyclic shift interleaver is compared to an

S-random interleaver for the same interleaver size. S was chosen to
be 15 for interleaver size 1024. For the cyclic shift interleaver, m
was set to S + 1 and B was chosen from the set (1,2,···, LN/m2 J).
The simulation results are shown in Fig. 7.16. It can be ob-
served that the turbo code with an S-random interleaver can achieve
a better performance relative to a cyclic shift interleaver with B =
1. However, increasing the value of B for the cyclic shift interleaver
improves the code performance. For interleaver size 1024, the cyclic
shift interleaver with m = 16 and B = 2 outperforms the S-random
interleaver with S = 15.

S-Random 8=15
~ CycllCshlftm=16B=1
-+- Cyclic shIft m=16 B=2
10-2

10-6

10-6
0.8 0.9 11 1.2 1.3 1.4 1.5 1.6
EbINo(dB)

Fig. 7.16: BER performance of the 16-state, rate 1/3, (1, 33/31) turbo
code with S-random and cyclic shift interleavers on an AWGN channel

In addition to the performance improvement, the cyclic shift in-

terleavers have the advantages of low design complexity and mem-
ory requirement. First, for given parameters, it is easier to con-
struct a cyclic shift interleaver than to search for an S-random
interleaver. Secondly, the interleaving vector of an S-random inter-
Performance of Turbo Codes with Cyclic Shift Interleavers 225

leaver must be stored in memory for both the turbo encoder and
decoder. However, for a cyclic shift interleaver, the interleaved or
deinterleaved sequence can be generated from the interleaving ma-
trix based on cyclic shifts. There is no need to store the interleaving
vector. Therefore, the cyclic shift interleavers reduce the memory
requirement and are easy to implement.
It is worth noting that the cyclic shift interleavers examined in
Fig. 7.16 have not been designed for a particular turbo code. For a
given turbo code, we could design a code matched interleaver based
on the cyclic shifts to further improve the turbo code performance
at high SNR's.
Bibliography

[1] J. G. Proakis, Digital Communications, 2nd Ed., McGraw-Hill,

New York, 1989.

[2] L. C. Perez, J. Seghers, and D. J. Costello, Jr., "A distance

spectrum interpretation of turbo codes," IEEE Trans. Inform.
Theory, vol. 42, no. 6, Nov. 1996, pp. 1698-1709.

[3] C. Berrou and A. Glavieux, "Near optimum error correcting

coding and decoding: Turbo-codes," IEEE Trans. Commun.,
vol. 44, no. 10, Oct. 1996, pp. 1261-1271.

[4] A. S. Barbulescu and S. S. Pietrobon, "Interleaver design for

turbo codes," Electron. Lett., vol. 30, no. 25, Dec. 1994, p.
2107.

[5] A. S. Barbulescu and S. S. Pietrobon, "Terminating the trellis

of turbo-codes in the same state," Electron. Lett., vol. 31, no.
1, Jan. 1995, pp. 22-23.

[6] A. S. Barbulescu and S. S. Pietrobon, "Interleaver design for

three dimensional turbo codes," in Proc. 1995 IEEE I8IT,
Whistler, BC, Canada, Sep. 1995, p. 37.

[7] O. Joerssen and H. Meyr, "Terminating the trellis of turbo-

codes," Electron. Lett., vol. 30, no. 16, Aug. 1994, pp. 1285-
1286.

[8] S. Benedetto and G. Montorsi, "Unveiling turbo-codes: Some

results on parallel concatenated coding schemes," IEEE
Trans. Inform. Theory, vol. 42, no. 2, Mar. 1996, pp. 409-428.
228 BIBLIOGRAPHY

[9] J. Yuan, B. Vucetic, and W. Feng, "Combined turbo codes

and interleaver design," IEEE Trans. Commun., vol. 47, no. 4,
Apr. 1999, pp. 484-487.

[10] W. Feng, J. Yuan, and B. Vucetic, "A code matched interleaver

design for turbo codes," in Proc. Int. Symposium on Personal,
Indoor and Mobile radio Communications (PIMRC'99), Os-
aka, Japan, Sep. 1999, pp. 578-582.

[11] E. Dunscombe and F. C. Piper, "Optimal interleaving scheme

for convolutional coding," Electron. Lett., vol. 25, no. 22, Oct.
1989, pp. 1517-1518.

[12] S. Dolinar and D. Divsalar, "Weight distributions for turbo

codes using random and nonrandom permutations," TDA
Progress Report 42-122, Jet Propulsion Lab., Aug. 1995, pp.
56-65.

[13] D. Divsalar and F. Pollara, "Thrbo codes for PCS applica-

tions," in Proc. ICC'95, Seattle, WA, June 1995, pp. 54-59.

[14] J. L. Ramsey, "Realization of optimum interleavers," IEEE

Trans. Inform. Theory, vol. 16, no. 3, May 1970, pp. 338-345.

[15] G. D. Forney, Jr., "Burst-correcting codes for the classic bursty

channel," IEEE Trans. Commun., vol. 19, no. 5, Oct. 1971, pp.
772-781.

[16] E. K. Hall and S. G. Wilson, "Convolutional interleavers for

stream-oriented parallel concatenated convolutional codes," in
Proc. 1 998 IEEE Int. Symposium on Inform. Theory, MIT,
Cambridge, MA USA, Aug. 1998, p. 33.

[17] J. D. Andersen, "Interleaver design for turbo coding," in Proc.

Int. Symposium on Turbo Codes and Related Topics, Brest,
France, Sep. 1997, pp. 154-156.

[18] M. Oberg and P. H. Siegel, "Lowering the error floor for turbo
codes," in Proc. Int. Symposium on Turbo Codes and Related
Topics, Brest, France, Sep. 1997, pp. 204-207.
BIBLIOGRAPHY 229

[19] J. Hokfelt and T. Maseng, "Methodical interleaver design for

turbo codes," in Proc. Int. Symposium on Turbo Codes and
Related Topics, Brest, France, Sep. 1997, pp. 212-215.

[20] F. Daneshgaran and M. Mondin, "Design of interleavers for

turbo codes based on a cost function," in Proc. Int. Symposium
on Turbo Codes and Related Topics, Brest, France, Sep. 1997,
pp. 255-258.

[21] O. Y. Takeshita and D. J. Costello, Jr., "New classes of alge-

braic interleavers for turbo codes," in Proc. 1998 IEEE Int.
Symposium on Inform. Theory, MIT, Cambridge, MA USA,
Aug. 1998, p. 419.

[22] K. S. Andrews, C. Heegard, and D. Kozen, "Interleaver design

methods for turbo codes," in Proc. 1998 IEEE Int. Symposium
on Inform. Theory, MIT, Cambridge, MA USA, Aug. 1998, p.
420.

[23] S. Crozier, J. Lodge, P. Guinand, and A. Hunt, "Performance

of turbo-codes with relative prime and golden interleaving
strategies," in Proc. Sixth Int. Mobile Satellite Conf., Ottawa,
June 1999, pp. 268-274.
Chapter 8

Turbo Coding for Fading

Channels

8.1 Introduction
In the previous chapters we have discussed the design, performance
analysis and iterative decoding of turbo codes for Gaussian chan-
nels. It is observed that turbo codes can achieve a remarkably low
bit error rate with iterative decoding at a signal-to-noise ratio close
to the Shannon capacity limit on AWGN channels.
However, on many real links such as radio, satellite and mobile
channels, transmission errors are mainly caused by variations in re-
ceived signal strength referred as fading. This severely degrades the
digital transmission performance and powerful error control coding
techniques are needed to reduce the penalty in signal-to-noise ratio.
In this chapter, we consider turbo and serial concatenated code
performance on fading channels. Analytical average upper bounds
for turbo and serial concatenated code performance based on the
union bound and code weight distributions are derived for indepen-
dent fading channels. For decoding, we consider the iterative MAP
and SOYA decoding methods with channel state information. The
turbo and serial concatenated code performance over independent
and correlated fading channels is discussed.

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
232 Turbo Coding for Fading Channels

8.2 Fading Channels

8.2.1 Multipath Propagation
In a cellular mobile radio environment, the surrounding objects,
such as houses, building, and trees, act as reflectors of radio waves.
These obstacles produce reflected waves with attenuated amplitude
and phase. If a modulated signal is transmitted, multiple reflected
waves of the transmitted signal will arrive at the receiving antenna
from different directions with different propagation delays. These
reflected waves are called multipath waves [1]. Due to the differ-
ent arrival angles and times, the multipath waves at the receiver
site will have different phases. When they are collected by the
receiver antenna at any point in space, they may combine either
in a constructive or a destructive way, depending on the random
phases. The sum of these multipath components forms a spatially
varying standing wave field. The mobile unit moving through the
multipath field will receive a signal which can vary widely in ampli-
tude and phase. When the mobile unit is stationary, the amplitude
variations in the received signal are due to the movement of sur-
rounding objects in the radio channel. The amplitude fluctuation
of the received signal is called signal fading. It is caused by the
time-variant multipath characteristics of the channel.

8.2.2 Doppler Shift

Due to the relative motion between the transmitter and the re-
ceiver, each multipath wave is subject to a shift in frequency. The
frequency shift of the received signal caused by the relative motion
is called the Doppler shift. It is proportional to the speed of the mo-
bile unit. Consider a situation when only a single tone of frequency
fe is transmitted and a received signal consists of only one wave
coming at an incident angle () with respect to the direction of the
vehicle motion. The Doppler shift of the received signal, denoted
by fd, is given by
(8.1)
Statistical Models for Fading Channels 233

where v is the vehicle speed and c is the speed of the light. The
Doppler shift effect in multipath propagation environment spreads
the bandwidth of the multipath waves into the range of fc ± fd max '
where hmax is the maximum Doppler shift and it is given by
vfc
fd max = ~ (8.2)

The maximum Doppler shift is also referred as the maximum

fade rate. As a result, a single tone transmitted gives rise to a re-
ceived signal with a spectrum of nonzero width. This phenomenon
is called frequency dispersion of the channel.

8.3 Statistical Models for Fading Chan-

nels
Because of multiplicity of factors involved in propagation in a cel-
lular mobile environment, it is necessary to apply statistical tech-
niques to describe signal variations. In this section we will introduce
Rayleigh and Rician fading models to describe signal variation in a
multipath environment.

8.3.1 Rayleigh Fading

In a common land mobile radio channel, we may assume the di-
rect wave is obstructed and the mobile unit receives only reflected
waves. When the number of reflected waves is large, according
to the central limit theorem, two quadrature components of the
channel impulse response will be uncorrelated Gaussian random
processes with a zero mean and variance a;.
As a result, the enve-
lope of the channel impulse response at any time instant undergoes
a Rayleigh probability distribution and the phase of the channel
response obeys a uniform distribution between -7r and 7r. The
probability density function (pdf) of the Rayleigh distribution is
given by
a _a 2 /a 2
a~O
p(a) = { '[/. e s (8.3)
a<O
234 Turbo Coding for Fading Channels

The mean value, denoted by m a , and variance, denoted by O"~,

of the Rayleigh distributed random variable are given by

(8.4)

If the probability density function in (8.3) is normalized so that

the average signal power (E[a 2 ]) is unity, then the Rayleigh distri-
bution will become

a2':O
(8.5)
a<O

The mean value and the variance will be

ma = 0.8862 (8.6)
O"~ = 0.2146
The pdf for Rayleigh distribution is shown in Fig. 8.1.

0.9.----,----__r---~--_,__--__r--_____,

01

°0L---~0.5---~--~1~.5--~2~--=2~.5~--~3
a

Fig. 8.1: The pdf of Rayleigh distribution

Statistical Models for Fading Channels 235

In fading channels, the received signal experiences a form of

frequency spreading and is band-limited between Ie ± Id. Assuming
an omnidirectional antenna with waves arriving in the horizontal
plane, a large number of reflected waves and uniform power received
for their incident angles, the power spectral density of the faded
amplitude due to the Doppler shift, denoted by IP(f)I, is given by

IP(f)1 = {27rJ~J-J2 if III ::;.lldl (8.7)

o otherwIse
where I is the frequency and Id is the fade rate. The value of
IdTs is the fade rate normalized by the symbol rate. It serves as a
measure of the channel memory. For correlated fading channels this
parameter is in the range 0 < IdTs < 1, indicating finite channel
memory. The autocorrelation function of the fading process is given
by
(8.8)
where J o(-) is the zero-order Bessel function of the first kind.

8.3.2 Rician Fading

In some propagation scenarios, such as satellite or micro cellular
mobile radio channels, there are essentially no obstacles on the
line-of-sight path. The received signal consists of a direct wave
and a number of reflected waves. The direct wave is a stationary
nonfading signal with a constant amplitude. The reflected waves
are independent random multipath signals. Their sum is called the
scattered component of the received signal. When the number of
reflected waves is large, the scattered component can be character-
ized as a Gaussian random process with a zero mean and variance
0";. The envelope of the scattered component will have a Rayleigh
probability distribution.
The sum of a constant direct signal and a Rayleigh distributed
scattered signal results in a signal with Rician envelope distribution.
The pdf of Rician distribution is given by

a~O (8.9)
a<O
236 Turbo Coding for Fading Channels

where D2 is the direct signal power and 10 (-) is the modified Bessel
function of the first kind and zero-order.
By assuming that the total average signal power is normalized
to unity, the pdf in (8.9) becomes

p(a) = { ~a(l + K)e~K~{HK)"' 10 (2aJK(K + 1)) a2:0

a<O
where K is the Rician factor denoting the power ratio of the direct
and the scattered signal components. The Rician factor is given by
D2
K=- (8.10)
2as2

The mean and variance of the Rician distributed random variable

are given by

(8.11)

where h (-) is the first order modified Bessel function of the first
kind. Small values of K indicate a severely faded channel. For
K = 0, there is no direct signal component and the Rician pdf
becomes a Rayleigh pdf. On the other hand, large values of K
indicate a slightly faded channel. For K approaching infinity, there
is no fading at all resulting in an AWGN channel. The Rician
distributions with various K are shown in Fig. 8.2.

8.4 Capacity of Fading Channels

The channel capacity of AWGN channels was considered in Chapter
1. In this section, we consider the channel capacity of the fading
channels as derived in [6]. The channel capacity can be used to
measure the turbo code performance on fading channels.
In the analysis, we will use the Rayleigh fading model, as the
Rayleigh model represents a worst fading case while offering ana-
lytical convenience. Let us assume that the channel is modelled by
Capacity of Fading Channels 237

2.------,------,------,,------,------,------,

18 K=O
K=2
K=5
1.6 K=10

14 . /,
/ \
/
12 I'

:§:1
c. '/. "
/ -- "- ,\
( , I
, I
08

/
/ ,
/'
,
\
\I
\I
06 I
,
"
/
,
/

,
/ I,
/ /
\,
,,,
/ I
/ \
I
/ , ....' .....

0.5 15 2 2.5 3
a

Fig. 8.2: The pdf of Rician distributions with various K

independent slow fading. Slow means that the fading bandwidth is

small compared to the signal bandwidth so that the receiver will be
able to track the phase variations, and hence compensate for it by
using a reference pilot tone or a phase-locked loop technique. This
will enable coherent detection. A slow fading process can be mod-
elled as a constant random variable during each symbol interval.
Furthermore, we assume that the channel interleaving depth is suf-
ficiently large, resulting in independent and identically distributed
(i.i.d.) fading amplitudes.
For BPSK modulation, let Xt be the transmitted signal at time
t. The received signal at time t is rt. The discrete-time channel
model is given by
(8.12)
where at is a random variable which represents the channel fading
variation, and nt is an additive white Gaussian noise with single
sided power spectral density No. The variance of the Gaussian
noise is a 2 = N o/2. In this model, the envelope amplitude of the
238 Turbo Coding for Fading Channels

fading attenuation at is a Rayleigh random variable.

Channel capacity is defined as the largest average mutual in-
formation between the channel input and output over the set of
input probability assignments. For fading channels, it is obvious
that the channel capacity depends on the knowledge of the values
of the fading attenuation, called channel state information (CSI),
available at the receiver. Here we consider the two extreme cases
of perfect CSI and no CSI.
By assuming that the fading attenuation can be perfectly recov-
ered at the receiver, the channel capacity is given by [6J

c = -11 p (a)p(r 1 x=1,a)

·log2 [~(1 + \lI(r, a))] drda (8.13)

where p(a) is the normalized Rayleigh pdf and p(r I x = 1, a) is a

Gaussian pdf with the mean of a and variance of (72. The term
\lI(r, a) is given by

\lI(r, a) = exp ( - :2 . a . r) (8.14)

If CSI is absent at the receiver, the channel capacity is give by

[6J

c -i 1 p(a)p (r I x = 1, a)

·log2 [~(1 + <p(r))] drda (8.15)

where
<P(r) = fap(a) . p (r I x = -1, a) da (8.16)
fap(a) . p (r I x = +1, a) da
The channel capacity of (8.13) and (8.15) can be calculated by
using numerical integration. The results from [6J are plotted in
Fig. 8.3. From the figure we can see that the penalty in Es/ No is
in excess of 0.8 1 dB when no CSI is utilized relative to the case
f'V

with perfect CSI on Rayleigh fading channels for codes of rate 1/4
to 1/2.
Capacity of Fading Channels 239

09

08

.,0.7 .- .-
~'" .- .- Ideal CSI
.0
I=-=-=- NoCSI
I
...
;:0.6
:0 .-
c.
.-
.- .-
U 0.5
a; .-
.,c:c:
.t: .- .-
U 0.4

0.3 .- .-

02

0.1
-2 0 2 4 6 8 10 12 14 16 18
ES/No (dB)

Fig. 8.3: Capacity of independent Rayleigh fading channels with coher-

ent BPSK signalling

From the channel coding theorem, we can derive the inequality

(8.17)

where R is the code rate and Hb(e) is the entropy of a binary source
with probability Pb(e). It is given by

(8.18)

Eqs. (8.17), (8.13) and (8.15) determine the channel capacity bounds
for fading channels. Due to the complexity of the channel capac-
ity expressions in (8.13) and (8.15), it is not possible to compute
analytically the pairs (Pb(e), Eb/No) satisfying (8.17) with equality.
However, by letting the code rate equal to the channel capacity,
we can obtain the smallest E b / No such that arbitrarily small error
performance is achievable. For codes with rates of 1/2 and 1/3,
the capacity limits for BPSK signalling on independent Rayleigh
240 Turbo Coding for Fading Channels

fading channels are given in Table 8.1. These values can be used
to indicate the channel capacity boundary of the Rayleigh fading
channels.

Table 8.1: Channel capacity limits for coherent BPSK

Code Independent Rayleigh Fading White
Rate CSI no CSI Gaussian
R=1/2 Eb/ N o=1.8 dB E b/No=2.6 dB Eb/ No=O.2 dB
R=1/3 Eb/ No=O.5 dB E b/ No=1.4 dB Eb/ No=-O.5 dB

8.5 Performance Upper Bounds on Fad-

ing Channels
The coded system structure is shown in Fig. 8.4. The channel
encoder may be a turbo encoder or a serial concatenated convolu-
tional encoder. The bit error probability of the coded system can
be upper-bounded by using a union bound that sums contributions
from all the codewords with various Hamming weights. As a
turbo or serial concatenated code can be represented by an equiva-
lent linear block code if the trellis termination is used to drive the
component encoders to the all-zero state, the union upper bound
for a block code can be applied to a concatenated code.
For example, a rate 1/3 turbo code with interleaver size N
and memory order 11 is equivalent to an (n, k) block code, where
n = 3(N + 11), k = N. In this section, we derive the upper bounds
on the pairwise error probability for block codes on an independent
Rician fading channel. Subsequently, we corne up with the average
upper bounds on the bit error probability for turbo and serial con-
catenated codes on a fading channel. In the analysis, we assume
that the channel interleaving depth is sufficiently large, resulting in
i.i.d. fading amplitudes.
Performance Upper Bounds on Fading Channels 241

Data Source -c Channel

Encoder
V BPSK
r-- Modulation -
X

Q9-- a

Etr-- n

Data Sink
- Channel
Decoder
BPSK
I+-- Demodulation I---
r

Fig. 8.4: Coded system block diagram

8.5.1 Upper Bounds on the Pairwise Error Prob-

ability
A. Decoding with Ideal CSI

Let a coded sequence be Vn = (VI, V2, ... , vn ), where n is the code

length. This sequence is modulated by a BPSK modulator. The
modulated sequence Xn = (Xl, X2, ... , Xn) is transmitted over the fad-
ing channel. At the receiver, the received sequence rn = (TI' T2, ... , Tn)
is observed. Furthermore, we assume that the fading attenuation
an = (aI, a2, ... , an) can be perfectly recovered at the receiver. The
decoder then performs soft decoding using the modified Euclidean
distance metric
n
m (rn' Xn; an) = L ITi - ai Xil 2 (8.19)
i=l

The pairwise error probability P(xn, xn) is the probability that

the decoder chooses as its estimate the sequence xn = (Xl, X2, ... , xn)
when the transmitted sequence was in fact xn (Xl, X2, ... , Xn).
This occurs if
(8.20)
242 'Thrbo Coding for Fading Channels

or equivalently
n n
L Iri - ai Xil 2 ~ L Iri - ai Xil 2 (8.21)
i=l i=l

For a given realization of the fading variable an, the conditional

pairwise error probability is given by
P(Xn, xnlan)
= Pr {~Iri - aiXil2 ~ ~ Iri - aiXil2}

= Pr {~2aini(Xi - Xi) ~ ~ lai(Xi - xi)1 2} (8.22)

Let us assume that phase recovery is ideal. Then ai, i =

1,2, ... , n, after the phase recovery is real. The term on the right
hand side of the inequality in (8.22) can be simplified as
n d
L lai(xi - xi)1 2 = 4 L a;k (8.23)
i=l k=l
where aik' k = 1,2, ... , d, represents the value of the fading ampli-
tude in the i k th bit interval in which the two sequences Xn and xn
are different, and d is the symbol Hamming distance between the
two sequences Xn and x n . The symbol Hamming distance between
two non-binary sequences is defined as the number of positions in
which they differ. For a given realization of the fading variable
an, the term on the right hand side of the inequality in (8.22) is a
constant which depends on an.
We introduce the random variable
n
X = L 2aini(xi - Xi) (8.24)
i=l
Since ni is a zero mean Gaussian noise with variance u 2 , the ran-
dom variable X is a Gaussian variable with a zero mean and vari-
ance ul = 16u 2"Lt=k a;k. Therefore, the pairwise probability con-
ditioned on an is

P(xn' xnlan) = Q ( 4 "L!: a;k)

Performance Upper Bounds on Fading Channels 243

(8.25)

where R = kin is the code rate, ~ is the signal-to-noise ratio per

bit, and Q(x) is the complementary error function defined by

(8.26)

Introducing the inequality

(8.27)

in (8.25) we obtain

(8.28)

The pairwise error probability P(xn, xn) can then be determined

by averaging over the random fading amplitudes aik'
k = 1,2, ... , d

P(Xn, xn) E [P(x n, Xnletn)]

< !E [ex p (_REb

2 No
t a;k)]
k=l
(8.29)

where the expectation operator E[·] represents the joint expectation

with respect to the components aik'
k = 1,2, ... , d. For the fully in-
terleaved fading channels, the fading amplitudes aik'
k = 1,2, ... , d,
are independent from interval-to-interval. Furthermore, for the Ri-
cian distributed random variable a, making use of the result

1 +K ( wK )
E [exp( -wa2 ) ] = 1 + K + w exp - 1 + K + w ,w > 0 (8.30)

the pairwise error probability P(xn, xn) can be upper bounded by

1 l+K KdR~
P x x <- ex ( _ (8.31)
1 + K + R~ ) + R~ )
No
1+K
d
( n, n) - 2 ( P
244 Turbo Coding for Fading Channels

B. Decoding without CSI

In deriving the above result we assumed that the value of fading
amplitude 8.n = (al' a2, ... , an) can be recovered perfectly. However,
in some systems it may not be practical to attempt CSI recovery,
so the pairwise error bound in the absence of CSI becomes relevant.
When no channel state information is available at the receiver,
the decoder performs soft decoding using the standard Euclidean
distance metric n
m (rn, xn) = L Iri - xil 2 (8.32)
i=l

The pairwise error probability P(xn' Xn) is given by

P(Xn' Xn) = Pr {~Iri - xil 2;::: ~ h - xil2}

Pr {~2ni(Xi - Xi) ;::: 4 E aik } (8.33)

As before we will introduce a random variable X equal to the

term on the left hand side of the inequality in (8.33). For a given
realization of the fading variable 8.n, X is a Gaussian variable with
a zero mean and variance 01 = 16da2 . The term on the right hand
side is a constant which depends on an. Therefore, the conditional
pairwise error probability can be expressed as

(8.34)

Using the bound of (8.27), the conditional pairwise error probability

can be upper-bounded by

P{x n , xnla.l ~ ~exp ( - ~ ~: (E a i' ) ' ) (8.35)

The pairwise error probability P(xn' xn) can then be determined

by averaging over the random fading amplitudes aik' k = 1,2, ... , d,
Performance Upper Bounds on Fading Channels 245

and is given by

P(Xn' Xn)

<
E [P(x n, xnlan)]

~ E [exp (- ~:, (E ai, 1 n (8.36)

In this analysis, we will introduce the random variable

(8.37)

As the fading amplitudes ~ = (aI, a2, ... , an) are independent iden-
tically distributed random variables, for large Hamming distance d
(larger than 5), according to the central limit theorem, the random
variable A is approximately Gaussian with the mean

mA = dma (8.38)

and the variance

(8.39)

where ma and a~ are respectively the mean and variance of the

Rician random variable a and they are given by (8.11). Noting
that

E [exp(wA2)] =V 1 exp ( wm 2A 2 ) ' if Re{w} < ~

1 - 2wa~ 1 - 2waA (f A

we get for the pairwise error probability upper-bound from (8.36)

(8.40)

For the turbo codes d is typically very large, so the above bound
holds in most cases.
246 Turbo Coding for Fading Channels

8.5.2 Average Upper Bounds on the Bit Error

Probability
For an (n, k) linear systematic block code, the input-redundancy
weight enumerating function can be represented as

A(W, Z) = L Aw,z WW ZZ (8.41)

w,z
where Aw,z is the number of the codewords of the block code with
the input information weight wand the parity check weight z. The
overall Hamming weight of the codeword is d = w + z.
The error coefficient which determines the contribution of the
codewords with the same weight d to the bit error probability, de-
noted by B d , is given by

= L
W
Bd kAw,z (8.42)
d=w+z
The error coefficient Bd is the average number of bit errors
caused by the transitions between the all-zero codeword and code-
words of weight d (d ~ dmin ), where dmin is the code minimum dis-
tance. The bit error probability of the code decoded by a maximum-
likelihood algorithm over a fading channel can be upper-bounded
by
Pb(e) ~ L BdPd(Xn, xn) (8.43)
d=dmin

where Pd(x n, xn) is the pairwise error probability when the Ham-
ming distance between the two sequences Xn and xn is d.
Therefore, the bit error probability upper-bound on an indepen-
dent Rician fading channel is given by

when ideal channel state information is available, and equal to

Performance Upper Bounds on Fading Channels 247

when there is no channel state information.

The above bounds can be evaluated as functions of ~ and K if
the code distance spectrum is known. Here, we assume the uniform
interleavers are used in the concatenated encoders so that the error
coefficient can be obtained by averaging over all possible input se-
quence permutations [7J [8J. For a turbo code, the error coefficient
can be written as [7J

(8.44)

where A~,z is the input-redundancy weight enumerating coefficient

of the equivalent block code for the component code, d = W+Zl +Z2,
W is the information weight, Zl and Z2 are the parity check weights
of the first and the second component codes, respectively. For a
serial code, the error coefficient is given by [12J

B = ' " ' " W ·n . (2N - l)! ACo . A C' (8.45)

d ~7 N.(2N)! w,l I,d

where A;,l and Af,'d are the corresponding input-output weight enu-
merating coefficients for the equivalent block codes of the outer and
inner code, respectively, W is input information weight of the outer
code, 1 is the weight of the output sequence for the outer code and
d is the weight of the output sequence for the inner code.
In the evaluation, we consider a turbo and a serial code with
the same code rate R = 1/3 and the memory order v = 2. For fair
comparison of the turbo and the serial code, the interleaver size
should be chosen in such a way that the decoding delay due to the
interleaver is the same. In other words, the input information block
sizes for both codes should be the same. We chose an information
size of N = 100 in the performance evaluation. For the turbo code,
the generator matrix of the component codes is

G(D) = [1 1 + D2 1
1 + D+ D2

The bit error probability upper-bounds for the turbo code with
variable K are illustrated in Fig. 8.5 for the case of ideal channel
248 Turbo Coding for Fading Channels

state information and in Fig. 8.6 for the case of no channel state
information. For the serial code, the generator matrix of the outer
code is

The generator matrix of the inner code is

The bit error probability upper-bounds for the serial code with
variable K are illustrated in Fig. 8.7 for the case of ideal channel
state information and in Fig. 8.8 for the case of no channel state
information. From the figures, we can see that for the Rayleigh
fading channel (K = 0), the error performance can be improved
by approximately 1.5 dB at the bit error rate of 10- 5 when ideal
channel state information is utilized relative to the case with no
channel state information. This improvement decreases as K is
getting larger. When K --t 00, the gain of channel state information
disappears.
In order to compare the performance of the turbo and serial
codes on Rayleigh fading channels, we show the distance spectra
and the bit error probability upper bounds for both codes in Figs.
8.9 and 8.10, respectively.
From Fig. 8.9 we can see that the serial code has smaller error
coefficients than the turbo code at low to medium Hamming dis-
tances. Since these distance spectral lines dominate the code error
performance in the region of high SNR's [9], the serial code can
achieve better performance than the turbo code on fading channels
at high SNR's as illustrated in Fig. 8.10. It may also be observed
from Fig. 8.9 that for medium distance values, the error coefficients
of both codes are almost the same, which will result in almost the
same error performance at low SNR's, as the code performance in
this region is determined by the medium distance spectral lines [9].
Iterative Decoding on Fading Channels 249

10°
\ \
\
\
\
\
10-'
\ \
\

10-2 \
\
\
\
,,
\

10-3
\

,,
\
,
II: , ,
W
<Xl
10-4

10-'

10-13

10-'
1.5 2 2.5 3 3.5 4 4.5 5
EblNo (dB)

Fig. 8.5: Bit error probability upper bound for the 4 state, rate 1/3 turbo
code with interleaver size 100 on independent Rician fading channels
with ideal channel state information. The curves are for Rician channels
with K =0, 2, 5, 50, starting from the top, with the bottom one referring
to an AWGN channel.

8.6 Iterative Decoding on Fading Chan-

nels
On AWGN channels, the turbo decoding is performed by a sub-
optimal iterative algorithm. The decoder consists of two identical
decoders of the component codes separated by an interleaver. The
component decoders are based on a maximum a posteriori algo-
rithm or a soft output Viterbi algorithm generating a weighted soft
estimate of the input sequence [2]-[5].
In the turbo decoding for AWGN channels, a branch metric
based on the standard Euclidean distance between the received and
transmitted signal sequences is used. This metric is optimum for
Gaussian channels. On fading channels, the decoding metric should
optimize the conditional probability p(rnlxn). Since this condition
250 Turbo Coding for Fading Channels

10·
I
I
I
I I
I I
I I
10-' I I
I

I
\ \
10-3 \
,,
II:
w
\
, ... ...
m
10-'
" ...

10-5

10-5
... ...

10-7
1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7
EbiNo (dB)

Fig. 8.6: Bit error probability upper bound for the 4 state, rate 1/3 turbo
code with interleaver size 100 on independent Rician fading channels
without channel state information. The curves are for Rician channels
with K =0, 2, 5, 50, starting from the top, with the bottom one referring
to an AWGN channel.

gives a fairly complicated metric computation, for practical reasons,

the AWGN metric is used as the decoding metric when no channel
state information is available at the receiver. So the MAP and
SOYA iterative decoding algorithms derived for AWGN channels
will be applied to decode turbo codes for fading channels without
channel state information.
If ideal channel state information is available at the decoder, the
branch metric is modified by replacing the transmitted signal Xi by
aiXi. This metric is not the optimum one, but it is a reasonable
trade-off between the complexity and performance, as the optimum
metric is very complex.
Based on this branch metric, the log-likelihood ratio and extrin-
sic information for MAP and SOYA decoding methods have been
modified. Note that the decoding methods described here are for
turbo codes. However, the algorithms can be directly extended to
Fig. 8.7: Bit error probability upper bound for the 4 state, rate 1/3 serial
code with information size 100 on independent Rician fading channels
with ideal channel state information. The curves are for Rician channels
with K =0, 2, 5, 50, starting from the top, with the bottom one referring
to an AWGN channel.

serial concatenated codes.

Without loss of generality, we assume that the bit sequence
C = (CI' C2, ••• , CN) is encoded by an (n, 1, v) RSC encoder. The
code trellis has a total number of Ms = 2/.1 distinct states, indexed
by the integer l, 1 = 0,1, ... , Ms - 1. The RSC encoder output
sequence is v = (VI,V2, ... ,VN), where Vi = (Vi,O,Vi,I, ..• ,Vi,n-l) are
defined by n encoder generators. The corresponding modulated
sequence x and received sequence r are given by

x (Xl, X2, ... , XN)

(XI,O, XI,I, ... , XI,n-l, X2,O, X2,1, •.•... , XN,O, XN,I, ... , xN,n-d

r (rb r2, ... , rN)

(rl,O, rl,l, ... , rl,n-l, r2,O, r2,1, ...... , rN,O, rN,I, ... , rN,n-l)
252 Turbo Coding for Fading Channels

10·
I
I
I
I
10-'

I I
I I
10-' I
\ \
I \
\ I
I \
10-' I
\
I \
~ I
I
\
\
I
10-4 \ I \
\ \
\
\
\ \
10-5
\
\
,
10-6

10-'
1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7
EbiNo (dB)

Fig. 8.8: Bit error probability upper bound for the 4 state, rate 1/3 serial
code with information size 100 on independent Rician fading channels
without channel state information. The curves are for Rician channels
with K =0, 2, 5, 50, starting from the top, with the bottom one referring
to an AWG N channel.

8.6.1 Modified MAP Decoding with CSI

In the MAP decoding method, the log-likelihood ratio for the input
symbol at time t, denoted by A (Ct), is given by [3] [4]

(8.46)

where ,; (rt, l', l) is the branch transition probability which can be

determined from the transition probabilities of the channel and the
encoder trellis. {}t-l (l') and f3t (l) are the forward and backward
recursive probability functions obtained from ,; (rt, l', l). For an
independent fading channel with ideal channel state information,
the branch transition probability,; (rt,l' , l) based on the modified
Iterative Decoding on Fading Channels 253

10" ,------.--------r----.-----,----,------,

10'"

/
10'· /
/
/
/
/
/
/
/

"

10-20
Turbo Code N=100
Serial Code N=100

10-30L-.._ _--'-_ _----'-_ _ _..l....-_ _--'-_ _ _L-.._ _--'

o 50 100 150 200 250 300
Hamming Distance

Fig. 8.9: Distance spectrum comparison of the 4 state, rate 1/3 turbo
and serial concatenated codes with information size 100

branch metric can be computed as

i(
'Yt rt"
l'l) _ (.). (Ej,:J(rt,j-at,jx~,j)2)
- Pt z exp - 20- 2 ' i = 0,1

where Pt(i) is the a priori probability of Ct = i and 0- 2 is the vari-

ance of the Gaussian noise. In a derivation similar to [3] [4] the
log-likelihood ratio for the input symbol at time t, A (Ct), can be
decomposed into

(8.47)

where the first term is the a priori information obtained from the
other decoder. The second term is the systematic information gen-
erated by the code information bit. Ae (Ct) is the extrinsic informa-
254 'furbo Coding for Fading Channels

10'"

10-3

-.
10'"

Turbo code, ideal CSI

10'" --+--Serial code, ideal CSI
- - Turbo code, no CSI
- +- Serial code, no CSI
10-7 t.:=:::=::J==;:t=:==~==:===~-:'::---'--L----l----'-_-.J
1.5 2 2.5 3 3.5 4 45 5 5.5 6 6.5 7
EblNo (dB)

Fig. 8.10: Bit error probability upper bound comparison of the 4 state,
rate 1/3 turbo and serial concatenated codes with information size 100
on independent Rayleigh fading channels

This extrinsic information Ae (Ct) is interleaved (or deinterleaved)

and then used as the a-priori information for the other decoder.

8.6.2 Modified SOYA Decoding with CSI

In the SOYA method, the path metric corresponding to path x at
time t, denoted by J.Lf, is calculated by [5]

(8.48)
Iterative Decoding on Fading Channels 255

where xf,j is the jth modulated symbol on path x at time t, Jl~-l is

the smallest metric of the path connected to path x at time t - 1
and logpt(i) is the logarithm of the a priori probability at time t
obtained from the previous decoder for the information bit ct = i.
The SOYA decoder provides a soft output in the form of an
approximate log-likelihood ratio of the posteriori probabilities of
the information bits 1 and O. The soft output of the SOYA decoder
for the input symbol at time t can be approximately expressed
as the metric difference of the maximum likelihood path and its
strongest competitor at time t. The strongest competitor of the
maximum likelihood path is the path which has the minimum path
metric among all paths obtained by replacing the trellis symbol
on the maximum likelihood path at time t by its complementary
symbol. The SOYA output, denoted by A (ct) at time t, can be
expressed as [5]

A(ct) _ I [P{ct =
og P {Ct
11
= 0 I r}
r}]
_ log [P{Xt,O = +11 r}]
P{Xt,O = -11 r}
Jl"t 1 - Jl~ (8.49)
where Jl"t 1 is the minimum path metric corresponding to Xt,O = -1
and Jl} is the minimum path metric corresponding to Xt,O = 1. The
path metric and the SOYA output have the additive property. Fol-
lowing the derivation in MAP, we can split the soft output into two
parts, the intrinsic information Ai (Ct) and the extrinsic information
Ae (ct), as follows
A (ct) = Ai (Ct) + Ae (ct) (8.50)
The intrinsic information, Ai (ct), is given by

Ai (Ct) = log [::~~~] + 4at,OTt,O (8.51)

Therefore, the extrinsic information Ae (Ct) can be obtained from

(8.50) and (8.51) as

(8.52)
256 'furbo Coding for Fading Channels

and it will be used to calculate the a priori probabilities ratio in

the next step of decoding after interleaving or deinterleaving.

8.7 Performance Simulation Results on

Fading Channels
8.7.1 Performance Comparison Between MAP
and SOYA Algorithms on Independent
Fading Channels
The bit error rate performance of the turbo codes on fading channels
is estimated by simulation. In the simulation, we considered the
rate 1/3 turbo code with memory order v = 4 and information size
N = 1024. The generator polynomials of the turbo code in the
octal form are go = (37) and gl = (21). In the receiver, iterative
decoding with symbol-by-symbol MAP and SOYA methods with
and without channel state information is employed. The number of
iterations is 8 for interleaver size 1024.
For the independent Rayleigh fading channels, the performance
simulation results are shown in Fig. 8.11. At a bit error rate of
10- 5 , both MAP and SOYA decoders with CSI give an improve-
ment of about 1dB relative to the respective decoders without CSI.
It can also be observed that MAP decoding is superior to SOYA
decoding by 0.5dB at the bit error rate of 10-5 , for both decoders
with and without channel state information. A similar result has
been obtained for AWGN channels [5].

8.7.2 Performance Comparison Between Turbo

and Serial Concatenated Codes on Inde-
pendent Fading Channels
The bit error rate performance comparison of the turbo and serial
concatenated coding schemes on fading channels was also carried
out by simulation. We simulated rate 1/3 turbo and serial codes
with memory order v = 2 and information sizes of N = 1024 and
Performance Simulation Results on Fading Channels 257

--..
10-'
--+-- SOVAwith CSI
SOYA without CSI
-+- MAP with CSI
10-'
- - MAP without CSI

~
!It.
,,
" \ \

"
,
10-3 \ \

.
\ \
\
cr:
w
m
, ",
>I-
10'"
\
, \
\
\ \
\ \
\ \
\
\ \
\ \
\
10--
" \

•
\

'+

10'"
15 2 2.5 3 3.5 4
EblNo (dB)

Fig. 8.11: Performance comparison of MAP and SOYA, with and with-
out CSI, for the 16 state, rate 1/3 turbo code on an independent Rayleigh
fading channel, information size 1024, the number of iterations 8

4096. In the receiver, iterative SOYA decoding is employed. The

number of iterations was 8 for information size 1024 and 18 for
information size 4096.

For the independent Rayleigh fading channels, the performance

simulation results for the turbo and serial codes are shown in Fig.
8.12. It can be observed that the turbo code has a better perfor-
mance at low SNR's than the serial code. However, the serial code
outperforms the turbo code at high SNR's. For the turbo code, a
BER error floor appears at 10- 6 • A similar result has been obtained
for AWGN channels [5].
258 Turbo Coding for Fading Channels

10-'

*"- ..,.-,
----M--
Turbo N=1024
- -K- .
Serial N=1024
,, ... "., --e--
-
Turbo N=4096
,, ')(
-()- - Serial N=4096
b ...
"- ,
,, " , "-
... x.
"-
lSI ,
~ 10-'
"., ,
"- ,,
X
10-'

", ,,
10-6 "-
"-
'b

10-'
28 3 32 3.4 3.6 3.8 4 4.2 4.4
EblNO (dB)

Fig. 8.12: Performance comparison for the 4 state, rate 1/3 turbo and
serial codes on an independent Rayleigh fading channel

8.7.3 Performance Comparison Between MAP

and SOYA Algorithms on Correlated Fad-
ing Channels
In some applications, such as mobile and satellite communications,
the channel interleaving depth is severely constrained by the maxi-
mum delay requirements. This results in a correlated fading chan-
nel. However, the performance of the communication systems with
correlated fades is not amenable to analytical evaluation. In this
section, we investigate the bit error rate performance of the coding
schemes on correlated Rayleigh fading channels by simulation.
In the simulations, the fade rate normalized by the symbol rate
was 0.01. The performance comparison of MAP and SOYA, with
and without channel state information for the rate 1/3 turbo code
with memory order v = 4 and information size N = 1024 on a
correlated Rayleigh fading channel is shown in Fig. 8.13. We can
see from the figure that the MAP decoder with CSI achieves an im-
Performance Simulation Results on Fading Channels 259

provement of about 1.2dB at a BER of 10- 5 relative to the MAP de-

coder without CSI. The corresponding improvement for the SOYA
decoder with ideal CSI is about 1.8dB. It can be also observed that
without CSI the performance degradation of the SOYA decoding
relative to the MAP decoding is approximately O.7dB. However,
the SOYA CSI decoder has almost the same performance as the
MAP CSI decoder at high Eb/No. By comparing Fig. 8.11 with
Fig. 8.13, we can see that the turbo code performance degrades
significantly due to the correlated fades in the channel. For the
specified normalized fade rate, the turbo code performance is about
1rv1.8dB worse relative to an independent fading channel.

--..
10-'

--+-- SOVA with CSI

SOVA without CSI
-+- MAP with CSI
, - - MAP without CSI

.. ,
10-2
\

"' \.
<- , \.
,
-10
,, '- , ,.
10- 3 ,,
"' \.

a:
w
'-
"t-, "' '- ,
'" \ '-
*, ,
\.
\.
10-' '-
... ~
",
'.,
" ,, '-
*'- ,
'- ,
10--
.... ... '-
.... "-
'- , ,.
"

10-6
1.5 2 2.5 3 3.5 4 4.5 5 5.5
EbINo (dB)

Fig. 8.13: Performance comparison of MAP and SOYA, with and with-
out CSI, for the 16 state, rate 1/3 turbo code on a correlated Rayleigh
fading channel, the fading rate normalized by the symbol rate is 10- 2 ,
information size 1024, the number of iterations 8
260 Turbo Coding for Fading Channels

8.7.4 Performance Comparison Between Turbo

and Serial Concatenated Codes on Corre-
lated Fading Channels
Performance comparison of the rate 1/3 turbo and serial codes with
memory order 1/ = 2 and various information sizes on a correlated
Rayleigh fading channel is shown in Fig. 8.14. In the simulation,
iterative SOYA decoding is employed and the information sizes are
1024 and 8192. The number of iterations is 8 for information size
1024 and 18 for information size 8192.

- Parallel N=1024 1=8

- ... - Serial N=1024 1=8
-- Parallel N=8192 1=18
--6- Serial N=8192 1=18

10-2 " . ,
~, 'I.,
., , , ,
, , ""
, ,

Q
,
"
" , ....
~
~

,,
10" \
, ,
\
,, "
, "x "
10'" 'I. " .,
",
\ '>
'\

""
10... ' - - - - ' - - - - ' - - - - ' - - - - ' - - - - - ' - - - - - ' - - . . . . . L - - - - ' - - - - ' - - - - - '
2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7
EblNO(dB)

Fig. 8.14: Performance comparison for the turbo and serial codes on a
correlated Rayleigh fading channel, the fading rate normalized by the
symbol rate is 10- 2 , information size N, the number of iterations I

We can see from the figure that the serial concatenated code
consistently outperforms the turbo code on correlated fading chan-
nels. At a BER of 10-5 , the corresponding improvement achieved
by the serial concatenated code relative to the turbo code is 1.25 dB
and 1.2 dB for the information size of 1024 and 8192, respectively.
Bibliography

[1] J. G. Proakis, Digital Communications, 2nd Ed., McGraw-Hill,

New York, 1989.

[2] C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon

limit error-correcting coding and decoding: Turbo-codes(I),"
in Proc. ICC'93, Geneva, Switzerland, May 1993, pp. 1064-
1070.

[3] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, "Optimal de-

coding of linear code for minimizing symbol error rate," IEEE
Trans. Inform. Theory, vol. 20, no. 2, Mar. 1974, pp. 284-187.

[4] J. Hagenauer, P. Robertson, and L. Papke, "Iterative ('Turbo')

decoding of systematic convolutional codes with the MAP and
SOYA algorithms," in Proc. of the 1994 ITC Conference on
Source and Channel Coding, Munich, Oct. 1994, pp. 1-9.

[5] B. Vucetic, "Iterative decoding algorithms," in Proc. of IEEE

Int. Symposium on Personal, Indoor and Mobile Radio Com-
munications (PIMRC'97j, Helsinki, Finland, Sep. 1997, pp.
99-120.

[6] E. K. Hall and S. G. Wilson, "Design and analysis of turbo

codes on Rayleigh fading channels," IEEE Journal on Selected
Areas in Communications, vol. 16, no. 2, Feb. 1998, pp. 160-
174.

[7] S. Benedetto and G. Montorsi, "Unveiling turbo-codes: Some

results on parallel concatenated coding schemes," IEEE
Trans. Inform. Theory, vol. 42, no. 2, Mar. 1996, pp. 409-428.
262 BIBLIOGRAPHY

[8] D. Divsalar, S. Dolinar, R. J. McEliece, and F. Pollara, "Trans-

fer function bounds on the performance of turbo codes," TDA
Progress Report 42-123, Jet Propulsion Lab., Aug. 1995, pp.
44-55.

[9] J. Yuan, B. Vucetic, and W. Feng, "Combined turbo codes

and interleaver design," IEEE Trans. Commun., vol. 47, no. 4,
Apr. 1999, pp. 484-487.

[10] J. Yuan and B. Vucetic, "Turbo code performance on Rician

fading channels," in Pmc. ICC'99, Vancouver, Bc, Canada,
June 1999, pp. 83-87.

[11] J. Yuan, W. Feng and B. Vucetic, "Parallel and serial concate-

nated codes for fading channels," in Proc. PIMRC'99, Osaka,
Japan, Sep. 1999, pp. 862-866.

[12] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Se-

rial concatenation of interleaved codes: Performance analysis,
design, and iterative decoding," IEEE Trans. Inform. Theory,
vol. 44, no. 3, May 1998, pp. 909-926.
Chapter 9

Turbo Trellis Coded

Modulation Schemes

9.1 Introduction
The effectiveness of trellis coded modulation (TCM) techniques,
proposed by Ungerboeck in 1982 [1], for communication over band-
width limited channels is well established. TCM schemes have been
applied to telephone, satellite and microwave digital radio channels,
where coding gains of the order of 3-6dB are obtained with no loss
of bandwidth or data rate.
Turbo codes can achieve remarkable error performance at a low
signal-to-noise ratio close to the Shannon capacity limit. However,
the powerful binary coding schemes are not suitable for bandwidth
limited communication systems.
In order to achieve simultaneously large coding gains and high
bandwidth efficiency, a general method is to combine turbo codes
with trellis coded modulations. In this chapter we discuss several
bandwidth efficient turbo coding schemes, their encoding/decoding
algorithms and performance on Gaussian and flat fading channels.
For Gaussian channels, turbo coded modulation techniques can
be broadly classified into binary schemes and turbo trellis coded
modulation. The first group can further be divided into "prag-
matic" schemes with a single component binary turbo code and
multilevel binary turbo codes. Turbo trellis coded modulation can

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
264 Turbo Trellis Coded Modulation Schemes

be classified into turbo coded modulation with parity symbol punc-

turing and turbo coded modulation with information symbol punc-
turing. For flat fading channels an I-Q turbo coded modulation
technique is considered.

9.2 Binary Turbo Coded Modulation

9.2.1 Pragmatic Binary Turbo Coded Modula-
tion
In pragmatic turbo coded modulation design [4] a single binary
turbo code of rate l/n is used as the component code. Its encoder
outputs are suitably multiplexed and punctured to obtain in parity
symbol and m - in information symbols, as shown in Fig. 9.l.
These encoded symbols are mapped into an M - PSK or M -QAM
signal set consisting of 2m points. If an M-PSK modulation is
used, m encoded symbols select one M - PSK signal according to
Gray code mapping. If an M-QAM modulation is used, we map
m/2 encoded symbols to an in phase (I) level of an M-QAM signal
and map the other m/2 encoded symbols to a quadrature (Q) level.
Separate Gray mapping for the I and Q channels is applied. The
spectral efficiency of the scheme is (m - in) bits/s/Hz.
Though an interleaver is included between the multiplexer and
puncturer modules, with the aim of symbol decorrelation in slow
fading channels, it is not needed on Gaussian channels.
The receiver, shown in Fig. 9.2, calculates the log-likelihood
function for each encoded binary digit based on the received noisy
symbol and the signal subsets in the signal constellation specified
by each binary digit. The stream of the bit likelihood values is then
deinterleaved and demultiplexed before passing to the binary turbo
decoder which can be based either on MAP or SOYA.
The pragmatic approach is simple, as only one turbo encoder
and one turbo decoder is used. By modifying the puncturing func-
tion and modulation signal constellation, it is possible to obtain a
large family of turbo coded modulation schemes.
Binary Thrbo Coded Modulation 265

systematic bits

I
I
Binary
Inter- Signal
Thrbo Mapper
c leaver
Encoder
Q

parity check bits

Fig. 9.1: Pragmatic turbo TCM encoder

systematic bit
I
I Bit
Binary
Demodu- LLR
plexer Thrbo
r lator leaver
tation

t
parity check bits

Fig. 9.2: Pragmatic turbo TCM decoder

To illustrate the pragmatic binary turbo coded modulations, we

consider an example of a rate 1/2 turbo coded 16-QAM, where the
parameters n, m and m in Fig. 9.1 are equal to 3, 4 and 2, respec-
tively. A rate 1/3 binary turbo code is employed in the example.
Every two encoding intervals, the encoded bits are multiplexed and
punctured according to a puncturing matrix as

p = [~ ~ 1 (9.1)

This leads to a rate 1/2 turbo code. Let us denote four encoded bits
at time t by Vt,i, i = 1,2,3 and 4. They are mapped to a 16-QAM
266 Turbo Trellis Coded Modulation Schemes

signal constellation to select a transmitted signal Xt, which is given

by
Xt = Xt,l + jXt,Q (9.2)
where Xt,l is the in-phase and Xt,Q is the quadrature components
of the transmitted signal Xt. A 16-QAM signal constellation with
Gray mapping is shown in Fig. 9.3.
Q

• • 11 • •

• • 10 • •
-----------+----------__ I
00 10 11

/.~
Vt,l Vt,2
• 00 • •

• • 01 • •
~1.4
Fig. 9.3: 16-QAM with Gray mapping

Assuming coherent detection, the received signal at time t, de-

noted by Tt, can be represented by
Tt = Tt,l + jTt,Q (9.3)
where the I and Q component signals are given by
Tt,l = + nt,I
Xt,l

Tt,Q = Xt,Q + nt,Q (9.4)

and where nt,I and nt,Q are two independent Gaussian noise signals
with zero mean and variance a 2 •
The log-likelihood ratio (LLR) for each encoded bit Vt,i, denoted
by A(Vt,i), can be calculated by [4] [10]

A( Vh.) -- K.l PT {Vt,i = 11 T t}

,
c og PT {Vt,i = 0 ITt } (9.5)
Binary Turbo Coded Modulation 267

where Kc is a constant. Since the in-phase and quadrature com-

ponent signals are affected by independent noise signals, the appli-
cation of Bayes' rule shows that the LLR's of Vt,l and Vt,2 depend
only on rt,I, while the LLR's of Vt,3 and Vt,4 depend only on rt,Q.
Thus the LLR's of the encoded bits can be further written as

i = 1,2

A(Vt,d = i = 3,4

where XI is the in-phase and xQ is the quadrature components of a

modulation signal of Vt,i = j, j = 0, 1. The log-likelihood values are
deinterleaved and demultiplexed and then fed to the binary turbo
iterative decoder.
It is worth noting that the LLR's of bits Vt,l and Vt,2 affected
by the same noise nt,I are not independent. The same applies to
bits Vt,3 and Vt,4 affected by nt,Q. In addition, for a given received
signal rt, the LLR's A( Vt,i) are not strictly Gaussian for all encoded
bits i = 1,2,3, and 4. Thus, the binary turbo decoder is no longer
optimal for the coded modulation scheme. Nevertheless, simulation
results indicate that, for large SNR values, the LLR's A( Vt,i) are
close to a Gaussian random variable with variance a 2 if the constant
Kc is set to a 2 /2. Therefore, the binary turbo decoder performs
quite well in the pragmatic approach [4] [10].

9.2.2 Multilevel Turbo Coding

The multilevel coding scheme, devised by Imai and Hirakawa [5], is
a combined coding and modulation method based on a number of
binary component codes. Multilevel turbo codes are constructed by
using binary turbo codes as the component codes. The transmitter
for an M -ary signal constellation consists of l = log2 M parallel
binary encoders as shown in Fig. 9.4.
Let a message sequence of length k be

(9.6)
268 Turbo Trellis Coded Modulation Schemes

cl
cf Turbo Encoder Cl I vi

Serial/
c Signal x
Parallel
Converter
c2 ITurbo Encoder c21 v2 Mapper

c1 I 1 v1
ITurbo Encoder C 1 I

Fig. 9.4: Multilevel turbo encoder

The sequence is split into 1 blocks. The ith block sequence is given
by
(9.7)
where k i is the length of the block and

(9.8)

Each message block c i is encoded by an individual binary turbo

encoder C i generating a codeword

(9.9)

where ni is the code length of the component turbo code C i . The

code rate of C i is R, = kdni. For simplicity, we assume that all
the component turbo codes have the same code length of n, that is,
nl = n2 = ... = nl = n. The output digits of the encoders at each
instant time t, denoted by v:' i = 1,2, ... , I, t = 1,2, ... , n, form a
binary label (vI, v;, ... , vD, which is mapped into a symbol Xt from
the M-ary signal constellation. The transmitted signal sequence is
represented by
(9.10)
The overall code rate R of the multilevel turbo code is R = k/nl.
The bandwidth efficiency of the coding scheme is
k
'TJ = Rlog2 M = -n bits/s/Hz. (9.11)
Binary Turbo Coded Modulation 269

The maximum likelihood decoder operates on the overall code

trellis. This decoder, in general, is too complex to implement. Al-
ternatively, a suboptimum technique, called multistage decoding,
can be used, resulting in the same asymptotic error performance
as the maximum likelihood decoding. A multistage decoder for a
multilevel turbo scheme is shown in Fig. 9.5.
The multistage decoder employs l component turbo decoders.
Each component code C i is successively decoded by the correspond-
ing decoder Di. The decisions at a lower level decoder will be used
in higher level decoders. At stage i, the decoder Di processes not
only the received signal sequence, but also the decisions of the pre-
vious decoding stages j, denoted by x.i, j = 1, 2, ... , i - 1. As long
as error free decisions xj = x j are produced by the decoder Dj, the
multistage decoder can achieve the maximum likelihood decoding
performance.
I I xl
I Turbo Decoder Dl I
xl-Ii ... x2xl

: Turbo Decoder D2 ~
x2 x2

r
,I Turbo Decoder DI : xl i ...
xl

Fig. 9.5: Multilevel turbo decoder

An important issue in the code design is the choice of component

codes and their code rates. Wachsmann and Huber [8] proposed a
technique for selecting the component code rates. In this design,
the component code rate at a particular modulation level, is chosen
to be equal to the capacity of the equivalent binary input chan-
nel associated with that level. As the overall channel capacity is
equal to the sum of the channel capacities for all levels, this de-
sign results, for infinite code lengths, in theory, in error free coding.
Turbo codes come close to the Shannon capacity limit and provide
almost error free coding. Therefore, they are suitable candidates
270 Thrbo Trellis Coded Modulation Schemes

for component codes in a multilevel scheme. Another good prop-

erty is that due to their good performance it can be assumed that
there is negligible error propagation between the modulation levels.
This is an important conclusion which enables the use of multistage
decoding, as it asymptotically leads to the optimum result. How-
ever, for small block sizes, there can be a significant loss in terms of
the signal-to-noise ratio needed to achieve a certain error rate. For
example, for block sizes of the order of several hundred, the loss of
a multilevel coding with multistage decoding is about 1dB relative
to random coding [8]. In this example of multistage decoding there
are no iterations between levels and only hard decisions are passed
from one stage to the next one.

9.3 Turbo Trellis Coded Modulation

9.3.1 Schemes with Alternate Puncturing of
Parity Digits
A method of combining turbo codes with multilevel modulation
[6] [7] involves parallel concatenation of two recursive Ungerboeck
type trellis codes with rate k/(k + 1). Fig. 9.6 shows the encoder
structure comprising of two recursive convolutional encoders linked
by a symbol interleaver and followed by a signal mapper. The
encoder operates on a message block of N groups of k information
bits, where N is the interleaver size. The message sequence, denoted
by c, is given by

where Ct is a group of k information bits at time t

The encoder maps the input sequence into a block consisting of N

encoded binary symbols,
Thrbo Trellis Coded Modulation 271

where Vt is an encoded symbol at time t, of length n (k + 1)

binary digits, given by

A symbol interleaver permutes the information bits in a symbol-

wise way. That is, the ordering of k information bits arriving at
the interleaver at a particular instant, remains unchanged.
For the component trellis code, some of the input bits may not
be encoded. In practical implementations these inputs do not need
to be interleaved, but are directly used to select the final point in a
signal subset. At the receiver the values of these bits are estimated
by subset decoding [1].
c
k n=k+l

>-.x
XU Selector
IEncoder 11 . IMapper 1:
x'
...
1 Symbol 1 1 Symbol .1
Interleaver Deinterleaver
...
n=k+l
xl
::1Encoder 21 .IMapper 2:

Fig. 9.6: Turbo TCM encoder with parity symbol puncturing

The encoded symbol sequence v is mapped into the modulated

sequence xu, given by
XU = (xr,x~, ... ,x~, ... ,x'N)
where x~ is a symbol from the modulation signal set, consisting of
2k+1 points, given by

U
xt = Xt,I + JXt,Q
U . U

where X~I is the in-phase and x~Q is the quadrature component of

the modulation signal x~. The mapping is based on the Ungerboeck
set partitioning method [1].
272 Turbo Trellis Coded Modulation Schemes

The output of the second encoder is deinterleaved. This ensures

that the k information bits which determine the encoded (k + 1)
binary digits of both the upper and lower encoder at a given time
instant are identical.
Note that the k input information bits are contained in both
modulated signals coming from the two parallel encoders. This
is in contrast to binary turbo codes where the information bits are
used only once, as they can be reconstructed from the other parallel
received sequence. To prevent the transmission of each information
bit twice, a selector is switched so that the modulated symbols
are transmitted alternately from the upper and the lower mapper.
In this way, for a signal set with 2k+1 points, a throughput of k
bits/sec/Hz is achieved.
An example for the 8-state encoder and 8-PSK modulation is
shown in Fig. 9.7. The system parameters are k = 2 and interleaver
size N = 6 symbols.
We will illustrate the encoding/modulation operation by an ex-
ample. Let us assume that a sequence of six information groups of
2 bits each (k = 2) enters the encoder

c= (00,01,11,10,00,11)

After encoding by a recursive systematic encoder and signal map-

ping based on the set partitioning rule, an 8-PSK sequence is gen-
erated by the upper encoder, denoted by xu,

XU = (0,2,7,5,1,6)
The information sequence is interleaved by a symbol interleaver and
encoded into the modulated sequence by the lower decoder, denoted
by xl,
Xl = (6, 7, 0, 3, 0, 4)
After deinterleaving this sequence becomes

Xl = (0,3,6,4,0, 7)
The selector alternately connects the upper and lower encoder to
the channel, transmitting the first symbol from the upper encoder,
Turbo Trellis Coded Modulation 273

Infobit pairs

).r=:::::r 8PSK symbols

_(_Crl,_C_2_,C_3_,_C_4,_C,5_'C_6_)_=<_0_0,_0_1,_1_1,_1_0,_0_0,_1_1
.....-I-----+-----,r------+j 8PSK 0,2,7,5,1,6
~lector h

Mapper ~ p

.-----.,~ 0,3,7,4,1,7

0,3,6,4,0,7

...---'--"-------, even positions to even positions

odd positions to odd positions Symbol
Deinterleaver

6,7,0,3,0,4
~______-4______- r______~8PSK 8PSK symbols
Mapper

Fig. 9.7: Example of a turbo trellis coded 8-PSK with parity symbol
puncturing

the second symbol from the lower encoder, the third symbol from
the upper encoder, the fourth symbol from the lower encoder, etc.
The transmitted sequence for this example is

x = (0,3,7,4,1,7)

Thus the parity symbol is alternately chosen from the upper and
lower encoder. Each information group appears in the transmitted
sequence only once.
For this coded modulation approach, the component code should
be chosen with no parallel transitions so that each information bit
274 Turbo Trellis Coded Modulation Schemes

can benefit from interleaving. However, this condition can be re-

laxed, if the interleaver does not keep a group of k information
bits unchanged [13J or if we desire a very high bandwidth efficiency
[7J. If the interleaver swaps the two information bits in the previ-
ous example, the code is allowed to have parallel transitions. This
operation ensures interleaving of information bits within a symbol.
To find good component codes, an exhaustive computer search
can be employed. The search goal is maximizing the minimum Eu-
clidean distance of each component code while randomly selecting
the parity check digits of each second symbol [6J.

9.3.2 Log-MAP Decoding Algorithm for Turbo

Trellis Coded Modulation with Punctured
Parity Digits
The decoder structure is shown in Fig. 9.8. In this scheme it is
impractical to use the MAP algorithm, as the extrinsic information
becomes either too large or too small, causing computational over-
flows. A Log-MAP algorithm is used instead in which the logarithm
of probabilities is computed and passed to the next stage. In all
other respects this algorithm follows the MAP algorithm, described
in [2J [3J.
The decoding process is very similar to the binary turbo code
except that the symbol probability is used as the extrinsic informa-
tion rather than the bit probability. The MAP decoding algorithm
for nonbinary trellises is called symbol-by-symbol MAP algorithm.
The MAP decoder computes the LLR of each group of informa-
tion bits Ct = i. The soft output A( Ct = i) is given by [3J

1 Pr{ Ct = ilrI}
A(Ct = i) = og Pr{ Ct = Olrr}
l:(I',I)EBi CXt-l (I') ,: (1',1) f3t(l)
(9.12)
log l:(l',I)EB~ CXt-l (I') ,2 (l', l) f3t(l)

,t
where i denotes an information group from the set, {O, 1, 2, ... , 2k -
I}, and the probabilities CXt(l), f3t(l) and (l' , l) can be computed
recursively [3J. The symbol i with the largest log-likelihood ratio
'furbo Trellis Coded Modulation 275

r _ _--+-"

. "
Decoder2
~----~ r-~----.
A2

Fig. 9.8: Turbo TCM decoder with parity symbol puncturing

in Eq. (9.12), i E {O, 1,2, ... , 2k - 1}, is chosen as the hard decision
output.
The MAP algorithm requires exponentiation and multiplication
operations to compute the log-likelihood ratio A(Ct). One way of
simplifying it is to work with the logarithms of probabilities of Cl!t(l) ,
f3t(l) and "(t(l', l), denoted by Cl!t(l), lit(l) and 'Yt(l', l), respectively.
The forward recursive variables can be computed as follows
M.-l
Cl!t (l) = log L eQt - 1 (1')+'Yt(l' ,1)
(9.13)
1'=0

with the initial condition

Cl!o(O) = 0

ao(l) = -00, 1 =J 0
and the backward recursive variables can be computed as

= log L
M.-l
lit (l) e13t+ 1 (1')+'Yt+1 (l,l') (9.14)
1'=0
276 Turbo 'Itellis Coded Modulation Schemes

with the initial condition

;B7(0) = 0

;B7(l) = -00, l =1= 0

where
2k-l
'Yt(l',l) = log L 'Y;(l',l) (9.15)
i=O
The branch transition probability at time t is calculated as

. { Pt(i)exp (_(rt'I-xtI(I))2+(rt,Q-xLQ(I))2) for (l' l) E Bi

'Y;(l', l) = Pt(O) 20'2 " t
0, otherwise

where rt,I and rt,Q are the channel output in-phase and quadrature
components at time t respectively, xtI(I) and xtQ(I) are the mod-
ulated in-phase and quadrature components at time t, associated
with the transition St-l = I' to St = I and input Ct = i, respectively,
and pt(i) is the a priori probability of Ct = i.
The (tt(l) and ;Bt(l) in Eqs. (9.13) and (9.14) can be calculated
using the Jacobian algorithm

log (e1h + e62) = max (6I, ( 2 ) + log (1 + e-162-611) (9.16)

= max (61 , ( 2 ) + Ie (162 - ( 1 1)
where le(') is a correction function, which can be implemented using
a look-up table.
The expression log (e 61 + e62 + ... + e6n ) can be computed ex-
actly by a recursive algorithm using Eg. (9.16) as follows

n _ log (6. + e6n ) , 6. = e61 + e62 + .. ·e6n - 1 = e6

logLe6; - max (log 6., 6n) + Ie (I log 6. - 6nl)
i=1 - max (6, 6n ) + Ie (16 - 6n l)
The iterative process of the symbol-by-symbol MAP algorithm
is similar to that of binary turbo decoders except that there is a
difference in the nature of information exchange between the two
component decoders [6]. For binary turbo decoders, a soft output
Turbo Trellis Coded Modulation 277

can be split into three terms. They are the a priori information gen-
erated by the other decoder, the systematic information generated
by the code information bit and the extrinsic information generated
by the code parity check digit. The extrinsic information is inde-
pendent of the a priori and systematic information. The extrinsic
information is exchanged between the two component decoders. In
contrast to binary turbo codes, in turbo TCM we cannot separate
the influence of the information and the parity-check components
within one symbol. The systematic information and the extrinsic
information are not independent. Thus both systematic and ex-
trinsic information will be exchanged between the two component
decoders. The joint extrinsic and systematic information of the first
Log-Map decoder, denoted by AI,es(Ct = i), can be obtained as

Al,es (Ct = Z') = AI (Ct = Z') - Iog Pt(O)

Pt ( i) (9.17)

The joint extrinsic and systematic information AI,es(Ct = i) is

used as the estimate of the a priori probability ratio at the next de-
coding stage. After interleaving, it is denoted by AI,es(Ct = i). The
joint extrinsic and systematic information of the second decoder is
given by
(9.18)
In the next iteration the a priori probability ratio in Eq. (9.17) is
replaced by the deinterleaved joint extrinsic and systematic infor-
mation from the second decoding stage, denoted by A2,es (Ct = i).
It is worth noting that for symbol-by-symbol MAP decoding,
each component decoder should avoid to use the same systematic
information twice in every iterative decoding step [6]. In turbo
TCM, each decoder alternately receives the noisy output of its own
encoder and that of the other encoder. That is, the parity symbols
in every second received signal belong to the other encoder and
need to be treated as punctured.
For example, we consider the first decoder. For every odd re-
ceived signal, the decoding operation proceeds as for the binary
turbo codes when the decoder receives the symbol generated by
its own encoder, The only difference is that the extrinsic informa-
tion is replaced by the joint extrinsic and systematic information.
278 Thrbo Trellis Coded Modulation Schemes

However, for every even received signal, the decoder receives the
punctured symbol in which the parity digit is generated by the
other encoder. The decoder in this case ignores this symbol by set-
ting the branch transition metric to zero. The only input at this
step in the trellis is the a priori component obtained from the other
decoder. This component contains the systematic information.

9.3.3 SOYA Decoding Algorithm for Turbo

Trellis Coded Modulation with
Punctured Parity Digits
The SOYA decoder structure is similar to that shown in Fig. 9.B.
As in decoding of binary turbo codes, we define the decoder soft
output A(Ct = i), where i E {O, 1,2, ... , 2k - 1}, as

Iog Pr{Ct = ilrI}

Pr{Ct = 0lrd
T

J-L~ - J-L~ (9.19)

where J-L~ is the minimum path metric for Ct = 0, and J-L~ is the
minimum path metric for Ct = i.
We define the branch metric, assigned to a trellis branch at time
t, as
(9.20)
where rt,I and rt,Q are the channel outputs of the in-phase and
quadrature components at time t, respectively, Xt,I and Xt,Q are
the modulated in-phase and quadrature components of the branch,
generated by input Ct = i, respectively, and pt(i) is the a priori
probability of Ct = i. The path metric, for a path x in the trellis,
can be computed as
1/:
T

J-Lx =L (9.21)
t=l

As in symbol-by-symbol MAP algorithm, for this scheme, we

cannot separate the information bit and the parity-check binary
digit in one coded symbol. Thus a decoder output can only be
split into the a priori component and joint extrinsic and systematic
Turbo Trellis Coded Modulation 279

components. The joint extrinsic and systematic information of the

first SOYA decoder, denoted by AI,es(Ct = i), is given by

1 pt(i)
AI,es (Ct = Z.) = AI (Ct = Z.) - og Pt(O) (9.22)

where Al (Ct = i) is the soft output of the first decoder and log ;:(~)
is the a priori information for Ct = i. The joint extrinsic and sys-
tematic information is used as the a priori probability at the next
decoding stage as

(9.23)

9.3.4 Performance of Turbo Trellis Coded Mod-

ulation with Punctured Parity Digits
Let us consider the performance of rate 2/3 turbo trellis coded
8-PSK schemes with bandwidth efficiency 2 bits/s/Hz on AWGN
channels. The generator polynomials of the 4, 8 and 16-state com-
ponent codes are shown in Table 9.1. The bit error probabilities of
the codes with various interleaver sizes of 1024 and 4096 are simu-
lated. The number of iterations is 8 for the interleaver size of 1024
and 18 for the size of 4096. The simulation results based on the
iterative SOYA decoding algorithm are shown in Figs. 9.9 - 9.11
[16].

Table 9.1: Rate 2/3 turbo trellis coded 8-PSK schemes

Trellis Generator Polynomials
States go gi g2
4 5 6 7
8 17 15 13
16 35 27 23

Performance comparison of the Log-MAP and SOYA for the 8-

state turbo trellis code with interleaver size 1024 is shown in Fig.
280 Thrbo Trellis Coded Modulation Schemes

N=1024,1=8
N=4096,1=18

10'"

3.4 3.6 3.8 4 4.2 44 4.6 4.8 5

EblNO(dB)

Fig. 9.9: Performance of the rate 2/3, 4-state turbo trellis coded 8-PSK
with various interleaver sizes on an AWGN channel, SOYA decoding
algorithm, the number of iterations I, bandwidth efficiency 2 bits/s/Hz

9.12. The SOYA is worse by 0.4 dB at the BER of 10-3 and by

0.2dB at the BER of 10- 5 .
The performance of rate 3/4 turbo trellis coded 16-QAM with
bandwidth efficiency 3bits/s/Hz on AWGN channels is also con-
sidered. The generator polynomials of the component codes with
various numbers of trellis states are given in Table 9.2. The effect of
the variable interleaver size on the performance of the codes with
various numbers of trellis states on AWGN channels is shown in
Figs. 9.13 - 9.15.

The comparative results of Log-MAP and SOYA decoded turbo

trellis coded 16-QAM with bandwidth efficiency 3 bits/s/Hz, with
various interleaver sizes and number of trellis states on AWGN
channels are also shown in Figs. 9.13 - 9.15. It is interesting to
note that the difference between the two algorithms is negligible
Turbo Trellis Coded Modulation 281

j=:= N=1024.1=8
N=4096. 1=18

10-3

10-6 ' - - - - ' - - - ' - - - - - ' - - - - - ' - - - - " ' - - - - ' - - - - ' - - - - ' - - - - ' - - - - '
34 3.5 3.6 3.7 38 3.9 4 4.1 4.2 43 4.4
EbINO(dS)

Fig. 9.10: Performance of the rate 2/3, 8-state turbo trellis coded 8-PSK
with various interleaver sizes on an AWGN channel, SOYA decoding
algorithm, the number of iterations I, bandwidth efficiency 2 bits/s/Hz

Table 9.2: Rate 3/4 turbo trellis coded 16-QAM schemes

Trellis Generator Polynomials
States go gl g2 g3
4 5 6 4 7
8 17 15 5 13
16 23 35 33 37

for a low number of states and increases with the growing number
of states, reaching a value of about O.3dB at the BER of 10- 5 for
the 16-state code, as shown in Fig. 9.15.
282 Turbo Trellis Coded Modulation Schemes

N=1024,1=8
N=4096, 1=18

3.6 3.7 3.8 3.9 4 4.1 4.2 4.3 4.4

EbINO(dB)

Fig. 9.11: Performance of the rate 2/3, 16-state turbo trellis coded 8-
PSK with various interleaver sizes on an AWGN channel, SOYA de-
coding algorithm, the number of iterations I, bandwidth efficiency 2
bits/s/Hz

9.3.5 Schemes with Puncturing of Systematic

Bits
Parallel concatenation of two recursive trellis codes with puncturing
of systematic bits was proposed by Benedetto, Divsalar, Montorsi
and Pollara [9]. The basic idea of the scheme is to puncture the
output symbols of each trellis encoder and select the puncturing
pattern such that the output symbols of the parallel concatenated
code contain the input information only once. In contrast to the
scheme with symbol interleaving and puncturing of parity digits,
this scheme uses multiple bit interleaving and punctures systematic
bits of both component trellis codes.
In the code structure, two high rate k/(k + 1) component en-
coders are linked in parallel. We assume that k is even. Each
encoder generates one parity and k information binary digits. Both
Turbo Trellis Coded Modulation 283

- N=1024, SOVA
~ N=1024,Log-~P

10-3

10-s '---'---'---':----'':-----::'':-----::'----::'----"-:---'''--J.----'
3.4 35 36 37 3.8 3.9 4 41 42 4.3 44
EblNO(dB)

Fig. 9.12: Performance comparison of the Log-MAP and SOYA for the
rate 2/3, 8-state turbo trellis coded 8-PSK with interleaver size 1024 on
an AWGN channel, bandwidth efficiency 2 bits/s/Hz

of these two parity digits are transmitted in their respective modu-

lated signals. In order to avoid transmission of each information bit
twice, the first k/2 information bits are used at the upper mapper
and the second k/2 at the lower mapper. That is, k/2 information
bits are punctured at each encoder. However, all k information
bits affect the states of both encoders. The total number of en-
coded bits is k + 2. These bits can be mapped to either M-PSK
or M-QAM signal constellations. If we map 1 + k/2 encoded bits
of each component trellis code to an M-PSK or M-QAM signal,
where M = 21+k/2, we can achieve a bandwidth efficiency of k/2
bits/s/Hz. Since two modulated signals are generated in every en-
coding interval, this scheme is equivalent to a multi-dimensional
M - PSK or M -QAM trellis code. For M -QAM we can also map the
1 + k/2 encoded bits of the first trellis code to I-channel and the
1 + k/2 encoded bits of the second trellis code to Q-channel, where
284 Turbo Trellis Coded Modulation Schemes

10-' , - - - - - - r - - - - - , - - - - - - - - - - , - - - - - r - - - - - ,

- N=1024, SOYA
~ N=4096, SOYA
- *- N=1024, Log-MAP
- e - N=4096, Log-MAP

Fig. 9,13: Performance comparison of the Log-MAP and SOYA for the
rate 3/4, 4-state turbo trellis coded 16-QAM with various interleaver
sizes on an AWGN channel, bandwidth efficiency 3 bits/s/Hz

M = 2k+2. In this case, a bandwidth efficiency of k bits/s/Hz can

be achieved.
To optimize the code performance, the component trellis code
should be designed to maximize the code effective free Euclidean
distance for a given modulation and signal mapping. The effective
free Euclidean distance of a turbo trellis coded modulation scheme
is defined by the minimum Euclidean distance between all pairs of
coded sequences, whose input sequences have a Hamming distance
of 2 [9].
Decoding is a straightforward application of the iterative symbol-
by-symbol MAP algorithm for the binary turbo codes. The only
difference is that 1) the extrinsic information computed for a sym-
bol needs to be converted to a bit level since they are carried out on
a bit level, and 2) after the interleaving/deinterleaving operations,
the bit a priori probabilities need to be converted to a symbol level
Turbo Trellis Coded Modulation 285

- N=1024, SOYA
- N=4096, SOYA
-*- N=1024, Log-MAP
- e - N=4096, Log-MAP
\
,
\
\
\
10-3 \ .
\
<1l
a: \
w
Ol \
\
\
10'" \
\

'q
\
\
10-<1
\
\

10-" '----.L---'----e----'-----'---$---'------'------'------'-----'
5 5.1 5.2 53 5.4 5.5 56 5.7 58 59 6
EblNO(dB)

Fig. 9.14: Performance comparison of the Log-MAP and SOYA for the
rate 3/4, 8-state turbo trellis coded 16-QAM with various interleaver
sizes on an AWGN channel, bandwidth efficiency 3 bits/s/Hz

since they will be used in the branch transition probability calcu-

lation in the symbol MAP algorithm.
For a symbol of a group of k information bits given by
(9.24)
where Cj = 0,1, j = 1,2, .,., k. If we denote the extrinsic informa-
tion of the symbol c by Ae(c), the extrinsic information of the jth
bit can be represented by [9]
EC:CJ =l eAe(c)
Ae(cj) = log Ec:c1 =o eAe(c) (9.25)

After the interleaving/deinterleaving operations, the a priori prob-

ability of any symbol can be given by [9]

(9.26)
286 Turbo Trellis Coded Modulation Schemes

10° r - - - , - - - - - - - , - - - - , - - - , . . - - - - , - - - - - - - ,

- N=1024. SOVA
-e- N=4096. SOVA
- • - N=1024. Log-MAP
- e - N=4096. Log-MAP

ffiIII 10-3 Q.
\
\
10~
~
\

,,
10-' , ,
\ :
\.
~
,,
Q. ,
10-6
4.8 5 5.2 5.4 5.6 5.8 6
EblNO (dB)

Fig. 9.15: Performance comparison of the Log-MAP and SOYA for the
rate 3/4, 16-state turbo trellis coded 16-QAM with various interleaver
sizes on an AWGN channel, bandwidth efficiency 3 bits/s/Hz

We will illustrate the turbo trellis coded modulation schemes

by examples. First we consider a coded 16-QAM scheme with 2
bits/s/Hz as shown in Fig. 9.16. The number of information bits
in a block, k, is 2. Two 16-state, rate 2/3 trellis codes are linked
by two bit interleavers in parallel. The generator polynomials are
go = (23), gl = (16), g2 = (27).
The encoded bits of the two component trellis codes are punctured
and then mapped to the I and Q branches of a 16-QAM signal
according to natural mapping, respectively. The squared effective
free Euclidean distance of the code is 7.2. The BER performance
of the coding scheme with two interleavers of size 16384 each and 8
iterations is shown in Fig. 9.17. The performance of the pragmatic
turbo coded 16-QAM with the same bandwidth efficiency and in-
terleaver size 32768 [14] is also shown in the figure. It is clear that
the turbo trellis coded 16-QAM with systematic symbol puncturing
Turbo Trellis Coded Modulation 287

outperforms the pragmatic turbo coded 16-QAM by about 0.1 dB

at a BER of 10- 5 .

Fig. 9.16: Thrbo trellis coded 16-QAM with systematic symbol punc-
turing

Another example is a turbo trellis coded 8-PSK modulation

scheme as shown in Fig. 9.18. Each parallel information sequence
is interleaved before passing to the lower encoder. For the given
example, the 16-state component codes have the same code rate
of 4/5 and the encoder contains four interleavers. The generator
polynomials are

go = (23), gl = (14), g2 = (16), g3 = (21), g4 = (31).

The encoded bits of each component trellis code are punctured and
then mapped to an 8-PSK signal constellation with reordered map-
ping [9]. The spectral efficiency is 2 bits/sec/Hz. The squared
effective free Euclidean distance of the code is 6.34. The BER per-
formance of the code with four interleavers of size 4096 each and
variable number of iterations is shown in Fig. 9.19 [9].
288 Turbo Trellis Coded Modulation Schemes

10-'
.. ..
.. . .. ....
.. --+- Turbo TCM, N=2*16384
-e-- Pragmatic turbo code, N=3276
10-2 ,.
.. ....
.... ..

10'"
.. .. .. .. ..
..

II:
w 10"
<Il
..

10-'
"
..

10'"
..: .. ..
..

10-'
2.6 265 2.7 275 2.8 2.85 2.9 295 3
EblNo (dB)

Fig. 9.17: Performance comparison of the turbo trellis coded 16-QAM

with systematic symbol puncturing and the pragmatic turbo coded 16-
QAM with bandwidth efficiency 2 bits/s/Hz and interleaver size 32768
on an AWGN channel, the number of iterations 8

9.4 I-Q Turbo Coded Modulation for

Fading Channels
The performance of a coded modulation scheme over a Rayleigh
distributed fading channel depends strongly on the code minimum
symbol-wise Hamming distance. In [15], Al-Semari and Fuja pro-
posed I-Q TCM schemes that could have substantially higher mini-
mum symbol-wise Hamming distances than the conventional TCM
schemes. They showed that significant performance improvement
could be achieved by using simple convolutional codes optimized in
terms of minimum Hamming distance combined with I-Q parallel
encoding and mapping. In this section, we use the turbo codes op-
timized for binary modulation combined with the I-Q approach to
construct good bandwidth efficient coding schemes [12]. In an I-Q
I-Q Turbo Coded Modulation for Fading Channels 289
c A
c:l! ~7-s0
c 7-B 0
A
-
c:1 7~
78
EKJ ,'"
-\,1,.1 -1''''
\,1,.1
,f-.
,17
,f-.
-\ !.I

rI~3~t~
,'"
\1,.1
~
U)
P-.
,1-. 00
\'l.I X
I-+-
A..... ~

/es o
U)
P-.
00
-So
A.-
A7-a
8
EKJ ,V
-1'''' -I't'\
~\;'!7
1"1-.
,p
I"~
'17
-
I"f-.
\.1/
,,- I-.
7"

Fig. 9.18: Turbo trellis coded 8-PSK with systematic symbol puncturing

turbo encoder, each group of information bits is associated with

its parity check digit which is alternately obtained from the first
and second turbo component encoders. The coded digits from the
two component encoders are mapped to the in-phase and quadra-
ture components of a modulation signal constellation, separately,
according to Gray code mapping. The coded modulation scheme
has the underlying I-Q pragmatic structure which can achieve a
larger minimum symbol-wise Hamming distance. In addition, the
less complex symbol-by-symbol MAP iterative decoding algorithm
can be employed in the decoder.
290 Turbo Trellis Coded Modulation Schemes

-+-- N=4*4096, 1=6

-- N=4*4096, 1=8

10~

10-7 L - _ - ' -_ _' - - - _ - - ' - _ - ' L - _ - ' - _ - - - ' ' - - - - _ - - ' - _ - - J

3.3 3.35 3.4 3.45 3.5 3 55 3.6 3.65 3.7
EbINo (dB)

Fig. 9.19: Performance of the turbo trellis coded 8-PSK with systematic
symbol puncturing with bandwidth efficiency 2 bits/s/Hz and interleaver
size 16384 on an AWGN channel, the number of iterations I

9.4.1 I-Q Coded Modulation Structure

We consider a rate 1/2 turbo coded 16-QAM scheme with band-
width efficiency 2 bits/s/Hz. The turbo encoder structure is illus-
trated in Fig. 9.20. Two identical component systematic recursive
convolutional encoders with the rate 1/2 and the memory order
v are separated by a random interleaver with size N. The first
encoder operates directly on the input sequence given by

C=(C1,C1],"',CN)' (9.27)

The output contains the information sequence

(9.28)

and the parity check sequence

VIp -- (v11P, v1p 1], ••• , vN1P ) . (9.29)
I-Q Turbo Coded Modulation for Fading Channels 291

The input sequence to the second encoder is permuted by the in-

terleaver I of size N. The parity check sequence of the second
component encoder is deinterleaved to ensure that its output se-
quence
v 2p = (V1
2P 2p 2P)
,V2 , " ' , v N (9.30)
has the same order as the input information sequence c. By using
an odd-even puncturing technique, the global turbo code rate is
1/2 and the turbo code sequence, denoted by v, consists of each
information bit followed by its parity check digit which is alternately
obtained from the first and the second component encoder. If we
assume the interleaver size N is even, the code sequence can be
represented as

v = ( v 11s ,V11p ,v21s ,V22p , " ' , VN-1,

1s 1p 1s 2P)
VN - 1 ' vN , vN (9.31)

The code sequence is mapped to a 16-QAM constellation. In the

given example, every two encoded digits select the in-phase or
quadrature component of the modulated signal according to Gray
code mapping. The output of the first turbo component encoder is
mapped to the in-phase component of the modulated signal while
the output of the second encoder is mapped to the quadrature com-
ponent.

Multi- Punc- Signal

1------:--+\
plexer turer

Fig. 9.20: I-Q turbo trellis coded 16-QAM

This I-Q coded modulation approach can be applied to produce

coding schemes with high bandwidth efficiency. For example, to
292 'furbo Trellis Coded Modulation Schemes

transmit 4 bits/s/Hz, a rate 2/3 turbo coded 64-QAM is used.

The three output digits from the first turbo component encoder
specify the in-phase component of a 64-QAM signal according to
the Gray code mapping, while the three output digits from the
second component encoder specify the quadrature component of a
64-QAM signal. With an even-odd puncturing, one 64-QAM signal
is produced every two turbo coding cycles.

9.4.2 The Decoder

In the proposed system, each group of information bits associated
with its parity check digit is mapped to the signal subsets in the sig-
nal constellation. An iterative decoder with the symbol-by-symbol
MAP algorithm, which has been derived for a non-binary trellis can
be applied. The algorithm has lower complexity than the bit-by-bit
MAP algorithm, as the receiver does not need to calculate the log-
likelihood ratio (LLR) for each encoded binary digit based on the
received noisy symbol before passing it to the binary turbo decoder
[4] or convert the extrinsic information between a bit level and a
symbol level [9].
Furthermore, we assume the channel fading attenuation can be
perfectly recovered at the receiver, and hence channel state infor-
mation may be used to decode turbo codes for fading channels. In
this case, the branch metric should be modified by multiplying the
transmitted signals in the trellis by the channel attenuations. This
metric is not the optimum one, but the use of channel state infor-
mation gives better performance than decoding with the standard
Euclidean distance metric.

9.4.3 Performance of I-Q Turbo Coded Modu-

lation on Fading Channels
The bit error rate performance of a coding scheme with bandwidth
efficiency 2 bits/s/Hz on an independent Rayleigh fading channel is
estimated by simulations. The generator polynomials are go = (23)
and gl = (35).
In the simulation, iterative decoding with symbol-by-symbol
I-Q TUrbo Coded Modulation for Fading Channels 293

Log-MAP algorithm is employed. The interleaver size is chosen to

be 1024 and 4096. The number of iterations is 8. The simulation
results are shown in Fig. 9.21.

I~ N=1024,1=8
N=4096, 1=8
I·,,

ffi
m
10-3

10-4

4.5 5,5 6 6.5

EblNo (dB)

Fig. 9.21: Performance of the I-Q turbo coded 16-QAM with bandwidth
efficiency 2 bits/s/Hz and various interleaver sizes on a Rayleigh fading
channel

This scheme is also compared with the pragmatic binary turbo

coded modulation of the same spectral efficiency reported in [4]. We
simulated a Log-MAP decoding with 8 iterations for the pragmatic
binary turbo coded modulation of interleaver size 4096. The result
is shown in Fig. 9.22. It is clear that the I-Q scheme outperforms
the pragmatic scheme by about 0.5 dB at a BER of 10- 5 for the
same bandwidth efficiency and interleaver size.
294 Turbo Trellis Coded Modulation Schemes

10-'.------.-------.------.------.
-+- 1-0 turbo code, N=·1096
-e- Pragmatic turbo code, N=40

4.5 5 5.5 6.5

EbINo(dB)

Fig. 9.22: Performance comparison of the I-Q turbo coded 16-QAM

and the pragmatic turbo coded 16-QAM with bandwidth efficiency 2
bits/s/Hz and interleaver size 4096 on a Rayleigh fading channel
Bibliography

[1] G. Ungerboeck, "Channel coding with multilevel/phase signal-

ing," IEEE Trans. Inform Theory, vol. 25, no. 1, Jan. 1982, pp.
55-67.
[2] J. Hagenauer, P. Robertson, and L. Papke, "Iterative ('Turbo')
decoding of systematic convolutional codes with the MAP and
SOYA algorithms," in Proc. of the 1994 ITC Conference on
Source and Channel Coding, Munich, Oct. 1994, pp. 1-9.
[3] B. Vucetic, "Iterative decoding algorithm," in PIMRC'97,
Helsinki, Finland, Sep. 1997, pp. 99-120.
[4] S. LeGoff, A. Glavieux, and C. Berrou, "Turbo codes and high
efficiency modulation," in Proc. of IEEE ICC'94, New Orleans,
LA, May 1994, pp. 645-649.
[5] H. Imai and S. Hirakawa, "A new multilevel coding method
using error correcting codes," IEEE Trans. Inform. Theory,
vol. 23, no. 3, May 1977, pp. 371-377.
[6] P. Robertson, and T. Woerz, "Novel coded modulation scheme
employing turbo codes," Electronics Letters, vol. 31, no. 18,
Aug. 1995, pp. 1546-1547.
[7] P. Robertson and T. Woerz, "Bandwidth efficient turbo trellis-
coded modulation using punctured component codes," IEEE
Journal on Selected Areas in Communications, vol. 16, no. 2,
Feb. 1998, pp. 206-218.
[8] L. U. Wachsmann and J. Huber, "Power and bandwidth ef-
ficient digital communication using turbo codes in multilevel
296 BIBLIOGRAPHY

codes," European Trans. Telecommun., vol. 6, no. 5, Sep./Oct.

1995, pp. 557-567.

[9] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, "Par-

allel concatenated trellis coded modulation," in Proc. of IEEE
ICC'96, 1996, pp. 974-978.

[10] T. M. Duman, "Turbo codes and turbo-coded modulation:

Analysis and performance bounds," Ph.D. dissertation, Elect.
Comput. Eng. Dep., Northeastern Univ., Boston, MA, May
1998.

[11] D. Divsalar and F. Pollara, "On the design of turbo codes,"

TDA Progress Report 42-123, JPL, Nov. 1995, pp. 99-12l.

[12] J. Yuan, B. Vucetic, and W. Feng, "I-Q turbo coded modu-

lation," in Proc. of IEEE GLOBECOM'98, Communications
Theory Mini Conference, Sydney, Australia, Nov. 1998, pp.
191-195.

[13] W. Blackert and S. Wilson, "Turbo trellis coded modulation,"

in Proc. CISS'96, 1996.
[14] S. A. Barbulescu, W. Farrell, P. Gray, and M. Rice, "Band-
width efficient turbo coding for high speed mobile satellite
communications," in Proc. International Symposium on Turbo
Codes, Brest, France, Sep. 1997, pp. 119-126.

[15] S. A. AI-Semari and T. E. Fuja, "I-Q TCM: Reliable commu-

nication over the Rayleigh fading channel close to the cutoff
rate", IEEE Trans. Inform Theory, vol. 43, no. 1, Jan. 1997,
pp. 250-262.
[16] B. Vucetic, W. Feng, and J. Yuan, "Performance of turbo
and serial concatenated convolutional codes," Technical Re-
port, The University of Sydney, Aug. 1997.
 Chapter 10

Applications of Turbo
Codes

Turbo and serial concatenated convolutional codes have been pro-

posed for various communication systems, such as deep space, cel-
lular mobile and satellite communication networks. The standard
turbo codes are presented in this chapter.

10.1 Turbo Codes for Deep Space Com-

munications
NASA's next generation deep space transponder will contain a
turbo code [1]. The Consultative Committee for Space Data Sys-
tems (CCSDS) has adopted 16-state rate 1/2, 1/3, 1/4 and 1/6
turbo codes as a new standard for telemetry channel coding [2]-[5].
The turbo encoder block diagram is shown in Fig. 10.1 [3].
The rate 1/3 turbo code is obtained by parallel concatenation of
two identical 16-state rate 1/2 RSC encoders, where the systematic
information bit of the second component encoder is eliminated. The
generator matrix of the RSC code is given by
G(D) = [1
1+D+D3+ D 4]
1+D3+D4

The rate 1/2 turbo code is obtained by puncturing every other

parity symbol of each component code in the rate 1/3 turbo en-
coder.

B. Vucetic et al., Turbo Codes

© Springer Science+Business Media New York 2000
 CIl
OJ
'"C
U
01 ~ Vo
0
-EI ~l+) ~l+) ~l+J--- VI
~
.....
01
CIl
~l+ ) l~l+J---V2
s::
0
'~I t--'--M+) I~l+) I~l+) T~l+J--- V3
.~
-a
~I c
I
VI
m •••
Not used
V3
I •
• = Take every symbol Rate 1/2 1/3 1/4 1/6
o = Take every other symbol
Fig. 10.1: CCSDS turbo encoder block diagram
ex>
0')
C"I
TUrbo Codes for CDMA2000 299

The rate 1/4 turbo code is obtained by parallel concatenation

of a 16-state rate 1/3 RSC encoder and a 16-state rate 1/2 RSC
encoder whose systematic information bit is eliminated. The gen-
erator matrix of the rate 1/3 RSC code is given by

G(D) = [1 1+D+D2+D3+D4
1+D3+D4

The rate 1/2 RSC code is the same as the component code in the
rate 1/3 turbo encoder.
The rate 1/6 turbo code is obtained by parallel concatenation
of a 16-state rate 1/4 RSC encoder and a 16-state rate 1/3 RSC
encoder whose systematic information bit is eliminated. The gen-
erator matrices of the rate 1/4 and 1/3 RSC codes are given by

G(D) = [1 l+D+D3+D 4
1+D3+D4
1+D+D2+D3+D4
1+D3+D4

and
1+D+D3+D4 1+D+D2+D3+D4
G(D) 1+D3+D4 1+D3+D4

respecti vely.

10.2 Turbo Codes for CDMA2000

In CDMA2000 proposal, turbo codes are recommended for both
forward and reverse supplemental channels in the 3rd generation
of wideband code division multiple access (CDMA) cellular mobile
systems [6]. The block diagram for the reverse link turbo encoder
is shown in Fig. 10.2. The component codes are identical (3, 1, 3)
RSC codes. The generator matrix of the RSC code is given by

G(D) = [1 1+D+D2+D3 ]
l+D+D3

By puncturing the component encoder outputs, variable code rates

of 1/4, 1/2 and 1/3 can be achieved. The rate 1/4 turbo code
is obtained by puncturing the parity digits V2 and v~ of the two
component encoders alternately. Puncturing the parity digits V1
and v~ results in a rate 1/3 turbo code. For the rate 1/2 turbo
300 Applications of Thrbo Codes

r-------------------------------------.Vo

c __---._.......+(

Fig. 10.2: The reverse link turbo encoder for CDMA2000

code, the parity digits V2 and v;,

and every other parity digits VI
and v~ for both encoders are punctured.
The forward link turbo encoder is based on two identical (3, 1,
3) RSC codes with generator matrix
G(D) -
-
[1 I+D+D3
I+D2+D3
I+D+D2+ D 3]
I+D2+D3

For the rate 1/4 turbo code, the parity digits VI from the first
component encoder and v; from the second component encoder
are alternately punctured. For the rate 1/3 turbo code, the parity
digits V2 and v;
for both encoders are punctured. The rate 1/2
turbo code is generated in a similar way as the punctured rate 1/2
code for the reverse link.

10.3 Turbo Codes for 3GPP

There are two candidates for the 3rd generation cellular mobile
communications based on the 3rd Generation Partnership Project
Turbo Codes for 3GPP 301

(3GPP) [7]. They include an 8-state rate 1/3 and 1/2 turbo code
and a 4-state rate 1/3 serial concatenated convolutional code.
The 8-state turbo code is generated by parallel concatenation
of two identical rate 1/2 RSC encoders with generator matrix

G(D) = [1
The output of the turbo encoders is punctured to produce coded
digits corresponding to the desired code rate of 1/3 or 1/2. For
rate 1/3, the systematic information bit of the second component
encoder is punctured. For rate 1/2, the parity digits produced
by the two component encoders are alternately punctured. The
encoder structure of the 8-state rate 1/3 turbo code is shown in
Fig. 10.3.
r--------------------------------------Vo
~------.t + 1 - - - -. . VI

}---------.(+ )----~ V2

Fig. 10.3: The turbo encoder for 3GPP

The encoder structure of the 4-state rate 1/3 serial concatenated

convolutional code is illustrated in Fig. IDA. The inner code is a
rate 1/2 RSC code with generator matrix
G(D) = [1
The outer code is of rate 2/3 obtained by puncturing a rate 1/2
RSC code with the same generator matrix as for the inner code.
302 Applications of Thrbo Codes

The rate 2/3 is obtained by puncturing every other parity-check

digit. The puncturing matrix is given by

p = [i ~l
Note that trellis termination is performed in both the turbo and
serial concatenated encoders. In the serial encoder, the tailing bits
of the outer encoder are included in the interleaver.

10.4 Turbo Codes for Satellite Commu-

nications
Turbo codes are also recommended for INMARSAT's new mobile
multimedia service [8] [9]. A data rate of 64 kbits/s is provided
based on the combined turbo coding and 16-QAM modulation. In
addition, INTEL SAT is currently investigating the application of
turbo codes for digital services.
 ~
d-
o
oo
0..
CD
rtl

0'
"1
en
~
5;
<:""t-
CD
o

Punc- Inter- ::s

I
r;.
turer leaver
~
~.

Fig. 10.4: The serial concatenated convolutional encoder for 3GPP

~
o
~
Bibliography

[1] C. D. Edwards, C. T. Steilzvied, L. J. Deutsch and L. Swanson,

"NASA's deep-space telecommunications road MAP," TMO
Progress Report 42-126, Feb. 1999.

[2] E. Vassallo, G. P. Calzolari, S. Benedetto, and G. Montorsi,

"More perform ant codes for deep-space missions: the turbo
codes option,"

[3] CCSDS recommendation for telemetry channel coding, Jun.

1998.

[4] D. Divsalar and F. Pollara, "Turbo codes for deep-space com-

munications," TDA Progress Report, 42-120, Feb. 1995.

[5] D. Divsalar and F. Pollara, "Multiple turbo codes for deep-

space communications," TDA Progress Report, 42-121, May.
1995.

[6] The cdma2000 ITU-R RTT Candidate Submission (0.18), Jul.

1998.

[7J 3rd Generation Partnership Project (3GPP) Technical Specifi-

cation Group: Radio Access Network Working Group 1, Mul-
tiplexing and Channel Coding, TS 25. 212 vl.l.O, June 1999.

[8] H. Feldman and D. V. Ramana, "An introduction to In-

mars at 's new mobile multimedia service," in Proc. Sixth Int.
Mobile Satellite Conj., Ottawa, June 1999, pp. 226-229.
[9] E. Trachman and T. Hart, "Research elements leading to the
development of Inmarsat's new mobile multimedia services,"
306 BIBLIOGRAPHY

in Proc. Sixth Int. Mobile Satellite Conj., Ottawa, June 1999,

pp. 209-212.

[10] S. A. Barbulescu, W. Farrell, P. Gray, and M. Rice, "Band-

width efficient turbo coding for high speed mobile satellite
communications," in Proc. International Symposium on Turbo
Codes, Brest, France, Sep. 1997, pp. 119-126.
Index

a posteriori probability, 118, carrier, 3

123, 138 catastrophic code, 52
a priori information, 277 CCSDS, 297
a priori probability, 145, 159 cellular mobile radio, 232
additive white Gaussian noise, channel, 3
6 channel capacity, 1, 6
AI-Semari S. A., 288 channel decoder, 4
asymptotic behavior, 170 channel encoder, 3
asymptotic coding gain, 28 channel model, 4
augmented state diagram, 63 channel noise, 4
average upper bound, 86 channel state information, 238
Cocke J., 30
backward recursion, 131, 145 code division multiple access,
Bahl L. R., 30 299
bandwidth efficiency, 5, 263, code matched interleaver, 213,
268, 283, 290, 292 219, 220
Benedetto S., 100, 282 code performance, 164
Berrou C., 1,73 code rate, 3
bidirectional SOYA, 127 coded modulation, 5
binary symmetric channel, 4 codeword, 3
bit error probability, 6, 20, 26, coding gain, 28, 263
80 communication system, 2
bit error rate, 121 complementary error function,
block code, 5, 13 25
block interleaver, 200 component code, 73, 107
helical simile, 204 compound error path, 82
odd-even, 202 concatenated code, 8, 73
branch, 31, 61 conditional weight enumerat-
branch metric, 124, 173 ing function, 22, 81
burst error, 73 constellation, 266
308 INDEX

controller canonical form, 48 concatenated, 201

convolutional code, 5, 37 error performance bound, 80,
convolutional interleaver, 206, 240
213, 222 Euclidean distance, 20
cross entropy, 165 extrinsic information, 162, 176,
cutoff rate, 87 277, 285
cyclic redundancy check, 165 fading channel, 254, 255
cyclic shift interleaver, 208, 222
fade rate, 233
data rate, 1, 2 fading channel, 233, 292
data transmission, 103 correlated, 258
decision depth, 125 independent, 256
deinterleaver, 193 power spectral density, 235
demodulator, 4 fading channel capacity, 236
digital communication, 1 feedback shift register, 47
discrete channel, 4 feedforward convolutional code,
distance spectral line, 95 87
distance spectrum, 26,93,103, feed forward shift register, 47
213 finite-state Markov process, 117,
dithered golden interleaver, 213 118
Divsalar D., 100, 211, 282 Forney G. D., Jr., 30, 37, 73,
Doppler shift, 232 206
Duman T., 87 forward recursion, 130, 145
free distance, 88
effective free distance, 92, 100, frequency dispersion, 233
111 Fuja T. E., 288
effective free Euclidean distance,
Gallager, 87
284
Gallieo, 8
Elias P., 37 Gaussian random variable, 20
encoder memory, 44 generating function, 63, 65
equivalent generator matrix, 46
generator matrix, 14, 40, 42,
error, 3 74, 100
error coefficient, 25, 26, 93, 246 generator polynomial, 41
error control coding, 1 Glavieux A., 1,73
error event probability, 24 Gray mapping, 264, 291
error floor, 171 greatest common divisor, 51
error path, 82
error pattern, 196 Hamming code, 22
INDEX 309

Hamming distance, 17, 25 iterative SOYA decoding, 171,

Hamming weight, 17,240 177, 180
hard decision decoding, 4
Jelinek F., 30
Hirakawa S., 267
joint extrinsic and systematic
Huber J., 269
information, 277, 279
I-Q trellis coded modulation, linear block code, 13
288 linear feedforward shift regis-
I-Q turbo coded modulation, ter, 38
290 log-likelihood ratio, 128, 138,
Imai H., 267 158, 266, 274
information source, 2 fading channel, 252
inner code, 73, 107 Log-MAP algorithm, 151, 274
input pattern, 198 Mason's rule, 63
length, 212 Massey-Berlekamp decoding, 8
low weight, 199, 213 Max-Log-MAP algorithm, 149
square or rectangular, 201 maximum a posteriori proba-
input-output weight enumer- bility, 74, 118, 138
ating function, 21 fading channel, 252
input-redundancy weight enu- maximum likelihood, 123
merating function, 21 maximum likelihood decoding,
interleaver, 73, 76, 194 18,30
delay, 195 maximum likelihood path, 124
device, 194 maximum spectral efficiency, 6
mapping function, 194 message, 2
size, 166, 197 military communication, 104
storage capacity, 195 minimum distance decoding, 19
structure, 198 minimum Euclidean distance,
vector, 194 29
interleaving performance gain, minimum free distance, 62
90, 197 minimum free Euclidean dis-
intrinsic information tance,62
fading channel, 255 minimum Hamming distance,
iterative decoding, 9, 157, 196 17,25
fading channel, 250 minimum path metric, 129
iterative MAP decoding, 159, mobile communication, 104,
163, 178 258,297
310 INDEX

modulation, 3 personal mobile communica-

M-PSK, 264,283 tion, 104
M -QAM, 264, 283 Pollara F., 100, 211, 282
16-QAM, 280, 286, 290 pragmatic turbo coded modu-
64-QAM, 292 lation, 264, 293
8-PSK, 272, 279, 287 primitive polynomial, 100
BPSK,23 pseudo-random interleaver, 77,
modulator, 3 210
Montorsi G., 100,282 punctured convolutional code,
multilevel coding, 267 67
multilevel turbo code, 267 puncturing, 79, 167, 265
multipath fading, 232 matrix, 79
multipath propagation, 232 period, 69
multipath waves, 232
Q-function, 25
multistage decoding, 269
Q-function bound, 25
NASA, 297 Ramsey J. L., 206
non-uniform interleaver, 210 random code, 196
nonrecursive convolutional code, random interleaver, 209, 220
87 rational transfer function, 48
normalized fade rate, 235 Raviv J., 30
number of iterations, 160 Rayleigh fading, 233
receiver, 4
observer canonical form, 48 recursive systematic convolu-
odd-even interleaver, 202 tional code, 74, 87
Odenwalder J. P., 7 Reed-Solomon code, 8, 73
optimal distance spectrum, 103 Rician factor, 236
optimum decoding, 157 Rician fading, 235
outer code, 73, 107
S-constraint, 211
pairwise error probability, 24 S-random interleaver, 211, 220,
parallel concatenated convolu- 222
tional code, 74 Salehi M., 87
parity check, 15 Sason 1.,87
parity check matrix, 16, 50 satellite communication, 7,103,
path, 31, 61 258, 297
path metric, 124, 173 satellite mobile communication,
perfect codes, 21 104
INDEX 311

serial concatenated code, 73, systematic block code, 15

107 systematic convolutional code,
serial concatenated convolutional 45
code, 74, 107, 182 systematic encoder, 49, 53
Shamai S., 87 systematic information, 277
Shannon C. E., 1
Shannon capacity limit, 7, 74, thermal noise, 4
196 Thitimajshima P., 1, 73
signal flow graph, 63 transfer function, 82, 88
signal-to-noise ratio, 6 modified, 83
signalling interval, 3 transmitter, 2
significant spectral line, 97 trellis based decoding
single error path, 82 algorithm, 117
sliding window SOYA, 135 trellis coded modulation, 9,
73, 263
soft decision, 73, 131
trellis construction, 30
soft decision decoding, 5
trellis diagram, 60
soft output, 117, 126
trellis termination, 77
soft output Viterbi algorithm,
turbo code, 1, 9
74,117,278
turbo coding, 73
fading channel, 254
turbo decoding, 74
soft-input soft-output, 147
turbo encoder, 74
soft-inputjsoft-output decoder,
turbo trellis coded modulation,
73
263, 270
source decoder, 4
puncturing of parity dig-
source encoder, 2
its, 270
space communication, 73, 103,
puncturing of systematic
297
bits, 282
spectral efficiency, 5
spread interleaver, 211 unequal error protection, 66
state, 31, 61 Ungerboeck G., 263
state diagram, 58 uniform interleaver, 81
survivor, 124 upper bound
symbol error rate, 121 bit error probability, 26, 85,
symbol rate, 3 108,246
symbol-wise Hamming distance, pairwise error probability,
288 241
system bandwidth, 5 word error probability, 25
312 INDEX

Viterbi A. M., 87
Viterbi algorithm, 30, 117, 122
voice transmission, 104
Voyager, 7

Wachsmann L. V., 269

weight distribution, 20, 63, 80
weight enumerating function,
21
Wolf J. K., 30
word error probability, 20, 25
word error rate, 121