Chapter 16

Turbo Coding
As noted in Chapter 1, Shannon's noisy channel coding theorem implies that arbitrarily low decoding
error probabilities can be achieved at any transmission rate R less than the channel capacity C by
using suÆciently long block (or constraint) lengths. In particular, Shannon showed that randomly
chosen codes, along with maximum likelihood decoding, can provide capacity achieving performance.
He did this by proving that the average performance of a randomly chosen ensemble of codes results
in an exponentially decreasing decoding error probability with increasing block (or constraint) length.
However, he gave no guidance about how to actually construct good codes, i.e., codes that perform at
least as well as the average, or to implement maximum likelihood decoding for such codes.
In the ensuing years after the publication of Shannon's paper in 1948, a large amount of research
was conducted into the construction of speci c codes with good error correcting capabilities and the
development of eÆcient decoding algorithms for these codes. Much of this research has been described in
the previous chapters of this book. Typically, the best code designs contain a large amount of structure,
either algebraic, as is the case with most block codes, such as RM and BCH codes, or topological,
as is the case with most convolutional codes, which can be represented using trellis or tree diagrams.
The structure is a key component of the code design, since it can be used to guarantee good minimum
distance properties for the code, as in the BCH bound, and since the particular decoding method, such
as the Berlekamp-Massey algorithm for BCH codes or the Viterbi algorithm for convolutional codes, is
based on this structure. In fact, one can say generally that the more structure a code contains, the easier
1
2 CHAPTER 16. TURBO CODING

it is to decode. However, structure does not always result in the best distance properties for a code,
and most highly structured code designs usually fall far short of achieving the performance promised
by Shannon.
Primarily because of the need to provide structure in order to develop easily implementable decoding
algorithms, little attention was paid to the design of codes with random-like properties, as originally
envisioned by Shannon. Random code designs, because they lacked structure, were thought to be too
diÆcult to decode. In this chapter, we discuss a relatively new coding technique, dubbed turbo coding
by its inventors, that succeeds in achieving a random-like code design with just enough structure to
allow for an eÆcient iterative decoding method. Because of this feature, these codes have exceptionally
good performance, particularly at moderate BER's and for large block lengths. In fact, for essentially
any code rate and information block lengths greater than about 104 bits, turbo codes with iterative
decoding can achieve BER's as low as 10 5 at SNR's within 1dB of the Shannon limit, i.e., the value
of Eb=N0 for which the code rate equals channel capacity. This typically exceeds the performance of
previously known codes of comparable length and decoding complexity by several dB, although the fact
that decoding is iterative results in large decoding delays.
Another important feature of turbo codes is that they are composed of two or more simple con-
stituent codes, arranged in a variation of the concatenation scheme introduced in Chapter 15, along
with a pseudorandom interleaver. Since the interleaver is part of the code design, a complete maximum
likelihood decoder for the entire code would be prohibitively complex. However, because more than one
code is used, it is possible to employ simple soft-in soft-out (SISO) decoders for each constituent code
in an iterative fashion, in which the soft output values of one decoder are passed to the other and vice
versa until the nal decoding estimate is obtained. Extensive experience with turbo decoders has shown
that this simple, suboptimum iterative decoding approach almost always converges to the maximum
likelihood solution.
In summary, turbo coding consists of two fundamental ideas: a code design which produces a code
with random-like properties, and a decoder design which makes use of soft output values and iterative
16.1. INTRODUCTION TO TURBO CODING 3
decoding. The basic features of turbo code design are developed in Sections 16.1 to 16.4 and the
principles of iterative turbo decoding are presented in Section 16.5.
16.1 Introduction to Turbo Coding

A block diagram of the basic turbo encoding structure is illustrated in Figure 16.1(a) and a speci c
example is shown in Figure 16.1(b). The basic system consists of an input information sequence, two
systematic feedback (recursive) convolutional encoders, and an interleaver. We will assume that the
input sequence includes m termination bits to return the rst encoder to the all-zero state S0 = 0,
where m is the memory order of the rst encoder. The information sequence (including termination
bits) is considered to be a block of length N and is represented by the vector
u = [u0 ; u1 ;


; uN 1 ] : (16.1)
Since encoding is systematic, the information sequence u is the rst transmitted sequence, i.e.,
h i
u = v(0) = v0(0) ; v1(0) ;


; vN(0) 1 : (16.2a)
The rst encoder generates the parity sequence
h i
v(1) = v0(1) ; v1(1) ;


; vN(1) 1 : (16.2b)
The interleaver reorders or permutes the N bits in the information block so that the second encoder
receives a permuted information sequence u0 di erent than the rst. (Note that the second encoder
may not be terminated.) The parity sequence generated by the second encoder is represented as
h i
v(2) = v0(2) ; v1(2) ;


; vN(2) 1 ; (16.2c)
and the nal transmitted sequence is given by the vector
h i
v = v0(0) ; v0(1) ; v0(2) ; v1(0) ; v1(1) ; v1(2) ;


; vN(0) 1; vN(1) 1; vN(2) 1 ; (16.3)
so that the overall code rate is R = (N m)=3N  1=3 for large N .
4 CHAPTER 16. TURBO CODING

(0) (0) (0) (0)

u=[u0,u1,.....,uN-1] v =[v0 ,v1 ,.....,vN-1]

(1) (1) (1) (1)

v =[v0 ,v1 ,.....,vN-1]
Encoder1
(2) (2) (2) (2)
v =[v0 ,v1 ,.....,vN-1]
π Encoder2

(a)

u (0)
v

(1)
v

π
(2)
v

(b)

Figure 16.1: The Basic Turbo Encoding Structure

16.1. INTRODUCTION TO TURBO CODING 5
The information sequence u along with the rst parity sequence v(1) is referred to as the rst
constituent code, while the permuted information sequence u0 (which is not transmitted) along with
the second parity sequence v(2) is referred to as the second constituent code. In Figure 16.1(b), both
constituent codes are generated by the same (2,1,4) systematic feedback encoder whose generator matrix
is given by
h i
G (D) = 1 1+D+D1+D+D +D :
2
4
3 4 (16.4)
We now make a number of remarks related to the typical operation of turbo codes. Explanations will
be given in later sections of this chapter.
Remarks:
 In order to achieve performance close to the Shannon limit, the information block length (inter-
leaver size) N is chosen to be very large, usually at least several thousand bits.
 The best performance at moderate BER's down to about 10 5 is achieved with short memory
order constituent encoders, typically m = 4 or less.
 The constituent codes are normally generated by the same encoder, as in Figure 16.1(b), but this
is not necessary for good performance. In fact, some asymmetric code designs have been shown
to give very good performance [1].
 Recursive constituent codes, generated by systematic feedback encoders, give much better perfor-
mance than nonrecursive constituent codes.
 Bits can be punctured from the parity sequences in order to produce higher code rates. For
example, puncturing alternate bits from v(1) and v(2) produces a rate 1=2 code.
 Additional constituent codes and interleavers can be used to produce lower rate codes. For ex-
ample, rate 1=4 can be achieved with three constituent codes and two interleavers, as shown in
Figure 16.2. This is called a multiple turbo code.
 The best interleavers reorder the bits in a pseudorandom manner. Conventional block interleavers
do not perform well in turbo codes, except at relatively short block lengths.
 Since it is only the ordering of the bits that is changed by the interleaver, the sequence u0 that
enters the second encoder has the same weight as the sequence u that enters the rst encoder.
 The interleaver is an integral part of the overall encoder, and thus the state complexity of turbo
codes is extremely large, making trellis-based maximum likelihood decoding impossible.
6 CHAPTER 16. TURBO CODING

 Suboptimum decoding, which employs individual SISO decoders for each of the constituent codes
in an iterative manner, achieves performance typically within a few tenths of a dB of maximum
likelihood decoding. The best performance is obtained when the BCJR, or MAP, algorithm is
used.
 Since the MAP decoder uses a forward-backward algorithm, the information is arranged in blocks.
Thus, the rst constituent encoder is terminated by appending m bits to return it to the 0 state.
Because the interleaver reorders the input sequence, the second encoder will not normally return
to the 0 state, but this has little e ect on performance for large block lengths. If desired, though,
modi cations can be made to insure termination of both encoders.
 Block codes can also be used as constituent codes in turbo encoders.
 Decoding can be stopped, and a nal decoding estimate declared, after some xed number of
iterations (usually on the order of 10-20) or based on a stopping criterion which is designed to
detect when the estimate is reliable with very high probability.

u v(0)

Encoder1 v(1)

π1 Encoder2 v(2)

π2 Encoder3 v(3)

Figure 16.2: A Rate 1=4 Multiple Turbo Code

The basic encoding scheme illustrated in Figures 16.1 and 16.2 is referred to as parallel concate-
nation, because of its similarity to Forney's original concatenation scheme. Compared to conventional
16.1. INTRODUCTION TO TURBO CODING 7
concatenation, in parallel concatenation it is the input to the encoder, rather than its output, that
enters the second encoder, although it is rst permuted by the interleaver. In other words, the two
encoders operate in parallel on di erent versions of the information sequence.
The example code shown in Figure 16.1(b), punctured to rate 1=2, is capable of achieving a 10 5
BER at an SNR of Eb=N0 = 0:7dB with an information block length N = 216 = 65536 bits after 18
iterations of a SISO MAP decoder. By comparison, the NASA standard (2,1,6) convolutional code
with ML decoding presented in Chapter 12 requires an Eb=N0 = 4:2dB to achieve the same BER. The
performance comparison of these two codes is shown in Figure 16.3. Thus the rate 1=2 turbo code
achieves a 3:5dB coding gain compared to the (2,1,6) convolutional code at a 10 5 BER. The decoding
complexity of the two codes is roughly equivalent, since a 16-state MAP decoder has about the same
complexity as a 64-state Viterbi decoder. This advantage of turbo codes over conventional methods of
coding is fairly typical over the entire range of possible code rates, i.e., several dB's of coding gain can
be achieved at moderate BER's with long turbo codes of the same rate and roughly the same decoding
complexity as conventional codes. In addition, the performance of turbo codes at moderate BER's is
within 1dB of capacity. For the example shown in Figure 16.3, the BER performance is only 0:7dB
away from capacity and only 0:5dB away from the capacity for binary input channels.
Turbo codes su er from two disadvantages: a large decoding delay, due to the large block lengths
and many iterations of decoding required for near capacity performance, and signi cantly weakened
performance at BER's below 10 5 (see Figure 16.3), due to the fact that the codes have a relatively
poor minimum distance, which manifests itself at very low BER's. The large delay appears to make turbo
codes unsuitable for real time applications such as voice transmission and packet communication in high
speed networks. It is possible, though, to trade delay for performance in such a way that turbo codes
may be useful in some real time applications involving block lengths on the order of a few thousand
bits, or even a few hundred bits. The fact that turbo codes typically do not have large minimum
distances causes the performance curve to atten out at BER's below 10 5, as shown in Figure 16.3.
This phenomenon, sometimes called an error oor, is due to the unusual weight distribution of turbo
8 CHAPTER 16. TURBO CODING

1e+00
(2,1,6) NASA
Turbo Code

1e-01

1e-02

1e-03
Pb(E)

1e-04
BPSK Capacity

1e-05
Capacity

1e-06

1e-07
-1 0 1 2 3 4 5 6
Eb/N0 (dB)

Figure 16.3: Performance Comparison of Convolutional Codes and Turbo Codes

16.1. INTRODUCTION TO TURBO CODING 9
codes, which will be discussed in the next section. Because of the error oor, turbo codes may not be
suitable for applications requiring extremely low BER's, such as some scienti c or command and control
applications. However, there are measures that can be taken to mitigate this problem. Interleavers can
be designed to improve the minimum distance of the code, thus lowering the error oor. Also, an outer
code, or a second layer of concatenation, can be used with a turbo code to correct many of the errors
caused by the small minimum distance, at a cost of a small decrease in overall rate. Both of these
techniques will be discussed later in this chapter.
The fundamental property of turbo codes that gives them such excellent performance at moderate
BER's is the random-like weight spectrum of the codewords produced by the pseudorandom interleaver
when systematic feedback encoders are used. To see this, we consider a series of examples.
Example 16.1
Consider the conventional (2,1,4) convolutional code with nonsystematic feedforward generator matrix

Gff (D) = 1 + D + D2 + D3 + D4 1 + D4  : (16.5)
The minimum free distance of this code is 6, obtained from the information sequence u = [1 1 0


].
If the encoder is converted to systematic feedback form, the generator matrix is then given by
h i
Gfb (D) = 1 1+D4 (16.6)
1+D+D2+D3+D4 :

Since the code is exactly the same, the free distance is still 6, but in this case the minimum weight
codeword is obtained from the information sequence u = [1 0 0 0 0 1 0 0


], i.e., u(D) = 1+ D5. The
two di erent encoders result in identical codes, but with di erent mappings between information blocks
and codewords. Now consider that each encoder is terminated after an information block of length N 4
by appending four bits to return the encoders to the 0 state. (Note that, for the feedback encoder, the
termination bits depend on the information block and are in general non-zero.) In this case we obtain
a (2N; N 4) block code with rate R = (N 4) =2N  1=2 for large N . Each of these block codes
contains exactly N 5 weight 6 codewords. This is due to the fact that the information sequence that
10 CHAPTER 16. TURBO CODING

produces the weight 6 codeword can begin at any of the rst N 5 information positions and generate
the same codeword. A similar analysis reveals that for weight 7 and other low weights, the number
of codewords is also large, on the order of N or larger. In Table 16.1(a), we give the complete weight
spectrum of the (32,12) code that results from choosing N = 16. Observe that the number of codewords
at each weight grows rapidly until it reaches a peak at length 16, half the block length. In other words,
the weight spectrum of the code is dense at the low end, and this results in a relatively high probability
of error at low SNR's even with ML decoding. 

In general, if an unterminated convolutional code has Ad codewords of weight d caused by a set

of information sequences fu(D)g whose rst one occurs at time 0, then it also has Ad codewords of
weight d caused by the set of information sequences fDu(D)g, and so on. Terminated convolutional
codes have essentially the same property for large block lengths. In other words, convolutional encoders
are time-invariant, and it is this property that accounts for the relatively large numbers of low weight
codewords.
Next, we look at an example where a pseudorandom interleaver is used to produce a parallel con-
catenation of two identical systematic feedback constituent encoders.
Example 16.2
Consider the systematic feedback encoder of (16.6), a length N = 16 input sequence, and the size 16
interleaving pattern given by the permutation:
Y
= [0; 8; 15; 9; 4; 7; 11; 5; 1; 3; 14; 6; 13; 12; 10; 2] : (16.7)
16
The input sequence is rst encoded by the parity generator 1 + D4
= 1 + D + D2 + D3 + D4
, pro-
ducing the parity sequence v(1) (D). Then the interleaver takes the 12 information bits plus the 4
termination bits and reorders them such that

u00 = u0 ; u01 = u8 ; u02 = u15 ;


; u015 = u2 : (16.8)

16.1. INTRODUCTION TO TURBO CODING 11
This permuted input sequence is then re-encoded using the same parity generator 1 + D4
= 1 + D + D2 + D3 + D4
,
thus producing another version of the parity sequence. In order to compare with the code of Exam-
ple 16.1, we now puncture alternate bits from the two versions of the parity sequence using the period
T = 2 puncturing matrix
 
1
P= 0 1 : 0 (16.9)

The result is a parallel concatenated code with the same dimensions as the code in Example 16.1, i.e.,
a (32,12) code. The weight spectrum for this parallel concatenated code is given in Table 16.1(b). We
see that there is a noticeable di erence between this weight spectrum and the one for the terminated
convolutional code shown in Table 16.1(a), even though the code generators are the same. This altered
weight spectrum is a direct result of the interleaver which permutes the bits for re-encoding. Note
that the free distance has decreased, from 6 to 5, but that there is only one weight 5 codeword. More
importantly, the multiplicities of the weight 6 through 9 codewords are less for the parallel concate-
nated code than for the terminated convolutional code. In other words, there has been a shift from
lower weight codewords to higher weight codewords relative to the convolutional code in the paral-
lel concatenated case. This shifting of low weight codewords towards higher weights in the parallel
concatenation of feedback convolutional encoders has been termed spectral thinning. The e ect is due
to the fact that interleaving causes almost all the low weight parity sequences in the rst constituent
code to be matched with high weight parity sequences in the second constituent code. For example,
consider the weight 2 input sequence u = [1000010


0] that causes the low weight parity sequence
v(1) = [11001100


0]. Thus without the interleaver, the convolutional code produces a codeword of
weight 6. The interleaved input sequence is given by u0 = [100000010


0] and produces the parity
sequence v(2) = [1100101111000110]. Combining v(1) and v(2) and then puncturing alternate bits using
the period T = 2 puncturing matrix given in (16.9) produces the parity sequence [1100100101000100].
Thus the same weight 2 input sequence produces a codeword of weight 8 in the parallel concatenated
code. This is typical of parallel concatenation, i.e., when feedback constituent encoders are used, most
12 CHAPTER 16. TURBO CODING

low weight codewords are shifted to higher weights. In the next example, we see that this spectral
thinning becomes more dramatic for larger block lengths. 

Example 16.3
Consider the same systematic feedback convolutional encoder as in Examples 16.1 and 16.2, but with a
block length of N = 32, including the m = 4 termination bits. The weight spectrum of the terminated
(64,28) convolutional code is shown in Table 16.2(a). Now consider the same encoder in a parallel
concatenation scheme, using the size 32 interleaving pattern given by
Y
= [0; 16; 7; 17; 12; 28; 19; 2; 8; 11; 22; 9; 4; 20; 18; 26;
32
1; 3; 14; 6; 13; 31; 10; 29; 25; 24; 15; 30; 5; 23; 27; 21] (16.10)
and the same period T = 2 puncturing matrix as in Example 16.2. This results in the (64,28) parallel
concatenated code whose weight spectrum is given in Table 16.2(b). We note that there is now a more
pronounced di erence between this weight spectrum and the one for the terminated convolutional code
given in Table 16.2(a) than in the N = 16 case. The free distance of both codes equals 6, i.e., there is
no change in dfree, but the multiplicities of the low weight codewords are greatly reduced in the parallel
concatenated code. This can be seen more clearly from the graphs of the two weight spectra shown in
Figure 16.4. In other words, the e ect of spectral thinning becomes more dramatic as the block length
(interleaver size) increases. In fact, for even larger values of N , the codeword and bit multiplicities of
the low weight codewords in the turbo code weight spectrum are reduced by roughly a factor of N , the
interleaver size, compared to the terminated convolutional code. This reduction by a factor of N in
the low weight multiplicities is referred to as the interleaver gain and will be veri ed analytically in the
next section. 

Remarks:
 Di erent interleavers and puncturing matrices would produce di erent results in the above exam-
16.1. INTRODUCTION TO TURBO CODING 13

Table 16.1: Weight Spectra of Two (32,12) Codes

(a) Terminated Convolutional (b) Parallel Concatenated
Weight Multiplicity Weight Multiplicity
0 1 0 1
1 0 1 0
2 0 2 0
3 0 3 0
4 0 4 0
5 0 5 1
6 11 6 4
7 12 7 8
8 23 8 16
9 38 9 30
10 61 10 73
11 126 11 144
12 200 12 210
13 332 13 308
14 425 14 404
15 502 15 496
16 545 16 571
17 520 17 558
18 491 18 478
19 346 19 352
20 212 20 222
21 132 21 123
22 68 22 64
23 38 23 24
24 11 24 4
25 2 25 4
26 0 26 1
27 0 27 0
28 0 28 0
29 0 29 0
30 0 30 0
31 0 31 0
32 0 32 0
14 CHAPTER 16. TURBO CODING

Table 16.2: Weight Spectra of Two (64,28) Codes

(a) Terminated Convolutional (b) Parallel Concatenated
Weight Multiplicity Weight Multiplicity Weight Multiplicity Weight Multiplicity
0 1 33 25431436 0 1 33 25716960
1 0 34 23509909 1 0 34 23669905
2 0 35 20436392 2 0 35 20464409
3 0 36 16674749 3 0 36 16623260
4 0 37 12757248 4 0 37 12662360
5 0 38 9168248 5 0 38 9024333
6 27 39 6179244 6 6 39 6012086
7 28 40 3888210 7 9 40 3729485
8 71 41 2271250 8 15 41 2156481
9 118 42 1226350 9 9 42 1160459
10 253 43 615942 10 80 43 573214
11 558 44 287487 11 119 44 262676
12 1372 45 124728 12 484 45 110369
13 3028 46 50466 13 1027 46 42264
14 6573 47 19092 14 3007 47 15269
15 14036 48 6888 15 6852 48 4556
16 29171 49 2172 16 17408 49 1416
17 60664 50 642 17 40616 50 335
18 122093 51 140 18 90244 51 103
19 240636 52 35 19 193196 52 20
20 457660 53 6 20 390392 53 5
21 838810 54 2 21 754819 54 1
22 1476615 55 0 22 1368864 55 0
23 2484952 56 0 23 2367949 56 0
24 3991923 57 0 24 3874836 57 0
25 6098296 58 0 25 5988326 58 0
26 8845265 59 0 26 8778945 59 0
27 12167068 60 0 27 12149055 60 0
28 15844169 61 0 28 15907872 61 0
29 19504724 62 0 29 19684668 62 0
30 22702421 63 0 30 22978613 63 0
31 24967160 64 0 31 25318411 64 0
32 25927128 32 26289667
16.1. INTRODUCTION TO TURBO CODING 15
1e+08
Terminated Convolutional
Parallel Concatenated

1e+07

1e+06

1e+05
Multiplicity

1e+04

1e+03

1e+02

1e+01

1e+00
0 10 20 30 40 50 60 70
Codeword Weight

Figure 16.4: An Illustration of Spectral Thinning

ples, but the behavior observed is typical in most cases.
 Spectral thinning has little e ect on the minimum free distance, but it greatly reduces the multi-
plicities of the low weight codewords.
 As the block length and corresponding interleaver size increase, the weight spectrum of parallel
concatenated convolutional codes begins to approximate a random-like distribution, i.e., the dis-
tribution that would result if each bit in every codeword were selected randomly from an i.i.d.
probability distribution.
 As will be seen in the next section, there is only a small spectral thinning e ect if feedforward
constituent encoders are used.
 One can explain the superiority of feedback encoders in parallel concatenation as a consequence
of the fact that they are IIR, rather than FIR, lters, i.e., their response to single input "1's" is
not localized to the constraint length of the code but extends over the entire block length. This
property of feedback encoders is exploited by a pseudorandom interleaver to produce the spectral
thinning e ect.
It is also worth noting that parallel concatenated codes are no longer time-invariant. This can
easily be seen by considering the e ect when the input sequence in Example 16.2 is delayed by one
16 CHAPTER 16. TURBO CODING

time unit, i.e., consider the input sequence u = [0100001000000000]. The rst parity sequence v(1) =
[0110011000000000] is also delayed by one time unit, but the interleaved input sequence is now u0 =
[0000000010010000], which produces the second parity sequence v(2) = [0000000011010011]. This is
clearly not a delayed version of the v(2) in Example 16.2. In other words, the interleaver has broken the
time-invariant property of the code, resulting in a time-varying code. To summarize, in order to achieve
the spectral thinning e ect of parallel concatenation, it is necessary both to generate a time-varying
code (via interleaving) and to employ feedback, i.e., IIR, encoders.
It is clear from the above examples that the interleaver plays a key role in turbo coding. As we
shall now brie y discuss, it is important that the interleaver has pseudorandom properties. Traditional
block or convolutional interleavers do not work well in turbo coding, particularly when the block length
is large. What is important is that the low weight parity sequences from the rst encoder get matched
with high weight parity sequences from the second encoder almost all the time. This requires that
the interleaver break the patterns in the input sequences which produce low weight parity sequences.
Interleavers with structure, such as block or convolutional interleavers, tend to preserve too many
of these \bad" input patterns, resulting in poor matching properties and limited spectral thinning.
Pseudorandom interleavers, on the other hand, break up almost all the \bad" patterns and thus achieve
the full e ect of spectral thinning. In Example 16.2, the 11 input sequences

u (D) = Di 1 + D5
; i = 0; 1;


; 10 (16.11)

are \bad", because they generate a low weight (4 in this case) parity sequence. As can be seen from
a careful examination of Figure 16.5, if a 4  4 block (row-column) interleaver is employed, 9 of these
11 sequences will maintain the same \bad" pattern after interleaving, resulting in a large multiplicity
of low weight codewords. The weight spectrum of the code in Example 16.2 with this block interleaver,
shown in Table 16.3, is clearly inferior to the weight spectrum shown in Table 16.1(b) obtained using
the interleaver of (16.7). Pseudorandom interleavers, such as those in (16.7) and (16.10), generate
a weight spectrum that has many of the same characteristics as the binomial distribution, which is
16.1. INTRODUCTION TO TURBO CODING 17
equivalent to the weight spectrum assumed by Shannon in his random coding proof of the noisy channel
coding theorem. In other words, codes with random (binomial) weight distributions can achieve the
performance guaranteed by Shannon's bound. Turbo coding with pseudorandom interleaving gives
us a way of constructing codes with weight spectra similar to a binomial distribution and a simple,
near-optimal iterative decoding method exists.

Write
Read

Figure 16.5: A (4  4) Block Interleaver for a (32,12) Turbo Code

Pseudorandom interleaving patterns can be generated in many ways, e.g., by using a primitive
polynomial to generate a maximum length shift register sequence whose cycle structure determines
the permutation. Another method uses a computationally simple algorithm based on the quadratic
18 CHAPTER 16. TURBO CODING

congruence
km(m + 1)
cm 
2 (mod N ); 0  m < N; (16.12)
to generate an index mapping function cm ! cm+1 (mod N ), where N is the interleaver size and k is an
odd integer. For example, for N = 16 and k = 1, we obtain
(c0; c1;


; c15) = (0; 1; 3; 6; 10; 15; 5; 12; 4; 13; 7; 2; 14; 11; 9; 8) : (16.13)
This implies that index 0 (input bit u00) in the interleaved sequence u0 is mapped into index 1 in the
original sequence u (i.e., u00 = u1), index 1 in u0 is mapped into index 3 in u (u01 = u3), and so on,
resulting in the permutation
Y
= [1; 3; 14; 6; 13; 12; 10; 2; 0; 8; 15; 9; 4; 7; 11; 5] : (16.14)
16
If this interleaving pattern is shifted cyclically to the right r = 8 times, we obtain the interleaving
pattern of (16.7). For N a power of two, it can be shown that these quadratic interleavers have statistical
properties similar to randomly chosen interleavers, and thus they give good performance when used in
turbo coding [2]. Other good interleaving patterns can be generated by varying k and r, and the special
case r = N=2 (used above to obtain (16.7)) results in an interleaver which simply interchanges pairs of
indices (see Problem 16.2). This special case is particularly interesting for implementation purposes,
since the interleaving and deinterleaving functions (both used in decoding) are identical. Finally, when
N is not a power of 2, the above algorithm can be modi ed to generate similar permutations with good
statistical properties [2].
The basic structure of an iterative turbo decoder is shown in Figure 16.6. (We assume here a
rate 1=3 parallel concatenated code without puncturing.) It employs two SISO decoders using the MAP
algorithm presented earlier in Chapters 12 and 14. At each time unit k, three output values are received
from the channel, one for the information bit uk = vk(0), denoted rk(0), and two for the parity bits vk(1)
and vk(2), denoted rk(1) and rk(2), and the 3N -dimensional received vector is denoted by
h i
r = r0(0) ; r0(1) ; r0(2) ; r1(0) ; r1(1) ; r1(2) ;


; rN(0) 1 ; rN(1) 1 ; rN(2) 1 : (16.15)
16.1. INTRODUCTION TO TURBO CODING 19

Table 16.3: The Weight Spectrum of a Block Interleaved (32,12) Turbo Code
Weight Multiplicity
0 1
1 0
2 0
3 0
4 0
5 0
6 21
7 6
8 13
9 40
10 67
11 154
12 190
13 308
14 411
15 486
16 555
17 532
18 493
19 350
20 230
21 140
22 64
23 28
24 3
25 4
26 0
27 0
28 0
29 0
30 0
31 0
32 0
20 CHAPTER 16. TURBO CODING

Now let each transmitted bit be represented using the binary values 0 ! +1 and 1 ! 1, so that
mod-2 addition is equivalent to real multiplication. Then for an AWGN channel with unquantized (soft)
   
outputs, we de ne the log-likelihood ratio (L-value) L vk(0) j rk(0) = L uk j rk(0) (before decoding) of
a transmitted information bit uk given the received value rk(0) as

P u = +1 j r (0) 
L(uk j rk(0) ) = ln 
k k

P uk = 1 j rk(0)
 
P rk(0) j uk = +1  P (uk = +1)
= ln  (0) 
P rk j uk = 1  P (uk = 1)
 
P rk(0) j uk = +1 P (uk = +1)
= ln  (0)  + ln
P (uk = 1)
P rk j uk = 1
 

e (Es =N0 ) rk 1
2

P (uk = +1)
(0)

= ln  2 + ln

P (uk = 1)
; (16.16)
e ( Es =N0 ) rk +1
(0)

where Es=N0 is the channel SNR and uk and rk(0) have both been normalized by a factor of pEs. This
simpli es to
 
  Es  (0) 2  (0) 2 P (uk = +1)
L uk j rk(0) = rk 1 rk + 1 + ln
N0 P (uk = 1)
= 4 NEs0 rk(0) + ln PP ((uuk == +1)
1)
k
= Lc rk(0) + La (uk ) ; (16.17)
where Lc =4 4(Es=N0) is called the channel reliability factor and La (uk ) is the a priori L-value of the
bit uk . In the case of a transmitted parity bit vk(i), given the received value rk(i), i = 1; 2; the L-value
(before decoding) is given by
   
L vk(i) j rk(i) = Lc rk(i) + La vk(i) = Lc rk(i) ; i = 1; 2; (16.18)
since in a linear code with equally likely information bits, the parity bits are also equally likely to be 0
or 1, and thus the a priori L-values of the parity bits are 0, i.e.,

  P v (i) = +1
La vk(i) = ln  (i)  = 0; i = 1; 2: (16.19)
k
P vk = 1
16.1. INTRODUCTION TO TURBO CODING 21
(We note here that La (uk ) also equals 0 for the rst iteration of Decoder 1, but that thereafter the a
priori L-values of the information bits are replaced by extrinsic L-values from the other decoder, as
explained below.)
The received soft channel L-values Lcrk(0) for uk and Lcrk(1) for vk(1) enter Decoder 1, while the
(properly interleaved) received soft channel L-values Lcrk(0) for uk and the received soft channel L-
values Lcrk(2) for vk(2) enter Decoder 2. The output of Decoder 1 contains two terms: (1) L(1) (uk ) =
h    i
ln P uk = +1 j r1; L(1) a =P uk = 1 j r1 ; L(1)a , the a posteriori L-value (after decoding) of each in-
h i
formation bit produced by Decoder 1 given the (partial) received vector r1 =
r0(0); r0(1); r1(0); r1(1);


; rN(0) 1; rN(1) 1
and the a priori input vector L(1)
h (1) (1) (1) i
(1)
a = La (u0 ); La (u1 );


; La (uN 1 ) for Decoder 1, and (2) Le (uk ) =
h i
L(1) (uk ) Lc rk(0) + L(2)e (uk ) , the extrinsic a posteriori L-value (after decoding) associated with each

information bit produced by Decoder 1, which, after interleaving, is passed to the input of Decoder 2 as
the a priori value L(2) (0) (2)
a (uk ). Subtracting the term in brackets, viz., Lc rk + Le (uk ), removes the e ect

of the current information bit uk from L(1) (uk ), leaving only the e ect of the parity constraints, thus
providing an independent estimate of the information bit uk to Decoder 2 in addition to the received
soft channel L-values at time k. Similarly, the output of Decoder 2 contains two terms, L(2) (uk ) =
h    i
ln P uk = +1 j r2; L(2) a =P uk = 1 j r2 ; L(2)a , where r2 is the (partial) received vector and L(2) a
h i
the a priori input vector for Decoder 2, and L(2) (2)
e (uk ) = L (uk ) Lc rk(0) + L(1)
e (uk ) , and the ex-

trinsic a posteriori L-values L(2)e (uk ) produced by Decoder 2, after deinterleaving, are passed back to

the input of Decoder 1 as the a priori values L(1) a (uk ). Thus, the input to each decoder contains three

terms, the soft channel L-values Lcrk(0) and Lcrk(1) (or Lcrk(2)) and the extrinsic a posteriori L-values
L(2) (1) (1) (2)
e (uk )= La (uk ) (or Le (uk ) = La (uk )) passed from the other decoder. (In the initial iteration of

Decoder 1, the extrinsic a posteriori L-values L(2) (1)

e (uk )= La (uk ) are just the original a priori L-values

La (uk ), which, as noted above, are all equal to 0 for equally likely information bits. Thus the extrinsic
L-values passed from one decoder to the other during the iterative decoding process are treated like
new sets of a priori probabilities by the MAP algorithm.) Decoding then proceeds iteratively, with each
decoder passing its respective extrinsic L-values back to the other decoder. This results in a turbo or
22 CHAPTER 16. TURBO CODING

bootstrapping e ect in which each estimate becomes successively more reliable. After a suÆcient number
of iterations, the decoded output is determined from the a posteriori estimate L(2) (uk ) at the output of
Decoder 2. Since the decoded output is taken only after the nal iteration, it is more accurate to refer to
the SISO constituent decoders as a posteriori probability (APP) estimators rather than MAP decoders,
since their outputs are extrinsic a posteriori L-values that are passed to their companion decoder for
more processing. A more complete discussion of iterative turbo decoding is given in Section 16.5.

While it is true, as stated above, that the extrinsic a posteriori L-values Le (uk ) passed between
decoders during the rst iteration of decoding are independent of uk , this is not so for subsequent
iterations. Thus the extrinsic information becomes less helpful in obtaining successively more reliable
estimates of the information bits as the iterations continue. Eventually, a point is reached where no
further improvement is possible, the iterations are stopped, and the nal decoding estimate is pro-
duced. Methods for determining when to stop the iterations, known as stopping rules, are discussed in
Section 16.5.

It is worth pointing out here that the name turbo in turbo coding has more to do with decoding
than encoding. Indeed, it is the successive feedback of extrinsic information from the SISO decoders in
the iterative decoding process that mimics the feedback of exhaust gases in a turbo engine.

Finally, before moving on to a more detailed discussion of turbo coding, we remark that many of
its features are similar to those observed for low-density parity-check codes (LDPCC's), to be discussed
in Chapter 17. Both encoding schemes produce codes with random-like weight distributions and a thin
weight spectrum. Both decoding methods make use of APP likelihoods in an iterative process, and they
both employ the concept of extrinsic information. In fact, it was the discovery of turbo coding in 1993
that led to a re-discovery of the merits of LDPCC's, which had been largely neglected by the research
community for more than 30 years.
 16.1.
INTRODUCTION TO TURBO CODING
Figure 16.6: Basic Structure of an Iterative Turbo Decoder

(0) (1) (0) (2)

Lcrk Lcrk Lcrk Lcrk

Interleaver
(2)
L (uk)
(2) (1) (0) (1) (1) Deinterleaver
Le (uk) Decoder 1 L (uk)-Lcrk Le (uk) Le (uk) Decoder 2 Decision
Interleaver
-

-
Deinterleaver
(2) (2) (0)
Le (uk) L (uk)-Lcrk

23
24 CHAPTER 16. TURBO CODING

16.2 Distance Properties of Turbo Codes

As illustrated in Examples 16.1 to 16.3, the fundamental property of turbo codes that allows them to
achieve such excellent performance is the random-like weight spectrum, or spectral thinning, produced
by the pseudorandom interleaver. In this section, we will examine the weight spectrum of turbo codes
in more detail. In particular, we consider a series of examples for Parallel Concatenated Codes (PCC's),
including both Parallel Concatenated Block Codes (PCBC's) and Parallel Concatenated Convolutional
Codes (PCCC's).
As noted in the remark following Example 16.3, the exact weight spectrum of a turbo code depends on
the particular interleaver chosen. Thus, in order to avoid exhaustively searching all possible interleavers
for the best weight spectrum for a speci c PCC, we introduce the concept of a uniform interleaver.

De nition 16.1 [3] A uniform interleaverof length


N is a probabilistic device which maps a given
 
N
input block of weight w into all its distinct w permutations with equal probability 1= w .  N

Using the notion of a uniform interleaver allows us to calculate the average weight spectrum of
a speci c PCC over all possible interleavers. This average weight spectrum is typical of the weight
spectrum obtained for a randomly chosen interleaver.
Example 16.4
Consider the (7,4,3) Hamming code in systematic form. The Weight Enumerating Function (WEF) for
this code is given by
A(X ) = 7X 3 + 7X 4 + X 7 ; (16.20)

i.e., in addition to the all-zero codeword, the code contains seven codewords of weight 3, seven codewords
of weight 4, and the all-one codeword. The complete list of 16 codewords is shown in Table 16.4. Splitting
the contributions of the information and parity bits gives the Input Redundancy Weight Enumerating
16.2. DISTANCE PROPERTIES OF TURBO CODES 25
Table 16.4: Codeword List for the (7,4,3) Hamming Code
Information Parity
0 0 0 0 0 0 0
1 0 0 0 1 0 1
0 1 0 0 1 1 0
1 1 0 0 0 1 1
0 0 1 0 0 1 1
1 0 1 0 1 1 0
0 1 1 0 1 0 1
1 1 1 0 0 0 0
0 0 0 1 1 1 1
1 0 0 1 0 1 0
0 1 0 1 0 0 1
1 1 0 1 1 0 0
0 0 1 1 1 0 0
1 0 1 1 0 0 1
0 1 1 1 0 1 0
1 1 1 1 1 1 1

Function (IRWEF)
A(W; Z ) = W (3Z 2 + Z 3 ) + W 2 (3Z + 3Z 2 ) + W 3 (1 + 3Z ) + W 4 Z 3 : (16.21)
In other words, there are three codewords with information weight 1 and parity weight 2, one codeword
with information weight 1 and parity weight 3, three codewords with information weight 2 and parity
weight 1, and so on. Finally, the Conditional WEF (CWEF) for each input weight is given by
A1 (Z ) = 3Z 2 + Z 3 ;
A2 (Z ) = 3Z + 3Z 2 ;
A3 (Z ) = 1 + 3Z;
A4 (Z ) = Z 3 : (16.22)


Now we examine how the uniform interleaver concept can be used to compute the average IRWEF
26 CHAPTER 16. TURBO CODING

Transmitted Bits

k n-k n-k

Systematic
Interleaver code C1(n,k)
length k
Systematic
code C2(n,k)

Figure 16.7: A Parallel Concatenated Block Code

for PCC's. First consider the general case shown in Figure 16.7 of a PCBC with two di erent (n; k)
systematic block constituent codes, C1 and C2, with CWEF's given by ACw (Z ) and ACw (Z ), w = 1;


; k,
1 2

connected by a uniform interleaver of size N = k. The original information block and both parity blocks
are transmitted, resulting in a (2n k; k) PCBC. Assume that a particular input block of weight w
enters the rst encoder, thereby generating one of the parity weights in ACw (Z ). From the de nition of
1

the uniform interleaver, it follows that there is an equal probability that the second encoder will match
that parity weight with any of the parity weights in ACw (Z ). Thus the average CWEF of the PCBC is
2

given by
AC (Z )  AC (Z )
APw C (Z ) = w  w (16.23)
1 2
;
N
w
and the average IRWEF is given by
X X
AP C (W; Z ) = Aw;z W w Z z = W w APw C (Z ): (16.24)
(w;z) (1wN )
Also, we can express the bit CWEF and the bit IRWEF as follows:
w PC
BwP C (Z ) = A (Z ) (16.25a)
N w
16.2. DISTANCE PROPERTIES OF TURBO CODES 27
and
X X
B P C (W; Z ) = Bw;z W w Z z = W w BwP C (Z ); (16.25b)
(w;z) (1wN )
where Bw;z = (w=N )Aw;z . Finally, the average codeword and bit WEF's are given by
X
AP C (X ) = Ad X d = AP C (W; Z )
W =Z =X
(dmin d)
X
= W w APw C (Z ) W =Z =X (16.26a)
1wN
and
X
B P C (X ) = Bd X d = B P C (W; Z )
W =Z =X
dmin d
X
= W w BwP C (Z ) W =Z =X : (16.26b)
1wN
Remarks:
 The codeword CWEF APwC (Z ), IRWEF AP C (W; Z ), and WEF AP C (X ) of the PCBC are all
average quantities over the entire class of uniform interleavers, and thus their coeÆcients may
not be integers, unlike the case when these quantities are computed for the constituent codes
individually.
 (16.26) represents two di erent ways of expressing the codeword and bit WEF's of a PCBC, one
as a sum over the codeword weights d that is valid for both systematic and nonsystematic codes
and the other as a sum over input weights w that applies only to systematic codes.
 The codeword and bit WEF expressions in (16.26a) and (16.26b) are general and apply to all
systematic codes. Expressions similar to the sums over input weights w that are valid for nonsys-
tematic codes are given by
X
A(X ) = A(W; X )jW =1 = W w Aw (X )jW =1 (16.26c)
(1wN )
and X
B (X ) = B (W; X )jW =1 = W w Bw (X )jW =1 ; (16.26d)
(1wN )
where A(W; X ); Aw (X ); B(W; X ), and Bw (X ) are the Input Output Weight Enumerating Func-
tions (IOWEF's) of the code.
 A more general class of PCBC's results if the two constituent codes, C1 and C2, have di erent
block lengths, n1 and n2.
28 CHAPTER 16. TURBO CODING

Example 16.4 (Continued)

For the (10,4) PCBC with two identical (7,4,3) Hamming codes as constituent codes, the CWEF's are
AP1 C (Z ) =
(3Z 2 + Z 3)2 = 2:25Z 4 + 1:5Z 5 + 0:25Z 6;
4
AP2 C (Z ) =
(3 Z + 3Z 2)2 = 1:5Z 2 + 3Z 3 + 1:5Z 4;
6 2
AP3 C (Z ) =
(1 + 3 Z)
= 0:25 + 1:5Z + 2:25Z 2;
4
AP4 C (Z ) =
(Z 3)2 = Z 6; (16.27)
1
the IRWEF's are


AP C (W; Z ) = W 2:25Z 4 + 1:5Z 5 + 0:25Z 6 + W 2 1:5Z 2 + 3Z 3 + 1:5Z 4
+


W 3 0:25 + 1:5Z + 2:25Z 2 + W 4Z 6 (16.28a)
and


B P C (W; Z ) = W 0:56Z 4 + 0:37Z 5 + 0:06Z 6 + W 2 0:75Z 2 + 1:5Z 3 + 0:75Z 4
+


W 3 0:19 + 1:12Z + 1:69Z 2 + W 4Z 6; (16.28b)
and the WEF's are
AP C (X ) = 0:25X 3 + 3X 4 + 7:5X 5 + 3X 6 + 0:25X 7 + X 10 (16.28c)
and
B P C (X ) = 0:19X 3 + 1:87X 4 + 3:75X 5 + 1:12X 6 + 0:06X 7 + X 10 : (16.28d)


Remarks:
 The coeÆcients of the codeword WEF's for the PCBC are fractional, due to the e ect of averaging.
 The sum of all the coeÆcients in AP C (X ) equals 15, the total number of nonzero codewords in
the PCBC.
16.2. DISTANCE PROPERTIES OF TURBO CODES 29
 The minimum distance of the (10,4) PCBC is almost certainly either 3 or 4, depending on the
particular interleaver chosen.
 In this example, a detailed analysis of the 4! = 24 possible interleaving patterns reveals that
exactly 6 result in a minimum distance of 3, while the other 18 result in a minimum distance of
4 [3].
 The average codeword multiplicities of the low weight codewords are given by A3 = 0:25 and
A4 = 3:0.
 The average bit multiplicities of the low weight codewords are given by B3 = 0:19 and B4 = 1:87.
Now we examine the more general case, illustrated in Figure 16.8, of forming the constituent codes
in a PCBC by concatenating h codewords of a basic (n; k) systematic code C to form an (hn; hk) h-
repeated code C h and using an interleaver size of N = hk. Again, the original information block and
both parity blocks are transmitted, resulting in a (2hn hk; hk) PCBC. In this case, the IRWEF of the
code C h is given by
 
AC (W; Z ) = 1 + AC (W; Z ) h 1; (16.29)
h

where we have included the \1's" in (16.29) to account for the fact that the all-zero codeword and a
non-zero codeword in the basic code C can be combined to form a non-zero codeword in the h-repeated
code C h. If ACwh (Z ) and ACwh (Z ) represent the CWEF's of two h-repeated constituent codes C1h and
1 2

C2h , then the average CWEF of the PCBC is given by

(Z ) = Aw (Z )N Aw (Z ) ;

C1h C2h
APw C (16.30)
w
the average IRWEF's are given by
X
AP C (W; Z ) = W w APw C (Z ) (16.31a)
(1whk)
and
X w w PC
B P C (W; Z ) = W Aw (Z ); (16.31b)
(1whk) hk
and the average WEF's can be computed using (16.26).
30 CHAPTER 16. TURBO CODING

Transmitted Bits

..... .....
n-k n-k

.....
k ..... k
1 h 1 h

Interleaver
length N=hk

Figure 16.8: A PCBC with Repeated Block Constituent Codes

Example 16.5
Consider the (20,8) PCBC formed by using h = 2 codewords from the (7,4,3) Hamming code, i.e., a
(14,8) doubly-repeated Hamming code, as both constituent codes, along with a uniform interleaver of
size N = hk = 8. The IRWEF of the (14,8) constituent code is
A(W; Z ) = 1 + W (3Z 2 + Z 3) + W 2(3Z + 3Z 2) + W 3(1 + 3Z ) + W 4Z 32 1
= W (6Z 2 + 2Z 3) + W 2(6Z + 6Z 2 + 9Z 4 + 6Z 5 + Z 6) +
W 3 (2 + 6Z + 18Z 3 + 24Z 4 + 6Z 5 ) + W 4 (15Z 2 + 40Z 3 + 15Z 4 ) +
W 5 (6Z + 24Z 2 + 18Z 3 + 6Z 5 + 2Z 6 ) + W 6 (1 + 6Z + 9Z 2 + 6Z 4 + 6Z 5 ) +
W 7 (2Z 3 + 6Z 4 ) + W 8 Z 6 ; (16.32)
and its CWEF's for each input weight are given by
A1 (Z ) = 6Z 2 + 2Z 3 ; A2 (Z ) = 6Z + 6Z 2 + 9Z 4 + 6Z 5 + Z 6 ;
A3 (Z ) = 2 + 6Z + 18Z 3 + 24Z 4 + 6Z 5 ; A4 (Z ) = 15Z 2 + 40Z 3 + 15Z 4 ;
(16.33)
A5 (Z ) = 6Z + 24Z 2 + 18Z 3 + 6Z 5 + 2Z 6 ; A6 (Z ) = 1 + 6Z + 9Z 2 + 6Z 4 + 6Z 5 ;
A7 (Z ) = 2Z 3 + 6Z 4 ; A8 (Z ) = Z 6 :

16.2. DISTANCE PROPERTIES OF TURBO CODES 31

Note that this code still has minimum distance 3, i.e., it is a (14,8,3) code, since each of the seven
weight 3 codewords in one code can be paired with the all-zero codeword in the other code, resulting
in a total of fourteen weight 3 codewords in the doubly-repeated code. Thus, by itself, this (14,8,3)
code would not be interesting, since it is longer (more complex) than the (7,4,3) code but does not
have better distance properties. However, when it is used as a constituent code for a (20,8) PCBC,
the resulting code has better distance properties than the (10,4) PCBC formed from a single Hamming
code.
Example 16.5 (Continued)
Using (16.30), we can compute the CWEF's of the (20,8) PCBC with two identical (14,8,3) doubly-
repeated Hamming codes as constituent codes as follows:

6Z 2 + 2Z 3
2 = 4:5Z 4 + 3Z 5 + 0:5Z 6;

1 (Z )
AP C = 8
AP2 C (Z ) = 6Z + 6Z 2 + 9Z 4 + 6Z 5 + Z 6
2
28
= 1:29Z 2 + 2:57Z 3 + 1:29Z 4 + 3:86Z 5 + 6:43Z 6 + 3Z 7 +
3:32Z 8 + 3:86Z 9 + 1:93Z 10 + 0:43Z 11 + 0:04Z 12;
AP3 C (Z ) = 2 + 6Z + 18Z 3 + 24Z 4 + 6Z 5
2
56
= 0:07 + 0:43Z + 0:64Z 2 + 1:29Z 3 + 5:57Z 4 + 5:57Z 5 +
7:07Z 6 + 15:43Z 7 + 14:14Z 8 + 5:14Z 9 + 0:64Z 10;
AP4 C (Z ) = 15Z 2 + 40Z 3 + 15Z 4
2
70
= 3:21Z 4 + 17:14Z 5 + 29:29Z 6 + 17:14Z 7 + 3:21Z 8;
AP5 C (Z ) =
6Z + 24Z 2 + 18Z 3 + 6Z 5 + 2Z 6
2
56
= 0:64Z 2 + 5:14Z 3 + 14:14Z 4 + 15:43Z 5 + 7:07Z 6 + 5:57Z 7 +
5:57Z 8 + 1:29Z 9 + 0:64Z 10 + 0:43Z 11 + 0:07Z 12;
32 CHAPTER 16. TURBO CODING

Z + 9Z 2 + 6Z 4 + 6Z 5 2


1 + 6
6 (Z ) =
AP C
28
= 0:04 + 0:43Z + 1:93Z 2 + 3:86Z 3 + 3:32Z 4 + 3Z 5 +
6:43Z 6 + 3:86Z 7 + 1:29Z 8 + 2:57Z 9 + 1:29Z 10;
Z 3 + 6Z 4 2


A7 (Z ) =
P C 2 = 0:5Z 6 + 3Z 7 + 4:5Z 8;
8
Z6 2


12
8 (Z )
AP C = 1 =Z : (16.34)
Then the IRWEF's AP C (W; Z ) and BP C (W; Z ) and the WEF's AP C (X ) and BP C (X ) can be computed
using (16.31) and (16.26) (see Problem 16.4). 

Remarks:
 The minimum distance of the (20,8) PCBC is almost certainly either 3 or 4, depending on the
particular interleaver chosen, the same as for the (10,4) PCBC.
 However, the average codeword multiplicities of the low weight codewords have decreased from
A3 = 0:25 to A3 = 0:07 and from A4 = 3:0 to A4 = 1:72, respectively, despite the fact that the
(20,8) PCBC contains 16 times as many codewords as the (10,4) PCBC.
 Also, the average bit multiplicities of the low weight codewords have decreased from B3 = 0:19 to
B3 = 0:03 and from B4 = 1:87 to B4 = 0:48, respectively.
 This is an example of spectral thinning, i.e., the multiplicities of the low weight codewords in a
PCBC are decreased by increasing the length of the constituent code and the interleaver size.
 Increasing the code length and interleaver size by further increasing the repeat factor h leads to
additional spectral thinning, which results in improved performance at low SNR's, but for block
constituent codes the bene cial e ect of increasing h diminishes for large h.
 A better approach would be to increase the interleaver size by using longer block constituent
codes, but eÆcient SISO decoding is more diÆcult for large block sizes.
Now we consider the case of PCCC's, where the constituent codes are generated by convolutional
encoders, as illustrated in Figure 16.1. An exact analysis, similar to the above examples for PCBC's, is
possible but is complicated by issues involving termination of the constituent encoders. Hence we make
the simplifying assumption that both constituent encoders are terminated to the all-zero state. (As
16.2. DISTANCE PROPERTIES OF TURBO CODES 33
noted previously, this is normally the case for the rst encoder, but not for the second encoder, because
the required termination bits are generated by the interleaver only with probability 1=2m, where m is
the encoder memory order.) The resulting analysis is nearly exact whenever the interleaver size N is at
least an order of magnitude larger than the memory order m of the constituent encoders. Since turbo
codes are most e ective for short memory orders and large interleaver sizes, this is an assumption which
always holds in practice.
We begin by illustrating the procedure for computing the CWEF's Aw (Z ) of the equivalent block
code produced by a convolutional encoder which starts in the all-zero state S0 = 0 and returns to
the all-zero state after an input sequence of length N , including termination bits. For an (n; 1; m)
convolutional encoder, the equivalent block code has dimensions (nN; N m). The situation here is
somewhat di erent from that presented in Chapter 11, where we were interested in computing the WEF
of all codewords which diverged from the all-zero state at a particular time and remerged only once.
This WEF was the appropriate one to consider for evaluating the event and bit error probabilities per
unit time of an encoder driven by semi-in nite (unterminated) input sequences. In order to evaluate
the block and bit error probabilities of PCCC's, however, the WEF must include the e ect of multiple
error events, i.e., error events which diverge from and remerge with the all-zero state more than once.
This is because the encoders are driven by nite length (terminated) input sequences, resulting in an
equivalent block code, and the error probability analysis must consider the entire block, rather than
just a particular time unit. Thus we will modify the single error event WEF from Chapter 11 to obtain
a multiple error event WEF appropriate for PCCC's.
From Figure 16.9, we can see that any multiple error event in a codeword belonging to a terminated
convolutional code can be viewed as a succession of single error events separated by sequences of 0's. We
begin by considering all codewords that can be constructed from a single error event of length   N ,
i.e., the error event is surrounded by (N ) 0's. Since the (N ) 0's are divided into two groups,
one preceding and one following the error event, the number of single error events is the number of ways
of summing two non-negative integers that add to N . Thus, the multiplicity of block codewords for
34 CHAPTER 16. TURBO CODING

single error events is given by

 
c [; 1] = N +1 =N  + 1: (16.35)
1
Next consider a pair of error events with total length   N , i.e., the two error events have a total of
(N ) 0's appearing before, after, or between them. In this case, the (N ) 0's are divided into three
groups and the number of double error events is the number of ways of summing three non-negative
integers that add to N . Thus, the multiplicity of block codewords for double error events is given
by
 
c [; 2] = N  + 2 = (N  + 2)(N  + 1) : (16.36)
2 2
In general, the number of h-error events with total length   N is the number of ways of summing
h +1 non-negative integers that add to N , and the multiplicity of block codewords for h-error events
is given by (see Problem 16.7)
 
c [; h] = N h + h : (16.37)

Figure 16.9: Multiple Error Events in a Terminated Convolutional Code

In order to proceed with the computation of the CWEF's Aw (Z ) of the equivalent block code
produced by a convolutional encoder which terminates after an input sequence of length N , we must
know the number A(1) w;z; of single error events of length   N with input weight w and parity weight z ,

the number A(2)w;z; of double error events of total length   N with input weight w and parity weight

z , and so on. The single-error event enumerators A(1) (1) z

P
w; (Z ) = (z) Aw;z; Z ;   N , can be obtained
directly from the IRWEF A(W,Z,L) of the unterminated convolutional code by simply dropping all
terms of order larger than LN and then collecting together all terms with input weight w and length
. In order to determine the double-error event enumerators A(2) w; Z ;   N , we must examine all
z
16.2. DISTANCE PROPERTIES OF TURBO CODES 35
pairs of terms in the IRWEF A(W; Z; L) for which the total length of the two error events is   N .
This can be accomplished by de ning the double-error event IRWEF A(2)(W; Z; L) of the unterminated
convolutional code as follows:
A(2) (W; Z; L) = [A (W; Z; L)]2 : (16.38)
The double-error event enumerators A(2)
w; (Z );   N , can now be obtained from the double-error event

IRWEF A(2)(W; Z; L) by dropping all terms of order larger than LN and then collecting together all
terms with input weight w and length . Higher order error event enumerators can be found in a similar
way. Then the CWEF's can be computed as
X
Aw (Z ) = c [; h] A(w;
h)
(Z ); (16.39)
(N; 1hhmax )
where hmax, the largest possible number of error events associated with a weight w input sequence,
depends on w and the code parameters.
Example 16.6
For the (2,1,2) systematic feedback convolutional encoder whose generator matrix is given by
h i
Gfb (D) = 1 1+D2 (16.40)
1+D+D2 :

and whose encoder block diagram and IRWEF state diagram are shown in Figure 16.10, consider the
(18,7) block code obtained from a length N = 9 input sequence, including two termination bits. The
IRWEF of this encoder is given by (see Problem in Chapter 11)
A(W; Z; L) = L3 W 3 Z 2 + L4 W 2 Z 4 + L5 (W 3 Z 4 + W 4 Z 2 ) +
L6 (2W 3 Z 4 + W 4 Z 4 ) + L7 (W 2 Z 6 + 2W 4 Z 4 + W 5 Z 2 + W 5 Z 4 ) +
L8 (2W 3 Z 6 + 3W 4 Z 4 + 2W 5 Z 4 + W 6 Z 4 ) +
L9 (3W 3 Z 6 + 3W 4 Z 6 + 3W 5 Z 4 + W 6 Z 2 + 2W 6 Z 4 + W 7 Z 4 ) +


: (16.41)
After dropping all terms of order larger than L9, the single-error event enumerators are obtained as
36 CHAPTER 16. TURBO CODING

follows:
A(1) 4
2;4 (Z ) = Z ; A(1)
2;7 (Z ) = Z
6

A(1) 2
3;3 (Z ) = Z ; A(1) 4
3;5 (Z ) = Z ; A(1) 4 (1) 6 (1)
3;6 (Z ) = 2Z ; A3;8 (Z ) = 2Z ; A3;9 (Z ) = 3Z
6

A(1) 2
4;5 (Z ) = Z ; A(1) 4
4;6 (Z ) = Z ; A(1) 4 (1) 4 (1)
4;7 (Z ) = 2Z ; A4;8 (Z ) = 3Z ; A4;9 (Z ) = 3Z
6

A(1) 2 4 (1) 4
5;7 (Z ) = Z + Z ; A5;8 (Z ) = 2Z ; A(1)
5;9 (Z ) = 3Z
4

A(1) 4
6;8 (Z ) = Z ; A(1) 2
6;9 (Z ) = Z + 2Z
4

A(1) 4
7;9 (Z ) = Z :
(16.42)
Note that there are no weight 1 input sequences for this code since a second "1" is needed to terminate
the encoder. In order to determine the double-error event enumerators A(2) w; (Z ), we rst form the

double-error event IRWEF A(2)(W; Z; L) as follows:

A(2) (W; Z; L) = [A(W; Z; L)]2
= L6W 6Z 4 + 2L7W 5Z 6 + L8(W 4Z 8 + 2W 6Z 6 + 2W 7Z 4) +
L9 (2W 5 Z 8 + 6W 6 Z 6 + 2W 7 Z 6 ) +


: (16.43)
Dropping all terms of order greater than L9 gives the double-error event enumerators
A(2)
4;8 (Z ) = Z 8
A(2) (2)
5;7 (Z ) = 2Z 6 ; A5;9 (Z ) = 2Z 8
(16.44)
A(2) (2) (2)
6;6 (Z ) = Z 4 ; A6;8 (Z ) = 2Z 6 ; A6;9 (Z ) = 6Z 6 ;
A(2) (2)
7;8 (Z ) = 2Z 4 ; A7;9 (Z ) = 2Z 6 :
Following the same procedure, the triple-error event IRWEF is given by
A(3) (W; Z; L) = [A(W; Z; L)]3 = L9 W 9 Z 6 +


; (16.45)
and we see that there is only one triple error event of total length   9. The triple-error event
enumerator is given by
A(3) 6
9;9 (Z ) = Z ; (16.46)
16.2. DISTANCE PROPERTIES OF TURBO CODES 37
and it is clear that there are no higher order error events of total length   9.
Before using (16.39) to compute the CWEF's, we must determine hmax for each input weight w.
From the double-error event IRWEF in (16.43) we see that there are no double error events with input
weight w < 4, i.e., hmax = 1 for input weights 2 and 3. Similarly, from (16.45) we see that there are no
triple error events with input weight w < 9, i.e., hmax = 2 for input weights 4 through 7. Finally, it is
clear that hmax = 3 for input weight 9. Now using (16.37) and (16.39), we can compute the CWEF's
for the (18,7) block code as follows:

A2 (Z ) = c[4; 1]A(1) (1) 4

2;4(Z ) + c[7; 1]A2;7(Z ) = 6Z + 3Z
6

A3 (Z ) = c[3; 1]A(1) (1) (1) (1) (1)

3;3(Z ) + c[5; 1]A3;5(Z ) + c[6; 1]A3;6(Z ) + c[8; 1]A3;8(Z ) + c[9; 1]A3;9(Z )
= 7Z 2 + 13Z 4 + 7Z 6
A4 (Z ) = c[5; 1]A(1) (1) (1) (1) (1) (2)
4;5(Z ) + c[6; 1]A4;6(Z ) + c[7; 1]A4;7(Z ) + c[8; 1]A4;8(Z ) + c[9; 1]A4;9(Z ) + c[8; 2]A4;8(Z )
= 5Z 2 + 16Z 4 + 3Z 6 + 3Z 8
A5 (Z ) = c[7; 1]A(1) (1) (1) (2) (2)
5;7(Z ) + c[8; 1]A5;8(Z ) + c[9; 1]A5;9(Z ) + c[7; 2]A5;7(Z ) + c[9; 2]A5;9(Z )
= 3Z 2 + 10Z 4 + 12Z 6 + 2Z 8
A6 (Z ) = c[8; 1]A(1) (1) (2) (2) (2)
6;8(Z ) + c[9; 1]A6;9(Z ) + c[6; 2]A6;6(Z ) + c[8; 2]A6;8(Z ) + c[9; 2]A6;9(Z )
= Z 2 + 14Z 4 + 12Z 6
A7 (Z ) = c[9; 1]A(1) (2) (2) 4
7;9(Z ) + c[8; 2]A7;8(Z ) + c[9; 2]A7;9(Z ) = 7Z + 2Z
6

A9 (Z ) = c[9; 3]A(3) 6
9;9(Z ) = Z : (16.47)

As a check, we note that the CWEF's include a total of 127 nonzero codewords, the correct number for
an (18,7) code. Finally, the IRWEF and WEF are given by


A(W; Z ) = W 2 6Z 4 + 3Z 6 + W 3 7Z 2 + 13Z 4 + 7Z 6
+ W 4 5Z 2 + 16Z 4 + 3Z 6 + 3Z 8
+
W5 3Z 2 + 10Z 4 + 12Z 6 + 2Z 8
+ W 6 Z 2 + 14Z 4 + 12Z 6
+
W7 7Z 4 + 2Z 6
+ W 9Z 6 (16.48a)
38 CHAPTER 16. TURBO CODING

and
A(X ) = 7X 5 + 11X 6 + 16X 7 + 20X 8 + 17X 9 + 17X 10 + 19X 11 + 15X 12 + 4X 13 + X 15 : (16.48b)


WL
(0)
v

S3

ZL ZL
u
WZL WL WZL
(1)
S0 S1 S2 S0
v
L

Figure 16.10: Encoder Diagram and IRWEF State Diagram for Example 16.6

Note that this terminated convolutional code has free distance 5, the same as the free distance of
the unterminated code, i.e., it is an (18,7,5) code. We can also see that it has codeword multiplicities
of A5 = 7; A6 = 11; A7 = 16;


, i.e., it has a dense weight spectrum. We will next calculate its
average weight spectrum when it is used, along with a size N = 9 interleaver, as a constituent code in
a (3N; N m) = (27; 7) PCCC.
Before proceeding with this calculation, however, we must modify the uniform interleaver concept to
take into account the fact that the termination bits of a convolutional encoder are not information bits.
In other words, for an input sequence of length N , the number of information bits is only N m, where
m is the encoder memory order, and the denominator in the uniform interleaver averaging expression
given in (16.23) must be modi ed so that we divide only by the number of valid input sequences of
each weight and not by all length N sequences of a given weight. That is, because the m termination
bits are xed for a given information sequence of length N m, only a fraction 1=2m of all length N
input sequences are valid. This technical detail complicates our example somewhat, but, as we will see
shortly, for large N and small m, i.e., normal turbo code parameter values, an approximate analysis
16.2. DISTANCE PROPERTIES OF TURBO CODES 39
can be used that makes this modi cation unnecessary.
Example 16.6 (Continued)
We begin by noting from (16.47) that there is one valid input sequence for weights 0 and 9, there are
no valid input sequences for weights 1 and 8, there are nine valid input sequences for weights 2 and 7,
and there are twenty seven valid input sequences for weights 3, 4, 5, and 6. Then, using the modi ed
version of (16.23), we can compute the average CWEF's of the (27,7) PCCC as follows:
Z 4 + 3Z 6 2


A2 (Z ) =
P C 6 = 4Z 8 + 4Z 10 + Z 12;
9
Z 2 + 13Z 4 + 7Z 6 2


A3 (Z ) =
P C 7
27
= 1:81Z + 6:74Z 6 + 9:89Z 8 + 6:74Z 10 + 1:81Z 12;
4
Z 2 + 16Z 4 + 3Z 6 + 3Z 8 2


A4 (Z ) =
P C 5
27
= 0:93Z 4 + 5:93Z 6 + 10:59Z 8 + 4:67Z 10 + 3:89Z 12 + 0:67Z 14 + 0:33Z 16;
3Z 2 + 10Z 4 + 12Z 6 + 2Z 8
2
AP5 C (Z ) =
27
= 0:33Z + 2:22Z 6 + 6:37Z 8 + 9:33Z 10 + 6:81Z 12 + 1:78Z 14 + 0:15Z 16;
4
Z 2 + 14Z 4 + 12Z 6 2


A6 (Z ) =
P C
27
= 0:04Z 4 + 1:04Z 6 + 8:15Z 8 + 12:44Z 10 + 5:33Z 12;
Z 4 + 2Z 6 2


A7 (Z ) =
P C 7 = 5:44Z 8 + 3:11Z 10 + 0:44Z 12;
9
Z6 2


12
9 (Z )
AP C = 1 =Z : (16.49)
Finally, the average IRWEF AP C (W; Z ) and the average WEF AP C (X ) can be computed using (16.24)
and (16.26a), respectively (see Problem 16.8). 

Remarks:
 The free distance of the (27,7) PCCC is 7, an increase of 2 compared to the (18,7) constituent
code. An increase in free distance is expected, though, since the (27,7) code has a lower rate than
the (18,7) code. Note that in this case the minimum weight codeword is produced by an input
sequence of weight 3.
40 CHAPTER 16. TURBO CODING

 The average multiplicities of the low weight codewords, viz., A7 = 1:81; A8 = 0:93, and A9 = 7:07,
are small compared to the low weight multiplicities of the (18,7) code indicated above. This is
another example of spectral thinning.
 Increasing the code length and interleaver size N leads to additional spectral thinning, which
results in improved performance at low SNR's. In fact, we will see shortly that, for large N , the
multiplicities of the low weight codewords in a PCCC are reduced by a factor of N compared to
the constituent codes. This multiplicity reduction factor is called the interleaver gain.
 The calculation of the average bit IRWEF's B (W; Z ) and B P C (W; Z ) for this example is also
complicated by the fact that the termination bits are not information bits, and the factor (w=N )
in (16.25) must be adjusted to consider only the information bits (see Problem 16.9).
 The observations that turbo coding produces (1) a normal increase in free distance due to the
reduction in code rate, and (2) a large decrease in codeword and bit multiplicities due to the
interleaver gain are illustrations of the fact that, unlike conventional code design, turbo codes are
designed to reduce the low weight multiplicities rather than to increase the free distance. This
results in much better performance than conventional codes at low and moderate SNR's, but
somewhat weaker performance at high SNR's.
 By including the termination bits for the rst encoder in the interleaver, we cannot guarantee that
the second encoder terminates (in fact, as noted earlier, it will do so only with probability 1=2m),
but this has little e ect on code performance for large N .
Before extending our analysis to larger block lengths and interleaver sizes, we note that the uniform
interleaver averaging represented
 
by (16.23) results in dividing the product of the constituent code
CWEF's by the factor Nw . Since there are no weight 1 input sequences to a terminated systematic
feedback encoder (because of the termination bits), the nonzero inputweight with the smallest division
factor in (16.23) is the w = 2 term, for which the division factor is N2 . For w = 3, the division
 
N
factor 3 is larger by roughly a factor of N . In other words, compared to their relative in uence
in the individual constituent codes, weight 3 input sequences in PCCC's are less important than weight
2 input sequences, since they are associated with lower average codeword and bit multiplicities. Thus,
particularly for large block lengths N , the codewords associated with weight 2 input sequences become
the most important contributors to the low order terms in the weight spectrum of PCCC's.
In order to develop an approximate weight spectrum analysis of PCCC's for large block lengths, we
16.2. DISTANCE PROPERTIES OF TURBO CODES 41
begin by simplifying the expression given in (16.37) for the number of codewords containing h error
events of total length  in a terminated convolutional code with block length N . Since, for the low
weight codewords of primary interest, both the number of error events h and the total length  of
error events cannot be large (or else the weight would also be large), for large N it is reasonable to
approximate the number of codewords containing h error events in a block as
 
c [h]  N ; (16.50)
h
independent of the total length  of the error events. Then the CWEF's of the equivalent (nN; N m)
block code are given by
X
Aw (Z ) = c [h] A(wh) (Z ); (16.51)
(1hhmax )
where A(wh)(Z ) is the h-error event enumerator for input weight w. (We note here that the h-error event
enumerators of (16.39) and (16.51), A(w; h)
(Z ) and A(wh)(Z ), respectively, are related by the expression
A(wh) (Z ) = (N ) A(w;
h)
(Z ), i.e., A(wh)(Z ) counts h-error events of all lengths.)
P

From (16.23), (16.50), and (16.51), the average CWEF's of the PCCC with the same (nN; N m)
terminated convolutional code as constituent codes are given by
X X c [h1 ] c [h2 ] (h )
AP C (Z ) =
w   A
N
(Z )A(h )(Z ):
w
1
w
2
(16.52)
(1h1 hmax ) (1h2 hmax )
w
For N  h, we can use the approximation
  h
N
h  Nh! (16.53)
to obtain
X X w!
APw C (Z )  N (h1 +h2 w)  A(wh1 ) (Z )A(wh2 ) (Z ): (16.54)
(1h1 hmax ) (1h2 hmax ) h 1 !  h 2 !
We can further approximate by saving only the (most signi cant) term in the double summation of
(16.54) with the highest power of N , i.e., the term corresponding to h1 = h2 = hmax, which gives us
the codeword CWEF
w! (2hmax w) A(hmax ) (Z ) 2
h i
APw C (Z )  N (16.55a)
(h !)2 max
w
42 CHAPTER 16. TURBO CODING

and the bit CWEF

w w! (2hmax w) A(hmax ) (Z ) 2 = w  w! N (2hmax
h i
1) A(hmax ) (Z ) 2 :
h i
BwP C (Z )   N w (16.55b)
N (hmax!)2 w
(hmax!)2 w

(It is worth noting that taking only the term corresponding to h1 = h2 = hmax in (16.54) is equivalent
to saying that, for a xed input weight w, the lowest weight codeword is likely to contain the maximum
number of error events, so that it is merged with the all-zero codeword as much as possible.) Finally,
the average IRWEF's of the PCCC are given by
X
AP C (W; Z ) = W w APw C (Z ) (16.56a)
(1wN )
and
X w w PC X
B P C (W; Z ) = W Aw (Z ) = W w BwP C (Z ); (16.56b)
(1wN ) N (1wN )
and the average WEF's can be computed using (16.26). (When calculating the approximate IRWEF's
from (16.56), the sums should be taken only over those w for which the approximations of (16.50),
(16.53), and (16.55) are reasonably accurate, i.e., w's of relatively low weight.) The extension of this
analysis to PCCC's with di erent constituent codes is straightforward.
Example 16.7
Consider a (3N; N 2) PCCC where both constituent codes are generated by the (2,1,2) systematic
feedback encoder of Example 16.6, but now for a block length of N  m = 2. For input weights w = 2
and 3, hmax = 1 and (16.55) simpli es to
h i2
APw C (Z )  w!  N (2 w)  A(1)
w (Z ) (16.57a)
and
h i2
BwP C (Z )  w  w!  N (1 w)  A(1)
w (Z ) ; (16.57b)
and for input weights w = 4 and 5, hmax = 2 and (16.55) is given by
w! h i2
APw C (Z )   N (4 w)  A(2) ( Z ) (16.58a)
4 w
16.2. DISTANCE PROPERTIES OF TURBO CODES 43
and
w! h i2
BwP C (Z )  w   N (3 w)  A(2) ( Z ) : (16.58b)
4 w

(Note that in Example 16.6 when N = 9, hmax = 2 also for w = 6 and 7, but this is not the case for
large N .) Including only these terms in the approximate average IRWEF's for the PCCC gives
X
AP C (W; Z )  W w APw C (Z )
(2w5)
=
h
2W 2 A(1) ( Z )
i2
+ 6 W 3 hA(1)(Z )i2 + 6W 4 hA(2)(Z )i2 +
2 N 3 4
30 W 5 A(2)(Z ) 2 +



h i
(16.59a)
N 5

and
X
B P C (W; Z )  W w BwP C (Z )
(2w5)
= 4 W 2 hA(1)(Z )i2 + 18 W 3 hA(1)(Z )i2 + 24 W 4 hA(2)(Z )i2 +
N 2 N2 3 N 4
150 W 5 A(2)(Z ) +


:
h i 2
(16.59b)
N2 5

The single and double error event enumerators A(1) (2)

w (Z ) and Aw (Z ) needed to evaluate (16.59) include

all single and double error events up to length N . However, the terms of most interest, i.e., the lowest
weight terms, can be obtained from Example 16.6, which counts all error events up to length N = 9.
Thus from (16.42) and (16.44), we obtain
A(1) (1)
2 (Z )  Z 4 + Z 6 A3 (Z )  Z 2 + 3Z 4 + 5Z 6 (16.60)
A(2)
4 (Z )  Z
8 A(2) 6
5 (Z )  2Z + 2Z
8

and (16.59) becomes

 2W 2 Z 4 + Z 6 2 + N6 W 3 Z 2 + 3Z 4 + 5Z 6 2 + 6W 4 Z 8 2 + 30 W 5 2Z 6 + 2Z 8 2 +



       
AP C (W; Z )
N
= W 2 2Z 8 + 4Z 10 + 2Z 12 + W 4 6Z 16
+ W 3 N1 (6Z 4 + 36Z 6 + 114Z 8 + 180Z 10 +

150Z 12) + W 5 N1 120Z 12 + 240Z 14 + 120Z 16
+


(16.61a)

44 CHAPTER 16. TURBO CODING

and
 W 2 N1 4Z 8 + 8Z 10 + 4Z 12 + W 4 N1 24Z 16 +



B P C (W; Z )
W3 2
1 18Z 4 + 108Z 6 + 342Z 8 + 540Z 10 + 450Z 12
+
N
1
W5 2 600Z 12 + 1200Z 14 + 600Z 16
+


(16.61b)
N
Note that the approximate expressions of (16.61) contain no terms of order (1=N )3 or higher. (These
terms clearly have little in uence on the IRWEF's for large values of N .) Finally, we obtain the
approximate average WEF's from (16.26) as follows:
6 36
AP C (X )  X 7 + X 9 + 2X 10 +
114 X 11 + 4X 12 + 180 X 13 + 2X 14 +


(16.61c)
N N N N
and
B P C (X ) 
18 X 7 + 108 X 9 + 4 X 10 + 342 X 11 + 8 X 12 + 540 X 13 + 4 X 14 +


: (16.61d)
N2 N2 N N2 N N2 N


Remarks:
 The minimum free distance of the (3N; N 2) PCCC still appears to be 7, but the average
codeword multiplicity is only A7 = 6=N . For large N , this means that the probability of a
particular interleaver resulting in a weight 7 codeword is very small, i.e., the free distance in
almost all cases will be greater than 7.
 (16.61c) and (16.61d) do not indicate the presence of any codewords of weight 8. This is due to
the fact that we kept only the most signi cant term in the double summation of (16.54). If the
term leading to weight 8 codewords had not been deleted from (16.54), however, it would have
order (1=N )2 in AP C (X ) and order (1=N )3 in BP C (X ), i.e., it would have a negligible e ect on
the WEF's for large N .
 The average multiplicity of weight 9 codewords is A9 = 36=N , i.e., for large N the probability of
a weight 9 codeword is very small.
 The average multiplicity of weight 10 codewords is A10 = 2, i.e., the interleaver does not purge
the code of weight 10 codewords, and in almost all cases the free distance of the PCCC will be 10.
We see that the weight 10 codewords are generated by weight 2 input sequences, thus illustrating
the importance of weight 2 input sequences when N is large that was noted earlier.
16.2. DISTANCE PROPERTIES OF TURBO CODES 45
 The (average of) two weight 10 codewords are produced by the N 2 input sequences u(D) =
D (1 + D3 ); 0    N 3, where we note that 1 + D3 is the shortest weight 2 input sequence
that terminates the encoder. This input sequence generates the weight 4 parity sequence v(D) =
1 + D + D2 + D3. A pseudorandom interleaver will typically leave two of these N 2 input
sequences unchanged (except for time shifts), and in these cases the second encoder will output
the same weight 4 parity sequence, resulting in codewords of weight 10. (To see this, assume that
the rst \1" in one of the above weight 2 input sequences is permuted to an arbitrary position
by the interleaver. There are then two positions, out of the remaining N 1 positions, that the
second \1" can appear and still give the same input pattern, viz., either three positions before or
three positions after the rst \1". Thus each of the N 2 \bad" input sequences has probability
2=(N 1) of retaining its \bad" property after interleaving, which leads to the average multiplicity
of A10 = 2 for weight 10 codewords.)
 For each constituent terminated convolutional code by itself, the low weight codeword multiplicities
increase linearly with N (because of time invariance), but for the PCCC the multiplicities of the
low weight codewords are smaller by factors of (1=N )p; p  1. Note that the largest factor, i.e.,
(1=N )1, is associated with terms resulting from input sequences of weight 2.
 From (16.61d), we see that the bit multiplicities of the low weight codewords are B7 = 18=N 2; B9 =
108=N 2; B10 = 4=N , and so on. Since the bit multiplicities of the low weight codewords in the
PCCC decrease with N , we say that the PCCC possesses interleaver gain.

Example 16.7 (Continued)

We continue the example by computing the approximate low weight codeword and bit multiplicities,
using (16.61c) and (16.61d), for three speci c cases: N = 100, 1000, and 10000. The results are pre-
sented in Table 16.5 and the entries for the bit multiplicities Bd clearly portray the interleaver gain
e ect. For each weight d, the value of Bd decreases by one or two orders of magnitude for each order of
magnitude increase in N . Note in particular that the interleaver gain for codeword weights 7 and 9 is
larger by roughly a factor of (1=N ) than for codeword weight 10. This is due to the fact that weight 10
codewords are generated by weight 2 input sequences, whereas weight 7 and 9 codewords are generated
by weight 3 input sequences. This is another illustration of the importance of weight 2 input sequences
when the block length N becomes very large. 
46 CHAPTER 16. TURBO CODING

Table 16.5: Approximate Codeword and Bit Multiplicities for the (3N,N-2) PCCC of Example 16.7
N 100 1000 10000
d Ad Bd Ad Bd Ad Bd
7 0.06 0.0018 0.006 0.000018 0.0006 0.00000018
8 0.00 0.0000 0.000 0.000000 0.0000 0.00000000
9 0.36 0.0108 0.036 0.000108 0.0036 0.00000108
10 2.00 0.0400 2.000 0.004000 2.0000 0.00040000
11 5.40 0.1620 0.540 0.001620 0.0540 0.00001620
12 4.00 0.0800 4.000 0.008000 4.0000 0.00080000
13 1.80 0.0540 0.180 0.000540 0.0180 0.00000540
14 2.00 0.0400 2.000 0.004000 2.0000 0.00040000
15 1.50 0.0450 0.150 0.000450 0.0150 0.00000450
16 0.00 0.0000 0.000 0.000000 0.0000 0.00000000
17 1.20 0.0600 0.120 0.000600 0.0120 0.00000600
18 0.00 0.0000 0.000 0.000000 0.0000 0.00000000
19 2.40 0.1200 0.240 0.001200 0.0240 0.00001200

In Problems 16.7 and 16.8, we repeat Examples 16.6 and 16.7 for the systematic feedback convolu-
tional encoder whose generator polynomials are inverted, i.e., the encoder given by
h i
Gfb (D) = 1 1+D+D2 2 : (16.62)
1+D

The (2,1,2) convolutional code generated by this encoder also has dfree = 5. However, in this case, the
minimum weight codeword is produced by an input sequence of weight 2, i.e., u(D) = 1 + D2. Thus,
when this code is converted to a (3N; N 2) PCCC, its minimum free distance for both small and large
values of N will be 8. (The parity weight corresponding to the weight 2 input sequence is three, which
is repeated twice in the PCCC.) In other words, when the minimum weight codeword is produced by
an input sequence of weight 2, increasing the block length N results in interleaver gain, which reduces
the bit multiplicities, but no improvement in free distance is achieved. This contrasts with the code in
Examples 16.6 and 16.7, where the free distance is improved from 7 to 10 by increasing the block length
N.
From the results of Examples 16.6 and 16.7 and Problems 16.10 and 16.11, we conclude that in
order to maximize the free distance of a PCCC, a constituent code should be chosen whose minimum
16.2. DISTANCE PROPERTIES OF TURBO CODES 47
weight codeword is produced by an input sequence of weight greater than 2. This is usually the case
when the feedback polynomial is chosen to be primitive, as in Examples 16.6 and 16.7. This is because
a primitive polynomial has maximum cycle length, i.e., it maximizes the length of the weight 2 input
sequence needed to terminate the encoder. Depending on the numerator polynomial, this normally
results in higher parity weight than one or more shorter, higher weight input sequences that terminate
the encoder. Since the free distance determines the asymptotic, i.e., large SNR, behavior of a code,
primitive feedback polynomials are good choices to optimize the large SNR performance of PCCC's.
One should not conclude, however, that the choice of primitive feedback polynomials also optimizes the
small SNR performance, where the low weight codeword multiplicities and the dynamics of iterative
decoding are more important than the free distance. In fact, experience with di erent code selections
seems to suggest that, in general, primitive feedback polynomials optimize large SNR behavior, but
non-primitive polynomials with shorter cycle lengths are better for small SNR's. A more extensive
analysis of constituent code design for PCCC's is included in Section 16.4
The above discussion, along with a close examination of the power of N , viz., 2hmax w 1, in
(16.55b), illustrates the fact that the input weights that produce low weight codewords in the constituent
code must be at least 2 in order to achieve interleaver gain in PCC's. Interleaver gain results only when
the power of N in (16.55b) is less than or equal to 1, so that the bit multiplicities decrease with
increasing N . Since hmax is always 1 for the lowest weight input sequences, this means that the lowest
input weight w that produces low weight codewords must be at least 2. When feedback encoders,
i.e., recursive convolutional codes, are used as constituent codes, almost all weight 1 input sequences
produce high weight parity sequences, and thus high weight codewords. (The only exception is when the
nonzero input doesn't appear until the end of the information block, but the number of these sequences
is small and does not grow linearly with N . Also, when the encoder is terminated, then at least one
more \1" must enter the encoder, and hence the total input weight is still at least 2 in this case.) Thus
for recursive constituent codes, all low weight codewords are produced by input sequences of weight at
least 2, and interleaver gain is achieved.
48 CHAPTER 16. TURBO CODING

This is not the case when short block codes are used as constituent codes. When a short block
code is repeated to achieve a large block length and interleaver size, there exists low weight codewords
produced by a single 1, thus negating the possibility of interleaver gain. If a single long block code is
used as a constituent code, the desired property can be achieved by choosing the weight of each row of
the generator matrix G to be large. However, eÆcient SISO decoding methods for such long codes are
diÆcult to realize.
Interleaver gain is also not possible when short constraint length feedforward convolutional encoders,
i.e., nonrecursive convolutional codes, are used as constituent codes. This is due to the fact that single
input 1's in short constraint length feedforward encoders produce short, low weight output sequences,
and thus large low weight codeword multiplicities result from parallel concatenation. This point is
illustrated in the next example.
Example 16.8
Consider the (3N; N 2) turbo code (PCCC) generated by the parallel concatenation of a (2,1,2)
nonsystematic feedforward encoder, as shown in Figure 16.11. (Recall that for nonsystematic encoders,
the weight spectrum is represented by the IOWEF rather than the IRWEF.) Assume the generator
matrix of the encoder is given by

Gff (D) = 1 + D + D2 1 + D2  : (16.63)
The code produced by this encoder is identical to the code produced by the systematic feedback encoder
of Examples 16.6 and 16.7, but the mapping between input sequences and codewords is di erent. That
is, the two codes have identical WEF's but di erent IOWEF's. To analyze the weight spectrum of this
PCCC, we must determine both the conditional IOWEF ACw (X ) of the nonsystematic nonrecursive
1

constituent code C1 and the conditional IRWEF ACw (Z ) of the (2,1,2) systematic nonrecursive con-
2

stituent code C2 whose parity generator is used to re-encode the input sequence after interleaving. In
our example, this systematic nonrecursive code is generated by the feedforward encoder

Gff (D) = 1 1 + D2  : (16.64)
16.2. DISTANCE PROPERTIES OF TURBO CODES 49
Then the e ect of uniform interleaving is represented by
ACw1 (X )  ACw2 (Z )
APw C (X; Z ) =   ; (16.65)
N
w
and the approximate expression for large N is
X X c[h1 ]c[h2 ] (h1 )
APw C (X; Z )     Aw (X )A(wh2 ) (Z ); (16.66)
(1h1 hmax ) (1h2 hmax ) N
w

where A(wh )(X ) and A(wh )(Z ) are the conditional IO and IR h-error event WEF's of the nonsystematic
1 2

and systematic constituent codes, C1 and C2, respectively. Now following the further approximations
of (16.53) - (16.55) results in
w!
APw C (X; Z ) 
h1max !  h2max !
 N (h1max+h2max w)  A(wh1max )(X )Aw(h2max)(Z ) (16.67a)
and
w!
BwP C (X; Z )  w 
h1max !  h2max
!  N (h max+h max
1 2 w 1)  A(h1max ) (X )A(h2max ) (Z ):
w w (16.67b)
Since the total codeword weights in the resulting PCCC are sums of powers of X and Z , the average
CWEF's are
APw C (X ) = APw C (X; Z ) Z =X (16.68a)
and
BwP C (X ) = BwP C (X; Z ) Z =X ; (16.68b)
the average IOWEF's are given by
X
AP C (W; X ) = W w APw C (X ) (16.69a)
(1wN )
and
X w w PC X
B P C (W; X ) = W Aw (X ) = W w BwP C (X ); (16.69b)
(1wN ) N (1wN )
50 CHAPTER 16. TURBO CODING

and the average WEF's are given by

AP C (X ) = AP C (W; X ) W =1 (16.69c)
and
B P C (X ) = B P C (W; X ) W =1 : (16.69d)

u
v(0)

v(1)
π
v(2)
Figure 16.11: A PCCC Using Nonrecursive Constituent Codes

Beginning with the nonsystematic constituent code generated by (16.63), its IOWEF is given by
(see Problem in Chapter 11)
A(W; X; L) = W X 5 L3 + W 2 L4 (1 + L)X 6 + W 3 L5 (1 + L)2 X 7 +


+ W w Lw+2 (1 + L)w 1 X w+4 +


:
(16.70)
Going directly to the large N case, we note that for any input weight w; w  1; hmax = w, since the
weight 1 single error event can simply be repeated w times, and therefore
A(ww) (X ) = X 5w ; w  1: (16.71)
16.2. DISTANCE PROPERTIES OF TURBO CODES 51
Now for the systematic constituent code generated by (16.64), its IRWEF is given by (see Problem in
Chapter 11)


A(W; Z; L) = W Z 2 L3 + W 2 Z 4 L4 + W 2 Z 2 L5 (1 + W Z 2 ) + W 3 Z 2 L6 2Z 2 + W Z 2 +


: (16.72)

Again, for any input weight w; w  1; hmax = w, and hence

A(ww) (Z ) = Z 2w ; w  1: (16.73)

Now substituting into (16.67) yields

w! N w 5w 2 w
APw C (X; Z )  N (w+w w) X 5w Z 2w = X Z (16.74a)
(w!)  (w!) w!

and
N (w 1) 5w 2w
BwP C (X; Z )  (16.74b)
(w 1)! X Z :
Finally, applying (16.68) and (16.69) gives the average IOWEF's
X N w 7w N2 N3
AP C (W; X )  Ww X = NW X 7 + W 2 X 14 + W 3 X 21 +


(16.75a)
(1wN ) w! 2 6
and
N 2 3 21
B P C (W; X )  W X 7 + NW 2 X 14 + W X +


(16.75b)
2
and the average WEF's
N 2 14 N 3 21
AP C (X )  NX 7 + X + X +


(16.75c)
2 6
and
N 2 21
B P C (X )  X 7 + NX 14 + X +


(16.75d)
2


Remarks:
52 CHAPTER 16. TURBO CODING

 The free distance of the (3N, N-2) PCCC is 7, and the average codeword multiplicity A7  N
grows linearly with N . Also, the average bit multiplicity of the weight 7 codewords is B7 = 1.
This is due to the fact that the weight 7 codewords are generated by weight 1 input sequences, and
the interleaver has no e ect (other than time shifts) on weight 1 input sequences. This contrasts
with the feedback encoder case, where the bit multiplicities all decrease with increasing N . Thus
interleaver gain does not apply to feedforward encoders.
 (16.75) does not indicate the presence of any codewords with weights between 7 and 14. This is
due to the fact that we kept only the most signi cant term in the double summation of (16.66).
If the terms leading to codeword weights between 7 and 14 are also retained, we nd that the
corresponding codeword and bit multiplicities do not grow linearly with N , and thus they will
have a negligible e ect for large N compared to codeword weights 7 and 14.
 From the above example, it is clear that when feedforward encoders are employed, hmax is always
equal to w. This is because, in the feedforward case, the weight 1 input sequence produces the
minimum length (equal to the constraint length v) terminated output sequence, and hence the
maximum number of error events that can be produced by a weight w input sequence is achieved
by simply repeating this shortest error event w times. This is not true for feedback encoders, since
in this case the weight 1 input sequence produces an unterminated output sequence.
 As in the case of block constituent codes, long constraint length feedforward encoders could be
used as constituent codes, but eÆcient SISO decoding methods for such long codes are diÆcult to
realize.
In the next section, we use the results of the weight spectrum analysis presented in this section,
along with standard union bonding techniques, to evaluate the performance of PCCC's.

16.3 Performance Analysis of Turbo Codes

The union bounding techniques developed in Chapter 12 can now be used to estimate the performance
of turbo codes. We will consider both the word error probability
X
Pw (E )  Ad Pd ; (16.76)
(dfreed)
and the bit error probability
X
Pb (E )  Bd Pd ; (16.77)
(dfreed)
16.3. PERFORMANCE ANALYSIS OF TURBO CODES 53
where Ad is the codeword multiplicity, Bd is the bit multiplicity, and Pd is the pairwise error probability
for a codeword of weight d, and dfree is the minimum free distance of the code. In this chapter, we will
concentrate on code performance for a binary input, unquantized output AWGN channel. In this case,
as shown in Chapter 12, Pd is loosely bounded by
p 
Pd  Q 2dREb=N0 ; (16.78)
and a somewhat tighter bound is given by
 
d RE
Pd  f free b
N0
 (e REb =N0 d ; ) (16.79)
where R is the code rate, Eb=N0 is the energy per information bit to one-sided noise power spectral
density ratio, i.e., the SNR, and the function f (
) is de ned as
p 
f (x) = ex  Q 2x : (16.80)
Now substituting (16.79) back into (16.76) and (16.77) gives us the expressions
   d
 f dfreeNREb
X
Pw (E ) Ad e REb =N0
0 (dfreed)
 
d RE
= f free b
N0
 A(X ) (16.81)
X =e REb =N0

and
   d
d RE X
Pb (E )  f free b Bd e REb =No
N0 (dfreed)
 
d RE
= f free b  B (X )
N0
: (16.82)
X =e REb =N0

Thus, in order to bound the performance of any turbo code, all one needs is the free distance dfree, the
codeword WEF A(X ) and the bit WEF B(X ). For values of Eb=N0 above the SNR at which the code
rate R equals the channel cuto rate R0, i.e., for
Eb 1
> ln 21 R 1 1 ;


N0 R
(16.83)
54 CHAPTER 16. TURBO CODING

the rst few terms of each WEF are all that is needed for an accurate estimate of performance. For values
of Eb=N0 below this limit, however, more terms are needed and the bound diverges, a characteristic
that is typical of all union bounds.
We now consider several examples illustrating the application of the above bounds to estimate the
performance of turbo codes.
Example 16.9
Consider the R = 1=3 PCCC generated by the systematic feedback encoder of Examples 16.6 and 16.7.
Approximations to the codeword and bit WEF's are given in (16.61c) and (16.61d) and the free distance
of this code is dfree = 7. (Note that, for large N, most interlevaers will result in a free distance of dfree
= 10, but in evaluating the bound we must use the smallest possible free distance, viz, dfree = 7. )
Substituting these expressions into (16.81) and (16.82) allows us to calculate the bounds on word and
bit error probability as functions of Eb=N0. Results are shown in Figure 16.12 for three values of the
interleaver size (information block length): N = 100, 1000, and 10000. (The program used to plot the
bounds of (16.81) and (16.82) was based on the approximations of the WEF's given in (16.61c) and
(16.61d) but included more terms for greater accuracy.) Pw (E ) is plotted in Figure 16.12(a) and Pb (E )
in Figure 16.12(b). For comparison we also show the three corresponding Pw (E ) simulation curves for
the optimum free distance (3,1,4) terminated convolutional code in Figure 16.12(a) and the correspond-
ing Pb (E ) simulation curve for the unterminated version of this code in Figure 16.12(b). A 16-state
convolutional code is chosen for comparison to the 4-state PCCC because their decoding complexities
are roughly equal. Finally, a simulation curve for the PCCC with N = 10000 is shown in each gure.
The PCCC simulations were generated using a typical pseudorandom interleaver and 18 iterations of
decoding. 

Remarks:
 The PCCC curves in Figure 16.12(b) clearly indicate the e ect of interleaver gain. For each factor
of 10 increase in N , the Pb (E ) curves improve by roughly the same factor. (Note, however, that
16.3. PERFORMANCE ANALYSIS OF TURBO CODES 55
(a)
1e+00
PCCC Simulation 10000
PCCC Bound 100
PCCC Bound 1000
PCCC Bound 10000
(3,1,4) Simulation 100
(3,1,4) Simulation 1000
(3,1,4) Simulation 10000
1e-01
Pw(E)

1e-02

1e-03

1e-04
0 1 2 3 4 5 6 7
Eb/N0 (dB)

(b)
1e+00
PCCC Simulation 10000
PCCC Bound 100
PCCC Bound 1000
1e-01 PCCC Bound 10000
(3,1,4) Simulation

1e-02

1e-03
Pb(E)

1e-04

1e-05

1e-06

1e-07

1e-08
0 1 2 3 4 5 6 7
Eb/N0 (dB)

Figure 16.12: (a) Word and (b) Bit Errror Probability Curves for a PCCC and a Convolutional Code
with R = 1=3
56 CHAPTER 16. TURBO CODING

the SNR di erence in the Pb (E ) curves, i.e., the coding gain, achieved by increasing N becomes
smaller for large Eb=N0).
 The Pw (E ) curves for the PCCC in Figure 16.12(a) merge for large N because the number of
weight 10 codewords remains xed at two and does not decrease with increasing N . For the
convolutional code, on the other hand, Pw (E ) becomes worse as N increases. This is to be
expected, since with increasing block length the occurrence of at least one error event becomes
more likely. This e ect is masked by the interleaver for the PCCC, because there are no low
weight codewords whose multiplicities increase linearly with N .
 Compared to the convolutional code, the PCCC o ers substantial coding gain, both in terms of
Pw (E ) and Pb (E ). For example, for Pb (E ) = 10 5 , the PCCC with N = 10000 achieves a 3:8dB
coding gain compared to the convolutional code. This is in spite of the fact that the convolutional
code has free distance 12, larger than the (most likely) free distance of 10 for the PCCC.
 The bounds are not extended to SNR's below 2dB, the cuto rate limit for R = 1=3 given by
(16.83), since they diverge below this point, as noted above. (In the case of N=10000, the PCCC
simulation is used to extend the bound below 2 dB.) At lower SNR's, the simulation curves show
much better performance than predicted by the bounds, but at higher SNR's the bounds predict
the performance very accurately. Tighter bounding arguments can be employed to improve the
bounds at low SNR's [4], however.
 If the bounds are being used to estimate performance for a particular interleaver and block length
for which the low order terms in the WEF's are known, the factor f (dfreeREb=N0) should be
adjusted to the actual value of dfree.

Example 16.10
Now consider the series of R = 1=3 PCCC's with di erent memory orders whose constituent encoders
are shown in Table 16.6. With the exception of Code E, each of the generators was chosen to have a
primitive feedback polynomial so as to achieve the best possible minimum free distance. The generator
for Code E has a nonprimitive feedback polynomial. The results of the Pb (E ) bound of (16.82) for
each code are shown in Figure 16.13 for two values of the interleaver size (information block length):
N = 100 and 1000. In addition, the simulated performance of the two 16-state codes, Code D and Code
E, is shown in Figure 16.14 for a pseudorandom interleaver of size N = 210 = 1024 and 18 iterations of
decoding. 
16.3. PERFORMANCE ANALYSIS OF TURBO CODES 57
Remarks:
 The bounds in Figure 16.13 indicate that substantial coding gains can be achieved at Pb (E )'s below
10 5 by increasing the memory order m of the constituent code. For example, for Pb (E ) = 10 8
and a block length of N = 1000, the 16-state primitive constituent code gains 6:5dB compared to
the 2-state constituent code.
 Selecting a larger memory order can be used as a substitute for a larger block length if decoding
delay is an issue. For example, the 16-state primitive code with N = 100 gains 0:5dB compared
to the 4-state code with N = 1000 for Pb (E ) = 10 7.
 The bounds in Figure 16.13 indicate that the 16-state primitive constituent code performs better
than the 16-state nonprimitive code, but the simulations in Figure 16.14 show that this is only
true at high SNR's. This is an illustration of the fact that the bounds only accurately predict
performance at SNR's above the cuto rate limit, 2 dB in this case. (Note that the simulation
results in Figure 16.14 show the performance of these two codes at a lower SNR range than the
bounds in Figure 16.13).
 The superior performance of the 16-state primitive code at higher SNR's is due to its larger free
distance (see Problem 16.15).
Table 16.6: Code Parameters for Figures 16.13 and 16.14
Code Generator
h Matrix i G(D )
A 1 1+D 1
h i
B 1 1+D2
1+D+D2
h i
C 1 1+D+D2+3D3
1+D+D
h i
D 1 1+D+D3 2+D4 4
1+D +D
h i
E 1 1+D4
1+D+D2+D3 +D4

Example 16.11
Finally, consider the two R = 1=3 PCCC's generated by the nonsystematic feedforward encoder of
Example 16.8 and by systematic feedback encoder of Examples 16.6 and 16.7. As we have noted previ-
ously, the constituent encoders by themselves generate identical R = 1=2 codes. The approximate bit
58 CHAPTER 16. TURBO CODING

1e-01
A 100
A 1000
B 100
1e-02 B 1000
C 100
C 1000
1e-03 D 100
D 1000
E 100
E 1000
1e-04

1e-05
Pb(E)

1e-06
B A A
C
1e-07
E

1e-08

1e-09
E B
D C D
1e-10
2 3 4 5 6 7 8 9 10
Eb/N0 (dB)

Figure 16.13: Bit Error Probability Bounds for R = 1=3 PCCC's with m = 1; 2; 3; and 4 and N = 100
and 1000
16.3. PERFORMANCE ANALYSIS OF TURBO CODES 59
1e-01
D
E

1e-02

1e-03 E D
Pb(E)

1e-04

1e-05

1e-06
0.2 0.4 0.6 0.8 1 1.2 1.4
Eb/N0 (dB)

Figure 16.14: Bit Error Probability Simulations for two R = 1=3 PCCC's with m = 4 and N = 1024
IOWEF of the nonrecursive PCCC is given in (16.75b). For large N , the free distance of this code is
dfree = 7. An approximation to the bit WEF is given in (16.75d). In Figure 16.15 we show the Pb (E )
bound obtained by substituting this expression into (16.82), along with the corresponding bound for the
recursive PCCC from Examples 16.6 and 16.7, shown previously in Figure 16.12(b). The interleaver size
(information block length) is N = 1000. Also plotted in the gure are the Pb (E ) bounds for the unter-
minated R = 1=3 convolutional codes obtained by simply repeating the second output of each R = 1=2
code without permuting the input sequence. These latter curves represent the equivalent R = 1=3
performance of the two constituent codes (nonrecursive and recursive) without parallel concatenation.


Remarks:

 The Pb (E ) curves for the unterminated convolutional codes without parallel concatenation (curves
A and C) indicate that the performance of the two codes is almost the same. This is to be
expected, since the two encoders generate identical codes and the only di erence in their BER's
60 CHAPTER 16. TURBO CODING

1
A: Convolutional Nonrecursive
B: PCCC Nonrecursive
C: Convolutional Recursive
0.1 D: PCCC Recursive

0.01

0.001

1e-04
Pb(E)

1e-05 B A C

1e-06

1e-07

D
1e-08

1e-09
2 3 4 5 6 7 8
Eb/N0 (dB)

Figure 16.15: Bit Error Probability Bounds for Nonrecursive and Recursive PCCC's and Convolutional
Codes with R = 1=3
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 61
is a consequence of the di erent mappings between information sequences and codewords.
 For the nonrecursive codes, there is little improvement achieved from parallel concatenation
(curves A and B). This is due to the fact that there is no interleaver gain for feedforward en-
coders.
 For the recursive codes, on the other hand, parallel concatenation results in substantial coding gain
(curves C and D). For example, for Pb (E ) = 10 6, the recursive PCCC with N = 1000 achieves
a 4:3 dB coding gain compared to its R = 1=3 equivalent code without parallel concatenation.
 The superiority of employing systematic feedback encoders rather than nonsystematic feedforward
encoders as constituent codes in a PCCC can clearly be seen by comparing curves B and D. For
example, for Pb (E ) = 10 6, the recursive PCCC gains 3:8 dB compared to the nonrecursive
PCCC.
In the next section, we continue our analysis of turbo codes by examining the parameters that
provide the best constituent code designs.
16.4 Design of Parallel Concatenated Turbo Codes

We will base our discussion of the design of PCCC's on the upper bound on bit error probability derived
in the previous section, which is valid for values of Eb=N0 above the cuto rate limit of (16.83). (For
smaller values of Eb=N0, performance bounds of suÆcient accuracy to determine code design rules are
not available.) We begin by using (16.25a) and (16.26b) to express the bit WEF B(X ) of a systematic
code as a sum over input weights w as follows:
X w w
B (X ) = W Aw (Z ) : (16.84)
(1wN ) N W =Z =X

Then, using the looser form of the bound of (16.82) (i.e., without the function f(
), a scaling factor that
depends only on dfree) and (16.84), we can approximate the bound on the bit error probability of a
PCCC as follows:
X w w
Pb (E )  W Aw (Z ) ; (16.85)
N
(wmin wN ) W =Z =e REb =N0

where wmin is the minimum weight nonzero valid input sequence, i.e., the minimum weight nonzero
input sequence that terminates the encoder. Now using the approximation for large N of (16.55a), we
62 CHAPTER 16. TURBO CODING

obtain
X w w w! (2hmax w)  A(hmax ) (Z ) 2
h i
Pb (E )  W N
(wmin wN ) N (hmax!)2 w
W =Z =e REb =N0
X w! (2hmax w 1)  W w A(hmax ) (Z ) 2
h i
= w
( h !)2 N w : (16.86)
(wmin wN ) max W = Z = e REb =N0

We begin our discussion of code design for PCCC's by noting that, for (n; 1; m) systematic constituent
encoders, the resulting PCCC has rate
R=
N m 1
(2n 1) N  2n 1 (for large N ): (16.87)
We rst consider nonrecursive PCCC's with (n; 1; m) systematic feedforward constituent encoders. In
this case, wmin = 1, since only zeros are required to terminate an information sequence for a feedforward
encoder. Also, as noted in the previous section, hmax = w for feedforward encoders. Finally, since for
any (n; 1; m) systematic feedforward encoder A(ww)(Z ) = [A(1)
1 (Z )]w (see Problem 16.16), we obtain from
(16.86) for nonrecursive PCCC's with (n; 1; m) systematic feedforward constituent encoders and large
N
X N (w 1) h i2w
Pb (E )   W w A(1)
1 ( Z ) : (16.88)
(1wN ) (w 1)! W =Z =e REb =N0

Remarks:
 From (16.88), we see that even in the best case (w = 1), the Pb (E ) does not decrease with
increasing N , i.e., no interleaver gain is possible with feedforward encoders.
 The same analysis applies to PCBC's with (hn; hk) block constituent codes and an interleaver
size of N = hk.
We now consider recursive PCCC's with (n; 1; m) systematic feedback constituent encoders. In this
case, wmin = 2, since a weight one input sequence cannot terminate the encoder, but there always exists
a weight two input sequence, of degree no greater than 2m 1, that does terminate the encoder (see
Problem 16.17). Also, since each single error event in a multiple error event requires at least two input
\1's" to terminate, hmax = bw=2c for feedback encoders. We now note that for the term involving the
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 63
block length N in (16.86)

N (2hmax w 1) = N 2if w is odd (16.89)
N 1if w is even.
Thus, for large N , the terms with odd w are negligible compared to the terms with even w, and in
(16.86) we need only consider terms of the type A(2ww)(Z ). Further, since for any (n; 1; m) systematic
h iw
(w) (1)
feedback encoder A2w (Z ) = A2 (Z ) (see Problem 16.16), we must examine the properties of the
term A(1)
2 (Z ), i.e., the parity weight enumerator for single error events with input weight two. We begin
by considering an example.
Example 16.12
Consider the IRWEF state diagram of the (2,1,2) systematic feedback encoder of Examples 16.6 and 16.7
shown in Figure 16.10. In this case, it is clear that the shortest single error event generated by a weight
two input follows the state sequence S0 ! S1 ! S3 ! S2 ! S0 and has parity weight 4. All other
single error events with input weight two are generated by adding one or more cycles around the loop
S2 ! S1 ! S3 ! S2 to the above shortest state sequence. Since each of these cycles adds parity weight
2,
Z4
A(1) 4 6 8
2 (Z ) = Z + Z + Z +


= 1 Z 2 : (16.90)


In general, for any (n,1,m) systematic feedback encoder, the input sequence u = 1000


will produce
a cycle in the weight enumerating graph with input weight zero which starts in the state S1 = (10


0),
arrives in the state S2m = (0


01) after at most 2m 2 steps, and then returns to the state S1 =
1

(10


0) in one step (see Problem 16.18). Let zc be the parity weight generated by this cycle. Also,
it is easy to show that a \1" input from the state S2m = (0


01) terminates the encoder (see
1

Problem 16.19). Further, as in Example 16.12, if each of the n 1 numerator polynomials has the form
1 +


+ Dm, i.e., they are monic polynomials of degree m, then the zero input branch connecting the
state S2m = (0


01) to the state S1 = (10


0) has zero parity weight. In this case, if zt is the
1
64 CHAPTER 16. TURBO CODING

total parity weight of the two input weight 1 branches which connect the initial and nal states to the
zero input weight cycle, the total parity weight associated with any input weight 2 single error event is
jzc + zt , where j is the number of zero input weight cycles included in the error event, and

zmin = zc + zt (16.91)
is the minimum parity weight associated with a weight 2 input. (If the above condition on the numerator
polynomials is not satis ed, the branch connecting state S2m = (0


01) to state S1 = (10


0) may
1

have nonzero parity weight. In this case, since this branch is not included in the path that achieves
minimum parity weight, its parity weight must be subtracted from the right side of (16.91).) Thus the
parity weight enumerator is given by
Z zmin
A(1) (2z
2 (Z ) = Z min + Z min
z zt ) + Z (3zmin 2zt) +


=
zt ) : (16.92)
1 Z (zmin

Example 16.12 (Continued)

For the encoder shown in Figure 16.10, zc = 2; zt = 2, and zmin = 4. 

Using (16.86), we can now state the approximate bit error probability bound for large N for recursive
PCCC's with (n; 1; m) systematic feedback constituent encoders as follows:
 2w
Pb (E ) 
X
2w 2w  N 1  W 2w  Z zmin
(1wbN=2c) w 1
Z (zmin zt ) W =Z =e REb =N0
   (2+2z ) w
=
X
2w 2w N 1   Y min
: (16.93)
(1wbN=2c) w 1 Y (zmin zt)2w Y =e REb=N0
Remarks:
 Unlike (16.88) for feedforward encoders, (16.93) contains the term N 1, indicating the presence
of interleaver gain for feedback encoders.
 The exponential behavior of the most signi cant term in the bound is determined by zmin, the
minimum parity weight associated with weight 2 input sequences. Thus, in order to guarantee
good performance for large N , zmin should be as large as possible.
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 65
 Since zmin depends primarily on zc, the parity weight in the zero input weight cycle should be as
large as possible. This implies that the cycle length should be as large as possible, which means
that a primitive denominator polynomial should be chosen to achieve a large zmin (see Problem
16.20).
Since the weight 2+2zmin is the coeÆcient of REb=N0 in the most signi cant term of the bound, we
de ne the e ective minimum free distance of a rate R = 1= (2n 1) PCCC with an (n; 1; m) constituent
encoder as
deff = 2 + 2zmin : (16.94)

Example 16.12 (Continued)

For the rate R = 1=3 PCCC based on the encoder shown in Figure 16.10, deff = 10. 

Example 16.13
Consider the rate R = 1=3 recursive PCCC's generated by the following (2,1,3) systematic feedback
encoders:
h i
Gp (D) = 1 1+1+ D+D +D
D+D
2
3
3
(16.95a)
and
h i
Gnp (D) = 1 1+D+D3 (16.95b)
1+D+D2+D3 ;
where Gp has a primitive feedback polynomial and Gnp has a nonprimitive feedback polynomial. The
rate R = 1=2 convolutional codes generated by these two encoders are identical. Both have free dis-
tance dfree = 6. For the primitive encoder, the shortest terminating weight two input sequence is
u(D) = 1+D7 , which generates the parity sequence v (1) (D) = 1+D2 +D3 +D4 +D6 +D7. Thus zmin = 6
and deff = 14 for the primitive PCCC. For the nonprimitive encoder, the shortest terminating weight
two input sequence is u(D) = 1+ D4, which generates the parity sequence v(1)(D) = 1+ D2 + D3 + D4,
and for the nonprimitive PCCC zmin = 4 and deff = 10. 
66 CHAPTER 16. TURBO CODING

Remarks:
 As expected, deff is larger for the PCCC based on a primitive feedback polynomial.
 Since the input sequence u(D) = 1+ D2 + D3 + D4 generates the parity sequence v(1)(D) = 1+ D4
for the primitive encoder, the actual minimum free distance of its corresponding PCCC, assuming
a uniform interleaver, is dfree = 8, smaller than for the nonprimitive encoder. However, since this
input sequence has weight 4 and produces a single error event, the multiplicity associated with
the weight 8 codewords decreases with N 2, and thus the probability that a particular interleaver
will result in a weight 8 codeword in the corresponding PCCC is very small for large N .

Theorem 16.1 [5] For an (n,1,m) systematic feedback encoder with generator matrix
h i
n1 (D) nn 1 (D)
G(D) = 1 d(D)


d(D) (16.96)

such that the denominator polynomial d(D) is a primitive polynomial of degree m and the numerator
polynomials ni (D) 6= d(D); 1  i  n 1, the minimum parity weight associated with weight 2 input
sequences satis es

zmin = s(2m 1 + 2) + (n s 1) 2m 1
 (n 1)(2m 1 + 2); (16.97)

where s; 0  s  n 1, is the number of numerator polynomials that are monic and of degree m.

Proof: We begin by noting that, since d(D) is primitive, it is the generator polynomial of a cyclic
(2m 1; 2m m 1) Hamming code. Also, the minimum degree binomial which has d(D) as a factor
is D2m 1 + 1, and the quotient polynomial q(D) = (D2m 1 + 1)=d(D) is the generator polynomial of a
cyclic (2m 1; m) maximum length code.
First, we consider numerator polynomials n(D) such that deg[n(D)] < m. In this case, the weight
two input sequence D2m 1 +1 generates the parity sequence n(D)q(D), which has weight exactly 2m 1,
since it is a nonzero codeword in a (2m 1; m) maximum length code. Also, if n(D) has degree m but
is not monic, it can be expressed as n(D) = Dr v(D); 1  r  m, where v(D) is monic and has degree
less than m. In this case, the parity sequence n(D)q(D) = Dr v(D)q(D) again has weight 2m 1.
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 67
Now consider numerator polynomials n(D) that are monic and of degree m, i.e., n(D) = 1+


+ Dm.
In this case, we can write
n(D) = Dn(1) (D) + n(2) (D); (16.98)
where n(1)(D) = Dm 1 and n(2)(D) = n(D) Dm both have degree less than m. Thus c(1)(D) =4

n(2)(D)q(D) are both nonzero codewords in a (2m 1; m) maximum length
n(1) (D)q (D) and c(2) (D) =
code and have weight 2m 1. We note that c(1)(D) has the maximum degree of 2m 2 for any code
polynomial and m 1 leading zeros, i.e., c(1)(D) = Dm 1 + c(1) (1) 2m 3 + D2m 2 , and
m D +


+ c2m 3 D
m

that c(2)(D) has a nonzero constant term. We can now write the cyclic shift X (1)(D) of c(1)(D) as
X (1)(D) = 1 + Dm + c(1) m+1 +


+ c(1) D 2m 2
mD 2m 3
= Dc(1) (D) + 1 + D2 1 : (16.99)
m

Since X (1)(D) and c(2)(D) are both codewords, X (1)(D)+ c(2)(D) is also a codeword, has a zero constant
term, and has weight 2m 1. Now the parity sequence can be written as
n(D)q (D) = Dc(1)(D) + c(2)(D)
= X (1)(D) + c(2)(D) + 1 + D2m 1: (16.100)
Since X (1)(D) + c(2)(D) has weight 2m 1, a zero constant term, and degree at most 2m 2, the parity
sequence n(D)q(D) has weight exactly 2m 1 + 2. (16.97) now follows by de ning s as the number of
numerator polynomials that are monic and of degree m. 

Remarks:
 In Problem 16.21, it is shown that the upper bound in (16.97) also holds for nonprimitive feedback
polynomials. Thus, since for primitive feedback polynomials Theorem 16.1 guarantees that the
upper bound can be achieved by properly selecting the numerator polynomials, we can achieve the
best performance for large N , i.e., the largest e ective free distance deff , by choosing constituent
codes with primitive feedback polynomials and monic numerator polynomials of degree m.
 Selecting constituent codes that result in large values of deff guarantees good high SNR perfor-
mance. For lower SNR's, however, codeword multiplicities and additional terms in the weight
spectrum must be considered.
68 CHAPTER 16. TURBO CODING

In searching for recursive systematic constituent codes that o er good performance as PCCC's over
the entire SNR range, it is important to maximize zmin, and thus the e ective free distance deff of
the PCCC, and to minimize the multiplicities of the minimum weight codewords produced by each of
the lowest weight input sequences, i.e., input weights w = 2; 3; 4;


. We denote the minimum weight
of codewords produced by weight w input sequences in a constituent code by dw and the number of
such codewords by Aw;dw , the coeÆcient of the lowest order term in the conditional IOWEF Aw (X ).
(Note that in this case deff = 2d2 2.) In Tables 16.7(a) and 16.7(b), we present lists of optimum
weight spectrum recursive systematic (n; k; K ) convolutional codes for code rates R = 1=4, 1=3, 1=2,
2=3, and 3=4. The optimization was performed by rst maximizing dw and then minimizing Aw;dw ,
successively, for increasing values of w. For each code, we list the rate k=n, the total encoder memory
K , the (right justi ed) octal generator sequences g0 , g1 , g2 , and g3 in Table 16.7(a) for rates 1=4, 1=3,
and 1=2 and the (right justi ed) octal parity check sequences h0, h1, h2, and h3 in Table 16.7(b) for
rates 2=3 and 3=4, and the pair (dw ; Aw;dw ) for w = 2, 3, 4, and 5. In Table 16.7(a), g0 represents the
denominator (feedback) polynomial of a systematic feedback encoder generator matrix G (D) and g1,
g2 , and g3 represent the numerator polynomials. For example, for the rate 1=3, 8-state (3; 1; 3) code in
Table 16.7(a), g0 = 13, g1 = 15, g2 = 17 and we have
h i h i
g1 (D) g2 (D) 1+D2 +D33 1+D+D2+3D3 :
G (D) = 1 g0 (D) g0 (D) = 1 1+D+D 1+D+D (16.101)
Similarly, in Table 16.7(b), h0 represents the denominator (feedback) polynomial of a systematic feed-
back encoder parity check matrix H(D) and h1, h2, and h3 represent the numerator polynomials. For
example, for the rate 2=3, 8-state (3; 2; 3) code in Table 16.7(b), h0 = 13, h1 = 17, h2 = 15 and we have
h i h i
h2 (D) h1 (D) D3 +D2 +1 D3 +D2 +D+1
H(D) = h0 (D) h0 (D) 1 = D3 +D+1 D3 +D+1 1 : (16.102)
The bold entries in the table denote the minimum free distance dfree of the code, the symbol 1 indicates
that there are no codewords corresponding to that input weight (only single error events are considered),
and the  entry indicates the existence of a code with the same d2 and a larger dfree.
Remarks:
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 69

Table 16.7: Recursive Systematic Convolutional Codes with Optimum Weight Spectrum [6]
(a) Low Rate Codes
k=n K g3 g2 g1 g0 d2 ; A2;d2 d3 ; A3;d3 d4 ; A4;d4 d5 ; A5;d5
1/4 1 2 3 1 3 6,1 1 1 1
1/4 2 6 7 5 7 10,1 10,1 14,2 16,1
1/4 3 11 17 15 13 20,1 12,1 14,1 18,2
1/4 4 27 37 35 23 32,1 16,1 14,1 18,3
1/4 5 71 45 51 67 56,1 23,1 20,3 15,1
1/3 1 - 2 3 3 5,1 1 1 1
1/3 2 - 5 7 7 8,1 8,1 10,1 12,1
1/3 3 - 17 15 13 14,1 10,2 10,1 14,6
1/3 4 - 33 37 23 22,1 12,1 10,1 12,1
1/3 5 - 45 51 67 38,1 17,2 16,6 11,1
1/3 6 - 101 131 163 70,1 23,1 14,1 11,1
1/2 1 - - 2 3 3,1 1 1 1
1/2 2 - - 5 7 6,1 5,1 6,1 7,1
1/2 3 - - 17 13 8,1 7,3 6,1 9,9
1/2 4 - - 37 23 12,1 8,3 6,1 10,14
1/2 5 - - 45 67 20,1 10,2 8,1 8,2
1/2 6 - - 101 147 36,1 13,1 8,2 7,1
1/2 6* - - 115 147 36,1 13,3 10,2 9,2

(b) High Rate Codes

k=n K h3 h2 h1 h0 d2 ; A2;d2 d3 ; A3;d3 d4 ; A4;d4 d5 ; A5;d5
2/3 1 - 2 2 3 2,1 1 6,4 1
2/3 2 - 6 5 7 4,3 3,1 4,1 5,1
2/3 3 - 15 17 13 5,1 4,1 5,1 6,6
2/3 4 - 27 35 23 8,2 5,3 6,9 7,29
2/3 5 - 73 51 75 12,2 6,3 6,3 6,5
2/3 6 - 135 133 147 20,2 8,4 6,3 6,2
3/4 1 2 2 2 3 2,3 1 5,6 1
3/4 2 6 2 7 5 3,3 3,3 4,7 5,15
3/4 3 17 13 15 11 4,1 4,3 4,1 5,5
3/4 4 33 31 27 25 6,2 4,1 5,8 5,4
3/4 5 63 51 71 45 9,2 5,2 5,5 5,2
70 CHAPTER 16. TURBO CODING

 Many of the codes in Table 16.7 are not optimum dfree codes, as can be seen by comparison to the
code lists in Chapter 12. However, the codes in Table 16.7 are optimum in the sense of minimizing
the number of low weight codewords corresponding to low weight input sequences.
 The constituent codes in Table 16.7 can be used to produce a variety of PCCC code rates, as
seen in Table 16.8. Other code rates can be achieved by puncturing, by using constituent codes
of di erent rates, or by employing multiple interleavers (see Figure 16.2).
Table 16.8: PCCC Code Rates for Di erent Constituent Code Rates
Constituent PCCC
Code Rate Code Rate
1/4 1/7
1/3 1/5
1/2 1/3
2/3 2/4
3/4 3/5

 It is important to note that in examining the weight spectrum of PCCC's, the values of interest
are given by the lowest order coeÆcients in the conditional IOWEF's Aw (X ) for small values of
w. This contrasts with the case for normal convolutional codes and block codes, where the low
order coeÆcients in the standard WEF A(X ) are the values of interest. This is due to the fact
that the interleaver in PCCC's has a di erent e ect on input sequences of di erent weight, thus
accentuating the importance of the codewords produced by low weight input sequences.
We now summarize the major points to remember in designing good PCCC's:
 The bit multiplicities in a PCCC include a factor of N 1 w , where N is the interleaver size, w is the
weight of an input sequence that produces a terminated codeword, and w  wmin, the minimum
weight input sequence that produces a terminated codeword.
 All feedback encoders have wmin = 2, resulting in an interleaver gain factor of 1/N for the
associated PCCC's. On the other hand, wmin = 1 for feedforward encoders and for repeated block
codes, so there is no interleaver gain in these cases, although some spectral thinning does occur.
 For large values of N and high SNR's, the performance of a PCCC is optimized by maximizing its
e ective free distance deff = 2 + 2zmin, where zmin is the minimum parity weight corresponding
to a weight 2 input sequence in the constituent code.
 For recursive systematic (n,1,m) convolutional codes, it is possible to achieve zmin = (n 1)(2m 1 +
2) by choosing a primitive denominator polynomial and monic numerator polynomials of degree
m.
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 71
 For lower SNR's, good performance is achieved by optimizing the low input weight terms in the
IOWEF of the constituent code.
 These code design rules are based on an average (uniform interleaver) analysis and performance
can vary in a particular case depending on the choice of an interleaver. Thus it is important to
support a design with appropriate computer simulations.
Finally, although pseudorandom interleavers such as the ones discussed earlier in this chapter give
very good performance on the average, it is possible to design the interleaver to improve the value of
deff and thus the high SNR performance of the code. A codeword of weight deff is produced when a
weight 2 input sequence that generates the weight zmin parity sequence in the rst constituent code is
interleaved in such a way that the interleaved input sequence maintains the same spacing between \1's",
thus also generating the weight zmin parity sequence in the second constituent code. If we refer to a
weight 2 input sequence that generates the weight zmin parity sequence in the rst constituent code as
a \bad" input sequence, interleaver designs that \break up" all bad input sequences, i.e., they change
the spacing between \1's", result in a larger value of deff . However, putting too much structure in the
interleaver can result in low weight codewords due to higher weight input sequences, i.e., the ability
of a pseudorandom interleaver to reduce the in uence of higher weight input sequences is mitigated
when structure is added. Interleaver designs which attempt to improve deff by breaking up the \bad"
input sequences, while also maintaining most of the characteristics of pseudorandom interleavers, are
referred to as semirandom interleavers [7]. For example, in an S -random interleaver, integers i in the
range 1  i  N are chosen randomly (without replacement) to denote the interleaved positions of
successive bits in the input sequence. If a selected integer is within a distance S of any of the S
previous selections, it is rejected and another integer is selected. If S is chosen equal to the cycle length
of the denominator polynomial, this guarantees that all \bad" weight 2 input sequences are broken
up. However, the algorithm is not guaranteed to successfully assign all N integers, so that the nal
permutation may have to be further altered to satisfy the above condition.
The above rules for designing good PCCC's implicitly assume the use of (optimum) maximum
likelihood decoding. Iterative decoding, on the other hand, is suboptimum, and di erent code design
72 CHAPTER 16. TURBO CODING

rules may apply in this case. At moderate to high SNR's, iterative decoding is essentially maximum
likelihood, and the above design rules will give the best codes. At low SNR's, below the channel cuto
rate, however, the dynamics of iterative decoding are more important in determining the performance
of PCCC's than the weight spectrum of the code.
In order to understand the dynamics of iterative decoding, it is helpful to introduce the extrinsic
information transfer chart, or EXIT chart [8], based on the techniques of density evolution [9, 10] and
threshold analysis [11]. These techniques relate a parameter of the input to a constituent decoder,
either the signal-to-noise ratio of the a priori L-value La(uk ) or the mutual information between an
information bit uk and its a priori L-value La (uk ), to a parameter of the decoder output, the signal-
to-noise ratio of the a posteriori extrinsic L-value Le(uk ) or the mutual information between uk and
its a posteriori extrinsic L-value Le(uk ), respectively. Following the mutual information approach, the
a priori L-value inputs to a constituent decoder are rst modeled as independent Gaussian random
variables with variance a2 and mean a = a2=2, where the sign of a depends on the value of uk .
(This Gaussian approximation is based on two facts: (1) the input channel L-values to a constituent
decoder (the terms Lcrk(0) in (16.17)) are independent Gaussian random variables with variance 2Lc and
mean Lc and (2) extensive simulations of the a posteriori extrinsic L-values Le(uk ) for a constituent
decoder with very large block lengths support this assumption [12].) Thus we can express the mutual
information Ia[uk ; La(uk )] between an information bit uk and its a priori L-value La(uk ) as (see [13])
1 X Z +1 2PLa (juk )
Ia [uk ; La (uk )] = PLa ( juk )log2 d; (16.103a)
2 uk = 1;+1 1 PLa ( juk = 1) + PLa ( juk = +1)

where the conditional probability density function PLa (juk ) of La(uk ) is given by
PLa ( juk ) = p 1 e ( )2 : (16.103b)
a2
uk = a2
2a
2

Next, histograms of the conditional pdf's PLe (juk = 1) and PLe (juk = +1) associated with the a
posteriori extrinsic L-values Le (uk ) are determined by simulating the BCJR algorithm with independent
Gaussian distributed a priori L-value inputs for a particular constituent code and a large block length
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 73

1
Eb/N0 = 0.25 dB
Eb/N0 = 0.50 dB
0.9
Eb/N0 = 0.75 dB
Eb/N0 = 1.00 dB
0.8 y=x
Mutual Information Ie at output of decoder

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Mutual Information Ia at input of decoder

Figure 16.16: Decoder Input-Output Transfer Curves

74 CHAPTER 16. TURBO CODING

N . We then determine the mutual information Ie [uk ; Le (uk )] between the information bit uk and its a
posteriori extrinsic L-value Le (uk ) using (16.103a).
A decoder input-output transfer curve can now generated by running a series of simulations of the
BCJR algorithm for one constituent code and di erent values of Ia[uk ; La(uk )] (completely determined
by a2), for a xed channel signal-to-noise ratio Eb=N0, and plotting the resulting values of Ie[uk ; Le(uk )].
Note that each point on the curve is obtained by simulating the BCJR algorithm just once for a
constituent code and does not require iterative decoding. A set of transfer curves is obtained by
repeating the above procedure for di erent values of the channel SNR, Eb=N0. An example is shown
in Figure 16.16 for the original turbo code of Figure 16.1(b), with alternate parity bits punctured to
produce a code rate of R = 1=2, and various values of the channel SNR. Note that a value of Ia = 0
correspond to the rst iteration of decoding when all a priori L-values are equal to 0 and that more
positive values of Ia correspond to more reliable a priori estimates. A key observation is that if a
transfer curve cuts below the straight line representing Ia = Ie, this corresponds to a situation where
an iteration results in an Ie below the Ia at the decoder input, i.e., the extrinsic estimates are getting
less reliable. In this case iterative decoding fails to converge to a reliable solution, resulting in a high
probability of decoding errors. On the other hand, if the transfer curve stays above the Ia = Ie line,
the extrinsic estimates are getting more reliable, and iterative decoding converges to a reliable solution.
We now de ne the SNR threshold of iterative decoding for a particular constituent code as the smallest
value of the channel SNR, Eb=N0, for which the transfer curve stays above the Ia = Ie line. For example
in Figure 16.16, we see that the iterative decoding threshold for the original rate R=1/2 turbo code
is approximately 0.5 dB. This threshold corresponds to the the value of the channel SNR, Eb=N0, at
which the bit error probability curve of a PCCC begins its characteristic sharp drop, referred to as the
waterfall region of the BER curve (see Figure 16.3).
An EXIT chart for a particular channel SNR Eb=N0 can be formed by plotting a transfer curve and
its mirror image (about the line Ia = Ie) on the same set of axes, as shown in Figure 16.17 for Eb=N0
= 1 dB. The two curves can be interpreted as representing the input-output transfer characteristics of
16.4. DESIGN OF PARALLEL CONCATENATED TURBO CODES 75

0.9
Mutual Information Ie (Ia) at output (input) of Decoder 1 (2)

Eb/N0= 1 dB
0.8

0.7

0.6 Decoder 1

0.5

0.4 Decoder 2

0.3

0.2

0.1

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Mutual Information Ia (Ie) at input (output) of Decoder 1 (2)

Figure 16.17: An Extrinsic Information Transfer (EXIT) Chart

76 CHAPTER 16. TURBO CODING

the two constituent decoders in an iterative turbo decoder. The chart can then be used to trace the
trajectory of iterative decoding as follows. For a given Eb=N0, assume initially Ia = 0, corresponding to
the rst iteration of Decoder 1, and determine the resulting Ie (vertically) using the transfer curve for
Decoder 1. Since the a posteriori extrinsic L-value of Decoder 1 becomes the a priori L-value of Decoder
2, the value of Ie from Decoder 1 becomes Ia for the rst iteration of Decoder 2, and the resulting Ie for
Decoder 2 is determined (horizontally) using the transfer curve for Decoder 2. This procedure is repeated
and traces the \staircase" trajectory of iterative decoding, as shown in Figure 16.17. If a tunnel exists
between the two transfer curves, as shown in Figure 16.17, iterative decoding converges. If no tunnel
exists, i.e. the two transfer curves cross each other, a decoder xed point exists and iterative decoding
does not converge. As the channel SNR is lowered, the two transfer curves come closer together, and
the smallest SNR for which the two curves do not touch is the threshold.
The EXIT chart is particularly useful for visualizing the dynamics of iterative decoding. For example,
we see from Figures 16.16 and 16.17 that as the channel SNR is lowered, the tunnel narrows and more
iterations are needed before the mutual information becomes large enough for the decoder to converge to
a reliable solution. We also note that the EXIT chart technique can be used to determine the threshold
of a turbo code with asymmetric constituent codes simply by plotting the transfer curves of the two
(di erent) constituent codes, which will no longer be the mirror image of one another.
The above discussion indicates that determining the transfer curves and EXIT charts of various
constituent codes is a useful tool in designing PCCC's with good low SNR performance, i.e., good
performance in the waterfall region of the BER curve. The design rules to follow in a particular case,
i.e., the high SNR uniform interleaver analysis or the low SNR threshold analysis, depend upon the
application. In communication systems designed to achieve very low bit error probabilities at moderate
SNR's, the uniform interleaver analysis is preferred, whereas in systems designed to achieve moderate
BER's at very low SNR's close to capacity, the threshold analysis is best.
The usefulness of the SNR threshold analysis discussed above depends on two factors: the accuracies
of the Gaussian approximation for the a priori L-values and the simulated estimates of the conditional
16.5. ITERATIVE DECODING OF TURBO CODES 77
pdf's for Le. Both these factors depend on the length of the input block N used in the simulations.
Experience with threshold analysis has shown that, in general, the constituent codes with the lowest
SNR thresholds have denominator polynomials with short cycle lengths, in contrast to the long cycle
lengths needed to achieve large values of deff . Asymmetric code designs for PCCC's attempt to exploit
these dual requirements by choosing di erent constituent codes, one with a short cycle length and the
other with a long cycle length [1].

16.5 Iterative Decoding of Turbo Codes

A brief summary of turbo decoding was given in Section 16.1. As noted there, the best performance is
achieved when the individual SISO decoders make use of the MAP, or APP, decoding algorithm. The
details of this algorithm have been presented earlier, both as an optimum bit-by-bit decoding algorithm
for convolutional codes in Chapter 12 and as an optimum method of trellis-based decoding of block codes
in Chapter 14. Thus in this section we will emphasize the use of this algorithm as an iterative decoder
of PCC's. Other SISO decoders that are simpler to implement, such as the SOVA or Max-Log-MAP
algorithms, can also be used to iteratively decode PCC's, but their performance is not as good.
The basic idea behind iterative decoding is to divide the decoding of a PCC, for which optimum de-
coding would be prohibitively complex, into a suboptimum iterative process involving only the relatively
simple APP decoders for the constituent codes. After each iteration the extrinsic L-values, representing
reliability information about the bits to be decoded, are passed from one decoder to the other, insuring
that very little information is lost relative to optimal decoding. The basic structure of an iterative turbo
decoder was shown in Figure 16.6. In the rst iteration of decoder 1, the inputs are the channel L-values
Lc rk(0) and Lc rk(1) for each received symbol (information and parity) of constituent code 1 and the a
priori L-values La (uk ) of each information bit, which all equal 0 for equally likely inputs. In the rst
iteration of decoder 2, the inputs are the channel L-values Lcrk(0) and Lcrk(2) for each received symbol
(information and parity) of constituent code 2 and the extrinsic a posteriori L-values L(1) e (uk ) of each

information bit received from the output of decoder 1. These extrinsic L-values, along with the L-values
78 CHAPTER 16. TURBO CODING

of the received information symbols, must be interleaved according to the same pattern used at the en-
coder before entering decoder 2. In all subsequent iterations of decoder 1, the a priori L-values L(1)
a (uk )

are replaced by the extrinsic a posteriori L-values L(2)

e (uk ) received, after proper deinterleaving, from

the output of decoder 2, and all subsequent iterations of decoder 2 are identical to the rst iteration,
except that the a priori inputs will be di erent. Finally, after a suÆcient number of iterations, the
decoded output can be derived from the a posteriori L-values of the information bits L(2)(uk ), properly
deinterleaved, at the output of decoder 2. Positive L-values are decoded as \+1" and negative L-values
as \-1".
We now present a simple example, using the Log-MAP algorithm rst presented in Chapter 12, to
illustrate the principle of iterative decoding.
Example 16.14
Consider the rate 1=3 PCCC formed by using the 2-state (2; 1; 1) RSCC with generator matrix
 1 
G(D) = 1 1+D (16.104)

as the constituent code. A block diagram of the encoder is shown in Figure 16.18(a). Also consider an
input sequence of length N = 4, including one termination bit, along with a 2  2 block (row-column)
interleaver, resulting in a (12; 3) PCCC with overall rate R = 1=4. (Recall that the termination bit
is not an information bit.) The length N = 4 decoding trellis for the constituent code is shown in
Figure 16.18(b), where the branches are labeled using the mapping 0 ! +1 and 1 ! 1. The input
block is given by the vector u = [u0; u1; u2; u3], the interleaved input block is u0 = [u00; u01; u02; u03] =
h i
[u0; u2; u1; u3], the parity vector in the rst constituent code is given by v(1) = v0(1); v1(1); v2(1); v3(1)
h i
and the parity vector in the second constituent code is v(2) = v0(2); v1(2); v2(2); v3(2) . Recalling that the
mapping 0 ! +1 and 1 ! 1 implies that mod-2 addition is replaced by real multiplication, we can
express the parity bits generated by the two constituent codes as

vk(1) = uk  vk(1)1 ; k = 0; 1; 2; 3; (16.105a)

16.5. ITERATIVE DECODING OF TURBO CODES 79
and
vk(2) = u0k  vk(2)1 ; k = 0; 1; 2; 3; (16.105b)
where v(1)1 = v(2)1 = +1 is required to make (16.105) consistent with the condition that the encoders start
in all-zero state S0 = 0. The 12 transmitted bits can be represented in a rectangular array, as shown in
Figure 16.19(a), where the input vector u determines the parity vector v(1) in the rst two rows and the
interleaved input vector u0 determines the parity vector v(2) in the rst two columns. For purposes of
illustration, we assume the particular bit values shown in Figure 16.19(b), where the parity bits satisfy
(16.105). We also assume a channel SNR of Eb=N0 = 1 (0dB), so that the received channel L-values
h i
corresponding to the received vector r = r0(0); r0(1); r0(2); r1(0); r1(1); r1(2); r2(0); r2(1); r2(2); r3(0); r3(1); r3(2) are
given by
   
E E
Lc rk(i) = 4 s rk(i) = 4 R b rk(i) = rk(i) ; k = 0; 1; 2; 3; i = 0; 1; 2: (16.106)
N0 N0
Again for the purposes of illustration, a set of particular received channel L-values is given in Fig-
ure 16.19(c).
In the rst iteration of Decoder 1 (row decoding), the Log-MAP algorithm is applied to the trellis of
the 2-state (2; 1; 1) code shown in Figure 16.18(b) to compute the a posteriori L-values L(1)(uk ) for each
of the four input bits and the corresponding extrinsic a posteriori L-values L(1)e (uk ) to pass to Decoder

2 (the column decoder). Similarly, in the rst iteration of Decoder 2, the Log-MAP algorithm uses the
extrinsic a posteriori L-values L(1) (2)
e (uk ) received from Decoder 1 as the a priori L-values, La (uk ) to

compute the a posteriori L-values L(2)(uk ) for each of the four input bits and the corresponding extrinsic
a posteriori L-values L(2)e (uk ) to pass back to Decoder 1, and further decoding proceeds iteratively in

this fashion. 

Before returning to our speci c example, we now develop the general expressions for the a poste-
riori L-values L(uk ) and the extrinsic a posteriori L-values Le (uk ) of the (2; 1; 1) constituent code in
Figure 16.18(b) using the Log-MAP algorithm. To simplify notation, we will denote an input bit as uk ,
80 CHAPTER 16. TURBO CODING

u (0)
v

(1)
v

π
(2)
v
(a)

S1 +1/+1,-1 S +1/+1,-1 S1
1
-1

-1

-1
/-1

/-1

/-1
,+

,+

,+
1

1
,-1

,-1

,-1
/-1

/-1

/-1
-1

-1

-1

S0 S0 S0 S0 S0
+1/+1,+1 +1/+1,+1 +1/+1,+1 +1/+1,+1
(b)

Figure 16.18: (a) A 2-state, Rate 1/3 Turbo Encoder, and (b) the Decoding Trellis for the (2,1,1)
Constituent Code with N = 4
16.5. ITERATIVE DECODING OF TURBO CODES 81
u0 u1 v(1)
0 v1
(1)
+1 -1 +1 -1 -0.8 -1.0 -0.1 +0.5

u2 u3 v(1)
2 v3
(1)
+1 -1 -1 +1 +1.8 -1.6 -1.1 +1.6

v(2)
0 v2
(2)
+1 -1 +1.2 -0.2

(2) (2)
v1 v3 +1 +1 -1.2 +1.1

(12,3) PCCC Coded values Received L-values

(a) (b) (c)

+0.32 +0.38 +0.88 +0.69 +0.40 +0.07

-0.77 -0.47 -0.23 +0.04 +0.80 -2.03

Soft output L-values after the

Extrinsic L-values after Extrinsic L-values after
first row and
first row decoding first column decoding
column decoding
(d) (e) (f)

+0.01 +0.01 +0.98 +0.81 +0.19 -0.18

-0.43 -0.77 -0.07 +0.21 +1.30 -2.16

Approximate extrinsic Approximate extrinsic Approximate soft output L-values

L-values after L-values after after the first row and
first row decoding first column decoding column decoding
(g) (h) (i)

+0.9 +0.9 +0.8 +0.8 +0.9 +0.7

-1.4 +0.3 -1.1 -0.1 -0.7 -1.4

Soft output L-values after the

Extrinsic L-values after Extrinsic L-values after
second row and
second row decoding second column decoding
column decoding
(j) (k) (l)

+0.1 +0.1 +0.9 +0.7 +0.2 -0.2

-0.6 -0.5 -0.3 +0.5 +0.9 -1.6

Approximate extrinsic Approximate extrinsic Approximate soft output L-values

L-values after L-values after after the second row and
second row decoding second column decoding column decoding
(m) (n) (o)

Figure 16.19: Iterative Decoding Example for a (12,3) PCCC

82 CHAPTER 16. TURBO CODING

a parity bit as pk , and the received vector as r = [r0; r1; r2; r3], where rk = [ruk ; rpk ], k = 0; 1; 2; 3, ruk
is the received symbol corresponding to the transmitted input bit uk , and rpk is the received symbol
corresponding to the transmitted parity bit pk . An input bit a posteriori L-value is now given by
P (uk = +1 j r)
L(uk ) = ln
P
P
(uk = 1 j r)
s0 ;s;uk =+1) p(s ; s; r)
0
= ln 0 P
( ; (16.107)
p(s0 ; s; r)
(s ;s;uk = 1)
where s0 represents a state at time k 1 (denoted s0 2 Pk 1), s represents a state at time k (denoted
P
s 2 k ), and the sums are over all state pairs (s0 ; s) for which uk = +1 or 1, respectively. Note that
every branch in the trellis connecting a state at time k 1 to a state at time k is included in one of the
sums in (16.107). We now introduce the notation rj<k to represent the portion of the received vector r
prior to time k and rj>k to represent the portion of r after time k. Using the fact that the probability
of a received branch rk , or a sequence of received branches rk , rk+1,


, depends only on the state at
time k 1, the joint probabilities p(s0; s; r) can be written as




p s0 ; s; r = p s0 ; rj<k  p s; rk j s0  p (rj>k j s)




= p s0 ; rj<k  p s j s0  p rk j s0 ; s  p (rj>k j s)



= k 1 s0  k s0 ; s  k (s) ; (16.108)
where k 1 (s0) = p (s0; rj<k ), k (s0; s) = p (s j s0)  p (rk j s0; s), and k (s) = p (rj>k j s) are the familiar
's, 's, and 's of the MAP algorithm (see Chapters 12 and 14). The three terms in (16.108), k 1 (s0),
k (s ; s), and k (s) represent the in uence of the received sequence before time k , at time k , and after
0

time k on the estimate of the input bit at time k, respectively. Since, given a state s0 at time k 1, there
are two possible states s at time k and their probabilities are given by P (uk = 1), the expression for
k (s ; s), when there is a branch in the trellis connecting state s to state s, can be rewritten as
0 0

k s0 ; s = p s j s0
 p rk j s0 ; s


= P (uk = 1)  p (rk j vk ) ; (16.109)
 
where vk = vk(0) ; vk(1) = (uk ; pk ) is the label on the trellis corresponding to the transition from state
16.5. ITERATIVE DECODING OF TURBO CODES 83
s0 to state s. We now write P (uk = 1) in terms of a priori L-values as follows:
[ P (uk = +1) =P (uk = 1)]1
P (uk = 1) =
1 + [P (uk = +1) = P (uk = 1)]1
La (uk )
= 1 +e eLa(uk)
f La (uk )=2g
= 1e + e La(uk)  euk La(uk )=2
= Ak  euk La(uk )=2; (16.110)
where the parameter Ak depends only on the a priori L-value La (uk ) and is independent of the actual
value of uk (see Problem 16.23). Also, for an AWGN channel, p (rk j vk ) is given by
p (r j v ) = p
1 e (Es=N )(ruk uk )  p 1 e (Es=N )(rpk pk )
0
2
0
2
k k
N0 N0
( Es =N0 )(ruk
2 +1+r 2 +1)
e
 e(2Es=N0)(uk ruk +pk rpk )
pk
= N0
= Bk  e(2Es =N0 )(uk ruk +pk rpk ) ; (16.111)
where the parameter Bk depends only on rk and thus is the same for all transitions from time k 1 to
time k. Combining (16.109), (16.110), and (16.111), we obtain
Ak Bk  euk La (uk )=2  e(2Es =No )(uk ruk +pk rpk )


k s0 ; s =
= Ak Bk  euk La (uk )=2  eLc (uk ruk +pk rpk )=2
= Ak Bk  euk (1=2)[La (uk )+Lc ruk ]  eLc pk rpk =2 : (16.112)
Continuing, the and terms in (16.108) can be determined using the MAP algorithm recursions
X
Forward recursion: k (s) = k (s0; s) k 1(s0 ) (16.113a)
(s0 2
P
k 1 )
and
X
Backward recursion: k 1(s0 ) = k (s0; s) k (s); (16.113b)
(s2 k ) P

the initial conditions 0(0) = 4(0) = 1, and (16.112), where the all-zero state S0 = 0 is both the initial
state and the nal state in the trellis diagram.
84 CHAPTER 16. TURBO CODING

Now we observe from Figure 16.18(b) that to determine the a posteriori L-value of bit u0, there is
only one term in each of the sums in (16.107), because at this time there is only one \+1" transition
and one \-1" transition in the trellis diagram. Thus, using the simpli ed notation S0 = 0 and S1 = 1,
we can express the a posteriori L-values of u0 as

= ln pp ((ss0 == 0;
0 0; s = 0; r)
L(u0 )
s = 1; r)
= ln 0(0)  1(s0 = 0; s = 0)  1(0) ln  0(0)  1(s0 = 0; s = 1)  1(1)


n o n o
= ln e+(1=2)[La(u )+Lcru ]  e+(1=2)Lcrp  1(0) ln e (1=2)[La(u )+Lcru ]  e (1=2)Lcrp  1(1)
0 0 0 0 0 0

+(1=2)Lcrp
= ln e+(1=2)[La(u )+Lcru ] ln e (1=2)[La(u )+Lcru ] + ln ee (1=2)Lcrp  1((01))
n o n o 0
0 0 0 0

1 0

= Lcru0 + La(u0) + Le(u0); (16.114a)

where

e+(1=2)Lcrp0  1 (0)
Le (u0 ) = ln (16.114b)
e (1=2)Lcrp0  1 (1)
represents the extrinsic a posteriori (output) L-value of u0. (Note that in (16.114a) the factor A0B0
appeared in both numerator and denominator and thus was canceled out. In exactly the same way,
the factors Ak and Bk always cancel out in the calculation of a posteriori L-values.) The nal form
of (16.114a) illustrates clearly the three constituents of the a posteriori L-value of u0 computed at the
output of a Log-MAP decoder:

 Lcru0: the received channel L-value corresponding to bit u0, which was part of the decoder input.
 La(u0): the a priori L-value of u0, which was also part of the decoder input. Except for the rst
iteration of Decoder 1, this term equals the extrinsic a posteriori L-value of u0 received from the
output of the other decoder. (For the rst iteration of Decoder 1, La(u0) = 0 for equally likely
input bits.)
 Le(u0): the extrinsic part of the a posteriori L-value of u0, which does not depend on Lcru0 or
La (u0 ). This term is then sent to the other decoder as its a priori input.
16.5. ITERATIVE DECODING OF TURBO CODES 85
We now proceed in a similar manner to compute the a posteriori L-value of bit u1. We see from
Figure 16.18(b) that in this case there are two terms in each of the sums in (16.107), because at this
time there are two \+1" and two \-1" transitions in the trellis diagram. Thus,
p (s0 = 0; s = 0; r) + p (s0 = 1; s = 1; r)
L(u1 ) = ln 0
p (s = 0; s = 1; r) + p (s0 = 1; s = 0; r)
= ln 1(0)  2(s0 = 0; s = 0)  2(0) + 1(1)  2(s0 = 1; s = 1)  2(1)


ln  1(0)  1(s0 = 0; s = 1)  2(1) + 1(1)  1(s0 = 1; s = 0)  2(0)

n h io
= ln e+(1=2)[La(u )+Lcru ]  e+(1=2)Lcrp  1(0) 2(0) + e (1=2)Lcrp  1(1) 2(1)
1 1 1 1

n h io
ln e (1=2)[La(u )+Lcru ]  e (1=2)Lcrp  1(0) 2(1) + e+(1=2)Lcrp  1(1) 2(0)
1 1 1 1

n o n o
= ln e+(1=2)[La(u )+Lcru ] ln e (1=2)][La(u )+Luru ] +
1 1 1 1

+(1=2)Lcrp (1=2)Lcrp
ln ee (1=2)Lcrp  1((00)) 2((10)) ++ ee+(1=2)Lcrp  1((11)) 2((01))
1 1

1
1 2 1
1 2
= Lcru1 + La(u1) + Le(u1); (16.115a)
where  
e+(1=2)Lcrp1  1 (0) 2(0) + e (1=2)Lcrp1  1 (1) 2 (1)
Le (u1 ) = ln  (1=2)Lcrp1  (16.115b)
e  1 (0) 2(1) + e+(1=2)Lcrp1  1(1) 2(0)
and the A1 and B1 factors have again been canceled. Continuing, we can use the same procedure to
compute the a posteriori L-values of bits u2 and u3 as
L (u2 ) = Lc ru2 + La (u2 ) + Le (u2 ) ; (16.116a)
where
 
e+(1=2)Lcrp2  2 (0) 3 (0) + e (1=2)Lcrp2  2 (1) 3 (1)
Le (u2 ) = ln  (1=2)Lcrp2 (16.116b)
 2 (0) 3 (1) + e+(1=2)Lcrp2  2 (1) 3 (0) ;

e
and
L (u3 ) = Lc ru3 + La (u3 ) + Le (u3 ) ; (16.117a)
where
+(1=2)Lcrp
Le (u3 ) = ln ee+(1=2)Lcrp  3 ((01)) 4 ((00))
3

33
4
86 CHAPTER 16. TURBO CODING

+(1=2)Lcrp
= ln ee+(1=2)Lcrp  3 ((01)) : (16.117b)
3

3 3

We now need expressions for the terms 1(0), 1(1), 2(0), 2(1), 3(0), 3(1), 1(0), 1(1),
2 (0), 2(1), 3 (0), and 3(1) that are used to calculate the extrinsic a posteriori L-values Le (uk ),
k = 0; 1; 2; 3. Using (16.112) and (16.113) and Figure 16.18(b), dropping all the Ak and Bk factors
(since they cancel out in the ratios used to form the extrinsic L-values), and using the shorthand
notation Luk =4 Lcruk + La (uk ) and Lpk =4 Lcrpk , k = 0; 1; 2; 3, we obtain (see Problem 16.24)
1 (0) = e+(1=2)[Lu0 +Lp0 ] (16.118a)
1 (1) = e (1=2)[Lu0 +Lp0 ] (16.118b)
2 (0) = e+(1=2)[Lu0 +Lp0 +Lu1 +Lp1 ] + e (1=2)[Lu0 +Lp0 Lu1 +Lp1 ] (16.118c)
2 (1) = e+(1=2)[Lu0 +Lp0 Lu1 Lp1 ] +e (1=2)[Lu0 +Lp0 +Lu1 Lp1 ] (16.118d)
3 (0) = e+(1=2)[Lu0 +Lp0 +Lu1 +Lp1 +Lu2 +Lp2 ] + e (1=2)[Lu0 +Lp0 Lu1 +Lp1 Lu2 Lp2 ] +
e+(1=2)[Lu0 +Lp0 Lu1 Lp1 Lu2 +Lp2 ] +e (1=2)[Lu0 +Lp0 +Lu1 Lp1 +Lu2 Lp2 ] (16.118e)
3 (1) = e+(1=2)[Lu0 +Lp0 +Lu1 +Lp1 Lu2 Lp ] + e
2 (1=2)[Lu0 +Lp0 Lu1 +Lp1 +Lu2 +Lp2 ] +
e+(1=2)[Lu0 +Lp0 Lu1 Lp1 +Lu2 Lp ] + e
2 (1=2)[Lu0 +Lp0 +Lu1 Lp1 Lu2 +Lp2 ] (16.118f)
3 (0) = e+(1=2)[Lu3 +Lp3 ] (16.118g)
3 (1) = e (1=2)[Lu3 Lp3 ] (16.118h)
2 (0) = e+(1=2)[Lu3 +Lp3 +Lu2 +Lp2 ] + e (1=2)[Lu3 Lp3 +Lu2 +Lp2 ] (16.118i)
2 (1) = e+(1=2)[Lu3 +Lp3 Lu2 +Lp2 ] +e (1=2)[Lu3 Lp3 Lu2 +Lp2 ] (16.118j)
1 (0) = e+(1=2)[Lu3 +Lp3 +Lu2 +Lp2 +Lu1 +Lp1 ] + e (1=2)[Lu3 Lp3 +Lu2 +Lp2 Lu1 Lp1 ] +
e+(1=2)[Lu3 +Lp3 Lu2 +Lp2 Lu1 Lp1 ]
+e (1=2)[Lu3 Lp3 Lu2 +Lp2 +Lu1 +Lp1 ] (16.118k)
1 (1) = e+(1=2)[Lu +Lp +Lu +Lp Lu +Lp ] + e
3 3 2 2 1 1 (1=2)[Lu3 Lp3 +Lu2 +Lp2 +Lu1 Lp1 ] +
e+(1=2)[Lu +Lp Lu +Lp +Lu Lp ] + e
3 3 2 2 1 1 (1=2)[Lu3 Lp3 Lu2 +Lp2 Lu1 +Lp1 ] : (16.118l)
We note here that the a priori L-value of a parity bit La (pk ) = 0 for all k, since a linear code with
16.5. ITERATIVE DECODING OF TURBO CODES 87
equally likely information bits, the parity bits are also equally likely. Also, unlike the information bits,
the L-values of the parity bits are not updated by the iterative decoding algorithm, and the parity bit
L-values remain equal to 0 throughout decoding. Thus we can write Lpk = Lc rpk + La (pk ) = Lc rpk ,
i.e., the shorthand notation for the L-values of information bits Luk and parity bits Lpk is consistent.
Finally, the extrinsic a posteriori L-values are given by
e+(1=2)Lp  1 (0)
Le (u0 ) = ln (1=2)Lp
0

e
n
 1 (1)
0

= ln e+(1=2)[Lp +Lu +Lp +Lu +Lp +Lu ] + e (1=2)[ Lp Lu Lp +Lu +Lp +Lu ]+
0 1 1 2 2 3 0 1 1 2 2 3

o
e+(1=2)[Lp Lu Lp Lu +Lp +Lu ] + e (1=2)[ Lp +Lu +Lp Lu +Lp +Lu ]
0 1 1 2 2 3 0 1 1 2 2 3

n
ln e+(1=2)[ Lp Lu +Lp +Lu +Lp +Lu ] + e (1=2)[Lp +Lu Lp +Lu +Lp +Lu ]+
0 1 1 2 2 3 0 1 1 2 2 3

o
e+(1=2)[ Lp +Lu Lp Lu +Lp +Lu ] + e (1=2)[Lp Lu +Lp Lu +Lp +Lu ] ; (16.119a)
0 1 1 2 2 3 0 1 1 2 2 3

e+(1=2)Lp  1 (0) 2(0) + e (1=2)Lp  1 (1) 2(1)

Le (u1 ) = ln (1=2)Lp
1 1

e
n
 1 (0) 2(1) + e+(1=2)Lp  1(1) 2(0)
1 1

= ln e+(1=2)[Lu +Lp +Lp +Lu +Lp +Lu ] + e (1=2)[ Lu Lp Lp +Lu +Lp +Lu ]+
0 0 1 2 2 3 0 0 1 2 2 3

o
e+(1=2)[ Lu Lp Lp Lu +Lp +Lu ] + e (1=2)[Lu +Lp +Lp Lu +Lp +Lu ]
0 0 1 2 2 3 0 0 1 2 2 3

n
ln e+(1=2)[Lu +Lp Lp Lu +Lp +Lu ] + e (1=2)[ Lu Lp +Lp Lu +Lp +Lu ]+
0 0 1 2 2 3 0 0 1 2 2 3

o
e+(1=2)[ Lu Lp +Lp +Lu +Lp +Lu ] + e (1=2)[Lu +Lp Lp +Lu +Lp +Lu ] ; (16.119b)
0 0 1 2 2 3 0 0 1 2 2 3

e+(1=2)Lp  2 (0) 3(0) + e (1=2)Lp  2 (1) 3(1)

Le (u2 ) = ln (1=2)Lp
2 2

e
n
 2 (0) 3(1) + e+(1=2)Lp  2(1) 3(0)
2 2

= ln e+(1=2)[Lu +Lp +Lu +Lp +Lp +Lu ] + e (1=2)[Lu +Lp Lu +Lp Lp Lu ]+

0 0 1 1 2 3 0 0 1 1 2 3

o
e+(1=2)[Lu +Lp Lu Lp Lp Lu ] + e (1=2)[Lu +Lp +Lu Lp +Lp +Lu ]
0 0 1 1 2 3 0 0 1 1 2 3

n
ln e+(1=2)[Lu +Lp +Lu +Lp Lp Lu ] + e (1=2)[Lu +Lp Lu +Lp +Lp +Lu ]+
0 0 1 1 2 3 0 0 1 1 2 3

o
e+(1=2)[Lu +Lp Lu Lp +Lp +Lu ] + e (1=2)[Lu +Lp +Lu Lp Lp Lu ] ; (16.119c)
0 0 1 1 2 3 0 0 1 1 2 3

and
+(1=2)Lp
Le (u3 ) = ln ee+(1=2)Lp  3 ((01))
3

n
3
3
= ln e +(1= 2)[Lu +L p +Lu +Lp +Lu +Lp ] + e
0 0 1 1 2 2 (1=2)[Lu0 +Lp0 Lu1 +Lp1 Lu2 Lp2 ] +
88 CHAPTER 16. TURBO CODING

o
e+(1=2)[Lu0 +Lp0 Lu1 Lp1 Lu2 +Lp2 ] +e (1=2)[Lu0 +Lp0 +Lu1 Lp1 +Lu2 Lp2 ]
n
ln e+(1=2)[Lu0 +Lp0 +Lu1 +Lp1 Lu2 Lp2 ] +e (1=2)[Lu0 +Lp0 Lu1 +Lp1 +Lu2 +Lp2 ]
+
o
e+(1=2)[Lu0 +Lp0 Lu1 Lp1 +Lu2 Lp2 ] +e (1=2)[Lu0 +Lp0 +Lu1 Lp1 Lu +Lp ] : (16.119d)
2 2

We note that in the expressions for the extrinsic L-values Le (uk ), the intrinsic L-value Luk does not
appear in any of the terms, i.e., the extrinsic L-value of bit uk does not depend directly on either the
received or a priori L-values of uk . We also note that the received L-value Lp3 = Lcrp3 cancels out of
all the expressions. This is due to the fact that, for the trellis of Figure 16.18(b), all branches at time
k = 3 have p3 = 0, and thus Lp3 conveys no information useful for decoding.

Example 16.14 (Continued)

We can now evaluate the extrinsic a posteriori L-values in our example using (16.119) and the received
channel L-values in Figure 16.19(c). Beginning with the rst iteration of Decoder 1 (row decoding) for
the bits u0, u1, u2, and u3, and recalling that the initial a priori L-values in the expressions for Lu0,
Lu1 , Lu2 , and Lu3 are equal to 0, we obtain

L(1) 0:75 + e0:15 + e 2:05 + e2:45

e0:35 + e1:25 + e 2:95 + e1:55 = 0:32;
e (u0 ) = ln e ln (16.120a)
L(1) 0:65 + e0:25 + e 2:05 + e2:45


e 2:95 + e1:55 + e0:25 + e1:15 = 0:38;
e (u1 ) = ln e ln (16.120b)
and, using similar calculations, Le(u2) = 0:77 and Le(u3) = 0:47. These extrinsic values after the
rst iteration of row decoding are listed in Figure 16.19(d). Now using these extrinsic a posteriori
L-values as the a priori inputs for Decoder 2 (column decoding), so that Luk = Lc ruk + L(1)
e (uk ), we

obtain for the bits u0, u2, u1, and u3

L(2) 0:93 + e1:96 + e 0:14 + e1:51


e 3:16 + e 0:27 + e 0:31 + e1:34 = 0:88;
e (u0 ) = ln e ln (16.121a)
L(2) 1:68 + e1:20 + e 0:58 + e1:06


e0:13 + e1:78 + e 2:40 + e0:48 =
e (u2 ) = ln e ln 0:23; (16.121b)
and, using similar calculations, L(2) (2)
e (u1 ) = 0:69 and Le (u3 ) = 0:04. (Note that in using (16.119)

to evaluate the expressions for L(2)

e (uk ), the roles of u1 and u2 are reversed because of the action
16.5. ITERATIVE DECODING OF TURBO CODES 89
of the interleaver.) These extrinsic values after the rst iteration of column decoding are listed in
Figure 16.19(e). Finally, the a posteriori L-values of the four information bits after the rst complete
iteration of decoding are given by
L(2) (u0 ) = Lc ru0 + L(2) (2)
a (u0 ) + Le (u0 ) = 0:8 + 0:32 + 0:88 = 0:40 (16.122a)
L(2) (u2 ) = Lc ru2 + L(2) (2)
a (u2 ) + Le (u2 ) = 1:8 0:77 0:23 = 0:80 (16.122b)
L(2) (u1 ) = Lc ru1 + L(2) (2)
a (u1 ) + Le (u1 ) = 1:0 + 0:38 + 0:69 = 0:07 (16.122c)
L(2) (u3 ) = Lc ru3 + L(2) (2)
a (u3 ) + Le (u3 ) = 1:6 0:47 + 0:04 = 2:03 (16.122d)
and shown in Figure 16.19(f), where the a priori L-values for Decoder 2, L(2) a (uk ), are equal to the

extrinsic a posteriori L-values, L(1)

e (uk ); k = 0; 1; 2; 3, after the rst iteration of Decoder 1. Thus, if a

nal decoding decision were made after just one iteration, the three information bits would be decoded
as
u^0 = +1; u^2 = +1; u^1 = +1; (16.123)
and bit u1 would be incorrectly decoded in this case.
A second iteration of decoding would use the extrinsic a posteriori L-values L(2) e (uk ) from the rst

iteration of Decoder 2 as a priori L-values L(1)

a (uk ), k = 0; 1; 2; 3, for Decoder 1, thus producing a new

set of extrinsic a posteriori L-values L(1)

e (uk ) at the output of Decoder 1. These would then be used as

a priori inputs to Decoder 2, and so on. In Figure 16.19(g), (h), and (i), we give the L-values L(1) e (uk ),

L(2) (2)
e (uk ), and L (uk ), respectively, k = 0; 1; 2; 3, for the second iteration of decoding (see Problem

16.25). Note that in this case, if a nal decoding decision were made after two iterations of decoding,
all information bits would be correctly decoded. 

Remarks:
 Although u3 is not an information bit and does not need to be decoded, its extrinsic L-values
must be updated throughout decoding.
 In this exmple, all input sequences that terminate Encoder 1, when interleaved, also terminate
Encoder 2. Thus the operation of the MAP algorithm is exactly the same for both decoders. In
90 CHAPTER 16. TURBO CODING

general, however, the interleaved input sequence will not terminate Encoder 2. In this case, the
MAP algorithm must specify initial values for the 's at each state, i.e., N (Si), i = 0; 1;


; 2K 1,
must be speci ed. These can either be set to equally likely values or to the nal values of the 's
at time N .
The most diÆcult calculations in the above example are the logarithms of sums of exponential terms
needed in the computation of extrinsic L-values. In this case the useful approximation
X
ln eaj  max a
(j ) j
(16.124)
(j )
allows us to simplify the calculation of the extrinsic L-values. The approximation of (16.124) is the
basis of the Max-Log-MAP algorithm discussed previously in Chapters 12 and 14. We now use this
approximation to revisit the above example.
Example 16.15
When the approximation of (16.124) is applied to the extrinsic a posteriori L-value calculations in
(16.120) and (16.121), we obtain for the rst iteration of Decoder 1
L(1) = ln e 0:75 + e0:15 + e 0:25 + e2:45
e0:35 + e1:25 + e 2:95 + e1:55


e (u0 ) ln
 2:45 1:55 = 0:9; (16.125a)
L(1) 0:65 + e0:25 + e 2:05 + e2:45
e 2:95 + e1:55 + e0:25 + e1:15


e (u1 ) = ln e ln
 2:45 1:55 = 0:9; (16.125b)
and, using similar caculations, L(1) (1)
e (u2 )  1:4 and Le (u3 )  0:3 Using these approximate extrinsic a

posteriori L-values as a priori L-values for Decoder 2, we obtain

L(2)
e (u0 ) = ln e 0:6 + e1:0 + e0:3 + e1:7
ln e 2:2 + e 0:6 + e 0:5 + e0:9

 1:7 0:9 = 0:8; (16.126a)
L(2) 0:75 + e0:85 + e 0:75 + e0:65
ln e0:55 + e1:95 + e 2:05 + e 0:45

e (u2 ) = ln e

 0:85 1:95 = 1:1; (16.126b)

16.5. ITERATIVE DECODING OF TURBO CODES 91
and, using similar calculations, L(2) (2)
e (u1 )  0:8 and Le (u3 )  0:1. These approximate extrinsic a

posteriori L-values for row and column decoding are listed in Figure 16.19(j) and 16.19(k), respectively.
Finally, the approximate a posteriori L-value of information bit u0 after the rst complete iteration of
decoding is calculated as

L(2) (u0 ) = Lc ru0 + L(2)

a (u0 ) + Le (u0 )  0:8 + 0:9 + 0:8 = 0:9; (16.127)

and the remaining approximate a posteriori L-values are obtained similarly as L(2)(u2)  0:7, L(2)(u1) 
0:7, and L(2)(u3)  1:4. These approximate a posteriori L-values are shown in Figure 16.19(l) and
we see that in this case the Max-Log-MAP algorithm would incorrectly decode both bits u1 and u2 if
decoding was stopped after one iteration. In Figure 16.19(m), (n), and (o), we give the approximate
L-values L(1) (2) (2)
e (uk ), Le (uk ), and L (uk ), respectively, k = 0; 1; 2; 3, for a second iteration of Max-Log-

MAP decoding (see Problem 16.25). We see from Figure 16.19(o) that if a nal decoding decision were
made after two iterations of decoding, the Max-Log-MAP algorithm would decode all bits correctly, the
same as the Log-MAP algorithm. 

The major lesson of Examples 16.14 and 16.15 can be stated as follows:
Fundamental Principle of Turbo Decoding [14]:
Iteratively estimate information symbols using the MAP algorithm, whose inputs include successively
updated a priori L-values obtained from all statistically independent information available after the
previous iteration of decoding.
Remarks:
 The extrinsic a posteriori L-values are no longer strictly independent of the other terms after the
rst iteration of decoding, which causes the performance improvement from successive iterations
to diminish over time.
 The concept of iterative decoding is similar to negative feedback in control theory, in the sense
that the extrinsic information from the output that is fed back to the input has the e ect of
amplifying the SNR at the input, leading to a stable system output.
92 CHAPTER 16. TURBO CODING

 Decoding speed can be improved by a factor of two by allowing the two decoders to work in
parallel. In this case, the a priori L-values for the rst iteration of Decoder 2 would be the same
as for Decoder 1 (normally equal to 0), and the extrinsic a posteriori L-values would then be
exchanged at the same time prior to each succeeding iteration.
 After a suÆcient number of iterations, the nal decoding decision can be taken from the a posteriori
L-values of either decoder, or from the average of these values, without a noticeable e ect on
performance.
 As noted above, the L-values of the parity bits remain equal to 0 throughout decoding. In serially
concatenated iterative decoding systems, however, parity bits from the outer decoder enter the
inner decoder, and thus the L-values of these parity bits must be updated during the iterations
[15].
 The above approach to iterative decoding is ine ective for nonrecursive nonsystematic constituent
codes, such as the one in Example 16.8, since channel L-values for the information bits are not
available as inputs to Decoder 2. However, the iterative decoder of Figure 16.6 can be modi ed
to decode PCCC's with recursive nonsystematic constituent codes [16].
 A row-column interleaver is used in Examples 16.14 and 16.15 for illustrative purposes only.
As noted previously, better performance is normally achieved with pseudorandom interleavers,
particularly for large block lengths, and the iterative decoding procedure remains the same.
 Further iterations of decoding in these examples would most likely improve the a posteriori L-
values of the input bits, i.e., L(2)(u0) and L(2)(u2) would become more positive, while L(2)(u1) and
L(2) (u3 ) would become more negative (see Problem 16.26). It is possible, however, particularly on
very noisy channels, for the decoder to converge to the correct decision and then diverge again,
or even to \oscillate" between correct and incorrect decisions.
 Iterations can be stopped after some xed number, typically in the range of 10-20 for most turbo
codes, or stopping rules based on reliability statistics can be used to halt decoding.
 The Max-Log-MAP algorithm is simpler to implement than the Log-MAP algorithm. However,
it typically su ers a performance degradation of about 0.5dB.
 It can be shown that the Max-Log-MAP algorithm is equivalent to the SOVA algorithm presented
in Chapter 12 [17].
The Log-MAP algorithm can be implemented by using the Max-Log-MAP approximation ln (ex + ey ) 
max(x; y) (see (16.124)) to simplify the calculation of the extrinsic a posteriori L-values and then adding
the correction term ln 1 + e jx yj
, which, given the input x y, can be accessed from a ROM. (For
16.5. ITERATIVE DECODING OF TURBO CODES 93
trellises with more than two bits per branch, this correction term can be applied in the same manner
successively to sums involving more than two terms.) Since
 
efmax(x;y)+ln(1+e
jx yj )g
= emax(x;y) 1 + e jx yj
= emax(x;y) + emax(x;y) jx yj
 x
= e + ex (x y) = ex + ey ; x > y (16.128)
ey + ey+(x y) = ey + ex ; x < y

this simpli ed implementation is equivalent to the Log-MAP algorithm.

The e ect of the number of decoding iterations on BER performance is illustrated in Figure 16.20 for
the R = 1=2 PCCC with block length N = 216 = 65536 obtained by using a pseudorandom interleaver
and puncturing alternate parity bits from the encoder of Figure 16.1(b). The performance improvement
with increasing iterations, as well as the diminishing e ect as the number of iterations gets large, is
clearly evident from the gure. This is the original code proposed in the paper by Berrou, Glavieux,
and Thitimajshima [18] that launched the eld of turbo coding. After 18 iterations, a BER of 10 5 is
achieved at an SNR of Eb=N0 = 0:7dB, which is only 0.5dB away from the Shannon limit for channels
with binary inputs.
We conclude this chapter with a brief discussion of stopping rules for iterative decoding, i.e., methods
of detecting when decoding has converged. One such method is based on the cross-entropy (CE) of
the APP distributions at the outputs of the two decoders. The cross-entropy D(P jj Q) of two joint
probability distributions P (u) and Q(u), assuming statistical independence of the bits in the vector
u = [u0 ; u1 ;


; uk ;


; uN 1 ], is de ned as (see [13])
X1
   
log QP ((uu)) log PQ ((uuk )) ;
N
D(P jj Q) = EP = EP (16.129)
k=0 k

where EP f
g denotes expectation with respect to the probability distribution P (uk ). D(P jj Q) is a
measure of the closeness of two distributions and

D (P jj Q) = 0 i P (uk ) = Q (uk ) ; uk = 1; k = 0; 1;


; N 1: (16.130)

94 CHAPTER 16. TURBO CODING

1
BPSK
1 iteration
2 iterations
3 iterations
0.1 6 iterations
18 iterations

0.01

0.001
Pb(E)

1e-04

1e-05

1e-06

1e-07
0 1 2 3 4 5 6
Eb/N0 (dB)

Figure 16.20: The E ect of Iterations on Decoding Performance

16.5. ITERATIVE DECODING OF TURBO CODES 95
Then when two distributions are nearly identical, the cross-entropy D(P jj Q) is close to 0. Thus the
value of D(P jj Q) can be used as an indication of when iterative decoding has converged and the
decoding iterations can be stopped.
The CE stopping rule is based on the di erence between the a posteriori L-values after successive
iterations at the outputs of the two decoders. For example, let
L(1) (1) (1)
(i) (uk ) = Lc ruk + La(i) (uk ) + Le(i) (uk ) (16.131a)
represent the a posteriori L-value at the output of Decoder 1 after iteration i, and let
L(2) (2) (2)
(i) (uk ) = Lc ruk + La(i) (uk ) + Le(i) (uk ) (16.131b)
represent the a posteriori L-value at the output of Decoder 2 after iteration i. Now using the facts that
L(1) (2) (2) (1)
a(i) (uk ) = Le(i 1) (uk ) and La(i) (uk ) = Le(i) (uk ), i.e., the a priori L-value at the input of one decoder

is the extrinsic a posteriori L-value at the output of the other decoder, and letting Q (uk ) and P (uk )
represent the a posteriori probability distributions at the outputs of Decoders 1 and 2, respectively, we
can write
L((Qi) ) (uk ) = Lc ruk + L(eP(i) 1) (uk ) + L(eQ(i)) (uk ) (16.132a)
and
L((Pi)) (uk ) = Lc ruk + L(eQ(i)) (uk ) + L(eP(i)) (uk ) : (16.132b)
The di erence in the two soft outputs can now be written as
4
L((Pi)) (uk ) L((Qi) ) (uk ) = L(eP(i)) (uk ) L(eP(i) 1) (uk ) =
L(eP(i)) (uk ) ; (16.133)
i.e.,
L(eP(i)) (uk ) represents the di erence in the extrinsic a posteriori L-values of Decoder 2 in two
successive iterations.
We now compute the CE of the a posteriori probability distributions P (uk ) and Q (uk ) as follows:
 
EP log QP ((uuk )) = P (uk = +1)log
P (uk = +1)
Q (uk = +1)
+ P (uk = 1)log QP ((uuk == 1)
1)
k k
96 CHAPTER 16. TURBO CODING

eL(i) (uk ) eL(i) (uk ) eL(i) (uk )

(P ) (P ) (Q)

= log 1 +
 (Q) +
1 + eL(i) (uk ) 1 + eL(i) (uk ) eL(i) (uk )
(P ) (P )

e L(i) (uk ) e L(i) (uk ) e L(i) (uk )

(P ) (P ) (Q)

log  1 + ; (16.134)
1 + e L(i) (uk) 1 + e L(i) (uk) e L(i) (uk )
(P ) (P ) (Q)

where we have used expressions for the a posteriori distributions P (uk = 1) and Q (uk = 1) analogous
to those given in (16.110). It can be shown that (see Problem 16.29) (16.134) simpli es to

P (uk )


L(eP(i))(uk ) 1 + e L i (uk )
Q ( )

EP log = + log (16.135)

( )
:
Q (uk ) 1 + eL i (uk )
P ( )
( ) 1 + e L i (uk )
P ( )
( )

This expression can be simpli ed further under the assumption that the hard decisions based on the
signs of the a posteriori L-values of both decoders do not change once decoding has converged, i.e., the
hard decisions after iteration i, u^(ki), satisfy
h i h i
u^(ki) = sgn L((Pi)) (uk ) = sgn L((Qi) ) (uk ) : (16.136)
Using (16.136) and noting that
h i
L((Pi)) (uk ) = sgn L((Pi)) (uk ) L((Pi)) (uk ) = u^(ki) L((Pi)) (uk ) (16.137a)
and
h i
L((Qi) ) (uk ) = sgn L((Qi) ) (uk ) L((Qi) ) (uk ) = u^(ki) L((Qi) ) (uk ); (16.137b)
it can be shown that (see Problem 16.29) (16.135) simpli es further to
u^(ki)
L(eP(i)) (uk ) e L i (uk )
  Q
P (uk ) 1 +
( )

EP log  + log (16.138)

( )
:
Q (uk ) 1 + e L i (uk )
P ( )
( )
1 + e L i (uk )
P ( )
( )

We now use the facts that once decoding has converged, the magnitudes of the a-posteriori L-values
are large, i.e.,
L((Pi)) (uk )  0 and L((Qi) ) (uk )  0; (16.139)
and that when x is large, e x is small and
1 + e x  1 and log 1 + e x
 e x: (16.140)
16.5. ITERATIVE DECODING OF TURBO CODES 97
Applying these approximations to (16.138), it can be shown that (see Problem 16.29)
   
P (uk ) LQ (uk ) u^ki
LePi (uk )

(i) (P)
EP log e 1 e 1 + u^k
Le(i)(uk ) : (16.141)
( ) ( ) ( )
i
Q (uk )
( ) ( )

Noting that the magnitude of
L(eP(i))(uk ) will be smaller than 1 when decoding converges, we can
approximate the term e u^k
Le i (uk ) using the rst two terms of its series expansion as follows:
i P
( ) ( )
( )

e u^k
Le(i) (uk )  1 u^(ki)
L(eP(i)) (uk );

(i) (P )
(16.142)
which leads to the simpli ed expression
 
P (uk ) h   i
e L i (uk ) 1 1 u^(ki)
L(eP(i))(uk ) 1 + u^(ki)
L(eP(i))(uk )
Q
EP log 
( )

Q (uk )
( )

2
(Q) h
(uk ) u^(i)
L(P ) (u ) i2
L(eP(i))(uk )
= e L (i)
k e(i) k = : (16.143)
i) (uk )
L((Q )
e
Now using (16.129) and (16.143) we can write the CE of the probability distributions P (u) and Q (u)
at iteration i as 2
  N (P )
X1
Le(i) (uk )
4 P (u)
D(i) (P jj Q) = EP log  ; (16.144)
Q (u) L Q (u )
k=0 e
( )
(i) k

where we note that the statistical independence assumption does not hold exactly as the iterations
proceed.
We next de ne 2
(P )
4 X
Le(i) (uk )
N 1
T (i) = ; (16.145)
L Q (u ) ( )
k=0 e (i) k
as the approximate value of the CE at iteration i given by
(16.144). T (i) can be computed after
each iteration. (Note that in computing T (1), we need to nd
L(eP(1)) (uk ) = L(eP(1)) (uk ) L(eP(0)) (uk ),
where L(eP(0)) (uk ) is taken to be the initial a priori L-value of the information bit uk , i.e., L(eP(0)) (uk ) = 0,
k = 0; 1;


; N 1, for equally likely inputs.) Experience with computer simulations has shown that
once convergence is achieved, T (i) drops by a factor of 10 2 to 10 4 compared to its initial value, and
thus it is reasonable to use
T (i) < 10 3 T (1) (16.146)
98 CHAPTER 16. TURBO CODING

as a stopping rule for iterative decoding.

The observation that led to the assumption of (16.136) can also be used to de ne a simpler stopping
rule based only on the hard decision outputs of each decoder. For example, if (16.136) is satis ed
for some number of consecutive iterations (5 is a reasonable choice), for all k = 0; 1;


; N 1, we
can declare that convergence has occured and stop decoding. While not quite as e ective in detecting
convergence as the CE rule, this hard decision aided (HDA) rule is extremely simple to implement. The
CE stopping rule was discussed in [12], while two HDA stopping rules were introduced in [19].

Another approach to stopping the iterations in turbo decoding is to concatenate a high rate outer
cyclic code with an inner turbo code, as illustrated in Figure 16.21. After each iteration, the hard
decision output of the turbo decoder is used to check the syndrome of the cyclic code. If no errors are
detected, decoding is assumed correct and the iterations are stopped. It is important to choose an outer
code with a low undetected error probability, so that iterative decoding is not stopped prematurely.
For this reason, it is usually advisable not to check the syndrome of the outer code during the rst few
iterations, when the probability of undetected error may be larger than the probability that the turbo
decoder is error free. This method of stopping the iterations is particularly e ective for large block
lengths, since in this case the rate of the outer code can be made very high, thus resulting in a negligible
overall rate loss.

For large block lengths, the above idea can be extended to include outer codes, such as BCH codes,
which can correct a small number of errors and still maintain a low undetected error probability. In
this case, the iterations are stopped once the number of hard decision errors at the output of the turbo
decoder is within the error correcting capability of the outer code. This method also provides a low
word error probability for the complete system, i.e., the probability that the entire information block
contains one or more decoding errors can be made very small. The idea of combining a tubo code with
a high rate outer BCH code was introduced in [20] and further analyzed in [21].
16.6. NOTES AND ASIDES 99
w Outer u (0)
v
Cyclic
Code
(1)
v
Encoder1
(2)

π Encoder2 v

Figure 16.21: A Concatenation of an Outer Cyclic Code With an Inner Turbo Code

16.6 Notes and Asides

The original concept of turbo coding was introduced in a paper by Berrou, Glavieux, and Thitimajshima
[18] delivered at the IEEE International Conference on Communications held in Geneva, Switzerland in
May 1993, and further elaborations were presented in [22, 23]. Many of the earlier ideas of Batill and
Hagenauer on random-like codes and iterative decoding were in uential in leading to the turbo coding
concept [24, 25, 26, 27]. Much of the research community was originally skeptical of the performance
claims, but by the end of 1994 the basic results had been con rmed by several other research groups
[28, 29]. Two papers by Benedetto and Montorsi [3] and [5] provided the rst theoretical justi cation
for the exceptional performance of the codes, and further insights were presented in [30] and [31].
The research group of Hagenauer was primarily responsible for illuminating the bene ts of iterative
decoding [12, 14], and additional understanding was achieved in [32] and [33]. The quarterly TDA
progress reports, beginning in 1995, of Divsalar, Pollara, Dolinar, and colleagues at the Jet Propulsion
Laboratory provide a detailed look at many aspects of turbo coding [34, 35, 36, 37] particularly as they
impact deep space communication issues. Numerous variations of turbo coding have also appeared in the
literature, such as serial concatenation, hybrid parallel and serial concatenation, and self concatenation
[15, 38, 39]. A comprehensive overview of the rst ve years of turbo coding is given in the book by
Heegard and Wicker [40], which also contains a thorough discussion of iterative decoding from the point
100 CHAPTER 16. TURBO CODING

of view of the theory of belief propagation. Another book covering roughtly the same material was
written by Vucetic [41], and a readable summary of the main aspects of turbo coding, with an emphasis
on iterative decoding, was published recently by Ryan [42].
Problems

16.1. Prove that the general rate 1=3 turbo encoder shown in Figure 16.1(a), where encoder 1 and
encoder 2 are linear convolutional encoders (not necessary identical) separated by an arbitrary
interleaver, is a linear system.
16.2. For the length N = 16 quadratic interleaver of (16.7), determine all pairs of indices which are
interchanged by the permutation.
16.3. Consider a PCBC with two di erent constituent codes: the (7; 4; 3) Hamming code and the (8; 4; 4)
extended Hamming code. Find the CWEF's, IRWEF's, and WEF's of this code assuming a
uniform interleaver.
16.4. Find the IRWEF's and WEF's for Example 16.5.
16.5. Repeat Example 16.5 for the case h = 4. What is the minimum distance of the (40,16) PCBC if
a 4  4 row-column (block) interleaver is used?
16.6. Consider a PCBC with the (24; 12; 8) extended Golay code in systematic form, with generator
polynomial g(x) = 1 + x + x2 + x3 + x4 + x7 + x10 + x12, as the constituent code. Assuming a
uniform interleaver,
(a) nd the CWEF's APwC (Z ); w =1, 2,


, 12,
(b) nd the IRWEF's AP C (W; Z ) and BP C (W; Z ), and
(c) nd the WEF's AP C (X ) and BP C (X )
(d) Now consider a PCBC with the 12-repeated (24; 12; 8) extended Golay code in systematic
form as the constituent code. Assume that the information bits are arranged in a square
 16.6. NOTES AND ASIDES 101
array and that a row-column interleaver is used, i.e., encoder 1 encodes across rows of the
array and encoder 2 encodes down columns of the array. Find the parameters (n, k, d) of
the PCBC.
16.7. Prove (16.37).
16.8. Find the codeword IRWEF and WEF for the PCCC in Example 16.6.
16.9. Find the bit IRWEF's and WEF's for Example 16.6.
16.10. Repeat Example 16.6 for the encoder with reversed generators given in (16.62).
16.11. Repeat Example 16.7 for the encoder with reversed generators given in (16.62).
16.12. Find the multiplicity of weight 8 codewords in Example 16.8.
16.13. Repeat Example 16.8 using the feedforward encoder

Gff (D) = 1 1 + D + D2 
for the second constituent code.
16.14. Consider a rate 1=4 multiple turbo code (PCCC) with constituent encoder
G(D) = [1 (1 + D + D2)=(1 + D)]
separated by two random interleavers (see Figure 16.2). Assuming a uniform interleaver and large
block size N,
(a) nd the approximate CWEF's APwC (Z ) and BwP C (Z ) for w= 2,3,4,5,
(b) nd the approximate IRWEF's AP C (W; Z ) and BP C (W; Z ),
(c) nd the approximate WEF's AP C (X ) and BP C (X ), and
(d) sketch the union bounds on Pw (E ) and Pb(E ) for N=1000 and N=10000, assuming a binary
input, unquantized output AWGN channel.
 102 CHAPTER 16. TURBO CODING

16.15. Find the minimum weight codewords corresponding to input weights 2 and 3 for the PCCCs whose
generators are given in Table 16.6. In each case determine the free distance dfree assuming large
N.
h iw
16.16. Show that for any (n, 1, m) systematic feedforward encoder A(ww)(Z ) = A(1)
1 ( Z ) and that for
h iw
any (n, 1, m) systematic feedback encoder A(2ww)(Z ) = A(1)
2 (Z ) .
16.17. Show that a weight one input sequence cannot terminate a systematic feedback encoder, but there
always exists a weight two input sequence, of degree no greater than 2m 1, that does terminate
the encoder.
16.18. For an (n,1,m) systematic feedback encoder, show that the input sequence u = (1000


) produces
a cycle with input weight 0 starting in state S1 = (10


0), arriving in state S2m = (0


01)1

after at most 2m 2 steps, and returning to state S1 = (10


0) in one step.

16.19. For an (n,1,m) systematic feedback encoder, show that a \1" input from state S2m = (0


01) 1

terminates the encoder.

16.20. Compute zmin and deff for the turbo codes with primitive and nonprimitive constituent codes D
and E of Table 16.6.
16.21. Show that the bound of (16.97) is also valid for nonprimitive denominator polynomials.
16.22. Use the S-random interleaver algorithm to construct a length N = 32 permutation that breaks up
all weight 2 input sequences with a spacing of S=4 or less betweens \1's".
16.23. In (16.97), show that ef L(uk )=2g =  1+e L(uk )  is independent of the actual value of uk .
16.24. Verify the and expressions in (16.118).
16.25. Verify all entries in Figure 16.19 that were not computed in the text.
 16.6. NOTES AND ASIDES 103
16.26. Complete 2 more iterations of decoding in Examples 16.14 and 16.15. Is there any change in the
decoded output?
u11 u12 v(1)
1
+1 +1 +1 +0.5 +1.5 +1.0

u21 u22 v(1)

2
+1 -1 -1 +4.0 +1.0 -1.5

(2) (2)
v1 v2 +1 -1 +2.0 -2.5

(8,4,3) PCBC Coded values Received L-values

(a) (b) (c)

+1
S1 S1
-1 -1

-1 -1

S0 S0 S0 S0
+1 +1 +1
Decoding trellis
(d)
Figure P-16.28

16.27. Calculate the cross-entropy at the end of each complete iteration in Example 16.14 and 16.15
and Problem 16.26.

16.28. [12] Consider the (8,4,3) PCBC formed by using h=2 codewords from the (3,2,2) systematic Single
Parity Check (SPC) code, i.e., a (6,4,2) doubly-repeated SPC code, as the constituent code, along
with a 2  2 block (row-column) interleaver of overall size N = 4. The information block is given
by the vector u = [u11; u12; u21; u22], where uij represents the j th information bit in the ith row
of the interleaver, i, j = 1,2, the ith parity bit in the row code is given by vi(1) = ui1  ui2, i=1,2,
 104 CHAPTER 16. TURBO CODING

and the j th parity bit in the column code is given by vj(2) = u1j  u2j , j=1,2. The arrangement is
shown in Figure P-16.28(a). Assume the particular bit values given in Figure P-16.28(b) and the
set of received channel L-values given in Figure P-16.28(c).
(a) Use the trellis shown in Figure P-16.28(d) and the Log-MAP algorithm to compute the
extrinsic L-values for the rst iteration of row and column decoding and the soft output
L-values after the rst complete iteration for each of the N=4 information bits.
(b) Repeat (a) using the Max-Log-MAP algorithm.
16.29. Starting from (16.134), derive the cross-entropy expressions given in (16.135), (16.138), and
(16.141).
References

[1] P. C. Massey and D. J. Costello Jr., \New developments in asymmetric turbo codes," in Proc. 2nd
International Symposium on Turbo Codes, (Brest, France), pp. 93{100, Sept. 2000.

[2] O. Y. Takeshita and D. J. Costello Jr., \New deterministic interleaver designs for turbo codes," in
IEEE Transactions on Information Theory, vol. IT-46, pp. 1988{2006, Sept. 2000.

[3] S. Benedetto and G. Montorsi, \Unveiling the turbo codes: Some results on parallel concatenated
coding," in IEEE Transactions on Information Theory, vol. IT-42, pp. 409{428, March 1996.
[4] I. Sason and S. Shamai (Shitz), \Improved upper bounds on the ML decoding error probability
of parallel and serial concatenated turbo codes via their ensemble distance spectrum," in IEEE
Transactions on Information Theory, vol. IT-46, pp. 24{47, Jan. 2000.

[5] S. Benedetto and G. Montorsi, \Design of parallel concatenated convolutional codes," in IEEE
Transactions on Communications, vol. COM-44, pp. 591{600, May 1996.

[6] S. Benedetto, R. Garello, and G. Montorsi, \A search for good convolutional codes to be used
in the construction of turbo codes," in IEEE Transactions on Communications, vol. COM-46,
pp. 1101{1105, Sept. 1998.
[7] S. Dolinar and D. Divsalar, \Weight distribution for turbo codes using random and nonrandom
permutations," TDA Progress Report 42-122, Jet Propulsion Laboratory, Aug. 1995.
[8] S. ten Brink, \Convergence of iterative decoding," in IEE Electronics Letters, vol. 35, pp. 806{808,
May 1999.
105
106 REFERENCES

[9] T. Richardson, \The geometry of turbo-decoding dynamics," in IEEE Transactions on Information

Theory, vol. IT-46, pp. 9{23, Jan. 2000.

[10] D. Divsalar, S. Dolinar, and F. Pollara, \Iterative turbo decoder analysis based on density evo-
lution," in IEEE Journal On Selected Areas in Communications, vol. SAC-19, pp. 891{907, May
2001.
[11] H. El Gamal and A. R. Hammons Jr., \Analyzing the turbo decoder using the Gaussian approxi-
mation," in IEEE Transactions on Information Theory, vol. IT-47, pp. 671{686, Feb. 2001.
[12] J. Hagenaeur, E. O er, and L. Papke, \Iterative decoding of binary block and convolutional codes,"
in IEEE Transactions on Information Theory, vol. IT-42, pp. 429{445, March 1996.
[13] T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley Interscience, 1991.
[14] J. Hagenauer, \The turbo principle: Tutorial introduction and state of the art," in Proc. 1st
International Symposium on Turbo codes, (Brest, France), pp. 1{11, Sept. 1997.

[15] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, \Serial concatenation of interleaved codes:
Performance analysis, design and iterative decoding," in IEEE Transactions on Information The-
ory, vol. IT-44, pp. 909{926, May 1998.

[16] O. M. Collins, O. Y. Takeshita, and D. J. Costello Jr., \Iterative decoding of non-systematic turbo
codes," in Proc. IEEE International Symposium on Information Theory (ISIT 2000), (Sorrento,
Italy), p. 172, June 2000.
[17] M. P. C. Fossorier, F. Burkert, S. Lin, and J. Hagenauer, \On the equivalence between SOVA and
Max-Log-MAP decoding," in IEEE Communications Letters, vol. 2, pp. 137{139, May 1998.
[18] C. Berrou, A. Glavieux, and P. Thitimajshima, \Near Shannon limit error-correcting coding and
decoding: Turbo codes," in Proc. IEEE International Conference on Communications (ICC 1993),
(Geneva, Switzerland), pp. 1064{1070, May 1993.
REFERENCES 107
[19] R. Y. Shao, S. Lin, and M. P. C. Fossorier, \Two simple stopping criteria for turbo decoding," in
IEEE Transactions on Communications, vol. COM-47, pp. 1117{1120, Aug. 1999.

[20] J. D. Andersen, \Turbo codes extended with outer BCH code," in IEE Electronics Letters, vol. 32,
pp. 2059{2060, Oct. 1996.
[21] O. Y. Takeshita, O. M. Collins, P. C. Massey, and D. J. Costello, Jr. , \On the frame error rate of
concatenated turbo-codes," in IEEE Transactions on Communications, vol. COM-49, pp. 602{608,
April 2001.
[22] C. Berrou and A. Glavieux, \Near optimum error correcting and decoding: Turbo-codes," in IEEE
Transactions on Communications, vol. COM-44, pp. 1261{1271, Oct. 1996.

[23] C. Berrou and A. Glavieux, \Re ections on the prize paper: Near optimum error correcting coding
and decoding: Turbo-codes," IEEE Information Theory Society Newsletter, vol. 48(2), pp. 24{31,
June 1998.
[24] G. Battail, M. Decouvelaere, and P. Godlewski, \Replication decoding," in IEEE Transactions on
Information Theory, vol. IT-25, pp. 332{345, May 1979.

[25] G. Battail, \Building long codes by combination of simple ones, thanks to weighted-output decod-
ing," in Proc. URSI International Symposium on Signals, Systems, and Electronics (ISSSE 1989),
(Erlangen, Germany), pp. 634{637, Sept. 1989.
[26] J. Hagenauer and P. Hoeher, \A Viterbi algorithm with soft-decision outputs and its applications,"
in Proc. IEEE Global Communications Conference (GLOBECOM 1989), (Dallas, Texas), pp. 1680{
1686, Nov. 1989.
[27] J. H. Lodge, P. Hoeher, and J. Hagenauer, \The decoding of multidimensional codes using separable
MAP lters," in Proc. 16th Biennial Symposium on Communications, (Kingston, Ontario), pp. 343{
346, May 1992.
108 REFERENCES

[28] J. D. Andersen, \Turbo coding for deep space applications," in Proc. IEEE International Sympo-
sium on Information Theory (ISIT 1995), (Whistler, British Columbia), p. 36, Sept. 1995.

[29] P. Robertson, \Illuminating the structure of code and decoder of parallel concatenated recursive
systematic (turbo) codes," in Proc. IEEE Global Communications Conference (GLOBECOM 1994),
(San Francisco, California), pp. 1298{1303, Dec. 1994.
[30] L. C. Perez, J. Seghers, and D. J. Costello Jr., \A distance spectrum interpretation of turbo codes,"
in IEEE Transactions on Information Theory, vol. IT-42, pp. 1698{1709, Nov. 1996.
[31] G. Battail, \A conceptual framework for understanding turbo codes," in IEEE Journal on Selected
Areas in Communications, vol. SAC-16, pp. 245{254, Feb. 1998.

[32] R. J. McEliece, D. J. C. MacKay, and J.-F. Cheng, \Turbo decoding as an instance of Pearl's belief
propagation algorithm," in IEEE Journal on Selected Areas in Communications, vol. SAC-16,
pp. 140{152, Feb. 1998.
[33] A. J. Viterbi, \An intuitive justi cation and a simpli ed implementation of the MAP decoder for
convolutional code," in IEEE Journal on Selected Areas in Communications, vol. SAC-16, pp. 260{
264, Feb. 1998.
[34] D. Divsalar and F. Pollara, \On the design of turbo codes," TDA Progress Report 42-123, Jet
Propulsion Laboratory, Nov. 1995.
[35] D. Divsalar and F. Pollara, \Multiple turbo codes for deep-space communications," TDA Progress
Report 42-121, Jet Propulsion Laboratory, May 1995.
[36] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, \A soft-input soft-output maximum a
posteriori (MAP) module to decode parallel and serial concatenated codes," TDA Progress Report
42-127, Jet Propulsion Laboratory, Nov. 1996.
REFERENCES 109
[37] S. Dolinar, D. Divsalar, and F. Pollara, \Code performance as a function of block size," TMO
Progress Report 42-133, Jet Propulsion Laboratory, May 1998.
[38] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, \Analysis, design and iterative decoding
of double serially concatenated codes with interleavers," in IEEE Journal on Selected Areas in
Communications, vol. SAC-42, pp. 231{244, Feb. 1998.

[39] D. Divsalar and F. Pollara, \Serial and hybrid concatenated codes with applications," in Proc. 1st
International Symposium on Turbo codes, (Brest, France), pp. 80{87, Sept. 1997.

[40] C. Heegard and S. B. Wicker, Turbo Coding. Kluwer Academic Publishers, 1999.
[41] B. Vucetic and J. Yuan, Turbo Codes. Kluwer Academic Publishers, 2000.
[42] W. E. Ryan, \Concatenated convolutional codes and iterative decoding," in Wiley Encyclopedia of
Telecommunications (J.G.Proakis, ed.), New York: Wiley and Sons, 2002.