See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/3079617

Applications of Error-Control Coding

Article  in  IEEE Transactions on Information Theory · November 1998

DOI: 10.1109/18.720548 · Source: IEEE Xplore

CITATIONS READS
789 6,991

4 authors, including:

Hideki Imai
The University of Tokyo
839 PUBLICATIONS   15,555 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Leader View project

All content following this page was uploaded by Hideki Imai on 19 September 2014.

The user has requested enhancement of the downloaded file.

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998 2531

Applications of Error-Control Coding

Daniel J. Costello, Jr., Fellow, IEEE, Joachim Hagenauer, Fellow, IEEE,
Hideki Imai, Fellow, IEEE, and Stephen B. Wicker, Senior Member, IEEE

(Invited Paper)

Abstract— An overview of the many practical applications of where we assume perfect Nyquist signaling, is the average
channel coding theory in the past 50 years is presented. The fol- signal energy in each signaling interval of duration ,
lowing application areas are included: deep space communication, and is the two-sided noise power spectral density. (In
satellite communication, data transmission, data storage, mobile
communication, file transfer, and digital audio/video transmis- this formulation of the capacity theorem, one quadrature (two-
sion. Examples, both historical and current, are given that typify dimensional) signal is transmitted in each -second signaling
the different approaches used in each application area. Although interval, and the nominal channel bandwidth . See
no attempt is made to be comprehensive in our coverage, the Wozencraft and Jacobs [2] for a more complete discussion of
examples chosen clearly illustrate the richness, variety, and im-
portance of error-control coding methods in modern digital
the concept of channel bandwidth.) The proof of the theorem
applications. demonstrates that for any transmission rate less than or equal
to the channel capacity , there exists a coding scheme that
Index Terms— Block codes, channel coding, convolutional
codes, error-control coding. achieves an arbitrarily small probability of error; conversely,
if is greater than , no coding scheme can achieve reliable
performance. However, since this is an existence theorem,
I. INTRODUCTION it gives no guidance as to how to find appropriate coding
schemes or how complex they may be to implement.
W ITH his 1948 paper, “A Mathematical Theory of
Communication,” Shannon [1] stimulated a body of
research that has evolved into the two modern fields of
Beginning with the work of Hamming, which was published
in 1950 [3] but was already known to Shannon in 1948,
Information Theory and Coding Theory. The fundamental many communication engineers and coding theorists have
philosophical contribution of [1] was the formal application of developed numerous schemes for a variety of applications in
probability theory to the study and analysis of communication an attempt to achieve performance close to what was promised
systems. The theoretical contribution of Shannon’s work in the by Shannon with reasonable implementation complexity. In
area of channel coding was a useful definition of “information” this paper, we will survey the progress made in applying
and several “channel coding theorems” which gave explicit error-control coding techniques to digital communication and
upper bounds, called the channel capacity, on the rate at storage systems over the past 50 years and see that great
which “information” could be transmitted reliably on a given advances have occurred in designing practical systems that
communication channel. narrow the gap between real system performance and channel
In the context of this paper, the result of primary interest capacity. In particular, we will focus on applications in six
is the “noisy channel coding theorem for continuous channels areas: space and satellite communications (Section II), data
with average power limitations.” This theorem states that the transmission (Section III), data storage (Section IV), digital
capacity of a bandlimited additive white Gaussian noise audio/video transmission (Section V), mobile communications
(AWGN) channel with bandwidth , a channel model that ap- (Section VI), and file transfer (Section VII). Included among
proximately represents many practical digital communication the applications covered in this survey are the Consultative
and storage systems, is given by Committee on Space Data Systems (CCSDS) standard coding
bits per second (bps) (1) scheme for space and satellite communications, trellis coding
standards for high-speed data modems, the Reed–Solomon
Manuscript received March 2, 1998; revised June 1, 1998.
D. J. Costello, Jr. is with the Department of Electrical Engineering,
coding scheme used in compact discs, coding standards for
University of Notre Dame, Notre Dame, IN 46556 USA (phone: +1-219- mobile cellular communication, and the CRC codes used in
631-5480, fax: +1-219-631-4393, e-mail: [email protected]). HDLC protocols. The reader may wish to consult the paper
J. Hagenauer is with the Institute of Communications Engineering (LNT),
Munich University of Technology (TUM), 80290 Munich, Germany (phone.:
published in 1974 by Jacobs [4], which reviewed applications
+49-89-28923467, fax: +49-89-28923490, e-mail: [email protected] of error-control coding over the first 25 years after the pub-
Muenchen.DE). lication of Shannon’s paper, to get an appreciation for the
H. Imai is with the Institute of Industrial Science, University of Tokyo,
Minato-ku Tokyo, 106-8558, Japan (phone: +81-3-3402-6231, fax: +81-3- accelerated rate at which coding techniques have been applied
3402-6425, e-mail: [email protected]). to real systems in recent years.
S. B. Wicker is with the School of Electrical Engineering, Cornell Univer- The result in (1) can be put into a form more useful for
sity, Ithaca, NY 14853 USA (phone: +1-607-255-8817, fax: +1-607-255-9072,
e-mail: wicker@ee. cornell.edu). the present discussion by introducing the parameter , called
Publisher Item Identifier S 0018-9448(98)06086-6. the spectral (or bandwidth) efficiency. That is, represents the
0018–9448/98$10.00  1998 IEEE
2532 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 1. Capacity curves and the performance of several coding schemes for data transmission applications.

average number of information bits transmitted per signaling scheme, no matter how complex, sending information
interval of duration seconds. Then bit per signal with an less than 0 dB will be unreliable.
(2) This interpretation indicates the maximum power reduction
available using appropriate coding schemes compared to an
and uncoded system. As an illustration, consider that an uncoded
(3) quadrature phase-shift keyed (QPSK) system with coherent
detection has a spectral efficiency of and achieves a bit-
where is the average energy per information bit. Substitut- error rate (BER) of at an dB. Since coding
ing the above relations into (1) and performing some minor can provide essentially error-free communication at the same
manipulations yields spectral efficiency and an dB, we say that a
(4) maximum power (or coding) gain of 7.8 dB is available in
which relates the spectral efficiency to the signal-to-noise ra- this case.
tio (SNR), represented by . The bound of (4) expresses Second, the capacity bound may be interpreted as giving the
the fundamental tradeoff between the spectral efficiency and maximum spectral efficiency at which a system may operate
the SNR . That is, increased spectral efficiency can be reliably for a fixed SNR. For example, if an dB
reliably achieved only with a corresponding increase in the is available, then there exists a coding scheme that operates
minimum required SNR. Conversely, the minimum required reliably with a bandwidth efficiency of information bit
SNR can be reduced only by decreasing the spectral efficiency per signal. Conversely, any coding scheme, no matter how
of the system. complex, sending more than information bit per signal
The bound of (4) is shown plotted in Fig. 1 and is labeled will be unreliable if dB. This interpretation indi-
the capacity bound. This curve represents the absolute best cates the maximum spectral efficiency increase available using
performance possible for a communication system on the appropriate coding schemes compared to an uncoded system.
AWGN channel. The performance of a particular system As an illustration, again consider an uncoded QPSK system
relative to the capacity bound may be interpreted in two with coherent detection. Since a coded system operating at
distinct ways. an dB can provide reliable communication at a
First, the capacity bound may be interpreted as giving spectral efficiency of information bits per signal, we
the minimum SNR required to achieve a specific spectral say that a maximum spectral efficiency (or rate) gain of 3.7
efficiency with an arbitrarily small probability of error. For bits per signal is available in this case.
example, if one wants to transmit information bit The capacity bound assumes that, for a given spectral effi-
per signal, then there exists a coding scheme that operates ciency, one is free to choose the signaling (modulation) scheme
reliably with an dB. Conversely, any coding which results in the best possible performance. However, in
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2533

real communication systems, there are many practical consid- many early applications of coding were developed for the
erations that come into play in choosing a modulation scheme. National Aeronautics and Space Administration (NASA) and
For example, satellite communication systems that use nonlin- the European Space Agency (ESA) deep-space and satellite
ear traveling-wave tube amplifiers (TWTA’s) require constant communication systems, where power was very expensive and
envelope signaling such as -ary phase-shift keying ( - bandwidth was plentiful. Second, no practical coding schemes
PSK) in order to avoid signal distortion. Also, for data trans- existed that could provide meaningful power gains at higher
mission over voice-grade telephone channels, the intersymbol spectral efficiencies. Thus our paper begins with a survey
interference caused by bandwidth limitations necessitates the of applications of error-control coding to space and satellite
use of large signal constellations that employ a combination communication systems.
of amplitude and phase modulation (AM/PM) to achieve high The deep-space channel turned out to be the perfect link on
data rates. It is, therefore, instructive to compute the maximum which to first demonstrate the power of coding. There were
spectral efficiency required to achieve reliable communica- several reasons for this, most notably those listed below.
tion given a particular modulation scheme and SNR. 1) The deep-space channel is almost exactly modeled as
For the discrete-input, continuous-output, memoryless the memoryless AWGN channel that formed the basis
AWGN channel with -ary phase-shift-keyed ( -PSK) or for Shannon’s noisy channel coding theorem. Thus all
quadrature amplitude modulation ( -QAM) modulation and the theoretical and simulation studies conducted for this
assuming equiprobable signaling, the capacity bound becomes channel carried over almost exactly into practice.
[2] 2) Plenty of bandwidth is available on the deep-space
channel, thus allowing the use of the low-spectral-
efficiency codes and binary-modulation schemes that
were most studied and best understood at the time.
3) Because of the large transmission distances involved,
(5) which caused severe signal attenuation, powerful, low-
rate codes, with complex decoding methods, were re-
where is a channel signal, is a Gaussian distributed noise quired, resulting in very low data rates. However, since a
random variable with mean and variance , and is deep-space mission is, by nature, a very time-consuming
the expectation operator. The bound of (5) is plotted in Fig. 1 process, the low data rates realized in practice did not
for equiprobable QPSK, 8-PSK, 16-QAM, 32-QAM, 64-QAM, present a problem.
128-QAM, and 256-QAM modulation. (For -QAM signal 4) A deep-space mission is also, by nature, a very ex-
sets with large , nonequiprobable signaling, called signal pensive undertaking, and thus the additional cost of
shaping, can improve performance by as much as 1.53 dB developing and implementing complex encoding and
compared to equiprobable signaling [5].) decoding solutions can be tolerated, especially since
For a specified signaling method and SNR, the bound of each decibel of coding gain realized resulted in an
(5) represents the maximum spectral efficiency required to overall savings of about $1 000 000 (in the 1960’s) in
achieve reliable communication. For example, a QPSK system transmitting and receiving equipment.
operating with an dB can transmit a maximum Thus it is no surprise that Massey, in a recent paper [6], called
of bits per signal. This is 0.44 bit per signal less deep-space communication and coding a “marriage made in
than an ideal system without any modulation constraints. heaven.”
Alternatively, to send information bits per signal As a starting point to understand the gains afforded by
using QPSK modulation requires a minimum coding on the deep-space channel, we consider an uncoded
dB. This is 0.76 dB more than an ideal system without any BPSK system with coherent detection. (Throughout this sec-
modulation constraints. tion we assume BPSK modulation, which transmits uncoded
In the sections that follow, we will have occasion to information at a rate of 1.0 information bit per signal. If coding
compare some of the coding systems discussed to the capacity is used, the code rate , in information bits per BPSK signal,
curves in Fig. 1 in order to assess their relative power and/or then represents the spectral efficiency of the coded system.)
spectral efficiency, at least in the cases where AWGN is the Simulation results and analytical calculations have shown that
primary signal impairment. In other cases, such as mobile uncoded BPSK achieves a BER of , considered “reliable”
communications, where signal fading is the major source of in many applications, at an dB. This point
errors, and data storage, where media contamination causes is plotted in Fig. 2 with the label BPSK. (All the points
most of the problems, a clear view of optimum performance in Figs. 1 and 2 are plotted for a BER of . Actual
is not as readily available. BER requirements vary from system to system in deep-space
applications depending on several factors, such as the nature
II. APPLICATIONS TO SPACE AND and sensitivity of the data and whether it has been compressed
SATELLITE COMMUNICATIONS on the spacecraft prior to transmission.)
Most of the early work in coding theory was directed at From the capacity curve in Fig. 2, it can be seen that the
the low spectral efficiency, or power-limited, portion of the minimum required to achieve error-free communica-
capacity curve. This was principally due to two factors. First, tion at a code rate bit per signal is 0.0 dB, and
2534 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 2. Capacity and cutoff rate curves and the performance of several coding schemes for deep-space applications.

thus a power savings of 9.6 dB is theoretically possible with is equivalent to the spectral efficiency.) This point is plotted in
an appropriate rate coding scheme. Looking at the Fig. 2 with the label “Mariner.” From Fig. 2, it is seen that the
BPSK capacity curve, however, reveals that to achieve a rate Mariner code requires 3.2 dB less power than uncoded BPSK
with BPSK modulation requires only a slightly larger for the same BER, but it requires more than five times the
dB. Thus for code rates or less, very bandwidth and is still 7.5 dB away from the BPSK capacity
little in potential coding gain is sacrificed by using BPSK curve at the same specrtral efficiency. It is important to note
modulation. This, combined with the difficulty in coherently that even with its significant bandwidth expansion, the coding
detecting signal sets with more than two points and the relative gain actually achieved by the Mariner code was rather modest.
abundance of bandwidth on the deep-space channel, resulted This is due to the fact that this code, as is typical of block codes
in the choice of BPSK modulation with code rates in general, has a relatively large number of nearest neighbor
bit/signal or below for deep-space communication. codewords, thus substantially reducing the available coding
One of the earliest attempts to improve on the performance gain at moderate BER’s. (At lower BER’s, a coding gain of up
of uncoded BPSK was the use of a rate biorthogonal, to 4.8 dB is achievable, but this is reduced to 3.2 dB at a BER
Reed–Muller block code, also referred to as the RM of by the code’s 62 nearest neighbors.) In fact, most of
code. This code was used on the 1969 Mariner and later the coding gain achieved by the Mariner code was due to the
Viking Mars missions in conjunction with BPSK modulation extra 2–3 dB obtained by using full soft-decision decoding,
and soft-decision maximum-likelihood decoding. The code a lesson that has carried over to almost all practical coding
consisted of 64 codewords, each 32 bits long, with a minimum implementations where coding gain is a primary consideration.
Hamming distance between codewords of . The 64 A significant advance in the application of coding to deep-
codewords can be viewed as a set of 32 orthogonal vectors space communication systems occurred later in the 1960’s
in 32-dimensional space, plus the complements of these 32 with the invention of sequential decoding [8] for convolutional
vectors, and thus the name “biorthogonal.” Full soft-decision codes and its subsequent refinement [9]. It was now possible
maximum-likelihood decoding (decoding using unquantized to use powerful long-constraint-length convolutional codes
demodulator outputs) was achieved by using a correlation with soft-decision decoding. Thus for the first time, practical
decoder, based on the Hadamard transform, developed by communication systems were capable of achieving substantial
Green at the Jet Propulsion Laboratory (JPL) that subsequently coding gains over uncoded transmission.
became known as the “Green Machine” [7]. Sequential decoding was first used in 1968 on an “exper-
The Mariner system had code rate imental” basis. (This was actually a deliberate stratagem to
bit/signal and it achieved a BER of with an circumvent lengthy NASA qualification procedures [6].) The
dB. (Note that, since BPSK modulation is assumed, each Pioneer 9 solar orbit space mission used a modified version
BPSK signal represents one code bit, and thus the code rate of a rate systematic convolutional code originally
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2535

constructed by Lin and Lyne [10], but the coding scheme was retransmission is possible, this variable decoding delay prop-
changed for subsequent missions. (A convolutional code is said erty of sequential decoding can be an advantage. For example,
to be in systematic form if the information sequence appears when a long delay occurs in decoding, indicating a very noisy
unchanged as one of the encoded sequences.) It is interesting to and therefore probably unreliable frame of data, the decoder
note that, even though the Mariner coding system was designed can simply stop and erase the frame, not delivering anything
first, the Pioneer 9 was actually launched earlier, and thus the to the user, or ask for a retransmission. A so-called “complete”
Lin–Lyne code was the first to fly in space. The Pioneer 10 decoder, on the other hand, would be forced to deliver a
Jupiter fly-by mission and the Pioneer 11 Saturn fly-by mission decoded estimate, which may very well be wrong in these
in 1972 and 1973, respectively, both used a rate , cases, resulting in what has been termed a “fools rush in
constraint length , i.e., a nonsystematic, where angels fear to tread” phenomenon [11]. However, fixed
quick-look-in (QLI) convolutional code constructed by Massey delay is desirable in many situations, particularly when high-
and Costello [11]. The two 32-bit code generator sequences speed decoding is required. In addition, the performance of
used for this code are given in octal notation by convolutional codes with sequential decoding is ultimately
limited by the computational cutoff rate (the rate at
(6)
which the average number of computations performed by
The code was chosen to be nonsystematic in order to give a sequential decoder becomes unbounded), which requires
it a larger minimum free Hamming distance , in this SNR’s higher than capacity to achieve reliable communication
case , compared to the best systematic code of at a given code rate [2], as shown in Fig. 2. For example, to
the same constraint length. This is true for convolutional achieve reliable communication at a code rate of
codes in general, i.e., for a given constraint length, a measure bit/signal using sequential decoding and BPSK modulation on
of decoding complexity, more free distance, and thus better the AWGN channel requires an dB, whereas
performance, can be achieved using a nonsystematic rather the capacity bound only requires an dB. The
than a systematic code. The fact that the two generators at which the Pioneer code achieves a BER of is
differ in only one bit position gives the code the “quick- only 0.3 dB away from the cutoff rate, and thus there is little
look” property, i.e., the capability of obtaining a reasonably to be gained with longer constraint length codes and sequential
accurate quick estimate of the information sequence from the decoding at this code rate and BER.
noisy received sequence prior to actual decoding. This is an These undesirable characteristics of sequential decoding
important capability in some situations that is always available and the possibility of higher decoding speeds led to the use
with systematic codes, but not, in general, with nonsystematic of maximum-likelihood Viterbi decoding [14] in the next
codes. Requiring this capability does result in some reduction generation of deep-space communication systems. The Viterbi
in free distance, however, and thus represents a compromise algorithm, like sequential decoding, is compatible with a vari-
between choosing the best possible code and retaining the ety of modulation and quantization schemes. Unlike sequential
“quick-look” property. Nevertheless, the above code has had decoding, though, the Viterbi algorithm has a fixed number of
a long and distinguished career in space, having been used, in computations per decoded branch and thus does not suffer
addition to the above two missions, on the Pioneer 12 Venus from incomplete decoding and, ultimately, is not limited by a
orbiter and the European Helios A and Helios B solar orbiter computational cutoff rate.
missions. The Voyager 1 and 2 space missions were launched in 1977
A sequential decoder using a modified version of the Fano to explore Jupiter and Saturn. They both used a
tree-searching algorithm with 3-bit soft decisions (3-bit quan- nonsystematic convolutional code with generator polynomials
tized demodulator outputs) was chosen for decoding. For
lower speed operation, in the kilobit-per-second (kbps) range,
decoding could be done in software. Faster hardware decoders (7)
were also developed for operation in the megabit-per-second
(Mbps) range [12], [13]. This scheme had code rate and . This code and a companion code
bit/signal and achieved a BER of at an with generators
dB (see Fig. 2, “Pioneer”), thus achieving a 6.9-
dB coding gain compared to uncoded BPSK, at the expense
of a doubling in bandwidth requirements. This represented a (8)
significant improvement compared to the Mariner system and
resulted in performance only 2.5 dB away from the BPSK
capacity curve. An excellent discussion of the considerations and , were both adopted as NASA/ESA Planetary
that went into the design of these early deep-space coding Standard Codes by the Consultative Committee on Space
systems is included in the paper by Massey [6]. Data Systems (CCSDS). The code has also been
Sequential decoding algorithms have a variable computation employed in numerous other applications, including satellite
characteristic which results in large buffering requirements, communication and cellular telephony, and has become a de
and occasionally large decoding delays and/or incomplete facto industry standard [15].
decoding of the received sequence. In some situations, such The above codes were decoded using a 3-bit soft-decision
as when almost error-free communication is required or when maximum-likelihood Viterbi decoder. Since the complexity
2536 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 3. The CCSDS concatenation standard.

of Viterbi decoding grows exponentially with code constraint be designed to correct almost all the residual errors from
length, it was necessary to choose these short constraint length the inner decoder, resulting in nearly error-free performance
codes rather than the long constraint length Pioneer codes used (BER’s, say, on the order of ). The most common
with sequential decoding. The codes chosen have a arrangement is a combination of a short constraint length inner
decoding trellis containing 64 states, considered reasonable in convolutional code with soft-decision Viterbi decoding and
terms of implementation complexity. The performance of these a powerful nonbinary Reed–Solomon (RS) outer code. This
codes is plotted in Fig. 2 with the label “Planetary Standard”. combination was eventually accepted in 1987 as the CCSDS
The code requires an dB to operate at Telemetry Standard [19].
a BER of . Though this code results in a 5.1-dB power In fact, though, several other concatenation schemes had
advantage compared to uncoded transmission, its performance been tried earlier. For example, on the 1971 Mariner mission,
is 1.8 dB worse than the Pioneer system, due to the short a RS outer code with symbols drawn from GF
constraint length used. However, its decoder implementation was used in conjunction with the RM code as an
complexity is simpler than a sequential decoder, it does not inner code, and on the two 1977 Voyager missions, a
suffer the long buffering delays characteristic of sequential extended Golay outer code was used together with the
decoding, and, because of its regular trellis structure, it is Planetary Standard code as an inner code [20]. In both cases,
adaptable to parallel implementation, resulting in decoding the data to be transmitted consisted mostly of uncompressed
speeds in the 100’s of Mbps [16]. image information, along with small amounts of sensitive
The Planetary Standard also played a major role in military scientific information. The outer codes were used only to
satellite communications well into the 1980’s (as incorporated give added protection to the scientific information, so that the
into the Satellite Data Link Standard (SDLS)). In general, overall coding rate was not reduced much below the inner-code
convolutional encoding with Viterbi decoding will continue rates. In the case of the Mariner system, each 6-bit symbol
to be used in earth-orbiting satellite communication systems from the outer code is encoded into one 32-bit codeword in the
well into the next century. The Globalstar and Iridium systems inner code. This “matching” between the outer code symbol
use , rate and , rate convolutional size and the information block size of the inner code means
codes, respectively. The rationale for the differing constraint that each block error from the inner decoder causes only one
lengths and rates lies with the nominal lengths (and consequent symbol error for the outer decoder, a desirable property for a
space loss) of the satellite-to-ground links for the two systems. concatenation scheme consisting of two block codes. Finally,
Globalstar satellites operate at altitudes of approximately 1400 although both inner decoders made use of soft-decision inputs
km, while Iridium operates at half that height [17]. from the channel, the outer decoders were designed to work
Coding gain beyond that provided by the Planetary Standard directly with the “hard decisions” made by the inner decoders.
can be achieved using code concatenation. Concatenation is a Outer decoders which also make use of soft-decision inputs
scheme first introduced by Forney [18] in which two codes, will be considered later in this section.
an “inner code” and an “outer code,” are used in cascade. The CCSDS Standard concatenation scheme consists of
The inner code should be designed to produce a moderate the Planetary Standard inner code along with a
BER (typically on the order of to ) with modest RS outer code, as shown in Fig. 3. (Note that the
complexity. The outer code can be more complex and should CCSDS Standard assumes that all the data is protected by
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2537

both codes, inner and outer, although it is clearly possible to the BPSK capacity curve. (See Fig. 2, “BVD.”) Compared
protect some data using only the inner code or to send some to the Planetary Standard code, it gains a full 2.8
data without any protection at all.) The RS code consists of dB. In a concatenated system with the RS outer
8-bit symbols chosen from the finite field GF based on code, the code requires an of just 0.9 dB
the primitive polynomial to achieve a BER of , is within 2.0 dB of capacity, and
is 1.6 dB more power efficient than the Voyager system. (See
(9)
Fig. 2, “Galileo.”) Although the above rate systems are
The generator polynomial (in cyclic code form) is given by 50% less bandwidth efficient than their rate counterparts,
it should be recalled that bandwidth is plentiful in deep space,
(10) and thus it is common to sacrifice spectral efficiency for added
power efficiency. In fact, an even less spectrally efficient
code is currently scheduled to be flown aboard the
where is a root of . From (10), we see that has 32 Cassini mission to Saturn [25]. However, further bandwidth
first-order roots, giving it degree 32, and thus the code contains expansion may be difficult to achieve due to the fact that
32 redundant symbols. Since RS codes are maximum-distance additional redundancy may reduce the energy per transmitted
separable (MDS), their minimum distance is always one more symbol below the level needed for reliable tracking by the
than the number of redundant symbols. Hence this code has phase-locked loops in the coherent demodulator.
and can correct any combination of 16 or fewer As a further improvement on the CCSDS Concatenation
symbol errors within a block of 255 symbols (2040 bits). Hard- Standard, errors-and-erasures decoding, a suboptimum form
decision decoding of the outer code is performed using the of soft-decision decoding, can be used to provide some per-
Berlekamp–Massey algorithm [21], [22]. Finally, in order to formance improvement if erased symbols are available from
break up possibly long bursts of errors from the inner decoder the inner decoder. One method of providing erased symbols is
into separate blocks for the outer decoder, thus making them based on two facts mentioned above: 1) a frame of several RS
easier to decode, a symbol interleaver is inserted between the code blocks is interleaved prior to encoding by the inner code,
inner and outer codes. Interleaver depths of between two and and 2) decoding errors from the inner decoder are typically
eight outer-code blocks are typical [23]. bursty, resulting in strings of consecutive error symbols.
With the CCSDS Standard concatenation scheme, the Although long strings of error symbols will usually cause
needed to achieve a BER of is reduced by a problems for an RS decoder, after de-interleaving they are
full 2.0 dB compared to using the Planetary Standard code more spread out, making them easier to decode. In addition,
alone, with only a slight reduction in code rate (from once a symbol error has been corrected by the RS decoder,
to ). The performance of the code in a symbols in the corresponding positions of the other codewords
concatenated system with the RS outer code is in the same frame can be flagged as erasures, thus making
shown in Fig. 2 with the label “Voyager.” The concatenated them easier to decode. This technique is known as “error
Voyager system operates in the same region as the forecasting” and has been discussed in a paper by Paaske [26].
Pioneer system, gaining about 0.2 dB with a 12.5% loss in Another method of improving the CCSDS Concatenation
rate. Thus short constraint length convolutional codes with Standard makes use of iterative decoding. In one approach, the
Viterbi decoding in concatenated systems can be considered RS codes in a given frame are assigned different rates, some
as alternatives to long constraint length codes with sequential higher that the code and some lower, such that
decoding. the average rate is unchanged. After an initial inner decoding
Variations on the concatenated CCSDS theme continue to of one frame, the most powerful (lowest rate) outer code is
be used at the end of the century in near-earth applications. decoded, and then its decoded information bits (correct with
For example, in 1998 the DirecTV satellite system was using very high probability) are fed back and treated as known
a concatenated convolutional-Viterbi/Reed–Solomon system to information bits (side information) by the inner decoder in a
bring television signals to three million subscribers. second iteration of decoding. This procedure can be repeated
Recently, technological advances have made it practical to until the entire frame is decoded, with each iteration using the
build maximum-likelihood Viterbi decoders for larger con- lowest rate outer code not yet decoded. The use of known
straint length convolutional codes. The culmination of this information bits by the inner decoder has been termed “state
effort was the Big Viterbi Decoder (BVD) designed by Collins pinning,” and the technique is discussed in a paper by Collins
to decode a code. The BVD was constructed at JPL and Hizlan [27].
for use on the Galileo mission to Jupiter [24]. The decoder A more general approach to iterative decoding of con-
trellis has states, making internal decoder catenated codes was proposed by Hagenauer and Hoeher
communication a formidable task, but decoding speeds up to with the introduction of the Soft-Output Viterbi Algorithm
1 Mbps were still achieved. The code has octal generators (SOVA) [28]. In the SOVA, reliability information about
each decoded bit is appended to the output of a Viterbi
(11) decoder. An outer decoder which accepts soft inputs can then
use this reliability information to improve its performance.
and , clearly a very powerful code. It achieves a If the outer decoder also provides reliability information
BER of at an dB, only 2.6 dB away from at its output, iterative decoding can proceed between the
2538 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 4. The “turbo” encoding/decoding system.

inner and outer decoders. In general, such iterative decoding parity generator separated by a pseudorandom interleaver. The
techniques for concatenated systems can result in additional generator matrix is given by
coding gains of up to about 1.0 dB. In the CCSDS system,
however, the outer RS decoder cannot make full use of such (12)
reliability information. Nevertheless, several combinations of
error forecasting, state pinning, and iterative decoding have (The code is the same as that generated by a conventional non-
been applied to the CCSDS system by various researchers, systematic feedforward encoder with generator polynomials
resulting in an additional coding gain of about 0.5 dB [20], and , but
[23]. it is important that it be encoded in systematic feedback form
As our final topic in this section, we discuss a significant because of the way the interleaver combines the two parity
new discovery called “turbo codes,” which is currently being sequences.) The encoder output consists of the information
considered as a software upgrade for the Cassini mission. sequence and two parity sequences, thus representing a code
Turbo codes, which are also known as parallel concatenated rate of . Alternately puncturing (deleting) bits from the
convolutional codes, were first introduced in a paper by two parity sequences produces a code rate of , and other
Berrou, Glavieux, and Thitimajshima [29]. Turbo codes com- code rates can be achieved by using additional parity gener-
bine a convolutional code along with a pseudorandom inter- ators and/or different puncturing patterns. The pseudorandom
leaver and maximum a posteriori probability (MAP) iterative interleaver re-orders the information sequence before encoding
decoding to achieve performance very close to the Shannon by the second parity generator, thus producing two different
limit. A block diagram of the turbo encoding/decoding system parity sequences. In essence, the interleaver has the effect of
is shown in Fig. 4. The encoder employs a simple matching “bad” (low-weight) parity sequences with “good”
code in systematic feedback form, using two copies of the (higher weight) parity sequences in almost all cases, thus
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2539

generating a code with very few low-weight codewords. For This changed dramatically with Ungerboeck’s discovery of
large enough information sequence blocklengths, performance trellis-coded modulation (TCM) [35].
very close to the Shannon limit can be achieved at moderate In his 1982 paper, Ungerboeck constructed trellis codes
BER’s, even though the free distance of turbo codes is not for amplitude modulation (AM), phase-shift-keyed (PSK), and
large. In fact, for most interleavers, the free distance of the quadrature amplitude modulation (QAM) modulation schemes.
above code is only , even for very long blocklengths. As an example, consider the simple four-state, 8-PSK code
The excellent performance at moderate BER’s is due rather shown in Fig. 5 along with its trellis diagram. The encoder
to a drastic reduction in the number of nearest neighbor has two input bits, and . The input bit is differentially
codewords compared to a conventional convolutional code. encoded to produce the encoded bit . (Differential encoding
In a paper by Benedetto and Montorsi [30], it was shown that of this bit is needed to take advantage of the “rotational
the number of nearest neighbors is reduced by a factor of , invariance” property of the code so that the correct information
where is the blocklength. This factor is referred to as the sequence can be recovered after decoding in the event the
“interleaver gain.” receiver demodulator locks onto the wrong phase. This is
The other important feature of turbo codes is the iterative an important practical consideration which will be discussed
decoder, which uses a soft-in/soft-out MAP decoding algo- further later in this section.) The input bit is encoded by
rithm first applied to convolutional codes by Bahl, Cocke, a systematic feedback convolutional encoder with
Jelinek, and Raviv [31]. This algorithm is more complex generator matrix
than the Viterbi algorithm by about a factor of three, and for
conventional convolutional codes it offers little performance (13)
advantage over Viterbi decoding. However, in turbo decoding,
the fact that it gives the maximum MAP estimate of each producing the two encoded bits (an information bit)
individual information bit is crucial in allowing the iterative and (a parity bit). (It should be noted here that the reason
decoding procedure to converge at very low SNR’s. Although for using systematic feedback encoders in TCM has nothing
the SOVA can also be used to decode turbo codes, significant to do with the reason for using them in turbo coding. In
improvement can be obtained with MAP decoding [32]. TCM, nonsystematic feedforward encoders can be used with
At almost any bandwidth efficiency, performance less than similar results, but systematic feedback encoders are preferred
1.0 dB away from capacity is achievable with short constraint because they provide a convenient canonical representation for
length turbo codes, very long blocklengths, and 10–20 itera- the high-rate codes, typically , commonly used
tions of decoding. In Fig. 2, the point marked “Turbo Code” in TCM and because the noisy received information bits can
shows performance at a BER of for rate and be directly accessed prior to decoding.) The signal mapper
information blocklength , with 18 iterations (modulator) then generates an 8-PSK signal according to the
of decoding. This is a full 3.8 dB better than the Planetary mapping shown in Fig. 5. Ignoring the differential encoding,
Standard code, with roughly the same decoding at each time unit the mapper input consists of two information
complexity, and is also 1.0 dB better than the very complex bits, and , and one parity bit, , and the mapper output
BVD code, which operates at 50% less spectral efficiency! consists of one 8-PSK signal. Thus the information rate, or
Using the same rate of , the turbo code outperforms spectral efficiency, of this scheme is bits/signal, the
the BVD code by about 1.9 dB. The major disadvantages same as an uncoded system using QPSK modulation. (The
of a turbo code are its long decoding delay, due to the fact that only one information bit, , enters the convolutional
large blocklengths and iterative decoding, and its weaker encoder and the other information bit, , remains “uncoded”
performance at lower BER’s, due to its low free distance. is a feature of many, but not all, TCM schemes. The best code
The long delays are not a major problem except in real-time design for a given situation may or may not contain uncoded
applications such as voice transmission, and performance at information bits, depending on the signal constellation, code
lower BER’s can be enhanced by using serial concatenation complexity, and spectral efficiency.) Decoding is performed by
[33], so turbo codes seem to be ideally suited for use on many a conventional soft-decision Viterbi decoder operating directly
future deep-space missions. on the noisy received 8-PSK signals. In this example, the
A comprehensive survey of the application of coding to decoding trellis has four states, as shown in Fig. 5. Because
deep-space communication as the decade of the 1960’s drew the code has one uncoded information bit, the trellis contains
to a close was given in a paper by Forney [34]. For a more so-called “parallel transitions,” i.e., parallel paths of length one
recent review of the subject, the article by Wicker [25] is an branch each connecting pairs of states at different time units.
excellent source. Parallel transitions in the trellis diagram are characteristic of
all TCM schemes that contain uncoded information bits.
The first important insight due to Ungerboeck was his
III. APPLICATIONS TO DATA TRANSMISSION realization that for the nonbinary signal constellations used
As mentioned previously in this paper, the focus of much in TCM, minimum free squared distance in Euclidean
of the early work in coding theory was on reducing power space, rather than minimum free Hamming distance , is
requirements at low spectral efficiencies. This was due, in part, the primary parameter that determines code performance. His
to the fact that no practical coding schemes existed that could technique of “mapping by set partitioning,” as opposed to
provide meaningful power gains at higher spectral efficiences. the conventional Gray mapping used in uncoded modulation,
2540 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 5. The four-state 8-PSK Ungerboeck code.

turned out to be the key to achieving performance gains example, his 64-state, , -ary AM code requires an
with TCM compared to uncoded modulation at high spectral dB to achieve a BER of with soft-
efficiencies. One of the important goals of mapping by set decision Viterbi decoding. This code has the same spectral
partitioning is to insure that parallel transitions in the trellis are efficiency as uncoded BPSK and requires 3.4 dB less power
mapped into signals far apart in signal space, thus minimizing to achieve the same BER! Note, however, that a
the probability of short, one-branch, error events due to convolutional code can also be used with conventional Gray
uncoded bits. For the four-state, 8-PSK code considered above, mapped QPSK modulation to achieve a BER of with
, whereas for uncoded QPSK, , assuming an dB, 1.7 dB less than the AM code, thus
unit energy signals in both cases. Hence this simple TCM illustrating the advantage of using quadrature modulation when
scheme has a factor of 2, or 3 dB, asymptotic, i.e., high SNR, possible. (See Fig. 1, “ QPSK.”) As another example,
coding gain compared to uncoded QPSK. (The asymptotic Ungerboeck’s 64-state, , 8-PSK code has a spectral
coding gain of any code is always somewhat larger than its efficiency of bits/signal and requires an of
“real” coding gain at moderate BER’s, due to the effect of only 6.0 dB to reach a BER of with soft-decision Viterbi
nearest neighbor codewords. In this case, the coding gain decoding. (See Fig. 1, “ 8-PSK.”) Thus the four-state
at a BER is 2.4 dB, as shown in Fig. 1 with the and 64-state, 8-PSK Ungerboeck codes require 2.4 and 3.6
notation “ 8-PSK.”) Also, unlike the case with binary dB less power, respectively, than uncoded QPSK to achieve
modulation, where coding gains are achieved only at the the same BER with the same spectral efficiency. Finally, the
expense of bandwidth expansion, the TCM scheme has the 64-state code is 3.3 dB away from the 8-PSK capacity curve,
same spectral efficiency, viz., 2.0 bits/signal, as uncoded which means that more than half the potential gain of rate
QPSK. Thus the 3-dB asymptotic coding gain is achieved , 8-PSK TCM can be achieved with relatively modest
without bandwidth expansion! In the case of TCM, it is complexity.
expanding the size of the signal constellation, rather than For spectral efficiencies of bits/signal or greater,
expanding the bandwidth, that results in the possibility of two-dimensional (2D) or multidimensional (MD) QAM signal
coding gain. This fundamental realization by Ungerboeck has sets offer better performance than PSK signal sets. (MD
led to an entire new set of application areas for coding in the signal sets are formed by mapping encoded bits into more
range of high spectral efficiencies. than one constituent 2D signal point at a time.) QAM signal
Ungerboeck was able to construct a number of interest- sets more efficiently pack the 2D signal space and thus can
ing codes as a result of his original work [36], [37]. For achieve the same minimum distance between signal points,
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2541

which effectively limits the performance of a trellis code, the previous standard of 2400 symbols/s. (In V.34, the data
with less average energy per signal than a comparable PSK rates are not necessarily integer multiples of the symbol rates,
signal set. This performance advantage of QAM signal sets since the signal mapping technique, called shell mapping [42],
has led modem designers to choose QAM, and QAM-based allows the mapping of fractional bits per symbol.) The V.34
trellis codes, for use in high-speed data transmission. standard also includes two other codes which offer slightly
The performance of Ungerboeck’s codes quickly dispelled better performance at a cost of increased complexity [41]. One
the belief that power reduction is only attainable with a is a 32-state, , linear 4D code developed by Williams
corresponding decrease in spectral efficiency, as is the case [43] that gains 0.3 dB compared to the 16-state Wei code,
when we limit our choice of modulation to BPSK. This was a but is four times more complex. The other is a variation of a
welcome result for modem designers, who had been frustrated 64-state, , nonlinear 4D code also developed by Wei
in their attempts to go beyond data rates of 9600 bps [38]. [40] that gains an additional 0.15 dB over the 16-state code,
Since the International Telecommunications Union’s ITU-T but is 16 times as complex. In both these cases, more bits are
V.29 modem standard was adopted in 1976, little progress was encoded by the convolutional code, so fewer uncoded bits are
made in increasing the speed and quality of data transmission needed to achieve a particular data rate.
over voice-grade telephone lines until the appearance of the A block diagram of the eight-state, nonlinear 2D code
V.32 and V.33 standards in 1986, both of which included a developed by Wei [39] is shown in Fig. 6, along with a sketch
TCM encoder/modulator. The V.29 standard used uncoded of the 32-CROSS constellation adopted for the ITU-T V.32
16-QAM modulation and a 2400-symbol/s signaling rate to standard. Note that the encoder has four input information
achieve a spectral efficiency of bits/signal and bits, and . and are uncoded
a transmission speed of 9600 bps in a half-duplex (one- and directly enter the 32-CROSS signal mapper (modulator).
way) mode. Due to the bandwidth constraints of the channel, and are first differentially encoded and then enter the
signaling rates higher than 2400 symbols/s were not considered systematic feedback nonlinear convolutional encoder,
feasible. Thus the only avenue to increased data rates was producing three output bits: and (information bits) as
to expand the size of the signal constellation. However, well as (a parity bit). The five encoded bits
due to the SNR constraints of the channel, this meant that and then enter the modulator and are mapped into one
signals had to be packed closer together, resulting in degraded of the 32-CROSS signals, as shown in Fig. 6. Since one 32-
performance. So a clear need developed for a scheme which CROSS signal is transmitted for every four information bits
would allow constellation expansion at the same signaling entering the encoder, the spectral efficiency of the code is
rate, thus achieving higher data rates, and yet provide a bits/signal. (The V.33 standard uses the same code
coding gain to at least recover the noise margin lost by along with four uncoded information bits and a 128-CROSS
the closer packing of signals. TCM proved to be just such constellation to achieve a spectral efficiency of
a scheme and, combined with some sophisticated signal- bits/signal.) As noted in the above example, soft decision
processing techniques, has resulted in a series of improvements Viterbi decoding, using an eight-state trellis with four-fold (16-
which have pushed modem speeds up to 33 600 bps full- fold in the V.33 case) parallel transitions, is performed based
duplex. on the noisy received symbols at the demodulator output. Since
In the remainder of this section, we will limit our discus- the encoder contains two AND gates, it is nonlinear and cannot
sion to two examples: the eight-state, , nonlinear 2D be described using the usual generator sequence notation. The
code developed by Wei [39] and adopted by ITU-T for the nonlinear encoder was needed to make the code invariant to
V.32 and V.33 standards and the 16-state, , linear 90 phase rotations, i.e., a rotation of the signal space by any
four-dimensional (4D) code also developed by Wei [40] and multiple of 90 still results in a valid code sequence. For a
adopted by ITU-T as one of the codes for the V.34 standard. code with this property, as long as the bits affected by the
The V.32 standard uses two uncoded bits and a 32-CROSS phase rotation are differentially encoded, the correct sequence
signal constellation, for a spectral efficiency of can still be decoded if the demodulator locks onto the wrong
bits/signal and a 9600 bps data rate, and achieves about 3.5 phase by any multiple of 90 . For 2D signal constellations, it
dB better performance (due to trellis coding) and full-duplex is impossible to achieve 90 invariance with linear codes (the
operation (due to echo cancellation) compared to the uncoded best that can be done is 180 invariance), and hence nonlinear
V.29 standard operating at the same data rate. In the V.33 codes are needed for full rotational invariance. This was a
standard, four uncoded bits and a 128-CROSS constellation are crucial insight made by Wei [39] in the design of this code.
used to achieve bits/signal and a 14 400-bps data rate. The eight-state, , nonlinear 2D Wei code used in the
The V.34 standard, considered the ultimate modem standard V.32 and V.33 standards has free distance , number
for the classic linear Gaussian model of the telephone channel of nearest neighbors , and achieves a coding gain
[41], uses an MD mapping, a constituent 2D signal constella- of 3.6 dB at a BER of compared to uncoded 16-QAM
tion containing a maximum of 1664 points, and as many as and 64-QAM modulation, respectively.
eight uncoded bits (ten information bits total) per symbol to As in the other examples, this coding gain is achieved without
achieve a spectral efficiency up to bits/signal and data bandwidth expansion. These points are shown in Fig. 1 with
rates as high as 33 600 bps. Advanced line probing, adaptive the designations “V.32” and “V.33.”
equalization, and precoding techniques are used to increase A block diagram of the 16-state, , linear 4D code
the signaling rates to as high as 3429 symbols/s, compared to developed by Wei [40] is shown in Fig. 7, along with a
2542 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 6. The eight-state V.32 code.

sketch of a 224-point constellation which, along with its mirror encoded bit , they enter the 4D signal mapper (modulator).
inverse, forms a constituent 448-point 2D signal constellation The constituent 2D constellation is divided into four subsets,
consistent with the ITU-T V.34 standard. The TCM portion of and thus there are 16 subsets in the 4D constellation. The
the encoder has three input information bits: two coded four encoded bits and are mapped into one
and and one uncoded . In addition, bits and of the 16 subsets in the 4D constellation. In the most basic
are differentially encoded, since they are affected by phase version of the standard, the additional uncoded information
rotations. Full 90 rotational invariance can be achieved with bits needed to achieve the desired spectral efficiency are then
a linear code in this case since the signal constellation is 4D. used to select the particular 4D signal point to be transmitted.
(In general, it is possible to achieve full rotational invariance For example, if a spectral efficiency of bits/signal is
with linear codes using MD signal constellations, at least
desired using a constituent 192-point 2D signal constellation,
for code rates with .) The
an additional 11 uncoded information bits are needed in each
systematic feedback linear convolutional encoder shown in
4D signaling interval. In this case, the ratio of the constellation
Fig. 7 (a simplified form) is equivalent to an encoder with
generator matrix size to the size of the uncoded constellation which would
support the same spectral efficiency, i.e., the constellation
expansion ratio (CER), is CER , or 50% less
(14) than that needed with 2D TCM. By choosing the constituent
2D constellation subsets and the mapping such that lower
and produces three output bits: and (information bits) energy signal points (the “inner points” in the constellation)
as well as (a parity bit). Along with the differentially are used more often than higher energy points (the “outer
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2543

Fig. 7. The 16-state V.34 code.

2544 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 8. The b-bit-per-chip semiconductor memory organization.

points” in the constellation) a technique called constellation more thorough discussion of the considerations that went into
shaping [44], an additional “shaping gain” is achieved. This the design of the V.34 modem can be found in [41].
concept can be extended by further expanding the basic 2D
signal constellation beyond 192 points in conjunction with the IV. DATA STORAGE APPLICATIONS
shell mapping approach. In this approach, the signal selection
(mapping) is done over several symbol intervals so that enough In this section, we cover two applications of coding to data
dimensions are included to produce an approximately spherical storage, high-speed computer memories and magnetic/optical
constellation, known to be optimal in high-dimensional spaces. discs. The use of RS codes in compact discs represents the
In the V.34 standard, shell mapping is applied to a 16- most widely used and best known application of error-control
dimensional constellation, i.e., eight 2D symbol intervals, to coding over the 50-year time span covered by this review.
achieve up to 0.8 dB of shaping gain. For the constituent
448-point 2D signal constellation shown in Fig. 7, a total of A. Error-Correcting Codes for High-Speed
20 information bits over eight 2D symbols are devoted to Computer Memories
shell mapping, and four information bits per symbol are left Error-correcting codes are widely used to enhance the relia-
uncoded. This arrangement gives a total spectral efficiency of bility of computer memories. In these storage applications, the
codes have unique features not seen in most communication
(coding) (shaping) (uncoded)
applications. This is due to the following conditions required
bits/signal. (15) for error control in computer memories.
The size of the constituent 2D signal set can vary depending 1) Encoding and decoding must be very fast to maintain
on the desired shaping gain and CER. Usually, an additional high throughput rates. Therefore, parallel encoding and
CER due to shaping of about 25% is enough to approach 0.8- decoding must be implemented with combinatorial logic
dB shaping gains. In this example, the constellation size of instead of the linear feedback shift-register logic usually
448 yields a CER , for a total of 75% used in other applications.
constellation expansion, 50% due to coding and 25% due to 2) Unique types of errors, such as byte errors, are caused
shaping. Shaping was included in the standard because it was by the organization of semiconductor memory systems.
ultimately a simpler way of picking up an additional 0.8 dB As memory systems become large, the -bit-per-chip
than by using a more complex code. In theory, as much as 1.5 organization shown in Fig. 8 tends to be adopted. Then
dB of shaping gain is available [5], but the methods needed a chip failure causes the word read out to have errors in
to achieve the extra gain are quite complex. a -bit block, called a -bit byte or simply a byte.
The 16-state, , linear 4D Wei code used in the
3) Redundancy must be small, because the cost per bit of
V.34 standard has free distance , number of nearest
high-speed semiconductor memories is high compared
neighbors per 2D signal, and achieves a coding
with magnetic memories and other low-speed memories.
gain of 4.2 dB at a BER of compared to uncoded
256-QAM modulation. (Note that the smaller The best-known class of codes for semiconductor mem-
free distance here compared to the eight-state V.32 code ories is the SEC/DED (single-error-correcting/double-error-
does not translate into less coding gain, since the uncoded detecting) codes. It can be viewed as a class of shortened
systems used to compute the coding gain are different). This versions of extended Hamming codes. These codes have
point is shown in Fig. 1 with the designation “V.34/16.” The minimum distance and are capable of correcting
slightly improved performance obtained by using 32- and 64- single errors and detecting double errors. Usually, however,
state codes is also indicated in Fig. 1 with the designations they can detect more than just double errors. Since the ad-
“V.34/32” and “V.34/64,” respectively. ditional error-detection capability depends on the structure of
A good summary of the state-of-the-art in modem design the parity-check matrix, many proposals for enhanced error-
circa 1990 was given in the review article by Forney [38]. A detection properties have been presented.
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2545

Among the many proposed schemes, the SEC/DED/SbED TABLE I

(single-error-correcting/double-error-detecting/single -bit MINIMUM NUMBER OF PARITY-CHECK BITS FOR THE BEST
KNOWN ERROR-CONTROL CODES FOR HIGH-SPEED MEMORIES
byte error-detecting) codes have been put into practical use.
These codes can correct a single bit error and detect any
errors in a single -bit byte as well as any two bit errors. Such
detection capability is useful for the memory organization of
Fig. 8, i.e., a -bit per chip organization, where a chip failure
causes a byte error.
As an example, let us consider the case of . An
SEC/DED/S4ED code is constructed as follows. Let
and let be the set of all binary -dimen-
sional column vectors of even weight if is odd and of odd
weight if is even. For , the following
matrices are defined as

assumed the major role in this area. On the other hand, RS

codes have been the principal error-correcting codes used in
optical-disc memories from the beginning. These applications
(16) were greatly expanded with the appearance of the compact
disc (CD) and the CD–read-only memory (CD-ROM) at the
where denotes the -dimensional column vector whose beginning of the 1980’s [46], [50].
components are all ones. Letting , we define The CD format uses a coding system called CIRC (cross-
a matrix as interleaved RS codes), which is a combination of two short-
ened RS codes of minimum distance over GF .
The information sequence is first encoded by a code
(17) and then by a code. For audio signals, an error
interpolating technique can be used to estimate the original
Then it is easily seen that the code having signal values by interpolating or extrapolating from reliable
parity-check matrix is an SEC/DED/S4ED code [45], [46]. values for those errors that cannot be corrected.
For example, in the case of , an SEC/DED/S4ED code However, such a technique is not applicable for data stored
of length 112 with eight check bits is constructed. Shortening in computer systems, and thus greater error correction ca-
this code by 40 bits, we have a SEC/DED/S4ED code. pability is required for the CD-ROM. Thus a CRC (cyclic
In order to have 64 information bits, SEC/DED codes require redundancy check) code with 32 check bits and a product of
at least eight check bits. The construction method described two shortened RS codes with minimum distance
above gives a code having the same redundancy as ordinary over GF are used in addition to the CIRC. The encoder
SEC/DED codes and, in addition, the capability of detecting for a CD-ROM first appends four check bytes of a CRC to the
any errors in a single 4-bit byte. user data of 2048 bytes with a 4-byte header and then adds
As high-speed semiconductor memories become larger, eight bytes of all zeros for future extension. Alternate bytes are
more powerful error-correcting codes are needed. This has taken from the resulting sequence to produce two sequences
made SbEC/DbED (single -bit byte error correcting/double of length 1032 bytes. Then, after rearranging the bytes, each
-bit byte error-detecting) codes practical. Such codes are sequence is encoded by a product of and RS
constructed from RS codes or Bose-Chaudhuri-Hocquengham codes to give a codeword of length 1170 bytes. Finally, the
(BCH) codes over GF . If each symbol of GF is two codewords are combined and a synchronization pattern of
expressed as a -bit byte, then RS codes or -ary BCH 12 bytes is added to make one sector of a CD-ROM having
codes having minimum distance are SbEC/DbED 2352 bytes, as shown in Fig. 9, which is then applied to
codes. In applying these codes to semiconductor memories, the CIRC encoder. A decoder for a CD-ROM is depicted in
they are usually substantially shortened to fit the number of
Fig. 10.
information bits to the respective application. In such cases,
There are various ways of decoding the product code [51].
we can sometimes obtain more efficient codes by slightly
Reliability information for each symbol is obtained from the
modifying RS codes [47].
CIRC decoding process. Unreliable symbols are interpolated
Table I shows the minimum number of parity-check bits for
for audio CD’s, but they are treated as erasures for CD-ROM’s.
the best known SEC/DED codes, SEC/DED/ SbED codes, and
If necessary, more detailed reliability information is available
SbEC/DbED codes when the number of information bits is 32,
and soft-decision iterative decoding can be applied to make the
64, and 128 for and [48].
decoding results as reliable as possible. However, the error rate
of current CD’s is not high, and the CIRC itself is an error-
B. Error-Correcting Codes for Magnetic/Optical Discs control coding scheme powerful enough for most CD’s. Thus
Error correction for magnetic-disc memories began with the if the error probability of CIRC decoding is small enough,
application of Fire codes [49] in the 1960’s. Later, RS codes simple erasure decoding for the two RS codes can be adopted,
2546 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 9. The structure of one sector of a CD-ROM.

Fig. 10. The CD-ROM decoder.

as is the case for most CD-ROM drive units. With a single A. Difference Set Cyclic Codes for FM Multiplexed
decoding of each of the two codes, all erasures in any eight or Digital Audio Broadcasting
fewer positions can be corrected, since the minimum distance There are many error-correcting codes that have been ap-
of the product code is . But there exists at least plied to DAB systems. Among them, the difference set cyclic
one pattern of nine erasures that cannot be corrected by any code [53] adopted for an FM multiplexed DAB system is an
decoder. If erasure decoding of the product code is iterated example of an application involving a majority logic decodable
a few times, though, many patterns of nine or more erasures code [54]. The initial use was for error correction in a
can be corrected. TV teletext system [55]. For Western languages, the
Each of the RS codes in the CD and CD-ROM systems extended Hamming code sufficed to keep the transmitted text
is decoded by directly calculating the coefficients of the error readable. However, for the Japanese language this was not the
locator polynomial from the syndromes. This technique, which case, because the language has ideograms called kanji (Chinese
is simpler than other methods of decoding RS codes, can be characters) and even one error may make the text unreadable
applied whenever the minimum distance is no more than or or incorrect.
, as is the case for these RS codes. Hence more powerful codes were desired. However, the
Since the epoch-making success of the application of the decoder had to be very simple, because it was to be installed
CIRC system to CD’s, the major error control roles for optical in a TV set for home use. For these reasons, a
and magnetic discs have been monopolized by RS codes. This difference set cyclic code was adopted as a simply decodable
popularity is also supported by the progress of implementation code with high error-correction capability. This code has
techniques for codes with high error-correcting capability. minimum distance and can correct eight or fewer
Recently, it has become common to use RS codes over GF errors. In practice, it was shortened by one bit to a
having very large minimum distances. For example, there code to fit the data format of the teletext.
exists a general purpose encoder/decoder chip for The code is constructed on the basis of a perfect
RS codes for which the number of check bytes can be difference set
varied up to 36 [52]. This chip can correct up to erasure
bytes and error bytes for any nonnegative integers and
satisfying at speeds of at least 33 Mbytes/s.

V. APPLICATIONS IN DIGITAL AUDIO/VIDEO TRANSMISSION Any codeword in this code satisfies the
In this section, we present a specific example of an applica- following parity-check equations:
tion of a cyclic code to digital audio broadcasting (DAB) and
then discuss more generally the various coding systems used
in DAB and digital video broadcasting (DVB) standards. (18)
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2547

Fig. 11. Majority logic decoder for the (273; 191) code.

where . Every symbol where denotes the estimate of , is a threshold given by

of a codeword except appears once and only once in these
equations, and is contained in all the equations. Let (21)
be the received symbol for the th bit of the transmitted
codeword, . ( may be an analog symbol in
the case of soft output demodulation.) We denote the result of and the weighting factor is defined as
a decision on at a previous stage of decoding by and
(22)
the error in by , i.e., .
Initially, is taken to be the hard quantized version of .
Then, defining a syndrome bit for the and are the probabilities that the number of nonzero
received word by errors appearing in (19) is even and odd, respectively. This
algorithm, which is called APP decoding [56], is near-optimal,
but it is not practical because it demands too much complexity
(19) to compute . Therefore, the following further approximation
we can correctly estimate the error in as the majority is commonly made [57].
element of when the number of errors Let be an error pattern corresponding to a previous
is no more than eight, because at least ten components of decision concerning the received symbols appearing in (19),
are if is erroneous and at most eight components of i.e.,
are otherwise. The errors in the other bits are estimated in
the same way after the received word is cyclically permuted.
Fig. 11 shows a decoder based on this principle. where
This majority logic decoder is much simpler than a BCH
decoder of comparable length and error-correcting capability.
In fact, a majority logic LSI decoder was fabricated with and
about one tenth the number of gates as a BCH decoder.
For such a simple implementation, code efficiency must be
compromised. However, considerable improvement in error- We then approximate as the probability of the error pattern
correcting capability is realized by introducing soft-decision having maximum probability among all of even weight.
decoding. Similarly, we approximate by the probability of the error
Optimal soft-decision decoding of the first bit in the pattern having maximum probability among all of odd
transmitted codeword requires a MAP estimate of , or equiv- weight. These and can be easily obtained by using
alently the error bit . In the remainder of this section, we the trellis shown in Fig. 12. Such an approximation of APP
consider MAP estimation of , i.e., estimating as the value decoding for the difference set cyclic code is called APP trellis
that maximizes the a posteriori probability (APP) , decoding.
given the received word . It is difficult to calculate The APP decoder is optimal if all 273-tuples
exactly. Therefore, some assumptions on the independence of satisfying (18) are codewords. Since this
the received symbols are needed. For an AWGN channel, the is not true in general, the decoding performance of APP
MAP estimate of is approximated as decoding can sometimes be improved by variable threshold
decoding [58], which involves iterative decoding with a
variable threshold. At first, the threshold is set high to avoid
(20) miscorrection. The decoding result for a received symbol then
replaces the previous decision in succeeding stages of APP
decoding. This operation gives so-called extrinsic information
2548 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 12. The trellis for computing e

3 and e
3 .
ei oi

Fig. 13. Decoding performance of the (273; 191) code and a (273; 192) BCH code.

[32] to the decoding of the other symbols. After each iteration, schemes. The basic principles of these schemes are described
is decreased to correct additional bits. When reaches the below.
value given by (21), decoding is complete. 1) Unequal and Adaptive Error Protection with RCPC
Fig. 13 shows the decoding performance of APP trellis Codes: Rate convolutional nonsystematic feedfor-
decoding and hard decision decoding over an AWGN channel. ward or systematic feedback encoders have the advantage of
In both cases, variable threshold decoding with three iterations a binary trellis and therefore employ a simple Viterbi or MAP
was adopted. In the figure, the hard-decision decoding perfor- decoder [32]. From a nonsystematic feedforward
mance of a shortened BCH code correcting nine or encoder with generator matrix
fewer errors is also shown for comparison. Variable-threshold
(23)
APP trellis decoding shows excellent decoding performance
in spite of the simplicity of its implementation. A decoder we obtain a systematic feedback encoder with a rational parity
chip for this algorithm has been implemented for application generator
to FM multiplexed DAB systems.
(24)
B. Rate-Compatible Punctured Codes for Both encoders generate the same set of codewords, but they
Digital Audio Broadcasting have slightly different BER performance. For very noisy
The European DAB System, which will be also used for channels, the BER curve of the feedback encoder is better
DVB and the MPEG audio system [59], makes extensive use than and approaches, but does not cross, the BER curve for
of rate-compatible punctured convolutional (RCPC) coding uncoded performance.
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2549

Fig. 14. BER performance of nonsystematic feedforward RCPC codes for K = 5 and R = 8=9 to R = 1=2 on the statistically independent
Rayleigh fading channel.

Higher rate codes are easily obtained by puncturing. If this or less. Using a psychoacoustic model of the human ear, the
is done in a rate-compatible way, such that all the code bits in audio band is divided into 32 subbands which employ adaptive
the higher rate code are also used for the lower rate codes, we quantization. The data stream sent to the multiplexer contains
obtain an RCPC code [60]. This code family, with a puncturing headers, a scalefactor and scalefactor-select information, and
period of , has rates equal to quantized samples with bits of different significance. This
format and the dynamic data rate require unequal and adaptive
for to (25) error protection. Therefore, the RCPC codes described above
have been designed for three different rates ( , , and
The lowest rate is the rate of the original rate code, ) using a constraint length mother code with
called the mother code. Puncturing tables for different rates are 64 states. Later, in [62], an even more sophisticated system
given in [60]. Even lower rates can be achieved by duplicating was proposed, where the rate pattern of the RCPC codes
bits, i.e., by replacing “ ” by “ ” or some higher integer in can be changed almost continuously within the data frame,
the puncturing tables. On a multiplicative, fully interleaved depending on the varying rate and protection requirements of
Rayleigh fading channel these systematic codes perform as the source bits. As a result, the resource “protection bits” are
shown in Fig. 14. By puncturing all of the nonsystematic bits, used according to the needs of the source bits and the variation
the rate (uncoded) system is easily incorporated. Therefore, of the audio signal. The overall system exhibits a more graceful
we have a great variety of coding options which can be degradation at the edge of coverage of the radio broadcast
easily adapted to source and channel requirements because system where the SNR deteriorates.
the encoder and the decoder are the same for the whole
code family and can be adapted using only the puncturing
control. At the receiver, punctured bits marked with “ ” in the C. Error-Correcting Codes for Digital Video Systems
puncturing tables are stuffed with zero values and repeated bits Decoding speed is at the top of the list of requirements for
are combined by adding their soft received values. Therefore, error control in digital video systems. For this reason, powerful
the decoder always operates with the mother code and its trellis error-correcting codes had not been considered for this appli-
remains unchanged. cation until the 1990’s. The first standardized error-correction
2) The RCPC Coding Scheme for the European DAB System: method for digital video transmission, recommended as a part
The European DAB system [61] uses the source-coding system of the ITU-T H.261 standard in 1990, is very simple. It uses a
of ISO/MPEG Audio [59]. In this case, PCM Audio at 2 times BCH code, which has minimum distance
768 kbps is compressed, typically down to 2 times 128 kbps and is capable of correcting two or fewer random errors and
2550 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 15. Simplified model of the mobile communications channel.

detecting three random errors. In addition, it is designed to algorithms in tandem, one for the equalizer and one for the
correct a burst error of length no longer than six. FEC decoder, all on one chip, which also includes source
Since then, however, more sophisticated error-correction compression for the speech signal, channel estimation, and
schemes based on RS codes have been put into practical use other tasks.
with advanced video coding standards like MPEG2. For DVB, In this section, we will give some Shannon theory back-
in particular, concatenated codes composed of an RS code ground for the mobile channel, describe the most widely used
as the outer code and a convolutional or trellis code as the FEC schemes in mobile communications, and conclude with
inner code have been used. For example, a code some advanced FEC features currently being discussed for
obtained by shortening the RS code with minimum digital mobile systems.
distance has been adopted as the outer code along
with the CCSDS convolutional code (sometimes A. The Mobile Radio Channel and the Shannon Limit
punctured) as the inner code in the European digital terrestrial The mobile communications channel can be modeled as
TV broadcasting system. shown in Fig. 15, where the received complex equivalent
Multilevel coding with multistage decoding [63] is a strong baseband signal after multipath distortion is
candidate for future DVB systems. By employing unconven-
tional signal labeling (mapping), the large number of nearest
(26)
neighbors associated with multistage decoding can be reduced
significantly. This results in highly efficient transmission for
multivalue modulation formats such as -PSK and -QAM with complex time-varying factors . At the receiver
with moderate complexity [64]. Also, because of its excellent we also have noise and interference, usually modeled as an
error performance, turbo coding [29] may be considered as additive AWGN component with one-sided power spectral
the inner code in a concatenated system for future applications. density . If the channel is nonfrequency-selective, i.e., if
Both turbo coding and multilevel coding suffer from relatively the delay is less than the symbol duration,
long decoding delays, but this might be allowed in some DVB then the channel induces only multiplicative distortion with
applications. . The amplitude
(27)

VI. APPLICATIONS IN MOBILE COMMUNICATIONS is then Rayleigh- or Rician-distributed. For the Rayleigh case
we have
The advent of digital mobile communications during the
last 15 years has resulted in the frequent use of error-control (28)
coding because the mobile channel exhibits raw error rates of
5 to 10%, mainly due to fades, bursts, and interference from and a uniformly distributed phase. If sufficient interleaving
other users and other cells. This also means that decoders must is applied, these random variables become statistically inde-
work at low values of . Consequently, all the features pendent. For coherently detected BPSK modulation, only the
known to decoder designers must be used, such as interleaving, amplitude fluctuations are relevant. These assumptions allow
soft decisions, channel state information, and concatenation. us to calculate Shannon’s channel capacity for this mobile
On the other hand, bandwidth is scarce and expensive in a channel. The capacity is calculated as a function of the
heavily booming cellular market. This means that the resource channel SNR
“redundancy” must be used with care, applying methods (29)
such as unequal and adaptive error protection. Of course, the
where represents the code rate. If we transmit at the capacity
dramatic increase in integrated circuit complexity has allowed
limit we obtain from the parameter equation
the use of very sophisticated error-control methods. It is quite
common to employ in a handheld mobile phone two Viterbi (30)
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2551

Fig. 16. FEC scheme for the full rate GSM speech coder.

TABLE II codes are used as outer codes in a concatenated scheme. We

SHANNON LIMIT FOR THERAYLEIGH FADING CHANNEL will concentrate here on the inner coding scheme, which is
primarily used for the full rate speech traffic channel. A 20-
ms slot of the 13-kbit/s full rate speech coder produces a block
of 260 bits, split into three sensitivity classes. 78 bits are
rather insensitive to errors and are unprotected. The 50 most
significant bits are initially protected by a 3-bit CRC used for
error detection. Then these 53 bits, along with the remaining
132 bits, or a total of 185 bits, are encoded with a
nonsystematic convolutional code with generator matrix
(32)
the Shannon limit
In order to terminate the trellis, four known tail bits are
(31) added, as shown in Fig. 16. This leads to a total of 378 coded
bits and, including the 78 uncoded speech bits, results in a
For an interleaved Rayleigh fading channel, depending on the block of 456 bits every 20 ms, for a rate of 22.8 kbit/s.
use of soft decisions and channel state information (CSI), the Together with seven other users, these bits are interleaved
Shannon SNR limits of Table II can be calculated [65]. in groups of 57 bits and spread over eight TDMA bursts,
This Shannon theory result tells the designer of an FEC which are transmitted over the mobile channel using GMSK
system for such a mobile channel that modulation. The bursts include a midamble of 26 bits ( two
• for low-rate codes, the mobile Rayleigh channel is not signaling bits) used by the receiver to estimate the channel
worse than the Gaussian channel, because low-rate in- tap values corresponding to the multipath model shown in
terleaved coding acts like a multiple diversity system Fig. 15. Using the channel tap estimates, an MLSE equalizer
which transforms independent Rayleigh channels into a employing the Viterbi algorithm outputs hard or soft values
Gaussian channel; for the 456 bits in a block. After deinterleaving, the Viterbi
• for rate codes, the Rayleigh channel degrades the decoder accepts 378 values and outputs 185 bits from the
system by 2 to 3 dB, and terminated 16-state trellis. If the check with the CRC code
is positive, the 50 plus 132 decoded bits, together with the
• soft decisions can gain up to 2.4 dB, and CSI, namely the 78 unprotected bits, are delivered to the speech decoder. A
fading amplitude, can gain roughly another decibel. negative CRC usually triggers a concealment procedure in the
speech decoding algorithm, rather than accepting any errors in
B. FEC Coding in the Global System for the 50 most important bits.
Mobile Communications (GSM) When, as defined in the GSM standard, frequency hopping
With currently some 80 million mobile phones worldwide, is used on the bursts in an urban enviroment, statistically
the error-control system of GSM is probably the most widely independent Rayleigh fading is a reasonable assumption. In
used form of nontrivial FEC besides the compact disc and this case, we notice from the Shannon limits in Table II that at
high-speed modems. The speech and data services of the a rate of we can gain up to 3.1 dB in required SNR if
GSM standard use a variety of FEC codes, including BCH the channel decoder uses CSI and soft values. Since a normal
codes and Fire codes, plus CRC codes for error detection and Viterbi equalizer delivers only hard decisions, a modification
codes for synchronization, access, and data channels. These is necessary leading to a SOVA or MAP type soft-in/soft-out
2552 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 17. A CDMA transmitter.

TABLE III the carrier phase, the delays of the multipath channel, and the
FEC CODES INGSM SPEECH CODING complex factors of the different propagation paths (i.e., perfect
CSI estimation). This leads to an -finger RAKE receiver
with maximum ratio combining (MRC) [67]. In each RAKE
finger, the respective path delay is compensated, followed
by a multiplication with the complex conjugate propagation
coefficient .
Classical demodulation after the coherent RAKE receiver
involves maximum-likelihood (ML) detection of the trans-
mitted Hadamard codeword by searching for the maximum
equalizer. Most GSM terminals now use some kind of soft value of the correlation vector, which is easily implemented
values from the equalizer as inputs to the decoder. with the fast Hadamard transform (FHT). The systematic
GSM is a good example how Shannon’s theory can help a bits (information bits) of the demodulated codeword are then
system designer. Although with such simple codes we are not weighted with the maximum value of the correlation vector.
operating at the absolute values of Shannon’s limit given in (The Hadamard code can also be decoded with a “soft-in/soft-
Table II (GSM operates at approximately 7 dB), the relative out” decoder [32].) Afterwards, they are passed on to the
gains between hard and soft decisions remain a good indicator. deinterleaver and the outer soft-in/hard-out Viterbi decoder.
Other FEC coding schemes used for the different speech The other case is a noncoherent receiver design without any
options defined by GSM are summarized in Table III. For knowledge of the phase of the signal. Here, in each RAKE
data services, the currently defined user data rates in GSM finger, the FHT must be performed for both the in-phase and
are 2.4, 4.8, and 9.6 kbit/s. When a full rate channel is used quadrature-phase components after the delay compensation.
the respective code rates are , , and . It will be a After square-law combining (SLC) [67] of the correlation
difficult task to accomodate higher user data rates in one full vectors, classical ML detection is also performed to find the
rate slot. maximum value of the correlation vector. Further decoding of
the outer code is straightforward, as in the coherent case.
C. FEC Coding in the CDMA Mobile System (IS-95)
In CDMA systems for mobile communication, FEC is
very important in combating multiuser interference (MUI) and D. Relations Between Source and Channel
the influence of multipath fading. Therefore, most systems Coding for Mobile Channels
use a low-rate channel code followed by direct sequence
A basic result of Shannon’s information theory is that source
(DS) spreading with a high processing gain. Others use a
and channel coding can be treated separately. Of course,
concatenation of two or more low-rate codes followed by DS
this is true in the information-theoretic sense: as long as
spreading, which provides privacy (a signature sequence), but
the entropy of the source is less than the channel capacity,
has a low processing gain. A well-known example of the latter
there exists a separable source and channel coding scheme
type of system is the uplink of the Qualcomm system based on
that allows transmission over the channel with arbitrarily low
the IS-95(A) standard [66]. As shown in Fig. 17, this system
error probability. Arising from this basic result, there has been
consists of an outer convolutional code with rate
a clear-cut interface between source and channel coding and
and constraint length , a block interleaver with 32
decoding. However, on a transmission channel with extensive
rows and 18 columns, and a -ary orthogonal modulation
noise, interference, multipath, fading, and shadowing, it is too
scheme. The orthogonal modulation (the inner code ) is
expensive in terms of bandwidth, delay, and complexity to
a so-called Hadamard code, or Walsh function, and can be
implement perfect channel coding.
viewed as a systematic block code of rate and
In a practical system, the different blocks of source coder,
minimum distance . The subsequent processing
channel coder, transmission channel, channel decoder, and
steps are: 1) DS spreading by a factor of , 2) quadrature
source decoder are linked in a variety of ways, as indicated
scrambling of the in- and quadrature-phase components with
in Fig. 18.
different PN sequences (IPN and QPN), and 3) offset QPSK
modulation (not shown in Fig. 17). • Source significance information (SSI) is passed to the
In the following, we describe two types of receiver struc- channel coder for static or dynamic unequal error pro-
tures. The first is a coherent receiver, where we must know tection. Unequal error protection can be easily performed
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2553

Fig. 18. Links between source and channel coding in a transmission over a mobile channel.

with block and convolutional codes, and in a very simple been achieved for speech, audio, still image, and video
way by RCPC codes, as described in Section VII-B1). sources [68], [69], [70].
• The channel decoder and the demodulator are not only The last three items can be implemented in existing systems
connected by the hard decisions of the demodula- because they do not require changes at the transmitter. They
tor/detector. Soft decisions and channel state information are “value-added” devices to be used only when appropriate,
(CSI) are also passed on. for instance, in a bad channel environment. An example for
• At the receiver, when the channel decoder makes errors GSM can be found in [69].
and these errors can be detected by a CRC or by other
means, it is common engineering practice to interpolate
the source sample values. One can also use more sophis- E. Turbo Decoding for Mobile Communications
ticated methods to conceal errors by using the decision on In 1993, decoding of two or more product-like codes was
the current bit and its reliability. This sophisticated con- proposed using iterative (“turbo”) decoding [29]. (See also
cealment, however, requires channel decoders that deliver the end of Section II.) The basic concept of this new coding
“soft outputs,” i.e., decisions and “decision reliability scheme is to use a parallel concatenation of at least two codes
information” (DRI) as shown in Fig. 18. with an interleaver between the encoders. Decoding is based
on alternately decoding the component codes and passing the
• Another approach links source and channel decoding to an
so-called extrinsic information to the next decoding stage. For
even greater extent. Source-controlled channel decoding
good soft-in/soft-out decoders this extrinsic information is an
[68] uses a priori and a posteriori information about the
additional part of the soft output. Even though very simple
source bits. For reasons of delay and complexity and
component codes are used, the “turbo” coding scheme is able
because of highly nonstationary sources, many source
to achieve performance close to Shannon’s bound, at least for
coding schemes still contain redundancy and some bits
large interleavers and at BER’s of approximately .
are highly correlated, at least at times. Under these cir-
Moreover, it turned out that the turbo method originally ap-
cumstances source and channel decoding should be linked
plied to parallel concatenated codes is much more general [71]
more closely. In fact, Shannon mentioned this possibility and can be sucessfully applied to many detection/decoding
in his 1948 paper [1]: “However, any redundancy in the problems such as serial concatenation, equalization, coded
source will usually help if it is utilized at the receiving modulation, multiuser detection, joint source and channel
point. In particular, if the source already has redundancy decoding, and others. We will mention here two applications
and no attempt is made to eliminate it in matching to for mobile communications to show the potential of turbo
the channel, this redundancy will help combat noise.” decoding.
Consequently, if some redundancy or bit correlation is 1) Parallel Concatenated (Turbo) Codes for GSM Applica-
left by the source coder, this should be utilized jointly tions: The turbo decoding method was not available at the
by the channel and source decoder. The source a priori/ time of GSM standarization. But it is interesting to consider
a posteriori information (SAI), as indicated in Fig. 18, what can be gained in the GSM System with turbo decoding.
tells the decoder with a certain probability the value of In [72], the FEC system for GSM was redesigned with parallel
the next bit to be decoded. It is much better to give concatenated codes, while keeping all the framing and the
the channel decoder all the information known about the interlever as in the full rate GSM System. This means that
bits which are to be decoded, i.e., about correlation or a frame length of only 185 information bits could be used.
bias values, than to allow it to make errors by blind This is different from the common usage of turbo codes,
decoding, thus causing it to conceal these errors later. The where several thousand bits are usually decoded in one frame.
goal of source-controlled channel decoding is to reduce Nevertheless, it was shown that on GSM urban channels with
the channel decoder error rate by supplying the channel frequency hopping, an improvement in channel SNR (i.e., an
decoder with source information. Especially on mobile additional coding gain) of 0.8 to 1.0 dB is possible. Although
channels, gains of several decibels in channel SNR have several iterations of the turbo decoder are necessary to achieve
2554 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

Fig. 19. Iterative decoding in a coherent receiver.

this gain, the total decoding complexity is comparable to the result from single-event upsets caused by equipment transients.
GSM system because the turbo component codes had only four The error-control systems for these applications use error
states, compared to the 16-state code used in standard GSM. detection coupled with retransmission requests to maximize
2) Turbo Detection in the IS-95 Mobile System: In this sec- reliability at some cost to throughput. At their simplest, such
tion, we explain the principle of turbo detection for the IS-95 error-control systems use parity-check bits or CRC codes
uplink when coherent demodulation is applied. An application to trigger retransmission requests. Since the retransmission
to the noncoherent uplink can be found in [73]. For iterative requests occur well below the application layer, and are in a
decoding, it is necessary to replace the outer Viterbi decoder sense “automatic,” such protocols are usually called automatic
with a modified soft-output Viterbi algorithm (SOVA) or a repeat request (ARQ) protocols. (The “RQ” in “ARQ” is taken
MAP decoder which delivers soft-output information with from the Morse code designation for a retransmission request.)
each decoded bit. The soft information from the outer decoding More complicated protocols include elements of FEC and
stage is interleaved and then fed back as a priori information packet combining to reduce the frequency of retransmission
in a redecoding of the inner code. This leads to a serial turbo requests.
decoding scheme. A schematic overview of the receiver is This section begins with a quick look at parity, CRC, and
given in Fig. 19. “pure” ARQ protocols. Several popular file-transfer protocols
From the -finger RAKE receiver we obtain a vector . are provided as examples. Consideration is then given to
The received log-likelihood values , hybrid protocols and packet combining.
including the fed back a priori values of the systematic
bits, are correlated with the Hadamard codewords using the A. Parity Bits and CRC’s
FHT. This results in the correlation vector . The approximate
MAP decoding rule then reduces to a simple expression In asynchronous file-transfer systems, each character is
treated as a separate entity [74]. Seven-bit ASCII is the
(33) most ubiquitous example (where ASCII stands for “American
Standards Committee for Information Interchange”). The 128
To obtain the a priori information, we must decode the outer possible characters cover upper and lower case Roman let-
code with a SOVA or MAP decoder. The extrinsic part of the ters, Arabic numerals, various punctuation marks, and several
soft ouput for the outer decoded bits is then used as the a control characters. The standard error-control technique for
priori information for the systematic bits of the inner asynchronous ASCII is the simplest possible: a single parity
code in the feedback loop shown in Fig. 19. bit is appended to each ASCII character. The value of the
Simulation results for the AWGN channel show that an appended bit is selected so that the sum of all eight transmitted
additional coding gain of about 1.0–1.5 dB can be achieved bits is modulo (even parity) or modulo (odd parity). In
with iterative decoding after five iterations. If the simple either case, the resulting code can detect all single errors, as
approximation in (33) is applied, the resulting degradation is well as a large number of error patterns of higher weight. The
less than 0.1 dB. encoding and decoding circuit is extremely simple, and can
be constructed using a small number of exclusive-OR gates
or a few lines of code. The Kermit data modem protocol is a
VII. FILE TRANSFER APPLICATIONS typical example of an asynchronous file-transfer system that
File transfer is the movement of digital information from uses simple parity checks.
one data terminal to another. This digital information is In synchronous transmission systems, error control can be
organized into characters, frames, and files. File transfer ap- expanded to cover entire frames of data. Since the frame is a
plications typically demand a very high level of reliability; larger transmission element than the character, the number of
unlike voice applications, a single erroneous bit can render encoding and decoding operations is reduced, along with the
a multimegabyte file useless to the end user. Communication number of potential requests for retransmission.
links in wireless and wired file transfer systems are designed to The most popular synchronous error-control techniques are
be highly reliable. In many systems, the majority of data errors based on shortened linear cyclic codes. Let be an
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2555

Fig. 20. A CRC encoder circuit.

cyclic code with a systematic encoder. Let be the subset of the message bits to complete the systematic encoding. The
codewords in whose high-order information coordinates resulting code polynomial has the form .
(i.e., the rightmost coordinates in the codeword) have the The encoder can also be used as a decoder. Note that, by
value zero. Let be the set of words obtained by deleting construction, must be divisible by
the rightmost coordinates from all of the words in . is without remainder. If the encoded data is not corrupted
an shortened systematic code that is usually during transmission, the “encoding” of the received version
referred to as a CRC, or cyclic redundancy check, code [75]. of will result in a zero remainder. The detection of a
CRC codes are also called polynomial codes. nonzero remainder after the processing of the received data is
Note that CRC codes obtained from cyclic codes through the trigger for a retransmission request.
shortening are almost always noncyclic. However, it is still There are a variety of “rules of thumb” for selecting CRC
possible to use the same shift-register encoders and decoders generator polynomials. Bertsekas and Gallager [76] note that
that are used with the original cyclic code. Consider the it is common practice to select , where
systematic cyclic encoder in Fig. 20. Before encoding begins, is primitive. The factor ensures that all odd-
the shift-register memory elements are set to zero. If the first weight error patterns are detectable. Table IV contains several
high-order message symbols are equal to zero, their insertion CRC codes of varying length that have been used in practice.
into the encoding circuit does not change the state of the shift Detailed examinations of various polynomials with different
register, and results only in the output of zeroes. The deletion degrees are provided in [77]–[79].
of high-order symbols from the message block thus has The error-detecting capability of a CRC code is best mea-
no impact on the encoding process except for the deletion sured in terms of the percentage of possible error patterns that
of zeroes in the corresponding message positions of the are detectable. Channel errors tend to occur in bursts in the
codeword. wired file-transfer applications that make the most use of CRC
The shift-register taps in Fig. 20 are the codes, so the memoryless channel assumption that underlies
coefficients of a degree- CRC generator polynomial a Hamming distance analysis is not valid. An -bit CRC can
detect % of all error patterns.
CRC codes are often altered in practice to avoid the possibil-
ity of confusing the check bits with protocol flag fields (frame
Given a message block delimiters, for example) or to maintain a minimum transition
density (an important consideration in magnetic recording
applications). The modification to the code typically involves
complementing some or all of the check bits. Though the code
and the associated message polynomial is no longer linear, error-detection performance is unaffected.
In some applications it is easier to generate an -bit check-
sum by simply segmenting the information to be protected
into -bit words and computing the sum (without carry). As
the encoder computes the remainder obtained from the with the unmodified CRC, the resulting code is linear and can
division of by . The remainder is appended to detect % of all error patterns.
2556 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

TABLE IV
SEVERAL CRC GENERATOR POLYNOMIALS THAT ARE COMMONLY USED IN PRACTICE

In the next section we consider the various means by which following throughput:
retransmission requests can be incorporated into a file-transfer
protocol. (35)

SW-ARQ protocols require very little buffering in the trans-

B. Retransmission Request Modes
mitter, and virtually none in the receiver. For this reason such
In this section we derive the basic relations and design protocols are often used in simple data modems (the “Kermit”
choices that determine the throughput performance of an ARQ protocol is a typical example). The disadvantage, of course,
protocol. We begin by determining the number of times that a lies in the inefficient use of the channel, an inefficiency that
given packet can expect to be transmitted before it is accepted increases with the distance between transmitter and receiver.
by the receiver. Given the probability that an error is GBN-ARQ protocols assume that the transmitter is capable
detected and a retransmission request generated, the expected of buffering packets. When the transmitter is informed
number of transmissions is computed as follows: by the receiver of an error in some packet, say packet
, the transmitter “goes back” to that packet and resumes
transmission from there. The parameter is the number of
packets that have been subsequently sent by the transmitter
since the th packet, and that must now be retransmitted. is
(34) a function of the roundtrip propagation and processing delays
for the transmitter and receiver and can be approximated by
. The throughput for a GBN-ARQ protocol based on a
There are three basic modes that are used in the im- rate error-detecting code is as follows:
plementation of retransmission-request protocols: stop-and-
wait (SW-ARQ), go-back-N (GBN-ARQ), and selective-repeat
(SR-ARQ) [75], [80], [81]. These three modes provide varying
levels of throughput by changing the amount of buffering (36)
required in the transmitter and/or receiver. Throughput Note that the transmitter in a GBN-ARQ protocol is continu-
is defined here to be the average number of encoded data ously transmitting. GBN-ARQ protocols are thus often called
blocks (packets) accepted by the receiver in the time it takes “Continuous RQ” protocols [74].
the transmitter to send a single -bit data packet. SR-ARQ protocols are also Continuous RQ protocols. In
SW-ARQ is by far the simplest of the three modes. In this case the transmitter only retransmits the specific packets
this mode, the transmitter sends a single packet and waits for which an ACK is not received. This requires significant
for an acknowledgment (ACK). If an ACK is received, the buffering in both the transmitter and the receiver, but max-
transmitter sends the next packet. Otherwise, the first packet imizes channel efficiency. The throughput in this case is no
is retransmitted. Since the transmitter is idle while waiting for longer a function of channel delay. The throughput for a rate
the response, this mode of ARQ is often called “Idle RQ” code is quickly computed as
[74]. Let be the number of bits that the transmitter could
have transmitted during this idle time. An SW-ARQ protocol (37)
based on a rate error-detecting code will provide the
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2557

In the next section we consider the means by which error • A Network Interface Layer (also called the link or data-
detection retransmission requests are incorporated into two link layer) that provides the actual interface to the hard-
popular file-transfer applications. ware of the individual networks (e.g. X.25, ATM, FDDI,
and packet radio networks).
C. High-Level Data-Link Control Protocol The Internet Protocol (IP) is the heart of the Internetwork
The High-Level Data-Link Control (HDLC) protocol is layer. It is a connectionless protocol that does not assume or
a general-purpose protocol that operates at the data link provide any measure of link reliability. The only error control
layer of the OSI reference model [82]. It was created by incorporated into the IP datagram is a 16-bit checksum that
the International Standards Organization (ISO) and is desig- only covers the header. The checksum is obtained by com-
nated ISO 4335. HDLC uses the services of a synchronous puting the 16-bit one’s complement of the one’s complement
physical layer and can provide a best effort or a reliable sum of all 16-bit words in the header (the checksum itself is
communications path between the transmitter and receiver assumed to be zero during this computation) [84]. If a received
(i.e., with acknowledged data transfer). The selected HDLC IP datagram has an incorrect checksum, the entire datagram is
mode determines the type of service. HDLC is generally discarded. There is no provision for a retransmission request,
implemented using a Continuous RQ mode. in part because it would not be known from whom to request
Link Access Procedure Version B (LAPB) is a subset the retransmission. It should also be noted that IP does not
of HDLC that is commonly used to control the transfer of provide a check on the data field within the datagram. This
information frames across point-to-point data links in public must be provided at the Transport layer.
and private packet-switching networks. Such networks are There are two basic transport-layer protocols: the Trans-
generally referred to as X.25 networks [74]. mission Control Protocol (TCP—connection oriented) and
The HDLC frame format has six basic elements [82]. the User Datagram Protocol (UDP—connectionless). Both
• An 8-bit opening flag. protocols “encapsulate” IP datagrams by appending transport-
• An 8-bit address field (expandable in 8-bit increments). layer headers. UDP has an optional 16-bit checksum that,
• An 8-bit control field (expandable in 8-bit increments). when used, is calculated in a manner similar to the IP
• An information field of arbitrary size. datagram checksum. The UDP checksum is a 16-bit one’s
• A 16-bit frame check field (32 bits optional). complement of the one’s complement sum of the encapsulated
• An 8-bit closing flag. IP header, the UDP header, and the UDP data. As with
IP, UDP does not allow for acknowledgments. UDP simply
The bits in the 16-bit frame check field are computed using
arranges received packets in the proper order and passes them
CRC-CCITT, as defined in Table IV. CRC-32 is the optional
on to the application. Internet telephony is typical of the
32-bit code that is used in applications requiring an extremely
applications that use UDP in that it emphasizes latency over
high level of reliability (e.g., high-speed local-area networks).
reliability.
In full-duplex mode, HDLC allows for the use of “piggy-
TCP, on the other hand, makes full use of its 16-bit
back acknowledgments”—a situation in which acknowledg-
checksum. TCP assigns sequence numbers to each transmitted
ments or requests for retransmission of packets on the forward
data block, and expects a positive acknowledgment from the
channel are combined with other data transmitted on the return
transport layer of the receiving entity. If an acknowledgment is
channel.
not received in a timely manner, the block is retransmitted. The
full retransmission protocol is a variation of SR-ARQ that uses
D. The Internet and TCP/IP
a window to control the number of pending acknowledgments.
The key technology underlying the Internet (and many The transmitter is allowed to transmit all packets within the
internets, intranets, and variations thereof) is the suite of window, starting a retransmission clock for each. The window
protocols that go by the name TCP/IP. TCP/IP is named after is moved to account for each received acknowledgment.
two constituent protocols, the Transmission Control Protocol
and the Internet Protocol, but the suite actually contains several
dozen protocols [83]. TCP/IP allows for the internetworking E. Hybrid-ARQ Protocols and Packet Combining
of computer networks that are based on different networking In 1960, Wozencraft and Horstein [85], [86] described and
technologies. For example, TCP/IP allows for the seamless analyzed a system that allowed for both error correction and
interconnection of token rings (IEEE 802.5) with Ethernet- error detection with retransmission requests. Their system,
based systems (IEEE 802.3). now known as a type-I hybrid-ARQ protocol, provides sig-
A TCP/IP internetworking architecture can be divided into nificantly improved performance over “pure” ARQ protocols.
four basic layers. The goal in designing such systems is to use FEC to handle the
• An Application Layer consisting of user processes (TEL- most frequently occurring error patterns. The less frequently
NET, FTP, SMTP, etc.). occurring (those most likely to cause FEC decoder errors)
• A Transport Layer that provides end-to-end data transfer. are handled through error detection and the request of a
• An Internetwork Layer (also called the Internet layer) that retransmission.
provides the image of a single virtual network to upper Error-control protocols based on algebraic block codes (e.g.,
layers by routing datagrams among the interconnected RS and BCH codes) and hard-decision decoding provide
networks. a natural form of type-I hybrid-ARQ protocol. The most
2558 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

commonly used hard-decision decoders for BCH and RS of almost any system involving the transmission or storage
codes use bounded distance decoding algorithms that correct of digital information. And as we look to the future and the
a number of errors up to some design distance . If there is no “digital revolution,” with such innovations as digital cellular
codeword within distance of the received word, the bounded telephony, digital television, and high-density digital storage, it
distance decoding algorithm fails, providing an indication of seems assured that the use of coding will become even more
uncorrectable errors. When decoder failures are used as a pervasive.
trigger for a retransmission request, the BCH- and RS-based In trying to assess the impact that various advances in
systems become examples of type-I hybrid-ARQ protocols coding theory have had in the practical arena, some general
[87]. The ReFLEX and cellular digital packet data (CDPD) trends become apparent. Although most of the early work in
wireless data protocols are typical of the applications that use coding was based on algebraic constructions and decoding
these strategies. methods, recent trends definitely favor probabilistic, soft-
Type-I hybrid-ARQ protocols can also be based on complete decision decoding algorithms. With a few notable exceptions,
decoders. Drukarev and Costello developed the “Time-Out” such as the ubiquitous Reed–Solomon codes, the 2–3-dB
and “Slope Control” algorithms for type-I protocols based on loss associated with hard-decision decoding can no longer
convolutional codes with sequential decoding [88]. Yamamoto be tolerated in most applications. In fact, one of the most
and Itoh [89] developed a similar protocol based on Viterbi exciting current areas of coding research is the development
decoding. of soft decoding methods for algebraically constructed block
Pure ARQ and Type-I hybrid-ARQ protocols generally codes with good distance properties. Because of the relative
discard received packets that trigger retransmission requests. In ease of soft-decision decoding of convolutional codes (due
1977, Sindhu [90] discussed a scheme that made use of these to the development of sequential decoding, and later the
packets. Sindhu’s idea was that such packets can be stored and Viterbi algorithm), convolutional codes became the preferred
later combined with additional copies of the packet, creating choice in most practical applications. The relative scarcity of
a single packet that is more reliable than any of its constituent nearest neighbor codewords in convolutional codes compared
packets. Since 1977 there have been innumerable systems to block codes also tipped the scale in their favor, particu-
proposed that involve some form of packet combining. These larly in applications requiring only moderate BER’s, where
packet-combining systems can be loosely arranged into two the number of nearest neighbors plays an important role in
categories: code-combining systems and diversity-combining performance. On the other hand, in applications where data
systems. integrity is paramount, or when there is a strong tendency
In code-combining systems, the packets are concatenated towards burst errors, powerful nonbinary error-correcting (e.g.,
to form noise-corrupted codewords from increasingly longer Reed–Solomon) or binary error-detecting (e.g., CRC) block
and lower rate codes. Code combining was first discussed in a codes are still preferred.
1985 paper by Chase [91], who coined the term. Subsequent Another noticable trend is the relatively recent emergence
code-combining systems are exemplified by the work of Kr- of many commercial applications of coding. It is fair to say
ishna, Morgera, and Odoul [92], [93]. An early version of a that the driving force behind almost all coding applications in
code-combining system was the type-II hybrid-ARQ protocol the first 25 years after the publication of Shannon’s paper was
invented by Lin and Yu [94]. The type-II system allows for government scientific (e.g., NASA) programs or military appli-
the combination of two packets, and is thus a truncated code- cations. However, in the most recent 25 years, particularly with
combining system. The system devised by Lin and Yu was the emergence of the compact disc, high-speed data modems,
developed for satellite channels. Type-II systems based on RS and digital cellular telephony, commercial applications have
and RM codes were later developed by Pursley and Sandberg proliferated. From our perspective, it seems fairly certain that
[95] and by Wicker and Bartz [96], [97], and similar systems this trend will not only continue but accelerate. Indeed, in
which utilize RCPC codes were introduced by Hagenauer [60]. this paper we have been able to touch on only a few of the
In diversity-combining systems, the individual symbols many current activities involving the development of coding
from multiple, identical copies of a packet are combined to standards for various commercial applications. Thus it is clear
create a single packet with more reliable constituent symbols. that error-control coding has moved from being a mathematical
Diversity-combining systems are generally suboptimal with curiosity to being a fundamental element in the design of
respect to code-combining systems, but are simpler to digital communication and storage systems. One can no longer
implement. These schemes are represented by the work of claim to be a communication or computer system engineer
Sindhu [90], Benelli [98], [99], Metzner [100], [101], and without knowing something about coding.
Harvey and Wicker [102]. As we look towards the future, it seems clear that parallel
and/or serial concatenated coding and iterative decoding (i.e.,
turbo coding) are destined to replace convolutional codes
VIII. CONCLUSION in many applications that require moderate BER’s and can
In this paper, we have presented a number of examples illus- tolerate significant decoding delay. This is another illustra-
trating the way in which error-control coding, an outgrowth of tion of the importance of minimizing the number of nearest
Shannon’s Information Theory, has contributed to the design neighbor codewords in applications requiring only moderate
of reliable digital transmission and storage systems over the BER’s. More generally, turbo coding has pointed the way
past 50 years. Indeed, coding has become an integral part towards more creative uses of code concatenation as a means
COSTELLO et al.: APPLICATIONS OF ERROR-CONTROL CODING 2559

of achieving exceptional performance with moderate decoding [16] G. Fettweis and H. Meyr, “High-speed parallel viterbi decoding: Algo-
complexity. The need for almost error-free performance in rithm and VLSI-architecture,” IEEE Commun. Mag., vol. 29, pp. 46–55,
May 1991.
many scientific applications and high-performance commer- [17] W. W. Wu, E. F. Miller, W. L. Pritchard, and R. L. Pickholtz, “Mobile
cial systems, such as CD’s and computer memories, seems satellite communications,” Proc. IEEE, vol. 82, pp. 1431–1448, Sept.
certain to continue to spur interest in extremely powerful code 1994.
[18] G. D. Forney, Jr., Concatenated Codes. Cambridge, MA: MIT Press,
designs with very large minimum distance, such as algebraic- 1966.
geometry codes and multilevel construction techniques. And [19] Consultative Committee for Space Data Systems, “Recommendations
for space data standard: Telemetry channel coding,” Blue Book Issue 2,
further advances in signal-processing technology are sure to CCSDS 101.0-B2, Jan. 1987.
speed the conversion of almost all decoding machines to [20] R. J. McEliece and L. Swanson, “Reed–Solomon codes and the ex-
the use of soft decisions. Finally, coding will continue to ploration of the solar system,” in Reed–Solomon Codes and Their
Applications, S. B. Wicker and V. K. Bhargava, Eds. Piscataway, NJ:
find use on increasingly difficult channels as demands for IEEE Press, 1994, pp. 25–40.
more reliable communications and data storage accelerate. The [21] E. R. Berlekamp, Algebraic Coding Theory. New York: McGraw-Hill,
technical challenges inherent in designing appropriate codes 1968. (Revised edition, Laguna Hills, CA: Aegean Park, 1984.)
[22] J. L. Massey, “Shift register synthesis and BCH decoding,” IEEE Trans.
for the many anticipated new mobile communication services, Inform. Theory, vol. IT-15, pp. 122–127, Jan. 1969.
which must operate in severe multipath and fading conditions, [23] J. Hagenauer, E. Offer, and L. Papke, “Matching Citerbi decoders and
and high-density data storage applications, with increasingly Reed–Solomon decoders in concatenated systems,” in Reed–Solomon
Codes and Their Applications, S. B. Wicker and V. K. Bhargava,
difficult read/write signal processing requirements, promise to Eds.Piscataway, NJ: IEEE Press, 1994, pp. 242–271.
keep coding theory an active research field well into the next [24] O. M. Collins, “The subleties and intracacies of building a constraint
length 15 convolutional decoder,” IEEE Trans. Commun., vol. 40, pp.
century. 1810–1819, Dec. 1992.
[25] S. B. Wicker, “Deep space applications,” in CRC Handbook on Coding
Theory, V. Pless, W. C. Huffman, and R. Brualdi, Eds. Boca Raton,
ACKNOWLEDGMENT FL: CRC, ch. 25, to be published in 1998.
[26] E. Paaske, “Improved decoding for a concatenated coding system
The authors wish to thank Dr. Lance C. Perez, Dr. G. David recommended by CCSDS,” IEEE Trans. Commun., vol. COM-38, pp.
Forney, Jr., Dr. Shu Lin, Dr. Robert H. Morelos-Zaragoza, 1138–1144, Aug. 1990.
and Dr. Oscar Y. Takeshita as well as Hermano A. Cabral for [27] O. M. Collins and M. Hizlan, “Determinate state convolutional codes,”
IEEE Trans. Commun., vol. 41, pp. 1785–1794, Dec. 1993.
valuable comments that contributed to the preparation of this [28] J. Hagenauer and P. Hoeher, “A Viterbi algorithm with soft-decision out-
paper. puts and its applications,” in Proc. 1989 IEEE Global Communications
Conference (Dallas, TX, Nov. 1989), pp. 47.1.1–47.1.7.
[29] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limit
REFERENCES error-correcting coding and decoding: Turbo codes,” in Proc. 1993
IEEE Int. Communications Conf. (Geneva, Switzerland, May 1993), pp.
[1] C. E. Shannon, “A mathematical theory of communication,” Bell Syst. 1064–1070.
Tech. J., vol. 27, pp. 379–423, 1948. [30] S. Benedetto and G. Montorsi, “Unveiling turbo codes: Some results
[2] J. M. Wozencraft and I. M. Jacobs, Principles of Communication on parallel concatenated coding schemes,” IEEE Trans. Inform. Theory,
Engineering. New York: Wiley, 1965. vol. 42, pp. 409–428, Mar. 1996.
[3] R. W. Hamming, “Error detecting and error correcting codes,” Bell Syst. [31] L. R. Bahl, J. Cocke, F. Jelinek, and J. Raviv, “Optimal decoding of
Tech. J., vol. 29, pp. 147–150, 1950. linear codes for minimizing symbol error rate,” IEEE Trans. Inform.
[4] I. M. Jacobs, “Practical applications of coding,” IEEE Trans. Inform. Theory, vol. IT-20, pp. 284–287, Mar. 1974.
Theory, vol. IT-20, pp. 305–310, May 1974. [32] J. Hagenauer, E. Offer, and L. Papke, “Iterative decoding of binary
[5] G. D. Forney, Jr., and L. F. Wei, “Multidimensional constellations—Part block and convolutional codes,” IEEE Trans. Inform. Theory, vol. 42,
I: Introduction, figures of merit, and generalized cross constellations,” pp. 429–445, Mar. 1996.
IEEE J. Select. Areas Commun., vol. 7, pp. 877–892, Aug. 1989. [33] S. Benedetto, D. Divsalar, G. Montorsi, and F. Pollara, “Serial concate-
[6] J. L. Massey, “Deep-space communication and coding: A marriage made nation of interleaved codes: Performance analysis, design, and iterative
in heaven,” in Lecture Notes on Control and Information Sciences 82, J. decoding,” IEEE Trans. Inform. Theory, vol. 44, pp. 909–926, May 1998.
Hagenauer, Ed.. Bonn, Germany: Springer-Verlag, 1992. [34] G. D. Forney, Jr., “Coding and its application in space communications,”
[7] R. R. Green, “A serial orthogonal decoder,” in Jet Propulsion Labo- IEEE Spectrum, vol. 7, pp. 47–58, June 1970.
ratory Space Programs Summary, vol. IV, no. 37–39, June 1966, pp. [35] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE
247–251. Trans. Inform. Theory, vol. IT-28, pp. 55–67, Jan. 1982.
[8] J. M. Wozencraft and B. Reiffen, Sequential Decoding. Cambridge, [36] , “Trellis-coded modulation with redundant signal sets—Part I:
MA: MIT Press, 1961. Introduction,” IEEE Commun. Mag., vol. 25, pp. 5–11, Feb. 1987.
[9] R. M. Fano, “A heuristic discussion of probabilistic decoding,” IEEE [37] , “Trellis-coded modulation with redendant signal sets—Part II:
Trans. Inform. Theory, vol. IT-9, pp. 64–74, Apr. 1963. State of the art,” IEEE Commun. Mag., vol. 25, pp. 12–21, Feb. 1987.
[10] S. Lin and H. Lyne, “Some results on binary convolutional code [38] G. D. Forney, Jr., “Coded modulation for band-limited channels,” IEEE
generators,” IEEE Trans. Inform. Theory, vol. IT-13, pp. 134–139, Jan. Inform. Theory Soc. Newslet., vol. 40, pp. 1–7, Dec. 1990.
1967. [39] L. F. Wei, “Rotationally invariant convolutional channel coding with
[11] J. L. Massey and D. J. Costello, Jr., “Nonsystematic convolutional codes expanded signal space. Part II: Nonlinear codes,” IEEE J. Select. Areas
for sequential decoding in space applications,” IEEE Trans. Commun. Commun., vol. SAC-2, pp. 672–686, Sept. 1984.
Technol., vol. COM-19, pp. 806–813, Oct. 1971. [40] , “Trellis-coded modulation with multidimensional constella-
[12] J. W. Layland and W. A. Lushbaugh, “A flexible high-speed sequential tions,” IEEE Trans. Inform. Theory, vol. IT-33, pp. 483–501, July
decoder for deep space channels,” IEEE Trans. Commun. Technol., vol. 1987.
COM-19, pp. 813–820, Oct. 1971. [41] M. V. Eyuboglu, G. D. Forney, Jr., P. Dong, and G. Long, “Advanced
[13] G. D. Forney, Jr., and E. K. Bower, “A high-speed sequential decoder: modulation techniques for V. fast,” Euro. Trans. Telecommun., vol. 4,
Prototype design and test,” IEEE Trans. Commun. Technol., vol. COM- pp. 243–256, May–June 1993.
19, pp. 821–835, Oct. 1971. [42] G. Lang and F. Longstaff, “A Leech lattice modem,” IEEE J. Select.
[14] A. J. Viterbi, “Error bounds for convolutional codes and an asymptot- Areas Commun., vol. 7, pp. 968–973, Aug. 1989.
ically optimum decoding algorithm,” IEEE Trans. Inform. Theory, vol. [43] British Telecom, “Code choice for V. fast,” Contribution D0, CCITT
IT-13, pp. 260–269, Apr. 1967. Study Group 14, Geneva, Switzerland, Aug. 1993.
[15] A. J. Viterbi, J. K. Wolf, E. Zehavi, and R. Padovani, “A pragmatic [44] A. R. Calderbank and L. H. Ozarow, “Nonequiprobable signaling on the
approach to trellis-coded modulation,” IEEE Commun. Mag., vol. 27, Gaussian channel,” IEEE Trans. Inform. Theory, vol. 36, pp. 726–740,
pp. 11–19, July 1989. July 1990.
2560 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 6, OCTOBER 1998

[45] T. R. N. Rao and E. Fujiwara, Error-Control Coding for Computer modulation and FEC,” in Proc. 1997 IEEE Int. Conf. Communications
Systems. Englewood Cliffs, NJ: Prentice-Hall, 1989. (Montreal, Que., Canada, June 1997).
[46] H. Imai, Ed., Essentials of Error-Control Coding Techniques. San [74] F. Halsall, Data Communications, Computer Networks, and Open Sys-
Diego, CA: Academic, 1990. tems, 4th ed. Wokingham, U.K.: Addison-Wesley, 1995.
[47] C. L. Chen, “Symbol error-correcting codes for computer memory [75] S. B. Wicker, Error Control Systems for Digital Communication and
systems,” IEEE Trans. Comp., vol. 41, pp. 252–256, Feb. 1992. Storage. Englewood Cliffs, NJ: Prentice Hall, 1995.
[48] E. Fujiwara, “Application of coding theory to computer systems,” in [76] D. Bertsekas and R. Gallager, Data Networks. Englewood Cliffs, NJ:
Proc. 1997 IEICE General Conf., Mar. 1997, vol. TA-3-8, pp. 572–573. Prentice Hall, 1987.
[49] P. Fire, “A class of multiple-error-correcting binary codes for non- [77] G. Castagnoli, J. Ganz, and P. Graber, “Optimum cyclic redundancy-
independent errors,” Sylvania Rep. RSL-E-2, Sylvania Electronics De- check codes with 16-bit redundancy,” IEEE Trans. Commun., vol. 38,
fense Lab., Reconnaissance Syst. Div., Mountain View, CA, Mar. 1959. pp. 111–114, Jan. 1990.
[50] K. A. S. Immink, “RS codes and the compact disc,” in Reed–Solomon [78] P. Merkey and E. C. Posner, “Optimum cyclic redundancy codes for
Codes and Their Applications, S. B. Wicker and V. K. Bhargava, Eds. noisy channels,” IEEE Trans. Inform. Theory, vol. IT-30, pp. 865–867,
Piscataway, NJ: IEEE Press, 1994, pp. 41–59. Nov. 1984.
[51] H. Imai and Y. Nagasaka, “On decoding methods for double-encoding [79] K. A. Witzke and C. Leung, “A comparison of some error detecting CRC
systems,” Electron. Commun. Japan, vol. 65-A, pp. 36–44, Dec. 1982. code standards,” IEEE Trans. Commun., vol. COM-33, pp. 996–998,
[52] M. Hattori, N. Ohya, M. Sasano, K. Sato, and N. Shirota, “Improved Sept. 1985.
general purpose Reed–Solomon erasure encoder/decoder chip,” in Proc. [80] R. J. Benice and A. H. Frey, “An analysis of retransmission systems,”
1993 Picture Coding Symp., Mar. 1993. IEEE Trans. Commun. Technol., vol. COM-12, pp. 135–145, Dec. 1964.
[53] E. J. Weldon Jr., “Difference set cyclic codes,” Bell Syst. Tech. J., vol. [81] S. Lin, D. J. Costello Jr., and M. J. Miller, “Automatic-repeat-request
45, pp. 1045–1055, 1966. error control schemes,” IEEE Commun. Mag., vol. 22, pp. 5–17, Dec.
[54] T. Kuroda, M. Takada, T. Isobe, and O. Yamada, “Transmission scheme 1984.
of high-capacity FM multiplex broadcasting system,” IEEE Trans. [82] R. L. Freeman, Practical Data Communications. New York: Wiley,
Broadcasting, vol. 42, pp. 245–250, Sept. 1996. 1995.
[55] O. Yamada, “Development of an error-correction method for data packet [83] D. E. Comer, Internetworking with TCP/IP, vol. I, 3rd ed. Englewood
multiplexed with TV signals,” IEEE Trans. Commun., vol. COM-35, pp. Cliffs, NJ: Prentice-Hall, 1995.
21–31, Jan. 1987. [84] E. Murphy, S. Hayes, and M. Enders, TCP/IP, Tutorial and Technical
[56] J. L. Massey, Threshold Decoding. Cambridge, MA: MIT Press, 1963. Overview, 5th ed. Upper Saddle River, NJ: Prentice-Hall, 1995.
[57] M. Takada and T. Kuroda, “Threshold decoding algorithm using trellis [85] J. M. Wozencraft and M. Horstein, “Digitalised communication over
diagram for majority logic decodable codes,” in Proc. 1996 Int. Symp. two-way channels,” presented at the Fourth London Symp. Information
Information Theory and Its Applications (Victoria, BC, Canada, Sept. Theory, London, U.K., Sept. 1960.
[86] , “Coding for two-way channels,” Tech. Rep. 383, Res. Lab.
1996), vol. 1, pp. 397–400.
[58] K. Yamaguchi, H. Iizuka, E. Nomura, and H. Imai, “Variable threshold Electron., MIT, Cambridge, MA, Jan. 1961.
[87] S. B. Wicker, “Reed–Solomon error control coding for data transmis-
soft decision decoding,” IEICE Trans., vol. 71-A, pp. 1607–1614, Aug.
sion over Rayleigh fading channels with feedback,” IEEE Trans. Veh.
1988 (in Japanese). English version available in Electron. Commun.
Technol., vol. 41, pp. 124–133, May 1992.
Japan, vol. 72, pp. 65–74, Sept. 1989,. [88] A. Drukarev and D. J. Costello Jr., “Hybrid ARQ error control using
[59] ISO-IEC, “Coding of moving pictures and associated audio up to about
sequential decoding,” IEEE Trans. Inform. Theory, vol. IT-29, pp.
1.5 Mbit/s,” ISO-ICE, Std. CD 11172-3, Part 3–Audio.
521–535, July 1983.
[60] J. Hagenauer, “Rate-compatible punctured convolutional codes and their
[89] H. Yamamoto and K. Itoh, “Viterbi decoding algorithm for convolutional
applications,” IEEE Trans. Commun., vol. 36, pp. 389–400, Apr. 1988.
codes with repeat request,” IEEE Trans. Inform. Theory, vol. IT-26, pp.
[61] G. Plenge, “A new sound broadcasting system,” EBU Tech. Rev., vol.
540–547, Sept. 1980.
246, pp. 87–112, Apr. 1991. [90] P. Sindhu, “Retransmission error control with memory,” IEEE Trans.
[62] C. Weck, “Unequal error protection for digital sound broadcast-
Commun., vol. COM-25, pp. 473–479, May 1977.
ing—Principle and performance,” presented at the 94th AES Conv., [91] D. Chase, “Code combining—A maximum-likelihood decoding ap-
preprint 3459, Berlin, Germany, Mar. 1993. proach for combining an arbitrary number of noisy packets,” IEEE
[63] H. Imai and S. Hirakawa, “A new multilevel coding method using error- Trans. Commun., vol. COM-33, pp. 385–393, May 1985.
correcting codes,” IEEE Trans. Inform. Theory, vol. IT-23, pp. 371–377, [92] H. Krishna and S. Morgera, “A new error control scheme for hybrid-
May 1977. ARQ systems,” IEEE Trans. Commun., vol. COM-35, pp. 981–990, Oct.
[64] R. H. Morelos-Zaragoza, M. P. C. Fossorier, S. Lin, and H. Imai, 1987.
“Multilevel block coded modulation with unequal error protection,” in [93] S. Morgera and V. Oduol, “Soft decision decoding applied to the
Proc. 1997 IEEE Int. Symp. Information Theory (Ulm, Germany, July generalized type-II hybrid-ARQ scheme,” IEEE Trans. Commun., vol.
1997), p. 441. 37, pp. 393–396, Apr. 1989.
[65] J. Hagenauer, “Zur Kanalkapazitaet bei Nachrichtenkanaelen mit Fading [94] S. Lin and P. S. Yu, “A hybrid-ARQ scheme with parity retransmission
und gebuendelten Fehlern,” AEU, Electron. Commun., vol. 34, pp. for error control of satellite channels,” IEEE Trans. Commun., vol.
229–237, 1980. COM-30, pp. 1701–1719, July 1982.
[66] Qualcomm Inc., “An overview of the application of code division mul- [95] M. B. Pursley and S. D. Sandberg, “Incremental redundancy transmis-
tiple access to digital cellular systems and personal cellular networks,” sion for meteor burst communications,” IEEE Trans. Commun., vol. 39,
Qualcomm, Inc., San Diego, CA, May 1992. pp. 689–702, May 1991.
[67] J. Proakis, Digital Communications. New York,: McGraw-Hill, 1989. [96] S. B. Wicker and M. Bartz, “Type-II hybrid-ARQ protocols using
[68] J. Hagenauer, “Source controlled channel decoding,” IEEE Trans. Com- punctured MDS codes,” IEEE Trans. Commun., vol. 42, pp. 1431–1440,
mun., vol. 43, pp. 2449–2457, Sept. 1995. Apr. 1994.
[69] C. Erben, T. Hindelang, and W. Xu, “Quality enhancement of coded and [97] , “The design and implementation of type-I and type-II hybrid-
corrupted speeches in GSM mobile systems using residual redundancy,” ARQ protocols based on first-order Reed-Muller codes,” IEEE Trans.
in Proc. ICASSP97 (Munich, Germany, Apr. 1997), vol. 1, pp. 259–262. Commun., vol. 42, pp. 979–987, Mar. 1994.
[70] W. Xu, J. Hagenauer, and J. Hollmann, “Joint source-channel decoding [98] G. Benelli, “An ARQ scheme with memory and soft error detectors,”
using the residual redundancy in compressed images,” in Proc. 1996 Int. IEEE Trans. Commun., vol. 33, pp. 285–288, Mar. 1985.
Conf. Communications (Dallas, TX, June 1996), pp. 142–148. [99] “An ARQ scheme with memory and integrated modulation,”
[71] J. Hagenauer, “The turbo principle: Tutorial introduction and state of IEEE Trans. Commun., vol. COM-35, pp. 689–697, July 1987.
the art,” in Proc. Int. Symp. Turbo Codes (Brest, France, Sept. 1997), [100] J. Metzner, “Improvements in block-retransmission schemes,” IEEE
pp. 1–11. Trans. Commun., vol. COM-27, pp. 524–532, Feb. 1979.
[72] F. Burkert, G. Caire, J. Hagenauer, T. Hindelang, and G. Lechner, [101] J. Metzner and D. Chang, “Efficient selective-repeat ARQ strategies
“Turbo decoding with unequal error protection applied to GSM speech for very noisy and fluctuating channels,” IEEE Trans. Commun., vol.
coding,” in Proc. 1996 IEEE Global Communications Conf. (London, COM-33, pp. 409–416, May 1985.
U.K., Nov. 1996), pp. 2044–2048. [102] B. A. Harvey and S. B. Wicker, “Packet combining systems based on

M
[73] R. Herzog, A. Schmidbauer, and J. Hagenauer, “Iterative decoding the Viterbi decoder,” IEEE Trans. Commun., vol. 42, pp. 1544–1557,
and despreading improves CDMA-systems using -ary orthogonal Apr. 1994.

View publication stats