IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO.

11, NOVEMBER 1989

1113

High-Rate Punctured Convolutional Codes for Viterbi and Sequential Decoding

DAVID HACCOUN,
SENIOR

MEMBER, IEEE,

AND

GUY BEGIN,

STUDENT MEMBER, IEEE

Abstruct-This paper investigates high-rate punctured convolutional codes suitable for Viterbi and sequential decoding. Results on known short-memory codes M I 8 discovered by others are extended. Weight spectra and upper bounds on the bit error probability of the best known punctured codes having memory 2 I M I 8, and coding rates 2/3 5 R I 7/8 are provided. Newly discovered rate-2/3 and 3/4 long-memory punctured convolutional codes with 9 % M s 23 are provided together with the leading terms of their weight spectra and their bit error performance bounds. Some results of simulation with sequential decoding are given.

I. INTRODUCTION OR discrete memoryless channels where the noise is essentially white (such as the space and satellite channels), systems using convolutional encoding at the transmitting end of the link and probabilistic decoding at the receiving end are among the most attractive means of approaching the reliability of communication predicted by the Shannon theory; these systems provide substantial coding gains while being readily implementable. By far, error control techniques using convolutional codes have been dominated by low-rate R = l / u codes [lJ-[3]. Optimal low-rate codes providing large coding gains are available in the literature [4]-[ lo], and practical implementations of powerful decoders such as Viterbi or sequential exist for data rates in the range of 10-40 Mbitsls [11]-[13]. However, as the trend for ever increasing data transmission rates and high error performance continues while conserving bandwidth, the needs arise for good high-rate R = b/u convolutional codes as well as practical encoding and decoding techniques for these codes. Unfortunately, a straightforward application of Viterbi and sequential decoding to high-rate codes becomes very rapidly impractical as the coding rate increases. Furthermore, a conspicuous lack of good nonsystematic long-memory (M > 9) convolutional codes with rates R larger than 2/3 prevails in the literature. A significant breakthrough occured recently with the advent of high-rate punctured convolutional codes [ 141 where the inherent difficulties of coding and decoding of high-rate codes can be almost entirely circumvented. Viterbi or sequential decoding of rate-b/u punctured convolutional codes is hardly more complex than for rate-llu codes and, furthermore, either technique may be easily applicable to adaptive and variablerate decoding [lo], [14]-[21]. In this paper, we present high-rate punctured convolutional codes suitable for Viterbi and sequential decoding. Extending results on good short-memory codes (M < 9) discovered by others 1141-[16], we provide the weight spectra and upper

bounds on the bit error probability of the best known punctured codes having memory 2 5 M 5 8 and coding rates 2/3 5 R 5 7/8. Using computer search techniques, newly discovered rate-213 and 314 long-memory punctured convolutional codes with 9 5 M 5 23 are derived from the best known rate-112 codes of the same memory. These codes, which are suitable for sequential decoding, are provided together with the leading terms of their weight spectra and their bit error performance bounds. The paper is structured as follows. Section I1 introduces the encoding of punctured codes and their perforation patterns. Section 1 1 presents Viterbi decoding for punctured codes and 1 the extension to sequential decoding. The search for good punctured code is the object of Section IV. This section contains the principal new results of the paper: weight spectra of known short-memory codes and new good rate-213 and 314 long-memory punctured codes, together with their weight spectra and bit error performance bounds. Accuracy of these results is verified by theoretical evaluation of the transfer functions of rate-213 and 415 codes. Finally, some simulation results for sequential decoding are given in Section V, demonstrating the advantages of using high-rate punctured convolutional codes over usual codes of the same rate. 11. BASIC CONCEPTS OF PUNCTURED CONVOLUTIONAL CODES A . Encoding of Punctured Codes A punctured convolutional code is a high-rate code obtained by the periodic elimination (i.e., puncturing) of specific code symbols from the output of a low-rate encoder. The resulting high-rate code depends on both the low-rate code, called the original code, and on both the number and specific positions of the punctured symbols. The pattern of punctured symbols is called the perforation pattern of the punctured code, and is conveniently described in a matrix form called the perfora-

tion matrix.
Consider constructing a high-rate R = b/u punctured convolutional code from a given original code of any low-rate R = l/uo. From every uob code symbols corresponding to the encoding of b information bits by the original encoder, a number S = (uob - U) symbols are deleted according to some chosen perforation pattern. The resulting rate is then R = b/(uob - S), which is equal to the desired target rate R = b/u. By a judicious choice of the original low-rate code and perforation pattern, any rate code may be thus obtained [20], 1211. For example, Fig. 1 shows the trellis diagram of a rate-1/2, memory M = 2 code where every fourth symbol is punctured (indicated by X on every second branch on the diagram). Reading this trellis two branches at a time and redrawing it as in Fig. 2, we see that it corresponds to a rate-213, memory M = 2 code. A punctured rate-213 code has therefore been obtained from an original rate-112 encoder.

Paper approved by the Editor for Coding Theory and Applications of the IEEE Communications Society. Manuscript received February 9, 1888; revised March 25, 1988. This work was supported in part by the Natural Sciences and Engineering Research Council of Canada and by the Fonds FCAR of Quebec. This paper was presented in part at the 1988 IEEE International Symposium on Information Theory, Kobe, Japan, June 1988. The authors are with the Department of Electrical Engineering, Ecole Polytechnique de Montreal, Montreal, P.Q. H3C 3A7, Canada. IEEE Log Number 8930918.

B. Perforation Patterns An encoder for rate R = b/u punctured codes may be visualized as consisting of an original low-rate R = l/uo convolutional encoder followed by a symbol selector or sampler which deletes specific code symbols according to a given perforation pattern. The perforation pattern may be

0090-6778/89/1100-1113$01 .OO 0 1989 IEEE

1114

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989

rli Fig. 1. Tels for M = 2, R = 1/2 convolutional code.

rli Fig. 2. Tels for punctured M = 2, R = 2/3 code.

expressed as a matrix [PIhaving uo rows and b columns, with only binary elements 0s and ls, corresponding to the deleting or keeping of the corresponding code symbol of the original encoder. Clearly, both the punctured code and its rate can be varied by suitably modifying the elements of the perforation matrix. For example, starting from an original rate-1/2 code, the perforation matrix of the rate-213 punctured code of Fig. 1 is given bv
r

whereas a r a t e 4 5 code could be obtained using the perforation matrix

code. With an orthogonal perforation pattern, the perforation matrix has uo = U rows and b columns, with each row containing only one element 1. Orthogonal perforation patterns ensure that all the different generators of the original low-rate Vuo code are used in deriving the desired punctured rate-b/u code. In particular, it can be shown that any punctured code can be obtained by means of an orthogonal perforation pattern. Using this concept punctured codes identical to the best known usual rate-2/3 and 3/4 codes have been obtained by the authors [2 13. The basic notions of encoding punctured codes having been established, the problem of their decoding by both the Viterbi algorithm and sequential decoding is examined in the next section.

m. DECODING PUNCTURED CODES OF

Variable-rate coding is readily obtained if all punctured rates of interest are obtained from the same low-rate encoder. Only the perforation matrices need to be modified accordingly as illustrated by [Pi] and [Pz]. Variable-rate coding may be further specialized by adding the restriction that all the code symbols of the higher rate punctured codes are required by the lower rate codes. This iia restriction implies m n m l modifications of the perforation matrix as the code rates vary. Punctured codes satisfying this restriction are said to be rate-compatible. Rate-compatible punctured codes are especially useful in some rate-adaptive ARQIFEC applications since only the incremental redundancy needs to be transmitted as the coding rate is decreased. Families of good noncatastrophic short-memory rate-compatible punctured codes with rates varying from 8/9to 114 have been found by Hagenauer [191. Finally, another class of perforation patterns called orthogonal perforation patterns plays an important part in the search of specific punctured codes [21]. An orthogonal perforation pattern is a pattern in which any code symbol that is not punctured on one of the b branches is punctured on all the other (b - 1) branches of the resulting rate-b/u punctured

A. Viterbi Decoding
Given the received sequence from the channel, Viterbi decoding consists essentially of computing the likelihood (that is the metric) that a particular sequence has been transmitted for every possible encoder state. For rate R = b/u codes, there are 2 paths merging at every state and only the path with the largest metric is selected at each state. The process is repeated for each of the encoder states, so that clearly, as b increases, the complexity of decoding increases also very rapidly. Now for punctured high-rate b/u codes, Viterbi decoding is hardly more complex than for the original low-rate l/uo code from which the punctured codes are derived. The decoding is performed on the trellis of the original low-rate code where the only modification consists of discarding the metric increments corresponding to the punctured code symbols. Given the perforation pattern of the code, this can be readily performed by inserting dummy data into the positions corresponding to the deleted code symbols. In the decoding process this dummy data is discarded by assigning them the same metric value (usually zero) regardless of the code symbol, 0 or 1. For either hard or soft-quantized channels, this procedure in effect

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING USUAL CODE

1115

inhibits the conventional metric calculation for the punctured symbols. Therefore, Viterbi codecs for high-rate punctured codes involve none of the complexity of the straightforward decoding of high-rate b / u codes. They can be implemented by adding relatively simple circuitry to the codecs of the original low-rate l/uo code. Furthermore, since a given low-rate l / u o code can give rise to a large number of high-rate punctured codes, the punctured approach leads to a very attractive realization of variable-rate Viterbi decoding. In addition to the metric inhibition mentioned above, the only coding ratedependent modification in a variable-rate codec is the truncation path length, which must be increased with the coding rate. All other operations of the decoder remain essentially unchanged [14]-[2 1).

U3

U
I W

In
I

Decoded Path Paths Examined by Decoder

- Incorrect

4
DEPTH (a)

PUNCTURED CODE

B. Sequential Decoding In sequential decoding, the decoding of the received message is performed one branch at a time without searching v) the entire tree. The exploration of the most-likely fraction of I ! the tree is performed along the path having the current largest U Decoded Path I likelihood function or Fano metric 121. Starting from the root W r - Incorrect Paths node of the tree, the path selected to be extended one step Examined by Decode1 further in the tree is the path whose metric is the largest among all previously examined paths. For practical reasons, the tree is limited to a depth of a few hundreds to a few thousands branches, and the path that first reaches the end of the tree is 1 1' 2 2' 3 3 4 4 5 5 6 6' ' ' ' accepted as the decoded path. DEPTH For binary convolutional codes of rate R = b/u, 2 h (b) branches each with U code symbols emerge from each node of Fig. 3. Paths explored by the stack algorithm: a) usual R = 2/3 code; b) the encoding tree. Since decoding is an exploration of tree punctured R = 2/3 code. paths, a straightforward application of the decoding algorithm to codes of high-rate b / u may become very quickly unacceptable. Several methods using discarding thresholds to eliminate stack algorithm for both an usual and punctured rate-2/3 code unlikely paths have been proposed to alleviate this difficulty are compared. Since each decoding cycle of the algorithm [22], 1231. However, even with discarding thresholds, the use involves 22 = 4 stack entries for the usual code, but only 2 for of sequential decoding for high-rate codes is somewhat limited the punctured code, then clearly, the small metric dips of the by the lack of suitable long memory codes [24]-[26]. By correct path are far more costly, in terms of computations and considering punctured codes instead, both difficulties of stack storage, for the usual than for the punctured code. For computational complexity and finding good high-rate codes example, Fig. 3 shows that in reaching the same decoded path, a substantially larger number of nodes must be explored with can be circumvented. Just as for Viterbi decoding, the punctured approach to the usual code than with the punctured code. An additional advantage of the punctured approach is that high-rate codes can, as easily, be applied to sequential decoding. Again, decoding is performed on the tree of the the flag procedure usually used to save stack entries and speedoriginal low-rate code rather than on the high-rate code, and in up the decoding of rate-l/u codes can be readily implemented principle any sequential decoding algorithm could be used. here at no additional cost [ 171, [ 181, [20]. The use of rejection However, only the Zigangirov-Jelinek or stack algorithm will thresholds to discard unlikely branch extensions can even be be considered here [27], [28]. Since it is very simple and added up at hardly any cost. Therefore, the stack algorithm readily amenable to variants and generalizations [29], [30], can be very easily adapted for the decoding of punctured this algorithm is a good candidate for the decoding of codes, and the method can lead to substantial simplifications punctured codes since it requires only minimal modifications and stack storage savings over the direct approach. Finally, as [ 171. Given the perforation pattern the decoder proceeds one with variable-rate Viterbi decoding, variable-rate sequential branch at a time just like for the decoding of the original low- decoding can be easily implemented. rate l/uo code. The algorithm is repeated through (b - 1) Long-memory codes present no problem in sequential elementary intermediate steps involving the original low-rate decoding. The achievement of low error probabilities, fast code. At each intermediate step, the top node of the stack is decoding speeds, and low overflow probabilities require the extended into its two uo-symbol branches from which specific selection of long memory codes having a large free distance symbols are discarded. Again this is accomplished by inhibit- and a good distance profile. Extensive lists of such codes exist ing the metric evaluation of these punctured symbols, that is, for low-rate l / u codes [25], [31]. However, with few by assigning them the same metric increment (usually zero). exceptions optimal long-memory codes suitable for sequential The advantages of the procedure is that the likelihood that decoding and with rates higher than 314 are not known [24], the decoder is following the correct path is evaluated on a bit- [26]. By taking the punctured approach, the search for good by-bit basis rather than on the basis of blocks of b bits like in a high-rate codes becomes much simplified. The search probbrute-force decoding. Hence, the decision to retreat in the tree lems for punctured codes is examined in Section IV. and follow another path may be taken earlier, at a possible reduction of the computational variability and required stack C. Error Performance storage. In addition, since each elementary path extension For discrete memoryless channels an upper bound on the bit involves only two branches, for a rate-b/u code, the forward error probability of a convolutional code can be obtained. The motion of the decoder involves 2b entries in the stack rather derivation of the bound is based on an union bound argument than the 2 b entries required of a straightforward decoding. on the transfer function T(D, B) of the code which describes This is illustrated in Fig. 3 where the paths explored by the the weight distribution, or weight spectrum, of the incorrect

"

'

'

'

'

1116

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989

codewords and the number of bit errors on these paths [l], [32]. The entire transfer function of the code is rarely known but an upper bound can still be calculated using only the first few terms of two series expansions related to the transfer function T ( 0 , B), that is

and

In these expressions, d- is the free distance of the code and

aj is the number of incorrect paths or adversaries of Hamming

weight j, j 5 dh, that diverge from the correct path and remerge with it sometime later. As for cj, it is simply the total number of bit errors in all the adversaries having a given Hamming weight j . Using the weight spectrum, an upper bound on the bit error probability PE of a code of rate R = b / u is given by

(3)
The evaluation of this bound depends on the actual expression of the pairwise error probability Pj, which in turn depends on the type of modulation and channel parameters used [13, [32]. For coherent PSK modulation and unquantized additive white Gaussian noise channels, the error probability between two codewords that differ over j symbols is given by

Pi= Q(d2jREb/No)
where

(4)

QW=

s, 6 (-z
- 1

-exp

z2) dz

(5)

and where Eb/No is the energy per bit-to-noise density ratio. A good evaluation of the bound on PErequires knowledge of the transfer functions (1) or (2). However, for the vast majority of codes only the first few terms of either functions are known, and very often only the leading coefficients a d f , and c d f , are available. But for channels with large EJN0 values such as those usually used with high-rate codes, the bound on PEis dominated by the first term, c d k . Naturally, bound (3) is also applicable for punctured codes whether decoded by a Viterbi or a sequential decoder. Therefore, in deriving the bit error performance of punctured codes, the free distances and at least the first few terms of the weight spectra of these codes must be obtained. Such partial weight spectra are provided in this paper for the best known short-memory codes given by others [l5], [16] as well as for the new long-memory codes discovered by the authors.

IV. SEARCH GOOD PUNCTURED CODES FOR Since punctured coding was originally devised for Viterbi decoding, the criterion of goodness for these codes was the free distance, and hence the maximal free distance punctured codes that first appeared in the literature were all shortmemory codes [141-[ 161. For sequential decoding, good longmemory punctured codes should have both a large free distance and a good distance profile. In searching for good punctured codes of rate b / u and memory M , one is confronted with the problem of finding both an original code of memory M and rate R = 1/UO, and its accompanying perforation pattern. Not unlike the search for usual convolutional codes, the search for punctured codes is often based on intuition and trial and error rather than on a strict mathematical construction 1211. An approach that

yielded good results is based on the intuition that good codes generate good codes. Consequently one could choose a known good code of memory M and rate R = l/uo, (e.g.. R = 1/2, 1/3, 1/4, - ) and exhaustively try out all possible perforation patterns to generate sood punctured codes of rates R = b / u and memory M . Naturally, if families of variablerate codes are desired, then all the perforation patterns must be applied to the same low-rate code. Furthermore, if the codes are to be rate-compatible, then the perforation patterns must be selected accordingly [191. Obviously puncturing a code reduces its free distance, and hence a punctured code cannot achieve the free distance of its original code. Although the freedistance of a code increases as its rate decreases, using original codes with rates l/uo lower than 1/2 does not always warrant punctured codes with larger free distances since, for a given b and U, the proportion of deleted symbols, S/uo b = 1 - (u/u,,b), also increases with UO. Consequently, good results and ease of implementation tend to favor the use of rate-1/2 original codes for generating good punctured codes with coding rates of the form R = (U - l)/u that may be suitable for either Viterbi decoding or sequential decoding. Further results on both short-(M 5 8) and long-memory punctured codes (M 2 9) that are derived from R = 1/ 2 codes are provided in this paper. Although one could select the punctured code on the basis of its free distance only, a finer method consists of determining the weight spectrum of the punctured code according to (1) and (2) and then calculating the bit error probability bound (3). The code yielding the best error performance may be thus selected as the best punctured code, provided it is not catastrophic. Clearly then, starting from a known optimal low-rate code, a successful search for good punctured codes hinges on the ability of determining the weight spectrum corresponding to each possible perforation pattern. Although seemingly simple, finding the weight spectrum of punctured codes turned out to be a very difficult task. This is due to the fact that even if the spectrum of the low-rate original code were available, the spectrum of the punctured code cannot be derived from it. One has to go back exploring the tree or trellis of the low-rate original code and apply to each path of interest the perforation pattern. For the well-known short-memory codes, the procedure is at best a rediscovery of their weight spectra, whereas for long-memory codes where often only the free distance is known, it is a novel determination of their spectra. The problem is further compounded by the fact that since puncturing a path reduces its Hamming weight, then in obtaining a given number of spectral terms, a larger number of paths must be explored over much longer lengths for a punctured than for a usual code. In our search we have used recently developed algorithms for the spectral determination of convolutional codes [33][35]. Each algorithm has been designed to be especially efficient within a given range of memory lengths, making it possible to obtain substantial extentions of the known spectra of the best convolutional codes. Obviously, the spectral terms obtained by these exploration methods must match exactly the corresponding terms derived from the transfer function of the code. This verification has been conducted for two different codes by evaluating their transfer function as detailed in the Appendix.

A . Short-Memory Punctured Codes A number of short-memory (M 5 8) punctured codes of rates R = (U - l)/u varying from 213 to 16/17 have been proposed by Cain et al. [14], Yasuda et al. 1151. [16], and more recently by Hagenauer [19] for rate-compatible codes. In particular, in [15] all the memory-6 punctured codes of rates varying from 213 to 16/17 have been derived from the same original memory-6, rate-112, optimal convolutional code due

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING TABLE I WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 s M s 8
original code

1117
TABLE 1 1 1 WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 5 M 1 8
original code M GI G ,

P
10

I 3 1

d,

punctured code (a.,n=d,.d,+l,dj+2, [c,,n = d,,d, l,d, + 2 ,

..)

. . .]

(1.4,14.40.116.339,991,2897,

18468,24752) [ I , 10,54,226,856,3072,10ffl7,35998, 119478,3909041

(3,11,35,114,381,1276,4257.14208,

2/3,

4/5,

P
10 11

dj

47413,158245)

#
2

1100

[ 10,43.200,826,3336,13032,49836,187480,
10

696290,25595211 f 1.0,27.0.345,0.4528.0. 59435.0)

[1,0,124,0,2721,0.50738,0,

862127,0]
10 11

(19.0,220,0,3089,0,427~,0.

588022,O) [96,0.1904,0,35936.0.638393. 0,10657411] 6

(1,16,48.158,642,2435,9174,

1000 1111

11 10

34705,131585.499608)
[3,70,285,1276.6160,27128,117019,

498860,2103891,8784123]
10 11

(9,35,104,372,1552,5905.

22148,85189,323823,1232139) l47.237.835.3637.17770.76162, 322120,1374323,5730015,23763275]

U1 I I PO
7 247 371
10 10

I i
[1,36,309,2060,12320,69343,375784, 1983150,10262827,52316541 5,36,200,1070,5919,32721,180476,

punctured code (o,,n= d,,d, l,d, + 2 , . . .) [c,,n = d,,d, 1,dl + 2 , . . .] 2 I (1,12,53,238,1091,4947,22459 102030,463451,2105128)

+ +

995885,5495386,30323667)

[14,194,1579,11313,77947,514705,3305113, 3,16,103,675,3969,24328.147313,

20808587,129003699,790098445

I 19259760,132078031,896198879]
I (7,54.307,2005,12970,83276,

897523,5447618,33133398) I11.78.753.6901.51737.386465.2746036.

~34556.343l703.22040ll0.l41546111 ) [40,381,325 1,27123.2 1345 1 , I fi2 1873, 1201l339,87380826,627189942,445L640~18] 3,24,I72.lI58.7409,4U729. 319861,209797113765538,90315667) 112.188.1732.15256.121372.945ffl5,
(20.115.694 4816.32027.210909. l3928669223171.610132364036650541

[ l68,1232,9120,78715,626483,4758850,
663936,4431049,29536078,197041 141)

561

753

1 10 1
[31,469.4205,34011,268650,2113955,

I 1 127923,544616,2313272,9721227]
TABLE I1 WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R 2 5 M s 8
M
original code GI GZ
2
=

I11.46.324.1594.6425.29069.

3/4,

TABLE IV WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R

original code punctured code (a,,n=d,,d,+l,d,+Z, ..) [ + , n = dj,d, 1,dl + 2 ,...]

= 5/6,

P
IO 1 1 10

d, 3

punctured code (a,,n=d,,d,+l,d,+Z, .) [c,,,n = d,,d, l,d, + 2 ,...]

P
10111 110 0 0

d,

(6,23,80,290,1050,3804,13782,

(2,26,129,633,3316,17194,88800,

j + $
3 15
6

49930.180890.655342)
[15,104,540,2557,11441,49340,207335,

459295,2375897,12288610)
[2,111,974,6857,45555,288020,1758617, 10487425,61445892,355061333]

854699,3471621,139363811 17
110 10 1

(29,0,532,0,10059,0,190112,0,

3593147.0) [124,0,4504,0,126M9,0,3156062,0,

10 10 0 1 10 1 1

(l5,96,601,3918,2539l,lffl481, 1065835,6906182,44749517,289960852) [63,697,6367,53574,42ffl71,3277878, 24573195,180823448,131 1630186,9405755195]

(5,37,309,2282,16614,122308,

110
[1,7,125,936,5915,36608,216972,

101 1 1 1 I 000

900991,6634698,48853474,359716572) [20,265,3248,32328,297825.2638257,
22710170,191432589,1587788458,13003147461]

100 111

42626,216131,1095495,5557252)
13,85.490.3198,20557.123384,

10 0 0 0 11 1I 1

(19,171,1251,9573,75167,585675, 4558463,35513472,276598647,2154'249884)
[100,1592.17441,166331,1591841,14627480, 131090525,1155743839,10049038363,86420307777] (14,69,654,4996,39699,315371,

133

171

1 I O

(8,31.160.892.451223307
121U~7.625059.32.14RR6.16753077I
2253373,13073811,75152755,428005675]

I 0I

11010 10 10 1

2507890,19921920,158275483,1257455600) [92,528.8694,79453,792114,7375573, 67884974,610875423,5427275376,4766421 5639) 4

(2,51,415,3044,25530,200878,

1 1 100 10 01 1
[239,0,11165,0,422030,0,14812557.

1628427,12995292,104837990,839661288)
[7,426,5244,49920,514857,4779338, 44929071,40647031 1,3672580016,32529952038]

0,493081189,Ol 10 0 236294,1236990,6488527,34056195)
[52,659,3265,21442,133697,805582, 4812492,28107867,162840763,935232173] 10 110 1 10 0 1

(19,187,1499,11809,95407,775775, 6281882,50851245,411841549,3335453320) [ 168,2469,25174,242850,232M29.21768364.

199755735,1807353406,16178271375,143502720686~

to Odenwalder [32]. A more complete list of rate 2/3 to 13/14 punctured codes has been derived from the optimal rate- 112 codes with memory M varying from 2 to 8 and compiled by Yasuda et al. [16]. In this list, for each code the perforation matrix is provided but the weight spectrum is limited to the first term only, that is, the term corresponding to dfree. Using the given perforation patterns, we have extended Yasuda's et al. results by determining the first 10 spectral terms for all the codes having memory lengths 2 5 M 5 8 and coding rates 213, 314,415, 516, 617, and 718. These results are given in Table I-Table VI, respectively. Each table lists the generators of the original low-rate code, the perforation matrix, the free distance of the resulting punctured code, and the coefficients a, and c, of the series expansions (1) and (2) of the corresponding weight spectra. These spectra determina-

tions have been conducted on SUN13 and APPOLO microcomputers and turned out to be quite time consuming. The amount of CPU time required for determining each further spectral coefficient increases with the coding rate by a factor varying from 3.8 to 11 as the coding rate varies from 2/3 to 7/8. Beyond 10 spectral coefficients the computer time becomes prohibitive, reaching several hundred hours with these computers. Now in order to verify the accuracy and validity of these results the entire transfer functions T ( D , B) have been evaluated for both rate-213 and rate-415 memory-2 punctured codes. As detailed in the Appendix, the transfer functions of the codes have been determined by solving the matrix equations describing the transitions between the different states of the encoder. As expected all the spectral terms

1118

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 11, NOVEMBER 1989

TABLE V
WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R = 6/7,

25M18
P
2
5 dj 2 ( % , n = d j , d j + l , d t + Z ,...) [%,n = d j , d j l , d , Z,.. .]

I
4 I 23 I 35

101111 1 1000 0

U
n

(4,39,221,1330,8190,49754. 302405.1~129.11194714.68101647~ [5,186,1942,16642,131415,'981578, ' 7076932,49784878,343825123~0813323] 1000 1 1 2 (1,25,188,1416,10757,81593, 111100 619023,46~330,3~3844,27w85463) [2,134,1696,18284,179989,1678667, 15082912,132368246,1140378555,9~489~26] 1 o 1 o 1 o I 3 I (14.100.828.7198.60847.513573. 1 1010 1 4~760,~675l720,31062M41,i628803066) [69,779,9770,113537,1203746,1~17198, 12070.1682,1167799637,11118330624,104537090090] 1 10 1 10 3 (5,55,517,4523,40476,~2074~2~848, 101001 288725n,2578984.L2,2303819072) 125,475,6302,73704,82~,8816834,

133

171

I
7

I 247 I 371

B I I [5,169,n25.32233.37~1,4169788,454174~, B 1 o 1 o o 1 I 4 II483171499,5063039490,5239471~1,536645404278] I (11,155,1399,13018,1~560,1154C87,

11101 0 10010 1

(1,20,~3,19a1,1~3.169175,15761~, 14656816.136394365.1269388805.11812597193~

TABLE VI
WEIGHT SPECTRA OF YASUDA et al. PUNCTURED CODES WITH R = 718,

2 s M s 8
P
2 5 7 1 1 00000 793483,5293846,35318216,235589046) [8,393,4248,38142,325739,2847528, dj punctured code ( o . , n = d j , d j + l , d j + Z ,...) [ k , n = d j , d j + 1,dj +2,. .]

15

17 11 11 1 0 1 1717289,14547758,123478503,1047166985) [4,219,3456,38973,437072,4492~,

453031M,4rl~B8,4265246076,40445094'
4 23 35 1010011 1 101 10 0

1
I

I 14019214,1~7So3~,1373401554,13593428075)
(49,1414,21358,284324,3544718,4m8392,
~9.l~.l195.12139.123889.1259682.
489726840,5552764343,6 1942612828,682203758430]

f13.145,1471.1~73.143110*1416407.

the rate 314 code than for the rate 415 code. Clearly, it is the coefficients e,,which adversely affect the error performance of the rate 314 code. This anomaly illustrates the fact that selecting a code according to the free distance only is good in general but may sometimes be insufficient. Knowledge of further terms of the spectrum will always provide more insight on the code performance. The theoretical bound on PB,for the original M = 6, R = 1/2 optimal-free-distance code is plotted in Fig. 4, together with all the punctured codes with rates R = 2/3, 3/4, 4/5, 5/6, 6/7, and 718 derived from it. For cornparison purposes Fig. 4 includes also the performance curve of the uncoded coherent PSK modulation. It shows that the performance degradation from the original rate-1/2 code is rather gentle as the the coding rate increases from 112 to 718. At PB = coding gains for the punctured rate-2/3 and 314 codes are 5.1 dF3 and 4.6 dF3, respectively. These results indicate that these codes are indeed very good, even though their free distances, which are equal to 6 and 5 , respectively, are slightly smaller than the free distances of the best known usual rate-213 and 314 codes which are equal to 7 and 6, respectively [24]. The error performance of these codes has been verified using an actual punctured Viterbi codec 1151, [16], and independently, using computer simulation [36]. Both evaluations have been performed using 8-level soft decision Viterbi decoding with truncation path lengths equal to 50,56,96, and 240 bits for the coding rates 213, 314, 718, and 15/16, respectively. Both hardware and software evaluations have yielded identical error performances which closely match the theoretical upper bound. At rate 15/16, the coding gain of the M = 6 code reaches a The fact that such a coding gain substantial 3 dB at PB = can be achieved with only a 7 percent redundancy and a Viterbi decoder that is hardly more complex than for a rate-112 code makes the punctured coding technique very attractive for short-memory codes. For longer codes and larger coding gains, Viterbi decoding becomes impractical and sequential decoding should be considered instead. Newly discovered long-memory-length punctured codes and their performance with sequential decoding are presented next.

B. Long Memory Punctured Codes izs34~i~,i30~ws~~,i331j13258,13562o351~~~ Following the same approach as for the short-memory codes, one could choose a known optimal long-memory code [So, 1360,18971,252751,3165885,3822720, 450898174,5217969030,59518129067,671110852141] of rate 112 and exhaustively try out all possible perforation 6 133 171 1 1 1 1 0 1 0 3 (2,46,499,5291,56179,599557. patterns to generate all punctured codes of high rate R = b / u . 6387194,~117821,72~~96,~41~67~) 10 00 10 1 [9,500,7437,105707,14~43,179092~, The selection of the punctured code is again based on its bit 222282289,27w822556,32434972565~83973015410] error performance, which is calculated from the series 7 247 371 1 0 1 0 1 0 0 4 (26,26L,~32,30389,32927,3571So7, expansion of its transfer function. Here, one of the difficulties 1 10101 1 38799203,421331658,457~8~2,49898647357) [258,3652,52824,746564,9825110,125472545, is that for the original low-rate and long-memory codes of 1567656165,192~335327,23319~14828,~948799~442] interest, only very partial knowledge of their weight spectra is 8 561 753 1 1 0 1 0 1 1 4 (6,132,1289,13986,1539,1694634, 18532588,2028~21,2~74919,24305050429) 1010100 available. In fact, beyond constraint length M = 15, very [70,1842,2~,337514,4548~4,586~237, often only the free distances of these codes are available in the 738611595,9148958248,111799479417,1351261813545] literature [34]. Extending previous work in this area [20], [2 11, a computer obtained by the search algorithms match exactly the corres- search for the best rate-213 and 314 punctured codes with memory length extending from 9 to 23 that are derived from ponding terms of the transfer functions. The bit error probability upper bound PB over both the best known rate-112 has been conducted. For each code the unquantized and hard quantized additive white Gaussian noise first few terms of the weight spectrum have been obtained for channels has been evaluated for all the codes listed in Tables I- each possible distinct perforation pattern. The obviously bad VI [36], using all the listed weight spectra terms. For a l these codes are discarded and the codes to be investigated further are l codes the error performance improves as the coding rate those that provide the largest free distance and the smallest decreases, indicating well-chosen perforation patterns. A number of bit errors in their weight spectra. However, the notable exception concerns the memory M = 3 code where h a l selection of the best punctured codes is based on the the error performance turned out to be slightly better at rate evaluation of the upper bound on the bit error probability. 415 than at rate 314. This anomaly may be explained by an Naturally, the codes obtained with this approach are suitable examination of the spectra of these two codes. As shown in for variable-rate decoding with sequential decoding. The Tables II and III, although the free distances of the rates 314 search has been initially limited to these two rates only since a and 415 codes are df = 4 and df = 3 respectively, the number fine comparison of the punctured codes with the best known terms are _ ___~- for nonsystematic high-rate-codes of the same memory length is of bit errors C , on the various s~ectral ~- far larger --. ~ 5 53 75 10 I1 10 I 1 1 o o o 1o 3

~~~~~

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING

1119
TABLE VUI BEST RATE-314, 9 5 M 5 23 PUNCTURED CODES WITH THEIR WEIGHT SPECTRA, PERFORATION MATRIX, AND ORIGINAL R = 1/2 CODES
punctured code

BEST RATE-213, 9 Sr M

TABLE VI1 23 PUNCTURED CODES WITH THEIR WEIGHT SPECTRA. PERFORATION MATRIX. AND ORIGINAL R = 1/2 CODES
5
origind code
ponctllred code

P I dt I ( o . . n = d , ..d , + l , d t +. 2 .
1

I I [%,n=df,d,+l,dl+Z,
7

,...)
]
IO0
1 I I

(1,10.29,94,415,1589, 5956.23060.88731.338932)

I
6

1 [%,a=d,,dj+l.d,+2.
(4,31.15L.774,3967,2I140,

[3,70,207,836.4411.19580, 82154,353734,1504013,6283078] 0 I 8 I 11 21 65.226 907 3397.

;3z23,~io~~;194iz~.7i8110)
[8,165,560.2321.10932,4692l, 204372,869373,3635291,15l75050]

I O 1 I 6 I 12.7.59.338.1M6.8918. M i i o m i , ~ ~ r ~ m ) [ 9,40,517.3731,22869,lU233, 864435,5134427,30224375,175613909

I2,55,236,1271,6853,364l8, 192287.1012294.5346233.28215351 1

; ;I I

111674,586630,3087541.16251939~

[38,27O,l~O.l05M,6360l,387227,'
2314232.135472~,78566879,451980738]

28907,111733,428615,1648700) [86.326.1379,6350,27l94,114590.
492275,2078228,8657889,35986562]

4,8,45,193,604,2383,

[ 25.65,411.1991.6925,11304,
22003,84614,324859,1248828) [ 46,268,1066,4344,19992,87284,
368272,1554340,6487550.26934394]

(34,182,965,6294.3~61.230668,' 1370985,8068081,47123543,272477579] 1 1 0 I 7 I 11 11 41 202.1334 6982.

14

55367

63121

I1 I0

10

(2,6.37,153,582,2184, 8329,32111.124791.476203)
[13,62.334.1606,7321,30635, 131195,555564,2366161,9790577]

3,19.95,529,2800,14824, 77841,413026,2182612,Il529739)
[28,159,1186,7461,46689.282212, 1659671,9787540,56862563,327592538]

15

111653

145665

II I0

10

(3,0,46,0,683.0. 10135.0.151488.0) (28.0.397.0.7735.0, 150284.0,2734434,0]

(8,45,145,567,2182,8242,

16

347241

246277

I 1 I 0

12

31574,122351,470446,1808049) [ 68,495,1569,7112,31556.131760,
554286,2349061,9797200,405872l7]

[5,49,311,2266.12976.78'198, 464998,2729868,15946528.92079794]

17

506477

673711

I 0
11

12

(2,24,79,320.1251.4784, 17972,69040,267659,1029502) [l I ,253,889,3978,18056.76298,

316866,1326374,5565438,23081423]

18

1352755

1771563

11 10

12

(2.11,27,137,514.1940, 7486,28593,110382,425046)
[18,105.276,1679,7028,29848,

I
111

I 1552898,9028366,52204067,299949760]
IO

[ 5,142,1375,6842,44160,265938,
(6,51,217,1014,5622,29034,

126696,529543,2222988,9245 160) 19 2451321 3546713

11 10
12

(1,3,14,71.249.995,3841,

14926,57232,21922'11

[ 9,21,139,715.2940,13629,
59239,254614,1068299.4455654]

[40.240,1219,6934,472l3,283532, l645614,9598742,55146996,315652I79]

19

2142513

3276171
[99,425.1425,6158.25037,105225,

443232,1836907,7578943,312l9806
100

70561,374042,1975961,10442453) [48.202,1248,7445,4698l,279184. 1640495,9587002,55315528,316950355

[40.249.1510,8120,53164]

[11,5,231,736,3096]

[143,266,2038,12128,73739]

[17,163,927.3614,14459]
I I

[79.1275,5279,32916,200731]

IO0
[39,170,852 ]
111

13

(2l,I4l,707) [292.2340,13196 ]

limited to the rate-213 and 314 codes. This is due to the fact that with very few exceptions, optimal long codes for sequential decoding are known for rates 213 and 314 only. Tables VI1 and VI11 list the characteristics of the best punctured codes of rate 213 and 314, respectively, with memory lengths varying from 9 to 23, that are derived from the best nonsystematic rate-112 codes. From M = 9 to M = 13 the original codes are the maximal-free-distance codes discovered by Larsen [37], whereas for 14 5 M 5 23 the original codes are those of Johannesson and Paaske [25]. In both tables, for each memory length the generators of the original code and its perforation matrix are given, together with the free distance of the resulting punctured code. Just as with short-memory codes, the first few terms a, and c,,, n = dbe, dbe + 1, df,, + 2, . of the series expansions of the weight spectra are also given for each punctured code. In deriving these spectral coefficients, the exploration of the original code had to be performed over a considerable length. In the search for the best punctured codes, the perforation patterns were chosen as to yield both a maximal free distance

and a good distance profile. Although all perforation patterns were exhaustively examined, the search was somewhat reduced by exploiting equivalences of the perforation patterns under cyclical shifts of their rows [21]. Among all the codes that were found, Tables VI1 and VILI list only those having the smallest number of bit errors cdfie at the free distance dh, and obviously, all the codes listed are noncatastrophic. Table VII lists two M = 19, R = 2/3 punctured codes derived from two distinct good M = 19, R = 1/2 original codes. The free distances of these two punctured codes are equal to 12 and 13, but an examination of the spectral coefficients shows that the coefficients c, are far larger for the d , = 13 code than they are for the dfre = 12 code. f Consequently, as confirmed by the calculation of the bit error bound PB,the code with dfre= 13 turned out to be worse than the code with df, = 12, by approximately 0.35 dB. This anomaly confirms further the need to determine at least the first few terms of the weight spectra when searching for the best codes. Fig. 5 plots the free distances of the original rate 112 codes

1120
BIT ERROR PERFORMANCE BOUNDS UNQUANTIZED CHANNEL M=0 CONVOLUTIONALCODES

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO.

BIT ERROR PERFORMANCE BOUNDS
UNOUANTlZED CHANNEL RATE = 2/3 CONVOLUTIONAL CODES

11, NOVEMBER 1989

PE

EblNO ( dE )

EblNO ( dB )

Fig. 4. Bit error performance bounds for the M = 6, R = 1/2 original and punctured rates 213, 314, 415, 516, 617, and 718 codes.

Fig. 6. Upper bounds on the bit error probability over unquantkd channel for R = 2/3 punctured codes with 2 5 M s 22, M even.

FREE DISTANCE OF PUNCTURED CODES

Original

18

f
0 -

15 12 9

3
2
4 6

10 12 14 16 18 2 0 22

M Fig. 5. Free distancesof original rate-112codes and punctured R = 2/3 and 314 codes derived f o them as a function of M , 2 I M I22, and upper rm bounds.

and the punctured rate-2/3 and 314 codes derived from them as a function of the memory length. Except for the two anomalies mentioned above, the behavior of the free distances is as expected: the free distance of the punctured codes of a given rate is generally nondecreasing with the memory, and for a given memory length the free distance decreases with increasing code rates. When the punctured codes of rate b / u are determined from the best original low-rate l / v code, an upper bound on the free distance of the punctured code can be derived [21]. This derivation, which is based on an analysis of the effect of the different perforation patterns on the spectrum of the original code, yields the bound where d m ) and d&(o) are the free distances of the punctured and best codes of rate l l v , respectively. This bound is also plotted on Fig. 5 for the rate-213 and 3/4 codes with memory M I 13. The upper bounds on the bit error probability have been evaluated for all the punctured codes of rates R = 2/3 and 3/4 and are shown for even values of memory in Figs. 6 and 7. This evaluation has been performed according to (3) for q u a n t i z e d Gaussian channels. These bounds indicate a normal behavior for all the punctured codes listed in Tables V I and VIII. The bit error performances all improve as the I coding rate decreases and/or as the memory increases, with approximately 0.4 dB improvement for each unit increase of

the memory length. At Pe = the M = 22, R = 2/3, and R = 3/4 punctured codes can yield impressive coding gains of 8.3 dB and 7.7 dB, respectively. The selection of the best punctured codes listed in Tables V I and VIII has been based on both the maximal free distance I and the calculated bit error probability bound. However, the choice of the best punctured code was often not a very clearcut one. It was often observed that different puncturing patterns yielded only marginally different error performances [36]. In some cases the performance curves were almost undistinguishable, and, hence, several best punctured codes having the same rate and memory may be obtained. However, since these codes are for sequential decoding applications, the final selection must also be based on the distance profile and computational performance. Short of analyzing the computational behavior, when in doubt, the h codes finally selected and listed in the Tables had t e fastest growing-column distance function. In the search for punctured codes, the above approach produces good but not optimal codes since the original lowrate code is imposed at the outset. A measure of the discrepancy between optimal and punctured codes of the same rate and memory is provided in Fig. 8, which shows the bit error performance bound curves of both the memory-9 punctured and maximal free distance codes of rates 213 and 3/4. These bounds have been computed using only the term at d h for the MFD codes and using both terms at d h and d h +1 for the punctured codes. Based on these terms only, the two MFD codes appear to be only slightly better than the punctured codes. Therefore, it may be concluded that although not optimal, the error performances of the rate-2/3 and 3/4 punctured codes of memory 9 closely match those of the MFD codes of the same rates and memory lengths. The same general conclusions may be made for the other punctured codes with different memory lengths [36]. Given an optimal usual high-rate code of rate R = b/v and memory length M, one could attempt to determine the lowrate l/v code which, after puncturing will yield a punctured code that is equivalent to that optimal code. This approach, which is the converse of the usual code searching method, can be used to find the punctured code equivalent to any known usual high-rate code. Based on this approach and using the notion of orthogonal perforation patterns, a systematic construction technique has been developed by the authors [21].

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING

BIT ERROR PERFORMANCE BOUNDS UNQUANTIZED CHANNEL
RATE
i

1121
BIT ERROR PERFORMANCE BOUNDS
UNQUANTIZED CHANNEL
M= 9

314

CONVOLUTIONAL CODES

PB

USUAL AND PUNCTURED CODES

PB

1oaL

5
EblNO ( dB 1

1 Fig. 7. Upper bounds on the bit error probability over unquantized channel for R = 3/4 punctured codes with 2 5 M 5 22, Meven.
EblNO ( d B

Fig. 8. Bit error performance bounds for maximal free distance (MFD) codes and punctured codes of rates 213 and 314 with memory M = 9.

Using this technique punctured codes equivalent to the best known nonsystematic rate-2/3 codes with memory lengths up to M = 24 have been found. Likewise, punctured codes equivalent to the best known rate-3/4 codes with memory lengths up to M = 9 have been tabulated [21]. Therefore, optimal high-rate codes may be obtained as punctured codes, but it should be pointed out that the punctured codes generated by this approach are not suitable for variable-rate applications since each punctured code has its own distinct low-rate original code. V. COMPUTER SIMULATION RESULTS The stack algorithm for sequential decoding of punctured codes has been simulated on a computer. In these simulations the results of interest concern both the computational effort and the bit error performance. For R/R,,,, values ranging between 0.85 and 0.99, meaningful error events could be observed with shortmemory-length codes only. In these cases the resulting error probabilities were always found to satisfy the upper bound computed from the first terms of the code spectra. As the memory lengths of the codes increase, no errors were collected within a reasonable simulation time. Therefore, as usually expected with sequential decoding of long-memory codes, the simulation results will concern mainly the computational effort. Fig. 9 shows a typical distribution of computation obtained for the memory-23 rate-2/3 punctured code equivalent to the best known usual code of the same memory and rate discovered by Johannesson and Paaske [25]. For comparison purposes, the computational distribution for the usual code is also plotted. Both punctured and usual codes were simulated over identical additive white Gaussian noise binary symmetric channels with &,/No= 5.76 dB corresponding to R/R,,,, = 0.94. In either case, no decoding errors were observed. The distributions of the computational efforts appear to be very nearly identical for both the punctured and usual codes. However, an examination of the decoding parameters demonstrate the advantages of using the punctured approach for the decoding of high-rate codes. The average number of stack entries, SAV, is equal to 1057 for the usual code, whereas it is only 656 for the punctured code. Counting a computation as the extension of the top node of the stack into all its successors, then clearly the decoding of punctured codes requires on the average more computations than the decoding of usual high-

rate codes, but these computations are far simpler. For our example, as indicated in Fig. 9 the average number of computations CAV is equal to 1.32 and 1.12 for the punctured and usual codes, respectively. A fairer comparison of the average computational effort using punctured and usual codes may be based on the overall simulation times, since using both the punctured and usual codes the decoding algorithms are identical with the exception of the branch extensions and metric inhibition of the punctured symbols. For example, for rate-2/3 codes, the decoding time of the punctured codes was observed to be approximately 16 percent of that required for the usual codes, whereas for rate-7/8 codes this proportion of decoding time drops to 5 percent. Based on extensive simulation results involving a large number of codes having different coding rates over a wide range of Eb/No values, it was observed that these decoding advantages of the punctured codes over the normal codes increase as both the coding rates and Eb/Noincrease [38]. VI. CONCLUSIONS In this paper, we have presented the encoding as well as Viterbi and sequential decoding of high-rate punctured convolutional codes. These codes are derived from well-known optimal low-rate convolutional codes, and depending on the choice of the perforation patterns they may yield easy implementations of variable-rate and rate-compatible coding. Using a computer search, we have extended results on previously discovered short-memory-length punctured codes by providing up to ten terms of their weight spectra. We have shown that sequential decoding can be easily adapted for the decoding of punctured codes and have provided new long-memory punctured codes with 9 5 M I23 and rates 2/3 and 314 that have been derived from the best known rate-1/2 codes of the same memory length. The first few terms of these code spectra have been provided together with their bit error probability upper bounds. The substantial advantages of using high-rate punctured codes over the usual high-rate codes open the way for powerful, yet practical, implementations of variable-rate codecs, extending from very low to very high coding rates. APPENDIX In this Appendix, the weight spectra for two of Yasudas et al. punctured codes are derived using the transfer function

1122

IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 37, NO. 1 1 , NOVEMBER 1989

over the input, yielding

T(D,B)=Xb/X,=G[Z-A]-F.

(A.7)

Inverting matrix [I - A] this equation yields the transfer function of the state diagram in closed form. For the purpose of comparing these results with those of the exploration technique, the expansion of T(D, B) as a power series in D is required. This is readily obtained since [I - A ] - may be expanded as an infinite series of power of matrices

[I-A]

= I + [ A ]+ [ A 2 ] [ A 3 ] [ A 4 ] * + + +

*.

(A.8)

Substituting in (A.7), we get

T(D,B)=GF+ G [ A ] F + G [ A Z ] F + G [ A 3 ] F + *

-(A.9)

loo

10

lo2

lo3

N Fig. 9. Empirical distribution of the number of computations per decoded branch for equivalent usual and punctured codes having R = 2/3 and M = 24 over a hard-quantized channel at EbNo = 5.76 dB.

which is the desired series expansion of T(D, B). One can easily obtain the terms U, and cj of the weight spectrum from T(D, B) as

technique [11, [32]. These results provide a validation for all the weight spectra that were obtained by the search technique. Punctured codes are considered high-rate codes in the true sense and hence their performance is evaluated assuming an optimal (i.e., Viterbi) decoder that makes comparisons only on true trellis nodes of the high-rate code, that is, nodes whose depths are a multiple of b for a rate-b/v code. The transfer function of a convolutional code may be evaluated by solving equations describing the transitions between the states of the finite-state encoder. In Fig. 10 the state diagram of the rate 213, memory-2 punctured code of Table I is given. Following the usual procedure, the self-loop at state zero is eliminated by splitting that state into two states X , and x b . The remaining states are arbitrarily labeled X I , X2, and X,. The state transitions are labeled with the indeterminate monomials that describe them: the exponent of B indicating the number of information bits 1 causing the transition and the exponent of D indicating the Hamming weight on the transition. From Fig. 10, the transition behavior within the state diagram may be described by the matrix equation

T ( D ,B ) J B = I =

j = d k

UjDj

(A.10)

dTg

)IB=I

=j = d f r e e cjDJ.

(A.11)

For the rate-213 code of memory M = 2 described by (A. l), the expansion (A.9) yields

T(D, B ) =D 3 B +D4(3B2 B4)

+ +Ds(B+7 B 3+ 5Bs + B7) +D6(B2 14B4+ 17B6+ 7B8+Bl0) + +D7(2B3+ 26B5+ 47B7+ 31 B9+ 9B11+ B13) +D8(Bz+ 6B4+ 48B6+ 1 15B8+ 108BO + 49BIZ+ 1 1B14+ B16)
+D9(3B3+ 19B5+93B7+262B9

+ 324B11+205B13+71 B15+ 13B17+B19)+ - -.

*

B2D2 (A.1 )
or in matrix notation,

(A.12)
From (A.12) the spectral coefficients aiare given by

T ( D , B)(B=l=

ajDj

x= [ A ] X + F X ,
from which it follows that

(A3 (A.3)

J=d&

= D 3 + 4 D 4 +14D5+40O6

x = [ I - A ] -FX,

+ 116D7+ 339D8+ 991D 9 + and

(A.13)

where the meaning of the symbols [A], I, and F is obvious from (A.1). W e also have

[xb]=[D2 2 D2] D
or,
x =G x . b

[$1

(A.4)

=D 3

+ l o o 4 + 540 + 226D6
. e - .

+856D7+3072D8+10647D9+ (A.5)

(A.14)

Comparing to the results of Table I, we see that the spectral Finally, combining (A.3) through (AS), we get the scalar coefficients obtained via the transfer function are identical to equation those obtained by our search algorithms. In order to verify our results for a still higher rate code, the X b = G [ I - A ] -FX,. (A.6) same calculation has been applied to the memory-2, r a t e 4 5 The transfer function is given by the ratio of the output punctured code of Table III. The state diagram of the code is

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIALDECODING

1123

BO

D2

Fig. 10. State diagram of the rate-2/3, memory-2 punctured code of Table I

Fig. 11. State diagram of the r a t e 4 5 , memory-2 punctured code of Table 111. In this diagram, each state transition pertains to all branch transitions between the encoder states

depicted in Fig. 11 and its matrix equation is given by

-

BD3+ B2D + B3D3+ B2D3 BO3+ B2D + B 3 D 3 B2D3 +

BD3+ B2D BD3+ 2B2D3 + B3D3+ B2D3 + B 3 D BD3+ B2D BD3+ B 2 D + B3D3+ B2D3 + B2D5+R 3 D

B2D4+2B3D2 B4+B3D2 B2D4 2B3D2 + + B4D2 +B3D4+B2D4 +B4D2

X=[A]X+FX,

-j
" 3

(A. 15)

[Xb] [2BD2+ B2D4 D 2 D 2+ BD4+ B2D2 BD2 2BD2+ B2D4+ D 2 ] X 2 + ( D 2 B + D 4 ( B + B 2 ) ) X , (A.17) = + +

"'I
x 3

Xb = GX + hX,.

(A. 16)

(A.18)

1124 As above, the transfer function is given by T ( D , B ) = - = G [ Z - A ] -IF+ h Xl L

and hence

IEEE TRANSACTIONSON COMMUNICATIONS,VOL. 37, NO. 11, NOVEMBER 1989 REFERENCES

x b

( A .19)

T ( D , B ) = h + GF+ GAF+ GA2F+CA 3F+

--

*.

(A.20)

Following the same approach as above, (A.20) for this code yields

T(D, B ) = D 2 B +D3(2B+4B2+ 3B3+ B5+ 2B6)

+ D 4 ( B + 2 B 2 + 6 B 3 1 1 B4+ 10B5+2B6 + 4 B 7 + 9B8+5B9+BI1+2BlZ)

+ D5(3B2+5B3+ 1 1 B4+ 19Bs+ 30B6 + 33B7+ 18B8+20B9+33B10+29B11 +7BI2+ 6BI3+ 14B14+7BI5+B17 + 2 B 9 +D6(4B2+13 B3+ 23B4+25B5+46B6+ 72B7 + 97B8+ 101 B9+ 88B*O+938 + 122BI2 + 124BI3+ 63BI4+43B + 67BI6+57BI7 + 14B18+8B19+ 19Bm+9BZ1 + BZ3 2B) + D7(B2+ 8B3+ 26B4 +
+ 6 8 B 5 + 105B6+ 117B7+ 173B8+274B9

+ 357B1+ 352BI1+ 366BI2+409B13 + 477B14+ 477B15+ 350B16+ 268BL7 + 309BI8+ 305BI9+ 150Bm+77B21
+ 113BU+94B23+23B24+ lOBZS
*... +24BZ6+ 11B27+B29+2B30)+
(A.21)
The spectra coefficients uj and cj are therefore given by

T ( D , B ) J B = ~ = UjDJ
j=d*

= D 2 + 12D3+53D4+238D5

+ 1091D6+4947O7+ m

* *

(A.22)

= D 2 36D3 309D4 2060D5

*
e.

+ + + 12320D6+ 69343D7+ +

(A.23)

Once again, the spectral coefficients determined by the transfer function approach match perfectly those obtained by the tree searching algorithms, thus validating all the results provided in Tables I through VII.
ACKNOWLEDGMENT The authors wish to thank C. Paquin, a graduate student at Ecole Polytechnique de MontrM, for her help in running the computer programs for the search of punctured codes.

A. J. Viterbi, Convolutional codes and their performance in communications systems, IEEE Trans. Commun. Technol., vol. COM-19, Oct. 1Wl. R. M. Fano, A Heuristic discussion of probabilistic decoding, IEEE Trans. Inform. Theory, vol. IT-9, Apr. 1962. I. M. Jacobs and E. R. Berlekamp, A lower bound to the distribution of computation for sequential decoding, IEEE Trans. Inform. Theory, vol. IT-13, pp. 167-174, Apr. 1967. I. M. Jacobs, Practical applications of coding, IEEE Trans. Inform. Theory, vol. IT-20, pp. 305-310, May 1974. E. R. Berlekamp, The technology of error-correcting codes, Proc. IEEE, vol. 68, pp. 564-593, May 1980. V. K. Bhargava, D. Haccoun, R. Matyas, and P. Nuspl, Digital Communications by Satellite. New York: Wiley, 1981. S. Lin and D. J. Costello, Jr., Error Control Coding. Englewood Cliffs, NJ: Prentice-Hall, 1983. W. W.Wu,Elements of Digital Satellite Communications, Volume II. Rockville, MD: Computer Science, 1985. A. Michelson and A. Levesque, Error-Control Techniques for Digital Communication. New York: Wiley, 1985. W. W. Wu, D. Haccoun, R.Peile, and Y.Hirata, Coding for satellite communication, IEEE J. Select. Areas. Commun., vol. SAC-5, pp. 724-748, May 1987. A. S. Acampora and R. Gilmore, Analog Viterbi decoding for high speed d g t l satellite channels, NTC 77 Conf. Rec., Los Angeles, iia CA, Dec. 1977, pp. 34.6.1-34.6.5. R. T. Clark and R. D. McCallister, Development of an LSI maximum-likelihood convolutional decoder for advanced forward error correction capability on the NASA 30120 GHz program, in Proc. AIAA 9th Commun. Satellite Syst. Conf., San Diego, CA, Mar. 1982, pp. 142-144. R. M. Orndoff et al., Viterbi decoder VLSI integrated circuit for bit e m r correction, in Proc. Nut. Telecommun. Conf., New Orleans, LA, Dec. 1981, pp. E1.7.1-E1.7.4. J. B. Cain, G. C. Clark, and J. Geist, Punctured convolutional codes of rate (n - l)/n and simplified maximum likelihood decoding, IEEE Trans. Inform. Theory, vol. IT-25, pp. 97-100, Jan. 1979. Y. Yasuda,Y. Hirata, K. Nakamura, and S. Otani, Development of a variable-rate Viterbi decoder and its performance characteristics, 6th Int. Conf. Digital Satellite Commun., Phoenix, AZ, Sept. 1983. Y. Yasuda, K. Kashiki, and Y. Hirata, High-rate punctured convoluot tional codes for s f decision Viterbi decoding, IEEE Trans. Commun., vol. COM-32, pp. 315-319, Mar. 1984. G. Begin and D. Haccoun, Sequential decoding of punctured convolutional codes, in Proc. 13th Biennial Symp. Commun., Kingston, Ont., Canada, June 1986, pp. A.3.5-A.3.8. , Decoding of punctured convolutional codes by the stack algorithm, Abstracts of Papers, 1986 IEEE Int. Symp. Inform. Theory, Ann Arbor, MI,Oct. 1986, p. 159. J. Hagenauer, Rate compatible punctured convolutional codes and their applications, IEEE Trans. Commun., vol. 36, pp. 389-400, Apr. 1988. D. Haccoun and G. Begin, Codage et dkodage skquentiel de codes convolutionnels perforbs, in Proc. 11th GRETSI Conf., Nice, France, June 1987, pp. 221-224. G. Begin and D. Haccoun, High rate punctured convolutional codes: Structure properties and construction techniques, IEEE Trans. Commun., to appear Dec. 1989. D. Haccoun and C. Naiyun, Variants of the stack algorithm for the decoding of high-rate codes by sequential decoding, in Proc. Int. Satellite Commun. Conf., Ottawa, Canada, June 1983, pp. 21.4.121.4.4. D. Haccoun, Dkodage dquentiel des codes convolutionels de taux ClevCs, Traitement du Signal, Paris, France, vol. 4, no. 6, pp. 471478, Dk. 1987. re E. Paaske, Short binary convolutional codes with maximal f e distance for rates 213 and 314, IEEE Trans. Inform. Theory, vol. IT-20, pp. 683-686, Sept. 1974. R. Johannesson and E. Paaske, Further results on binary convolutional codes with an optimum distance profile, IEEE Trans. Inform. Theory, vol. IT-24, pp. 264-268, Mar. 1978. J. Hagenauer, High rate convolutional codes with good profiles, IEEE Trans. Inform. Theory, vol. IT-23, pp. 615-618, Sept. 1977. F. Jelinek, A fast sequential decoding algorithm using a stack, IBA4 J. Res. Develop., vol. 13, pp. 675-685, Nov. 1969. K. Zigangirov, Some sequential decoding procedures, Problemii Peredachi Informatsii, vol. 2, pp. 13-15, 1966.

HACCOUN AND BEGIN: CODES FOR VITERBI AND SEQUENTIAL DECODING

1125
and is currently Head of the Communication Department. From June 1984 to May 1985 he was a Research Visiting Professor at Concordia University, Montrkal, P.Q., Canada. His current research interests include theory and applications of error-control coding, VLSI implementation of powerful codecs, mobile communications, and digital communications systems by satellite. Dr. Haccoun is member of the Order of Engineers of Quebec and Sigma Xi. He was Treasurer of the 1983 IEEE International Symposium on Information Theory, St. Jovite, P.Q., Canada, September 1983, the Chairman of the Communications Chapter (1976-1978) and the founder of the Vehicular Communications Chapter (1984-1985) in the Montreal section of the IEEE, and a cofounder and President of the Canadian Society of Information Theory (1986-1987). He has been serving as a consultant to several government agencies and industries. He is the author or coauthor of several papers in these areas and the coauthor of the book Digital Communications by Satellite: Modulation, Multiple Access and Coding (New York: Wiley, 1981). A Japanese translation of the book was published in 1984.

D. Haccoun and M. J. Ferguson, Generalized stack algorithms for decoding convolutional codes, IEEE Trans. Inform. Theory, vol. IT-21, pp. 638-651, NOV. 1975. D. Haccoun, Variabilite de calcul et dkbordements de decodeurs sequentiels a pile, Traitement du signal, Paris, France, vol. 3, no. 3, pp, 127-143, Dec. 1986. R. Johannesson, Some long rate one-half binary convolutional codes with an optimal distance profile, IEEE Trans. Inform. Theory, vol. IT-22, pp. 629-631, Sept. 1976. J. P. Odenwalder, Optimal decoding of convolutional codes, Ph.D. dissertation, Dept. Elect. Eng., U.C.L.A., Los Angeles, 1970. P. Montreuil, Algorithmes de determination de spectres des codes convolutionnels, M.Sc.A. thesis, Dep. Elect. Eng., Ecole Polytechnique de Montreal, 1987. D. Haccoun and P. Montreuil, Weight spectrum determination of convolutional codes, to be submitted to IEEE Trans. Commun. , Algorithms for the spectrum determination of convolutional codes, Book of Abstracts, 1988 Int. Symp. Inform. Theory, Kobe, Japan, June 1988, pp. 49-50. D. Haccoun and G. Begin, High rate punctured convolutional codes, Rapport Technique EPMIRT-88/1, Ecole Polytechnique de Montreal, Jan. 1988, 63 pages. K. Larsen, Short convolutional codes with maximal free distance for rates 1/2, 113, and 1/4, IEEE Trans. Inform. Theory, vol. IT-19, pp. 371-72, May 1973. G. Begin and D. Haccoun, Simulation results of sequential decoding of high rate punctured convolutional codes, to be submitted to IEEE

Trans. Commun.

.k
David Haccoun (S62-M67-S69-M74-SM84) received the Engineer and B.A.Sc. degrees in engineering physics from h o l e Polytechnique de Montreal, P.Q., Canada, in 1965, the S.M. degree in electrical engineering from the Massachusetts Institute of Technology, Cambridge, in 1966, and the Ph.D. degree in electrical engineering from McGill University, MontrCal, P.Q., Canada, in 1974 Since 1966 he has been with the Department of Electrical Engineering, Ecole Polytechnique de Montreal, where he is Professor of Electrical Engineering since 1980. He was Head of the Department of Communication and Computer from 1980 to 1982, Guy Begin (S86) was born in Jonquieres, P.Q , Canada, in 1960. He received the B.Eng. degree in unified engineering from Universite du Quebec a Chicoutim in 1983. He is currently working towards the Ph.D. degree in electrical engineering at Ecole Polytechnique de Montreal, P.Q., Canada. Since 1985, he has been a recipient of a postgraduate scholarship from Fonds FCAR of Qutbec. His research interests include theory and applications of error-control coding, VLSI implementations of codecs and cryptology.