Unit 4: Channel Coding By

Dr. Shashibhushan Sharma

Assistant Professor
JK Institute of Applied Physics and
Technology,
University of Allahabad,
Prayagraj
Content
• Convolutional Codes-
• Convolutional encoding; Representation of convolutional Encodes;
The state diagram; The tree diagram; The trellis diagram;
• Optimum decoding of the Convolutional codes- The Viterbi algorithm,
Sequential decoding; Comparison and Limitations of Viterbi and
Sequential decoding;
• Distance properties of convolutional codes; Error correcting
capability of convolutional codes.
Convolutional Encoding
Brief introduction:
• The Linear Block Codes are described by two integer n and k, and generator matrix or
polynomial. The integer k is the number of data bits that form an input to a block encoder.
• The integer n is the total number of bits in the associated codeword out of the encoder.
• A characteristics of liner block codes is that each codeword n- tuples is uniquely determined
by the input message k-tuple. The ratio k/n is called the rate of the code- a measure of the
amount of added redundancy.
• A convolutional code is described by three integers, n, k, and K, where the ratio k/n has the
same code rate significance (information per coded bit) that it has for the block codes;
however n does not define a block or codeword length as it does for block codes.
• The integer K is the parameter known as the constraint length; it represents the number of k-
tuples stages in the encoding shift register.
• An important characteristics of convolutional codes, different from block codes, is that the
encoder has memory- the n-tuple emitted by the convolutional encoding procedure is not only
a function of an input k-tuple, but is also a function of the previous K - 1 input k-tuples.
• In practice, n and k are small integers and K is varied to control the capability and complexity
of the code.
 Convolutional Encoding
• Figure 1 represents a typical block diagram of a digital communication system.

Figure 1. Block diagram of a typical digital

communication system.
Convolutional Encoding Contd…
• A version of this functional diagram, focusing primarily on the convolutional encode/decode
and modulate/demodulate portions of communication link is shown in Figure 2.
• The input message source is denoted by the sequence m= m1 , m2 ,..., mi ,..., mk , where each mi
represents a binary digit (bit) one or zero and is equally likely. The binary digits are
independent to each other.
• Being independent means, lacks of
redundancy in the bit sequence exist. That
is knowledge about mi gives no
information about m j  i  j  .
• The encoder transform the each sequence
m into a unique codeword sequence
U=G(m).

Figure 2. Encode/decode and modulate/demodulate

portions of a communication link
Convolutional Encoding Contd…
• The sequence m uniquely defines the sequence U, since the encoding of each k-tuple is not
only a function of that k-tuple but is also a function of the K-1 input k-tuples that precede it.
• The sequence U can be partitioned into a sequence of branch words: U= U1 ,U 2 ,...,U i ,...,U n .
Each branch word U i is made up of binary code symbols, often called channel symbols,
channel bits or code bits.
• Unlike the input message bits, the code symbols are not independent.
• In a typical communication application, the codeword sequence U, modulates a waveform
s(t).
• During transmission, the waveform s(t) is corrupted by noise, resulting in received
waveform s  t  and a demodulated sequence Z= Z1 , Z 2 ,..., Z i ,..., Z n as indicated in Figure 2.
• The task of decoder is to produce an estimate m  m1 , m 2 ,..., mi ,..., m n of the original message
sequence, using the received sequence Z together with a priori knowledge of the encoding
procedure.
 Convolutional Encoding Contd…
• A general convolutional encoder, shown in Figure 3, is mechanized with a kK-stage shift
register and n modulo-2 adders, where K is the constraint length.

The constraint length K

• At each unit of time, k bits are shifted represents the number of k-bit
into the first k stages of the register; shifts over which a single
all bits in the register are shifted k information bit can influence
stages to the right, and the outputs of the encoder output.
the n adders are sequentially sampled
to yield the binary code symbols or
code bits.
• These code symbols are used by the
modulator to specify the waveforms
to be transmitted over the channel.
• Since there are n code bits for each
input group of k message bits, the
code rate is k/n message bit per code Figure 3. Convolutional encoder with constraint length K and rate k/n.
bit, where k<n.
Convolutional Encoding Contd…
• We shall consider only most commonly used binary convolutional encoder for which k=1,
that is, those encoders in which the message bits are shifted into the encoder one bit at a
time.
• For the k=1 encoder, at the ith unit of time, message bit mi shifted into the first shift
register stage; all previous bits in the register are shifted one stage to the right, and as in
the more general case, the output of the n adders are sequentially sampled and
transmitted.
• Since there are n code bits for each message bit, the code rate is 1/n. The n code symbols
occurring at time ti comprise the i th branch word.
• Note for the 1/n encoder, the kK-stage shift register can be referred to simply as a K-stage
shift register, and the constraint length K, which was expressed in unit of k-tuple stages,
can be referred to as constraint length in units of bits.
Convolutional Encoder Representation

• To describe a convolutional code, one need to characterize the encoding function G(m),
so that given an input sequence m, one can readily compute the output sequence U.
Several methods are used for representing a convolutional encoder. The most popular
being the Connection pictorial, Connection Vectors or polynomials, the state diagram, the
tree diagram, and the trellis diagram.
The state diagram
• The simple encoder is shown in Figure 4 and its
state diagram is shown in Figure 5.
• The states, shown in the boxes of the diagram,
possible content of the rightmost K-1 stages of the
register. The paths between the states represents
the output branch words resulting from such state
transitions.
• The states of the register are designated a=00,
b=10, c=01, and d=11.

Figure 4. Convolutional encoder Figure 5. Encoder state diagram with rate ½, K=3
with rate ½, K=3
The state diagram
• The state diagram shown in Figure 5 illustrates all the
state transitions that are possible for the encoder in
Figure 4.
• There are only two transitions emanating from each state
corresponding to the two possible input bits.
• Each path between state is written the output branch
word associated with the state transition.
• In drawing the path we use the convention that a solid
line denotes a path associated with an input bit, zero, and
a dashed line denotes a path associated with an input bit,
one.
• Notice that it is not possible to move any other state in a
single transition. Shifting in one bit at a time, there are
only two possible state transitions that the register can
make at each bit time. For example, present encoder
state is 00, the only possibilities for the state at the next
shift are 00 or 10. Figure 5. Encoder state diagram with rate ½, K=3
The state diagram
Example 1. Convolution Encoding
For the encoder shown in Figure 4, show the
state changes and the resulting output
codeword sequence U for the message
sequence m = 11011, followed by K-1=2
zeros to flush the register. Assume that initial
contents of the register are all zeros.
Solution:

Figure 4. Convolutional encoder

with rate ½, K=3
The state diagram
Example 2. Convolutional Encoding
In Example 1, the initial contents of the
register are all zeros. This is equivalent
to the condition that the given input
sequence is preceded by two zero bits
(the encoding is a function of the
present bit and the K-1 prior bits).
Repeat Example 1, with the assumption
that the given input sequence is
preceded by two one bits, and verify
that now the codeword sequence U for
input sequence m=11011 is different
than the codeword found in Example 1.

By comparison with the result of Example 1, we can see that each branch word of the output
sequence U is not only a function of the input bit, but is also a function of the K-1 prior bits.
The tree diagram
• Although the state diagram completely characterizes the
encoder, one can not easily use it for tracing the encoder
transitions as a function of time since the diagram can
not represent time history.
• The tree diagram adds the dimension of time to the state
diagram. The tree diagram, for the convolutional encoder
shown in Figure 4, is illustrated in Figure 6.
• At each successive input bit time, the encoding
procedure can be described by traversing the diagram
from left to right, each tree branch describing an output
branch word.
• The branching rule for finding a codeword sequence is as
follows: if the input bit is a zero, its associated branch Figure 6. Tree
word is found by moving to the next rightmost branch in representation of
the upward direction. If the input bit is a one, its branch encoder (rate ½, K=3)
word is found by moving to the next rightmost branch in
the downward direction.
The tree diagram
• Assuming that the initial contents of the encoder is all zeros,
the diagram shows that if the first input bit is a zero, the output
branch word is 00 and, if the first input bit is a one, the output
branch word is 11.
• Similarly, if the first input bit is a one, and the second input bit
is a zero, the second output branch word is 10. or if the first
input bit is a one, and the second input bit is a one, the second
output branch word is 01.
• Following this procedure, we see that the input sequence 1 1 0
1 1 traces the heavy line drawn on the tree diagram in the
Figure 6. This path corresponds to the output codeword
sequence 1 1 0 1 0 1 0 0 0 1.
• The added dimension of time in the tree diagram (compared to
the state diagram) allow one to dynamically describe the
encoder as a function of a particular input sequence.
• It is very difficult to design this type of code on paper due to Figure 6. Tree
less space and also we would lose patience. If the L is number representation of
of branch word in the sequence, then the number of branches encoder (rate ½, K=3)
increases as a function of 2 L .
The trellis diagram
• We label each node in the tree of figure 6 to correspond
to four possible states in the shift register as follows:
a=00, b=10, c=01, and d=11.
• The first branching of the tree structure, at time t1 ,
produces a pair of nodes labeled a and b. At each
successive branching the number of nodes double. The
second branching, at time t2 , results in four nodes
labeled, a, b, c, and d. After the third branching, there
are a total of eight nodes.
• We can see that all branches emanating from two nodes
of the same state generate identical branch word
sequences. From this point on, the upper and lower
halves of the tree are identical. The reason for this
Figure 6. Tree
should be obvious from examination of the encoder in representation of
Figure 4. encoder (rate ½, K=3)
The trellis diagram
• As the fourth input bit enters the encoder on the left, the
first input bit is ejected on the right and no longer
influences the output branch word.
• Consequently, the input sequences 10 0 x y . . . and 0 0 0
x y . . . , where leftmost bit is the earliest bit, generate
the same branch words after the (K=3)rd branching. This
means that any two nodes having the same state label at
the same time ti can be merged, since all succeeding
paths will be indistinguishable.
• If we do this to the tree structure of Figure 6, we obtain
another diagram called the trellis diagram.
• The trellis diagram, by exploiting the repetitive
structure, provides a more manageable encoder Figure 6. Tree
description than does the tree diagram. The trellis representation of
diagram for the convolution encoder of Figure 6 is encoder (rate ½, K=3)
shown in Figure 7.
The trellis diagram
• In drawing the trellis diagram, we use the same convention that we introduced with the
state diagram-a solid line denotes the output generated by an input bit zero, and a dashed
line denotes the output generated by an input bit one.
• The nodes of the trellis characterize the encoder states; the first row nodes correspond to
the state a= 00, the second and subsequent rows correspond to the states, b=10, c=01, and
d=11.
• At each unit of time, the trellis requires 2 K 1 nodes to represent the 2 K 1 possible encoder
states.
• The trellis in our example assumes a fixed
periodic structure after trellis depth 3 is
reached (at time t4 ).
• In general case, the fixed structure prevails
after depth K is reached.
• At this point and there after, each of the states
can be entered from either of two preceding
states.
Figure 7. Encoder trellis
diagram (rate ½ , K=3)
The trellis diagram
• Also, each of the states can transition to one of the two states.
• Of the two outgoing branches, one correspond to input bit zero, and the other correspond
to an input bit one.
• The output branch words corresponding to the state transitions appear as labels on the
trellis branches.
• One time-interval section of a fully-formed encoding trellis structure completely defines
the code.
• The only reason for showing several sections
is for viewing a code symbol sequence as a
function of time.
• The state of convolutional encoder is
represented by the contents of the rightmost
K-1 stages in the encoder register.

Figure 7. Encoder trellis

diagram (rate ½ , K=3)
The trellis diagram
• A code symbol sequence is characterized by N branches (representing N data bits) occupying
N intervals of time and associated with a particular state at each of N+1 times (from start to
finish).
• Thus, we lunch bits at times t1 , t2 ,..., t N , and are interested in state metrics at times t1 , t2 ,..., t N 1 .

• We refer to this time as the start time and

label it t1 . We refer to the concluding time of
the last transition as the terminating time and
label it t N 1 .

Figure 7. Encoder trellis

diagram (rate ½ , K=3)
Viterbi algorithm
• Viterbi decoding algorithm was discovered and analyzed by Viterbi in 1967.
• The Viterbi algorithm essentially performs maximum likelihood decoding.
• It reduces the computational load by taking advantage of the special structure in the code
trellis.
• The advantage of Viterbi decoding, compared with brute-force decoding, is that the
complexity of a Viterbi decoder is not a function of the number of symbols in the
codeword sequence.
• The Viterbi algorithm removes those trellis paths that could not possibly be candidates for
the maximum likelihood choice.
• When two paths enter the same state, the one having the best metric is chosen; this path is
called the surviving path. In the selection of surviving path, the decoder takes decisions
by eliminating the least likely paths. The early rejection of unlikely paths reduces the
decoding complexity.
• In 1969, Omura demonstrated that the Viterbi algorithm is, in fact, maximum likelihood.
• The goal of selecting the optimum path is based on choosing the codeword with
maximum likelihood metric, or choosing the codeword with the minimum distance
metric.
Viterbi algorithm
• On the board.
Sequential Decoding
• The sequential decoding originally proposed by Wozencraft before the discovery of an optimum
algorithm by Viterbi.
• Consider the encoder shown in Figure 4. A message sequence m= 1 1 0 1 1 is encoded into the
codeword sequence U= 1 1 0 1 0 1 0 0 0 1 as shown in Example 1. Receiver receives the sequence Z.
• The decoder has available a replica of the encoder code tree shown in Figure 6 and can use the
received sequence Z to penetrate the tree.
• The decoder starts at the time t1 node of the tree and generates both paths leaving that node and
follows the path agreeing with the first group of n code symbols.

• If received sequence Z is correct version of U, then the

tree has been quickly penetrated.
• If the received sequence Z is a corrupted version of U. It
takes time to penetrated the tree.
• The decoder starts at time t1 node of the code tree and
generates both paths leading from that node as shown in
Figure 8 in next slide.

Figure 4. Convolutional encoder with rate ½, K=3

Sequential Decoding
• If there is not agreement, the decoder follows the most likely
path but keeps a cumulative count on the number of
disagreements between the received symbols and the branch
words on the path being followed.
• If two branches appear equally likely, the receiver uses an
arbitrary rule, such as following the zero input path.
• At each new level in the tree, the decoder generates new
branches and compares them with the next set of n received
code symbols.
• The search continues to penetrate the tree along the most
likely path and maintains the cumulative disagreement count.
• If the disagreement count exceeds a certain number (which
may increase as penetrate the tree), the decoder decides that
it is on an incorrect path, backs out of the path, and tries
another.
• The decoder keeps track of discarded pathways to avoid
repeating any path excursions.
Figure 8. Sequential decoding example
Sequential Decoding
• Let us consider that the encoder in Figure 4 is used to encode
the message sequence m=1 1 0 1 1 into codeword sequence
U as seen in example 1. Suppose that the fourth and seventh
bits of the transmitted sequence U are received in error, such
that:

• Now, we follow the decoder path trajectory in the Figure 8.

Assume that a cumulative path disagreement count of 3 is
the criterion for backing up and trying an alternate path.
• In Figure 8, the numbers along the path trajectory represent
the current disagreement count. Figure 8. Sequential decoding example
Comparison and Limitations of Viterbi and
Sequential decoding
• The major drawback of the Viterbi algorithm is that while error probability decreases
exponentially, with constraint length, the number of code states and consequently decoder
complexity grows exponentially with constraint length.
• Next, the computational complexity of the Viterbi algorithm is independent of channel
characteristics (compared to hard-decision decoding, soft-decision decoding requires
only a trivial increase in the number of computations).
• Sequential decoding achieves asymptotically the same error probability as maximum
likelihood decoding.
• In fact, with sequential decoding, the number of states searched is essentially independent
constraint length, due to this it is possible to use very large (K=41) constraint lengths.
This is an important factor in providing such low error probability.
• The major drawback of sequentially decoding is that the number of state metrics searched
is a random variable.
Comparison and Limitations of Viterbi and
Sequential decoding
• For sequential decoding, the expected number of poor hypotheses and backward searches
is a function of the channel SNR.
• With low SNR, more hypotheses must be tried than with a high SNR. Because of this
variability in computational load, buffers must be provided to store the arriving
sequences.
• Under low SNR, the received sequences must be buffered while the decoder is laboring to
find a likely hypothesis.
• If the average symbol arrival rate exceeds the average symbol decode rate, the buffer will
overflow, no matter how large it is, causing a loss of data. The sequential decoder
typically puts out error-free data until the buffer overflows, at which time the decoder has
to go through a recovery procedure.
• The buffer overflow threshold is a very sensitive function of SNR. Therefore, an
important part of a sequential decoder specification is the probability of buffer overflow.
Comparison and Limitations of Viterbi and
Sequential decoding
• The bit error probability ( Pb ) versus Eb N 0 curves for
these two popular solutions to the convolutional
decoding problem, Viterbi decoding and sequential
decoding illustrate their comparative performance
using coherent BPSK over an AWGN channel in
Figure 9.
• The curves compare Viterbi decoding (rates ½ and 1/3
hard decision, K=7) versus Viterbi decoding (rates ½
and 1/3 soft decision, K=7) versus sequential decoding
(rates ½ and 1/3 hard decision, K=41).
• One can see from Figure 9 that coding gains of
approximately 8 dB at Pb  106 can be achieved with
sequential decoders.

Figure 9. bit error performance for various Viterbi and sequential

decoding schemes using coherent BPSK over an AWGN channel.
Distance properties of convolutional codes
• In the linear block code, we are interested in the minimum distance between all pairs of
such codeword sequences in the code, since the minimum distance is related to the error-
correcting capability of the code.
• A convolutional code is a group or linear code, there is no loss in generality in simply
finding the minimum distance between each of the codeword sequences and the all-zeros
sequence. In other word, any test message is just as “good” as any other test message.
• Assume that the all-zeros input sequence was transmitted which follow the all zeros states.
• In this transmission, an error occurs whenever the all zeros path does not survive. An error
of interest is associated with a surviving path that diverges from and then remerges to the
all-zeros path.
• Diverges and remerges both are necessary, because, we can observe that error has been
occurred. If we see only the divergence, we can detect the error. At the point of occurance
of an error, decoder produce the garbage value for rest of the message duration.
• We want to quantify the decoder capability in terms of errors, that will usually take place.
Distance properties of convolutional codes
• The minimum distance for making such an error can be found by exhaustively examining
every path from the 00 state to the 00 state.
• We redraw the trellis diagram as shown in Figure 10, labeling each branch with its
hamming distance from the all zeros codeword instead of with its branch word symbols.
• Consider all the paths that diverge from the all zeros path and then remerge for the first
time at some arbitrary node.
• We can compute the distances of these path from the all-zeros path. There is one path at
distance 5 from all-zeros path; this path departs from the all-zeros path at time t1 and
merge with it at time t4.
• Similarly, there are two paths at distance 6, one which departs at time t1 and merge at
time t5 and the other which departs at time t1 and merge at time t6, and so on.
• We can also see from the dashed and solid lines of the diagram that the input bits for the
distance 5 path are 1 0 0; it differs in only one input bit from the all-zeros input sequence.
• Similarly the input bits for the distance 6 paths are 1 1 0 0 and 1 0 1 0 0; each differs in
two positions from the all-zeros path.
Distance properties of convolutional codes
• The minimum distance in the set of all arbitrarily long paths that diverge and remerge
called the minimum free distance or simply the free distance is seen to be 5 in this
example as shown with the heavy lines in figure 10.

Figure 10. Trellis diagram labeled with distances from the all zeros path.
Distance properties of convolutional codes
• We can find the number of error, which can be corrected by the code, is expressed as
 d f  1
t  1
 2 
where  x  is the largest integer but not greater than x, and d f is the minimum free
distance or free distance. Setting d f  5 , we see that code can correct any two channel
errors.
• Although, the Figure 10 presents the computation of free distance in a straight forward
way, a more direct closed form expression can be obtained by starting with state diagram.
• First we label the branches of state diagram as either D 0  1 , D1 or D 2 shown in Figure
11,where the exponent of D denotes the Hamming distance from the branch word of that
branch to the all-zeros branch. The self loop at node a can be eliminated since it
contributes nothing to the distance properties of a codeword sequence relative to the all-
zero sequence. Furthermore node a can be split into two nodes (labeled a and e), one of
which represent the input and the other the output of the state diagram.
• All paths originating at a=00 and terminating at e=00 can be traced on the modified state
diagram of Figure 11.
Distance properties of convolutional codes
• We can calculate the transfer function of the path a b c e in terms of indeterminate
“placeholder” D, as D 2 D D 2  D5 . The exponent of D represent the Hamming distance.
• Similarly, the path a b d c e and a b c b c e each have the transfer function D 6 and thus a
Hamming distance of 6 from the all-zeros path. We now write the state equations as
X b  D2 X a  X c
X c  DX b  DX d
2
X d  DX b  DX d
X e  D2 X c
where X a ,..., X e are the dummy
variables for the partial paths to the
intermediate nodes.
• The transfer function, T(D),
sometimes called the generating
function of the code can be expressed Figure 11. State diagram, labeled according to the distance from
T  D  Xe X a
the all-zeros path
as 3a
Distance properties of convolutional codes
• By solving state equations shown in Equation 2, we obtain
D5
T  D   D5  2 D 6  4 D 7  ...  2l D l 5  ... 3b
1  2D
• The transfer function for this code indicates that there is a single path of distance 5 from
the all-zeros path, two of distance 6, four of distance 7, and in general 2l paths of distance
l+5 from the all-zeros path, where l=0, 1, 2, ….
• The free distance d f of the code is the Hamming weight of the lowest-order term in the
expansion of T(D). In this example d f  5 . In evaluating distance properties, the transfer
function T(D), can not be used for long constraint lengths since the complexity of T(D)
increases exponentially with constraint length.
Distance properties of convolutional codes
• The transfer function can be used to provide more detailed information than just the
distance of the various paths.
• Let us introduce a factor L into each branch of the state diagram so that the exponent of L
can serve as a counter to indicate the number of branches in any given path from state a =
00 to state e=00. Furthermore we introduce a factor N into all branch transitions caused
by the input bit one. Thus, as each branch is traversed, the cumulative exponent on N
increases by one, only if that branch transition is due to an input bit one.
• For convolutional code characterized in Figure 4, the state diagram is modified. The
additional factor L and N are shown on the modified state diagram of Figure 12. The
Equation 2 can be modified as follows:
X b  D 2 LNX a  LNX c
X c  DLX b  DLX d
4
X d  DLNX b  DLNX d
X e  D 2 LX c Figure 12. State diagram, labeled
according to distance, length, and
number of input ones.
Distance properties of convolutional codes
• The transfer function of this augmented state diagram is
D5 L3 N
T  D , L, N    D 5 L3 N  D 6 L4 1  L  N 2  D 7 L5 1  L  N 3  ...  D l 5 Ll 3 N l 1  ...
2

1  DL 1  L  N 5
• Thus, we can verify some of the path properties displayed in Figure 11. There is one path
of distance 5, length 3, which differs in one input bit from the all-zeros path. There are
two path of distance 6, one of which is length 4, the other length 5 and both differ in two
input bits from the all-zeros path. Also of the distance 7 paths, one is of length 5, two are
of length 6, and one is of length 7, all four paths correspond to input sequences that differ
in three input bits from the all-zeros path.
Thus if the all-zeros path is the correct path the noise
causes us to choose one of the incorrect paths of
distance 7, three bit error will be made.

Figure 12. State diagram, labeled

according to distance, length, and
number of input ones.
Error-Correcting Capability of Convolutional
Codes
• In the study of block code, we have seen the t error can corrected from each block length
with maximum likelihood decoding.
• In the convolutional code, error correcting capability can not stated so succinctly.
However, from Equation 1, we can say that code can, with maximum likelihood decoding,
correct t errors within a few constraint lengths, where few here means 3 to 5. The exact
length depends on how the errors are distributed. The Equation 1 re-expressed as
 d f  1
t  6
 2 
• For a particular code and error pattern, the length can be bounded using transfer function
methods.
Class Test