DSC – Chapter 4 University of Trento

4. ERROR CONTROL CODING

◼ INTRODUCTION

◼ ARQ AND FEC SYSTEMS

◼ CODE TYPOLOGIES

◼ BLOCK CODES

◼ REPETITION CODE

◼ PARITY-CHECK CODE

◼ HAMMING DISTANCE

◼ ERROR DETECTION AND CORRECTION CAPABILITY

1
DSC – Chapter 4 University of Trento

4. ERROR CONTROL CODING

◼ BLOCK CODE PROPERTIES

◼ SYSTEMATIC CODES

◼ HAMMING CODE

◼ SYNDROME DECODING

◼ CYCLIC CODE

◼ CONVOLUTIONAL CODE

◼ VITERBI DECODING

◼ SEQUENTIAL DECODING

2
DSC – Chapter 4 University of Trento

INTRODUCTION

 A real communication system is subject to transmission errors

due to the presence of noise on the channel.

 The error probability Pbe on a bit sent on the channel is related

to several factors, including:
 average power of the received signal SR

 transmission bit rate rb

 power density spectrum of the noise

 type of channel (with additive noise, with multiplicative noise,

with time-variance characteristics, etc…)

3
DSC – Chapter 4 University of Trento

INTRODUCTION

 In the study of information theory, we have seen that, if the information

rate R is lower than the channel capacity, it is possible to make the error
probability on the information bits arbitrarily small by using appropriate
channel coding techniques.
 Channel coding consists of adding redundancy bits to the message that
one desires to transmit. During the reception phase, the presence of
these bits allows to detect or correct any errors introduced into the
message by the channel noise.
 The disadvantage of channel coding is that more bits than are actually
necessary to represent the message need to be transmitted, increasing
transmission time (or bandwidth).

4
DSC – Chapter 4 University of Trento

INTRODUCTION

◼ Actually, channel coding does not change the error probability

of the bits that “physically” travel on the channel.

◼ It reduces the error probability of the information bits (i.e., the

message bits).

◼ Below, we will see the main coding strategies and the main
codes that can be used to protect the message from the
noise present on the channel.

5
DSC – Chapter 4 University of Trento

INTRODUCTION

 There are two possible code categories for using error-

control codes:

 codes to detect errors (errors are recognized but not

corrected)

 codes to correct errors (errors are recognized and

corrected)

6
 DSC – Chapter 4 University of Trento

AUTOMATIC REPEAT AND

REQUEST (ARQ) SYSTEMS

 They use codes only to reveal the presence of errors in the

received message.

 When an error is detected, the receiver requests retransmission of

the message to the transmitter (a return channel is thus needed).

information bits information bits

Input buffer and Output buffer and

Encoder Tx Channel Rx Decoder
control system control system

information bits Return channel

ACK/NAK
+ redundancy bits

7
DSC – Chapter 4 University of Trento

FORWARD ERROR
CORRECTION (FEC) SYSTEMS

 They exploit codes both to reveal the presence of errors

and to correct them.

 They are particularly suitable in cases where it is not

possible to retransmit the message.

information bits Encoder Tx Channel Rx Decoder information bits

information bits + redundancy bits

8
DSC – Chapter 4 University of Trento

CODES: DEFINITIONS

 We define:
 detection capability of a code: maximum number of errors it
can detect in a codeword

 correction capability of a code: maximum number of errors it

can correct in a codeword

 The performance of a code is measured on the basis of its

error detection and/or correction capability, its efficiency
Rc=k/n (k → message bits, n → codeword length) and its
implementation complexity.

9
DSC – Chapter 4 University of Trento

CODE TYPOLOGIES

 There are different types of codes that are distinguished by the

way redundancy is introduced into the message.

 In particular, codes can be divided into two large classes: the

class of block codes and that of convolutional codes.

 The class of block codes can be further divided into

subclasses.

10
DSC – Chapter 4 University of Trento

CODE TYPOLOGIES

CODES

BLOCK CONVOLUTIONAL

LINEAR NON LINEAR

REPETITION PARITY HAMMING CYCLIC

CHECK

11
DSC – Chapter 4 University of Trento

CODE TYPOLOGIES
BLOCK CODES (n,k)

 The message is divided into blocks of k symbols, to which

codewords consisting of n=k+q symbols are associated. q
redundancy bits are then introduced.

 In block codes, there is no correlation between the symbols

of one codeword and those of the next.
Xi Xi+1
0 1 0 01 0 11 0 0 0 1 0 0 0 0 11 1 0 0 1 0 1 1 0 0 1 1 0 1 1 1 0

k q k q

12
DSC – Chapter 4 University of Trento

CODE TYPOLOGIES

CONVOLUTIONAL CODES

 The structure does not consist of single codewords: one

message symbol influences the generation of many
symbols of the transmitted sequence.

It is not possible to associate each bit of the encoded

sequence with a particular message. The value of each bit
is influenced by multiple messages.

0 0 1 0 11 0 10 011 1 0 11 0 1 0 1 1 1 0 1 1 0 01 1 0 11 1 0

13
DSC – Chapter 4 University of Trento

BLOCK CODES (n,k)

 Let us focus on block codes (n,k).

 In particular, let us start with two simple examples of block

codes:

 Repetition code

 Parity-check code

14
DSC – Chapter 4 University of Trento

REPETITION CODE

 Repetition codes are linear and systematic codes (we will

see the definition of linearity and systematicity later).

 The encoding strategy consists in building codewords that

repeat n times the bit of information to be transmitted:
n

0 → 000…00
1 → 111...11

 In repetition codes, we have: k=1, q=n-1. Therefore, we are

in the presence of (n,1) codes.

15
DSC – Chapter 4 University of Trento

REPETITION CODE

 Let us calculate the value of the error probability that is

obtained with a repetition code.

 If transmission errors occur randomly and independently with

probability Pbe=a (a <<1), the probability P(i,n) that i errors
occur in an n-bit codeword is equal to:

n i n i
P (i, n ) =  a (1 − a )   a
  n −i

i i
◼ Repetition codes are efficient if a is small enough to satisfy the
condition P(i+1,n) << P(i,n).

16
DSC – Chapter 4 University of Trento

REPETITION CODE: EXAMPLE

 Consider a (3,1) repetition code (i.e., with n=3 and k=1).

This code is characterized by the following words:
0 → 000 e 1 → 111

 Whenever in reception we obtain words other than 000 and

111 (for example 001 or 101) we are clearly in the presence
of errors.
 In reception, depending on the coding/decoding strategy
adopted, we can:
 detect the presence of errors and request retransmission of
the information (ARQ strategy)
 detect the presence of errors and correct them (FEC
strategy)
17
DSC – Chapter 4 University of Trento

REPETITION CODE: EXAMPLE

Message Codeword Single Double Triple

error error error

0 000 (001 010 100) (011 110 101) 111

1 111 (011 101 110) (100 010 001) 000

18
DSC – Chapter 4 University of Trento

REPETITION CODE: EXAMPLE

 With n=3, in the case where we use the code to reveal any
error, we are able to identify 1 or 2 errors (single or double
errors).

 On the contrary, we are not able to detect triple errors (3

errors transform a codeword into another codeword).

 Therefore, probability Pwe of not revealing the presence of

errors in a received word is given by:

Pwe =P(3,3) = a3

19
DSC – Chapter 4 University of Trento

REPETITION CODE: EXAMPLE

 Now suppose we use the same code as before to correct errors by

exploiting a majority decision rule: if a word does not belong to the
code, we transform it into the closest codeword assuming that at
least (n+1)/2 bits are correct.

 If we assume that 2 bits are received correctly (and 1 is wrong),

we can correct the error.

 If, instead, we have 2 or 3 errors, it is not possible to perform their

correction.

 Therefore, the probability Pwe of making a mistake in the correction

is given by:

Pwe= P(2,3)+P(3,3) = 3a 2-2a3

20
DSC – Chapter 4 University of Trento

PARITY-CHECK CODE

 Another very simple type of linear and systematic (n,k) block codes
is that of parity-check codes.

 In parity-check codes:

 a word is said to have even parity when it contains an even

number of “1”s

 a word is said to have odd parity when it contains an odd

number of “1”s

 The words of a (n,n-1) parity-check code contain n-1 information

bits and 1 control bit which is chosen so as to obtain words all
characterized by the same parity.

21
DSC – Chapter 4 University of Trento

PARITY-CHECK CODE:
EXAMPLE (n=3, k=2)
Not revealed because of even parity

Message Codeword Single Double Triple

error error error
00 000 (001 010 100) (011 110 101) 111

01 011 (010 001 111) (000 101 110) 100

10 101 (100 111 001) (110 011 000) 010

11 110 (111 100 010) (101 000 011) 001

Revealed because of odd parity

22
DSC – Chapter 4 University of Trento

PARITY-CHECK CODE

 A parity-check code is able to detect an odd number of

errors.

 The probability Pwe of not revealing the presence of errors in

a received word is then given by:
n  n 2 n(n − 1) 2
Pwe =  P (i , n )  P (2 , n ) =  
 2a (1 − a )n−2
 a
  2
iieven
pari

a<<1

 A parity-check code has no correction capability.

23
DSC – Chapter 4 University of Trento

PARITY-CHECK CODE:
EXAMPLE (n=10, k=9)

 Suppose we are in the case where n=10, k=9 and a=10-3.

 The probability Pwe of not revealing the presence of errors in a

received word is given by:
n(n − 1) 2
Pwe  a = 45  10 −6
2

 If we had not used encoding, we would have had a probability

of getting one message word wrong equal to:
 n − 1 0
Puwe = 1 − P(0,n − 1) = 1 −  a (1 − a ) 
n −1

 0 
= 1 − (1 − a )  1 − 1 − (n − 1)a  = (n − 1)a = 9  10
n −1 −3

24
DSC – Chapter 4 University of Trento

BLOCK CODES (n,k)

 We have seen 2 simple examples of (n,k) block codes.

 In general, given a (n,k) block code:

 It consists of 2k words
-

 Its efficiency is:

nuber-collec
Rc=k/n < 1
maxim
 Its properties (detection and correction capabilities) depend
on how the codewords are chosen and how these words
are separated in their associated n-dimensional space.

25
DSC – Chapter 4 University of Trento

CODE SPACE
 Let us see how the codewords obtained with the
repetition code (3,1) and the parity-check code (3,2)
are represented in the code space of size n=3: Zerzons
3
ords ↓ desoue
:
Z

arances I
001
↑ 001
Codeword 011 011
101 101
Word not belonging 111 111
to the code 000 010 000 010
af
100 · Since they're cosen
100
110 robunues 110
s scolle

Pl 31
,
=
(3/Pb
 In the first case, the codewords are more spaced P(3 31 .
=
(3) Obe

apart a larger number of errors can be

revealed.
26
DSC – Chapter 4 University of Trento

HAMMING DISTANCE

 To mathematically quantify the concept of codeword separation,

we define the Hamming distance and the minimum code distance.

 The Hamming distance d(X,Y) between two binary vectors X and Y

of equal length is the number of different bits that appear in the
two vectors at corresponding positions.
Encorerla
tryto increase desanc
 Example: -

X:101 to
poieu
d(X,Y) = 2 codren
Y:110
"allerenbus
 The minimum code distance dmin is the minimum Hamming
distance between all possible pairs of codewords.

27
DSC – Chapter 4 University of Trento

DETECTION CAPABILITY

 If a number of errors less than dmin has occurred, the word

received is not a word expected by the decoder and therefore it is
possible to reveal the errors.

 If a number of errors greater than or equal to dmin has occurred, the

word received could correspond to a codeword. In this case, it
would not be possible to reveal the errors.

 The rule to evaluate the detection capability of a code is the

following:

Detection capability equal to  errors per word dmin   +1

28
 *
DE SPACE

①
Or x
UNACTIONZID
U ee

a
a

St
.

&
minimal de Sara
un authatd
&

&

0
S
-
...,
NO DETECTED ,

IT BECONGS

To A PretBET
92"-2"-unauthouza
I
①"N ↓ Sequence

detection capabsity : duin-1

stig s

- THANNEL >
-
-
DSC – Chapter 4 University of Trento

CORRECTION CAPABILITY

 If the number of errors is less than dmin/2, we can correct the

received word by transforming it into the closest codeword.

 If the number of errors is greater than or equal to dmin/2, we cannot

correct the received word (the closest codeword may not be the
word that was transmitted).

 The rule to evaluate the correction capability of a code is the

following:

Correction capability equal to t errors per word dmin  2t +1

29
DSC – Chapter 4 University of Trento

DETECTION AND CORRECTION

CAPABILITY

 O
EXAMPLE: Repetition Code (3,1)

 Codewords: X1=(000), X2=(111)

 Hamming distance: d(X1,X2)=3

 Minimum code distance: dmin=3

dmin   +1   3 –1=2

dmin  2t +1 2t  3 –1=2 t1

 So, as we have already observed, a repetition code (3,1) can

reveal up to 2 errors or correct only 1 error.

30
DSC – Chapter 4 University of Trento

DETECTION AND CORRECTION

CAPABILITY

 EXAMPLE : Parity-Check Code (3,2)

 Codewords: X1=(000), X2=(011), X3=(101), X4=(110)

 Hamming distances:

d(X1,X2)=2, d(X1,X3)=2, d(X1,X4)=2, d(X2,X3)=2, d(X2,X4)=2, d(X3,X4)=2

 Minimum code distance: dmin=2

dmin   +1   2 –1=1

dmin  2t +1 2t  2 –1=1 t½

 So (as already seen), a (3,2) parity-check code can detect 1 error, while
it cannot be used for error correction.
31
X Xt) :

dd(xy4
: 1 +
MIN

%na (* , y9)
-
>
Depens on
the
wih

↓ ~ [ 1 + MAX)da(x9 491/
message
.

-1 +a

101
- Entr den (x 4),
= 1 +g

·
9 :
2 =

admin/ax : 3
DSC – Chapter 4 University of Trento

DETECTION AND CORRECTION

CAPABILITY

 The minimum distance of a code (and therefore its

detection and/or correction capability) depends on the
number of redundancy bits that are added.

 In particular, in the case of a block code (n,k), the following

relationship holds:
dmin  n – k +1

 Unfortunately, the upper limit is reached only in the case of

repetition codes which, however (having k=1) have a very
low efficiency:
Rc=k/n = 1/n
32
DSC – Chapter 4 University of Trento

BLOCK CODE PROPERTIES

 We have seen that one of the fundamental steps in building

a code is to choose the words in such a way as to maximize
dmin.

 There are other properties that a block code should satisfy

in order to ease the encoding and decoding operations.

 These properties are:

 linearity

 sistematicity

33
DSC – Chapter 4 University of Trento

LINEARITY

 Consider two generic codewords that can be represented

as 2 vectors X=(x1 x2 … xn) and Z=(z1 z2 … zn), where the
elements xi and zi are bits.

 The sum operation between the two codewords is defined

as:
X+Z=(x1 z1 x2  z2 … xn  zn)

 A code is defined as linear if it contains the null vector and if

the sum of two generic words belonging to it always
produces another codeword.

34
Repritions
1 Yaxe lite

Pain
DSC – Chapter 4 University of Trento

LINEARITY

 Let us define the weight w(X) as the number of non-zero

elements of the vector X.

 It can be easily verified that the Hamming distance between

two codewords is given by:

d(X,Z)=w(X+Z)

 By exploiting the linearity property, it can be shown that:

dmin=[w(X)] min X(0 0 …. 0)

 The minimum distance of a linear code is equal to the

smallest of the weights of the codewords (excluding the null
vector).
35
DSC – Chapter 4 University of Trento

SISTEMATICITY

 A systematic code is defined as a block code in which the

first bits xi (i=1,…,k) correspond to the message bits, while
the control bits xj (j=k+1,…,n) are grouped at the end of the
codeword.

 A word of a systematic code is thus structured as:

X = (m1 m2 … mk c1 c2 … cn-k ) =

=(M|C)

36
DSC – Chapter 4 University of Trento

MATRIX REPRESENTATION OF
LINEAR SYSTEMATIC CODES

 Matrix representation is very useful because it allows to generate

codewords by means of a simple row-by-column multiplication of
matrices.

 In fact, given a message M, its codeword can be written as:

X=MG

 Matrix G (of size kxn) is called generator matrix and has the following

t
structure:
1 0 ... 0 p1,1 p1,2 ... p1,n − k 
 
0 1 ... 0 p 2 ,1 p 2 ,2 ... p 2 ,n − k 
 
def
G = Ik P =
   
  k
9
0 0 ... 1 p k ,1 ... ... p k ,n − k 
~ Lexul Cexul Min
M G
bina alienis X = +

37
:
[MC]
DSC – Chapter 4 University of Trento

MATRIX REPRESENTATION OF
LINEAR SYSTEMATIC CODES

 Row-by-column matrix multiplication follows the classical

rules, except that a modulo-2 sum is used instead of the
natural sum.

 Therefore, control (redundancy) bits are calculated as:

cj =m1 p1,j  m2 p2,j  …  mk pk,j

OBSERVATION

 There is no analytical method to determine the elements of

the submatrix P and relate them to dmin.

 The determination of good codes is mostly due to

inspiration and perseverance.
38
DSC – Chapter 4 University of Trento

HAMMING CODE

 A Hamming code is a linear block code for which the

following relations hold:

q=n-k3 and n=2q-1

 The code rate is therefore equal to:

9
-
ORc -
>1

k q
Rc = = 1 − q 3 an = = =
k 4
2 −1 q
=

n
=
=

9 4 = 15
: D K ll
=

i
39
 es[ii) 3
DSC – Chapter 4 University of Trento

HAMMING CODE

 It is possible to verify that, independently of q, the minimum

distance is constant and equal to:

dmin=3

 Thus, a Hamming code can detect up to two errors or correct

one error for each codeword.

 To construct a systematic Hamming code, it is enough to

choose a submatrix P consisting of rows of q bits, all of which
have two or more bits set to “1”.

I
trivovoscon non
>
-

40
DSC – Chapter 4 University of Trento

HAMMING CODE: EXAMPLE

 Consider a systematic Hamming code with q=3. -

 An example of generator matrix G is the following:

1

0 0 0 1 0 1

se [Mic]

G=
0 1 0 0 1 1 1 cics
0 0 1 0 1 1 0 mx

0

0 0 1 0 1 1
4 =

[1]
( My
Control bits are therefore: [i]
=

c1 =m1  m2  m3  0
c2 =0  m2  m3  m4
C = Mx
[])
c3 =m1  m2  0  m4
41
DSC – Chapter 4 University of Trento

HAMMING CODE: EXAMPLE

 The encoder can then be constructed as follows:

mi
Buffer message bits

m4 m3 m2 m1
X

c3 c2 c1 control bits

42
DSC – Chapter 4 University of Trento

HAMMING CODE: EXAMPLE

 The resulting 24 codewords are shown in the following

table:

 It is noteworthy that the smallest non-zero weight w(X) is 3.

This confirms that dmin =3.

43
 a
= 4 =
an = 13 = Bk = 1

10 - 0000000
198
- O
G
I
= - &
01 -
S
, o
&
O O 1 -
-
-
100
11x15 0
o
-O ⑧ 1 110
&
1
S
I
&
001
I S
1
S

I 0 10 I
G
&
1 101
S

1 &
& 01 I I
&
I

=
I 1 S 1 O
I I
-

↑ 1 S 110 I

1
1' I'l
0 0000 e 0 0 O 8

-
-
P
Ik
X 16
=
: M +
[IniP]
[M In : M .

P]
= [MIc]
control bis
it (m ..
mony . . -

Mil

C =
(C(zx)
colora de P
-
prodotto de M per
1

S
collicient ( = m . Mi MEDMSMDM
-
W2m2

C =
prodotto der
per
2 colora de P

>
-
Mi
M
tr

X =
MG
i

[Iv1, dea : to use

I fer decode
DSC – Chapter 4 University of Trento

DECODING OF LINEAR
SYSTEMATIC CODES

 Suppose we transmit codeword X and receive a vector Y. If

transmission errors have occurred, YX.

 One way to perform error detection is to compare Y with

each of the codewords all 2k codewords need to be
memorized. >
- problemat, ma and
computation
(2k n)
=

EXAMPLE

 Hamming code with q=5 n=31 and k=26 need

to memorize 226 words!

44
DSC – Chapter 4 University of Trento

DECODING: PARITY-CHECK MATRIX

 A more efficient method to perform decoding and detect errors is to

use submatrix P.

 Parity-check matrix H is defined as:

m

 
def
· H =O
P Iq
T

(axk) (axa)
qq identity matrix
 This matrix has the following property:

- T
XH = ( 0 0 0 … 0 )

where X is any codeword.

 If Y is not a codeword, product YHT contains at least one non-zero

-

element.
45
DSC – Chapter 4 University of Trento

SYNDROME DECODING:
ERROR DETECTION

lexal (exulluxal
 The quantity S=YHT is called syndrome and allows to detect possible
errors present in the received vector Y.

 If S is a null vector, 2 possible cases emerge:

1. no errors occurred

2. errors occurred, and Y is a codeword but different from the

codeword X transmitted (errors are not detectable)

 If syndrome S contains at least one non-zero element, errors are

present (and thus detectable).

46
DSC – Chapter 4 University of Trento

SYNDROME DECODING:
ERROR CORRECTION

 Error correction is more complex, but can still exploit the

properties of syndrome S.

 Let us define an error vector E consisting of n bits. This vector

indicates the position of transmission errors contained in Y.

EXAMPLE

X= ( 1 0 1 1 0 )
obinayvector
I

E=(00101)
accred tlat
Y= ( 1 0 0 1 1 ) oror
in

poster

47
DSC – Chapter 4 University of Trento

SYNDROME DECODING:
ERROR CORRECTION

 In general, one can write:

Y = X+E

and thus also:

X = Y+E

 Syndrome S can be rewritten as: the platan

t
-D

O
S=YHT=(X+E)HT = X HT +E HT= E HT

The syndrome therefore depends only on the errors that

occurred and not on the specific codeword X transmitted.

48
DSC – Chapter 4 University of Trento

SYNDROME DECODING:
ERROR CORRECTION Error sce

Overine
SPACESINDR

PROBLEM


021-e
The possible errors on a codeword encoded with n bits are
2n-1. Space Sindrome 29 :

 The syndrome, being made up of n-k bits, can only

represent 2n-k -1 errors.

Each syndrome vector S corresponds to several error

vectors E. It is therefore necessary to identify the error that
is most likely to be associated with each specific syndrome
value.
49
DSC – Chapter 4 University of Trento

MAXIMUM LIKELIHOOD DECODING

 The strategy known as maximum likelihood decoding consists in

choosing the most probable error vector Ê among all possible E vectors
that generate the same syndrome S=EHT. NEAMESTPONT W

-
SPAC
I
 It can be shown that this vector is the one that contains the smallest
number of incorrect bits. This choice corresponds to determining the
codeword X̂ that has the smallest Hamming distance from the received
vector Y.

 Maximum likelihood decoding is optimal in the sense that it minimizes

-

the probability of error per word Pwe.

-

50
DSC – Chapter 4 University of Trento

MAXIMUM LIKELIHOOD DECODING

𝒆ො 𝒏 … 𝒆ො 𝒌 … 𝒆ො 𝟐

Y + Ê = X̂

Y
yn … yk … y2 y1 In reality, we are only
… … interested in the first k bits
as they are the ones that
Syndrome computation contain the message.
S
Look-up table
Ê

51
DSC – Chapter 4 University of Trento

CYCLIC CODE
heghzonela
e

 It can be shown that, in order to achieve a correction capability t 1

and Rc 1, it is necessary to have n>>1, k>>1.

 Unfortunately, in general, the hardware complexity required to encode

and decode codewords with large length is very high.

 In order to overcome this limitation, one can resort to cyclic codes.

 Cyclic codes are a subclass of linear block codes whose structure

allows a reduction in the complexity of the encoder and decoder even
in the presence of codewords characterized by a large length.
Rc =E = #
Y
u+ 7
q COR
>
-

U Dduu RET
=
Dq
=
52 &P
DSC – Chapter 4 University of Trento

CYCLIC CODE
m
 Consider a generic codeword consisting of n bits:

X=(xn-1 xn-2 … x1 x0 )

 A code is said to be cyclic if:

 It satisfies the linearity property.

 Each cyclic shift of the bits of codeword X generates another

codeword:

xn-1 xn-1 … x1 x0

For instance, X’=(xn-2 … x1 x0 xn-1 ) is a codeword.

53
DSC – Chapter 4 University of Trento

-
CYCLIC CODE
I

 Cyclic codes can be treated mathematically by associating

a polynomial X(p) to the codeword X:
Postor lestone
X(p) =xn-1 pO
n-1 + x
n-2 pO… + x p + x
n-2
1 0

with p an arbitrary real variable.

 The power to which p is raised indicates the position in the

codeword of the bit represented by the coefficient
associated with p.

 As seen above, cyclic codes are also called polynomial

codes.

[100111935x6 p php p2
+
y = : +
2 54
DSC – Chapter 4 University of Trento

CYCLIC CODE: MATHEMATICAL

FORMULATION

 The mathematical formulation that provides the theoretical

basis for the treatment of cyclic codes relies on the Galois
fields.

 Without going into details, let us assume the following:

 the sum of two polynomials is obtained by the modulo-2 sum

I of their respective coefficients.

 since these coefficients can only assume the value “1” or “0”
and 11=0 subtraction operation is equal to modulo-2
addition: X.
(p( Xz(p) X3(9)
+ =

X(p)+Z(p)=0 X(p)=Z(p)
55
DSC – Chapter 4 University of Trento

CYCLIC CODE: MATHEMATICAL

FORMULATION

 Let us analyze what relationship exists between 2 codewords in

a cyclic code. Consider the following two polynomials:

pX(p) =xn-1 pn + xn-2 pn-1+… + x1 p2 + x0 p

X’(p) =xn-2 pn-1 +… + x1 p2 + x0 p + xn-1

 Summing up pX(p) and X’(p), one gets:

pX(p)+X’(p)=xn-1 pn + (xn-2 xn-2 ) pn-1+…+(x1 x1) p2 +(x0 x0) p+xn-1

0 0 0

pX(p)+X’(p)=xn-1 pn + xn-1 X’(p)= pX(p)+ xn-1 (pn +1)

56
 - --

[Xn Yne---
X =

.. XXo]
2 X(p Yn-ph Xep" Xu-sphs: + + +
Xep2 Xope +

X [Xne Xas XXoYnc] --

X (pl :

Xneph Xnsph + -
----Xopt :
po
CROTATED

TANSCATION :
px(1 +
Yuzph
2
Xn- > p X, p tope
.

: + +
+ +
--

x((p) px()
+ = Xn - ph + Xn po =
Xn -1(p4 +
1)

=
px(p) Xn 1(ph 1) + - +

La correction
Thoudated
Generator polyrarial
G(pl -G(pl pa
ga 1 pa geph pas
= + +
- +.. -

facta el phie

G(P)
And :And G
DSC – Chapter 4 University of Trento

CYCLIC CODE: GENERATOR

POLYNOMIAL

 By performing multiple shifts, we obtain results similar to those

seen previously for a single shift.

 The polynomial (pn+1) and its coefficient xn-1 play a very important
role in cyclic codes.

 In particular, a cyclic code (n,k) is defined by means of a

generator polynomial G(p) having the following form:

G(p) = pq + gq-1 pq-1 +…+ g1 p + 1

where coefficients gi are such that G(p) is a factor of the

polynomial (pn +1).

57
DSC – Chapter 4 University of Trento

CYCLIC CODE: GENERATOR

POLYNOMIAL

 It can be shown that each codeword can be written as:

X(p) = QM (p)· G(p)

where QM (p) is a polynomial representing a block of k message bits.

 Any factor of (pn+1) that has degree q can be used as a generator

polynomial, but it does not necessarily provide a good code.

 The most commonly used polynomial codes include the cyclic

Hamming, Golay, and Bose-Chaudhuri-Hocquenghem (BCH) codes.

58
DSC – Chapter 4 University of Trento

CYCLIC CODE: EXAMPLES

e

of the most used cyclic codes.

Codice n k Rc dmin
n 7=

↑
The following table reports the generator polynomials of some

G(p)
i Ok it's
offe

7 4 0.57 3 genota
evid a 1 011 p3 +
p
+2

Hamming 15 11 0.73 3 10 011

31 26 0.84 3 100 101
15 7 0.46 5 111 010 001
BCH 31 21 0.68 5 11 101 101 001

OO
63 45 0.71 7 1 111 000 001 011 001 111
Golay 23 12 0.52 7 101 011 100 011

approva i Muodes en
layer cod
59
&
DSC – Chapter 4 >
University of Trento

SYSTEMATIC CYCLIC CODE

 In general, a cyclic code can be systematic or non-

systematic. This depends on the structure of the QM (p) term.

 In the following, we will focus on systematic cyclic codes.

Codwords ?
G(p) -

X a Xmpmm
-pr(p +A d
mush
M - (p) 7 no]: 0 :

[Mu . The ... Me

PM(p) pk
1
= = mk -

1 + wk 29
pl M
+ la +
,
C [Ca (2
--

CC]
.

=
.
....

=>
C(p1 Ca :
.

1 pq Ca pa 2
-

. . .
+
---
Gp' Cope
pa
-
op
ani Re
60
DSC – Chapter 4 University of Trento

SYSTEMATIC CYCLIC CODE

 Let us consider a generic systematic cyclic code and define

the polynomial M(p) of the message bits and the polynomial
C(p) of the control bits:

M(p) =mk-1 pk-1 +… + m1 p + m0

C(p) =cq-1 pq-1 +… + c1 p + c0

 For the code to be systematic, codeword X(p) associated

with message M(p) must be expressed as:

X(p) = pqM(p) + C(p)

61
DSC – Chapter 4 University of Trento

SYSTEMATIC CYCLIC CODE

 From what seen above, the following relationship must be

valid:

X(p) = QM (p)· G(p)= pq M(p) + C(p)

p q M ( p) C( p)
= QM ( p ) +
G( p ) G( p )

 The polynomial of control bits can then be calculated as the

remainder of the division of pq M(p) over G(p):
SG :Col

 q

C ( p ) = p M ( p ) mod G ( p )
↑ M(pl p
=
X =

+
2
1

Modulo operator >

-

62
 -pel C =

DSC – Chapter 4 #2
University of Trento [10 +1 = , J

= X =

[oo 11101

CYCLIC CODES: SYNDROME

 Even for cyclic codes, decoding can be performed using the

syndrome.

 Calculation of syndrome S(p) in reception is straightforward:

S ( p ) = Y ( p ) mod G( p )

 If polynomial Y(p) associated with the received word is a

codeword, then S(p) = 0 since G(p) is a factor of Y(p). If instead
S(p)  0, it means that the received word is affected by the ↑

presence of errors. Chat -

↑
- siti -

-(p
x(p)

63
DSC – Chapter 4 University of Trento

CYCLIC CODE: ADVANTAGES

 The main advantages of cyclic codes are:

 coding simplicity lugh cookzate ,

corre
nugh Dir copobetis
 syndrome calculation simplicity

 well-defined mathematical structure that allows the

development of very efficient decoding methods. Such
methods drastically reduce the need to memorize
information and therefore make it possible to use codes
with n>>1, k>>1 Rc1.

64
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE
 Convolutional codes, unlike block codes, do not generate sequences
organized into well-defined codewords.

 In order to understand the operating principle of convolutional codes,

let us analyze the following scheme:
input sine of Reggim
state (past defines
SHEFT REGISTER
mi ...
mi mi-1 mi-L D
gi ={0,1}
message bit

-
Weiglis g0 g1 ... gL -
seems
the
predcorig
xi

similar to convolution
L integral
xi = mi-L gL  …  mi-1 g1  mi g0 =  mi − j g j
j =0
65
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE

 Encoded bit xi depends on input mi and the state of the

register (i.e., the previous L message bits).

 Message bit mi influences the subsequent L+1 bits.

 The mechanism illustrated so far allows a sequence of bits

to be transformed into a different sequence, but does not
add redundancy.

 How can this be used to introduce redundancy?

66
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE

 To add the redundancy bits needed for error control, we

may resort to two or more modulo-2 adders:
input
state
gp,j = {0,1} indicates whether the
mi j-th cell of the shift register is
mi mi-1 ... mi-L
message bit connected to the p-th adder.
g1,0 g1,1 ... g1,L
L
x 'i =  m i− j g 1, j
j =0
X = ....x'i x"i .....
g2,0 g2,1 ... g2,L

L
x "i =  m i− j g 2 , j 67
j =0
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE

 In general, if one uses n modulo-2 adders, code rate Rc=1/n.

 Message bit mi influences the following n(L+1) bits.

 Quantity n(L+1) is called constraint length.

 L, namely the length of the shift register, refers to its

memory.

 Similarly to what was done for block codes, convolutional

codes adopt the following parameterization (n,k,L).

68
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE

 In order to obtain a code rate Rc>1/n, it is necessary to use

k2 shift register. In particular, using k shift registers, it is
possible to achieve Rc=k/n.

 The next slide shows an example of how to build an

encoder to obtain a convolutional code characterized by
Rc=2/3.

69
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE: EXAMPLE

: how
mary
- mach

 O
Encoder scheme of a convolutional code (3,2,1) pacious

euroz
(i.e., characterized by n=3, k=2 e L=1).
stilTs
,
med

input state register

mi-1 mi-3
message bit
we're robeing in

mi mi-2 Temol
LeAtte

8
~
X = . . . . x 'i x "i x 'i ' ' . . . .

3 combination

Combi-convo
:
poder d
3 bisie

70
(n =

3)
ave stadwelsin pote
a
cauwe
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
-

OBSERVATIONS

 To obtain a code rate Rc>1/n, as an alternative to using k

registers, it is possible to use a single register of length k(L+1)
and shift the bits in groups of k.

 Convolutional codes typically used in FEC systems are

characterized by small values ​of k and n and by a constraint
length between 10 and 30.

 It is possible to obtain convolutional codes in systematic form,

that is, characterized by the following structure: ↓
Input pocifut
X = m1 x1'' x'2'' .........mi x'i' x'i'' .......

71
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE: GRAPHICAL

REPRESENTATIONS

 Three different types of representation can be used for

convolutional codes:

 Tree representation

 Trellis representation

 State diagram representation

72
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE: EXAMPLE

 Let us consider the convolutional code (2,1,2) (i.e. having

n=2, k=1, L=2) obtained with the following encoder:
input
state

mi
mi mi-1 mi-2
message bit

X = . . . . x 'i x "i . . . .

 Suppose the register contains all “0”s when the first

message bit m1 arrives.

73
DSC – Chapter 4 University of Trento

TREE REPRESENTATION
00
00 a
output a 11
00 b
State a 10
11 c
a=00 b 01
00 d
b=10 a 11
10 a
c
c=01 11 00 b
b 01
d=11 01 c
d 10 d
a
00
11 a
a
11 b
state transition that occurs as 10 c 10
a result of an input mi=0 00 c
b 01
11 d
b 11
01 a
c
00 b
state transition that occurs as 01
a result of an input mi=1 d 01
10 c
d 10 d 74
DSC – Chapter 4 University of Trento

TRELLIS REPRESENTATION

current state output successive state

State 00
a a
a=00 11
b=10 b 11 b
00
10
c=01 c
c 01
d=11 01
d d
10

state transition that occurs as

a result of an input mi=0

state transition that occurs as

a result of an input mi=1
75
DSC – Chapter 4 University of Trento

STATE DIAGRAM REPRESENTATION

output

b
State
11 01
a=00
b=10 00 a 00 10 d 10
c=01
11 01
d=11
c

state transition that occurs as

a result of an input mi=0

state transition that occurs as

a result of an input mi=1
76
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE: EXAMPLE

 Given an input sequence and the initial state, it is possible to

obtain the sequence of bits generated by the encoder by
analyzing the trellis or the state diagram.

Input 1 1 0 1 1 1 0 0 1 0 0 0

State a b d c b d d c a b c a a

Output 11 01 01 00 01 10 01 11 11 10 11 00

77
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
FREE DISTANCE

 Correction and/or detection capabilities of a block code

depend on its minimum distance.
 In block codes, the minimum distance is determined by
analyzing the weights of the codewords.
 In the case of a convolutional code, since codewords
cannot be identified, the weight of the entire sequence X
generated by a given sequence of messages is considered.
 The free distance of a convolutional code is defined as the
quantity:
dl=[w(X)] min X(0 0 0 ….)

78
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
FREE DISTANCE

 The calculation of dl can be easily done without having to

analyze the weight w of all possible sequences w(X).

 One must identify the path with minimum weight that detaches
itself from the trivial all-zero path and return to it in a stable
way.

 Suppose we end the sequence with a series of “0”s in order to

return the shift register to its initial state.

 This procedure eliminates some branches from the final L

transitions of the trellis (i.e., trellis termination).

79
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
FREE DISTANCE

 Consider the trellis in the previous example and associate

each branch with the number of "1"s that appear in the
corresponding coded sequence:

0 0 0 0
a a a a F
2 2
2
b 1 2 b 1 2 0 b 2
1

c 0 1 c 1 c c
1 1 final state
1
d 1
d d
1

80
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
FREE DISTANCE

 Consider the path with minimum weight (not all zero) that
starts from state a and arrives at the final state F:

0
a a a a F
2
1 2
b b b

c c c c
final state

d d d

 Let us now examine the sequence of inputs X which yields

the sequence of states aaaa…abcF.

 The weight of this sequence is w(X)=5.

81
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE:
FREE DISTANCE

 It can be checked that there are no paths corresponding to

non-identically zero input sequences X with a weight w(X)
less than that of the path aaaa…abcF.

 Therefore, the free distance is dl=5.

 The above-described strategy allows to easily calculate dl

without having to analyze all possible paths X.

82
DSC – Chapter 4 University of Trento

DECODING CONVOLUTIONAL CODES

 There are two main methods for decoding convolutional

codes:
 Viterbi Algorithm: performs optimal maximum likelihood
decoding, but requires complex hardware

 Sequential Decoding: can make different tradeoffs between

hardware complexity and decoding quality

83
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM

 An optimal maximum likelihood decoder needs to examine the

entire received sequence Y and find the “authorized” path that
has the smallest Hamming distance from Y.

 If the transmitted message consists of N bits (i.e., Y consists of

Nn/k bits) there are 2N possible paths.

 In real cases, being typically N>>1, an exhaustive comparison

of Y with all possible paths is prohibitive.

84
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM

 The Viterbi algorithm, independently of N, allows to limit the

comparison to 2kL surviving paths, making it possible to apply optimal
maximum likelihood decoding.

 Each branch of each path is assigned a metric that indicates its

Hamming distance from the corresponding branch of Y.

 The metric of each path is obtained by adding the values ​of the
metric of the branches that compose it.

 The sequence Y is decoded as the path with minimum metric.

85
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 Let us study how Viterbi decoding works with an example.

 Consider the decoder associated with the n=2, k=1, L=2

convolutional code seen previously, and suppose we
receive the sequence Y = 11 01 11.

 Basing on its code trellis, we will understand how:

 the metric is assigned to the different branches

 the surviving paths can be selected

 the Viterbi algorithm is applied

86
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 Consider the first 3 possible transitions from initial node a0:

Y= 11 01 11

2 00 2 1
00 3 2 00 5
a0 a1 a2 a3
11 0
11 0
11 2
1 3
0
b1 b2 1 11 3 b3
2
00
10 2 1 4
0 0
4
c2 01 10 c3
2
0 1
01 1
d2 01 4
d3
Branch metric Path metric 1 10
1

87
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 We observe that each state at the third transition (i.e., either a3 or b3 or c3

or d3) can be reached with 2 different paths.

 For example, we can reach b3 by going from a0,a1,a2,b3 or from a0,b1,c2,b3.

Regardless of what will be transmitted next, the path that includes the
transitions a0,a1,a2,b3 has a smaller metric and is therefore more likely to
represent the transmitted sequence X.

 Reasoning in this way, for each of the nodes a3 ,b3 ,c3 ,d3 that we have at
the third transition, we can eliminate the path with the larger metric, thus
reducing the number of paths to be memorized by keeping only the
surviving paths.
88
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 In our example, we obtain 2kL=4 surviving paths of length

j=L+1=3:
Y= 11 01 11

00 00
a0 a1 a2 2 a3
11 11
b1 b2 11 b3
3
10
c2 1 c3
01
d2 01 1
d3
10
89
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 At this point, the Viterbi algorithm iteratively increases j (j=j+1)

and applies the same procedure to grow the paths (all consisting
of one more branch at each iteration).

 A new set of 2kL=4 surviving paths is generated (each

characterized by length j).

 This iterative process ends when the last bit of the received
sequence Y is reached.

90
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: EXAMPLE

 Example of what happens in the case of transmission of N=12 bit:

Maximum
likelihood path

91
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM

 With the Viterbi algorithm, it is necessary to compute 2

metrics for each node and store 2kL surviving paths (each
consisting of N branches).

 Its complexity grows exponentially with L and linearly with N.

 This limits the application of this algorithm to codes with small

values ​of L.

92
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: METRIC-

DIVERGENCE EFFECT

 When N>>1 , a simplified strategy can be adopted that allows to

reduce the amount of memory required by the algorithm.

 This strategy is based on the observation that, if two surviving

paths start from a node, the metric of the least probable path
tends to increase much more rapidly than that of the other
surviving path.

 This effect becomes evident after about 5L branches.

93
DSC – Chapter 4 University of Trento

VITERBI ALGORITHM: METRIC-

DIVERGENCE EFFECT

 After the first 5Ln bits have been received, the first k bits of the
message can be decoded (and consequently the first sets of
branches can be erased from memory).

 Subsequent sets of k bits are decoded after each additional

set of n bits received.

 In this way, (simplified) decoding does not require waiting for

the end of the transmitted sequence.

94
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING

 The application of the Viterbi algorithm is limited to codes with a

small value of L.

 Problem: small value of L limited corrective capacity

 Possible solution: increase L and use a non-optimal, but

computationally simple decoding technique

sequential decoding

95
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING

 Sequential decoding is based on the metric-divergence effect.

 Starting from the initial state a0, a sequential decoder takes the path that
has the smallest metric at the next node (if 2 paths have the same metric,
one is chosen randomly).

 If the decoder takes a path that turns out to be “too” unlikely, in a few
branches the metric of that path tends to grow very quickly. In these
cases, the decoder can “decide” to go back and try another path
(backtracking).

96
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING

 Backtracking decision is based on the expected value of the

metric at a certain node.

 The expected value of the metric at the j-th node is equal to

jnPbe (with Pbe the bit error probability). This value coincides
with the expected number of incorrect bits in the first jn bits of
the sequence Y.

 The decoder performs the backtracking operation if the metric

of a path exceeds by a certain threshold  the quantity jnPbe.

97
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING

 If  is small: the decoder tends to analyze all possible

paths and the performance approaches that of a maximum
likelihood decoder (however, in this case, frequent
backtracking makes decoding more complex than that of the
Viterbi algorithm).

 If  is large: decoding time and complexity are

reduced, but probability of errors increases.

98
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING: EXAMPLE

path associated
with the transmitted
sequence

99
DSC – Chapter 4 University of Trento

SEQUENTIAL DECODING: EXAMPLE

path associated with

jnPbe+ the transmitted
sequence

jnPbe

100
DSC – Chapter 4 University of Trento

CONVOLUTIONAL CODE: PROS/CONS

 In general, the encoding operation is simple, while in decoding

high performance can be obtained only by means of sophisticated
algorithms.

 Convolutional codes are particularly suitable for satellite

communication systems. In such systems, the use of simple
encoders is essential (mounted on board the satellites), while the
use of complex decoders is acceptable (carried out at the ground
segment which typically includes powerful computers).

101
DSC – Chapter 4 University of Trento

APPLICATION EXAMPLES OF CODES

 The choice of coding approach (FEC/ARQ) and code type

depends on the application of interest.

 Let us consider, for example, some important cases:

 AWGN channel coding

 Compound channel coding

 Data-storage coding

102
DSC – Chapter 4 University of Trento

AWGN CHANNEL CODING

 When considering channels that can be modeled with AWGN

noise (e.g., in satellite radio communications and in deep space
communications), the use of FEC strategies is particularly
recommended.

 Depending on the desired coding gain, different types of codes

can be chosen:

 Moderate encoding gain: convolutional codes with

small constraint-lengths and simplified Viterbi decoding

 High encoding gain: convolutional codes with large

constraint-lengths and sequential decoding

103
DSC – Chapter 4 University of Trento

COMPOUND CHANNEL CODING

 Compound channels are characterized by the mixed presence of

independent errors and burst-type errors (continuous sequences
of errors).

 Examples are:

 telephone channels (burst errors occur due to impulsive

noise)

 radio channels (burst errors appear because of atmospheric

noise, interference, etc.)

 In these cases, ARQ strategies using cyclic linear block codes are
generally used.

104
DSC – Chapter 4 University of Trento

DATA-STORAGE CODING

 This type of application involves the use of error control

codes in computer memories, digital magnetic tapes,
magnetic disks, etc.

 In these cases, both ARQ and FEC strategies can be

exploited.

 Typically, linear block codes are used (e.g., Hamming

codes).

105