Viet Nam National University Ho Chi Minh City

University of Science
Faculty of Electronics & Telecommunications

Chapter 8a: Error Correcting Codes:

Linear Block codes

Dang Le Khoa
[email protected]
 Outline
⚫ Channel coding
⚫ Linear block codes
– The error detection and correction capability
– Linear block codes: Encoding and decoding
– Hamming codes
– Cyclic codes

2
 Block diagram of a DCS

Source Channel Pulse Bandpass

Format
encode encode modulate modulate

Digital modulation

Channel
Digital demodulation

Source Channel Demod.

Format Detect
decode decode Sample

3
 What is channel coding?
◼ Channel coding:
Transforming signals to improve
communications performance by increasing
the robustness against channel impairments
(noise, interference, fading, ..)
◼ Structured sequences: Transforming data
sequences into better sequences, having
structured redundancy.
-“Better” in the sense of making the decision process less
subject to errors.

4
 Error control techniques
⚫ Automatic Repeat reQuest (ARQ)
– Full-duplex connection, error detection codes
– The receiver sends a feedback to the transmitter, saying that if
any error is detected in the received packet or not (Not-
Acknowledgement (NACK) and Acknowledgement (ACK).
– The transmitter retransmits the previously sent packet if it
receives NACK.
⚫ Forward Error Correction (FEC)
– Simplex connection, error correction codes
– The receiver tries to correct some errors
⚫ Hybrid ARQ (ARQ+FEC)
– Full-duplex, error detection and correction codes
5
 Why using error correction coding?
◼ Error performance vs. bandwidth

PB
Coding gain:
Coded
For a given bit-error probability,
the reduction in the Eb/N0 that can be
A
realized through the use of code:
F
 Eb   Eb 
G [dB] =   [dB] −   [dB].
C B

 N0 u  N 0 c D
E
Uncoded

Eb / N0 (dB)

6
 Some definitions

⚫ Binary field :
– The set {0,1}, under modulo 2 binary addition and
multiplication forms a field.

Addition Multiplication
0 0 =0 0 0 = 0
01 =1 0 1 = 0
1 0 =1 10 = 0
1 1 =0 1 1 = 1
– Binary field is also called Galois field, GF(2).

7
 Linear block codes

◼ The information bit stream is chopped into blocks of k bits.

◼ Each block is encoded to a larger block of n bits.
◼ The coded bits are modulated and sent over the channel.
◼ The reverse procedure is done at the receiver.

Channel
Data block Codeword
encoder
k bits n bits

n-k Redundant bits

k
Rc = Code rate
n

8
 Linear block codes

⚫ The Hamming weight of vector U, denoted by w(U),

is the number of non-zero elements in U.
⚫ The Hamming distance between two vectors U and
V, is the number of elements in which they differ.
d (U, V ) = w(U  V )
⚫ The minimum distance of a block code is
d min = min d (Ui , U j ) = min w(Ui )
i j i

9
 Linear block codes – cont’d

⚫ Error detection capability is given by

e = dmin − 1

⚫ Error correcting-capability t of a code, which is

defined as the maximum number of guaranteed
correctable errors per codeword, is
 d min − 1 
t= 
 2 

10
 Linear block codes – cont’d

◼ Encoding in (n,k) block code

U = mG
 V1 
V 
(u1, u2 , , un ) = ( m1, m2 , , mk )   2
 
 
◼ The rows of G are linearly independent.
 Vk 
(u1, u2 , , un ) = m1  V1 + m2  V2 + + m2  Vk .

Lecture 9 11
 Linear block codes – cont’d
⚫ Example: Block code (6,3)

Message vector Codeword

000 000000
 V1  110100  1 00 11 01 00 .
  
G =  V2  = 011010  01 0 011 01 0
 V3  101001  11 0 1 0111 0
001 1 01001
1 01 0111 01
011 11 0011
111 000111

12
 Linear block codes – cont’d

⚫ Systematic block code (n,k)

– For a systematic code, the first (or last) k elements in the codeword
are information bits.

G = [P Ik ]
I k = k  k identity matrix
Pk = k  (n − k ) matrix

U = (u1, u2 ,..., un ) = ( p1, p2 ,..., pn − k , m1, m2 ,..., mk )

parity bits message bits

13
 Linear block codes – cont’d

⚫ For any linear code we can find an matrix H( n − k )n , such

that its rows are orthogonal to the rows of G :

GH = 0 T

⚫ H is called the parity check matrix and its rows are

linearly independent.
⚫ For systematic linear block codes:

H = [I n −k PT ]

14
 Linear block codes – cont’d

⚫ Standard array
n−k
– For row i = 2,3,...,2 find a vector in Vn of minimum
weight which is not already listed in the array.
– Call this pattern e i and form the i:th row as the corresponding
coset
zero
codeword U1 U2 U 2k
e2 e2  U 2 e 2  U 2k coset
.

coset leaders
e 2n − k e 2n − k  U 2 e 2 n − k  U 2k

15
 Linear block codes – cont’d
⚫ Example: Standard array for the (6,3) code

codewords

000000 110100 011010 101110 101001 011101 110011 000111

000001 110101 011011 101111 101000 011100 110010 000110
000010 110111 011000 101100 101011 011111 110001 000101
000100 110011 011100 101010 101101 011010 110111 000110
.
001000 111100
010000 100100 coset
100000 010100
010001 100101 010110

Coset leaders

16
 Linear block codes – cont’d

Data source Format

m Channel U Modulation
encoding
channel
Channel Demodulation
Data sink Format
decoding r Detection
m̂

r = U+e
r = (r1 , r2 ,...., rn ) received codeword or vector
e = (e1 , e2 ,...., en ) error pattern or vector.

◼ Syndrome testing:
◼ S is the syndrome of r, corresponding to the error pattern e.
S = rHT = eHT
17
 Linear block codes – cont’d

⚫ Standard array and syndrome table decoding

1. Calculate S = rH .
T

2. Find the coset leader, eˆ = ei , corresponding to S .

3. Calculate Uˆ = r + eˆ and corresponding m̂ .

– ˆ = r + eˆ = (U + e) + eˆ = U + (e + eˆ )
Note that U
If eˆ = e , error is corrected.
If eˆ  e , undetectable decoding error occurs.

18
 Linear block codes – cont’d

Error pattern Syndrome

U = (101110) transmitted
000000 000
r = (001110) is received.
000001 101 The syndrome of r is computed:
000010 011
S = rHT = (001110)HT = (100)
000100 110
. Error pattern corresponding to this syndrome is
001000 001
eˆ = (100000)
010000 010 The corrected vector is estimated
100000 100
Uˆ = r + eˆ = (001110) + (100000) = (101110)
010001 111

19
 Hamming codes

◼ Hamming codes
◼ Hamming codes are a subclass of linear block codes and
belong to the category of perfect codes.
◼ Hamming codes are expressed as a function of a single
integer m  2 .
Code length: n = 2m − 1
Number of information bits: k = 2m − m − 1.
Number of parity bits: n-k = m
Error correction capability: t = 1
◼ The columns of the parity-check matrix, H, consist of all
non-zero binary m-tuples.

20
 Hamming codes

⚫ Example: Systematic Hamming code (7,4)

1 0 0 0 1 1 1
H = 0 1 0 1 0 1 1 = [I 33 PT ].
0 0 1 1 1 0 1

0 1 1 1 0 0 0
1 0 1 0 1 0 0 
G= = [P I 44 ].
1 1 0 0 0 1 0
 
1 1 1 0 0 0 1

21
Other representations of the Hamming Code
⚫ Example: Systematic Hamming code (7,4)
0 1 1 1 0 0 0
1 0 1 0 1 0 0
G [P I4 4 ]
1 1 0 0 0 1 0
1 1 1 0 0 0 1

⚫ Rearrange the matrix in column order 3,2,4,1,5,6,7

1 1 1 0 0 0 0
1 0 0 1 1 0 0
G
0 1 0 1 0 1 0
1 1 0 1 0 0 1

22
 Cyclic block codes
⚫ Cyclic codes are a subclass of linear block codes.
⚫ Encoding and syndrome calculation are easily performed
using feedback shift-registers.
– Hence, relatively long block codes can be implemented with a
reasonable complexity.
⚫ BCH and Reed-Solomon codes are cyclic codes.

23
 Cyclic block codes

◼ A linear (n,k) code is called a Cyclic code if all cyclic shifts

of a codeword are also codewords.

U = (u0 , u1, u2 ,..., un −1) “i” cyclic shifts of U

U (i )
= (un −i , un −i +1,..., un −1, u0 , u1, u2 ,..., un −i −1).
◼ Example:

U = (1101)
U(1) = (1110) U(2) = (0111) U(3) = (1011) U(4) = (1101) = U.

24
 Cyclic block codes

⚫ Algebraic structure of Cyclic codes, implies expressing

codewords in polynomial form

U( X ) = u0 + u1 X + u2 X 2 + ... + un−1X n−1 degree (n - 1).

⚫ Relationship between a codeword and its cyclic shifts:
XU( X ) = u0 X + u1 X 2 + ..., un − 2 X n −1 + un −1 X n
= un −1 + u0 X + u1 X 2 + ... + un − 2 X n −1 + un −1 X n + un −1 .
U(1) ( X ) un −1 ( X n +1)

= U(1) ( X ) + un −1 ( X n + 1)

– Hence: U(1) ( X ) = XU( X ) modulo ( X n + 1)

By extension
U(i ) ( X ) = X i U( X ) modulo ( X n + 1)
25
 Cyclic block codes
⚫ Basic properties of Cyclic codes:
– Let C be a binary (n,k) linear cyclic code
1. Within the set of code polynomials in C, there is a
unique monic polynomial g( X )with minimal degree
r  n. g ( X ) is called the generator polynomials.
g( X ) = g0 + g1 X + ... + g r X r
2. Every code polynomial U( X ) in C, can be expressed
uniquely as U( X ) = m( X )g( X )

3. The generator polynomial g( X ) is a factor of

X n +1

26
 Cyclic block codes

The orthogonality of G and H in polynomial form is

expressed as g( X )h( X ) = X + 1 . This means h( X ) is
n
also a factor of X n + 1

1. The row i, i = 1,..., k , of generator matrix is formed by the

coefficients of the "i − 1" cyclic shift of the generator
polynomial.

 g0 g1 gr 0
 g ( X )   
 Xg( X )   g0 g1 gr 
G= = 
   
 k −1   g0 g1 gr 
 X g( X )  
0 g0 g1 g r 

27
 Cyclic block codes

⚫ Systematic encoding algorithm for an (n,k) Cyclic code:

1. Multiply the message polynomial m( X ) by X n − k

2. Divide the result of Step 1 by the generator

polynomial g( X ) . Let p( X ) be the reminder.

n−k
3. Add p ( X ) to X m( X ) to form the codeword
U( X ).

28
 Cyclic block codes

⚫ Example: For the systematic (7,4) Cyclic code with

generator polynomial g ( X ) = 1 + X + X 3.
1. Find the codeword for the message m = (1011).
n = 7, k = 4, n − k = 3
m = (1011)  m( X ) = 1 + X 2 + X 3
X n − k m( X ) = X 3m( X ) = X 3 (1 + X 2 + X 3 ) = X 3 + X 5 + X 6 .
Divide X n − k m( X ) by g( X) :
X 3 + X 5 + X 6 = (1 + X + X 2 + X 3 ) (1 + X + X 3 ) + 1
quotient q(X) generator g(X) remainder p ( X )
Form the codeword polynomial:
U(X ) = p(X ) + X 3m( X ) = 1 + X 3 + X 5 + X 6
U=(1 0 0 1 0 1 1)
parity bits message bits

29
 Cyclic block codes
– Find the generator and parity check matrices, G and H,
respectively.
g( X ) = 1 + 1  X + 0  X 2 + 1  X 3  ( g 0 , g1, g 2 , g3 ) = (1101).
1 1 0 1 0 0 0
0 1 1 0 1 0 0 
Not in systematic form.
G= We do the following:
0 0 1 1 0 1 0 row(1) + row(3) → row(3)
 
0 0 0 1 1 0 1 row(1) + row(2) + row(4) → row(4)
1 1 0 1 0 0 0
0 1 1 0 1 0 0  1 0 0 1 0 1 1 
G=
1 1 1 0 0 1 0
. H = 0 1 0 1 1 1 0  .
  0 0 1 0 1 1 1 
1 0 1 0 0 0 1
P I 4 4 I 33 PT

30
 Cyclic block codes

◼ Syndrome decoding for Cyclic codes:

◼ Received codeword in polynomial form is given by

Received r ( X ) = U ( X ) + e( X ) Error
codeword pattern

◼ The syndrome is the remainder obtained by dividing the

received polynomial by the generator polynomial.
r ( X ) = q( X )g( X ) + S( X ). Syndrome

◼ With syndrome.
S( X ) = 0 : no error
S( X )  0 : error , retransmission
31
 Example of the block codes

PB
8PSK

QPSK

Eb / N0 [dB]
32