3/4/2018

EE140 Introduction to
Communication Systems
Lecture 24

Instructor: Prof. Xiliang Luo

ShanghaiTech University, Spring 2018

1

Architecture of a (Digital) Communication System

Transmitter

A/D Source Channel

Source Modulator
converter encoder encoder

Absent if
source is Noise Channel
digital

D/A Source Channel

User Detector
converter decoder decoder

Receiver

1
 3/4/2018

Contents

• Linear block code introduction

• Hamming codes

• Linear block codes

• Decoding of LBC

• Cyclic code, BCH code, R-S code

Linear Block Code

• : A block code of block length over an
alphabet χ is a non-empty set of -tuples of
symbols from χ.
– ,..., ,..., ,...,
– The -tuples of the code are called codewords.
• Rate:
– channel alphabet: symbols
– information symbols: tuples
– number or codewords:
– code length:
– rate
1
log
– , code
4

2
 3/4/2018

Block Codes: Examples

• 00010110
– Block length 8, 1, rate 1⁄8 log 1 0.
– Codes with rate 0 are called useless code.
– Could be used for error rate analysis or byte sync.

• 00, 01, 10, 11

– Block length 2, 4, rate 1⁄2 log 4 1.
– No redundancy. Can neither correct nor detect errors.

• 000, 111
– Block length 3, 2, rate 1⁄3 log 2 0.333.
– Repetition code, can correct one bit error.

Hamming Distance
• The Hamming distance between -tuples is the
number of components in which the n-tuples differ
1, if
, ∑ , , where ,
0, if
– , 0 with equality if and only if (nonnegativity)
– , , (symmetry)
– , , , (triangle inequality)
– Hamming distance is a coarse or pessimistic measure of
difference.
• Other useful distances in error control coding:
– , distance on a circle, is applicable to phase
shift coding.
– , is used with sensewords in .
6

3
 3/4/2018

Minimum Distance
• The minimum (Hamming) distance ∗ of a block
code is the minimum distance between any two
codewords:
∗ min , ∶ , are codewords and
• Properties of minimum distance:
– ∗ 1 since Hamming distance between distinct codewords
is a positive integer.
– ∗ if code has two or more codewords.
– ∗ 1 or ∗ ∞ for the useless code with only one
codeword. (This is a convention, not a theorem.)
– ∗ ∗ if ⊆ —smaller codes have larger (or
equal) minimum distance.
• The minimum distance of a code determines both
error-detecting ability and error-correcting ability.
7

Error-detecting Ability
• Suppose that a block code is used for error
detection only.
– If the received -tuple is not a codeword, a detectable error
has occurred
– If the received -tuple is a codeword but not the
transmitted codeword, an error has occurred that cannot be
detected.

• : The guaranteed error-detecting ability is

∗ 1.

• The error-detecting ability is a worst case measure

of the code.

4
 3/4/2018

Error-correcting Ability
• We may think of a block code as a set of vectors
in an -dimensional space. The geometric intuition:

• Nearest-neighbor decoding
argmin , ∶ is a codeword
9

Error-correcting Ability (cont’d)

• : Using nearest-neighbor decoding, errors of
weight t can be corrected if and only if 2 ∗ . (For
Hamming distance, equivalently, 2 1 ∗ ).

• : The “spheres” of radius surrounding the

codewords do not overlap. Otherwise, there would
be two codewords 2 distant. Therefore when
errors occur, the decoder can unambiguously decide
which codeword was sent.

10

5
 3/4/2018

Decoding Outcomes
• A particular block code can have more than one
decoder, depending on
– the type of errors expected or observed
– the error rate
– the computational power available at the decoder
• Suppose that codeword is transmitted, senseword
is received, and ̂ is the decoder’s output. possible
outcomes:
– ̂ decoder success Successful correction
(including no errors)
– ̂ ? decoder failure Uncorrectable error detected,
no decision (not too bad)
– ̂ decoder error Miscorrection (very bad)

11

Decoding Outcomes (cont’d)

• The decoder cannot distinguish the outcome ̂
from ̂ .
• However, it can assign probabilities to the
possibilities; more bit errors corrected suggests a
higher probability that the estimate is wrong.
– ̂ ? Pued Error detected but not corrected (decoder
failure)
– ̂ Pmc Miscorrection (decoder error)
– ̂ Pue Undetectable error (error detection only)
• : A complete decoder is a decoder that
decodes every senseword to some codeword; i.e.,
the decoder never fails (to make hard decisions).
– For a complete decoder, Pued 0.

12

6
 3/4/2018

Decoding Outcomes (cont’d)

• : A bounded-distance decoder corrects all
errors of weight but no errors of weight .
More than errors results in failure or error.
• For a fixed code, if we reduce then decoder failure
becomes more common while decoder error
becomes less likely.

13

Example: Repetition Code

• Transmit a single information symbol times.
• For the binary alphabet, there are two codewords,
i.e., 00 00, 11 11 .
– Rate: 1/
– Minimum distance: ∗
– Error detection: up to 1 symbol errors
– P{undetected error} =
– Error correction: up to 1 /2 symbol errors
– P{miscorrection} = P{more than n/2 errors} (for binary
alphabet)
• For binary alphabet, any two complementary
codewords could be used.
– Example: ′ 01010101, 10101010 55, .

14

7
 3/4/2018

Example: Parity-check Codes

• Append one check bit to data bits so that all
codewords have the same overall parity - either
even or odd.
• Even-parity codewords are defined by a single
parity-check equation:
⊕ ⊕ ⊕ mod 2 0,
where ⊕ denotes the exclusive-or operation.
• encoder
⊕ ⊕ ⊕
• Note: Any bit can be considered to be the check
bit because it can be computed from the other 1
bits: ∑

15

Example: Product Codes

• Arrange data bits in a two-dimensional array.
Append parity check bits at the end of each row and
column.

• Error detection and correction

– single error: failure of one row equation and one column
equation.
– Double error: can be detected—two rows or two columns
(or both) have the wrong parity—but cannot be corrected.
– Exercise: what fraction of errors is not detected?
16

8
 3/4/2018

Contents

• Linear block code introduction

• Hamming codes

• Linear block codes

• Decoding of LBC

• Cyclic code, BCH code, R-S code

17

Hamming Codes
• Parity check is simple but inefficient:
– a failed parity-check equation locates row or column of
error
– however, a satisfied equation gives little information
• Can we locate the error?
• (7, 4) Hamming parity check code (encoding rules)

18

9
 3/4/2018

Parity Check Equations

• Parity check equations
,
,
,
• Codewords , , , , , ,
0000 000 0100 011 1000 110 1100 101

0001 101 0101 110 1001 011 1101 000

0010 111 0110 100 1010 001 1110 010

0011 010 0111 001 1011 100 1111 111

• Codeword is uniquely determined from the

information bits.
19

Properties of Hamming Codes

• Generator matrix G and parity check matrix

1 0 0 0 Systematic
0 1 0 0
 
0 0 1 0 0 0 0 1 1 1 1
 
G  0 0 0 1 , H  0 1 1 0 0 1 1
 
0 1 1 1 1 0 1 0 1 0 1
 
1 0 1 1
1 1 0 1 

• Parity check
0

20

10
 3/4/2018

Properties of Hamming Codes (cont’d)

• Codeword can be generated from the generator
matrix
1 0 0 0
0 1 0 0
   u0 
0 0 1 0  
  u1
c  Gu  0 0 0 1  
u2 
0 1 1 1  
  u3
1 0 1 1  
1 1 0 1

• The summation of two codewords is still a valid

codeword
• Parity check matrix 0
21

Properties of Hamming Codes (cont’d)

• Minimum distance 3
• Decoding algorithm: nearest neighborhood
• (7, 4) Hamming code:
– Detect 1 2 bits of error
– Correct 1 bit of error
• General Hamming code:
– 2 1, 2
– 2 1
– Example: 4, (15, 11) Hamming code
– 3

– Coding efficiency:

22

11
 3/4/2018

Exercise
• Given generator matrix G,
– Codewords
– Minimum distance
– Error control ability

1 0 0
0 1 0
 
0 0 1
G 
1 0 1
0 1 1
 
1 1 0
23

Exercise (cont’d)
• Extended Hamming codes: add an overall parity-
check bit
,
– Codewords
– Minimum distance
– Error control ability

24

12
 3/4/2018

Contents

• Linear block code introduction

• Hamming codes

• Linear block codes

• Decoding of LBC

• Cyclic code, BCH code, R-S code

25

Minimum Weight
• The Hamming weight is the number of
nonzero components of .
• Facts:
– 0,
– 1, 2 1 2 2 1
– 0 if and only if 0
• Definition: The minimum (Hamming) weight of a
block code is the weight of the nonzero codeword
with smallest weight:
∗
min min ∶ ∈ , 0
• Examples of minimum weight:
– Simple parity-check codes: ∗ 2.
– Repetition codes: ∗ .
– (7,4) Hamming code: ∗ 3. 26

13
 3/4/2018

Minimum Distance = Minimum Weight

• Theorem: For every linear block code, ∗ ∗.

Proof : We show that ∗ ∗

and ∗ ∗
.
– (≥) Let be a nonzero minimum-weight codeword. the 0
vector is a codeword, so
∗ ∗
0,
– (≤) Let be two closest codewords. Then is a
nonzero codeword, so
∗ ∗
,
Combining these two inequalities, we obtain ∗ ∗
.
• It is easier to find minimum weight than minimum
distance because the weight minimization considers
only a single parameter.

27

Generator Matrix
• Definition: A generator matrix for a linear block
code of blocklength and dimension is any
matrix whose rows form a basis for .
• Every codeword is a linear combination of the
columns of a generator matrix :
 u0 
u 
c  Gu   g 0 g1 L g k 1 nk  1 
 M
 
 uk 1 

28

14
 3/4/2018

Parity-check Matrix
• Definition: The dual code of is the orthogonal
complement C  .
• Definition: A parity-check matrix for a linear block
code is any matrix , whose rows span the
orthogonal complement C . Obviously, .
• Example: and for 5, 4 simple parity-check
code.
1 0 0 0
1 1 0 0
 
G  0 1 1 0 , H  1 1 1 1 1
 
0 0 1 1
0 0 0 1 
– is generator matrix for 5, 1 repetition code - the dual code
29

Example
• 7, 4 Hamming code

1 0 0 0
0 1 0 0
 
0 0 1 0 0 0 0 1 1 1 1
 
G  0 0 0 1 , H  0 1 1 0 0 1 1
 
0 1 1 1 1 0 1 0 1 0 1
 
1 0 1 1
1 1 0 1 

30

15
 3/4/2018

Codewords are Defined by

• Theorem: If is an , linear block code with
parity-check matrix , then an -tuple is a
codeword if and only if 0.

31

Generator vs. Parity-check Matrices

• Usually we choose to consist of independent
rows, so is .
– May exist with redundant rows (for example, binary BCH
codes).
• Each row of corresponds to an equation satisfied
by all codewords.
• Since each column of is a codeword, for any
parity-check matrix ,
0
– Conversely, if 0 and then is a
parity-check matrix.
• How do we find from ?
– We could find from by finding linearly
independent solutions of the linear equation 0.
32

16
 3/4/2018

Contents

• Linear block code introduction

• Hamming codes

• Linear block codes

• Decoding of LBC

• Cyclic code, BCH code, R-S code

33

Syndrome (校正子) Decoding

• Encoding for LBC is vector-matrix multiplication.
• Maximum-likelihood decoding: maximize |
• Decoding is inherently nonlinear. Fact: linear
decoders are very weak.
• However, several steps in the decoding process are
linear:
– syndrome computation
– correction after error pattern and location have been found
– extracting estimated message from estimated codeword
• Definition: The error vector or error pattern is the
difference between the received -tuple and the
transmitted codeword :
≜ ⇒
34

17
 3/4/2018

Syndrome Decoding (cont’d)

• Multiply both sides of the equation by :
≜ 0
– The syndrome of the senseword is defined to be .
– The syndrome of (known to receiver) equals the
syndrome of the error pattern e (not known to receiver but
must be estimated).
• Decoding consists of finding the most plausible (貌
似真实的) error pattern such that

• “Plausible” depends on the error characteristics:
– For binary symmetric channel, most plausible means
smallest number of bit errors. Decoder picks error pattern
of smallest weight satisfying .
– For bursty channels, error patterns are plausible if the
symbol errors are close together.
35

Syndrome Decoding (cont’d)

• Syndrome table decoding:
1. Calculate syndrome of received -tuple.
2. Find most plausible error pattern with .
3. Estimate transmitted codeword: ̂ .
4. Determine message from the encoding equation ̂ .
• Only step 2 requires nonlinear operations.
• For small values of , lookup tables can be used
for step 2.
• Step 4 is not needed for systematic encoders, since
̂ 1: .

36

18
 3/4/2018

Syndrome Decoding -- Concept

• Decoding region

37

Example

38

19
 3/4/2018

Contents

• Linear block code introduction

• Hamming codes

• Linear block codes

• Decoding of LBC

• Example LBC
– Cyclic code,

– BCH code,

– R-S code

39

Polynomial Description of Cyclic Shifts

• Cyclic shift:
– Original:
– Shifted version:

• Lemma: The right cyclic shift of is

1
mod 1
Proof:

• When 0, the right cyclic shift is same as the

right logical shift:
0 ⇔

40

20
 3/4/2018

Example
• Example generator matrix

41

Basic Properties
• By iteration, all cyclic shifts of codewords are
codewords; thus the right cyclic shift positions of
any codeword c(x) is a codeword:
mod 1
• Since cyclic codes are linear, sums of cyclic shifts
are codewords; thus for any polynomial
,

mod 1 mod 1

is a codeword.

42

21
 3/4/2018

Examples of Binary Cyclic Codes

• Even-parity simple parity-check codes are cyclic:
1 1

• Repetition codes have generator polynomial

1

43

Examples of Binary Cyclic Codes (cont’d)

• Over 2 , 1 1 1 1 .
There are 2 8 divisors of 1 and thus 8 cyclic
codes of blocklength 7. If 1, a
primitive polynomial, we obtain a cyclic Hamming
code.
– Show each codewords are actually cyclic
– Show that this code is a Hamming code

44

22
 3/4/2018

Cyclic Codes: Review

• A cyclic code is a LBC such that every cyclic shift of
a codeword is a codeword.
• A cyclic code has generator polynomial that is a
divisor of every codeword.
• The generator polynomial is a divisor of 1,
where is the blocklength.
• The parity-check polynomial is
• Codewords can be characterized by (and errors
detected by):
– mod 0
– 0 mod 1

45

Examples of Cyclic codes: CRC

• Cyclic codes are often used for error detection
• CRC-16

– Generator polynomial: 1 1 1
– Minimum distance for CRC-16, CRC-CCITT is 4. Both codes
correct single errors while detecting double errors, or detect
up to 3 errors.
• CRC-CCITT
– Generator polynomial: 1 1 ⋯
⋯ 1
46

23
 3/4/2018

Contents

• Hamming codes

• Linear block codes

• Decoding of LBC

• Example LBC
– Cyclic code,

– BCH code,

– R-S code

47

BCH Introduction
• BCH codes, which are cyclic code, were first
discovered by A. Hocquenghem in 1959 and
independently by R. C. Bose and D. K. Chaudhuri in
1960.

• The original applications of BCH codes were

restricted to binary codes of length 2 1 for some
integer . These were extended later by Gorenstein
and Zieler (1961) to the nonbinary codes with
symbols from Galois field .

48

24
 3/4/2018

BCH Introduction
• The code is designed according to the error-
correcting ability, which is different from the linear
block codes discussed before.

• For any positive integers m, and t with m ≥3,

t<2m-1, there exists a BCH code of length 2m-1.

• Thus, the minimum distance d 2 1 , and can

correct t random errors

49

Example I
• GF(24) generated by the primitive polynomial
1 over GF(2).
• Correct 1 bit error 1, 15.
• Generate polynomial
,
1, 1
1
• Degree of 4, so we have generate
polynomil for 1 bit error correcting 15,11 .
• Hamming distance of codewords 2 1 3.

50

25
 3/4/2018

Example II
• GF(24) generated by the primitive polynomial
1 over GF(2).
• Correct 2 bit error 2, 15.
• Generate polynomial
, , ,
1, 1,
1, 1
1
• Degree of 8, so we have generate
polynomil for 2 bit error correcting 15,7 .
• Hamming distance of codewords 2 1 5.

51

Contents

• Hamming codes

• Linear block codes

• Decoding of LBC

• Example LBC
– Cyclic code,

– BCH code,

– R-S code

52

26
 3/4/2018

RS Introduction
• The RS codes are nonbinary cyclic codes with code
symbols from a Galois field. They were discovered
in 1960 by . and . . The work was
done when they were at MIT Laboratory.

• The most important RS codes are codes with

symbols from 2 .

• RS code is established based on the bytes, rather

than a single bit 1 or 0, it is a nonbinary BCH code.
It well performed to deal with burst errors.

53

RS Codes

• For any positive integer 2 1, there exists a

-symbol-error-correcting RS code with symbols
from 2 and following paramers:
– 2 1
– 2
– 2 2 1
– 2 1 1

54

27
 3/4/2018

Example
• Construct a 3, 15, 4 RS code.
– Distance: 2 1 7 28
– Parity check: 2 6 24
– Information: 6 9 36
– Length of codeword: 15 60
– This is (15,9) RS code, regard as a (60,36) binary code.
– Generate matrix:
– …
1
– is a element of 2

• The degree of in RS code is always 2 !

55

Error Performance of R-S Code

56

28
 3/4/2018

Thanks for your kind attention!

Questions?

57

29