ELEC1200: A System View of

Communications: from Signals to Packets

Lecture 13
• Review
– Channel Coding
– Error correction vs detection
– (n,1) repetition codes
• More complex block codes
– (k+1,k) parity bit code with d = 2
– (8,4) code with d = 3
– (9,4) code with d = 4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
1
 (n,k) Block Codes
• Arrange the message into k-bit blocks
• Make each codeword n bits long, by adding (n-k) bits to each block.
k n-k
The entire block is called
a “codeword”
Message bits Extra bits

• d = minimum Hamming Distance between codewords

Determines the error detection/correction capabilities of the code
• Sometimes we include d in the description and indicate (n,k,d)

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
2
 Error Detection vs Correction
• Error detection
– We try to detect errors, but don’t try to fix them.
• Error correction
– We try to correct errors.
• Note that we must choose between error detection and error
correction.

• The minimum Hamming distance, d, determines how many bit errors

we can detect or correct.
• We can EITHER
– Detect, but not correct, d-1 or fewer bit errors
OR
– Correct (d-1)/2 or fewer bit errors

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
3
 (n,1) repetition code
• Repetition codes are block codes, where
– The message is split to 1-bit blocks.
– Blocks are expanded to n bits by repeating the bit n times.
– The minimum Hamming distance between valid codewords is d=n.
• Example: (3,1) repetition code
– Two valid codewords
• Input bit 0  Codeword 000
• Input bit 1  Codeword 111
– Hamming distance d=3.
– We can EITHER
• Detect (d-1) = 2 or fewer bit errors
OR
• Correct (d-1)/2 = 1-bit errors

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT 4
 ELEC1200: A System View of
Communications: from Signals to Packets
Lecture 13
• Review
– Channel Coding
– Error correction vs detection
– (n,1) repetition codes
• More complex block codes
– (k+1,k) parity bit code with d = 2
– (8,4) code with d = 3
– (9,4) code with d = 4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
5
 More complex block codes
• We can create more complex block codes than the repetition code by
adding multiple parity bits
– Different parity bits consider different subsets of the message block.

• In this lecture, we introduce three block codes

– (k+1,k) parity bit code
• Examples: (2,1), (3,2), (4,3), (5,4) ….
• (k+1)-k = 1 parity bit added to make entire codeword have even parity.
• d = 2
– (8,4) block code
• 8-4 = 4 parity bits added,
• d = 3
– (9,4) block code
• 9-4 = 5 parity bits added,
• d = 4
ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
6
 (k+1,k) Parity Bit Block Code
• Add a single parity bit chosen • Examples (cont.)
so that the number of 1’s in the – (4,3) parity bit code
(k+1)-bit codeword is even. • 000 → 0000
• 001 → 0011
• 010 → 0101
• Examples • 011 → 0110
• 100 → 1001
– (2,1) parity bit code
• 101 → 1010
• 1 → 11
• 110 → 1100
• 0 → 00 • 111 → 1111
– (3,2) parity bit code
• 00 → 000
• 01 → 011
• 10 → 101 𝒌𝒌
code rate =
• 11 → 110 𝒌𝒌 + 𝟏𝟏

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
7
 Error Detection
• It turns out that d=2 for this code.
• We can only detect (d-1) = 1-bit errors.
– Sum the received bits. If the sum is odd, there’s been an error.

• Example: (8,7) code

parity bit 7
code rate =
8
Example 1: 0 1 1 0 0 1 0 → original 7-bit message
0 1 1 0 0 1 0 1 → codeword
0 1 1 0 0 0 0 1 → received codeword - single-bit error (detected)

Example 2: 1 0 1 1 1 0 0 → original 7-bit message

1 0 1 1 1 0 0 0 → codeword
1 0 1 1 1 1 1 0 → received codeword - 2-bit errors (undetected)

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
8
 ELEC1200: A System View of
Communications: from Signals to Packets
Lecture 13
• Review
– Channel Coding
– Error correction vs detection
– (n,1) repetition codes
• More complex block codes
– (k+1,k) parity bit code with d = 2
– (8,4) code with d = 3
– (9,4) code with d = 4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
9
 (8,4) block code D1 D2 D3 D4

• Four parity bits (P) are computed by P1 is parity bit

– Arranging the message bits (D) into four subsets, for row #1
corresponding to the rows and columns of a 2x2
square. D1 D2 P1
– Adding a parity bit to each row or column so that it
has even parity.
D3 D4 P2

D1 D2 D3 D4 P1 P2 P3 P4 P3 P4 P4 is parity bit
for column #2

• Example:
0 1 1

Message block: 0 1 1 1 Codeword: 0 1 1 1 1 0 1 0

1 1 0

1 0

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
10
 Performing Error Detection
• It turns out that d = 3 for this code.
D1 D2 D3 D4 P1 P2 P3 P4
– The proof is beyond the scope of this class.
(Step 1)

• We can detect (d-1) = 2 or fewer bit

D1 D2 P1
errors in the codeword.
1. Take the codeword and rearrange it D3 D4 P2

2. Add syndrome bits to the rows and columns P3 P4

to make them even parity
(Step 2)
• Even though the rows and columns were even
parity when they were transmitted, they may
not be in the received codeword due to bit D1 D2 P1 S1
errors.
D3 D4 P2 S2
• The receiver computes the syndrome bits to
check for these errors. P3 P4
3. If all syndrome bits are 0, there is no error. S3 S4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
11
 Examples of Error Detection
Received Rearranged Computed
Error?
Codeword Codeword Syndrome

0 1 1 0 1 1 0

no
1 1 0 0
01111010 1 1 0 1 0
1 0 0 0

0 1 1 0 1 1 0

yes
1 0 0 1
01101010 1 0 0 1 0
1 0 0 1

0 1 1 0 1 1 0

yes
1 0 1 0
01101110 1 0 1 1 0
1 0 0 1

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
12
 Performing Error Correction
• We can correct (d-1)/2 = 1-bit errors.
D1 D2 D3 D4 P1 P2 P3 P4
1. Rearrange the codeword
2. Add syndrome bits to the rows and columns to (Step 1)
make them even parity
3. Check the syndrome bits. D1 D2 P1

• If all are 0, there is no error. D3 D4 P2

• If only one syndrome bit is 1, there is an error in P3 P4

the corresponding parity bit
– e.g., S2=1 indicates error in P2 (Step 2)

• If the syndrome bit for column i is 1 and

D1 D2 P1 S1
the syndrome bit for row j is 1, then
there is an error in the data bit at i,j D3 D4 P2 S2
– e.g., S2=1 and S3=1 indicates error in D3
P3 P4

S3 S4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
13
 Examples of Error Correction
Received Rearranged Computed Corrected
Codeword Codeword Syndrome Data

0 1 1 0 1 1 0
01111010 1 1 0 0 0111
1 1 0 1 0
no error
1 0 0 0

0 1 1 0 1 1 0
01101010 1 0 0 1 0111
1 0 0
1 0 D4 incorrect
1 0 0 1

0 1 1 0 1 1 0
01111110 1 1 1 1 1 1 1 0111
1 0
1 0 0 0 P2 incorrect

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
14
 Error Correction Summary
• Take an input message stream: 011011101101

0110
• Break the message stream into k-bit blocks (e.g., k = 4).
1110
1101

01101111
• Add (n-k) parity bits to form n-bit codeword (e.g., n = 8). 11100101
11010110

01100111
• Transmit data through noisy channel and receive codewords with
some errors. 11110101
11000110
• Perform error correction.
• Extract the k=4 message bits from each corrected codeword. 0110 1110 1101
ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
15
 ELEC1200: A System View of
Communications: from Signals to Packets
Lecture 13
• Review
– Channel Coding
– Error correction vs detection
– (n,1) repetition codes
• More complex block codes
– (k+1,k) parity bit code with d = 2
– (8,4) code with d = 3
– (9,4) code with d = 4

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
16
 (9,4) block code
• Add an extra parity bit, P5, to the (8,4) code.
– P5 chosen so the 9-bit codeword has even parity.
Message D1 D2 D3 D4

• The minimum Hamming distance increases to d=4. D1 D2 P1

Add
– Proof is out of the scope of this class. parity D3 D4 P2
bits
P3 P4 P5

• With the (9,4) code, we can EITHER

– Detect errors in (d-1) = 3 or fewer bits. D1 D2 D3 D4 P1 P2 P3 P4 P5
OR Codeword
𝒅𝒅−𝟏𝟏
– Correct 1-bit errors and detect 2-bit errors: = 𝟏𝟏. 𝟓𝟓
𝟐𝟐

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
17
 Creating (9,4) codewords
• Parity bits Message D1 D2 D3 D4
– P1: parity for D1,D2
– P2: parity for D3,D4
– P3: parity for D1,D3
– P4: parity for D2,D4 D1 D2 P1
– P5: parity for D1,D2,D3,D4,P1,P2,P3,P4 Add
parity D3 D4 P2
bits
• Examples P3 P4 P5
1 0 1
– Message Block: 1010 Codeword: 101011000
1 0 1

(Add Parity Bits) 0 0 0 D1 D2 D3 D4 P1 P2 P3 P4 P5

Codeword
1 1 0
– Message Block: 1110 Codeword: 111001011
1 0 1
(Add Parity Bits) 0 1 1

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
18
 Performing Error Correction
• To correct errors, we assume at most 2 bits have errors.
• Procedure
1. Compute 5 syndrome bits by checking D1 D2 P1 S1
• First two rows for even parity (S1,S2)
D3 D4 P2 S2
• First two columns for even parity (S3,S4)
• Entire codeword for even parity (S5) P3 P4 P5
2. Check the syndrome bits.
3 4 S S S5
if all Si = 0 (i = 1,2,3,4 and 5)
no error
else
if S5 = 1, (1-bit error occurred)
Check the S1-S4 following the (8,4) code to find which bit to correct.
else a 2-bit error occurred
We can detect, but not correct, this.

• Alternatively, we can detect, but not correct, errors in 3 or fewer bits. How?

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
19
 Examples of Error Correction
Received Rearranged Computed Corrected
Codeword Codeword Syndrome Data

0 1 1 0 1 1 0
011110101 1 1 0 0 0111
1 1 0 1 0 1
1 0 1 no errors
0 0 0

0 1 1 0 1 1 0
011010101 1 0 0 1 0111
1 0 0
1 0 1 D4 incorrect
1 0 1 0 1 1

0 1 1 0 1 1 0
1 1 1 1 0111
011111101 1 1 1 P2 incorrect
1 0 1
1 0 1 0 0 1

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
20
 Examples of Error Correction
Received Rearranged Computed Corrected
Codeword Codeword Syndrome Data

0 1 1 0 1 1 0
011110101 1 1 0 0 0111
1 1 0 1 0 1
1 0 1 no errors
0 0 0

0 1 1 0 1 1 0
011111111 1 1 1 1 ????
1 1 1
1 1 1 2-bit error
1 1 1 0 1 0

The reason we cannot correct these 2-bit errors is that there is another
valid codeword that differs from the received codeword by 2-bit errors.

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
21
 Inability to correct two-bit errors
Two valid codewords differ
from the received codeword
by 2 bits. Valid
codeword

Received
codeword

Valid
codeword

ELEC 1200 22
 Summary
• We create more complex codes by adding parity bits for subsets of
the message block.

• The (8,4) code rearranges the 4 message bits into a 2x2 array and
adds a parity bit for each row/column.
• Since d=3, we can EITHER
– Detect errors in 2 or fewer bits, OR
– Correct 1-bit errors.

• The (9,4) code adds an overall parity bit to the (8,4) code.
• Since d=4, we can EITHER
– Detect errors in 3 or fewer bits, OR
– Correct 1-bit errors and detect, but not correct, 2-bit errors.

ELEC 1200 Some content taken with permission from material for the course EECS6.02 by C. Sodini, M. Perrot and H. Balakrishnan at MIT
23