EEE530 Advanced Digital

Communications
Lecture 2
Source and Channel Coding

1
Learning Objectives
• Understand and study the main elements of Digital communication systems
• Understand and apply the source and channel coding techniques

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Error Detection

21
Error Detection
• The objective is to detect transmission errors
• With a possibility to correct errors
• Error detection not 100% reliable!
• protocol may miss some errors, but rarely
• Frames are transmitted as follow

• EDC= Error Detection and Correction bits (redundancy) Appended to the message
• D = Data protected by error checking, may include header fields

NOTE: larger EDC field yields better detection and correction

22
Error Detection and Correction

23
Parity Check – Single bit parity
• Single bit parity: detects single bit error
• Parity bit can be:
• Even: the number of ones, including parity bit should be even
Example: 01110110001 → even parity bit is 0
11110011010 → even parity bit is 1
• Odd: the number of ones, including parity bit should be odd
Example: 01110110001 → odd parity bit is 1
11110011010 → odd parity bit is 0
Problems with parity bit. If two bits flip, error is not detected
Even parity
0
Original message 01110110001 received message 01 101100010 → error
Original message 01110110001 received message 010101000010 → No error

24
Single bit parity design

25
Hamming Distance
We define:
• Hamming distance: d(x, y) between two words x and y (of the same
size) is the number of differences between the corresponding bits
(XOR)
Example: d(000, 011) = 2 since 000 ⊕ 011 = 011 (two 1s)
d(10101, 11110) = 3 since 10101 ⊕ 11110 = 01011 (three 1s)

• Minimum Hamming distance: is the smallest Hamming distance

between all possible pairs in a set of words
Example: Find the minimum Hamming distance of the words in the
00000 following table
01011
10101 Solution: dmin = 3
11110
26
Hamming Distance
• To guarantee the detection of up to s errors in all cases, the
minimum Hamming distance in a block code must be dmin = s + 1.
• To guarantee correction of up to t errors in all cases, the minimum
Hamming distance in a block code must be dmin = 2t + 1.
Example: A code has a Hamming distance dmin = 4. What is the error detection and
correction capability the code?
Solution: This code guarantees the detection of up to three errors (4
= s + 1 → s = 3), but it can correct up to one error (4 = 2t + 1 → t = 3/1
= 1).

27
Block Coding
In block coding, we divide our message into blocks, each of k bits, called datawords. We add r
redundant bits to each block to make the length
n=k+r

The resulting n-bit blocks are called codewords.

• With k bits, we can create a combination of 2K datawords
• With n bits, we can create a combination of 2n codewords

Since n > k, the number of possible codewords is larger than the number of possible datawords.

The block coding process is one-to-one; the same dataword is always encoded as the same
codeword. This means that we have 2n− 2K codewords that are not used. We call these codewords
invalid or illegal.

28
Block Coding
Error detection

Error correction

29
Linear Block Coding
• In a linear block code, the exclusive OR (XOR) of any two valid codewords creates another valid
codeword
• The minimum Hamming distance is the number of 1s in the nonzero valid codeword with the
smallest number of 1s
Example: A simple parity-check code is a single-bit error-detecting code in which n = k + 1 with
dmin =2

The Syndrome is the

addition (⊕) of the
codeword bits
Syndrome = 0
meaning that the
codeword is OK

30
Linear Block Coding

31
Cyclic Code
Cyclic codes are linear codes with one extra property, if a codeword is cyclically
shifted (rotated), the result is another codeword.

In this lecture, we will discuss a category of cyclic codes: the cyclic redundancy
check (CRC) used in networks such as LANs and WANs
Example: CRC Code with C(7, 4) n=7 k = 4
Dataword Codeword Dataword Codeword
0000 0000000 1000 1000101
0001 0001011 1001 1001110
0010 0010110 1010 1010011
0011 0011101 1011 1011000
0100 0100111 1100 1100010
0101 0101100 1101 1101001
0110 0110001 1110 1110100
32
0111 0111010 1111 1111111
Cyclic Code

33
Cyclic Code - Encoder
In the encoder, the dataword has k bits (4); the
codeword has n bits (7).

The size of the dataword is augmented by adding

n − k (3) 0s to the right-hand side of the word.

The n-bit result is fed into the generator. The

generator uses a divisor of size n − k + 1 (4),
predefined and agreed upon.

The generator divides the augmented

dataword by the divisor (modulo-2 division).

The quotient of the division is discarded;

the remainder (r2r1r0) is appended to the
34
dataword to create the codeword.
Cyclic Code - Decoder
The decoder receives the possibly corrupted
codeword.

A copy of all n bits is fed to the checker which is a

replica of the generator.

The remainder produced by the checker is a

syndrome of n − k (3) bits, which is fed to the
decision logic analyzer.

The analyzer has a simple function.

If the syndrome bits are all 0s,
the 4 leftmost bits of the codeword
are accepted as the dataword

otherwise, the 4 bits are discarded 35

CRC Encoder – Example

36
CRC Decoder – Example

37
Using polynomial
A binary number can be converted to a polynomial as follow:
If there is a 1 in the position i of the binary number add xi-1 to the polynomial

Example: 110101 → x0 + x2 + x4 + x5 = x5 + x4 + x2 + 1
1001000 → x6 + x3 Dataword in the previous example
1011 → x3 + x1 + 1 Divisor in the previous example

= 1001110
38
Standard polynomials
In a LAN
• The sender computes CRC of the message and appends
the computed CRC at the end of message
• The receiver computes CRC on the message and the
received CRC, if result is 0 message is OK

This table shows some standard polynomials used in Networking

39
Internet Checksum
Sender:
• treat segment contents as sequence of 16-bit integers
• checksum: addition (1’s complement sum) of segment contents
• sender puts checksum value into UDP checksum field

Receiver:
• compute checksum of received segment
• check if computed checksum equals checksum field value:
• NO - error detected
• YES - no error detected.

40
Substitution Ciphers
• One popular substitution
• Each letter is uniquely “cipher” for some Internet
replaced by another. posts is ROT13.
• There are 26! possible
substitution ciphers.
• There are more than
4.03 x 1026 such ciphers.

Public domain image from http://en.wikipedia.org/wiki/File:ROT13.png

41
Cipher Coding

42
Classical Substitution Ciphers
• Letters of plaintext are replaced by other letters or by numbers or
symbols
• or if plaintext is viewed as a sequence of bits, then substitution
involves replacing plaintext bit patterns with ciphertext bit patterns

43
Caesar Cipher
• earliest known substitution cipher
• by Julius Caesar
• first attested use in military affairs
• replaces each letter by 3rd letter to the right
• example:
meet me after the toga party
PHHW PH DIWHU WKH WRJD SDUWB

44
Caesar Cipher

• can define transformation as:

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
D E F G H I J K L M N O P Q R S T U V W X Y Z A B C

• mathematically give each letter a number

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

• then have Caesar cipher as:

c = E(p) = (p + k) mod (26)
p = D(c) = (c – k) mod (26)

45
Cryptanalysis of Caesar Cipher
• only have 26 possible ciphers
• A maps to A,B,..Z
• could simply try each in turn
• a brute force search
• given ciphertext, just try all shifts of letters
• Easy to recognize when we have a plaintext
• eg. break ciphertext "GCUA VQ DTGCM"

Possible answers: to, of, in, an, on

46
Monoalphabetic Cipher
• rather than just shifting the alphabet
• could shuffle (jumble) the letters arbitrarily
• each plaintext letter maps to a different random ciphertext letter
• hence key is 26 letters long

Plain: abcdefghijklmnopqrstuvwxyz
Cipher: DKVQFIBJWPESCXHTMYAUOLRGZN

Plaintext: ifwewishtoreplaceletters
Ciphertext: WIRFRWAJUHYFTSDVFSFUUFYA

47
Monoalphabetic Cipher Security
• now have a total of 26! = 4 x 10^25 keys
• with so many keys, might think is secure
• but would be !!!WRONG!!!
• problem is language characteristics: Letters like E and I appear more
than others!!!

48
Language Redundancy and Cryptanalysis
• human languages are redundant
• eg "th lrd s m shphrd shll nt wnt"
• letters are not equally commonly used
• in English E is by far the most common letter
• followed by T,R,N,I,O,A,S
• other letters like Z,J,K,Q,X are fairly rare
• have tables of single, double & triple letter frequencies for various
languages

49
English Letter Frequencies

50
Playfair Cipher
• not even the large number of keys in a monoalphabetic cipher
provides security
• one approach to improving security was to encrypt multiple letters
• the Playfair Cipher is an example
• invented by Charles Wheatstone in 1854, but named after his friend
Baron Playfair

51
Playfair Key Matrix
• a 5X5 matrix of letters based on a keyword
• fill in letters of key (without duplicates)
• fill rest of matrix with other letters in alphabetical
order
• eg. using the keyword MONARCHY
M O N A R
C H Y B D
E F G I/J K
L P Q S T
U V W X Z
52
Encrypting and Decrypting
• plaintext is encrypted two letters at a time
1. if a pair is a repeated letter, insert filler like 'X’
2. if both letters fall in the same row, replace each
with letter to right (wrapping back to start from
end)
3. if both letters fall in the same column, replace
each with the letter below it (again wrapping to
top from bottom)
4. otherwise each letter is replaced by the letter in
the same row and in the column of the other letter
of the pair

53
 Playfair Example
• Consider the key: “playfairExample”
• We wish to encrypt the sentence: Hide the gold in
the tree stump
• Step 1: construct the playfair matrix

P L A Y F • Fill key first

• No duplicates
I/J R E X M • I/J together
• Then fill rest in
B C D G H alphabetical
order
K N O Q S
T U V W Z 54
 Playfair Example
• Step 2: Hide the gold in the tree stump
• Hi de th eg ol di nt he tr ex es tu mp
• Notice the X inserted between the 2 e of “tree”
• Then take each 2 letters and located them in the playfair
matrix. Check if same row, same column or not.
• Look next!

55
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

56
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

57
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

58
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

NOT all the letters of the plaintext are shown!!!!!

You can complete this at home! 59
Security of Playfair Cipher
• security much improved over monoalphabetic
• since have 26 x 26 = 676 digrams
• would need a 676 entry frequency table to analyze (versus 26 for a
monoalphabetic)
• and correspondingly more ciphertext
• was widely used for many years
• eg. by US & British military in WW1
• it can be broken, given a few hundred letters
• since still has much of plaintext structure
60
Substitution Ciphers
• One popular substitution
• Each letter is uniquely “cipher” for some Internet
replaced by another. posts is ROT13.
• There are 26! possible
substitution ciphers.
• There are more than
4.03 x 1026 such ciphers.

Public domain image from http://en.wikipedia.org/wiki/File:ROT13.png

61
Classical Substitution Ciphers
• Letters of plaintext are replaced by other letters or by numbers or
symbols
• or if plaintext is viewed as a sequence of bits, then substitution
involves replacing plaintext bit patterns with ciphertext bit patterns

62
Caesar Cipher
• earliest known substitution cipher
• by Julius Caesar
• first attested use in military affairs
• replaces each letter by 3rd letter to the right
• example:
meet me after the toga party
PHHW PH DIWHU WKH WRJD SDUWB

63
Caesar Cipher

• can define transformation as:

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
D E F G H I J K L M N O P Q R S T U V W X Y Z A B C

• mathematically give each letter a number

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

• then have Caesar cipher as:

c = E(p) = (p + k) mod (26)
p = D(c) = (c – k) mod (26)

64
Cryptanalysis of Caesar Cipher
• only have 26 possible ciphers
• A maps to A,B,..Z
• could simply try each in turn
• a brute force search
• given ciphertext, just try all shifts of letters
• Easy to recognize when we have a plaintext
• eg. break ciphertext "GCUA VQ DTGCM"

Possible answers: to, of, in, an, on

65
Monoalphabetic Cipher
• rather than just shifting the alphabet
• could shuffle (jumble) the letters arbitrarily
• each plaintext letter maps to a different random ciphertext letter
• hence key is 26 letters long

Plain: abcdefghijklmnopqrstuvwxyz
Cipher: DKVQFIBJWPESCXHTMYAUOLRGZN

Plaintext: ifwewishtoreplaceletters
Ciphertext: WIRFRWAJUHYFTSDVFSFUUFYA

66
Monoalphabetic Cipher Security
• now have a total of 26! = 4 x 10^25 keys
• with so many keys, might think is secure
• but would be !!!WRONG!!!
• problem is language characteristics: Letters like E and I appear more
than others!!!

67
Language Redundancy and Cryptanalysis
• human languages are redundant
• eg "th lrd s m shphrd shll nt wnt"
• letters are not equally commonly used
• in English E is by far the most common letter
• followed by T,R,N,I,O,A,S
• other letters like Z,J,K,Q,X are fairly rare
• have tables of single, double & triple letter frequencies for various
languages

68
English Letter Frequencies

69
Playfair Cipher
• not even the large number of keys in a monoalphabetic cipher
provides security
• one approach to improving security was to encrypt multiple letters
• the Playfair Cipher is an example
• invented by Charles Wheatstone in 1854, but named after his friend
Baron Playfair

70
Playfair Key Matrix
• a 5X5 matrix of letters based on a keyword
• fill in letters of key (without duplicates)
• fill rest of matrix with other letters in alphabetical
order
• eg. using the keyword MONARCHY
M O N A R
C H Y B D
E F G I/J K
L P Q S T
U V W X Z
71
Encrypting and Decrypting
• plaintext is encrypted two letters at a time
1. if a pair is a repeated letter, insert filler like 'X’
2. if both letters fall in the same row, replace each
with letter to right (wrapping back to start from
end)
3. if both letters fall in the same column, replace
each with the letter below it (again wrapping to
top from bottom)
4. otherwise each letter is replaced by the letter in
the same row and in the column of the other letter
of the pair

72
 Playfair Example
• Consider the key: “playfairExample”
• We wish to encrypt the sentence: Hide the gold in
the tree stump
• Step 1: construct the playfair matrix

P L A Y F • Fill key first

• No duplicates
I/J R E X M • I/J together
• Then fill rest in
B C D G H alphabetical
order
K N O Q S
T U V W Z 73
 Playfair Example
• Step 2: Hide the gold in the tree stump
• Hi de th eg ol di nt he tr ex es tu mp
• Notice the X inserted between the 2 e of “tree”
• Then take each 2 letters and located them in the playfair
matrix. Check if same row, same column or not.
• Look next!

74
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

75
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

76
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

77
 Playfair Example
• Step 3:Hi de th eg ol di nt he tr ex es tu mp

NOT all the letters of the plaintext are shown!!!!!

You can complete this at home! 78
Conclusion
• In this chapter, we studied different source and channel coding
techniques

79