Scribd
Upload
70 views18 pages

Lesson 15 - Information Theory

This document provides an overview of various coding techniques and information theory concepts used in communication systems, including: - Line coding, channel coding, source coding, and cryptographic coding techniques - Linear block codes, convolutional codes, and turbo codes - Shannon's information theory and the channel capacity formula - Source coding methods like Huffman coding - Intersymbol interference in baseband pulse transmission and eye patterns It includes examples and references key concepts in coding theory and information transmission.

Uploaded by

Mafe Lopez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
70 views18 pages

Lesson 15 - Information Theory

This document provides an overview of various coding techniques and information theory concepts used in communication systems, including: - Line coding, channel coding, source coding, and cryptographic coding techniques - Linear block codes, convolutional codes, and turbo codes - Shannon's information theory and the channel capacity formula - Source coding methods like Huffman coding - Intersymbol interference in baseband pulse transmission and eye patterns It includes examples and references key concepts in coding theory and information transmission.

Uploaded by

Mafe Lopez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

COMMUNICATION

SYSTEMS

DISCLAIMER
This presentation is meant for educational purposes only.
LESSON 15

Information Theory
Coding theory
• Line coding: digital baseband modulation or digital
baseband transmission method, e.g. Manchester,
NRZ, AMI.

• Channel coding: forward error correction (FEC) to find


codes which transmit quickly, contain many valid code
words and can correct or at least detect many errors.

• Source coding: lossless and lossy data compression


algorithms.
•
• Cryptographic coding: algorithms for performing
encryption or decryption.
[http://en.wikipedia.org/]
Channel coding
Linear codes: algebraic coding theory
• Properties
• Code word length
• Total number of valid code words
• The minimum distance between two valid code words
• Classification
❖ Block codes
✓ Cyclic codes (e.g., Hamming codes)
✓ Repetition codes
✓ Parity codes
✓ Polynomial codes (e.g., BCH codes)
✓ Reed–Solomon codes
✓ Algebraic geometric codes
✓ Reed–Muller codes
✓ Perfect codes.

❖ Convolutional codes
❖ Turbo codes

[http://en.wikipedia.org/]
4/12/2017 5

Information Theory
• Shannon  1948: “A Mathematical Theory of
Communication”

• Information: 𝐼𝑥 = −log 𝑏 𝑃(𝑥)

• Average: 𝐼ഥ𝑥 = σ𝑖 𝐿 𝑥𝑖 𝑃 𝑥𝑖

• Entropy: 𝐻𝑥 = − σ𝑖 𝑃(𝑥𝑖 ) log 𝑏 𝑃(𝑥𝑖 )


Source coding
Lossless data compression algorithms
• Run-length encoding (RLE)
• Lempel-Ziv (LZ)
• Lempel-Ziv-Welch (LZW): GIF format
• DEFLATE: PKZIP, Gzip, PNG.
• Lempel-Ziv-Renau: Zip
Entropy encoding
• Arithmetic algorithm
• Huffman coding: optimal prefix code

[http://en.wikipedia.org/]
4/12/2017 7

Example: Huffman coding

Source: https://en.wikipedia.org/wiki/Huffman_coding
4/12/2017 8

Example: Huffman coding

Source: https://en.wikipedia.org/wiki/Huffman_coding
4/12/2017 9

Example: Huffman coding


Char Freq Code
space 7
a 4
e 4
f 3
h 2
i 2
m 2
n 2
s 2
t 2
l 1
o 1
p 1
r 1
u 1
x 1
Source: https://en.wikipedia.org/wiki/Huffman_coding
4/12/2017 10

Example: Huffman coding


Char Freq Code
space 7 111
a 4 010
e 4 000
f 3 1101
h 2 1010
i 2 1000
m 2 0111
n 2 0010
s 2 1011
t 2 0110
l 1 11001
o 1 00110
p 1 10011
r 1 11000
u 1 00111
x 1 10010
Source: https://en.wikipedia.org/wiki/Huffman_coding
Channel capacity
• Shannon’s Theorem: “For any given degree of noise
contamination of a communication channel, it is
possible to communicate discrete data
(digital information) nearly error-free up to a
computable maximum rate through the channel.”

Example: an additive white Gaussian noise (AWGN)


channel with B Hz bandwidth and signal-to-noise
ratio S/N is the Shannon–Hartley theorem:
𝑆
𝐶 = 𝐵 log 2 1+
𝑁
[http://en.wikipedia.org/]
Baseband pulse transmission
Matched filter

The matched filter is the linear filter, ℎ, that maximizes the


output signal-to-noise ratio.

∗ℎ 𝑡 =

𝑡
ℎ 𝑡 =Π [http://en.wikipedia.org/]
𝑇
Baseband pulse transmission
Intersymbol Interference (ISI)
Form of distortion of a signal in which one symbol
interferes with subsequent symbols.

• Multipath propagation
• Bandlimited channels.

Transmit data
SLICE LEVEL
Receive signal
ERRORED
“ZERO” ERRORED
“ONE”
Receive data

[http://en.wikipedia.org/]
Baseband pulse transmission
Nyquist ISI criterion
1; 𝑛 = 0
ℎ 𝑛𝑇𝑠 = ቊ
0; 𝑛 ≠ 0

[http://en.wikipedia.org/]
Baseband pulse transmission
Eye patterns

[http://en.wikipedia.org/]
Baseband pulse transmission
Eye patterns

[http://www.edn.com]
Baseband pulse transmission
Eye patterns
• The width of the eye opening defines the time interval over
which the received wave can be sampled without error from
ISI. It is apparent that the preferred time for sampling is the
instant of time at which the eye is open widest.
• The sensitivity of the system to timing error is determined by
the rate of closure of the eye as the sampling time is varied.
• The height of the eye opening, at a specified sampling time,
defines the margin over noise.

[http://en.wikipedia.org/]
4/12/2017 18

References
[Wikipedia] Wikipedia in English, the free Encyclopedia.
Visited on Jan. 2014.
[Frenzel, 2007] Frenzel, Louis E. Principles of Electronic
Communication Systems”, 3rd Ed. McGraw-Hill, 2007.
[Haykin, 2001] Haykin, Simon. Communication Systems.
4th Ed. Wiley, 2001.
[FCC]. www.fcc.gov. Propagation characterization

You might also like