Information Theory, Shannon Limit and Error Correction Codes for Terrestrial DTV Broadcasting

Bo Rong, Yiyan Wu, and Gilles Gagnon Communications Research Centre Canada ATSC BOOT CAMP Iowa Public Television-Johnston, IA Oct. 13, 2011
1

Presentation Outline
Information Theory and Shannon Limit History of Error Correction Codes (ECC) ECC Performance: BER, S/N & Eb/No Block Codes, Convolutional Codes, and Concatenated Codes Turbo Codes vs. LDPC Codes A Discussion: Data Rate vs. Robustness

Information Theory
History & Definition
History: The publication of Claude Shannons 1948 paper entitled A Mathematical Theory of Communication. Definition: Information theory is the science of compression, storage, and transmission of information. In other words, it was developed to find fundamental limits on above operations.
3

Shannon Limit
Who is Shannon
Claude Elwood Shannon (April 30, 1916 February 24, 2001) was an American mathematician, electronic engineer, and cryptographer known as "the father of information theory". He taught at MIT from 1956 until his retirement in 1978.

Shannon Limit
What is Shannon Limit
Shannon showed that any communications channel a telephone line, a radio band, a fiber-optic cable could be characterized by two factors: bandwidth and noise. Bandwidth is the range of electronic, optical or electromagnetic frequencies that can be used to transmit a signal; Noise is anything that can disturb that signal. Given a channel with particular bandwidth and noise characteristics, Shannon showed how to calculate the maximum rate at which data can be sent without error. He called that rate the channel capacity. But people always call it Shannon limit.

Shannon Limit
Channel Capacity: The maximum data rate at which the error-free communication over the channel is performed.
Channel capacity on AWGN channel (Shannon-Hartley capacity C/W [bits/s/Hz] theorem):

S C = W log 2 1 + N
W [Hz] : Bandwidth

[bits/s ]

Unattainable region

Practical region

S = Eb Rb [ Watt ] : Average received signal power N = N 0 BW [Watt] : Average noise power

SNR [dB] 6

Error Correction Schemes

There are two methods to correct the transmission errors: 1. Error Detection and Retransmission: Good for point-to-point two-way communications. Return channel and retransmission (Automatic Repeat reQuest-ARQ) needed; 2. Forward Error Correction (FEC) Addition parity bits are transmitted, which can be used to correct the transmission errors in the receiving-end. The broadcasting service are point-to-multi-point real time services. Forward Error Correction is suitable for broadcasting applications. The rest of the presentation will focus on FEC codes.
7

Error Correction Codes

There are two types of error correction codes:
1. 2. Block codes: Hamming, BCH, Reed-Solomon, Turbo, Turbo Product, LDPC. Convolutional codes and Trellis Coded Modulation (TCM: combines convolutional code and modulation);

History of Error Correction Codes

9 Source: Intel tutorial

History of Error Correction Codes

10 Source: Intel tutorial

Block Codes
A block code is any code defined with a finite codeword length:
The code rate R = k/n, where - k is the lengths of useful information bits - n is the total length codeword. If the codeword is constructed by appending parity symbols to the payload Data Field, it is called a systematic code.

Examples of block codes: BCH, Hamming, Reed-Solomon, Turbo, Turbo Product, LDPC. Essentially all iteratively-decoded codes are block codes.
11

Convolutional Codes
Convolutional codes are generated using a shift register to apply a polynomial to a stream of data:
The convolutional code can be systematic, if the data is transmitted in addition to the redundancy bits, but it often isnt.

The code above is naturally R = . By deleting selected output bits, or puncturing the code, the code rate can be increased to 2/3, 3/4, 7/8...
12

Convolutional Codes
Convolutional codes are typically decoded using Viterbi algorithm
The Viterbi decoder is a Maximum Likelihood Sequence Estimator, that estimates the encoder state using the sequence of transmitted codewords.

All paths merge

Pros: Viterbi algorithm works well under noisy and fading channel; Cons: when making a mistake, it can lose track of the sequence and generate burst errors until it reestablishes code lock.
13

Concatenated Codes
Concatenation of an inner convolutional code with an outer block code
A very common and effective coding structure as shown below:

The convolutional code works better in low SNR range, but its BER curve roll-off slowly and can have irreducible error floor in fading channel. The Reed-Solomon code has fast BER roll-off and well suited to correct the bursty output errors common with a Viterbi decoder. An interleaver can be used to spread the Viterbi output error bursts across multiple R-S codewords for burst error correction.
14

What is an Interleaver
Interleaver and de-interleaver are a pair of devices that can break up (or de-correlate) the long burst of error symbols, occurred in transmission or generated by inner decoder, so that they can be easily corrected by error correction code; The simplest interleaver is an orthogonal memory, which read-in data symbol line-by-line and read-out column by column; There are different type of interleavers, such as, block interleaver, covolutional interleaver, random interleaver, etc.

A Simple Block Interleaver

Data write-in If there are three lines of Transmission errors, the interleaver out put will only have 3 symbols in consecutive errors. It is easier for decoder to Correct them.

Data read-out

To encoder or decoder

Convolutional Interleaver in ATSC A/53

1 2
From Reed-Solomon Encoder
M(=4Bytes)

2M

To Pre-Coder and Trellis Encoder

51 (B=)52

(B-2)M (B-1)M

M=4, B=52, N=208, R-S Block =207, BXM=N

Code Performance Measurement

Bit Error Rate (BER), Symbol Error Rate (SER), Packet Error Rate (PER), or Block Error Rate; Signal to Noise Ratio (SNR) or Eb/No, where
- Eb is energy per bit, - No is power spectrum density (power per Hertz);

BER vs S/N or Eb/No curve is the most important performance measurement; Other measurements: complexity, delay, etc.

18

S/N vs. Eb/No

Eb Rb Eb Rb S = = N N 0 BW N 0 BW
Eb is energy per bit Rb is bit rate [bits/s] BW is channel bandwidth No is the noise spectrum density

Eb Rb S (dB) = (dB) + 10 log( ) N N0 BW Eb Rb S (dB) = (dB) 10 log( ) N0 N BW

Eb/No is independent of data rate and bandwidth!
19

S/N vs. Eb/No

Eb Rb S (dB) = (dB) 10 log( ) N0 N BW
Example: for ATSC system, S/N = 15 dB, BW = 6 MHz, Rb = 19.4 Mbps, Eb/No = 15 10 log(19.4/6) = 9.9 dB When say S/N, system bandwidth MUST be specified. For North American TV industry, BW is always 6 MHz. For telecom industry, BW is always 3dB bandwidth. ATSC system 3dB BW = 5.38 MHz, system S/N = 15.5 dB ATSC channel BW = 6.0 MHz, system S/N = 15.0 dB.

Eb/No is bandwidth and bit rate independent!!!

20

Turbo Codes
History
Introduced by Berrou, Glavieux, and Thitimajshima (from TelecomBretagne, former ENST Bretagne, France) in their paper: "Near Shannon Limit Error-correcting Coding and Decoding: Turbo-codes" published in the Proceedings of IEEE International Communications Conference (ICC1993). Turbo-codes are also called Parallel (or Serial) Concatenated Convolutional Code (PCCC or SCCC). Since the introduction of the original parallel turbo codes (PCCC) and iterative decoding algorithm in 1993, many other classes of turbo code have been discovered, including serial versions (SCCC) and repeataccumulate (RA) codes. Iterative Turbo decoding methods have also been applied to more conventional FEC systems, including Reed-Solomon/convolutional code structure.
21

Turbo Codes
Encoder-Decoder Structure
PCCC: Parallel Concatenated Convolutional Code.

Encoder Decoder

: is an interleaver

22

Turbo Codes
Performance (BER vs Eb/No, PCCC vs SCCC)

23

Turbo Codes

24

Turbo Codes
Turbo codes have good performance in low codes rate, such as R < 0.5. It is often used in mobile communication systems for low SNR fading channel (3GPP, LTE, etc.); The existence of error-floor requires an outer code (R-S or BCH code) to be used for error-free transmission.

25

LDPC Codes History

The Low-Density Parity-Check (LDPC) code concept was invented by Robert G. Gallager in his doctoral dissertation at MIT in 1963. Deemed impractical to implement when first developed (that was pre-silicon and microprocessor days), LDPC codes were forgotten, but they were rediscovered in 1996. In the last few years, the advances in LDPC codes have surpassed turbo codes in terms of error floor and performance in the higher code rate range, leaving turbo codes better suited for the lower code rates (mobile communications).
26

LDPC Codes
Parity Check Matrix

H=
(dv=3,dc=4) An LDPC code is called regular code, if its column and row weights (dv,dc) are constant for each column and row. Otherwise, it is called irregular LDPC code. A special class of irregular LDPC codes, namely QC-LDPC, prevails in most standards.

27

LDPC Codes
Bipartite Graph (equivalent to H)
Check Nodes Edges Variable Nodes (Codeword bits) dc=4

dv=(2,3)
28

LDPC Codes
Iterative Decoding (Belief Propagation)
Edges Check Nodes dc=4 ri

qi dv=(2,3) Variable Nodes

29

LDPC Codes
Performance of DVB-S vs. DVB-S2

30

LDPC Codes
Performance of DVB-T2 vs. DVB-T

31

Error Correction Codes in Wireless Communications and Broadcasting

ATSC: TCM + R-S; DVB-T and ISDB-T: Convolutional + R-S; DTMB: LDPC(8k) + BCH; DVB-T2: LDPC (16/64k) + BCH; Wi-Max and WiFi: LDPC(0.5/1/2k) + BCH; ATSC M/H: Turbo + R-S/CRC product codes; 3GPP and LTE: Turbo + R-S;

32

A Discussion:

Data Rate vs. Robustness for Terrestrial DTV

33

Data Rate vs. Robustness

The Information theory is about to trade off between data rate and reliability; Recently, DVB-T2 and NHK Super Hi-vision both aimed at higher data rate for higher spectrum efficiency, which might lead to less robust reception. Question: Can terrestrial broadcasting system win the data rate race over satellite/Cable/IPTV? Answer: Unlikely!!!
34

Data Rate vs. Robustness

The real advantage of terrestrial broadcasting Mobile and Portable reception anywhere, anytime, at home, on the go. How to strike a balance between spectrum efficiency and reception robustness? Are there any alternatives?

35

Recommendation
Using advanced audio/video coding to achieve better spectrum efficiency, so that more services are available within 6 MHz channel; Using advanced error correction codes to achieve more robustness in reception; Additionally, using diversity technology and distributed transmission to reach a greener terrestrial broadcast DTV eco-system.

36

Thank You!
Bo Rong
Communications Research Centre Canada [email protected]
613-991-4438
37