Channel Coding 2

Dr.-Ing. Dirk Wübben

Institute for Telecommunications and High-Frequency Techniques
Department of Communications Engineering
Room: N2300, Phone: 0421/218-62385
[email protected]
Lecture Tutor
Tuesday, 08:30 – 10:00 in N3130 Shayan Hassanpour
Exercise Room: N2390
Wednesday, 14:00 – 16:00 in N2420 Phone 218-62387
Dates for exercises will be announced [email protected]
during lectures.

www.ant.uni-bremen.de/courses/cc2/
 Outline Channel Coding II
 1. Concatenated Codes
 Serial Concatenation & Parallel Concatenation (Turbo Codes)
 Iterative Decoding with Soft-In/Soft-Out decoding algorithms
 EXIT-Charts
 BiCM
 LDPC Codes
 2. Trelliscoded Modulation (TCM)
 Motivation by information theory
 TCM of Ungerböck, pragmatic approach by Viterbi, Multilevel codes
 Distance properties and error rate performance
 Applications (data transmission via modems)
 3. Adaptive Error Control
 Automatic Repeat Request (ARQ)
 Performance for perfect and disturbed feedback channel
 Hybrid FEC/ARQ schemes

2
 Chapter 1. Concatenated Codes
 Introduction
 Serial and Parallel Concatenation
 Interleaving
 Serial Concatenation
 Direct approach, Product Codes, Choice of Component Codes
 Parallel Concatenation
 Modification of Product Codes, Turbo-Codes, Choice of Component Codes
 Distance Properties and Performance Approximation
 Decoding of Concatenated Codes
 Definition of Soft-Information, L-Algebra, General Approach for Soft-Output
Decoding,
 BCJR-Algorithm, Iterative Decoding, General Concept of Iterative Decoding
 EXtrinstic Information Transfer (EXIT)-Charts
 Bitinterleaved Coded Modulation (BiCM)
 Low Density Parity Check (LDPC) Codes

3
 Introduction
 Achieving Shannon’s channel capacity is the general goal of coding theory
 Block- and convolutional codes of CC-1 are far away from achieving this limit Claude E. Shannon

 Decoding effort increases (exponentially) with performance

 Questionable, if Shannon’s limit can be achieved by these codes

 Concatenation of Codes
David Forney
 Forney (1966): proposed combination of simple codes
 Berrou, Glaxieux, Thitimajshima: Turbo-Codes (1993):
Clever parallel concatenation of two convolutional codes
achieving 0.5 dB loss at Pb=10-5 to channel capacity

 Principal Idea: Claude Berrou Alain Glavieux Punya Thitimajshima

 Clever concatenation of simple codes in order to generate a total code with high performance
and enabling efficient decoding
 Example:
 Convolutional Code with LC = 9  28 = 256 states
 2 Convolutional Codes with LC = 3  2·22 = 8 states  complexity reduction by a factor of 32
repeated decoding (6 iterations)  6·8 = 48 states  reduction by a factor of 5

4
 Serial and Parallel Code Concatenation
 Serial Code Concatenation

C1 C2 D2 D1
inner code
outer code
 Subsequent encoder obtains whole output stream of previous encoder
 redundancy bits are also encoded
C1 P
 Parallel Code Concatenation
 Each encoder obtains only
information bits C2
 Parallel-serial converter generates
serial data stream
 Example: Turbo Codes
Cq S

5
 Interleaving
𝑥𝑥 𝑥𝑥′
 Interleaver performs permutation of symbol sequence Π
channel
 Strong impact on performance of concatenated codes 𝑦𝑦 𝑦𝑦𝑦
Π−1
 Also used to split burst errors into single errors for fading channels
 Block interleaver
write Column-wise write in, but
row-wise read out leads to
read
permutation of symbol sequence
𝑥𝑥0 𝑥𝑥3 𝑥𝑥6 𝑥𝑥9 𝑥𝑥12
interleaving depth LI = 5:
𝑥𝑥1 𝑥𝑥4 𝑥𝑥7 𝑥𝑥10 𝑥𝑥13 neighboring symbols of the input
stream have a distance of 5 in the
output stream
𝑥𝑥2 𝑥𝑥5 𝑥𝑥8 𝑥𝑥11 𝑥𝑥14  given by number of columns

 input sequence: x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x10, x11, x12, x13, x14
 output sequence: x0, x3, x6, x9, x12, x1, x4, x7, x10, x13, x2, x5, x8, x11, x14
LI

6
 Interleaving
 Assumption: burst errors of length b should 𝑥𝑥 𝑥𝑥′
Π
be separated channel
𝑦𝑦 𝑦𝑦𝑦
 Aspects of dimensioning block interleaver Π−1
 Number of columns
 affects directly the interleaver depth LI Burst error
 LI ³ b is required, so that burst error of length b is broken into single errors by Π−1
 Number of rows
 Example: For a convolutional code with LC = 5, five successive code words are correlated
 for Rc =1/2 ten successive code bits are correlated
 In order to separate these ten bits (by LI to protect them from burst errors), the number of
rows should correspond to LC/Rc=10
 Time delay (latency)
 The memory is read out after the whole memory is written =
∆t rows ⋅ columns ⋅ Tb
 Notice: For duplex speech communication only an overall delay of 125 ms is tolerable
400
=
 Example: data rate 9,6 kbit/s and interleaver size 400 bits 2 ⋅ ∆t 2= 83,3 ms
96001/ s

7
 Interleaving
 Convolutional Interleaver
(N-1) L
L

2L channel 2L

L
(N-1) L

 Consists of N registers and multiplexer

 Each register stores L symbols more than the previous register
 Principle is similar to block interleaver
 Random Interleaver
 Block interleaver has a regular structure  output distance is directly given by input
distance  leading to bad distance properties for Turbo-Codes
 Random interleavers are constructed as block interleavers where the data positions
are determined randomly
 A pseudo-random generator can be utilized for constructing these interleavers

8
 Serial Code Concatenation: Direct Approach
 Concatenation of (3,2,2)-SPC and (4,3,2)-SPC code
C1 C2
u c1 c2 w H (c 2)
00 000 0000 0 Rc = 2/4 = 1/2
01 011 0110 2
Concatenation does not automatically
10 101 1010 2
dmin = 2 result in a code with larger distance
11 110 1100 2

 Concatenation of (4,3,2)-SPC and (7,4,3)-Hamming code

u c1 c22 wH
w (c22))
H(c c2 w H (c 2)
Rc = 3/7
000 0000
0000 0000
0000 000
000 00 0000 000 0
001 0011
0011 0011 001
0011 001 33 0001 111 4 original concatenation:
010 0101
0101 0101 010
0101 010 33 0110 011 4
011 0110
0110 0110 011
0110 011 44 0111 100 4
dmin = 3
100 1001
1001 1001 100
1001 100 33 1010 101 4
101 1010
1010 1010 101
1010 101 44 1011 010 4
110 1100
1100 1100 110
1100 110 44 1100 110 4 optimized concatenation:
111 1111
1111 1111 111
1111 111 77 1101 001 4 dmin = 4

9
 Serial Code Concatenation: Product Codes
kH n H - kH
 Information bits arranged in (kV,kH)-
matrix u
 Row-wise encoding with systematic
kV (nH, kH, dH)-code CH of rate kH/ nH
u pH CH  each row contains a code word
 Column-wise encoding with systematic
(nV , kV , dV)-code CV of rate kV / nV
 each column contains a code word
n V - kV pV p+
 Entire code rate:
CV kH ⋅ kV
=
Rc = Rc ,H ⋅ Rc ,V
„checks on checks“ nH ⋅ nV
CH Π CV  Minimum Hamming distance:
block interleaver =
d min d min,H ⋅ d min,V

10
 Serial Code Concatenation: Examples of Product Codes
Detection capability of
(12,6,4) product code (28,12,6) product code horizontal code exceeded

x0 x4 x8 x0 x7 x14 x21 x0 x7 x14 x21

x1 x5 x9 x1 x8 x15 x22 x1 x8 x15 x22
x2 x6 x10 x2 x9 x16 x23 x2 x9 x16 x23
x3 x7 x11 x3 x10 x17 x24 x3 x10 x17 x24
x4 x11 x18 x25 x4 x11 x18 x25
 Horizontal: (3,2,2)-SPC code
 Vertical: (4,3,2)-SPC code x5 x12 x19 x26 x5 x12 x19 x26
 Code rate: 1/2
 dmin = 2 · 2 = 4
x6 x13 x20 x27 x6 x13 x20 x27
 Correction of 1 error & detection of 3  Horizontal: (4,3,2)-SPC code
errors possible  Vertical: (7,4,3)-Hamming code
Interleaver combines 3 info words  increase of eff. block length  dmin = 2 · 3 = 6  correction of 2 errors possible

11
 Parallel Code Concatenation: Modified Product Codes
kH n H - kH
 Information bits u row-wise and
column-wise encoded with CH and
CV, respectively

kV  Parity check bits of component

u pH CH codes not encoded twice
(no checks on checks)
 Entire code rate
kH ⋅ kV
n V - kV pV Rc =
nH ⋅ nV − (nH − kH ) ⋅ (nV − kV )
CV 1
=
1 / Rc ,H + 1 / Rc ,V − 1
CH
 Minimum Hamming distance:
Π CV d min = d min,H + d min,V − 1

12
 Parallel Code Concatenation: Examples
modified (11,6,3) product code modified (25,12,4) product code

x0 x4 x8 x0 x7 x14 x21
x1 x5 x9 dmin = 3 x1 x8 x15 x22
x2 x6 x10 x2 x9 x16 x23
x3 x7 x3 x10 x17 x24
x4 x11 x18
 Horizontal: (3,2,2) SPC code x5 x12 x19
 Vertical: (4,3,2) SPC code
x6 x13 x20
 Code rate: 6/11
 Horizontal: (4,3,2) SPC code
 dmin = 2 + 2 -1 = 3
 Vertical: (7,4,3) Hamming code
 1 error correctable
 dmin = 2 + 3 -1 = 4  1 error correctable

13
 Union Bound on Bit Error Rate for Product Codes
 Product codes using same (n,k,3)-Hamming code
 Only taking into account minimum distance dmin=3+3-1=5
 results only valid for high signal to noise ratios
0 0
10 10
(7,4) (7,4)
(15,11) (15,11)
(31,26) (31,26)

Pb 10-5 -5
P 10
b

-10 -10
10 10
0 2 4 6 8 10 0 2 4 6 8 10
10log10 ( Es / N 0 ) 10log10 ( Eb / N 0 )

14
 Parallel Code Concatenation: Turbo Codes
General structure with q constituent codes special case with 2 constituent codes
u
u
u1 c1
Π1 C1 u=u1 c1
C1 c

c2
Π2 u2 C2 P c Π P
u2 c2
C2

uq cq
Πq Cq  Interleaver P1 neglectable
 Information bits generally not punctured
 Presented in 1993 by Berrou,  Code rate: R = 1
1 / Rc ,1 + 1 / Rc ,2 − 1
Glavieaux, Thitimajshima c

15
 Potential of Turbo Codes
0
Comparison convolutional codes / turbo codes for Rc=1/2
10
Lc=3  Optimized interleaver of
Lc=5
-1 Lc=7 length 256 x 256 = 65536 bits
10 Lc=9  For this interleaver, gain of
TC
nearly 3 dB over convolutional
-2
10 code with Lc = 9
Pb  Gap to Shannon‘s channel
-3
10 capacity only 0.5 dB
(C = 0.5 at Eb/N0 = 0.19 dB)
-4  Tremendous performance
10
loss for smaller interleavers
0.5 dB
-5
 World record: 0.08 dB gap to
10 Shannon capacity by Stephan
0 1 2 3 4 5 6
0.19 dB
10 log10(Eb/N0) ten Brink

16
 Influence of Constituent Codes
 Systematic recursive convolutional encoders employed in turbo codes
 Constituent codes generate only parity bits
1
 Conventionally codes with small constraint length (3 ≤ Lc ≤ 5) and rate 𝑅𝑅𝑐𝑐 =
𝑛𝑛
(codes of larger rate can be achieved by puncturing)

 Error probability depends on interleaver size 𝐿𝐿𝜋𝜋 and minimum input weight 𝑤𝑤𝑚𝑚𝑚𝑚𝑚𝑚 of
constituent encoders that leads to finite output weight
Pb ∼ L1π− wmin
 Only recursive encoders require at least 𝑤𝑤min = 2 for finite output weight
 Interleaving gain only achievable for recursive encoders due to 𝑃𝑃𝑏𝑏 ~𝐿𝐿−1
𝜋𝜋
 Nonrecursive encoders with 𝑤𝑤min = 1 do not gain from enlarging interleaver size
(𝑃𝑃𝑏𝑏 ~𝐿𝐿0𝜋𝜋 )
RSC-Encoders are used as constituent codes
 performance improves with length of interleaver!

17
 Influence of Constituent Codes
 Instead of free distance df the effective distance deff is crucial
d eff= wmin + 2 ⋅ cmin
 Interpretation: Turbo codes are systematic codes
 Total weight of code words depends on weight of information bits 𝑤𝑤min
 𝑐𝑐min denotes minimum weight of parity bits of one encoder for input weight 𝑤𝑤min = 2
 Assuming same constituent codes, minimum weight for 𝑤𝑤min = 2 is given by 𝑑𝑑eff

 Consequence:
 Suitable constituent codes should maximize parity weight for input weight 𝑤𝑤min = 2
 Aim is achieved if feedback polynomial of constituent encoders is prime
 Shift register generates sequence of maximum length (m-sequence)
 may have larger weight than shorter sequences
Feedback polynomial of constituent encoders should be prime!

18
Example of Turbo Code with 2 Codes (Lc = 3), Rc = 1/2

c1
c
u1
T T

Π C1 g1 = 58
c2 g 2 = 78
u2 P
T T 1 0
P= 
C2  0 1 

19
 Example of Turbo Code with 2 Codes (Lc = 3), Rc = 1/2
 Recursive polynomial: g2(D) = 1 + D + D2 0/0
 g2(D) is prime 00
1/1
1/1
g2(0) = 1 + 0 + 0 = 1 and g2(1) = 1 + 1 + 1 = 1
1/0
 Shift register achieves sequence of maximum 10 01
length (m-sequence) with L = 22-1=3 0/0
max 0/1 0/1
 Max dist. 𝑑𝑑eff = 𝑤𝑤min + 2 ⋅ 𝐿𝐿 + 1 = 2 + 2 ⋅ 4 = 10 11
 u = [1 0 0 1]  c1 = [1 1 1 1] 1/0

 Recursive polynomial: g1(D) = 1+ D2 0/0

 g1(D) = (1+D)(1+D)  non-prime 00

1/1 1/1
 Shift register generates sequence of length L = 2 0/0
max 10 01
 Max dist.𝑑𝑑eff = 𝑤𝑤min + 2 ⋅ 𝐿𝐿 + 1 = 2 + 2 ⋅ 3 = 8 0/1
 u = [1 0 1]  c1 = [1 0 1] 1/0 1/0
Feedback polynomial g1(D) would lead to degraded 11

performance! 0/1

20
 Example of Turbo Code with 2 Codes (Lc = 5), Rc = 2/3
= =
g1 238 g 2 358
u

c1
c
u1
T T T T

Π C1
c2

u2
T T T T 1 0 0 0
P= 
C2  0 0 1 0 

21
 LTE Turbo Code with 2 Codes (Lc = 4)
g1 =+
1 D 2 + D3 =
138 g 2 =+
1 D + D3 =
158
u

c1
c
u1
T T T Rate
Matching
Π C1
c2

u2
T T T

C2

22
 Influence of Interleaver
1  Eb  cd: total number of nonzero info bits
Pb ≤ ∑ cd ⋅ erfc  d ⋅ Rc  associated with code sequences of
2 d  N 0  Hamming weight d
 Avoiding output sequences with low Hamming weight at both encoders
 If output c1 of C1 has low Hamming weight  permutation of input sequence u2 for C2
should result in output sequence c2 with high Hamming weight
 Higher total average Hamming weight / Hamming distance d
 Interleaver directly influences minimum distance
 Number of sequences with low weight reduced due to interleaving
 Small coefficients cd
 Even more important than minimum distance that acts only asymptotically
 Randomness of interleaver is important
 Simple block interleavers perform bad due to symmetry
 Pseudo-random interleavers are much better → random codes ( Shannon)

23
 Distance Properties of Turbo Codes: Definitions

 General IOWEF (Input Output Weight Enumerating Function) of encoder:

k n
A (W=
, D) ∑∑ A
=
w 0=
d 0
w,d ⋅ W w
⋅ D d Aw,d: number of code words with
input weight w and output weight d

 Conditioned IOWEF’s (specific input weight w or specific output weight d):

n k
A (=
w, D ) ∑A
d =0
w,d ⋅D d
A (=
W,d ) ∑A
w=0
w,d ⋅W w

 Important for parallel concatenation: weight c of parity bits

, C ) ∑∑ Aw,c ⋅ W w ⋅ C c
A (W= with d= w + c All encoders have same input weight w
w c Encoders generate only parity bits
 consider weight c of parity bits
 Corresponding conditioned IOWEF:
A (=
w, C ) ∑A
c
w ,c ⋅Cc

24
 Distance Properties of Turbo Codes: Uniform Interleaver
 Problem: concrete interleaver has to be considered for distance spectrum /
IOWEF
→ determination of IOWEF computationally expensive
 Uniform interleaver (UI): theoretic device comprising all possible permutations
0 0 1 1
1/6
1/6 0 1 0 1

uniform 1/6 0 1 1 0  4
0 1 0 1  2  possibilities
interleaver 1/6
1/6 1 0 0 1  

1 0 1 0
1/6
1 1 0 0
 UI provides average distance spectrum (incl. good and bad interleavers)

25
 Distance Properties of Turbo Codes: Results
 Parallel concatenation:
 Both encoders have same input weight w
 Weights c1 and c2 of encoder outputs are added
 A1(w,C)·A2(w,C) combines output sequences with same input weight w and covers all
possible combinations of output sequences (uniform interleaver)
 Denominator achieves averaging w.r.t. number of permutations of w ones in length Lπ
A1 ( w, C ) ⋅ A2 ( w, C )
Apar (=
w, C )
 π
L
= ∑A par
w ,c ⋅Cc
=cd ∑
w
⋅ Awpar,c
w c
w+ c =
Lπ
  d

 Serial concatenation:
 Output weight  of outer encoder equals input weight of inner encoder
A1 (W , ) ⋅ A2 (, D)
A (W=
ser
, D) ∑ = ∑∑ ser
⋅ w
⋅ d

∑ L ⋅R
A W D w

 Lπ  w,d = cd 1 ⋅ Aser
w ,d
   w d
π c
  w

26
 Distance Properties of Turbo Codes
 Codes
10
5 TC, Lπ=100
TC, Lπ=400  Turbo Code
CC g1 = 58, g2 = 78
 Convolutional Code with
Lc=9
 Rc=1/3
cd 100  Observations
 UI  cd < 1 is possible
 TC has lower df but
coefficients cd are much
smaller
-5  effect becomes more
10 obvious with increasing
interleaver length Lπ
5 10 15 20 25 30
distance d

27
 Analytical Error Rate Estimation of Turbo Codes
0
10  Observations
TC, L=100
TC, L=400  For small SNR the TC
-2
10 outperforms CC
FC
significantly
 Gain increases with Lπ
-4
10  For increasing SNR the
Pb
Flat Rayleigh Fading BER of TC flattens,
-6 whereas the curve of CC
10 decreases
 Explanations
-8
10  df dominates BER for
large SNR
AWGN
-10
 For small SNR the
10 number of sequences
0 2 4 6 8 10
with specific weight is of
10log10 ( Eb / N 0 ) larger importance

28
Decoding of Concatenated Codes

 Definition of Soft-Information
 L-Algebra
 General Approach for Soft-Output Decoding
 Soft-Output Decoding using the Dual Code
 Soft-Output Decoding for (4,3,2)-SPC-Code
 BCJR Algorithm for Convolutional Codes

29
 Decoding of Concatenated Codes
 Optimum Maximum Likelihood Decoding of concatenated codes is too complex
 Constituent codes C1 and C2 are decoded by separated decoders D1 and D2
 Decoders D1 and D2 are allowed to exchange “information” in to improve their
performance
 probability of information and/or code bits is of interest
 soft output decoding is required!
 What is a useful soft output?
 Assumption: uncoded transmission over AWGN channel y= x + n
 BPSK modulation ⊕ 0 1 ∙ +1 -1
u=
0 → x=
+1
x = 1 − 2u 0 0 1 +1 +1 -1
u=
1 → x=
−1 1 1 0 -1 -1 +1
 MAP criterion (Maximum a posteriori) considers unequal distribution of symbols

Pr {u 0= > Pr u 1| y
| y} < > Pr x =
= { } Pr { x =
+1| y} < { −1| y}

30
 Decoding of Concatenated Codes
 Conditional Probability Pr { x =
+1 y} =
p{x =
+1, y} Pr { y}
p{x = +1, y} > p { x = −1, y} p{x =
+1, y} p { y | x =
+1} Pr { x =
+1} >
< = ⋅ < 1
Pr { y} Pr { y} p{x =
−1, y} p { y | x =
−1} Pr { x =
−1}
 Log-Likelihood-Ratio (LLR) (or L-values) derived by Hagenauer
p { x = +1, y} >
=L( xˆ ) L=
( x, y ) L=
( x | y ) ln < 0
p { x = −1, y}
p{ y | x = +1} Pr { x = +1}
= ln + ln =
L( y | x) + La ( x)
{y | x =
 Sign ofpLLR’s

−1} hard
equals
 
{ x = −
Prdecision

1} Joachim
Hagenauer
L ( y| x ) La ( x )

 Sign sgn 𝐿𝐿 𝑥𝑥� corresponds to hard decision

 Magnitude 𝐿𝐿 𝑥𝑥� indicates reliability of hard decision Addition of LLRs requires
statistically independency
 Another possible definition would be (not used)
Pr { x =+1} − Pr { x =−1}
of variables!
L( x) =

31
 Log-Likelihood-Ratio
 For an uncoded transmission the LLR consists of two components
 L(y|x) depends on channel statistics and therefore on the received signal y
 La(x) represents a-priori knowledge about symbol x
8 Pr { x = +1}
La ( x) = ln
6 Pr { x = −1}
4  Symmetric with respect to (0,5 ; 0)
2  Pr{x = +1} > 0,5
La(x)
0  +1 more likely than -1
 positive La(x)
-2
 The larger the difference between
-4 Pr{x=+1} and Pr{x=-1} the larger La(x)
-6  suitable value for reliability
 Pr{x = +1} = 0,5  La(x) = 0  decision
-8
0 0.2 0.4 0.6 0.8 1 would be random
Pr{x = +1}

32
 LLR for a Memoryless Channel
 Memoryless channel (AWGN or 1-path fading channel) y =α x + n
 Channel information
 1
( )
2
exp  − 2 y − α Es / Ts 
p { y | x = +1}  2σ 
= L ( y x ) ln= ln
p { y | x = −1}  1
( )
2
exp  − 2 y + α Es / Ts 
 2σ 
1
( ) 1
( )
2 2
= + α − − α N0
2σ 2
y E s / T s
2σ 2
y E s / Ts with σ 2
=
2Ts
4α y Es / Ts Es / Ts 2 Es
= = 4 α y = 4 α y′
2σ 2
N 0 / Ts N0 normalized received signal
 
y
y′ =
Lch

α Es / Ts
 Lch= reliability of the channel (depends on SNR ES / N0 and channel gain |α|2)

33
 LLR for a Memoryless Channel
Es
 Reliability of channel: Lch= 4 α
2

N0
 LLR is simply a scaled version of the matched filter  motivation for ln
L(y|x) versus y
60
high channel reliability
40

20
L ( y x )= 4 α
Es
y′
2

N0 0
0 dB
-20 2 dB
4 dB
-40 6 dB
8dB
-60
-2 -1 0 1 2
y

34
 LLRs for BSC and BEC
1-Pe
X0 Y0
 Binary Symmetric Channel (BSC) Pe
Pe
X1 1-Pe Y1

 1 − Pe
for y = Y0 = +1
5
 ln
p { y | x = +1}  1 − Pe

𝐿𝐿(𝑦𝑦|𝑥𝑥)
Pe
L ( y x )= ln =  = y ⋅ ln 0

p { y | x = −1}  P Pe
ln e for y = Y1 = −1 -5
 1 − Pe
0 0.2 0.4 0.6 0.8 1
𝑃𝑃𝑒𝑒
 Binary Erasure Channel (BEC)
 1− P
ln q
for y = Y0
 0
 +∞ for y = Y0 X0 1-Pq Y0
Pq 
L ( y x=)  P
Pq
ln for =
y Y=2  0 for =
y Y2 Y2
 q −∞ for y = Y1 Pq
  X1
1-Pq
Y1
0
ln for y = Y1
 1 − Pq

35
 Relation between LLRs and Probabilities (1)
 Matched filter corresponds to LLR  Task: Find arithmetic to perform operation
with respect to LLR instead of probabilities  L-algebra by Hagenauer

 Basic relation
 Using completeness (Pr{x = +1} + Pr{x = -1} = 1) in LLR
Pr { x =
+1| y} Pr { x = +1| y} 1 − Pr { x =−1| y}
=
L( xˆ ) L=
( x | y ) ln = ln = ln
Pr { x =
−1| y} 1 − Pr { x =+1| y} Pr { x = −1| y}
e L ( xˆ ) 1
Pr { x = +1| y} = L ( xˆ ) = − L ( xˆ )
1+ e 1+ e
1
Pr { x = −1| y} = L ( xˆ )
1+ e
 With respect to symbol x∈{+1,-1} the general relation holds
e L ( xˆ )/2 i⋅L ( xˆ )/2 1
Pr {=
x i|=y} ⋅ e = − sgn( i )⋅ L ( xˆ )
with i ∈ {−1, +1}
1+ e L ( xˆ )
1+ e

36
 Relation between LLRs and Probabilities (2)
 Probability of a correct decision
 For x = +1 decision is correct, if 𝐿𝐿(𝑥𝑥)
� is positive
L ( xˆ )
e L ( xˆ )
Pr { xˆ correct x =+1} = L ( xˆ ) = L ( xˆ )
e
1+ e 1+ e
 For x = -1 decision is correct, if 𝐿𝐿(𝑥𝑥)
� is negative
L ( xˆ )
Pr { xˆ correct= } 1 + e1L (= 1 e
x -1= =
− L ( xˆ )
1+ e 1+ e
xˆ ) L ( xˆ )

L ( xˆ )
e
Pr { xˆ is correct} =
1+ e
L ( xˆ )

 Soft bit: expected value for antipodal tx signal

e L ( xˆ ) 1 e L ( xˆ ) − 1 L( xˆ )
λ= E { xˆ}= ∑ i ⋅ Pr { xˆ= i}= ( +1) + ( − 1 ) = = tanh
i =±1 1 + e L ( xˆ ) 1 + e L ( xˆ ) e L ( xˆ ) + 1 2
λ +1
Pr { xˆ = +1} = +1
2
-1

37
 L-Algebra
 Parity bits are generated by modulo-2-sums of certain information bits
 how can we calculate the L-value of a parity bit?  Hagenauer
 Assumption: Single parity check code (SPC) p= u1 ⊕ u2 L(p) = ?
1+ x
 x1 and x2 are statistically independent 2artanh ( x ) = ln
1− x
Pr {u1 ⊕ u2 =
0} Pr { x1 ⋅ x2 =+1}
L ( p ) = L ( u1 ⊕ u2 ) = ln = ln = L( x1 ⋅ x= ) ex −1
(2) x
Pr {u1 ⊕ u2 =
1} Pr { x1 ⋅ x2 =−1} λ tanh
=
2 x
e +1
Pr { x1 =+1} Pr { x2 =+1}
⋅ +1
Pr { x1 =+1} ⋅ Pr { x2 =+1} + Pr { x1 =−1} ⋅ Pr { x2 =−1} Pr { x1 =−1} Pr { x2 =−1}
L ( x1 ⋅ x2 ) ln = ln
Pr { x1 =+1} ⋅ Pr { x2 =−1} + Pr { x1 =−1} ⋅ Pr { x2 =+1} Pr { x1 =+1} Pr { x2 = +1}
+
Pr { x1 =−1} Pr { x2 = −1}
e L ( x1 ) ⋅ e L ( x2 ) + 1 e L ( x1 )+ L ( x2 ) + 1
L ( x1 ⋅ x2 ) ln L=
= ln L ( x1 ) boxplus
e ( x1 )
+e L ( x2 )
e +e L ( x2 )
operation
  L( x1 )   L( x2 )  
= 2artanh  tanh   ⋅ tanh  =   2artanh [=
λ1 ⋅ λ 2 ] L( x1 ) + L( x2 )
  2   2 
38
 L-Algebra
 mod-2-sum of 2 statistically independent random variables:
  L( x1 )   L( x2 )  
L ( u=
1 ⊕ u 2) 2artanh  tanh   ⋅ tanh  =   2artanh [=
λ1 ⋅ λ 2 ] L( x1 ) + L( x2 )
  2   2 
≈ sgn [ L( x1 ) ] ⋅ sgn [ L( x2 ) ] ⋅ min { L( x1 ) , L( x2 ) }
L( x1 ) λ1
tanh(x/2) -1
+1
+1 L( x1 ⋅ x2 )
2 artanh(x)
-1

L( x2 ) λ2  n  n

tanh(x/2) = L ( u1 ⊕  ⊕ un ) 2artanh
= ∏
 i =1
tanh ( i )
L ( x ) / 2

∑ L( x )
i =1
+
i

n
 mod-2-sum of n variables: ≈ min { L( xi ) } ⋅ ∏ sgn [ L( xi ) ]
i
i =1

39
 General Approach for Soft-Output Decoding
 For FEC encoded sequence MAP criterion should be fulfilled
p ( ui = 0, y )
 Symbol-by-Symbol MAP Criterion: ( i )
L uˆ = ln
p u = 1, y ( i )
 L-value for estimation of information bit ui given by receive sequence y
 Joint probability density function p(ui=0/1,y) not available  elementary conversions

 Using the completeness, the code space is split into two subsets
P (a ) = ∑ P (a, bi )
i

Γi( 0 ) =
contains all c with ui 0 ( ui 0=
p= ,y) ∑ c∈Γi( 0 )
p ( c, y )

Γi(1) =
contains all c with ui 1 ( ui
p= y) ∑
1,= c∈Γi(1)
p ( c, y )

=
∑
L ( uˆ ) ln=
p ( c, y ) c∈Γi( 0 )
ln
∑ c∈Γi( 0 )
p ( y | c ) ⋅ Pr {c} sum over 2k/2=2k-1 code words
in numerator and in
∑ p ( c, y )
i
c∈Γi(1) ∑ c∈Γi(1)
p ( y | c ) ⋅ Pr {c} denominator

40
 General Approach for Soft-Output Decoding
 Assuming statistical independency of the yj (transmission over AWGNC)
 Succeeding noise terms nj are independent, but of course not succeeding code bits
cj (interdependencies introduced by encoder)!
 p(y|c) represents probability density conditioned on the hypothesis c
n −1

∑ ∏ p( y | c j ) ⋅ Pr {c}
 yj are statistically independent random variables
n −1 j

p ( y | c) = ∏ p ( y j | c j ) L ( uˆi ) = ln i
c∈Γ j =0
(0)

n −1
j =0
∑ ∏ p( y j | c j ) ⋅ Pr {c}
c∈Γi(1) j = 0

 Each codeword c is uniquely determined by the corresponding info word u

n −1 k −1
p ( y j | c j ) ⋅ ∏ Pr {u j }
(ui are statistically independent)
k −1 ∑ ∏
=
Pr {c} Pr= {u} ∏ Pr {u j } L ( uˆi ) = ln i n−1
= c∈Γ ( 0 ) j 0=j 0
k −1

∑ ∏ p ( y j | c j ) ⋅ ∏ Pr {u j }
j =0

Symbol-by-Symbol MAP = c∈Γi(1) j 0=j 0

41
 General Approach for Soft-Output Decoding
 Symbol-by-Symbol MAP for systematic encoders
 For systematic encoder ui = ci holds for 0 ≤ i ≤ k-1 i-th term p(yi|ci) is constant in
numerator and denominator  can be separated together with P(ui)
n −1 k −1

∑ ∏ p( y
j j | c ) ⋅ ∏ Pr {u j }
= p ( yi | ui 0=
) + ln Pr {ui 0} + ln
=c∈Γi j 0=j 0
(0)

L ( uˆi )= ln
j ≠i j ≠i

= p ( yi | ui 1=
) Pr {ui 1} n −1 k −1

=
∑ ∏ p( y
j j | c ) ⋅ ∏ Pr {u j }
c∈Γi j 0=j 0
(1)
j ≠i j ≠i

= Lch ⋅ yi + La ( ui ) + Le ( ui )
 Soft-Output can be split into 3 statistically independent parts:
♦ Systematic part Lch·yi
♦ A-priori information La(ui)
♦ Extrinsic information Le(ui): information provided by code bits connected with ui

42
 General Approach for Soft-Output Decoding
 Compact description of extrinsic information
n −1 k −1 n −1  p ( y | c ) ⋅ Pr {u } 0 ≤  < k
∏ p (=
y j | c j ) ⋅ ∏ Pr {u j } ∏ p ( y j ; c j )=
with p ( y ; c ) 
j 0 =j 0=j 0  p ( y | c ) k ≤<n
j ≠i j ≠i j ≠i
n −1 k −1 n −1

=
j
j 0=j 0
j ∑ ∏ p( y | c ) ⋅ ∏ Pr {c j }
=
∑ ∏ p ( y ;c )
j j
c∈Γi
(0)
c∈Γi( 0 ) j 0
Le ( uˆ ) j ≠i j ≠i j ≠i
i ln= ln
n −1 k −1 n −1

=
∑ ∏ p ( y j | c j ) ⋅ ∏ Pr {c j }
0=j 0 =
∑ ∏ p ( y ;c )
j j
c∈Γi(1) j c∈Γi(1) j 0
j ≠i j ≠i j ≠i

 Calculation of extrinsic information with LLR‘s:

n −1

∑ ∏ exp − L(c ; y ) ⋅ c  j j j

c∈Γi( 0 ) j =0
 Lch ⋅ y + La (u ) 0 ≤  < k
Le ( uˆi )
j ≠i
ln= n −1
with L(c ; y ) 
 Lch ⋅ y k ≤<n
∑(1) ∏  j j j 
exp  − L (
j =0
c ; y ) ⋅ c 
c∈Γi
j ≠i

43
 Soft-Output Decoding of Repetition Codes
 Code word 𝐜𝐜 = 𝑐𝑐0 𝑐𝑐1 ⋯ 𝑐𝑐𝑛𝑛−1 contains 𝑛𝑛 repetitions of information word 𝐮𝐮 = [𝑢𝑢0 ]
 Set of all code words for 𝑛𝑛 = 3 is given by Γ = {000, 111}
n −1 n −1
∑ ∏ p( y j | c j ) ⋅ Pr {c} ∏ p( y j | 0 ) ⋅ Pr {c =
[000]}
L ( uˆ0 ) ln= ln
c∈Γ 0( 0 ) j = 0 j =0
n −1 n −1

=
∑ ∏ p ( y j | c j ) ⋅ Pr {c} ∏ p ( y |1) ⋅ Pr {c =
j [111]}
c∈Γ 0(1) j 0=j 0

p ( y0 | 0 ) ⋅ p ( y1 | 0 ) ⋅ p ( y2 | 0 ) ⋅ Pr {ui =
0}
= ln
p ( y0 |1) ⋅ p ( y1 |1) ⋅ p ( y2 |1) ⋅ Pr {ui =
1}
p ( y0 | 0 ) p ( y1 | 0 ) p ( y2 | 0 ) Pr {ui = 0}
= ln + ln + ln + ln
p ( y0 |1) p ( y1 |1) p ( y2 |1) Pr {ui = 0}
= L ( y0 | c0 ) + L ( y1 | c1 ) + L ( y2 | c2 ) + La ( u0 )
 Corresponds to averaging of LLRs

44
 Soft-Output Decoding using the Dual Code
 Calculation of extrinsic information requires summation over all code words c of
the code space Γ
 The (255,247,3) Hamming code contains 2247 = 2.3·1074 code words
 Instead of calculating the LLR over all code words c of the code , it is also
possible to perform this calculation with respect to the dual code ┴
 Beneficial, if the number of parity bits is relatively small
 dual code for (255,247,3) Hamming code contains only 28 = 256 code words
 Calculation of extrinsic information with dual code:
c′
n −1
  L ( c ; y  
) 
∑ ∏  tanh 




c′∈Γ ⊥  = 0  2
Le ( uˆi ) = ln
 ≠i
c′
n −1
  L ( c ; y 
) 
∑ ( ) ∏
′
−
ci  Summation over all 2n-k code
1  tanh  
′
c ∈Γ ⊥
 = 0   2  words c’ of the dual code
 ≠i

45
 Soft-Output Decoding of (4,3,2)-SPC using the Dual Code
 Calculation of extrinsic information requires summation over 23 = 8 code words.
Instead, the dual code contains only 2n-k = 2 words Γ┴ = {0000, 1111}.
 Calculation of LLR
n −1
  L(c ; y )   First term in numerator and
1 + ∏  tanh   denominator (c=0000) is one.
 =0   2 
L ( uˆi ) = Lch ⋅ yi + ln
 ≠i
n −1
  L(c ; y )   1+ x
1 − ∏  tanh   with ln = 2artanh ( x )
 =0   2  1− x
 ≠i
Each cÎG fulfills ccT=0, i.e. ci is given
 n−1 
  L(c ; y )   
= Lch ⋅ yi + 2artanh  ∏  tanh 
by modulo-2-sum of all other code
  =0    bits cj:
 2   n
  ≠i  ci = ∑ c j Le ( ci ) = ∑+ L( x j )
n −1
≈ Lch ⋅ yi + min { L(c ; y ) } ⋅ ∏ sgn [ L(c ; y ) ]
j ≠i j =1
j ≠i
 ≠i
 =0
 ≠i

46
 Soft-Output Decoding for (4,3,2)-SPC-Code
Lch = 4 ⋅ EsN 0lin
Es/N0 = 2dB = 4 ⋅10 EsN 0dB /10
=
4 ⋅102/10 =
6,34
u 1 0 1 Lch·y -5.1 +7.0+1.9+2.5
n

encoding + Approximation for Le ( ci ) = ∑+ L( x j )

j =1
j ≠i
c 1 0 1 0 Le(û) +1.9 -1.9 -2.5 -1.9 Pr{û correct}

BPSK = Lch·y+ Le(û) 0.960.990.650.65

HD
x -1 +1 -1 +1 L(û) -3.2 +5.1 -0.6 +0.6 -1 +1 -1 +1

error corrected
AWGN
HD error detected.
y -0.8 +1.1+0.3+0.4 -1 +1 +1 +1
but not corrected

47
 BCJR Algorithm for Convolutional Codes
 Symbol-by-Symbol MAP Decoding: Bahl, Cocke, Jelinek, Raviv (1972)
p (ui = 0, y )
L ( uˆi ) ln= ln
∑ ( s ', s ), ui = 0
= ln
p ( s ', s, y ) ∑ ( s ', s ), ui = 0
p ( s ', s, y k <i , y i , y k >i )
p (ui = 1, y ) ∑ ( s ',s ), u =1 p(s ', s, y ) ∑ ( s ',s ), u =1 p(s ', s, y k <i , y i , y k >i )
i i

 Efficient calculation of LLR based on the Trellis diagram (exploiting Markov prop.)
state s‘ state s
p ( s ', y k <i ) p (y i , s | s ') p(y k >i | s)
Trellis of a RSC with Lc=3
00

ui = 1
10
ui = 0
01
y = [ y1 y 2  y N ]
y i =  yi ,0 yi ,1  yi ,n−1 
11
i-1 i

48
 BCJR Algorithm for Convolutional Codes state s‘ state s
p ( s ', y k < i ) p (y i , s | s ') p(y k >i | s)

 Splitting up the observations yk>i

p=
( s ', s, y k <i , y i , y k >i ) p (y k >i s ', s, y k <i , y i ) ⋅ p ( s ', s, y k <i , y i )
i-1 i
 Backward probability: Probability of the sequence yk>i, if the trellis is assumed
in state s at time instant i If state s at time instant i is known, the
βi ( s ) p=
(y k >i s ', s, y k <i , y i ) p (y k >i s ) parameter s’, y , y are not relevant i k<i

 Splitting up the observations yi

p=
( s ', s, y k <i , y i ) p ( s, y i s ', y k <i ) ⋅ p ( s ', y k <i )
 Transition probability: Probability of observing yi under the condition that the
transition from s’ to s takes place at time instant i  yk<i not relevant
= γ i ( s ', s ) p=
( s, y i s ', y k <i ) p ( s, y i s ') p{yi|s’,s}: transition probability

Pr {s ', s}
of channel
= = p (y i s ', s ) = p (y i s ', s ) ⋅ Pr {s s '}
p ( s ', s, y i ) Pr{s|s’} : a-priori-information
Pr {s '} Pr {s '}
 Possibility to use a-priori knowledge within the decoding process  Pr{s|s’} ~ ui

49
 BCJR Algorithm for Convolutional Codes state s‘ state s
p ( s ', y k < i ) p (y i , s | s ') p(y k >i | s)

 Forward probability: α i −1 ( s′) = p ( s ', y k <i ) the trellis is assumed in state

Probability of sequence yk<i, if
s’
 Probability density splits into three terms at time instant i-1
p ( s ', s, y k <i , y i , y k >i ) = α i −1 ( s ') ⋅ γ i ( s ', s ) ⋅ βi ( s ) i-1 i

 Compact description of Symbol-by-Symbol MAP

L ( uˆi ) = ln
∑ p ( s ', s, y k <i , y i , y k >i )
( s ', s ), ui 0=
= ln
∑ α ( s ') ⋅ γ i ( s ', s ) ⋅ βi ( s )
( s ', s ), ui 0 i −1

∑ ( s ',s ), u 1 =
i
p ( s ', s , y k <i , y i , y k >i ) ∑ ( s ',s ), u 1 αi−1 (s ') ⋅ γ i (s ', s) ⋅ βi (s)
i

 Recursive Calculation ai-1(0) gi(0,0) ai(0)

 Forward probability=
α i ( s ) p=
( s, y k <i +1 ) ∑ γ (s ', s) ⋅ α
s'
i i −1 ( s ')
ai-1(1)
=
 Backward probability βi −1 ( s ') p=
(y k >i −1 s ') ∑ γ (s ', s) ⋅ β (s)
i i
gi(3,1) βi(1)
s βi-1(3) gi(3,3) βi(3)
 Initialization Terminated code otherwise
1 s ' = 0 1 s ' = 0
α 0 ( s ') =  β N (s) =  β N ( s ) = 2− m
0 s ' ≠ 0 0 s ' ≠ 0
(m memory elements)

50
 BCJR Algorithm for Convolutional Codes
 Symbol-by-Symbol MAP Decoding:
p (ui = 0, y )
L ( uˆi ) ln= ln
∑ α ( s ') ⋅ γ i ( s ', s ) ⋅ βi ( s )
( s ', s ), ui = 0 i −1
ui = 1
= p (ui 1, y ) ∑ ( s ',s ), u =1 αi−1 (s ') ⋅ γ i (s ', s) ⋅ βi (s)
i
ui = 0

𝛼𝛼3 0 = 𝛼𝛼2 0 𝛾𝛾3 0,0

𝛼𝛼0 0 = 1 𝛾𝛾 0,0 𝛼𝛼1 0 𝛾𝛾2 0,0 𝛼𝛼2 0 𝛾𝛾3 0,0 +𝛼𝛼2 1 𝛾𝛾3 1,0 𝛽𝛽𝑁𝑁−2 0 𝛾𝛾𝑁𝑁−1 0,0 𝛽𝛽𝑁𝑁−1 0 𝛾𝛾𝑁𝑁 0,0 𝛽𝛽𝑁𝑁 0 = 1
1
00

𝛼𝛼0 2 = 0 𝛼𝛼1 2 𝛼𝛼2 2 𝛼𝛼3 2 𝛽𝛽𝑁𝑁−2 2 𝛽𝛽𝑁𝑁−1 2 𝛽𝛽𝑁𝑁 2 = 0

10

𝛼𝛼0 1 = 0 𝛼𝛼1 1 𝛼𝛼2 1 𝛼𝛼3 1 𝛽𝛽𝑁𝑁−2 1 𝛽𝛽𝑁𝑁−1 1 𝛽𝛽𝑁𝑁 1 = 0

01

𝛼𝛼0 3 = 0 𝛼𝛼1 3 𝛼𝛼2 3 𝛼𝛼3 3 𝛽𝛽𝑁𝑁−2 3 𝛽𝛽𝑁𝑁−1 3 𝛽𝛽𝑁𝑁 3 = 0

11
0 1 2 3 𝑁𝑁 − 2 𝑁𝑁 − 1 𝑁𝑁

51
 Calculation in Logarithmic Domain
 Implementation with respect to probabilities is complicated
 numerical problems  implementation in the logarithmic domain favorable
 Transition variable= γ i ( s ', s ) ln p (y i s ', s ) + ln Pr {s s '}
γi ( s ', s ) ln=
1
C − 2 y i − x( s ', s ) + ln Pr {ui =
= u ( s ', s )}
2

2σ N
 Forward variable
α i ( s ) = ln α i ( s ) = ln
( ∑ γ (s ', s) ⋅ α
s'
i i −1
) ( ∑ exp ( γ (s ', s) + α
( s ') = ln
s'
i i −1 ( s ') )
)
 Backward variable
=βi −1 ( s ') ln=
βi −1 ( s ') ln
( ∑ γ (s ', s) ⋅ β (s)) = ln ( ∑ exp ( γ (s ', s) + β (s)))
s
i i
s
i i

 Initialization Terminated code otherwise

 0 s' = 0  0 s' = 0 βN ( s ) = const.
α 0 ( s ') =  βN ( s ) = 
−∞ s ' ≠ 0 −∞ s ' ≠ 0

52
 Calculation in Logarithmic Domain: Jacobi Logarithm
 In recursion, ln of sum of exponents occur

(
ln e x1 +
= e x2) (
max [ x1 , x2 ] + ln 1 + e =
1 2 − x −x
)
max * [ x1 , x2 ]

 Proof
 For x1 > x2

max * [ x1 , x2 ] = ( (
ln e x1 1 + e
−( x1 − x2 )
)) =ln ( e ) + ln (1 + e ( ) ) =x + ln (1 + e
x1 − x1 − x2
1
− x1 − x2
)
 For x1 ≤ x2

( (
max * [ x1 , x2 ] =ln e x2 1 + e
−( x2 − x1 )
)) =ln ( e ) + ln (1 + e ( ) ) =x
x2 − x2 − x1
2 (
+ ln 1 + e
− x1 − x2
)
 Second term has small range between 0 and ln 2
 efficiently be implemented by a lookup table w.r.t |x1-x2|

53
 Calculation in Logarithmic Domain: Jacobi Logarithm
 Simplify logarithm of sums ln e x1 + = e x2 ( )
max * [ x=
1 , x2 ] max [ x1 , x2 ] + ln 1 + e (
− x1 − x2
)
 Forward variable
=α i ( s ) ln=
(
α i ( s ) ln ∑ exp ( γi ( s ', s ) + α i −1 ( s ') )
s'
)
max * [ γi ( s1′, s ) + α i −1 ( s1′ ), γi ( s2′ , s ) + α i −1 ( s2′ ) ]
correction term
=
=∆i ( γi ( s1′, s) + αi −1 ( s1′) )
= max [ γi ( s ', s ) + α i −1 ( s ') ] + ln (1 + e )
− ∆i
s'
− ( γi ( s2′ , s ) + α i −1 ( s2′ ) )
 Backward variable
βi −1 ( s′) =
ln βi −1 ( s′) =
max *  γi ( s′, s1 ) + βi ( s1 ), γi ( s′, s2 ) + βi ( s2 )  correction term
= ∆ i ( γi ( s′, s1 ) + βi ( s1 ) )
= max  γi ( s′, s ) + βi ( s )  + ln (1 + e )
− ∆i
s
− ( γi ( s′, s2 ) + βi ( s2 ) )
 Declaration:
 Log-MAP: implementation of BCJR in log-domain with correction term
 Max-Log-MAP: implementation in log-domain without correction term

54
 Iterative Decoding

 General Structure for Parallel Concatenated Codes

 Turbo Decoding for (24,16,3)-Product Code
 Simulation Results
 Turbo Decoding for Serially Concatenated Codes

55
 General Concept for Iterative Decoding
 Parallel Concatenated Codes
y0
u=u1 c1 𝐿𝐿(𝑢𝑢)
� Le,1
C1 y1 D1 Π
y2 La,1
u c0

Π Π

𝐿𝐿(𝑢𝑢)
� Le,2
u2 c2 D2 Π−1
C2 La,2

systematic (message) bits

soft-decision estimates
parity bits Decoder for message bits
A priori information

56
 Turbo Decoding for (24,16,3) Modified Product Code (1)

u 1 0 0 1 x -1 +1 +1 -1 +1 LLR 0.6 7.6 1.3 -3.2 6.3

Lch · y
0 1 1 1 encoding +1 -1 -1 -1 -1 AWGN 5.1 -4.4 3.8 -0.6 -9.5
1 0 1 0 BPSK -1 +1 -1 +1 +1 -7.6 3.2 -5.7 7.6 1.3
SNR=2 dB
0 0 0 1 +1 +1 +1 -1 -1 1.3 -1.3 8.2 -9.5-12.7
+1 -1 +1 -1 1.9 -5.7 7.6 -7.0
Vertical extrinsic info serves
as horizontal a-priori info 1. vertical extrinsic
| | | decoding information
𝐿𝐿−𝑎𝑎,1 𝐮𝐮
� = 𝐿𝐿𝑒𝑒,1 𝐮𝐮
� � = Lch · y + 𝐿𝐿𝑒𝑒,1 𝐮𝐮
𝐿𝐿1 𝐮𝐮 �
|
-1.3 -1.3 -3.8 -0.6 -0.7 6.3 -2.5 -3.8 � -1.3 -1.3 -3.8 -0.6
𝐿𝐿𝑒𝑒,1 𝐮𝐮
-0.6 1.3 -1.3 -3.2 4.5 -3.1 2.5 -3.8 Lch · y -0.6 1.3 -1.3 -3.2
0.6 -1.3 1.3 0.6 -7.0 1.9 -4.4 8.2 + L|e,1 (û) 0.6 -1.3 1.3 0.6
-0.6 3.2 -1.3 -0.6 0.7 1.9 6.9 -10.1 -0.6 3.2 -1.3 -0.6

57
Turbo Decoding for (24,16,3) Modified Product Code (2)
Lch · y + 𝐿𝐿−𝑎𝑎,1 𝐮𝐮
�
-0.7 6.3 -2.5 -3.8 6.3 𝐿𝐿−𝑒𝑒,1 𝐮𝐮
� 2.5 -0.7 0.7 0.7 𝐿𝐿−1 𝐮𝐮
� 1.8 5.6 -1.8 -3.1
4.5 -3.1 2.5 -3.8 -9.5 1. horizontal -2.5 2.5 -3.1 2.5 Lch · y + 2.0 -0.6 -0.6 -1.3
-7.0 1.9 -4.4 8.2 1.3 -1.3 1.3 -1.3 1.3 -8.3 3.2 -5.7 9.5
decoding L–e,1 (û) +
0.7 1.9 6.9 -10.1-12.7 1.9 0.7 0.7 -0.7 L–a,1 (û) 2.6 2.6 7.6 -10.8
1.9 -5.7 7.6 -7.0
|
�
Lch · y + 𝐿𝐿𝑎𝑎,2 𝐮𝐮 𝐿𝐿−𝑒𝑒,1 𝐮𝐮
�
|
3.1 6.9 2.1 -2.5 6.3 = 𝐿𝐿𝑎𝑎,2 𝐮𝐮� û1 0 0 1 1
2.6 -1.9 0.7 1.9 -9.5 0 1 1 1
-8.9 4.5 -7.0 8.9 1.3 1 0 1 0
3.2 -0.6 8.9 -10.2-12.7 0 0 0 1
1.9 -5.7 7.6 -7.0

58
 Turbo Decoding for (24,16,3) Modified Product Code (3)
Lch y + L|a,2 (u)
3.1 6.9 2.1 -2.5 6.3 û2 1 0 0 1 L–2 (û) -1.9 7.6 2.1 -1.9
2.6 -1.9 0.7 1.9 -9.5 0 1 1 1 1.5 -2.1 -1.4 1.4
-8.9 4.5 -7.0 8.9 1.3 1 0 1 0 -7.0 3.9 -6.3 1.3
3.2 -0.6 8.9 -10.2-12.7 x x 0 1 0 0 6.9 0.6
1.9 -5.7 7.6 -7.0
Lch · y +
2. vertical Lch y + L–
a,2 (u) L–e,2 (û) +
decoding -1.3 7.0 0.6 -1.3 6.3 L–a,2 (û)

L|e,2 (û) -1.9 -0.6 -0.7 1.9 L2|(û) 3.2

1.2 -3.8 1.4 -3.1
6.3 1.7 0.6 -9.5 L–e,2 (û) -0.6 0.6 1.3 -0.6
-1.9 0.6 -2.1 -2.5 Lch · y + -5.7 2.6 -5.0
0.7 -1.3 5.7 1.3 2. horizontal
-1.4 -0.6 -1.7 1.7 -3.1 1.7
1.9 -0.6 0.7 -1.9 -0.6
-7.0 0.6
3.9 -6.3 7.0 -12.7 decoding
7.5 -7.6 -1.3 1.3 -1.3 1.3
L|e,2 (û)
-1.9 1.9 -0.7 1.9 1.9 1.3 7.6
1.3 -5.7 8.2 -7.0
-8.3 0.6 -0.6 -0.6 0.6

59
 Turbo Decoding for (24,16,3) Modified Product Code (4)
Lch y + L|a,3 (u)
0.0 8.2 2.6 -3.8 6.3 û3 1 0 0 1 L–2 (û) -1.9 6.3 1.9 -2.7
3.4 -2.7 0.7 1.1 -9.5 0 1 1 1 3.9 -1.3 -1.3 -3.2
-8.9 4.5 -7.0 8.9 1.3 1 0 1 0 -8.9 2.6 -6.3 8.8
1.9 -1.9 8.2 -8.9-12.7 0 0 0 1 2.7 2.7 8.8 -9.7
1.9 -5.7 7.6 -7.0
Lch · y +
3. vertical Lch y + L–a,3 (u) L–e,3 (û) +
decoding
-1.3 5.7 0.6 -2.1 6.3 L–a,3 (û)

L|e,3 (û) -1.9 -1.9 -0.7 1.1 L3|(û) -1.9

5.1 -2.5 1.9 -4.4
6.3 1.2 -2.7 -9.5 L–e,3 (û) -0.6 0.6 1.3 -0.6
0 1.9 -2.6 -3.8 Lch · y + -7.6 1.3 -5.0
3.4 -0.8 7.5 1.3 3. horizontal
-1.9 -2.7 -1.2 1.2 -2.5 1.2
0 -1.9 0.7 -1.1 1.3 1.4
-8.9 2.6 -6.3 7.8 -12.7 decoding
7.5 -8.4 -1.3 1.3 -1.3 1.3
L|
e,3 (û)
0 2.7 -0.7 1.1 0.8 7.6
1.9 -5.7 7.5 -7.0
-7.8 1.4 1.3 1.3 -1.3

60
 Turbo Decoding for Parallel Concatenated Codes
𝐿𝐿a,1 𝐮𝐮 =𝐿𝐿e,2 𝐮𝐮 −1
Π

𝐿𝐿ch ⋅ 𝐲𝐲
D1
� = 𝐿𝐿ch ⋅ 𝐲𝐲s
𝐿𝐿1 𝐮𝐮
Π D2
� = 𝐿𝐿ch ⋅ 𝐲𝐲s
𝐿𝐿2 𝐮𝐮
+𝐿𝐿e,1 𝐮𝐮 𝐿𝐿ch ⋅ 𝐲𝐲s + 𝐿𝐿a,2 𝐮𝐮
+𝐿𝐿a,1 𝐮𝐮 +𝐿𝐿a,2 𝐮𝐮
+𝐿𝐿e,2 𝐮𝐮
only Lch y2 is fed to decoder D2

 Both decoders estimate same information word u and each decoder receives
corresponding channel outputs
 Systematic information bits ys are fed to D2 via D1 and Π
 Each decoder generates extrinsic information for bit u serving as a priori
LLRs for other decoder
 A priori LLRs improve decoders’ performance in each iteration as long as they
are statistically independent of regular inputs

61
Simulation Results for Modified Product Codes (7,4,3)-Hamming Codes
0
10
It.1
 Observations
It.2
It.3  Gains decrease with
analyt. number of iterations
-2
10  Same info bits are
estimated and correlation
Pb of a-priori information
increases
-4
10
 With the larger interleaver
length the gains of
subsequent iterations are
generally larger 
-6
10 statistical independence
0 2 4 6 8 of bits is required
10log10 ( Eb / N 0 )

62
 Simulation Results for Modified Product Codes
(15,11,3)-Hamming-Codes
0
10
It.1
 Observations
It.2
It.3  Larger interleaver leads
analyt. to improved statistic
-2
10  gains for iteration 3

Pb

-4
10

-6
10
0 1 2 3 4 5 6
10log10 ( Eb / N 0 )

63
 Simulation Results for Modified Product Codes
(31,26,3)-Hamming-Codes
0
10
It.1
 Observations
It.2
It.3  Larger interleaver leads
analyt. to improved statistic
-2
10  gains for iteration 3

Pb  For larger SNR the BER

flattens
-4  minimum distance
10 dominates error rate for
large SNR

-6
10
0 1 2 3 4 5 6
10log10 ( Eb / N 0 )

64
 Simulation Results for Modified Product Codes
 Hamming codes have 𝑑𝑑min = 3 for all lengths 𝑛𝑛
 Analyzed product codes have same 𝑑𝑑min  similar error rates versus 𝐸𝐸𝑠𝑠/𝑁𝑁0
 Code rates are different  longer product codes are better versus 𝐸𝐸𝑏𝑏/𝑁𝑁0
0 0
10 10
(7,4) (7,4)
(15,11) (15,11)
(31,26) (31,26)
-2 -2
10 10
Pb Pb

-4 -4
10 10

-6 -6
10 10
0 2 4 6 0 2 4 6
10log10 ( Es / N 0 ) 10log10 ( Eb / N 0 )

65
 Simulation Results for Turbo Codes (Lc = 3)
0
10x10 Block-Interleaver 30x30 Block-Interleaver
0
10 10
It. 1 It. 1
It. 2 It. 2
-1
10 It. 3 10
-1
It. 3
It. 4 It. 4
It. 5 It. 5
-2 It. 6 It. 6
Pb 10 Pb 10
-2

-3 -3
10 10

-4 -4
10 10

-5 -5
10 10
1 2 3 4 5 6 1 2 3 4 5 6
10log10 ( Eb / N 0 ) 10log10 ( Eb / N 0 )

 Gains decrease with number of iterations

 Increase of interleaver size leads to
reduced BER

66
 Simulation Results for Turbo Codes (Lc = 3)
900-Random-Interleaver, Rc=1/3 0
Comparison of different interleavers
0
10 10
It. 1 CC, Lc=9
It. 2 BIL-100
-1 -1 BIL-400
10 It. 3 10
It. 4 BIL-900
It. 6 RIL-900
-2 It. 10 -2 RIL-900,Rc=1/3
Pb 10 Pb 10

-3 -3
10 10

-4 -4
10 10

-5 -5
10 10
1 2 3 4 5 6 1 2 3 4 5 6
Eb / N 0 in dB Eb / N 0 in dB
 Usage of random interleaver leads to  Random interleaver (RIL) achieves
significant performance improvements in larger gains in comparison to block
comparison to block interleaver interleaver (BIL)

67
 Turbo Decoding for Serially Concatenated Codes

C1 Π C2
outer encoder systematic inner encoder

𝐿𝐿𝑎𝑎,2 𝐜𝐜�1 = Π{𝐿𝐿e,1 𝐜𝐜1 } 𝐿𝐿2 𝐜𝐜1 = 𝐿𝐿ch ⋅ 𝐲𝐲�S

Π +𝐿𝐿a,1 𝐜𝐜1
+𝐿𝐿e,1 𝐜𝐜1
−1
D2 Π D1
�
𝐿𝐿 𝐮𝐮
𝐿𝐿2 𝒄𝒄�1 = 𝐿𝐿ch ⋅ 𝐲𝐲s
+𝐿𝐿e,2 𝐜𝐜�1 𝐿𝐿ch ⋅ 𝐲𝐲�S + 𝐿𝐿a,1 𝐜𝐜1
+𝐿𝐿a,2 𝐜𝐜�1

 Outer decoder receives information only from inner decoder

 Outer decoder delivers estimates on information bits u as well as extrinsic LLRs
of code bits c1 being information bits of inner code C2
 Extrinsic LLRs of code bits c1 serve as a priori LLRs for inner code C2

68
Comparison of Serial and Parallel Concatenation
0
10
serial
parallel
-1 n = 120
10 n = 1200
n = 12000
-2
10
BER

Results for specific setup,

-3
no generalization possible!
10

-4
10

-5
10
0 1 2 3 4 5 6
Eb / N 0 in dB

69
 Repeat Accumulate Code by ten Brink
 Approximately 100 decoding iterations are needed
 Half-rate outer repetition encoder and rate-one inner recursive convolutional
encoder 100

10-1

10-2

BER
10-3

10-4

10-5
0 0.1 0.2 0.3 0.4 0.5 0.6
Eb/N0 in dB

70
Repeat Accumulate Code by Stephan ten Brink

71
EXtrinsic Information Transfer Chart

(EXIT-Charts)

Stephan ten Brinn

 Mutual Information for Turbo Decoder
 Parallel Concatenation

Π−1

C D1 Π D2

73
 Mutual Information for Single Decoder

C BPSK D

74
 General Concept of Iterative „Turbo“ Decoding

 BER curve shows three different regions

 At low SNR the iterative decoding performs worse than uncoded transmission
 At low to medium SNR the iterative decoding is extremely effective  waterfall
region
 At high SNR the decoding converges already in few iterations  error floor
 How to understand this varying behavior?
 Extrinsic information is exchanged between decoders
 Analysis of iterative process by semi-analytic approach
 Determine analytically mutual information 𝐼𝐼 𝑢𝑢; 𝐿𝐿𝑎𝑎 𝑢𝑢 between information bits and
a-priori input of decoder
 Determine by simulation mutual information 𝐼𝐼 𝑢𝑢; 𝐿𝐿𝑒𝑒 𝑢𝑢 between information bits and
extrinsic output of decoder for specific a-priori information at input
 Draw relationship between both mutual information's
 Combine diagrams of both contributing decoders into one chart:
→ EXIT chart: EXtrinsic Information Transfer chart

75
 Distribution of Extrinsic Information
 Investigation of extrinsic decoder output Le ( uˆ=
i) L ( uˆi ) − Lch ⋅ yi − La ( ui )
 Example: [7,5]-RSC at Eb/N0 = 0, …, 2 dB +1)
pe ( ξ xi =
 PDF of extrinsic estimate is given for 0.2
xi = +1 and xi = -1 separately 0.0 dB
0.15 0.5 dB
1.0 dB
 Extrinsic information is nearly 0.1
1.5 dB
Gaussian distributed 0.05 2.0 dB

 With increasing SNR 0

-20 -15 -10 -5 0 5 10 15 20
 the mean’s absolute value is increased
pe ( ξ xi =
−1)
 the variance is increased 0.2
0.0 dB
0.15 0.5 dB
Iterative Decoding: With increasing 1.0 dB
0.1
number of iterations the extrinsic 1.5 dB
information approaches a Gaussian 0.05 2.0 dB

distribution 0
-20 -15 -10 -5 0 5 10 15 20

76
 Analytical Model for the A-Priori Information
 Extrinsic information of decoder 1 becomes a-priori-information of decoder 2
and vice versa
 For EXIT analysis the a-priori information A=La is modeled as A =µ A ⋅ x + nA
 Gaussian random variable nA of zero mean and variance 𝜎𝜎𝐴𝐴2 is
added to the value x of the transmitted systematic bit multiplied by µ A = 12 σ A
2

σ = σ = σ A =

 (ξ − )
2 2 2
 2
 1 16 36
σ 2 A A
A
⋅ xi 
0.4 0.06 0.035

A ( ξ xi )
1 2
p= exp −
 
0.35 0.03

2πσ A  2σ 2
A 
0.05

 
0.3
0.025
0.04
0.25

 Normalization of a-priori information with 1

𝜎𝜎𝐴𝐴2
0.02

2 0.2 0.03

 With increasing variance the probability

0.015
0.15
0.02
functions are more separated and do not 0.1
0.01

overlap anymore 0.01

0.005
0.05

0 0 0
-50 0 50 -50 0 50 -50 0 50

77
 Motivation for Modeling A-Priori Information
 LLR for uncoded transmission over AWGNC is given by (
y = x + n ~  ±1, σ 2n )
p { y | x = +1}
L ( y x ) = ln
E
= 4 s y = Lch ⋅ y = Lch ⋅ ( x + n ) with Es 1 2
p { y | x = −1} N0 =
L 4= 4 =
 2σ 2n σ 2n
ch
N0
Lch
and
L ( y x )=
2 2 N Es
⋅ x + ⋅n σ n2 = 0 σ 2x= = 1
σn
2
σn
2
2Ts Ts
 LLR is Gaussian distributed with mean mA and variance sA2

{ {( )}
2
µA = E L( y x = i) = 2 ⋅i  2 
2 2 4
σn =σ 2
E 2
⋅=n  2 = ⋅ σ 2
σ2n
σ σ 2n
A n
 n
 The mean’s absolute value equals the half of the variance
 Model for a-priori LLR  2 4 
A =La =µ A ⋅ x + nA A ~  ( ± 12 σ 2A , σ 2A =
)  ± σ2 , σ2 

 n n 

78
 Mutual Information of A-Priori Information and Info Bits
 Mutual information between systematic bits and a-priori LLR
∞
2 p A ( ξ xi )
∑ ∫ A ( i ) 2 p (ξ x =
1
I A (=
σ A ) I ( X=
; A) p ξ x log dξ
2 xi ={+1,−1} −∞ A i −1) + p A ( ξ xi =
+1)

( )
∞

∫
1
{ }
exp − 2 σ1 2 ( ξ − 12 σ 2A ) log 2 (1 + e −ξ ) d ξ = 1 − E log 2 (1 + e −ξ ) = J (σA )
2
= 1−
−∞ 2πσ A A

1
 0 £ IA £ 1 0.9

 Integral has to be solved numerically 0.8

 J(sA) is monotonically increasing in sA 0.7

 has a unique inverse function sA = J-1(IA) 0.6

 Close approximation for J-function I A ( σ A ) 0.5

J ( σ A ) I A ( σ A ) ≈ (1 − 2 )
0.4
1.1064
2⋅0.8935
−0.3073⋅σ
= A
0.3

0.2
and its inverse
1 0.1

 
log 2 (1 − I 1/1.1064 )
1 2⋅0.8935
σ A ≈ J −1 ( I A ) =  − 
0

σA
0 1 2 3 4 5 6 7 8
A
 0.3073 

79
 Mutual Information of Extrinsic Information and Info Bits
 Mutual information between systematic bits and extrinsic LLR
∞
2 pE ( ξ xi )
∑ ∫ E ( i ) 2 p (ξ x =
1
I ( X;E) =
IE = p ξ x log dξ
2 xi ={+1,−1} −∞ E i −1) + pE ( ξ xi =
+1)
 0 £ IE £ 1

 Semi analytical approach to determine the dependency of mutual information at

decoder input and output
 Perform encoding for a random information sequence u  c = f(u) and x = 1-2c
 Transmit BPSK signals over AWGN channel y=x+n
 For given IA determine sA using the inverse J-function sA= J-1(IA)
 Model a-priori information using analytical model: A = mA x + nA
 Perform decoding of noisy receive signal y using a-priori information A
 Determine mutual information IE for extrinsic information using histogram for
approximating pdf pE( ξ|xi)
 Transfer characteristic shows dependency of IE and IA I E = Tr ( I A , Eb / N 0 )

80
 Measurement of the Mutual Information
 By application of ergodic theorem (expectation is replaced by time average), the
mutual information can be measured for large number N of samples
{
I ( L; X ) = 1 − E log 2 (1 + e −L
)} 1 N
(
≈ 1 − ∑ log 2 1 + e − xn ⋅Ln
N n=1
)

 Measurement setup I ( LE , X )
( )
systematic bits − x⋅ Le
1 − log 2 1 + e Average
n
u ∈ {0,1} 2 / σ 2n L ( y x )
 Le ( x )
x ∈ {±1}  σ 2A = 4 / σ 2n′A
 2 4 
LA= A ~  ( ± 12 σ 2A , σ 2A =
)  ± σ2 , σ2 

La ( x )  n′A n′A 

n′A 2 / σ n2′A
I ( LA , X )
(
1 − log 2 1 + e − x⋅ La
) Average

81
 Dependency of Mutual Information at Decoder Input and Output
𝐿𝐿ch ⋅ 𝑦𝑦 𝐿𝐿𝑒𝑒
𝐿𝐿𝑎𝑎 Dec.
1
 Transfer characteristic for
(37,23r)8- RSC code
0.8
 Decoder processes L(y|x) and
La(x)

I E = 0.6  Observations
-0.5 db
 IE increases with growing SNR
I ( u; Le (u ) ) 0.0 db and IA
0.4 0.5 db  IA=0  no a-priori information
1.0 db available
1.5 db  IA=1  perfect a-priori
0.2 2.0 db  IE is reliable regardless of
2.5 db SNR
3.0 db  For high SNR, nearly no a-
0 priori information is required
0 0.2 0.4 0.6 0.8 1
I A = I ( u; La (u ) )
for good decoding results

82
 Behavior of different Convolutional Codes
1
 Transfer characteristic if
0.9
only a-priori information is
0.8 provided to the decoder
0.7 Lc = 3 (c.f. serial concatenation)
IE = 0.6
 Weak codes better for
I ( u; Le (u ) ) 0.5 low a-priori information
0.4  Strong codes better for
0.3 Lc = 5 high a-priori information
0.2  Point of intersection for all
convolutional codes close
0.1 Lc = 7
to (0.5,0.5)
0 (explanation for this
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
I A = I ( u; La (u ) ) behavior unknown!)

Serial concatenation: Outer decoder gets only a-priori information of inner decoder 𝐿𝐿𝑎𝑎 𝐿𝐿𝑒𝑒
Dec.
 Transfer function of outer decoder is independent of SNR

83
 Comparison of MAP and Max-Log-MAP
1 Dec.

0.8  High channel SNR leads

to high extrinsic
information
0.6
 Large a-priori information
can compensate bad
0.4 Eb / N0 = -1 dB
channel conditions
Eb / N0 = 0 dB  Max-Log-MAP decoder
Eb / N0 = 1 dB
performs nearly as good
0.2
as optimal MAP decoder
Eb / N0 = 2 dB
Eb / N0 = 3 dB
0
0 0.2 0.4 0.6 0.8 1

84
 EXtrinsic Information Transfer (EXIT) Charts
 Extrinsic information provided
by one decoder is used
as a-priori information
for other decoder D2 D1

 For EXIT charts the transfer function of both constituent codes are drawn into
one diagram with exchanging the abscissa and ordinate for the second code

 Assumptions
 A large interleaver is assumed to assure statistical independence of IA and IE
 For inner decoders in a serial concatenated scheme and for parallel concatenated
schemes the input parameters are Lch and IA
 For outer decoders in a serial concatenation only IA(outer) appears as input which is
taken form the interleaved signal IE(inner)
(Transfer function of outer decoder is independent of SNR)

85
 EXIT Charts for Serial Concatenation
1
Eb/N0 = -1.0 dB
Eb/N0 = 0.0 dB
0.8
Eb/N0 = 1.0 dB
Eb/N0 = 1.2 dB
Eb/N0 = 2.0 dB
Eb/N0 = 3.0 dB
0.6
outer decoder

 Outer non-recursive
0.4
convolutional encoder
pinch-off SNR: (15,13)8, 𝑅𝑅𝑐𝑐 = 3/4
minimum SNR 0.2
for convergence
of turbo  Inner recursive
decoder
convolutional encoder
0
0 0.2 0.4 0.6 0.8 1 (13,15r)8, 𝑅𝑅𝑐𝑐 = 2/3

86
 EXIT Charts for Serial Concatenation
1  Outer non-recursive
convolutional encoder
(15,13)8, 𝑅𝑅𝑐𝑐 = 3/4
0.8

 Inner recursive
0.6 convolutional encoder
(13,15r)8, 𝑅𝑅𝑐𝑐 = 2/3
0.4

0.2

0
0 0.2 0.4 0.6 0.8 1

87
 EXtrinsic Information Transfer (EXIT) Charts
1  1 Eb 2 
Pb ≈ erfc  + J ( I A ) + J ( IE ) 
−1 −1
2
8 Rc
2  2 2 N 0 
Outer convolutional code
1
Inner Walsh-Hadamard code
0.9
0.8
0.7
0.6
I ( u; L1e (u ) ) = I ( u; L2a (u ) ) 0.5
0.4
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
I ( u; L1a (u ) ) = I ( u; L2e (u ) )

88
 EXtrinsic Information Transfer (EXIT) Charts
 Determining pinch-off SNR: minimum SNR for which convergence is maintained
1
0.9
0.8
0.7
0.6

I ( u; L1e (u ) ) = I ( u; L2a (u ) )
0.5
0.4
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

10log10(Eb/N0) = - 0.3 dB I ( u; L1a (u ) ) = I ( u; L2e (u ) )

89
Code Design for Half-Rate Repeat-Accumulate Code
1

Signal-to-Noise ratio
0.8

0.6

0.4
Outer repetition code

0.2
outer repetition code
Inner recursive
inner code
convolutional encoder
0
0 0.2 0.4 0.6 0.8 1

90
 Bitinterleaved Coded Modulation

 General Structure for Serially Concatenated Blocks

 Calculation of LLRs
 Simulation Results

91
 Bit-Interleaved Coded Modulation (BICM)
Π

channel channel
encoder Π mapper channel demapper
Π-1 decoder

 Coded transmission with higher order modulation:

 Binary vector of length m is mapped to one of 2m symbols of the alphabet 𝕏𝕏
 Usually Gray mapping employed 𝑥𝑥 ∈ 𝕏𝕏  minimizes bit error probability without channel coding
 Good properties regarding the capacity of a BICM system
 Interpretation as serially concatenated system
 Insertion of interleaver between encoder and mapper leads to pseudo random mapping of bits
onto specific levels and is crucial for iterative turbo detection
 Iterative detection and decoding: demapper and decoder exchange extrinsic information

 How to perform turbo detection / decoding?

 Are there better mapping strategies than Gray mapping?

92
 Soft-Output Demapping
 LLR for each of the m bits (for one specific time instant 𝑘𝑘):
∑ p ( y c ) ⋅ Pr {c}
p ( y, cµ = 0 )
( cµ y ) ln p y,=
( cµ ) L= c∈GF( 2 ) , cµ =
m

L=  
dem 0
ln
( cµ = 1)
 ∑ p ( y c ) ⋅ Pr {c}
c∈GF( 2 ) , cµ =
m
1

 y−x2  m
∑ p ( y x ) ⋅ Pr { x} ∑0 exp  − σ2  ⋅ ∏ Pr {cν ( x )}
ln=
x∈ , cµ =0
ln
x∈µ  n  ν=1
∑ p ( y x ) ⋅ Pr { x}  y−x2  m
x∈ , cµ =1 ∑1 exp  − σ2  ⋅ ∏ Pr {cν ( x )}
x∈µ  n  ν =1
 A priori information 𝐿𝐿𝑎𝑎 𝑐𝑐𝜈𝜈̃ provided by decoder
e ν( ) a( ν )
m m − c x L c

∏ Pr {cν ( x )} = ∏ − La ( cν )
ν 1 = ν 1 1 + e

93
 Soft-Output Demapping
 Denominator of a priori information cancels when inserted into 𝐿𝐿dem 𝑐𝑐𝜇𝜇̃
 y − x 2  m − c ( x ) L ( c )
∑0 exp  − σ2  ⋅ ∏ e ν a ν
  ν=1
( µ) ( µ )
x∈µ n
=
Ldem 
c L=
c y ln
 y − x 2  m − c ( x ) L ( c )
∑1 exp  − σ2  ⋅ ∏ e ν a ν
x∈µ  n  ν =1
 Intrinsic information 𝐿𝐿dem 𝑖𝑖 𝑐𝑐𝜈𝜈̃ is independent of a priori information 𝐿𝐿𝑎𝑎 𝑐𝑐𝜈𝜈̃
L=
dem
i ( cµ ) Ldem ( cµ ) − La ( cµ )
 y − x 2  m −c ( x)L (c )
∑0 exp  − σ2  ⋅ ν= ∏ e ν a ν
= ln
x∈µ  n  1,ν ≠µ
 y − x 2  m −c ( x)L (c )
∑1 exp  − σ2  ⋅ ν= ∏ e ν a ν
x∈µ  n  1,ν≠µ

94
 Soft-Output Demapping for 16-QAM
Ldem ( c1 ) Ldem ( c4 )
Im Im
0111 0110 0010 0011 0111 0110 0010 0011

0101 0100 0000 0001 0101 0100 0000 0001

1101 1100 1000 1001 Re 1101 1100 1000 1001 Re

1111 1110 1010 1011 1111 1110 1010 1011

∑ exp ( − ) ⋅ ∏ Pr {c ( x )}
m
y−x
1 2
σ2n ν

Ldem ( c1 ) = ln
x∈10 ν =1

∑ exp ( − ) ⋅ ∏ Pr {c ( x )}
m
y−x
1 2
σ2n ν
x∈11 ν=1

95
 System Model for BICM

𝐜𝐜 𝑘𝑘 = 𝑐𝑐1 [𝑘𝑘] , … , 𝑐𝑐𝑚𝑚 [𝑘𝑘] 𝑥𝑥[𝑘𝑘]

𝑢𝑢 channel
encoder Π mapper

Transmitter
channel

𝐿𝐿dem
𝑖𝑖 𝑐𝑐𝜈𝜈̃
dem
𝑢𝑢� 𝐿𝐿 𝑐𝑐𝜈𝜈̃ +𝐿𝐿𝑎𝑎 𝑐𝑐𝜈𝜈̃ 𝑦𝑦[𝑘𝑘]
Π
channel -1 𝑖𝑖 soft
decoder demapper

Π 𝐿𝐿dec 𝑐𝑐𝜈𝜈̃ = 𝐿𝐿𝑎𝑎 𝑐𝑐𝜈𝜈̃

Receiver

96
 Selected Bit-Mappings for 8-PSK
011 101
010 1 1 001 110 2 2 011
1 1 3 2
110 Gray 000 001 d23 000
1 1 010 2 3
111 1 1 100 011 1 2 001 010 2 2 111
101 3 1 100
100 natural 000
101 1 3 001
101 2 1 111
110 2 2 011 110 3 2 111
1 2 110 2 Anti- 3
111 d21 000 011 000
2 1 3 Gray 2
001 2 2 100 100 2 3 101
010 010

97
 EXtrinsic Information Transfer Charts
1
I0 I2  Demapper: a priori information
Eb/N0 = 5 dB mutual information I (c; L dem
a )
8
0.8
a )
) = I (c; L dec

 Detection and decoding only

0.6 once
1 8  Gray is best
I (c; L dem

0.4
Gray  Iterative detection and decoding
i

1 natural
d21  Anti-Gray is best
0.2 d23
Anti-Gray
BCH(8,4)
0
0 0.2 0.4 0.6 0.8 1
I (c; L dem
a ) = I (c; L dec )

98
 Bit Error Rates
0
10  Simulation parameters
 BCH(8,4)
-1
10  8-PSK
 Alamouti scheme
10
-2
 360 coded bits per frame
 Independent Rayleigh fading

BER

-3 Channel const. for 24 symbols

10
 First detection and decoding
-4
10 Gray  Gray good, Anti-Gray bad
natural  After four iterations
-5 d21
10 d23  Anti-Gray is best
Anti-Gray 4 iterations
10
-6
 Same results as predicted
0 5 10 15 20 by EXIT charts
Eb/N0 [dB]

99
 Low Density Parity Check Codes

 Definition and properties of LDPC codes

 Iterative decoding
 Simulation results

100
 LDPC Codes
 Low Density Parity Check Codes
 Invented by Robert G. Gallager in his PhD thesis, 1963
 Re-invented by David J.C. Kay in 1999

 LDPC codes are linear block codes with sparse parity check matrix H
 contains relatively few ‘1’ spread among many ‘0’ (for binary codes)

 Iteratively decoded on a factor graph of the check matrix

 Advantages
 Good codes
 Low decoding complexity

101
 Introduction
 Recall: For every linear binary (n, k) code  with code rate Rc = k/n
 There is a generator matrix G ∈ GF(q)k×n such that code words x ∈ GF(q)n and info
words u ∈ GF(q)k are related by
x= u ⋅ G

 There is a parity-check matrix H ∈ GF(q)m×n of rank{H} = n-k, such that

x ⋅ HT =
0

 Relation of generator and parity check matrix

G ⋅ HT =
0

102
 Regular LDPC-Codes
 Definition: A regular (dv,dc)-LDPC code of length n is defined by a parity-check
matrix H ∈ GF(q)m×n with dv ones in each column and dc ones in each row. The
dimension of the code (info word length) is k = n – rank{H}

 Example:
 n = 8, m = 6, k = n - rank{H} = 4 (!), RC= 1/2
 dv = 3, dc = 4
1 0 0 1 1 1 0 0
1 0 1 0 1 1 0 0
 
1 0 1 1 0 1 0 0
H= 
0 1 0 1 0 0 1 1
0 1 0 0 1 0 1 1
 
0 1 1 0 0 0 1 1

103
 Regular LDPC-Codes
 Design Rate: The true rate RC and the design rate Rd are defined as
k dv
RC = and d R = 1 − with RC ≥ Rd
n d c

 Proof: The number of ones in the check matrix m·dc = n·dv. Some parity check
equations may be redundant, i.e., m ≥ n-k, and thus k = 1 − n − k ≥ 1 − m = 1 − d v
n n n dc
 The check matrices can be constructed randomly or deterministic
 Encoding
 LDPC codes are usually systematically encoded, i.e., by a systematic generator
matrix G =  I P 
 k ×k k ×n − k 
 The matrix P can be found by transforming H into another check matrix of the code,
that has the form
H′ =  − PkT×n − k I n − k ×n − k 

104
 Factor Graph
 A factor graph of a code is a graphical representation of the code constraints
defined by a parity-check matrix of this code
x ⋅ HT =
0

 The factor graph is a bipartite graph with

 a variable node for each code symbol,
 a check node for each check equation,
 an edge between a variable node and a check node if the code symbol participates
in the check equation

 Notice that each edge corresponds to one ‘1’ in the check matrix.

105
 Factor Graph
T
 Example: 1 0 0 1 1 1 0 0 
1 0 1 0 1 1 0 0 
 
1 0 1 1 0 1 0 0 
x ⋅ HT [ x0 x1  x7 ] 
=  0
0 1 0 1 0 0 1 1 
0 1 0 0 1 0 1 1 
 
 0 1 1 0 0 0 1 1 
x0 ⊕ x3 ⊕ x4 ⊕ x5 =
0 chk0 x0 n = 8 columns (code word length)
x1 chk0 n-k = 6 parity check equations
x0 ⊕ x2 ⊕ x4 ⊕ x5 =
0 chk1 chk1
x2 Each check node represents one row of

x0 ⊕ x2 ⊕ x3 ⊕ x5 =
0 chk2 x3 chk2
parity check matrix

x1 ⊕ x3 ⊕ x6 ⊕ x7 =
0 chk3 x4 chk3
x1 ⊕ x4 ⊕ x6 ⊕ x7 =
0 chk4 x5 chk4
x6
x1 ⊕ x2 ⊕ x6 ⊕ x7 =
0 chk5 chk5
x7
Variable nodes Check nodes
106
 Decoding with the Sum-Product Algorithm
 Similar to Turbo Decoding, extrinsic information is exchanged
 Check nodes „collect“ extrinsic information from the connected variable nodes
 Variable nodes „collect“ extrinsic information from the connected check nodes
Check node example Variable node example

chk0 = ∑+ Li A0 = ∑ E0k
L0
x0 i x0 k E00
chk0 chk0
boxed plus
E01 chk1
L3
x3 extrinsic info E02 chk2
L4
x4

∑L
x5 L5 E kj = +
i
i≠ j
i∈K k
 Iterative decoding procedure Stop if x ⋅ H =
T
0
 Also called „message passing“ or “believe propagation”

107
 Decoding with the Sum-Product Algorithm
 First check equation x0 ⊕ x3 ⊕ x4 ⊕ x5 = 0
 Is the check equation fulfilled? chk0 = L( x0 ) + L( x3 ) + L( x4 ) + L( x5 )
 Extrinsic information
x0 = x3 ⊕ x4 ⊕ x5 L0e ( x0 ) = L( x3 ) + L( x4 ) + L( x5 )

L(x0) = Lch y0 x0 L0e ( x3 ) = L( x0 ) + L( x4 ) + L( x5 )

L(x1) = Lch y1 x1 chk0
L(x2) = Lch y2 x2 chk1 L0e ( x4 ) = L( x0 ) + L( x3 ) + L( x5 )
L(x3) = Lch y3 x3 chk2 L0e ( x5 ) = L( x0 ) + L( x3 ) + L( x4 )
L(x4) = Lch y4 x4 chk3
L(x5) = Lch y5 x5 chk4
L(x6) = Lch y6 x6
chk5
L(x7) = Lch y7 x7

108
 Decoding with the Sum-Product Algorithm
 Second check equation x0 ⊕ x2 ⊕ x4 ⊕ x5 =
0 L1e ( x0 ) = L( x2 ) + L( x4 ) + L( x5 )
L1e ( x2 ) = L( x0 ) + L( x4 ) + L( x5 )
 Third check equation x0 ⊕ x2 ⊕ x3 ⊕ x5 =
0 L1e ( x4 ) = L( x0 ) + L( x2 ) + L( x5 )
L1e ( x5 ) = L( x0 ) + L( x2 ) + L( x4 )
x0
chk0 L2e ( x0 ) = L( x2 ) + L( x3 ) + L( x5 )
x1
x2 chk1 L2e ( x2 ) = L( x0 ) + L( x3 ) + L( x5 )
x3 chk2 L2e ( x3 ) = L( x0 ) + L( x2 ) + L( x5 )
x4
L2e ( x5 ) = L( x0 ) + L( x2 ) + L( x3 )
chk3
x5 chk4
x6
chk5
x7

 …

109
 Decoding with the Sum-Product Algorithm
 Variable update
 Collect extrinsic information of check nodes and update variable nodes

A0 = ∑ E0k
k
L(x0) = Lch y0 +A0 x0
L(x1) = Lch y1 +A1 x1 chk0
L(x2) = Lch y2 +A2 x2 chk1
L(x3) = Lch y3 +A3 x3 chk2
L(x4) = Lch y4 +A4 x4 chk3
L(x5) = Lch y5 +A5 x5 chk4
L(x6) = Lch y6 +A6 x6
chk5
L(x7) = Lch y7 +A0 x7

110
 Example: BEC

X0 1-Pq Y0
+∞ y = Y0
Pq 
L ( y ) =
= 0 y ?
?
−∞ y = Y
Pq  1
X1 Y1
1-Pq

x0 0
chk0
x1 0
x2 ? chk1
x3 0 chk2
x4 0 chk3
x5 0 chk4
x6 ? chk5
x7 0

111
 Example: BEC
 Check equations  calculate extrinsic information
x0 0
chk
L
0
1
e ( x2 ) = L (
0
x0 ) + L ( x4 ) + L ( x5 ) = +∞ L=
1
e ( x0 ) L=
1
e ( x4 ) Le ( x5 ) 0
=
1

x1 0
chk1 Le ( x2 ) = L( x0 ) + L( x3 ) + L( x5 ) = +∞
2
+∞
x2 ? +∞ 00
x3 0
0 chk2 L5 ( x ) L=
= ( x1 ) + L( x6 ) + L( x7 ) 0
e 2
x4 0 chk3
L3e ( x6 ) = L( x1 ) + L( x3 ) + L( x7 ) = +∞
x5 0 +∞ chk4
chk5 e ( x6 ) = L( x1 ) + L( x4 ) + L( x7 ) = +∞
+∞ 4
x6 ? 0 L
x7 0
=L5e ( x6 ) L=
( x1 ) + L( x2 ) + L( x7 ) 0

 Variable check
La ( x2 ) = L1e ( x2 ) + L2e ( x2 ) + L5e ( x2 ) = 0 L5e ( x2 ) = L1e ( x2 ) + L2e ( x2 ) = +∞

La ( x6 ) = L3e ( x6 ) + L4e ( x6 ) + L5e ( x6 ) = 0 L5e ( x6 ) = L3e ( x6 ) + L4e ( x6 ) = +∞

112
 Irregular LDPC-Codes
 Properties:
 Generalization of regular LDPC codes
 Lower error rates, i.e., better performance
 Irregular number of ones per column and per row
 Variable nodes of different degrees
 Check nodes of different degrees
 Example:
1 0 0 1 1 1 0 0 x0
1 0 1 0 1 1 0 0 x1 chk0
  x2 chk1
1 1 1 1 0 1 0 0
H=  x3 chk2
0 1 0 1 0 0 1 1 x4 chk3
1 1 0 1 1 0 1 1 x5
  chk4
1 1 1 1 0 0 1 1 x6
chk5
x7

113
 Irregular LDPC-Codes
 Irregular number of ones per column and per row:
 i : proportion of left (variable) nodes of degree i
 ri : proportion of right (check) nodes of degree i
x0
 In example: x1 chk0
 3 = 5 / 8 4 = 1 / 8 5 = 2 / 8 x2 chk1
 r4 = 3 / 6 r5 = 1/ 6 r6 = 2 / 6 x3 chk2
x4 chk3
 Proportions of edges: x5 chk4
 λi : proportion of edges incident to left nodes of degree i x6
chk5
 ri : proportion of edges incident to right nodes of degree i x7

 In example:
 l3 = 15 / 29 l4 = 4 / 29 l5 = 10 / 29
 r4 = 12 / 29 r5 = 5 / 29 r6 = 12 / 29

114
 Irregular LDPC-Codes
 LDPC codes are optimized via Density Evolution or EXIT analysis
 Probability density functions describing the distribution of check and variable nodes
in a parity check matrix
 Specific codes can be found via random code generation following these distributions
 PDFs will only be nearly fulfilled due to the finite number of checks and variables
 Quality may vary in such an ensemble of codes due to random generation
 Example: Rc=1/2 LDPC Code with n=4096 and k=2048
 Variable node distribution:
Degree 2 3 6 7 20
PDF 0.48394942887 0.29442753267 0.29442753267 0.074055964589 0.062432620582
Number 1986 1202 349 303 256

 Check node distribution

Degree 8 9
PDF 0.74193548387 0.25806451612
Number 1850 529

115
 0
Simulation Results
10

Irregular LPDC  Irregular and regular

Regular LPDC LDPC code
10
-1  IR as previous slide
 Regular: n=4096, k=2048
 3 ones in a column
-2  Random generation
10
 Performance
BER

 Irregular better in
-3
waterfall region
10
 Error floor depends on n
Waterfall Region IR LDPC Ensemble  lower error floor
possible
-4
10
 Remarks
Error Floor  Regular codes are
easier to attain
-5
10
0 0.5 1 1.5 2 2.5 3
Eb/N0 in dB

116
 BER Performance of LDPC Code
0  Number info bits
10
k = 9507

-1
10  Code word length
#1 N = 29507
-2
10
BER

 Code rate
-3
RC = 0.322
10
#10 #5
-4 #15
10

0 1 2 3 4 5 6
Eb=N 0 in dB

117