Status of Knowledge on Non-Binary LDPC Decoders

Part I: From Binary to Non-Binary Belief Propagation Decoding

D. Declercq 1
1 ETIS - UMR8051 ENSEA/Cergy-University/CNRS France

IEEE SSC SCV Tutorial, Santa Clara, October 21st, 2010

D. Declercq (ETIS - UMR8051)

1 / 59

Outline

Introduction

Belief Propagation on a General Graph

Binary Belief Propagation Decoder

Non-Binary Belief Propagation Decoding

D. Declercq (ETIS - UMR8051)

2 / 59

Outline

Introduction

Belief Propagation on a General Graph

Binary Belief Propagation Decoder

Non-Binary Belief Propagation Decoding

D. Declercq (ETIS - UMR8051)

3 / 59

Small History of Binary LDPC Codes: Landmarks

Gallager 1962 regular LDPC codes, proof of convergence (MLD), algo. A (bit ipping), algo. B , Tanner 1981 composite codes on graphs, link with product codes and LDPC codes, MacKay 1995 Belief Propagation (BP) decoding, link with iterative turbo-decoding, irregular LDPC codes, Rich. et Urb.
2001

proof of convergence (BP), optimization of irregularity, codes approaching capacity (BEC, BI-AWGN),

Since then Optimization for other types of channels (freq. selective, multilevel, multi-user, turbo-equalization, joint source-channel coding), nding good matrices for small sizes, lowering the error oor. Golden age of LDPC codes, application in many standards.

D. Declercq (ETIS - UMR8051)

4 / 59

Small History of Non-Binary LDPC Codes: Landmarks

Gallager 1963 LDPC codes in Galois Fields, iterative hard decoding algo. B for dv = 3, MacKay 1998 Advantages for small blocks/high rates, ultra-sparse dv = 2 LDPC codes in high-order Fields,
2003-2006 2006-2010

Development of practical decoders for Non-Binary LDPC codes, Attempts to nd applications where NB-LDPC codes outperform binary LDPC codes, Golden age of Non-Binary LDPC codes ?

2010-xxx

[DAVEY 98] M. DAVEY AND D.J.C. M ACKAY, L OW DENSITY PARITY CHECK CODES OVER GF( Q ), IEEE communication letter, VOL . 2, PP 165167, J UNE 1998 [M ACKAY 99] D.J.C. M ACKAY AND M. DAVEY, E VALUATION OF GALLAGER CODES FOR SHORT BLOCK LENGTH AND HIGH RATE APPLICATIONS , proc. of IMA workshop on codes, systems and graphical models, 1999

D. Declercq (ETIS - UMR8051)

5 / 59

Denitions and Quantities

LDPC Code CH = c GF (q)N | H.c

LDPC Code
GF (q)

= 0

CG = c = G.u, u GF (q)K

with H(M N), parity check matrix, with G(N K ), generator matrix.

8 < of a codeword of information Size : of the redundancy Density of H :

N K M

R = K /N (if H is full rank)

elements in H dH = nb. nonzeroM.N

LDPC :

dH

N+

D. Declercq (ETIS - UMR8051)

6 / 59

Tanner Graph Representation of a Binary LDPC Code

Tanner Graph is a Bi-partite graph with Adjacency Matrix H LDPC: Low Density Parity Check Codes

c0 1 1 H= 0 0 0 1 1 0 0 0 1 1 1 1 0 1 1 c0 0 0 1 1 0 1 1 0 0 0 0 0 1 1

c1

c2

c3

c4

c5

c6

c7

codeword

: message 0 Parity checks

c1

c2

c3

c4

c5

c6

c7

codeword

Interleaver

Parity checks

D. Declercq (ETIS - UMR8051)

7 / 59

Tanner graph Representation of a Non-Binary LDPC code

Tanner graph of an Irregular Non-Binary LDPC code in GF (8)

c0 3 4 H= 1 0 0 6 5 0 0 0 3 1 6 2 0 0 7 c0 0 0 3 7 0 1 5 0 0 0 0 0 1 2 0

c1

c2

c3

c4

c5

c6

c7

Codeword

. . . . . . . .

: message

parity checks Codeword

c1

c2

c3

c4

c5

c6

c7

Interleaver

parity checks

D. Declercq (ETIS - UMR8051)

8 / 59

parameters for Non-Binary LDPC code irregularity

Irregularity distribution, Irregularity prole

edges proportions :

i : proportion of nonzero values {Hkl } in degree i columns, j : proportion of nonzero values {Hkl } in degree j rows,
2

node proportions :

i : proportion of columns in H with degree i, j : proportion of rows in H with degree j,

(x) =
3 dvmax X i=2

i x i1

(x) =

dcmax X j=2

j x j1

non-zero values distribution :

hij () : uniformly distributed in GF(q)\0.

D. Declercq (ETIS - UMR8051)

9 / 59

Notion of Code Family

A LDPC code family is dened by ((x), (x), N)
for characterization, proofs, theoretical studies

A xed LDPC code is dened by ((x), (x), N, , {hij })

for practical application

(x) =

4 3 2 8 3 9 8 x+ x + x + x 24 24 24 24

(x) = x 5

D. Declercq (ETIS - UMR8051)

10 / 59

Outline

Introduction

Belief Propagation on a General Graph

Binary Belief Propagation Decoder

Non-Binary Belief Propagation Decoding

D. Declercq (ETIS - UMR8051)

11 / 59

Concept of Iterative Decoder on Graph

From Local Computation to Global Optimization

The general concept of LDPC decoders is based on message passing between nodes in the Tanner graph of the code, so that iterative updates of the messages lead to a stable state of the messages: convergence to a xed point. Messages represent probability density functions of the random variables. For a discrete random variable in a set of q elements:
q1 X i=0

(0) = Prob (xn = 0|.) . . . (q 1) = Prob (xn = q 1|.)

(k) = 1

The decoding result is the a posteriori probability of one random variable xn : Prob (xn |y0 , y1 , . . . , yN1 ) a particular scheduling of the computation of the messages denes a decoding iteration.

D. Declercq (ETIS - UMR8051)

12 / 59

Terminology and some History

Belief Propagation: BP

Articial Intelligence Statistical learning : Pearls Belief Propagation (1981-86) Neural Networks : sum-product algorithm () (1985-86)

Information Theory Gallager iterative decoders for LDPC (1963), Viterbi (1967), BCJR (1974): can be analysed as BP on Factor graphs,

Statistical Physics BP = Bethe approximation of the global free energy of complex systems (1935), Generalized BP = Kikuchi approximation of the free energy (1951)

D. Declercq (ETIS - UMR8051)

13 / 59

Example
Tanner graph (for PC codes) is a special case of Factor Graph
Let A(), B(), C(), D() be dependant random variables. Let A , B , C , D be their noisy observation.

B A A p(B|A,C,D) B

With Belief propagation on a tree, we get the a posteriori density : optimal solution.

D. Declercq (ETIS - UMR8051)

14 / 59

Notations for bi-partite graphs

x xn f
variable node
(x
f , f x )

F(.)

function node

: messages = p.d.f.

The graph is not oriented: messages needed in both directions. 2 types of nodes = 2 types of local updates: data node update and function node update

D. Declercq (ETIS - UMR8051)

15 / 59

Concept of Computational Tree

Expansion of the graph from a symbol/check node.

F1

F2

Nodes seen after 1 iteration

Nodes seen after 2 iterations

LLR

LLR

LLR

D. Declercq (ETIS - UMR8051)

16 / 59

Concept of Computational Tree

Past of the graph = set of nodes.

F1

F2

Nodes seen after 1 iteration

Nodes seen after 2 iterations

LLR

LLR

LLR

Past of F1:

S F1

Past of F2:

S F2

D. Declercq (ETIS - UMR8051)

17 / 59

Concept of Computational Tree

Independence assumption

Independent F1 F2

Nodes seen after 1 iteration

Nodes seen after 2 iterations

LLR

LLR

LLR

Past of F1:

S F1

Past of F2: Disjoint

S F2

D. Declercq (ETIS - UMR8051)

18 / 59

Variable Node Update: Bayesian Merging

bitnode/symbol updates for LDPC codes

F(.) f4 F(.)
2 3
x

x1 x1
x

F(.)

f3

f2

F(.)
4 Y i=2

1 f [k] x

if

x [k]

k = 0 . . . q 1

ASSUMPTION: input messages if x are independent ASSUMPTION: noisy symbol sets leading to if x are disjoint

Update equation is NOT normalized

D. Declercq (ETIS - UMR8051)

19 / 59

Function Node Update: Bayesian Marginalization

checknode updates for LDPC codes

F( x1 x2 x, x4 ) , , 3 x1
1 f x

x4

x4
f

x3

f 2 x f

x3 x2

1 x [k1 ] = f

X
k2 ,k3 ,k4

4 Y i ` F x1 = k1 , x2 = k2 , x3 = k3 , x4 = k4 x i=2

f [ki ]

k1 = 0 . . . q 1 are independent ASSUMPTION: input messages ASSUMPTION: noisy symbol sets leading to if x are disjoint if x

D. Declercq (ETIS - UMR8051)

20 / 59

Function Node Update: Bayesian Marginalization

Non-Binary Parity-check case in GF(q)

1 x [k1 ] = f

X
k2 ,k3 ,k4

4 ` Y i F x1 = k1 , x2 = k2 , x3 = k3 , x4 = k4 x i=2

f [ki ]

let k GF(q) = 0, 1, , 2 , . . . , q1 Parity-check case: the function node reduces to an indicator function: F x1 = 1 , x2 = 2 , x3 = 3 , x4 = 4 = 1 if 1 + 2 + 3 + 4 = 0 F x1 = 1 , x2 = 2 , x3 = 3 , x4 = 4 = 0 if 1 + 2 + 3 + 4 = 0

Parity-check case: results in one less sum dimension in the marginalization: X 1 x [1 ] = 2 f [2 ] 3 f [3 ] 4 f [1 + 2 + 3 ]. f x x x

2 ,3

D. Declercq (ETIS - UMR8051)

21 / 59

Scheduling and Denition of Iteration

This ordering of the messages is called ooding schedule One decoding iteration = APP

D. Declercq (ETIS - UMR8051)

22 / 59

Concept of Computational Tree (contd)

APP

Nodes seen after 1 iteration

Nodes seen after 2 iterations

LLR

LLR

LLR

Computational span of L iterations: in L iterations, a maximum of dv (dv 1)L1 (dc 1)L nodes are seen from the top of the tree. As a consequence, a usual assumption is that the BP decoder needs at least L = log(N) iterations to converge (to see all LLRs). As a consequence, the independence assumption for the BP decoder breaks after at most L = log(N) iterations.

D. Declercq (ETIS - UMR8051)

23 / 59

Concept of Computational Tree (contd)

Breaking the independance assumption

wrong APP wrong update

wrong update

Nodes seen after 1 iteration

Nodes seen after 2 iterations

LLR

LLR

LLR

a crucial parameter of the graph is its girth g, i.e. the size of the smallest closed path/cycle,

As a consequence, only g/4 decoding iterations correspond to an exact inference !!!

D. Declercq (ETIS - UMR8051)

24 / 59

Alternate Scheduling of messages (1)

Layered BP or Shufed Scheduling

D. Declercq (ETIS - UMR8051)

25 / 59

Alternate Scheduling of messages (2)

Layered BP or Shufed Scheduling

D. Declercq (ETIS - UMR8051)

26 / 59

Alternate Scheduling of messages (3)

Layered BP or Shufed Scheduling

D. Declercq (ETIS - UMR8051)

27 / 59

Alternate Scheduling of messages (4)

Layered BP or Shufed Scheduling

Advantage: for bitnodes with degree dv 3 Messages are computed several times during ONE iteration Faster convergence.

D. Declercq (ETIS - UMR8051)

28 / 59

Outline

Introduction

Belief Propagation on a General Graph

Binary Belief Propagation Decoder

Non-Binary Belief Propagation Decoding

D. Declercq (ETIS - UMR8051)

29 / 59

Binary Belief Propagation Algorithm in the Log-Domain

denition of messages in the log-domain
[0] k f x k [1] f x [0] k x f k [1] x f

uk = log

vk = log

message update through the 2 types of nodes

U0 (LLR) C Uk Vm

vm = u0 +

dv X k =1,k=m

uk

tanh

uk = 2

dc Y m=1;m=k

tanh

vm 2

Vm

Uk

D. Declercq (ETIS - UMR8051)

30 / 59

From the Probability-Domain to the Log-Domain (1)

Bitnode Update

dv f [k] = x

dY v 1 i=1

if

x [k]

k = 0, 1

Let us consider a dv = 3 bitnode with v3 as output message:

v3

= = =

log log

3 f [0] x 3 f [1] x 0 [0] 1 x [0] 1 x [0] f f

0 [1] 1 x [1] 1 x [1] f f u0 + u1 + u2

D. Declercq (ETIS - UMR8051)

31 / 59

From the Probability-Domain to the Log-Domain (2)

Checknode Update

dc x [dc ] f

X
1 ,...,dc 1

dY c 1 i=1

ix f [i ]

! X
k

k = 0

k = 0, 1

Let us consider a dc = 3 bitnode with u3 as output message: 3 x [0] f 3 x [1] f = = 1 f [0]2 f [0] + 1 f [1]2 f [1] x x x x 1 f [0]2 f [1] + 1 f [1]2 f [0] x x x x

D. Declercq (ETIS - UMR8051)

32 / 59

From the Probability-Domain to the Log-Domain (3)

Checknode Update Intermediate step: decoding in the Fourier Domain

Now compute the factorization of the sum ... (3 x [0] + 3 x [1]) f f = (1 f [0] + 1 f [1]) (2 f [0] + 2 f [1]) x x x x

... and the factorization of the difference (3 x [0] 3 x [1]) f f = (1 f [0] 1 f [1]) (2 f [0] 2 f [1]) x x x x

"

we can write in vector form: #" # " 1 1 3 x [0] 1 f = 1 1 3 x [1] 1 f

1 1

#"

1 f [0] x 1 f [1] x

#!

"

1 1

1 1

#"

2 f [0] x 2 f [1] x

#!

D. Declercq (ETIS - UMR8051)

33 / 59

From the Probability-Domain to the Log-Domain (4)

Checknode Update Intermediate step: decoding in the Fourier Domain

with the denitions of the Fourier Transforms F = 1 1 1 1 F 1 =

1 2

1 1

1 1

we obtain the checknode update in the Fourier Domain: " 3 x [0] f 3 x [1] f # = F 1 F " 1 f [0] x 1 f [1] x # F " 2 f [0] x 2 f [1] x #!

D. Declercq (ETIS - UMR8051)

34 / 59

From the Probability-Domain to the Log-Domain (5)

link between the probability domain and the log-domain 3 f [0] = x eu3 eu3 + 1 3 f [1] = x 1 e u3 + 1

From previous equations, we have: (3 x [0] 3 x [1]) = (1 f [0] 1 f [1]) (2 f [0] 2 f [1]) x x x x f f e u3 1 = e u3 + 1 eu3 /2 eu3 /2 = eu3 /2 + eu3 /2 tanh e v2 1 e v2 + 1 ! ! v1 /2 ev1 /2 e ev2 /2 ev2 /2 ev1 /2 + ev1 /2 ev2 /2 + ev2 /2 ev1 1 ev1 + 1

u3 v1 v2 = tanh tanh 2 2 2

D. Declercq (ETIS - UMR8051)

35 / 59

Final Step: Remove all Products

Lets compute the BP checknode update in the Log-Domain: |u3 | 2 |v1 | |v2 | + log tanh 2 2

log tanh

log tanh

The sign of the message is computed in a parallel stream: u3 sign tanh 2 sign (u3 ) = v1 v2 sign tanh sign tanh 2 2 sign (v1 ) sign (v2 )

D. Declercq (ETIS - UMR8051)

36 / 59

Binary Belief Propagation Algorithm in the Log-Domain

denition of messages in the log-domain
k [0] f x k [1] f x k [0] x f k [1] x f

uk = log

vk = log

message update through the 2 types of nodes

U0 (LLR) C Uk Vm

log tanh
dv X k =1,k =m

|uk | 2

dc X m=1;m=k dc Y m=1;m=k

log tanh

|vm | 2
Vm

Uk

vm = u0 +

uk

sign(uk ) =

sign(vm )

D. Declercq (ETIS - UMR8051)

37 / 59

From the Log-Domain BP to the Min-Sum decoder

From previous equations: 3 x [0] f 3 x [1] f = = 1 f [0]2 f [0] + 1 f [1]2 f [1] x x x x 1 f [0]2 f [1] + 1 f [1]2 f [0] x x x x

u3

= =

log log

3 x [0] f 3 x [1] f

= =

e v1 e v2 1 1 + v e v1 + 1 e v2 + 1 e 1 + 1 ev2 + 1 1 1 ev2 e v1 + v log ev1 + 1 ev2 + 1 e 1 + 1 e v2 + 1 ` ` log ev1 +v2 + 1 log ev1 + ev2 max (v1 + v2 , 0) max (v1 , v2 )

` where max (x, y) = log ex + ey denotes the Jacobian Logarithm.

D. Declercq (ETIS - UMR8051)

38 / 59

From the Log-Domain BP to the Min-Sum decoder

After some transformations: u3 = = max (v1 + v2 , 0) max (v1 , v2 ) max(v1 + v2 , 0) max(v1 , v2 ) + log sign(v1 ) sign(v2 ) min (|v1 | , |v2 |) + log 1 + e|v1 +v2 | 1 + e|v1 v2 | !

1 + e|v1 +v2 | 1 + e|v1 v2 |

The additionnal term log

1 + e|v1 +v2 | 1 + e|v1 v2 |

! can be replaced by a constant value.

noting that this term is negative when v1 and v2 have the same sign, and the term is positive when v1 and v2 have different signs, ...

D. Declercq (ETIS - UMR8051)

39 / 59

We nally get the Corrected Min-Sum decoder

1

Bitnode update: same as for BP

dv X k=1,k=m

vm = u0 +

uk

Checknode update: 0 uk = @
dc Y m=1;m=k

1 sign(vm )A . min |vm |

m=k

Compensation/Correction: uk = max(0, uk ) uk = min(0, uk + ) if uk > 0 if uk < 0

D. Declercq (ETIS - UMR8051)

40 / 59

Comments on the different Decoding Algorithms

Shufed Scheduling can be parallelized if the LDPC is properly designed increased throughput, Shufed Scheduling converges approximately 2 to 3 times faster than ooding schedule reduced latency,

Bit-ipping, Gal-A and Gal-B: easier to get theorems on theoretical performance, Min-Sum with proper offset correction approaches BP for regular or slightly irregular LDPC codes, In some particular cases, the Min-Sum decoder can surpass the BP decoder in the error oor region.

D. Declercq (ETIS - UMR8051)

41 / 59

Outline

Introduction

Belief Propagation on a General Graph

Binary Belief Propagation Decoder

Non-Binary Belief Propagation Decoding

D. Declercq (ETIS - UMR8051)

42 / 59

Belief Propagation in the Probability Domain

Check Node Equations: augmenting the Factor graph representation
Now the code is dened from Non Binary Parity Check equations
dc X j=1

hij .cj = 0

in GF (q)

with GF (q) = 0, 0 , 1 , . . . , q1

c1 c1
f
f

c2

c3
Permutation Nodes

c2

c3

p
x

h 1c1
p h1c1 + h2c2 + h3c 3
c

h 2c2
c
p

h 3c3

D. Declercq (ETIS - UMR8051)

43 / 59

Belief Propagation in the Probability Domain

Variable Node Equations

Now the code is dened from Non Binary Parity Check equations
dc X j=1

hij .cj = 0

in GF (q)

with GF (q) = 0, 0 , 1 , . . . , q1

Variable node Update is the Kronecker product of all incomming messages: dv p [k ] = v

dY v 1 i=1

ip v [k]

k = 0, . . . , q 1

Or in vector form:
1 dv p = 0 v . . . dv v v p p

D. Declercq (ETIS - UMR8051)

44 / 59

Explaining the Permutation Step

cyclic permutation / rotation

GF(q) is a cyclic Field, as such, multiplication by hij acts on the symbols as a cyclic permutation of the Field elements:

ip c [k ] = iv

p [k ]

k = hij k

k = 0, . . . , q 1

GF(8)
0 0 1 2 3 4 5 6

GF(8)
0 0= 1= 2= 3= 4= 5= 6= 2 5 2 6 2 0 2 1 2 2 2 3 2 4

Internal operation in GF(8)

ci

h ji . ci

D. Declercq (ETIS - UMR8051)

45 / 59

Belief Propagation in the Probability Domain

Check Node Equations
Now the code is dened from Non Binary Parity Check equations
dc X j=1

hij .cj = 0

in GF (q)

with GF (q) = 0, 0 , 1 , . . . , q1

Check node Update is still a Bayesian marginalization. Case of dc = 3 and GF(4):

h 1c1

h 2c2
c

h 3c3
c1 \ c2 0 1 2 3 0 0 1 2 3 1 1 0 3 2 2 2 3 0 1 3 3 2 1 0

c +

D. Declercq (ETIS - UMR8051)

46 / 59

Belief Propagation in the Probability Domain

Check Node Equations

h 1c1

h 2c2
c

h 3c3
c1 \ c2 0 1 2 3 0 0 1 2 3 1 1 0 3 2 2 2 3 0 1 3 3 2 1 0

c +

Check node Update is still a Bayesian marginalization. Case of dc = 3 and GF(4):

3 p [0] = 1 c [0]2 c [0] + 1 c [1]2 c [1] + 1 c [2]2 c [2] + 1 c [3]2 c [3] c p p p p p p p p 3 p [1] = 1 c [0]2 c [1] + 1 c [1]2 c [0] + 1 c [2]2 c [3] + 1 c [3]2 c [2] c p p p p p p p p 3 p [2] = 1 c [0]2 c [2] + 1 c [2]2 c [0] + 1 c [1]2 c [3] + 1 c [3]2 c [1] c p p p p p p p p 3 p [3] = 1 c [0]2 c [3] + 1 c [0]2 c [3] + 1 c [1]2 c [2] + 1 c [2]2 c [1] p p p p p p c p p

The number of terms in the above equations grows as q 2 .

D. Declercq (ETIS - UMR8051) 47 / 59

Belief Propagation in the Probability Domain

How to simplify the checknode update ?

h 1c1

h 2c2
c

h 3c3

c + Fourier ?

D. Declercq (ETIS - UMR8051)

48 / 59

Tensorial Notation of Messages

in case of binary extension elds GF(2p ), the symbols c GF(q) could be represented by a binary map, or a polynomial: c = [c1 , . . . , cp ] c(x) =
p X i=1

with with

{c1 , . . . , cp } {0, 1} {c1 , . . . , cp } {0, 1}

ci x i1

Let put the probability weights (c = k ) in a size-2, p-dimensional tensor indexed by binary values {c1 , . . . , cp }.
Prob(c(x)=0)

C=

Prob(c(x)=1) Prob(u(x)=x) Prob(c(x)=1+x)

C[0,0] C[1,0] = C[0,1] C[1,1]

C[i,j]=

D. Declercq (ETIS - UMR8051)

49 / 59

Tensorial Notation of Messages

a GF(8) example

Prob(c(x)=0) Prob(c(x)=1) Prob(c(x)=x) Prob(c(x)=x 2) Prob(c(x)=1+x) Prob(c(x)=x+x 2) Prob(c(x)=1+x+x 2) Prob(c(x)=1+x 2)

C=

C[0,0,0] C[1,0,0] C[0,1,0] C[0,0,1] = C[1,1,0] C[0,1,1] C[1,1,1] C[1,0,1]

C[i,j,k]=

D. Declercq (ETIS - UMR8051)

50 / 59

Tensorial Notation of Messages

a GF(8) example

Prob(c(x)=0) Prob(c(x)=1) Prob(c(x)=x) Prob(c(x)=x 2) Prob(c(x)=1+x) Prob(c(x)=x+x 2) Prob(c(x)=1+x+x 2) Prob(c(x)=1+x 2)

C=

C[0,0,0] C[1,0,0] C[0,1,0] C[0,0,1] = C[1,1,0] C[0,1,1] C[1,1,1] C[1,0,1]

C[i,j,k]=

D. Declercq (ETIS - UMR8051)

51 / 59

Tensorial Notation of Messages

a GF(8) example

Prob(c(x)=0) Prob(c(x)=1) Prob(c(x)=x) Prob(c(x)=x 2) Prob(c(x)=1+x) Prob(c(x)=x+x 2) Prob(c(x)=1+x+x 2) Prob(c(x)=1+x 2)

C=

C[0,0,0] C[1,0,0] C[0,1,0] C[0,0,1] = C[1,1,0] C[0,1,1] C[1,1,1] C[1,0,1]

C[i,j,k]=

D. Declercq (ETIS - UMR8051)

52 / 59

Tensorial Notation of Messages

a GF(8) example

Prob(c(x)=0) Prob(c(x)=1) Prob(c(x)=x) Prob(c(x)=x 2) Prob(c(x)=1+x) Prob(c(x)=x+x 2) Prob(c(x)=1+x+x 2) Prob(c(x)=1+x 2)

C=

C[0,0,0] C[1,0,0] C[0,1,0] C[0,0,1] = C[1,1,0] C[0,1,1] C[1,1,1] C[1,0,1]

C[i,j,k]=

D. Declercq (ETIS - UMR8051)

53 / 59

Fast Fourier Transform applied to Tensors

Expression of the Fourier Transform C = F (C) = C 1 F 2 F . . . p F

F =

1 1

1 1

where k denotes the tensor product in the k-th dimension of the tensor C(i1 , . . . , ip ).

in the k-th dimension, (i1 , . . . , ik 1 , ik+1 , . . . , ip ) {0, 1}p1

C (i1 , . . . , ik 1 , 0, ik +1 , . . . , ip ) = C(i1 , . . . , ik1 , 0, ik +1 , . . . , ip ) + C(i1 , . . . , ik 1 , 1, ik +1 , . . . , ip ) C (i1 , . . . , ik 1 , 1, ik +1 , . . . , ip ) = C(i1 , . . . , ik1 , 0, ik +1 , . . . , ip ) C(i1 , . . . , ik 1 , 1, ik +1 , . . . , ip )

for the Fourier Transform in one dimension, we perform 2p = q operations, the total number of operations for F (.) is then p 2p = q log(q) operations: Fast Fourier Transform

D. Declercq (ETIS - UMR8051)

54 / 59

Illustration of the FFT in multiple dimensions

1 1 1 1 1

1 1

GF(8) : 3 dimensions
1 1 1 1

1 1 GF(4) : 2 dimensions 1
1

1 1

1 1

D. Declercq (ETIS - UMR8051)

55 / 59

Belief Propagation Decoding Steps in the Fourier Domain

product Information Symbols

Vpv Vcp

U vp U pc

permutation Permutation Nodes

Interleaver

Fourier Tranform

F
Fourier

Product Node product

D. Declercq (ETIS - UMR8051)

56 / 59

Belief Propagation in the Log-Domain (1)

recursive use of the max operator.

Quantization impacts on the performance are very strong in the Probability Domain

u(k) = log

c p [k] c p [0]

k = 0, . . . , q 1

v (k) = log

p c [k] p c [0]

k = 0, . . . , q 1

Case of dc = 3 and GF(4):

3 p [0] = 1 c [0]2 c [0] + 1 c [1]2 c [1] + 1 c [2]2 c [2] + 1 c [3]2 c [3] c p p p p p p p p 3 p [1] = 1 c [0]2 c [1] + 1 c [1]2 c [0] + 1 c [2]2 c [3] + 1 c [3]2 c [2] c p p p p p p p p 3 p [2] = 1 c [0]2 c [2] + 1 c [2]2 c [0] + 1 c [1]2 c [3] + 1 c [3]2 c [1] c p p p p p p p p 3 p [3] = 1 c [0]2 c [3] + 1 c [0]2 c [3] + 1 c [1]2 c [2] + 1 c [2]2 c [1] c p p p p p p p p

D. Declercq (ETIS - UMR8051)

57 / 59

Belief Propagation in the Log-Domain (2)

Limitations

After some manipulations: u3 (1) u3 (2) u3 (3) K = = = = max (v1 (1) , v2 (1) , v1 (2) + v2 (3) , v1 (3) + v2 (2)) K max (v1 (2) , v2 (2) , v1 (1) + v2 (3) , v1 (3) + v2 (1)) K max (v1 (3) , v2 (3) , v1 (1) + v2 (1) , v1 (2) + v2 (1)) K max (0 , v1 (1) + v2 (1) , v1 (2) + v2 (2) , v1 (3) + v2 (3))

The number of max operators grows in O(q 2 ), Its a recursive implementation: approximations (e.g. use of max instead of max , small LUT) become rapidly catastrophic, Log-Domain implementation and the FFT complexity reduction O(q 2 ) O(q log(q)) are not compliant.

D. Declercq (ETIS - UMR8051)

58 / 59

Conclusion on Non-Binary Belief Propagation Decoding

The bottleneck of the decoder complexity is the check node update

Belief Propagation in the Time/Probability-Domain,

[Davey,1998] M. DAVEY AND D.J.C. M ACKAY, L OW DENSITY PARITY CHECK CODES OVER GF( Q ), IEEE communication letter, VOL . 2, PP 165167, J UNE 1998

Belief Propagation in the Time/Log-Domain (limits to GF(16)),

[Wymeersch,2004] H. W YMEERSCH , H. S TEENDAM AND M. M OENECLAEY, L OG DOMAIN DECODING OF LDPC CODES
OVER

GF (q), Proceedings of IEEE ICC, PARIS , F RANCE , J UNE 2004.

Belief Propagation in the Frequency/Probability Domain,

[Davey,1998] M. DAVEY AND D.J.C. M ACKAY, L OW DENSITY PARITY CHECK CODES OVER GF( Q ), IEEE communication letter, VOL . 2, PP 165167, J UNE 1998 [Barnault,2003] L. B ARNAULT AND D. D ECLERCQ , F AST DECODING ALGORITHM FOR LDPC CODES OVER GF (2q ), Proceedings of IEEE Information theory workshop, PARIS , F RANCE , M ARCH , 2003.

Belief Propagation in the Frequency/Log-Domain (partially - limits to GF(16)),

[Song,2003] H. S ONG AND J.R. C RUZ , R EDUCED -C OMPLEXITY D ECODING OF Q- ARY LDPC C ODES FOR M AGNETIC R ECORDING , IEEE Transactions on Magnetics, VOL . 39(2), M ARCH 2003

D. Declercq (ETIS - UMR8051)

59 / 59