Crypto 03
Crypto 03
Cyclic codes
CHAPTER 3: CYCLIC CODES, CHANNEL CODING, LIST
DECODING
Cyclic codes are very special linear codes. They are of large interest and importance for
several reasons:
They posses a rich algebraic structure that can be utilized in a variety of ways.
They have extremely concise specifications.
Their encodings can be efficiently implemented using simple machinery - shift
registers.
Many of the practically very important codes are cyclic.
Channel codes are used to encode streams of data (bits). Some of them, as
Concatenated codes and Turbo codes, reach theoretical Shannon bound concerning
efficiency, and are currently used very often in practice.
List decoding is a new decoding mode capable to deal, in an approximate way, with cases
of many errors, and in such a case to perform better than classical unique decoding.
Locally decodable codes can be seen as theoretical extreme of coding theeory with deep
theoretical implications.
prof. Jozef Gruska IV054 3. Cyclic codes 2/82
IMPORTANT NOTE
In order to specify a non-linear binary code with 2k codewords of length n one may need
to write down
2k
codewords of length n.
In order to specify a linear binary code of the dimension k with 2k codewords of length n
it is sufficient to write down
k
codewords of length n.
0 0 0 1 1 1 1
is equivalent to a cyclic code.
(iii) The binary linear code {0000, 1001, 0110, 1111} is not cyclic, but it is equivalent to
a cyclic code.
(iv) Is Hamming code Ham(2, 3) with the generator matrix
» –
1 0 1 1
0 1 1 2
(a) cyclic?
(b) or at least equivalent to a cyclic code?
prof. Jozef Gruska IV054 3. Cyclic codes 4/82
FREQUENCY of CYCLIC CODES
Comparing with linear codes, cyclic codes are quite scarce. For example, there are 11 811
linear [7,3] binary codes, but only two of them are cyclic.
Trivial cyclic codes. For any field F and any integer n ≥ 3 there are always the following
cyclic codes of length n over F :
No-information code - code consisting of just one all-zero codeword.
Repetition code - code consisting of all codewords (a, a, . . . ,a) for a ∈ F .
Single-parity-check code - code consisting of all codewords with parity 0.
No-parity code - code consisting of all codewords of length n
For some cases, for example for n = 19 and F = GF (2), the above four trivial cyclic
codes are the only cyclic codes.
c2 = 0101110
c1 = 1011100 c3 = 0010111
c1 + c3 = 1001011
c1 + c2 = 1110010 c2 + c3 = 0111001
c1 + c2 + c3 = 1100101
and it is cyclic because the right shifts have the following impacts
c2 → c3 ,
c1 → c2 , c3 → c1 + c3
c1 + c3 → c1 + c2 + c3 ,
c1 + c2 → c2 + c3 , c2 + c3 → c1
c1 + c2 + c3 → c1 + c2
If x 3 + x + 1 is divided by x 2 + x = 1, then
x 3 + x + 1 = (x 2 + x + 1)(x − 1) + x
or C can be seen as
a set of polynomials of the degree (at most) n − 1
a0 + a1 x + a2 x 2 + . . . + an−1 x n−1
For any polynomial f (x), the set of all polynomials in Fq [x] of degree less than deg (f (x)),
with addition and multiplication modulo f (x), forms a ring denoted Fq [x]/f (x).
Example Calculate (x + 1)2 in F2 [x]/(x 2 + x + 1). It holds
(x + 1)2 = x 2 + 2x + 1 ≡ x 2 + 1 ≡ x (mod x 2 + x + 1).
How many elements has Fq [x]/f (x)?
Result |Fq [x]/f (x)| = q deg (f (x)) .
Example Addition and multiplication tables for F2 [x]/(x 2 + x + 1)
+ 0 1 x 1+x • 0 1 x 1+x
0 0 1 x 1+x 0 0 0 0 0
1 1 0 1+x x 1 0 1 x 1+x
x x 1+x 0 1 x 0 x 1+x 1
1+x 1+x x 1 0 1+x 0 1+x 1 x
deg (a(x)) < deg (f (x)), deg (b(x)) < deg (f (x)).
If f (x) is not reducible, then it is said to be irreducible in Fq [x].
Theorem The ring Fq [x]/f (x) is a field if f (x) is irreducible in Fq [x].
prof. Jozef Gruska IV054 3. Cyclic codes 10/82
RING (Factor ring) Rn = Fq [x]/(x n − 1)
Theorem A binary code C of words of length n is cyclic if and only if it satisfies two
conditions
(i) a(x), b(x) ∈ C ⇒ a(x) + b(x) ∈ C
(ii) a(x) ∈ C , r (x) ∈ Rn ⇒ r (x)a(x) ∈ C
Proof
(1) Let C be a cyclic code. C is linear ⇒
(i) holds.
(ii)
If a(x) ∈ C , r (x) = r0 + r1 x + . . . + rn−1 x n−1 then
r (x)a(x) = r0 a(x) + r1 xa(x) + . . . + rn−1 x n−1 a(x)
is in C by (i) because summands are cyclic shifts of a(x).
(2) Let (i) and (ii) hold
Taking r (x) to be a scalar the conditions (i) and (ii) imply linearity of C .
Taking r (x) = x the conditions (i) and (ii) imply cyclicity of C .
and therefore
r (x) ≡ −q(x)g (x) (mod x n − 1) and
r (x) ∈ C ⇒ r (x) = 0 ⇒ g (x) is therefore a factor of x n − 1.
Definition If
C = hg (x)i,
for a cyclic code C , then g is called the generator polynomial for the code C .
The last claim of the previous theorem gives a recipe to get all cyclic codes of the
given length n in GF(q)
Therefore, we have the following generator polynomials and cyclic codes of length 3.
x 7 − 1 = (x − 1)(x 3 + x + 1)(x 3 + x 2 + 1)
Since we want to determine binary codes, all computations should be modulo 2 and
therefor all minus signs can be replaced by plus signs. Therefore
x 7 + 1 = (x + 1)(x 3 + x + 1)(x 3 + x 2 + 1)
h(x) = hk + hk−1 x + . . . + h0 x k
input
output
input
m 0 g0
(m0 g1 + m1 g0 )x
+
(m0 g2 + m1 g1 + m2 g0 )x 2
+
(m0 g3 + m1 g2 + m2 g1 + m3 g0 )x 3
+
..
.
prof. Jozef Gruska IV054 3. Cyclic codes 26/82
HAMMING CODES as CYCLIC CODES I
Theorem If p(x) is a primitive polynomial over GF (2) of degree r , then the cyclic code
hp(x)i is the code Ham (r , 2).
{0, 1, x, x 2 , x 3 = x + 1, x 4 = x 2 + x, x 5 = x 2 + x + 1, x 6 = x 2 + 1}
The parity-check matrix for a cyclic version of Ham (3, 2)
0 1
1 0 0 1 0 1 1
H = @0 1 0 1 1 1 0A
0 0 1 0 1 1 1
(a0 , a1 , . . . , ar −1 )>
and consider the binary r × (2r − 1) matrix
r
H = [1 α α2 . . . α2 −2 ].
Let now C be the binary linear code having H as a parity check matrix.
Since the columns of H are all distinct non-zero vectors of V (r , 2), C = Ham (r , 2).
Putting n = 2r − 1 we get
C = {f0 f1 . . . fn−1 ∈ V (n, 2)|f0 + f1 α + . . . + fn−1 αn−1 = 0} (1)
= {f (x) ∈ Rn |f (α) = 0 in F2 [x]/p(x)} (2)
If f (x) ∈ C and r (x) ∈ Rn , then r (x)f (x) ∈ C because
r (α)f (α) = r (α) • 0 = 0
and therefore, by one of the previous theorems, this version of Ham (r , 2) is cyclic.
prof. Jozef Gruska IV054 3. Cyclic codes 29/82
EXAMPLES of CYCLIC CODES
Golay codes G24 and G23 were used by spacecraft Voyager I and Voyager II to transmit
color pictures of Jupiter and Saturn. Generator matrix for G24 has the form
01 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 1 1 0 0 0 1 01
B0 1 0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 1 1 1 0 0 0 1C
B0 0 1 0 0 0 0 0 0 0 0 0 1 1 0 1 1 0 1 1 1 0 0 0C
B C
B0 0 0 1 0 0 0 0 0 0 0 0 1 0 1 0 1 1 0 1 1 1 0 0C
B C
B0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 1 0 1 1 0 1 1 1 0C
B C
B0
G =B 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 0 1 1 0 1 1 1CC
B0 0 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 0 1 1 0 1 1C
B C
B0 0 0 0 0 0 0 0 1 0 0 0 1 1 1 1 0 0 0 1 0 1 1 0C
B C
B0 0 0 0 0 0 0 0 0 1 0 0 1 0 1 1 1 0 0 0 1 0 1 1C
@ A
0 0 0 0 0 0 0 0 0 0 1 0 1 1 0 1 1 1 0 0 0 1 0 1
0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 1 1 1 0 0 0 1 0
G24 is (24, 12, 8)-code and the weights of all codewords are multiples of 4. G23 is
obtained from G24 by deleting last symbols of each codeword of G24 . G23 is
(23, 12, 7)-code. It is a perfect code.
Golay code G23 is a (23, 12, 7)-code and can be defined also as the cyclic code generated
by the codeword
11000111010100000000000
This code can be constructed via factorization of x 23 − 1.
In his search for perfect codes Golay observed that
3
!
X 23
= 223−12 = 211
j=0
j
Golay code G24 was used in NASA Deep Space Missions - in spacecraft Voyager 1 and
Voyager 2. It was also used in the US-government standards for automatic link
establishment in High Frequency radio systems.
Example For the binary polynomial code with n = 5 and m = 2 generated by the
polynomial g (x) = x 2 + x + 1 all codewords are of the form:
a(x)g (x)
where
a(x) ∈ {0, 1, x, x + 1, x 2 , x 2 + 1, x 2 + x, x 2 + x + 1}
BCH codes and Reed-Solomon codes belong to the most important codes for
applications.
Definition A polynomial p is said to be minimal for a complex number x in GF (q) if
p(x) = 0 and p is irreducible over GF (q).
Definition A cyclic code of codewords of length n over GF (q), where q is a power of a
prime p, is called BCH code1 of the distance d if its generator g (x) is the least common
multiple of the minimal polynomials for
ω l , ω l+1 , . . . , ω l+d−2
for some l, where
ω is the primitive n-th root of unity.
m
If n = q − 1 for some m, then the BCH code is called primitive.
Applications of BCH codes: satelite communications, compact disc players,disk drives,
two-dimensional bar codes,...
Comments: For BCH codes there exist effcient variations of syndrome decoding. A
Reed-Solomon code is a special primitive BCH code.
1
BHC stands for Bose and Ray-Chaudhuri and Hocquenghem who discovered these codes in 1959.
prof. Jozef Gruska IV054 3. Cyclic codes 35/82
REED-SOLOMON CODES - basic idea behind - I
Having more than k points of the polynomial allows to determine exactly, through the
Lagrangian interpolation, the original polynomial (message).
Variations of Reed-Solomon codes are obtained by specifying ways distinct points are
generated and error-correction is performed.
They are very useful especially in those applications where one can expect that errors
occur in bursts - such as ones caused by solar energy.
Reed-Solomon (RS) codes were discovered in 1960 and since that time they have been
applied in CD-ROOMs, wireless communications, space communications, DVD, digital
TV.
There several mathematical nontrivial descriptions of RS codes. However the basic idea
behind is quite simple.
PM = m0 + m1 x + m2 x 2 + . . . + mk−1 x k−1
Reed-Solomon codes RSC(k, q), for k ≤ q. are codes generator matrix of which has rows
labeled by polynomials X i , 0 ≤ i ≤ k − 1, columns are labelled by elements
0, 1, . . . , q − 1 and the element in a row labeled by a polynomial p and in a column
labeled by an element u is p(u).
Channel coding is concerned with an efficient encoding of the streams of data and
sending them, at the highest possible rate, over a given communication channel
and then obtaining the original data reliably, at the receiver side, by decoding the
received data efficiently.
Shannon’s channel coding theorem says that over many common channels there exist
data coding schemes that are able to transmit data reliably at all rates smaller than a
certain threshold, called nowadays the Shannon channel capacity of a given channel.
As already mentioned, a breakthrough came when D. Forney, in his PhD thesis in 1972,
showed that concatenated codes could be used to achieve exponentially decreasing error
probabilities at all data rates less than the Shannon channel capacity, with decoding
complexity increasing only polynomially with the code block length.
Let X and Y be raandom variables representing the input and output of the channel.
C = sup I (X , Y )
PX (x)
where „ «
XX PX ,Y (x, y )
I (X , Y ) = PX ,Y (x, y ) log
PX (x)PY (y )
y ∈Y x∈X
The task of channel coding is to encode streams of data in such a way that if they
are sent over a noisy channel errors can be detected and/or corrected by the receiver.
The code rate express the amount of redundancy in the code - the lower is the
rate, the more redundant is the code.
Codes with lower code rate can usually correct more errors. Consequently, the
communication system can operate
with a lower transmit power;
transmit over longer distances;
tolerate more interference from the environment;
use smaller antennas;
transmit at a higher data rate.
These properties make codes with lower code rate energy efficient.
On the other hand such codes require larger bandwidth and decoding is usually of higher
complexity.
The selection of the code rate involves a tradeoff between energy efficiency and
bandwidth efficiency.
Central problem of channel encoding: encoding is usually easy, but decoding is usually
hard.
prof. Jozef Gruska IV054 3. Cyclic codes 45/82
CONVOLUTION CODES
as follows
C =I ·G
EXAMPLE 1
(x 3 + x + 1) · G1 = (x 3 + x + 1) · (x 2 + 1, x 2 + x + 1)
= (x 5 + x 2 + x + 1, x 5 + x 4 + 1)
EXAMPLE 2
„ «
1+x 0 x +1
(x 2 + x, x 3 + 1) · G2 = (x 2 + x, x 3 + 1) ·
0 1 x
The way infinite streams are encoded using convolution codes will be Illustrated on the
code CC1 .
and
C1 (x) = C10 + C11 x + . . . = (x 2 + x + 1)I (x).
The first multiplication can be done by the first shift register from the next figure; second
multiplication can be performed by the second shift register on the next slide and it holds
C0i = Ii + Ii+2 , C1i = Ii + Ii−1 + Ii−2 .
That is the output streams C0 and C1 are obtained by convolving the input stream with
polynomials of G1 .
input
will multiply the input stream by x 2 + 1 and the second shift register
output
input
The following shift-register will therefore be an encoder for the code CC1
input
output streams
For BIAGWN channels, that well capture deep space channels, this limit is (by so-called
Shannon-Hartley theorem):
„ «
S
R < W log 1 + {bits per second}
N
Till 1993 channel code designers were unable to develop codes with performance
close to Shannon capacity limit, that is so called Shannon capacity approaching
codes, and practical codes required about twice as much energy as theoretical
minimum predicted.
Therefore, there was a big need for better codes with performance (arbitrarily)
close to Shannon capacity limits.
Concatenated codes and Turbo codes have such a Shannon capacity approaching
property.
prof. Jozef Gruska IV054 3. Cyclic codes 53/82
CONCATENATED CODES - I
The basic idea of concatenated codes is extremely simple. Input is first encoded by one
code C1 and C1 -output is then encoded by second code C2 . To decode, at first C2
decoding and then C1 decoding are used.
In 1972 Forney showed that concatenated codes could be used to achieve exponentially
decreasing error probabilities at all data rates less than channel capacity in such a way
that decoding complexity increases only polynomially with the code block length.
Let Cout : B K → B N be an [N, K , D] code over alphabet B with |B| = |A|k symbols.
Concatenation of Cout (as outer code) with Cin (as inner code), denoted Cout ◦ Cin is the
[nN, kK , dD] code
Cout ◦ Cin : AkK → AnN
that maps an input message m = (m1 , m2 , . . . , mK ) to a codeword
0 0 0
(Cin (m1 ), Cin (m2 ), . . . , Cin (mN )), where
0 0 0
(m1 , m2 , . . . , mN ) = Cout (m1 , m2 , . . . , mK )
super
channel
super
channel
Of the key importance is the fact that if Cin is decoded using the maximum-likelihood
principle (thus showing an exponentially decreasing error probability with increasing
length) and Cout is a code with length N = 2n r that can be decoded in polynomial time
in N, then the concatenated code can be decoded in polynomial time with respect to
n2nr and has exponentially decreasing error probability even if Cin has exponential
decoding complexity.
super
channel
When the primary antenna failed to deploy on the Galileo mission to Jupiter in 1977,
heroic engineering effort was undertaken to design the most powerful concatenated code
conceived up to that time, and to program it into the spacecraft computer.
The inner code was a 214 convolution code, decoded by the Viterbi algorithm.
The system achieved a coding gain of more than 10dB at decoding error probabilities of
the order 10−7 . Original antenna was supposed to send 100,000 bits per second. With a
small antenna only 10 b/s could be sent. After all reparations and new codings it was
possible to send up to 1000 b/s.
Nowadays when so called iterative decoding is used concatenation of even very simple
codes can yield superb performance.
Channel coding was revolutionized by invention of Turbo codes. Turbo codes were
introduced by Berrou, Glavieux and Thitimajshima in 1993. Turbo codes are specified by
special encodings.
A Turbo code can be seen as formed from the parallel composition of two (convolution)
codes separated by an interleaver (that permutes blocks of data in a fixed
(pseudo)-random way).
A Turbo encoder is formed from the parallel composition of two (convolution)
encoders separated by an interleaver.
input x i
interleaver
convolution
encoder parity bit b2
interleaver
convolution
encoder parity bit b2
n020kceeacj
and if the same four positions are damaged the output will be
n020kc....j
However, after the inverse permutation the output actually is
c.n.j.200k.
which is easy to decode correctly!!!!
prof. Jozef Gruska IV054 3. Cyclic codes 62/82
DECODING and PERFORMANCE of TURBO CODES
Though Shannon developed his capacity bound already in 1940, till recently code
designers were unable to come with codes with performance close to theoretical limit.
In 1990 the gap between theoretical bound and practical implementations was still
at best about 3dB
A decibel is a relative measure. If E is the actual energy and Eref is the theoretical
lower bound, then the relative energy increase in decibels is
E
10 log10
Eref
Since log10 2 = 0.3 a two-fold relative energy increase equals 3dB.
For code rate 12 the relative increase in energy consumption is about 4.8 dB for
convolution codes and 0.98 for Turbo codes.
Turbo codes encoding devices are usually built from two (usually identical) recursive
systematic convolution encoders , linked together by nonuniform interleaver
(permutation) devices.
Soft decoding is an iterative process in which each component decoder takes
advantage of the work of other at the previous step, with the aid of the original
concept of intrinsic information.
For sufficiently large size of interleavers , the correcting performance of turbo codes,
as shown by simulations, appears to be close to the theoretical shannon limit.
Permutations performed by interleaver can often by specified by simple polynomials
that make one-to-one mapping of some sets {0, 1, . . . , q − 1}.
In the unique decoding model of error-correction, considered so far, the task is to find,
for a received (corrupted) message wc , the closest codeword w (in the code used) to wc .
This error-correction task/model is not sufficiently good in case when the number of
errors can be large.
In the list decoding model the task is for a received (corrupted) message wc and a given
to output (list of) all codewords with the distance at most ε from wc .
List decoding is considered to be successful in case the outputted list contains the
codeword that was sent.
It has turned out that for a variety of important codes, including the Reed-Solomon
codes, there are efficient algorithms for list decoding that allow to correct a large variety
of errors.
For a polynomial-time list decoding algorithm to exist we need that any Hamming ball of
a radius pn around a received word (where p is the fraction of errors in terms of the block
length n) has a small number of codewords.
This is because the list size itself is a lower bound for the running time of the algorithm.
Hence it is required that the list size has to be polynomial in the block length of the
code.
For a given q-nary input word w of length n and a given error bound ε the task is to
output a list of codewords of C whose Hamming distance from w is at most ε
If for every q-nary word w of length n the number of codewords of C withing Hamming
distance pn from w is at most L, then the code C is said to be (p, L)-list-decodable.
Theorem let q ≥ 2, 0 ≤ p ≤ 1 − 1/q and ε ≥ 0 then for large enough block length n if
the code rate R ≤ 1 − Hq (p) − ε, then there exists a (p, O(1/ε)))-list decodable code.
[Hq (p) = p logq (q − 1) − p logq p − (1 − p) logq (1 − p) is q-ary entropy function.]
Moreover, if R > 1 − Hq (p) + ε, then every (p, L)-list-decodable code has L = q Ω(n)
APPENDIX
Reed-Solomon codes have been widely used in mass storage systems to correct the
burst errors caused by media defects.
Special types of Reed-Solomon codes have been used to overcome unreliable nature
of data transmission over erasure channels.
Several bar-code systems use Reed-Solomon codes to allow correct reading even if a
portion of a bar code is damaged.
Reed-Solomon codes were used to encode pictures sent by the Voyager spacecraft.
Modern versions of concatenated Reed-Solomon/Viterbi decoder convolution coding
were and are used on the Mars Pathfinder, Galileo, Mars exploration Rover and
Cassini missions, where they performed within about 1-1.5dB of the ultimate limit
imposed by the shannon capacity.
APPENDIX
A group G is a set of elements and an operation, call it *, with the following properties:
G is closed under *; that is if a, b ∈ G , so is a ∗ b.
The operation * is associative, hat is a ∗ (b ∗ c) = (a ∗ b) ∗ c, for any a, b, c ∈ G .
G has an identity e element such that e ∗ a = a ∗ e = a for any a ∈ G .
Every element a ∈ G has an inverse a−1 ∈ G , such that a ∗ a−1 = a−1 ∗ a = e.
A group G is called an Abelian group if the operation ∗ is commutative, that is
a ∗ b = b ∗ a for any a, b ∈ G .
A ring R is a set with two operations + (addition) and · (multiplication) , having the
following properties:
R is closed under + and ·.
R is an Abelian group under + (with a unity element for addition called zero).
The associative law for multiplication holds.
R has an identity element 1 for multiplication
The distributive law holds: a · (b + c) = a · b + a · c for all a, b, c ∈ R.
A ring is called a commutative ring if multiplication is commutative.
A field F is a set with two operations + (addition) and · (multiplication) , with the
following properties:
F is a commutative ring.
Non-zero elements of F form an Abelian group under multiplication.
A non-zero element g is a primitive element of a field F if all non-zero elements of F
are powers of g .
Theorem If p is a prime, then the integers modp, GF (p), constitute a field. Every finite
field F contains a subfield that is GF (p), up to relabeling, for some prime p and p · α = 0
for every α ∈ F .
If a field F contains the prime field GF (p), then p is called the characteristic of F .
Theorem (1) Every finite field F has p m elements for some prime p and some m.
(2) For any prime p and any integer m there is a unique (up to isomorphism) field of p m
elements GF (p m ).
(3) If f (x) is an irreducible polynomial of degree m in Fp [x], then the set of polynomials
in Fp [x] with additions and multiplications modulo f (x) is a field with p m elements.
GF(4) = {0, 1, ω, ω 2 }
2. Let Z2 [x] be the set of polynomials whose coefficients are integers mod 2. GF(4) is
also Z2 [x] (mod x 2 + x + 1) therefore the set of polynomials
0, 1, x, x + 1
3. Let p be a prime and Zp [x] be the set of polynomials with coefficients mod p. If p(x)
is a irreducible polynomial modp of degree n, then Zp [x] (mod p(x)) is a GF(p n ) with
p n elements.
prof. Jozef Gruska IV054 3. Cyclic codes 82/82