Coding Theory: Linear Codes and Distances

A Generic Communication Channel

Transmitter Channel Receiver

Data x Encoder Noise e Decoder Decoded Data x̂

x = (x1 , . . . , xk ) ∈ Fkq 7→ c = (c1 , . . . , cn ) = (x1 , . . . , xk ) [Ik |A] ∈ Fnq

| {z } | {z }
codeword, n≥k systematic
generator matrix
Generator matrix Parity check matrix

C = {c = xG, x ∈ Fkq }
C ⊥ = {c = xH, x ∈ C = {v ∈ Fnq , HvT = 0}
Fn−k
q } C ⊥ = {v ∈ Fnq , GvT = 0}
Hamming distance

For two vectors

x, y ∈ Fnq , their
Hamming distance
d(x, y) is the number of
coordinates in which
they differ.

(source: wikipedia)
 Hamming Distance
 Axioms of distance

d(x, y) ≥ 0 : d counts a number of coordinates, it varies

between 0 and n for all x, y ∈ Fnq .
d(x, y) = 0 ⇐⇒ x = y : d(x, y) = 0 means x and y differ
in zero coordinate.
d(x, y) = d(y, x) : d counts the differences between x and
y (which is the same as between y and x).
d(x, z) ≤ d(x, y) + d(y, z) : suppose x and z differ in one
coordinate; it cannot be that y agrees with both x and z
on this coordinate. If y agrees with either x or z, this
contributes to 1 to d(x, y) + d(y, z), if y disagrees with
both, it contributes to 2.
Weight of a vector
• wt((1, 0, 1, 0, 0)) = 2
For x ∈ Fnq , its weight • d(x, y) = wt(x − y) for
wt(x) counts the x, y ∈ Fnq . Indeed, the
number of nonzero vector x − y will have
coordinates of x. zero coordinates exactly
where x and y agree on
their coordinates.
Hamming distance of C
• This works only for linear
codes.
For an (n, k) linear code
C over Fq , its minimum • If dH (C) = d, the notation
distance dH (C) is the (n, k, d) is sometimes
minimum weight of the used.
nonzero codewords of C.
 Hamming Distance
 Examples

The (n, 1) repetition code

• The dimension is k = 1 and the length is n.

• (x1 ) 7→ (x1 , . . . , x1 ).
 Hamming Distance
 Examples

The (n, 1) repetition code

• The dimension is k = 1 and the length is n.

• (x1 ) 7→ (x1 , . . . , x1 ).
• Its minimum distance is dH (C) = n: indeed, there is only
one codeword which is not zero, namely (1, . . . , 1) whose
weight is n.
 Hamming Distance
 Examples

The (n, n − 1) single parity check code

• The dimension is k = n − 1 and the length is n.

• (x1 , . . . , xk ) 7→ (x1 , . . . , xk , ki=1 xi ).
P
 Hamming Distance
 Examples

The (n, n − 1) single parity check code

• The dimension is k = n − 1 and the length is n.

• (x1 , . . . , xk ) 7→ (x1 , . . . , xk , ki=1 xi ).
P

• Its minimum distance is dH (C) = 2: indeed, the codeword

(1, 0, . . . , 0, 1) has weight 2. It is not possible to have a
codeword with weight 1, because if there is a single data
symbol which is not zero, then the parity symbol is also
not zero. And if there are at least two data symbols, then
the weight is at least 2.
 Hamming Distance
 Examples

The (7, 4) Hamming code

• It has dimension k = 4, length n = 7 and alphabet F2 .

• Encoding given by the generator matrix:
 
1 0 0 0 0 1 1
 0 1 0 0 1 0 1 
G=  0 0 1 0
,
1 1 0 
0 0 0 1 1 1 1
so
x 7→ (x1 , x2 , x3 , x4 , x2 + x3 + x4 , x1 + x3 + x4 , x1 + x2 + x4 ).
 Hamming Distance
 Examples

The (7, 4) Hamming code

• x 7→ (x1 , x2 , x3 , x4 , x2 + x3 + x4 , x1 + x3 + x4 , x1 + x2 + x4 ).
• If x1 = 1 and x2 = x3 = x4 = 0, then we get a weight of 3.
If we have two data symbols that are not zero, say x1 and
x2 , the only way to get a smaller weight would be that all
parities are 0, which is not possible since x1 and x2 appear
at different positions. Thus dH (C) = 3.
 Hamming Distance
 Examples

The (4, 2) tetracode

• It has dimension k = 2, length n = 4 and alphabet F3 .

• Encoding given by the generator matrix:
 
1 0 1 1
0 1 1 −1

so a codeword is of the form (x1 , x2 , x1 + x2 , x1 − x2 ).

 Hamming Distance
 Examples

The (4, 2) tetracode

• It has dimension k = 2, length n = 4 and alphabet F3 .

• Encoding given by the generator matrix:
 
1 0 1 1
0 1 1 −1

so a codeword is of the form (x1 , x2 , x1 + x2 , x1 − x2 ).

• If say x1 = 1 and x2 = 0, then we get a weight of 3. To get
a smaller weight, we could try to have both x1 , x2 non-zero,
but then the parities should both be 0, which is not
possible. So dH (C) = 3.
 Hamming Distance
 Examples

n k Fq name dH
n 1 Fq repetition n
n n−1 Fq single parity check 2
7 4 F2 Hamming 3
4 2 F3 tetracode 3
Distance and Erasures
Any two codewords differ
in d coordinates.
An (n, k) linear code C
over Fq with minimum If up to d − 1 coordinates
Hamming distance are missing, there is still
dH (C) = d can recover at least one left to
from d − 1 erasures. recognize the codeword.
 Erasure Recovery
 Examples

The (4, 3, 2) single parity check code

Over F2 , codewords are

(0, 0, 0, 0), (1, 0, 0, 1), (0, 1, 0, 1), (1, 1, 0, 0),

(0, 0, 1, 1), (1, 0, 1, 0), (0, 1, 1, 0), (1, 1, 1, 1).

If we receive (0, ∗, 1, 0),

 Erasure Recovery
 Examples

The (4, 3, 2) single parity check code

Over F2 , codewords are

(0, 0, 0, 0), (1, 0, 0, 1), (0, 1, 0, 1), (1, 1, 0, 0),

(0, 0, 1, 1), (1, 0, 1, 0), (0, 1, 1, 0), (1, 1, 1, 1).

If we receive (0, ∗, 1, 0), we must have sent (0, 1, 1, 0).

If we receive (0, ∗, 1, ∗),
 Erasure Recovery
 Examples

The (4, 3, 2) single parity check code

Over F2 , codewords are

(0, 0, 0, 0), (1, 0, 0, 1), (0, 1, 0, 1), (1, 1, 0, 0),

(0, 0, 1, 1), (1, 0, 1, 0), (0, 1, 1, 0), (1, 1, 1, 1).

If we receive (0, ∗, 1, 0), we must have sent (0, 1, 1, 0).

If we receive (0, ∗, 1, ∗), we could have sent (0, 1, 1, 0) or
(0, 0, 1, 1).
 Erasure Recovery
 Examples

The (4, 2, 3) tetracode

Over F3 , codewords are of the form

(x1 , x2 , x1 + x2 , x1 − x2 ):

(0, 0, 0, 0), (0, 1, 1, 2), (0, 2, 2, 1), (1, 0, 1, 1),

(1, 1, 2, 0), (1, 2, 0, 2), (2, 0, 2, 2), (2, 1, 0, 1), (2, 2, 1, 0).

If we receive (0, ∗, ∗, 0),

 Erasure Recovery
 Examples

The (4, 2, 3) tetracode

Over F3 , codewords are of the form

(x1 , x2 , x1 + x2 , x1 − x2 ):

(0, 0, 0, 0), (0, 1, 1, 2), (0, 2, 2, 1), (1, 0, 1, 1),

(1, 1, 2, 0), (1, 2, 0, 2), (2, 0, 2, 2), (2, 1, 0, 1), (2, 2, 1, 0).

If we receive (0, ∗, ∗, 0), we must have sent (0, 0, 0, 0).

If we receive (∗, ∗, 1, ∗),
 Erasure Recovery
 Examples

The (4, 2, 3) tetracode

Over F3 , codewords are of the form

(x1 , x2 , x1 + x2 , x1 − x2 ):

(0, 0, 0, 0), (0, 1, 1, 2), (0, 2, 2, 1), (1, 0, 1, 1),

(1, 1, 2, 0), (1, 2, 0, 2), (2, 0, 2, 2), (2, 1, 0, 1), (2, 2, 1, 0).

If we receive (0, ∗, ∗, 0), we must have sent (0, 0, 0, 0).

If we receive (∗, ∗, 1, ∗), we could have sent (0, 1, 1, 2) or
(1, 0, 1, 1) or (2, 2, 1, 0).
Parity check matrix and
C = {v ∈ Fnq , HvT = 0}
Hamming distance
If c ∈ C, HcT = 0.
If c ∈ C, an (n, k) code, If hi are columns of H,
the columns of the then
parity check matrix H
corresponding to the c1 h1 +c2 h2 +. . .+hn cn = 0.
nonzero coordinates of c
are linearly dependent. The nonzero ci create a
linear dependency among
their columns.
Parity check matrix and
If hi are columns of H
Hamming distance
and
Conversely, if a linear v1 h1 +v2 h2 +. . .+hn vn = 0
dependency with
nonzero coefficients for exactly w nonzero vi .
exists among w columns Then for v = (v1 , . . . , vn ),
of H, then there is a HvT = 0.
codeword of weight w
Since
whose nonzero
C = {v ∈ Fnq , HvT = 0}
coordinates correspond
we have that v ∈ C.
to these columns.
Parity check matrix and
C has minimum Hamming
Hamming distance
distance d if and only if
there is a codeword c of
A linear code C has
weight d but no codeword
minimum Hamming
of weight less than d.
distance d if and only if
Therefore there are
its parity check matrix
linearly dependent
H has a set of d linearly
columns of H
dependent columns but
corresponding to the
no set of d − 1 linearly
nonzero coordinates of c,
dependent columns.
and no d − 1 columns that
are linearly dependent.
 Hamming Distance
 Examples

The (n, 1) repetition code

dH (C) = d if and only if H has a set of d linearly dependent

columns but no set of d − 1 linearly dependent columns.
• parity check matrix:
 
−1
 .. 
 . In−1 
−1

• We have n linearly dependent columns since the first colum

is obtained as minus the sum of the others, but any set of
n − 1 columns is linearly independent, thus dH (C) = n.
 Hamming Distance
 Examples

The (7, 4) Hamming code

dH (C) = d if and only if H has a set of d linearly dependent

columns but no set of d − 1 linearly dependent columns.
•  
0 1 1 1 1 0 0
H= 1 0 1 1 0 1 0 
1 1 0 1 0 0 1
• Columns 4,5,6,7 are linearly dependent, but so are columns
1,6,7. All two columns are linearly independent, so
dH (C) = 3.
 Hamming Distance
 Examples

n k Fq name dH recovers from

n 1 Fq repetition nX n − 1 erasures
n n−1 Fq single parity check 2 1 erasure
7 4 F2 Hamming 3X 2 erasures
4 2 F3 tetracode 3 2 erasures
Weight of a vector
Hamming distance
Connection to erasure recovery
Two ways of computing the Hamming
distance