Coding Theory: Linear Codes

What is Coding Theory?

 Communication channels

Transmitter Channel Receiver

Data Noise Corrupted Data

 A Generic Communication Channel

Transmitter Channel Receiver

Data x Encoder Noise e Decoder Decoded Data x̂

x = (x1 , . . . , xk ) 7→ c = (c1 , . . . , cn ) −→ c+e → x̂ = (x̂1 , . . . , x̂k )

| {z }
codeword, n≥k
Encoding
An encoding for k = 1:
(x1 ) 7→ (x1 , . . . , x1 ), that
Given an alphabet A, a
is c1 = c2 = . . . = cn .
map that sends k data
symbols If the alphabet A is
(x1 , . . . , xk ) ∈ Ak to A = {0, 1}, then
n ≥ k encoded symbols (0) 7→ (0, . . . , 0) and
(c1 , . . . , cn ) ∈ An . The (1) 7→ (1, . . . , 1).
encoded vector
c = (c1 , . . . , cn ) is called
a codeword.
 Discrete Alphabets
 Arithmetic Modulo p

Modulo 2 Modulo 3

+ 0 1 · 0 1 + 0 1 2 · 0 1 2
0 0 1 0 0 0 0 0 1 2 0 0 0 0
1 1 0 1 0 1 1 1 2 0 1 0 1 2
2 2 0 1 2 0 2 1

Modulo p (a prime): take the remainder of the Euclidean

division by p.
 Discrete Alphabets
 Arithmetic Modulo p

Modulo 4

+ 0 1 2 3 · 0 1 2 3
0 0 1 2 3 0 0 0 0 0
1 1 2 3 0 1 0 1 2 3
2 2 3 0 1 2 0 2 0 2
3 3 0 1 2 3 0 3 2 1

Modulo m (m not a prime): what is the difference?

 Discrete Alphabets
 Arithmetic Modulo p

So 2 · 2 ≡ 0 (mod 4). What does it change?

 Discrete Alphabets
 Arithmetic Modulo p

So 2 · 2 ≡ 0 (mod 4). What does it change?

Actually a lot...
• If 2x = 0, it is not true that x = 0, it could also be that
x = 2.
• This also shows that a polynomial of degree 1 can have two
solutions...
• and that two numbers different from 0 once multiplied
together actually give 0.
• Also 2x = 1 does not have a solution.
Finite field Fp
Fp is finite means
|Fp | = p < ∞.
For p a prime, the set of
integers modulo p Informally, that Fp is a
represented by field means that
{0, 1 . . . , p − 1} is a computations work as
finite field, denoted by usual, namely we can
Fp . add, subtract, multiply, in
a commutative manner,
and divide as long as it is
not by 0.
Inverse in Fp
The inverse of 3 modulo
5:
For x a non-zero
element in Fp , its
(multiplicative) inverse
is the element in Fp
denoted by x−1 which
satisfies that
x · x−1 = x−1 · x = 1.
Inverse in Fp
The inverse of 3 modulo
5: 2 is the inverse of 3
For x a non-zero
since 2 · 3 ≡ 1 (mod 5).
element in Fp , its
(multiplicative) inverse Find two elements that
is the element in Fp are their own inverse
denoted by x−1 which modulo 7:
satisfies that
x · x−1 = x−1 · x = 1.
Inverse in Fp
The inverse of 3 modulo
5: 2 is the inverse of 3
For x a non-zero
since 2 · 3 ≡ 1 (mod 5).
element in Fp , its
(multiplicative) inverse Find two elements that
is the element in Fp are their own inverse
denoted by x−1 which modulo 7: 1 is its own
satisfies that inverse since 1 · 1 ≡ 1
x · x−1 = x−1 · x = 1. (mod 7), but so is 6 since
6 · 6 = 36 ≡ 1 (mod 7).
 Discrete Alphabets
 Arithmetic Modulo p

Exercise. Prove that if m is not a prime integer, then integers

modulo m cannot form a finite field.
 Discrete Alphabets
 Arithmetic Modulo p

Exercise. Prove that if m is not a prime integer, then integers

modulo m cannot form a finite field.

If m is not a prime, then m is a composite number, that is

m = ab for a, b some integers which are not zero. Then ab ≡ 0
mod m.
 Discrete Alphabets
 Arithmetic Modulo p

Exercise. Prove that if m is not a prime integer, then integers

modulo m cannot form a finite field.

If m is not a prime, then m is a composite number, that is

m = ab for a, b some integers which are not zero. Then ab ≡ 0
mod m. Now a cannot be invertible, if it were, consider a−1 and
multiply ab ≡ 0 mod m by a−1 to get b ≡ 0 mod m, a
contradiction.
 Discrete Alphabets
 Another finite field

Suppose there exists an element ω which is a zero of

X 2 + X + 1 (mod 2). Then ω 6= 0, 1,
ω2 = ω + 1 (mod 2), ω 3 = ω(ω + 1) = ω 2 + ω = 1 (mod 2).
 Discrete Alphabets
 Another finite field

Suppose there exists an element ω which is a zero of

X 2 + X + 1 (mod 2). Then ω 6= 0, 1,
ω2 = ω + 1 (mod 2), ω 3 = ω(ω + 1) = ω 2 + ω = 1 (mod 2).

F4

+ 0 1 ω ω2 · 0 1 ω ω2
0 0 1 ω ω2 0 0 0 0 0
1 1 0 ω2 ω 1 0 1 ω ω2
ω ω ω2 0 1 ω 0 ω ω2 1
ω2 ω2 ω 1 0 ω2 0 ω2 1 ω
 Discrete Alphabets
 Finite fields

We will denote by Fq a finite field with q elements.

So far, we know Fp and F4 , we will know more later, but in
the meantime, we will use the notation.
Linear Encoding Since we work over Fq :
Codewords belong to Fnq ,
Given a finite field Fq , a which is a vector space.
linear map that sends k
Since the encoding is a
data symbols
linear map, by definition
(x1 , . . . , xk ) ∈ Fkq to
(1) the sum of two
n ≥ k encoded symbols
codewords is again a
(c1 , . . . , cn ) ∈ Fnq .
codeword, and (2) a
multiple of a codeword is
again a codeword.
Linear (n, k) code The whole zero codeword
0 ∈ C.
Given a finite field Fq , a
set of codewords If c ∈ C, so is −c.
{(c1 , . . . , cn ), ci ∈ Fq } ∈ C forms a linear subspace
Fnq is said to form a of Fnq , it thus has a
linear code (or dimension, namely k, and
codebook) C if (1) the we call n the length.
sum of two codewords is
again a codeword, and
(2) a multiple of a
codeword is again a
codeword.
An (n, k) linear code C Let b1 , . . . , bk be a basis for C.
over Fq contains q k Then codewords are obtained
codewords. as every possible linear
combination:
We write |C| = q k .
x1 b1 + . . . + xk bk ,

there are q possible values for

each xi , i = 1, . . . , k.
 Linear (n, k) codes
Linear algebra

Linear algebra Linear codes

For V, W finite-dimensional Given Fkq , Fnq , fix a basis for

vector spaces, with a basis for each space, a linear encoding is
each space, a linear map can be represented by a generator
represented by a matrix in the matrix.
given bases.
Generator matrix
There are many generator
matrices.
Given a finite field Fq , a
generator matrix G for There is a unique
an (n, k) linear code C generator matrix of the
is a k × n matrix, which form G = [Ik |A] where Ik
contains as rows the is the identity matrix.
basis vectors of C. The code is said to be in
systematic form.
 Linear Codes
 Generator matrices

 
a11 a1,n−k
(x1 , . . . , xk )  Ik .. ..
= (x1 , . . . , xk , ck+1 , . . . , cn )
 
| {z } . . | {z }
information data | ak,1 ak,n−k codeword
{z }
generator matrix G
 Linear Codes
 Generator matrices

The (n, 1) repetition code

• Dimension: k = 1.
• Length: n.
• Encoding: (x1 ) 7→ (x1 , . . . , x1 ) ∈ Fnq .
 Linear Codes
 Generator matrices

The (n, 1) repetition code

• Dimension: k = 1.
• Length: n.
• Encoding: (x1 ) 7→ (x1 , . . . , x1 ) ∈ Fnq .
(x1 )[1, . . . , 1] = (x1 . . . , x1 ).
 Linear Codes
 Generator matrices

The (n, n − 1) single parity check code

• Dimension: k = n − 1.
• Length: n
Pk
• Encoding: (x1 , . . . , xk ) 7→ (x1 , . . . , xk , i=1 xi ) ∈ Fnq
 Linear Codes
 Generator matrices

The (n, n − 1) single parity check code

• Dimension: k = n − 1.
• Length: n
• Encoding: (x1 , . . . , xk ) 7→ (x1 , . . . , xk , ki=1 xi ) ∈ Fnq
P
 
1
(x1 , . . . , xk )  I ...  = (x1 . . . , xk , x1 + . . . + xk ).
 
k
1
 Linear Codes
 Generator matrices

Exercise. Consider the following codebook of length n = 4

over F2 : {(1, 0, 0, 0), (0, 1, 0, 1), (1, 1, 0, 1), (0, 0, 0, 1)}. Is this
code a linear code? If so, provide a generator matrix.
 Linear Codes
 Generator matrices

Exercise. Consider the following codebook of length n = 4

over F2 : {(1, 0, 0, 0), (0, 1, 0, 1), (1, 1, 0, 1), (0, 0, 0, 1)}. Is this
code a linear code? If so, provide a generator matrix.

The code cannot be linear, because (0, 0, 0, 0) does not

belong to the code.
 Linear Codes
 Generator matrices

Exercise. Consider the following codebook of length n = 4

over F2 : {(1, 0, 0, 0), (0, 1, 0, 1), (1, 1, 0, 1), (0, 0, 0, 1)}. Is this
code a linear code? If so, provide a generator matrix.

The code cannot be linear, because (0, 0, 0, 0) does not

belong to the code.
The code cannot be linear, because (1, 1, 0, 1) + (0, 0, 0, 1)
does not belong to the code.
 Linear Codes
 Generator matrices

Exercise. Consider the codebook of length n = 5 over F2

containing: (0, 0, 0, 0, 0),(1, 0, 0, 1, 0), (0, 1, 0, 1, 1), (1, 1, 0, 0, 1),
(0, 0, 1, 0, 1), (1, 0, 1, 1, 1), (0, 1, 1, 1, 0), (1, 1, 1, 0, 0). Is this code
a linear code? If so, provide a generator matrix.
 Linear Codes
 Generator matrices

Exercise. Consider the codebook of length n = 5 over F2

containing: (0, 0, 0, 0, 0),(1, 0, 0, 1, 0), (0, 1, 0, 1, 1), (1, 1, 0, 0, 1),
(0, 0, 1, 0, 1), (1, 0, 1, 1, 1), (0, 1, 1, 1, 0), (1, 1, 1, 0, 0). Is this code
a linear code? If so, provide a generator matrix.

The code is linear. The first three coefficients run through every
possible vectors in F32 , namely (0, 0, 0),(1, 0, 0), (0, 1, 0), (1, 1, 0),
(0, 0, 1), (1, 0, 1), (0, 1, 1), (1, 1, 1).
 Linear Codes
 Generator matrices

Exercise. Consider the codebook of length n = 5 over F2

containing: (0, 0, 0, 0, 0),(1, 0, 0, 1, 0), (0, 1, 0, 1, 1), (1, 1, 0, 0, 1),
(0, 0, 1, 0, 1), (1, 0, 1, 1, 1), (0, 1, 1, 1, 0), (1, 1, 1, 0, 0). Is this code
a linear code? If so, provide a generator matrix.

The code is linear. The first three coefficients run through every
possible vectors in F32 , namely (0, 0, 0),(1, 0, 0), (0, 1, 0), (1, 1, 0),
(0, 0, 1), (1, 0, 1), (0, 1, 1), (1, 1, 1). We next show that there is a
generator matrix (which is enough to conclude the code is
linear):  
1 0 0 a11 a12
(x1 , x2 , x3 ) 0 1 0 a21 a22  .
0 0 1 a31 a32
 Linear Codes
 Generator matrices

Exercise. Consider the codebook of length n = 5 over F2

containing: (0, 0, 0, 0, 0),(1, 0, 0, 1, 0), (0, 1, 0, 1, 1), (1, 1, 0, 0, 1),
(0, 0, 1, 0, 1), (1, 0, 1, 1, 1), (0, 1, 1, 1, 0), (1, 1, 1, 0, 0). Is this code
a linear code? If so, provide a generator matrix.

 
1 0 0 a11 a12
(1, 0, 0) 0 1 0 a21
 a22  = (1, 0, 0, a11 , a12 )
0 0 1 a31 a32
 
1 0 0 1 0
(x1 , x2 , x3 ) 0 1 0 1 1 .
0 0 1 0 1
 Linear (n, k) codes
Linear algebra

Linear algebra Linear codes

A subspace W of a vector space Given an (n, k) linear code over

V is the kernel of some linear Fq , there exists an (n − k) × n
transformation (the projection matrix H such that
onto W ).
C = {x ∈ Fnq , HxT = 0},

called a parity check matrix.

If G = [Ik |A] is a If c ∈ C, c = xG and
generator matrix for the HcT = HGT xT .
(n, k) code C, then
H = [−AT |In−k ].

We have
 
I
G = kT
T
A

and
HGT = −AT + AT = 0.
If G = [Ik |A] is a If c ∈ C, c = xG and
generator matrix for the HcT = HGT xT .
(n, k) code C, then Thus C is contained in the
H = [−AT |In−k ]. kernel of the linear map
v 7→ HvT . As H has rank
We have n − k, this map has a kernel of
  dimension k, which is the
I
G = kT
T dimension of C.
A

and
HGT = −AT + AT = 0.
 Linear Codes
 Parity check matrices

The (n, 1) repetition code

Generator matrix in systematic form:

[1, 1 . . . , 1] = [Ik |A].

| {z }
A

Parity check matrix in systematic form:

 Linear Codes
 Parity check matrices

The (n, 1) repetition code

Generator matrix in systematic form:

[1, 1 . . . , 1] = [Ik |A].

| {z }
A

Parity check matrix in systematic form:

 
−1
T  ..
[−A |In−k ] =  . I

n−k 
−1
Data - Encoder - Channel - Decoder
(n, k) linear code
Generator matrix
Parity check matrix
Fp , F4