Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Detection and Correction of Errors In Coding

Theory

ADWIP KASHYAP

Roll No-MSI20012

Department Of Mathematical Sciences

Tezpur University

October 10, 2024

1 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Outline

1 Introduction

2 Pre-requisite

3 Encoding

4 Error detection

5 Error Correction

6 Applications

2 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Introduction

Communications are done through some medium called as

channels. Mostly channels are not noise free that is almost every
channels have disturbances. Although the channels are unreliable,
but we can make reliable communication to a large extent with the
help of some techniques or method. We call it coding.
In coding, we add some extra digits to our message. They are
called as parity check digits. While sending this coded message
there may be certain errors which need to be detected and if
possible corrected.
A good code has the capability of detecting or correcting as much
as errors having minimum number of parity check digits.
The process of encoding, detecting and correction of codes is
introduced in this presentation.

3 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite

Coding
A coding is a rule that assigns each source symbol by exactly one
code word uniquely.

4 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite

Coding
A coding is a rule that assigns each source symbol by exactly one
code word uniquely.

Block Code
A block code of length n is a subset of n - tuples over q - code
symbols.

4 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite

Coding
A coding is a rule that assigns each source symbol by exactly one
code word uniquely.

Block Code
A block code of length n is a subset of n - tuples over q - code
symbols.

Code Rate
The code rate of a code of length n where information bits is K is
defined by
C = Kn

4 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite
Binary Symmetric Channel
One simple communication channel is Binary Symmetric Channel
(BSC).

Figure: Binary Symmetric Channel

In BSC, the input and output is always 0 and 1.

Error probability is same.
Error probability in each bit is independent of error probability
in previous bits.
5 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite

Hamming Distance
For a given two words, U = (u1 , u2 , · · · , un ) and
V = (v1 , v2 , · · · , vn ) then the Hamming Distance is defined as
d(U, V ) = Number of indices, where i = 1, 2, 3 · · · , n with ui ̸= vi .
For example: U = (1, 2, 0, 3) and V = (0, 2, 1, 3) then d(U, V ) = 2

6 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Pre-requisite

Hamming Distance
For a given two words, U = (u1 , u2 , · · · , un ) and
V = (v1 , v2 , · · · , vn ) then the Hamming Distance is defined as
d(U, V ) = Number of indices, where i = 1, 2, 3 · · · , n with ui ̸= vi .
For example: U = (1, 2, 0, 3) and V = (0, 2, 1, 3) then d(U, V ) = 2

Minimum Distance
The minimum distance of a block code is the smallest of all
distances between any two codewords. In other words, if C is a
block code of length n, then the minimum distance of C is defined
as
d(C ) = min{d(u, v )| u, v ∈ C }

6 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

Let us consider a Binary Symmetric Channel. We apply the

encoding as follows:

0 −→ 000
1 −→ 111

This type of codes are called Repetition Code (K3 ).

Let us do the encoding on the codeword 1010. So, 1010 becomes

1010 −→ 111 000 111 000

We will now send this encoded codeword.

7 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

Now due to some noises in the channel, let the received codeword
be 111 010 110 000. If we perform the decoding process now, then
we get

111 010 110 000 −→ 111 000 111 000 −→ 1010

If a1 a2 a3 is the received vector then we decode the received vector

by majority i.e if 1 is occurred more times, then we decode it as
111 and if 0 occurred more times we decode it as 000.

Let the received codeword is 101 111 001 000 then

101 000 011 000 −→ 111 000 111 000 −→ 1010

In this case, the decoding took place correctly.

8 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

But, when the received codeword is 010 111 001 000 then

010 111 001 000 −→ 000 111 000 000 −→ 0100

In this case, the decoding took place incorrectly.

From the above example, it can be seen that the decoding can
take place correctly if only at most single component get disturbed
and decode incorrectly if at least two components get disturbed.

If p is the probability of incorrect decoding then

Perr (K3 ) = p 3 + 3p 2 (1 − p)

If p=0.001, then Perr (K3 ) = 3 ∗ 10−6 .

9 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

If we apply the encoding as follows:

0 −→ 00000
1 −→ 11111

This type of codes are called Repetition Code (K5 ).

The decoding can take place correctly if only at most two

component get disturbed and decode incorrectly if at least three
components get disturbed.

The probability of incorrect decoding then

Perr (K5 ) = p 5 + 5p 4 (1 − p) + 10p 3 (1 − p)2

If p=0.001, then Perr (K5 ) = 10−8 .

10 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

It can be seen that the average number of bits getting disturbed

per symbols is getting decreased when the number of parity check
digits increases.

So, if we continue with the repetition code K7 , K9 , · · · we can

keep the error probability as small as we want. So,

Perr → 0 as n → ∞

11 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Encoding of Codewords

This simple code is very useful code but it has some

disadvantages also. It takes lots of space and time.
Speed of communication is measured for a code by code rate
or information rate.
Repetition code (Kn ) has code rate ( n1 ) which is very low for
higher value of n.

12 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Better Code
Let us consider a code
Information Bits Codeword
000 000 000
100 100 011
010 010 101
001 001 110
101 101 101
110 110 110
111 111 111

This code has a capability of correcting only single errors. Its code
rate R is
3 1
R= 6 = 2
13 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Better Code

This code rate is better than K3 .

Error Probability of the code = 1- (probability of correct decoding )

= 1 − {(1 − p)6 + 6(1 − p)5 p }

If p=0.001, then

= 1 − {(1 − 0.001)6 + 6(1 − 0.001)5 0.001} = 1.5 ∗ 10−5

Hence, this code is better than K3 from code rate point of view
and not from error probability point of view.

14 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Error Detection

Error Detection
The error detection concept is based on the idea that if we receive
a word and if received word is not a codeword. We call it detection
of error.
A code is said to detect t or less errors if for each codeword u, the
received word u’, obtained from u by disturbing t or less
components is not a codeword.

15 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Detection

Theorem
A code C detect t or less errors if and only if the distance of the
code C is greater than t.

Suppose the code C detect all t or less errors. We need to prove

that d(C ) > t.

If possible, let d(C ) ≤ t. Then there will be at least one pair of

codewords u and v such that

d(u, v ) ≤ t

16 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Detection

If u is the sent codeword and v is the received vector with t or less

components getting disturbed then the code will not be able to
detect the t or less errors as v is also a codeword.

This is a contradiction. So,

d(C ) > t

Conversely, let d(C ) > t .

17 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Detection

Suppose, u is a sent codeword and t or less components in u get

disturbed resulting a received vector w . Then

d(u, w ) ≤ t

Again, since for every pair of codewords x, y ∈ C ,

d(x, y ) > t ≥ d(u, w )

So, w cannot be a codeword. Therefore the code can detect t or

less errors.

18 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Example

Example
If C1 = {0000, 1100, 0011, 1111} then we can see that
d(C1 ) = 2. So, C1 can detect all single errors.

19 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Example

Example
If C1 = {0000, 1100, 0011, 1111} then we can see that
d(C1 ) = 2. So, C1 can detect all single errors.

Example
If C2 = {11101, 01110, 11001} then we can see that d(C2 ) = 1.
So, C2 can detect zero or no errors.

19 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Error Correction

Error Correction
Error correction of a code is done by maximum likely hood
decoding method.
If w is the received vector. We compute the disturbances of all
codewords with w and pick that codewords as the same codeword
which has the smallest disturbance. This is called maximum likely
hood decoding.

20 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Correction

Theorem
A code C corrects all t or less errors if and only if the distance of
the code C is greater than 2t.

Let d(C ) > 2t, we need to prove that the code correct all t or less
errors. Let u be the sent codeword and w be the received vector
after t or less errors.

d(u, w ) ≤ t

Again for any v (̸= u) ∈ C

21 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Correction

d(w , v ) + d(w , u) ≥ d(v , u)

⇒ d(w , v ) ≥ d(v , u) − d(w , u)
⇒ d(w , v ) ≥ 2t − t
⇒ d(w , v ) > t ≥ d(w , u)
⇒ d(w , v ) > d(w , u)

So, by maximum likelyhood decoding, the received vector w is

decoded to u correctly. This mean the code C can correct all t or
less errors.
Conversely, suppose the code C corrects all t or less errors then we
need to prove

d(C ) > 2t

22 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Correction

If possible let d(C ) ≤ 2t. Then there exists atleast two codewords
u and v such that d(u, v ) ≤ 2t.

Case 1: If 0 ≤ d(u, v ) ≤ t, then if u is sent codeword and v is

the received vector, then the numbers of error occured is less
then or equal to t. As v is a codeword so the code C cannot
detect it as a result cannot correct it. This is a contradiction.

Case 2: If t + 1 ≤ d(u, v ) ≤ 2t
Let u = (u1 , u2 , u3 , · · · ut , ut+1 , · · · u2t , 0, 1, 2, · · · )
v = (v1 , v2 , v3 , · · · vt , vt+1 , · · · v2t , 0, 1, 2, · · · )
u ′ = (v1 , v2 , v3 , · · · vt , ut+1 , · · · u2t , 0, 1, 2, · · · )
The distance between u and u’ is d(u, u ′ ) = t and
d(v , u ′ ) ≤ t.
So, d(v , u ′ ) ≤ d(u, u ′ ).
23 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Theorem on Error Correction

But this is not possible as u is sent codeword and u’ is received

vector after having t errors, but the maximum likelihood decoding
will not lead to a unique u as the closest codeword. This is
contradiction to the fact that C corrects all t or less errors.
Therefore,

d(C ) > 2t

24 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Example

Example
If C1 = {0000, 1001, 0011, 1111} then we can see that
d(C1 ) = 2. So, C1 cannot correct any errors.

25 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Example

Example
If C1 = {0000, 1001, 0011, 1111} then we can see that
d(C1 ) = 2. So, C1 cannot correct any errors.

Example
If C2 = {10001, 11111} then we can see that d(C2 ) = 3. So, C2
can correct all single errors.

25 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

Applications

Encoding, Decoding, Error detection and Error correction is used in

the following fields:
Communication systems: It is widely used in
communication systems, particularly in the design and analysis
of coding schemes. For instance, error-correcting codes based
on Shannon’s theory are used in digital communication
systems to reduce errors in transmission.
Compression: Information theory provides a framework for
data compression, which is the process of reducing the size of
data files to save storage space or reduce transmission time.
Cryptography: It is also used in cryptography, the practice of
secure communication. It helps in developing encryption and
decryption algorithms that can be used to secure data
transmission over public networks.

26 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

References

[1] Adámek Jiřı́ Foundations of Coding: Theory and Applications of

Error-Correcting Codes with an Introduction to Cryptography and
Information Theory, John Wiley Sons, 1991.

27 / 28
Introduction Pre-requisite Encoding Error detection Error Correction Applications References

THANK YOU

28 / 28