REPUBLIC OF CAMEROON RÉPUBLIQUE DU CAMEROUN

Peace – Work – Fatherland Paix-Travail-Patrie

THE UNIVERSITY OF BAMENDA UNIVERSITÉ DE BAMENDA

HIGHER TECHNICAL TEACHERS TRAINING ECOLE NORMALE SUPERIEURE DE

Information theory and elements of

Credit: 2

By
DADA Jean-Pierre (PhD)
H.T.T.T.C. Bambili, The University of Bamenda
 TABLE OF CONTENTS
CHAPTER ONE: GENERALITIES ....................................................................................................... - 3 -
1. Introduction ................................................................................................................................... - 3 -
2. Number systems ........................................................................................................................... - 3 -
2.1. From a base to another ...................................................................................................................... - 3 -
2.2 Arithmetic in base 2............................................................................................................................. - 5 -
2.3 Arithmetic in base 16........................................................................................................................... - 5 -
3) XOR operations ............................................................................................................................. - 5 -
4) Modular arithmetic ....................................................................................................................... - 5 -
5) ASCII codes .................................................................................................................................... - 5 -
CHAPTER TWO: INFORMATION THEORY ......................................................................................... - 7 -
1. Causes of errors during transmission of information........................................................... - 7 -
2. Error detection without correction ........................................................................................ - 7 -
2.1. Cyclic Redundancy Codes (CRC) ..................................................................................................... - 7 -
2.2 Parity checks ...................................................................................................................................... - 10 -
3. Auto-checking and autocorrecting errors ................................................................................... - 10 -
3.1. Block codes ........................................................................................................................................ - 11 -
3.2. Hamming codes ................................................................................................................................. - 13 -
3.2.1 Hamming codes (7,4): syndrome decoding ............................................................................... - 13 -
3.2.2 Hamming codes (12,4): syndrome decoding ............................................................................. - 16 -
3.2.3. Generalization: Hamming codes ( , ) ................................................................................... - 17 -
3.3. BCH codes ......................................................................................................................................... - 19 -
4. Analog signals .............................................................................................................................. - 19 -
3. Digital or numeric signals ......................................................................................................... - 20 -
5. Advantages of digitization ........................................................................................................ - 20 -
6. Notion of entropy ....................................................................................................................... - 21 -
7. The notion of join entropy .......................................................................................................... - 22 -
8. Notion of mutual Information ..................................................................................................... - 22 -
CHAPTER THREE:
THREE: ELEMENTS OF DIGITAL COMMUNICATION ................................................... - 24 -
1. Introduction ................................................................................................................................. - 24 -
2. Source .......................................................................................................................................... - 24 -
2. Input transducer .......................................................................................................................... - 24 -
3. Source encoder............................................................................................................................ - 24 -
4. Channel encoder ......................................................................................................................... - 24 -
5. Digital modulation ....................................................................................................................... - 25 -
6. Channel ........................................................................................................................................ - 25 -
7. Digital demodulator .................................................................................................................... - 25 -
8. Channel decoder ......................................................................................................................... - 25 -
9. Source decoder............................................................................................................................ - 25 -
10. Output transducer ..................................................................................................................... - 25 -
11. Output signal ............................................................................................................................. - 25 -
CHAPTER FOUR: TUTORIALS..........................................................................................................
TUTORIALS - 26 -

Information theory and elements of digital communication (Dr. J.P. DADA, University of Bamenda) Page - 2 -