Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

Information Theory and Coding L P C

3 3

Discipline(s) / EAE / OAE Semester Group Sub-group Paper Code

CSE-CS 6 PC PC CS-310T
EAE 6 CS-EAE CS-EAE-1 CS-310T

Marking Scheme:
1. Teachers Continuous Evaluation: 25 marks
2. Term end Theory Examinations: 75 marks
Instructions for paper setter:
1. There should be 9 questions in the term end examinations question paper.
2. The first (1st) question should be compulsory and cover the entire syllabus. This question should be
objective, single line answers or short answer type question of total 15 marks.
3. Apart from question 1 which is compulsory, rest of the paper shall consist of 4 units as per the syllabus.
Every unit shall have two questions covering the corresponding unit of the syllabus. However, the student
shall be asked to attempt only one of the two questions in the unit. Individual questions may contain upto
5 sub-parts / sub-questions. Each Unit shall have a marks weightage of 15.
4. The questions are to be framed keeping in view the learning outcomes of the course / paper. The standard
/ level of the questions to be asked should be at the level of the prescribed textbook.
5. The requirement of (scientific) calculators / log-tables / data – tables may be specified if required.
Course Objectives :
1. To understand the efficient, error-free and secure delivery of information using binary streams.
2. To have in-depth knowledge of error-control coding.
3. To learn the process of encoding and decoding of digital data streams.
4. To learn and apply the methods of generation of these codes and evaluate the performance of them
over the noisy communication channels.
Course Outcomes (CO)
CO 1 To be able to understand the principles behind an efficient and secure transmission of digital data
stream.
CO 2 To be able to demonstrate the knowledge of channel capacity and coding.
CO 3 To be able to implement the knowledge of encoding and decoding of digital data stream using Linear
& Cyclic Codes.
CO 4 To be able to analyse the encoding and decoding of digital data stream using Convolutional codes.
Course Outcomes (CO) to Programme Outcomes (PO) mapping (scale 1: low, 2: Medium, 3: High)
PO01 PO02 PO03 PO04 PO05 PO06 PO07 PO08 PO09 PO10 PO11 PO12
CO 1 3 1 1 1 - - - 2 - 3 - 1
CO 2 3 3 3 3 1 1 - 2 - 2 - 2
CO 3 3 3 3 3 3 1 - 2 - 2 - 2
CO 4 3 3 3 3 3 1 - 2 - 2 - 2

UNIT-I

Introduction to Information Theory, Uncertainty & Information, Mutual Information, Average mutual
information, Entropy, Relative Entropy, Extension of an Information source and Markov Source, Maximum
Entropy Principle, Information measure of Continuous random Variables, Maximum Entropy Principle,
Jensen’s Inequality, Fano’s Inequality, Introduction to lossless coding, Source coding theorem Block code and
its properties, Instantaneous code and its properties, Kraft-Mcmillan equality, Huffman Coding, Shannon Fano
coding, Lempel Ziv Algorithm.

UNIT-II

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 947
 Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

Introduction to discrete information channels, Equivocation and Mutual Information, Properties of different
information channels, Reduction of information channels, Noiseless channel, Properties of Mutual
information, Introduction to channel capacity, Shannon’s Channel Coding theorem, Bandwidth – S/N Trade
Off, Channel capacity theorem, Shannon Limit, Channel capacity for MIMO system

UNIT-III

Definition of Terms: Redundancy, code efficiency, systematic codes, Hamming distance, Hamming Weight,
Hamming Bound, Types of Code: Parity check codes, Hamming codes, Linear Block Codes, Generator and Parity
Check matrix, Syndrome decoding. Introduction to Cyclic Codes, Generation and detection of cyclic codes.

UNIT – IV

Burst Error Detecting and correcting codes, Convolutional codes, Code Tree, Trellis and State diagram,
Decoding of convolutional codes, Viterbi’s Algorithm, Sequential Decoding, Transfer function and Distance
properties of convolutional codes, Bound on bit error rate, Coding Gain.

Textbook(s):
1. Ranjan Bose, “Information Theory Coding & Cryptography”, 3 rd Edition, McGraw Hill, 2017.
2. T.M. Cover and J.A Thomas, “Elements of Information Theory”, 2 nd Edition, Wiley India Pvt Ltd, 2013.

References:
1. Salvatore Gravano, Introduction to Error Control Codes, Oxford University Press, 2017.

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 948
 Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

Information Theory and Coding Lab L P C

2 1

Discipline(s) / EAE / OAE Semester Group Sub-group Paper Code

CSE-CS 6 PC PC CS-310P
EAE 6 CS-EAE CS-EAE-1 CS-310P

Marking Scheme:
1. Teachers Continuous Evaluation: 40 marks
2. Term end Theory Examinations: 60 marks
Instructions:
1. The course objectives and course outcomes are identical to that of (Information Theory and Coding) as this
is the practical component of the corresponding theory paper.
2. The practical list shall be notified by the teacher in the first week of the class commencement under
intimation to the office of the Head of Department / Institution in which the paper is being offered from the
list of practicals below. Atleast 10 experiments must be performed by the students, they may be asked to
do more. Atleast 5 experiments must be from the given list.

1. Write a MATLAB program to compute entropy and mutual information for Noise Free and Noisy Channel.
2. Write a MATLAB program to compute entropy and mutual information for Error Free and Binary Symmetric
Channel.
3. Write a MATLAB program to implement algorithm for generation and evaluation of Shannon- Fano coding
and decoding. Compute entropy, average length and coding efficiency.
4. Write a MATLAB program to implement algorithm for generation and evaluation of Huffman coding and
decoding. Compute entropy, average length and coding efficiency.
5. Write a MATLAB program to implement algorithm for generation and evaluation of Lempel Ziv dictionary
method. Compute entropy, average length and coding efficiency.
6. Write a MATLAB program to implement the algorithm for encoding and decoding of Linear Block Code.
7. Write a MATLAB program to implement the algorithm for encoding and decoding of Cyclic Code.
8. Write a MATLAB program to implement the algorithm for generating Convolutional code by Code Tree.
9. Write a MATLAB program to implement the algorithm for generating Convolutional code by Code Trellis.
10. Write a MATLAB program to implement the algorithm for encoding and decoding of BCH Code.

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 949