C 1200ECT306052401 Pages: 3

Reg No.:_______________ Name:__________________________

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (S, FE) / S6 (PT) (S, FE) Examination December 2024 (2019 Scheme)

Course Code: ECT306

Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours
PART A
Answer all questions, each carries 3 marks. Marks
1 Define the term entropy of a discrete memory-less source. Calculate the entropy (3)
of a DMS with two symbols with respective probabilities { , }.

2 Differentiate between source coding and channel coding (3)

3 State Shannon’s channel coding theorem (3)
4 What is meant by the differential entropy? Derive an expression for the same (3)
5 Evaluate the expression over the prime field modulo-5 operation (3)

6 Differentiate between block codes and convolutional codes. (3)

7 Elaborate on the features of cyclic codes and their benefits. (3)
8 Explain the parameters of Hamming codes (3)
9 Draw the convolutional encoder whose connection vectors are, g1=(111) and (3)
g2=(101).
10 Explain the features of any one ‘capacity approaching’ code. (3)
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) Determine the mutual information between two discrete memoryless sources X (7)
and Y with conditional probability matrix given by

P(Y/X) =

Verify the formula H(X,Y)= H(X) + H(Y/X). Given p(x1)=0.6, p(x2)=0.3 &
p(x3)=0.1.
b) For any discrete memoryless source S= {s1, s2, s3, ... sq}, prove that (7)
0 ≤ H(S) ≤ log2q , where H(S) is the entropy of the source.
OR

Page 1of 3
 1200ECT306052401

12 a) Consider a DMS with source probabilities {0.35, 0.25, 0.2, 0.15, 0.05} (10)
i. Determine the binary Huffman code for this source
ii. Determine the ternary Huffman code for this source using symbols A, B
and C.
iii. Determine the average code word length and compare.
b) A given source alphabet consists of 300 words, of which 15 occur with (4)
probability 0.06 each and the remaining words occur with probability 0.00035
each. If 1000 words are transmitted each second, what is the average rate of
information transmission?
Module II
13 a) A binary channel has the following noise characteristics: (7)

P(Y/X) =

Find the channel capacity, efficiency and redundancy of the channel, if the
source symbols are occurred with probabilities p(x1) = 2/3 and p(x2) = 1/3.
b) State and derive Shannon-Hartley theorem (7)
OR
14 a) A communication system employs a continuous source. The channel noise is (7)
white and Gaussian. The bandwidth of the source output is 10 MHz and signal to
noise power ratio at the receiver is 100.
i. Determine the channel capacity
ii. If the SNR drops to 10, how much bandwidth is needed to achieve the
same channel capacity?
b) Find the capacity of a channel with infinite bandwidth. Discuss Shannon’s limit (7)
Module III
15 a) Consider a systematic block code whose parity check equations are: (10)
p1 = m1+m2+m4
p2 = m1+m3+m4
p3 = m1+m2+m3
p4 = m2+m3+m4
i. Construct the G and H matrices for this code
ii. How many errors can the code detect and correct?
iii. Is the vector 10101010 a valid codeword?
iv. Is the vector 01011100 a valid codeword?

Page 2of 3
 1200ECT306052401

b) Which among the following block codes are perfect codes? Justify your answer. (4)
i. (7,3)
ii. (7,4)
OR
16 a) The parity check matrix for a (7,4) LBC is given below:
(7)
P=

Draw the syndrome calculator circuit for the code.

b) Explain the standard array decoding of linear block code. Derive the standard (7)
array for a (5,1) repetition code.
Module IV
17 a) Encode the message 101 in systematic form for a cyclic code with generator (5)
polynomial g(X) = 1 + X + X2 +X4.
b) Design a feedback shift register encoder for an (8,5) cyclic code with generator (9)
polynomial g(X) = 1 + X + X2 +X3. Use the encoder to find the codeword for the
message 10101 in systematic form.
OR
18 a) Consider the generator polynomial of (7,4) cyclic code as g(X) = 1+X2+X3. (8)
i. Determine the parity check polynomial for this code
ii. Find the generator and parity check matrices in systematic form
iii. Determine the error detecting and correcting capability of the code.
b) A (15,5) cyclic code has a generator polynomial as follows: (6)
g(X) = 1 + X + X2 +X5+ X8 +X10.
i. Find the code polynomial for the message 10101
ii. Has any error occurred during transmission for the received word
100010101000001?
Module V
19 a) Draw the state, tree and trellis diagram for the K=3, rate 1/3 code generated by, (14)
g1(X) = X+X2
g2(X) =1+ X
g3(X) =1+ X+X2
OR
20 a) Explain decoding of convolutional codes using Viterbi algorithm. (14)
****

Page 3of 3
 1200ECT306052401

Scheme of Valuation/Answer Key

(Scheme of evaluation (marks in brackets) and answers of problems/key)
APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (S, FE) / S6 (PT) (S, FE) Examination December 2024 (2019 Scheme)
Course Code: ECT306
Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Average information content per source symbol. Expression for entropy.(2mark) (3)
H(S) = 1 bit/symbol (1 marks)
2 Definition of source coding and channel coding ( 1.5 marks each) (3)
3 Statement (3)
4 Definition --------1 Mark (3)
Derivation of expression ------- 2 Marks
5 Additive inverse of 4 = 1 → 2-4 = 2+1 = 3 (3)
Multiplicative inverse of 3 = 2 → ((2-4) x 4) /3 = (3 x 4) x 2 = 4
6 Three differences --- 1 mark each (3)
7 Any three features/ benefits ---- 1 Mark each (3)
8 For any positive integer m ≥ 3; n = 2m – 1, k = 2m – m – 1, n – k = m, t = 1 (3)
9 (3)

10 LDPC (3)
PART B
Answer onefull question from each module, each carries 14 marks.
Module I
11 a) (7)
P(X,Y) = ------ 1mark

p(y1) = 0.45, p(y2) = 0.35 & p(y3) = 0.2

H(Y) = 1.5128 bits/symbol ------ 1mark
H(Y/X) = 1 bit/symbol ------ 1mark

Page 1of 6
 1200ECT306052401

I(X,Y) = H(Y) – H(Y/X) = 0.5128 bits/symbol ------ 1mark

H(X,Y) = 2.2955 bits/symbol ------ 1mark
H(X) = 1.2955 bits/symbol ------ 1mark
⸫ H(X,Y) = H(X) + H(Y/X) ------ 1mark
b) Proof of lower bound --------1 mark (7)
Proof for upper bound ---------6 marks
OR
12 a) Binary Huffman (4 marks) Ternary Huffman (4 marks) (10)
Msg prob, pi
Code Length li Code Length, li
0.35 00 2 B 1
0.25 01 2 C 1
0.20 11 2 AA 1
0.15 100 3 AB 2
0.05 101 3 AC 2
Average
Codeword
2.2 bits 1.4 bits
length=pili
(2 marks)
b) H(S) = 4.798 bits/symbol ----2 Mark (4)
Rate of information transmission = rs x H(S)
= 1000 x 4.798 = 4798 bits/sec ----2Marks
Module II
13 a) C = 1-H(p) = 0.8112 bit/symbol ------ 2 Marks (7)

P(X,Y) =

p(y1) = 7/12 & p(y2) = 5/12 ------ 1 Mark

H(Y) = 0.9796 bits/symbol ------ 1 Mark
H(Y/X) = H(p) = 0.8112 bit/symbol
⸫ I(X;Y) = H(Y) – H(Y/X) = 0.1684 bits/symbol ------ 1 Mark
Channel efficiency = = 0.8919 = 89.19% ------ 1 Mark

Channel redundancy = 1-efficiency = 10.81% ------ 1 Mark

b) B = log2 (1 + ) bits/sec ------ 1 Marks (7)

Page 2of 6
 1200ECT306052401

Derivation ------ 6 Marks

OR
14 a) i. B = log2 (1 + ) bits/sec ------ 1 Marks (7)

= 6.66 X 107 bps ------ 2 Marks

ii. B = 19.25 MHz ------ 4 Marks
b) Derivation for C∞ = 1.44 (7)

Derivation for Shannon Limit = -1.63dB

Module III
15 a) (10)
i) G= -------- 1 Mark

H= -------- 1 Mark

ii) from the H matrix, the min number of columns that sum to zero vector is 4 (
Eg. 3+5+6+7) ⸫ dmin = 4 -------- 2 Marks
No of errors detected= 3 -------- 1 Mark
No of errors corrected = 1 -------- 1 Mark
iii) s=rHT = [0 0 1 1] ≠ 0 , so not a codeword -------- 2 Marks

iv) s=rHT = [0 0 0 0] = 0 , so a valid codeword -------- 2 Marks

b) (7,3) not a perfect code because not able to include all the double bit error (4)
patterns in the std array.
(7,4) is a perfect code because all the single bit error patterns are included in the
std array
OR
16 a) (7)
H=

[s1 s2 s3] = [r1 r2 r3 r4 r5 r6 r7] HT

s1 = r1+r3+r4+r5
s2 = r1+r2+r3+r6
s3 = r2+r3+r4+r7

Page 3of 6
 1200ECT306052401

b) steps in std array decoding -------3 Marks (7)

std array of (5,1) repetition code -----4 Marks
Module IV
17 a) (n,k) = (7,3) (5)
X(n-k).m(X) = X4.(1+X2) ---------- 1 Mark
r(X) = X3+X2 ---------- 2 Marks
c(X) = r(X) + X(n-k).m(X) = X2+X3+X4+X6 ---------- 1 Mark
c = [0 0 1 1 1 0 1] ---------- 1 Mark
b) (9)

Input queue Shift number Register contents

10101 0 000
1010 1 111
101 2 100
10 3 101
1 4 101
- 5 010 →Parity
⸫ the final codeword is 01010101
OR
18 a) h(X) = (X7+1)/g(X) = 1+X2+X3+X4 -----2Marks (8)

Page 4of 6
 1200ECT306052401

Gnon systematic = =

Gsystematic = -----2Marks

⸫ Hsystematic = -----1Mark

Minimum number of columns sum to zero (Eg. Col3, Col5 & Col6) is 3. -----
1Mark
⸫dmin = 3
Error detection = dmin -1 = 2 -----1Mark
Error correction = =1 -----1Mark

b) c(X) = 1+X+X3+X6+X7+X8+X9+X14 ----------2Marks (6)

Since the remainder obtained when the given code polynomial is divided by
g(X) = X9+X7+X4+X3+X ≠ 0, the received vector has affected by noise during

transmission. ---------4 Marks

Module V

Page 5of 6
 1200ECT306052401

19 a) (14)

OR
20 a) Viterbi decoding with an example (14)
****

Page 6of 6
C 1200ECT306012401 Pages: 3

Reg No.:_______________ Name:__________________________

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (R,S) / S6 (PT) (R,S) Examination May 2024 (2019 Scheme)

Course Code: ECT306

Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Define conditional entropy. (3)

2 Find Huffman code for the source with symbol probabilities {1/2,1/3,1/6}. (3)
3 State Shannon’s channel coding theorem. What is its significance in digital (3)
communication system?
4 An analog signal is bandlimited to B Hz, sampled at the Nyquist rate, and the (3)
samples are quantized into 4 levels. The quantization levels are assumed
independent and occur with probabilities P1 = P4 = 3 / 8 and P2 = P3 =1/ 8.Find
the information rate of the source assuming B = 100Hz.
5 List the properties of group. Give an example. (3)
6 Explain Repetition code with general example. (3)
7 What are the features of Hamming Code. (3)
8 List the features of Reed Solomon code. (3)
9 What is Tanner graph representation of LDPC codes. (3)
10 What is trellis diagram. (3)
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) State and prove Kraft Inequality. (7)
b) Show that the entropy is maximum when all the messages are equally likely. (7)
OR
12 a) Explain uniquely decodable and prefix-free property of source code. (4)
b) A transmitter has an alphabet consisting of five letters{ x1,x2,x3,x4,x5}and the (10)
receiver has an alphabet of four letters { y1,y2,y3,y4}.The joint probabilities of
the systems are shown as

Page 1of 3
 1200ECT306012401

y1 y2 y3 y4
x1 0.25 0 0 0
P(X,Y) = x2 0.1 0.3 0 0
x3 0 0.05 0.1 0
x4 0 0 0.05 0.1
x5 0 0 0.05 0
Find Mutual information.
Module II
13 a) Explain Binary Symmetric Channel and derive its channel capacity. (10)
b) Also draw the Entropy vs Probability and Channel capacity vs Probability (4)
graphs of the Binary Symmetric Channel.
OR
14 a) State and prove Shannon Hartley theorem (10)
b) Also explain Bandwidth-S/N trade off. (4)
Module III
15 a) Explain with examples how error detection and correction is possible with Block (7)
codes.
b) Draw and explain the general decoding circuit of a (n,k) Linear Block codes. (7)
OR
16 a) Explain single parity check bit coding for error detection. (7)
b) Explain Rings and finite fields. (7)
Module IV
17 a) Draw the general encoding circuit for a (n,k) Systematic Cyclic code and (10)
explain its operation.
b) The generator polynomial of a (7,4) Cyclic code is G(p) = 1 + p + p3 . Find the (4)
code vector corresponding to the message vector 1010 in NonSystematic form.
OR
18 a) Draw the encoder circuit for a (7,4) Cyclic code with G(p) = 1 + p + p3 and (10)
obtain the codeword in systematic form for the message sequence 1110.
b) Explain the features of BCH codes. (4)

Page 2of 3
 1200ECT306012401

Module V
19 With the help of figures, explain Viterbi algorithm for decoding of (14)
Convolutional code with example.
OR
20 a) Explain LDPC code with general example. (7)
b) Draw a (3,2,1) Convolutional encoder with generator sequences g1(1) = (1 1) , (7)
g1(2) = (0 1) , g1(3) = (1 1) and g2(1) = (0 1) , g2(2) = (1 0) , g2(3) = (1 0).
****

Page 3of 3
 1200ECT306012401

Scheme of Valuation/Answer Key

(Scheme of evaluation (marks in brackets) and answers of problems/key)
APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (R,S) / S6 (PT) (R,S) Examination May 2024 (2019 Scheme)
Course Code: ECT306
Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Definition of conditional entropy ( 3 marks ). (3)

2 Coding procedure (2 marks) (3)
Code word( 1 marks )
3 Statement ( 2 marks ) ; Significance ( 1marks). (3)
4 The average information H = 1.8 bits / message.( 1.5marks ) (3)
The information rate R = r.H = 2.B.(1.8) = 360 bits / Second.( 1.5 marks )
5 Properties ( 2 marks ) ; Example ( 1 marks ). (3)
6 Explanation with an example ( 3 marks ). (3)
7 Three points on Hamming code .( 3 marks ) (3)
8 Three features of Reed Solomon code. ( 3 marks ) (3)
9 Explanation with an example using suitable diagram. ( 3 marks ) (3)
Note : Marks should be given even for drawing Tanner graph with an
example
10 Explanation with the help of suitable diagram.( 3 marks ) (3)
Note: Marks should be given even if explained with an example
PART B
Answer onefull question from each module, each carries 14 marks.
Module I
11 a) Statement (3 marks) ; Proof (4 marks). (7)
b) Proof (7 marks ) (7)
OR
12 a) Uniquely decodable property (2 marks); prefix-free property ( 2 marks). (4)
Note : Marks should be given even if only example is mentioned of uniquely
decodable code and Prefix-free code

Page 1of 3
 1200ECT306012401

b) Marks should be given if Mutual information is calculated using any (10)

method :
1. If I(X:Y) calculated using equation I(X:Y) = H(X) - H(X/Y) then :
H(X) -(3 marks), H(X/Y) - (5 marks), I(X:Y) - (2 marks)

2. If I(X:Y) calculated using equation (IX:Y) = H(Y) - H(Y/X) then:

H(Y) - (3 marks), H(Y/X) - (5 marks), I(X:Y) - (2 marks).

If I(X:Y) calculated using equation I(X:Y) = H(X)+H(Y) - H(X,Y) then :

H(X) - (3 marks), H(Y) - (3 marks), H(X:Y)- (2 marks), I(X:Y)- (2 marks).

Module II
13 a) Channel diagram or Channel Matrix( 4marks); Derivation (6 marks). (10)
b) Entropy Vs Probability graph (2 marks) ;Channel capacity Vs Probability (4)
graph (2 marks).
OR
14 a) Statement (4 marks); Proof (6 marks). (10)
b) Using Noise power N = η.B, find Channel capacity C, then findC ∞ .(2 (4)
marks);Explanation of Bandwidth-S/N trade off (2 marks).
Module III
15 a) Error detection (3.5 marks) (7)
Error Correction (3.5 marks)
b) Decoding circuit(4 marks);Explanation (3 marks) (7)
Note : Mark should be given even if the Decoder is explained using an
example.
OR
16 a) Coding explanation(3.5 marks) ; Error detection explanation(3.5 marks). (7)
b) Explanation of Rings (3.5 marks) ; Explanation of finite fields( 3.5 marks). (7)
Module IV
17 a) Circuit (6 marks) ; Explanation (4 marks). (10)
Note : Mark should be given if an example is used to explain the Decoder
b) Message polynomial (1 mark) ; Codeword polynomial ( 2 marks) ; (4)

Page 2of 3
 1200ECT306012401

Codeword = ( 1110010) (1 mark).

Note: Mark should be given even if the students hasn’t attempted the
question
OR
18 a) Encoder Circuit (5 marks); Finding Codeword (5 marks);Codeword = (10)
[1110010]
Note :
1.Marks should be given if Systematic codeword is structured as coefficients
of message polynomial first followed by coefficients of message polynomial.
Then codeword = 0101110
2. Marks should be given if codeword is found using systematic G matrix or
using the encoder circuit.

b) Three points on BCH Codes (4marks). (4)

Module V
19 Explanation using example and suitable diagram: (14)
Encoder circuit /State diagram /State transition table - (4 marks), Trellis
Diagram -(5marks), Decoding - (5 marks)
OR
20 a) Explanation of LDPC code(4marks); General example on LDPC (7)
code(3marks).
b) Encoder Circuit (7marks). (7)
****

Page 3of 3
Reg No.:_______________ Name:__________________________
1200ECT306052303
APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (S, FE) / S6 (PT) (S) Examination January 2024 (2019 Scheme)

Course Code: ECT306

Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 A source is transmitting six different messages at the rate of 100 msg/sec. The (3)

probabilities of the messages are ½, ¼, 1/6, 1/8, 1/32 and 1/32. Calculate the
information rate.
2 Define Entropy and list its properties. (3)
3 State Shannon’s second theorem on channel capacity. Give the Positive and (3)
Negative statements.
4 Explain Binary Symmetric Channel. (3)
5 1 0 0 0 1 1 (3)
A (6, 3) linear block code has a generator matrix given by 0 1 0 1 0 1.
0 0 1 1 1 0
Find all the code vectors.
6 Explain the error detection and error correction capability of a linear block code. (3)
7 The generator polynomial of a (7, 4) Cyclic code is 𝐺(𝑝) = 𝑝 + 𝑝 + 1. Find the (3)
code vectors in systematic form for the message vector 0101.
8 Explain the BCH codes. (3)
9 Draw the encoder circuit of a (2, 1, 3) convolutional encoder, with sequences are (3)
[1 0 0 1] and [1 1 1 1] as impulse responses.
10 Explain LDPC Codes. (3)
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) State and prove Kraft’s inequality. (7)
b) A Discrete memoryless system has 6 codes with probabilities 1/3, 1/4, 1/8, 1/8, 1/12, (7)
and 1/12.Obtain the Huffman codes and find the code efficiency and redundancy.

Page 1of 3
 1200ECT306052303

OR
12 a) The joint probability matrix of a communication system is given by (10)
0.1 0.05 0.05
𝑃(𝑋𝑌) = 0.2 0.15 0.1 . Find H(X), H(Y), H(X,Y), H(X/Y) and H(Y/X).
0.1 0.2 0.05
b) Explain uniquely decodable and prefix free property of source code (4)
Module II
13 a) Derive the capacity of a Binary Erasure Channel. (6)
b) Given a binary channel, (8)

1. Find the channel matrix.

2. Find P(y1) and P(y2) given P(x1)= P(x2)= 0.5
3. Find the joint probability P(x1, y2) and P(x2, y1).
OR
14 a) A voice grade channel of a telephone network has a bandwidth of 3.4KHZ. (7)
a. Calculate the channel capacity of the telephone channel or a signal to noise
ratio of 30dB.
b. Calculate the minimum signal to noise ratio required to support
information transmission through the telephone channel at the rate of
4800bits/sec.
b) State Shannon Hartely theorem and explain the significance of Shannon Limit. (7)
Module III
15 a) The parity check list of an (8,4) linear block code is given by (12)
C5= d1+d2+d4
C6= d1+ d2+d3
C7=d1+ d3+d4
C8= d2+d3+d4
a. Find the generator and parity check matrix.

Page 2of 3
 1200ECT306052303

b. Find the minimum weight.

c. Calculate the error correction and detection capability.
d. Implement the encoder circuit.
b) List the properties to be satisfied by a linear block code? (2)
OR
16 a) Define ring and field. Discuss its properties. (5)
b) The parity matrix for a (7,4) linear block code is given by (9)
1 0 1
1 1 1
P=
1 1 0
0 1 0
a. Find all code vectors.
b. Find Parity check matrix.
c. Implement an decoder circuit of the code
Module IV
17 a) The generator polynomial of a (7, 4) Cyclic code is 𝐺(𝑝) = 𝑝3 + 𝑝 + 1. Find the code (10)
vectors corresponding to the message vectors 1011 and 1101 in Non-systematic form
b) Discuss the properties of Hamming codes. (4)
OR
18 a) Draw syndrome circuit for a (7, 4) cyclic code generated by𝑔(𝑥) = 1 + 𝑥 + 𝑥 . If (10)
the received code word r is [0010110], what is the syndrome of r? Explain the circuit
with a table showing the contents of syndrome register.
b) Explain decoding of cyclic codes. (4)
Module V
19 For a Convolutional encoder, the generator sequences are given as, g (1) = (1,1,0) (14)
and g(2) = (1,0,1). Obtain the state diagram and encode the message sequence
(1011). Also verify the output sequence using Transform domain approach.
OR
20 For a (2,1,2) convolutional encoder with generator sequences g(1) = (1,1,1) and (14)
g(2) = (1,0,1). Draw the Trellis diagram and perform Viterbi decoding on this trellis
for the received sequence {01, 10, 10, 11, 01, 01, 11} and obtain the estimate of
the transmitted sequence.
****

Page 3of 3
 1200ECT306052303

DRAFT Scheme of Valuation/Answer Key

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY

SIXTH SEMESTER B.TECH DEGREE (R,S)EXAMINATION, May/July 2023(2019 Scheme)
Course Code: ECT306
Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Equation – 1 marks, Information rate=193.7bits/sec- 2 marks (3)

2 Definition – 1 marks, Properties- 2 marks (3)
3 Statement- 2 mark, Positive and negative statements- 1 marks (3)
4 Channel Diagram - 1 marks, Channel Matrix – 1 marks, Explanation- 1 marks (3)
5 Calculation of code vectors- 1 mark, All code vectors- 2 marks (3)
6 Error correction- 1.5 marks, Error detection- 1.5 marks (3)
7 Equation to calculate code vector- 1 mark, Final answer- [0101101] - 2 marks (3)
8 Explanation – 3 matrix (3)
9 Circuit diagram- 3 marks (3)
10 Explanation – 3 matrix (3)
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) Statement- 2 marks, Proof- 5 marks (7)
b) Huffman code (4 marks), Efficiency(98.31%) (2 marks), Redundancy= 1.69% (1 (7)
marks)
OR
12 a) 5 entropies- 2 marks each (10)
H(X)= 1.5128 bits/sec
H(Y)= 1.527 bits/sec
H(XY)= 2.9841 bits/sec
H(X/Y)= 1.4622 bits/sec
H(Y/X)= 1.4718 bits/sec
b) Explanation- 3marks, Example- 1 marks (4)

Page 1of 3
 1200ECT306052303

Module II
13 a) Derivation- 4 marks, Explanation – 2 marks (6)
b) channel matrix-𝑃(𝑌/𝑋) = 0.9 0.1 2 marks (8)
0.2 0.8
P(y1)= 0.55 and P(y2)=0.45- 3 marks
P(x1, y2)=0.05 and P(x2, y1)=0.1 - 3 marks
OR
14 a) Channel capacity= 33.8kbps (4 marks), minimum signal to noise ratio=2.2dB (3 (7)
marks)
b) Shannon Hartely Theorem (2 marks), significance of Shannon limit (5 marks) (7)
Module III
15 a) a. the generator and parity check matrix- 4 marks (12)
1 1 1 0
1 1 0 1
𝑃=
0 1 1 1
1 0 1 1
𝐺 =[I P]
b. minimum weight=4 - 1 mark
c. error correction capability=2 - 1 mark
error detection capability=3 - 1 mark
d. encoder circuit- 5 marks
b) Properties- 2 marks (2)
OR
16 a) Define ring and field- 3 marks, Properties- 2 marks (5)
b) a. all code vectors- 4 marks (9)
V=U.G , G is the generator matrix
G=[I P]
b. Parity check matrix- 2 marks
H=[𝑃 𝐼]
c. Implement an decoder circuit of the code- 3 marks
Module IV
17 a) Code vector corresponding to [1011]= [1000101] (10)
Code vector corresponding to [1101]= [1111001]
b) Properties- 4 marks (4)

Page 2of 3
 1200ECT306052303

OR
18 a) Syndrome circuit- 5 marks (10)
Syndrome register- 5 marks
b) Explanation- 4 marks (4)
Module V
19 State table (3 marks), State diagram (4 marks) encoding of the message sequence (14)
1011 (3 marks), verification using Transform domain approach (4 marks)
OR
20 State diagram (4 Marks), Trellis Diagram (4 mark), Viterbi Decoding (4 marks (14)
Transmitted sequence (2 marks)
****

Page 3of 3
C 1200ECT306052301 Pages: 3

Reg No.:_______________ Name:__________________________

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
B.Tech Degree S6 (R, S) / S6 (PT) (R) Examination June 2023 (2019 Scheme)

Course Code: ECT306

Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Consider a source S = {S1, S2, S3} with P = {1/2,1/4,1/4)}. Find self-information (3)
of each message and entropy of source S?
2 Define coding and explain the necessity of coding? (3)
3 State the positive and negative statements of Shannon’s channel coding theorem. (3)
4 Explain BSC with its channel diagram. (3)
5 Distinguish between block codes and convolutional codes. (3)
6 Explain how generator matrix is obtained for a linear block code. (3)
7 What is BCH code? (3)
8 List any three properties of cyclic code. (3)
9 For a (2,1,2) convolutional encoder with impulse response g(1)=[111] and (3)
g(2)=[101], find the output sequence for the message D =1 0 0 1 1 by time domain
approach.
10 Explain trellis diagram. (3)
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) Show that H (X, Y) = H(X/Y) + H(Y). (7)
b) Given the messages X1, X2, X3, X4, X5 and X6 with respective probabilities of 0.4, (7)
0.2, 0.2, 0.1, 0.07 and 0.03. Construct a binary code by applying Huffman
encoding procedure. Also determine its efficiency and redundancy.
OR
12 a) Construct an instantaneous binary code for a source producing 5 symbols S1 to S5 (7)
with respective probabilities of {1/2, 1/6,1/6,1/9,1/18}. Also find the code
efficiency and redundancy?

Page 1of 3
 1200ECT306052301

b) The joint probability of a pair of random variable is given below. Determine H(X), (7)
H(Y), H(X,Y), H(X/Y), H(Y/X) and I(X,Y) .Verify the relation between joint,
1/3 1/3
conditional and marginal entropies? P(X,Y)= [ ]
0 1/3

Module II
13 a) Explain Bandwidth-efficiency relation and Shannon’s Limit of a continuous (7)
Gaussian channel.
b) A message source produces two independent symbols A and B with probabilities (7)
P(A)=0.4 and P(B)=0.6 and transmitted through a BSC. Calculate the Information
transmission rate of the system if the symbols are received in average with 4 in
every 100 symbols in error.
OR
14 a) Calculate the capacity of a Gaussian channel with a bandwidth of 1MHz and S/N (7)
ratio of 30dB.
b) Explain Binary Erasure Channel and derive its channel capacity. (7)
Module III
15 a) Construct the encoding circuit of a systematic (6,3) LBC with the parity matrix (7)
1 0 1
given by P = [0 1 1]
1 1 0
Also find all possible code vectors.
b) Explain the error detection and correction capabilities of linear block code. (7)
OR
16 a) Explain the procedure to prepare the standard array of LBC. Also construct the (7)
1 1
standard array of (4,2) LBC whose parity matrix is given by, P = [ ]
1 0
b) For a systematic (7,4) linear block code, the parity matrix P is given by, (7)
1 1 1
P=
1 1 0
1 0 1
0 1 1
A single error has occurred in the received vectors [0111110] and [1011100].
Detect and correct the error?
Also draw the syndrome calculation circuit?

Page 2of 3
 1200ECT306052301

Module IV
17 a) Draw the general encoding circuit for (n,k) cyclic code and explain its operation. (7)
b) Find the generator and parity check matrices of (7,4) cyclic code where (7)
xn+1= x7+1 = (1+x+x3) (1+x+x2+x4).
OR
18 For a (7,4) cyclic code, the received vector Z(x) is 1110101 and the generator (14)
polynomial is g(x)=1+x+x3. Draw the syndrome calculation circuit and explain the
procedure with the register contents. Also correct the single error in the received
vector.
Module V
19 Consider a (3,1,2) convolutional code with g(1)=(1 1 0), g(2)=(1 0 1) and g(3)=(1 1 1). (14)
i) Draw the encoder block diagram.
ii) Find the generator matrix.
iii) Find the code vector corresponding to the information sequence (1 1 1 0 1)
using time domain and transfer domain approach.
OR
20 Explain Viterbi algorithm for decoding of convolutional code with example with the (14)
help of figure.
****

Page 3of 3
 1200ECT306052301

Scheme of Valuation/Answer Key

(Scheme of evaluation (marks in brackets) and answers of problems/key)
APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
SIXTH SEMESTER B.TECH DEGREE (R,S)EXAMINATION, June 2023(2019 Scheme)
Course Code: ECT306
Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Self information with necessary equation(1.5 marks)[Self information of s1=1 bit, (3)
s2=2 bit, s3=2 bit]
Source entropy with necessary equation(1.5 mark)[H(S)=1.5 bits /message
symbol]
2 Definition of coding(1.5 mark) (3)
Necessity of coding(1.5 marks)
3 Positive statement(1.5 mark) and negative statement(1.5 mark) (3)
4 (3)

Definition of BSC(1 mark) Diagram(2 mark)

5 Three factors of comparison(3 marks) (3)
6 Equation of check bits(1 marks) (3)
Equation of code vector in matrix form(1 mark)
Generator matrix in terms of parity matrix(1 mark)
Full marks can be given if explained with an example.
7 Three points on BCH code(3 mark) (3)
8 Any three properties of cyclic code(3 marks) (3)
9 Code vector[C]=[11,10,11,11,01,01,11](3 marks) (3)
Can use any approach to find code vector.Full marks can be given if code vector
is correct.
10 Explanation with an example using suitable diagram(3 mark) (3)
PART B

Page 1of 4
 1200ECT306052301

Answer one full question from each module, each carries 14 marks.
Module I
11 a) Derivation(7 mark) (7)
b) Huffman code table and code[x1-1, x2-01, x3=000, x4=0010, x5=00110, (7)
x6=00111] (4 mark),
(4 marks can be given to any proper Huffman code with average length 2.3)
Efficiency=H(S)/L=2.209/2.3=96.04%(2 marks)
Redundancy(1 mark)
OR
12 a) Any instantaneous code meeting the condition(4 mark) (7)
Efficiency(2 marks)
Redundancy(1 mark)

b) Finding the values of entropies(5 mark) (7)

H(X,Y)=1.589bits/message

H(X/Y)=0.666bits/message

H(Y/X)=0.666bits/message

I(X,Y)=0.2522bits/message

Relation verification(2 mark)

Module II
13 a) Bandwidth-efficiency relation(3.5 mark) (7)
Shannon’s Limit(3.5 mark)
b) Channel diagram(BSC)(2 mark) (7)

I(X;Y)=0.7331bits/message symbol(4 marks)

Transmission rateRt=73.31bits/sec(2 marks)

Page 2of 4
 1200ECT306052301

OR
14 a) S/N=10^3(3 mark) (7)
Channel capacity=9.967Mbits/sec(4 mark[2 marks can be given to equation for
channel capacity and 2 marks to Channel capacity=9.967Mbits/sec]
)
b) Channel diagram(2 mark) (7)
Derivation(5 mark)
Module III
15 a) Code vectors(4 mark) (7)
Encoding circuit(3 mark)
b) Error detection and correction capabilities with proof(7 mark) (7)
OR
16 a) Procedure to prepare the standard array(3.5 mark) (7)
Preparation of the standard array(3.5 mark)
b) Error detection and correction(4 marks) (7)
For [0111110] error vector is [0100000] and for [1011100] error vector is
[0010000]
Syndrome calculation circuit(3 mark)
Marks can be given for solutions obtained using G = [P: Ik ]
Module IV
17 a) Circuit(3.5 mark) Explanation(3.5 mark) (7)
b) Generator matrix(3.5 mark) 1 1 0 1 0 0 0 (7)
0 1 1 0 1 0 0
1 1 1 0 0 1 0
0 0 0 0 0 0 1
Parity check matrix(3.5 mark)
1 0 0 1 0 1 1
0 1 0 1 1 1 0
0 0 1 0 1 1 1

Marks can also be given for non-systematic G and H matrix, as it is easy to find
that directly from generator and parity check polynomial.

Non-systematic G and H matrix,

Page 3of 4
 G=
(3.5 marks)
Non-systematic H can be obtained from xk h(x-1) as

(3.5 marks)
OR
18 a) Syndrome calculation circuit(4 marks) (14)
Contents of shift register table(6 mark)
Finding error vector E[0010000](2 mark)
Error correction[corrected code vector is 1100101](2 mark)

Module V
19 i)Encoder block diagram(6 mark) (14)
ii)Generator matrix(4 mark)(The four marks for generator matrix can be
distributed to the (i) and (iii)as 2 marks for(i) and 2 marks for(iii) since matrix
description for convolutional codes is not specified in syllabus )
iii)Code vector using time domain approach(4 mark)
Code vector using transfer domain approach(4 mark)
[111 010 001 110 100 101 011]
(Full marks(8) can be given for finding the code vector using any one method)

OR
20 Algorithm(6 marks) (14)
Explanation using example and suitable diagram(8 mark)

****

Page 4of 4
C 0300ECT306052201 Pages: 4

Reg No.:_______________ Name:__________________________

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY
Sixth Semester B.Tech Degree Examination June 2022 (2019 Scheme)

Course Code: ECT306

Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours
PART A
Answer all questions, each carries 3 marks. Marks

1 A source transmits two independent messages with probabilities p and (1-p) (3)
respectively. Prove that the entropy is maximum when both the messages are
equally likely and plot the graph for Entropy H.
2 Define (3)
i. Entropy
ii. Information rate
3 State Shannon’s second theorem on channel capacity. Give the Positive and (3)
Negative statements.
4 Illustrate channel diagram for a Discrete memoryless channel with an example. (3)
5 Differentiate (3)
i. Systematic and non-systematic codes
ii. Linear and non-linear codes
6 For a (7,4) linear block code, the Parity Check matrix is (3)

Obtain the Generator matrix

7 The generator polynomial of a (15,7) Cyclic code is, (3)

Find the code vector in Systematic form for the message vector 1101100

8 Explain the various parameters of RS codes? (3)

9 Draw the encoder circuit of a (2, 1, 3) convolutional encoder, if the generator (3)
sequences are g(1)=(1 0 0 1) and g(2) = (1 1 0 1) respectively.
10 Explain the concept of Tanner graph in LDPC Code? (3)

Page 1of 4
 0300ECT306052201

PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) A discrete source emits one of six symbols once in every milliseconds. The (7)

source probabilities are .Find the source entropy and

Information rate.
b) State and prove Kraft’s inequality. (7)
OR
12 a) A DMS has 6 codes with probabilities as p(x1)=0.25, p(x2)=0.3, p(x3)=0.12, (7)
p(x4)=0.2, p(x5)=0.08 and p(x6)=0.05.Obtain the Huffman codes and find the
code efficiency and redundancy.
b) Calculate the information rate of a telegraphy system which uses a dash and dot (7)
as symbols. Assume that the dash is twice as long as dot and half as probable.
The data last for 0.2 milliseconds and the same interval exists for the pause
between the symbols.
Module II
13 a) For the Channel matrix given below, find H(X), H(Y) and H(X,Y). Assume all (7)
input symbols are equally likely.

b) Explain the relation between differential entropy and entropy. (7)

OR
14 a) Given an AWGN channel with 5 K Hz bandwidth and the noise power spectral (7)
density η/2 =10-9 W/Hz. The signal power required at the receiver is 0.2 mW.
Calculate the capacity of this channel.
b) Derive the capacity of a Binary Symmetric Channel. (7)
Module III
15 a) For a systematic linear block code, the three parity check bits, c4, c5 and c6 are (7)
given by:

Page 2of 4
 0300ECT306052201

i. Construct the Generator matrix

ii. Decode the received code word 101100.

b) For a (7,4) linear block code, the Generator matrix is (7)

i. Find all the codewords.

ii. Comment on the error detection and correction capability.

iii. Draw the encoder circuit

OR
16 a) Write notes on (7)
i. Rings
ii. Finite fields.
b) The Parity check matrix of a (7,4) linear block code is given as (7)

Construct the code words and show that this is a Hamming code.
Module IV
17 a) The generator polynomial of a (7,4) Cyclic code is . Find the (7)
code vectors corresponding to the message vectors 1011 and 1101 in Non-
Systematic form.

b) Draw the encoder circuit for a (7,4) Cyclic code with and (7)
obtain the codeword for the message sequence 1001.
OR
18 a) Obtain the generator matrix of a (7,4) Cyclic code for the generator polynomial (7)

Page 3of 4
 0300ECT306052201

in Non-systematic form and using that find the codeword

for the message vector 1100 and 1111.
b) For a (7,4) Cyclic code, the received vector Y is 1110101 and the generator (7)
polynomial is p3+p+1. Draw the syndrome calculation circuit and correct the
single error in the received vector.
Module V
19 a) For a Convolutional encoder, the generator sequences are given as, (7)
g(1) = (1,1,0) and g(2) = (1,0,1).Obtain the
a. Code Tree for the message sequence 1011
b. State diagram
b) Explain the Message-passing decoding scheme for LDPC codes. (7)
OR
20 a) For a Convolutional encoder, the generator sequences are given as, (7)
g(1) = (1,0,1) and g(2) = (0,1,1).
a. Draw the encoder circuit.
b. Output sequence for a message sequence 1011 using Time domain
approach.
b) For the convolutional encoder shown in figure , decode the sequence , (7)
Y= 11 01 11 00 01 10 using Viterbi algorithm.

****

Page 4of 4
 0300ECT306052201
Pages 6

APJ ABDUL KALAM TECHNOLOGICAL UNIVERSITY

Scheme for Valuation/Answer Key
Scheme of evaluation (marks in brackets) and answers of problems/key
Sixth Semester B.Tech Degree Examination June 2022 (2019 scheme)
Course Code: ECT306
Course Name: INFORMATION THEORY AND CODING
Max. Marks: 100 Duration: 3 Hours

PART A
Answer all questions, each carries 3 marks. Marks

1 Entropy for (2 Marks) (3)

Graph (1 Marks)
2 i. Entropy – Definition (1 Marks), Equation (0.5 Marks) (3)
ii. Information rate – Definition (1 Marks), Equation (0.5 Marks)
3 Shannon’s theorem on channel capacity: (3)
Positive statement (1.5 Marks)
Negative statement (1.5 Marks)
4 Channel diagram: (3)
Significance (1 Marks), Elements in channel diagram (1 Marks), Example (1 Marks)
5 i. Difference between Systematic and non-systematic codes (1.5 Marks) (3)
ii. Linear and non-linear codes (1.5 Marks)
6 (3)

→G →

7 →Remainder polynomial (where q=n-k) (3)

Code word polynomial = X(p) = pn-k M(p) + Rem{pn-k M(p) /G(p)}

Codeword for 1101100 →110110001100111- (3 Marks)

8 3 parameters (3) (3)

Page 1of 6
 0300ECT306052201
Pages 6

9 (3)

10 Bipartite graphs-Tanner graphs (3)

Left-hand nodes - variable (symbol) nodes→ correspond to elements of the codeword.
Right-hand nodes - check nodes→correspond to the set of parity-check constraints
satisfied by codewords
Said to be regular→ all the nodes of a similar kind have exactly the same degree.
Marks can be awarded for description on either Regular or irregular LDPC codes.
PART B
Answer one full question from each module, each carries 14 marks.
Module I
11 a) H(X): Equation (1 marks), Ans=1.9375 bits/symbol (3 marks), (7)
Information Rate: Equation (1 marks), Ans=1937.5 bits/sec (2 marks)
b) Statement (2 marks), Proof (5 marks) (7)
OR
12 a) Huffman Code (3 marks), Lmin, L (1 marks) (7)
Code word lengths {2, 2, 2, 3, 4, 4}.
Entropy, H= 2.3601 bits/symbol, Average codeword length, L = 2.38 bits/symbol
Code Efficiency = 99.15% (2 marks)
Redundancy =0.85% (1 marks)
b) Symbol rate: Equation (1 marks), Ans=2.14x103 symbols/sec (2 marks) (7)
Entropy: Equation (1 marks), Ans=0.92 bits/symbol (2 marks)
Information rate=1971 bits/sec (1 marks)
Module II
13 a) P(X,Y) matrix (1 marks) (7)
P(y1)=0.104,p(y2)=0.188,p(y3)=0.458,p(y4)=0.25 (2 marks)
H(X)=2 bits/symbol (1 marks) , H(Y) =1.808 bits/symbol (1 marks)

Page 2of 6
 0300ECT306052201
Pages 6

H(X,Y) =3.1696 bits/symbol (2 marks)

b) Equations (2 marks), Relation (5 marks) (7)
OR
14 a) N=10-5 W (1.5 marks), S/N=20 (2 marks), (7)
Capacity: Equation (1 marks), Ans=21.96x103 bits/sec (2.5 marks)
b) Channel diagram- (1 marks) (7)
P(Y/X)- (2 marks)
H(Y/X)- (2 marks)
] (2 marks)

Module III
15 a) (7)
Parity matrix (2 marks)

Generator matrix (1 marks)

Syndrome=110 (2 marks), Correct codeword=111100 (2 marks)

If the student has not attempted the Qn. 15 b), the 7 marks of Qn. No. 15 b) can
be distributed to Qn. No. 15 a).Then, total marks assigned to Qn. 15 a) will be 14.
b) There is a mistake in the question: (7)
1. Full 7 marks can be awarded if the student has attempted solving (6,3) code (partial
or complete) as per the given matrix.

OR

2. If the student has not attempted the question 15 b), the 7 marks can be
distributed to Qn. No. 15 a). Then total marks assigned to Qn. 15 a) will be
14.

OR
16 a) i. Rings-Explanation (3.5 Marks) (7)
ii. Finite fields-Explanation (3.5 Marks)

Page 3of 6
 0300ECT306052201
Pages 6

b) Generator matrix-(2 Marks) (7)

Codewords-(2 Marks)
Minimum distance-(2 Marks)
Explanation (1 Marks)
OR
7 Marks can be awarded if the student answers the question by finding the
hamming bound.
Module IV
17 a) Since Non-systematic Cyclic code is not specified in the syllabus, (7)
1. Full 7 marks can be awarded if the student has attempted the question 17 a) (partial
or complete)
M=1011, M(p)=p3+p+1 - (1 Marks),
X(p)=G(p)M(p)=( p3+p+1)( p3+p+1) =p6+p2+1- (3 Marks),
➔ Codeword=1000101 (3 Marks)
M=1101, M(p)=p3+p2 +1 - (1 Marks),
X(p)=G(p)M(p)= ( p3+p+1) ( p3+p2+1) =p6+p5 + p4 +p3 + p2+p+1 - (3 Marks),
➔ Codeword=1111111 (3 Marks)
OR
2. If the student has not attempted the Qn. 17 a), 7 marks of Qn. 17 a) can be distributed
to 17 b). Then total marks for question 17 b) will be 14.
b) (7)

Page 4of 6
 0300ECT306052201
Pages 6

Circuit-(3 Marks)
Table-(3 Marks)
Generated code vector- 0111001-(1 Marks)
If the student has not attempted the question 17 a) , 7 marks of 17 a) can be distributed
to 17 b).Then total marks for question 17 b) will be 14.
OR
18 a) Since Non-systematic Cyclic code is not specified in the syllabus, (7)
1. Full 7 marks can be awarded if the student has attempted (partial or complete)
the question 18 a)
OR
2. If the student has not attempted the question 18 a) , 7 marks of Qn. No. 18 a) can
be distributed to 18 b). Then total marks for question 18 b) will be 14.
b) (7)

Circuit-(4 marks)
Syndrome-001-(6 marks)
Corrected codeword-1100101-(4 marks)
If the student has not attempted the question 18 a) , 7 marks of Qn. No. 18 a) can be
distributed to 18 b). Then total marks for question 18 b) will be 14.
Module V
19 a) Since Code tree is not specified in the syllabus, full 7 marks can be awarded if the (7)
student has completed state diagram
State diagram -(7 marks)
b) Concept-(3 marks) (7)
Steps-(4 marks)

Page 5of 6
 0300ECT306052201
Pages 6

OR
20 a) (7)

Circuit-(2 marks)
Output-(5 marks)
v(1) = (1 0 1 1) * (1 0 1) =(1 0 0 1 1 1)
v(2) = (1 0 1 1) * (0 1 1) =(0 1 1 1 0 1)
Output v = (10, 01, 01, 11, 10, 11)

b) (7)

Diagram –(3 marks)

Maximum likelihood path-(2 marks)
Decoded signal-(2 marks)
****

Page 6of 6