ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR

GUESS PAPERS SOLUTION

(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Unit 1:
Short Answers
Q. 1 Define information theory
Solution :

Q. 2 Explain the uncertainty? Also define information? How can we measure the information
Solution :
Conditions of Occurrence of Events
 If we consider an event, there are three conditions of occurrence.
 If the event has not occurred, there is a condition of uncertainty.
 If the event has just occurred, there is a condition of surprise.
 If the event has occurred, a time back, there is a condition of having some information
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 3 What is entropy? Explain its properties
Solution :
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4 What is conditional entropy and joint entropy. Define the relation between the joint and conditional
entropy.
Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 5 State the source coding theorem.

Ans:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Descriptive Answers: (5 to 20 Marks)
Q. 1 A discrete source emits one of the six symbols once every m-sec. the symbol probability are 1/2, 1/4, 1/8, 1/16, 1/32,
and 1/32 respectively. Find the source entropy and information rate

Q. 2 What is mutual information? Prove the following properties of mutual information

I(X;Y)= H(X)+H(Y)-H(X/Y)

I(X;Y)= H(X)- H(X/Y)

Solution:
Mutual information is a measure of the reduction of randomness of a variable given knowledge of another variable.
Mutual information is a quantity that measures a relationship between two random variables that are sampled
simultaneously. In particular, it measures how much information is communicated, on average, in one random
variable about another. Intuitively, one might ask, how much does one random variable tell me about another? For
example, suppose X represents the roll of a fair 6-sided die, and Y represents whether the roll is even (0 if even, 1 if
odd). Clearly, the value of Y tells us something about the value of X and vice versa. That is, these variables share
mutual information.
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
(i)

(ii)
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 3 A binary channel matrix is given by

𝟐/𝟑 𝟏/𝟑
𝟏/𝟏𝟎 𝟗/𝟏𝟎

Given that Px(x1)=1/3 Px(x2)=2/3

i. Determine H(X),H(X/Y), H(Y), H(Y/X) and I(X;Y)

ii. Also draw binary channel matrix
Solution:
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4. Given a DMS X with two symbols X1 and X2 and P(X1) = 0.9, P(X2) = 0.1. Symbol X1 and X2 are
encoding as follows.
Xi P(xi) Code
X1 0.9 0
X2 0.1 1
Find the efficiency and redundancy of this code.
Solution:

L = P(x1)n1 + P(x2)n2
= 0.9 * 1 + 0.1 * 1
=1

H (X) = ( 0.9 log2 (1/0.9) + 0.1 log2 (1/0.1)

= 0.469 b/symbol

Efficiency = (H(x) / L) = (0.469/1) = 0.469 = 46.9 %

Code Redundancy = 1 – code efficiency = 1 – 0.469 = 0.531 = 53.1%

Q. 5 A source Produces one of four possible messages during each interval having probabilities p1=1/2,
p2=1/4, p3=1/8 and p4=1/8. Obtain the information content of each of these messages
Solution:
I= log 1/p(x)
I1=1 bit
I2=2 bit
I3=3 bit
I3=3 bit
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY,
TECHNOLOGY JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01
01 INFORMATION THEORY AND CODEING
Unit 2:
Short Answers

Q. 1 What do you mean by Prefix Code?

Solution:
A prefix code is a type of code system (typically a variable
variable-length
length code) distinguished by its possession of the
“prefix property”,
ty”, which requires that there is no whole code word in the system that is a prefix (initial segment) of
any other code word in the system. “
Q. 2 What is channel capacity?
Solution:
Definition 1 (Discrete Channel) A system consisting of an input alphabet X and output alphabet Y and a probability
transition matrix p(y|x).
Definition 2 (Channel
Channel capacity) The “information” channel capacity of a discrete memoryless channel is
C = max p(x) I(X; Y )
where the maximum is taken over all possible input distribut
distribution p(x).
■ Operational definition of channel capacity: The highest rate in bits per channel use at which information can be
sent.
■ Shannon’s second theorem: The information channel capacity is equal to the operational channel capacity.
Properties of Channel Capacity
 C ≥ 0.
 C ≤ log |X |.
 C ≤ log |Y|.
 I(X; Y ) is a continuous function of p(x),
 I(X; Y ) is a concave function of p(x),
Q. 3 What is lemple ziv codes
Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4 What is the difference between Shannon-fano coding and Huffman coding techniques.
Solution:
BASIS FOR
HUFFMAN CODING SHANNON FANO CODING
COMPARISON

Basic Based on source symbol Based on the cumulative

probabilities distribution function

Efficiency Better Moderate

Developed by David Huffman Claude Shannon and Robert Fano

Invented in the year 1952 1949

Optimization provided High Low

Q. 5 Define coding and code efficiency

Solution:
A coding is a mapping from the discrete set of symbols {0, · · · , M − 1} to finite binary sequences
 For each symbol, m their is a corresponding finite binary sequence σm
 |σm| is the length of the binary sequence
code efficiency:
The ratio of minimum length code to average code word length
𝐿
𝜂=
𝐿
𝐿 ≥ 𝐻(𝑥)
𝐿 = 𝐻(𝑥)
𝐻(𝑥)
𝜂=
𝐿
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Descriptive Answers:
Q. 1 Apply the Shannon- fano coding procedure for the given message

[X] X1 X2 X3 X4 X5 X6 X7

[P] 0.4 0.2 0.12 0.08 0.08 0.08 0.04

Assume M=2

Calculate the efficiency of code

Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 2 Determine the Huffman code for the following message with their probability given

[X] X1 X2 X3 X4 X5 X6 X7

[P] 0.05 0.15 0.2 0.05 0.15 0.3 0.1

Calculate the efficiency of code

Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 3 Explain the types of error and classification of codes

Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 4 State and prove krafts inequality theorem

Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 5 Seven alphabets with Probabilities {0.25, 0.25, 0.125, 0.125, 0.125, 0.0625, and 0.0625} for source symbols
{s1, s2, s3, s4, s5, s6, s7}. Find Huffman Code of each source symbol.

Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Unit 3:
Short Answers

Q. 1 Explain use of Block code.

Solution :
Block codes are used For error detection and correction
A block code is a rule for converting a sequence of source bits , of length K, say, into a transmitted sequence of
length N bits, where, in order to add redundancy, N will of course be greater than K.

Q. 2 What is role of generator matrix?

Solution :
The linear block code (n, k), using the language of vector spaces, can be defined as a subspace of a vector space over
finite field GF(q), where q is the prime power. Every space is described by its basis—a set of linearly independent
vectors. The number of vectors in the basis determines the dimension of the space. Therefore, for an (n, k) linear
block code the dimension of the space is n, and the dimension of the code subspace is k. The basis vectors may be
put in a matrix of dimensions k × n, known as the generator matrix, in which every row represents a vector from the
coding subspace, and the columns represent corresponding vector components:

By elementary operations on rows in the generator matrix, the code may be transformed into systematic form
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 3 define the following :
i. Code word

An (n, k) linear code is a k-dimensional subspace of the vector space of all the binary n-tuples, so it is
possible to find k linearly independent code words g0, g1, · · · , gk−1 to span this space
ii. Code rate
Code rate =k/n
iii. Hamming distance
Hamming distance between two codewords c1 and c2, d(c1, c2), is defined as the number of locations in which
their respective elements differ. Hamming weight, w(c), of a code vector c is defined as the number of
nonzero elements in the vectors. The minimum distance, dmin, of a linear block code is defined as the smallest
Hamming distance between any pair of code vectors in the code.
iv. Code efficiency
Ratio of message bits to the no. of transmitted bits per block

Q. 4 What are systematic codes and non systematic codes?

Solution:

Q. 5 What are hamming codes?

Solution:
An (n, k) linear block code of minimum distance dmin can correct up to t errors if, and only if, t ≤ ⌊1/2(dmin − 1)⌋
(where ⌊ ⌋ denotes the largest integer less than or equal to the enclosed quantity) or, equivalently, dmin ≥ 2t + 1. If we
are only interested in detecting ed errors, then dmin ≥ ed + 1. Finally, if we are interested in detecting ed errors and
correcting ec errors, then dmin ≥ ed + ec + 1. The Hamming (7, 4) code is therefore a single error-correcting and
double error-detecting code. More generally, a family of (n, k) linear block codes with parameters
 Block length: n = 2m − 1
 Number of message bits: k = 2m − m – 1
 Number of parity bits: n − k = m
 dmin = 3
where m ≥ 3, are known as Hamming codes.
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Descriptive Answers
Q. 1 For the systematic linear block code, the three parity check bits are c4,c5,c6 are given

𝒄𝟒 = 𝒎𝟏 ⊕ 𝒎𝟐 ⊕ 𝒎𝟑

𝒄𝟓 = 𝒎𝟏 ⊕ 𝒎𝟐

𝒄𝟔 = 𝒎𝟏 ⊕ 𝒎𝟑

i. Construct generator matrix

ii. Construct all the codes generated by this matrix
iii. Realize an encoder for this code
iv. Determine the error detecting capability and error correcting capability
v. Prepare a suitable decoding table
vi. Decode the received word 101100 and 000110
Solution:
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 2 Write short note on linear block codes. Explain how are the received codeword are decode. What are the
advantages of LBC?
Solution :
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 3 Consider a (7, 4) linear code whose generator matrix is
G=
1 0 0 0 1 0 1

0 1 0 0 1 1 1

0 0 1 0 1 1 0

0 0 0 1 0 1 1

(i) Find all the code vectors of this code.

(ii) Find the parity check matrix of this code.
(iii) Find the minimum weight of this code
(iv) Prove equation CHT =0
Solution :
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4
i. prove that all error vector that differ by a code vector have same syndrome.

ii. Show that if Ci and Cj aretwo code vectors in an (n,k) LBC,then their sum is also a code vector

Q. 5 An error control code has the following parity check matrix.

𝟏𝟎𝟏 𝟏𝟎𝟎
𝑯 = 𝟏𝟏𝟎 𝟎𝟏𝟎
𝟎𝟏𝟏 𝟎𝟎𝟏
i. Determine the generator matrix
ii. Find the code word that begins with 101……
iii. Decode the received code word 110110. Also determine the error detecting capability of this code.
Solution:
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Unit 4:
Short Answers

Q. 1 Define cyclic codes.

Solution :
Cyclic codes form an important subclass of linear codes. These codes are attractive for two reasons:
 Encoding and syndrome computation can be implemented easily by employing shift registers with feedback
connections (or linear sequential circuits).
 Because they have considerable inherent algebraic structure, it is possible to find various practical methods
for decoding them.
An (n, k) linear code C is called a cyclic code if every cyclic shift of a code vector in C is also a code vector in C.

Q. 2 what are advantages and disadvantages of cyclic codes

Solution:
Advantages
 Encoder and decoder are simpler
 Simple and easy error correcting and decoding method of cyclic codes
 Burst errors can be detected
Disadvantages
 Error detection is simpler but error correction is complicated

Q. 3 differentiate between source coding, line coding and error detection coding
Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4 What are systematic codes and non systematic cyclic codes?

Q. 5 Define galois fields(GF) what are the properties of galois fields(GF)

 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Descriptive Answers: (5 to 20 Marks)
Q. 1 The generator polynomial for (7,4) cyclic code is given by G(p)=1+p+p 3
i. Determine the non systematic form of generator matrix
ii. Determine the systematic form of generator matrix
iii. Determine the parity check matrix from the generator matrix in (ii)
iv. Determine all the systematic and nonsystematic code vectors
v. Draw the encoder
vi. Draw the syndrome decoder
Solution :
1011000
0101100
(i) non systematic form of generator matrix g=
0010110
0001011
1000101
0100111
(ii) the systematic form of generator matrix g=
0010110
0001011
1110100
(iii)the parity check matrix h= 0111010
1101001
(iv) Determine all the systematic and nonsystematic code vectors
C=dg
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
v. Draw the encoder

vi. Draw the syndrome decoder

 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 2 A message 101101 is to be transmitted in cyclic code with a generator polynomial G(d)=d 4+d3+1. Obtain
the transmitted code word. how many check bits does the encoded message contain? Draw the encoding
arrangement for the same
Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 3 A cyclic (7,3) code has the generator polynomial g(x)=x 4+x2+x+1

Find the generator matrix for this code. Also find all the possible code words. Further find the minimum
distance for this code
Solution:
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 4 the generator polynomial for (15,7) cyclic code is g(x)=1+x 4+x6+x7+x8
i. Find the code vector for the message polynomial m(x)=x 2+x3+x4
ii. Assume that the first and last bit of the code vector X(x) for m(x)= x 2+x3+x4 is in error
Solution:

reminder =0
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 5 write a short note on BCH codes and R-S codes

Bose-Chaudhuri-Hocquenghem (BCH) codes offer flexibility in the selection of the block length and code
rate, and can be designed for correction of any given number of errors. A fast decoding algorithm can be
employed for hard-decision decoding of the BCH codes. For any integer m≥3 and 0<t<2m−1, there exists a t-
error-correcting BCH (n, k) code with n=2m−1 and n−k≤mt, whose minimum distance dmin is bounded as
follows: 2t+1≤dmin≤2t+2. BCH codes can be defined in the binary field, such as the Hamming codes, and in
the non-binary (symbol) field, such as the Reed-Solomon codes
For single error-correcting codes, if the total number of bits in a transmitted codeword is n,
then m=n−k check bits must be able to indicate at least n+1 different states. Of these, one state means no
error, and n states indicate the location of an error in each of the n positions, where it is also possible to have
an error in the redundancy bits themselves. So n+1 states must be discoverable by n−k bits, and n−k bits can
indicate 2n−k different states. Therefore, we must have 2n−k≥n+1 or equivalently 2m−1≥n, for an (n, k) code
with single error-correcting capability.
Hamming codes have dmin=3, and thus t=1, i.e., a single error can be corrected regardless of the number of
parity-check bits. An (n, k) Hamming code has m=n−k parity-check bits, where n=2m−1 and k=2m−1−m,
for m≥3. The parity-check matrix H of a Hamming code has m rows and n columns, and the last n−k columns
must be chosen such that it forms an identity matrix. No column consists of all zeros; each column is unique
and has m elements. In view of this, the syndrome of all single errors will be distinct and single errors can be
detected. By increasing the number of parity check bits k, the error-correcting capability remains unchanged
(i.e., t=1), but the code rate kn improves, of course at the expense of additional encoding and decoding
complexity
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Unit 5:
Short Answers

Q. 1 Define convolution codes

Solution:

Convolution code is an alternative to block codes, where n outputs at any given time unit depend on k inputs at that
time unit as well as m previous input blocks. An (n, k, m) convolution code can be developed with k inputs, n output
sequential circuit, and m input memory. The memory m must be large enough to achieve low error probability,
whereas k and n are typically small integers with k < n. An important special case is when k = 1, the information
sequence is not divided into blocks and can be processed continuously.
Convert any length message to a single ‘codeword’ • Encoder has memory and has n outputs that at any time depend
on k inputs and m previous input blocks
• Typically described by 3 parameters: –
n= no. of bits produced at encoder output at each time unit –
k= no. of bits input to encoder at each time unit
– And one of:
• m = memory of encoder = no. of prev. input blocks used to generate each output
(or)
• K = constraint length = max. no. of bits in a single output stream that can be affected by any input bit = 1+ maxi mi
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 2 what are advantages and disadvantages of convolution codes
Solution :
Advantages
 Storage hardware required by convolution decoder is less
 Decoding delay is small
 Synchronization problem doesn’t affect the performance of convolution codes
Disadvantages :
 They are difficult to analyse
 They are not developed

Q. 3 write a short note on interleaving

Solution :
ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING

Q. 4 Define the following

i. Code rate
ii. Constrained length
iii. Code dimension
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY,
TECHNOLOGY JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01
01 INFORMATION THEORY AND CODEING
Q. 5 compare linear block code and convolutio
convolution codes
Solution :

Descriptive Answers

Q. 1 Write a short note on sequential decoding

coding
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY, JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01 INFORMATION THEORY AND CODEING
Q. 2 Describe viterbi algorithm
Implementation: The Viterbi algorithm is implemented according to Eq., where in place of gk(p, q) we use the
simplified additive branch metric μ^k(p,q)=|rk−s(p,q)|2. There is no a priori term in the branch metrics, because all
transmitted symbols are independent and uniformly distributed over the 16-QAM alphabet. Because the channel
memory is μ = 3 and the alphabet size is M = 16, the number of states in the trellis is Mμ = 4096. Furthermore, there
will be M = 16 branches emanating from each node in the trellis. The overall complexity is thus extremely high,
significantly higher than the suboptimal equalizers considered above.
•
Performance: The gray curve in Fig. 8.16 shows the performance of the Viterbi algorithm. At high SNR, the Viterbi
algorithm significantly outperforms all of the linear, decision feedback, and THP strategies. At Pe = 10−5, the SNR
advantage of Viterbi over MMSE THP is 1.7 dB, and the advantage over ZF LE is more than 5.3 dB. Interestingly,
the performance advantage of Viterbi is not as significant at extremely low SNR values. In fact, when the SNR is so
low that the error probability exceeds 20%, the Viterbi algorithm is seen to perform slightly worse than the
significantly less complex THP strategies. The dashed curve labeled “MFB” shows the error probability of
the matched-filter bound, which is computed by substituting Eq. (8.10) into Eq. . The Viterbi algorithm closely
approaches the MFB at high SNR, falling only 0.2 dB short at Pe = 10−5.

Q. 3 Write a short note on Maximum likelihood decgding of convolution codes

 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY,
TECHNOLOGY JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01
01 INFORMATION THEORY AND CODEING
Q. 4 Consider the convolution encoder shown in figure below.the message bits are shifted into the encoder two
bits at a time.

i. Find the constraint length and the rate efficiency of the code
ii. Assume the initial content of the register to be zero and find the code block for the messgae input
message block 1101010
Solution:
Constraint length =nN
Where n=3
N=2
Code rate = k/n=2/3
Input :1101 01
C1=D’1+D2’
C2=d1’+d2’+d2’’
C3=d1’+d1’’+d2’’

I/P D1’ D1’’ D2’ D2’’ C1 C2 C3

11 11 00 110
01 01 11 100
01 01 01 010
Shift 00 01 011
Shift 00 00 000
Code word : 110 100 010 011 000
 ARYA INSTITUTE OF ENGINEERING & TECHNOLOGY,
TECHNOLOGY JAIPUR
GUESS PAPERS SOLUTION
(IIIyr B. Tech. V Semester)
5CS3-01
01 INFORMATION THEORY AND CODEING
Q. 5 Write a short note on trellis code