Scribd
Upload
118 views16 pages

Information Theory Questions

The document outlines several problems related to coding theory and information theory, including finding optimal codewords for given probabilities, calculating channel capacity, decoding a Lempel-Ziv code, and analyzing a binary discrete memoryless source. It also covers entropy, mutual information, decision rules for receivers, and linear block codes. Each section requires specific calculations and solutions to demonstrate understanding of the concepts.

Uploaded by

noopur807
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
118 views16 pages

Information Theory Questions

The document outlines several problems related to coding theory and information theory, including finding optimal codewords for given probabilities, calculating channel capacity, decoding a Lempel-Ziv code, and analyzing a binary discrete memoryless source. It also covers entropy, mutual information, decision rules for receivers, and linear block codes. Each section requires specific calculations and solutions to demonstrate understanding of the concepts.

Uploaded by

noopur807
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

1.

Find an optimal set of codewords for an instantaneous code for each of the following probabilities:
10 9 8 7 7
(a) 𝑝 = (41 , 41 , 41 , 41 , 41)
9 9 1 9 1 2 9 1 3
(b) 𝑞 = (10 , (10) (10) , (10) (10) , (10) (10) , … )

Solution:

2. Consider the channel shown below. Assume 𝑃(𝑋 = 0) = 𝑞.


𝜖

𝑋 𝑌

(a) Find the channel capacity 𝐶.


(b) Find the input probability distribution that achieves capacity.
(c) What is the input probability and capacity for the special cases 𝜖 = 1 and 𝜖 = 0.

Solution:

1
3. Consider a discrete memoryless source with the alphabet set X = {𝐴, 𝐵, 𝐶}. Decode the following Lempel-Ziv
code and obtain the source string. Show all the steps.

(000, 𝐴) (000, 𝐵) (010, 𝐶) (011, 𝐴) (010, 𝐴) (100, 𝐴) (110, 𝐵)

Solution:

2
4. Consider a binary discrete memoryless source with the alphabet set X = {𝑥1 , 𝑥2 } that emits symbols at the rate
𝑣𝑠 = 100 bits/s. Assume 𝑃(𝑥1 ) = 0.3. The channel is described by the transition matrix
0.4 0.6
𝑃=[ ]
0.75 0.25
(a) Find the entropy and the information rate of the source.

(b) Find the mutual information and the information rate of the channel.

(c) Find the decision rule for the receiver yielding the minimum error probability.

(d) Find the probability of error 𝑃𝑒 .

Solution:

3
5. Consider a linear block code described the generator matrix given by
1 0 0 1 0
𝐺 = [0 1 0 1 1]
0 0 1 0 1
(a) Find all the codewords.

(b) Construct the standard array.

(c) Find all the syndromes corresponding to the correctable errors.

Solution:

6.

4
Solution:

5
6
00011

7
7.

Solution:

8
8.

Solution:

9
9.

10
Solution:

10.

Solution:

11
11.

Solution:

12
13
12.

14
Solution:

15
16

You might also like