Information Theory Questions
Information Theory Questions
Find an optimal set of codewords for an instantaneous code for each of the following probabilities:
10 9 8 7 7
(a) 𝑝 = (41 , 41 , 41 , 41 , 41)
9 9 1 9 1 2 9 1 3
(b) 𝑞 = (10 , (10) (10) , (10) (10) , (10) (10) , … )
Solution:
𝑋 𝑌
Solution:
1
3. Consider a discrete memoryless source with the alphabet set X = {𝐴, 𝐵, 𝐶}. Decode the following Lempel-Ziv
code and obtain the source string. Show all the steps.
Solution:
2
4. Consider a binary discrete memoryless source with the alphabet set X = {𝑥1 , 𝑥2 } that emits symbols at the rate
𝑣𝑠 = 100 bits/s. Assume 𝑃(𝑥1 ) = 0.3. The channel is described by the transition matrix
0.4 0.6
𝑃=[ ]
0.75 0.25
(a) Find the entropy and the information rate of the source.
(b) Find the mutual information and the information rate of the channel.
(c) Find the decision rule for the receiver yielding the minimum error probability.
Solution:
3
5. Consider a linear block code described the generator matrix given by
1 0 0 1 0
𝐺 = [0 1 0 1 1]
0 0 1 0 1
(a) Find all the codewords.
Solution:
6.
4
Solution:
5
6
00011
7
7.
Solution:
8
8.
Solution:
9
9.
10
Solution:
10.
Solution:
11
11.
Solution:
12
13
12.
14
Solution:
15
16