Scribd
Upload
51 views2 pages

Homework 4: Problem 1

This document contains the instructions for Homework 4 assigned in the course EE5581 - Information Theory and Coding at the University of Minnesota for Fall 2018. It includes 4 problems related to computing channel capacity for various communication channels. The problems involve binary symmetric channels, general discrete memoryless channels, and channels with additive noise. Students are asked to determine channel capacities and optimal input distributions for the channels described. The homework is due on November 12 at 2:30 PM.

Uploaded by

Sanjay Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
51 views2 pages

Homework 4: Problem 1

This document contains the instructions for Homework 4 assigned in the course EE5581 - Information Theory and Coding at the University of Minnesota for Fall 2018. It includes 4 problems related to computing channel capacity for various communication channels. The problems involve binary symmetric channels, general discrete memoryless channels, and channels with additive noise. Students are asked to determine channel capacities and optimal input distributions for the channels described. The homework is due on November 12 at 2:30 PM.

Uploaded by

Sanjay Kumar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

EE5581 - Information Theory and Coding UMN, Fall 2018, Nov.

Homework 4
Instructor: Soheil Mohajer Due on: Nov. 12, 2:30 PM

Problem 1

Consider a Binary Symmetric Channel with cross probability p, given by


 
1−p p
p(y|x) =
p 1−p

(a) Compute C(p), the capacity of this channel.

Now, consider a communication channel with X = Y = {1, 2, 3, 4}, with transition probability matrix given
by
 
p 1−p 0 0
1 − p p 0 0 
p(y|x) =  
 0 0 q 1 − q
0 0 1−q q

where 0 < p < q < 1/2.

(b) Compute capacity of this channel, and write it in terms of C(p) and C(q).
Hint: Assume PX (1, 2, 3, 4) = (αr, (1 − α)r, β(1 − r), (1 − β)(1 − r)).

Next, consider a more general setting: there are two channels Ch1 : X1 → Y1 and Ch2 : X2 → Y2 , where
X1 ∩ X2 = ∅ and Y1 ∩ Y2 = ∅. Let P1 and P2 denote the transition probability matrices of channels, and the
capacity of two channels are given by C1 and C2 , respectively. A combined channel Ch : X1 ∪ X2 → Y1 ∪ Y2 is
defined by
 
P1 0
P = .
0 P2

(c) Compute the capacity of the channel P in terms of C1 and C2 .

Problem 2

Consider the communication channel with transition probabilities given in Fig. 1.

(a) Is this channel strongly symmetric? why?

1
2/3
1 1
1/3
1/3
1/3
X 2 2 Y
1/3
1/3
3 2/3
3

Figure 1: Channel

(b) It it weekly symmetric? why?

(c) Compute the channel capacity. (simplify the result to a real number)
Hint: You may assume p(X = 1) = p(X = 3) in the optimum input probability distribution.

(d) What does the channel with its optimum distribution remind you? Make a connection between this channel
and one of the standard channels have seen in the class, and compare their capacities.

Problem 3

(a) Consider the channel with X = Y = {0, 1}, and Y = XZ, where X and Z are independent binary random
variables. Let Z ∼ Ber(α), i.e., P (Z = 1) = α. Find the capacity of this channel and the maximizing
distribution on X.

(b) Now suppose that the receiver can observe Z as well as Y . What is the capacity?

Problem 4

Consider a channel with input alphabet X = {0, 1, 2, 3} where the output is given by Y = X + Z, and Z is
independent from X and uniformly distributed Z = {a, b, c} (here + denotes usual summation over real numbers,
not over a finite field).

(a) What is H(Z)? Write the capacity expression for this channel.

(b) If you want to maximize the capacity, what alphabet should you choose for Z? Determine a, b, c.

(c) Give the maximizing distribution for p(x) to achieve the capacity.

(d) What is the minimum capacity over all choices for the Z alphabet? Give distinct integer values a, b, c and a
distribution p(x) on X to achieve this minimum.

You might also like