Channel Coding I

Exercises

– SS 2018 –

Lecturer: Dirk Wübben

Tutor: Shayan Hassanpour
SPT, Room C 3220, Tel.: 0421/218-62387
E-mail: {wuebben, hassanpour}@ant.uni-bremen.de

Universität Bremen, FB1

Institut für Telekommunikation und Hochfrequenztechnik
Arbeitsbereich Nachrichtentechnik
Prof. Dr.-Ing. A. Dekorsy
Postfach 33 04 40
D–28334 Bremen

WWW-Server: http://www.ant.uni-bremen.de

Version from April 20, 2018

 April 20, 2018 I

General Information
• The dates for the exercises are arranged in the lectures. Tutorials take place in room N 1250. The exercises
cover the contents of past lectures and contain a theoretical part and a programming part, in general. The
students are advised to repeat the corresponding chapters and to prepare the exercises for presenting them at
the board.
• All references to passages in the text (chapter- and equation numbers) refer to the script: V. Kühn, “Kanal-
codierung I+II” in german language. References of equations in the form of (1.1) refer to the script, too,
whereas equations in the form (1) refer to solutions of the exercises.

• Although M ATLAB is used in the exercises, no tutorial introduction can be given due to the limited time. A
tutorial and further information can be found in
– MATLAB Primer, 3rd edition, Kermit Sigmon
– Practical Introduction to Matlab, Mark S. Gockenbach
– NT Tips und Tricks für MATLAB, Arbeitsbereich Nachrichtentechnik, Universität Bremen
– Einführung in MATLAB von Peter Arbenz, ETH Zürich
available on http://www.ant.uni-bremen.de/teaching/kc/exercises/.
• PDF-files of the tasks, solutions and M ATLAB codes are available on
http://www.ant.uni-bremen.de/de/courses/cc1/.

Within the university net the additional page

http://www.ant.uni-bremen.de/de/courses/cc1/
is available. Beside the tasks and solutions you will find additional information on this page, e.g the matlab
primer, the original paper of C. E. Shannon A mathematical theory of communication, some scripts and a
preliminary version of the book ”Error-Control Coding” of B. Friedrichs!
1 I NTRODUCTION April 20, 2018 1

1 Introduction

Exercise 1.1 Design of a discrete channel

a) Given is a transmitting alphabet consisting of the symbols −3, −1, +1, +3. They shall be transmitted over
an AWGN channel, which is real-valued and has the noise variance σn2 = 1. At the output of the channel
a hard-decision takes place, i.e. the noisy values are again mapped to the 4 symbols (Aout = Ain ), where
the decision threshold in each case lies in the middle of two adjacent symbols. Calculate the transition
probabilities Pr{Yµ |Xν } of the channel and represent them in a matrix.

b) Check the correctness of the resulting matrix by checking the sums of probabilities to one.
c) Determine the joint probabilities Pr{Xν , Yµ } for equiprobable transmitting symbols.
d) Calculate the occurrence probabilities for the channel output symbols Yµ .

e) Calculate the error probability Pe {Xν } for the transmitting symbols Xν and the mean overall error proba-
bility Pe .

Exercise 1.2 Statistics of the discrete channel

Given are the channels in figure 1. Complete the missing probabilities.

Channel 1 Channel 2
0.7 0.8
0.2 X0 Y0 0.3 0.2 X0 Y0

Y1 Y1 0.08

X1 Y2 X1 Y2
0.8
Fig. 1: Discrete channel models

Exercise 1.3 Binary symmetric channel (BSC)

a) The word 0110100 is transmitted by a BSC with the error probability Pe = 0.01. Specify the probability of
receiving the word 0010101, wrongly.
b) Determine the probability of m incorrectly received bits at the transmission of n bits.
c) For a BSC with the error probability Pe = 0.01, the probability of more than 2 errors at words of length 31
shall be determined.

Exercise 1.4 Serial concatenation of two BSCs

Two BSCs with Pe,1 and Pe,2 shall be connected in series. Determine the error probability of the new channel.

Exercise 1.5 Transmission of coded data over a BSC

Use Matlab to simulate the transmission of coded data over a Binary Symmetric Channel (BSC) with error
probability Pe = 0.1. Apply repetition coding of rate Rc = 1/5 for error protection.
2 S URVEY OF I NFORMATION T HEORY April 20, 2018 2

2 Survey of Information Theory

Exercise 2.1 Entropy

a) The average information content H(Xν ) of the signal Xν (also called partial entropy) shall be maxi-
mized. Determine the value Pr{Xν }, for which the partial entropy reaches its maximum value and specify
H(Xν )max . Check the result with M ATLAB by determining the partial entropy for Pr{Xν } = 0 : 0.01 : 1
and plotting H(Xν ) over Pr{Xν }.
b) The random vector (X1 X2 X3 ) can exclusively carry the values (000), (001), (011), (101) and (111) each
with the probability of 1/5.
Determine the entropies:

1. H(X1 ) 6. H(X2 |X1 )

2. H(X2 ) 7. H(X2 |X1 = 0)
3. H(X3 )
4. H(X1 , X2 ) 8. H(X2 |X1 = 1)
5. H(X1 , X2 , X3 ) 9. H(X3 |X1 , X2 )

Exercise 2.2 Channel capacity of a discrete memoryless channel

Determine the channel capacity for the following discrete memoryless channel on condition that Pr{Xν } = 1/3
is valid.

X0 1/2 Y0
1/3
1/6
1/6 1/2
X1 Y1
1/3
1/3
1/6
X2 Y2
1/2

Exercise 2.3 Channel capacity of the BSC

a) Derive the capacity for equiprobable input symbols Pr{X0 } = Pr{X1 } = 0.5 in dependence on the error
probability for a binary symmetric channel (BSC).
b) Prepare a M ATLAB program, which calculates the (input-output)mutual information of an asymmetric binary
channel for the input probabilities Pr{X0 } = 0 : 0.01 : 1. It shall be possible to set the error probabilities
Pe,X0 and Pe,X1 at the program start. (Attention: Pr{Y } must be calculated!)
c) For a BSC determine the (input-output)mutual information for several error probabilities Pe at a fixed input
probability Pr{X} within M ATLAB. Plot the corresponding curves of I(X; Y ).

Exercise 2.4 Channel capacity of the AWGNC

Derive the channel capacity for a Gaussian channel (σn2 = N0 /2) with normal distribution at input (σx2 = Es ).

x y = x+n