Scribd
Upload
33 views6 pages

CS CT 2 Alvi

1. Entropy is a measure of uncertainty in a probability distribution. It is calculated as the negative sum of the probability of each outcome multiplied by its log probability. 2. An event is a specific outcome or set of outcomes in a random process. Surprise measures how unexpected an event is based on its probability. 3. Instantaneous codes allow immediate recognition and decoding without look ahead or backtracking. Non-singular codes have no code word prefixes, ensuring unique decoding. Uniquely decodable codes are non-singular, while instantaneous codes are a superset and are not necessarily non-singular.

Uploaded by

Shahnaj Masud
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
33 views6 pages

CS CT 2 Alvi

1. Entropy is a measure of uncertainty in a probability distribution. It is calculated as the negative sum of the probability of each outcome multiplied by its log probability. 2. An event is a specific outcome or set of outcomes in a random process. Surprise measures how unexpected an event is based on its probability. 3. Instantaneous codes allow immediate recognition and decoding without look ahead or backtracking. Non-singular codes have no code word prefixes, ensuring unique decoding. Uniquely decodable codes are non-singular, while instantaneous codes are a superset and are not necessarily non-singular.

Uploaded by

Shahnaj Masud
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Communication System I

CT – 2
Information Theory
Topic:
1. What is Entropy? If the Sample space S = {S1, S2} and P(S1) = 0.96875
and P(S2) = 0.03125. Find Entropy. [2021]
2. What is Event & Surprise?
3. What do you know about instantaneous code? Write down the
properties of instantaneous code. [2021]
4. Design instantaneous binary code with length of 3,2,3,2,2 and 2,3,2,2,2.
[2021]
5. Define non-singular code, uniquely decodable code and instantaneous
code. Draw a relation among them. [2021]
6. For the PCM bit sequence 11010, draw the waveforms using the
following line coding techniques: [2021]
a. Polar NRZ
b. Dipolar (antipodal)
7. For the PCM bit sequence 1011, draw the waveforms using the following
line coding: [CT]
a. Polar NRZ
b. Dipolar (OOK)
c. Bipolar NRZ
d. Unipolar RZ
8. Design an instantaneous ternary code with lengths {2,3,1,1,2} [CT]’
9. If the sample space S ={s1,s2,s3,s4,s5,s6} and P(Si) = 1/6 for I = 1,2,3,4,5,6.
Find Entropy. [CT]
10. Define Line Coding. [2020]
11. For the PCM bit sequence 1101001, draw the waveforms using the
following line coding: [2020]
a. Unipolar RZ
b. Bipolar NRZ
c. Polar NRZ
12. Compare the performance among the polar (RZ), unipolar (NRZ),
bipolar (RZ) and bipolar (NRZ) line coding. [2020]
13. What is conditional entropy and redundancy? [2020]
Answer:
1)
Entropy: Entropy is a measure of the uncertainty or randomness in a
random variable or probability distribution.
If the sample space is S = {S1,S2,….,Sn} with probability distribution P, then
the entropy of the probability distribution is given by

Math: Example 1.2, Books: Roberto Togneri, Page: 5


2)
Event: An event is a specific outcomes or a set of outcomes in a random
process.
Surprise:

3)
instantaneous code:
Properties:

4)
5)
Non-singular code: A non-singular code is a type of code where no code word is a prefix of
any other code word.

A code is said to be uniquely decodable if, and only if, the nth extension of the code is non-singular
for every finite value of n.

Now, let's draw the relationship among these concepts:

1. Non-singular code ⊆ Uniquely decodable code: Every non-singular code is a


uniquely decodable code, as the absence of prefix code words ensures that decoding
can be done without any ambiguity.
2. Uniquely decodable code ⊆ Instantaneous code: Every uniquely decodable code is
not necessarily an instantaneous code. However, there exist uniquely decodable
codes that are also instantaneous codes. The instantaneous code property provides
an additional guarantee of immediate recognition and decoding without the need
for look ahead or backtracking.
3. Instantaneous code ⊄ Non-singular code: An instantaneous code can have code
words that are prefixes of other code words. While it ensures immediate recognition
and decoding, it does not guarantee the absence of prefix code words. Therefore, an
instantaneous code is not necessarily a non-singular code.

To summarize, non-singular codes are a subset of uniquely decodable codes, and


uniquely decodable codes are a subset of instantaneous codes. However,
instantaneous codes may or may not be non-singular codes.

6)
7)

8)

9)
10) Line coding is a technique that converts binary data into a specific electrical or optical
signal for reliable transmission in digital communication systems.

11)

You might also like