25‐01‐2021

Principles of Communication
Engineering I Course Outcomes
1. Understand random variables and random processes.
Principles of Communication • Course No : ELC2420 2. Analyse different amplitude modulation schemes.
Engineering I • Credits : 4 3. Analyse different angle modulation schemes.
• Course Category : Departmental Core
4. Explain sampling processes and reconstruction.
• Pre‐requisite(s) : ELC2410 (Signals and Systems)
• Contact Hours (L‐T‐P) : 3‐1‐0 5. Analyse the behaviour of communication system in the
• Type of Course : Theory presence of noise.

Syllabus
Unit I: Random Variables and Stochastic Processes
• Review of Random Variables; Probability Distribution and Probability Density Functions; Books
Uniform, Gaussian, Exponential and Poisson Random Variables; Statistical Averages;
Random Processes; Correlation; Power Spectral Density; Analysis of Linear Time 1. Simon Haykin, Communication Systems, 4th Edition, John Wiley
Invariant Systems With Random Input; Noise and Its Representations & Sons, 2001.
Unit II: Amplitude Modulation
• Introduction to Modulation; Amplitude Modulation Systems (AM, DSBSC, SSBSC, VSB
2. G R Cooper and C D McGillem, Probabilistic Methods of Signals
and Systems Analysis, Oxford University Press, 1998.
Unit I
Modulation/Demodulations); Frequency Division Multiplexing; Superhetrodyne Radio
Receiver; Equivalent Receiver Model, Noise in CW Receivers Using Coherent Detection, 3. H Taub, D L Schilling and G Saha, Principles of Communication Random Variables and Processes
Noise in CW Receivers Using Envelope Detector Systems, 3rd Edition, Tata McGraw Hill, 2008.
Unit III: Angle Modulation
• Angle Modulation: Frequency and Phase Modulation; Generation and Demodulation of
4. A B Carlson, Communication Systems, McGraw Hills, 2002.
Narrowband and Wideband FM; FM Broadcasting; Non‐linear Effects in FM Systems; 5. J G Proakis and M Salehi, Communication Systems Engineering,
Noise in FM Receivers, FM Threshold Effect 2nd Edition, Pearson Education, 2006.
Unit IV: Sampling and Pulse Modulation
• Sampling Theorem; Various Sampling Techniques; Sampling of Low Pass and Bandpass
Signals; Time Division Multiplexing; Generation and Recovery of PAM, PWM and PPM
Signals

Contents Introduction Probability

• Probability theory is rooted in phenomena that, explicitly or implicitly,
• Introduction can be modeled by an experiment with an outcome that is subject to
• Fourier transform is a mathematical tool for the representation of chance.
• Probability
deterministic signals.
• Probability Distribution and Probability Density Functions • Probability theory is devoted to the study of uncertainty and
• Random Variables • Deterministic signals: The class of signals that may be modeled as variability
• Statistical Averages completely specified functions of time. • Probability quantifies how uncertain we are about future events.
• Random Processes • A signal is “random” if it is not possible to predict its precise value • Sample point: If an experiment has K possible outcomes, then for the
• Mean, Correlation and Covariance Functions in advance. kth possible outcome is called a sample point, which we denote by sk.
With this basic framework, we make the following definitions:
• Transmission of a Random Process through a Linear Filter • A random process consists of an ensemble (family) of sample • Sample space: The set of all possible outcomes of the experiment is
• Power Spectral Density
functions, each of which varies randomly with time. called the sample space, which we denote by S.
• Gaussian Process
• Noise • A random variable is obtained by observing a random process at a • An event corresponds to either a single sample point or a set of
• Narrowband Noise fixed instant of time. sample points in the space S.
 25‐01‐2021

Probability Probability Probability

• A single sample point is called an elementary event.
• The entire sample space S is called the sure event; and the null set 
• The following properties of probability measure 𝑃 may be
is called the null or impossible event
easily derived:
• Two events are mutually exclusive if the occurrence of one event
precludes the occurrence of the other event. 𝑃 𝐴̅ 1 𝑃𝐴
• A probability measure 𝑃 is a function that assigns a non‐negative • When events A and B are not mutually exclusive:
number to an event 𝐴 in the sample space S and satisfies the
following three properties (axioms): 𝑃 𝐴 ∪ 𝐵 =𝑃 𝐴 +𝑃 𝐵 −𝑃 𝐴 ∩ 𝐵
1. 0 𝑃 𝐴 1 • If 𝐴 , 𝐴 ,..., 𝐴 are mutually exclusive events that include all
2. 𝑃 𝑆 1 possible outcomes of the random experiment, then
3. If 𝐴 ∪ 𝐵 are the union of two mutually exclusive events, then
𝑃 𝐴∪𝐵 𝑃𝐴 𝑃𝐵
𝑃𝐴 𝑃𝐴 ⋯ 𝑃𝐴 1

Probability
• Let 𝑃 𝐵|𝐴 denote the probability of event 𝐵, given that event 𝐴
Conditional Probability Extra
has occurred. The probability 𝑃 𝐵|𝐴 is called the conditional
probability of 𝐵 given 𝐴. • Suppose that the condition probability 𝑃 𝐵|𝐴 is simply
• 𝑃 𝐵|𝐴 is defined by: equal to the elementary probability of occurrence of
𝑃 𝐴∩𝐵 event B, that is
𝑃 𝐵|𝐴
𝑃𝐴 𝑃 𝐵|𝐴 =𝑃 𝐵 𝑃 𝐴∩𝐵 =𝑃 𝐵 𝑃 𝐴
• Bayes’ rule ∩
𝑃 𝐴|𝐵 𝑃𝐴
• We may re‐write as 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐵|𝐴 𝑃 𝐴
• It is apparent that we may also write 𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴|𝐵 𝑃 𝐵 • Events A and B that satisfy this condition are said to be
• Provided 𝑃 𝐴 ≠ 0, we may determine 𝑃 𝐵|𝐴 by using the relation statistically independent.
𝑃 𝐴|𝐵 𝑃 𝐵
𝑃 𝐵|𝐴
𝑃 𝐴

Identity Name
A∪B = B∪A Commutative laws
A∩B = B∩A Commutative laws
(A∪B)∪C = A∪(B∪C) = A∪B∪C
(A∩B)∩C = A∩(B∩C) = A∩B∩C
Associative laws
Associative laws
Example
A∪(B∩C) = (A∪B)∩(A∪C) Distributive laws
A∩(B∪C) = (A∩ B)∪(A∩C) Distributive laws
A∪∅ =A Identity laws • An experiment is performed in which a fair coin is • Example: Of all bit sequences of length 8, an 8‐bit sequence
A∩U = A Identity laws tossed. Let Nh denote the number of times head is selected at random. Assuming that the probability of a bit
A∪U = U Domination laws shows up as the trail outcome of N trails. Make a being 0 is equal to that being 1, determine the probability
A∩∅ =∅ Domination laws program to plot the relative frequency of head that the selected bit sequence starts with a 1 or ends with
A∪A = A Idempotent law (Nh/N) versus N where N varies from 1 to 100. the two bits 00.
A∩A = A Idempotent law Comment of the plot obtained.
(Ac)c = A Complementation law
A∪Ac = U Complement laws
A∩Ac = ∅ Complement laws
A−B = A∩Bc Relative complement law
A∪(A∩B) = A Absorption laws
A∩(A∪B) = A Absorption laws
(A∪B)c = Ac∩Bc De Morgan’s laws
(A∩B)c = Ac∪Bc De Morgan’s laws
A⊆B iff A∪B = B Consistency laws
A⊆B iff A∩B = A Consistency laws
|A∪B| = |A|+|B|− ∣A∩B∣ Inclusion–exclusion principle
 25‐01‐2021

• Example: A box contains 20 balls, numbered from 1 to 20 inclusive. • Example: Of all bit sequences of length 8, an 8‐bit
Determine the probability that the number on a ball chosen at random is sequence is selected at random. Assuming that the
either a prime number or a multiple of 4. probability of a bit being 0 is equal to that being 1,
• Example: In a game of darts, the dartboard has a radius of 22.5 cm. determine the probability that the selected bit
Consider the sample space is the entire dartboard, it is thus a sequence starts with a 1 or ends with the two bits 00.
continuous sample space. Assuming a dart is thrown at random and
lands on the board, determine the probability that it lands in a region
• Solution: Let A1 be the event that includes prime numbers, and A2 be • Solution: The sample space S includes 256 equally likely outcomes, as we
within 7.5 cm from the center of the board, and the probability that can construct a bit sequence of length eight in 28 = 256 ways, i.e. |S| = 256.
the dart lands exactly at the center of the board. the event that includes numbers that are multiples of 4, that is we have The event A includes bit sequences of length 8 that all begin with a 1, we
can also construct such sequences in 28−1 = 27 = 128 ways, i.e. |A| = 128.
• Solution: The total area of the dartboard is (22.5)2𝜋 cm2. The event of A1 = {2, 3, 5, 7, 11, 13, 17, 19} → P(A1) = 8/20 = 2/5 The event B includes bit sequences of length eight that all end with the two
bits 00, we can also construct such sequences in 28−2 = 26 = 64 ways, i.e. |B|
interest, which represents the area of a region within 7.5 cm from the A2 = {4, 8, 12, 16, 20} → P(A2) = 5/20 = ¼ = 64. Some of the ways to construct a bit sequence of length 8 starting with
center, is (7.5)2𝜋 cm2. We also have the following: a 1 are the same as the ways to construct a bit sequence of length 8 that
ends with the two bits 00, there are 28−3 = 25 = 32 ways to construct such a
Hence, the probability of interest is (7.5)2𝜋/((22.5)2𝜋)=(7.5/22.5)2 A1 ∩ A2 = ∅ → P(A1 ∩ A2) = 0 sequence, i.e. |A∩ B| = 32.
Using the principle of inclusion–exclusion, the number of bit sequences
=(1/3)2=1/9 starting with the bit 1 or ending with the two bits 00 is thus as follows:
As the events A1 and A2 are mutually exclusive, we can use Axiom III to |A ∪ B| = |A| + |B| − |A ∩ B| = 128 + 64 − 32 = 160
Although it is possible that the dart lands exactly at the center of the determine the probability that the number on a ball chosen at random The probability of selecting the random sequence of interest is then
board, the probability of its occurrence is zero, as the center of the is either a prime number or a multiple of 4. We thus have 160/256 = 5/8.

board is one of the infinitely many points that the dart can hit.
P(A1 ∪ A2) = P(A1) + P(A2) = 2/5 + 1/4= 13/20

• Example: There are 10 000 smart phones in a warehouse,

• Example: Determine the conditional probability that a family out of which 10 are defective. Two phones are selected at
with two children has two girls, given they have at least one girl. random and tested one by one. Determine the probability
Example: Suppose for the events A and B, Assume the probability of having a girl is the same as the
we have P(A) = 0.8 and P(B) = 0.6. Show that probability of having a boy. that both phones are defective.
P(A∩B)≥0.4.
• Solution: Let A denote the event that the first phone tested
• Solution: The four possibilities GG, GB, BG, and BB, each is defective, we therefore have
reflecting the order of birth, are equally likely, where G
• Solution represents a girl and B represents a boy. Let A be the event P(A) = 10/10000 = 1/1000
that a family with two children has two girls, and let C be the
event that a family with two children has at least one girl. We Let B denote the event that the second phone tested is
We have defective. Therefore, the event B ∣A represents that the
therefore have the following events and probabilities:
second phone tested is also defective, and we have
P(A∪B) = P(A) + P(B) − P(A∩B) ≤ 1 A = {GG} → P(A) = 1/4
P(B ∣ A) = 9/9999 = 1/1111
→ 0.8 + 0.6 −P(A∩B) ≤ 1 C = {GG,GB,BG} → P(C) = 3/4
We can thus obtain
→ P(A∩B) ≥ 0.4 A ∩ C = {GG} → P(A ∩ C) = 1/4
P(A ∩ B) = P(B ∣ A)P(A) =( 1/1111)( 1/1000)
We can thus conclude
≅ 0.9 × 10−6
P(A ∣ C) = P(A ∩ C)/P(C) =(1/4)/(3/4)= 1/3

• Example: In the network of switches shown in Figure, each • Example: A technician has three resistors, with values 100, 300, and 900 Ω.
They are connected randomly in the configuration, as shown in figure below;
switch operates independently of all others and each switch
has a probability of 0.5 of being closed. What is the 1.
now, find.
What is the probability that the equivalent resistance of this network will
Random Variables
be 390 Ω?
probability of a complete path through the network? 2. What is the probability that the equivalent resistance will be >390 Ω? • A function whose domain is a sample space and whose
3. What is the probability that the equivalent resistance will be <1000 Ω? range is a set of real numbers is called a random
variable of the experiment.
• Random variable is used to describe the process of
assigning a number to the outcome of a random
experiment.
• Solution: All possible resistor configurations will form a set. • Random variables may be discrete and take only a finite
S={(100+300||900), (100+900||300), (300+100||900), number of values, e.g. coin‐tossing experiment.
• Solution: Let A be the event of a complete path through the (300+900||100), (900+100||300), (900+300||100)}
network. The path in the network will be completed by the • It may be continuous and take a range of real values, e.g.
1. Probability P(A) of getting equivalent resistance of 390 Ω can be
following switch combination. amplitude of a noise voltage.
found by tracking the combinations that give 390 Ω. They are 300
+ 100||900 and 300 + 900||100. • If the outcome of an experiment is 𝑠, then the random
{(S1, S2, S3) , (S1, S4) , (S1, S2, S4) , (S1, S3, S4) , (S1, S2, S3, S4)}
P(A) = 2/6 =1/3
variable is denoted as 𝑋 𝑠 or just 𝑋.
2. Probability P(B) of getting equivalent resistance >390 Ω can be • A particular outcome of a random experiment by 𝑥; that
Therefore, the probability of a complete path through the
found by tracking combinations that give equivalent resistance of is, 𝑋 𝑠 𝑥.
network is:
>390 Ω. They are 900 +100||300, 900 + 300||100. P(B) = 1/3
P(A)= 𝑃 𝐴 3. What is the probability P(C) that the equivalent resistance will be
<1000 Ω? This will happen for all resistor combinations. P(C) = 1
 25‐01‐2021

Random Variables
Probability Mass Function Distribution Functions

• For a discrete‐valued random variable, the probability • This is the probability that the random variable X takes
mass function describes the probability of each any value less than or equal to x. The distribution
possible value of the random variable. For the coin‐
tossing experiment, if it is a fair coin, the probability function is written as 𝐹 𝑥 so
mass function of the associated random variable may 𝐹 𝑥 P𝑋 𝑥
be written as:
1 • 𝐹 𝑥 is a function of 𝑥, not of the random variable X.
, 𝑥 0
2 • The distribution function has two basic properties
𝑃𝑋 𝑥 1
, 𝑥 1 1. The distribution function 𝐹 𝑥 is bounded between
2
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 zero and one.
2. The distribution function 𝐹 𝑥 is a monotone
nondecreasing function of x; that is,
• Sample space, random variables, and probability 𝐹 𝑥 𝐹 𝑥 𝑖𝑓 𝑥 𝑥

Probability Density Function Random Variables Random Variables

Example of a Uniform Distribution
• If 𝑋 is a continuous‐valued random variable and 𝐹 𝑥 is • Probability of the event x1 < X ≤ x2 equals
differentiable with respect to 𝑥, then a third commonly used
function is the probability density function, denoted by 𝑃𝑥 𝑋 𝑥 𝑃𝑋 𝑥 𝑃𝑋 𝑥
where
𝜕 𝐹 𝑥 𝐹 𝑥
𝑓 𝑥 𝐹 𝑥
𝜕𝑥
• A probability density function has three basic properties: 𝑓 𝑥 𝑑𝑥
1. Since the distribution function is monotone nondecreasing,
it follows that the density function is nonnegative for all
values of 𝑥.
2. The distribution function may be recovered from the density • Probability density function must always be a non‐
function by integration, as shown by: negative function, and with a total area of one.

𝐹 𝑥 𝑓 𝑠 𝑑𝑠

3. Property 2 implies that the total area under the curve of the
density function is unity.