Probability and Random

Variables

Dr. Sadiq Ali

 Course Notes
• Lecture Slides.

• Probability, Statistics, and Random Processes for

Electrical Engineering, 3rd Edition, Alberto Leon-
Garcia.

• Advanced Engineering Mathematics by Erwin

Kreyzig.
 Assessment Breakdown

• Final - 50%.
• Midterm – 20%
• Quizzes – 20%
• Assignments 5%
• Attendance – 5%
 Course Outline
• Basic concepts in probability.
• Conditional probability and independent events.
• Discrete and continuous random variables,
distributions and density functions.
• Probability distributions.
• Linear regression and curve fitting.
• Stochastic processes.
• Applications to real life.
• Basic concept of Monte Carlo Simulations.
 Lecture Outline

• Applications of Probability
• Probability Basics
Probabilistic vs. Deterministic Events
• Probability is the likelihood of an event to occur.

– Characterized by randomness. That is to say that it is

applied to a Probabilistic Event (random experiment),
the outcome of which is not certain e.g. coin toss,
cricket match, noise in digital communication, etc.

– Does not apply to Deterministic Events where a

specific outcome is certain, e.g. V = IR
Applications – Digital Communication
Probability is essential for understanding Digital
Communication, particularly in analyzing message bits
passing through error-prone channels.
Applications – Analog Communication

Analog communication involves recovering a message

from a noisy signal. Understanding noise requires a
theory of randomness.
 Applications – Analog Communication

• Signal recovery involves estimating the original signal

from noisy data, relying on probability theory.
Applications – Wireless Communication
• Probability theory models noise, interference, and
fading (channels), crucial for systems like cell phones
and urban wireless networks.

• Amplifiers boost weak signals to extend

communication range but introduce thermal noise.
 Applications – Electrical Power
Engineering
• Analyze power system reliability by assessing failure
likelihood and predicting fluctuations.

• Design resilient systems to ensure stability under

uncertainty, using probability distributions to model
random failures and load variations.

• Evaluate voltage fluctuation risks from load variations

affecting equipment.

• Use normal, binomial, and Poisson distributions to

model failures and load changes
Probability - Basics
 Probability - Basics
• Many systems of interest involve phenomena that
exhibit unpredictable variation and randomness.

• Random Experiment an experiment in which the

outcome varies unpredictably when the experiment is
repeated under the same conditions.

• Deterministic models are not appropriate for random

experiments since they predict the same outcome for
each repetition of an experiment.
 Probability - Basics
• Example of a random experiment: suppose a ball
is selected from an urn containing three identical
balls, labeled 0, 1, and 2.
• The urn is first shaken to randomize the position of
the balls, and a ball is then selected.
• The number of the ball is noted, and the ball is then
returned to the urn.
• The outcome of this experiment is a number from
the set 𝑆 = {0, 1, 2}.
 Probability - Basics
• We call the set S of all possible outcomes the
Sample Space.

• The outcome of this experiment cannot consistently

be predicted correctly.

• Figure on the next slide shows the outcomes in 100

repetitions (trials) of a computer simulation of this
urn experiment.
Probability - Basics

Outcomes of urn experiment.

 Probability - Basics
• Suppose that the above urn experiment is repeated 𝑛
times under identical conditions.

• Let 𝑁0(𝑛), 𝑁1(𝑛), and 𝑁2(𝑛) be the number of times

the outcomes are balls 0, 1, and 2, respectively.

• Then the relative frequency of the outcome 1 is the

number of times the outcome 1 occurs divided by the
number of trials,
𝑓1 𝑛 = 𝑁1𝑛𝑛
 Probability - Basics
• Similarly, the relative frequency of the outcome 0 is
the number of times the outcome 0 occurs divided by
the number of trials,
𝑁0 𝑛
𝑓0 𝑛 = 𝑛

• And the relative frequency of the outcome 2 is the

number of times the outcome 2 occurs divided by the
number of trials,
𝑓2 𝑛 = 𝑁2𝑛𝑛
 Probability - Basics
• In general, the relative frequency of an outcome 𝑘
can be defined as,
𝑓𝑘 𝑛 = 𝑁𝑘𝑛𝑛

• Note that 𝑓𝑘(𝑛) varies less and less about a constant

value as 𝑛 is made large, that is,
lim 𝑓𝑘 𝑛 = 𝑝𝑘
𝑛→∞

• The constant 𝑝𝑘 is called the probability of the

outcome 𝑘. In our case k=0,1,2.
 Probability - Basics

lim 𝑓𝑘 𝑛 = 𝑝𝑘
𝑛→∞

𝑁𝑘 𝑛
lim 𝑓𝑘 𝑛 = lim = 𝑝𝑘
𝑛→∞ 𝑛→∞ 𝑛

This expression states that the probability of an outcome

is the ratio of the number of times the outcome arises to
the total number of trials in a long sequence of trials.
 Probability - Basics

• The Figures on the following two slides depict how the

relative frequencies for the three outcomes (𝑓0(𝑛),
𝑓1(𝑛), 𝑓2(𝑛)) approach a value close to 1/3 when the
number of trials n is increased.

• The fact that the relative frequencies are converging to

the value 1/3 is in agreement with our intuition that
the three outcomes are equiprobable.
 Probability - Basics

𝟏/𝟑

Relative Frequencies for the outcomes 0, 1 and 2 for a

maximum of 50 trials.
 Probability - Basics

𝟏/𝟑

Relative Frequencies for the outcomes 0, 1 and 2 for a

maximum of 500 trials.
 Probability - Basics
• Suppose that in our urn experiment we have 10
trials, i.e., 𝑛 = 10.

• In other words, we select a ball from the urn and

after noting the number on the ball we place the ball
back in the urn – and we do this 10 times.

• Then the number of times that the outcome is 1

cannot be less than zero or greater than 10,

0 ≤ 𝑁1 𝑛 ≤ 10
 Probability - Basics
• Similarly, the number of times that the outcome is 0
cannot be less than zero or greater than 10,
0 ≤ 𝑁0 𝑛 ≤ 10

• The number of times that the outcome is 2 cannot be

less than zero or greater than 10,
0 ≤ 𝑁2 𝑛 ≤ 10

• In general,
0 ≤ 𝑁𝑘 𝑛 ≤ 𝑛
 Probability - Basics
0 ≤ 𝑁𝑘 𝑛 ≤ 𝑛

• If we divide this equation by 𝑛,

𝑁𝑘 𝑛
0≤ 𝑛
≤1
• We get

0 ≤ 𝑓𝑘 𝑛 ≤ 1
 Probability - Basics

• The probability of an event cannot be less than 0

or greater than 1.

• Probability of an outcome/event/result A can be

expressed as,

Number of Possible ways that A can occur

PA =
Total number of possible outcomes
 Basics - Example
• An urn contains 10 identical balls numbered
0,1, … . , 9. A random experiment involves selecting a
ball from the urn and noting the number of the ball.
Find the probability that the number of the ball
selected is odd.
 Basics - Example

P odd Ball is Selected

=
Number of Possible ways to get an odd ball
Total number of possible outcomes
 Basics - Example
Number of Possible ways that A can occur
PA =
Total number of possible outcomes

• Let event A refer to the selected ball number being odd

(i.e. 1, 3, 5, 7, 9). Then there are 5 possible ways that
event A can occur.

• Actually, the selected ball can be any number between 0

and 9, so there are 10 possible outcomes.
P[ball is odd] = 5 / 10 = 0.5
 Basics - Example
• An urn contains 10 identical balls numbered 0,1, … . , 9.
A random experiment involves selecting a ball from the
urn and noting the number of the ball. Find the
probability that the number of the balls selected is a
multiple of 3.
 Basics - Example
Number of Possible ways that A can occur
PA =
Total number of possible outcomes

• Let event A refer to the selected ball number being a

multiple of 3 (i.e. 𝟑, 𝟔, 𝟗). Then there are 3 possible
ways that event A can occur.

• But the selected ball can be any number between 0 and

9, so there are 10 possible outcomes in total.
P[ball number is multiple of 3] = 3 / 10 = 0.3
 Basics - Example
• An urn contains 10 identical balls numbered 0,1, … . , 9.
A random experiment involves selecting a ball from the
urn and noting the number of the ball. Find the
probability that the number of the ball selected is less
than 5.
 Basics - Example
Number of Possible ways that A can occur
PA =
Total number of possible outcomes

• Let event A refer to the selected ball number being less

than 5 (i.e. 0, 1, 2, 3, 4). Then there are 5 possible ways
that event A can occur.

• But the selected ball can be any number between 0 and

9, so there are 10 possible outcomes in total.
P[ball number is less than 5] = 5 / 10 = 0.5
 Home Assignment 01 (Next Week)
Question 1:
Consider the following three random experiments: Experiment 1: Toss a coin.
Experiment 2: Toss a die. Experiment 3: Select a ball at random from an urn
containing balls numbered 0 to 9.
a) Specify the sample space of each experiment.
b) Find the relative frequency of each outcome in each of the above experiments
in a large number of repetitions of the experiment. Explain your answer.
Question 2:
An urn contains three electronically labeled balls with labels 00, 01, 10. Lisa,
Homer, and Bart are asked to characterize the random experiment that involves
selecting a ball at random and reading the label. Lisa’s label reader works fine;
Homer’s label reader has the most significant digit stuck at 1; Bart’s label
reader’s least significant digit is stuck at 0.
a) What is the sample space determined by Lisa, Homer, and Bart?
b) What are the relative frequencies observed by Lisa, Homer, and Bart in a
large number of repetitions of the experiment?