EE3331

Chapter2: Random Variables

The slides are based on the notes of Prof. So.

Page 1 Semester B 2023-2024

Random Variables

Chapter Intended Learning Outcomes:

(i) Understand discrete and continuous random variables

(ii) Know common distributions of random variables

(iii) Able to generate common random variables

(iv) Able to compute expectation of random variables

(v) Able to apply random variables and common probability

distributions to solve problems

Page 2 Semester B 2023-2024

Random Variables
A random variable (RV) is a real-valued number assigned
to every outcome of an experiment.

http://www.est.uc3m.es/icascos/eng/probability_notes/discrete-random-variables.html

RV is a function from the sample space to a number, allowing

numerical designation for events which are set functions.
Page 3 Semester B 2023-2024
Recall dice rolling, event with odd number is a set {1, 3, 5}.
In this case, we may assign RV if the outcome is an
odd number, i.e., .
Traditionally, RV is denoted by an upper-case letter (e.g., )
and any specific value of RV is a lower-case letter (e.g., )
While we may assign RV for even number. Hence is
random, with value 0 or 1.
For an unbiased dice, we have:

and
Page 4 Semester B 2023-2024
Note that we can assign/define RVs in many different ways
for a particular sample space, e.g., we may also set
and for odd and even number events, respectively.
But we can define the RVs based on a specific problem so
that it will be solved in an easier manner.
RV can be classified as:
▪ Discrete: its range is countable, with finite or a countably
infinite number of values on the real line.
▪ Continuous: It has a range with uncountably infinite
number of values on the real line.
Example 2.1
Consider the experiment of flipping a coin until a tail occurs.
Define a RV for this experiment and discuss its property.

Page 5 Semester B 2023-2024

We can define a RV which is the number of trials until the
experiment ends (i.e., tail occurs). Hence the RV is discrete
with only positive integer values:

Example 2.2
Can we define a discrete RV on a continuous sample space?
And can we define a continuous RV on a discrete sample
space? For each case, use an example for illustration.

Yes, for both. First, suppose the sample space is any point in
the interval (0,1]. We can define a RV that maps (0,0.5] to 0
and (0.5,1] to 1.
Consider the outcome of flipping a coin, which is either head
(H) or tail (T). We can define a RV which is selected in (0,0.5]
to represent that H occurs, while (0.5,1] for T.
Page 6 Semester B 2023-2024
Distribution Functions for Random Variables

For discrete RV , we use probability mass function (PMF)

to assign probability to each RV value.

Suppose must be one of the values , , , … then:

(2.1)

(2.2)

where is the range. Suppose , the probability of

is thus equal to:

Page 7 Semester B 2023-2024

Apart from PMF, the cumulative distribution function (CDF)
can also be used to describe the RV distribution.

Assuming that , the CDF is defined as:

(2.3)
Consider rolling a dice again, we may directly assign the
outcomes “1” to “6” as to . Hence we have:

If refers to the event with odd number, then

and

Page 8 Semester B 2023-2024

Also,

is the probability that the outcome is less than or equal to 3.

Apparently, CDF is a non-decreasing function with minimum

and maximum values 0 and 1, respectively.

If PMF is known, then CDF can be determined and vice versa.

Example 2.3
Consider the experiment of rolling 2 fair dice and the outcome
is the sum of the faces. Assign RV for this experiment and
then plot the PMF and CDF. Compute the probability that the
sum is between 6 and 8.
Page 9 Semester B 2023-2024
To facilitate the probability computation, we assign as the
sum value, i.e., .
Applying the probability theory, e.g., the sum is 2 only when
both dice give “1”, hence , we obtain:

Page 10 Semester B 2023-2024

From the computed probabilities, we construct PMF:

The required probability is:

Or

Page 11 Semester B 2023-2024

The CDF is:

Page 12 Semester B 2023-2024

Example 2.4
Consider the experiment of flipping a coin until a head occurs.
Assume the probability of coming up a head is , and
successive flips are independent. Design a suitable RV and
find the probability that a head occurs at the th flip.
Following Example 2.1, we define as the RV which is the
number of trials until the head (H) appears. Also, it is clear
that the probability of getting tail (T) is .
We have:

Hence the required probability is:

What is P(a head occurs within infinite trials)?

Page 13 Semester B 2023-2024
Binomial Distribution

Consider an experiment which results in two possible

outcomes: success with probability , and failure with
probability .
It is called a Bernoulli trial. If we assign RV for success
and for failure, then:

(2.4)

where is called Bernoulli RV.

Binomial RV extends the Bernoulli RV by combining
independent Bernoulli trials, where we are interested in the
number of successes. Assigning RV to represent the
success number, we have .
Page 14 Semester B 2023-2024
The PMF and CDF of the binomial RV are:

(2.5)

where accounts for the number of combinations. The

binomial distribution is characterized by and .

Example 2.5
4 fair coins are flipped and assuming that the outcomes are
independent. Compute the probability of obtaining 2 heads.
How about the probability of obtaining at most 2 heads?
Let RV be the number of heads. With and , the
Page 15 Semester B 2023-2024
required probability of the former is:

For the second, we use:

Page 16 Semester B 2023-2024

Geometric Distribution

Considering independent trials, each having probability of

being a success (and failure with probability ), are
performed until a success occurs.

Define RV to be the number of trial(s) until the experiment

ends, which is called geometric RV.

From Example 2.1, we know . As in Example 4, the

PMF is:

(2.6)

Why combination as in (2.1) is not considered?

What is/are the parameter(s) of geometric RV?
Page 17 Semester B 2023-2024
Example 2.6
Determine CDF of geometric RV with success probability .

Page 18 Semester B 2023-2024

Poisson Distribution
A RV is said to be Poisson RV with parameter if
its PMF has the form of:
(2.7)
The Poisson function is the limiting case of the binomial
function for large and small , with .

Hence can be used to represent the number of successes in

trials if is large enough.

Using (2.5) and considering :

Page 19 Semester B 2023-2024

because

The Poisson RV can be interpreted as the number of events

in a given time interval.

Page 20 Semester B 2023-2024

Example 2.7
The number of hits at a website in any time interval is a
Poisson RV with an average rate of 2 hits/s. What is the
probability that there are no hits in an interval of 0.25s? What
is the probability that there are no more than 2 hits in an
interval of 1s?

For no hits, it means in (2.9). Together with

, the probability is:

When , the probability with no more than 2

hits in 1s is:

Page 21 Semester B 2023-2024

Example 2.9
The number of database queries processed by a computer in
any 10s is a Poisson RV with an average of 5 queries. What
is the probability that there will be no queries processed in a
10-second interval? What is the probability that at least 2
queries will be processed in a 2-second interval?

For 10-second interval, . Hence we have:

Noting that , and for a 2-second interval,

, then the required probability is:

Page 22 Semester B 2023-2024

For continuous RV , where is any real number in ,
we use probability distribution function (PDF)
and CDF .

The PDF and CDF are related as:

(2.10)

(2.11)

indicating that is differentiable.

As in the discrete RV case, if PDF is known, then CDF can be

determined and vice versa.

Page 23 Semester B 2023-2024

From (2.10)-(2.11):

Page 24 Semester B 2023-2024

Uniform Distribution
When RV is uniformly distributed over the interval , its
value can be any value in this range with equal probability of
occurrence.
We write , and the PDF is:

(2.12)

where the unit step function is defined as:

Apparently, a uniform RV can be characterized by and .

Page 25 Semester B 2023-2024
It is commonly used in modeling quantization error, say,
rounding a real number to its nearest integer, e.g.,
and or , and we have and :

Why p(x) =1 when x is between (-0.5, 0.5)?

Page 26 Semester B 2023-2024
Example 2.10
Find the CDF of a RV uniformly distributed between . If
is uniformly distributed over (0,10), calculate the probability
that .

Using (2.11), we get:

Setting and , the required probability is

or is:

Page 27 Semester B 2023-2024

Gaussian Distribution
Gaussian (or normal) RV is the most popular continuous RV
which is characterized by two parameters, namely, mean
and standard deviation (or variance ).
We write and its PDF is:
(2.13)

Gaussian Distribution (princeton.edu)

Page 28 Semester B 2023-2024
Note that is symmetric around .

Many real-world data particularly when the size is sufficiently

large, e.g., population’s heights, population’s weights,
students’ examination marks, follow a bell shape aligning
with the Gaussian PDF.

For example, the heights of adult men in US are

approximately normally distributed with a mean of 70 inches
and a standard deviation of 3 inches. Then we may estimate
that 99.7% of US adult men have heights between 61 and 79
inches. And we expect the larger the , the larger the spread.

For , it is called a standard Gaussian RV:

(2.14)

Page 29 Semester B 2023-2024

The standard normal complementary CDF is:

(2.15)

The Standard Normal Distribution | Examples, Explanations, Uses (scribbr.com)

We can see that for :

(2.16)
Page 30 Semester B 2023-2024
A general Gaussian RV is related to :

This implies that we can easily shift and scale a standard

Gaussian RV to produce different Gaussian RVs.

Example 2.11
Consider a Gaussian RV . What is ?
Furthermore, what is ?

According to (2.13)-(2.14), the probability is

Page 31 Semester B 2023-2024

Example 2.12
In an optical fiber transmission system, the probability of a
bit error is where is the signal-to-noise ratio. What
is the minimum value of that produces a bit error rate not
exceeding ?
Using (2.15):

The command norminv can be used to find :

norminv(1-1e-6)
= 4.7534

Hence

Page 32 Semester B 2023-2024

Exponential Distribution

A RV is called exponential RV if its PDF has the form:

(2.17)

where it is characterized by the parameter . The

corresponding CDF is:

(2.18)

Exponential distribution may be interpreted as a continuous

version of the geometric distribution. That is, the former
models the amount of time, while the latter corresponds to
the number of trial(s), until a specific event occurs.
Page 33 Semester B 2023-2024
Example 2.13
Suppose the probability that a telephone call lasts no more
than min. is modelled as an exponential RV:

Find the probability that a call lasts between 2 and 4 min.

Applying (2.18) with , the probability is:

Page 34 Semester B 2023-2024

Expectation of Random Variable
The expectation of a RV refers to the average values of the
RV, which includes mean (1st moment) and power or mean
square value (2nd moment), denoted by and ,
respectively, where is the expectation operator.
For , the definitions of discrete and continuous cases are:

(2.19)

(2.20)

These can be generalized to nth moment, e.g., for (2.20):

(2.21)
Page 35 Semester B 2023-2024
The moment can also be computed about a point , i.e.,

(2.22)

In particular, when , (2.22) is the central moment.

The expected values are important because the properties of

RV can be obtained, e.g., we may make the best decision
under uncertainty (most probable value and risk), and
characterize the RV such as normal RV which depends only
on mean and 2nd central moment or variance .

Example 2.14
Determine the mean of the Bernoulli RV with success
probability . When , what is the expected number of
experiment(s) to obtain one success?
Page 36 Semester B 2023-2024
Using (2.19):

Thus the expected number of successes is just the

probability that the trial will be a success.

When , the expected number of experiments to obtain

one success is .
Extending the computation to binomial RV with parameters
and , we can obtain .
When and , we then have ,
corresponding to the consistent result.

Page 37 Semester B 2023-2024

Example 2.15
Suppose the PMF of RV is: , , and
. Calculate and .

Note that .

Example 2.16
Calculate for Poisson RV with the use of .

Based on (2.7):

Page 38 Semester B 2023-2024

Example 2.17
Calculate and for uniform RV with PDF:

The average value of aligns with our intuitive

understanding. For a standard uniform RV , the
mean and power are 0.5 and 1/3.

Page 39 Semester B 2023-2024

Example 2.18
Calculate and variance for Gaussian RV.

Using (2.13) and (2.20), we have:

Note that by the symmetric property, the first integral is 0.

Page 40 Semester B 2023-2024
The variance is:

where the Gaussian integral formula is used.

Page 41 Semester B 2023-2024

Example 2.19
Consider a continuous RV with PDF . Express in
terms of and .

Let for notational simplification. We have:

Note that the identity holds for both continuous and discrete
cases:
(2.23)
Page 42 Semester B 2023-2024
The expectation operator can be generalized to a function of
RV , i.e., and it is clear that is a RV:
(2.24)

(2.25)

For example, when and , (2.25) will

become (2.20) and (2.22), respectively.
Example 2.20
Consider a uniform RV . Calculate .
Applying (2.25), we get

Page 43 Semester B 2023-2024

Alternatively, we can start from the CDF. Using the result in
Example 2.13 with and :

That is, for .

Let . As , we have .

Applying (2.10)

Page 44 Semester B 2023-2024

Hence according to (2.20):

Writing , its PDF can also be obtained as:

Now . Again, with , we

then compute:

Combining the results, we get the same .

Page 45 Semester B 2023-2024
The same result is obtained. Comparing the two approaches,
the first method is much simpler while the second method
can provide the CDF and PDF.

Page 46 Semester B 2023-2024

For discrete RV, the PMF of is:

(2.26)

Example 2.21
Suppose the amplitude of signal is a discrete RV with PMF:

Let be the signal power. Find and determine

.

only has 4 possible values, 4.5 if , 2 if ,

0.5 if and 0 if .
Page 47 Semester B 2023-2024
Using (2.26) together with the given , we have:

Note that implies , i.e., ,

, . Applying (2.19),
is:

Alternatively, we can use (2.24):

Page 48 Semester B 2023-2024

Example 2.22
Given where is a continuous RV while and
are constants. Determine .

According to (2.20), we have:

That is,
(2.27)

Note that (2.27) also holds for discrete RV.

Page 49 Semester B 2023-2024
Moment Generating Function

The moment generating function of a continuous RV is

defined as the expected value of , denoted by :

(2.28)

In general:

Hence we can use for moment computation:

(2.29)
Page 50 Semester B 2023-2024
Note that the characteristic function , which is
similar to , has a relationship with , which is analogous
to Fourier transform:

(2.30)

For discrete RV , the moment generating function is:

(2.31)

where (2.29) is still valid.

Example 2.23
Repeat Example 2.23 to calculate and variance for a
Gaussian RV using (2.28).
Page 51 Semester B 2023-2024
We first consider . Based on (2.28):

Since , then . Hence:

Applying chain rule in differentiation yields:

From (2.23):

Page 52 Semester B 2023-2024

Example 2.24
Calculate and variance for binomial RV which is
characterized by and using (2.31).

Using (2.5) and (2.31), we have:

From (2.23):

Page 53 Semester B 2023-2024

References:
1. S. M. Ross, Introduction to Probability Models, Academic Press,
2014
2. D. G. Childers, Probability and Random Processes: Using
MATLAB with Applications to Continuous and Discrete Time
Systems, Irwin, 1997
3. D. Morin, Probability: For the Enthusiastic Beginner,
CreateSpace Independent Publishing Platform, 2016
4. R. D. Yates and D. J. Goodman, Probability and Stochastic
Processes: A Friendly Introduction for Electrical and Computer
Engineer, Wiley, 2014
5. S. Kay, Intuitive Probability and Random Processes using
MATLAB, Springer, 2006
6. H. Pishro-Nik, Introduction to Probability, Statistics, and
Random Processes, Kappa Research LLC, 2014
https://www.probabilitycourse.com

Page 54 Semester B 2023-2024