Random Variable

Random Variable Definition

 It frequently occurs that in performing an experiment we are mainly interested
in some functions of the outcome as opposed to the outcome itself
Rotary, casino, game, etc.

 These quantities of interest, or more formally, these real-valued functions

defined on the sample space, are known as random variables (RVs).

 More mathematical expression of RVs

Given a probability space (S, F, P), a random variable is a measurable function (mapping)
from S to the real line X: S  R
 X({H}) = 100, X({H}) = -50
 X({1,2}) = 100, X({3,4,5,6}) = -50

 Since the value of a random variable is defined by the outcome of the

experiment, we may assign probabilities to the possible values of the random
variable
Sample
space ω
1

ω2
X(ω1) X(ω2) 0 1
Random Variable: Examples
Ex 2.1] Letting X denote the random variable that is defined
as the sum of two fair dice
The outcomes of two dice = (ω1, ω2)
RV X(ω1, ω2) = ω1+ω2
Possible values of X = {2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}
Probabilities to the possible values of the random variable
 P(X=2)=P{(1,1)}=1/36
 P(X=3)=P{(1,2), (2,1)}=2/36
6
…
 P(X=11)=P{(5,6), (6,5)}=2/36 5
 P(X=12)=P{(6,6)}=1/36
4
 1 = 𝑃𝑃 ⋃12
𝑛𝑛=2{𝑋𝑋 = 𝑛𝑛}
= ∑12𝑛𝑛=2 𝑃𝑃 𝑋𝑋 = 𝑛𝑛 3
2

1
1 2 3 4 5 6
Ex.2.3] Suppose that we toss a coin having a probability p of
coming up heads, until the first head appears. Letting N
denote the number of flips required, then assuming that the
outcome of successive flips are independent, N is a random
variable taking on one of the values 1, 2, 3, . . . , with
respective probabilities
P{N = 1} = P{H} = p,
P{N = 2} = P{(T, H)} = (1 − p)p,
P{N = 3} = P{(T, T, H)} = (1 − p)2p,
...
P{N = n} = P{(T, T, … , T, H)} = (1 − p)n−1p, n ≥ 1

Indicator random variable

IE(x) = 1 if x∈E, 0 otherwise.
Ex. 2.5] Suppose that independent trials, each of which
results in any of m possible outcomes with respective
𝑚𝑚
probabilities p1, . . . , pm, ∑𝑖𝑖=1 𝑝𝑝𝑖𝑖 = 1, are continually
performed. Let X denote the number of trials needed until
each outcome has occurred at least once.
Instead of 𝑃𝑃 𝑋𝑋 = 𝑛𝑛 , we will first determine 𝑃𝑃 𝑋𝑋 > 𝑛𝑛 the probability
that at least one of the outcomes has not yet occurred after 𝑛𝑛 trials.
Letting 𝐴𝐴𝑖𝑖 denote the event that outcome 𝑖𝑖 has not yet occurred
after 𝑛𝑛 first trials, 𝑛𝑛 = 1, … , 𝑚𝑚, then
𝑃𝑃 𝑋𝑋 > 𝑛𝑛 = 𝑃𝑃 ⋃𝑚𝑚 𝑖𝑖=1 𝐴𝐴𝑖𝑖
𝑚𝑚
= ∑𝑖𝑖=1 𝑃𝑃 𝐴𝐴𝑖𝑖 − ∑ ∑𝑖𝑖<𝑗𝑗 𝑃𝑃 𝐴𝐴𝑖𝑖 𝐴𝐴𝑗𝑗
+ ∑ ∑ ∑𝑖𝑖<𝑗𝑗<𝑘𝑘 𝑃𝑃 𝐴𝐴𝑖𝑖 𝐴𝐴𝑗𝑗 𝐴𝐴𝑘𝑘 − ⋯ + −1 𝑚𝑚+1 𝑃𝑃(𝐴𝐴1 … 𝐴𝐴𝑚𝑚 )
𝑃𝑃 𝐴𝐴𝑖𝑖 is the probability that each of the first n trials results in a
non-i outcome

𝑃𝑃 𝐴𝐴𝑖𝑖 𝐴𝐴𝑗𝑗 is the probability that the first n trials all result in a non-i
and non-j outcome

𝑃𝑃 𝑋𝑋 = 𝑛𝑛 = 𝑃𝑃 𝑋𝑋 > 𝑛𝑛 − 1 − 𝑃𝑃 𝑋𝑋 > 𝑛𝑛

CDF
If a random variable takes on either a finite or a countable number
of possible values, the RV is called discrete.

If a continuum of possible values, continuous.

The cumulative distribution function (CDF) F( ) of the random

variable X is defined for any real number b, -∞<b<∞ by
F(b)=P{X≤b}.

Properties of CDF F
1. F(b) is a nondecreasing function of b
2. limb∞F(b)=F(∞)=1
3. limb-∞F(b)=F(-∞)=0

Ex. P{a<X≤b}=F(b)-F(a) for all a<b

Types and Properties of RVs
Discrete RVs
 A random variable that can take on at most countable number of possible
values

 Probability Mass Function (pmf): for a discrete random variable X, pmf p(a) of X
is defined a p(a) = P{X=a}.

 Properties of pmf
1. p(xi)>0, i=1,2,…
2. p(x)=0 all other values of x
3. ∑i={1,..,∞}p(xi)=1
4. cdf vs. pmf: F(a)=∑xi≤ap(xi)

 Ex. p(1)=1/2, p(2)=1/3, p(3)=1/6

0 𝑓𝑓𝑓𝑓𝑓𝑓 𝑎𝑎 < 1
1
𝑓𝑓𝑓𝑓𝑓𝑓 1 ≤ 𝑎𝑎 < 2
2
𝐹𝐹 𝑎𝑎 = 5
𝑓𝑓𝑓𝑓𝑓𝑓 2 ≤ 𝑎𝑎 < 3
6
1 𝑓𝑓𝑓𝑓𝑓𝑓 3 ≤ 𝑎𝑎

 Discrete random variables are often classified according to their pmf

Ex: Bernoulli RV, Binomial RV, Geometric RV, Poisson RV
Example of Discrete RVs: The
Bernoulli RV
A random variable X is said to be a Bernoulli RV for given
sample space S={A,Ac} and some p ∈ (0,1) if its pmf is given
by
P{A}=p, P{Ac}=1-p

For example, suppose that a trial, or an experiment, whose

outcome can be classified as either a “success” or as a
“failure” is performed. If we let X equal 1 if the outcome is a
success and 0 if it is a failure, then the probability mass
function of X is given by
p(0) = P{X = 0} = 1 − p,
p(1) = P{X = 1} = p (2.2)
where p, 0 ≤ p ≤ 1, is the probability that the trial is a
“success.”
Example of Discrete RVs:
The Binomial RV
 Suppose that n independent trials, each of which results in a “success” with
probability p and in a “failure” with probability 1−p, are to be performed.

 If X represents the number of successes that occur in the n trials, then X is said
to be a binomial random variable with parameters (n, p).

 The pmf𝑛𝑛 of a binomial RV with (n,p)

𝑝𝑝 𝑖𝑖 = 𝑝𝑝𝑖𝑖 (1 − 𝑝𝑝)𝑛𝑛−𝑖𝑖 for i = 0, 1, . . . , n (2.3)
𝑛𝑛𝑖𝑖 𝑛𝑛!
where = .
𝑖𝑖 𝑛𝑛−𝑖𝑖 !𝑖𝑖!

𝑛𝑛 𝑖𝑖
 ∑𝑛𝑛𝑖𝑖=0 𝑝𝑝(𝑖𝑖) = ∑𝑛𝑛𝑖𝑖=0 𝑝𝑝 (1 − 𝑝𝑝)𝑛𝑛−𝑖𝑖 = (𝑝𝑝 + 1 − 𝑝𝑝 )𝑛𝑛 = 1
𝑖𝑖

 Ex. 2.7] It is known that any item produced by a certain machine will be
defective with probability 0.1, independently of any other item. What is the
probability that in a sample of three items, at most one will be defective?
If X is the number of defective items in the sample, then X is a binomial random
variable with parameters (3, 0.1). Hence, the desired probability is given by
P{X = 0} + P{X = 1} =(3,0)(0.1)0(0.9)3+(3,1)(0.1)1(0.9)2 = 0.972
Example of Discrete RVs:
The Geometric RV
Suppose that independent trials, each having probability p of
being a success, are performed until a success occurs.

If we let X be the number of trials required until the first

success, then X is said to be a geometric random variable
with parameter p.

Its probability mass function is given by

p(n) = P{X = n} = (1 − p)n−1p, n = 1, 2, . . . (2.4)

To check that p(n) is a probability mass function, we note

that ∞ ∞

� 𝑝𝑝(𝑛𝑛) = 𝑝𝑝 � (1 − 𝑝𝑝)𝑛𝑛−1 = 1
𝑛𝑛=1 𝑛𝑛=1
Example of Discrete RVs:
The Poisson RV
 A random variable X, taking on one of the values 0, 1, 2, . . . , is said to be a Poisson
random variable with parameter𝑖𝑖
λ, if for some λ > 0,
𝜆𝜆
𝑝𝑝 𝑖𝑖 = 𝑃𝑃 𝑋𝑋 = 𝑖𝑖 = 𝑒𝑒 −𝜆𝜆 , i=0,1,2,3,… (2.5)
𝑖𝑖!

 To check that p(n) is a∞probability

∞ mass function,
𝑖𝑖
∞ we note that
−𝜆𝜆
𝜆𝜆 𝜆𝜆𝑖𝑖
� 𝑝𝑝(𝑖𝑖) = � 𝑒𝑒 = 𝑒𝑒 � = 𝑒𝑒 −𝜆𝜆 𝑒𝑒 𝜆𝜆 = 1
−𝜆𝜆
𝑖𝑖! 𝑖𝑖!
𝑖𝑖=0 𝑖𝑖=0 𝑖𝑖=0

 Binomial vs. Poisson RVs

n>>1, p<<1, let λ=np
𝜆𝜆 𝑛𝑛
𝑛𝑛 𝑖𝑖 𝑛𝑛−𝑖𝑖 𝑛𝑛! 𝜆𝜆 𝑖𝑖 𝜆𝜆 𝑛𝑛−𝑖𝑖 𝑛𝑛 𝑛𝑛−1 …(𝑛𝑛−𝑖𝑖+1) 𝜆𝜆𝑖𝑖 1−𝑛𝑛
𝑃𝑃 𝑋𝑋 = 1 = 𝑝𝑝 1 − 𝑝𝑝 = 1− =
𝑖𝑖 𝑛𝑛−𝑖𝑖 !𝑖𝑖! 𝑛𝑛 𝑛𝑛 𝑛𝑛𝑖𝑖 𝑖𝑖!
1−
𝜆𝜆 𝑖𝑖
𝑛𝑛
For large n and small p
𝜆𝜆 𝑛𝑛 𝜆𝜆 𝑖𝑖 𝑛𝑛 𝑛𝑛−1 …(𝑛𝑛−𝑖𝑖+1)
 1− ≈ 𝑒𝑒 −𝜆𝜆 , 1 − ≈ 1, ≈1
𝑛𝑛 𝑛𝑛 𝑛𝑛𝑖𝑖
𝜆𝜆𝑖𝑖
𝑃𝑃 𝑋𝑋 = 𝑖𝑖 ≈ 𝑒𝑒 −𝜆𝜆
𝑖𝑖!

 Ex. 2.10] Suppose that the number of typographical errors on a single page of this
book has a Poisson distribution with parameter λ = 1. Calculate the probability that
there is at least one error on this page.
P{X≥1}=1-P{X=0}=1-e-1=0.633
Continuous RVs
RV X is said to be a continuous RV if there exists a non-negative
function f(x), defined for all real x∈{-∞,∞} having the property that
for any set B of real numbers
P{X∈B}=∫Bf(x)dx=1

The function f(x) is called the probability density function (pdf) of

the random variable X

Properties of pdf of X
∞
𝑃𝑃 𝑋𝑋 ∈ −∞, ∞ = ∫−∞ 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑 = 1
𝑏𝑏
𝑃𝑃 𝑎𝑎 ≤ 𝑋𝑋 ≤ 𝑏𝑏 = ∫𝑎𝑎 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑
𝑎𝑎
𝑃𝑃 𝑋𝑋 = 𝑎𝑎 = ∫𝑎𝑎 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑 = 0
𝑎𝑎
𝐹𝐹 𝑎𝑎 = 𝑃𝑃 𝑋𝑋 ∈ −∞, 𝑎𝑎 = ∫−∞ 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑
𝑑𝑑𝑑𝑑(𝑎𝑎)
 = 𝑓𝑓(𝑎𝑎)
𝑑𝑑𝑑𝑑

Ex. Uniform Rv, Exponential Rv, Gamma RV, Normal RV

Example of Continuous RVs:
Uniform RV
A RV is said to be uniformly distributed over the interval (α,β)
if its probability density function is given by
1
, 𝛼𝛼 < 𝑥𝑥 < 𝛽𝛽
𝑓𝑓 𝑥𝑥 = �𝛽𝛽 − 𝛼𝛼
0, 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜

Ex. 2.14] X~U(0,10)

P{X<3}= ?
P{X>7}= ?
P{1<X<6}= ?
Example of Continuous RVs:
Uniform RV
A continuous RV is said to be uniformly distributed over the
interval (α,β) if its probability density function is given by

1
, 𝛼𝛼 < 𝑥𝑥 < 𝛽𝛽
𝑓𝑓 𝑥𝑥 = �𝛽𝛽 − 𝛼𝛼
0, 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜

Ex. 2.14] X~U(0,10)

P{X<3}= 1/10∫X∈(0,3)dx=3/10
P{X>7}= 1/10∫X∈(7,10)dx=3/10
P{1<X<6}= 1/10∫X∈(1,6)dx=1/2
Example of Continuous RVs:
Exponential RV
A continuous RV is said to be an exponential RV with
parameter λ if its probability density function is given, for
λ>0, by
𝜆𝜆𝑒𝑒 −𝜆𝜆𝑥𝑥 , 𝑖𝑖𝑖𝑖 𝑥𝑥 ≥ 0
𝑓𝑓 𝑥𝑥 = �
0, 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜
𝑎𝑎
𝐹𝐹 𝑎𝑎 = � 𝜆𝜆𝑒𝑒 −𝜆𝜆𝑥𝑥 𝑑𝑑𝑑𝑑 = 1 − 𝑒𝑒 −𝜆𝜆𝑎𝑎
0
Example of Continuous RVs:
Gamma RV
A continuous RV is said to be a gamma RV with parameter α,
λ, if its probability density function is given, for α >0, λ>0, by

𝜆𝜆𝑒𝑒 −𝜆𝜆𝑥𝑥 𝜆𝜆𝑥𝑥 𝛼𝛼−1

𝑓𝑓 𝑥𝑥 = � , 𝑖𝑖𝑖𝑖 𝑥𝑥 ≥ 0
Γ(𝛼𝛼)
0, 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜

where
∞
Γ 𝛼𝛼 = � 𝑒𝑒 −𝑥𝑥 𝑥𝑥 𝛼𝛼−1 𝑑𝑑𝑑𝑑
0
Example of Continuous RVs:
Normal RV
A continuous RV is said to be a normal RV with parameter μ
and σ2, if its probability density function is given by

1 (𝑥𝑥−𝜇𝜇)2
−
𝑓𝑓 𝑥𝑥 = 𝑒𝑒 2𝜎𝜎2
2𝜋𝜋𝜎𝜎
for -∞<x<∞.
Expectation of a function of a RV
The expected value of X , E[X], is a weighted average of the possible
values that X can take on, each value being weighted by the probability
that X assumes that value.

If RV X is a discrete RV having a pmf p(x), then E[X] is defined by

𝐸𝐸 𝑋𝑋 = � 𝑥𝑥𝑥𝑥(𝑥𝑥)
𝑥𝑥:𝑝𝑝 𝑥𝑥 >0
Ex. 2.16~2.19] find the followings
 Expectation of a Bernoulli Random Variable
 Expectation of a Binomial Random Variable
 Expectation of a Geometric Random Variable
 Expectation of a Poisson Random Variable

If RV X is a continuous RV having ∞a pdf f(x), then E[X] is defined by

𝐸𝐸 𝑋𝑋 = � 𝑥𝑥𝑥𝑥(𝑥𝑥) 𝑑𝑑𝑑𝑑
−∞
Ex. 2.20~2.22] find the followings
 Expectation of a Uniform Random Variable
 Expectation of a Exponential Random Variable
 Expectation of a Normal Random Variable
Expectation of a function of a RV
Given a RV X and its probability distribution, what is the
expectation of a function of X?
1. Since g(X) is itself a random variable, it must have a probability
distribution, which should be computable from a knowledge of the
distribution of X. Once we have obtained the distribution of g(X),
we can then compute E[g(X)] by the definition of the expectation.
2. Another way is to compute the expectation of a function of X from
a knowledge of the distribution of X. See Proposition 2.1.

Ex. 2.23] Suppose X has the following probability mass function: p(0)
= 0.2, p(1) = 0.5, p(2) = 0.3. Calculate E[X2].
Letting Y = X2, we have that Y is a random variable that can take on
one of the values 02, 12, 22 with respective probabilities
pY (0) = P{Y = 02} = 0.2,
pY(1) = P{Y = 12} = 0.5,
pY(4) = P{Y = 22} = 0.3
Hence, E[X2] = E[Y] = 0(0.2) + 1(0.5) + 4(0.3) = 1.7
Note that E[X2] ≠ E[X]2
Expectation of a function of a RV
Proposition 2.1
1. If X is a discrete random variable with probability mass function p(x),
then for any real-valued function g,
𝐸𝐸 𝑔𝑔(𝑋𝑋) = � 𝑔𝑔(𝑥𝑥)𝑝𝑝(𝑥𝑥)
𝑥𝑥:𝑝𝑝 𝑥𝑥 >0

2. If X is a continuous random variable with probability density

function f (x), then for any
∞ real-valued function g,
𝐸𝐸 𝑔𝑔(𝑋𝑋) = � 𝑔𝑔(𝑥𝑥)𝑓𝑓(𝑥𝑥) 𝑑𝑑𝑑𝑑
−∞

 Ex 2.26] Let X be uniformly distributed over (0,1), E[X3]?

1 1
 𝐸𝐸 𝑋𝑋 3 = ∫0 𝑋𝑋 3 𝑑𝑑𝑑𝑑 =
4

 Corollary 2.2
If a and b are constants, then E[aX + b] = aE[X] + b
Expectation of a function of a RV
 The expected value of a RVX, E[X], is also referred to as the mean or the first
moment of X.

 The quantity E[Xn], n≥1, is called the nth moment of X

 The variance of a RV X, denoted by Var(X), is defined by

Var(X)=E[(X-E[X])2],
deviation of X from the mean.

 Var(X)=E[(X-E[X])2]
=E[X2-2E[X]X+E[X]2]
=E[X2]-E[2E[X]X]+E[E[X]2]
=E[X2]-2E[X]2+E[X]2
=E[X2]-E[X]2

 Ex 2.27] Var(X) of the normal RV with μ and σ.

𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋 = 𝐸𝐸 𝑋𝑋 − 𝜇𝜇 2
∞
1 𝑥𝑥−𝜇𝜇 2 1 ∞ 𝑥𝑥−𝜇𝜇 2 1 ∞ 𝑦𝑦 2
− 2 − 2𝜎𝜎 2 2 −2
= � 𝑥𝑥 − 𝜇𝜇 2 𝑒𝑒 2𝜎𝜎 2 𝑑𝑑𝑑𝑑 = � 𝑥𝑥 − 𝜇𝜇 𝑒𝑒 𝑑𝑑𝑑𝑑 = � 𝑦𝑦 𝑒𝑒 𝑑𝑑𝑦𝑦
−∞ 2𝜋𝜋𝜎𝜎 2𝜋𝜋𝜎𝜎 −∞ 2𝜋𝜋 −∞
𝜎𝜎 2 ∞ −𝑦𝑦2
= � 𝑒𝑒 2 𝑑𝑑𝑑𝑑
2𝜋𝜋 −∞
= 𝜎𝜎 2
 Jointly Distributed RVs
For any two RVs X and Y, the joint cumulative probability
distribution function of X and Y is defined as
𝐹𝐹 𝑎𝑎, 𝑏𝑏 = 𝑃𝑃 𝑋𝑋 ≤ 𝑎𝑎, 𝑌𝑌 ≤ 𝑏𝑏 , -∞≤a,b≤∞
𝐹𝐹𝑋𝑋 𝑎𝑎, 𝑏𝑏 = 𝑃𝑃 𝑋𝑋 ≤ 𝑎𝑎 = 𝑃𝑃 𝑋𝑋 ≤ 𝑎𝑎, 𝑌𝑌 ≤ ∞ = 𝐹𝐹 𝑎𝑎, ∞
If X and Y are discrete RVs, the joint pmf of X and Y is defined as
p 𝑥𝑥, 𝑦𝑦 = 𝑃𝑃 𝑋𝑋 = 𝑥𝑥, 𝑌𝑌 = 𝑦𝑦

The joint2 pdf of X and Y, f(x,y) is defined 𝑎𝑎

as𝑏𝑏
𝑑𝑑
𝐹𝐹 𝑥𝑥, 𝑦𝑦 = 𝑓𝑓 𝑥𝑥, 𝑦𝑦 , 𝐹𝐹 𝑎𝑎, 𝑏𝑏 = � � 𝑓𝑓 𝑥𝑥, 𝑦𝑦 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
𝑑𝑑𝑥𝑥𝑑𝑑𝑦𝑦 −∞ −∞
𝑃𝑃 𝑥𝑥, 𝑦𝑦 ∈ 𝐷𝐷 = � 𝑓𝑓 𝑥𝑥, 𝑦𝑦 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
𝐷𝐷

A variation of Proposition 2.1

= ∑𝑥𝑥 ∑𝑦𝑦 𝑔𝑔(𝑥𝑥, 𝑦𝑦)𝑝𝑝(𝑥𝑥, 𝑦𝑦)
𝐸𝐸 𝑔𝑔 𝑋𝑋, 𝑌𝑌 (discrete case)
∞ ∞
= ∫−∞ ∫−∞ 𝑔𝑔(𝑥𝑥, 𝑦𝑦)𝑓𝑓(𝑥𝑥, 𝑦𝑦) 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 (continuous case)
E.g) E[aX+bY]=aE[X]+bE[Y]
 Properties of Joint Distribution
1. The 𝐹𝐹(𝑥𝑥, 𝑦𝑦) is such that
𝐹𝐹 −∞, 𝑦𝑦 = 0, 𝐹𝐹 𝑥𝑥, −∞ = 0, 𝐹𝐹 ∞, ∞ = 1

2. The event {𝑥𝑥1 < 𝑋𝑋 ≤ 𝑥𝑥2 , 𝑌𝑌 ≤ 𝑦𝑦} consists of all points {𝑋𝑋, 𝑌𝑌}
in the vertical half-strip 𝐷𝐷2 and the event {𝒙𝒙 ≤ 𝑥𝑥, 𝑦𝑦1 < 𝑌𝑌 ≤
𝑦𝑦2 } consists of all points {𝒙𝒙, 𝒚𝒚} in the vertical half-strip 𝐷𝐷3 .
We maintain that
𝑃𝑃 𝑥𝑥1 < 𝑋𝑋 ≤ 𝑥𝑥2 , 𝑌𝑌 ≤ 𝑦𝑦 = 𝐹𝐹 𝑥𝑥2 , 𝑦𝑦 − 𝐹𝐹 𝑥𝑥1 , 𝑦𝑦
𝑃𝑃 𝑋𝑋 ≤ 𝑥𝑥, 𝑦𝑦1 < 𝑌𝑌 ≤ 𝑦𝑦2 = 𝐹𝐹 𝑥𝑥, 𝑦𝑦2 − 𝐹𝐹 𝑥𝑥, 𝑦𝑦1

3. 𝑃𝑃 𝑥𝑥1 < 𝑋𝑋 ≤ 𝑥𝑥2 , 𝑌𝑌 ≤ 𝑦𝑦 = 𝐹𝐹 𝑥𝑥2 , 𝑦𝑦2 − 𝐹𝐹 𝑥𝑥1 , 𝑦𝑦2

Y −𝐹𝐹 𝑥𝑥2 , 𝑦𝑦1 + 𝐹𝐹 𝑥𝑥1 , 𝑦𝑦1
y2
R0
y1

From Chap.6 in Papoulis’s (4Ed) x1 x2

Jointly Distributed RVs: Example
Example 2.30 As another example of the usefulness of Equation
(2.11), let us use it to obtain the expectation of a binomial random
variable having parameters n and p. Recalling that such a random
variable X represents the number of successes in n trials when each
trial has probability p of being a success, we have X = X1+ X2 + · · ·
+ Xn, where Xi is 1, if the ith trial is a success, otherwise 0.
E[X]=np from a variation of Proposition 2.1

Example 2.31 At a party N men throw their hats into the center of
a room. The hats are mixed up and each man randomly selects one.
Find the expected number of men who select their own hats.
Letting X denote the number of men that select their own hats, we can
best compute E[X] by noting that X = X1 + X2 + · · · + XN where Xi =1, if
the ith man selects his own hat, otherwise 0.
Now, because the ith man is equally likely to select any of theN hats, it
follows that P{Xi = 1} = P{ith man selects his own hat} = 1/N
E[X]=E[X ]+E[X ]+ · · · +E[X ]=N(1/N)=1
1 2 N
Independent RVs:
The random variables X and Y are said to be independent if,
for all a, b,
P{X ≤ a, Y ≤ b} = P{X ≤ a}P{Y ≤ b}
⇔ F(a, b) = FX(a)FY(b) for all a, b
⇔ f(x, y) = fX (x)fY(y) for continuous cases
⇔ p(x, y) = pX (x)pY(y) for discrete cases

Proposition 2.3 If X and Y are independent, then for any

functions h and g
E[g(X)h(Y)] = E[g(X)]E[h(Y)]
∞ ∞
Proof] 𝐸𝐸[𝑔𝑔 𝑋𝑋 ℎ(𝑌𝑌)]=
∞
∫ ∫ 𝑔𝑔(𝑥𝑥)h(y)𝑓𝑓(x,y)dxdy
∞ −∞ −∞
= � � 𝑔𝑔(𝑥𝑥)h(y)𝑓𝑓𝑋𝑋 (x)𝑓𝑓𝑌𝑌 (y)d𝑥𝑥dy
∞ −∞
−∞ ∞
= � h(y) 𝑓𝑓𝑌𝑌 (y)dy � 𝑔𝑔(𝑥𝑥)𝑓𝑓𝑋𝑋 (x)d𝑥𝑥
−∞ −∞
= 𝐸𝐸 ℎ 𝑌𝑌 𝐸𝐸[𝑔𝑔 𝑋𝑋 ]
Covariance and Variance of Sums of
Random Variables
The covariance of any two random variables X and Y,
denoted by Cov(X, Y), is defined by
Cov(X, Y) = E[(X − E[X])(Y − E[Y])]
= E[XY − YE[X] − XE[Y] + E[X]E[Y]]
= E[XY] − E[Y]E[X] − E[X]E[Y] + E[X]E[Y]
= E[XY] − E[X]E[Y]

Note that if X and Y are independent, then by Proposition

2.3 it follows that Cov(X, Y) = 0.

Example 2.33 The joint density function of X, Y is,

1 −(𝑦𝑦+𝑥𝑥/𝑦𝑦)
𝑓𝑓 𝑥𝑥, 𝑦𝑦 = 𝑒𝑒 , 0 < 𝑥𝑥, 𝑦𝑦 < ∞
𝑦𝑦
Verify that the preceding is a joint density function.
Find Cov(X, Y).
By yourself
Properties of Covariance
For any random variables X, Y,Z and constant c,
1. Cov(X,X) = Var(X),
2. Cov(X, Y) = Cov(Y,X),
3. Cov(cX, Y) = c Cov(X, Y),
4. Cov(X, Y + Z) = Cov(X, Y) + Cov(X,Z).
Proof] Whereas the first three properties are immediate, the final one
is easily proven as follows:
Cov(X, Y + Z) = E[X(Y + Z)] − E[X]E[Y + Z]
= E[XY] − E[X]E[Y] + E[XZ] − E[X]E[Z]
= Cov(X, Y) + Cov(X,Z)

 A useful expression for the variance of the sum of random

variables can be obtained as follows:
𝑉𝑉𝑉𝑉𝑉𝑉 ∑𝑛𝑛𝑖𝑖=1 𝑋𝑋𝑖𝑖 = 𝐶𝐶𝐶𝐶𝐶𝐶 ∑𝑛𝑛𝑖𝑖=1 𝑋𝑋𝑖𝑖 , ∑𝑛𝑛𝑗𝑗=1 𝑋𝑋𝑗𝑗 (2.16)

If Xi, i = 1, . . . , n are independent random variables, then

Equation (2.16) reduces to
𝑉𝑉𝑉𝑉𝑉𝑉 ∑𝑛𝑛𝑖𝑖=1 𝑋𝑋𝑖𝑖 = ∑𝑛𝑛𝑖𝑖=1 𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋𝑖𝑖
Definition 2.1 If 𝑋𝑋1 , … , 𝑋𝑋𝑛𝑛 are independent and identically
distributed, then the random variable 𝑋𝑋� = ∑𝑖𝑖=1𝑛𝑛
𝑋𝑋𝑖𝑖 is called the
sample mean.

Proposition 2.4 Suppose that 𝑋𝑋1 , … , 𝑋𝑋𝑛𝑛 are independent and

identically distributed with expected value 𝜇𝜇 and variance 𝜎𝜎 2 .
Then,
a. 𝐸𝐸 𝑋𝑋� = 𝜇𝜇
b. Var 𝑋𝑋� = 𝜎𝜎 2 /𝑛𝑛
c. Cov 𝑋𝑋,� 𝑋𝑋𝑖𝑖 − 𝑋𝑋� = 0, i=1, …,n
 Proofs…

 Example 2.34 (Variance of a Binomial Random Variable)

Compute the variance of a binomial random variable 𝑋𝑋
with parameters 𝑛𝑛 and 𝑝𝑝.
 One Function of Two RVs
Given two random variables 𝑋𝑋 and 𝑌𝑌 and a function 𝑔𝑔(𝑋𝑋, 𝑌𝑌), we
form a new random variable 𝑍𝑍 as 𝑍𝑍 = 𝑔𝑔(𝑋𝑋, 𝑌𝑌).

Given the joint p.d.f 𝑓𝑓𝑋𝑋𝑋𝑋 (𝑋𝑋, 𝑌𝑌), how does one obtain 𝑓𝑓𝑍𝑍 (𝑍𝑍) the
p.d.f of 𝑍𝑍?
𝑓𝑓𝑍𝑍 𝑍𝑍 = 𝑃𝑃 𝑍𝑍 𝜁𝜁 ≤ 𝑧𝑧 = 𝑃𝑃 𝑔𝑔 𝑋𝑋, 𝑌𝑌 ≤ 𝑧𝑧 = 𝑃𝑃 𝑋𝑋, 𝑌𝑌 ∈ 𝐷𝐷𝑧𝑧 = ∬𝐷𝐷 𝑓𝑓 𝑥𝑥, 𝑦𝑦 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑
𝑍𝑍

Problems of this type are of interest from a practical standpoint.

For example, a receiver output signal usually consists of the
desired signal buried in noise, and the above formulation in that
case reduces to 𝑍𝑍 = 𝑋𝑋 + 𝑌𝑌.

From Chap.6 in Papoulis’s (4Ed)

 Example of One Function of Two
RVs

From Chap.6 in Papoulis’s (4Ed)

Distribution of X+Y
Suppose first that X and Y are continuous, X having
probability density f and Y having probability density g. Then,
letting FX+Y(a) be the cumulative distribution function of 𝑋𝑋 +
𝑌𝑌, we have
𝐹𝐹𝑋𝑋+𝑌𝑌 𝑎𝑎 = 𝑃𝑃{𝑋𝑋 + 𝑌𝑌 ≤ 𝑎𝑎}= ∫ ∫𝑥𝑥+𝑦𝑦≤𝑎𝑎 𝑓𝑓(𝑥𝑥)g(y)d𝑥𝑥dy
∞ 𝑎𝑎−𝑦𝑦 ∞ 𝑎𝑎−𝑦𝑦
= ∫−∞ ∫−∞ 𝑓𝑓(𝑥𝑥)g(y)d𝑥𝑥dy = ∫−∞ ∫−∞ 𝑓𝑓(𝑥𝑥)d𝑥𝑥 g(y)dy
∞
= ∫−∞ 𝐹𝐹𝑋𝑋 (𝑎𝑎 − 𝑦𝑦)g(y)dy
 The cumulative distribution function 𝐹𝐹𝑋𝑋+𝑌𝑌 is called the convolution of the
distributions 𝐹𝐹𝑋𝑋 and 𝐹𝐹𝑌𝑌 .

𝑑𝑑 ∞
𝑓𝑓𝑋𝑋+𝑌𝑌 (𝑎𝑎)= ∫ 𝐹𝐹 𝑎𝑎 − 𝑦𝑦 g(y)dy
𝑑𝑑𝑑𝑑 −∞ 𝑋𝑋
∞ 𝑑𝑑 ∞
= ∫−∞ 𝐹𝐹𝑋𝑋 𝑎𝑎 − 𝑦𝑦 g(y)dy = ∫−∞ 𝑓𝑓 𝑎𝑎 − 𝑦𝑦 g(y)dy
𝑑𝑑𝑑𝑑
 Example 1
Example 2.36 (Sum of Two Independent Uniform Random
Variables) If 𝑋𝑋 and 𝑌𝑌 are independent random variables both
uniformly distributed on (0, 1), then calculate the probability
density of 𝑋𝑋 + 𝑌𝑌.
Answer?
 fY (x ) f X ( z − x) f X ( z − x ) fY ( x )

x x x
1 z −1 z z

(a ) 0 ≤ z < 1

fY (x ) f X ( z − x) f X ( z − x ) fY ( x )

x x x
1 z −1 z
z −1 1
( b) 1 ≤ z < 2

f Z (z )

z
0 1 2

39
From Chap.6 in Papoulis’s (4Ed)
Joint Probability Distribution of
Functions of Random Variables
Let X1 and X2 be jointly continuous random variables with
joint probability density function 𝑓𝑓(𝑥𝑥1 , 𝑥𝑥2 ).

Suppose 𝑌𝑌1 = 𝑔𝑔1 (𝑥𝑥1 , 𝑥𝑥2 ), 𝑌𝑌2 = 𝑔𝑔2 (𝑥𝑥1 , 𝑥𝑥2 ).

𝐹𝐹𝑌𝑌1,𝑌𝑌2 𝑦𝑦1 , 𝑦𝑦2 = ∬𝐷𝐷 𝑓𝑓𝑌𝑌1,𝑌𝑌2 𝑦𝑦1 , 𝑦𝑦2 𝑑𝑑𝑦𝑦1 𝑑𝑑𝑦𝑦2
𝑦𝑦1 ,𝑦𝑦2
= 𝑃𝑃 𝑌𝑌1 ≤ 𝑦𝑦1 , 𝑌𝑌2 ≤ 𝑦𝑦2 = 𝑃𝑃 𝑔𝑔1 𝑥𝑥1 , 𝑥𝑥2 ≤ 𝑦𝑦1 , 𝑔𝑔1 𝑥𝑥1 , 𝑥𝑥2 ≤ 𝑦𝑦2
= 𝑃𝑃 𝑥𝑥1 , 𝑥𝑥2 ∈ 𝐷𝐷𝑌𝑌1,𝑌𝑌2 = ∬𝐷𝐷 𝑓𝑓𝑋𝑋1,𝑋𝑋2 𝑥𝑥1 , 𝑥𝑥2 𝑑𝑑𝑥𝑥1 𝑑𝑑𝑥𝑥2
𝑌𝑌1 ,𝑌𝑌2
−1 𝑑𝑑𝑥𝑥 𝑑𝑑𝑥𝑥
= ∬𝐷𝐷 𝑓𝑓𝑋𝑋1,𝑋𝑋2 𝑥𝑥1 , 𝑥𝑥2 𝐽𝐽(𝑥𝑥1 , 𝑥𝑥2 ) 1 2
𝑋𝑋1 ,𝑋𝑋2
−1
𝑓𝑓𝑌𝑌1,𝑌𝑌2 𝑦𝑦1 , 𝑦𝑦2 = 𝑓𝑓𝑋𝑋1,𝑋𝑋2 𝑥𝑥1 , 𝑥𝑥2 𝐽𝐽 𝑥𝑥1 , 𝑥𝑥2
𝜕𝜕𝑔𝑔1 𝜕𝜕𝑔𝑔1
𝜕𝜕𝑥𝑥1 𝜕𝜕𝑥𝑥2 −1
where 𝐽𝐽(𝑥𝑥1 , 𝑥𝑥2 ) = 𝜕𝜕𝑔𝑔2 𝜕𝜕𝑔𝑔2
= 𝐽𝐽 𝑦𝑦1 , 𝑦𝑦2
𝜕𝜕𝑥𝑥1 𝜕𝜕𝑥𝑥2
Meaning of 𝐽𝐽(𝑥𝑥1 , 𝑥𝑥2 )

w y
D′
C D C′
∆w
∆i
w yi
θ ϕ B′
A B A′
∆z
z z xi x
(a) (b)
Moment Generating Functions
The moment generating function φ(t) of the random variable X is
defined for all values t by
∑𝑥𝑥 𝑒𝑒 𝑡𝑡𝑥𝑥 𝑝𝑝(𝑥𝑥)
ϕ 𝑡𝑡 = 𝐸𝐸 𝑒𝑒 𝑡𝑡𝑡𝑡 = � ∞ 𝑡𝑡𝑡𝑡
∫−∞ 𝑒𝑒 𝑓𝑓 𝑥𝑥 𝑑𝑑𝑑𝑑

Properties
𝑑𝑑 𝑑𝑑 𝑡𝑡𝑡𝑡
ϕ′ (𝑡𝑡) = 𝐸𝐸 𝑒𝑒 𝑡𝑡𝑡𝑡 = 𝐸𝐸 𝑒𝑒 = 𝐸𝐸 𝑋𝑋𝑒𝑒 𝑡𝑡𝑡𝑡
𝑑𝑑𝑡𝑡 𝑑𝑑𝑑𝑑
𝑑𝑑 𝑑𝑑
ϕ 𝑡𝑡 = ϕ′ (𝑡𝑡) = 𝐸𝐸
′′
(𝑋𝑋𝑒𝑒 𝑡𝑡𝑡𝑡 ) = 𝐸𝐸 𝑋𝑋 2 𝑒𝑒 𝑡𝑡𝑡𝑡
𝑑𝑑𝑑𝑑 𝑑𝑑𝑑𝑑
ϕ(𝑛𝑛) 𝑡𝑡 = 𝐸𝐸 𝑋𝑋 𝑛𝑛 𝑒𝑒 𝑡𝑡𝑡𝑡
 ϕ′ 0 = 𝐸𝐸 𝑋𝑋 , ϕ′′ 0 = 𝐸𝐸 𝑋𝑋 2 , ϕ(𝑛𝑛) 0 = 𝐸𝐸 𝑋𝑋 𝑛𝑛
An important property of moment generating functions is that the
moment generating function of the sum of independent random
variables is just the product of the individual moment generating
functions. To see this, suppose that X and Y are independent and
have moment generating functions φX(t) and φY(t), respectively.
Then φX+Y(t), the moment generating function of X + Y, is given by
φX+Y(t) = E[et(X+Y)]= E[etXetY]= E[etX]E[etY]= φX(t)φY(t)
 Limit Theorems: Strong Law of Large
Numbers
Let X1, X2, . . . be a
Theorem 2.1 (Strong Law of Large Numbers)
sequence of independent random variables having
a common distribution, and let E[Xi] = μ. Then, with
𝑋𝑋1 +𝑋𝑋2 +⋯+𝑋𝑋𝑛𝑛
probability 1, → 𝜇𝜇
𝑛𝑛

Example: suppose that a sequence of independent

trials is performed. Let 𝐸𝐸 be a fixed event and
denote by 𝑃𝑃(𝐸𝐸) the probability that 𝐸𝐸 occurs on
any particular trial. Letting
1, 𝑖𝑖𝑖𝑖 𝐸𝐸 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜 𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡 𝑖𝑖𝑖𝑖𝑖 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡
𝑋𝑋𝑖𝑖 = �
0, 𝑖𝑖𝑖𝑖 𝐸𝐸 𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜 𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡 𝑖𝑖𝑖𝑖𝑖 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡
we have by the strong law of large numbers that,
with probability 1,
𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛
→ 𝐸𝐸 𝑋𝑋 = 𝑃𝑃(𝐸𝐸)
𝑛𝑛
Limit Theorems: Central Limit
Theorem
Theorem 2.2 (Central Limit Theorem)
Let X1,X2, . . . be a sequence of
independent, identically distributed random variables,
each with mean μ and variance σ2. Then the distribution of
𝑋𝑋1 +𝑋𝑋2 +⋯+𝑋𝑋𝑛𝑛 −𝑛𝑛𝜇𝜇
tends to the standard normal as n→∞. That is
𝜎𝜎 𝑛𝑛 2
𝑥𝑥
𝑋𝑋1 +𝑋𝑋2 +⋯+𝑋𝑋𝑛𝑛 −𝑛𝑛𝜇𝜇 1 𝑎𝑎 −
P ≤ 𝑎𝑎 = ∫ 𝑒𝑒 2 𝑑𝑑𝑑𝑑 s n→∞.
𝜎𝜎 𝑛𝑛 2𝜋𝜋 −∞
If 𝑋𝑋 is binomially distributed with parameters 𝑛𝑛 and 𝑝𝑝, then 𝑋𝑋
has the same distribution as the sum of 𝑛𝑛 independent
Bernoulli random variables, each with parameters 𝑝𝑝. Hence,
the distribution of
𝑋𝑋 − 𝐸𝐸 𝑋𝑋 𝑋𝑋 − 𝑛𝑛𝑛𝑛
=
𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋 𝑛𝑛𝑛𝑛(1 − 𝑝𝑝)
approaches the standard normal distribution as 𝑛𝑛 approaches
∞.