Probabilistic Methods in

Engineering
Lecture 11
Dr. Maha Ali
• Conditional PMF
• Conditional Expectation
• Law of Total Expectation
• Joint PMF
• Conditioning on a R.V.
Conditional PMF PCA B P

• The conditional PMF of a random variable X, conditioned on a

particular event A with P(A) > 0, is defined by
PxX

• Note that the events {X = x} ∩ A are disjoint for different values of x,

their union is A, and, therefore,

i
H
Example
• Let X be the roll of a fair six-sided die and let A be the event that the
roll is an even number. Then, by applying the preceding formula, we
obtain
k

P 4 24
Px A

4 21416
k
P roll iseven L
Px f otherwise
Conditional Expected Value and conditional
Variance EEX Ʃ Px x
• Definition: Given an event A ⊆ S with P(A) ≠0 and a discrete random
variable X associated with S,
• the conditional expected value of X given A is defined as

E X A
PXAG My A
• the Conditional Variance of X given A is defined as

WAR XIA E X
My
The Law of Total Expectation
MAY
• The law of total expectation: 4 B
PCA B PCB
• Let X be a random variable and B be an event. Then PIA BC PCB
PNB PNB

E X E X B PCB E X BY P BY

• The proof is by using the law of total probability and definition of

conditional expectation.
The Law of Total Expectation , General form
• Let B1, B2, . . . , Bn be n be pairwise disjoint (mutually exclusive)
events whose union equals the sample space S.
• Let X be a discrete random variable associated with S. Then

E X E X B P Be E X B2 PLBE ECRIBAPU

E X 2 E X Bi P BD
The Law of Total Expectation, Example
• There are two biased coins in a box. For one of them P(H) = 1/ 3 and
for the other one P(H) = 2/ 3 . 0

• A coin is selected at random from the box and is flipped 10 times. Let
X be the total number of H’s observed. What is E[X]?
• Let B be the event that the first coin is selected. Then we know that

E X B 10 E X Ba 10 33
YP p B1 P B2 L
• Then we have

E X E X Be PLA E X Ba PC Ba
do I 10 12 5
Example
• Messages transmitted by a computer in Boston through a data
network are destined for New York with probability 0.5, for Chicago
with probability 0.3, and for San Francisco with probability 0.2. The
transit time X of a message is random. Its mean is 0.05 seconds if it is
destined for New York, 0.1 seconds if it is destined for Chicago, and
0.3 seconds if it is destined for San Francisco. Then. E[X] is easily
calculated using the total expectation theorem as
IN a or

SF
if E A chickpt
E X SF 0
o I sec
3 sec

X E X NY P Y E X Chicas
E
0 5 0.05 0 3 0.1 0 2 0 3 0 115seen If
iClicker (Example)
• A miner is trapped in a mine containing 3 doors. The first door leads
to a tunnel that will take him to safety after 3 hours of travel. The
second door leads to a tunnel that will return him to the mine after 5
hours of travel. The third door leads to a tunnel that will return him to
the mine after 7 hours. If we assume that the miner is at all times
equally likely to choose any one of the doors, what is the expected
length of time until he reaches safety?

E PEP 13 PE P D 2
E PED
EEX 033 7 FEE
D1 I 120 in
E X D2 5 E X X 3 5 14 7 EEN
EG 15 hours
Pairs of random variables
• For a random experiment with sample space Ω, a pair of random variable (X,
Y ) is a function that assigns a point in R2 , (X(ζ), Y (ζ)), to each outcome ζ ∈
Ω.
• Example 1: Consider a random experiment of selecting a student ζ from a
class. Define X(ζ) as the height of student ζ and Y (ζ) as weight of student ζ.
Then (X(ζ), Y (ζ)) is a pair of random variables associated to ζ.

• Example 2: Consider the experiment of rolling a fair dice twice. Let ζ denote
the outcome, X(ζ) be the first number and Y (ζ) be the sum of the two
numbers. Then (X(ζ), Y (ζ)) is a pair of random variables associated to ζ.
Experiment and 2 R.V.

Sometimes it is helpful; to look at it

as a vector geometry in 2-D

As a point in the
XY plane
Model as 2 d R.V

Location on GPS
High and low temperature of the day
JOINT PMFS OF MULTIPLE RANDOM VARIABLES
• The probabilities of the values that X and Y can take are captured by
the joint PMF of X and Y, denoted PX,Y

P X x AY P x s P X x Y y PLAY
 ftp.t
Joint PMF cont.

P XY E A Ʃ Pay S
NS GA
Marginal PMF from Joint PMF

P LEI P X K ƩPCX K Y y
y

• We can calculate the PMFs of X and Y by using the formulas

PxyC4S Px 1 15
Px X PCS

marginal joint
Joint and
Marginal
probabilities

Pp PIFE.mx PLY ne I PLY D

Note: Once I projected the
marginal I lost the
correlation and can’t go back
again.

Just knowing the marginal is

not enough to recreate the
joint probability.
We can have same marginal
probability from different
joint probability
Conditioning one Random Variable on another
PCA B B
• Let X and Y be two discrete random variables. The conditional PMF of
X given Y is

P
Pxy 14

PH PH xH s
Px 1H41
Effy yffd
Conditional pmf cont.
Conditional Probability Cont.
Pay NY
plans PLACB Pl

Py Px 44
Pay x 5 Y
y

PxCx LYA
Piya
Independence of Random Variables P AIB
PIAMB PIA PIB
Intuitively, independence means that the value of Y provides no LILY
information on the value of X.

Py 4 15 Px x Pyls for all x y

1244147 PxLx for all 1 with Py s o

anal all
Functions of Multiple Random Variables
• When there are multiple random variables of interest, it is possible to generate
new random variables by considering functions involving several of these random
variables.
• For example : Z = g(X. Y) of the random variables X and Y defines another
random variable.
its PMF can be calculated from the joint PMF Px.y according to

Pz C2 Eggen
E g GSI 8 451 Pay As
Example
• Let X be a coin flip, Y be a die. The sample space of X is {0, 1}, whereas
the sample space of Y is {1, 2, 3, 4, 5, 6}. The joint PMF, according to
our definition, is the probability P[X = x and Y = y], where x takes a
binary state and Y takes one of the 6 states. The following table
summarizes all the 12 states of the joint distribution.
Y
1 2 3 4 5 6
0 1/12 1/12 1/12 1/12 1/12 1/12
X
1 1/12 1/12 1/12 1/12 1/12 1/12

In this example, we observe that if X and Y are not interacting

with each other (formally, independent), the joint PMF is the
product of the two individual probability
Example

• Now if we define A = {X +Y = 3}, what is the probability P[A]?

013
P A
EnP 9
Pay Pay 11,2
Example

• If B = {min(X, Y ) = 1}, find P[B]

P
PCB 14 1451 a
iClicker Example

see

PLX o 0.1 0.2 0 3

P X 1
0 7
iClicker