Scribd
Upload
55 views

Notes 3

The document discusses conditional probability and expectation. It provides examples of calculating conditional probabilities and expectations. Specifically: 1) It defines conditional probability and expectation, and shows how to calculate expected value given an event. 2) It provides an example of a coin toss experiment and calculates conditional probabilities and expectations. 3) It discusses the law of total expectation and how to use it to calculate unconditional expected values from conditional expectations.

Uploaded by

thinh
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
55 views

Notes 3

The document discusses conditional probability and expectation. It provides examples of calculating conditional probabilities and expectations. Specifically: 1) It defines conditional probability and expectation, and shows how to calculate expected value given an event. 2) It provides an example of a coin toss experiment and calculates conditional probabilities and expectations. 3) It discusses the law of total expectation and how to use it to calculate unconditional expected values from conditional expectations.

Uploaded by

thinh
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Probability 2 - Notes 3

The conditional distribution of a random variable X given an event B.


Let X be a random variable defined on the sample space S and B be an event in S. Denote
and B)
P(X = x|B) P(X=x
by fX|B (x). This is a probability mass function. We can therefore
P(B)
find the expectation of X conditional on B. E[X|B] = x x fX|B (x).

Example We toss a coin twice. Let X count the number of heads, so X Binomial 2, 12 , and
let B1 be the event that the first outcome is a head and B2 be the event that the first outcome is a
tail. Then P(B1 ) = P({HT, HH}) = 21 and P(B2 ) = P({T H, T T }) = 12 .
Hence P(X = 0|B1 ) = 0, P(X = 1|B1 ) =
E[X|B1 ] = 32 .
Also P(X = 0|B2 ) =
E[X|B2 ] = 12 .

P({T T }
P(B2 )

P({HT }
P(B1 )

= 12 , P(X = 1|B2 ) =

1
2

and P(X = 2|B1 ) =

P({T H}
P(B2 )

1
2

P({HH}
P(B1 )

= 21 . Then

and P(X = 2|B2 ) = 0. Therefore

We can also obtain the conditional distribution of X|B1 and X|B2 by considering the implications
of the experiment. If B1 occurs then X|B1 equals 1+ Y where Y counts the number of heads
in the second toss of the coin, so Y Bernoulli 12 . If B2 occurs then X|B equals Y . Hence
E[X|B1 ] = 1 + E[Y ] = 1 + 21 and E[X|B2 ] = E[Y ] = 12 .
We will now look at a similar law to the law of total probability which is for expectations. This
can be used to find the expected duration of the sequence of games (expected number of games
played) for the gamblers ruin problem.
The law of total probability for expectations
From the law of total probability, if B1 , ..., Bn partition S then for any possible value of x,
n

P(X = x) =

P(X = x|B j )P(B j ) =

j=1

fX|B j (x)P(B j ).

j=1

Multiplying by x and summing we obtain the Law of Total Probability for Expectations
n

E[X] =

E[X|B j ]P(B j )

j=1

Example. Consider the set-up for a geometric distribution. We have a sequence of independent
trials of an experiment, with probability p of success at each trial. X counts the number of trials
till the first success.
Let B1 be the event that the first trial is a success and B2 be the event that the first trial is a
failure.

When B1 occurs, X must equal 1. So P(X = 1 and B1 ) = P(B1 ) and P(X = x and B1 ) = 0 if
x > 1. Hence the distribution of X|B1 is concentrated at the single value 1 i.e. X|B1 is identically
equal to 1.
If B2 is the event that the first trial is a failure, then the number of trials until a success in the
subsequent trials, Y , has the same distribution as X. We also have carried out the first trial.
Hence X|B2 is equal to 1 +Y where Y has the same distribution as X.
Hence E[X|B1 ] = 1 and E[X|B2 ] = 1 + E[Y ] = 1 + E[X]. Therefore

E[X] = E[X|B1 ]P(B1 ) + E[X|B2 ]P(B2 ) = p 1 + q (1 + E[X]).


Therefore E[X] = 1p .
The gamblers ruin problem, the expected duration of the game.
We use the same notation as before. The gambler plays a series of games starting with a stake
of k units. He stops playing when he reaches either M or N units, where M k N. Let Tk be
the random variable for the number of games played (the duration of the game). Set Ek = E[Tk ].
Theorem. The expectations Ek satisfy the following difference equations:
Ek = 1 + pEk+1 + qEk1 , if M < k < N; EM = EN = 0.
Proof. Denote by B1 and B2 the events the gambler wins the first game and the gambler loses
the first game. These events form a partition and the law of total probability for expectations is
just
E[Tk ] = E[Tk |B1 ]P(B1 ) + E[Tk |B2 ]P(B2 )
If he wins the first game he has k + 1 units so the distribution of Tk given B1 has the same
distribution as 1 + Tk+1 where Tk+1 measures the duration of the game starting from k + 1 units.
Hence E[Tk |B1 ] = 1 + E[Tk+1 ]. Similarly E[Tk |B2 ] = 1 + E[Tk1 ]. Then

Ek = p(1 + Ek+1 ) + q(1 + Ek1 )


and hence we obtain the difference equation

Ek = 1 + pEk+1 + qEk1
EM = EN = 0 since the gambling stops playing immediately. 2
The equation for Ek is sometimes wriiten in the following equivalent form
pEk+1 Ek + qEk1 = 1
1
. When p = 21 a
When p 6= 12 a particular solution to this equation is Ek = Ck where C = qp
particular solution is Ek = Ck2 where C = 1. Now, as for differential equations, the general

solution to the particular difference equation is the particular solution just obtained plus the
general solution to the general equation pEk+1 Ek + qEk1 = 0.
Case when p 6= 12 .
k
k
q
Ek =
+A+B
q p
p
Since 0 = EM =

M
qp

+A+B

M
q
p

and 0 = EN =

N
qp

+A+B

N
q
p

,B=

and

M
A = (qp)

q
p

q M
p

N .
qp

(NM)

M N
(qp) qp qp

If we write Ek as Ek (M, N) to explicitly include the bound-

aries we obtain

k
Ek (M, N) =

q
p

(k M) (N M)

(q p)
(q p) q N

M
q
p

M
q
p

Case when p = 12 .
Ek = k2 + A + Bk
Since 0 = EM = M 2 + A + BM and 0 = EN = N 2 + A + BN, B = N + M and A = MN.
Hence writing Ek as Ek (M, N) to explicitly include the boundaries

Ek (M, N) = (k M)(N k)
Conditional distribution of X|Y where X and Y are random variables.
For any value y of Y for which P(Y = y) > 0 we can consider the conditional distribution of
X|Y = y and find the expectation and variance of X over this conditional distribution, E[X|Y = y]
and Var(X|Y = y). Let fX|Y (x|y) = P(X = x|Y = y). Consider the function of Y which takes the
value E[X|Y = y] when Y = y. This is a random variable which we denote by E[X|Y ]. Similarly
we define Var(X|Y ) and E[g(X)|Y ] to be the functions of Y (so random variables) which take
value Var(X|Y = y) and E[g(X)|Y ] when Y = y.
Theorem. (i) E[X] = E[E[X|Y ]], (ii) Var(X) = E[Var(X|Y )] +Var(E[X|Y ]) and (iii) GX (t) =
E[E[t X |N]].
Proof We show that E[g(X)] = E[E[g(X)|Y ]]. Now

E[g(X)|Y = y] =

g(x) fX|Y (x|y) =

x=0

E[E[g(X)|Y ]]

g(x)

x=0

P(X = x,Y = y)
P(Y = y)

=
y=0 E[g(X)|Y = y]P(Y = y)
P(X=x,Y =y)

=
y=0 x=0 g(x) P(Y =y) P(Y = y)

=
x=0 g(x) y=0 P(X = x,Y = y)
=
x=0 g(x)P(X = x) = E[g(X)]

(i) If we let g(X) = X we immediately obtain E[X] = E[E[X|Y ]].


(ii) If we let g(X) = X 2 we obtain E[X 2 ] = E[E[X 2 |Y ]].
Now Var(X|Y )) = E[X 2 |Y ] (E[X|Y ])2 and hence
E[Var(X|Y )] = E[E[X 2 |Y ]] E[(E[X|Y ])2 ] = E[X 2 ] E[(E[X|Y ])2 ]

Var(E[X|Y ]) = E[(E[X|Y ])2 ] (E[E[X|Y ]])2 = E[(E[X|Y ])2 ] (E[X])2


Therefore E[Var(X|Y )] +Var(E[X|Y ]) = E[X 2 ] (E[X])2 = Var(X).
(iii) If we let g(X) = t X we obtain GX (t) = E[t X ] = E[E[t X |N]].
Example Let X Binomial(n, p) and Y Binomial(m, p) where X and Y are independent.
Then R = X +Y Binomial(n + m, p).

P(X = x|R = r) =

P(X=x,R=r)
P(R=r)

P(X=x,Y =rx)
P(R=r)

rx qmr+x
nC px qnx mC
x
rx p
n+mC pr qn+mr
r

P(X=x)P(Y =rx)
P(R=r)

nC mC
x
rx
n+mC
r

Hence the conditional distribution of X|R = r is hypergeometric. This provides the basis of the
2 2 contingency table test of equality of two binomial p parameters in statistics.
Example The number of spam messages Y in a day has Poisson distribution with parameter .
Each spam message (independently) has probability p of not being detected by the spam filter.
Let X be the number getting through the filter. Then X|Y = y has Binomial distribution with
parameters n = y and p. Let q = 1 p.
Hence E[X|Y = y] = py, Var(X|Y = y) = pqy and E[t X |Y = y] = (pt + q)y so that E[X|Y ] = pY ,
Var(X|Y ) = pqY and E[t X |Y ] = (pt + q)Y . Therefore:

E[X] = E[E[X|Y ]] = E[pY ] = pE[Y ] = p

Var(X) = E[Var(X|Y )] +Var(E[X|Y ]) = E[pqY ] +Var(pY ) = p(1 p) + p2 = p

GX (t) = E[E[t X |Y ]] = E[(pt + q)Y ] = GY (pt + q) = e((pt+q)1) = e p(t1)


But this is the p.g.f. of a Poisson r.v. with parameter = p. Hence by the uniqueness of the
p.g.f., X Poisson(p).
Random Sums.
Let X1 , X2 , X3 , .... be a sequence of independent identically distributed random variables (i.i.d.
random variables), each with the same distribution, each having common mean , variance 2
and p.g.f. GX (t). Consider the random sum Y = Nj=1 X j where the number in the sum, N is
also a random variable and is independent of the X j . Then we can use our results for conditional
expectations.
Since E[Y |N = n] = E[nj=1 X j ] = nj=1 E[X j ] = n, we obtain the result that E[Y ] = E[E[Y |N]] =
E[N] = E[N].
Similarly Var(Y |N = n) = n2 so that
Var(Y ) = E[Var(Y |N)] +Var(E[Y |N]) = E[N2 ] +Var(N) = 2 E[N] + 2Var(N)
Also we can obtain an expression for the p.g.f. of Y .
h n
i
n
E[t Y |N = n] = E e j=1 X j = GX j (t) = (GX (t))n
j=1

so that
h
i
N
GY (t) = E[E[t |N]] = E (GX (t)) = GN (GX (t))
Y

Example
Let X j be the amount of money the jth customer spends in a day in a shop. The X 0 s are i.i.d.
random variables with mean 20 and variance 10. The number of customers per day N has
Poisson distribution parameter 100. The total spend Y in the day is Y = Nj=1 X j . So E[Y ] =
(20)(100) = 2000 and Var(Y ) = (10)(100) + (20)2 (100) = 41000.

You might also like