ELEC2600 Tutorial 10

Conditional Probability
Conditional Expectation
A Function of Two Random Variables
 Conditional Probability and Independence
2
 X and Y are discrete RVs
P Y  k , X  j  p X ,Y ( j , k )
pY | X (k | j )   if p X ( j )  0 ; undefined, otherwise
P  X  j

pX ( j )

 
k 
pY | X (k | j )  1

 p X ,Y ( j , k )  pY | X (k | j ) p X ( j ) p X ,Y ( j , k )  p X |Y ( j | k ) pY (k )

 If X and Y are independent

p X ,Y ( j , k ) p X ( j ) pY (k )
pY | X (k | j )    pY (k )
pX ( j) pX ( j)
p X ,Y ( j , k ) p X ( j ) pY (k )
p X |Y ( j | k )    pX ( j)
pY (k ) pY (k )
 Total probability theorem: P[Y in A]    p X ,Y ( j , k )   P[Y in A | X  j ] p X ( j )
j k in A j
 Conditional pdf
3
 X and Y are continuous RVs
f  x,   d 
y

 FY X  y x  
 X ,Y
if f X  x   0 and undefined otherwise
fX  x
f X ,Y  x, y 
 fY X  y x  
fX  x
 Properties of conditional pdf
 f X ,Y ( x, y )  f X |Y ( x | y ) fY ( y ) f X ,Y ( x, y )  fY | X ( y | x) f X ( x)

 Total probability theorem: fY ( y )   fY | X ( y | x) f X ( x)dx
 Bayes Theorem: f X |Y ( x | y )   fY | X ( y | x) f X ( x)
 fY | X ( y | x) f X ( x)dx


 If X and Y are independent f X |Y ( x | y )  f X ( x)

 Conditional Expected Value
4

 The conditional expectation of Y given X = x


E[Y | x ]   y fY | X  y | x  dy if Y is continuous


E[Y | x ]   j  pY | X ( j | x ) if Y is discrete and integer valued

j

 g(Y) is a random variable E[ g (Y ) | x]   g ( y ) f Y | X ( y | x) dy




Example 1
The total number of defects X on a chip is a Poisson random variable
with mean α. Suppose each defect has a probability p of falling in a
specific region R and that the location of each defect is independent
of the others. Let Y be the number of defects inside the region R, and
let Z be the number of defects outside the region.
(a) Find the joint pmf of Y and Z.
(b) Find the conditional pmf of Z given Y. (Hint: !
)

5
Example 1 Solution
X=Y+Z is a Poisson random variable, , for 0,1, …
!
(a) , , , ,
| 1 ,
!
, 0,1, …
(b) ∑ , , ∑ 1
!
∑ 1
! ! ! !

∑
! ! ! !

, , !
,
| | =
!
!

6
Example 2
Suppose that X and Y are continuous random variables whose joint
density is constant in the shaded area shown and zero otherwise.
y
1

0 0.5 1 x

(a) Find the conditional densities | | and | | , for 0

, 1.
(b) Find the conditional expected values | for 0 1, and
| for 0 1.
7
Example 2 - Solution
(a)

8
Example 2 - Solution
(b)

9
 Example 3
10

The number of customers that arrive at a bank during a time t is a

Poisson RV with parameter βt. The time required to service each
customer is an exponential RV with parameter α. Assume that
customer arrivals and service times are independent.
(a) Find the pmf of the number of customers N that arrive during the
service time T of a specific customer.
(b) Find the mean and variance of the number of customer arrivals, N,
during the service time, T, of a particular customer.

Solution
(a) pN |T (k | t )  (  t ) k
e   t for k  {0,1, 2,...}
k!
fT (t )   e  t u (t ) where u (t ) is the unit step
 Example 3 Solution
11

By the total probability theorem, pN (k )  0 pN |T (k | t )  fT (t )dt
 ( t )k   t  k 
pN ( k )    t
0 e
e   e dt  t k  (   ) t
dt for k  {0,1, 2,...}
0 k! k!
 k 1  k r
Letting r  (   )t and dr  (   )dt , pN (k )  (   ) k 1 k ! 0 r e dr

It turns out the integral is a gamma function, ( z )   x z 1e  x dx z  0
 k 1 0
pN (k )  (k  1)
(   ) k !
k 1

For non-negative integer k , (k  1)  k !

k
 k
    
Thus, pN (k )      for k  {0,1, 2,...}
(   ) k 1          

N is a geometric RV with p 
 
Example 3 - Solution
(b) Since the service time T is exponential, E[T] = 1/ and Var[T] =
1/2.
Since the number of customer arrivals is Poisson when the service
time is known, E[N|T] = T. Thus, E[N] =E[E[N|T]]= .
Intuitively, the average number of customer arrivals is the average
service time 1/ multiplied by the rate at which customers arrive, 
(customers/min).
Var[N] = E[N2] – (E[N])2.
Since N is Poisson, Var[N|T] = T and
E[N2|T] = Var[N|T] + (E[N|T)2 = T + 2T2
Thus, E[N2] =E[N2] = E[E[N2|T]]= T] + 2E[T2] and
Var[ N ]   E[T ]   2 E[T 2 ]   2 (E[T ]) 2
  E[T ]   2 Var[T ]   /    2 /  2
12
 Example 4
13
Suppose in a communication system, the received signal is the
transmit signal corrupted by the noise. The voltage of the information
signal and the voltage of the noise are both continuous and they are
independent from each other. The voltage of the information signal is
uniformly distributed on the interval [-1, 1] and the noise voltage is
also uniformly distributed on the interval [-0.5, 0.5].
a) Find the mean and variance of the received signal.
b) Find the conditional density function for the received signal given
the information signal.
c) Find the probability density function for the received signal.
d) Find the conditional expected value of the received signal given
the information signal.
 Example 4 Solution
14

a)

b)
FY | X  y | x   P Y  y | x   P  X  N  y | x   P  N  y  X | x 
 FN | X  y  x | x   FN  y  x 
 Example 4 Solution
15

c)

d)
 One Function of Two Discrete Random Variables
16

 Z = g(X, Y), X and Y are discrete random variable

 pmf of Z:

 Example 1
Let K and N be independent random variables with nonnegative
integer values.
(a) Find an expression for the pmf of M = K + N.
(b) Find the pmf of M if K and N are binomial random variables
with parameters (k, p) and (n, p)
Function of Two Discrete Random Variables – Example
1 Solution
17
Example 2
Alice and Bob arrive at different queues in the supermarket at the
same time. The time Alice needs to wait (measured in minutes)
follows a geometric distribution with mean 3 minutes and the waiting
time of Bob is also geometrically distributed but with mean 5
minutes. Assume the waiting times of Alice and Bob are independent.
Since they intend to leave together, the longest waiting time of them
max , becomes important.
Note: The range of both and is 1, 2, 3, ⋯ .
(a) Find the cdf of .
(b) Find the pmf of .
(c) Find the value of .
Hint: You may need the following equality:
1
1 , where 0 1.

18
Example 2 Solution
(a) Considering the geometric distribution defined on 1, 2, 3, ⋯, we have
max , , P
2 4
1 1
3 5
2 4 8
1 1
3 5 15
(b) From the results of (a), we have
1
1 2 1 4 7 8
1
3 3 5 5 15 15
(c) From the results of (b), we have
1 2 1 4 7 8
3 3 5 5 15 15
15 41
3 5
7 7

19
 Function of Two Continuous Random Variables
20

 Z = g(X, Y), X and Y are continuous random variable

 Solution Approach
Function of Two Continuous Random Variables – Example 1
Suppose the lifetimes of two bulbs are independent and exponentially
distributed with mean 1 year.
a) Find the cumulative density function of the absolute value of the
difference of their lifetimes.
b) Use the result of part (a) to find the expectation of the absolute
value of the difference of their lifetimes.

1  e  z z0
FZ  z   
0 otherwise
21
Function of Two Continuous Random Variables –
Example 1 Solution
22
Function of Two Continuous Random Variables – Example 2
Consider the MTR Tseung Kwan O line starting from North Point towards Po
Lam and LOHAS Park. To operate this line, MTR instituted a “3+1" service
whereby, for every 4 successive trains, the first three run to Po Lam and the fourth
train runs to LOHAS Park. Suppose that the amount of time MTR trains stay at
Tseung Kwan O station is independent and exponentially distributed with a mean
value of 1 minute. Assume the time interval between 2 successive trains begins
when the first train leaves and ends when the next train arrives. Regardless how
long a train stays at Tseung Kwan O station, the time intervals between any 2
successive trains are independent and exponentially distributed with a mean value
of 3 minutes.
(a) Suppose Jack arrives at Tseung Kwan O station when a train just arrived,
however he decides not to take this train. Find the pdf and cdf of the amount
of time he will wait until the next train arrives.
(b) Suppose Jack arrives at Tseung Kwan O station when a train to LOHAS Park
just left. What is the expected value of the amount of time that he will wait
until the next train to LOHAS Park arrives?
(c) Suppose Jack arrives at Tseung Kwan O station when a train just left. Find
the expected value of the amount of time he will wait until the next train to
Po Lam arrives. 23
Function of Two Continuous Random Variables – Example 2

(a) Let denote the amount of time that a train stays at the station, Y denote the time interval
of 2 successive trains, denote the amount of time that the next train will arrive. Therefore,
.
As the mean of exponential distribution is , we can get the pdf of X and Y as:

, 0 , 0
,
0, 0,
Then the joint pdf of and is easy to know since their independence relationship.
1
, , 0, 0
, 3
0,
To compute the cdf of , we should integrate , , over the region
, : , 0, 0
1 1 3
, , 1
3 2 2
Differentiate:
1 1
2 2

24
Function of Two Continuous Random Variables – Example 2 solution

(b) Let denote the amount of time that the next train runs to LOHAS Park will
arrive. Therefore, . Because X and Y are
independent random variables, we can obtain the expected value of as,
. 3 1 4 3 15
(c) Let denote the event that the next train runs to LOHAS Park, denote that
the next train runs to Polam. B denote the waiting time for the next train runs to
Polam.
We can compute that , .
If happens, the expectation of B is | 1 2 3 7,
if happens, the expectation of B is | 3.
Then the expected time he will wait is
| | 4

25