Conditional Distributions and Independent random variables

1. Conditional Distributions

Definition 1. Let Z = (X, Y ) be a random vector of discrete type with support EZ , joint
d.f. FZ and joint p.m.f. fZ . Then X and Y are discrete type random variables.

For a fixed y with P (Y = y) > 0, the function fX|Y (.|y) : R −→ R defined as

fX|Y (x|y) = P (X = x|Y = y), ∀ x ∈ R,
is called the conditional probability mass function of X, given Y = y. Thus, the conditional
probability mass function of X, given Y = y, is
P (X = x, Y = y) fZ (x, y)
fX|Y (x|y) = P (X = x|Y = y) = =
P (Y = y) fY (y)
( f (x,y)
Z
, if x ∈ EX|Y =y
= fY (y)
0, otherwise,
where EX|Y =y = {x ∈ R | (x, y) ∈ EZ } and fY is the marginal p.m.f. of Y .
The conditional cumulative distribution function of X, given Y = y, is defined as
FX|Y (x|y) = P (X ≤ x|Y = y)
P (X ≤ x, Y = y)
=
P (Y = y)
X fZ (xi , y)
=
fY (y)
xi ∈EX|Y =y ∩(−∞,x]
X
= fX|Y (xi |y), where xi ∈ EX|Y =y .
xi ≤x

In the similar manner, we can define the conditional probability mass function and
conditional cumulative distribution function of Y , given X = x, provided P (X = x) > 0.
Definition 2. Let Z = (X, Y ) be a random vector of continuous type with joint c.d.f. FZ
and joint p.d.f. fZ . Then X and Y are continuous type random variables. Let y ∈ R be
such that fY (y) > 0, where fY (y) > 0 is the marginal p.d.f. of Y .
The function fX|Y (.|y) : R −→ R defined as
fZ (x, y)
fX|Y (x|y) = , ∀ x ∈ R,
fY (y)
is called the conditional probability density function of X, given Y = y.
Also, the conditional cumulative distribution function of X, given Y = y, is defined as
Zx
FX|Y (x|y) = fX|Y (t|y)dt
−∞
Zx
fZ (t, y)
= dt
fY (y)
−∞
1
 In the similar manner, we can define the conditional probability density function and
conditional cumulative distribution function of Y , given {X = x}, provided fX (x) > 0,
where fX (x) > 0 is the marginal p.d.f. of X.

Note: Definition 1 and 2 can be generalized if we replace random variables X and Y by

random vectors X and Y .
Example 3. Let Z = (X, Y ) be a random vector with joint p.d.f.
(
6xy(2 − x − y), if 0 < x < 1, 0 < y < 1
f (x, y) =
0, otherwise

Then find the conditional p.d.f. of X, given Y = y, where 0 < y < 1.

Solution: The conditional p.d.f. of X, given Y = y, is

f (x, y)
fX|Y (x|y) =
fY (y)
 6xy(2−x−y)
 R1 , if 0 < x < 1
6xy(2−x−y)dx
=
0
0, otherwise
(
6x(2−x−y)
4−3y
, if 0 < x < 1
=
0, otherwise

Example 4. Let Z = (X, Y, Z) be a random vector with joint p.m.f.

(
xyz
, if (x, y, z) ∈ {1, 2} × {1, 2, 3} × {1, 3}
f (x, y, z) = 72
0, otherwise

(1) Find the conditional p.m.f. of X, given (Y, Z) = (2, 1).

(2) Find the conditional p.m.f. of (X, Z), given Y = 3.

Solution:

(1) The conditional p.m.f. of X, given (Y, Z) = (2, 1), is

f (x, 2, 1)
fX|(Y,Z) (x|(2, 1)) =
P ((Y, Z) = (2, 1))
(
2x
, if x ∈ EX|(Y,Z)=(2,1) = {x ∈ R | (x, 2, 1) ∈ EZ }
= 72P (Y =2,Z=1)
0, otherwise
(
2x
, if x ∈ {1, 2}
= 72P (Y =2,Z=1)
0, otherwise
P
Now, P (Y = 2, Z = 1) = f (x, 2, 1), where R(2,1) = {x ∈ R | (x, 2, 1) ∈
x∈R(2,1)
1
EZ } = {1, 2}. Hence, P (Y = 2, Z = 1) = f (1, 2, 1) + f (2, 2, 1) = 12
. Therefore,
(
x
, if x ∈ {1, 2}
fX|(Y,Z) (x|(2, 1)) = 3
0, otherwise
2
 (2) The conditional p.m.f. of X, given Y = 3, is
f (x, 3, z)
f(X,Z)|Y ((x, z)|3) =
P (Y = 3)
(
3xz
if x ∈ EX,Z)|Y =3 = {(x, z) ∈ R | (x, 3, z) ∈ EZ }
72P (Y =3)
,
=
0, otherwise
(
3xz
, if (x, z) ∈ {1, 2} × {1, 3}
= 72P (Y =3)
0, otherwise
P
Now, P (Y = 3) = f (x, 3, z), where R3 = {(x, z) ∈ R | (x, 3, z) ∈ EZ } =
(x,z)∈R3
{1, 2}×{1, 3}. Hence, P (Y = 3) = f (1, 3, 1)+f (1, 3, 3)+f (2, 3, 1)+f (2, 3, 3) = 12 .
Therefore,
(
xz
, if x ∈ {1, 2} × {1, 3}
f(X,Z)|Y ((x, z)|3) = 12
0, otherwise

2. Independent random variables

Definition 5. The random variables X1 , X2 , . . . , Xn are said to be independent if for any

sub-collection {Xi1 , Xi2 , . . . , Xik }, 2 ≤ k ≤ n, we have
k
Y
FXi1 ,...,Xik (x1 , x2 , · · · , xk ) = FXij (xj ), ∀ (x1 , x2 , · · · , xk ) ∈ Rk
j=1

where FXi1 ,...,Xik is the joint c.d.f. of (Xi1 , Xi2 , . . . , Xik ) and FXij is the marginal c.d.f. of
Xij , for 1 ≤ j ≤ k.
Theorem 6. Let X = (X1 , X2 , . . . , Xn ) : S −→ Rn be a n−dimensional (n ≥ 2) random
vector with joint c.d.f. FX . Let FXi be the marginal c.d.f. of Xi , for 1 ≤ i ≤ n. Then the
random variables X1 , X2 , . . . , Xn are independent if and only if
n
Y
FX (x1 , x2 , · · · , xn ) = FXi (xi ), ∀ (x1 , x2 , · · · , xn ) ∈ Rn .
i=1

Theorem 7. Let X = (X1 , X2 , . . . , Xn ) : S −→ Rn be a n−dimensional (n ≥ 2) random

vector of either discrete or continuous type. Let fX be the joint p.m.f. (or p.d.f.) of X
and fXi be the marginal p.m.f. (or p.d.f.) of random variable Xi , for 1 ≤ i ≤ n. Then

(1) the random variables X1 , X2 , . . . , Xn are independent if and only if

n
Y
fX (x1 , x2 , · · · , xn ) = fXi (xi ), ∀ (x1 , x2 , · · · , xn ) ∈ Rn .
i=1
n
Q
(2) the random variables X1 , X2 , . . . , Xn are independent ⇒ EX = EXi , where EX
i=1
is the support of random vector X and EXi is the support of random variable Xi ,
for 1 ≤ i ≤ n.
Theorem 8. Let X1 , X2 , . . . , Xn be the independent random variables.

(1) Let ψi : R −→ R be a function such that ψi (A) ∈ BR , for all A ∈ BR , for

i = 1, 2, · · · , n. Then the random variables ψ1 (X1 ), ψ2 (X2 ), . . . , ψn (Xn ) are inde-
pendent.
3
 (2) For Ai ∈ BR , i = 1, 2, · · · , n, we have
n
Y
P ({Xi ∈ Ai , i = 1, 2, · · · , n}) = P ({Xi ∈ Ai ).
i=1

Remark 9. X = (X1 , X2 ) be a random vector of either discrete or continuous type. Let

D = {x2 ∈ R | fX1 |X2 (.|x2 ) is defined}. Then for x2 ∈ D, X1 and X2 are independent if
and only if fX1 |X2 (x1 |x2 ) = fX1 (x1 ), for all x1 ∈ R,

i.e,
X1 and X2 are independent if and only if ∀ x2 ∈ D, the conditional distribution of X1 ,
given X2 = x2 , is the same as unconditional distribution of X1 .
Example 10. Let Z = (X, Y, Z) be a random vector with joint p.m.f.
(
xyz
, if (x, y, z) ∈ {1, 2} × {1, 2, 3} × {1, 3}
f (x, y, z) = 72
0, otherwise

(1) Are X, Y and Z independent random variables?

(2) Are X and Z independent random variables?

Solution:

(1) The supports of X, Y and Z are

EX = {x ∈ R | (x, y, z) ∈ EZ for some (y, z) ∈ R2 } = {1, 2}
EY = {y ∈ R | (x, y, z) ∈ EZ for some (x, z) ∈ R2 } = {1, 2, 3}
and
EZ = {z ∈ R | (x, y, z) ∈ EZ for some (x, y) ∈ R2 } = {1, 3},
respectively. For x ∈ EX , Rx = {(y, z) ∈ R2 | (x, y, z) ∈ EZ } = {1, 2, 3} × {1, 3}.
So the marginal p.m.f. of X is
 P
 f (x, y, z), if x ∈ EX
fX (x) = (y,z)∈Rx
0, otherwise
(
x
, if x ∈ {1, 2}
= 3
0, otherwise
Similarly the marginal p.m.f. of Y and Z are
(
y
, if y ∈ {1, 2, 3}
fY (y) = 6
0, otherwise
and
(
z
if y ∈ {1, 3}
4
,
fZ (z) =
0, otherwise

respectively. Clearly f (x, y, z) = fX (x)fY (y)fZ (z), for all (x, y, z) ∈ R3 . Thus
X, Y and Z are independent.
(2) Let X = (X, Y ). The support of X is EX = {(x, z) ∈ R2 | (x, y, z) ∈ EZ for some y ∈
R} = {1, 2} × {1, 3}. For (x, z) ∈ EX , R(x,z) = {x ∈ R | (x, y, z) ∈ EZ } = {1, 2, 3}.
4
 So the marginal p.m.f. of X is
 P
 f (x, y, z), if (x, z) ∈ EX
fX (x, z) = y∈R(x,z)
0, otherwise
(
xz
, if (x, z) ∈ {1, 2} × {1, 3}
= 12
0, otherwise
Thus fX (x, z) = fX (x)fZ (z), for all (x, z) ∈ R2 . Thus X and Z are independent.
Example 11. Let Z = (X, Y ) be a random vector with joint p.d.f.
(
1
x
, if 0 < y < x < 1
fZ (x, y) =
0, otherwise.
Are X and Y independent?

Solution: By Example 7 of Lecture 14, the marginal p.d.f. of X and Y are

(
1, if 0 < x < 1
fX (x) =
0, otherwise
and
(
− ln y, if 0 < y < 1
fY (y) =
0, otherwise
Clearly, fZ (x, y) 6= fX (x)fY (y). Hence, X and Y are not independent.

Alternative solution: The support of Z is EZ = {(x, y) ∈ R2 | 0 < y < x < 1}, and
the support of X and Y are (0, 1). Hence, EZ 6= EX × EY . Therefore, X and Y are not
independent.