23

2.2. Multivariate Probability Distributions

(I) Bivariate Probability Distribution
Joint Probability Distribution (Density) Function:
 Let X 1 and X 2 be discrete random variables. The joint probability
distribution function is
f  x1 , x 2   P X 1  x1 , X 2  x2  ,
where x1 and x 2 are possible values of X 1 and X 2 , respectively. f
satisfies the following conditions:
1. f  x1 , x 2   0 for all x1 , x 2 ;
2.  f  x , x   1 .
x2 x1
1 2

 Let X 1 and X 2 be continuous random variables. The joint probability

density function satisfies the following conditions:
1. f  x1 , x 2   0 for    x1  ,  x 2   ;
 
2. , x 2  dx1 d x 2  1 .
  f x
  
1

Cumulative Distribution Function:

 For any random variables X 1 and X 2 , the joint (cumulative) distribution
function is given by F  x1 , x 2   P X 1  x1 , X 2  x2  .
 As X 1 and X 2 are discrete,
P  a  X 1  b, c  X 2  d     f x ,x  . 1 2
c  x2  d a x1 b

As X 1 and X 2 are continuous,

d b
P a  X 1  b, c  X 2  d     f  x1 , x2  dx1dx2 .
c a

 For any random variables X 1 and X 2 with joint cumulative distribution

function F  x1 , x 2  ,
1. F   ,   F   , x 2   F  x1 ,   0;
2. F  ,    1 ;
3. if b  a, d  c , then
P a  X 1  b, c  X 2  d   F  b, d   F  b, c   F  a, d   F  a, c  .

Example 1:
Form a collection of 3 white balls, 2 black balls, and 1 red ball, 2 balls is to be
randomly selected. Let X 1 denote the number of white balls and X 2 the
number of black balls.
(a) Find the joint probability distribution table of X 1 and X 2 .

23
 24

(b) F 1,1 and F  2,0  .

[solution:]
(a)
 3  2  1 
   
i j 2i  j 
P  X 1  i, X 2  j   f  i, j      ,0  i, j  2; i  j  1 or 2;
 6
 
 2
For example,
 3  2  1 
   
1 1 0 3 2 6 .
P X 1  1, X 2  1  f 1,1       
6 15 15
 
 2
The joint probability distribution table is
x1
x2 0 1 2 Total
0 0 3 3 6
15 15 15
1 2 6 0 8
15 15 15
2 1 0 0 1
15 15
Total 3 9 3 1
15 15 15
(b) F 1,1  P X 1  1, X 2  1  f  0,0  f  0,1  f 1,0  f 1,1  0  2  3  6  11 .
15 15
033 6
F  2,0  P X 1  2, X 2  0  f  0,0  f 1,0  f  2,0  
15 15

Example 2:
Let f  x1 , x 2   2 x1 ,0  x1  k ;0  x 2  1; and f  x1 , x 2   0, otherwise.
(a) Find k .
(b) Find F  0.7,0.5 , F  2,0  and F  0.2,3 .
[solution:]
(a)
 

 x 
1 k 1
f  x1 , x 2 dx1 dx 2 
1
2 k
   2 x1dx1dx2  dx 2  k dx 2  k 2  1  k  1 .
2
1 0 0
   0 0 0

24
 25

 k  1.
(b)
0.5 0.7 0.50.7
F  0.7,0.5  P  X 1  0.7, X 2  0.5   f  x1 , x2  dx1dx2    2 x dx dx1 1 2
  0 0

 x 
0.5 0.5
2 0.7

0
1 0 dx2   0.49dx
0
2  0.49  0.5  0.245

0 2 0 1
F  2,0   P X 1  2, X 2  0    f  x1 , x2  dx1dx2    2 x1dx1dx2
   0 0

 
0 0
1
  x12 0 dx2   1dx2  1  0  0
0 0
3 0.2 1 0.2
F  0.2,3  P  X 1  0.2, X 2  3   f  x1 , x2  dx1dx2    2 x dx dx
1 1 2
   0 0

 x 
1 1
2 0.2
 1 0 dx2   0.04dx2  0.04
0 0

Marginal Probability Distribution (Density) Function:

 Let X 1 and X 2 be discrete random variables. The marginal probability
distribution function for X 1 and X 2 are
f1  x1   P X 1  x1    f  x1 , x 2  ,
x2

and
f 2  x 2   P X 2  x 2    f  x1 , x 2  ,
x1

respectively.
 Let X 1 and X 2 be continuous random variables. The marginal
probability density function for X 1 and X 2 are

f 1  x1    f x 1 , x 2 dx 2 ,


and

f 2  x2    f x 1 , x 2 dx1 ,


respectively.

Conditional Probability Distribution (Density) Function:

 Let X 1 and X 2 be discrete random variables. The conditional probability
distribution function X 1 given X 2 is

P X 1  x1 , X 2  x 2  f  x1 , x 2  ,
f1  x1 | x 2   P X 1  x1 | X 2  x 2   
P X 2  x2  f 2  x2 

25
 26

provided that f 2  x 2   0 . Similarly, the conditional probability distribution

function X 2 given X 1 is

P X 1  x1 , X 2  x2  f  x1 , x 2  ,
f 2  x 2 | x1   P X 2  x 2 | X 1  x1   
P X 1  x1  f1  x1 

provided that f 1  x1   0 .
 Let X 1 and X 2 be continuous random variables. The conditional
probability density function X 1 given X 2 is

 f  x1 , x 2 
 , f 2  x2   0
f1  x1 | x 2    f 2  x 2 
0, otherwise.
Similarly, the conditional probability density function X 2 given X 1 is

 f  x1 , x 2 
 , f1  x1   0
f 2  x 2 | x1    f 1  x1 
0, otherwise.

Example 1 (continue):
(c) Find the marginal probability distribution function of X 1 .
(d) Given one of chosen balls being black, what is the distribution for the number
of balls being white?
[solution:]
(c) f1 (0)  f  0,0  f  0,1  f  0,2   0  2  1  3 ,
15 15
3 6 0 9 ,
f 1 (1)  f 1,0   f 1,1  f 1,2   
15 15
300 3
f1 (2)  f  2,0   f  2,1  f  2,2   
15 15

(d) f 2 (1)  f  0,1  f 1,1  f  2,1  2  6  0  8

15 15

2
f  0,1 15 1 ,
f1  0 | 1  P X 1  0 | X 2  1   
f 2 1 8 4
15

26
 27

6
f 1,1 15 3 ,
f1 1 | 1  P X 1  1 | X 2  1   
f 2 1 8 4
15
f  2,1 0
f1  2 | 1  P X 1  2 | X 2  1   0
f 2 1 8
15

Example 2 (continue):
(c) Find the marginal probability density function for X 1 and X 2 .
(d) Find the conditional probability density function for X 1 given X 2 and the
conditional probability density function for X 2 given X 1
[solution:]
(c)
 1
f1  x1    f  x1 , x 2  dx 2   2 x1 dx 2  2 x1 ,0  x1  1; .
 0

 
1
f 2  x2    f  x , x  dx
1
1 2 1   2 x1 dx1  x12 0
 1,0  x 2  1;
 0

(d) For 0  x1  1,0  x 2  1,

f  x1 , x 2 
f 1  x1 | x 2    2 x1
f 2  x2 

and

f  x1 , x 2  .
f 2  x2 | x1   1
f1  x1 

Independence of Two Random Variables:

Let X 1 and X 2 be random variables. If X 1 and X 2 are independent if and
only if
f  x1 , x 2   f1  x1  f 2  x 2  ,
for all pairs of  x1 , x 2  .

Note: As f 2  x 2   0 and X 1 and X 2 are independent, then

f1  x1 | x 2   f1  x1  .
Further, as X 1 and X 2 are independent,

27
 28

 
P X 1  x1 , X 2  x 2  f  x1 , x 2   f1  x1  f 2  x 2   P X 1  x1  P X 2  x 2 
 P A  B   P A P B  , A   X 1  x1  , B   X 2  x 2 .

Example 1 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
No since f 1,0  3  5  6  9  6  f1 1 f 2  0 .
15 25 25 15 15

Example 2 (continue):
(e) Are X 1 and X 2 independent?
[solutions:]
Yes since f  x1 , x 2   2 x1  f1  x1  f 2  x 2  .

Example 3:
Let and Are
f  x1 , x 2   6 x1 x 22 ,0  x1  k ;0  x 2  1; f  x1 , x 2   0, otherwise. X1

and X 2 independent?
[solution:]

 
1
f 1  x1   f  x1 , x 2  dx2   6 x1 x 22 dx 2  2 x1 x 23
1

 0
0
 2 x1 ,0  x1  1;


 
1
f 2  x2   f  x1 , x 2  dx1   6 x1 x 22 dx1  3x12 x 22
1

 0
0
 3 x 22 ,0  x1  1.

Thus,
.
f  x1 , x 2   6 x1 x 22  f1  x1  f 2  x 2 
Therefore, X 1 and X 2 are independent.
(II) Multivariate Probability Distribution
Joint Probability Distribution (Density) Function:
 Let X 1 , , X n be discrete random variables. The joint probability
distribution function is
f  x1 ,  , x n   P  X 1  x1 ,  , X n  x n  ,
where x1 ,  , x n are possible values of X 1 ,  , X n , respectively. f
satisfies the following conditions:
1. f  x1 ,  , x n   0 for all x1 ,  , x n ;
2.   f  x , , x   1 .
x2 xn
1 n

28
 29

 Let X 1 ,  , X n be continuous random variables. The joint probability

density function satisfies the following conditions:
1. f  x1 ,  , x n   0 for    x1 ,  , x n  ; ;
 
2. ,  , x n  dx1  d x n  1 .


 f x

1

Independence of Multiple Random Variables:

Let X 1 ,  , X n be random variables. If X 1 ,  , X n are independent if and only if
f  x1 , , x n   f1  x1   f n  x n  ,
for all pairs of  x1 , , x n  , where f j is the marginal probability distribution (or
density) function of X j .
Likelihood Function:
Let x1 ,  , x n be sample observations taken on corresponding random variables
X 1 ,  , X n .whose joint probability distribution or density function depends on
the parameters 1 ,  ,  k . The likelihood of sample is
l 1 , , k   f  x1 , , x n | 1 , , k  .
As X 1 ,  , X n are a random sample from a common probability distribution (or
density) with the parameters 1 ,  ,  k , i.e., the probability distribution or
density function equal to f  x | 1 , , k  and X 1 ,  , X n being i.i.d. (identically
independent distributed), the likelihood is
n
l 1 ,  ,  k    f  xi | 1 ,  ,  k   f  x1 | 1 ,  ,  k   f  xn | 1 ,  ,  k  .
i 1

Example 4:
Let X 1 ,  , X n are a random sample from a normal distribution N   ,  2  . Then,
n
 xi    2    xi   2
l  ,  2      2 2 
n 2 2
e n i 1
2 2
2
e
i 1 2 2

Example 5:
Let X 1 ,  , X n are a random sample from a binomial distribution B  m, p  .
Then,
n
n
 m  xi  n  m   xi n
nm  xi
l  p      p 1  p      p 1  p  i 1
m xi i 1

i 1  xi   i 1  xi 

Example 6:
Let X 1 ,  , X n are a random sample from a Poisson distribution Poisson   .
Then,

29
 30

n
 xi
n
e  xi e n  i 1
l      n
xi !
i 1
x!
i 1
i

30