Engineering Data Analysis

Joint Probability
Distribution
MPS Department | FEU Institute of Technology
 Engineering Data Analysis

Joint Probability Distribution of

Two or More Discrete Random
Variables
MPS Department | FEU Institute of Technology
 Subtopic 1
OBJECTIVES

 Enumerate the types of random variables and their probability distributions

 Calculate probabilities of events on each type of distribution
 Construct the probability distribution of two random variables
 Calculate the marginal and conditional distributions of each random variable.
 Subtopic 1
Joint Probability Distribution of Two
or More Discrete Random Variables
 Joint Probability Distributions
 Marginal Probability Distribution
 Conditional Probability Distribution
 More than Two Random Variables
• If X and Y are two discrete random variables, the probability distribution
for their simultaneous occurrence can be represented by a function with
values f(x,y) for any pair of values (x, y) within the range of the random
variables X and Y. It is customary to refer to this function as the joint
probability distribution of X and Y.
• Hence, in the discrete case,
• f(x,y) = P(X = x,Y = y)
• For example, if a television set is to be serviced and X
represents the age to the nearest year of the set and Y
represents the number of defective tubes in the set,
• then p(5,3) is the probability that the television set is 5 years
old and needs 3 new tubes.
The function
p(x,y) = P[X = x, Y = y]
is called the joint probability function of X and Y.
Quite often there will be 2 or more random variables (X, Y, etc) defined for
the same random experiment.
Example:
A bridge hand (13 cards) is selected from a deck of 52 cards.
X = the number of spades in the hand.
Y = the number of hearts in the hand.
In this example we will define:
p(x,y) = P[X = x, Y = y]
 Note:
The possible values of X are 0, 1, 2, …, 13
The possible values of Y are also 0, 1, 2, …, 13 and X + Y ≤
13.
p  x, y   P  X  x , Y  y  The number of ways of
choosing the y hearts for
the hand
The number of ways of
choosing the x spades for
the hand
The number of ways of
 13   13   26  completing the hand with
    diamonds and clubs.
 x   y   13  x  y 

 52  The total number of ways of
  choosing the 13 cards for the
 13  hand
  13   13   26 
   
 
    
x y 13 x y
Table: p(x,y)  52 
 
 13 
0 1 2 3 4 5 6 7 8 9 10 11 12 13
0 0.0000 0.0002 0.0009 0.0024 0.0035 0.0032 0.0018 0.0006 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000
1 0.0002 0.0021 0.0085 0.0183 0.0229 0.0173 0.0081 0.0023 0.0004 0.0000 0.0000 0.0000 0.0000 -
2 0.0009 0.0085 0.0299 0.0549 0.0578 0.0364 0.0139 0.0032 0.0004 0.0000 0.0000 0.0000 - -
3 0.0024 0.0183 0.0549 0.0847 0.0741 0.0381 0.0116 0.0020 0.0002 0.0000 0.0000 - - -
4 0.0035 0.0229 0.0578 0.0741 0.0530 0.0217 0.0050 0.0006 0.0000 0.0000 - - - -
5 0.0032 0.0173 0.0364 0.0381 0.0217 0.0068 0.0011 0.0001 0.0000 - - - - -
6 0.0018 0.0081 0.0139 0.0116 0.0050 0.0011 0.0001 0.0000 - - - - - -
7 0.0006 0.0023 0.0032 0.0020 0.0006 0.0001 0.0000 - - - - - - -
8 0.0001 0.0004 0.0004 0.0002 0.0000 0.0000 - - - - - - - -
9 0.0000 0.0000 0.0000 0.0000 0.0000 - - - - - - - - -
10 0.0000 0.0000 0.0000 0.0000 - - - - - - - - - -
11 0.0000 0.0000 0.0000 - - - - - - - - - - -
12 0.0000 0.0000 - - - - - - - - - - - -
13 0.0000 - - - - - - - - - - - - -
Bar graph: p(x,y)

0.0900
p(x,y)
0.0800

0.0700

0.0600

0.0500

0.0400

0.0300

0.0200
12
0.0100 9
6
-
3
0 1 2 3 4 5 6 7 8 0
9 10 11 12 13 y

x
Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
Now
p(x,y) = P[X = x, Y = y]
The possible values of X are 0, 1, 2, 3, 4, 5.
The possible values of Y are 0, 1, 2, 3, 4, 5.
and X + Y ≤ 5
 A typical outcome of rolling a die n = 5 times will be a sequence
F5FF6 where F denotes the outcome {1,2,3,4}. The probability of
any such sequence will be:

x y 5 x  y
1 1 4
     
6 6 6

where
x = the number of sixes in the sequence and
y = the number of fives in the sequence
 Now
p(x,y) = P[X = x, Y = y]
x y 5 x  y
1 1 4
K     
6 6 6
Where K = the number of sequences of length 5 containing x sixes
and y fives. 5 5 x 5 x y
   
    
 
x y  5  x  y 


5!   5  x ! 

5!
 x ! 5  x  ! 
 y ! 5  x  y  ! 
 x ! y ! 5  x  y  !
  
Thus
p(x,y) = P[X = x, Y = y]
x y 5 x  y
5! 1 1 4
      
x ! y ! 5  x  y  !  6   6   6 

if x + y ≤ 5 .
Table: p(x,y)
x y 5 x  y
5! 1 1 4
      
x ! y ! 5  x  y  !  6   6   6 

0 1 2 3 4 5
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001
1 0.1646 0.1646 0.0617 0.0103 0.0006 0
2 0.0823 0.0617 0.0154 0.0013 0 0
3 0.0206 0.0103 0.0013 0 0 0
4 0.0026 0.0006 0 0 0 0
5 0.0001 0 0 0 0 0
Bar graph: p(x,y)
p(x,y)

0.1800

0.1600

0.1400

0.1200

0.1000

0.0800

0.0600

0.0400
5
4
0.0200
3
0.0000 2
1
0
1 0
2
3
4
5
y

x
General properties of the joint probability function;
p(x,y) = P[X = x, Y = y]

1. 0  p  x, y   1

2.   p  x, y   1
x y

3. P     p  x, y 
 X , Y   A 
 x, y   A
Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
What is the probability that we roll more sixes than fives
i.e. what is P[X > Y]?
Table: p(x,y)
x y 5 x  y
5! 1 1 4
      
x ! y ! 5  x  y  !  6   6   6 
0 1 2 3 4 5
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001
1 0.1646 0.1646 0.0617 0.0103 0.0006 0
2 0.0823 0.0617 0.0154 0.0013 0 0
3 0.0206 0.0103 0.0013 0 0 0
4 0.0026 0.0006 0 0 0 0
5 0.0001 0 0 0 0 0

P  X  Y     p  x, y   0.3441
x y
Marginal And Conditional
Distributions
Definition:
Let X and Y denote two discrete random variables with joint
probability function
p(x,y) = P[X = x, Y = y]
Then
pX(x) = P[X = x] is called the marginal probability function of X.

and
pY(y) = P[Y = y] is called the marginal probability function of Y.
Note: Let y1, y2, y3, … denote the possible values of Y.
pX  x   P  X  x
 P X  x, Y  y1    X  x, Y  y2   

 P  X  x, Y  y1   P  X  x, Y  y2  

 p  x, y1   p  x, y2  

  p  x, y j    p  x, y 
j y
Thus the marginal probability function of X, pX(x) is obtained from the joint
probability function of X and Y by summing p(x,y) over the possible values
of Y.
Also
pY  y   P Y  y 
 P X  x1 , Y  y   X  x2 , Y  y  

 P  X  x1 , Y  y   P  X  x2 , Y  y  

 p  x1 , y   p  x2 , y  

  p  xi , y    p  x, y 
i x
Table: p(x,y)
x y 5 x  y
5! 1 1 4
      
x ! y ! 5  x  y  !  6   6   6 

0 1 2 3 4 5
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001
1 0.1646 0.1646 0.0617 0.0103 0.0006 0
2 0.0823 0.0617 0.0154 0.0013 0 0
3 0.0206 0.0103 0.0013 0 0 0
4 0.0026 0.0006 0 0 0 0
5 0.0001 0 0 0 0 0
Bar graph: p(x,y)
p(x,y)

0.1800

0.1600

0.1400

0.1200

0.1000

0.0800

0.0600

0.0400
5
4
0.0200
3
0.0000 2
1
0
1 0
2
3
4
5
y

x
 Note:
The possible values of X are 0, 1, 2, …, 13
The possible values of Y are also 0, 1, 2, …, 13 and X + Y ≤
13.
p  x, y   P  X  x , Y  y  The number of ways of
choosing the y hearts for
the hand
The number of ways of
choosing the x spades for
the hand
The number of ways of
 13   13   26  completing the hand with
    diamonds and clubs.
 x   y   13  x  y 

 52  The total number of ways of
  choosing the 13 cards for the
 13  hand
Table: p(x,y)
x y 5 x  y
5! 1 1 4
      
x ! y ! 5  x  y  !  6   6   6 

0 1 2 3 4 5
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001
1 0.1646 0.1646 0.0617 0.0103 0.0006 0
2 0.0823 0.0617 0.0154 0.0013 0 0
3 0.0206 0.0103 0.0013 0 0 0
4 0.0026 0.0006 0 0 0 0
5 0.0001 0 0 0 0 0
Bar graph: p(x,y)
p(x,y)

0.1800

0.1600

0.1400

0.1200

0.1000

0.0800

0.0600

0.0400
5
4
0.0200
3
0.0000 2
1
0
1 0
2
3
4
5
y

x
Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.

0 1 2 3 4 5 p X (x )
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.4019
1 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.4019
2 0.0823 0.0617 0.0154 0.0013 0 0 0.1608
3 0.0206 0.0103 0.0013 0 0 0 0.0322
4 0.0026 0.0006 0 0 0 0 0.0032
5 0.0001 0 0 0 0 0 0.0001
p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001
Conditional
Distributions
Definition:
Let X and Y denote two discrete random variables with joint
probability function
p(x,y) = P[X = x, Y = y]
Then
pX |Y(x|y) = P[X = x|Y = y] is called the conditional probability function
of X given Y = y

and
pY |X(y|x) = P[Y = y|X = x] is called the conditional probability
function of Y given X = x
Note
pX Y  x y   P 
 X  x Y  y
P  X  x, Y  y  p  x, y 
 
P Y  y  pY  y 
and
pY X  y x   P Y  y X  x

P  X  x, Y  y  p  x, y 
 
P  X  x pX  x
• Marginal distributions describe how one variable behaves ignoring the
other variable.
• Conditional distributions describe how one variable behaves when the
other variable is held fixed
Example:
A die is rolled n = 5 times
X = the number of times a “six” appears.
Y = the number of times a “five” appears.
y
0 1 2 3 4 5 p X (x )
0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.4019
1 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.4019
x 2 0.0823 0.0617 0.0154 0.0013 0 0 0.1608
3 0.0206 0.0103 0.0013 0 0 0 0.0322
4 0.0026 0.0006 0 0 0 0 0.0032
5 0.0001 0 0 0 0 0 0.0001
p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001
The conditional distribution of X given Y = y. pX |Y(x|y) = P[X = x|Y = y]
y
0 1 2 3 4 5 p X (x )
x 0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.4019
1 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.4019
2 0.0823 0.0617 0.0154 0.0013 0 0 0.1608
3 0.0206 0.0103 0.0013 0 0 0 0.0322
4 0.0026 0.0006 0 0 0 0 0.0032
5 0.0001 0 0 0 0 0 0.0001
p Y (y ) 0.4019 0.4019 0.1608 y
0.0322 0.0032 0.0001

0 1 2 3 4 5
0 0.3277 0.4096 0.5120 0.6400 0.8000 1.0000
1 0.4096 0.4096 0.3840 0.3200 0.2000 0.0000
x 2 0.2048 0.1536 0.0960 0.0400 0.0000 0.0000
3 0.0512 0.0256 0.0080 0.0000 0.0000 0.0000
4 0.0064 0.0016 0.0000 0.0000 0.0000 0.0000
5 0.0003 0.0000 0.0000 0.0000 0.0000 0.0000
 The conditional distribution of Y given X = x.

pY |X(y|x) = P[Y = y|X = x]

y
0 1 2 3 4 5
0 0.3277 0.4096 0.2048 0.0512 0.0064 0.0003
1 0.4096 0.4096 0.1536 0.0256 0.0016 0.0000
2 0.5120 0.3840 0.0960 0.0080 0.0000 0.0000
x 3 0.6400 0.3200 0.0400 0.0000 0.0000 0.0000
4 0.8000 0.2000 0.0000 0.0000 0.0000 0.0000
5 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000
The joint probability function;
p(x,y) = P[X = x, Y = y]

1. 0  p  x, y   1

2.   p  x, y   1
x y

3. P     p  x, y 
 X , Y   A 
 x, y   A
Elementary Statistics by Bluman
 Engineering Data Analysis

Joint Probability Distribution of

Two or More Continuous Random
Variables
MPS Department | FEU Institute of Technology
 Subtopic 2
OBJECTIVES

 Solve for the cumulative and probability density functions of functions

of random variables
 Find the density functions of random variables
 Subtopic 1
Joint Probability Distribution of Two or More Discrete
Random Variables

 Multiple Integration
 The Bivariate Normal Distribution
 Marginal Probability Distribution
 Conditional Probability Distribution of
More than Two Random Variables
Multiple Integration
 A  f  x, y  dxdy

f(x,y)
If the region A = {(x,y)| a ≤ x ≤ b, c ≤ y ≤ d} is a rectangular region with
sides parallel to the coordinate axes:
y
d

a b x
Then
A  f  x, y  dxdy
d
b  b
d 
f(x,y)    f  x, y  dx  dy  a  c f  x, y  dy  dx
c a  
To evaluate
d b

  f  x, y  dxdy    f  x, y  dxdy
c a
A
d b

   f  x, y  dx  dy
c a 
First evaluate the inner integral
b
G  y   f  x, y  dx
a

Then evaluate the outer integral

d b d

  f  x, y  dxdy   G  y  dy
c a c
 y
d
dy
y

a b x
b
G  y   f  x, y  dx = area under surface above the line where y is
a constant
d b d

  f  x, y  dxdy   G  y  dy
c a c
Infinitesimal volume under surface
above the line where y is constant
The same quantity can be calculated by integrating first with respect
to y, than x.
b d

  f  x, y  dxdy    f  x, y  dydx
a c
A
b d

   f  x, y  dy  dx
a c 
First evaluate the inner integral
d
H  x   f  x, y  dy
c

Then evaluate the outer integral

b d b

  f  x, y  dydx   H  x  dx
a c a
 y
dx
d

a x b x
d
H  x   f  x, y  dy = area under surface above the line where x is
c constant
b d b

  f  x, y  dydx   H  x  dx
a c a
Infinitesimal volume under surface
above the line where x is constant
Example: Compute
1 1 1 1

 x y  xy  dxdy     x y  xy  dydx

2 3 2 3

0 0 0 0
Now
1 1
 1 1

   
x y  xy dxdy     x y  xy dx  dy 
2 3 2 3

0 0 0 0 
1  3 2 x 1

x x
  y y3  dy
0  x 0 
3 2
 
4 y 1
1 1 3
1 2
1 y 1 y
   y  y  dy  
0  
3 2 3 2 2 4 y 0
1 1 7
  
6 8 24
The same quantity can be computed by reversing the order of
integration
1 1 1
1 
 x y  xy  
dydx     x y  xy dy  dx 
2 3 2 3

0 0 0 0 
1 y2 y 4 y 1

  x 
2
x  dx
0  y 0 
2 4
 

2 x 1
1 2 1 
1 3
1 x 1 x
   x  x  dx  
0 
2 4  2 3 4 2 x 0
1 1 7
  
6 8 24
Definition: Two random variable are said to have joint probability
density function f(x,y) if

1. 0 f  x, y 
 
2.   f  x, y  dxdy  1
 

3.  X , Y   A 
P    f  x, y  dxdy
A
Definition: Let X and Y denote two random variables with joint
probability density function f(x,y) then
the marginal density of X is


fX  x   f  x, y  dy


the marginal density of Y is


fY  y   f  x, y  dx

Definition: Let X and Y denote two random variables with joint
probability density function f(x,y) and marginal densities fX(x), fY(y)
then
the conditional density of Y given X = x
f  x, y 
fY X  y x  fX  x

conditional density of X given Y = y

fX Y  x y 
f  x, y 
fY  y 
The Bivariate Normal
Distribution
Let
1
1  Q  x1 , x2 
f  x1 , x2   e 2

 2   1 2 1  2

where

 x1  1 
2
 x1  1   x2   2   x2   2  
2

   2     

            
Q  x1 , x2  
1 1 2 2

1  2
This distribution is called the bivariate Normal
distribution.
The parameters are 1, 2 , 1, 2 and .
Surface Plots of the bivariate Normal
distribution
Note:
1
1  Q  x1 , x2 
f  x1 , x2   e 2

 2   1 2 1  2

is constant when

 x1  1 
2
 x1  1   x2   2   x2   2  
2

   2     

            
Q  x1 , x2  
1 1 2 2

1  2
is constant.
This is true when x1, x2 lie on an ellipse centered at 1, 2 .
Elementary Statistics by Bluman