Chapter 4

Joint and Conditional

Probability Distributions
1
 So far we have dealt with only a single random variable
X and its probability distribution/density.
 It is possible to extend the rv to more than one variables
at a time.
 For instance, we may observe the income (X) and
expenditure (Y) of a household. Thus, we observe a pair
of random variables (X, Y). If we add family size, we add
a 3rd variable Z and observe (X, Y, Z).
 Such case is known as multivariate (the intersection of
more than two events).
 The special case that involves only two random variables
(X & Y) is known as a bivariate (the intersection of two
events).

2
4.1 Joint and Marginal Distributions
4.1.1. Joint and Marginal Distributions: Discrete
Suppose X and Y are two discrete random variables
The probability that a random variable X assumes a given
value x and a random variable Y assumes a given value
y is known as the joint probability of x and y and is written
as f(x, y) = P(X = x, Y = y).
The domain is the set of ordered pairs (x, y), where x and y
are possible values for X and Y, respectively and the
range is the set of probability values corresponding to the
ordered pairs in its domain.
Properties of Joint Probability Function
1. f  x , y   0 , x , y
2.  f  x ,y   1
x y 3
2.The probability that a rv X will assume a given value x
whatever the value of a rv Y is known as the marginal
probability of x and the distribution is called the marginal
distribution of X. Mathematically,
f ( x)  P( X  x)  f ( xi , y1 )  f ( xi , y2 )  f ( xi , y3 )  ...
 f ( x )   f ( x, y )
y
 This implies that the marginal probability distributions of xi’s
are the horizontal summation of f(x, y).
 The probability that a Y will assume a given value y
whatever the value of a rv X is known as the marginal
probability of y and the distribution is called the marginal
distribution of Y. Mathematically,
f ( y )  P(Y  y )  f ( x1 , yi )  f ( x2 , yi )  f ( x3 , yi )  ...
 f ( y )   f ( x, y )
x 4
Using a Table

Note that:
 f ( x)   f ( y )   f ( x, y )  1
x y x y

5
4. The joint (bivariate) distribution function F(x, y) is given by F(x, y) =
P(X x; Y y). Mathematically,

x y x y
F ( x, y )    f u , v 
u  v 
   P U  u,V  v 
u  v 
Properties of bivariate cumulative Distribution Functions F(x, y):

1.

F (, y )  lim F ( x, y )  0 for all y, F ( x, )  lim F ( x, y)  0 for all x

x  y 
2.

lim F ( x, y )  1
x 
y 

6
 Examples
1. Throw two fair dice.
Let X = the outcome on the 1st die, and
Y = the outcome on the 2nd die.
Then we define 1
 , for x=1,2,...,6; y=1,2,...,6
f(x, y) = P(X = x, Y = y) =  36
0 , Otherwise

7
8
2: In the above exercise let
X = the number of 4's shown up
Y = the number of 1's shown up
Derive the joint probability distribution of X and Y.
 Note that:

1) The sample space consists of 36 points.

2) We can observe the number of 4's and 1's either 0 or 1
or 2 times in each case.
The random variable X = 0 for the following 25 points:
{(1,1), (1,2), (1,3), (1,5), (1,6), (2,1), (2,2), (2,3), (2,5),
(2,6), (3,1), (3,2), (3,3), (3,5), (3,6), (5,1), (5,2) (5,3),
(5,5), (5,6), (6,1), (6,2), (6,3), (6,5), (6,6)},

9
 X = 1 for the following 10 points {(1,4), (2,4), (3,4), (4,1),
(4,2), (4,3), (4,5), (4,6), (5,4), (6,4)}
 X = 2 is observed in only 1 point {(4, 4)}

Similarly,
 Y = 0 can be observed in 25 points {(2,2), (2,3), (2,4),
(2,5), (2,6),(3,2), (3,3), (3,4), (3,5), (3,6), (4,2), (4,3),
(4,4), (4,5), (4,6), (5,2), (5,3), (5,4), (5,5), (5,6), (6,2),
(6,3), (6,4), (6,5), (6,6)},
 Y = 1 for the following 10 points {(1,2), (1,3), (1,4), (1,5),
(1,6), (2,1), (3,1), (4,1), (5,1), (6,1)}, and
 Y = 2 is observed in only 1 point {(1, 1)}

10
 Intersections of X and Y
a) The intersection of X = 0 and Y = 0, has the following 16 points
{ (2,2), (2,3), (2,5), (2,6), (3,2), (3,3), (3,5), (3,6), (5,2) (5,3),
(5,5), (5,6), (6,2), (6,3), (6,5), (6,6)}
b) The intersection of X = 1 and Y = 0, has the following 8 points
{(2, 4), (3, 4), (4, 2), (4, 3), (4, 5), (4, 6), (5, 4), (6, 4)}
c) The intersection of X = 2 and Y = 0 has the following 1 points
{(4, 4)}
d) The intersection of X = 0 and Y = 1, has the following 8 points
{(1, 2), (1, 3), (1, 5), (1, 6), (2, 1), (3,1), (5,1), (6,1)}
e) The intersection of X = 1 and Y = 1, has the following 2 points
{(1, 4), (4, 1)},

11
f) The intersection of X = 0 and Y = 2, has the following 1
point {(1, 1)} and
g) Note that when X = 2, Y cannot take values greater than 0
and vice versa, also when X = 1, Y cannot take values
greater than 1 and vice versa.
These results can be easily presented using the following
table:

12
3. A box contains 3 black, 2 red and 4 white balls. Two
balls are selected at random without replacement.
 Let the rv X denote number of black balls in the sample of 2 balls
selected
 Let the rv Y denote number of red balls in the sample of 2 balls
selected
 since there are nine balls in all and we are selecting two balls, there
are  9  9! 98
     36
ways of doing so, and
 2  (9  2)!2! 2
 3
 
 x
 ways of selecting x black balls out of the 3 black balls,
 2
 
 y
ways of selecting y red balls out of the 2 red balls, and


given that 2 - x - y is the number 
4of white balls that can be selected,
 
2 x  y
we can select white balls in ways. 13
 The joint (bivariate) probability distribution for X and Y is
given as
f  x , y   P  X  x ,Y  y 
3  2  4 
      
x   y   2 x y 
 , x  0 ,1 , 2 ; y  0 ,1 , 2 ; &x y  2
 9
  
 2
 0, otherwise


 This is a bivariate hypergeometric distribution.

 3 2 4  3! 2! 4!
     
0 0 2  0  0  3!0! 2!0! 2!2! 1 1 6 6 1
f (0, 0)          
9 36 36 36 6
 
 2
14
Cont.

15
 This can be represented in tabular form as follows:

 From the above examples we note the following properties

16
 We also observe that, for the last example
P(X = 0) = f (0) = 15/36
P(X = 1) = f (1) = 18/36
P(X = 2) = f (2) = 3/36
These are the sums of the 3 rows.
Thus, we have
This is the marginal probability distribution of X.

Similarly, if we sum the three columns we get the marginal

probability distribution of Y as follows:

17
4. Find the value of k in the following joint probability
distribution:

Solution: Consider the pairs: (-1, -1), (-1, 2), (-1, 3), (0, -1),
(0, 2), (0, 3), (1, -1), (1, 2), (1, 3), (3, -1), (3, 2), (3, 3)

18
b. find the p(X+Y>2). Solution: The pairs are: (0, 3), (1, 2), (1, 3), (3, 2),
(3, 3) Then,

c. Find F(1, 2)= P(X≤ 1, Y≤2)

19
4.1.2. Joint and Marginal Densities: Continuous
1. A bivariate function can serve as a joint probability density
function of a pair of continuous random variables X and Y if
it values, f(x, y), satisfy the following two conditions:

2. If X and Y are continuous random variables and f(x, y) is the

value of their joint probability density at (x, y), the function given
by:
is known as the
marginal density of X.
Correspondingly, the function given by
is known as the marginal density of Y.
20
 If X and Y are continuous random variables, the function
given by:

 Where is the value of the joint probability density of X and Y

at (s, t), is known as the joint distribution function of X
and Y.
 Note that:

21
Examples:
1. Consider the following function:

a. Verify that it is joint pdf

b. Find the marginal density function for X and Y
c. Find the joint distribution function

22
 Conditional Distribution and Statistical
Independence
 Conditional distribution: The conditional probability
distribution/density of X given Y = y is defined as

 and the conditional probability distribution/density of Y

given X = x is defined as

23
 Examples
1. The bivariate random variables X & Y has the pdf:

a) Find the value of the constant k

b) Compute the marginal density functions for X and Y
c) Calculate the conditional probability density functions
for X given Y and Y given X

24
2. Given the joint pdf:

a) Compute the marginal density functions for X and Y

b) Calculate the conditional probability density
functions for X given Y and Y given X

25
Statistical independence
Formally, two random variables, X and Y, are said to be
independent if and only if any one of the following three
conditions holds:

Examples

Are X and Y independent?

26
 Expectation and Variance
4.3.1. Mathematical Expectation
 If X and Y are two discrete random variables with joint
probability function f(x,y), then:

27
b. If X and Y are two continuous random variables with joint
probability density function f(x,y), then

28
 If g(x, y) is any function of X and Y, then

Examples:
 1. Given the pdf in the form of table:

 Find the expected value of X and Y 29

30
31
 In terms of summation and integration notations:

32
Find the variance of X

Find the variance of Y

33
4.4. Covariance and Correlation
 An important measure when we have two random
variables is that of covariation of X and Y. It is defined as
follows:

 In the case where the two random variables are

continuous, the covariance between the two random
variables can be evaluated as follows:

34
 Note the following:

 The coefficient of correlation denoted by  or x,y

It measures linear relationship between X and Y

35
Properties of the coefficient of correlation
1. -1   1.
2. Cov(X, Y) = Cov(Y, X), and
3. x,y = yx
Example: Find the covariance and correlation coefficient of
our example under the mean for the discrete.

36
37
Solution
1. E(X) = 0x5/12+1 x ½+2 x 1/12 = 2/3
2. E(Y) = 0 x 7/12+1 x 7/18+2 x 1/36 = 4/9
3. E(X2) = 0 x 5/12+1 x ½+4 x 1/12 = 5/6
4. E(Y2) = 0 x 7/12+1 x 7/18+4 x 1/36 = ½
5. X2 =E(X2) - E(X)2 =5/6 - (2/3)2 = 7/18
6. Y2 =E(Y2) - E(Y)2 = ½ - (4/9)2 = 49/162
7. E(XY) = 0 x 0 x 1/6+0 x 1 x 2/9+0 x 2 x 1/36+1 x 0 x 1/3+1
x 1 x 1/6+... = 1/6
8. Cov(X,Y) = E(XY) - E(X) x E(Y) = 1/6 - 2/3 x 4/9 = -7/54
 Note: The covariance ends up negative because of the
restriction given by x + y  2.
Cov( x, y ) 7 / 54 7 / 54 1
 XY    
 XY 7 /18  49 /162 7 / 54 7 7
38
 4.4. Conditional Expectation
Conditional expectation of X given Y=y is defined as

Conditional expectation of Y given X=x is defined as

39
 Note that: The conditional expectation of Y given X = x
turns out to be the function of x. That is, E(Y/X=x) =
m(x), or is called the regression function of Y on X. It
tells us how the mean of Y varies with changes in X.
 Example: In our earlier example given as follows

40
 The problem is now to
a) Find f(y|X = x)

b). Find f(x|Y = y)

41
 Now, regression of Y on X is defined as m( x)  E (Y | X  x)
that is,

42
HW

The results of this exercise were tabulated as

43
The problem is now to
a) Find f(y|X = x)

b) Find f(x|Y = y)

44
Regression of Y on X is defined as m( x)  E (Y | X  x) . Thus,

45
 This can be tabulated as

 This could also be graphed as:

 As a result the slope of the regression of Y on X is linear

with the slope of -1/3.

46
4.5. Independence and Expectation
Let X and Y be two random variables with probability function
f(x, y), then
1. If X and Y are independent, then
 Proof:

47
 If X and Y are two independent random variables, then
Cov (X, Y)=0 and x,,y= 0
Proof:

 If two random variables are independence, then they

have zero correlation. However, the converse is not
necessarily true.
 That is, two random variables may have zero correlation;
it does not necessarily mean that they are independent.

48
Example: Let the joint probability distribution of two random
variables X and Y be given as follows

E(X)= 0 E(Y) = 0 and E(XY) = 0 therefore Cov(X,Y) = 0

However, P(X=-1,Y =-1) = f(-1,-1)=0 is not equal to

P(X=-1). P(Y =-1) = 1/4.1/4 = 1/16.

49
Note also that f(y/X=x) is not equal to f(y)

 If X and Y are independent f(y/X=x) = f(y) for all X, but in

the above example this does not hold, therefore, the two
variables are not independent.

50
4.6. Sums of random variables
1. Let X and Y be two random variables and define a new
random variable Z as Z = X +Y
Proposition:
Var(Z) = Var(X) + Var(Y) + 2Cov(X, Y)
Proof:

51
2. Let X and Y be two random variables and define a new
random variable Z as Z = X + Y where  and  are
constants.
Proposition:
Var(Z) = 2Var(X) + 2Var(Y) + 2Cov(X,Y)
Proof:

52