CHAPTER IV: Joint and Conditional Probability

Distribution
Introduction
 So far we have focused on probability distributions for single random
variables(one dimensional case).
 However, we are often interested in probability statements concerning
two or more random variables.
Examples:
 Education and Earnings; Height and Longevity; Attendance
and Learning Outcomes; Sex-ratios and Areas Under Rice
Cultivation; income and expenditure; rainfall(R) and average
temperature(T) Genetic Make-Up and Disease etc.
 In this chapter, we will discuss(both for discrete and continuous) Joint,
Conditional, Marginal Distributions and Independency of random
variables.
4.1 Joint and Marginal Distributions
 Business and economic applications of statistics are often concerned about the
relationships between/among variables.
Illustrations:
 Products at different quality levels have different prices.
 Age groups have different preferences for clothing, for automobiles, and for
music.
 The percent returns on two different stocks may tend to be related (positive
relationship; negative relationship).
 When we work with probability models for problems involving relationships
between variables, it is important that the effect of these relationships is included
in the Probability Model.
 For example, assume that a car dealer is selling the following automobiles: (1) a red
two-door compact, (2) a blue minivan, and (3) a silver full-size sedan; the
probability distribution for purchasing would not be the same for women in their
20s, 30s, and 50s.
  Thus, it is important that probability models reflect the joint
effect of variables on probabilities.
Joint Probability: is probability of two or more independent
events occurring together or in succession.
 The joint probability of two or more independent events is
equal to the product of their marginal probabilities.
 In particular, if A and B are independent events, the
probability that both A and B will occur is given by:

Example 1: Suppose we toss a coin twice. The probability that in

both cases the coin will turn up head is given by:
 In general, if X and Y are two random variables, the probability distribution
that defines their simultaneous behavior is called a Joint Probability
Distribution.
 It is denoted as:

 If X and Y are discrete, this distribution can be described with a Joint

Probability Mass Function (Joint PMS).
 If X and Y are continuous, this distribution can be described with a Joint
Probability Density Function.
A) Joint and Marginal Distributions: Discrete
Examples (Joint discrete variables)
 Years in College Vs. Number of credits taken
 Number of Cigarettes smoked per day Vs. Day of the week
 Average temperature Vs. Day of the Year
 Number of students Vs. Number of professors in a College
Definition: Let X and Y be discrete random variables defined on the
sample space that take on values and respectively.
 Then, the Joint Probability Mass Function of is:

 The values give the probability/likelihood that outcomes X and

Y occur at the same time.

Example 1: CDs are covered with a rectangular plastic.

Measurements for the length and width are rounded to the nearest mm
(so they are discrete). Let X denote the length and Y denote the width.
 The possible values of X are 129, 130, and 131 mm while the
possible values of Y are 15 and 16 mm (Thus, both X and Y are
discrete).
 Thus, there are six possible areas (pairs of X and Y) of the plastic cover.
 The probability for each pair is shown in the following table:
X: length
Y: 129 130 131
Width 15 0.12 0.42 0.06
16 0.08 0.28 0.04

 From the table,

 The sum of all the probabilities is 1.0.
 The combination with the highest probability is (130, 15).
 The combination with the lowest probability is (131, 16).
 The joint probability mass function is the function
 For example, we have
Recall: A marginal or unconditional probability is the simple probability of
the occurrence of an event.
 Which is the probability of the occurrence of a single event i.e .
Examples: The probability a card drawn is red is and the probability of a card
drawn is a heart is .
 These are marginal Probabilities.
Marginal Probability Distribution: Discrete
Definition: Suppose that the discrete random variables X and Y have joint
PMF, Let denote the possible values of X, and denote the possible values of Y.
 The marginal probability mass functions (marginal PMF’S) of X and Y are
respectively given by:
 Thus, if we are given a joint PMF for X and Y, we can obtain the individual
probability distribution for X or for Y (and these are called the Marginal
Probability Distributions).
 Given the previous EXAMPLE,
i) Find the probability that a CD cover has length of 129 mm (i.e. ).
ii) What is the probability distribution of X?
Solution:

X: length Row
Y: 129 130 131 Sum
Width
15 0.12 0.42 0.06 0.60
16 0.08 0.28 0.04 0.40
ii. Then, the probability distribution for X appears in the column totals, given
as:
X 129 130 131
P( 0.20 0.70 0,10

 The same way, the probability distribution for Y appears in the raw
totals, given as:
Y 15 16
P( 0.60 0.40
 Because the PMF for X and Y appear in the margins of the table (i.e.
column and row totals), they are often referred to as the Marginal
Distributions for X and Y.
 We can also compute the Expected Values (i.e. and ) and variance of a
single random variable from the joint probability distribution
function.

Example 2: A fair coin is tossed three times independently: let X

denote the number of heads on the first toss and Y denote the total
number of heads.
 Find the joint probability mass function of X and Y .
Solution: the joint distribution of can be summarized in the following
table:
X|Y 0 1 2 3
0 1/8 2/8 1/8 0
1 0 1/8 2/8 1/8
 Suppose that we wish to find the PMF of Y from the joint PMF of X and
Y in the previous example:

 In general, to find the frequency function of Y, we simply sum down

the appropriate column of the table given the joint PMF of X and Y.
 For this reason, is called the marginal probability mass function of
Y.
 Similarly, summing across the appropriate rows gives,
which is the marginal PMF of X.
 For the above example, we have
Properties of Joint Discrete Probability Distribution
i.
ii. the sum of the joint probabilities over all possible pairs of
values must be 1, i.e.
iii. For any region A in the plane ,

Exercise 1: A basket contains two Red, three Black, and four White
balls.
 Let X be the number of chosen black balls and Y be number of chosen
red balls. If two balls are randomly selected from the basket(without
replacement),
A) Find the joint probability distribution function
B) The variance of X and Y.
Exercise 2: Given the joint PMF of two random variables as follow,
0 10 20
10 1/5 1/10 1/5
25 5/100 3/20 3/10
A) Find the probability that .
B) Find the probability that .
C) Find the marginal probability of X, and marginal probability of Y,
D) Compute the expected value and variances of the two random variables.
Exercise 3: Consider two random variable X and Y with joint PMF as given below.
10 15 20
0 1/4 1/6 1/8
1 1/8 1/6 1/6
A) Find the probability that and .
B) Find the probability that .
B) Joint and Marginal Densities: Continuous
 A bivariate function can serve as a joint probability density function
of a pair of continuous random variables X and Y if it values, , satisfy the
following two conditions:
i) for and

 If X and Y are continuous random variables and is the value of their joint
probability density at , the function given by:

is known as the marginal density of X.

 Correspondingly, the function given by:

is known as the marginal density of Y.

Example 1: Consider the following function: for and
a) Verify that it is joint PDF.
b) Find the marginal density function for X and Y.
Exercise 1: The bivariate random variables X & Y has the pdf:
, ,
A) find the value of the constant k
B) Compute the marginal density functions for X and Y

Exercise 2: Given the joint pdf: , ,

A) Compute the marginal density functions for X, and
B) Compute the marginal density functions for Y
4.2 Conditional Distributions and Independence
Recall: The probability of an event occurring, given that another event has
occurred is Conditional Probability:
 The conditional probability of event A, given that event B has already
occurred is written as: Similarly, we may write
 The vertical bar is read as: ‘given’ and events appearing to the right of
the bar are those that you know have occurred.
 Two events A and B are said to be independent if and only or Otherwise,
events are said to be dependent.
Example :
 In a toss of a coin twice, the : Independent events
 In a box containing 2 white and 2 black balls,
 , with replacement: Independent Events
 , without replacement: Dependent Events.
Conditional Distribution: The conditional probability distribution/density of
X given is defined as:
if
and the conditional probability distribution/density of Y given is defined as:
if
 Note that: This definition holds for both discrete and continuous random
variables. You can also notice the similarity between the conditional probability
and conditional probability distribution.
Example:
Age Group (X)
Purchase 1 2 3
Decision (Y) (16 to 20) (26 to 46) (46 to 65)
1(buy) 0.10 0.20 0.10 0.40
2(not buy) 0.25 0.25 0.10 0.60
0.35 0.45 0.20 1.00
 Statistical Independence: The condition of statistical independence
arises when the occurrence of one event has no effect upon the
probability of occurrence of any other event.
For example, in a fair coin toss the outcome of each toss is an event that is
statistically independent of the outcomes of every other toss of the coin.
Note: The probability computations under statistical dependent and
independent events is very different.
Under Statistical dependence

I. and
II. OR

III. and
 When the information that the random variable X takes a particular
value x is irrelevant to the determination of the probability that
another random variable Y takes, we say that Y is independent of
X.
 Formally, two random variables, X and Y, are said to be
independent if and only if any one of the following three conditions
holds:
i. for
ii. , for
iii. , for
Exercise 1: Given the joint PDFs in the previous exercises,
compute the conditional probability density functions for X given Y
and for Y given X.
4.3 Expectations
Mathematical Expectation
 If X and Y are two discrete random variables with joint PDF, then:
,

 Given
, for discrete random variable.
 The expected value for Y is also given in the same way.
 If X and Y are two continuous random variables with joint PDF, then:

 Given,
, for continuous random variable.
 The expected value for Y is also given in the same way.
 If X and Y are two random variables, then,

 If g is any function of X and Y, then,

 , if X and Y are discrete random variables.
 , if X and Y are continuous random variables.
Example: Given the PMF in the form of table as below, find,
, and

X|Y 0 1 2 3
0 1/8 2/6 1/8 0 4/8
1 0 2/8 1/8 1/8 4/8
1/8 3/8 3/8 1/8 1
Solution:



 The Variance,

 In terms of summation and integration notations:

 , for discrete random variable.
 , for continuous random variable.
 Note that the variance for Y, is also expressed in the same way, replacing Y in
place of X in the expressions above.

Exercise: Compute the variance for X and Y for the previous example.
Answer:
4.3.1 Covariance and Correlation
 An important measures when we have two random variables is that
of covariation and correlation of X and Y.
 The relationship between two variables can be represented by a
covariance and a correlation, which then suggests that there may
be a relationship that differs from 0.
Covariance: measures the amount of linear dependence between two
random variables.
 Is a measure of the joint variability of two variables.
 A positive covariance indicates that two random variables move in
the same direction, while a negative covariance indicates they
move in opposite directions.
 Interpreting the magnitude of a covariance can be a little tricky.
 It is defined as follows:

  , in the case of discrete. , in the case of continuous.
 Note the following:

Correlation: the degree of association between a dependent (response) variable Y

and two or more independent variables.
 is an alternative measure of dependence between X and Y that solves the “units”
problem of the covariance.
 Specifically, the correlation between X and Y is the covariance between X and Y
divided by their standard deviations, denoted by or is:

 This is called the coefficient of correlation, which measures linear relationship

between X and Y.
 Thus, Covariance measures the joint variability of two random variables
while Correlation measures the strength/the degree to which variables are
moving together.
Properties of the coefficient of correlation

1) and

Exercise 1: Find the covariance and coefficient of correlation of our

example under the mean for the discrete.
Answer:
Exercise 2: Given with ; , and ; and its results tabulated as follow:

X|Y 0 1 2
0 1/6 2/9 1/36 5/12
1 1/3 1/6 0 1/2
2 1/12 0 0 1/12
 Find the correlation
7/12coefficient7/18 1/36Y.
between X and 1
Answer:
 ;;;
 ;;;
 Note: The covariance ends up negative because of the restriction given by .
So that when X increases, Y must go down and thus, the negative relationship
between the two random variables.

4.3.2 Conditional Expectation

 Conditional expectation of X given is defined as:
 , if discrete.
 , if continuous.
 Similarly, the conditional expectation of Y given is defined as:
 , if discrete.
 , if continuous.
 Note that: The conditional expectation of Y given turns out to be the
function of X. That is, or is called the regression function of Y on X.
 It tells us how the mean of Y varies with changes in X.
Example: In our earlier example given as follows
X|Y 0 1 2 3
0 1/8 2/6 1/8 0 4/8
1 0 2/8 1/8 1/8 4/8
1/8 3/8 3/8 1/8 1
 Find,

Solution: a) Conditional probability distribution of X

Y
0
1
2
3
a)𝒇 (𝒙 |𝒀 =𝒚 ), Conditional Probability distribution of Y
X 𝒇 (𝒙 |𝒀 =𝒚 ) 𝒇 (𝒙 |𝒀 =𝒚 ) 𝒇 (𝒙 |𝒀 =𝒚 ) 𝒇 (𝒙 |𝒀 =𝒚 )
0

 Now, regression of Y on X is defined as . That is,

X
0 1
 In the1 form of function: 2
4.3.3 Independence and Expectation
 Let X and Y be two random variables with probability function , then
1) If X and Y are independent, then
Proof:

, as X and Y are independent

2) If X and Y are independent, then and

Proof:

 Note: If two random variables are independent, then they have zero correlation.
However, the converse is not necessarily true. That is, two random variables may
have zero correlation; it does not necessarily mean that they are independent.
Example: Let the joint PMF of two random variables X and Y be given as follows:
X|Y -1 0 1
-1 0 1/4 0 1/4
0 1/4 0 1/4 1/2
1 0 1/4 0 1/4
1/4 1/2 1/4 1
 ; and , therefore
 However, is not equal to .
 Note also that is not equal to ; but X and Y are dependent.

Y Y
-1 0.5 -1 1/4
0 0 0 1/2
1 0.5 1 1/4
1 1
  The sum of the conditional distribution should add up to unity.
 Similarly, the sum of the marginal distribution also adds up to one.
 If X and Y are independent for all X, but in the above example this does
not hold, therefore, the two variables are not independent.
 Relationship between Independence and
Uncorrelatedness
 Independence: Two random variables are independent when there
is no relationship of any form between the two RVs (X & Y).
 Uncorrelatedness: Two random variables are uncorrelated when
there is no relationship of a particular form between the two RVs (X
& Y).
 Thus, independent random variables are uncorrelated but
uncorrelated random variables may not be independent.
END!