University of Benghazi

Faculty of Engineering
Electrical and Electronics Engineering Department

Probability and (Stocastic) Random Process Course

EE277

Salma Elkawafi
[email protected]
Goals
Understand the following:
Jointly Distributed Random Variables
• Distributions functions (PMF, PDF) (Discrete and Continuous)

• Cumulative distribution function (cdf): (Continuous and Discrete)

• Marginal distributions

• Conditional distributions

• Independent random variables

• Expectation, Covariance, Correlation

This is quite similar to what we had before, but now we are jointly
describing the two random variables.
 Joint Probability Distributions
If 𝑋 and 𝑌 are two random variables, the probability distribution for their simultaneous
occurrence can be represented by a function with values 𝑝 𝑥, 𝑦 or 𝑓(𝑥, 𝑦) for any pair of
values (𝑥, 𝑦) within the range of the random variables 𝑋 and 𝑌 . It is customary to refer to
this function as the joint probability distribution of X and Y .

The function 𝑝(𝑥, 𝑦) is a joint probability The function 𝑓(𝑥, 𝑦) is a joint density
distribution or probability mass function of function of the continuous random
the discrete random variables 𝑋 and 𝑌 if: variables 𝑋 and 𝑌 if:

1. 𝑝 𝑥, 𝑦 ≥ 0 for all (𝑥, 𝑦) 1. 𝑓 𝑥, 𝑦 ≥ 0 for all (𝑥, 𝑦)

∞ ∞
2. σ𝑥 σ𝑦 𝑝(𝑥, 𝑦) = 1 2. ‫׬‬−∞ ‫׬‬−∞ 𝑓(𝑥, 𝑦) = 1
3. 𝑃 𝑋 = 𝑥, 𝑌 = 𝑦 = 𝑝 𝑥, 𝑦 = 𝑃𝑋𝑌 (𝑥, 𝑦) 3. 𝑃 𝑋, 𝑌 ∈ 𝐴 = ‫𝑥 𝑓 𝐴׬ ׬‬, 𝑦 𝑑𝑥 𝑑𝑦 ,
4. For any region A in the 𝑥𝑦 plane, for any region A in the 𝑥𝑦 plane.
𝑃[(𝑋, 𝑌 ) ∈ 𝐴] = σ σ𝐴 𝑝(𝑥, 𝑦)
Joint Probability Distributions
Example: Two ballpoint pens are selected at random from a box that contains 3 blue pens, 2 red pens, and 3 green
pens. If 𝑋 is the number of blue pens selected and Y is the number of red pens selected, find
(a) the joint probability function 𝑝(𝑥, 𝑦),
(b) 𝑃[(𝑋, 𝑌 ) ∈ 𝐴], where A is the region {(𝑥, 𝑦)|𝑥 + 𝑦 ≤ 1}.
Solution:
The possible pairs of values (x, y) are (0, 0), (0, 1), (1, 0), (1, 1), (0, 2), and (2, 0).
𝐶03 𝐶02 𝐶23
𝑝 0,0 =
𝐶28
𝐶03 𝐶12 𝐶13 𝑝(𝑥, 𝑦)
𝑝 0,1 =
𝐶28
𝐶13 𝐶02 𝐶13
𝑝 1,0 =
𝐶28
𝐶13 𝐶12 𝐶03
𝑝 1,1 =
𝐶28
𝐶23 𝐶02 𝐶03
𝑝 2,0 =
𝐶28
𝐶03 𝐶22 𝐶03
𝑝 0,2 =
𝐶28
Joint Probability Distributions
Example: Two ballpoint pens are selected at random from a box that contains 3 blue
pens, 2 red pens, and 3 green pens. If 𝑋 is the number of blue pens selected and Y is the
number of red pens selected, find
(a) the joint probability function 𝑝(𝑥, 𝑦),
(b) 𝑃[(𝑋, 𝑌 ) ∈ 𝐴], where A is the region {(𝑥, 𝑦)|𝑥 + 𝑦 ≤ 1}.
Solution:
The possible pairs of values (x, y) are (0, 0), (0, 1), (1, 0), (1, 1), (0, 2), and (2, 0).
3
𝐶𝑥3 𝐶𝑦2 𝐶2−𝑥−𝑦
𝑝 𝑥, 𝑦 = 𝑓𝑜𝑟 𝑥, 𝑦 = 0,1,2, 𝑎𝑛𝑑 0 ≤ 𝑥 + 𝑦 ≤ 2
𝐶28

𝑝(𝑥, 𝑦)
b) 𝑃 𝑋, 𝑌 ∈ 𝐴 = 𝑃 𝑋 + 𝑌 ≤ 1
8
= 𝑝 0, 0 + 𝑝 0, 1 + 𝑝 1, 0 =
14
Joint Probability Distributions
Example :A privately owned business operates both a drive-in facility and a walk-in facility. On a randomly
selected day, let 𝑋 and 𝑌 , respectively, be the proportions of the time that the drive-in and the walk-in facilities are
in use, and suppose that the joint density function of these random variables is
2
𝑓(𝑥, 𝑦) = ቐ5 2𝑥 + 3𝑦 , 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
∞ ∞
(a) Verify that ‫׬‬−∞ ‫׬‬−∞ 𝑓(𝑥, 𝑦) = 1.
1 1 1
(b) Find 𝑃[(𝑋, 𝑌 ) ∈ 𝐴], where 𝐴 = 𝑥, 𝑦 0 < 𝑥 < , < 𝑦 < }
2 4 2
Solution:
Joint Probability Distributions
Example :A privately owned business operates both a drive-in facility and a walk-in facility. On a randomly selected day, let 𝑋 and 𝑌 , respectively,
be the proportions of the time that the drive-in and the walk-in facilities are in use, and suppose that the joint density function of these random
variables is
2
𝑓(𝑥, 𝑦) = ቐ5 2𝑥 + 3𝑦 , 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
∞ ∞
(a) Verify that ‫׬‬−∞ ‫׬‬−∞ 𝑓(𝑥, 𝑦) = 1.
1 1 1
(b) Find 𝑃[(𝑋, 𝑌 ) ∈ 𝐴], where 𝐴 = 𝑥, 𝑦 0 < 𝑥 < 2 , 4 < 𝑦 < 2}

Solution:
 Joint Probability Distributions
Cumulative distribution function (cdf):
The joint behavior of two random variables 𝑋 and 𝑌 is determined by the joint cumulative
distribution function (cdf):
𝐹 𝑥, 𝑦 = 𝑃 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦 = 𝑃((𝑋 ≤ 𝑥) ∩ (𝑌 ≤ 𝑦))
where 𝑋 and 𝑌 are continuous or discrete.
The joint cumulative function of the The joint cumulative function of the
discrete random variables 𝑋 and 𝑌with continuous random variables 𝑋 and 𝑌
joint pmf 𝑝(𝑥𝑖 , 𝑦𝑗 ) : with joint density 𝑓(𝑥, 𝑦):
𝑦 𝑥
𝐹 𝑥, 𝑦 = ෍ ෍ 𝑝(𝑥𝑖 , 𝑦𝑖 ) 𝐹 𝑥, 𝑦 = න න 𝑓 𝑢, 𝑣 𝑑𝑢𝑑𝑣
𝑥𝑖 ≤𝑥 𝑦𝑖 ≤𝑦 −∞ −∞

The probability that (𝑋, 𝑌 ) belongs to a given rectangle is

𝑃(𝑥1 < 𝑋 ≤ 𝑥2 , 𝑦1 < 𝑌 ≤ 𝑦2 ) = 𝐹(𝑥2 , 𝑦2 ) − 𝐹(𝑥2 , 𝑦1 ) − 𝐹(𝑥1 , 𝑦2 ) + 𝐹(𝑥1 , 𝑦1 ).
Joint Probability Distributions
Examples
Suppose 𝑋 and 𝑌 both take values in [0,1] with density 𝑓(𝑥, 𝑦) = 4𝑥𝑦. Find the
joint cdf for the random variables.

Solution:
𝑦 𝑥 𝑦 𝑥
𝐹 𝑥, 𝑦 = න න 𝑓 𝑢, 𝑣 𝑑𝑢𝑑𝑣 = න න 4𝑢𝑣 𝑑𝑢𝑑𝑣 = 𝑥 2 𝑦 2
−∞ −∞ 0 0
Joint Probability Distributions
Example:
Roll two dice. Let 𝑋 be the value on the first die and let 𝑌 be the value on the second die. Then both
1
𝑋 and 𝑌 take values 1 to 6 and the joint pmf is 𝑝 𝑖, 𝑗 = for all 𝑖 and 𝑗 between 1 and 6. the joint
36
probability table below , compute 𝐹(3.5, 4).
Solution

𝐹 𝑥, 𝑦 = ෍ ෍ 𝑝(𝑥𝑖 , 𝑦𝑖 )
𝑥𝑖 ≤𝑥 𝑦𝑖 ≤𝑦
12 1
𝐹 3.5, 4 = 𝑃 𝑋 ≤ 3.5 ∩ 𝑌 ≤ 4 = =
36 3
Joint Probability Distributions
Marginal distributions
Given the joint probability distribution 𝑝(𝑥, 𝑦)/𝑓(𝑥, 𝑦) of the random variables 𝑋 and 𝑌 , the
probability distribution 𝑔(𝑥) of 𝑋 alone is obtained by summing 𝑝(𝑥, 𝑦) / integrating 𝑓(𝑥, 𝑦) over
the values of 𝑌 . Similarly, the probability distribution ℎ(𝑦) of 𝑌 alone is obtained by summing
𝑝(𝑥, 𝑦) / integrating 𝑓(𝑥, 𝑦) over the values of 𝑋. We define 𝑔(𝑥) and ℎ(𝑦) to be the marginal
distributions of X and Y , respectively.

The marginal distribution for discrete The marginal distribution for continuous
case of 𝑋 alone and of 𝑌 alone are: case of 𝑋 alone and of 𝑌 alone are:
∞
𝑔 𝑥 = ෍ 𝑝(𝑥, 𝑦) 𝑔 𝑥 = න 𝑓 𝑥, 𝑦 𝑑𝑦
𝑦 −∞

ℎ 𝑦 = ෍ 𝑝(𝑥, 𝑦) ℎ 𝑦 = න 𝑓 𝑥, 𝑦 𝑑𝑥
−∞
𝑥
 Two ballpoint pens are selected at random from

Joint Probability Distributions a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
selected and Y is the number of red pens
Marginal distributions selected, find
Example:
Show that the column and row totals of give the marginal distribution of X alone and of Y alone.
Solution:
𝑔 𝑥 = ෍ 𝑝(𝑥, 𝑦) , ℎ 𝑦 = ෍ 𝑝(𝑥, 𝑦) 𝑝(𝑥, 𝑦)
𝑦 𝑥
3 3 1 5
𝑔 0 = 𝑝 0, 0 + 𝑝 0, 1 + 𝑝 0, 2 = + + = ,
28 14 28 14
9 3 15
𝑔 1 = 𝑝 1, 0 + 𝑝 1, 1 + 𝑝 1, 2 = 𝑔 1 = + + 0 =
28 14 28
3 3
𝑔(2) = 𝑝(2, 0) + 𝑝(2, 1) + 𝑝(2, 2) = + 0 + 0 =
28 28
In a similar manner we could show that the values of ℎ(𝑦) are given by the row totals. In tabular form, these
marginal distributions may be written as follows:
Joint Probability Distributions
Marginal distributions
Example:
Find 𝑔(𝑥) and ℎ(𝑦) for the joint density function of the previous Example
2
𝑓(𝑥, 𝑦) = ቐ5 2𝑥 + 3𝑦 , 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
Solution:
∞ 1
2 4𝑥𝑦 6𝑦 2 𝑦=1 4𝑥 + 3
𝑔 𝑥 = න 𝑓 𝑥, 𝑦 𝑑𝑦 = න 2𝑥 + 3𝑦 𝑑𝑦 = + ቚ = ,
−∞ 0 5 5 10 𝑦=0 5
For 0 ≤ 𝑥 ≤ 1 and 𝑔 𝑥 = 0 elsewhere

∞ 1
2 2(1 + 3𝑦)
ℎ 𝑦 = න 𝑓 𝑥, 𝑦 𝑑𝑥 = න 2𝑥 + 3𝑦 𝑑𝑥 =
−∞ 0 5 5

For 0 ≤ 𝑦 ≤ 1 and ℎ 𝑦 = 0 elsewhere

Joint Probability Distributions
Conditional probability distribution;

Let X and Y be two random variables, discrete or continuous. The conditional distribution
of the random variable 𝑌 given that 𝑋 = 𝑥 is
𝑓 𝑥,𝑦
𝑓 𝑦𝑥 = , provided 𝑔(𝑥) > 0.
𝑔 𝑥
Similarly, the conditional distribution of 𝑋 given that 𝑌 = 𝑦 is
𝑓 𝑥,𝑦
𝑓 𝑥𝑦 = ℎ 𝑦
, provided ℎ(𝑦) > 0.

The probability that the discrete random The probability that the continuous random
variable 𝑋 falls between 𝑎 and 𝑏 when it is variable 𝑋 falls between 𝑎 and 𝑏 when it is
known that the discrete variable 𝑌 = 𝑦, known that the continuous variable 𝑌 = 𝑦,
𝑏
𝑃(𝑎 < 𝑋 < 𝑏 | 𝑌 = 𝑦) = ෍ 𝑝 𝑥 𝑦 𝑃(𝑎 < 𝑋 < 𝑏 | 𝑌 = 𝑦) = න 𝑓 𝑥 𝑦 𝑑𝑥
𝑎<𝑥<𝑏 𝑎
 Two ballpoint pens are selected at random from

Joint Probability Distributions a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
selected and Y is the number of red pens
Conditional probability distribution; selected, find
Example:
Referring to the previous Example, find the conditional distribution of 𝑋, given that 𝑌 = 1, and use it
to determine 𝑃(𝑋 = 0 | 𝑌 = 1).
Solution: 𝑝(𝑥, 𝑦)
𝑝 𝑥,𝑦
𝑝 𝑥𝑦 = , provided ℎ(𝑦) > 0
ℎ 𝑦
2
3 3 3
ℎ 1 = 𝑃 𝑌 = 1 = ෍ 𝑝 𝑥, 1 = + +0=
14 14 7
𝑥=0
𝑝 𝑥, 1 7
𝑝 𝑥1 = = 𝑝 𝑥, 1 , 𝑥 = 0,1,2
ℎ 1 3
7 7 3 1
𝑝 01 = 𝑝 0,1 = ∗ =
3 3 14 2
7 7 3 1
𝑝 11 = 𝑝 1,1 = ∗ =
3 3 14 2
7 7
𝑝 21 = 𝑝 2,1 = ∗ 0 = 0
3 3
 Two ballpoint pens are selected at random from

Joint Probability Distributions a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
selected and Y is the number of red pens
Conditional probability distribution; selected, find
Example: (Cont.)
Referring to the previous Example, find the conditional distribution of 𝑋, given that 𝑌 = 1, and use it
to determine 𝑃(𝑋 = 0 | 𝑌 = 1).
Solution: 𝑝(𝑥, 𝑦)

the conditional distribution of X, given that Y = 1, is

𝑥 0 1 2
𝑝(𝑥|1) ½ ½ 0

Finally,
1
𝑃 𝑋=0𝑌=1 =
2
if it is known that 1 of the 2 pen refills selected is red, we have a probability equal to 1/2 that the
other refill is not blue.
Joint Probability Distributions
Conditional probability distribution;
Example:
10𝑥𝑦 2 , 0<𝑥<𝑦<1
The joint density for the random variables (𝑋, 𝑌 ), is 𝑓(𝑥, 𝑦) = ቊ
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
where 𝑋 is the unit temperature change and 𝑌 is the proportion of spectrum shift that a certain
atomic particle produces,
(a) Find the marginal densities 𝑔(𝑥), ℎ(𝑦), and the conditional density 𝑓(𝑦|𝑥).
(b) Find the probability that the spectrum shifts more than half of the total observations, given that
the temperature is increased by 0.25 unit.
Solution:
(a) 𝑔 𝑥 = ‫׬‬−∞ 𝑓 𝑥, 𝑦 𝑑𝑦 = ‫ 𝑥׬‬10𝑥𝑦 2 𝑑𝑦 = 3 𝑥𝑦 3 |𝑦=𝑥
∞ 1 10 𝑦=1 10
= 𝑥 1 − 𝑥3 , 0 < 𝑥 < 1
3
∞ 𝑦 𝑥=𝑦
ℎ 𝑦 = න 𝑓 𝑥, 𝑦 𝑑𝑥 = න 10𝑥𝑦 2 𝑑𝑥 = 5𝑥 2 𝑦 2 ቚ = 5𝑦 4 , 0<𝑦<1
−∞ 0 𝑥=0

𝑓(𝑥, 𝑦) 10𝑥𝑦 2 3𝑦 2
𝑓 𝑦𝑥 = = = ,0 < 𝑥 < 𝑦 < 1
𝑔(𝑥) 10 3 1 − 𝑥 3
𝑥 1−𝑥
3
Joint Probability Distributions
Conditional probability distribution;
Example: (Cont.)
10𝑥𝑦 2 , 0<𝑥<𝑦<1
The joint density for the random variables (𝑋, 𝑌 ), is 𝑓(𝑥, 𝑦) = ቊ
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
where 𝑋 is the unit temperature change and 𝑌 is the proportion of spectrum shift that a certain
atomic particle produces,
(a) Find the marginal densities 𝑔(𝑥), ℎ(𝑦), and the conditional density 𝑓(𝑦|𝑥).
(b) Find the probability that the spectrum shifts more than half of the total observations, given that
the temperature is increased by 0.25 unit.
Solution:
3𝑦 2
𝑓 𝑦𝑥 = ,0 < 𝑥 < 𝑦 < 1
1 − 𝑥3

1 1 1 3𝑦 2 8
(b) 𝑃 𝑌 > 𝑋 = 0.25 ) = ‫׬‬1 𝑓 𝑦|𝑥 = 0.25 𝑑𝑦 = ‫׬‬1 1−0.253 𝑑𝑦 =9
2 2 2
Joint Probability Distributions
Statistically independent
Let 𝑋 and 𝑌 be two random variables, discrete or continuous, with joint probability
distribution 𝑓(𝑥, 𝑦) and marginal distributions 𝑔(𝑥) and ℎ(𝑦), respectively. The random
variables 𝑋 and 𝑌 are said to be statistically independent if and only if
𝑓 𝑥, 𝑦 = 𝑔 𝑥 ℎ 𝑦 = 𝑃{𝑋 ∈ 𝐴, 𝑌 ∈ 𝐵} = 𝑃{𝑋 ∈ 𝐴}𝑃{𝑌 ∈ 𝐵}
Or 𝑓 𝑦 𝑥 = ℎ 𝑦 , and 𝑓 𝑥 𝑦 = 𝑔 𝑥 ,

for all (𝑥, 𝑦) within their range.

Properties
𝑃 𝑋 ≤ 𝑎, 𝑌 ≤ 𝑏 = 𝑃 𝑋 ≤ 𝑎 𝑃 𝑌 ≤ 𝑏
𝐹(𝑎, 𝑏) = 𝐹(𝑎)𝐹(𝑏) 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑎, 𝑏
Joint Probability Distributions
Two ballpoint pens are selected at random from
a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
Statistically independent selected and Y is the number of red pens
selected, find
Example:
Show that the random variables of the previous Example are not independent
Solution:

Consider the point (0, 1). From the table we find the three probabilities 𝑝(0, 1), 𝑔(0), and ℎ(1) to be

3
𝑝 0, 1 =
14 𝑝(𝑥, 𝑦)

5
𝑔 0 =
14

3
ℎ 1 =
7

𝑝 0, 1 ≠ 𝑔 0 ℎ(1)
Joint Probability Distributions
Joint Probability Distributions

All the concepts we defined can also be

generalized easily to more than two random
variables.
 Joint Probability Distributions
Expected Value
Let 𝑋 and 𝑌 be random variables with joint probability distribution 𝑝(𝑥, 𝑦) for discrete and 𝑓(𝑥, 𝑦)
for continues random variables. The mean, or expected value, of the random variable 𝑔(𝑋, 𝑌 ) is
∞ ∞
𝜇𝑔 𝑋, 𝑌 = 𝐸[𝑔(𝑋, 𝑌)] = ෍ ෍ 𝑔 𝑥, 𝑦 𝑝(𝑥, 𝑦) 𝜇𝑔 𝑋, 𝑌 = 𝐸[𝑔(𝑋, 𝑌)] = න න 𝑔(𝑥, 𝑦)𝑓(𝑥, 𝑦) 𝑑𝑥𝑑𝑦
−∞ −∞
𝑥 𝑦

if 𝑋 and 𝑌 are continuous.

if 𝑋 and 𝑌 are discrete,
∞ ∞
𝐸[𝑋] = න න 𝑥 𝑓(𝑥, 𝑦) 𝑑𝑥𝑑𝑦
𝐸[𝑋] = ෍ ෍ 𝑥𝑝(𝑥, 𝑦) −∞ −∞
∞ ∞
𝑥 𝑦
𝐸[𝑌] = න න 𝑦 𝑓(𝑥, 𝑦) 𝑑𝑥𝑑𝑦
𝐸[𝑌] = ෍ ෍ 𝑦 𝑝(𝑥, 𝑦) −∞ −∞
𝑥 𝑦
Joint Probability Distributions
Two ballpoint pens are selected at random from
a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
selected and Y is the number of red pens
selected, find

Expected Value
Example:
Find the expected value of 𝑔(𝑋, 𝑌 ) = 𝑋𝑌 .
𝑝(𝑥, 𝑦)
Solution:

𝜇𝑔 𝑋, 𝑌 = 𝐸[𝑔(𝑋, 𝑌)] = ෍ ෍ 𝑔 𝑥, 𝑦 𝑝(𝑥, 𝑦)

𝑥 𝑦
2 2
3
𝐸 𝑋𝑌 = ෍ ෍ 𝑥𝑦𝑝 𝑥, 𝑦 = 𝑝 1,1 =
14
𝑥=0 𝑦=0
Joint Probability Distributions
Covariance (Cov(X,Y))
Let 𝑋 and 𝑌 be random variables with joint probability distribution 𝑝(𝑥, 𝑦) for discrete and 𝑓(𝑥, 𝑦) for
continues random variables. The covariance, of the random variable 𝑔(𝑋, 𝑌 ) is
𝜎𝑋𝑌 = 𝐸[(𝑋 − 𝜇𝑋 )(𝑌 − 𝜇𝑌 )] = ෍ ෍ 𝑥 − 𝜇𝑋 𝑦 − 𝜇𝑌 𝑝(𝑥, 𝑦)
𝑥 𝑦
if 𝑋 and 𝑌 are discrete,
∞ ∞
𝜎𝑋𝑌 = 𝐸[(𝑋 − 𝜇𝑋 )(𝑌 − 𝜇𝑌 )] = න න (𝑥 − 𝜇𝑋 )(𝑦 − 𝜇𝑌 )𝑓(𝑥, 𝑦) 𝑑𝑥𝑑𝑦
−∞ −∞
if 𝑋 and 𝑌 are continuous.
• The covariance between two random variables is a measure of the nature of the association
between the two.
• The sign of the covariance indicates whether the relationship between two dependent random
variables is positive or negative.
• When 𝑋 and 𝑌 are statistically independent, it can be shown that the covariance is zero.
• Two variables may have zero covariance and still not be statistically independent. Note that the
covariance only describes the linear relationship between two random variables ( 𝑋 and 𝑌 may
have a nonlinear relationship).
Joint Probability Distributions
Covariance
The covariance of two random variables 𝑋 and 𝑌 with means 𝜇𝑋 and 𝜇𝑌 ,
respectively, is given by
𝜎𝑋𝑌 = Cov(𝑋, 𝑌) = 𝐸(𝑋𝑌 ) − 𝜇𝑋 𝜇𝑌
Joint Probability Distributions 𝜎𝑋𝑌 = Cov 𝑋, 𝑌 = 𝐸 𝑋𝑌 − 𝜇𝑋 𝜇𝑌

Covariance
The covariance has the following properties:

• Cov(𝑋, 𝑋) = 𝑉𝑎𝑟(𝑋);
• 𝑖𝑓 𝑋 𝑎𝑛𝑑 𝑌 are independent then Cov(𝑋, 𝑌) = 0;
• Cov(𝑋, 𝑌) = Cov(𝑌, 𝑋);
• Cov(𝑎𝑋, 𝑌) = 𝑎Cov(𝑋, 𝑌);
• Cov(𝑋 + 𝑐, 𝑌) = Cov(𝑋, 𝑌);
• Cov(𝑋 + 𝑌, 𝑍) = Cov(𝑋, 𝑍) + Cov(𝑌, 𝑍)
Joint Probability Distributions
Correlation coefficient
Let 𝑋 and 𝑌 be random variables with covariance 𝜎𝑋𝑌 and standard deviations 𝜎𝑋 and
𝜎𝑌 , respectively. The correlation coefficient of 𝑋 and 𝑌 is
𝑋−𝐸 𝑋 𝑌−𝐸 𝑌 𝑋 𝑌 𝜎𝑋𝑌
𝜌𝑋𝑌 = Cov , = Cov( , ) =
𝜎𝑋 𝜎𝑌 𝜎𝑋 𝜎𝑌 𝜎𝑋 𝜎𝑌

• The correlation coefficient satisfies the inequality −1 ≤ 𝜌𝑋𝑌 ≤ 1.

• It assumes a value of zero when 𝜎𝑋𝑌 = 0.
• Where there is an exact linear dependency, say 𝑌 ≡ 𝑎 + 𝑏𝑋, 𝜌𝑋𝑌 = 1 if 𝑏 >
0 and 𝜌𝑋𝑌 = −1 if 𝑏 < 0.

Consider two random variables 𝑋 and 𝑌:

• If 𝜌(𝑋, 𝑌) = 0, we say that 𝑋 and 𝑌 are uncorrelated.
• If 𝜌(𝑋, 𝑌) > 0, we say that 𝑋 and 𝑌 are positively correlated.
• If 𝜌(𝑋, 𝑌) < 0, we say that 𝑋 and Y are negatively correlated.
Joint Probability Distributions
Two ballpoint pens are selected at random from
a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
Covariance and Correlation coefficient selected and Y is the number of red pens
selected, find
Example:
Find the covariance of 𝑋 and 𝑌 .
Find the correlation coefficient between X and Y
Solution:
𝐶𝑜𝑣 𝑋, 𝑌 = 𝜎𝑋𝑌 = 𝐸(𝑋𝑌 ) − 𝜇𝑋 𝜇𝑌
2
5 15 3 3
𝜇𝑋 = ෍ 𝑥𝑔 𝑥 = 0 ∗ +1∗ +2∗ =
14 28 28 4
𝑥=0
2
15 3 1 1 2 2
𝜇𝑌 = ෍ 𝑦ℎ 𝑥 = 0 ∗ +1∗ +1∗ = 𝐸 𝑋𝑌 = ෍ ෍ 𝑥𝑦𝑝 𝑥, 𝑦
28 7 28 2 𝑥=0 𝑦=0
𝑦=0
3
3 3 1 9 = 𝑝 1,1 =
𝐶𝑜𝑣 𝑋, 𝑌 = 𝜎𝑋𝑌 = − ∗ =− 14
14 4 2 56
𝜎𝑋𝑌
𝜌𝑋𝑌 =
𝜎𝑋 𝜎𝑌
𝜎𝑋2 = 𝐸 𝑋 2 − 𝜇𝑋2 , 𝜎𝑌2 = 𝐸 𝑌 2 − 𝜇𝑌2
Joint Probability Distributions
Two ballpoint pens are selected at random from
a box that contains 3 blue pens, 2 red pens, and
3 green pens. If X is the number of blue pens
Covariance and Correlation coefficient selected and Y is the number of red pens
selected, find
Example: (cont.)
Find the covariance of X and Y .
Find the correlation coefficient between X and Y
Solution:
5 15 3 27
𝐸 𝑋 2 = 02 ∗ + 12 ∗ + 22 ∗ =
14 28 28 28

15 3 1 4
𝐸 𝑌 2 = 02 ∗ + 1 2 ∗ + 22 ∗ =
28 7 28 7
3
𝜇𝑋 =
4
1
𝜇𝑌 =
2
27 3 2 45 4 1 2 9
𝜎𝑋2 = 𝐸 𝑋 2 − 𝜇𝑋2 = − = , 𝜎𝑌2 = 𝐸 𝑌 2 − 𝜇𝑌2 = − =
28 4 112 7 2 28
9
𝜎𝑋𝑌 −
𝜌𝑋𝑌 = = 56 = − 1
𝜎𝑋 𝜎𝑌 45 9 5
∗
Joint Probability Distributions
Covariance and Correlation coefficient
Example:
The fraction 𝑋 of male runners and the fraction 𝑌 of female runners who compete in marathon races are described
by the joint density function.
8𝑥𝑦 , 0 ≤𝑦≤𝑥≤1
𝑓 𝑥, 𝑦 = ቊ
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
Find the covariance and the correlation coefficient of 𝑋 and 𝑌.
Solution:
∞ ∞
𝐶𝑜𝑣 𝑋, 𝑌 = 𝜎𝑋𝑌 = 𝐸(𝑋𝑌 ) − 𝜇𝑋 𝜇𝑌 , 𝜇𝑋 = ‫׬‬−∞ 𝑥 𝑔 𝑥 𝑑𝑥 𝜇𝑌 = ‫׬‬−∞ 𝑦 ℎ 𝑦 𝑑𝑦
∞ 𝑥 𝑦=𝑥
𝑔 𝑥 = න 𝑓 𝑥, 𝑦 𝑑𝑦 = න 8𝑥𝑦 𝑑𝑦 = 4𝑥𝑦 2 ቚ = 4𝑥 3 , 0≤𝑥≤1
−∞ 0 𝑦=0
∞ 1 𝑥=1
ℎ 𝑦 = න 𝑓 𝑥, 𝑦 𝑑𝑥 = න 8𝑥𝑦 𝑑𝑥 = 4𝑥𝑦 2 ቚ = 4𝑦(1 − 𝑦 2 ) , 0≤𝑦≤1
−∞ 𝑦 𝑥=𝑦
1 4 1 8
𝜇𝑋 = ‫׬‬0 4𝑥 4 𝑑𝑥 = , 𝜇𝑦 = ‫׬‬0 4𝑦 2 (1 − 𝑦 2 ) 𝑑𝑦 =
5 15

1 1 1 1
4
𝐸 𝑋𝑌 = න න 𝑥𝑦𝑓 𝑥, 𝑦 𝑑𝑥𝑑𝑦 = න න 8𝑥 2 𝑦 2 𝑑𝑥𝑑𝑦 =
0 𝑦 0 𝑦 9
4 4 8 4
𝐶𝑜𝑣 𝑋, 𝑌 = 𝜎𝑋𝑌 = − ∗ =
9 5 15 225
Joint Probability Distributions
Covariance and Correlation coefficient
Example: (Cont.)
Solution:
𝜎
𝜌𝑋𝑌 = 𝑋𝑌 , 𝜎𝑋2 = 𝐸 𝑋 2 − 𝜇𝑋2 , 𝜎𝑌2 = 𝐸 𝑌 2 − 𝜇𝑌2
𝜎𝑋 𝜎𝑌
4 8 4
𝜇𝑋 = , 𝜇𝑦 = , 𝜎𝑋𝑌 =
5 15 225
3
𝑔 𝑥 = 4𝑥 , 0≤𝑥≤1
ℎ 𝑦 = 4𝑦(1 − 𝑦 2 ) , 0≤𝑦≤1

∞ 1 2
𝐸 𝑋 2 = ‫׬‬−∞ 𝑥 2 𝑔 𝑥 𝑑𝑥 = ‫׬‬0 4𝑥 5 𝑑𝑥 = ,
3
∞ 1
2
1
𝐸 𝑌 =න 𝑦 ℎ 𝑦 𝑑𝑦 = න 4𝑦 3 1 − 𝑦 2 𝑑𝑦 =
2
−∞ 0 3
2 4 2 2 1 8 2 11
𝜎𝑋2 = − = , 𝜎𝑌2 = − =
3 5 75 3 15 225
4
225 4
𝜌𝑋𝑌 = =
2 11 66
∗
75 225