Statistics (I)

Joint Probability
Distributions

王泰裕教授
成功大學工業與資訊管理學系

1/78
 Learning Objectives
1. Define two discrete random variables
2. Describe two continuous random variables
3. Discuss covariance and correlation
4. Define bivariate normal Distribution
5. Describe linear combination of random variables
6. Discuss Chebyshev’s inequality
 Why Two Random Variables ?
◼ To find out the simultaneous behavior of two
random variables.
◼ Example:

X and Y are random variables defined for

the two dimensions of a part of a production
line. What is the probability that X is within the
range of 2.95 ~ 3.05 and Y is within the range of
7.60~7.80?
 Why Two Discrete Random
Variables ?
◼ Example:
Tossing a fair dice twice, let
S = {(x, y )|1 ≤ x ≤ 6, 1 ≤ y ≤ 6}
where x is the points appeared at
the first time, y is the points
appeared at the second time.
 Why Two Discrete Random
Variables ?
◼ Example:
so we call
f(X=3, Y=5)=1/36
as the joint probability mass function.
 Two Discrete
Random Variables
◼ If X and Y are discrete random variables, the
joint distribution of X and Y is a description of
the set of points (x, y) in the range of (X, Y)
along with the probability of each point.
◼ It is sometimes referred to as the bivariate
probability distribution or bivariate
distribution.
 Two Discrete
Random Variables
◼ Definition
◼ The joint probability mass function of the discrete
random variable X and Y, denoted as fXY(x,y) ,
satisfies
1. f XY ( x, y )  0
2.  f XY (x,y) = 1
x y

3. f XY (x,y) = P ( X = x, Y = y )
 Two Discrete Random Variables -
--Example
◼ A financial company uses X=1, 2,3 to
represent low, medium, and high income
customers respectively. Also they use
Y=1,2,3,4 to represent mutual funds, bonds,
stocks, and options respectively.
◼ Then the joint probability of X and Y could be
represented by following table.
Two Discrete Random Variables -
--Example
Y=1 Y=2 Y=3 Y=4

X=1 0.1 0 0 0

X=2 0.2 0.1 0.2 0

X=3 0.1 0 0.1 0.2

(1) f XY ( x , y )  0
(2)  f XY ( x , y ) = 1
x y
Two Discrete Random Variables -
--Example

0.2
0.15
0.1
0.05
Y 0
1 3
2 2
3
4 1
Marginal Probability
Distribution (1/2)

f X ( x ) = P( X = x ) =  f XY ( x, y )
Rx

fY ( y ) = P(Y = y ) =  f XY ( x, y )
Ry
 Marginal Probability
Distribution (2/2)

where
Rx denotes the set of all points in the range of
(X, Y) for which X=x and
Ry denotes the set of all points in the range of
(X, Y) for which Y=y
 Mean of Marginal
Probability Distribution
◼ If the marginal probability distribution of X
has the probability mass function fX(x), then
 Example(1/3)
◼ A financial company use X=1,2,3 to
represent low, medium, and high income
customers respectively. Also they use
Y=1,2,3,4 to represent mutual funds,
bonds, stocks, and options respectively.
◼ Then the joint probability of X and Y could
be represented by following table.
Example(2/3)

Y=1 Y=2 Y=3 Y=4

X=1 0.1 0 0 0

X=2 0.2 0.1 0.2 0

X=3 0.1 0 0.1 0.2

 Example(3/3)

◼ Please find marginal probability of fX(x,y),

fY(x,y), and their mean and variance.
Solution to Example(1/3)
Solution to Example(2/3)
 Solution to Example(3/3)

Y=1 Y=2 Y=3 Y=4 fX(x)

X=1 0.1 0 0 0 0.1

X=2 0.2 0.1 0.2 0 0.5

X=3 0.1 0 0.1 0.2 0.4

fY(Y) 0.4 0.1 0.3 0.2 1

 Conditional Probability
Distribution
◼ Given discrete random variable X and Y
with joint probability mass function
fXY(x, y) the conditional probability mass
function of Y given X = x is
Properties of Conditional
Probability Distribution
 Example(1/3)

◼ Same example mentioned above

Y=1 Y=2 Y=3 Y=4

X=1 0.1 0 0 0

X=2 0.2 0.1 0.2 0

X=3 0.1 0 0.1 0.2

 Example(2/3)

◼ Computed results
Y=1 Y=2 Y=3 Y=4 fX(x)

X=1 0.1 0 0 0 0.1

X=2 0.2 0.1 0.2 0 0.5

X=3 0.1 0 0.1 0.2 0.4

fY(Y) 0.4 0.1 0.3 0.2 1

 Example(3/3)

◼ The conditional probability of Y given X =

2 is
 Properties of Conditional
Probability Distribution
Conditional mean of Y given X = x

Conditional variance of Y given X = x

 Example

Please find E(X|y=1)

 Example

Please find V(X|y=1).

 Independence of Two Discrete
Random Variables(1/2)
◼ For discrete random variables X and Y, if
any one of the following properties is
true, then the others are also true, and X
and Y are independent .
 Independence of Two Discrete
Random Variables(2/2)

(1) f XY ( x , y ) = f X ( x ) f Y ( y ) for all x and y

( 2 ) f Y | x ( y ) = f Y ( y ) for all x and y with f X ( x ) > 0
( 3) f X | y ( x ) = f X ( x ) for all x and y with f Y ( y ) > 0
( 4) P ( X = x, Y = y ) = f X ( x ) f Y ( y )
for all ( x , y ) in the range of X and Y
 Example

In above example, are X and Y independent ?

 Solution to Example

P(X=1, Y=1)=0.1
fX(Y=1)=0.4
fY(X=1)=0.1
P(X=1, Y=1) ≠ fX(Y=1)*fY(X=1)
Thus X and Y are dependent!
Two Continuous
Random Variables
 Two Continuous
Random Variables
◼ A joint probability density function for the
continuous random variables X and Y, denoted
as fXY(x, y), satisfied the following properties.
Two Continuous
Random Variables
 Example

◼ Suppose X is the time to failure of a

component, and Y is also the time to failure
of its spare part. If the joint probability
density function of X and Y is:

Please find P(X≦1000, Y≦2000)=?

 Solution to Example

∞ ∞ ∞ ∞
න න 𝑓𝑋𝑌 𝑥, 𝑦 𝑑𝑦𝑑𝑥 = න න 6 × 10−6 𝑒 −0.001𝑥−0.002𝑦 𝑑𝑦 𝑑𝑥
−∞ −∞ 0 𝑥
∞ ∞
= 6 × 10−6 ‫׬‬0 ‫ 𝑒 𝑥׬‬−0.002𝑦 𝑑𝑦 𝑒 −0.001𝑥 𝑑𝑥

∞ 𝑒 −0.002𝑥
=6× 10−6 ‫׬‬0 𝑒 −0.001𝑥 𝑑𝑥
0.002

6×10−6 ∞ −0.003𝑥 6×10−6 1

=
0.002
‫׬‬0 𝑒 𝑑𝑥 = 0.002 0.003
=1
∞
න 𝑒 −0.001𝑥 𝑑𝑥
0
 Solution to Example

1000 2000
𝑃 𝑋 ≤ 1000, 𝑌 ≤ 2000 = න න 𝑓𝑋𝑌 𝑥, 𝑦 𝑑𝑦𝑑𝑥
0 𝑥

1000 2000 −0.002𝑦

= 6 × 10−6 ‫׬‬0 ‫𝑥׬‬ 𝑒 𝑑𝑦 𝑒 −0.001𝑥 𝑑𝑥

−6 1000 𝑒 −0.002𝑥 −𝑒 −4
= 6 × 10 ‫׬‬0 𝑒 −0.001𝑥 𝑑𝑥
0.002

6×10−6 1000 −0.003𝑥 −4 −0.001𝑥

= ‫׬‬0 𝑒 𝑒 𝑒 𝑑𝑥
0.002

= 3 × 10−3 316.738 − 11.578 = 0.915

 Marginal Probability
Distributions
◼ If the joint probability density function of continuous
random variables X and Y is fXY (x,y), then the
probability density functions of X and Y are

Rx denotes the set of all points in the range of (X,Y) for which X=x
and
Ry denotes the set of all points in the range of (X,Y) for which Y=y
 Example

From above example, please find P(Y>2000)=?

Solution to Example

0.05 Why?
 Conditional Probability
Distribution
Given continuous random variable X and Y
with joint probability density function
fXY(x, y) the conditional probability density
function of Y given X = x is

f XY ( x , y )
f Y |x (x, y) =
fX (x)
for all fX (x) > 0
Properties of Conditional
Probability Distribution
 Example

From above example, please find the

conditional density function Y given that
X=x.
Solution to Example

p.225
 Properties of Conditional
Probability Distribution
Conditional mean of Y given X = x

Conditional variance of Y given X = x

 Properties of Conditional
Probability Distribution
Conditional mean of Y given X = x

Conditional variance of Y given X = x

 Independence of Two Continuous
Random Variables (1/2)
◼ For continuous random variables X
and Y, if any one of the following
properties is true, then the others are
also true, and X and Y are
independent .
 Independence of Two continuous
Random Variables(2/2)

(1) f XY ( x , y ) = f X ( x ) f Y ( y ) for all x and y

( 2 ) f Y | x ( y ) = f Y ( y ) for all x and y with f X ( x ) > 0
( 3) f X | y ( x ) = f X ( x ) for all x and y with f Y ( y ) > 0
( 4 ) P ( X A, Y  B) = P ( X A) P(Y  B)
for any sets A and B in the range of X
and Y, respectively.
Covariance and Correlation
 Covariance

◼ Expected value of a function of two random

variables h(x, y)

R  h( x, y ) f XY ( x, y ) X,Y discrete 

 
E[h( X , Y )] =  h( x, y ) f ( x, y )dxdy X,Y continuous 

R
 XY


 Covariance

The covariance between the random variables

X and Y, denoted as cov(X,Y) or σXY ,

 XY = E[( X −  X )(Y − Y )] = E ( XY ) −  X Y
 Example(1/3)

A financial company use X=1,2,3 to

represent low, medium, and high
income customers respectively. Also
they use Y=1,2,3,4 to represent mutual
funds, bonds, stocks, and options
respectively.
Then the joint probability of X and Y could be
represented by following table.
Example(2/3)

Y=1 Y=2 Y=3 Y=4

X=1 0.1 0 0 0

X=2 0.2 0.1 0.2 0

X=3 0.1 0 0.1 0.2

 Example(3/3)

Please find the covariance of X and Y.

Solution to Example
 XY = E ( XY ) −  X Y
= 1  1  0 .1 + 2  1  0 . 2 + 2  2  0 .1
+ 2  3  0.2 + 3  1 0.1 + 3  3  0.1
+ 3  4  0 . 2 − 2 . 3  2 .3
= 5.7 − (2.3) 2

= 0.41
 Correlation

The correlation between the random variables X and

Y, denoted as XY ,

cov( X , Y ) σ XY
ρ XY = =
V ( X )V (Y ) σ Xσ Y
where - 1  ρ XY  1
 Correlation

If X and Y are independent random variables,

then

σ XY = ρ XY =0
 Example

From the above example, please find the

correlation of X and Y.
Solution to Example

cov( X , Y ) σ XY
ρ XY = =
V ( X )V (Y ) σ Xσ Y
0.41
=
(0.41)(1.41)
 Covariance Matrix for Multi-
Dimensional Data
◼ For multi-dimensional data, X=[x1, x2, …xn],
the covariance matrix is the diagonal values
of the covariance matrix and can be shown
as:
𝑛
1
𝐶𝑂𝑉 = ෍ 𝑋𝑖 − 𝑋ത 𝑋𝑖 − 𝑋ത 𝑇
𝑛−1
𝑖=1
Bivariate Normal Distribution
 Bivariate Normal
Distribution

The probability density function of a

bivariate normal distribution
f XY ( x, y;σ X ,σ Y ,μ X ,μ Y ,ρ )
1  - 1  ( x − μ X ) 2 2ρ (x- X )( y − μ Y ) ( y − μ Y ) 2  
= exp 2 
− + 
2σ Xσ Y  2(1-ρ )  σ X σ Xσ Y σ Y 
2 2
1 −ρ 2

for -∞<x<∞ and -∞<y<∞, with parameters σX >0,

σy >0 , -∞<μX<∞ and -∞< μY <∞, and -1<ρ<1
 Bivariate Normal
Distribution

If X and Y have a bivariate normal distribution

with joint probability density fXY(x,y; σX, σY, μX, μY,
ρ), then the marginal probability distributions of X
and Y are normal with means μX and μY, and
standard deviations σX and σY, respectively.
 Bivariate Normal
Distribution

If X and Y have a bivariate normal distribution

with joint probability density fXY(x,y; σX, σY, μX, μY,
ρ), then the correlation between X and Y is ρ
 Bivariate Normal
Distribution

If X and Y have a bivariate normal

distribution with ρ=0, then X and Y are
independent.
Linear Combinations of Random
Variables
 Linear Combinations of
Random Variables

Given random variables X1, X 2, …., Xp and

constants c1, c 2, …., cp , then
Y= c1 X1+ c 2 X 2 …., cp Xp
is a linear combination of X1, X 2, …., Xp .
 Linear Combinations of
Random Variables

If Y= c1 X1+ c 2 X 2 …., cp Xp , then

E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp)
 Linear Combinations of
Random Variables
If Y= c1 X1+ c 2 X 2 …., cp Xp , then
V (Y ) = c12V ( X 1 ) + c22V ( X 2 ) + ... + c 2pV ( X p )
+ 2  ci c j cov( X i , X j )
i j =2

Furthermore, if X1, X 2, …., Xp , are

independent, then
V (Y ) = c12V ( X 1 ) + c22V ( X 2 ) + ... + c 2pV ( X p )
 Linear Combinations of
Random Variables

If X = ( X1 + X2 +...+ X p ) / p with E(Xi)=μ for i=1,2,…,p,

then
E( X ) =μ

Furthermore, if X1, X 2, …., Xp , are also

independent, with V(Xi ) =σ 2 for i=1,2,…,p, then
V ( X ) =σ 2 / p
 Reproductive Property
of the Normal Distribution
If X1, X 2, …., Xp , are independent, normal
2
random variables with E(Xi)=μi and V(Xi)=σi , for
i=1,2,..,p, then
Y= c1 X1+ c 2 X 2 …., cp Xp
is a normal random variable with
E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp) and
V (Y ) = c  + c 
2
1
2
1
2
2
2
2 + ... + c 
2
p
2
p
Chebyshev’s Inequality
 Chebyshev’s Inequality

For any random variable X with mean μ and

variance σ2 ,
P(|X- μ| ≧ c σ) ≦ 1/c 2
for c >0
 Example

The process of drilling holes in PCB produces

diameters with a standard deviation of 0.01
millimeter. How many diameters must be
measured so that the probability is at least 8/9
that the average of the measured diameters is
within 0.005 of the process mean.
 Solution to Example(1/2)

Let X1, X2, …, Xn be the independent random

variables that denote the diameters of n holes.
So the average measured diameter is :
X = ( X 1 + X 2 + ... + X n ) / n
 E ( X ) =  and V ( X ) = 0.01 / n2

 σ X = 0.01 / n 2
 Solution to Example(2/2)

P(|X- μ| ≧ c σ) ≦ 1/c 2
So P (| X −  | c (0.012 / n)1/ 2 )  1 / c 2
let c = 3
P (| X −  | 3(0.012 / n)1/ 2 )  1 / 9
i.e. P (| X −  | 3(0.012 / n)1/ 2 )  8 / 9
So 3(0.012 / n)1/ 2 = 0.005
and n = 36