JointProbability

Distributions

CHAPTEROUTLINE
51TwoorMoreRandomVariables
51.1JointProbabilityDistributions
51.2MarginalProbability
Distributions
51.3ConditionalProbability
Distributions
51.4Independence
51.5MoreThanTwoRandom
Variables

52CovarianceandCorrelation
53CommonJointDistributions
53.1MultinomialProbability
Distribution
53.2BivariateNormalDistribution
54LinearFunctionsofRandom
Variables
55GeneralFunctionsofRandom
Variables

Chapter4TitleandOutline
1

Outline

JointProbabilityMass/DistributionFunction
MarginalProbabilityDistribution
Mean
Variance
ConditionalProbability
Independence
Covariance
Correlation

Sec2

3
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

ConceptofJointProbabilities
Somerandomvariablesarenotindependentofeach
other,i.e.,theytendtoberelated.
Urbanatmosphericozoneandairborneparticulatematter
tendtovarytogether.
Urbanvehiclespeedsandfuelconsumptionratestendto
varyinversely.

Thelength(X)ofainjectionmoldedpartmightnotbe
independentofthewidth(Y).Individualpartswillvary
duetorandomvariationinmaterialsandpressure.
Ajointprobabilitydistributionwilldescribethe
behaviorofseveralrandomvariables,say,X andY.The
graphofthedistributionis3dimensional:x,y,and
f(x,y).
Chapter5Introduction

4
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

Example51:SignalBars
Youuseyourcellphonetocheckyourairlinereservation.Theairlinesystem
requiresthatyouspeakthenameofyourdeparturecitytothevoice
recognitionsystem.
LetYdenotethenumberoftimesthatyouhavetostateyourdeparture
city.
LetXdenotethenumberofbarsofsignalstrengthonyoucellphone.
y=numberof x=numberofbars
timescity
ofsignalstrength
2
3
nameisstated 1
1
0.01 0.02 0.25
2
0.02 0.03 0.20
3
0.02 0.10 0.05
4
0.15 0.10 0.05

Figure51Jointprobability
distributionofXandY.Thetablecells
aretheprobabilities.Observethat
morebarsrelatetolessrepeating.

0.25
0.2
Y1

0.15

Y2

0.1

Y4
Y3

0.05

Y2

0
1

Y3
Y4

Y1
2

Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

JointProbabilityMassFunctionDefined
The joint probability mass function of the
discrete random variables X and Y ,
denoted as f XY x, y , satifies:
(1) f XY x, y 0
(2)

f x, y 1
XY

All probabilities are non-negative

The sum of all probabilities is 1

(3) f XY x, y P X x, Y y

Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

(5-1)

MarginalProbabilityDistributions(discrete)
Y=

fY(y)

1
0

0.15

0.10

0.02

0.10
0.03

0.20

0.01

0.02

0.25

f X x f xy
y

fY y f xy
Sec2

0.05

0.02

0.05

2
1
x

0.15

0.10

0.05

0.02

0.10

0.05

0.02

0.03

0.20

0.01

0.02

0.25

X=

fX(x)

JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

MeanandVariance
y

Y=

fY(y)

0.30

0.17

0.25

0.28

0.15

0.10

0.05

0.02

0.10

0.05

0.02

0.03

0.20

0.01

0.02

0.25

X=

fX(x)

0.20

0.25

0.55

E(X)=
E(X2)=
V(X)=

E(Y)=
E(Y2)=
V(Y)=

Sec2

10
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

JointProbabilityDensityFunctionDefined
Thejointprobabilitydensityfunctionforthecontinuousrandom
variablesX andY,denotesasfXY(x,y),satisfiesthefollowing
properties:
(1) f XY x, y 0 for all x, y

(2)

f XY x, y dxdy 1

(3) P X , Y R f XY x, y dxdy

(5-2)

Figure52Jointprobability
densityfunctionfortherandom
variablesX andY.Probabilitythat
(X,Y)isintheregionR is
determinedbythevolume of
fXY(x,y)overtheregionR.
Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

11

JointProbabilityMassFunctionGraph

Figure53Jointprobabilitydensityfunctionforthe
continuousrandomvariablesX andY ofdifferent
dimensionsofaninjectionmoldedpart.Notethe
asymmetric,narrowridgeshapeofthePDF
indicatingthatsmallvaluesintheX dimensionare
morelikelytooccurwhensmallvaluesintheY
dimensionoccur.
Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

12

Example52:ServerAccessTime1
LettherandomvariableX denotethetime(msecs)untila
computerserverconnectstoyourmachine.LetY denotethe
timeuntiltheserverauthorizesyouasavaliduser.X andY
measurethewaitfromacommonstartingpoint (x <y).The
rangeofx andy areshownhere.

Figure54Thejointprobability
densityfunctionofX andY isnonzero
overtheshadedregionwherex <y.

Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

13

Example52:ServerAccessTime2
Thejointprobabilitydensityfunctionis:
f XY x, y ke 0.001x 0.002 y for 0 x y and k 6 106

Weverifythatitintegratesto1asfollows:

0.001x 0.002 y
0.002 y 0.001x
f XY x, y dxdy ke
dy dx k e
dy e
dx
0 x
0 x

e 0.002 x 0.001x
0.003 x
e
dx
0.003
e
dx
k

0.002
0
0

1
0.003
1
0.003
Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

14

Example52:ServerAccessTime2
Nowcalculateaprobability:
P X 1000, Y 2000

1000 2000

1000

1000

f XY x, y dxdy

2000 0.002 y 0.001x

dy e
dx
e
x

e 0.002 x e 4 0.001x
dx

e
0.002
1000

0.003

e 0.003 x e 4 e 0.001x dx

1 e 3 4 1 e 1
0.003
e

0.003
0.001

0.003 316.738 11.578 0.915

Figure55Regionof
integrationfortheprobability
thatX <1000andY <2000is
darklyshaded.

Sec51.1JointProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

16

MarginalProbabilityDistributions(continuous)
RatherthansummingadiscretejointPMF,we
integrateacontinuousjointPDF.
ThemarginalPDFsareusedtomakeprobability
statementsaboutonevariable.
Ifthejointprobabilitydensityfunctionofrandom
variablesX andY isfXY(x,y),themarginal
probabilitydensityfunctionsofX andY are:

f X x f XY x, y dy
y

fY y f XY x, y dx

(5-3)

Sec51.2MarginalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

17

Example54:ServerAccessTime1
Fortherandomvariables
timesinExample52,find
theprobabilitythatY
exceeds2000.
IntegratethejointPDFdirectly
usingthepictureto
determinethelimits.
P Y 2000

2000

Dark region

f XY x, y dy dx f XY x, y dy dx
2000 x
2000

left dark region

right dark region

Sec51.2MarginalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

18

Example54:ServerAccessTime2
Alternatively,findthemarginalPDF andthenintegrate
thattofindthedesiredprobability.
y

fY y ke

0.001 x 0.002 y

dx

P Y 2000

fY y dy

2000

e 0.002 y 1 e 0.001 y dy

ke 0.002 y e 0.001x dx

6 10

e 0.001x y

ke 0.002 y
0.001 0

e 0.002 y e 0.003 y

6 103

0.002 2000 0.003 2000

1 e

ke 0.002 y

0.001

0.001 y

2000

e 4
e 6
6 10

0.05

0.002 0.003
3

6 103 e 0.002 y 1 e 0.001 y for y 0

Sec51.2MarginalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

19

Mean&VarianceofaMarginalDistribution
MeansE(X)andE(Y)arecalculatedfromthe
discreteandcontinuousmarginaldistributions.
Discrete
E X x fX x
R

E Y y fY y
R

Continuous
x f X x dx X
R

y fY y dy Y
R

V X x 2 f X x X2 x 2 f X x dx X2
R

V Y y 2 fY y Y2 y 2 fY y dy Y2
R

Sec51.2MarginalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

20

ConditionalProbabilityDistributions
P A B
Recall that P B A
P A

4
3
2
1

0.15

0.10

0.05

0.02

0.10

0.05

FromExample51

0.02

0.03

0.20

0.01

0.02

0.25

P(Y=1|X=3)=
P(Y=2|X=3)=
P(Y=3|X=3)=
P(Y=4|X=3)=

Y=

3
3

f (y|X=3)

Sec2

21
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

ConditionalProbabilityDensityFunctionDefined
Given continuous random variables X and Y with
joint probability density function f XY x, y ,
the conditional probability densiy function of Y given X =x is
f XY x, y
fY x y
for f X x 0
fX x

(5-4)

which satifies the following properties:

(1) fY x y 0
(2)
(3)

f y dy 1
P Y B X x f y dy for any set B in the range of Y
Yx

Yx

Sec51.3ConditionalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

23

Mean&VarianceofConditionalRandomVariables
y

4
3
2
1
0

FromExample51
0.15

0.10

0.05

0.02

0.10

0.05

0.02

0.03

0.20

0.01

0.02

0.25

P(Y=1|X=3)=0.25/0.55=0.455
P(Y=2|X=3)=0.20/0.55=0.364
P(Y=3|X=3)=0.05/0.55=0.091
P(Y=4|X=3)=0.05/0.55=0.091
E(Y|X=3)=
x

V(Y|X=3)=
Y=

f (y|X=3) 0.455 0.364 0.091 0.091

Sec2

27
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

Mean&VarianceofConditionalRandomVariables

TheconditionalmeanofY givenX =x,

denotedasE(Y|x)orY|x is:
E Y x y fY x y dy

(5-6)

TheconditionalvarianceofY givenX =x,

denotedasV(Y|x)or2Y|x is:

V Y x y Y x
y

fY x y dy y 2 fY x y Y2 x
y

V(Y|x) = E(Y2|x) 2Y|x

Sec51.3ConditionalProbabilityDistributions
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

28

JointRandomVariableIndependence
Randomvariableindependencemeansthat
knowledgeofthevaluesofXdoesnotchange
anyoftheprobabilitiesassociatedwiththe
valuesofY.
X andY varyindependently.
DependenceimpliesthatthevaluesofX are
influencedbythevaluesofY.
Doyouthinkthatapersonsheightandweight
areindependent?
Sec51.4Independence

31
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

Exercise510:IndependentRandomVariables

Inaplasticmoldingoperation,
X 1 if the part conforms to color specs
0 otherwise
eachpartisclassifiedasto
whetheritconformstocolor Y 1 if the part conforms to length specs
0 otherwise
andlengthspecifications.

Figure510(a)showsmarginal&joint
probabilities,fXY(x,y)=fX(x)*fY(y)

Figure510(b)showtheconditional
probabilities,fY|x(y)=fY(y)

Sec51.4Independence

32
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

PropertiesofIndependence
ForrandomvariablesX andY,ifanyoneofthefollowing
propertiesistrue,theothersarealsotrue.ThenX
andY areindependent.
(1) f XY x, y f X x fY y
(2) fY x y fY y for all x and y with f X x 0
(3) f X y y f X x for all x and y with fY y 0
(4) P X A, Y B P X A P Y B for any
sets A and B in the range of X and Y , respectively.

Sec51.4Independence

(5-7)

33
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

RectangularRangefor(X,Y)
ArectangularrangeforX andY isanecessary,
butnotsufficient,conditionforthe
independenceofthevariables.
IftherangeofXandYisnotrectangular,then
therangeofonevariableislimitedbythe
valueoftheothervariable.
IftherangeofXandYis rectangular,thenone
ofthepropertiesof(57)mustbe
demonstratedtoproveindependence.
Sec51.4Independence

34
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

Covariance
Covarianceisameasureoftherelationship
betweentworandomvariables.
First,weneedtodescribetheexpectedvalueofa
functionoftworandomvariables.Leth(X,Y)
denotethefunctionofinterest.
h x, y f XY x, y for X , Y discrete

E h X , Y
(5-13)
h x, y f XY x, y dxdy for X , Y continuous

Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

49

Example519:E(Functionof2RandomVariables)
Task:CalculateE[(XX)(YY)]
=covariance

Mean

Marginal

Joint

y
1
1
3
3
3
1
3

1
2
1
2
3

1
2
3
X =
Y =

f(x,y) xX yY Prod
0.1
1.4
1.0 0.14
0.2
1.4
0.0 0.00
0.2
0.6
1.0 0.12
0.2
0.6
0.0 0.00
0.3
0.6
1.0 0.18
0.3 covariance= 0.20
0.7
0.3
0.4
0.3
2.4
2.0

Figure512Discretejoint
distributionofXandY.

Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

50

***CovarianceDefined
The covariance between the random variables X and Y,
denoted as cov X , Y or XY is

XY E X X Y Y E XY X Y

(5-14)

The units of XY are units of X times units of Y .

For example, if the units of X are feet
and the units of Y are pounds,
the units of the covariance are foot-pounds.
Unlike the range of variance, - XY .
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

51

Proof
)

Note,however,that

,
,

Similarly,itcanbeshownthat

,thus

Sec2

52
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

CovarianceandScatterPatterns
XandYare
independent.

XandYare
NOT
independent.

Figure513Jointprobabilitydistributionsandthesignofcov(X,Y).
**Notethatcovarianceisameasureoflinearrelationship.
**Variableswithnonzerocovariancearesaidtobecorrelated.
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

53

Example520:IntuitiveCovariance
y=numberof x=numberofbars
ofsignalstrength
timescity
2
3
nameisstated 1
1
0.01 0.02 0.25
2
0.02 0.03 0.20
3
0.02 0.10 0.05
4
0.15 0.10 0.05

0.25
0.2
Y1

0.15

Y2

0.1

Y4
Y3

0.05

Y3
Y4

Y2

0
1

Y1
2

TheprobabilitydistributionofExample51isshown.
Byinspection,notethatthelargerprobabilitiesoccurasX
andY moveinoppositedirections.Thisindicatesanegative
covariance.
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

54

***Correlation( =rho)
The correlation between random variables X and Y,
denoted as XY , is

XY

cov X , Y

XY

V X V Y X Y

(5-15)

Since X 0 and Y 0,

XY and cov X , Y have the same sign.

We say that XY is normalized, so 1 XY 1

(5-16)

Note that XY is dimensionless.

Variables with non-zero correlation are correlated.
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

55

Example521:Covariance&Correlation

Joint

Figure514Discretejoint
distribution,f(x,y).

StDev

Mean

Marginal

Determinethe
covarianceand
correlation.

y
0
1
1
2
2
3
0
1
2
3

0
1
2
1
2
3

0
1
2
3
X =
Y =

f(x,y) xX yY Prod
0.2
1.8
1.2 0.42
0.1
0.8
0.2 0.01
0.1
0.8
0.8 0.07
0.1
0.2
0.2 0.00
0.1
0.2
0.8 0.02
0.4
1.2
1.8 0.88
0.2 covariance= 1.260
0.2 correlation= 0.926
0.2
0.4
Notethestrong
0.2 positivecorrelation.
0.2
0.2
0.4
1.8
1.8

X= 1.1662
Y= 1.1662

Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

56

Example521
Calculatethe
correlation.

Figure515Discretejoint
distribution.Steepnessofline
connectingpointsisimmaterial.
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

57

IndependenceImplies =0
IfX andY areindependentrandomvariables,
XY =XY =0

(517)

XY =0isnecessary,butnotasufficientcondition forindependence.
Figure513d(x,y plotsasacircle)providesanexampleofdependence
DESPITE havingXY =XY =0.
Figure513b(x,y plotsasasquare)providesanexampleof
independence AND havingXY =XY =0.

Independent XY =XY =0

Dependentbut XY =XY =0

Dependentand XY ,XY 0

Ingeneral,arectangularpatterindicatesindependence,andthusXY =
XY =0.
But,anonrectangularpatternwouldindicatedependence.
Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

58

Example523:IndependenceImpliesZeroCovariance
Let f XY xy x y 16 for 0 x 2 and 0 y 4
Show that XY E XY E X E Y 0
EX

1 2
x
ydx

dy
16 0 0

x
1
y
16 0 3

dy
0

3 2

1 y 8 1 16 4

16 2 0 3 6 2 3

4 2

1
E Y xy 2 dx dy
16 0 0

4
2 2
1
2 x
y
dy
16 0 2 0

2 4

2 y3

16 3

1 64 8

8 3 3
0

4 2
1 2 2
E XY x y dx dy
16 0 0

4
3
1
2 x
y
16 0 3

dy
0

1
8
y 2 dy
16 0 3
4

1 y3

6 3

1 64 32

6 3
9
0

Figure515Aplanar
jointdistribution.

XY E XY E X E Y

32 4 8
0
9 3 3

Sec52Covariance&Correlation
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

59

Summary1:MarginalProbabilities
Marginalprobabilitydistributionfunction
f x f xy
f X x f XY x, y dy
X

fY y f xy
x

ExpectationandVariance

fY y f XY x, y dx
x

Sec2

61
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.

Summary2:CovarianceandCorrelation
Covariance
Correlation

IfX andY areindependentrandomvariables,

XY =XY =0
TheoppositeisNOT true.
Sec2

62
JohnWiley&Sons,Inc. AppliedStatisticsandProbabilityforEngineers,byMontgomeryandRunger.