Applied Statistics and

Probability for Engineers

Sixth Edition
Douglas C. Montgomery George C. Runger

Chapter 5
Joint Probability Distributions

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Joint Probability

5
CHAPTER OUTLINE
5-1 Two or More Random Variables
Distributions

5-5 General Functions of Random Variables

5-1.1 Joint Probability Distributions 5-6 Moment Generating Functions
5-1.2 Marginal Probability Distributions
5-1.3 Conditional Probability Distributions
5-1.4 Independence
5-1.5 More Than Two Random Variables

5-2 Covariance and Correlation

5-3 Common Joint Distributions
5-3.1 Multinomial Probability Distribution
5-3.2 Bivariate Normal Distribution
5-4 Linear Functions of Random Variables

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Learning Objectives for Chapter 5
After careful study of this chapter, you should be able to do the
following:
1. Use joint probability mass functions and joint probability density
functions to calculate probabilities.
2. Calculate marginal and conditional probability distributions from joint
probability distributions.
3. Interpret and calculate covariances and correlations between random
variables.
4. Use the multinomial distribution to determine probabilities.
5. Properties of a bivariate normal distribution and to draw contour plots
for the probability density function.
6. Calculate means and variances for linear combinations of random
variables, and calculate probabilities for linear combinations of
normally distributed random variables.
7. Determine the distribution of a general function of a random variable.
8. Calculate moment generating functions and use them to determine
moments and distributions

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Joint Probability Mass Function

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Joint Probability Density Function
The joint probability density function for the continuous
random variables X and Y, denotes as fXY(x,y), satisfies the
following properties:

Figure 5-2 Joint probability

density function for the random
variables X and Y. Probability
that (X, Y) is in the region R is
determined by the volume of
fXY(x,y) over the region R.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-2: Server Access Time-1
Let the random variable X denote the time until a computer
server connects to your machine (in milliseconds), and let Y
denote the time until the server authorizes you as a valid user (in
milliseconds). X and Y measure the wait from a common
starting point (x < y). The joint probability density function for X
and Y is
f XY  x, y   ke0.001x0.002 y for 0  x  y   and k  6 106

Figure 5-4 The joint probability

density function of X and Y is
nonzero over the shaded region
where x < y.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-2: Server Access Time-2
The region with nonzero probability is shaded in
Fig. 5-4. We verify that it integrates to 1 as follows:

  
  0.001x 0.002 y   0.002 y  0.001x
 

 f XY  x, y dydx     ke dy dx  k    e dy  e dx
  00  00 
 
 e0.002 x  0.001x
 k e dx  0.003  e 0.003 x
dx
0
0.002  0

 1 
 0.003   1
 0.003 

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-2: Server Access Time-3
Now calculate a probability:
1000 2000
P  X  1000, Y  2000     f XY  x, y dydx
0 x

 2000 0.002 y  0.001x

1000
k    e dy  e dx
0  x 
1000
 e 0.002 x  e 4  0.001x
k   e dx
0 
0.002 
1000
 0.003 
0
e 0.003 x  e 4 e 0.001x dx
Figure 5-5 Region of
integration for the probability
 1  e 3  4  1  e 1  
 0.003  e   that X < 1000 and Y < 2000
 0.003   0.001  is darkly shaded.
 0.003  316.738  11.578   0.915

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Marginal Probability Distributions (discrete)
The marginal probability distribution for X is found by summing the probabilities
in each column whereas the marginal probability distribution for Y is found by
summing the probabilities in each row.

f X  x    f  xy  y = Response x = Number of Bars of

time(nearest Signal Strength
y second) 1 2 3 f (y )

fY  y    f  xy  1
2
0.01
0.02
0.02
0.03
0.25
0.20
0.28
0.25
x
3 0.02 0.10 0.05 0.17
4 0.15 0.10 0.05 0.30
f (x ) 0.20 0.25 0.55 1.00

Marginal probability distributions of X and Y

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Marginal Probability Density Function (continuous)

If the joint probability density function of

random variables X and Y is fXY(x,y), the
marginal probability density functions of X
and Y are:

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-4: Server Access Time-1
For the random variables that
denotes times in Example 5-2,
find the probability that Y
exceeds 2000 milliseconds.

Integrate the joint PDF directly

using the picture to determine
the limits.

2000
   
 
P Y  2000      f XY  x, y  dy dx     f XY  x, y  dy dx
0  2000  2000  x 
Dark region  left dark region  right dark region

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-4: Server Access Time-2
Alternatively, find the marginal PDF and then
integrate that to find the desired probability.
y 
fY  y    ke 0.001 x  0.002 y
dx P Y  2000    fY  y dy
0 2000
y 

 ke 0.002 y  e 0.001x dx  6 10 3

 e 0.002 y 1  e 0.001 y dy
2000
0

e 
0.001 x y
 e 0.002 y    e 0.003 y   
 ke 
0.002 y
  6 103   
 0.001 0    
 0.002 2000   0.003 2000  
 
 e 4 e 6 
0.002 y  1  e 
0.001 y 3
 6 10     0.05
 ke    0.002 0.003 
 0.001 
 6 103 e 0.002 y 1  e 0.001 y  for y  0

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Mean & Variance of a Marginal Distribution
E(X) and V(X) can be obtained by first calculating the marginal
probability distribution of X and then determining E(X) and V(X) by
the usual method.

E  X    x  fX  x
R

V  X    x 2  f X  x    X2
R

E  Y    y  fY  y 
R

V Y    y 2  fY  y   Y2
R

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Mean & Variance for Example 5-1
y = Response x = Number of Bars
time(nearest of Signal Strength f (y ) y *f (y ) y 2*f (y )
second)
1 2 3
1 0.01 0.02 0.25 0.28 0.28 0.28
2 0.02 0.03 0.20 0.25 0.50 1.00
3 0.02 0.10 0.05 0.17 0.51 1.53
4 0.15 0.10 0.05 0.30 1.20 4.80
f (x ) 0.20 0.25 0.55 1.00 2.49 7.61
x *f (x ) 0.20 0.50 1.65 2.35
x 2*f (x ) 0.20 1.00 4.95 6.15

E(X) = 2.35 V(X) = 6.15 – 2.352 = 6.15 – 5.52 = 0.6275

E(Y) = 2.49 V(Y) = 7.61 – 2.492 = 7.61 – 16.20 = 1.4099

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Conditional Probability Density Function

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-6: Conditional Probability-1
From Example 5-2, determine the conditional PDF for Y given X=x.

f X  x    k  e 0.001x 0.002 y dy
x

 e 0.002 y  
 ke 0.001x  
 0.002 x 
 
0.001 x  e 
0.002
 ke  
 0.002 
 0.003e 0.003 x for x  0
f XY  x, y  ke 0.001x 0.002 y
fY x  y   
f X ( x) 0.003e 0.003 x
 0.002e0.002 x 0.002 y for 0  x and x  y

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-6: Conditional Probability-2
Now find the probability that Y exceeds 2000 given that X=1500:

P Y  2000 X  1500 

  fY 1500  y  dy
2000

  0.002e0.0021500 0.002 y
2000

 e 0.002 y  
 0.002e3  
 0.002 2000 
 
 e 4
 1
 0.002e 
3
  e  0.368
 0.002 

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Mean & Variance of Conditional Random Variables

• The conditional mean of Y given X = x,

denoted as E(Y|x) or μY|x is

E  Y x    y  fY x  y 
y

• The conditional variance of Y given X = x,

denoted as V(Y|x) or σ2Y|x is


V Y x    y  Y x   fY x  y    y 2  fY x  y   Y2 x
2

y y

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-8: Conditional Mean And Variance
From Example 5-2 & 5-6, what is the conditional mean for
Y given that x = 1500?
 
E Y X  1500    y  0.002e 0.0021500   0.002 y
dy  0.002e 3
 y  e 0.002 y dy
1500 1500

 e 0.002 y  
 e 0.002 y  
 0.002e  y
3
    dy 
  0.002 1500 1500 
 0.002  
  0.002 y


3  1500 3 e
 0.002e e  
 0.002   0.002  0.002  1500  
  

3 1500 3 e 3 
 0.002e  e  
 0.002  0.002  0.002 
3  e 
3
 0.002e   2000   2000
 0.002 
If the connect time is 1500 ms, then the expected time to be authorized is 2000 ms.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-9
For the discrete random variables in Exercise 5-1,
what is the conditional mean of Y given X=1?
y = Response x = Number of Bars
time(nearest of Signal Strength f (y )
second) 1 2 3
1 0.01 0.02 0.25 0.28
2 0.02 0.03 0.20 0.25
3 0.02 0.10 0.05 0.17
4 0.15 0.10 0.05 0.30
f (x ) 0.20 0.25 0.55 y*f(y|x=1) y2*f(y|x=1)
1 0.050 0.080 0.455 0.05 0.05
2 0.100 0.120 0.364 0.20 0.40
3 0.100 0.400 0.091 0.30 0.90
4 0.750 0.400 0.091 3.00 12.00
Sum of f(y|x) 1.000 1.000 1.000 3.55 13.35
12.6025
0.7475

The mean number of attempts given one bar is 3.55 with variance of 0.7475.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Independent Random Variables
For random variables X and Y, if any one of the
following properties is true, the others are also true.
Then X and Y are independent.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-11: Independent Random Variables
• Suppose the Example 5-2 is modified such that the joint
PDF is:
f XY  x, y   2 10 e
6 0.001x 0.002 y
for x  0 and y  0.

• Are X and Y independent?

 

f X  x    2 10 e
6 0.001 x  0.002 y
dy fY  y    2 106 e0.001x 0.002 y dx
0 0

 0.001e0.001x for x  0  0.002e0.002 y for y > 0

• Find the probability

P  X  1000, Y  1000   P  X  1000   P Y  1000 

 e1  1  e2   0.318

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Joint Probability Density Function

The joint probability density function for the continuous

random variables X1, X2, X3, …Xp, denoted as
f  x , x ,..., x  satisfies the following properties:
X1 X 2 ... X p 1 2 p

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-14: Component Lifetimes
In an electronic assembly, let X1, X2, X3, X4 denote
the lifetimes of 4 components in hours. The joint
PDF is:
f X1 X 2 X3 X 4  x1 , x2 , x3 , x4   9 1012 e0.001x1 0.002 x2 0.0015 x3 0.003x4 for xi  0

What is the probability that the device operates

more than 1000 hours?
The joint PDF is a product of exponential PDFs.
P(X1 > 1000, X2 > 1000, X3 > 1000, X4 > 1000)
= e-1-2-1.5-3 = e-7.5 = 0.00055

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Marginal Probability Density Function

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Mean & Variance of a Joint Distribution
The mean and variance of Xi can be
determined from either the marginal PDF, or
the joint PDF as follows:

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-16
Points that have positive probability in the
joint probability distribution of three random
variables X1 , X2 , X3 are shown in Figure.
Suppose the 10 points are equally likely
with probability 0.1 each. The range is the
non-negative integers with x1+x2+x3 = 3

List the marginal PDF of X2

P (X2 = 0) = f x1x2 x3(3,0,0) + f x1x2 x3(0,0,3) + f x1x2 x3 (1,0,2) + f x1x2 x3(2,0,1) = 0.4
P (X2 = 1) = f x1x2 x3(2,1,0) + f x1x2 x3(0,1,2) + f x1x2 x3 (1,1,1) = 0.3
P (X2 = 2) = f x1x2 x3(1,2,0) + f x1x2 x3(0,2,1) = 0.2
P (X2 = 3) = f x1x2 x3(0,3,0) = 0.1

Also, E(x2) = 0(0.4) + 1(0.3) + 2(0.2) + 3(0.1) = 1

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Distribution of a Subset of Random Variables

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Conditional Probability Distributions
• Conditional probability distributions can be
developed for multiple random variables by
extension of the ideas used for two random
variables.
• Suppose p = 5 and we wish to find the
distribution conditional on X4 and X5.
f X1 X 2 X 3 X 4 X 5  x1 , x2 , x3 , x4 , x5 
f X1 X 2 X 3 X 4 X 5  x1 , x2 , x3  
f X 4 X 5  x4 , x5 
for f X 4 X 5  x4 , x5   0.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Independence with Multiple Variables
The concept of independence can be extended to
multiple variables.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-18: Layer Thickness
Suppose X1,X2, and X3 represent the thickness in μm of a
substrate, an active layer and a coating layer of a chemical
product. Assume that these variables are independent and
normally distributed with parameters and specified limits as
tabled.
Parameters Normal
What proportion of the product and specified Random Variables
meets all specifications? limits X1 X2 X3
Answer: 0.7783, 3 layer product. Mean (μ) 10,000 1,000 80
Std dev (σ) 250 20 4
Lower limit 9,200 950 75
Which one of the three
Upper limit 10,800 1,050 85
thicknesses has the least
P(in limits) 0.99863 0.98758 0.78870
probability of meeting specs?
P(all in limits) = 0.77783
Answer: Layer 3 has least prob.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Covariance
• Covariance is a measure of the relationship
between two random variables.
• First, we need to describe the expected value
of a function of two random variables. Let
h(X, Y) denote the function of interest.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-19: Expected Value of a Function of Two Random Variables

For the joint probability distribution of the two random variables in

Example 5-1, calculate E [(X-μX)(Y-μY)].

The result is obtained by multiplying x - μX times y - μY, times fxy(X,Y)

for each point in the range of (X,Y). First, μX and μy were determined
previously from the marginal distributions for X and Y:

μX = 2.35 and μy = 2.49

Therefore,

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Covariance Defined

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Correlation (ρ = rho)

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-21: Covariance & Correlation
Determine the covariance x y f(x, y) x-μX y-μY Prod
and correlation to the figure 0 0 0.2 -1.8 -1.2 0.42
1 1 0.1 -0.8 -0.2 0.01
below. 1 2 0.1 -0.8 0.8 -0.07

Joint
2 1 0.1 0.2 -0.2 0.00
2 2 0.1 0.2 0.8 0.02
3 3 0.4 1.2 1.8 0.88
0 0.2 covariance = 1.260
1 0.2 correlation = 0.926
2 0.2

Marginal
3 0.4 Note the strong
0 0.2 positive correlation.
1 0.2
2 0.2
3 0.4
Mean

μX = 1.8
μY = 1.8
Figure 5-13 Discrete joint
σX = 1.1662
StDev

distribution, f(x, y).

σY = 1.1662

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Independence Implies ρ = 0
• If X and Y are independent random
variables,
σXY = ρXY = 0

• ρXY = 0 is necessary, but not a sufficient

condition for independence.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-23: Independence Implies Zero Covariance
Let f XY  xy   x  y 16 for 0  x  2 and 0  y  4
Show that  XY  E  XY   E  X   E Y   0
1  2 2 
4 2

1  2
4 2
 E  XY      x y dx dy
EX  
16 0  0
 x ydx dy 16 0  0 

1
4  3 2

x  2 x
1
4 3 2
 y  dy
  y dy 16 0  3 0
16 0  3 0   
 
8 
4
1
1  y 2   8  1 16 4
4
  y 2   dy
      16 0  3 
16  2 0   3  6 2 3
 
1  y3  1 64 32
4

1 
4 2
     
E Y      xy 2 dx dy 6 3
 0

6 3 9 Figure 5-15 A planar
16 0  0  joint distribution.
 2 2
 XY  E  XY   E  X  .E Y 
4
1 2 x
 y  dy
16 0  2 0 
  32 4 8
   0
2  y 3  1 64 8 9 3 3
4

    
16  3 0  8 3 3
 

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Multinomial Probability Distribution
• Suppose a random experiment consists of a series of n trials.
Assume that:
1) The outcome of each trial can be classifies into one of k
classes.
2) The probability of a trial resulting in one of the k outcomes is
constant, and equal to p1, p2, …, pk.
3) The trials are independent.
• The random variables X1, X2,…, Xk denote the number of
outcomes in each class and have a multinomial distribution
and probability mass function:

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-25: Digital Channel
Of the 20 bits received over a digital channel, 14 are of excellent
quality, 3 are good, 2 are fair, 1 is poor. The sequence received was
EEEEEEEEEEEEEEGGGFFP. Let the random variables X1 , X2 , X3,
and X4 denote the number of bits that are E, G, F , and P, respectively,
in a transmission of 20 bits. What is the probability that 12 bits are E, 6
bits are G, 2 are F, and 0 are P?

20!
P  X 1  12, X 2  6, X3  2, X 4  0   0.6120.360.0820.020  0.0358
12!6!2!0!

Using Excel
0.03582 = (FACT(20)/(FACT(12)*FACT(6)*FACT(2))) * 0.6^12*0.3^6*0.08^2

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Multinomial Mean and Variance
The marginal distributions of the multinomial
are binomial.

If X1, X2,…, Xk have a multinomial distribution,

the marginal probability distributions of Xi is
binomial with:

E(Xi) = npi and V(Xi) = npi(1-pi)

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Bivariate Normal Probability Density Function
The probability density function of a bivariate normal distribution is

1
f XY  x, y;  X ,  X , Y ,  Y ,    eu
2 X  Y 1   2

  x   X  2   x   X  y  Y   y  Y  
2 2
1
where u     
2 1      X  XY  Y 
2 2 2

for    x   and    y  .

 x  0,    x  ,
Parameter limits:  1    1
 y  0,    y  ,

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Marginal Distributions of the Bivariate Normal Random Variables

If X and Y have a bivariate normal distribution with

joint probability density function

fXY(x,y;σX,σY,μX,μY,ρ)

the marginal probability distributions of X and Y

are normal with means μX and μY and standard
deviations σX and σY, respectively.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Conditional Distribution of Bivariate Normal Random Variables

If X and Y have a bivariate normal distribution with

joint probability density fXY(x,y;σX,σY,μX,μY,ρ), the
conditional probability distribution of Y given X = x is
normal with mean and variance as follows:

Y
Y x  Y    x  X 
X

 Y2 x   Y2 1   2 

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Correlation of Bivariate Normal Random Variables

If X and Y have a bivariate normal

distribution with joint probability density

function fXY(x,y;σX,σY,μX,μY,ρ), the

correlation between X and Y is ρ.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Bivariate Normal Correlation and Independence

• In general, zero correlation does not imply

independence.
• But in the special case that X and Y have
a bivariate normal distribution, if ρ = 0,
then X and Y are independent.

If X and Y have a bivariate normal

distribution with ρ=0, X and Y are
independent.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Linear Functions of Random Variables
• A function of random variables is itself a
random variable.
• A function of random variables can be
formed by either linear or nonlinear
relationships. We limit our discussion here
to linear functions.
• Given random variables X1, X2,…,Xp and
constants c1, c2, …, cp
Y= c1X1 + c2X2 + … + cpXp
is a linear combination of X1, X2,…,Xp.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Mean and Variance of a Linear Function

If X1, X2,…,Xp are random variables, and Y= c1X1 + c2X2 +

… + cpXp , then

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-31: Error Propagation
A semiconductor product consists of three layers.
The variances of the thickness of each layer is 25,
40 and 30 nm. What is the variance of the finished
product?

Answer:

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Mean and Variance of an Average

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Reproductive Property of the Normal Distribution

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-32: Linear Function of Independent Normal Random variables

Let the random variables X1 and X2 denote Parameters of

the length and width of a manufactured X1 X2
Mean 2 5
part. Their parameters are shown in the Std Dev 0.1 0.2
table. What is the probability that the
perimeter exceeds 14.5 cm?
Let Y  2 X 1  2 X 2  perimeter
E Y   2 E  X 1   2 E  X 2   2  2   2  5   14 cm
V Y   22V  X 1   22V  X 2   4  0.1  4  0.2   0.04  0.16  0.20
2 2

SD Y   0.20  0.4472 cm
 14.5  14 
P Y  14.5   1      1   1.1180   0.1318
 .4472 
Using Excel
0.1318 = 1 - NORMDIST(14.5, 14, SQRT(0.2), TRUE)

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
General Function of a Discrete Random Variable

Suppose that X is a discrete random

variable with probability distribution fX(x).
Let Y = h(X) define a one-to-one
transformation between the values of X
and Y so that the equation y = h(x) can be
solved uniquely for x in terms of y. Let this
solution be x = u(y), the inverse transform
function. Then the probability mass
function of the random variable Y is
fY(y) = fX[u(y)]

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Example 5-34: Function of a Discrete Random Variable

Let X be a geometric random variable with probability

distribution
fX(x) = p(1-p)x-1 , x = 1, 2, …
Find the probability distribution of Y = X2.
Solution:
– Since X ≥ 0, the transformation is one-to-one.
– The inverse transform function is X = y .
– fY(y) = p(1-p) y -1 , y = 1, 4, 9, 16,…

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
General Function of a Continuous Random Variable

Suppose that X is a continuous random variable

with probability distribution fX(x). Let Y = h(X)
define a one-to-one transformation between the
values of X and Y so that the equation y = h(x) can
be solved uniquely for x in terms of y. Let this
solution be x = u(y), the inverse transform
function. Then the probability distribution of Y is

fY(y) = fX[u(y)]∙|J|

where J = u’(y) is called the Jacobian of the

transformation and the absolute value of J is used.

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-35: Function of a Continuous Random Variable

Let X be a continuous random variable with probability

distribution: x
f X ( x)  for 0  x  4
8
Find the probability distribution of Y = h(X) = 2X + 4

Note that Y has a one-to-one relationship to X .

y4 1
x  u  y  and the Jacobian is J  u '  y  
2 2

fY  y  
 y  4 2 1 y  4
  for 4  y  12.
8 2 32

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Definition of Moments about the Origin

The rth moment about the origin of the

random variable X is

 X r f ( x), X discrete

 'r  E ( X )  
r


r
X f ( x)dx, X continuous
 

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Definition of a Moment-Generating Function
The moment-generating function of the random variable X is
the expected value of etX and is denoted by MX (t). That is,

 etX f ( x), X discrete


M X (t )  M (e )   tX
tX

  e f ( x)dx, X continuous


Let X be a random variable with moment-generating

function MX (t). Then

d r M X (t )
 'r  r
|t  0
dt

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-36 Moment-Generating Function for a Binomial Random Variable-1
n
Let X follows a binomial distribution, that is f ( x)    p x (1  p)n x , x  0,1,...., n
 x

Determine the moment generating function and use it to verify that the
mean and variance of the binomial random variable are μ=np and
σ2=np(1-p).
The moment-generating function is
n
n x n
n t x
M X (t )   e   p (1  p)    ( pe ) (1  p) n  x
tx n x

x 0  x x 0  x 

which is the binomial expansion of

[ pet  (1  p)]n

Now the first and second order derivatives will be

M x' (t )  npet [1  p(et  1)]n 1 and
M x'' (t )  npet (1  p  npet )[1  p(et  1)]n 2

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-36 Moment-Generating Function for a Binomial Random Variable-2

If we set t = 0 in the above two equations we get

M x' (t )  1'    np and

M x'' (t )  2'  np(1  p  np)

Now the variance is

 2  2'   2  np(1  p  np)  (np)2

 np  np 2
 np(1  p)

Hence, the mean is   np and variance is  2  np(1  p).

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Properties of Moment-Generating Function

If X is a random variable and a is a constant, then

1. M X  a (t )  e at M X (t )
2. M aX (t )  M X (at )
If X 1 , X 2 ,..., X n are independent random variables with
moment generating functions M X1 (t ), M X 2 (t ),..., M X n (t )
respectively, and if Y  X 1  X 2  ...  X n then the moment
generating function of Y is
3. M Y (t )  M X1 (t ).M X 2 (t ). ... .M X n (t )

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
Example 5-38 Distribution of a Sum of Poisson Random Variables
Suppose that X1 and X2 are two independent Poisson random variables
with parameters λ1 and λ2, respectively. Determine the probability
distribution of Y = X1 + X2.

The moment-generating function of a Poisson random variable with

parameter λ is
M X (t )  e ( e 1)
t

1 ( et 1) 2 ( et 1)
Hence for X1 and X2, M X1 (t )  e and M X 2 (t )  e
Using M Y (t )  M X1 (t ).M X 2 (t ). ... .M X n (t ) , the moment-generating
function of Y = X1 + X2 is
M Y (t )  M X1 (t ).M X 2 (t )
1 ( et 1) 2 ( et 1)
e e
( 1  2 )( et 1)
e
Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.
 Important Terms & Concepts for Chapter 5
Bivariate distribution
Bivariate normal distribution General functions of random
Conditional mean variables
Conditional probability density Independence
function Joint probability density function
Conditional probability mass Joint probability mass function
function Linear functions of random
Conditional variance variables
Contour plots Marginal probability distribution
Correlation Multinomial distribution
Covariance Reproductive property of the
Error propagation normal distribution

Copyright © 2014 John Wiley & Sons, Inc. All rights reserved.