Chapter 5 Functions of Random Variables

Chapter 5
Functions of Random Variables

Contents
 5-1. Introduction
 5-2. Transformation of Variables
 5-3. Moment Generating Functions
 5-4. Characteristic Functions
 5-5. Limit and Convergence
 5-6. Important Theorems
5-1 Introduction
 Many statistical methods involve functions of one or more

random variables. Statistical inference requires the
distributions of these functions, which are usually
obtained through transformations of variables.
 The moment generating functions will be introduced for
determining moments of distributions and for establishing
distributions of functions of random variables.
 The moment generating functions are then generalized
into the characteristic functions, which enable us to
prove both the Weak Law of Large Numbers and the
Central Limit Theorem.
5-2 Transformations of Variables
 Given fX (x) and Y = g(X), find fY (y).

 Start by determining the cumulative distr. function of Y:
FY ( y )  P (Y  y )  P ({ X : g ( X )  y})
 P({ X : X  g 1 (( , y ])})
Then   f X ( x) dx
{ x: g ( x ) y}
dFY ( y )
fY ( y )  .
dy
1-1 Mapping
 If Y = g(X) is 1-1 mapping, then
dx
fY (y) = fX (x) , where x  g 1 ( y ).
dy
 If Y = f (X) is 1-1 mapping and X is discrete, then
1
f (y) = f ( g ( y )).
Y X
[Example] Linear Function Y = a X + b :

1 y b
fX ( ).
fY (y) = a a
Example (Power-Law Function)
 Consider Y = X 2:
P (Y  y )  P ( X 2  y )
 P( y  X  y)
 FX ( y )  FX ( y ).
By differentiation,
1
fY ( y )  [ f X ( y )  f X (  y )].
2 y
Non-1-1 Mapping
 If Y = g(X)
dx1 dx2
pY ( y )  p X ( x1 )  p X ( x2 ) , where xi  g 1 ( y ).
dy dy
Simulating A Random Variable
Simulating A Random Variable
Example
Example
Theorem
Example
Sol:
5-3 Moment Generating Functions
 The moment generating functions will be introduced for

determining moments of distributions and for establishing
distributions of functions of random variables.
 The expected value of a function g(X) = X r, for r = 0,1,2,…,
is called the r-th moment (about the origin) of X.
Moment Generating Function
 Although the moments of X can be determined directly
from the definition, there exists an alternative way
utilizing the moment generating function.
 If the moment generating function of X does exist, it can

be used to generating all the moments of X.
Theorem
[Example] Find the moment generating function of the
binomial R. V. X and verify that   np and  2  np (1  p).
(Solution)
Moment Generating Functions
 The moment generating function of X having a normal

distribution with mean  and variance 2 is given by
M X (t )  exp(  t  12  2t 2 ).
 The moment generating function of X having a chi-squared
distribution with  degrees of freedom is given by
M X (t )  (1  2 t )  2 .

M X  a (t )  e at M X (t ), M aX (t )  M X (at ).
 If Y = X1 + X2 + … + Xn, where Xi are independent R. V.s, then
M Y (t )  M X1 (t ) M X 2 (t ) M Xn (t ).
Theorems
5-4 Characteristic Functions
 The characteristic function of X is defined as

 X (t )  E[eitX ]  E[cos(tX )]  iE[sin(tX )]

= M X (it )   eitX f X ( x)dx.

 Given a characteristic function  X (t ), we can recover the

pdf f X ( x) through the (inverse) Fourier transform:
1 
f X ( x) =  e itX  X (t )dt.
2
Moments from Characteristic function
k
k 1 d  X (t )
E[ X ]  k .
i d tk t 0
It can be shown that the characteristic function of X having

a normal distribution with mean  and variance  2 is given by
it   2t 2 2
 X (t )  e e ,
which can be derived from the moment generating function
M X (it ) of a normal X.
5-5 Limit and Convergence
 In probability theory, there exist several different notions of

convergence of random variables. The convergence of
sequences to some limit random variable is an important
concept in probability theory, and its applications.
 The concepts formalize the idea that a sequence of random
events can sometimes be expected to settle down into a
behavior that is essentially unchanging when items far
enough into the sequence are studied.
 The different notions of convergence relate to how such a
behavior can be characterized; two readily understood
behaviors are that the sequence eventually takes a constant
value, and that values in the sequence can be described by
an unchanging probability distribution.
Modes of Convergence
 lim X  X defines the limit of a sequence of R.V.s.

n
n
There exist, however, several different ways (modes) to

interpret the convergence to a limit.
 Convergence with probability 1
P ( lim X n  X ) = P({ : lim X n ( )  X ( ),   })  1.
n n
This mode of convergence is also known as an almost

sure (a.s.) convergence. It converges almost
everywhere except for a set of events with probability
zero.
Modes of Convergence
 Convergence in probability
lim P ( X n  X > ) = 0,   0
n
 Convergence in mean square

2
lim E ( X n  X ) = 0
n
 Convergence in distribution
lim FX n ( x)  FX ( x)
n
i.e., the sequence of FX n ( x) converges pointwise toFX ( x).

Examples
 Convergence in distribution of a binomial R.V.

to a Gauss R.V.
 Convergence of the Poisson distribution to the
Gauss distribution.
If X n 
n Z
 , then Xn converges to Z with
n 1
probability 1 (as long as P ( Z  )  1 ), and

converges in mean square to Z if E(Z2) is finite.
Convergence Relationship
 The statement convergence with probability 1 is

stronger than (implies) the statement
convergence in probability.
 Convergence in mean square also implies
 The weakest mode of convergence is
convergence in distribution, which is implied by
5-6 Important Theorems
 If two random variables have the same characteristic

function, they have the same distribution function.
 Continuity Theorem: The convergence of characteristic
functions (of a sequence of R.V.s) implies convergence of
the corresponding distribution functions.
 Weak Law of Large numbers:
Let X1, X2,…,be independent, identically distributed R.V.s
having finite mean  and set Sn=X1+X2+…+Xn. Then for any
>0 Sn
lim P (     )  0. (converge in proba.)
n n
Central limit theorem
Let X1, X2,…,be independent, identically distributed

R.V.s having mean  and nonzero variance 2. Then
S n  n
lim P(  x)   ( x), (converge in distr.)
n  n
where  ( x) is the standard normal distribution function.
If we are sampling from a population with unknown distr.,
X (sample mean)
the sampling distribution of will still be
approximately normal provided that the sample size is large
(n 30).

Chapter 5 Functions of Random Variables

Uploaded by

Copyright:

Available Formats

Chapter 5 Functions of Random Variables

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 5 Functions of Random Variables

Uploaded by

Copyright:

Available Formats

Chapter 5

Functions of Random Variables

 Many statistical methods involve functions of one or more

 Given fX (x) and Y = g(X), find fY (y).

[Example] Linear Function Y = a X + b :

 The moment generating functions will be introduced for

 If the moment generating function of X does exist, it can

 The moment generating function of X having a normal

 The characteristic function of X is defined as

 Given a characteristic function  X (t ), we can recover the

It can be shown that the characteristic function of X having

 In probability theory, there exist several different notions of

 lim X  X defines the limit of a sequence of R.V.s.

There exist, however, several different ways (modes) to

This mode of convergence is also known as an almost

 Convergence in mean square

i.e., the sequence of FX n ( x) converges pointwise toFX ( x).

 Convergence in distribution of a binomial R.V.

probability 1 (as long as P ( Z  )  1 ), and

 The statement convergence with probability 1 is

 If two random variables have the same characteristic

Let X1, X2,…,be independent, identically distributed

You might also like