HIFI moment

generating functions
probability generating functions
orelationshiptofrequencytransfoms
wc.ie

of probability functions for multiple r.ve's

An especially important class of functions are sums e
g Z Y

Start from the discrete case where X and Y are independent with
pmts Px and Py respectively Then ZEE

Pz.CZ PCE z x.Y y

ty.gg

IPC x̅ x FZ x x
p z x

For those of you who took EE 202 signals systems you may recall
this as the convolution operator

f g n
Es f n k g n f g t
if t s g s as

Likewise for independent continuous r v s X Y with pdfs fx fy

respectively and Z Y note that
P Z z x̅ x IP Y Z x̅ x

P x YE z P YE z x
Fy z x

Fy z x z f x
fz.ly Zlx f lx fy z x

fz z
ff z x z dx f x f z x dx
 which is another convolution

independent normal x̅ N Mx 0 2 and Y N My Ty

Example v.v s

Then Z Y has pdf

tale f.ir expL

EtfJ.EgexpEEffax
go through the tedious calculation details here but
We won't it turns

out that

I n
mail.tt tEEIE
Example difference between two i i d exponential r.ve's X Y Exp X
Note that f g y f y for the exponential distribution so we
can treat x̅ 5 like the sum between x̅ and Y

fx y
z
f x z x dx
f f x f x z dx

xe tx.net ax

Ietf via straightforward

calculation
while this holds for 220 we can compute the Z o case via similar

calculation
fy z f x y
Z
x y Z

You may realize that for more complex distributions computing the

convolution sum or
integral can become quite tedious In the rest of

today's lecture we will introduce a new

way to look at distributions
expectations and higher moments
 samation
contained in a
sequence 19 3 of real
numbers is via a
generating function
Ga s
E a S SER sit sum
converges

Note the sequence a 3 can now be reconstructed from Ga via

ith derivative
a E Gas

We can generating functions to simplify the convolution process

even use

seen before Consider two sequences a and b 3 Let c 3 be

the sequence formed via convolution
c E a b do b a b e t a_bo
Then the generating functions can be related as

G s cns
E E a b e s

a S bn i 5 Gals Gb S

Thus c a b implies G GaGb i e the original convolution becomes

a
generator function multiplication Same of you may remember a
similar fact from
signals systems convolution in the time domain
yields multiplication in the frequency domain

Def The plabilityneratingfunction Cpf of discrete v.v x̅ is

defined to be the generating function

FETIEL of its pmf
For general r.v s including continuous case if is more common to

substitute s et
Examples
discrete ru x̅ taking values in 0,1 2

Then G s
If skp k Note that G o 0 and G 1 1

Bmolk v.v is a special case of above

G s 1
p so p s
ftp.t
Geometric v.v Geom p

G Ésk E Get p
_IpI
s 1
p p ps L

by geometric series formula

Poisson v.v x̅ Pois X

G s
Ésk e e
IT by Taylor series

Theorems If x̅ has pgf G s then E x̅

f G

This can be directly verified via the pgf formula

G s
If skpy.lk G s
Efksk p k

Gx 1
fkP K E E X

by definition

Def The momentgeneratingfunction mgt of a v.v x̅ is the

function M IR 11230 defined by My t

Tf
EEfet
In the discrete v.v case this formula becomes
 My t e px x

while for continuous v.v s

My t
fet f x dx

you may notice the resemblance between this formula and the Laplace
transform from EE 202 For this reason generating functions are also
often called transforms

If you are more computing higher moments of X

interested in

rather than its probability distribution the mgt may be more useful
than the pgf The change of variable sheet can be motivated for
the discrete rv case the series of My t
by expanding power
Mx.CH
E et k
E k

É E kpx.sk I E x̅

FEES 2
which is the exponential generating function of x E x̅ I

Examples
Esson r v x̅ Pois X

Mx t
E et py.lk E.tk If
e
E.ly e xexet fexEy
Following I the moments of x̅ can now be computed via
taking
the derivative
 M LH
I E x̅ IE X IE x̅ ÉE x̅ E x

M t IE x̅ E X EE 3
MxCo E X

4
E M t E X E X E 2M o E x̅

E X4
M t E X E X
1M o E x

In general neIN
ÉM t
o
E x̅T
Examplescontinued continnousr.v.tt
Exponential r v x̅ Exp X

Mx t
f.be xe xxdx
xfet xxdx
it
e

Note that we require tox so that the integral doesn't ta

To obtain the moments for a continuous r.us from its mgt we have
a similar process
µ t
fet x dx

My t
fetx f x dx
fxe x dx

My o
fx f x dx
fxf x dx E X

1 Mx t x etxf x dx
1 Mx o x f x dx E E X

In general Unein
f iMEfx̅If t
o
 Note that in our derivation above we interchanged the derivative and
the integral For example

fetx f x dx et f x dx
fxet f x dx

This is not something you can always do Leibniz integral rule see

However for most applications where the limits of

integration have
nothing to do with the variable you are differentiating with respect to
this interchange may be performed

Now we address the main concern posed to us at the start of this

lecture sums of r v s and convolutions of pmfs pdfs

Theorem Leartransformations Let My t be the

mgt associated
with v.v x̅ Then the mgt associated with YE a b is
given
a bt
My E IE e ebtIE eat ebt M at Ha belR 2

Theorems Sums If X Y are independent with respective mgt

Mx My then M s M s M s 3

Note that pgfs G also satisfy properties

2 and 3

Def For two discrete v.v s x̅ Y which take values in 79 the

jointpf is defined
by
E E SET
Gxy s t s t ER
This definition can be extended to an arbitrary number of v.v s

Theory Discrete v.v s x̅ which take values in are independent

if Gxy s t
G s
Gy t s t ER
Examplescontinued sums of v.v s

Binomial v.v 8 Bin n

p
let Xi X2 In Bem p iid Then each has generating
function G s E 1 p
ps
By nature of the Binomial v v we can write S x̅ t In
Thus applying 3 repeatedly yields

Gs s
G s 1 p
ps
Moreover the sum of two Binomial vv s S Bin n
p S2 Bin m
p
with the same
p has generating function
ⁿtm
Gsits s 1 p
ps

which is the same as Sit 52 Bin ntm p i.e adding m more

Bernoulli trials after the first n Bernoulli trials

Normal v.v x̅ NCM 02

First let's compute the mgt of the standard normal Y N 0,1

My t
fitz e ets dy

tze.lk ttyd
etkf é dy by completing
the square
Thisisthepf
of distributed
a v.v
like N t 1
 eᵗ by normalization probability axiom

Thus by 2 the mgt of x̅ which is

equal to rYtM is
given

tmt7
et tht
M t IE eat letting eat my a eᵗᵗ
Now we can compare the result from the example at thestartof
today's lecture with the mgt characterization method Suppose we

have two independent normal v.v s x̅ N Mx 0 2 and Y N My Ty

Their mgts are
Met Myt
My t My t

Then by 3 the mgt of Z Y is

t Mx that
Mz My t My t

In conclusion mgts pgfs provide a much simpler characterization

of riv s and functions of ru s Namely it does not require one
to compute complicated integrals of pdfs