Expectation and Generating Functions.

1 Introduction.
When a large collection of observations is assembled, we are usually inter-
ested not in the any individual observation, but rather in some quantity by
which we can represent the complete set of observation such as some central
value (arithmetic mean, median etc.). Similarly, in case of the random vari-
able we are interested in value around which the probability distribution of
the random variable is cantered. The mean of the probability distribution
of given random variable is termed as its expectation.

Expectation should always be some finite quantity, as there is no meaning

of infinite expectation. To ensure this we required integrability/summability
of the function whose expectation.

Let X be a Borel measurable function, let

X + = max(X, 0) and X − = min(−X, 0),

X + and X − are known as positive part and negative parts of X respectively.

Therefore X = X + − X − . Also | X |= X + + X − , X + and X − are Borel
measurable functions.

R
Definition 1.1. A function X is said to be integrable iff | X | dP < ∞,
i.e., iff X + dP and X − dP are both finite.
R R

Definition 1.2. Let X be an non-negative, extended, real valued, measurable

function on the measure space (Ω, F, P ), i.e., X be a random variable defined
on the probability space (Ω, F, P ). Then
Z X
E[X] = xdP = xP [X = x]
x∈Ω x∈Ω

provided, Z X
| x | dP < ∞ or | x | P [X = x] < ∞.
x∈Ω x∈Ω

1
 Example. Let in a coin tossing experiment, we define a random variable
X takes value 1 if heads turns up and takes value 0 in case of tails. Let
probability of heads turns up in p. Then

E[X] = 1 · P [X = 1] + 0 · P [X = 0] = p.

Example. Let X be a uniform random variable distributed on the interval

[0; 1]. Then
Z 1
E(X) = xdx = 1/2.
0

1.1 Basic Properties of Expectation.

Theorem 1.1. Let X be a random variable such that X ≥ a, where a is a
constant. Then E[X] ≥ a.
Proof. Given X ≥ a, then
Z
E[X] = xfX (x)dx
x∈ΩX
Z
≥ a fX (x)dx = a.
x∈ΩX

Theorem 1.2. Let X be a random variable and Y = aX + b, where a and

b are constants. Then E[Y ] = aE[X] + b.
Proof. Given Y = aX +b, and let Y, X are continuous random variables.
Then

E[Y ] = E[aX + b]
Z
= (ax + b)fX (x)dx
Zx∈Ω Z
= axfX (x)dx + bfX (x)dx
x∈Ω x∈Ω
Z Z
= a xfX (x)dx + b fX (x)dx
x∈Ω x∈Ω
R
= aE[X] + b, since x∈Ω fX (x)dx = 1.

Similarly we can prove in case of discrete varables.

Theorem 1.3. Let X and Y are two random variables. Then E[X + Y ] =
E[X] + E[Y ].

2
 Proof. Let Y, X are continuous random variables. Then
Z Z
E[X + Y ] = (X + Y )fXY (x, y)dydx
X∈ΩX Y ∈ΩY
Z Z
= XfXY (x, y)dydx
X∈ΩX Y ∈ΩY
Z Z
+ Y fXY (x, y)dydx
X∈ΩX Y ∈ΩY
Z Z
= X fXY (x, y)dydx
X∈ΩX Y ∈ΩY
Z Z
+ Y fXY (x, y)dydx
ZX∈ΩX Y ∈ΩY Z
= XfX (x)dx + Y fY (y)dy
X∈ΩX Y ∈ΩY
= E[X] + E[Y ].

Theorem 1.4. Let X and Y are two random variables and let X ≥ Y. Then
E[X] ≥ E[Y ].
Proof. Given X ≥ Y, then

X −Y ≥ 0
⇒ E[X − Y ] ≥ 0
⇒ E[X] − E[Y ] ≥ 0
⇒ E[X] ≥ E[Y ].

Theorem 1.5. Let X is a random variable. Then | E[X] |≤ E | X | .

Proof. Since X ≤| X | and −X ≤| X |, then from the previous result
we have
E[X] ≤ E[| X |], (1)
and
−E[X] = E[−X] ≤ E[| X |]. (2)
Now from equation (1) and (2), we have

| E[X] |≤ E | X | .

Theorem 1.6. Let X and Y are two independent random variables. Then
E[XY ] = E[X] · E[Y ].

3
 Proof. Given X and Y are independent random variables. Then
Z Z
E[XY ] = XY fXY (x, y)dydx
X∈ΩX Y ∈ΩY
Z Z
= XY fX (x)fY (y)dydx
X∈ΩX Y ∈ΩY
since X and Y are independent.
Z Z
= XfX (x)dx · Y fY (y)dy.
X∈ΩX Y ∈ΩY
= E[X] · E[Y ].

Theorem 1.7. Let E[XY ] = E[X] · E[Y ]. Then it is not necessary that X
and Y are two independent random variables.

Proof. To prove the above statement, we are using following counter

example. Let X and Y are two random variables having their joint pmf as
follows:
X \Y -1 0 1 P(X=x)
-1 0 1/4 0 1/4
0 1/4 0 1/4 1/2
1 0 1/4 0 1/4
P(Y= y) 1/4 1/2 1/4 1

E[X] = 1 · P [X = 1] + 0 · P [X = 0] + (−1) · P [X = −1] = 0,

E[Y ] = 1 · P [Y = 1] + 0 · P [X = 0] + (−1) · P [Y = −1] = 0,

E[X · Y ] = 1 · [P [X = 1, Y = 1] + P [X = −1, Y = −1]]

+ (−1) · [P [X = −1, Y = 1] + P [X = 1, Y = −1]]
+ 0 · [P [XY = 0]]
= 1 · 0 + (−1) · 0 + 0 · [1/4 + 1/4 + 1/4 + 1/4 + 1/4] = 0.

This implies
E[X · Y ] = E[X] · E[Y ].
Now
P [X = 0, Y = 0] 1
P [X = 0 | Y = 0] = = 0 6= = P [X = 0].
P [Y = 0] 2

This implies that E[X · Y ] = E[X] · E[Y ] does not implies that X and Y

4
are independent.

NOTE:) From Theorem 1.6 and Theorem 1.7, we can conclude that for
independent random variable X and Y the following condition is necessary
but not sufficient
E[X · Y ] = E[X] · E[Y ].
X \Y 1 -1 P(X=x)
1 1/4 1/4 1/2
Example.
-1 1/4 1/4 1/2
P(Y= y) 1/2 1/2 1

E[X] = 1 · P [X = 1] + (−1) · P [X = −1] = 0,

E[Y ] = 1 · P [Y = 1] + (−1) · P [Y = −1] = 0,

E[X · Y ] = 1 · [P [X = 1, Y = 1] + P [X = −1, Y = −1]]

+ (−1) · [P [X = −1, Y = 1] + P [X = 1, Y = −1]]
= [1/4 + 1/4] − [1/4 + 1/4] = 0.

E[X · Y ] = E[X] · E[Y ].

While
P [X = 1 | Y = −1] = 1/2 = P [X],
P [X = 1 | Y = 1] = 1/2 = P [X].
Theorem 1.8. Let IA be an indicator function and let X = IA , then E[X] =
P [A].
Proof. Given X = IA , then

E[X] = E[IA ]
Z
= fX (x)dx
x∈A
= P [x ∈ A] = P [A].

Example. Consider n trails of a coin tossing experiment. Let X be a

random variable and takes value 1 if heads turns upPand 0 in case of tails.
Let probability of heads turns up in p. Define Sn = ni=1 Xi . Then
n
X n
X
E[Sn ] = E[ Xi ] = E[Xi ] = np.
i=1 i=1

5
 Example. Let outcome of an experiment is an integer from {1, 2, 3, 4}.
Also let the experiment preformed n times. Let Xi denotes the sum of
outcomes after ith trial. Find out E[Xn ].
Solution. Let Yi denotes that out come of ith trial, then Xn = ni=1 Yi .
P
This implies
Xn n
X
E[Xn ] = E[ Yi ] = E[Yi ]. (3)
i=1 i=1

Now,
4
X 1 5
E[Yi ] = y= . (4)
4 2
y=1

Using (4) in (3), we get

n
X 5 5n
E[Xn ] = = .
2 2
i=1

2 Conditional Expectation.
In previous chapter we defined the conditional probability measure. Can we
define expectation with respect to new probability measure?

To answer it let us consider a random variable X defined on the proba-

bility space (Ω, F, P ). Also let B be an arbitrary but fixed subset of Ω and
B ∈ F. Consider a class of event

B ∩ F = {B ∩ A : A ∈ F} = FB ,

This class is a σ−algebra of subsets of B. Also FB be a sub-σ-algebra of

σ-algebra F i.e., FB ⊆ F. Let X be is integrable with respect to P. Then
we can easily show that X is integrable with respect to PB .
Now, we can define expectation E[X] with respect to this new probability
measure PB .
X
E[X | B] = xP [{X = x} | B]
x∈Ω
1 X
= xP [X = x].
P [B]
x∈B

6
In continuous case
Z
1
E[X | B] = xfX (x)dx.
P [B] x∈B

Some times we need more general conditioning as compare to above one.

For example instead of given subset of Ω, we would like to condition on a
given random variable.
Let X and Y are two random variables and P [Y ∈ B] 6= 0. Then
1 X
E[X | Y ∈ B] = xP [X = x, Y = y]
P [Y ∈ B]
x∈Ω,y∈B
1 X
= E[X | FB ]
P [Y ∈ B]
y∈B

In continuous case
Z
1
E[X | Y ∈ B] = xfX,Y (x, y)dx.
P [Y ∈ B] x∈Ω,y∈B

Now we can define the conditional expectation of random variable X

with respect to algebra/σ-algebra FB is a random variable E[X | FB ] such
that following conditions satisfied

i. E[X | FB ] is FB measurable.

ii. for every FB -measurable function Y,

E[XY | FB ] = E[E[X | FB ]Y ].

Example. A die rolled twice, Y be the outcome of the first roll and X
be the sum of the two outcomes. Show that E[X | Y ] is random variable
with respect to FY , where FY , is the smallest algebra for which Y is random
variable. Also Show that E[E[X | Y ]] = E[X].

Solution. The outcome of the first roll Y (ω) = ω where can take val-
ues from the set {1, 2, 3, 4, 5, 6}. Hence {1}, {2}, . . . {6} are the atoms of
the algebra for which Y is a random variable. Also P [Y = i] = 61 for all
i = 1, 2, . . . , 6.

7
 Then,
6
X
E[X | Y = i] = (i + j)P [X = i + j | Y = i]
j=1
6
X P [X = j + i, Y = i]
= (i + j)
P [Y = i]
j=1
6
1 X
= (i + j)P [X = j + i, Y = i]
P [Y = i]
j=1
6
1 X
= (i + j)P [Z = j]P [Y = i]
P [Y = i]
j=1
6
X
= (i + j)P [Z = j],
j=1

where Z be the outcome of the second roll.

6
X 6
X
E[X | Y = i] = iP [Z = j] + jP [Z = j]
j=1 j=1
6 6
X 1 X 1
= i + j
6 6
j=1 j=1
6
1X
= i+ j
6
j=1
= i + 3.5

This shows that the value of E[X | Y ] depends on the value of Y and take
6 different value for the corresponding to the 6 different values of y. This
implies that

{ω : E[X | Y ] = 3.5 + i} = {ω : Y = i} ∈ FY .

Hence E[X | Y ] is a random variable with respect to FY . Also

FE[X|Y ] ⊆ FY ,

where FE[X|Y ] be the smallest algebra form which E[X | Y ] is a random

variable.

8
 Now consider,
6
X
E[E[X | Y ]] = E[X | Y = j] · P [Y = j]
j=1
6
" 6 #
X X
= xP [X = x | Y = j] · P [Y = j]
j=1 x=1
6
" 6 #
X X P [X = x, Y = j]
= x · P [Y = j]
P [Y = j]
j=1 x=1
6
" 6 #
X X
= xP [X = x, Y = j]
j=1 x=1
6
X 6
X
= x P [X = x, Y = j]
x=1 j=1
6
X
= xP [X = x] = E[X].
x=1

Example. Let Ω := [0, 1] with σ−algebra B of Borel sets and P be

the probability measure on [0, 1]. Let X(ω) := 2ω and Y (ω) := 2 are two
random variables. Find E[X | Y ].

Solution.

E[X | Y ] = E[IΩ X] = E[X]

Z 1
= 2xdx = 1.
0

Example. Let X and Y are two random variables and following table
shows the joint probability mass function of X and Y :

Y \X -1 0 1 P[Y=y]
0 1/15 2/15 1/15 4/15
1 3/15 2/15 1/15 6/15
2 2/15 1/15 2/15 5/15
P[X=x] 6/15 5/15 4/15 1

9
Find E[X | Y = 2].
Solution. Condition probability mass function of X given Y = 2 is as
follows:
P [X = x, Y = 2] 1
P [X = x | Y = 2] = = P [X = x, Y = 2].
P [Y = 2] 3

i.e., 
 2/5, x = −1;
P [X = x | Y = 2] = 1/5, x = 0;
2/5, x = 1.


Hence

E[X | Y = 2] = −1·P [X = −1 | Y = 2]+0·P [X = 0 | Y = 2]+1·P [X = 1 | Y = 2] = 0.

Example. Let X and Y are two random variables having joint probability
mass function as
 1
2x+1
, x ≥ y, x, y = 1, 2, 3, . . .;
P [X = x, Y = y] =
0, otherwise.

Find E[Y | X = 2].

Solution. The marginal distribution of X is

x
X 1
P [X = x] = ,
2x+1
y=1

and the marginal distribution of Y is

P [X = x, Y = y] 1
P [Y = y] = = .
P [X = x] x

The conditional distribution of Y given X = 2 is as follows:

1
P [Y = y | X = 2] = .
2
Hence
1 1
E[Y | X = 2] = 1 · + 2 · = 3/2.
2 2

10
3 Moments & Moment Generating Functions.
The term moments in statistics has a resemblance with the term moments
in physics. In physics, moment shows amount of some physical quantity
applied to the some object and it is defined as the product of distance of an
object from a fixed reference point and some physical quantity such as the
force, charge, etc. The rth moment is denoted by µr and defined as
Z
µr = xr ρ(x)dx,

where x is the distance from some fix reference point (or origin) and ρ(x)
be some physical quantity. In particular, the first moment in physics used
to find the center of mass in a system of point masses. In statistics also we
use the same formula for calculating moments, the only difference is that in
statistics instead of ρ(x) we use probability density function fX (x) of vari-
able X. Hence the first moment in statistics gives the center of probability
mass instead of center of mass. In the similar way, in physics the second
moment (moment of inertia), shows the distribution of the point mass with
regards of a fix axis (or origin), and defined as the sum of all the elemen-
tal point masses each multiplied by the square of its perpendicular distance
from the some fix reference point (or origin). While in statistics second mo-
ment is defined as: sum of the square of distance from the some fix reference
point (or origin) multiplied by the corresponding probability masse. The
second statistical moment use to measure the dispersion from the fix ref-
erence point (or origin). Similar correspondence exists for higher moments
also. The above discussion shows the statistical moment are equally impor-
tant as moments in the physics. Hence we can conclude that, statistical
moments are used to characterize any statistical population. Now we are
interested in precise mathematical definition of moments.

Definition 3.1. The rth moment of a variable x about any point A, is

denoted by m0r and define as

m0r = E[(x − A)r ].

Note. when A = 0, then m0r is called the rth moment about the
origin and when A = X̄, then mr is called the rth moment about mean.
E | X − A |r is known as rth absolute moment about A.

11
 Now first moment about mean is

E[X − X̄] = E[X] − E[X̄] = E[X] − E[X] = 0.

and second moment about mean is

[E[(X − X̄)2 ] = E[X 2 − 2X X̄ + (X̄)2 ]

= E[X 2 ] − 2X̄E[X] + (X̄)2
= E[X 2 ] − 2X̄ X̄ + (X̄)2
= E[X 2 ] − (X̄)2 .

Note. First moment about mean is always equal to zero.

Example. Let X be a uniform random variable, having the following prob-
ability function
 1
P [X = x] = N , x = 1, 2, . . . , N ;
0, otherwise.

find first two moments about origin and first two moments about mean of
X.
Solution. The First moment about origin is
N N
X x 1 X N +1
E[X] = = x= ,
N N 2
x=1 x=1

and second moment about origin is

N N
X x2 1 X 2 (N + 1)(2N + 1)
E[X 2 ] = = x = .
N N 6
x=1 x=1

Example. Let X be a binomial random variable, having the following

probability function
 n
x n−x , x = 0, 1, 2, . . . , n, 0 ≤ p ≤ 1;
P [X = x] = x p (1 − p)
0, otherwise.

find first two moments about origin and first two moments about mean of
X.

12
Solution. The First moment about origin is
n  
X n
E[X] = xpx (1 − p)n−x
x
x=0
n
X n!
= xpx (1 − p)n−x
x!(n − x)!
x=0
n
X (n − 1)!
= np px−1 (1 − p)n−x
(x − 1)!(n − x)!
x=0
n
X (n − 1)!
= np px−1 (1 − p)n−x
(x − 1)!(n − x)!
x−1=0
= np.

and second moment about origin is

n  
2
X n
E[X ] = x2 px (1 − p)n−x
x
x=0
n
X n!
= x2 px (1 − p)n−x
x!(n − x)!
x=0
n
X (n − 1)!
= np xpx−1 (1 − p)n−x
(x − 1)!(n − x)!
x=0
n
X (n − 2)!
= n(n − 1)p2 px−2 (1 − p)n−x
(x − 2)!(n − x)!
x−2=0
n
X (n − 1)!
+ np px−1 (1 − p)n−x
(x − 1)!(n − x)!
x−1=0
= n(n − 1)p2 + np.

Example. Let X be a geometric random variable, having the following

probability function

p(1 − p)x−1 , x = 1, 2, . . . , 0 ≤ p ≤ 1;

P [X = x] =
0, otherwise.

find first two moments about origin and first two moments about mean of
X.

13
Solution. The First moment about origin is
∞
X
E[X] = xp(1 − p)x−1
x=1
X∞
= p x(1 − p)x
x=1
1 1
= p 2
= .
(1 − (1 − p)) p

and second moment about origin is

∞
X
E[X 2 ] = x2 p(1 − p)x−1
x=1
X∞
= p x2 (1 − p)x−1
x=1
1 + (1 − p)
= p.
p3
2−p
= .
p2
Example. Let X be a Gamma random variable, having the following prob-
ability function
( n
λ −λx xn−1 , x ≥ 0, λ > 0, n > 0;
P [X = x] = Γ(n) e
0, otherwise.

find first two moments about origin and first two moments about mean of
X.
Solution. The First moment about origin is
Z ∞
λn
E[X] = xe−λx xn−1 dx
Γ(n) 0
Z ∞
λn
= e−λx xn dx
Γ(n) 0
λn Γn + 1
= ·
Γ(n) λn+1
n
= .
λ

14
and second moment about origin is
Z ∞
λn
2
E[X ] = x2 e−λx xn−1 dx
Γ(n) 0
Z ∞
λn
= e−λx xn+1 dx
Γ(n) 0
λn Γn + 2
= ·
Γn λn+2
n(n + 1)
= .
λ2
Example. Let X be a normal random variable, having the following prob-
ability function
1
( 2
√ 1 e− 2σ2 (x−µ) , −∞ ≤ x ≤ ∞; σ 2 > 0, −∞ ≤ µ ≤ ∞;
P [X = x] = 2πσ
0, otherwise.

find first two moments about origin and first two moments about mean of
X.
Solution. The First moment about origin is
Z ∞
1 1 2
E[X] = √ xe− 2σ2 (x−µ) dx
2πσ −∞

x−µ
Let y = σ ,
Z ∞
1 1 2
E[X] = √ (yσ + µ)e− 2 y dy
2π −∞
Z ∞ Z ∞
1 − 21 y 2 µ 1 2
= √ ye dy + √ e− 2 y dy
2π −∞ 2π −∞

= 0 + µ,

first integral is 0 as the integrand is an odd function, while in second integral

we are integrating standard normal density over its entire range. Now second
moment about origin is
Z ∞
1 1 2
2
E[X ] = √ x2 e− 2σ2 (x−µ) dx
2πσ −∞

15
 x−µ
Let y = σ ,
Z ∞
1 1 2
E[X 2 ] =√ (yσ + µ)2 e− 2 y dy
2π −∞
Z ∞ Z ∞
σ2 2 − 21 y 2 µ2 1 2
= √ y e dy + √ e− 2 y dy
2π −∞ 2π −∞
Z ∞
2σµ 1 2
+√ ye− 2 y dy
2π −∞
2 Z ∞
σ 1 2
= √ y 2 e− 2 y dy + µ2 + 0.
2π −∞
in second integral we are integrating standard normal density over its entire
range and the third integral is 0 as the integrand is an odd function. Now
let z = y 2 /2, then

2σ 2 ∞ 3/2−1 −z
Z
2
E[X ] = √ z e dz + µ2 + 0
π 0
2σ 2
= √ Γ(3/2) + µ2
π
σ 2
= √ Γ(1/2) + µ2 = σ 2 + µ2 .
π
p
Note . Γ(1/2) = (π).

Moment Generating Function.

As we already know the importance of moments, but calculating all the

moment separately is a tedious job. Moment generating function appears
as a result of motivation of designing a single function from which we can
derive all the moments of a given distribution.

Definition 3.2. The moment generating function(m.g.f.) of a random

variable X having the probability function is define as
 R tx
tX e f (x)dx, for continuous r.v. X;
MX (t) = E[e ] = P tx
x e P [X = x], for discrete r.v. X.

the integration and summation is extended to the whole range of random

variable X, and t ∈ R.

16
 (tX)2 (tX)3
MX (t) = E[etX ] = E[1 + tX + + + . . .]
2! 3!
t2 E[X 2 ] t3 E[X 3 ]
= 1 + tE[X] + + + ...
2! 3!
t2 m02 t3 m03
= 1 + tm01 + + + ... (5)
2! 3!
where m0r is the rth moment about origin. If we differentiate (5) r times
with respect to t, and substituting t = 0, we get the rth moment about
origin, i.e.,
∂r
m0r = (MX (t)) .
∂tr t=0
Example. Let X be a Poisson random variable, having the following prob-
ability function
 e−λ λx
P [X = x] = x! , x = 0, 1, 2, . . . , λ > 0;
0, otherwise.

Find out the moment generating function of X.

Solution.

MX (t) = E[etX ]
∞
X e−λ λx
= etx
x!
x=0
∞
X (et λ)x
= e−λ
x!
x=0
−λ et λ t
= e e = e−λ(1−e ) .

Example. Let X be a negative exponential random variable, having the

following probability function

λe−λx , x ≥ 0 λ > 0;

fX (x) =
0, otherwise.

Find out the moment generating function of X.

17
Solution.
MX (t) = E[etX ]
Z ∞
= λ etx e−λx dx
0
Z ∞
= λ e−(λ−t)x dx
0
t −1
 
λ
= = 1− .
λ−t λ
Theorem 3.1. Let X be a random variable, with corresponding moment
generating functions MX (t). Then
MaX+b (t) = exp(bt)MX (at),
where a, b are constants.
Proof. Let Y = aX + b, then characteristic function of Y is as follows:
MY (t) = E[exp(Y t)] = E[exp((aX + b)t)]
= exp(bt)E[exp(aXt)]
= exp(bt)MX (at).
Theorem 3.2. Let X1 and X2 are two independent random variables, with
corresponding characteristic functions MX1 (t) and MX2 (t). Then
MX1 +X2 (t) = MX1 (t)MX2 (t).
Proof. Let Y = X1 + X2 , then characteristic function of Y is as follows:
MY (t) = E[exp(Y t)] = E[exp((X1 + X2 )t)]
= E[exp(X1 t)]E[exp(X2 t)]
= MX1 (t)MX2 (t).
The previous theorem can be extended to any number of independent vari-
ate.
Theorem 3.3. Let Xi , i = 1, 2, . . . , n are n independent random variables,
with corresponding characteristic functions MXi (t), i = 1, 2, . . . , n. Let Y =
P n
i=1 Xi Then
Yn
MY (t) = MXi (t).
i=1
Proof. we can prove the above statement in a very similar way used in
the previous theorem.

18
4 Characteristic Function.
Although, moment generating function is very important tool for generating
the moments of a given
P txdistribution. It does
R txnot exist for the distributions,
for which the sum x e p(x) or integral x e f (x)dx does not converge for
real valued t.
For example, the moment generating function of following mass and density
does not exist
a
P [X = x] = 2 , , x = 0, 1, 2, . . . , n,
x
and
1 1
fX (x) = , −∞ < X < ∞.
π 1 + x2
This problem can be solved by designing a more general generating function.

Definition 4.1. Let X be a random variable with distribution function

FX (x). Then the characteristic function(C.F.) of a random variable X
is define as
 R itx
itX e f (x)dx, for continuous r.v. X;
φX (t) = E[e ] = P itx
x e f (x), for discrete r.v. X.

the integration and summation is extended to the whole range of random

variable X, and t ∈ R.

Example. Find out the characteristic function of binomial distribution.

Solution.
n  
X n itx x
φX (t) = e p (1 − p)n−x
x
x=0
n  
X n
= (eit p)x (1 − p)n−x
x
x=0

n
= (1 − p) + ei tp .

Example. Let X be a Gamma random variable, having the following prob-

ability function
( n
λ −λx xn−1 , x ≥ 0, λ > 0, n > 0;
P [X = x] = Γ(n) e
0, otherwise.

19
Find out the characteristic function of X.
Solution.

φX (t) = E[eitX ]
Z ∞
λn
= eitx e−λx xn−1 dx
Γ(n) 0
Z ∞
λn
= λ e−(λ−it)x xn−1 dx
Γ(n) 0
λn Γ(n)
=
Γ(n) (λ − it)n
it −n
 
= 1− .
λ

Necessary and Sufficient conditions for Characteristic Function

1. φ(0) = 1.

2. | φ(t) |≤ φ(0) = 1.

3. φ(t) is uniformly continuous for t ∈ R.

4. φ(t) = φ(−t).

Proof.

1. Z ∞
φ(0) = fX (x)dx = 1.
−∞

2.
Z ∞
| φ(t) | ≤ | eitx f (x)dx |
−∞
Z ∞ Z ∞
≤ | eitx | f (x)dx = fX (x)dx = 1.
−∞ −∞
p
Since eitx = cos(tx)+i sin(tx) and hence | eitx |= cos2 (tx) + sin2 (tx) =
1.

20
 3. For h 6= 0,
Z ∞
| φ(t + h) − φ(t) | = | (ei(t+h)x − eitx )f (x)dx |
−∞
Z ∞
≤ | ei(t+h)x − eitx | f (x)dx
Z−∞∞
= | eitx (eihx − 1) | f (x)dx
−∞
Z ∞
= | (eihx − 1) | f (x)dx,
−∞

p
since | eitx |= cos2 (tx) + sin2 (tx) = 1. Now taking h → 0, we get
Z ∞
lim | φ(t + h) − φ(t) |≤ lim | (eihx − 1) | f (x)dx = 0.
h→0 −∞ h→0

⇒ φ(t + h) = φ(t); ∀t.

4.
φ(t) = E[eitx ] = E[cos(tx) + i sin(tx)]
and

φ(t) = E[e−itx ] = E[cos(tx) − i sin(tx)]

= E[cos(−tx) + i sin(−tx)]
= E[e−itx ] = φ(−t)

Theorem 4.1. Inversion Theorem. Let distribution function FX (x) con-

tinuous over (x − h, x + h). Then
Z T
1 sin(ht) −itx
FX (x + h) − FX (x − h) = lim e φX (t)dt,
T →∞ 2π −T t

where φX (t) be the characteristic function.

Proof. Starting with

Z T
1 sin(ht) −itx
FX (x + h) − FX (x − h) = lim e φX (t)dt,
T →∞ 2π −T t

21
 T
FX (x + h) − FX (x − h)
Z
1 sin(ht) −itx
⇒ = lim e φX (t)dt,
2h T →∞ 2π −T ht
taking h → 0, we get
Z ∞
FX (x + h) − FX (x − h) 1 sin(ht) −itx
lim = lim e φX (t)dt,
h→0 2h 2π −∞ h→0 ht
Z ∞
0 1
fX (x) = FX (x) = e−itx φX (t)dt,
2π −∞
which is the Inverse Laplace Transformation of φX (t).

Theorem 4.2. Uniqueness Theorem. Characteristic function is uniquely

determines the distribution function, i.e., a necessary and sufficient condi-
tion for two random variable X and Y with probability density/mass func-
tions fX (·) and gY (·) to be identical is that their characteristic functions are
identical.
Proof. If fX (·) and gY (·), then by the definition of characteristic func-
tion, we have
Z ∞
φX (t) = eitx fX (x)dx
Z−∞
∞
= eitx gY (x)dx = φY (t).
−∞

Since fX (·) and gY (·) are identical.

Conversely, by the inversion theorem for characteristic function,

Z ∞
1
fX (x) = e−itx φX (t)dt
2π −∞
Z ∞
1
e−itx φY (t)dt = gY (x).
2π −∞

Since φX (t) and φX (t) are identical.

Note. 1. Let X be a random variable with characteristic function

φX (t), and if m0r (the rth moment about origin) exists. Then
 r 
0 r ∂
mr = (−1) (φX (t)) .
∂tr t=0

22
Theorem 4.3. Let X be a random variable, with characteristic functions
φX (t). Then
φaX+b (t) = exp(ibt)φX (at),
where a, b are constants.

Proof. Let Y = aX + b, then characteristic function of Y is as follows:

φY (t) = exp(iY t) = exp(i(aX + b)t)

= exp(ibt) exp(iaXt)
= exp(ibt)φX (at).

Theorem 4.4. Let X1 and X2 are two independent random variables, and
with corresponding characteristic functions φX1 (t) and φX2 (t). Then

φX1 +X2 (t) = φX1 (t)φX2 (t).

Proof. Let Y = X1 + X2 , then characteristic function of Y is as follows:

φY (t) = exp(iY t) = exp(i(X1 + X2 )t)

= exp(iX1 t) exp(iX2 t)
= φX1 (t)φX2 (t).

The previous theorem can be extended to any number of independent vari-

ate.

Theorem 4.5. Let Xi , i = 1, 2, . . . , n are n independent random variables,

with corresponding characteristic functions φXi (t), i = 1, 2, . . . , n. Let Y =
P n
i=1 Xi Then
Yn
φY (t) = φXi (t).
i=1

Proof. we can prove the above statement in a very similar way used in
the previous theorem.

Theorem 4.6. The distribution function of a random variable is symmetric

i.e., FX (x) = F−X (x)or 1 − FX (x) = FX (−x), iff φX (t) is an even function
of t.

23
 Proof.
Z ∞ Z ∞
φX (t) = itx
e f (x)dx = e−ity f (−y)dy = φX (−t)
−∞ −∞

This implies that φX (t) is an even function.

Also, we have
φX (t) = φX (−t)
this implies that φX (t) is real valued.
Conversely, if φX (t) is real;

φX (t) = φX (t)

i.e., X and −X have same distribution function.

Example. Find out the characteristic function of a random variable,

having following density function:
1
( 2
√ 1 e− 2σ2 (x−µ) , −∞ < x < ∞, −∞ < µ < ∞, σ 2 > 0;
fX (x) = 2πσ
0, otherwise.
X−µ
Solution. Let Y = σ , then

φX (t) = eitµ φY (tσ). (6)

Now characteristic function of Y is as follows:

Z ∞
1 1 2
φY (t) = √ eity e− 2 y dy
2π −∞
Z ∞
1 1 2
= √ e− 2 (y −2ity dy
2π −∞
t2 Z
∞
e2 1 2 2
= √ e− 2 (y −2ity+t dy
2π −∞
t2 Z
∞
e− 2 1 2
= √ e− 2 (y−it) dy
2π −∞
t2
= e− 2 . (7)

Now using (7) in (6), we get

1 2 −2itµ)
φX (t) = e−( 2 (t .

Exercise.

24
1. Let 3 dice tossed together

(a) Find out the expected value of the sum of face numbers on the
dice.
(b) Find out the expected value of the sum of face numbers on the
dice.

2. Consider a bag having m balls numbered from 1 to m, out of these

balls n(< m) balls are drawn. Let Sn we the sum of the number on
these n balls. Find out first two moments around origin of S.

3. In a series of n independent tosses of a fair coin. Find the first two

moments of the number of heads.

4. Let X1 , X2 , . . . , Xn are n i.i.d. P

random variable with moment gener-
n
Xi
ating function MX (t). let X = i=1 n . Find out moment generating
function of X.

5. Find out the moment generating function of random variable whose

rth moment about origin is A1r .

6. Find out the moment generating function of random variable whose

rth moment about origin is (r + 1)!Ar .

7. Find out the first two moment about origin of the following distribu-
tions with the help of their respective moment generating functions

(a) Poisson distribution with parameter λ.

(b) Uniform (Rectangular) distribution with parameter a, b(> a).
(c) Gamma distribution with parameter (α, β).

8. Using moment generating function show that sum of Poisson random

variables is again a Poisson variate.

9. Find Characteristic function of following distributions:

(a) Binomial distribution with parameter n, p.

(b) Geometric distribution with parameter p.

25
 (c) Beta distribution with parameter m, n.

10. Using characteristic function show that the probability density func-
tion of a standard normal variate is symmetric about origin.

11. Check whether following function is a characteristic function or not?

φX (t) = e−|t| .

12. Let X and Y are two random variables from exp(λ). Using charac-
teristic function show that X + Y is Gamma variate with parameter
(λ, 2).

13. Let X and Y are two random variables from N (µ, σ 2 ). Using charac-
teristic function show that X + Y is again follow normal variate with
parameters 2µ and 2σ 2 .

26