66

7. Week 7

Remark 7.1 (Identically distributed RVs). Let X and Y be two RVs, possibly defined on different
probability spaces.

(a) Recall from Remark 3.19 that their law/distribution may be the same and in this case,
we have FX = FY , i.e. FX (x) = FY (x), ∀x ∈ R. The statement ‘X and Y are equal in
law/distribution’ is equivalent to ‘X and Y are identically distributed’.
(b) Recall from Remark 3.20 that the DF uniquely identifies the law/distribution, i.e. if FX =
FY , then X and Y are identically distributed.
(c) Suppose X and Y are discrete RVs. Recall from Remark 4.13, the p.m.f. is uniquely
determined by the DF and vice versa. In the case of discrete RVs, X and Y are identically
distributed if and only if the p.m.f.s are equal (i.e., fX = fY ).
(d) Suppose X and Y are continuous RVs. Recall from Note 4.24 that the p.d.f.s in this case
are uniquely identified upto sets of ‘length 0’. We may refer to such an almost equal p.d.f.
as a ‘version of a p.d.f.’. Recall from Note 4.27, the p.d.f. is uniquely determined by the
DF and vice versa. In the case of continuous RVs, X and Y are identically distributed if
and only if the p.d.f.s are versions of each other. In other words, X and Y are identically
distributed if and only if there exist versions fX and fY of the p.d.f.s such that fX = fY ,
i.e. fX (x) = fY (x), ∀x ∈ R.
(e) Suppose X and Y are identically distributed and let h : R → R be a function. Then we have
that the RVs h(X) and h(Y ) are identically distributed. In particular, Eh(X) = Eh(Y ),
provided one of the expectations exists.
(f) Suppose X and Y are identically distributed. By (e), X 2 and Y 2 are identically distributed
and EX 2 = EY 2 , provided one of the expectations exists. More generally, the n-th moments
EX n and EY n of X and Y are the same, provided they exist.
(g) There are examples where EX n = EY n , ∀n = 1, 2, · · · , but X and Y are not identically
distributed. We may discuss such an example later in this course. Consequently, the
moments do not uniquely identify the distribution. Under certain sufficient conditions on
 67

the moments, such as the Carleman’s condition, it is however possible to uniquely identify
the distribution. This is beyond the scope of this course.
(h) Suppose X and Y are identically distributed and suppose that the MGF MX exists on
(−h, h) for some h > 0. By the above observation (e), the MGF MY exists and MX = MY ,
i.e. MX (t) = MY (t), ∀t ∈ (−h, h).
(i) We now state a result without proof. Suppose the MGFs MX and MY exist. If MX (t) =
MY (t), ∀t ∈ (−h, h), then X and Y are identically distributed. Therefore, the MGF
uniquely identifies the distribution.
d
Notation 7.2. We write X = Y to denote that X and Y are identically distributed.

Example 7.3. If Y is an RV with the MGF MY (t) = (1−t)−1 , ∀t ∈ (−1, 1), then by Example 6.47,
we conclude that Y is a continuous RV with p.d.f.

 e−x , if x > 0
fY (x) =
 0, otherwise.
Example 7.4. If X is a discrete RV with support SX and p.m.f. fX , then the MGF MX is of the
form
etx fX (x).
X
MX (t) =
x∈SX

We can also make a converse statement. Since the MGF uniquely identifies a distribution, if an
MGF is given by a sum of the above form, we can immediately identify the corresponding discrete
RV with its support and p.m.f.. For example, if MX (t) = 1
2
+ 13 et + 16 e−t , then X is discrete with
the p.m.f. 
1



 2
, if x = 0,



1, if x = 1,


3
fX (x) :=
1
6, if x = −1,







0, otherwise.



Notation 7.5. We may refer to expectations of the form EetX as exponential moments of the RV
X.
68

We now discuss a function similar to the MGF, arising through the Fourier transform. We first
recall some notations from Complex Analysis.

Remark 7.6. Recall that exp(iy) = cos(y) + i sin(y), ∀y ∈ R. Moreover, | exp(iy)| = 1, ∀y ∈ R.

Let X be a discrete/continuous RV defined on a probability space (Ω, F, P) with DF FX ,

p.m.f./p.d.f. fX and support SX .

Definition 7.7 (Characteristic Function). The Characteristic function of X, denoted by ΦX :

R → C is defined by ΦX (t) := EeitX = E cos(tX) + i E sin(tX), ∀t ∈ R.

Remark 7.8. Since cos and sin are bounded continuous functions on R, the random variables
cos(tX) and sin(tX) are bounded for all t ∈ R and all RVs X. Using Proposition 6.22(d), we
conclude that E cos(tX) and E sin(tX) exist for all t ∈ R and all RVs X. As such, the Characteristic
function exists for all RVs X.

Note 7.9. If X is discrete/continuous with p.m.f./p.d.f. fX , then following the definition of an

expectation of an RV, we write

ΦX (t) = EeitX

eitx fX (x), for discrete X,
 X



= x∈SX

 ∞ eitx fX (x) dx, for continuous X.

R
−∞

= E cos(tX) + i E sin(tX)

X X
cos(tx)fX (x) + i sin(tx)fX (x), for discrete X,




= x∈SX x∈SX

 ∞
R R∞

−∞ cos(tx)fX (x) dx + i −∞ sin(tx)fX (x) dx, for continuous X.

Note 7.10. For any RV X, ΦX (0) = 1.

 69

Example 7.11. For c ∈ R, consider the constant/degenerate RV X given by the p.m.f. (see
Example 6.8) 
1, if x = c


fX (x) = 
0,
 otherwise.
Here, the support is SX = {c} and ΦX (t) = EeitX = eitx fX (x) = eitc = cos(tc) +
P
x∈SX

i sin(tc), ∀t ∈ R.

Note 7.12. Take constants c, d ∈ R with c ̸= 0. Then, the RV Y = cX + d is discrete/continuous,

according to X being discrete/continuous and moreover,

ΦY (t) = Eeit(cX+d) = eitd ΦX (ct)

exists for all t ∈ R.

Note 7.13. We have Φ−X = Φ̄X .

We state the next result without proof.

Theorem 7.14. If E|X|k < ∞ for some positive integer k, then

r
" #
(r)
 
(r) d
(a) ΦX (u) = ir E X r eiuX , where MX (0) = MX (t) is the r-th derivative of MX (t) at
dtr t=0
the point 0 for each r ∈ {1, 2, · · · , k}.
(r)
(b) In particular, E (X r ) = (−i)r ΦX (0) for each r ∈ {1, 2, · · · , k}.
(k) Pk ij j j
(c) ΦX (u) = j=0 j! (EX )u + o(|u|k ), as u → 0.

Remark 7.15. Suppose X and Y are two RVs, possibly defined on different probability spaces. If
ΦX (t) = ΦY (t), ∀t ∈ R, then X and Y are identically distributed. Therefore, the Characteristic
function uniquely identifies the distribution.

Definition 7.16 (Symmetric Distribution). An RV X is said to have a symmetric distribution

d
about a point µ ∈ R if X − µ = µ − X.

Proposition 7.17. Let X be an RV which is symmetric about 0.

70

(a) If X is discrete, then the p.m.f. fX has the property that fX (x) = fX (−x), ∀x ∈ R. Further,
EX n = 0, ∀n = 1, 3, 5, · · · , provided the moments exist.
(b) If X is continuous, then the p.d.f. fX has the property that fX (x) = fX (−x), ∀x ∈ R.
Further, EX n = 0, ∀n = 1, 3, 5, · · · , provided the moments exist.

Proof. We prove the statement when X is a continuous RV. The proof for the case when X is
discrete is similar.
d
If X is symmetric about 0, then X = −X and hence for any x ∈ R, FX (x) = F−X (x) and
hence FX (x) = P(X ≤ x) = P(−X ≤ x) = P(X ≥ −x) = 1 − FX (−x). This implies fX (x) =
fX (−x), ∀x ∈ R.
R∞
Assume that the moments in question exist. Then EX n = −∞ xn fX (x) dx = 0, since the
function x 7→ xn fX (x) is odd. □

Remark 7.18. Let X be a continuous RV, which is symmetric about µ ∈ R. As argued in the above
proposition, we have for all x ∈ R

FX (µ + x) = P(X ≤ µ + x) = P(X − µ ≤ x) = P(µ − X ≤ x) = P(X − µ ≥ −x) = 1 − FX (µ − x)

and hence fX (µ + x) = fX (µ − x), ∀x. Conversely, given a continuous RV X such that fX (µ + x) =

fX (µ − x), ∀x for some µ ∈ R, we have FX−µ = Fµ−X and hence X is symmetric about µ.

We now look at some special examples of discrete RVs.

Example 7.19 (Degenerate RV). We have already mentioned this example earlier in Example 6.8.
Fix c ∈ R. Say that X is degenerate at c if its distribution is given by the p.m.f.

1, if x = c


fX (x) = P(X = x) =
0, otherwise.



This is a discrete RV with support SX = {c}. As computed earlier, EX = c. We also have EX n =

cn , ∀n ≥ 1, MX (t) = etc , ∀t ∈ R and ΦX (t) = eitc , ∀t ∈ R. Note that V ar(X) = EX 2 − (EX)2 = 0.

Remark 7.20 (Bernoulli Trial). Suppose that a random experiment has exactly two outcomes, iden-
tified as a ‘success’ and a ‘failure’. For example, while tossing a coin, we may think of obtaining a
 71

head as a success and a tail as a failure. Here, the sample space is Ω = {Success, F ailure}. A single
trial of such an experiment is referred to as a Bernoulli trial. In this case, P robability({Success}) =
1 − P robability({F ailure}). If we define an RV X : Ω → R by X(Success) = 1 and X(F ailure) =
0, then X is a discrete RV with p.m.f.




1 − P robability({Success}), if x = 0,


fX (x) = P(X = x) = P robability({Success}), if x = 1,




0, otherwise.



If P robability({Success}) = 0 or P robability({F ailure}) = 0, then X is degenerate at 0 or 1,

respectively. The case when P robability({Success}) ∈ (0, 1) is therefore of interest.

Example 7.21 (Bernoulli(p) RV). Let p ∈ (0, 1). An RV X is said to follow Bernoulli(p) distri-
bution or equivalently, X is a Bernoulli(p) RV if its distribution is given by the p.m.f.




1 − p, if x = 0,


fX (x) = p, if x = 1,



0,

otherwise.

In relation with the Bernoulli trial described above, p may be treated as the probability of success.
Here, EX = p, EX 2 = p, V ar(X) = EX 2 − (EX)2 = p − p2 = p(1 − p), MX (t) = 1 − p + pet , t ∈ R
and ΦX (t) = 1 − p + peit , t ∈ R. By standard arguments, we can establish the existence of these
moments.

Notation 7.22. We may write X ∼ Bernoulli(p) to mean that X is a Bernoulli(p) RV. Similar
notations shall be used for other RVs and their distributions.

Example 7.23 (Binomial(n, p) RV). Fix a positive integer n and let p ∈ (0, 1). By the Binomial
theorem, we have
n
!
n n k
p (1 − p)n−k
X
1 = [p + (1 − p)] =
k=0 k
72

and hence the function f : R → [0, 1] given by


 
 n px (1 − p)n−x , if x ∈ {0, 1, · · · , n},


x
f (x) =
0, otherwise.



is a p.m.f.. An RV X is said to follow Binomial(n, p) distribution or equivalently, X is a Binomial(n, p)

RV if its distribution is given by the above p.m.f.. Here,
n n
!
n! k n−k n − 1 k−1
p (1 − p)n−k = np [p + (1 − p)]n−1 = np,
X X
EX = k p (1 − p) = np
k=0 k!(n − k)! k=1 k − 1

and
n n
!
n! n − 2 k−2
pk (1−p)n−k = n(n−1)p2 p (1−p)n−k = n(n−1)p2 .
X X
EX(X−1) = k(k−1)
k=0 k!(n − k)! k=2 k − 2

Then EX 2 = EX(X − 1) + EX = n(n − 1)p2 + np and V ar(X) = EX 2 − (EX)2 = n(n − 1)p2 +

np − n2 p2 = np(1 − p). Also
n
!
tX tk n k
p (1 − p)n−k = (1 − p + pet )n , ∀t ∈ R.
X
MX (t) = Ee = e
k=0 k

By standard arguments, we can establish the existence of these moments. We also have ΦX (t) =
 
Pn n
EeitX = k=0 e
itk
k
pk (1 − p)n−k = (1 − p + peit )n , ∀t ∈ R.

Note 7.24. Observe that Binomial(1, p) distribution is the same as Bernoulli(p) distribution. We
shall explore the connection between Binomial and Bernoulli distributions later in the course.

Remark 7.25 (Factorial moments). In the computation for EX 2 for X ∼ Binomial(n, p), we first
computed EX(X − 1), which is easy to compute. It turns out that expectations of the form
EX(X − 1), EX(X − 1)(X − 2) etc. are often easy to compute for integer valued RVs X. We refer
to such expectations as factorial moments of X.

Remark 7.26 (Symmetry of Binomial(n, 12 ) distribution). Let X ∼ Binomial(n, p) and let Y :=

n − X. Since MX (t) = (1 − p + pet )n , ∀t ∈ R, we have

MY (t) = EetY = Eet(n−X) = e−nt MX (−t) = ent (1 − p + pe−t )n = (p + (1 − p)et )n .

 73

Since MGFs determine the distribution, we conclude that Y ∼ Binomial(n, 1 − p). In particular,
d n d
if p = 12 , then Y = n − X = X ∼ Binomial(n, 12 ). Rewriting the relation, we get 2
− X = X − n2 .
Therefore, X ∼ Binomial(n, 21 ) is symmetric about n2 .

We now look at more examples of discrete RVs. Later in the course, we shall discuss their
motivation through various random experiments.

Example 7.27 (Uniform RVs with support on a finite set). Consider a discrete RV X with support
SX = {x1 , x2 , · · · , xn } and p.m.f. fX : R → [0, 1] given by

1, if x ∈ SX ,


n
fX (x) =
0, otherwise.



We had considered the case SX = {1, 2, · · · , 6} in Example 6.9 and computed the expectation. In
the general setting, we have
1 X 1 X 2 1 X tx
EX = x, EX 2 = x, MX (t) = EetX = e , ∀t ∈ R
n x∈SX n x∈SX n x∈SX

and hence V ar(X) can be computed by the formula EX 2 − (EX)2 . By standard arguments, we
can establish the existence of these moments.

P∞ λk
Example 7.28 (Poisson (λ) RV). Fix λ > 0. Note that eλ = k=0 k! and hence the function
f : R → [0, 1] given by 
x
e−λ λ , if x ∈ {0, 1, 2, · · · },


x!
f (x) =
0, otherwise.



is a p.m.f.. An RV X is said to follow Poisson(λ) distribution or equivalently, X is a Poisson(λ)

RV if its distribution is given by the above p.m.f.. Recall that we have already computed the
following EX = λ, V ar(X) = λ and MX (t) = eλ(e −1) , ∀t ∈ R in Example 6.46. As done for the
t

case of Binomial(n, p) RVs, we can compute factorial moments. For example,

∞ k ∞
−λ λ 2 −λ λk−2
= λ2 .
X X
EX(X − 1) = k(k − 1)e =λ e
k=0 k! k=2 (k − 2)!
74

In fact, EX(X − 1) · · · (X − (n − 1)) = λn for all n ≥ 1. Here, the Characteristic function is

Φ (t) = eλ(e −1) , ∀t ∈ R.
it
X

P∞
Example 7.29 (Geometric (p) RV). Fix p ∈ (0, 1). Note that k=0 p(1 − p)k = 1 and hence the
function f : R → [0, 1] given by

p(1 − p)x , if x ∈ {0, 1, 2, · · · },


f (x) =
0, otherwise.



is a p.m.f.. An RV X is said to follow Geometric(p) distribution or equivalently, X is a Geometric(p)

RV if its distribution is given by the above p.m.f.. Let us compute the MGF. Here,
∞
tX p
etk p(1 − p)k =
X
MX (t) = Ee = ,
k=0 1 − (1 − p)et
 
1
for all t such that 0 < (1 − p)et < 1 or equivalently, t < ln 1−p
. Looking at the derivatives of
1−p 1−p
MX and evaluating at t = 0, we have EX = p
and V ar(X) = p2 . The Characteristic function
itX p
is given by ΦX (t) = Ee = 1−(1−p)eit
,t ∈ R.

We now look at special examples of continuous RVs.

Example 7.30 (Uniform(a, b) RV). Fix a, b ∈ R with a < b. An RV X is said to follow

Uniform(a, b) distribution or equivalently, X is a Uniform(a, b) RV if its distribution is given
by the p.d.f. 
 1
, if x ∈ (a, b),


fX (x) =  b−a
0, otherwise.


We had considered the case a = 0, b = 1 in Example 6.13 and computed the expectation. In the
general setting, we have
Z b
x b 2 − a2 a+b Z b
x2 b 3 − a3 a2 + ab + b2
EX = dx = = , EX 2 = dx = =
a b−a 2(b − a) 2 a b−a 3(b − a) 3
 75

and hence V ar(X) can be computed by the formula EX 2 − (EX)2 . The MGF is given by

tb ta
 e −e ,
if t ̸= 0,

Z b
etx

EetX = dx = t(b−a)
a b−a 1, if t = 0.



By standard arguments, we can establish the existence of these moments. Further, observe that
fX ( a+b
2
− x) = fX ( a+b
2
+ x), ∀x ∈ R. Using Remark 7.18, we conclude that X is symmetric about
its mean.

Example 7.31 (Cauchy(µ, θ) RV). Let θ > 0 and µ ∈ R. An RV X is said to follow Cauchy(µ, θ)
distribution if its distribution is given by the p.d.f.
θ 1
fX (x) = , ∀x ∈ R.
π θ + (x − µ)2
2

x−µ
The fact that fX is a p.d.f. is easy to check. Set y = θ
and observe that
Z ∞
1Z∞ 1 2Z∞ 1 2
fX (x) dx = 2
dy = 2
dy = tan−1 (y) |∞
0 = 1.
−∞ π −∞ 1 + y π 0 1+y π
We have already considered the case µ = 0, θ = 1 in Example 6.15 and Example 6.48, where we
have seen that EX and the MGF do not exist for this distribution. In the general setting, note
X−µ
that θ
∼ Cauchy(0, 1) and by a similar argument, we can show that EX and MGF do not
exist. Moreover, fX (µ + x) = fX (µ − x), ∀x ∈ R and using Remark 7.18, we conclude that X is
symmetric about µ. The Characteristic function for X is given by

ΦX (u) = exp(iuµ − θ|u|), ∀u ∈ R.

R∞
Example 7.32 (Exponential(λ) RV). Let λ > 0. Note that 0 exp(− λx ) dx = λ and hence the
function f : R → [0, ∞) given by

1 exp(− λx ), if x > 0,


λ
f (x) =
0, otherwise.



is a p.d.f.. An RV X is said to follow Exponential(λ) distribution or equivalently, X is an

Exponential(λ) RV if its distribution is given by the above p.d.f.. We have already considered
76

the case λ = 1 in Example 6.47, where we computed the moments and the MGF. Following similar
arguments, in the general setting we have
1
EX n = λn n!, V ar(X) = λ2 , MX (t) = (1 − λt)−1 , ∀t < .
λ
By standard arguments, we can establish the existence of these moments.

R ∞ α−1 −x
Definition 7.33 (Gamma function). Recall that the integral x e dx
0 exists if and only if
R ∞ α−1 −x
α > 0. On (0, ∞), consider the function α 7→ 0 x e dx. It is called the Gamma function and
the value at any α > 0 is denoted by Γ(α).

Remark 7.34. We recall some important properties of the Gamma function.

(a) For α > 0, we have Γ(α) > 0.

(b) Γ(α) = (α − 1)Γ(α − 1), if α > 1.
R ∞ −x
(c) Γ(1) = 0 e dx = 1 and hence using (b), Γ(n) = (n − 1)! for all positive integers n.
R∞ 1 √ 2
(d) Γ( 12 ) = 0
√
x
e−x dx = π. Putting x = y2 , this relation may be rewritten as

√ Z∞ √
!
1 y2
 
Γ = 2 exp − dy = π.
2 0 2

y R ∞ α−1 −x
(e) Fix β > 0. Putting x = β
, in the integral for Γ(α) = 0 x e dx, we get Γ(α) =
R ∞ α−1 −α
0 y β exp(− y ) dy. β

Example 7.35 (Gamma(α, β) RV). Fix α > 0, β > 0. By the properties of the Gamma function
described above, the function f : R → [0, ∞) defined by

 1 xα−1 β −α exp(− βx ), if x > 0,


Γ(α)
f (x) = 
0,
 otherwise.

is a p.d.f.. An RV X is said to follow Gamma(α, β) distribution or equivalently, X is a Gamma(α, β)

RV if its distribution is given by the above p.d.f.. Note that for α = 1, we get back the p.d.f.
for an Exponential(β) RV (see Example 7.32), i.e. Gamma(1, β) distribution is the same as
 77

Exponential(β) distribution. For general α > 0, β > 0, we have

1
EX = αβ, V ar(X) = αβ 2 , MX (t) = (1 − βt)−α , ∀t < .
β
By standard arguments, we can establish the existence of these moments.
R ∞ 1 −t √
Example 7.36 (Normal(µ, σ 2 ) RV). Fix µ ∈ R, σ > 0. Note that Γ( 21 ) = 0
√ e
t
dt = π (see
y2 2
 
∞
√1 exp − y2
R
Remark 7.34). Putting t = 2
and after suitable manipulation, we have 2π −∞ dy = 1.
Putting y = σ1 (x − µ) (equivalently, x = σy + µ), we have
!
1 Z∞ (x − µ)2
√ exp − dx = 1.
σ 2π −∞ 2σ 2
Therefore, the function f : R → [0, ∞) defined by
!
1 (x − µ)2
f (x) = √ exp − , ∀x ∈ R
σ 2π 2σ 2
is a p.d.f.. An RV X is said to follow Normal(µ, σ 2 ) distribution or equivalently, X is a Normal(µ, σ 2 )
RV, denoted by X ∼ N (µ, σ 2 ) if its distribution is given by the above p.d.f.. If X ∼ N (µ, σ 2 ),
X−µ
from our above discussion we conclude that Y = σ
∼ N (0, 1). Now,
!
tY 1 Z ∞ ty y2
MY (t) = Ee =√ e exp − dy
2π −∞ 2
! !
t2 1 Z∞ (y − t)2
= exp √ exp − dy
2 2π −∞ 2
!
t2
= exp , ∀t ∈ R.
2
t2
In particular, ψY (t) = ln MY (t) = 2
, ∀t ∈ R with ψ ′ (t) = t, ψ ′′ (t) = 1, ∀t ∈ R. Evaluating at
t = 0, by Proposition 6.45 we conclude that EY = 0 and V ar(Y ) = 1. But X = σY + µ and
hence EX = µ, V ar(X) = σ 2 . This yields the interpretation of the parameters µ and σ in the
distribution of X. Further, MX (t) = EetX = Eet(σY +µ) = eµt MY (σt) = exp(µt + 12 σ 2 t2 ), ∀t ∈ R.
The Characteristic function for X is given by
1
ΦX (u) = exp(iuµ − σ 2 u2 ), ∀u ∈ R.
2
78

Definition 7.37 (Standard Normal RV). We say X is a Standard Normal RV if X ∼ N (0, 1), i.e.
EX = 0 and V ar(X) = 1.

Notation 7.38. Normal RVs are also referred to as Gaussian RVs and Normal distribution as
Gaussian distribution.

Remark 7.39 (Symmetry of Gaussian Distribution). If X ∼ N (µ, σ 2 ), note that fX (µ + x) =

fX (µ − x), ∀x ∈ R and using Remark 7.18, we conclude that X is symmetric about its mean µ.