MULTINOMIAL DISTRIBUTION :

A random experiment that satisfies the following properties:

(i) It has a repeated number of repeated trials.

(ii) Each trial has a discrete number of possible outcomes.

(iii) On any given trial, the probability that a particular outcome will occur is constant.

(iv) The trials are independent of each other;

is said to be a multinomial experiment. A binomial experiment is a special case of

multinomial experiment.
Probability distribution:
The probabilities of the possible outcomes of a multinomial experiment are defined by a
probability distribution known as multinomial distribution
Definition: Let a multinomial experiment have k(k > 2) possible outcomes; where each
of the k possible outcomes is of type i, i = 1, 2, .., k. Also let the probability that a trial
results in outcome i be pi . Further let Xi be the number of the n trials that are of type i;
then the probability distribution function of X1 , X2 , .., Xk−1 is given by

n! xk−1 xk
X
k−1
f (x1 , x2 , .., xk−1 ) = px1 1 px2 2 ....pk−1 pk ; xi ≤ n
x1 !x2 !...xk−1 !xk ! i=1

Pk−1 P
where pk = 1 − pi and xk = n − k−1i=1 xi
Pk i=1
Remark: Given that i=1 xi = n; then essentially we only have k − 1 random variables.
The random variables X1 , X2 , .., Xk−1 are components of random vector X having multi-
nomial distribution.

1
 Mean vector and variance covariance martix

for the mean vector:

the expected value of each component is E[Xi ] = npi ; hence
 
np1
 
 np2 
µ=  .. 

 . 
npk−1

For the variance-covariance matrix:

For each component Xi ;V ar(Xi ) = npi (1 − pi ) and for each pair (Xi , Xj );
Cov(Xi Xj ) = −npi pj , i ̸= j.
This values give rise to the matrix
 
np1 (1 − p1 ) −np1 p2 ... −np1 pj ... −np1 pk−1
 
 −np2 p1 np2 (1 − p2 ) . . . −np2 pj ... −np2 pk−1 
Σ=  .. .. .. .. .. .. 

 . . . . . . 
−npk−1 p1 −npk−1 p2 . . . −npk−1 pj . . . npk−1 (1 − pk−1 )

Marginal distributions

(a) For each Xi ,

XX XX X
f (xi ) = ... .. f (x1 , x2 , .., xk−1 )
x1 x2 xi−1 xi+1 xk−1
| {z }
k−2summations
n!
= pxi i (1 − pi )n−xi ; xi = 0, 1, 2, .., n
xi !(n − xi )!

marginal distribution of each component is binomial distribution.

(b) The marginal distribution of X1 , X2 , .., Xr (r < k) is given by

!n−∑ri=1 xi
n! X
r
f (x1 , x2 , .., xr ) = Pr px1 1 ..pxr r 1 − pi
x1 !x2 !...xr ! (n − i=1 xi )! i=1

marginal distribution is multinomial distribution.

2
 Conditional distribution
the conditional distribution of X1 , X2 , .., Xr given Xr+1 , Xr+2 , .., Xk−1 is given by
!xk
m! Xr
f (x1 , x2 , .., xr ) = q x1 q x2 ....qrxr 1 − qj
x1 !x2 !...xr !xk ! 1 2 j=1

where
X
k−1
pj
m=n− xi and qj = ; j = 1, 2, .., r
r+1
p1 + p2 + ...pr + pk
Generating function of multinomial distribution:
Let X′ = [X1 , X2 , .., Xk−1 ] be a random vector having multinomial distribution.

(a) The joint moment generating function of the distribution is given by

XX X n! Y
k−1
M (t1 , t2 , ..tk−1 ) = .. (pi eti )xi pxkk
x1 x2 xk−1
x1 !x2 !...xk−1 ! i=1
t1
= [p1 e + p2 et1 + ... + pk−1 etk−1 + pk ]n

where t = (t1 , t2 , .., tk−1 )ϵℜk−1

(b) The joint probability generating function of the distribution is given by

XX X n! Yk−1
G(s1 , s2 , ..sk−1 ) = .. (pi si )xi pxkk
x1 x2 xk−1
x1 !x2 !...xk−1 ! i=1
= [p1 s1 + p2 s2 + ... + pk−1 sk−1 + pk ]n

where s = (s1 , s2 , .., sk−1 )

(c) The joint characteristic function of the distribution is given by

XX X n! Y
k−1
Φ(t1 , t2 , ..tk−1 ) = .. (pj eitj )xj pxkk
x1 x2 xk−1
x1 !x2 !...xk−1 ! j=1
it1
= [p1 e + p2 eit1 + ... + pk−1 eitk−1 + pk ]n

where t = (t1 , t2 , .., tk−1 )ϵℜk−1 and i is imaginary number with property i2 = −1

3
 Finding moments using generating functions:

(a) To obtain the first two moments; E[Xj ] and E[Xj2 ]

(i) Using moment generating function

 
∂
E[Xj ] = M (t)
∂tj t=(0,0,..,0)
i.e. obtain the first derivative of the moment generating function with respect
to tj and then substitute for all values of vector t equal to zero.
Next
 
∂2
E[Xj2 ] = M (t)
∂t2j t=(0,0,..,0)
i.e. obtain the second derivative of the moment generating function with respect
to tj and then substitute for all values of vector t equal to zero.
(ii) Using probability generating function
 
∂
E[Xj ] = G(s)
∂sj s=(1,1,..,1)
i.e. obtain the first derivative of the probability generating function with respect
to sj and then substitute for all values of vector s equal to one.
Next
 
∂2
E[Xj2 ] = G(s)
∂s2j s=(1,1,..,1)
i.e. obtain the second derivative of the probability generating function with
respect to sj and then substitute for all values of vector s equal to one.
(iii) Using characteristic function
h i
∂
∂tj
Φ(t)
t=(0,0,..,0)
E[Xj ] =
i
i.e. obtain the first derivative of the characteristic function with respect to tj
and then substitute for all values of vector t equal to zero. Divide the derivative
by imaginary number i

4
 Next
h i
∂2
∂t2j
Φ(t)
t=(0,0,..,0)
E[Xj2 ] =
i2
i.e. obtain the second derivative of the characteristic function with respect to tj
and then substitute for all values of vector t equal to zero. Divide the derivative
by the square of the imaginary number i

(b) To obtain E[Xj , Xh ]

(i) Using moment generating function

  
∂ ∂
E[Xj Xh ] = M (t)
∂tj ∂th t=(0,0,..,0)

(ii) Using probability generating function

  
∂ ∂
E[Xj Xh ] = G(s)
∂sj ∂sh s=(1,1,..,1)

(iii) Using characteristic function

h  i
∂ ∂
∂tj ∂th
Φ(t)
t=(0,0,..,0)
E[Xj Xh ] =
i2

Example 1:
Let X′ = [X1 , X2 , X3 ] have joint probability distribution function

20!
f (x1 , x2 , .., x3 ) = 0.2x1 0.25x2 0.35x3 0.2(20−x1 −x2 −x3 )
x1 !x2 !x3 !(20 − x1 − x2 − x3 )!

Find

(i) Pr(X1 = 4, X2 = 6, X3 = 7)

(ii) the conditional distribution of X2 given that X1 = 4, X3 = 8

5
 Solution:

(i)

20!
Pr(X1 = 4, X2 = 6, X3 = 7) = 0.24 0.256 0.357 0.23 = 0.0094
4!6!7!3!

(ii) the marginal distribution of X1 , X3 is

20!
h(x1 , x3 ) = 0.2x1 0.35x3 (1 − 0.2 − 0.35)20−x1 −x3
x1 !x3 ! (20 − x1 − x3 )!

Let g(x2 |x1 , x3 ) be the conditional distribution required

f (x1 , x2 , x3 )
g(x2 |x1 , x3 ) =
h(x1 , x3 )
20!
= 0.2x1 0.25x2 0.35x3 0.2(20−x1 −x2 −x3 )
x1 !x2 !x3 !(20 − x1 − x2 − x3 )!
20!
÷ 0.2x1 0.35x3 (1 − 0.2 − 0.35)20−x1 −x3
x1 !x3 ! (20 − x1 − x3 )!
  x2  20−x1 −x2 −x3
(20 − x1 − x3 )! 0.25 0.2
=
x2 !(20 − x1 − x2 − x3 )! 1 − 0.2 − 0.35 1 − 0.2 − 0.35

given X1 = 4, X3 = 8; hence
  x2  20−4−x2 −8
(20 − 4 − 8)! 0.25 0.2
g(x2 |x1 , x3 ) =
x2 !(20 − 4 − x2 − 8)! 1 − 0.2 − 0.35 1 − 0.2 − 0.35
 x2  12−x2 2
12! 0.25 0.2
=
x2 !(12 − x2 )! 0.45 0.45

Example 2:
Let the joint characteristic function of the distribution of a random vector X be

Φ(t1 , t2 , t3 , t4 ) = [0.1eit1 + 0.3eit2 + 0.1eit3 + 0.2eit4 + 0.3]1 5

Find E[X1 , X4 |X2 = 5, X3 = 2] and Var[X1 , X4 |X2 = 5, X3 = 2]

6
Solution:
Using the uniqueness property of characteristic function; the joint probability distri-
bution of X is
15!
f (x1 , x2 , x3 , x4 ) = 0.1x1 0.3x2 0.1x3 0.2x4 0.315−x1 −x2 −x3 −x4
x1 !x2 !x3 !x4 !(15 − x1 − x2 − x3 − x4 )!

Next the marginal characteristic function of X2 , X3 is

Φ(0, t2 , t3 , 0) = [0.1ei0 + 0.3eit2 + 0.1eit3 + 0.2ei0 + 0.3]15

= [0.1 + 0.3eit2 + 0.1eit3 + 0.2 + 0.3]15
= [0.3eit2 + 0.1eit3 + 0.6]15

Hence by the uniqueness property of characteristic functions, the marginal distribu-

tion of X2 , X3 is
15!
h(x2 , x3 ) = 0.3x2 0.1x3 0.615−x2 −x3
x2 !x3 !(15 − x2 − x3 )!

Thus the conditional distribution is

f (x1 , x2 , x3 , x4 )
g(x1 , x4 |x2 , x3 ) =
h(x2 , x3 )
15!
= 0.1x1 0.3x2 0.1x3 0.2x4 0.315−x1 −x2 −x3 −x4
x1 !x2 !x3 !x4 !(15 − x1 − x2 − x3 − x4 )!
15!
÷ 0.3x2 0.1x3 0.615−x2 −x3
x2 !x3 !(15 − x2 − x3 )!
 x1  x4  15−x1 −x2 −x3 −x4
(15 − x2 − x3 )! 0.1 0.2 0.3
=
x1 !x4 !(15 − x1 − x2 − x3 − x4 )! 0.6 0.6 0.6

Given X2 = 5, X3 = 2

 x1  x4  8−x1 −x4

8! 0.1 0.2 0.3
g(x1 , x4 |x2 , x3 ) =
x1 !x4 !(15 − x1 − 5 − 2 − x4 )! 0.6 0.6 0.6
 x1  x4  8−x1 −x4
8! 0.1 0.2 0.3
=
x1 !x4 !(8 − x1 − x4 )! 0.6 0.6 0.6

The conditional distribution is a trinomial with two components X1 , X4

7
Hence
" # " #
E[X1 |X2 = 5, X3 = 2] 8 × 1/6 = 4
E[X1 , X4 |X2 = 5, X3 = 2] = = 3
E[X4 |X2 = 5, X3 = 2] 8 × 2/6 = 8
3

and
" # " #
8 × 1/6 × 5/6 −8 × 1/6 × 2/6 10
− 94
V ar(X1 , X4 |X2 = 5, X3 = 2) = = 9
−8 × 2/6 × 1/6 8 × 2/6 × 4/6 − 49 16
9