5.

Special Distributions
1. Bernoulli density function
The random variable 𝑋𝑋 takes 0 with probability𝑞𝑞 and 1 with probability 𝑝𝑝.
𝑝𝑝0 = 𝑝𝑝(0) = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 = 0] = 𝑞𝑞
𝑝𝑝1 = 𝑝𝑝(1) = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 = 1] = 𝑝𝑝
𝑝𝑝𝑖𝑖 = 𝑝𝑝(𝑖𝑖) = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 = 𝑖𝑖] = 0, 𝑖𝑖 ≠ 0,1
Where 𝑝𝑝 + 𝑞𝑞 = 1

The mean and variance of this distribution are:

𝜇𝜇 = 𝐸𝐸[𝑋𝑋] = 0𝑞𝑞 + 1𝑝𝑝 = 𝑝𝑝
𝜎𝜎 2 = 𝑝𝑝𝑝𝑝
2. Binomial distribution

Let
𝑋𝑋 = 𝑋𝑋1 + 𝑋𝑋2 + ⋯ + 𝑋𝑋𝑛𝑛

Where 𝑋𝑋𝑖𝑖 follows Bernoulli distribution, 𝑖𝑖 = 1, . . . , 𝑛𝑛, and they are independent, then
𝑋𝑋follows binomial probability distribution. It is the number of “successes” in 𝑛𝑛
independent Bernoulli trials

32
Let 𝑝𝑝𝑖𝑖 be the probability that there are 𝑖𝑖 times of successes in the 𝑛𝑛 independent
Bernoulli trials, then
𝑛𝑛 𝑛𝑛!
𝑝𝑝𝑖𝑖 = � � 𝑝𝑝𝑖𝑖 𝑞𝑞 𝑛𝑛−𝑖𝑖 = 𝑝𝑝𝑖𝑖 𝑞𝑞 𝑛𝑛−𝑖𝑖 𝑖𝑖 = 0,1,2, . . . , 𝑛𝑛
𝑖𝑖 𝑖𝑖! (𝑛𝑛 − 𝑖𝑖)!
where 𝑝𝑝 is the probability of success of each Bernoulli trial.

The mean and variance of this distribution are:

𝜇𝜇 = 𝐸𝐸[𝑋𝑋] = 𝐸𝐸[𝑋𝑋1 ] + 𝐸𝐸[𝑋𝑋2 ] + ⋯ + 𝐸𝐸[𝑋𝑋𝑛𝑛 ] = 𝑝𝑝 + 𝑝𝑝 + ⋯ + 𝑝𝑝 = 𝑛𝑛𝑛𝑛

𝜎𝜎 2 = 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋) = 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋1 ) + 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋2 ) + ⋯ + 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑛𝑛 ) =
= 𝑝𝑝𝑝𝑝 + 𝑝𝑝𝑝𝑝 + ⋯ + 𝑝𝑝𝑝𝑝 = 𝑛𝑛𝑛𝑛𝑛𝑛

For 𝑝𝑝 = 0.5, 𝑛𝑛 = 5 we have:

5 5!
𝑝𝑝0 = � � 0. 50 0. 55−0 = 0. 50 0. 55−0 = 0. 55 = 0.03125
0 0! (5 − 0)!
5 5! 5!
𝑝𝑝1 = � � 0. 51 0. 55−1 = 0. 51 0. 55−1 = 0. 55 = 5 × 0.03125 = 0.15625
1 1! (5 − 1)! 4!

and so on. Figure 7.2 shows the distribution with different values of 𝑝𝑝and 𝑛𝑛

33
34
3. Geometric distribution
In 𝑛𝑛 independent Bernoulli trials, the probability 𝑝𝑝𝑖𝑖 that the first failure occurs in the 𝑖𝑖 -
th trial, where 𝑝𝑝 is the probability of success of each trial.
𝑝𝑝𝑖𝑖 = 𝑃𝑃𝑃𝑃[ 𝑌𝑌 = 𝑖𝑖] = 𝑝𝑝𝑖𝑖−1 𝑞𝑞
Figure 7.3 shows the distribution with different values of 𝑝𝑝and 𝑛𝑛

The mean and variance of this distribution are

1 𝑝𝑝
𝜇𝜇 = , 𝜎𝜎 2 =
𝑞𝑞 𝑞𝑞2

35
4. Poisson distribution
A discrete random variable with the following distribution function:
𝜆𝜆𝑖𝑖 𝑒𝑒 −𝜆𝜆
𝑝𝑝𝑖𝑖 = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 = 𝑖𝑖] = , 𝑖𝑖 = 0,1,2, . ..
𝑖𝑖!
For example, for 𝜆𝜆 =2.5, we have

(2.5)0 𝑒𝑒 −2.5
𝑝𝑝0 = = 0.082
0!
(2.5)1 𝑒𝑒 −2.5
𝑝𝑝1 = = 2.5 × 0.082 = 0.205,
1!
(2.5)2 𝑒𝑒 −2.5 0.205
𝑝𝑝2 = = 2.5 × = 0.257
2! 2
(2.5)3 𝑒𝑒 −2.5 0.257
𝑝𝑝3 = = 2.5 × = 0.214,
3! 3
… ..
(2.5)10 𝑒𝑒 −2.5
𝑝𝑝10 = = 0.0002
10!
For 𝜆𝜆 =4.5, we have
(4.5)0 𝑒𝑒 −4.5
𝑝𝑝0 = = 0.011,
0!
(4.5)1 𝑒𝑒 −4.5
𝑝𝑝1 = = 4.5 × 0.011 = 0.05,
1!
(4.5)2 𝑒𝑒 −4.5 0.05
𝑝𝑝2 = = 4.5 × = 0.1125
2! 2
(4.5)3 𝑒𝑒 −4.5
𝑝𝑝3 = = 4.5 × 0.1125/3 = 0.169,
3!
.....
(4.5)10 𝑒𝑒 −4.5
𝑝𝑝10 = = 0.01
10!

36
Figure 7.4 shows the distribution function with different values of 𝜆𝜆.

The mean and variance of this distribution function are:

𝜇𝜇 = 𝜆𝜆
𝜎𝜎 2 = 𝜆𝜆
Table 7.1 gives the cases when Poisson distribution can be used as an approximation to
the Binomial distributions

37
5. Uniform distribution
A continuous distribution with the variable taking all values equally likely within the
defined range. The density function is:
1
𝑓𝑓(𝑥𝑥) = �𝑏𝑏 − 𝑎𝑎 , 𝑎𝑎 ≤ 𝑥𝑥 ≤ 𝑏𝑏
0, otherwise
The cumulative function is:
0, 𝑥𝑥 < 𝑎𝑎
𝑥𝑥 − 𝑎𝑎
𝐹𝐹(𝑥𝑥) = � , 𝑎𝑎 ≤ 𝑥𝑥 ≤ 𝑏𝑏
𝑏𝑏 − 𝑎𝑎
1, 𝑥𝑥 > 𝑏𝑏
The mean and variance of this distribution function are:
𝑎𝑎 + 𝑏𝑏
𝜇𝜇 =
2
2
(𝑏𝑏 − 𝑎𝑎)2
𝜎𝜎 =
12

Figure 7.5 shows the distribution function.

38
6. The negative exponential distribution
The density function is:
−𝜆𝜆𝜆𝜆
𝑓𝑓(𝑥𝑥) = �𝜆𝜆𝑒𝑒 , 0 ≤ 𝑥𝑥 ≤ ∞
0, 𝑥𝑥 < 0
The cumulative function is:
−𝜆𝜆𝜆𝜆
𝐹𝐹(𝑥𝑥) = �1 − 𝑒𝑒 , 0 ≤ 𝑥𝑥 ≤ ∞
0, 𝑥𝑥 < 0
The mean and variance of this distribution function are:
1
𝜇𝜇 =
𝜆𝜆
1
𝜎𝜎 2 = 2
𝜆𝜆
For example, when 𝜆𝜆 =1, we have:
𝑓𝑓(0) = 𝜆𝜆𝑒𝑒 −𝜆𝜆𝜆𝜆 = 𝑒𝑒 0 = 1, 𝑓𝑓(0.5) = 𝑒𝑒 −0.5 = 0.61, 𝑓𝑓(0.95) = 𝑒𝑒 −0.95 = 0.387,
𝑓𝑓(3.2) = 𝑒𝑒 −3.2 = 0.041
when 𝜆𝜆 =4, we have
𝑓𝑓(0) = 4𝑒𝑒 −𝜆𝜆𝜆𝜆 = 4𝑒𝑒 0 = 4, 𝑓𝑓(0.5) = 4𝑒𝑒 −4×0.5 = 0.409, 𝑓𝑓(0.95) = 4𝑒𝑒 −4×0.95 = 0.089,
𝑓𝑓(3.2) = 4𝑒𝑒 −4×3.2 = 0.00001

Figure 7.6 shows the distribution function with different values of 𝜆𝜆.

39
“Memoryless” property of negative exponential distribution.
As shown in Figure 7.7 below:

40
 Let 𝑋𝑋′ = 𝑋𝑋 − 𝑠𝑠, then the conditional probability:
𝐺𝐺(𝑥𝑥) = 𝑃𝑃𝑃𝑃[ 𝑋𝑋′ ≤ 𝑥𝑥|𝑋𝑋 > 𝑠𝑠] = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 − 𝑠𝑠 ≤ 𝑥𝑥|𝑋𝑋 > 𝑠𝑠]
𝑃𝑃𝑃𝑃[ 𝑋𝑋 ≤ 𝑥𝑥 + 𝑠𝑠 and 𝑋𝑋 > 𝑠𝑠]
= 𝑃𝑃𝑃𝑃[ 𝑋𝑋 ≤ 𝑥𝑥 + 𝑠𝑠|𝑋𝑋 > 𝑠𝑠] =
𝑃𝑃𝑃𝑃[ 𝑋𝑋 > 𝑠𝑠]
𝑥𝑥+𝑠𝑠 𝑥𝑥+𝑠𝑠 𝑥𝑥+𝑠𝑠
−𝜆𝜆𝜆𝜆
𝑃𝑃𝑃𝑃[ 𝑠𝑠 < 𝑋𝑋 ≤ 𝑥𝑥 + 𝑠𝑠] ∫𝑠𝑠 𝜆𝜆𝑒𝑒 −𝜆𝜆𝜆𝜆 𝑑𝑑𝑑𝑑 −𝑒𝑒 �𝑠𝑠 𝑒𝑒 −𝜆𝜆𝜆𝜆 �𝑠𝑠 𝑒𝑒 −𝜆𝜆(𝑥𝑥+𝑠𝑠) − 𝑒𝑒 −𝜆𝜆𝜆𝜆
= = ∞ = = ∞ =
𝑃𝑃𝑃𝑃[ 𝑋𝑋 > 𝑠𝑠] ∫𝑠𝑠 𝜆𝜆𝑒𝑒 −𝜆𝜆𝜆𝜆 𝑑𝑑𝑑𝑑 −𝑒𝑒 −𝜆𝜆𝜆𝜆 |∞
𝑠𝑠 𝑒𝑒 −𝜆𝜆𝜆𝜆 �𝑠𝑠 0 − 𝑒𝑒 −𝜆𝜆𝜆𝜆

𝑒𝑒−𝜆𝜆(𝑥𝑥+𝑠𝑠)
= + 1 = 1 − 𝑒𝑒 −𝜆𝜆𝜆𝜆 = 𝐹𝐹(𝑥𝑥) = 𝑃𝑃𝑃𝑃[ 𝑋𝑋 ≤ 𝑥𝑥]
−𝑒𝑒−𝜆𝜆𝜆𝜆

This result shows that the conditional probability is the same as if the condition does not
exist. The opposite is also true as any continuous memoryless distribution will be a
negative exponential distribution.

7. Normal distribution
The density function is:
1 2 ⁄2𝜎𝜎 2
𝑓𝑓(𝑥𝑥) = 𝑒𝑒 −(𝑥𝑥−𝜇𝜇) , −∞ ≤ 𝑥𝑥 ≤ +∞
√2𝜋𝜋𝜎𝜎
The cumulative function is:
𝑥𝑥
1 2 ⁄2𝜎𝜎 2
𝐹𝐹(𝑥𝑥) = � 𝑒𝑒 −(𝑡𝑡−𝜇𝜇) 𝑑𝑑𝑑𝑑
−∞ √2𝜋𝜋𝜎𝜎

The mean is 𝜇𝜇and variance is 𝜎𝜎 2

41
8. Central Limit Theorem
Let 𝑋𝑋1 , 𝑋𝑋2 , . . . , 𝑋𝑋𝑛𝑛 , . .. be a sequence of mutually independent, identically distributed
random variables with
𝐸𝐸[𝑋𝑋𝑖𝑖 ] = 𝜇𝜇 and 𝑉𝑉𝑉𝑉𝑉𝑉[𝑋𝑋1 ] = 𝜎𝜎 2
We consider:
𝑆𝑆𝑛𝑛 = 𝑋𝑋1 + 𝑋𝑋2 +. . . +𝑋𝑋𝑛𝑛
We know that the mean of 𝑆𝑆𝑛𝑛 is 𝐸𝐸[𝑆𝑆𝑛𝑛 ] = 𝑛𝑛𝑛𝑛. Since 𝑋𝑋1 , 𝑋𝑋2 , . . . , 𝑋𝑋𝑛𝑛 , . .. are independent,
we have 𝑉𝑉𝑉𝑉𝑉𝑉[𝑆𝑆𝑛𝑛 ] = 𝑛𝑛𝜎𝜎 2

To standardize this variable, we define:

𝑆𝑆𝑛𝑛 − 𝑛𝑛𝑛𝑛 (𝑋𝑋1 − 𝜇𝜇) + (𝑋𝑋2 − 𝜇𝜇) + ⋯ + (𝑋𝑋𝑛𝑛 − 𝜇𝜇)
𝑍𝑍𝑛𝑛 = =
√𝑛𝑛𝜎𝜎 √𝑛𝑛𝜎𝜎
so that it will have mean 0 and variance 1.

42
The Central Limit Theorem
If 𝐹𝐹𝑛𝑛 (𝑥𝑥) is the distribution function of 𝑍𝑍𝑛𝑛 , then
𝑥𝑥
1 2 ⁄2
𝑙𝑙𝑙𝑙𝑙𝑙 𝐹𝐹𝑛𝑛 (𝑥𝑥) = 𝑙𝑙𝑙𝑙𝑙𝑙 𝑃𝑃𝑃𝑃[ 𝑍𝑍𝑛𝑛 ≤ 𝑥𝑥] = � 𝑒𝑒 −𝑢𝑢 𝑑𝑑𝑑𝑑
𝑛𝑛→∞ 𝑛𝑛→∞ √2𝜋𝜋 −∞

It says that 𝐹𝐹𝑛𝑛 (𝑥𝑥) approaches standard normal distribution as n becomes large.

Example 7.3
For a sequence of independent variables, 𝑋𝑋1 , 𝑋𝑋2 , . . . , 𝑋𝑋𝑛𝑛 , . .., following the same uniform
distribution in the range of −√3 ≤ 𝑥𝑥 ≤ √3. Let 𝑆𝑆𝑛𝑛 = 𝑋𝑋1 + 𝑋𝑋2 +. . . +𝑋𝑋𝑛𝑛 . Since all of
them have mean 0 and variance 1, the standardized 𝑍𝑍𝑛𝑛 is
𝑆𝑆𝑛𝑛
𝑍𝑍𝑛𝑛 =
√𝑛𝑛
The density function of 𝑍𝑍𝑛𝑛 for 𝑛𝑛 = 1,2,3 are shown in Figure 7.9 below.

43
A stronger version of the central limit theorem says that:
When 𝑋𝑋1 , 𝑋𝑋2 , . . . , 𝑋𝑋𝑛𝑛 , . .. are independent random variables with means 𝜇𝜇1 , 𝜇𝜇2 , . . . , 𝜇𝜇𝑛𝑛 , . .. and
variances 𝜎𝜎1 , 𝜎𝜎2 , . . . , 𝜎𝜎𝑛𝑛 , . .., then for 𝑆𝑆𝑛𝑛 = 𝑋𝑋1 + 𝑋𝑋2 +. . . +𝑋𝑋𝑛𝑛
we have
𝑆𝑆𝑛𝑛 − ∑𝑛𝑛𝑖𝑖=1 𝜇𝜇𝑖𝑖
𝑍𝑍𝑛𝑛 =
�∑𝑛𝑛𝑖𝑖=1 𝜎𝜎𝑖𝑖2

approach N(0,1) distribution when n approaches infinity.

9. Approximations to probability distributions (H stands for Hypergeometric distribution)

44