Lecture #5
• The moment generating function.
• Some Discrete probability distributions:
➢ Bernoulli distribution.
➢ Binomial distribution.
1
The moment generating function
Definition: The moment generating function of a discrete random variable 𝑿 is
defined as follows:
𝑴𝑿 𝒕 = 𝑬 𝒆𝒕𝑿 = 𝒆𝒕𝒙 𝒇 𝒙
∀𝒙
Example: Find the moment generating function of the random variable 𝑿
with the probability function:
𝑥 0 1 2
f(𝑥) = 𝑃(𝑋 = 𝑥) 10/28 15/28 3/28
Solution:
𝟏𝟎 𝟏𝟓 𝟑
𝑴𝑿 𝒕 = 𝑬 𝒆𝒕𝑿 =𝒆𝟎𝒕∗ 𝒕
+𝒆 ∗ 𝟐𝒕
+𝒆 ∗
𝟐𝟖 𝟐𝟖 𝟐𝟖
𝟏𝟎 𝟏𝟓 𝒕 𝟑 𝟐𝒕
= + 𝒆 + 𝒆
𝟐𝟖 𝟐𝟖 𝟐𝟖
2
The moment generating function
Example: Let the moment generating function of the random variable 𝑿 is given by:
𝟑 𝒕 𝟏 𝟐𝒕 𝟏 𝟓𝒕 𝟐 𝟕𝒕
𝑴 𝒕 = 𝒆 + 𝒆 + 𝒆 + 𝒆
𝟏𝟎 𝟓 𝟏𝟎 𝟓
(a) Find the probability distribution function (pdf) of the random variable 𝑿.
(b) Find the mean and variance of 𝑿.
Solution:
(a) 𝑥 1 2 5 7
𝑓(𝑥) = 𝑃(𝑋 = 𝑥) 3/10 1/5 1/10 2/5
𝟑 𝟏 𝟏 𝟐 𝟐𝟔
(b) 𝝁𝑿 = 𝒙𝒇(𝒙) = 𝟏 ∗ +𝟐∗ +𝟓∗ +𝟕∗ =
𝒂𝒍𝒍𝒙
𝟏𝟎 𝟓 𝟏𝟎 𝟓 𝟏𝟎
𝟑 𝟏 𝟏 𝟐 𝟐𝟑𝟐
𝑬 𝑿𝟐 = 𝒙𝟐 𝒇 𝒙 =𝟏 𝟐 ∗ + 𝟐𝟐 ∗ + 𝟓𝟐 ∗ + 𝟕𝟐 ∗ =
𝟏𝟎 𝟓 𝟏𝟎 𝟓 𝟏𝟎
𝒂𝒍𝒍𝒙
𝑽𝒂𝒓 𝑿 = 𝑬 𝑿𝟐 − 𝝁𝟐 = 𝟏𝟔. 𝟒𝟒
3
The moment generating function
Theorem: Let 𝑴𝑿 𝒕 be the moment generating function of a discrete random
variable 𝑿, then the mean and the variance of 𝑿 can be calculated as follows:
𝝁𝑿 = 𝑬 𝑿 = 𝑴′ 𝑿 𝟎 , 𝑬 𝑿𝟐 = 𝑴″ 𝑿 𝟎
𝑽𝒂𝒓 𝑿 = 𝑴″ 𝑿 𝟎 − 𝑴′ 𝑿 𝟎 𝟐
Proof: Similarly,
𝒅𝟐 𝒅𝟐 𝒕𝒙
∵ 𝑴𝑿 𝒕 = 𝑬 𝒆𝒕𝑿 = 𝒆𝒕𝒙 𝒇 𝒙 ∴ 𝟐 𝑴𝑿 𝒕 = 𝒅𝒕𝟐 ( 𝒆 𝒇 𝒙 )
𝒅𝒕 ∀𝒙
∀𝒙
𝒅 𝒅 = 𝒆𝒕𝒙 𝒙𝟐 𝒇 𝒙
∴ 𝑴𝑿 𝒕 = ( 𝒆𝒕𝒙 𝒇 𝒙 ) = 𝒆𝒕𝒙 𝒙𝒇 𝒙
𝒅𝒕 𝒅𝒕 ∀𝒙
∀𝒙 ∀𝒙
At 𝒕 = 𝟎
At 𝒕 = 𝟎 ∴ 𝑴′ 𝑿 𝟎 = 𝒙𝒇 𝒙 = 𝝁𝑿 ∴ 𝑴″ 𝑿 𝟎 = 𝒙 𝟐 𝒇 𝒙 = 𝑬 𝑿 𝟐
∀𝒙 𝒂𝒍𝒍𝒙
4
The moment generating function
Example: Let the moment generating function of the random variable 𝑿 is given by:
𝟓
𝑴𝑿 (𝒕) = (𝟎. 𝟑 + 𝟎. 𝟕𝒆𝒕 )
and 𝒀 = 𝟐 − 𝟒𝑿, find 𝝈𝟐𝒀 .
Solution:
𝝈𝟐𝒀 = 𝝈𝟐𝟐−𝟒𝑿 = (−𝟒)𝟐 ∗ 𝝈𝟐𝑿 = 𝟏𝟔 ∗ (𝑬 𝑿𝟐 − 𝝁𝟐 )
𝟒 𝟒
𝑴′ 𝑿 𝒕 = 𝟓 ∗ (𝟎. 𝟑 + 𝟎. 𝟕𝒆𝒕 ) ∗ 𝟎. 𝟕𝒆𝒕 = 𝟑. 𝟓𝒆𝒕 ∗ (𝟎. 𝟑 + 𝟎. 𝟕𝒆𝒕 )
𝝁𝑿 = 𝑴′ 𝑿 𝟎 = 𝟑. 𝟓
𝟒 𝟑
𝑴″ 𝑿 𝒕 = 𝟑. 𝟓𝒆𝒕 ∗ (𝟎. 𝟑 + 𝟎. 𝟕𝒆𝒕 ) + 14 ∗ 𝟎. 𝟕𝒆𝟐𝒕 ∗ (𝟎. 𝟑 + 𝟎. 𝟕𝒆𝒕 )
𝑬 𝑿𝟐 = 𝑴″ 𝑿 𝟎 = 𝟑. 𝟓 + 𝟗. 𝟖 = 𝟏𝟑. 𝟑
𝝈𝟐𝒀 = 𝟏𝟔 ∗ 𝟏𝟑. 𝟑 − 𝟏𝟐. 𝟐𝟓 = 𝟏𝟔. 𝟖
5
Some discrete probability distributions
A random variable is characterized by either its probability density function
or its cumulative distribution function. The other characteristics of a
random variable are its mean, variance and moment generating function. In
this chapter, we explore some frequently encountered discrete distributions
and study their important characteristics
Some Discrete Probability Distributions
Hypergeometric
Binomial
Poisson Geometric
6
Bernoulli Distribution
A Bernoulli trial is a random experiment in which there are precisely two
possible outcomes, which we conveniently call ‘failure’ (F) and ‘success’ (S).
The probability of success is 𝒑 and the probability of failure is 𝟏 − 𝒑.
We can define a random variable 𝑿 from the sample space {S, F} represent
the number of successes when the experiment is performed.
𝑿 𝑭 = 𝟎, 𝑿 𝑺 =𝟏 ∴ 𝒙 = 𝟎, 𝟏
The probability density function of this random variable is
𝒇 𝟎 =𝑷 𝑿=𝟎 =𝟏−𝒑
𝒇 𝟏 =𝑷 𝑿=𝟏 =𝒑
where 𝒑 denotes the probability of success. Hence
𝒇 𝒙 = 𝒑𝒙 𝟏 − 𝒑 𝟏−𝒙 , 𝒙 = 𝟎, 𝟏
7
Bernoulli Distribution
Theorem: If 𝑿 is a Bernoulli random variable with parameter 𝒑, then the mean,
variance and moment generating functions are respectively given by
𝝁𝑿 = 𝒑 𝝈𝟐𝑿 = 𝒑(𝟏 − 𝒑) 𝑴𝑿 𝒕 = 𝟏 − 𝒑 + 𝒑𝒆𝒕
Proof: The mean of the Bernoulli random variable is
𝟏 𝟏
𝝁𝑿 = 𝒙𝒇 𝒙 = 𝒙𝒇 𝒙 = 𝒙𝒑𝒙 𝟏 − 𝒑 𝟏−𝒙 =𝒑
∀𝒙 𝒙=𝟎 𝒙=𝟎
Similarly, the variance of X is given by
𝟏
𝑬(𝑿𝟐 ) = 𝒙𝟐 𝒇 𝒙 = 𝒙𝟐 𝒇 𝒙
∀𝒙 𝒙=𝟎
𝟏
= 𝒙𝟐 𝒑𝒙 𝟏 − 𝒑 𝟏−𝒙 = 𝟎 𝟐 ∗ 𝒑 𝟎 𝟏 − 𝒑 + 𝟏𝟐 ∗ 𝒑 𝟏 𝟏 − 𝒑 𝟎 =𝒑
𝒙=𝟎
𝝈𝟐𝑿 = 𝒑 − 𝒑𝟐 = 𝒑 𝟏 − 𝒑
8
Bernoulli Distribution
Next, we find the moment generating function of the Bernoulli random
variable
M X ( t ) = E ( etX ) = et x f ( x )
x
1
= et x p x (1 − p ) = (1 − p ) + et p
1− x
x =0
Then, if 𝑿 is a Bernoulli random variable with parameter 𝒑, then the mean,
variance and moment generating functions are respectively given by
𝝁𝑿 = 𝒑 𝝈𝟐𝑿 = 𝒑(𝟏 − 𝒑) 𝑴𝑿 𝒕 = 𝟏 − 𝒑 + 𝒑𝒆𝒕
9
Binomial Distribution
10
Binomial Distribution
➢ Consider a fixed number 𝒏 of mutually independent Bernoulli trails.
➢ Suppose these trials have same probability of success, say 𝒑.
➢ A random variable X is called a binomial random variable if it represents the total
number of successes in 𝒏 independent Bernoulli trials. Thus, to find the probability
density function of 𝑿, we have to find the probability of x successes in 𝒏 independent
trails.
➢ Then, in the binomial distribution:
𝒏 : represent the number of trails.
𝒑 : represent the probability of success in each trial of the 𝒏 trials.
x : represent the number of successes in the 𝒏 trials.
11
Binomial Distribution
➢ If we have x successes in 𝒏 trials, then the probability of each 𝒏 -tuple with x
successes and 𝒏 − 𝒙 failures is
𝒑𝒙 𝟏 − 𝒑 𝒏−𝒙
12
Binomial Distribution
𝒏
However, there are tuples with x successes and 𝒏 − 𝒙 failures in 𝒏 trials.
𝒙
Hence
𝒏 𝒙 𝒏−𝒙
𝒇 𝒙 = 𝒑 𝟏−𝒑
𝒙
Therefore, the probability density function of 𝑿 is
𝒏 𝒙 𝒏−𝒙
𝒇 𝒙 = 𝒑 𝟏−𝒑 𝒙 = 𝟎, 𝟏, 𝟐, … , 𝒏
𝒙
where 0 < 𝒑 < 1 is the probability of success.
We will denote a binomial random variable with parameters 𝒑 and 𝒏 as
𝑿~𝑩𝑰𝑵 𝒏, 𝒑
13
Binomial Distribution
Example: Suppose that 25% of the products of a manufacturing process are
defective. Three items are selected at random, inspected, and classified as
defective (D) or non-defective (N). Find the probability distribution of the
number of defective items.
Solution:
Let 𝑿 be a random variable represent the number of defective items in the sample.
Then, we need to find the probability distribution of 𝑿
Note that: the experiment is performed 𝒏 = 𝟑 times with probability of success 𝒑 = 𝟎. 𝟐𝟓
∴ 𝑿~𝑩𝑰𝑵 𝟑, 𝟎. 𝟐𝟓
𝒙 𝟑−𝒙
𝟑 𝟏 𝟏
∴𝒇 𝒙 =𝑷 𝑿=𝒙 = 𝟏− 𝒇𝒐𝒓 𝒙 = 𝟎, 𝟏, 𝟐, 𝟑
𝒙 𝟒 𝟒
14
Binomial Distribution
𝒙 𝒇 𝒙 =𝑷 𝑿=𝒙
0 3
3 1 3 27
0 0 4 4
=
64
1 2
3 1 3 27
1 1 4 4
=
64
2 1
3 1 3 9
2 =
2 4 4 64
3 0
3 1 3 1
3 3 4 4
=
64
Total 1
15
Binomial Distribution
Example: In the previous example, find the following probabilities:
(1) The probability of getting at least two defective items.
(2) The probability of getting at most two defective items.
Solution:
X 0 1 2 3
𝑓(𝑥) = 𝑃(𝑋 = 𝑥) 27/64 27/64 9/64 1/64
𝟗 𝟏 𝟏𝟎
(1) 𝑷(𝑿 ≥ 𝟐) = 𝑷(𝑿 = 𝟐) + 𝑷(𝑿 = 𝟑) = 𝒇(𝟐) + 𝒇(𝟑) = + =
𝟔𝟒 𝟔𝟒 𝟔𝟒
(2) 𝑷(𝑿 ≤ 𝟐) = 𝑷(𝑿 = 𝟐) + 𝑷(𝑿 = 𝟏) + 𝑷(𝑿 = 𝟎)
𝟗 𝟐𝟕 𝟐𝟕 𝟔𝟑
= 𝒇(𝟐) + 𝒇(𝟏) + 𝒇(𝟎) = + + =
𝟔𝟒 𝟔𝟒 𝟔𝟒 𝟔𝟒
𝟏 𝟔𝟑
or 𝑷(𝑿 ≤ 𝟐) = 𝟏 − 𝑷(𝑿 > 𝟐) = 𝟏 − 𝑷(𝑿 = 𝟑) = 𝟏 − =
𝟔𝟒 𝟔𝟒
16
Binomial Distribution
Example: On a six-question multiple-choice test, each question on the test has
five possible answers, and only one is correct. If a student guesses randomly
and independently, what is the probability that she is correct only on two
questions?
Solution:
Let 𝑿 be a random variable represent the number of correct questions.
Note that: the experiment is performed 𝒏 = 𝟔 times with probability of success 𝒑 = 𝟎. 𝟐
∴ 𝑿~𝑩𝑰𝑵 𝟔, 𝟎. 𝟐
𝟐 𝟔−𝟐
𝟔 𝟏 𝟏
∴ P(correct on two questions) =𝑷 𝑿=𝟐 = 𝟏− = 0.2048
𝟐 𝟓 𝟓
17
Binomial Distribution
Theorem: If 𝑿 is binomial random variable with parameters 𝒑 and 𝒏, then the
mean, variance and moment generating functions are respectively given by
𝒏
𝝁𝑿 = 𝒏𝒑 𝝈𝟐𝑿 = 𝒏𝒑(𝟏 − 𝒑) 𝑴𝑿 𝒕 = 𝟏 − 𝒑 + 𝒑𝒆𝒕
Proof: First, we determine the moment generating function 𝑴𝑿 𝒕 of X and
then we generate mean and variance from 𝑴𝑿 𝒕 .
M X ( t ) = E ( etX ) = et x f ( x )
x
n
= e p (1 − p ) = ( p e ) (1 − p ) = ( p e + (1 − p ) )
n
n
tx n x n − x t x n− x t n
x =0 x x =0 x
Note: Binomial expansion formula
𝒏
∴ 𝑴𝑿 𝒕 = 𝟏 − 𝒑 + 𝒑𝒆𝒕
18
Binomial Distribution
Therefore
X = E ( X ) = M X ( 0 ) = ( p e + 1 − p ) = n pe ( p e + 1 − p )
d n t t n −1
t
dt t =0
= 𝒏𝒑
Similarly
d2
E ( X 2 ) = M X ( 0 ) = 2 ( p e +1− p) = ( )
n d n −1
+ −
t t t
n pe p e 1 p
dt dt
= n pe ( p e + 1 − p ) + n ( n − 1) ( pe ) ( pe +1− p)
t t n −1 t 2 t n−2
t =0
= n p + n ( n − 1) p 2
Hence
= n p + n ( n − 1) p 2 − ( np )
2
𝑉 𝑥 = 𝐸 𝑋 2 − 𝜇𝑋 2
= 𝒏𝒑 𝟏 − 𝒑
19
Binomial Distribution
Example: Suppose that 25% of the products of a manufacturing process are
defective. Three items are selected at random, inspected, and classified as
defective (D) or non-defective (N). Find the expected value (mean) and the
variance of the number of defective items.
Solution:
Let 𝑿 be a random variable represent the number of defective items in the sample.
Then, we need to find the probability distribution of 𝑿
Note that: the experiment is performed 𝒏 = 𝟑 times with probability of success 𝒑 = 𝟎. 𝟐𝟓
∴ 𝑿~𝑩𝑰𝑵 𝟑, 𝟎. 𝟐𝟓
The expected number of defective items is
𝑬(𝑿) = = 𝒏 𝒑 = (𝟑) (𝟏/𝟒) = 𝟑/𝟒 = 𝟎. 𝟕𝟓
The variance of the number of defective items is
𝑽𝒂𝒓(𝑿) = 𝒏 𝒑 (𝟏 − 𝒑) = (𝟑) (𝟏/𝟒) (𝟑/𝟒) = 𝟗/𝟏𝟔 = 𝟎. 𝟓𝟔𝟐𝟓
20
Binomial Distribution
Example: A package of 6 fuses are tested where the probability an individual
fuse is defective is 0.05. (That is 5% of all fuses manufactured are defective).
1- What is the probability that one fuse will be defective?
2- What is the probability that at least one fuse will be defective?
3- Find mean and standard deviation for the number of defective items?
Solution:
Let X be a random variable represent the number of defective fuses in the package.
Note that: the experiment is performed 𝒏 = 𝟔 times with probability of success 𝒑 = 𝟎. 𝟎𝟓
∴ 𝑿~𝑩𝑰𝑵 𝟔, 𝟎. 𝟎𝟓
1- The probability that one fuse will be defective = 𝑷 𝑿 = 𝟏
= 𝟔𝑪𝟏 𝟎. 𝟎𝟓 𝟏 𝟎. 𝟗𝟓 𝟓 = 𝟎. 𝟐𝟑𝟐𝟏
21
Binomial Distribution
2- The probability that at least one fuse will be defective = 𝑷 𝑿 ≥ 𝟏
=𝟏−𝑷 𝑿<𝟏 =𝟏−𝑷 𝑿=𝟎
= 𝟏 − 𝟔𝑪𝟎 𝟎. 𝟎𝟓 𝟎 𝟎. 𝟗𝟓 𝟔 = 𝟎. 𝟐𝟔𝟒𝟗
3- The mean and standard deviation:
𝝁𝑿 = 𝒏𝒑 = 𝟔 𝟎. 𝟎𝟓 = 𝟎. 𝟑
𝑽𝒂𝒓 𝑿 = 𝒏𝒑 𝟏 − 𝒑 = 𝟔 𝟎. 𝟎𝟓 𝟎. 𝟗𝟓 = 𝟎. 𝟐𝟖𝟓
𝝈𝑿 = 𝑽𝒂𝒓 𝑿 = 𝟎. 𝟐𝟖𝟓
22
Binomial Distribution
Example: The probability that a patient recovers from a rare blood disease is
0.4. If 15 people are known to have contracted this disease, what is the
probability that:
1- at least 10 survive?
2- from 3 to 8 die?
3- at most 3 survive?
Solution:
1- Let X be a random variable represent the number of patients that will survive.
Note that: the experiment is performed 𝒏 = 𝟏𝟓 times with probability of success 𝒑 = 𝟎. 𝟒
∴ 𝑿~𝑩𝑰𝑵 𝟏𝟓, 𝟎. 𝟒
1- The probability that at least 10 survive = 𝑷 𝒙 ≥ 𝟏𝟎 = 𝟏 − 𝑷 𝑿 < 𝟏𝟎
=𝟏−𝑷 𝑿≤𝟗
9
15
=1− 0.4𝑥 1 − 0.4 15−𝑥 = 𝟎. 𝟎𝟑𝟑𝟖
𝑥
𝑥=0
23
Binomial Distribution
2- Let 𝑿 be a random variable represent the number of patients that will die.
Note that: the experiment is performed 𝒏 = 𝟏𝟓 times with probability of success 𝒑 = 𝟎. 𝟔
∴ 𝑿~𝑩𝑰𝑵 𝟏𝟓, 𝟎. 𝟔
The probability that f𝐫𝐨𝐦 𝟑 𝐭𝐨 𝟖 𝐝𝐢𝐞 = 𝑷 𝟑 ≤ 𝑿 ≤ 𝟖
8
15
= 0.6𝑥 1 − 0.6 15−𝑥 = 𝟎. 𝟖𝟕𝟕𝟗
𝑥
𝑥=3
3- Let 𝑿 be a random variable represent the number of patients that will survive.
∴ 𝑿~𝑩𝑰𝑵 𝟏𝟓, 𝟎. 𝟒
The probability that a𝐭 𝐦𝐨𝐬𝐭 𝟑 𝐬𝐮𝐫𝐯𝐢𝐯𝐞 = 𝑷 𝑿 ≤ 𝟑
3
15
= 0.4𝑥 1 − 0.4 15−𝑥
𝑥
𝑥=0
24
Binomial Distribution
Example: It is known that disks produced by a certain company will be defective with
probability 0.01 independently of each other. The company sells the disks in packages
of 10 and offers a money-back guarantee that at most 1 of the 10 disks is defective ( The
guarantee is that the customer can return the entire package of diskettes if he or she
finds more than one defective diskette in it)
(a) What proportion of packages is returned? (probability of returning the package)
(b) If someone buys three packages, what is the probability that exactly one of them
will be returned?
Solution: ∴ 𝑿~𝑩𝑰𝑵 𝟏𝟎, 𝟎. 𝟎𝟏
(a) Let 𝑿 be a random variable represent the number of defective disks.
Note that: the experiment is performed 𝒏 = 𝟏𝟎 times with probability of success 𝒑 = 𝟎. 𝟎𝟏
The proportion of packages is returned = 𝑷 𝑿 > 𝟏 = 𝟏 − 𝑷 𝑿 ≤ 𝟏
= 𝟏 − (𝑷 𝑿 = 𝟏 + 𝑷 𝑿 = 𝟎 )
1
15
=1− 0.01𝑥 1 − 0.01 15−𝑥 = 𝟎. 𝟎𝟎𝟓
𝑥
𝑥=0
25
Binomial Distribution
(b) Let 𝒀 be a random variable represent the number of packages that the person will
have to return.
Note that: the experiment is performed 𝒏 = 𝟑 times with probability of success
𝒑 = 𝟎. 𝟎𝟎𝟓
∴ 𝒀~𝑩𝑰𝑵 𝟑, 𝟎. 𝟎𝟎𝟓
The probability that exactly one of them will be returned = 𝑷 𝒀 = 𝟏
3
= 0.0051 1 − 0.005 2
1
26
Finish
See you next week
