5.

Discrete probability distributions

Jong-Min Lee, PhD

Computational NeuroImage Analysis Lab.,
Dept. of Biomedical Engineering,
Hanyang University
 Distributions

• Discrete Uniform Distribution

• Binomial and Multinomial Distribution

• Hypergeometric Distribution

• Poisson Distribution

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Discrete Uniform Distribution

• The simplest PDF for discrete RVs

• Probability of observing a particular value of X is equal for all

possible values of X

• RV X: values of x1, x2, . . . , xk with equal probabilities, then the

discrete uniform distribution is given by
1
• f(x; k) = , for x = x1, x2, . . . , xk
k
• Since f(x) is dependent on k, the function is written as f(x; k)

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Discrete Uniform Distribution:
Mean and variance

k
k ∑i=1 xi
∑
•
μ = E[X] = xi f(xi) =
i=1
k
k
k ∑i=1 (xi − μ)2
2 2
∑
•
σ = Var(X) = (xi − μ) f(xi) =
i=1
k

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 1

• X = die #1 result
1

{0
for x = 1,2,...,6
Uniform PDF: f(x) = 6
•
otherwise
k
∑i=1 xi 1 + 2 + . . . + 6 21
•μ= = = = 3.5
k 6 6
k
∑i=1 (xi − μ)2 (1 − 3.5)2 (6 − 3.5)2
2
• σ = = + ... + = 2.9167
k 6 6

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Binomial distribution

• n trials

• Two possible outcomes (success or failure) of a trial

• Trials are independent of each other

• Probability of success: p

• Remains constant for each trial

• Probability of a failure: 1 − p

• RV X: # of successes

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 2: Field goal kicking

• X: # of eld goals that are successful

• X = 0,1,2,3,4

• P(S) = 0.6: success,P(F) = 0.4: failure

P(X = 0) = P(1stF ∩ 2ndF ∩ 3rdF ∩ 4thF )
= P(1stF ) ⋅ P(2ndF ) ⋅ P(3rdF ) ⋅ P(4thF ) ← independent
•
= P(F ) ⋅ P(F ) ⋅ P(F ) ⋅ P(F ) ← each trial has the same probability
= (1 − p)4 = 0.44 = 0.0256

P(X = 1) = P(1stS ∩ 2ndF ∩ 3rdF ∩ 4thF )

x f(x)
+P(1stF ∩ 2ndS ∩ 3rdF ∩ 4thF )
+P(1stF ∩ 2ndF ∩ 3rdS ∩ 4thF ) 0 0.0256
+P(1stS ∩ 2ndF ∩ 3rdF ∩ 4thS ) 1 0.1536
= P(S ) ⋅ P(F ) ⋅ P(F ) ⋅ P(F ) + P(F ) ⋅ P(S ) ⋅ P(F ) ⋅ P(F )
2 6(0.6)2(0.4)2=0.3456
+P(F ) ⋅ P(F ) ⋅ P(S ) ⋅ P(F ) + P(F ) ⋅ P(F ) ⋅ P(F ) ⋅ P(S )
• = 0.6 ⋅ 0.4 ⋅ 0.4 ⋅ 0.4 + 0.4 ⋅ 0.6 ⋅ 0.4 ⋅ 0.4 3 4(0.6)3(0.4)1=0.3456
+0.4 ⋅ 0.4 ⋅ 0.6 ⋅ 0.4 + 0.4 ⋅ 0.4 ⋅ 0.4 ⋅ 0.6
4 1(0.6)4(0.4)0=0.1296
= 4 ⋅ 0.61 ⋅ 0.43 = 0.1536
Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University
fi
 Binomial PDF & CDF

f(x; n, p) = b(x; n, p)

(x)
Binomial PDF: n
• = p x(1 − p)n−x x = 0,1,2,...,n

F(x; n, p) = P(X ≤ x) = B(x; n, p)

∑(t)
x
Binomial CDF: n t n−t
• = p (1 − p)
t=0

∑(t)
1
n
F(1) = p t(1 − p)n−t
Example: t=0

(0) (1)
• n 0 n n
= p (1 − p) + p 1(1 − p)n−1

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Bernoulli PDF

• Special case of binomial PDF

• # of trials: 1 (i.e. n = 1)

• f(x; 1,p) = b(x; 1,p) = p x(1 − p)1−x for x = 0,1

• Since ( x ) is 1 for x = 0 or 1
1

• Suppose X1, X2, . . . , Xn are independent RVs with Bernoulli PDF

n

(y)
n y n−y
∑ i
•
Y = X has a Binomial PDF of f(y; n, p) = p (1 − p)
i=1

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Binomial distribution:
Mean and variance
• μ = E[X] = np

• σ 2 = Var(X) = np(1 − p)
n n
n!
p x(1 − p)n−x
∑ ∑ x!(n − x)!
E(X ) = xf(x) =
x=0 x=0
n
(n − 1)! x−1
p (1 − p)n−x
∑ x!(n − x)!
= np x
x=0
n
(n − 1)! x−1
p (1 − p)n−x
∑ x!(n − x)!
Proof: = np x
x=1
n
(n − 1)!
p x−1(1 − p)n−x
∑ (x − 1)!(n − x)!
= np
• x=1
n−1
(n − 1)!
p y(1 − p)n−y−1 = np ⋅ 1 = np
∑ y!(n − y − 1)!
= np
y=0

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 3

• n = 4, p = 0.6
x f(x) x∗f(x)

0 0.0256 0

• Find the mean and variance 1 0.1536 0.1536

2 0.3456 0.6912
• μ = 4 * 0.6 = 2.4 3 0.3456 1.0368

4 0.1296 0.5184
2
• σ = 4 × 0.6 × (1 − 0.6) = 0.96
4

∑
What would μ be if we just used the formula of μ = xf(x)?
•
x=0
4

∑
•
μ= xf(x) = 0 + 0.1536 + 0.6912 + 1.0368 + 0.5184 = 2.4
x=0

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Shape of the binomial PDF

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Hypergeometric Distribution

• Suppose

• Pot has N balls with k of them being red and N − k of

them being white, and

• n ≤ N balls are drawn from the pot without replacement

• x: # of red balls drawn out

• Hypergeometric PDF

(x) ( n − x )
k N−k
f(x; N, n, k) = h(x; N, n, k) =
(n)
• N

• where, max{0,n − (N − k)} ≤ x ≤ min{k, n}

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University
 Example 4

• N = 10, k = 4, n = 3 and x = 2

(x) ( n − x)
k N−k
f(2) =
( n)
N

(2) ( 3 − 2)
4 10 − 4
=

( 3)
10
•
6*6 3
= =
120 10

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Hypergeometric Distribution:
Mean and variance
nk
• μ = E[X] =
N
2 N−n k k
σ = Var(X) = n (1 − )
N−1 N N
nk(N − n)(N − k)
=
N 2(N − 1)

[N − 1]
• N−n
= npq

k N−k
• where, p = , q=
N N
• Proof: similar to the proof for the binomial PDF

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 5

• Tea taster claimed that, when drinking tea, she could distinguish
whether milk of tea was added to the cup rst

• Design an experiment in which she tasted N = 8 cups of tea

• k = 4 cups which had milk added rst, and the other N − k = 4

cups had tea added rst

• She was told there were four cups of each type

• The cups were presented in random order

Guess Pour First

Milk Tea

Poured Milk n=4

First
Tea 4
k=4 N-k=4 N=8

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

fi
fi
fi
 Example 5

• Suppose she really can not differentiate

• What is the probability that she guesses all k = 4 of the milk

added rst cups correct? i.e, what is f(4) = P(X = 4)?

(x) ( n − x ) (4) (0 − 4)
k N−k 4 8−4
1
f(4) = = = = 0.0143
(n) ( 4)
• N 8 70

x f(x)
• What are the probabilities of observing
0 0.0143
the other possible values of X?
1 0.2286
2 0.5143
3 0.2286
4 0.0143

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

fi
 Example 5

• Suppose she did get all 4 correct

• Given this PDF, what do you think about whether or not

she really can differentiate?

• Answering questions like this will be VERY important for

hypothesis testing

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Poisson Distribution

• Poisson PDF: # of outcomes in a given time interval or

speci ed region, t, and the average # of outcomes per
unit of time or region, λ

e −λt(λt)x
• f(x; λt) = p(x; λt) = for x = 0,1,2,...
x!

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

fi
 Example 6

• Consider a telemarketing operator who, on the average,

handles 5 phone calls every 3 minutes

• What is the probability that there will be no phone calls

in the next 3 minutes?

• X: # of phone calls in a time interval

• 3 min. is 1 unit (t = 1) and λ = 5

e −5(5)0 −5
• P(X = 0) = f(0) = = e = 0.0067
0!

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 6

• Consider a telemarketing operator who, on the average,

handles 5 phone calls every 3 minutes

• What is the probability that there will be no phone calls

in the next 1 minute?

• 1 min. is 1/3 unit (t = 3 ) and λ = 5

1

e −5/3(5/3)0 −5/3
• P(X = 0) = f(0) = = e = 0.1889
0!

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 6

• Consider a telemarketing operator who, on the average, handles

5 phone calls every 3 minutes

• 2 or more phone calls in the next 1 minute?

P(X ≥ 2) = P(X = 2) + P(X = 3) + P(X = 4) + P(X = 5) + . . .
= 1 − P(X = 0) − P(X = 1)
e −5/3(5/3)0 e −5/3(5/3)1
• =1− −
0! 1!
= 1 − 0.1889 − 0.3148 = 0.4963

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Poisson Distribution:
Mean and variance
• μ = E[X] = λt

• σ 2 = Var(X) = λt
∞ ∞
e −λt(λt)x
∑ ∑
E(X ) = xf(x) = x
x=0 x=0
x!
∞ x
(λt)
= e −λt
∑ x!
x
x=1
∞ x ∞
−λt (λt) −λt (λt)x−1
∑ (x − 1)! ∑ (x − 1)!
Proof: =e = e (λt)
x=1 x=1
∞ y
(λt)
= e −λt(λt)
• ∑ (y)!
y=0
a ∞
b
= e −λt(λt) ⋅ e λt since e b =
∑ a!
= λt
a=0

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Example 7

• Consider a telemarketing operator who, on the average,

handles 5 phone calls every 3 minutes

• For the one unit of time (3 minutes),

• μ = E[X] = 5, σ 2 = Var(X) = 5
• For the 1/3 unit of time (1 minute),

• μ = E[X] = 5/3, σ 2 = Var(X) = 5/3

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University

 Shape of the Poisson distribution

Computational NeuroImage Analysis Lab., Dept. of Biomedical Engineering, Hanyang University