19/01/2014

The Binomial and Poisson

Probability Distributions

You can download this presentation (and check for updates) from
http://www1.gantep.edu.tr/~andrew/eee283/BinomialPoisson.pdf

Dr Andrew Beddall
Version 1 (19/01/2014)

Content
- Permutations and Combinations
- The Binomial Coefficient
- Distributions of Outcomes
- Bernoulli Trials
- The Binomial pmf
- The Poisson pmf

1
 19/01/2014

Permutations and Combinations

are the arrangement of elements under different rules.
We are interested in counting how many arrangements there are.

Take a set of n=3 elements {A,B,C}

If we dont allow repetition of elements, ABC BAC CAB
and if order counts, then there are
3! = 6 permutations:
ACB BCA CBA

If we now form arrangements with two ABC BAC CAB

elements and if order counts, then there
are 3!/(3-2)! = 6/1 = 6 permutations: ACB BCA CBA

If order does not count then there are ABC BAC CAB
3!/2!(3-2)! = 6/2(1) = 3 combinations: ACB BCA CBA

Permutations and Combinations are the arrangement of elements.

We are interested in counting how many arrangements there are.

PERMUTATIONS: COMBINATIONS:
arrangements where arrangements where
order is counted order is not counted
repetition repetition repetition repetition
is counted is not counted is not counted is counted
For a set of nFor a set of n For a set of n For a set of n
elements, elements there are elements there is elements,
select k n! permutations. one combination. select k
elements Select k elements Select k elements elements
there are there are there are there are
nk P(n,k) = n! / (n-k)! C(n,k) = P(n,k) / k! (n+k-1)!
permutations. permutations. = n! / k!(n-k)! k!(n-1)!
combinations. combinations.
We are interested in the third case.
This is called the Binomial Coefficient and it is very important in probability.
4

2
 19/01/2014

COMBINATIONS: arrangements where order is not counted

For a set of n elements there is one combination.
Repetition (of elements) Select k elements there are
is not counted C(n,k) = P(n,k) / k! = n! / k!(n-k)! combinations.

Example 1: 4! = 24 permutations
Take the set of n=4 elements { A, B, C, D }
ABCD ABDC ACBD ACDB ADBC ADCB
We do not allow repetition of elements, BACD BADC BCAD BCDA BDAC BDCA
and the order of the elements is not CABD CADB CBAD CBDA CDAB CDBA
counted; so there is only one combination, DABC DACB DBAC DBCA DCAB DCBA
i.e. ABCD
AB BA CA DA
Select k=2 elements, there are P(4,2) =
AC BC CB DB
(4)(3) = 12
C(n,k) = P(n,k) / k! AD BD CD DC
permutations
= n! / k!(n-k)! = 4!/2!(4-2)!
= 6 combinations.
AB C(4,2) =
We divide by k! because there are k! AC BC (4)(3)/2! = 6
ways we can order the combinations AD BD CD combinations
but order is not counted.
5

Example 2 (more elements):

Take the set of n=5 5! = 120 permutations
elements { A, B, C, D, E } ABCDE ABCED ABDCE ABDEC ABECD ABEDC
Select k=2 elements ACBDE ACBED ACDBE ACDEB ACEBD ACEDB
ADBCE ADBEC ADCBE ADCEB ADEBC ADECB
AEBCD AEBDC AECBD AECDB AEDBC AEDCB
P(5,4) = (5)(4) = 20 permutations BACDE BACED BADCE BADEC BAECD BAEDC
AB BA CA DA EA BCADE BCAED BCDAE BCDEA BCEAD BCEDA
BDACE BDAEC BDCAE BDCEA BDEAC BDECA
AC BC CB DB EB
BEACD BEADC BECAD BECDA BEDAC BEDCA
AD BD CD DC EC CABDE CABED CADBE CADEB CAEBD CAEDB
AE BE CE DE ED CBADE CBAED CBDAE CBDEA CBEAD CBEDA
CDABE CDAEB CDBAE CDBEA CDEAB CDEBA
CEABD CEADB CEBAD CEBDA CEDAB CEDBA
AB DABCE DABEC DACBE DACEB DAEBC DAECB
AC BC DBACE DBAEC DBCAE DBCEA DBEAC DBECA
AD BD CD DCABE DCAEB DCBAE DCBEA DCEAB DCEBA
AE BE CE DE DEABC DEACB DEBAC DEBCA DECAB DECBA
EABCD EABDC EACBD EACDB EADBC EADCB
C(5,2) = n! / k!(n-k)! = 5!/2!(5-2)! EBACD EBADC EBCAD EBCDA EBDAC EBDCA
= 120/12 = 10 combinations ECABD ECADB ECBAD ECBDA ECDAB ECDBA
EDABC EDACB EDBAC EDBCA EDCAB EDCBA

We do not allow repetition of elements, and the order of the elements is not counted.
6

3
 19/01/2014

The Binomial Coefficient

The number of combinations C(n,k) where
order is not counted and repetition (of elements) is not counted
is called the Binomial Coefficient.
See http://en.wikipedia.org/wiki/Binomial_coefficient

Sometimes this is written as which reads n select k.

It is sometimes more convenient to evaluate as a ratio of k terms:

k terms

e.g.

Some properties:

And remember n! = n(n-1)(n-2)(n-3)...(1) for n>0

and 0! = 1.

The Binomial Coefficient is an important tool in probability.

In the following pages we develop probability and use the

binomial coefficient to form distributions of probabilities.

4
 19/01/2014

Consider throwing three dice

{111 112 113 114,115,116, 411,412,413,414,415,416,
121,122,123,124,125,126, 421,422,423,424,425,426,
131,132,133,134,135,136, 431,432,433,434,435,436,
141,142,143,144,145,146, 441,442,443,444,445,446,
Set of all possible outcomes, 151,152,153,154,155,156, 451,452,453,454,455,456,
161,162,163,164,165,166, 461,462,463,464,465,466,
each equally likely.
211,212,213,214,215,216, 511,512,513,514,515,516,
There are 221,222,223,224,225,226, 521,522,523,524,525,526,
63 = 216 permutations 231,232,233,234,235,236, 531,532,533,534,535,536,
(order is counted). 241,242,243,244,245,246, 541,542,543,544,545,546,
251,252,253,254,255,256, 551,552,553,554,555,556,
Now what is the probability 261,262,263,264,265,266, 561,562,563,564,565,566,
of obtaining exactly two 4s? 311,312,313,314,315,316, 611,612,613,614,615,616,
321,322,323,324,325,326, 621,622,623,624,625,626,
Solution: 331,332,333,334,335,336, 631,632,633,634,635,636,
there are 15 such 341,342,343,344,345,346, 641,642,643,644,645,646,
outcomes out of 216, so 351,352,353,354,355,356, 651,652,653,654,655,656,
the probability 361,362,363,364,365,366, 661,662,663,664,665,666}
P(k=2) = 15/216.
Note that: P(k=0) = 125/216, P(k=1) = 75/216, P(k=2) = 15/216, P(k=3) = 1/216
9

Distributions of Outcomes

no 4s P(k=0) = 125/216
one 4s P(k=1) = 75/126
two 4s P(k=2) = 15/216
three 4s P(k=3) = 1/216
Sum = 216/216

We see that the probability of k outcomes is distributed between 0 and n=3.

The total probability 216/216 = 1; that is all outcomes have been considered
(Axiom 2 of probability).
But how can we calculate such probabilities without forming and counting huge
lists of outcomes?
The solution lies with the theory of Bernoulli Trials and the Binomial Coefficient ...

10

5
 19/01/2014

Bernoulli Trials
A Bernoulli trial has only two possible outcomes,
for example outcome A and outcome B (which is not A).
The probability of outcome A ocurring k-times in n trials can be
calculated as follows:
One possible outcome is: A1.A2.A3....Ak . B1.B2.B3....Bn-k
k terms n-k terms
The probability of this outcome is
P(A1 A2 A3.. ..Ak B1 B2 B3.. ..Bn-k)
= P(A1)P(A2)P(A3)...P(Ak) . P(B1)P(B2)P(B3)....P(Bn-k)

= P(A)k . (1-P(A))n-k assuming P(Ai) are all equal.

= k (1- ) n-k where P(A) = and P(B) = 1-

But there are a number of ways we can obtain k outcomes ....
11

We have one possible outcome: A1.A2.A3....Ak . B1.B2.B3....Bn-k

The number of ways we can obtain k outcomes can be considered by counting
the combinations where we do not allow repetition of elements, and the order
of the elements is not counted.
This is n,k combinations and is described by the Binomial Coefficient:

k terms

For example:
for (n=3,k=2) = 3!/2!(1!) = 3 A1A2B1 , A1B1A2 , B1A1A2

And for

A1A2B1B2B3 , B1A1A2B2B3 , B1B2A1A2B3 , B1B2B3A1A2 , A1B1A2B2B3 ,

B1A1B2A2B3 , B1B2A1B3A2 , A1B1B2A2B3 , B1A1B2B3A2 , A1B1B2B3A2

12

6
 19/01/2014

The Binomial pmf

Combining Bernoulli trials and the Binomial coefficient we get:

This discrete distribution (a function of k) is the Binomial probability mass

function. It describes the distribution of probabilities of k out of n outcomes
given that the probability of an individual outcome is .

For example throw three dice, the probability of obtaining exactly two 4s:

= 1/6, k = 2, n = 3

= 15/216.

This is the value we calculated earlier by counting a large list of outcomes!

13

Experiment (computer simulation)

A die is thrown 5 times (65 = 7776 permutations), and the probability of a
number (k=0,1,2,3,4,5) of 4s is calculated from 100 million trials.

k n(k) n(k) / n Binomial

0 40,188,405 0.401884 0.401878 = 1/6
0.4
1 40,191,089 0.401910 0.401878
2 16,075,721 0.160757 0.160751 0.3

3 3,210,682 0.032107 0.032150 0.2

4 321,285 0.003213 0.003215 0.1

5 12,818 0.000128 0.000129 0.0 k

0 1 2 3 4 5
total 100,000,000 1.000000 1.000000

We can see that experiment and theory are in good agreement,

i.e. the Binomial pmf gives the distribution of the probability of k out of n outcomes.
http://www1.gantep.edu.tr/~andrew/eee283/src/cpp/bernoulli_dice.cpp
http://www1.gantep.edu.tr/~andrew/eee283/src/cpp/binomial_pmf.cpp
14

7
 19/01/2014

Note that the mean number of outcomes of these distributions is

mean = n, also variance 2 = n (1-) ( =2), skewness = (1-2)/
15

Example
A department purchases a box of 16 switches.
If there is a 5% chance of each switch being defective,
what is the probability that there are less than three
defective switches in the box?
Solution: use the Binomial pmf with n=16 and = 0.05 and we want P(k<3).
Remember that the Binomial pmf provides a distribution of probabilities,
so P(k<3) = P(k=0) + P(k=1) + P(k=2)

= 1 (0.05)0 (0.95)16
+ 16 (0.05)1 (0.95)15
+ 16(15)/2 (0.05)2 (0.95)14
= 0.4401+0.3706+0.1463
= 0.9570

there is 95.7% chance of less

than three defective switches.

16

8
 19/01/2014

The Poisson pmf

A large number of rare random processes (large n, small )
We often experience random events that are the result of a large number of
processes each with a small probability of an outcome.
For example:
- The number of radioactive decays per minute
(very large number of nuclei, very low prob. of a nuclei decaying).
- The number of web hits per second
(very large number of web users, low prob. of visiting the website).
- The number of phone calls to a help line per hour.
(very large number of phone users, low prob. of each phoning).
In such case, calculating the Binomial pmf is numerically difficult;
however, it can be approximated by the Poisson pmf:

Where is the mean

number of outcomes.

17

Derivation of the Poisson pmf from the Binomial pmf

In the Binomial pmf set n large, and small k << n

Binomial pmf

Binomial
k terms

Poisson pmf

18

9
 19/01/2014

For convenience the Poisson pmf is often used in place of the binomial pmf.
But remember it is an approximation (though sometimes a good one).

Consider the example of throwing 5 dice.

What is the probability of obtaining two 4s?

We should use the Binomial pmf with n=5 and k=2 and = 1/6,

Now lets try the Poisson approximation with = n = 5/6:

This Poisson approximation has an error of about -6%.

19

Properties (the Binomial pmf properties with the limit 0, and =n)
mean = variance 2 = ( =2 = ) skewness = 1/
20

10
 19/01/2014

Example
100 million people access a web site (at a random time) once a year.
What is the probability of the web site receiving more than 3 hits in a one-second period?
Solution:
We have n = 108 , and = 1 / (365 24 3600) = 3.171 10-8 per second.
We can use the Poisson pmf with = n = 3.171

The sum of the terms in this infinite distribution rapidly

converges to 1 (the sum of the first 8 terms is 0.994) .
We want P(k > 3)
0.20
= 1 P(k 3)
0.15
= 1 [ P(k=0) + P(k=1) + P(k=2) + P(k=3) ]
0.10
= 1 [(3.1710 / 0!
0.05 + 3.1711 / 1!
+ 3.1712 / 2!
0.00 k + 3.1713 / 3!) e-3.171 ]
0 1 2 3 4 5 6 7 8 ....
= 1 [0.608981] = 0. 39102 39%
21

Summary

The Binomial pmf

This discrete distribution is the Binomial probability mass function.

It describes the distribution of probabilities of k out of n outcomes given that
the probability of an individual outcome is .

The Poisson pmf

This discrete distribution (a function of k) is the Poisson probability mass

function. It describes the distribution of probabilities of k given the mean
number of outcomes .

22

11