STAT2001_CH02B Page 1 of 14

Combinatorics (counting tools)

The mn rule
If a {a1 , , am } and b {b1 , , bn } , then the number of different (ordered) pairs
(a,b) is mn.
Example 7

Find the pr of getting at least one 6 on 2 rolls of a die.

Let (5,3) denote a 5 coming up on the 1st roll and a 3 on the 2nd, etc, and abbreviate
(a,b) by ab. Then the sample space is
S = {11,12,13,...,66} = {ab : a {1,..., 6}, b {1,..., 6}} .
Hence (by the mn rule with m = 6 and n = 6), nS = 6 6 = 36 .
Next let A = At least one 6 comes up.
Then A ={16, 26, 36, 46, 56, 66, 65, 64, 63, 62, 61}, and so nA =11.
Since all possible outcomes are equally likely, P(A) = nA / nS = 11/36.
Illustration with Venn diagram:

Factorials
For n a positive integer, we define n! = n(n 1)(n 2)...(3)(2)(1).
We also define 0! = 1.
Eg: 4! = 4(3)(2)(1) = 24.

STAT2001_CH02B Page 2 of 14

Theorem 5

The number of different possible arrangements of n distinct objects in

a row is n!.

Example 8

How many different 3-letter words can be formed from the letters
A, C, T (assuming that no letter can be used more than once).

There are 3! = 3(2)(1) = 6 words (ACT, ATC, CAT, CTA, TAC, TCA).

Permutations
An ordered arrangement of objects is called a permutation (eg CAT above). The
number of different permutations of r objects which can be formed from n distinct
objects is denoted Prn .
(We say n-permute-r. Other notations: P(n,r),

Theorem 6 Prn =

Example 9

n!
(n r )!

Pr ,

Pr )

n(n 1)...(n r + 1) (n r )!

=
.
=

+
n
(
n
1)...(
n
r
1)

(n r )!

How many different words of length 3 can be formed from the letters
A, C, T, E, W (assuming that no letter can be used more than once)?

There are P35 =

5!
120
=
= 60 words (ACT, ACE, ..., CAT, ..., TAC, ..., WET).
(5 3)!
2

(To see this another way, there are 5 possibilities for the 1st letter in the word,
then 4 possibilities for the 2nd, and finally 3 for the 3rd. Thus P35 = 5*4*3 = 60.)

Combinations
An unordered collection of objects is called a combination.
The number of different combinations of r objects that can be selected
n
from n distinct objects is denoted .
r
(We say n-choose-r. Other notations: C(n,r), Crn , nCr ,

Theorem 7

n
n!
=

r r !(n r )!

Cr .)

Prn n(n 1)...(n r + 1)

.
=
=
r ! r (r 1)...(r r + 1)

STAT2001_CH02B Page 3 of 14

Example 10 How many different combinations of 3 letters can be formed from

A, C, T, E, W (assuming that no letter can be used more than once)?
5
5!
120
There are =
=
= 10 combinations
3 3!2! 6(2)
(ACT, ACE, ACW, ATE, ATW, AEW, CTE, CTW, CEW, TEW).
(Note: Each of these combinations corresponds to 3! = 6 permutations in Example 9.
For instance, ACT corresponds to permutations ACT, ATC, CAT, CTA, TAC, TCA.
5 P 5 60
Thus, another way to express the number of combinations is as = 3 =
= 10 .)
6
3 3!
A related question: How many different committees of size 3 can be selected from 5
5
people? The answer is the same, = 10. Think of the persons as labelled 1,2,3,4,5
3
& consider how many unordered sets of 3 numbers can be selected from these labels.
Note that another way to derive the same answer is to think about how many sets of 2
5
numbers can be selected from 1,...,5 (so as to form the 'non-committee'), ie = 10.
2

Example 11 A committee of 5 is to be selected randomly from 12 people.

What is the probability that it will contain the two oldest people?
12 12!
The total number of committees is =
= 792.
5 5!7!
12 2 10 10!
The number of committees with the 2 oldest people is
= =
= 120.
5 2 3 3!7!
(If the two oldest persons are definitely on the committee,
then 3 more people need to be selected from the remaining 10.)
Hence the required probability is 120/792 = 5/33 = 0.152
(since all 792 committees are equally likely).

Example 12 Two separate committees of size 3 and 4 are to be selected from 15

people. How many different pairs of committees are possible?

STAT2001_CH02B Page 4 of 14
15
There are possibilities for the 1st committee.
3
15 3 12
For each of these, there are
= possibilities for the 2nd committee.
4 4
So the number of possible pairs of committees is
1512
= 15! 12! = 15! = 225225 .
3 4 3!12! 4!8! 3!4!8!

15
Note 1: Alternatively, there are ways to choose the 2nd committee, and then
4
11
ways to choose the 1st committee. So the number of possible pairs is
3
15

11 = 15! 11! = 15! = 225225 (the same).

3 4!11! 3!8! 3!4!8!
4
15
Note 2: Yet again, there are ways to choose the 'non-committee, and then
8

7

3

ways to select the 1st committee. So the number of possible pairs is

15

7 = 15! 7! = 15! = 225225 .

3 8!7! 3!4! 3!4!8!
8

Note 3: (Theorem 2.3 in the text) The number of ways of partitioning n distinct
objects into k distinct groups, containing r1 ,..., rk objects, respectively is given by

n
n
n r1 n r1 r2 n r1 r2 ... rk 2
n!

...
=
,
r

r r r

r
r

r ! r !
1

k 1

where it is assumed that r1 + ... + rk = n .

In Example 12, n = 15, r1 = 3, r2 = 4 and
15
so that the answer may be written
3 4

r3 = 8 (the number of persons not selected),

15
12
= = 225225.
4
8 3

Note 4: If we allow for the possibility that r1 + ... + rk < n , we must generalise our

n
n
n!

as equal to
definition of
r1 ... rk rk +1 = r !...r !r ! , where
r1 rk
1
k k +1

rk +1 = n r1 ... rk . We may then write

3
n
n

are short for

Likewise, and
r
n r
r

15 15
as
or
, etc.
3 4 8 3
4 8
n
n
and
, respectively.
n r r
n r

15

STAT2001_CH02B Page 5 of 14

Example 6 (again)

Whats the pr of getting 2 Hs on 5 tosses of a coin?

S = {HHHHH, HHHHT, ...}. How many sample points are there in S?

nS = 25 = 32 by the mn rule (or, rather, an obvious extension of it).
Let A = Get 2 Hs. How many sample points are there in A?
Now A = {HHTTT, HTHTT, HTTHT, ..., TTTHH}.
12

1 3

45

<---- position numbers

We see that nA is the number of combinations of 2 numbers that can be selected from
5
1,2,3,4,5, namely = 10 (12, 13, 14, 15, 23, 24, 25, 34, 35, 45).
2
Thus nA =10 and P(A) = nA / nS =10/32 = 5/16 = 0.3125.

The bijection principle

In the solution to Example 6 above we made use of the bijection principle, which says
that if there is a bijection (or one-to-one-correspondence) between two sets then those
sets have the same number of objects. (The two sets were A = {HHTTT, HTHTT,
HTTHT, ..., TTTHH} and {12, 13, ...,45}, with each set having 10 objects.)
To give another illustration of the bijection principle, consider the problem of
counting the number of ways that three identical rings can be arranged on the five
fingers of one hand, assuming that each finger can have any number of rings.
To solve this problem we set up a bijection between arrangements of rings and
arrangements of 0, 0, 0, +, +, +, + in a row. Here, each 0 stands for a ring
and each + represents a division between two fingers. For example:
000++++

<----->

All 3 rings on Finger 1

00+++0+

<----->

Two rings on Finger 1 and one ring on Finger 4

++0+0+0

<----->

One ring on each of Fingers 3, 4 and 5, etc.

7
We see that the total number of arrangements of rings must be = 35.
3
What if the rings are all different? Then each arrangement of rings can be represented
by an arrangement of 1, 2, 3, +, +, +, + in a row. Now, if the numbers were 1, 2, 3, 4,
5, 6, 7, the answer would be 7!. But there are 4! ways that 4, 5, 6, 7 can be arranged in
a row. So 7! is too large by a factor of 4! and the answer must be 7!/4! = 7*6*5 = 210.

STAT2001_CH02B Page 6 of 14

Lattice paths
Consider the following lattice and all the possible paths from P = (0,0) to Q = (4,3) ,
moving only up and right along the lines:

Each path must go up exactly 3 times and go right exactly 4 times, in any order.
Some possibilities are RURRUUR (shown), RRRRUUU and RRRUURU.
We see that there is a one-to-one correspondence between paths and arrangements of
4 + 3 7
= = 35.
4 R's and 3 U's in a row. So the total number of paths must be
3 3
Now consider the fact that each path must pass through either (3,3) or (4,2).

3 + 3
,
The number of paths going through (3,3) is
3
4 + 2
.
and the number going through (4,2) is
2
4 + 3 3 + 3 4 + 2
6
+
. (Check: +
It follows that
=
3 3 2
3

6
= 20 + 15 = 35.)
2

Generalizing this logic, we now consider all paths from P = (0,0) to Q = (a,b) and get

a + b (a 1) + b a + (b 1)

=
+
.
b
b
b 1
m + 1 m m
This can also be written
= +
, which is known as Pascal's Identity.
k k k 1

STAT2001_CH02B Page 7 of 14

Note 1: Pascal's Identity can also be proved algebraically, as follows:

m!
m!
m+ m =
+
k k 1 k !(m k )! (k 1)!(m k +1)!

m !(m k +1)
m !k
+
k !(m k + 1)! k !(m k +1)!

m!
((m k + 1) + k )
k !(m k + 1)!

m + 1
(m +1)!
=
.
k !((m +1) k )! k

Note 2: Another proof of Pascal's Identity is to consider the number of committees of

k persons that can be formed from m men and 1 woman. The answer is obviously
m +1

. But this must be the same as the number of committees with k men (i.e., no
k
m
women), which is , plus the number with k 1 men, which is
k

.
k 1

Note 3: Pascal's Identity is a special case of Vandermonde's Identity:

m + w m
m
w m w
w+ m
w.

= +
+ ... +

0
k 1 k 1
1 k
k 0
k 1
This can be proved using lattice paths. But the simplest proof is to consider the
number of committees of k persons that can be formed from n men and w women. The
m + w
m w
. But the number of committees with no men is ,
answer is obviously
k
0 k

m w
the number with one man is
, and so on. Summing up proves the result.
1 k 1

Note 4: In Vandermonde's Identity, k may be greater than m or w. For example,

3 +1 = 31+ 31+ 31 = 1 0 + 31 + 31 = 0 + 3 + 3 = 6.
2 1
1 2
0
2 0

STAT2001_CH02B Page 8 of 14

In some cases the sample point method is impractical, even with assistance from
combinatorics. This leads us to consider the second major strategy for computing
probabilities.

The event composition method (2 steps)

1.

Express the event of interest, A (say), as a composition (ie function) of other

events (using unions, intersections, complementations, etc).

2.

Apply any relevant results to this composition in an attempt to compute P(A).

Example 13 Two dice are rolled . Find the pr that the sum of the numbers which
come up is at least 4.
1.

Let A = Sum is at least 4 and B = Sum is less than 4.

We may write A = B (or A = S B ).
(A is expressed as a function of another event, B, or of other events, S and B).

2.

Now B = {11, 12, 21}, so that P ( B ) = nB / nS = 3 / 36 = 1/12 .

So P ( A) = P ( B ) = 1 P ( B )

(by Theorem 3 from earlier)

= 1 1/12
= 11/12.
We have here given an example involving one of the simplest forms of composition ,
namely complementation.
Another such example involving complementation is provided by an alternative
solution to Problem 7 (Find the pr of getting at least one 6 on 2 rolls of a die).
Let A = "At least one six". Then A = "No sixes" = "Get 1 to 5 on each of the rolls",
so that nA = 5 5 = 25 (using the mn rule again) and hence P ( A) = nA / nS = 25 / 36 .
Thus P ( A) = 1 P ( A) = 1 25/36 = 11/36, as before.
This solution via the event composition method is only slightly simpler and easier than
the solution via the sample point method used in Example 7 (where A was listed and
all 11 of its elements counted). However, in many problems the event composition
method will be very much simpler and easier to apply than the sample point method.

STAT2001_CH02B Page 9 of 14

Lets now develop some more concepts and results that are useful for computing
probabilities using the event composition method.

Conditional probability
For any two events A and B such that P(B) > 0,
the conditional probability of A given B is defined as
P ( AB )
P( A | B) =
.
P( B)
If P(B) = 0, then P(A|B) is undefined.

Example 14 A die is rolled. What is the pr that the number which comes up is even,
given that it is greater than 3?
Let A = An even number comes up and B = A number greater than 3 comes up.
Then A = {2,4,6}, B = {4,5,6}, AB = {4,6}, P(AB) = 2/6 and P(B) = 3/6 (> 0).
Hence P(A|B) = P(AB)/P(B) = (2/6)/(3/6) = 2/3.
Alternatively, P(A|B) = nAB / nB = 2 / 3 .

Another interpretation: If someone rolled a die and we learn that a number greater
than 3 came up, then we can be 66.7% confident that that number was even.
Yet another interpretation: Suppose we will roll a die repeatedly until a number
greater than 3 comes up. Then the probability that the last number will be even is 2/3.
Yet another: We roll a die millions of times and each time write the number that
comes up if it's more than 3. Then about 2/3 of the written numbers will be even.

STAT2001_CH02B Page 10 of 14

Independence
Two events A and B are said to be independent if P(AB) = P(A)P(B),
in which case we write A B .
If P(AB) P(A)P(B), then A and B are dependent and we write A B .

It can be shown that: If P(A|B) = P(A) or P(B|A) = P(B), then A B .

If P(A|B) P(A) or P(B|A) P(B), then A B .

Example 15 In Example 14, are A and B independent events?

P(A)P(B) = (3/6)(3/6) = 9/36 and P(AB) = 2/6. Thus P(AB) P(A)P(B).
So no, A and B are not independent; they are dependent.
(Another solution: P(A|B) = 2/3 3/6 = P(A) A B .)

The multiplicative law of probabilitity (MLP)

P(AB) = P(A)P(B|A).
(This follows directly from the definition of conditional pr.)

Example 16 Two cards are to be drawn from 5 white cards and 3 black cards.
Find the pr that a white card will be drawn first and then a black card.
Let A = A white card is drawn first and B = A black card is drawn second.
Then P(A) = 5/8 and P(B|A) = 3/7.
Hence P(AB) = P(A)P(B|A) = (5/8)(3/7) = 15/56 = 0.268.

The law of total probability (LTP)

P ( A) = P ( AB ) + P ( AB )
= P( B) P( A | B) + P( B ) P( A | B ) .
A useful corollary: P ( AB ) = P ( A) P ( AB ) .

STAT2001_CH02B Page 11 of 14
Proof of LTP:

P ( A) = P ( AS )
= P ( A( B B ))

= P (( AB ) ( AB )) by the distributive laws

= P ( AB ) + P ( AB ) by Theorem 2 earlier (since ( AB )( AB ) = )
= P ( B ) P ( A | B ) + P ( B ) P ( A | B ) by the MLP.

Example 17
We will consider a location where the following has been found to be true (approx.):
If its cloudy tonight, the pr of rain tomorrow is 70%.
If its not cloudy tonight, the pr of rain tomorrow is only 40%.
The pr of it being cloudy tonight is 20%.
Find the probability that it will rain tomorrow.
Let R = Rain tomorrow and C = Clouds tonight.
Then P(R|C) = 0.7, P ( R | C ) = 0.4 , P(C) = 0.2 (and P (C ) = 0.8 ).
So:

P(CR) = P(C)P(R|C) = 0.2(0.7) = 0.14

(the pr of clouds tonight and rain tomorrow)
P (CR ) = P (C ) P ( R | C ) = 0.8(0.4) = 0.32
(the pr of no clouds tonight and rain tomorrow).

Thus the overall probability of rain tomorrow is

P ( R ) = P (CR ) + P (CR ) = 0.14 + 0.32 = 0.46.
Equivalently, we could write:

P ( R ) = P (C ) P ( R | C ) + P (C ) P ( R | C )
= 0.2(0.7) + 0.8(0.4) = 0.46.

Note that the unconditional pr of rain tomorrow (46%) is an appropriately weighted

average of the two conditional prs (70% and 40%).

STAT2001_CH02B Page 12 of 14

Illustration with Venn diagram:

The additive law of probability

P ( A B ) = P ( A) + P ( B ) P ( AB ) .
Proof: A B = ( A B ) ( AB ) ( B A)
(the union of three disjoint events, as shown in the Venn diagram below)
So P ( A B ) = P ( A B ) + P ( AB ) + P ( B A)

= {P ( A B ) + P ( AB )} + {P ( B A) + P ( AB )} P ( AB )
= P ( A) + P ( B ) P ( AB ) .

STAT2001_CH02B Page 13 of 14

Example 18 Refer to Ex. 17. Whats the pr it will be cloudy tonight

or rain tomorrow?
P (C R ) = P (C ) + P ( R ) P (CR ) = 0.2 + 0.46 0.14 = 0.52.
Note that this covers the possibility of clouds tonight and rain tomorrow.
So, whats the pr it will either be cloudy tonight or rain tomorrow, but not both?
P (C R ) + P ( R C ) = P (C R ) P (CR ) = 0.52 0.14 = 0.38.

Bayes rule (or theorem)

P( B) P( A | B)
P ( A)
P ( B) P( A | B)
=
.
P( B) P( A | B) + P( B ) P( A | B )

P ( B | A) =

(The two equalities here follow trivially from the multiplicative law of pr,
the LTP and the defn of conditional pr.)
We call P(B) the prior probability of B, and P(B|A) the posterior probability of B.
Bayes' rule shows that the posterior is the prior multiplied by a factor P ( A | B ) / P ( A) .

Example 19 Refer to Ex. 17.

Suppose tomorrow has come and its raining.
Whats the pr that it was cloudy last night?
P (C | R ) =

P (C ) P ( R | C ) 0.2(0.7)
=
= 0.304
P( R)
0.46
(the posterior pr that it was cloudy last night).

(Alternatively, P (C | R ) =

P (CR ) 0.14
=
= 0.304 .)
P( R)
0.46

Observe that the posterior pr is higher than the prior pr, P(C) = 0.2.
This makes sense because being cloudy is associated with a higher chance of rain.
Now suppose that tomorrow has come and its not raining. What then is the pr of it
being cloudy last night? We suspect that this pr is lower than 0.2. Let's find out.

STAT2001_CH02B Page 14 of 14

P (C | R ) =

P (CR ) P (C ) P ( R | C ) P (C ){1 P ( R | C )} 0.2(1 0.7)

=
=
=
= 0.111 ,
P( R )
P( R )
1 P ( R )
1 0.46

which is indeed lower than P(C) = 0.2.

Summary of probabilities in Example 17 to 19:

Event

Probability

Description

----------------------------------------------------------------------------------C

0.2

(prior pr. of clouds tonight)

C'

0.8

(prior pr. of no clouds)

R|C

0.7

(conditional pr. of rain, given clouds)

R|C

0.4

(conditional pr. of rain, given no clouds)

0.46

(unconditional pr. of rain tomorrow)

CR

0.14

C|R

0.14/0.46 = 0.304

CR

0.06

C|R

0.06/0.54 = 0.111 (posterior pr. of clouds, given no rain)

CR

0.48

(posterior pr. of clouds, given rain)

etc.
------------------------------------------------------------------------------------------Venn diagram: