CHAPTER 2

Introduction to Probability

PART 2 Counting Techniques

I General Rules and Constructs

Combinatorics is the field of studying questions of counting

mathematics

and arrangement in discrete systems The goal is to develop methods to avoid

individual outcomes in calculations
counting probability

For example let us count the different ways we can

arrange
3 objects
a b and c next to each other We list all possibilities

a bc b a c cab
so 6
a e b be a cba ways

We could have figured this out with the following reasoning in the
first spot choose the first letter 3 options in the second spot choose

one of the 2 remaining letters 2 options and the last letter in the
goes
third spot I option We get 3 2 1 6 ways This number is
3 factorial
denoted by 3
 Product Rule

Prop If an experiment has a finite number 1 of outcomes and can be

done in k stages such that there are n possibilities for the ith stage
then we have
IS n x Max X Nk

Example Toss a coin twice We have 2x 2 outcomes

Suppose an un contains 3 balls white Green Black If
we draw 2 balls one after the other without replacement we

have 3 2 6 outcomes If they are drawn with replacement

we have 3 3 9 outcomes
W

W G W
G
B B
W
W G
G G
B B
W W
Case 1 B G Case 2 B b
B
Exercise 1 How many words of size 4 are there
2 If the letters are picked randomly what is the probability
that no letter is repeated
 Exercise 1 How many words of size 4 are there
2 If the letters are picked randomly what is the probability
that no letter is repeated

Solution 1 26 26 26 x 26 26

2 ht E be the event in the question There is efuiprobability so

PCE 26 from question 1 The number of
and A1
E
outcomes where we have 4 different letters is
26 IT 24 23 26 25 24 23
Hence IP E 26 25 24 23
264

The answer of points to general pattern of counting ordered

2 selections

of 4 objects out of 26 This is what we call a permutation an

ordered arrangement
 Permutations and Combinations

Recall the first example of permuting 3 objects a b and c Suppose we are

looking for the number of permutations of n objects There are n options for
the first spot n 1 for the second and I for the last By the product
rule there are thus n n 1 2.1 total possibilities
Def The number n n n 1 n 2 3.2.1 denotes the number of
permutations of all the elements of a set of cardinality n

In general permuting means that the order of elements matters Now what
if we want to permute te objects out of a set of n elements

Def The number of permutations of r elements out of a set of n

elements is
given by
pun n ren
Ch r
Example From of 10 people
a 6 will pose for a
photo
group
The number of possible photos we can take is
10 x 9 x 8 x 7 x 6 x 5
10 9.8 7 6.5.4 3 2 7 10 9 8.7.6
or Po 19 J
4 3 2 I

Note that the order matters here which is why we for Palo
go
Note that order does not need to be mentioned literally when a certain
distinction between the chosen objects is imposed an ordered arrangement
is in question For example if we want to select a committee of
3 people a president VP and treasurer out of 10 people then we have
B to LO x 9x 8

What if order does not matter A combination is an unordered

selection of elements in a set Two combinations are thus different only
if they differ in composition
 We derive the formula for a combination of r elements out of n In the
example above suppose the did not include
committee
specific positions Then
every
6 permutation correspond to one combination Why 6 There are
3 positions so 3 6 We thus need
ways to assign the positions
to divide the total number of permutations by 6 and we thus have
10 different combinations
3

Def The number of combinations of r elements out of a set of n

elements is
given by
Cnn
r
n
Ch r I ren

So in general

1 Pi

I is the binomial coefficient and it is read n choose r

Exercise In how many can 2 maths books and 3 biology books
ways
be selected from 8 maths and 6 biology books
Exercise In how many can 2 maths books and 3 biology books
ways
be selected from 8 maths and 6 biology books
Solution
There are Iways
to select the 2 maths books and 5 ways to
select 3 bio books Using the product rule the total number of ways is

x S 8 6 560
92 3 31

From ordinary deck of cards seven cards are drawn at random

Exercise an

without replacement what is the probability that at least one card is a king
Exercise In how many can 2 maths books and 3 biology books
ways
be selected from 8 maths and 6 biology books
Solution
There are Eways
to select the 2 maths books and 5 ways to
select 3 bio books Using the product rule the total number of ways is

x S 8 6 560
92 3 31

From ordinary deck of cards seven cards are drawn at random

Exercise an

without replacement what is the probability that at least one card is a king
Solution
Considerthe event E at least one king It is easier to study E no king

Total outcomes E
Favourable outcomes of E Y chosen out of 48 4 kings

PCE L IPC E E 0.4496

E
 Multinomial Coefficient

Def A multinomial coefficient describes the number of possible partitions

of n objects into k
groups
of sizes ni nu ink s t ni n
If
n

ni nai ing hi ne ne

We show how the formula is derived an example

using

Example We want to split a of 10 children into 3 groups 3 will

group
painting 4 will basketball and 3 will
go play go hiking

First stage 3 kids for painting

choose s
Second
stage
choose 4 kids for basketball out of the 7
remaining I
Third stage the remaining 3 go for hiking 3 1
 Now using the product rule the total number of partitions ways to distribute
the 10 children into 3 groups
is

10
E 7 3
317 1
t
II 3 3 3

Another IMPORTANT interpretation of the multinomial coefficient is the following

Suppose we want to permute all elements of a multiset of size ns.t some

times the 2nd nz times

say the first
elements are is repeated n
repeated
and the k element
na times and n that t
na n Then the total number

of permutations of full size n is

given by the
multinomial coefficient
n inn

Example The number of distinct to permute the letters of the word

ways
MISSISSIPPI ID 4 Is U Sss 2 Ps is

11
1,44 2 34650
1141612
I SamplingModels

In this section we summarizethe different ways to count outcomes in the

context of sampling scenarios You will see in the examples to come that
don't always have to consider this formalism Note here that the
you
population of size n consists of n distinguishable objects
As we have seen before the word order is not to be taken too literally

We have a population of size n and we want to draw a

sample
of size k We have 4 different models
ordered
sample
with replacement
unordered sample

ordered
sample
without replacement
Kan unordered sample
 We quickly considerthe 6 cases The first 3 cases are known to us

1 without replacement and with ordering Pan

Sampling In no

2 Sampling
without replacementand without ordering k n
k N K

3 Sampling with replacementand with ordering n

The last case is more complicated We give the proof in a later section when
we consider its corresponding allocation model Only this part is new

4 Sampling
with replacement and without ordering
th
1
II
Example The number of ways to choose 3 scoops of ice cream from a place
3
that only has 2 flavours n 2 k 3 is ith 13
4 4
I Allocation Models

Main Question How many are there to allocate k objects into n

ways
distinguishable cells boxes
As with sampling models we have 4 different models It all depends on
whether or not the objects themselves are
distinguishable or not and on

whether the allocation occurs with or without exclusion

With exclusion each box can have at most object
one

Without exclusion each box can have any number of objects

distinguishable objects
without exclusion
indistinguishable objects

distinguishable objects
with exclusion
Ren
indistinguishable objects
It turns out that there is a perfect correspondence between the 4 sampling
models and the 4 allocation models Suppose we want to
sample k balls
from a n balls The boxes correspond to the balls in the
jar containg jar
and the objects correspond to the balls drawn from the
jar
In this context sampling
with replacement means we can draw the same ball

many
times i.e the box can contain more than one ball no exclusion

As we have discussed before the order in sampling is equivalent to labelling

the objects s it corresponds to an allocation where we only care how many
balls fall in each box there is no distinction between the objects

Allocation
Sampling
with replacement ordered without exclusion
distinguishable
with replacement
unordered without exclusion indistinguishable
withoutreplacement ordered with exclusion distinguishable
without replacement unordered with exclusion indistinguishable
This allows us to compute the formulae for allocation models

Allocation
Sampling

ordered unordered distinguishable indistinguishable

with without
replacement
n
Ith exclusion
n
it
without with
replacement
Pan y exclusion
Phin
I

How to prove the formula for Ith We use allocation This is

by itself an important case that is very useful in problem solving

Suppose we want to count the number of ways to place k indistinguishable
objects into n boxes s t each box can contain number of objects
any
without exclusion let n 4 and k 5 What matters is how
many
objects fall in each box indistinguishable
We model the 4 boxes using 3 internal separators I
n ingeneral in 1 the
k 5 objects with 5 stars in general k This is an example of an outcome

In a 11am
1 26
Fyi
3

We can thus
map
all the outcomes he the permutations of h 1 bars and k
stats Out of n I k spots choose k spots for the stars Hence the
number of outcomes is
th
ni
as desired
Exercise let f IR s IR beanalytic function of to variables How
an

many partial
derivatives of order 5 exist
Solution Partial derivatives do not depend on order of differentiation We thus

want an unordered sample of size k 5 from a population n to with replacement

5
I
o
ft Try to think about the corresponding allocation problem

Exercise We have seen in the above that the number of ways to allocate 5
4 5
indistinguishable ball into 4 boxes no exclusion is Ith 1 E
Now how many are there to do this s t no box remains empty
ways

Solution Put one of the 5 balls into each of the 4 boxes so that none
remain empty
W a
W W A
W
There is one ball left How many
ways are there to allocate this one ball
1
to 4 boxes 4 The answer is 4
1st
1
I Examples
The only way to master combinatorics is through
practice A lot of practice

We start with an
interesting paradox
Exercise The Birthday Problem How should one randomly
many people
collect so the chance that at least 2 people share the same birthday
is at least o 5
I Examples
The only way to master combinatorics is through
practice A lot of practice

We start with an
interesting paradox
Exercise The Birthday Problem How should one randomly
many people
collect so the chance that at least 2 people share the same birthday
is at least 0.5

Solution We sample N people We compute the probability of the

complement IP no two people share the same birthday Pn365

365N

PN 365
We thus want NS.t 1
N I 0 5
365

365 364 x x 365 NH

F O 5
N
365

computing software
N 23 This means that in a class
Using a we
get
of only 23 students there is a 50 chance that 2 sharethe same birthday
Exercise 1 How words of size 4 are such that exactly one letter
many
is
repeated twice
Solution
Stl Choose 3 letters to be used 3
St 2 Of those 3 pick which letter is to be repeated
twice E
St 3 Now arrange them
Y MISSISSIPPI

We
get 3 x x
Y 93600 different words

2 What if the word consists of 2 letters each repeated twice

Solution
St 1 Choose the 2 letters E
St 2 Arrange 242

4
We get 4 2 2
1950
3 Hander How many 4 letter words of exactly 2 letters are there
3 Hander How many 4 letter words of exactly 2 letters are there
Solution

split the problem into different cases and add

Sometimes we need to
the number of outcomes favourable to each case in the end

Case1 Both letters are repeated twice ab ab We have

4 It outcomes question 2

case 2 One letter repeated 3 times and the second

only once abbb
Stl Choose 2 letters R
St 2 Choose which letter is the one repeated
3times
St3 Arrange
4
3 1
So the number of outcomes of case 2 is E x2 x

The total number of words is 1 42 E x 2x 434

Exercise An un contains 5 whiteballs 5 Red balls and 5 Black balls
5 balls are drawn without replacement whatis the probability that
1 All are white
2 All five are of one color
3 Exactly 2 balls are white
4 Exactly 2 are white and 2 are black
5 At least 2 are white
Exercise An un contains 5 whiteballs 5 Red balls and 5 Black balls
5 balls are drawn without replacement whatis the probability that
1 All are white
2 All five are of one color
3 Exactly 2 balls are white
4 Exactly 2 are white and 2 are black
5 At least 2 are white
Solution
1 IPL All White E 1

E E
2 St 1 Choose which color 3 3
Sf 2 Choose 5 of this color 5
IPC All of one color 3 15
E
3 Stl Choose 2 White balls E
St 2 Choose 3 out of the 10 Black Red balls 5
 IP 2 white I 5
5
4 Plz white 2 Black E E i

5 A Avoid over counting

There are4 disjoint events Ei Exactly i white balls for i 2,314.5

For each case the probability is

PIE Pl exactly i white balls 7 Ei

E
Now add the different cases
just like g
3 in the exercise before
Probabilistically speaking we have a union of 4 disjoint events so

IPL at least 2 white P QE E Ez PIE E Ei

E
Exercise 1 An un contains 3 red and 7 white balls are all drawn
They
one by one and without replacement What is the probability that
the 6th draw is a red ball
Exercise 1 An un contains 3 red and 7 white balls are all drawn
They
one by one and without replacement What is the probability that
the 6th draw is a red ball

Solution R
Choose 2 spots for the remaining red balls out of the 9 spots
The number of favourable outcomes is E
The total number of outcomes is I choose 3 spots for the red balls

9
Pl 6thdraw is red 12 7 9 3
19 3 2 10 2

2 If they are drawn with replacement what is the probability

that the 6th draw is a red ball

Solution 3 The draws are

10 independent
Exercise Roll a die n times Compute Pl all numbers appear if
1 n 6 2 no 7 3 n 8
Exercise Roll a die n times Compute Pl all numbers appear if
1 n 6 2 n 7 3 n 8
Solution
1 Note that this problem is equivalent to choosing a number without replacement
at each roll The total number of outcomes is 6
The number of favourable outcomes is the number of permutations of
123,4 5,61 so all
appeon 6
6
IPL all appear
66

2 If all numbers
appear
then exactly one of the numbers occurs twice
St 1 Choose which number occurs twice E
St 2 Shuffle MISSISSIPPI 7
2

The total number of outcomes is 67

PC all appear 9 x
67
3 If n 8 we have 2 disjoint cases

Carel One number appears

3 time Case 2 Two numbers
appear
twice each
and the others and the others appear only once
appear only once
example 12232456 example 12323456

Stl Pick one to appear 3 times Stl Pick 2 numbers tapper twice
St2 Shuffle 8
St 2 Shuffle 83
2 2

The total number of cases is 68 Hence the probability is

Pl all appear 9 8 6 Ez
68
What if n 9 Hint 3 disjoint cases
Exercise An ordinary deck of 52 card is randomly dealt to 4 players s r
each player receives B cards Compute the
probability of the event
E each player receives an ace
Solution
The total number of to allocate 52 cards in 4 distinct
ways groups
of 13 cards each is
gz
Ir 13,13 13,13 93413113
13

Now there are 4 aces and 48 other cards To compute the favourable cases
Stl Choose an ace for 1stplayer Y
SEZ Choose 12 other cards for 1stplayer Y
St 3 Choose an ace for 2ndplayer
St 4 Choose 12 other cards
for 2ndplayer
St 5 Choose
ace for 3rdplayer k
St 6 Choose 12 cards for 3rd
player
Y
St 7 Remaining cards go to 4th i 1
 IE 9 1 F E 4 IN E 4 40
4
42
4 98 13
PIE
52

The sample space A of the positions of the 52 and deck

consists

that are occupied by 4 aces but it does not matter which ace is
in which position In this case we only care about the position of aces
I Y or Mississippi
In this scenario the first 13 spots to the first player the next
go
13 spots to the second and so on

St 1 Choose
spot of 1st ace among
1st B B 13
St 2 Choose
spot of 2nd ace among
2nd B B 13
St 3 Choose spot of 3rd ace among 3rd 13 B 13
St4 Choose
spot of 4th ace among 4th B R 13 11 13
134 4 48 134
PL E same answer
5421
52
 Consider all couds to be distinguishable and as before the first B
cards to the first second 13 to the 2ndplayer and so on
go
Ir 52
Stl Choose which ace to which player Pu
goes 4
St 2 3 4 5 Repeat the 4 steps of 172 3 7 1 1 134

St 6 Shuffle all remaining 48 cards 48

4 13 98
PCE same answer
52

Conclusion The way think about the problem and set

Important you up the
steps for the product rule depends largely on how
define
you your sample space
Exercise 1 6 adults and 6 children to be divided into 3 distinctgroups of
are
size 4 each What is the probability that each consists of
group
exactly 2 adults and 2 children if allocation is random
Exercise 1 6 adults and 6 children
to be divided into 3 distinct groups of
are
size 4 each What is the probability that each consists of
group
exactly 2 adults and 2 children if allocation is random
Solution The total number of allocations is u u u II
Favourable cases

Stl Choose 2 adults and 2 children for first

group
EE
SEZ Choose 2 adults and 7 children for 2ndgroup I E
3 I E
Stl Remaining in group 1
El E E E I 61 and PCE 6 4
12 x 26

2 How are there to form these of 2 adults

many ways groups
R 2 children if the are not distinct
groups
Exercise 1 6 adults and 6 children
to be divided into 3 distinct groups of
are
size 4 each What is the probability that each consists of
group
exactly 2 adults and 2 children if allocation is random
Solution The total number of allocations is u u u II
Favourable cases

Stl Choose 2 adults and 2 children for first

group
EE
SEZ Choose 2 adults and 7 children for 2ndgroup I E
3 I E
Stl Remaining in group 1
El E 6 E A 61 and PCE

2 How are there to form these of 2 adults

many ways groups
R 2 children if the are not distinct
groups
Solution
The answer 18 counts one outcome 3 6 times allocating 3 sets
to 3 Hence the answer is 16 1350
groups 26 31
Exercise 10 distinguishable balls are thrown randomly in 3 boxes s t each ball
has the same chance to fall in ofthe boxes What is the probability
any
that no box remains empty
Exercise 10 distinguishable balls are thrown randomly in boxes s t each ball
3
has the same chance to fall in any ofthe boxes What is the probability
that no box remains empty

Solution Note that we cannot consider the scenario of indistinguishable balls

because this method
only
counts the number of balls in each box

let Ei be the event that box i stays empty i 1,2 3 The event we wantis
MEIN EJ AMBI E U E U Ez

10
For each i IP Ei 210310
3 distinguishable balls without exclusion
For each i PC Ei NE 119310
g 43

PIE ME NED O obviously

Now using the principle of inclusion exclusion

PL E UE UE I PL E UE UE
PLED PLED PIE PPLEED BLEED
I
I
PL E E PL E ELE
I 3x 143 3x b t o

8339 0.948

If we considered the balls to be indistinguishable

7 1

IP no box empty 3 0.5454 different answer

q y
To convince yourself why this is case compute the probability of the
the
allocation lo o d in each of the 2 cases

Further exercise What if we had 5 boxes instead of 3

Exercise To indistinguishable balls are placed in 5 distinguishable urns Compute

the probability of the following events

1 Exactly 2 uns remain empty

2 Uns 4 and 5 combined receive at most 3 balls
Exercise To indistinguishable balls are placed in 5 distinguishable urns Compute

the probability of the following events

1 Exactly 2 uns remain empty

2 Uns 4 and 5 combined receive at most 3 balls

Solution
1
1 The total number of cases is 111 1915 4
To count the number of favourable cases
Stl Choose which 2 uns remain empty E
St 2 Place a ball in the other 3 and distribute
the remaining 7 balls on 3 wins IT
IPLExactly 2 wins empty
E E
2 The event consists of 3 disjoint cases
Case Ans 4 and 5 Combined receive 0 balls
1 3 1
Distribute 10 balls on 3 boxes 1

Case2 Uns Yand 5 combined receive 1 ball

1
Stl Distribute the ball on wins 9 and 5 I
St 2 Distribute 9 balls the other 3 boxes 9 1
on
3 g
4 15

Case3 Mns 4 and 5 combined receive 2 balls Same reasoning as

2 1 8 1
case 2 we
get 3 8 3 8

and
k 3 s 3 8
get
PYat
s

most 3 balls
Exercise A class contains 8 boys and 4 girls We line them randomly
up
in a straight line What is the probability that exactly 2 girls
come together and that the 2 other girls are separated

example BG GB GB B GBB BB
Exercise A class contains 8 boys and 4 girls We line them
up randomly
in a straight line What is the probability that exactly 2 girls
come together and that the 2 other girls are separated

example BG GB GB B GBB BB
Solution
In this sort of problems it is best to think about the situation as an
allocation problem The 8 boys are considered to be separators for 9
boxes The
girls constitute 4 indistinguishable balls The event we
want to study is equivalent to one box containing exactly 2 balls
2 other boxes containing exactly 1 ball each the others remaining empty

4 49 1
Total Cases Irl Y
Favourable Cases Stl Choose box balls
containing 2 I 9
St 2 Choose 2 others with I balleach

IP event 9 E