Probability

Recall our example from last week:

Example. If the experiment is to throw a standard die and record the outcome then:

sample space S = {1, 2, 3, 4, 5, 6}

Q. What is the probability P (1) of rolling a 1?

A. 1/6.

Q. What is the following sum?

P (1) + P (2) + P (3) + P (4) + P (5) + P (6)

which we can denote using summation notation:
6
X
P (i)
i=1

A. 1. Why? this should make sense because it means that one of the events in the sample space
must occur. The probability of no event occurring is 0.

Let’s make our understanding of discrete probability a little more formal with some definitions.
Definition A probability function P assigns to each outcome x in a sample space S a number P (x)
so that:
0 ≤ P (x) ≤ 1 ∀x ∈ S
and X
P (x) = 1
x∈S

This then tells us that if we know, P (E) the probability of an event E occurring, then we can easily
determine the probability of the event not occurring. We indicate that the event that E does not
occur as E.

Theorem. Let E be an event. The probability of E, the complement of E satisfies:

P (E) + P (E) = 1

Example.
The Birthday Problem. What is the probability that in a group of n people at least two have the
same birthday?

1
Solution. Let’s represent the days of the year by the integers 1, 2, . . . , 365 and use E to represent
the event that at least two people have the same birthday.

Q. What could we define our sample space S to be?

A. We need for each person, one of the days in the year, so n - tuples of the integers 1-365. For
example, if n = 10 then (1, 1, 4, 14, 98, 11, 2, 204, 363, 101) might be one element in the sample
space.

Q. Let’s assume all combinations or birthdays are equally likely. How many different ways are
there for the n birthdays to fall?

A. Each position in the n-tuple has 365 choices (think permutation with repetition) or 365 n .

Is it easy to count the number of tuples with 2 or more values the same? Think about this carefully.
How many tuples have 2 or more elements the same....if n is small, we can do it fairly easily but if
n is large, it becomes difficult.

Q. How can we rephrase the problem to make it easier?

A. Consider 1 − P (E). E is the complement of E - ie., the event that everyone has a distinct
birthday.

Q. What if n > 365?

A. Then P (E ) = 1 as there are more people than possible birthdays so there have to be two people
with the same birthday. This follows from the Pigeon Hole Principle. We will see the Pigeon Hole
Principle soon!

Q. Suppose n ≤ 365. What is P (E)?

A. There are two ways to count the number of ways to get distinct birthdays (order should matter
because each person is distinct):

1. Select n days from 365 and then have n ways to permute them. So C(365, n) · n!.

2. Consider 365 options for first person, 364 for second etc. to 365 − n + 1 for last person,
which the number n permutations of a 365 element set or 365 · 364 · · · (365 − n + 2 ) · (365 − n + 1 ).

C (365 ,n)·n!
Therefore P (E ) = 1 − 365 n

2
Example. Consider tossing a coin five times. What is the probability of getting the same result on
the first two tosses or the last two tosses?

Solution.
Let E be the event that the first two tosses are the same and F be the event that the last two
tosses are the same. Let n(E) be the number of ways event E can occur. Our sample space is
S = {HHHHH , HHHHT , HHHTH , HHTHH , HTHHH , THHHH , HHHTT , etc., ..., TTTTT }

|E | = The first toss can be H or T and the second toss must be the same so choices are:
= 2 · 1 · 2 · 2 · 2 = 24 = 16.
|F | = The second last toss can be H or T and the last toss must be the same so choices are
= 2 · 2 · 2 · 2 · 1 = 24 = 16.
|S | = Size of the sample space is then 25 = 32

To compute the probability of getting the same result on the first two tosses or the last two tosses,
one might do the following:

|E| 16 1
P (E) = = =
|S| 32 2
|F | 16 1
P (F ) = = =
|S| 32 2
Therefore
1 1
P (E or F ) = P (E) + P (F ) = + =1
2 2
Sanity Check Does this make sense??

Q. How did we go wrong? Stop and figure this out before continuing...

3
Q. Are there any situations where an event belongs to both E and F ? Another way to think of this,
is there any overlap between E and F ? Is their intersection empty? I.e., is E ∩ F = ∅?

A. E ∩ F 6= ∅.

In fact, E ∩ F = all those tosses where both the first two are the same and the last two are the
same. This means we have over counted the overlap!.

Let’s figure out the probability that we have both the first two tosses and the last two tosses the
same.

|(E and F )| = |E ∩ F |
= We have two choices for the first pair, two choices for the middle toss,
two choices for the last pair
2 · 2 · 2 = 23
= 8.

8
So P (E and F )| = 32
.

Now we can calculate the probability P (E or F ):

P (E or F ) = P (E) + P (F ) − Overlap in E and F .

= P (E) + P (F ) − P (E and F )
16 16 8
= + −
32 32 32
24 3
= =
32 4

Theorem (The Sum Rule) If E and F are events in an experiment then the probability
that E or F occurs is given by:

P (E or F ) = P (E) + P (F ) − P (E and F )

Example Suppose we are roll two fair dice. What is the probability of getting doubles or a sum of
6?

4
The sample space will be of size 36 (all pairs of numbers from 1 to 6).

Let E1 denote the event we roll doubles and E2 denote the event get a sum of 6.

What is P (E1 )? There are 6 ways to roll doubles so

6
P (E1 ) =
36
What is P (E2 )? We can have a sum of 6 by rolling (1, 5), (2, 4), (3, 3), (4, 2), (5, 1):
5
P (E2 ) =
36
Then the number of ways we can have both E1 and E2 is (3, 3). Therefore,
1
P (E1 ∩ E2 ) =
36
What then is the final probability for P (E1 or E2 )?

P (E1 or E2 ) = P (E1 ) + P (E2 ) − P (E1 ∩ E2 )

6 5 1
= + −
36 36 36
10 5
= =
36 18