Sample Spaces, Events, Probability

Foundations of Data Analysis

January 25, 2022

Brain Teaser

You have cards numbered 1 through 10. If you pick five

cards at random, what is the probability that you selected
the cards 1, 2, 3, 4, 5 (not necessarily in that order)?
Sets

Definition
A set is a collection of unique objects.

Here “objects” can be concrete things (people in class,

schools in ACC), or abstract things (numbers, colors).

Examples:

A = {3, 8, 31}
B = {apple, pear, orange, grape}
Not a valid set definition: C = {1, 2, 3, 4, 2}
Sets
I Order in a set does not matter!
{1, 2, 3} = {3, 1, 2} = {1, 3, 2}
I When x is an element of A, we denote this by:

x ∈ A.

I If x is not in a set A, we denote this as:

x∈
/ A.

I The “empty” or “null” set has no elements:

∅ = {}
Sample Spaces

Definition
A sample space is the set of all possible outcomes of an
experiment. We’ll denote a sample space as Ω.

Examples:
I Coin flip: Ω = {H, T}
I Roll a 6-sided die: Ω = {1, 2, 3, 4, 5, 6}
I Pick a ball from a bucket of red/black balls:
Ω = {R, B}
Some Important Sets
I Integers:

Z = {. . . , −3, −2, −1, 0, 1, 2, 3, . . .}

I Natural Numbers:

N = {0, 1, 2, 3, . . .}

I Real Numbers:

R = “any number that can be written in decimal form”

5 ∈ R, 17.42 ∈ R, π = 3.14159 . . . ∈ R
Building Sets Using Conditionals

I Alternate way to define natural numbers:

N = {x ∈ Z : x ≥ 0}

I Set of even integers:

{x ∈ Z : x is divisible by 2}

I Rationals:

Q = { p/q : p, q ∈ Z, q 6= 0}
Subsets

Definition
A set A is a subset of another set B if every element of
A is also an element of B, and we denote this as A ⊆ B.

Examples:
I {1, 9} ⊆ {1, 3, 9, 11}
I Q⊆R
I {apple, pear} * {apple, orange, banana}
I ∅ ⊆ A for any set A
Events

Definition
An event is a subset of a sample space.

Examples:
I You roll a die and get an even number:
{2, 4, 6} ⊆ {1, 2, 3, 4, 5, 6}
I You flip a coin and it comes up “heads”:
{H} ⊆ {H, T}
I Your code takes longer than 5 seconds to run:
(5, ∞) ⊆ R
Set Operations: Union

Definition
The union of two sets A and B, denoted A ∪ B is the set
of all elements in either A or B (or both).

When A and B are events, A ∪ B means that event A or

event B happens (or both).

Example:
A = {1, 3, 5} “an odd roll”
B = {1, 2, 3} “a roll of 3 or less”
A ∪ B = {1, 2, 3, 5}
Venn Diagram: Union

A B

Ω
Venn Diagram: Union

A∪B
Ω
Set Operations: Intersection
Definition
The intersection of two sets A and B, denoted A ∩ B is
the set of all elements in both A and B.

When A and B are events, A ∩ B means that both event

A and event B happen.
Example:
A = {1, 3, 5} “an odd roll”
B = {1, 2, 3} “a roll of 3 or less”
A ∩ B = {1, 3}
Note: If A ∩ B = ∅, we say A and B are disjoint.
Venn Diagram: Intersection

A B

Ω
Venn Diagram: Intersection

A∩B
Ω
Set Operations: Complement

Definition
The complement of a set A ⊆ Ω, denoted Ac , is the set
of all elements in Ω that are not in A.

When A is an event, Ac means that the event A does not

happen.

Example:
A = {1, 3, 5} “an odd roll”
Ac = {2, 4, 6} “an even roll”
Venn Diagram: Complement

Ω
Venn Diagram: Complement

AC
Ω
Set Operations: Difference

Definition
The difference of a set A ⊆ Ω and a set B ⊆ Ω,
denoted A − B, is the set of all elements in Ω that are in
A and are not in B.
Example:
A = {3, 4, 5, 6}
B = {3, 5}
A − B = {4, 6}
Note: A − B = A ∩ Bc
Venn Diagram: Difference

A B

Ω
Venn Diagram: Difference

A-B

Ω
De Morgan’s Laws

Complement of union or intersection:

(A ∪ B)c = Ac ∩ Bc

(A ∩ B)c = Ac ∪ Bc

What is the English translation for both sides of the

equations above?
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

A B A B

Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

A B

A∪B
Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

A B

(A∪B)C
Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

AC
(A∪B)C
Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

(A∪B)C BC
Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

(A∪B)C AC BC
Ω Ω
Venn Diagram: De Morgan

(A ∪ B)c = Ac ∩ Bc

(A∪B)C AC ∩ B C
Ω Ω
Exercises

Check whether the following statements are true or false.

(Hint: you might use Venn diagrams.)
I A−B⊆A
I (A − B)c = Ac ∪ B
I A∪B⊆B
I (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
Probability

Definition
A probability function on a finite sample space Ω
assigns every event A ⊆ Ω a number in [0, 1], such that
1. P(Ω) = 1
2. P(A ∪ B) = P(A) + P(B) when A ∩ B = ∅
P(A) is the probability that event A occurs.
Equally Likely Outcomes

The number of elements in a set A is denoted |A|.

If Ω has a finite number of elements, and each is equally

likely, then the probability function is given by

|A|
P(A) =
|Ω|
Example: Rolling a 6-sided die
I P({1}) = 1/6
I P({1, 2, 3}) = 1/2
Repeated Experiments

If we do two runs of an experiment with sample space Ω,

then we get a new experiment with sample space

Ω × Ω = {(x, y) : x ∈ Ω, y ∈ Ω}

The element (x, y) ∈ Ω × Ω is called an ordered pair.

Properties:
Order matters: (1, 2) 6= (2, 1)
Repeats are possible: (1, 1) ∈ N × N
More Repeats

Repeating an experiment n times gives the sample

space

Ωn = Ω × · · · × Ω (n times)
= {(x1 , x2 , . . . , xn ) : xi ∈ Ω for all i}

The element (x1 , x2 , . . . , xn ) is called an n-tuple.

If |Ω| = k, then |Ωn | = kn .

Probability Rules

Complement of an event A:

P(Ac ) = 1 − P(A)

Union of two overlapping events A ∩ B 6= ∅:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Exercise

You are picking a number out of a hat, which contains

the numbers 1 through 100. What are the following
events and their probabilities?
I The number has a single digit
I The number has two digits
I The number is a multiple of 4
I The number is not a multiple of 4
I The sum of the number’s digits is 5
Permutations

A permutation is an ordering of an n-tuple. For instance,

the n-tuple (1, 2, 3) has the following permutations:

(1, 2, 3), (1, 3, 2), (2, 1, 3)

(2, 3, 1), (3, 1, 2), (3, 2, 1)

The number of unique orderings of an n-tuple is

n factorial:

n! = n × (n − 1) × (n − 2) × · · · × 2

How many ways can you rearrange (1, 2, 3, 4)?

Binomial Coefficient or “n choose k”

n


The binomial coefficient, written as k and spoken as
“n choose k”, is the number of ways you can select k
items out of a list of n choices.

Formula:  
n n!
=
k k!(n − k)!
Binomial Coefficient or “n choose k”

Example: You have cards numbered 1 through 10. If

you pick five cards at random, what is the probability that
you selected the cards 1, 2, 3, 4, 5 (not necessarily in
that order)?
Answer:
|A|
We’ll use the formula P(A) = |Ω| .

There is only one combination that gives us cards

1,2,3,4,5, so |A| = 1.

The total number of possible 5 card selections is

 
10 10!
|Ω| = = = 252
5 5!(10 − 5)!
So, finally the probability is

|A| 1
P(A) = = ≈ 0.00397 = 0.397%
|Ω| 252