Lecture 3:

Probability

Alexey Rubtsov

Ryerson University

Alexey Rubtsov (Ryerson University) Probability and Statistics I 1 / 27

 Lecture content

1 Sample Space, Events, and Probabilities

2 Counting Rules
3 Sets Theory Concepts
4 Probability
5 Conditional Probability

Alexey Rubtsov (Ryerson University) Probability and Statistics I 2 / 27

 1. Sample Space, Events, and Probabilities

Definition
An experiment is the process which leads to well-defined results
called outcomes (measurements).

Definition
The sample space S is the set of all possible outcomes of a
probability experiment.

Discrete sample space: it has finitely many or countably many

elements.
Continuous sample space: the elements constitute a continuum
(points on a line segment).

Alexey Rubtsov (Ryerson University) Probability and Statistics I 3 / 27

 Definition
A simple event is the outcome that is observed on a single repetition
of the experiment. An event A is a collection of simple events (a
subset of the sample space).

Example
Rolling a die.
Sample space: S = {1, 2, 3, 4, 5, 6}.
Event: A = {1, 2, 3}
Time to failure of a laptop.
Sample space: S = {x | x ∈ [0, ∞)} in years.
Event: A = {x | x ∈ [2, 10]} in years.

Definition
Two events are mutually exclusive if, when one event occurs, the
other cannot, and vice versa.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 4 / 27

 Frequency Interpretation of Probability: The probability of an event is
the proportion of times the event occurs in an infinite series of repeated
experiments.

Suppose that an experiment is performed n times. The relative

frequency for an event A is:
number of times A occurs f
=
n n
The probability of A would be:
f
P (A) = lim
n→∞ n

Shortcut: If in the sample space, S, there are m equally likely

possibilities and the event A contains k then:
k
P (A) = .
m

Alexey Rubtsov (Ryerson University) Probability and Statistics I 5 / 27

 Requirement for simple-event probabilities:
Each probability must lie between 0 and 1.
The sum of the probabilities for all simple events in S equals 1.

Definition
The probability of an event A is equal to the sum of the
probabilities of the simple events contained in A.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 6 / 27

 Example (Rolling a die)
Sample space: S = {1, 2, 3, 4, 5, 6}.
Event: A = {1, 2, 3} .
Number of times experiment is performed, n = 3
Results: 1, 4, 2
Relative Frequency: 32
True probability: P (A) = 12 .

Note that, in order to compute the probability of an event, we have to:

1 Learn Sets Theory in order to operate with different sets of
elements.
2 Learn Counting Techniques in order to compute ”the size of the
sample space” and ”the size of events”.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 7 / 27

Example 4.8
A candy dish contains one yellow and two red candies. You
close your eyes, choose two candies one at a time from the
dish, and record their colors. What is the probability that
both candies are red?

1
Example 4.8 – Solution (1 of 3)
Since no probabilities are given, you must list the simple
events in the sample space. The two-stage selection of the
candies suggests a tree diagram.

Figure 4.4

2
Example 4.8 – Solution (2 of 3)
There are two red candies in the dish, so you can use the
letters R1, R2, and Y to indicate that you have selected the
first red, the second red, or the yellow candy, respectively.

Since you closed your eyes when you chose the candies, all
six choices should be equally likely and are assigned
probability 1 ∕ 6.

If A is the event that both candies are red, then

A = {R1R2, R2R1}

3
Example 4.8 – Solution (3 of 3)
Thus,
P(A) = P(R1R2) + P(R2R1)

4
Calculating Probabilities Using Simple Events (5 of 5)

How to Calculate the Probability of an Event

1. List all the simple events in the sample space.
2. Assign an appropriate probability to each simple event.
3. Determine which simple events result in the event of
interest.
4. Sum the probabilities of the simple events that result in
the event of interest.

5
 2. Counting Rules

1 Multiplication: If sets A1 , ..., Ak contain, respectively, n1 , ..., nk

elements, then there are n1 · ... · nk ways of choosing first an
element of A1 , then an element of A2 ,... and finally an element of
Ak .
2 Permutations: The number of permutations of r objects selected
from a set of n distinct objects is:
n!
Prn =
(n − r)!
3 Combinations: The number of ways in which r objects can be
selected from a set of n distinct objects is:
n!
Crn =
r! · (n − r)!

Alexey Rubtsov (Ryerson University) Probability and Statistics I 8 / 27

 Example (Rolling a die)
Sample space: S = {1, 2, 3, 4, 5, 6}.
Events: A = {1, 2, 3, 4}, B = {1, 5}
Multiplication: There are 4 · 2 ways of choosing first an element of A,
then an element of B:
{1, 1}; {1, 5}; {2, 1}; {2, 5}; {3, 1}; {3, 5}, {4, 1}; {4, 5}.
Permutation: The number of permutations of 3 objects selected from
4!
A is (order is important): 1! = 24 :
{1, 2, 3}; {1, 3, 2}; {2, 1, 3}; {3, 1, 2}; {2, 3, 1}; {3, 2, 1}.
{1, 2, 4}; {1, 4, 2}; {2, 1, 4}; {4, 1, 2}; {2, 4, 1}; {4, 2, 1}.
{1, 3, 4}; {1, 4, 3}; {3, 1, 4}; {4, 1, 3}; {3, 4, 1}; {4, 3, 1}.
{2, 3, 4}; {2, 4, 3}; {3, 2, 4}; {4, 2, 3}; {2, 4, 2}; {4, 3, 2}.
Combinations: The number of ways in which 3 objects can be selected
4!
from A is (the order is not important): 3!·1! =4:
{1, 2, 3}; {1, 2, 4}, {1, 3, 4}; {2, 3, 4}.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 9 / 27

 Example (Multiplication)
A manufacturer is experiencing difficulty getting consistent readings of
wireless signals among three iPhones, there are also four possible
technicians who operate at least one of the iPhones regularly. How
many iPhones-technicians pairs must be included in a designed
experiment where every technician tries every iphone.

A1 = T echnician = {M1 , M2 , M3 , M4 }
A2 = iP hones = {A, B, C}
n1 = 4, n2 = 3

Answer : 4 · 3 = 12.
Find probability that technician M2 will be chosen for a quick testing
of signals.
Find probability that either technician M2 or technician M1 are chosen.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 10 / 27

 Example (Permutations)
A computer is made of five main components and can be assembled in
any order. A quality control technician wants to test each order for
efficiency. How many orders are there?
(The order of the choice is important!)

5!
P55 = = 120
(0)!
What is the probability that component ”1” is chosen first for a
randomly selected assemble?

Alexey Rubtsov (Ryerson University) Probability and Statistics I 11 / 27

 Example (Combinations)
Three members of a 5-person committee must be chosen to form a
subcommittee. How many different subcommittees could be formed?
(The order of the choice is not important!)

5!
C35 = = 10.
3! · (5 − 3)!
What is the probability that person A is part of the subcommittee?
Find probability that either person A or B are part of the
subcommittee.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 12 / 27

Example 4.10
A candy dish contains one yellow and two red candies. Two
candies are selected one at a time from the dish, and their
colors are recorded. How many simple events are in the
sample space S?

1
Example 4.10 – Solution
The first candy can be chosen in m = 3 ways.

Since one candy is now gone, the second candy can be

chosen in n = 2 ways.

The total number of simple events is mn = (3)(2) = 6

2
Example 4.12
A truck driver can take three routes from city A to city B, four
from city B to city C, and three from city C to city D. When
traveling from A to D, if the driver must drive from A to B to
C to D, how many possible A-to-D routes are available?

Solution:
Let
n1 = Number of routes from A to B = 3
n2 = Number of routes from B to C = 4
n3 = Number of routes from C to D = 3

3
Example 4.12 – Solution
Then the total number of ways to construct a complete
route, taking one subroute from each of the three groups,
(A to B), (B to C), and (C to D), is
n1n2n3 = (3)(4)(3) = 36

4
Example 4.13
Three lottery tickets are drawn from a total of 50. If the
tickets will be distributed to each of three employees in the
order in which they are drawn, the order will be important.

How many simple events are associated with the

experiment?

5
Example 4.13 – Solution
The total number of simple events is

= 50(49)(48)

= 117,600

6
Example 4.16
Five manufacturers produce a certain electronic device,
whose quality varies from manufacturer to manufacturer. If
you were to select three manufacturers at random, what is
the chance that the selection would contain exactly two of
the best three?

7
Example 4.16 – Solution (1 of 3)
The experiment consists of randomly selecting three
manufacturers from a group of five, three of which are
designated as “best” and two as “not best.” The event of
interest is
A: select exactly two of the “best” three manufacturers

You can think of a bowl

containing the names of the
manufacturers, from which
you select three, as shown in
figure.
Figure 4.6

8
Example 4.16 – Solution (2 of 3)
To find P(A), we need to calculate

The number of ways to select three manufacturers from a

group of five is

To find nA, notice that A will occur only when you select two
of the “best” three and one of the “not best”—a two-step
process.

9
Example 4.16 – Solution (3 of 3)
There are

ways to select two of the “best” three and

ways to select one of the two “not best.”

Applying the mn Rule, we find there are nA = (3)(2) = 6 of

the 10 simple events in event A and P(A) = nA ∕ N = 6 ∕ 10.

10
 Example
Consider a box containing 2 red lollipops and 48 yellow lollipops, and
suppose that 3 lollipops are chosen at random without replacement.
a) What is the probability that none of the lollipops chosen are red?
b) What is the probability that one of them is red?
c) What is the probability that at most one is red?
d) What is the probability that three are red?

Alexey Rubtsov (Ryerson University) Probability and Statistics I 13 / 27

 Example
There are 5 stops left on a subway line and 3 passengers on a train.
Assume everyone is equally likely to get off at any stop. What is the
probability that all 3 passengers get off at different stops?

Alexey Rubtsov (Ryerson University) Probability and Statistics I 14 / 27

 3. Sets Theory Concepts
Let A and B be two events then:
A ∩ B denotes intersection {x ∈ A ∩ B if and only if x ∈ A and
x ∈ B}.
A ∪ B denotes union {x ∈ A ∪ B if and only if x ∈ A or x ∈ B}.
A denotes complement {x ∈ A if and only if x ∈
/ A}.
A and B are mutually exclusive events if they have no elements
in common.

Example (Rolling a die)

Sample space: S = {1, 2, 3, 4, 5, 6}.
Events: A = {1, 2, 3}, B = {1, 5}
A ∩ B = {1}.
A ∪ B = {1, 2, 3, 5}
A = {4, 5, 6}
A and B are not mutually exclusive.

Alexey Rubtsov (Ryerson University) Probability and Statistics I 15 / 27

Sets theory
The following figures show Venn diagrams for A ∪ B, A ∩ B,
and respectively.

Figure 4.7 Figure 4.8 Figure 4.9

1
Sets theory

You can see from the Venn diagram in figure that A and
are mutually exclusive and that the entire
sample space.

The complement of an event

Figure 4.9

2
Sets theory

When two events A and B are mutually exclusive or

disjoint, it means that when A occurs, B cannot, and vice
versa. The Venn diagram in figure shows two such events
with no simple events in common.

Two disjoint or mutually exclusive events

Figure 4.11

3
 4. Probability

A probability P is defined on the sample space S by assigning to each

event A, a number P (A) with the following properties:

Axioms of Probability for Finite Sample Space

1 0 ≤ P (A) ≤ 1 for each event A in S.
2 P (S) = 1.
3 If A and B are mutually exclusive events in S, then
P (A ∪ B) = P (A) + P (B).

Alexey Rubtsov (Ryerson University) Probability and Statistics I 16 / 27

 Elementary Theorems

Generalization of the third axiom (for discrete and continuous

sample space): If A1 , ..., An are mutually exclusive events in a
sample space S, then

P (A1 ∪ ... ∪ An ) = P (A1 ) + ... + P (An )

Let us also assume that if A1 , ... (infinite sequence) are mutually

exclusive events then:
∞
S ∞
P
P ( Ai ) = P (Ai ).
i=1 i=1
Addition Rule: If A and B are any events in S, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Complement Rule: If A is any event in S, then P (A) = 1 − P (A).

Alexey Rubtsov (Ryerson University) Probability and Statistics I 17 / 27

Example 4.17
Two fair coins are tossed, and the outcome is recorded.
These are the events of interest:

A: Observe at least one head

B: Observe at least one tail

Define the events A, B, A ∩ B, A ∪ B, and as collections

of simple events, and find their probabilities.

1
Example 4.17 – Solution (1 of 3)
We know that the simple events for this experiment are

E1 : HH (head on first coin, head on second)

E2 : HT
E3 : TH
E4 : TT

and that each simple event has probability 1 ∕ 4.

2
Example 4.17 – Solution (2 of 3)
Event A, at least one head, occurs if E1, E2, or E3 occurs, so
that

and

3
Example 4.17 – Solution (3 of 3)
Similarly,

Note that (A ∪ B) = S, the sample space, and is thus certain

to occur.

4
Calculating Probabilities for Unions and Complements

When two events A and B are mutually exclusive or

disjoint, it means that when A occurs, B cannot, and vice
versa. This means that the probability that they both occur,
P(A ∩ B), must be zero. The Venn diagram in figure shows
two such events with no simple events in common.

Two disjoint or mutually exclusive events

Figure 4.11

5
Calculating Probabilities for Unions and Complements

When two events A and B are mutually exclusive, then

P(A ∩ B) = 0 and the Addition Rule simplifies to

P(A ∪ B) = P(A) + P(B)

A second rule deals with complements of events.

6
Calculating Probabilities for Unions and Complements

You can see from the Venn diagram in figure that A and
are mutually exclusive and that the entire
sample space.

The complement of an event

Figure 4.9

7
Calculating Probabilities for Unions and Complements

It follows that

Rule for Complements

8
Example 4.19 (1 of 3)
In a survey of 1000 adults, respondents were asked about
the cost of a college education.

The respondents were classified according to whether they

currently had a child in college and whether they thought the
loan burden for most college students is too high, the right
amount, or too little.

9
Example 4.19 (2 of 3)
The proportions responding in each category are shown in
the probability table. Suppose one respondent is chosen at
random from this group.

Table 4.6

10
Example 4.19 (3 of 3)
1. What is the probability that the respondent has a child in
college?
2. What is the probability that the respondent does not
have a child in college?
3. What is the probability that the respondent has a child in
college or thinks that the loan burden is too high or both?

11
Example 4.19 – Solution (1 of 3)
Table 4.6 gives the probabilities for the six simple events in
the table. For example, the entry in the top left corner of the
table is the probability that a respondent has a child in
college and thinks the loan burden is too high (A ∩ D).
1. The event that a respondent has a child in college will
occur regardless of his or her response to the question
about loan burden. That is, event D consists of the
simple events in the first row:
P(D) = .35 + .08 + .01 = .44

12
Example 4.19 – Solution (2 of 3)
In general, the probabilities of marginal events such as D
and A are found by summing the probabilities in the
appropriate row or column.
2. The event that the respondent does not have a child in
college is the complement of the event D denoted by
The probability of is found as

13
Example 4.19 – Solution (3 of 3)
3. The event of interest is P(A ∪ D). Using the Addition
Rule
P(A ∪ D) = P(A) + P(D) − P(A ∩ D)
= .60 + .44 − .35
= .69

14
 5. Conditional Probability

Definition
If A and B are any events in S and P (B) 6= 0, the conditional
probability of A given B is:

P (A ∩ B)
P (A | B) = .
P (B)

Conditional probability of A given B ⇔ Probability of A given the

occurrence of other event B.

Definition
Two events A and B are independent events if P (A | B) = P (A).

The occurrence of B does not change the probability of A!

Alexey Rubtsov (Ryerson University) Probability and Statistics I 18 / 27
 Multiplication Rule: If A and B are any events in S then

P (A) · P (B | A) if P (A) 6= 0
P (A ∩ B) =
P (B) · P (A | B) if P (B) 6= 0

Two events A and B are independent if and only if

P (A ∩ B) = P (A)P (B)

Rule of elimination (Law of total probability): If B1 , ..., Bn are

n
S
mutually exclusive events of which one must occur (S = Bi ),
i=1
then
n
X
P (A) = P (Bi ) · P (A | Bi )
i=1

Alexey Rubtsov (Ryerson University) Probability and Statistics I 19 / 27

 Theorem (Bayes)
If B1 , ..., Bn are mutually exclusive events of which one must occur
n
S
(S = Bi ), then
i=1

P (Br )P (A | Br )
P (Br | A) = n
P
P (Bi ) · P (A | Bi )
i=1

Alexey Rubtsov (Ryerson University) Probability and Statistics I 20 / 27

 Example
Let the following table represent market conditions on 100 companies
(Commodities C, Technology T and Services S) classify into W :
performance above 2% and B: performance below 2%:
C T S
W 10 10 15
B 20 35 10
a) Find the probability that a randomly selected commodity company
is classified as W .
b) Find the probability that a randomly selected company is in
commodities and it is classified as W .
c) Find P ((W ∩ C) ∪ (S ∩ B)) and P (W ).

Alexey Rubtsov (Ryerson University) Probability and Statistics I 21 / 27

 Example
A store stocks headphones from three suppliers. Suppliers A, B, and C supply 10%,
20%, and 70% of the products respectively. It has been determined that company
A’s products are 1% defective while company B’s are 3% defective and company C’s
are 4% defective. Denote defective as D.
a) What is the probability that a randomly selected product is defective?

P (D) = P (D | A)P (A) + P (D | B)P (B) + P (D | C)P (C)

= 0.01(0.1) + 0.03(0.2) + 0.04(0.7)

b) If a headphone is selected at random and found to be defective, what is the

probability that it came from supplier B?
P (B)P (D | B)
P (B | D) =
P (A)P (D | A) + P (B)P (D | B) + P (C)P (D | C)
0.2(0.03)
= ≈ 0.1714.
0.1(0.01) + 0.2(0.03) + 0.7(0.04)

Alexey Rubtsov (Ryerson University) Probability and Statistics I 22 / 27

Conditional Probability

Colorblindness
A researcher notes a person’s gender and whether or not
the person is colorblind to red and green. Does the
probability that a person is colorblind change, depending on
whether the person is male or not?

1
Conditional Probability

Define two events:

A: Person is a male
B: Person is colorblind
In this case, because colorblindness is a male sex-linked
characteristic, the probability that a man is colorblind will be
greater than the probability that a person chosen from the
general population will be colorblind.
The probability of event B, that a person is colorblind,
depends on whether or not event A, that the person is a
male, has occurred. We say that A and B are dependent
events.

2
Conditional Probability

Tossing Dice
On the other hand, consider tossing a single die two times,
and define two events:
A: Observe a 2 on the first toss
B: Observe a 2 on the second toss
If the die is fair, the probability of event A is P(A) = 1 ∕ 6.

Consider the probability of event B. Regardless of whether

event A has or has not occurred, the probability of observing
a 2 on the second toss is still 1 ∕ 6.

3
Conditional Probability

We could write:
P(B given that A occurred) = 1 ∕ 6
P(B given that A did not occur) = 1 ∕ 6
Since the probability of event B is not changed by the
occurrence of event A, we say that A and B are independent
events.

The probability of an event A, given that the event B has

occurred, is called the conditional probability of A, given
that B has occurred, and written as

4
Example 4.20
In a color preference experiment, eight toys are placed in a
container. The toys are identical except for color—two are
red, and six are green. A child is asked to choose two toys
at random. What is the probability that the child chooses the
two red toys?

5
Example 4.20 – Solution (1 of 3)
Use a tree diagram as shown
in figure and define the
following events:
R: Red toy is chosen
G: Green toy is chosen

Figure 4.12

6
Example 4.20 – Solution (2 of 3)
The event A (both toys are red) can be written as the
intersection of two events:
A = (R on first choice) ∩ (R on second choice)
Since there are only two red toys in the container, the
probability of choosing red on the first choice is 2 ∕ 8.
However, once this red toy has been chosen, the probability
of red on the second choice is dependent on the outcome of
the first choice.

7
Example 4.20 – Solution (3 of 3)
If the first choice was red, the probability of choosing a
second red toy is only 1 ∕ 7 because there is only one red toy
among the seven remaining. Using this information and the
Multiplication Rule, you can find the probability of event A.
P(A) = P(R on first choice ∩ R on second choice)

8
Conditional Probabilities

Colorblindness, continued
Suppose that in the general population, there are 51% men
and 49% women, and that the proportions of colorblind men
and women are shown in the following probability table:

9
Conditional Probabilities

If a person is drawn at random from this population and is

found to be a man (event B), what is the probability that the
man is colorblind (event A)? If we know that the event B has
occurred, we must restrict our focus to only the 51% of the
population that is male.

The probability of being colorblind, given that the person is

male, is 4% of the 51%, or

10
Conditional Probabilities

What is the probability of being colorblind, given that the

person is female? Now we are restricted to only the 49% of
the population that is female, and

Notice that the probability of event A changed, depending

on whether event B occured. This indicates that these two
events are dependent.

11
Example

Coin Tosses at Football Games

A football team is involved in two overtime periods during a
given game, so that there are three coin tosses. If the coin is
fair, what is the probability that they lose all three tosses?

Solution
If the coin is fair, the event can be described in three steps:
A: lose the first toss
B: lose the second toss
C: lose the third toss

12
Example

Since the tosses are independent, and because P(win) =

P(lose) = .5 for any of the three tosses,

P(A ∩ B ∩ C) = P(A)P(B)P(C) = (.5)(.5)(.5) = .125

13
Example 4.21
Toss two coins and observe the outcome. Define these
events:
A: Head on the first coin
B: Tail on the second coin
Are events A and B independent?
Solution:
From previous examples, you know that
S = {HH, HT, TH, TT}. Use these four simple events to find

14
Example 4.21 – Solution

Since we have

P (A) P(B) = P(A ∩ B) and the two events must be

independent.

15
The Difference between Mutually Exclusive and
Independent Events

Many students find it hard to tell the difference between

mutually exclusive and independent events.
• When two events are mutually exclusive or disjoint, they
cannot both happen together when the experiment is
performed. Once the event B has occurred, event A
cannot occur, so that

or vice versa. The

occurrence of event B certainly affects the probability that
event A can occur.

16
The Difference between Mutually Exclusive and
Independent Events

• Therefore, mutually exclusive events must be dependent.

• When two events are mutually exclusive or disjoint,
P(A ∩ B) = 0 and P(A ∪ B) = P(A) + P(B).
• When two events are independent,
P(A ∩ B) = P(A)P(B), and
P(A ∪ B) = P(A) + P(B) − P(A)P(B).

17
Example 4.23
Two cards are drawn from a deck of 52 cards. Calculate the
probability that the draw includes an ace and a ten.

Solution:
Consider the event of interest:
A: Draw an ace and a ten
Then A = B ∪ C, where
B: Draw the ace on the first draw and the ten on the second
C: Draw the ten on the first draw and the ace on the second

18
Example 4.23 – Solution (1 of 2)
Events B and C were chosen to be mutually exclusive and
also to be intersections of events with known probabilities;
that is,
B = B1 ∩ B2 and C = C1 ∩ C2
where
B1 : Draw an ace on the first draw
B2 : Draw a ten on the second draw
C1 : Draw a ten on the first draw
C2 : Draw an ace on the second draw

19
Example 4.23 – Solution (2 of 2)
Applying the Multiplication Rule, you get

and

Then, applying the Addition Rule,

20