School of Mathematics and Statistics

UNSW Sydney

Introduction to Probability and Stochastic Processes

OPEN LEARNING
CHAPTER 1

2 / 77
Outline:

1.1 Introduction
1.2 Experiments, Sample Space and Events
1.3 Axioms and Basic Results
1.4 Counting Rules
1.5 Conditional Probability
1.6 Independent Events
1.7 Additional Probability Laws
1.8 Bayes Rule Formula
1.9 Supplementary Material

3 / 77
1.1 Introduction

4 / 77
Statistics is concerned with making decisions based on
data, where the data has uncertainty.

In the science of statistics, we are developing and studying

methods for collecting, analysing, interpreting and
presenting data.

In order to deal with uncertainty, we need to develop a

language for discussing it, namely probability theory.

We will derive in this chapter some fundamental results

from probability theory that will be useful later on.

5 / 77
1.2 Experiments, Sample Space and
Events

6 / 77
Definition
An experiment is any process leading to recorded
observation.

Example
Examples of experiments include
coin tossing, blood pressure measurements, the
progress of an economic indicator, shopping
behaviour at a grocery store, etc

7 / 77
Definition
An outcome is a possible result of an experiment.

The set S of possible outcomes is the sample space of an

experiment.

The sample space S is discrete if it contains a countable

(finite or countably infinite) number of outcomes.

Note that Ω also denotes the sample space S in some textbooks of

probability and statistics.

8 / 77
Definition
An event is a set of outcomes (a subset of S).

An event occurs if the result of the experiment is one of the

outcomes of that event.

9 / 77
Definition
Events are mutually exclusive (or disjoint) if they have
no outcomes in common. That is if they cannot both occur.

If A and B are mutually exclusive events, we say that

A ∩ B = ∅, where ∅ denotes the empty set and ∩ represents
the intersection.

Definition
If A is an event, we denote Ā as the complement of A.
That is, Ā is the event that A does not occur.

Note that, Ac is also used to denote the complement of A.

10 / 77
Example
Consider the experiment of tossing a coin three times and
recording the results.

Let H denotes HEADS and T TAILS.

The sample space S for this experiment is

S = {HHH, HHT, HT H, T HH, T T H, T HT, HT T, T T T }.

Note that HHH represents getting three heads after

tossing the coin three times.

Observe that the sample S is discrete since we have a

countable number of events.

11 / 77
Example (Continued)
Let A be the event at least one head. That is,

A = {HHH, HHT, HT H, T HH, T T H, T HT, HT T }.

Let B be the event exactly two heads. That is,

B = {HHT, HT H, T HH}.

Let C be the event at least two tails. That is,

C = {T T H, T HT, HT T, T T T }.

12 / 77
Example (Continued)
Which of the events A, B, and C are mutually
exclusive?

13 / 77
Example (Continued)
Which of the events A, B, and C are mutually
exclusive?

Solution
B and C are mutually exclusive events since B ∩ C = ∅.

13 / 77
Example (Continued)
Suppose that HHT is the result of the experiment.
Which of A, B, and C have occurred?

14 / 77
Example (Continued)
Suppose that HHT is the result of the experiment.
Which of A, B, and C have occurred?

Solution
Events A and B have occurred.

14 / 77
1.3 Axioms and Basic Results

15 / 77
For every event A in a sample space S, we assign a number,
P (A), representing the probability that the event A has
occurred. The function P must satisfy the following
axioms.

Axioms
(i) for each A ⊂ S, P (A) ≥ 0

16 / 77
For every event A in a sample space S, we assign a number,
P (A), representing the probability that the event A has
occurred. The function P must satisfy the following
axioms.

Axioms
(i) for each A ⊂ S, P (A) ≥ 0
(ii) P (S) = 1

16 / 77
For every event A in a sample space S, we assign a number,
P (A), representing the probability that the event A has
occurred. The function P must satisfy the following
axioms.

Axioms
(i) for each A ⊂ S, P (A) ≥ 0
(ii) P (S) = 1
(iii) if A1 , A2 , . . . are mutually exclusive events, i.e.
Ai ∩ Aj = ∅ for all i, j with i ̸= j, then
∞
[  ∞
X
P Ai = P (Ai ).
i=1 i=1

16 / 77
It follows that
1 for each A ⊂ S, 0 ≤ P (A) ≤ 1, we have the probability
of the complement of event A is P(Ā) = 1 − P(A).
2 P(∅) = 0
3 if A1 , A2 , . . . , Ak are mutually exclusive events, then
k
[  Xk
P Ai = P(Ai ).
i=1 i=1

4 if B ⊂ A, then P (B) ≤ P (A)

if B occurs implies that A occurs, then P (B) ≤ P (A).

17 / 77
 Examples of Calculating
Probabilities when Sample Space S
is discrete

18 / 77
Example
If S is discrete and mutually exclusive, then P (A) is the
sum of the probability of the outcomes in A.
This follows from Axiom (iii) Jump to Axiom slide

If S = {s1 , s2 , s3 , s4 , . . . } and A = {s2 , s5 , s9 }, then we view

A as
A = {s2 } ∪ {s5 } ∪ {s9 },
which is a union of mutually exclusive events.
Hence,

P (A) = P ({s2 }∪{s5 }∪{s9 }) = P ({s2 })+P ({s5 })+P ({s9 }).

19 / 77
Example
Consider tossing three coins simultaneously.

Recall that there are 23 = 8 possible outcomes, and each

outcome is equally likely Jump to Three coin example .

So, the probability of each event is 18 . Hence

P (A) = P (at least one head)

= P {HHH} ∪ {HHT } ∪ {HT H} ∪ {T HH}


∪ {T T H} ∪ {T HT } ∪ {HT T }
= P ({HHH}) + P ({HHT }) + P ({HT H})
+P ({T HH}) + P ({T T H}) + P ({T HT })
+P ({HT T })
1 1 1 1 1 1 1
= + + + + + +
8 8 8 8 8 8 8
7
= .
8
20 / 77
Example (Continued)
Similarly,

P (B) = P (exactly two heads)

= P ({HHT }) + P ({HT H}) + P ({T HH})
3
=
8

P (C) = P (at least two tails)

= P ({T T T }) + P ({T T H}) + P ({T HT })
+P ({HT T })
4 1
= =
8 2

21 / 77
1.4 Counting Rules

22 / 77
Counting Rule 1
If there are k experiments with ni , i = 1, 2, . . . , k, possible
outcomes in the ith experiments, then the total number of
possible outcomes forQ the k experiments is
n1 × n2 × · · · × nk = ki=1 ni .

23 / 77
Example (of Counting Rule 1)
Suppose you toss a six-sided die three times and consider
the possible results of the three tosses.
What is k and ni in this case?

24 / 77
Example (of Counting Rule 1)
Suppose you toss a six-sided die three times and consider
the possible results of the three tosses.
What is k and ni in this case?
Solution:
Here k = 3 and n1 = n2 = n3 = 6.

24 / 77
Example (of Counting Rule 1)
Suppose you toss a six-sided die three times and consider
the possible results of the three tosses.
What is k and ni in this case?
Solution:
Here k = 3 and n1 = n2 = n3 = 6.

What is the number of outcomes in the sample space?

24 / 77
Example (of Counting Rule 1)
Suppose you toss a six-sided die three times and consider
the possible results of the three tosses.
What is k and ni in this case?
Solution:
Here k = 3 and n1 = n2 = n3 = 6.

What is the number of outcomes in the sample space?

Solution:
Q3
There are i=1 ni = 6 × 6 × 6 = 216 possible outcomes.

24 / 77
Definition
An ordered arrangement of a set of distinct objects is called
a permutation

25 / 77
Definition
An ordered arrangement of a set of distinct objects is called
a permutation

Example
How many permutations are there for six things? For
example, if there are six different statistics textbooks on a
bookshelf, how many ways can you order the book?

25 / 77
Definition
An ordered arrangement of a set of distinct objects is called
a permutation

Example
How many permutations are there for six things? For
example, if there are six different statistics textbooks on a
bookshelf, how many ways can you order the book?
Solution:
The books can be arranged in
6! = 6 × 5 × 4 × 3 × 2 × 1 = 720 ways.

25 / 77
Counting Rule 2
The number of possible permutations of r objects selected
from n distinct objects is

n n!
Pr = ,
(n − r)!

where n! = n × (n − 1) × · · · × 3 × 2 × 1 for n ≥ 1 and the

convention 0! = 1.

26 / 77
Proof Counting Rule 2.

n
Pr = n × (n − 1) × (n − 2) × · · · × (n − (r − 1))

(n − r) × (n − r − 1) × · · · 3 × 2 × 1
= n × (n − 1) × (n − 2) × · · · × (n − (r − 1)) ×
(n − r) × (n − r − 1) × · · · 3 × 2 × 1

n × (n − 1) × · · · × (n − r) × (n − r − 1) × · · · 3 × 2 × 1
=
(n − r) × (n − r − 1) × · · · 3 × 2 × 1

n!
=
(n − r)!

27 / 77
Example
A particular committee has four members. One member
must be the committee chair, and a different committee
member must take the meeting minutes.

How many different ways are there of choosing a Chair and

Minute-taker for this committee? (That is, how many
possible choices are there?)

28 / 77
Example
A particular committee has four members. One member
must be the committee chair, and a different committee
member must take the meeting minutes.

How many different ways are there of choosing a Chair and

Minute-taker for this committee? (That is, how many
possible choices are there?)

Solution:
4! 4×3×2×1
There are 4 P2 = 2!
= 2×1
= 12

28 / 77
Example
A particular committee has four members. One member
must be the committee chair, and a different committee
member must take the meeting minutes.

How many different ways are there of choosing a Chair and

Minute-taker for this committee? (That is, how many
possible choices are there?)

Solution:
4! 4×3×2×1
There are 4 P2 = 2!
= 2×1
= 12

Note that the number of possible permutations of r

r!
objects with n = r is r Pr = 0! = r!

28 / 77
Counting Rule 3
The number of ways of choosing r objects from n distinct
objects is n choose r, nr , i.e.

 
n n!
= , 0 ≤ r ≤ n.
r r! (n − r)!

29 / 77
Proof of Counting Rule 3.
The number of permutations (ordered arrangements) is n Pr .

However, in this case, we are ignoring order and only

considering the selection;

so that a particular set of r objects contributes r!

permutations to the total.

Now count as one selection.

Hence, the number of ways of choosing r objects from n

distinct objects is
n
Pr n!
=
r! r! (n − r)!.

30 / 77
Example
From a committee of four persons, two committee members
must present the committee’s recommendations to the
board of directors.

How many ways are there of choosing two committee

members to report to the board of directors? (How many
different selections of two people from 4 persons?)

31 / 77
Example
From a committee of four persons, two committee members
must present the committee’s recommendations to the
board of directors.

How many ways are there of choosing two committee

members to report to the board of directors? (How many
different selections of two people from 4 persons?)

Solution:
4 4! 4××3×2×1


There are 2
=4 C2 = 2! (4−2)!
= 2×1 2×1
=6

31 / 77
1.5 Conditional Probability

32 / 77
Definition
For any two events A and B, where P (B) ̸= 0, we define
the conditional probability that an event A occurs given
that the event B has occurred is
P(A ∩ B)
P(A B) = , if P(B) ̸= 0.
P(B)

33 / 77
Given that B has occurred, the total probability for
possible results of the experiment equal P (B), so the
probability that A occurs equals the total probability for
outcomes in A (only those in A ∩ B) divided by the total
probability, P (B).

34 / 77
Note that we can rearrange the conditional probability
formula to get

P (A ∩ B) = P (A B) P (B).

We can also write A ∩ B = B ∩ A, so it follows that

P (A ∩ B) = P (B ∩ A).

Interchange A and B in the conditional probability

definition, we get

P (B ∩ A)
P (B A) =
P (A)

if P (A) ̸= 0.

35 / 77
Lemma
P (A B) = P (A) if and only if P (B A) = P (B)

36 / 77
Proof of Lemma.
If P (A B) = P (A) then

P (B ∩ A)
P (B A) = by definition of conditional probability
P (A)
P (A ∩ B)
=
P (A)
P (A B) P (B)
=
P (A)
P (A) P (B)
= since P (A B) = P (A)
P (A)
= P (B).

To show that P (A B) = P (A) interchange A and B in the

above derivation.

37 / 77
1.6 Independent Events

38 / 77
Definition
Any two events A and B are independent if

P (A ∩ B) = P (A) P (B).

39 / 77
Definition
Any two events A and B are independent if

P (A ∩ B) = P (A) P (B).

(If A and B are not independent, they are dependent.)

39 / 77
For any two events A and B, we say that A and B are
independent if and only if

P (A B) = P (A)

or
P (B A) = P (B),
which is a consequence of the definition of independent.

Recall the Lemma on conditional probability

Jump to Lemma on Conditional Probability slide .

This slide shows that A and B are independent.

40 / 77
Example
Suppose two fair dice are tossed, and consider the sum of
numbers on the first and second die. There are 62 = 36
possible outcomes in the sample space S, each with
1
probability 36 .
The sample sample S consists of

S = {(1, 1), (1, 2), . . . , (1, 6), (2, 1), . . . (2, 6), . . . (6, 6)}.

Let
A be the event four appears on the first die
B be the event sum of the two dice is seven
C be the event sum of the two dice is eight.

41 / 77
Example (Continued)
We see immediately that B and C are NOT independent
since P (B ∩ C) = 0, where

B = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), (6, 1)} and

C = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}.

However, P (B) > 0, P (C) > 0 so P (B) P (C) > 0.

This implies that P (B ∩ C) ̸= P (B) P (C).

42 / 77
Example (Continued)
Show that A and B are independent.

43 / 77
Example (Continued)
Show that A and B are independent.

Solution:
6 1 6
We have P (A) = 36
= 6
and P (B) = 36
= 61 .
1
Note that P (A ∩ B) = P ((4, 3)) = 36
and
1 1 1
P (A) P (B) = 6
× 6
= 36
.

Therefore P (A ∩ B) = P (A) P (B).

Hence, A and B are independent.

43 / 77
Example (Continued)
Show that A and C are dependent.

44 / 77
Example (Continued)
Show that A and C are dependent.

Solution:
1 5
Note that P (A) = 6
and P (C) = 36
.
1
P (A ∩ C) = P ((4, 4)) = 36
and
1 5 5
P (A) P (C) = 6
× 36
= 216
.

We see that P (A ∩ C) ̸= P (A) P (C).

Hence, A and C are dependent.

44 / 77
Definition
A countable sequence of events {Ai }, i = 1, 2, 3, . . . , are
pairwise independent if

P (Ai ∩ Aj ) = P (Ai ) P (Aj ) for all i ̸= j.

45 / 77
Definition
A countable sequence of events {Ai }, i = 1, 2, 3, . . . , are
pairwise independent if

P (Ai ∩ Aj ) = P (Ai ) P (Aj ) for all i ̸= j.

Definition
The events are (mutually) independent if for any
collections Ai1 , . . . , Ain ,

P (Ai1 ∩ Ai2 ∩ · · · ∩ Ain ) = P (Ai1 ) P (Ai2 ) · · · P (Ain )

45 / 77
Definition
A countable sequence of events {Ai }, i = 1, 2, 3, . . . , are
pairwise independent if

P (Ai ∩ Aj ) = P (Ai ) P (Aj ) for all i ̸= j.

Definition
The events are (mutually) independent if for any
collections Ai1 , . . . , Ain ,

P (Ai1 ∩ Ai2 ∩ · · · ∩ Ain ) = P (Ai1 ) P (Ai2 ) · · · P (Ain )

Independence implies pairwise independence. However,

pairwise independence does not imply independence. We
will see this in the next example.

45 / 77
Example (of pairwise independent does not imply
independence)
A coin is tossed twice. The sample space S is

S = {HH, HT, T H, T T }.

Let
A be the event that heads on the first toss (i.e.
A = {HH, HT }),
B be the event that heads on the second toss (i.e.
B = {HH, T H}),
C be the event that exactly one head turned up (i,e,
C = {HT, T H})

46 / 77
Example (Continued)
We observe that A, B, and C are pairwise independent.
1
P (A ∩ B) = P (HH) = 4
and
1 1
P (A) = 2
and P (B) = 2
, so
1 1 1
P (A) P (B) = 2
× 2
= 4
.

Similarly, we can see B and C are pairwise independent;

and A and C are pairwise independent.

We can easily see P (A ∩ B ∩ C) = 0.

1 3
= 81 .


However, the P (A) P (B) P (C) = 2

Hence A, B, C are not independent.

47 / 77
Example
A ball is drawn randomly from an urn containing four balls
numbered 1,2,3,4. Let
A = {1, 2} i.e. Ball 1 or Ball 2 is drawn
B = {1, 3} i.e. Ball 1 or Ball 3 is drawn
C = {1, 4} i.e. Ball 1 or Ball 4 is drawn.

Show that A, B, and C are pairwise independent but not

independent.

48 / 77
Example (Continued)

Solution:
To show that the pairs of AB, AC, and BC are pairwise
independent, we need to show that
P (A ∩ B) = P (A) P (B), P (A ∩ C) = P (A) P (C), and
P (B ∩ C) = P (B) P (C), respectively.

Observe that P (A) = P (B) = P (C) = 24 = 12 and

P (A ∩ B) = P ({1}) = 41 . So, P (A ∩ B) = P (A) P (B).

Similarly, the other two pairs are pairwise independent.

49 / 77
Example (Continued)

Solution:
To see that A, B, C are not independence, we observe that
1  1 3 1
P (A∩B∩C) = P ({1}) = ̸= P (A) P (B) P (C) = = .
4 2 8

Hence, A, B, and C are not (mutually) independent.

50 / 77
1.7 Additional Probability Laws

51 / 77
The Multiplicative Law
For two events A1 and A2 , we have

P (A1 ∩ A2 ) = P (A2 ∩ A1 ) = P (A2 A1 ) P (A1 ).

For three events A1 , A2 and A3 , we have

P (A1 ∩ A2 ∩ A3 ) = P (A3 ∩ A2 ∩ A1 )
= P (A3 A1 ∩ A2 ) P (A1 ∩ A2 )
= P (A3 A1 ∩ A2 )P (A2 A1 ) P (A1 ).

The sample pattern applies to higher numbers of events.

This law is particularly useful when we have a sequence of
dependent trials.

52 / 77
Example
To enter a selective high school, students must pass three
tests. Suppose 20% fail the first test, and they are excluded.
Of the 80% who passed the first test, 30% failed the second,
and these unsuccessful students are excluded. Of those who
passed the second test, only 60% will pass the third.

53 / 77
Example
To enter a selective high school, students must pass three
tests. Suppose 20% fail the first test, and they are excluded.
Of the 80% who passed the first test, 30% failed the second,
and these unsuccessful students are excluded. Of those who
passed the second test, only 60% will pass the third.

Summary of the Example:

First, we need to define our events. Let Ai be the event the
student passes the i test, i = 1, 2, 3. From the question, we
see that
P (Ā1 ) = 0.20,
P (Ā2 A1 ) = 0.3 and
P (A3 A2 ∩ A1 ) = 0.60

53 / 77
Example (Continued)
Question 1: What proportion of students pass the first
two tests? Use the multiplicative law to answer this
question.

54 / 77
Example (Continued)
Question 1: What proportion of students pass the first
two tests? Use the multiplicative law to answer this
question.
Solution:
We want P (A1 ∩ A2 ). That is,

P (A1 ∩ A2 ) = P (A2 A1 )P (A1 ) = 0.7 × 0.8 = 0.56.

The proportion of students that passed the first two tests is

56%.

Note that P (A1 ) = 1 − P (Ā1 ) = 1 − 0.2 = 0.8 and

P (A2 A1 ) = 1 − P (Ā2 A1 ) = 1 − 0.3 = 0.70.

54 / 77
Example (Continued)
Question 2: What proportion of students gained entry to
the selective high school?

55 / 77
Example (Continued)
Question 2: What proportion of students gained entry to
the selective high school?
Solution:
We want P (A1 ∩ A2 ∩ A3 ). That is,

P (A1 ∩ A2 ∩ A3 ) = P (A3 A1 ∩ A2 ) P (A2 A1 )P (A1 )

= 0.6 × 0.7 × 0.8 = 0.336.

The proportion of students who gained entry to the

selective high school is 33.6%.

Note that P (A1 ) = 1 − P (Ā1 ) = 1 − 0.2 = 0.8 and

P (A3 A1 ∩ A2 ) = 0.60.

55 / 77
Example (Continued)
Question 3: What proportion of students passed the first
two tests but failed the third?

56 / 77
Example (Continued)
Question 3: What proportion of students passed the first
two tests but failed the third?
Solution:
We want P (A1 ∩ A2 ∩ Ā3 ). That is,

P (A1 ∩ A2 ∩ Ā3 ) = P (Ā3 A1 ∩ A2 ) P (A2 A1 )P (A1 )

= 0.4 × 0.7 × 0.8 = 0.224.

The proportion of students who passed the first two tests

but failed the third is 22.4%.

Note that
P (Ā3 A1 ∩ A2 ) = 1 − P (A3 A1 ∩ A2 ) = 1 − 0.6 = 0.4

56 / 77
The Additive Law
For two events A and B, we have

P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

57 / 77
Proof of Additive Law for Two Events
From the Venn Diagrams,

A B A B

Figure: The shaded area is Figure: The shaded area is

A ∩ B. A ∪ B.

we see that

A ∪ B = A ∪ (Ā ∩ B) and B = (A ∩ B) ∪ (Ā ∩ B).

58 / 77
Proof of Additive Law for Two Events - Continued.
Note that A and Ā ∩ B are mutually exclusive
Jump to the definition of mutually exclusive events and A ∩ B and Ā ∩ B

are mutual exclusive. So, from the probability axioms,

Jump to the Probability Axioms (iii) , we have

P (A ∪ B) = P (A) + P (Ā ∩ B) (1)

P (B) = P (A ∩ B) + P (Ā ∩ B). (2)

From Equation (1), we have P (Ā ∩ B) = P (A ∪ B) − P (A).
Substituting this expression into Equation (2) and
rearranging, we obtain the required result

P (A ∪ B) = P (A) + P (B) − P (A ∩ B).

59 / 77
The additive law can be extended to more than two events.

Definition
For events A, B and C,

P (A ∪ B ∪ C) = P (A) + P (B) + P (C)

−P (A ∩ B) − P (B ∩ C) − P (A ∩ C)
+P (A ∩ B ∩ C).

60 / 77
Example
Suppose three letters are placed randomly into three
envelopes, one in each envelope. What is the probability
that none is in the correct envelope?

61 / 77
Example
Suppose three letters are placed randomly into three
envelopes, one in each envelope. What is the probability
that none is in the correct envelope?

Solution:
Let A, B, C be the events that envelopes 1,2,3 contain the
correct letters, respectively. We solve this example in two
ways.

First solution: There are six possible arrangements of three

letters into three envelopes: {123, 132, 213, 231, 312, 321}.
So,
2 1
P ( none are in the correct envelope) = P ({231, 312}) = = .
6 3

61 / 77
Example (Continued)
Suppose three letters are placed randomly into three
envelopes, one in each envelope. What is the probability
that none is in the correct envelope?

62 / 77
Example (Continued)
Suppose three letters are placed randomly into three
envelopes, one in each envelope. What is the probability
that none is in the correct envelope?

Solution:

Second Solution: We have

P (A) = P (B) = P (C) = 13 ,
1 1
P (A ∩ B) = P (B A)P (A) = 2 × 3 = 16 ; and similarly,
P (B ∩ C) = P (A ∩ C) = 16
1 1 1
P (A ∩ B ∩ C) = P (C A ∩ B) P (B A) P (A) = 1 × 2 × 3 = 6

62 / 77
Example (Continued)

Solution:

By Jump to Additive Law for Three Events , we have

1 1 1 1 1 1 1 2
P (A ∪ B ∪ C) = + + − − − + = .
3 3 3 6 6 6 6 3
Therefore,
2
P ( none are in the correct envelope) = 1 − P (A ∪ B ∪ C) = 1 = 3 = 13 .

63 / 77
Law of Total Probability
Suppose that A1 , A2 , . . . , Ak are mutually exclusive events
(i.e. Ai ∩ Aj = ∅ for all i ̸= j) and exhaustive events (i.e.
Sk
i=1 Ai = S, in order words, A1 , A2 , . . . , Ak form a
partition of S.) Then, for any event B,
k
X
P (B) = P (B Ai )P (Ai ).
i=1

64 / 77
Proof of the Law of Total Probability.
Write B as a disjoint union, i.e. B = ki=1 (B ∩ Ai ) since
S
the Ai ’s are disjoint. By the finite case of probability
axioms Finite case of Probability Axioms ,
k
[ 
P (B) = P (B ∩ Ai )
i=1
k
X
= P (B ∩ Ai )
i=1
k
P (Ai ∩B)
X
= P (B Ai ) P (Ai ) since P (B|Ai ) = P (Ai ) .
i=1

We obtain the desired result.

65 / 77
Example
Urn I contains three red and four white balls. Urn II contains two red
balls and four white balls. A ball is drawn from Urn I and placed
unseen into Urn II. A ball is now drawn at random from Urn II.

What is the probability that this second ball is red?

66 / 77
Example
Urn I contains three red and four white balls. Urn II contains two red
balls and four white balls. A ball is drawn from Urn I and placed
unseen into Urn II. A ball is now drawn at random from Urn II.

What is the probability that this second ball is red?

Solution:

Let
A1 be the event that first ball drawn is red
A2 be the event that first ball drawn is white
B be the event second ball drawn is red.
The events A1 and A2 are mutually exclusive (because they cannot both occur) and exhaustive
(because one must occur). So

P (B) = P (B A1 ) P (A1 ) + P (B A2 ) P (A2 )

3 3 2 4
= × + ×
7 7 7 7
17
= .
49

17 .
The probability that the second ball drawn is read is 49

66 / 77
1.8 Bayes Rule Formula

67 / 77
Bayes rule calculates the conditional probability when the
ordering of conditioning is reversed.

In the simple two-events case, the Bayes rule is

P (A ∩ B) P (B A) P (A)
P (A B) = = .
P (B) P (B)

68 / 77
Generalised Bayes rule
For a partition A1 , A2 , . . . , Ak and an event B,

P (B Aj ) P (Aj ) P (B Aj ) P (Aj )
P (Aj B) = = Pk .
P (B) i=1 P (B A i )P (Ai )

69 / 77
Example
Recall the previous example about the two urns
Example of the two urns .

Given that the second ball drawn is red, what is the

probability that the first ball was white?

70 / 77
Example
Recall the previous example about the two urns
Example of the two urns .

Given that the second ball drawn is red, what is the

probability that the first ball was white?
Solution:
By Bayes rule, we have

P (B A2 ) P (A2 )
P (A2 B) =
P (B)
2 4
7 × 7
= 17
49
8
= .
17
The probability that the first ball was white, given that the second
8
ball drawn is red, is 17 .

70 / 77
Example
A diagnostic test for a certain disease is claimed to be 90%
accurate because, if a person has the disease, the test will
show a positive result with a probability of 0.9. In contrast,
if a person does not have the disease, the test will show a
negative result with a probability of 0.9. Only 1% of the
population has the disease.

If a person is chosen at random from the

population and tests positive for the disease, what
is the probability that the person does in fact have
the disease?

71 / 77
Example (Continued)

Solution:
Let A be the event that person has the disease and B be
the event that person tests positive.

We have
P (A ∩ B) P (B A) P (A)
P (A B) = = ,
P (B) P (B A) P (A) + P (B Ā) P (Ā)

since A and Ā form a partition, which are mutually

exclusive and exhaustive.

72 / 77
Example (Continued)

Solution - continued:
From the example, we have
P (B A) = 0.9, P (B Ā) = 0.1,
P (B A) = 0.9, P (B Ā) = 0.1
Therefore,

P (B A) P (A)
P (A B) =
P (B A) P (A) + P (B Ā) P (Ā)
0.9 × 0.01
=
0.9 × 0.01 + 0.1 × 0.99
1
=
12

73 / 77
1.9 Supplementary Material

74 / 77
A Venn diagram illustrates the relationships between two
or more events.

A B A B

Figure: A ∩ B. Figure: A \ B

75 / 77
A∪B∪C

A B

76 / 77
 A B A B

C C

Figure: A \ (B ∪ C) Figure: A ∩ B ∩ C

77 / 77