Note 4

The probability of A occurring given that B has occurred is

the conditional probability:

P(A ∩ B)
P(A|B) =
P(B)

Don’t confuse P(A|B) and P(B|A)

Events A and B are independent if P(A ∩ B) = P(A)P(B)

Product rule: P(A ∩ B) = P(A|B)P(B), so if A and B are
independent, P(A|B) = P(A).
The method:

P(A1 ∩ · · · ∩ An) = P(A1)P(A2|A1)P(A3|A2 ∩ A1) · · ·

Partition theorems
For any event B, we have that B ∪ Bc = Ω:

S
B Bc =

So if A is any other event, A = (A ∩ B) ∪ (A ∩ Bc ):

S
A =

A∩B A ∩ Bc

Formally,
A = A ∩ Ω = A ∩ (B ∪ Bc ) = (A ∩ B) ∪ (A ∩ Bc )
Because A ∩ B and A ∩ Bc are disjoint, their probabilities add:

P(A) = P(A ∩ B) + P(A ∩ Bc ) .

Together with the multiplication rules

P(A ∩ B) = P(A|B)P(B) and P(A ∩ Bc ) = P(A|Bc )P(Bc ) ,

we arrive at
Theorem (The partition rule)
For any two events A, B, we have

P(A) = P(A|B)P(B) + P(A|Bc )P(Bc ) .

Remark
This is also called the rule of total probability or the rule of
alternatives.
A general partition rule
Definition
By a (finite) partition of Ω we mean events {B1 , B2 , . . . , Bn }
such that Bi ∩ Bj = ∅ for i 6= j and n
S
i=1 Bi = Ω.

Theorem (General partition rule)

Let {B1 , . . . , Bn } be a partition of Ω. Then for any event A,

X
n
P(A) = P(A|Bi )P(Bi ) .
i=1
Example (More coins)
A box contains 3 double-headed coins, 2 double-tailed coins
and 5 conventional coins. You pick a coin at random and flip it.
What is the probability that you get a head?
Example (More coins)
A box contains 3 double-headed coins, 2 double-tailed coins
and 5 conventional coins. You pick a coin at random and flip it.
What is the probability that you get a head?
Let H be the event that you get a head and let A, B, C be the
events that the coin you picked was double-headed,
double-tailed or conventional, respectively.
Example (More coins)
A box contains 3 double-headed coins, 2 double-tailed coins
and 5 conventional coins. You pick a coin at random and flip it.
What is the probability that you get a head?
Let H be the event that you get a head and let A, B, C be the
events that the coin you picked was double-headed,
double-tailed or conventional, respectively.
Then by the (general) partition rule

P(H) = P(H|A)P(A) + P(H|B)P(B) + P(H|C)P(C)

3 2 1 5
= (1 × 10 ) + (0 × 10 ) + ( 2 × 10 )
3 1 11
= 10 + 4 = 20
Example (Medical tests)
A virus infects a proportion p of individuals in a given
population. A test is devised to indicate whether a given
individual is infected. The probability that the test is positive for
an infected individual is 95%, but there is a 10% probability of a
false positive. Testing an individual at random, what is the
chance of a positive result?
Example (Medical tests)
A virus infects a proportion p of individuals in a given
population. A test is devised to indicate whether a given
individual is infected. The probability that the test is positive for
an infected individual is 95%, but there is a 10% probability of a
false positive. Testing an individual at random, what is the
chance of a positive result?
Let P denote the event that the result of the test is positive and
V the event that the individual is infected.
Example (Medical tests)
A virus infects a proportion p of individuals in a given
population. A test is devised to indicate whether a given
individual is infected. The probability that the test is positive for
an infected individual is 95%, but there is a 10% probability of a
false positive. Testing an individual at random, what is the
chance of a positive result?
Let P denote the event that the result of the test is positive and
V the event that the individual is infected. Then

P(P) = P(P|V)P(V) + P(P|V c )P(V c )

Example (Medical tests)
A virus infects a proportion p of individuals in a given
population. A test is devised to indicate whether a given
individual is infected. The probability that the test is positive for
an infected individual is 95%, but there is a 10% probability of a
false positive. Testing an individual at random, what is the
chance of a positive result?
Let P denote the event that the result of the test is positive and
V the event that the individual is infected. Then

P(P) = P(P|V)P(V) + P(P|V c )P(V c )

= 0.95p + 0.1(1 − p)
Example (Medical tests)
A virus infects a proportion p of individuals in a given
population. A test is devised to indicate whether a given
individual is infected. The probability that the test is positive for
an infected individual is 95%, but there is a 10% probability of a
false positive. Testing an individual at random, what is the
chance of a positive result?
Let P denote the event that the result of the test is positive and
V the event that the individual is infected. Then

P(P) = P(P|V)P(V) + P(P|V c )P(V c )

= 0.95p + 0.1(1 − p)
= 0.85p + 0.1
Example (Noisy channels)
Alice and Bob communicate across a noisy channel using a bit
stream. Let S0 (resp. S1 ) denote the event that a 0(resp. 1) was
sent, and let R0 (resp. R1 ) denote the event that a 0 (resp. 1)
was received. Suppose that P(S0 ) = 74 and that due to the noise
P(R1 |S0 ) = 18 and P(R0 |S1 ) = 61 . What is P(S0 |R0 )?
Example (Noisy channels)
Alice and Bob communicate across a noisy channel using a bit
stream. Let S0 (resp. S1 ) denote the event that a 0(resp. 1) was
sent, and let R0 (resp. R1 ) denote the event that a 0 (resp. 1)
was received. Suppose that P(S0 ) = 74 and that due to the noise
P(R1 |S0 ) = 18 and P(R0 |S1 ) = 61 . What is P(S0 |R0 )?

P(S0 ∩ R0 ) P(S0 ∩ R0 )
P(S0 |R0 ) = =
P(R0 ) P(S0 ∩ R0 ) + P(S1 ∩ R0 )
Example (Noisy channels)
Alice and Bob communicate across a noisy channel using a bit
stream. Let S0 (resp. S1 ) denote the event that a 0(resp. 1) was
sent, and let R0 (resp. R1 ) denote the event that a 0 (resp. 1)
was received. Suppose that P(S0 ) = 74 and that due to the noise
P(R1 |S0 ) = 18 and P(R0 |S1 ) = 61 . What is P(S0 |R0 )?

P(S0 ∩ R0 ) P(S0 ∩ R0 )
P(S0 |R0 ) = =
P(R0 ) P(S0 ∩ R0 ) + P(S1 ∩ R0 )

1 3 1
P(S1 ∩ R0 ) = P(R0 |S1 )P(S1 ) = P(R0 |S1 )(1 − P(S0 )) = 6 × 7 = 14
Example (Noisy channels)
Alice and Bob communicate across a noisy channel using a bit
stream. Let S0 (resp. S1 ) denote the event that a 0(resp. 1) was
sent, and let R0 (resp. R1 ) denote the event that a 0 (resp. 1)
was received. Suppose that P(S0 ) = 74 and that due to the noise
P(R1 |S0 ) = 81 and P(R0 |S1 ) = 16 . What is P(S0 |R0 )?

P(S0 ∩ R0 ) P(S0 ∩ R0 )
P(S0 |R0 ) = =
P(R0 ) P(S0 ∩ R0 ) + P(S1 ∩ R0 )

1 3 1
P(S1 ∩ R0 ) = P(R0 |S1 )P(S1 ) = P(R0 |S1 )(1 − P(S0 )) = 6 × 7 = 14
7 4 1
P(S0 ∩ R0 ) = P(R0 |S0 )P(S0 ) = (1 − P(R1 |S0 ))P(S0 ) = 8 × 7 = 2
Example (Noisy channels)
Alice and Bob communicate across a noisy channel using a bit
stream. Let S0 (resp. S1 ) denote the event that a 0(resp. 1) was
sent, and let R0 (resp. R1 ) denote the event that a 0 (resp. 1)
was received. Suppose that P(S0 ) = 74 and that due to the noise
P(R1 |S0 ) = 18 and P(R0 |S1 ) = 61 . What is P(S0 |R0 )?

P(S0 ∩ R0 ) P(S0 ∩ R0 )
P(S0 |R0 ) = =
P(R0 ) P(S0 ∩ R0 ) + P(S1 ∩ R0 )

1 3 1
P(S1 ∩ R0 ) = P(R0 |S1 )P(S1 ) = P(R0 |S1 )(1 − P(S0 )) = 6 × 7 = 14
7 4 1
P(S0 ∩ R0 ) = P(R0 |S0 )P(S0 ) = (1 − P(R1 |S0 ))P(S0 ) = 8 × 7 = 2

. .
1
∴ P(S0 |R0 ) = 2 ( 12 + 1
14 ) = 1
2
4
7 = 7
8
Conditional partition rule
Theorem
Let {B1 , . . . , Bn } be a partition of Ω and let C be an event with
P(C) > 0. Then for any event A,

X
n
P(A|C) = P(A|Bi ∩ C)P(Bi |C) .
i=1
Example
There are number of different drugs to treat a disease and each
drug may give rise to side effects. A certain drug C has a 99%
success rate in the absence of side effects and side effects only
arise in 5% of cases. If they do arise, however, then C has only
a 30% success rate. If C is used, what is the probability of the
event A that a cure is effected?
Example
There are number of different drugs to treat a disease and each
drug may give rise to side effects. A certain drug C has a 99%
success rate in the absence of side effects and side effects only
arise in 5% of cases. If they do arise, however, then C has only
a 30% success rate. If C is used, what is the probability of the
event A that a cure is effected?
Let B be the event that no side effects occur. We are given that

P(A|B ∩ C) = 0.99 P(B|C) = 0.95 P(A|Bc ∩ C) = 0.3 ,

whence P(Bc |C) = 0.05.

Example
There are number of different drugs to treat a disease and each
drug may give rise to side effects. A certain drug C has a 99%
success rate in the absence of side effects and side effects only
arise in 5% of cases. If they do arise, however, then C has only
a 30% success rate. If C is used, what is the probability of the
event A that a cure is effected?
Let B be the event that no side effects occur. We are given that

P(A|B ∩ C) = 0.99 P(B|C) = 0.95 P(A|Bc ∩ C) = 0.3 ,

whence P(Bc |C) = 0.05. By the conditional partition rule

corresponding to the partition {B, Bc } and condition C,

P(A|C) = P(A|B ∩ C)P(B|C) + P(A|Bc ∩ C)P(Bc |C)

Example
There are number of different drugs to treat a disease and each
drug may give rise to side effects. A certain drug C has a 99%
success rate in the absence of side effects and side effects only
arise in 5% of cases. If they do arise, however, then C has only
a 30% success rate. If C is used, what is the probability of the
event A that a cure is effected?
Let B be the event that no side effects occur. We are given that

P(A|B ∩ C) = 0.99 P(B|C) = 0.95 P(A|Bc ∩ C) = 0.3 ,

whence P(Bc |C) = 0.05. By the conditional partition rule

corresponding to the partition {B, Bc } and condition C,

P(A|C) = P(A|B ∩ C)P(B|C) + P(A|Bc ∩ C)P(Bc |C)

= (0.99 × 0.95) + (0.3 × 0.05) = 0.9555 ' 96%
Bayes’s rule
Recall the product rule

P(A ∩ B) = P(A|B)P(B)
Bayes’s rule
Recall the product rule

P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A) ,

Bayes’s rule
Recall the product rule

P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A) ,

which immediately gives

Theorem (Bayes’s rule)
P(B|A)P(A)
P(A|B) =
P(B)
Bayes’s rule
Recall the product rule

P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A) ,

which immediately gives

Theorem (Bayes’s rule)
P(B|A)P(A)
P(A|B) =
P(B)

Using the partition rule P(B) = P(B|A)P(A) + P(B|Ac )P(Ac ) we

get a modified version of Bayes’s rule:
Theorem (Bayes’s rule too)
P(B|A)P(A)
P(A|B) =
P(B|A)P(A) + P(B|Ac )P(Ac )
Example (False positives)
You get tested for the virus in the earlier example and it shows
positive. What is the probability that you are actually infected?
Example (False positives)
You get tested for the virus in the earlier example and it shows
positive. What is the probability that you are actually infected?
In the notation of the earlier example , we want to compute P(V|P).
Example (False positives)
You get tested for the virus in the earlier example and it shows
positive. What is the probability that you are actually infected?
In the notation of the earlier example , we want to compute P(V|P). By
Bayes’s rule

P(P|V)P(V) 0.95p
P(V|P) = =
P(P) 0.85p + 0.1
Example (False positives)
You get tested for the virus in the earlier example and it shows
positive. What is the probability that you are actually infected?
In the notation of the earlier example , we want to compute P(V|P). By
Bayes’s rule

P(P|V)P(V) 0.95p
P(V|P) = =
P(P) 0.85p + 0.1

So that if half the population is infected (p = 0.5), then

P(V|P) ' 90% and the test looks good, but if the virus affects
only one person in every thousand (p = 10−3 ), then
P(V|P) ' 1%, so not very conclusive at all!
Example (Multiple choice exam)
A student is taking a multiple choice exam, each question
having c available choices. The student either knows the
answer to the question with probability p or else guesses at
random with probability 1 − p. Given that the answer selected is
correct, what is the probability that the student knew the
answer?
Example (Multiple choice exam)
A student is taking a multiple choice exam, each question
having c available choices. The student either knows the
answer to the question with probability p or else guesses at
random with probability 1 − p. Given that the answer selected is
correct, what is the probability that the student knew the
answer?
Let A denote the event that the answer is correct and let K
denote the event that the student knew the answer. We are
after P(K|A).
Example (Multiple choice exam)
A student is taking a multiple choice exam, each question
having c available choices. The student either knows the
answer to the question with probability p or else guesses at
random with probability 1 − p. Given that the answer selected is
correct, what is the probability that the student knew the
answer?
Let A denote the event that the answer is correct and let K
denote the event that the student knew the answer. We are
after P(K|A). Bayes’s rule says

P(A|K)P(K)
P(K|A) = ,
P(A)

so we need to compute P(A).

Example (Multiple choice exam)
A student is taking a multiple choice exam, each question
having c available choices. The student either knows the
answer to the question with probability p or else guesses at
random with probability 1 − p. Given that the answer selected is
correct, what is the probability that the student knew the
answer?
Let A denote the event that the answer is correct and let K
denote the event that the student knew the answer. We are
after P(K|A). Bayes’s rule says

P(A|K)P(K)
P(K|A) = ,
P(A)

so we need to compute P(A). We will use the partition rule,

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc ) .

Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c,
Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c, whence

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc )

Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c, whence

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc )

= (1 × p) + ( c1 × (1 − p))
Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c, whence

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc )

= (1 × p) + ( c1 × (1 − p))
1−p
=p+ c .
Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c, whence

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc )

= (1 × p) + ( c1 × (1 − p))
1−p
=p+ c .

Finally,
p cp
P(K|A) = = .
p + (1 − p)/c 1 + (c − 1)p
Example (Multiple choice exam – continued)
We notice that P(A|K) = 1 and P(A|Kc ) = 1/c, whence

P(A) = P(A|K)P(K) + P(A|Kc )P(Kc )

= (1 × p) + ( c1 × (1 − p))
1−p
=p+ c .

Finally,
p cp
P(K|A) = = .
p + (1 − p)/c 1 + (c − 1)p
Notice that the larger the number c, the more likely that the
student knew the answer.
Conditional probability in Mendelian genetics
Let us consider a gene contained inside the X
chromosome and having two alleles: A and a.
We will assume that a male with allele A in his one X
chromosome does not present the genetic trait, whereas
one with a does.
We will assume that a female will present the trait if and
only if both her X chromosomes contain the allele a.
One says that allele A is dominant and a is recessive.
We will assume the following laws of inheritance:
a son gets one of his mother’s two X chromosomes at
random
a daughter gets her father’s X chromosome and one of her
mother’s at random
Males can therefore be A or a, whereas females can be
AA, Aa and aa. (We don’t distinguish between Aa and aA.)
Conditional probability in Mendelian genetics
Example
Suppose that a male with genotype A and a female with
genotype Aa have a daughter. She can have genotype AA or
Aa, both with probability 12 . Now suppose that she herself has a
son with genotype A. What is the (conditional) probability that
she has genotype AA?
Conditional probability in Mendelian genetics
Example
Suppose that a male with genotype A and a female with
genotype Aa have a daughter. She can have genotype AA or
Aa, both with probability 12 . Now suppose that she herself has a
son with genotype A. What is the (conditional) probability that
she has genotype AA?
Let GAA (resp. GAa ) denote the event that the daughter has
genotype AA (resp. Aa) and let SA denote the event that the
daughter’s son has genotype A. We want P(GAA |SA ). Notice
that P(SA |GAA ) = 1 and P(SA |GAa ) = 21 .
Conditional probability in Mendelian genetics
Example
Suppose that a male with genotype A and a female with
genotype Aa have a daughter. She can have genotype AA or
Aa, both with probability 12 . Now suppose that she herself has a
son with genotype A. What is the (conditional) probability that
she has genotype AA?
Let GAA (resp. GAa ) denote the event that the daughter has
genotype AA (resp. Aa) and let SA denote the event that the
daughter’s son has genotype A. We want P(GAA |SA ). Notice
that P(SA |GAA ) = 1 and P(SA |GAa ) = 21 . By the partition rule

P(SA ) = P(SA |GAA )P(GAA ) + P(SA |GAa )P(GAa )

= (1 × 21 ) + ( 12 × 21 ) = 1
2 + 1
4 = 3
4 .
Conditional probability in Mendelian genetics
Example
Suppose that a male with genotype A and a female with
genotype Aa have a daughter. She can have genotype AA or
Aa, both with probability 12 . Now suppose that she herself has a
son with genotype A. What is the (conditional) probability that
she has genotype AA?
Let GAA (resp. GAa ) denote the event that the daughter has
genotype AA (resp. Aa) and let SA denote the event that the
daughter’s son has genotype A. We want P(GAA |SA ). Notice
that P(SA |GAA ) = 1 and P(SA |GAa ) = 21 . By the partition rule

P(SA ) = P(SA |GAA )P(GAA ) + P(SA |GAa )P(GAa )

= (1 × 21 ) + ( 12 × 21 ) = 1
2 + 1
4 = 3
4 .
.
1 3
Bayes’s: P(GAA |SA ) = P(SA |GAA )P(GAA )/P(SA ) = 2 4 = 23 .
Summary
The partition rule: P(A) = P(A|B)P(B) + P(A|Bc )P(Bc )
This generalises to a partition {Bi } of the sample space:
X
P(A) = P(A|Bi )P(Bi )
i

It also applies to conditional probability:

X
P(A|C) = P(A|Bi ∩ C)P(Bi |C)
i

Bayes’s rule allows us to compute P(A|B) from a

knowledge of P(B|A) via
P(B|A)P(A) P(B|A)P(A)
P(A|B) = =
P(B) P(B|A)P(A) + P(B|Ac )P(Ac )