Bayes’ Theorem

In this section, we look at how we can use information

about conditional probabilities to calculate the reverse
conditional probabilities such as in the example below. We
already know how to solve these problems with tree
diagrams. Bayes’ theorem just states the associated
algebraic formula.
Bayes’ Theorem
In this section, we look at how we can use information
about conditional probabilities to calculate the reverse
conditional probabilities such as in the example below. We
already know how to solve these problems with tree
diagrams. Bayes’ theorem just states the associated
algebraic formula.

Example Suppose that a factory has two machines,

Machine A and Machine B, both producing jPhone touch
screens. Forty percent of their touch screens come from
Machine A and 60% of their touch screens come from
Machine B. Ten percent of the touch screens produced by
Machine A are defective and five percent of the touch
screens from Machine B are defective. If I randomly choose
a touch screen from those produced by both machines and
find that it is defective, what is the probability that it came
from machine A?
Bayes’ Theorem

We can draw a tree diagram representing the information

we are given. If we choose a touch screen at random from
those produced in the factory, we let MA be the event that
it came from Machine A and let MB be the event that it
came from Machine B. We let D denote the event that the
touch screen is defective and let ND denote the event that
it is not defective. Fill in the appropriate probabilities on
the tree diagram on the left on the next page.
Bayes’ Theorem

xD
xxx
x
xx
xx
MA ND
{{
{{{
{
{{
0 CC
CC
CC
CC
MBFF D
FF
FF
FF
ND
Bayes’ Theorem

P(M A ∩ D)
We can now calculate P M AD = =
P(D)
P(M A ∩ D)


. Note the
P DM A · P(M A) + P DM B · P(M B)
event D is shown in red above and the event M A ∩ D is
shown in blue.
rr
D
0.1rrrrr
r
rrr
t
MA ND
0.4ttttt 0.9
tt
ttt
0 JJJ
JJJ
JJJ
0.6 JJ 0.05
MB LL D
LLL
LL
0.95 LLL
ND
Bayes’ Theorem
rr
D
0.1rrrrr
r
rrr
t
MA ND
0.4ttttt 0.9
tt
ttt
0 JJJ
JJJ
JJJ
0.6 JJ 0.05
MB LL D
LLL
LL
0.95 LLL
ND

P(M A ∩ D)
P M A D = =
P(D)
P(M A ∩ D)


=
P D M A · P(M A) + P DM B · P(M B)


0.4 · 0.1 0.04 0.04 4

= = =
0.4 · 0.1 + 0.6 · 0.05 0.04 + 0.03 0.07 7
Bayes’ Theorem
Let E1 and E2 be mutually exclusive events (E1 ∩ E2 = ∅)
whose union is the sample space, i.e. E1 ∪ E2 = S. Let F
be an event in S for which P(F ) 6= 0. Then


P(E1 ∩ F ) P(E1 ∩ F )
P E 1 F = = =
P(F ) P(E1 ∩ F ) + P(E2 ∩ F )


P(E1 )P F E1


.
P(E1 )P F E1 + P(E2 )P F E2

Note that if we cross-classify outcomes in the sample space

according to whether they belong to E1 or E2 and whether
they belong to F or F 0 , we get a tree diagram as above
from which we can calculate the probabilities.
Bayes’ Theorem

P(F| E 1) P(F| E 1)
F
||
|||
|
|| P(E1) P(E1)

E1 0
F
~~
~~~
~
~~
0 @@ P(E2) P(F| E2)
@@
@@
@@
E2 B F
BB
BB
BB
B
F0
Predictive value of diagnostic test
The above analysis allows us to gain insight a commonly
misunderstood point about the accuracy of tests for
diseases and drugs. The predictive value of a diagnostic
test does not depend entirely on the sensitivity of the test.
It also depend on the prevalence of the disease. Many
people when asked the following question “If a swimmer
fails a drug test that is known to be 95 percent
accurate(whether they have drugs in their system or not),
how likely is it that he/she is really guilty?” will answer 95
percent, but of course you know that you need more
information in order to answer the question. Check out the
following article on the subject:
Doctors flunk quiz on screening-test math
Predictive value of diagnostic test
The above analysis allows us to gain insight a commonly
misunderstood point about the accuracy of tests for
diseases and drugs. The predictive value of a diagnostic
test does not depend entirely on the sensitivity of the test.
It also depend on the prevalence of the disease. Many
people when asked the following question “If a swimmer
fails a drug test that is known to be 95 percent
accurate(whether they have drugs in their system or not),
how likely is it that he/she is really guilty?” will answer 95
percent, but of course you know that you need more
information in order to answer the question. Check out the
following article on the subject:
Doctors flunk quiz on screening-test math
If you are more legally inclined, here is a discussion of
Bayes Theorem as it applies to criminal trials.
Predictive value of diagnostic test
The above analysis allows us to gain insight a commonly
misunderstood point about the accuracy of tests for
diseases and drugs. The predictive value of a diagnostic
test does not depend entirely on the sensitivity of the test.
It also depend on the prevalence of the disease. Many
people when asked the following question “If a swimmer
fails a drug test that is known to be 95 percent
accurate(whether they have drugs in their system or not),
how likely is it that he/she is really guilty?” will answer 95
percent, but of course you know that you need more
information in order to answer the question. Check out the
following article on the subject:
Doctors flunk quiz on screening-test math
If you are more legally inclined, here is a discussion of
Bayes Theorem as it applies to criminal trials.
Let’s see what happens in a few examples.
Predictive value of diagnostic test
Example Suppose, for example a test for the HIV virus is
95% accurate.The test gives a positive result for 95% of
those taking the test who are HIV positive. Also the test
gives a negative result for 95% of those taking the test who
are not HIV positive.

(a) According to a recent estimate, approximately one

million people in the U.S. are HIV positive. The population
of the U.S. is approximately 308 million. Suppose a random
U.S. resident takes the aids test and tests positive, what is
the probability that the person is infected given

that they
have tested positive, That is what is P I P ?

(We let P denote the event that a person chosen at random
from the population tests positive, we let I denote the
event that a person chosen at random is infected.)
Predictive value of diagnostic test
ll P lll P
lll lll
l
lllll 0.05
lll
l ll
lll lll
lll 307 ll
mm NI NP mm NI NP
mmm mmm
m 0.95
mmm 308
mmm
mmmmm mmm
m
mmm mmm
0 QQQQ 0 QQQQ
QQQ QQQ
QQQ QQQ
QQQ Q
QQQ
Q 1 QQQQQQ 0.05
I RRRRR NP 308 I RRRRR NP
RRR RRR
RRR RRR
RRR R
RRR
R 0.95 RRRRRR
P P
1

· 0.95 308
P I P ≈ 307 1 =
· 0.05 + 308
308
· 0.95
0.00308441558442
=
0.04983766233766 + 0.00308441558442
0.00308441558442
≈ 0.0583 = 5.83%
0.05292207792208
Predictive value of diagnostic test
(b) In country X, forty percent of the residents are HIV
positive. Suppose a random resident of Country X takes
the aids test and tests positive, what is the probability that
the person is infected given

that they have tested positive,
That is what is P I P ?

Predictive value of diagnostic test
(b) In country X, forty percent of the residents are HIV
positive. Suppose a random resident of Country X takes
the aids test and tests positive, what is the probability that
the person is infected given

that they have tested positive,
That is what is P I P ?

lll P
lllll
lll
lll
lll
NI NP
mmm
mmmmm
mm
mmm
mmm
Q
0 QQQ
QQQ
QQQ
QQQ
QQQ
Q
I RRRRR NP
RRR
RRR
RRR
RRR
R
P
Predictive value of diagnostic test
(b) In country X, forty percent of the residents are HIV
positive. Suppose a random resident of Country X takes
the aids test and tests positive, what is the probability that
the person is infected given

that they have tested positive,
That is what is P I P ?

lll P lll P
lllll 0.05 lll
l
lll l lll
lll lll
lll lll
NI NP mm NI NP
mmm m 0.95
mmmm 0.6 mmm
mmm mmm
m
mmm mmm
mmm mmm
Q
0 QQQ Q
0 QQQ
QQQ QQQ
QQQ QQQ
QQQ Q
QQQ
Q 0.4 QQQQQQ 0.05
I RRRRR NP I RRRRR NP
RRR RRR
RRR RRR
RRR R
RRR
R 0.95 RRRRRR
P P
Predictive value of diagnostic test
(b) In country X, forty percent of the residents are HIV
positive. Suppose a random resident of Country X takes
the aids test and tests positive, what is the probability that
the person is infected given

that they have tested positive,
That is what is P I P ?

lll P lll P
lllll 0.05 lll
l
lll l lll
lll lll
lll lll
NI NP mm NI NP
mmm m 0.95
mmmm 0.6 mmm
mmm mmm
m
mmm mmm
mmm mmm
Q
0 QQQ Q
0 QQQ
QQQ QQQ
QQQ QQQ
QQQ Q
QQQ
Q 0.4 QQQQQQ 0.05
I RRRRR NP I RRRRR NP
RRR RRR
RRR RRR
RRR R
RRR
R 0.95 RRRRRR
P P

0.4 · 0.95 0.38 0.38
P I P = = = ≈
0.6 · 0.05 + 0.4 · 0.95 0.03 + 0.38 0.41
0.9268 ≈ 93%
Predictive value of diagnostic test
Example A test for Lyme disease is 60% accurate when a
person has the disease and 99% accurate when a person
does not have the disease. In Country Y, 0.01% of the
population has Lyme disease. What is the probability that
a person chosen randomly from the population who test
positive for the disease actually has the disease?
Predictive value of diagnostic test
Example A test for Lyme disease is 60% accurate when a
person has the disease and 99% accurate when a person
does not have the disease. In Country Y, 0.01% of the
population has Lyme disease. What is the probability that
a person chosen randomly from the population who test
positive for the disease actually has the disease?
lP lll P
lll l
lll 0.01 lll
lllll l lll
ll lll
lll lll
m NI NP mm NI NP
mmm mmm
m 0.99
mmmmm 0.9999
mmm
m m
mmm mmm
mmm mmm
0 QQQQ 0 QQQQ
QQQ QQQ
QQQ QQQ
QQQ Q
QQQ
Q 0.0001 QQQQQQ 0.4
I RRRRR NP I RRRRR NP
RRR RRR
RRR RRR
RRR R
RRR
R 0.6 RRRRRR
P P

0.0001 · 0.06 0.00006 0.00006
P I P = 0.9999 · 0.01 + 0.0001 · 0.06
=
0.00006 + 0.009999
=
0.010059
≈
0.006.
A legal example
A crime has been committed and the only evidence is a
blood spatter that could only have come from the
perpetrator. The chance of a random individual having the
same blood type as that of the spatter is 10%. Joe has
been arrested and charged. The trial goes as follows.
Prosecutor: Since there is only a 10% chance that Joe’s
blood would match, there is a 90% chance that Joe did it.
That’s good enough for me.
Defence Lawyer: There are two hundred people in the
neighborhood who could have done the crime. Twenty of
them will have the same blood type as the sample. Hence
1
the chances that Joe did it are = 5% so there is a 95%
20
chance that Joe is innocent. That’s good enough for me.
The ghost of the Reverend Thomas Bayes: You’re all
nuts!
The ghost of the Reverend Thomas Bayes: You’re all
nuts!
./figures/guilty.{ps,eps} not found (or no BBox)
If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If you think that Joe is guilty, then x = 0 and after

seeing
the evidence you still think Joe is guilty, P I M = 0.
If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If you think that Joe is guilty, then x = 0 and after

seeing
the evidence you still think Joe is guilty, P I M = 0.
If you think that Joe is innocent, then x = 1 and after
seeing
 the

evidence you still think Joe is innocent,
P I M = 1.
If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If you think that Joe is guilty, then x = 0 and after

seeing
the evidence you still think Joe is guilty, P I M = 0.
If you think that Joe is innocent, then x = 1 and after
seeing
 the

evidence you still think Joe is innocent,
P I M = 1.
If you think that Joe there is a 40% change that Joe is
guilty,
 then

x = 0.4 and after seeing the evidence
P I M = 0.0625.

If P(I) = x and so P(G) = 1 − x then

0.1 · x 0.1x
P I M = =
0.1 · x + 1 · (1 − x) 1 − 0.9x
If you think that Joe is guilty, then x = 0 and after

seeing
the evidence you still think Joe is guilty, P I M = 0.
If you think that Joe is innocent, then x = 1 and after
seeing
 the

evidence you still think Joe is innocent,
P I M = 1.
If you think that Joe there is a 40% change that Joe is
guilty,
 then

x = 0.4 and after seeing the evidence
P I M = 0.0625.

If you think that the police just searched a blood type
database until they came up with a name in the
neighborhood, then you should probably start with the
19
defense lawyer’s idea that x = P (I) = = 95%. Now after
20

0.095 0.095
seeing the evidence P I M = = = 0.66.
1 − 0.855 0.145
General Bayes’ Theorem

Let E1 , E2 , . . . , En be (pairwise) mutually exclusive events

such that E1 ∪ E2 ∪ · · · ∪ En = S, where S denotes the
sample space. Let F be an event such that P(F ) 6= 0, Then

P(E1 ∩ F )
P E1 F = =
P(F )
P(E1 ∩ F )
=
P(E1 ∩ F ) + P(E2 ∩ F ) + · · · + P(En ∩ F )


P(E1 )P F E1




P(E1 )P F E1 + P(E2 )P F E2 + · · · + P(En )P F En
General Bayes’ Theorem

Example A pile of 8 playing cards has 4 aces, 2 kings and

2 queens. A second pile of 8 playing cards has 1 ace, 4
kings and 3 queens. You conduct an experiment in which
you randomly choose a card from the first pile and place it
on the second pile. The second pile is then shuffled and you
randomly choose a card from the second pile. If the card
drawn from the second deck was an ace, what is the
probability that the first card was also an ace?
General Bayes’ Theorem
Let A be the event that you draw an ace, K the event that
you draw a king and Q be the event that you draw a queen.

mm A2
mmm
mmm
mmmmm
mmm
A QQQ K2
 QQQ
 QQQ
QQQ
 QQQ
 Q
 Q2



 A2
 m
 mmm
mmm
 mmm
 mmmmm
m
06 K QQQ K2
66 QQQ
66 QQQ
QQQ
66 QQQ
66 Q
66 Q2
66
66
66
66
66 m A2
66 mmm
66 mmm
6 mmmmm
m
mmm
Q QQQ K2
QQQ
QQQ
QQQ
QQQ
Q
Q2
General Bayes’ Theorem
In the first round there are 4 + 2 + 2 = 8 cards so the
probabilities in the first round are
m A2
mmm
mmm
mmm
mmm
mmm
A QQQ K2
QQQ
 QQQ
 QQQ
QQQ
 Q
 Q2

.5


 mm A2
 mmm
 mmm

mmmmm
 0.25 mmm
06 K QQQQ K2
66 QQQ
66 QQQ
66 QQQ
QQQ
66
66 Q2
66
6
0.25 666
66
66 A2
66 mmm
mmm
66 mmm
6 mmmmm
mm
Q QQQ K2
QQQ
QQQ
QQQ
QQQ
Q
Q2
General Bayes’ Theorem
In the second round there are 1 + 4 + 3 + 1 = 9 cards and
the probabilities are different at the various nodes. If you
draw an ace in round 1 the cards are 2 aces, 4 kings and 3
queens so we get
A2
2 mmm
9 mmm
mmm
mmmmm 4
mm 9
A QQQ K2
QQQ
 QQQ
QQQ
 QQQ

3
Q
 9
Q2

.5


 mmm
m A2
 mmm
 mmm
 mmm
 0.25 mmm
06 K QQQQ K2
66 QQQ
66 QQQ
66 QQQ
QQQ
66
66 Q2
66
6
0.25 666
66
66
mm A2
66 mmm
66 mmm
6 mmmmm
mmm
Q QQQ K2
QQQ
QQQ
QQQ
QQQ
Q
Q2
General Bayes’ Theorem
If you draw a king in round 1 the cards are 1 ace, 5 kings
and 3 queens so we get
A2
2 mmm
9 mmm
mmm
mmmmm 4
mm 9
A QQQ K2
 QQQ
 QQQ
QQQ
 3 QQQ
 Q

9
Q2

.5


 A2
 1 mmm
 9mmmmm
 mmmm5
m
 0.25 mmm 9
06 K QQQQ K2
66 QQQ
66 Q QQQ
66 3 QQQQ
66 9
Q
66 Q2
66
66
0.25 66
66
66
mm A2
66 mmm
66
mm mmm
6 mmm
mmm
Q QQQ K2
QQQ
QQQ
QQQ
QQQ
Q
Q2
General Bayes’ Theorem
If you draw a queen in round 1 the cards are 1 ace, 4 kings
and 4 queens so we get
A2
2 mmm
9 mmmm
mmm
mmm 4
mmm 9
A QQQ K2
QQQ
 QQQ
 QQQ
QQQ
 3
Q

9
Q2

.5


 1 mmm
A2
 9mmmmm

mmmmm5
 0.25 mmm 9
06 K QQQQ K2
66 QQQ
66 QQQ
66 QQQ
3 QQQ
66 9
66 Q2
66
66
0.25 66
66
66 A2
66 1 mmm
9 mmm
66 mmm
6 mmmmm 4
mm 9
Q QQQ K2
QQQ
QQQ
QQQ
4 QQQ
9 Q
Q2
 A2
General Bayes’ Theorem mmm
mmm
2
9 mmmm
mm4m
9
mmm

A QQQ K2
 QQQ
 QQQ
QQQ
 QQQ

3
Q
 9
Q2

.5


 A2
 1 mmm
 9 mmm
 mmm
 mmmmm 5
0.25 mm 9
06 K QQQ K2
66 QQQ
66 QQQ
QQQ
66 3 QQQ
66 9
Q
66 Q2
66
6
0.25 666
66
66 A2
66 1 mmm
66 9 mmmm
mmm
6 mmm 4
mmm 9
Q QQQ K2
QQQ
QQQ
QQQ
4 QQQ
9 Q
Q2

The question asks for

1

P(A ∩ A2) · 292
P AA2 = = 1 =
P(A2 ) 2
· 29 + · 19 + 41 ·
1
4
1
9
4
36 4 2
4 1 1 = = .
36
+ 36 + 36 6 3