0% found this document useful (0 votes)
4K views21 pages

Conditional Probability, Total Probability Theorem, Bayes Rule

Conditional probability provides a way to quantify the likelihood of an outcome A given that another outcome B has occurred. The conditional probability of A given B, written as P(A|B), is defined as the probability of A and B occurring together divided by the total probability of B occurring. Some examples include finding the probability of getting a 6 on the first roll of a die given that the sum of two rolls is 9, or the probability that a medical test is negative given that a person does not have a disease. The total probability theorem and multiplication rule allow us to calculate conditional probabilities in multi-step scenarios.

Uploaded by

Brian
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4K views21 pages

Conditional Probability, Total Probability Theorem, Bayes Rule

Conditional probability provides a way to quantify the likelihood of an outcome A given that another outcome B has occurred. The conditional probability of A given B, written as P(A|B), is defined as the probability of A and B occurring together divided by the total probability of B occurring. Some examples include finding the probability of getting a 6 on the first roll of a die given that the sum of two rolls is 9, or the probability that a medical test is negative given that a person does not have a disease. The total probability theorem and multiplication rule allow us to calculate conditional probabilities in multi-step scenarios.

Uploaded by

Brian
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

Conditional Probability (1/2)

• Conditional probability provides us with a way to reason


about the outcome of an experiment, based on partial
information
– Suppose that the outcome is within some given event B , we
wish to quantify the likelihood that the outcome also belongs
some other given event A

– Using a new probability law, we have the conditional probability


 
of A given B , denoted by P A B , which is defined as:

P A  B  A B
 
P AB 
P B 
• If PB  has zero probability, P A B  is undefined
• We can think of P A B  as out of the total probability of the
elements of B, the fraction that is assigned to possible
outcomes that also belong to A
Probability-Berlin Chen 2
Conditional Probability (2/2)

• When all outcomes of the experiment are equally likely,


the conditional probability also can be defined as

 
P AB 
number of elements of A  B
number of elements of B

• Some examples having to do with conditional probability


1. In an experiment involving two successive rolls of a die, you are told that
the sum of the two rolls is 9. How likely is it that the first roll was a 6?
2. In a word guessing game, the first letter of the word is a “t”. What is the
likelihood that the second letter is an “h”?
3. How likely is it that a person has a disease given that a medical test was
negative?
4. A spot shows up on a radar screen. How likely is it that it corresponds to
an aircraft?

Probability-Berlin Chen 3
Conditional Probabilities Satisfy the Three Axioms

• Nonnegative:
 
P AB 0

• Normalization:
P   B  P B 
 
PB 
P B 

P B 
1

• Additivity: If A1 and A2 are two disjoint events


P  A1  A 2   B 
P A1  
A2 B 
P B  distributive
P  A1  B    A 2  B 

A1 A2 P B 
P  A1  B   P  A 2  B  disjoint sets

B P B 
  
 P A1 B  P A 2 B  Probability-Berlin Chen 4
Conditional Probabilities Satisfy General
Probability Laws
• Properties probability laws
– P A1  A 2 B   P A1 B   P A 2 B 
–      
P A1  A2 B  P A1 B  P A2 B  P A1  A2 B  
– …

Conditional probabilities can also be viewed as a probability


law on a new universe B , because all of the conditional
probability is concentrated on B .

Probability-Berlin Chen 5
Simple Examples using Conditional Probabilities (1/3)

Probability-Berlin Chen 6
Simple Examples using Conditional Probabilities (2/3)

Probability-Berlin Chen 7
Simple Examples using Conditional Probabilities (3/3)

F SF FF
N
S SS FS
S F
C

Probability-Berlin Chen 8
Using Conditional Probability for Modeling (1/2)

• It is often natural and convenient to first specify


conditional probabilities and then use them to determine
unconditional probabilities

• An alternative way to represent the definition of


conditional probability
 
P  A  B   P B P A B

Probability-Berlin Chen 9
Using Conditional Probability for Modeling (2/2)

P A  B 


P A  Bc 


P Ac  B 

P Ac  B c 

Probability-Berlin Chen 10
Multiplication (Chain) Rule

• Assuming that all of the conditioning events have


positive probability, we have

     
P in1 Ai  P  A1 P A2 A1 P A3 A1  A2  P An in11Ai 
– The above formula can be verified by writing

 
P in1 Ai  P  A1 
P  A1  A 2  P  A1  A2  A 3  P 
 n
i 1 Ai 
P  A1  P  A1  A2 


P in11Ai 
– For the case of just two events, the multiplication rule is simply
the definition of conditional probability


P  A1  A2   P  A1 P A2 A1 
Probability-Berlin Chen 11
Multiplication (Chain) Rule: Examples (1/2)

• Example 1.10. Three cards are drawn from an


ordinary 52-card deck without replacement
(drawn cards are not placed back in the deck).
We wish to find the probability that none of the
three cards is a “heart”.

P  Ai   the ith card is not a heart, i  1,2,3

 
P  A1  A2  A3   P  A1 P A2 A1 P A3 A1  A2 
39 38 37
   C339
52 51 50 52
?
C3
Probability-Berlin Chen 12
Multiplication (Chain) Rule: Examples (2/2)

• Example 1.11. A class consisting of 4 graduate and 12


undergraduate students is randomly divided into 4 groups of 4. What
is the probability that each group includes a graduate student?

A1  graduate students 1 and 2 are at different groups


A2  graduate students 1 ,2, and 3 are at different groups
A3  graduate students 1 ,2, 3, and 4 are at different groups
 
P  A3   P  A1  A2  A3   P  A1 P A2 A1 P A3 A1  A2 
12
P  A1  
15 12


P A2 A1 
8
14
 8


P A3 A1  A2   4
13 4
12 8 4
 P  A3    
15 14 13
Probability-Berlin Chen 13
Total Probability Theorem (1/2)

• Let A1 ,, An be disjoint events that form a partition of the


sample space and assume that P Ai   0 , for all i .
Then, for any event B , we have

P B   P  A1  B     P  An  B 
 
 P  A1 P B A1    P  An P B An  
– Note that each possible outcome of the experiment (sample
space) is included in one and only one of the events A1 ,, An

Probability-Berlin Chen 14
Total Probability Theorem (2/2)

Figure 1.13:

Probability-Berlin Chen 15
Some Examples Using Total Probability Theorem (1/3)
Example 1.13.

Probability-Berlin Chen 16
Some Examples Using Total Probability Theorem (2/3)

Example 1.14.

(1,3),(1,4) (2,2),(2,3),(2,4) (4)

Probability-Berlin Chen 17
Some Examples Using Total Probability Theorem (3/3)
• Example 1.15. Alice is taking a probability class and at the end of
each week she can be either up-to-date or she may have fallen
behind. If she is up-to-date in a given week, the probability that she
will be up-to-date (or behind) in the next week is 0.8 (or 0.2,
respectively). If she is behind in a given week, the probability that
she will be up-to-date (or behind) in the next week is 0.4 (or 0.6,
respectively). Alice is (by default) up-to-date when she starts the
class. What is the probability that she is up-to-date after three weeks?
U i : up - to - date    
PU 3   PU 2 P U 3 U 2  PB2 P U 3 B2  PU 2   0.8  PB2   0.4
Bi : behind PU 2   PU1 P U 2 U1   PB1 P U 2 B1   PU 1   0.8  PB1   0.4
PB2   PB1 P B2 U1   PB1 P B2 B1   PU1   0.2  PB1   0.6
As we know that PU1   0.8 , PB1   0.2  PU 0   1.0 
 PU 2   0.8  0.8  0.2  0.4  0.72 Recursion formulea
P U i 1   P U i   0.8  P Bi   0.4
PB2   0.8  0.2  0.2  0.6  0.28
P Bi 1   P U i   0.2  P Bi   0.6
 PU 3   0.72  0.8  0.28  0.4  0.688 P U1   0.8, P B1   0.2

Probability-Berlin Chen 18
Bayes’ Rule

• Let A1, A2 ,, An be disjoint events that form a partition of


the sample space, and assume that P  Ai   0, for all i .
Then, for any event B such that P B   0 we have
P  Ai  B 
P Ai B  
P B  Multiplication rule

P  Ai P B Ai 

P B  Total probability theorem
P  Ai P B Ai 

 nk 1 P  Ak P B Ak 
P  Ai P B Ai 

P  A1 P B A1     P  An P B An 
Probability-Berlin Chen 19
Inference Using Bayes’ Rule (1/2)

惡性腫瘤

良性腫瘤

Figure 1.14:

Probability-Berlin Chen 20
Inference Using Bayes’ Rule (2/2)

• Example 1.18. The False-Positive Puzzle.


– A test for a certain disease is assumed to be correct 95% of the time: if
a person has the disease, the test with are positive with probability 0.95
( P B A   0 .95 ), and if the person does not have the disease, the test
results are negative with probability 0.95 ( P  B c A c   0 . 95 ). A random
 
person drawn from a certain population has probability 0.001
( P  A   0 . 001 ) of having the disease. Given that the person just tested
positive, what is the probability of having the disease ( P A B  ) ?

• A : the event that the person has a disease


• B : the event that the test results are positive


P A B  
P  A P B A 
P B 
P  B A c   1  P  B c A c   0 . 05

P  A P B A     
  

 
P  A P B A  P A c P B A c
0 . 001  0 . 95
  0 . 0187
0 . 001  0 . 95  0 . 999  0 . 05
Probability-Berlin Chen 21
Recitation

• SECTION 1.3 Conditional Probability


– Problems 11, 14, 15
• SECTION 1.4 Probability Theorem, Bayes’ Rule
– Problems 17, 23, 24, 25

Probability-Berlin Chen 22

You might also like