Calculating
probabilities of two
events
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Alexander A. Ramírez M.
CEO @ Synergy Vision
Independence
Given that A and B are events in a random experiment, the conditions for independence of A
and B are:
1. The order in which A and B occur does not a ect their probabilities.
2. If A occurs, this does not a ect the probability of B.
3. If B occurs, this does not a ect the probability of A.
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
Measuring a sample
Generate a sample that represents 1000 throws of two fair coin ips
from scipy.stats import binom
sample = binom.rvs(n=2, p=0.5, size=1000, random_state=1)
array([1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 2, 0, 1, 1, 1, 0, 0, 2, 2,...
Find repeated data
from scipy.stats import find_repeats
find_repeats(sample)
RepeatedResults(values=array([0., 1., 2.]), counts=array([249, 497, 254]))
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
FOUNDATIONS OF PROBABILITY IN PYTHON
Measuring a biased sample
Using biased_sample data generated, calculate the relative frequency of each outcome
from scipy.stats import relfreq
relfreq(biased_sample, numbins=3).frequency
array([0.039, 0.317, 0.644])
FOUNDATIONS OF PROBABILITY IN PYTHON
Joint probability calculation
Engine Gear box P_Eng_fail = 0.01
Fails 0.01 0.005 P_GearB_fail = 0.005
Works 0.99 0.995 P_both_fails = P_Eng_fail*P_GearB_fail
print(P_both_fails)
P (Engine f ails and Gear box f ails) =? 0.00005
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with cards
P (Jack or King) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with cards (Cont.)
P (Jack or King) = P (Jack) + ...
4
P (Jack or King) = + ...
52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with cards (Cont.)
P (Jack or King) = P (Jack) + P (King)
4 4
P (Jack or King) = +
52 52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with cards (Cont.)
P (Jack or King) = P (Jack) + P (King)
4 4 8 2
P (Jack or King) = + = =
52 52 52 13
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with cards (Cont.)
P (Jack or King) = P (Jack) + P (King)
4 4 8 2
P (Jack or King) = + = =
52 52 52 13
FOUNDATIONS OF PROBABILITY IN PYTHON
Probability of A or B
FOUNDATIONS OF PROBABILITY IN PYTHON
Probability of A or B (Cont.)
FOUNDATIONS OF PROBABILITY IN PYTHON
Probability of A or B (Cont.)
P (A or B) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Probability of A or B (Cont.)
P (A or B) = P (A) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Probability of A or B (Cont.)
P (A or B) = P (A) + P (B)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap
P (Jack or Heart) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap (Cont.)
P (Jack or Heart) = P (Jack) + ...
4
P (Jack or Heart) = + ...
52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap (Cont.)
P (Jack or Heart) = P (Jack) + P (Heart) ...
4 13
P (Jack or Heart) = + ...
52 52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap (Cont.)
P (Jack or Heart) = P (Jack) + P (Heart)...
4 13
P (Jack or Heart) = + ...
52 52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap (Cont.)
P (Jack or Heart) = P (Jack) + P (Heart) − P (Jack and Heart)
4 13 1
P (Jack or Heart) = + −
52 52 52
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A or B) with overlap (Cont.)
P (Jack or Heart) = P (Jack) + P (Heart) − P (Jack and Heart)
4 13 1 16 4
P (Jack or Heart) = + − = =
52 52 52 52 13
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) = P (A) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) = P (A) + P (B) ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) = P (A) + P (B) ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) = P (A) + P (B) − P (A and B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Diagram of P(A or B) (Cont.)
P (A or B) = P (A) + P (B) − P (A and B)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(Jack or Heart) calculation in Python
P_Jack = 4/52
P_Hearts = 13/52
P_Jack_n_Hearts = 1/52
P_Jack_or_Hearts = P_Jack + P_Hearts - P_Jack_n_Hearts
print(P_Jack_or_Hearts)
0.307692307692
FOUNDATIONS OF PROBABILITY IN PYTHON
Let's calculate
probabilities of two
events
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Conditional
probabilities
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Alexander A. Ramírez M.
CEO @ Synergy Vision
Dependent events
FOUNDATIONS OF PROBABILITY IN PYTHON
Dependent events (Cont.)
4
P (Jack) = ≃ 7.69%
52
FOUNDATIONS OF PROBABILITY IN PYTHON
Dependent events (Cont.)
3
P (Jack) = ≃ 5.88%
51
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability formula
P (A and B) = P (A)P (B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability formula (Cont.)
P (A and B) = P (A)P (B∣A)
P (A and B)
P (B∣A) =
P (A)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Red∣Jack) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Jack and Red)
P (Red∣Jack) =
P (Jack)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Jack and Red)
P (Red∣Jack) =
P (Jack)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Jack and Red) X
P (Red∣Jack) = = 4
P (Jack) 52
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Jack and Red) 52
P (Red∣Jack) = = 4
P (Jack) 52
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Jack and Red) 52 2 1
P (Red∣Jack) = = 4
= =
P (Jack) 52
4 2
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Jack and Red) 52 2 1
P (Red∣Jack) = = 4
= =
P (Jack) 52
4 2
FOUNDATIONS OF PROBABILITY IN PYTHON
P(Red | Jack) calculation in Python
P_Jack = 4/52
P_Jack_n_Red = 2/52
P_Red_given_Jack = P_Jack_n_Red / P_Jack
print(P_Red_given_Jack)
0.5
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability
P (Jack∣Red) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Red and Jack)
P (Jack∣Red) =
P (Red)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Red and Jack)
P (Jack∣Red) =
P (Red)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Red and Jack)
P (Jack∣Red) =
P (Red)
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
P (Red and Jack) X
P (Jack∣Red) = = 26
P (Red) 52
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Red and Jack) 52
P (Jack∣Red) = = 26
P (Red) 52
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Red and Jack) 52 2 1
P (Jack∣Red) = = 26
= =
P (Red) 52
26 13
FOUNDATIONS OF PROBABILITY IN PYTHON
Conditional probability (Cont.)
2
P (Red and Jack) 52 2 1
P (Jack∣Red) = = 26
= =
P (Red) 52
26 13
FOUNDATIONS OF PROBABILITY IN PYTHON
P(Jack | Red) calculation in Python
P_Red = 26/52
P_Red_n_Jack = 2/52
P_Jack_given_Red = P_Red_n_Jack / P_Red
print(P_of_Jack_given_Red)
0.0769230769231
FOUNDATIONS OF PROBABILITY IN PYTHON
Let's condition
events to calculate
probabilities
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Total probability law
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Alexander A. Ramírez M.
CEO @ Synergy Vision
Deck of cards example
FOUNDATIONS OF PROBABILITY IN PYTHON
Deck of cards example (Cont.)
P (F ace card) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Deck of cards example (Cont.)
P (F ace card) = P (Club and F ace card) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Deck of cards example (Cont.)
P (F ace card) = P (Club and F ace card) + P (Spade and F ace card) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Deck of cards example (Cont.)
P (F ace card) = P (Club and F ace card) + P (Spade and F ace card)+
P (Heart and F ace card) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Deck of cards example (Cont.)
P (F ace card) = P (Club and F ace card) + P (Spade and F ace card)+
P (Heart and F ace card) + P (Diamond and F ace card)
FOUNDATIONS OF PROBABILITY IN PYTHON
Face card example in Python
Total probability calculation, FC is Face card in the code
P_Club_n_FC = 3/52
P_Spade_n_FC = 3/52
P_Heart_n_FC = 3/52
P_Diamond_n_FC = 3/52
P_Face_card = P_Club_n_FC + P_Spade_n_FC + P_Heart_n_FC + P_Diamond_n_FC
print(P_Face_card)
The probability of a face card is:
0.230769230769
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) =?
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V 1 and D) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V 1 and D) + P (V 2 and D) + ...
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V 1 and D) + P (V 2 and D) + P (V 3 and D)
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V 1)P (D∣V 1) + P (V 2)P (D∣V 2) + P (V 3)P (D∣V 3)
FOUNDATIONS OF PROBABILITY IN PYTHON
Damaged parts example in Python
A certain electronic part is manufactured by three di erent vendors, V1, V2, and V3.
Half of the parts are produced by V1, and V2 and V3 each produce 25%. The probability of a
part being damaged given that it was produced by V1 is 1%, while it's 2% for V2 and 3% for V3.
What is the probability of a part being damaged?
FOUNDATIONS OF PROBABILITY IN PYTHON
Damaged parts example in Python (Cont.)
What is the probability of a part being damaged?
P_V1 = 0.5
P_V2 = 0.25
P_V3 = 0.25
P_D_g_V1 = 0.01
P_D_g_V2 = 0.02
P_D_g_V3 = 0.03
FOUNDATIONS OF PROBABILITY IN PYTHON
Damaged parts example in Python (Cont.)
We apply the total probability formula
P_Damaged = P_V1 * P_D_g_V1 + P_V2 * P_D_g_V2 + P_V3 * P_D_g_V3
print(P_Damaged)
The probability of being damaged is:
0.0175
FOUNDATIONS OF PROBABILITY IN PYTHON
Let's start using the
total probability law
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Bayes' rule
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
Alexander A. Ramírez M.
CEO @ Synergy Vision
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A and B) for independent events
P (A and B) = P (A)P (B)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A and B) for dependent events
P (A and B) = P (A)P (B)
P (A and B) = P (A)P (B∣A)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A and B) for dependent events (Cont.)
P (A and B) = P (A)P (B)
P (A and B) = P (A)P (B∣A)
P (B and A) = P (B)P (A∣B)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A and B) is equal to P(B and A)
P (A and B) = P (A)P (B)
P (A and B) = P (A)P (B∣A)
P (B and A) = P (B)P (A∣B)
FOUNDATIONS OF PROBABILITY IN PYTHON
P(A and B) is equal to P(B and A) (Cont.)
P (A and B) = P (A)P (B)
P (A and B) = P (A)P (B∣A)
P (B and A) = P (B)P (A∣B)
P (A)P (B∣A) = P (A and B) = P (B and A) = P (B)P (A∣B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Bayes' relation
P (A and B) = P (A)P (B)
P (A and B) = P (A)P (B∣A)
P (B and A) = P (B)P (A∣B)
P (A)P (B∣A) = P (A and B) = P (B and A) = P (B)P (A∣B)
P (A)P (B∣A) = P (B)P (A∣B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Bayes' rule
P (A)P (B∣A) = P (B)P (A∣B)
P (A)P (B∣A)
⟹ P (A∣B) =
P (B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability
P (D) = P (V1 and D) + P (V2 and D) + P (V3 and D)
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V1 and D) + P (V2 and D) + P (V3 and D)
P (V1 and D) = P (V1 )P (D∣V1 )
P (V2 and D) = P (V2 )P (D∣V2 )
P (V3 and D) = P (V3 )P (D∣V3 )
P (D) = P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Total probability (Cont.)
P (D) = P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Bayes' formula
P (A)P (B∣A)
P (A∣B) =
P (B)
FOUNDATIONS OF PROBABILITY IN PYTHON
Bayes' formula (Cont.)
Bayes' formula:
P (A)P (B∣A)
P (A∣B) =
P (B)
The probability of a part being from vendor i, given that it is damaged:
P (Vi )P (D∣Vi )
P (Vi ∣D) =
P (D)
FOUNDATIONS OF PROBABILITY IN PYTHON
Bayes' formula (Cont.)
Bayes' formula:
P (A)P (B∣A)
P (A∣B) =
P (B)
The probability of a part being from vendor i, given that it is damaged:
P (Vi )P (D∣Vi )
P (Vi ∣D) =
P (D)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
...
FOUNDATIONS OF PROBABILITY IN PYTHON
Visual representation of Bayes' rule (Cont.)
P (V1 )P (D∣V1 )
P (V1 ∣D) =
P (V1 )P (D∣V1 ) + P (V2 )P (D∣V2 ) + P (V3 )P (D∣V3 )
FOUNDATIONS OF PROBABILITY IN PYTHON
Factories and parts example in Python
A certain electronic part is manufactured by three di erent vendors, V1, V2, and V3.
Half of the parts are produced by V1, and V2 and V3 each produce 25%. The probability of a
part being damaged given that it was produced by V1 is 1%, while it's 2% for V2 and 3% for V3.
Given that the part is damaged, what is the probability that it was manufactured by V1?
FOUNDATIONS OF PROBABILITY IN PYTHON
Factories and parts example in Python (Cont.)
Given that the part is damaged, get the probability that it was manufactured by V1.
P_V1 = 0.5
P_V2 = 0.25
P_V3 = 0.25
P_D_g_V1 = 0.01
P_D_g_V2 = 0.02
P_D_g_V3 = 0.03
P_Damaged = P_V1 * P_D_g_V1 + P_V2 * P_D_g_V2 + P_V3 * P_D_g_V3
FOUNDATIONS OF PROBABILITY IN PYTHON
Factories and parts example in Python (Cont.)
P_V1_g_D = (P_V1 * P_D_g_V1) / P_Damaged # P(V1|D) calculation
print(P_V1_g_D)
A randomly selected part which is damaged is manufactured by V1 with probability:
0.285714285714
FOUNDATIONS OF PROBABILITY IN PYTHON
Let's exercise with
Bayes
F O U N D AT I O N S O F P R O B A B I L I T Y I N P Y T H O N
