Conditional Probability, Independent Events and Bay’s Rule

Conditional Probability: Let A and B be two events in a probability space (𝛺, 𝐴̃, 𝑃[. ]). The conditional
probability of event A given event B, denoted by 𝑃[𝐴|𝐵], is defined by
𝑃[𝐴𝐵]
𝑃[𝐴|𝐵] = , if 𝑃[𝐵] > 0.
𝑃[𝐵]

𝑃[𝐴𝐵]
Similarly, 𝑃[𝐵|𝐴] = , if 𝑃[𝐴] > 0 is the conditional probability of event B given event A.
𝑃[𝐴]

From the above relation, we see that 𝑃[𝐴𝐵] = 𝑃[𝐴|𝐵]𝑃[𝐵] = 𝑃[𝐵|𝐴]𝑃[𝐴],

where 𝑃[𝐴] > 0 𝑎𝑛𝑑 𝑃[𝐴] > 0.

Independent Events: Two events A and B defined on a probability space (𝛺, 𝐴̃, 𝑃[. ]) are said to be
independent if 𝑃[𝐴𝐵] = 𝑃[𝐴 ∩ 𝐵] = 𝑃[𝐴]𝑃[𝐵].

### Similarly, any n events 𝐴1 , 𝐴2 , … … … , 𝐴𝑛 defined on a probability space (𝛺, 𝐴̃, 𝑃[. ]) are said to be
independent if and only if the following conditions are satisfied:

𝑃[𝐴𝑖 𝐴𝑗 ] = 𝑃[𝐴𝑖 ]𝑃[𝐴𝑗 ], 𝑓𝑜𝑟 𝑖 ≠ 𝑗.

𝑃[𝐴𝑖 𝐴𝑗 𝐴𝑘 ] = 𝑃[𝐴𝑖 ]𝑃 [𝐴𝑗 ]𝑃[𝐴𝑘 ] 𝑓𝑜𝑟 𝑖 ≠ 𝑗, 𝑖 ≠ 𝑘, 𝑗 ≠ 𝑘.

:
:
𝑃[𝐴1 𝐴2 … … 𝐴𝑛 ] = 𝑃[𝐴1 ]𝑃[𝐴2 ] … … 𝑃[𝐴𝑛 ].
Theorem: Show that the following conditions are equivalent:
a. 𝑃[𝐴𝐵] = 𝑃[𝐴]𝑃[𝐵],
b. 𝑃[𝐴|𝐵] = 𝑃[𝐴] , if 𝑃[𝐵] > 0.
c. 𝑃[𝐵|𝐴] = 𝑃[𝐵] , if 𝑃[𝐴] > 0.
Proof. First we prove that (a)⟹(b).
Let 𝑃[𝐴𝐵] = 𝑃[𝐴]𝑃[𝐵] (1)
𝑃[𝐴𝐵]
By definition 𝑃[𝐴|𝐵] = (2)
𝑃[𝐵]

From (1) and (2), 𝑃[𝐴|𝐵] = 𝑃[𝐴] . Hence (a)⟹(b).

Next we show that (b)⟹(c).

Let 𝑃[𝐴|𝐵] = 𝑃[𝐴] (3)

By definition of conditional probability, we have
𝑃[𝐴𝐵] = 𝑃[𝐴|𝐵]𝑃[𝐵] = 𝑃[𝐵|𝐴]𝑃[𝐴]. (4)
Therefore, 𝑃[𝐵|𝐴]𝑃[𝐴] = 𝑃[𝐴]𝑃[𝐵], using (3) ⟹ 𝑃[𝐵|𝐴] = 𝑃[𝐵].
Hence (b) ⟹ (c).
Lastly, we show that (c) ⟹ (a).
𝑃[𝐵|𝐴] = 𝑃[𝐵], 𝑤ℎ𝑒𝑟𝑒 𝑃[𝐴] > 0. (5)
From (4) and (5), we have 𝑃[𝐴𝐵] = 𝑃[𝐴]𝑃[𝐵] 𝑓𝑜𝑟 𝑃[𝐴] > 0.
If 𝑃[𝐴] = 0, then 𝑃[𝐴]𝑃[𝐵] = 0 and by (4), 𝑃[𝐴𝐵] = 0.
Hence 𝑃[𝐴𝐵] = 𝑃[𝐴]𝑃[𝐵].
Theorem: If A and B are independent events, then

i. 𝐴 and 𝐵̅ are independent.

ii. 𝐴̅ and 𝐵 are independent and
iii. 𝐴̅ and 𝐵̅ are independent.
Proof: (Self)
1
Example 1. If A and B are independent and 𝑃[𝐴] = 𝑃[𝐵] = 2 , what is 𝑃[𝐴𝐵̅ ∪ 𝐴̅𝐵]?

Solution: Since A and B are independent, therefore all 𝐴, 𝐵, 𝐴̅ and 𝐵̅ are independent.
1 1 1
Consequently, 𝑃[𝐴𝐵̅] = 𝑃[𝐴]𝑃[𝐵̅] = 2 (1 − 2) = 4 ,

1 1 1
𝑃[𝐴̅𝐵] = 𝑃[𝐴̅]𝑃[𝐵] = (1 − 2) 2 = 4 .
1 1 1
Now 𝑃[𝐴𝐵̅𝐴̅𝐵] = 𝑃[𝐴𝐵̅] + 𝑃[𝐴̅𝐵] − 𝑃[𝐴𝐵̅ ̅𝐴𝐵] = 4 + 4 − 𝑃[𝜑] = 2 , as 𝐴 ̅𝐴 = 𝜑 𝑎𝑛𝑑 𝐵̅𝐵 = 𝜑.

Example 2. Five persons of the people have high blood pressure. Of the people with high blood pressure,
75% drink alcohol; whereas, only 50% of the people without high blood pressure drink alcohol. What
percent of the drinkers have high blood pressure?
Solution: Let 𝐴 denote the event that people have high blood pressure and 𝐵 denote the people who drink
alcohol.
We have 𝑃[𝐴] = 0.05, 𝑃[𝐵|𝐴] = 0.75, 𝑃[𝐵| ̅𝐴] = 0.50.
𝑃[𝐴∩𝐵]
We have to find 𝑃[𝐴|𝐵] = . (1)
𝑃[𝐵]

We know 𝐵 = 𝐴𝐵 ∪ ̅𝐴𝐵 and 𝐴𝐵 ∩ ̅𝐴𝐵 = 𝜑

⟹ 𝑃[𝐵] = 𝑃[𝐴𝐵] + 𝑃[ ̅𝐴𝐵] = 𝑃[𝐴 ∩ 𝐵] + 𝑃[ ̅𝐴 ∩ 𝐵]. (2)

𝑃[𝐵∩𝐴]
Now 𝑃[𝐵|𝐴] = 0.75 ⟹ = 0.75 ⟹ P[A ∩ B] = 0.75 × 0.05 = 0.0375 (3)
𝑃[𝐴]

𝑃[𝐵∩𝐴̅] 1
Also 𝑃[𝐵|𝐴̅] = 0.50 ⟹ 𝑃[𝐴̅]
= 0.50 ⟹ 𝑃[𝐵 ∩ 𝐴̅] = 2 (1 − 𝑃[𝐴])

1
⟹ 𝑃[𝐵 ∩ 𝐴̅] = 𝑃[𝐵] − 𝑃[𝐴 ∩ 𝐵] = 2 (1 − 0.05) = 0.475. by (2)

Therefore, 𝑃[𝐵] = 0.0375 + 0.475 = 0.5125, by (3).

 𝑃[𝐴∩𝐵] 0.0375 3
Hence, by (1), 𝑃[𝐴|𝐵] = 𝑃[𝐵]
= 0.5125 = 41 = 0.073.

Hence the required percentage is 73%.

Exercises:

1. If 𝑃[𝐴] = 𝑃[𝐵] = 𝑃[𝐵|𝐴] = 0.5, are A and B are independent?

2. If 𝑃[𝐴] = 𝑎, 𝑃[𝐵] = 𝑏, then show that 𝑃[𝐴|𝐵] ≥ (𝑎 + 𝑏 − 1)/𝑏.
3. Suppose A and B are events for which 𝑃[𝐴] = 𝑝1 , 𝑃[𝐵] = 𝑝2 , and 𝑃[𝐴 ∩ 𝐵] = 𝑝3 . Evaluate:
(a) 𝑃[𝐴̅ ∩ 𝐵], (b) 𝑃[𝐴̅ ∪ 𝐵], (c) 𝑃[𝐴 ∩ 𝐵̅], (d) 𝑃[𝐴̅ ∩ 𝐵̅],
̅̅̅̅̅̅̅
(e) 𝑃[𝐴 ∩ 𝐵], ̅̅̅̅̅̅̅
(f) 𝑃[𝐴 ∪ 𝐵], (g) 𝑃[𝐴̅ ∪ 𝐵̅], (h) 𝑃[𝐴|𝐵],

(i) 𝑃[𝐵|𝐴̅], (j) 𝑃[𝐴̅|𝐵̅], (k) 𝑃[𝐴̅ ∩ (𝐴 ∪ 𝐵)], (l) 𝑃[𝐴 ∪ (𝐴̅ ∩ 𝐵)].
(Ans. 𝑝2 − 𝑝3 , 1 − 𝑝1 + 𝑝3 , 𝑝1 − 𝑝3 , 1 − 𝑝3 , 1 − 𝑝3 , 1 − 𝑝1 − 𝑝2 + 𝑝3 , 1 − 𝑝1 − 𝑝2 + 𝑝3 ,
𝑝3 ⁄𝑝2 , (𝑝2 − 𝑝3 )⁄(1 − 𝑝1 ), (1 − 𝑝1 − 𝑝2 + 𝑝3 )⁄(1 − 𝑝2 ), 𝑝2 − 𝑝3 , 𝑝1 + 𝑝2 − 𝑝3 respectively.)

4. Suppose an urn contains 𝑀 balls of which 𝐾 are black and 𝑀 − 𝐾 are white. A sample of size 𝑛 is
drawn with replacement. Find the probability that the 𝑗-th ball drawn is black given that the sample
contains 𝐾 black balls. (Ans. 𝐾/𝑛)
Theorem of Total Probability:

Let 𝐵1 , 𝐵2 , … … … … , 𝐵𝑛 be a collection mutually disjoint events in the probability space (𝛺, 𝐴̃, 𝑃[. ]) such
that 𝛺 = ⋃𝑛𝑗=1 𝐵𝑗 and 𝑃[𝐵𝑗 ] > 0, 𝑗 = 1, 2, … … , 𝑛.

Then 𝑃[𝐴] = ∑𝑛𝑗=1 𝑃[𝐴|𝐵𝑗 ]𝑃[𝐵𝑗 ] , 𝑓𝑜𝑟 𝑒𝑎𝑐ℎ 𝐴 ∈ 𝐴̃.

Proof: We have 𝐴 = 𝐴 ∩ 𝛺 = 𝐴 ∩ (⋃𝑛𝑗=1 𝐵𝑗 ) = ⋃𝑛𝑗=1(𝐴 ∩ 𝐵𝑗 ) = ⋃𝑛𝑗=1 𝐴𝐵𝑗 .

Since 𝐵𝑗 ’s are mutually disjoint, therefore 𝐴𝐵𝑗 ’s are mutually disjoint.

Therefore, 𝑃[𝐴] = 𝑃[⋃𝑛𝑗=1 𝐴𝐵𝑗 ] = ∑𝑛𝑗=1 𝑃[𝐴𝐵𝑗 ]. (1)

By definition, 𝑃[𝐴𝐵𝑗 ] = 𝑃[𝐴|𝐵𝑗 ]𝑃[𝐵𝑗 ]. (2)

From (1) and (2), we get 𝑃[𝐴] = ∑𝑛𝑗=1 𝑃[𝐴|𝐵𝑗 ]𝑃[𝐵𝑗 ].

Corollary: If 𝐴, 𝐵 ∈ 𝐴̃; then 𝑃[𝐴] = 𝑃[𝐴|𝐵]𝑃[𝐵] + 𝑃[𝐴|𝐵̅]𝑃[𝐵̅], 𝑃[𝐵] > 0.

Proof: We have 𝛺 = 𝐵 ∪ 𝐵̅ , where 𝐵 and 𝐵̅ are mutually disjoint.

Hence by the above theorem, 𝑃[𝐴] = 𝑃[𝐴|𝐵]𝑃[𝐵] + 𝑃[𝐴|𝐵̅]𝑃[𝐵̅], 𝑃[𝐵] > 0.

Bay’s Theorem:

Let 𝐵1 , 𝐵2 , … … … … , 𝐵𝑛 be a collection mutually disjoint events in the probability space (𝛺, 𝐴̃, 𝑃[. ]) such
that 𝛺 = ⋃𝑛𝑗=1 𝐵𝑗 and 𝑃[𝐵𝑗 ] > 0, 𝑗 = 1, 2, … … , 𝑛.

Then for each 𝐴 ∈ 𝐴̃ satisfying 𝑃[𝐴] > 0, we have

𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ]
𝑃[𝐵𝑘 |𝐴] = ∑𝑛 , this is known as Bay’s formula.
𝑗=1 𝑃[𝐴|𝐵𝑗 ]𝑃[𝐵𝑗 ]

Proof: By the definition of conditional probability, we have

𝑃[𝐵𝑘 𝐴] 𝑃[𝐴𝐵𝑘 ]
𝑃[𝐵𝑘 |𝐴] = and 𝑃[𝐴|𝐵𝑘 ] = with 𝑃[𝐴] > 0 𝑎𝑛𝑑 𝑃[𝐵𝑘 ] > 0 (1)
𝑃[𝐴] 𝑃[𝐵𝑘 ]

𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ]
Using these two, we obtain 𝑃[𝐵𝑘 |𝐴] = 𝑃[𝐴]
. (2)

By the theorem of total probability, we have

𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ]
𝑃[𝐵𝑘 |𝐴] = ∑𝑛 .
𝑗=1 𝑃[𝐴|𝐵𝑗 ]𝑃[𝐵𝑗 ]

Hence proved.
𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ]
Corollary: If 𝐴, 𝐵 ∈ 𝐴̃; then 𝑃[𝐵|𝐴] = 𝑃[𝐴|𝐵]𝑃[𝐵]+𝑃[𝐴|𝐵
̅ ]𝑃[𝐵̅]
, 𝑃[𝐵] > 0.

Proof: We have 𝛺 = 𝐵 ∪ 𝐵̅ , where 𝐵 and 𝐵̅ are mutually disjoint.

𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ]
Hence by the Bay’s theorem, 𝑃[𝐵|𝐴] = 𝑃[𝐴|𝐵]𝑃[𝐵]+𝑃[𝐴|𝐵
̅ ]𝑃[𝐵̅]
, 𝑃[𝐵] > 0.
 1
Example 1. Suppose 𝐵1 , 𝐵2 and 𝐵3 are mutually exclusive events. If 𝑃[𝐵𝑘 ] = 3 𝑎𝑛𝑑 𝑃[𝐴|𝐵𝑘 ] =
𝑘
6
𝑓𝑜𝑟 𝑘 = 1, 2, 3. What is 𝑃[𝐴]?

Solution: By the theorem of total probability, we have

3 3
𝑘 1
𝑃[𝐴] = ∑ 𝑃[𝐴|𝐵𝑘 ]𝑃[𝐵𝑘 ] = ∑ × = 1/3.
6 3
𝑘=1 𝑘=1

Example 2. The probability that a person can hit the target is 3/5 and the probability that another person
can hit the same target is 2/5. But the first person can fire 8 shots in a given time while the second person
fires 10 shots. They fire together. What is the probability that the second person shoots the target?
Solution: Let E denote the event of shooting the target, E1 and E2 respectively denote the events that the
first person and the second person shoot the target, we are given
3 2
𝑃[𝐸|𝐸1 ] = 𝑎𝑛𝑑 𝑃[𝐸|𝐸2 ] = .
5 5
The ratio of the shots of the first person to those of the second person in the same time is 8/10 = 4/5. Thus
4
𝑃[𝐸1 ] = 𝑃[𝐸2 ]. By Bay’s theorem we get
5

2
𝑃[𝐸|𝐸2 ]𝑃[𝐸2 ] 𝑃[𝐸2 ]
𝑃[𝐸2 |𝐸] = = 5
𝑃[𝐸|𝐸1 ]𝑃[𝐸1 ] + 𝑃[𝐸|𝐸2 ]𝑃[𝐸2 ] 3 × 4 𝑃[𝐸 ] + 2 𝑃[𝐸 ]
5 5 2 5 2

5
𝑃[𝐸2 |𝐸] = .
11

Example 3. An urn contains 10 white and three black balls, while another urn contains 3 white and 5 black
balls. Two balls are drawn from the first urn and put into the second urn and then a ball is drawn from the
latter. What is the probability that it is a white ball?
Solution: The two balls are drawn from the first urn may be:
(i) both white or (ii) both black or (iii) one white and one black.
Let these events be denoted by A, B C respectively. Then
10 3 10 3
.𝐶2 15 .𝐶2 1 .𝐶1 .𝐶1 10
𝑃[𝐴] = 13𝐶
= , 𝑃[𝐵] = 13𝐶
= , 𝑃[𝐶] = 13𝐶
= .
. 2 26 . 2 26 . 2 26
When two balls are transferred from the first urn to the second urn, the second urn contains
(i) 5 white and 5 black balls or (ii) 3 white and 7 black balls or (iii) 4 white and 6 black balls.
Let W denotes the event of the drawing a white ball from the second urn in the above three cases. Then
5 3 4
𝑃[𝑊|𝐴] = , 𝑃[𝑊|𝐵] = , 𝑃[𝑊|𝐶] = .
10 10 10
5 15 3 1 4 10 59
Hence 𝑃[𝑊] = 𝑃[𝑊|𝐴]𝑃[𝐴] + 𝑃[𝑊|𝐵]𝑃[𝐵] + 𝑃[𝑊|𝐶]𝑃[𝐶] = 10 × 26 + 10 × 26 + 10 × 26 = 130 .
Exercises:
1. An urn contains 𝑎 white and 𝑏 black balls, while another urn contains 𝑐 white and 𝑑 black balls. One
ball is transferred from the first urn and put into the second urn and then a ball is drawn from the latter.
𝑎𝑐+𝑏𝑐+𝑎
What is the probability that it will be a white ball? Ans. (𝑎+𝑏)(𝑐+𝑑+1)
2. Three urns 𝐴1 , 𝐴2 , 𝐴3 contain respectively 3 red, 4 white, 1 blue; 1 red, 2 white, 3 blue; 4 red, 3 white,
2 blue balls. One urn is chosen at random and a ball is withdrawn. It is found to be red. Find the
probability that it comes from the urn 𝐴2 . Ans. 12/71.
3. An insurance company insured 2000 scooter drivers, 4000 car drivers and 6000 truck drivers. The
probability of any accident involving a scooter, a car and a truck are 0.01, 0.03 and 0.15 respectively.
One of the insured person meets with an accident. What is the probability that he is a scoter driver?
Ans. 1/52.
4. In a bolt factory machines A, B, C manufacture respectively 25, 35 and 40 percent of the total. Out of
their output 5, 4 and 2 percent ate defective bolts. A bolt is drawn from the produce and is found
defective. What is the probabilities that it was manufactured by A, B and C. (Ans. 25/69, 28/69, 16/69.)
5. Suppose that in answering a question in a multiple choice test, an examinee knows the answer with
probability 𝑝 or he guesses with probability 1 − 𝑝. Assume that the probability of answering a question
correctly is unity for an examinee who knows the answer and 1/𝑚 for the examinee who guesses,
where 𝑚 is the number of multiple choice alternatives. Show that the probability that an examinee
𝑚𝑝
knows the answer to a problem, given that he has correctly answered it, is 1+(1−𝑚)𝑝 .