Unit – IV

Basics of Probability (Part 2)

 Bayes’ Theorem
• Revising probability when new information is obtained is an important
phase (discussed in Conditional probability).

• Often, we begin our analysis with initial or prior probabilities estimates for
specific events of interest.
• Then, from sources such as sample, a special report, a product test and so on
we obtain additional information about the events.
• Given this new information, we update the prior probability values by
calculating revised probabilities, referred to as posterior probabilities.
 Bayes’ Theorem
• We need sound method to compute revised or posterior probabilities.
• Bayes’ theorem gives us a way to do this.

• The steps in this probability revision process are shown in the following
diagram.

Posterior
Prior New Application of
Probabilities
Probabilities Information Bayes' Theorem
 Bayes’ Theorem (Thomas Bayes; 1763)

• If E1, E2, …, Ek are mutually disjoint event with P(Ei) ≠ 0,

(i =1, 2, 3, …, k), then for any arbitrary event A which is a subset
𝑘
of 1 𝐸𝑖 such that P(A) > 0, we have:

𝑃(𝐴∩𝐸𝑖 ) 𝑃 𝐸𝑖 . 𝑃 𝐴 𝐸𝑖
P(𝐸𝑖 |A)= = ; i =1, 2, 3, …, k
𝑖=1 𝑃(𝐸𝑖 ). 𝑃 𝐴 𝐸𝑖
𝑃(𝐴) 𝑘
 Bayes’ Theorem (Thomas Bayes; 1763)

𝑃(𝐴∩𝐸𝑖 ) 𝑃 𝐸𝑖 . 𝑃 𝐴 𝐸𝑖
P(𝐸𝑖 |A)= = ; i =1, 2, 3, …, k
𝑖=1 𝑃(𝐸𝑖 ). 𝑃 𝐴 𝐸𝑖
𝑃(𝐴) 𝑘

• P(E1), P(E2), …, P(Ek) are termed as ‘a prior probabilities’ (because they exist before we gain
information from the experiment itself).

• The probabilities; 𝑃 𝐴 𝐸𝑖 ; i =1, 2, 3, …, k are ‘likelihoods’ (because they indicate how likely
the event A under consideration is to occur, given each and every a prior probability.

• The probability 𝑃 𝐸𝑖 𝐴 ; i =1, 2, 3, …, k are called ‘posterior probabilities’ (because they

are determined after the results of the experiment are known).
 The Tabular Approach of Bayes’ Theorem

Step1: Prepare the following columns

a) The list of mutually exclusive events Ei; i = 1, 2, …, k; occur in the problem.
b) The prior probabilities P(Ei); i =1, 2, 3, …, k for the events.
c) Conditional probabilities of the new information (A) given each event. i. e.,
P(A/Ei); i = 1, 2, …, k.
d) Compute the joint probabilities for each event using the formula:
P(EiꓵA) = P(Ei).P(A/Ei)

Step2: Sum the joint probability column to find the probability of the new information;
i.e., P(A).

Step3: Compute the posterior probabilities using the basic relationship of conditional
𝑃(𝐸𝑖∩𝐴)
probability, i.e., P(Ei/A) = ; i = 1, 2, …, k.
𝑃(𝐴)
 Application of Bayes’ Theorem – Ex. 1
Consider a manufacturing firm that receives shipment of parts from two suppliers.
Let
A1 denote the event that a part is received from supplier 1
A2 is the event the part is received from supplier 2
 We get 65 percent of our parts
from supplier 1 and 35 percent
from supplier 2.

Thus:
P(A1) = 0.65 and P(A2) = 0.35
 Quality levels differ between suppliers
Percentage Percentage
Good Parts Bad Parts
Supplier 1 98 2
Supplier 2 95 5

Let G denote that a part is good and B denote the event that a part is bad.
Thus, we have the following conditional probabilities:

P(G|A1 ) = 0.98 and P(B|A1 ) = 0.02

P(G|A2 ) = 0.95 and P(B|A2 ) = 0.05

Tree Diagram for Two-Supplier Example
Step 2 Experimental
Step 1
Condition Outcome
Supplier
(A1, G)
G

A1
B
(A1, B)

A2 G (A2, G)

B
(A2, B)
Each of the experimental outcomes is the intersection of 2 events.

For example, the probability of selecting a part from supplier 1 that is

good is given by:

P( A1 , G )  P( A1  G )  P( A1 ).P(G | A1 )
Probability Tree for Two-Supplier Example
Step 2
Step 1 Probability of Outcome
Condition
Supplier
P(G | A1) P( A1  G )  P( A1 ) P(G | A1 )  0.6370
0.98
P(A1)
P(B | A1)

0.65 0.02 P( A1  B)  P( A1 ) P( B | A1 )  0.0130

P(G | A2)
P(A2) P( A2  G )  P( A2 ) P(G | A2 )  0.3325
0.95
0.35
P(B | A2)
P( A2  B)  P( A2 ) P(G | A2 )  0.0175
0.05
 A bad part broke in one of machines!
What is the probability the part came from suppler 1?

We know from the law of conditional probability that:

P( A1  B)
P( A1 | B) 
P( B)
Observe from the probability tree that:

P( A1  B)  P( A1 ) P( B | A1 )
 Bayes’ Theorem for 2 events

P( A1 ) P( B | A1 )
P( A1 | B) 
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )

P( A2 ) P( B | A2 )
P( A2 | B) 
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )
 P( A1 ) P( B | A1 )
P( A1 | B) 
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )
(0.65)(0.02) 0.0130
   0.4262
(0.65)(0.02)  (0.35)(0.05) 0.0305

P( A2 ) P( B | A2 )
P( A2 | B) 
P( A1 ) P( B | A1 )  P( A2 ) P( B | A2 )
(0.35)(0.05) 0.0175
   0.5738
(0.65)(0.02)  (0.35)(0.05) 0.0305
 Prior Conditional Joint Posterior
Events Probabilities Probabilities Probabilities Probabilities
A1 0.65 0.02 0.013 0.426229508
A2 0.35 0.05 0.0175 0.573770492
 Bayes’ Theorem

P( Ai ).P( B | Ai )
P( Ai | B) 
P( A1 ).P( B | A1 )  P( A2 ).P( B | A2 )  ...  P( An ) P( B | An )
 Ex. 2
• A factory produces a certain type of output by 3 types of machines.
The respective daily production figures are:
• Machine 1: 3000 units; Machine 2: 2500 units; Machine 3: 4500 units.
• Past experience shows that 1% of the output produced by Machine 1 is
defective.
• The corresponding fractions of defectives for the other two machines
are 1.2% and 2% respectively.
• An item is drawn at random from the day’s production run and is
found to be defective.
• What is the probability that it comes from the output of:
i) Machine 1; ii) Machine 2; iii) Machine 3
• Let E1, E2, and E3 denote the events that the output is produced by
machines, I, II, and III respectively.
• And, let A denote the event that the output is defective.
• Then, we have,
P (E1) = 3000/10000 = 0.30
P (E2) = 2500/10000 = 0.25
P (E3) = 4500/10000 = 0.45

Also, P (A|E1) = 1% = 0.01

P (A|E2) = 1.2% = 0.012
P (A|E3) = 2% = 0.02
• The probability that an item selected at random from day’s production is
defective is given by:
3
P (A) = 𝑖=1 𝑃 (𝐸𝑖 ∩ 𝐴)
3
= 𝑖=1 𝑃 (𝐸𝑖 ) . 𝑃 (𝐴|𝐸𝑖 )
= (0.30x0.01) + (0.25x0.012) + (0.45x0.0.2)
= 0.015
• By Bayes’ rule, the required probabilities are given by:

𝑃 𝐸1 . 𝑃(𝐴|𝐸1 ) 0.30𝑥0.01 0.003 1

i) P (𝐸1 |A) = = = =
𝑃(𝐴) 0.015 0.015 5

𝑃 𝐸2 . 𝑃(𝐴|𝐸2 ) 0.25𝑥0.012 0.003 1

ii) P (𝐸2 |A) = = = =
𝑃(𝐴) 0.015 0.015 5

𝑃 𝐸3 . 𝑃(𝐴|𝐸3 ) 0.45𝑥0.02 0.009 3

iii) P (𝐸3 |A) = = = =
𝑃(𝐴) 0.015 0.015 5
OR
The posterior probabilities can also be obtained in tabular form as given below:

Events Prior probabilities Conditional Joint Posterior

Ei P (Ei) probabilities probabilities probabilities
P (A|Ei) P (Ei  A) P (Ei|A)
(1) (2) (3) (4) = (2) x (3) (5) = (4) / P(A)
E1 0.30 0.010 0.003 0.003 1
=
0.015 5

E2 0.25 0.012 0.003 0.003 1

=
0.015 5

E3 0.45 0.020 0.009 0.009

=
3
0.015 5

Total 1.00 P(A) = 0.015 1

 Ex. 3
• The probabilities of X, Y, and Z becoming managers are 4/9, 2/9, and
1/3 respectively.

• The probabilities that the Bonus Scheme will be introduced if X, Y,

and Z become managers are 3/10, 1/2, and 4/5 respectively.

i) What is the probability that Bonus scheme will be introduced, and

ii) If the Bonus scheme has been introduced, what is the probability
that the manager appointed was X?
• Let E1, E2, and E3 denote the events that the X, Y, and Z become managers
respectively.
• And, let A denote the event that ‘Bonus scheme’ will be introduced.
• Then, we have,
P (E1) = 4/9
P (E2) = 2/9
P (E3) = 1/3

Also, P (A|E1) = 3/10

P (A|E2) = 1/2
P (A|E3) = 4/5
• The event A can materialize in the following mutually exclusive ways:
i) Mr. X becomes manager and bonus scheme is introduced.
i.e., E1 ∩ 𝐴 happens.
ii) E2 ∩ 𝐴 happens,
iii) E3 ∩ 𝐴 happens.

Thus, A = (E1 ∩ 𝐴)  (E2 ∩ 𝐴)  (E3 ∩ 𝐴) where (E1 ∩ 𝐴), (E2 ∩ 𝐴), and (E3 ∩ 𝐴)
are disjoint.
3
P (A) = 𝑖=1 𝑃 (𝐸𝑖 ∩ 𝐴)
3
= 𝑖=1 𝑃 (𝐸𝑖 ) . 𝑃 (𝐴|𝐸𝑖 )
= (4/9)x(3/10) + (2/9)x(1/2) + (1/3)x(4/5)
= 23/45
• By Bayes’ theorem, the required probability is:

𝑃 𝐸1 . 𝑃(𝐴|𝐸1 )
P (𝐸1 |A) =
𝑃(𝐴)
4 3
9
𝑥(10)
= 23
(45)
6
=
23
OR
The posterior probabilities can also be obtained in tabular form as given below:
Events Prior probabilities Conditional Joint Posterior
Ei P (Ei) probabilities probabilities probabilities
P (A|Ei) P (Ei  A) P (Ei|A)
(1) (2) (3) (4) = (2) x (3) (5) = (4) / P(A)
E1 4/9 3/10 12/90 6
23

E2 2/9 1/2 1/9 = 10/90 5

23
E3 1/3 4/5 4/15 = 24/90 12
23
Total 1.00 P(A) = 46/90 1
 Ex. 4
• A letter is known to have come either from TATANAGAR or from
CALCUTTA.

• On the envelope just two consecutive letters TA are visible.

• What is the probability that the letter came from CALCUTTA.

• Let E1, and E2 denote the events that the letter came from TATANAGAR
and CALCUTTA respectively.
• And, let A denote the event that two consecutive visible letters on the
envelop are TA.
• Then, we have,
P (E1) = 1/2
P (E2) = 1/2

Also, P (A|E1) = 2/8

P (A|E2) = 1/7
• By Bayes’ theorem, the required probability is:

𝑃 𝐸2 . 𝑃(𝐴|𝐸2 )
P (𝐸2 |A) =
𝑃(𝐴)
𝑃 𝐸2 . 𝑃(𝐴|𝐸2 )
=
𝑃 𝐸1 . 𝑃(𝐴|𝐸1 )+𝑃 𝐸2 . 𝑃(𝐴|𝐸2 )
1 1
𝑥( )
2 7
= 1 2 1 1
2
𝑥 8
+ 2
𝑥(7)
4
=
11
Thank you