Module No.

1
Probability

Lecture -2

1
Few terms
• Uncertainty
• Probability
• Simple Probability
• Joint Probability
• Conditional Probability
• Total Probability
• Reverse Probability

2
Uncertainty
• Uncertainty means working with imperfect or
incomplete information.
• Uncertainty is hard to bear for human beings.
• But in machine learning, there are certain algorithms that help to find
your way around this limitation.
• The Naive Bayes machine learning algorithm is one of the tools to
deal with uncertainty with the help of probabilistic methods.

3
Probability
• Probability is a field of math that enables us to reason about uncertainty
and assess the likelihood of some results or events.
• When you work with predictive ML modeling, you have to predict
uncertain future.
• For example, you may try to predict the performance of an Olympic
champion during the next Olympics based on past results.
• Even if they won before, it doesn’t mean they will win this time.
Unpredictable factors such as an argument at home this morning or no
time to have breakfast may or may not influence their results.
• Three main sources of uncertainty in machine learning are noisy data,
incomplete coverage of the problem, and imperfect models
4
Probability-examples
• A coin is thrown 3 times .what is the probability that atleast one head
is obtained?
Sol: Sample space = [HHH, HHT, HTH, THH, TTH, THT, HTT, TTT]
Total number of ways = 2 × 2 × 2 = 8. Fav. Cases = 7
P (A) = 7/8

5
 Mutually Exclusive
• When two events (call them "A" and "B") are Mutually Exclusive two events are said to be mutually exclusive if they
cannot occur at the same time or simultaneously:
• P(A and B) = 0
• "The probability of A and B together equals 0 (impossible)"
• Example: King AND Queen
• A card cannot be a King AND a Queen at the same time! The probability of a King and a Queen is 0 (Impossible)
• But, for Mutually Exclusive events, the probability of A or B is the sum of the individual probabilities:
• P(A or B) = P(A) + P(B)
• "The probability of A or B equals the probability of A plus the probability of B"
• Example: King OR Queen
• In a Deck of 52 Cards:
• the probability of a King is 1/13, so P(King)=1/13 the probability of a Queen is also 1/13, so P(Queen)=1/13

• When we combine those two Events: The probability of a King or a Queen is (1/13) + (1/13) = 2/13

• P(King or Queen) = (1/13) + (1/13) = 2/13

6
Not Mutually Exclusive

Probability of Hearts or Kings is:

all the Hearts (13 of them) + all the Kings (4 of them)

But that counts the King of Hearts twice!

So we correct our answer, by subtracting the extra "and" part:set hearts kings sum

16 Cards = 13 Hearts + 4 Kings − the 1 extra King of Hearts

Count them to make sure this works!

Formula : P(A or B) = P(A) + P(B) − P(A and B)

"The probability of A or B equals the probability of A plus the probability of B minus the
probability of A and B"

Here is the same formula, but using ∪ and ∩:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

7
Mutually exclusive vs Independent events
• Mutually exclusive events are those that cannot happen simultaneously,
whereas independent events are those whose probabilities do not affect one
another.
• Events A and B are said to be independent if the probability of B occurring
is unaffected by the occurrence of the event A happening.
• For example, now suppose that we are tossing a coin twice. Let A be the
event that the first coin toss lands on heads.
• In addition, let B be the event that the second coin toss lands on heads.
Clearly the result of the first coin toss does not affect the result of the
second coin toss.
P(A and B) = P(A) P(B)

8
Conditional Probability
• We can try to answer the question of what is the probability that an
athlete wins the race given their results for the past 10 years.
• This is how conditional probability is defined:
The probability of a, given b = the joint probability of both a and b
happening, divided by the probability of b.

9
Conditional Probability
Definition 8.2: Conditional Probability

If events are dependent, then their probability is expressed by conditional probability.

The probability that A occurs given that B is denoted by .

Suppose, A and B are two events associated with a random experiment. The
probability of A under the condition that B has already occurred and is given by

10
Conditional Probability
Corollary 8.1: Conditional Probability

or

For three events A, B and C

For n events A1, A2, …, An and if all events are mutually independent to each other

Note:
if events are mutually exclusive
if A and B are independent
otherwise,

11
Conditional Probability
• Generalization of Conditional Probability:

∵ P(A) = P(B)

By the law of total probability : P(B) =

12
Conditional Probability

In general,

13
Conditional Probability-Example
•A die is rolled twice and two numbers are obtained, let X be the outcome of first roll and Y be the
outcome of the second roll. Given that X+Y=5, what is the probability of X=4 or Y=4?
Assume, A be the event the getting 4 as X or Y, and B be the event of X+Y=5, therefore
A={(4,1), (4,2), (4, 3), (4,4), (4,5), (4,6), (1,4), (2,4), (3,4), (4,4), (5,4), (6,4)}
B={ (1,4),(4,1), (2, 3), (3,2)}

A⋂ B= {(1,4), (4, 1)}

We are interested in finding the probability of A given B

P(A⋂ B)= 2/ 36
As die is rolled out two times, total sample space= 36

So, P(A|B)= P(A⋂ B)/ P(B)

P(B)= 4/ 36

= 2/4, or
= 1/2. 14
Total Probability
• It is a useful way to find the probability of some event A when we
don’t directly know the probability of A but we do know that
events B1, B2, B3… form a partition of the sample space S.
• Let be n mutually exclusive and exhaustive events associated with a
random experiment. If A is any event which occurs with , then

15
Suppose there are two bags in a box, which contain the following marbles:
Bag 1: 7 red marbles and 3 green marbles
Bag 2: 2 red marbles and 8 green marbles
If we randomly select one of the bags and then randomly select one marble from that
bag, what is the probability that it’s a green marble?
let P(G) = probability of choosing a green marble.
P(G|B1) = 3/10 = 0.3
P(G|B2) = 8/10 = 0.8
Thus, using the law of total probability we can calculate the probability of choosing
a green marble as:
P(G) = ΣP(G|Bi)*P(Bi)
P(G) = P(G|B1)*P(B1) + P(G|B2)*P(B2)
P(G) = (0.3)*(0.5) + (0.8)*(0.5)
P(G) = 0.55
16
Inverse or Reverse Probability
Inverse Probability or Reverse Probability is the probability of an entity that is not taken into
account. In other words, more technically, the probability distribution of an unobserved entity.
The Inverse Probability can be found with Conditional Probability.

Eg:Let us consider two students in a class. Student 1 is having a Green bag (G) and Student 2 is
having a Blue bag (B).
Each bag consists of a countable number of notebooks.
The green bag consists of 3 short notebooks (S) and 9 long notebooks (L).
The blue bag consists of 5 short notebooks and (S) 6 long notebook (L).
Q: Find the probability or chance of getting a short notebook from the green bag given all
the short notebooks.

17
•Each bag consists of a countable number of notebooks.
•The green bag consists of 3 short notebooks (S) and 9 long notebooks (L).
•The blue bag consists of 5 short notebooks and (S) 6 long notebook (L).
• Probability of getting a notebook from green bag P(Green Bag) = 12 / 23
• Probability of getting a notebook from blue bag = 11/23
• P ( Green Bag|Short Notebook )=???
• P( Short Notebook ) = 8 / (12 + 11) = 8 / 23
• P( Short Notebook | Green Bag ) = 3 / 12

P( Green bag | Short Notebooks) = ((3 / 12) * (12 / 23)) / (8 / 23)= 3/8= 0.75
18
Marginal Probability:
• Marginal probability is defined as the probability of an
event A occurring independent of any other event B.
Further, it is considered as the probability of evidence
under any consideration.
• P(A) = P(A|B)*P(B) + P(A|~B)*P(~B)

Here ~B represents the event that B does not

occur. 19