Scribd
Upload
30 views

Lecture Notes - 5

This document provides an overview of Bayes' theorem including its statement, formula, derivation, examples and related terms. It defines Bayes' theorem as a method to determine the probability of an event based on prior events. The summary explains the key elements of the theorem and provides three examples for illustration.

Uploaded by

12112004it
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
30 views

Lecture Notes - 5

This document provides an overview of Bayes' theorem including its statement, formula, derivation, examples and related terms. It defines Bayes' theorem as a method to determine the probability of an event based on prior events. The summary explains the key elements of the theorem and provides three examples for illustration.

Uploaded by

12112004it
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Artificial Intelligence

Lecture Notes - 5
(Subject Code: CSC 602, ITC 651)

Prepared by
Dr. Sourabh Jain
Indian Institute of Information Technology
Sonepat
Bayes Theorem
Bayes theorem is a theorem in probability and statistics, named after the
Reverend Thomas Bayes, that helps in determining the probability of an event
that is based on some event that has already occurred. Bayes theorem has many
applications such as bayesian interference, in the healthcare sector - to
determine the chances of developing health problems with an increase in age
and many others.

What is Bayes Theorem?


Bayes theorem, in simple words, determines the conditional probability of an
event A given that event B has already occurred. Bayes theorem is also known
as the Bayes Rule or Bayes Law. It is a method to determine the probability of an
event based on the occurrences of prior events. It is used to calculate conditional
probability. Bayes theorem calculates the probability based on the hypothesis.

Now, let us state the theorem and its proof. Bayes theorem states that the
conditional probability of an event A, given the occurrence of another event B, is
equal to the product of the likelihood of B, given A and the probability of A. It is
given as:

Here, P(A) = how likely A happens (Prior knowledge)- The probability of a


hypothesis is true before any evidence is present.
P(B) = how likely B happens(Marginalization)- The probability of observing the
evidence.
P(A/B) = how likely A happens given that B has happened (Posterior)-The
probability of a hypothesis is true given the evidence.
P(B/A) = how likely B happens given that A has happened (Likelihood)- The
probability of seeing the evidence if the hypothesis is true.
Bayes Theorem - Statement
The statement of Bayes Theorem is as follows: Let E1, E2, E3,..., En be a set of
events associated with a sample space S, where all events E1, E2, E3,..., En have
non-zero probability of occurrence and they form a partition of S. Let A be any
event which occurs with E1orE2orE3...or En, then according to Bayes Theorem,

 Here Ei ∩ Ej = φ, where i ≠ j. (i.e) They are mutually exhaustive events


 The union of all the events of the partition, should give the sample space.
 0 ≤ P(Ei) ≤ 1

Bayes Theorem Formula


Bayes theorem formula exists for events and random variables. Bayes Theorem
formulas are derived from the definition of conditional probability. It can be
derived for events A and B, as well as continuous random variables X and Y.

Bayes Theorem Formula for Events


The formula for events derived from the definition of conditional probability is:
Derivation:
According to the definition of conditional probability,

Hence, the Bayes theorem formula for events is derived.

Bayes Theorem for Continuous Random


Variables
The formula for continuous random variables X and Y derived from the
definition of the conditional probability of continuous variables is:

Derivation:
According to the definition of conditional density or conditional probability of
continuous random variables, we know that:

Hence, the Bayes Theorem formula for random continuous variables is derived.
Difference Between Conditional Probability and
Bayes Theorem

Conditional Probability Bayes Theorem

Bayes Theorem is derived using the


Conditional Probability is the probability of
definition of conditional probability. The
an event A that is based on the occurrence
Bayes theorem formula includes two
of another event B.
conditional probabilities.

Terms Related to Bayes Theorem


Few terms related to the concept which have been used in the Bayes theorem
formula and derivation are the following:
 Conditional Probability - Conditional Probability is the probability of an
event A based on the occurrence of another event B. It is denoted by P(A|B)
and represents the probability of A given that event B has already
happened.
 Joint Probability - Joint probability measures the probability of two more
events occurring together and at the same time. For two events A and B, it
is denoted by P(A∩B).
 Random Variables - Random variable is a real-valued variable whose
possible values are determined by a random experiment. The probability
of such variables is also called the experimental probability.
 Posterior Probability - Posterior probability is the probability of an
event that is calculated after all the information related to the event has
been accounted for. It is also known as conditional probability.
 Prior Probability - Prior probability is the probability of an event that is
calculated before considering the new information obtained. It is the
probability of an outcome that is determined based on current knowledge
before the experiment is performed.
Bayes Theorem Examples:
Example 1: Amy has two bags. Bag I has 7 red and 2 blue balls and bag II has 5
red and 9 blue balls. Amy draws a ball at random and it turns out to be red.
Determine the probability that the ball was from the bag I using the Bayes
theorem.
Solution: Let X and Y be the events that the ball is from the bag I and bag II,
respectively. Assume A to be the event of drawing a red ball. We know that the
probability of choosing a bag for drawing a ball is 1/2, that is,
P(X) = P(Y) = 1/2

Since there are 7 red balls out of a total of 11 balls in the bag I, therefore,
P(drawing a red ball from the bag I) = P(A|X) = 7/11

Similarly, P(drawing a red ball from bag II) = P(A|Y) = 5/14


We need to determine the value of P(the ball drawn is from the bag I given that
it is a red ball), that is, P(X|A). To determine this we will use Bayes Theorem.
Using Bayes theorem, we have the following:

Answer: Hence, the probability that the ball is drawn is from bag I is 0.64
Example 2: Assume that the chances of a person having a skin disease are 40%.
Assuming that skin creams and drinking enough water reduces the risk of skin
disease by 30% and prescription of a certain drug reduces its chance by 20%. At
a time, a patient can choose any one of the two options with equal probabilities.
It is given that after picking one of the options, the patient selected at random
has the skin disease. Find the probability that the patient picked the option of
skin screams and drinking enough water using the Bayes theorem.
Solution: Assume E1: The patient uses skin creams and drinks enough water;
E2: The patient uses the drug; A: The selected patient has the skin disease
P(E1) = P(E2) = 1/2
Using the probabilities known to us, we have
P(A|E1) = 0.4 × (1-0.3) = 0.28
P(A|E2) = 0.4 × (1-0.2) = 0.32
Using Bayes Theorem, the probability that the selected patient uses skin creams
and drinks enough water is given by,

Answer: The probability that the patient picked the first option is 0.47
Example 3: A man is known to speak the truth 3/4 times. He draws a card and
reports it is king. Find the probability that it is actually a king.
Solution:
Let E be the event that the man reports that king is drawn from the pack of cards
A be the event that the king is drawn
B be the event that the king is not drawn.
Then we have P(A) = probability that king is drawn = 1/4
P(B) = probability that king is drawn = 3/4
P(E/A) = Probability that the man says the truth that king is drawn when
actually king is drawn = P(truth) = 3/4
P(E/B)= Probability that the man lies that king is drawn when actually king is
drawn = P(lie) = 1/4
Then according to Bayes theorem, the probability that it is
actually a king = P(A/E)

Answer: Thus the probability that the drawn card is actually a king = 0.5

You might also like