Scribd
Upload
18 views6 pages

Class 4 Naive Bayes Classification 2

Uploaded by

pratyushksingh2001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
18 views6 pages

Class 4 Naive Bayes Classification 2

Uploaded by

pratyushksingh2001
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 6

Bayes Theorem

The Bayes in Naïve Bayes comes from Bayes’ Theorem. If you paid attention to
probability and statistics in your maths class, there’s little chance you haven’t already
heard of Bayes’ theorem. Let’s recall it:

Bayes’ Theorem describes the probability of an event based on prior knowledge of the
conditions that might be related to the event.

This equation is derived from the formula of conditional probability given below:

Let’s take a silly little example – Say the likelihood of a person having Arthritis if they are
over 65 years of age is 49%.

Now, let’s assume the following:

 Class Prior: The probability of a person stepping in the clinic being >65-year-old is 20%
 Predictor Prior: The probability of a person stepping into the clinic having Arthritis is
35%

What is the probability that a person is >65 years given that he has Arthritis? This is Let’s
calculate this with the help of Bayes’ theorem!
In fact, Bayes’ Theorem is nothing but an alternate way of calculating the conditional
probability of an event. When the reverse conditional probability is easier to calculate
than the joint probability, Bayes’ Theorem is used.

You can use Bayes’ theorem to build a learner ML model, from an existing set of
attributes, that predicts the probability of the response variable belonging to some class,
given a new set of attributes.

Consider the previous equation again. Now, assume that event A is the response variable
and event B is the input attribute. So according to the equation,

 P(A) or Class Prior is the prior probability of the response variable


 P(B) or Predictor Prior is the evidence or the probability of training data
 P(A|B) or Posterior Probability is the conditional probability of the response variable
being of a particular value given the input attributes
 P(B|A) or Likelihood is basically the reverse of the posterior probability or the
likelihood of training data
Stepwise Bayes Theorem
Let’s come back to the problem at hand. Looks like you’re very serious with your
resolution this time given that you have been keeping track of the weather outside for the
past two weeks:

Step 1 – Collect raw data

Next, you need to create a frequency table for each attribute of your dataset.

Step 2 – Convert data to a frequency table(s)


Then, for each frequency table, you will create a likelihood table.

Step 3 – Calculate prior probability and evidence

Step 4 – Apply probabilities to Bayes’ Theorem equation

Let’s say you want to focus on the likelihood that you go for a run given that it’s sunny
outside.
P(Yes|Sunny) = P(Sunny|Yes) * P(Yes) / P(Sunny) = 0.625 * 0.571 / 0.428 = 0.834

The Naïve Bayes Algorithm


Naïve Bayes assumes conditional independence over the training dataset. The classifier
separates data into different classes according to the Bayes’ Theorem. But assumes that
the relationship between all input features in a class is independent. Hence, the model is
called naïve.

This helps in simplifying the calculations by dropping the denominator from the formula
while assuming independence:

Let’s understand this through our running resolution example:

Say you want to predict if on the coming Wednesday, given the following weather
conditions, should you go for a run or sleep in:

Outlook: Rainy

Humidity: Normal

Wind: Weak

Run: ?

Likelihood of ‘Yes’ on Wednesday:

P(Outlook = Rainy|Yes) * P(Humidity = Normal|Yes) * P(Wind = Weak|Yes) * P(Yes)


= 1/8 * 1/8 * 9/9 * 8/14 = 0.0089

Likelihood of ‘No’ on Wednesday:

P(Outlook = Rainy|No) * P(Humidity = Normal|No) * P(Wind = Weak|No) * P(No)

= 3/6 * 3/6 * 2/5 * 6/14 = 0.0428

Now, to determine the probability of going for a run on Wednesday, you just need to
divide P(Yes) with the sum of the likelihoods of Yes and No.

P(Yes) = 0.0089 / (0.0089 + 0.0428) = 0.172

Similarly, P(No) = 0.0428 / (0.0089 + 0.0428) = 0.827

According to the model, it looks like there’s an almost 83% probability that you’re going
to stay under the covers next Wednesday!

You might also like