Scribd
Upload
105 views18 pages

Naïve Bayes Classifier

Naive Bayes is a probabilistic classifier that assumes independence between features. It calculates the posterior probability of a class given features using Bayes' theorem, where the class prior, feature likelihoods, and evidence are estimated from the training data. Naive Bayes represents probabilities learned from data and makes predictions by selecting the class with the highest posterior. It is commonly used for text classification, spam filtering, and recommendations due to its simplicity and efficiency.

Uploaded by

d
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
105 views18 pages

Naïve Bayes Classifier

Naive Bayes is a probabilistic classifier that assumes independence between features. It calculates the posterior probability of a class given features using Bayes' theorem, where the class prior, feature likelihoods, and evidence are estimated from the training data. Naive Bayes represents probabilities learned from data and makes predictions by selecting the class with the highest posterior. It is commonly used for text classification, spam filtering, and recommendations due to its simplicity and efficiency.

Uploaded by

d
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Naïve Bayes Classifier

• What is a classifier?

A classifier is a machine learning model that is used to discriminate different


objects based on certain features.

• Principle of Naive Bayes Classifier:


A Naive Bayes classifier is a probabilistic machine learning model that’s used
for classification task.
Bayesian methods
The variable h is the class variables and D represents parameters of features.
D is given as:
D=(d1,d2.d3………,dn)
By substituting for D and expanding using the chain rule we get:
P(h|d1,d2,d3,…….,dn)=(P(h)P(d1,d2,…..dn)|h)/ P(d1,d2,…..,dn)
Posterior=(Prior X Likelihood)/Evidence
Assume that all features in d are mutually independent, conditional under the category h. Under
this assumption:

P(h,d1,…….,dn)= P(h)P(d1|h)P(d2|h)………P(dn|h)
=P(h)∏ P(di|h)
The conditional distribution over the class variable h is

P(h|d1,…….,dn)= (1/Z)P(h)∏ P(di|h)

Where Z is a scaling factor dependent only on d1, d2,……dn, that is a constant if the values of the
feature variables are known.
Properties of Bayes Classifier
• Incrementality
• Combines prior knowledge and observed data
• Probabilistic hypothesis
Representation Used By Naive Bayes Models
• The representation for naive Bayes is probabilities:
A list of probabilities are stored to file for a learned naive Bayes model. This
includes:
• Class Probabilities: The probabilities of each class in the training dataset.
• Conditional Probabilities: The conditional probabilities of each input value given
each class value.
Calculating Class Probabilities:
• a binary classification the probability of an instance belonging to class 1 would be calculated as:
P(class=1) = count(class=1) / (count(class=0) + count(class=1))

Calculating Conditional Probabilities:


• if a “weather” attribute had the values “sunny” and “rainy” and the class attribute had the class
values “go-out” and “stay-home“, then the conditional probabilities of each weather value for
each class value could be calculated as:
• P(weather=sunny|class=go-out) = count(instances with weather=sunny and class=go-out) /
count(instances with class=go-out)
• P(weather=sunny|class=stay-home) = count(instances with weather=sunny and class=stay-home)
/ count(instances with class=stay-home)
• P(weather=rainy|class=go-out) = count(instances with weather=rainy and class=go-out) /
count(instances with class=go-out)
• P(weather=rainy|class=stay-home) = count(instances with weather=rainy and class=stay-home) /
count(instances with class=stay-home)
Example:
Applications

• Real time Prediction


• Multi class Prediction
• Text classification/ Spam Filtering/ Sentiment Analysis
• Recommendation System
Conclusion
• Naive Bayes algorithms are mostly used in sentiment analysis, spam filtering, recommendation
systems etc. They are fast and easy to implement but their biggest disadvantage is that the
requirement of predictors to be independent.
Thank You

You might also like