NAÏVE BAYES MODELS
Naïve Bayes is comprised of 2 words Naïve and Bayes, Which can be
described as:
Naïve: It assumes that the occurrence of a certain feature is
independent of the occurrence of other features.
Eg: if the fruit is identified on the basis of color, shape and taste then
red, spherical, and sweet fruit is recognized as an apple. Here each
feature individually contributes to identify that it is an apple without
depending on each other.
Bayes: It depends on the principle of Bayes' Theorem.
NAÏVE BAYES MODELS
•Naïve Bayes- Supervised learning algorithm based on Bayes theorem.
•Used in text classification that includes a high-dimensional training dataset.
•Simple and most effective Classification algorithms which helps in building
the fast machine learning models that can make quick predictions.
•It is a probabilistic classifier, which means it predicts on the basis of the
probability of an object.
Examples - spam filtration, Sentimental analysis, and classifying articles.
NAÏVE BAYES MODELS
•Bayes‘ theorem/Bayes' Rule/Bayes' law, is used to determine the probability of a
hypothesis with prior knowledge. It depends on the conditional probability.
•The formula for Bayes' theorem is given as:
Where,
P(A|B) is Posterior probability: Probability of hypothesis A on the observed event B.
P(B|A) is Likelihood probability: Probability of the evidence given that the probability of a hypothesis is true.
P(A) is Prior Probability: Probability of hypothesis before observing the evidence.
P(B) is Marginal Probability: Probability of Evidence.
NAÏVE BAYES MODELS
Working of Naïve Bayes' Classifier:
The dataset is divided into two parts, namely, feature matrix and the response
vector.
Assumption:
The fundamental Naive Bayes assumption is that each feature makes an:
• independent
• equal
contribution to the outcome.
NAÏVE BAYES MODELS
Working of Naïve Bayes' Classifier:
Working of Naïve Bayes' Classifier can be understood with the help of the below
example:
Suppose we have a dataset of weather conditions and corresponding target
variable "Play". So using this dataset we need to decide that whether we should
play or not on a particular day according to the weather conditions.
So to solve this problem, we need to follow the below steps:
1.Convert the given dataset into frequency tables.
2.Generate Likelihood table by finding the probabilities of given features.
3.Now, use Bayes theorem to calculate the posterior probability.
NAÏVE BAYES MODELS
Problem: If the weather is sunny, then the Player should play or not?
Solution: To solve this, first consider the below dataset:
Outlook Play
0 Rainy Yes
1 Sunny Yes
2 Overcast Yes
3 Overcast Yes
4 Sunny No
5 Rainy Yes
6 Sunny Yes
7 Overcast Yes
8 Rainy No
9 Sunny No
10 Sunny Yes
11 Rainy No
12 Overcast Yes
AL3391/AI/II AI&DS/III SEM/KG-KiTE
13 Overcast Yes
NAÏVE BAYES MODELS
Frequency table for the Weather Conditions: Likelihood table weather condition:
Weather Weather No Yes
Yes No
Overcast
Overcast
Rainy
Rainy
Sunny Sunny
Total All
NAÏVE BAYES MODELS
Frequency table for the Weather Conditions: Likelihood table weather condition:
Weather Weather No Yes
Yes No
Overcast 0 5 5/14= 0.35
Overcast 5 0
Rainy 2 2 4/14=0.29
Rainy 2 2
Sunny 3 2 Sunny 2 3 5/14=0.35
Total 10 4 All 4/14=0.29 10/14=0.71
NAÏVE BAYES MODELS
P(Yes|Sunny)= P(Sunny|Yes)*P(Yes)/P(Sunny) P(No|Sunny)= P(Sunny|No)*P(No)/P(Sunny)
P(Sunny|Yes)= 3/10= 0.3 P(Sunny|NO)= 2/4=0.5
P(Sunny)= 0.35 P(No)= 0.29
P(Yes)=0.71 P(Sunny)= 0.35
So P(Yes|Sunny) = 0.3*0.71/0.35= 0.60 So P(No|Sunny)= 0.5*0.29/0.35 = 0.41
So as we can see from the above calculation that P(Yes|Sunny) > P(No|Sunny)
Hence on a Sunny day, Player can play the game.
NAÏVE BAYES MODELS
Advantages of Naïve Bayes Classifier:
•Fast and easy ML algorithms to predict a class of datasets.
•Used for Binary as well as Multi-class Classifications.
•Performs well in Multi-class predictions as compared to the other Algorithms.
•Most popular choice for text classification problems.
Disadvantages of Naïve Bayes Classifier:
•Naive Bayes assumes that all features are independent or unrelated, so it cannot learn the
relationship between features.
Applications of Naïve Bayes Classifier:
•Credit Scoring.
•Medical data classification.
•Real-time predictions because Naïve Bayes Classifier is an eager learner.
•Text classification such as Spam filtering and Sentiment analysis.
NAÏVE BAYES MODELS
Types of Naïve Bayes Model:
Multinomial Naive Bayes — These types of classifiers are usually used
for the problems of document classification. It checks whether the
document belongs to a particular category like sports or technology or
political etc and then classifies them accordingly. The predictors used for
classification in this technique are the frequency of words present in the
document.
Complement Naive Bayes — This is basically an adaptation of the
multinomial naive bayes that is particularly suited for imbalanced
datasets.
NAÏVE BAYES MODELS
Types of Naïve Bayes Model:
Bernoulli Naive Bayes — This classifier is also analogous to
multinomial naive bayes but instead of words, the predictors are
Boolean values. The parameters used to predict the class variable
accepts only yes or no values, for example, if a word occurs in the text or
not.
Out-of-Core Naive Bayes — This classifier is used to handle cases of
large scale classification problems for which the complete training
dataset might not fit in the memory.
Gaussian Naive Bayes — In a Gaussian Naive Bayes, the predictors
take a continuous value assuming that it has been sampled from a
Gaussian Distribution. It is also called a Normal Distribution.
NAÏVE BAYES MODELS- TRY YOURSELF
Consider a situation where you have 1000 fruits which are either
‘banana’ or ‘apple’ or ‘other’. These will be the possible classes of
the variable Y.
The data for the following X variables all of which are in binary (0
and 1):
•Long
•Sweet
•Yellow
AL3391/AI/II AI&DS/III SEM/KG-KiTE
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
The main agenda of the classifier is to predict if a given fruit is a ‘Banana’ or an
‘Apple’ or ‘Other’ when the three attributes(long, sweet and yellow) are known.
Consider a case where you’re given that a fruit is long, sweet and yellow and you
need to predict what type of fruit it is. This case is similar to the case where you
need to predict Y only when the X attributes in the training dataset are known.
You can easily solve this problem by using Naive Bayes.
The thing you need to do is to compute the 3 probabilities,i.e. the probability of
being a banana or an apple or other. The one with the highest probability will be
your answer.
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
NAÏVE BAYES MODELS- TRY YOURSELF
