Machine Learning
Naive Bayes classifiers
Dr. Maiwan Bahjat Abdulrazzaq
P(+) = 6/10 P(R) = 6/10
P(-) = 4/10 P(B) = 4/10
Introduction
• Naive Bayes is a probabilistic machine learning algorithm that is
commonly used for classification tasks is based on Bayes' theorem,
which describes the probability of an event based on prior knowledge
of conditions that might be related to the event.
• Naive Bayes assumes that all features are independent of each other,
which is where the "naive" comes from. This assumption allows the
algorithm to work efficiently with large datasets and to make
predictions with high accuracy, even with limited amounts of training
data
• Naive Bayes can be used for binary classification problems or for multi-
class classification problems.
Types of Naive Bayes classifiers
• 1. Gaussian Naive Bayes 2. Multinomial Naive Bayes 3. Bernoulli Naive
Bayes.
• Gaussian Naive Bayes is a type of computer program that can help us
figure out which category something belongs to. For example, it can
help us figure out whether an email is spam or not. It does this by
looking at different characteristics of the email, like the words used or
the length of the email
Naive Bayes Formula
• P(A|B) =
P(A|B) is the posterior of class (A, target) given prediction (B,
attributes).
P(A) is the prior probability of class.
P(B|A) is the likelihood which is the probability of predictor
given class.
P(B) is the prior probability of predictor (Marginal probability)
Play Tennis Example
Play
Day Outlook Temperature Humidity Wind
Tennis
D1 sunny Hot High Weak No
D2 sunny Hot High Strong No
D3 Overcast Hot High Weak Yes
D4 Rain Mild High Weak Yes
D5 Rain Cool Normal Weak Yes
D6 Rain Cool Normal Strong No
D7 Overcast Cool Normal Strong Yes
D8 sunny Mild High Weak No
D9 sunny Cool Normal Weak Yes
D10 Rain Mild Normal Weak Yes
D11 sunny Mild Normal Strong Yes
D12 Overcast Mild High Strong Yes
D13 Overcast Hot Normal Weak Yes
D14 Rain Mild High Strong No
Learning phase
Outlook Play=Yes Play=No Temperature Play=Yes Play=No
Sunny 2/9 3/5 Hot 2/9 2/5
Overcast 4/9 0/5 Mild 4/9 2/5
Rain 3/9 2/5 Cool 3/9 1/5
Humidity Play=Yes Play=No Wind Play=Yes Play=No
High 3/9 4/5 Strong 2/9 2/5
Normal 6/9 1/5 Weak 4/9 2/5
Test Phase
• X= (Outlook=sunny, Temperature=Cool, Humidity=High, Wind=Strong)?
• Let’s see how much achieved in the learning phrase
• P(Outlook= sunny | Play= Yes) = 2/9 • P(Outlook= sunny | Play= No) = 3/5
• P(Temperature = Cool | Play= Yes) = 3/9 • P(Temperature = Cool | Play= No) = 1/5
• P(Humidity= High | Play= Yes) = 3/9 • P(Humidity = High | Play= No) = 4/5
• P(Wind= Strong | Play= Yes) = 3/9 • P(Wind = Strong | Play= No) = 3/5
P(Play = yes) = 9/14 P(Play = No) = 5/14
P(Yes | X’) = [P(sunny | Yes) P(Cool | Yes) P(High | Yes) P(Strong | Yes) ] P(Play=Yes) = 0.0053
P(No | X’) = [P(sunny | No) P(Cool | No) P(High | No) P(Strong | No) ] P(Play= No) = 0.0206
Given the fact P(Yes| X’) < P(No| X’). we label X’ to be “NO”.
Advantage
1. Simplicity: Naive Bayes is a simple and easy-to-understand algorithm that requires minimal
tuning of parameters.
2. Speed: Naive Bayes is a fast algorithm that can scale well to large datasets with many features. It
can also handle high-dimensional data effectively.
3. Good performance on small datasets: Naive Bayes can perform well on small datasets with few
training examples because it makes strong assumptions about the conditional independence of
features, which can help to reduce the risk of overfitting.
Disadvantage
1. Feature independence assumption: The algorithm assumes that all input features are
independent, which may not be true in real-world scenarios.
2. Sensitivity to input data: Gaussian Naive Bayes can be sensitive to input data that contains
outliers or extreme values, which can lead to inaccurate predictions.
Applications of Naïve Bayes Algorithms
Recommendation System
• Text classification
• Real time prediction
• Spam Filtering
• Sentiment Analysis
