Machine Learning
[email protected]
QN3DYCG1XF
Naive Bayes Classifier
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Naive Bayes Classification
• Will my flight be on time? It is Sunny, Hot, Normal Humidity, and not
Windy!
• Data from the last several times we took this flight
OUTLOOK TEMPERATURE HUMIDITY WINDY Flight On Time
Rainy Hot High 0 No
Rainy Hot High 1 Yes
Overcast Hot High 0 Yes
[email protected]QN3DYCG1XF Sunny Mild High 0 No
Sunny Cool Normal 0 Yes
Sunny Cool Normal 1 No
Overcast Cool Normal 1 Yes
Rainy Mild High 0 No
Rainy Cool Normal 0 Yes
Sunny Mild Normal 0 Yes
Rainy Mild Normal 1 Yes
Overcast Mild High 1 Yes
Overcast Hot Normal 0 Yes
Sunny Mild High 1 No
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Probability Review
• If A is any event, then the complement of A, denoted by , is
the event that A does not occur.
• The probability of A is represented by P(A), and the
probability of its complement P( ) = 1 – P(A).
• Let A and B be any events with probabilities P(A) and P(B).
[email protected]QN3DYCG1XF
• If you are told that B has occurred, then the probability of A
might change. The new probability of A is called the conditional
probability of A given B
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Probabilistic Independence
• Probabilistic independence means that knowledge of one
event is of no value when assessing the probability of the
other.
• The main advantage to knowing that two events are
independent is that in that case the multiplication rule
[email protected] simplifies to: P(A and B) = P(A) P(B).
QN3DYCG1XF
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Bayes’ Rule
• P(A|B), reads “A given B,” represents the probability of A if B
was known to have occurred.
• In many situations we would like to understand the relation
between P(A|B) and P(B|A).
[email protected]
QN3DYCG1XF
• You are planning an outdoor event tomorrow. When it
actually rains, the weatherman correctly forecasts rain 90%
of the time. When it doesn't rain, he incorrectly forecasts rain
10% of the time. Historically it has rained only 5 days each
year. Unfortunately, the weatherman has predicted rain for
tomorrow. What is the probability that it will rain tomorrow?
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Bayes’ Rule Continued
• Let A1 through An be a set of mutually exclusive outcomes.
• The probabilities of the As are P(A1) through P(An). These
are called prior probabilities.
• Because an information outcome might influence our
thinking about the probabilities of any Ai, we need to find the
[email protected]QN3DYCG1XF
conditional probability P(Ai|B) for each outcome Ai. This is
called the posterior probability of Ai.
• Using Bayes’ Rule:
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Bayes’ Rule Continued
• In words, Bayes’ rule says that the posterior is the likelihood
times the prior, divided by a sum of likelihoods times priors.
• The denominator in Bayes’ rule is the probability P(B).
[email protected]
QN3DYCG1XF
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
So will our flight be on time?
[email protected]
QN3DYCG1XF
This file is meant for personal use by [email protected] only.
Proprietary content. ©GreatSharing
Learning. All Rights
or publishing Reserved.
the contents Unauthorized
in part or full is liable foruse oraction.
legal distribution prohibited
Naïve Bayes Classifiers
• Probabilistic models based on Bayes’ theorem.
• It is called “naive” due to the assumption that the features in the
dataset are mutually independent
• In real world, the independence assumption is often violated, but naïve
Bayes classifiers still tend to perform very well
[email protected]QN3DYCG1XF
• Idea is to factor all available evidence in form of predictors into the
naïve Bayes rule to obtain more accurate probability for class
prediction
• It estimates conditional probability which is the probability that
something will happen, given that something else has already
occurred. For e.g. the given mail is likely a spam given appearance of
words such as “prize”
• fBeing relatively robust, easy to implement, fast, and accurate, naive
Bayes classifiers are used in many different fields
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by
[email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Naïve Bayes Classifiers - Pros and Cons
• Advantages
• Simple, Fast in processing and effective
[email protected]
• Does well with noisy data and missing data
QN3DYCG1XF
• Requires few examples for training (assuming the data set is a true
representative of the population)
• Easy to obtain estimated probability for a prediction
• Dis-advantages
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
Gaussian Naive Bayes classifier
• When some of our independent variables are continuous we cannot
calculate conditional probabilities!
• In Gaussian Naive Bayes, continuous values associated with each
feature (or independent variable) are assumed to be distributed
according to a Gaussian distribution
[email protected]
QN3DYCG1XF • All we would have to do is estimate the mean and standard deviation of
the continuous variable.
Proprietary content. ©Great Learning. All Rights Reserved. Unauthorized use or distribution prohibited
This file is meant for personal use by [email protected] only.
Sharing or publishing the contents in part or full is liable for legal action.
