Scribd
Upload
31 views

Machine Learning Department of Electrical Engineering

The document discusses naive Bayes classifiers and their use in machine learning. It introduces naive Bayes, describing how it makes assumptions of conditional independence between features. It then provides examples of how to calculate probabilities for naive Bayes classification using maximum likelihood estimation and maximum a posteriori estimation. Finally, it discusses extensions of naive Bayes for continuous features using Gaussian distributions, and compares naive Bayes to logistic regression classification.

Uploaded by

champion 88
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
31 views

Machine Learning Department of Electrical Engineering

The document discusses naive Bayes classifiers and their use in machine learning. It introduces naive Bayes, describing how it makes assumptions of conditional independence between features. It then provides examples of how to calculate probabilities for naive Bayes classification using maximum likelihood estimation and maximum a posteriori estimation. Finally, it discusses extensions of naive Bayes for continuous features using Gaussian distributions, and compares naive Bayes to logistic regression classification.

Uploaded by

champion 88
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Machine Learning Department of Electrical Engineering

Summary of Last Lecture

• Bayes Classifiers to learn P(Y|X)

• MLE & MAP estimates for parameters

• Conditional Independence

• Naïve Bayes >>> makes Bayesian learning practical

Machine Learning Department of Electrical Engineering


Naive Bayes in Nutshell

• Bayes Rule

• Assuming conditional independence among Xjs

Machine Learning Department of Electrical Engineering


Training Naive Bayes (Example 1)

X1 =>Sunny, X2=>Temp, X3=>Humidity, X4=>Wind, X5=>Water, X6=>Forecast


Sunny =1 Warm = 1 High = 1 Strong = 1 Warm = 1 Same = 1
Rainy = 0 Cold = 0 Norm = 0 Mild = 0 Cool = 0 Change = 0

Machine Learning Department of Electrical Engineering


Training Naive Bayes (Example 1)

Probabilities using Probabilities using


MLE MAP
P(sky = sunny | play)
P(sky = sunny | ~play)
P(Temp = warm | play)
P(Temp = warm | ~play)
P(hum = hi | play)
P(hum = hi | ~play)
P(wind = strong | play)
P(wind = strong | ~play)

Machine Learning Department of Electrical Engineering


Training Naive Bayes (Example 1)

Probabilities using Probabilities using


MLE MAP
P(water = cool | play)
P(water = cool | ~play)
P(forecast = same | play)
P(forecast = same | ~play)

New Example:

Sky = Sunny >>> X1 = 1


Temp = Cold >>>X2 = 0
Humidity = High >>> X3 = 1
Wind = Strong >>>X4 = 1
Water = Cool >>>X5 = 0
Forecast = Change >>>X6 = 0

Machine Learning Department of Electrical Engineering


Training Naive Bayes (Example 2)

Y = 1 (live in Islamabad) G = 1 (Shop at Centaurus)


D = 1 (Drive to EME by car/bike/bus) B = 1 (Visited Lake View Park)

P (Y = 1) = P (Y = 0) =

Probabilities Probabilities Probabilities Probabilities


using MLE using MAP using MLE using MAP
P(G|Y) P(~G|Y)
P(G| ~Y) P(~G| ~Y)
P(D|Y) P(~D|Y)
P(D| ~Y) P(~D| ~Y)
P(B|Y) P(~B|Y)
P(B| ~Y) P(~B| ~Y)

New Example
G = 1, D = 1, B = 0 P(Y=1) =?
Machine Learning Department of Electrical Engineering
Naive Bayes: Another perspective

Machine Learning Department of Electrical Engineering


What if Xis are Continuous?

Machine Learning Department of Electrical Engineering


Gaussian (Normal) Distribution

Machine Learning Department of Electrical Engineering


What if Xis are Continuous?

Machine Learning Department of Electrical Engineering


Gaussian Naïve Bayes (GNB) Algorithm

Machine Learning Department of Electrical Engineering


Estimating Parameters

jth training
example

δ ( ) = 1, if Yj = yk else 0
ith feature
kth class

Machine Learning Department of Electrical Engineering


Gaussian Pdf with Topographic Image

Machine Learning Department of Electrical Engineering


GNB: Big Picture

Example: Y = Play Basket Ball (Boolean), X1 = Height


X2 = GPA

Machine Learning Department of Electrical Engineering


Logistic Regression

Key Idea:

• Naïve Bayes allows computing P(Y) and P(Y|X) by


learning P(Y) and P(X|Y).

• Why not P(Y|X) directly?

Machine Learning Department of Electrical Engineering


Logistic Regression

Machine Learning Department of Electrical Engineering


Derivation of Logistic Regression

Machine Learning Department of Electrical Engineering

You might also like