Example <)><

({{Smelly, Color, Temperature, Goodness}, {yes, green, cool, bad}, {no, white, cold, good}, {yes, red, cool, sellable}, {yes, red, cold, good}, {yes, white, cool, good}, {yes, red, hot, bad}})

Multinomial Naive Bayes - "Categorical"

Parameters to learn

(Subscript[θ, b]) ≡ P(Y=bad) = ((#(Y=bad))/(# examples))=(2/6)=(1/3)

(Subscript[θ, s])=P(Y=sellable)=(1/6)

(Subscript[θ, g])=P(Y=good)=(3/6)=1-(1/3)-(1/6)=(1/2)

P(Smelly=yes|Y=bad)=(2/2)=1

P(Smelly=no|Y=bad)=0

P(Smelly=yes|Y=sellable)=1

P(Smelly=no|Y=sellable)=0

P(Smelly=yes|Y=good)=(2/3)

P(Smelly=no|Y=good)=(1/3)

Summary:

({{□, bad, sellable, good}, {P(Y), (1/3), (1/6), (1/2)}})         ({{Smelly, bad, sellable, good}, {yes, 1, 1, (2/3)}, {no, 0, 0, (1/3)}})

({{Color, bad, sellable, good}, {green, (1/2), 0, 0}, {red, (1/2), 1, (1/3)}, {white, 0, 0, (2/3)}})         ({{Temp, bad, sellable, good}, {cold, 0, 0, (2/3)}, {cool, (1/2), 1, (1/3)}, {hot, (1/2), 0, 0}})

"testing"

(Subscript[f, 1]):  Smelly = yes, Color = red, Temperature = cool

P(Y=good|X)=((P(X|Y=good) P(Y=good))/(P(X)))

P(Y=good|X)=(1/(P(X)))P(Smelly=yes|Y=good) P(Color=red|Y=good) P(Temp=cool|Y=good) P(Y=good)

=(1/(P(X))) (2/3)(1/3)(1/3)(1/2)=(1/27)(1/(P(X)))

P(Y=bad| X) =

(1/(P(X))) P(Smelly=yes|Y=bad) P(Color=red|Y=bad)P(Temp=cool|Y=bad) P(Y=bad)

=1(1/2)(1/2)(1/3)=(1/12)(1/(P(X)))

= (1/(P(X)))

P(Y=sellable|X)=(1/(P(X)))(1/6)

P(X) = (∑_(y=1)^k)P(X,Y=k)

This fish is Sellable despite being Smelly

(Subscript[f, 2]): Smelly = yes, Color = white, Temperature = cool

Recall Summary:

({{□, bad, sellable, good}, {P(Y), (1/3), (1/6), (1/2)}})         ({{Smelly, bad, sellable, good}, {yes, 1, 1, (2/3)}, {no, 0, 0, (1/3)}})

({{Color, bad, sellable, good}, {green, (1/2), 0, 0}, {red, (1/2), 1, (1/3)}, {white, 0, 0, (2/3)}})         ({{Temp, bad, sellable, good}, {cold, 0, 0, (2/3)}, {cool, (1/2), 1, (1/3)}, {hot, (1/2), 0, 0}})

P(Y=bad,X)

=P(Smelly=yes|Y=bad)P(Color=white|Y=bad) P(Temp=cool|Y=bad)P(Y=bad)

=1 0

Laplace smoothing

θ ≡ P(Y=bad) = ((#(Y=bad)+1)/(# (examples)+3))=(□/□)

P(Color=white|Y=bad) = ((#(Y=bad, Color=white )+1)/(# ( Y=bad)+#(bad fish hallucinated for all Colors)))=(□/□)

no zeros!

Gaussian Naive Bayes

Parameters to learn

θ ≡ P(Y=1) =same way as for Mult. NB

P(Smelly|Y=bad) ~N((Subscript[μ, 01]), (Subscript[σ, 01]))

(Subscript[μ, 01])= (1/(#(Y=bad)))(∑_(bad examples))value of Smelly

What's the problem?

Smelly: Binary -> {0,1}

Temperature: can get the numeric values -> Gaussian makes sense since closer numbers are more "similar"

Color: green=0, red=1, white=2?

doesn't really make sense... can still model as multinomial and combine with the gaussian features! (why not? ← rhetorical)