BAYESIAN CLASSIFICATION (UPTU 2011 2) ‘The Bayesian Classification represents a supervised learning method as well as a statistical method fot classification. Assumes an underlying probabilistic model and it allows us to capture uncertainty 3b0!cusunesnon sno Patrons rma pms ay yn ee cm er caanet Cassiaion named fer Thomas Bays (702-76 who rg he Rayos Thee resonate as cl arn ll ooo tn aft Baya Clsication provides a uefa pete for meaning and eating ay itt ese eis iene omnes nes clair ae stitial assis They can edit ease met proaes cease pmb ta gen pe longo apres Bayvnncsaeton el ren Sasi clin anaes van et ee ae Basen cerca ertenance eth reson ede Save Dyesian cls: sme tnt te effet of an aie va on a ven las i fin vuoi Tesi ete ts contol npc sss mt) te compton Ill ann th Some enseed ave Bayon 2a gral mon nich ne mv eyes cass alow ie epeeraton dt ng ssc ofates Bayesian et cheno edo seston ‘Bayes Theorem ‘erie a data sample whose class label i unknown, Let be some hypothesis, such as that he data ‘Erne Xbslongs oa specified class C. For classification problems, we want wo determine PH X).the ne abies of he outcnmes, Kean salve diagnostic and prota that the pothesis Hho given the observe data sample X: PU |X) i the posterior obabily ox posterior probability, of H coi For example, suppose the world of data samples consists of fruits, described by their color and tage, Suppose tht Xs red and round, and that His the hypothesis that X isan ape. Then UH |X fats our confidence that X isan apple given tat we have seen that X is red and round In contrast “Rss prior probabil, ors prior probably of H.Simily, PX) ithe prior probability of X ‘or our example, thi is the probability that any given data sample isan apple, regardless of ow the dua sample looks The posterior probability, PH |) is based on more information (such as teciground knowledge) than the prior probably, PCH, whichis independent of X ‘Similar, PUC| His the posterior probability of Xcooitioned on H. Tats, tithe probability ‘ta: red and round given that we know that its tue that Xs an apple. POX) is the pir probability ‘of. Using our example: its the probability that data sample rom our st of fruits sre and round. 'PUX), PCH), and PC | H) may be estimated rom the given dats. Bayes theorem i useful in that provides «way of calculating the posterior probability, PC |) from PD, PX), and POX| #, Bayes theorem i _ POC HPH) pat |x) = Or Naive Bayesian Classification ‘The naive Bayesian classifier, or simple Bayesian classifier, works as follows: 1 eh ds mpl reenented by an nsmensional fear Yer, X= 9 0M Each dase Ppa ote ale fom nates, eapectvely Ay Ay Ay aaa ret uses Co Ca a Gy vn nentnown dea Le ear ee lic wil psi ht Xtelonas tothe cls avin the hes aang a aiuoned on That the nave Bayesian classifi sign a9 esnown sample Xt the ls fan only if [ TAG 13> MG)13) foe sj JotDara Mine avo DATA Wa rOUSING 5G, for which PU wm we masiie MG | XT nim otro yore Bayes theorem PLXJE DPE) ne) OR ov fra seo te rit POX|C).PC) me be and a comin aimed tn ces ca Cy ns NG) an ne woul hie ns Se calms TC.) ete sas por pti a8 MC) training tuples of class; in D. 4. For large data set, computation of PCX | C) is very complex. In ord to reduce comput, in evaluating P(X | C).the naive assumption of class conditional independence is mae Using this assompeion, we ean express PUX | C) a8 5, As PO is const lass prior probabil IG, oD. where [ol 1 the number paicp= [1Pa1G) = Pos, G) x Play | €) x «XP IC) ere refers tothe valu of atvibute A, for tuple X. We can easily estimate the probe Psy] G), Psy] Cs PC |G) fom the training apes For each atiibue, we look at whether the atibute is categorical or continuus-valus Fe Instance to compute PX | C), we consider the following (os categorical, then Pi |G) isthe number of tuples of class C, in. D having th we +5 for Ay divided by [C, othe numberof tuples of class C; in D. (O)IE A, is contnaoas-valued, then we need to do bit more work. A continuous bs! sibut is typically assumed to have a Gaussian dstribuion with a mean wand ste! deviation 0, defined by fo 1 808.He0e) = —e Me 80 wher (Hy the Gassian (arma) density funtion for tbe A, whieh and ate the mean and variance respectively given the values for atte, fo wil samples of class C, . an 5.1m oder to predict the class abel of X, PUK |C).20C) 7 POX | ).2(C) a evaluated foreach class Class pedis tha the cls ltl of ple Xs the las italy POX |G) PC) > PKG) PC) ft <<, Inthory, Bayesian clair hae the a eat nimam ero rat in comparison tol te cas Beye lasiies are also useful in tht hey provide a theoretical justine fr eter ce wont pl we Byes here Forex under ain smpin cn Imeny neural nework and curve-fiting algrithns ouput the masini prrerir hypothesis the naive Bayesian laste, m aaa PUI)an Cusinernon si Parocrons Trample: Suppose we wish o predict the clas abel ofan unknown sample win sano Te ening a ae Follows ce ‘Table 6.1 Training data tuples from the customer database ae ‘income [student | ereditraing | Class: buys computer al | ao) hah = Tae 0 yf so | high "0 excelent a | | Sogo | wien to tat yes en ee ‘ie | yes f | So | tow ves fie yes & | 30 | tw yes cellent ~ + | so | tow se ‘cellent ye & | 0) | aetiom | no fae 7 5 | 30 | tow ys fat i wo | So | medion | ye fae yes th {<0 | media | yes cet | yes |i | 3o40 | medium | to creelen ye | ie | oto | wi vs ta ye | [te | | maton | oo cellent » | “Te data tuples are described by the aibutes age, come, student and credit raring. The class titanate bu computer as two diferent values [965 no. Lat C; earespond tothe clas buys. computer = jes and C,corespond to buys. computer =o. ‘The ninown sample we wish to classify is "X= (age = "<20", income = medium, student = yes, credit ating = fi) ‘We ned to maximize P(X | C).P(C), for i= 12. PCC). the prior probability ofeach class, ean te omuted based on the taining simples Pobays_computer = yes) = 914 = 0.683 ‘Pibuys_computer = no) = 5/14 = 0.357 “ocompute P| C), for ‘= 1,2, we compute the following conditional probabil: ‘lage ="<30"|buys_computer= yes)= 299 = 0.222 Plage =“<30" | buys_computer = no)= 315 = 0.6 income = medium | buys. computer = yes ‘lincome = medium | buys_computer = no) = 25 = 04 Pistaden = yes | buys computer = yes) \661 ‘Ptstadent = yes | buys computer =o) = 15 = 02 Pleretit ating = fair | buys_compater = yes) = 619 = 0.667 leit ating = air | buys_computer = 90) 4 Using he above probabilities, we obtain Ox | buys.computer = yes) = 0222 x 0444 0667 x 0.667 = 0.044 0x | buys_computer = no) = 0.600 x 04 x 0.200 x 0.400 = 0.019 ‘AX buys_computer = yes) Pbuys computer = yes) = 0.044 x 0.63 = 0.028 0x | buy computer = no) Pbuys. computer = no) = 0019 x 0.387 = 0.007 “Thecefore, te naive Bayesian classifier predits "buys computer = yes for sample X.