| | Decision Tree Decision Tree~ A decision tee is a structure that includes a vot node, branches and leaf nodes. Each internal node denotes a test on an attribute each branch _olenotes the outcome of atest, and each leaf node holds a class label The topmost node in the Tree is the _voot _ node. Example: The following decision tree is for the concept __buy- computtey that indicates whether a cuistome. ay at a company is likely to buy a compuler or aot: Each APR PRPPIPMIDPPIDA PP PPR Mae intemal node __vepresents a Test T zy Ir on an attribute, Each leaf Node 4 5Connotea) yepresents a class- a0.ge young credit-vatin. iddlle— { Student 2 ose (ve) EN Tae \a Decision Tree Tnduction:- TE is a — Supewised learning method _useol in data. mining for chssification anal regression Methods. a helps us __in dkcision making purposes - Tt creates classification or regression, models. It separat iS a deta get into smaller isubsets, ond at the same time the decision tree _dleveloped steacli The final tree is with decision Nodes and leat node."| Key factors: Entropy Entropy refers to a common way to measure impurity. In_ the decision tree, it measures the ! - Lvandomness or impurity in data sets. Tnformation Gain: Tntormation gain refers 10 the decline in entropy after the data sets split - Tt is also called Entropy Redluction. _Buildling a dbecisio tree is all about discovering attribules that return the _ highest dato. gain. Why _ave decision Tree useful? E enables us to analyae | the possible consequences of a f ~ decision throughly ; it provide s us framework “tb : measure the values of outcomes and the probability of accomplishing e&ORE them. Examp le Day Outlook Temp Humidhty Wind — Decision 1 sunny | Hot | High | weak | Ni 2 [sunny | Hot | High Strong_|Na 3 lovercast | Hot High weak | Ye 4 | Rain Mild | High | weak | Yas 5 _| Rain Cool | Normal | weak | Ya 6 | Rain cool | Normal strong | Ni 7 lovercast | cool |Noxmal Strong | Ye 8 | sunny | Mild High _|weak | Ng 4 | eunoy | cool_| Normal | weak Ye 10 | Rain ‘ Mild |Novmal | weak | Ye | Sunoy | Mild [Normal | stvong | Ye 12 | Overcast | Mild | High | ctvong | Ye 13 | Overcast | Hot [Normal | weak Ye 14 | Rain Mild | High | strong | NShep 1 Find information gain _of t target altributes. I1G--P log fP\-N bos \ Pan > Spent PNM Pen Entwopy (S)2 -4 log (4-5 (2) oe 14 \ Fe “Ty 7X yt = 0-410 + 6-530 = 0-44 Step 2 ind entropy of yemaining attributes one by one. Entwopy (5, outlook) :- es No Total sunny 3 2 31 overcast | 4 0 4 rainy r | 3 5 praneireararioravannnsipp re nale 4 Ss.7 EA) = & PrNt 7 (PNe) TPN LG |E(s outlook) 2 _ 2 | - 2 log ([B 5S) 9 (BY Ps Ne3 SUNNY e A be = 0:52940.44Q = 0.4944 E(Sroutlok) <= 4 fog (4) - 2 log (2) ps4 neo Overcast weer ae : =0 E (Ss outlook) . - 3 oq (3)- 2 log (aly 20H on) 3 (3 oS Si? = OWA + 0-529 = 0-474 E(s,outlook) =F (Era) (E(40)) +5 (E(32)), eH t = 5 (0974) +4 (o)+ 5 (0-49) 14 ea = 0:6 936saan Gqaiia dS Outlook) Gain = 1G - EA) = 0:94- 0-6936 = 0 R464, Now, we will Yepeate step 2 & 3. fox other oitvibutes. Entvopy (S, temperature ):- Yes No Total Hot 2 2 4 Mild 4 [2 6 Cool 3 1 4 (252) E (Ss) temnpevatuve ) wd log. (& 3 2 log hot q y re = 0:5+0-5 (4a = E (5, temperetine) = “4 log, (4. - 2 log (2 mild le = 0-340 + 0-525 = 0-918 APerPpv?Pz?7IVI90P Prope PPPR KhAls 4 (3 IE (5, temperature). -3 log (3\- 4 log cool q qT y Z| ch : p{Mon {Rue Wed rh) CR Sar] = 0-311 +0-5 = 0-311 i E(S , temperature) = 4. (El22)) EU 2 t iq 4 E(>)1)) 4 = 4 (1)4 4 (0-918)+ 4 0-811) ty 1% TY Ce = 0.914 Gain 5 impale - = 0-44-0914 = 0.0249 Entropy (5, Humidity ) = Yes lo Total High 3 4 te Normal 6 ah 7 @) ECS) Humidity) . -3 log (2-4 log (4 + a eat high = 0-594 #¢ 0-461 Y = 0-985 wre s,VrVTsVTVZFsFVRe(61) yh. : Q E (S)Humidity) = 6 loq_6 -4 lo nosmal et 7 Sq 7S = 0-1906 +0-401 a 2 6-592. Gq E (5 humidity) = 4 (€ (3)4)) +2 (E60) & 14 TH a & = E (09-9854) + (0-59 7 (0-4 4s (0-594) e = = 0-186 = ain) (St Humidity) :- e ~ O-G 4 = 0-783 o = 0-152 & Entzo Wind ):- © Tota] & Yes lo lola - weak 6 2 8 & strong 3 3 6 a (6 9) & E(S,wind) 2 -& |o4 (4)- 2 lo (=) & weak e i: “a « 0-311 + 0-45 & 2 0- 0-5 ~ 2 0-911 a a a- “3 bg (3.)-3 bg 3 E(s, wind) 3. 7 6 d= 393) Strong 6 2 0510-5 = Ai = 8 (osemy +6, ). : iy JE(S; Si wind) = 8.(6 (E(6a) 4 + 4 (E00) — = 0893 a ain (Sy wind )- = 0-94- 0.898 = 0-048 From gain of all four _atttibutes Gain (S$, outlook) has maximum value, so Qutloole will be the toot node of this clecision tree. out | ook Sunny Rainy. . cast eg ee ge eae ner at afemmaS) = Ds outlook have +hvee values » 50 . we will divide given table into x three tables actording + each value. a Outlook Temp Humidity iNind Decision __ sunny | Hot | High | weak _| No & sunny | Hot | High strong No * sunny | Mild | High [weak | No - sunny | Cool | Nowmal | weak | Yes om sunny | Mile | Normal | strong | Yes eo a utlool Temp Humidity Wind Decision © overcast | Hot | High weak | Yes 7 overcast [Cool | Normal | strong | Yes a overcast_| Mild | High styong_| Yes | overcast | Hot | Normal | weak Yes 3 Outlook Temp Humiohiy Wind Decisi | Rain [Mild | High | weeute Yos « Rain | Cool _| Normal | weak Yes ss Rainy Cool | Normal Strong | No 1 Kai Mild _| Novmal weak Yes s Rain | Mill | High strong | No o(Mon The Wea) im RDSa) br Since overcast contains only examples of class ‘Yes? we can set it as Yes- Thot means if outlook is overcas match will be played. Glew uny Fainy t > : b > > > Now, we will solve 4st table: Sunn ¥ is Ast _Step - E (Sumy )= 3g) = 5 18 Cs) = 0444 40.529 = 0-974 :« iy 2 ; a | Step 2&3 for all _ottvibutes: =a net e E (Sunny 7 Temperature )- a Yes. No Total & Hot ° 2 2 & Mild | 4 1 2 id Coo] L 0 1 a ae E(S,T) . -2 |vg (2)-2 log (2) e Rot a i Q ve i = 0 o EST)» -t fog (2) +L log, (£ e mild a a et o 7 > = 65 40-5 i: ee 7 EC z “4 log. (4) +0. log (2) a ihe < E (Sunny, Temperature )= 4 2 (E(o0a)) + 2 (E(3.1)) + ea Z (Ela) + g(E(4 1) + (Eo) og = 2 (0) *% (4) +3 (0) = 04 e ajain (Sunny, Temperature): | 2 0-FH- 0-4 = 0-542 E (Sunny, Humidity ):- Yes No Total High © 3 3 Normal Zz ° 2. E (SH) high 3 3 2 0 E(S)H 2-2 Jog, (2)- 2 [09 (a : 1 z *3 (3) a 9. ) E (Sunny , humidity ) = 3 (E(3)) +2 (E = 3 (0)+2 (0)=0 G ain (S unny ) Humidity )- = 0-4H-0 > 0-974E(Suany, Wind):- Yes No __lotal strong i. 2 | a Weak 1 2 | 3 S 2.4 L\_2 zL E( sw) = 2 bog, (4) 2 bog 5 = 0523+ 0-390 = 0-918 E (Ss, W) 2 = log, (+£) - 4 log (4 Strong = 65+0-5 s 4 E( Sunny, Wind) = 2 (E(41)) +3 (E (La) 22 (1) +3 (0-918) 0 9508 Gain (Sunny » Wind): = 0-91- 0-H0g = 0.020 DIPKAPTEPTIDIAPPAAP OPP PRK A ¢(tou (Tue (Wealihal( Ay Ser) | | Here» Gain (Sunny, Humidity) isthe br laygest value. So humidity is the node that comes under sunny - Now, we will form the table for humidity if outlook. sunny. Yes No Total High o |4 3 Normal 2 } 2 Hf outlook is sunny and humioh is high then decision willbe No, and _ if humidity is Nowm A then decision will be Yes, Now , we will solve 31d table.Step 2% 3 for _all_attvibutes:- - EER — = Rain :- € et Step= " gz E (Rain) = “3 log (3)-2 log, (2) & a ae a = 0-474 % e& = & E (Rain, Tempeyature ):- Z| o Ves Total Hot | ° ° o a wild | a i 3 a (ool | 4 4 [2 5 Ss eps 4 E( ) 2 509, (3) -4 log, (4) ‘| x = 0-390+ 0-528 NX = 0-913 q E(RT) = -4 log (4) -2 q por 8 gfe = 0.5405 S ial = 4 s »E (Rain, Temperature) =O +3 (E(a4)) +2 (E(1,4) 5 > 2 3 + 2 (0.918) us (4) = 0-9508 ain (Rain, Temperature ):- = 0-944 - 0-950 = 0-020 E (Rain, Humidity )-- Yes No Total Heh | | ee Normal 2 1 3 E (RH) = -2 log (L)-2 log (4) nigh TP rere = OF+05-4 tap Dell 255. delbg Id ) E (RH) + -2 log, (2) 5 logit = 0340 +0523 = 0-918 TEST VV PPT SWIFT TFT FOV Te Oea E (Rain, Humichty) = % (Eb1)) +3 (ela) - 2 (1) +3 (0.918) gS = 0-950 ain ( Rain , Humidity ):- 0-F#L- 0-508 = 0.020 he Wind) Yes No Total Stro 9, o 2 2 weak 3 ° 3 E(Rain ,Wind ) = 0 Gain (Rain, Wied) 2 0-474 -0 _ O-FfFt Here Goin ( Rain , Wind) has the maximum value: BQIIARATIAIT AP PPIDPM9I Daa alnln le wma al,br So,» It : is clear that if outlook | 6 yain then the next node E will be Wind. | If willd is strong , then b decision “will be No. And # b wind iy weak, then decision will be Yes. Decision Tree: 7 | out look ]