Scribd
Upload
22 views29 pages

Decision Trees

Uploaded by

dineshragav90
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
22 views29 pages

Decision Trees

Uploaded by

dineshragav90
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 29

Decision Trees

Premalatha M
Decision Trees
 The core algorithm for building decision trees
called ID3 by J. R. Quinlan which employs a
top-down, greedy search through the space of
possible branches with no backtracking. ID3
uses Entropy and Information Gain to
construct a decision tree.
Entropy
 A decision tree is built top-down from a root
node and involves partitioning the data into
subsets that contain instances with similar
values (homogenous). ID3 algorithm uses
entropy to calculate the homogeneity of a
sample. If the sample is completely
homogeneous the entropy is zero and if the
sample is an equally divided it has entropy of
one.
Entropy
To build a decision tree, we need to calculate two types of entropy using
frequency tables as follows:

a) Entropy using the frequency table of one attribute:


Data
Day Outlook Temperat Humidity Wind Class: Play ball
ure
D1 Sunny Hot High False No
D2 Sunny Hot High True No
D3 Overcast Hot High False Yes
D4 Rain Mild High False Yes
D5 Rain Cool Normal False Yes
D6 Rain Cool Normal True No
D7 Overcast Cool Normal True Yes
D8 Sunny Mild High False No
D9 Sunny Cool Normal False Yes
D10 Rain Mild Normal False Yes
D11 Sunny Mild Normal True Yes
D12 Overcast Mild High True Yes
D13 Overcast Hot Normal False Yes
D14 Rain Mild High True No
D=14
 Yes = 9  In Humidity,
 No=5  High=7 (Y=3, N=4)
 In Outlook,  Normal=7 (Y=6,
 Sunny=5 (Y=2, N=1)
N=3)  In Wind,
 Overcast=4 (Y=4,  False=8 (Y=6,
N=0)
N=2)
 Rainy=5 (Y=3, N=2)  True=6 (Y=3 N=3)
 In Temp,
 Hot=4 (Y=2, N=2)
 Mild=6 (Y=4, N=2)
 Cool=4 (Y=3, N=1)
Entropy -
In Outlook,
Gain Outlook Sunny=5 (Y=2, N=3)
Overcast=4 (Y=4, N=0)
 Gainout(d)=Info(D)-Infooutlook
(D) Rainy=5 (Y=3, N=2)
In Temp,
Gain Temp Hot=4 (Y=2, N=2)
Mild=6 (Y=4, N=2)
 Gaintemp(d)=Info(D)-Infotemp
(D) Cool=4 (Y=3, N=1)
In Humidity,
Gain Hum High=7 (Y=3, N=4)
 Gainhum(d)=Info(D)-Infohum(D) Normal=7 (Y=6,
N=1)
In Windy,
Gain Windy False=8 (Y=6,
 Gainwind(d)=Info(D)-Infowind(D) N=2)
True=6 (Y=3
N=3)

 Maximum Gain -> Outlook = 0.246


Tree with root = Outlook
D=5 [Sunny]
 Yes = 2
 No = 3
 In Temp,
 Hot=2 (Y=0, N=2)
 Mild=2 (Y=1, N=1)
 Cool=1 (Y=1, N=0)
 In Humidity,
 High=3 (Y=0, N=3)
 Normal=2 (Y=2,
N=0)
 In Windy,
 False=3 (Y=1, N=2)
 True=2 (Y=1 N=1)
Info(D) - Sunny
In Temp,
Gain Temp - Sunny Hot=2 (Y=0, N=2)
Mild=2 (Y=1, N=1)
 Gaintemp(d)=Info(D)-Infotemp
(D) Cool=1 (Y=1, N=0)
In Humidity,
Gain Hum - Sunny High=3 (Y=0, N=3)
Normal=2 (Y=2, N=0)
 Gainhum(d)=Info(D)-Infohum(D)
In Windy,
Gain Windy - Sunny False=3 (Y=1, N=2)
True=2 (Y=1 N=1)
 Gainwind(d)=Info(D)-Infowind(D)

 Maximum Gain -> Outlook -> Sunny = Hum =


0.971
D=5 [Rainy]
 Yes = 3
 No = 2
 In Temp,
 Hot=0 (Y=0, N=0)
 Mild=3 (Y=2, N=1)
 Cool=2 (Y=1, N=1)
 In Humidity,
 High=2 (Y=1, N=1)
 Normal=3 (Y=2,
N=1)
 In Windy,
 False=3 (Y=3, N=0)
 True=2 (Y=0 N=2)
Info(D) - Rainy
In Temp,
Gain Temp - Rainy Hot=0 (Y=0, N=0)
Mild=3 (Y=2, N=1)
 Gaintemp(d)=Info(D)-Infotemp
(D) Cool=2 (Y=1, N=1)
In Humidity,
Gain Hum - Rainy High=2 (Y=1, N=1)
Normal=3 (Y=2, N=1)
 Gainhum(d)=Info(D)-Infohum(D)
In Windy,
Gain Windy - Rainy False=3 (Y=3, N=0)
True=2 (Y=0 N=2)
 Gainwind(d)=Info(D)-Infowind(D)

 Maximum Gain -> Outlook -> Rainy = Windy =


0.971
Predict:
 Outlook=Overcast, Temperature=Mild, Humidity=Normal,
Windy=False
To do

You might also like