C4.5 Algorithm decision tree
C4.5 Algorithm decision tree
5)
1
Training Examples for PlayTennis
(Mitchell 1997)
Outlook
No
K
Humidity
Normal
Yes
Yes
Strong
No
Wind
Weak
Yes
3
Decision Tree Algorithm (C4.5)
i=1
(9Y, 5N)
4
Decision Tree Algorithm (C4.5)
|Humidir T
= -El(s/S)nfaS,)]
Igo,(8)
5
Decision Tree Algorithm (C4.5)
S
Splitlnfo X)=
6
Decision Tree Algorithm (C4.5)
7
Decision Tree Algorithm (C4.5)
Step 1
Use Entropy:-P.log,P-P.log,P.
Atfirst, we have 9(YES) and 5(NO)
Starting Entropy:-9/14 log, 9/14 -5/14 log,5/14-0.94
Ingo( S)=-eg(C,
=1
Sy|s]log, [freg(C, SY|S|)
8
Entropy (Info)
1.0
0.5
Entropy(S)
0.0 0. 1.0
9
Decision Tree Algorithm (C4.5)
Step 2
Compute the gain ratio for each attribute
10
Decision Tree Algorithm (C4.5)
Step 2(Con t)
Wind Temperature
11
Decision Tree Algorithm (C4.5)
log,,X log,,X
log, X=
Step 2(Con't) log,,2 0.30103
=(0.5281H04421)=0.9702
Outlook E{-4/4 log,4/4 -0)=(-1log, 1 -0)-0;
E(-3/5 log,3/5 -2/5 log, 2/5)
(-0.6* (-0.7369)+0.4 *(-1.3219)= 0.9702
2+ 3 4+ 0
Gain (X)= hnfo (S)-Info,(S)
Splitlnfo =-5/14 log, 5/14 - 4/14 log,4/14 -0.94 -5/14*0.9702- 4/14*0 - 5/14*0.9702
-5/14 log, 5/14=1.577 =0.94 -0.693=0.247
Step 2(Con't)
Compute the gain ratio for each attribute
13
Decision Tree Algorithm (C4.5)
Step 2(Con t)
Gain ratio=0.0292/1.556=0.01876
14
Decision Tree Algorithm (C4.5)
Step 2(Con t)
Infogain-0.94 -6/14*1 =
- 8/14*0.8112 0.048
15
Decision Tree Algorithm (C4.5)
Step 2(Con t)
16
Decision Tree Algorithm (C4.5)
Step 3
Decide other attribute under root node
Yes
17
Decision Tree Algorithm (C4.5)
Step 3(Con t)
E (Outlook=Sunny)
=-2/5 log,2/5 - 3/5 log,3/5= 0.97
Sunny
243
18