See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/330138092

Study and Analysis of Decision Tree Based Classification Algorithms

Article in International Journal of Computer Sciences and Engineering · October 2018

DOI: 10.26438/ijcse/v6i10.7478

CITATIONS READS
405 48,227

2 authors:

Harsh Patel Purvi Prajapati

California State University, Fullerton Charotar University of Science and Technology
1 PUBLICATION 405 CITATIONS 19 PUBLICATIONS 512 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Purvi Prajapati on 15 July 2019.

The user has requested enhancement of the downloaded file.

 International Journal of Computer Sciences and Engineering Open Access
Research Paper Vol.-6, Issue-10, Oct. 2018 E-ISSN: 2347-2693

Study and Analysis of Decision Tree Based Classification Algorithms

Harsh H. Patel1*, Purvi Prajapati2

1,2
Dept. of Information Technology, CSPIT, Charotar University of Science and Technology, Changa, Gujarat, India.
*
Corresponding Author: [email protected], Tel.: +91 7043046700

Available online at: www.ijcseonline.org

Accepted: 13/Oct/2018, Published: 31/Oct/2018

Abstract— Machine learning is to learn machine on the basis of various training and testing data and determines the results in
every condition without explicit programmed. One of the techniques of machine learning is Decision Tree. Different fields
used Decision Tree algorithms and used it in their respective application. These algorithms can be used as to find data in
replacement statistical procedures, to extract text, medical certified fields and also in search engines. Different Decision tree
algorithms have been built according to their accuracy and cost of effectiveness. To use the best algorithm in every situations
of decision making is very important for us to know. This paper includes three different algorithms of Decision Tree which are
ID3, C4.5 and CART.

Keywords— Machine Learning, Decision Tree (DT), WEKA tool.

I. INTRODUCTION TO DECISION TREE both situations whether one has discrete or continuous data
as input. The example of Decision Tree is as follow [15].
Classification is the task of giving objects to categories
which have many diverse applications.

Fig. 1: Classification of mapping attribute set (X) to its class

label (Y)

Decision Tree
A normal tree includes root, branches and leaves. The same
structure is followed in Decision Tree. It contains root node,
branches, and leaf nodes. Testing an attribute is on every
internal node, the outcome of the test is on branch and class
label as a result is on leaf node [3, 4]. A root node is parent of
all nodes and as the name suggests it is the topmost node in
Tree. A decision tree is a tree where each node shows a Fig. 2: Example of Decision Tree on what to do when
feature (attribute), each link (branch) shows a decision (rule) different situations occur in weather.
and each leaf shows an outcome (categorical or continues
value) [4]. As decision trees mimic the human level thinking When data does not offer benefits while splitting, it directly
so it’s so simple to grab the data and make some good stops the execution. Try to find one test at a time rather than
interpretations. The whole idea is to create a tree like this for optimize the whole tree together.
the entire data and process a single outcome at every leaf.
Talking about the characteristics of Decision Tree, the ID3
II. RELATED WORK ON DECISION TREE algorithm is simulated only on WEKA tool and the data type
of data set is only categorical. ID3 can not take continuous
Decision Tree is similar to the human decision-making data set for simulation. Similarly, CART and C4.5 have same
process and so that it is easy to understand. It can solve in

© 2018, IJCSE All Rights Reserved 74

 International Journal of Computer Sciences and Engineering Vol.6(10), Oct 2018, E-ISSN: 2347-2693

characteristics as ID3 has. The only difference is that C4.5 datasets. The
and CART both can take continuous data set as input for technique called
simulation purpose [11]. “PRUNNING”,
solves the
Table-1: Characteristics of DT. problem of over-
Decision Tree Data Types Numerical Possible filtering [9].
Algorithm Data Splitting Tool C5.0 Improved version of C5.0 allows to
Method the C4.5 whether estimate
CHAID Categorical N/A SPSS missing values as
answer a function of other
tree attributes or
ID3 Categorical No Restriction WEKA apportions the
C4.5 Categorical, No Restriction WEKA case statistically
Numerical among the results
CART Categorical, Binary Splits CART 5.0 [13].
Numerical CHAID (CHi- Predates the original For a nominal
square ID3 implementation. scaled variable,
The decision tree makes explicit all possible alternatives and Automatic this type of
Interaction decision tree is
traces each alternative to its conclusion in a single view, to
Detector) [6] used. The
make easy comparison among the various alternatives [12]. technique detects
Transparent in nature is one of the best advantages of the dependent
Decision Tree. variable from the
categorized
Another main advantage is the ability to selecting the most variables of a
biased feature and comprehensibility nature. It is also easy to dataset [3, 11].
classify and Interpretable easily. Also used for both MARS (multi- Used to find the best In order to
continuous and discrete data sets. adaptive split. achieve the best
Variable screening and feature section are good enough in regression split, we can use
decision tree [19]. By talking on its performance, non-linear splines) the regression tree
does not affect any of the parameters of the decision tree. based on MARS
[2, 10].
III. DECISION TREE ALGORITHMS
IV. METRICS
Decision tree algorithms are used to split the attributes to test
at any node to determine whether splitting is “Best” in According to the values of the splitting attribute, the training
individual classes. The resulting partitioned at each branch is data are partitioned into several subsets. Until all instances in
PURE as possible, for that splitting criteria must be identical. a subset belong to the same class in any Decision Tree the
algorithm proceeds recursively [6].
Table- 2: Decision tree algorithms
Algorithm Classification Description Table- 3: Splitting Criteria
name
CART Uses Gini Index as a By applying
Metrics Equation
(Classification metric. numeric splitting, Information
and Regression we can construct Gain
Trees) the tree based on
CART [4].
ID3 (Iterative Uses Entropy function The only concern
Dichotomiser 3) and Information gain with the discrete Gain Ratio Gain Ratio=I(p,n)-E(A)
as metrics. values. Therefore, I(p,n)= Information before
continuous dataset splitting
must be classified
E(A)= Information after splitting
within the discrete
data set [5].
C4.5 The improved version Deals with both Gini Index
on ID 3 discrete as well as
a continuous
dataset. Also, it
can handle the
incomplete

© 2018, IJCSE All Rights Reserved 75

 International Journal of Computer Sciences and Engineering Vol.6(10), Oct 2018, E-ISSN: 2347-2693

Information Gain is biased towards multivariate attributes terms of persons carries (persons), the size of luggage boot
which are the main drawback of Information Gain [6]. The (lug_boot) and estimated the safety of the car (safety).
unbalanced split of data where one of the child nodes has
more number of entries compared to the others Gain Ratio Number of Instances: 1728
generally prefers that [7, 12]. Gini Index gives unfavorable Number of Attributes: 6
results as with more than two categories in the data set. Missing Attributes Value: None
These are the drawbacks of splitting criteria [15].
Attributes Value:
V. EVALUATION MECHANISM
Attribute Attribute
If the values are close to each other, the set can be said to be Values
precise. If their average is close to the true value of the buying v-high, high, med,
quantity being measured, the set can be said to be accurate. low
Only if given a set of data points from repeated maint v-high, high, med,
measurements of the same quantity then one can measure low
above two terms [13]. doors 2, 3, 4, 5-more
persons 2, 4, more
( ) lug_boot small, med, big
( )
safety low, med, high

Class Distribution (Number of instances per class):

( )
Class N N [%]
Unacc 1210 70.023%
TP = True positive, TN = True Negative Acc 384 22.222%
FP = False Positive, FN = False Negative good 69 3.993%
v–good 65 3.762%

VII. EXPERIMENT

The experiment is simulating on WEKA tool. For data

mining tasks, WEKA is a collection of machine learning
algorithms. For data pre-processing, classification,
regression, clustering, association rules, and visualization
Weka contains tools. Weka is open source software issued
under the GNU General Public License. It is also well-suited
for developing new machine learning schemes. The
algorithms can either be applied directly to a dataset or called
from your own Java code [18].

Table- 4: Theoretical results

Fig. 3: Confusion Matrix sample in Decision Tree. Algorithm Attribute Missing Pruning Outlier
Type Value Strategy Detection
VI. DATASET DESCRIPTION ID3 Only No No Susceptible
categorical to outlier
The dataset which is used in the experiment is car dataset. By values
applying this dataset on three algorithms of the decision tree CART Categorical Yes Cost Can handle
which are ID3, C4.5, and CART. Dataset description is as and complexity
follow. Numerical pruning is
both used
The car dataset is in two parts. One is Car Acceptability and C4.5 Categorical Yes Error based Susceptible
other is Technical Characteristic. Overall price (buying) and and pruning is to outlier
Numerical used
Price of Maintenance (maint) are two attributes of Car
both
Acceptability. The Number of doors (doors), Capacity in

© 2018, IJCSE All Rights Reserved 76

 International Journal of Computer Sciences and Engineering Vol.6(10), Oct 2018, E-ISSN: 2347-2693

For the experiment, this paper distributes the same data sets
on three different decision tree algorithms like ID3, C4.5,
and CART. The results of all three algorithms in the terms
time and accuracy with the help of the outcome from the
below table [17]. The splitting Criteria column gives
information about how the algorithm split in order to get a
better result. The attribute type column gives information
about what type of values the algorithm can handle. Whether
the algorithm finds the missing value or not, the result
defines from the Missing Value column and thus the Fig. 4 – Confusion matrix for CART
algorithm is accurate or not we can find.
VIII. CONCLUSION
Table- 5: Practical results
Algorithm Time Taken Accuracy Precision The Decision Tree algorithms ID3 C4.5 and CART were
(Seconds) (%) applied on the dataset. Decision tree outperforms others in
ID3 0.02 89.35 0.964 terms of accuracy, time and precision. It quite relies on the
CART 0.5 97.11 0.972 algorithm used for recommendation to find interesting
C4.5 0.06 92.36 0.924 resources. At last, the comprehensive study is done about
decision tree algorithms and this paper concludes that CART
As we can see the above table is the practical result of three is the algorithm for this dataset is very precise and most
algorithms ID3, C4.5, and CART. One can notice that CART accurate among the others.
takes 0.5 seconds to execute an algorithm, ID3 takes 0.02
seconds and C4.5 takes 0.06 seconds. The slowest execution IX. FUTURE WORK
is of CART and fastest is ID3.
In the future, this will be installed in the Apache server thus
Though CART takes too much time or we can say it is the published on the internet. [17] Datasets are updated
slowest one among them, accuracy is highest and it gives continuously and it will take online rating for the prediction.
very precise result than the other algorithms which are ID3 The prediction approaches can also be tried in different
and C4.5. So, we can conclude from the above table that if datasets to check the performance of the system.
we do the comparative study of all three algorithms, the
CART is best to choose. REFERENCE

Confusion Matrix: Research Papers

[1]. Sorower MS. A literature survey on algorithms for multi-label
learning. Oregon State University, Corvallis. 2010 Dec;18.
[2]. Utku A, Hacer (Uke) Karacan, Yildiz O, Akcayol MA.
Implementation of a New Recommendation System Based on
Decision Tree Using Implicit Relevance Feedback. JSW. 2015
Dec 1;10(12):1367-74.
[3]. Gershman A, Meisels A, Lüke KH, Rokach L, Schclar A, Sturm
A. A Decision Tree Based Recommender System. InIICS 2010
Jun 3 (pp. 170-179).
[4]. Jadhav SD, Channe HP. Efficient recommendation system using
Fig. 2 – Confusion matrix for ID3 decision tree classifier and collaborative filtering. Int. Res. J. Eng.
Technol. 2016;3:2113-8.
[5]. Beel J, Langer S, Genzmehr M, Nürnberger A. Introducing
Docear's research paper recommender system. InProceedings of
the 13th ACM/IEEE-CS joint conference on Digital libraries 2013
Jul 22 (pp. 459-460). ACM.
[6]. Zhang X, Jiang S. A Splitting Criteria Based on Similarity in
Decision Tree Learning. JSW. 2012 Aug;7(8):1775-82.
[7]. Bhargava N, Sharma G, Bhargava R, Mathuria M. Decision tree
analysis on j48 algorithm for data mining. Proceedings of
International Journal of Advanced Research in Computer Science
and Software Engineering. 2013 Jun;3(6).
[8]. Anyanwu MN, Shiva SG. Comparative analysis of serial decision
Fig. 3 – Confusion matrix for C4.5 tree classification algorithms. International Journal of Computer
Science and Security. 2009 Jun;3(3):230-40.

© 2018, IJCSE All Rights Reserved 77

 International Journal of Computer Sciences and Engineering Vol.6(10), Oct 2018, E-ISSN: 2347-2693

[9]. Freund Y, Mason L. The alternating decision tree learning

algorithm. Inicml 1999 Jun 27 (Vol. 99, pp. 124-133).
[10]. Pandey M, Sharma VK. A decision tree algorithm pertaining to the
student performance analysis and prediction. International Journal
of Computer Applications. 2013 Jan 1;61(13).
[11]. Priyama A, Abhijeeta RG, Ratheeb A, Srivastavab S. Comparative
analysis of decision tree classification algorithms. International
Journal of Current Engineering and Technology. 2013
Jun;3(2):334-7.
[12]. Anyanwu MN, Shiva SG. Comparative analysis of serial decision
tree classification algorithms. International Journal of Computer
Science and Security. 2009 Jun;3(3):230-40.
[13]. Quinlan JR. Induction of decision trees. Machine learning. 1986
Mar 1;1(1):81-106.
[14]. Drazin S, Montag M. Decision tree analysis using weka. Machine
Learning-Project II, University of Miami. 2012:1-3.
[15]. Banu GR. A Role of decision Tree classification data Mining
Technique in Diagnosing Thyroid disease. International Journal of
Computer Sciences and Engineering. 2016;4(11):111-5.
[16]. Jayakameswaraiah M, Ramakrishna S. Implementation of an
Improved ID3 Decision Tree Algorithm in Data Mining System.
International Journal of Computer Science and
EngineeringVolume-2, Issue-3 E-ISSN. 2014.

Books
[17]. Larose D.T. (2005), Discovering Knowledge in Data: An
Introduction to Data Mining, Wiley.
[18]. DATA MINING WITH DECISION TREES: Theory and
Applications (2nd Edition) by Lior Rokach and Oded Maimon.
[19]. Lior R. Data mining with decision trees: theory and applications.
World Scientific; 2014 Sep 3.

Authors Profile
Harsh H. Patel pursuing Bachelor in
Information Technology from Chandubhai S.
Patel Institute of Technology, CHARUSAT.
His area of interest is Data Mining.

Purvi Prajapati is working as an assistant

professor at Department of Information
Technology in Chandubhai S. Patel Institute
of Technology, CHARUSAT since June
2006. She had received the degree of Master
of Technology in Computer Engineering
from Chandubhai S Patel Institute of Technology,
CHARUSAT in 2012 and Bachelor of Engineering in
information technology from A D Patel Institute of
Technology in 2004. She has 11 years of teaching experience
including subject proficiency in Data Structure, Computer
Network, Data Mining, Language Processor and
Programming Languages (C, C++, Java, Python). Her
research interest includes data mining in machine learning.
She has published 09 papers in international journal and
conferences and also a member of ACM.

© 2018, IJCSE All Rights Reserved 78

View publication stats