SURVEY ON “CLASSIFICATION TECHNIQUES”

Harshit Gupta Pratik

Dept. of Computer Science Dept. of Computer Science
RVCE, Bangalore, India RVCE, Bangalore, India

Abstract -The data mining has a basic patterns or relationships describing the
principle for analysing the data from data), and predictive (i.e., predicting or
different angles and categorize it and finally classifying the behaviour of the model
to condense it. In today’s world data mining based on available data). It is an
have increasingly become very interesting
interdisciplinary field with a general goal of
and popular in terms of all application. The
Data Mining refers to extracting or mining
predicting outcomes and uncovering
knowledge from huge volume of data. relationships in data. Some of the data
Classification is an important data mining mining techniques are Classification,
technique with broad applications. It Clustering and Rule Mining.
classifies data of various kinds.
Classification is used in every field of our life. Classification techniques in data mining are
Classification is used to classify each item in capable of processing a large amount of
a set of data into one of predefined set of data. It can be used to predict categorical
classes or groups. In this paper different class labels and classifies data based on
classification techniques of Data Mining are
training set and class labels and it can be
compared using diverse datasets from
University of California, Irvine (UCI). used for classifying newly available data.
Accuracy and time required for execution by The aim of this study is to investigate the
each technique is observed. This paper
performance of different classification
provides an inclusive survey of different
classification algorithms (Decision Tree, algorithms: Decision Tree Induction, Naïve
Naïve Bayes, Support Vector, J48, Logical Bayesian Classification, Support Vector
analysis of data, k-NN and Radical Basis Machine, LAD Tree Algorithm, k-NN
Function Network) and their features and classification, J48-graft algorithm, Radial
limitations. Basis Function Network and Naïve Bayes
classifier using Apriori algorithm.
INTRODUCTION
Data mining refers to extracting or
DIFFERENT CLASSIFICATION
“mining” knowledge from large volume of
TECHNIQUES
data. But in broadly data mining can be
Classification Techniques varies with
defined as the task of extracting implicit,
different datasets, attributes, tuples and
non-trivial, previously unknown potential
system configuration so classification
useful information or pattern from data in algorithms accuracy depends on type of
large databases. Data mining is “the process datasets (bivariate and multivariate) and
of using variety of data analysis tools to platform used (WEKA, MATLAB 2016,
discover patterns and relationships in data SAS etc.).
that may be used to make valid
predictions”. Data mining tasks can be  DECISION TREE INDUCTION
It is the hierarchical decision
descriptive, (i.e., discovering interesting
making approach used to partition
 the dataset. It is an approximate is used to solve problems,
discrete function technique for recognition and classification of
retrieving useful expressions. data among the different categories
According to this approach, the of data, and as soon as possible this
dataset is classified to n mutual technique can form the model that
exclusive datasets and each dataset lead to issues such as classification
will be defined with a label. Now with 2 or more classes.
the data points are identified by a
certain action to its relative data  SUPPORT VECTOR MACHINE
class. Decision Tree is a supervised Support vector machine are
learning technique that defines a basically binary classification
transparent tree based structure to algorithms. The basic support
perform the action on available vector machine takes a set of input
dataset. The tree structure itself data and predicts, for each given
contains number of nodes that are input, which of two possible classes
connected by the edges. These forms the output, making it a non-
nodes basically define the probabilistic binary linear classifier.
conditions and the edges define the Given a set of training examples,
relative event based on the each marked as belonging to one of
condition. Each edge itself defines a two categories, a support vector
separate class. The decision is here machine training algorithm builds a
being taken respective to the true or model that assigns new examples
false case of a conditional analysis. into one category or the other.
If the condition is true, one category
will be elected otherwise second  LAD TREE ALGORITHM
Logical Analysis of Data (LAD)
categorization.
tree is the classifier for binary target
variable based on learning a logical
 NAÏVE BAYESIAN
CLASSIFICATION expression that can distinguish
This algorithm is derived from the between positive and negative
Bayesian theory that is based on the samples in a data set. The central
probability of occurrence or non- concept in LAD tree algorithm is
occurrence of a phenomenon for that of classification, clustering, and
classification. Based on the intrinsic other problems. The construction of
properties of the possibility LAD model for a given data set
(especially Share probability), typically involves the generation of
Naive Bayes will provide good large set patterns and the selection
results with received initial training. of a subset of them that satisfies the
In Naive Bayes‚ the method of above assumption such that each
learning is a kind of learning by pattern in the model satisfies certain
observer and controller. In fact, this requirements in terms of prevalence
technique can be formed models for and homogeneity.
classification and prediction of
several purposes. Also Naive Bayes
 k-NN CLASSIFICATION to discover association rules
The output is a class membership. between items in large database sale
An object is classified by a majority transactions. Proposed algorithm
vote of its neighbours, with the incorporates the frequent item idea
object being assigned to the class which effectively increases the
most common among its k nearest overall accuracy. Not only
neighbours. If k = 1, then the object considered each and every word as
is simply assigned to the class of independent and mutually exclusive
that single nearest neighbour. but also frequent words as a single,
independent and mutually
 J48-GRAFT ALGORITHM exclusive.
J48-graft algorithm generates a
grafted decision tree from a J48 tree DIFFERENT COMPARISION
algorithm. The grafting technique is PARAMETERS
an inductive process that adds nodes To analyse all the algorithms following
to inferred decision trees. The parameters are used:
grafting technique is an inductive  Kappa statistic
process that adds nodes to inferred The Kappa Statistic can be defined
decision trees with the purpose of as measuring degree of agreement
reducing pre-diction errors. between two sets of categorized
data (reliability of the data collected
 RADIAL BASIS FUNCTION
and validity). Kappa result varies
NETWORK
A radial basis function network is between 0 to 1 intervals. Higher the
an artificial neural network that uses value of Kappa means stronger the
radial basis functions as activation agreement/ bonding. If Kappa = 1,
functions. The output of the then there is perfect agreement. If
network is a linear combination of Kappa = 0, then there is no
radial basis functions of the inputs agreement. If values of Kappa
and neuron parameters. Radial basis statics are varying in the range of
function networks have many uses, 0.40 to 0.59 considered as
including function approximation, moderate, 0.60 to 0.79 considered
time series prediction, as substantial, and above 0.80
classification, and system control. considered as outstanding.

 MAE (Mean Absolute Error)

 NAÏVE BAYES CLASSIFIER Mean absolute error can be defined
USING APRIORI ALGORITHM as sum of absolute errors divided by
Naïve Bayes is considered as one of number of predictions. It is measure
the most effectual and significant set of predicted value to actual value
learning algorithms for machine i.e. how close a predicted model to
learning and data mining and also actual model.
has been treated as a core technique
in information retrieval. Apriori
Algorithm Frequent Itemset is used
  RMSE (Root Mean Square tasks are supported by Weka. It is an
Error) open source application which is
Root mean square error is defined as freely available.
square root of sum of squares error In Weka datasets should be
divided number of predictions. It is formatted to the ARFF format. The
measure the differences between Weka Explorer will use these
values predicted by a model and the automatically if it does not
values actually observed. Small recognize a given file as an ARFF
value of RMSE means better file. Classify tab in Weka Explorer
accuracy of model. So, minimum of is used for the classification
RMSE & MAE better is prediction purpose. A large different number
and accuracy. of classifiers are used in weka such
as Bayes, function, tree etc. WEKA
 RAE (Relative Absolute Error) is a collection of machine learning
Relative absolute error is very algorithms for data mining tasks.
similar to the relative squared error The algorithms can either be applied
in the sense that it is also relative to directly to a dataset or called from
a simple predictor, which is just the our own Java code.
average of the actual values. In this
case, though, the error is just the  MATLAB 2016
total absolute error instead of the CLASSIFICATION LEARNER
total squared error. Thus, the APP
relative absolute error takes the total The Classification Learner app
absolute error and normalizes it by trains models to classify data. Using
dividing by the total absolute error this app, you can explore supervised
of the simple predictor. machine learning using various
classifiers. You can explore your
DIFFERENT DATA ANALYTICS data, select features, specify cross-
TOOLS validation schemes, train models,
and assess results. You can perform
 WEKA automated training to search for the
The full form of WEKA: Waikato best classification model type,
Environment for Knowledge including decision trees,
Learning. Weka is a computer discriminant analysis, support
program that was developed by the vector machines, logistic
student of the University of Waikato regression, nearest neighbours, and
in New Zealand for the purpose of ensemble classification.
identifying information from raw You can perform supervised
data gathered from agricultural machine learning by supplying a
domains. Data pre-processing, known set of input data
classification, clustering, (observations or examples) and
association, regression and feature known responses to the data (e.g.,
selection these standard data mining labels or classes). You use the data
to train a model that generates
 ACCURACY
predictions for the response to new 86 84.0741
data. To use the model with new 84
data, or to learn about programmatic 82
classification, you can export the 80

Accuracy %
77.6
model to the workspace or generate 78 76.6667 76.3 76.5 76.08
75.1852
MATLAB® code to recreate the 76 74.34
74
trained model.
72
70
68
RESULT & ANALYSIS 1 2 3 4 5 6 7 8

Kappa statistics, Mean Absolute Error, Figure 1 Graphical Representation of measured accuracy
Root Mean Square Error, Relative Absolute CONCLUSION
Error and accuracy parameters are used to In this paper, the analysis of different
judge classification algorithms. These classification algorithms has been
reading taken with a medical dataset performed under five different parameters
classification. Table below shows analysis: called RSME, Relative absolute error, mean
absolute error, kappa statistic and accuracy
analysis for medical datasets. The obtained
KAP MA RM RAE ACCUR results show that the SVM is the most
PA E SE ACY robust, consistent and reliable classification
DT 0.527 0.03 NIL NIL 76.6667 algorithm whereas RBFN is the worst
1 algorithm for the classification in this
SVM 0.676 0.08 0.38 82.73 84.0741 situation.
2 + +
NB 0.107 0.26 0.36 93.263 76.30
4 77 41 3 REFERENCES
J48 0 0.28 0.37 98.172 76.5 [1] http://en.wikipedia.org/data-mining.
18 54 4 [2] Han, J., Kamber, M.: Data Mining;
LAD NIL 0.31 0.40 70.08 76.08 Concepts and Techniques, Morgan Kaufmann
+ + + Publishers (2000).
RBFN NIL 0.35 0.42 79.49 74.34
+ + + [3] Tapas Ranjan Baitharu, Subhendu Kumar
KNN 0.498 0.00 NIL NIL 75.1852 Pani, A Survey on Application of Machine
8 3 Learning Algorithms on Data Mining,
NB+AA NIL NIL NIL NIL 77.6 International Journal of Innovative Technology
FI and Exploring Engineering (IJITEE) ISSN:
2278-3075, Volume-3, Issue-7, December
2013.
[4] Rahman, Rashedur M., and Farhana Afroz.
Plotting accuracy measured of "Comparison of various classification
classification algorithms: techniques using different data mining tools for
diabetes diagnosis." Journal of Software
Engineering and Applications 6.03 (2013): 85.
[5] Mittal, Pooja, and Nasib Singh Gill. "A
Comparative Analysis of Classification
Techniques on Medical Data Sets."
International Journal of Research in
Engineering and Technology 3.06 (2014): 454-
460.
[6] Upadhyay, Nitya, and Vinodini Katiyar. "A
Survey on the Classification Techniques in
Educational Data Mining." International
Journal of Computer Applications Technology
and Research 3 (2014): 725-728.
[7] Vaithiyanathan, V., et al. "Comparison of
different classification techniques using
different datasets." International Journal of
Advances in Engineering & Technology 6.2
(2013): 764.
[8] G. Parthiban, A. Rajesh, S.K.Srivatsa,
“Diagnosis of Heart Disease for Diabetic
Patients using Naïve Bayes Method”,
International Journal of Computer
Applications,Vol. 24, No.3, pp. 7–11, June
2011.