2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)

Comparative Analysis of Multi-label Classification

Algorithms

Seema Sharma Deepti Mehrotra

Amity University Amity University
Uttar Pradesh, India Uttar Pradesh, India
[email protected]

Abstract—Multi-label classification has generated multi-label classification and its usage in different
enthusiasm in many fields over the last few years. It allows the applications can be found in [17].
classifications of dataset where each instance can be associated
with one or more label. It has successfully ended up being The objective of taking this research work is the analysis
superiorstrategy as compared to Single labelclassification. In and application of Multi-label classification algorithms on
this paper, we provide an overview of multi-label classification undergraduate student dataset.
approaches. We also discussed the various tools thatutilizes
This research is undertaken to comprehend the effect of
MLC approaches. Lastly, we have presented an experimental
study to compare different algorithms of multi-label
Graduate student personality traits like proficiency in English
classification. After applying and studying the accuracies of & expressiveness, participation in extra co-curricular
various multilabel classification techniques, we have found that activities along with their academic performance on getting
performance of Random Forest is better than the rest of the successful Campus placement and chances of perusal of
other compared multilabelclassification algorithms with 96% higher studies. College management could use the results to
accuracy. think ahead and provide the appropriate career guidance to
students. This study was initiated to figure out what fits best
Keywords— Multi-label classification (MLC), Multi-label for the student career path.
dataset (MLD), Single-label, Problem transformation, Algorithm
adaptation, Ensemble. The paper has been organized as per below structure:
after the introduction in section I, section II details the Multi-
I. INTRODUCTION label classification, Section III highlights challenges faced by
Presently, there is a colossal amount of information being MLC, Section IV discusses tools available for MLC, section
gathered and put away in databases throughout the globe. So Vdiscusses the Experimental Study & Results achieved,
what do we do of this information? The appropriate Section VI provides conclusion and the last section refers to
response lies in the idea of data mining. Data mining is a the References.
computational procedure of extraction of information from II. MULTILABEL CLASSIFICATION
considerable datasets to recognize patterns inside the mass
data. Over the years numerous techniques have been “Multi-label classification can be defined as the problem
developed to process and extract conclusions from the vast of finding a model that maps input A to binary vector B,
datasets. Some of the techniques are classification, rather than scalar outputs as in case of single label
clustering, regression and association. In this paper, we will classification problem.
concentrate on the classification techniques, particularly on
Initially inspired by text categorization and medical
the multi-label classification.
diagnosis, multi-label classification is now expanding its
Classification is a supervised learning technique which horizon in other fields like audio categorization,
provides label to instances in the given dataset based on bioinformatics and many more. MLC has successfully gained
training data. This classification can be categorized into popularity in the recent times because of its ability to solve
single label and multi-label. real-time problems. For instance, single-label classification
can label an email message as work, or research but not both,
Single label classification associates one label with each where the truth of the matter is, it could be tagged as both
instance whereas multi-label classification associates a research and work. MLC labels it as both, as it contributes
subset of labels with each instance. Multi-label classification towards the association of the object to one or more classes.
ought not to be mistaken for the multiclass arrangement. In This property of multi-label classification makes it more
multiclass classification the object cannot simultaneously competitive than the previously used classification approach.
belong to more than one class. For example, in scenery MLC methods can be divided into three types.
theobject belongs to the class ’sky’, class ‘water’, class ‘bird’
all at the same time. Multi-label classification has an edge A. Problem Transformation Method
over generally utilised classification techniques as multi-
label classification deals with the association of each B. Algorithm Adaptation Method
instance with one or more classes or labels. This paper aims C. Ensemble methods
at giving a brief overview of multi-label classification
approaches and its challenges. Further, we have compiled an Problem Transformation method transforms the multi-
experimental study in which we have compared different label classification problem into a set of binary classification
MLC approaches. A recent survey on the advancement of problem, which is then handled using single-class classifiers.
Algorithm Adaptation method merely modifies single label
classifiers into multi-label classification.An ensemble

978-1-5386-6373-8/18/$31.00 ©2018 IEEE 35

 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)

method uses either algorithm adaptation or problem dependency

transformation asbaseclassifiers. between
features
In this paper, the focus is on some of the problem and labels
transformation, algorithm adaptation and ensemble multi- is resolved
iteratively[
label classifications approaches. Table 1 gives a brief 19]
comparison of these approaches. Rando Ensemble First- Does not Classification
m method order suffers interpretation is
TABLE 1. COMPARISON OF VARIOUS MULTI-LABEL CLASSIFICATION Forest from the difficult
METHOD [7,3, 5] problem of
over-fitting.
Algori Basic Order of Pros Cons
thm Concept correlati
ons III. CHALLENGES FACED BY MULTILABEL
BR Problem first- Fast, simple Does not take CLASSIFICATION
trnsformati order & easy to consideration of
on method. implement correlations
MLC has been used widely in many domains in the last
Learns l among labels. few years. Although researchers have developed many
number of Also suffers the algorithms to train multi-label classifiers, there are still some
classifiers problem of class challenges that need to be investigated in depth[6,15].
(one imbalance Below are the listofkey challenges.
classifier
for each A. High dimensionality of data emerged to be the most
label) challengeable issue in MLC[15]. A large size of features
where l is
increases the complexity of algorithms. Classifiers must
the total
number of consider a significant feature selection method to achieve
labels in the high classification accuracy.
dataset. Aconsiderable amount of attention must be given to the
Divides the optimised multi-label feature selection method [7]. It would
dataset into be better if feature selection methods consider correlations
n
datasets[4] among label and label specific features .
CC Problem high- Maintains Could not use B. Challenge of learning from multi-label dataset is to
trnsformati order computatio unlabeled data
on method. nal for classification
cope up with large size of the output set, i.e.number of label
Learns l complexity set grows exponentially as the number of class labels
classifiers of BR. increases. It would beessential to exploit high order
as in the considers correlations among labels with linear complexity to facilitate
case of BR label the learning process [9,11,14].
but all the dependency
classifiers C. Dealing with the missing & noisy labels [10].
are chained
together. D. In many real-lifescenarios,there would be a need for
[18] hierarchical multi-label classification. In applications like
Label Problem high- Considers Suffers from text categorisation, functional genomics the labels could be
Power trnsformati order correlation overfitting of
set on method. among data,
organised hierarchically. Different structures like a tree or
Assign one labels Complex and directed acyclic graph could be used to represent the
class to expensive to hierarchical nature of the dataset. In tree-likestructure, each
each unique compute label has a single parent whereas in DAG each label can have
occurrence more than one parent [12].
of label set.
Rando Ensemble high- More High E. Challenge with stratification. Stratified sampling is a
m K- of LP[2] order predictive computational method of dividing the target population into two or more
label capability complexity. groups based upon some common attributes. Stratification
sets Considers Balanced
label training set & ensures that the chosen sample is a representative of the
correlation Predicts unseen target population.
Overcome label sets Stratified sampling is easy to implement in a single-label
drawback classification since every instance is labeled with only one
LP’s with mutually exclusive label. Whereas in MLC, the stratification
less number
of unique
is challenging to implement [13] [15].
label- sets F. Learning from the imbalanceddataset is another issue
ML- Algorithm first- Improved Lazy learner
kNN adaptation order performanc
innately associated with multi-label classification[16].
based e Firstly, Due to the nature of MLDs there is a variation of
ML- Algorithm first- Splits data Cannot consider label distribution. In an MLD with a large number of labels,
DT adaptation order based on correlation it is natural that few of them are much more frequent than
based important among labels others. Secondly, due to the transformation applied to
features MLDs. Problem transformation approach for instance in the
SVM- Algorithm second- Considers Reduced
HF adaptation order correlation accuracy due to
case of BR, it causes the imbalance of labels.
based among unlabeled data
cyclic classes. consideration.

36
 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)

IV. TOOLS AVAILABLE FOR MULTI-LABEL TABLE I. DATASET ATTRIBUTES

CLASSIFICATION # Attribute Format # Attribute Format

1 10th Marks Integer 5 Participated in Integer
A. MULAN:Mulan is the most evolved multi-label extra curricular
programming and presumably the most generally utilized activites
open source library. It is written in Java and is built upon 2 12th Marks Integer 6 Proficiency in Boolean
WEKA. It gives no graphical user interface for the English
development and only provides API for the library users for 3 Theory Integer 7 Project Integer
the programming. Percentage Outcome
MULAN supports most of the advanced algorithms for 4 Practical Integer
multi-label classification and label ranking. . Percentage
Additionally it offers simple baseline feature selection
methods for dimensionality reduction. It also provides Table III gives some of the basic stats of dataset, for
methods to calculate a large variety of multi-label evaluation instance total number of records,attributes number, the label
measures through hold-out evaluation and cross-validation. density and the label cardinality.Label cardinality(LC) is the
average number of labels per instance, while label density
B. MEKA: Like MULAN, MEKA is built on WEKA. (LD)is the label cardinality divided by total labels in dataset.
In contrast to MULAN, MEKA provides a graphical user
interface which is very much similar to WEKA. MEKA TABLE II. MULTI- LABEL DATASET STATISTICS
provides a framework for the development, execution and
Domain # of Instances # attributes LC LD
evaluation of multi-label classifiers [1].
Academics 493 7 0.482 0.241
C. R: It is the most frequently used tool for data
B. Multi-label evaluation measure
analysis. It uses mldr package to load, amend and to
analyses ML dataset and supports only binary multilabel Let M be a MLD, containing |M| multilabel instances (ti ,
classifiers only. R is easily extendable through functions. It Ai), i = 1. |M|, Ai is subset of L (label Set). Let c be a
also provides the way for the development of new ML multilabel classification model and Pi = c(ti) is label sets
classifiers. predicted by c for instance ti.
D. Scikit-multilearn: Scikit-multilearn is Python-based x Accuracy is an example based measure that
module for multi-label classification. It provides native computes the proportion of prediction of correct labels
Python implementation of various ML classifiers. It also among all predicted and accurate labels[8]. In this study
provides a wrapper class for Meka/Mulan to access all of Random forest algorithm provides more accuracy.
their functions which is not yet implemented in Scikit- |M|
multearns. 1 |Ai∩Pi|
Accuracy(c,M)= ෍
V. EXPERIMENTAL STUDY |M| |Ai Pi|
i=1
A. Dataset x Hamming Loss is also an example based
Our exploratory study has utilised multi-label dataset of measurethat determines the proportion of wrongly
undergraduate Engineering students collected from a private classified instances i.e. correct label which actually
university. belong to the instance is not predicted. If the value of
hamming loss is small, the performance of MLC
Table IIsummarizes the dataset attributes.In this study, algorithm will be considered better.
the primary focus is on two target variables |M|
i.e.HigherEducation and Ccampusplacement.Higher 1 |Ai∩Pi|
Education shows the interest of student in higher education Hamming Loss(c,M)= ෍
|M| |L|
after graduation whereas campus placement shows whether i=1
student is seeking campus placement or not. x F1 is an example based measure that calculates
harmonic average between recall and precision. Its value
The Five MLC algorithms were compared &analyzed is a mean over all examples in the dataset. F1 value lies
using MEKA tool for this dataset. MEKA is a java based between 1 and 0 where 1 is considered to be best value
open source multi-label tool which is built on WEKA and 0 is worst.
software.
For applying the algorithms, the dataset was arbitrarily
split into two sets; one partwas used to train the classification TABLE III. EVALUATION MEASURE FOR MLC ALGORITHM
model and second onewas used for testing the model.
Dataset had 493 records of students and it was split into Evaluation Metric BR CC PS LC Random forest
training data set having 66% of records and and testing data
Accuracy 0.839 0.839 0.836 0.836 0.964503
set containing having 44% records.
Hamming Loss 0.086 0.086 0.092 0.092 0.03144
For the unbiased estimate of the five prediction models, a
10-fold cross validation method was used. F1 0.82 0.82 0.803 0.803 0.972955
TableIVcomparesthe performanceof Binary Relevance(BR), Fig. 5.1 is the graphical representation of performance
Classifier Chains(CC), Pruned Set(PS), Label powerset(LC) measure for classification algorithms.
& Random forest multilabel classification algorithms.

37
 2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)

[14] Tsoumakas, G., & Vlahavas, I., Sechidis, K., (2011), “On the
stratification of multi-label data,” Machine Learning and Knowledge
Discovery in Databases, 145-158.
[15] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. 2008,
“Effective and efficient multilabel classification in domains with large
number of labels,” In Proceedings of the ECML/PKDD 2008
Workshop on Mining Multidimensional Data (MMD’08)
[16] Raed Alazaidah, Farzana Kabir Ahmad,“Trending Challenges in
Multi Label Classification,” (IJACSA) International Journal of
Advanced Computer Science and Applications,Vol. 7, No. 10, 2016
[17] Charte, F., Rivera, A.J., del Jesus, M.J., Herrera, F.,“MLSMOTE:
approaching imbalancedmultilabel learning through synthetic instance
generation. Knowl.-Based Syst,” Elsevier. 89, 385–397(2015)
Fig. 5.1. Comparison of MLC algorithm [18] Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas.“Mining
Multi-label Data. O.Maimon, L. Rokach (Ed.),” Springer, 2nd edition,
VI. CONCLUSION 2010.(1-20)
[19] Jesse Read, Bernhard Pfahringer, Geoffrey Holmes, Eibe Frank,
The paper reviewed multi label classifications and “Classifier chains for multi-label classification,” In Proceeding(s) of
additionally highlighted how different algorithms are utilized the ECML/PKDD, volume 5782 , pages 254–269. Springer, 2009
through an experimental study to solve problems. Using the [20] Yuhong Guo , Suicheng Gu,“Proceedings of the Twenty-Second
MEKA tool,algorithms of BR, CC, PS, LS and Random International Joint Conference on Artificial Intelligence”number of
Forest were applied. Conclusion of the algorithm labels,” In Proceedings of the ECML/PKDD 2008 Workshop on
applications that the Random Forest offers superior Mining Multidimensional Data (MMD’08)
approach. Random Forest performance was followed up by
BR, CC,PS and LS. We observed that multi-label
arrangement although offers superior progressions, though it
faces many difficulties also.
REFERENCES
[1] Read, J., Reutemann, P., Pfahringer, B., & Holmes, G. (2016).
“MEKA: A multi-label/multi-target extension to WEKA,” Journal of
Machine Learning Research, 17(21), 1–5.
[2] Asma Aldrees and Azeddine Chikh and Jawad Berri,“Comparative
Evaluation of Four Multi-labels ClassificationAlgorithms in
Classifying Learning Objects,” David CWyld etal. (Eds): CCSEA,
CLOUD, DKMP, SEA, SIPRO – 2016.
[3] Herrera, F., riveria, A.J. del Jesus, M.J ,”Multilabel Classification
Problem Analysis, Metrics and Techniques,” 2016,XVI, 194 P. 72
Illus., Hardcover ISBN:978-3-319-4110-1
[4] Jadon Mayurisingh Nareshpalsingh1, Prof. Hiteshri N. Modi,” Multi-
label Classification Methods:A Comparative Study,” International
Research Journal of Engineering and Technology (IRJET), Volume:
04 Issue: 12 | Dec-2017.
[5] Zeynep Ceylan, Ebru Pekel,”Comparison of Multi-Label
Classification Methods for Prediagnosis of Cervical Cancer,” IJISAE,
2017, 5(4), 232-236.
[6] M. Zhang, Z. Zhou, “A review on multi-label learning algorithms,”
IEEE Trans. Knowl. Data Eng. 26 (8) (2014) 1819–1837,
doi:10.1109/TKDE.2013.39.
[7] EVA GIBAJA, SEBASTIA´N VENTURA,“A Tutorial on Multi-
Label Learning,” ACM Computing Surveys, Vol. 9, No. 4, Article 39,
Publication date: March 2010
[8] Hyunki Lim, Jaesung Lee, Dae-Won Kim,”Optimization approach for
feature selection in multi-label classification,” Elsevier,2017
[9] Slovenia G. Tsoumakas, M.-L. Zhang, and Z.-H. Zhou, “Tutorial on
learning from multi-label data,” in ECML PKDD, Bled, ,2009
[10] M.-L. Zhang and K. Zhang, “Multi-label learning by exploiting label
dependency,” in Proc. 16th ACM SIGKDD Int. Conf.
KDD,Washington, DC, USA, 2010, pp. 999–1007.
[11] Amirhossein Akbarnejad, Mahdieh Soleymani Baghshah, “A
probabilistic multi-label classifier with missing and noisy
labelshandling capability,” 0167-8655/© 2017
[12] Wei Wenga,b , Yaojin Linc , Shunxiang Wua , Yuwen Li a , Yun
Kanga , “Multi-label learning based on label-specific features and
local pairwise label correlation,” 0925-2312/© 2017 Elsevier
[13] Wei Bi , James T. Kwok, “Multi-Label Classification on Tree- and
DAG-Structured Hierarchies,” International Conference on Machine
Learning, Bellevue, WA, USA, 2011

38