A comprehensive survey on support vector machine

classification: Applications, challenges and trends

sciencedirect.com/science/article/abs/pii/S0925231220307153

,,,

https://doi.org/10.1016/j.neucom.2019.10.118 Get rights and content

Abstract
In recent years, an enormous amount of research has been carried out on support vector
machines (SVMs) and their application in several fields of science. SVMs are one of the
most powerful and robust classification and regression algorithms in multiple fields of
application. The SVM has been playing a significant role in pattern recognition which is an
extensively popular and active research area among the researchers. Research in some
fields where SVMs do not perform well has spurred development of other applications
such as SVM for large data sets, SVM for multi classification and SVM for unbalanced
data sets. Further, SVM has been integrated with other advanced methods such as
evolve algorithms, to enhance the ability of classification and optimize parameters. SVM
algorithms have gained recognition in research and applications in several scientific and
engineering areas. This paper provides a brief introduction of SVMs, describes many
applications and summarizes challenges and trends. Furthermore, limitations of SVMs
will be identified. The future of SVMs will be discussed in conjunction with further
applications. The applications of SVMs will be reviewed as well, especially in the some
fields.

Introduction
Machine Learning is a highly interdisciplinary field which builds upon ideas from cognitive
science, computer science, statistics, optimization among many other disciplines of
science and mathematics. In machine learning, classification is a supervised learning
approach used to analyze a given data set and to build a model that separates data into a
desired and distinct number of classes [1].

There are many good classification techniques in the literature including k-nearest-
neighbor classifier [2], [3], Bayesian networks[4], [5], artificial neural networks [6], [7], [8],
[9], [10], decision trees [11], [12] and SVM [13], [14], [15]. K-nearest-neighbor methods

1/8
have the advantage that they are easy to implement, however, they are usually quite slow
if the input data set is very large. On the other hand, these are very sensitive to the
presence of irrelevant parameters [2], [3].

Decision trees have also been widely used in classification problems. These are usually
faster than neural networks in the training phase, however, they do not have flexibility to
modeling the parameters [11], [12]. Neuronal networks are one of the most used
techniques [16], [17], [18], [19], [20]. Neural networks have been widely used in a large
number of applications as a universal approach. However, many factors must be taken
into account to building a neural network to solve a given problem: the learning algorithm,
the architecture, the number of neurons per layer, the number of layers, the
representation of the data and much more. In addition, these are very sensitive to the
presence of noise in the training data [21], [22].

From these techniques, SVM is one of the best known techniques to optimize the
expected solution [13], [15]. SVM was introduced by Vapnik as a kernel based machine
learning model for classification and regression task. The extraordinary generalization
capability of SVM, along with its optimal solution and its discriminative power, has
attracted the attention of data mining, pattern recognition and machine learning
communities in the last years. SVM has been used as a powerful tool for solving practical
binary classification problems. It has been shown that SVMs are superior to other
supervised learning methods [23], [24], [25], [26], [27], [28], [29]. Due to its good
theoretical foundations and good generalization capacity, in recent years, SVMs have
become one of the most used classification methods.

Decision functions are determined directly from the training data by using SVM in such a
way that the existing separation (margin) between the decision borders is maximized in a
highly dimensional space called the feature space. This classification strategy minimizes
the classification errors of the training data and obtains a better generalization ability, i.e.,
classification skills of SVMs and other techniques differ significantly, especially when the
number of input data is small. SVMs are a powerful technique used in data classification
and regression analysis. A notable advantage of SVMs lies in the fact that they obtain a
subset of support vectors during the learning phase, which is often only a small part of the
original data set. This set of support vectors represents a given classification task and is
formed by a small data set.

The rest of this paper is divided as follows: in Section 2 the theoretical basis of SVM are
presented; in addition, their characteristics, advantages and disadvantages are
described. In Section 3 weaknesses of SVM are introduced and reviewed. In Section 4 a
set of SVM implementations are presented. Section 5 shows some applications of SVM in
real world problems. Finally, Section 6 closes the paper with trends and challenges.

Section snippets

Theoretical basis of SVMs

2/8
The principal objective in pattern classification is to get a model which maximizes the
performance for the training data. Conventional training methods determine the models in
such a way that each input–output pair is correctly classified within the class to which it
belongs. However, if the classifier is too fit for the training data, the model begins to
memorize training data rather than learning to generalize, degrading the generalization
ability of the classifier.

The main motivation of SVM

Weaknesses of SVM
Despite the generalization capacity and many advantages of the SVM, they have some
very marked weaknesses, among which are: the selection of parameters, algorithmic
complexity that affects the training time of the classifier in large data sets, development of
optimal classifiers for multi-class problems and the performance of SVMs in unbalanced
data sets.

SVM implementations
Currently there are several implementations of SVM in the literature. Table 2 shows a list
of the most used SVM implementations. SVMs must solve a quadratic programming
problem to find a hyperplane that separates the classes. The main reason for multiple
implementations is because computational time depends mainly on the heuristics used to
divide the problem into small fragments. In small data sets, the computational time of the
SVMs is not important, however the computational complexity of the

Applications in real-world problems

SVM applications have been used to solve many real-world problems, in this section we
describe some of them.

Trends and challenges

Large amounts of data are generated and collected at each moment. The supervised and
unsupervised learning methods of machine learning are the responsible for transforming
these data into useful information. SVMs have proven to be one of the best supervised
learning methods in various applications; however, since SVM development several
challenging problems have been identified to be able to use this classifier with very large
data sets, also in dynamic environments such as data streams with

Conclusions
Due to its good theoretical foundations and generalization capacity among other
advantages, the SVMs have been implemented in many real-world applications. SVM
algorithms have been implemented in many research fields like: Text (and hypertext)

3/8
categorization, Protein fold and remote homology detection, Image classification,
Bioinformatics (protein classification and cancer classification), Hand-written character
recognition, Face detection, Generalized predictive control and many more. Many

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal
relationships that could have appeared to influence the work reported in this paper.

Jair Cervantes received the B.S. degree in Mechanical Engineering from Orizaba
Technologic Institute, Veracruz, Mexico, in 2001 and the M.S degree in Automatic Control
from CINVESTAV-IPN, México, in 2005. In 2009 he got a Ph.D. in Computer Science at
Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional
(CINVESTAV-IPN). His research interests include support vector machines, pattern
classification, neural networks, fuzzy logic, clustering and genetic algorithms.

References (303)
Y. Zhang et al.
A novel ensemble method for k-nearest neighbor

Pattern Recogn.

(2019)

B.G. Marcot et al.

Advances in bayesian network modelling: integration of modelling
technologies

Environ. Modell. Software

(Jan 2019)

M. Fratello et al.
Decision trees and random forests
Z.-Q. Zhao et al.
A mended hybrid learning algorithm for radial basis function neural
networks to improve generalization capability

Appl. Math. Model.

(2007)

4/8
 J.-X. Du et al.
Shape recognition based on neural networks trained by differential
evolution algorithm

Neurocomputing

(2007)

J. Du et al.
A novel full structure optimization algorithm for radial basis
probabilistic neural networks

Neurocomputing

(2006)

M. Zhang et al.
Supervised learning in spiking neural networks with noise-threshold

Neurocomputing

(Jan 2017)

L. Huang et al.
Impact of noise structure and network topology on tracking speed of
neural networks

Neural Networks

(Dec 2011)

X. Liang et al.
Multi-task ranking SVM for image cosegmentation

Neurocomputing

(Jul 2017)

S. Kasnavi et al.
The effect of kernel selection on genome wide prediction of discrete
traits by support vector machine

Gene Reports

(2018)

Cited by (583)

5/8
 Discrimination of wheat flour grade based on PSO-SVM of
hyperspectral technique
2023, Spectrochimica Acta - Part A: Molecular and Biomolecular Spectroscopy

A hybrid mental health prediction model using Support Vector

Machine, Multilayer Perceptron, and Random Forest algorithms
2023, Healthcare Analytics

A Convolutional Neural Network ensemble model for Pneumonia

Detection using chest X-ray images
2023, Healthcare Analytics

A performance analysis of dimensionality reduction algorithms in

machine learning models for cancer prediction
2023, Healthcare Analytics

Multi-resource fusion of nondestructive evaluation data for bridge

deck assessment using discrete wavelet transform and Dempster-
Shafer theory
2023, Measurement: Journal of the International Measurement Confederation

Solid waste management techniques powered by in-silico

approaches with a special focus on municipal solid waste
management: Research trends and challenges
2023, Science of the Total Environment

View all citing articles on Scopus

Recommended articles (6)

Research article

Special issue on advanced intelligent computing: Theory and

applications
Neurocomputing, Volume 408, 2020, p. 188

Research article

A novel selective naïve Bayes algorithm

Knowledge-Based Systems, Volume 192, 2020, Article 105361

Research article

Multi-task ranking SVM for image cosegmentation

Neurocomputing, Volume 247, 2017, pp. 126-136

6/8
 Research article

Decision Trees and Random Forests

Encyclopedia of Bioinformatics and Computational Biology, Volume 1, 2019, pp.
374-383

Research article

Neural network and support vector machine for the prediction of

chronic kidney disease: A comparative study
Computers in Biology and Medicine, Volume 109, 2019, pp. 101-111

Research article

Bagged support vector machines for emotion recognition from

speech
Knowledge-Based Systems, Volume 184, 2019, Article 104886

Jair Cervantes received the B.S. degree in Mechanical Engineering from Orizaba
Technologic Institute, Veracruz, Mexico, in 2001 and the M.S degree in Automatic Control
from CINVESTAV-IPN, México, in 2005. In 2009 he got a Ph.D. in Computer Science at
Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional
(CINVESTAV-IPN). His research interests include support vector machines, pattern
classification, neural networks, fuzzy logic, clustering and genetic algorithms.

Farid García-Lamont received a B.Sc. degree in Robotics from ESIME-IPN, México in

2000; in 2004 obtained a M.Sc. in Automatic Control from the Centro de Investigación y
de Estudios Avanzados del IPN (CINVESTAV-IPN), México. In 2010 received a PhD
degree in Computer Science from CINVESTAV-IPN. His research interests are pattern
recognition, applications of artificial intelligence and robotics.

7/8
Lisbeth Rodríguez-Mazahua received the B.S. degree in informatic and the M.S. degree
in computer science from the Instituto Tecnológico de Orizaba, Veracruz, Mexico, in 2004
and 2007, respectively. In 2012 she got a Ph.D. in Computer Science at Centro de
Investigación y de Estudios Avanzados del Instituto Politécnico Nacional (CINVESTAV-
IPN). From 2012 to 2014, she was a Professor of computer science at Universidad
Autónoma del Estado de México, Centro Universitario UAEM Texcoco. Since February
2014 she is doing a postdoctoral research at Instituto Tecnológico de Orizaba. Her
current research interests include distribution design of databases, database theory,
autonomic database systems, multimedia databases, data mining and Big Data.

Asdrúbal López-Chau, received his B.S. degree in Communications and electronics

from ESIME-IPN, Mexico, in 1997; in 2000, received his M.S. the degree in Computer
Engineering from CIC-IPN, Mexico; in 2013, his Ph.D. degree in Computer Science from
CINVESTAV-IPN, Mexico. Since 2009, he is a computer science researcher at
Universidad Autonoma del Estado de Mexico, CU UAEM Zumpango, Mexico.
View full text
© 2020 Elsevier B.V. All rights reserved.

Copyright © 2023 Elsevier B.V. or its licensors or contributors. ScienceDirect® is a

registered trademark of Elsevier B.V.

8/8