Cyber Threat Detection Using Machine Learning

Techniques: A Performance Evaluation Perspective

Kamran Shaukat, Suhuai Luo, Shan Chen Dongxi Liu,
The University of Newcastle, Australia Data61, Commonwealth Scientific and Industrial Research
[email protected] Organization, Australia

Abstract — The present-day world has become all dependent security which contributes to strengthening cybercrimes,
on cyberspace for every aspect of daily living. The use of including unqualified users, the weak configuration of system
cyberspace is rising with each passing day. The world is spending resources and limited access to clean data [3]. The future of
more time on the Internet than ever before. As a result, the risks cyber security is all about automated cyber security. Advanced
of cyber threats and cybercrimes are increasing. The term 'cyber and automated cyber security techniques are highly needed.
threat' is referred to as the illegal activity performed using the They possess the ability to learn from experience to detect
Internet. Cybercriminals are changing their techniques with new polymorphic cyberattacks to keep pace with the evolving
time to pass through the wall of protection. Conventional cybercrimes [4].
techniques are not capable of detecting zero-day attacks and
sophisticated attacks. Thus far, heaps of machine learning The cyber threat is an act in which someone will try or
techniques have been developed to detect the cybercrimes and attend to steal the information, violate the integrity rules and
battle against cyber threats. The objective of this research work harm the computing device or network. Cyber threats include
is to present the evaluation of some of the widely used machine phishing, malware, attack on IoT devices, denial of service
learning techniques used to detect some of the most threatening attack, spam, intrusion on network or mobile device, financial
cyber threats to the cyberspace. Three primary machine learning fraud, ransomware, to name a few [5, 6]. Malware detection,
techniques are mainly investigated, including deep belief intrusion detection and spam detection are discussed in this
network, decision tree and support vector machine. We have paper.
presented a brief exploration to gauge the performance of these
machine learning techniques in the spam detection, intrusion An email that is unwanted or unsolicited is called spam
detection and malware detection based on frequently used and email. Spam emails are mostly used for advertisement or
benchmark datasets. spreading fraudulent material. It occupies the network and
computer resources such as the bandwidth of network,
Keywords— Cyber Threat; Cybercrime; Performance memory and wastage of time [7]. Another cyber threat is
Evaluation; Machine Learning Application; Intrusion Detection malware. Malware, as a short for malicious software, is a
System; Malware Detection; Spam Classification software that is installed on a computer to disrupt its operation
and harm the electronic data. Viruses, worms, ransomware,
I. INTRODUCTION adware, spyware, malvertising, and Trojan horse are
The cyberspace refers to the global environment that considered as significant types of malware [8]. Malign
facilitates the sharing of electronic resources from all over the intrusions over the computer network and devices are another
world. Resources can be an electronic document, audio, video, cyber threat to cyberspace. These intrusions are used to
image, and tweet. The cyberspace incorporates a wide range identify and scan the vulnerabilities of a network or computer
of components, including the Internet, technically skilled system. An intrusion detection system (IDS) is used to protect
users, system resources, data and untrained users. The against these intrusions. There are three classifications of
cyberspace is providing a global arena to infinitely gain access intrusions, namely, signature/misuse-based, anomaly-based
to information and resources. At present, the cyberspace is and hybrid [9, 10].
playing the leading role in data transfer and information Machine learning (ML) is the most effective and
exchange with all its vastly growing losses and gains. After fundamental strategy to compete against cyber threats and
2017 the cyberspace gained more popularity. Internet usage overcome the limitations of conventional security systems
has risen 81% in developed countries and still growing all [11]. Despite having all its charms, machine learning
over the globe [1]. The elevating cyberspace has also given techniques have their constraints and limitations. Machine
rise to the risks of cybercrimes and cyber threats. learning is a subclass of artificial intelligence (AI) [12]. The
With the growing range of cyber threats, cyber security has fascinating quality of machine learning techniques is that
also made a considerable number of enhancements to compete machine learning techniques do not need to be explicitly
against cybercrimes. The cyber security refers to a set of programmed as they can automatically learn from their
technologies, technology experts and processes that are used experience to generate the results [13].
to make safety measures to protect the cyberspace from On the strength of all the benefits of machine learning
cybercriminals [2]. There are two main approaches of cyber techniques, ML techniques are expanding their scope in
security, i.e., conventional cyber security and automated cyber almost every area of life, including cyber security [14],
security. There are numerous downsides of conventional cyber

978-1-7281-6840-1/20/$31.00 ©2020 IEEE

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.
medical science [15, 16], educational purposes [17, 18], Authors in [32] came up with a statistical semi-supervised
intrusion detection [19, 20], spam detection [21, 22] and machine learning technique for intrusion detection in Android
malware detection [23]. Almost all famous machine learning mobile devices. The increase in data traffic will also give rise
techniques have been applied to detect and classify different to cybercrimes. Consequently, to protect Android mobile
cyber threats. Commonly used machine learning techniques devices against advanced cybercrimes, more advanced
are decision tree, random forest, naive Bayes, support vector machine learning techniques are needed to be developed to
machine, K-nearest neighbor, deep belief network, artificial detect malicious activities.
neural network, K-mean, to name a few [24, 25]. However, we
In this paper, we have provided a comprehensive review of
have considered the decision tree, deep belief network, and
widely used machine learning techniques to gauge the
support vector machine techniques for this article. We have
performance of machine learning techniques to detect some
provided a comparison of machine learning techniques based
widely known cybercrimes. We have analyzed three widely
on frequently used and benchmark datasets.
used machine learning techniques, namely: decision tree, deep
belief network and support vector machine. Most of the
II. LITERATURE REVIEW review articles only focused on a particular threat. However,
Authors in [26] analyzed the applications of widely used we have considered three major cyber threats. An intrusion
machine learning techniques to protect the cyberspace from detection, spam detection and malware detection are
cybercriminals. The authors also depicted various obstacles considered for this study. We have provided a comprehensive
faced during the implementation of machine learning comparison to see the performance of each classifier based on
techniques. The work concluded that although the machine frequently used datasets. We have mentioned the
learning techniques are expanding various ways to protect computational complexity of each classifier. The following
cyberspace against cybercriminals, still there is an immense section will discuss the fundamentals of machine learning, an
number of advancements needed to protect the classifiers from overview of considered classifiers and evaluation criteria to
adversarial attacks. Machine learning classifiers themselves evaluate the performance of a classifier. The discussion
are incredibly vulnerable to cyber threats and adversarial section will discuss cyber threats and provide the performance
attacks. evaluation in the form of accuracy, recall and precision.
Lastly, the conclusion section will conclude the study.
Authors in [27] bestowed a brief review of several
publications related to the implementation of machine learning
models to enhance cyber security. They addressed some III. FUNDAMENTALS OF MACHINE LEARNING
commonly faced barriers to machine learning techniques in Artificial intelligence is a branch of computer science based
finding appropriate datasets with most efficient applicability on simulation of the human brain by an artificial entity to
for a specific security problem. automate a necessary process. Machine learning is a sub-
branch of AI. It achieves a specific goal by using the results
Authors in [28] presented a brief performance comparison from experience without explicitly being programmed. Hence
of different machine learning techniques, specifically in machine learning does not require to be fed explicitly with data
anomaly detection. They gauged the performance efficiency of [33]. There are three sub-branches of machine learning,
feature selection in ML for IDS. They claimed that the namely, supervised learning, unsupervised learning and semi-
convolutional neural network (CNN) classifier is an underused supervised learning. In supervised learning, the targeted
classifier and it could have brought vast advancements in class/label is known in advance, whereas the targeted classes
cyber security if it was used to its full potential. are unknown in unsupervised learning. Unsupervised learning
Authors in [29] analyzed the role of various machine divides the data into different clusters based on the similarity
learning techniques in spam detection, malware detection and between data objects. Semi-supervised learning combines
intrusion detection. They claimed that there is no machine characteristics of both: supervised learning and unsupervised
learning technique that is not vulnerable to cyberattacks. learning.
Every machine learning technique is still struggling to keep a Decision tree, random forest, naive Bayes, support vector
pace with continuously upgrading cybercrimes. machine, K-nearest neighbour, deep belief network, artificial
Authors in [30] proposed a novel machine learning neural network, K-mean are widely used learning techniques to
technique for spam detection in text messages using content- detect cyber threats. We have considered three techniques that
based features. They concluded that the proposed averaged are decision tree, deep belief network, and support vector
neural network and content-based feature selection outplayed machine. We have briefly described each technique below.
most of the recent machine learning techniques in terms of A deep belief network (DBN) is a complex representation
accuracy on the same dataset. Authors in [31] stated that the of middle layers of Restricted Boltzmann Machine (RBM).
signature-based classification techniques generate results with Deep belief network follows a greedy approach. Every layer
high error rates when it comes to mobile malware detection.
communicates with the previous layer and the next layer. In
They proposed an image-based deep learning technique for each layer of the deep belief network, the nodes do not
mobile malware detection, aiming to demonstrate the communicate laterally with other nodes. In a deep belief
discrimination between the family of malicious attributes and network, every layer is assigned with both input and output
the legitimate attributes by obtaining grey-scale images. tasks, excluding the first layer and the last layer. The end layer
is the classifier layer. The computation complexity of DBN is

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.
O((n+N)k) where k is the number of iterations, n represents the Precision
number of records, and N is the number of parameters in DBN
The precision is a percentage of the total number of
[34].
positive instances classified to the total number of positive
Decision tree (DT) is a supervised machine learning instances.
technique. The main components of a decision tree are nodes,
Precision= TPositive / (TPositive + FPositive) (1)
paths and leaf nodes. A node can be a root node or an
intermediate node. Decision tree follows the if-then rule to find
the best suitable root node at each level. Leaf node or terminal Error Rate
node is an ending node. The decision class is denoted by the The error rate (ERate) is a percentage of the total number
leaf node [35]. The time complexity of DT is O(mn2) where n of misclassified instances to all instances of the dataset.
represents the number of instances and m shows the number of
attributes [36, 37]. ERate = (FPositive + FNegative)/ (TNegative + FPositive + FNegative + TPositive) (2)
TABLE I. CONFUSION MATRIX Recall
Predicted as Predicted as The recall is a percentage of correctly classified positive
Normal Attack instances to the total number of positive instances classified in
Actual Labeling as
TPositive FNegative the dataset.
Normal
Actual Labeling as Recall = TPositive / (TPositive + FNegative) (3)
FPositive TNegative
Attack

Support vector machine (SVM) is another widely used IV. DISCUSSION AND PERFORMANCE EVALAUTION
supervised machine learning model. SVM works to find There is a wide range of cybercrimes that try to breach the
hyperplane with most suitable dataset distribution by privacy of user’s data daily on a computer network or mobile
classifying the data into two classes on both sides of the devices. An extensive range of machine learning techniques
hyperplane. Both sides of the hyperplane donate a separate have been developed to battle against cybercrimes. However
class. The class of every data point depends on the side of the those techniques are still lagging a step behind as compared to
hyperplane it lands. Support vector machine has a high cybercrimes. In our review, we have mainly focused on the
consumption of space and time to handle larger and noisier detection of three cardinal cyber threats, namely: IDS, malware
datasets [25]. The computational complexity of SVM is O(n2) detection and spam detection. We have considered three
where n represents the number of instances [38, 39]. learning models that are decision tree, support vector machine,
A matrix that is used to evaluate the performance of and deep belief network. Datasets play an important role in
machine learning classifier is called a confusion matrix [40], as completing all the significant tasks as the results are all
depicted in Table 1. TPositive means the number of normal dependent on the type and size of the dataset. The diversity of
instances that are correctly classified as normal. TNegative means the dataset helped to evaluate the performance of the classifier
the number of attack instances that are correctly classified as an in the training and testing phases. Real-time and diverse
attack. FNegative means the number of normal instances that are datasets produce better results than a customized dataset. In
misclassified as an attack. FPositive means the number of attack this review, we have considered frequently used and
instances that are misclassified as normal. benchmark datasets that are KDD CUP 99 [41], Spambase
[42], Twitter dataset [43], Enron [44], NSL-KDD [45],
DARPA [46], and malware datasets [47]. We have compared
the performance of the machine learning models on detecting
these cyber threats.
TABLE II. PERFORMANCE RESULTS OF SPAM DETECTION USING MACHINE LEARNING MODELS

Cyber Learning Published Performance Results

Dataset Reference Sub-Domain
Threat Model Year Precision Accuracy Recall
[48] 2011 Email Spam 93.12 % 96.90 % 95.00 %
Support Spambase
[49] 2015 Email Spam 79.02 % 79.50 % 68.67 %
Vector
Machine [50] 2018 Spam Tweets 92.91 % 93.14 % 93.14 %
Twitter Dataset
[51] 2015 Spam Tweets 95.20 % 93.60 %
[52] 2016 Email Spam 98.00 % 96.00 % 94.00 %
Enron
Spam Decision [52] 2016 Email Spam 98.00 % 96.00 % 94.00 %
Detection Tree [53] 2014 Email Spam 91.51 % 92.08 % 88.08 %
Spambase
[54] 2014 Email Spam - 94.27 % 91.02 %
[55] 2016 Email Spam 96.49 % 95.86 % 95.61 %
Enron
[56] 2016 Email Spam 98.39 % 97.50 % 98.02 %
DBN
[57] 2007 Email Spam 94.94 % 97.43 % 96.47 %
Spambase
[58] 2018 Email Spam 96.00 % 89.20 % -

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.
 We have taken accuracy, recall and precision as evaluation precision value of 98.39 % using Enron dataset [56]. DBN also
factors to measure the performance of classification models. outperformed in terms of recall and precision over SVM and
Table 2, Table 3, and Table 4 present the performance of three DT. On Spambase dataset, SVM performed better than DT
learning models for spam detection, malware detection, and with an accuracy of 96.90 % [48]. Using Enron dataset, the
intrusion detection, respectively. Cyber Threat and Learning decision tree has shown better precision than SVM and similar
Model columns are self-explanatory. Dataset column shows the precision to DBN [52]. It is apparent from Table 2 that DBN
frequently used and benchmark dataset for each particular has performed better than other learning models for these
threat. Reference column depicts the citation of specific paper particular datasets. Based on the above evaluation metrics, the
authors recommend using DBN for spam detection.
that shows the evaluation results. Values for the sub-domain
column is different for each cyber threat. Performance results
column shows the performance results of each cited article. B. Intrusion Detection
Following sub-sections will present the discussion on each Malign intrusions over the computer network and devices
cyber threat. are another cyber threat to cyberspace. These intrusions are
used to identify the vulnerabilities of a network [60]. Intrusions
A. Spam Detection identify the weakness within a computer system for further
Spam is a threat to computer and network resources. It is a attacks. An intrusion detection system is used to protect against
term used for an unwanted message. Spam can be in different these intrusions. There are three classifications of intrusions,
mediums. It can be in the form of text messages, images and namely, signature/misuse-based, anomaly-based and hybrid
videos on mobile devices [59]. Spam tweets and spam emails [61]. The intrusions can be detected on the network or a host
are the mediums that are mostly used over the computing computer. Conventional techniques are unable to cope with the
devices and network. Spam messages consume a lot of network pace to detect intrusions. Commonly used datasets are DARPA
resources, such as bandwidth. Spam emails in the form of and KDD versions. However, these datasets are older for more
unnecessary advertisements consume a lot of time. Machine than fifteen years. Table 3 presents the evaluation results of
learning techniques have been applied in the literature to intrusion detection. DBN performed better than SVM and DT
distinguish between a genuine email and a spam email, as in terms of accuracy. DBN has shown better accuracy results of
shown in Table 2. SVM and DT have shown a good accuracy 96.70 % using NSL-KDD dataset [62].
of 96.90 % [48]. However, DBN has outperformed with a

TABLE III. PERFORMANCE RESULTS OF INTRUSION DETECTION SYSTEM USING MACHINE LEARNING MODELS

Cyber Learning Published Performance Results

Dataset Reference Sub-Domain
Threat Model Year Precision Accuracy Recall
NSL-KDD [63] 2019 Anomaly-Based - 89.70 % -
Support 82.00 %
[41] 2014 Hybrid-Based 74.00 % 82.37 %
Vector
Machine DARPA [64] 2007 Hybrid-Based - 69.80 % -
[65] 2014 Anomaly-Based - 95.11 % -
[66] 2018 Misuse-Based - 99.96 % -
Intrusion Decision KDD
[67] 2017 Hybrid-Based - 86.29 % 78.00 %
Detection Tree
[68] 2019 Hybrid-Based - 93.40 % -
NSL-KDD
[69] 2017 Hybrid-Based 91.15 % 90.30 % 90.31 %
KDD [61] 2015 Anomaly-Based - 97.50 % -
DBN [62] 2015 Hybrid-Based 97.90 % 96.70 % -
NSL-KDD
[70] 2017 Anomaly-Based 88.60 % 90.40 % 95.30 %

TABLE IV. PERFORMANCE RESULTS OF MALWARE DETECTION USING MACHINE LEARNING MODELS

Cyber Learning Published Performance Results

Dataset Reference Sub-Domain
Threat Model Year Precision Accuracy Recall
[71] 2017 Static - 94.37 % -
Support Malware Dataset
[72] 2013 Dynamic - 95.00 % -
Vector
Machine [73] 2015 Dynamic - 97.10 % -
Enron [52] 2016 Static 84.74 % 91.00 % 100 %
Custom [74] 2016 Static 99.40 % 99.90 % -
Malware Decision
Detection Tree [75] 2017 Static - 84.70 % -
Malware Dataset
[76] 2014 Static 97.90 % - 96.70 %
[77] 2016 Dynamic 78.08 % 71.00 % 59.09 %
Custom
[77] 2016 Static 83.00 % 89.03 % 98.18 %
DBN
[77] 2016 Hybrid 95.77 % 96.76 % 97.84 %
KDD CUP99
[78] 2015 Hybrid - 91.40 % 95.34 %

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.
 However, the decision tree has shown outstanding security purposes. In future, we will focus on analyzing more
accuracy of 99.96 % and better than DBN and SVM using learning techniques for cyber threat detection.
KDD dataset [66]. The decision tree has shown the best
efficiency among the learning classifiers of 99.96 % REFERENCES
regardless of the dataset [66]. DBN has reported the best
[1] "ICT Facts and Figures 2017." Telecommunication Development
recall and precision values of 95.30 % and 97.90 %, Bureau,International Telecommunication Union (ITU), Technical Report.
respectively [62, 70]. Based on the considered articles, the https://www.itu.int/en/ITU-D/Statistics/Pages/facts/default.aspx (accessed
decision tree is recommended as the best learning classifier October 09, 2019).
[2] "What is Cyber-Security?" https://www.kaspersky.com.au/resource-
for intrusion detection, as depicted in Table 3. center/definitions/what-is-cyber-security (accessed January 11, 2020).
[3] F. Farahmand, S. B. Navathe, P. H. Enslow, and G. P. Sharp, "Managing
vulnerabilities of information systems to security incidents," in Proceedings of
C. Malware detection the 5th international conference on Electronic commerce, 2003: ACM, pp. 348-
Malware, short for malicious software, is a software that 354.
[4] P. Szor, The Art of Computer Virus Research and Defense: ART COMP VIRUS
is installed on a computer to disrupt its operation and harm RES DEFENSE _p1. Pearson Education, 2005.
the electronic data. Viruses, worms, ransomware, adware, [5] M. Jump, "Fighting Cyberthreats with Technology Solutions," Biomedical
instrumentation & technology, vol. 53, no. 1, pp. 38-43, 2019.
spyware, malvertising, and Trojan horse are considered as [6] N. Kostyuk and C. Wayne, "Communicating Cybersecurity: Citizen Risk
significant types of malware [79]. Malware interrupts the Perception of Cyber Threats," 2019.
normal flow of computer operations. With a growing pace of [7] A. K. Jain, D. Goel, S. Agarwal, Y. Singh, and G. Bajaj, "Predicting Spam
Messages Using Back Propagation Neural Network," Wireless Personal
usage of computing and mobile devices, the cybercriminal is Communications, vol. 110, no. 1, pp. 403-422, 2020.
finding it easy to compromise the integrity of data. Malware [8] "Malware Types and Classifications." https://www.lastline.com/blog/malware-
also disrupts the availability of computer and network types-and-classifications/ (accessed April 18,2020).
[9] N. Sultana, N. Chilamkurti, W. Peng, and R. Alhadad, "Survey on SDN based
resources. Machine learning techniques are being used to network intrusion detection system using machine learning approaches," Peer-to-
detect malware. The performance of each learning classifier Peer Networking and Applications, vol. 12, no. 2, pp. 493-501, 2019.
is depicted in Table 4. Static detection is a sub-domain of [10] M. Pradhan, C. K. Nayak, and S. K. Pradhan, "Intrusion Detection System (IDS)
and Their Types," in Securing the Internet of Things: Concepts, Methodologies,
malware detection in which applications are tested for Tools, and Applications: IGI Global, 2020, pp. 481-497.
malware without executing them. However, in dynamic [11] I. Firdausi, A. Erwin, and A. S. Nugroho, "Analysis of machine learning
detection, the applications or software are tested by techniques used in behavior-based malware detection," in 2010 second
international conference on advances in computing, control, and
executing them. Hybrid detection is a mixture of both static telecommunication technologies, 2010: IEEE, pp. 201-203.
and dynamic detection [80]. The decision tree has shown [12] A. V. Joshi, Machine Learning and Artificial Intelligence. Springer, 2020.
[13] D. Michie, D. J. Spiegelhalter, and C. Taylor, "Machine learning," Neural and
overall best accuracy of 99.90 % on custom data collected by Statistical Classification, vol. 13, 1994.
the author [74]. However, on a malware dataset, SVM [14] K. Shaukat, A. Rubab, I. Shehzadi, and R. Iqbal, "A Socio-Technological
performed better than decision tree in terms of accuracy. analysis of Cyber Crime and Cyber Security in Pakistan," Transylvanian Review,
vol. 1, no. 3, 2017.
SVM reported the best recall value of 100% [52]. SVM is [15] K. Shaukat, N. Masood, A. B. Shafaat, K. Jabbar, H. Shabbir, and S. Shabbir,
recommended based upon the cited papers to detect and "Dengue fever in perspective of clustering algorithms," arXiv preprint
classify applications from malware. arXiv:1511.07353, 2015.
[16] K. Shaukat, N. Masood, S. Mehreen, and U. Azmeen, "Dengue fever prediction:
A data mining problem," Journal of Data Mining in Genomics & Proteomics,
V. CONCLUSION vol. 2015, 2015.
[17] K. Shaukat, I. Nawaz, and S. Zaheer, Students Performance: A Data Mining
Cyber threats are increasing at a growing pace. The Perspective. LAP Lambert Academic Publishing, 2017.
[18] K. Shaukat, I. Nawaz, S. Aslam, S. Zaheer, and U. Shaukat, "Student's
conventional security techniques are not capable enough of performance in the context of data mining," in 2016 19th International Multi-
coping with these threats. Machine learning techniques are Topic Conference (INMIC), 2016: IEEE, pp. 1-8.
being applied to overcome the limitations of conventional [19] S. Dey, Q. Ye, and S. Sampalli, "A machine learning based intrusion detection
scheme for data fusion in mobile clouds involving heterogeneous client
security systems. Machine learning techniques are playing networks," Information Fusion, vol. 49, pp. 205-215, 2019.
their role at both ends: at defender-end and attacker-end. We [20] B. Geluvaraj, P. Satwik, and T. A. Kumar, "The future of cybersecurity: Major
role of artificial intelligence, machine learning, and deep learning in cyberspace,"
have presented a performance comparison of three learning in International Conference on Computer Networks and Communication
models to detect and classify the intrusion, spam and Technologies, 2019: Springer, pp. 739-747.
malware. We have considered frequently used and [21] A. A. Alurkar et al., "A Comparative Analysis and Discussion of Email Spam
Classification Methods Using Machine Learning Techniques," Applied Machine
benchmark datasets to compare the evaluation results in Learning for Smart Data Analysis, p. 185, 2019.
terms of recall, precision, and accuracy. In the previous [22] E. G. Dada, J. S. Bassi, H. Chiroma, A. O. Adetunmbi, and O. E. Ajibuwa,
section, we have discussed and concluded that we cannot "Machine learning for email spam filtering: review, approaches and open
research problems," Heliyon, vol. 5, no. 6, p. e01802, 2019.
recommend a particular learning technique for every cyber [23] P. Jain, "Machine Learning versus Deep Learning for Malware Detection," 2019.
threat detection. Different learning models are being used for [24] P. Thiyagarajan, "A Review on Cyber Security Mechanisms Using Machine and
specific different cyber threats. On the other hand, there is a Deep Learning Algorithms," in Handbook of Research on Machine and Deep
Learning Applications for Cyber Security: IGI Global, 2020, pp. 23-41.
vast number of authors who have worked to highlight the [25] S. S. Iyer and S. Rajagopal, "Applications of Machine Learning in Cyber
constraints faced by machine learning techniques. We have Security Domain," in Handbook of Research on Machine and Deep Learning
observed and suggested that there is a dare need of latest Applications for Cyber Security: IGI Global, 2020, pp. 64-82.
[26] V. Ford and A. Siraj, "Applications of machine learning in cyber security," in
benchmark dataset to test the latest advancement in the field Proceedings of the 27th International Conference on Computer Applications in
of machine learning for cyber threat detection. Available Industry and Engineering, 2014.
datasets lack in terms of diversity and sophisticated attacks [27] H. Jiang, J. Nagra, and P. Ahammad, "Sok: Applying machine learning in
security-a survey," arXiv preprint arXiv:1611.03186, 2016.
and contain missing values. There is a need for specific and [28] E. Hodo, X. Bellekens, A. Hamilton, C. Tachtatzis, and R. Atkinson, "Shallow
customized learning models specifically designed for and deep networks intrusion detection system: A taxonomy and survey," arXiv
preprint arXiv:1701.02145, 2017.

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.
[29] G. Apruzzese, M. Colajanni, L. Ferretti, A. Guido, and M. Marchetti, "On the [56] I. J. Alkaht and B. J. I. R. C. S. Al-Khatib, "Filtering SPAM Using Several
effectiveness of machine and deep learning for cyber security," in 2018 10th Stages Neural Networks," vol. 11, p. 2, 2016.
International Conference on Cyber Conflict (CyCon), 2018: IEEE, pp. 371-390. [57] G. Tzortzis and A. Likas, "Deep belief networks for spam filtering," in 19th
[30] S. Sheikhi, M. Kheirabadi, and A. Bazzazi, "An Effective Model for SMS Spam IEEE International Conference on Tools with Artificial Intelligence (ICTAI
Detection Using Content-based Features and Averaged Neural Network," 2007), 2007, vol. 2: IEEE, pp. 306-309.
International Journal of Engineering, vol. 33, no. 2, pp. 221-228, 2020. [58] Y. Rizk, N. Hajj, N. Mitri, M. J. A. C. Awad, and Informatics, "Deep belief
[31] F. Mercaldo and A. Santone, "Deep learning for image-based mobile malware networks and cortical algorithms: A comparative study for supervised
detection," Journal of Computer Virology and Hacking Techniques, pp. 1-15, classification," 2018.
2020. [59] A. Sharaff, N. K. Nagwani, and A. Dhadse, "Comparative study of classification
[32] J. Ribeiro, F. B. Saghezchi, G. Mantas, J. Rodriguez, S. J. Shepherd, and R. A. algorithms for spam email detection," in Emerging research in computing,
Abd-Alhameed, "An autonomous host-based intrusion detection system for information, communication and applications: Springer, 2016, pp. 237-244.
android mobile devices," Mobile Networks and Applications, vol. 25, no. 1, pp. [60] C. Yin, Y. Zhu, J. Fei, and X. He, "A deep learning approach for intrusion
164-172, 2020. detection using recurrent neural networks," Ieee Access, vol. 5, pp. 21954-21961,
[33] C. Chen et al., "A performance evaluation of machine learning-based streaming 2017.
spam tweets detection," IEEE Transactions on Computational social systems, [61] M. Z. Alom, V. Bontupalli, and T. M. Taha, "Intrusion detection using deep
vol. 2, no. 3, pp. 65-76, 2015. belief networks," in 2015 National Aerospace and Electronics Conference
[34] Z. Chen, S. Liu, K. Jiang, H. Xu, and X. Cheng, "A data imputation method (NAECON), 2015: IEEE, pp. 339-344.
based on deep belief network," in 2015 IEEE International Conference on [62] S. Jo, H. Sung, and B. Ahn, "A comparative study on the performance of
Computer and Information Technology; Ubiquitous Computing and intrusion detection using decision tree and artificial neural network models,"
Communications; Dependable, Autonomic and Secure Computing; Pervasive Journal of the Korea Society of Digital Industry and Information Management,
Intelligence and Computing, 2015: IEEE, pp. 1238-1243. vol. 11, no. 4, pp. 33-45, 2015.
[35] D. M. Farid, N. Harbi, and M. Z. Rahman, "Combining naive bayes and decision [63] J. Lee, J. Kim, I. Kim, and K. Han, "Cyber Threat Detection Based on Artificial
tree for adaptive intrusion detection," arXiv preprint arXiv:1005.4496, 2010. Neural Networks Using Event Profiles," IEEE Access, vol. 7, pp. 165607-
[36] Q. J. Ross, "C4. 5: programs for machine learning," San Mateo, CA, 1993. 165626, 2019.
[37] P. S. Oliveto, J. He, and X. Yao, "Time complexity of evolutionary algorithms [64] L. Khan, M. Awad, and B. Thuraisingham, "A new intrusion detection system
for combinatorial optimization: A decade of results," International Journal of using support vector machines and hierarchical clustering," The VLDB journal,
Automation and Computing, vol. 4, no. 3, pp. 281-293, 2007. vol. 16, no. 4, pp. 507-521, 2007.
[38] C. J. Burges, "A tutorial on support vector machines for pattern recognition," [65] R. Kokila, S. T. Selvi, and K. Govindarajan, "DDoS detection and analysis in
Data mining and knowledge discovery, vol. 2, no. 2, pp. 121-167, 1998. SDN-based environment using support vector machine classifier," in 2014 Sixth
[39] G. D. Forney, "The viterbi algorithm," Proceedings of the IEEE, vol. 61, no. 3, International Conference on Advanced Computing (ICoAC), 2014: IEEE, pp.
pp. 268-278, 1973. 205-210.
[40] X. Deng, Q. Liu, Y. Deng, and S. Mahadevan, "An improved method to construct [66] P. Mishra, V. Varadharajan, U. Tupakula, E. S. J. I. C. S. Pilli, and Tutorials, "A
basic probability assignment based on the confusion matrix for classification detailed investigation and analysis of using machine learning techniques for
problem," Information Sciences, vol. 340, pp. 250-261, 2016. intrusion detection," vol. 21, no. 1, pp. 686-728, 2018.
[41] M. S. Pervez and D. M. Farid, "Feature selection and intrusion classification in [67] J. Kevric, S. Jukic, A. J. N. C. Subasi, and Applications, "An effective combining
NSL-KDD cup 99 dataset employing SVMs," in The 8th International classifier approach using tree algorithms for network intrusion detection," vol.
Conference on Software, Knowledge, Information Management and Applications 28, no. 1, pp. 1051-1058, 2017.
(SKIMA 2014), 2014: IEEE, pp. 1-6. [68] A. Ahmim, L. Maglaras, M. A. Ferrag, M. Derdour, and H. Janicke, "A novel
[42] "Spambase Dataset. Center for Machine Learning and Intelligent Systems at UC hierarchical intrusion detection system based on decision tree and rules-based
Irvine." https://archive.ics.uci.edu/ml/datasets/Spambase (accessed January 31, models," in 2019 15th International Conference on Distributed Computing in
2020). Sensor Systems (DCOSS), 2019: IEEE, pp. 228-233.
[43] D. Gunawan, R. F. Rahmat, A. Putra, and M. F. Pasha, "Filtering Spam Text [69] B. Ingre, A. Yadav, and A. K. Soni, "Decision tree based intrusion detection
Messages by Using Twitter-LDA Algorithm," in 2018 IEEE International system for NSL-KDD dataset," in International Conference on Information and
Conference on Communication, Networks and Satellite (Comnetsat), 2018: IEEE, Communication Technology for Intelligent Systems, 2017: Springer, pp. 207-218.
pp. 1-6. [70] D. Kwon, H. Kim, J. Kim, S. C. Suh, I. Kim, and K. J. Kim, "A survey of deep
[44] B. Klimt and Y. Yang, "Introducing the Enron corpus," in CEAS, 2004. learning-based network anomaly detection," Cluster Computing, pp. 1-13, 2017.
[45] B. Ingre and A. Yadav, "Performance analysis of NSL-KDD dataset using [71] Y. Cheng, W. Fan, W. Huang, and J. An, "A Shellcode Detection Method Based
ANN," in 2015 International Conference on Signal Processing and on Full Native API Sequence and Support Vector Machine," in IOP Conference
Communication Engineering Systems, 2015: IEEE, pp. 92-96. Series: Materials Science and Engineering, 2017, vol. 242, no. 1: IOP
[46] A. Chahal and R. Nagpal, "Performance of Snort on Darpa Dataset and Diferent Publishing, p. 012124.
False Alert Reduction Techniques," in 3rd International Conference on [72] A. Mohaisen and O. Alrawi, "Unveiling zeus: automated classification of
Electrical, Electronics, Engineering Trends, Communication, Optimization and malware samples," in Proceedings of the 22nd International Conference on
Sciences (EEECOS). World Wide Web, 2013: ACM, pp. 829-832.
[47] H. Kim, T. Cho, G.-J. Ahn, and J. H. Yi, "Risk assessment of mobile applications [73] P. Shijo and A. J. P. C. S. Salim, "Integrated static and dynamic analysis for
based on machine learned malware dataset," Multimedia Tools and Applications, malware detection," vol. 46, pp. 804-811, 2015.
vol. 77, no. 4, pp. 5027-5042, 2018. [74] Q. Jamil and M. A. Shah, "Analysis of machine learning solutions to detect
[48] W. Awad, S. J. I. J. o. C. S. ELseuofi, and I. Technology, "Machine learning malware in android," in 2016 Sixth International Conference on Innovative
methods for spam e-mail classification," vol. 3, no. 1, pp. 173-184, 2011. Computing Technology (INTECH), 2016: IEEE, pp. 226-232.
[49] R. Karthika and P. J. W. T. C. Visalakshi, "A hybrid ACO based feature selection [75] D. Moon, H. Im, I. Kim, and J. H. Park, "DTB-IDS: an intrusion detection
method for email spam classification," vol. 14, pp. 171-177, 2015. system based on decision tree using behavior analysis for preventing APT
[50] G. Jain, M. Sharma, and B. J. I. J. o. K. D. i. B. Agarwal, "Spam detection on attacks," The Journal of supercomputing, vol. 73, no. 7, pp. 2881-2895, 2017.
social media using semantic convolutional neural network," vol. 8, no. 1, pp. 12- [76] Z. Salehi, A. Sami, M. J. C. F. Ghiasi, and Security, "Using feature generation
26, 2018. from API calls for malware detection," vol. 2014, no. 9, pp. 9-18, 2014.
[51] C. Chen et al., "A performance evaluation of machine learning-based streaming [77] Z. Yuan, Y. Lu, Y. J. T. S. Xue, and Technology, "Droiddetector: android
spam tweets detection," vol. 2, no. 3, pp. 65-76, 2015. malware characterization and detection using deep learning," vol. 21, no. 1, pp.
[52] Z. Khan and U. Qamar, "Text Mining Approach to Detect Spam in Emails," in 114-123, 2016.
The International Conference on Innovations in Intelligent Systems and [78] Y. Li, R. Ma, R. J. I. J. o. S. Jiao, and I. Applications, "A hybrid malicious code
Computing Technologies (ICIISCT2016), 2016, p. 45. detection method based on deep learning," vol. 9, no. 5, pp. 205-216, 2015.
[53] S. A. Saab, N. Mitri, and M. Awad, "Ham or spam? A comparative study for [79] R. Vinayakumar, M. Alazab, K. Soman, P. Poornachandran, and S.
some content-based classification algorithms for email filtering," in MELECON Venkatraman, "Robust intelligent malware detection using deep learning," IEEE
2014-2014 17th IEEE Mediterranean Electrotechnical Conference, 2014: IEEE, Access, vol. 7, pp. 46717-46738, 2019.
pp. 339-343. [80] A. Damodaran, F. Di Troia, C. A. Visaggio, T. H. Austin, and M. Stamp, "A
[54] Y. Zhang, S. Wang, P. Phillips, and G. J. K.-B. S. Ji, "Binary PSO with mutation comparison of static, dynamic, and hybrid analysis for malware detection,"
operator for feature selection using decision tree applied to spam detection," vol. Journal of Computer Virology and Hacking Techniques, vol. 13, no. 1, pp. 1-12,
64, pp. 22-31, 2014. 2017.
[55] A. Tyagi, "Content Based Spam Classification-A Deep Learning Approach,"
University of Calgary, 2016.

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 18,2021 at 21:37:56 UTC from IEEE Xplore. Restrictions apply.