Cloud-based Network Intrusion Detection System using Deep

Learning
Archana C Chaitra H P Khushi M
PES University, Bangalore, India PES University, Bangalore, India PES University, Bangalore, India
[email protected] [email protected] [email protected]

Pradhiksha Nandini T Sivaraman E Prasad Honnavalli

ABSTRACT Sharjah, United Arab Emirates. ACM, New York, NY, USA, 6 pages. https:
With the increase in the internet traffic worldwide, it has been //doi.org/10.1145/3485557.3485562
observed that there is a major spike in the influx of network traffic
into any system over the years. The internet traffic and the number 1 INTRODUCTION
of attempts at malicious access to any organization have increased Network Security has become an important aspect because com-
over the years. The attacker can take advantage of this and over puter networks are used extensively in all the fields. Various sensi-
flood the organization with dummy traffic and make the systems tive user data is handled by networks which are prone to various
unresponsive. The current existing models of Network Intrusion kinds of attacks. There are a variety of tools like firewall, antimal-
Detection System (NIDS) have a good prediction rate but somehow ware, antivirus etc. used to detect and exploit attacks. These attacks
have high False Positive Rate (FPR). We propose a Deep Learning are diverse and are evolving with sophisticated algorithms which
(DL) model to reduce the FPR which offers a scalable solution by makes it undetectable by traditional tools leading to data breach.
deploying the model on cloud to increase the responsiveness of These kinds of malicious attacks pose security challenges which
the NIDS during high loads, hence increasing the availability. The makes it important to have a reliable and flexible Intrusion Detec-
model which is running on docker containers on the cloud instance tion System (IDS). An IDS is a software application or a system
can be accessed using REST APIs as it is deployed as microservice. that monitors network traffic for any suspicious activity and an
The experimental results showed that Deep Neural Networks (DNN) alert is issued to the management system on discovery of such ab-
with five hidden layers achieved the best accuracy of 95.02% and a normal events [1]. Anything that violates the CIA (Confidentiality,
least accuracy of 88.75% by Long Short-Term Memory (LSTM) with Integrity and Availability) triad must be caught by the IDS and it
two layers. In traditional ML algorithms, Random Forest approach can be installed on any individual devices such as desktop, servers.
have achieved 86% accuracy. Network Intrusion Detection system (NIDS) monitors the net-
work traffic flowing in and out of the entire system to detect traces
CCS CONCEPTS of malicious activity [2]. It is usually placed at the crucial points
• Security and privacy; • Networks; • Computing methodolo- where all the traffic traverses. IDS uses the following methods to
gies; identify the attacks: Signature based Intrusions can be detected by
classifying access to the systems into predefined signature, patterns,
KEYWORDS malicious activities. Signatures need to be updated regularly and
it is accurate in finding known types of attacks. Anomaly based
Network Intrusion Detection, Deep Learning, Docker, Container,
IDS analyses the normal behavior of the network and alerts when
Microservice
there’s a deviation [3]. It is used to detect unknown attacks. Hybrid
ACM Reference Format: based IDS is a combination of both signature-based and anomaly-
Archana C, Chaitra H P, Khushi M, Pradhiksha Nandini T, Sivaraman E, based IDS. In this research, a novel and scalable cloud-based IDS is
and Prasad Honnavalli. 2021. Cloud-based Network Intrusion Detection proposed which can handle high network traffic loads and reduce
System using Deep Learning. In ArabWIC 2021: The 7th Annual International FPR.
Conference on Arab Women in Computing in Conjunction with the 2nd Forum
of Women in Research, Sharjah, UAE (ArabWIC 2021), August 25, 26, 2021, 2 RELATED WORKS
To enhance the performance of IDS, many studies are focusing on
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed applying ML techniques both supervised and unsupervised. Some
for profit or commercial advantage and that copies bear this notice and the full citation of the ML algorithms used are Naive bayes, Support Vector Ma-
on the first page. Copyrights for components of this work owned by others than ACM chine (SVM), Random Forest, Decision Trees, K-Nearest Neighbour
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a (KNN), Decision Trees etc. [3]. In [4], the authors have presented a
fee. Request permissions from [email protected]. NIDS model which is designed and implemented using Recurrent
ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates Neural Network. Performance of the model is studied for both mul-
© 2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8418-6/21/08. . . $15.00 ticlass classification and binary classification and how the accuracy
https://doi.org/10.1145/3485557.3485562 is impacted by various combinations of the parameters like learning
ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates Archana C et al.

rate and number of neurons. A Deep Neural Network model with algorithms to increase the responsiveness of the IDS during high
one input layer, five hidden layers and an output layered architec- loads, hence increasing the reliability. The IDS can be scaled up or
ture on the KDD99 dataset as the hybrid intrusion detection system down depending on the traffic. This way there won’t be concerns
approach is presented by Vinayakumar et al. [5]. Another research regarding investment on infrastructure and wastage of resources
by Stefan Dlugolinsky et al. discusses about the harmony between when the load is high and low respectively. The model prefers DL
large scale data processing and Deep Learning techniques [6]. The algorithm over ML algorithm because DL learns new features which
model by coupling filter-based feature selection with Feed Forward majorly define a model on its own without any human intervention,
Deep Neural Networks (FFDNNs) algorithm is proposed by Yanxia whereas ML requires the user to feed the important features to the
Sun et al. [7]. A model which uses a hybrid of Convolutional Neural model. It trains a model using DL algorithms that learns about the
Network (CNN) and weight-dropped Long Short-Term Memory characteristics of a malicious activity using a pre-existing dataset
(LSTM) is presented by researchers [8]. The authors in [9] have [14] and with each detection, improves and optimizes the model.
proposed an IDS based on the CNN to reinforce the security of It will be hosted on the cloud and be configured as microservices
the internet. A hierarchical Convolutional Neural Network and which can be scaled depending on the network traffic.
Recurrent Neural Network called LuNet, is proposed by Hui Guo
et al. [10] to detect intrusions. Farrukh Aslam Khan et al. [11] have 4 UNSW-NB15 DATASET
used a stacked Autoencoder (AE) based two staged Deep Learning The dataset used in our research is UNSW-NB15 [14] and it was
model by using soft-max for classification. To enhance the detec- collected by the Australian Center for Cyber Security (ACCS) in
tion rate of the intrusion detection system, [12] mainly focuses on 2015 by the research group of cyber security. It is a network intru-
false positive and false negative performance metrics. A genetic sion dataset. The raw network packets of the UNSW-NB15 data set
algorithm based on exhaustive search and fuzzy C-means clustering are created by the IXIA PerfectStorm tool. The training set consists
algorithm is used in Genetic Convolutional Neural Network [13]. of 175,341 records. Testing set contains 82,232 records from the
different types, attack and normal. UNSW-NB15 contains approxi-
3 RESEARCH CHALLENGES AND PROPOSED mately 2.5 million records. The dataset was generated using two
diverse simulation periods of 15 hours and 16 hours respectively.
SOLUTION
This dataset consists of both synthesized attack activities and real
First challenge is that the NIDS usually has a large FPR and classi- modern normal activities and of the network traffic. TCP dump
cal ML classifiers have high computation rate with high FPR since is employed to capture 100GB of the raw traffic. It contains nine
they learn the characteristic of simple features of TCP and/or IP families of attacks, namely, Reconnaissance, Shellcode, Backdoors,
locally [2]. Whereas DL is a complex subset of ML that acquires Analysis, DoS, Exploits, Generic, and Worm.
hidden sequential relationships and hierarchical feature represen-
tations by sending the TCP and/or IP information on subsequent 5 PROPOSED METHODOLOGY AND
hidden layers making it reduce the FPR. Since the DL algorithms
IMPLEMENTATION
require substantial computational resources, the advent of cloud-
based platforms can streamline the process of implementation of The proposed NIDS architecture is to build the IDS model as mi-
models using Deep Learning. Next challenge is the IDS built using croservices on the cloud. The IDS will classify the input data in-
ML have achieved a high accuracy with small amounts of input stance as normal or different types of attacks based on the features
data. However, if the large dataset is used its very time consuming of the initial stage. Using Neural Network models like Convolutional
and has high latency for large dataset [3]. There are some issues Neural Network (CNN) [17], Long Short-Term Memory (LSTM) [16]
with these classical algorithms. ML techniques fail in multi class or Hybrid models with appropriate activations at each layer, the
classification because of a greater number of features. There are model can perform better than the classical ML algorithms [18].
issues such as overfitting and inducing high bias due to redundant
or irrelevant features which affect the model metrics. Most of the 5.1 Proposed Cloud-based NIDS Architecture
studies have used old publicly available dataset which doesn’t cover The proposed architecture for cloud-based NIDS which consists of
today’s wide range of attacks. There is also the issue of scalability, Docker Containers, REpresentational State Transfer Application
where the model can take a long time to respond if it is bombarded Programming Interfaces (REST APIs) over postman and a Load
with requests. The signature-based IDS has evolved over the years Balancer is illustrated in Figure 1. There are three microservices
into anomaly-based IDS to detect anomalous behavior posed by the namely, Training, Testing and Database. The testing/classification
attacker [2]. However, these attacks are always evolving in both of packets can be done by sending REST APIs to the testing mi-
quality and quantity. With increase in the internet traffic worldwide, croservice. The testing microservice will in turn send this now
the influx of network traffic into any system has also seen a major classified packet into the database to ensure “real time” data is
spike over the years. The attacker can take advantage of this and being stored. The idea is to have weekly updates of the training
over flood the system with dummy traffic and make the systems microservice on the data in the database. For experimentation, the
unresponsive. In such environments having a single running in- proposed NIDS model on Amazon EC2 instances as microservices.
stance of an IDS to alert the organization about a potential attack in The docker containers are configured accordingly and the train-
real-time can be quite challenging. The proposed research is to use ing and testing containers are accessed through REST APIs. The
a DL algorithm-based NIDS which will be hosted on the cloud to training and testing phases are implemented as a flask app hosted
classify malicious activities. It will offer a scalable solution using DL locally. Both can be accessed through REST APIs over Postman.
Cloud-based Network Intrusion Detection System using Deep Learning ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates

Figure 1: Proposed Cloud-based NIDS Architecture

Depending on the amount of data to be processed by the docker with a maxpool layer and a flattening layer. Lastly, DNN uses the
containers, the number of running instances will be either scaled-up Adam method for optimizing the network and minimizing the loss
or scaled-down accordingly via docker SDK. function. Adam is an alternative to the gradient descent optimizer
Across multiple targets, the incoming traffic is automatically and solves its limitations. Every node in the output and the hidden
distributed by the load balancer to respective EC2 instances. It layers have their own activation functions chosen on trial and error.
monitors the registered target’s health, and it routes the traffic only The prediction of accuracy depends on the computed weights and
to the healthy instances. The load balancer can scale itself according biases which is why we need to choose the hyperparameters cor-
to the change in incoming traffic changes with time. For clients, rectly. The model was run for 50 epochs with batch size as 64. The
it serves as a single point of contact. This ensures the availability experimental results of the proposed DL algorithms are conversed
of the proposed NIDS application. One or more listeners can be in the following section.
added to the load balancer. The model will get the incoming traffic
features from the training dataset. The traffic is fed to the neural
network model which extracts features and trains the model to 6 EXPERIMENTAL RESULTS AND
classify traffic as attack or benign.
DISCUSSION
Initially, we have experimented the proposed cloud-based IDS by
5.2 NIDS Implementation using Deep Learning creating an AWS EC2 instance with two containers using flask
Algorithms framework for making API calls. We have compared the results
A two-stage model is considered in this research. The initial stage of different ML and DL models on the UNSW-NB15 dataset. The
is to extract minimal meaningful features which best represent the ML models implemented includes Logistic Regression, K-Nearest
data. The next stage is to classify the input data instance as normal, Neighbor (KNN), Naive Bayes, Random Forest, Decision Trees and
or attack based on the features of the initial stage. Neural networks Adaboost [3]. The first step is preprocessing the dataset which
such as LSTM, Gated Recurrent Unit (GRU), CNN and DNN models will be used for model implementation. We were able to reduce 42
are used with appropriate activation functions [21, 22]. The activa- features to 29 which has improved the performance of the model.
tion function at each layer is fixed after experimentation. During Figure 2 shows the comparison of different ML algorithms, and it
optimization, hyperparameter tuning like learning rate, number can be perceived that Random Forest & Decision Tree have achieved
and depth of hidden layers used, number of epochs can be fixed the highest accuracy of 86%.
after trial and experimentation. Initially, LSTM with different lay- The experimental results of different DL algorithms were calcu-
ers is used with Rectified activation Linear function (ReLu) as an lated. Highest accuracy of 88.81% is achieved from LSTM model
activation function and using Stochastic Gradient Descent (SGD) with one hidden layer and GRU achieved 88.79% with both one and
as the optimization function. LSTM is an improvement on the Re- two hidden layers. Usually, adding more hidden layers would in-
current Neural Network (RNN) [16]. RNN suffers from vanishing crease the accuracy of the model but in this study, it’s observed that
gradient problems so the layers won’t learn much and will forget adding a greater number of hidden layers did not help in increasing
the information it has learned [20]. Next, GRU is considered which the accuracy rather it has decreased the performance of the model.
is an improvised form of standard RNN. It solves the disappearing CNN with 2 convolutional layers achieved an accuracy of 93.11%.
gradient problem of a standard RNN. CNN is also considered which Deep Neural Network is implemented using Batch Normalization
has one or more convolutional layers along with a pooling layer optimizer which gave the highest accuracy compared to other ML
and a flattening layer. Our CNN model has 2 convolutional layers and DL models (refer Figure 3).
ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates Archana C et al.

Figure 2: Performance Analysis of Machine Learning Algorithms

Figure 3: Performance Analysis of DNN with Different Hidden Layers

Table 1: Confusion Matrix for the Proposed DNN Model

Condition Predicted Benign Predicted Attack

The DNN model with 4 hidden layers achieved the highest ac- differentiate between the two classes. The ROC curve for different
curacy of 95.02%. DNN with 1 hidden layer achieved the lowest layers of the DNN model is plotted in the Figure 4. It is noted that
FPR of 1.70% followed by four hidden layers with 1.75% FPR. For AUC increases with hidden layers and DNN with four hidden layers
DNN layers adding more hidden layers improves the accuracy but has the highest AUC of 0.97.
the training time is also increased as more hidden layers are added.
Table 1 represents the confusion matrix for the DNN model. For an 7 CONCLUSION
IDS, FPR should be less as the users shouldn’t be alarmed falsely
We have used AWS instances to run the model on the cloud and
when there is no attack because if the FPR rate is high users tend
configure the docker containers accordingly. We have implemented
to ignore an alarm of real attack thinking it as a false alarm [23].
different DL algorithms like LSTM, GRU, DNN and compared with
ROC curve is used for performance comparison at different
traditional ML models. Based on the experimental results, it is
thresholds. Ideally, FPR should be 0 and TPR should have the value
noted that using Deep Learning over Machine Learning appreciably
of 1. Area Under the Curve (AUC) tells us how well the model can
increases the performance of the model for larger datasets. The
Cloud-based Network Intrusion Detection System using Deep Learning ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates

Figure 4: ROC Curve for DNN Model

performance metrics are calculated for the proposed model and 14-25. IEEE. DOI:https://doi.org/10.1109/OJCS.2021.3050917.
compared with the other studies. The Deep Neural Network model [10] Peilun Wu, Hui Guo, 2019. LuNet: A Deep Neural Network for Network Intrusion
Detection. In Proceedings of IEEE Symposium Series on Computational Intelli-
achieved good accuracy. Using batch normalization has significantly gence (SSCI), Xiamen, China. IEEE. DOI: https://doi.org/10.1109/SSCI44817.2019.
improved the accuracy of the model. This model can offer real 9003126.
[11] Farrukh Aslam Khan, Abdu Gumaei , Abdelouahid Derhab, and Amir Hussain,
time detection provided, the input to the model for testing is given 2019. A Novel Two-Stage Deep Learning Model for Efficient Network Intrusion
according to the format specified and follows necessary. A proactive Detection. IEEE Access. IEEE, pp.30373 – 30385. DOI: https://doi.org/10.1109/
model which can detect future attacks may be implemented in ACCESS.2019.2899721.
[12] Mohammad Almseidin, Maen Szilveszter Kovacs and Mouhammd Alkasass-
future for prior detection of attacks. beh, 2017. Evaluation of Machine Learning Algorithms for Intrusion Detec-
tion Systems. In Proceedings of IEEE 15th International Symposium on In-
REFERENCES telligent Systems and Informatics (SISY), Subotica, Serbia. IEEE, DOI: https:
//doi.org/10.1109/SISY.2017.8080566.
[1] Sabahi F and Movaghar A, 2008. Intrusion Detection: A Survey. In Proceedings of [13] Minh Tuan Nguyen and Kiseon Kim, 2020. Genetic Convolutional Neural Network
the Third International Conference on Systems and Networks Communications, for Intrusion Detection Systems. Future Generation Computer Systems, 113, 418-
Sliema, Malta. IEEE Xplore, DOI:10.1109/ICSNC.2008.44. 427. Elsevier. DOI: https://doi.org/10.1016/j.future.2020.07.042.
[2] Karen Scarfone and Peter Mell, 2007. Guide to Intrusion Detection and Prevention [14] UNSW-NB15 Dataset. Australian Centre for Cyber Security (ACCS). https://www.
Systems (IDPS). NIST Special Publication 800-94. National Institute of Standards kaggle.com/mrwellsdavid/unsw-nb15.
& Technology, Gaithersburg, MD, United States. [15] Zouhair Chiba Noreddine Abghour Khalid Moussaid Amina El omri and Mo-
[3] Sharfuddin Khan, E Sivaraman and Prasad B. Honnavalli, 2020. Performance hamed Rida, 2019. Intelligent Approach to build a Deep Neural Network based
Evaluation of Advanced Machine Learning Algorithms for Network Intrusion IDS for Cloud Environment Using Combination of Machine Learning Algorithms.
Detection System. In Proceedings of the International Conference on IoT Inclusive Computers & Security, 86, 291-317. Elsevier. DOI: https://doi.org/10.1016/j.cose.
Life (ICIIL 2019), NITTTR Chandigarh, India. Lecture Notes in Networks and 2019.06.013.
Systems, 51-59. Springer, Singapore. DOI: https://doi.org/10.1007/978-981-15- [16] Sydney Mambwe Kasongo and Yanxia Sun, 2019. A Deep Long Short-Term Mem-
3020-3_6. ory Based Classifier for Wireless Intrusion Detection System. ICT Express, 6
[4] Chuanlong Yin, Yuefei Zhu, Jinlong Fei and Xinzheng He, 2017. A Deep Learning (2), 98-103. Korean Institute of Communications Information Sciences. DOI:
Approach for Intrusion Detection using Recurrent Neural Networks. IEEE Access, https://doi.org/10.1016/j.icte.2019.08.004.
21954 –21961. IEEE. DOI: https://doi.org/10.1109/ACCESS.2017.2762418. [17] Kehe Wu, Zuge Chen and Wei Li, 2018. A Novel Intrusion Detection Model for a
[5] R. Vinayakumar, Mamoun Alazab, K. P. Soman, Prabaharan Poornachandran, Massive Network Using Convolutional Neural Networks. IEEE Access, 50850 –
Ameer Al-nemrat, Sitalakshmi Venkatraman, 2019. Deep Learning Approach 50859. IEEE. DOI: https://doi.org/10.1109/ACCESS.2018.2868993.
for Intelligent Intrusion Detection System. IEEE Access. IEEE, pp.41525 – 41550. [18] Jose Camacho, Roberto Therón, Jose M. García-Giménez, Gabriel Maciá-
DOI:https://doi.org/10.1109/ACCESS.2019.2895334. Fernández and Pedro García-Teodoro, 2019. Group-Wise Principal Component
[6] Giang Nguyen, Stefan Dlugolinsky, Viet Tran, Alvaro Lopez Garcia, 2020. Deep Analysis for Exploratory Intrusion Detection. IEEE Access, 113081 – 113093. IEEE.
Learning for Proactive Network Monitoring & Security Protection. IEEE Access. DOI: https://doi.org/10.1109/ACCESS.2019.2935154.
IEEE, pp. 19696 - 19716. DOI:https://doi.org/10.1109/ACCESS.2020.2968718. [19] Ao Liu and Bin Sun, 2019. An Intrusion Detection System Based on Quantitative
[7] Sydney Mambwe Kasongo, Yanxia Sun, 2019. A Deep Learning Method with Filter Model of Interaction Mode Between Ports. IEEE Access, 161725 – 161740. IEEE.
Based Feature Extraction for Wireless Intrusion Detection System. IEEE Access. DOI: https://doi.org/10.1109/ACCESS.2019.2951839.
IEEE, pp. 38597 - 38607. DOI:https://doi.org/10.1109/ACCESS.2019.2905633. [20] Jin Yang, Tao Li, Gang Liang, Wenbo He and Yue Zhao, 2019. A Simple Recurrent
[8] Mohammed Mehedi Hassan, Abdu Gumaei, Ahmed Alsanad, Majed Alrubaian, Unit Model Based Intrusion Detection System With DCGAN. IEEE Access, 83286
Giancarlo Fortino, 2019. A Hybrid Deep Learning Model for Efficient Network – 83296. IEEE. DOI: https://doi.org/10.1109/ACCESS.2019.2922692.
Intrusion Detection in Big Data Environment. Information Sciences. Elsevier. [21] Shahid Anwar, Jasni Mohamad Zain, Mohamad Fadli Zolkipli, Zakira Inayat,
DOI: https://doi.org/10.1016/j.ins.2019.10.069. Suleman Khan, Bokolo Anthony and Victor Chang, 2017. From Intrusion Detec-
[9] Samson Ho, Saleh Al Jufout, Khalil Dajani, Mohammed Mozumdar, 2021. A Novel
Intrusion Detection Model for Detecting Known and Innovative Cyberattacks Us- tion to An Intrusion Response System: Fundamentals, Requirements, And Future
ing Convolutional Neural Network. IEEE Open Journal of the Computer Society, Directions. Algorithms, 10 (2), 39. MDPI. DOI: https://doi.org/10.3390/a10020039.
ArabWIC 2021, August 25, 26, 2021, Sharjah, United Arab Emirates Archana C et al.

[22] Enamul Kabir, Jiankun Hu, Hua Wang, Guangping Zhuo, 2018. A Novel Statistical [23] Adel Binbusayyis and Thavavel Vaiyapuri, 2019. Identifying and Benchmarking
Technique for Intrusion Detection Systems. Future Generation Computer Systems, Key Features for Cyber Intrusion Detection: An Ensemble Approach. IEEE Access,
79 (1), 303-318. Elsevier. DOI: https://doi.org/10.1016/j.future.2017.01.029. 106495 – 106513. IEEE. DOI: https://doi.org/10.1109/ACCESS.2019.2929487.