A Novel IoT-Based Explainable

Deep Learning Framework for

Intrusion Detection Systems
Zakaria Abou El Houda, Bouziane Brik, and Sidi-Mohammed Senouci

Abstract
The growth of the Internet of Things (IoT) is accompanied by serious cybersecurity risks, especially with the emergence of IoT botnets. In
this context, intrusion detection systems (IDSs) proved their efficiency in detecting various attacks that may target IoT networks, especially
when leveraging machine/deep learning (ML/DL) techniques. In fact, ML/DL-based solutions make “machine-centric” decisions about
intrusion detection in the IoT network, which are then executed by humans (i.e., executive cyber-security staff). However, ML/DL-based
solutions do not provide any explanation of why such decisions were made, and thus their results cannot be properly understood/exploited
by humans. To address this issue, explainable artificial intelligence (XAI) is a promising paradigm that helps to explain the decisions of ML/
DL-based IDSs to make them understandable to cyber-security experts. In this article, we design a novel XAI-powered framework to enable
not only detecting intrusions/attacks in IoT networks, but also interpret critical decisions made by ML/DL-based IDSs. Therefore, we first
build an ML/DL-based IDS using a deep neural network (DNN) to detect and predict IoT attacks in real time. Then we develop multiple
XAI models (i.e., RuleFit and SHapley Additive exPlanations, SHAP) on top of our DNN architecture to enable more trust, transparency, and
explanation of the decisions made by our ML/DL-based IDS to cyber security experts. The in-depth experiment results with well-known IoT
attacks show the efficiency and explainability of our proposed framework.

Introduction such systems do not give any explanation/interpretation about

why such decisions are made; hence, their results cannot be
The Internet of Things (IoT) is an emerging paradigm that has understood by humans. In other words, the main drawback of
gained momentum and is now shaping our future [1–3]. IoT existing ML/DL-based IDSs, particularly the most accurate ones,
aims to transform daily life by deploying billions of smart devic- are the black-box decisions whose internal functioning is hidden
es, around 75 billion IoT devices by 2025 [2], to perform daily and not understood.
tasks. Thus, IoT is becoming a key pillar of different sectors, Recently, explainable artificial intelligence (XAI) has emerged
including healthcare, agriculture, transportation, and manu- as a promising paradigm to develop new approaches explaining
facturing [1, 2]. However, with the rapid deployment of IoT, how ML/DL models work. XAI aims to make ML/DL models
numerous IoT vulnerabilities have emerged as well. In fact, new understandable for experts in the domain [9, 10]. This also
sophisticated and destructive IoT attacks are increasing. For enables experts to trust and adapt such models and hence
instance, the Mirai IoT botnet performed a huge attack using release their decisions (models).
many compromised IoT devices, including IoT gateways, closed In this article, we design a new XAI-powered framework that
circuit television cameras, and routers. This subsequently result- comprises two main modules:
ed in the unavailability of many Internet services such as Twitter • DL-based IDS: We first build a DL-based prediction model
and Amazon for several hours [4]. In addition, such IoT attacks of intrusions for IoT applications. To do so, we leverage the
may cause extensive financial loss and huge damage. Accord- UNSW-NB15 dataset [11] and design a neural network to
ing to a recent report [5], it is estimated that the financial loss create our prediction model.
caused by the IoT attacks was about US$20 billion in 2021. • XAI-enabled IDS: We develop several XAI models on top of
To deal with IoT attacks, different security measures are our DL-based IDS to add more transparency and interpret-
used, including firewalls, anti-virus, and access control, in order ability to our DL-based IDS’s decisions.
to filter and control incoming network traffic [6]. However, Specifically, we implement RuleFit and SHapley Addi-
these measures are not sufficient/efficient to protect the net- tive exPlanations (SHAP) as white-box models related to our
work [4], especially with the emergence of IoT attacks. As a DL-based black-box model. Therefore, our framework enables
solution, intrusion detection systems (IDSs) should be efficiently not only timely detection of intrusions in IoT networks, but also
designed to secure the IoT network against various attacks interpretation of decisions made by our DL-based model. This
ranging from distributed denial of service (DDoS) to scanning introduces more trust and transparency among our DL-based
attacks. In this context, IDSs have proved their efficiency in IDS and experts who will execute its decisions. This article is
detecting various attacks that may target IoT networks, especial- organized as follows. The next section gives a review of related
ly when leveraging machine/deep learning (ML/DL) techniques work. We then describe the design and specification of our pro-
[7, 8]. In fact, ML/DL techniques consist of learning the char- posed XAI-powered framework. Following that, we present the
acteristics of each attack so that we can quickly and efficiently performance evaluation of our proposed XAI-powered frame-
identify/detect existing and new IoT attacks, without having to work. Finally, we conclude the article.
update traditional IDS rules. Hence, ML/DL-based IDSs make
“machine-centric” decisions about intrusion detection, which Related Work
are then executed by humans (i.e., executive staff). However, In this section, we briefly present the main works that address
the explainability of ML/DL-based IDSs, along with their lim-
Zakaria Abou El Houda is with the University of Montreal, Canada.
itations. In [12], a deep neural network (DNN) is first used for
Bouziane Brik and Sidi-Mohammed Senouci are with the University of Bourgogne
the network IDS, and then an XAI-based framework is designed
Franche-Comté, France.
to improve the transparency of the DL model. The authors
Digital Object Identifier: 10.1109/IOTM.005.2200028 leveraged the NSL KDD dataset to implement and validate

20 2576-3180/22/$25.00 © 2022 IEEE IEEE Internet of Things Magazine • June 2022

Authorized licensed use limited to: UNIVERSITY OF SHARJAH. Downloaded on January 10,2024 at 12:27:01 UTC from IEEE Xplore. Restrictions apply.
 several XAI approaches, including SHAP, the Decisions or recommendations
contrastive explanations method, LIME, and
ProtoDash. Similarly, another framework used
the SHAP approach to improve the transparen-

Explanation Interface
IDS Deep
Predictions
cy of IDSs of any ML/DL-based IDS system, in
Learning Model
Who ? Users of the models
Objective ? Understand and trust the model itself,
[9]. The authors used also the NSL-KDD dataset Data
to test the performance of the framework. Who ? Executive staff
In [10], XAI is integrated with an ML-based IDS XAI
Objective ? Understand and execute the model’s decision,
Explanations

IDS to deal with adversarial attacks. First, a

Model

random forest classifier is built to detect net- IoT Network

work intrusions. Then the SHAP approach is
applied to explain and interpret the outputs Figure 1. General architecture of our XAI-based framework for IoT IDSs.
of the random-forest-based model. The perfor-
mance of this scheme is evaluated using the
CICIDS dataset. Furthermore, a layer-wise relevance propaga- ber (dtcpb) [0;4*10 9]” and “Source TCP sequence number
tion (LRP) method is used in [13] to determine input feature (stcpb) [0;4*109]”) have higher values than other features (e.g.,
relevance and send offline and online feedback to end users, “Source IP address (srcip) [0;39]” and “Destination to source
to help them deduce which features have more impact on the time to live (dttl) [0;254]”); which may impact the final model
predictions made by the IDS. In [14], an explanation approach decisions. The latter may miss important features that have
is proposed to deal with incorrect classifications made by ML/ minimum values (i.e., source time to live (dttl)). Thus, we have
DL-based IDSs. This approach helps to determine the modifica- applied the standardization technique to overcome this issue.
tions needed to correctly classify a given dataset sample. These Finally, we encoded the Labels/output features (e.g., back-
modifications are also exploited to deduce the most important doors, Shellcode, and Fuzzers) into numerical values.
features that justify the reason for the incorrect classification. Furthermore, to test the effectiveness of our proposed
The designed approach is evaluated and tested using the NSL- XAI-powered framework, we constructed a DNN model with
KDD dataset. Even the above works leveraged XAI to interpret an input layer of 49 neurons, which corresponds to the dimen-
ML/DL-based IDS; however, some of these works are limited sion of the input sample of the UNSW-NB15 dataset, five hid-
to traditional ML algorithms, which are less complex and easy den layers with a leaky rectified linear unit, and an output layer
to interpret compared to DL [10]. In addition, most of them of one neuron. We implemented our proposed XAI-powered
designed a general XAI framework whatever the targeted ML/ framework using Pytorch and the SHAP Library [15], an open
DL-based IDS [9, 13]. This may not be realistic, since each ML/ source library that includes various functions to explain the out-
DL-based IDS model has its own specific input features and per- put of ML/DL-based models. In this work, we have considered
formance, and the XAI framework should consider such char- two techniques, namely RuleFit and SHapley Additive exPlana-
acteristics as input to then be able to explain an ML/DL-based tions (SHAP), to effectively interpret a DL-based IDS model’s
IDS’s decisions. decisions/classifications. The objective is to explore linear and
nonlinear methods, including local and global explanations. In
O F
ur ramework to xplain E DL-B
ased IDS RuleFit, we learn sparse linear models/forms that include the
effects of interaction in a decision making rule-based form;
I TA
of o pplications it creates new features in the form of decision making rules
In this section, we first present our system architecture. Then we and constructs a transparent model with these features. RuleFit
present our DNN architecture to predict IoT intrusions/attacks in includes two steps: it trains a tree-based model and uses it to
real time, and our XAI models to explain our DL-based IDS. create the decision rules; and it trains a sparse linear model
(e.g., LASSO) to select the most informative/significant features.
System Architecture SHAP is a well-known unified framework for model interpreta-
Figure 1 shows the general system architecture of our designed tion; it explains the predictions of an input data sample by cal-
framework. The data collected from the IoT network will be exploit- culating the contribution of each feature to the final decision/
ed, on one hand, to build a DL-based model to predict/detect prediction. This contribution can be either positive or negative.
intrusions in the IoT network. On the other hand, an XAI model is The main advantage of SHAP is that it can be applied to any
created that leverages both the sensed data and DL-based mod- model, rather than only simple/linear models. Also, instead of
el’s predictions in order to explain/interpret such predictions. This looking only at local decisions/interpretations, SHAP looks at
enables explaining not only how the DL-based model works, but the overall/global interpretations by summing the input values
also why its predictions and hence decisions are made. Note that of the features and averaging all columns/features individually.
performed predictions with their explanations are shown in an
online way to different audiences through an explanation inter- P
erformance valuation E
face. Moreover, our framework targets both users of the DL-based The feature importance scores shows the most important/rel-
model and executive staff. The users of the model should under- evant features among all features of a dataset; these features
stand and trust the model predictions before transferring the mod- have more significant impact on the model predictions than
el’s decisions to the executive staff, and should also understand the other features. Our proposed XAI-powered framework investi-
received decisions and execute them. In the following, we present gates the use of both RuleFit and SHAP methods to select the
our explainable DL-based IDS suitable for IoT applications. most informative/significant features and explores their effect
on final model predictions. Figure 2 shows the feature impor-
Explainable Deep-Learning-Based IDS of IoT Applications tance scores on UNSW-NB15 dataset using RuleFit and SHAP,
In this work, we leverage the UNSW-NB15 dataset for the attack respectively; it shows the highest scoring features in descending
traffic. UNSW-NB15 is a synthetic network security dataset that order.
contains 100 GB of network data samples, including several For the RuleFit method, the highest scoring features corre-
IoT attacks (e.g., backdoors, DoS, and worms). For the pre-pro- spond to the following:
cessing phase, we have encoded the categorical/non-numeric • sttl: Source to destination time to live
input features (i.e., “service,” “proto,” and “state”) into numeric • ct_state_ttl: The number of each state (e.g., ACC, CLO)
values using one hot encoding techniques. Some of the the according to a range of values for source/destination time to
UNSW-NB15 features (e.g., “Destination TCP sequence num- live (ttl)

IEEE Internet of Things Magazine • June 2022 21

Authorized licensed use limited to: UNIVERSITY OF SHARJAH. Downloaded on January 10,2024 at 12:27:01 UTC from IEEE Xplore. Restrictions apply.
 a) b)
Figure 2. Feature importance scores on the UNSW-NB15 dataset for: a) RuleFit; b) SHAP.

• sport: Source port number

• dstip: Destination IP address
Figure 3 shows the data distribution of dataset
features. Figure 3a shows some of the highest
scoring features based on RuleFit and SHAP,
while Fig. 3b shows the other non-irrelevant fea-
tures. We observe that the most relevant fea-
tures, computed based on RuleFit and SHAP,
can effectively distinguish the two classes (i.e.,
Normal and Attack), because the data distribu-
tion of the two classes is completely different,
while the data distribution of the two classes
a)
is similar for the other non-irrelevant features,
which makes classification difficult for the IDS.
Figure 4 shows the interpretation of our DNN
model on the UNSW-NB15 dataset using the
SHAP method. Instead of examining decisions of
our DNN model locally, we examine the overall/
global feature importance of the UNSW-NB15
dataset using SHAP, we sum up the input values,
and we average all the columns/features indi-
vidually. For a particular observation, each input
feature value (e.g., (Sload) source bits per sec-
ond, (stcpb) source TCP sequence, and (dtcpb)
destination TCP sequence) has either a positive
or negative contribution to the final decision
b) (i.e., base value). In our analysis, we have exam-
ined three observations. Figure 4a shows the first
Figure 3. Data samples distribution of features of the UNSW-NB15 dataset in terms observation in which the data sample is Normal
of: a) the highest scoring features using RuleFit and SHAP; b) the other non- (i.e., non-attack) and the DNN model correctly
irrelevant features. predicted/detected as a Normal data sample. In
this observation, the values of the input features
• service: The used protocol (e.g., http, dns, ssh) are as follows: Sload is equal to 4.5 * 104, stcpb is equal to 1.43
• dsport: The destination port number * 109, and dtcpb is equal 3.5 * 109. Figure 4b shows the second
• ct_srv_dst: The number of connections that contain the same observation in which the data sample is an IoT attack and the
service and destination address in the last 100 connections DNN model correctly predicted/detected it as an IoT attack. In
• sbytes: Source to destination bytes this observation, the values of the input features are as follows:
For the SHAP method the highest scoring features correspond Sload is equal to 4.9*105, stcpb is equal to 0, and dtcpb is equal
to the following features: 0. Figure 4c shows the last studied observation in which the data
• srcip: The source IP address sample is an IoT attack that the DNN model predicted as a nor-
• ct_dst_src_ltm: The number of connections that contain the same mal data sample (i.e., false negative, FN). In this observation, the
service and destination address in the last 100 connections values of the input features are as follows: Sload is equal to 1.8
• ct_dst_sport_ltm: The number of connections pf the same desti- * 109, stcpb is equal to 5.8 * 109, and dtcpb is equal 2.7 * 109.
nation address and the source port in the last 100 connections In all these observations, the blue features push the prediction of

22 IEEE Internet of Things Magazine • June 2022

Authorized licensed use limited to: UNIVERSITY OF SHARJAH. Downloaded on January 10,2024 at 12:27:01 UTC from IEEE Xplore. Restrictions apply.
 Figure 4. Interpretation of our DNN model on the UNSW-NB15 dataset with: a) Sload of 4.5 * 104, a stcpb
of 1.43 * 109, and a tcpb of 3.5 * 109; b) Sload of 4.9 * 105, a stcpb of 0, and a dtcpb of 0; c) Sload of
1.8 * 109, a stcpb of 5.8 * 109, and a dtcpb of 2.7 * 109.

[4] Z. Abou El Houda, L. Khoukhi, and A. Senhaji Hafid, “Bringing Intelligence to

the data sample to be normal (i.e., class 0). The larger the shaft, Software Defined Networks: Mitigating DDoS Attacks,” IEEE Trans. Network
the more effect this input feature of the UNSW-NB15 dataset and Service Management, vol. 17, no. 4, 2020, pp. 2523–35.
[5] S. Morgan, “Global Ransomware Damage Costs Predicted to Reach $20 Bil-
has on the final detection/prediction. In the first scenario (Fig. lion (USD) by 2021”; https://cybersecurityventures.com/.
4a), we observe that the most contributing/significant features [6] S. B. Saad, A. Ksentini, and B. Brik, “A Trust Architecture for the SLA Manage-
are Sload and dtcpb. The red feature (i.e., stcpb) reduces the ment in 5G Networks,” Proc. IEEE ICC 2021, 2021.
probability for a data sample to be normal. In the second sce- [7] Z. A. El Houda, A. S. Hafid, and L. Khoukhi, “A Novel Machine Learning Frame-
work for Advanced Attack Detection Using SDN,” Proc. IEEE GLOBECOM, 2021.
nario (Fig. 4b), we observe that the most contributing/significant [8] Z. A. El Houda et al., “When Federated Learning Meets Game Theory: A
features are stcpb and dtcpb. The red features (i.e., stcpb and Cooperative Framework to Secure Iiot Applications on Edge Computing,” IEEE
dtcpb) drive the probability for a data sample to be an attack. In Trans. Industrial Informatics, 2022.
the last scenario (Fig. 4c), we observe that the most contributing/ [9] M. Wang et al., “An Explainable Machine Learning Framework for Intrusion
Detection Systems,” IEEE Access, vol. 8, 2020, pp. 73,127–41.
significant features are Sload and dtcpb. The red feature (i.e., [10] S. Wali and I. Khan, “Explainable AI and Random Forest Based Reliable
stcpb) reduces the probability for a data sample to be normal. Intrusion Detection System Detection System,” Dec. 2021. DOI:10.36227/
Thus, such solid knowledge makes cybersecurity experts more techrxiv.17169080.v1.
convinced of the decisions made by ML/DL-based IDSs. [11] N. Moustafa and J. Slay, “Unsw-nb15: A Comprehensive Data Set for Net-
work Intrusion Detection Systems (Unsw-nb15 Network Data Set),” Proc.
conclusIon 2015 Military Commun. Info. Systems Conf., 2015.
[12] S. Mane and D. Rao, “Explaining Network Intrusion Detection System Using
We propose a novel XAI-powered framework that enables not Explainable AI Framework,” 2021; https://www.researchgate.net/publica-
only the detection of IoT attacks, but also the interpretation of tion/350061199_Explaining_Network_Intrusion_Detection_System_Using_
Explainable_AI_Framework.
critical decisions made by ML/DL-based IDSs. First, we built a [13] K. Amarasinghe and M. Manic, “Improving User Trust on Deep Neural Net-
DNN model to detect IoT attacks in real time. Then we devel- works Based Intrusion Detection Systems,” Proc. IECON 2018 — 44th Annual
oped multiple XAI models on top of our DNN architecture to Conf. IEEE Industrial Electronics Society, 2018, pp. 3262–68.
enable more trust, transparency, and explainability of the deci- [14] D. L. Marino et al., “An Adversarial Approach for Explainable Ai in Intrusion
Detection Systems,” 2018. DOI: 10.1109/IECON.2018.8591457.
sions made by our ML/DL-based IDS to cyber-security experts. [15] Shap (shapley additive explanations) library; https://shap.readthedocs.io/en/
The in-depth experiment results on well-known IoT attacks latest/index.html.
show the efficiency of our framework. This makes it a promising
cyber-security framework for accurate IoT attack detection and bIographIes
expalainability of a deep learning framework for IDSs. ZAKARIA ABOU EL HOUDA received his Ph.D. degree in computer science from
both the University of Montreal, Canada, and the University of Technology of
reFerences Troyes, France. His current research interests include ML/DL-based intrusion
detection and blockchain.
[1] B. Brik et al., “Thingsgame: When Sending Data Rate Depends on the Data
Usefulness in IoT Networks,” Proc. 14th Int’l. Wireless Commun. Mobile Com-
BOUZIANE BRIK is currently working as an associate professor at the University
puting Conf., 2018, pp. 886–91.
of Burgundy, France. His research interests include IoT, IoT in industrial systems,
[2] L. Horwitz, “The Future of IoT Miniguide: The Burgeoning IoT Market Con-
smart grid, and vehicular networks.
tinues”; https://www.cisco.com/c/en/us/solutions/internet-ofthings/future-
of-iot.html.
SIDI-MOHAMMED SENOUCI received his Ph.D. in computer science in October 2003
[3] S. Ben Saad, A. Ksentini, and B. Brik, “An End-to-End Trusted Architecture for
from the Sorbonne University and his HDR from INP Toulouse, France. He is a
Network Slicing in 5G and Beyond Networks,” Security and Privacy, vol. 5, no.
professor at ISAT, a major French post-graduate school located in Nevers, France.
1, 2022, p. e186.

IEEE Internet of Things Magazine • June 2022 23

Authorized licensed use limited to: UNIVERSITY OF SHARJAH. Downloaded on January 10,2024 at 12:27:01 UTC from IEEE Xplore. Restrictions apply.