RMDNet-Deep_Learning_Paradigms_for_Effective_Malware_Detection_and_Classification
RMDNet-Deep_Learning_Paradigms_for_Effective_Malware_Detection_and_Classification
ABSTRACT Malware analysis and detection are still essential for maintaining the security of networks
and computer systems, even as the threat landscape shifts. Traditional approaches are insufficient to keep
pace with the rapidly evolving nature of malware. Artificial Intelligence (AI) assumes a significant role in
propelling its design to unprecedented levels. Various Machine Learning (ML) based malware detection
systems have been developed to combat the ever-changing characteristics of malware. Consequently, there
is a growing interest in exploring advanced techniques that leverage the power of Deep Learning (DL)
to effectively analyze and detect malicious software. DL models demonstrate enhanced capabilities for
analyzing extensive sequences of system calls. This paper proposes a Robust Malware Detection Network
(RMDNet) for effective malware detection and classification. The proposed RMDNet model branches the
input and performs depth-wise convolution and concatenation operations. The experimental results of the
proposed RMDNet and existing DL models are evaluated on 48240 malware and binary visualization image
dataset with RGB format. Also on the multi-class malimg and dumpware-10 datasets with grayscale format.
The experimental results on each of these datasets demonstrate that the proposed RMDNet model can
effectively and accurately categorize malware, outperforming the most recent benchmark DL algorithms.
INDEX TERMS Binary classification, concatenation, convolution, cyber security, deep learning, depthwise
convolution, malware, multiclass classification.
2024 The Authors. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.
82622 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ VOLUME 12, 2024
S. Puneeth et al.: RMDNet-DL Paradigms for Effective Malware Detection and Classification
personal or sensitive information [1]. Particularly within the models have demonstrated exceptional performance, their
realm of Internet of Things (IoT), the need for strong security decision-making process often lacks transparency, hindering
measures is paramount to uphold consumer trust. However, the understanding of how and why a classification or
due to factors such as cost, size, performance, and security detection decision is made [10]. Explore methods such as
management often takes a lower priority in IoT deployments. attention mechanisms and interpretability techniques to shed
Consequently, IoT becomes vulnerable to security breaches, light on the reasoning behind the DL model outputs. Through
resulting in substantial, financial and reputation damages [2]. extensive experimentation and evaluation, empirical evidence
Internet security is facing a significant menace as malware showcasing the effectiveness of DL techniques in malware
attacks continue to surge at an exponential rate. It can affect analysis. Results obtained in this paper reveal improvements
regular processes, gather sensitive information, and obtain in detection accuracy, robustness against emerging malware
superuser rights to carry out malicious acts. Attackers deliver variants, and the capacity to identify hidden patterns and
it to the victim’s PC by exploiting security weaknesses in characteristics that may not be detectable using standard
operating systems or application software [3]. analytic approaches. This work intends to contribute to the
Traditional techniques to malware detection and analysis improvement of malware analysis by harnessing the power
are struggling to keep up with the rising sophistication of DL techniques. By capitalizing on the capabilities of
and diversity of malware threats. While effective against neural networks, we can enhance the accuracy, efficiency, and
known malware variants, signature-based methods, and adaptability of malware detection and analysis systems.To
rule-based systems often fail to detect novel and evolving defend against zero-day attacks, the proposed RMDNet
threats [4]. As a result, there is an essential need to excels at detecting anomalies and recognizing novel attack
investigate sophisticated methodologies capable of adapting signatures, providing a proactive defense against previously
to and successfully analyzing the ever-changing environment unseen threats. RMDNet’s ability to autonomously adapt
of malicious software [5]. DL is a subfield of ML that to evolving threats without explicit programming makes it
focuses on artificial neural networks with multiple layers, valuable for real-time detection and mitigation. The key
enabling the automatic learning and extraction of complex contributions of this study are as follows:
patterns and features from data. Inspired by the structure 1) A RMDNet-deep learning based malware classification
and function of the human brain, DL algorithms excel at algorithm is proposed to efficiently differentiate classes
tasks such as image and speech recognition, natural language of malware samples while retaining high accuracy
processing, and pattern recognition. The depth of the neural on different malware datasets and also emphasizing
networks allows them to autonomously learn hierarchical computational efficiency.
representations, making DL particularly powerful for tasks 2) Propose a DCOCO block, that performs depth wise
requiring intricate and nuanced understanding of data. [6]. convolution and concatenation methods and has the
This ability makes DL an appealing approach for tackling the ability to perform effective and efficient feature
challenges in malware analysis. The fundamental objective of extraction.
malware analysis is to uncover and understand the behavior, Rest of the article is organized as follows: Section II
objectives, and capabilities of malicious software [7]. This discusses various methods for categorizing and detecting
process involves static analysis, which examines the structure malware. The proposed RMDNet architecture model’s com-
and content of malware samples without executing them, and prehensive description and the datasets are the main topics
dynamic analysis, which observes malware’s behavior when of Section III. The proposed RMDNet model’s training and
executed in a controlled environment. implementation details are described in Section IV. The
DL techniques can significantly contribute to both static experimental results of our model are detailed in Section V
and dynamic analysis by automating feature extraction, along with comparisons of various benchmark models.
improving detection accuracy, and providing insights into the Section VI of our paper presents the results of our research.
inner workings of malware. In this paper, we delve into the
realm of malware analysis using DL techniques. We aim to II. RELATED WORK
explore the potential of various DL models in effectively Malware detection is crucial for securing computer systems
analyzing and detecting malware [8]. To accomplish our and user data in the area of computer security. To fight
objectives, we leverage large-scale datasets consisting of the constantly changing threat landscape, several malware
binary and diverse types of malware samples. These datasets detection techniques have been developed as shown in
encompass a wide range of malware families, such as Figure 1. These techniques are classified as static analysis,
viruses, worms, and trojans, providing a comprehensive dynamic analysis, feature extraction techniques, and DL
representation of the threat landscape [9]. Discussing the techniques.
pre-processing steps involved in preparing the data for
training DL models, ensuring data integrity, and appropriate A. STATIC ANALYSIS
feature extraction. Furthermore, addressing the challenges In this approach, most of the antivirus software used
associated with the interpretability and explainability of for detection uses the signature-based technique. These
DL models in the context of malware analysis. While DL signatures are generated by gazing at the disassembled
VOLUME 12, 2024 82623
S. Puneeth et al.: RMDNet-DL Paradigms for Effective Malware Detection and Classification
code or using the string command. Disassembling portable analysis is essential in the identification and categorization
executables is made easier by a variety of disassemblers and of malware. Dynamic analysis involves executing malware
debuggers. Thus, features are taken from disassembled code, samples in a controlled environment to observe their behavior
and it is analyzed. Therefore, these characteristics are crucial during runtime. An effective method used to investigate
in creating the signature of a specific malware family. Static malware is by analyzing the program’s behavior during
analysis is a technique for assessing malware without running execution [16], [17]. This approach, also known as behavioral
it [11]. This is often accomplished by analyzing the coding analysis, involves observing and collecting information from
of a binary file in order to comprehend its operation and the operating system including API call sequence analysis,
discover any malicious activities. Potential security issues in a system call monitoring, network traffic analysis, sandboxing,
sample can be discovered via static analysis without affecting and memory behavior analysis. However, dynamic analysis
the analysis environment. Static analysis does not involve the does come with some limitations. For example, accurately
execution of the actual malware. It is safer because of this simulating the conditions necessary to trigger the malware’s
non-execution environment, which lowers the possibility of dangerous functionalities, especially when targeting specific
unintentional activation or unexpected repercussions during vulnerable applications, can be quite challenging [18].
dynamic analysis [12]. They are quick, safe, and can readily Additionally, determining the precise time frame required
detect multi-path malware. But they may make errors while to observe harmful behavior for each malware instance
analyzing malware that is unknown. Attackers employ several remains uncertain. To accomplish categorization, the ML
ways to find vulnerable devices and transform them into approach makes use of various aspects of the malware
bots using infection scripts. Data botnet arrays are then samples [19]. This technique can give great accuracy, but
leveraged to conduct network traffic attacks. Opcodes are it involves significant effort to run the malware files and
building blocks that have historically been utilized for does not guarantee the execution of the malware’s entire
malware detection and statically analyzing program activity. code, thus the harmful section of the malware code may not
The control flow graph method (CFG) is used to extract be identified. Executing malware in controlled environments
the executable opcodes, representing behavior characteristic poses risks of unintentional infection and contamination if
executable [13], [14]. Features are chosen from CFG-based proper isolation measures are not in place. Dynamic anal-
and text-based sequences, if packed malware cannot be ysis can be computationally intensive, requiring substantial
unpacked, this strategy is rendered insignificant. Formulating resources and potentially impacting system performance
a Convolution Recurrent Neural Network (CRNN) to detect during analysis [20]. Some extremely sophisticated malware
malware using an N-extracted opcode sequence from a binary can identify the analysis environment and change their
file without execution doesn’t reflect in indirect branching behavior to avoid detection, decreasing the effectiveness of
instructions. Information about the program or its intended dynamic analysis.
behavior is gathered from explicit and implicit observations
in its binary/source code through static analysis [15]. Static
C. FEATURE EXTRACTION TECHNIQUES
analysis solutions are often created using signature-based
approaches, but even with their extreme precision, they are Several efforts have been made to adapt feature extraction
ineffective against unknown malicious code. techniques for the classification of malware. There have been
surveys of numerous visualization approaches, including
image processing for malware analysis. The visualization of
B. DYNAMIC ANALYSIS malware as images [21], which presented the first studies
As traditional static approaches fail to keep up with the on the viewing of binary data as images, improved the
increasing sophistication of malicious software, dynamic capabilities of text-based hex editors, and provided a method
for converting binary files into images called byteplots. Malware detection and classification using a deep random
A method for visualizing static malware samples as grayscale forest approach, and a sliding window were proposed [28].
images, discovering that images from the same malware However, because it uses smaller versions of the input
family appear very similar in structure and texturing for image for each sliding window, this uses more memory.
many malware families [22]. Figure 2 depicts the process of To investigate informative aspects from the one-dimensional
converting malware binary data to images. Malware binary structure of binary executables, a byte-level 1D CNN model
files, which are in the form of bits are grouped into eight-bit was presented [29]. While binary executables were being
vectors and then converted to grayscale images. converted and resized to larger images, such as 128 ×
A recurring element of these initiatives is the transforma- 128, this 1D CNN did not always perform better. And
tion of binary malware samples into various image formats, also suggest learning the useful characteristics from larger
followed by the implementation of image classification images may require a more complex model, like ResNet
algorithms to categorize based on the image representation or EfficientNet. Deep Image Mal Detect (DIMD) model
of the malware. Using local gray level cooccurrence matrices was proposed, consisting of Deep Neural Nework (DNN)and
and global color moments, features from both grayscale and Long Short Term Memory (LSTM). The highest accuracy of
color byteplots are extracted and then sent to classifiers. this CNN-LSTM model was 96% with a cross-validation of
This was tested on fifteen malware families and showed 10 fold. Results in this model suggest a scope of improvement
that scoring 97 % in accuracy, the combined feature sets is required to develop DL complex model [30]. Experiments
outperformed either local or global features alone. There were conducted on different ResNet models and transfer
hasn’t been a lot of study on malware classification using learning for malware classification, with more complicated
space-filling curves, in contrast to the byteplot related ResNet variants that did not yield a meaningful improvement
work that was mentioned [23]. An approach using Hilbert in results [31]. An Alex Net and Resnet hybrid DNN was
curves and a Self-Organizing Incremental Neural Network presented, integrating the two pre-trained networks to provide
were employed to classify malware, very small sample a feature vector and fully connected layers for categorization.
size was used and it is insufficient to properly show the The limitations of the model were the adversary’s attacks
benefits of classifying malware using the Hilbert curve, were not tested using crafted inputs, and complexity in rises
Overfitting makes it difficult for the model to generalize more hidden layers [32]. A model on Convolution Recurrence
to new samples [24]. SimHash keeps the malware’s unique (ConRec), based on VGG16 and BiLSTM was used along
characteristics while encoding them to identical lengths. with image augmentation on the malware samples. Model
When converting SimHash bits to grayscale images, each performance without image augmentation was less [33].
SimHash value may be viewed as a pixel. SimHash can It was suggested to employ transfer learning-based architec-
be improved by employing multiple cascade hash functions ture rather than class balancing techniques to identify mal-
rather than a single hash result [25]. Bitmap Image Converter, ware from different families [34]. It utilized spatial attention
a technique that accepts binary files from Windows Portable created by CNN, as well as feedforward and dropout layers
Executables (PE) as input and converts them into bitmap with less trainable parameters. On the Malimg benchmark
images in order to visualize them. In order to assess the dataset, the performance was examined and an accuracy
similarity of the original binary files, each line of bitmap of 97.68 % was obtained. Malware classification utilizing
images has an entropy value, which is calculated by the Co-Lab image, VGG 16, and Support vector machine
entropy graph generator, and these values are used to create (MalCVS) presented, with image feature extraction using
entropy graphs, it incorrectly classified malware binary files a fine-tuned VGG16 model. Following that, the retrieved
belonging to few families of malware [26]. The bytes transfer features are used to construct a multi-class SVM mode.
probability matrix based Markov images are fixed-size pixel Due to malware’s ability to change or confuse the header
matrices. It ignores the scaling issue when compared to field of PE files after packing, this visualization technique
grayscale images. Malware binaries are seen as a stream of cannot classify malware that has been packaged [35].
bytes that may be visualized as a stochastic process [27]. Using transfer learning using ShuffleNet and DenseNet-201,
in the final classification layer, an ensemble configuration
D. DEEP LEARNING TECHNIQUES of Support Vector Machines (SVM) with Optimal Error
Due to the improved feature learning ability of convolu- Correction Output Coding (ECOC). This model reported an
tion neural network (CNN) from malware images, several accuracy of 99.14 % on the malimg dataset and 96.62 % on
researchers have tried to contribute elegant DL techniques dumpware 10 datasets [37]. VGG16 and ResNet-50 ensemble
to work on malware analysis. Figure 3 shows the basic of CNN architectures to extract malware image bottleneck
block diagram or workflow diagram of the DL method. The characteristics, which were subsequently used to train SVM
malware datasets are pre-processed and split into training classifiers. The computational complexity of this model was
and testing data. Training data is applied on different DL high [41].
algorithms and its performance is evaluated by applying test Static approach remains invaluable in identifying known
datasets. Models are built on the continuous evaluation of the threats and lays the groundwork for subsequent stages
performance of the model using quality metrics. of analysis. Although dynamic malware analysis provides
VOLUME 12, 2024 82625
S. Puneeth et al.: RMDNet-DL Paradigms for Effective Malware Detection and Classification
useful insights into the behavior of harmful software, it is into two sections, section I and section II. Section I starts with
not a solution. Its limitations, including evasion techniques, a convolutional layers (Conv2D) with 64 filters, each having
time-dependent behavior, and the complexity of real-world a kernel size of 3 × 3 and ReLU activation. These layers are
environments. However, the true innovation of this study responsible for extracting high-level features from the input
lies in the fusion of DL techniques with the established images. Batch normalization is applied after each Conv2D
practices. DL with its ability to learn feature hierarchies layer to accelerate training and stabilize the learning process
independently from malware data, overcomes the constraints of the model. Following batch normalization, max pooling is
of handcrafted features. To overcome these limitations, performed to downsample the feature maps spatially while
a robust malware detection and classification model is preserving essential information. The output equation of
required, and to address this issue, we propose the RMDNet Section I is given in equation1, the ReLU activation function
architecture for efficiently differentiate classes of malware is represented as ϕ, batch normalization is represented as BN,
samples while retaining high accuracy on different malware and the max pooling layer as ϑ.
datasets. Detailed description about proposed RMDNet is
presented in Section III. Xsec1 = ϑ(BN(ϕ(Xin ∗ W3×3 ))) (1)
B. DCOCO BLOCK
The proposed DCOCO module has two paths shown in
FIGURE 4. Proposed RMDNet architecture.
Figure 5. Let Xin be the input to the DCOCO block
derived from the stage 2 output Xstage2, as indicated in
adjustment is made to effectively decrease the total number of equation (6). The input xin is processed using depthwise
parameters within the module, as described in the equation 5 convolution, and the output of path 1 is given in equation (9).
Depthwise convolution encourages feature separability by
Xsec4 = ϑ(BN(ϕ(Xstage1 ∗ W3×3 ))) (5) learning distinct features in each input channel. This can be
particularly helpful when dealing with diverse and complex
The output of these two sections is concatenated together,
feature representations in multi-channel data like RGB
represented in equation 6.
images, where different channels represent different color
Xstage2 = Xsec3 Xsec4 (6) information. When adapting pre-trained models to new tasks
or datasets, depthwise convolution can be advantageous. Due
In Conv2D, each filter is applied to all input channels, to its parameter efficiency, depthwise convolution allows for
resulting in a large number of operations. However, in depth- faster fine-tuning and adaptation to new data, reducing the
wise convolution, each channel is convolved separately, risk of overfitting when the target dataset is small.
significantly reducing the number of operations. This makes
depthwise convolution computationally more efficient than Outpath1 = ϑ(BN(ϕ(Xin ∗ W3×3 depthwise ))) (9)
Conv2D. Depthwise convolution requires fewer parameters
compared to Conv2D. There are a lot of learnable parameters In the second path, input Xin is processed through two
in Conv2D since each filter has the same amount of param- depth-wise convolution layers, Let X2conv be the output given
eters as the input channels. A single filter is used for each in (10). Applying depthwise convolution twice allows the
2 ∗ Re ∗ Pr
F1 = (16)
Re + Pr
where Re = Recall and Pr = Precision.
FLOPs measure the computational capability of a com-
puting entity, while training and testing time represent the
time required to train a DL architecture. Trainable parameters
represent the bulkiness of the architecture. True Positive
Rate (TPR) measures the proportion of positive instances
correctly classified as positive by a model, presented in
equation (17). While False Positive Rate (FPR) quantifies the
ratio of negative instances incorrectly classified as positive
given in equation (18).
TP
TPR = (17)
FN + TP
FIGURE 7. Malimg malware samples (dataset 2). FP
FPR = (18)
TN + FP
C. TRAINING SETUP
This section gives a detailed overview of the training setup
used for conducting experiments of on the proposed RMDNet
model and benchmark models. The training process was
performed on Kaggle’s cloud based environment with a
preconfigured Linux distribution, utilizing the GPU Kernel-
Tesla P100. The NVIDIA Tesla P100 is a high-performance
GPU with 16 GB High Bandwidth Memory (HBM-2), based
on NVIDIA Pascal architecture having 3584 CUDA cores.
The model is implemented using the Keras API framework
and Tensorflow 2.11.0. The proposed and benchmark models
utilized in this study are trained with a batch size of 32 for
40 epochs on the binary dataset, 100 epochs on the mailing
FIGURE 8. Dumpware 10 malware samples (dataset 3). dataset, and 100 epochs on the dumpware 10 dataset. It uses
the Adam optimizer with early stopping. The best of these
model’s performance using four computational complexity values are reported below after all of these models were
metrics and computes the total number of trainable parame- trained five times on each of the three datasets. No data
ters. The confusion metrics predict True Positive (TP), False augmentation was performed on any of these datasets.
Positive (FP), True Negative (TN), and False Negatives (FN).
Accuracy is a metric used to assess a classification model’s D. ABLATION STUDY
correctness, given in equation (13). With precision, indicating Ablation study is the systematic analysis of the impact
the proportion of accurately predicted positive observations of eliminating or changing certain components, features,
relative to the total number as shown in equation (14). Recall or parameters inside a model to understand their individual
quantifies the percentage of correctly predicted positive contributions to the model’s performance. The purpose is
outputs, evaluating the architecture’s ability to acquire to determine the importance of each component in overall
all positive outputs without missing any represented in operation of the RMDNet.
equation (15). F1-score calculates the ratio of accurately 1) Intermediate stage 3: In this abliation study elimanating
predicted positive observations to the actual number of the stage 1 and stage 2 of the RMDNet the model
positive observations in the class represented in equation (16). performance was studies on all the three datasets and
TNC + TPC the number of parameters were high.
Acc = (13) 2) Intermediate stage 2 and 3: In this abliation study
FPC + FNC + TPC + TNC
only the stage 1 of the RMDNet was elimated to
where TPC = True Positive Calculated, TNC = True
observe the model performance on all the three
Negative Calculated, FPC = False Positive Calculated and
datasets, the number of parameters were reduced but
FNC = False Negative Calculated.
their was no significant improvements were found in
TPC the performance metrics. The stage 1 of the proposed
Pr = (14)
FPC + TPC model initially perform the Conv2D operation. From
TP the results, it is seen that eliminating the Conv2D
Re = (15)
FN + TP operation reduces the accuracy and other performance
VOLUME 12, 2024 82629
S. Puneeth et al.: RMDNet-DL Paradigms for Effective Malware Detection and Classification
TABLE 5. Performance metrics on binary dataset. TABLE 7. Performance metrics on dumpware 10 dataset.
TABLE 6. Performance metrics on malimg dataset. it numerically. Regardless of the type of model employed,
be it a statistical model or a neural network approach such
as DNN or CNN, an appropriate metric is indispensable
for evaluating performance. Numerous evaluation metrics
exist for DL problems, and this discussion will explore
some of the popular ones and delve into the insights
they provide regarding model performance. knowing models
perceive the data is crucial for gaining valuable insights
metrics of the model. To capture the spatial hierarchies into its strengths, weaknesses, and overall effectiveness.
and local patterns efficiently the Conv2D opertaion By examining these metrics, a deeper understanding of
must be performed in the initial layers over the the model’s behavior and decisions about its optimization
seperable convolution method. The table 2 gives the and potential enhancements can be determined. Using the
ablation study on binary dataset, table 3 and 4 gives evaluation metrics, the performance of the proposed RMDNet
the ablation results of the malimg and dumpware model is compared to ResNeXt [39], VGG 19 [34], LiverNET
10 datasets respectively. [38], EfficientNet-B0 [39], and DenseNET 121 [40]. After
training and testing on all the mentioned algorithms for
V. RESULTS AND DISCUSSIONS all three datasets obtained accuracy, precision, recall, and
In this section, we present the results of our comprehensive f1-score are listed in Table 5 for binary dataset, Table 6 for
study. Our study aimed to develop a robust malware detec- malimg dataset, and Table 7 for dumpware 10 dataset.
tion & classification model on malware datasets. Compare Confusion matrix for binary, malimg, and dumpware
the results with the state-of-the-art models using different 10 datasets are shown in figures 9, 10, and 11 respectively.
performance metrics, as detailed in sections V-A and V-B. Table 8 and 9 presents the TPR and FPR corresponding
to each malware class for different DL algorithms trained on
A. COMPARISON WITH BENCHMARK MODELS the Malimg dataset and dumpware 10 datasets respectively.
Performance metrics play a crucial role and DL pipeline, Loss and accuracy are essential values to consider while
providing valuable insights into progress and quantifying training DL models. We can check whether our model
FIGURE 12. Learning curve of the binary dataset. Training and validation (a) accuracy graph
and (b) loss graph.
FIGURE 13. Learning curve of mailing dataset. Training and validation (a) accuracy graph
and (b) loss graph.
FIGURE 14. Learning curve of dumpware 10 dataset. Training and validation (a) accuracy
graph and (b) loss graph.
Figures 12, 13, and 14 show the learning curves of RMDNet model excelled overall in terms of quality metrics
RMDNet on binary, malimg, and dumpware 10 datasets irrespective of the malware datasets. This demonstrates that
respectively. Training, validation accuracy and loss of proposed RMDNet architecture is capable of identifying and
dataset 1 and 2 are similar compared to dataset 3. The model classifying image based malware in more accurately. The
is converging prior 15 epochs. second best result for the binary dataset was with VGG 19,
for the malimg dataset was with ResNeXt, and for dumpware
VI. CONCLUSION 10 was with DenseNET 121. By leveraging domain-specific
A novel RMDNet model was proposed for accurately iden- knowledge and experimenting with different architectural
tification of malware variants. The performance of several components, it was possible to design and develop a
existing DL models, including VGG19, ResNeXt, LiverNet, robust deep learning model which was able to effectively
EfficientNet B0, and DenseNet was compared with the captured the essential features and designs of malware
proposed RMDNet. After conducting a thorough evaluation images. The results obtained from proposed RMDNet model
and analysis, the results demonstrated that the presented highlighted the importance of exploring custom architectures
and leveraging domain expertise when dealing with complex [14] S. Jeon and J. Moon, ‘‘Malware-detection method with a convolutional
classification tasks, such as image-based malware detection. recurrent neural network using opcode sequences,’’ Inf. Sci., vol. 535,
pp. 1–15, Oct. 2020, doi: 10.1016/j.ins.2020.05.026.
While existing DL models provide strong baselines, tailoring [15] Y. Ding, W. Dai, S. Yan, and Y. Zhang, ‘‘Control flow-based opcode
the architecture to the specific task can lead to significant behavior analysis for malware detection,’’ Comput. Secur., vol. 44,
performance improvements. It was worth noting that the self- pp. 65–74, Jul. 2014, doi: 10.1016/j.cose.2014.04.003.
[16] Y. Ki, E. Kim, and H. K. Kim, ‘‘A novel approach to detect malware based
created model’s success does not diminish the significance on API call sequence analysis,’’ Int. J. Distrib. Sensor Netw., vol. 11, no. 6,
of the existing DL models. These models have undergone Jun. 2015, Art. no. 659101, doi: 10.1155/2015/659101.
considerable study and validation in a variety of domains, [17] C.-Y. Wang, C.-Y. You, F.-H. Hsu, C.-H. Lee, C.-H. Liu, and Y. Zhuang,
‘‘SMS observer: A dynamic mechanism to analyze the behavior of
making them useful tools for categorization tasks. However, SMS-based malware,’’ J. Parallel Distrib. Comput., vol. 156, pp. 25–37,
in order to obtain the optimum performance, the special Oct. 2021, doi: 10.1016/j.jpdc.2021.05.004.
needs of image-based malware detection demanded the [18] M. Tang and Q. Qian, ‘‘Dynamic API call sequence visualisation for
development of a customized DL architecture. This RMDNet malware classification,’’ IET Inf. Secur., vol. 13, no. 4, pp. 367–377,
Jul. 2019, doi: 10.1049/iet-ifs.2018.5268.
shows the effectiveness of a self-designed DL architecture [19] E. Amer and I. Zelinka, ‘‘A dynamic windows malware detection
in binary and multi-class classification of image-based and prediction method based on contextual understanding of API call
malware. The superior performance of the model highlighted sequence,’’ Comput. Secur., vol. 92, May 2020, Art. no. 101760, doi:
10.1016/j.cose.2020.101760.
the importance of tailoring the architecture to the specific task [20] J. Ragaventhiran, P. Vigneshwaran, M K. Mallikarjun, S T. Ahmed,
at hand. The findings of this work contribute to the ongoing R. Prabu, and P. Megantoro, ‘‘An unsupervised malware detection system
research and development of more accurate and efficient for windows based system call sequences,’’ Malaysian J. Comput. Sci.,
pp. 79–92, 2022, doi: 10.22452/mjcs.sp2022no2.7.
methods for malware recognition and categorization using [21] J. Homer, A. Varikuti, X. Ou, and M. A. McQueen, ‘‘Improving attack
DL techniques. graph visualization through data reduction and attack grouping,’’ 2008, doi:
10.1007/978-3-540-85933-8_7.
REFERENCES [22] L. Nataraj, S. Karthikeyan, G. Jacob, and B. S. Manjunath, ‘‘Malware
images,’’ in Proc. 8th Int. Symp. Visualizat. Cyber Secur., Jul. 2011, doi:
[1] D. B. Rawat, R. Doku, and M. Garuba, ‘‘Cybersecurity in big data 10.1145/2016904.2016908.
era: From securing big data to data-driven security,’’ IEEE Trans. [23] J. Fu, J. Xue, Y. Wang, Z. Liu, and C. Shan, ‘‘Malware visualization for
Services Comput., vol. 14, no. 6, pp. 2055–2072, Nov. 2021, doi: fine-grained classification,’’ IEEE Access, vol. 6, pp. 14510–14523, 2018,
10.1109/TSC.2019.2907247. doi: 10.1109/ACCESS.2018.2805301.
[2] Y. Lu and L. D. Xu, ‘‘Internet of Things (IoT) cybersecurity research: A
[24] I. Baptista, ‘‘Binary visualisation for malware detection,’’ Plymouth
review of current research topics,’’ IEEE Internet Things J., vol. 6, no. 2,
Student Scientist, vol. 11, no. 1, pp. 223–237, 2018.
pp. 2103–2115, Apr. 2019, doi: 10.1109/JIOT.2018.2869847.
[25] K. S. Han, J. H. Lim, B. Kang, and E. G. Im, ‘‘Malware analysis using
[3] M. Roopak, G. Yun Tian, and J. Chambers, ‘‘Deep learning models
visualized images and entropy graphs,’’ Int. J. Inf. Secur., vol. 14, no. 1,
for cyber security in IoT networks,’’ in Proc. IEEE 9th Annu. Comput.
pp. 1–14, Feb. 2015, doi: 10.1007/s10207-014-0242-0.
Commun. Workshop Conf. (CCWC), Jan. 2019, pp. 0452–0457, doi:
[26] S. O’Shaughnessy and S. Sheridan, ‘‘Image-based malware classification
10.1109/CCWC.2019.8666588.
hybrid framework based on space-filling curves,’’ Comput. Secur., vol. 116,
[4] A. Souri and R. Hosseini, ‘‘A state-of-the-art survey of malware detection
May 2022, Art. no. 102660, doi: 10.1016/j.cose.2022.102660.
approaches using data mining techniques,’’ Human-Centric Comput. Inf.
Sci., vol. 8, no. 1, Dec. 2018, doi: 10.1186/s13673-018-0125-x. [27] B. Yuan, J. Wang, D. Liu, W. Guo, P. Wu, and X. Bao, ‘‘Byte-level malware
[5] S. Kumar, P. Tiwari, and M. Zymbler, ‘‘Internet of Things is a revolutionary classification based on Markov images and deep learning,’’ Comput. Secur.,
approach for future technology enhancement: A review,’’ J. Big Data, vol. 92, May 2020, Art. no. 101740, doi: 10.1016/j.cose.2020.101740.
vol. 6, no. 1, Dec. 2019, doi: 10.1186/s40537-019-0268-2. [28] S. A. Roseline, S. Geetha, S. Kadry, and Y. Nam, ‘‘Intelligent vision-
[6] Y. Li and Q. Liu, ‘‘A comprehensive review study of cyber-attacks and based malware detection and classification using deep random for-
cyber security; emerging trends and recent developments,’’ Energy Rep., est paradigm,’’ IEEE Access, vol. 8, pp. 206303–206324, 2020, doi:
vol. 7, pp. 8176–8186, Nov. 2021, doi: 10.1016/j.egyr.2021.08.126. 10.1109/ACCESS.2020.3036491.
[7] D. Ucci, L. Aniello, and R. Baldoni, ‘‘Survey of machine learning [29] W.-C. Lin and Y.-R. Yeh, ‘‘Efficient malware classification by binary
techniques for malware analysis,’’ Comput. Secur., vol. 81, pp. 123–147, sequences with one-dimensional convolutional neural networks,’’ Math-
Mar. 2019, doi: 10.1016/j.cose.2018.11.001. ematics, vol. 10, no. 4, p. 608, Feb. 2022, doi: 10.3390/math10040608.
[8] Y. Ye, L. Chen, S. Hou, W. Hardy, and X. Li, ‘‘DeepAM: A heterogeneous [30] R. Vinayakumar, M. Alazab, K. P. Soman, P. Poornachandran,
deep learning framework for intelligent malware detection,’’ Knowl. Inf. and S. Venkatraman, ‘‘Robust intelligent malware detection using
Syst., vol. 54, no. 2, pp. 265–285, Feb. 2018, doi: 10.1007/s10115-017- deep learning,’’ IEEE Access, vol. 7, pp. 46717–46738, 2019, doi:
1058-9. 10.1109/ACCESS.2019.2906934.
[9] K. Shaukat, S. Luo, and V. Varadharajan, ‘‘A novel deep learning-based [31] J. Kim, A. Sim, J. Kim, K. Wu, and J. Hahm, ‘‘Transfer learning
approach for malware detection,’’ Eng. Appl. Artif. Intell., vol. 122, approach for botnet detection based on recurrent variational autoencoder,’’
Jun. 2023, Art. no. 106030, doi: 10.1016/j.engappai.2023.106030. in Proc. 3rd Int. Workshop Syst. Netw. Telemetry Analytics, Jun. 2020, doi:
[10] M. S. Akhtar and T. Feng, ‘‘Detection of malware by deep learning as 10.1145/3391812.3396273.
CNN-LSTM machine learning techniques in real time,’’ Symmetry, vol. 14, [32] Ö. Aslan and A. A. Yilmaz, ‘‘A new malware classification frame-
no. 11, p. 2308, Nov. 2022, doi: 10.3390/sym14112308. work based on deep learning algorithms,’’ IEEE Access, vol. 9,
[11] M. E. Ahmed, S. Nepal, and H. Kim, ‘‘MEDUSA: Malware detection using pp. 87936–87951, 2021, doi: 10.1109/ACCESS.2021.3089586.
statistical analysis of system’s behavior,’’ in Proc. IEEE 4th Int. Conf. [33] A. Mallik, A. Khetarpal, and S. Kumar, ‘‘ConRec: Malware classification
Collaboration Internet Comput. (CIC), Nepal, Oct. 2018, pp. 272–278, using convolutional recurrence,’’ J. Comput. Virol. Hacking Techn., vol. 18,
doi: 10.1109/CIC.2018.00044. no. 4, pp. 297–313, Feb. 2022, doi: 10.1007/s11416-022-00416-3.
[12] M. Alazab, S. Venkataraman, and P. Watters, ‘‘Towards understanding [34] M. J. Awan, O. A. Masood, M. A. Mohammed, A. Yasin, A. M.
malware behaviour by the extraction of API calls,’’ in Proc. 2nd Zain, R. Damaševičius, and K. H. Abdulkareem, ‘‘Image-based malware
Cybercrime Trustworthy Comput. Workshop, Jul. 2010, pp. 52–59, doi: classification using VGG19 network and spatial convolutional attention,’’
10.1109/CTC.2010.8. Electronics, vol. 10, no. 19, p. 2444, Oct. 2021, doi: 10.3390/electron-
[13] A. Shabtai, R. Moskovitch, C. Feher, S. Dolev, and Y. Elovici, ‘‘Detecting ics10192444.
unknown malicious code by applying classification techniques on OpCode [35] M. Xiao, C. Guo, G. Shen, Y. Cui, and C. Jiang, ‘‘Image-based malware
patterns,’’ Secur. Informat., vol. 1, no. 1, Dec. 2012, doi: 10.1186/2190- classification using section distribution information,’’ Comput. Secur.,
8532-1-1. vol. 110, Nov. 2021, Art. no. 102420, doi: 10.1016/j.cose.2021.102420.
[36] B. Saridou, J. R. Rose, S. Shiaeles, and B. Papadopoulos, ‘‘SAGMAD—A SHYAM LAL (Senior Member, IEEE) received
signature agnostic malware detection system based on binary visualisation the M.Tech. degree in electronics and communi-
and fuzzy sets,’’ Electronics, vol. 11, no. 7, p. 1044, Mar. 2022, doi: cation engineering from the National Institute of
10.3390/electronics11071044. Technology, Kurukshetra, Haryana, India, in 2007,
[37] D. Vasan, M. Alazab, S. Wassan, B. Safaei, and Q. Zheng, ‘‘Image- and the Ph.D. degree in image processing from
based malware classification using ensemble of CNN architectures the Birla Institute of Technology, Mesra, Ranchi,
(IMCEC),’’ Comput. Secur., vol. 92, May 2020, Art. no. 101748, doi: India, in 2013. He has been an Associate Pro-
10.1016/j.cose.2020.101748.
fessor with the Department of Electronics and
[38] W. K. Wong, F. H. Juwono, and C. Apriono, ‘‘Vision-based malware
Communication Engineering, National Institute
detection: A transfer learning approach using optimal ECOC-SVM
configuration,’’ IEEE Access, vol. 9, pp. 159262–159270, 2021, doi:
of Technology Karnataka (NITK) at Surathkal,
10.1109/ACCESS.2021.3131713. Surathkal, India. He has published around 100 research papers in reputed
[39] J. H. Go, T. Jan, M. Mohanty, O. P. Patel, D. Puthal, and M. Prasad, journals and conferences. His research interests include machine learning,
‘‘Visualization approach for malware classification with ResNeXt,’’ in deep learning, cyber security, digital image processing, satellite remote
Proc. IEEE Congr. Evol. Comput. (CEC), Jul. 2020, pp. 1–7, doi: sensing, and medical image processing.
10.1109/CEC48606.2020.9185490.
[40] A. A. Aatresh, K. Alabhya, S. Lal, J. Kini, and P. P. Saxena, ‘‘LiverNet:
Efficient and robust deep learning model for automatic diagnosis of sub-
types of liver hepatocellular carcinoma cancer from H&E stained liver MAHENDRA PRATAP SINGH (Member, IEEE)
histopathology images,’’ Int. J. Comput. Assist. Radiol. Surg., vol. 16, no. 9, received the Ph.D. degree from the Depart-
pp. 1549–1563, Sep. 2021, doi: 10.1007/s11548-021-02410-4. ment of Computer Science and Engineering,
[41] R. Chaganti, V. Ravi, and T. D. Pham, ‘‘Image-based malware repre- Indian Institute of Technology Kharagpur (IIT
sentation approach with EfficientNet convolutional neural networks for Kharagpur). He is currently an Assistant Professor
effective malware classification,’’ J. Inf. Secur. Appl., vol. 69, Sep. 2022,
with the Department of Computer Science and
Art. no. 103306, doi: 10.1016/j.jisa.2022.103306.
Engineering, National Institute of Technology
[42] J. C. Kimmell, M. Abdelsalam, and M. Gupta, ‘‘Analyzing machine
Karnataka (NITK) at Surathkal, Surathkal, India.
learning approaches for online malware detection in cloud,’’ in Proc. IEEE
Int. Conf. Smart Comput. (SMARTCOMP), Aug. 2021, pp. 189–196, doi: He has published more than 21 research papers
10.1109/SMARTCOMP52413.2021.00046. in reputed international journals and conferences.
His research interests include network security, information security, and
privacy.