IEEE - 61001

Automated Malware Detection Using Memory

2024 15th International Conference on Computing Communication and Networking Technologies (ICCCNT) | 979-8-3503-7024-9/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICCCNT61001.2024.10725096

Forensics
Sarath Jayan Nair Sreelakshmi R Syam
Center for Cybersecurity Systems and Networks Center for Cybersecurity Systems and Networks
Amrita Vishwa Vidyapeetham Amrita Vishwa Vidyapeetham
Amritapuri, India Amritapuri, India
[email protected] [email protected]

Abstract—Malware is getting advanced and uses different memory images for any malicious activity could streamline
techniques to be hidden. Some malware reveal their harmful the process by sifting through different artifacts provided
nature only upon execution. Detection of such advanced malware by the memory image. Additionally, this aids in the better
poses challenges. Memory forensics plays a crucial role in
identifying such malware, as it involves the real-time recording understanding of the suspicious behaviours in the system by a
of system activity and inspection of vital artifacts like active forensic investigator who might not be a specialist in malware
processes, network activity, and DLLs loaded by programs. This analysis. However, a malware analyst’s analysis is required for
study proposes a Python-based command-line program, utilizing a thorough examination of the memory image.
Volatility3 for extracting and analyzing memory artifacts to Former research works such as [5] used Cuckoo sandbox
identify suspicious processes. It examines all running processes,
regardless of their network activity, and checks the origins of to execute the malware and collect the memory dump. The
these processes. Suspicious processes are then cross-verified with details extracted from the dump using Volatility are used to
VirusTotal to assess their malicious nature. The final outputm train and test a Recurrent Neural Network (RNN) model.
of the tool is a comprehensive report detailing the findings from Another work by A.D Tran et al. [6] proposes a classification
VirusTotal, providing a streamlined and efficient approach to tool with a dataset of 2750 samples consisting of 8 different
malware detection in memory dumps. This work aims to en-
hance the capabilities of forensic investigators, particularly those malware types. The memory dumps are converted into images
not specialized in malware analysis, in rapidly and accurately using “bin2png” tool. The machine learning model fetches
identifying potential threats within a system’s memory. features like HOG and GIST from the memory dump. The
Index Terms—Malware, Memory forensics, Volatility3, Virus- training and testing are performed using multiple algorithms
Total namely Decision Tree, Random Forest, and Support Vector
Machine. The main limitation in these two works is the limited
I. I NTRODUCTION set of samples considered for building the model. In the
A malicious software that interferes with the operation of a work done by P. H Rughani [7], the proposed automated tool
computer system by collecting private or sensitive information uses Volatility to extract the process details that make open
from the system or disrupting the system’s operation could connections. These processes are extracted and cross-checked
be considered malware. Malware is classified into different with VirusTotal for their maliciousness. The main limitation
categories such as ransomware, trojan, virus, worm, spyware, of this approach is that it considers only processes that use
keyloggers, etc based on its functioning [1]. The recent ad- network connections.
vancements in the malware show several evasion mechanisms The goal of this work is to automate the process of detecting
incorporated to go undetected by the antivirus software. Some malware from memory dumps using a command line program,
unveil the malicious behaviour only when they are executed. malware scanner, based on Python. Using Volatility3, the util-
Fileless malware has become popular recently due to its ability ity will extract and analyze the memory artifacts and identify
to reside within the volatile memory and leave no trace on the any running suspicious processes on the system. The analysis
disks [2][3]. It will be challenging to detect such malware is not limited to those processes that have network connections
using traditional detection mechanisms. Memory forensics, but also checks each process and verifies the location from
which can record system activity in real-time, can be particu- which the processes are loaded. The identified suspicious
larly useful in identifying malware since all processes must processes will be extracted and then sent to VirusTotal [8]
be put into volatile memory before they can be executed. for analysis of their level of maliciousness. Finally, the tool
Details like active processes, network activity, DLLs loaded by will provide a report with the final outcomes from VirusTotal.
individual programs, registry activity, etc. may all be inspected
with it. These are valuable information that can be used to II. R ELATED W ORK
determine the presence of malware [4]. A manual analysis Malware has evolved significantly in recent years, and
of the memory artifacts can be time-consuming and leads their evasion strategies have also evolved. Consequently, a
to a delayed response. An automated mechanism to analyze substantial amount of research is done on malware detection

15th ICCCNT IEEE Conference,

Authorized licensed use limited to: Amrita School Of Engineering - Kollam.June 24-28, 2024,
Downloaded on November 15,2024 at 09:40:55 UTC from IEEE Xplore. Restrictions apply.
IIT - Mandi, Kamand, India
 IEEE - 61001

techniques. Memory forensics can be very helpful in discover- licious activities that evade conventional security measures.
ing malicious applications that are being run because it tracks USIM uses userspace abstractions to monitor runtime behavior
actions within a system in real-time. An automated malware deviations. It consists of a collection agent, which gathers
detection system can speed up timely action and cut down on system data, and an appraiser, which evaluates this data against
the time needed for manual investigation. Most of the research predefined rules. USIM effectively identified threats in tests
works involve adopting machine learning or deep learning against standard implant methods and the Hidden Wasp Linux
techniques to automate malware detection [9][10]. In the work rootkit.
by A. H. Lashkari et al., [11] they propose a tool based The research in [3] utilizes Multiple Memory Images (MMI)
on Volatility that extracts 36 features from memory dumps, analysis to detect fileless malware by capturing and analyzing
parses them, and writes them to a CSV file. These extracted memory images at different timestamps, addressing malware
features are then used to generate machine learning models evasion tactics. Various fileless malware samples, including
using algorithms such as AdaBoost, decision tree, KNN, and Gargoyle variants and APT 34, are injected using different
random forest for the detection of malware. The study is techniques. Memory dumps from VMs running Windows XP,
performed using 19 memory dumps and noises are added to 7, and 10 are analyzed with a Python script and Volatility
the data to prevent the model from over-fitting. The study plugins to detect anomalies and compare processes across
by S Lyles et al. [12] characterizes the memory behaviour timestamps. The MMI method achieves 100% accuracy, sig-
of a Windows machine and incorporates machine learning nificantly outperforming the traditional single image analysis
methods to detect malware. Collects 165 memory samples of method Ptenum, which has 50% accuracy and higher false
benign applications running and 200 samples with malware positives. However, MMI requires more time and storage due
performing DLL injection, shell code injection, PE injection to the analysis of multiple memory images.
and process hollowing are executed. Details are collected from An automated tool was proposed in [7] named ForMaLity.
the EPROCESS, ETHREAD, process environment block, The tool is aimed at aiding forensic investigators in conduct-
and virtual address descriptor tree structures. An ML model ing preliminary analysis for malware detection. Utilizing the
using a random forest algorithm is being used to detect Volatility framework and VirusTotal for its analysis streamlines
malware. The drawback of studies using machine learning or the detection process by requiring only the path to the mem-
deep learning is the limited availability of memory-specific ory dump file. Nonetheless, the tool has certain limitations,
datasets, hence the existing works involve fewer data samples including its focus only on processes with external communi-
for modeling. cations for detection purposes. Consequently, it is imperative
The study by Vurdelja et al., [13] provides a thorough to explore different automated mechanisms to detect malware
examination of automated malware analysis methodologies, with comprehensive memory analysis methods performed to
concentrating on containerization and type 2 hypervisors identify malware.
as two sandboxing strategies. Compared to containerization, Although previous studies have shown great progress in
which shares the host operating system among all containers automated malware detection through memory forensics, a
and thus increases the attack surface, type 2 hypervisors are number of constraints still exist. Previous research frequently
recommended because of their separate operating systems, uses out-of-date tools and approaches, concentrates exclusively
which limit the impact of vulnerabilities to particular VMs. on processes with network connections, or only look at a
The study also covers the different ways that malware evades small number of samples. Furthermore, the scope of machine
detection, including anti-VM and anti-sandbox strategies that learning and deep learning models has been limited by the
identify virtual environments and sandbox presence, respec- lack of publicly available memory-specific datasets.
tively. Also emphasizes the importance of adequate system This paper introduces an innovative approach by automating
resources. This can be useful when conducting research that malware detection from memory dumps using a Python-
includes the analysis and execution of malware. based command-line program. Leveraging Volatility3, this
The study depicted in [2] surveys memory acquisition and utility conducts a comprehensive analysis of memory arti-
analysis tools for malware detection, categorizing them using facts, extending beyond processes with network connections
a taxonomy by Latzo et al. based on criteria like user level, to include a thorough check of each process and its origin.
kernel level, and hypervisor level. It discusses tools such By integrating with VirusTotal, the tool not only identifies
as WinPmem, LiME, and VMWare, and examines system suspicious processes but also provides a detailed report on
management-level tools like SmmBackdoor. In memory analy- their maliciousness. This approach enhances the robustness
sis, it highlights the widespread use of Volatility and traditional and accuracy of malware detection, offering a more holistic
forensic methods, including signature-based and sandbox dy- solution to the evolving challenges in cybersecurity.
namic analysis. The research also explores machine learning
approaches, noting the potential of feature engineering and III. M ETHODOLOGY
computer vision techniques despite challenges like the lack of The overall architecture of this work is depicted in figure
publicly available memory dump images. 1. The victim machine is the system of which the memory is
In the research [14], authors introduce USIM (Userspace analyzed. Getting the memory image is the first step. Further,
Integrity Measurement), a tool to detect memory-based ma- the digital artifacts of the memory image are extracted and

15th ICCCNT IEEE Conference,

Authorized licensed use limited to: Amrita School Of Engineering - Kollam.June 24-28, 2024,
Downloaded on November 15,2024 at 09:40:55 UTC from IEEE Xplore. Restrictions apply.
IIT - Mandi, Kamand, India
 IEEE - 61001

examined. Following examination, the skeptical processes are

extracted from the dump and verified using the VirusTotal
website. A Python script automates each of these operations.
The tool is designed to be user-friendly, and the user need not
remember any commands of volatility to examine the dump.
To begin the analysis, it requires only the path to the extracted
memory image as a command line argument when executing
the Python file. There is no requirement for user interaction Fig. 2. Virtual machine ID
once the execution has started.

Fig. 3. Memory dump generation

Fig. 1. System Architecture of the Proposed Tool

in this study to do memory forensics on the obtained dump

A. Malware Execution Environment [15]. Volatility offers a number of plugins that can be used to
To execute the malware, this work sets up a Windows 10 examine the dump and retrieve various digital artifacts. This
Virtual Machine (VM) using Oracle VirtualBox. The malware tool makes use of different plugins, including windows.info,
samples are downloaded from the MalwareBazaar website. windows.cmdline, windows.psscan, and windows.netscan.
Sophisticated and novel malware can conceal its malevolent
actions by identifying a monitored environment. The virtual
machine’s guest additions are not set up. Since certain malware
can identify the system as a virtual machine with guest
additions, this is done to offer a context as similar to the actual
system as possible. In addition, the machine is allotted 4GB
RAM, 70GB storage and 4 CPU cores. This has been done
since some malware recognizes a virtual environment based
on the system resources assigned [13]. To make it easier to
return to a clean Windows before running another piece of
malware, a snapshot of the virtual machine is created at the
beginning before any malware is executed. Fig. 4. Software Architecture of the Proposed Tool

B. Memory Acquisition Figure 4 shows the software architecture or code flow of

Memory acquisition involves capturing the data from the the proposed tool. The execution begins with deducing high-
volatile memory of a system. Since an Oracle VirtualBox level information about the memory sample being examined
virtual machine is used in this work as the victim machine, using “windows.info” plugin. This includes information on the
the memory dump is obtained using VBoxManage, a utility operating system, architecture, profile, time of sample collec-
built-in to VirtualBox. The command displayed in figure 2 tion, KDBG address, and other parameters. When evaluating
can be used to list all of the available virtual machines’ IDs. a sample using Volatility2, this is an essential step because
As shown in figure 3, the memory image can be obtained by the profile needs to be specified in the instructions. Since the
utilizing the ID, the debugvm, and dumpcore options. proposed tool utilizes Volatility3, the profile of the memory
sample need not be mentioned.
C. Memory Analysis Using the “windows.cmdline” plugin, information about the
Memory analysis involves inspecting the obtained memory processes is first collected for the malware investigation. It
dump which is carried out in this work on an Ubuntu ma- displays both the path to the location from which a specific
chine. This analysis can be performed using various open- process is started and the command-line arguments. This
source tools such as Volatility, Rekall, LibVMI, etc. Volatility information can be very useful as certain default processes
is among the most popular and advanced memory forensic of the Windows OS will be launched from locations such
frameworks available. Volatility is the foundation of Rekall as “C:\windows\system32”, “C:\windows\systemapps”, “C:\
but is no longer maintained. Volatility3 release v2.5.0 is used Program Files (x86)\”, etc. Any suspect locations detected in

15th ICCCNT IEEE Conference,

Authorized licensed use limited to: Amrita School Of Engineering - Kollam.June 24-28, 2024,
Downloaded on November 15,2024 at 09:40:55 UTC from IEEE Xplore. Restrictions apply.
IIT - Mandi, Kamand, India
 IEEE - 61001

the location from which the processes are loaded will have The tool displays the output at each stage of the execution
their respective process IDs transferred to a suspicious process which is depicted in the architectural diagram of the tool in
list. figure 4. The output of the “windows.info” plugin that gives
Details about the processes that were active when the mem- details about the memory dump is shown in figure 6.
ory dump was collected are provided by the “windows.psscan”
plugin. Additionally, it provides information about hidden or
unlinked processes, as well as previously terminated ones.
In case of terminated processes, the exit time will also be
mentioned. The plugin’s output is compared to the default
Windows system process names that were retrieved from a
previously acquired, non-malicious memory snapshot. This is
another method incorporated to identify suspicious processes.
In addition to the local IP address and any foreign IP
addresses that are collected, the “windows.netscan” plugin
offers details on the network connections that processes have
made. Given that many malware programs involve network
connectivity and connections to external systems, such as Fig. 6. Memory Image Details
command and control servers, this information can be signifi-
The outputs during the execution of “windows.cmdline”,
cantly helpful. There is a chance of the same process showing
“windows.psscan”, and “windows.netscan” will be saved as
multiple instances in the output with a connection to the same
CSV files. This can be useful if further analysis is required.
IP address. The tool will remove the duplicate ones before
The suspicious processes will be identified by analyzing the
identifying the suspicious process IDs.
outputs from each plugin and the corresponding process IDs
Further, all the identified suspicious process executables are will be saved into a suspicious process list.
dumped using the “windows.dumpfiles” plugin. All the DLLs
and associated files specific to the processes are dumped along
with the executable image. These files can be used for detailed
analysis later if required.
The dumped executable image files will be scanned for
malware with the help of VirusTotal. This web-based platform
aggregates malware scanners from several security vendors.
It supports scanning of files, IP addresses and URLs. The
VirusTotal API integrated with the tool programmatically
interacts with the platform. The extracted files from the dump
are submitted to the platform through the API and retrieves
the results after successful scanning.

IV. R ESULTS
The proposed tool for automating the malware detection
process requires only the path to the memory dump file as Fig. 7. Result from VirusTotal
input from the user. It has to be given as a command line
argument during the execution of the tool. Figure 5 shows
the execution of the tool. The tool will generate an output
directory with the timestamp at the analysis is performed. All
the output files including the final report generated and the
files extracted will be saved within this output directory.

Fig. 8. Scan Summary

Once the suspicious processes have been found, these pro-

Fig. 5. Execution of the Proposed Tool cesses are extracted using “windows.dumpfiles” plugin. The

15th ICCCNT IEEE Conference,

Authorized licensed use limited to: Amrita School Of Engineering - Kollam.June 24-28, 2024,
Downloaded on November 15,2024 at 09:40:55 UTC from IEEE Xplore. Restrictions apply.
IIT - Mandi, Kamand, India
 IEEE - 61001

extracted files associated with each of the processes will be [7] P. H. Rughani, “Formality: Automated forensic malware analysis using
saved into separate directories with the particular process ID volatility.,” International Journal of Advanced Research in Computer
Science, vol. 8, no. 3, 2017.
as the directory name. Subsequently, the executable images of [8] VirusTotal, “Subsidiary of google.” https://www.virustotal.com/.
the extracted suspicious processes are uploaded to VirusTotal [9] B. A. V. Vidyapeetham, “Api call based malware detection approach
using its API and retrieve the scan results. The results of a using recurrent neural network—lstm,” in Intelligent Systems Design
and Applications: 18th International Conference on Intelligent Systems
process executable submitted to VirusTotal are shown in figure Design and Applications (ISDA 2018) held in Vellore, India, December
7. For files that have been identified as malicious, details such 6-8, 2018, Volume 1, vol. 940, p. 87, Springer, 2019.
as the security vendor name, the version of the engine used [10] K. M. Balasubramanian, S. V. Vasudevan, S. K. Thangavel, G. Kumar,
K. Srinivasan, A. Tibrewal, and S. Vajipayajula, “Obfuscated malware
to check the file, the malware variant identified by the vendor, detection using machine learning models,” in 2023 14th International
and the date on which the engine was updated. Conference on Computing Communication and Networking Technologies
After the completion of a successful analysis, the summary (ICCCNT), pp. 1–8, IEEE, 2023.
[11] A. H. Lashkari, B. Li, T. L. Carrier, and G. Kaur, “Volmemlyzer: Volatile
of the scanning will also be generated as shown in figure 8. memory analyzer for malware classification using feature engineering,”
All these outputs and the summarized scanning results will be in 2021 Reconciling Data Analytics, Automation, Privacy, and Security:
saved into the final report generated. A Big Data Challenge (RDAAPS), pp. 1–8, IEEE, 2021.
[12] S. Lyles, M. Desantis, J. Donaldson, M. Gallegos, H. Nyholm, C. Taylor,
and K. Monteith, “Machine learning analysis of memory images for
V. C ONCLUSION process characterization and malware detection,” in 2022 52nd Annual
IEEE/IFIP International Conference on Dependable Systems and Net-
This study proposes a Python-based tool to automate the works Workshops (DSN-W), pp. 162–169, IEEE, 2022.
malware detection mechanism in a memory dump. Owing to [13] I. Vurdelja, I. Blažić, D. Bojić, and D. Drašković, “A framework
the fact that the process has to be loaded into the RAM for its for automated dynamic malware analysis for linux,” in 2020 28th
Telecommunications Forum (TELFOR), pp. 1–4, IEEE, 2020.
execution, memory forensics can be considerably helpful in [14] J. A. Pendergrass, N. Hull, J. Clemens, S. C. Helble, M. Thober,
malware detection tasks. Popular and open-source advanced K. McGill, M. Gregory, and P. Loscocco, “Runtime detection of
memory forensics utility, Volatility3 is being used for the userspace implants,” in MILCOM 2019-2019 IEEE Military Commu-
nications Conference (MILCOM), pp. 1–6, IEEE, 2019.
purpose of extracting digital artifacts from the memory dump. [15] Volatilityfoundation, “Volatility3.” https://github.com/volatilityfoundat
Based on the analysis, certain processes are shortlisted to ion/volatility3.
be suspicious. They are extracted from the memory dump
and submitted to VirusTotal through the API integrated into
the tool. The results are retrieved and a final report will be
generated with the identified results. The tool is intended
to aid a forensic investigator who is not an expert malware
analyst in getting a preliminary idea about the maliciousness
of a system. Additionally, it can save the time required to
perform manual analysis which in turn helps in providing
timely response. The continuing advancements in this field
could focus on identifying advanced malware that employs
evasion techniques. More advanced memory analysis methods
can be integrated into the tool for advanced malware detection
mechanisms.

R EFERENCES
[1] S. Talukder, “Tools and techniques for malware detection and analysis,”
arXiv preprint arXiv:2002.06819, 2020.
[2] H. Nyholm, K. Monteith, S. Lyles, M. Gallegos, M. DeSantis, J. Don-
aldson, and C. Taylor, “The evolution of volatile memory forensics,”
Journal of Cybersecurity and Privacy, vol. 2, no. 3, pp. 556–572, 2022.
[3] S. M. Pottaraikkal and A. S. Sugatha, “Effectiveness of multiple
memory-images in detecting fileless malware,” in 2023 11th Interna-
tional Symposium on Digital Forensics and Security (ISDFS), pp. 1–5,
IEEE, 2023.
[4] M. Swapna and J. Ramkumar, “Multiple memory image instances
stratagem to detect fileless malware,” in International Conference on
Advancements in Smart Computing and Information Security, pp. 131–
140, Springer, 2023.
[5] M. Murthaja, B. Sahayanathan, A. Munasinghe, D. Uthayakumar, L. Ru-
pasinghe, and A. Senarathne, “An automated tool for memory foren-
sics,” in 2019 International Conference on Advancements in Computing
(ICAC), pp. 1–6, IEEE, 2019.
[6] A.-D. Tran, N.-H. Vo, Q.-K. Tran, H.-D. Nguyen, and M.-T. Tran,
“Os-independent malware detection: Applying machine learning and
computer vision in memory forensics,” in 2021 17th International
Conference on Computational Intelligence and Security (CIS), pp. 616–
620, IEEE, 2021.

15th ICCCNT IEEE Conference,

Authorized licensed use limited to: Amrita School Of Engineering - Kollam.June 24-28, 2024,
Downloaded on November 15,2024 at 09:40:55 UTC from IEEE Xplore. Restrictions apply.
IIT - Mandi, Kamand, India