Received December 13, 2020, accepted December 23, 2020, date of publication January 1, 2021, date of current version

January 13, 2021.

Digital Object Identifier 10.1109/ACCESS.2020.3048848

Bypassing Anti-Analysis of Commercial

Protector Methods Using DBI Tools
YOUNG BI LEE , JAE HYUK SUK , AND DONG HOON LEE , (Member, IEEE)
Graduate School of Information Security, Korea University, Seoul 02841, South Korea
Corresponding author: Dong Hoon Lee ([email protected])
This work was supported by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government Ministry of
Science and ICT (MSIT) under Grant NRF-2017R1A2B3009643.

ABSTRACT As most malware is infectious, anti-analysis and packing techniques supported by commercial
protectors are conventionally applied to hinder analysis. When analyzing to detect and block such protected
malware, it is necessary to do so in a virtual environment to prevent infection. In terms of packing,
it is necessary to analyze using dynamic binary instrumentation (DBI), a dynamic analysis tool, which is
advantageous for unpacking because DBI inserts code at run time and analyzes it dynamically. However,
malware terminates on its own when it detects a virtual environment or DBI due to anti-analysis techniques.
Therefore, it is necessary to also bypass anti-VM and anti-DBI techniques in order to successfully analyze
malware in a virtual environment using DBI. It is very difficult for analysts to bypass anti-VM and anti-DBI
techniques that are used in commercial protectors because analysts generally have little information on what
methods are used or how to even bypass these techniques. In this paper, we suggest guidelines to aid in easy
analysis of malware protected by anti-VM and anti-DBI techniques supported by commercial protectors.
We analyzed the techniques used by five of the most common commercial protectors, and herein present
how to bypass anti-VM and anti-DBI techniques supported by commercial protectors via a detailed algorithm
analysis. We performed a bypass experiment after applying each commercial protector to 1573 executable
files containing vulnerabilities provided by the National Institute of Standards and Technology (NIST).
To our knowledge, this is the first empirical study to suggest detailed bypassing algorithms for anti-VM
and anti-DBI techniques used in commercial protectors.

INDEX TERMS Obfuscation, commercial protectors, anti-analysis, anti-VM, anti-DBI, DBI tool.

I. INTRODUCTION dynamically analyze program behavior and make it possible

There is a steady increase in attacks utilizing malware to effectively assess obfuscation and packing techniques. Pre-
on computer systems. Most malware is distributed via vious studies using DBI tools have already suggested various
application of anti-analysis techniques and packing, a code methods for unpacking or effectively analyzing programs
obfuscation technique, while still supported by commercial protected by commercial protectors.
protectors, inhibiting antivirus programs from detecting the It is important to use a virtual environment when analyzing
presence of such malware. However, if anti-analysis tech- actual malware with certain commercial protectors that use an
niques are applied, the malware may be forcibly terminated analysis method coupled with a DBI tool. This is because the
depending on the existence of a virtual environment or a host OS may become inadvertently infected while performing
debugger. Packing is a technique that changes the control flow the malware analysis. However, if an anti-virtual machine
of program execution, making it difficult to analyze. As such, (anti-VM) technique is applied to the malicious code, anal-
many analysts use dynamic binary instrumentation (DBI) ysis methods using DBI tools cannot be used in a virtual
tools to analyze the packing of commercial protectors [1]–[3]. environment. In addition, even when anti-DBI techniques are
DBI tools insert executable code at run time to help applied, analysis methods using DBI tools still cannot be
used. Therefore, it is crucial to bypass anti-VM and anti-DBI
The associate editor coordinating the review of this manuscript and techniques in order to analyze malware used on commercial
approving it for publication was Porfirio Tramontana . protectors using DBI tools more efficiently in the field.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

VOLUME 9, 2021 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ 7655
 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

Assessment in this context is time consuming, as analysts Section VI presents our implementation and evaluation.
must bypass the protected techniques before using DBI tools Finally, we give our discussion and conclusion in Section VII
to analyze the malware. The longer it takes to analyze the mal- and VIII, respectively.
ware, the more damage can occur because antivirus programs
will not be able to update. In light of this, analysts must also II. BACKGROUND
understand anti-VM and anti-DBI techniques used in com- A. CODE OBFUSCATION
mercial protectors. In order to analyze malware in relation Code obfuscation, a technique used to transform a program,
to the anti-analysis technique applied, many researchers are hinders analysis because it modifies the internal code while
now studying anti-anti-analysis techniques, that is, bypassing maintaining functionality of the program. There are four
anti-analysis techniques. categories of code obfuscation [7]: layout, data, control flow,
Although anti-anti-analysis techniques have been studied and preventive.
to a degree in terms of a theoretical framework [4]–[6], these • Layout Obfuscation: This technique modifies or
are not directly suitable for anti-analysis techniques pro- removes detailed elements that do not affect the execu-
vided by commercial protectors. Further analysis is necessary tion of programs. It mainly corresponds to a technique
because analysts cannot know which anti-analysis technique that makes it difficult to transform variable names or
to use for commercial protectors. Also, even if the analyst remove comments.
knows which anti-analysis technique to use, the theoretical • Data Obfuscation: This technique transforms data
content cannot be applied as is because the anti-analysis values in a program or reconstructs data structures.
technique may be modified. Therefore, empirical studies It mainly corresponds to a technique that transforms
showing the results of various anti-analysis techniques used variable values in a complex way or reconstructs array
in commercial protectors are needed. structures.
The goal of this paper is to enable analysts to use DBI tools • Control Flow Obfuscation: This technique inserts
when analyzing malware employing anti-analysis techniques dummy codes or modifies the control flow from inside
supported by commercial protectors in a virtual environment. the program. It mainly corresponds to a technique that
Therefore, we propose guidelines to aid in the easy analysis inserts meaningless branch statements or transforms the
of malware protected by anti-VM or anti-DBI techniques control flow graph.
supported by commercial protectors. In addition, we present • Preventive Obfuscation: This technique inserts anal-
bypass algorithms for anti-VM and anti-DBI techniques used ysis prevention routines or analysis prevention codes
in commercial protectors along with our implementation and inside programs. It mainly corresponds to a technique
evaluation results. that prevents or terminates the operation of an analysis
Contributions: The following presents a detailed discus- tool (e.g., a debugger or a disassembler).
sion of our contributions. In actuality, preventive obfuscation is based on the same
• We propose guidelines to aid in the easy analysis of concept as anti-analysis techniques, and this paper aims to
malware protected by anti-VM and anti-DBI techniques implement bypass anti-analysis technique modules.
supported by commercial protectors. We present the
results of our detailed analysis for anti-VM and anti-DBI B. DYNAMIC BINARY INSTRUMENTATION
algorithms used in commercial protectors, and we also DBI is a technique primarily used for the dynamic anal-
categorize and present the techniques used with each ysis of programs [8], and it can be analyzed by inserting
tool. Therefore, our findings can be helpful for analysts instrumentation code during program execution. As such,
who want to analyze malware protected by commercial DBI is used to measure program performance, and ana-
protectors. lysts can use DBI tools to observe memory and register
• We present how to bypass anti-VM and anti-DBI tech- values during program execution, detect memory allocation
niques supported by commercial protectors through a errors, and perform security threat analysis. Because of these
detailed analysis of the algorithms used. This paper advantages, it has also recently been used to analyze mal-
features the first empirical research results produced via ware using DBI. Tools such as PIN [9], DynamoRIO [10],
a detailed analysis of anti-analysis techniques provided and Valgrind [11] typically provide a framework to
by commercial protectors, and the first to suggest actual perform DBI.
algorithms accordingly. In this paper, we chose to use PIN from the numerous
The organization of this paper is as follows: Section II DBI tools. PIN is a DBI framework provided by Intel that
describes the background of existing anti-analysis techniques can be used in IA-32, x86-64, and MIC instruction-set archi-
and their respective categories, and Section III presents tectures. It is widely used in security research because it
related works about bypassing anti-analysis. Section IV allows for measurement at the granular level and provides an
categorizes and presents anti-analysis techniques provided intuitive application programming interface (API). We chose
by commercial protectors, which are the actual targets for to use PIN for these reasons, but it should be noted that our
analysis. Section V presents an algorithm that bypasses proposed algorithm and results can still be used with other
anti-analysis techniques used in commercial protectors. DBI tools.

7656 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

C. ANTI-ANALYSIS E. ANTI-DBI
Anti-analysis is a technique that prevents or interferes with Anti-DBI techniques detect the situation under analysis with
program analysis. To make program analysis difficult, a pro- a DBI tool and forcibly terminates the program, which conse-
gram in which anti-analysis is applied can detect the anal- quentially means that dynamic analysis cannot be performed
ysis environment or analysis tools and forcibly terminate using a DBI tool. Anti-DBI performs DBI tool detection
itself. Commercial protectors also offer diverse anti-analysis using the principles or features of a certain DBI tool. Cur-
techniques, among these the most representative being anti- rently, commercial protectors do not provide any standalone
debugging, anti-VM, anti-patching, and anti-dumping. Anti- anti-DBI option. In fact, the anti-DBI techniques provided
debugging is a technique that prevents analysis using a by commercial protectors simply overlap with the techniques
debugger, and anti-VM techniques achieve this in a vir- provided in anti-debugging options. So far, few cases have
tual environment. Anti-patching mainly detects whether a been systematically studied to block DBI tools. The anti-DBI
file has been patched using a checksum value, and finally, techniques currently used in commercial protectors can be
anti-dumping detects and defends memory dumps performed classified into four basic categories.
by dump tools. Note that anti-debugging techniques do not • Overhead-based: The difference between the general
only detect debugging tools. Some also detect DBI tools, program execution time and the execution time of a pro-
which are referred to as anti-DBI techniques. Although gram being analyzed using the DBI tool is that the DBI
anti-DBI techniques is not a separate option provided by com- tool can be detected through the difference in execution
mercial protectors, we analyzed some techniques provided time of the analysis target program.
by anti-debugging options in commercial protectors to detect • JIT complier-based: Unlike general programs, the DBI
and prevent DBI tools. tool uses a just-in-time (JIT) compiler to patch and com-
pile certain instructions in the program to be analyzed
D. ANTI-VM in real-time to perform program analysis. A DBI tool is
Anti-VM is a technique that detects a virtual machine envi- detected using patching and compiling the instructions
ronment and terminates the program so that it cannot be in the DBI tool cache.
• API-based: Windows provides anti-debugging applica-
analyzed in a virtual machine environment. This technique is
currently provided by various commercial protectors and is tion programming interfaces (APIs). Some APIs that
used most commonly to prevent analysis in a virtual environ- perform anti-debugging can detect not only debuggers
ment. The virtual environment detection methods currently but also DBI tools, and anti-DBI uses these APIs to
used in commercial protectors can be classified into three detect DBI tools.
• Exception-based: In most general situations, programs
basic categories as follows:
perform exception handling where the exception occurs.
• Registry-based: When Windows is installed in a virtual During analysis using a DBI tool, programs operate
environment, the registry is set as information about the without executing exception handling in the part where
virtual machine. Therefore, there is a difference between the exception occurs. Therefore, if an exception occurs,
the registry value of the guest OS and that of the host a routine is inserted into the general program and the
OS, such that virtual environment detection can be per- normal operation code of the program is inserted into
formed using this. the exception handling part. Then, a DBI tool can be
• Hardware-based: In a virtual environment, logically detected as it operates the program without processing
separated H/W is used through the hypervisor, not the exception.
the physical H/W used in the host PC. Accordingly, A detailed outline of the anti-VM and anti-DBI techniques
there is a difference between the H/W information of used in commercial protector tools is provided in section IV.
the guest OS and that of the host OS. Virtual envi-
ronment detection is performed using the difference III. RELATED WORKS
between the H/W information of the guest OS and the This section describes existing research related to malware
host OS. analysis and unpacking using DBI tools, and it also explains
• Process/Service-based: In a virtual environment, spe- how the bypassing anti-VM and anti-DBI techniques pro-
cific programs are installed to use each virtual machine posed in this paper can be used efficiently in the field only
more effectively. Therefore, there are certain programs when they are provided together.
that only exist in the guest OS and are absent from Suk et al. [3] used PIN to analyze Themida. Based on
the host OS. For example, a virtual environment is the analysis results, Themida’s unpacking method was imple-
detected using a specific program that exists only in the mented by the algorithm, and the unpacking results were ver-
guest OS. ified using a large data set. This is the first empirical study to
There has been limited research on anti-VM techniques up to analyze almost all Themida-based obfuscation options simul-
this point, and only a limited offering of empirical research taneously while reconstructing them closely to the original
has explored which techniques are used in real commercial program. However, Suk et al. treats the anti-VM option as
protectors and how to bypass them. out of scope even though Themida has an anti-VM option.

VOLUME 9, 2021 7657

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

Suk et al. can remove in the unpacking process even if the information to the empirical analyst through the algorithm
anti-VM option is applied, but this is limited to the case proposed in this paper.
in which unpacking is performed in the host OS. However, Cheng, Binlin, et al. [14] proposed a new unpacking pro-
unpacking the malware from the host OS may lead to a cess called rebuilt-then-called, which differs from the existing
situation where the host OS becomes infected by the malware. written-then-executed process. Rebuilt-then-called uses the
When analyzing malware, it is important to analyze malware feature of redesigning the import address table (IAT) and
in a virtual environment, making it is necessary to consider an calling the API just before the original execution code is
anti-VM option. Therefore, by applying Themida’s anti-anti- executed in the packed program. It is also a technique of
VM algorithm presented in this paper, the analysis results will searching for the original entry point (OEP). This process
be enhanced with a more empirical study that can be applied can be analyzed without being detected by anti-debugging,
in the field. anti-DBI, or other anti-analysis techniques because it uses
M.Polino et al. [12] collected a large volume of malware, kernel-level DLL hijacking without using dynamic analy-
analyzed the anti-DBI techniques applied to them, and sug- sis tools. For performance evaluation, the authors collected
gested countermeasures. The researchers classified anti-DBI 238,835 packed malware and conducted experiments. A lap-
techniques into four categories based on the 7,006 malware top was used as the environment in which the malware was
programs they collected and analyzed in terms of how many unpacked, and as a result of the experiment, the tool described
anti-DBIs were applied. The results of this analysis and unpacking to be successful with a 97.3% probability. This
bypass were also described, and in order to show that their experiment demonstrated that unpacking is possible even
results could be applied to commercial packers, unpackers when anti-VM techniques are applied. However, there is
were produced in prototype form. Subsequently, the authors no part that explains whether this success was due to the
also evaluated bypassing anti-DBI techniques and unpacking application of the anti-anti-VM technique, or if it was merely
results for commercial packers. However, although this study not necessary to bypass because the experiment was con-
described 1,093 of the 7,006 malware programs as applying ducted in the host OS. When experimentation occurs in the
an anti-DBI technique, the existence of a ground truth appears host OS, if the part that the authors suggest would redesign the
to be lacking. Therefore, the method lacks a definitive way IAT is not found, the host OS can be infected with malware.
to check whether or not anti-DBI techniques were applied In fact, the researchers state that unpacking has failed with
to the 1,093 malware. Accordingly, there is a limitation in a 2.7% probability. Also, owing to the custom packer of the
that the possibility of false positives or false negatives can- malicious behavior payload that does not utilize IAT, there
not be excluded. In fact, the researchers do not separately is a limitation in that it is difficult to prevent packing-based
indicate that an anti-DBI technique has been applied in the malware from affecting the host OS. Therefore, it is important
1,093 malware included in the study. In addition, anti-DBI to perform the analysis in a virtual environment to cope with
bypassing and unpacking experiments were performed on various malware attack scenarios. In this paper, we propose
commercial packers used by malware in the wild. During the an anti-anti-VM study to complement the limitations of the
experiment, there was a case in which unpacking was not technique suggested by Cheng, Binlin, et al. Therefore, if the
performed correctly. This was because the experiment was above study were to consider the results of this paper, analysis
performed in a virtual machine environment and an anti-VM may be conducted in a safer environment.
technique was applied to the malware. Therefore, in order D’Elia, Daniele Cono, et al. [15] proposed Bluepill,
to empirically analyze malware using dynamic tools, not a human-centered dynamic analysis system to facilitate mal-
only anti-anti-DBI but also anti-anti-VM techniques must be ware analysis. Bluepill is based on DBI tool, and the authors
provided. of the above mentioned study configured a rule set for auto-
Park et al. [13] suggested automatic anti-debugging tech- matic bypass by analyzing known anti-analysis techniques.
nique detection and bypass methods using the PIN for a num- The rule set is a form in which a bypass algorithm is built
ber of self-selected commercial protectors. The researchers for each detailed anti-analysis technique, such that it can be
performed DBI detection analysis along with anti-debugging automatically bypassed when using the framework. There-
techniques and suggested a bypass method for DBI detection. fore, when analyzing malware using Bluepill, analysts can
In addition, it was described that the PIN works normally assess only anti-analysis techniques that are not included in
for a program applying anti-analysis via each commercial the existing framework and add them to the framework’s
protector. However, there is insufficient explanation about rule set. However, in the case of the anti-analysis technique
the anti-DBI algorithms used by each commercial protec- provided by commercial protectors, it is difficult to analyze
tor and proposed bypassing methods. Therefore, there is a anti-analysis technique algorithms and add them to the rule
limitation in that sufficient information cannot be provided set because obfuscation is applied simultaneously. Therefore,
to analysts who want to assess the anti-analysis techniques it is possible to supplement the anti-analysis bypass meth-
of commercial protectors using the information presented in ods of commercial protectors by updating Bluepill using the
that study. In response to this, we present our results after research results of this paper.
analyzing detailed algorithms used by each commercial pro- Choi, Seokwoo, et al. [16] proposed x64Unpack, which
tector, which means that it is possible to provide sufficient analyzes the packed executable file and unpacks it.

7658 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

TABLE 1. Summary of anti-VM and anti-DBI techniques used in commercial protectors.

In addition, the study presents analysis results on how the A. THEMIDA

program packed with VMProtect 3.4 works using x64Unpack Themida is a protector tool that provides 21 obfuscation
and which API is used. The researchers used x64Unpack options, including anti-analysis techniques at the binary
to bypass and unpack VMProtect, Themida, UPX, and level [19], and it continues to operate with new versions being
MPRESS anti-reversing techniques. x64Unpack is a form released. In this paper, we performed our analysis on Themida
of running a packed program using a CPU emulator in a version 2.4.5. Note that Themida offers various anti-analysis
host environment. If anti-reversing techniques appear in the options, but no anti-DBI techniques.
process of running, they are bypassed by having the authors’
proposal follow a predefined API and exception handling rou- 1) ANTI-VM OF THEMIDA
tine. To extend x64Unpack so that other commercial protec- Themida detects a virtual environment using two types of
tors can be unpacked, an anti-reversing technique bypassing anti-VM techniques. When detecting a virtual environment,
the API and exception handling routine must be added after a message box like in Fig.1 appears and the program is
additional analysis of the commercial protector. Therefore, terminated.
the commercial protector bypass methods presented in this
paper may improve the x64Unpack extension.

IV. ANTI-VM & ANTI-DBI ANALYSIS OF COMMERCIAL

PROTECTORS
In this section, we describe the detailed algorithm after
analyzing anti-VM and anti-DBI techniques provided by
five commonly used commercial protectors (i.e., Themida,
Enigma, VMProtect, Obsidium, and ACProtect). Upon ana- FIGURE 1. A pop-up message box that appears when executing the
program applied Themida’s anti-VM in a virtual environment.
lyzing the five commercial protectors, we summarized
anti-VM and anti-DBI techniques provided by each commer-
cial protector as shown in Table 1. Anti-VM techniques are
provided as an option in all commercial protectors except a: DETECTION USING REGISTRY KEY VALUE
one. All the tools that provided the anti-VM option also In each registry key value, there are various features that can
provided virtual environment detection techniques using the be assumed to be virtual environments. However, features
hardware features of the guest OS. Unfortunately, the anti- used by VMware and VirtualBox are utilized differently.
DBI technique has not yet been widely made available in When executing a program with Themida obfuscation applied
commercial protectors. Tools either contain one or three in a VMware environment, it reads the DriverDesc1 value
anti-DBI techniques, which all provide an array of technique VMware SVGA 3D among the values stored in the registry and
types. Later in this section, detailed algorithms are described performs virtual environment detection through string match.
related to anti-VM and anti-DBI used by each commercial See the VMware part in Fig.2.
protector. With the detailed algorithms of anti-VM, analysis In VirtualBox, the virtual environment is detected through
was performed on VMware and VirtualBox, which are two of string match by reading the VBOX −1 value of Sys-
the most commonly used virtual machines. VMware and Vir- temBiosVersion and the Oracle VM VirtualBox value of
tualBox in a virtual environment are representative enough to
be featured in two performance comparison papers spanning 1 HKEY_LOAL_MACHINE\SYSTEM\ControlSet001\Control\
a wide window of time [17], [18]. Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0000

VOLUME 9, 2021 7659

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

FIGURE 4. Bypassing a detection technique through modification of the

RegOpenKeyExA API.

FIGURE 2. Registry of VMware and VirtualBox used by Themida.

c: DETECTION USING IN INSTRUCTION
The VMware virtual machine has an I/O port communication
VideoBiosVersion among the values stored in the registry2 channel, through which it exchanges data between the host
when executing the program with Themida obfuscation. In OS and the guest OS. If analysts use the IN instruction,
addition, detection is performed by using the VBOX__ value, they can obtain information about I/O port. Additionally,
which is the folder name inside HARDWARE in HKLM.3 if analysts execute the IN instruction by inserting the value
See the VirtualBox part in Fig.2. 0x5658 (i.e.,VX) in the DX register (i.e., the communication
channel), the value containing the information of the virtual
b: BYPASSING DETECTION USING REGISTRY KEY VALUE machine is stored in the EAX or EBX register. Therefore, it is
In order to perform a DriverDesc value string match in possible to detect the virtual machine using the IN instruction.
the VMware registry, the memory value is read using the
memmove API. When calling the memmove API, the third d: BYPASSING DETECTION USING IN INSTRUCTION
parameter, a source address value, is checked to see if it has Themida uses the IN instruction twice to detect the VMware
the value as the VMware SVGA 3D. As shown in Fig.3, if the environment. In the first case, 0x564D5868 (i.e.,VMXh),
value exists, the anti-VM can be bypassed by modifying it. which indicates a magic number, is put in the EAX register,
and 0x14 (memory size request) is put in the ECX register.
The host OS cannot execute the IN instruction, so a 0x0 value
is entered in the EAX register. However, the IN instruction
is executed in the guest OS, and a value other than 0x0 is
received. Therefore, the anti-VM can be bypassed by modi-
fying the EAX register value to 0 x 0 after the IN instruction
executes as shown in Fig.5.

FIGURE 3. Bypassing a detection technique through modification of the

memmove API.

In VirtualBox, memmove API is used to read VBOX-1 and

Oracle VM VirtualBox values from the memory to perform
FIGURE 5. Bypassing a detection technique through modification of the
a string match. Therefore, bypass is possible through the register (IN1 instruction).
detection and modification of the corresponding value.
Additionally, VirtualBox is detected by checking whether In the second case, 0x564D5868 (i.e.,VMXh) is put in
the VBOX__ folder exists in the registry using the the EAX register, and 0xA (a request VMware Version
RegOpenKeyExA API. Therefore, as shown in Fig.4, it is type) is put in the ECX register. Subsequent execution of
possible to bypass detection by modifying the value of the the IN instruction in the guest OS leaves a unique value
registry folder name existing in the second argument. of 0x564D5868 (i.e.,VMXh) in the EBX register. Therefore,
2 HKEY_LOAL_MACHINE \HARDWARE\DESCRIPTION\Control
the anti-VM can be bypassed by setting the EBX register
\System value to 0x0 after the IN instruction executes as shown
3 HKEY_LOAL_MACHINE \HARDWARE\ACPI\DSCT\VBOX__ in Fig.6.

7660 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

FIGURE 8. A pop-up message box that appears when executing a

program applying Enigma’s anti-VM option in a virtual environment.

FIGURE 6. Bypassing a detection technique through modification of the Therefore, the bypass method is identical to that used in
register (IN2 instruction). Themida.

b: DETECTION USING RUNNING PROCESS

B. ENIGMA
Enigma detects VirtualBox by detecting the use of VBoxSer-
Companies still operate and manage Enigma as a commercial vice.exe, an additional program provided by VirtualBox,
protector, which provides anti-VM but not anti-DBI. In this which is a file created by installing VirtualBox Guest Addi-
paper, we performed our analysis on the Enigma version 6.00. tions. By installing VirtualBox Guest Additions, Virtual-
Box users can conveniently adjust the window size and res-
1) ANTI-VM OF ENIGMA olution. Therefore, most users who use VirtualBox will also
Enigma provides an anti-VM option to other environments install and use VirtualBox Guest Additions.
besides VMware and VirtualBox as shown in Fig.7 Because
the target of this paper is either VMware or Virtual- c: BYPASSING DETECTION USING A RUNNING PROCESS
Box, we did not perform any analysis for other environ- To detect VBoxService.exe, Enigma uses processs32next API
ments. Despite this, we made sure to verify all options to obtain information on currently running processes, and this
before performing the experiment. Among the anti-VM information is stored in Unicode in the memory. After this,
options provided by Enigma, there are three virtual envi- Enigma uses WideCharToMultiByte API to compare against
ronment detection techniques used in both VMware and the predefined VBoxSevice.exe ASCII value. Note that this
VirtualBox. The three options are VMware, VirtualBox, API converts Unicode values stored in the memory to ASCII.
and Hyper-V (CPU feature enabled). Each anti-VM option After that, it performs a string comparison using the process
detection message box is shown in Fig.8. The Hyper-V name changed to ASCII and performs detection. As shown
(CPU feature enabled) option can detect both VMware in Fig.9, the analyst can modify the process name value in
and VirtualBox through the CPU information of the virtual ASCII format stored in the memory, or they can bypass it by
environment. modifying the comparison routine.

d: DETECTION USING CPUID INSTRUCTION

The CPUID instruction returns various processor informa-
tion, such as the serial number and manufacturer ID according
to the EAX register value. Since most guest OS run on the
hypervisor, processor information that is distinct from the
host OS appears. Enigma uses these differences to detect
virtual environments.

e: BYPASSING DETECTION USING CPUID INSTRUCTION

FIGURE 7. Anti-VM option provided by Enigma.
If 0x1 is put in EAX and the CPUID instruction is executed,
information about the model and type of the processor is
a: DETECTION AND BYPASSING USING IN INSTRUCTION returned. After executing the CPUID instruction, the 31st bit
Programs applying Enigma detect VMware using the of the ECX register indicates the existence of a hypervisor.
IN instruction. Similarly, Enigma uses the Themida 2nd IN Therefore, by using this bit, the host OS and the guest OS can
instruction method, which executes 0xA in ECX register. be differentiated. If the bit is 1, it is deemed to be the

VOLUME 9, 2021 7661

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

a message box like that shown in Fig.11 appears and the

program is terminated.

FIGURE 11. A pop-up message box that appears when executing a

program applying VMProtect’s anti-VM option in a virtual environment.

a: DETECTION USING FIRMWARE TABLE

VMProtect detects a virtual machine environment via a string
match routine from the firmware table information obtained
FIGURE 9. Bypassing a detection technique through modification of the
WidecharToMultibyte API (VBoxService.exe).
using GetSystemFirmwareTable API.

b: BYPASSING DETECTION USING THE FIRMWARE TABLE

guest OS CPU, and Enigma determines that it is a virtual The program applying VMProtect’s anti-VM option has a
environment. To bypass the virtual environment detection routine to string match VMware and VirtualBox strings using
using the CPUID instruction, the anti-VM can be bypassed by information in the firmware table. VMware is detected by
changing it to 0 through an eXclusive OR (XOR) operation comparing the information of the firmware table by 1 byte
when the 31st bit of the ECX register is set to 1 as shown sequentially from the front of 0x564D7761 (i.e., VMwa),
in Fig.10. and VirtualBox is detected by comparing 1 byte sequentially
from the front of 0x56697274 (i.e., Virt). As shown in Fig.12,
the routines that VMProtect detects using firmware table
information are notably unique, such as CMP BYTE PTR
DS:[EDX], 0x56. Therefore, the analyst can locate a rou-
tine to check the firmware table using a unique comparison
routine instruction. To bypass this, the value stored in the
memory can be altered as shown in Fig.12 since the first byte
of both VMware and VirtualBox is 0x56.

c: DETECTION AND BYPASSING USING CPUID

INSTRUCTION
Like Enigma, VMProtect puts 0x1 in the EAX register when
executing CPUID and detects that it is a virtual environment
with the 1 and 0 of the 31st bit of ECX after execution. The
bypass method is identical to the one described in the Enigma
portion of this paper.
The program did not operate normally when analyzed
FIGURE 10. Bypassing a detection technique through modification of the using PIN, and the anti-DBI was applied.
register (CPUID instruction).

2) ANTI-DBI OF VMProtect
VMProtect does not have an anti-DBI option, but it is
C. VMProtect applied through an anti-debugging option. Anti-debugging
VMProtect is a powerful protector for virtualization obfus- is provided in two modes: the user mode and the
cation that provides various functions. In this paper, we ana- user mode + kernel mode. For this paper, we conducted
lyzed VMProtect version 3.0.9, which includes both anti-VM research on both modes. The anti-DBI technique provided by
and anti-DBI techniques. VMProtect is included in both modes. In addition, even when
the anti-debugging option was not set and only the anti-VM
1) ANTI-VM OF VMProtect option was set, it was confirmed that the anti-DBI effect
VMProtect uses three types of anti-VM techniques to detect also occurred. The analyst can see that the program does not
virtual environments. When detecting a virtual environment, work normally and terminates as shown in Fig.13 when using

7662 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

b: BYPASSING DETECTION USING SINGLE STEP (ANTI-VM

OPTION)
In order to bypass the anti-DBI technique in the anti-VM
option, the program operates normally by performing XOR
after checking whether or not the stack address is a trap flag
(0x100) before the POPFD instruction, as shown in Fig.14.

FIGURE 12. Bypassing a detection technique through modification of the FIGURE 14. Bypassing a detection technique by a trap flag XOR operation
memory (firmware table information). (single step (anti-VM)).

c: DETECTION USING SINGLE STEP (ANTI-DEBUGGING

OPTION)
This technique is the same as the single step technique applied
in the VMProtect anti-VM option. Though the detection pro-
cess is similar with that used in anti-VM, there are differences
in the bypassing process.

FIGURE 13. A program that has applied VMProtect anti-VM and d: BYPASSING DETECTION USING SINGLE STEP
anti-debugging options is terminated during analysis using DBI. (ANTI-DEBUGGING OPTION)
With single step, exception handling occurs during program
execution. However, with debugging or DBI, it is executed
normally without any exception handling. This means that
DBI for the programs to which the VMProtect anti-debugging single step can be used to detect the DBI tool by using this
and anti-VM are applied. There is one anti-DBI technique principle. When using a DBI tool, exception handling does
provided by VMProtect, but the bypass method of this not occur and the program executes, resulting in abnormal
technique is different depending on the anti-debugging and termination of the program, thus revealing the DBI tools and
anti-VM options. debuggers. Therefore, if a trap flag is set in the execution
timing of the POPFD instruction, as shown in Fig.15, it is
a: DETECTION USING SINGLE STEP (ANTI-VM OPTION) possible to bypass the program, thus allowing the program-
In VMProtect’s anti-VM option, a technique that forcibly mer to operate it as intended by forcing exception handling
terminates DBI uses a single step exception handling tech- using the API provided by the DBI tool.
nique. Single step is a technique that forcibly handles excep-
tions using the trap flag, which is activated by setting D. OBSIDIUM
0x100 in the EFLAGS register. Therefore, if 0x100 is Obsidium, like previously developed tools, is a protector that
put on the stack and the POPFD instruction is executed, continues to release new versions. In this paper, we analyzed
exception handling occurs. In a normal program, exception Obsidium version 1.6.7, which provides both anti-VM and
processing occurs by single stepping, but exception process- anti-DBI techniques.
ing does not occur in the state of analysis using a debug-
ger or a DBI tool. Therefore, programs not being analyzed 1) ANTI-VM OPTION OF OBSIDIUM
and using DBI work normally when the two anti-VM tech- The anti-VM technique provided by Obsidium includes three
niques are bypassed in a virtual environment, but those being types of techniques to detect virtual environments, during
analyzed using a DBI tool are abnormally terminated by which a message box like in Fig.16 appears and the program
single step. is terminated.

VOLUME 9, 2021 7663

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

FIGURE 15. Bypassing a detection technique by raise exception

(VMProtect single step (anti-debugging)).

FIGURE 17. Bypassing a detection technique through modification of the

FIGURE 16. A program applying Obsidium’s anti-VM option is terminated memory (disk drive value).
in a virtual environment.

d: DETECTION USING VBoxGuest FILE (ANTI-VM OPTION)

a: DETECTION AND BYPASSING USING IN INSTRUCTION Obsidium detects the existence of the VBoxGuest name file in
Programs applying Obsidium’s anti-VM option detect the order to detect the VirtualBox. VBoxGuest files are created by
VMware environment by using a feature that keeps the installing VirtualBox Guest Additions in VirtualBox. Regard-
VMXh unique value in the EBX register when IN instruc- less of whether it is VirtualBox or not, if VirtualBox Guest
tion is used in VMware. This technique is the same as Additions is not installed, it is certain that theVBoxGuest file
the second IN instruction technique used in Themida. As does not exist, and as such, the virtual environment cannot be
such, the bypass method is also identical to that used in detected by the corresponding technique.
Themida.
e: BYPASSING DETECTION USING VBoxGuest FILE
b: DETECTION USING DISK DRIVE VALUE (ANTI-VM OPTION)
The hardware contains various information such as model In Obsidium, CreateFileW API is used to check the exis-
names and serial numbers. Since the guest OS uses vir- tence of the VBoxGuest file. As shown in Fig.18, the pro-
tual hardware, different features from the host OS envi- gram inserts \\.\VBoxGuest as the file name argument value
ronment appear. Obsidium performs detection using the and OPEN_EXISTING as the mode argument value in
disk drive model name information. The disk drive model CreateFileW API.
names of VMware and VirtualBox include the VMware
and VBOX strings, which Obsidium uses to detect virtual
environments.

c: BYPASSING DETECTION USING DISK DRIVE VALUE

As in Fig.17, we found a routine in the program with Obsid-
ium and checked the virtual environment using the disk drive
model name.
In fact, this program has a routine to string match VMware
and VirtualBox strings from the disk drive model name. Note
that the first letter of both VMware and VBOX is 0x56
(i.e., V). Therefore, analysts can see that there is a comparison
routine such as CMP BYTE PTR DS: [EAX], 0x56 as shown FIGURE 18. Bypassing a detection technique through modification of the
register (VBoxGuest file).
in Fig.17. It is possible to bypass the anti-VM technique by
modifying the memory data in the EAX register value before
execution or by modifying the instruction to a value other If the return value of the EAX register is 0xFFFFFFFF
than 0x56. (−1) after executing the API, it means that the VBoxGuest

7664 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

file does not exist, and it is not a VirtualBox. Conversely, If it indeed has that value, it can be bypassed by modifying
if the return value of the EAX register returns with a value 0x56, the first character in the memory, to 0x0.
other than 0xFFFFFFFF, it means that theVBoxGuest file
exists, and it is a VirtualBox. Therefore, it is possible to 2) ANTI-DBI OF OBSIDIUM
bypass the anti-VM technique by changing the EAX register Obsidium’s anti-DBI technique is available when the
value to 0xFFFFFFFF in the comparison routine CMP EAX, anti-debugging option is applied. There are three type of
0xFFFFFFFF immediately after CreateFileW API. anti-DBI techniques used in Obsidium. If an analyst attempts
to perform an analysis of the program applying Obsidium’s
f: DETECTION USING SMBIOS INFORMATION ON THE anti-DBI technique using a DBI tool, the program will termi-
FIRMWARE TABLE nate as shown in Fig.20.
System Management BIOS (SMBIOS) is a standard for data
structures that is used to read information stored in the BIOS
on a computer. It contains a variety of information, such as
BIOS information and system information. The guest OS has
BIOS and system features that are distinct from the host OS,
FIGURE 20. A program applying the Obsidium anti-debugging option is
which makes it detectable using these features. terminated during analysis using DBI.

g: BYPASSING DETECTION USING SMBIOS INFORMATION

FROM THE FIRMWARE TABLE
a: DETECTION USING ZwQueryInformationProcess API
Obsidium uses GetSystemFirmwareTable API to obtain infor-
ZwQueryInformationProcess API is an API that allows you
mation about SMBIOS. In the information obtained through
to search for information in the process, and it can also detect
the API in the virtual environment is system informa-
debugging. If an analyst enters 0x7 (ProcessDebugPort) as
tion, including VMware and VirtualBox strings. In Obsid-
the second argument to this API, the address value of the third
ium, string matching is performed on the obtained system
argument returns a value indicating whether or not the debug-
information string. First, check the first character 0x56
ger is running. If debugging is in progress, 0xFFFFFFFF
(i.e., V) through the CMP BYTE PTR [ESI], 0x56 instruction
(−1) is returned. This technique detects debugging through
as shown in Fig.19. As shown in Fig.19, the first character
ZwQueryInformationProcess API as documented in MSDN.
0x56 (i.e., V) is checked through the instruction CMP BYTE
However, there is an additional undocumented debugging
PTR [ESI], 0x56 and then the routine to check the rest of the
detection option. If an analyst calls the API with 0x1F as
string is executed. After that, a virtual environment is detected
the second argument, the analyst can check whether debug-
using the signature stored in the EAX register (CMP EAX,
ging is being performed. If 0 is returned for the third address
0x117A8875(=signature)), which is calculated through the
value, debugging is in progress; if 1 is returned, debug-
operation in advance. Signatures are configured differently
ging is not in progress. The method of detecting by way of
for each virtual environment.
positioning the documented 0x7 as an argument does not
detect DBI tools, but if 0x1F is inserted, the API can detect
them.

b: BYPASSING DETECTION USING

ZwQueryInformationProcess API
Obsidium detects DBI by inserting 0x1F in ZwQueryIn-
formationProcess API. Generally speaking, to bypass this,
the DBI tool is used to track the ZwQueryInformationProcess
API. If the second argument is 0x1F, it can be bypassed by
changing the value stored in the third address to 1 after API
execution. However, in order to inhibit analysis in programs
FIGURE 19. Bypassing a detection technique through modification of the applying Obsidium, some APIs are not directly called. So
memory (firmware table information).
even after using the DBI tool to trace all the ZwQueryInfor-
mationProcess APIs used in the program, it cannot be traced
In the instruction to check the first character, the ESI well. Therefore, it is necessary to locate the ZwQueryInfor-
register value as the address has a value of 0x564D7761 (i.e., mationProcess API that performs the anti-DBI technique via
VMwa) in the case of VMware and 0x56697274 (i.e., Virt) a feature of argument values before calling ZwQueryInfor-
in the case of VirtualBox. Therefore, to bypass this, when mationProcess API. As shown in Fig.21, the nearest call is
the first character check routine is performed, the ESI reg- found using the argument value as a feature. As such, if the
ister value as the address is checked to see if it has a value value stored in the third address value after call is 0, it can be
of 0x564D7761 (i.e., VMwa) or 0x56697274 (i.e., Virt). bypassed by changing it to 1.

VOLUME 9, 2021 7665

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

FIGURE 21. Bypassing a detection technique by modifying the return

value (ZwQueryInformationProcess(0x1F) API).

FIGURE 22. Bypassing a detection technique by modifying the return

value (VBoxGuest file).

c: DETECTION USING VBoxGuest FILE (ANTI-DEBUGGING

OPTION)
However, cycles in which single step techniques appear are
If the VBoxGuest file does not exist while the program that has quite frequent and must be bypassed.
applied Obsidium is being analyzed through DBI, the process
is forcibly terminated as shown in Fig.20. If the program is
f: BYPASSING DETECTION USING SINGLE STEP
not being analyzed, it executes normally. However, when the
To bypass a single step, the AND EAX, 0x7 instruction is
program is being analyzed using DBI, the DBI is detected
performed after the CALL DWORD PTR [EBX+0x12C]
using the existence of the VBoxGuest file. Therefore, when
instruction as shown in Fig.23. After executing the AND
the VBoxGuest file is installed in the VirtualBox environment,
EAX, 0x7 instruction, we confirmed that various values were
it can be analyzed without being detected if the program is
entered in the EAX register. The single step routine proceeds
analyzed using DBI. The difference from VBoxGuest detec-
only when the EAX register is 0x5 or 0x2. Therefore, bypass
tion in the anti-VM option is that, in the anti-debugging
is possible by changing the EAX register to a value other than
option, the VBoxGuest file must exist to bypass it.
0x5 or 0x2 after executing the AND EAX, 0x7 instruction.

d: BYPASSING DETECTION USING VBoxGuest FILE

(ANTI-DEBUGGING OPTION)
In order to bypass DBI detection using VBoxGuest, the ana-
lyst must do the opposite of what is necessary for the anti-VM
option. When the CreateFileW API is executed, the analyst
checks the argument to verify whether or not VBoxGuest is
set as the file name, and OPEN_EXISTING is set as the mode.
After that, CMP EAX, 0xFFFFFFF instruction appears as
shown in Fig.22. If the EAX register value is 0xFFFFFFFF,
program debugging is detected, so the analyst can bypass
the anti-DBI technique by changing 0xFFFFFFFF to another
value.

e: DETECTION USING SINGLE STEP

Obsidium’s anti-debugging option uses a single step tech-
nique to forcefully terminate the program during analysis
using DBI. This single step technique is performed sporad-
ically and not every time the program is executed, which
means that sometimes the analysis of the program can be FIGURE 23. Bypassing a detection technique through modification of the
completed using DBI without bypassing the single step. register (single step).

7666 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

E. ACProtect
ACProtect, unlike other tools, is a protector that does not
continue to release new versions. In this paper, we analyzed
the ACProtect version 2.0, which offers a variety of options
like anti-debugging. However, ACProtect does not provide an
anti-VM option.

1) ANTI-DBI OF ACProtect
Unlike previously developed protectors, even if the
anti-debugging option is not applied in ACProtect, the anti-
FIGURE 25. A program applying ACProtect crashes due to
DBI technique is applied. ACProtect has one anti-DBI tech- self-modification during analysis using DBI.
nique, and when the ACProtect-applied program is analyzed
by a DBI tool, the program does not operate normally and is
terminated as shown in Fig.24.
instructions are put into the cache and executed line by line
as shown in Fig.25. When the memory write instruction
MOV DWORD PTR SS: [EBP+F], EAX is executed in DBI
cache, the memory area of the program is altered, but not the
instruction in the DBI cache. Therefore, the DBI tool causes
a crash by executing an instruction that is not self-modifying.
To bypass this, the analyst should check the instructions that
write the memory region in the DBI cache. Once this is
located, the analyst should check if the instruction to write
the memory modifies the instructions in the DBI cache. If the
instruction to be modified is an instruction included in the
DBI cache, it can be bypassed by clearing the cache after
executing the instruction to write memory and collect the
FIGURE 24. A program applying ACProtect is terminated during analysis changed instruction again.
using DBI.

V. ANTI-ANTI VM & ANTI-ANTI DBI OF COMMERCIAL

a: DETECTION USING SELF-MODIFICATION CODE PROTECTORS
ACProtect uses self-modification code to detect DBI tools. This section presents an algorithm that can bypass anti-VM
This code modifies its own instruction code while the pro- and anti-DBI techniques for each tool by using a DBI tool
gram is running. As shown in Fig.25, this poses no prob- based on the preceding analysis.
lems for normal program execution, but when an analyst In all algorithms, the current instruction (curINS), the cur-
executes a program with DBI tool, the program terminates. rent instruction address (curAddr), and the current API
This is because DBI tools use a just-in-time (JIT) compiler routine (curRTN) are entered as arguments. Input options
to compile and run programs in real time. When a DBI tool curINS,curRTN, or curAddr helps the algorithm determine
executes, instructions for programs of a certain size are put in whether or not an anti-analysis technique is applied based on
the cache, and the program is executed through real-time JIT which is received.
compilation. When self-modification is applied, the instruc-
tion appropriately self-modifies during program execution. If A. ANTI-ANTI-VM OF THEMIDA
the modified code is placed in DBI cache before modification, Themida detects the virtual environment using two types
the modified code is not applied. Therefore, when using a of techniques. The algorithm to bypass Themida’s anti-VM
DBI tool, the instruction is executed before modification so option using a DBI tool is shown as Alg. 1.
that the program does not execute, indicating that, as a result, The program applying Themida’s anti-VM technique
the DBI tool can be detected using this principle. detects a virtual environment using the IN instruction. If the
EDX register contains value 0 x 5658 (VX) in the execution
b: BYPASSING DETECTION USING SELF-MODIFICATION order of the IN instruction, it means that information about the
CODE virtual machine is obtained from the port. The algorithm then
In Fig.25, the MOV DWORD PTR SS: [EBP+F], EAX removes the IN instruction to bypass the anti-VM technique
instruction is a memory write instruction. If the instruction and checks if the value of the ECX register is 0 x 14 or 0xA.
to write the memory is executed, MOV EAX, 5A is changed If the ECX register value is 0x14, the EAX register changes
to MOV EAX, ACP_NONE.0041B462. However, when a pro- to 0. Also, if the ECX register is 0xA, the EBX register
gram is executed using a DBI tool, a certain amount of value changes to 0. By changing the EAX and EBX register

VOLUME 9, 2021 7667

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

Algorithm 1 Anti-Anti-VM Algorithm for Themida Algorithm 2 Anti-Anti-VM Algorithm for Enigma
Input: Current Instruction (curINS), Current API (curRTN) Input: Current Instruction (curINS), Current API (curRTN)
1: if (curINS = IN _ins) AND (EDX = ‘‘VX ’’) then 1: CPUID_Flag ← False
2: INS_Delete(curINS) 2: GuestOS_Sig ← 0x80000000
3: if ECX = Memsize_Request (= 0x14) then 3: if CPUID_Flag = True then
4: EAX ← 0 4: if (ECX & GuestOS_Sig) 6= 0 then
5: else if ECX = Version_Request (= 0xA) then 5: ECX ← (ECX xor GuestOS_Sig)
6: EBX ← 0 6: end if
7: end if 7: CPUID_Flag ← False
8: end if 8: end if
9: if curRTN = memmove_API then 9: if curINS = CPUID_ins then
10: if Src = (‘‘VMware_SVGA_3D’’ OR ‘‘Oracle_VM ’’) 10: CPUID_Flag ← True
then 11: end if
11: Src ← 0 12: if (curINS = IN _ins) AND (EDX = ‘‘VX ’’) then
12: end if 13: INS_Delete(curINS)
13: end if 14: if ECX = Version_Request (= 0xA) then
14: if curRTN = RegOpenKeyA_API then 15: EBX ← 0
15: if hKey = HKLM then 16: end if
16: if Subkey = ‘‘VBOX __’’ then 17: end if
17: Subkey ← 0 18: if curRTN = WideCharToMultiByte_API then
18: end if 19: if WideCharStr = ‘‘VBoxService.exe’’ then
19: end if 20: WideCharStr ← 0
20: end if 21: end if
22: end if

values to 0, the analyst can bypass the anti-VM using the IN

instruction. is 0 x 5658 (VX). If the ECX register is 0xA, an analyst can
The program applying the Themida anti-VM option detects bypass the anti-VM using the IN instruction by changing the
a virtual environment using registry values with an algo- EBX register value to 0.
rithm that saves each signature for three special registry Enigma checks whether or not VBoxSevices.exe has been
values used by Themida. The algorithm checks for the executed to detect VirtualBox, but the string match deter-
existence of VMware SVGA 3D and Oracle VM through mines whether or not VBoxSevices.exe executes. In a program
memmove API tracking, and if it exists, it changes to 0. to which Enigma is applied, WideCharToMultiByte API must
To check the VBOX registry key of Virtualbox, the algo- be used to match string format, which allows a string match
rithm traces RegOpenKey_A API. When opening HKLM of a VBoxSevices.exe string to be performed. Therefore, when
(HKEY_LOCAL_MACHINE), if the VBOX__ is included, changing the string format of VBoxSevices.exe by tracking
it is possible to bypass anti-VM by changing the value to 0. WideCharMultiByte API, the anti-VM option can be bypassed
by changing the string to 0.
B. ANTI-ANTI-VM OF ENIGMA
Enigma detects a virtual environment using two types of tech- C. ANTI-ANTI-VM OF VMProtect
niques. The algorithm to bypass Enigma’s anti-VM option VMProtect detects virtual environments using two types of
using a DBI tool is shown as Alg. 2. techniques. In order to bypass VMProtect’s anti-VM option
The program applying Enigma’s anti-VM option detects using DBI, one of the anti-DBI techniques (single step) needs
a virtual environment using the CPUID instruction, which to be bypassed. The algorithm used to bypass using a DBI tool
is tracked by the algorithm. After executing the CPUID in this case is shown as Alg. 3.
instruction to check whether the 31st bit of the ECX register The program applying the VMProtect anti-VM option uses
is set to 1, the algorithm performs an AND operation of the CPUID instruction to detect a virtual environment. The
0x80000000 (GuestOS_Sig) to the value of the ECX register. bypass method for the CPUID instruction is the same as the
If the 31st bit is set to 1, a bypass is possible by changing the method used in Enigma.
bit value to 0 through an XOR operation. The program applying the VMProtect anti-VM option uses
The program applying to Enigma’s anti-VM option uses firmware table information to detect a virtual environment.
the IN instruction to detect a virtual environment. The dif- In the firmware table information, the string ‘‘VMware’’
ference from Themida is that only the IN instruction is used exists for VMware and the string ‘‘Virtual’’ exists for
when the ECX register is 0xA. The algorithm traces the IN VirtualBox. There is a unique instruction called CMP BYTE
instruction and removes it if the value of the EDX register PTR DS:[EDX], 0x56 that is used for virtual environment

7668 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

Algorithm 3 Anti-Anti-VM Algorithm for VMProtect Algorithm 5 Anti-Anti-VM Algorithm for Obsidium
Input: Current Instruction (curINS), Current API(curRTN) Input: Current Instruction (curINS), Current API (curRTN)
1: VM _CheckRoutine ← ‘‘CMP BYTE PTR 1: VM _Diskdirve ← ‘‘CMP BYTE PTR [EAX], 0x56’’
DS:[EDX], 0x56’’ 2: VM _firmware ← ‘‘CMP BYTE PTR [ESI], 0x56’’
2: CPUID_Flag ← False 3: VGuest_Cmp ← ‘‘CMP EAX, 0xFFFFFFFF’’
3: GuestOS_Sig ← 0x80000000 4: VGuest_Flag ← False
4: if CPUID_Flag = True then 5: if (curINS = IN _ins) AND (EDX = ‘‘VX ’’) then
5: if (ECX & GuestOS_Sig) 6 = 0 then 6: INS_Delete(curINS)
6: ECX ← (ECX xor GuestOS_Sig) 7: if ECX = Version_Request (= 0xA) then
7: end if 8: EBX ← 0
8: CPUID_Flag ← False 9: end if
9: end if 10: end if
10: if curINS = CPUID_ins then 11: if curINS = VM _Diskdirve then
11: CPUID_Flag ← True 12: if ∗EAX = (‘‘VMware’’ OR ‘‘VBOX ’’) then
12: end if 13: ∗EAX ← 0
13: if curINS = VM _CheckRoutine then 14: end if
14: if ∗EDX = (‘‘VMware’’ OR ‘‘Virtual’’) then 15: end if
15: ∗EDX ← 0 16: if (curINS = VGuest_Cmp) AND VGuest_Flag then
16: end if 17: if EAX 6= 0xFFFFFFF then
17: end if 18: EAX ← 0xFFFFFFF
18: if curINS = POPFD_ins then 19: VGuest_Flag ← False
19: if (∗ESP & TrapFlag(= 0x100)) 6 = 0 then 20: end if
20: ∗ESP ← (∗ESP xor TrapFlag(= 0x100)) 21: end if
21: end if 22: if curRTN = CreateFileW _API then
22: end if 23: if FileName = \\.\VBoxGuest then
24: if Mode = OPEN _EXISTING then
25: VGuest_Flag ← True
Algorithm 4 Anti-Anti-DBI Algorithm for VMProtect
26: end if
Input: Current Instruction (curINS), Current API (curRTN)
27: end if
1: if curINS = POPFD_ins then
28: end if
2: if (∗ESP & TrapFlag(= 0x100)) 6 = 0 then
29: if curINS = VM _firmware then
3: DBI _RaiseException(EIP + 1, ESP + 1))
30: if ∗ESI = (‘‘VMware’’ OR ‘‘Virtual’’) then
4: end if
31: ∗ESI ← 0
5: end if
32: end if
33: end if

detection with the corresponding firmware table information.

With this, a CMP BYTE PTR DS:[EDX], 0x56 instruction
trace is performed and the value is checked against the value the single step. There is a slight difference from the method
of the EDX register as an address before the instruction is used in the anti-VM option of VMProtect. The algorithm
executed. If a the string ‘‘VMware’’ or ‘‘Virtual’’ value exists, performs a POPFD instruction trace and checks whether
it can be bypassed by changing the value to 0. 0x100 (trap flag) is set in ESP before executing the POPFD
As described above, an anti-DBI technique must be instruction. If 0x100 (trap flag) is set, a bypass is possible
bypassed to circumvent the anti-VM option. To bypass a by forcibly making an exception using the exception API
single step, thePOPFD instruction must be traced. Before provided via the DBI tool.
executing the POPFD instruction, the analyst should check
whether 0 x 100 (trap flag) is set in ESP. If 0 x 100 (trap flag) E. ANTI-ANTI-VM OF OBSIDIUM
exists, it can be bypassed by removing the trap flag through Obsidium detects a virtual environment using three types of
an XOR operation. techniques. The algorithm that bypasses Obsidium’s anti-VM
option using a DBI tool is shown as Alg. 5.
D. ANTI-ANTI-DBI OF VMProtect The program applying Obsidium’s anti-VM option uses
There is only one technique that can detect DBI tools the IN instruction to detect virtual environments. The IN
in VMProtect. The algorithm that bypasses VMProtect’s instruction technology used in Obsidium is the same as the
anti-debugging option using a DBI tool is shown as Alg. 4. technique that inserts 0xA into the ECX register and executes
In order to bypass the anti-DBI technique provided by it in Enigma. The bypass method is similar to that of the
VMProtect’s anti-debugging option, an analyst must bypass previously discussed method for Enigma.

VOLUME 9, 2021 7669

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

The program applying the Obsidium anti-VM technique Algorithm 6 Anti-Anti-DBI Algorithm for Obsidium
detects the virtual environment using disk drive information. Input: Current Instruction (curINS), Current API (curRTN)
In the disk drive information, the string ‘‘VMware’’ exists 1: Debug_Check_Address ← 0
for VMware and the string ‘‘VBOX’’ exists for VirtualBox. 2: ZwQuery_Flag1 ← False
There is a unique detection instruction called CMP BYTE 3: ZwQuery_Flag2 ← False
PTR DS:[EAX], 0x56, which uses the corresponding disk 4: VGuest_Cmp ← ‘‘CMP EAX, 0xFFFFFFFF’’
drive information. Therefore, the CMP BYTE PTR DS:[EAX], 5: VGuest_Flag ← False
0x56 instruction trace is performed, and the value is checked 6: Single_Sig1 ← ‘‘CALL DWORD PTR
against the value of the EAX register as an address before the [EBX+0x12C]’’
instruction is executed. If the string ‘‘VMware’’ or ‘‘VBOX’’ 7: Single_Sig2 ← ‘‘AND EAX, 0x7’’
value exists, it can be bypassed by changing the value to 0. 8: Single_Flag ← False
The program applying the Obsidium’s anti-VM option 9: if ZwQuery_Flag2 then
detects VirtualBox by the existence of VBoxGuest files. Cre- 10: if ∗Debug_Check_Address = 0x0 then
ateFileW API is executed in order to determine the existence 11: ∗Debug_Check_Address ← 0x1
of a VBoxGuest file, and when executing the CreateFileW 12: end if
API, \\.\VBoxGuest exists in the FileName value. After exe- 13: end if
cuting the CreateFileW API, it is determined whether or 14: if (curINS = RET _ins) AND ZwQuery_Flag1 then
not the VBoxGuest file exists through the result value that 15: ZwQuery_Flag2 ← True
is stored in the EAX register. If the file does not exist, 16: end if
0xFFFFFFFF is stored in the EAX register, and if the file 17: if ZwQueryInformation_Argument_Check then
exists, a value other than 0xFFFFFFFF is stored. There- 18: if curINS = CALL_ins then
fore, to bypass this, an analyst must check if \\.\VBoxGuest 19: Debug_Check_Address ← ∗(ESP + 12)
exists in the FileName value in the CreateFileW API. After 20: ZwQuery_Flag1 ← True
that, the algorithm traces whether or not the CMP EAX, 21: end if
0xFFFFFFFF instruction appears. If the EAX register is 22: end if
any value other than 0xFFFFFFFF, it can be bypassed by 23: if (curINS = VGuest_Cmp) AND VGuest_Flag then
changing it to 0xFFFFFFFF. 24: if EAX 6= 0xFFFFFFF then
The program applying the Obsidium anti-VM option uses 25: EAX ← 0xFFFFFFF
the firmware table information to detect virtual environments. 26: VGuest_Flag ← False
In the firmware table information, the string ‘‘VMware’’ 27: end if
exists for VMware and the string ‘‘Virtual’’ exists for Virtual- 28: end if
Box. There is a unique instruction CMP BYTE PTR DS:[ESI], 29: if curRTN = CreateFileW _API then
0 x 56 that is used for virtual environment detection with the 30: if FileName = \\.\VBoxGuest then
corresponding firmware table information. Therefore, a CMP 31: if Mode = OPEN _EXISTING then
BYTE PTR DS:[ESI], 0x56 instruction trace is performed, 32: VGuest_Flag ← True
and the value is checked against the value of the ESI register 33: end if
as an address before the instruction is executed. If either 34: end if
the string ‘‘VMware’’ or ‘‘Virtual’’ value exists, it can be 35: end if
bypassed by changing the value to 0. 36: if (curINS = Single_Sig2) AND Single_Flag then
37: if (EAX & 0x7) = (0x2 OR 0x5) then
F. ANTI-ANTI-DBI OF OBSIDIUM 38: EAX ← 0x0
There are three types of techniques that can be used to 39: Single_Flag ← False
detect DBI tools in Obsidium. The algorithm that bypasses 40: end if
Obsidium’s anti-debugging option using a DBI tool is shown 41: end if
as Alg. 6. 42: if curINS = Single_Sig1 then
In order to bypass the anti-DBI provided by Obsidium’s 43: Single_Flag ← True
anti-debugging option, an analyst must bypass the ZwQuery- 44: end if
InformationProcess(0x1F) API. In general, the analyst can
bypass the anti-DBI option by tracking the ZwQueryInfor-
mationProcess(0x1F) API using the API tracking function of
the PIN and modifying the return value, but in the case of through the analysis. Obsidium’s anti-DBI technique sequen-
Obsdium, API tracking is not possible because the ZwQuery- tially executes instructions for push 0x4, push EDX, and
InformationProcess(0x1F) API is not called directly. There- push 0x1F in order to insert argument values and call
fore, Obsidium’s anti-DBI technique should be tracked and ZwQueryInformationProcess(0x1F) API. In the algorithm,
bypassed using the argument value pattern inserted when this series of processes is expressed as ZwQueryInfor-
calling the ZwQueryInformationProcess(0x1F) API found mation_Argument_Check. After that, Obsidium’s anti-DBI

7670 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

Algorithm 7 Anti-Anti-DBI Algorithm for ACProtect and the last given value in the DBI cache. The algorithm
Input: Current Address (curAddr), Current Instruction stores which address was written when each instruction that
(curINS) writes memory is executed. If the newly written address
1: DBI _Head ← TRACE_Address(trace) value is included between the DBI cache start address value
2: DBI _Tail ← DBI _Head + TRACE_Size(trace) and the last address value, a bypass is possible through the
3: FirstWritten ← 0x0 DBI API that collects the cache again. Through this process,
4: if INS_MemoryOperandIsWritten(curINS) then the instructions altered during execution are newly collected
5: Write_addr ← MemoryWriteAddress and executed in the DBI cache, so that the program operates
6: if DBI _Head ≤ Write_addr < DBI _Tail then normally.
7: FirstWritten ← Write_addr
8: end if VI. IMPLEMENTATION & EVALUATION
9: end if We implemented a tool using the DBI framework to exper-
10: if curAdd < FirstWritten ≤ curAdd + ins_size then iment with the proposed algorithm as shown in Fig. 26.
11: FirstWritten ← 0x0 The input of the tool is malware, into which the anti-VM
12: DBI _Cache_Collection(curAdd) and anti-DBI techniques of commercial protectors are
13: end if applied. When the malware goes through the three modules,
it bypasses the anti-VM or anti-DBI option, and the program
runs normally without interruption. Therefore, malware can
be analyzed using DBI in a virtual environment.
technique indirectly calls the ZwQueryInformationPro- The following describes the three aforementioned mod-
cess(0x1F) API using the CALL instruction. Since these ules.
patterns appear, the algorithm tracks them to locate which API Trace Module traces and bypasses the API used
part calls the ZwQueryInformationProcess(0x1F) API. The by anti-DBI and anti-VM techniques by using the API
return value of the ZwQueryInformationProcess(0x1F) API trace function of a DBI tool. This module traces mem-
is stored in value with the 3rd parameter value (ESP+12) move, RegOpenKeyA, CreateFileW API, and others to bypass
as the address. Therefore, Alg. 6 stores the value of ESP anti-VM and anti-DBI techniques.
+12 in the Debug_Check_Address variable in advance. When Instruction Trace Module tracks and bypasses instruc-
the ZwQueryInformationProcess(0x1F) API is terminated, tions used by anti-DBI and anti-VM techniques using the
the RET instruction is called. Therefore, after the RET instruc- instruction tracking function of a DBI tool. This module
tion, if it is in a debugging environment, 0 is stored in the detects and bypasses special instructions that are used to
value addressed to Debug_Check_Address such that it can be detect virtual environments, such as CPUID, IN instructions.
bypassed when changed to 1. Memory Write Check Module bypasses self-modification
Obsidium’s anti-DBI option detects DBI using the exis- technique, which employs the JIT compiler characteristics
tence of VBoxGuest files. The algorithm performs a trace among anti-DBI techniques. This module detects instructions
of the CreateFileW API and checks if the \\.\VBoxGuest is that write memory, and then it checks whether the newly
included as an argument. After that, the algorithm traces written memory address is included in the DBI cache. If it
whether or not the CMP EAX, 0xFFFFFFFF instruction is included, DBI collects a new cache to bypass the anti-DBI
appears. If the EAX register is 0xFFFFFFFF, it can be technique.
bypassed by changing that value to anything other than For our evaluation, 1,573 test execution files were gen-
0xFFFFFFFF. erated using Juliet Test Suite version 1.3 provided by the
Obsidium’s anti-DBI option uses a single step technique National Institute of Standards and Technology (NIST). The
to detect DBI and then forcibly terminates the program. Juliet Test Suite code is categorized into a set of 118 secu-
However, the single step routine does not appear every time rity weaknesses based on common weakness enumeration
a program is executed, and its appearance is dependent on (CWE). Each set has one or more flaw types that cause
the state of the EAX register after the CMP DWORD PTR security weaknesses, and the Juliet Test Suite code contains
[EBX+0 x 12C] instruction and AND EAX, 0x7 instruction. 1,617 vulnerability types. Since Juliet Test Suite code can
If the EAX register is 0x2 or 0x5 after the AND EAX, trigger vulnerabilities, it can also be used to create malware.
0x7 instruction, the single step routine proceeds so it can be Therefore, in this experiment, the experiment was performed
bypassed by changing the EAX register to 0x0. assuming that the Juliet Test Suite code was malware.
The evaluation environment is as follows. The virtual
G. ANTI-ANTI-DBI OF ACProtect environment used in the experiment was performed in
There is one type of technique for ACProtect to detect DBI VMware Workstation 15 Player and Oracle VirtualBox 6.0,
tools. The algorithm to bypass ACProtect’s anti-DBI option both of which were configured in Windows 7. The experi-
using a DBI tool is shown as Alg. 7. ment was performed using PIN version 2.7 of a DBI tool.
In order to bypass ACProtect’s anti-DBI, self-modification Experiments were performed by applying anti-VM and
must be bypassed. The algorithm stores the start address value anti-DBI techniques to 1,573 test portable executable (PE)

VOLUME 9, 2021 7671

 Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

FIGURE 26. Overview of implementation.

files using five commercial protector tools (Themida 2.4.5, nique is the most used when investigating the anti-DBI
Enigma 6.00, VMProtect 3.0.9, Obsidium 1.6.7, and ACPro- technique applied to malware. We are presenting a bypass
tect 2.00). ACProtect does not provide any function that algorithm for both techniques. Therefore, by finely adjusting
obfuscates large data sets automatically, and as such, the bypass algorithm proposed in this paper, it will be possible
no obfuscation options provided by ACProtect were applied to cover most cases. As such, we believe it would be inac-
to every test set. As a result, 30 test sets were manually curate to claim that the general applicability of our proposal
generated and evaluated. is insufficient in that the experiment was performed on five
In terms of our experimental method, it was confirmed typical commercial protectors.
that the test set with an anti-VM technique in a virtual There are many options for any one commercial protector.
machine environment was bypassed using PIN, and the pro- In this paper, since only anti-VM and anti-DBI techniques
gram executed successfully. We also performed an anti- were analyzed, it is difficult to present experimental results
anti-DBI experiment to check whether the test set with an on whether the proposed bypass algorithm works properly
anti-DBI technique was bypassed, and the program was even when other obfuscation options are applied. However,
successfully executed using PIN. The evaluation results are in the case of Themida, it is a structure in which unpack-
shown in Table 2. Test data sets obfuscated through each ing is executed if it is not detected after the anti-analysis
protector were all bypassed using the proposed algorithm. technique is performed, and these two parts can be consid-
The experimental results indicate that if the malware uses ered to be separate [3]. Even when other protection tech-
an anti-VM technique and an anti-DBI technique from each niques are applied, anti-analysis techniques are performed
commercial protector, the proposed algorithm makes it pos- first and followed by other techniques. Therefore, even if
sible to bypass. other options are applied, deobfuscation should be applied
after bypassing the anti-analysis technique, according to Suk
TABLE 2. Proposed anti-anti-VM & anti-anti-DBI algorithm evaluation. et al. [3]. Since other commercial protectors have a simi-
lar structure, the bypass algorithm presented in this paper
can work normally even when other obfuscation options
are applied.

VIII. CONCLUSION
In order to analyze malware protected by commercial pro-
tectors in a virtual environment using DBI, we needed
to additionally analyze anti-VM and anti-DBI techniques.
VII. DISCUSSION We presented a detailed analysis and proposed bypass algo-
At first glance, the method presented in this paper may rithms for anti-VM and anti-DBI techniques for commercial
appear to be applicable only to anti-DBI and anti-VM protectors in this paper, which is the first empirical study
techniques for five commercial protectors. However, from to propose detailed bypass algorithms of this capacity. The
the results of previous studies, the types of anti-DBI and results of this study can serve as guidelines for easy anal-
anti-VM options are somewhat limited [5], [6], [12], [20], ysis of malware protected by an anti-VM or an anti-DBI
[21]. Branco, Rodrigo Rubira, Gabriel Negreira Barbosa, and option supported by commercial protectors. In addition, other
Pedro Drimel Neto [20] mentioned that IN instruction is recent research has focused on unpacking DBI tools, and
the most used anti-VM technique found in malware. Polino, we believe that our bypass algorithm will aid in achieving
Mario, et al. [12] mentioned that the Self-modification tech- higher success rates in future research. However, our paper

7672 VOLUME 9, 2021

Y. B. Lee et al.: Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools

only addresses solutions for current techniques used in com- [21] H. Shi, J. Mirkovic, and A. Alwabel, ‘‘Handling anti-virtual machine
mercial protectors, and malware using customized protec- techniques in malicious software,’’ ACM Trans. Privacy Secur., vol. 21,
no. 1, pp. 1–31, Jan. 2018.
tors along with new versions of commercial protectors using
anti-analysis techniques are emerging constantly. Consider-
ing these developments, anti-anti-analysis techniques must
be studied further, which will form the basis of our future
research.

REFERENCES
[1] S. D’Alessio and S. Mariani, ‘‘PinDemonium: A DBI-based generic
unpacker for Windows executables,’’ Black Hat USA, Tech. Rep., 2016.
[2] D. Reynaud and J.-Y. Marion, ‘‘Dynamic binary instrumentation for deob- YOUNG BI LEE received the B.S. degree in infor-
fuscation and unpacking,’’ in Proc. In-Depth Secur. Conf. Eur. (Deepsec), mation security engineering from Soonchunhyang
2009. University, Asan, South Korea, in 2019. He is
[3] J. H. Suk, J.-Y. Lee, H. Jin, I. S. Kim, and D. H. Lee, ‘‘UnThemida: Com- currently pursuing the M.S. degree in information
mercial obfuscation technique analysis with a fully obfuscated program,’’ security with the Graduate School of Information
Softw., Pract. Exper., vol. 48, no. 12, pp. 2331–2349, Dec. 2018. Security, Korea University. His research interests
[4] X. Chen, J. Andersen, Z. M. Mao, M. Bailey, and J. Nazario, ‘‘Towards include software protection, program obfuscation,
an understanding of anti-virtualization and anti-debugging Behavior in program deobfuscation, reverse engineering, mal-
modern malware,’’ in Proc. IEEE Int. Conf. Dependable Syst. Netw. FTCS ware analysis, and digital forensic.
DCC (DSN), 2008, pp. 177–186.
[5] D. C. D’Elia, E. Coppa, S. Nicchi, F. Palmaro, and L. Cavallaro, ‘‘SoK:
Using dynamic binary instrumentation for security (And how you may get
caught red Handed),’’ in Proc. ACM Asia Conf. Comput. Commun. Secur.,
Jul. 2019, pp. 15–27.
[6] P. Chen, C. Huygens, L. Desmet, and W. Joosen, ‘‘Advanced or not? A
comparative study of the use of anti-debugging and anti-vm techniques in
generic and targeted malware,’’ in IFIP Int. Conf. ICT Syst. Secur. Privacy
Protection. Springer, 2016, pp. 323–336.
[7] C. Collberg, C. Thomborson, and D. Low, ‘‘A taxonomy of obfuscating
transformations,’’ Dept. Comput. Sci., Univ. Auckland, Auckland, New
Zealand, Tech. Rep. 148, 1997.
[8] G.-R. Uh et al., ‘‘Analyzing dynamic binary instrumentation overhead,’’ in JAE HYUK SUK received the B.S. degree in elec-
Proc. WBIA Workshop ASPLOS, 2006.
trical and computer engineering from the Uni-
[9] Pin. Accessed: Aug. 13, 2020. [Online]. Available: https://software. versity of Seoul, Seoul, South Korea, in 2012,
intel.com/en-us/articles/pin-a-dynamic-binary-instrumentation-tool
and the M.S. degree in information security from
[10] Dynamorio. Accessed: Aug. 13, 2020. [Online]. Available: https://
Korea University, Seoul, in 2014, where he is cur-
dynamorio.org/
rently pursuing the Ph.D. degree in information
[11] Valgrind. Accessed: Aug. 13, 2020. [Online]. Available: https://valgrind.
security with the Graduate School of Informa-
org/
tion Security. His research interests include soft-
[12] M. Polino et al., ‘‘Measuring and defeating anti-instrumentation-equipped
ware protection, program obfuscation, program
malware,’’ in Proc. Int. Conf. Detection Intrusions Malware, Vulnerability
Assessment. Cham, Switzerland: Springer, 2017, pp. 73–96.
deobfuscation, reverse engineering, and malware
analysis.
[13] J. Park, Y.-H. Jang, S. Hong, and Y. Park, ‘‘Automatic detection and bypass-
ing of anti-debugging techniques for microsoft windows environments,’’
Adv. Electr. Comput. Eng., vol. 19, no. 2, pp. 23–29, 2019.
[14] B. Cheng, J. Ming, J. Fu, G. Peng, T. Chen, X. Zhang, and J.-Y. Marion,
‘‘Towards paving the way for large-scale windows malware analysis:
Generic binary unpacking with Orders-of-Magnitude performance boost,’’
in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Oct. 2018,
pp. 395–411.
[15] D. C. D’Elia, E. Coppa, F. Palmaro, and L. Cavallaro, ‘‘On the dissec-
tion of evasive malware,’’ IEEE Trans. Inf. Forensics Security, vol. 15,
pp. 2750–2765, 2020.
[16] S. Choi, T. Chang, C. Kim, and Y. Park, ‘‘X64Unpack: Hybrid emulation
DONG HOON LEE (Member, IEEE) received
unpacker for 64-bit windows environments and detailed analysis results on
VMProtect 3.4,’’ IEEE Access, vol. 8, pp. 127939–127953, 2020. the B.S. degree from Korea University, Seoul,
[17] P. Li, ‘‘Selecting and using virtualization solutions: Our experiences South Korea, in 1985, and the M.S. and Ph.D.
with VMware and virtualbox,’’ J. Comput. Sci. Colleges, vol. 25, no. 3, degrees in computer science from The University
pp. 11–17, 2010. of Oklahoma, Norman, OK, USA, in 1988 and
[18] D. T. Vojnak, B. S. Eordevic, V. V. Timcenko, and S. M. Strbac, ‘‘Per- 1992, respectively. Since 1993, he has been with
formance comparison of the type-2 hypervisor VirtualBox and VMWare the Faculty of Computer Science and Information
workstation,’’ in Proc. 27th Telecommun. Forum (TELFOR), Nov. 2019, Security, Korea University. He is currently a Pro-
pp. 1–4. fessor with the Graduate School of Information
[19] Themida. Accessed: Aug. 13, 2020. [Online]. Available: https://www. Security, Korea University. His research interests
oreans.com/themida.php include cryptographic protocol, applied cryptography, functional encryption,
[20] R. R. Branco, G. N. Barbosa, and P. D. Neto, ‘‘Scientific but not academ- software protection, mobile security, vehicle security, and ubiquitous sensor
ical overview of malware anti-debugging, anti-disassembly and anti-vm network security.
technologies,’’ Black Hat, vol. 1, pp. 1–27, Jul. 2012.

VOLUME 9, 2021 7673