electronics

Article
Fine-Grained Modeling of ROP Vulnerability Exploitation
Process under Stack Overflow Based on Petri Nets
Liumei Zhang 1, * , Wei Zhang 1 , Yichuan Wang 2,3 , Bowen Xia 1 and Yu Han 2

1 School of Computer Science, Xi’an Shiyou University, Xi’an 710065, China;

[email protected] (W.Z.); [email protected] (B.X.)
2 School of Computer Science and Engineering, Xi’an University of Technology, Xi’an 710048, China;
[email protected] (Y.W.); [email protected] (Y.H.)
3 Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an 710048, China
* Correspondence: [email protected]; Tel.: +86-180-9233-0186

Abstract: Software vulnerability discovery is currently a hot topic, and buffer overflow remains a
prevalent security vulnerability. One of the key issues in vulnerability discovery and analysis is
how to quickly analyze buffer overflow vulnerabilities and select critical exploitation paths. Existing
modeling methods for vulnerability exploitation cannot accurately reflect the fine-grained execution
process of stack overflow exploitation paths. This paper, based on the discussion of buffer overflow
exploitation techniques, proposes a fine-grained modeling and analysis method based on Petri nets
for the selection and execution of exploitation processes, specifically focusing on the return-oriented
programming in stack overflow. Through qualitative analysis, we compared the simulated time of the
software with the execution time of existing exploitation tools, achieving timeout-based simulation
experiments. We validated the model’s effectiveness using symbolic execution and dynamic analysis
techniques. The results indicate that this model performs well for vulnerable programs with Position
Independent Executable (PIE) protection enabled and has an advantage in selecting exploitation
paths, enabling timeout-based simulation. This method provides a reference for rapidly constructing
exploitation implementations.

Citation: Zhang, L.; Zhang, W.;

Keywords: ROP (return-oriented programming); Petri net; vulnerability exploitation; stack overflow
Wang, Y.; Xia, B.; Han, Y.
Fine-Grained Modeling of ROP
Vulnerability Exploitation Process
under Stack Overflow Based on Petri
Nets. Electronics 2023, 12, 4741.
1. Introduction
https://doi.org/10.3390/ Buffer overflow [1] is a common vulnerability found in various applications and op-
electronics12234741 erating systems [2]. Due to the lack of automatic buffer overflow detection instructions
in languages like C/C++ and the high time cost of manually writing code to check for
Academic Editors: Yun Zhou,
Zhaoyun Ding and Qianzhen Zhang
buffer overflow occurrences in real time, these vulnerabilities are easily exploitable by
attackers. Although fuzz testing techniques helped us automatically discover the pro-
Received: 23 October 2023 gram vulnerabilities [3], and parallel fuzz testing platforms efficiently uncover buffer
Revised: 17 November 2023 overflow vulnerabilities in programs, both defenders and attackers are more concerned
Accepted: 21 November 2023 about whether these program vulnerabilities or errors can be exploited. Rapidly analyzing
Published: 22 November 2023 buffer overflow vulnerabilities and selecting critical paths for exploitation is one of the
key issues in current vulnerability discovery and analysis. Petri nets are modeling and
analysis tools with a strict mathematical definition and powerful graphical representation
Copyright: © 2023 by the authors.
capabilities [4]. They can simulate processes that describe the static structure and micro-
Licensee MDPI, Basel, Switzerland. dynamic behavior. While there has been extensive research on buffer overflow in the past,
This article is an open access article these studies did not delve into the micro-level processes of stack overflow exploitation,
distributed under the terms and and they failed to provide an in-depth and detailed analysis of vulnerability mechanisms
conditions of the Creative Commons that reflect the essence of exploitation. Therefore, this paper discusses the use of Petri nets
Attribution (CC BY) license (https:// to model the process of buffer overflow exploitation, analyzing factors such as register
creativecommons.org/licenses/by/ states and memory information relevant to buffer overflow exploitation. This establishes a
4.0/). fine-grained Petri net model for vulnerability exploitation in stack overflows, specifically

Electronics 2023, 12, 4741. https://doi.org/10.3390/electronics12234741 https://www.mdpi.com/journal/electronics

Electronics 2023, 12, 4741 2 of 15

focusing on Return-Oriented Programming (ROP) exploitation [5]. The paper also describes
the path selection method during vulnerability exploitation, effectively enabling buffer
overflow exploitation, and enhancing research on vulnerability discovery and software
vulnerability exploitability issues.
In this study, our objective is to achieve the analysis and rapid construction of vulnera-
bility exploitation by establishing Petri nets on the ROP vulnerability exploitation process.
The main contributions are as follows:
(1) The Petri nets model for ROP vulnerability exploitation under stack overflow is
proposed. In comparison to other studies, this paper delves into the fundamental
principles of stack overflow and provides a detailed description of the process of
constructing vulnerability exploitation using ROP techniques for programs with stack
overflow. It employs Petri nets to model the payload construction process, offering a
fine-grained description of the exploitation process.
(2) The model is compared with the existing vulnerability exploitation tools, facilitating
real-time simulation experiments. The average time for 10 sets of experiments is
calculated for two ROP vulnerability exploitation methods, and it is found that this
model takes less time and has an advantage in path selection.
(3) Based on the model’s approach to vulnerability exploitation, the model’s effective-
ness is verified using symbolic execution and dynamic analysis. The results indicate
that this model performs well for vulnerable programs with PIE protection enabled
and provides a reference method for rapidly constructing vulnerability exploita-
tion implementations.
The remaining parts of this paper are organized as follows. Section 2 presents Petri
nets and related work on buffer overflow exploitation. Section 3 introduces the Petri net
modeling of ROP buffer overflow exploitation. Section 4 provides a detailed analysis of
the model. Section 5 presents an analysis of experimental results for the model. Section 6
summarizes the relevant research in this paper.

2. Related Work
Buffer overflow attacks are initiated by overwriting the return address saved in the
function stack frame of the called function that redirects the execution to arbitrary code
injected by the attacker. Stack overflow is one of the most common vulnerabilities in
buffer overflow. Aleph One [6] detailed the architecture of the Linux stack in 1996 and
proposed a method for exploiting buffer overflow vulnerabilities based on the stack to gain
control of the program’s control flow. Attackers can remotely gain administrative privileges
through this vulnerability and then execute malicious instructions with those privileges.
Buffer overflow has been identified as one of the most common and dangerous security
vulnerabilities to date [7]. Many detection and mitigation techniques have been developed
and deployed to reduce the likelihood of buffer overflow attacks [8–10]. Li, S et al. [11]
proposed a machine learning-based buffer overflow detection method, which demonstrates
good accuracy in detecting remote buffer overflow attacks. Piromsopa, K [12] introduced
a framework to prevent buffer overflow attacks and created an effective buffer overflow
prevention tool using this framework. Ozdoganoglu et al. [13] introduced Smashguard, a
hardware stack implemented in the Central Processing Unit to protect return addresses, first
implemented in the Alpha architecture. No-Execute (NX) [14] and Address Space Layout
Randomization (ASLR) [15] at the hardware and operating system levels also provide some
mitigation for buffer overflow vulnerabilities. Although current detection and mitigation
techniques help us address the discovery of program vulnerabilities [1,16–19], these miti-
gation measures are not foolproof. Return-oriented Programming (ROP) [20] attacks can
effectively bypass NX. ROP is an advanced form of code reuse attack that achieves control
flow bypassing protection mechanisms by using pre-existing small instruction sequences.
Therefore, research on ROP exploitation under buffer overflow is of great significance.
Vulnerability exploitation refers to leveraging security flaws within software to achieve
objectives that cannot be attained through normal software operation. The primary method
Electronics 2023, 12, 4741 3 of 15

for exploiting buffer overflow vulnerabilities relies on the absence of boundary checks
for reading and writing data in C and C++ programming languages. This allows data
placement outside the boundaries of a buffer, potentially overwriting the return address
of a program function, thereby hijacking control flow and leading to program crashes or
gaining system control privileges. D. Brumley [21] and others introduced an Automatic
Exploit Generation (APEG) method based on binary patch comparison. Its core idea
involves adding filtering conditions to patched programs to prevent triggering the original
program’s crashes. For example, a patch might increase the length of a buffer to fix a
buffer overflow vulnerability. A D. Federico [22] and colleagues proposed a vulnerability
exploitation technique capable of bypassing NX (No-Execute) and ASLR (Address Space
Layout Randomization) protections. This technique exploits the dynamic linking process
of ELF executable files to overcome complex security mechanisms. T. Avgerinos [23]
and his team introduced an effective method for automated vulnerability discovery and
exploitation called AEG. The core concept of this method is to use program verification
techniques to identify inputs that lead the program into an insecure state that can be
exploited. Such insecure states may involve memory overflows, malicious format string
usage, and more. Various AEG solutions have emerged for different vulnerabilities and
exploitation methods [24,25]. This research extends beyond the software layer, delving
deep into the Linux kernel to enhance security [26–29].
Existing research on vulnerability exploitation can efficiently discover a large num-
ber of program errors, and the description of exploitation methods has become more
refined [30–32]. However, current research lacks fine-grained modeling and analysis of
vulnerability exploitation scenarios, overlooks details in attack scenarios, and has vague
path selection analysis for exploitation. Most approaches directly explore program paths,
making it challenging to address the problem of path explosion.
The control flow and sensitive data of a program can be disrupted by successful
buffer overflow exploitation. This paper models the micro-level process of Return-Oriented
Programming (ROP) vulnerability exploitation using Petri nets, with states and operations
as the basic granularity. A Petri net model for stack overflow exploitation is established,
addressing the issue of traditional exploitation method models being too macroscopic and
providing a detailed description of the micro-level processes of stack overflow vulnerabili-
ties. By exploring the path selection method for vulnerability exploitation in this Petri net
model, efficient identification of exploitable vulnerabilities in programs can be achieved,
offering a more fine-grained representation for enhancing the understanding of the buffer
overflow exploitation process.

3. Petri Net Model for Stack Overflow Vulnerability Exploitation

In this section, we describe a Petri net model for vulnerability exploitation based on
stack overflow. The Petri net model for stack overflow automatic vulnerability exploitation
is established based on the Return to C Library (Ret to libc) [33] and Return to C Start-Up
(Ret to csu) [34] vulnerability exploitation methods. Through a fine-grained analysis of the
exploitation process, states and behaviors in the exploitation process are mapped one to one
to the places and transitions of the Petri net, creating a model that reflects the stack overflow
exploitation process. This model detects stack buffer overflows by checking whether the EIP
(Extended Instruction Pointer) register is contaminated and collects relevant information
during runtime to automatically generate the vulnerability exploitation method paths.

3.1. Petri Net Modeling Rules

Petri nets are directed graphs used to describe relationships between events and
conditions. They consist of places, transitions, and directed arcs. Places are represented
by circles to describe possible local states of the system. Transitions are represented by
rectangles to describe events that modify the system’s state. Directed arcs connect from
Place nodes to Transition nodes or from Transition nodes to Place nodes, representing
connections between Places and Transitions. The following is the definition of Petri nets.
Electronics 2023, 12, 4741 4 of 15

Definition 1. A triple PN = (P, T, F) is called a Petri net if it satisfies the following conditions:
• P is a finite set of places, T is a finite set of transitions;
• Where P 6= ∅ , T 6= ∅ , P ∩ T = ∅;
• F = ( P × T ) ∩ ( T × P) represents the flow relationship of PN.

The mapping relationship between the established Petri net model and automatic
vulnerability exploitation is shown in Table 1.

Table 1. Petri Net Rules.

Petri Elements Automatic Vulnerability Exploitation

Place States
Transition Events
Direct arc State Transition Process

In formal description and modeling, states primarily include the state of return address
contamination, stack protection status, the Ret to libc exploitability status, library function
offset address acquisition status, and shellcode execution capability status. Operations
primarily encompass dynamic analysis, overwriting return addresses, and invoking a shell,
among other behaviors.
In this study, places are used to represent various system states or states within the
stack in the model. For example, states like library function offset address acquisition,
exploitation construction in progress, and stack protection enabled are represented as places.
Transitions are used to indicate operations during the construction of the vulnerability
exploitation. For instance, operations like reading the offset address and overwriting
the return address are indicated by transitions. Transitions in the model possess the
characteristic of describing fine-grained execution operations in the automatic vulnerability
exploitation process. Therefore, aspects related to states in various vulnerability exploitation
operation scenarios are represented by places, while aspects related to operations are
represented by transitions.
Within places, token values can be set based on the scenario of the actual vulnerability
exploitation process. Token values represent the quantity of resources available in the
model system or the extent of information leakage. These values are determined based on
the specific exploitation scenario. According to the rules of the Petri net, when the model is
running, a transition can occur only if all the places corresponding to that transition have
tokens and the number of tokens satisfies the weight of the directed arcs. Token values
decrease in the group of places preceding the transition according to the weight of the
directed arcs, and token values increase in the group of places following the transition
according to the weight of the directed arcs.

3.2. Petri Net Modeling for Vulnerability Exploitation

The process of constructing a stack overflow vulnerability exploitation can be divided
into two main parts: accessing information within the stack and constructing the payload.
According to the Petri net modeling rules, the entire system process is divided into sets
of places and transitions, and an automatic vulnerability exploitation Petri net model is
established. In this model, the main places include the state of overwriting the return
address in the stack, the exploitability state of the vulnerability, various states related to
vulnerability construction, and the state of acquiring library function offset addresses.
These states are represented by circles in the Petri net model. States such as the stack-based
return address overwrite and the stack protection enabled represent information access
within the stack, while states related to vulnerability construction and library function
offset address leakage represent payload construction.
The main transitions include operations for retrieving information from the stack,
reading function offset addresses, and overwriting return addresses. These operations are
Electronics 2023, 12, 4741 5 of 15

represented by squares in the Petri net model. Specific token values can be set at the initial
state, and in the system, the weight of directed arcs represents the number of times an
action is performed and the quantity of resources.
According to the Petri net modeling rules proposed in this paper, the Petri net model
for stack overflow automatic vulnerability exploitation is shown in Figure 1.

T8 P10

T6
P1

P9 T7
T1 P4

P0 T0 P2 P6 T4 P15

T2 P5 T3 P7 T10 P12

P3
P8 T5 P11 T9 P13

T11 P14 T12

Figure 1. Stack Overflow Automatic Vulnerability Exploitation Model.

To address the selection of specific and appropriate exploitation methods during

vulnerability exploitation, various branches are used in the model described above. The
start and end states of each stage or substage of the construction are indicated using place
nodes. The meanings of places in the vulnerability exploitation model are as shown in
Table 2.

Table 2. Meaning of Place in Vulnerability Exploitation Petri Net.

Place Description
P0 Initial state
P1 PIE contamination state
P2 State Transition Process
P3 Stack Protection Disabled State
P4 Other Vulnerability Exploitation States
P5 Vulnerability Exploitation State
P6 Ret to libc construction
P7 Exploitation Construction State
P8 Ret to csu Construction
P9 Ret to libc Vulnerability Point Exploitation Information Acquisition State
P10 libc Library Function Offset Address Leakage State
P11 Ret to csu Vulnerability Point Exploitation Information Acquisition State
P12 write Function Offset Address Leakage State
P13 Library Function Offset Address Acquisition State
P14 BSS Segment Exploitable Vulnerability State
P15 Shell Establishment Exploitable State

This model can finely describe various states and changes during the construction of
different vulnerability exploitation processes related to stack overflow. From the model in
Figure 1, it can be observed that P6 and P8 represent two different states of vulnerability
exploitation construction, mainly resulting in root privilege leakage to unauthorized users
through the acquisition of library function offset addresses, return address contamination,
Electronics 2023, 12, 4741 6 of 15

and the construction of the system (/bin/sh) function. P4 represents the state of a failed
stack overflow vulnerability exploitation, while P15 represents the state of a successfully
established shell, indicating the successful completion of the exploitation.
In the vulnerability exploitation model, the meanings of transitions are as shown in
Table 3.

Table 3. Meaning of Transitions in Vulnerability Exploitation Petri Net.

Transition Description
T0 Symbolic execution
T1 Return-oriented exploitation construction
T2 Exploitation construction
T3 Dynamic analysis for path selection
T4 Query initial address of got-puts
T5 Acquire csu return address
T6 Acquire got-puts offset address using the puts function
T7 Read system offset address and bin/sh using the read function
T8 Leak libc library offset address
T9 Overwrite return address to hijack control flow to libc-csu-init
T10 Acquire libc library offset address
T11 Read function reads the system offset address and bin/sh
T12 Exploit using system /bin/sh

From the model perspective, Transition T3 is the dynamic analysis path selection
operation and a crucial step in constructing suitable vulnerability exploitation. Transitions
T7 and T12 represent the operations of invoking the shell using the system function,
indicating the successful exploitation of the stack overflow vulnerability. P9 in the model is
the state required when enabling the Ret to libc exploitation method, while P11 and P14
are the states required when enabling the Ret to csu exploitation method. The quantity of
tokens set for P0 indicates the number of times the model selects vulnerability exploitation.

4. Model Analysis
This section analyzes how to construct vulnerability exploitation scenarios in the
presence of stack overflow vulnerabilities in a program. It explores the construction of
suitable vulnerability exploitation methods, such as Ret to libc exploitation, and discusses
the path selection during construction. It provides an in-depth explanation of the Petri net
model for stack overflow vulnerability exploitation, describing the meanings represented
by Places and Transitions in the model. It maps the process of constructing payloads for
stack overflow scenarios to the Petri net model of vulnerability exploitation to illustrate the
effectiveness of the model. Finally, it explains how to set token values in Places and the
meanings of some of the weights on directed arcs.

4.1. Ret to Libc Method Model Analysis

Ret to libc is a major attack technique primarily targeting dynamically linked programs.
When a program is dynamically linked, the libc library functions are loaded, and these
library functions offer a wide range of system functionalities for program use. Therefore, it
is possible to obtain control of the target program by searching for usable system function
addresses in memory and combining stack overflow to overwrite the function’s return
address. This model divides the details of the program’s control flow hijacking process
in the context of stack overflow vulnerability exploitation and sets a Place node at the
beginning and end of each stage or substage to mark its start and end states. The stack’s
internal structure for the Ret to libc method is shown in Figure 2.
Electronics 2023, 12, 4741 7 of 15

First-time Second-time
Unused padding data Unused padding data
Overflow
“a”× n “a”× n
(n is the buffer size) (n is the buffer size)

puts-plt
Return system-adds
address

Return address of puts Start-addr “a”× 4 Return address of system

Output content of puts puts-got /bin/sh-addr Arguments of system

Figure 2. Stack Internal Structure of the Ret to libc Method.

When conducting vulnerability exploitation attacks on a target program with a stack

overflow vulnerability using the Ret to libc method, obtaining stack frame information and
library function address offsets is essential to prepare for further payload construction. In
Figure 2, vulnerability exploitation is achieved through two stages of payload construction.
First, by overflowing the stack and overwriting the return address, control flow is hijacked
to the “puts” function, with “puts-got” set as the parameter. This allows the retrieval of the
actual address of the “puts” function from the “got” table, calculating the leaked library
function offset address. After obtaining the necessary information for exploitation, the
return address is overwritten to point to “system-addr”, and the address of “/bin/sh” string
is placed on the stack. The program is then hijacked to the "system" function, ultimately
leading to the acquisition of a shell.
The Petri net model for stack overflow vulnerability exploitation proposed in this
paper is abstracted from the vulnerability exploitation methods of ROP, Ret to libc and Ret
to csu. It is particularly well suited for the detection and generation of vulnerabilities in the
context of stack overflow.

4.2. Vulnerability Exploitation Path Selection Analysis

The path selection analysis process primarily involves two components: path selection
and branch acquisition. The goal is to reproduce the crash path, capture the program’s state
when it crashes, and obtain information such as path constraints and memory snapshots.
With PIE and NX enabled in the stack space, and considering that the base address
of the libc library is random, the model utilizes symbolic execution methods for the path
selection in vulnerability exploitation. It aims to detect input that triggers stack buffer
overflow vulnerabilities and employs dynamic analysis methods for the dynamic execution
and analysis of base address offsets in binary programs.
First, the path selection program assesses whether stack protection is enabled. It
determines the status of PIE and NX protection and selects the execution path for Return-
oriented Programming (ROP) attacks. After obtaining addresses for library functions and
character sets, the “puts” function activates the Ret to libc vulnerability exploitation path,
as depicted in Figure 3 and Table 4.

Table 4. Table of Place and Transition variations of the Petri model.

Place Description Transition Description

P0 Ret to libc construction begins T0 Obtaining Vulnerability Point Information
Ret to libc vulnerability point information
P1 T1 puts function retrieves the got-puts offset address
retrieval state
P2 Libc Library Function Offset Address Leak State T2 Leaking libc library offset address
P3 Establishing a Shell-Exploitable State T3 call a shell
Electronics 2023, 12, 4741 8 of 15

T2

P0 T0 P1 T1 P2

P3
T3

Figure 3. Ret to libc Model.

Formalizing the path selection of Ret to libc in a Petri net abstraction, after obtaining
token resources at the Place P0 node, Transition T0 has the authority to fire. Following
the occurrence of behavior in T0, state P1 is triggered. However, at this point, since the
library function address offset cannot be obtained, it is necessary to execute behavior in
T1, constructing the first payload. After obtaining the function address offset, state P2 has
token resources. By constructing the payload once again, Transition T3 gains the authority
to fire, ultimately activating state P3 to call the shell and acquire program control flow. The
payloads constructed in two iterations are shown in Table 5.

Table 5. Construction of Payloads for Ret to libc.

Ret to Libc Attack Code

Payload1 ‘a’*n + p64 (puts_plt) + p64 (start_addr) + p64 (put_got)
Payload2 ‘a’*n + p64 (system_addr) + P64 (‘a’*8) + p64 (bin/sh_addr)

5. Experimental Analysis
The experimental environment configured for implementing the model in this paper is
as follows: Operating System: Ubuntu 22.04, AMD R7-5800H CPU 64-bit operating system;
Memory: 16 GB; Modeling tools and runtime environment: Tina v3.6.0, angr v9.0.10576
symbolic execution engine [35], and radare2 v5.62 dynamic analysis module [36]. Target
programs: CTF and CVE programs with stack buffer overflow vulnerabilities. Comparison
tool: Zeratool vulnerability exploitation tool [37].

5.1. Model Analysis

In this section, this article provides statistics for the time taken to reach the first
successful exploitation state in Petri net models, the time taken to reach the first Ret to libc
vulnerability exploitation state, and the time taken to reach the first Ret to csu vulnerability
exploitation state. It also includes the actual time for successful exploitation using the
Zeratool tool. Zeratool is a Python script that takes a binary file as a parameter and
focuses on exploiting buffer overflow and format string vulnerabilities. Table 6 presents
the corresponding experimental system environments, simulation software, and system
version for the verification experiments.

Table 6. The environment of the experiment.

Environment Configuration
OS Windows10
CPU AMD® Ryzen™ 7-5800H
Tina tools 3.7.0
Python 3.10.12
Libc 2.35-0ubuntu3-amd64
Electronics 2023, 12, 4741 9 of 15

According to the Petri net model in the third section, a model was established and
simulation experiments were conducted using the Tina tool [38]. The representation of
reaching the vulnerability exploitation state through the Ret to libc exploitation method in
the Petri net model is shown in Figure 4. The value of P0 represents the number of attempts
to exploit the vulnerability in the target program. In the experiment, the token value of
P0 was set to eight. Colored transitions indicate states waiting for resources to become
available. By selecting the exploitation method, a payload is constructed and executed.
When T0 becomes active, P2 and P3 can acquire tokens, representing the acquisition of the
EIP register status and the execution of the ROP exploitation method. Resources flow to
P5, allowing T3 acquisition of tokens for path selection, obtaining memory status, function
address offsets, and other information. After P9 acquires resources, both T6 and T7 become
active, executing the Ret to libc exploitation method path. In the end, P9 and P10 jointly
possess the necessary resources, completing the payload construction. The payload is
injected into the target program, and upon execution of T7, resources flow to P15, granting
control over the program, thus resulting in a successful vulnerability exploitation.

T8 P10

T6
P1

P9 T7
T1 P4

P0 T0 P2 P6 T4 P15

T2 P5 T3 P7 T10 P12

P3
P8 T5 P11 T9 P13

T11 P14 T12

Figure 4. Execution path of the Ret to libc exploitation method.

In the Petri net model diagram, the number of tokens possessed by all Places is
represented by the quantity of black dots. Transitions with a red color indicate transitions
with the right to occur. Throughout the entire model, transitions are designed to be
executed randomly, so the time it takes to reach the successful exploitation state P15 using
the exploitation method is random with each attempt. Similarly, P0’s token value is set to
eight. Figure 5 illustrates the selection of the Ret to csu exploitation method to reach the
exploitation state.

T8 P10

T6
P1

P9 T7
T1 P4

P0 T0 P2 P6 T4 P15

T2 P5 T3 P7 T10 P12

P3
P8 T5 P11 T9 P13

T11 P14 T12

Figure 5. Execution path of the Ret to csu exploitation method.

Electronics 2023, 12, 4741 10 of 15

The duration for which tokens are emitted in the model can be manually adjusted.
The triggering duration is expressed as the rate of token propagation within the model.
The faster the tokens flow, the shorter the duration. In our experiments, we set the trig-
gering duration for tokens to 1 s and enabled the random, unpredictable timing option.
Figure 6 displays ten sets of experiments. The average time for reaching the exploitation
state for both the Ret to libc method and the Ret to csu method pathways is 24.2 s and
29.3 s, respectively. This article also uses the Zeratool vulnerability exploitation tool to
simulate the time required for vulnerability exploitation in stack overflow scenarios. The
simulated programs used are “downunderctf2020_return” and “utctf2021_resolve”. The
time required for both symbolic execution and dynamic analysis processes is calculated,
including the time needed to compute path constraints. Figure 7 presents data from eight
sets of experiments, with average times of 33.4 s and 36.6 s, respectively. The experimental
results indicate that the runtime of Petri net vulnerability exploitation simulation proposed
in this article is shorter than the runtime of existing vulnerability exploitation tools using
Zeratool. The simulation environment allows for near real-time simulation.

20.5
10 32.1
18.5
9 17.2
7.5
8 21.2
35.2
7 43.6
52.5
6 38.2
18.7
5 55.1
20.2
4 8.5
40.1
3 27.5
16.3
2 18.9
12.9
1 30.7

0 20 40 60
Ret to csu Ret to libc
Time(s)
Figure 6. Simulation Time Statistics for Petri Net Models.
Electronics 2023, 12, 4741 11 of 15

39.1
10 32.2
30.5
9 35.2
42.1
8 33.7
35.6
7 33.7
38.2
6 32.1
39.3
5 32.1
32.1
4 40.5
38.2
3 31.6
35.7
2 31.5
35.6
1 32.2

0 10 20 30 40 50
downunderctf2020_return utctf2021_resolve
Time(s)
Figure 7. Simulation Time Statistics for Zeratool.

5.2. Comparative Analysis of Experiments

Based on the abstract concept of the stack overflow automatic vulnerability exploita-
tion model in Sections 3 and 4, a program is written in the Python language to validate
the effectiveness of this model. The experiments are conducted on a 64-bit Ubuntu 22.04
system under VMware. Symbolic execution is performed using the angr framework [39],
and dynamic analysis is carried out using radare2 [40]. The experiments focus on CTF
(Capture The Flag) programs with stack buffer overflow vulnerabilities, most of which
can be found on CTFTime [41]. CTF is a cybersecurity competition that is used to test and
develop computer security skills. CTF programs can be considered simplified versions of
real-world programs and are used to demonstrate the principles of vulnerabilities more
concisely. Since CTF programs are designed to test participants’ skills in competitions, their
vulnerabilities can be exploited.
Based on the established ROP vulnerability exploitation Petri net model, whether the
program is protected by PIE (Position-Independent Executable) is used as a selection crite-
rion to demonstrate the effectiveness of this model in selecting and executing vulnerability
exploitation methods as efficiently as possible. The experiments are conducted in environ-
ments with ASLR (Address Space Layout Randomization) and NX (No-Execute) protection
enabled [5]. Table 7 shows the results of this model on 10 CTF programs. This model can
automatically select paths and construct vulnerabilities efficiently and effectively.
Electronics 2023, 12, 4741 12 of 15

Table 7. Results of the Petri Model Program on CTF.

Program NX ASLR PIE Rop Branch

√ √
downunderctf2021_deadcode √ × √ Ret to libc
csictf2020_pwn0x1 √ ×
√ √ Ret to libc
csictf2020_pwn0x2 √ √ √ Ret to libc
csictf2020_pwn0x3 √ √ √ Ret to libc
dctf2021_babybof √ √ √ N/A
utctf2021_resolve √ √ √ Ret to csu
dicectf2021_babyrop √ √ √ Ret to csu
nahamconctf2021_smol √ √ Ret to csu
redpwnctf2020_coffer √ √ × Other
downunderctf2021_out × Other

For programs with PIE (Position-Independent Executable) protection enabled, this

model can effectively implement Return-Oriented Programming (ROP) exploitation tech-
niques and choose different vulnerability exploitation paths. However, in programs that
do not have PIE protection enabled, the model opts for executing other vulnerability
exploitation techniques, such as “Ret to Shellcode”.
Table 8 provides a fine-grained qualitative analysis and comparison of several typical
vulnerability exploitation methods. Liu, Z et al. [42] proposed a vulnerability exploitation
generation framework that uses a crash reproduction algorithm to mitigate the path ex-
plosion problem. Biswas et al. [43] introduced a two-level branch predictor data structure
reuse technique for buffer overflow or return-oriented programming, but it did not involve
code analysis. Bu, W et al. [44] proposed a method to monitor the program execution
process through dynamic binary instrumentation, analyzing the buffer overflow vulnera-
bility exploitation process and achieving transplantation and utilization of vulnerability
samples. However, this method lacks in-depth analysis of key information such as program
loading addresses. Overall, existing research has not provided a fine-grained description
and in-depth analysis of ROP vulnerability exploitation processes under stack overflow.
This paper, exemplifying the Ret to libc exploitation method under ROP vulnerability
exploitation, achieves the mapping of path selection and payload construction to a fine-
grained formalized model. It extends the analysis of exploitation details, addressing the
shortcomings of existing methods.

Table 8. Qualitative Analysis Table.

Automatic Software Formal

Modeling Method Code Level Clear State
Operation Simulation Analysis
√ √
Liu, Z et al. [42] ×
√ × ×
Biswas et al. [43] ×
√ √ ×
√ × ×
Bu, W et al. [44] √ √ √ ×
√ ×
√
This article

6. Conclusions
In this paper, we consider the stack buffer overflow vulnerability, which occurs when
the number of bytes written by a program to a buffer variable on the stack exceeds the
buffer’s requested size. This can lead to the overwriting of a function’s return address,
thereby hijacking the program’s control flow. Based on this characteristic and according to
the modeling rules defined in this paper, we establish a fine-grained Petri nets model for
ROP vulnerability exploitation. Recording exploit path selection and branch acquisition,
we conduct timeout-based simulation experiments, qualitatively analyzing and separately
calculating the simulation time of the software and the execution time of existing exploit
tools. Because it does not involve the processing of actual data, the model operates at a
faster speed. Consequently, the results of the model’s execution can serve as a theoretical
Electronics 2023, 12, 4741 13 of 15

decision reference for constructing practical exploit implementations. Through the fine-
grained partitioning of the manipulation of function stack frames and state changes, the
model can provide a theoretical reference for building Petri net models in the context of
exploiting vulnerabilities using other methods. Using symbolic execution and dynamic
analysis techniques to implement simulation experiments, the experimental results indicate
that the model performs well in exploiting programs with PIE protection enabled and
demonstrates advantages in exploit path selection. However, for programs that do not have
PIE protection enabled and programs where obtaining library function offset addresses is
not possible, there are limitations to the vulnerability exploitation method of this model.
Model deadlock issues may be encountered when choosing to execute the Ret to CSU
path, and inevitable path explosion problems may arise when using symbolic execution
to verify the model. In the future, we plan to extend our research to programs without
PIE protection and further optimize path selection based on this Petri nets model. We
aim to address the issue of path explosion in symbolic execution and implement more
exploitation methods, thus providing a more comprehensive and systematic approach to
program vulnerability exploitation.

Author Contributions: Conceptualization, W.Z. and L.Z.; methodology, Y.W.; software, W.Z.; valida-
tion, B.X. and W.Z.; formal analysis, W.Z.; investigation, W.Z. and Y.H.; resources, Y.W. and L.Z.; data
curation, W.Z., L.Z. and B.X.; writing—original draft preparation, W.Z.; writing—review and editing,
W.Z. and Y.W.; supervision, W.Z.; project administration, L.Z.; funding acquisition, W.Z., L.Z. and
Y.W. All authors have read and agreed to the published version of the manuscript.
Funding: This research work is supported by the National Natural Science Founds of China (62072368,
U20B2050), the Key Research and Development Program of Shaanxi Province (2021ZDLGY05-09,
2022CGKC-09), the Open Project Funds of Shaanxi Key Laboratory for Network Computing and
Security Technology (NCST2021YB-04), and Natural Science Basic Research Program of Shaanxi
Province (2023-JC-QN-0742).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data are contained within the article.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Cowan, C.; Wagle, F.; Pu, C.; Beattie, S.; Walpole, J. Buffer overflows: Attacks and defenses for the vulnerability of the decade.
In Proceedings of the DARPA Information Survivability Conference and Exposition. DISCEX’00, Hilton Head, SC, USA, 25–27
January 2000; Volume 2, pp. 119–129.
2. Ahmad, D. The rising threat of vulnerabilities due to integer errors. IEEE Secur. Priv. 2003, 1, 77–82. [CrossRef]
3. Elwan, E.M. Automatic generation of control, flow hijacking exploits for software vulnerabilities. Int. J. Cult. Inherit. Soc. Sci.
2023, 5, 22–82.
4. Murata, T. Petri nets: Properties, analysis and applications. Proc. IEEE 1989, 77, 541–580. [CrossRef]
5. Schwartz, E.J.; Avgerinos, T.; Brumley, D. Q: Exploit hardening made easy. In Proceedings of the 20th USENIX Security
Symposium (USENIX Security 11), San Francisco, CA, USA, 8–12 August 2011.
6. One, A. Smashing the stack for fun and profit. Phrack Mag. 1996, 7, 14–16.
7. Tobah, Y.; Kwong, A.; Kang, I.; Genkin, D.; Shin, K.G. SpecHammer: Combining spectre and Rowhammer for new speculative
attacks. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 22–26 May 2022;
pp. 681–698.
8. Youssef, A.; Abdelrazek, M.; Karmakar, C. Use of Ensemble Learning to Detect Buffer Overflow Exploitation. IEEE Access 2023,
11, 52009–52025. [CrossRef]
9. Li, Y.; Chung, Y.C.; Bao, Y.; Lu, Y.; Guo, S.; Lin, G. KPointer: Keep the code pointers on the stack point to the right code. Comput.
Secur. 2022, 120, 102781. [CrossRef]
10. Li, W.; Liu, Z. Stack Data Protection Mechanism for LLVM Intermediate Representation. In Proceedings of the 2023 5th
International Conference on Communications, Information System and Computer Engineering (CISCE), Guangzhou, China,
14–16 April 2023; pp. 240–244.
Electronics 2023, 12, 4741 14 of 15

11. Li, S.; Zheng, R.; Zhou, A.; Liu, L. A Machine Learning-Based Method for Detecting Buffer Overflow Attack with High Accuracy.
In Proceedings of the 2020 International Conference on Computer, Network, Communication and Information Systems (CNCI
2020), Qingdao, China, 27–29 March 2020.
12. Piromsopa, K.; Enbody, R.J. Buffer-Overflow Protection: The Theory. In Proceedings of the 2006 IEEE International Conference
on Electro/Information Technology, Berkeley, CA, USA, 25–26 May 2006.
13. Ozdoganoglu, H.; Vijaykumar, T.; Brodley, C.; Kuperman, B.; Jalote, A. SmashGuard: A Hardware Solution to Prevent Security
Attacks on the Function Return Address. IEEE Trans. Comput. 2006, 55, 1271–1285. [CrossRef]
14. Xu, S.; Sandhu, R.; White, G.; Winsborough, W.; Korkmaz, T. Protecting Cryptographic Keys and Functions from Malware
Attacks. 2010. Available online: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.298.8685&rep=rep1&type=pdf
(accessed on 10 June 2023).
15. Marco-Gisbert, H.; Ripoll Ripoll, I. Address space layout randomization next generation. Appl. Sci. 2019, 9, 2928. [CrossRef]
16. Rani, P.; Jain, S.G. Buffer Overflow: Proof of Concept Implementation. Ph.D. Thesis, 2014. Available online: http://hdl.handle.
net/10266/3149 (accessed on 10 June 2023).
17. Butt, M.A.; Ajmal, Z.; Khan, Z.I.; Idrees, M.; Javed, Y. An in-depth survey of bypassing buffer overflow mitigation techniques.
Appl. Sci. 2022, 12, 6702. [CrossRef]
18. Alzahrani, S.M. Buffer Overflow Attack and Defense Techniques. Int. J. Comput. Sci. Netw. Secur. 2021, 21, 207–212.
19. ALHusayn, S.M.S. The Buffer Overflow Attack and How to Solve Buffer Overflow in Recent Research. AJRSP J. 2020, 2, 1–13.
20. Xu, S.; Wang, Y. Defending against Return-Oriented Programming attacks based on return instruction using static analysis and
binary patch techniques. Sci. Comput. Program. 2022, 217, 102768. [CrossRef]
21. Brumley, D.; Poosankam, P.; Song, D.; Zheng, J. Automatic patch-based exploit generation is possible: Techniques and implications.
In Proceedings of the 2008 IEEE Symposium on Security and Privacy (SP 2008), Oakland, CA, USA, 18–22 May 2008; pp. 143–157.
22. Di Federico, A.; Cama, A.; Shoshitaishvili, Y.; Kruegel, C.; Vigna, G. How the ELF Ruined Christmas. In Proceedings of the 24th
USENIX Security Symposium (USENIX Security 15), Washington, DC, USA, 12–14 August 2015; pp. 643–658.
23. Avgerinos, T.; Cha, S.K.; Rebert, A.; Schwartz, E.J.; Woo, M.; Brumley, D. Automatic exploit generation. Commun. ACM 2014,
57, 74–84. [CrossRef]
24. Xu, S.; Wang, Y. BofAEG: Automated Stack Buffer Overflow Vulnerability Detection and Exploit Generation Based on Symbolic
Execution and Dynamic Analysis. Secur. Commun. Netw. 2022, 2022, 1251987. [CrossRef]
25. Nicula, S, .; Zota, R.D. Exploiting stack-based buffer overflow using modern day techniques. Procedia Comput. Sci. 2019, 160, 9–14.
[CrossRef]
26. Chen, W.; Zou, X.; Li, G.; Qian, Z. {KOOBE}: Towards facilitating exploit generation of kernel {Out-Of-Bounds} write vulnerabili-
ties. In Proceedings of the 29th USENIX Security Symposium (USENIX Security 20), 12–14 August 2020; pp. 1093–1110. Available
online: https://www.usenix.org/conference/usenixsecurity20/presentation/chen-weiteng (accessed on 15 June 2023).
27. Chen, Y.; Lin, Z.; Xing, X. A Systematic Study of Elastic Objects in Kernel Exploitation. In Proceedings of the CCS ’20: 2020 ACM
SIGSAC Conference on Computer and Communications Security, Virtual, 9–13 November 2020.
28. Chen, Y.; Xing, X. Slake: Facilitating slab manipulation for exploiting vulnerabilities in the linux kernel. In Proceedings of the
2019 ACM SIGSAC Conference on Computer and Communications Security, London, UK, 11–15 November 2019; pp. 1707–1722.
29. Wu, W.; Chen, Y.; Xing, X.; Zou, W. {KEPLER}: Facilitating control-flow hijacking primitive evaluation for Linux kernel
vulnerabilities. In Proceedings of the 28th USENIX Security Symposium (USENIX Security 19), Santa Clara, CA, USA, 14–16
August 2019; pp. 1187–1204.
30. Li, Y.; Liu, M.; Cao, C.; Li, J. Communication-Traffic-Assisted Mining and Exploitation of Buffer Overflow Vulnerabilities in
ADASs. Future Internet 2023, 15, 185. [CrossRef]
31. Shafana, N.J.; Pawar, K. Exploitation Analysis of Buffer Overflow in SL-Mail Server. In Proceedings of the 2021 Fifth Interna-
tional Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, India, 11–13 November 2021;
pp. 1361–1370.
32. Pei, Z.; Chen, X.; Yang, S.; Duan, H.; Zhang, C. TAICHI: Transform Your Secret Exploits Into Mine From a Victim’s Perspective.
IEEE Trans. Dependable Secur. Comput. 2023, 20, 5278–5292. [CrossRef]
33. Tran, M.; Etheridge, M.; Bletsch, T.; Jiang, X.; Freeh, V.; Ning, P. On the expressiveness of return-into-libc attacks. In Proceedings of
the Recent Advances in Intrusion Detection: 14th International Symposium, RAID 2011, Menlo Park, CA, USA, 20–21 September
2011; Proceedings 14; Springer: Berlin/Heidelberg, Germany, 2011; pp. 121–141.
34. Marco-Gisbert, H.; Ripoll, I. Return-to-csu: A new method to bypass 64-bit Linux ASLR. In Proceedings of the Black Hat Asia
2018, Singapore, 20–23 March 2018.
35. Shoshitaishvili, Y.; Wang, R.; Salls, C.; Stephens, N.; Vigna, G. SOK: (State of) The Art of War: Offensive Techniques in Binary
Analysis. In Proceedings of the 2016 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA, 22–26 May 2016.
36. Cha, S.K.; Avgerinos, T.; Rebert, A.; Brumley, D. Unleashing mayhem on binary code. In Proceedings of the 2012 IEEE Symposium
on Security and Privacy, San Francisco, CA, USA, 20–23 May 2012; pp. 380–394.
37. Christopher, R. Zeratool: Automatic Exploit Generation (AEG) and Remote Flag Capture for Exploitable CTF Problems (Version
2.2). Available online: https://github.com/ChrisTheCoolHut/Zeratool (accessed on 30 August 2023).
38. Berthomieu, B.; Ribet, P.-O.; Vernadat, F. The tool TINA—Construction of abstract state spaces for petri nets and time petri nets.
Int. J. Prod. Res. 2004, 42, 2741–2756. [CrossRef]
Electronics 2023, 12, 4741 15 of 15

39. Audrey, R. Documentation for the Angr Suite (Version 9.0.10576). Available online: https://github.com/angr/angr-doc (accessed
on 30 August 2023).
40. radare2. UNIX-Like Reverse Engineering Framework and Command-Line Toolset (Version 5.8.8). Available online: https:
//github.com/radareorg/radare2 (accessed on 30 August 2023).
41. CTFtime. All about CTF. Available online: https://ctftime.org (accessed on 15 July 2023).
42. Liu, Z.; Wang, Z.; Zhang, Y.; Liu, T.; Fang, B.; Pang, Z. Automated Crash Analysis and Exploit Generation with Extendable
Exploit Model. In Proceedings of the 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), Guilin, China,
11–13 July 2022; pp. 71–78.
43. Biswas, A.; Dee, T.M.; Guo, Y.; Li, Z.; Tyagi, A. Multi-Granularity Control Flow Anomaly Detection with Hardware Counters.
In Proceedings of the 2021 IEEE 7th World Forum on Internet of Things (WF-IoT), New Orleans, LA, USA, 14–31 July 2021;
pp. 449–454.
44. Bu, W.; Kang, F.; Zhao, Y.; Xu, J. Vulnerability Sample Analysis Based on Dynamic Binary Platform. In Proceedings of the 2021
International Conference on Aviation Safety and Information Technology, Changsha, China, 18–20 December 2021; pp. 567–573.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.