Formal Fuzzing Amplification
Formal Fuzzing Amplification
Abstract—Softwarization and virtualization in 5G and beyond verification framework, which integrates model checkers and
require rigorous testing against vulnerabilities and unintended cryptographic protocol verifiers by applying the abstraction-
emergent behaviors for critical infrastructure and network secu- refinement principle. In contrast, fuzz testing can offer a
rity assurance. Formal methods operates efficiently in protocol-
arXiv:2307.05758v1 [cs.CR] 11 Jul 2023
level abstract specification models, and fuzz testing offers com- detailed and comprehensive experimental evaluation and detect
prehensive experimental evaluation of system implementations. potential vulnerabilities in the 5G code implementation plat-
In this paper, we propose a novel framework that leverages form [5] and has been proven to be successful in discovering
the respective advantages and coverage of both formal and critical security bugs in implemented software [6]. However,
fuzzing methods to efficiently detect vulnerabilities from proto- the limitations of conventional pick-and-choose fuzz testing
col logic to implementation stacks hierarchically. The detected
attack traces from the formal verification results in critical and formal analysis still exist, especially with the large-scale
protocols guide the case generation of fuzz testing, and the software stacks in the system. Given that each approach pos-
feedbacks from fuzz testing further broaden the scope of the sesses unique strengths, we have proposed a tandem connec-
formal verification. We examine the proposed framework with tion between the fuzz testing and formal methods to achieve
the 5G Non Standard-Alone (NSA) security processes, focusing a more comprehensive vulnerability detection and enable high
on the Radio Resource Control (RRC) connection process. We
first identify protocol-level vulnerabilities of user credentials via assurance in the security analysis.
formal methods. Following this, we implement bit-level fuzzing Further, we propose an integrated framework fortified by
to evaluate potential impacts and risks of integrity-vulnerable a reinforcing loop to detect vulnerabilities and unintended
identifier variation. Concurrently, we conduct command-level
mutation-based fuzzing by fixing the assumption identifier to as- emergent behaviors in system design and implementations.
sess the potential impacts and risks of confidentiality-vulnerable Our approach advocates for a harmonized application of fuzz
identifiers. During this approach, we established 1 attack model testing and formal analysis, aiming to establish a symbiotic
and detected 53 vulnerabilities. The vulnerabilities identified used cycle between these two methods. This integrated strategy
to fortify protocol-level assumptions could further refine search is designed to facilitate the identification of vulnerabilities
space for the following detection cycles. Compared to the state
of art fuzz testing, this unified methodology significantly reduces throughout the entire search space, thereby providing a com-
computational complexity, transforming the computational cost prehensive and robust mechanism for vulnerability detection.
from exponential to linear growth. Consequently, it addresses Formal verification provides valuable guidance and assump-
the prevalent scalability challenges in detecting vulnerabilities tions in reducing and directing fuzz testing. Conversely, fuzz
and unintended emergent behaviors in large-scale systems in 5G testing broaden formal verification’s scope by classifying
and beyond.
Index Terms—Formal verification, fuzz testing, reinforcing
uncertainty areas. Importantly, this integrated approach enables
loop, integrated solution, Non Standard-Alone 5G Network mutual amplification between the two methodologies. Follow-
ing this approach, we discover multiple vulnerabilities due to
I. I NTRODUCTION absence of rudimentary man-in-the-middle (MITM) protection
within the protocol, which is unexpected considering that the
Verticals in 5G and next-generation infrastructure create Transport Layer Security (TLS) solution to this issue has
a diverse and intricate environment consisting of software, been in existence for well over a decade [7]. Our framework,
hardware, configurations, instruments, data, users, and various characterized by robust automation, scalability, and usability,
stakeholders [1]. With system complexity and its lack of promises to enhance security assurance and resilience across
security emphasis by domain scientists, the formed ecosystem both infrastructure and domain levels, striving to guarantee
requires a comprehensive evaluation and in-depth validation the absence of additional security issues within the system.
for improving transitional Critical Infrastructure (CI) security Additionally, the proposed approach could be applied to vari-
posture [2]. ous open programmable communication platforms [8], [9] Our
Two major state-of-the-art approaches, formal verification major contributions are summarized below:
and fuzz testing, have been proposed to detect various vul-
nerabilities and unintended emergent behaviors of the 5G • We designed and implemented the integrated formal
network. Formal verification provides a high-level protocol guided fuzz testing framework, which significantly im-
concept and logical proof of security and vulnerability [3]. proves efficiency and achieves scalability for large-scale
For example, Hussian [4] et al. proposed a cross-layer formal 5G systems and discovery of new and exploited vulnera-
bilities in the Non Standard-Alone (NSA) 5G communi-
cation authentication process. 2) Formal
Verification
• We performed in-depth formal analysis on the NSA 5G Start
3) Search Space
Isolation
authentication process by converting informal protocols Undefined
attack
into a symbolic flowchart (Fig. 2), enabling comprehen- Symbolic
Transfer Provable
sive formal analysis. Security
Provable
• We discover multiple vulnerabilities due to absence of 1) Protocol
Attack
Abstraction
rudimentary MITM protection that needs to be addressed End
the 3GPP technical standards and protocols, despite the 5) Formal Verification
4)Fuzz Guide Fortification
TLS solution to this issue has been in existence for well Quantification
over a decade. $ISMI
$RAND
• With the proposed integrated formal and fuzz testing Impletation ?/$d
Code ******
framework, we connected vulnerabilities detected by for-
Vulnearbility
mal analysis to real-life attack models and discovered new Detection
vulnerabilities. Fig. 1: System Design
The rest of the paper is organized as follows. Section II-A
traces have been detected. Fuzz testing serves a dual
introduces the structure of our proposed framework. We then
purpose: it is not only deployed to identify runtime
discuss our design and formal symbolic transfer of the NSA
vulnerabilities, thereby complementing the detection of
5G communication establishment process in Section II-B,
vulnerabilities through logical proofs on protocols, but it
followed by a detailed analysis and illustration of the formal
also functions as a stochastic approach for those uncertain
verification results in Section III. Then, we propose proven
areas that cannot be verified through formal methods.
solutions for each detected formal attack model, along with
5) Fortification of Protocol and Formal Verification :
some novel suggestions. In Section IV, we use the assumptions
We verify the vulnerabilities detected by fuzz testing
as a guide to apply our proposed fuzz testing framework.
and feedback to the formal result and search space. By
Lastly, in Section V, we use mathematical proof to analyze the
defining the space more precisely, formal verification can
efficiency of different fuzzing strategies across varied scopes
be further optimized, consequently extending the scope
of fuzz testing.
of the security assurance area.
II. S YSTEM D ESIGN The proposed framework inter-connected with our previous
A. Architecture Overview fuzzing platform [11] [12] is capable of performing mutation-
We design and implement a hybrid multi-model vulnerabil- based identifiers fuzzing and permutation-based command
ity and unintended emergent behaviors detection framework fuzzing following the direction from the formal method con-
for 5G and other communication systems. As shown in Fig. 1, clusion. Formal verification, guided fuzzing analysis of results
to achieve the amplification and cross-validation of fuzz testing from the actual 5G testbed, and the real-time analysis and
and formal verification, the proposed framework composites feedback construct a reinforcing loop in our system.
the following components to build up a reinforcing loop: B. Abstraction of NSA 5G Authentication Protocol
1) Protocol Abstraction: At the beginning of the system, Compared to Standard-Alone (SA) 5G network architec-
we abstract the protocol into symbolic language. Logical ture, NSA 5G architecture is still widely adopted but more
transfer can easily exploit vulnerabilities in design. vulnerable because the complexity introduced by the Long
2) Formal Analysis: In the formal verification process, Term Evolution (LTE) compatibility in protocol designs and
we employed Proverif [10], a robust tool, to conduct infrastructure implementation, especially for authentication
an in-depth analysis of our system’s protocols. Proverif and authorization. Therefore, we focus on the authentication
offers a logical proof of security properties and potential process in NSA 5G architecture. As shown in Fig. 2, the
vulnerabilities, facilitating a robust and comprehensive abstracted protocol authentication process in NSA architecture
evaluation of the system’s security integrity. includes four parts: Radio Resource Control (RRC) Connec-
3) Search Space Isolation: The output of formal veri- tion Setup, Mutual Authentication, Non-Access Stratum
fication divides the search space into three sets: no (NAS) Security Setup and Access Stratum (AS) Security
vulnerabilities, attack trace detected, and uncertain areas Setup. Considering the scope of this paper and the critical
that need further investigation. The division of the search level among them, we pilot on the RRC connection setup for
space effectively narrows down the uncertain regions and in-depth analysis.
enables the scalability of vulnerability detection. The RRC Connection Setup is a pivotal step in the initial
4) Formal Guided Fuzz Framework: With the guidance of establishment of communication between a mobile device and
a formal verification conclusion, we initiate fuzz testing the network in the LTE and 5G New Radio (NR) frameworks.
on runtime binary systems, focusing particularly on the This procedure is instigated by the network upon receiving a
predefined uncertain areas and those areas where attack connection request from the User Equipment (UE), commonly
Legend
UE gNB CN
UE-identity establishmentCause
Confidentiality
Integrity
RRC Con n ect i on Set u p RRC Connection Request
Authentication
(UE-identity, establishmentCause) Accounting
Same
RRC Connection Setup RRC
(radioResourceConfigDedicated) Con n ect i on radioResource Sequence
RRC Connection Setup Complete (RRC-Transaction ConfogDedocated
Identifier, selectedPLMN-Identity, dedicatedInfoNAS)
Set u p Integrity Confidentiality
LTE K RAND SQN SN ID Authentication Request (RAND, AUTNHSS, RRC-Transaction selectedPLMN dedicated
KSIASME = 1)[not ciphered, not integrity protected] EPS AKA Algorithm Identifier Authentication Accounting
-Identity InfoNAS
*:Deeper color represent higher
EPS AKA Algorithm Check (AUTNUE = AUTNHSS) security level in special property
AUTNHSS XRES K ASME
Authentication Response (RES)
AUTNUE RES K ASME [not ciphered, not integrity protected]
Check (AUTNUE = AUTNHSS)
Ciphered and Integrity Protected NAS Signaling (K NASenc K NASint ) A. User Credentials Disclosure
K ASME NAS Uplink Count
In this attack, the adversary can exploit the transparency of
KDF
Attach Accept
(UE Capability, K eNB )
RRC Connection Setup process to effortlessly access critical
K eNB
AS Secu i r t y Set u p AS Security Mode Select (encryption/ user identity information, which includes but is not limited
integrity algorithm)
Command (AS Ciphering
Algorithm=EEA1, AS K eNB Alg-ID Distinguisher to the UE identity and establishment cause. This illicit access
Integrity Algorithm=EIA1,
MAC-I) [AS integrity
protected]
KDF enables the adversary to acquire user information and use the
Compute K eNB
K eNB Alg-ID Distinguisher AS Secuirty Mode
K RRCenc K RRCint K UPenc
ensuing session key for nefarious activities such as eavesdrop-
Complete (MAC-I)
KDF [AS integrity protected] ping and manipulation of subsequent communications.
K RRCenc K RRCint K UPenc Assumption. Analyzing Fig 3, we can conclude that the
Ciphered and Integrity Protected RRC Signaling (K RRCenc K RRCint ) adversary can exploit the transparency of RRC Connection
Ciphered User/ Data Plane (K UPenc ) Setup process to directly access any identifier within the mes-
sage. Furthermore, the adversary is also capable of establish
a fake UE or a MITM relay to eavesdrop and manipulate the
Fig. 2: 5G NSA Security Process. messages within the RRC Connection Setup process. To verify
the security properties of identifiers within the RRC Connec-
in response to an initiating event such as a call or data session
tion Setup process, including aspects such as confidentiality
initiation. RRC connection setup process aims to build up
and consistency, we converted the aforementioned assumptions
connections in RRC layer.
into ProVerif code.
Further, we abstract and derive the dependency table, pre-
Vulnerability. As depicted in Fig. 2, the UE initiates the
sented as Table II, from the defined protocol, considering
process by sending an RRC connection request to the core
four essential security properties: confidentiality, integrity,
network (CN). Upon receiving this request, the CN responds
authentication, and accounting. Utilizing Table II, we construct
by transmitting the radioResourceConf igDedicated back
the corresponding dependency graph, as depicted in Fig 3,
to the UE. The UE, in turn, obtains authentication from the
to provide a visual representation of the security dependency
CN and responds with the RRC − T ransactionIdentif ier,
relationships.
selectedP LM N − Identity and dedicatedInf oN AS to fi-
C. Formal Guided Fuzz Framework nalize the RRC connection setup. Nevertheless, this process
presents an exploitable vulnerability as an adversary can access
Compared to traditional fuzz testing, which needs a com- all message identifiers. Such unprotected identifiers run the
plete understanding of code implementation, like LZFUZZ [6], risk of being eavesdropped upon and modified, potentially
we propose a novel formal-guided identifier-based fuzzing enabling the adversary to orchestrate a MITM relay attack.
framework. In our proposed fuzzing framework, we first fix Attack Trace Description. Employing formal verification,
the value of critical identifiers under the assumption proved we analyzed the confidentiality of identifiers within the RRC
by the formal verification and collect the communicated com- Connection Setup process. Through this methodical investi-
mands. Then we set up a relay attack mechanism on srsRAN gation, we identified two categories of identifiers with the
platform [13] following the attack traces, which are detected most significant impact: user identities and RRC configuration
by formal verification. identifiers. As illustrated in Fig. 4, an attacker can access
the identifiers marked in red, delineating the pathway of the
III. F ORMAL D ETECTED ATTACK M ODEL AND A NALYSIS
attack. In the initial scenario, an adversary with the access to
In this section, we present a proof-of-concept via an il- the user identity, like U E − identity, is capable of launch
lustrative attack model detected using Proverif [10]. A com- DoS attack with real U E − identity. Contrary to traditional
prehensive summary of all identified attack models in 5G DoS attacks, which aim to overwhelm a system’s capac-
authentication and authorization process from our findings ity, an U E − identity-based DoS attack efficiently disrupts
Assumpt- New at-
Attack Vulnerability Solution Guidance to fuzz
ion tack?
Modified commands known • Hash value protection for UE identity
Modification of RRC Inspired Fuzz testing can start with dif-
will disable the RRC C-RNTI • Integrity protection
Connection by [4] ferent RRC status.
functions or TMSI
• Ensured confidentiality Authentication
Deny of Service and Key agreement (EC-AKA) [15] Repeat authentication request
UE accepts authenti-
(DoS) or Disconnect Inspired • Hashed international mobile subscriber commands can be fuzzed at
cation request without No
using Authentication by [14] identity (IMSI) [16] random time to test DoS and
integrity.
Request. • Hashed IMSI with integrity check [17] cutting of device attack.
All NAS information known • Asymmetric encryption NAS fuzz testing can start
Exposing KN ASenc Inspired
will be monitored, hi- IMSI, • Hashed IMSI based encryption with known KN ASenc and
and KN ASint by [18]
jacked and modified. MITM KN ASint .
RRC fuzz testing start with
All RRC and UP
Exposing KRRCenc , known • Asymmetric encryption known KRRCenc and
information will be
KRRCint and IMSI, Yes • Hashed IMSI based encryption KRRCint ; User Plane (UP)
monitored, hijacked
KU P enc MITM fuzz testing start with known
and modified.
KU P enc .
1067
different fuzzing strategies
Fig. 5: Comparison of Different Command-level Fuzzing Brute Force Fuzzing Rule-based Fuzzing Formal Guided Fuzzing
Strategy Efficiency Fig. 6: Comparison of Different Bit-level Fuzzing Strategy
Efficiency
V. P ERFORMANCE AND E FFICIENCY A SSESSMENT
Our proposed framework has the capacity not only to
Our proposed fuzz testing framework, guided by formal validate the impact and security of identifiers, but also to detect
verification, affirms the viability of our integrative approach unintended vulnerabilities based on high-risk assumptions,
combining both formal and fuzz testing frameworks. In this such as an identifier set that is accessible to an adversary. As
corroborated by the evidence presented in Section IV-B, our [2] M. Shatnawi, H. Altaleb, and R. Zoltán, “The digital revolution with ne-
framework proves highly efficient in detecting vulnerabilities, sas assessment and evaluation,” in 2022 IEEE 10th Jubilee International
Conference on Computational Cybernetics and Cyber-Medical Systems
underscoring its potential utility in enhancing system security. (ICCC). IEEE, 2022, pp. 000 099–000 104.
[3] A. Peltonen, R. Sasse, and D. Basin, “A comprehensive formal analysis
VI. C ONCLUSION of 5g handover,” in Proceedings of the 14th ACM Conference on Security
In this paper, we have introduced an innovative framework and Privacy in Wireless and Mobile Networks, 2021, pp. 1–12.
[4] S. R. Hussain, M. Echeverria, I. Karim, O. Chowdhury, and E. Bertino,
that integrates formal verification and fuzz testing to fortify “5Greasoner: A property-directed security and privacy analysis frame-
the security of 5G systems, effectively addressing the vulner- work for 5G cellular network protocol,” in Proceedings of the ACM
abilities from protocol logic to implementation stacks. The dy- Conference on Computer and Communications Security. Association
for Computing Machinery, 11 2019, pp. 669–684.
namic feedback loop within this framework has demonstrated [5] G. Klees, A. Ruef, B. Cooper, S. Wei, and M. Hicks, “Evaluating
its strength in both the refinement of undefined areas and the fuzz testing,” in Proceedings of the ACM Conference on Computer and
exhaustive detection of potential vulnerabilities. This work has Communications Security, 2018.
[6] S. Bratus, A. Hansen, and A. Shubina, “Lzfuzz: a fast compression-
been illuminated through an application on a continuous loop based fuzzer for poorly documented protocols,” 2008.
in the RRC Connection Setup process, illustrating the practi- [7] D. G. Berbecaru and G. Petraglia, “Tls-monitor: A monitor for tls
cability and effectiveness of our proposed methodology. In the attacks,” in 2023 IEEE 20th Consumer Communications & Networking
Conference (CCNC). IEEE, 2023, pp. 1–6.
initial phase, our framework identifies a formal attack model [8] O-RAN Alliance, “O-RAN: Towards an Open and Smart RAN,” O-RAN
through the application of formal verification. Subsequently, Alliance, no. October, 2018.
leveraging the protocol-level exposure of user credentials, the [9] Software Radio Systems, “srsRAN is a 4G/5G software radio suite
developed by SRS,” 2021.
proposed framework employs bit-level and command-level [10] B. Blanchet, “Modeling and verifying security protocols with the
fuzzing to execute comprehensive impact identification and applied pi calculus and proverif,” Foundations and Trends® in Privacy
simulate plausible attacks. As a result, by relying on the and Security, vol. 1, no. 1-2, pp. 1–135, 2016. [Online]. Available:
http://dx.doi.org/10.1561/3300000004
verified impact and the security status of the identifier or [11] J. Yang, Y. Wang, T. X. Tran, and Y. Pan, “5g rrc protocol and stack
command determined by the fuzz test, our framework robustly vulnerabilities detection via listen-and-learn.” 2023 IEEE Consumer
reinforces protocol-level assumptions and refines the detection Communications & Networking Conference, January 2023.
[12] Y. Wang, A. Gorski, and A. da Silva, “Development of a Data-
area. Notably, this integrated approach significantly mitigates Driven Mobile 5G Testbed: Platform for Experimental Research,” in
computational complexity, transitioning it from exponential IEEE International Mediterranean Conference on Communications and
to linear growth. This scalability ensures that the framework Networking, 2021.
[13] I. Gomez-Miguelez, A. Garcia-Saavedra, P. D. Sutton, P. Serrano,
can accommodate larger datasets or more complex scenarios C. Cano, and D. J. Leith, “srslte: An open-source platform for lte
without a drastic increase in computational resources or pro- evolution and experimentation,” in Proceedings of the Tenth ACM
cessing time, making it suitable for extensive applications in International Workshop on Wireless Network Testbeds, Experimental
Evaluation, and Characterization, 2016, pp. 25–32.
5G security testing. [14] J.-K. Tsay and S. F. Mjølsnes, “A vulnerability in the umts and lte
To conclude, our research presents a pioneering step towards authentication and key agreement protocols,” in Computer Network Se-
bolstering 5G security by employing an integrated, hierarchical curity: 6th International Conference on Mathematical Methods, Models
approach to vulnerability detection. This work contributes and Architectures for Computer Network Security, MMM-ACNS 2012,
St. Petersburg, Russia, October 17-19, 2012. Proceedings 6. Springer,
substantially to the ongoing efforts to secure the next genera- 2012, pp. 65–76.
tion of wireless communications and provides a foundation [15] J. B. Bou Abdo, H. Chaouchi, and M. Aoude, “Ensured confidentiality
for future research in this domain. Further studies might authentication and key agreement protocol for eps,” in 2012 Symposium
on Broadband Networks and Fast Internet (RELABIRA), 2012, pp. 73–
explore extending this approach to other advanced wireless 77.
technologies to ensure robust security in our increasingly [16] 3GPP, “Universal Mobile Telecommunications System (UMTS); LTE;
connected world. Mobility Management Entity (MME) Visitor Location Register (VLR)
SGs interface specification,” 3rd Generation Partnership Project
ACKNOWLEDGMENT (3GPP), Technical Specification (TS) 29.118, 01 2015, version
8.5.0. [Online]. Available: https://portal.3gpp.org/desktopmodules/
This effort was sponsored by the Defense Advanced Specifications/SpecificationDetails.aspx?specificationId=1601
Research Project Agency (DARPA) under grant no. [17] M. Khan, P. Ginzboorg, K. Järvinen, and V. Niemi, “Defeating the
downgrade attack on identity privacy in 5g,” in International Conference
D22AP00144. The views and conclusions contained herein on Research in Security Standardisation. Springer, 2018, pp. 95–119.
are those of the authors and should not be interpreted as [18] M. T. Raza, F. M. Anwar, and S. Lu, “Exposing lte security weaknesses
necessarily representing the official policies or endorsements, at protocol inter-layer, and inter-radio interactions,” in Security and
Privacy in Communication Networks: 13th International Conference,
either expressed or implied, of DARPA or the U.S. SecureComm 2017, Niagara Falls, ON, Canada, October 22–25, 2017,
Government. Proceedings 13. Springer, 2018, pp. 312–338.
[19] J. Yang, Y. Wang, Y. Pan, and T. X. Tran, “Systematic meets unintended:
R EFERENCES Prior knowledge adaptive 5g vulnerability detection via multi-fuzzing,”
[1] J. Alcaraz-Calero, I. P. Belikaidis, C. J. B. Cano, P. Bisson, D. Bourse, arXiv preprint arXiv:2305.08039, 2023.
M. Bredel, D. Camps-Mur, T. Chen, X. Costa-Perez, P. Demestichas, [20] Z. Salazar, H. N. Nguyen, W. Mallouli, A. R. Cavalli, and E. M. Montes
M. Doll, S. E. Elayoubi, A. Georgakopoulos, A. Mämmelä, H. P. Mayer, De Oca, “5Greplay: A 5G Network Traffic Fuzzer - Application to
M. Payaro, B. Sayadi, M. S. Siddiqui, M. Tercero, and Q. Wang, “Lead- Attack Injection,” in ACM International Conference Proceeding Series.
ing innovations towards 5G: Europe’s perspective in 5G Infrastructure Association for Computing Machinery, 8 2021.
Public-Private Partnership (5G-PPP),” in IEEE International Symposium
on Personal, Indoor and Mobile Radio Communications, PIMRC, vol.
2017-October, 2018.