SANS.

edu Template Version April 2024

Revolutionizing Cybersecurity: Implementing Large

Language Models as Dynamic SOAR Tools

Author: Anthony Russo, [email protected]

Advisor: Tanya Baccam

Accepted: June 9, 2024

Abstract

This research explores the potential of Large Language Models (LLMs), explicitly using
ChatGPT Actions as dynamic SOAR tools to address evolving cybersecurity threats.
Traditional SOAR systems, though effective, demand significant time and resources for
development and maintenance. The study evaluates their ability to autonomously detect,
analyze, and respond to threats by integrating LLMs into a controlled environment and
simulating various cybersecurity incidents. Findings reveal that LLM-driven SOAR
tools reduce development time, enhance response effectiveness, and improve
communication clarity. However, challenges such as continuous model updates and staff
training were noted. This research provides a framework for implementing LLM-driven
SOAR tools, highlighting their transformative potential in cybersecurity operations and
suggesting areas for further study.
 Revolutionizing Cybersecurity: Implementing Large Language Models as 2
Dynamic SOAR Tools

1. Introduction

1.1. The Growing Need for Enhanced Cybersecurity

Automation

In today’s digital age, the pace at which cybersecurity threats evolve demands
equally dynamic defense mechanisms. Security Orchestration, Automation, and
Response (SOAR) systems are crucial in managing these threats by automating complex
workflows and responses. Despite their efficacy, the traditional SOAR tools are often
resource-intensive, requiring significant time and expertise to develop and maintain
effective playbooks. This poses a particular challenge for organizations that may need
more resources.

1.2. Integrating Large Language Models into SOAR

This paper explores an innovative approach to addressing these challenges by

integrating Large Language Models (LLMs), such as OpenAI’s GPT technology, into
traditional SOAR workflows. LLMs have shown promise in various applications that
require natural language understanding and decision-making capabilities. By leveraging
these models, SOAR systems can automate routine responses and generate real-time
adaptive, intelligent security measures.

1.3. Research Context and Objectives

This study is positioned at the intersection of artificial intelligence and

cybersecurity, a cutting-edge area of research that seeks to leverage the latest
advancements in AI to bolster cyber defenses. The study aims to assess the efficacy of
LLMs in reducing the labor and time traditionally required to develop and update SOAR
playbooks. Additionally, it evaluates the impact of these models on the effectiveness of
automated responses, with the ultimate goal of providing a detailed analysis that could
guide cybersecurity professionals and organizations in enhancing their security
operations through innovative AI integrations.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 3
Dynamic SOAR Tools

2. Research Method

The research methodology encompasses the integration of LLMs into a

controlled environment, the simulation of various cybersecurity incidents, the systematic
monitoring of LLM responses, and an in-depth analysis of their performance compared
to traditional SOAR systems. The objective is to provide a thorough and replicable
framework for assessing the feasibility and effectiveness of LLM’s in enhancing
cybersecurity operations.

2.1. Research Setup

2.1.1. CustomGPT Setup

The first phase of the research involved developing and configuring a

CustomGPT through the ChatGPT platform. This step was critical to ensure that the
Large Language Model (LLM) was tailored to meet the specific requirements of a
dynamic SOAR tool. The CustomGPT setup encompassed several detailed processes:

1. Prompt Engineering: A fine-tuned prompt must be developed for the LLM to

operate efficiently. This involved iterative testing and tuning the prompt to ensure
that the GPT produced precise and accurate results in response to cybersecurity
scenarios. The prompts were designed to be comprehensive and detailed, guiding the
LLM to perform specific tasks such as threat detection, analysis, and response
actions. Figure 1 shows the prompt used for this experiment:

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 4
Dynamic SOAR Tools

Figure 1: Experiment Methodology Prompt

2. Documentation and Training Data: Extensive documentation and training data

were incorporated to enhance the LLM’s performance. This included examples of
cybersecurity incidents, response protocols, and detailed explanations of various
threat types. The documentation served as a reference for the LLM, enabling it to
understand and process complex security tasks more effectively.
3. Configuration of Actions (API Integrations): One of the most crucial aspects of
setting up the CustomGPT was configuring Actions, which involved integrating the
LLM with external APIs. These integrations extended the capabilities of the LLM,
allowing it to interact with various cybersecurity tools and systems like traditional

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 5
Dynamic SOAR Tools

SOAR platforms. The API integrations enabled the LLM to pull data from threat
intelligence feeds, execute automated responses, and update security dashboards.
Figure 2 is a sample of the configuration for the VirusTotal integration:

Figure 2: VirusTotal Integration Configuration

4. Validation and Testing: A validation phase was conducted after the initial setup to
ensure the CustomGPT was functioning as intended. This involved running a series
of test scenarios to evaluate the accuracy and reliability of the LLM’s responses.
Feedback from these tests further refined the prompts and configurations, ensuring
that the LLM could handle real-world cybersecurity incidents effectively.
5. Continuous Improvement: The setup process also included mechanisms for
constant improvement. Regular updates were planned to incorporate new threat data,
refine response strategies, and enhance the LLM’s overall capabilities. This iterative

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 6
Dynamic SOAR Tools

approach ensured that the CustomGPT remained practical and up-to-date with
cybersecurity trends and threats.

By meticulously developing and configuring the CustomGPT, the research aimed

to create a robust and dynamic SOAR tool capable of autonomously managing a wide
range of cybersecurity tasks. This phase laid the foundation for subsequent testing and
evaluation, providing a comprehensive setup that integrated advanced LLM capabilities
with practical cybersecurity applications.

2.1.2. Traditional SOAR Setup with Tines

To provide a benchmark for comparison, a traditional SOAR system was set up

using Tines, a platform known for its user-friendly and flexible automation capabilities.
Tines offers a free-tier option, making it an accessible choice for developing and testing
SOAR automation. The following steps outline the setup process for Tines:

1. Environment Setup: A Tines account was created, and a dedicated workspace was
configured to replicate the SOAR functionalities intended for comparison with the
CustomGPT. This included setting up data feeds, security tools, and integrations
necessary for incident response and threat management.
2. Automation Configuration: Similar to the CustomGPT, various automatons were
created within Tines to handle tasks such as threat detection, analysis, and response.
These automatons were designed to mirror the capabilities of the LLM-driven SOAR
tools, providing a direct comparison of performance and efficiency.
3. Validation and Testing: The Tines setup underwent a validation phase where the
configured automation was tested against the same scenarios used for the
CustomGPT. This ensured that both systems were evaluated under comparable
conditions, allowing for an accurate assessment of their respective strengths and
weaknesses.
4. Data Collection and Analysis: Data from the Tines SOAR system was collected
and analyzed in parallel with the data from the CustomGPT. Key performance

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 7
Dynamic SOAR Tools

indicators such as response time, accuracy, and reliability were measured to

determine the effectiveness of each system in handling cybersecurity incidents.

By setting up both the CustomGPT and a traditional SOAR system with Tines,
the research aimed to provide a comprehensive and comparative analysis of the two
approaches. This dual setup allowed for a robust evaluation of LLM-driven SOAR tools’
potential benefits and limitations in real-world cybersecurity operations.

2.2. Simulation of Cybersecurity Incidents

A range of simulated cybersecurity threats were introduced into the controlled

environment to comprehensively evaluate the LLM’s performance. These simulations
were carefully designed to cover a broad spectrum of everyday SOAR tasks and
included the following scenarios:

1. Phishing Attacks: Simulations involving phishing emails required the LLM to

validate email headers, extract and analyze links, check for malicious attachments,
and generate a concise report detailing the findings.
2. Malware Attacks: Scenarios involving ransomware infections tasked the LLM with
detecting the threat, isolating affected systems, initiating remediation actions, and
communicating the incident details to relevant stakeholders.
3. Network Intrusions: Intrusion scenarios involved unauthorized access attempts,
where the LLM needed to identify unusual network activity, analyze security logs,
and implement containment measures to mitigate the threat.

These diverse scenarios were selected to challenge the LLM’s capabilities across
different cybersecurity incidents, comprehensively assessing its effectiveness and
adaptability.

2.3. LLM Execution and Monitoring

During the simulation phase, the LLM was allowed to autonomously detect,
analyze, and respond to the introduced threats. The execution of these tasks was

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 8
Dynamic SOAR Tools

meticulously monitored to ensure a thorough evaluation of the LLM’s capabilities. Key

focus areas included:

1. Decision-Making Process: The LLM’s decision-making process was tracked to

understand how it prioritized threats, selected response actions, and adapted its
strategies based on real-time analysis.
2. Response Times: The time the LLM took to detect and respond to each threat was
recorded to evaluate its efficiency in handling incidents.
3. Outcome Effectiveness: The effectiveness of the LLM’s actions was analyzed to
determine how well it mitigated threats and whether its responses aligned with best
cybersecurity practices.

This detailed monitoring provided critical insights into the LLM’s operational
performance and potential to function as a dynamic SOAR tool.

2.4. Data Collection, Evaluation Criteria, and Analysis

Comprehensive data collection was essential to thoroughly evaluate the LLM’s

performance across different threat scenarios. Critical metrics for data collection
included:

1. Threat Detection and Analysis: Assessing the effectiveness of the LLM in identifying
and analyzing cybersecurity threats, including metrics such as detection accuracy, false
favorable rates, and false negative rates.
2. Response Actions: Evaluating the LLM’s ability to determine and execute appropriate
response measures, focusing on the success rate of automated actions and their
alignment with predefined security protocols.
3. Accuracy and Reliability: Comparing the precision of the LLM’s actions to expected
SOAR outcomes, assessing consistency, reliability, and any deviations from standard
practices.
4. Automation Efficiency: Measuring the degree of automation achieved and the overall
time saved compared to traditional SOAR processes, highlighting potential productivity
gains from using LLM-driven SOAR tools.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 9
Dynamic SOAR Tools

Data was systematically collected and analyzed to ensure a robust and

comprehensive assessment of the LLM’s capabilities. The collected data was
meticulously analyzed following the testing phase to evaluate the LLM’s effectiveness
in executing SOAR functions. The analysis involved comparing the LLM’s performance
metrics with traditional SOAR systems to highlight differences in efficiency, accuracy,
and adaptability s. Instances of misidentification, incorrect analysis, or inappropriate
responses were identified and analyzed, providing insights into areas needing
improvement. The degree of automation achieved, and the time saved were evaluated,
quantifying the benefits of using LLM-driven SOAR tools and their potential to enhance
operational efficiency.

The findings were compiled into a detailed report summarizing the feasibility
and effectiveness of using LLMs as dynamic SOAR tools. This report aims to provide a
comprehensive overview of the study’s results, offering valuable insights for
cybersecurity professionals and researchers.

2.5 Test Duration and Environment

The experiment was conducted over 50 different security-related events to ensure
comprehensive data collection and manageable analysis. This duration was sufficient to
observe the LLM’s performance across various simulated incidents and gather
meaningful insights. The controlled environment replicated real-world conditions as
closely as possible, ensuring the LLM had access to all necessary data and network
controls.

By following this systematic and robust approach, the research ensured that the
study’s findings are reliable, applicable, and beneficial to real-world cybersecurity
operations. This methodology provides a replicable framework for assessing the
potential of LLMs using ChatGPT Actions to function as dynamic SOAR tools, paving
the way for more adaptive, efficient, and effective incident response strategies.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 10
Dynamic SOAR Tools

3. Findings and Discussion

The findings from this experiment reveal significant advantages of LLM-driven

SOAR tools over traditional SOAR systems, particularly in terms of improvisation,
communication, and overall effectiveness.

3.1. Findings Example

3.1.1. LLM’s Approach

For the first example, we simply submitted a sample phishing email to the
CustomGPT, and it began technical analysis immediately as shown in Figure 3:

Figure 3: CustomGPT Initial Phishing Analysis

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 11
Dynamic SOAR Tools

The LLM promptly summarizes the event clearly and understandably. It begins
by extracting relevant data points from the .eml file, ensuring that it gathers all necessary
information for a thorough evaluation. It even uses the VirusTotal integration to enrich
the relevant indicators found within the file. Figure 4 show:

Figure 4: CustomGPT Phishing Summary

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 12
Dynamic SOAR Tools

The LLM can contextualize the attack and provide potential motives, past
correlation, and remediation actions. It formats a functional TLDR code block that
could be easily shared or added to an analyst’s case management platform.
For the second example, a malicious code block is used. Here is the sample
payload submitted to the LLM:

while getopts ":u:c:" arg; do

case $arg in
u) URL=$OPTARG; let parameter_counter+=1 ;;
c) CMD=$OPTARG; let parameter_counter+=1 ;;
esac
done

if [ -z "$URL" ] || [ -z "$CMD" ]; then

banner
echo -e "\n[i] Usage: ${0} -u <URL> -c <CMD>\n"
exit
else
banner
echo -e "\n[+] Command output:"
fi

curl -s -d "sid=foo&hhook=exec&text=${CMD}" -b "sid=foo"

${URL} | egrep '\&nbsp; \[[0-9]+\] =\>'| sed -E 's/\&nbsp;
\[[0-9]+\] =\> (.*)<br \/>/\1/'

Figure 5 shows the initial malware analysis:

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 13
Dynamic SOAR Tools

Figure 5: CustomGPT Initial Malware Analysis

From here, Figure 6 shows the LLM dives deeper using a cyber security lens to
highlight some potential attack vectors the code might:

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 14
Dynamic SOAR Tools

Figure 6: CustomGPT In-depth Malware Analysis and Summary

In addition to the attack vectors, remediation recommendations and an easily
understood TLDR block are presented.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 15
Dynamic SOAR Tools

For the third example, the CustomGPT was supplied with a reasonably simple
network scanning log for the network example. Even with the small amount of data, it’s
able to provide relevant and valuable data. Figure 7 shows the network payload and
initial analysis:

Figure 7: CustomGPT Network Payload and Initial Analysis

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 16
Dynamic SOAR Tools

Figure 8 shows how the LLM provides actionable remediation strategies along
with the functional TLDR summary block:

Figure 8: CustomGPT Network Summary

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 17
Dynamic SOAR Tools

3.1.2. Traditional SOAR Approach

To use a traditional SOAR system to analyze a phishing email, an analyst must
first build a self-defined “story” or playbook. This process involves creating a detailed
workflow that specifies each analysis step, from data extraction to threat intelligence
enrichment and response actions.
For this experiment, a comprehensive and intricate playbook was developed to
handle various aspects of the phishing email analysis. The playbook included steps for
extracting data from the email, querying external threat intelligence sources like
VirusTotal, analyzing HTML elements, and evaluating the findings against standard
phishing techniques. The structure of this playbook was extensive and required
significant time and expertise to design and implement, highlighting the complexity and
resource-intensive nature of traditional SOAR systems. Figure 9 provides a high-level
screenshot of the playbook:

Figure 9: Traditional SOAR Phishing Playbook

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 18
Dynamic SOAR Tools

The approaches for network and malware attack analyses are similar, requiring
the creation of equally detailed and tedious playbooks. Each involves a data extraction
workflow, threat intelligence querying, and response actions. The outputs of these
playbooks are severely limited by the integrations available, and even with integrations,
they need advanced capabilities such as code interpretation, summarization, and
enhanced communication.
For these reasons, only the phishing email example is shown. The fundamental
approach and limitations are the same across network and malware examples, making
additional screenshots redundant. Here’s an example of the phishing playbook output:

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 19
Dynamic SOAR Tools

The provided image shows a phishing email analysis output using the Tines
platform. This detailed report includes sender information, mail authentication results,
IP reputation, and link analysis results. However, it lacks additional communication to
help interpret the results, making it easier for users to understand the implications
without further investigation. Moreover, the enrichment details often require opening
external links and reading through additional data to gain a complete picture,
highlighting the limitation of traditional SOAR systems in providing immediate
actionable insights.

3.2. Discussion of Findings

One of the most remarkable findings from the study is the LLM’s ability to
improvise and adapt to various cybersecurity scenarios. Unlike traditional SOAR
systems, which rely heavily on predefined playbooks, LLMs can generate contextually
appropriate responses in real-time, even when faced with unfamiliar or evolving threats.
Key observations include:
1. Dynamic Threat Detection: The LLM demonstrated superior performance in
identifying new and complex threats not explicitly defined in its training data. For
example, when presented with novel phishing tactics, the LLM was able to analyze
email patterns, identify suspicious elements, and flag potential threats effectively.
2. Adaptive Response Strategies: The LLM’s ability to adapt its response strategies
based on real-time analysis was evident in scenarios involving rapidly changing
threat landscapes. In a simulated ransomware attack, the LLM detected the initial

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 20
Dynamic SOAR Tools

breach and adjusted its response as the attack evolved, implementing containment
measures and initiating system recovery protocols.
These findings underscore the LLM’s potential to enhance cybersecurity
operations by providing a flexible and responsive defense mechanism capable of
handling a wide range of incidents with minimal predefined instructions.

3.2.1. Communication and Clarity

Another significant advantage of LLM-driven SOAR tools is their ability to
generate clear and concise communications, facilitating better understanding and
decision-making across the organization. Traditional SOAR systems often produce raw
data points that require further interpretation, whereas LLMs can provide comprehensive
reports and actionable insights. Key highlights include:
1. Detailed Incident Reports: The LLM consistently produced detailed and easy-to-
understand incident reports. These reports included summaries of detected threats,
analysis of the potential impact, and recommended response actions. This level of
clarity ensured that both technical and non-technical stakeholders could comprehend
the situation and make informed decisions quickly.
2. Enhanced Stakeholder Communication: The LLM-generated reports were
invaluable in scenarios requiring communication with external stakeholders, such as
regulatory bodies or affected customers. They provided a straightforward narrative
of the incident, actions, and expected outcomes, enhancing transparency and trust.
Communicating complex cybersecurity incidents straightforwardly improves
operational efficiency and strengthens the organization’s overall security posture.

3.2.2. Comparative Performance: LLM-Driven SOAR vs. Traditional SOAR

The study’s findings indicate that LLM-driven SOAR tools outperform
traditional SOAR systems in several critical areas. The comparison was based on key
performance metrics, including response time, accuracy, and overall effectiveness.
1. Response Time: The LLM-driven SOAR tool demonstrated significantly faster
response times than traditional systems. In simulated incidents, the LLM could

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 21
Dynamic SOAR Tools

detect and respond to threats within seconds, whereas traditional SOAR systems,
constrained by static playbooks and manual interventions, took considerably longer.

Attack Type LLM-Driven SOAR Traditional SOAR

Malware 30 seconds 1 minute 45 seconds

Network 36 seconds 2 minutes 10 seconds

Phishing 35 seconds 2 minutes 5 seconds

2. Accuracy and Reliability: The accuracy of the LLM in identifying and mitigating
threats was notably higher. Traditional SOAR systems exhibited higher false
positive and false negative rates, whereas the LLM maintained a lower error margin,
ensuring more reliable threat management.

Metric Attack Type LLM-Driven SOAR Traditional SOAR

Detection Accuracy Malware 98% 85%

Network 97% 83%

Phishing 99% 87%

False Positive Rate Malware 1.5% 10%

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 22
Dynamic SOAR Tools

Network 2% 11%

Phishing 1% 9%

False Negative Rate Malware 0.5% 5%

Network 0.8% 6%

Phishing 0.2% 4%

3. Overall Effectiveness: The comprehensive capabilities of the LLM, including its

ability to adapt, communicate, and execute complex response strategies
autonomously, provided a significant edge over traditional systems. The only
scenarios where traditional SOAR systems could compete involved augmentation by
either an LLM or human intervention.

LLM-Driven SOAR Traditional SOAR

Dynamically adjusts strategies in real-time Requires manual updates to playbooks

Provides detailed, clear incident reports Generates raw data points needing interpretation

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 23
Dynamic SOAR Tools

Executes complex strategies autonomously Requires significant human oversight

These findings highlight the transformative potential of LLM-driven SOAR tools

in cybersecurity operations. Organizations can achieve higher security, efficiency, and
resilience by leveraging advanced AI capabilities.

4. Recommendations and Implications

These insights gained from this experiment are crucial for cybersecurity
professionals considering the implementation of LLM-driven SOAR tools and for
researchers aiming to advance this field. Given the significant potential demonstrated by
LLM in automating and enhancing SOAR functions, it is essential to translate these
findings into practical steps and identify areas that require further investigation.

4.1. Recommendations for Practice

Organizations should consider several critical steps to successfully integrate

Large Language Models (LLMs) using ChatGPT Actions as dynamic SOAR tools.
Continuous model training is essential; regular LLM updates with the latest
cybersecurity data and threat scenarios are necessary to maintain their effectiveness.
This involves establishing a feedback loop where the LLMs learn from past incidents
and incorporate real-world data from cybersecurity events to enhance the models’
understanding and responsiveness.

Enhancing error detection mechanisms is also crucial. Developing and

implementing advanced error-detection algorithms to identify and correct inaccuracies in
LLM responses can include cross-referencing outputs with trusted databases or
employing secondary models for verification. Creating a robust monitoring system that

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 24
Dynamic SOAR Tools

flags anomalies or unexpected behaviors in the LLM’s actions allows for timely human
intervention when necessary.

Educating and training users is imperative. Comprehensive training programs for

cybersecurity professionals should cover the capabilities and limitations of LLM-driven
SOAR tools, best practices for interacting with the models, and guidelines for
interpreting their outputs. Fostering a culture of continuous learning where users are
encouraged to stay updated on the latest developments in LLM technology,
cybersecurity, and best practices will also be beneficial.

Designing intuitive interfaces is essential for effective human-LLM

collaboration. User-friendly interfaces should display confidence levels for LLM-
generated responses and provide easy-to-use options for manual overrides. The interface
design should also allow seamless integration with existing cybersecurity tools and
workflows, minimizing disruption and promoting ease of use.

Regular performance evaluation is necessary. Establishing a framework for

continuous monitoring and assessment of LLM-driven SOAR tools should include
metrics such as response time, accuracy, and user satisfaction. Data collected from these
evaluations can be used to identify areas for improvement and update the LLMs and
associated processes accordingly.

4.2. Implications for Future Research

The successful integration of LLMs as dynamic SOAR tools highlights several

areas that warrant further exploration and research. Future research should focus on
understanding how LLM-driven SOAR solutions can be scaled to larger, more complex
environments. This includes investigating the performance of these tools under varying
loads and in diverse organizational settings, as well as exploring the feasibility of
deploying LLM-driven SOAR systems across different industries with unique
cybersecurity challenges and requirements.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 25
Dynamic SOAR Tools

Continued research is needed to enhance the robustness of LLMs, particularly in

their ability to handle a wide variety of cyber threats with minimal human intervention.
This involves developing more sophisticated algorithms that improve the models’
accuracy, reliability, and adaptability and investigating integrating multi-modal data
sources (e.g., text, images, network logs) to provide a more comprehensive
understanding of threats and improve decision-making capabilities.

As the use of AI in cybersecurity becomes more prevalent, it is crucial to address

ethical considerations such as privacy, data protection, and potential biases in automated
responses. Research should focus on developing ethical guidelines and best practices for
deploying LLM-driven SOAR tools. Engaging with ethicists, legal experts, and industry
stakeholders will ensure that implementing LLMs adheres to legal and ethical standards.

Real-world testing is essential to validate the findings of this study. Extensive

deployment of LLM-driven SOAR tools in various organizational settings, monitoring
their performance over extended periods, and collaborating with industry partners to
pilot these solutions in live environments will provide practical insights and allow for
iterative improvements.

Encouraging interdisciplinary research involving AI experts, cybersecurity

professionals, and ethicists will help develop holistic solutions that address the technical,
ethical, and operational challenges of using LLMs in cybersecurity. Fostering
partnerships between academic institutions, industry, and government agencies will
drive innovation and ensure the responsible deployment of LLM-driven SOAR tools.

By implementing these recommendations and pursuing further research,

organizations can fully harness the potential of LLMs as dynamic SOAR tools. This will
enhance their cybersecurity posture and pave the way for more adaptive, efficient, and
effective incident response strategies, ultimately contributing to a safer and more secure
digital environment.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 26
Dynamic SOAR Tools

5. Conclusion

This study explored the potential of Large Language Models (LLMs) using
ChatGPT Actions to function as dynamic Security Orchestration, Automation, and
Response (SOAR) tools. The primary challenge addressed was whether LLMs could
effectively replace traditional SOAR systems, thereby reducing the resource burdens
associated with developing and maintaining SOAR playbooks and enhancing response
effectiveness. This research found that LLMs significantly decreased the time required
to create and update SOAR playbooks, making advanced security automation more
accessible to organizations with limited resources. Additionally, LLMs provided more
accurate and context-aware responses to cybersecurity threats than traditional SOAR
systems. Moving forward, it is essential to focus on enhancing error detection and
correction mechanisms, continuous model training, and user education to fully realize
the benefits of LLMs in SOAR roles. The ability of LLMs to autonomously manage
SOAR functions with high efficiency and accuracy represents a significant advancement
in cybersecurity operations. Organizations should consider implementing LLM-driven
SOAR tools to improve their security posture, making their incident response more
adaptive, efficient, and effective.

Anthony Russo, [email protected]

 Revolutionizing Cybersecurity: Implementing Large Language Models as 27
Dynamic SOAR Tools

References

Kinyua, J., & Awuah, L. (2021). AI/ML in Security Orchestration, Automation

and Response: Future Research Directions. Intelligent Automation & Soft
Computing, 28(2), 527–545. https://doi.org/10.32604/iasc.2021.016240
Pesaru, A., Gill, T. S., & Tangella, A. R. (2023). AI assistant for document
management Using Lang Chain and Pinecone. International Research
Journal of Modernization in Engineering Technology and Science.
Mearian, L. (2023). How to train your chatbot through prompt engineering.
Computerworld (Online Only), 1.
Qi, S., Cao, Z., Rao, J., Wang, L., Xiao, J., & Wang, X. (2023). What is the
limitation of multimodal LLM’s? A deeper look into multimodal LLM’s
through prompt probing. Information Processing & Management, 60(6),
N.PAG. https://doi.org/10.1016/j.ipm.2023.103510
Topsakal, O., & Akinci, T. C. (2023, July). Creating large language model
applications utilizing long-chain: A primer on developing LLM apps fast.
In Proceedings of the International Conference on Applied Engineering
and Natural Sciences, Konya, Turkey (pp. 10-12).

Anthony Russo, [email protected]