Next-Generation Intrusion Detection System Using

Advanced AI Techniques and Federated Learning

Nilamadhab Mishra1[0000-0002-7286-7207], Deepika Ajalkar2[0009-0003-8867-3559]

Jerome Daniel3, Sameer Kulkarni4 and Aarif Sheikh5

1,2,3,4,5
G H Raisoni College of Engineering & Management, Pune 412207, India
1 [email protected]
2 [email protected]
3 [email protected]
4 [email protected]

5 [email protected]

Abstract. This project proposes a next-generation intrusion detection system

(IDS) using modern AI technologies. It leverages large language models (LLMs),
federated learning, deep learning, and other machine learning techniques to en-
hance threat detection and automate reporting. The system is built upon exhaus-
tive preprocessing, feature extraction, and PCA (Principal Component Analysis)
based dimensionality reduction on the CIC-IDS 2017 dataset. Random Forest
achieved an accuracy of 99.77% which was better than Logistic Regression
(86.99%) and KNN (98.87%) in multi-classifier evaluations with Naive Bayes,
K-Nearest Neighbors (KNN), Decision Trees, Support Vector Machines (SVM),
and Logistic Regression. An LSTM-based deep neural network is also used for
learning temporal patterns, achieving 98.71% accuracy which proves effective
against zero-day and advanced persistent threat attacks. To support adaptive pri-
vacy-preserving learning in distributed frameworks, a reinforcement-enhanced
federated learning approach is applied which achieves near 90% accuracy at a
global model level. After classifying the cyber threat, a custom LLM assistant is
developed which takes three different outputs as inputs and automatically gener-
ates cyber threat summaries and reports. One can view or query these reports to
extract pertinent information from the SQLite database. The system can be hosted
on cloud platforms like AWS EC2 using Flask and Gradio for easier scalability.
The approach blends interpretability, automation, and robustness which is true
for the proposed IDS framework to significantly reduce false positives while
maintaining real-time multi-task detection, responsiveness, and adaptability to
swiftly changing network environments.

Keywords: Intrusion Detection System, Large Language Models, Federated

Learning, Generative Adversarial Networks, Reinforcement Learning.
2 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

1 Introduction

The scale and complexity of cybersecurity threats have significantly expanded due to
the rapid development of cloud-native technologies and digital infrastructure.
According to Mishra and Mishra [1], traditional signature-based intrusion detection
systems (IDS) often struggle to keep pace with evolving attack vectors and zero-day
exploits. On the other hand, machine learning and deep learning techniques have
emerged as powerful tools for anomaly detection, offering greater accuracy and
adaptability in dynamic threat environments. Techniques such as Random Forest,
Support Vector Machines (SVM), and feature-based dimensionality reduction have
demonstrated notable effectiveness [3], [4], [9].

To further enhance performance, recent research has focused on designing hybrid

IDS architectures that integrate deep learning models with traditional classifiers. Mande
et al. [7] and Arthy et al. [3] demonstrated that multi-model systems significantly
reduce false positives while improving classification accuracy. Furthermore, research
addressing the challenge of data imbalance—such as that by Talukder et al. [11] and
Fan et al. [12]—has promoted the adoption of techniques like SMOTE to ensure
balanced learning during IDS training.

Federated Learning (FL), a cutting-edge approach to collaborative intrusion

detection, has gained traction for enabling data privacy and real-time learning across
decentralized systems. The proposed model incorporates federated reinforcement
learning, wherein distributed agents can detect threat patterns without compromising
local data, inspired by the works of Azar et al. [13] and Praveena et al. [15].
Additionally, the integration of Long Short-Term Memory (LSTM) networks within
this framework supports the effective identification of temporal dependencies and
stealthy threats such as advanced persistent threats (APTs).

This study leverages the CIC-IDS 2017 dataset to achieve high-accuracy

classification, enhanced threat explainability, and data privacy. The resulting system is
robust, portable, and well-suited for deployment in modern cybersecurity scenarios,
particularly within distributed or high-traffic network environments.
Next-Gen Intrusion Detection System Using Advanced AI Techniques and Federated Learning 3

2 Literature Survey

Examining recent research on machine and deep learning techniques in cybersecurity

is necessary to create a robust intrusion detection system (IDS). Some research has
ventured into ML-based IDS models, noting their merits, demerits, and possible im-
provements with AI-based techniques. Table 1 illustrates a systematic review of main
research contributions, describing the methodologies used, outcomes, and areas of re-
search gap in IDS development.

Table 1. Literature Survey.

No. Year Name Key Findings Gap Findings Methodology

1 2024 Mishra & Reviewed ML-based Emphasizes need for Literature review
Mishra [1] IDS techniques and real-time scalability and comparative
their effectiveness and accuracy analysis
2 2024 Philip et al. Applied deep learn- Did not explore fed- Experimental DL
[2] ing for detecting erated or reinforce- approach
web-based intrusions ment learning
3 2024 Arthy et al. Used feature embed- No integration with End-to-end DL with
[3] ding and DL to en- privacy-preserving embedding tech-
hance network traffic techniques like FL niques
analysis
4 2024 Kotnur et al. Surveyed DL and ML Lacks FL or RL inte- Comprehensive sur-
[4] techniques in IDS gration and explaina- vey and gap analy-
bility sis
5 2023 Mishra & Proposed ML-based Lacks temporal mod- ML model design
Mishra [5] novel intrusion detec- eling or LSTM inte- and benchmarking
tion architecture gration
6 2022 Mishra et al. Introduced random No federated model DL with class bal-
[6] oversampling with training or GAN- ancing
deep learning for im- based augmentation
balanced datasets
7 2022 Mande et al. Evaluated multiple No RL or explainable Classifier-wise
[7] classifiers for IDS AI elements included benchmarking
8 2021 Mishra & Applied SVM and Dataset outdated; no Empirical ML eval-
Gupta [8] other ML models on GAN or FL applica- uation
NSL-KDD tion
9 2018 Mishra & Used SVM for binary No deep learning or Basic SVM applica-
Mishra [9] classification in IDS temporal modeling tion
10 2018 Mishra et al. Explored IoT intru- No FL or real-time IoT-focused ML ap-
[10] sion detection with modeling included proach
ML
4 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

3 Proposed Methodology

3.1 System Architecture

Fig. 1. System Architecture for Model Building.

The CIC-IDS 2017 dataset is used for data preparation and aggregation at the start of
the proposed Next-Generation IDS workflow, which is shown in Figure 1. Cleaning,
normalization with StandardScaler, and dimensionality reduction with PCA are applied
to the raw traffic data. Random Forest importance ratings and correlation analysis are
used to guide feature selection. Several supervised classifiers, such as Naive Bayes,
KNN, Decision Trees, SVM, and Logistic Regression, are used to detect network in-
trusions; Random Forest and LSTM are selected as the final models based on their
superior performance. While federated learning with PyTorch allows for distributed,
privacy-preserving model updates, reinforcement learning improves sample selection.
Data synthesis with GANs increases the model's resistance to new threats. A custom-
ized LLM-powered assistant receives the final prediction outputs and automatically
creates structured cyber threat reports. For subsequent examination and analysis, these
reports are kept in a SQLite database. Real-time interaction, monitoring, and automated
report generation are made possible by the system's Flask backend and Gradio-based
user interface, which can be deployed to AWS EC2 or hosted on localhost. In the sec-
tions that follow, each part of this pipeline is covered in detail.

3.2 CIC-IDS 2017 Dataset

The foundation of the intrusion detection system used in this project is the CIC-IDS
2017 dataset, which was created by the Canadian Institute for Cybersecurity. It offers
an extensive collection of real-world network traffic statistics, including both benign
flows and contemporary attack types such as Web-based attacks, DDoS, port scanning,
brute force, and infiltration as shown in Fig.2. In contrast to older datasets, CIC-IDS
2017 accurately mimics actual enterprise traffic patterns by including comprehensive
flow-level information like packet size, flow duration, and byte rates. Both binary and
multi-class classification tasks are made possible by the attacks' broad classification.
Next-Gen Intrusion Detection System Using Advanced AI Techniques and Federated Learning 5

Because of its great quality, diversity, and alignment with contemporary cybersecurity
concerns, this dataset was chosen. It resolves significant problems with old attack sig-
natures and a lack of protocol-level variety that were evident in previous benchmarks
like KDD'99 and NSL-KDD. It is perfect for training conventional machine learning
models, deep learning networks like LSTM, and assessing the efficacy of federated and
reinforcement-based learning strategies because it also provides labeled data for both
supervised and semi-supervised learning. In contemporary IDS research, its temporal
depth and structured style make it appropriate for robust anomaly detection and sequen-
tial modeling.

Fig.2. Examining the CIC-IDS 2017 attacks following data cleaning.

3.3 Data Pre-Processing

In order to ensure that the CIC-IDS 2017 network traffic is clean, consistent, and ap-
propriate for machine learning pipelines, data pre-processing is essential to the effec-
tiveness of the suggested intrusion detection system. The raw data is thoroughly cleaned
using Python tools such as Pandas, NumPy, and Scikit-learn. This includes handling
missing values, deleting infinite values, getting rid of duplicates, and standardizing col-
umn names. To reduce noise and inconsistency that could impair model accuracy, this
step is crucial. Label encoding is used to encode categorical fields, like attack types,
because the dataset includes both string-based and numeric features. This makes it eas-
ier for supervised algorithms to classify the data. StandardScaler, which standardizes
characteristics by eliminating the mean and scaling to unit variance, is used to normal-
ize numerical features. In both deep and conventional models, this guarantees steady
convergence. Principal Component Analysis (PCA), which compresses the feature
space while maintaining crucial variance, minimizes overfitting, and boosts training
efficiency, is then used to reduce dimensionality. SMOTE also addresses class imbal-
ance, allowing for balanced class distributions in binary and multi-class classification
applications. In order to maximize model generalization and avoid memory problems,
the processed data is further sampled. All things considered, the preprocessing pipeline
turns unprocessed traffic into a clean, insightful dataset, speeding up training and
greatly improving the system's capacity to identify intrusions with high accuracy.
6 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

3.4 Model Training

The core functionality of the proposed intrusion detection system lies in its ability to
accurately identify malicious behavior through a synergistic use of deep learning archi-
tectures, reinforcement-enhanced federated learning, and traditional machine learning
models. The CIC-IDS 2017 dataset serves as the foundation, where multiple supervised
classifiers—Logistic Regression, Naive Bayes, K-Nearest Neighbors (KNN), Support
Vector Machines (SVM), and Decision Trees—are trained to differentiate between be-
nign and various attack types. Logistic Regression is used as a baseline for binary clas-
sification, while SVM employs kernel methods to manage high-dimensional decision
boundaries. Naive Bayes and Decision Trees offer interpretability and efficiency for
structured input, whereas KNN classifies based on proximity to labeled instances.

Random Forest is chosen for its ensemble capabilities, contributing both high accu-
racy and feature importance insights. To further enhance performance, XGBoost is in-
corporated as a gradient boosting method, known for its ability to avoid overfitting
through built-in regularization. For learning temporal dependencies and sequential be-
havior patterns in network traffic—critical for detecting time-sensitive threats such as
infiltration or brute-force attacks—Long Short-Term Memory (LSTM) networks are
implemented.

A custom reinforcement learning component is integrated using PyTorch and a light-

weight policy network, dynamically prioritizing samples during training based on ex-
pected rewards. Federated learning is simulated via PyTorch to enable decentralized
training while aggregating privacy-preserving updates into a global model. Addition-
ally, Generative Adversarial Networks (GANs) are utilized to synthesize realistic attack
traffic, strengthening the system’s resilience against novel threats.

As shown in Figure 3, the Random Forest model achieved a slightly higher test ac-
curacy (0.98) compared to the LSTM model (0.97), confirming the effectiveness of
ensemble methods in structured data scenarios. Throughout the training phase, model
evaluation is performed using metrics such as accuracy, precision, recall, F1-score, and
ROC curves, ensuring robustness across both centralized and federated configurations.

Fig.3. Two Models with the Highest Selected Accuracy.

Next-Gen Intrusion Detection System Using Advanced AI Techniques and Federated Learning 7

3.5 Deployment

Using a combination of Flask and Gradio interfaces, the suggested intrusion detection
system is designed for real-time monitoring, automation, and user-friendliness. The
system as a whole operates locally or on cloud-based infrastructure like AWS EC2,
which provides scalable computational resources to effectively manage fluctuating net-
work traffic demands. As a lightweight API layer, the Flask backend controls user au-
thentication, initiates the IDS pipeline, and facilitates communication between the
LLM-based report generator, federated learning modules, and machine learning mod-
els. Users can run the entire detection process straight from the front-end interface after
successfully logging in. Users can create dynamic threat reports with the use of inter-
active features like "Run Full Pipeline" and "Load Batch Reports," which are supported
by the front end, which was constructed with custom HTML and Gradio as shown in
Fig..4. Rapid deployment and user-friendly result presentation are made possible by the
Gradio based interface, and generated reports are stored in SQLite for subsequent que-
rying or viewing in DB Browser. The system can be deployed to AWS EC2 instances
for wider accessibility; however, it is typically hosted at http://127.0.0.1:7860/ for local
deployment. The federated and reinforcement learning modules can adjust to fresh traf-
fic data because of the architecture's support for continuous monitoring. A proprietary
LLM assistant automatically generates structured summaries of risks using outputs
from RF, LSTM, and FL-RL models to support analyst explainability. Understanding
the type and seriousness of detected intrusions is made easier with the help of this re-
porting architecture. Matplotlib and Pandas-based summaries are used to create visual
dashboards and logs that facilitate easy comprehension of IDS activity. From real-time
detection to LLM-based reporting, the deployment pipeline guarantees end-to-end au-
tomation, guaranteeing operational scalability, explainability, and high availability.

Fig.4. Project Pipeline Monitor.

8 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

4 Results and Discussion

This study evaluated several machine learning, deep learning, and federated learning
algorithms for detecting complex network intrusions using the CIC-IDS 2017 dataset.
Among the traditional models, Decision Tree and Random Forest achieved the highest
test accuracies—99.85% and 99.77%, respectively—demonstrating their effectiveness
in managing structured, high-dimensional input features. The LSTM model also per-
formed remarkably well, with a test accuracy of 98.71%, proving particularly valuable
for capturing long-range temporal dependencies associated with sophisticated and per-
sistent threat patterns. In contrast, models such as Naive Bayes and Logistic Regression
exhibited significantly lower performance, indicating their limitations in dealing with
non-linear and intricate interactions present in modern attack scenarios.

As shown in Figure 5, the Gradio-based LLM assistant interface provides real-time,

model-backed threat classification and report generation. The assistant integrates out-
puts from models such as Random Forest and LSTM to classify traffic into specific
attack types and present structured reports. These reports are enriched with threat sum-
maries and mitigation steps, enhancing operational interpretability.

The batch evaluation interface is presented in Figure 6, where reports for multiple
threat instances—including DDoS and Infiltration—are generated and displayed. This
interface enables users to interactively load and download grouped reports, which is
particularly beneficial for real-time operations in high-volume traffic environments.
Furthermore, Figure 7 illustrates the persistent storage of these reports in an SQLite
database. Each report includes fields for predicted attack type, model insights, and sum-
mary interpretation. This structured storage approach not only aids historical analysis
but also supports downstream integration with other forensic or SIEM systems.

Validation performance across models confirmed that Random Forest and Decision
Tree maintained minimal validation loss, demonstrating strong generalization without
overfitting. LSTM and KNN also achieved low validation losses, reflecting consistent
reliability, whereas Naive Bayes and Logistic Regression showed higher validation
losses, suggesting underfitting or limited flexibility to complex data distributions. The
ensemble nature of Decision Tree and Random Forest provides resilience to noisy fea-
tures, while LSTM's temporal modeling capabilities allow it to detect threats that may
be missed by traditional models.

These outcomes underscore the effectiveness of hybrid AI strategies in enhancing

intrusion detection accuracy. Random Forest is highly suitable for fast, real-time clas-
sification, while LSTM offers deeper insight into dynamic, sequential threat behavior.
Federated learning and reinforcement learning collectively contributed to a global
model accuracy of approximately 90%, supporting both privacy preservation and dis-
tributed adaptability. Future enhancements include broader deployment to AWS EC2,
integration of real-time traffic streams, and increased usage of synthetic data generated
through GANs to improve generalization. Model training and experimentation were
Next-Gen Intrusion Detection System Using Advanced AI Techniques and Federated Learning 9

accelerated using PyTorch on Google Colab with GPU support, ensuring scalable and
efficient development cycles.

Table 2. Models And Their Accuracy Scores

Sr. No. Algorithm Name Test Accuracy

1 Decision Tree 99.85
2 Random Forest 99.77
3 K-Nearest Neighbors (KNN) 98.87
4 Long Short-Term Memory (LSTM) 98.71
5 Support Vector Machine (SVM) 96.88
6 Naive Bayes 87.14
7 Logistic Regression 86.99

Fig.5. Single Report Generation of Next - Gen IDS Assistant.

10 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

Fig.6. Batch Reports Generation of Next - Gen IDS Assistant.

Fig.7. DB Browser for SQLite.

Next-Gen Intrusion Detection System Using Advanced AI Techniques and Federated Learning 11

5 Methodology

The methodology for this intrusion detection system is structured around a multi-stage
pipeline that integrates data processing, model training, federated learning, and auto-
mated threat reporting. The CIC-IDS 2017 dataset forms the foundation and is sub-
jected to a comprehensive pre-processing routine that includes cleaning, handling miss-
ing values, normalization using StandardScaler, and dimensionality reduction via Prin-
cipal Component Analysis (PCA). Class imbalance is addressed using SMOTE to en-
sure equitable model training across attack categories.

Following this, a suite of supervised learning algorithms—Logistic Regression, Na-

ive Bayes, K-Nearest Neighbors, Support Vector Machines, and Decision Trees—is
trained and evaluated to identify the most effective baseline models. Random Forest is
selected not only for its accuracy but also for feature importance analysis, while LSTM
is employed to capture sequential patterns within the network traffic. Ensemble tech-
niques such as XGBoost are included to further improve generalization.

Reinforcement learning is introduced via a custom policy network to enhance sam-

ple selection and model adaptability. Federated learning is implemented using PyTorch
to simulate decentralized training, preserving data locality and privacy. Synthetic attack
samples are generated using GANs to strengthen model robustness against unseen
threats. Finally, the outputs from selected models are passed into an LLM-based report-
ing module, which produces structured analytical summaries. The system is deployed
through a Flask and Gradio-based interface, and can be hosted on local or cloud infra-
structure, enabling real-time execution, monitoring, and reporting.

6 Conclusion

In conclusion, by combining deep learning, federated learning, and classical machine

learning methods, this study offers an improved intrusion detection system that greatly
improves cybersecurity defenses. The system provides excellent detection accuracy and
flexibility to a variety of changing cyber-attack patterns by utilizing models including
Decision Trees, Random Forest, LSTMs, SVM, and KNN. While ensemble models like
Random Forest and Decision Tree provide great precision and robustness, the addition
of LSTM allows learning of temporal connections, which is essential for identifying
complex threats.

A realistic simulation of contemporary network traffic is guaranteed by the use of

the CIC-IDS 2017 dataset, and data privacy issues are addressed by the use of federated
learning, which permits decentralized training. Furthermore, scalable, real-time moni-
toring and dynamic model upgrades are guaranteed via cloud-based deployment. These
findings show that the AI-powered IDS performs better in terms of accuracy, adapta-
bility, and privacy than traditional systems.

The groundwork for creating intelligent, scalable, and privacy-conscious cybersecu-

rity systems that can fend off next-generation cyberthreats is laid by this study.
12 Nilamadhab Mishra | Deepika Ajalkar | Jerome Daniel Sameer Kulkarni | Aarif Sheikh

References
1. Mishra, N., Mishra, S.: A review of machine learning-based intrusion detection system. EAI
Endors. Trans. Internet Things 10, (2024)
2. Philip, J., Harshini, M., Patil, S., Shareef, S.K.K., Reddy, B.V., Sridevi, S.: Deep learning
for web intrusion detection. In: Proc. 2nd Int. Conf. Sustainable Computing and Smart Sys-
tems (ICSCSS), pp. 10–12 (2024)
3. Arthy, J., Variar, S.D., Tharakeshwar, N., Narayan, R.A.: End-to-end network traffic exam-
ination for intrusion detection using feature embedding learning. In: Proc. 4th Int. Conf.
Intelligent Technologies (CONIT), pp. 21–23 (2024)
4. Kotnur, H., Muthusamy, S., Ravindran, S., Vijean, V.: A comprehensive survey of intrusion
detection system using machine learning and deep learning approaches. In: Proc. 10th Int.
Conf. Advanced Computing and Communication Systems (ICACCS), pp. 14–15 (2024)
5. Mishra, N., Mishra, S.: A novel intrusion detection technique of the computer networks us-
ing ML. Int. J. Intell. Syst. Appl. Eng. 11(5s), 247–260 (2023)
6. Mishra, N., Mishra, S., Patnaik, B.: A novel IDS based on random oversampling and deep
neural network. Indian J. Comput. Sci. Eng. 13(6), 1924–1936 (2022)
7. Mande, S., Ramachandran, N., Kumar, C.K., Priyanka, C.N.: A brief analysis on machine
learning classifiers for intrusion detection to enhance network security. In: Proc. Int. Conf.
Automation, Computing and Renewable Systems (ICACRS), pp. 1–6 (2022)
8. Mishra, N., Mishra, S.: On the NSL-KDD dataset, a survey of machine learning-based in-
trusion detection systems. J. Huazhong Univ. Sci. Technol. 50(4), 4512 (2021)
9. Mishra, N., Mishra, S.: Support vector machine used in network intrusion detection. In: Nat.
Workshop on Internet of Things (IoT) (2018)
10. Mishra, N., Mishra, S., Patnaik, B.: Intrusion detection using IoT. In: The Things Services
and Applications of Internet of Things, pp. 79–84 (2018)
11. Talukder, M.A., Islam, M.M., Uddin, M.A., Hasan, K.F., Sharmin, S., Alyami, S.A., et al.:
Machine learning-based network intrusion detection for big and imbalanced data using over-
sampling, stacking, feature embedding and feature extraction. J. Big Data 11(1), 33 (2024)
12. Fan, Z., Sohail, S., Sabrina, F., Gu, X.: Sampling-based machine learning models for intru-
sion detection in imbalanced dataset. Electronics 13(10), 1878 (2024)
13. Azar, A.T., Shehab, E., Mattar, A.M., Hameed, I.A., Elsaid, S.A.: Deep learning-based hy-
brid intrusion detection systems to protect satellite networks. J. Netw. Syst. Manag. 31(4),
82 (2023)
14. Rani, M., Gagandeep: Effective network intrusion detection by addressing class imbalance
with deep neural networks. Multimed. Tools Appl. 81(6), 8499–8518 (2022)
15. Praveena, V., Vijayaraj, A., Chinnasamy, P., Ali, I., Alroobaea, R., et al.: Optimal deep
reinforcement learning for intrusion detection in UAVs. Comput. Mater. Contin. 70(2),
2639–2653 (2022)