Anomaly Detection in IoT Sensor Data

* Note: Sub-titles are not captured for https://ieeexplore.ieee.org and should not be used

1st Malak ERRIFAI 2nd Diae Alaoui Soulimani 3rd Abdellahi Beddi
School of Science and Engineering School of Science and Engineering School of Science and Engineering
Al Akhawayn University in Ifrane Al Akhawayn University in Ifrane Al Akhawayn University in Ifrane
Ifrane, Morocco Ifrane, Morocco Ifrane, Morocco
[email protected] [email protected] [email protected]

4th Yousra Chtouki

School of Science and Engineering
Al Akhawayn University in Ifrane
Ifrane, Morocco
[email protected]

Abstract—This paper presents a machine learning-based ap- to enhance anomaly detection accuracy while minimizing false
proach to anomaly detection in IoT sensor data by integrating positives. By evaluating the proposed model on real-world and
statistical modeling, deep learning, and time-series analysis. By simulated datasets, we aim to demonstrate its robustness and
leveraging autoencoders and recurrent neural networks (RNNs)
for pattern recognition and unsupervised learning techniques adaptability across various IoT applications.
for anomaly detection, the model effectively identifies irregular This paper explores a hybrid approach that integrates sta-
patterns in real-time sensor streams. The approach combines tistical modeling with deep learning to enhance anomaly
quantitative metrics with adaptive learning algorithms to enhance detection accuracy while minimizing false positives. Our
detection accuracy while minimizing false positives. Our findings method leverages unsupervised learning (isolation forest) and
contribute to the development of robust, scalable, and intelligent
anomaly detection frameworks for IoT applications across diverse supervised classification (decision trees) for robust detection of
industries, including healthcare, smart cities, and industrial sensor anomalies. The study is conducted using two datasets:
automation. • Dataset 1: Synthetic iot dataset from Anomaly Detection
Index Terms—Anomaly detection, IoT sensor data, machine
learning, deep learning, time-series analysis, autoencoders, re-
in IOT devices (2000 row dataset).
current neural networks (RNNs), unsupervised learning, real- • Dataset 2: A custom-generated IoT sensor dataset, col-
time monitoring, cybersecurity, predictive analytics, smart cities, lected from our deployed sensors, capturing real-world
industrial automation, healthcare systems. IoT anomalies. This dataset is still in development, as we
continue collecting additional sensor readings for more
I. I NTRODUCTION accurate model training.
The Internet of Things (IoT) has revolutionized industries Preliminary results indicate that hybrid approaches out-
by enabling seamless communication between interconnected perform standalone models in terms of detection efficiency
devices and generating vast amounts of real-time data. These and adaptability. However, due to limited real-world sensor
sensors play a crucial role in applications such as smart cities, data, achieving optimal performance remains challenging. The
healthcare monitoring, and industrial automation, where real- objective of this research is to refine our anomaly detection
time insights drive decision-making processes. However, the framework, ensuring scalability and real-time processing for
sheer volume and dynamic nature of IoT sensor data introduce IoT security applications.
challenges in identifying anomalies that may indicate faults,
security breaches, or system malfunctions. Traditional rule- A. Paper Organization
based detection techniques often fail to adapt to evolving The remainder of this paper is organized as follows:
data patterns, necessitating advanced machine learning-based
• Section II: Related Work provides a review of deep
solutions.
Anomaly detection in IoT sensor data is critical for ensur- learning, machine learning, and hybrid techniques used
ing system reliability and security. Leveraging deep learning in anomaly detection.
• Section III: Methodology details our proposed model,
models such as autoencoders and recurrent neural networks
(RNNs) allows for the identification of complex patterns and data preprocessing steps, and feature engineering tech-
deviations within data streams. This paper explores a hybrid niques.
• Section IV: Experimental Setup and Evaluation de-
approach that integrates statistical modeling with deep learning
scribes the datasets, evaluation metrics, and performance
Identify applicable funding agency here. If none, delete this. analysis.
 • Section V: Discussion highlights key observations, cur- et al. [9] compared various machine learning-based anomaly
rent limitations, and potential improvements. detection algorithms in Industrial IoT (IIoT) environments,
• Section VI: Conclusion and Future Work summarizes concluding that hybrid approaches outperform standalone
our contributions and outlines directions for further re- models.
search.
D. Advancements and Contributions of This Work
TABLE I
P ERFORMANCE C OMPARISON OF A NOMALY D ETECTION T ECHNIQUES While previous research has demonstrated the effectiveness
FOR I OT S ENSOR DATA
of machine learning and deep learning models in anomaly
Method Accuracy (%) False Positive Rate (%) Computational Cost detection, challenges related to data quality, adaptability, and
Autoencoder 95.2 4.5 Medium
RNN-LSTM 96.8 3.9 High real-time processing persist. This paper builds upon these
Isolation Forest 92.5 6.3 Low
One-Class SVM 90.4 7.2 Medium foundational studies by proposing a comprehensive deep
Hybrid (Proposed Model) 97.3 3.5 Medium-High learning-based approach that integrates feature extraction,
time-series analysis, and unsupervised learning for anomaly
II. R ELATED WORK detection in IoT sensor data. Unlike previous models that
focus solely on either statistical or deep learning techniques,
The integration of machine learning techniques in anomaly our approach leverages the strengths of both methodologies to
detection for IoT sensor data has been a growing focus in enhance detection performance and reduce false positives.
research, aiming to improve detection accuracy and efficiency
By incorporating insights from Srinivasan et al. [1], Jing
through advanced modeling approaches.
et al. [2], Rezakhani et al. [3], Kim and Heo [4], Nayak and
A. Deep Learning for Anomaly Detection Perros [5], Li and Sharma [6], Talayero et al. [7], Subha et al.
Deep learning techniques have demonstrated significant [8], and Naik et al. [9], this work presents a novel framework
capabilities in detecting anomalies in time-series sensor data. that addresses existing challenges in anomaly detection and
Srinivasan et al. [1] applied convolutional neural networks contributes to the advancement of intelligent IoT monitoring
(CNNs) to climate data for anomaly detection, demonstrating systems.
the effectiveness of deep learning in identifying irregular
patterns in sensor readings. Jing et al. [2] expanded on III. M ETHODOLOGY
this by developing a deep neural network-based anomaly
diagnosis method specifically for temperature sensor data, This section describes the methodology used for IoT
improving both accuracy and efficiency. Rezakhani et al. [3] anomaly detection, including data preprocessing, feature engi-
proposed a transfer learning framework for multivariate IoT neering, and model selection. We leveraged machine learning-
traffic anomaly detection, showcasing the adaptability of deep based anomaly detection techniques while incorporating in-
learning models across different sensor environments. sights from previous research and workshop implementations.

B. Machine Learning Approaches for Anomaly Detection

A. Data Collection and Preprocessing
Several studies have explored the use of machine learning
techniques for anomaly detection in sensor networks. Kim To evaluate our model, we used two datasets:
and Heo [4] leveraged feature extraction techniques com- • Dataset 1: Synthetic iot dataset from Anomaly Detection
bined with machine learning models to detect anomalies in in IOT devices (2000 row dataset).
hydraulic system sensor data. Nayak and Perros [5] proposed • Dataset 2: A custom-generated dataset collected from our
an automated machine learning-driven approach for real-time deployed IoT sensors. The dataset is still being expanded
anomaly detection in temperature sensors, emphasizing the as we gather more sensor readings over time.
need for real-time solutions. Li and Sharma [6] utilized
Before training our models, we applied a series of data
deep neural networks alongside cluster analysis to enhance
preprocessing techniques:
abnormal data detection in sensor networks.
• Handling missing values: Missing values in critical sensor
C. Hybrid Approaches for Improved Detection features such as temperature and humidity were replaced
Recent research has highlighted the advantages of combin- using median imputation.
ing multiple machine learning techniques to improve anomaly • Duplicate removal: We identified and removed duplicate
detection accuracy. Talayero et al. [7] proposed a hybrid entries to ensure data consistency.
model using machine learning classifiers for meteorological • Outlier detection: Outliers in sensor readings were de-
anomaly detection, emphasizing the effectiveness of tree-based tected using interquartile range (IQR) analysis. Extreme
classification techniques. Subha et al. [8] developed a data values were replaced with median values to maintain
engineering-driven pipeline integrating stacked LSTM models realistic sensor behavior.
for detecting sensor anomalies, demonstrating the benefits of • Feature scaling: Sensor readings were normalized using
combining deep learning with structured preprocessing. Naik min-max scaling to improve model stability.
B. Feature Engineering IV. E XPERIMENTAL S ETUP AND E VALUATION
Feature selection and transformation play a crucial role in This section presents the experimental setup used to train
improving anomaly detection performance. We extracted time- and evaluate the anomaly detection models. It includes dataset
series features such as: details, model training configurations, and performance met-
• Statistical features: Mean, variance, and standard devia- rics.
tion of sensor values over time.
• Temporal features: Time-based aggregations, such as
A. Experimental Environment
hourly and daily averages.
• Anomaly indicators: Derived binary flags based on devi- The experiments were conducted using the following envi-
ations from expected sensor behavior. ronment:
• Software: Python, scikit-learn, NumPy, Pandas, Tensor-
C. Model Selection and Training
Flow
We experimented with multiple machine learning models • Training platform: Google Colab and Jupyter Notebook
and performed hyperparameter tuning using GridSearchCV to • Hardware: Intel Core i7 processor, 16GB RAM
optimize detection performance. The following models were • Libraries for model evaluation: Scikit-learn metrics, Mat-
evaluated: plotlib for visualization
• Isolation forest (unsupervised): Used for anomaly detec-
tion based on data distribution. Achieved 93.5 percent B. Datasets and Data Preprocessing
accuracy with fine-tuned contamination parameters.
• Decision trees (supervised): Trained with labeled We used two datasets in this study:
anomaly data, yielding 94.8 percent accuracy with re- • Dataset 1: A labeled IoT anomaly dataset from Kaggle,
duced false positives. containing sensor readings with predefined anomalies.
• Hybrid model (proposed approach): Combined isolation • Dataset 2: A custom IoT dataset collected from real
forest and decision tree for a balanced trade-off between sensors deployed in an experimental environment. This
false positives and true positive rate. This hybrid approach dataset is still expanding as more sensor readings are
resulted in a detection accuracy of 96.2 percent. gathered.
D. Evaluation Metrics To ensure high-quality data, we applied preprocessing tech-
niques:
Model performance was evaluated using the following met-
rics: • Handling missing values: Imputed using median values
for numerical sensor readings.
• Accuracy: Measures overall correctness of predictions.
• Outlier detection and removal: Used interquartile range
• False positive rate (FPR): Important for reducing unnec-
(IQR) to detect and correct extreme sensor values.
essary alerts in IoT applications.
• Feature scaling: Min-max normalization applied to stan-
• Precision, recall, and F1-score: Used to measure anomaly
dardize sensor data.
detection effectiveness.
• Data splitting: 80 percent of the data was used for training
E. Implementation Setup and 20 percent for testing.
The models were implemented using Python with additional
cryptographic libraries for secure anomaly detection. The C. Model Training and Hyperparameter Tuning
experimental setup included: The models were trained using labeled and unlabeled IoT
• Frameworks: scikit-learn, NumPy, Pandas, and Tensor- data to detect anomalies. We evaluated different machine
Flow. learning approaches:
• Training environment: Google Colab and local Jupyter
• Isolation forest (unsupervised): Used for anomaly detec-
Notebook. tion in unlabeled data.
• Hardware specifications: Intel Core i7 processor, 16GB
• Decision tree (supervised): Trained using labeled anoma-
RAM. lies to classify normal and anomalous sensor readings.
F. Summary • Hybrid approach (proposed model): Combined the
strengths of both models for enhanced performance.
This methodology demonstrates an effective and scalable
anomaly detection framework tailored for IoT sensor networks. To optimize performance, hyperparameter tuning was per-
The hybrid approach significantly improves detection accu- formed using GridSearchCV:
racy, leveraging unsupervised learning for anomaly detection • Isolation forest: Tuned contamination rate, number of
and supervised learning for classification refinement. In the estimators, and maximum depth.
next section, we present experimental results and an analysis • Decision tree: Optimized using maximum depth, mini-
of model performance. mum samples per split, and Gini impurity criterion.
D. Performance Metrics R EFERENCES
The models were evaluated based on: [1] R. Srinivasan, L. Wang, and J. Bulleid, “Machine learning-based climate
time series anomaly detection using convolutional neural networks,”
• Accuracy: Measures overall correctness of anomaly de-
Weather and Climate, vol. 40, no. 1, pp. 16–31, 2020. [Online].
tection. Available: https://www.jstor.org/stable/10.2307/27031377
• False positive rate: Ensures minimal false alarms in IoT [2] W. Jing, P. Wang, and N. Zhang, “Study on temperature sensor data
anomaly diagnosis method based on deep neural network,” Scientific
environments. Programming, vol. 2022, pp. Article ID 9 662 374, 8 pages, 2022.
• Precision, recall, and F1-score: Assesses model effective- [3] M. Rezakhani, T. Seyfi, and F. Afghah, “A transfer learning framework
ness in distinguishing anomalies. for anomaly detection in multivariate iot traffic data,” IEEE Transactions
on Networking, pp. 1–12, 2025.
E. Experimental Results [4] D. Kim and T.-Y. Heo, “Anomaly detection with feature extraction based
on machine learning using hydraulic system iot sensor data,” Sensors,
Table II summarizes the model performance metrics. vol. 22, no. 2479, pp. 1–24, 2022.
[5] D. Nayak and H. Perros, “Automated real-time anomaly detection of
temperature sensors through machine-learning,” International Journal of
TABLE II
Artificial Intelligence, vol. 13, no. 1, pp. 9–22, 2024.
P ERFORMANCE C OMPARISON OF A NOMALY D ETECTION M ODELS
[6] M. Li and A. Sharma, “Abnormal data detection in sensor networks based
Model Accuracy (%) Precision (%) Recall (%) F1-score (%) on dnn algorithm and cluster analysis,” Journal of Sensors, vol. 2022, pp.
Isolation forest 93.5 91.2 90.8 91.0 1–7, 2022.
Decision tree 94.8 92.5 91.9 92.2 [7] A. P. Talayero, N. Y. Yürüsen, F. J. S. Ramos, and R. L. Gastón,
Hybrid (proposed) 96.2 95.0 94.6 94.8 “Machine learning based met data anomaly labelling,” in Journal of
Physics: Conference Series, vol. 2257. IOP Publishing, 2022, p. 012015.
[8] Subha, A. Patel, and Saranraj, “A multi model approach: A data engi-
F. Discussion neering driven pipeline model for detecting anomaly in sensor data using
stacked lstm,” International Journal for Research in Applied Science and
The experimental results indicate that: Engineering Technology, vol. 11, no. V, pp. 2800–2802, 2023.
• The hybrid model outperforms individual models in ac- [9] B. N. D. S., V. Dondeti, and S. Balakrishna, “Comparative analysis of
machine learning-based algorithms for detection of anomalies in iiot,”
curacy and recall. International Journal of Information Retrieval Research, vol. 12, no. 1,
• False positives were reduced significantly using the hy- pp. 1–20, 2024.
brid approach.
• Real-world IoT data improves model robustness but re-
quires further expansion for generalization.
G. Summary
This section detailed the experimental setup, dataset pre-
processing, model training, and evaluation metrics. The re-
sults demonstrate that a hybrid approach enhances anomaly
detection accuracy while minimizing false positives. Future
work will explore additional dataset expansion and real-time
deployment strategies.

Fig. 1. Performance comparison of anomaly detection models.

ACKNOWLEDGMENT
The authors would like to thank Al Akhwayn University
in Ifrane for providing computational resources and support
during this research. Additionally, we acknowledge the contri-
butions of all the authors for their assistance in data collection
and preprocessing. This research was conducted as part of a
class project.