INNOVATIVE DIVER DROWSINESS

DETECTION USING IOT AND DEEP

LEARNING
B.Devakrsihnan G.Hiran Krishna Mr.R.Sundar
Department of Computer Science and Department of Computer Science and Department of Computer Science and
Engineering Engineering Engineering
Sathyabama Insitute of Science Sathyabama Institute of Science Sathyabama Institute of Science
and Technology and Technology and Technology
Chennai, India Chennai, India Chennai, India
[email protected] [email protected] [email protected]

Abstract—The detection of driver drowsiness is one of the detection techniques, hybrid approaches, have been proved
most important part related to road safety and has been very more accurate and more robust.
widely explored using different methods. Recent developments
make use of technologies such as monitoring eye activity, deep Although most of the techniques are proposed for
learning algorithms, or non-intrusive techniques for the identifying driver drowsiness, further study is required in
accurate identification of drowsy states. In this respect, various terms of maximizing the detection accuracy, guaranteeing
techniques reached high accuracies—often more than 97%— real-time processing, and addressing the privacy concerns. In
through the use of CNN, RNN with LSTM units, or hybrid this paper, the authors review driver drowsiness detection
models. The non-intrusive approaches include the latest technologies with a focus on recent innovations, analyse their
computer vision techniques and IoT-based frameworks that effectiveness, and show areas for future research and
ensure driver comfort and privacy while providing real-time development.
monitoring. Integrating multimodal data sources and edge
computing further enhances the detection capabilities. All the
same, much work is left to be done in this line of work to satisfy II. LITERATURE REVIEW
the optimization of real-time processing and privacy concerns, Ahmet Kolus [1] proposed a model for develop a fully
hence a pointer that more studies and development are needed functional system for driver drowsiness detection that shall
for a more robust and scalable solution integrate activity measures for the eyes with the latest
technologies. This systematic review of existing methodology
Keywords—CNN, Image Recognition, ResNet, Drowsiness and approaches shall help in integrating new insights into their
Detection, Deep Learning enhancement for methods of early detection and prevention,
in order to improve road safety by avoiding drowsiness-
I. INTRODUCTION related accidents
Driver drowsiness is among the top rated or leading causes Alexey Kashevnik [2] This project aims to contribute to
of road traffic accidents, which severely threatens road safety road safety by tackling driver distraction, making a critical
and human lives. Since driving is related to continuous review of the existing methods for its detection and their
attention and a fast response rate, due to a decrease in attention integration into a general framework. The work considers
and alertness, the driver will be unable to respond in time to manual, visual, and cognitive distractions and provides an
any unexpected situations that may result in fatality. This ordered summary of approaches and technologies applied.
difficulty in detecting the drowsy drive has provoked massive Other than practical implications, this work enhances the
research into developing an efficient monitoring system. They quality of driver distraction/inattention detection systems.
are usually intrusive, based on physiological signals, or non- Ayman Altameema et al [3] it enhances the safety of a road
intrusive, directly observed, with limitations in their scope. by developing a system that detects drowsiness and emotional
Recent advances in technology have us here in more states of drivers through real-time image segmentation and
sophisticated approaches to drowsiness detection, involving machine learning. By face expression analysis using SVM,
measures of eye activity, such as blinking rate and duration of the system will capture facial expressions associated with
eyelid closure, with machine learning techniques. Deep fatigue, anger, and inattentiveness and alert the driver; it is
learning algorithms, particularly convolutional neural also capable of automatically adjusting vehicle control to
networks and recurrent neural networks with long short-term prevent an accident in real-time.
memory units, performed very well in detecting the first signs Bao Ma et al [4] presents an improved YOLOv8 detection
of drowsiness. Computer-vision-based non-intrusive methodology for the detection of distracted driving
approaches by facial recognition and emotion detection can behaviours and driver emotions. In this study, it incorporates
provide very friendly privacy without sacrificing high the MHSA attention mechanism into the CNN module for
detection accuracy. better accuracy and convergence. Trained on the FER2013
The combination of Internet of Things and edge dataset and tested on Jetson Nano, it performed better in real
computing has been used to solve the problem of real-time time and accuracy, maximizing volume and speed.
processing in a drowsiness detection system without Carmen Bisogni et al [5] proposed a model for identify
scarifying driver comfort. Multi-sensors with multiple how Industry 4.0 technologies can be applied to help ensure
workplace safety through worker drowsiness monitoring. The respectively, thereby outperforming the approaches proposed
research focuses on the feasibility of existing biometric earlier.
technologies within real-world use cases, researching their
constraints and precision. The goal is to identify gaps and N. Sukumar et al [11] proposed a model for state detection
propose solutions for their effective integration in large-scale system combines physiological and physical sensors for
safety applications. estimating the driver's stress level, drowsiness, and behavior.
It would include ECG recording for stress estimation using
Hanane Lamaazi et al [6] This paper has proposed synchro squeezing-continuous wavelet transform, accelerator
Driver Drowsiness Detection System using smart edge data for drowsiness detection, and an IDT capacitive sensor
computing to resolve public safety concerns due to drowsy for hand position on the steering wheel. The proposed
driving. The suggested system, as represented in Fig. 4.1, system has an accuracy of 91.82% in driver stress detection
delivers driver privacy while detecting drowsiness through and 97.30% in driver drowsiness detection. This, therefore,
mobile devices for real-time analysis by adopting a distributed shows a very huge improvement in detection compared to
edge architecture with Main and Local Edge Nodes. The previous methods on the capture of distinct driver states.
proposed framework combines facial expression analysis via
CNN and car path monitoring through YoLov5 with a two- Shahzeb Ansari et al [12] proposed a model for mental
layer LSTM algorithm, which resulted in drowsiness detection fatigue detection in a driver that is necessary for prevention
accuracy of 97.7%. against the occurrence of crashes due to sudden changes in
real-world driving. This paper thus proposes a novel way of
M. Adil Khan et al [7] This document presents the estimating mental fatigue and drowsiness by monitoring
development of a non-intrusive Internet of Things (IoT) head position measurements using the XSENS motion
framework aimed at monitoring driver behavior to enhance capturing system. The proposed method makes use of 15
road safety in logistics and public transport. It features an subjects in a driver-in-the-loop simulator of MATHWORKS
integrated system that includes embedded technology, edge and further makes use of the bidirectional long short-term
computing, and cloud modules, complemented by a mobile memory system, which is the modified system in question,
application that detects drowsiness, sleep, yawning, and with a rectified linear unit layer.
distraction through image processing.. The design mitigates
latency and data loss through edge computing and provides V. Vijaypriya et al [13] the increasing number of mishaps
real-time monitoring with 96% accuracy. This will deliver a due to driver drowsiness has underlined the need for detecting
robust, portable, and user-friendly solution for the continuous systems. This paper presents an MCNN framework for the
evaluation of drivers for safety improvement. classification of drowsiness on video sequences extracted
from YAWDD and NTHU-DDD datasets. In this proposed
Monagi H. Alkinani et al [8] this paper detects abnormal approach, keyframes are extracted and further pre-processed
driving behaviours, including driver distraction, driver by the Dlib library for facial point localization and Cross
fatigue, and aggressive driving behavior, with the help of Guided Bilateral Filtering on images. The features are
deep learning algorithms. Reasons for these behaviors—the extracted using hybrid dual tree complex wavelet transforms
driver's experience, demographics—are classified in this combined with Walsh-Hadamard transforms.
paper. The present work offers a comparative review of deep
learning-based systems and techniques while presenting Vishnu Yarlagadda et al [14] proposed a model for
fundamental requirements and open research challenges for transition state from wakefulness to sleep that, owing to a lack
the improvement of detection and understanding of human of focus and fatigue, normally leads to serious accidents. In
driver inattentive and unusual behaviors. this paper, the approach was made to detect drowsiness
through facial parameters from eye and mouth regions. Hence,
Mohammad Shahbakhti et al [9] the paper presents the the approach provides the ability for attaining an accuracy of
enhancement in the driver drowsiness detection using 97.25% using RNNs and LSTM units, which have been
features related to blink derived from EEG data. In this effective in dealing with sequential multimedia data. The
paper, an algorithm is proposed for continous extraction and proposed technique exploits the power of computer vision
removal of eye-blink features from low-channel prefrontal and deep learning in the execution of effective driver
EEG. Following blink intervals identification, filtering with drowsiness detection.
PCA and wavelet transforms, and states classification with
SVM, this algorithm realized a high accuracy improvement of Yashar Jebraeily et al [15] This proposed model is for
78.1% against previous methods of 71.2%. This approach Several traffic accident reports have indicated that drowsiness,
emphasizes the fact that eye blinks act both as an fatigue, and lack of alertness are major factors that cause
informational component and as possible interference in EEG- accidents due to slow reactions and unsatisfactory driving
based schemes. conditions. This paper proposes a neural architecture search
mechanism for the design of the optimal CNN to identify
Muneeb Ahmed et al [10] this paper presents a novel driver drowsiness. Key contributions are the enrichment of
ensemble deep learning architecture to overcome the datasets related to drowsy driving by video frame labeling, a
limitations of the automated drowsiness detection systems. genetic algorithm for CNN structure optimization, and
For the integration of features from eyes and mouth transfer learning to improve feature extraction
subsamples, such a system will be created by applying two
InceptionV3 modules in parallel for efficient feature III. METHODOLOGY
extraction. These features are combined through a weighted
average method for the determination of driver drowsiness. In this work, sensor-based systems that use ECG,
This model was trained on the NTHU-DDD dataset and accelerometers, and capacitive sensors for measuring
achieved high training accuracy, validation accuracy, and physiological and physical indicators; computer vision
evaluation accuracy of 99.65%, 98.5%, and 97.1%, techniques for facial expression and eye movement analysis;
and state of the art deep learning techniques. It utilizes time-
frequency techniques like CWT-SST for heart-rate variability, states, there remain significant gaps in applying these
machine learning algorithms like SVM for the drowsiness technologies on a large scale.
classification, and ensemble models coupled with neural
networks for feature extraction from simple statistical and
time-frequency methods. The methodologies include low-
channel EEG with regard to the extraction of features related
to blinks, as well as filtering; optimization of YOLOv8 by
attention mechanisms and CNNs in emotion detection; and
facial expression analysis through real-time image
segmentation using machine learning. Other methods include
non-intrusive IoT frameworks that combine edge computing
and cloud systems, motion capture systems with LSTM
networks for head postures analysis, and convolutional neural
networks optimized using genetic algorithms and transfer
learning for the classification of drowsiness.

Fig. 2. Enhanced architecture of MCNN with FSA.

The convolution layer in CNN applies a series of filters to

input data, creating feature maps, which in truth means
processing the feature obtained with an activation function,
such as 𝜑(𝑥)=1.7159×tanh(𝑥) to represent non-linear
activation. Following the overall architecture, there is then a
max-pooling layer that actually reduces the feature maps
dimensionality provided by selecting the maximum value
from the overlapping regions, therefore supplying
translation invariance. Lastly, the classification layer
converts the downsampled feature maps into a 1D vector

IV. RESULT AND CONCLUSION

The new system combines deep learning and IoT technology
to detect drowsiness with high accuracy and reliability. Tests
show that using computer vision along with data from
multiple sensors improves detection rates. By using edge
computing, the system reduces delays and sends real-time
alerts to drivers. It works well in different situations, such as
changes in lighting or with different types of drivers.

Fig. 1. Flowchart of the drowsiness detection system

[3] methodology is outlined in a four-step process for
detecting driver fatigue using a prototype system. First, the
method is introduced with preparation activities. Next, eye-
state examination is performed to track eye movement and
assess drowsiness. The third step involves nod analysis to
detect signs of falling asleep. Finally, a warning stage issues
alerts if drowsiness is detected. The system employs non-
intrusive analysis of external signals, transitioning from a
proof of concept model to a stable software framework,
laying the groundwork for future research and development.

A. Work flow
Multi-scale Convolutional Neural Networks (MCNN)
effectively detect driver drowsiness by using convolutional Fig. 3. Detection results
and pooling layers for feature extraction, followed by fully
connected layers for classification. The analysis reveals that However, there are still some issues to address. Real-time
while advancements in sensor technology, data analytics, monitoring raises privacy concerns, so strong encryption and
and automation have improved our ability to monitor data anonymization methods are needed. Also, for large-scale
driver
use, like in public transportation, affordable hardware is JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS,
VOL. 26, NO. 3, MARCH 2022
necessary. Future versions of this system should focus on
[10] Muneeb Ahmed et al “Intelligent Driver Drowsiness Detection for
these areas to make it more user-friendly and widely adopted. Traffic Safety Based on Multi CNN Deep Model and Facial
Subsampling,” IEEE access, IEEE TRANSACTIONS ON
V. CONCLUSION AND FUTURE WORK INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 23, NO. 10,
OCTOBER 2022
Deep learning and IoT technologies have impacted the [11] N. Sukumar et al “Physiological and Physical Sensors for Stress
Level,Drowsiness Detection, and Behaviour Analysis,” IEEE
domain of driver sleepiness detection more constructive in TRANSACTIONS ON CONSUMER ELECTRONICS, VOL. 70, NO.
terms of accuracy, real-time operations, and user experience. 1, FEBRUARY 2024
The paper successfully integrates RNNs, IoTs, and CNNs, [12] Shahzeb Ansari et al “Driver Mental Fatigue Detection Based on Head
providing a robust and scalable solution for sleepiness Posture Using New Modified reLU-BiLSTM Deep Neural Network,”
IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION
detection. Edge computing is enhancing the real-time SYSTEMS, VOL. 23, NO. 8, AUGUST 2022
features of systems and processes by augmenting the already [13] V. Vijaypriya et al “Facial Feature-Based Drowsiness Detection With
low latency. Multi-Scale Convolutional Neural Network,” IEEE access DOI
10.1109/ACCESS.2023.3288008
However, it’s notable that there are many areas that are still [14] Vishnu Yarlagadda et al “Driver Drowsiness Detection Using Facial
not covered such as large scale deployment and privacy and Parameters and RNNs with LSTM,” 2020 IEEE 17th India Council
security. It can be stated that more research is to be conducted International Conference (INDICON) | 978-1-7281-6916-3/20/$31.00
©2020 IEEE | DOI: 10.1109/INDICON49873.2020.9342348
on Algorithm Optimization: creation of light models that can
[15] Yashar Jebraeily et al “Driver Drowsiness Detection Based on
work with edge gadgets while remaining accurate. Privacy Convolutional Neural Network Architecture Optimization Using
Assurance: Attaching high end encryption and Genetic Algorithm,” IEEE access, DOI
anonymization techniques to protect user information. Cost 10.1109/ACCESS.2024.3381999
Efficient Hardware: Developing affordable hardware for
integration and use in cars. In-Situ Testing: Conducting more
tests with different populations and locations to strengthen
the system. Multimodal Integration: Adding other sources
such as voice recognition or car telemetry information to
compensate for lack of accuracy. Work to improve and
mitigate areas will provide scope to build more efficient and
easily deployable drowsiness detection systems that will lead
further.

REFERENCES
[1] Ahmet Kolus, “A Systematic Review on Driver Drowsiness
Detection Using Eye Activity Measures,” IEEE access, DOI
10.1109/ACCESS.2024.3424654
[2] Alexey Kashevnik, ” Driver Distraction Detection Methods:A
Literature Review and Framework,” IEEE access, DOI
10.1109/ACCESS.2021.3073599
[3] Ayman Altameema et al, “Early Identification and Detection of Drive
Drowsiness by Hybrid Machine Learning,” IEEE access, DOI
10.1109/ACCESS.2021.3131601
[4] Bao Ma et al “Distracted Driving Behaviour and Driver’s Emotion
Detection Based on Improved YOLO With Attention Mechanism,”
IEEE access, DOI 10.1109/ACCESS.2024.3374726
[5] Carmen Bisogni et al “Drowsiness Detection in the Era
Of Industry 4.0: Are We Ready?,” IEEE access, TRANSACTIONS
ON INDUSTRIAL INFORMATICS, VOL. 18, NO. 12, DECEMBER
2022
[6] Hanane Lamaazi et al “Smart Edge-Based Driver Drowsiness
Detection,” IEEE access, DOI 10.1109/ACCESS.2023.3250834
[7] M. Adil Khan et al “IoT-Based Non-Intrusive Automated Driver
Drowsiness Monitoring Framework for Logistics and Public Transport
Applications to Enhance Road Safety,” IEEE access, DOI
10.1109/ACCESS.2023.3244008
[8] Monagi H. Alkinani “Detecting Human Driver Inattentive and
Aggressive Driving Behavior Using Deep Learning: Recent Advances,
Requirements,” SPECIAL SECTION ON ARTIFICIAL
INTELLIGENCE (AI)-EMPOWERED, IEEE access, DOI
10.1109/ACCESS.2020.2999829
[9] Mohammad Shahbakhti et al “Simultaneous Eye Blink
Characterization and Elimination From Low-Channel Prefrontal EEG
Signals Enhances Driver Drowsiness Detection,” IEEE access, IEEE