2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON), 26-27 May 2022

Eye Blink Detection for Drowsy Drivers

2022 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON) | 978-1-6654-9602-5/22/$31.00 ©2022 IEEE | DOI: 10.1109/COM-IT-CON54601.2022.9850475

Yanshee Sharma, Vanshika Bohra, Ananya Khurana, Suresh Kmuar, Savita Sindhu, Kamlesh Sharma, Monica
Department of Computer Science and Engineering, Faculty of Engineering and Technology
Manav Rachna International Institute of Research and Studies, Faridabad, India
[email protected], [email protected], [email protected],
[email protected], [email protected], [email protected], [email protected]

Abstract: In recent times, road causalities caused by driving cars consistent comparisons of the driver's facial expressions
have increased gradually. These accidents result in severe further help in calculating the level of drowsiness using an Eye
injuries and deaths, evolving as a serious social and traffic issue. Aspect Ratio (EAR) threshold value. The GPS has been used
As a matter of fact, fatigue and drowsiness amongst drivers has to locate nearby trauma centres so that immediate help can be
been detected as the primary reason behind such causalities.
provided to the driver if he meets any road crash and to avoid
Therefore, it is essential to design and implement a system that
can fix this issue using known techniques. In this paper, a such crashes primarily, voice speakers have been used to
computer-based process that uses camera to detect the driver's alarm the drowsy driver as soon as possible and when
facial expressions and his eye blink rate to detect if the driver is detected.
under fatigue or drowsy. Once a drowsy driver is detected, IoT is
used to issue warning message for the driver, thereby alerting II. LITERATURE REVIEW
him using an inbuilt voice speaker. According to traffic accidents reports, drowsy drivers are four
Keywords: Drowsy drivers, detection techniques, eye blink to six times more responsible for road crashes than active
detection, EAR ratio drivers. There is clear evidence [3, 8] for sleep-related effects
that produce impaired performance and hence, accidents. Also,
I. INTRODUCTION Li et al. [5] has stated that drowsiness can be considered a
Drowsy driving and driver fatigue are the main contributors to major reason for inadequate performance abilities due to
fatal road collisions. It refers to the driver's condition where he which drivers can have a hard time focusing or their driving.
is tired, drunk, or sleep-deprived, due to which he lacks
alertness and leads to motor vehicle collisions. Studies have Drowsiness detection technologies and methodologies are
proved that drowsy driving can affect the human brain as mainly categorized under three major categories: physiological
much as alcohol does [1]. Road traffic crashes lead to nearly methodology, vehicle-based methodology, and computer
3200 deaths each day worldwide and about 1.36 million every vision methodology [18]. The former of these three
year, as revealed by National Highway Traffic Safety measurement techniques focus on detecting a drowsy driver by
Administration (NHTSA) [1]. NHTSA also calculated that tracing and monitoring certain internal signals like variations
sleep-deprived drivers' driving led to 2.4 percent of fatal traffic in the driver's heart rate and brain activity using psychological
accidents in the USA in the last decade. Alongside, Korean sensors. However, these sensors can be a great distraction
reports on motor vehicle accidents suggest that drowsy driving themselves, making this methodology less feasible [6].
is one of the top reasons behind car accidents and sudden
deaths [2]. Secondly, the vehicle-based methodology is one of the
traditions ones and focuses on calculating the number of driver
There can be many ways to detect and avoid drowsiness behavior metrics like lane deviations and steering wheel
amongst drivers, and one of the most natural is manual movements. Finally, computer vision methodology uses
detection by the driver himself. However, this self-assessment Artificial Neural Networks(ANNs) to solve many road crash
can prove to be an excellent coping mechanism initially. Still, issues like at toll plazas, traffic safety analysis, and
it cannot be relied upon to get completely rid of drowsy identification of driver's behavioral changes [6,9]. In recent
driving-related road accidents [5]. This leads to the urge and years, drowsiness detection amongst drivers has become a
needs to create a drowsiness detection technique that is major attraction center of attraction, drawing focus from object
reliable enough and produces warning alarms to warn/wake recognition and the computer vision community. Park [22]
the driver and locate the nearest trauma centers for immediate took a significant step at designing an automated system for
help to the driver in case of any mishappening. the same by using ensemble strategies along with Flow
ImageNet, Alex Net, and VGG-Face Net to classify whether a
The paper describes one such technique for detecting the level driver is drowsy or not using his input video. Similarly,
of driver's drowsiness using an existing and dependable research has developed a 3D deep neural network and
algorithm, the eye blink detection algorithm. It has been used boosting framework-based detection mechanisms for exposing
to detect the eye position of drowsy driver and the number of drowsiness [21].
blinks made per unit time. These number of blinks along with

978-1-6654-9602-5/22/$31.00 ©2022 IEEE 534

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.
The eyeblink detection technique is a computer vision This technique is used in this paper to monitor the behavior of
technique that calculates the number of blinks in a specified the driver by measuring his eye blinks. Continuous
amount of time [13]. video/image capturing is required for the proposed solution so
that the system can compare the eye state. Eye detection is
accelerated by using the distance (of eye/head movement)
using the Euclidean formula to check if the driver is falling
asleep or lacking alertness. Along with this, car sensors and
GPS modules must be installed so as to sense any car crash
and provide the required aid by tracking the driver's current
location [13, 18].
III. CASE STUDIES
A. Hyundai initiative
Hyundai Mobi's system is an excellent initiative for detecting
drowsiness amongst drivers. They aim to do so by detecting
the driver's face, recognizing it, and monitoring its breathing
patterns. An alarm beeps as soon as the driver's eyes stop
staring/focusing in the front. This alarm is designed to be
activated as per eyelid's blink time and speed. Surveillance
cameras are used to detect the driver's facial parts like eyes,
nose, and mouth, which further help in providing statistical
data. Inspired by this, other domestic brands of Korea have
established various techniques such as giving an alarm to the
driver using car sensors. When the gap between his car and the
front car is less, the alarm beeps to alert the driver to stop/put
brakes and avoid any mishappening. Another technique makes
use of the "tilt and vibration sensor" which is placed on the
driver's ear flap. It generates a vibration as and when a specific
(a) tilt value is detected.
B. Toyota's lexus
A well-known Japanese brand, Toyota came up with a brand
new Lexus car that had inbuilt drowsiness detection
protection. There, infrared cameras were placed behind the
steering wheel to detect driver's drowsiness and hence sound a
warning in return. They use about 240 camera points and a
thousand meshes. These mesh points are used to extract
information about the driver's expressions. Apart from this,
they use numerous factors to increase the accuracy of results.
IV. TECHNIQUES
A. Face and Eye Detection by Machine Learning (ML) and
Deep Learning (DL) Algorithms. Jabbar et al.[19] come up
with a Convolution Neural Network algorithm(ConvNet).
This technique is a machine learning-based algorithm to
detect drowsiness & microsleep conditions. The fatigue &
drowsiness of the driver is seen with the help of facial
landmarks. These landmarks are achieved by passing the
input image through this ConvNet algorithm. Along with
the algorithm, several data sets such as with and without
spectacles, in day and night are used for experimental
classification. Thus, it ensures high precision under all
conditions. Saleh et al. also developed the LSTM
(b) algorithm and RNN, i.e., Recurrent Neutral Network. This
classifies the driver's actions with the help of sensors.
Fig 1: Flowchart of detection
(a) Drowsiness (b) Collision [32]

535

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.
B. Eye Recognition System Based on Wavelet Network E. Fatigue and collision detection using smart devices:
Algorithm. Wavelet Technique is used to warn the drowsy Smart techniques refer to the methods that automatically
drivers. This technique tends to track the driver's eye can take action on finding drowsy levels of the driver.
movements using Wavelet Network Classyfying Earlier, a smart glass was used to detect drowsiness and
Algorithm. This algorithm further relies on Fast Wavelet fatigue. If it is seen, a message is sent with flashing of rear
Transform based on a binary decision. The system uses 6 lights automatically using cloud environment or IoT
points for eye detection and several points, called mesh module. A better system was then proposed which used
points, to detect the diver's facial behavior and use Eye eye blinking sensors to detect fatigue and drowsiness along
Aspect Ratio (EAR) to calculate driver's drowsiness rate. with detecting road collisions with the help of vibration
EAR gives an estimated value that tells whether the driver sensors.
is drowsy, nearly drowsy or in conscious state. The
wavelet technique is further used to calculate psychology In this model, the GPS module is integrated with the sensors
aspects, which are the rate of heart beat and to track the driver's location in case of any mishappening or
electrocardiogram and are extracted using this collision. The GSM module trasmits messages to the drowsy
transformation technique[18].
driver's contacts to inform them. Although Arduino board with
sensors can be used to control the cause of such traffic
C. Fatigue Detection Using Vehicle State (Steering Wheel) accidents, Jang and Ahn [30] proposed a system that can
Algorithm. The system detects drowsiness amongst drivers detect alcohol addict drivers and their drowsiness conditions
based on steering wheel information that uses neuro-fuzzy with the help of Raspberry Pi controller module integration. In
inference mechanism and particle swarm algorithm and addition to this, IoT modules in combination with WebCam is
vector machine. However, this method alone results in to be used to report this abnormal behaviour of the driver.
false detection results. To resolve this issue, Mutya Further, to improve the alert processing, voice recognition and
implemented a system that uses the steering wheel machine learning processes are applied.
algorithm and is made on the pictorial or picture formed
steering movement. In integration to this, the Convolution V. COMPONENTS
Neural Network is applied for properly classifying The proposed system is made up of the following primary
drowsiness conditions. components [2].
D. Drowsy Alert System Designed Using A. Pi Camera Module V2. This Raspberry Pi Camera module
Electroencephalography (EEG), Electrocardiography v2 can be used instead of the original camera module to
(ECG), Electrooculogram (EOG), and Electromyogram capture high-definition video with still picture with the
(EMG) Algorithm. Budak et al. [26] designed drowsiness help of Sony IMX219 8-megapixel sensor [23] . It operates
detection using ECG that included components such as with 1080p30, 720p60, and VGA90 video modes which
VGGNet method, AlexNet method, and Wavelet transform connect to a CSI port via a 15 cm ribbon cable on the
algorithm to analyse the state of drowsiness through Raspberry Pi module.
camera, brain indicator signal and sensors (activated using
ML techniques to alert the driver). Waleed and Hayawi
[16] have made efforts to detect drowsiness with the help
of variations in Heart Rate and Electrooculogram to
measure eyball movement and construct Arduino
controller embedded alert system that gives an alert
message when the driver is detected under fatigue/drowsy
condition.

Electromyogram (EMG) sensors signals are used to Fig 2: Pi Camera Module V2 [32]
measure the movements of muscles and eyelid closure.
These signals use ESP8266 to function and monitor
Internet's drowsy data. Electroencephalography (EEG) B. Raspberry Pi3 Model B Module. It is a credit card-sized
signals provide robust platform for detection of drowsiness computer which is embedded with Quad Core 1.2 GHz
using Deep learning to ensure good accuracy. Broadcom BCM2837 64-bit CPU, 1 GB RAM, and other
Deep learning is also used to observe the facial parts. Raspbian OS is integrated, and its microprocessor is
movements through AdaBoost Classifier and Multiblock- designed for the Windows operating system [23].
local-binary-patterns to detect the drowsiness level using
fuzzy inference. However, drivers wearing glasses create a
problem in accurate monitoring. For this reason, IoT
modules make the system more intelligent and improve
accuracy of drowsy detection.

536

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.
 from the facial image. Afterwards various points on the
eyes are marked which are then joined in a manner to draw
vertical & horizontal lines. These lines are used to calculate
Eucledian distance which in turn helps in the deduction of
EAR values. If the EAR value comes out to be greater than
0.25, which implies person is drowsy else stated as non-
drowsy. If the drowsiness of driver is detected, then a
required amount of voltage is supplied as an alert to
generate regular programmed voice sound using the
Fig 3: Raspberry Pi3 Module [32] speaker. In order to make system more effective one more
measure is taken which is in the form of electronic
C. Speaker. It is an audio- or voice-generating device which equipment which are inserted that provides safety and
can convert electromagnetic waves into sound. If the measure the impact through vibrations. So as to provide
driver's drowsiness is detected, then a voltage is supplied immediate medical treatment to prevent any further mishap
as an alert to generate regular programmed voice sound. GPS module is embedded in the system so that mishap
location is easily traced & can be sent to the concerned
person.

Fig 4: Speaker [32]

D. Crash Sensor. It is an electronic equipment that provides

safety and detects an impact through vibrations. Due to
drowsiness, any type of crash may happen; then, it detects
the impact of collision.

E. Force Sensitive Resistor Sensor (FSR). Special types of

resistor known as force sensitive resistor, whose
resistance may di er, according to pressure or force on it
[28]. So, it is made by a thick polymer sheet which is
operated in di erent environments. Due to good shocking
resistance, it is integrated in this system.

F. GPS Module. This module is used to track the vehicle's Fig 5: Facial Coordinates Representation [31]
location, if any serious collision occurs with a drowsy
driver. It is the main component to detect the latitude and
longitude of any area on the Earth with date and time
from satellite. In this system, the mishap location is easily
traced and the location is forwarded using GPS [32].

VI. IMPLEMENTATION
Here, the detailed working proposed framework for driver
drowsiness detection is presented. A brief explanation of
the process for detection of driver's communication is as Fig 6: Eye Points (6 coordinates of the opened eye) [32]
follows. Raspberry pi camera is mounted on the dashboard
to capture video & still images of the person's eye sitting at VII. RESULT
the driving seat is captured. Several points are marked on
the face to which face detection by Machine Learning and The section envisions the proper consequence of the
deep algorithms are applied to calculate accurate alignment successfully imposed proposed method while driving the
of face to predict drowsiness. We have used Open CV vehicle. The driver's drowsiness is detected with the help of
library here for image processing & Computer Vision tasks. EAR observations of the condition of driver. It provides the
In our case, the area of interest from the face is extracted

537

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.
result of the eye's condition, opening, and closing and provides This proposed model detects the drowsy state of the eye and
data about their drowsiness and its impacts. sends a warning message. But the slow repercussions of the
driver may result in a fatal accident. To avoid this, a system
synchronized with the vehicle speed can be developed. The
system itself slows the speed of the vehicle after a warning
alert.

Technological development and a linked application for the

registered vehicles for the owners can be developed showing
the history of the driver, the alert messages sent to the driver
during any journey can be developed for the proper tracking.

The development of these applications and synchronized

systems will help decrease the fatal accidents occurring around
the globe.

IX. CONCLUSION
(a)
In this paper, we have evaluated the potential of eyeblink
detection algorithm to identify a driver's sleepiness or
fatigue condition by visualizing the signals from the eyes
with the help of image capturing & video recording using
Raspberry pi camera rather than usual cameras so as to get
more defined & still pictures. This study gives more
defined results as it involves constant monitoring, which
allows the test to be performed for a longer duration. From
future aspects the aim may be of increasing the validity and
applicability of the system is being presented here and
extending the implementations to evaluate more diverse
datasets to check the adaptability of the proposed
framework. We can also try to expand the basic structure of
the framework to be used in microcomputing systems to
reduce the operating and financial cost for a better system.
(b)

Fig. 7. EAR mapped (a) when eyes are open (b) when eyes are
closed [32] REFERENCES
[1] Singh, S. Critical Reasons for Crashes Investigated in the National
Drowsiness Prediction: EAR values are calculated using the Motor Vehicle Crash Causation Survey; Technical Report; (Traffic
Safety Facts Crash•Stats. Report No. DOT HS 812 115); National
movement of the eyelids. The frequency of change is the EAR Highway Traffic Safety Administration: Washington, DC, USA, 2015.
values are at a higher pace, which is determining if the driver [2] National Center for Statistics and Analysis. Drowsy Driving 2015;
is drowsy or not. When the value of EAR is more than 0.25, it (Crash•Stats Brief Statistical Summary. Report No. DOT HS 812
indicates that the eyes are open and the detected face is not 446); National Highway Traffic Safety Administration: Washington,
DC, USA, 2017.
recognized as drowsy. The driver is considered to be in a [3] Williamson, A.; Lombardi, D.A.; Folkard, S.; Stutts, J.; Courtney, T.K.;
fatigue state if the EAR value of the detected face is less than Connor, J.L. The link between fatigue and safety. Accid. Anal. Prev.
0.25, meaning that the eyes are closed of the detected face, and 2011, 43, 498–515
then; the driver is being by the help of repeated voice sounds. [4] Stevenson, M.R.; Elkington, J.; Sharwood, L.; Meuleners, L.; Ivers, R.;
Boufous, S.; Williamson, A.; Haworth, N.; Quinlan, M.;
Grunstein, R.; et al. The role of sleepiness, sleep disorders, and the work
VIII. FUTURE WORK environment on heavy-vehicle crashes in 2 Australian
States. Am. J. Epidemiol. 2014, 179, 594–601.
[5] Li, Z.; Li, S.E.; Li, R.; Cheng, B.; Shi, J. Online Detection of Driver
The closing of the eyes for the successive frames was Fatigue Using Steering Wheel Angles for Real Driving Conditions.
detected, thereafter, it is classified as a drowsy state condition Sensors 2017, 17, 495.
otherwise, it is said as a normal blink and the loop for [6] Fletcher, A.; McCulloch, K.; Baulk, S.D.; Dawson, D. Countermeasures
capturing the eye image and visualize the state of the driver is to driver fatigue: A review of public awareness campaigns
and legal approaches. Aust. N. Z. J. Public Health 2005, 29, 471–476.
being carried out again and again. In this, the drowsy state the [7] Williamson, A.; Friswell, R.; Olivier, J.; Grzebieta, R. Are drivers aware
eye is not surrounded by a circle and it is not detected and of sleepiness and increasing crash risk while driving? Accid. Anal. Prev.
displays the corresponding message. 2014, 70, 225–234.

538

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.
[8] Blommer, M.; Curry, R.; Kozak, K.; Greenberg, J.; Artz, B. [20] Lenskiy, A.; Lee, J. Driver's eye blinking detection using novel color
Implementation of controlled lane departures and analysis of simulator and texture segmentation algorithms. Int. J. Control Autom. Syst. 2012,
sickness for a drowsy driver study. In Proceedings of the 2006 Driving 10, 317–327
Simulation Conference Europe, Paris, France, 4–6 October 2006. [21] Pauly, L.; Sankar, D. Detection of drowsiness based on HOG features
[9] Lawoyin, S. Novel Technologies for the Detection and Mitigation of and SVM classifiers. In Proceedings of the 2015 IEEE International
Drowsy Driving. Ph.D. Thesis, Virginia Commonwealth University, Conference on Research in Computational Intelligence and
Richmond, VA, USA, 2014. Communication Networks (ICRCICN), Kolkata, India, 20–22 November
[10] Sayed, R.; Eskandarian, A. Unobtrusive drowsiness detection by neural 2015; pp. 181–186.
network learning of driver steering. Proc. Inst. Mech. Eng. J. Automob. [22] Moujahid, A.; Dornaika, F.; Arganda-Carreras, I.; Reta, J. Efficient and
Eng. 2001, 215, 969–975. compact face descriptor for driver drowsiness detection. Expert Syst.
[11] Tsuchida, A.; Bhuiyan, M.; Oguri, K. Estimation of driversdrive Appl. 2021, 168, 114334
drowsiness level using a neural network based 'Error Correcting Output [23] Singh, A.; Chandewar, C.; Pattarkine, P. Driver Drowsiness Alert
Coding' method. In Proceedings of the 13th International IEEE System with Effective Feature Extraction. Int. J. Res. Emerg. Sci.
Conference on Intelligent Transportation Systems, Madeira, Portugal, Technol. 2018, 5, 14–19.
19–22 September 2010; pp. 1887–1892. [24] Viola, P.; Jones, M. Rapid object detection using a boosted cascade of
[12] LM, K.; Nguyen, H.; Lal, S. Early driver fatigue detection from simple feature. In Proceedings of the 2001 IEEE Computer Society
electroencephalography signals using artificial neural networks. In Conference on CVPR, Kauai, Hawaii, 8–14 December 2001; pp. 11–
Proceedings of the International IEEE Conference of the Engineering in 518.
Medicine and Biology Society, New York, NY, USA, 30 August–3 [25] Abdullah-Al-Wadud, M.; Kabir, M.H.; Dewan, M.; Chae, O. A dynamic
September 2006; pp. 2187–2190. histogram equalization for image contrast enhancement. IEEE Trans.
[13] Bakheet, S.; Al-Hamadi, A. Chord-length shape features for license plate Consum. Electron. 2007, 53, 593–600.
character recognition. J. Russ. Laser Res. 2020, 41, 156–170. [26] Sadek, S.; Al-Hamadi, A.; Michaelis, B.; Sayed, U. Image Retrieval
[14] Abdelwahab, H.T.; Abdel-Aty, M.A. Artificial neural networks and logit using Cubic spline Neural Networks. Int. J. Video Image Process. Netw.
models for traffic safety analysis of toll plazas. Transp. Res. Rec. 2002, Secur. 2009, 9, 17–22.
1784, 115–125. [27] Bakheet, S.; Al-Hamadi, A. Computer-Aided Diagnosis of Malignant
[15] Wijnands, J.; Thompson, J.; Aschwanden, G.; Stevenson, M. Identifying Melanoma Using Gabor-Based Entropic Features and Multilevel Neural
behavioural change among drivers using long short-term memory Networks. Diagnostics 2020, 10, 822.
recurrent neural networks. Transp. Res. Part F Traffic Psychol. Behav. [28] Dalal, N.; Triggs, B. Histograms of oriented gradients for human
2018, 53, 34–49. detection. In Proceedings of the IEEE Conference Computer Vision and
[16] Schmidhuber, J. Deep learning in neural networks: An overview. Neural Pattern Recognition, San Diego, CA, USA, 20–26 June 2005; pp. 886–
Netw. 2015, 117, 61–85. Brain Sci. 2021, 11, 240 15 of 15 893.
[17] Park, S.; Pan, F.; Kang, S.; Yoo, C.D. Driver Drowsiness Detection [29] Bakheet, S. An SVM Framework for Malignant Melanoma Detection
System Based on Feature Representation Learning Using Various Deep Based on Optimized HOG Features. Computation 2017, 5, 4.
Networks. In Computer Vision—ACCV 2016 Workshops Part III; Chen, [30] Wintersberger, S.; Azmat, M.; Kummer, S. AreWe Ready to Ride
C.S., Lu, J., Ma, K.K., Eds.; Springer: Taipei, Taiwan, 2017; pp. 154– Autonomous Vehicles? A Pilot Study on Austrian Consumers'
164. Perspective. Logistics 2019, 3, 20.
[18] Huynh, X.; Park, S.; Kim, Y. Detection of driver drowsiness using 3D [31] Blog- https://towardsdatascience.com/drowsiness-detection-system-in-
deep neural network and semi-supervised gradient boostingmachine. In real-time-using-opencv-and-flask-in-python-b57f4f1fcb9e
Computer Vision—ACCV 2016 Workshops Part III; Chen, C.S., Lu, J., [32] JOUR;Chen, Chien-Ming; Biswal, Anil Kumar; Singh, Debabrata;
Ma, K.K., Eds.; Springer: Taipei, Taiwan, 2017; pp. 134–145. Pattanayak, Binod Kumar; Samanta, Debabrata. IoT-Based Smart Alert
[19] Jabbar, R.; Al-Khalifa, K.; Kharbeche, M.; Alhajyaseen, W.; Jafari, M.; System for Drowsy Driver Detection, 10.1155/2021/6627217
Jiang, S. Real-time Driver Drowsiness Detection for Android
Application Using Deep Neural Networks Techniques. Procedia
Comput. Sci. 2018, 130, 400–407

539

Authorized licensed use limited to: ANNA UNIVERSITY. Downloaded on March 30,2023 at 05:28:09 UTC from IEEE Xplore. Restrictions apply.