2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS)

Improved Authentication and Drowsiness

2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS) | 979-8-3503-9737-6/23/$31.00 ©2023 IEEE | DOI: 10.1109/ICACCS57279.2023.10112846

Detection from Facial Features using Deep

Learning Framework in Real Time Environments
Santhiya S Divyabharathi T
Department of Computer Science Department of Computer Science
and Engineering and Engineering
M.Kumarasamy College of M.Kumarasamy College of
Engineering, Thalavapalayam, Karur, Engineering, Thalavapalayam, Karur,
Tamilnadu, India -639113 Tamilnadu, India -639113
[email protected] [email protected]

Jeno J Kawin D
Department of Computer Science Department of Computer Science
and Engineering and Engineering
M.Kumarasamy College of M.Kumarasamy College of
Engineering, Thalavapalayam, Karur, Engineering, Thalavapalayam, Karur,
Tamilnadu, India -639113 Tamilnadu, India -639113
[email protected] [email protected]

Abstract—The automobile industry has seen a rise in global I. INTRODUCTION

popularity in recent years. Unfortunately, though, automobile Everywhere in the world, the usage of automobiles is
crime is growing exponentially. The majority of cars are becoming significant, and preventing theft is necessary [1].
currently operated via mechanical keys, safety cards, In order to prevent thefts, especially of cars, auto
passwords, and pattern recognition systems. However, with
advancements in IoT and embedded technologies, vehicle
manufacturers are enhancing the security elements of their
security systems have been constantly improving. There are vehicles with cutting-edge embedded and automated
numerous recognized techniques for biometric identification technologies. The majority of the time, both biometric and
and validation available. Various security applications have non-biometric approaches offer these security characteristics
used iris, face features, and fingerprints. A reliable biometric [2]. We are pursuing these clever hidden car safety gadgets
method is face recognition. For car security as well as alarm that will be more efficient within real time because password
devices since it is based upon information about the features of hacks and the encrypting of decrypted data can cause these
a human face and can function in a variety of settings. The systems to malfunction at times [3]. This paper's main goal is
creation of a car guard as well as alarm system employing to present an advanced vehicle security system made up of a
biometric security based on systems for facial recognition is
suggested in this survey research. And, extend the framework
face recognition system, a Wi-Fi module, as well as an IoT
to predict drowsiness based on eye features monitoring. The control platform. The optimized algorithm used in the face
sensitivity of various eye monitoring systems that have been recognition system can identify faces inside vehicles,
presented up to this point to changing illumination conditions is specifically four-wheeled vehicles, when no one should be
a frequent issue. We can implement the hybrid system for present and will trigger a loud alarm. Technology has made
combining both face recognition and drowsiness detection. The security in the modern world the most advanced. To deter
Driver Face Recognition-Anti Theft System is a security theft, various security systems have been created. Closed-
measure that can be implemented in a driving setting. It begins circuit Television (CCTV) cameras are commonly used in
with the driver's face being registered and then uses facial commercial establishments due to their effectiveness and
recognition for identification. And to use the HAAR Cascade
algorithm to recognize faces and the higher accuracy
cost-efficiency in preventing and solving crimes. Many
Convolutional Neural Network technology to track the states of establishments also use burglar alarms, which emit a loud
the eyes. If it is detected that a driver is acting strangely while warning to discourage break-ins and unauthorized access.
having their eyes closed, an automatic alarm may go off, alerting Additionally, button alarms are available to rapidly notify the
the nearby residents utilize the Python Framework to put this nearby police headquarters of a crime that is being committed
survey to the test while driving. or an attempted crime. Numerous more security measures are
also readily available. Additionally, many forms of
Keywords—Authentication system, Face recognition, Eye identification, including biometric and non-biometric
detection, Drowsiness detection, Deep learning. features like face, retinal, iris, and fingerprint recognition, are
employed to boost security in various types of devices. Of the
979-8-3503-9737-6/23/$31.00 ©2023 IEEE

1638
Authorized licensed use limited to: Universidad Nacional Pedro Ruiz Gallo. Downloaded on January 31,2024 at 00:39:55 UTC from IEEE Xplore. Restrictions apply.
 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS)

security features discussed, recognition software is the most

sophisticated and secure. Face detection, however, can be Federico Gede-Fernandez, et.al,…[2] Make a
difficult because faces might appear in a variety of poses recommendation for a technique to detect fatigue using
(frontal or non-frontal), as well as in occlusions, orientation, variations in the breathing signal. The breathing signal was
lighting, and expressions. Extraction of the essential facial collected using an induction plethysmography belt, and
characteristics can be shown in Fig 1. whether the driver was awake or asleep was determined by
real-time analysis of the data. This study examines whether
the suggested algorithm can alert the driver when the first
signs of fatigue appear, indicating that it might be a beneficial
Input Image
safety feature in cars that can detect episodes. In conclusion,
the acquired results are comparable for various training
criteria, but a combination of respiratory signal quality has
Pre-processing
been discovered to significantly increase performance.
Therefore, as provided as the criteria remained unaffected by
Features class distribution, the algorithm could be maximized using
Extraction various criteria.
Output Image
Muhammad Ramzan, et.al,…[3] provides information
Classifier (Known /
about drowsiness detection methods based on behavioral,
Unknown)
vehicular, and physiological characteristics. The study found
that while none of the techniques examined were completely
accurate, techniques based on physiological parameters
tended to be more accurate than the other non-intrusiveness
Face database of these operations can be further decreased by using
Wireless sensing devices positioned on vehicle components
including the body, seat, seat cover, and steering wheel.
Combining multiple approaches, such as employing
physiological measurements together with driving or
behavioral data, can improve the detection of sleepiness and
decrease the shortcomings of individual procedures. For
Figure 1: Basic Diagram for facial recognition instance, it has been shown that combining ECG and EEG
features produces highly effective outcomes, proving that
II. THE REVIEW OF LITERATURE combining physiological data is more effective than using
them separately
Wanghuadeng, et.al,…[1] Researchers propose in unique
method that makes use of visual key point detection and face Rateb Jabbar, et.al,…[4] described a machine learning-
tracking to measure driver fatigue. To enhance the based CNN-based improvement to the drowsiness detection
effectiveness initial implementation of the KCF method, we method. The objective is to create a platform that is
create a brand-new technique and suggest the MCKCF lightweight, appropriate for embedded devices, and capable
approach, which tracks the person's face which both CNN of being maintained and performing at a high level. In order
and MTCNN. Foundation for face region detection is the to recognize drowsy driving behavior, a CNN-trained deep
identification of key facial features. Additionally, we suggest learning model is used, which uses the system's ability to
a cutting-edge technique for determining tiredness that makes recognize face landmarks in mobile device photos. This
use of the conditions of the mouth and eyelids. As a result of project's success was partly attributable to the creation a deep
its quick processing, DriCare technology operates almost learning model's structure with a mini computational
instantly. However, because the KCF technique utilizes footprint and a high level of accuracy. The average accuracy
solely a Felzenszwalb histogram of oriented gradient of this model for all locations, where the maximum size, is
information, it performs poorly for tracking faces in a 83.33% was no greater than 75KB. The developed
challenging environment. The tracked target within the frame technology can easily be included into the dashboards of
is identified using the KCF algorithm in addition to a human future cars to offer improved driver assistance programs or it
method. In the case of KCF, a human method in addition to can even be loaded on a mobile device to send notifications
the KCF algorithm tracked the target within a frame. The when drivers are drowsy, helping aimed at enhancing road
tracker is unable to immediately retrieve the target in the case safety.
of KCF once it has returned after leaving the detection region.
It also loses sight of the face of a person.

1639
Authorized licensed use limited to: Universidad Nacional Pedro Ruiz Gallo. Downloaded on January 31,2024 at 00:39:55 UTC from IEEE Xplore. Restrictions apply.
 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS)

Chao Zhang, et.al,… [5], A system for detecting fatigue

based on concurrent detection of yawnings, blinkings and LCD
blood volume pulse detection was developed. Our method's Display
Heartbeat
main structure is 9-channel SOBI, which simultaneously sensor
isolates a number of physiological sources. The sources from H-
the several outputs are automatically identified by combining MICR DC
Bridge
Body O MOT
short-term energy and kurtosis. The final step in determining
Temperatur CONT OR
drowsiness is a parallel analysis of the yawn, blink, as well as ROLLE
BVP signals. A system is suggested for simultaneously e sensor
R Buzzer
extracting several physiological details from facial
recordings. Several outputs corresponding to various sources Eye blink
are acquired simultaneously with the use of multi-channel sensor Relay
SOBI. Our solution has an algorithm structure that is simpler
than earlier fusion techniques

III. EMBEDDED BASED MONITORING SYSTEM

GPS GSM
The sleepiness detection aiming to decrease accidents
Figure 2: Driver monitoring system using Sensors.
through this system caused by tired drivers. Worldwide, a
common problem that contributes many traffic accidents is IV. SAFETY AND SECURITY SYSTEM
driver weariness. The objective of this system is to create an
automatic A smart warning or attention system for a vehicle Modern technology has led to the introduction of a variety
using these techniques [4]. To examine the driver's of automotive kinds that offer convenience to people in their
behavioral and visual-based attitudes, monitoring facial daily lives. On the other hand, it causes a lot of issues for
movements and eye blinks are used as indicators to determine people, including the numerous occurrences of accidents are
the driver's condition. The focus here is on using eye blink caused by drowsy drivers, poor night time visibility, loss of
detection to identify drowsy driving. A threshold value of concentration, etc [8]. Various security applications have
Eye Aspect Ratio (EAR) above 0.25 is considered to indicate used iris, face features, and fingerprints. While recognising a
alertness, without any signs of fatigue [5]. The proportion of fingerprint, specific traits are sought in the finger's surface's
the eye's width to that of the skin around it (EAR) falls below line pattern. Some people's fingerprints might not be
a particular level when a driver starts to nod off and their eyes recognisable or even readable. Even the absence of
start to close. The amount of video frames in which the fingerprints at birth is caused by a hereditary condition.
camera car are closed is represented by a threshold level for Additionally, people occasionally suffer financial losses as a
drowsy eye blinks. The driver is probably drowsy if the result of vehicle theft [9]. So, a procedure for the vehicle's
number of consecutive frames exceeds this threshold. The safety and security is suggested. The suggested system is
current technologies for detecting fatigue use a using Support divided into two sections, including a drowsiness detection
Vector Machine (SVM) to classify video clips into alert or system and a face authentication system.
non-alert driving events. Various techniques for detecting
driver fatigue include those based on vision and artificial A. Face Authentication System
intelligence, including employing image processing to
determine how often someone tilts their head, facial To extract facial characteristics, the HAAR Cascade
recognition, as well as eye blinking [6][7]. The system classifier is utilized, which is built on the Haar Wavelet
measures the driver's heart rate, skin temperature, as well as method to analyze pixels in the image and divide them into
eye blink using sensors including the Heartbeat Sensors, squares by function [10]. It uses the "integral image" concept
Body Temperature Probe, and Eye Blink Sensor. Another to determine the discovered "features." Haar Cascades
technique for identifying drowsy driving is tracking the employ the Ada-boost method to choose a small number of
health of the driver using sensors like temperature and critical traits from a large set in order to produce efficient
heartbeat monitors. Driver monitoring system using Sensors classifiers [11][12]. These classifiers then employ cascade
is show in fig.2 approaches to identify faces inside a picture. The process of
extracting features can be illustrated in Figure 3

1640
Authorized licensed use limited to: Universidad Nacional Pedro Ruiz Gallo. Downloaded on January 31,2024 at 00:39:55 UTC from IEEE Xplore. Restrictions apply.
 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS)

this image in order to detect sleepiness. Offer an alert system

that includes voice, SMS, and email alerts from an admin
with a good reputation. The eye status monitoring system
with alarm system is depicted in Fig. 4 [1].

Figure 3: HAAR like features

Step 1: The image is cut up into smaller pieces before

being transmitted to the classifier (or sub windows as shown
in the illustration)

Step 2: Using a cascade arrangement of N detectors, each

one learns to recognize a combination of different visual
features, such as lines, edges, circles, and rectangles, as they Figure 4: Eye state monitoring system
pass by. After the extracted features are finished, a
confidence value is given to each sub-part. V. EXPERIMENTAL RESULTS

Step 3: The images (or sub-images) that have the greatest Real-time datasets are utilized in this chapter. Face
likelihood of being faces are accepted and transferred to the detection and identification methods were used in this
accumulator. The cascade then continues the procedure by framework. Accuracy metrics can be used to gauge
retrieving the following frame or image, if it is available. performance. Accuracy is evaluated as.
B. Drowsiness Detection
(𝑇𝑃 + 𝑇𝑁)
Convolutional neural network techniques are used to 𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = ∗ 100
categorize different eye states. CNN are feed-forward neural (TP + TN + FP + FN)
networks that include various combinations of convolution
layer, max pooling layers, and totally related layers. They
also take use of spatially localized correlation by enforcing a The suggested algorithm improves accuracy compared to
proximate connection sample between the neurons of machine learning models.
adjacent layers [13]. Max pooling and Convolutional layers
alternate, simulating the persona of complex and clear cells Table(1) displays accuracy findings
in mammalian visual cortex. Convolutional and maximum .
pooling layers can be combined to create an CNN, which
ultimately results in neural networks that are fully Algorithm Accuracy(%)
interconnected. There are distinct types of weariness from the
viewpoint of each person's organ capabilities, which covers
the following situations: Local physical weariness (In Naive bayes 20
skeletal or ocular muscle areas.), Recognized physical fatigue
caused by intense manual labor, major worried fatigue
(drowsiness), and cerebral fatigue are the first four symptoms SVM 5
(no longer having the power to do something). With this
machine, the more precisely adjust the ocular states while still
enforcing the system for face detection via linear discriminate CNN 70
assessment When driver's close the eyes, a warning signal for
corrective action may be elevated [14][15]. After, Monitoring
driver's head, eye movements from the images the camera Table (1) Accuracy table
captured, the algorithm moves on to the analysis stage. Then,
several image processing algorithms are used to pre-process

1641
Authorized licensed use limited to: Universidad Nacional Pedro Ruiz Gallo. Downloaded on January 31,2024 at 00:39:55 UTC from IEEE Xplore. Restrictions apply.
 2023 9th International Conference on Advanced Computing and Communication Systems (ICACCS)

for Driver Drowsiness Detection Based on Eye State." Rev.

d'IntelligenceArtif. 33.6 (2019): 461-466.
[7] Arefnezhad, Sadegh, et al. "Driver drowsiness detection based on
steering wheel data applying adaptive neuro-fuzzy feature selection."
Sensors 19.4 (2019): 943.
[8] Mahmoodi, Mohammad, and Ali Nahvi. "Driver drowsiness detection
based on classification of surface electromyography features in a
driving simulator." Proceedings of the Institution of Mechanical
Engineers, Part H: Journal of Engineering in Medicine 233.4 (2019):
395-406.
[9] Bakheet, Samy, and Ayoub Al-Hamadi. "A Framework for
Instantaneous Driver Drowsiness Detection Based on Improved HOG
Features and Naïve Bayesian Classification." Brain Sciences 11.2
(2021): 240.
[10] Tanveer, M. Asjid, et al. "Enhanced drowsiness detection using deep
learning: an fNIRS study." IEEE access 7 (2019): 137920-137929.
[11] G. D. Furman, A. Baharav, C. Cahan and S. Akselrod, "Early detection
of falling asleep at the wheel: A heart rate variability approach", Proc.
Comput. Cardiol., vol. 35, pp. 1109-1112, Jul. 2008.
[12] Y. Li and S. Cha, "Implementation of robust face recognition system
using live video feed based on CNN", Proc. Comput. Vis. Pattern
Recognit., 2018
Figure 5: Performance report [13] Viola Paul and Michael J. Jones, "Robust real-time face
detection", International journal of computer vision 57, vol. 2, pp. 137-
Performance report is show in fig.5 As per performance 154, 2004.
[14] N. S. Karuppusamy and B.-Y. Kang, ‘‘Multimodal system to detect
graph, CNN offers greater accuracy levels than the current driver fatigue using EEG, gyroscope, and image processing,’’IEEE
machine learning algorithms. False positive findings are Access, vol. 8, pp. 129645–129667, 2020, doi:
decreased by the suggested system. The high false rate is 10.1109/ACCESS.2020.3009226.
decreased by the suggested system. [15] A. Chowdhury, R. Shankaran, M. Kavakli, and M. M. Haque, ‘‘Sensor
applications and physiological features in drivers’ drowsiness
detection: A review,’’ IEEE Sensors J., vol. 18, no. 8, pp. 3055–
VI. CONCLUSION 3067,Apr.2018,doi:10.1109/JSEN.2018.2807245

The proposed method aims to increase the security and

safety of drivers. The technology uses the driver's eye
conditions to detect sleepiness before identifying verifying
user's face for system access. The HAAR Cascade classifier,
a well-known face identification method, is the basis of the
proposed crop Eye algorithm, which splits the face into
sections and identifies the left and right eye. The device can
determine if the user's eyes are open or shut by examining
and intensity distribution and calculating the separation of the
eyelashes from the eyebrows. Eyes are regarded as being
closed when the estimated distance exceeds the threshold.
Otherwise, they're accessible. There is an active warning
when the eyelids are shut repeatedly for several frames.

REFERENCES

[1] Deng, Wanghua, and Ruoxue Wu. "Real-time driver-drowsiness

detection system using facial features." IEEE Access 7 (2019): 118727-
118738.
[2] Guede-Fernandez, Federico, et al. "Driver drowsiness detection based
on respiratory signal analysis." IEEE access 7 (2019): 81826-81838.
[3] Ramzan, Muhammad, et al. "A survey on state-of-the-art drowsiness
detection techniques." IEEE Access 7 (2019): 61904-61919.
[4] Jabbar, Rateb, et al. "Driver drowsiness detection model using
convolutional neural networks techniques for android application."
2020 IEEE International Conference on Informatics, IoT, and Enabling
Technologies (ICIoT). IEEE, (2020).
[5] Zhang, Chao, et al. "Driver drowsiness detection using multi-channel
second order blind identifications." IEEE Access 7 (2019): 11829-
11843.
[6] Chirra, Venkata Rami Reddy, SrinivasuluReddyUyyala, and Venkata
Krishna Kishore Kolli. "Deep CNN: A Machine Learning Approach

1642
Authorized licensed use limited to: Universidad Nacional Pedro Ruiz Gallo. Downloaded on January 31,2024 at 00:39:55 UTC from IEEE Xplore. Restrictions apply.