Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic

Reasoning

Student ID:

Student Name:

Project title:

Course:
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Contents
Introduction:...............................................................................................................................................4
Aims and objective:.....................................................................................................................................6
Literature review:........................................................................................................................................7
Background..................................................................................................................................................7
Detection of Eye and Face by utilizing Machine learning and Deep Learning techniques.......................7
Field Programmable Gate Array (FPGA) Based Drowsiness/Sleepiness Detection System......................8
Wavelet Network Algorithm-Based Eye Recognition System..................................................................8
Detection of Fatigue based on the Vehicle (steering wheel) Algorithm..................................................8
Drowsy Alarming System by employing the Electrooculogram (EOG), Electroencephalography (EEG),
Electromyogram (EMG), and Electrocardiography (ECG) Algorithms......................................................9
Collision and Fatigue Alarming System by Employing the Smart Techniques........................................10
The imaging processing system is used for the Diver Drowsiness Detection automatically..................10
Convolutional Neural Network..............................................................................................................10
By using Haarcascade Samples for the Real-time Drowsy Driver Detection..........................................11
A study on alarm systems and drowsiness detection of drivers............................................................11
Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal Network...................11
An Efficient K–NN Approach for Automatic Drowsiness Detection Using Single-Channel EEG Recording
...............................................................................................................................................................12
Target Recognition in Infrared Circumferential Scanning System via Deep Convolutional Neural
Networks...............................................................................................................................................12
Prediction of Drowsy Driver Detection by Using Soft Computing Technique........................................13
Non-intrusive driver drowsiness detection system...............................................................................13
Modeling...................................................................................................................................................14
CNN (convolutional neural network).....................................................................................................14
Implementation.........................................................................................................................................16
Raspberry pi:..........................................................................................................................................17
Pi camera:..............................................................................................................................................18
Gas sensor.............................................................................................................................................18
Vibration sensor....................................................................................................................................19
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
Serial port..............................................................................................................................................19
Buzzer....................................................................................................................................................20
GPS........................................................................................................................................................21
Circuit....................................................................................................................................................21
Result and conclusion................................................................................................................................23
References.................................................................................................................................................26

List of figures

Figure 1 Block diagram_________________________________________________________________________15

Figure 2 Raspberry pi___________________________________________________________________________17
Figure 3 Pi camera_____________________________________________________________________________18
Figure 4 Gas sensor____________________________________________________________________________19
Figure 5 serial port_____________________________________________________________________________20
Figure 6 Buzzer________________________________________________________________________________20
Figure 7 GPS__________________________________________________________________________________21
Figure 8 Drowsiness detection circuit______________________________________________________________22
Figure 9 system starts__________________________________________________________________________23
Figure 10 alcohol detected______________________________________________________________________24
Figure 11 vehicle starts_________________________________________________________________________24
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Introduction:
The main reason for the major accidents is drowsiness. To control the ratio of accidents that
occurs by the drowsiness some type of advanced system is required that uses the advanced
technology and detect the real-time activities of the driver and also control the vehicle if the
driver is behaving abnormally. The main aim of this project is to build an advanced system that
controls the drowsiness of the driver, generates the alarm, and also controls the vehicle by using
advanced techniques by using advanced computer language. Drowsiness is a natural thing that
occurs due to some reasons. The drowsiness happens due to an unfavorable climate situation. A
lot of accidents happen due to the tiredness of the driver. Fatigue is the main reason for the
driver. When the driver feels sleepy then he lost control over the vehicle. The system measures
the drowsiness of the driver. Drowsiness means feeling tired or sleepy. This case is usually
because it’s often noticed that during driving drivers feel sleepy or tired which resulted in major
accidents. To control this type of major accident this system is very helpful. So this project
proposed an advanced system that also monitors the facial expression and movement of the eye
and also controls the vehicle if the driver feels sleepy and applies the brake. The driver is the
main element in driving. During driving driver's condition is the most important thing. The
purpose of this proposed system is to notice the condition of the driver by using modern
technologies like artificial intelligence. In this project, the developed system measures the
movement of the eye and the facial expression of the driver during driving. It’s also called
drowsiness. This system is measuring the condition of the driver in real-time and sends data back
to the main hub. A camera is used in this system. The functionality of this camera is to detect the
facial expression and movement of the eye in real-time. The camera is recorded every aspect of
the driver and sends the data back to the main hub. The camera that is used in the system is
integrated by the raspberry pi. The main component of this system is raspberry pi. The
raspberry pi is used here because it supports the functionality of the camera and as well as
supports the different languages that are used in a computer program. The size of the raspberry pi
is small but it works like the original operating system. In this system, the camera is directly
connected with raspberry pi which sends the instruction to the camera to detect the facial
expression and the movement of the eye in real-time. The programming language that used in
this system is python. Python is a popular and strong language. Python is used in almost every
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
field. The python language helps in getting the real-time expression of the driver and also detect
the movement of the eye. The algorithm that is used with the python language is CNN and the
RNN algorithm. Each algorithm measures the real-time face expression of the driver. So, by
using these two algorithms detect the real-time face expression of the driver. It detects the facial
expression of the driver and then it compares with the face of a normal person. The system is
behaving like a smart system. During the comparison, if the face is detected as sleepy then the
system Is directly connected with the brake system of the vehicle and automatically applied the
brake and slows down the vehicle and stops the vehicle. The major components that are used in
the project are the raspberry pi which is the main hub of this system, the buzzer which gives an
alarm if the driver has a sleepy mode, and the camera which measures the real-time expression of
the face. Every component that used in this system has its function. The GPS Module is also
used in this system which tracks the location if major accidents happen. As discussed before the
programming language that is used in this system is python. To measure the real-time, face
detection of the driver the python library that is used in this system is open cv and the dlib. The
dlib library contains a lot of pertained models or algorithms. It also helps in measuring or
estimating the location of the driver. Dlib contains the different learning algorithms. Here this
system used the dlib library because during the driving situation it continuously measures the
face expression of the driver and then compares collected face expression with a learning
algorithm and with a trained model. After the comparison continuously changes their state. The
real-time face expression of the driver is taken by the python library named as open cv. The open
cv is designed for real-time applications and also calculates the computational efficiency. By
using the camera, it detects real-time the person. The open cv library is used to take real-time
pictures or videos of the driver and then on that base algorithm checks if the driver behaving
normally or not. The vibration sensor is used in this system to vibrate the driver if he feels any
drowsiness. The gas sensor is a sensor is used in this system that checks if the driver is drinking
alcohol or not. The GPS sensor is used which sends the current location of the driver. The buzzer
is also used in this system. which generates an alarm if the system detects that the driver is
feeling sleepy. The system is the complete package and it's very helpful for the driver and the
traffic. This system is based on the advanced technique and used the advanced programming
technique that gives more accuracy to this system. This system is very helpful for the drivers and
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
it’s also decreasing the ratio of accidents that happen due to drowsiness. Different types of
sensors are used in this system.

The majority of the accidents are caused mainly due to driver fatigue, monotonous road
situations or conditions, and severe weather conditions (Vural et al., 2007). World Health
Organization and National Highway Traffic, around 1.35 million people die every year as a
result of automobile crashes across the world. In the majority of situations, road accidents are
caused by insufficient driving skills (Jabbar et al., 2020). If the driver is drunken or drowsy, this
type of incident can occur (Turan and Gupta, 2013). The most popular sorts of fatal or lethal
accidents are mostly linked to the driver's tiredness or exhaustion. The control of the car is lost
when the driver feels sleepy and falls asleep (Gupta and Garima, 2014). Therefore, advanced
technology is required to build intelligent or smart vehicle systems (Malekian et al., 2016). This
research work suggests a technique that alarms the drivers if they are daydreaming or in
drowsiness condition. A camera is placed inside the vehicle to detect the driver's facial detection,
eyes closure, eye blinking, head position, and other behaviors using the landmark algorithm
along with the Euclidean distance in the behavior modification method. These characteristics aid
in detecting the tiredness of the driver and immediately warning him through voice speaker and
e-mail to a perspective person (usually the owner of the car) who can help him reawaken
(Mollicone et al., 2019). The IoT module, which depends on a wireless transmission mechanism,
is mostly employed to forward or send an e-mail to a target (Miranda et al., 2018). However, the
suggested device includes a credit card-sized microprocessor called the Raspberry Pi and Pi3 a
camera that can track eye movement (Khunpisuth et al., 2016). As a result, the strength of
collision impacts that occur at the accident time is monitored, and the owners and hospitals'
emergency wards who are close to the accident place are alerted, along with the accident GPS
location (Hossain and George, 2018).

Aims and objective:

The aim of this report is to develop a system, a webcam records the video and driver’s with
acceptable accuracy in each frame employing image processing technique. Facial expression are
detected and subsequently the eye aspect ratio, mouth opening ratio and nose length ration are
computed and depending on their values, drowsiness is detected based on these values. This
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
system also check driver is alcoholic or not. If the driver is drowning system will generate an
alarm and stop the car automatically

The objectives of this project are:

 To develop the hardware which is very advanced product related to driver safety on the roads.

 To reduce the ratio of accident due to drowsy driving

 To aid in the prevention of accidents passenger and commercial vehicles.

Literature review:

Background
The driver's drowsiness can be identified by employing a variety of ways such as psychological,
vehicle-based, and behavioral metrics or measurements performed with the help of various
predictive algorithms. The details of different aspects used for the identification of the driver's
drowsiness are detailed in the below-given sections.

Detection of Eye and Face by utilizing Machine learning and Deep Learning
techniques
To identify sleepiness and microsleep, a researcher named Jabbar suggested a Convolutional
Neural Network (CNN) approach utilizing the ML algorithm (Jabbar et al., 2020). In this
research paper, a virtual camera is employed to detect the driver's facial landmarks, which is
further fed into a CNN algorithm to diagnose drowsiness more. The practical classification or
categorization of eye detection is carried out here using a variety of different data sets like with
and without glasses in both day and night vision. As a result, it works with the different android
modules for the detection of drowsiness with relatively high precision. According to the findings
of Chakrabarty and Sanyal, the Deep CNN algorithm was developed to detect or identify the eye
blink and recognize its states (Sanyal and Chakrabarty, 2019). A researcher name Saleh used the
Recurrent Neural Networks (RNN) and LSTM model to classify the driving behavior using
different sensors (Saleh et al., 2017). The RNN algorithm method was further used by the
researcher Ed-Doughmi to assess the driver's behavior (Ed-Doughmi et al., 2020). It more
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
specifically concentrates on the implementation of real-time fatigue identification/detection
systems to prevent road collisions or accidents. This technology uses stacked three-dimensional
(3D) CNN tools to predict drowsy or sleepy drivers and has an acceptance rate of 92%.

Field Programmable Gate Array (FPGA) Based Drowsiness/Sleepiness Detection

System
A researcher named Vitabile devised low-intrusive drowsiness or sleepiness detection system
employing an FPGA (Vitabile et al., 2011). This gadget specifically focuses on bright pupil
eyeballs detection by using an infrared sensor which is a light source placed or embedded in a
vehicle. With the help of this effect, the eye's retinas were recognized up to 90%, which helps in
finding or identifying drivers' eyes for detecting tiredness/drowsiness over a variety of frames to
avert major accidents. By employing the cyclone II FPGA mechanism, a researcher named
Navaneethan created a real-time system for tracking human eyes (Navaneethan et al., 2019).

Wavelet Network Algorithm-Based Eye Recognition System

The wavelet networking technique was firstly utilized by the researcher named Jemai to develop
or implement a drowsiness warning system (Jemai et al., 2013). That network follows the
movements of people's eyes using classification algorithms such as the wavelet network
classifier, which completely depends on the fast wavelet transform to make binary decisions
(nonconscious or conscious). The findings of Babaeian revealed that the most important features
which are thoroughly considered are electrocardiogram and the rate of heartbeat that is
repeatedly retrieved via wavelet modification with the regression method for the identification of
fatigue (Babaeian et al., 2016). This technique was used to classify heart rate data by employing
a wavelet network mechanism to establish an average way of detecting drowsiness.

Detection of Fatigue based on the Vehicle (steering wheel) Algorithm

Arefnezhad suggested noninterfering drowsiness or tiredness detection system that is fully relied
on the data collected from the vehicle steering, the collected data is then further integrated with
the neuro-fuzzy mechanism with the help of the vector device and the swarm particle
optimization technique (Arefnezhad et al., 2019). A researcher named Mutya recently developed
a method which completely based on the steering wheel algorithm to overcome the drowsiness
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
problem (Arefnezhad et al., 2019). It is based on the formation of image or steering movement
based on the pictorial mechanism as well as the CNN algorithms for effective drowsiness
classification that can lead to the reduction of wrong drowsiness detection rates.

Drowsy Alarming System by employing the Electrooculogram (EOG),

Electroencephalography (EEG), Electromyogram (EMG), and Electrocardiography
(ECG) Algorithms
A researcher named Budak constructed a drowsiness detection or identification system by
employing the EEG technique or approach, which comprised of different components such as the
VGGNet technique, the AlexNet technique, and the wavelet algorithm (Budak et al., 2019). This
procedure more efficiently examines the condition of sleepiness by utilizing the brain/mind
indication signal called EEG, sensors, and the cameras which are activated by employing
machine learning techniques to notify the drivers. Waleed and Hayawi suggested a method for
detecting tiredness with the help of EEG sensors to measure the heart rate variability signal
(Hayawi and Waleed, 2019). Waleed and Hayawi further continued their work and developed an
intrusive approach for eyeball movement measurement by employing the EOG methodology to
build a fatigue alarming system which includes that includes an Arduino controller along with
the Nearest Neighbors classifier to boost the system's overall accuracy. A researcher named Song
suggested a method to detect driver weariness based on muscular skin movement of the eyes,
which is evaluated by employing the EMG sensors and a human-device interface (Song et al.,
2019). Similar work was conducted by two researchers Ma and Artanto (Artanto et al., 2017).
They both developed the system based on fatigue identification with the help of EEG signals.
From their findings, the movements of muscle parts and eyelid closure were also observed by
utilizing the EEG sensors which mostly work with the support of ESP8266 for monitoring or
providing drowsiness/sleepiness data available on the global Internet. It offered a dependable
framework for tiredness or drowsiness detection that utilizes a deep learning technique to
determine fatigue accuracy by employing the various EEG signals. This technique, however, is
mostly based on the primary principal elemental examination or analysis termed the PCANet,
that pre-developed the EEG data to enhance the overall detection accuracy. This procedure was
first integrated into offline mode and limited sample size, on the other hand, it fails in terms of
the accuracy requirements when applied to the huge amounts of the samples/pieces in real-time
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
scenarios. As a result, the IoT module further is utilized to examine huge sample volumes of data
both online and offline. The researcher Ma developed a useful application for detecting driver
drowsiness by analyzing facial expressions (Ma et al., 2019). The movement of the face was
identified with the help of the deep learning approach of MB-LBP and the classifier termed
AdaBoost. On the other hand, employing a fuzzy inference approach or technique may also be
employed to quickly and correctly detect driver tiredness. Since the accuracy or precision of
detection is reduced while car drivers wear different glasses. Consequently, IoT modules are
normally utilized to enhance it smarter and improve fatigue detection accuracy.

Collision and Fatigue Alarming System by Employing the Smart Techniques

A smart glass was used by Chen for fatigue detection (Gowri et al., 2019). The vehicle's
backlight flashes automatically with the written text being sent via the IoT module. The
researchers named Patil and Kinage suggested a system that comprised of human eyes blinking
sensors to identify drowsiness as well any collisions or accidents; then with the help of a
vibration sensor combined with the heart rate sensor to send a warning message to an authorized
user (Kinage and Patil, 2019). As a result, it is also connected to a GSM and GPS device for
location tracking and message delivery.

Kumari and Siva Reddy presented a method that used an Arduino board with sensors and a
camera to regulate the source of unintentional disasters.

The imaging processing system is used for the Diver Drowsiness Detection
automatically.
Sleeping is a natural phenomenon of nature and each human must need to sleep at least 7 to 8
hours a day. If someone has taken less sleep than the mentioned period, he is not able to perform
his daily tasks efficiently. Particularly drivers must need adequate sleep to perform their duty
without put any error. If they will not get a sufficient amount of rest time and they will sleep
during driving and which can cause damage. Drowsiness during driving can cost lives and it
happened if someone driving for so long or he didn’t get enough sleep. In both cases, there is a
chance of hazard that this can be resolved by using the modern computation devices made for the
solution of drowsiness by alarming the driver in case he is sleepy.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
The updated systems that are used for the prevention of drowsiness monitor the blinking of the
driver's eye, yawning and the portion of the mouth to distinguish the drowsiness condition and
alarm the driver.

Convolutional Neural Network

In this study, the deep learning technique is used with multiple layers of the artificial neural
network (ANN). It has been developed immensely in the last decades. It has the capacity for data
processing with a large volume. It is also an advanced technique that is used instead of other
techniques due to its power full capabilities.

Convolutional Neural Network (CNN) is the most famous and the deepest method that is being
used. Convolution is the name that represents the linear operation of mathematics. Several layers
the CNN comprise include a non-linear layer, pooling layer, complete connected layer, and
coevolutionary. CNN has great capability of computer learning. Applications particularly dealing
with picture data, such as natural language processing (NLP), the world's biggest image
classification data collection (Image Net), and computer vision, have yielded fantastic results.
This study has investigated and discussed all the significant issues and elements connected to
CNN and its operations. The knowledge of artificial neural networks and machine learning is a
must for the understanding of this study (Albawi et al., 2017).

By using Haarcascade Samples for the Real-time Drowsy Driver Detection

As there is an immense increase in the population, the number of road accidents has also grown
up. In India. according to the study, there are over half of a million incidents occur per year. The
tired rider affects the three main regions due to its driving skill under tiredness which is c)
Perception loss, b) hinder balance, and c) reaction cycle which affects slower driving abilities. In
this study, a real-time monitoring device was created which uses the image recognition
technologies like head movement and facial recognition. Tests used in the blink of the eye and
detection in exhaustion /drowsiness are the haar cascade tests (Suryaprasad et al., 2013).

A study on alarm systems and drowsiness detection of drivers

While traveling or driving the safety of the traveler or driver is the most valuable factor. A
mistake during driving can lead to dangerous consequences. These consequences include the
losses such as major economic losses, physical injury, and even mortality. There are advanced
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
devices obtainable in the market like multiple sensors and navigation. This study showed that
they are an increase in the number of accidents and their major cause includes the sleep of the
driver. The alarm system has been recommended for the drivers so they can be tracked by
driving and the alarm goes on as they feel drowsy (Choudhary et al., 2016).

Real-Time System for Driver Fatigue Detection Based on a Recurrent Neuronal

Network
In recent years, car crash deaths have risen abruptly across the globe. Road safety is the primary
concern of the countries to minimize the probability of crashes occurring due to human error,
particularly drowsiness. The most advanced technology is providing the driver information about
road activity such as navigation, car sensors, automatic braking, etc, to prevent accidents. The
leading research topic right now is to observe the activity of the driver during driving specifically
somnolence as it has the highest impact on the accident and death on the highway. In this paper,
the Recurrent Neural Network technique is used to monitor the activity of the driver and predict
drowsiness. Neural Network Multi-Layer 3D Convolutional Networks for Driver Detection
Drowsiness was established using a dataset to mold and approve our concept and supply repeated
implementations. After the training session, the accuracy was measured of the system was 92%
(Ed-Doughmi et al., 2020).

An Efficient K–NN Approach for Automatic Drowsiness Detection Using Single-

Channel EEG Recording
This study was stated that traffic incidents occurred and one of the major causes is drowsy
driving. The main purpose of this study is to use the k-nearest neighbors (K-NN) algorithm to
developed an automated method for the detection of drowsiness. Short-term Fourier transform
(STFT) is used in this study to develop the power distribution in time frequency. Then afterward
for each EEG subband, the mean power value was determined across time segments of 0.5.
Standard deviation (SD) and Shanon entropy linked to rising time-segment were also calculated
in the time domain. A minimum of 11 apps have been selected to detect tiredness and alertness.
To give a rapid classifier, Kd-trees are employed as a search technique for closest neighbors. The
experimental results provided that the introduced material in this study successfully diagnosed
the drowsiness with a 91% accuracy (Jalilifard and Pizzolato, 2016).
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
Target Recognition in Infrared Circumferential Scanning System via Deep
Convolutional Neural Networks
We will achieve long-term oversight across a wide field of vision with an IRCSS (Infrared
Circumferential Scanning System). Increasing environmental consciousness in the
computerization cycle, particularly with the framework of protection, requires automatically
recognizing goals within the range of vision. Identification and identification, are two subtasks in
the recognition of objectives, which refer to the target location and type, respectively. The study
proposes the end-to-end realization of the goal in IRCSS by using the deeply coevolutionary
Neural Network (DCNN). An infrared object recognition dataset was created to boost the
adaptability of the algorithm and solve the data limitation. For networking learning the technique
used is the optimum cross-domain software transition. 82.7 mAP was obtained by the proposed
method in the test (Chen and Wang, 2020).

Prediction of Drowsy Driver Detection by Using Soft Computing Technique

This study focused on the facial recognition of drivers to detect drowsiness and avoid accidents
that occurred due to drowsy driving. The methodology of this research is to take the driver's face
picture and through handling images search for facial expressions and assess the degree of
drowsiness by using the hybrid strategy (Singh et al., 2016).

Non-intrusive driver drowsiness detection system

The research key contribution is an exclusive algorithm for sleepiness identification and tracking
that incorporates data from a vehicle's performance characteristics

as well as a road vision system. By the implementation of a service computer vector detection
with an effective and reliable alert mechanism with sleepiness, the algorithm is refined to more
precisely recognize the degree of drowsiness. The Support Vector Machine (SVM) detection
technology reduced drowsiness levels by combining non-intrusive devices with standard
equipment sensors, resulting in the elimination of road casualties caused by drowsy drivers. The
results are founded, to assess data, collection of somnolence databases that span about 60 hours
of driving time data have gathered. By using the SVM, a car model with all components has
been developed for the identification of drowsiness. After applying certain methods and
executing them, in result they gave the sign of strong drowsiness (Abas, 2013).
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
In this study, a control has been designed for the detection of drowsiness with the help of a
camera. This method will be useful for the identification of the drowsiness of the driver during
driving by using the CNN algorithm. The python 3 version has been used for this study. This will
help to avoid accidents and prevent economic loss and casualties. IR sensor is used to detect the
seat belt. If the driver has not fastened the seat belt the alarm will go on to identify the seat belt
error. Afterward, the gas sensor will sense the alcohol concentration. If the concentration will
exceed the standard for driving, the car will not start and the alarm will also turn on as a warning.
A GPS is installed for location tracking purposes. A vibration sensor is installed in the car to
monitor the sudden vibration that occurs due to drowsy driving. A camera has been installed in
the car which identifies the eye features. In case of drowsy condition of the driver, this camera
will detect this and turn on the alarm. The alarm will only turn off when the driver presses the off
button, otherwise, it will not turn off automatically. This is the visual technique to identify
drowsiness and keep the driver informed about its consciousness. This study mainly focused to
identify drowsiness and by using this control method we can avoid serious accidents that
happened due to this particular problem.

Modeling
This section models the drowsiness system by analyzing the eyes. for this system face detection,
the not the main purpose of the system once the clear face is detected by a face detection
technique then the face detects the eye for the eyes uses some other algorithm. For the face,
expression detection uses the CNN algorithm because this is the best technique for facial
recognition to perform some analysis. The model for the drowsiness detection is trained on some
relevant images like drowse, yawing, eye enclosure analysis, and eye blink rate.

CNN (convolutional neural network)

In python, multiple algorithms are used for face detection among all the techniques this has a
specific key feature that makes it better. In the CNN algorithm size of the image is decided
before the input that is the reason the exact size of the image should be required for the
processing. As in the project the camera is used for the processing to get a continuous image like
the video from which get the exact image which is required for the processing then apply the
facial recognition technique. when taking the images for the video compare it with the database
after making a comparison for the result based on facial expression. The data received from the
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
camera is stored in the system database after that these images are used in the convolutional
techniques to generate the vectors. The CNN has multiple layers that are connected to gather to
carry out the best result to detect the driver’s drowsiness. A layer named reule is directed
connected with the convolutional layer in this layer filters are used and each filter has its
specification like length width and depth. By calculating the scalar product of local image and
filters then draw a map to carry out the results. To speed up the process of face detection CNN
algorithm uses the average and Max (pooling). Pooling is the key feature of CNN which is used
to reduce the mapping size. An input data image of interest to perform the technique is divided
into multiple regions. During the max-pooling process, the region with the higher value is
selected and placed at the point where it should be to produce the results. This CNN technique is
used for face detection there is no need to detect the face expression because the eyes state issued
for the drowsiness detection.

The CNN is used for the eye because the driver's face is very important for the detection of
facial features. The facial features that need to extract from the driver’s face are eye blink rate,
eye closures and yawning, etc.

For drowsiness monitoring the need, the eye blink rate is very important and the method used for
the detection of drowsiness analyses the eye blinking of a normal person when sitting inactive
state. The normal eye blinking rate is 10 times is a minute. When a driver is in drowsy condition
then the eye blinking rate decreases according to the algorithm at that time drowsiness condition
is condition sissified and generates the alarm to alert the driver.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Figure 1 Block diagram

In the model, a camera helps to get the real-time image, and the python CNN algorithm works on
it to detect the drowsiness. First, detect the face from the video then resize the image according
to the system requirement once the image is passed to the convulsion system then do some
required calculation and map it to create the final result image. The model works step by step
first to detect the presence of a driver after that it goes through the seat belt detection procedure
then the system checks the alcohol concentration when the driver breath using the gas sensor.
When all the sensor sends a clear signal the vehicle start and the camera gets the images of the
driver's face to detect face once the face is detected then it focuses on the eyes and checks the
eye condition for drowsiness detection.

Implementation
In this section of the report implement the design of the model to detect the drowsiness of the
driving person while driving the designed model is based on visual and real-time detection of
face to get the results. This system uses multiple sensors that help to detect the face seat belt and
alcohol. Each sensor has its own defined criteria for the detection purpose as for alcohol
detection a gas sensor is used because when a person breathing at that time exhales the air which
is gas in that air the detector detects the presence of alcohol particles. In the visual-based
drowsiness detection system, all the sensors are integrated to perform safe driving, such as
camera module, gas sensor, vibration sensor, buzzer or alarm, relay, and GPS. Each sensor has
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
its job to start and stop the engine depending on the conditions that are manipulated by the
central controller which is raspberry pi. To control the car and alert the driver use facial
recognition technology which gets the real-time face input using a camera or image sensor after
the once face is detected analyze the face. Once the driver's face is detected then only from the
face focus on the eye because when eyes are open have a specific value of eye aspect ratio based
on these values system decide diver is feeling sleepy or active. If the value of the eye aspect ratio
is less than the threshold value it means that the drowsiness condition is satisfied after that
controller takes action according to the situation to save the car and traveling person from any
serious injury. A known as GPS is used in this system in case of an accident to get the location
of the incidence place to rescue the passages for first aid. To implement the visual-based
drowsiness system proteus software is used where all components are integrated with raspberry
pi to meet the requirement. The following components are used for this purpose:

1- Raspberry pi
2- Camera
3- Gas sensor
4- Vibration sensor
5- Serial port
6- Buzzer
7- GPS module

Raspberry pi:
It is a mini-computer that is very low in cost and used for practice to integrate the sensors to
build the system. It allows plugging a tv screen, mouse, keyboard, and internet cable to control
like a computer after that get real-time values to analyze the scenario of interest. This mini-
computer is a simple platform where write a program in python langue for learning and
experiment purposes, good thing about its that can reuse if a task is done and just overwrite
another program. To use this minicomputer in the real-world application it has some general-
purpose input and out ports which are known as GPIO pins these specific pins are used for the
input and output data devices such as the major devices for the raspberry pi is the mouse,
keyboard, monitor and speaker. In this project, this is integrated with some other devices for the
input and output data all these devices are connected through the GPIO pins. The main purpose
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
to use raspberry pi is that it can solve complex real-world problems using python languages
techniques and algorithms. In python, multiple packages and libraries are available in it to
simplify the task complexity. In the proteus, the raspberry module behaves like a controller
because all the sensors are integrated with it to perform the job. The controller has some defined
conditions that are written in the program for the guidance of the controller and sensors. 1 st of all
import all the required sensors after that make the connection between the controller and sensor.
In proteus software use the source code window to write the code for the guidance of the sensors
and controller who get the data and monitor the driver's state while driving. The main controller
or the computer get data for any input sensor after that the detection techniques are applied then
according to the situation send a quick signal to the output sensor, if necessary, action is required
otherwise get the continuous data from the input sensors to continue to monitor the driver state.

Figure 2 Raspberry pi

Pi camera:
The camera is also known as an image sensor this is a device used to capture objects, some
visual and video. Same as the simple camera the module is used for the raspberry pi is a pi
camera to get the image of an object. The Pi camera module is only connected with and
integrated able with the raspberry pi to capture and process the image in a proper way for face
detection using some python algorithms. The pi camera is a small and lightweight camera
module that is connected using the CSI cable. Raspberry pi camera module for the
communication serial interface protocol is used which is named MIPI. This camera is mostly
used for, surveillance and machine learning projects as in this project get the image of a person
who is driving a car apply some facial algorithm to detect the eye state. The raspberry pi camera
module is sued with pi has a resolution of 5 megapixels and can record a video with the
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
resolution of 1080 with 30 fps (frames per second). The drowsiness detection is a real-time
application this is the reason the camera module is the best choice for the analysis.

Figure 3 Pi camera

Gas sensor
The gas sensor is a module that is used for gas detection by using this sensor multiple types of
gases can be detected such as LPG, CH4, CO, Benzine alcohol, etc. Gas leakage may cause a
serious situation so these sensors are highly sensitive detect the gas as soon as they can. These
gas sensors are integrated with the microcontrollers that get the value in real-time and take action
if needed. In this project a gas sensor is used for alcohol detection if ad river is drunk then the
system does not allow the driver to start the engine. Multiple types of gas sensors are available in
the market but for alcohol detection, MQ3 is the best sensor that detects the presence of ethanol
in air. This senor is detecting the ethanol and provides the concentration of it after. When a
person breathes Infront of a gas sensor then it calculates the value of ethanol concentration on the
breath after that based on that concentration value take action. A threshold for the ethanol is set
in the python program if the value is high then that threshold then it generates the alarm and does
not allow the driver to start the engine. The gas sensor uses the GPIO pins of the raspberry pi for
the integration of the sensor to send the data.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Figure 4 Gas sensor

Vibration sensor
To detect the vibration a vibration sensor is used which is also known as a piezoelectric sensor.
A simple vibration can be used for the detection of vibrational change, force, pressure,
acceleration of a system. the vibration sensor can be integrated with the raspberry pi and Arduino
for various applications depending on the requirements. When the vibration sensor detects any
vibration then it converts it to the voltage the value of the voltage is high that’s means high
vibration is present or detected. It is a digital sensor that provides the output signal as “0” for low
and ‘’1” for high vibration.

Serial port
The serial port looks like a socket through which input and output devices can connect easily for
the exchange of data. The data acquisition process for the various sensor takes place through the
serial ports in the sense of computers these ports are sued for the mouse, keyboard, monitor, and
modem. In this circuit, the simulation ton gets the image from the camera gets in the real-time
use the serial port by these ports get the image and video which is then further process by the
python algorithms to detect the eyes and face expression.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Figure 5 serial port

Buzzer

The buzzer is purely an output device that generates the sound when the condition is satisfied or

power is supplied to it. In actuality, a buzzer is used to convert an audio input signal and produce

sound. It is a Dc operated device that is the reason this is integrated with the GPIO pint because

at that point DC signal is available. The use of a buzzer in the system is to alert the driver when

derive is feeling sleepy or drowsiness.

Figure 6 Buzzer
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
GPS
GPS is a global positions system that is used to get the location of any device that has a GPS
module built-in or integrated. With the help of this system, a person gets the exact location of his
presence. This works with the help of a satellite system which is the reason it provides the exact
location of an object. This can be used for a type of field so the sued is dependent on the
requirement of that domain. In this project, the GPS is used to locate the position of a vehicle in
case the driver is feeling sleepy and hits the vehicle and for a quick rescue, the person knows the
exact place of incidence.

Figure 7 GPS

Circuit
The circuit is implemented in the proteus where raspberry pi and required sensors are available
in the part library. Find all the components for design requirements and make the connection
using the default connection pin all the input and output devices are connected with the GPIO pin
to the raspberry pi to process the data which meets the goal of the system. Once the system is
implemented accurately that it works step by step. For the very first step, the gas sensor check
for alcohol is not detected then go for the seat belt detection all the initial requirements are
satisfied then start the vehicle. When the vehicle start at that moment camera starts getting the
facial expression to send them to analysis algorithms sued for detection purpose. The circuit
diagram is given below for the designed system.
Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Figure 8 Drowsiness detection circuit

 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Result and conclusion

In this report, a visual-based drowsiness system is implemented which helps to drive safely and
reach the destination safe and sound. To control the activeness of driving person. In this project
learn the integration of various sensors with the raspberry pi and work. In this the car is started
once the driver sits on the driving seat first it checks for some conditions if the person sits on the
car, it says welcome as can see in the following image

Figure 9 system starts

Which shows the presence of the person in the driving seat after that for safety purposes check if
he fastens the seat belt or not if the seat belt detects it shows the message on the screen seat belt
is detected. If the driver did not fasten the seat belt, then it does not goes to the next step once the
seat belt is detected after the gas sensor actively monitors the breathing of the driver if the value
of ethanol is more than the limited value the message print on the screen is alcohol is detected.
This is not allowed them to start the car if all the conditions are satisfied. When alcohol is
detected, the message is printed on the along with-it buzzer to start producing the sound.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

Figure 10 alcohol detected

When all the required conditions to drive a vehicle are satisfied that a message is printed on the
screen “vehicle start”. When a vehicle is in running condition then the image sensor starts to get
input and fed to the analyzing algorithms for drowsiness detection.

Figure 11 vehicle starts

Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning

References
 ABAS, A. B. 2013. Non-intrusive driver drowsiness detection system. University of Bradford.
 ALBAWI, S., MOHAMMED, T. A. & AL-ZAWI, S. Understanding of a convolutional neural network.
2017 international conference on engineering and technology (ICET), 2017. Ieee, 1-6.
 AREFNEZHAD, S., SAMIEE, S., EICHBERGER, A. & NAHVI, A. 2019. Driver drowsiness detection
based on steering wheel data applying adaptive neuro-fuzzy feature selection. Sensors, 19, 943.
 ARTANTO, D., SULISTYANTO, M. P., PRANOWO, I. D. & PRAMESTA, E. E. Drowsiness detection
system based on eye-closure using a low-cost EMG and ESP8266. 2017 2nd International
conferences on Information Technology, Information Systems and Electrical Engineering
(ICITISEE), 2017. IEEE, 235-238.
 BABAEIAN, M., BHARDWAJ, N., ESQUIVEL, B. & MOZUMDAR, M. Real time driver drowsiness
detection using a logistic-regression-based machine learning algorithm. 2016 IEEE Green Energy
and Systems Conference (IGSEC), 2016. IEEE, 1-6.
 BUDAK, U., BAJAJ, V., AKBULUT, Y., ATILA, O. & SENGUR, A. 2019. An effective hybrid model for
EEG-based drowsiness detection. IEEE sensors journal, 19, 7624-7631.
 CHEN, G. & WANG, W. 2020. Target recognition in infrared circumferential scanning system via
deep convolutional neural networks. Sensors, 20, 1922.
 CHOUDHARY, P., SHARMA, R., SINGH, G. & DAS, S. 2016. A survey paper on drowsiness
detection & alarm system for drivers. International Research Journal of Engineering and
Technology (IRJET), 3, 1433-1437.
 ED-DOUGHMI, Y., IDRISSI, N. & HBALI, Y. 2020. Real-time system for driver fatigue detection
based on a recurrent neuronal network. Journal of imaging, 6, 8.
 GOWRI, S. M., ANITHA, P., SRIVAISHNAVI, D. & NITHYA, M. Internet of things based accident
detection system. 2019 Third International conference on I-SMAC (IoT in Social, Mobile,
Analytics and Cloud)(I-SMAC), 2019. IEEE, 159-163.
 GUPTA, S. & GARIMA, E. 2014. Road Accident Prevention System Using Driver’s Drowsiness
Detection by Combining Eye Closure and Yawning. International Journal of Research, 1, 839-842.
 HAYAWI, A. A. & WALEED, J. Driver's drowsiness monitoring and alarming auto-system based on
EOG signals. 2019 2nd International Conference on Engineering Technology and its Applications
(IICETA), 2019. IEEE, 214-218.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
 HOSSAIN, M. Y. & GEORGE, F. P. IOT based real-time drowsy driving detection system for the
prevention of road accidents. 2018 International Conference on Intelligent Informatics and
Biomedical Sciences (ICIIBMS), 2018. IEEE, 190-195.
 JABBAR, R., SHINOY, M., KHARBECHE, M., AL-KHALIFA, K., KRICHEN, M. & BARKAOUI, K. Driver
drowsiness detection model using convolutional neural networks techniques for android
application. 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies
(ICIoT), 2020. IEEE, 237-242.
 JALILIFARD, A. & PIZZOLATO, E. B. An efficient K-NN approach for automatic drowsiness
detection using single-channel EEG recording. 2016 38th Annual International Conference of
the IEEE Engineering in Medicine and Biology Society (EMBC), 2016. IEEE, 820-824.
 JEMAI, O., TEYEB, I. & BOUCHRIKA, T. 2013. A novel approach for drowsy driver detection using
eyes recognition system based on wavelet network. International Journal of Recent
Contributions from Engineering, Science & IT (iJES), 1, 46-52.
 KHUNPISUTH, O., CHOTCHINASRI, T., KOSCHAKOSAI, V. & HNOOHOM, N. Driver drowsiness
detection using eye-closeness detection. 2016 12th International Conference on Signal-Image
Technology & Internet-Based Systems (SITIS), 2016. IEEE, 661-668.
 KINAGE, V. & PATIL, P. Iot based intelligent system for vehicle accident prevention and detection
at real time. 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and
Cloud)(I-SMAC), 2019. IEEE, 409-413.
 MA, Y., CHEN, B., LI, R., WANG, C., WANG, J., SHE, Q., LUO, Z. & ZHANG, Y. 2019. Driving fatigue
detection from EEG using a modified PCANet method. Computational intelligence and
neuroscience, 2019.
 MALEKIAN, R., KAVISHE, A., MAHARAJ, B. T., GUPTA, P. K., SINGH, G. & WASCHEFORT, H. 2016.
Smart vehicle navigation system using hidden Markov model and RFID technology. Wireless
Personal Communications, 90, 1717-1742.
 MIRANDA, M., VILLANUEVA, A., BUO, M. J., MERABITE, R., PEREZ, S. P. & RODRIGUEZ, J. M.
Portable Prevention and Monitoring of Driver’s Drowsiness Focuses to Eyelid Movement Using
Internet of Things. 2018 IEEE 10th International Conference on Humanoid, Nanotechnology,
Information Technology, Communication and Control, Environment and Management
(HNICEM), 2018. IEEE, 1-5.
 Visual Based Driving Drowsiness Monitoring System based On Advanced Synergistic
Reasoning
 MOLLICONE, D., KAN, K., MOTT, C., BARTELS, R., BRUNEAU, S., VAN WOLLEN, M., SPARROW, A.
R. & VAN DONGEN, H. P. 2019. Predicting performance and safety based on driver fatigue.
Accident Analysis & Prevention, 126, 142-145.
 NAVANEETHAN, S., NANDHAGOPAL, N. & NIVEDITA, V. 2019. An FPGA-based real-time human
eye pupil detection system using E2V smart camera. Journal of Computational and Theoretical
Nanoscience, 16, 649-654.
 SALEH, K., HOSSNY, M. & NAHAVANDI, S. Driving behavior classification based on sensor data
fusion using LSTM recurrent neural networks. 2017 IEEE 20th International Conference on
Intelligent Transportation Systems (ITSC), 2017. IEEE, 1-6.
 SANYAL, R. & CHAKRABARTY, K. Two stream deep convolutional neural network for eye state
recognition and blink detection. 2019 3rd International Conference on Electronics, Materials
Engineering & Nano-Technology (IEMENTech), 2019. IEEE, 1-8.
 SINGH, M., SRIVASTAVA, G. & KUMAR, P. 2016. Internet traffic classification using machine
learning. International Journal of Database Theory and Application, 9, 45-54.
 SONG, M.-S., KANG, S.-G., LEE, K.-T. & KIM, J. 2019. Wireless, skin-mountable EMG sensor for
human–machine interface application. Micromachines, 10, 879.
 SURYAPRASAD, J., SANDESH, D., SARASWATHI, V., SWATHI, D. & MANJUNATH, S. Real time
drowsy driver detection using haarcascade samples. CS & IT Conference Proceedings, 2013. CS
& IT Conference Proceedings.
 TURAN, G. & GUPTA, S. 2013. Road accidents prevention system using driver’s drowsiness
detection. International Journal of Advanced Research in Computer Engineering & Technology
(IJARCET) Volume, 2.
 VITABILE, S., DE PAOLA, A. & SORBELLO, F. 2011. A real-time non-intrusive FPGA-based
drowsiness detection system. Journal of Ambient Intelligence and Humanized Computing, 2,
251-262.
 VURAL, E., CETIN, M., ERCIL, A., LITTLEWORT, G., BARTLETT, M. & MOVELLAN, J. Drowsy driver
detection through facial movement analysis. international workshop on human-computer
interaction, 2007. Springer, 6-18.