CHAPTER ONE

INTRODUCTION
1.1 Background of the Study
Humans have always invented machines and developed technologies to make their
lives easier and safer, whether for everyday activities like commuting to work or for
more interesting purposes like air travel. As technology advanced, transportation
evolved and our dependence on it began to increase exponentially. It has greatly
impacted our life as we know it. Now we can travel to places at a pace even our
grandparents never thought possible.

In modern times, almost everyone in this world uses some form of transportation
every day. Some people are rich enough to have their own car, while others use public
transportation. However, there are some rules and codes of conduct for those who
drive regardless of their social status. One of them is staying alert and active while
driving. Neglecting our responsibilities to safer travel has resulted in hundreds of
thousands of tragedies being associated with this wonderful invention each year. It
may seem trivial to most people, but complying with road traffic rules and regulations
is of paramount importance. On the road a car exerts the most power and in
irresponsible hands it can be destructive and sometimes this carelessness can even
damage the lives of people on the road. One type of negligence is not admitting when
we're too tired to drive.

Driver fatigue has become one of the main reasons for a large number of traffic
accidents and has a significant impact on road safety. Many traffic accidents could be
avoided if overtired drivers were warned in good time. Drowsiness is defined as the
state of being sleepy when one needs to rest. It can cause symptoms that have a major
impact on task performance: slowed reaction time, intermittent lack of consciousness,
or microsleep (blinks with a duration of over 500ms), to name a few examples
(Dinges, 2015).

In fact, prolonged fatigue can cause performance impairments similar to alcohol

(Dawson, Reid, & Fatigue, 2017). While driving, these symptoms are extremely
dangerous as they greatly increase the likelihood that drivers will miss traffic signs or

1
exits, veer into different lanes, or even collide with their vehicle and cause an accident
(Soares, Monteiro, Lobo, Couto, Cunha, & Ferreira, 2020).

To monitor and prevent a destructive result of such neglect, many researchers have
written research on driver drowsiness detection systems. But sometimes some of the
points and observations made by the system are not accurate enough. Hence, this
research work is proposed to provide data and another perspective on the problem at
hand to improve their implementations and further optimize the solution.

Therefore, what distinguishes this work from previous research is the use of a non-
intrusive system capable of detecting fatigue from image sequences, which is
currently an open problem. In most of the available works, the experimental
methodology consists of extracting and classifying individual frames from each video
and checking whether the classification is correct or not, but this approach does not
take into account the intrinsic relationship between consecutive images and their
measures for false positives results less reliable. There is currently little work testing
the systems with full video and counting the number of alarms issued during each
video (which is required to assess the number of false alarms generated during a given
period). Therefore, the methodology proposed in this research work can be seen as a
starting point for the design of such systems.

1.2 Statement of Problem

If you've ever driven a car, at some point you've felt sleepy at the wheel. We hate to
admit it, but it is an important issue with serious implications that needs to be
addressed.

Drowsy driving can be just as dangerous as drunk or distracted driving. Unfortunately,

work, household chores and other commitments can result in sleepless nights, making
driving dangerous for you.

One in four car accidents is caused by driver fatigue. The scariest part is that sleepy
driving isn't just about falling asleep while driving. Drowsiness while driving can be
as minor as a brief state of unconsciousness if the driver is not paying full attention to

2
the road. Drowsy driving results in over 71,000 injuries, 1,500 deaths, and N12.5
billion in monetary losses per year (Anirban Dasgupta, 2018).

In addition, it is believed that sleepiness can also negatively affect people in work and
classrooms. Although sleep deprivation and study go hand-in-hand, workplace
drowsiness, especially when working with heavy machinery, can lead to serious
injuries similar to those experienced when driving a car. Driving while being drowsy
has become one of the major reasons of causing road accidents. Drivers who drive at
night or for long distances without a break are more prone to accidents. Because of
this, many fatal injuries and deaths occur; Hence, it has become an active area of
research. Due to the consequence(s) that may arise from this problem of drowsiness
while on steering, it is important to develop a drowsiness detection solution, especially
in the early stages, to avoid accidents.

1.3 Aim and Objectives of the study

The primary aim of the research work is to design and implement driver drowsiness
detection system with the following objectives:
1. To examine and analyse driver drowsiness and its implication to life and
properties.

2. To design a system that can detect driver’s drowsiness while driving using
computer vision techniques.
3. To implement the drowsiness detection system using Python Programming
Language.

1.4 Significance of the Study

This research will be important in its implementation as driver fatigue and drowsiness
contribute to the percentage of road accidents. Detecting driver alertness and
drowsiness is an efficient way to prevent traffic accidents. With this system, drowsy
drivers are warned by an alarm to regulate the driver's awareness, alertness and
concentration; thus, helps to reduce the number of traffic accidents.

It can be use in commercial vehicle for safety of life and properties. The system will
also be useful in heavy vehicles for examples trucks since the drivers of truck have

3
long driving periods. Overloaded cranes as well as mobile cranes are no exception
since heavy things are lifted by using cranes for transporting them to other places.

This research work is an active topic that is still being enhanced and improved by
researches and can be applied in many areas such as detecting the attention-level of
students in classrooms and lectures. This is also relevant to the researcher’s field of
study since it requires the researcher to apply the knowledge of programming and
algorithms.

1.5 Scope of the Study

This research is specific to Car drivers, Buses drivers as well as heavy-duty truck
drivers. The parameters to be used to detect drowsiness are eye blinking, eye pupil
position and reflex actions such as yawning.

1.6 Definition of Terms

Drowsiness: refers to feeling more sleepy than normal during the day.
Fatigue: is a term used to describe an overall feeling of tiredness or lack of energy.
Algorithm: a process or set of rules to be followed in calculations or other problem-
solving operations, especially by a computer.
Accident: an unfortunate incident that happens unexpectedly and unintentionally,
typically resulting in damage or injury.
Automobile: usually four-wheeled vehicle designed primarily for passenger
transportation and commonly propelled by an internal-combustion engine using a
volatile fuel.
Computer Vision: Computer vision is a field of artificial intelligence (AI) that
enables computers and systems to derive meaningful information from digital images,
videos and other visual inputs — and take actions or make recommendations based on
that information.

Image classification: Image classification refers to the task of extracting information

classes from a multiband raster image. The resulting raster from image classification
can be used to create thematic maps.

4
Artificial Intelligence: Artificial intelligence is the simulation of human intelligence
processes by machines, especially computer systems.

Deep Learning: Deep learning is a subset of machine learning, which is essentially a

neural network with three or more layers.

Convolutional neural Network: A convolutional neural network (CNN) is a type of

artificial neural network used in image recognition and processing that is specifically
designed to process pixel data.

Image Processing: Image processing is a method to perform some operations on an

image, in order to get an enhanced image or to extract some useful information from
it.

5
 CHAPTER TWO
LITERATURE REVIEW
2.1 Drowsiness Detection
According to WorkplaceTesting (2019), Drowsiness describes the state of being
drowsy or sleepy. A person experiencing drowsiness may feel tired or sluggish and
have difficulty staying awake. Individuals are usually less alert when drowsy and may
fall asleep but are able to be roused. When used by medical professionals, the term
drowsiness describes a symptom that may arise from a variety of causes. A person
will naturally become drowsy when their body is in need of rest. However, in some
cases an individual can experience drowsiness either at inappropriate times or to
excess. Drowsiness is also referred to as somnolence or sleepiness.

WorkplaceTesting (2019) further explained that, in the workplace, excessive

drowsiness can be a safety hazard. A person experiences drowsiness is less alert and
may even fall asleep while on the job. A worker who is drowsy due to sleep
deprivation may have slower reaction time, reduced motor control, and impaired
cognitive abilities. Drowsiness reduces the brain's ability to process information,
slowing productivity and limiting an individual's ability to respond to stimuli. Some
researchers have concluded that a severely fatigued driver is just as dangerous as a
driver who has consumed excessive alcohol (Friedrichs & Yang, 2010).

Drowsiness that occurs during the hours associated with wakefulness is sometimes
referred to as excessive daytime sleepiness. Of course, for individuals engaged in shift
work, daytime sleepiness may be the norm. In modern times, it may be more accurate
to refer to drowsiness as either normal or abnormal. Medications, neurological
disorders, sleep apnea, lack of sleep or insomnia, or substance abuse are all potential
causes of drowsiness (Chau et al., 2017).

2.1.1 Methods of measuring the drowsiness level

When measuring the drowsiness level of a driver, there are two different approaches
according to the origin of the data used for this measuring. On the one hand, there are

6
systems that monitor the vehicle state to assess the fatigue of the driver, while on the
other hand, there are systems that use parameters obtained from the own driver.

(a) Systems focused on the vehicle: Among works that focus on the analysis of the
vehicle state and its relation to fatigue, the most common measures that are studied are
steering wheel behaviors or lane departures (McDonald et al., 2012). In Samiee et al
(2014), other parameters of the car are used, such as the vehicle position or the
steering wheel angle, and they perform data fusion on multiple measures to achieve a
more reliable system. However, even if the diminishing performance over skill-based
tasks by the driver can actually be a consequence of drowsiness, it appears at a later
stage and it cannot be used to detect the early symptoms of fatigue (Yang et al., 2009).

(b) Systems focused on the driver: One of the most reliable ways of estimating fatigue
is by using electroencephalograms (EEG) in combination with electrooculograms
(EOG) (Sommer & Golz, 2010), but in real driving environments, these kinds of
systems are usually rejected by drivers. Their main drawback is that they require that
the driver has attached electrodes around the eyes and over the head, which makes
them intrusive systems that produce discomfort and rejection by drivers. Because of
this limitation, the most used fatigue detection systems are those in which the driver’s
state is detected through a camera placed on the vehicle that takes images of the
driver. In this work, we will focus on the detection of the early symptoms of
drowsiness by using the driver’s state.

It's worth mentioning that each method has its own limitations and accuracy may vary
due to various factors, such as vehicle’s state and environmental conditions.
Therefore, it's important to use multiple methods and cross-reference the results to
improve accuracy.

2.1.2 Application of driver drowsiness detection system

A driver drowsiness detection system is a technology that uses various sensors and
algorithms to detect when a driver is becoming drowsy or distracted while driving.
This system can be very helpful in preventing accidents caused by drowsy or

7
distracted driving, which can be particularly dangerous on long journeys or when
driving at night. (Fletcher, 2017)

Some of the practical applications of driver drowsiness detection systems include:

i. Safety: The primary application of a driver drowsiness detection system is to

ensure safety. When a driver is drowsy or distracted, their reaction time slows
down, and they may not be able to respond to changing road conditions or
hazards on the road in time. A drowsiness detection system can alert the
driver to take a break, preventing accidents caused by fatigue or distraction.
(Kapoor et al., 2018)

ii. Fleet Management: Companies that manage fleets of vehicles, such as taxi and
trucking companies, can use driver drowsiness detection systems to monitor
their drivers' behavior and ensure that they are not driving while drowsy or
distracted. This can help prevent accidents, reduce downtime due to
accidents, and improve overall fleet safety. (Tziviskou et al., 2019)

iii. Insurance: Some insurance companies are now offering lower premiums to
drivers who have a drowsiness detection system installed in their cars. This is
because the system reduces the risk of accidents and can therefore lead to
lower insurance claims. (National Sleep Foundation, 2020)

iv. Public Safety: Driver drowsiness detection systems can also be used by public
transportation companies, such as buses and trains, to ensure that their drivers
are alert and focused on the road. This can improve public safety and reduce
the risk of accidents caused by driver fatigue. (National Highway Traffic
Safety Administration, 2021)

v. Medical Conditions: Some medical conditions, such as sleep apnea, can cause
drivers to become drowsy while driving. A drowsiness detection system can
help drivers with these conditions monitor their drowsiness levels and take
appropriate action to prevent accidents. (National Sleep Foundation, 2020)

8
Overall, a driver drowsiness detection system is a valuable tool for improving road
safety and preventing accidents caused by driver fatigue or distraction. (Kapoor et al.,
2018).

2.1.3 Drowsiness detection algorithms

Drowsiness detection algorithms are computer programs designed to analyze data
collected from various sensors to determine a driver's level of drowsiness. These
algorithms use different methods to analyze driver behavior and physiological
responses to identify when a driver is becoming drowsy and alert them to take a break.

There are several types of drowsiness detection algorithms, including:

i. Eye Blinking Analysis: This algorithm analyzes the frequency and duration of
eye blinks to detect drowsiness. When a person is drowsy, their eye blinking
frequency decreases, and the duration of their eye blinks increases. This
algorithm tracks the changes in eye blinking patterns over time to detect
drowsiness (Kapoor, Verma, & Taneja, 2018).
ii. Pupil Dilation Analysis: This algorithm measures the diameter of a person's
pupils and looks for changes in pupil size. When a person is drowsy, their
pupils dilate, and the algorithm tracks the changes in pupil size to detect
drowsiness (Tziviskou, Kostoulas, & Tziviskou, 2019).
iii. Head Movement Analysis: This algorithm analyzes the movement of a person's
head and neck to detect drowsiness. When a person is drowsy, their head and
neck movement decrease, and the algorithm tracks these changes in movement
to detect drowsiness (Tziviskou et al., 2019).
iv. EEG Analysis: This algorithm measures the electrical activity in a person's
brain to detect drowsiness. When a person is drowsy, the frequency of their
brain waves decreases, and the algorithm tracks these changes to detect
drowsiness (Kapoor et al., 2018).

These algorithms are often combined to create a more comprehensive drowsiness

detection system. For example, a system may use both eye blinking analysis and head
movement analysis to detect drowsiness more accurately. These algorithms may also

9
be combined with other sensors, such as steering wheel sensors or GPS systems, to
provide a more complete picture of a driver's behavior (Tziviskou et al., 2019).

In general, drowsiness detection algorithms are designed to provide a warning to

drivers when they are becoming drowsy and may be at risk of falling asleep at the
wheel. This warning can be in the form of an audible alert, a vibration or other
physical feedback, or a visual alert displayed on a dashboard or other screen in the
vehicle. By providing drivers with a warning, drowsiness detection algorithms can
help prevent accidents caused by drowsy driving (National Highway Traffic Safety
Administration, 2021).

2.2 Machine Learning Techniques

Machine learning is a type of artificial intelligence that enables machines to learn
from data, rather than being explicitly programmed (Alpaydin, 2010). It is a powerful
tool that has been used in various applications, including natural language processing,
computer vision, and predictive analytics.

There are different types of machine learning techniques, but the most common ones
are supervised learning, unsupervised learning, and reinforcement learning.
Supervised learning involves training a model using labeled data, where the input
features and the corresponding output values are provided (Hastie, Tibshirani, &
Friedman, 2009). The model is trained to map the input features to the correct output
values, allowing it to make predictions on new, unseen data. This technique is widely
used in classification and regression problems.

Unsupervised learning involves training a model using unlabeled data, where the input
features are provided, but the output values are not (Alpaydin, 2010). The model is
trained to find patterns and structure in the data, allowing it to group similar data
points together or identify anomalies in the data. This technique is commonly used in
clustering, dimensionality reduction, and anomaly detection.

Reinforcement learning is a technique in which an agent learns to make decisions in

an environment based on feedback in the form of rewards or penalties (Jordan &

10
Mitchell, 2015). The agent takes actions in the environment, and the environment
provides feedback in the form of a reward or penalty, based on the outcome of the
action. The goal of the agent is to learn to take actions that maximize the reward over
time. Machine learning algorithms are used to build models that can learn from data
and make predictions or decisions based on that learning. These algorithms can be
applied to a wide range of problems, from image classification to fraud detection to
personalized recommendations.

Some popular machine learning algorithms include logistic regression, decision trees,
random forests, support vector machines, and neural networks (Goodfellow, Bengio,
& Courville, 2016). Each algorithm has its strengths and weaknesses, and the choice
of algorithm depends on the specific problem being addressed. Machine learning is a
rapidly growing field, and its applications are expanding rapidly. It has become an
essential tool for many businesses and industries, including healthcare, finance, and
transportation.

Achieving the classification task using conventional ML techniques requires several

sequential steps, specifically pre-processing, feature extraction, wise feature selection,
learning, and classification. Furthermore, feature selection has a great impact on the
performance of ML techniques. Biased feature selection may lead to incorrect
discrimination between classes. Conversely, DL has the ability to automate the
learning of feature sets for several tasks, unlike conventional ML methods (LeCun, et
al., 2015). DL enables learning and classification to be achieved in a single shot. DL
has become an incredibly popular type of ML algorithm in recent years due to the
huge growth and evolution of the field of big data (Najafabadi, et al., 2015). It is still
in continuous development regarding novel performance for several ML tasks (Yusuf,
2020) and has simplified the improvement of many learning fields, such as image
super-resolution (Yang, et al., 2019), object detection, and image recognition.
Recently, DL performance has come to exceed human performance on tasks such as
image classification.

11
2.2.1 Driver drowsiness detection using Machine Learning Techniques

Driver drowsiness detection using machine learning techniques has gained much
attention due to its potential to improve road safety. Machine learning algorithms can
be trained on various input data sources, including physiological signals, computer
vision data, and behavioral data, to detect driver drowsiness in real-time. Some
common machine learning techniques used for driver drowsiness detection include
support vector machines (SVMs), artificial neural networks (ANNs), convolutional
neural networks (CNNs), and deep learning models.

SVMs are popular machine learning algorithms for drowsiness detection, as they can
be trained to classify features extracted from physiological signals, such as
electroencephalography (EEG), electrocardiography (ECG), and electromyography
(EMG), with high accuracy (Yu et al., 2018). ANNs have also been used for driver
drowsiness detection, achieving accuracy rates of up to 94% (Boukhechba et al.,
2019). CNNs, which are commonly used for image classification, have been applied
to extract features from facial images and detect driver drowsiness (Chen et al., 2018).
Deep learning models, such as recurrent neural networks (RNNs) and long short-term
memory (LSTM) networks, have also been used for driver drowsiness detection,
achieving high accuracy rates of up to 98% (Ma et al., 2021).

While machine learning techniques have shown promising results in driver drowsiness
detection, several challenges need to be addressed, including dealing with unbalanced
datasets, selecting appropriate features, and ensuring real-time performance.
Furthermore, the reliability and generalizability of machine learning models need to
be validated in real-world driving scenarios to ensure their effectiveness.

Hence, driver drowsiness detection using machine learning techniques has the
potential to improve road safety by detecting drowsiness in real-time. Various
machine learning algorithms, including SVMs, ANNs, CNNs, and deep learning
models, have been used for driver drowsiness detection with promising results.
However, further research is needed to address the challenges associated with the

12
development of effective and reliable driver drowsiness detection systems using
machine learning techniques.

2.3 Convolutional Neural Network

A Convolutional Neural Network (CNN) is a type of deep learning architecture that is
particularly well-suited for image and video data. CNNs are composed of multiple
layers, including a series of convolutional layers, pooling layers, and fully connected
layers (LeCun et al., 2015).
The convolutional layers are responsible for learning local features in the input data,
such as edges and textures. These features are then passed through pooling layers,
which reduce the dimensionality of the data and help to make the network more robust
to small changes in the input. Finally, the fully connected layers are responsible for
combining these local features into a global representation of the input data, which can
be used for tasks such as image classification or object detection.

The use of convolutional layers in CNNs was first proposed by LeCun et al. (1989) to
improve the performance of handwritten digits recognition. They proposed the
architecture of multi-layer perceptron (MLP) with convolutional and subsampling
layers, which they called LeNet-5.

Since then, CNNs have been used in a wide range of image and video processing
tasks, such as object recognition (Krizhevsky et al., 2012), image segmentation (Long
et al., 2015), and facial expression recognition (Sun et al., 2013).

CNNs have also been used for image generation tasks, such as image super-resolution
(Dong et al., 2016) and style transfer (Gatys et al., 2015).

CNNs are a powerful tool for image and video processing tasks, owing to their ability
to learn local features in the input data, reduce dimensionality through pooling, and
combine these features into a global representation for tasks such as classification.

13
2.3.1 Application Convolutional Neural Network to driver’s drowsiness
detection
Convolutional Neural Network (CNN) has shown promising results in driver
drowsiness detection. A CNN is a deep learning algorithm that can learn features
automatically from images, making it suitable for processing visual data such as
images or video frames. In drowsiness detection, CNNs are used to extract features
from facial images or videos to predict driver drowsiness.

One study that used CNNs for driver drowsiness detection was conducted by Chen et
al. (2018). The study proposed a deep learning-based approach that used a CNN to
extract features from facial images to predict driver drowsiness. The CNN was trained
on a dataset of driver facial images with corresponding labels of alert or drowsy states.
The extracted features were then fed to a support vector machine (SVM) classifier to
make the final prediction.

The study achieved a high accuracy of 98.6% in drowsiness detection, demonstrating

the effectiveness of the proposed approach. The approach had several advantages over
previous methods, such as not requiring specific facial feature extraction and
achieving high accuracy even in challenging lighting conditions.

Another study by Jin et al. (2018) used a CNN to detect driver drowsiness from
images captured by a camera mounted on the dashboard. The CNN was trained on a
large dataset of images and achieved an accuracy of 88% in detecting driver
drowsiness. The study also compared the performance of different CNN architectures
and found that a deeper network with more layers achieved higher accuracy.

CNNs have shown great potential in driver drowsiness detection, achieving high
accuracy and robustness to challenging conditions. However, more research is needed
to evaluate the performance of different CNN architectures and optimize their
hyperparameters to improve their effectiveness in real-world applications.

14
2.4 Review of related work
Driver's drowsiness detection system is a crucial technology that has the potential to
improve road safety by alerting drivers when they are at risk of falling asleep at the
wheel. This technology has been a topic of interest in research and has gained much
attention in recent years. In this literature review, the researcher will examine the
various approaches to driver drowsiness detection and the current state of the art.

Various methods have been employed to develop systems for detecting drowsiness,
such as computer vision, machine learning algorithms, and physiological signals. In
recent years, deep learning techniques have been particularly promising for detecting
driver drowsiness. For instance, Chen et al. (2018) proposed a method that utilizes a
convolutional neural network (CNN) to extract features from facial images to predict
driver drowsiness. This deep learning-based approach achieved a drowsiness detection
accuracy of 98.6%, which makes it a viable option for real-world applications.

Another approach to driver drowsiness detection is using physiological signals.

Electroencephalography (EEG), which measures brain activity, has been used to
detect driver drowsiness. For instance, a study by Li et al. (2018) proposed a system
that uses EEG and a support vector machine (SVM) to detect driver drowsiness. The
system achieved an accuracy of 91.8%, which demonstrates the feasibility of using
physiological signals for driver drowsiness detection.

Computer vision-based approaches have also been used for driver drowsiness
detection. These approaches use a camera to monitor the driver's facial expressions
and eye movements. For example, a study by Hassan et al. (2017) proposed a system
that uses an SVM to classify the driver's facial features and eye movements to detect
drowsiness. The system achieved an accuracy of 96.3%, demonstrating the potential
of computer vision-based approaches for driver drowsiness detection.

15
One study evaluated the performance of different computer vision-based algorithms
for driver drowsiness detection. The study found that the combination of eye closure
and head pose features provided the best accuracy, achieving an accuracy of 96.6%
(Wang et al., 2017).

Another study used a convolutional neural network (CNN) to detect driver drowsiness
from images captured by a camera mounted on the dashboard. The CNN was trained
on a large dataset of images and achieved an accuracy of 88% (Jin et al., 2018).

Also, physiological signals have also been used to detect driver drowsiness.
Electroencephalography (EEG) is a popular physiological signal that has been used to
detect drowsiness. In one study, a system was developed to detect driver drowsiness
using EEG signals. The system achieved an accuracy of 85% in detecting drowsiness
(Munir et al., 2020).

Driver drowsiness can also be detected using machine learning algorithms. A study
utilized a Support Vector Machine (SVM) classifier to detect drowsiness by analyzing
features from physiological signals and driver behavior. The system demonstrated
86.7% accuracy in detecting drowsiness (Yu et al., 2018). In another study, a deep
neural network (DNN) was used to identify driver drowsiness based on multiple
physiological signals. The DNN obtained a 94.8% accuracy in detecting sleepiness
(Ma et al., 2021).

2.4.1 Summary of Literature review

The literature review reveals that driver drowsiness detection systems have been
developed using various techniques, including computer vision, physiological signals,
and machine learning algorithms. Deep learning techniques, such as convolutional
neural networks (CNNs), have shown promising results in driver drowsiness
detection, achieving high accuracy rates of up to 98.6%. Physiological signals, such as
electroencephalography (EEG), have also been used to detect driver drowsiness,
achieving accuracy rates of up to 91.8%. Computer vision-based approaches have
been used to monitor the driver's facial expressions and eye movements, achieving
accuracy rates of up to 96.6%. Machine learning algorithms, such as support vector

16
machines (SVMs) and deep neural networks (DNNs), have also been used for driver
drowsiness detection, achieving accuracy rates of up to 94.8%. These findings
demonstrate the feasibility and potential of various techniques for developing effective
driver drowsiness detection systems.

17
 CHAPTER THREE
ANALYSIS AND DESIGN
3.1 Research Methods
This section of the study outlines the various steps used in the analysis and design of
the proposed driver drowsiness system. All steps taken together should ensure the
realization of a working computer vision technique for detecting a drowsy driver. The
steps taken in detecting a drowsy driver are as follows: analysis of the existing system,
limitations of the existing system, proposed model, system design, all using the
Knowledge Discovery Database Methodology (KDD-M). The procedures to be
followed are as follows:

3.2 Facts Finding

In a research work, facts are pieces of information that can be verified as true or false
through objective evidence. They are typically used as the foundation for a study, and
are used to support the conclusions and recommendations made in the work. Suitable
fact-finding techniques are very important developing and implementing a new
system. For the purpose of this research, primary and secondary source of data
collection were employed:

Primary source: Primary data is collected specifically for the research project at hand,
and is often considered to be more reliable and valid than secondary data; for the
purpose of this study, information was collected through observation.

Secondary source: This work shall exploit a public dataset acquired from Kaggle will
be used to train the model.

3.3 System Analysis

In this section the existing system is analysed to determine its strong points and also
highlights its weak point which this research tend to improve on. Also, analysis of the
proposed system is carried out to determine all the components of the system and how
they work individually and as a unit before the design and implementation.

18
3.3.1 Analysis of the existing system
Driver drowsiness detection systems use a variety of techniques to detect when a
driver is becoming fatigued and may be at risk of falling asleep at the wheel. Some
common methods include:

 Eye state Trackers: These systems use cameras to track the position and
movement of the driver's eyes to detect signs of drowsiness, such as prolonged
periods of time with the eyes closed or a lack of blinking (Zhou, et al., 2020).
 Head position and movement Detectors: These mechanisms use cameras or
other sensors to track the position and movement of the driver's head, looking
for signs that the driver may be nodding off, such as a downward head tilt or a
lack of head movement (Wang et al., 2019).
 Facial expression monitors: A webcam is used here to monitor driver face to
detect specific facial muscle movement, yawning and drowsiness (Kim, et al.,
2020)
 Heart rate meters: These systems are capable of measuring the driver's heart
rate, which can increase as a person becomes more tired (Rajesh, et al., 2020).
 Lane departure and steering wheel movement sensors: These are cameras or
sensors capable of detecting the driver is drifting or deviating from their lane,
which can be an indication that they are drowsy or otherwise distracted (Paik,
et al, 2020).

These systems typically provide an alert to the driver, such as a sound or vibration,
when they detect signs of drowsiness. Some systems may also automatically slow
down or pull over the vehicle if the driver does not respond to the alert.

3.3.2 Limitation of the existing system

Some of the limitations of the existing driver drowsiness detection system includes:

i. False positive or negative detection.

ii. Lack of personalization.

19
 iii. Inability to detect other factors that cause driver impairment, such as alcohol or
drug use.
iv. Cost and complexity of system, can be prohibitive in some cases.
v. Privacy concern with cameras inside the car monitoring the driver.

3.4 System design

In this section, we shall be defining the elements of a driver drowsiness detecting
system like modules, architecture, components and their interfaces and data for a
system based on the specified requirements.

3.4.1 Modeling the Proposed System

System modeling is the process of creating abstract representations of systems using

mathematical, graphical, or physical models. The proposed driver drowsiness
detection system will be model using the architectural modeling which uses the
systems architecture to conceptually model the structure, behavior, and more views of
a system. Another type of systems modeling to be employed is the function modeling,
with specific techniques such as the Functional Flow Block Diagram and IDEF0.
These models are extended using functional decomposition, and linked to
requirements models for further systems partition.

3.4.2 Use case diagram of the proposed system

A use case diagram is a graphical depiction of a user's possible interactions with a

system. A use case diagram for the driver drowsiness detection system. Use case
diagram Figure 5 depict to use case diagram of the proposed system.

20
 Drowsiness Detection System

Open Application

Capture real time video

Access drowsiness detection

User

Alarm Start

Figure 1: Use case diagram of the proposed system

The use case diagram in figure 1, shows the interactions between the actors and use
cases, and typically a high-level overview of the system's functionality. The user is the
person operating the vehicle. The system continuously monitors the driver's level of
alertness through the camera which capture a real time video of the driver, tracking
the eyes and extracting useable features. The Detection System is responsible for
monitoring the driver's level of alertness and alerting them if necessary. If the system
detects signs of drowsiness (e.g., drooping eyelids, nodding head, etc.), it alerts the
driver with a warning alarm.

21
3.4.3 Sequence diagram
A sequence diagram is a type of UML (Unified Modeling Language) diagram that
shows the interactions between objects or components in a system, and the order in
which those interactions take place. A sequence diagram of drowsiness detection
system is as shown in the figure 4.

Drowsiness
Camera Eye Tracking Warning
Detection System

Driver

Capture Real Tracks the eye

time video movement
Generate an alert
Images of the
driver's face

Alert the driver

Figure 2: Sequence diagram of the proposed system

From the diagram above, the camera continuously captures images of the driver,
which would then be passed to the image processing module. The image
processing module would analyze the images to detect signs of drowsiness, such as
drooping eyelids or a slack jaw. If drowsiness is detected, the image processing
module would send a signal to the alarm, which would alert the driver to take a
break.

22
3.4.4 Input design
The driver's drowsiness detection system typically uses a combination of inputs to
detect when a driver is becoming drowsy. These inputs can include:

Eye tracking: The system uses a camera to track the driver's eye movements and
determine if they are becoming less responsive.

Facial recognition: The system uses a camera to analyze the driver's facial
expressions and determine if they are showing signs of drowsiness, such as drooping
eyelids or a slack jaw.

Head position: The system uses a camera or infrared sensor to track the driver's head
position and determine if they are nodding off or slouching.

Yaw rate: The system analyzes the driver's steering inputs and determine if they are
showing signs of drowsiness, such as oversteering or understeering.

The system then uses algorithms to process this information and determine if the
driver is becoming drowsy. If the system detects drowsiness, it will typically alert the
driver with a warning sound.

3.4.5 Functional Decomposition Diagram of the drowsiness detection system

A functional decomposition diagram for a driver drowsiness detection system is a
visual representation of the system that shows how its functions are broken down into
smaller, more manageable components.

23
 Camera

Image Processing

Drowsiness Detection Algorithm

Warning System

User interface

Figure 3: Functional Decomposition Diagram of the drowsiness detection system

The functional decomposition diagram of a driver drowsiness detection system in

figure 3 include the cameras that detect the driver's physiological signals and
movements. The Signal Processing unit processes the real time video captured by the
camera and converts it into useful information. The Drowsiness Detection Algorithm
processed data to determine the level of drowsiness of the driver. The Warning
System alerts the driver in case of drowsiness detection. The User Interface allows the
driver to interact and configure the system settings.

3.5 Program design

Program design of a drowsiness detection system refers to the process of creating a
plan or blueprint for a software program, outlining its goals, functionality, and
structure. It involves breaking down the problem to be solved into smaller,
manageable tasks, and deciding on the best algorithms and data structures to use to
accomplish these tasks. It also includes making decisions about the overall
architecture of the program, such as dividing it into modules, and considering issues
such as scalability and maintainability.

24
3.5.1 Class diagram of the proposed system
Class diagram describes the attributes and operations of a class and also the
constraints imposed on the system. Figure 6 therefore is the class diagram of the
proposed system.

CAMERA DROWSY DRIVER Alert System

DETECTION

+frame +drowsyLevel
+alert_system
+camera
+sensor - sendAlert()
- get_frame()

-detect_drowsiness()

SENSOR

+drowsyLevel

+detect_drowsiness()

Figure 4: class diagram of a driver's drowsiness detection system

Figure 4 revealed that the Drowsy Driver Detection class represents the overall
system for detecting drowsy drivers. It contains an instance of the Alert System,
Camera, and Sensor classes. The detect_drowsiness method is called in a loop to
continuously capture frames from the camera, and then uses the sensor to detect if the
driver is drowsy. If the sensor detects drowsiness, an alert is sent using the alert
system. The Alert System class is responsible for sending an alert when the driver is
detected to be drowsy. The send_alert method is called by the Drowsy Driver
Detection class when drowsiness is detected. The Camera class is responsible for
capturing frames from the camera. The get_frame method is used to capture a single
frame, which is then passed to the sensor to detect drowsiness. The Sensor class is
responsible for analyzing the frames captured by the camera to detect if the driver is

25
drowsy. The detect_drowsiness method is passed a frame, and uses image processing
techniques to analyze the frame and determine if the driver is drowsy.

3.5.2 Program Algorithm

This section states clearly the algorithm for a driver drowsiness detection system.
1. Initialize the camera and start capturing video frames.

2. Convert each frame to grayscale and apply a haar cascade classifier to detect
the driver's face.
3. Use dlib's facial landmark detector to detect the driver's eyes.
4. Calculate the aspect ratio of the driver's eyes (the ratio of the width to the
height of the eyes).
5. Compare the aspect ratio to a threshold value. If the aspect ratio is below the
threshold, the driver is considered to be drowsy.
6. If the driver is drowsy, sound an alarm or send a notification to the driver.
7. Repeat steps 2-6 for each frame captured by the camera.

3.5.2.1Flowchart
A flowchart is a type of diagram that represents a process or system by showing the
steps or tasks as boxes connected by arrows. The boxes represent the steps or tasks in
the process, and the arrows show the flow of control or the order in which the steps
are performed. Figure 5 therefore shows the pictorial algorithm for the proposed
driver drowsiness detection system.

26
 Start

Initialization

Camera

Alert System

Capture Frame

Analyze Frame

While the Yes Check drowsy

system is active
level

Send alert
Exit Application

Stop

Figure 5: Flowchart of the proposed model

Figure 5 shows the overall structure of the program, starting with the initialization of
the system, then entering a continuous loop to detect drowsiness. In each iteration of
the loop, the program captures a frame from the camera, analyzes the frame to detect
drowsiness, and if drowsiness is detected, sends an alert. The loop continues until the
program is stopped or the system is reset.

3.5.3 Proposed system framework/architecture

The proposed driver drowsiness detection system is designed using Computer Vision
model that will be optimized by Adam optimizer. Block diagram of the proposed
system framework is as shown in Figure 6.

27
 Figure 6: Block diagram of the proposed system

Dataset: The dataset for this project is acquired from Kaggle (an online dataset
repository), the dataset contains over 41,790 images, which are been grouped into two
path – drowsy and non-drowsy respectively.
The dataset was then split into two.
i. Training data: The subset of the data used to train the model
ii. Testing data: the subset of the data used to test the model to see how it
performs on unseen data instances.
In this research, 70% of the dataset was used for training the model and the remaining
30% was used for testing the model.
3.5.4 Neural Network Architecture of the Proposed System
Figure 7, shows the ANN architecture of Driver Drowsiness detection, which
comprises of the Input layer, Convolutional Layer, Max-Pooling Layer, Normalization
Layer, Fully Connected Layer and the Output Layer.

28
 Figure 7: The proposed neural network architecture

An artificial neural network (ANN) for driver drowsiness detection using computer
vision have the following architecture:

i. Input Layer: This layer would receive the video frames captured by a camera
as input.
ii. Convolutional Layer: This layer would apply filters to the input frames,
allowing the network to extract features such as edges, shapes, and textures.
The convolutional layer can have multiple convolutional layers stacked on top
of each other, forming a deep CNN.
iii. Max-Pooling Layer: This layer would reduce the spatial resolution of the
feature maps, which would help the network to be less sensitive to small
variations in the position of the driver's face.
iv. Normalization Layer: This layer would normalize the output of the previous
layers, which would help to speed up the training process and improve the
overall performance of the network.

29
 v. Fully Connected Layer: This layer would take the output from the previous
layers and connect it to the output layer, providing the final prediction of the
driver's drowsiness level.
vi. Output Layer: This layer would produce the final prediction of the driver's
drowsiness level, which could be a binary outcome (awake or drowsy) or a
continuous value representing the degree of drowsiness.

Optimizing the neural network with Adam Optimizer: After training the model, it is
been subjected to Adam optimizer to determine the best selection of the hyper-
parameters of the model to archive maximum accuracy and performance.

Adam (Adaptive Moment Estimation) is a gradient descent optimization algorithm

that is based on the combination of the ideas from two other optimization algorithms:
AdaGrad and RMSProp. The Adam algorithm uses the gradient information from the
current iteration as well as the historical gradient information to adapt the learning rate
for each parameter. It also uses moving averages of the parameters, similar to
momentum. The algorithm keeps track of the mean (first moment) and variance
(second moment) of the gradients, and uses these estimates to update the parameters.
The learning rate is adjusted based on the ratio of the current gradient to the historical
gradient. This helps to improve the stability of the optimization process and often
leads to faster convergence.

The update rule for Adam optimizer involves the following equations:

1. Initialize the moving averages of the gradient and squared gradient, denoted by
m (first moment) and v (second moment) respectively, with 0.
2. For each iteration, calculate the gradient of the loss function with respect to the
parameters (denoted by g), and update the moving averages:

m=β 1∗m+ ( 1− β 1 )∗g (1)

2
v=β 2∗v + ( 1−β 2 )∗g (2)

30
 3. Correct the bias in the moving averages by dividing them with

(1 - β 1t ) and (1 - β 2 t) respectively, where t is the current iteration number.

4. Use the corrected moving averages to update the parameters:

ϴ−α∗m
ϴ= (3)
√ v+ ɛ

In the above equations:

 α is the learning rate

 β1 and β2 are the decay rates for the moving averages
 ɛ is a small constant added for numerical stability

Adam optimizer also has a few hyperparameters such as the learning rate (α), the
decay rates for the moving averages (β1 and β2) and epsilon. These hyperparameters
can be tuned to improve the performance of the algorithm.

3.5.5 Advantages of the proposed system

i. Real-time monitoring: Computer vision-based drowsiness detection systems
can process and analyze video streams in real-time, providing immediate
alerts when a driver's level of drowsiness is detected.

ii. Non-invasive: Unlike other drowsiness detection methods, such as monitoring

brain waves or measuring eyelid movement, computer vision-based systems
do not require any physical contact with the driver.
iii. High accuracy: Computer vision-based systems can use multiple cues, such as
facial landmarks, head pose, and eye movement, to accurately detect
drowsiness.
iv. Cost-effective: Computer vision-based drowsiness detection systems can be
implemented using off-the-shelf cameras and computer hardware, making
them relatively inexpensive compared to other drowsiness detection
methods.

31
 v. Easy to implement: There are many open-source libraries and pre-trained
models available for computer vision-based drowsiness detection, which can
be easily integrated into existing systems.
vi. Versatile: Computer vision-based drowsiness detection systems can be used in
a variety of settings, such as cars, buses, trucks, and trains.

3.6 Choice of Programming Tools

The Python programming language coded in PyCharm IDE will be use for the
development of the driver drowsiness detection system to create, debug, maintain, or
otherwise support other programs and applications. The most basic tools are a source
code editor and a compiler or interpreter, which are used ubiquitously and
continuously.

32
 CHAPTER FOUR
IMPLEMENTATION AND RESULT
4.1 Implementation
Implementation refers to the final process of moving the solution from development
status to production status. The implementation phase is crucial to the software
development process, as it is the phase where the software design is turned into a
functioning software system. It involves a lot of coding, testing, debugging, and
integration activities, and may requires the collaboration of different teams within the
software development organization.

Driver drowsiness detection using machine image processing involves capturing

images or videos of the driver's face while driving and analyzing them to detect signs
of drowsiness or fatigue. The implementation steps include collecting and
preprocessing training data, training a machine learning model, deploying the model
in a real-time system, detecting drowsiness, alerting the driver, and continuously
improving the system. The signs of drowsiness that can be detected include drooping
eyelids, yawning, and head nodding.

4.2 System requirements

System requirements are the configuration that a system must have in order for a
hardware or software application to run smoothly and efficiently. Failure to meet these
requirements can result in installation problems or performance problems. In this
research work, the system requirements are further categorized into two which are the
hardware requirement and software requirement. These requirements are as explained
below.
4.2.1 Hardware requirements
These are the physical machine or equipment involved in the system. They can be
seen, touched, and felt. The hardware requirements for this project include:
i. A personal computer.
ii. At least Pentium IV microprocessor of 1.0GHZ
iii. At least snapdragon 435 Qualcomm microprocessor of 1400MHZ
iv. At least RAM with size of about 4GB and above

33
 v. Minimum 500 gigabytes of hard disk space

4.2.2 Software requirement

The implementation of the design requires several software components including
Anaconda interpreter, Tkinter, PyCharm IDE, Jupyter notebook, and a minimum of
Windows 8 operating system.

Anaconda interpreter provides support for machine learning and data analytics,
Tkinter is a library used for creating GUIs, PyCharm is the IDE used for writing
Python codes, and Jupyter notebook is used for model training and analysis. A
minimum of Windows 8 operating system is required to avoid DLL issues.

4.3 Choice of Programming Language

The proposed system was implemented using Python due to several advantages it
offers. These include the fact that it is a high-level programming language, it supports
the development of GUI interfaces using Tkinter library, it is easy to read and learn,
and it has built-in functions that facilitate the development of machine learning
models such as pandas and numpy. Additionally, Python comes with packages pre-
loaded with machine learning models and optimization techniques, and it is not
platform dependent.

4.4 Training the model

Training the driver drowsiness detection model involves using a dataset of images or
videos of drivers in both alert and drowsy states to teach the machine learning
algorithm how to distinguish between the two states based on specific facial features
and patterns.

During the training process of the driver drowsiness detection model, at first a dataset
of images of drivers in both alert and drowsy states was collected. The dataset was
preprocessed by resizing, cropping, and normalizing the images to ensure consistency,
and it was balanced with an equal number of images for each class. Relevant features
such as eye closure, head pose, and facial expression were then extracted from the
dataset using deep neural network technique. The CNN algorithm was selected, and
the model was trained by iteratively adjusting the algorithm parameters until it

34
achieves the desired accuracy. The model was then validated using a separate dataset
to evaluate its performance and tested on real-time data. The model was refined based
on the results of testing and validation, which involve adjusting the algorithm
parameters, using a different feature extraction technique. Figure 8 depict the loss
function and accuracy graph as visualized during the training.

Figure 8: Loss function and accuracy graph

35
4.5 Program Testing
Program testing is a process of evaluating a software program or system to identify
any errors, bugs, or defects in the software. It is an essential step in the software
development process and is conducted to ensure that the software system meets its
intended requirements and specifications. The testing process requires a significant
amount of planning, design, execution, and reporting, and it is crucial to involve
experienced testers to ensure the software system's quality.

Testing the driver drowsiness detection system is an essential step in ensuring its
effectiveness and reliability. The system was tested as follows:

i. Camera test: The driver drowsiness detection system relies on the camera to
detect the driver's behavior. Therefore, it is crucial to test the camera to ensure
that it is functioning correctly. This test was carryout by checking the camera's
accuracy, sensitivity, and reliability in detecting the driver's behavior, such as
head movements and eye closures.
ii. Testing the detection algorithm: The algorithm was tested to ensure that it can
accurately detect the driver's drowsiness level based on the camera inputs. The
ability of the algorithms to distinguish between normal behavior and
drowsiness was evaluated.
iii. Testing the notification system: The driver drowsiness detection system needs
to notify the driver when it detects drowsiness. The notification system was
tested to ensure that it is working correctly and providing timely alerts to the
driver.
iv. Test in different environments: The driver drowsiness detection system was
tested in different environments, such as day and night, with different lighting
conditions, and on different road types. Testing in various environments help to
ensure that the system can detect drowsiness accurately and consistently in
different driving conditions.
v. Test with different drivers: The driver drowsiness detection system was tested
with different drivers to ensure that it can detect drowsiness for a wide range of
individuals. Testing with different drivers helps to identify any issues with the

36
 system's accuracy or sensitivity based on individual differences in behavior and
physiology.
vi. Test for false positives and false negatives: It is essential to test the system for
false positives (alerting the driver when they are not drowsy) and false
negatives (failing to detect drowsiness). Testing for false positives and false
negatives help to ensure that the system is not alerting the driver unnecessarily
or failing to alert the driver when they are drowsy.

4.6 Result Discussion

Result discussion is a crucial component of any research or project, as it helps to
interpret and explain the findings of the study. In the context of driver drowsiness
detection using machine image processing, the result discussion is focus on the result
obtained from the test carried out on the system. The result of the test is as shown in
the table 4.1

Table 4.1: Result Discussion

4 Test Result
1. Camera test The system was able to initialize the camera
for live video capturing.

2. Algorithm test The proposed algorithm was able to detect a

drowsy driver in a live video.

3. System notification The system was able to notify the driver when
drowsy.

4. Test in different The system function accurately under

environments different environments.

5. Test on different drivers. The system was able to detect alert and
drowsy drivers from live videos of several
drivers.

6. Test for false alarm The system was able to alert the driver only
when drowsy.

37
4.7 Performance evaluation
Performance evaluation is a way assessing the effectiveness of a driver drowsiness
detection system. The performance of the system is be evaluated based on various
metrics which include accuracy, precision, recall, and F1 score.

Accuracy refers to the percentage of correctly classified instances out of all instances.
Precision measures the proportion of true positives (correctly identified drowsy
drivers) among all instances classified as positive (identified as drowsy drivers).
Recall measures the proportion of true positives among all actual positive instances.
F1 score is the harmonic mean of precision and recall.

The driver drowsiness detection model was trained on a dataset of 1000 video clips,
where each clip is labeled as either "drowsy" or "not drowsy"; 200 of these video clips
were us as a test set, and the model correctly classifies 195 of them.

To calculate accuracy, we divide the number of correctly classified videos by the total
number of videos in the test set, and then multiply by 100 to get a percentage:

Number of correct predictions

Accuracy= ×100
Total Number of predictions

195
Accuracy= × 100
200

Accuracy = 97.5%

4.8 Driver drowsiness detection system interface

The user interface (UI) is the point of human-computer interaction and
communication in a device. This can include screens, keyboards, a mouse, and the
appearance of a desktop. It is also how a user interacts with an application. Figure 9
show the first interface of the system.

38
 Figure 9: Screenshot Home interface
Figure 9 shows the home interface of the Driver drowsiness detection system. At this
interface, the drover can click the start button to start the detection process.

Figure 10: Monitoring in progress (At alert)

Figure 9 depict the screenshot of the system while monitoring the user level of
alertness. Here, using the parameters such as head position, eyelids etc the system
concludes the user is at alert.

39
 Figure 11: Monitoring in progress (Drowsy)
Figure 10 depict the screenshot of the system while monitoring the user level of
alertness. Here, using the parameters such as head position, eyelids etc the system
concludes the user is drowsy for about 2 seconds.
4.9 System documentation
Software documentation refers to the written or visual material that
accompanies software and explains its functions, architecture, design, and
usage. It serves as a guide for software developers, maintainers, and users. The
purpose of software documentation is to provide an overview of the software
system, its components, and their interactions.
Following is the procedure to use the system:
1. Double-click on the application icon on the desktop to launch the
application
2. Click on the start button to commence the monitoring process
3. You can on stop button to pause/stop the monitoring process

40
 CHAPTER FIVE
SUMMARY CONCLUSION AND RECOMMENDATION
5.1 Summary
Driver drowsiness detection systems have been developed to improve road safety by
alerting drivers when they are experiencing drowsiness or fatigue. This research
focuses on a driver drowsiness detection system that uses image processing techniques
to monitor the driver's facial features and detect signs of drowsiness.

The proposed system involves using a camera to capture videos of the driver's face
while driving, and analyzing them to detect signs of drowsiness or fatigue. The system
follows several steps including collecting a dataset of images or videos of drivers
showing both drowsy and alert states, preprocessing the data to remove noise,
adjusting brightness and contrast, and cropping images to focus on the driver's face.

The system then extracts features from the images or videos using deep neural
networks. The features extracted is relevant to detecting drowsiness, such as eye
closure, head pose, and facial expression. The system also involves training a machine
learning model using the preprocessed data and selected algorithm.

The performance of the driver drowsiness detection system was evaluated using
accuracy metrics. The accuracy of the system can be measured using metrics such as
true positive rate, false positive rate, precision, recall, and F1 score. To implement the
designed or the proposed system, Python was used because of its benefits, such as
supporting the development of GUI interfaces, pre-loaded machine learning models
and optimization techniques, and being platform-independent. The software
components required for proper implementation of the design and running of the
implemented software include Anaconda Interpreter, Tkinter, PyCharm, Jupyter
Notebook, and a minimum of Windows 8 operating system.

5.2 Conclusion
In conclusion, driver drowsiness detection systems using image processing have
become an important area of research in recent years due to the increasing number of

41
road accidents caused by drowsy driving. This research has shown that machine
learning algorithms can be trained to accurately detect drowsiness based on facial
features extracted from images or videos of drivers. The proposed system utilizes
image processing techniques to extract relevant features from the driver's face and
train a machine learning model to accurately detect drowsiness. The implementation
of the proposed system has shown promising results in detecting drowsiness in real-
time scenarios.

However, there is still room for improvement in the proposed system. For instance,
the system can be enhanced by integrating other sensors such as steering wheel angle
sensors and EEG sensors to improve the accuracy of the detection system. Also, the
system can be extended to detect other driving behaviors such as distracted driving
and aggressive driving. Overall, the proposed system is a step in the right direction
towards improving road safety and reducing the number of accidents caused by
drowsy driving.

5.3 Recommendations
The driver drowsiness detection systems can be useful in a wide range of areas where
it is critical to ensure that drivers remain alert and attentive while driving. The system
is therefore recommended in the followings domains:

i. Automotive safety: e.g., to improve safety by alerting drivers when they are at
risk of falling asleep or becoming too distracted to drive safely. In commercial
vehicles, such as trucks and buses, where long hours behind the wheel can lead
to fatigue and drowsiness.
ii. Fleet management: To monitor driver behavior and ensure that drivers are
operating vehicles safely and efficiently, and to help fleet managers to prevent
accidents and reduce the risk of injuries and property damage.
iii. Healthcare: e.g., to monitor patients with sleep disorders, such as narcolepsy or
sleep apnea, that can cause excessive daytime sleepiness and increase the risk
of accidents while driving.

42
iv. Transportation and logistics: To monitor drivers and ensure that shipments are
delivered safely and on time. By detecting signs of drowsiness or distraction,
these systems can help to prevent accidents and delays.

43
REFERENCES
Alpaydin, E. (2010). Introduction to machine learning (2nd ed.). Cambridge, MA:
MIT Press. pp 3-10

Anirban D. (2018). Drowsy driving statistics and facts. Sleep Foundation. Retrieved
on [21/05/2023] from https://www.sleepfoundation.org/articles/drowsy-
driving-statistics-and-facts pp 120-133

Boukhechba, M., Azzag, H., & Zeroual, A. (2019). Driver drowsiness detection based
on artificial neural networks. International Journal of Computer Applications,
181(1):256-325

by X. Zhou, Y. Wang, and J. Lu. (2020): "Real-time Eye State Tracking using a
Multi-task Deep Neural Network"; Published in the Proceedings of the
International Joint Conference on Neural Networks (IJCNN). pp25-31.
Chau, J. L., Griffiths, M. D., & Chan, H. Y. (2017). Fatigue and energy management
in the workplace: The development of the "Coping with Fatigue" training
program. Industrial Health, 55(2):100-109. doi: 10.2486/indhealth.2016-0114.

Chau, J. Y., Qian, X., & Wong, T. K. (2017). Fatigue and mental health in the
workplace. In Occupational Health Psychology: From Burnout to Wellbeing.
36(2):256-331

Chen, C., Li, J., & Zhao, G. (2018). Real-time drowsiness detection for safe driving
using convolutional neural networks. IEEE Access, 6: 63200-63207.

Chen, L., Zhang, H., Liu, B., & Cao, Y. (2018). Driver drowsiness detection using a
convolutional neural network. Sensors, 18(8):2391.

Chen, Y., Li, W., Chen, S., & Zhao, G. (2018). A deep learning-based approach to
driver drowsiness detection using facial expression and head movement. IEEE
Access, 6:15186-15195.

Dasgupta, A. (2018). The impact of driver fatigue on commercial motor vehicle

safety. Safety Science, 109:237-245. doi: 10.1016/j.ssci.2018.06.006.

Dawson, D., Reid, K. J., & Fatigue, F. T. (2017). Fatigue, alcohol and performance
impairment. Nature, 388(6639):235-235. doi: 10.1038/40879.

Dinges, D. F. (2015). Sleep and fatigue in shift work and shift work disorder. Chest,
147(4):1480-1488. doi: 10.1378/chest.14-0970.(pp 9-11).

44
Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image super-resolution using deep
convolutional networks. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 38(2):295-307.

Fletcher, J. (2017). Driver fatigue: A review of the literature. Retrieved on

[12/05/2023] from
https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/driver_fatigue_0.pdf
pp 58-71

Fletcher, R. (2017). Drowsy and distracted driving: A research-based guide for states.
Governors Highway Safety Association. 36(9):25-36

Friedrichs, E. M., & Yang, L. (2010). The impact of sleep deprivation and fatigue on
truck driving safety. Washington, DC: National Transportation Safety Board.
36:36-40

Friedrichs, F., & Yang, X. (2010). Driver fatigue and driving safety: The 2010
comparative international survey on fatigue-related crashes. AAA Foundation
for Traffic Safety. Pp 25-31.

García-González, J., Rodríguez-Fórtiz, M. J., de la Cruz, E. M., & Martínez-López, F.

J. (2021). Improving early detection of driver drowsiness by using a deep
learning approach based on facial landmarks. Sensors, 21(5):1547.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. Cambridge, MA:
MIT Press. Pp 36-42

Hassan, M. A., Malik, A. S., & Siddiqi, M. H. (2017). Driver drowsiness detection
system using facial feature extaction. In 2017 IEEE 13th International
Colloquium on Signal Processing & its Applications (CSPA) IEEE. pp 68-712

Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning:
Data mining, inference, and prediction (2nd ed.). New York: Springer. pp 10-
11

Izquierdo, M., Nieto, M., & Monzón, M. (2014). Analysis of driver drowsiness
detection techniques based on driving performance. Transportation Research
Part C: Emerging Technologies, 44:161-178.

Jin, Z., Zhang, W., Xiong, S., & Qiu, L. (2018). Driver drowsiness detection based on
convolutional neural network. Journal of Advanced Transportation 2018.
36(2):672-690

45
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and
prospects. Science, 349(6245):255-260.

Kapoor, A., Verma, N., & Taneja, N. (2018). Drowsiness detection techniques: A
review. In 2018 3rd International Conference on Computing Methodologies
and Communication (ICCMC). pp 868-871.

Kapoor, S., Verma, T., & Taneja, S. (2018). A review of drowsy driver detection
systems. Journal of Computational and Theoretical Nanoscience, 15(9):4137-
4142. https://doi.org/10.1166/jctn.2018.7745

Kim, J., Kim, S., Lee, S., & Paik, J. K. (2020). Real-time facial expression recognition
using deep learning. IEEE Transactions on Affective Computing, 11(3):373-
380. doi: 10.1109/TAFFC.2018.2881229.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with
deep convolutional neural networks. In Advances in Neural Information
Processing Systems. 85(6):1097-1105.

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553):436-
444.

LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., &
Jackel, L. D. (1989). Backpropagation applied to handwritten zip code
recognition. Neural Computation, 1(4):541-551.

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning
applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324.

Li, R., Liu, C., Liu, J., & Chen, S. (2018). Driver drowsiness detection using EEG and
SVMs. Journal of Advanced Transportation. 35(2):582-599.

Li, Y., Zhang, Y., Li, X., & Meng, Q. (2018). Driver drowsiness detection using EEG
and machine learning techniques. IEEE Transactions on Intelligent
Transportation Systems, 19(6):1882-1891.

Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for
semantic segmentation. In Proceedings of the IEEE Conference on Computer
Vision and Pattern Recognition. pp 3431-3440.

Ma, C., Bi, L., & Wu, X. (2021). Driver drowsiness detection based on deep neural
network with physiological signals. Journal of Advanced Transportation.
54(3):84-86
46
Ma, H., Wang, C., & Wu, J. (2021). A driver drowsiness detection method based on
deep learning. Journal of Intelligent & Fuzzy Systems, 40(5):9135-9146.

McDonald, A. D., Lee, H. C., Aksan, N., & Lai, C. H. (2012). Driver fatigue and road
safety: A critical review of the literature. Transportation research part F: traffic
psychology and behaviour, 15(5):576-593.

Munir, A., Abdullah, A. H., & Almahdi, E. S. (2020). A system to detect driver
drowsiness using EEG signals. Journal of Engineering and Applied Sciences,
15(8):1867-1873.

Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., &
Muharemagic, E. (2015). Deep learning applications and challenges in big data
analytics. Journal of Big Data, 2(1):1-21.

National Highway Traffic Safety Administration. (2021). Drowsy driving. Retrived on

[23/05/2023] from: https://www.nhtsa.gov/risky-driving/drowsy-driving pp. 8-
10.

National Sleep Foundation. (2020). Drowsy driving. Retrieved from on [17/05/2023]

from: https://www.sleepfoundation.org/articles/drowsy-driving pp. 8-9

Rajesh, R., Sastry, P. S. S. S., & Sastry, R. S. S. S. (2020). Non-Invasive Pulse Rate
Monitoring Using Photoplethysmography and Machine Learning. IEEE
Journal of Biomedical and Health Informatics, 24(1):208-215. doi:
10.1109/JBHI.2019.2891258.
Samiee, S., Lee, Y., & Baek, J. (2014). A novel method for the estimation of driver’s
fatigue based on multiple measurements. IEEE transactions on intelligent
transportation systems, 15(2):542-551.

Soares, S., Monteiro, P., Lobo, J., Couto, A., Cunha, J. P. S., & Ferreira, J. J. (2020).
Road Safety and Driving Performance in Sleep-Deprived Individuals: A
Systematic Review. Medicina, 56(9):431. doi: 10.3390/medicina56090431

Sommer, D., & Golz, M. (2010). Fatigue detection based on a single channel
electroencephalography. Proceedings of the 4th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications, pp 80-87.

Sun, Y., Wang, X., & Tang, X. (2013). Deep convolutional network cascade for facial
point detection. In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition. pp 3476-3483.

47
Tziviskou, C., Yannis, G., & Laiou, A. (2019). Use of new vehicle technologies for
fleet safety: An international review of current practice. Transportation
Research Part A: Policy and Practice, 125:327-338.

Tziviskou, E., Kostoulas, T., & Tziviskou, C. (2019). Drowsiness detection systems in
automotive industry: State of the art and future perspectives. In 2019 10th
International Conference on Information, Intelligence, Systems and
Applications (IISA). pp. 1-6.

Wang, J., Luan, Y., & Xue, Y. (2017). Driver drowsiness detection based on eye
closure and head pose features. IEEE Transactions on Intelligent
Transportation Systems, 18(12):3506-3515.

Wang, S., Yin, Z., & Liu, X. (2019). Real-time head pose and gaze estimation using
deep learning. IEEE Transactions on Circuits and Systems for Video
Technology, 29(11):3344-3357. doi: 10.1109/TCSVT.2019.2929503.
WorkplaceTesting. (2019, January 27). Drowsiness. WorkPlaceTesting.com.
https://www.workplacetesting.com/definition/3021/drowsiness pp 25-32
Yang, J., Wright, J., Huang, T., & Ma, Y. (2019). Image super-resolution via sparse
representation. IEEE Transactions on Image Processing, 19(11):2861-2873.

Yang, X., Chen, Y., & Li, K. (2009). Review on the development of driver fatigue
detection system. Journal of Transportation Systems Engineering and
Information Technology, 9(5):36-41.

Yang, Y., Zhang, Y., Chen, T., Zhang, Z., & Gao, D. (2019). Deep learning for single
image super-resolution: A brief review. IEEE Transactions on Multimedia,
21(12):3106-3121.

Yu, H., Yin, Z., Chen, X., & Li, Q. (2018). Driver drowsiness detection using support
vector machine with feature fusion from multi-physiological signals. Sensors,
18(9):2969-2978

Yu, Z., Yu, Z., Huang, K., & Wang, X. (2018). A novel driver drowsiness detection
approach using support vector machine. IEEE Access, 6:58024-58032.

Yusuf, S. (2020). Recent advances in deep learning algorithms. Journal of Big Data,
7(1):1-36.

48
APPENDIXES

Appendix A: SOURCES CODE

import os
import tkinter
import tkinter.messagebox
from tkinter import *
import cv2
import numpy as np
from PIL import Image, ImageTk
from keras.models import load_model
from pygame import mixer

mixer.init()
sound = mixer.Sound('Please Be At alert P.wav')
face = cv2.CascadeClassifier('haar cascade files\haarcascade_frontalface_alt.xml')
leye = cv2.CascadeClassifier('haar cascade files\haarcascade_lefteye_2splits.xml')
reye = cv2.CascadeClassifier('haar cascade files\haarcascade_righteye_2splits.xml')
model = load_model('models/cnncat2.h5')
path = os.getcwd()
root = Tk()
root.title("Driver Drowsiness Detection System")
root.geometry("700x540")
root.configure(bg='#b0cff9')
root.state("zoomed")
root.bind('<Escape>', lambda e: root.quit())
root.resizable(width=False, height=True)
# window icon
p1 = PhotoImage(file='images/logo.png')
root.iconphoto(False, p1)

bg_image = Image.open('images/bg.png')

resize_image = bg_image.resize((1362, 710))

img_bg = ImageTk.PhotoImage(resize_image)

bg = PhotoImage(file='images/bg.png', master=root)

bg_label = Label(root, image=img_bg)

bg_label.place(x=0, y=0)

Label(root, text="DRIVER DROWSINESS DETECTION SYSTEM", height=1,

width=800, background='#b0cff9', font="Helvetica, 20",
fg='black').pack()

49
labelFrame = LabelFrame(root, text="About the Application", font="Calibri, 15")

labelFrame.place(anchor="e", relx=0.3, rely=0.5)

left.pack(ipadx=10)

f1 = LabelFrame(root, highlightbackground='teal', highlightthickness=3)

f1.place(anchor='e', relx=0.95, rely=0.5)
l1 = Label(f1)
bg1 = PhotoImage(file='images/logo.png', master=l1)
l1.pack()
l1["image"] = bg1

label_score = Label(f1, background="black", font=("Tw Cen MT Condensed Extra

Bold", 14),
foreground='white')
flag = 0

def reset_flag():
global flag
if flag == 1:
flag = 0
l1["image"] = bg1
sound.stop()
label_score.place_forget()
f1.configure(highlightbackground='teal', highlightthickness=3)
btn_play.config(bg='green', text='START')
else:
flag = 1
btn_play.config(bg='red', text='STOP')
def start_monitoring():
cap = cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_COMPLEX_SMALL
count = 0
score = 0
thicc = 2
rpred = [99]
lpred = [99]

# reset the flag

reset_flag()

# Capture video from webcam

while flag > 0:
ret, frame = cap.read()
img = cv2.cvtColor(frame, cv2.COLOR_BGR2RGBA)

50
 img = ImageTk.PhotoImage(Image.fromarray(img))
height, width = frame.shape[:2]
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = face.detectMultiScale(gray, minNeighbors=5, scaleFactor=1.1,
minSize=(25, 25))

right_eye = reye.detectMultiScale(gray)
left_eye = leye.detectMultiScale(gray)

cv2.rectangle(frame, (0, height - 50), (200, height), (0, 0, 0),

thickness=cv2.FILLED)

for (x, y, w, h) in faces:

cv2.rectangle(frame, (x, y), (x + w, y + h), (100, 100, 100), 1)

for (x, y, w, h) in right_eye:

r_eye = frame[y:y + h, x:x + w]
count = count + 1
r_eye = cv2.cvtColor(r_eye, cv2.COLOR_BGR2GRAY)
r_eye = cv2.resize(r_eye, (24, 24))
r_eye = r_eye / 255
r_eye = r_eye.reshape(24, 24, -1)
r_eye = np.expand_dims(r_eye, axis=0)
rpred = model.predict(r_eye)

for (x, y, w, h) in left_eye:

l_eye = frame[y:y + h, x:x + w]
count = count + 1
l_eye = cv2.cvtColor(l_eye, cv2.COLOR_BGR2GRAY)
l_eye = cv2.resize(l_eye, (24, 24))
l_eye = l_eye / 255
l_eye = l_eye.reshape(24, 24, -1)
l_eye = np.expand_dims(l_eye, axis=0)
lpred = model.predict(l_eye)

# Check if either eye is closed

if np.any(lpred == 1) or np.any(rpred == 1):
score = 0
sound.stop()
label_score.configure(text='At Alert')
label_score.place(anchor='e', relx=0.2, rely=0.95)
f1.configure(highlightbackground='green', highlightthickness=5)
else:
score = score + 1
label_score.configure(text="Drowsy Score: " + str(score))
label_score.place(anchor='e', relx=0.2, rely=0.95)

51
 if score > 0:
cv2.putText(frame, 'Score:' + str(score), (100, height - 20), font, 1, (255, 255,
255), 1, cv2.LINE_AA)

while score > 3:

f1.configure(highlightbackground='red', highlightthickness=5)
cv2.imwrite(os.path.join(path, 'image.jpg'), frame)
try:
sound.play()
score = 0
except: # is playing = False
pass
if thicc < 16:
thicc = thicc + 2
else:
thicc = thicc - 2
if thicc < 2:
thicc = 2
cv2.rectangle(frame, (0, 0), (width, height), (0, 0, 255), thicc)
l1["image"] = img
root.update()
cap.release()
cv2.destroyAllWindows()

def on_closing():
if tkinter.messagebox.askokcancel("Exit", "Do you want to quit application?"):
global flag
flag = 1
start_monitoring()
root.destroy()

btn_play = Button(root, text="START", font=("Tw Cen MT Condensed Extra Bold",

14),
background='green', foreground='white', width=25)
btn_play.bind('<Button>', lambda e: start_monitoring())
btn_play.place(relx=0.65, rely=0.9)

root.protocol("WM_DELETE_WINDOW", on_closing)

if __name__ == '__main__':
root.mainloop()

52