Driver Drowsiness Detection

A mini project report submitted by

VARSHA S (URK18CS273)

In partial fulfillment for the award of the degree of

BACHELOR OF TECHNOLOGY
In
COMPUTER SCIENCE AND ENGINEERING

Under the supervision of

MRS. T. MATHU, Asst. Professor

COMPUTER SCIENCE AND ENGINEERING

KARUNYA INSTITUTE OFTECHNOLOGY AND SCIENCES

(Declared as Deemed to be University -under Sec-3 of the UGC Act, 1956)

Karunya Nagar, Coimbatore - 641 114. INDIA

MARCH 2021
 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

BONAFIDE CERTIFICATE

This is to certify that the project report entitled, “Driver Drowsiness Detection “is a bonafide record of Mini
Project work done during the even semester of the academic year 2020-2021 by

VARSHA S (Reg. No: URK18CS273)

in partial fulfillment of the requirements for the award of the degree of Bachelor of Technology in Computer
Science and Engineering of Karunya Institute of Technology and Sciences.

Submitted for the Viva Voce held on _________________

Project Coordinator Signature of the Guide

 ACKNOWLEDGEMENT

First and foremost, I praise and thank ALMIGTHY GOD whose blessings have bestowed in me the

will power and confidence to carry out my project.

I am grateful to our beloved founders Late.Dr. D.G.S. Dhinakaran, C.A.I.I.B, Ph.D and Dr. Paul

Dhinakaran, M.B.A, Ph.D, for their love and always remembering us in their prayers.

I extend my thanks to our Vice Chancellor Dr.P. Mannar Jawahar, Ph.Dand our Registrar Dr.

Elijah Blessing, M.E., Ph.D,for giving me this opportunity to do the project.

I would like to thank Dr. Prince Arulraj, M.E., Ph.D.,Dean, School of Engineering and Technology

for his direction and invaluable support to complete the same.

I would like to place my heart-felt thanks and gratitude toDr. J. Immanuel John Raja, M.E., Ph.D.,

Head of the Department, Computer Science and Engineering for his encouragement and guidance.

I feel it is a pleasure to be indebted to, Mr. J. Andrew, M.E, (Ph.D.), Assistant Professor,

Department of Computer Science and Engineering and Mrs. T. Mathu for their invaluable support, advice and

encouragement.

I also thank all the staff members of the Department for extending their helping hands to make this

project a successful one.

I would also like to thank all my friends and my parents who have prayed and helped me during the

project work.
 Abstract

Drowsiness and fatigue are one of the main causes leading to road accidents. They can be
prevented by taking effort to get enough sleep before driving, drink coffee or energy drink, or
have a rest when the signs of drowsiness occur. The popular drowsiness detection method uses
complex methods, such as EEG and ECG. This method has high accuracy for its measurement
but it need to use contact measurement and it has many limitations on driver fatigue and
drowsiness monitor [18]. Thus, it is not comfortable to be used in real time driving. This paper
proposes a way to detect the drowsiness signs among drivers by measuring the eye closing rate
and yawning.

This project describes on how to detect the eyes and mouth in a video recorded from the. In the
video, a participant will drive the driving simulation system and a webcam will be place in front
of the driving simulator. The video will be recorded using the webcam to see the transition from
awake to fatigue and finally, drowsy. The designed system deals with detecting the face area of
the image captured from the video. The purpose of using the face area so it can narrow down to
detect eyes and mouth within the face area. Once the face is found, the eyes and mouth are found
by creating the eye for left and right eye detection and also mouth detection.

The parameters of the eyes and mouth detection are created within the face image. The video
were change into images frames per second. From there, locating the eyes and mouth can be
performed. Once the eyes are located, measuring the intensity changes in the eye area determine
the eyes are open or closed.

If the eyes are found closed for 4 consecutive frames, it is confirm that the driver is in
drowsiness condition.
 CONTENTS

Acknowledgement i
Abstract ii

1. Introduction 1
1.1 Introduction
1.2 Objectives
1.3 Motivation
1.4 Overview of the Project
1.5 Chapter wise Summary

2. Analysis and Design 6

2.1 Functional Requirements
2.2 Non-Functional Requirements
2.3 Architecture
2.4 Use case diagram
2.5. Sequence Diagram

3. Implementation . 8
3.1. Modules Description
3.2Implementation Details
3.3. Tools used

4. Test results/experiments/verification.
4.1. Testing 17
4.2. Results
4.3. Verification

5. Conclusions and Further Scope 20

References 23

1
 1. Introduction

1.1 Introduction

Drowsiness is a state of near sleep, where the person has a strong desire for sleep. It has two
distinct meanings, referring both to the usual state preceding falling asleep and the chronic
condition referring to being in that state independent of a daily rhythm [16]. Sleepiness can be
dangerous when performing tasks that require constant concentration, such as driving a vehicle.
When a person is sufficiently fatigue while driving, they will experience drowsiness and this
leads to increase the factor of road accident.

Figure 1: Statistic of Road Accident from 2005 to 2009

Figure 1 shows the statistic of road accident. The numbers of vehicles involved in road accident
keep increasing each year. From Figure 1, car and taxi type of vehicles shows about nearly
400,000 cases of road accident has been recorded. It keeps increasing every year and by the year
2009, it shows the number of road accident was recorded by nearly 500,000.

2
 Figure 2 shows the difference between fatigue and drowsiness condition.

Figure 2: Examples of Fatigue & Drowsiness Condition

The development of technologies for detecting or preventing drowsiness while driving is a major
challenge in the field of accident avoidance system. Because of the hazard that drowsiness
presents on the road, methods need to be developed for counteracting its affects.

The aim of this project is to develop a simulation of drowsiness detection system. The focus will
be placed on designing a system that will accurately monitor the open or closed state of the
driver’s eyes and mouth. By monitoring the eyes, it is believed that the symptoms of driver's
drowsiness can be detected in sufficiently early stage, to avoid a car accident. Yawning detection
is a method to assess the driver’s fatigue. When a person is fatigue, they keep yawning to ensure
that there is enough oxygen for the brain consumption before going to drowsiness state .
Detection of fatigue and drowsiness involves a sequence of images of a face, and the observation
of eyes and mouth open or closed duration. Another method to detect eye closure is PERCLOS.
This detection method is based on the time of eyes closed which refers to percentage of a
specific time.

The analysis of face images is a popular research area with applications such as face recognition,
and human identification and tracking for security systems. This project is focused on the
localization of the eyes and mouth, which involves looking at the entire image of the face, and
determining the position of the eyes and mouth, by applying the existing methods in image-
processing algorithm. Once the position of the eyes is located, the system is designed to
determine whether the eyes and mouth are opened or closed, and detect fatigue and drowsiness.

3
1.2 Objectives

The project focuses on these objectives, which are:

• To suggest ways to detect fatigue and drowsiness while driving.
• Study on eyes and mouth from the video images of participants in the experiment of driving
simulation that can be used as an indicator of fatigue and drowsiness.
• To investigate the physical changes of fatigue and drowsiness.
• To develop a system that use eyes closure and yawning as a way to detect fatigue and drowsiness.

1.3 Motivation

Driver drowsiness is a significant factor in the increasing number of accidents on today’s roads and has been
extensively accepted. This proof has been verified by many researchers that have demonstrated ties between
driver drowsiness and road accidents. Although it is hard to decide the exact number of accidents due to
drowsiness, it is much likely to be underestimated. The above statement shows the significance of a research
with the objective of reducing the dangers of accidents anticipated to drowsiness. so far, researchers have tried
to model the behavior by creating links between drowsiness and certain indications related to the vehicle and
to the driver.

Previous approaches to drowsiness detection primarily make pre-assumptions about the relevant behavior,
focusing on blink rate, eye closure, and yawning. The automobile business also has tried to build several
systems to predict driver drowsiness but there are only a few commercial products available today. The
systems do not look at driver performance and overlook driver ability and characteristics. Naturally, most
people would agree that different people drive differently. The system that being develop able to adapt to the
changes of the driver’s behavior.

1.4 Overview of the Project

Drowsiness detection is a safety technology that can prevent accidents that are caused by drivers who fell
asleep while driving.

The objective of this intermediate Python project is to build a drowsiness detection system that will detect that
a person’s eyes are closed for a few seconds. This system will alert the driver when drowsiness is detected.

1.5 Chapter wise Summary

During long journeys, it’s possible that the driver may lose attention because of drowsiness, which may be a
potential reason for fatal accidents. With technologies like Driver Drowsiness Detection getting it is possible
to detect driver’s driving behavior that may prove fatal to the vehicle as well as the people boarding it.

4
Having such sleep detection system in cars embedded in vehicles could protect precious lives and property
worth billion dollars. The outcome would be positive – it would be suitable for fleet owners as well as
individual vehicle users. In either case, the objective is identical by sleep detection while driving. The driver
drowsiness detection system uses Image Processing to analyze the driver’s eye blink pattern by sitting on the
vehicle’s dashboard
If the eye lid movements are abnormal than usual then the detection system triggers the alarm thus alerting the
driver about the condition.

2. Analysis and Design

2.1 Functional Requirements

Anaconda Software:
Anaconda is a free and open-source distribution of the Python and R programming languages for
scientific computing (data science, machine learning applications, large-scale data processing,
predictive analytics, etc.), that aims to simplify package management and deployment.

Python:
We have used pythons 3.6 version which support opencv and dlib packages for face recognition. Python is an
interpreter, general purpose programming language. It is easy to understand.

OpenCV :
It is a library of many programming functions which is mainly used for real time computer vision program.
Dlib:
It is used for creating more complex software and used to solve many real time based problems.
Webcam:
To detect face and eye movements.
Play sound:
If eyes are closed for more interval of time with the help of play sound application it will be able to play
the alarm.

2.2 Non-Functional Requirement

Laptop, Camera, Monitor, Alarm, Mouse, Keyboard

5
2.3 Architecture

Fig 1: architecture

2.4 Use case diagram

Fig 2: use case diagram

6
2.5. Sequence Diagram

Fig 3 :sequence diagram

7
 3. Implementation
.

3.1. Modules Description

A camera is setup that looks for faces in the input video stream and monitors frames of faces. In
the event that a face is identified, facial milestone identification is connected and the eye district is
removed from the earge of video stream

Fig4. Practical design of the system

Fig5.Eye Aspect Ratio Calculation

8
If the aspect ratio of the eye indicates that they have been closed for greater than a dedicated fixed time, we
will sound an alarm system so that the driver wakes up.

Picture 6: Eye marks when the eye is open

Picture 7: Eye marks when the eye is closed
Eye Aspect Ratio plotted over time, the downfall in the aspect ratio graph shows a blink of the driver

We observe the aspect ratio of the eye remains constant for a period of time indicating that the eye
was open, then it falls rapidly to zero and then increases again which indicates the person blinked

We observe the aspect ratio of the eye remains constant for a period of time indicating that the eye
was open, then it falls rapidly to zero and then increases again which indicates the person blinked

9
We will be observing this eye aspect ratio in our drowsiness detector case to see if the value
remains constant or falls to zero but not increases again implying that the driver has closed his
eyes for extended period

Developing Image Processing solutions using OpenCV & dlib

OpenCV was developed keeping image processing in mind. Every function and data struct of
OpenCV concerns itself with an Image Processing library. Comparatively, Matlab is hugely of
generic use & slow.

Any usefulness can be accomplished by methods for tool kits in OpenCV, it might be money
related tool compartments or explicit DNA tool stash

Fig8. Facial landmarks by opencv

10
Also the dlib library comes with a oriented gradients based face detector histogram a facial
landmark predictor comes bundled in the library.

Facial landmarks generated by dlib is an indexable list as described in below image

Fig9:Facial landmarks set which is detected via dlib

11
3.2. Implementation Details

Fig 10: flow chart

12
Algorithmic steps:-

The overall algorithm is pretty straightforward one. First we have used a camera which is
setup at desirable position in a car that looks for faces stream.

If face gets detected, the facial landmark detection task is applied and region of eyes is
extracted.

Once we get the eye region, we calculate the Eye Aspect Ratio to find out if the eye-lids
are down for a substantial amount of time.

On the off chance that the Eye Aspect Ratio demonstrates that the eyes are shut for a
considerably long measure of time, the alert will sound noisy to wake the driver up.For
the functionalities of the system and to make it work efficiently we have used OpenCv,
dlib and Python.

The implementation of the drowsiness detector system includes machine learning

algorithms which are in turn included in OpenCv ML algorithms. There are numerous
ML algorithms but for our purpose we required only the face detector algorithm.

It is fundamentally an item discovery ground-breaking application. Additionally,

prepared frontal face identifier is accessible with the OpenCv circulation.

It works efficiently well overall. It can also be used to detect various different types of
objects with the required software.

Starting to build the detector system with OpenCv

13
First, we create a new file drowsy_detect.py and write the following script in it : -

Now for calculating the eye-aspect ratio we need to compute the Euclidean distance between the
facial landmarks points which in turn needs SciPy package in python. (It not a strict requirement
but SciPy is needed if work related to computer vision or image processing is intended).

Also the package named imutils is needed for image processing and computer vision functions to
assist the working with OpenCv.

The thread class is imported so that we can beep the alarm in a different thread from main thread
so that it is ensured that our script doesn’t stop/pause executing while the alarm beeps. In order to
play a file of the wav or mp3 format, we need playsound library.

Facial landmarks and eye aspect ratio calculation

For detecting and localising facial landmarks we will require the dlib library hence we import it.

Eye_aspect_ratio function is defined to calculate the distance between the eye landmarks taken
vertically and distances between the eye landmarks taken horizontally.

14
So, when the eye is open, the value returned for the eye aspect ratio will be a constant
approximately. Then the value will rapidly decrease reaching zero in case of an eye blink.

When the eye is closed, eye aspect ratio again approaches to an approximate constant value which
is very smaller compared to that when the eye is open.

Therefore, the dip in the aspect ratio indicates blink of the eyes.

Fig11.Plot of eye aspect ratio over time

15
Important variables in the script

EYE_AR_THRESH: it’s a threshold value for the eye aspect ratio. If ratio becomes lower than this
value, counter starts for the number of frames the eyes remain closed.

EYE_AR_CONSEC_FRAMES: If the value for the number of frames for which the eyes remain
closed exceeds this variable’s value , the alarm is activated.

dlib library for face detection

the dlib library serves us with a facial landmark detector as well as facial landmark
predictor.Below is the facial landmarks that are produced by the library.

Now from these landmarks, we just churn out the eye regions successfully.

3.3. Tools used

OpenCV

OpenCV is an open source computer vision library accessible in python coding language to code
for visionary capabilities of our smart pc.

16
OpenCV was expected for computational capability and having a high focus on ongoing picture
location and distinguishing proof. OpenCV is coded with streamlined C and can take work with
multicore processors. If we need progressively programmed improvement utilizing Intel models
[Intel], you can purchase Intel's Integrated Performance Primitives (IPP) libraries [IPP]. These
comprise of low-level schedules in different algorithmic regions which are streamlined. OpenCV
consequently utilizes the IPP library, at runtime if that library is introduced.

4. Test results/experiments/verification.

4.1. Testing

Following is the table representing four test cases that are to encountered while doing this project
that concerns with the drowsiness of the driver.

Test cases Eyes Detected Eye closure Result

Case1 NO NO No result

Case2 NO NO No result

Case3 YES NO No alarm

Case4 YES YES Alarm beeps

Table1 . Test Cases

17
At the point when the eyes are shut for more than certain measure of edges then we find that the
driver is feeling tired. Henceforth these cases are distinguished is and a caution sounded.

4.2. Results

Table and Analysis: -

Numerous examples with shifting exactnesses were assembled and consequently a table plotted for
them

Every individual who volunteered for the test will be approached to squint multiple times and act
languid multiple times amid the test procedure. The eye squinting exactness was determined by
beneath referenced recipe

I/p Eyes Drowsiness Accuracy

Detection Accuracy

Sample 1 100% 87.5 %

Sample 2 95% 100%

Sample 3 80% 62.5%

Sample 4 100% 87.5%

Sample 5 100% 100%

TOTAL 95% 87.5%

18
Every individual who volunteered for the test will be approached to squint multiple
times and act languid multiple times amid the test procedure. The eye squinting
exactness was determined by beneath referenced recipe

Eye Detection Accuracy = total number of times eyes detected / (total no. of eyes
detected+ total no of times eyes not detected)

Drowsiness Detection Accuracy = total no. of times alarm sounds / (total no.
of times alarm sounds + total no of times alarm didn’t sound)

Face or eyes sometimes might not be detected due to lack of ample ambient light. It will
in general be seen from the above table that in case model 3 isn't mulled over, at that
point the framework has an accuracy of about 100%. That said; the high proportion of
disappointments in test 3 exhibits that the framework is slanted to botch and has certain
obstacles. In test 3 we didn't utilize ample backdrop lights for the webcam. The
subsequent poor lighting conditions gave a very error prone result.

Limitations: -

Dependency on proper ambient light:

With poor lighting conditions once in a while the framework is unfit to perceive the eyes. So it gives
a wrong result which must be managed. Continuously
Circumstance infrared setting enlightenments should be used to repel from poor lighting
conditions.

An optimum range is required: -

Exactly when the division among face and webcam isn't at perfect range then certain
issues develop. Exactly when face is unreasonably close webcam (less than 25 cm), then
the framework is unfit to perceive the face from the image. Right when face is a long
way from the webcam (more than 80cm) by then the setting light is missing to edify the
face fittingly. So eyes are not related to high precision which results in botch in
recognizable proof of sluggishness. This issue isn't truly considered as progressively
circumstance, the partition between driver's face and webcam is perfect so the issue
never develops.

19
Orientation of face: -

At the point when the face is tilted to a specific degree it will in general be perceived,
anyway past this the framework can't identify the face. So when the face isn't recognized,
eyes are also not distinguished.

Problem with multiple faces: -

In case more than one face is recognized by the webcam, at that point framework
gives an incorrect result. This issue isn't huge as we have to recognize the tiredness
of a solitary driver

Poor detection of a person’s eyes with spectacles: -

At the point when the driver wears glasses the system may not detect eyes which is the most
noteworthy disadvantage of these systems. This issue has not yet been settled and is a test for
practically all eye detection systems structured up until now.

4.3. Verification

Figure 12: Drowsiness Detected in Graph

The peak
value in the
graph show
the driver is v The driver
in drowsiness is drowsy
state. because
the driver
closed his
eyes for
more than
4 frames
per
second.

20
 In the image above, the drowsiness state occur from frame number 168 to 179, where
the driver closed it eyes for 11 seconds. The result of the driver yawns just after the
drowsiness state occurs and it is shown in the graph in Figure 31.

Driver yawns
after drowsiness
state occurs.

Figure 13: Driver Yawning Detected

From the experimentation and modeling, this project successfully detects each eyes and
mouth for drowsiness detection system to meet the objectives of mini Project. This
system is reliable because it detects the transition of the drowsiness state where the
algorithms detects each eyes and mouth changes each frame per second. Therefore, the
Drowsiness Detection System by using Webcam has successfully done. All the data and
result from the simulation algorithm has been shown this chapter.

5. Conclusions

Thus we would have successfully designed and developed partial

implementation of the Driver Drowsiness Detector using Python and OpenCv
along with the cam to detect the face.

The system to be developed is to be tested and limitations are identified. T he

rest of the work will be done according to what is planned already.
21
 Future Scope

This framework can be stretched out further to have abundant security highlights, for
example, just a certain no of individuals can have specialist get to or work the vehicle.

If there should be an occurrence of an endeavor to robbery, the vehicle's motor don't

begin or an alarm sounds.

A picture of the burglar is taken in an attempted theft &sent to the owner of the vehicle who can
register a case against the thief of the vehicle.

References

[1]. Ameratunga.S , Bailey.J, Connor.J, Civil.I, Dunn.R , Jackson.R , Norton.R, and

Robinson.E, ―Driver sleepiness and risk of serious injury to car
occupants: Population control study.‖British Medical Journal, vol. 324, 2002, pp. 1125–
1129.

[2]. Bronte.S, Bergasa.L, Delgado.B, Garcia.I, Hernandez.N and Sevillano.M,

―Vision- based drowsiness detector for a realistic driving simulator,‖ in IEEE
Intelligent Transportations Systems Conference (ITSC), 2010.

[3]. Distante.A, D’Orazio.T, Guaragnella.C and Leo.M, ―A visual approach

for driver inattention detection,‖Pattern Recogn., vol. 40, no. 8, 2007, pp.
2341–2355.

[4]. Bradski.

[5]. Igarashi.K ,Itou.K, Itakura.F, Miyajima.C, Ozawa.T, Takeda.K

andWakita.T, ―Driver identification using driving behavior
signals,‖IEICE -Trans. Inf. Syst., vol. E89- D, 2006.[6]. Nakano.T,
Suzuki.M, Yamamoto.N, Yamamoto.O and Yamamoto.S,―Measurement of
driver’s consciousness by image processing a method for presuming driver’s
drowsiness by eye- blinks coping with individualdifferences.‖Systems, Man
and Cybernetics, vol. 4, 2006

22