A

PROJECT REPORT
Based on
DRIVER’S DROWSINESS DETECTION SYSTEM
Submitted
By
ABHISHEK
(ROLL NO. – 6318013, 7th SEM)
Submitted
to
Mr. Tejbir Rana(Asst. prof.)
DEPARTMENT OF COMPUTER SCIENCE &
ENGINEERING

GURU NANAK INSTITUTE OF TECHNOLOGY, MULLANA

AFFILIATED TO
KURUKSHETRA UNIVERSITY, KURUKSHETRA, HARYANA
2019-23

1|Page
 ACKNOWLEDGMENT

I take this opportunity to express my sincere gratitude to the principal OF GNIT MULLANA

for providing this opportunity to carry out the present work.

I am highly grateful to the Dr. Siddharth Arora (HOD CSE, GNIT), for providing this

opportunity to make this project in order to enhance my skills. I would like to expresses my

gratitude to other faculty members of Computer Science & Engineering department of GNIT

for providing academic inputs, guidance & Encouragement throughout the training period.

The help rendered by Mr. Sachin Chawla (Dean) for Experimentation is greatly

acknowledged. Finally, I express my indebtedness to all who have directly or indirectly

contributed to the successful completion of my project.

Student name: Abhishek

2|Page
 TABLE OF CONTENT

SR. NO CONTENT PAGE NO.

1 ABSTRACT 4

2 INTRODUCTION 5

3 OBJECTIVE 5

4 LITERATURE REVIEW 6

5 PROPOSED WORK 6

6 WORK FLOW 7

7 MODEL ARCHITECTURE 8

8 PROJECT 9
PREREQUISITES

9 PICTORIAL 11
REPRESENTATION

3|Page
ABSTRACT

Fatigue has costly effects on the safety, health, and quality of life of the American
public. Whether fatigue is caused by sleep restriction due to a new baby waking every couple
of hours, a late or long shift at work, hanging out late with friends, or a long and monotonous
drive for the holidays – the negative outcomes can be the same. These include impaired
cognition and performance, motor vehicle crashes, workplace accidents, and health
consequences.

Tackling these issues can be difficult when our lifestyle does not align with avoiding drowsy
driving. In a 24/7 society, with an emphasis on work, longer commutes, and exponential
advancement of technology, many people do not get the sleep they need. Effectively dealing
with the drowsy-driving problem requires fundamental changes to societal norms and
especially attitudes about drowsy driving.

How are drivers affected by drowsy driving?

Drowsy driving significantly increases the risk of car accidents. Microsleeps are when a

person dozes off for just a few seconds5, and when they occur while driving, it's easy for the

car to run off the road or collide with another vehicle. The damage from these crashes

increases when they occur at high speeds

4|Page
INTRODUCTION
As we know countless number of people drive on the highway day and night. Taxi
drivers, bus drivers, truck drivers and people traveling long-distance suffer from lack of
sleep. Due to which it becomes very dangerous to drive when feeling sleepy. The majority of
accidents happen due to the drowsiness of the driver.

So, to prevent these accidents we will build a system using Python, OpenCV, and Keras
which will alert the driver when he feels sleepy.

This project is to build a drowsiness detection system that will detect that a person’s eyes are
closed for a few seconds. This system will alert the driver when drowsiness is detected.

OBJECTIVE OF THE PROJECT:

The objective of this project is to build a drowsiness detection system that will detect
that a person’s eyes are closed for a few seconds. This system will alert the driver when
drowsiness is detected.

LITERATURE REVIEW:

Drowsy driving significantly increases the risk of car accidents. Microsleeps are
when a person dozes off for just a few seconds5, and when they occur while driving, it’s easy
for the car to run off the road or collide with another vehicle. The damage from these crashes
increases when they occur at high speeds.
Drowsy driving is dangerous even if a person doesn’t actually fall asleep. Research shows
that sleep deprivation leads to mental impairment that is similar to drunkenness6 with 24
hours of sleep deprivation roughly equating to a blood alcohol content (BAC) of 0.10%.
This impairment makes a person less attentive to their surroundings and more easily
distracted. It slows their reaction time, making it harder to avoid dangers in the roadway.
Insufficient sleep is also tied to worsened decision-making, which can lead to risk-taking
behind the wheel.

5|Page
PROPOSED WORK / METHODOLOGY

Let’s now understand how our algorithm works step by step.

Step 1 – Take Image as Input from a Camera

With a webcam, we will take images as input. So to access the webcam, we made an infinite
loop that will capture each frame. We use the method provided by
OpenCV, cv2.VideoCapture(0) to access the camera and set the capture object
(cap). cap.read() will read each frame and we store the image in a frame variable.
Step 2 – Detect Face in the Image and Create a Region of Interest (ROI)
To detect the face in the image, we need to first convert the image into grayscale as the
OpenCV algorithm for object detection takes gray images in the input. We don’t need color
information to detect the objects. We will be using haar cascade classifier to detect faces.
This line is used to set our classifier face = cv2.CascadeClassifier(‘ path to our haar
cascade xml file’). Then we perform the detection using faces =
face.detectMultiScale(gray). It returns an array of detections with x,y coordinates, and
height, the width of the boundary box of the object. Now we can iterate over the faces and
draw boundary boxes for each face.
for (x,y,w,h) in faces:

cv2.rectangle(frame, (x,y), (x+w, y+h), (100,100,100), 1 )

Step 3 – Detect the eyes from ROI and feed it to the classifier
The same procedure to detect faces is used to detect eyes. First, we set the cascade classifier
for eyes in leye and reye respectively then detect the eyes using left_eye =
leye.detectMultiScale(gray). Now we need to extract only the eyes data from the full image.
This can be achieved by extracting the boundary box of the eye and then we can pull out the
eye image from the frame with this code.
l_eye = frame[ y : y+h, x : x+w ]

l_eye only contains the image data of the eye. This will be fed into our CNN classifier which
will predict if eyes are open or closed. Similarly, we will be extracting the right eye
into r_eye.
Step 4 – Classifier will Categorize whether Eyes are Open or Closed

6|Page
We are using CNN classifier for predicting the eye status. To feed our image into the model,
we need to perform certain operations because the model needs the correct dimensions to
start with. First, we convert the color image into grayscale using r_eye =
cv2.cvtColor(r_eye, cv2.COLOR_BGR2GRAY). Then, we resize the image to 24*24
pixels as our model was trained on 24*24 pixel images cv2.resize(r_eye, (24,24)). We
normalize our data for better convergence r_eye = r_eye/255 (All values will be between 0-
1). Expand the dimensions to feed into our classifier. We loaded our model using model =
load_model(‘models/cnnCat2.h5’) . Now we predict each eye with our model
lpred = model.predict_classes(l_eye). If the value of lpred[0] = 1, it states that eyes are
open, if value of lpred[0] = 0 then, it states that eyes are closed.
Step 5 – Calculate Score to Check whether Person is Drowsy
The score is basically a value we will use to determine how long the person has closed his
eyes. So if both eyes are closed, we will keep on increasing score and when eyes are open, we
decrease the score. We are drawing the result on the screen using cv2.putText() function
which will display real time status of the person.

cv2.putText(frame, “Open”, (10, height-20), font, 1, (255,255,255), 1, cv2.LINE_AA )

A threshold is defined for example if score becomes greater than 15 that means the person’s
eyes are closed for a long period of time. This is when we beep the alarm using sound.play().

WORK FLOW AND STEPS TO BE PERFORMED

In this Python project, we will be using OpenCV for gathering the images from
webcam and feed them into a Deep Learning model which will classify whether the person’s
eyes are ‘Open’ or ‘Closed’. The approach we will be using for this Python project is as
follows:

Detect the Classifier

Detect the Calculate
face in the will
Take image eyes from score to
image and categorize
as input ROI and check
create a whether
from a feed it to whether
Region of eyes are
camera. the the
7 |person
Interest open or Page
classifier. is drowsy.
(ROI). closed.
Step 1 – Take image as input from a camera.
Step 2 – Detect the face in the image and create a Region of Interest (ROI).
Step 3 – Detect the eyes from ROI and feed it to the classifier.
Step 4 – Classifier will categorize whether eyes are open or closed.
Step 5 – Calculate score to check whether the person is drowsy.

DRIVER DROWSINESS DETECTION DATASET

The dataset used for this model is created by us. To create the dataset, we wrote a
script that captures eyes from a camera and stores in our local disk. We separated them into
their respective labels ‘Open’ or ‘Closed’. The data was manually cleaned by removing the
unwanted images which were not necessary for building the model. The data comprises
around 37000 images of people’s eyes under different lighting conditions. After training the
model on our dataset, we have attached the final weights and model architecture file
“models/cnnCat2.h5”.

Now, you can use this model to classify if a person’s eye is open or closed

THE MODEL ARCHITECTURE

The model we used is built with Keras using Convolutional Neural Networks
(CNN). A convolutional neural network is a special type of deep neural network which
performs extremely well for image classification purposes. A CNN basically consists of an
input layer, an output layer and a hidden layer which can have multiple layers. A convolution
operation is performed on these layers using a filter that performs 2D matrix multiplication
on the layer and filter.
The CNN model architecture consists of the following layers:

 Convolutional layer; 32 nodes, kernel size 3

 Convolutional layer; 32 nodes, kernel size 3
 Convolutional layer; 64 nodes, kernel size 3

8|Page
  Fully connected layer; 128 nodes
The final layer is also a fully connected layer with 2 nodes. A Relu activation function is used
in all the layers except the output layer in which we used Softmax.

PROJECT PREREQUISITES

The requirement for this Python project is a webcam through which we will capture
images. You need to have Python (3.6 version recommended) installed on your system, then
using pip, you can install the necessary packages.

1. OpenCV – OpenCV (Open Source Computer Vision Library) is an open source

computer vision and machine learning software library. OpenCV was built to provide a

common infrastructure for computer vision applications and to accelerate the use of machine

perception in the commercial products. Being a BSD-licensed product, OpenCV makes it

easy for businesses to utilize and modify the code.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of

both classic and state-of-the-art computer vision and machine learning algorithms. These

algorithms can be used to detect and recognize faces, identify objects, classify human actions

in videos, track camera movements, track moving objects, extract 3D models of objects,

produce 3D point clouds from stereo cameras, stitch images together to produce a high

resolution image of an entire scene, find similar images from an image database, remove red

eyes from images taken using flash, follow eye movements, recognize scenery and establish

markers to overlay it with augmented reality, etc. The library is used extensively in

companies, research groups and by governmental bodies.

9|Page
2. TensorFlow – TensorFlow is an end-to-end open source platform for machine learning.

It has a comprehensive, flexible ecosystem of tools, libraries and community resources that

lets researchers push the state-of-the-art in ML and developers easily build and deploy ML

powered applications.

Why to use Tesnor Flow?

 Easy model building

Build and train ML models easily using intuitive high-level APIs like Keras
with eager execution, which makes for immediate model iteration and easy
debugging.

 Robust ML production anywhere

Easily train and deploy models in the cloud, on-prem, in the browser, or on
device no matter what language you use.
 Powerful experimentation for research

A simple and flexible architecture to take new ideas from concept to code, to
state-of-the-art models, and to publication faster.

3. Keras

Keras is a deep learning API written in Python, running on top of the machine
learning platform TensorFlow. It was developed with a focus on enabling fast
experimentation. 

Keras is:

 Simple -- but not simplistic. Keras reduces developer cognitive load to free you to
focus on the parts of the problem that really matter.
 Flexible -- Keras adopts the principle of progressive disclosure of complexity: simple
workflows should be quick and easy, while arbitrarily advanced workflows should
be possible via a clear path that builds upon what you've already learned.

10 | P a g e
  Powerful -- Keras provides industry-strength performance and scalability: it is used
by organizations and companies including NASA, YouTube, or Waymo.

4. Matplotlib
Matplotlib is an amazing visualization library in Python for 2D plots of arrays.
Matplotlib is a multi-platform data visualization library built on NumPy arrays and designed
to work with the broader SciPy stack. It was introduced by John Hunter in the year 2002.

One of the greatest benefits of visualization is that it allows us visual access to huge amounts
of data in easily digestible visuals. Matplotlib consists of several plots like line, bar, scatter,
histogram etc.

PICTORIAL REPRESENTATION

Fig-1

11 | P a g e
 Here the person who is sitting in front of the webcam has his open so we can see in
the bottom-left the application is telling us the eyes of the person is open by using the
word ‘open’.

fig-2

 Now the person has closed his eyes so now the application will sense danger and tell
us the eyes of the user is closed by using word ‘closed’ and Start its counter.

12 | P a g e
 FIG-3

 Here the person has not opened his eyes and counter has reached to certain limit so
here the boundries of the frame will get red and alarm will start playing in order to tell
the user you should take rest to avoid any mishappening.

CONCLUSION
In this Python project, we have built a drowsy driver alert system that you can

implement in numerous ways. We used OpenCV to detect faces and eyes using a cascade

classifier and then we used a CNN model to predict the status. Here we are building the

application that will detect the drowsiness of the person if the drowsiness is detected the

application will alert the person in order to make the journey safe and secure. This application

can be used at large scale by taxi drivers, truck drivers, commuters etc in order to make

themselves safe from any mishappening.

REFERENCES

13 | P a g e
 National Centre for Chronic Disease Prevention and Health Promotion, Division of
Population Health. (2017, May 2). CDC - Data and Statistics - Sleep and Sleep
Disorders. Retrieved January 12, 2021,
fromhttps://www.cdc.gov/sleep/data_statistics.html
 National Center for Chronic Disease Prevention and Health Promotion, Division of
Population Health. (2017, March 21). Drowsy Driving. Retrieved January 12, 2021
fromhttps://www.cdc.gov/sleep/about_sleep/drowsy_driving.html

 National Highway Traffic Safety Administration. (n.d.). Drowsy Driving. Retrieved

January 12, 2021, fromhttps://www.nhtsa.gov/risky-driving/drowsy-driving

14 | P a g e