Volume 5, Issue 5, May – 2020 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Drowsy State Detection & Warning Using EAR and

Shape Predictor_68_Facial Landmark
{ Aneesha Shaly George, Christy Shaji, Harikrishnan A, Jose Paul } B.Tech Students
Ashly Thomas Assistant Professor
Department of Computer Science and Engineering
St Joseph’s College of Engineering and Technology Palai, Kerala

Abstract:- Drowsy driving is a major problem in our arrive on time, thus leading to more deaths caused due to
society. Because of this road accidents are increasing at road accidents.
a tremendous rate and it leads to the loss of human life
and many other problems. So that the drowsiness The Driver drowsiness detection techniques can
detection system is the perfect solution for this problem. minimize the risk of accidents by alerting the driver on
In this system, we consider the parameters like the face fatigue conditions. To prevent drowsiness of the driver
detection, position of the head and most importantly the while driving, it requires a method that constantly detects
blinking of the eye. Image processing algorithms are whether there is a fall in the driver alertness. Microsleeps are
used to ensure proper detection of drowsiness in order short periods of sleep that last for 2 to 4 seconds and these
to avoid road accidents. In the proposed method, the are the best sign for the drowsiness condition. By
USB camera that captures drivers face and eye. After continuously monitoring the eye and mouth movement of
processing these images, it will be sent to the drowsiness the driver’s drowsy state can be easily detected.
detection system. When the drowsiness is detected, the
system gives awareness to the driver through an alarm This project uses EAR and Shape predictor_68_facial
and also an alert message is sent to the concerned landmark for the drowsiness detection, SMS alert is done
person’s mobile. Haar Cascade Classifiers are used to by Nexmo API and the location of the driver is located by
detect the eye blink duration of the driver and then the using a GPS module.
Eye Aspect Ratio (EAR) value is calculated. The
drowsiness detection is based on this calculation. II. EXISTING SOLUTION

Keywords:- Image Processing Algorithms, EAR, Haar There are several techniques for analysing driver’s
Cascade Classifier. drowsiness. These techniques include the image
processing-base techniques[1], the electroencephalogram
I. INTRODUCTION based techniques [2] and artificial neural network-based
techniques[3],[4]. In the image processing-based techniques
The main objective of our project is to reduce the it is divided into three categories such as template
accident ratio occurring due to drowsy driving. Major matching[7], eye blinking and Yawning based
accidents occur due to the driver’s fault which leads to technique.[5],[6]. These techniques are based on computer
several other problems, hence to reduce the chance of vision which uses the image processing technique. In the
falling asleep we are making this project. Nowadays, the computer vision technique, the facial gestures such as the
traffic is increasing at a tremendous rate. Because of the eye blinking, the head movement of the driever are used to
heavy traffic emergency vehicles like ambulances can’t identify drivers' drowsy state.

Fig 1

IJISRT20MAY158 www.ijisrt.com 172

Volume 5, Issue 5, May – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
III. PROPOSED METHODOLOGY  Buzzer:
Buzzer is known as an audio signalling device which
Drowsy driving warning system will be implemented produces sound. The applications of the buzzer include
on a computer system using a webcam with the help of alarm on devices, timers, and user input confirmation such
image processing algorithms like facial landmark detection as click on the mouse etc.
algorithm which is used to ensure the proper detection of
drowsy state and to prevent road accidents. Driver’s face  GPS:
fatigues such as eye blinking, mouth position will regularly The Global Positioning System (GPS) is used as a
monitor by USB camera. If the driver face expressions radio navigation system which is based on satellites. The
match with drowsy parameters then the system will alert Global Navigation Satellite Systems (GNSS) provides the
the driver. For a certain interval of time, if drowsiness is geolocation and the time information to a GPS receiver
detected for several number of times then the system will anywhere on the earth. The GPS does not require the user to
send an SMS to the nearby police station or close relatives. transmit any data and operates independently of any internet
reception, though these technologies can enhance the
This project uses EAR and Shape predictor_68_facial usefulness of the GPS positioning information. The GPS
landmark for the drowsiness detection, SMS alert is done applications includes providing critical positioning
by Nexmo API and the location of the driver is located by capabilities to military, civil, and commercial users across
using a GPS module. the world.

IV. HARDWARE IMPLEMENTATION V. SOFTWARE IMPLEMENTATION

 Raspberry Pi:
The Raspberry Pi is a Linux based programmable  OpenCV Library:
computer that does all the normal operations in PC. An OpenCV (Open Source Computer Vision) is a
Raspberry Pi 4 Model B has a 1.5 GHz 64-bit quad core library of python functions developed to solve the computer
ARM Cortex-A72 processor which is used in this system. vision problems.The OpenCV-Python uses Numpy, which
There is a total of four ports, two USB 2.0 ports and two is a highly optimized library for numerical operations. All
USB 3.0 ports. The Pi 4 is also powered via USB-C port. the OpenCV array structures are transformed to and from
The model comes with a standard 40 pin GPIO header. The the Numpy arrays. So whatever operations performed in
board is operated in such a way that the code starts to Numpy, can be combined with OpenCV. The OpenCV
execute as the power is turned ON. The output is obtained supports many algorithms associated with computer vision
through the audio jack. and machine learning etc.. and it is growing day-by-day. As
of now, the OpenCV is being supported by a huge variety
 USB Camera: of programming languages such as Java,Python,C++ etc
The cameras which use USB ports to transfer data and is out there on different platforms such as iOS, Linux,
such as images are known as USB Cameras. These Windows, OS X, Android etc. Hence, OpenCV- Python is
Cameras help to interface with the computer systems by an appropriate tool for fast prototyping of computer vision
using the same USB technology that is present in almost problems.
every computer. USB cameras are ideal for many imaging
applications due to the accessibility of USB technology in  Haar Cascade Classifier:
computer systems and also provides 480 Mb/s transfer rate Harr feature-based cascade classifiers are an efficient
of USB port 2.0. An increasing selection of USB port 3.0 object detection method[8]. Object detection from other
Cameras has a data transfer rate of 5Gb/s. The proposed images is done by cascade function which is trained from
method will use a camera that captures drivers face and eyes, tons of positive and negative images. That means images of
then processes the image to detect the driver’s fatigue. faces and images without faces respectively.

Initially, the algorithm needs the cascade function to

train the classifier. Then extract features from it. Haar
features are used for this. Each feature is a single value
obtained by subtracting the sum of pixels under the white
rectangle from the sum of pixels under the black rectangle.

IJISRT20MAY158 www.ijisrt.com 173

Volume 5, Issue 5, May – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 2

For each feature calculation, all possible sizes and locations of every kernel are used and we need to obtain the sum of the
pixels under white and black rectangles.

But, most of the features calculated are irrelevant. For instance, consider the image below. The top row shows two good
features. The first feature selected focuses on the property that the region of the eyes is often darker than the region of the nose and
cheeks. The second feature selected depends on the property that the eyes are darker than the bridge of the nose. But the same
windows applied to the cheek or any other place is irrelevant. It is achieved by Adaboost.

Fig 3

Using Adaboost we select the features with the lowest error rate, which means they are the features that most accurately
classify the face and non-face images.

In a picture, most of the image is non-face region. Therefore, the focus will be on the regions where there can be a face. For
this, the concept of Cascade of Classifiers is used.

 Shape Predictor_68_Facial Landmark Detection:

A single annotation consists of the face region, and the labelled points. The face region can be easily obtained by any face
detection algorithm. Instead the points have to be detected by already-available landmark detectors and models (SP68). Training
options are a set of parameters. Shape Predictor can be generated from a set of images, annotations and training options that
defines characteristics of the trained model.

IJISRT20MAY158 www.ijisrt.com 174

Volume 5, Issue 5, May – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 4

 Eye Aspect Ratio (EAR): REFERENCES

The eye blink is a fast closing and reopening of a
person’s eye. Everyone has a different pattern of blinking. [1]. Pour Sadeghiyan, M., Mazloumi, A., Nasl Saraji, G.,
The pattern will differ based on the speed of closing and Baneshi, M. M., Khammar, A., & Ebrahimi, M. H.
opening, then the degree in which the eye’s are kept closed (2018). Using Image Processing in the Proposed
and the time taken to blink the eyes. The eye blink will Drowsiness Detection System Design. Iranian
happen for approximately 100-400 Ms. In this system the journal of public health, 47(9), 1371–1378.
facial landmark detectors are used to isolate the eyes and [2]. Li, Ming-Ai & Zhang, Cheng & Yang, Jinfu. (2010).
eyelid shapes. From the landmarks that are located , the eye An EEG-based method for detecting drowsy driving
aspect ratio (EAR) value is derived, that is used as an state. Proceedings - 2010 7th International Conference
estimate of the eye-opening state. on Fuzzy Systems and Knowledge Discovery, FSKD
2010. 5. 2164-2167. 10.1109/FSKD.2010.5569757.
Each eye is represented by a 6-(x,y)-coordinates. The [3]. Vats, M., & Garg, A. (2012). Security System for
width and the height of these coordinates are related to each Drowsy Driver by using Artificial Neural Network
other. The Eye Aspect Ratio (EAR) value is determined by and Varying Layer Method.
the following equation: [4]. Girado, Javier & Sandin, Daniel & Defanti, Thomas &
Wolf, Laura. (2003). Real-time camera- based face
 Nexmo SMS: detection using a modified LAMSTAR neural
Nexmo is a tool within the Voice and SMS category of network system. Proceedings of SPIE - The
a tech stack. The Nexmo SMS API allows the app International Society for Optical Engineering. 5015.
developers to merge their product with the Nexmo SMS 10.1117/12.477405.
services,thus allowing their users to send and receive [5]. Galarza, Eddie & Egas, Fabricio & Silva, Franklin &
messages from Nexmo directly.The Nexmo will attach your Velasco, Paola & Galarza, Eddie. (2018). Real Time
applications directly to carriers around the world. It Driver Drowsiness Detection Based on Driver’s Face
integrates SMS and Voice messages using one simple API. Image Behavior Using a System of Human Computer
Interaction Implemented in a Smartphone.
VI. CONCLUSION 10.1007/978-3-319- 73450-7_53.
[6]. Kaur, Parminder & kalyane, shubhangi. (2014).
The proposed system helps to reduce traffic accidents. DRIVER DROWSINESS MONITORING BASED
The system will monitor the driver's facial cues and it will ON EYE AND YAWN DETECTION.
alert the driver of fatigue. Also, the system will send an INTERNATIONAL JOURNAL OF
SMS alert to the responsible one. ENGINEERING SCIENCES & MANAGEMENT.
[7]. Pamnani, R., Siddiqui, F.A., DharmikGajara, Gupta,
ACKNOWLEDGEMENT A., & Pandya, K.G. (2015). DRIVER DROWSINESS
DETECTION USING HAAR CLASSIFIER AND
The project is being funded by Kerala State Council TEMPLATE MATCHING.
for Science, Technology and Environment (KSCSTE).

IJISRT20MAY158 www.ijisrt.com 175

Volume 5, Issue 5, May – 2020 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
[8]. P. Viola and M. Jones, "Rapid object detection using
a boosted cascade of simple features,"
Proceedings of the 2001 IEEE Computer Society
Conference on Computer Vision and Pattern
Recognition. CVPR 2001, Kauai, HI, USA, 2001, pp.
I-I.
[9]. Malla, Amol & Davidson, Paul & Bones, P.J. &
Green, Richard & Jones, Richard. (2010). Automated
Video-based Measurement of Eye Closure for
Detecting Behavioral Microsleep. Conference
proceedings : ... Annual International Conference of
the IEEE Engineering in Medicine and Biology
Society. IEEE Engineering in Medicine and Biology
Society. Conference. 2010. 6741-4.
10.1109/IEMBS.2010.5626013.
[10]. Babu, T. & Sanjiv, Ashwin & Naidu, Mukul &
Muthukumaaran, C. & Raghavan, C.. (2019). Sleep
Detection and Alert System for Automobiles.
10.1007/978-981-13-6374-0_14.

IJISRT20MAY158 www.ijisrt.com 176