See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/325100971

Fatigue Detection Using Raspberry Pi 3

Article in International Journal of Engineering & Technology · April 2018

DOI: 10.14419/ijet.v7i2.24.11993

CITATIONS READS

29 12,960

5 authors, including:

Mandi Sushmanth Reedy Ezhilarasie R R

Indian Institute of Information Technology Allahabad SASTRA University
4 PUBLICATIONS 67 CITATIONS 24 PUBLICATIONS 166 CITATIONS

SEE PROFILE SEE PROFILE

Kanimozhi Suguna .. S. Arumugam Umamakeswari

13 PUBLICATIONS 102 CITATIONS

SASTRA University
145 PUBLICATIONS 1,087 CITATIONS
SEE PROFILE
SEE PROFILE

All content following this page was uploaded by Mandi Sushmanth Reedy on 18 October 2018.

The user has requested enhancement of the downloaded file.

 International Journal of Engineering & Technology, 7 (2.24) (2018) 29-32

International Journal of Engineering & Technology

Website: www.sciencepubco.com/index.php/IJET

Research paper

Fatigue Detection Using Raspberry Pi 3

Akalya Chellappa *, Mandi Sushmanth Reddy, R.Ezhilarasie, S.Kanimozhi Suguna, A.Umamakeswari

School of Computing, SASTRA Deemed to be University, Thanjavur-613401.

*
Corresponding Author Email: [email protected]

Abstract

Driver drowsiness is a primary cause of several highway calamities leads to severe physical injuries, loss of money, and loss of human
life. The implementation of driver drowsiness detection in real-time will aid in avoiding major accidents. The system is designed for
four-wheelers wherein the driver’s fatigue or drowsiness is detected and alerts the person. The proposed method will use 5-megapixel
Raspbian camera that captures driver’s face and eyes and processes the images to detect driver’s fatigue. On the detection of drowsiness,
the programmed system cautions the driver through an alarm to ensure vigilance. The proposed method constitutes of various stages to
determine wakefulness of the driver. According to this output, the warning message is generated. Haar Cascade Classifiers is used to
detect the blink duration of the driver and Eye Aspect Ratio (EAR) is calculated. Finally, the alert message along with car plate number
is sent to the concerned person mobile with help of Ubidots cloud service and Twilio API. For this Raspberry Pi 3 with Raspbian (Linux
Based) Operating System is used.

Keywords: Driver drowsiness detection, Raspberry Pi 3, Raspbian camera, OpenCV, Feature Extraction, Eye Aspect Ratio (EAR).

1. Introduction and visual cues involve direct computation, whereas driving

performance involves indirect computation [10]. It is suitable for
the real-time application, because of no need for sensing electrodes.
In recent years, driving has become an important part of our day-
Automatic eye detection in video sequence involves several
to-day life, especially in urban areas sleepiness-related accidents
methods like motion estimation in the eye. The eye region is
are occurring infrequent [1]. Road accidents are apparently a global
evaluated by sparse tracking, adaptive thresholding or by frame-to-
hazard in our country. Based on the survey of the National Crime
frame intensity variation and from optical flow [16].
Records Bureau (NCRB) about 1, 35, 000 traffic-related demise
The driver fatigue is detected using computer vision [6]. The driver
happen every year in India. These factors lead to the development
fatigue is determined by the frequency of eye blinking and head
of Intelligent Transportation System (ITS). If the accident caused
tilting and the drowsiness level is calculated by Raspberry Pi3 along
by abnormalities of the driver, it can be prevented by placing
with a Pi camera [8]. The experiment is carried out on ten
abnormality detecting system within the vehicle [2].
volunteers under different lighting conditions and they obtain face
Tiredness can be caused by several phenomena like psychosocial
and eye detection rate up to 99.59% [7]. Real-time video-based
factors, health factors, and physiological factors [3]. Based on the
vision processing method is recommended to estimate fatigue of the
survey of National Highway Traffic Safety Administration of the
driver and a buzzer is used to alert the driver [11]. Relative Areal
United States of America (USA), police studied around 1, 00, 000
Ratio (RAR) is used to identify facial landmarks and to calculate
crashes occurred due to the drowsiness of the driver, it causes major
six landmark points between each eye [12]. Driver fatigue is
losses like 71,000 wounds, $12.5 billion budgetary mishap and
monitored by serially interfacing Raspberry Pi board loaded with
1,550 passings [4]. The issues can be overcome by implementing
raspbian OS to Arduino Uno, and 8-megapixels digital USB camera
several methods which include Support Vector Machines (SVM),
[13].
fuzzy-based system, neuro-fuzzy approach and Artificial Neural
Networks (ANN) [5] for detecting the drowsiness. It is difficult to
correctly say about an exact number of sleep-related accidents, but 2. Methodology
traffic research shows up to 20% of accidents happen due to fatigue
of the driver [6]. Drowsiness detection can be carried out by two The main aim is to detect drowsiness of driver, it can be done in
techniques. The first technique is intrusive and second is non- different ways like detecting facial expression of the driver and
intrusive. The intrusive technique involves computation of mind measuring Eye Aspect Ratio (EAR). Blinking pattern is different
wave monitoring, heart-beat rate etc. for each and every individual. The pattern gets varied in terms of
Non-invasive techniques are appropriate to find facial appearance squeezing degree of eye, blink duration and speed of closing and
for tiredness detection. Mouth gaping and Eye closure are the well- opening the eye [16]. The proposed method involved with the
known symptoms of the drowsiness detection [9]. The non- following methodologies such as Haar Cascade Classifiers, Shape
intrusive technique involves head pose, eye blinking rate, yawn Predictor_68_facial landmark detection, Eye Aspect Ratio (EAR),
detection, eye closure, etc., [8]. Another non-invasive way to detect Ubidots cloud service and Twilio API.
fatigue can be divided into three scenarios: visual cues,
physiological measurements, driving performance. Physiological

Copyright © 2018 Authors. This is an open access article distributed under the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
 30 International Journal of Engineering & Technology

2.1 Haar Cascade Classifiers

║𝒑𝟐−𝒑𝟔║+║𝒑𝟑−𝒑𝟓║
In Haar Cascade Classifiers, a lot of similar and dissimilar images EAR = (1)
are trained in order to detect fatigue of the driver. OpenCV is a
𝟐║𝒑𝟏−𝒑𝟒║
learning-based method, packed with a detector as well as a trainer.
Where p1,…,p6 are the two-dimensional landmark location,
For training, a separate database is maintained for face and eye with
represented in Fig.2. The EAR is mostly stable when an eye is open
several positive and negative images having eye closed and opened
and is getting close to zero while the eye is not in open state. If the
conditions and different set facial images [15]. In 2013, Patil et al
person viewing the camera continuously, the Eye Aspect Ratio
suggested a vision-based drowsiness is carried out using Support
(EAR) is found to be normal and it reaches low value when he/she
Vector Machine and Haar Cascade Classifiers [14].
closing the eye for a longer time. When the lower value is reached,
then drowsiness is detected. In 2012, Ubidots elaborated on
2.2 Shape predictor_68_Facial Landmark detection and
connected software and hardware solutions to remotely control,
Eye Aspect Ratio (EAR) automate processes for healthcare clients and monitor. Twilio is a
cloud communication platform as a service (PaaS), it allows a
In order to predict the face and eye region in the live video stream, software developer to programmatically send and receive text
shape predictor is used. Fig.1 shows the sleepiness which is message using its web service APIs, make and receive phone calls.
measured by calculating the eye aspect ratio (Euclidean distance The paper introduces alerting process when the driver feels sleepy
between the eyes are calculated), the arguments are passed to the using cloud server and mobile API to send a message and at the
predefined dataset and facial landmark detection is carried out. For same time providing an alarm signal to the driver.
every video sequence, the eye landmarks are located. The aspect
ratio between width and height of the eye is calibrated.
3. Programming Algorithm
Raspberry Pi 3 is effective with Python IDLE. It involves
programming in software with extensions of OpenCV computer
vision installed. The entire algorithm for drowsiness detection is
shown with the help of a flowchart shown in Fig 2. The program
will start to run and it can be terminated using command line
interfaces in Raspberry Pi 3 or the system is turned off. In order to
initiate the program execution, it will import the following libraries
like numpy, OpenCV, play sound, argparse, dlib, distance, timer,
client, ApiClient, and picamera

Fig.1: Close and open eyes with landmark pi detected automatically by [1].
The EAR is calculated for several frames of a video. A single blink is
represented.

Fig. 2: Drowsiness Detection algorithm

 International Journal of Engineering & Technology 31

4. Result and Discussion closing rate is measured after every 0.5 seconds and if the value
crosses already existed threshold value, then the raspberry pi 3
receives the alert signal from alarm connected to the GPIO pins of
The driver drowsiness can be measured using Eye Aspect
Pi 3 board. Fig 5.When the person closing his eyes for more than
Ratio(EAR). The ratio of the eye can vary for each and every
fixed threshold range then the alert signal is generated to wake up
person. Fig.4. The following case is tested for ten different set of
the driver from sleepy state and also through the cloud service the
people with two conditions. One is calculated for eye-opening
alert message is sent to the owner of the car along with the car plate
condition and another one for eye closing condition. Fig.3. Eye
number.

Fig. 3: Drowsiness detection for open and closed eyes with drowsiness count and message send to mobile after 10 continuous blinks from the cloud server.

Drowsiness Detection
Person

EAR(open)
EAR(close)

Eye Aspect Ratio (EAR)

Fig. 4: The result of fatigue detection for ten different samples

Fig. 5: Output message with car plate number in the cloud server

5. Conclusion will help to calculate drowsiness level. Frequent detection of eye

blinking and head tilting is measured properly and it helps to
indicates drowsiness. When he/she reaches maximum threshold the
Driver drowsiness detection is designed mainly to keep the driver
driver will be alarmed by a loud warning that will wake up the
awake while driving to avoid the accident due to sleepiness. The
driver from the sleep state. In future, the implementation can be
alert signal is generated from embedded device to awake driver
carried out in a bright room with consistent light, for different
from sleepy state. The Pi along with Raspbian camera is used to
lighting conditions and also for the persons with dark skin can be
calculate the drowsiness of the driver in real time. Fatigue is
considered.
measured by detecting Eye and face using Haar Cascade Classifier,
especially facial landmarks is detected using shape-predictor and
Eye Aspect Ratio (EAR) by calculating the Euclidean distance
between the eyes. Accurate eye detection and faces in every frame
 32 International Journal of Engineering & Technology

References Network”, CiiT International Journal of Programmable Device

Circuits and Systems,Print: ISSN 0974 – 973X & Online: ISSN
0974 – 9624, Issue : November 2011, PDCS112011008.
[1] L. Jia, D. Zhao, K. Zheng, Z. Li, G. Sun, and F. Zhang, [27] T. Padmapriya, V.Saminadan, “Performance Improvement in long
“Smartphone-based fatigue detection system using the progressive term Evolution-advanced network using multiple imput multiple
locating method”, IET Intell. Transp. Syst, vol. 10, no. 3, pp. 148– output technique”, Journal of Advanced Research in Dynamical
156, 2016. and Control Systems, Vol. 9, Sp-6, pp: 990-1010, 2017.
[2] J. S. Jayasenan and P. S. Smitha, “Driver Drowsiness Detection
System”, vol. 4, no. 1, pp. 34–37, 2014.
[3] V. Triyanti and H. Iridiastadi, “Challenges in detecting drowsiness
based on driver’s behavior”, IOP Conf. Ser. Mater. Sci. Eng, vol.
277, 2017.
[4] F. Omidi and G. N. Saraji, “Non-intrusive Methods used to
Determine the Driver Drowsiness : Narrative Review Articles”, pp.
186–191, 2016.
[5] P. P. Bhatt GHPatel PG and J. A. Trivedi Patel PG, “Various
Methods for Driver Drowsiness Detection : An Overview”, Int. J.
Comput. Sci. Eng, vol. 9, no. 3, pp. 70–74, 2017.
[6] D. Sarkar and A. Chowdhury, “A Real-Time Embedded System
Application for Driver Drowsiness and Alcoholic Intoxication
Detection”, Int. J. Eng. Trends Technol, vol. 10, no. 9, pp. 461–465,
2014.
[7] R. Ahmed, Kazi Emrul Kayes Emon, and M. F. Hossain, “Robust
driver fatigue recognition using image processing”, Int. Conf.
Informatics, Electron. Vis, pp. 1–6, 2014.
[8] O. Khunpisuth, T. Chotchinasri, V. Koschakosai, and N. Hnoohom,
“Driver Drowsiness Detection Using Eye-Closeness Detection”,
12th Int. Conf. Signal-Image Technol. Internet-Based Syst, pp.
661–668, 2016.
[9] L. F. Ibrahim et al., “Using Haar classifiers to detect driver fatigue
and provide alerts”, Multimedia. Tools Appl, vol. 71, no. 3, pp.
1857–1877, 2014.
[10] T. Soukupová and J. Cech, “Real-Time Eye Blink Detection using
Facial Landmarks”, 21st Comput. Vis. Winter Work, 2016.
[11] N. G. Prajapati, “Driver Drowsiness Detection with Audio-Visual
Warning”, vol. 3, no. 1, pp. 294–300, 2016.
[12] A. Suganya and A. Robertson, “On-Road Drowsiness Alarm of
Drivers using Raspberry Pi”, Int. J. Recent Trends Eng. Res, vol. 3,
no. 11, pp. 199–204, 2017.
[13] S. Indexed and R. Agrawal, “A LOW-COST DESIGN TO
DETECT”, vol. 8, no. 9, pp. 1138–1149, 2017.
[14] B. Sivakumar and K. Srilatha, “A novel method to segment blood
vessels and optic disc in the fundus retinal images”, Res. J. Pharm.
Biol. Chem. Sci, vol. 7, no. 3, pp. 365–373, 2016.
[15] N. L. Fitriyani, C. K. Yang, and M. Syafrudin, “Real-time eye state
detection system using haar cascade classifier and circular Hough
transform”, IEEE 5th Glob. Conf. Consum. Electron. GCCE, pp.
5–7, 2016.
[16] [G. J. AL-Anizy, M. J. Nordin, and M. M. Razooq, “Automatic
Driver Drowsiness Detection Using Haar Algorithm and Support
Vector Machine Techniques”, Asian Journal of Applied , vol. 8, no.
2. pp. 149–157, 2015.
[17] T. Azim, M. A. Jaffar, and A. M. Mirza, “Fully automated real-time
fatigue detection of drivers through Fuzzy Expert Systems”, Appl.
Soft Comput. J, vol. 18, pp. 25–38, 2014.
[18] A. Manuscript, “NIH Public Access”, Traffic, vol. 11, no. 2, pp.
126–136, 2010.
[19] P. Thiffault and J. Bergeron, “Monotony of road environment and
driver fatigue: A simulator study”, Accid. Anal. Prev, vol. 35, no.
3, pp. 381–391, 2003.
[20] L. Li, Y. Chen, and Z. Li, “Yawning detection for monitoring driver
fatigue based on two cameras”, IEEE Conf. Intell. Transp. Syst.
Proceedings, pp. 12–17, 2009.
[21] R. S. Rawal and S. S. Nagtilak, “Drowsiness Detection Using
RASPBERRY-Pi Model Based On Image Processing”, pp. 328–
331, 2016.
[22] S. Salehian and B. Far, “Embedded Real-Time Blink Detection
System for Driver Fatigue Monitoring”,
Ksiresearchorg.Ipage.Com, 2015.
[23] B. Reddy, Y. H. Kim, S. Yun, C. Seo, and J. Jang, “Real-Time
Driver Drowsiness Detection for Embedded System Using Model
Compression of Deep Neural Networks”, IEEE Comput. Soc. Conf.
Comput. Vis. Pattern Recognit. Work, pp. 438–445, 2017.
[24] K. U. Anjali, A. K. Thampi, A. Vijayaraman, M. F. Francis, N. J.
James, and B. K. Rajan, “Real-time nonintrusive monitoring and
detection of eye blinking in view of accident prevention due to
drowsiness”, Proc. IEEE Int. Conf. Circuit, 2016.
[25] M. F. Abulkhair and L. F. Ibrahim, “Using Mobile Platform to
Detect and Alerts Driver Fatigue”, vol. 123, no. 8, pp. 27–35, 2015.
[26] S.V.Manikanthan and K.Baskaran “Low Cost VLSI Design
Implementation of Sorting Network for ACSFD in Wireless Sensor

View publication stats