EYE BEHAVIOUR BASED DROWSINESS DETECTION SYSTEM

JAVED AHMEDt, JIAN-PING LI 1, SAEED AHMED KRAN 2 , RIAZ AHMED SHAIKR 1

I School of Computer Science & Engineering, VESTC, Chengdu 611731, China

2Departmnet ofElectrical Engineering, Sukkur Institute ofBusiness Administration, Sindh Pakistan
E-MAIL: [email protected]@uestc.edu.cn. [email protected].
[email protected]

Abstract: steering wheel , heart rate variability or grip quality.

A non-intrusive computer vision based ideas has been Systems that utilize a camera for the tracking of eye
utilized for the development of a Drowsy Driver Detection movements have already been created [1]. In any case, so
System. The sm all camera has been used by system that far no framework has turned out to be adequately reliable.
focuses straight towards the face of driver and checks the Previous method relies on the LED's and multiple cameras
driverfs eyes with a specific end goal to recognize fatigue. A
to estimates the facial expressions however moving vehicle
warning sign is issued to caution the driver, in such situation
when fatigue is recognized. This paper illustrates the process introduces new difficulties like variable lighting and
of locate the eyes of driver, and to decide whether the eyes of running backgrounds. This paper presents a solution for
driver are open or cIose. The system manages utilizing data minimizing the road accident caused by the drowsiness of
gained for the image which is in binary form to locate the face driver by alerting tbrough a single camera placed on the
edges, wh ich gets the location where the eyes of a person may dash board of vehicle. Smart vehicle vendors have
exist. If the eyes of driver are found cIose for five successive developed this technology by applying different techniques.
frames, the proposed system assurnes that the driver is The algorithm of eye detection system integrated with
nodding off and a signal of warning has been issued. The hardware to develop the smart vehicles, which can
framework is also capable to recognize in such situation when
implement nationwide to avoid the road accidents.
the eyes canft be discovered, and works in sensible lighting
circumstances. The result demonstrates that eye-tracking Microcontroller and camera are used to make and
drowsiness functions admirably for a few drivers the length of intelligent hardware and software integrated system.
the squint acknowledgment works appropriately. The camera The purpose of this study is to build up a model of
based drowsiness measures give an appreciated contribution. Drowsiness Detection System. The system will precisely
check in real time, the open or close condition of the eyes
Keywords: of driver. By checking the driver's eyes, the indications of
Drowsiness Detection: Driver fatigue: eye-tracking: EEG: driver drowsiness can be identified in the beginning to
EOG: protect from vehicle accident.

1. Introduction 1.1 Drowsiness

Driver drowsiness or fatigue is an important element in Drowsiness is astate where a person sleep or alrnost
vehicle accidents. Latest statistics evaluate that every year likely. It alludes to a failure to keep awake or a drive to rest
1550 deaths and 71,000 injuries can be ascribed to [2]. Drowsiness and sleepiness considered in this paper as
accidents due to the driver fatigue. (Drowsiness is astate equivalent words. Although, in this paper sleepiness is
where a man sleep or likely). It alludes to a powerlessness utilized as apart of distinctive term as fatigue, wh ich is a
to keep wakeful or a drive to sleep. The advancement of great tiredness because of physical and mental action.
technology for recognizing or forestalling drowsiness in the Drowsiness can likewise be depicted by the evaluation of
driverfs seat is a noteworthy challenge in the field of alertness or vigilance. Attentiveness is the same as
accident evasion frameworks. It would in this manner be sharpness or a condition of sleep inability, while vigilance
advantageous to figure out how to recognize drowsiness can be depicted as watchfulness or astate where one is
before it happens and to have the capacity to caution the readied for something to happen [3]. The steering response
driver in time. A few systems already had been created, in in frequency domain of an active driver and a drowsy driver
light of recording of head developments, movement of is show in Figure 1. It is hard to give a defmite measure of

978- 1-4673-8266-3115/$31.00 ©20151EEE 268

drowsiness in the way that is possible with for instance 3. Drowsy Drivers Detection Techniques
alcohol. Moreover, drowsiness is a transient state, which
additionally makes the discovery hard. 3.1 Monitoring Physiological Characteristics
Isway by drowsy drivi!l§ I

Physiological measures have much of the time been

utilized for drowsiness discovery as they can give an
immediate and objective measure. Conceivable measures
are EEG, eyelid closure, movements of eye, heart rate, size
of pupil, skin conductance and creation of the cortical [2].
Among these procedures, the systems that are best, in light
Steerlrll( lrequency
of precision are the ones in view of physiological
Fig.l Drowsy Driver Characteristics experience of human [9]. There are two ways for
2. Literature Review implementing this procedure. Measurement of changes in
physiological signs for example, waves of human brain,
blinking of eyes and heart rate; and physical changes
Advance driver assistance system (ADAS) is a general
measurement for example, drooping stance, leaning of the
tenn that is used for automotive intelligent system for
head of driver and the open/close conditions ofthe eyes [9].
assisting drier in two ways. Inductive power link
framework based on power oscillator has been proposed by
Ma [4]. A system based on conventional class-E PA requires 3.2 Electroencephalogram (EEG)
high drive requirement and decrease the efficiency of
overall link. Adaptive Cruise Control and Lane Departure EEG is a technique for measuring the electrical action
Warning. ADAS with the feature of detection of driver created by the nerve cells of the hwnan brain, basically the
drowsiness has been introduced by major automakers cortical movement. The EEG-action is available all the time
nowadays. ADAS system are adopt by many companies and recording show both arbitrary and periodic behavior.
such as Attention Assist (Daimler [5]), Driver Alert (Ford The fundamental inception of the EEG is the neuronal
[6]), Driver Alert Control (Volvo [7]) and Fatigue Detection action in the cerebral cortex; however some action Iikewise
System (Volkswagen [8]) are able to monitor driving starts from the thalamus and from subcritical parts of the
behavior and issue alarm in a visual manner if necessary. human brain. The EEG speaks to the summation of
Attention assistant system was introduced by Daimler excitatory and inhibitory postsynaptic possibilities in the
automotive company. Between the speed of 80 and 180 nerve cells. The musical movement is because of the
km/h the system is active. These systems analyze the synchronous actuation of the nerve cells [2]. In Figure 2
staring action pattern in fIrst few minute and cheek the different sensors are shown.
staring action by comparing with previous pattern [5].
Driver alert system was introduced by Ford
automotive company [6]. The Driver Alert System is
intended to alann drowsy drivers by observing the vehicle's
movement contrasted with path markings that are followed
by a camera mounted on the windshield. On the off chance
that the framework distinguishes a driving pattern
predictable with a drowsy driver, a fIrst-level toll will sound
and an espresso mug warning will show up on the Fig.2 Different EEG techniques for drowsiness detection
dash board instrument bunch to suggest the driver to take a
break. Volkswagen [8] built up the driver Fatigue Detection 3.3 Electrooculography (EOG)
framework consequently investigations the driving qualities
and if they demonstrate possible fatigue, suggests that the Electrooculography is a strategy utilized for measuring
driver take a break. If fatigue is identifIed, the driver is the potential distinction between the front and back of the
cautioned by information in the Multi-function Display and eye ball. The EOG can in this manner be utilized for
an acoustic sign. In [14], to achieve higher PAC for location of eye movements and flickers. The eye is a dipole
biomedical applications a high efficiency differential POSC with the positive cornea in the front and the negative retina
has been used. in the back. While moving the eyes an adjustment in
potential is distinguished as the posts come closer or more

269
remote far from the terminals. In Figure 3(a) the indication response time to any surpnsmg occasions likewise gets
of the change relies on the course of the movement of eyes longer with increased sleepiness [13].
[2]. Vertical recording is typically monocular, which
implies that the recording is made crosswise over one eye, 4. Methodology
while horizontal recording normally is binocular. Figure 3(b)
demonstrates how the electrodes are put. Eye blinks are 4.1 Aigorithm for Eye Detection
distinguished by utilizing vertical recording [2]. a) Take an image.
~V)
30' 200 b) Convert RGB to gray image.
O· 30' o' c) Apply histogram equalization.

~y
d) Remove binary noise from image
e) Select the Region of interest of image, such as
Width, height and pupil.
lime[s) f) Find eyes position from face .
g) Determine whether eyes closed or open.
h) Repeat the process.
A color image is consist ofthree basic color RGB (Red,
Green, Brown) and computer read the image either binary
or gray so, for detecting features from the color image such
as eye, nose, head gray image is required. First the webcam
is used for acquired the image for processing. Then to
search and detect the faces we use the Haarcascade file
face.xml in each individual frame.
(b)
Fig.3 (a) Change in EOG potential when looking 30 0 to the
right. (b) EOG Sensor Heimet

3.4 Non-intrusive image based method

Individuals in exhaustion display certain visual

attitudes effectively recognizable from the changes of their
facial features such as head, eyes, and face . Distinctive
visual features detectable from the image of a man with
lessened sharpness level incorporate moderate eyelid
movement. All these visual prompts, however flawed they
are exclusively, if consolidated systematically, can give an
exact characterization of a visualization level of driver [10,
11]. It can robustly track eyes under different face
orientations, illuminations, head movements, and
open/close eye, yawning. Head orientation is estimated by
pupil shape, pupil position, pupil orientation and pupil size.

3.5 Driving performance measures

(e)
Driving performance measures incorporate movements Fig.4 (a) Normal Color Image (b) Gray Scale Image (c)
of steering wheel, horizontal position, speed variability and Histogram Equalization ofGray Scale Image (d) Eyes
response time. Studies demonstrate that the controlling Detection from Equalized Image (e) Eyes not Detected in
wheel variability increments with the measure of sleepiness Equalized Image.
[12]. The movements of steering additionally get to be
bigger and happen less frequently, and the parallel position Figure 4(a) is representing a color image of the face.
variability increments as the driver get drowsier. The Gray scale image has the greater intensity as shown in
Figure 4(b), after converting color image to grayscale.

270
Histogram equalization of image has done in Figure 4(c). intensity is occur. From the amount of intensity changes the
Histogram equalization is a method of contrast adjustment. program will judge that, either the eyes are close or open. If
Form equalized image it is relative easy to detect eye. In the eyes of driver will remain close for five consecutive
Figure 4(d) detected eyes are encircled. Haarcascade is used image frame then system will warn the driver. If it will
to detect the features from the image In Figure 4(e) no eye remain less than five consecutive frames then system will
is detected form Equalize image because eyes are close. not warn for but show in display that you are drowsy. The
The main purpose of this paper is to utilize the program is design to monitor and warn in drowsy condition
reflection of retina as a way to discovering the eyes on the as it starts. It will not take time to analyze any situation.
human face, and as a method utilizing the nonappearance of This prototype is straightforward for checking eye and
this reflection distinguishing whether the eyes are closed. It detects drowsiness on the basis of eye condition, whether
was observe that this technique may not be the good for the eye is close or open. The system has difficulties in
system for checking the eyes due to the two reasons. First, fmding that, whether the person is Active or not. The
in situation when lighting is lower, the retinal reflection system detects only one eye and second does not detect
amount declines; and next, the retinal reflection may not because of glasses. The eye detection system has
appear, ifthe individual has little eyes, as seen in Figure 5. restrictions when the individual is wearing spectacles.
Localizing the eyes of person is not an issue, but to decide
wh ether the driver's eyes are closed or opened is an issue of
~-:;._.
this created system. In figure 7 test sampie of a person is
k _ _

taken who is
Fig.5 No retinal reflections present

4.2 Drowsy Detection Program

Now we discuss about the logic that we are design for

making the most effective and efficient Drowsiness System.
The running of working of our system is shown in
flowchart of figure 6. Fig.7 Eyeglasses Sampie
More test image sampies are show in test sampies of
different persons. Figure 8(a) shows the active condition
Elf
DetKllo D .,
IH;'Iognun EqualWttion l and the same person drowsy sampie is in Figure 8(b).
r uctlol
I RmlOveNoise I Another person's image sampies are given in Figure 8(a)
• and Figure 9(b). The system encircles detected eye and
show the u are active".

Dro"-sIDf;u
Dtt K IIOIL
ructloll

Fig.8 (a) Active sampie (b) Drowsy sampie

Fig.6 Flow chart of Drowsy Detection Program

This prototype program in C++ by using the computer
vision libraries of OpenCY.First of all an image will be
capture from camera stream and convert it into grayscale
then histogram equalization will be applying on it. After
then it will pass through filters for removing ofnoise. Then
program will detect face to and width and find the eyes by Fig.9 (a) Active sampie (b) Drowsy sampie
intensity changer because in eye region rapid change of

271
 It is clear that there are impediments to the system. Detection", 11th International Conference on
The most noteworthy impediment is that it won't work with Enhanced Safety ofVehieles Munich", 1994.
dark skin peoples. Binarization is the core of algorithm on [4] Ma, Q., Haider, M., R., Yuan, S., Islam, S., K.,
which system is dependant. Binarization doesn't work for "Power-oscillator based high efficiency inductive
the dark skinned individuals. There cannot be any power-link for transcutaneous power transmission"
intelligent items behind the driver, it is another limitation. IEEE International Midwest Symposium on Circuits
and Systems, pp.537-540, Aug. 2010.
5. Conclusions and Future Work [5] Daimler AG, http://www.daimler.com. last visited:
1112512013
A non-intrusive drowsy driver detection system has [6] Ford Motor Company, http://www .ford.com. last
been developed to concentrate the eyes of driver and check visited: 11125/2013
the drowsiness. During the experiment the system has the [7] Volvo Car Corporation, http://www.volvocars.com.
capacity to choose wh ether the driver's eyes are opened or last visited: 1/25/2013
closed. At the point when the eyes are close for a really [8] Volkswagen AG, http://www.volkswagen.com. last
long time, a warning sign is issued to driver. Also, visited: 11/25/2013
throughout observation, the framework has the capacity [9] Weirwille, W.W. (1994). "Overview of Research on
suddenly identify any eye confining error that may have Driver Drowsiness Definition and Driver Drowsiness
happened. Ifthere should arise an occurrence ofthis kind of Detection," 14th International Technical Conference
error, the system has the capacity to recover and accurately on Enhanced Safety ofVehicles,pp 23-26.
localize the person eyes. Image processing accomplishes [10] Wierville, W.W. (1994) Overview of research on
greatly precise and trustworthy finding of sleepiness, a driver drowsiness definition and driver drowsiness
drowsiness detection system which judges the alertness detection. ESV, Munich.
level of driver on the basis of nonstop eye closures. [11] Dinges, D.F., Mallis, M. , Maislin, G. & Powell, J.W.
Future work may be to design the alarm system an (1998) Evaluation of techniques for ocular
ergonomie design so that, it can work effectively and warn measurement as an index of fatigue and the basis for
the driver when drowsy condition is occurs. For alarm alertness management. Department of Transportation
circuit design a prototype of alarm that is connecting with Highway Safety Publication 808762, April 1998.
image processing device (Computer prototype) using serial [12] J. Huang, D. Ii, X. Shao, and H. Wechsler, "Pose
(RS-232). The system has limitation it does not work for discrimination and eye detection using support vector
dark skinned. By using adaptive light source this can be machines (svms)," in Proceeding of NATO-ASI on
making correct. Face Recognition: From Theory to Applications, 1998,
pp.528-536.
Acknowledgment [13] Y.Matsumoto, J. Heinzmann and A. Zelinsky "The
Essential Components of Human- Friendly Robot
This paper was supported by the National Natural Systems". Procs. Of the International Conference on
Science Foundation of China (Grant No.61370073), the Field and Service Robotics (FSR' 99). Pittsburgh,
National High Technology Research and Development Pennsylvania, USA, 1999.
Program of China (Grant NO.2007 AAO lZ423). [14] Ma, Q., Haider, M.R., Islam, S.K., "A high efficiency
inductive power link and backward telemetry for
References biomedical applications", IEEE in Sensors, pp. 89-93,
2010.
[1] L. Hartley, T. Horberry, N. Mabbott, and G. Krueger,
"Review of fatigue detection and prediction
technologies", National Road Transport Commission
report, 2000.
[2] Ulrika Svensson, Blink behaviour based drowsiness
detection, Master's thesis project in Applied Physics
and Electrical Engineering, Dept. Biomedical
Engineering, Linköping University, Sweden, 2004.
[3] Wierville. "Overview of Research on Driver
Drowsiness Definition and Driver Drowsiness

272