VINEELA
VINEELA
VINEELA
USING ANN
BACHELOR OF TECHNOLOGY
In
ELECTRONICS AND COMMUNICATION ENGINEERING
Submitted by
Assistant Professor
CERTIFICATE
External Examiner
ACKNOWLEDGEMENT
The Mini Project work carried out by our team in the Department of
Electronics and Communication Engineering, Malla Reddy College of
Engineering for Women, Hyderabad. This work is original and has not
been submitted in part or full for any degree or diploma of any other
university.
We wish to acknowledge our sincere thanks to our project guide Mrs
D.Anitha, Assistant Professor in Department of Electronics &
Communication Engineering for formulating the problem, analysis, guidance
and her continuous supervision during the project work.
We convey our special thanks to the Entire Teaching faculty and non-
teaching staff members of the Electronics & Communication Engineering
Department for their support in making this project work a success.
4.PROPOSED SYSTEM 14
5.MODULE DESCRIPTION 16
6.SYSTEM REQUIREMENTS 19
9. APPENDIX 27
10. BIBLIOGRAPHY 40
ii
ABSTRACT
LIST OF FIGURES
iii
FIG.NO FIGURE NAME PAGE NO
7.4 25
Yawning Mouth gets Drowsiness Alert
LIST OF TABLES
iv
TABLE NO TABLE NAME PAGE NO
v
CHAPTER1: INTRODUCTION
1.1.Background Introduction:
1
Electroencephalogram (EEG), heartbeat, pulse rate etc. are monitored and
from these metrics, drowsiness or fatigue level is detected.
1.2Goals:
After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block
1.3 OBJECTIVES
2
are detecting Drowsiness in a driver. This application will use inbuilt
webcam to read pictures of a driver and then using OPENCV SVM algorithm
extract facial features from the picture and then check whether driver in
picture is blinking his eyes for consecutive 20 frames or yawning mouth
then application will alert driver with Drowsiness messages.
3
of the driver i.e., eye blinking, eye closing, yawn, head bending etc. are
analyzed to detect drowsiness. This is also nonintrusive measurement as
simple camera is used to detect these features.
After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block
are discussed below.
4
What Is Machine Learning and How Does It Work?
The concept of machine learning has been around for a long time
(think of the World War II Enigma Machine, for example). However, the idea
of automating the application of complex mathematical calculations to big
data has only been around for several years, though it’s now gaining more
momentum. At a high level, machine learning is the ability to adapt to new
data independently and through iterations. Applications learn from
previous computations and transactions and use “pattern recognition” to
produce reliable and informed results.
5
Machine Learning is complex, which is why it has been divided into
two primary areas, supervised learning and unsupervised learning. Each
one has a specific purpose and action, yielding results and utilizing various
forms of data. Approximately 70 percent of machine learning is supervised
learning, while unsupervised learning accounts for anywhere from 10 to 20
percent. The remainder is taken up by reinforcement learning.
1. Supervised Learning:
6
In this case, the model tries to figure out whether the data is an apple or
another fruit. Once the model has been trained well, it will identify that the
data is an apple and give the desired response.
Here is the list of top algorithms currently being used for supervised
learning are:
1. Polynomial regression
2. Random Forest
3. Linear regression
4. Logistic regression
5. Decision trees
6. K-nearest neighbors
7. Naive Bayes
2. Unsupervised Learning:
7
Fig1.2: Unsupervised Learning
In this case, the unknown data consists of apples and pears which look
similar to each other. The trained model tries to put them all together so
that you get the same things in similar groups.
The top 7 algorithms currently being used for unsupervised learning are:
2. Fuzzy means
4. K-means clustering
5. Apriori
6. Hierarchical clustering
3. Reinforcement Learning:
8
Why is Machine Learning Important?
9
CHAPTER 2: LITERATURE SURVEY
11
CHAPTER 3:EXISTING SYSTEM
Mainly, using of two cameras in the system one for monitoring the head
movement and the other one for facial expressions. The other disadvantage
is aging of sensors and all these sensors are attached to the driver’s body
which may affect the driver.
12
CHAPTER 4: PROPOSED SYSTEM
After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block
are discussed in the next chapter.
13
fig 4.1: The Block Diagram of Proposed Drowsiness Detection
14
CHAPTER 5:MODULE DESCRIPTION
A. Data Acquisition:
B. Face Detection:
After extracting the frames, first the human faces are detected.
Numerous online face detection algorithms are there. In this study,
histogram of oriented gradients (HOG) and linear SVM method [10] is used.
In this method, positive samples of descriptors are computed on them.
Subsequently, negative samples (samples that do not contain the required
object to be detected i.e., human face here) of same size are taken and HOG
descriptors are calculated.
For the test image, the fixed size window is translated over the image
and the classifier computes the output for each window location. Finally, the
maximum value output is considered as the detected face and a bounding
box is drawn around the face. This non-maximum suppression step removes
the redundant and overlapping bounding boxes.
15
C. Facial Landmark marking:
After detecting the face, the next task is to find the locations of
different facial features like the corners of the eyes and mouth, the tip of the
nose and so on. Prior to that, the face images should be normalized in order
to reduce the effect of distance from the camera, non-uniform illumination
and varying image resolution. Therefore, the face image is resized to a width
of 500 pixels and converted to grayscale image.
D. Feature Extraction:
E. Classification:
After computing all the three features, the next task is to detect
drowsiness in the extracted frames. In the beginning, adaptive thresholding
is considered for classification. Later, machine learning algorithms are used
to classify the data. For computing the threshold values for each feature, it
is assumed that initially the driver is in complete awake state.
This is called setup phase. In the setup phase, the EAR values for first
three hundred (for 10s at 30 fps) frames are recorded. Out 4of these three
16
hundred initial frames containing face, average of 150 maximum values is
considered as the hard threshold for EAR.
The higher values are considered so that no eye closing instances will
be present. If the test value is less than this threshold, then eye closing (i.e.,
drowsiness) is detected. As the size of eye can vary from person to person,
this initial setup for each person will reduce this effect. Similarly, for
calculating threshold of MOR, since the mouth may not be open to its
maximum in initial frames (setup phase) so the threshold is taken
experimentally from the observations.
If the test value is greater than this threshold then yawn (i.e.,
drowsiness) is detected. Head bending feature is used to find the angle made
by head with respect to vertical axis in terms of ratio of projected nose
lengths. Normally, NLR has values from 0.9 to 1.1 for normal upright
position of head and it increases or decreases when head bends down or up
in the state of drowsiness.
To make this thresholding more realistic, the decision for each frame
depends on the last 75 frames. If at least 70 frames (out of those 75) satisfy
drowsiness conditions for at least one feature, then the system gives
drowsiness detection indication and the alarm.
17
CHAPTER 6:SYSTEM REQUIREMENTS
• Mouse : Logitech.
19
Face Detection Using OpenCV:
This seems complex at first but it is very easy. Let me walk you through the
entire process and you will feel the same.
Step 2: This step involves making use of OpenCV which will read the image
and the features file. So at this point, there are NumPy arrays at the primary
data points.
All we need to do is to search for the row and column values of the face
NumPy N dimensional array. This is the array with the face rectangle
coordinates.
Step 3: This final step involves displaying the image with the rectangular
face box.
SVM Description:
20
Intuitively, the further from the hyperplane our data points lie, the
more confident we are that they have been correctly classified. We therefore
want our data points to be as far away from the hyperplane as possible,
while still being on the correct side of it.
So, when new testing data is added, whatever side of the hyperplane it
lands will decide the class that we assign to it.
Or, in other words, how do we best segregate the two classes within the
data?
The distance between the hyperplane and the nearest data point from
either set is known as the margin. The goal is to choose a hyperplane with
the greatest possible margin between the hyperplane and any point within
the training set, giving a greater chance of new data being classified
correctly.
Project Description:
21
Screen shots:
To run this project double click on ‘run.bat’ file to get below screen;
22
fig 7.2: Checking Drowsiness of a person
In above screen we can see web cam stream then application monitor all
frames to see person eyes are open or not, if closed then will get below
message
Similarly, if mouth starts yawn then also will get alert message
23
fig 7.4: Yawning Mouth gets Drowsiness Alert
In the above figure, the yawning of person then person will get alert message
fig 7.5: Closed Eyes or Sleepiness Eyes also gets an alert message.
CHAPTER 9: APPENDIX
25
One of the most popular languages is Python. Guido van Rossum released
this language in 1991. Python is available on the Mac, Windows, and
Raspberry Pi operating systems. The syntax of Python is simple and
identical to that of English. When compared to Python, it was seen that the
other language requires a few extra lines. It is an interpreter-based language
because code may be run line by line after it has been written. This implies
that rapid prototyping is possible across all platforms. Python is a big
language with a free, binary-distributed interpreter standard library.
Python Features:
26
7) Dynamical typed: Python is a dynamically typed language, therefore the
type of the value will be determined at runtime.
* Tkinter, the most widely used GUI technique, is used for all of them.
* The Tk GUI toolkit offered by Python is used with the conventional Python
interface.
* Tkinter is the easiest and quickest way to write Python GUI programs.
* Python and Tkinter together give a straightforward and quick way. The Tk
GUI toolkit's object-oriented user interface is called Tkinter.
* Making a GUI application is easy using Tkinter. Following are the steps:
4) Set up the main event loop such that it reacts to each user-initiated
event.
27
Machine Learning
For large classes of machine learning, many algorithms are used. We must
provide algorithms with more precise data for them to complete certain jobs.
In some circumstances, a computer will utilize data to gather information,
check its output against the desired outcome, and make necessary
corrections.
28
a) Supervised Machine Learning
* That method looks for patterns in the labeled data set to obtain results.
Data labelling in supervised learning requires human intervention. To train
the algorithm with labeled inputs and the intended output, supervised ML
requires human participation. ML under supervision is good for a task like;
II. Multi-classification: The division of data into more than two categories,
*This method searches for patterns in the data collection without relying on
labeled data or human interaction. Data labeling is not necessary for this
strategy. ML Unsupervised is effective for tasks like;
III. Association mining identifies the item or group of items that commonly
appear together in data.
IV. Data point identification for anomaly detection in the data set
c) Semi-supervised Learning:
29
* There are several contexts in which semi-supervised machine learning (ML)
may be used.
*The news is appropriately adjusted if a user alters the design and doesn't
read anything from that particular group the following week. Applications of
machine learning (ML) include business intelligence, human resource
information systems, autonomous vehicles, and virtual assistants.
Advantages:
Disadvantage:
30
Steps to choosing the suitable ML model
The issue is solved by selecting the best ML model, which might take some
time. The steps are as follows:
1) For the difficulty with the pure date alignment, the input should be
thought about.
3) To put the right algorithms to use and test them to determine how well
they perform.
Libraries Used
Pandas:
NumPy:
Matplotlib:
Scikit-learn:
31
* The most stable and practical machine learning library for Python is scikit-
learn. Regression, dimensionality reduction, classification, and clustering
are just a few of the helpful tools it provides through the Python interface for
statistical modeling and machine learning. It is an essential part of the
Python machine learning toolbox used by JP Morgan. It is frequently used in
various machine learning applications, including classification and
predictive analysis.
Keras:
h5py:
* The h5py Python module offers an interface for the binary HDF5 data
format. Thanks to p5py, the top can quickly halt the vast amount of
numerical data and alter it using the NumPy library. It employs common
syntax for Python, NumPy, and dictionary arrays.
SOURCE CODE
import tkinter
import numpy as np
import imutils
import dlib
import cv2
32
main = tkinter.Tk()
main.geometry("500x400")
def EAR(drivereye):
return ear_aspect_ratio
def MOR(drivermouth):
# taking average
Ypoint = (point1+point2)/2.0
mouth_aspect_ratio = Ypoint/point
return mouth_aspect_ratio
defstartMonitoring():
svm_predictor_path = 'SVMclassifier.dat'
EYE_AR_THRESH = 0.25
EYE_AR_CONSEC_FRAMES = 10
MOU_AR_THRESH = 0.75
COUNTER = 0
yawnStatus = False
yawns = 0
svm_detector = dlib.get_frontal_face_detector()
svm_predictor = dlib.shape_predictor(svm_predictor_path)
while True:
prev_yawn_status = yawnStatus
rects = svm_detector(gray, 0)
shape = face_utils.shape_to_np(shape)
leftEye = shape[lStart:lEnd]
rightEye = shape[rStart:rEnd]
34
mouth = shape[mStart:mEnd]
leftEAR = EAR(leftEye)
rightEAR = EAR(rightEye)
mouEAR = MOR(mouth)
leftEyeHull = cv2.convexHull(leftEye)
rightEyeHull = cv2.convexHull(rightEye)
mouthHull = cv2.convexHull(mouth)
COUNTER += 1
else:
COUNTER = 0
if mouEAR> MOU_AR_THRESH:
35
cv2.putText(frame, "Yawning, SLEEPINESS ALERT! ", (10,
70),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
yawnStatus = True
else:
yawnStatus = False
yawns+=1
cv2.imshow("Frame", frame)
if key == ord("q"):
break
cv2.destroyAllWindows()
webcamera.release()
title.config(bg='black', fg='white')
36
title.config(font=font)
title.config(height=3, width=120)
title.place(x=0,y=5)
upload.place(x=50,y=200)
upload.config(font=font1)
pathlabel = Label(main)
pathlabel.config(bg='DarkOrange1', fg='white')
pathlabel.config(font=font1)
pathlabel.place(x=50,y=250)
main.config(bg='blue1')
main.mainloop()
37
CHAPTER 10: BIBLIOGRAPHY
1. W. L. Ou, M. H. Shih, C. W. Chang, X. H. Yu, C. P. Fan, "Intelligent
Video- Based Drowsy Driver Detection System under Various
Illuminations and Embedded Software Implementation", 2015
international Conf. on Consumer Electronics - Taiwan, 2015.
2. S. Singh, N. P. papanikolopoulos, “Monitoring Driver Fatigue using
Facial Analysis Techniques”, IEEE Conference on Intelligent
Transportation System, pp 314-318.
3. M. Karchani, A. Mazloumi, G. N. Saraji, A. Nahvi, K. S. Haghighi, B.
M. Abadi, A. R. Foroshani, A. Niknezhad, “The Steps of Proposed
Drowsiness Detection System Design based on Image Processing in
Simulator Driving”, International Research Journal of Applied and
Basic Sciences, vol. 9(6), pp 878-887, 2015.
4. R. Ahmad, and J. N. Borole, “Drowsy Driver Identification Using Eye
Blink Detection,” IJISET - International Journal of Computer Science
and Information Technologies, vol. 6, no. 1, pp. 270-274, Jan. 2015.
5. Abas, J. Mellor, and X. Chen, “Non-intrusive drowsiness detection by
employing Support Vector Machine,” 2014 20th International
Conference on Automation and Computing (ICAC), Bedfordshire, UK,
2014, pp. 188- 193.
6. Sengupta, A. Dasgupta, A. Chaudhuri, A. George, A. Routray, R.
Guha; "A Multimodal System for Assessing Alertness Levels Due to
Cognitive Loading", IEEE Trans. on Neural Systems and Rehabilitation
Engg., vol. 25 (7), pp 1037-1046, 2017.
7. K. T. Chui, K. F. Tsang, H. R. Chi, B. W. K. Ling, and C. K. Wu, “An
accurate ECG based transportation safety drowsiness detection
scheme,” IEEE Transactions on Industrial Informatics, vol. 12, no. 4,
pp. 1438- 1452, Aug. 2016.
8. V. Kazemi and J. Sullivan; "One millisecond face alignment with an
ensemble of regression trees", IEEE Conf. on Computer Vision and
Pattern Recognition, 23-28 June, 2014, Columbus, OH, USA.
38
9. Richard O. Duda, Peter E. Hart, David G. Stork, “Pattern
Classification”, Wiley student edition.
39