VINEELA

Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 47

PLANT DISEASE DETECTION AND CLASSFICATION

USING ANN

A Mini Project Report submitted to


JNTU Hyderabad in partial fulfillment
of the requirements for the award of the degree

BACHELOR OF TECHNOLOGY
In
ELECTRONICS AND COMMUNICATION ENGINEERING

Submitted by

VINEELA NAGIREDDY 20RG1A0437

SURAKANTI SRINITHA 20RG1A0450

BODDU PAVANI 20RG1A0410

AKUBATHINI CHANDANA 20RG1A0403

BADDULA TEJASWINI 20RG1A0412

Under the Guidance of


Mrs. D. ANITHA
B.Tech., M.Tech

Assistant Professor

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING


MALLA REDDY COLLEGE OF ENGINEERING FOR WOMEN
(Approved by AICTE New Delhi and Affiliated to JNTUH)
(An ISO 9001: 2015 Certified Institution)
(B.Tech Programs(CSE & ECE) Accredited by NBA)
Maisammaguda,Medchal (M), Hyderabad-500100, T.S
SEPTEMBER 2023
DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING
MALLA REDDY COLLEGE OF ENGINEERING FOR WOMEN
(Approved by AICTE New Delhi and Affiliated to JNTUH)
(An ISO 9001: 2015 Certified Institution)
(B.Tech Programs(CSE,ECE) Accredited by NBA)
Maisammaguda, Medchal (M), Hyderabad-500100, T. S.
SEPTEMBER 2023

CERTIFICATE

This is to certify that the Major project entitled “PLANT LEAF


DISEASE DETECTION AND CLASSIFICATION USING ANN” has been
submitted by NAGIREDDY VINEELA (20RG1A0437),
SURAKANTI SRINITHA (20RG1A0450), BODDU PAVANI
(20RG1A0410), AKUBATHINI CHANDANA (20RG1A0403),
BUDDULA TEJASWINI (20RG1A0412) in partial fulfillment of the
requirements for the award of BACHELOR OF TECHNOLOGY in
ELECTRONICS & COMMUNICATION ENGINEERING. This record of
bonafide work was carried out by them under my guidance and supervision.
The result embodied in this mini-project report has not been submitted
to any other University or Institute for the award of any degree.

Mrs. D. Anitha Dr. Selvamani Indrajith


Assistant Professor Head of the Department
Project Guide

External Examiner
ACKNOWLEDGEMENT
The Mini Project work carried out by our team in the Department of
Electronics and Communication Engineering, Malla Reddy College of
Engineering for Women, Hyderabad. This work is original and has not
been submitted in part or full for any degree or diploma of any other
university.
We wish to acknowledge our sincere thanks to our project guide Mrs
D.Anitha, Assistant Professor in Department of Electronics &
Communication Engineering for formulating the problem, analysis, guidance
and her continuous supervision during the project work.

We acknowledge our sincere thanks to Dr. Vaka Murali Mohan,


Director, MRCEW. Dr.Kanaka Durga Returi, Principal and Professor of
Computer Science & Engineering Department. Dr.Selvamani Indrajith,
Professor and Head of the Electronics & Communication Engineering
Department, for their kind cooperation in making this Mini Project work a
success.

We extend our gratitude to Sri. Ch. Malla Reddy, Founder-Chairman


and Sri. Ch. Mahender Reddy, Secretary, MRCEW and
Sri. P. Praveen Reddy Director, MRGI for their kind cooperation in
providing the infrastructure for successful completion of our Mini project
work.

We convey our special thanks to the Entire Teaching faculty and non-
teaching staff members of the Electronics & Communication Engineering
Department for their support in making this project work a success.

NAGIREDDY VINEELA (20RG1A0437)


SURAKANTI SRINITHA (20RG1A0450)
BODDU PAVANI (20RG1A0410)
AKUBATHINI CHANDANA (20RG1A0403)
BUDDULA TEJASWINI (20RG1A0412)
i
INDEX
Chapter Page No
ABSTRACT iii
LIST OF FIGURES iv
LIST OF TABLES v
1.INTRODUCTION 1
1.1 Back ground Introduction 1
1.2 Goals 2
1.3 Objectives 3
2.LITERATURE SURVEY 10
3.EXISTING SYSTEM 13

4.PROPOSED SYSTEM 14

5.MODULE DESCRIPTION 16

6.SYSTEM REQUIREMENTS 19

6.1 Hardware Requirements 19

6.2 Software Requirements 19

7.RESULTS AND DISCUSSIONS 20

8.CONCLUSION & FUTURE SCOPE 26

9. APPENDIX 27

10. BIBLIOGRAPHY 40

ii
ABSTRACT

In this project, Visual Behavior of a driver is monitoring with webcam


and machine learning SVM (support vector machine) algorithm, we are
detecting drowsiness in a driver. This application will use inbuilt webcam to
read pictures of a driver and then using OPENCV SVM algorithm extract
facial features from the picture and then check whether driver in picture is
blinking his eyes for consecutive 20 frames or yawning mouth then
application will alert driver with Drowsiness messages. We are using SVM
pre-trained drowsiness model and then using Euclidean distance function
we are continuously checking or predicting EYES and MOUTH distance
closer to drowsiness, if distance is closer to drowsiness, then application will
alert driverso we can prevent accidents happening because of driver fatigue
and sleepiness.

LIST OF FIGURES
iii
FIG.NO FIGURE NAME PAGE NO

1.1 Supervised Learning 6

1.2 Unsupervised Learning 7

1.3 Machine Learning Process 9

4.1 Block Diagram of Proposed Drowsiness Detection 15

7.1 Checking Behaviour Using Webcam 23

7.2 Checking Drowsiness of a person 24

7.3 Closed Eyes gets Driver Drowsiness Alert 24

7.4 25
Yawning Mouth gets Drowsiness Alert

7.5 Closed Eyes or Sleepiness Eyes also gets an alert 25


message.

LIST OF TABLES

iv
TABLE NO TABLE NAME PAGE NO

1 Difference between supervised and 6


unsupervised learning

v
CHAPTER1: INTRODUCTION

1.1.Background Introduction:

Occurring in road accidents. The truck drivers who drive for


continuous long hours (especially at night), bus drivers of long-distance
route or overnight buses are more susceptible to this problem. Driver
drowsiness is an overcast nightmare to passengers in every country. Every
year, a large number of injuries and deaths occur due to fatigue related road
accidents. Hence, detection of driver’s fatigue and its indication is an active
area of research due to its immense practical applicability.

The basic drowsiness detection system has three blocks/modules;


acquisition system, processing system and warning system. Here, the video
of the driver’s frontal face is captured in acquisition system and transferred
to the processing block where it is processed online to detect drowsiness. If
drowsiness is detected, a warning or alarm is sent to the driver from the
warning system. Generally, the methods to detect drowsy drivers are
classified in three types; vehicle based, behavioural based and physiological
based.

In vehicle-based method, a number of metrics like steering wheel


movement, accelerator or brake pattern, vehicle speed, lateral acceleration,
deviations from lane position etc. are monitored continuously. Detection of
any abnormal change in these values is considered as driver drowsiness.

This is a nonintrusive measurement as the sensors are not attached


on the driver. In behavioral based method, the visual behavior of the driver
i.e., eye blinking, eye closing, yawn, head bending etc. are analyzed to detect
drowsiness.

This is also nonintrusive measurement as simple camera is used to


detect these features. In physiological based method [8,9], the physiological
signals like Electrocardiogram (ECG), Electooculogram (EOG),

1
Electroencephalogram (EEG), heartbeat, pulse rate etc. are monitored and
from these metrics, drowsiness or fatigue level is detected.

This is intrusive measurement as the sensors are attached on the


driver which will distract the driver.

Depending on the sensors used in the system, system cost as well as


size will increase. However, inclusion of more parameters/features will
increase the accuracy of the system to a certain extent. These factors
motivate us to develop a low-cost, real time driver’s drowsiness detection
system with acceptable accuracy. Hence, we have proposed a webcam-based
system to detect driver’s fatigue from the face image only using image
processing and machine learning techniques to make the system low-cost as
well as portable

1.2Goals:

Proposed driver drowsiness monitoring system has been depicted in


Fig 1. At first, the video is recorded using a webcam. The camera will be
positioned in front of the driver to capture the front face image. From the
video, the frames are extracted to obtain 2-D images. Face is detected in the
frames using histogram of oriented gradients (HOG) and linear support
vector machine (SVM) for object detection.

After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block

are discussed below

1.3 OBJECTIVES

In this project by monitoring Visual Behaviour of a driver with


webcam and machine learning SVM (support vector machine) algorithm we

2
are detecting Drowsiness in a driver. This application will use inbuilt
webcam to read pictures of a driver and then using OPENCV SVM algorithm
extract facial features from the picture and then check whether driver in
picture is blinking his eyes for consecutive 20 frames or yawning mouth
then application will alert driver with Drowsiness messages.

We are using SVM pre-trained drowsiness model and then using


Euclidean distance function we are continuously checking or predicting
EYES and MOUTH distance closer to drowsiness, if distance is closer to
drowsiness, then application will alert driver.

Drowsy driving is one of the major causes of deaths occurring in road


accidents. The truck drivers who drive for continuous long hours (especially
at night), bus drivers of long-distance route or overnight buses are more
susceptible to this problem. Driver drowsiness is an overcast nightmare to
passengers in every country. Every year, a large number of injuries and
deaths occur due to fatigue related road accidents.

Hence, detection of driver’s fatigue and its indication is an active area


of research due to its immense practical applicability. The basic drowsiness
detection system has three blocks/modules; acquisition system, processing
system and warning system. Here, the video of the driver’s frontal face is
captured in acquisition system and transferred to the processing block
where it is processed online to detect drowsiness.

If drowsiness is detected, a warning or alarm is sent to the driver from


the warning system. Generally, the methods to detect drowsy drivers are
classified in three types; vehicle based, behavioral based and physiological
based. In vehicle-based method, a number of metrics like steering wheel
movement, accelerator or brake pattern, vehicle speed, lateral acceleration,
deviations from lane position etc. are monitored continuously.

Detection of any abnormal change in these values is considered as


driver drowsiness. This is a nonintrusive measurement as the sensors are
not attached on the driver. In behavioral based method, the visual behavior

3
of the driver i.e., eye blinking, eye closing, yawn, head bending etc. are
analyzed to detect drowsiness. This is also nonintrusive measurement as
simple camera is used to detect these features.

In physiological based method, the physiological signals like


Electrocardiogram (ECG), Electooculogram (EOG), Electroencephalogram
(EEG), heartbeat, pulse rate etc. are monitored and from these metrics,
drowsiness or fatigue level is detected. This is intrusive measurement as the
sensors are attached on the driver which will distract the driver. Depending
on the sensors used in the system, system cost as well as size will increase.

However, inclusion of more parameters/features will increase the


accuracy of the system to a certain extent. These factors motivate us to
develop a low-cost, real time driver’s drowsiness detection system with
acceptable accuracy. Hence, we have proposed a webcambased system
todetect driver’s fatigue from the face image only using image processing
and machine learning techniques to make the system low-cost as well as
portable.

A block diagram of the proposed driver drowsiness monitoring system


has been depicted in Fig 1. At first, the video is recorded using a webcam.
The camera will be positioned in front of the driver to capture the front face
image. From the video, the frames are extracted to obtain 2-D images. Face
is detected in the frames using histogram of oriented gradients (HOG) and
linear support vector machine (SVM) for object detection.

After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block
are discussed below.

4
What Is Machine Learning and How Does It Work?

For starters, machine learning is a core sub-area of Artificial


Intelligence (AI). ML applications learn from experience (or to be accurate,
data) like humans do without direct programming. When exposed to new
data, these applications learn, grow, change, and develop by themselves. In
other words, machine learning involves computers finding insightful
information without being told where to look. Instead, they do this by
leveraging algorithms that learn from data in an iterative process.

The concept of machine learning has been around for a long time
(think of the World War II Enigma Machine, for example). However, the idea
of automating the application of complex mathematical calculations to big
data has only been around for several years, though it’s now gaining more
momentum. At a high level, machine learning is the ability to adapt to new
data independently and through iterations. Applications learn from
previous computations and transactions and use “pattern recognition” to
produce reliable and informed results.

How Does Machine Learning Work?

Machine Learning is, undoubtedly, one of the most exciting subsets of


Artificial Intelligence. It completes the task of learning from data with
specific inputs to the machine. It’s important to understand what makes
Machine Learning work and, thus, how it can be used in the future. The
Machine Learning process starts with inputting training data into the
selected algorithm.

Training data being known or unknown data to develop the final


Machine Learning algorithm. The type of training data input does impact the
algorithm, and that concept will be covered further momentarily. New input
data is fed into the machine learning algorithm to test whether the algorithm
works correctly. The prediction and results are then checked against each
other.

What are the Different Types of Machine Learning?

5
Machine Learning is complex, which is why it has been divided into
two primary areas, supervised learning and unsupervised learning. Each
one has a specific purpose and action, yielding results and utilizing various
forms of data. Approximately 70 percent of machine learning is supervised
learning, while unsupervised learning accounts for anywhere from 10 to 20
percent. The remainder is taken up by reinforcement learning.

Factors Supervised learning Unsupervised


learning
Input Known and Labeled data Unknown data

Complexity Very complex Less complex


Number of classes Known Unknown
Accuracy Accurate and reliable Moderately Accurate
and reliable
Table -1: Difference between Supervised and Unsupervised Learning

1. Supervised Learning:

In supervised learning, we use known or labeled data for the training


data. Since the data is known, the learning is, therefore, supervised, i.e.,
directed into successful execution. The input data goes through the Machine
Learning algorithm and is used to train the model. Once the model is
trained based on the known data, you can use unknown data into the model
and get a new response.

fig 1.1:Supervised Learning

6
In this case, the model tries to figure out whether the data is an apple or
another fruit. Once the model has been trained well, it will identify that the
data is an apple and give the desired response.

Here is the list of top algorithms currently being used for supervised
learning are:

1. Polynomial regression

2. Random Forest

3. Linear regression

4. Logistic regression

5. Decision trees

6. K-nearest neighbors

7. Naive Bayes

2. Unsupervised Learning:

In unsupervised learning, the training data is unknown and unlabeled


- meaning that no one has looked at the data before. Without the aspect of
known data, the input cannot be guided to the algorithm, which is where
the unsupervised term originates from. This data is fed to the Machine
Learning algorithm and is used to train the model. The trained model tries
to search for a pattern and give the desired response. In this case, it is often
like the algorithm is trying to break code like the Enigma machine but
without the human mind directly involved but rather a machine.

7
Fig1.2: Unsupervised Learning

In this case, the unknown data consists of apples and pears which look
similar to each other. The trained model tries to put them all together so
that you get the same things in similar groups.

The top 7 algorithms currently being used for unsupervised learning are:

1. Partial least squares

2. Fuzzy means

3. Singular value decomposition

4. K-means clustering

5. Apriori

6. Hierarchical clustering

7. Principal component analysis

3. Reinforcement Learning:

Like traditional types of data analysis, here, the algorithm discovers


data through a process of trial and error and then decides what action
results in higher rewards. Three major components make up reinforcement
learning: the agent, the environment, and the actions. The agent is the
learner or decision-maker, the environment includes everything that the
agent interacts with, and the actions are what the agent does.

8
Why is Machine Learning Important?

To better answer the question “what is machine learning” and


understand the uses of Machine Learning, consider some of the applications
of Machine Learning: the self-driving Google car, cyber fraud detection, and
online recommendation engines from Facebook, Netflix, and Amazon.
Machines make all these things possible by filtering useful pieces of
information and piecing them together based on patterns to get accurate
results.

fig1.3: Machine Learning Process


Main Uses of Machine Learning:

Typical results from machine learning applications usually include


web search results, real-time ads on web pages and mobile devices, email
spam filtering, network intrusion detection, and pattern and image
recognition. All these are the by-products of using machine learning to
analyze massive volumes of data.

9
CHAPTER 2: LITERATURE SURVEY

Intelligent D. Dakopoulos, et al [1]. An intelligent video-based drowsy


driver detection system, which is unaffected by various illuminations, is
developed in this study. Even if a driver wears glasses, the proposed system
detects the drowsy conditions effectively. By a near-infrared-ray (NIR)
camera, the proposed system is divided into two cascaded computational
procedures: the driver eyes detection and the drowsy driver detection.

Hugo Fernandesc, et al [2]A vision-based real-time driver fatigue detection


system is proposed for driving safely. The driver's face is located, from color
images captured in a car, by using the characteristic of skin colors. Then,
edge detection is used to locate the regions of eyes. In addition to being used
as the dynamic templates for eye tracking in the next frame, the obtained
eyes' images are also used for fatigue detection in order to generate some
warning alarms for driving safety. The system is tested on a Pentium III 550
CPU with 128 MB RAM.

The experiment results seem quite encouraging andpromising. The


system can reach 20 frames per second for eye tracking, and the average
correct rate for eye location and tracking can achieve 99.1% on four test
videos. The correct rate for fatigue detection is l00%, but the average
precision rate is 88.9% on the test videos.

The average open/closed eyes detection rates without/with glasses are


94% and 78%, respectively, and the accuracy of the drowsy status detection
is up to 91%. By implementing on the FPGA-based embedded platform, the
processing speed with the 640×480 format video is up to 16 frames per
second (fps) after software optimizations.

Sabarish S [3] In this paper, we describe a non-intrusive vision-based


system for the detection of driver fatigue. The system uses a color video
camera that points directly rewards the driver's face and monitors the
10
driver's eyes in order to detect micro-sleeps (short periods of sleep). The
system deals with skin-color information in order to search for the face in
the input space. After segmenting the pixels with skin like color, we perform
blob processing in order to determine the exact position of the face.

We reduce the search space by analyzing the horizontal gradient map


of the face, taking into account the knowledge that eye regions in the face
present a great change in the horizontal intensity gradient. In order to find
and track the location of the pupil, we use gray scale model matching. We
also use the same pattern recognition technique to determine whether the
eye is open or closed. If the eyes remain closed for an abnormal period of
time (5-6 sec), the system draws the conclusion that the person is falling
asleep and issues a warning signal.

Pooja Sharma [4] Drowsiness detection has many implications including


reducing roads traffic accidents importance. Using image processing
techniques is amongst the new and reliable methods in sleepy face. The
present pilot study was done to investigate sleepiness and providing images
of drivers' face, employing virtual-reality driving simulator. In order to
detecting level of sleepiness according to the signal, information related to
25 drivers was recorded with imaging rate of 10 fps. Moreover, on average
3000 frames was analysed for each driver.The frames were investigated by
transforming in grey scale space and based on the Cascade and Viola &
Jones techniques and the images characteristics were extracted using
Binary and Histogram methods. The MPL neural network was applied for
analysing data.70% of information related to each driver were inserted to the
network of which 15% for test and 15% for validation. In the last stage the
accuracy of 93% of the outputs were evaluated. The intelligent detection and
usage of various criteria in long-term time frame are of the advantages of the
present study, comparing to other researches. This is helpful in early
detection of sleepiness and prevents the irrecoverable losses by alarming.

11
CHAPTER 3:EXISTING SYSTEM

The existing system of driver drowsiness detection system has following


disadvantages.

Mainly, using of two cameras in the system one for monitoring the head
movement and the other one for facial expressions. The other disadvantage
is aging of sensors and all these sensors are attached to the driver’s body
which may affect the driver.

So to overcome all these disadvantages we designed a system in which a


live camera is used for monitoring the driver drowsiness condition and alert
the driver which reduces the road accidents.

12
CHAPTER 4: PROPOSED SYSTEM

A block diagram of the proposed driver drowsiness monitoring system has


been depicted in Fig 1. At first, the video is recorded using a webcam. The
camera will be positioned in front of the driver to capture the front face
image. From the video, the frames are extracted to obtain 2-D images. Face
is detected in the frames using histogram of oriented gradients (HOG) and
linear support vector machine (SVM) for object detection.

After detecting the face, facial landmarks like positions of eye, nose,
and mouth are marked on the images. From the facial landmarks, eye
aspect ratio, mouth opening ratio and position of the head are quantified
and using these features and machine learning approach, a decision is
obtained about the drowsiness of the driver. If drowsiness is detected, an
alarm will be sent to the driver to alert him/her. The details of each block
are discussed in the next chapter.

13
fig 4.1: The Block Diagram of Proposed Drowsiness Detection

14
CHAPTER 5:MODULE DESCRIPTION

A. Data Acquisition:

The video is recorded using webcam (Sony CMU-BR300) and the


frames are extracted and processed in a laptop. After extracting the frames,
image processing techniques are applied on these 2D images. Presently,
synthetic driver data has been generated. The volunteers are asked to look
at the webcam with intermittent eye blinking, eye closing, yawning and head
bending. The video is captured for 30 minutes duration.

B. Face Detection:

After extracting the frames, first the human faces are detected.
Numerous online face detection algorithms are there. In this study,
histogram of oriented gradients (HOG) and linear SVM method [10] is used.
In this method, positive samples of descriptors are computed on them.
Subsequently, negative samples (samples that do not contain the required
object to be detected i.e., human face here) of same size are taken and HOG
descriptors are calculated.

Usually, the number of negative samples is very greater than number


of positive samples. After obtaining the features for both the classes, a linear
SVM is trained for the classification task. To improve the accuracy of VM,
hard negative mining is used. In this method, after training, the classifier is
tested on the labeled data and the false positive sample feature values are
used again for training purpose.

For the test image, the fixed size window is translated over the image
and the classifier computes the output for each window location. Finally, the
maximum value output is considered as the detected face and a bounding
box is drawn around the face. This non-maximum suppression step removes
the redundant and overlapping bounding boxes.

15
C. Facial Landmark marking:

After detecting the face, the next task is to find the locations of
different facial features like the corners of the eyes and mouth, the tip of the
nose and so on. Prior to that, the face images should be normalized in order
to reduce the effect of distance from the camera, non-uniform illumination
and varying image resolution. Therefore, the face image is resized to a width
of 500 pixels and converted to grayscale image.

After image normalization, ensemble of regression trees [11] is used to


estimate the landmark positions on face from a sparse subset of pixel
intensities. In this method, the sum of square error loss is optimized using
gradient boosting learning. Different priors are used to find different
structures. Using this method, the boundary points of eyes, mouth and the
central line of the nose are marked and the number of points for eye, mouth
and nose. The facial landmarks are shown in Fig 2. The red points are the
detected landmarks for further processing.

D. Feature Extraction:

After detecting the facial landmarks, the features are computed as


described below. Eye aspect ratio (EAR): From the eye corner points, the eye
aspect ratio is calculated as the ratio of height and width of the eye as given
by them.

E. Classification:

After computing all the three features, the next task is to detect
drowsiness in the extracted frames. In the beginning, adaptive thresholding
is considered for classification. Later, machine learning algorithms are used
to classify the data. For computing the threshold values for each feature, it
is assumed that initially the driver is in complete awake state.

This is called setup phase. In the setup phase, the EAR values for first
three hundred (for 10s at 30 fps) frames are recorded. Out 4of these three

16
hundred initial frames containing face, average of 150 maximum values is
considered as the hard threshold for EAR.

The higher values are considered so that no eye closing instances will
be present. If the test value is less than this threshold, then eye closing (i.e.,
drowsiness) is detected. As the size of eye can vary from person to person,
this initial setup for each person will reduce this effect. Similarly, for
calculating threshold of MOR, since the mouth may not be open to its
maximum in initial frames (setup phase) so the threshold is taken
experimentally from the observations.

If the test value is greater than this threshold then yawn (i.e.,
drowsiness) is detected. Head bending feature is used to find the angle made
by head with respect to vertical axis in terms of ratio of projected nose
lengths. Normally, NLR has values from 0.9 to 1.1 for normal upright
position of head and it increases or decreases when head bends down or up
in the state of drowsiness.

The average nose length is computed as the average of the nose


lengths in the setup phase assuming that no head bending is there. After
computing the threshold values, the system is used for testing. The system
detects the drowsiness if in a test frame drowsiness is detected for at least
one feature.

To make this thresholding more realistic, the decision for each frame
depends on the last 75 frames. If at least 70 frames (out of those 75) satisfy
drowsiness conditions for at least one feature, then the system gives
drowsiness detection indication and the alarm.

17
CHAPTER 6:SYSTEM REQUIREMENTS

6.1 HARDWARE REQUIREMENTS:

• System : Pentium IV 2.4 GHz.

• Hard Disk : 40 GB.

• Floppy Drive : 1.44 Mb.

• Monitor : 15 VGA Colour.

• Mouse : Logitech.

• Ram : 512 Mb.

6.2 SOFTWARE REQUIREMENTS:

 Web technology : HTML, CSS, Java scripts, AJAX


 Python Web Server : Django
 Programming Language : Python
 Algorithm : Deep Neural Network
 Database : MYSQL

CHAPTER 7:RESULTAND DISCUSSIONS


18
In this project we used a web cam for monitoring the behavior of the
driver ,we read few pictures of the driver by using OPEN CV SVM algorithm
we extracted the facial features and calculated the eye aspect ratio, mouth
aspect ratio, Euclidean distance of eye and mouth for consecutive 20
frames .if the calculated Euclidean distance is less for the eyes or more for
mouth the driver will receive an alert message.

We are using SVM pre-trained drowsiness model and also Euclidean


distance function so that we can continuously check or predict EYES and
MOUTH distance to detect drowsiness so, this application can alert
driver.To implement above concept, we are using following modules

 Video Recording: Using this module we will connect application to


webcam using OPENCV built-in function called Video Capture.
 Frame Extraction: Using this module we will grab frames from
webcam and then extract each picture frame by frame and convert
image into 2-dimensional array.
 Face Detection & Facial Landmark Detection: Using SVM algorithm
we will detect faces from images and then extract facial expression
from the frames.
 Detection: Using this module we will detect eyes and mouth from the
face
 Calculate: Using this module we will calculate distance with
Euclidean Distance formula to check whether given face distance
closer to eye blinks or yawning, if eyes blink for 20 frames
continuously and mouth open as yawn then it will alert driver.

OpenCV is an artificial intelligence API available in python to perform


various operation on images such as image recognition, face detection, and
convert images to gray or colored images etc. This API written in C++
languages and then make C++ functions available to call from python using
native language programming. Steps involved in face detection using
OpenCV.

19
Face Detection Using OpenCV:

This seems complex at first but it is very easy. Let me walk you through the
entire process and you will feel the same.

Step 1: Considering our prerequisites, we will require an image, to begin


with. Later we need to create a cascade classifier which will eventually give
us the features of the face.

Step 2: This step involves making use of OpenCV which will read the image
and the features file. So at this point, there are NumPy arrays at the primary
data points.

All we need to do is to search for the row and column values of the face
NumPy N dimensional array. This is the array with the face rectangle
coordinates.

Step 3: This final step involves displaying the image with the rectangular
face box.

SVM Description:

Machine learning involves predicting and classifying data and to do so


we employ various machine learning algorithms according to the dataset.
SVM or Support Vector Machine is a linear model for classification and
regression problems. It can solve linear and non-linear problems and work
well for many practical problems. The idea of SVM is simple: The algorithm
creates a line or a hyperplane which separates the data into classes.

In machine learning, the radial basis function kernel, or RBF kernel,


is a popular kernel function used in various kernelized learning algorithms.
In particular, it is commonly used in support vector machine classification.
As a simple example, for a classification task with only two features (like the
image above), you can think of a hyperplane as a line that linearly separates
and classifies a set of data.

20
Intuitively, the further from the hyperplane our data points lie, the
more confident we are that they have been correctly classified. We therefore
want our data points to be as far away from the hyperplane as possible,
while still being on the correct side of it.

So, when new testing data is added, whatever side of the hyperplane it
lands will decide the class that we assign to it.

How do we find the right hyperplane?

Or, in other words, how do we best segregate the two classes within the
data?

The distance between the hyperplane and the nearest data point from
either set is known as the margin. The goal is to choose a hyperplane with
the greatest possible margin between the hyperplane and any point within
the training set, giving a greater chance of new data being classified
correctly.

Project Description:

Drowsy driving is one of the major causes of road accidents and


death. Hence, detection of driver’s fatigue and its indication is an active
research area. Most of the conventional methods are either vehicle based, or
behavioural based or physiological based. Few methods are intrusive and
distract the driver, some require expensive sensors and data handling.

Therefore, in this study, a low-cost, real-time driver’s drowsiness


detection system is developed with acceptable accuracy. In the developed
system, a webcam records the video and driver’s face is detected in each
frame employing image processing techniques. Facial landmarks on the
detected face are pointed and subsequently the eye aspect ratio, mouth
opening ratio and nose length ratio are computed and depending on
theirvalues, drowsiness is detected based on developed adaptive
thresholding.

21
Screen shots:

To run this project double click on ‘run.bat’ file to get below screen;

fig 7.1: Checking Behavior Using Webcam

In above screen click on ‘Start Behaviour Monitoring Using Webcam’ button


to connect application with webcam, after clicking button will get below
screen with webcam streaming

22
fig 7.2: Checking Drowsiness of a person

In above screen we can see web cam stream then application monitor all
frames to see person eyes are open or not, if closed then will get below
message

fig 7.3: Closed Eyes gets Driver Drowsiness Alert

Similarly, if mouth starts yawn then also will get alert message
23
fig 7.4: Yawning Mouth gets Drowsiness Alert

In the above figure, the yawning of person then person will get alert message

fig 7.5: Closed Eyes or Sleepiness Eyes also gets an alert message.

CHAPTER 8: CONCLUSION& FUTURE SCOPE


24
8.1 CONCLUSION:

We can detect the drowsiness of the driver in three methods ; Vehicle


based, Visual Behavioral based and Psychological based. a low-cost, real-
time driver drowsiness monitoring system has been proposed based on
visual behaviorand machine learning. Here, visual behavior features like eye
aspect ratio, mouth opening ratio and nose length ratio are computed from
the streaming video, captured by a webcam. An adaptive thresholding
technique has been developed to detect driver drowsiness in real time. The
developed system works accurately and helps in detecting the drowsiness of
a driver .

8.2 FUTURE SCOPE:

In the future we can enhance these models by adding zoom in features,


where the system can predict at a high-rate accuracy and it will be more
efficient. With the update of technology and AI techniques we can completely
upgrade our system such that cameras will be only able to capture the eyes
of the driver instead of complete facial expression.

CHAPTER 9: APPENDIX

25
One of the most popular languages is Python. Guido van Rossum released
this language in 1991. Python is available on the Mac, Windows, and
Raspberry Pi operating systems. The syntax of Python is simple and
identical to that of English. When compared to Python, it was seen that the
other language requires a few extra lines. It is an interpreter-based language
because code may be run line by line after it has been written. This implies
that rapid prototyping is possible across all platforms. Python is a big
language with a free, binary-distributed interpreter standard library.

It is inferior to maintenance that is conducted and is straightforward to


learn. It is an object-oriented, interpreted programming language. It
supports several different programming paradigms in addition to object-
oriented programming, including functional and procedural programming. It
supports several different programming paradigms in addition to object-
oriented programming, including practical and procedural programming.
Python is mighty while maintaining a relatively straightforward syntax.
Classes, highly dynamic data types, modules, and exceptions are covered.
Python can also be utilised by programmes that require programmable
interfaces as an external language.

Python Features:

1) Easy: Because Python is a more accessible and straightforward language,


Python programming is easier to learn.

2) Interpreted language: Python is an interpreted language; therefore, it


can be used to examine the code line by line and provide results.

3) Open Source: Python is a free online programming language since it is


open-source.

4) Portable: Python is portable because the same code may be used on


several computer standard libraries: Python offers a sizable library that we
may utilize to create applications quickly.

6) GUI: It stands for GUI (Graphical User Interface)

26
7) Dynamical typed: Python is a dynamically typed language, therefore the
type of the value will be determined at runtime.

Python GUI (Tkinter)

* Python provides a wide range of options for GUI development (Graphical


User Interfaces).

* Tkinter, the most widely used GUI technique, is used for all of them.

* The Tk GUI toolkit offered by Python is used with the conventional Python
interface.

* Tkinter is the easiest and quickest way to write Python GUI programs.

* Using Tkinter, creating a GUI is simple.

* A part of Python's built-in library is Tkinter. The GUI programs were


created.

* Python and Tkinter together give a straightforward and quick way. The Tk
GUI toolkit's object-oriented user interface is called Tkinter.

* Making a GUI application is easy using Tkinter. Following are the steps:

1) Install the Tkinter module in place.

2) The GUI application Makes the primary window

3) Include one or more of the widgets mentioned above in the GUI


application.

4) Set up the main event loop such that it reacts to each user-initiated
event.

Although Tkinter is the only GUI framework included in the Python


standard library, Python includes a GUI framework. The default library for
Python is called Tkinter. Tk is a scripting language often used in designing,
testing, and developing GUIs. Tk is a free, open-source widget toolkit that
may be used to build GUI applications in a wide range of computer
languages.

27
Machine Learning

Artificial intelligence (AI), which includes machine learning, enables


computer systems to learn without being explicitly programmed. It has to do
with statistics and applied mathematics. Mike Robert's definition of machine
learning. As a computer gathers and learns from the data it provides, it may
operate more correctly via machine learning.

For large classes of machine learning, many algorithms are used. We must
provide algorithms with more precise data for them to complete certain jobs.
In some circumstances, a computer will utilize data to gather information,
check its output against the desired outcome, and make necessary
corrections.

For instance, when someone texts on a phone, the phone


learns about spelling errors and either autocorrects the offending word or
suggests a replacement. For many top organizations, machine learning is a
critical component of the creation of new products.

ML is an important factor in the operations of many companies, like


Facebook and Google. Data science uses machine learning in many different
ways. Data scientists rely on ML approaches to carry out their Modeling.
Regression and classification are of utmost relevance in data science; hence,
the main tool utilized in ML is to accomplish such objectives.

ML applies applicable to practically all phases of data science and is most


often associated with the data Modeling phase. Python has been the primary
computer language used for data processing. Several Python packages are
used in ML settings. The three sections of Python are huge data,
optimizing your code, and data files in memory.

1.6 Types of Machine Learning

There are three fundamental forms of machine learning: -supervising, semi-


supervised, and machine learning

28
a) Supervised Machine Learning

* That method looks for patterns in the labeled data set to obtain results.
Data labelling in supervised learning requires human intervention. To train
the algorithm with labeled inputs and the intended output, supervised ML
requires human participation. ML under supervision is good for a task like;

I. Classify the data using a binary system into two groups.

II. Multi-classification: The division of data into more than two categories,

III. Modeling imaging continuous value using regression.

IV. Assembling: Compiling the estimates from many ML models to provide a


precise estimate.

b) Unsupervised Machine Learning

*This method searches for patterns in the data collection without relying on
labeled data or human interaction. Data labeling is not necessary for this
strategy. ML Unsupervised is effective for tasks like;

I. Dimensionality reduction: Reduce the number of variables in the data


collection.

II. Clustering: Grouping the dataset based on similarities.

III. Association mining identifies the item or group of items that commonly
appear together in data.

IV. Data point identification for anomaly detection in the data set

c) Semi-supervised Learning:

*For this method, you require labeled data. As a consequence, human


interaction is also necessary, but the process still moves forward. In this
kind of learning, the algorithm is given a tiny quantity of labeled data by
data scientists, and as a result, the algorithm gains knowledge about the
data set's dimension, which it may then apply to mother del, unlabeled data.

29
* There are several contexts in which semi-supervised machine learning (ML)
may be used.

I. Machine translation: Language conversion using a learning system.

II. Data labelling: An algorithm trained on modest amounts of data will


automatically apply data labels to enormous collections.

1.7 Uses of Machine Learning

*Machine learning is used in many areas nowadays. The most well-known


example is the machine learning recommendation engine that drives a
book's news feed. This engine makes an effort to reinforce established
patterns in a user's online activity inside a certain Facebook group.

*The news is appropriately adjusted if a user alters the design and doesn't
read anything from that particular group the following week. Applications of
machine learning (ML) include business intelligence, human resource
information systems, autonomous vehicles, and virtual assistants.

Advantages:

• ML helps enterprises in comprehending their clients. ML assists in


improving goods in response to client demand by gathering the necessary
user data and associating it with shifting behaviour. Some companies'
business models are heavily reliant on machine learning, such as Uber,
which uses an algorithm to connect drivers and customers. To surface the
advertising in searches, Google employs ML.

Disadvantage:

• ML might be expensive. High wages for machine learning are a result of


data emotions command on the project. These initiatives also often demand
expensive software infrastructure.

• In addition to that, when an algorithm is trained on a data set, ML bias


might develop. That has flaws in it that might provide erroneous results.

30
Steps to choosing the suitable ML model

The issue is solved by selecting the best ML model, which might take some
time. The steps are as follows:

1) For the difficulty with the pure date alignment, the input should be
thought about.

2) Gather, label, and prepare the data as appropriate.

3) To put the right algorithms to use and test them to determine how well
they perform.

Libraries Used

Pandas:

* Pandas is a Python computer language library for data analysis and


manipulation. It offers a specific operation and data format for handling time
series and numerical tables. It differs significantly from the release3-clause
of the BSD license. It is a well-liked open-source of opinion that is utilized in
machine learning and data analysis.

NumPy:

* The NumPy Python library for multi-dimensional, big-scale matrices adds a


huge number of high-level mathematical functions. It is possible to modify
NumPy by utilizing a Python library. Along with line, algebra, and the
Fourier transform operations, it also contains several matrices-related
functions.

Matplotlib:

* It is a multi-platform, array-based data visualization framework built to


interact with the whole SciPy stack. MATLAB is proposed as an open-source
alternative. Matplotlib is a Python extension and a cross-platform toolkit for
graphical plotting and visualization.

Scikit-learn:

31
* The most stable and practical machine learning library for Python is scikit-
learn. Regression, dimensionality reduction, classification, and clustering
are just a few of the helpful tools it provides through the Python interface for
statistical modeling and machine learning. It is an essential part of the
Python machine learning toolbox used by JP Morgan. It is frequently used in
various machine learning applications, including classification and
predictive analysis.

Keras:

* Google's Keras is a cutting-edge deep learning API for creating neural


networks. It is created in Python and is designed to simplify the development
of neural networks. Additionally, it enables the use of various neural
networks for computation. Deep learning models are developed and tested
using the free and open-source Python software known as Keras.

h5py:

* The h5py Python module offers an interface for the binary HDF5 data
format. Thanks to p5py, the top can quickly halt the vast amount of
numerical data and alter it using the NumPy library. It employs common
syntax for Python, NumPy, and dictionary arrays.

SOURCE CODE

from tkinter import *

import tkinter

fromscipy.spatial import distance as dist

from imutils import face_utils

import numpy as np

import imutils

import dlib

import cv2

32
main = tkinter.Tk()

main.title("Driver Sleepiness Alert Monitoring")

main.geometry("500x400")

def EAR(drivereye):

point1 = dist.euclidean(drivereye[1], drivereye[5])

point2 = dist.euclidean(drivereye[2], drivereye[4])

# compute the euclidean distance between the horizontal

distance = dist.euclidean(drivereye[0], drivereye[3])

# compute the eye aspect ratio

ear_aspect_ratio = (point1 + point2) / (2.0 * distance)

return ear_aspect_ratio

def MOR(drivermouth):

# compute the euclidean distances between the horizontal

point = dist.euclidean(drivermouth[0], drivermouth[6])

# compute the euclidean distances between the vertical

point1 =dist.euclidean(drivermouth[2], drivermouth[10])

point2 =dist.euclidean(drivermouth[4], drivermouth[8])

# taking average

Ypoint = (point1+point2)/2.0

# compute mouth aspect ratio

mouth_aspect_ratio = Ypoint/point

return mouth_aspect_ratio

defstartMonitoring():

pathlabel.config(text="Webcam Connected Successfully")


33
webcamera = cv2.VideoCapture(0)

svm_predictor_path = 'SVMclassifier.dat'

EYE_AR_THRESH = 0.25

EYE_AR_CONSEC_FRAMES = 10

MOU_AR_THRESH = 0.75

COUNTER = 0

yawnStatus = False

yawns = 0

svm_detector = dlib.get_frontal_face_detector()

svm_predictor = dlib.shape_predictor(svm_predictor_path)

(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]

(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]

(mStart, mEnd) = face_utils.FACIAL_LANDMARKS_IDXS["mouth"]

while True:

ret, frame = webcamera.read()

frame = imutils.resize(frame, width=640)

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

prev_yawn_status = yawnStatus

rects = svm_detector(gray, 0)

for rect in rects:

shape = svm_predictor(gray, rect)

shape = face_utils.shape_to_np(shape)

leftEye = shape[lStart:lEnd]

rightEye = shape[rStart:rEnd]
34
mouth = shape[mStart:mEnd]

leftEAR = EAR(leftEye)

rightEAR = EAR(rightEye)

mouEAR = MOR(mouth)

ear = (leftEAR + rightEAR) / 2.0

leftEyeHull = cv2.convexHull(leftEye)

rightEyeHull = cv2.convexHull(rightEye)

mouthHull = cv2.convexHull(mouth)

cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 255), 1)

cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 255), 1)

cv2.drawContours(frame, [mouthHull], -1, (0, 255, 0), 1)

if ear < EYE_AR_THRESH:

COUNTER += 1

cv2.putText(frame, "Eyes Closed ", (10, 30),cv2.FONT_HERSHEY_SIMPLEX,


0.7, (0, 0, 255), 2)

if COUNTER >= EYE_AR_CONSEC_FRAMES:

cv2.putText(frame, "SLEEPINESS ALERT!", (10,


50),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

else:

COUNTER = 0

cv2.putText(frame, "Eyes Open ", (10, 30),cv2.FONT_HERSHEY_SIMPLEX,


0.7, (0, 255, 0), 2)

cv2.putText(frame, "EAR: {:.2f}".format(ear), (480,


30),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

if mouEAR> MOU_AR_THRESH:
35
cv2.putText(frame, "Yawning, SLEEPINESS ALERT! ", (10,
70),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

yawnStatus = True

output_text = "Yawn Count: " + str(yawns + 1)

cv2.putText(frame, output_text, (10,100),cv2.FONT_HERSHEY_SIMPLEX,


0.7,(255,0,0),2)

else:

yawnStatus = False

if prev_yawn_status == True and yawnStatus == False:

yawns+=1

cv2.putText(frame, "MAR: {:.2f}".format(mouEAR), (480,


60),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

cv2.putText(frame,"Visual Behaviour & Machine Learning Sleepiness


Detection @ Sleepiness",(370,470),cv2.FONT_HERSHEY_COMPLEX,0.6,
(153,51,102),1)

cv2.imshow("Frame", frame)

key = cv2.waitKey(1) & 0xFF

if key == ord("q"):

break

cv2.destroyAllWindows()

webcamera.release()

font = ('times', 16, 'bold')

title = Label(main, text='Driver Sleepiness Alert Monitoring System using


Visual\n Behaviour and Machine Learning',anchor=W,
justify=LEFT)

title.config(bg='black', fg='white')
36
title.config(font=font)

title.config(height=3, width=120)

title.place(x=0,y=5)

font1 = ('times', 14, 'bold')

upload = Button(main, text="Start Behaviour Monitoring Using Webcam",


command=startMonitoring)

upload.place(x=50,y=200)

upload.config(font=font1)

pathlabel = Label(main)

pathlabel.config(bg='DarkOrange1', fg='white')

pathlabel.config(font=font1)

pathlabel.place(x=50,y=250)

main.config(bg='blue1')

main.mainloop()

37
CHAPTER 10: BIBLIOGRAPHY
1. W. L. Ou, M. H. Shih, C. W. Chang, X. H. Yu, C. P. Fan, "Intelligent
Video- Based Drowsy Driver Detection System under Various
Illuminations and Embedded Software Implementation", 2015
international Conf. on Consumer Electronics - Taiwan, 2015.
2. S. Singh, N. P. papanikolopoulos, “Monitoring Driver Fatigue using
Facial Analysis Techniques”, IEEE Conference on Intelligent
Transportation System, pp 314-318.
3. M. Karchani, A. Mazloumi, G. N. Saraji, A. Nahvi, K. S. Haghighi, B.
M. Abadi, A. R. Foroshani, A. Niknezhad, “The Steps of Proposed
Drowsiness Detection System Design based on Image Processing in
Simulator Driving”, International Research Journal of Applied and
Basic Sciences, vol. 9(6), pp 878-887, 2015.
4. R. Ahmad, and J. N. Borole, “Drowsy Driver Identification Using Eye
Blink Detection,” IJISET - International Journal of Computer Science
and Information Technologies, vol. 6, no. 1, pp. 270-274, Jan. 2015.
5. Abas, J. Mellor, and X. Chen, “Non-intrusive drowsiness detection by
employing Support Vector Machine,” 2014 20th International
Conference on Automation and Computing (ICAC), Bedfordshire, UK,
2014, pp. 188- 193.
6. Sengupta, A. Dasgupta, A. Chaudhuri, A. George, A. Routray, R.
Guha; "A Multimodal System for Assessing Alertness Levels Due to
Cognitive Loading", IEEE Trans. on Neural Systems and Rehabilitation
Engg., vol. 25 (7), pp 1037-1046, 2017.
7. K. T. Chui, K. F. Tsang, H. R. Chi, B. W. K. Ling, and C. K. Wu, “An
accurate ECG based transportation safety drowsiness detection
scheme,” IEEE Transactions on Industrial Informatics, vol. 12, no. 4,
pp. 1438- 1452, Aug. 2016.
8. V. Kazemi and J. Sullivan; "One millisecond face alignment with an
ensemble of regression trees", IEEE Conf. on Computer Vision and
Pattern Recognition, 23-28 June, 2014, Columbus, OH, USA.

38
9. Richard O. Duda, Peter E. Hart, David G. Stork, “Pattern
Classification”, Wiley student edition.

39

You might also like