Distracted Driver Detection

A BS Final Year Project by

Haseeb Ahmad
2504/FET/BSEE/F15

Muhammad Abdul Mateen Qazi

2478/FET/BSEE/F15

Inamullah khan
2491/FET/BSEE/F15

Supervised by
Engr. Rehan Ahmad

Department of Electrical Engineering

Faculty of Engineering & Technology
International Islamic University, Islamabad

June 2019
 Certificate of Approval

It is certified that we have checked the project presented and demonstrated by Haseeb Ahmad
2504-FET/BSEE/F15, Muhammad Abdul Mateen Qazi 2478-FET/BSEE/F15, Inamullah
khan 2491-FET/BSEE/F15 and approved it.

External Examiner Internal Examiner

Engr. Adeel Jadoon Engr. Zahoor ud Din Sheikh
Lecturer Assistant Professor

Supervisor
Engr. Rehan Ahmad
Lecturer

Department of Electrical Engineering,

Faculty of Engineering & Technology, IIUI

Page ii
In the name of Allah (SWT), the most beneficent and the most merciful

Page iii
 A BS Final Year Project submitted to the
Department of Electrical Engineering
International Islamic University, Islamabad
In partial fulfillment of the requirements
For the award of the degree of
Bachelor of Science in Electronic Engineering

Page iv
 DECLARATION

We hereby declare that this work, neither as a whole nor as a part there of has been copied out
from any source. No portion of the work presented in this report has been submitted in support
of any application for any other degree or qualification of this or any other university or institute
of learning. We further declare that the referred text is properly cited in the references.

Haseeb Ahmad
2504 -FET/BSEE/F15

Muhammad Abdul Mateen Qazi

2478-FET/BSEE/F15

Inamullah khan
2491-FET/BSEE/F15

Page v
 ACKNOWLEDGMENTS

This BS thesis in Electronic Engineering has been conducted at Department of Electrical Engineering,
Faculty of Engineering & Technology, International Islamic University, as part of the degree program.
We would like to thank Engr. Rehan Ahmad for providing us an opportunity to work on this project,
under his supervision and guidance throughout the project. . We would also like to thank Engr. Rehan
Ahmad for his help, efforts and dedicated support throughout the project.

Further we are particularly thankful to almighty Allah and grateful to our parents, brothers and sisters
who always supported and encouraged us during our project and studies at IIUI.

Haseeb Ahmad

M. Abdul Mateen Qazi

Inamullah Khan

Page vi
Project Title: Distracted Driver Detection

Undertaken By: Haseeb Ahmad (2504-FET/BSEE/F15)

Muhammad Abdul Mateen Qazi (2478-FET/BSEE/F15)

Inamullah khan (2491-FET/BSEE/F15)

Supervised By: Engr. Rehan Ahmad

Lecturer

Supervised By:

Date Started: September 2018

Date Completed: June 2019

Tools Used:
 Anaconda IDE
 Python
 Raspberry pi
 Raspbian OS
 Raspberry pi Camera
 SD Card Formatter

Page vii
 ABSTRACT

For the past few years there is a continuous increase in the number of road accidents worldwide.
According to WHO nearly 1.25 million deaths annually are due to road accidents and nearly one out of
five accidents are mistakes of distracted drivers. Motivated by the performance of Convolutional Neural
Networks in computer vision, we present a CNN based system that not only detects the distracted driver
but also identifies the cause of distraction. We worked to develop an accurate and robust system for
detecting distracted driver and alert him against his distraction. We used Convolution Neural Network
(CNN) to solve this problem. We designed a Convolution Neural Network (CNN) based system that
detects the driver and alert him if necessary. The system consist of ten classes one of them is safe driving
while the remaining are distractions.

Page viii
 Table of Contents
CHAPTER 1 ........................................................................................................ 1

Introduction ......................................................................................................... 1
1.1 History ......................................................................................................................... 1

1.2 Motivation ................................................................................................................... 1

1.3 Machine Learning ....................................................................................................... 2

1.3.1 Deep Learning ...................................................................................................... 2

1.3.2 Convolutional Neural Network ................................................................................. 2

1.3.3 Input Layer ................................................................................................................ 3

1.3.4 Rectified Linear Units (ReLU) Layer ........................................................................ 3

1.3.5 Pooling Layer ....................................................................................................... 3

1.3.6 Dropout Layer ...................................................................................................... 3

1.3.7 Fully Connected Layer ......................................................................................... 3

1.4 Existing Systems ......................................................................................................... 4

1.5 Our Approach .............................................................................................................. 4

CHAPTER 2 ........................................................................................................ 5

Project Overview ................................................................................................. 5

2.1 Related Work.............................................................................................................. 5

2.2 Dataset and features ................................................................................................... 5

2.2 Data Split ..................................................................................................................... 5

2.3 Classes ......................................................................................................................... 6

CHAPTER 3 ........................................................................................................ 8

System Design ...................................................................................................... 8

3.1 Python.......................................................................................................................... 8

3.2 Anaconda ..................................................................................................................... 8

3.2.1 Why Anaconda..................................................................................................... 8

3.3 Libraries ...................................................................................................................... 8

Page ix
 3.3.1 Tensor Flow ......................................................................................................... 8

3.3.2 Keras .................................................................................................................... 9

3.3.3 Numpy.................................................................................................................. 9

3.3.4 Matplotlib............................................................................................................. 9

3.3.5 Scikit Learn .......................................................................................................... 9

3.3.6 SciPy .................................................................................................................... 9

3.3.7 OpenCV ............................................................................................................... 9

3.4 Convolution Neural Network .................................................................................... 10

............................................................................................................................. 11
3.5 Results ....................................................................................................................... 12

3.6 Embedded Board ....................................................................................................... 13

3.6.1 Raspberry Pi ............................................................................................................ 14

3.7 Raspbian .................................................................................................................... 16

Installation of OS in Raspberry PI:................................................................................... 16

3.8 Installing OpenCV..................................................................................................... 16

3.9 Libraries .................................................................................................................... 18

3.9.1 Tensor Flow ....................................................................................................... 18

3.9.2 Keras .................................................................................................................. 18

3.9.3 Numpy................................................................................................................ 19

3.9.4 Matplotlib........................................................................................................... 19

3.9.5 Scikit Learn ........................................................................................................ 19

3.9.6 SciPy .................................................................................................................. 19

3.10 Raspberry Pi Camera ................................................................................................. 20

3.11 Testing ....................................................................................................................... 20

Results ................................................................................................................ 21
Future Recommendations..................................................................................................... 23

References .......................................................................................................... 24

Page x
Page xi
 List of Figures

Figure 1: Deep learning ............................................................................................................. 2

Figure 2: CNN............................................................................................................................ 3

Figure 3:Distracted Driver ......................................................................................................... 5

Figure 4: Block Diagram of CNN Model ................................................................................ 11

Figure 5: Accuracy vs Epochs ................................................................................................. 12

Figure 6: Losses vs Epochs ...................................................................................................... 13

Figure 7: Raspberry Pi ............................................................................................................. 14

Figure 8: Test/Predict ............................................................................................................... 20

Page xii
 List of Tables

Table 1: 10 Classes ................................................................................................................... 6

Table 2: Pretrained Models ...................................................................................................... 10

Table 3: Raspberry Pi vs Nvidia Tegra TK1 ........................................................................... 13

Table 4: Raspberry Pi Model Comparison............................................................................... 15

Page xiii
 List of Abbreviations

WHO World Health Organization

CNN Convolution Neural Network

ADAS Advanced Driver Assistance Systems

CDC Center for Disease Control and Prevention

ReLU Rectified Linear Unit

Page xiv
 CHAPTER 1
Introduction

1.1 History
According to the World Health Organization (WHO) survey, 1.3 million people worldwide die in traffic
accidents each year, making them the eighth leading cause of death and an additional 20-50 million are
injured/ disabled. The report also states that the total number of deaths have risen to 1.46 lakhs in 2015
and driver error is the most common cause behind these traffic accidents. The number of accidents
because of distracted driver has been increasing since few years. National Highway Traffic Safety
Administrator of United States (NHTSA) reports deaths of 3477 people and injuries to 391000 people
in motor vehicle crashes because of distracted drivers in 2015 [2]. In the United States, everyday
approximately 9 people are killed and more than 1,000 are injured in road crashes that are reported to
involve a distracted driver [1]. NTHSA describes distracted driving as “any activity that diverts attention
of the driver from the task of driving” which can be classified into Manual, Visual or Cognitive
distraction [2] [1]. As per the definitions of Center for Disease Control and Prevention (CDC), cognitive
distraction is basically “driver’s mind is off the driving”. In other words, even though the driver is in
safe driving posture, he is mentally distracted from the task of driving. He might be lost in thoughts,
daydreaming etc. Distraction because of inattention, sleepiness, fatigue or drowsiness falls into visual
distraction class where “driver’s eyes are off the road”. Manual distractions are concerned with various
activities where “driver’s hands are off the wheel”. Such distractions include talking or texting using
mobile phones, eating and drinking, talking to passengers in the vehicle, adjusting the radio, makeup
etc.

1.2 Motivation
Distracted driving is an ongoing problem that seems to only be getting worse with the dependence on
technology. Not only is the general public becoming more reliant on vehicles, but also on distractions
like cell phones and GPS systems within them. Most modern day vehicles come with touch screen
navigation that requires some level of interaction to operate. Cellphones, too, are a big part of everyday
life interactions, with constant incoming notifications and calls. These upward trends in dependence
seem to have a correlation to the number of crashes as a result of distracted driving.

It is evident that distracted driving is an ongoing issue, and perhaps the very technology that typically
causes distractions can be used to alleviate them. With the current advances in machine learning and
hardware, computer vision techniques has also followed suit and have shown their powerful
capabilities. This work is a stepping stone in the ultimate goal of using computer vision to help fight
the distracted driver problem. If a system within the vehicle is able to detect if a driver is distracted, it
could perhaps remind the driver to focus their attention on driving. Many drivers may not realize what
they do is taking their attention away from driving, possibly because the distractions are so habitual and
ingrained in their daily routine. The hypothetical distracted driver detection system in the car could,
through feedback, make the drivers consciously aware of their actions, and help correct them over time.
This kind of proposed work would, of course, have to eventually be properly engineered, fine-tuned,
and tested before becoming a successful prevention in distracted driving. However, until then, small
steps need to be taken in solving the problem of distracted driver detection

Page 1
1.3 Machine Learning
Machine learning is an application of artificial intelligence that provides systems the ability to
automatically learn and improve from experience without being explicitly programmed. Machine
learning focuses on the development of computer programs that can access data and use it learn for
themselves.

1.3.1 Deep Learning

Deep learning is a part of machine learning methods which is based on artificial neural networks. It has
three types supervised learning, unsupervised learning and semi supervised learning.

Figure 1: Deep learning

1.3.2 Convolutional Neural Network

In order to tackle the distracted driver detection problem, the Convolutional Neural Network (CNN)
model is utilized. CNNs have proven to perform remarkably well on classifying images, and as such,
are a great fit for this problem. It is worth noting that CNNs generally suffer from overfitting, which
occurs when a model adapts too well on trained data, but performs poorly on new data, and thus is said
to generalize poorly. This is an issue that we try to mitigate as much as possible throughout this work.

CNNs rely on the idea that local understanding of an image is good enough, with the practical advantage
of having fewer parameters, as a result reducing the computation time and data required to train the
model. Rather than have a fully connected layer for every pixel, CNNs have enough weights to look at
small parts of the image at a time. The process generally involves a convolution layer, followed by a
pooling and an activation function, but not necessarily in that exact order. These three separate
operations can be applied as separate layers to the original image, generally multiple times. Finally, a
fully connected layer or layers are added at the end in order to classify an image accordingly. With the
highly variable number of combinations and permutations, it can be difficult to find the exact one that
gives the optimal performance. CNN designs are driven by the community and research who thankfully
have made some successful results, and made their CNNs publicly available for others to use and

Page 2
improve upon.

Figure 2: CNN

1.3.3 Input Layer

The input layer holds the pixel values of the input image. Our project uses full color images, so the
input layer is 224 x 224 x1 . Because to reduce computation we need to reduce data and the best way to
reduce data here is to convert these image to gray scale. This will reduce the data by three time of
original data also we don’t need colored data.

1.3.4 Rectified Linear Units (ReLU) Layer

The ReLU layer applies an element-wise non-linear activation function to increase nonlinearity in the
model.

1.3.5 Pooling Layer

A pooling layer reduces the 2D dimensions of the input volume (no change in the depth) to prevent the
model from overfitting and getting too large to compute. It is done independently for each depth slice
of the input volume by applying a small filter for example 2× 2. The most popular pooling function is
the max function (e.g. outputting the maximum values of 2× 2 regions, as a result reducing the
dimension by 75%).

1.3.6 Dropout Layer

Adding dropout layers is a regularization method to prevent overfitting. A dropout layer randomly sets
some unit activations to zero and thus removes some feature detectors. Because some feature detectors
make the model adapt to very complex functions in the context of some other very specific feature
detectors. By dropping out some activations, some of the high variance due to this is removed.

1.3.7 Fully Connected Layer

Fully-connected layers, as the name suggests, is like ordinary NN where each neuron is connected to
all the outputs from the previous layer. The last Fully Connected layer computes probabilities for each
class. For multi-class classification, SoftMax is a popular choice. That is, SoftMax trains the final layer
to correctly predicts with maximal confidence for each image.

Page 3
1.4 Existing Systems
Nowadays, Advanced Driver Assistance Systems (ADAS) are being developed to prevent accidents by
offering technologies that alert the driver to potential problems and to keep the car’s driver and
occupants safe if an accident does occur. But even today’s latest autonomous vehicles require the driver
to be attentive and ready to take the control of the wheel back in case of emergency. Tesla autopilot’s
crash with the white truck-trailer in Williston, Florida in May 2016 was the first fatal crash in testing of
autonomous vehicle. Recently in March 2018, Uber’s self-driving car with an emergency backup driver
behind the wheel struck and killed a pedestrian in Arizona. In both of these fatalities, the safety driver
could have avoided the crashes but evidences reveal that he was clearly distracted. This makes detection
of distracted driver an essential part of the self-driving cars as well. We believe that distracted driver
detection is utmost important for further preventive measures. If the vehicle could detect such
distractions and then warn the driver against it, number of road crashes can be reduced.

1.5 Our Approach

We are focused on detecting manual distractions where driver is engaged in other activities than safe
driving and also identify the cause of distraction. We present a Convolutional Neural Network based
approach for this problem. We also attempt to reduce the computational complexity and memory
requirement while maintaining good accuracy which is desirable in real time applications.

Page 4
 CHAPTER 2
Project Overview
2.1 Related Work
This section summaries review of some of the relevant and significant work from literature for distracted
driver detection. The major cause of manual distractions is usage of cellphones. Many solutions already
exist because this problem was a public challenge hosted by State Farm, an insurance company, on
Kaggle. Many top solutions used pre-trained CNN models and the most popular are VGG-16 and
ResNet, which were state-of-the-art one or two years ago and had been improving. But arrives when
we fit this trained model into the small memory of Raspberry pi because of smaller memory size these
did not help.

2.2 Dataset and features

The data was supplied by State Farm for a public Kaggle challenge. Kaggle hosts numerous machine
learning challenges for the research community with large prize pools. The dataset consists of training
and testing images of people either driving safely or doing one of nine kinds of distracted behaviors. A
little problem arrives when we came across the situation of driving which makes a difference here in
sub-continent. To overcome the problem we decided to create our own dataset with sub-continental
driving side and we did that with perfection. An example input image is shown in Fig. 1.

Figure 3:Distracted Driver

The training images come with correct labels and the challenge is to make the best multi-class
classifications possible.

2.2 Data Split

In this work, however, in order to train and evaluate the performance of the models, the 3200 training
labeled images were split up further into 70% actual training data, 20% validation, and 10% testing
data. This allows us to evaluate the model, but also test the model against unseen data and apply our
own testing measures. The initial concern is that it might not be enough data to train the model; however
this will be supplemented by data augmentation and transfer learning techniques.
Page 5
To evaluate the success of models, the training images were split into train and validation sets. As the
images are highly correlated and each driver only appears in either the training or test set (i.e. none of
the drivers that appear in the provided training set appear in the large test set), it was important to choose
the images of a certain driver in the training set to be the validation images. Only then can these
validation images be independent from the remaining training images of other drivers and thus avoid a
falsely high test accuracy. Few drivers were chosen at random and all of their images were 1 separated
as validation set, which equated to being 20% of the whole training set.

2.3 Classes
There are ten classes in total:

CLASSES DESCRIPTION

c0: Safe driving

c1: texting – left

c2: talking on the phone – left

c3: texting – right

c4: talking on the phone – right

c5: reaching behind

c6: operating the radio

c7: Drinking

c8: talking to passenger

c9: hair and makeup

Table 1: 10 Classes

The dataset consists of images of drivers performing one of 10 possible activities, one of which is safe
driving. The rest of the images belong to classes where the driver can be considered distracted (e.g.
texting). An example of the images used can be seen in Figure 1. It’s worth noting that all the images
are typically taken from the same angle, in the same environmental conditions (i.e. daytime) which is
both good and bad in training the model. It could be considered a positive event because, if a subset of
this data is used as the test set, then the model will most likely perform well since the test images have
the same conditions. However, this posses an issue when the model is generalized to images that don’t
necessarily meet the same conditions as the ones in the original dataset. The hypothesis is that a drastic
change in angle, or incoming light will cause the model to perform very poorly. This hypothesis is hard
to confirm without finding a new dataset of distracted drivers in varying conditions, which, to our
knowledge, does not exist.

Page 6
In order to train a model to detect distracted drivers, a dataset of images of both distracted and non-
distracted drivers is required. This would only require a binary classifier model that would predicted if
a driver is distracted or not. A more interesting problem would be one where the drivers are distracted
in different ways, like eating or fixing their hair for example. This is a harder problem and requires a
multi classifier model and a more specific dataset.

Page 7
 CHAPTER 3
System Design

Designing system consists of two main parts. First of all we need to gather the required data already
explained in the previous chapter for training the model that we are about to develop and then we are
going to load that trained model on our embedded board for real time predictions. Hence our designing
process can simply be divided in to two main categories Software and Hardware parts. Now let’s explain
software part first.

3.1 Python
Python is a general purpose and one of the high-level programming languages. A variable can be used
directly without its declaration while writing code in python.

Python is known for its simplicity, easy to read concerning code and recognized as one of the high-level
programming languages. Python consists of inbuilt constructs which have the feature of executing clear
code on small as well as large scale. Python is an object-oriented programming language.

3.2 Anaconda
The open-source Anaconda Distribution is the easiest way to perform Python/R data science and
machine learning on Linux, Windows, and Mac OS X. With over 11 million users worldwide, it is the
industry standard for developing, testing, and training on a single machine, enabling individual data
scientists to.

3.2.1 Why Anaconda

 Quickly download 1,500+ Python/R data science packages

 Manage libraries, dependencies, and environments with Conda

 Develop and train machine learning and deep learning models with scikit-learn, TensorFlow,
and Theano

 Analyze data with scalability and performance with Dask, NumPy, pandas, and Numba

 Visualize results with Matplotlib, Bokeh, Datashader, and Holoviews

3.3 Libraries
Advance libraries and high level APIs are essential part of this project. Python supports more than 1500
libraries. We need to install only those libraries that are essential for our work. Here we are going to
install Keras, Tensor Flow, OpenCV, Scikit-learn, Sci py etc. Commands for installation of these are
given below

3.3.1 Tensor Flow

TensorFlow is an end-to-end open source platform for machine learning. It has a
comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers
push the state-of-the-art in ML and developers easily build and deploy ML powered applications.

conda install -c conda-forge tensorflow

Page 8
 3.3.2 Keras
Keras is an open-source neural-network library written in Python. It is capable of running on
top of TensorFlow, Microsoft Cognitive Toolkit, Theano. Designed to enable fast experimentation with
deep neural networks,

conda install -c conda-forge keras

3.3.3 Numpy
NumPy is the fundamental package for scientific computing with Python. It contains among
other things:

 a powerful N-dimensional array object

 sophisticated (broadcasting) functions

 useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container
of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily
integrate with a wide variety of databases.

conda install -c anaconda numpy

3.3.4 Matplotlib
Matplotlib is a Python 2D plotting library which produces publication quality figures in a
variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in
Python scripts, the Python and IPython shells, the Jupyter notebook, web application servers, and four
graphical user interface toolkits.

conda install -c conda-forge matplotlib

3.3.5 Scikit Learn

Scikit-learn is a free software machine learning library for the Python programming language.
It features various classification, regression and clustering algorithms including support vector
machines and etc.

conda install -c anaconda scikit-learn

3.3.6 SciPy
SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics,
science, and engineering

conda install -c anaconda scipy

3.3.7 OpenCV
OpenCV (Open Source Computer Vision Library) is an open source computer vision and
machine learning software library. OpenCV was built to provide a common infrastructure for computer

Page 9
vision applications and to accelerate the use of machine perception in the commercial products. Being
a BSD-licensed product, OpenCV makes it easy for businesses to utilize and modify the code.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of both
classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be
used to detect and recognize faces, identify objects, classify human actions in videos

conda install -c conda-forge opencv

3.4 Convolution Neural Network

Designing the perfect CNN was the complex and very much difficult task that’s why our first priority
was to use the pretrained model. We have tested the few of them and check out the performance. Here
are the some of results from them.

MODEL ACCURACY

Alex net 95.77%

Inception V3 95.98%

VGG 16 95.54%

Table 2: Pretrained Models

Our all these models perform very well and the highest performing can be chosen but problem arrives
when we have to fit these models on Raspberry Pi small memory. Where all the got failed to fit that
was the point when we decided to make our own model and also reduced the data by going for gray
scaled images instead of going for RGB three times the gray scaled data.

The designed model output graphs are quit impressive but when came unseen data exposes its potential
hence possess some overfitting .

Overfitting occurs when your model perform exceptionally well on trained data but when came across
unseen data fails to predict right. This is where concept of fine tuning came to help reduce overfitting.

Keras fine tuning is there to help reduce the overfitting in your CNN model by adding some Dropouts
explained in previous chapter, reducing the convolution layers and also by increasing the trained data
through some techniques or by introducing new data.

In our case Dropout and data increasing help a lot in achieving good accuracy but we are also trading
with approximately 66% of our data by converting RGB to Gray Scale. Hence we need to increase the
data and by doing so our model performs quit well.

Currently our model perform quit good in real time but our trained model accuracy was about 80%
which was quit good but clearly shows the tradeoff that we made with Pretrained model to meet our
hardware compatibility.

In our model we use train images to train our CNN model to learn ten classes patterns and required
dimension well enough so that when came across fresh data then model possess the ability to make
correct predictions. Now we are able test our models by pushing them to face some fresh data. This
block diagram completely demonstrate the hierarchy of our model flow.

Page 10
 Training Neural
Data Network
Model

Image Classifier Prediction

Acquisition

Figure 4: Block Diagram of CNN Model

Page 11
One important thing here is don’t use same data for training as well as for validation this will lead you
to overfitted model and model will not perform well on unseen data. So for that purpose we need to
select few driver whom pictures should be kept separate for random testing.

3.5 Results
As discussed above that the model performs well but not quit well as compared to the results of
pretrained model. The comparison between the accuracy and losses was necessary to discuss here.
Overall accuracy achieved through this model was 80% .some results are shown here in figures.

Figure 5: Accuracy vs Epochs

Page 12
Figure 6: Losses vs Epochs

Now it’s the time to move on second part of the project which is hardware.

3.6 Embedded Board

Firstly we are considering the GPU to be most suitable because of the computation. But when we are
good to go with the gray scale image and our memory and computation requirement also changed and
we move down from Nvidia Tegra TK1 500 $ device to Raspberry Pi 30 $ device. Here is the
comparison between both devices.

Features Raspberry Pi 3B+ Nvidia Tegra TK1

CPU ARM Cortex A53 Cortex A 15

Single core Quad core

GPU Dual core Video core IV Multimedia NVIDIA Kepler GK20a

Co-Processor

Speed 1.4 MHz 2.4 GHz

USB 4 1/1

RAM 1GB 2GB DDR3L

Table 3: Raspberry Pi vs Nvidia Tegra TK1

Page 13
 3.6.1 Raspberry Pi
The Raspberry Pi is a credit-card-sized single-board computer developed in the UK by the
Raspberry Pi Foundation with the intention of promoting the teaching of basic computer science in
schools.

The secret sauce that makes this computer so small and powerful is the Broadcom BCM2837. The ARM
core hardware is a quad core ARM Cortex (ARMv8) cluster. The ARM cores are capable to run up to
1.4GHz and the Video Core IV runs at 400MHz. The ARM core is 64-bit, while the Video Core IV is
32-bit. The model of Raspberry Pi used for this project is 3B+.

 The biggest advantage of raspberry pi is its small size as credit card. it works as if a
normal computer at a relatively low price.
 One of other advantage of raspberry pi is usually speeding: problems can be solved in
‘hardware’ rather than in software, make use of parallelization.

Figure 7: Raspberry Pi

Page 14
 Specifications:
 A 1.4GHz 64-bit quad-core ARM Cortex-A53 CPU
 2.4GHz and 5GHz IEEE 802.11.b/g/n/ac wireless LAN, Bluetooth 4.2,
BLE
 Faster Ethernet (Gigabit Ethernet)
 Power-over-Ethernet support (with separate PoE HAT)
 Improved PXE network and USB mass-storage booting
 Improved thermal management
 1GB LPDDR2 SDRAM
 4 × USB 2.0 ports
 H.264, MPEG-4 decode (1080p30); H.264 encode (1080p30); OpenGL
ES 1.1, 2.0 graphics
 40-pin GPIO header

Raspberry Pi Model A+ Model B Model B+ 2, Model B 3, Model B 3, Model B

Chipset Broadcom BCM2835 Broadcom Broadcom Broadcom

BCM2836 BCM2837 BCM2837B0

64-bit 64-bit

Processor ARMv6 single core ARMv7 quad ARM Cortex- ARM Cortex-
core A53 single A53 single core
core

Processor 700 MHz 900 MHz 1.2 GHz 1.4 GHz

Speed

GPU Dual Core Video Core IV Multimedia Co-processor

USB 2.0 1 2 4

Wireless LAN None Integrated Integrated

802.11b/g/n 2.4GHz &
5GHz
802.11b/g/n/c

Table 4: Raspberry Pi Model Comparison

Page 15
3.7 Raspbian
The new Raspberry pi does not come with any operating system. Raspbian is the recommended
operating system for normal use, which is a Debian based operating system. Many other operating
system can also run on raspberry pi like RISC OS, LibreELEC, OSMC (Open Source Media Center),
Windows IOT Core and Arch Linux.
For this project Raspbian is installed in R-pi.

Installation of OS in Raspberry PI:

 The Raspberry Pi is not generally supplied with an operating system. To setup raspberry pi
download an iso disk image for the from https://www.raspberrypi.org/downloads/ click on
Raspbian and download the full version of Raspbian Stretch.
 You’ll also need a freshly formatted microSD card for raspberry pi. The format raspberry pi
uses is FAT32. For good performance you’ll need a SD card with high reading and writing
speed.
 After downloading the operating system, you will need a software to write the downloaded OS
on SD card, for this purpose Balena Etcher is used.
 Download Balena Etcher from https://www.balena.io/etcher/ and install it.
 Connect SD card to your PC with card reader.
 Open Balena Etcher and select from your hard drive the raspberry pi .img or .zip file you wish
to write on SD card.
 Select the SD card, Click on FLASH to begin writing data to the SD card. The process time
will depend on the speed of your SD card.
 After the Process will Complete eject SD card from computer and insert it in raspberry pi and
power it on.
 The first boot will take longer than usual. You will be shown raspi-config after booting into
Raspbian. To open the configuration tool after this, simply run the following from the command
line: sudo raspi-config
 The sudo is required because you will be changing files that you do not own as the Pi user. You
should see a blue screen with options in a grey box in the center, like so:
 Raspi-config aims to provide functionality to make most common configuration changes This
automatically edits to /boot/config/.txt

3.8 Installing OpenCV

For this purpose, you’ll need to write commands in terminal
sudo apt update
sudo apt-get upgrade
Above commands will upgrade dependencies and libraries that are pre-avaliable after the installation of
operating system in raspberry pi.

Now install some developer tools like CMAKE, which will help us configure the OpenCV build
process.

sudo apt-get install build-essential cmake pkg-config

Now, we need to install some image and video I/O packages that allow us to load various image and
video file formats from disk.

sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev

sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

Page 16
 sudo apt-get install libxvidcore-dev libx264-dev

Now we need to compile the highgui which is used to display images and build basic GUIs. module,
we need to install the GTK development library:

sudo apt-get install libgtk2.0-dev libgtk-3-dev

Many operations inside of OpenCV can be optimized further by installing a few extra dependencies:
sudo apt-get install libatlas-base-dev gfortran

These optimization libraries are especially important for resource constrained devices such as the
Raspberry Pi.

Now install both the Python 2.7 and Python 3 header files so we can compile OpenCV with Python
bindings:

sudo apt-get install python2.7-dev python3-dev

Download the OpenCV source code

Now that we have our dependencies installed, now download the 3.3.0 archive of OpenCV from the
official OpenCV repository.
cd ~
get -O opencv.zip https://github.com/Itseez/opencv/archive/3.3.0.zip
unzip opencv.zip

We’ll want the full install of OpenCV 3 to have access all features so we also need to grab the
opencv_contrib repository as well:

wget -O opencv_contrib.zip
https://github.com/Itseez/opencv_contrib/archive/3.3.0.zip
unzip opencv_contrib.zip

Python 2.7 or Python 3?

Before we can start compiling OpenCV on our Raspberry Pi, we first need to install pip , a
Python package manager:

we are using Python3 for our project that is why we are using downloading pip3.

wget https://bootstrap.pypa.io/get-pip.py
sudo python3 get-pip.py

Now we install NumPy a Python package used for numerical processing

pip3 install numpy
We are now ready to compile and install OpenCV now we can setup our build using CMake
Page 17
 cd ~/opencv-3.3.0/
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-
3.3.0/modules \
-D BUILD_EXAMPLES=ON ..
Before starting the compiling process, we should increase swap space size. This enables
OpenCV to compile with all four cores of the raspberry pi without memory problems.

CONF_SWAPSIZE=1024

After changing the swap size we need to start the raspberry pi

sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

Finally, we are now ready to compile OpenCV

make
sudo make install

verify that your OpenCV installated properly.

python3
>>> import cv2
>>> cv2.__version__

3.9 Libraries
You need to install all the libraries that you have install in anaconda environment and used them while
developing your model. This require you adopt the procedure again.

3.9.1 Tensor Flow

TensorFlow is an end-to-end open source platform for machine learning. It has a
comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers
push the state-of-the-art in ML and developers easily build and deploy ML powered applications.

sudo apt install libatlas-base-dev

pip3 install tensorflow

3.9.2 Keras
Keras is an open-source neural-network library written in Python. It is capable of running on
top of TensorFlow, Microsoft Cognitive Toolkit, Theano. Designed to enable fast experimentation with
deep neural networks,
Page 18
sudo pip3 install keras

3.9.3 Numpy
NumPy is the fundamental package for scientific computing with Python. It contains among
other things:

 a powerful N-dimensional array object

 sophisticated (broadcasting) functions

 tools for integrating C/C++ and Fortran code

 useful linear algebra, Fourier transform, and random number capabilities

Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container
of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily
integrate with a wide variety of databases.

sudo apt-get install python3-numpy

3.9.4 Matplotlib
Matplotlib is a Python 2D plotting library which produces publication quality figures in a
variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in
Python scripts, web application servers, and four graphical user interface toolkits.

sudo pip3 install matplotlib

3.9.5 Scikit Learn

Scikit-learn is a free software machine learning library for the Python programming language.
It features various classification, regression and clustering algorithms including support vector
machines and etc.

sudo pip3 install scikit-learn

3.9.6 SciPy
SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for
mathematics, science, and engineering

sudo apt-get install python3-scipy

Page 19
3.10 Raspberry Pi Camera
The Camera used for this project is a standard USB type camera.

 8 mega pixels
 Still image capture up to 640*480 pixels
 Automatic color and brightness adjustment
 Resolution 352*288(CIF)
First you need to enable Raspberry PI Camera using raspi-config go to Camera option and enable to
interface camera correctly.

3.11 Testing
Load the trained model save earlier and save it in Raspberry Pi card through USB or Mail the “DDD.h5”.
Now load it and perform preprocessing on it that you have perform while training like gray scale,
resizing etc.

Now capture the frame and pass that frame to classifier to get predictions.

Image Classifier Prediction

Acquisition

Figure 8: Test/Predict

Page 20
 Results

As discussed above that the model performs well but not quit well as compared to the results of
pretrained model. The comparison between the accuracy and losses was necessary to discuss here.
Overall accuracy achieved through this model was 80% .some results are shown here in figures.

Figure 7: Accuracy vs Epochs

Page 21
Figure 8: Losses vs Epochs

Page 22
 Future Recommendations
 Now there are three system that already exist
 Sleep Detector
 Eye Detector
 Distracted Driver Detection
 Integrate these systems to develop more satisfying system
 Increase data for eastern drivers

Page 23
 References
[1] Center for disease control and prevention. https: //www.cdc.gov/motorvehiclesafety/
distracted_driving/.

[2] National highway traffic safety administration traffic safety facts.

https://www.nhtsa.gov/risky-driving/ distracted-driving/.

[3] State farm distracted driver detection. https://www.kaggle.com/c/ state-farm-distracted-

driver-detection.

[4] https://elinux.org/Jetson_TK1

[5] https://www.wikipedia.org/

Page 24