REAL TIME OBJECT RECOGNITION AND

CLASSIFICATION USING TENSORFLOW

 Prof. Prerana Chaithra,
Ambarish Parthasarthy, Ankita,
Associate Professor,
Dept. of Information Science and UG Student, UG Student,
Engineering, Dept. of Information Science and Dept. of Information Science and
Engineering, Engineering,
Sapthagiri College of Engineering. Sapthagiri College of Engineering Sapthagiri College of Engineering
Ankita, Adarsh Kumar Humney,
UG Student, UG Student,
Dept. of Information Science and Dept. of Information Science and
Engineering, Engineering,
Sapthagiri College of Engineering Sapthagiri College of Engineering
 Abstract— corners and have equal side classification of the type of the selected study the process
Efficient and accurate object lengths are needed. class that we want. of object detection and
detection has been an Fig 1(A): choosing the best among the
important topic in the An image Convolutional Neural Network neural networks i.e.
advancement of computer recognition algorithm convolutional neural network
vision systems. With the (acknowledged as an image and why it suit the need to be
advent of deep learning classifier) takes an image (or a selected. The proposed work
techniques, the accuracy for patch of an image) as input II. Literature Survey
object detection has and outputs what the image
increased drastically. The contains. In other words, the
project aims to incorporate output is a class label (e.g.
state-of-the-art technique for “cat”, “dog”, “table” etc.).
object detection and species How does an image
classification with the goal of recognition algorithm know
achieving high accuracy with the contents of an image?
a real-time performance. A Well, you have to train the
major challenge in many of algorithm to learn the
the object detection systems differences between different
is the dependency on other classes. If you want to find
computer vision techniques cats in images, you need to
for helping the deep learning train an image recognition
based approach, which leads algorithm with thousands of
to slow and non-optimal images of cats and thousands
performance. In this project, of images of backgrounds that
we use a completely deep do not contain cats. Needless
learning based approach to to say, this algorithm can only
solve the problem of object understand objects / classes it
detection in an end-to-end has learned.
fashion. Pre trained model
of Keras is used, mostly on To simplify we have
the most challenging publicly used pre trained models of presents the methods of
available dataset (Microsoft Keras. The pre-trained models A survey on the techniques teaching the model to process
COCO), Iam Dataset, include may include models that can be used to design a the features of the object and
Flower17 dataset. The such as vgg16, inceptionv3
resulting system is fast and which are designed by the
accurate, thus aiding those Google which helps not to
applications which require make a custom model from the
object detection. Feature- scratch and the pre trained
Based Methods include model are capable of detecting
search to find feasible any type of object detection
matches between object algorithm may it be. The pre-
features and image features. trained model are used here
The primary constraint is not only for object detection
that a single position of the but also for the detection of
object must account for all of flowers and dog breed.
the feasible matches and
methods that extract features This approach can
from the objects to be be used for the detection and
recognized and the images to classification of the object
be searched. present in the frame of
reference. With this approach
Keywords— CNN, BOW, in a frame we would be able to
Keras, TensorFlow, COCO, identify the type of the object
Resnet50, Imagenet, present and would be able to
computer vision. recognise the types of species
(animal, plant) taken in the
context.
I. INTRODUCTION
Every object class There has been a lot to recognise the features of
has its own of work in object detection model to recognise object is that particular object. It
special features that helps in using traditional computer made. The different design discusses about the usage of
classifying the class – for vision techniques. However, techniques of eight papers that RELU function at the first
example all circles are round. they lack the accuracy of deep can be used to design the where the RELU function or
Object class detection uses learning based techniques. model are compared. It also the rectified linear unit is a
these special features. For Among the deep learning covers the significant process where the function
example, when looking for based techniques, CNN is improvements in Object converts the maxpool outcome
circles, objects that are at a being used here. CNN or the Detection Model with the use which are complicated and not
particular distance from a convolutional neural network of different branch of artificial linear function into linear
point (i.e. the centre) are is the branch of artificial neural network is done. In function so that the calculation
sought. Similarly, when neural network which has addition, it discusses about the made can be less. In addition,
looking for squares, objects denser and deep inner network similarity and differences in it also talks about the
that are perpendicular at which helps in the the techniques that are used in challenges of implementing
the max pooling function features are being extracted recognise which breed does
which is one of the main from the pixels and the the particular dog belongs to.
features of the convolutional features that are relevant are A. Object Recognition The output gives a list of dogs
neural network. Max Pooling saved to identify the class of
is basically a function where the object that is being used as
the images are converted into the input image from the user.
smaller matrix forms, such that After the feature extraction the
each matrix is divided into 4x4 recognition of the image takes
matrix which contains the place with the help of the
binary value or the pixel value neural network build and from
of the particular class of the inputs that neural network
image, from the matrix the acquired from the along
object which has the highest processes and after the
pixel is selected to be the recognition the output is
choosing element of the class generated as the class the
or is selected as a feature system thinks or predict the
extraction process which helps input image belongs to.
to distinguish the particular
class of the object. Fig in which the breed with the
highest probability is placed
4(a): Identifying object first followed by the breed
III. names that can resemble the

Proposed System In
Fig3 (a): Use-Case diagram
this module
the process
of object
detection
takes place.
An input
image is
provided
from the
user and the
system
undergoes processes to particular dog from the pic. It
identify the image that is being also follows with the usage of
provided by the user. Here the the library from the keras and
use of pre-trained model of the tensorflow.
keras and tensorflow is used
where the tensorflow is the
Fig 2(a): System library mainly used in the C. Flower Recognition
Architecture The Use-Case machine learning in the object
diagram represent the work of
The Proposed the system and the user. Here
System is as follows, first the the User is provided with the
user interacts with the system following set of task where the
and the user in the interaction user has either to upload and
transfers the image which it image or to cancel the image
wants to know about. After the that the user has uploaded. The
interaction with the user the system on the other hand has
system then starts the pre- more task than the user, the
processing of the image where system initialize the image,
in the process of pre- and it has the function to pre-
processing three different process the image.
process takes place. First the Segmentation is the next task detection process. Same way
image is converted to which in itself include three the keras is used, it provides Fig 4(c): Flower
grayscale, from grayscale it is processes which is grayscale with different models and with Classification
then converted to binary form conversion, binary conversion the help of models the object
and from that it is then send to and normalization. Then the detection process is possible. This model is based
the process of normalisation of system has segmentation and on the pre-trained model build
the object sent to the system. recognition as the next set of B. Dog Breed classification by the application model
After this the system then start process and at the last the provided by the keras library.
the process of segmentation system then generates the Fig 4(b): The model consist of vgg16,
where the image is cut into output from the input it rasnet50 among others. The
Dog breed Classification
smaller units of pixel and the collected from the given set of model does not have to build
most occurring and the pixel In this module the by the user nor must the layers
process.
with the value that matches the system tries to identidy the that has to use be build by the
features of the different classes species of the dog that is being user. In this model the image
of the object is kept for the provided by the user. The of flower is being provided by
latter process. The feature system takes the input from the user and the system tries to
extraction is next where the IV. Module Description the user and then it tries to predict and classify the species
of the flower that the user has etc. or Local descriptors CNN could be used as a convolution
provided to the system. such as SIFT, SURF, ORB Feature Extractor. (25*25)
etc. These are hand-crafted Convolution:
features thatrequires domain 3 filters 3*3
level expertise. ReLU function
Output:
But here comes 3 tensors (25*25)
Convolutional Neural
Networks (CNN)! Instead of
 Pooling
using hand-crafted features,
Deep Neural Nets Input:
automatically learns these 3 tensors
features from images in a from previous
D. Handwritten Recognition
convolution
B. Keras Pre-Trained Model (25*25)
Pooling:
1 filter 4*4
Fig 5(a): Keras Pre-Trained (max function)
Model Output:
3 tensors
(6*6)
C. Dependencies

The following Python  Fully Connected

hierarchical fashion. Lower packages are required: Input:
Fig 4(d): Handwritten
layers learn low-level  TensorFlow 3 tensors
Classification
features such as Corners,  Keras from previous
In this module the system tries Edges whereas middle layers  NumPy
to identify the handwritten learn color, shape etc. and  Scikit-learn convolution (6*6)
document provided by the higher layers learn high-  Matplotlib Fully connected
user. The system tries to level features representing  Seaborn layer:
predict the words or the the object in the image.  H5py 6*6*3
numerals which ever the Instead of making a CNN as inputs
system finds and tries to give a model to classify images, D. Algorithm Output:
the output. Here the Iam we used it as a Feature 4 neurons
Dataset is used to train the Extractor by taking the  1st Convolution
model. Tensorflow Library is activations available before Input:
used in the process of making the last fully connected layer 3 channel E. Simple Steps for Deep
the model run. of the network image (25*25) Learning
(i.e. before the final softmax Convolution:
V. Implementation classifier). These activations 2 filters  Prepare the training
For the module of Flower will be acting as the feature 5*5 dataset with flower
Classification we will be vector for a machine ReLU images and its
using the pre-trained Deep learning model (classifier) function corresponding
Neural Nets trained on the which further learns to Output: labels.
ImageNet challenge that are classify it. This type of 2 tensors  Specify your own
made publicly available in approach is well suited for (25*25) configurations
Keras. We will specifically Image Classification in conf.json file.
use FLOWERS17 dataset problems, where instead of  2nd Convolution  Extract and store
from the University of training a CNN from scratch 2 tensors from features from the last
Oxford. The pre-trained (which is time-consuming previous fully connected
models we will consider are and tedious), a pre-trained layers (or
VGG16, VGG19,Inception- intermediate layers)
v3, Xception, ResNet50, of a pre-trained
InceptionResNetv2 and Deep Neural Net
MobileNet. Instead of (CNN)
creating and training deep using extract_feature
neural nets from scratch s.py.
(which takes time and  Train a Machine
involves many iterations). Learning model such
as Logisitic
A. Feature Extraction using Regression using
ConvNets these CNN extracted
features and labels
Traditional machine learning using train.py.
approach uses feature  Evaluate the trained
extraction for images using model on unseen
Global feature descriptors data and make
such as Local Binary further optimizations
Patterns (LBP), Histogram if necessary.
of Oriented Gradients
(HoG), Color Histograms
E. Result (ICCE – Asia),
Pages: 1 – 4.

VII. References

[1] Malay Shah,

Professor Rupal
Kapdi Features for
Object Detection
using deep neural
network and about
conventional neural
network,
International
Conference on
Intelligent
Computing and
control System
(ICICCS), 2017 pp
787 – 790.

[2] Lihong Yang, Liewei

Wang, Shuo Wu
features about real-
time object
Recognition based
on deep
convolutional neural
network in 2018

[3] Guobin Chen, Tony

X. Han, Zhihai He,
Roland Kays, and
Tavis Forrester
featured about Deep
Convolutional
Neural Network
based species
recognition for wild
animals monitoring.
In 2014 IEEE
International
Conference on
Image Processing
(ICIP), pages: 858 –
862.

[4] Huieun Kim Huien

Kim, Youngwan
Lee, Byeounghak
Yim, Eunsoo Park,
Hakil Kim for on-
road object detection
using deep neural
network.
In 2016 IEEE
International
Conference on
Consumer
Electronics – Asia