International Journal of Computer Applications (0975 – 8887)

Volume 162 – No 8, March 2017

Study on Object Detection using Open CV - Python

Bhumika Gupta, PhD Ashish Chaube Ashish Negi Umang Goel
Assistant Professor, B.Tech IV Year B.Tech IV Year B.Tech IV Year
C.S.E.D G.B.P.E.C, Pauri G.B.P.E.C, Pauri G.B.P.E.C, Pauri
G.B.P.E.C, Pauri Uttarakhand, India Uttarakhand, India Uttarakhand, India
Uttarakhand, India

ABSTRACT it will be easier, faster and convenient to pluck the apples and
Object detection [9] is a well-known computer technology bananas rather than the manual plucking.
connected with computer vision and image processing that The standard performance measure that is commonly used for
focuses on detecting objects or its instances of a certain class the object category segmentation problem is called
(such as humans, flowers, animals) in digital images and Intersection-over-Union (IOU) [3]. Given an image, the
videos. There are various applications of object detection that similarities between the predicted region and the ground-truth
have been well researched including face detection, character region for an object present in the image can be found with
recognition, and vehicle calculator. Object detection can be the help of IOU measures and can be defined as the size of the
used for various purposes including retrieval and surveillance. intersection divided by the union of the different regions. For
In this study, various basic concepts used in object detection example, if any particular algorithm predicts each and every
while making use of OpenCV library of python 2.7, pixel of an image to be its background, the IOU measure can
improving the efficiency and accuracy of object detection are effectively penalize for that, as the intersection between the
presented. predicted and ground-truth regions would be zero, it will
produce an IOU count of zero.
Keywords
Object Detection, IOU, OpenCV, Python, Matlab. OpenCV library implemented in python2.7 along with the
help of Numpy is used and the world of object detection is
1. INTRODUCTION explored, a virtual Artificial Neural Network is created using
Object detection [9] and location in digital images has become Sci-kit tool.
one of the most important applications for industries to ease
user, save time and to achieve parallelism. This is not a new 2. CONCEPT
technique but improvement in object detection is still required 2.1 Theory
in order to achieve the targeted objective more efficiently and Every object class has its own special features that help in
accurately. classifying the object. Object recognition is that sub-domain
of computer vision which helps in identifying objects in an
The main aim of studying and researching computer vision is
image or video sequence. With more efficient algorithms,
to simulate the behavior and manner of human eyes directly
objects can even be recognized even when they are partially
by using a computer and later on develop a system that
obstructed from the direct view. Various approaches to this
reduces human efforts. Computer vision is such kind of
task have been implemented in the past years.
research field which tries to perceive and represents the 3D
information for world objects. Its main purpose is Various terms related to object detection are:
reconstructing the visual aspects of 3D objects after analyzing
the 2D information extracted. Real life 3D objects are 2.1.1 Edge matching
represented by 2D images.  Uses edge detection techniques to find the edges
The process of object detection analysis is to determine the  Effect of changes in lighting and color
number, location, size, position of the objects in the input
image. Object detection is the basic concept for tracking and  Count the number of overlapping edges.
recognition of objects, which affects the efficiency and 2.1.2 Divide and Conquer search
accuracy of object recognition. The common object detection
 All positions are to be considered as a set.
method is the color-based approach, detecting objects based
on their color values [4]. The method is used because of its  The lower bound is determined at best position in
strong adaptability and robustness, however, the detection the cell.
speed needs to be improved, because it requires testing all
possible windows by exhaustive search and has high  The cell is pruned if the bound is too large.
computational complexity.  The process stops when a cell becomes small
Object detection from a complex background is a challenging enough.
application in image processing. The goal of this project is to
identify objects placed over a surface from a complex 2.1.3 Grayscale matching
background image using various techniques. The detection of  Edges give a lot of information being robust to
the objects can be extended using automation and robotics for illumination changes.
plucking of the objects like apples, bananas from the
corresponding tree using the image processing techniques and

17
 International Journal of Computer Applications (0975 – 8887)
Volume 162 – No 8, March 2017

 Pixel distance is computed as a function of both Arbitrary data types can be defined. This allows NumPy to
pixel intensity and position. The same thing can seamlessly and speedily integrate with a wide variety of
compute with color too. databases.

2.1.4 Gradient matching NumPy is licensed under the BSD license, enabling reuse with
 Comparing image gradients can also be helpful in few restrictions.
making it robust to illumination changes.
2.4 Object Classification In Moving Object
 Matching is performed like matching greyscale Detection
images. Object classification approach is based on shape, motion,
The ease of dealing with the image is that it is made up of color and texture. The classification can be done under
pixels, so in most cases, the location of next point is easily various classes such as trees, animals, humans, objects etc.
found and can be connected with our current pixel anytime. Tracking objects and analyzing their features is a key concept
Consider the following example for calculating Euclidean of object classification.
distance between center of the circle and the connected points.
2.4.1 Shape Based
Take an image consisting a circle, convert it to a grayscale
A mixture of image-based and scene-based object parameters
image, detect edges, move along edges, and draw normal
such as image blob (binary large object) area, the aspect ratio
which will intersect at center. Now repeat this process for
of blob bounding box and camera zoom is given as input to
entire circle or find connected edges and then
this detection system. Classification is performed on the basis
calculate Euclidean distance between center of the circle and
of the blob at each and every frame. The results are kept in the
the connected points.
histogram.
2.2 Open CV 2.4.2 Motion Based
OpenCV (Open Source Computer Vision) is an open source When a simple image is given as an input with no objects in
computer vision and machine learning software library [10]. motion, this classification is not needed. In general, non-rigid
OpenCV was initially built to provide a common articulated human motion shows a periodic property, hence
infrastructure for applications related to computer vision and this has been used as a strong clue for classification of moving
to increase the use of machine perception in the commercial objects. Based on this useful clue, human motion can be
products. As it is a BSD-licensed product so it becomes easy distinguished from other objects motion.
for businesses to utilize and modify the existing code in
OpenCV. 2.4.3 Color Based
Around 3000 algorithms are currently embedded inside Though color is not an appropriate measure alone for
OpenCV library, all these algorithms being efficiently detecting and tracking objects, but the low computational cost
optimized. It supports real-time vision applications. These of the color based algorithms makes the color a very good
algorithms are categorized under classic algorithms, state of feature to be exploited. For example, the color-histogram-
art computer vision algorithms and machine learning based technique is used for detection of vehicles in real-time.
algorithms. These algorithms are easily implemented in Java, Color histogram describes the color distribution in a given
MATLAB, Python, C, C++ etc. and are well supported by region, which is robust against partial occlusions.
operating system like Window, Mac OS, Linux and Android. 2.4.4 Texture-Based
A full-featured CUDA and OpenCL interfaces are being The texture-based approaches with the help of texture pattern
actively developed for the betterment of technology. There are recognition work similar to motion-based approaches. It
more than 500 different algorithms and even more such provides better accuracy, by using overlapping local contrast
functions that compose or support those algorithms. OpenCV normalization but may require more time, which can be
is written natively in C++ and has a templated interface that improved using some fast techniques.
works seamlessly with STL containers.
3. STEPS INVOLVED IN OBJECT
For OpenCV to work efficiently with python 2.7 we need to
install NumPy package first.
DETECTION IN PYTHON 2.7
Let us start with an image (im.jpg) and detect various objects
2.3 NumPy in it.
NumPy is the fundamental package for scientific computing
with Python [11].It can be treated as an extension of the
3.1 Install OpenCV-Python
Below Python packages are to be downloaded and installed to
Python programming language with support for
their default location - Python-2.7.x, NumPy and Matplotlib.
multidimensional matrices and arrays. It is open source
Install all packages into their default locations. Python will be
software with many contributors. It contains among other
installed to C/Python27/. Open Python IDLE. Enter import
things:
NumPy and make sure NumPy is working fine. Download
• A powerful N-dimensional array object. OpenCV from Sourceforge. Go to OpenCV/build/python/2.7
folder. Copy cv2.pyd to C:/Python27/lib/site-packages.
• Broadcasting functions.
• Tools for integrating C/C++ and FORTRAN code.
• Useful linear algebra, Fourier transform, and
random number capabilities.
Besides its obvious scientific uses, NumPy can also be used as
an efficient multi-dimensional container of generic data.

18
 International Journal of Computer Applications (0975 – 8887)
Volume 162 – No 8, March 2017

3.2 Read an Image  Free and open.

Use the function CV2.imread() to read an image. The image  Multiple functions can be packaged in one module.
should be in the current working directory otherwise, we need
to specify the full path of the image as the first argument. The  More choices in graphics packages and toolsets.
second argument is a flag which specifies the way image
should be read. 5. VARIOUS OBJECT DETECTION
1. CV2.IMREAD_COLOR: This function is used to load a
ALGORITHMS IMPLEMENTED IN
color image. Transparency of image, if present will be PYTHON
neglected. It is the default flag. 5.1 Haar-like features
2. CV2.IMREAD_GRAYSCALE: Loads image in grayscale It is an effective object detection technique which is proposed
mode by Paul Viola and Michael Jones in 2001. It is a machine
learning based method for object detection where we train a
3. CV2.IMREAD_UNCHANGED: Loads image as such classifier from a lot of images. This classifier is then used in
including alpha channel. detecting objects in an image.

3.3 Feature detection and description Initially, the algorithm needs images with faces (positive
 Understanding features (What are the main features images) and images without faces (negative images) to train a
in an image? How can finding those features be classifier and then extract features from this classifier.
useful to us?) This method introduces a concept cascade of the classifier.
 Corner detection (Okay, Corners are good features? Instead of applying all the features at once we group the
But how do we find them) features into different stages of the classifier and apply one by
 Feature matching (We know a great deal about one. Discard the window if it fails in the first stage. If it
feature detectors and descriptors. So let us now passes the stage then continue the process. The window which
learn to match different descriptors. OpenCV passes through all the stages will be our desired region.
provides two techniques, Brute-Force matcher, and
FLANN based matcher.)
 Homography (As we are aware of feature matching,
so let us now blend it with Camera calibration and
3D reconstruction (calib3d module) to find objects
for description in a complex image.)

Fig:2 Face Detection using Haar Cascade

Fig: 1 Flow Chart for Object Detection 5.2 Circular Hough Transformation
Hough transformation was invented by Richard Duda and
4. PYTHON VS OTHER LANGUAGES Peter Hart in 1992, this transformation was initially meant to
detect arbitrary shapes from an image. It was later modified to
FOR OBJECT DETECTION detect circular objects in low-contrast noisy images and
Object detection is a domain-specific variation of the machine
referred as Circular Hough Transformation [6].
learning prediction problem.
CHT relies on equations [6] for circles:
Intel‟s OpenCV library that is implemented in C/C++ has its
interfaces available in a number of programming environment r2 = (x-a)2 +(x-b)2
such as C#, Matlab, Octave, R, Python etc.
where a and b are the coordinate of the center, and r is the
Some of the benefits of using Python codes over other radius of the circle.
language codes for object detection are
CHT relies on three parameters, which require larger
 More compact and readable code. computation time and memory and it increases the complexity
to extract information from the image.
 Python uses zero-based indexing.
For simplicity, CHT programs are provided with a constant
 Dictionary (hashes) support is offered. value of radius or provided with a range of radius prior to
 Simple and elegant Object-oriented programming. running the application.

19
 International Journal of Computer Applications (0975 – 8887)
Volume 162 – No 8, March 2017

5.3 Template matching 6. APPLICATIONS AND FUTURE

Template matching [1, 2] & [5] is a high-level machine vision SCOPE
technique to detect objects from an image that matches a Computer vision is still a developing discipline, it has not
given image pattern. This technique matches the source image been matured to that level where it can be applied directly to
with the template image or patch. real life problems.
If the template image has strong features, the feature-based After few years‟ computer vision and particularly the object
approach may be used otherwise template based approach is detection would not be any more futuristic and will be
used. ubiquitous. For now, we can consider object detection as a
sub-branch of machine learning.
5.4 Blob detection
This method is used to detect regions in an image that differs Some common and widely used application of object
in properties. A blob is a region in the image in which all the detection are:
points can be considered to be similar to each other. There are
two classes of blob detection method: differential method and 6.1 Face Detection
local extrema method. Have you ever wondered how Facebook detects your face
when you upload a photo? Not only it detects, it remembers
5.5 The Gradient-based method the face too. This is a simple application of object detection
The gradient-based method uses spatial and temporal partial that we see in our daily life.
derivatives to estimate image flow at every position in the
image. If the motion is not known in advance to be restricted 6.2 Counting objects/peoples
to a small range of possible values then a multi- scale analysis Object detection can be also used for counting purpose, it is
must be applied so that the scale of the smoothing prior to used for keeping a count of particular or all objects in an
derivatives estimation is appropriate to the scale of the image or a frame. For e.g. from a group photograph it can
motion. This can make this method computationally count the number of persons and if implemented smartly you
expensive. may also find out different people with different dresses.

5.6 Local Binary Pattern 6.3 Vehicle detection

LBP is a visual descriptor proposed in 1990. It is a powerful Similarly, when the object is a vehicle, object detection along
method for texture classification. with tracking can be used for finding the type of vehicle, this
application may be extended to even make a traffic calculator.
The LBP feature vector is created in the following manner
Divide window into cells. 6.4 Industries
 For each pixel in the cell, compare it with its 8 Object detection is also used in industrial processes for the
adjacent pixels. identification of different products. Say you want your
machine to only detect objects of a particular shape, you can
 If the value of the pixel is greater than its neighbor, achieve it very easily. For e.g. Hough circle detection
write “0‟‟, otherwise “1”. This gives an 8 digit transform [6] can be used for detecting circular objects.
binary number which is converted to decimal for
convenience 6.5 Security
Identification of unwanted or suspicious objects in any
 Aggregate histogram of all the cells is calculated particular area or more specifically object detection
and compared. techniques are used for detecting bombs/explosives. It is also
 Normalize the histogram. Concatenate histogram of even used for personal security purpose.
all cells. This gives the feature vector of the entire
window.
6.6 Biometric recognition
Biometric recognition uses physical or behavioral traits of
This vector now can be used to classify the images humans to recognize any individuals for security and
authentication purpose [1]. It uses distinct biological traits like
5.7 Bag-of-words method fingerprints, hand geometry, retina and iris patterns etc.
This method can be applied to image classification. Images
are treated like visual words. These visual words are 6.7 Surveillance
important points in the images. These points are called Objects can be recognized and tracked in videos for security
features. This method can be used for image classification by purpose. Object recognition is required so that the suspected
creating a large vocabulary of many visual words and person or vehicle can be tracked [1].
representing each histogram of the frequency words that are in
the image. 6.8 Medical analysis
Object detection is used to detect diseases like a tumor,
5.8 Deep Face method stones, cancer in MRI images [1].
Facebook AI research group has developed Deep Face
Software in Menlo Park, California by the support of an 6.9 Optical character recognition
advanced deep learning neural network. A piece of software Characters in scanned documents can be recognized using
that simulates an approximation of how real neurons works is object recognition [1].
called a Neural Network. Deep Face Learning can be
performed by Machine Learning. It can be defined as a huge 6.10 Human computer interaction
body of data that develops a high-level abstraction by looking Human gestures can be stored in the system and can be used
for recurring faces. for recognition in a dynamic environment by computers to
interact with humans [1].

20
 International Journal of Computer Applications (0975 – 8887)
Volume 162 – No 8, March 2017

Object detection‟s scope is not yet limited here. You can use it 9. REFERENCES
for any purpose you can think of. For e.g. for solving number [1] Khushboo Khurana and Reetu Awasthi,“Techniques for
puzzles by just giving their images as input and applying
Object Recognition in Images and Multi-Object
some proper algorithms after detecting different numbers and
Detection”,(IJARCET), ISSN:2278-1323,4th, April 2013.
their places from the input image.
[2] Latharani T.R., M.Z. Kurian, Chidananda Murthy
7. CHALLENGES M.V,“Various Object Recognition Techniques for
The main purpose is to recognize a specific object in real time Computer Vision”, Journal of Analysis and Computation,
from a large number of objects. Most recognition systems [2] ISSN: 0973-2861.
are poorly scalable with many recognizable objects.
Computational cost rises as the number of objects increases. [3] Md Atiqur Rahman and Yang Wang, “Optimizing
Comparing and querying images using color, texture, and Intersection-Over-Union in Deep Neural Networks for
shape are not enough because two objects might have same Image Segmentation,” in Object detection, Department
attributes. Designing a recognition system with abilities to of Computer Science, University of Manitoba, Canada,
work in the dynamic environment and behave like a human is 2015.
difficult. Some main challenges to design object recognition [4] Nidhi, “Image Processing and Object Detection”, Dept. of
system are lighting, dynamic background, the presence of Computer Applications, NIT, Kurukshetra, Haryana,
shadow, the motion of the camera, the speed of the moving 1(9): 396-399, 2015.
objects, and intermittent object motion weather conditions etc.
[5] R. Hussin, M. Rizon Juhari, Ng Wei Kang, R.C.Ismail,
8. CONCLUSION A.Kamarudin, “Digital Image Processing Techniques for
The possibilities of using computer vision to solve real world Object Detection from Complex Background
problems are immense. The basics of object detection along Image,”Perlis, Malaysia: School of Microelectronic
with various ways of achieving it and its scope has been Engineering, University Malaysia Perlis, 2012.
discussed. Python has been preferred over MATLAB for
[6] S.Bindu, S.Prudhvi, G.Hemalatha, Mr.N.Raja Sekhar, Mr.
integrating with OpenCV because when a Matlab program is
V.Nanchariah,“Object Detection from Complex
run on a computer, it gets busy trying to interpret all that
Background Image using Circular Hough Transform”,
Matlab code as Matlab code is built on Java. OpenCV is
IJERA, ISSN: 2248-9622, Vol. 4, Issue 4(Version 1),
basically a library of functions written in C\C++.
April 2014, pp.23-28.
Additionally, OpenCV is easier to use for someone with little
programming background. So, it is better to start researching [7] Shaikh, S.H; Saeed, K, and Chaki.N,“Moving Object
on any concept of object detection using OpenCV-Python. Detection Using Background Subtraction” Springer,
ISBN:978-3-319-07385-9.
Feature understanding and matching are the major steps in
object detection and should be performed well and with high [8] Shijian Tang and Ye Yuan,“Object Detection based on
accuracy. Deep Face is the most effective face detection Conventional Neural Network”.
method that is preferred over Haar-Cascade by most of the
social applications like facebook, snap chat, Instagram etc. [9] (2017, January 17). Object Detection [Online].Available:
http://en.m.wikipedia.org/wiki/Object_detection
In the coming days, OpenCV will be immensely popular
among the coders and will also be the prime requirement of IT [10] Opencv.org, „About OpenCV‟, 2017. [Online].
companies. To improve the performance of object detection Available:http://www.opencv.org/about
IOU measures are used. [11]Numpy.org, 2017. [Online]. Available:
http://www.numpy.org

IJCATM : www.ijcaonline.org
21