OBJECT TRACKING USING IMAGE PROCESSING

A Mini-Project II Report

Submitted in Partial Fulfillment of the Requirements

for the Degree of

BACHELOR OF TECHNOLOGY
IN
ELECTRONICS & COMMUNICATION ENGINEERING
By
Puneet Kaushik (15BEC089)
Ravi Kapadiya (15BEC096)
Under the Guidance of
Dr. Niranjan Devashrayee

Department of Electronics & Communication Engineering

We would like to express our deep sense of gratitude to our Supervisor, Dr.
Niranjan Devashrayee Professor, Electronics and Communication Engineering
Department for his valuable guidance and motivation throughout our study.

We would like to express our sincere respect and profound gratitude to Prof. (Dr.)
D.K. Kothari Professor & Head of Electronics and Communication Engineering
Department for supporting and providing the facilities for our project work.

We would also like to thanks all our friends who have helped us directly or
indirectly for the completion of our project.

No words are adequate to express our indebtedness to our parents and for their
blessings and good wishes. To them we bow in the deepest reverence.

- Puneet Kaushik (15BEC089)

i
 CERTIFICATE
This is to certify that the mini-project II entitled “OBJECT TRACKING USING
IMAGE PROCESSING” submitted by Puneet Kaushik (Roll No. 15BEC089) &
Ravi Kapadiya (Roll No. 15BEC096) as the partial fulfillment of the
requirements for the award of the degree of Bachelor of Technology in Electronics
& Communication Engineering, Institute of Technology, Nirma University is the
record of work carried out by them under my supervision and guidance. The work
submitted in our opinion has reached a level required for being accepted for the
examination.

Date:

MAY 3, 2018

Dr. Niranjan Devashrayee Prof. (Dr.) D.K. Kothari

Professor, Prof. & Head of Department,

Department of EC Engineering, Department of EC Engineering,

Institute of Technology, Institute of Technology,

Nirma University Nirma University

ii
 ABSTRACT
We know that 21st century is a century of information and technology. In this
modern age we all are connected with digital technology one or other way. As a
step towards a perceptual user interface, an object tracking algorithm is developed
and demonstrated tracking human faces. Computer vision algorithms that are
intended to form part of a perceptual user interface must be fast and efficient. They
must be able to track in real time and yet not absorb a major share of
computational resources. An efficient, new algorithm is described here based on
the mean shift algorithm. The mean shift algorithm robustly finds the mode (peak)
of probability distributions. In our case, we want to track the mode of an object's
probability distribution within a video scene. Since the probability distribution of
the object can change and move dynamically in time, the mean shift algorithm is
modified to deal with dynamically changing probability distributions.

iii
 LIST OF FIGURE
FIGURE FIGURE TITEL PAGE
No. No.

3.1 Basics details of Raspberry Pi 3 Model B+………………. 3

3.2 Raspberry Pi with Pi camera ………………..……………. 4

4.1 Quadrant algorithm …………………………………..…... 6

4.2 Robot movement ………………………….……………… 6

4.3 Simulation result ………………………….……………… 9

iv
 INDEX
CHAPTER PAGE
TITEL
No. No.
i
Acknowledgement………………………………………………...
ii
Certificate …………………………………………………………
iii
Abstract……………………………………………………………
iv
List of Figures…………………………………………………….

1 Introduction………………………………………....................... 1

2 About OpenCV…………………………….……………………... 2

3 Raspberry PI 3 (Model B+)…………………….……………….. 3

4 Object detection & tracking using Raspberry Pi…………….… 5

4.1 Introduction…………………………………………………... 5

4.2 Block Diagram……………………………………………….. 5

4.3 Quadrant based algorithm……………………………………. 6

4.4 Code………………………………………………………….. 7

4.5 Simulation Result…………………..………………………… 9

Conclusion………………………………………………………… 10

References…………………………………………….................. 11

v
CHAPTER 1
INTRODUCTION
 Digital image processing is the use of computer algorithms to perform image
processing on digital images.
 As a subcategory or field of digital signal processing, digital image processing has many
advantages over analog image processing.
 It allows a much wider range of algorithms to be applied to the input data and can avoid
problems such as the build-up of noise and signal distortion during processing. Since
images are defined over two dimensions (perhaps more) digital image processing may be
modeled in the form of multidimensional systems.
 Digital image processing allows the use of much more complex algorithms, and hence,
can offer both more sophisticated performance at simple tasks, and the implementation of
methods which would be impossible by analog means.
 Digital image processing is the process of performing some mathematical functions and
operations on an image, images or video. The output that we get after image processing is
a set of parameters or some altered image, images or videos.

1
CHAPTER 2
ABOUT OPEN CV
OpenCV (Open Source Computer Vision Library) is associate degree open source
computer vision and machine learning software package library. OpenCV was designed to
supply a standard infrastructure for computer vision applications and to accelerate the
utilization of machine perception within the business product. Being a BSD-licensed product,
OpenCV makes it straightforward for businesses to utilize and modify the code.

It has C++, Python, Java and MATLAB interfaces and supports Windows,
Linux, Android and Mac OS. OpenCV leans mostly towards real-time vision applications
and takes advantage of MMX and SSE instructions when available.

The library has more than 2500 optimized algorithms, which includes a comprehensive set of
both classic and state-of-the-art computer vision and machine learning algorithms. These
algorithms can be used to detect and recognize faces, identify objects, classify human actions
in videos, track camera movements, track moving objects, extract 3D models of objects,
produce 3D point clouds from stereo cameras, stitch images together to produce a high
resolution image of an entire scene, find similar images from an image database, remove red
eyes from images taken using flash, follow eye movements, recognize scenery and establish
markers to overlay it with augmented reality, etc.

2
CHAPTER 3
RISPBERRY PI 3 (MODEL B+)
3.1 INTRODUCTION
 Taking into account the relatively high performance requirements of image processing in
general and the equipment currently available to the faculty, as a relatively inexpensive and
powerful embedded platform the Raspberry Pi was an obvious choice. The hardware
specifications taken into consideration for this work can be seen in Table 1.

Courtesy: https://www.techrepublic.com
Fig:3.1 Basics details of Raspberry Pi 3 Model B+

3
 Another contributing factor to this choice was the availability of the Pi camera module, which can
capture high-definition video as well as stills and requires the user to simply update Raspberry’s
firmware to the latest version. It can easily be used in OpenCV based applications. Although using
the officially supported camera module, which can be accessed through the MMAL and V4L APIs and
is supported by numerous third-party libraries, any USB web-camera can be used.

Courtesy: https://www.aliexpress.com
Fig: 3.2 Raspberry Pi with Pi camera

4
CHAPTER 4
OBJECT DETECTION & TRACKING USING RISPBERRY PI
4.1 INTRODUCTION

The project mainly focuses on the basis to implement the object detection and tracking based
on its color, which is a visual based project i.e. the input to the project will be the
video/image data which is continuously captured with the help of a webcam which is
interfaced to the Raspberry Pi. It will detect the object and it tracks that object by moving the
camera in the direction of the detected object. First of all the Linux O.S is installed into the
Raspberry Pi board via Micro SD card and appropriate code is written in Python language for
the object detection using the Open CV libraries and is dumped in the board. The visual data
captured by the webcam is processed in the Raspberry Pi and the object is detected based on
the color and once if the object is detected, the servo motor is rotated in such a way that
wherever the object moves, the camera will be pointing to that object. We can control the
angle of servo rotations by the Arduino board through its PWM pins i.e., by varying the pulse
widths.

4.2 BLOCK DIAGRAM

5
4.3 QUADRANT BASED ALGORITHM
Step 1: Capture live video.

Step 2: Pre-process the frames. This step includes converting of the frames to gray scale
and also removing the noise from the frames.
Step 3: Further converting the frames to black and white frames.

Courtesy: http://ijettjournal.org
Fig: 4.1 Quadrant algorithm

Step 4: Black and white frames are then virtually divided into grids as shown in the figure
4.1
Step 5: Object is identified by finding out its contour and calculate its centroid.
Step 6: The main idea is to ensure that the object lies in the 7th quadrant. When the object
moves away from 7th quadrant, suitable signals are sent to the robot to take action
accordingly. Robot movement is given according to the direction shown in the figure 4.2.
The arrows depict the movement of the robot based on the position of the object.

Courtesy: http://ijettjournal.org
Fig:4.2 Robot movement

6
4.4 CODE
import cv2

import numpy as np

import argparse

import imutils

from collections import deque

ap = argparse.ArgumentParser()

ap.add_argument("-b", "--buffer", type=int, default=64,

help="max buffer size")

args = vars(ap.parse_args())

#def image_processing (color):

#print ("deciding color...............")

#if color == "blue":

#lower = (86,31,4)

#upper = (220,88,50)

#if color == "green":

lower = (29,86,6)

upper = (64,255,255)

#if color == "red":

#lower = (17,15,100)

#upper = (50,56,200)

#print (color)

#greenLower = (29, 86, 6)

#greenUpper = (64, 255, 255)

pts = deque(maxlen=args["buffer"])

7
camera = cv2.VideoCapture(0)

while True:

(grabbed, frame) = camera.read()

frame = imutils.resize(frame, width=300)

hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

cv2.imshow('image',hsv)

mask = cv2.inRange(hsv,lower,upper)

cv2.imshow('image1',mask)

kernel = np.ones((5,5),np.uint8)

mask = cv2.erode(mask, kernel, iterations=2)

cv2.imshow('image2',mask)

kernel = np.ones((5,5),np.uint8)

mask = cv2.dilate(mask, kernel, iterations=2)

cv2.imshow('image3',mask)

cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL,

cv2.CHAIN_APPROX_SIMPLE)[-2]

print(cnts)

center = None

if len(cnts) > 0:

c = max(cnts, key=cv2.contourArea)

((x, y), radius) = cv2.minEnclosingCircle(c)

M = cv2.moments(c)

center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"]))

if radius > 10:

cv2.circle(frame, (int(x), int(y)), int(radius),(0, 255, 255), 2)

8
 cv2.circle(frame, center, 5, (0, 0, 255), -1)

pts.appendleft(center)

cv2.imshow("Frame", frame)

key = cv2.waitKey(1)

if key == ord("q"):

break

#camera will off when we press the 'q' key

camera.release()

cv2.destroyAllWindows()

4.5 RESULTS

Fig: 4.3 Simulation result

9
CONCLUSION
In this modern age, we are all connected one or another way with the many digital
technologies. So by using that technology and by doing some modification in them we use
that technology in new application. This report gives brief demonstrates how objects can be
recognized, reconstructed, and localized within a 3D map, using observations and matching
of SIFT features in key frames. The key frames arise as part of a frame-rate process of
parallel camera tracking and mapping, in which the key frame camera poses and 3D map
points are refined using bundle adjustment. The object reconstruction process runs
independently, and in parallel to, the tracking and mapping processes. Detected objects are
automatically labeled on the user's display using predefined annotations. The annotations
are also used to highlight areas of interest upon the objects to the user.

10
REFERENCES
 https://www.techrepublic.com
1. G. Bradski, "Computer Vision Face Tracking For Use in a Perceptual User
Interface", Intel Technology Journal, 1998, [online] Available:
http://developer.intel.com/technology/itj/q21998/artic les/art 2.htm.
2. Y. Cheng, "Mean shift mode seeking and clustering", IEEE Trans. Pattern
Anal. Machine Intell., vol. 17, pp. 790-799, 1995.

3. P. Fieguth, D. Terzopoulos, "Color-based tracking of heads and other mobile

11