International Journal of Engineering and Advanced Technology (IJEAT)

ISSN: 2249 – 8958, Volume-8, Issue-5S, May, 2019

Robotic ARM using Computer Vision

M Sandeep, Lavanya Shivgonda, Rajeswari M, Kaushik S, Nikhil Tengli

Abstract— A Bot which pursues Human hand developments. Its The majority of the scientists arranged motion
unlimited authority lies with the client and doesn't have any acknowledgment framework into primarily three stages
knowledge of its own. Programmed robots having man-made subsequent to getting the info picture from camera. These
brainpower are a danger to society and may cause hurt in certain are: Extraction Process, highlights estimation and grouping
situations. Subsequently, having full oversight over the robot is a or acknowledgment as appeared in Figure 2.1
protected method to work with such robots. In this paper, we
have proposed a comparable arrangement of a robot. catching
Extractio Highlights Grouping
pictures from the PC web cam progressively condition and
procedure them as we are required. By utilizing open source n Process Estimation
Computer vision library (OpenCV for short), a picture can be
caught on the basis of its Hue saturation value (HSV) extend. Fig 2.1: Data processing
The fundamental library capacities for picture dealing with and
handling are utilized. Fundamental library capacities are  Extraction process and picture pre-handling
utilized for stacking a picture, making windows to hold picture at
run time, sparing pictures, and to separate pictures dependent on Division is the primary procedure for perceiving hand
their shading values. I have additionally connected capacity to signals. It is the way toward isolating the information picture
edge the yield picture so as to diminish the twisting in it. While into Districts isolated by limits. The division procedure is
handling, the pictures are changed over from their essential
relies upon the kind of motion, in the event that it is dynamic
plain Red, Green, and Blue (RGB) to an increasingly reasonable
one that is HSV.
signal, at that point the hand motion should be found and
followed however on the off chance that it is static motion, at
that point input picture must be fragmented as it were. The
I. INTRODUCTION hand ought to be found right off the bat so a jumping box is
utilized to indicate the skin shading, since it is simple and
PC vision is an interdisciplinary field that manages how PCs invariant to scale, interpretation, and revolution changes. In
can be made to pick up an abnormal state understanding from division process the shading space is utilized yet shading
advanced pictures or videos.it depends on ongoing spaces are delicate to lighting changes for these reason HSV
innovation of Computer vision. It doesn't requires any gloves shading models are utilized. This system focuses on the
or other overwhelming contraption. This technique requires shades of the pixel used to standardized R-G shading space.
just web camera to removes the video outline. Today the Some pre-preparing tasks are connected, for example,
greater part of the workstations have an incorporated foundation subtraction, edge discovery and standardization
webcam alongside it so it is an effectively accessible gadget to upgrade the sectioned hand picture.

The thought here is to computerize assignments that the  Highlights Estimation

human visual frameworks can do. In this way, a PC ought to
almost certainly perceive objects like the essence of an Great division process prompts impeccable highlights
individual or even a statue. In this the camera input extraction procedure and it assume an essential job in an
identifying and detecting a hand using various algorithms in effective acknowledgment process. Highlights vector of the
OpenCV. fragmented picture divided picture can be removed in various
ways as indicated by specific application. Different
The computer reads any image as a range of values between 0 techniques have been utilized for speaking to the highlights
and 255. For any color image, there are 3 primary channels extraction.
— red, green and blue.
A few techniques utilized the state of the hand, for example,
II. RELATED WORK hand form recognition while others identify fingertips
position or palm focus.

This segment quickly talks about a portion of the work done  Gestures Grouping
by different creators in the field of PC vision and mechanical
arms. In the wake of demonstrating and examination of the
information picture, signal arrangement technique is utilized
Revised Manuscript Received on April 24, 2019. to perceive the motion.
Mr. M Sandeep, School of C & I T, REVA University, Acknowledgment process is influenced by appropriate choice
Ms. Lavanya Shivgonda, School of C & I T, REVA University, of highlights parameters and
Ms Rajeswari M, School of C & I T, REVA University,
Mr. Kaushik S, School of C & I T, REVA University,
reasonable order calculation.
Mr. Nikhil Tengli, School of C & I T, REVA University,

Published By:
Blue Eyes Intelligence Engineering
Retrieval Number: E10010585S19/19©BEIESP 1 & Sciences Publication
 Robotic ARM using Computer Vision

For instance edge identification or shape administrators can't outcome is a solid classifier made out of a course of the chose
be utilized for motion acknowledgment since many hand week classifiers.
stances are created and could deliver misclassification. The we select skin-shading to get normal for hand. The
measurable instruments utilized for motion characterization skin-shading is an unmistakable signal of hands and it is
are Hidden Markov invariant to scale and turn. In the subsequent stage we utilize
the assessed hand state to separate a few hand highlights to
Show (HMM), Finite State Machine (FSM), Principal characterize a deterministic procedure of finger
Component Analysis (PCA) and Neural system has been acknowledgment. After the hand is divided from the
generally connected for extraction the hand shape. Other foundation, a counter is extricated. The counter vector
delicate processing instruments are Fuzzy C Means grouping contains the arrangement of directions of edges of hand. At
(FCM), that point the handling of counter vector gives the area of the
fingertip.
Hereditary Algorithms GAs. In vision based methodology,
we study of every accessible strategy utilized for hand As indicated by another paper titled "Vision Based Hand
identification, motion acknowledgment. From that we chose Gesture Recognition for Human Computer Interaction" by X.
HSV for Pre-handling, AdaBoost for hand discovery and Zabulisy, H. Baltzakisy and A. Argyroszy [3], they
Haar classifier for preparing and redesign reason which has concentrate to vision-based acknowledgment of hand
quick identification with great precision. signals. The initial segment of the paper gives a diagram of
The paper "Continuous Finger Tracking and Contour the present best in class with respect to the acknowledgment
Detection for Gesture Recognition utilizing OpenCV" by of hand motions as these are watched and recorded by regular
Ruchi Manish Gaurav and Premanand K. Kadbe [1] camcorders.
recommends that these days new advancements of Human
Computer Interaction (HCI) are being created to convey As per another paper titled "Hearty Real-Time Tracking of
client's direction to the robots. Clients can interface with Non-unbending Objects" by Richard Y. D. Xu, John G. Allen
machines through hand, head, outward appearances, voice and Jesse S. Jin Video Object Tracking assumes an essential
and contact. The goal of their paper is to utilize one of the job in numerous vision applications. Aside from applications
critical methods of association for example hand signals to customarily for video observation, object acknowledgment,
control the robot or for workplaces and family unit and video division and ordering, ongoing article following is
applications. currently widely utilized in broad media discourse
The straightforward Haar-like highlights (which are acknowledgment (Liu 2002), human signal
processed correspondingly to the coefficients in the Haar acknowledgment, and item based video compressions, for
wavelet change) are utilized in the Viola and Jones example, MPEG-4.
calculation. The Haar-like highlights are strong to clamor Their paper, then again, examines object following utilizing
and different lighting condition since they register the dim data from a client chose area of the underlying edge, which
dimension contrast between the white and dark territory of contains objects of enthusiasm without priori data. A solid
square shapes. The clamor and lighting varieties influence focal point of this paper is execution. Powerful continuous
the pixel esteems all in all component territory. The following of non-unbending items is required in numerous
fundamental picture at the area of pixel [x, y] contains the applications.
whole of the pixel power esteems found straightforwardly tracks key purposes of the form subsequent to portioning
over the pixel area [x,y] and at the left half of this pixel. So each casing utilizing Canny channel and combining them by
A[x,y] is the first picture and AI[x,y] is the indispensable separation testing. The key focuses utilized in following are
picture that is determined by a condition. the neighborhood maxima or the defining moments of the
shapes.
The AdaBoost based learning calculation improves organize
by stage generally speaking exactness, by utilizing a straight This paper examines following articles utilizing quick
blend of these independently frail classifiers [4]. The shading thresholding as shading data gives a productive
AdaBoost learning calculation at first relegates an equivalent element. They are powerful to fractional impediment and
load to each preparation test. We begin with the choice of a geometry invariant, and computationally productive.
Haar-like element based classifier for the principal organize
and showed signs of improvement than half grouping Following Using Colour Clustering
precision. In following stage this classifier is added to 975 i. Colour Space
the straight mix with the quality that is relative to the ii. Colour Representation
subsequent exactness. So the preparation test loads are iii. Region Grouping and Noise Filtering
refreshed for example preparing tests that are missed by the iv. Foreground Object Extraction
past classifier are supported in understanding. The following a. Colour Clusters Determination
grouping stage must accomplish better exactness for these b. Foreground Extraction Mask
misclassified preparing tests so the mistake can be decreased. v. Contour Extraction
By this methodology we can improve the general order vi. Alpha Blending with Edge Map
exactness at further stage. The emphasis goes on by adding vii.
new classifiers to the direct mix until the general exactness
meets to the required dimension. At the last dimension the

Published By:
Blue Eyes Intelligence Engineering
Retrieval Number: E10010585S19/19©BEIESP
2 & Sciences Publication
 International Journal of Engineering and Advanced Technology (IJEAT)
ISSN: 2249 – 8958, Volume-8, Issue-5S, May, 2019

III. METHODOLOGY
1. Colour Detection
The algorithms in the OpenCV library are used in order to
identify a user’s hand through camera and is tracked in real
To detect colour, the BGR colour space is converted
time.
into an HSV colour space.
The BGR camera input is converted into a HSV (Hue,
Saturation, Value) space and a specific colour is detected
The tint of a pixel is an edge from 0 to 359 the
using the mentioned HSV values.
estimation of each point chooses the shade of the
The detected colour entity is tracked in real time frame by
pixel the request of the shading is same yet in turn
frame.
around as the request in rainbow request from red to
All these data are processed and the signals are sent to the
violet and again back to red. The Saturation is
robotic arm through an Arduino Uno board.
essentially how soaked the shading is, and the Value
is the manner by which splendid or dull the shading
The system majorly consists of three modules:
is.
i. Data Acquisition Module
So the scope of these are as per the following:
ii. Data Processing Module
iii. Hardware Module
 Hue is mapped – >0º-359º as [0-179]
Data Acquisition Module: This module deals with the
• Saturation is map - > 0%-100% as [0-255]
acquisition of data from the user through a camera input by
identifying and detecting a hand using various algorithms in • Value is 0-255 (there is no mapping)
OpenCV. This acquired data is forwarded to the processing
module. So as to identify a shading, we have to choose a
range for HSV esteem for that specific shading as
Data processing Module: This module processes the data
there are loads of variety of the shading.
received from the acquisition module. The data includes the
shape of the hand being distinctly identified from the Presently we make another paired picture of same
background using the Convex Hull algorithm and is size as unique picture called a veil and we'll ensure
processed by the Arduino Uno Microcontroller. just those pixels that are in this HSV range will be
permitted to be in the cover. That way just that
Hardware Module: The processed data is received from the shading items will be in the veil.
processing module through the microcontroller and is
implemented on the hardware robotic arm.

Fig 3.2 BGR Colour Space

Fig 3.1: System Architecture

Published By:
Blue Eyes Intelligence Engineering
Retrieval Number: E10010585S19/19©BEIESP 3 & Sciences Publication
 Robotic ARM using Computer Vision

processed and the servo motors of the robotic arm

rotate accordingly moving the hand in the desired
direction.

IV. RESULTS AND DISCUSSION

The robotic arm built receives signals from the Arduino uno
board and performs actions based on the inputs received by
web cam using OpenCV library.
Fig 3.3 Raw Mask Output (HSV Colour space)
Even though the system is a robot capable of doing various
tasks in different fields, it can only work when a trained user
is operating it minimizing the risks of accidents due to
self-aware robots.
2. Filtering the Mask

In the above mask, the colour is detected but it has V. CONCLUSION AND FUTURE SCOPE
some false positives called noise which may affect
the efficient tracking of the hand. PC vision speaks to the "product sensor" of things to come.
In order to filter it out, we need to do some PC vision exchanges interesting equipment for
morphological activity called opening and shutting. programming.
Robots help individuals with assignments that would be
Opening the cover will clear every one of the dabs troublesome, dangerous, or exhausting for a genuine
haphazardly showing up out of sight. individual to do alone.
This robot is a step for the near future, giving complete
Shutting will fill the little gaps present inside the control to robots may be a threat to mankind.
genuine item (hand). Thus, this Robot is a machine which has no artificial
intelligence, but still can perform numerous tasks with
human control.
We can exhibit that this Hand motion location and
acknowledgment, joined with different advances, can deliver
successful and ground-breaking applications.

VI. REFERENCES

[1]. Ruchi Manish Gurav and Premanand K. Kadbe, "Continuous

Finger Tracking and Contour Detection for Gesture Recognition
utilizing OpenCV " 2015 International Conference on Industrial
Instrumentation and Control (ICIC).
[2]. Ivan Culjak, David Abram, Tomislav Pribanic, Hrvoje Dzapo and
Mario Cifrek, ―A brief introduction to OpenCV‖ University of
Zagreb, MIPRO 2012.
[3]. Viraktamath SV, Mukund Katti, Aditya Khatawkar, Pavan
Kulkarni, ―Face Detection and Tracking using OpenCV,‖ The SIJ
Transaction on Computer Networks & Communication
Engineering (CNCE), 2013.
[4]. Nidhi ―Image Processing and Object Detection‖ Department of
omputer Applications, NIT Kurukshetra, Haryana, India 2015
[5]. Mamata s.kalas ―Real time face detection and tracking using
opencv‖ Department of IT, KIT’S College of Engg., Kolhapur 2014
[6]. Shamsheer Verma ―Hand Gestures Remote Controlled Robotic
Arm‖ Advance in Electronic and Electric Engineering 2013.
[7]. Kari Pulli, Anatoly Baksheev, Kirill Kornyakov and Victor
Eruhimov ―Real-Time Computer Vision with OpenCV‖ 2012
[8]. Neetu Saini, Sukhwinder Kaur, Hari Singh A Review:Face
Detection Methods And Algorithms, International Journal
of Engineering Research & Technology June – 2013

Fig 3.4 Mask Open and Mask Close

3. Processing

The coordinates of the detected hand are then sent to

an Arduino Uno board in real time and the data is

Published By:
Blue Eyes Intelligence Engineering
Retrieval Number: E10010585S19/19©BEIESP
4 & Sciences Publication
 International Journal of Engineering and Advanced Technology (IJEAT)
ISSN: 2249 – 8958, Volume-8, Issue-5S, May, 2019

Authors Profile Ms Rajeswari M pursuing Bachelor’s of Computer Science

& Information Technology from REVA University,
Mr. M Sandeep pursuing Bachelor’s of Computer Science Bangalore since 2015.
& Information Technology from REVA University,
Bangalore since 2015. Mr. Kaushik S pursuing Bachelor’s of Computer Science &
Information Technology from REVA University, Bangalore
Ms. Lavanya Shivgonda pursuing Bachelor’s of Computer since 2015.
Science & Information Technology from REVA University,
Bangalore since 2015. Mr.NikhilTengli Assistant Professor, School of Computing
& Information Technology, REVA University, Bangalore.

Published By:
Blue Eyes Intelligence Engineering
Retrieval Number: E10010585S19/19©BEIESP 5 & Sciences Publication