International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019

RESEARCH ARTICLE OPEN ACCESS

Design and Implementation of Real Time Pick and Place Robot

Controlled by Hand Gesture
K.Parvateesam1, P.Satya Sruthi2
1(Electronics and Communication Engineering, , Aditya College of Engineering and Technology, Kakinada, AP
Email: [email protected])
2 (Computer Science and Engineering, Aditya College of Engineering and Technology, Kakinada, AP
Email: [email protected])

Abstract:
Today human machine interaction is moving away from keypad and is becoming pervasive and much more
compatible with the physical world. Robots are playing an important role in automation across the entire sectors like
construction, military, medical, manufacturing, etc. Gestures are playing vital role in the automation. The main purpose of
using gesture is that it provides a more natural way of controlling and provides a rich and intuitive form of interaction with
the robotic system. The pick and place means to pick the certain object from one location and place it to another desired
location. This work can be done by machine called pick and place robot. The robotic arm kit is made of two sections. The
bottom driving unit takes the robot to left, right, backward, forward and other six direction motions. These all six direction
motions are controlled by the accelerometer. The top gripper unit is to pick and place any object. The gripper unit is
controlled by the flex sensor. The driving unit has four dc motors and two motor drivers. The two sections in this project are
transmitter and receiver will be communicated through ZigBee/XBee wireless protocol. The robot is strong enough to handle
the task like rescue, transporting etc. The pick and place robots are designed, implemented in various fields such as in bottle
filling industry, packing industry, used in surveillance to detect and destroy the bombs etc. The pick and place robot is
controlled by Arduino UNO and ZigBee/XBee module.

Keywords — Pick & Place Robo, Arduino UNO, ZigBee/XBee module, Hand Gesture.

nothing but human hand with flex sensors,

I.INTRODUCTION
accelerometer & receiver is robot manipulator.
In today’s life automation plays very important role. Motion of transmitter is wirelessly transmitted to
Robotic arm is called as robot manipulator which receiver through ZigBee/XBee module. Robotic
can perform various functions as human arm arm which is receiver is nothing but a mechanical
performs. Many industries use a robot for various system formed by different joints and end and
functions where important part of any robot is effectors i.e. gripper movements of these fingers or
robotic arm or called as robot manipulator should gripper can be carried out using DC motors when
be controlled precisely depending upon application. user carry out motion of hand for any application at
In industry or any application robot manipulator can transmitter side same movement is copied by
be used for applications like welding, trimming; receiver as on transmitter there are flex sensors,
picking etc. Advantage of such robotic arm is it can accelerometer mounted on glove at transmitter
work in hazards area, which cannot be accessed by which change its resistance depending on
human. Many parameters of robot are designed movement of user. Robotics is a current emerging
according to requirement. There are different ways technology in the field of science. Nowadays,
to control robotic arm like Voice Controlled, robots are increasingly being integrated into
Keypad Control, Gesture Control, etc. Our project working tasks to replace humans especially to
is based on gesture control. Implemented system perform the repetitive task. In general, robotics can
consists of transmitter & receiver. Transmitter is be divided into two areas, industrial and service

ISSN: 2395-1303 http://www.ijetjournal.org Page 109

 International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019
robotics these robots are currently used in many new node to the network is decided, according to
fields of applications including office, military tasks,
the application and security policies.
hospital operations, dangerous environment and Communication can happen right after the
agriculture. Besides, it might be difficult or association. Direct addressing uses both radio
dangerous for humans to do some specific tasks like address and endpoint identifier, whereas indirect
picking up explosive chemicals, defusing bombs or addressing uses every relevant field (address,
in worst case scenario to pick and place the bomb endpoint, cluster and attribute) and requires that
somewhere for containment and for repeated pick they be sent to the network coordinator, which
and place action in industries. Hence, it is proposed maintains associations and translates requests for
to develop something that breaks the barriers and communication. Indirect addressing is particularly
brings the digital world closer to the human world useful to keep some devices very simple and
using accelerometers, gestures (flex sensors), latest minimize their need for storage. Besides these two
wireless communication. methods, broadcast to all endpoints in a device is
available, and group addressing is used to
II.HARDWARE DESIGN OF THE SYSTEM
communicate with groups of endpoints belonging to
In order for applications to communicate, their a set of devices.
comprising devices must use a common application
protocol (types of messages, formats and so on); A Block diagram
these sets of conventions are grouped in profiles.
Furthermore, binding is decided upon by matching
input and output cluster identifiers, unique within
the context of a given profile and associated to an ACCELEROMETER
incoming or outgoing data flow in a device.
Binding tables contain source and destination pairs.
Depending on the available information, device
ARDUINO
discovery may follow different methods. When the
network address is known, the IEEE address can be
requested using unicast communication. When it is FLEX SENSOR
not, petitions are broadcast (the IEEE address being
part of the response payload). End devices will
simply respond with the requested address, while a
network coordinator or a router will also send the
addresses of all the devices associated with it. This
extended discovery protocol permits external ZigBee
devices to find out about devices in a network and TRANSMITTER
the services that they offer, which endpoints can
report when queried by the discovering device
(which has previously obtained their addresses).
Matching services can also be used. The use of
cluster identifiers enforces the binding of Fig:1a: pick and place robot Transmitter section

complementary entities by means of the binding

tables, which are maintained by ZigBee/XBee
coordinators, as the table must be always available
within a network and coordinators are most likely
to have a permanent power supply. Backups,
managed by higher-level layers, may be needed by
some applications. Binding requires an established
communication link; after it exists, whether to add a

ISSN: 2395-1303 http://www.ijetjournal.org Page 110

 International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019
backward side, robot change its state and start
moving in backwards direction until other
command is given. When we tilt it in left side
ZigBee DC MOTORS
RECEIVER Robot get turn left till next command. When we tilt
hand in right side robot turned to right. And for
stopping robot we keeps hand in stable.

ARDUINO MOTOR DRIVER

MOTOR GRIPPER/
DRIVER ROBOTIC ARM

Fig:1b: pick and place robot Receiver section

Gesture Controlled Robot is divided into two

Fig:2 Transmitter schematic
sections: 1. Transmitter part and 2. Receiver part.
In transmitter part an accelerometer, flex sensor and Flex sensors are analog resistors. It works as
a transmitter unit is used. As we know that variable analog voltage divider. Internally it
accelerometer gives an analog output so here we consists of carbon resistive element with thin
need to convert this analog data in to digital. For substrate. As substrate is flexible when it is bent,
this purpose we have used 4 channel comparator sensor produces resistive output which is equivalent
circuit in place of any ADC. By setting reference to bend radius. The flex sensor provides greater
voltage we gets a digital signal and then apply this accuracy for small movements also. Smaller the
signal to HT12E encoder to encode data or radius higher will be the resistance value. These
converting it into serial form and then send this data flex sensors are mounted on human palm as user
by using ZigBee/XBee transmitter into the moves palm for particular applications flex sensor
environment. At the receiver end we have used also bends by same amount as they are flexible. The
zigbee receiver to receive data and then applied to changes due to tilt of accelerometer are in forward,
HT12D decoder. This decoder IC converts received backward i.e. X-axis, left, right i.e. Y-axis and up,
serial data to parallel and then read by using down i.e. Z-axis. Accelerometer converts deflection
arduino. According to received data we drive robot into proportionate voltage & Analog to Digital
by using DC motors in forward, reverse, left, right converter convert analog signal into proportionate
and stop direction and pick and place the object. digital value. So according to positive & negative
deflections motor either rotates in clockwise or in
B Working Principle anticlockwise direction.
Gesture controlled robot moves according to hand C Receiver schematic
movement as we place transmitter in our hand.
When we tilt hand in front side, robot start to In the transmitter side ZigBee/XBee transmit the
moving forward and continues moving forward signal and this signal is received by the receiver
until next command is given. When we tilt hand in ZigBee/XBee and then fed to the microcontroller
 International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019
which drives motor through motor driver to control the Mac, the serial port is probably something like
movement of robot manipulator. The sensor glove /dev/tty.usbmodem241 (for an Uno or Mega2560 or
reads values from the flex sensors and Leonardo) or /dev/tty.usbserial-1B1 (for a
correspondingly sends them wirelessly using XBee Duemilanove or earlier USB board), or
protocol to the robotic hand. /dev/tty.USA19QW1b1P1.1 (for a serial board
connected with a Keyspan USB-to-Serial adapter).
On Windows, it's probably COM1 or COM2 (for a
serial board) or COM4, COM5, COM7, or higher
(for a USB board) - to find out, you look for USB
serial device in the ports section of the Windows
Device Manager. On Linux, it should be
/dev/ttyACMx , /dev/ttyUSBx or similar. Once
you've selected the correct serial port and board,
press the upload button in the toolbar or select the
Upload item from the Sketch menu. Current
Arduino boards will reset automatically and begin
the upload. With older boards (pre-Diecimila) that
lack auto-reset, you'll need to press the reset button
on the board just before starting the upload. On
most boards, you'll see the RX and TX LEDs blink
as the sketch is uploaded. The Arduino Software
(IDE) will display a message when the upload is
complete, or show an error.

Fig:3 Receiver schematic

C Serial Monitor

This displays serial sent from the Arduino or

Genuino board over USB or serial connector, as
III. SOFTWARE IMPLEMENTATION
shown in Fig.4.2. To send data to the board, enter
A Arduino IDE
text and click on the "send" button or press enter.
Choose the baud rate from the drop-down menu
The first time you run the Arduino software, it will that matches the rate passed to Serial.begin in your
automatically create a directory for your sketch. Note that on Windows, Mac or Linux the
sketchbook. You can view or change the location of board will reset (it will rerun your sketch) when you
the sketchbook location from with connect with the serial monitor.
the Preferences dialog. The source code for the
IDE is released under the GNU General Public
License, version 2. The Arduino IDE supports the
languages C and C++ using special rules of code
structuring. The Arduino IDE supplies a software
library from the Wiring project, which provides
many common input and output procedures.

B Uploading

Before uploading your sketch, you need to select

the correct items from the Tools > Board and Tools
> Port menus. The boards are described below. On Fig:5 Serial monitor window

ISSN: 2395-1303 http://www.ijetjournal.org Page 112

 International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019

D Algorithm
IV. RESULT AND DISCUSSION
Step1: Start
Step2: Initialize the ZigBee module
Step3:Initially set the predefined values
for accelerometer and flex sensor
Step4:Read the input values from the
hand gesture
Step5:Compare the input values with
predefined values
Step6:After comparing encode the data
Step7:The encoded data is sent to the
receiver
Step8:Encoded data will decode and
passes to the motors
Step9:Based on the movements the robot
will pick and place the object
Fig:7 Transmitter module
Step10:Stop

E Flowchart
Based on hand movements with holding
accelerometer inside, changing the movement
START
robot works as follows: forward: Robot moves in
forward direction. In reverse: Robot moves in
reverse direction. In left: Robot moves in left
INITIALIZE direction In right: Robot moves in right direction.
THE ZIGBEE We are placing two flex sensors for two different
fingers. If the finger1 bend forward means robot
arm will open and if the finger1 bend backward
READ THE INPUT means robot arm will close. If the finger2 bend
HAND VALUES FROM forward means robot arm will pick the object .And
GESTURE ACCELEROMETER if the finger2 bend backward means robot arm will
AND FLEX SENSOR
place the object.

COMPARE THE INPUT VALUES WITH

THE PREDEFINED VALUES AND
ENCODE

SEND THE ENCODED

SIGNAL FROM
TRANSMITTER TO
RECEIVER

STOP
Fig:8 Receiver module

Fig:6 Flowchart
 International Journal of Engineering and Techniques - Volume 5 Issue 3, May- June 2019
VI.CONCLUSION PattewarBalaji , Deshmukh Saurabh Professor, Student,
RSCOE Pune, (India).
The objective of this paper has been achieved with
developed hardware and software for an flex sensor 3. International Journal for Research in Applied Science &
and accelerometer controlled robotic arm. From Engineering Technology (IJRASET) Volume 6 Issue III,
March 2018 Hand Gesture Controlled Robot Using Arduino
observations it is clear that movement of arm and
Ms.Asmita Jadhav , Ms. Deepika Pawar , Ms.Kashmira
car is precise, accurate, and is easy to control and Pathare , Ms.Prachi Sale , Prof.R.Thakare, Department of
user friendly to use .This robot control method is Electronics and Telecommunication, Bharati Vidyapeeth
expected to overcome the problem such as placing College Of Engineering, Belpada, Navi Mumbai.
or picking object that is away from the user or
4. International Research Journal of Engineering and
picking any hazardous object in a very fast and easy
Technology (IRJET) Volume: 05 Issue: 04 Apr-2018 Robotic
manner. This type of the hand gesture technology Vehicle Movement and Arm Control through Hand Gestures
can be used where the humans are unable to sustain using Arduino N Meenatchi Sundaram, R Shiva Rama
in the difficult or harsh environments. This might Krishnan, T K Suraj Kamal, J Rajeswari, UG
reduce some of the labour, that is used in industry Students,Department of Electrical and Electronics
Engineering, Velammal College of Engineering and
and also the life risk factor.
Technology, Madurai.
ACKNOWLEDGMENT 5. International Journal of Innovative Research in Science,
We are very grateful to our Head of the Department Engineering and Technology , Vol. 4, Issue 6, June
of Electronics and Communication Engineering, 2015 ,Gesture Controlled Mobile Robotic Arm Using
Mr.D.Kishore, M.tech,(Ph.D) , Associate Professor, Accelerometer Vivek Bhojak, Girish Kumar Solanki , Sonu
Daultani, Associate Professor, Department of Electronics and
who helped us in choosing a research area Communication, Anand International College of Engineering,
Embedded Systems that has enlightened our Jaipur, Rajasthan, India.
thinking skills and laid a foundation for the
completion of this paper. We express our sincere 6. IRA-International Journal of Technology & Engineering
thanks to Dr. T.K. Ramakrishna Rao, M.Tech, Ph.D ISSN 2455-4480; Vol.07, Issue 01 (2017) Gesture Control of
Robotic Arm Dharshan Y , Vivek S , Saranya S , Aarthi V.R ,
Principal of Aditya College of Engineering And Madhumathi T,Assistant Professor, Department of EIE Sri
Technology for providing appropriate environment Ramakrishna Engineering College, Coimbatore, India
required for our work. We also express our
gratitude to college management and all those who 7. International Journal of Advance Engineering and
have indirectly helped us in completion of this work. Research Development Volume 5, Issue 05, May -2018 Voice
and gesture recognition of pick and place robot using pic
microcontroller Dr Naveen B , Poornesh.P , Priya.U ,
REFERENCES Rekha.D , Suraj.H.D,Associate Professor ,U.G.Student,
BGSIT Visveswaraya Technological University, Karnataka,
1. Gesture Controlled Pick and Place Robot Prateek Pal , India.
Amrit Mishra , M.C. Srivastava, Ashwani Sharma,UG Scholar,
Assistant Professor,Mechanical and Automation Engineering 8. International Journal of Engineering Research &
Amity University, Lucknow, India. Technology (IJERT) Vol. 6 Issue 04, April-2017 Design and
Implementation of Robotic Hand Control Using Gesture
2. International Journal of Advanced Technology in Recognition Sakshi Sharma, Shubhank Sharma, Piyush Yadav
Engineering and Science Volume No.03, Issue No. 04, UG Student , Assistant Professor, Department of Electronics
April 2015 HAND GESTURE CONTROLLED PICK AND and Communication Engineering G.L.Bajaj Institute of
PLACE ROBOT Prof.R.R. Itkarkar , Dixit Rohan , Technology and Management Greater Noida, India.

ISSN: 2395-1303 http://www.ijetjournal.org Page 114