Hand Gesture Controlled Robot
Hand Gesture Controlled Robot
HAND MOVEMENT
By
Asadullah FA15-BEE-028
Irslan Sultan FA15-BEE-076
Muhammad Umer Mir FA15-BEE-069
Faculty of Engineering
MIRPUR UNIVERSITY OF SCIENCE AND TECHNOLOGY
Mirpur Azad Jammu Kashmir
June 2019
i
Copyright © 2019 by MUST Student
All rights reserved. Reproduction in whole or in part in any form requires the prior
written permission of Asadullah, Irslan Sultan, Muhammad Umer Mir or designated
representative.
ii
CERTIFICATE OF APPROVAL
It is certified that the research word titled “ROBOTIC VEHICLE CONTROLLED
BY HAND MOVEMENT” carried out by “Asadullah, Irslan Sultan, Muhammad
Umer Mir”, Roll. No. “FA15-BEE-028” “FA15-BEE-076” “FA15-BEE-069”, under
the supervision of Ma’am Benish Kanwal at Mirpur University of Science And
Technology, Mirpur is fully adequate, in scope and in quality, as a thesis for the
degree of B.Sc in Electrical Engineering.
Supervisor: -------------------------
Madam Benish Kanwal
Lecturer
Department of Electrical Engineering
Mirpur University of Science And Technology, Mirpur (AJK)
Chairman: ----------------------------
Dr. Anzar Mehmood
Professor
Department of Electrical Engineering
Dean: ----------------------------
Dr.Anwar Khitab
Professor
Dean Faculty of Engineering
Mirpur University of Science And Technology, Mirpur (AJK)
iii
ACKNOWLEDGMENT
All devotion belongs to Allah al mighty . Without support of Allah man is unable to
complete any task . We would like to acknowledge the Moral and technical support of
our supervisor Madam Benish Kanwal, Electrical Engineering Department, Must
University, Mirpur. Without her help this project would not have seen these steps
towards completion on time. Also, we admire her for her consultations guidance, and
Encouragement.
iv
DECLARATION
It is declared that this is an original piece of my own work, except where
otherwise acknowledged in text and references. This work has not been submitted in
any form for another degree or diploma at any university or other institution for
tertiary education and shall not be submitted by me in future for obtaining any
degree from this or any other University or Institution.
Asadullah
FA15-BEE-028
Irslan Sultan
FA15-BEE-076
June 2019
v
ABSTRACT
.
The Robotic vehicles are tasked to obtain some specific works which we cannot do
and take more time to do. we use the robotic vehicle for work like surveillance
purposes and lifting weight robots are designed to perform extend the utilization of
robots wherever Condit certain things on the basis of human instructions. It can
perform in harmful condition which human cannot work so we can apply these tasks
to robots. in the fyp we describe a robotic vehicle which works on the basis of hand
gesture robot is divided in two parts the transmitter side and receiver side in
transmitter side the information from accelerometer and flex sensor is send to
Arduino and Bluetooth module. Bluetooth module on the receiver sides ends this
information to Arduino a motor to rotate in the desired direction
vi
Table of Contents
Acknowledgment ........................................................................................................................ iv
Declaration .................................................................................................................................... v
Abstract ......................................................................................................................................... vi
Table of Contents ....................................................................................................................... vii
List of Figures............................................................................................................................... x
List of tables…………………………………………………………………………..xi
Chapter 1
Introduction
1.1Overview............................................................................................................................1
1.2Statement of Problem....................................................................................................1
1.3Purpose of the Research/Project .................................................................................2
Application of the research……….............................................................................2
1.4
1.5
Theoretical Bases and Organization...........................................................................2
1.5.1 Transmitter..................................................................................................................3
1.5.2 Receiver…...................................................................................................................3
1.6 Summary………………………………...........................................................................3
Chapter 2
Literature Review……………………………………………….......................4
vii
3.1.6 Crystal oscillator....................................................................................................14
3.1.7 Quartz crystal..........................................................................................................14
3.1.8 LM7805....................................................................................................................15
3.1.9 Serial to usb converter.........................................................................................16
3.1.10 Usb to serial .........................................................................................................17
3.1.11 Bluetooth module................................................................................................18
3.1.12 Three-wheel chassis ..........................................................................................19
3.1.13 Dc motor ...............................................................................................................20
3.2 Software(s), Simulation tool(s) used...................................................................20
3.3 Summary......................................................................................................................21
Chapter 4
viii
Chapter 5
Result and analysis................................................................................................................48
5.1 Presentation of the findings................................................................................48
5.1.1 Hardware Results...........................................................................................48
5.1.2 Software Results..........................................................................................48
5.2 Discussion of the finding........................................................................... 49
5.2.1 Comparison with initial GOAL.................................................................49
5.2.2 Reason for Short Coming...........................................................................49
5.3 Limitations........................................................................................................49
5.4 Recommendations............................................................................................49
5.5 Summary..........................................................................................................49
Chapter 6
Conclusion……………………………………………………………………...…50
References…………………………………………………………………………51
ix
LIST OF FIGURES
x
LIST OF TABLES
xi
LIST OF ACRONYMS
Bot Robot
RF Radio frequency
LCDs Liquid crystal displays
MC Micro controller
PCB Printed Circuit Board
TS Transmitter Side
RS Receiver Side
xii
Chapter 1
INTRODUCTION
Recently, sturdy efforts are administered to develop clever bridge between human
and computer . hand Gestures made bridge between man and computer. This
technique is expounded to the golem and their style, manufacturer, application.
AI analysis these days is concentrated on designing of systems which have
tendency, burden acceptance and few alternative researchers square measure on
utterly automating a producing method or a task, by providing detector based
mostly to the golem arm. During this extremely developing business and
manpower square measure crucial constraints for completion of the task. to avoid
wasting human efforts the automation taking part in a necessary part. method
being helpful to ordinary frequently supported tasks. Unique in every main
utmost ordinarily performed task selecting, inserting duties from supply to end
point. within the earlier the motion of the human hand is perceived by the golem
through sensors and it follows constantly. because the person moves their hand,
the measuring system conjointly begins moving consequently motion of the hand
detector displaces and this detector senses object or parameter in line with motion
of the hand.
1.1 Overview
1
1.2 Statement of Problem
user contains a adxl335 sensor and flex sensor patched on glove and
traditional hand gestures are controlled. we have a power bank power of cell
make decision about how sturdy is the vehicle if we add more power banks
makes the receiver side a lot of large and less power demages capability.
Change the working unwearable device so we can change the working as 3cm
to 5cm that economical for tiny measure however would like use our project on
huge measures .
2
1.5.1 Transmitter
1.5.2 Receiver
receiver side have a bot which contains a pcb buzzer, LCD ,Bluetooth
module and microcontroller on receiver side information from transmitter is
send to Arduino and which have capability to make decision from
information it haven in the form of code makes decision and send the
information to motor drive ic which rotates motors in desired direction
• go to forward side
• go to reverse side
• go to left side
1.6 Summary
Recently, robust efforts are applied to make smart basic connection between user
and glove based systems which support human movements. AI analysis nowadays
is focusing on the systems which are litheness, laying-off, and a few different
researchers square measure on fully automating a producing method or a task, by
providing sensing element based mostly to the robot arm.
3
Chapter 2
LITERATURE REVIEW
There exist some works within the field gesture recognition within which
instruments area unit designed and build for human robot bridge. in contrast to
audio data, a visible mechanism are going to desirable as in creaking environments
or at things wherever sound would cause a disturbance. there's a simplification
employed in this project, that wasn't found in any recognition ways researched. the
amount of various gestures recognized, and therefore the recognition accuracy area
unit amongst the most effective found hand handled mechanism do on the basis of
device that feed gestures transmit the recorded gestures which are made as result
of hand movements gives low voltage signals to microcontroller . the data sanded
to microcontroller, microcontroller make decision according information saved in
it and send it to motor drive ic there exit work in this field of robotic arm and leap
motion and joy stick controlled robot. We can also add camera in it and also add
obstacle detection faction in the robot
Figure 2.1 basic mechanisms
4
2.1 Related Technologies
2.1.1 Related Technology 1
5
2.3 Related Studies
Project control the robotic automotive supported our hand movement and
gestures. Knowledge from a Leap Motion management that detects gestures and
hand movement area unit captured by Matlab and also the control signals area
unit sent from Matlab to Robotic automotive that has Arduino and Bluetooth
module. Leap Motion, This project uses associate Arduino based mostly
automaton automotive, which might be controlled by gestures employing a
controller device referred to as as Leap Motion .Gesture-based Home
Automation system is a sophisticated methodology of dominant home appliances
victimisation spartan6 FPGA kit and gesture recognition kit. The gesture kit
accepts numerous hand gestures such North American country, left to right, right
to left, prime to bottom and bottom to prime and send the information to FPGA
kit. Finally FPGA management door lock open/close and bulb On/Off through
relay. This project controls the house appliances supported our hand movement
and gestures.
• Batteries are quite big and occupy more space also motor quite so better
replacing them with suitable option
• We are using transmitter receiver modules which have less coverage area
2.5 Summary
here exist some works within the field gesture recognition within which
instruments are designed and build for the human pc interaction employing
single hand language characters range signals within AI field, many analysis
efforts are created to make easy robots, implementing user interfaces like colour
bit screens, a 3D joystick. Finger gesture recognition: It aims to create it possible
to act transportable mechanism or a laptop through the popularity of hand
movements’ insight Technology.
6
Chapter 3
Figure 3.1.1
7
Table 3.1.1
Accelerometer
FEATURE:
3-pivot detecting
Less size
Less current: 350 micro ampere
1.8 V to 3.6 V
Temperature resistant
8
3.1.2 Adxl335
Figure 3.1.2
9
3.1.3 FLEX SENSOR
Figure 3.1.3
Flex Sensor
10
3.1.4 Atmega 328p
Figure 3.1.4
Pin diagram
11
Pin description
Table 3.1.4
12
3.1.5 L298N Dual Motor Controller
Module pinouts
Figure 3.1.5
13
3.1.6 The Crystal Oscillator
Fig 3.1.6
Figure 3.1.7
14
3.1.8 LM 7805
LM 7805 IC Rating
• In voltage 7v-35v
• Current flowing i=2ma
• Out voltage 5.2-4.8
Pin description
Figure 3.1.8
15
3.1.9 Serial to USB (CH-340)
Description
Features
Pinouts
(All of the following signals are supported and are tested at Art of Circuits Lab
using Bray Terminal)
DCD: in
RX: in
TX: out
DTR: data
DSR: input
16
3.1.10 Usb to serial converter
Pinout diagram
Figure 3.1.9
17
3.1.11 HC-05 Bluetooth Module
HC-05 PINOUT
Figure 3.1.10
18
3..1.12 Three Wheel Chassis
The chassis involves a outline which ropes a manmade vehicle .similar with animal
structure .just like chassis of car under the car and chassis of motor bike on which all
other thing are relied on all the other thing are mounted on it engine
Figure 3.1.11
19
3.1.13 Dc motors
Figure 3.1.13
20
3.3 Summary
we have described about various hardware module we have cast-off their features
pinout and pin description we have described about their technichal specification
and their algorithim and libraries
21
Chapter 4
METHODOLOGIES
According to the gesture within which the user directs his hands to manoeuvre the
robot, controlling signals are generated, that are distinctive for every direction. for
example, if the user desires to progress the robot within the forward direction, tiny
low voltage proportional to the gravitational field acting once the device
(accelerometer) is made and the forward gesture is given. Totally different sets of
values in X and Y axes once different gestures are used. These command signals are
then transmitted wirelessly to the Bluetooth module (HC-05) via the Bluetooth
module available in transmitted part i.e. on the glove. A Gesture Controlled robot
could be a reasonably mechanism which might be controlled by hand gestures not
by recent buttons. One simply must wear a tiny low transmittal device in his hand.
The transmitter device includes an Arduino accelerometer and a Bluetooth device.
This may transmit an acceptable command to the robot to induce the specified robot.
At the receiving part this information is received by another Bluetooth module that
is then processed by another Arduino motor supporting IC to regulate movements of
motors.
22
4.1.3 Driving the motors of the robot car
l293d ic obtain digital data and send the info as input to controller, provides output
to motors of bot. Speed handling of the DC motors may be achieved by this IC.
This is the particular bot involving of hardware Bluetooth module HC05, controller
and L289N motor drive cct on the bot chassis having voltage often provide by the
recharging batteries. The whole system is solely controlled by the gesture
23
4.3.1.2 FLEX SENSOR
Hc-05 module is capable of transmitting information from one place to way beyond
the range it works on 2.4 GH frequencies and coded with serial communication
code.
Specification:
● Freq: 2.4GHz
● Temp: - 20 ~ +75 °C
4.3.1.4 DC MOTORS
A device which has ability to convert electrical energy into mechanical energy is
termed as dc machine. it works very simple the current passing from conductor and
conductor is inserted in changing MF , so there is a power exerted on conductor and
the coil present will start to rotate. Speed of our motor is measured by speed of shaft
present in side.
24
4.3.1.5 Wheels
Wheel types
Pair of wheel
Three wheel
Four wheel
25
26
27
28
4.3.2.2 Receiver side code
29
30
31
32
33
34
35
4.4 Verification of functionalities
It has been verified looking at simulations, sampling and values of virtual terminal
that by tilting hand forward backward left right the motors moves accordingly and
displays the result as forwarding backward leftward rightward
36
4.5 Details about simulation / mathematical modeling
4.5.1 Schematics of transmitter side
Figure 4.5.1
37
4.5.2 PCB OF TRANSMITTER SIDE
Figure 4.5.2
38
4.5.3 SCHEMATICS OF RECIEVER SIDE
Figure 4.5.3
39
4.5.4 PCB OF RECIRVER SIDE
Figure 4.5.4
40
4.5.5 SAMLING OF SENSOR
4.5.5 .1 Backward
Figure 4.5.5.1
41
4.5.5 .2 Forward
Figure 4.5.5.2
42
4.5.5 .3 Leftward
Figure 4.5.5.3
43
44
4.5.5 .4 Rightward
Figure 4.5.5.4
45
4.5.5.5 SIMULATIONS
4.5.5.5 Figure
46
4.5.5.6 Values of virtual terminal
Figure 4.5.5.6
4.6 Summary
According to the gesture during which the user directs his hands to maneuver the
robot, control signals are generated, that are distinctive for every direction. L293d
motor driver ic achieve signal from controller and send this signal to motors. this is
often particular mechanism -Bluetooth module HC05, controller and L293d
47
Chapter 5
RESULTS AND ANALYSIS
5.1 Presentation of the findings
Various experiments are conducted to see the most effective fitting correlation worth
for this project on gesture controlled bot. a pair of axes of the accelerometer is
employed for the project .The analog result obtained by measuring device were from
the vary 0-5 volts that were mapped to ten output channels that were starting from 0-
1023 taking analogRead() operate. For forward and backward motion, Y axis values
area unit found to be constant. Equally for left and right motion X axis values were
found to be constant
Divided project in two parts. we sending data received from the flex sensor and
adxl335 sensor to the robot on the receiver side on the receiver side controller is
present for decision-making ifs sensor for the x-axis is 290-325 and y-axis value is
280-300 it sends the command to move forward to receiver side but it moves forward
if and only if we give some speed ie if the flex sensor value is 700-900 then it will go
forward similar procedure would be followed backward, leftward and rightward
Project is divided into three part glove and transmitter side and robot ist we have
design transmitter side ie making of glove and interfacing of adxl335 sensor and flex
sensor interfacing is done by using their libraries and then see their result on the serial
monitor by tilting palm and bending flex sensor and doing sampling 5000 times and
define some thresholds result receiver side we have Bluetooth which will receive it
and send it to controller the result on the reciever side can be seen for all types of
movement
48
Bluetooth and adxl335 and is used to connect to the robot so that it may perform
functions associated with it like left, right, forward, backward. All these tasks will be
performed by hand movements
We start its looks easy we are asumming that we will windup project before it actual
date but alter we find some problems that our modules are not intercaing with each
other require mudules are not available in market so we have to shft our modules
instead of using rf transmitter we have to use bluestooth mudule
5.3 Limitations
• The batteries on board are bulky and they take much greater space. We
will have to use an alternate way to supply power to the motor or
replace the motor with a motor of less power.
• We have used radio frequency for sending data the range is limited to 49-
81.
5.4 Recommendations
I recommend using 2g for sending data. 2g used across the globe. We can use
camera for if the robot is to be used in long range mount a camera where we can
receive the camera recording.
5.5 Summary
Robot consists of two parts input and output. Input section consists of two sensing
elements and micro controller. Data from microcontroller is send to output side using
Bluetooth. Receiver side consists chases, motor, Bluetooth. All the programming
was done on C.
49
Chapter 6
CONCLUSION
In our project we developed a vehicle which is controlled through the hand gestures..
The vehicle is wireless. Bluetooth operates on frequency of 433 MC and its range is
49-81m. Robot can be improved to notice human life in disasters like shaking of earth
by installing the detector consequently. Robot can be improved to bombing works
because it has an arm that can be used. Global positioning system can be added for
tracking purposes.
50
References
1.A. Kun, W. T. Miller, "Adaptive dynamic balance of abiped robot using neural
networks", IEEE Conference on Robotics and Automation, pp. 240-245, 1996.
51
2. T. Nagasaki, S. Kajita, K. Kaneko, K. Yakoi, K. Tanie, "A running experiment
of humanoid biped", Proc. of IROS2004, pp. 136-141, 2004.
52
53