ISSN (Online) 2581-9429

IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

Survey Paper on Hand Gesture Recognition Using

Deep Learning
Prof. Nilam Honmane1, Sourabh Pawar2, Ashitosh Nevase3, Vaibhav Purane4, Saurabh Sawant5
Professor, Department of Information Technology1
Students, Department of Information Technology2,3,4,5
Zeal College of Engineering and Research, Pune, Maharashtra, India
Savitribai Phule Pune University, Pune, Maharashtra, India
[email protected]

Abstract: Hand gesture recognition is an applications with a small number of defined gestures. If
attractive research field with a wide range of computers can understand the movements or gestures of
applications. Another important application of human hands, we can close the gap, and the task will
hand gesture recognition is the translation of become much easier. Human hand recognition is used in
sign language. The importance of hand gesture image processing, network security, robotics, and many
recognition has increased due to the prevalence mother fields. Gesture recognition has been a very active
of touchless applications The software research topic lately. As the interest and value of gesture
proposed in this project is created using detection increase, experiments are performed to validate
Python, NumPy, Open-CV, Tk-inter, labeling, application and system results. Many countries have their
and TensorFlow. Recent research has proved own standards for sign language gestures and
the supremacy of Convolutional Neural interpretation. For example, the alphabet of Korean Sign
Networks (CNN) for image representation and Language is different from that of Indian Sign Language. It
classification. the CNN model, the provided highlights the richness of sign languages, but also their
image or video will be classified as the ambiguity. Deep learning requires mastery of gestures to
respective Alphabet or number from the achieve reasonable accuracy. Any technique can be used to
American Sign Language Set. The proposed identify the logo.
system is evaluated on a very challenging
dataset, which consists of 11 dynamic hand II. LITERATURE OVERVIEW
gestures. The results show that the proposed Hand gestures are a form of nonverbal communication
system outperforms state-of- the-art widely used in deaf communication, robotic control,
approaches, demonstrating its effectiveness. human- computer interaction (HCI), home automation, and
The model was trained on 800 images and medical applications. Gesture-based research papers use
tested on 400 images. The model with many different techniques, including those based on
augmented data achieved an accuracy of instrumented sensor technology and computer vision.
86.75%. Gesture recognition can be recognized through human
movement. But the automatic recognition of human
Keywords: Convolutional Neural Networks, gestures is a big challenge. This article reviews some
OpenCV, Computer vision, Deep Learning, previous research. In this article, we focus on creating the
TensorFlow, and Gestures. most accurate way to communicate or interact between
computers and humans. We used a
I. INTRODUCTION A webcam-based approach where user-provided gestures
Gestures are an aspect of body language that can be are captured, processed, than predicted by the machine, and
conveyed by the center of the palm, the position of the then disabled for specific gestures. For the construction of
fingers, and the shape of the hand. Other existing systems the project model, we used the CNN algorithm, but to draw
ignore the local configuration of the fingers and only this conclusion, which model to use and why, we studied
consider the global configuration of the body. These some previous research papers. In addition, the price of
systems have been used successfully in some HCI such equipment is quite high. For this investigation, they
Copyright to IJARSCT DOI: 10.48175/568 16
www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

used a different approach, based on computer vision of appropriate, and applicable technology because it enables
gestures. contactless communication between humans and computers.
Human arm motion recognition using deep reinforcement The cameras are available in different configurations such as
learning in this approach, they used a deep reinforcement monoculars, fisheye, TOF, and IR. However, this technique
algorithm. 2019 Jaya Prakash Sahoo, Simit Ari, Sarat involves several challenges, including lighting changes,
Kumar Patra Gesture recognition using PCA. Deep CNN background issues, occlusion effects, complex backgrounds,
reduced functions and SVM classifier authors in this model and trade-off processing time for resolution and frame rate
used a Support Vector Machine (SVM) model). In 2018 pictures and show foreground or background objects that
Guillaume DeVine au, Wang Xi, and Fabien Mortared, and have the same skin colour or otherwise appear as hands. The
Jie Yang used Deep Learning Skeletal Data Gesture article [2] studies deep reinforcement learning methods.
Recognition In this model, they use a CNN to create a Algorithms using this approach learn patterns from sensors
skeletal model of the hand. using only reward feedback without class labels. It allows
Year Author Title Methodologies users to control IoT devices and generate desired arm
2020 Munir Hand Gesture It uses a model movement patterns without creating beacons. In this paper,
Oudh, Ali Recognition of the the performance of a convolutional neural network (CNN)
Al-Nnaji Based on instrumental with a DQN model is compared to that of an LSTM model
and Javan Computer glove with DQN.
Chahil Vision approach. Reinforcement learning has become very popular since the
2018 W.Soek, Recognition of In this appearance of AlphaGo, an artificial intelligence Go
Y.Kim, Human Arm approach, they program developed by Google Deep Mind. Therefore, in
C.Park. Movement have used a this investigation, we found that the model used is a deep
Using Deep deep reinforcement algorithm, which is too advanced for the
Reinforcement reinforcement system and requires a huge data set. Deep features are
Learning algorithm. extracted from the fully connected layers of the pre-trained
2019 Jaya Prakash Hand Gesture In this model, Alex Net. The PCA dimensionality reduction technique is
Sahoo, Simit Recognition they have used then used to reduce redundant features in the feature vectors.
Ari, Sarat using PCA- the support The proposed technique does not require any manual
Kumar Patra based Deep vector machine segmentation or localization technique. However, in this
CNN Reduced (SVM) model. survey, support vector machines (SVM) are used for image
Features and classification, which is not good for large terabyte datasets.
SVM Classifier In the article [4], they proposed a New 3D gesture
Author recognition method based on a deep learning model. In their
2018 Guillaume Deep Learning In this model, paper [1], they propose a novel convolutional neural
Devineau, for Hand they created a network (CNN) in which sequences of skeletal hand joint
Wang Xi Gesture hand-skeletal positions are processed by parallel convolutions; we then
and, Fabien Recognition on model using study the performance of this model on the performance of
Moutardel Skeletal Data CNN. the gesture sequence classification task. The model only
and Jie Yang uses hand skeletal data, not depth images. Thus, through all
The authors of the article [1] used an approach based on an these studies, we have successfully used deep learning to
instrumented glove model since these wearable sensors can create models through convolutional neural networks.
be used to capture hand movement and position.
Additionally, they can easily provide precise coordinates of III. PROBLEM STATEMENT
palm and finger position, orientation, and configuration Suppose you work as a data scientist in a home electronics
using sensors on the glove. However, this method requires company that produces modern smart TVs. You want to
the user to be physically connected to the computer, which develop a cool feature in your smart TV that can recognize
reduces the convenience for the user to interact with the five different gestures made by the user, which will help the
computer. In this investigation, they used a different user to control the TV without using a remote control. A
approach, namely that camera vision sensors are a common, webcam mounted on the TV constantly monitors gestures.
Each gesture corresponds to a specific command: Thumbs
Copyright to IJARSCT DOI: 10.48175/568 17
www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

up: Increase volume, Thumbs up: Decrease volume, Swipe

left: go back 10 seconds, Swipe right: go forward 10
seconds, Stop put the movie paused.

IV. RESULT AND ANALSIS

The first article describes how a wearable glove-based
sensor was used to capture hand movement and position.
However, this method requires the user to be physically
connected to the computer. However, modern glove-based
approaches use touchscreen technology, a more promising
technology considered for industrial-grade haptics.
The sensor they created based on electronic camera vision is
therefore a universally applicable technology as it enables
contactless communication between humans and machines.
However, this technique involves several challenges,
including lighting changes, background issues, occlusion
effects, complex backgrounds, and trade-offs between
processing time, resolution, and frame rate. pictures. picture. V. CONCLUSION
images, and the appearance of skin tones or hands displayed It discusses different approaches to hand tracking, webcams,
in the foreground or background. In the second paper, they wireless technologies, CNNs, OpenCV, deep learning, and
introduced the deep reinforcement learning method, which Tensor Flow. After analyzing five articles on gesture
uses deep reinforcement learning algorithms to identify recognition from different authors, we concluded that the
human arm movement patterns using IoT sensors. Recent article Hand Gesture Recognition Using Convolutional
studies have investigated supervised learning-based Neural Network is the best. Experimental results show that
approaches such as CNNs and RNNs to implement HCI the proposed system outperforms all other search methods in
devices. terms of recognition rate, demonstrating its effectiveness.
In this paper, the performance of a convolutional neural
network (CNN) with a DQN model is compared to that of a VI. ACKNOWLEDGMENTS
long-short-term memory (LSTM) model with DQN. The We would like to thank the Department of Information
results show that the CNN-based DQN model is more stable Technology, Zeal School of Engineering and Research.
than the LSTM-based model, with a classification accuracy Thanks to Prof. Nilam Humane for her encouragement and
of 98.33% in predicting arm motion patterns. In the third help. We would like to express our sincere gratitude to co-
paper, they classify gesture poses using a classifier based on ordinator Dr. T. Praveen Blessington, Professor, Department
a support vector machine (SVM) with a linear kernel. In this of Information Technology, for his coordination, and
analysis, the variance of the sum of deep features greater encouragement throughout this project. We are especially
than 99.9% are considered redundant features. The final thankful to Professor Balaji Chaugule, Head of the
reduction function studies gesture recognition performance. Information Technology Department, for his valuable advice
The tabular results show that the performance of the and unflinching support in ensuring the completion of this
proposed technique is 87.83%. which is better than the project.
average accuracy result of the Alex-Net only "FC6"
function. They introduce a new 3D gesture recognition REFERENCES
method based on deep learning models. [1]. Guillaume DeVine au and Wang Xi and Fabien
Moutarde1 and Jie Yang, DEEP Learning for Hand
Gesture Recognition on Skeletal Data [2018].
[2]. Dhal, Shubham Vashist, Garima Aggarwal,
Automated Hand Gesture Recognition using a
Deep Convolutional Neural Network [2018].
[3]. W. Seok, Y. Kim, C. Park, Recognition of Human
Arm Movement Using Deep Reinforcement
Copyright to IJARSCT DOI: 10.48175/568 18
www.ijarsct.co.in
 ISSN (Online) 2581-9429
IJARSCT
International Journal of Advanced Research in Science, Communication and Technology (IJARSCT)

Volume 3, Issue 1, April 2023

Impact Factor: 7.301

Learning [2018].
[4]. Jaya Prakash Sahoo, Simit Ari, Sarat Kumar Patra,
Hand Gesture Recognition using PCA-based Deep
CNN Reduced Features and SVM classifier [2019].
[5]. Munir Oudh, Ali Al-Nnaji and Javan Chahil, Hand
Gesture Recognition Based on Computer Vision: A
Review of Techniques [2020].

Copyright to IJARSCT DOI: 10.48175/568 19

www.ijarsct.co.in