International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 09 Issue: 02 | Feb 2022 www.irjet.net p-ISSN: 2395-0072

A Survey Paper on Controlling Computer using Hand Gestures

Pradnya Kedari1, Shubhangi Kadam2, Rutika Chaudhary3, Rajesh Prasad4
1Student, Department of Computer Science and Engineering, MIT School of Engineering, MIT Art, Design and
Technology, Pune, Maharashtra, India
2Student, Department of Computer Science and Engineering, MIT School of Engineering, MIT Art, Design and

Technology, Pune, Maharashtra, India

3Student, Department of Computer Science and Engineering, MIT School of Engineering, MIT Art, Design and

Technology, Pune, Maharashtra, India

4Professor, Department of Computer Science and Engineering, MIT School of Engineering, MIT Art, Design and

Technology, Pune, Maharashtra, India

---------------------------------------------------------------------***----------------------------------------------------------------------
Abstract - The availability of the cheap webcams with, at OpenCV is a real-time open-source computer vision and
least, sufficient qualities open up new chances concerning the image-processing library. We’ll use it via the OpenCV python
implementation of human computer interaction (HCI) package.
interfaces. Gesture is one among the foremost detailed and
powerful approach of communications between human and There has been a special significance recently on Human
computer. Hence, there has been a growing interest to form Computer Interaction (HCI) study to form convenient
feasible interfaces by directly using the natural interfaces by directly using common communication and
communication and management capability of humans. Hand handling expertise of humans. Among different chassis parts,
gesture recognition system contains a decent surveillance the hand is the most helpful interaction tool, because of its
present days thanks to simple and straightforward adroitness. The word gesture is employed for several
intercommunication between human and machine. The main different cases involving human motion particularly of the
target of developing hand gesture is to make a much better hands, arms, and face, only some of these are co-operative or
conveyance between human and computer for delivering informative.
information.
Gesture recognition is a crucial field that specify human
Hand gestures are a kind of nonverbal way to interact that gesture using computer vision techniques and algorithms.
may be employed in several fields. Research and survey papers There are numerous bodily motion which may develop
supported hand gestures have acquire so many alternative gesture however the usual type of gesture generation arises
techniques, including those supported on sensor technology from the face and hands. The complete policy of tracing
and computer vision. This paper mainly focuses on a review of gesture to their representation and changing them to some
the related work readily available hand gesture techniques useful command is called as gesture recognition. Various
and introduces their excellence and restrictions under technologies has been used for the design and
different situations. This paper could be a general summary of implementation of such kind of application.
hand gesture implementation. It shows all methods that were
employed for hand gesture recognition in numerous research The CNN or convolutional neural networks are the extremely
papers. The aim of this study is to perform a scientific popular used technique for image classification domain. An
literature review for identifying the foremost prominent image classifier takes an image, or sequence of images that is
techniques, applications and difficulties in controlling video as an input and classifies it into one of the possible
computer using hand gestures. categories or classes that it was trained to recognize. They
have applications in several different fields like driver less
Key Words: hand gestures, computer vision, image cars, defense, education, medical field, fraud detection, etc.
processing, gestures recognition, human-computer There are many algorithms for image classification and also
interaction. there are some challenges like data overfitting, environment
like background color, structure, etc.
1. INTRODUCTION
2. LITERATURE REVIEW
Gesture recognition is a full of life analysis field in Human
Computer Interaction technology. It has several The initial approach of communication with computer
employments in virtual environment management, medical employing hand gesture was first projected by Myron W.
applications, sign language translation, robot control, music Krueger in 1970 [1]. The purpose of the approach was
creation, or home automation. During this project attained and also the mouse cursor control was
Controlling a Computer using Hand Gestures, we are going to accomplished using an external webcam (Genius FaceCam
make a real-time application using OpenCV and Python. 320), a software package that would paraphrase hand

© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1050
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 02 | Feb 2022 www.irjet.net p-ISSN: 2395-0072

gestures and so turned the acknowledged gestures into OS interactive learning, immersive gaming, interacting with
commands that handled the mouse operations on the display virtual objects on screen.
screen of the computer [2]. Selecting hand gesture as an
interface in HCI will permit the implementation of a good Arduino UNO and ultrasonic sensors based hand gesture to
vary of applications with none physical contact with the control a computer where they can play pause videos and
computing environments [3]. Nowadays, majority of the HCI scroll up and down pages [11]. This paper [12] suggest an
relies on devices like keyboard, or mouse, however an efficient ultrasonic based hand gesture monitoring system
enlarging significance in a category of techniques based on which is designed with the help of Arduino microcontroller
computer vision has been came out because of skill to ATMEGA32. The hand gestures recognized effectively with
acknowledge human gestures in a habitual manner [4]. ultrasonic sensors. It is proved that no extra hardware is
needed to identify hand gestures and proved that simple
The primary aim of gesture recognition is to spot a specific inexpensive ultrasonic sensors can be employed to find
human gesture and carry information to the computer. several ranges to identify hand gestures. In this paper [13],
General objective is to create the computer acknowledged Arduino UNO ARDUINO and python programming with
human gestures, to manage remotely with hand poses a good wired ultrasonic sensor based hand gesture system to
sort of devices [5]. The automated vision-based recognition control a computer where they can zoom in/out and rotate
of hand gesture for management of tools, such as digital TV, the image is developed. This is successful trial of working of
play stations and for sign language was take into account as hand motion sensing system using sensors i.e. ultrasonic
a significant exploration topic lately. However the common sensors and finger contact sensors and using in it to Arduino
issues of those works arise because of several problems, like kits in wireless mode using radio frequency. HCI for MS
the complicated and disturbing environments, tone color of office and media player and have their own dataset, used
skin and also the kind of static and dynamic hand gestures. skin coloured based technique [14]. Application that switch
Hand gestures recognition for TV management is suggested in a Web browser, Web page scrolling, Task switching,
by [6]. During this system, just one gesture is employed to Changing the slides of the presentation, Play/pause the
regulate TV by operating user hand. On the display, a hand video, Video forward and rewind is implemented. Arduino,
icon seems that follows the hand of user. In this paper [7], PySerial, PyautoGUI, etc. used [15].
the actual HCI system that based on gestures and accept
gestures uniquely operating one monocular camera and This project is built in order to make smart home appliance
reach out the system to the HRI case has been evolved. The system. Two deep CNN architectures are evolved in this
came out system depends on a Convolution Neural Network system which are revised from AlexNet and VGGNet,
classifier to grasp features and to acknowledge gestures. respectively [16]. Done the implementation of the system
using Convolutional Neural Network and Back Propagation
The Hidden Markov Model delivers as a crucial tool for the methodologies. They built a gesture controlling and hand
recognition of dynamic gestures in real time. The method recognition system for the one who are disables [17]. In this
employed HMM, works in actual and is built to operate in paper they have used the surface electromyography (sEMG)
static environments. The approach is to make the use of LRB sensors with wearable hand gesture devices and mostly
topology of HMM in association with the Baum Welch applied classifier is Artificial Neural network for sign
Algorithm for training and also the Forward and Viterbi language hand gesture. In this authors faced overfitting
Algorithms for testing and checking the input finding problem in the dataset [18]. The vision based real-time
sequences and producing the most effective attainable state system with Python programming language and OpenCV
sequence for pattern recognition [8]. In this paper [9], the libraries and Linux framework was implemented. A real-
system is designed even it appears to be easy to use as time human gesture recognition using an automated
compared to latest system or command based system technology called Computer Vision is demonstrated using
however it is less powerful in spotting and recognition. Linux operating system and Lenet is a CNN architecture used
Require to upgrade the system and attempt to construct for training of the gestures [19]. This approach is relied on
further strong algorithm for both detection and recognition image comparison and motion recognition mechanism to try to
despite of the confused background environment and a usual do mouse indicator activities and make choice of the icon.
lighting environment. Also require to upgrade the system for
several additional categories of gestures as system is built A Virtual gesture control mouse is an approach is developed
for just six classes. However this system can use to manage to help the cursor of the mouse and perform its operations
applications like power point presentation, games, media with the help of real-time camera. This method is also built
player, windows picture manager etc. In this paper [10], on the basis of image comparison and motion recognition
hand gesture laptop makes the use of an Arduino Uno, mechanism to try to do mouse indicator operations and
Ultrasonic sensors and a laptop to perform the activities like choice of icons [20]. Developed a system that assists user
controlling media, playback and volume. Arduino, Ultrasonic friendly interaction such as full-body game and system
sensors, Python used for serial connection. This type of generating a three-dimensional environment. It is an
technology can be employed in the classroom for easier and exploratory study on the gesture selection for operations
and also settled an information recapture system to address
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1051
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 02 | Feb 2022 www.irjet.net p-ISSN: 2395-0072

several doubt phase of users [21]. Hyperparameter of CNN This paper [32] have made the use of various features and
structures which are relied on Alex Net model, are advanced multiclass SVM model to recognize and trace uncover hand,
by heuristic optimization algorithms. This suggested and to manage an system by command produced by a
approach is tried out on gestures language digits and at the grammar in a complex background environment, via skin
same time Thomas Moeslund’s gesture detection datasets. In tone color detection and outline curves comparison
this paper they achieved the 94.2% accuracy. algorithm. Also K-means clustering algorithm and scale
Simultaneously, for Thomas Moeslund’s gesture recognition invariance feature transform (SIFT) have used to retrieve the
dataset, the established method got 98.09% average important attributes from the images that have trained. But,
accuracy performance and 94.33% average recognition the segmentation and localization technique is not clear for
performance [22]. In this paper, they have built an algorithm the application and also there is no diligent geometric details
for real-time hand gesture recognition using convolutional of the object components. In [33], a system has been
Neural Network (CNN) and they got an average accuracy of suggested that primarily capture the human hand
98.76% on the dataset that they have proposed. Dataset movements that is gestures into the disk and then transform
consists of total nine hand gestures and 500 image samples them into binary image by deriving frame from each video
for each gesture [23]. A wireless hand gestured controlled one by one and then form 3D Euclidian space for binary
fan system is formed. Gyro sensor is employed so as to work image, for classifying hand gestures. For training part they
out the modification in co-ordinates of the hand. Also used a have used back propagation algorithm and supervised feed-
microcontroller and particularly an Arduino in which coded forward neural network which is convenient for just very
functions have been run. Hand gesture detection achieved easy type of gesture and having not much complex
with testing accuracy of 98.61% [24]. To build the gesture background. This paper [34] proposed hand gesture
detection system and also to minimize the unnecessary recognition using three dimensional depth sensors. There
information of EMG signals and optimize the dimension of are various form of fields including static hand motion, 3D
the signal, the Principal Component Analysis (PCA) and hand modeling, and hand route gesture within this
GRNN neural network are employed. In this system they application. This paper focused on gesture identification
have taken total nine static gestures as input observations techniques and in which domains those methods are
and made the use of electromyography instrument to employed is also described. This paper [35] suggests a
examine the characteristics of the signal. The all in all method for detecting the uncover hand with none color cap.
recognition accuracy of the system came to 95.1% [25]. The RGB image is modified into Hue image. The technique
which removes arm region, the hand skin segmentation
This paper [26] suggest a quick, easy and powerful gesture technique is employed for localization and segmentation. As
recognition algorithm for automation application that in [36], they have proposed a hand gesture identification
automatically acknowledges a restricted samples of gestures. method for home automation using depth sensor. This
However, the segmentation method ought to be strong and technique have mainly two steps that is in initial step the
needed to be cope with temporal trailing, obstruction and dataset is formed for application and in second step,
three dimensional modelling of hand. In paper [27], they different attributes are derived from the labeled hand parts
have proposed hand gesture for controlling power point and that are employed for providing command to system.
VLC media player. This paper [28] thought about the
advanced design of hand segmentation that escapes the 3. CONCLUSIONS
confused classification of objects on kalman filtering and TSL
color space that offer high accuracy. This method gets a I With the growth of present technology, and as humans
sensible hand segmentation. This paper [29] targeted on generally makes the use of hand movements that is hand
vision based recognition system. Skin tone color model and gestures in the daily communication in order to make
thresholding method with template matching using PCA intentions more clear, hand gesture identification is treated
were the parameters on which database of hand gesture was to be a crucial portion of Human Computer Interaction (HCI),
based on. The primary step is hand segmentation which is which provides devices the capability of detecting and
done by applying the skin color model, followed by the classifying hand gestures, and perform activities
second step that is differentiate the image of hand from subsequently. Research and analysis in the field of hand
background environment. In last step, PCA, a format based gestures has become more popular and exciting. It also
matching is formed in order to identification of gesture. allows a way of natural and simple interaction. Standard
Vision based hand gesture recognition techniques from interactive techniques based on several tools like a mouse,
surveys of past 16 years are analyzed. Also this paper keyboard/touch pad, or touch screen, joystick for gaming
analyses 26 publicly accessible hand gesture datasets [30]. and consoles for system management.
This paper [31] implemented hand gesture recognition
application for HCI that was relied on vision based method. In this paper we have discussed the overall review of gesture
They established real time system to dam the mouse’s acquisition methods, the feature extraction process, the
movements in windows by without using ANN training and classification of hand gestures, the challenges that face
employing detection technique which was color based. researchers in the hand gesture recognition process.

© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1052
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 02 | Feb 2022 www.irjet.net p-ISSN: 2395-0072

Hand gesture recognition used in many applications like HCI, [13] Sarita K., Gavale Yogesh, S. Jadhav, “HAND GESTURE
robotics, sign language, digit and alphanumeric value, home DETECTION USING ARDUINO AND PYTHON FOR
automation, etc. In this survey paper we have discussed in SCREEN CONTROL”, International Journal of
brief on basic methods of hand gesture recognition and find Engineering Applied Sciences and Technology, 2020,
that Arduino and ultrasonic sensor is used widely in Vol. 5, Issue 3, ISSN No. 2455-2143, July 2020.
comparison of vision based technology. Also can conclude [14] Ram Pratap sharma, Gyanendra Varma, “Human
that recognition of static hand gesture needs less computer interaction using hand gesture”, 2015.
computation gesture in comparison to dynamic hand. Hand [15] J. S. Vimali, Senduru Srinivasulu, J. Jabez, S. Gowr, “Hand
gestures recognition provides an interesting interaction field Gesture Recognition Control for Computers Using
in a several different computer applications. Arduino”, 2020.
[16] Chung, Hung-Yuan, Yao-Liang Chung, and Wei-Feng Tsai.
REFERENCES "An efficient hand gesture recognition system based on
deep CNN." 2019 IEEE International Conference on
[1] M. KRUEGER, Artificial reality II, Addison-Wesley Industrial Technology (ICIT). IEEE, 2019.
Reading (Ma), 1991. [17] Varun, Kollipara Sai, I. Puneeth, and T. Prem Jacob.
[2] Horatiu-Stefan Grif and Cornel Cristian Farcas, "Mouse "Hand gesture recognition and implementation for
Cursor Control System Based on Hand Gesture", 9th disables using CNN’S." 2019 International Conference on
International Conference Interdisciplinarity in Communication and Signal Processing (ICCSP). IEEE,
Engineering, INTER-ENG 2015, 8-9 October 2015, Tirgu 2019.
Mures, Romania. [18] Yasen, Mais, and Shaidah Jusoh. "A systematic review on
[3] H.A JALAB, "Static hand Gesture recognition for human hand gesture recognition techniques, challenges and
computer interaction", Asian Network for Scientific applications." PeerJ Computer Science 5 (2019): e218.
Information technology journal, pp. 1-7, 2012. [19] Suresh, Malavika, Avigyan Sinha, and R. P. Aneesh. "Real-
[4] J C.MANRESA, R VARONA, R. MAS and F. PERALES, Time Hand Gesture Recognition Using Deep Learning."
"Hand tracking and gesture recognition for human- International Journal of Innovations and
computer interaction", Electronic letters on computer Implementations in Engineering 1 (2019).
vision and image analysis, vol. 5, pp. 96-104, 2005. [20] Shibly, Kabid Hassan, et al. "Design and development of
[5] H. HASAN and S. ABDUL-KAREEM, "Static hand gesture hand gesture based virtual mouse." 2019 1st
recognition using neural networks", Artificial International Conference on Advances in Science,
Intelligence Review, vol. 41, pp. 147-181, 2014. Engineering and Robotics Technology (ICASERT). IEEE,
[6] W. FREEMAN and C. WEISSMAN, "Television control by 2019.
hand gestures", Proc. of Intl. Workshop on Automatic [21] Wu, Huiyue, et al. "The gesture disagreement problem in
Face and Gesture Recognition 1995, pp. 179-183, 1995. free-hand gesture interaction." International Journal of
[7] Pei Xu, "A Real-time Hand Gesture Recognition and Human–Computer Interaction 35.12 (2019): 1102-1114.
Human-Computer Interaction System", [22] Ozcan, Tayyip, and Alper Basturk. "Transfer learning-
arXiv:1704.07296v1 [cs.CV] 24 Apr 2017. based convolutional neural networks with heuristic
[8] J.R Pansare, Malvika Bansal, Shivin Saxena, Devendra optimization for hand gesture recognition." Neural
Desale, "Gestuelle: A System to Recognize Dynamic Hand Computing and Applications 31.12 (2019): 8955-8970.
Gestures using Hidden Markov Model to Control [23] Zhan, Felix. "Hand gesture recognition with convolution
Windows Applications", International Journal of neural networks." 2019 IEEE 20th International
Computer Applications (0975 – 8887) Volume 62– Conference on Information Reuse and Integration for
No.17, January 2013. Data Science (IRI). IEEE, 2019.
[9] Ram Pratap Sharma and Gyanendra K. Verma, "Human [24] Alon, Alvin Sarraga, and Julie Ann B. Susa. "Wireless
Computer Interaction using Hand Gesture", Eleventh Hand Gesture Recognition for an Automatic Fan Speed
International Multi-Conference on Information Control System: Rule-Based Approach." 2020 16th IEEE
Processing-2015 (IMCIP-2015). International Colloquium on Signal Processing & Its
[10] Rohit Mukherjee, Pradeep Swethen, Ruman Pasha, Applications (CSPA). IEEE, 2020.
Sachin Singh Rawat, “Hand Gestured Controlled laptop [25] Qi, Jinxian, et al. "Surface EMG hand gesture recognition
using Arduino”, International Journal of Management, system based on PCA and GRNN." Neural Computing and
Technology And Engineering, October 2018. Applications 32.10 (2020): 6343-6351.
[11] Ayushi Bhawal, Debaparna Dasgupta, Arka Ghosh, [26] A. Malima, E. Ozgur and M. Cetin, "A Fast Algorithm for
Koyena Mitra, Surajit Basak, “Arduino Based Hand Vision-Based Hand Gesture Recognition For Robot
Gesture Control of Computer”, IJESC, Volume 10, Issue Control.Z " IEEE Signal Processing and Communications
No.6, June 2020. Applications, pp. 1–4, (2006).
[12] Udit Kumar, Sanjana Kintali, Kolla Sai Latha, Asraf Ali, N. [27] Jalab, Hamid A., and Herman K. Omer. "Human computer
Suresh Kumar, “Hand Gesture Controlled Laptop Using interface using hand gesture recognition based on
Arduino”, April 2020. neural network." In Information Technology: Towards

© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1053
 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 09 Issue: 02 | Feb 2022 www.irjet.net p-ISSN: 2395-0072

New Smart World (NSITNSW), 2015 5th National

Symposium on, pp. 1-6. IEEE, 2015.
[28] Mo, Shu, Shihai Cheng, and Xiaofen Xing. "Hand gesture
segmentation based on improved kalman filter and TSL
skin color model." In Multimedia Technology (ICMT),
2011 International Conference on, pp. 3543-3546. IEEE,
2011.
[29] Ahuja, Mandeep Kaur, and Amardeep Singh. "Static
vision based Hand Gesture recognition using principal
component analysis." In MOOCs, Innovation and
Technology in Education (MITE), 2015 IEEE 3rd
International Conference on, pp. 402-406. IEEE, 2015.
[30] Pisharady, Pramod Kumar, and Martin Saerbeck.
"Recent methods and databases in vision-based hand
gesture recognition: A review." Computer Vision and
Image Understanding 141 (2015): 152-165.
[31] Chetan Dhule, Trupti Nagrare, "Computer Vision Based
HumanComputer Interaction Using Color Detection
Techniques." Fourth International Conference on
Communication Systems and Network Technologies,
pp.934-938, 2014.
[32] N. H. Dardas and N. D. Georganas, "Real-Time Hand
Gesture Detection and Recognition using Bag-of-
Features and Support Vector Machine Techniques", IEEE
Transaction on Instrumentations and Measurement, vol.
60(11), pp. 3592–3607, November (2011).
[33] Murthy and Jadon, "Hand Gesture Recognition using
Neural Networks," 2nd IEEE International Advance
Computing Conference (IACC), pp. 134–138, (2010).
[34] Cheng, Hong, Lu Yang, and Zicheng Liu. "Survey on 3D
hand gesture recognition." IEEE Transactions on Circuits
and Systems for Video Technology 26, no. 9 (2016):
1659-1673.
[35] Xu, Yishen, Jihua Gu, Zhi Tao, and Di Wu. "Bare hand
gesture recognition with a single color camera." In
Image and Signal Processing, 2009. CISP'09. 2nd
International Congress on, pp. 1-4. IEEE, 2009.
[36] Dong-Luong Dinha, Jeong Tai Kimb, and Tae-Seong
Kimc, "Hand Gesture Recognition and Interface via a
Depth Imaging Sensor for Smart Home Appliances," 6th
International Conference on Sustainability in Energy and
Buildings, SEB-14, pp. 576 – 582, 2014.

© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 1054