Gesture Recognition System

Prerna Sharma Naman Sharma

Computer Science Engineering, Computer Science Engineering,
Krishna Engineering College, Inderprastha Engineering College,
Ghaziabad-201007, India. Ghaziabad-201010, India.
[email protected] [email protected]

Abstract. Mostly communication in current scenario is eigenvectors of AAT , and columns of V are orthonormal
happened via vocal sounds with body language gestures. Vocal eigenvectors of ATA.
sound play an important role in communication but at the same
time, various body language expressions add more importance to
communication. Even in few cases, body language played a very B. PCA
important role like communication between deaf & dumb people
or traffic signal etc. In this paper, a method is defined for Principal Component Analysis is a linear transformation
recognizing Gesture and Posture. This approach is tested on 6 method used in statistical techniques. This method used for
gesture classes and 6 posture classes. data dimension reduction and feature extraction. It defines
through the multiplication of two matrices, which shows the
Keywords: SVD-PCA approach, Posture Recognition, Gesture co-relation between various features. [10]
Recognition, Feed Forward neural Network.
Theorem is presented as follows:
Y=P’X (2)
I. INTRODUCTION here X is correlated variable matrix, Y is uncorrelated
variable matrix which is evaluated by multiplying X and co-
Body gestures have broad range of gestures and variance matrix, P'.
postures like eye movements, variation in the pitch of vocal
sounds etc. Generally body language involves hand
movements. Gestures performed through hands gives a C. Feed-Forward Neural Network
better expression to your words like representation of a ―An ANN consists of a sequence of layers; each layer
number, expressing any feeling etc. Gestures are important consists of a set of neurons. All neurons of every layer are
tool of communication in any sign language or gesture linked by weighted connections to all neurons on the
based machine control. So, an algorithm is designed which preceding and succeeding layersǁ. [9]
works on recognizing gestures .
This recognition system can also be used in various areas Every neuron in network can receive the signal and
like Tele-presence system [13], 3D animation system [12], process that signal and generate the output signal. Neurons
sign language recognition [11], Virtual reality [14] and are interconnected to each other, and every connection is
many more. calculated through a real number, called weight coefficient,
Application of this work is in area of HCI (Human that shows the degree of importance of that connection in
Computer Interaction) based system. For extracting the network.
features of input images, Principal Component Analysis
(PCA), Singular value Decomposition (SVD) methods are
used and neural network is trained using these features for
classifying the gestures.

A. SVD
SVD is an approach used for extracting the silent
features of image used for data dimension reduction and
training purposes. ―SVD is based on mathematical theorem
which states that a rectangular matrix A can be broken down
into product of three matrices - an orthogonal matrix U, a
diagonal matrix S, and the transpose of an orthogonal matrix
Vǁ. [10] Fig. 1. ANN Architecture
The equation of this theorem is generally presented like:
T Two functions are defined for predicting the behavior of any
A =U SV
mn mm Mnnn neuron in any particular layer
(1)
1. Network Input function
here UTU = I, VTV = I; S is diagonal matrix containing
square roots of eigen values from U or V. U are orthonormal 2. Network Output function

Authorized licensed use limited to: Motilal Nehru National Institute of Technology. Downloaded on December 15,2024 at 15:29:57 UTC from IEEE Xplore. Restrictions apply.
978-1-7281-1253-4/19/$31.00 © 2019 IEEE
 Output function, Can be calculated as on network layer i, network match these features with the trained images and
weight wi , input xi is defined as follows: classify the gestures.

neti = ∑ wixi + bias (3)

IV. ALGORITHM
Following steps represent the proposed method:
II. RELATED STUDIES 1. Input Dataset
Licsar [4] introduced a system for hand gesture Input dataset is prepared that shows an image containing
recognition on vision based approach used Fourier hand and upper body portion and through this image skin
Descriptor algorithm for classifying of static gestures in pixels and edges are detected which used in gesture-posture
hand images only and remove the background using recognition.
Background Subtraction method.
2. Basic Processing Methods
A Survey paper [6] published on gesture recognition. In a) Gesture Recognition
that survey, they discussed on various gesture recognition
Skin pixels are detected using RGB color space and then
techniques using different algorithms like neural networks, some specific morphological operations are performed.
HMM (Hidden Markov Model), particle filtering, etc. Hu
[7] explained a method for pose recognition using joint edge b) Posture Recognition
detection. He detected face, torso and skin pixels and detects Gray conversion is done on input image and then canny
proper joint locations in input image and markov chain-Monte edge detection algorithm is used for detection of edges and
Carlo Model is used for recognizing the posture. contours are obtained.

Nguyen [8], 2013 explained a method for gesture 3. Feature Extraction

recognition usinh neural network model in static input
images. He used background subtraction and skin pixel a) Gesture Recognition: SVD-PCA method is used to
detection techniques to extract the main reason in image and extract the feature which used signal matrix, eigen vectors
implement median filtering to remove the noise and of input image which is used to train the network and
recognize the gesture. classifying purpose. A image of SVD features is shown
below:

III. PROPOSED METHODOLOGY

A novel method is defined and implemented of
recognizing the gesture and body pose recognition using
SVD-PCA approach and feed forward artificial neural
network. Methodology is explained below clearly:

1. Posture Recognition System

In case of recognizing postures, firstly two types of pose
images are taken, one for train purpose and other for testing
purpose. Then system converts the images into gray levels
Fig. 2. SVD-PCA features of gesture class
and detects the edges. After that segmentation is applied
using thresholding technique and tracing the contours to
extract the silent features of input image. And features are b) Posture Recognition
extracted using SVD-PCA approach and features of train SVD-PCA features are extracted; for each class,
dataset images is used to train the neural network and features are calculated through SVD-PCA approach and
features of test dataset is given as an input to the network, so used for training and classifying. Following graph show the
that it can identify the features and match them with the SVD-PCA features for a posture class:
trained image features and classify the postures.

2. Gesture Recognition System

In case of recognizing gestures, similarly as the above
system, trained hand dataset images are taken and then hand
is detected through skin detection technique. After that
various morphological operation are performed on that
image to improve the quality so that it can clearly show the
skin pixels. Now Features are extracted from trained image
dataset using SVD-PCA approach and use that features to
train the network. In case of testing the system, test dataset
hand images are taken and apply the same procedure till
morphological operations and after extracting features,
Fig. 3. SVD-PCA features of posture class

Authorized licensed use limited to: Motilal Nehru National Institute of Technology. Downloaded on December 15,2024 at 15:29:57 UTC from IEEE Xplore. Restrictions apply.
978-1-7281-1253-4/19/$31.00 © 2019 IEEE
 V. RESULTS REFERENCES
Results are calculated on the basis of confusion [1] Ibraheem N.A., Khan R.Z.,: Vision based gesture recognition using
matrices generated by the neural network. And here, we neural networks approaches: a review. International Journal of human
Computer Interaction (IJHCI). 3(1), (2012).
formalize them in tabular method which shows the correct
[2] Symeonidis K.,: Hand Gesture Recognition Using Neural Networks,
recognition percentage of detected gestures and postures. MS Thesis. University of Surrey, UK, (2000).
[3] Ranganath C.W. Ng, S.,: Real-time gesture recognition system and
application, Image and Vision Computing, Vol. 20, Issues 13-14, pp.
993-1007, (2002).
Dataset No. of Classes Accuracy
[4] Licsar, Sziranyi T., : Dynamic training of hand gesture recognition
system, IEEE, 0-7695-2128-2/04, (2004).
Gesture 6 95.9 %
[5] Ivekovic S., and Trucco E.,: Human body pose estimation with PSO.
IEEE Congress on Evolutionary Computation Sheraton Vancouver
Posture 6 90.3 % Wall Centre Hotel. Vancouver, BC, Canada, (2006).
Table 1. Accurate Results [6] Mitra S., Acharya T., : Gesture Recognition – A survey, IEEE
Transactions on systems, man, and cybernetics—Part C: Application
and Reviews, Vol. 37, No. 3, (2007).
[7] Hu Z., Wang G., Lin X., Yan H., : Recovery of upper body poses in
VI. CONCLUSION static images based on joints detection. Pattern Recognition Letters
(30). pp. 503– 512, (2009).
Proposed methodology recognized specific postures [8] Nguyen T. N., Huynh H. H., : Static hand gesture recognition using
artificial neural network, Journal of Image and Graphics, Volume 1,
and gestures in any body image. Results are more accurate and No.1, (2013).
than the previous work done. But here, in proposed [9] Anderson J. A., : An Introduction to Neural Network. 3rd Ed. Library
of Congress Cataloging in publication Data. 651p, (1997).
technique, background is uniform and only some type of [10] Baker K., : Singular Value Decomposition Tutorial. 24p, (2005).
gestures and postures are recognized. [11] Brun T., : Teckensprks Lexikon. Bokforlaget Spektra AB, Halmstad,
(1974).
In future scope, Defined work can be performed on [12] Starner T., and Pentland A., : Real-time American sign language
recognition from video using hidden markov models. Technical
complex background images. This method tracks posture in Report No. 375, M.I.T Media Laboratory Perceptual Computing
Section, (1995).
upper body only. It can be extended to complete body [13] Schlenzig J., Hunter E., and Jain R., : Recursive spatio-temporal
posture recognition. analysis: Understanding Gestures. Technical report, Visual
Computing Laboratory, University of San Diego, California, (1995)
[14] Grant H., Lai C. K., : Simulation modeling with artificial reality
technology (smart): an integration of virtual reality simulation
modeling. Proceedings of the Winter Simulation Conference, (1998)
[15] Marcel S. ―Hand Posture Recognition in a Body Centered Spaceǁ,
IEEE, (1999).

978-1-7281-1253-4/19/$31.00
Authorized © National
licensed use limited to: Motilal Nehru 2019 IEEE
Institute of Technology. Downloaded on December 15,2024 at 15:29:57 UTC from IEEE Xplore. Restrictions apply.