KUMARAGURU COLLEGE OF TECHNOLOGY

COIMBATORE-641049
(An Autonomous Institution Affiliated to Anna University)

DEPARTMENT OF ELECTRONICS AND COMMUNICATION

ENGINEERING

U18INI5600 - ENGINEERING CILINCS – V

Project Report

HAND GESTURE RECOGNITION USING CNN

IN MATLAB

Submitted by

HIGH COURT DURAI P Reg.No.:18BEC181

SHANJAI SETHUPATHY V Reg.No.:18BEC184
JAYASATHYAN S Reg.No.:18BEC178
SANJAI D Reg.No.:18BEC207
SIVARAJ S Reg.No.:18BEC158

December 2020

Faculty Coordinator

1
 ACKNOWLEDGEMENT

First, we would like to express our praise and gratitude to the Lord, who
has showered his grace and blessing enabling us to complete this project in an
excellent manner. He has made all things in beautiful in his time. We express
our sincere thanks to our beloved Joint Correspondent, Shri. Shankar
Vanavarayar for his kind support and for providing necessary facilities to carry
out the project work.
We would like to express our sincere thanks to our beloved Principal
Dr.J.Srinivasan, Ph.D who encouraged us with his valuable thoughts. We
would like to express our sincere thanks and deep sense of gratitude to
Dr.S.A.Pasupathy, Ph.D (UK)., Professor and Head, Department of Electronics
and Communication Engineering for his valuable suggestions and
encouragement which paved way for the successful completion of the project.
In particular, we wish to thank and express our everlasting gratitude to
Dr.B.Gopinath, Ph.D, Associate Professor, Department of Electronics and
Communication Engineering for the expert counselling in each and every steps
of project work and we wish to convey our deep sense of gratitude to all
teaching and non-teaching staff members of Electronics and Communication
Engineering Department for their help and cooperation.

2
 ABSTRACT

Hand gesture is a natural way for humans to interact with the

computers to perform variety of applications. Using Deep learning which is
efficient for image recognition system is used to find the hand gesture which is
captured dynamically. In particular the Convolutional neural network is used for
better performance. The model is trained with static hand gesture images. The
Convolutional neural network is created without using a Pre-trained model.

3
 TABLE OF CONTENTS

CHAPTER NO TITLE PAGE NO

ABSTRACT 3

1 INTRODUCTION 5
2 LITERATURE SURVEY 6
3 PROPOSED BLOCK DIAGRAM 7
4 CODING PART 8
LIST OF TOOLS / COMPONENTS USED
5 9
AND THEIR DESCRIPTION
GESTURE RECOGNITION USING CNN
6 9
DEEP LEARNING
7 NETWORK ARCHITECTURE 10
8 RESULTS AND DISCUSSION 11
9 APPLICATIONS 12

10 CONCLUSION 13

11 REFERENCES 13

4
 1. INTRODUCTION

Gesture Recognition is more important when it comes to Real time

applications. Due to wide spread of Digital cameras availability many researchers
are in research regarding the applications of gesture recognition. Still there are
many challenges because of the complexity of gestures recognition. Hence this
problem is solved by Deep learning using Convolutional neural network . Deep
learning is more efficient when it comes to image recognition.

Here the ASL dataset which contains the hand gesture containing (0-9) is
used. Generally an image is preprocessed where it plays a vital role for extracting
the gesture in a static image ( i.e ) background subtraction , image binarization . the
feature is extracted from all the images after binarization .

Convolutional neural network’s are made of neurons with learnable

weights and biases. Every neurons receives several inputs, and takes a weighted
sum over them. It is then passed through an activation function and responds with
an output.

5
 2. LITERATURE SURVEY

Author proposed a real time vision based system for hand gesture
recognition for human computer interaction in many applications. The system can
recognize any different hand gestures given by Indian and American Sign
Language or ISL and ASL at faster rate with virtuous accuracy. RGB-to-GRAY
segmentation technique was used to minimize the chances of false detection.
Authors proposed a method of improvised Scale Invariant Feature Transform
(SIFT) and same was used to extract features. The system is model using
MATLAB. To design and efficient user friendly hand gesture recognition system, a
GUI model has been implemented.

6
 3. PROPOSED BLOCK DIAGRAM

Fig no 1: Block diagram

4. CODING PART

TESTING
clc;
close all;
clear all;
warning off;
c=webcam;
load myNet;
x=0;
y=0;
height=200;
7
width=200;
bboxes=[x y height width];
while true
e=c.snapshot;
IFaces=insertObjectAnnotation(e,'rectangle',bboxes,'Processing
Area');es=imcrop(e,bboxes);
es=imresize(es,[227,227]);
label=classify(myNet,es);
imshow(IFaces);
title(char(label));
drawnow;
end

TRAINING

clc;
clear all;
close all;
warning off;
g=alexnet;
layers=g.Layers;
layers(23)=fullyConnectedLayer(7);
layers(25)=classificationLayer;
allImages=imageDatastore('Hand
Dataset','IncludeSubfolders,true','LabelSource','foldername');
opts=trainOptions('sgdm','InitialLearnRate',0.001,'MaxEpochs',20,'MinBatchSize',
64);
myNet=trainNetwork(allImages,layers,opts);
save myNet
8
 5. LIST OF TOOLS / COMPONENTS USED AND THEIR
DESCRIPTION

The major components used in this project are,

a) MATLAB R2020b
b) AI,Deep learning toolbox
c) WEB camera

6. GESTURE RECOGNITION USING CNN DEEP LEARNING

Initially the dataset is loaded into the network for training . Preprocessing
is done before the feature is extracted. The training is done in Convolutional
neural network . After training an input image is given by capturing from a
webcam. The given input image is tested for recognizing the gesture . A confusion
matrix is produced accordingly to the produced output with its mean accuracy.
A ConvNet is a popular machine learning algorithm. Which is one of the
techniques of Deep learning and is a learning model used to execute classification
tasks through images, video, text, and sound .CNNs specifically give better results
for identifying patterns in an image, which leads to recognizing of hand gesture,
face, and any object. The advantage of CNN is it don’t require any feature
extraction to train the model. CNN is invariant to the scaling and rotation.

9
 7. NETWORK ARCHITECTURE

 ImageInputLayer: An imageInputLayer is the place you initialize the

size of input image,here, 128-by-128-by-1 is used. These numbers
represent height, width, and the number ofchannels. In this case, input
data is a grayscale image, hence the number of channel is 1.
 Convolutional Layer: Input arguments for this layer are filtering size,
the number of filters, and padding. Here, the filter of size 10 is used,
which determines 10 x 10 filter.The number of channels used is 10,
means 10 neurons are connected. Padding of 1 specifies that the size of
the output image is same as that of an input image.
 ReLU Layer: ReLU (rectified linear unit) layer is a batch
normalization layer, which is placed after initializing a nonlinear
activation function. Importance of this layer is to
 decrease the sensitivity and increase the pace of the training.
 Max Pooling Layer: Max pooling layer is one of the downsampling
technique which is used for convolutional layers. In this architecture,
poolSize is set to 3 and training function’s step size is 3.
 Fully Connected Layer: Fully connected layers follow max pooling
layer. In this layer, all the neurons of all layers are interconnected to the
previous layer. The given input argument for this layer is 10, which
indicate 10 classes.
 Softmax Layer: Fully connected layers are followed by softmax layer,
which is normalization technique. This layer generates positive
numbers as output such that the sum of numbers is one. Classification
layer uses these numbers for lassification.

10
  Classification Layer: Classification layer is the final layer of the
architecture. This layer classifies the classes based on probabilities
obtained from softmax layer and also calculate cost function.
 Training Options
The maximum number of epochs set to 15 and initial learning rate is 0.001.

8. RESULTS AND DISCUSSION

Fig no 2: Confusion Matrix

Hand gesture recognition system received great attention in the recent

few years because of its manifoldness applications and the ability to interact with
machine efficiently through human computer interaction. Advantages and
drawbacks of the discussed systems are explained finally

11
 Fig no 3: Real time hand gesture recognition

9. APPLICATIONS

 Hand Image controlling & Scaling

 To Control Mouse
 Sign Language Recognition
 Gaming Interface
 Robot Control
 Controlling Machines
 System Control and Image Scaling
 Physically abled people can communicate through mobile applications using
gestures.The gestures will convert into letters,numbers and audio signals
through this method.

12
 10. CONCLUSION

Fully connected layers Scaling Human hand gestures provide the most
important means for non-verbal interaction among people.

• At present, artificial neural networks are emerging as the technology of choice

formany applications, such as pattern recognition, gesture recognition,
prediction, system identification, and control.
• ANN provides good and powerful solution for gesture recognition in MATLAB.
• The ability of neural nets to generalize makes them a natural for gesture
recognition.

11. REFERENCES

• Sebastian Marcel, Oliver Bernier, Jean Emmanuel Viallet and Daniel Collobert.
(2000). “HandGesture Recognition using Input –Output HiddenMarkov Models”,
Proc. of the Fourth IEEE International Conference on Automatic Face and Gesture
Recognition, pp.456 - 461.

• Xia Liu and Kikuo Fujimura. (2004). “HandGesture Recognition using Depth
Data”, Proc. of the Sixth IEEE International conference on automatic Face and
Gesture Recognition,pp. 529-534.

13
14