Sign Language Recognition based

on Hands Symbols
Classification
 Content
● Introduction
● Motivation
● Problem statement
● Objective
● Literature review
● Dataset description
● Proposed work
● Result
● Conclusion and Future work
● References
 Introduction

● The world is hardly live without communication, no matter

whether it is in the form of texture, voice or visual
expression.

● The communication among the deaf and dumb people

is carried by text and visual expressions.

● Gestural communication is always in the scope

of confidential and secure communication.

● Hands and facial parts are immensely influential to express

the thoughts of human in confidential communication.
 Motivation
● Sign language is learned by deaf and dumb, and usually it is
not known to normal people, so it becomes a challenge for
communication between a normal and hearing impaired
person.

● Its strike to our mind to bridge the between hearing

impaired and normal people to make the communication
easier.

● Sign language recognition (SLR) system takes an input

expression from the hearing impaired person gives output to
the normal person in the form text or voice.
 Problem Statement

● Understanding the context of

expressions exact symbolic dumb
of
challenging job indeaf people
real life until unless is
it is properly
specified. and the
 Objective

● Communication is always having a great impact in

every domain and how it is considered the
meaning of the thoughts and expressions that
attract the researchers to bridge this gap for every
living being.

● The objective of this project is to identify the

symbolic expression through images so that the
communication gap between a normal and hearing
impaired person can be easily bridged.
 Literature Review

Author Publication Year Problem Methodolog Remark

y
Naresh Kumar ICCCA 2017 Hand Sign SVM & LDA Classification
Language to recognition
Recognition sign language
symbols
(97.3%).
Tse-Yu Pan, IEEE 2016 Real-time Sign SVM, PCA & Images of the
Li-Yun Lo, Language LDA same gesture
Chung-Wei Recognition were captured
Yeh, Jhe-Wei in different
Li, Hou-Tim lighting
Liu, Min- Conditions
Chun Hu (94%)
 Cont.

Author Publication Year Problem Methodolog Remark

y
Salem Ameen, University of 2017 Classify American CNN This paper
Sunil Vadera Salford Sign Language explores the
Fingerspelling from applicability of
Depth and Colour deep learning for
Images interpreting sign
language,
precision of 82%
and recall of
80%.
Arabic sign IEEE 2017 3D Convolutional CNN The system
language Neural Network achieved 98%
recognition with (CNN) was used to accuracy for
3D recognize 25 observed data
convolutional gestures from an and 85% average
neural networks Arabic sign accuracy for new
language dictionary data.
 Dataset description

● We analyze 4,800 images of sign images which is ISL of the English alphabet,
which have a spread of 26 class labels assigned to them. Each class label is a set of
sign images of the English alphabet.

● All the images are resized to 640 x 480 pixels, and we perform both the model
optimization and predictions on these downscaled images.
 Cont.

● Below figure shows an example from every class of sign images dataset.
 Proposed Work

● Inimpaired
this work, we proposed an idea for feasible communication between hearing
and normal person with the help of-
• Deep Learning
■ CNN
■ AlexNet
• Machine Learning
■ SVM
■ Random Forest
■ KNN
 Cont.

Work flow Diagram:

 Cont.

Figure. Typical Convolutional Neural Network Architecture

 Cont.

Figure. Typical AlexNet Architecture

 Result

● CNN
• SGD optimizer with learning rate 0.01 and dropout 0.25
• Model Accuracy
■ Accuracy: 98.74%
• Model Loss
■ Loss: 6.53%
 Cont.

● AlexNet
• SGD optimizer with learning rate 0.01, momentum 0.9, nesterov and dropout 0.25
• Model Accuracy
■ Accuracy: 99.79%
• Model Loss
■ Loss: 0.79%
 Cont.

Comparison table of CNNs and AlexNet:

Model Optimizer Dropout Learning Momentum Nestrove Accuracy

rate %

CNNs SGD 0.25 0.01 --- --- 98.74

CNNs SGD 0.50 0.01 --- --- 98.74

AlexNet SGD 0.25 0.01 98.42

AlexNet SGD 0.50 0.01 --- --- 95.70

AlexNet SGD 0.25 0.01 0.9 True 99.79

AlexNet SGD 0.50 0.01 0.9 True 99.79

AlexNet Adam 0.25 0.001 --- --- 99.69

AlexNet Adam 0.50 0.001 --- --- 99.58

 Cont.

● Machine Learning
• SVM: 91.226%
• Random Forest: 95.719%
• KNN: 87.542%
 Conclusion and Future Work

● In this project, we proposed an idea for feasible

communication between hearing impaired and normal person
with the help of deep learning and machine learning approach.

● This proposed work ensures the accuracy of 91.22% using

SVM, 95.71% using Random Forest, 87.54% using KNN,
98.74% using CNN and 99.79 using AlexNet.

● There is ever the sounding challenge to develop a sign

language system in data the collection remains invariant of the
unconstraint environment. This project can be extended to the
real time data.
 References
1Ameen, S., & Vadera, S. (2017). A convolutional neural network to classify
American Sign Language fingerspelling from depth and colour images.
Expert Systems.
2Naresh Kumar(2017). Sign Language Recognition for Hearing Impaired
People based on Hands Symbols Classification. International Conference on
Computing, Communication and Automation (ICCCA2017)
3Menna ElBadawy, A. S. Elons, Howida A. Shedeed and M. F. Tolba. Arabic
sign language recognition with 3D convolutional neural networks. 2017
Eighth International Conference on Intelligent Computing and Information
Systems (ICICIS)
[4] Pan, T. Y., Lo, L. Y., Yeh, C. W., Li, J. W., Liu, H. T., & Hu, M. C.(2016,
April). Real-time sign language recognition in complex background scene
based on a hierarchical clustering classification method. In Multimedia Big
Data (BigMM), 2016 IEEE Second International Conference on (pp. 64-67).
IEEE.
 Cont.

5Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2014,

September). Sign language recognition using convolutional neural
networks. In Workshop at the European Conference on Computer Vision (pp.
572-578). Springer International Publishing.
6Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of
initialization and momentum in deep learning. In: Proceedings of the 30th
International Conference on Machine Learning (ICML-13). pp. 1139{1147
(2013)
7Tao Liu, Wengang Zhou, and Houqiang Li. Sign Language Recognition With
Long Short-Term Memory . 2016 IEEE International Conference on
Image Processing (ICIP)