International Conference on Internet of Things (ICIoT) 2023

26th – 28th April 2023

Indian Sign Language Classifier using Machine

Learning
Paper ID: 650
Presented by:
1. Bhavesh Jain (SRM Institute of Science and Technology)
2. Kevin S Paul (SRM Institute of Science and Technology)
3. A.M.J Muthu Kumaran (SRM Institute of Science and Technology)
Introduction/Abstract
• Humans rely heavily on communication because it fosters relationships between people and the
exchange of ideas. In addition to voice, body language, hand gestures, reading, writing, and
sketching are other ways we might communicate. Nonetheless, speech is the most popular form
of communication. Yet, people who are deaf or have difficulty in hearing are limited to using
signs, which mainly rely on non-verbal means of communication. In India itself, millions of people
are hearing impaired. Indian Sign Language (ISL). This project exhibits a system prototype that can
automatically recognize sign language in order to aid deaf individuals in communicating with
others more efficiently.

International Conference on Internet of Things (ICIoT) 2023 2

Introduction/Abstract
• Deaf individuals utilize sign language as their native speech; it is a visual language. A
person's thoughts can be dynamically expressed using sign language rather than
through the use of acoustically transmitted sound patterns. It is accomplished by fusing
together hand gestures, arm, and body movements, and facial emotions all at once. It
may be used to communicate with persons who have hearing impairments, by people
who have trouble speaking, by those who can hear but cannot talk, and by normal
people.

• The foundation of this project is the text-to-sign language conversion, which can be done
by using Python modules, to transform text input to respective sign language signals.
The text is broken down into manageable chunks using machine learning. In order for
the program to show the converted text into the sign language, predetermined data sets
of Indian sign language are utilized as the input.
International Conference on Internet of Things (ICIoT) 2023 3
Objectives
• Usually deaf people seek the help of sign language interpreters for translating their
thoughts to normal people and vice versa. But this system is very costly and does
not work throughout the life period of a deaf person. So a system that
automatically recognizes sign language gestures is necessary. Such a system can
minimize the gap between deaf people and normal people in society.

• Moreover, hearing people have difficulties in learning sign language and likewise
the majority of those people who were born deaf or who became deaf early in life,
have only a limited vocabulary of the accordant spoken language of the community
in which they live. Hence a system of translating sign language to spoken language
would be of great help for deaf as well as for hearing people. A second aspect is
that sign language recognition serves as a good basis for the development of
gestural human-machine interfaces.
International Conference on Internet of Things (ICIoT) 2023 4
Literature Survey/Existing
Systems
TITLE AUTHOR AND PUBLICATION TECHNOLOGY USED

Sign language recognition using J. L. Raheja, A. Mishra & A. Support Vector Machine
SVM Chaudhary (SVM)
5 June 2016

Sign Language Recognition Using Lionel Pigou, Sander Dieleman, Convolutional Neural Networks
Convolutional Neural Networks Pieter-Jan Kindermans & (CNN)
Benjamin Schrauwen
1 January 2015

Sign Language Recognition Using Sinchana S Bharadwaj, Anagha G, Convolutional Neural Networks
Machine Learning Sumana M R (CNN)
5 August 2022

International Conference on Internet of Things (ICIoT) 2023 5

Literature Survey/Existing
Systems
TITLE AUTHOR AND PUBLICATION TECHNOLOGY USED

ML Based Sign Language K Amrutha, P Prabhu SLR Model

Recognition System 12 February 2021

American Sign Language Kshitij Bantupalli, Ying Xie Recurrent Neural Network
Recognition using Deep Learning 24 January 2019 (RNN)
and Computer Vision

International Conference on Internet of Things (ICIoT) 2023 6

Proposed
Methodology/Algorithms
Here is the proposed methodology and algorithm used –
• Text Recognition
At first, we take input in the form of text and then, by using various Natural Language Processing
(NLP) tools, one of which includes stop word removal. Through that process, the input is then
processed an further converted into the respective sign language. If the word is complicated, it is
broke down to individual alphabets and then expressed in sign language.
• Algorithm
We have used NLTK, which is a very robust package consisting of various algorithms which are used
for Natural Language Processing.

International Conference on Internet of Things (ICIoT) 2023 7

Proposed
Methodology/Algorithms
• Tokenization
The breaking down of text into smaller units is called tokens. tokens are a small part of that text.
If we have a sentence, the idea is to separate each word and build a vocabulary such that we
can represent all words uniquely in a list.
• Stop Words removal
When we use the features from a text to model, we will encounter a lot of noise. These are the
stop words like the, he, her, etc … which don’t help us and, just be removed before processing
for cleaner processing inside the model.
• Lemmatization
Lemmatization is the process of converting a word to its base form. It considers he context and converts
the word to its meaningful base form.
• POS Tagging
Part of Speech tagging is used in text processing to avoid confusion between two same words
that have different meanings. With respect to the definition and context, we give each word a
particular tag and process them.

International Conference on Internet of Things (ICIoT) 2023 8

Software Tools/Datasets used (if
any)
• For this project, we have used the official Indian Sign Language(ISL)
dataset .
• For the main coding part of the project, we have used VS Code as the
code editor.
• Languages used – HTML, CSS, Python, Django, SQLite 3
• We have also used NLTK (Natural Language Toolkit) which is a suite of
libraries used for Natural Language Processing.
• We have also used Blender for the animations.

International Conference on Internet of Things (ICIoT) 2023 9

Findings/Results
• This project converts text input to the respective signals according to the Indian Sign Language,
and the output is presented as the animation of the signals as shown in the image –

International Conference on Internet of Things (ICIoT) 2023 10

Findings/Results
• It also breaks down the words into individual alphabets and shows individual animations in order
to facilitate the learning process for the hearing impaired.
• The animation used to showcase the output is shown below –

International Conference on Internet of Things (ICIoT) 2023 11

Conclusion
As part of the deployment of our system, we created a text to Sign language converter that is useful and can help many
individuals who are deaf or hard of hearing and make communicating with them much simpler. For those who are less
wealthy, this desktop programme will be incredibly helpful and make their lives easier. When Indian Sign Language is
widely used throughout the nation, it will be very beneficial. This technique is applicable to colleges, universities,
airports, and schools, to name a few places.
This project can be further enhanced by incorporating the following points -
• As deaf persons typically lack the ability to communicate normally with others, they must rely on an interpreter or
another kind of visual communication. This effort can assist reduce reliance on the interpreter because they are no
longer constantly available.
• To fully comprehend the context of the input text, the system may be expanded to add knowledge of various
languages as well.
• A web-based and mobile version of the application will broaden its audience.
• Using computer vision to integrate a hand gesture detection system to create a two-way communication system.

International Conference on Internet of Things (ICIoT) 2023 12

References
1. J. Atick (1994). neural network recognition of Indian sign language. 1165–1168 in IEEE International
Conference on Neural Networks
2. Kumar, R. K., and Rajesh, R. (2011). Hidden Markov model recognition for Indian sign language. 22(9), 24-
29, International Journal of Computer Applications.
3. Banerjee, S.; Roy, A. (2016). recognition of Indian sign language using recurrent and convolutional neural
networks. SPICES, 1–5, is the name of the IEEE International Conference on Signal Processing, Informatics,
Communication, and Energy Systems.
4. Basu, D. K., Kundu, M. K., and Sarkar, S. (2017). Recognition of Indian sign language using customized
characteristics. 8(2), 38–44, International Journal of Computer Science and Information Technology.
5. Goel, S., Tandon, P., and Singal, A. (2019). Deep learning and text-to-speech synthesis are being used to
recognize and translate Indian sign language. ICAC3, the International Conference on Advances in
Computing, Communication, and Control, 1–5

International Conference on Internet of Things (ICIoT) 2023 13

References
6. S. Bansal, S. Chaudhary, et al (2018). utilizing deep learning to recognize Indian sign language. 35(1), 853-
860, Journal of Intelligent & Fuzzy Systems.
7. R. Bhatt, R. Shah, and R. (2020). Recognition of Indian sign language using machine learning methods.
IJITEE, 9(3), 1732-1739. International Journal of Innovative Technology and Exploring Engineering.
8. Kotecha and Dangar, D. M. (2020). utilizing deep learning to recognize Indian sign language. 1–5 IEEE
International Conference on Inventive Computation Technologies.
9. M. Gupta and N. Garg (2021). Convolutional neural networks are used for Indian sign language recognition.
IEEE CICT: Computational Intelligence & Communication Technologies International Conference, 1-6.
10. Rafique, A.; Khan, M. U. (2021). Deep learning method used to recognise Indian sign language in real time.
MLICOM, the IEEE International Conference on Machine Learning and Intelligent Communications, pages 1–
6

International Conference on Internet of Things (ICIoT) 2023 14

 Thank You

International Conference on Internet of Things (ICIoT) 2023 15