Oriental Institute of Science and Technology, Bhopal

Department of CSE-(Artificial Intelligence & Machine Learning)

SIGN LANGUAGE
DETECTION :

YOGESH PATIDAR VIKASH RAJ YASHASVI DUGAYA RESHU YADAV

Department of artificial Department of artificial Department of artificial Department of artificial
inteligence and machine inteligence and machine inteligence and machine inteligence and machine
learning , O.I.S.T. learning , O.I.S.T. learning , O.I.S.T. learning , O.I.S.T.
[email protected] [email protected] [email protected] [email protected]
c.in c.in c.in c.in
Introduction
please stop
SIGN LANGUAGE DETECTION
 Do You Know?
 63 million people in India and 1.5 billion people worldwide live with hearing
impairments.
 OnlyI 50%
love Cleanworldwide use sign language for communication.
of deaf individuals
 90% you
of children born deaf are raised in families where sign language isn't used,
making communication even harder.

 The Challenge:
Imagine the difficulty for a deaf person in performing everyday tasks like:
More
 Ordering food help
 Talking on the phone
 Expressing needs
 Navigating streets with no awareness of surroundings

 Our Objective:
No
 To develop yes
an Al-powered system that translates sign language gestures into text or
speech, unlocking seamless communication and empowering the deaf to interact
confidently with the world.
 Detail idea of project
The goal of this project is to create a sign language detection system that recognizes hand gestures used in sign
language and translates them into text or speech in real-time. This system aims to break down communication barriers
for people with hearing or speech impairments, allowing them to communicate seamlessly with non-signers.

 Key Features:
 Real-time Gesture Recognition: The system will capture hand gestures using a camera and process them instantly.
 Preprocessing: The captured image or video will undergo preprocessing, such as background removal and hand
segmentation, to improve accuracy.
 Machine Learning Model: A neural network or classification algorithm will be trained to recognize specific hand
gestures from a dataset of sign language images/videos.
 Text/Speech Output: Once the gesture is recognized, it will be translated into corresponding text, which can then be
converted to speech using a text-to-speech engine.

 Objectives:
 Accurate Recognition: Ensure the system can correctly interpret a wide range of sign language gestures.
 User-friendly: The interface should be simple, with minimal input required from the user.
 Scalability: The system should support multiple sign languages and expand to include more gestures over time.
 DESIGN PROCESS
The design process for the sign language detection system involves several key steps,
from capturing the input to generating the output. Below is the high-level overview of
the process:

1. Input : Gesture Capture A camera (either a built-in webcam or external device) is used to capture the hand
gestures of the person using sign language. This serves as the system’s raw input.

2. Preprocessing : Image Processing techniques like background subtraction and hand detection are applied to
isolate the hand gestures from the background. Segmentation is performed to focus on the hand region and
remove unnecessary parts of the image. This step ensures that the gesture is captured clearly and is ready
for analysis.

3. Feature Extraction The preprocessed image is analyzed to extract key features of the hand gesture (e.g.,
shape, movement, or key points like finger positions). This feature set is used as input for the machine
learning model.

4. Model Training and Classification: A machine learning model (like a convolutional neural network, CNN) is
trained using a dataset of various sign language gestures. The trained model is then used to classify new
gestures in real time based on the features extracted. The model identifies which sign language gesture the
user is making.

5. Output: Text or Speech The recognized gesture is mapped to its corresponding text representation.
Optionally, the text can be fed into a text-to-speech (TTS) engine to generate spoken output, making the
 Technical Concepts
Using a CNN (Convolutional Neural Network) model for the classification of Indian Sign Languages
involves several technical concepts and algorithms. CNN is a deep learning architecture known for its
ability to process grid-like data such as images and videos effectively. When applying CNN
architecture for Indian Sign Language classification, we encountered the following technical concepts
and algorithms:
● CNN Architecture: Convolutional Neural Network (CNN) architecture is designed for processing and
analysing computer vision tasks. It utilizes various layers to train very deep neural networks effectively on
image and video dataset.

● Convolutional Layers: These layers automatically extract features from hand gesture images. They're crucial
for recognizing patterns in images of the hand, helping the model identify signs language and classify them
correctly.

● Pooling layers: These layers downgrade the dimensions of the image input by reducing its size. This helps in
reducing the computational usage and focuses on important functions while retaining its features.

● Fully Connected Layers: These layers connect to every neuron in one layer with another layer, enabling the
network to classify and predict based on the learned weights and features
 Gesture Input
Preprocessing Feature Extraction
START (Camera captures
sign language
(Image cleaning, (Extract hand gesture
hand segmentation) features)
gesture)

Model
Classification Output as Text
FLOWCHART
Optional: Text to-
Speech
(Classify the gesture
using a trained
(Display the
recognized sign as (Convert text to END
machine learning text) speech output)
model)
RESULT
 References
 RESEARCH ARTICLES AND PAPERS:

"Hand Gesture Recognition Using Deep Learning" – Journal of Artificial Intelligence Research, 2023.
"Real-Time Sign Language Detection with CNNs" – IEEE Transactions on Neural Networks, 2022.
"Image Preprocessing Techniques for Gesture Recognition" – Springer AI Applications, 2021.

 Sign language recognition with advanced computer vision:

https://towardsdatascience.com/sign-language-recognition-with-advanced-comput
er-vision-7b74f20f3442
 World deafness data by WHO:
https://www.who.int/india/campaigns/world-hearing-day-2023

 Deafness in India:
https://journals.lww.com/ijoo/fulltext/2016/22020/deafness_in_india.1.aspx

 A mom fights to get an education for her deaf daughter:

https://www.npr.org/sections/goatsandsoda/2018/01/14/575921716/a-mom-fights
-to-get-an-education-for-her-deaf-daughters
Thank You!