VISVESVARAYA TECHNOLOGICAL UNIVERSITY

"JNANA SANGAMA”, MACHHE, BELAGAVI-590018

Seminar Report
on
Deep learning-based sign language recognition system for static
signs
Submitted in partial fulfillment of the requirements for the VIII semester
Bachelor of Engineering
in
Computer Science and Engineering
of
Visvesvaraya Technological University, Belagavi.
by
Spoorthi S V
(1CD21CS158)

Under the Guidance of

Mr. Arun P
Assistant Professor
Dept. of CSE

Department of Computer Science and Engineering

CAMBRIDGE INSTITUTE OF TECHNOLOGY
AN AUTONOMOUS INSTITUTION AFFILIATED TO VTU,
BANGALORE - 560 036
2024-2025
 CAMBRIDGE INSTITUTE OF TECHNOLOGY
AN AUTONOMOUS INSTITUTION AFFILIATED TO VTU
K.R. Puram, Bangalore-560 036
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

CERTIFICATE

Certified that Ms. Spoorthi S V bearing USN 1CD21CS158, a bonafide student of Cambridge
Institute of Technology, has successfully completed technical seminar entitled “Deep learning-
based sign language recognition system for static signs” in partial fulfillment of the
requirements for VIII semester Bachelor of Engineering in Computer Science and Engineering
of Visvesvaraya Technological University, Belagavi during academic year 2024- 2025. It is
certified that all Corrections/Suggestions indicated for Internal Assessment have been incorporated
in the report deposited in the departmental library. The seminar report has been approved as it
satisfies the academic requirements in respect of technical seminar prescribed for the Bachelor of
Engineering degree.

----------------------------- ----------------------------- --------------------------

Seminar Guide Seminar Co-Ordinator Head of the Dept.
Prof. Arun P Prof. Vasumathi A K Dr. Shreekanth Prabhu M
Dept. of CSE, CITech Dept. of CSE. CITech Dept. of CSE. CITech
 DECLARATION

I, Spoorthi S V, a student of VIII semester BE, Computer Science and Engineering, Cambridge
Institute of Technology, hereby declare that the technical seminar entitled “Deep learning-based
sign language recognition system for static signs” has been carried out by me and submitted in
partial fulfillment of the course requirements of VIII semester Bachelor of Engineering in
Computer Science and Engineering as prescribed by Visvesvaraya Technological University,
Belagavi, during the academic year 2024-2025.

I also declare that, to the best of my knowledge and belief, the work reported here does
not form part of any other report on the basis of which a degree or award was conferred on an
earlier occasion on this by any other student.

Date: 25/03/2025 Spoorthi S V

Place: Bangalore (1CD21CS158)
 ACKNOWLEDGEMENT
I would like to place on record my deep sense of gratitude to Shri. D. K. Mohan, Chairman,
Cambridge Group of Institutions, Bangalore, India for providing excellent Infrastructure and

Academic Environment at CITech without which this work would not have been possible.

I am extremely thankful to Dr. G. Indumathi, Principal, CITech, Bangalore, for providing me the
academic ambience and everlasting motivation to carry out this work and shaping our careers.

I express my sincere gratitude to Dr. Shreekanth Prabhu M., HOD, Dept. of Computer Science
and Engineering, CITech, Bangalore, for his stimulating guidance, continuous encouragement and
motivation throughout the course of present work.

I also wish to extend my thanks to Ms. Vasumathi AK, Assistant Professor, Seminar Coordinator,
Dept. of CSE, CITech, Bangalore, for her critical, insightful comments, guidance and constructive
suggestions to improve the quality of this work.

I also wish to extend my thanks to Mr. Arun p, Assistant. Professor, Dept. of CSE, CITech for his
guidance and impressive technical suggestions to complete my seminar.

I express my gratitude to Thomas Bocek, Bruno B. Rodrigues, Tim Strasser, and Burkhard Stiller,
whose paper entitled “Blockchains Everywhere – A Use-case of Blockchains in the Pharma
Supply-Chain” forms the base for this report.

Finally, I would like to express my deepest gratitude to my friends and classmates for their
unwavering support, especially in technical aspects. I’m also thankful to my faculty members for
their guidance and encouragement. Lastly, I extend my heartfelt thanks to my parents, whose
constant support and encouragement were my pillar of strength in completing this work.

Spoorthi S V
 ABSTRACT

Sign language is an effective medium of communication for individuals with hearing and speech
impairments. With the rapid progress in computer vision, researchers are increasingly focusing on
developing automated sign language recognition systems to enhance accessibility. Traditional
approaches to Indian Sign Language (ISL) recognition often concentrate on a small set of distinct
static signs, limiting their applicability in real-world scenarios. This paper proposes a robust deep
learning-based system for the recognition of static ISL signs using Convolutional Neural Networks
(CNNs). A large and diverse dataset of 35,000 images representing 100 static signs was collected
from multiple users under varying conditions to ensure model robustness and generalizability. The
system's performance was extensively evaluated using around 50 CNN architectures and tested
with different optimizers to identify the most efficient configuration. The proposed model achieved
a remarkable training accuracy of 99.72% on colored images and 99.90% on grayscale images. In
addition to accuracy, evaluation metrics such as precision, recall, and F-score were used to validate
the system's reliability. The results demonstrate significant improvement over previous works,
which were limited to recognizing only a few hand signs. This research contributes to building a
more comprehensive and scalable sign language recognition system, paving the way for better
human-computer interaction and accessibility solutions.

i
 CONTENTS
Abstract i

Contents ii

List of Figures iii

Chapters Page No.

Chapter 1 Introduction 1
1.1 Overview of sign language recognition 1
1.2 Advancements in Deep Learning and Computer Vision 1
1.3 Problem Statement and Motivation 2
1.4 Sign AI: A Deep Learning-Based Static Sign
Recognition System
Chapter 2 Literature Survey 3
Chapter 3 Architecture 5

3.1 Back-End Components 5

3.1.1 Convolutional Neural Network (CNN) Model 5
3.1.2 Model Training Environment 6
3.1.3 Relational Database 6
3.1.4 Server 6
3.2 Front-End Components 6
3.2.1 Mobile Devices 6
3.2.2 IoT Sensors 6
3.3 Workflow 7
Chapter 4 Implementation 8
4.1 Method 8
4.1.1 Data Acquisition 8
4.1.2 Data Processing 8
4.1.3 Model Training 9
4.1.4 Data Storage and Processing 9
4.1.5 Testing 9

i
 4.2 Challenges 10
4.2.1 Dataset Imbalance and Collection Difficulty 10
4.2.2 Variations in Lighting and Background 10
4.2.3 Overfitting in CNN Models 11
4.2.4 Computational Resource Constraints 11
4.1.5 Accuracy Trade-off in Similar Signs 11
Chapter 5 Real-World Applications 12
5.1 Deep learning in sign language recognition 12
5.1.1 Real -Time Sign-to-Text Conversion 12
System
5.1.2 Education tools for Learning Sign Language 15
In real-time
5.1.3 Assistive Communication Devices for 13
Accessibility
5.1.4 Integration into Mobile and Web Applications 13
Platforms
5.1.5 Enhanced Customer Support Accessibility 13
5.1.6 Smart Classrooms and Inclusive Education 13
5.3 Modum: Blockchain-Enabled Temperature 14
Monitoring in Pharmaceutical Supply Chains
5.4 Challenges in Real-World Implementations 16
Conclusion 17
References 18

ii
 List of Figures

Figure No. Figure Name Page No.

3.1 High-level general CNN Architecture 5

5.1 Sample Dataset 14
5.2 Classification Performance 15

iii
CHAPTER 1
INTRODUCTION

1.1 Overview of sign language recognition

Sign language is a visual method of communication that uses hand gestures, facial expressions, and
body movements to convey meaning, primarily used by individuals with hearing and speech
impairments. Each gesture or sign represents a word, letter, or phrase, forming a complete language
system. Sign language recognition (SLR) aims to bridge the communication gap between the hearing-
impaired community and the general population by translating these signs into readable or audible
formats. SLR can be categorized into static (still images) and dynamic (continuous gestures)
recognition. Static signs are easier to detect as they involve a fixed hand posture captured in a single
frame.

1.2 Advancements in Deep Learning and Computer Vision

Deep learning has revolutionized the field of computer vision by enabling machines to automatically
learn patterns and features from large datasets without manual intervention. Convolutional Neural
Networks (CNNs), a key architecture in deep learning, have shown remarkable success in tasks like
image classification, object detection, and gesture recognition. These models are capable of
identifying subtle variations in hand shapes, orientations, and positions, making them ideal for sign
language recognition. Unlike traditional methods that required handcrafted features or external
devices, deep learning models can work directly on raw image data. This advancement has led to
more accurate, scalable, and real-time recognition systems. The integration of computer vision with
deep learning has thus opened new possibilities in developing intelligent and accessible
communication tools.

1.3 Problem Statement and Motivation

Despite the importance of sign language, many existing recognition systems are limited to a small
number of basic or easily distinguishable signs. This restricts their usefulness in real-world
applications, especially in diverse and dynamic communication settings. Indian Sign Language (ISL),
in particular, has received less research attention compared to other global sign languages. There is a
need for a robust system that can recognize a larger set of static ISL signs with high accuracy. Deep
learning offers a promising solution due to its ability to learn complex visual patterns.

B.E.,Dept of CSE.,CITech 2024-25 Page 1

 Deep learning-based sign language recognition system for static signs Introduction

Deep learning, especially Convolutional Neural Networks (CNNs), offers powerful capabilities in
feature extraction and image classification. This motivates the development of a CNN-based system
that can handle varied hand shapes, lighting conditions, and user differences. The ultimate goal is to
bridge the communication gap and create a supportive tool that enhances accessibility and
independence for the hearing-impaired community.

1.4 Sign AI: A Deep Learning-Based Static Sign Recognition System

Sign AI is a deep learning-powered system developed to recognize static hand gestures from Indian
Sign Language (ISL) using Convolutional Neural Networks (CNNs). The system aims to bridge the
communication gap between hearing-impaired individuals and the wider population by accurately
identifying 100 different static signs.
Key Features of SignAI’s Solution:
1. Large-Scale Data Collection – A dataset of 35,000 images of 100 ISL static signs collected
from multiple users ensures model robustness and variation handling.
2. CNN-Based Gesture Recognition – Deep learning models trained using CNNs automatically
extract features and classify static signs with up to 99.90% accuracy.
3. Color and Grayscale Compatibility – The system performs effectively on both color and
grayscale images, offering flexibility in real-world environments.
4. Performance Evaluation Metrics – Model performance is evaluated using precision, recall,
F-score, and training accuracy, ensuring reliable gesture recognition.
5. Scalability and Accessibility – The system is designed to scale to additional signs and can be
integrated into assistive communication apps or devices to promote inclusivity.

B.E.,Dept of CSE.,CITech 2024-25 Page 2

Chapter 2
LITERATURE SURVEY

2 Introduction

Sign language recognition has evolved significantly with the introduction of machine learning
and deep learning techniques. Early research efforts primarily utilized traditional machine
learning algorithms, which required manual feature extraction and offered limited accuracy.
However, recent advances in deep learning, especially convolutional neural networks (CNNs),
have significantly improved the performance and accuracy of sign language recognition
systems.

2.1 Early Approaches Using Machine Learning [1]

Early research efforts in sign language recognition primarily relied on conventional machine
learning techniques such as Support Vector Machines (SVM), k-Nearest Neighbors (k-NN),
and Decision Trees. These approaches required handcrafted features extracted manually from
images, such as contour, edge, or skin-color features. However, the dependency on manually
engineered features limited the scalability and accuracy of the systems. Such models failed to
generalize well across users, lighting conditions, and different sign styles. Moreover, these
techniques performed poorly on large-scale sign datasets, especially when the signs were
captured in varied environments. l

2.2 CNN-Based Sign Language Recognition Systems [2]

The introduction of Convolutional Neural Networks (CNNs), sign language recognition
systems became more accurate and robust. CNNs automatically extract spatial hierarchies of
features from input images, making them highly suitable for image classification tasks. Nagi
et al. implemented a CNN using color segmentation and achieved 96% accuracy on six hand
gesture classes. Rioux-Maldague and Giguère used Kinect images and deep belief networks,
achieving 99% precision for known users. Huang et al. utilized 3D CNNs on temporal gesture
data, resulting in a recognition rate of 94.2%, and later improved it to 98.9% using RealSense
sensors. Pigou et al. used CNNs with Nesterov’s Accelerated Gradient optimizer, achieving
91.7% accuracy in Italian sign language classification. These studies proved the strength of
CNNs in handling noise, variability, and high-dimensional input.

B.E.,Dept of CSE.,CITech 2024-25 Page 3

Deep learning-based sign language recognition system for static signs Literature Survey

2.3 Diverse Architectures and Innovations [3]

Several researchers proposed customized CNN architectures to improve sign language
recognition. Molchanov et al. introduced a multi-sensor system using depth, radar, and optical
sensors with CNNs, reaching 94.1% accuracy. Tang et al. applied LeNet-5 CNN on 36 hand
postures using Kinect data and found Deep Belief Networks outperforming CNNs with 98.12%
accuracy. Yang and Zhu simplified hand segmentation in Chinese Sign Language recognition
using CNNs and showed Adadelta optimizer yielding better results than Adagrad.

2.4 Indian Sign Language and Hybrid Models [4]

Indian Sign Language (ISL) has gained attention recently due to the lack of real-time
recognition systems. Rao et al. developed a selfie-based ISL recognition system using CNNs
and stochastic pooling, achieving an accuracy of 92.88% across 200 signs in varied lighting
and backgrounds. Koller et al. proposed a hybrid CNN-HMM model combining CNN's visual
recognition capability with HMM's sequence modeling for continuous sign recognition,
improving accuracy over standalone CNNs. Kumar et al. built a two-stream CNN model using
JDTD and JATD descriptors, trained on a 50,000-sign video dataset.

2.5 Gesture Recognition Using Autoencoders and Deep Belief Networks [5]
Autoencoders and Deep Belief Networks (DBNs) have been explored for their capacity to
extract abstract and layered features from gesture images. Oyedotun and Khashman used
Stacked Denoising Autoencoders (SDAE) and CNNs for static ASL gestures, achieving
accuracies of 91.33% and 92.83% respectively. These models were trained on public gesture
databases and showed significant improvement over shallow learning models. DBNs were
particularly effective in learning hierarchical representations, while SDAEs helped denoise and
refine the input images before classification. These findings demonstrate that unsupervised
deep models can provide an effective alternative to standard CNNs. By fine-tuning the final
layers of these models, researchers were able to leverage powerful feature extractors without
needing extensive data.

B.E.,Dept of CSE.,CITech 2024-25 Page 4

CHAPTER 3
ARCHITECTURE
The CNN architecture includes three main layers: input, feature extraction, and classification. The
input layer accepts preprocessed static hand gesture images. The feature extraction layer uses
convolutional layers to detect spatial features. ReLU activations introduce non-linearity for better
learning. Max-pooling layers reduce spatial size and retain important features. Flattening
transforms the 2D features into a 1D vector. The classification layer contains fully connected
layers to map features to gesture classes. A dropout layer prevents overfitting during training. The
softmax layer gives final probabilities for each sign class..

Fig 3.1: High-level general CNN architecture

3.1 Back-End Components:

3.1.1 Convolutional Neural Network (CNN) Model:
• The core of the system lies in the CNN-based deep learning model used for
classifying static Indian Sign Language gestures.

• The CNN model consists of multiple convolutional layers followed by ReLU

activations, max-pooling layers, dropout layers to avoid overfitting, and fully
connected layers.

B.E.,Dept of CSE.,CITech 2024-25 Page 5

Deep learning-based sign language recognition system for static signs Architecture

3.1.2 Relational Database:

• A relational database is used to store recognized signs, timestamps, user details,
accuracy scores, and session logs.
• A structured storage system (e.g., cloud storage or local servers) is used to store
labeled image datasets. This facilitates training, testing, and validation with
efficient access to the required data.

3.1.3 Model Training Environment:

• This includes GPU-enabled environments like TensorFlow or PyTorch for
training the CNN on large datasets.

3.1.4 Server:
• The server acts as a central hub for managing model inference, handling client
requests, and interacting with both the database and the blockchain.
• Manages smart contracts and stores related data in the database.

3.2 Front-End Components:

3.2.1 Mobile Devices:
• Mobile devices serve as the primary interface for users to interact with the system.
The mobile application includes a user-friendly interface that captures real-time sign
images via the camera, processes them for background subtraction.

3.2.2 IoT Sensors:

• In advanced use cases, IoT-based gloves or sensors may be integrated to enhance
gesture recognition accuracy.

3.3 Workflow
The system begins with the user performing a static or dynamic sign in front of the camera
on a mobile device. The camera captures the frame and applies preprocessing techniques like
background subtraction or grayscale transformation. The image is then sent to the server
where the CNN model processes the input through multiple layers including convolutional,
ReLU, pooling, and fully connected layers to classify the sign.

B.E.,Dept of CSE.,CITech 2024-25 Page 6

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Supply-Chain Architecture

Fig 3.2. System flowchart

B.E.,Dept of CSE.,CITech 2024-25 Page 7

CHAPTER 4
IMPLEMENTATION

4.1 Methods
The proposed Sign Language Recognition System is implemented through four key phases: Data
Acquisition, Data Preprocessing, Model Training, and Testing. Each phase is crucial in
developing a robust and accurate CNN-based classifier for recognizing static Indian Sign
Language (ISL) signs. The system aims to bridge the communication gap for individuals with
hearing or speech impairments by translating visual gestures into meaningful text.

4.1.1. Data Acquisition

• This phase involves capturing RGB images of static signs using a camera.
The dataset contains 35,000 images representing 100 different sign classes.
• Each class includes 350 images covering English alphabets, digits, and common words.
Examples of words include "water", "bowl", "hand", and "fever".
• Images are taken under varying environmental conditions like lighting and background.
This diversity improves the system’s ability to generalize across real-world settings.
• The collected images are then passed to the preprocessing module for further steps.

4.1.2 Data Preprocessing

• Preprocessing enhances image quality and prepares data for training.
Morphological operations are applied to remove noise from the images.
• All images are resized uniformly to 128×128 pixels.
Normalization is applied to adjust pixel values to mean 0 and variance 1.ensuring real-
time monitoring and compliance verification.

• These steps ensure consistency in image size and scale for CNN input.
Preprocessed images are stored and used in both training and testing phases.

B.E.,Dept of CSE.,CITech 2024-25 Page 8

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Methods and challenges

4.1.3 Model Training

• Training is done using a Convolutional Neural Network (CNN) architecture.
The model is trained on a system with Tesla K80 GPU, 12 GB memory, and 64 GB RAM.
• The dataset is shuffled to ensure randomness and then split 80:20 for training and
validation.CNN learns spatial features from images like shape, texture, and edges.
• The classifier categorizes each input image into one of the 100 sign classes.
Training uses multiple epochs and tuning to reduce loss and improve accuracy.

4.1.4 Data Storage and Processing

• While blockchain ensures immutable and decentralized record-keeping, raw temperature
data is stored in a PostgreSQL relational database due to storage limitations on the
blockchain.
• Smart contracts store only verification results and metadata, with a URL linking to the full
dataset stored off-chain.
• The Ethereum node, hosted on the modum.io AG server, monitors blockchain transactions
and processes smart contract executions.
• JSON-based REST APIs are used to enable efficient communication between different
system components.

4.1.5 Testing
• The model is evaluated by testing it with unseen data after training.
Around 50 different CNN models with various optimizers are tested.
• Testing helps in identifying the model with the best performance.
Fine-tuning is done by adjusting parameters to enhance accuracy.
• The best-performing model is finalized for deployment.
Testing validates the system’s effectiveness in recognizing static ISL signs.

4.2 Challenges
During the development of the system, we faced several challenges such as collecting a balanced and
diverse dataset, managing variations in lighting and background during image capture, and avoiding
overfitting in CNN models.

B.E.,Dept of CSE.,CITech 2024-25 Page 9

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Methods and Challenges

4.2.1 Dataset Imbalance and Collection Difficulty

Creating a balanced dataset was one of the major challenges in this project. Certain sign gestures
were easier to perform and capture, while others required multiple attempts to get consistent
images. As a result, some classes had fewer samples, which affected the model's learning.
Additionally, variations in user hand shapes, skin tones, and orientations made the dataset less
uniform. Manual collection also introduced fatigue and inconsistency in gesture performance.
Ensuring equal representation of all 100 classes took considerable effort and time. This imbalance
risked biasing the model toward frequently represented signs.

4.2.2 Variations in Lighting and Background

Sign images were captured in different environmental conditions, leading to inconsistent image
quality. Changes in lighting, shadows, and varying backgrounds created noise in the images,
making feature extraction difficult for the CNN. Some signs got misclassified due to poor visibility
or overlapping backgrounds. Controlling the environment for every capture was not always
feasible, especially during dataset expansion. Although preprocessing techniques helped reduce
this issue, some noise still passed through. These inconsistencies affected the generalization
performance of the model during testing.

4.2.3 Overfitting in CNN Models

During training, the model often learned the training data too well, resulting in poor performance
on new data — a classic case of overfitting. The model showed high training accuracy but low
validation accuracy, indicating it wasn’t generalizing well. Techniques like dropout layers, L2
regularization, and data augmentation were introduced to reduce overfitting. However, fine-tuning
these techniques for optimal performance took several iterations. Overfitting also varied
depending on the optimizer and CNN configuration used. Regular validation and testing became
essential to keep the model in check.

4.2.4 Computational Resource Constraints

Training a deep learning model, especially on a dataset of 35,000 images, demands high
computational power. While the project used a Tesla K80 GPU and 64 GB RAM, the training
process still took several hours. Experimenting with multiple CNN architectures and optimizers
further increased the processing time.

B.E.,Dept of CSE.,CITech 2024-25 Page 10

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Methods and Challenges

Real-time tuning and debugging were delayed due to limited hardware availability. Handling large
image batches sometimes caused memory overflow or system lag. These constraints slowed down
experimentation and development cycles significantly.

4.1.5 Accuracy Trade-off in Similar Signs

Some signs in Indian Sign Language have very subtle visual differences, such as those for "stand",
"stop", or "sit". These signs confused the model, especially when the hand orientation or finger
positioning was slightly off. Even minor variations by the same person performing the same sign
led to misclassifications. The CNN struggled to identify features that were distinguishable enough
across similar-looking classes. Increasing model depth helped slightly, but also increased training
time and risk of overfitting. More advanced techniques like attention mechanisms might be needed
for future improvement.

B.E.,Dept of CSE.,CITech 2024-25 Page 11

CHAPTER 5

REAL WORLD APPLICATIONS

Deep learning models, once trained on static sign images, can be integrated into applications that
convert signs into corresponding alphabets or words in real time. These applications can run on
desktops or mobile devices and use the device’s camera to capture input gestures. The processed
output is instantly displayed as text, which can help deaf or mute individuals interact easily with
people who don’t understand sign language.This enables smooth communication without the need
for a human interpreter. The model can be optimized for low-latency performance, making it
suitable for real-time interactions. Users can practice their signs and get immediate feedback,
improving their accuracy and confidence. With multilingual support, the output text can be
translated into different languages.

5.1 Deep Learning in Sign Language Recognition

5.1.1 Real-Time Sign-to-Text Conversion Technology
Deep learning models, once trained on static sign images, can be integrated into applications that
convert signs into corresponding alphabets or words in real time. These applications can run on
desktops or mobile devices and use the device’s camera to capture input gestures. The processed
output is instantly displayed as text, which can help deaf or mute individuals interact easily with
people who don’t understand sign language. This technology promotes inclusivity by breaking
communication barriers in daily life. It also reduces the dependency on human interpreters in
casual or professional settings.

5.1.2 Educational Tools for Learning Sign Language System

CNN-based recognition systems are being embedded into e-learning platforms, making it easier
for students or caregivers to learn sign language. The application provides feedback by identifying
whether the user is performing the correct gesture. This form of interactive learning is far more
effective than traditional textbook methods. Learners can practice at their own pace and track their
progress over time. Visual feedback and scoring systems help in improving accuracy and
motivation.

B.E.,Dept of CSE.,CITech 2024-25 Page 12

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Supply-Chain Real-World

5.1.3 Assistive Communication Devices for Accessability

Deep learning models are used in assistive devices that serve as gesture-to-speech converters. These
devices help convert static hand signs into spoken language using a speaker, enabling real-time
conversation. This enhances communication in public places, workplaces, hospitals, and schools,
making the world more inclusive for hearing-impaired individuals. Such devices can be wearable,
portable, and user-friendly, allowing seamless communication on the go. They reduce the need for
a human interpreter in many everyday situations.

5.1.4 Integration into Mobile and Web Applications Platforms

Many startups and research teams are integrating sign recognition models into mobile apps. These
applications can detect hand gestures through the phone’s camera and translate them into English
or any local language. The offline functionality of deep learning models is making it possible to
use these tools even in remote areas without internet access. These apps can be especially helpful
in emergency situations where quick communication is essential. Features like voice output, text
display, and gesture history enhance user experience and accessibility.

5.1.5 Enhanced Customer Support Accessibility Solutions

Banks, government offices, and hospitals can adopt this technology to make their customer service
accessible to deaf or mute individuals. For example, a static-sign recognition kiosk at a railway
station can help users choose travel options, make payments, or ask for help—all
This not only improves user experience but also promotes equality in public service delivery.
Touch-free interactions also contribute to hygiene and safety in high-traffic areas. Automated sign
recognition systems can reduce waiting times and dependency on staff. Over time, this can lead to
cost savings and more efficient, inclusive service models. pharmaceutical industry.

5.1.6 Smart Classrooms and Inclusive Education

By integrating sign recognition systems into smart classroom tools, hearing-impaired students can
be given equal learning opportunities. Real-time sign translation can help them understand
content, respond to questions, and interact with teachers through technology-assisted
communication. This fosters an inclusive learning environment where no student feels left behind.
It also empowers educators to adapt their teaching methods to meet diverse student needs more
effectively.

B.E.,Dept of CSE.,CITech 2024-25 Page 13

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Supply-Chain Real-World

5.3 CNN: Data Collection, Preprocessing, and CNN-Based Image Recognition

for Sign Language

Fig 5.1: Sample Dataset

To build a deep learning model for static sign language recognition, the dataset is primarily
collected from available public datasets of Indian Sign Language (ISL) or similar sign language
datasets. The data collection involves capturing hand gestures representing different sign language
symbols (such as alphabets or common words) using high-resolution cameras under various
lighting conditions. To ensure the model is diverse and generalizable, the dataset includes images
from different users with various hand orientations, skin tones, and backgrounds. To prevent
overfitting and enrich the dataset, data augmentation techniques are applied. These include small
random rotations to simulate different hand orientations, horizontal flipping to introduce variation,
slight zoom to handle gestures performed at different distances, and brightness adjustments to
simulate different lighting environments. These augmentations help increase the dataset's
diversity, improving the model's robustness. Each image is labeled according to the corresponding
gesture (e.g., "A", "B" for alphabet recognition). For word-based recognition, labels represent
entire words or phrases. The data is organized into directories by gesture. The data is organized
into directories by gesture class, facilitating easy access for training the model. Before feeding the
data into the model, preprocessing steps are applied.

B.E.,Dept of CSE.,CITech 2024-25 Page 14

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Supply-Chain Real-World

Sign Precision Recall F1-score Sign Precision Recall F1-score

A 1.00 0.96 0.98 Me 1.00 1.00 1.00

Afraid 0.97 0.97 0.97 Nose 0.98 1.00 0.99
B 1.00 1.00 1.00 Oath 1.00 1.00 1.00
Bent 0.97 1.00 0.99 Open 1.00 0.97 0.98
Coolie 0.97 0.94 0.96 P 1.00 0.97 0.98
Claw 1.00 1.00 1.00 Pray 1.00 1.00 1.00
D 0.79 0.97 0.87 Q 0.97 1.00 0.99
Doctor 0.98 1.00 0.99 S 0.95 1.00 0.97
Eight 0.96 0.90 0.93 Sick 1.00 1.00 1.00
Eye 1.00 1.00 1.00 Strong 0.97 1.00 0.98
Fever 0.95 1.00 0.97 T 0.99 1.00 0.99
Fist 0.97 0.98 0.97 Tongue 0.99 1.00 0.99
Gun 0.97 1.00 0.99 Trouble 1.00 0.95 0.97
H 1.00 1.00 1.00 U 1.00 0.99 0.99
Hand 0.97 1.00 0.98 V 1.00 1.00 1.00
I 1.00 1.00 1.00 West 1.00 0.93 0.96
Jain 0.99 1.00 0.99 Water 0.93 0.98 0.95

Fig 5.2: Classification Performance

The classification performance of the deep learning model is evaluated based on its ability to
accurately recognize static sign language gestures from the input images. Metrics such as
accuracy, precision, recall, and F1-score are used to assess the model's performance. Accuracy
measures the percentage of correctly classified gestures out of all predictions, while precision and
recall provide insight into the model's ability to minimize false positives and false negatives,
respectively. The F1-score balances precision and recall, offering a single metric to evaluate the
model's overall effectiveness. A confusion matrix is also used to visualize misclassifications and
identify patterns in the model's errors. By optimizing the model through hyperparameter tuning
and data augmentation, classification performance can be improved, ensuring robust and accurate
sign language recognition. Additionally, cross-validation techniques are employed to assess the
model’s ability to generalize to unseen data, preventing overfitting. A thorough evaluation on both
training and test datasets helps fine-tune the model, providing a reliable solution for real-world
applications. Continuous monitoring of performance ensures that the model remains efficient and
adaptable over time. Regular updates to the dataset, including adding more diverse sign language
gestures, help improve the model’s robustness. Furthermore, fine-tuning the model with domain-
specific data can enhance its effectiveness in particular settings, such as educational or medical
environments.

B.E.,Dept of CSE.,CITech 2024-25 Page 15

Blockchains Everywhere: A Use-case of Blockchains in the Pharma Supply-Chain Real-World

5.4 Challenges in Real-World Implementations

1. Environmental Variations: Different lighting conditions, camera angles, and clut.
Shadows, brightness changes, or unclear hand positioning often confuse the model.
Dynamic environments make it harder to detect hand gestures consistently.
2. Hardware Limitations: Running CNNs or deep models on mobile or embedded devices
Real-time recognition requires high frame rates, which low-end devices can't sustain.
Memory and power constraints on mobile platforms reduce system responsiveness.
Lightweight models or hardware acceleration (e.g., GPUs, TPUs) are required for deployment.
Many pharmaceutical companies rely on traditional supply chain management systems that
may not be compatible with blockchain technology. Transitioning
3. Dataset Scarcity: Lack of diverse, labeled datasets for Indian Sign Language (ISL) limits
model training. Most publicly available datasets are limited to American Sign Language A
small or unbalanced dataset causes overfitting and poor real-world performance.
Creating comprehensive ISL datasets with user diversity is essential but time-consuming.
tracking capabilities, affecting supply chain efficiency.
4. Integration and Usability: Integrating with existing systems like mobile apps, IoT sensors,
Ensuring a smooth and intuitive interface for users from varied backgrounds is difficult.
-technical users may struggle with system setup and troubleshooting.
Non-UI/UX design must focus on accessibility and minimal user effort. Pharmaceutical
companies must comply with strict regulations set by authorities such as the FDA (U.S.), EMA
(Europe), and WHO. While blockchain
5. Sign Ambiguity and Similarity: Many signs resemble one another closely, making
Subtle differences in finger position or motion aren't always captured clearly.
Advanced spatial-temporal modeling is needed for accurate recognition.. Striking a balance
between transparency and privacy is crucial for industry-wide adoption.

B.E.,Dept of CSE.,CITech 2024-25 Page 16

 CONCLUSION

The proposed Deep Learning-Based Sign Language Recognition System efficiently recognizes
static signs from Indian Sign Language using a customized Convolutional Neural Network (CNN).
The system achieves high accuracy by extracting spatial features through convolutional, pooling,
ReLU, and fully connected layers. A modular architecture with front-end components like cameras
and back-end components such as a trained CNN model, server, and database ensures smooth
functioning. The dataset, collected under varied environmental conditions, enhances the system’s
robustness. Preprocessing techniques like background subtraction and image resizing further
improve recognition performance. Metrics like precision, recall, and F1-score confirm the system's
reliability. The dropout layer prevents overfitting, ensuring better generalization. The web camera-
based interface allows real-time interaction, making the system accessible and user-friendly. The
CNN model is trained to classify digits, alphabets, and common words in ISL with high
confidence. The system helps bridge communication gaps for hearing or speech-impaired
individuals. Its scalable design allows future extensions such as dynamic sign recognition and
integration with mobile platforms. The hybrid architecture supports multi-class classification and
efficient data handling. With nearly 50 model variations tested, the chosen configuration shows
superior performance. This sign recognition system represents a valuable application of AI in
assistive technology, making communication more inclusive.

B.E.,Dept of CSE.,CITech 2024-25 Page 17

 REFERENCES

[1] Corballis MC (2003) From mouth to hand: gesture, speech and the evolution of right-handedness.
Behav Brain Sci 26(2):199–208.
[2] Oyedotun OK, Khashman A (2017) Deep learning in vision- based static hand gesture recognition.
Neural Comput Appl 28(12):3941–3951.
[3] Nagi J, Ducatelle F, Di Caro GA, Cires¸an D, Meier U, Giusti A, Gambardella LM (2011) Max-
pooling convolutional neural net- works for vision-based hand gesture recognition. In: IEEE
international conference on signal and image processing appli- cations (ICSIPA), pp 342–347.
[4] Huang J, Zhou W, Li H, Li W (2015) Sign language recognition using 3D convolutional neural
networks. In: IEEE international conference on multimedia and expo (ICME), pp 1–6.
[5] Arora, S.; Roy, A. Recognition of sign language using image processing. Int. J. Bus. Intell. Data
Min. 2018, 13, 163–176.
[6] Lin, H.; Hong, X.; Wang, Y. Object Counting: You Only Need to Look at One. arXiv 2021,
arXiv:2112.05993.
[7] Rioux-Maldague L, Giguere P (2014) Sign language finger- spelling classification from depth and
color images using a deep belief network. In: IEEE Canadian conference on computer and robot
vision (CRV), pp 92–97.
[8] Dhulipala, S.; Adedoyin, F.F.; Bruno, A. Sign and Human Action Detection Using Deep Learning.
J. Imaging 2022, 8, 192.
[9] Alvarez-Estevez, D.; Rijsman, R.M. Inter-database validation of a deep learning approach for automatic
sleep scoring. PLoS ONE 2021, 16, e0256111.
[10] Kaluri, R.; Pradeep Reddy, C.H. Sign gesture recognition using modified region growing algorithm
and Adaptive.
[11] A.M.; Kamel, A.E.; Slim, S.O.; Abdallah, M.S.; Cho, Y.I. MediaPipe’s Landmarks with RNNfor
Dynamic Sign Language Recognition. Electronics 2022, 11, 3228.
[12] Dang, C.N.; Moreno-García, M.N.; De La Prieta, F. Hybrid Deep Learning Models for Sentiment
Analysis. Complexity 2021, 2021, 9986920.
[13] Aly, S.; Aly, W. DeepArSLR: A novel signer-independent deep learning framework for isolated
arabic sign language gestures recognition. IEEE Access 2020, 8, 83199–83212.
[14] Huang,Y.; Huang, J.; Wu, X.; Jia, Y. Dynamic Sign Language Recognition Based on CBAM with
Autoencoder Time Series Neural Network. Mob. Inf. Syst. 2022, 2022, 3247781.
[15] Mekala, P.; Gao, Y.; Fan, J.; Davari, A. Real-time sign language recognition based on neural
network architecture. In Proceedings of the 2011 IEEE 43rd Southeastern Symposium on System
Theory, Auburn, AL, USA, 14–16 March 2011; pp. 195–199.
[16] Al-Shaheen, A.; Çevik, M.; Alqaraghuli, A. American Sign Language Recognition using YOLOv4
Method. Int. J. Multidiscip. Stud. Innov. Technol. 2022, 6, 61.
[17] Kothadiya, D.; Bhatt, C.; Sapariya, K.; Patel, K.; Gil-González, A.B.; Corchado, J.M. Deepsign:
Sign Language Detection and Recognition Using Deep Learning. Electronics 2022, 11, 1780.
[18] Gunji, B.M.; Bhargav, N.M.; Dey, A.; Zeeshan Mohammed, I.K.; Sathyajith, S. Recognition of
Sign Language Based on Hand Gestures. J. Adv. Appl. Comput. Math. 2022, 8, 21–32.
[19] Agarwal, S.R.; Agrawal, S.B.; Latif, A.M. Sentence Formation in NLP Engine on the Basis of
Indian Sign Language using Hand Gestures. Int. J. Comput. Appl. 2015, 116, 18–22.

B.E.,Dept of CSE.,CITech 2024-25 Page 18