1.

INTRODUCTION
Sign language is a key tool in order to survive in society
for hearing and handicapped persons. Linguistic
communication minimises discomfort through visual
speech instead of hearing in their day-to-day lives.
Communication is extremely important for men, as it
allows the US to determine itself. Speech is one of the
most unknottable things used among them, which is
communicating through speech, gesture, visual
communication, reading, writing or visual support.
Unfortunately, however, there is a communication gap for
the speech and hard-to-hear minority. Area unit used for
human activity with the visual aids, or interpreters of AN.
These ways, however, unit area rather cumbersome and
expensive, and cannot be used in an emergency.
Linguistic communications mainly transmit what means
by manual communication. This combines the hand
forms, directions and the movement of the hands with the
thoughts of the speaker. They are much less varied than
spoken languages. India has its own language
communication under the Indian language
communications name (ISL). Only a few colleges for deaf
students can be found in developing countries. d
1
equivalent Languages that have been signed Government
rates for adults with surface areas in developing countries
are terribly high. Ethnologue's information says that the
overall population, performance rate and variety of
teaching children among the deaf people in India are
incredibly smaller. It continues to state that official
recognition of sign languages increases significantly the
availability of interpreters and transcripts in sign
languages. Signs in the area of sign languages unit wor
appear favourable. The area of sign languages has mainly
been developed to assist deaf and dumb people. They use
a combination of hand movements, hand forms, and
orientation to provide explicit information in a chronical
and specific way. One of the languages used in Southeast
Asia is the Indian linguistic communication (ISL) system.
The boundary between ISL and alternative sign languages
is that ISL empties any temporal inflections in the
fingerprint system chart, as well as the use of each hand.
With the presence of unnaturally smart algorithms, not to
mention the provision of massive information and
enormous procedural resources, LEDs have increased
considerably in tending, robotics, automotive, HLI, etc.
The detection of hand gestures could be a hard task,
significantly because of its use of each hand the ISL
2
recognition is difficult. During this respect, however,
several studies are conducted with the victimisation
sensors (such as a glove sensor) and alternative imaging
methods (such as edge Sign Language Recognition Using
Machine Learning Algorithm Dept of CSE 10 2020-21
detection technique, etc.). But the performances in this
field have increased considerably with the new deep
learning techniques like CNN which led to many new
perspectives for the future. Many people in India have
discomfort in speech and/or hearing and use hand gestures
to talk to others. But, apart from a few, most are not
attentive to this linguistic communication and may need a
non-convenient and expensive an interpreter. This project
seeks to narrow this communication gap by developing a
package that can predict the alphanumeric gestures of the
ISL hand in real time.

1.1 Sign Language Recognition

The language of signs may be a visual language used by
people with disabilities. It makes it easier for people to
communicate. Because of the lack of signature
3
recognition, it is of great social importance for people
with hearing impairments to develop a human-computer
interaction system that offers a platform to speak to
others. The language of the sign is that with completely
different gestures of hand, the only medium that
especially capable individuals can connect to the rest of
the world. The recognition of gesticulations has become
extremely important with the progress made in machine
learning.

1.2. Solution Implemented in the Project

The input aspect a video camera employs or with
integrated image processing system allows the calculation
of vectors options for capturing an imagery to overcome
the inconveniences of the existing system. The process of
segmentation takes place compared to images on the train
sign recognition, sign can realise a proof language
interface to alphabet, which is converted to completely
different classifiers and the accuracy is high and user
friendly, the main benefit of this project is that the user
can't wear anything and that we can use the integrated
camera.
• The goal is to create a simple human interface wherever
4
the laptop perceives human signature
• This uses completely different classifiers to convert a
text to a sign language
• Results square measure in terms of maximum precision

1.3 Justification of the Project

From time immemorial communication has been the
means for building relationships, understanding
individuals, perceiving technology, and allowing the
world to grow and develop. Traditional people
communicate their ideas and thoughts through speech to
others. One important element is that the use of signing is
one of communication methodologies for the deaf
community. The people's lakhs use signing as their
principal means of communication.

1.4. Objectives
Our project aims to bridge the gap between people with
speech impairment and ordinary individuals. The main
idea for this project is to create a system that stupid
people can use their ordinary gestures to communicate
significantly with all others. SLR aims at developing
5
algorithms and methods for properly identifying and
understanding a sequence of the signs produced. Many
methods of SLR treat the problem correctly as gesture
recognition. The goal is to develop an easy-to-use
interface to understand the language of human signs. The
interface uses different classifiers to convert the sign
language into text The results are shown in the highest
precision
➢ The main purpose of this project is to thoroughly
identify hand gestures, images in order to achieve a
communication gap between an individual with a normal
hearing impairment
➢ The main aim of this project was to establish the
usability of the system for gesture recognition
➢ Pre-processing the extracted data to eliminate
unnecessary information (like removing any unnecessary
information and keeping the required database
➢ The algorithm with different features will be trained to
produce the correct output
➢ And spell correction of a word will be generated
➢ And we also add additional feature like text to speech
conversion

6
 2. SYSTEM ANALYSIS

2.1 Identification of the Need

Identifying the need for sign language recognition using
machine learning algorithms involves understanding the
challenges faced by the deaf and hard of hearing
community, as well as recognizing the potential benefits
that such technology can offer. Here are some key points
to consider:
Communication Barrier:
Deaf and hard of hearing individuals face significant
communication barriers in interacting with the
hearing world, particularly in situations where sign
language interpreters are not available.

Sign language recognition technology can help

bridge this communication gap by enabling
real-time translation of sign language into
7
spoken or written language, facilitating
communication with hearing individuals.
Accessibility:
Many digital platforms and services are not
accessible to individuals who primarily use
sign language for communication.

Sign language recognition technology can

enhance the accessibility of these platforms by
enabling sign language users to interact with them
more effectively
Empowerment and Independence:
Access to sign language recognition technology
empowers deaf and hard of hearing individuals to
communicate more independently and confidently
in various settings, such as schools, workplaces, and
public spaces.

It reduces reliance on intermediaries like sign language

interpreters, giving users greater autonomy over their
communication choices and facilitating their participation
in social and professional activities.
Education and Learning:
Sign language recognition can facilitate the

8
 teaching and learning of sign language, both for
deaf individuals and hearing individuals
interested in learning sign language.

Emergency Situations:
In emergency situations, such as medical emergencies or
natural disasters, communication barriers can have life-
threatening consequences for deaf and hard of hearing
individuals.

Sign language recognition technology integrated into

emergency response systems can enable more effective
communication between emergency responders and
individuals who use sign language, ensuring timely
assistance and support.

Inclusivity and Diversity:

Embracing sign language recognition
reflects a commitment to inclusivity and
diversity,acknowledging the linguistic and
cultural rights of deaf communities.

9
 It promotes a more inclusive society where all
individuals, regardless of their
communication preferences or abilities, can
participate fully and contribute meaningfully.

2.2. Project Planning and Scheduling

Gantt Chart:
A Gantt chart is a popular type of bar chart that illustrates
a project schedule. Gantt charts illustrate the start and
finish dates of the terminal elements and summary
elements of a project. Terminal elements and summary
elements compromise the work breakdown structure of
the project.

1.
PERT Chart:
10
A PERT (Program Evaluation and Review
Technique) Chart is another project management tool
in which the stages of a project are graphically
represented on a chart.
Fig: Pert Chart

2.3. Software Requirement Specifications

Software requirements for the implementation and
testing project are mentioned below
1. Operating system : Windows 10
2. Coding Language : Python
3. Interpreter : Python 3.6
4. IDE : Python IDE
5. ML APIS : Sklearn, numpy, pandas, matplotlib,
keras, tensorflow

Software Tools Used:

Python:
Python is an interpreted object-oriented, complicated
semantic high-level programming language. It is
ideal in screening and rapid application development
with high-level built-in data structures coupled with
11
dynamic timing and dynamic binding. Python's easy
to learn syntax puts readability premium on it,
reducing maintenance costs for software. Python
supports modules and bundles, enabling modularity
of software and reuse of code. The Python parser is
open source, as is a wide standard library. Python is
widely considered to be the best language for
teaching and learning. The word ML (Machine
Learning) refers to the process of learning something
new. Python is a major eason why machine learning
is so widely known. It is easy to read and is loved by
experienced developers and experimental students
both. It is easy to read. Python's simplicity allows
developers to focus rather than spend their entire time
on solving the machine learning problem. Python is a
major reason why machine learning is so popular. It
is easy to read and is loved by experienced
developers and innovative students both. It is easy to
read. Python's simplicity means that developers can
concentrate on fixing the situation of machine
learning rather than spending time. Due to its
portable and extendable character, many cross-
language operations can easily be carried out on
Python. Python learners can easily improve their
12
knowledge of machine learning, which only makes
them more popular.

Features:
• Interpreted Language
• Large Standard Library
• High-Level Language
• Dynamically Typed Language
• Easy to code
• Free and Open Source
• Object-Oriented Language
• Extensible feature
• Python is Portable language
• Python is Integrated language

Python IDLE:
An integrated development environment (IDE) is a
software development programme that integrates a
number of tools designed specifically for software
development. These methods can be used in a variety
of ways.
o A code editor is a piece of software that lets you
edit code.
o Source control software o Creation, execution, and
debugging tools When you install Python, IDLE is
13
 already installed.
o Its core capabilities include the Python shell
window, self finishing, syntax emphasis, smart
indentation and a simple built-in Debugger.
o Python 3.6.8 was used in this project.

2.4. Software Engineering Paradigm

Applied

1. Requirements Engineering Begin by clearly

defining the requirements of the sign language
recognition system. This involves understanding the
target user group, the context in which the system
will be used, and the specific gestures or signs to be
recognized.
2. Modeling: Use modeling techniques such as UML
(Unified Modeling Language) to create diagrams that
represent the system's structure and behavior. This
can include class diagrams, sequence diagrams, and
state diagrams to help visualize the system's
architecture and interactions.
14
3. Agile Development: Adopt agile methodologies such
as Scrum or Kanban for iterative development and
frequent feedback loops. This allows for continuous
improvement and adaptation based on user feedback
and evolving requirements.
4. Version Control: Utilize version control systems
like Git to manage the source code of the project.
This facilitates collaboration among team members,
allows for easy tracking of changes, and ensures that
the codebase is always in a stable state.
5. Testing: Implement various testing strategies
including unit testing, integration testing, and end-to-
end testing to ensure the correctness and robustness
of the sign language recognition system. This
involves creating test cases that cover different
scenarios and edge cases.
6. Continuous Integration and Deployment (CI/CD):
Set up CI/CD pipelines to automate the build, testing,
and deployment processes. This helps in detecting
and fixing issues early in the development cycle, and
ensures that the system can be deployed quickly and
reliably.
7. Documentation: Document the design,
implementation, and usage of the sign language
15
 recognition system to aid in understanding and
maintenance. This includes writing API
documentation, user manuals, and developer guides.
8. Security: Pay attention to security considerations
such as data privacy, authentication, and
authorization, especially if the sign language
recognition system involves sensitive information or
interactions with users.
9.Performance Optimization: Optimize the
performance of the system by profiling and analyzing
its resource usage, identifying bottlenecks, and
implementing optimizations where necessary. This
ensures that the system can run efficiently and handle
real-time sign language recognition tasks effectively.

2.4. Data Models

DATA FLOW DIAGRAM:

16
 Fig: data flow diagram

System Architecture:

17
 Fig: System Architecture

Sequence Diagram:

18
 Fig: Sequence Diagram

3. SYSTEM DESIGN

3.1. Modularisation Details

Modularization in sign language recognition using
machine learning algorithms involves breaking down the
system into smaller, more manageable components or
19
modules, each responsible for a specific task or aspect of
the recognition process. Here's a breakdown of how you
might modularize such a system:

1. Data Acquisition Module :

This module is responsible for capturing sign language

data. It could involve using cameras or sensors to record

hand movements and gestures.
2. reprocessing Module:
Raw data captured may need preprocessing to enhance
features or remove noise. This module may involve tasks
such as noise reduction, normalization, or feature
extraction.
3. Feature Extraction Module:
This module extracts relevant features from the
preprocessed data. For sign language recognition, features
might include hand shape, movement trajectory, hand
orientation, or finger positions.
4.Model Training Module:
Here, machine learning algorithms are trained on the
extracted features. This module involves tasks such as
selecting appropriate algorithms (e.g., SVM, Random
Forest, Neural Networks) and optimizing
20
hyperparameters.
5.Model Evaluation Module:
After training, the performance of the models needs to be
evaluated. This module typically involves techniques such
as cross-validation, confusion matrix analysis, or
precision-recall curves.
6. Integration Module:
Once individual modules are developed and tested, they
need to be integrated into a cohesive system. This module
ensures seamless communication and data flow between
different components.
7. Real-time Recognition Module:
In a real-time system, this module continuously processes
incoming data from sensors or cameras, applies the
trained models, and recognizes sign language gestures on-
the-fly.
8. User Interface Module:
This module provides a user-friendly interface for
interaction with the system. It could be a graphical
interface displaying recognized gestures or a feedback
mechanism for users to correct misinterpretations.
9.Feedback and Adaptation Module:

21
This module collects user feedback and adapts the system
over time. It might involve techniques such as online
learning or retraining the model with new data to improve
performance.
10. Deployment and Maintenance Module:
Finally, this module handles the deployment of the system
in real-world environments and ongoing maintenance
tasks such as software updates, bug fixes, and
performance monitoring.
Modularization allows for easier development, testing,
and maintenance of the sign language recognition system,
as each module can be developed and optimized
independently before integration into the larger system. It
also promotes reusability, as individual modules can be
repurposed for other projects or integrated into different
applications.

3.2. Data Integrity and Constraints

Data integrity and constraints are crucial considerations in
sign language recognition using machine learning
algorithms to ensure accurate and reliable results. Here's
how these concepts apply to such a system:
Data Integrity:
22
Data Quality: Ensuring that the data used for training and
testing the machine learning models is of high quality is
essential. This involves collecting data from reliable
sources and ensuring it accurately represents the sign
language gestures you want to recognize.
Data Cleaning: Before training the models, it's important
to clean the data to remove any inconsistencies, errors, or
noise. This could involve techniques such as outlier
detection, missing value imputation, and normalization.
Data Augmentation: To enhance the diversity and
richness of the dataset, data augmentation techniques can
be employed. This involves generating additional training
samples by applying transformations such as rotation,
translation, or scaling to existing data.
Data Privacy and Security: Since sign language data
may contain sensitive information, maintaining data
privacy and security is paramount. Implementing
encryption, access controls, and anonymization
techniques can help protect the privacy of individuals
whose sign language data is used.
Constraints:

23
Computational Resources: Machine learning algorithms,
especially deep learning models, can be computationally
intensive. Constraints related to computational resources
such as memory, processing power, and latency need to
be considered when designing the system.
Real-time Processing: If the sign language recognition
system needs to operate in real-time, there are constraints
on the processing time and latency. Algorithms and
architectures optimized for real-time performance may
need to be prioritized.
Accuracy Requirements: Depending on the application,
there may be constraints on the accuracy and reliability of
the recognition system. Balancing accuracy with other
factors such as computational complexity and real-time
performance is important.
Hardware Constraints: The hardware platform on which
the system will run may impose constraints. For example,
if the system is intended for deployment on mobile
devices, there may be limitations on memory, battery life,
and processing power.
User Interface Constraints: Constraints related to the
user interface, such as the resolution and field of view of
the camera capturing sign language gestures, may impact
24
the design and performance of the system.
By addressing data integrity issues and considering
constraints during the design and development of the sign
language recognition system, you can ensure that it
delivers accurate, reliable, and efficient performance in
real-world applications.

3.3. Object Oriented Design

Creating an object-oriented design (OOD) for a sign
language recognition system using machine learning
involves several key components. The system should be
modular, reusable, and maintainable. Here’s a breakdown
of the design:
1. Core Classes and Their Responsibilities
A. SignLanguageRecognitionSystem

• Responsibilities:
1. Main entry point for the system.
2. Coordinates different components of the system.
3. Manages the lifecycle of the system (initialization,
training, prediction, evaluation).
• Attributes:
25
 1. DataManager dataManager
2. Preprocessor preprocessor
3. FeatureExtractor featureExtractor
4. Model model
5. Evaluator evaluator
• Methods:
1. initialize()
2. train(data: Data)
3. predict(input: InputData): Prediction
4. evaluate(testData: Data): EvaluationResult
B. DataManager

• Responsibilities:
1.Handles data loading, storage, and augmentation.
• Attributes:
1. dataSource
2. dataFormat
• Methods:
1. loadData(source: String): Data
2. saveData(data: Data, destination: String)
3. augmentData(data: Data): Data
C. Preprocessor

• Responsibilities:
26
 • Preprocesses raw data (e.g., image resizing,
normalization).
• Attributes:
• preprocessingSteps
• Methods:
• applyPreprocessing(rawData: Data):
PreprocessedData
D. FeatureExtractor

• Responsibilities:
• Extracts relevant features from preprocessed
data.
• Attributes:
• featureExtractionMethod
• Methods:
• extractFeatures(data: PreprocessedData):
FeatureSet
E. Model
• Responsibilities:
• Represents the machine learning model.
• Handles training and prediction tasks.

• Attributes:
27
 • modelType
• hyperparameters
• trainedModel
• Methods:
• train(features: FeatureSet, labels: Labels)
• predict(features: FeatureSet): Prediction
F. Evaluator

• Responsibilities:
• Evaluates the performance of the trained model.
• Attributes:
• evaluation Metrics
• Methods:
• evaluate(predictions: Predictions, groundTruth:
Labels): EvaluationResult
2. Class Interactions
• Initialization: The SignLanguageRecognitionSystem
initializes by creating instances of DataManager,
Preprocessor, FeatureExtractor, Model, and
Evaluator.
• Training:
• The system calls DataManager to load the
training data.
28
 • The raw data is then passed to Preprocessor to apply
necessary preprocessing steps.
• The Preprocessor output is fed into FeatureExtractor
to derive a set of features.
• These features, along with corresponding labels, are
used by Model to train the machine learning model.
• Prediction:
• For a given input, the system repeats
preprocessing and feature extraction steps.
• The extracted features are passed to Model to
predict the corresponding sign language gesture.
• Evaluation:
• The system uses Evaluator to compare model
predictions against ground truth labels and assess
performance.
+-------------------------------+
| 5. Implementation Considerations
• Extensibility: Ensure that each class can be extended
or modified without affecting others. For example,
different preprocessing techniques can be added by
extending the Preprocessor class.

29
 • Modularity: Each component should handle a
specific aspect of the process. This makes the system
easier to maintain and debug.
• Reusability: Components like FeatureExtractor and
Model should be reusable across different projects or
datasets with minimal modifications.
• Scalability: The design should accommodate scaling
up the data processing and model training, possibly
incorporating distributed computing techniques if
necessary.
By following this object-oriented design, the system
becomes robust, modular, and easy to maintain or extend,
facilitating efficient sign language recognition using
machine learning algorithms.

3.4. User Interface Design

Designing a user interface (UI) for a sign language
recognition system using machine learning involves
creating an intuitive and user-friendly experience that
caters to various user needs, including users who are deaf
or hard of hearing. Here’s a detailed outline of the UI
design:
30
1. User Personas
• Deaf or Hard of Hearing Individuals: Primary users
who rely on sign language for communication.
• Educators and Researchers: Users who might use the
system for educational purposes or research.
• General Public: Users interested in learning sign
language or curious about the technology.
2. Key UI Components
A. Home Screen
• Welcome Message: Brief introduction to the system
and its purpose.
• Navigation Menu: Links to different sections such as
Live Translation, Training Mode, Settings, Help, and
About.
• Quick Access Buttons:
• Start Live Translation
• Learn Sign Language
• Upload Video for Translation
B. Live Translation Screen
• Video Feed:

31
 • Real-time camera view for capturing sign
language gestures.
• Start/Stop button for the live translation.
• Option to upload a pre-recorded video.
• Translation Output:
• Text display area showing the recognized sign
language translation.
• Voice output toggle for converting text to
speech.
• Feedback Mechanism:
• Thumbs up/down or rating system to provide
feedback on translation accuracy.
• Option to report errors or provide suggestions.
C. Training Mode Screen
• Upload Section:
• Interface to upload training videos.
• Instructions on how to create high-quality
training data.
• Training Progress:
• Visualization of training progress and model
accuracy.

32
 • Logs and statistics showing the performance of
the model.
• Manual Correction:
• Interface for manually correcting misidentified
signs to improve the model.
D. Learn Sign Language Screen
• Learning Modules:
• Categorized lessons on different sign language
gestures.
• Video demonstrations with text descriptions.
• Practice Area:
• Interactive practice section where users can sign
and receive feedback.
• Quizzes and exercises to reinforce learning.
E. Settings Screen
• Profile Management:
• User profile with personal information and
preferences.
• Customization Options:
• Language preferences.
• Video quality settings.
• Notification settings.
33
 • Privacy Settings:
• Options to manage data privacy and sharing
permissions.
F. Help and Support Screen
• FAQs:
• Frequently Asked Questions to help users
troubleshoot common issues.
• User Guide:
• Detailed user guide explaining how to use each
feature.
• Contact Support:
• Form to contact technical support or provide
feedback.
3. User Flow
Launch Application: User opens the application and
lands on the Home Screen.
Start Translation:
• User selects "Start Live Translation" from the
Home Screen.
• System accesses the camera and displays the
Live Translation Screen.

34
 • User signs in front of the camera, and the system
translates the signs in real-time.
Feedback and Correction:
• User provides feedback on the translation
accuracy.
• If necessary, user can switch to Training Mode to
upload videos and improve the model.
Learning:
• User navigates to the Learn Sign Language
Screen to access lessons and practice.
Settings and Help:
• User configures preferences in the Settings
Screen.
• User accesses Help and Support for any issues or
additional information.
+5. Implementation Considerations
• Accessibility: Ensure the UI is accessible to all users,
including those with disabilities. Use appropriate
color contrasts, text sizes, and provide alternative text
for images.
• Localization: Support multiple languages to cater
to a diverse user base.
35
• Responsiveness: Design the UI to be responsive and
work seamlessly across different devices (desktop,
tablet, mobile).
• User Feedback: Continuously gather and incorporate
user feedback to improve the UI and overall user
experience

3.5. Test Cases

Unit Test Cases

Designing a user interface (UI) for a sign language

recognition system using machine learning involves
creating an intuitive and user-friendly experience that
caters to various user needs, including users who are
deaf or hard of hearing. Here’s a detailed outline of
the UI design:
1. User Personas
Deaf or Hard of Hearing
Individuals: Primary users who
rely on sign language for
communication.

36
Educators and Researchers: Users
who might use the system for
educational purposes or research.
General Public: Users interested
in learning sign language or
curious about the technology.
2. Key UI Components
A. Home Screen
Welcome Message: Brief
introduction to the system and its
purpose.
Navigation Menu: Links to
different sections such as Live
Translation, Training Mode,
Settings, Help, and About.
Quick Access Buttons:
Start Live Translation
Learn Sign Language
Upload Video for Translation
B. Live Translation Screen
Video Feed:
37
Real-time camera view for
capturing sign language gestures.
Start/Stop button for the live
translation.
Option to upload a pre-recorded
video.
Translation Output:
Text display area showing the
recognized sign language
translation.
Voice output toggle for
converting text to speech.
Feedback Mechanism:
Thumbs up/down or rating system
to provide feedback on translation
accuracy.
Option to report errors or provide
suggestio
C. Training Mode Screen
Upload Section:
Interface to upload training
videos.
38
Instructions on how to create
high-quality training data.
Training Progress:
Visualization of training progress
and model accuracy.
Logs and statistics showing the
performance of the model.
Manual Correction:
Interface for manually correcting
misidentified signs to improve the
model.
D. Learn Sign Language Screen
Learning Modules:
Categorized lessons on different
sign language gestures.
Video demonstrations with text
descriptions.
Practice Area:
Interactive practice section where
users can sign and receive
feedback.

39
Quizzes and exercises to reinforce
learning.
E. Settings Screen
Profile Management:
User profile with personal
information and preferences.
Customization Options:
Language preferences.
Video quality settings.
Notification settings.
Privacy Settings:
Options to manage data privacy
and sharing permission
F. Help and Support Screen
FAQs:
Frequently Asked Questions to
help users troubleshoot common
issues.
User Guide:
Detailed user guide explaining
how to use each feature.
40
Contact Support:
Form to contact technical support
or provide feedback.
3. User Flow
Launch Application: User opens
the application and lands on the
Home Screen.
Start Translation:
User selects "Start Live
Translation" from the Home
Screen.
System accesses the camera and
displays the Live Translation
Screen.
User signs in front of the camera,
and the system translates the signs
in real-time.
Feedback and Correction:
User provides feedback on the
translation accuracy.

41
If necessary, user can switch to
Training Mode to upload videos
and improve the model.
.
Settings and Help:
User configures preferences in the
Settings Screen.
User accesses Help and Support
for any issues or additional
information.
4. Wireframes and Mockups
Creating wireframes and mockups
helps visualize the UI design.
5. Implementation Considerations

Accessibility: Ensure the UI is

accessible to all users, including
those with disabilities. Use
appropriate color contrasts, text
sizes, and provide alternative text
for images.

42
 Localization: Support multiple
languages to cater to a diverse
user base.
Responsiveness: Design the UI to
be responsive and work
seamlessly across different
devices (desktop, tablet, mobile).
User Feedback: Continuously
gather and incorporate user
feedback to improve the UI and
overall user experience.

By following this detailed UI

design, the sign language
recognition system can provide an
effective, user-friendly interface
that meets the needs of its diverse
users.
System Test Cases
System test cases for a sign language recognition system
involve testing the entire application to ensure all
components work together correctly in real-world
scenarios. These tests are higher-level compared to unit
43
and integration tests and focus on end-to-end
functionality. Here’s a comprehensive list of system test
cases:

1. End-to-End Functionality Tests

A. Real-time Sign Language Recognition
test_real_time_translation_accuracy:
Description: Verify that the system accurately
translates real-time sign language gestures into text.
Steps:
1. Launch the application and navigate to the Live
Translation Screen.
2.Start the camera feed.
3.Perform a series of known sign language gestures.
4.Check the translated text output.
Expected Result: The text output matches the
performed gestures.
test_real_time_translation_speed:
Description: Verify the response time of the system
when translating gestures in real-time.
Steps:
1. Start the real-time translation.
2. Perform a sign language gesture.
44
 3. Measure the time taken for the text translation to
appear.
Expected Result: The translation appears within an
acceptable time frame (e.g., less than 1 second).
B. Video Upload and Translation
test_video_upload_translation:
Description: Verify that the system correctly
translates gestures from an uploaded video.
Steps:
1.Navigate to the Live Translation Screen.
2.Upload a pre-recorded video containing sign
language gestures.
3.Check the translated text output.
Expected Result: The text output accurately
represents the gestures in the video.
2. Accuracy and Performance Tests
A. Translation Accuracy
test_accuracy_with_varied_signers:
Description: Ensure the system accurately translates
gestures from different signers with varying styles.
Steps:

45
 1.Collect videos from multiple signers performing the
same set of gestures.
2.Run the system on these videos.
3.Compare the text output with the expected
translations.
Expected Result: The system provides accurate
translations for all signers.
B. Model Performance
test_model_performance_with_large_dataset:
Description: Verify that the system can handle a
large dataset efficiently.
Steps:
1.Load a large dataset for training.
2.Train the model using this dataset.
3.Measure the training time and evaluate the model
performance.
Expected Result: The system completes training
within a reasonable time and maintains high
accuracy.
3. User Experience Tests
A. Usability
test_usability_of_live_translation_interface:
46
 Description: Ensure the live translation interface is
user-friendly.
Steps:
1.Navigate to the Live Translation Screen.
2.Test all available features (e.g., start/stop
translation, upload video, feedback mechanism).
3.Collect feedback from users on ease of use.
Expected Result: Users find the interface intuitive
and easy to use.
test_feedback_mechanism:
Description: Verify that the feedback mechanism
works correctly.
Steps:
1.Perform a sign language gesture.
2.Provide feedback on the translation accuracy using
the thumbs up/down system.
3.Check that the feedback is recorded and processed.
Expected Result: Feedback is correctly recorded and
used for improving the system.
B. Accessibility
test_accessibility_features:
Description: Ensure the system is accessible to users
with different needs.

47
 Steps:
1.Test the application with screen readers.
2.Check color contrast and font sizes for readability.
3.Verify the presence of alternative text for images
and videos.
Expected Result: The system is fully accessible and
complies with accessibility standards.
4. Robustness and Error Handling
A. Handling Invalid Inputs
test_invalid_video_format_upload:
Description: Verify that the system handles invalid
video formats gracefully.
Steps:
1.Attempt to upload a video in an unsupported
format.
2.Observe the system’s response.
Expected Result: The system displays an
appropriate error message without crashing.
test_unexpected_gesture_handling:
Description: Ensure the system handles unexpected
gestures or noise gracefully.
Steps:

48
 1.Perform random, non-sign language movements in
front of the camera.
2.Observe the system’s response.
Expected Result: The system does not produce
incorrect translations and may prompt the user to
perform recognizable gestures.
5. Integration with Other Systems
A. Voice Output Integration
test_voice_output_toggle:
Description: Verify that the text-to-speech feature
works correctly.
Steps:
1.Perform a gesture and receive the text translation.
2.Enable the voice output toggle.
3.Check that the translated text is correctly converted
to speech.
Expected Result: The spoken output matches the
text translation.

These system test cases ensure that the sign language

recognition system works correctly in real-world
scenarios, providing accurate, efficient, and user-friendly
functionality.

49
 4. CODING

4.1. Data Base

50
 Fig: Dataset
The dataset consists of Sign Language Hand Gestures
containing 26 signs with different variations Each sample
consists of a person signing the corresponding letter while
facing directly at the camera

4.2. Data Collection

Collecting data for a sign language recognition system
using machine learning is a crucial step to ensure the
accuracy and reliability of the model. The data must be
51
comprehensive, diverse, and representative of the sign
language being modeled. Here's a detailed guide on how
to approach data collection for such a system:

1. Define Objectives and Scope

A. Objectives
• To collect a diverse dataset of sign language gestures.
• To ensure the data covers all necessary signs and
variations.
• To collect high-quality video data suitable for
machine learning.
B. Scope
• Determine the specific sign language (e.g., ASL,
BSL, etc.).
• Define the number of gestures to be collected.
• Identify any specific demographics or user groups to
include.
2. Sources of Data
A. Primary Data Collection
• Recording Sessions: Organize recording sessions
with native signers.

52
 • Participants: Include a diverse group of
participants of different ages, genders, and
ethnicities.
• Environment: Use a controlled environment
with good lighting and a plain background.
• Equipment: Use high-resolution cameras to
capture the gestures clearly from multiple angles
if possible.
B. Secondary Data Collection
• Public Datasets: Utilize existing sign language
datasets if available.
• Examples include:
• RWTH-PHOENIX-Weather 2014
• ASLLVD (American Sign Language
Lexicon Video Dataset)
• CSL (Chinese Sign Language) Dataset
• Online Videos: Collect data from educational videos,
sign language dictionaries, or YouTube channels with
proper licensing and permissions.

53
3. Data Collection Methodology
A. Recording Protocol
• Standardized Phrases: Prepare a list of gestures and
phrases to be recorded.
• Multiple Takes: Record each gesture multiple times
to capture variability.
• Annotation: Ensure each video is annotated with the
correct sign label.
B. Ethical Considerations
• Informed Consent: Obtain consent from all
participants, explaining the purpose and use of the
data.
• Privacy: Ensure participants' privacy is protected,
and data is anonymized if necessary.
4. Data Preprocessing
A. Annotation
• Label each video with the corresponding gesture.
• Use tools like VIA (VGG Image Annotator) or
custom annotation tools to mark the start and end of
gestures.

54
B. Cleaning
• Remove any corrupted or low-quality videos.
• Standardize video formats and resolutions.
C. Augmentation
• Geometric Transformations: Apply rotations,
translations, and flips to augment the dataset.
• Temporal Augmentation: Vary the speed of gesture
playback to simulate different signing speeds.
5. Data Storage and Management
A. Organization
• Store videos in a structured format (e.g., folder per
gesture).
• Maintain a metadata file containing details of each
video (e.g., signer, date, gesture label).
B. Backup
• Regularly backup the collected data to prevent loss.
• Use cloud storage solutions for scalability and
accessibility.

55
6. Quality Assurance
A. Validation
• Manually review a subset of the data for accuracy.
• Use automated scripts to check for inconsistencies in
annotations.
B. Consistency Checks
• Ensure that gestures are uniformly captured across
different sessions.
• Verify that the lighting and background remain
consistent within a session.
7. Example Workflow
Here’s a simplified example of the workflow for data
collection:

A. Preparation
Define the list of gestures to collect.
Recruit participants and schedule recording
sessions.
Set up recording equipment in a controlled
environment.

56
B. Recording Session
Explain the process to participants and obtain consent.
Record each gesture multiple times with breaks in
between.
Annotate the videos in real-time or immediately after
recording.
C. Post-Processing
Clean and preprocess the videos.
Apply data augmentation techniques.
Store the data in an organized structure with metadata.
D. Verification
Review a random sample of videos for quality assurance.
Run consistency checks on the annotations.
8. Tools and Technologies
A. Hardware
• High-resolution cameras (e.g., DSLRs, webcams).
• Tripods and lighting equipment.
B. Software
• Annotation tools: VIA, Labelbox, RectLabel.

57
 • Video processing libraries: OpenCV, FFmpeg.
• Cloud storage: AWS S3, Google Cloud Storage.
9. Challenges and Considerations
A. Diversity
• Ensure the dataset includes diverse signers to make
the model robust.
B. Quality Control
• Continuously monitor the quality of data being
collected.
C. Data Volume
• Collect enough data to cover all variations and
contexts of the gestures.
10. Future Enhancements
• Crowdsourcing: Consider using platforms like
Amazon Mechanical Turk for annotating larger
datasets.
• Collaboration: Partner with institutions and
organizations specializing in sign language for
broader data collection.

58
 • Real-time Data Collection: Implement mechanisms
for users to contribute data via mobile applications or
online platforms.

By following these guidelines, you can systematically

collect high-quality data for training a robust and accurate
sign language recognition system using machine learning
algorithms.

4.3. Complete Project Coding

import numpy as np

import keras

from keras.models import load model

from spellchecker import Spellchecker import pyttsx3

#One time initialization

engine = pyttsx3.init()

# Set properties before you add things to say

59
engine.setProperty('rate', 125) #Speed percent (can go
over 100)

engine.setProperty('volume', 1) # Volume 0-1

def nothing(x):

pass

def get_class_label (val, dictionary):

"""

Function returns the key (Letter: a/b/c/...) value from

the alphabet dictionary

based on its class index (1/2/3/...)

"""

for key, value in dictionary.items():

if value == val:

return key

60
model = load_model('model.h5')

spell = SpellChecker ()

# create alphabet dictionary to label the letters ('a':1, ...,

'nothing':29)

alphabet = (chr(i+96) .upper():i for i in range (1,27)}

alphabet ['del'] = 27

alphabet ['nothing'] = 28

alphabet ['space'] = 29

video_capture = cv2.VideoCapture (0)

cv2.namedWindow("Model Image',
cv2.WINDOW_NORMAL)

# set the ration of main video screen

61
video_capture.set (cv2.CAP_PROP_FRAME_WIDTH,
640)

video_capture.set (cv2.CAP_PROP_FRAME_HEIGHT,
480)

# set track bar of threshold values for Canny edge

detection

# more on Canny edge detection here:

#https://opencv-python-
tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/
py_canny/py_canny.html

cv2.createTrackbar ('lower_threshold', 'Model Image', 0,

255, nothing)

cv2.createTrackbar ('upper_threshold', 'Model Image', 0,

255, nothing)

cv2.setTrackbar Pos ('lower_threshold', 'Model Image',

100)

cv2.setTrackbar Pos ('upper_threshold', 'Model Image', 0)

62
# VARIABLES INITIALIZATION

# THRESHOLD - ratio of the same letter in the last

N_FRAMES predicted letters

THRESHOLD = 0.85

N_FRAMES= 10

IMG SIZE 100

SENTENCE = '' # string that will hold the final output

letter =' '#temporary letter

LETTERS = np.array([], dtype='object') # array with

predicted letters

START = False # start/pause controller

bot = False # start/pause controller

#supportive text

63
description_text_1 = "Press 's' for 'CHAR' / 'W' for
'WORD' (START/PAUSE)"

description_text_2 = "Press 'D' to erase the output. "

description_text_3 = "Press 'Q' to quit."

def speak out (text_input):

engine.say (text input)

# Flush the say () queue and play the audio

engine.runAndWait()

while True:

blank_image = np.ones ((100,800,3), np. uint8)

ret, frame = video_capture.read() # capture frame-by-

frame

# set the corners for the square to initialize the model

picture frame

x_0 = int (frame.shape[1] * 0.1)

64
 y_0= int (frame. shape [0] * 0.25)

x_1 = int (x_0200)

y_1 = int (y_0 + 200)

# MODEL IMAGE INITIALIZATION

hand frame.copy() [y_0:y_1, x_0:x_1] # crop model

image

gray = cv2.cvtColor (hand, cv2.COLOR_BGR2GRAY)

# convert to grayscale

#noise reduction

blured= cv2.GaussianBlur (gray, (5, 5), 0)

blured cv2.erode (blured, None, iterations=2)

blured= cv2.dilate (blured, None, iterations=2)

#get the values from tack bar

lower = cv2.getTrackbar Pos ('lower_threshold', 'Model

Image')

65
 upper = cv2.getTrackbar Pos ('upper_threshold', 'Model
Image')

edged = cv2.Canny (blured, lower, upper) # aplly edge

detector

model _image = -edged # invert colors

model_image = cv2.resize(

model_image,

dsize=(IMG_SIZE, IMG_SIZE),

interpolation-cv2.INTER_CUBIC

model_image = np.array (model_image)

model_image = model_image.astype ('float32') / 255.0

try:

model_image = model_image.reshape(1, IMG_SIZE,

IMG_SIZE, 1)
66
 predict = model.predict (model_image)

for values in predict:

if np.all (values < 0.5):

#if probability of each class is less than 5 return

a message

letter = 'Cannot classify: ('

else:

predict = np.argmax (predict, axis=1) + 1

letter = get_class_label (predict, alphabet)

LETTERS = np.append(LETTERS, letter)

#print (letter)

except:

pass

if START == True:
67
 # append the final output with the letter

if (np.mean (LETTERS [-N_FRAMES:]== letter) >=

THRESHOLD) & (len (LETTERS) >= N_FRAMES):

if letter = 'space':

SENTENCE = SENTENCE [:-1] + ' ' + '_'

LETTERS = np.array([], dtype='object')

elif letter == 'del':

SENTENCE = SENTENCE [:-2]+' '

LETTERS = np.array([], dtype='object')

elif letter == 'nothing':

pass

else:

SENTENCE = SENTENCE [-1]+ letter +

LETTERS = np.array([], dtype='object')

# apply spell checker after double space

68
if len (SENTENCE) > 2:

if SENTENCE [-3:] == ' _':

SENTENCE = SENTENCE.split(' ')

word = SENTENCE [-3]

corrected_word = spell.correction (word)

SENTENCE [-3] = corrected_word. upper()

SENTENCE = ' ' .join (SENTENCE [:-2]) + '_'

engine.say (corrected_word)

# Flush the say () queue and play the audio

engine.runAndWait()

elif bot == True:

# append the final output with the letter

if (np.mean (LETTERS [-N_FRAMES:] == letter)

>= THRESHOLD) & (len (LETTERS) >= N_FRAMES):
69
#ADD for all the letter

if letter == 'A':

speak out ("I am hungry")

elif letter == 'B':

speak out ("I need help")

elif letter == 'C':

speak out ("How are you?")

elif letter == 'D':

speak out ("I am fine")

elif letter == 'B':

speak out ("Good Morning")

elif letter == 'F':

speak out ("Good Afternoon")

elif letter == 'G':

speak_out ("Good Evening")

70
elif letter == 'H':

speak out ("Good Night")

elif letter == 'I':

speak out ("I believe in you")

elif letter == 'J':

speak out ("Had Lunch")

elif letter == 'K':

speak_out ("How was the day")

elif letter == 'L':

speak out ("What are you doing")

elif letter == 'M':

speak out ("I am waiting for my friend")

elif letter: == 'N':

speak out ("I am sorry")

elif letter == '0':

71
 speak_out ("Congratulations")

elif letter == 'p':

speak_out ("Excuse me")

elif letter == 'Q':

speak_out ("Excellent")

elif letter == 'R':

speak out ("Have a good day")

speak out ("Nice to meet you")

#helper texts

elif letter == 'U':

cv2.putText(

speak out ("I feel that")

img=frame,

elif letter == 'V':

text=description_text_1,

speak out ("I understand your problem")

org=(10,440)

72
 elif letter == 'W':
fontFace=cv2.FONT_HERSHEY_PLAIN,

speak out ("All right")

color=(0,0,225),

elif letter == 'X':

fontscale=1

speak out
("Wrong") )

elif letter == 'Y':

speak out ("Hi")

cv2.putText(

elif letter == '2':

img=frame,

speak out ("Hello")

text=description_text_2,

elif letter 'nothing':

org=(10,455),

pass
fontFace=cv2.FONT_HERSHEY_PLAIN,
73
color=(0,0,225),

fontscale=1

if START == False and bot

False: )

paused text 'Waiting!'

elif START == True and bot == False:

cv2.putText(

paused text = 'Char Mode

img=frame,

elif START== False and bot == True:

test=description_text_3,

paused text = 'Word Mode

org=(10,470),

fontFace=cv2.FONT_HERSHEY_COMPLEX_SMALL,

color=(255,255,255),
74
 #TEXT INITIALIZATION
fontscale=1

#paaused text )

cv2.putText(

img=frame,
# current letter

text=paused_text,
cv2.putText(

org=(x_0+110,y_0+195),
img=frame,

font Face=cv2.FONT HERSHEY PLAIN,

text=letter,

color=(0,0,255),
org=(x_0-30,y_0-10),

fontScale=1
fontFace=cv2.FONT_HERSHEY_COMPLEX_SMALL,

)
color=(255,255,255),

75
fontscale=1

#helper texts

cv2.putText( #cu
rrent letter

img=frame,
cv2.putText(

text=description_text_1,
img=frame,

org=(10,440),
text=letter,

font Face=cv2.FONT HERSHEY PLAIN,

org=(x_0+10,y_0+20)

fontScale=1
fontFace=cv2.FONT_HERSHEY_PLAIN,

)
color=(0,0,255),
76
fontscale=1

cv2.putText( )

img=frame,

text-description_text_2,

# final output

cv2.putText(

img=blank_image,

text='Result:+SENTENCE,

org (10, 50),

fontFace=cv2.FONT HERSHEY
COMPLEX_SMALL,

thickness=1,

77
 color=(0,0,255),

fontScale=1

#print (SENTENCE)

# draw rectangle for hand placement

cv2.rectangle (frame, (x_0, y_0), (x_1, y_1), (0, 255,

0), 2)

# display the resulting frames

cv2.imshow('Main Camera', frame)

cv2.imshow('Edge Output', edged)

cv2.imshow('Output', blank_image)

if cv2.waitKey (30) & 0xFF== ord('w') :

bot = not bot

78
 if cv2.waitKey(30) & 0xFF == ord('d'):

SENTENCE = ''

if cv2.waitKey (30) & 0xFF== ord('s'):

START= not START

if cv2.waitKey (30) & 0xFF == ord ('q'):

break

# When everything is done, release the capture

video_capture.release()

cv2.destroyAllWindows ()

# save the resulted string into the file

text_file = open("Output.txt", "w")

text_file.write("You said: %s" % SENTENCE)

79
text_file.close()

80
81
82
83
84
85
86
87
88
89
90
 5. STANDARDIZATION OF THE CODE
PACKAGES:
➢ NumPy:
What is NumPy, exactly?
• NumPy is a Python package containing a strong n-
dimensional array object. NumPy is a Python package. •
It's used for array computing on the market.
• NumPy strives to be 50 times faster than regular Python
lists when it comes to array objects.
▪ The following are some of NumPy’s features:
▪ Advanced (broadcasting) capabilities
▪ C/C++ and Fortran code integration tools
▪ An effective N-dimensional array object
▪ Linear algebra, Fourier transform, and random numbers
awareness
▪ NumPy's Applications o NumPy includes a high-

91
performance multidimensional array as well as basic
computation and manipulation methods.
o It's a Python replacement for lists and arrays that
consumes very little memory.
o We are capable of performing a wide range of
mathematical operations.
o Shape Manipulation o It's compatible with Pandas,
SciPy, Tkinter, and other Python libraries.

➢ Open CV:
o What is Open CV?
OpenCV is a cross-platform library for building computer
vision applications in real time. It focuses on image
processing, video grabing and analytics, including the
detection of faces and objects. OpenCV was developed to
provide common infrastructure for computer vision
applications and accelerate the use of machine perception
in commercial products. OpenCV, a BSD-licensed
product, makes it easier for companies to use and modify
the code. The library has more than 2500 optimised
algorithms, including a full range of both classic and
advanced computer vision and master learning
algorithms. These Sign Language Recognition Using

92
Machine Learning Algorithm Dept of CSE 35 2020-21
algorithms can be used for identification of faces, objects
identification, classification of human activities into
videos, camera tracking, moving objects and 3D object
models deleted. Their images can be stitched to produce
highly-intensive 3D point clouds. Color segmentation or
colour filtration is commonly used for identifying certain
objects/regions of a certain colour in OpenCV. The most
common colour area is RGB. It is called an additive
colour space since the three colour shades add to the
image.

o Features of Open CV:

• Write and read images
• Take photos and videos
• Work with photographs (filter and transform)
• Analyse the video or images
Standard neural networks have an architecture different
from the Convolutionary neural networks. Regular
Networks (RNNs) restructure data through a hidden layer
sequence. Every layer consists of a range of neurons and
every layer is linked to one or all neurons in the preceding
layer. Finally, a fully connect layer and an output layer
represent the forecasts. CNNs are a bit different from
93
others. CNNs are a little different. The layers are arranged
in three dimensions first and foremost: width, height and
depth. In addition, the neurons in a single layer are not
connected in the next layer to all of the neurons, but only
a few of them. Finally, a single vector of shift values
along the axis will reduce the final output. Square
Convolutionary Neural Networks measure an entirely
different small amount. 1st of all, the three-dimensional
square layers: width, height and depth. In addition,
neurons in one layer do not hook up to only a small low
area with all neurons within the next layer. Finally, the
final output is reduced to an organised, depth-dimensional
vector of changing scores • Perform feature extraction

o Applications of Open CV:

The computer vision is used widely in different domains
such as
• Robotics
• Medical
• Security
• Automation

➢ Keras
• What is keras?
94
 ▪ Keras is a lightweight python library that makes
building deep learning models based on Tensorflow
simple. Keras was created with the aim of rapidly define
deep learning models.
▪ Keras contains numerous implements on common
neural network compilation layers, targets, activation
features, optimizers and a host of tools to simplify the
coding needed to create deep neural network code,
making working with image and text data easier. A great
approach for deep learning is to create a convolutions
neural network to classify images (CNN). The Python
Keras library makes the construction of a CNN quite
simple.

Features of keras:
▪ Minimal Structure
▪ Consistent, simple
▪ High Scalability and computation
▪ User Friendly
▪ It supports on multiple platforms

2 FUNCTIONS GAUSSIAN BLUR:

The picture is convolved with a Gaussian filter instead of

the box filter in the Gaussian Blur process. The Gaussian
95
filter is a low-pass filter that eliminates high-frequency
components from audio signals
. • The Gaussianblur() method is used to perform this
operation on an image.
• The GaussianBlur() method of OpenCV can be used to
blur an image.
• GaussianBlur() is a function that uses the Gaussian
kernel. The kernel's height and width should both be
positive and odd numbers. Then you must define the X
and Y directions, which are denoted by the letters sigmaX
and sigmaY, respectively. Both are considered the same if
only one is mentioned. Syntax: blur_image =
cv2.GaussianBlur(img, (7,7), 0)
➢ CONVERTING INTO GRAYSCALE: Gray scaling is
the method of converting an image from other colour
spaces to shades of grey, such as RGB, CMYK, HSV, and
so on. It can be completly black and complete white.
Importance of Gray scaling:
● Reduced Dimensions
● Reduces Model Complexity This function takes the
original image as its first input. It receives the colour
space c conversion code as a second input. We use the
COLOR BGR2GRAY code to transform our original
image from BGR to grey. Gray = cv2.cvtColor (image,
96
cv2.COLOR_BGR2GRAY

6. TESTING
OVERVIEW OF TECHNOLOGY:
CONVULATIONAL NEURAL NETWORK (CNN):
A CNN is a deep-learn algorithmic programme that can
absorb the input image, allocate weights and biases in
different aspects/objects in the image, and distinguish
between them. The CNN is an algorithmic programme of
deep learning. The pre-processing requirement of a
ConvNet is significantly less than other classification
algorithms. While filters are traditionally manufactured
by hand, ConvNets can learn these features with sufficient
training. The visual field design impressed the ConvNet
design, which is similar to the pattern of Neurons in the
Human Brain. Only a small area of the visual field called
the receptive field is used to respond to stimuli by
individual neurons. A group of such fields overlap the
whole cortical region. By using relevant filters, an image
of abstraction and temporal dependencies can be captured
successfully in a ConvNet. The design is more robustly
aligned with the image data set because of the reduction
in the number of parameters involved and the reusability
97
of the weights. In other words, the Network can be
equipped for a higher level of image advancement.
Standard neural networks have an architecture different
from the Convolutionary neural networks. Regular
Networks (RNNs) restructure data through a hidden layer
sequence. Every layer consists of a range of neurons and
every layer is linked to one or all neurons in the preceding
layer. Finally, a fully connect layer and an output layer
represent the forecasts. CNNs are a bit different from
others. CNNs are a little different. The layers are arranged
in three dimensions first and foremost: width, height and
depth. In addition, the neurons in a single layer are not
connected in the next layer to all of the neurons, but only
a few of them. Finally, a single vector of shift values
along the axis will reduce the final output. Square
Convolutionary Neural Networks measure an entirely
different small amount. 1st of all, the three-dimensional
square layers: width, height and depth. In addition,
neurons in one layer do not hook up to only a small low
area with all neurons within the next layer. Finally, the
final output is reduced to an organised, depth-dimensional
vector of changing scores.

98
 7.SYSTEM SECURITY MEASURES

Provided an endless number of resources and

time, every device is feasible. Unfortunately, in
the real world, this condition does not exist. As
a result, it's both important and wise to assess
the system's viability as soon as possible. If an
unwell developed structure is identified early
within the description part, months or years of
effort, thousands of rupees, and a great deal of
professional embarrassment can be avoided. In
several ways, practicability and risk analysis are
related. In cases of a low project risk, a high-
quality package can be produced feasibly. In
this situation, there are three key issues of
concern.
FEASIBILITY STUDY:
When given limitless resources and endless
time, all systems are feasible. Unfortunately, in
the real world, this situation does not exist. As a

99
result, assessing the system's viability as soon as
possible is both appropriate and wise. If the
early definition of an impermeable system,
months or years of work are identified,
thousands of rupees and unspeakable
professional humiliations can be avoided. In
several cases, feasibility and risk analysis are
related. When project risk is high, the likelihood
of delivering high-quality applications is
lowered. There are three main fields of concern
in this situation.
1.Performance Analysis The project is run with
the help of a safe networking setup for the entire
practicality of the project work. The operating
system is typically Windows. The main goal of
this initiative is to help deaf and dumb people
communicate with others. The aim of
performance analysis is to retrieve information
in a safe and reliable manner. It is important that
the performance measurement and definition
100
methods be carried out in parallel.
2. Technical Analysis The technical
requirements of the company Simply stated, this
check of practicability determines whether the
scheme will succeed until designed and
implemented, as well as whether there are any
significant obstacles to implementation. There
are some things to consider in relation to these
issues in technical analysis:
• Changes to the method to be implemented: All
improvements should be in a constructive
direction, resulting in a higher degree of quality
and customer support.
• Required abilities: This project's platforms and
technologies are commonly used. As a result,
the professional workforce is immediately
available inside the business.
• Acceptability: The system's configuration is
unbroken enough that there shouldn't be any
problems.
101
3. Economical Analysis Economical analysis is
used to assess the importance of growth in
comparison to the ultimate financial gain or
benefits resulting from the developed
environment. We don't want to use
highperformance servers to operate this
technique. Package modules are used to enforce
all of the features. We don't seem to be using
any physical equipment for affiliation during
this system. That the scheme is viable from a
financial standpoint.

DEVELOPMENT METHODS :
The development of software is usually a
process step by step. It involves the software
design process before the implementation
process. A software design is a description of
the component structure, the data that is part of
the system, the interfaces between the
components and the algorithms used sometimes.

102
Designers do not immediately achieve a finished
design but iteratively develop the design in a
number of different versions. The design
process involves adding formalities and details
in the development of the design with constant
retrofitting to correct previous designs. Software
design is an ad hoc process in many software
development projects. An informal design is
prepared based on the set of requirements,
usually in natural language. The coding starts
and when the system is implemented the design
stage will be modified. When the stage of
implementation is complete, the design usually
has changed from the initial specification to an
incorrect and incomplete description of the
system by the original design document. The
design phase offers various advantages. Some of
the following are listed:
• The design stage helps user needs understand
and helps user demands to be mapped into the
103
implementation stage.
• Design iterations help to incorporate in the
final software as many user requirements as
possible. Different design options need to be
considered at each stage during the design
process. Design phase aims at producing the
software's overall design.

8.REPORTS

8.1. Screenshots of the Result

104
Fig: output(i) In this hand gesture showing a sign ‘A’ so
its displaying letter ‘B’ in the box as it is in char mode

fig: In this, sign is showing ‘U’ so it is displaying letter

105
‘U’ in the box as it is in char mode

Fig: In this, sign is showing letter ‘A’ as it is in word

mode the speech output will be ‘I am hungry

106
Fig: In this, sign is showing letter ‘B’ as it is in word
mode the speech output will be ‘I need help’

Fig: In this, sign is showing letter ‘L’ as it is in word

mode the speech output will be ‘What are you doing?’

9. FUTURE SCOPE AND FURTHER

ENHANCEMENT OF THE SCOPE
Only static ISL numeral signs can benefit from the

107
system. The ISL recognizer system cannot be regarded as
a complete system since we need to include ISL
alphabets, words, and sentences to fully understand sign
language. These signals may be used in the future.
Additional extraction algorithms, such as transform
Wavelet, invariant moments, Shape lets descriptors and
other existing methods, can also be used to enhance the
results of the experiments. Other classifications can be
used in experiments for increasing recognition rates, such
as Multi-class Support Vector Machine (SVM), Principal
Component Analysis (PCA) or Linear Discriminant
Analysis (LDA), or a combination of them.
✓ Use Dynamic Loading for the Dataset: Our original
dataset was very big and would require a server with a lot
of RAM and disc space to run. Splitting the file names
into preparation, Sign Language Recognition Using
Machine Learning Algorithm Dept of CSE 42 2020-21
validation, and test sets and dynamically loading images
in the Dataset class might be one solution.
✓ We might train the model on more samples in the
dataset if we used such a loading technique.
✓ Sign language recognition is a diverse field of study.
✓ More methods for shading detection, image separation,

108
and grouping can also be used. We plan to focus on more
approaches in the future, as well as build a framework for
continuous sign language recognition for more terms and
sentences.
✓ We used tools learned in computer vision and machine
learning to implement an automated sign language gesture
recognition system in real-time in this project.
✓ We discovered that often simple solutions are better
than complicated solutions. Despite using a sophisticated
segmentation algorithm, the most effective skin masks
were extracted using a simple skin segmentation model.
We also recognised the time constraints and challenges of
building a dataset from the ground up.
✓ Other symptoms, such as the Indian Sign Language, as
currently available for American Sign Language, should
be added to our model.
✓ Further training of the neuro-network to recognise
symbols that effectively involve two hands in the model's
recognition of common terms and expressions. Using
linear classifiers to improve the performance.

109
 10. BIBLIOGRAPHY
1. Hand Gesture Recognition for Indian Sign Language
using Skin Color Detection and Correlation-Coefficient
algorithm with Neuro-Fuzzy Approach.
110
2. Jeegar Trivedi.
3. Real-time Sign Language Recognition based on Neural
Network Architecture.
4. Kusumika Krori Dutta, Sunny Arokia Swamy Bellary
5. Sign Language Learning System with Image Sampling
and Convolutional Neural.
6. Network, Yangho Ji, Sunmok Kim, and Ki-Baek Le.
7. Sign Language Recognition Using Deep Learning on
Custom Processed Static Gesture
8. Aditya Das, Shantanu Gawde, Khyati Suratwala and
Dr. Dhananjay Kalbande.
9. Hand Gesture Detection based Real-time American
Sign Language Letters Recognition.
10. Support Vector Machine, Xinyun Jiang, Wasim
Ahmad.
11. End to end Residual Neural Network with Data
Augmentation for sign Language.
12. Recognition, Mengyi Xie, Xin Ma
13. Ian Lim, Joshua Lu, Claudine Ng, Thomas Ong and
Clement Ong.
14. “Sign language Recognition through Gesture &
Movement Analysis” .
15. Two hand Indian Sign language dataset for
benchmarking classification models of machine Learning,
111
Leela surya teja manga muri, Lakshay jain, Abhishek.

112
113