10 VI June 2022

https://doi.org/10.22214/ijraset.2022.44296
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

Gesture Vocalizer Using Flex Sensor and Software

Visualization
Katikaneni Nitish Kumar1, Thota Guna Prashant2, Pendli Rishith Reddy3, Mrs. B. Priyanka4
1, 2, 3
B.E. Scholars, Dept. of Electronics and Communications Engineering, Sreenidhi Institute of Science and Technology,
Hyderabad, Telangana
4
Associate Professor, Dept. of Electronics and Communications Engineering, Sreenidhi Institute of Science and Technology,
Hyderabad, Telangana

Abstract: Hardware version of the project: Gesture Vocalizer is a multi-microcontroller-based framework intended to work with
the correspondence among the imbecilic, hard of hearing and visually impaired networks and their correspondence with the
ordinary individuals. This system can be flexibly changed to function as a "smart device.". In this project we showcased a
gesture vocalizer based on a microcontroller and sensors. Gesture vocalizer design is basically a hand glove and a
microcontroller system. A hand glove can detect practically all hand gestures, and a microcontroller-based system can translate a
few of those gestures into identifiable speech. The hand glove has two types of sensors: flex sensors that act as bend sensors and
accelerometers that act as tilt sensors. This device is advantageous to deaf people because it allows their hands to speak once
they have worn the gesture vocalizer glove. In this project we use flex sensors and Arduino Mega microcontroller which are the
backbone of the project. The flex sensor otherwise called the twist sensor is a sort of sensor that measures the measure of bowing
or avoidance or flexing as the name proposes. As the sensors are bowed the sensors give us an obstruction esteem, the sensor
gives so the more the sensors are twisted, more opposition esteem. When estimated with a multimeter, the flex sensor at a level
and consistent position gives an opposition esteem close to 25k ohm while when the sensors are completely bowed, they give an
obstruction worth of 72k ohm. The flex sensors are typically of a slim construction and are entirely agreeable to utilize, and can
be twisted without utilizing a lot of power.
Software version of the project: We describe a revolutionary real-time approach for hand gesture identification in this paper.
The background subtraction method is used in our framework to remove the hand region from the background. The palm and
fingers are then subdivided so that the fingers may be detected and recognised. Finally, to predict hand gesture labels, a rule
classifier is used. Experiments on a 1300 image data set reveal that our solution works well and is quite efficient. Furthermore,
on another data set of hand movements, our method outperforms a state-of-the- art method. The background subtraction
approach is used to detect the hand region from the backdrop. The palm and fingers are segmented after that. The fingers in the
hand image are discovered and recognised based on the segmentation. Hand gesture recognition is performed using a simple
rule classifier. On a data set of 1300 hand photographs, the performance of our technique is tested. The results of the
experiments suggest that our method works effectively and is suitable for real-time applications. Furthermore, on an image
collection of hand movements, the suggested technique beats the state-of-the-art FEMD. The proposed method's performance is
significantly dependent on the results of hand detection. If there are moving objects with a color that is similar to that of the
skin, the items are detected as a result of the hand detection and reduce the hand gesture recognition performance. Machine
learning algorithms, on the other hand, can distinguish the hand from the backdrop. ToF cameras provide depth information
that can help enhance hand detection performance. To address the complicated background problem and improve the robustness
of hand identification, ML algorithms and TOF cameras may be applied in future research.

I. INTRODUCTION
A. Deaf-Dumb Hand Gestures
There are 22 official languages in India, as well as 415 living languages.When it comes to communicating between villages, groups,
and states, such variation in languages poses a barrier. The Deaf and Mute communities in India utilize Indian Sign Language (ISL)
as one of their living languages. The total percentage of population of India that is handicapped with hearing cumulates to nearly 1.3
million people out of the total 21.9 million people with disabilities. Previously, India's educational system was based on the oral-
aural approach.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2294
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

As more Indian Sign Language is employed, the situation is improving. In India, there are 22 official languages and 415 additional
languages that are still spoken. When it comes to communicating between villages, groups, and states, such variation in languages
poses a barrier. Indian Sign Language (ISL) is a live language in India that is spoken by Deaf and Mute people. Previously, the
Indian educational system was based on the oral-aural approach. The situation is improving as more Indian Sign Language is used
Sign languages are languages that use the visual-manual modality to convey the meaning or messages. This language is expressed
by a combination of manual signs and non- manual features. Sign languages are distinct natural languages with their own syntax and
lexicon. This indicates that sign languages are not universal or mutually comprehensible, despite the fact that there are obvious
parallels between them. It's uncertain how many sign languages are now in use around the world. Each country has its own native
sign language, and some countries have many sign languages. The Ethnologue 2021 edition counts 150 sign languages, but the
SIGN-HUB Atlas of Sign Language Structures lists over 200, with the caveat that there are likely to be more that have yet to be
recorded or discovered. Some sign languages have received legal recognition in some manner. Natural sign languages are
distinguished from other systems that are antecedents to or derived from them, such as devised manual codes for spoken languages,
home sign, and "baby sign," according to linguists, and signs learned by non-human primates. There are striking similarities among
the signs in the sign language such as:
1) Direction of the Sign: When the same sign is interpreted in multiple ways, it might have completely distinct meanings. When
the sign "give" is directed toward the one signing, it indicates "give to me," but when it is directed away from the one signing, it
indicates "give to you.".
2) Hand positioning: The palm of your hand should face the person(s) you're speaking with. Maintain a comfortable position for
your arm. Your hand should not be in the way of your mouth.
3) Flow and Rhythm: With a little break between words, each letter should be signed clearly, distinctly, and crisply. A pause is
indicated by holding the last letter of the word for several seconds.
4) Vocalization: You should speak the words you finger spell at the same moment. The letters themselves should not be uttered
aloud.
5) Double letters in a word: Between the first and second letters, the hand is slightly opened.
6) Understanding Signs: It's not uncommon for novice or inexperienced signers to have trouble comprehending what others are
saying to them. Deaf or speech-impaired people who use sign language will be patient with you if you don't understand them

Sign Language is divided into three groups such as: -

 Everyday Terms: This class incorporates terms that are utilized in regular correspondence like Sad, Happy, Good, How Are
You, Sleepy, Angry, etc.
 Basic terms: This classification incorporates the terms that are utilized for our fundamental necessities like Alphabets (A, B, C,
...), Numbers (1, 2, 3, ...), Days (Sunday, Monday, ...), Months (January, February, ...), Etc.
 Emergency terms: This class incorporates the terms that are utilized for a critical need or to make mindful of certain perils like
Thirsty, Emergency, Discharge, Call Ambulance, Etc.

B. Gesture Vocalizer
People have the voice capacity for cooperation and correspondence among one another. Lamentably, not every person has the
capacity of talking and hearing. Communication via gestures utilized among the local area of individuals who can't talk or hear as
the methods for correspondence. Communication through signing is a motion portrayal that includes at the same time joining hand
shapes, direction and development of the hands, arms or body, and looks to communicate fluidly with a speaker's musings.
Individuals who can't talk utilizes the communications via gestures to speak with other individual vocally weakened individual and
even with other typical individuals who knows the implications of gesture based communications or a translator is expected to
decipher the implications of gesture based communications to others who can talk and don't have the foggiest idea about the
implications of gesture based communications.
The communication gap between -special person and a normal person is one of the main obstacles that this unique person faces.
Deaf and dumb persons have a hard time communicating with normal people. This enormous challenge makes them uneasy, and
they believe they are being discriminated against in society. It's anything but consistently workable for a person to associate with
constantly to decipher the gesture-based communications and not every person can get familiar with the gesture-based
communications.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2295
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

On account of a breakdown in correspondence Deaf and moronic individuals accept they can't impart, and therefore, they can't pass
on their feelings. In this manner, another option is that we can utilize a PC or an advanced mobile phone as an arbiter. The HGRVC
(Hand Gesture Recognition and Voice Conversion) innovation finds and tracks the hand movements of those who are hard of
hearing and unable to speak so they can speak with others. The PC or a PDA could take a contribution from the vocally disabled
individual and give its text based just as and sound type of yield.

II. BACKGROUND
1) “Real-Time Hand Gesture Recognition Using Finger Segmentation” paper written by Zhi- hua Chen, Jung-Tae Kim, Jianning
Liang, Jing Zhang, and Yu-Bo Yuan has mentioned that for human-computer interaction, hand gesture recognition is critical.
We describe a revolutionary real-time approach for hand gesture identification in this paper. The hand locale is separated from
the setting in our system utilizing the foundation deduction technique. The palm and fingers are then partitioned so the fingers
might be identified and perceived. Finally, to predict hand gesture labels, a rule classifier is used. Experiments on a 1300 image
data set reveal that our solution works well and is quite efficient. Furthermore, on another data set of hand movements, our
method outperforms a state-of- the-art method.
2) “Gesture Based Vocalizer for Deaf and Dumb” paper written by Supriya Shevate , Nikita Chorage, Siddhee Walunj, Moresh M.
Mukhedkar have pointed out that gesture vocalizer is a socially conscious project. According to the results of the survey, deaf
persons find it extremely difficult to communicate with others. Hard of hearing people typically speak with hand signals,
making it hard for others to appreciate their communication through signing. We intend to execute a motion-based vocalizer
that will perceive all hard of hearing individuals' motions, make an interpretation of them to discourse, and show them on a
LCD screen. To do as such, we're utilizing an ARM 7 regulator to associate the entirety of the sensors and the discourse
synthesizer. The information glove comprises two kinds of sensors: a flex sensor and an accelerometer that fills in as a slant
sensor. Remote information gloves, which are standard material driving gloves with flex sensors along the length of each finger
and the thumb, are utilized. Dumb persons can wear the gloves to make hand gestures, which are then translated into speech so
that others may understand them. This technique allows persons who are deaf to communicate with others.
3) “Hand Gesture Recognition and Voice Conversion for Deaf and Dumb” paper written by Cheshta has told that Communication
is the primary means by which people communicate with one another. Birth defects, accidents, and oral disorders have all
contributed to a rise in the number of deaf and dumb people in recent years. Because deaf and dumb persons are unable to speak
with normal people, they must rely on visual communication. People sometimes misinterpret these communications, whether
through sign language, lip reading, or lip sync. This project is designed to enable these persons with special needs participate in
society on an equal footing.
4) “Orientation Histograms for Hand Gesture Recognition” paper written by William T. Freeman and Michal Roth were
informative and expressed that they offer a method for recognizing hand gestures that is based on McConnell's” pattern
recognition methodology that uses histograms of local orientation. For gesture categorization and interpolation, we employ the
orientation histogram as a feature vector. This approach is simple and quick to compute, and it provides some resistance to
fluctuations in scene light. We created a real- time version that can discriminate between a small set of hand movements
(around ten). The entire computation takes place on a workstation, with separate hardware only being utilized to digitize the
image. A user can use hand gestures to control a computer graphic crane or play a game. We go over the method's drawbacks.
The histogram of the spatio- temporal gradients of picture intensity forms the comparable feature vector for moving or
"dynamic gestures," and may be beneficial for dynamic gesture detection.
5) “Hand Gesture recognition using a real-time tracking method and hidden Markov models” paper written by Feng-Sheng Chen,
Chih-MingFu, Chung-LinHuang said that They present a technique for recognising continuous gestures in front of a stationary
background. Ongoing hand following and extraction, including extraction, covered up Markov model (HMM) preparation, and
motion acknowledgment are the four segments that make up the framework. To follow the moving hand and concentrate the
hand locale, we first utilize a continuous hand following and extraction strategy, then, at that point we utilize the Fourier
descriptor (FD) to describe spatial highlights and movement investigation to portray fleeting qualities. As our feature vector, we
combine the spatial and temporal properties of the incoming image sequence. We use HMMs to recognise the input gesture
after extracting the feature vectors. Different HMMs are used to score the gesture to be recognised. The gesture is shown by the
model with the greatest score. We tried our system to recognise 20 various gestures in the experiments, and the recognition rate
was above 90%.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2296
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

6) “Microcontroller and sensor based gesture vocalizer” by Ata-Ur-Rehman Ata-Ur-Rehman, Salman Afghani, Muhammad
Akmal, Raheel Yousaf have given insight that Gesture Vocalizer is a large-scale multi-microcontroller-based system that aims
to improve communication between the dumb, deaf, and blind groups, as well as between them and the general public.This
system may be configured to function as a mobile "smart gadget". A gesture vocalizer based on a microcontroller and sensors is
presented in this work. The gesture vocalizer under discussion is essentially a data glove with a microprocessor. A data glove
can detect practically all hand movements, and a microcontroller-based system can translate some of those movements into
identifiable human speech. The data glove has two types of sensors: flex sensors that serve as bend sensors and accelerometers
that serve as tilt sensors. This device is advantageous to deaf people because it allows their hands to speak once “they have
worn the gesture vocalizer data glove.
7) “Hand Gesture Vocalizer for Dumb and Deaf People” paper Written by Sanish Manandhar, Sushana Bajracharya, Sanjeev
Karki, Ashish Kumar Jha have told that The significant objective of this work is to introduce a framework that changes over an
offered hint utilized by a handicapped individual into its appropriate composed, hear-able, and graphical structure using normal
segments like Arduino Mega, Flex sensors, and Accelerometer. A wearable glove regulator is planned with flex sensors on each
finger that permit the framework to recognize finger developments, just as a Gy-61 accelerometer that identifies the debilitated
individual's hand development. The gathered input signal is sent to the system for processing by the wearable input glove
controller. On the present training model, the system employs the Random Forest calculation to anticipate the right yield with a
precision of 85%.

III. PROPOSED METHODOLOGY

A. Random forest Algorithm
Random forests, also known as random decision forests, are an ensemble learning method or supervised classification algorithm for
classification, regression, and other tasks that work by training a large number of decision trees and then outputting the class that is
the mode of the classes (classification) or the mean prediction (regression) of the individual trees. This algorithm generates a forest
with a large number of trees, and the more trees in the forest, the more resilient it seems. The decision trees' characteristic of
overfitting to their training set is corrected by random decision forests. Ensemble algorithms are algorithms that integrate many
methods of the same or different types to categorize things.
For example, perform a prediction using Naive Bayes, SVM, and Decision Tree, and then vote on the final class for the test item.
Similarly, with the random forest classifier, a larger number of trees in the forest means that the outputs are more accurate. The
random forest classifier produces a collection of decision trees from a randomly selected subset of the training set, then combines
the votes from the individual decision trees to determine the test object's final class.
Finally, predictions are formed based on the majority of votes from each of the decision trees. This approach works effectively
because a single decision tree can be susceptible to noise, but aggregating numerous decision trees reduces the influence of noise,
resulting in more accurate results for various input sets.
The random forest may use the weight notion to assess the influence of each decision tree's output. When a tree's mistake rate is
high, it has a low weight value, and vice versa. This algorithm has two stages: the first is to generate a random forest classifier, and
the second is to produce a prediction using the random forest classifier built in the first step. The entire procedure is detailed here
There are four major steps of the DCP-based image de-hazing, which are atmospheric light estimation, transmission map estimation,
transmission map refinement, and image reconstruction.

B. Random Forest Prediction Pseudo Code

Takes the test characteristics, predicts the outcome using the rules of each randomly generated decision tree, and saves the projected
result (target).
Determine the number of votes for each forecasted goal.
Consider the random forest algorithm's final forecast to be the target with the most votes.
When we utilize the random forest technique to solve any classification issue, the problem of overfitting will never arise.
Both classification and regression tasks may be solved using the same random forest approach.
For feature engineering, the random forest method might be utilized.
It entails selecting the most essential characteristics from the training dataset's available feature

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2297
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

Training Sample Sample

Sample
set 1 3
2

Decision Decision Decision

Tree 1 Tree 2 Tree 3

Test Set
Voting

prediction

Random forest is regarded as an approach that is both accurate and reliable. This approach is free of the overfitting problem and
may be used to solve both classification and regression problems. Random forest may also manage missing values in two ways: by
replacing continuous variables with median values or by computing the proximity-weighted average of missing values.
Because it has several decision trees, the random forest takes a long time to provide predictions. This algorithm takes a long time to
complete. In comparison to a decision tree, where you can readily make a decision by following the route in the tree, the model is
harder to comprehend. When a user makes a hand motion, the glove controller sends eight user inputs to the system, including a
three-axis accelerometer signal and five flex sensor signals. The characteristics for detecting a given gesture are three axis values
and five flex sensor values that have been acquired from the user. These gesture characteristics were sent into a random forest
classifier, which then categorized the motions based on the features, and the output was recognised.
For the real-time application of the trained machine, the flex sensor and accelerometer data from the glove movement were gathered
and processed by the Arduino, turning those raw data to useful data. These values aid in the extraction of features. Those data were
delivered from the Arduino to the serial port, collected by Python via the serial interface, and recorded in CSV format as a
temporary dataset. Finally, the computer ran the temporary dataset through the machine, which extracted the dataset's characteristics
and predicted the proper output for that gesture using the Random Forest Classifier model. Because the model identified the output,
the output was displayed to the user on the laptop screen and over the speaker

IV. CONCLUSION
This examination presents another technique for hand signal distinguishing proof in this investigation. The foundation deduction
approach is utilized to recognize the hand area from the scenery. The palm and fingers are portioned after that. The fingers in the
hand picture are found and perceived dependent on the division. Hand motion acknowledgment is performed utilizing a
straightforward guideline classifier. On an informational collection of 1300 hand photos, the exhibition of our strategy is tried. The
consequences of the investigations propose that our strategy works adequately and is appropriate for continuous applications.
Besides, on a picture assortment of hand developments, the recommended procedure beats the cutting edge FEMD.

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2298
 International Journal for Research in Applied Science & Engineering Technology (IJRASET)
ISSN: 2321-9653; IC Value: 45.98; SJ Impact Factor: 7.538
Volume 10 Issue VI June 2022- Available at www.ijraset.com

The proposed technique's presentation is altogether reliant "on the consequences of hand location. On the off chance that there are
moving articles with a shading that is like that of the skin, the things are distinguished because of the hand identification and
decrease the hand signal acknowledgment execution.However, AI computations can distinguish the hand from the setting. ToF
cameras give profundity data that can help upgrade hand discovery execution. To address the confounded foundation issue and
improve the heartiness of hand distinguishing proof, AI calculations and ToF cameras might be applied in future exploration.
Currently, research efforts have primarily concentrated on recognizing static ISL indications from pictures or video sequences
captured under controlled conditions. The dimensionality of the sign recognition process will be minimized by employing the LDA
method. Noise will be decreased and with excellent accuracy as a result of dimensionality reduction. This project will be improved
in the future by determining the numbers that will be Displayed in words.
We sought to build this system by integrating a variety of image processing techniques and basic picture attributes. The recognition
of gestures has been accomplished using LDA algorithms. Remembering that every God creature has value in society, let us
endeavor to incorporate hearing challenged persons in our daily lives and live together.

V. FUTURE SCOPE
This research article discusses the design and operation of a machine that allows dumb, deaf, and blind people to communicate with
each other and with regular people. The stupid people utilise a common signal language that is difficult to understand by common
humans, and blind humans are unable to perceive their motions. This computer translates the signal language into a voice that is
easily understood by both blind and non-blind people. Sign language is also translated into a few written content formats to help
deaf people.

VI. ACKNOWLEDGEMENT
It gives us great pleasure in presenting the preliminary project report on ‘GESTURE VOCALIZER USING FLEX SENSOR AND
SOFTWARE VISUALIZATION’.

REFERENCES
[1] Sanish Manandhar, Sushana Bajracharya, Sanjeev Karki, Ashish Kumar Jha, “Hand Gesture Vocalizer for dumb and deaf people”, SCITECH Nepal,2019.
[2] www.elprocus.com
[3] en.wikipedia.org
[4] www.hindawi.com
[5] www.ijarcce.com
[6] Y C Venkatesh,A. S Srikantappa, P Dinesh. “Smart Brake Monitoring System with Brake Failure Indication for Automobile Vehicles”, IOP Conference
Series:Materials Science and Engineering,2020.
[7] Circuitdigest.com
[8] Sawant, Shreyashi Narayan, and M.S.Kumbhar.”Realtime sign language Recognition using PCA”,2014 IEEE International Conference on Advanced
Communications Control and Computing Technologies,2014.
[9] Yikai Fang, Kongqiao Wang,Jian Cheng, Haning Lu.”A Real-Time Hand Gesture Recognition
[10] mafiadoc.com
[11] link.springer.com
[12] dl.acm.org
[13] A. D. Bagdanov, A. Del Bimbo, L. Seidenari, and L. Usai, “Real-time hand status recognition from RGB-D imagery,” in Proceedings of the 21st International
Conference on Pattern Recognition (ICPR '12), pp. 2456–2459, November 2012.View at: Google Scholar
[14] M. Elmezain, A. Al-Hamadi, and B. Michaelis, “A robust method for hand gesture segmentation and recognition using forward spotting scheme in conditional
random fields,” in Proceedings of the 20th International Conference on Pattern Recognition (ICPR '10), pp. 3850–3853, August 2010.View at: Publisher Site |
Google Scholar
[15] C.-S. Lee, S. Y. Chun, and S. W. Park, “Articulated hand configuration and rotation estimation using extended torus manifold embedding,” in Proceedings of
the 21st International Conference on Pattern Recognition (ICPR '12), pp. 441–444, November 2012.View at: Google Scholar

©IJRASET: All Rights are Reserved | SJ Impact Factor 7.538 | ISRA Journal Impact Factor 7.894 | 2299