sensors

Article
Novel Wearable System to Recognize Sign Language in Real Time
İlhan Umut 1, * and Ümit Can Kumdereli 2

1 Department of Electronics and Automation, Corlu Vocational School, Tekirdag Namik Kemal University,
Tekirdag 59850, Türkiye
2 Department of Computer Engineering, Faculty of Engineering, Trakya University, Edirne 22030, Türkiye;
[email protected]
* Correspondence: [email protected]

Abstract: The aim of this study is to develop a practical software solution for real-time recognition of
sign language words using two arms. This will facilitate communication between hearing-impaired
individuals and those who can hear. We are aware of several sign language recognition systems
developed using different technologies, including cameras, armbands, and gloves. However, the
system we propose in this study stands out for its practicality, utilizing surface electromyography
(muscle activity) and inertial measurement unit (motion dynamics) data from both arms. We address
the drawbacks of other methods, such as high costs, low accuracy due to ambient light and obstacles,
and complex hardware requirements, which have limited their practical application. Our software
can run on different operating systems using digital signal processing and machine learning methods
specific to this study. For the test, we created a dataset of 80 words based on their frequency of
use in daily life and performed a thorough feature extraction process. We tested the recognition
performance using various classifiers and parameters and compared the results. The random forest
algorithm emerged as the most successful, achieving a remarkable 99.875% accuracy, while the naïve
Bayes algorithm had the lowest success rate with 87.625% accuracy. The new system promises to
significantly improve communication for people with hearing disabilities and ensures seamless
integration into daily life without compromising user comfort or lifestyle quality.

Keywords: artificial intelligence; computer software; human–computer interaction; inertial measurement

unit; sign language recognition; surface electromyography
Citation: Umut, İ.; Kumdereli, Ü.C.
Novel Wearable System to Recognize
Sign Language in Real Time. Sensors
2024, 24, 4613. https://doi.org/ 1. Introduction
10.3390/s24144613 The World Health Organization reports that around 466 million individuals have
Academic Editors: Olfa Kanoun and
hearing impairments, with this number expected to rise to 700 million by 2050 [1]. Sign
Rim Barioul
language uses hands, facial movements, and body posture to express thoughts, feelings,
and information instead of verbal communication. There are several types of sign languages
Received: 24 May 2024 with alphabets and signs used worldwide. TSL, like other sign languages, has a different
Revised: 12 July 2024 word order and grammar than Turkish. Although hearing-impaired individuals use sign
Accepted: 12 July 2024
language to communicate with each other, they may face difficulties when interacting
Published: 16 July 2024
with others. As a result, the prevalence of psychological problems among them is high [2].
Writing and reading are standard methods of communication between hearing people
and people who are deaf or hard of hearing. Writing is the most effective method of
Copyright: © 2024 by the authors.
communicating with a person with hearing impariment, mainly when accuracy is crucial.
Licensee MDPI, Basel, Switzerland.
However, in some situations, writing may not be feasible. Speechreading is another method
This article is an open access article
of communication, but many spoken Turkish sounds do not appear on the lips. Therefore, a
distributed under the terms and real-time sign language recognition (SLR) system is necessary to translate signs into sound
conditions of the Creative Commons or text.
Attribution (CC BY) license (https:// Although the focus of our paper is on SLR, most of the work in this area consists
creativecommons.org/licenses/by/ of hand gesture recognition. Hand gestures are used to express only letters and num-
4.0/). bers. However, communicating in this way can be slow and difficult. It is much faster

Sensors 2024, 24, 4613. https://doi.org/10.3390/s24144613 https://www.mdpi.com/journal/sensors

Sensors 2024, 24, 4613 2 of 18

and easier to sign words using hand, arm, and facial expressions instead of individual
letters [3]. Studies have been conducted on the subject and solutions have been devel-
oped. The majority of these studies have focused on specific sign languages, particularly
American Sign Language (ASL) and Chinese Sign Languages (CSLs) [3]. Various tech-
nologies have been employed, including Microsoft Kinect [4,5], Leap Motion [6–11], data
gloves [12,13], cameras [14–25], surface electromyography (sEMG), and inertial measure-
ment unit (IMU) [26–36]. Although some systems have achieved high levels of accuracy,
none are likely to be suitable for everyday real-life situations. Refer to Table 1 [37] for a
comparison of current technology-based systems.

Table 1. Comparison of existing it based solutions.

Device/Technology Accuracy (Around) Mobility User Convenience

Kinect 90% Not a mobile solution User must stand in front of the sensor
Data glove 80% Not a mobile solution User must wear device
Leap Motion 95–98% Not a mobile solution User must stand in front of the sensor
Image processing 90% Not a mobile solution User must stand in front of the camera

Motion data in image-oriented techniques is acquired when using a camera, while so-
lutions using data gloves acquire position, direction, and velocity information of movement
through sensors. Although most developers prefer image-based techniques for gesture
recognition, these techniques have several disadvantages. For instance, gestures are highly
dependent on perspective, which can cause different gestures to appear the same to a
poor-quality passive camera. Furthermore, image-based methods necessitate consistent
lighting and fixed camera placements while requiring high processing power, memory,
and energy. Although Leap Motion, a different visual-based method, has advantages over
cameras, it remains theoretical since it must be in a fixed position.
In solutions where sensors are used, the user wears gloves or armbands containing
various sensors to detect movements instead of relying on images. Fels et al. [38] are
pioneering teams in glove-based SLR methods. They achieved good classification success
(94%) using artificial neural networks to classify 203 ASL signals. However, wearing gloves
is uncomfortable under normal conditions, and technical obstacles limit the use of this
method in laboratory environments. When comparing armbands and gloves, it is essential
to note that armbands can help reduce sweating and provide a more aesthetically pleasing
appearance. They also require fewer sensors and can communicate wirelessly. Furthermore,
armband systems that use sEMG and IMU technology are popular due to their affordability
and portability. IMU sensors are specifically used to capture arm movements. sEMG
sensors, on the other hand, are suitable for acquiring movement information of hands and
fingers [39]. For instance, although some signs in TSL share similarities in hand and arm
movements, the movements of the fingers differ. This makes it challenging to differentiate
between IMU data alone and these movements. However, with the aid of sEMG data, finger
movements can also be distinguished, enabling the classification of signs. In a study on CSL
recognition, accelerometers and sEMG sensors were employed together. The study used a
hierarchical decision tree as the classification algorithm and achieved 95.3% accuracy [40].
The study most similar to ours was conducted by Wu et al. [41]. They developed a system
that utilized a wrist-worn IMU and four sEMG sensors to recognize 80 different ASL signs.
The study achieved an average accuracy of 96.16% using a support vector machine classifier.
However, their hardware was a prototype, and they did not have software like ours, which
can obtain and classify data simultaneously. Therefore, their study remained theoretical.
This study aims to develop software for real-time SLR using two Myo armbands.
The system offers critical contributions regarding practicality, affordability, and mobility.
Unlike many current technologies that are limited to controlled environments, this system
is designed to be used in everyday real-life situations. The system utilizes Myo armbands,
which are compact, light, and less costly than other technologies, enhancing mobility and
 This study aims to develop software for real-time SLR using two Myo armbands. The
system offers critical contributions regarding practicality, affordability, and mobility. Un-
Sensors 2024, 24, 4613 like many current technologies that are limited to controlled environments, this system
3 of 18is
designed to be used in everyday real-life situations. The system utilizes Myo armbands,
which are compact, light, and less costly than other technologies, enhancing mobility and
reducinguser
reducing userinconvenience
inconvenienceassociated
associatedwith
withmore
morecumbersome
cumbersomedevices.
devices.Our
Ourresearch
researchdid
did
not find a comparable system appropriate for daily
not find a comparable system appropriate for daily use.use.

2.2.Materials
Materialsand
andMethods
Methods
2.1. System Hardware
2.1. System Hardware
Data
Datafrom
fromthe
thearm
armwas
wascollected
collectedusing
usingtwo
twoMyo
Myoarmbands
armbandsfromfromThalmic
ThalmicLabs,
Labs,each
each
consisting of an IMU and sEMG sensors [42]. The device is depicted in Figure
consisting of an MU and sEMG sensors [42]. The device is depicted in Figure 1. 1.

Figure1.1.Myo
Figure Myoarmband.
armband.

Myoisisaacommercial
Myo commercialmotion motioncontroller
controllerthat thatcontains
containsan anARM
ARMCortex-M4
Cortex-M4120 120MHZ
MHZ
microprocessor, eight dry sEMGs with a sampling rate of 200
microprocessor, eight dry sEMGs with a sampling rate of 200 Hz, and a nine-axis IMU with Hz, and a nine-axis MU
awith a sampling
sampling rate of rate
50 Hz.of The
50 Hz. IMU The MU provides
provides ten different
ten different typesThe
types of data. of data. The accel-
accelerometers
erometers
in the device inmeasure
the device measure acceleration
acceleration in terms of the in gravitational
terms of the constant
gravitational constant
(G), while the (G),
gy-
while the
roscope gyroscope
measures measures
rotation in termsrotation in terms
of rad/s. of rad/s.
The device The device
includes includes magnetom-
magnetometers to obtain
orientation
eters to obtainand positioning
orientation measurements,
and positioningwhich are used towhich
measurements, determine the movement
are used to determine of
the
thearm. Additionally,
movement the device
of the arm. has eight
Additionally, thedry sEMG
device hassensors
eight drythatsEMG
allowsensors
for the that
detection
allow
of
forfinger movements.
the detection of finger movements.
The
TheMyo
Myoarmband
armbandwas wasthe theideal
idealchoice
choice forforthethe
study
studyoutlined
outlined in this manuscript
in this manuscript duedueto
its
to exceptional
its exceptionalfeatures thatthat
features make it particularly
make it particularlysuited for real-time
suited SLR.SLR.
for real-time The Myo armband
The Myo arm-
was
band selected for its integrated
was selected sensors,sensors,
for its integrated including IMU andMU
including sEMG andsensors. The IMUThe
sEMG sensors. sensors
MU
capture
sensorsarm movements,
capture while the sEMG
arm movements, while sensors
the sEMG detect muscle
sensors activity
detect related
muscle to hand
activity and
related
finger
to hand movements.
and finger This combination
movements. provides comprehensive
This combination data necessary
provides comprehensive fornecessary
data accurate
gesture
for accurate gesture recognition. The Myo armbands are wireless and highly portable,for
recognition. The Myo armbands are wireless and highly portable, allowing al-
practical,
lowing for everyday
practical, use outside use
everyday laboratory
outsideenvironments. Additionally,
laboratory environments. the armbands
Additionally, the
enable
armbands real-time
enableprocessing of the captured
real-time processing of the data.
captured Thedata.
Myo Thearmband processesprocesses
Myo armband data in
real time, making it ideal for translating sign language into text
data in real time, making it ideal for translating sign language into text or speech with or speech with minimal
delays.
minimal The sensors
delays. Thehave a high
sensors haveresolution, ensuringensuring
a high resolution, precise muscle
preciseactivity
muscleand motion
activity and
dynamics detection. This study’s results demonstrate that the
motion dynamics detection. This study’s results demonstrate that the random forest algo- random forest algorithm
achieved an accuracy
rithm achieved rate of
an accuracy 99.875%.
rate of 99.875%. TheThe Myo Myoarmband
armband design
design isisuser-friendly
user-friendlyand and
comfortable
comfortable to wear, which is crucial for ensuring user acceptance and continuoususe.
to wear, which is crucial for ensuring user acceptance and continuous use.Itt
can
canbe beeffortlessly
effortlesslyput putononandandtaken
taken offoffandanddoes
does notnot
necessitate
necessitatethe the
useruser
to handle complex
to handle com-
setups. Compared
plex setups. Comparedto other motion
to other capture
motion devices
capture such assuch
devices dataasgloves and camera-based
data gloves and camera-
systems, Myo armbands are a cost-effective option. This technology is highly accessible
based systems, Myo armbands are a cost-effective option. This technology is highly acces-
and practical for large-scale deployment thanks to its enhanced accessibility. The Myo
sible and practical for large-scale deployment thanks to its enhanced accessibility. The
armband boasts a robust application programming interface (API) that empowers the
creation of tailored applications, such as the SLR system mentioned in this manuscript.
This support allows for developers to personalize applications to suit the unique demands
of their projects. The Myo armband is an excellent choice for developing an SLR system
that is effective and practical for everyday use. It addresses many of the limitations found
in previous technologies.
Sensors 2024, 24, 4613 4 of 18

2.2. Developed Software

The application was developed using the Delphi programming language and Embar-
cadero RAD Studio 11 Integrated Development Environment. Two versions were created,
for Windows and Android platforms, with the possibility of creating additional versions
for Linux and IOS platforms using the same code. SQLite was used as the database. Fig-
ure 2 presents a flowchart outlining the sequential steps of the proposed methodology for
recognizing sign language using the Myo armband. A detailed description of Figure 2
using pseudocode is shown below. On the left is the test and on the right is the description
of the training algorithm.

START START
CONNECT MYO devices // Check Bluetooth and CONNECT MYO devices
location services then connects the MYO devices to :BEGIN
the system. Choose sign // Select the sign to use in
: BEGIN classification training
DO DO
Read raw data // Read raw data form connected Read raw data
devices (IMU and sEMG). Recognize movement
Recognize movement // Is there a change above WHILE movement recognized
the threshold value in any or both values taken DO
from the two devices? Read raw data
WHILE movement recognized Record data
// If motion is detected, proceed to the recording Recognize movement
cycle WHILE not movement recognized
DO // If motion is not detected, stop recording and
Read raw data proceed to the train
Record data // Save raw data to memory Stop recording
Recognize movement Extract features and save to the database //
WHILE no movement is recognized Extract Time domain, Frequency domain, and
// If motion is not detected, stop recording and entropy-based features and save them to the
proceed to the classify database with the label of the selected sign.
Stop recording IF continue record THEN GO TO BEGIN // go
Extract features and classify // Classify with the to BEGIN if record section is not closed.
selected classification algorithm by extracting Time Choose methods and parameters.
domain, Frequency domain, and Entropy-based Train and save model //Perform classification
features. training with the selected algorithm and parameters.
Show result // Show classification result. Play Then, save the model as a file
with voice via TTS STOP
IF not exit the app, THEN GO TO BEGIN // go
to BEGIN if the app is not closed
STOP

The program has five different user interfaces. All interfaces are given in Figure 3.

2.2.1. Main User Interface

Upon running the program for the first time, the user interface depicted in (Windows
Figure 3a and Android Figure 3b) will appear. The interface is designed with simplicity in
mind. Pressing the connect button triggers the program to check for location and Bluetooth
permissions. If permission is not granted, the program will request it from the user. If
permission is granted but Bluetooth connection or location access is turned off, the program
will turn them on. Finally, the program will establish a connection to both Myo armbands
using the Bluetooth Low Energy (BLE) protocol. After establishing the connection, the label
and oval-shaped markers on the left of the screen appear if the EMG and IMU services
are successfully subscribed. These markers turn blue when there is a change above the
threshold value in the data. The blue oval indicator under the TEST label indicates that
the program can make predictions from instantaneous movements. The guessed word
Sensors 2024, 24, 4613 5 of 18

is displayed in written form in the upper middle of the screen. The color transition in
the three-dimensional arms can be used to observe changes in the EMG data of each arm.
The battery status of each armband is displayed as a number ranging from 0 (empty)
to 100 (fully charged) in the upper left and right corners of the interface. To return the
three-dimensional arms to their reference position, click the refresh icon in the lower right
Sensors 2024, 24, x FOR PEER REV EW
corner. Access the settings interface by clicking on the three-line icon located in the 5lower
of 20
right corner.

Figure 2.
Figure 2. Flow chart of the
the software:
software: (a) test and (b) train.

The program has five different user interfaces. All interfaces are given in Figure 3.
Sensors 2024,
Sensors 24,24,
2024, 4613
x FOR PEER REV EW 6 of 2018
6 of

Figure 3. An example of the graphical user interfaces: (a) Windows main user interface; (b) Android
main user interface; (c) settings user interface; (d) records user interface; (e) train user interface;
(f) dictionary user interface.
Sensors 2024, 24, 4613 7 of 18

2.2.2. Settings User Interface

The interface (Figure 3c) allows for access and modification of all device and software
settings. It is possible to adjust the device’s data, vibration, lock, sleep, and wake functions.
Data Segmentation Settings:
Precise data segmentation settings can be adjusted individually. Each EMG and IMU
datum is stored in a global variable for segmentation. Motion is detected by calculating the
difference between the current and previous measurements of 200 Hz (EMG) and 50 Hz
(IMU) data. The user can adjust the limit values in the settings interface to suit their needs.
Movement Track Bar: The pause between two signals can be adjusted between 20 ms
and 1 s (1000 ms). If you wish to speak quickly, select the minimum value of 20 ms. In the
example interface, this value is set to 60 ms. If the limit value specified in the EMG or IMU
is not exceeded for 60 ms (12 measurements, each taking 5 ms since the device operates at
200 Hz), the sign is considered finished. Following the end of the sign, the data properties
are calculated and sent to the artificial intelligence algorithm for classification.
EMG Trackbar: The maximum value of the EMG Trackbar is 256, corresponding to
the range of EMG data from the device (−127 to 128). In the example interface, this value
is set to 40. Therefore, any absolute change of 40 in the values of the eight EMG sensors
indicates motion.
IMU Trackbar: The IMU orientation data determines the device’s position in the x, y,
and z planes and detects motion. The IMU Trackbar maximum value is set to 360, as this
is the maximum value of the angles being measured. In the provided interface, the value
is set to 20. If the absolute value of the mean of the changes in the Euler angle labels (α,
β, and γ) exceeds 20 degrees, it indicates movement. The Euler angles are obtained from
the IMU sensor using the quaternion method. To convert the given Euler angle labels to
quadrilateral labels, use the following conversion. The values of roll, pitch, and yaw angles
from the Euler angle notation are represented by α, β, and γ, respectively. The assumed
rotation order is from pitch to roll from deflection. The corresponding quarter q is defined
as follows:
   α β β 
qw cos 2 cos 2 cos γ2 + sin α2 sin 2 sin γ2
β β
 qx   sin α2 cos 2 cos γ2 − cos α2 sin 2 sin γ2 

q= =
 qy  cos α sin β cos γ + sin α cos β sin γ 
 (1)
2 2 2 2 2 2

qz β
cos α cos sin γ − sin α sin cos γ
β
2 2 2 2 2 2
Min. Rec. Time Track Bar: The minimum duration of the movement is set with the
minimum record time, which is 0.4 s in the example interface. A certain quantity of data is
required to calculate certain features. An option has been added to prevent the software
from giving errors.
Interface Features and Controls
Additionally, detailed information can be displayed or hidden using checkboxes for
Log, 3D Arm, Progress, Sound, and Graph for simplicity and performance improvement.
The Log Checkbox allows for instant viewing of all data and software/device changes.
The 3D Arm Checkbox displays arm movements on the screen in real time. The feature
can be hidden to reduce performance and battery consumption.
The Progress Checkbox displays recording and training times as a progress bar.
The signs are read aloud using the operating system’s text-to-speech feature if the
Sound Checkbox is selected. This software feature enables the user’s phone to convert sign
language movements into sound.
The Graph Checkbox allows for visualization of device data in chart form.
MYO Armband Classification Features
Additionally, the MYO Armband’s classification features can control the application
(e.g., putting the device to sleep, waking it up, and starting and stopping). Figure 4 shows
that the device classifies five hand movements as hardware and sends them over the BLE
service. The application can also utilize these features to generate ten different commands
from two different devices. For instance, when the user makes a ‘Fist’ movement with their
 The Graph Checkbox allows for visualization of device data in chart form.
MYO Armband Classification Features
Additionally, the MYO Armband’s classification features can control the application
(e.g., putting the device to sleep, waking it up, and starting and stopping). Figure 4 shows
Sensors 2024, 24, 4613 that the device classifies five hand movements as hardware and sends them over the8 of BLE18
service. The application can also utilize these features to generate ten different commands
from two different devices. For instance, when the user makes a ‘Fist’ movement with
right
their arm,
rightthe 3Dthe
arm, arms3Dmove
arms to the reference
move position.
to the reference To display
position. To the logs,the
display click on the
logs, icon
click on
of
thethree
iconlines in a lines
of three row in
in the lower
a row right
in the corner
lower of corner
right the mainof graphical interface. interface.
the main graphical The logs
will
The appear when
logs will the ‘Wave
appear when theIn’ movement is made with
‘Wave n’ movement is the
maderight arm
with and
the disappear
right arm and when
dis-
the samewhen
appear movement is made
the same again.is made again.
movement

Figure 4. The
Figure 4. The five
five gestures
gestures recognized
recognized by
by the
the Myo
Myo armband
armband[43].
[43].

When the ‘Fingers Spread’ movement is made with the right arm, the BLE charac-
When the ‘Fingers Spread’ movement is made with the right arm, the BLE character-
teristics of the devices are subscribed, meaning that the data begins to be received. In
istics of the devices are subscribed, meaning that the data begins to be received. n other
other words, the program starts. When the movement is repeated, the program stops. This
words, the program starts. When the movement is repeated, the program stops. This fea-
feature enables the user to modify the Movement Track Bar, including adjustments to the
ture enables the user to modify the Movement Track Bar, including adjustments to the
speech rate or other application settings, more efficiently. A multitude of commands can
speech rate or other application settings, more efficiently. A multitude of commands can
be executed via the wristband without the necessity of manual input from the user, even
be executed via the wristband without the necessity of manual input from the user, even
when the phone is stored within a pocket.
when the phone is stored within a pocket.
2.2.3. Records User Interface
2.2.3. Records User nterface
This interface was designed to collect data for training artificial intelligence. Figure 3d
Thisainterface
displays list box onwas
thedesigned to collect
left showing words data
andfor
thetraining
numberartificial intelligence.
of records associatedFigure
with
3d displays
each a listthe
word. Select box on the
word left showing
to create words
a new record andand the
click thenumber of records
‘New Rec’ associated
button. Recording
begins when the motion starts and continues until it ends. When the recording is complete,
the word count increases by one. If the Auto Checkbox is selected, the list box will
automatically switch to the next word. When the movement starts, the data for the next
word are saved. If it reaches the last word in the list, it will move on to the first word and
continue until the Stop Rec button is clicked. This interface allows for a graphical display
of data from the device during recording. Additionally, suppose a gif file with the same
name as the word exists. In that case, the movement video will be displayed if a gif file
with the same name as the word exists. The list box at the top displays the record numbers
of the selected words in the database. By selecting the record number, record that record’s
data are graphically displayed, allowing for the identification of any incorrect records. To
remove unwanted records, double-click (double Tab on the phone) on the record number
and select ‘yes’ in the warning message. The interface displays the graph of record number
244 for the word ‘correct’.

2.2.4. Train User Interface

Training involves using the data obtained after extracting features from the IMU and
EMG raw data. The algorithm and parameters selected in this interface are used to train
existing data. The training is then tested using the 10-fold cross-validation method. The
performance values of the end-of-training algorithm are displayed in Figure 3e. The sample
interface uses the K-nearest neighbor algorithm with a K = 3 parameter.

2.2.5. Dictionary User Interface

This interface allows for adding and removing words or sentences from the database
(refer to Figure 3f). Additionally, words can be uploaded from a txt file for convenience.
However, please note that loading from the file will result in deleting all existing records. If
accepted, a new dictionary will be added. As the application is personalized, it can create a
unique experience by allowing for users to choose their own words, create custom records,
and train it using artificial intelligence.
Sensors 2024, 24, 4613 9 of 18

2.3. Data Collecting

Determining the signs of TSL is a crucial aspect of this study, which the system will
test. To achieve this, we received support from Hülya AYKUTLU, a licensed sign language
Sensors 2024, 24, x FOR PEER REV EW
trainer and translator with years of experience in the Special Education Department. 10 The
of 20
testing location is shown in Figure 5.

Figure5.5.Testing
Figure Testingthe
thesystem
systemwith
withaasign
signlanguage
languageinstructor.
instructor.

Thesystem
The system waswas designed
designed to to predict
predict moremore than
than 80
80 words,
words, butbut to
to test
test its
its performance,
performance,
80words,
80 words,which
which is is the
the maximum
maximum number number of of words
words used
used inin similar
similar studies,
studies, were
were chosen.
Requests from
Requests from sign
signlanguage
language instructors
instructors and andusers
userswere
wereevaluated
evaluated while
whileselecting
selecting these
these
80words.
80 words.
To
Totest
testthe
thesystem,
system,80 80frequently
frequentlyused usedwords
wordsin insign
signlanguage
languageand anddaily
dailyspeech
speechwere
were
selected.
selected. The
The system
system alsoalsosupports
supportsadditional
additionalwords.words.
Below
Belowarearecategories
categoriesfor forthe
theselected
selectedwords:
words:
•• Greeting
Greeting and introduction
introduction words words (hello,
(hello, ’mI’mglad,
glad,meet,
meet, etc.),
etc.), Turkish
Turkish (merhaba,
(merhaba, se-
sevindim, görüşürüzvb.);
vindim, görüşürüz vb.);
•• Family
Familywords
words(mother,
(mother,father,
father,brother,
brother,etc.),
etc.),Turkish
Turkish(anne,
(anne,baba,
baba,abiabivb.);
vb.);
•• Pronouns and person signs (I, you, they etc.), Turkish (ben,
Pronouns and person signs ( , you, they etc.), Turkish (ben, sen, onlar vb.); sen, onlar vb.);
•• Common
Commonverbs verbs(come,
(come,go,go,take,
take,give,
give,etc.),
etc.),Turkish
Turkish(gel,
(gel,git,
git,al,
al,ver,
ver,vb.);
vb.);
•• Question words (what, why), Turkish
Question words (what, why), Turkish (ne, neden); (ne, neden);
•• OtherOther daily
daily life
life words
words (home,
(home, name,
name, good,
good, warm,
warm, easy,
easy,married,
married, year,
year,etc.),
etc.),Turkish
Turkish
(ev,
(ev,isim,
isim,iyi,
iyi,sıcak,
sıcak,kolay,
kolay,evli,
evli,yıl
yılvb.).
vb.).
The
The80-word
80-worddictionary
dictionarywas wasrepeated
repeated 1010times, with
times, each
with word
each wordbeing recorded
being by the
recorded by
IMU sensors of the Myo armband device 50 times per second.
the MU sensors of the Myo armband device 50 times per second. The data collected in- The data collected include
10 measurements,
clude consisting
10 measurements, of gyroscope
consisting accelerometer
of gyroscope (x, y, and
accelerometer z) and
(x, y, and z) orientation (x,
and orienta-
y, z, and w). Additionally, data from eight sEMG sensors of the
tion (x, y, z, and w). Additionally, data from eight sEMG sensors of the device are meas- device are measured
200
ured times
200 per second
times and stored
per second in memory
and stored during during
in memory recording. At the end
recording. of the
At the endrecording,
of the re-
cording, 1044 features are extracted from the stored data, including raw and feature-ex-
tracted data, which are then stored in the database. f the sign recording lasted for 1 s, 4200
raw data and 1044 feature data would be stored. The data were initially segmented using
either a fixed time or a fixed reference arm position. Users did not receive the slow and
tiring nature of the application testing well. A motion detection system is employed in-
Sensors 2024, 24, 4613 10 of 18

1044 features are extracted from the stored data, including raw and feature-extracted data,
which are then stored in the database. If the sign recording lasted for 1 s, 4200 raw data
and 1044 feature data would be stored. The data were initially segmented using either
a fixed time or a fixed reference arm position. Users did not receive the slow and tiring
nature of the application testing well. A motion detection system is employed instead of
a fixed time to ensure fast and effective communication during data segmentation. The
sensitivity setting can be adjusted in the settings section to determine the most suitable
option. Another important aspect of this feature is that hearing-impaired individuals may
produce signs at varying speeds depending on their level of excitement and emotional
state. As a result, the duration of the same sign may differ. Ten different recordings were
taken for each received sign to account for this.

2.4. Feature Extraction

In machine learning, distinguishable features are crucial, as they enhance the system’s
success and performance. The accuracy and performance of the developed SLR system
were improved by using feature extraction methods from the EMG data in Table 2 [44].
Some features listed in Table 2 were also utilized for the IMU data.

Table 2. Features extracted from sEMG signals.

Time Domain Frequency Domain Entropy-Based

Mean absolute value Peak frequency Shannon
Integrated EMG Median frequency Spectral
Higuchi fractal dimension Modified median frequency SVD
Petrosian fractal dimension Modified mean frequency Fisher
Detrended fluctuation analysis Intensity weighted mean frequency
Nonlinear energy Intensity weighted bandwidth
Slope Total spectrum
Line length Mean power spectrum
Willison amplitude Wavelet energy
Standard deviation AR coefficient 1
Min value AR modelling error 1
Max value AR coefficient 2
Hurst exponent AR modelling error 2
Minima AR coefficient 4
Maxima AR modelling error4
Skewness AR coefficient 8
Kurtosis AR modelling error 8
Zero crossings
Zero crossings of 1 derivative
Zero crossings of 2 derivatives
RMS amplitude
Inactive samples
Mobility
Activity
Complexity

2.4.1. Time Domain Features

This section describes the application of time domain features, including mean abso-
lute value, root mean square, zero crossing rate, and Willison amplitude. These features
provide additional information to that provided by frequency domain features, thereby
enhancing the classifier’s performance.
Mean absolute value (MAV) is a simple and effective time domain feature that repre-
sents the average of the absolute values of the signal.

1 N
N n∑
MAV = | Xn | (2)
=1

This equation calculates the average of the absolute values of the signal Xn , where N
is the total number of samples. In our study, MAV was used to quantify the overall activity
Sensors 2024, 24, 4613 11 of 18

level of the sEMG signals. This feature helps distinguish between different muscle activities
based on their intensity.
Root mean square (RMS) is another commonly used time domain feature that provides
a measure of the signal’s magnitude.
v
u1 N
u
RMS = t ∑ xn2 (3)
N n =1

This equation calculates the square root of the mean of the squares of the signal values
xn . It provides a measure of the signal’s energy. RMS was used to capture the energy
content of the sEMG signals, which is important for distinguishing between gestures with
different levels of muscle contraction.
Zero crossing rate (ZCR) measures the rate at which the signal changes sign, indicating
the frequency of signal oscillations.

1 N −1 h  i
ZCR = ∑
N − 1 i =1
I x ( i ) · x ( i +1) < 0 (4)

Here, I is an indicator function that equals 1 if the product x(i) · x(i+1) is less than
0, indicating a zero crossing, and 0 otherwise. ZCR was used to measure the frequency
content of the sEMG signals in the time domain. This feature is useful for identifying rapid
changes in muscle activity.
Willison amplitude (WAMP) counts the times the absolute difference between consec-
utive signal samples exceeds a predefined threshold.

N −1 h  i
WAMP = ∑ I x ( i +1) − x ( i ) > θ (5)
i =1

This equation sums the number of times the absolute difference between consecu-
tive samples x(i) and x(i+1) exceeds a threshold θ. WAMP was used to count the number
of significant changes in muscle activity, providing insights into the frequency and in-
tensity of the gestures. This feature helped distinguish between gestures with different
movement patterns.

2.4.2. Frequency Domain and Entropy-Based Features

This section outlines the application of the Fourier transform, wavelet transform, and
entropy-based methods for feature extraction. The application of these methods resulted in
the generation of a comprehensive and complementary set of features, which collectively
enhanced the performance of the classifier.
The Fourier transform is used to convert time domain signals into frequency do-
main representations. This helps in identifying the dominant frequencies present in the
sEMG signals.
N −1
X( f ) = ∑ x(n) .e− j2π f n/N (6)
n =1

The discrete Fourier transform (DFT) is derived from the continuous Fourier transform
but adapted for discrete signals. Here, x(n) represents the signal in the time domain, and
X( f ) represents its frequency domain representation. N is the total number of samples,
and j is the imaginary unit. The Fourier transform was applied to the sEMG signals to
convert them from the time domain to the frequency domain. This helped in identifying
the dominant frequencies, which are crucial for differentiating between various muscle
activities corresponding to different gestures.
Sensors 2024, 24, 4613 12 of 18

The wavelet transform is another method for analyzing signals in both time and
frequency domains.
Z ∞  ′
t −t

W(t,a) = xt′ ψ∗ dt′ (7)
−∞ a
The continuous wavelet transform (CWT) involves scaling and translating a wavelet
function ψ. t represents the translation parameter, and a represents the scale parameter. ψ∗
is the complex conjugate of the wavelet function.
The wavelet transform was used to extract time frequency features from the sEMG
and IMU signals. These features provided detailed information about the signal’s behavior
at different scales and time points, enhancing the classifier’s performance.
Entropy-based features captured the randomness and complexity of the signals. This
was important for distinguishing between similar gestures that might have different levels
of muscle activity and motion variability.
n
H(X ) = − ∑ P( xi ) logP( xi ) (8)
i =1

Shannon entropy is derived from information theory. It quantifies the amount of

uncertainty or surprise associated with random variables xi and their probabilities P( xi ) .
Shannon entropy was calculated for each segment of the sEMG and IMU signals to capture
the complexity and variability of the muscle activity and motion dynamics. This feature
was crucial for distinguishing between different sign language gestures.
The methods used converted all IMU and EMG data from two devices into 1044 pieces
of data, resulting in a high level of classification performance. However, training with raw
data was not successful.

2.5. Classification
In this study, we used various classification methods to address research objectives.
The Weka deep learning (WDL) algorithm was employed to harness the power of deep
learning for extracting features and classifying data. The K-nearest neighbor (KNN) method,
a non-parametric algorithm, was utilized for pattern recognition based on data point
proximity in the feature space. We also employed the multilayer perceptron (MLP), a
type of artificial neural network, for its ability to model complex relationships within the
data through its layered structure. Naïve Bayes (NB), a probabilistic classifier, was chosen
for its simplicity and efficiency in managing datasets. The random forest (RF) method,
an ensemble learning technique, was applied to combine the predictions from numerous
decision trees, improving the classification performance. Support vector machines (SVMs),
known for their effectiveness in high-dimensional spaces, were employed to determine
the optimal hyperplane to separate data. Each of these classification methods was selected
to make use of their specific strengths and capabilities in tackling the complexities of the
research problem, enabling a comparative study for a thorough analysis.
This study compared the classification performance of features obtained through
feature extraction using various classification algorithms and parameters. All algorithms
were tested using the 10-fold cross-validation method. The Weka application programming
interface (API), developed specifically for the Windows platform, allows for the use of all
algorithms available in Weka by converting the data in the database into ARFF file for-
mat [45]. Trainings are saved as a model file containing the algorithm name and parameters.
Therefore, a previously performed training can be predicted using the model file without
the need for retraining. Only the KNN algorithm is used on the Android platform due to
its classification performance and fast operation [46]. The aim is to incorporate additional
algorithms in the future, including the Weka API on the Android platform.
To add Weka algorithms to the program, edit the ‘Data\weka\algorithms.txt’ file
located in the program’s installation folder. Here is an example of the file’s contents:
bayes.NaiveBayes
Sensors 2024, 24, x FOR PEER REV EW 14 of 20
Sensors 2024, 24, 4613 13 of 18

bayes.NaiveBayes
lazy.IBk
lazy. Bk-K -K11-W-W00-A-A
trees.J48
trees.J48-C -C0.25
0.25-M
-M22
functions.MultilayerPerceptron
functions.MultilayerPerceptron-L -L0.3
0.3-M
-M0.2
0.2-N-N5050-V-V00-S-S00-E
-E20
20-H
-H50
50
trees.RandomForest
trees.RandomForest -P 100 - 100 -num-slots 1 -K 0 -M 1.0 -V 0.001-S-S11
-P 100 -I 100 -num-slots 1 -K 0 -M 1.0 -V 0.001
bayes.BayesNet
bayes.BayesNet-D -D-Q-Q
When the program
When the program is executed,
is executed,each added
each line is
added displayed
line as a combo
is displayed box, as box,
as a combo shownas
in Figurein6,Figure
shown when 6,the training
when interfaceinterface
the training is accessed.
is accessed.

Figure6.6.An
Figure Anexample
exampleWeka
Wekatraining
traininguser
usergraphical
graphicalinterface
interface(only
(onlyWindows).
Windows).

3.3.Results
Results
Innthe
themeasurements
measurements taken taken for
for aatotal
totalof of800
800data
datapoints,
points,consisting
consisting of
of80
80signs
signsandand
10
10repetitions,
repetitions,thetheaverage
averagetime
timetaken
takenforfor
feature
featureextraction from
extraction fromthethe
rawraw
datadata
andand
classifica-
classi-
tion afterafter
fication the signal ended
the signal was was
ended 21.2 21.2
ms. This
ms. Thisdemonstrates
demonstratesthe system’s ability
the system’s to perform
ability to per-
in real time, as the time was imperceptible to the users testing the
form in real time, as the time was imperceptible to the users testing the system. system.
The
The training results of
training results ofsix
sixdifferent
differentalgorithms,
algorithms, selected
selected based
based on on their
their popularity
popularity and
and classification success, were compared. Table 3 presents the
classification success, were compared. Table 3 presents the results of the training results of the training
con-
conducted
ducted using using a total
a total of 800
of 800 data
data points,
points, with with 80 signs
80 signs andand 10 records
10 records forfor each
each sign.
sign. TheThe10-
10-fold cross-validation method was used for testing. This method uses
fold cross-validation method was used for testing. This method uses all data for both test-all data for both
testing
ing and and training.
training. The
The default
default parametersofofthe
parameters thealgorithms
algorithmsin in Weka
Weka were
were used
used for
for this
this
comparison, as their performance was quite high. The default parameters
comparison, as their performance was quite high. The default parameters of the algo- of the algorithms
in Weka were used for this comparison, as their performance was quite high. No alternative
rithms in Weka were used for this comparison, as their performance was quite high. No
parameters were tested.
alternative parameters were tested.
In another application, the training was conducted by splitting the data at different
rates instead of using 10-fold cross-validation. Some of the randomly selected records from
10 records for each sign were used for training, while the remaining records were used for
testing. The results of these classifications, made using the same algorithm and parameters,
are also shown in Table 4.
Sensors 2024, 24, 4613 14 of 18

Table 3. Tenfold cross-validation classification performances of different algorithms.

Root Mean
Algorithm Accuracy (%) Kappa Statistic
Squared Error
Weka deep learning (WDL) 99.8750 0.9987 0.0053
K-nearest neighbor (KNN) 95.5000 0.9542 0.0020
Multilayer perceptron (MLP) 98.0000 0.9797 0.0201
Naïve Bayes (NB) 87.6250 0.8747 0.0556
Random forest (RF) 99.8750 0.9987 0.037
Support vector machine (SVM) 97.6250 0.9759 0.11

Table 4. Classification performances of different algorithms with split data.

Number of Records Used Algorithm Accuracy (%)

Training Test MLP WDL RF KNN SVN NB
1 1 73.75 62.5 30 71.25 62.5 16.25
2 2 90.625 73.75 89.375 78.125 81.25 26.875
3 3 91.25 99.5833 99.1667 89.166 90 45.833
4 4 92.1875 99.375 98.75 91.25 90 61.875
5 5 95 99.75 99.25 92.25 94 67.75
6 4 95 91.5625 99.0625 95 95.312 75.312
9 1 97.5 96.25 100 97.5 98.75 85

The classification results obtained from different algorithms indicate that the random
forest algorithm outperformed the naïve Bayes algorithm. It is important to note that train-
ing using a single record resulted in very low success rates. Therefore, it is recommended to
repeat each sign at least three times to create a record. Increasing the number of repetitions
is directly proportional to the increase in performance. Despite the variations in recording
speeds, the classification performance remains consistently high.
The variation in performance among different algorithms in a 10-fold cross-validation
classification task can be attributed to several factors, such as the algorithms’ nature, their
handling of data complexity, and their sensitivity to the specifics of the dataset used. In
this section, we will evaluate the performance of the listed algorithms based on three key
metrics: accuracy, kappa statistic, and root mean squared error (RMSE).
WDL and RF performance: WDL and RF demonstrated exceptional accuracy of
99.875%, with identical kappa statistics of 0.9987, indicating almost perfect classification
capabilities compared to a random classifier. However, it is worth noting that WDL
outperforms RF regarding RMSE, with an impressively low value of 0.0053, compared to
RF’s RMSE of 0.037. The analysis shows that WDL is more consistent in its predictions
across the dataset, possibly due to better handling of outlier data or noise within the dataset.
KNN performs moderately well, with an accuracy of 95.5% and a kappa statistic of
0.9542. It has the lowest RMSE among all algorithms at 0.0020, indicating tight clustering
around the true values. KNN is a strong model despite its lower accuracy compared to
WDL and RF. It is important to note that KNN may require parameter tuning, such as the
choice of ‘k’, and may be sensitive to noisy data.
MLP exhibits a strong performance with an accuracy of 98% and a kappa statistic
of 0.9797 despite its relatively higher RMSE of 0.0201. The higher RMSE compared to its
accuracy and kappa indicates variations in prediction errors, possibly due to the complexity
of the model and the need for careful tuning of its layers and neurons.
NB: In contrast, NB demonstrates the lowest performance among all evaluated models,
with an accuracy of 87.625%, a kappa statistic of 0.8747, and a relatively high RMSE of 0.0556.
While NB may encounter difficulties when dealing with datasets where features are not
independent, which is a core assumption of the algorithm, SVM can handle such datasets.
Although SVM has the highest RMSE of 0.11 among the algorithms, its superior
performance in other areas makes it the recommended choice. The analysis demonstrates
that the SVM algorithm outperforms the NB algorithm in terms of accuracy and kappa
Sensors 2024, 24, 4613 15 of 18

statistics. Although SVM has the highest RMSE of 0.11 among the algorithms, its superior
performance in other areas makes it the recommended choice. Although SVM has the
highest RMSE of 0.11 among the algorithms, its superior performance in other areas makes
it the recommended choice. The high RMSE, despite good accuracy and kappa statistic,
suggests that SVM’s decision boundary may be less stable or more sensitive to individual
data points, possibly due to the choice of kernel or regularization parameters.
WDL and RF outperform the other models in terms of accuracy and kappa statistics,
likely due to their robustness to data imperfections and their ability to model complex
patterns. WDL is superior in handling outliers or noise compared to RF, as evidenced by its
lower RMSE. The other models’ performance is dependent on their intrinsic assumptions
and sensitivity to data characteristics. It is imperative to select the appropriate model based
on the specific requirements and nature of the dataset.

4. Discussion
In their review article, Kudrinko et al. examined all systematic literature review studies
to date and identified the essential features that should be present in SLR systems [47].
• Real-life usability: Sign language technology should be applicable to real-life situations
outside of the laboratory.
• Accuracy and minimal delay: The technology must accurately convert sequences of
movements into text and speech with minimal delay.
• Aesthetics and comfort: The technology design should prioritize aesthetics and com-
fort for the user.
• Wireless and rechargeable: To be useful, it must be wireless and easily rechargeable.
• User-centric design: When designing systems.
The existing literature focuses mainly on recognizing sign language movements,
but fails to meet these essential criteria, thereby limiting its practical applicability. To
address this gap, our study presents a mobile application developed in collaboration with
10 individuals with disabilities who provided valuable feedback confirming its utility and
effectiveness in real-life scenarios. In particular, our approach emphasizes personalized
application development, recognizing the inherent variability of sEMG data between users.
By tailoring the application to individual physiological characteristics, we mitigate the
challenges posed by inter-user variability, thereby enhancing performance and usability.
In addition, the inclusion of multiple users in the training sets in previous studies may be
why classification performance decreases as the number of signs increases, highlighting the
importance of personalized approaches to account for individual differences and optimize
system performance. Through these efforts, we aim not only to advance the theoretical
understanding of SLR but also to provide practical solutions that address the diverse needs
of people with hearing impairments, ultimately facilitating seamless communication and
integration into society.
The KNN algorithm and its corresponding model were chosen for the mobile appli-
cation due to their exceptional performance and accuracy. This algorithm has a single
effective parameter, the K parameter.
The user can add any number of words to the dictionary. Since biological data such
as EMG may vary from person to person, the application is for a single person’s use.
When different people’s data have been used for training on the same model in other
studies, success rates have been low. The user creates their model by creating their own
training data by performing the desired movements. They can adjust the desired delay
and sensitivity settings. Since the Text to Speech (TTS) feature in the application uses the
TTS system in the operating system, voiceover can be made in the Turkish language. The
application allows for users to create their dictionary, store data, and train and use it with
the existing artificial intelligence algorithm. In addition, the application allows for users
to control the device using sign language, including putting it to sleep, waking it up, and
starting and stopping it. All signals are easily recognizable when using the Myo wristbands.
Sensors 2024, 24, 4613 16 of 18

The application is now being actively used by disabled Please state the version number of
the software.people who participated in the trials.

Limitations
Despite the promising results of the proposed real-time SLR (v1.0) software, some
limitations should be acknowledged.
Firstly, the results presented in Tables 3 and 4 were obtained from a single subject,
limiting the findings’ generalizability. Although the device demonstrated high accuracy
and performance for this individual, there is no concrete evidence to suggest that it would
perform similarly in a wider population. Future studies will include a more diverse group
of subjects to confirm the device’s effectiveness and ensure its applicability to a wider user
base. In addition, the between-subject variance will be calculated and reported to provide a
full understanding of the variability in the device’s performance.
Secondly, although the software shows potential for use with other sign languages,
the current study focuses only on TSL. A shortcoming is the lack of classification results
for other languages, such as ASL. Extending the study to more sign languages would
provide concrete evidence of the adaptability and reliability of the software in different
linguistic contexts.
In conclusion, while the initial results are promising, addressing these limitations in
future research will be crucial to validate the effectiveness of the software and increase
its robustness and applicability across different user groups and sign languages. In the
future, we will redesign the software to use the device we have developed. Although we
encountered a challenge during this study with the discontinuation of the Myo armband,
we overcame this obstacle by embarking on a project to develop our own device. We aim to
create a novel system that comprehensively addresses the needs of people with disabilities.
We envision this system not only as a replacement for the Myo armband but also as
an innovative solution that surpasses its capabilities. Through this initiative, we aim to
democratize access to assistive technology and ensure that people with disabilities around
the world can benefit from our system. Our goal is to make this device readily available
and accessible to anyone in need, thereby promoting inclusivity and empowerment within
the disability community.

Author Contributions: Conceptualization, Ü.C.K.; Data curation, Ü.C.K.; Formal analysis, Ü.C.K.;
Funding acquisition, Ü.C.K.; Investigation, Ü.C.K.; Methodology, İ.U. and Ü.C.K.; Project adminis-
tration, İ.U.; Resources, Ü.C.K.; Software, İ.U.; Supervision, İ.U.; Validation, İ.U.; Visualization, İ.U.;
Writing—original draft, İ.U.; Writing—review and editing, Ü.C.K. All authors have read and agreed
to the published version of the manuscript.
Funding: No funding was received for conducting this study.
Data Availability Statement: Access to these data are restricted, but interested researchers may
contact Assoc. Prof. İlhan UMUT ([email protected]) for inquiries regarding data access.
Acknowledgments: The authors acknowledge the support given by Hülya AYKUTLU in providing
them with knowledge.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. World Health Organization. Deafness and Hearing Loss. Available online: https://www.who.int/news-room/fact-sheets/
detail/deafness-and-hearing-loss (accessed on 6 April 2023).
2. Crealey, G.E.; O’Neill, C. Hearing loss, mental well-being and healthcare use: Results from the Health Survey for England (HSE).
J. Public Health 2018, 42, 77–89. [CrossRef]
3. Cheok, M.J.; Omar, Z.; Jaward, M.H. A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn.
Cybern. 2019, 10, 131–153. [CrossRef]
4. Yang, H.-D. Sign Language Recognition with the Kinect Sensor Based on Conditional Random Fields. Sensors 2015, 15, 135–147.
[CrossRef]
Sensors 2024, 24, 4613 17 of 18

5. Zafrulla, Z.; Sahni, H.; Bedri, A.; Thukral, P.; Starner, T. Hand detection in American Sign Language depth data using domain-
driven random forest regression. In Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic
Face and Gesture Recognition (FG), Ljubljana, Slovenia, 4–8 May 2015; Volume 1, pp. 1–7. [CrossRef]
6. Abdullahi, S.B.; Chamnongthai, K. American Sign Language Words Recognition of Skeletal Videos Using Processed Video Driven
Multi-Stacked Deep LSTM. Sensors 2022, 22, 1406. [CrossRef] [PubMed]
7. Abdullahi, S.B.; Chamnongthai, K. American Sign Language Words Recognition Using Spatio-Temporal Prosodic and Angle
Features: A Sequential Learning Approach. IEEE Access 2022, 10, 15911–15923. [CrossRef]
8. Abdullahi, S.B.; Chamnongthai, K. IDF-Sign: Addressing Inconsistent Depth Features for Dynamic Sign Word Recognition. IEEE
Access 2023, 11, 88511–88526. [CrossRef]
9. Abdullahi, S.B.; Chamnongthai, K.; Bolon-Canedo, V.; Cancela, B. Spatial–temporal feature-based End-to-end Fourier network for
3D sign language recognition. Expert Syst. Appl. 2024, 248, 123258. [CrossRef]
10. Katılmış, Z.; Karakuzu, C. ELM based two-handed dynamic Turkish Sign Language (TSL) word recognition. Expert Syst. Appl.
2021, 182, 115213. [CrossRef]
11. Chuan, C.H.; Regina, E.; Guardino, C. American Sign Language Recognition Using Leap Motion Sensor. In Proceedings of the
2014 13th International Conference on Machine Learning and Applications, Detroit, MI, USA, 3–6 December 2014; pp. 541–544.
[CrossRef]
12. Wang, H.; Leu, M.; Oz, C. American Sign Language Recognition Using Multidimensional Hidden Markov Models. J. Inf. Sci. Eng.
2006, 22, 1109–1123.
13. Tubaiz, N.; Shanableh, T.; Assaleh, K. Glove-Based Continuous Arabic Sign Language Recognition in User-Dependent Mode.
IEEE Trans. Hum.-Mach. Syst. 2015, 45, 526–533. [CrossRef]
14. Garg, A. Converting American Sign Language to Voice Using RBFNN. Ph.D. Thesis, San Diego State University, San Diego, CA,
USA, 2012.
15. Starner, T.; Weaver, J.; Pentland, A. Real-time american sign language recognition using desk and wearable computer-based video.
IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20, 1371–1375. [CrossRef]
16. Thepade, S.D.; Kulkarni, G.; Narkhede, A.; Kelvekar, P.; Tathe, S. Sign language recognition using color means of gradient slope
magnitude edge images. In Proceedings of the 2013 International Conference on Intelligent Systems and Signal Processing (ISSP),
Vallabh Vidyanagar, India, 1–2 March 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 216–220.
17. Kim, T. American Sign Language fingerspelling recognition from video: Methods for unrestricted recognition and signer-
independence. arXiv 2016, arXiv:1608.08339.
18. Sadeddine, K.; Chelali, F.Z.; Djeradi, R. Sign language recognition using PCA, wavelet and neural network. In Proceedings of the
2015 3rd International Conference on Control, Engineering & Information Technology (CEIT), Tlemcen, Algeria, 25–27 May 2015;
pp. 1–6. [CrossRef]
19. Yang, R.; Sarkar, S.; Loeding, B. Handling Movement Epenthesis and Hand Segmentation Ambiguities in Continuous Sign
Language Recognition Using Nested Dynamic Programming. IEEE Trans. Pattern Anal. Mach. Intell. 2010, 32, 462–477. [CrossRef]
[PubMed]
20. Bilge, Y.C.; Cinbis, R.G.; Ikizler-Cinbis, N. Towards Zero-Shot Sign Language Recognition. IEEE Trans. Pattern Anal. Mach. Intell.
2023, 45, 1217–1232. [CrossRef] [PubMed]
21. Liu, Z.; Pang, L.; Qi, X. MEN: Mutual Enhancement Networks for Sign Language Recognition and Education. IEEE Trans. Neural
Netw. Learn. Syst. 2022, 35, 311–325. [CrossRef] [PubMed]
22. Mohandes, M.; Deriche, M.; Liu, J. Image-Based and Sensor-Based Approaches to Arabic Sign Language Recognition. IEEE Trans.
Hum.-Mach. Syst. 2014, 44, 551–557. [CrossRef]
23. Han, X.; Lu, F.; Yin, J.; Tian, G.; Liu, J. Sign Language Recognition Based on R (2+1) D With Spatial–Temporal–Channel Attention.
IEEE Trans. Hum.-Mach. Syst. 2022, 52, 687–698. [CrossRef]
24. Rajalakshmi, E.; Elakkiya, R.; Subramaniyaswamy, V.; Alexey, L.P.; Mikhail, G.; Bakaev, M.; Kotecha, K.; Gabralla, L.A.; Abraham,
A. Multi-Semantic Discriminative Feature Learning for Sign Gesture Recognition Using Hybrid Deep Neural Architecture. IEEE
Access 2023, 11, 2226–2238. [CrossRef]
25. Rajalakshmi, E.; Elakkiya, R.; Prikhodko, A.L.; Grif, M.G.; Bakaev, M.A.; Saini, J.R.; Kotecha, K.; Subramaniyaswamy, V. Static and
Dynamic Isolated Indian and Russian Sign Language Recognition with Spatial and Temporal Feature Detection Using Hybrid
Neural Network. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 2022, 22, 26. [CrossRef]
26. Paudyal, P.; Banerjee, A.; Gupta, S.K.S. SCEPTRE: A Pervasive, Non-Invasive, and Programmable Gesture Recognition Technology.
In Proceedings of the 21st International Conference on Intelligent User Interfaces, Sonoma, CA, USA, 7–10 March 2016. [CrossRef]
27. Fatmi, R.; Rashad, S.; Integlia, R.; Hutchison, G. American Sign Language Recognition using Hidden Markov Models and
Wearable Motion Sensors. Trans. Mach. Learn. Data Min. 2017, 10, 41–55.
28. Savur, C.; Sahin, F. American Sign Language Recognition system by using surface EMG signal. In Proceedings of the 2016 IEEE
International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016; pp. 002872–002877.
[CrossRef]
29. Seddiqi, M.; Kivrak, H.; Kose, H. Recognition of Turkish Sign Language (TID) Using sEMG Sensor. In Proceedings of the 2020
Innovations in Intelligent Systems and Applications Conference (ASYU), İstanbul, Turkey, 15–17 October 2020; pp. 1–6. [CrossRef]
Sensors 2024, 24, 4613 18 of 18

30. Zhang, Z.; Yang, K.; Qian, J.; Zhang, L. Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial
Neural Network. Sensors 2019, 19, 3170. [CrossRef] [PubMed]
31. Zhang, Z.; Su, Z.; Yang, G. Real-Time Chinese Sign Language Recognition Based on Artificial Neural Networks. In Proceedings of
the 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China, 6–8 December 2019; pp. 1413–1417.
[CrossRef]
32. Jane, S.P.Y.; Sasidhar, S. Sign Language Interpreter: Classification of Forearm EMG and IMU Signals for Signing Exact English. In
Proceedings of the 2018 IEEE 14th International Conference on Control and Automation (ICCA), Anchorage, AK, USA, 12–15
June 2018; pp. 947–952. [CrossRef]
33. Yang, X.; Chen, X.; Cao, X.; Wei, S.; Zhang, X. Chinese Sign Language Recognition Based on an Optimized Tree-Structure
Framework. IEEE J. Biomed. Health Inform. 2017, 21, 994–1004. [CrossRef] [PubMed]
34. Lee, B.G.; Lee, S.M. Smart Wearable Hand Device for Sign Language Interpretation System with Sensors Fusion. IEEE Sens. J.
2018, 18, 1224–1232. [CrossRef]
35. Fatmi, R.; Rashad, S.; Integlia, R. Comparing ANN, SVM, and HMM based Machine Learning Methods for American Sign Lan-
guage Recognition using Wearable Motion Sensors. In Proceedings of the 2019 IEEE 9th Annual Computing and Communication
Workshop and Conference (CCWC), Las Vegas, NV, USA, 7–9 January 2019; pp. 0290–0297. [CrossRef]
36. Zheng, Z.; Wang, Q.; Yang, D.; Wang, Q.; Huang, W.; Xu, Y. L-Sign: Large-Vocabulary Sign Gestures Recognition System. IEEE
Trans. Hum.-Mach. Syst. 2022, 52, 290–301. [CrossRef]
37. Madushanka, A.L.P.; Senevirathne, R.G.D.C.; Wijesekara, L.M.H.; Arunatilake, S.M.K.D.; Sandaruwan, K.D. Framework for
Sinhala Sign Language recognition and translation using a wearable armband. In Proceedings of the 2016 Sixteenth International
Conference on Advances in ICT for Emerging Regions (ICTer), Negombo, Sri Lanka, 1–3 September 2016; pp. 49–57. [CrossRef]
38. Fels, S.S.; Hinton, G.E. Glove-Talk: A neural network interface between a data-glove and a speech synthesizer. IEEE Trans. Neural
Netw. 1993, 4, 2–8. [CrossRef] [PubMed]
39. Jiang, S.; Lv, B.; Guo, W.; Zhang, C.; Wang, H.; Sheng, X.; Shull, P.B. Feasibility of Wrist-Worn, Real-Time Hand, and Surface
Gesture Recognition via sEMG and IMU Sensing. IEEE Trans. Ind. Inform. 2018, 14, 3376–3385. [CrossRef]
40. Zhang, X.; Chen, X.; Li, Y.; Lantz, V.; Wang, K.; Yang, J. A Framework for Hand Gesture Recognition Based on Accelerometer and
EMG Sensors. IEEE Trans. Syst. Man Cybern.—Part A Syst. Hum. 2011, 41, 1064–1076. [CrossRef]
41. Wu, J.; Sun, L.; Jafari, R. A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG
Sensors. IEEE J. Biomed. Health Inform. 2016, 20, 1281–1290. [CrossRef] [PubMed]
42. Visconti, P.; Gaetani, F.; Zappatore, G.A.; Primiceri, P. Technical Features and Functionalities of Myo Armband: An Overview on
Related Literature and Advanced Applications of Myoelectric Armbands Mainly Focused on Arm Prostheses. Int. J. Smart Sens.
Intell. Syst. 2018, 11, 1–25. [CrossRef]
43. Cognolato, M.; Atzori, M.; Faccio, D.; Tiengo, C.; Bassetto, F.; Gassert, R.; Muller, H. Hand Gesture Classification in Transradial
Amputees Using the Myo Armband Classifier. In Proceedings of the 2018 7th IEEE International Conference on Biomedical
Robotics and Biomechatronics (Biorob), Enschede, The Netherlands, 26–29 August 2018; pp. 156–161.
44. Umut, İ. PSGMiner: A modular software for polysomnographic analysis. Comput. Biol. Med. 2016, 73, 1–9. [CrossRef]
45. Eibe, F.; Hall, M.A.; Witten, I.H. The WEKA Workbench. In Online Appendix for Data Mining: Practical Machine Learning Tools and
Techniques, 4th ed.; Morgan Kaufmann: San Francisco, CA, USA, 2016.
46. Daniel, T.L.; Chantal, D.L. k Nearest Neighbor Algorithm. In Discovering Knowledge in Data: An Introduction to Data Mining; Wiley:
Hoboken, NJ, USA, 2014; pp. 149–164.
47. Kudrinko, K.; Flavin, E.; Zhu, X.; Li, Q. Wearable Sensor-Based Sign Language Recognition: A Comprehensive Review. IEEE Rev.
Biomed. Eng. 2021, 14, 82–97. [CrossRef] [PubMed]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.