IMA: DESIGN AND IMPLEMENTATION OF

FILIPINO SIGN LANGUAGE TRANSLATOR

VIA SMART WEARABLE GLOVE
TECHNOLOGY
CONRADO M. GALANG JR
JORGE NEIL ERICKSON A. BAGASOL
CLEIFE DAREN I. PALERACIO
DARYLL KEITH A. OASAY
ISAIAH DAVID A. SUNIGA

CAPSTONE PROJECT
THE PROBLEM
COMMUNICATION BETWEEN
THE DEAF AND MUTE AND
HEARING PEOPLE REMAINS A
CHALLENGE that needs to be addressed.
Thus, more attention is needed by information
system/information technology (IT) researchers to
solve this two-way communication problem.
 1
2 0
2 ORGANIZATION
WORLD HEALTH

0 3 3 0 1
0 40 120
1 4
PEOPLE IN THE WORLD HAVE PARTIAL AND FULL

1 51 231
SPEAKING AND HEARING DISABILITY

2 5
7 7 77 7 777
8 8 88 8 888
WORLD HEALTH ORGANIZATION

7 0 00 0 000
PEOPLE IN THE WORLD HAVE PARTIAL AND FULL
SPEAKING AND HEARING DISABILITY
ASSISTIVE DEVICES

GLOVE BASED SYSTEM

RESEARCH ARTICLE

PROPOSED IT INTERVENTION
ASSISTIVE DEVICES
GLOVE BASED SYSTEM

- Abhijith Bhaskaran et al. -

The glove’s sensors sensed and define the hand gesture

through sign language that corresponds to a specific
message.

RESEARCH ARTICLE

PROPOSED IT INTERVENTION
 ASSISTIVE DEVICES
GLOVE BASED SYSTEM

RESEARCH ARTICLE

- CALIFORNIA -
SpaceX designs and builds its reusable rockets and spacecraft at its headquarters in
Hawthorne, California. As a company, SpaceX is vertically integrated, building the vast
- Barbour
majority of the vehicle on the Hawthorne etSpaceX
campus. al - headquarters remains one of the
few facilities in the world where you can see an entire launch vehicle or spacecraft come
Numerous systems have alsounder
together been developed using sensors,
one roof.

image processing, and smartphones to facilitate

communication between the two parties.

PROPOSED IT INTERVENTION
 ASSISTIVE DEVICES
GLOVE BASED SYSTEM

RESEARCH ARTICLE

PROPOSED IT INTERVENTION
- CALIFORNIA -
SpaceX designs and builds its reusable rockets and spacecraft at its headquarters in
Hawthorne, California. As a company, SpaceX is vertically integrated, building the vast
- CALIFORNIA
majority of the vehicle on the Hawthorne campus. SpaceX - headquarters remains one of the
few facilities in the world where you can see an entire launch vehicle or spacecraft come
SpaceX designs and builds itstogether
reusableunder
rockets
oneand spacecraft at its headquarters in
roof.
Hawthorne, California. As a company, -SpaceX
IMA - is vertically integrated, building the vast
majority of the vehicle on the Hawthorne campus. SpaceX headquarters remains one of the
thisfew facilities in the world where you can see an entire launch vehicle or spacecraft come
device uses the analysis of the Filipino Sign Language (FSL)
together under one roof.
alphabet for which every sign corresponds to a specific message.
OBJECTIVES
 design an assistive device for
people with hearing/speech

01
impairment that interprets hand
sign languages into a readable
text messages and recognizable
voice audio;
 determine the accuracy of

02
the sensors after conducting
the calibration process
 refine a system based on user
input and ensure that the

03 gloves are culturally sensitive

and appropriate for the users
SIGNIFICANCE OF THE STUDY

Deaf and Mute Individuals and SPED Teachers

Relatives of the Hearing –
impaired Individuals
 SCOPE AND LIMITATIONS

Exclusion of Bluetooth Filipino Sign Language Predefined Vocabulary

Module
 PROCESS OUTPUT
INPUT 1. Detection of hand
1. FSL hand 1. Translated text
gestures by the gesture- messages: Converted
gestures: Primary based sensors in the FSL gestures into
input for smart glove readable text for
glove translation. 2. Interpretation of communication.
detected gestures using 2. Recognizable voice
2. User input & predefined mapping to audio of translated
feedback: Guides FSL alphabet messages: Audible
tech adaptation and 3. Translation of FSL representation of
improvement. gestures into readable translated messages for
text messages and accessibility.
3. Sensor data: recognizable voice audio 3. User feedback on
Captures hand 4. Output of translated system accuracy and
movements for messages through voice usability: Valuable input
accurate gesture output via speaker and for improving system
text output displayed on performance and user
interpretation. LCD experience.

OUTCOME:
1. Enhanced communication for deaf and hard-of-hearing individuals
2. Improved accessibility through wearable technology
3. Facilitated real-time translation of FSL into text or speech
4. Improved communication between deaf/mute individuals and hearing relatives
5. Provided valuable user feedback for future refinement and enhancement of the technology
CONCEPTUAL
FRAMEWORK
 PROCESS OUTPUT
INPUT 1. Detection of hand
1. FSL hand 1. Translated text
gestures by the gesture- messages: Converted
gestures: Primary based sensors in the FSL gestures into
input for smart glove readable text for
glove translation. 2. Interpretation of communication.
detected gestures using 2. Recognizable voice
2. User input & predefined mapping to audio of translated
feedback: Guides FSL alphabet messages: Audible
tech adaptation and 3. Translation of FSL representation of
improvement. gestures into readable translated messages for
text messages and accessibility.
3. Sensor data: recognizable voice audio 3. User feedback on
Captures hand 4. Output of translated system accuracy and
movements for messages through voice usability: Valuable input
accurate gesture output via speaker and for improving system
text output displayed on performance and user
interpretation. LCD experience.

OUTCOME:
1. Enhanced communication for deaf and hard-of-hearing individuals
2. Improved accessibility through wearable technology
3. Facilitated real-time translation of FSL into text or speech
4. Improved communication between deaf/mute individuals and hearing relatives
5. Provided valuable user feedback for future refinement and enhancement of the technology
CONCEPTUAL
FRAMEWORK
LOCALE OF THE STUDY
LOCALE OF THE STUDY

Surveys wcarried out at select schools in Ilocos Norte with Special Education
(SPED) programs, namely Agripino P. Santos Elementary School, Bible Believing
Baptist Church Educational Ministries Foundation

Inc., and Ilocos Norte National High School.

RESEARCH DESIGN
The research used experimental analysis
during the design and implementation of
Filipino Sign Language translator via smart
wearable glove technology. This study
evaluated the capabilities of the IMA in terms
of accuracy, comfortability, and effectiveness
to the users.
VARIABLES OF THE STUDY
This study examined how accurately the gloves
translate sign language gestures into text or
spoken language and assessed the speed at which
the gloves can recognize and translate gestures in
real-time. In addition, the device’s durability was
observed as it will be essential for its long-term
usability.

Users’ hand size, familiarity with sign language,

and their speed in hand gestures was considered
since not all deaf-mute and hearing people are
capable.
Project Conceptualization
and Construction
Project Conceptualization
and Construction
Project Conceptualization
and Construction
Flowchart of IMA: Design and
Implementation of Filipino Sign
Language Translator via Smart Wearable
Technology.
Project Conceptualization
and Construction
DESIGN PHASE

Inspiration of the Design vs Initial Design

DEVELOPMENT PHASE

This phase involved programming, hardware

development, and integration of sensors and
software components.
TESTING PHASE
IMA involved a comprehensive and
systematic approach to ensure its accuracy,
calibration, and user acceptability.
EVALUATION PHASE
DATA GATHERING PROCEDURE
 DATA GATHERING PROCEDURE

As the user performs sign language gestures, the data of the sensors incorporated
to IMA was recorded.
DATA GATHERING PROCEDURE
 DATA GATHERING PROCEDURE

Survey questionnaire served as the primary research instrument to assess the

acceptability and satisfaction of the IMA among users, particularly deaf and mute
individuals.
DATA GATHERING PROCEDURE
DATA GATHERING PROCEDURE
DATA GATHERING PROCEDURE
DATA GATHERING PROCEDURE
DATA ANALYSIS

Statistical Range Descriptive Interpretation

1.0 – 1.5 Very Unacceptable

1.51 – 2.5 Somewhat Unacceptable

2.51 – 3.5 Neutral

3.51 – 4.5 Somewhat Acceptable

4.51 – 5.0 Very Acceptable

RESULTS AND DISCUSSION
RESULTS AND DISCUSSION
Gesture Recognized

Frequency Percentage
A 50 100%
B 50 100%
C 50 100%
D 50 100%
E 50 100%
F 47 94%
G 49 98%
H 46 92%
I 45 90%
J 45 90%
K 46 92%
L 45 90%
M 44 88%
N 43 86%
 H 46 92%
I 45 90%
J 45 90%
RESULTS AND DISCUSSION
K
L
46
45
92%
90%
M 44 88%
N 43 86%
O 43 86%
P 49 98%
Q 48 96%
R 44 88%
S 41 82%
T 44 88%
U 46 92%
V 49 98%
W 44 88%
X 49 98%
Y 40 80%
Z 40 80%
Accuracy 92.08%
 RESULTS AND DISCUSSION
Gesture Recognized Not Recognized

Frequency Percentage Frequency Percentage

Welcome 50 100% 0 0%
Yes 40 80% 10 20%
No 41 82% 9 18%
We 48 96% 2 4%
Hello 50 100% 0 0%
Hungry 45 90% 5 10%
I love you 44 88% 6 12%
50 100% 0 0%
Thank you
Accuracy 92%
 Degree of Acceptability of IMA in terms of Usage
Experience, Accuracy and Functionality
Criteria Mean Descriptive Interpretation

1. Ease of use 4.73 Very Acceptable

2. IMA interpret FSL gestures 3.87 Somewhat Acceptable
into readable text messages

3. IMA translate FSL gestures 3.73 Somewhat Acceptable

into recognizable voice audio

Overall Mean 4.11 Somewhat Acceptable

Degree of Acceptability of IMA in terms of User
Interface and Comfort
Criteria Mean Descriptive Interpretation

1. User interface design of 4.47 Somewhat Acceptable

IMA

2. IMA is comfortable to wear 4.27 Somewhat Acceptable

for extended periods

Overall Mean 4.37 Somewhat Acceptable

Overall Degree of Acceptability of IMA
Criteria Mean Descriptive Interpretation
Usage Experience, 4.11 Somewhat Acceptable
Accuracy and
Functionality
User Interface and 4.37 Somewhat Acceptable
Comfort
How likely are you to 4.6 Very Acceptable
recommend IMA to others

Overall Satisfaction 4.36 Somewhat Acceptable

Score
Summary
CONCLUSION
 The design and implementation
of the assistive device, IMA, have
addressed the need for a tool that
interprets hand sign languages
into readable text messages and
recognizable voice audio.

01
Through testing and
experimentation, IMA has
demonstrated its capability to
facilitate communication for
individuals with hearing/speech
impairment.
 Following a calibration process,
the accuracy of the IMA has been
determined. The system achieves
an overall accuracy rate of
approximately 92.04% in
translating both letter and word
gestures into text and speech.

02
While certain gestures exhibit
lower recognition rates,
particularly for letters S, Y, and
Z, the results indicate a high level
of precision in the system's
recognition capabilities.
 Surveys conducted among users,
including students from schools with
Special Education programs, have
provided valuable insights into the
usability, accuracy, and functionality of
the IMA. Additionally, respondents'
preferences for using the IMA in
educational environments and police
stations underscore its potential to

03
facilitate communication in various
settings and scenarios. The overall
satisfaction score, calculated at 4.36,
indicates a high level of acceptance and
endorsement of the project within the
target population, positioning it as
‘somewhat acceptable’ based on the
users' feedback.
Recommendations
1. expand the predefined vocabulary and linguistic scope beyond the
basic communication tasks currently facilitated by the IMA. This could
involve incorporating a wider range of words, phrases, and expressions
commonly used in everyday communication among individuals who are
deaf or hard of hearing.
2. develop comprehensive user training materials and educational
resources to familiarize users with the functionality and operation of the
IMA.
3. utilize Bluetooth connectivity, improve contact sensors for the dual-
glove system to address the need for gestures that involve the use of
both hands in Filipino Sign Language (FSL)
4. utilize a higher memory microcontroller unit aside from Arduino
nano 33 ble sense rev2, Raspberry pi 3 to incorporate machine
learning into the dual-glove system.