Proceeding Paper

An IoT Braille Display towards Assisting Visually Impaired

Students in Mexico †
Oscar I. Ramos-García 1 , Anuar A. Vuelvas-Alvarado 1 , Néstor A. Osorio-Pérez 1 , Miguel Á. Ruiz-Torres 1 ,
Fermín Estrada-González 1 , Laura S. Gaytan-Lugo 2 , Silvia B. Fajardo-Flores 1 and Pedro C. Santana-Mancilla 1, *

1 School of Telematics, Universidad de Colima, Colima 28040, Mexico

2 School of Mechanical and Electrical Engineering, Universidad de Colima, Coquimatlán 28400, Mexico
* Correspondence: [email protected]
† Presented at the 9th International Electronic Conference on Sensors and Applications, 1–15 November 2022;
Available online: https://ecsa-9.sciforum.net/.

Abstract: According to the World Health Organization, 2.2 billion people globally have some vision
impairment. Blind and vision impairment children can undergo poor motor, language, and cognitive
evolution, bringing lower levels of educational success. Our proposal aims to design and develop a
one-character refreshable braille display that is affordable and easy to use through the Internet of
Things (IoT) technology. Reading is essential to acquire knowledge by allowing an affordable form of
reading based on braille, a handy tool for teaching and training blind and visually impaired people
can be reached.

Keywords: assistive technologies; braille cell; visual impairment; accessibility; internet of things;
haptic interface

Citation: Ramos-García, O.I.;

Vuelvas-Alvarado, A.A.;
Osorio-Pérez, N.A.; Ruiz-Torres, 1. Introduction
M.Á.; Estrada-González, F.; According to the World Health Organization, 2.2 billion people globally have some
Gaytan-Lugo, L.S.; Fajardo-Flores,
vision impairment [1]. The impairment prevalence in low- and middle-income areas, like
S.B.; Santana-Mancilla, P.C. An IoT
Mexico and Latin America, is estimated to be 400% higher than in high-income regions. In
Braille Display towards Assisting
Mexico, 8.97 million people were identified with some visual impairment in 2020 [2]. In
Visually Impaired Students in
addition, the blind and vision impairment population faces severe social and academic ex-
Mexico. Eng. Proc. 2022, 27, 11.
clusion. For instance, children can undergo poor motor, language, and cognitive evolution,
https://doi.org/10.3390/
ecsa-9-13194
bringing lower levels of educational success.
Braille script represents a language’s characters in a rectangular arrangement of raised
Academic Editor: Francisco Falcone dots so that blind people can read the symbols and obtain written knowledge through
Published: 1 November 2022
the touch sense. Braille characters are generally printed in books or on commonly used
surfaces, such as ATMs. However, printing is resource intensive as it needs to be done in
Publisher’s Note: MDPI stays neutral each case. For this reason, electronic Braille displays were developed, which connect to
with regard to jurisdictional claims in
a computer or smartphone and provide a tactile representation of the text displayed on
published maps and institutional affil-
the screen. Unfortunately, accessing this technology is difficult for people in developing
iations.
countries because it is very costly.
Hence, this paper describes the design and development of a refreshable single-
character Braille display that is affordable and easy to use through the Internet of Things
Copyright: © 2022 by the authors.
(IoT) technology. Reading is essential to acquire knowledge by allowing an affordable
Licensee MDPI, Basel, Switzerland. form of reading based on braille, a handy tool for teaching and training blind and visually
This article is an open access article impaired people can be reached.
distributed under the terms and
conditions of the Creative Commons
2. Braille Displays as Assistive Technology
Attribution (CC BY) license (https:// Screen readers are now gaining ground as assistive technology for blind and low-
creativecommons.org/licenses/by/ vision people as they are natively integrated into modern desktop and mobile operating
4.0/). systems. Although Braille displays (also known as Braille lines) have been around for many

Eng. Proc. 2022, 27, 11. https://doi.org/10.3390/ecsa-9-13194 https://www.mdpi.com/journal/engproc

Eng. Proc. 2022, 27, 11 2 of 5

2. Braille Displays as Assistive Technology

Eng. Proc. 2022, 27, 11 Screen readers are now gaining ground as assistive technology for blind and low- 2 of 5
vision people as they are natively integrated into modern desktop and mobile operating
systems. Although Braille displays (also known as Braille lines) have been around for
many years, their advantages over screen readers are that they allow for more significant
years, their advantages over screen readers are that they allow for more significant user
user interaction [3] and better usability [4].
interaction [3] and better usability [4].
A wide variety of Braille displays for visual impairment are available, but due to cost
A wide variety of Braille displays for visual impairment are available, but due to
reasons, they are mostly accessible only to the high-income population in developed coun-
cost reasons, they are mostly accessible only to the high-income population in developed
tries. Attempts to develop Braille displays date back decades of research, with the main
countries. Attempts to develop Braille displays date back decades of research, with the
limitation being the
main limitation actuator
being technology
the actuator that can
technology becan
that used
betoused
interact with the
to interact user’s
with hap-
the user’s
tics. They have experimented with solenoids, shape memory alloys (SMAs),
haptics. They have experimented with solenoids, shape memory alloys (SMAs), ferrofluids, ferrofluids,
Micro-Electromechanical
Micro-Electromechanical SystemsSystems (MEMs), flexible polymers,
(MEMs), flexible polymers,andandother
othermaterials
materials[2–7].
[2–7].In
Inallall
cases, the main challenges are the weight, volume, energy consumption, and highhigh
cases, the main challenges are the weight, volume, energy consumption, and costs
costs
of the ofdevices
the devices developed.
developed.
AArefreshable
refreshableBraille
Braillecell is is
cell proposed
proposed to address these
to address challenges
these and and
challenges develop a de-a
develop
vice suitable for the local context, this research designed the device using push-pull
device suitable for the local context, this research designed the device using push-pull elec-
tromagnetic
electromagneticsolenoids connected
solenoids to an ESP-32
connected microcontroller
to an ESP-32 board. board.
microcontroller Each solenoid, when
Each solenoid,
moved vertically, represents a braille dot that blind people can read from the
when moved vertically, represents a braille dot that blind people can read from the device device by
touching.
by touching.

3.3.Methods
Methods
Themethodology
The methodologyused usedininthis
thisproject
projectwas
wasSCRUM
SCRUM[5], [5],which
whichuses
usesan anagile
agileapproach
approach
totodevelop
developsystems
systemsbased
basedon onsoftware
softwareandandhardware.
hardware.ItItisisbased
basedon onrapid
rapiditerations
iterationswith
with
specific objectives. This methodology was selected due to its flexibility to generate
specific objectives. This methodology was selected due to its flexibility to generate con- constant
improvements
stant improvements sincesince
it seeks to provide
it seeks functional
to provide prototypes
functional prototypesin short periods.
in short periods.

3.1.System
3.1. SystemArchitecture
Architecture
Figure11illustrates
Figure illustratesthe
theIoT
IoTsystem
systemarchitecture
architecturefor
forthe
theBraille
Braillecell.
cell.

Figure
Figure1.1.The
TheIoT
IoTsystem
systemarchitecture
architecturefor
forthe
thebraille
braillecell.
cell.

The
Thearchitecture
architecture was designed based
was designed basedon onthetheedge
edge computing
computing paradigm
paradigm [6,7],
[6,7], ena-
enabling
bling near real-time communication close to where the data is generated, that
near real-time communication close to where the data is generated, that is, the edge of the is, the edge
ofInternet
the Internet [8]. Avoiding
[8]. Avoiding the steps
the steps of sending
of sending information
information from an from an internet-connected
internet-connected device
device
to the to the cloud,
cloud, from which
from which the Braille
the Braille cell read
cell must musttheread the information,
information, which in
which results results
faster
incommunication
faster communication
betweenbetween
softwaresoftware and hardware.
and hardware.
Communication
Communication between
between the device
device and
andthethecontents
contentstotobebedisplayed
displayed in in
thethe Braille
Braille cell
cell is done
is done through
through the the Message
Message Queuing
Queuing Telemetry
Telemetry Transport
Transport (MQTT) (MQTT) protocol,
protocol, a well-
a well-known
known IoT messaging
IoT messaging protocolprotocol
that wasthat was
used toused to transmit
transmit the text
the digital digital
andtext and convert
convert it to
it to physical
braille text.
physical MQTT
braille text. uses
MQTT a publish and subscribe
uses a publish model,model,
and subscribe which which
typically operates
typically with a
operates
small network overhead. This attribute makes it flawless for operating on microcontrollers
in edge computing applications that demand high-speed data communication between
devices over a wireless local area network (WLAN).
Eng. Proc. 2022, 27, 11 3 of 5

with a small network overhead. This attribute makes it flawless for operating on micro-
Eng. Proc. 2022, 27, 11 3 of 5
controllers in edge computing applications that demand high-speed data communication
between devices over a wireless local area network (WLAN).

3.2.
3.2. System
System Design
Design
Figure
Figure22shows
showsthe prototype
the prototype hardware.
hardware.TheThe
central elements
central are the
elements digital
are processing
the digital pro-
and control
cessing andphases implemented
control within an ESP32-WROOM
phases implemented microcontroller
within an ESP32-WROOM board. This
microcontroller
device
board.wasThisselected
device because it has because
was selected integrated Wi-Fi
it has (802.11 b/g/n
integrated Wi-Fiup to 150b/g/n
(802.11 Mbps),upa to
32-bit
150
dual-core Xtensa LX6 processor operating at 240 MHz, 4 MB of Flash memory,
Mbps), a 32-bit dual-core Xtensa LX6 processor operating at 240 MHz, 4 MB of Flash and 520 KB
of SRAM, and digital terminals. This is sufficient for controlling the actuators that
memory, and 520 KB of SRAM, and digital terminals. This is sufficient for controlling the produce
the sequences
actuators that of the Braille
produce symbols. of the Braille symbols.
the sequences

Figure 2.
Figure 2. Prototype
Prototype schematics
schematics diagram.
diagram.

Six
Sixsolenoids
solenoidswere
wereused
used toto
display
displaythethe
tactile symbols
tactile symbolson the
on matrix, which
the matrix, due to
which their
due to
nature, operate
their nature, with higher
operate voltagevoltage
with higher and current than the
and current digital
than system system
the digital can supply. Hence,
can supply.
an external
Hence, power source
an external powercapable
source of providing
capable 5 V and a5 current
of providing V and aofcurrent
9 ampsofis9required.
amps is re-
The
quired. schematic diagram presents two blocks separated according to their operating
voltage
Theand current:diagram
schematic one lowpresents
power and twoone highseparated
blocks current. As seen in to
according these
theirblocks, the
operating
digital
voltagecontrol terminals
and current: onearelow
isolated
powerwith
andsix optocouplers
one high current. to ensure
As seen theinblock’s protection
these blocks, the
from discharges up to 5 kV, while the high current fed into each solenoid is
digital control terminals are isolated with six optocouplers to ensure the block’s protection controlled
independently
from discharges byup
activated
to 5 kV, relays
whilethrough
the highthecurrent
outputfed
of the
intocorresponding
each solenoidoptocoupler.
is controlled
independently by activated relays through the output of the corresponding optocoupler.
4. Results
Figure 3 shows the prototype developed in this iteration of the methodology. As can
4. Results
be seen, the structure was built with wood since it represents a lower cost than 3D printing
Figure 3 shows the prototype developed in this iteration of the methodology. As can
and is durable and resistant.
be seen, the structure was built with wood since it represents a lower cost than 3D printing
At the top is the cell in the form of a 2 × 3 matrix, whose pins are visible when they
and is durable and resistant.
are in the active state, representing a Braille character.

Testing
The effectiveness of the prototype was tested using the battery of tests proposed in [9]:
• In-sequence-character output accuracy test.
• Random-character output accuracy test.
• Correct display of pins during reading mode.
• All tests were passed successfully.
Eng. Proc. 2022, 27, 11 4 of 5
Eng. Proc. 2022, 27, 11 4 of 5

Figure 3. Braille cell prototype.

Figure 3. Braille cell prototype.

5. Discussion
At the top is the cell in the form of a 2 × 3 matrix, whose pins are visible when they
are in the active
During the state, representing
first iterations, thea control
Braille character.
system maintained the active state of the
solenoids to represent the Braille letters indefinitely. However, over time, excessive power
Testing
was found to be dissipated by these devices in the form of heat, which means high power
consumption and risk ofofdamage,
The effectiveness including
the prototype wasfire. To address
tested these
using the challenges,
battery of teststhe solenoids
proposed in
were
[9]: programmed to be activated for only four seconds, enough time for each of the
symbols
• to be interpreted and
In-sequence-character keep accuracy
output them inactive
test. the rest of the time.
• The
Random-character output accuracy test. of the few options available in Mexico, as it
solenoids used in the project were one
was impossible to obtain others with more desirable characteristics, such as low operating
• Correct display of pins during reading mode.
voltage, a higher impedance of the inductor, and even smaller physical dimensions to
• All tests were passed successfully.
organize them for the size of a single fingerprint. Due to this, in the project’s next iteration,
various hardware components that can achieve the miniaturization of the Braille cell and
5. Discussion
reduce energy consumption are analyzed.
During the first iterations, the control system maintained the active state of the sole-
6.noids to represent the Braille letters indefinitely. However, over time, excessive power
Conclusions
wasWith
foundthe to development
be dissipated of bythis
these devices
work in the form
in progress, of heat,
it was which means
demonstrated thathigh power
creating a
consumption and risk of damage, including fire. To address these
Braille cell with accessible costs for the local context is possible. The following stages challenges, the sole-
of
noids
the were
project programmed
will to be activated
consist of reducing for onlysize
the prototype’s four seconds,
and carryingenough time
out tests withforusers
eachtoof
the symbols
validate to be interpreted
its usability, usefulness, and
andkeep them inactive the rest of the time.
efficacy.
The solenoids used in the project were one of the few options available in Mexico, as
it was Contributions:
Author impossible to obtain others withP.C.S.-M.,
Conceptualization, more desirable
L.S.G.-L.characteristics, such as low
and S.B.F.-F.; methodology, operat-
P.C.S.-M.;
ing voltage,
software, a higher
O.I.R.-G., impedance
A.A.V.-A., of the
N.A.O.-P. andinductor,
M.Á.R.-T.; and even smaller
validation, physical
F.E.-G., dimensions
L.S.G.-L. to
and S.B.F.-F.;
formal
organizeanalysis,
themP.C.S.-M. andof
for the size F.E.-G.; investigation,
a single fingerprint.O.I.R.-G., A.A.V.-A.,
Due to this, in theN.A.O.-P.
project’s and
nextM.Á.R.-T.;
iteration,
writing—original
various hardware draftcomponents
preparation, P.C.S.-M.
that can and F.E.-G.;
achieve thewriting—review
miniaturization and ofediting, L.S.G.-L.
the Braille cell and
and
S.B.F.-F.; project administration, P.C.S.-M.
reduce energy consumption are analyzed. All authors have read and agreed to the published version
of the manuscript.
6. Conclusions
Funding: This research was partially funded by the Human-Computer Interaction Lab (IHCLab) of
the School
Withofthe
Telematics at the University
development of Colima
of this work in Mexico.
in progress, it was demonstrated that creating a
Braille cell with
Institutional accessible
Review costs forNot
Board Statement: theapplicable.
local context is possible. The following stages of
Eng. Proc. 2022, 27, 11 5 of 5

Informed Consent Statement: Not applicable.

Data Availability Statement: Not applicable.
Acknowledgments: The authors thank Esli Castellanos for his advice on developing the embedded
software of the prototype.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. WHO. Blindness and Vision Impairment; World Health Organization: Geneva, Switzerland, 2021.
2. INEGI. Población Con Limitación En La Actividad Para Ver (Personas); México en Cifras; Instituto Nacional de Estadística y Geografía:
Aguascalientes, Mexico, 2020.
3. Hatzfeld, C.; Khoury, M.E.; Nakic, C.; Staab, M.; Winterstein, T.; Schlaak, H.F. Electrothermal Actuators for Portable Braille
Devices. In the Proceedings of Eurohaptics 2016; Springer: London, UK, 2016.
4. Russomanno, A.; O’Modhrain, S.; Gillespie, R.B.; Rodger, M.W.M. Refreshing Refreshable Braille Displays. IEEE Trans. Haptics
2015, 8, 287–297. [CrossRef]
5. Mariño, S.I.; Alfonzo, P.L. Implementación de SCRUM En El Diseño Del Proyecto Del Trabajo Final de Aplicación. Sci. Et Tech.
2014, 19, 413–418.
6. Cao, K.; Liu, Y.; Meng, G.; Sun, Q. An Overview on Edge Computing Research. IEEE Access 2020, 8, 85714–85728. [CrossRef]
7. Shi, W.; Cao, J.; Zhang, Q.; Li, Y.; Xu, L. Edge Computing: Vision and Challenges. IEEE Internet Things J. 2016, 3, 637–646.
[CrossRef]
8. Santana-Mancilla, P.C.; Anido-Rifón, L.E.; Contreras-Castillo, J.; Buenrostro-Mariscal, R. Heuristic Evaluation of an IoMT System
for Remote Health Monitoring in Senior Care. Int. J. Environ. Res. Public Health 2020, 17, 1586. [CrossRef] [PubMed]
9. Garcillanosa, M.M.; Apuyan, K.N.T.; Arro, A.M.; Ascan, G.G. Audio-Assisted Standalone Microcontroller-Based Braille System
Tutor for Grade 1 Braille Symbols. In Proceedings of the 2016 IEEE Advanced Information Management, Communicates,
Electronic and Automation Control Conference (IMCEC), Xi’an, China, 3–5 October 2016; pp. 439–442.