Assistive Navigation Application for Blind People

using a White Cane Embedded System

Adrian Mocanu Valentin Sita Camelia Avram
Automation Department Automation Department Automation Department
Technical University of Cluj Napoca Technical University of Cluj Napoca Technical University of Cluj Napoca
Cluj Napoca, Romania Cluj Napoca, Romania Cluj Napoca, Romania
[email protected] [email protected] [email protected]

Dan Radu Adina Aştilean

Automation Department Automation Department
Technical University of Cluj Napoca Technical University of Cluj Napoca
Cluj Napoca, Romania Cluj Napoca, Romania
[email protected] [email protected]

Abstract—The need to move independently is one of the most increased cost at individual level, but also could require a
important factors conditioning an active life. A relative reduced minimum level of knowledge and an increased effort, leading
number of devices and applications proved to have a real utility sometimes to really challenging situations the user must face.
in this field, many of them presenting limitations and requiring
improvements. The present paper proposes a system capable to In the case of persons suffering from a specified type of
assist impaired visual people to travel independently in smart disability, the support devices created to increase their quality
cities. It is based on the enhancing of white canes with of life are designed to replace the absent perception sense with
capabilities to read and interpret codes of colors special created another, correspondingly modifying the information
to express previously elaborated routes for a given area. A originating in the surrounding environment. The developing
decision part of the system compares the translated route with strategies of smart cities include many facilities offered to
the real trial detected by sensors and transmits to the blind impaired people, giving them a particular attention. The
person guiding and warning coded tactile signals. The main integration of new infrastructure elements and solutions
functionalities of the system were modeled and tested in a offered by internet of Things have the needed potential to
laboratory environment, it proving to be reliable and easy to increase the quality of life for this category of persons.
use.
II. STATE OF THE ART
Keywords—Impaired visual people, assistive technology, white
cane, color sensor, Raspberry Pi The consequences of vision loss are similar, even if,
depending on many factors, there is a variety of causes leading
I. INTRODUCTION to visual impairments and the degree of visual acuity loss
differ. The impossibility or reduced capability to distinguish
Blindness and vision loss can be caused by a number of
images and colors, to read, to have quick and easy access to
conditions and diseases, as well as accidents. Recent statistics
modern technologies and devices, deeply affect the daily life
show that worldwide there are around 253 million persons
and activities of this category of people. The negative impact
having visual impairments [1] and about 2.2 billion people
is more than evident when visual impaired people make a
suffer from eye conditions [2]. Although the modality in
career choice, participate to social activities, navigate outside
which these persons interacts with the environment could be
and inside [3], and in many others situations. The result has a
influenced by different circumstances including total or
considerable influence on their future evolution and emotional
partial, progressive or accidental, acquired or from birth vision
health.
loss, the main issues facing by this category of population are
similar. Consequently, to a greater or lesser extent, there is the From all the above enumerated aspects, the need to move
need to ensure assistance during their entire daily activities. independently is one of the most important factors
An important category of the raised issues is connected with conditioning an active life. An important cause of the almost
the necessity to travel on different distances, inside (where unsurmountable difficulties confronting unassisted visual
they can memorize the obstacles) or outside, following known impaired persons in new, unknown places is their incapacity
or new routes. The need of continuous orientation, keeping or to take a decision for a correct orientation, to appropriately
changing the direction to successfully reach different change the direction based on a continuously, renewed
destinations, the avoidance of various obstacles and situations information about current localizations. Even in the case of
which can lead to unexpected accidents are only a few of the walking in places where the fixed landmarks are preserved,
specific challenges which are still awaiting new solutions with the presence of unexpected, occasional, obstacles, as well as
better performances. If some years ago there were a few any changes including object positions and characteristics,
options to help the vision impaired people during daily could become a source of accidents.
activities, most of them including the human factor (a sighted
person for guide them), the rapid registered technological During the last years, besides the classical solutions
progress lead not only to the increasing of the degree of utility implying guide dogs, sighted volunteers and simple white
of older devices, but also to the possibility to develop new canes, more sophisticated devices and special structures
facilities. However. it must be mentioned that the complexity embedded in environment, based on the last technologies,
of the solutions based on advanced technologies, capable to were proposed for a more efficient support offered to this
bring important supplementary benefits, implies not only an category of persons. In [4], analyzing the responses given in a
special created questionnaire, the authors highlight the

978-1-7281-7166-1/20/$31.00©2020 European Union

Authorized licensed use limited to: University of Southern Queensland. Downloaded on July 03,2020 at 17:24:43 UTC from IEEE Xplore. Restrictions apply.
importance of facilities and assisting tools made available for marking the end of the streets which meet in the considered
visual impaired people. The effectiveness of tactile ground point, represent indicators for all possible changes of the route.
surfaces in parallel with their disadvantages are discussed in An example is given in Fig.1, representing two connected
[5]. Wireless communications, including applications of crossroads mentioned by their identity (CR-Id), and four travel
mobile telephony and GPS, underpin smarter and sustainable directions marked by brown, blue, orange and green colors.
systems or category of devices proposed in the field. In this The different colored portions of street segments have
direction, a lot of examples can be given, only some, very few correspondingly designed dimensions to be easily detected by
of them being presented in [6], [7], [8], and [9] research the cane. A reduced superposition area of two directions is
papers. A special place is occupied by assistive systems using represented by a supplementary distinct color to allow the
image processing. These can detect obstacles and sometimes users to return to a reference point if they fail to follow the
recognize objects, then notifying impaired visual people by indicated right way. A decision component of the system will
different means [10], [11], [12]. Other possible categories of permanently analyze the signals received from the color
solutions, based on the actual technologies and promising sensor. Based on both stored and continuously received
present and future utilities, are analyzed in [13]. Kahn et al., information and applying a set of rules, this component will
[14] highlight the advantages resulting from the development generate and send warning and guiding tactile messages to the
of the traditional guiding navigation devices, referring user. For example, warning signals will be transmitted when
especially to the white cane, considered as “a symbol of the user leave for a specified time interval the colored segment
identification for blind people”. between two crossroads, being in the incapacity to maintain
the direction, or when the succession of colors detected when
Despite the great number of proposals and published the crossroad is passed doesn’t correspond with the
research papers it is obviously that the number of devices memorized predetermined order, indicating the necessary
having a real utility remains relative reduced, many change of direction.
applications presenting limitations requiring improvements.
Taking into account the above considerations, the present It must be mentioned that for a predetermined distance, the
paper proposes a system capable to assist impaired visual sidewalks entering in a crossroad will be painted and
people to travel independently in smart cities. It is based on represented with different colors in order to facilitate a simple
the enhancing of white canes with capabilities to read and description of directions changing. Supposing that the long
interpret codes of colors special created to express previously segments are all painted in yellow, and four, the maximum
elaborated routes for a given area. A decision part of the number of terminal segments of the sidewalk before the
system compares the translated route with the real trial crossroad are red, blue, orange and green, a possible change
detected by sensors and transmits to the blind person guiding of direction for a given route will be described on the
and warning coded tactile signals. generated map as the following succession: 30 yellow, 1 red,
1 blue, 75 green. Fig. 1 better illustrates this representation
III. SYSTEM DESCRIPTION rule. A reduced superposition area of two directions is
The proposed system has three main parts: represented by a supplementary distinct color to allow the
users to return to a reference point if they fail to follow the
• A dedicated city infrastructure composed of colored paint indicated right way. A decision component of the system will
bands, the colors being different in the case of the long permanently analyze the signals received from the color
segments of the sidewalks and of the short segments sensor. Based on both stored and continuously received
preceding the crossroads; information and applying a set of rules, this component will
• A software component, used to establish the route and to generate and send warning and guiding tactile messages to the
guide and monitor the position of the blind user taking into user. For example, warning signals will be transmitted when
account the colors recognition; the user leave for a specified time interval the colored segment
• An enhanced white cane capable to follow the painted between two crossroads, being in the incapacity to maintain
segments and to signalize the change of colors, transforming the direction, or when the succession of colors detected when
them in different interpretable messages understandable for the crossroad is passed doesn’t correspond with the
memorized predetermined order, indicating the necessary
blind people.
change of direction.
A short description of the role and existing connections
among these parts is given below.
The user communicates to the guiding system the source
and the destination of his next travel, utilizing an existing
device adapted for blind people. The information is
transmitted to the computing system which, based on the
memorized colored streets map of the city, will generate the
route. Unlike the usual routes described using the street
names, this particular one will contain numbered and colored
street segments and crossroads, enumerated in the
corresponding succession. So, for a predetermined distance,
the sidewalks entering in a crossroad will be painted and
represented with different colors in order to facilitate a simple Fig.1. Generic representation of infrastructure
description of directions changing. A possible scenario is that
in which the long street segments are painted in yellow. Red The long street segments are painted in yellow. Red is the
is the color chosen to represent the entrance segment to the color chosen to represent the entrance segment to the
crossroad, while other different colored short segments, crossroad, while other different colored short segments,
marking the end of the streets which meet in the considered

Authorized licensed use limited to: University of Southern Queensland. Downloaded on July 03,2020 at 17:24:43 UTC from IEEE Xplore. Restrictions apply.
point, represent indicators for all possible changes of the route. Where:
An example is given in Fig.1, representing two connected
crossroads mentioned by their identity (CR-Id), and four travel P1 – a pedestrian approaching; T1 – determine (calculate the
directions marked by brown, blue, orange and green colors. colored dashes from the pathway) the path; P2 – colored
The different colored portions of street segments have dashes calculated; T2 – entering on the crossroad; T3, T5, T7
correspondingly designed dimensions to be easily detected by and T12 – exits from the crossroad (possible paths to follow);
the cane. P4, P5, P6 and P7 – the pedestrian is exited from the crossroad
following one of the four possibilities; T4, T6, T8 and T13 –
A comprehensive description of the system behavior is walking on the next pathway segments; In case of taking the
given by the Petri Nets based model, represented in Fig. 2. wrong exit on P4, P5, P6 or P7 there is a possibility to return
to the crossroad by firing one of T9, T10, T11, T14 transitions.

Fig.2. The Petri nets based model for selecting the path near a
crossroad
Where:
P0 – the stored trajectory and will enable only one transition to
be fired. P1, P2, P3, P4, P5, P6, P7 and P8 – color detection. P10,
P11, P12 and P13 – output color detected. P9 – the pedestrian is
on the intersection. T1, T2, T3 and T4 – entering on the
intersection. T5, T6, T7 and T8 – exits from the intersection.
A detailed description of a crossroad is presented in Fig 3.

Fig. 4. The experimental device

Fig. 3. The Petri nets based model of a crossroad Fig. 5. The electronic components of box “A”

Authorized licensed use limited to: University of Southern Queensland. Downloaded on July 03,2020 at 17:24:43 UTC from IEEE Xplore. Restrictions apply.
 Compared with the tactile ground surfaces, the proposed In Fig. 7 is presented a sample of how the results are
solution doesn’t require any real modification of represented on a graph. The sensor detects a color and sends
infrastructure, is cost effective and easy to maintain. From the an impulse to the development board on the input pin (wired
point of view of blind users it can offer an increased safety and connection), the signal is roughly filtered locally and then sent
speed during their travel, being thought, by design, free of
static obstacles. The blind users of the system are secured
against the obstacles by using both a dedicated path and the
white cane the guiding role being reserved for the embedded
software component.
IV. SYSTEM DEVELOPMENT
For experimental purposes an experimental stick was
developed, presented in Fig. 4, which include two elements
placed on two sides. Upper side (close to handler, noted with
letter “A”, electronic components, placed in a box “B”, for
reading and interpreting the signals from sensors and taking
decisions to guide the impaired person on the street, an
ultrasonic sensor “C”, connected with a cable “D”) and lower
side of the stick (the box “E” with the color sensor “F”).
Fig. 5 presents the electronic components of the device, it
contains the development board “A” powered with
rechargeable batteries “B”, relay module “C”, speaker “D”,
vibration motor “E” and wireless communication module “F”.
The ultrasonic sensor “H” and the color sensor “G” are wired
connected to the development board.
For this project a development board using ATmega2560
microcontroller and a 16 MHz crystal oscillator were chose.
This microcontroller can be used for more complex
projects where multiple sensors and actuators are installed.
The development board contain a USB connection for
programming, a power jack for connection to a power supply
or battery, has 39 digital input/output pins, 15 PWM output
pins that can be also used as digital input/output pins, 16
analog input pins and 4 serial ports. For powering the
development board we used a 5V 8000mA rechargeable
battery pack that contains a micro USB connector for
recharging that can be easily be recharged from any USB from
a PC or power supply with USB socket and two USB male A
for supplying consumers.
As a sensing element for direction a small DC motor vibrator
was used. This motor is also utilized in cell phones powered
at 5V DC through a relay board also with 5V DC relays. For
on/off the device we used also a speaker as a second Fig. 6. The developed device
acknowledgement element. The communication with the PC
can be established using three methods: USB wired to PC. Normally the colors are detected as impulses and then
connection or wireless with ESP8266 WiFi module or on the PC are extrapolated resulting a colored lines.
Sparkfun Bluetooth Mate Gold. To detect the obstacles and to
avoid them it is used the HC-SR04 ultrasonic module. Color VI. CONCLUSIONS
detection is realized with TCS230 sensor. A costly effective application to offer support and
In Fig. 6 is presented the developed device with the independency to visual impaired people during their travel in
displacement of the electronic components, the box being new, unknown places was design and implemented. The core
placed near the handler for easy use of controls. element of the designed system is an enhanced white cane. As
a whole, the assembly application requires a minimum of non-
V. SYSTEM TESTING essential modifications, concretized as sidewalks painted
The tests and verifications were accomplished in laboratory bands in the targeted area. The main functionalities of the
environment. The signals read from sensors were filtered and system were modeled and tested in a laboratory environment
stored on a PC. All the communications between the device it proving to be reliable and easy to use. For further
and PC were realized using Ethernet. The device has access to improvements, others tests, accomplished with the
internet using WiFi and communicates through sockets with a participation of many categories of visual impaired people,
server installed on the PC. The server store the measurements affected by different degrees of vision loss, are planned in the
on a local database. The measurements are used later for future.
analysis purposes.

Authorized licensed use limited to: University of Southern Queensland. Downloaded on July 03,2020 at 17:24:43 UTC from IEEE Xplore. Restrictions apply.
 Fig. 7. The representation of the signal sent by the device

REFERENCES [8] Jafri R., Ali S.A. (2014) A GPS-Based Personalized Pedestrian Route
Recording Smartphone Application for the Blind. In: Stephanidis C.
[1] Ackland P, Resnikoff S, Bourne R. World blindness and visual (eds) HCI International 2014 - Posters’ Extended Abstracts. HCI 2014.
impairment: despite many successes, the problem is growing. Communications in Computer and Information Science, vol 435.
Community Eye Health. 2017; 30:71–73. Springer, Cham
[2] Haile Fentahun Darge, Getahun Shibru, Abiy Mulugeta, and Yinebeb [9] Brilhault A, Kammoun S, Gutierrez O, Truillet P, Jouffrais C (2011)
Mezgebu Dagnachew, The Prevalence of Visual Acuity Impairment Fusion of artificial vision and GPS to improve blind pedestrian
among School Children at Arada Subcity Primary Schools in Addis positioning. International conference on new technologies, mobility
Ababa, Ethiopia, Journal of Ophthalmology, Volume 2017, Article ID and security, IEEE, France
9326108, 7 pages, https://doi.org/10.1155/2017/9326108 [10] Jain, B & Thakur, Shwetha & Suresh, K. (2018). Visual Assistance for
[3] AbbasRiazi, Fatemeh Riazi, RezvanYoosfi, FatemehBahmee, Outdoor Blind Using Image Processing. 0499-0503.
difficulties experienced by a group of visually impaired Iranian people, 10.1109/ICCSP.2018.8524251
Volume 28, Issue 2, June 2016, Pages 85-90 [11] Hotaka Takizawa and Mayumi Aoyagi (December 20th 2017).
[4] Zeng, L., (2015). A Survey: Outdoor Mobility Experiences by the Assistive Systems for the Visually Impaired Based on Image
Visually Impaired. In: Weisbecker, A., Burmester, M. & Schmidt, A. Processing, Causes and Coping with Visual Impairment and Blindness,
(Hrsg.), Mensch und Computer 2015 – Workshopband. Berlin: De Shimon Rumelt, IntechOpen, DOI: 11.5772/intechopen.70679
Gruyter Oldenbourg. (S. 391-397). [12] Rodríguez A, Yebes JJ, Alcantarilla PF, Bergasa LM, Almazán J, Cela
[5] A. Pembuain, S. Priyanto and L.B. Suparma, The evaluation of tactile A. Assisting the visually impaired: obstacle detection and warning
ground surface indicator condition and effectiveness on the sidewalk in system by acoustic feedback. Sensors (Basel). 2012;12(12):17476–
Yogyakarta City, Indonesia, IATSS Research, 17496. Published 2012 Dec 17. doi:10.3390/s121217476
https://doi.org/10.1016/j.iatssr.2019.04.002 [13] Ádám Csapó, György Wersényi, Hunor Nagy and Tony Stockman, A
[6] Velázquez, Ramiro & Pissaloux, Edwige & Rodrigo, Pedro & survey of assistive technologies and applications for blind users on
Carrasco, Miguel & Giannoccaro, Nicola & Lay-Ekuakille, Aimé. mobile platforms: a review and foundation for research, Journal on
(2018). An Outdoor Navigation System for Blind Pedestrians Using Multimodal User Interfaces, 2015, Volume 9, Number 4, Page 275
GPS and Tactile-Foot Feedback. Applied Sciences. 8. 578. [14] Khan I, Khusro S, Ullah I. Technology-assisted white cane: evaluation
10.3390/app8040578. and future directions. PeerJ. 2018;6:e6058. Published 2018 Dec 10.
[7] M.Saranya , K.Nithya, Campus Navigation and Identifying Current doi:10.7717/peerj.6058
Location through Android Device to Guide Blind People, International
Research Journal of Engineering and Technology (IRJET), Vol.2,
Issue 8, Nov-2015

Authorized licensed use limited to: University of Southern Queensland. Downloaded on July 03,2020 at 17:24:43 UTC from IEEE Xplore. Restrictions apply.