DATE : 24.09.

2024

SEMINAR PRESENTATION

ANALYSIS OF DIFFERENT
ASSISTIVE TECHNOLGY FOR
THE VISUALLY IMPAIRED
Project Guide Presented by:
Dr. Varun G Menon Jyothish T R
Department of Computer Science SCM21CS074
SSET, Karukutty
 CONTENTS
1. Introduction
2. Motivation
3. Objectives
4. Literature review
5. Different Technologies
a. Smart Cane
b. Deep-See
c. Robot Dog
6. Conclusion
7. References

Analysis of different assistive technolgy for the visually impaired 2/24

 INTRODUCTION
With increasing number of blind people in the society the need to develop
technologies that help them becomes relevant.
Technologies like smart wearables and smart canes are currently used to help the
blind reach from point A to point B.

Fig 1. [KMP Dunque et al. Obstacle Detector and Qibla Finder for Visually Impaired
Muslim Community]
Global trends and predictions of numbers of people who are blind or moderately and
severely vision impaired, from 1990 to 2050

Analysis of different assistive technolgy for the visually impaired 3/24

 INTRODUCTION
Visually impaired individuals face significant challenges in navigation and mobility,
impacting their independence and quality of life
Advances in technology have enabled the development of smart assistive devices to
aid visually impaired people in obstacle detection, navigation, and environmental
awareness

Analysis of different assistive technolgy for the visually impaired 4/24

 INTRODUCTION
This presentation will explore existing technologies that help the visually impaired and
blind.
An analysis of existing technologies will provide a overview of which technologies are best
and which can be used to create a new, better method to achieve the same goal

Fig 2. Tapu et. al. Wearable assistive devices for visually impaired: A state of the art survey

Analysis of different assistive technolgy for the visually impaired 5/24

 MOTIVATION

According to WHO the ratio of blind people in the world will

only increase in the coming years.
Robots that navigate both indoors and outdoors efficiently and
can be used without any help are necessary.

Analysis of different assistive technolgy for the visually impaired 6/24

 OBJECTIVES

To analyse existing blind assist

technologies.
To understand the advantages and
disadvantages of different
systems.

Analysis of different assistive technolgy for the visually impaired 7/24

 LITERATURE REVIEW
Journal
Name with Paper Title Authors Findings Challenges
year

There has been sustained growth in

research on assistive technology for
Many systems still
visually impaired people, from less than 50
struggle with accurately
publications per year in the mid-1990s to
detecting and classifying
Sensor-Based close to 400 scientific publications per year
obstacles, especially in
Assistive Devices in 2014.
Wafa dynamic and cluttered
for Visually- The research field encompasses multiple
Elmannai real-world environments
Sensors Impaired People: disciplines, including computer vision,
and Khaled There is a need for more
Current Status, sensor processing, cognitive psychology,
Elleithy extensive real-world
Challenges, and and human-computer interaction. Four
testing and validation of
Future Directions main research communities were identified:
assistive devices with
multisensory research, accessible content
actual visually impaired
processing, accessible user interface
users
design, and mobility and accessible
environments research.

Analysis of different assistive technolgy for the visually impaired 8/24

 LITERATURE REVIEW
Journal
Name with Paper Title Authors Findings Challenges
year

Mobile platforms, especially smartphones, Designing user interfaces

have become a de facto standard for that are appropriate for
implementing assistive technologies due to the sensorimotor
their increasing computational capacity and capabilities of blind users
An insight into
sensor capabilities. remains a significant
assistive
Journal on Assistive applications now cover a wide challenge, both for input
technology for the
Multimodal User Alexy B. et range of functionalities, including and output interactions.
visually impaired
Interfaces al. navigation, object recognition, text reading, Ensuring the reliability
and blind people:
and social interaction support for visually and accuracy of assistive
state-of-the-art
impaired users. technologies in diverse
and future trends
There's a growing trend towards using real-world environments,
computer vision algorithms and artificial especially for critical tasks
intelligence to enhance the capabilities of like navigation, continues
assistive devices. to be a major challenge.

Analysis of different assistive technolgy for the visually impaired 9/20

 LITERATURE REVIEW
Journal
Name with Paper Title Authors Findings Challenges
year

Research on navigation assistants for

visually impaired people has increased,
with publications growing from around 50 Despite technological
per year in the 1990s to about 400 per year advancements, many
A survey of in 2014. navigation assistants
assistive Navigation assistants have evolved from have not been widely
technologies and simple obstacle detection devices to adopted by the visually
Journal on applications for Ádám sophisticated systems integrating computer impaired community.
Multimodal User blind users on Csapó et vision, GPS, and artificial intelligence to Accurate obstacle
Interfaces mobile platforms: al. provide comprehensive environmental detection and
a review and information. classification in dynamic,
foundation for The field has diversified into multiple real-world environments
research research communities, including remains a significant
multisensory research, accessible content challenge for many
processing, accessible user interface systems.
design, and mobility in accessible
environments.

Analysis of different assistive technolgy for the visually impaired 10/20

 TYPES OF ASSISTIVE TECHNOLOGY

FIG 3: Jyoti et. al “A Qualitative and Quantitative Analysis of Research in Mobility Technologies for Visually Impaired People”

Analysis of different assistive technolgy for the visually impaired 11/20

 TECHNOLOGIES DISCUSSED
Smart cane integrates ultrasonic sensors, camera modules with

Smart Cane
facial recognition capabilities, and haptic feedback to provide
visually impaired users with enhanced obstacle detection,
environmental awareness, and social interaction support.

Smart cane integrates ultrasonic sensors, camera modules with

DEEP SEE facial recognition capabilities, and haptic feedback to provide
visually impaired users with enhanced obstacle detection,
environmental awareness, and social interaction support.

Robot Guide A quadrupedal robot (Mini Cheetah) that uses a leash and hybrid
physical interaction model to safely guide visually impaired people
Dog through narrow and cluttered spaces.

Analysis of different assistive technolgy for the visually impaired 12/20

 SMART CANE Core Technologies

Arduino UNO Brains of the system

Key Features:
To sense the
PIR and Ultrasonic
Obstacle Detection: Dual surrounding and get
sensors
ultrasonic sensor system inputs

Facial Recognition: Integrated

ESP32 camera module ESP 32 Camera module For Imaging
User Alerts: Haptic feedback
through vibration
Vibration module Feedback mechanism
Power Efficiency: PIR sensor-
triggered camera activation
Li-ion rechargeable
Adaptability: Can be integrated Power Supply
battery
with existing white canes
Fig 4. Sivanpalanirajan M et.
al. Design of Smart Cane with
integrated camera module
for visually impaired people

Analysis of different assistive technolgy for the visually impaired 13/20

 SMART CANE
The figure depicts the different positions the cane would be in when it is in use.
The camera module’s FOV is 78 degree and this combined with the position will provide an
FOV of 150-160 degrees
Even if the cane is tilted the camera will be able to capture all objects around it.
Also if the angle abruptly changes(ie: if the cane is dropped) the relatives will receive an
Fig 5. Sivanpalanirajan M et al. Design of alert
Smart Cane with integrated camera
module for visually impaired people

Fig 5. Sivanpalanirajan M et al. Design of

Smart Cane with integrated camera
module for visually impaired people

Analysis of different assistive technolgy for the visually impaired 14/20

 SMART CANE
The ESP32 camera module, triggered by a PIR motion sensor, captures images of people
approaching the cane user. These images are compared to a stored database of known
individuals. If a match is found, the system provides an audio message to the user, identifying
the person by name.
For unfamiliar faces, it alerts the user that an unknown person is present. This feature
enhances the user's awareness of their surroundings, providing valuable social context and
Fig 6. Sivanpalanirajan M et al. potential security benefits. The camera's wide 160° field of view ensures comprehensive
Design of Smart Cane with
coverage, even when the cane is tilted during normal use.
integrated camera module for
visually impaired people
This graph illustrates the ultrasonic sensors’ performance. The graph demonstrates a linear
relationship, aligning with the theoretical distance calculation formula. The sensor's response
is particularly sensitive in the first 50 cm, providing high-resolution detection for nearby
obstacles. Beyond this range, the response settles into a steady linear increase up to the
maximum range of 400 cm. This characteristic ensures accurate distance measurements for
both close and distant obstacles, allowing the smart cane to provide timely alerts to users
about potential hazards in their path.
Fig 7. Sivanpalanirajan M et al. Design of
Smart Cane with integrated camera
module for visually impaired people

Analysis of different assistive technolgy for the visually impaired 15/24

 DEEP-SEE

FIG 8 & 9. Tapu et al. “DEEP-SEE: Joint Object

Detection, Tracking and Recognition with
Application to Visually Impaired Navigational
Assistance”

Analysis of different assistive technolgy for the visually impaired 16/24

 DEEP-SEE
Deep-see makes use of 2 CNNs trained offline, one
for motion patterns and one for visual awareness
The initial position is obtained using motion based
CNN and further refined using occlusion detection
and handling and visual appearance updating
model
Uses quad-tree decomposition of object bounding
box, compares image patches using DeepCompare
algorithm, adjusts bounding box size when
occlusion detected
The visual appearance model maintains set of
reference object instances is updated based on
visual similarity constraints and is used to re-
FIG 10. Tapu et al. “DEEP-SEE: Joint Object Detection, Tracking and Recognition with
initialize tracker after occlusions
Application to Visually Impaired Navigational Assistance” It makes use of offline learning, hence
computationally efficient
Achieves high accuracy and robustness while
maintaining real-time performance

Analysis of different assistive technolgy for the visually impaired 17/24

 DEEP-SEE
DEEP-SEE's tracking system matches top competitors in
accuracy and robustness while operating four times faster, at 20
fps on standard hardware. It achieved the highest accuracy on
the VOT 2016 dataset, ranking fourth in failure rate. This balance
of accuracy and speed makes it ideal for real-time applications
like assisting visually impaired users.
DEEP-SEE achieves impressive object detection and
classification, with precision and recall around 90% and a 5%
improvement in F1 score over previous methods. It robustly
identifies various obstacles in urban settings, handling
challenging conditions like dynamic scenes and lighting changes
FIG 11. Tapu et al. “DEEP-SEE: Joint Object Detection, Tracking and effectively.
Recognition with Application to Visually Impaired Navigational DEEP-SEE demonstrates high accuracy in detecting various
Assistance” obstacles in urban environments, crucial for assisting visually
impaired users. The system shows robust performance despite
challenging conditions. This reliability in diverse real-world
scenarios makes it an effective tool for enhancing the mobility
and safety of visually impaired individuals in outdoor settings.

Analysis of different assistive technolgy for the visually impaired 18/24

 ROBOT DOG

A quadrupedal robot(Mini It is capable of navigating narrow and

Cheetah) for guiding the blind cluttered spaces
and visually impaired. It employs a hybrid system that
It uses a leash for physical switches between taut and slack leash
human-robot interaction modes

FIG 12. A Xiao et al. “Robotic Guide Dog: Leading a Human with
Leash-Guided Hybrid Physical Interaction”

Analysis of different assistive technolgy for the visually impaired 19/20

 ROBOT DOG
Developed a hybrid dynamic model for taut and slack leash modes
Created a data-driven leash tension model
Implemented a global A* planner for overall path generation
Designed a local planner using mixed-integer optimization to handle hybrid modes

FIG 13. Hybrid modes switches in the discrete

dynamics model Σ. The hybrid modes switches
from Ss to St when leash becomes taut and
the switch action is denoted as ∆s→t. When
the cable becomes slack, it switches back to
Ss with action denoted as ∆t→s. This hybrid
mode transitions are shown in the discrete-
time manner. [6]

FIG 15. A. Xiao et al.Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical
Interaction

Analysis of different assistive technolgy for the visually impaired 20/24

 ROBOT DOG

Successfully guided blindfolded individuals through narrow spaces (e.g., doorways)

Demonstrated hybrid mode switching to navigate tight areas
Achieved collision-free navigation in cluttered environments
Validated the system with multiple test subjects

FIG 16. A. Xiao et al.Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid FIG 17. A. Xiao et al.Robotic Guide Dog: Leading a Human
Physical Interaction with Leash-Guided Hybrid Physical Interaction

Analysis of different assistive technolgy for the visually impaired 13/20

 CONCLUSION
Analyzed three innovative assistive technologies for the visually impaired:
Smart Cane
DEEP-SEE
Robot Guide Dog
Key Findings
Smart Cane: Integrates multiple sensors and facial recognition for enhanced navigation and social interaction.
DEEP-SEE: Achieves high accuracy in object detection and tracking with real-time performance.
Robot Guide Dog: Demonstrates successful navigation in complex environments using hybrid physical interaction.
Continued advancements in AI and sensor technologies will further improve assistive devices.
Future Outlook
Integration of these technologies may lead to more comprehensive solutions.
Emphasis on user testing and feedback is crucial for widespread adoption.
Challenges remain in creating reliable, user-friendly, and affordable assistive technologies for the visually impaired.

Analysis of different assistive technolgy for the visually impaired 16/20

 REFERENCES
[1] S. Khan et. al. "Waypoint navigation system implementation via a mobile robot using global positioning system (GPS) and
global system for mobile communications (GSM) modems", Int. J. Comput. Eng. Res., vol. 3, no. 7, pp. 49-54, 2013.

[2] Fig 1. from “Obstacle Detector and Qibla Finder for Visually Impaired Muslim Community”.

[3] Tapu et al. “Wearable assistive devices for visually impaired: A state of the art survey”.

[4] Sivapalanirajan. M et al. “Design of Smart Cane with integrated camera module for visually impaired people”.

[5] Jyoti Madake et al. “A Qualitative and Quantitative Analysis of Research in Mobility Technologies for Visually Impaired
People”.

[6] Anxing Xiao et al. “Robotic Guide Dog: Leading a Human with Leash-Guided Hybrid Physical Interaction”.

[7] Tapu et al. “DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational
Assistance”

Analysis of different assistive technolgy for the visually impaired 13/20

 THANK YOU
Presentation by Jyothish T R

Analysis of different assistive technolgy for the visually impaired 13/20