ACKNOWLEDGMENT

Presenting the report of the project we worked on during our

last year of B.E. is a great joy. I am especially grateful to Prof.
Mahima Jain, an assistant professor in the Army Institute of
Technology, Pune's Department of E&TC, for her unwavering
support during this entire process. We also want to express our
gratitude to Prof. Girish Kaapse, an assistant professor in the
E&TC department at the Army Institute of Technology in Pune,
and Dr. P. B. Karandikar, an assistant professor in the E&TC
department, for helping to make this possible by sharing his
expertise at every stage of the project's development. We
would also like to take this opportunity to thank the
departmental teachers for their helpful support and
collaboration as my project was being developed. Finally, but
just as importantly, we thank our friends for helping us finish
the project.

Sujeet K Gupta
Aman kumar
Fense N Raju

AIT 2024-2025

1
 LIST OF FIGURES

Figure Illustrations Page no.

Fig 1 Block diagram 8

Fig 2 Flow Chart diagram 9

Fig 3 Types of image 10

annotation

Fig 4 Components 12
Specification

AIT 2024-2025

2
 ABSTRACT

People deal with problems on a daily basis. Even walking without assistance is
impossible for them. They frequently depend on others to assist them. Several
technologies have been created to help those who are visually impaired. A
variety of technologies are being used.

Help the blind, Solutions based on computer vision are becoming one of the
most promising choices because they are accessible and reasonably priced. For
those who are blind or visually handicapped, this research suggests a system.
For visually challenged individuals, the suggested system seeks to provide a
wearable visual assistance that can respond to verbal commands. The capability
it offers deals with item and signboard identification.

This will make it easier for the person with vision impairments to go about their
daily lives and navigate their environment. The Open CV platform uses the
Raspberry Pi to develop artificial vision using the Python programming
language. Keywords: Raspberry Pi, Python, OpenCV, speech commands, video
recording, and navigation.

AIT 2024-2025

3
 INTRODUCTION

The Internet of Things and artificial intelligence gained enormous appeal

because of their ease of processing and vast amounts of data. It is feasible to
significantly ease the lives of these individuals. In order to provide a super sight
till they obtain the necessary resources, Care for them. Blind persons or others
who are visually challenged might use this to make their lot easy in their daily
lives. Those who are blind encounter numerous challenges.

This project's goal is to assist blind individuals in their daily lives by utilizing
the listen. There are a few key components to this visual-based endeavor,
including a camera, Vibrators, sensors, microphones, and a Raspberry Pi
mounted together with extra Internet-based working technologies are
interconnected.

The project's input will be an image or video (many frames), the picture was
taken and examined using the camera with an IoT and AI interface for the
Raspberry Pi. Thus, the object is identified. and the blind person receives
information using earphones that are transmitted via audio. Blind people's lives
are improved by this method since it will use the most recent technology and is
designed to help people with vision impairments live unrestricted lives.

Lack of visual recognition due to physiological or neurological factors is known

as visual deficit. People with virtual impairment may find it challenging to carry
out their regular everyday tasks. A new estimate states that 285 million
individuals suffer from eyesight impairment. 217 million people have moderate
to severe visual impairment, while 39 billion people are blind. The affected
people and their relatives endure great human pain as a result of losing their
sight. A person's ability to see permits them to observe their surroundings world.

AIT 2024-2025

4
 LITERATURE SURVEY

1. "IOT Based Assistive System for Visually Impaired and Aged

People"

• Publication: IEEE, 2022

• Details: This paper discusses an IoT-based assistive system

that uses sensors to detect obstacles, providing real-time
audio feedback to aid navigation. The solution is designed to
help visually impaired and elderly individuals move safely by
identifying nearby obstacles.

• Highlights:

• Focus on safe navigation through obstacle detection.

• IoT integration for enhanced accessibility.

2. "Obstacle Detection System for Visually Impaired Person"

• Publication: IRJET, 2020

• Details: This study presents a smart walking stick that uses

ultrasonic sensors and alerts users through vibrations and
sound. The cane enhances mobility and independence by
warning users of nearby obstacles.

• Highlights:

• Physical and auditory feedback for obstacle detection.

• Cost-effective and straightforward approach.

3. "Object Detection and Localization for Visually Impaired

People using CNN"

• Publication: IRJET, 2023

• Details: This paper focuses on using Convolutional Neural

Networks (CNN) to detect and locate objects, giving visually
impaired users audio cues about their surroundings in real
time. The system aids in recognizing and localizing objects
within the environment.

• Highlights:

AIT 2024-2025

5
 • Uses deep learning for real-time object detection.

• Provides auditory assistance to improve environmental

awareness.

• Limited to indoor object detection and navigation.

PROBLEM STATEMENT

1. Visually impaired individuals face challenges in navigating their

surroundings independently, particularly in unfamiliar or complex
environments.

2. Common tasks like identifying objects, avoiding obstacles, and

maintaining spatial awareness are difficult without visual cues,
leading to limited mobility and increased dependence on others.

3. Existing assistive devices can be prohibitively expensive, lack real-

time feedback, or are not designed for practical daily use.

4. There is a need for a portable, affordable, and effective visual

assistant device that provides real-time audio feedback, helping
visually impaired individuals navigate independently and safely.

5. This project seeks to address these challenges by developing a user-

friendly device that enhances mobility and improves the quality of
life for visually impaired individuals.

AIT 2024-2025

6
 AIM
The aim of this project is to develop a portable, real-time visual assistant for
visually impaired individuals that enhances their independence by providing
audio feedback on surrounding objects and obstacles. Using a Raspberry Pi,
camera module, and speaker, the device detects objects in the user’s
environment and communicates relevant information via voice prompts,
allowing for safer and more informed navigation in both indoor and outdoor
settings. This project seeks to address the daily navigation challenges faced by
visually impaired individuals, offering a practical, affordable, and user-friendly
solution to improve their quality of life.

OBJECTIVE

The primary objective of this project is to design and develop a portable visual
assistant device that uses a Raspberry Pi, camera, and speaker to provide
visually impaired individuals with real-time object detection and audio
feedback. The system aims to reliably identify common objects and obstacles,
enabling users to navigate safely in various environments, including both indoor
and outdoor settings. Through a text-to-speech (TTS) system, the device will
deliver immediate audio feedback, giving users clear information about their
surroundings and enhancing their situational awareness. Additionally, we aim to
optimize the device’s performance by minimizing processing latency and
ensuring a long-lasting battery for extended usability. User-centered testing with
visually impaired individuals will be conducted to refine the system based on
real-world feedback, ensuring it is user-friendly, accurate, and effective.
Ultimately, this project strives to create an affordable, accessible assistive tool
that empowers visually impaired individuals, supporting greater independence
and improving their quality of life through enhanced mobility.

AIT 2024-2025

7
 Block Diagram

Flow Chart Diagram

AIT 2024-2025

8
Methodology

1. Requirements Analysis

In the initial stages of our project, we conducted a thorough requirements

analysis to understand the needs and challenges of visually impaired
individuals. To gather authentic insights, we visited a local blind center,
where we spoke with individuals and caregivers about their daily
experiences and observed the environment. These conversations revealed
several specific challenges that influenced our design choices. For
instance, many visually impaired individuals struggle with recognizing
common objects around them, avoiding obstacles in crowded spaces, and
maintaining spatial awareness, especially in unfamiliar environments.
Recognizing these needs, we aimed to design a device that would be both
functional and user-friendly, addressing the everyday struggles they
shared with us.

From this research, we identified that real-time object detection would be

a critical feature, as it enables immediate feedback on surrounding
objects, allowing for more independent movement. Additionally, audio
feedback emerged as the most effective way to convey information to
users without requiring visual cues or manual operation. By prioritizing
features that provide reliable object detection and clear audio feedback,
we designed a solution that would offer practical support, enhancing the
user’s situational awareness and overall independence. This analysis
shaped our approach, ensuring that each aspect of the device’s
functionality was grounded in the real-world needs of its intended users.

2. Hardware Setup:-

Our project’s core hardware components are a Raspberry Pi, a camera

module, and a speaker. The Raspberry Pi serves as the primary processor
for managing tasks and running software for object detection and voice
output. The camera captures live video frames, which the Raspberry Pi
AIT 2024-2025

9
 processes to identify nearby objects. The speaker is used to communicate
detected objects and obstacles to the user in real-time. By choosing these
compact components, we aim to keep the device lightweight and portable.

Raspberry Pi

The Raspberry Pi 4 is a powerful single-board computer designed to

be versatile for various applications, from educational projects to
advanced technical applications. Here’s a summary of its key
theoretical aspects:
1. Processor: It uses a Broadcom BCM2711, Quad-core Cortex-A72
(ARM v8) 64-bit SoC, clocked at 1.5GHz, offering significantly better
performance compared to previous models.
2. Memory: Available in multiple RAM configurations (2GB, 4GB, and
8GB LPDDR4), allowing for smoother multitasking and support for
more memory-intensive applications.
3. Connectivity: It includes dual-band Wi-Fi, Bluetooth 5.0, and
Gigabit Ethernet, making it suitable for both wired and wireless
applications.
4. Graphics: Equipped with a VideoCore VI GPU, it supports dual 4K
display output, making it capable of handling visual processing for
applications requiring high-quality graphics.

AIT 2024-2025

10
 5. USB Ports: It features two USB 3.0 and two USB 2.0 ports,
supporting faster data transfer rates for peripherals.
6. GPIO Pins: The 40-pin GPIO (General Purpose Input/Output) allows
users to connect external components, enabling a wide range of
electronics and IoT applications.
7. Power Supply: Requires a USB-C 5V 3A power source, providing
more stable and reliable power than previous models.

Camera Module for Raspberry pi

The Raspberry Pi Camera Module is an accessory designed specifically for the Raspberry Pi
boards, enabling high-quality image and video capture. Here’s an overview of its theoretical
aspects:
1. Sensor: The camera module typically features a high-resolution sensor (like the Sony
IMX219 for older versions or Sony IMX477 for newer high-quality versions) that
allows capturing images and videos with high clarity and detail.
2. Resolution: The module offers different resolutions depending on the version:
 Camera Module v2: 8 MP resolution, capable of capturing still images and
Full HD (1080p) video at 30 fps.
 High Quality (HQ) Camera Module: 12.3 MP resolution, with support for
various lens mounts (e.g., CS and C-mount), providing greater flexibility and
improved image quality for advanced applications.
3. Lens and Field of View (FoV): The standard modules have a fixed-focus lens with a
broad field of view (around 62.2° horizontally for the v2 module). The HQ Camera
AIT 2024-2025

11
 allows for interchangeable lenses, which helps customize the field of view and depth
of focus.
4. Interface: The camera connects to the Raspberry Pi via the CSI (Camera Serial
Interface) port using a ribbon cable. The CSI interface allows high-speed data
transmission, enabling real-time image and video processing.
5. Image Processing: With built-in support in the Raspberry Pi OS, the camera module
works seamlessly with software libraries like OpenCV, Python’s PiCamera library,
and other imaging tools, making it easy to integrate into applications requiring
computer vision.
6. Applications: Due to its compact design and good performance, the camera module is
ideal for projects involving surveillance, facial recognition, object detection, and, in
your case, creating a visual assistant for blind individuals.

3. Software Design:-

Object Detection: We implemented object detection using OpenCV and a pre-trained

model that can recognize common objects. The camera captures images, and the
Raspberry Pi processes them to detect objects, converting them into recognizable
names.

Text-to-Speech (TTS): For voice feedback, we integrated a TTS library (such as

gTTS) that reads out detected objects or warnings. This voice output provides the user
with immediate information about their surroundings.

System Integration: The object detection and TTS modules work together in real
time. When an object is detected, the TTS module converts the object’s name to
speech, which the speaker then relays to the user, ensuring timely and accurate
feedback.

4. Data Processing and Training:-

1. Model Selection
Chose a pre-trained model (e.g., MobileNet or YOLO) that balances high accuracy
and efficient processing, suitable for real-time use on the Raspberry Pi.

2. Dataset Selection
Utilized standard datasets like COCO (Common Objects in Context) to ensure the
model recognizes a wide range of objects relevant to daily environments.

AIT 2024-2025

12
 3. Data Preprocessing
Applied preprocessing steps such as resizing images, normalizing pixel values, and
data augmentation to improve the model’s robustness to different conditions.
Performed data augmentation techniques like rotating, flipping, and adjusting
brightness to make the model more adaptable to real-world conditions.

4. Model Training and Optimization

If additional training was needed, trained the model on a selected dataset using
specific hyperparameters (e.g., learning rate, batch size) to achieve a good balance
between accuracy and processing speed.
Fine-tuned the model on specific object categories that were most relevant for visually
impaired users, like common furniture or household objects.

5. Inference and Real-Time Processing

Optimized the model for inference on the Raspberry Pi to ensure smooth real-time
performance.
Reduced processing delays by adjusting the frame rate and optimizing object
detection algorithms, allowing immediate feedback for the user.

6. Testing and Validation

Validated the model’s performance in various environments (indoor, outdoor) to
ensure it could detect objects accurately in diverse conditions.
Monitored key metrics like accuracy, latency, and processing speed, refining the
model and preprocessing steps as needed to meet the device’s real-time requirements.

5. Testing and Calibration

To ensure the reliability and accuracy of our visual assistant, we conducted extensive
testing and calibration across various environments and conditions. Testing began
with each individual component, including the camera, object detection algorithm,
and speaker. We tested the camera’s ability to capture clear images in different
lighting conditions and angles, which was critical for accurate object recognition.
Adjustments were made to optimize exposure settings for both low-light and bright
environments, as lighting significantly affects detection accuracy.

After confirming that each component worked effectively on its own, we conducted
integrated system tests. We placed the device in controlled environments with
common obstacles like chairs, tables, and doors to assess its ability to detect and
describe objects. During these tests, we adjusted the sensitivity of the detection
AIT 2024-2025

13
algorithm, ensuring it accurately identified objects at practical distances without
overwhelming the user with unnecessary details about objects that were too far away.

Field testing with blind individuals provided crucial insights into the practical
usability of the device. We tested the system both indoors (in rooms, hallways, and
cluttered spaces) and outdoors (on streets and open spaces), noting how the device
responded in real-world settings. The feedback from users helped us fine-tune the
timing of audio feedback so that voice cues were delivered promptly without being
delayed or overly frequent. Additionally, we calibrated the text-to-speech volume and
speed to ensure that feedback was clear and easily understandable, even in noisy
environments.

Final calibration involved optimizing power usage to ensure a longer battery life,
allowing the device to be functional over extended periods. After several rounds of
testing and calibration, we achieved a setup where the device consistently delivers
accurate, timely information about the user’s surroundings, enhancing both its
usability and effectiveness for visually impaired individuals. This thorough testing
and calibration process ensured that the device would be practical, reliable, and
comfortable for real-world usage.

AIT 2024-2025

14
 EXPERIMENTATION

Among different Algorithms, we used YOLO for this project.

What is YOLO:

YOLO (You Only Look Once): YOLO is a popular real-time object detection
system. It's known for its speed and accuracy, making it suitable for applications
like aerial surveillance. YOLOv5, the latest version, offers improved accuracy
and efficiency.

Steps:-
AIT 2024-2025

15
1. Component Testing

 Conducted individual tests on each component (camera, Raspberry

Pi, speaker) to ensure they functioned correctly and met
performance requirements.

 Verified camera quality under different lighting conditions to

ensure clear image capture for reliable object detection.

2. Environment-Based Testing

 Tested the device in various environments, including indoor spaces

(rooms, hallways) and outdoor settings (sidewalks, open areas), to
evaluate object detection performance in diverse conditions.

 Assessed the accuracy of object recognition under changing light

conditions, like low light and bright sunlight, making adjustments
as needed.

3. User Testing with Feedback

 Conducted usability testing with visually impaired individuals to

evaluate the device’s functionality, ease of use, and overall
effectiveness.

 Collected feedback on voice prompt clarity, timing, and whether

the device provided valuable information for navigating different
spaces.

4. Real-Time Performance Analysis

AIT 2024-2025

16
  Measured real-time performance, including detection speed,
latency, and responsiveness, to ensure the device provided quick
feedback without noticeable delays.

 Optimized processing parameters to reduce lag, providing users

with timely and accurate information.

5. Battery Life and Portability Assessment

 Tested the device's power consumption under normal use to

evaluate battery life and ensure the device remains operational for
extended periods.

 Evaluated portability, ensuring that the device’s weight and design

were comfortable for users to carry or wear.

6. Error Rate and Accuracy Tracking

 Monitored error rates in object detection and false positives to

improve the system’s reliability.

 Adjusted model parameters to reduce detection errors, ensuring

that the device provided accurate feedback in practical situations.

7. Calibration and Refinement

 Used insights from testing to calibrate sensitivity and timing of

feedback, making it suitable for real-time use.

 Implemented refinements based on user feedback to improve

overall functionality and user satisfaction.

CODE :-
AIT 2024-2025

17
 Object Detection Pipeline:

Explanation: This code initializes a YOLO model for object detection

using pre-trained weights and configuration files. Each video frame is
converted into a "blob" (suitable format) and fed into the model, which
returns detected objects. The model runs on the CPU by default, but it can
be set to use a GPU.

OCR Pipeline:

• Explanation: This code uses EasyOCR to extract text from specified

regions within an image. It first crops the region of interest (ROI) based
on the provided bounding box, then converts this area to grayscale to
enhance OCR accuracy. Finally, EasyOCR processes the grayscale ROI
and returns any detected text.

AIT 2024-2025

18
 Audio Feedback Using gTTS:

• Explanation: This code takes recognized text as input and converts it to

audio output using gTTS. The generated audio file (output.mp3) is then
played back, providing real-time spoken feedback.

AIT 2024-2025

19
 CHALLENGES DUE TO HARDWARE

1. OCR Performance:

• Initially used Tesseract OCR, but it struggled with accurate text

recognition.

• Switched to EasyOCR for improved text detection and faster processing.

2. Frame Skipping:

• Implemented frame skipping for video input, processing fewer frames to

reduce overall load.

3. Optimized Text Processing:

• Converted text images to grayscale instead of color, reducing processing

time for OCR.

4. High CPU Usage:

• Running both object detection and OCR is CPU-intensive, requiring

optimization to maintain speed and accuracy.

Challenges Faced by Blind People:-

• Navigating indoors and outdoors, with risks of colliding with obstacles or
missing steps.

• Locating personal items such as wallets, keys, or phones, which can be

difficult to find if misplaced.

• Understanding signs and labels in public spaces (e.g., bus stops, restroom
signs), which are often inaccessible.

AIT 2024-2025

20
• Recognizing and identifying people, which can lead to social discomfort
and awkwardness.

• Crossing streets and navigating crowded areas without visual cues,

increasing dependency on auditory signals.

• Handling everyday objects safely (e.g., knives, cups) without risk of

injury.

• Reading text on devices and objects, leading to reliance on others for

assistance.

• Interpreting social cues and non-verbal communication, which can affect

social interactions.

• Identifying the right bus or location, particularly in transit settings where

visual signs are crucial.

• Keeping track of belongings in public spaces to avoid loss or

misplacement.

Challenges Addressed by Our Project:

• Navigating Indoors and Outdoors
Solution: Object detection and distance-based vibration feedback help
users avoid obstacles, reducing the risk of collisions and falls.

AIT 2024-2025

21
• Understanding Signs and Labels in Public Spaces
Solution: Text-to-speech for reading signs and labels provides access to
critical information in public areas, enhancing independence.

• Recognizing and Identifying People

Solution: Facial recognition allows users to recognize friends or family,
improving social engagement and reducing discomfort.

• Handling Everyday Objects Safely

Solution: Object detection identifies potentially hazardous items,
ensuring safer handling of sharp or hot objects.

• Reading Text on Devices and Objects

Solution: Text-to-speech capabilities enable users to read text on phones
and other objects, increasing autonomy.

AIT 2024-2025

22
Project Schedule

AIT 2024-2025

23
 CONCLUSION

This paper presents a novel technique for assisting

visually impaired people. The proposed system has a
simple architecture and makes it user-friendly thus,
making the subject independent in his/her home. The
system also aims at helping the blind to navigate in
his/her surroundings by detecting obstacles, locate his
necessities, read signboards and texts. Preliminary
experiments show promising results the user can freely
navigate in his surroundings safely. The system is made
much more user-friendly by accepting speech as the input
to access his basic necessitate.

AIT 2024-2025

24
 REFERENCES

 Kiran G Vetteth, Prithvi Ganesh K, Durbha Srikar “Collision

Avoidance Device For Visually Impaired”, INTERNATIONAL
JOURNAL OF SCIENTIFIC & TECHNOLOGY RESEARCH
VOLUME 2, ISSUE 10, OCTOBER 2013
 S. Sivakumar, T. M. Pavithra, K. Pooja Shree, S. Reshma
“Intelligent Virtual Assistant for Visually Impaired People
using Raspberry Pi”, International Journal of Engineering
Science and Computing, Issue 3, March 2020.
 The Linn, Ali Jwaid, Steve Clark, “Smart Glove for Visually
Impaired”, IEEE, Issue 18, July 2017.
 Steven Edwin Moore, "Drishti: An Integrated Navigation
System for the Visually Impaired and Disabled”, IEEE,
Issue 8 October 2002. [5]. Anitha. J, Subalaxmi. A
Vijayalakshmi. G, “Real-Time Object Detection For Visually
Challenged Person”, International Journal of Innovative
Technology and Exploring Engineering, Issue-8 June 2019.
 Heika Gada, Vedant Gokani, Abhinav Kashyap, Amit
Deshmukh, "Object Recognition for the Visually Impaired "
IEEE, Issue 4 January 2019.

AIT 2024-2025

25