100% found this document useful (1 vote)
145 views

Real Time Braille To Speech Using Python

This document discusses a system to convert braille text to speech in real-time using a camera and image processing. Braille is an important text format for visually impaired individuals but converting existing braille documents is challenging. The proposed system aims to solve this by capturing an image of braille text, detecting the braille characters, and converting them to audio that can be understood. This would help with preserving existing braille content and allow visually impaired people to access information more easily without relying on hard copy braille. The system works by preprocessing the image, segmenting individual braille cells, extracting features, and post-processing to convert the recognized braille text to audio output.

Uploaded by

Kali Linux
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
145 views

Real Time Braille To Speech Using Python

This document discusses a system to convert braille text to speech in real-time using a camera and image processing. Braille is an important text format for visually impaired individuals but converting existing braille documents is challenging. The proposed system aims to solve this by capturing an image of braille text, detecting the braille characters, and converting them to audio that can be understood. This would help with preserving existing braille content and allow visually impaired people to access information more easily without relying on hard copy braille. The system works by preprocessing the image, segmenting individual braille cells, extracting features, and post-processing to convert the recognized braille text to audio output.

Uploaded by

Kali Linux
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 10

The Report is Generated by DrillBit Plagiarism Detection Software

Submission Information

Author Name Renuka


Title Braille
Paper/Submission ID 766189
Submission Date 2023-06-08 17:27:35
Total Pages 6
Document type Research Paper

Result Information

Similarity 23 %
1 10 20 30 40 50 60 70 80 90

Sources Type Report Content


Student
Internet Quotes
Paper
0.95% 1.88%
1.57% Words <
14,
5.77%

Ref/Bib
Journal/ 7.87%
Publicatio
n 20.48%

Exclude Information

Quotes Excluded
References/Bibliography Excluded
Sources: Less than 14 Words Similarity Excluded
Excluded Source 0%
Excluded Phrases Excluded

A Unique QR Code use to View/Download/Share Pdf File


DrillBit Similarity Report

A-Satisfactory (0-10%)
B-Upgrade (11-40%)

23 10 B C-Poor (41-60%)
D-Unacceptable (61-100%)
SIMILARITY % MATCHED SOURCES GRADE

LOCATION MATCHED DOMAIN % SOURCE TYPE

1 IEEE 2015 International Conference on Electrical Engineering and Inf Publication


10
by

2 www.dx.doi.org Publication
6

3 Automated Electronic Pen Aiding Visually Impaired in Reading, Publication


2
Visualizing and Un by Josh-2012

4 IEEE 2020 4th International Conference on Vocational Education and Publication


1
Training (I

5 www.dx.doi.org Publication
1

6 Braille character recognition using find contour method Student Paper


1

7 journals.pan.pl Internet Data


1

8 Identifying community structure in a multi-relational network employing Publication


<1
non-nega by Verma-2016

9 ijircce.com Publication
<1

10 QEFSMmodelandMarkovAlgorithmfortranslatingQuranrecitingrulesinto Student Paper


<1
Braillecode

EXCLUDED PHRASES

1 mitmysore
2 dept. e&ce
Real Time Braille to Speech Conversion using  
Python  
Mrs.Renuka.C   Maanav.S.K   Jeevan.T.V  
7 Associate Professor   Student   Student  
Department of Electronics and   Department of Electronics and   Department of Electronics and  
Communication Engineering,   Communication Engineering,   Communication Engineering,  
Maharaja Institute of Technology   Maharaja Institute of Technology   Maharaja Institute of Technology  
Mysore, India   Mysore, India   Mysore, India  
email: [email protected]   email: [email protected]   email: [email protected]  
Mohith.S   Vidyashree  
Student   Student  
Department of Electronics and   Department of Electronics and  
Communication Engineering,   Communication Engineering,  
Maharaja Institute of Technology   Maharaja Institute of Technology  
Mysore, India   Mysore, India  
email:[email protected]   email: [email protected]  
10
Abstract— There are around 314 million visually challenged   Braille, originally developed by the French military for  
people in the world. One way for people with visual   nighttime communication, was later adapted by Louis  
impairments to read and write is using Braille, which is a text-   Braille in 1829 specifically for the blind. It is a code  
based format. However, braille text communication is a time-   consisting of dots embossed on paper, representing letters,  
consuming and difficult task. Hard-copy Braille is bulky and   numbers, and symbols. Blind individuals can perceive these  
takes up a lot of room to store and maintain. Old Braille books   dots with their fingers to read. Each Braille letter is  
and documents must be preserved because of their importance.   composed of six dots arranged in a cell. Braille is widely  
Automation of the Braille reading and digitization processes  
used in cell phone keypads, computer keyboards, display  
will aid in the preservation, network distribution, and demand-  
driven hard copy reproduction of Braille. The requirement to  
boards, ATMs, and currency notes.  
digitize Braille has prompted numerous researchers to devise   In the case of contracted blindness, individuals lose their  
and put optical Braille recognition systems into practice.   vision later in life as a result of an illness or an accident. In  
This paper aims to solve these issues. The system which   these situations, learning Braille on one's own becomes  
consists of a camera, captures an image of any page in a book,   challenging, and teaching such a person the Braille script  
detects the braille text and converts this text into audio. The   might be tough. To help with this, there are numerous  
goal of braille text recognition is to convert the recognized   programs and applications. There is still a lacuna in a  
braille text from hardcopy into desired format (audio). The  
process involves several steps like pre- processing,  
medium for the blind to put forth their thoughts in Braille,  
segmentation, feature extraction, post- processing. This system   since the code is not universally understood.  
basically would be used by visually challenged people as well as   Optical Braille recognition (OBR) makes it possible to  
students to learn Braille.   extract Braille letters from written text, translate them into  
Keywords—Braille, digitization, recognition, extraction   images, and then translate the images back into natural  
language. Documents are reproduced using this method in  
I. INTRODUCTION   order to preserve them for future use. OBR presents certain  
Worldwide, there are 314 million visually challenged   difficulties because no ink is used to produce the documents,  
people. Out of these, 45 million are blind, 124 million have   making it difficult to distinguish between raised dots 3 and  
low vision and 145 million have uncorrected refractive   flat surfaces. Several image enhancing techniques are used  
errors. 1.4 million children under the age of 15 are blind.   to enhance the quality of the image.  
2
More than 90% of the world’s visually impaired people live   Speech synthesis is the production of human voice or  
in low income countries[1]. The most severe sensory   speech by a machine. It used to convert written textual  
impairment, which impairs human experience in all senses   content into audible speech content for convenience. A text-  
by about 90%, is vision loss which has an impact on almost   to-speech (TTS) system performs this function. One form of  
all of the visually challenged person’s activities. For reading   speech synthesis is concatenation, which involves  
and writing visually challenged persons always use Braille   rearranging of voice samples spoken by humans into words  
representation of different alphabets, digits, symbols etc.   and sentences.  
Braille is the only source for visually impaired person to read  
and write[2]. Figure 1 shows Braille character representation.   This paper presents the implementation and analysis of  
real-time braille text-to-speech conversion. It aims to bridge  
the gap between normal and visually challenged person.  
Audiobooks are more preferred over physical books as they  
are user-friendly and portability. A similar system for braille  
books will help the visually challenged people enjoy the  
experience of reading a book virtually.2This system will also  
help normal people with sight to understand the braille  
script without any prior knowledge about braille.  

Figure 1 Braille character representation  

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE  


II. LITERATURE SURVEY  
The authors of [1] discuss the procedures for developing  
a character recognition system that converts Hindi
2 Braille  
into text using two approaches—Conventional Approach or  
SVM approach and Deep Learning Approach. The first  
method maps a Braille character to its associated Hindi  
letter by first converting it to a binary sequence. In the  
second method, a Convolutional Neural Network named  
AlexNet is trained using a dataset that contains images of  
Braille letters utilising Deep Learning ideas (specifically,  
Convolutional Neural Networks). It was discovered that the  
conventional approach's accuracy was higher as long as the  
manually established thresholds didn't need to be adjusted  
for every image. Deep learning produced good outcomes in  
terms of robustness because thresholds didn't need to be  
manually set.  
The authors of [2] have developed a prototype of an  
application that aims to help the Blind in learning Braille  
independently. The application translates the scanned  
Braille image into speech. The image is acquired  
automatically using edge detection and4 preprocessing is  
done for braille feature extraction. The find contours  
algorithm from OpenCV is used to find the presence of dots.  
After translation, the application produces a speech using   Figure 2 Block Diagram of Real Time Braille to Speech Conversion using  
the Text to Speech (TTS) mechanism from Android Studio.   Python  
The authors of [3] devised a methodology
2 to using Field   IV. METHODOLOGY  
Programmable Gate Arrays 5 (FPGAs) to convert 2 Kannada  
Braille to text or speech. The number of dots in each Braille   Images taken with a mobile camera or a flatbed scanner  
cell served as the foundation for classification, and case   often contain noise, and certain dots in the images may be  
statements were used to identify the character based on the   not visible due to different lighting conditions. To improve  
presence or absence of a dot in each of the six places. By   the visibility and facilitate the recognition of Braille dots,  
compiling a database of just phonemes obtained from MP3   different preprocessing techniques are utilized to acquire  
audio recordings and concatenating certain phonemes to   images that are free from noise.  
create any word or phrase, the authors present  a A. Conversion to Gray scale  
concatenative method of voice synthesis for the Kannada   5
language.   The first step is to convert the captured RGB image to a  
Grayscale image. Using the following equation we convert  
The authors of [4] have conducted a survey and   the RGB/BGR to Gray channel.  
comparative analysis on different algorithms and   Y = (0.299 * R) + (0.587 *G) + (0.114 *B)   (1)  
methodologies offered and presented by researchers in the   Here Y is the Gray channel which can be used to  
development of an Optical Braille Recognition System. The   construct the Gray scale image. Figure 3 shows the  
effectiveness of various preprocessing, image segmentation,   Grayscale image.  
and feature extraction techniques employed by the majority  
of researchers were evaluated. Because conventional  
procedures are still unable to attain 100% accuracy with a  
high translation rate, they have arrived to the conclusion that  
more effective Braille image preprocessing and picture  
acquisition methods are required.  
1
The authors of [5] have designed a real time braille  
recognition system. The system can be called real time   Figure 3  Grayscale image  
because a webcam is used to get braille pictures and  
webcam is connected to a computer system. Cropping,   B. Segmentation  using Binary Thresholding and OTSU  
6 Thresholding  
grayscale, thresholding, erosion and dilation techniques are  
used for image preprocessing. Then, find contour method   Thresholding is the simplest form of segmentation.  
and image recognition lookup table method were used to   Segmentation is the process in which different regions of an  
recognize the braille characters. The system could recognize   image is identified. By identifying different regions, we can  
braille characters with 100% accuracy even when the braille   effectively find the impressed dots in the image by  
image is tilted up to 0.5 degrees.   segmenting it from the background. Equation (2) is used for  
binary thresholding.  
III. BLOCK DIAGRAM  
Figure 2 shows the block diagram of the Real time   (2)  
braille to speech conversion using python.  
Where T(x,y) is the threshold of the pixel at x,y with  
respect to 2d cartesian plane. The Otsu's algorithm tries to  
find a threshold value (t) which minimizes the weighted  
within-class variance given by the relation in equation (3)  
below,  
Figure 6 After applying Dilation  
(3)  
F. Contours  
C. Removing Noise   Contours are straightforward curves that connect all  
Using Gaussian Blurring and Median Blurring we   continuous points (along a boundary) and have the same  
reduce noise in the captured image as much as possible.   colour or intensity. From the binary image that was  
produced during the preprocessing stage, contours can be  
D. Conversion to Binary Image   retrieved. We find contours by a method proposed by  
1
Binary image is an image that has only the color scale   Satoshi Suzuki et el. Figure 7 shows the contours drawn on  
values of 0 and 255, or it could be regarded as black and   original image.  
white. In this process, adaptive thresholding is used. The  
adaptive threshold uses a range of value from minimum to  
maximum. Figure 4 shows the Binary image obtained after  
thresholding.  

Figure 7  Contours drawn on original image  

Figure 4  Binary thresholding of Braille text   G. Braille Character Segmentation  


Segmentation is very simply built and unique to our  
E. Erosion and Dilation   study. The Segmentation process is as follows.  
Erosion is2 a morphological operation that erodes or  
shrinks the boundaries of objects within an image. It   G.1 Finding the Centre Coordinates of Each Dot  
achieves this by scanning the image with a structuring   After the pixel coordinate data of each member of the  
element and determining the minimum pixel value within   black dot is obtained, then the data is searched for the  
that neighborhood. The resulting image is formed by   minimum and maximum values which can determine the  
replacing each pixel with the minimum coordinates of the centre point of the black dot. To find the  
1 value found in its  
neighborhood. As shown in Figure 4, there were black spots   value of the centre point can be used Eq. 4 and Eq. 5 below.  
noise after the thresholding result. After the erosion process,  
the noise disappeared as shown in Figure 5.   (4)  

(5)  

Where centreX, centreY are the centre x and y  


coordinates respectively. In Figure 8, the results of reading  
(a) Before applying erosion  
the centre point coordinates of each black dot are shown.  

(b) After applying erosion   Figure 8 Reading centre point coordinates  

Figure 5 Before and after applying Erosion   G.2 Finding Average Coordinates of Row and column  
9 The average rows and column are detected because in  
Dilation, on the other hand,
2 is a morphological operation   one row or in one column, it is not necessary to have the
8  
that expands or enlarges the boundaries of objects within an  
image. It involves scanning the image with a structuring   same x and y coordinates. So it is necessary to find the  
element and determining the maximum pixel value within   average value of the coordinate values. To compute average  
that neighbourhood. Each pixel in the resulting image is   coordinate values Eq. 6 and Eq.7 are used as shown below.  
replaced with the maximum value found in its  
neighbourhood. As shown in Figure 6, dilation results in  
black dot enlarged.  
1
TABLE I.   DATA VALUE OF BRAILLE CHARACTER COMBINATION  
(6)  

(7)  
5
Where aXk, aYb are the average x and y coordinates of  
each row, k is1column value, b is the row value and n is the  
count of dot in each row and column. The results of finding  
the average x-y coordinates of each row and column are  
shown in Figure 9.  

Figure 9 Result of finding average x-y coordinates of each row and column  

G.3 Segmentation of each Braille Character  


The Segmentation of braille character is based on the  
average value of x and y coordinates of each of the six  
points of the x and y coordinates. The calculations used to   V. IMPLEMENTATION  
determine the area segmentation is given by Eq.8.  
A. Experimental setup  
The project’s goal is to use image processing to  
(8)   recognize the Braille code printed on a sheet of paper. The  
setup is shown in Figure 12.  
Results of the segmentation process is shown in Figure  
10. The segmentation area of one braille character is marked  
with red line.  

Figure 12 Experimental Setup  


Figure 10 Braille characters bounded by a bounding rectangle  
Braille paper images are taken using a web camera with  
G.4 Braille Character Recognition   a resolution of 1024 x 720 to create braille images. The  
process produces image files of the jpeg format with an  
After segmentation for each braille character, the next   RGB colour scale.  
process is reading the conditions in area of each dot. The  
reading of the conditions in area of each dot is done by  
reading the pixel values
2 of area of each 1dot. In these   B. Hardware and Software requirements  
readings if we read a pixel value of 0 (black) then assumed a  
value of 1, while the pixel value of 255 is assumed to be 0.   The system was developed and tested using the Python  
Figure 11 shows reading pixel values in the area   programming language on a general- purpose laptop with  
segmentation. TABLE I shows the various combinations of   Intel Core i5-10300H processor. The laptop used 8GB  
the 6-dot braille code and their corresponding character   RAM and a 64-bit Windows 10 operating system. A CMOS  
equivalents.   webcam (Logitech C170) was connected to the USB 2.0 port  
of the laptop. The webcam is shown in Figure 13.  

Figure 11 Reading pixel values in the area segmentation  


Figure 13 Logitech Webcam C170  
The specifications of Logitech Webcam C170 are shown in   It is evident that all of the dots under varied lighting  
TABLE II below:   circumstances have been successfully retrieved. However, it  
is also apparent that the dots' positions have changed as a  
TABLE II.   SPECIFICATIONS OF LOGITECH WEBCAM C170  
result of changes in lighting. As a result, since the algorithm  
counts the dots based on their placements, it would be  
Feature   Description  
incorrect for identifying Braille letters.  
Sensor type   VGA sensor   Consistent and uniform lighting is vital for accurate  
Video capture resolution   1024 x 768   identification of Braille letters. It helps to avoid noise and  
Focus type   Fixed   maintain dot detection accuracy. Finding the optimal  
Focal length   2.3 mm  
lighting intensity is crucial to prevent missed or  
misinterpreted dots.  
Maximum frame rate   30 fps  
Additional features   Logitech fluid crystal technology  
B. Dimensions of dots  
The dot diameter in a Braille display is carefully chosen  
to strike a  balance between tactile sensitivity and  
The key software requirements for the real-time Braille to   compactness. A larger dot diameter may limit the amount of  
speech conversion project implemented in Python are  
information that can be displayed within a given space.  
NumPy, OpenCV, gTTS, Tkinter, TensorFlow, and  
Tesseract. These packages provide essential functionalities   To study the accuracy of character recognition based on  
for data manipulation and processing, image and video   dimension of dots, we have took four different dimensions  
processing, text-to-speech conversion, GUI development,   and compared their performance as shown in TABLE V.  
machine learning tasks, and optical character recognition.  
TABLE V.   COMPARISION OF DETECTION OF BRAILLE DOTS  
C. Experimental procedure   BASED ON DIAMETER  
We obtained many Braille sheets with words printed in  
English. Images of the Braille character were captured   Dia   Distanc   Image of   Detection of   To No.  Er  
using a simple web camera situated at a distance of 15   mete   e Braille   Braille Character   tal   of   ro  
centimeters. Three cameras of different resolutions were   r of   betwee   character   no   cha   r
used to capture the images to compute the accuracy of   dots   n .of   ract   (
braille character detection based on resolution for each   camera   ch   ers   %
Detect-   Binary  
word. The specifications of the cameras used are shown in   and   ar   dete   )
ed   image  
braille   ac   cted  
TABLE III below.   sheet  
image  
te  
rs  
TABLE III.   SPECIFICATIONS OF THE CAMERAS USED  
5 15 cm   3 3 0
Name of the camera   Resolution of the camera   mm  

Logitech Webcam C170   5 megapixel  


Vivo Y91i   13 megapixel  
4 15 cm   3 3 0
Redmi Note 8 Pro   64 megapixel   mm  

3
3 15 cm   3 0
mm  
VI. RESULTS  

A. Lighting conditions   2 15 cm   3 2 33  
mm  
A few test cases of Braille dot extraction for various  
lighting conditions have been shown in TABLE IV.  
It was observed that for the larger dimensions of the dot,  
TABLE IV.   BRAILLE DOT EXTRACTION FOR VARIOUS   the percentage of error of character detection was less. For 2  
LIGHTING CONDITIONS   mm diameter, there was a high percentage of error for  
Lighting   Image of Braille   Detection of Braille   character recognition and some characters were not  
condition   Character   Character   detected. Since the standard Braille dimensions are greater  
Dark   than 2.5mm, the proposed project can recognize characters  
intensity   in original Braille documents.  
Medium   C. Resolution of the camera  
intensity   The resolution of the camera effects the quality of image  
taken. To compute the accuracy of braille detection based on  
High   resolution, we have used 10 braille images of different  
intensity  
Braille cell lengths. Three cameras were used to capture the  
images and the accuracy of braille character detection for  
each word was computed and their average was taken.  
TABLE VI shows the observed values.  
TABLE VI.   COMPARISION OF ACCURACY BASED ON   for VivoY8li mobile. The accuracy was much high for  
RESOLUTION OF DIFFERENT CAMERAS  
Redmi Note 8 pro which was 89.7%.  
The accuracy level increases with the resolution of the  
Name of   Accuracy of character detection per each Braille Cell   camera. These results demonstrate that this system can  
Camera   (%)   effectively recognize Braille characters with high accuracy  
levels.  
Length of cell (mm)  

1 mm   2 mm   3 mm   4 mm   5 mm   VII. CONCLUSION  
Logitech   100%   66%   50%   20%   0%   The Real-time Braille to Speech Conversion system  
Webcam   using Python is a valuable contribution to assistive  
C170  
technology for 5 visually challenged individuals. By  
recognizing Braille characters and translating them into  
Vivo   100%   87.5%   67.2%   66%   66%   alphanumeric text and speech, this system aids in reading  
Y91i   and learning. It utilizes a webcam to capture Braille images  
Redmi   100%   87.5%   80%   75%   66%  
Note 8   and employs image processing techniques such as  
Pro   thresholding, erosion, dilation, and contour detection to  
identify the positions of the embossed dots. The system  
A graph is plotted based on the values given in Figure 14.   experienced a high error rate and failed to detect certain  
characters when using a 2 mm diameter for Braille.  
However, since standard Braille dimensions are larger than  
2.5 mm, the proposed system successfully recognized  
characters in original Braille documents. The system  
achieves 89.7% accuracy with a 64MP camera, although  
accuracy decreases with lower resolutions. Consistent  
illumination is crucial to minimize noise and ensure accurate  
dot detection. This technology has diverse applications in  
education, communication, and navigation. It enables  
visually impaired students to access Braille materials swiftly  
and assists teachers in preparing resources. Additionally, it  
facilitates the conversion of Braille messages into text  
messages or emails, enhancing communication for visually  
impaired individuals. Future research can explore further  
potential and advancements in this system.  

REFERENCES  
Figure 14 Comparison of Accuracy based on Resolution   [1] P.  Kaur, S. Ramu, S. Panchakshari and N. Krupa, "Conversion of  
Hindi Braille to Speech using Image and Speech Processing," 2020  
IEEE 7th Uttar Pradesh Section International Conference on  
From the graph, it is observed that the accuracy of Redmi   Electrical, Electronics and Computer Engineering (UPCON), 2020,  
pp. 1-6, doi:10.1109/UPCON50219.2020.9376566  
note 8 Pro camera with 64 MP resolution is high. The  
[2] J.  Tri Ardiansah and Y. Okazaki, "The Design and Prototyping of  
algorithm works well for different Braille cell lengths.   Braille to Speech Application as a Self-Learning Support Media for  
Visually Impaired Person," 2020 4th International Conference on  
Based on the calculation of accuracy for the three cameras,   Vocational Education and Training (ICOVET), 2020, pp. 224-  
the accuracy percentage of character detection is computed   228,doi:10.1109/ICOVET50258.2020.9230060  
and shown in graph shown in Figure 15 below.   [3] S. R. Rupanagudi, S. Huddar, V. G. Bhat, S. S. Patil and Bhaskar M.  
K., "Novel methodology for Kannada Braille to speech translation  
using image processing on FPGA," 2014 International Conference on  
Advances in Electrical Engineering (ICAEE), 2014, pp. 1-6,  
doi:10.1109/ICAEE.2014.6838445  
[4] M.  Hanumanthappa and V. V. Murthy, "Optical Braille recognition  
and its correspondence in the conversion of Braille script to text — A  
literature review," 2016 International Conference on Computation  
System and Information Technology for Sustainable Solutions  
(CSITSS), 2016, pp. 297-301, doi: 10.1109/CSITSS.2016.7779374  
[5] J.  Subur, T. A. Sardjono and R. Mardiyanto, "Braille character  
recognition using find contour method," 2015 International  
Conference on Electrical Engineering and Informatics (ICEEI), 2015,  
pp. 699-703,doi:10.1109/ICEEI.2015.7352588  

Figure 15 Comparison of Accuracy Percentage of character recognition  

The accuracy of character recognition was found to be  


60.20% for the Webcam. The accuracy increased to 85.75 %  

You might also like