AN ELECTRONIC EYE FOR THE VISUALLY IMPAIRED

(NANOTECHNOLOGY)

ABSTRACT detect the color of the traffic

lights. The process of detecting a

This paper proposes a
crossing is a pre-process
method for detecting frontal
followed by the process for
pedestrian crossings from image
detecting the state of the traffic
data obtained with a single
lights. It is important for the
camera as a travel aid for the
visually challenged to know
visually challenged. This would
whether or not a frontal area is a
be mounted on a pair of glasses,
crossing. The existence of a
will be capable of detecting the
crossing is detected in two steps.
existence and location of a
In the first step, edge detection
pedestrian crossing, to measure
and pattern detection are
the width of the road, and to
employed to identify the  Functioning of the
system
crossing. In the second step, the
 Image Analyzer
existence of a crossing is
 Calculation of width of
detected by checking the road and time required
to cross it
periodicity of white lines on the
 Traffic light detector
road using projective invariants.
 Timing unit
Then the traffic light detector is  Voice speech system
 Functional block
used to check the pedestrian
 Conclusion
light and the time display. The

calculated time is then INTRODUCTION

compared with the average time Blindnes

needed for a blind person to s is the most feared of all human
ailments. CROSSING busy
cross. The observations are
roads can be a challenge for
relayed through voice signals people with good vision. For
using the voice vision blind people, it is a perilous
activity. Our electronic eye aims
technology. Thus, this effective
at helping millions of blind and
technology aids mobility for the visually impaired people lead
visually impaired throughout the more independent lives.

globe. The
electronic eye can be adapted to
help the blind or visually
INDEX
impaired get around without a
walking stick or seeing-eye dog.
 Introduction
Canes and other travel aids with
 An overview of our
sonar or lasers can alert the user
Electronic Eye
to approaching objects. Global
Positioning Systems can tell
what streets, restaurants, parks An
and other landmarks the user is effective navigation system
passing. Devices like these are would improve the mobility of
very good at giving locations millions of blind people all over
and directions. But the the world. Our new “eye” will
limitations of G.P.S. technology allow blind people to cross busy
mean that they cannot pin down roads in total safety for the first
the location of a curb or time. Our “electronic eye”,
crosswalk and frequently fail in which would be mounted on a
areas that have many tall pair of glasses, will be capable
buildings and high traffic. None of detecting the existence and
of these devices are able to location of a pedestrian crossing,
specifically identify a and at the same time measure the
crosswalk, nor do they have the width of the road to the nearest
potential for figuring out the step and detect the color of the
state of the traffic signals. traffic lights.

AN OVERVIEW OF OUR ELECTRONIC EYE

A camera to
capture the image
of crossroads and
traffic signals

A voice speech
generator - used to
instruct the user

We have developed a system By measuring the width of the

that is able to detect the road and the color of traffic
existence of a pedestrian lights, this single camera can
crossing in front of a blind now give the blind all the
person using a single camera. information they need to cross a
road in safety. The camera system and give vocal
would be mounted at eye level, commands and information
and be connected to a tiny through a small speaker placed
computer. It will relay near the ear
information using a voice speech
.

FUNCTIONING OF THE SYSTEM

CROSS
CAMERA IMAGE
ROAD
ANALYSER
DETECTOR

TRAFFIC VOICE
LIGHT SPEECH TIMING
DETECTOR GENERATOR
1 UNIT

2
3

1 2 3

TO USERS
EAR

1 – Tells the user whether any cross road is present

2 - Tells the user whether the traffic signal is favorable or not
3 – Tells the user the time taken to cross the road.

The style of crosswalks speech system instructs the user

commonly used in India are to cross the road.
known as zebra crossings and
The timer unit calculates
they feature a series of thick
the average time required by the
white bands that run in the same
visually challenged person to
direction as the vehicle traffic.
cross the road and ‘tells’ it to the
To detect the
user via the voice speech system.
presence of a zebra crossing we
use the “projective invariant”
High-level scene
which takes the distance
interpretation applied to the
between the white lines and a set
processed images will produce a
of linear points on the edges of
symbolic description of the
the white lines. This gives an
scene. The symbolic description
accurate way of detecting
is then converted into verbal
whether crossing is present in a
instructions appropriate to the
given image or not.
needs of the user by using voice
The length of a
speech software.
pedestrian crossing is measured
by projective geometry. The IMAGE ANALYSER
camera makes an image of the
white lines painted on the road, The image analyzer
and then the actual distances are contains the bitmap image,
determined using the properties which has to be processed to
of geometric shapes as seen in detect the presence of a zebra
the image. crossing. Given an X-bit per
pixel image, slicing the image at
The traffic light detector
different planes (bit-planes)
checks images for symmetrical
plays an important role in image
shapes and compares them to a
processing.
list of road signs. If the
pedestrian light is ON, the voice
EDGE DETECTION lines from edges detected in an

One way to detect edges earlier processing step.

or variations within a region of

an image is by using the gradient
operator. There are several well- If the pixels detected fall on a

known gradient filters. In this straight line then they can be

experiment we use the Sobel expressed by the equation:

gradients, which are obtained by Y=mx

convolving the image with two +c

kernels, one for each direction. The basis of the Hough

transform is to translate the

CROSSROAD PATTERN points in (x, y) space into (m,c)

space using the equation:
DETECTION
c= (-x)
The zebra crossing has
m+y
alternate white bands running
Thus each point in (x,y) space
across the width of the road.
(i.e. the image) represents a line
This pattern has to be recognized
in (m,c) space. Where three or
to confirm the presence of a
more of these lines intersect a
crossing. To detect basic shapes
value can be found for the
within the image, we make use
gradient (m) and intercept (c) of
of the Hough transform. At its
the line that connects the (x,y)
simplest the Hough transform
space points.
can be used to detect straight

CALCULATION OF Calculation of the width

THE WIDTH OF THE of the road is based on the

concept of projection invariants.
ROAD AND TIME
This requires us to define the
REQUIRED TO CROSS
term Cross Ratio.
IT
L1

P4
L2
 P3
P2 L3

P1 L4

The cross ratio can defined for the four collinear points as,
(P1, P2, P3, P4) = (P1P3/P2P3)/(P1P4/P2P4)
Where P1P2 is the distance P1 to P2. The cross ratio of the four lines is
given by,
(L1, L2, L3, L4) = (sin13/sin23)/ (sin14/sin24)
Where ij is the angle between Li and Lj. L1

L2
P1

L3
P2
P3 L4
P4

“projective invariant” which

In the above figure, lines
takes the distance between the
are constructed from the
white lines and a set of linear
collinear points and in the
points on the edges of the white
adjacent figure a line is formed
lines. The system effectively
by joining the points in the lines
draws a virtual line out into the
L1, L2, L3, L4.
road. If a crosswalk is present,
A useful fact is that the cross-
the edges of the painted white
ratio of the original four points is
lines will form a predictable
equal to the cross-ratio of the
series of points along the virtual
constructed lines.
line.
To detect the presence of
a zebra crossing we use the
 Let M and N be two For an arbitrary point (,) 
distinct points of the projective (1,0) we can rescale (,) to
space. Here we take points M =1 and represent A by its
and N as the points on the edges ``affine coordinates'', (, 1) or
of the line formed on the image.
just  for short. Since we have
The projective line between M
mapped M to infinity, this is just
and N consists of all points A of
linear distance along the line
the form
from N.
The time required to cross the

Here (,) are the coordinates of road is calculated based on an

A in the 2D linear subspace assumption that the user covers a

spanned by the coordinate distance of one foot in a minute

vectors M and N. N is on an average. So, the time

represented by (0, 1) and serves required to cover the calculated

as the ``origin'' and M is distance is calculated based on a

represented by (1, 0) and serves simple logic.

as the ``point at infinity''. Generally, the time taken, T, to

cross the road can be found out
by

T= Calculated width of the road

Distance covered by the user in one second

TRAFFIC LIGHT the traffic light must be detected

DETECTOR and compared with the time

required by the user to cross the
The function of the
road. If the user can cross the
traffic light detector is to
road safely, the voice speech
recognize if the pedestrian light
system will instruct him to cross
is on for the user to cross the
the road. This process can be
road. If the user happens to reach
effectively done by having an
the road when the pedestrian
image database in the system
light is already on, the time
and comparing the obtained
indicated by the timer display in
image of camera to detect if how fast a planar contour is
pedestrian light is on and to turning. Contour evolution is
detect the time left to cross the achieved by first parametrizing
road. using arc length. This involves
sampling the contour at equal
We have a large number of intervals and recording the 2-D
images and wish to select some coordinates of each sampled
of them, which are similar to a point. The result is a set of 2
certain image (for example, the coordinate functions (of arc
image of the pedestrian light). length), which are then
So we need a content-based convolved, with a Gaussian filter
image database system, which of increasing width or standard
accepts an image as its input and deviation. Next step is to
retrieves all images like that by compute curvature on each
using some image properties smoothed contour.
such as color, texture, shape and As a result, curvature zero-
keywords. crossing points can be recovered
Every image is processed to and mapped to the CSS image in
recover the boundary contour, which the horizontal axis
which is then represented by represents the arc length
three global shape parameters parameter on the original
and the maxima of the curvature contour, and the vertical axis
zero-crossing contours in its represents the standard deviation
Curvature Scale Space image. of the Gaussian filter.
The features recovered from a
CURVATURE SCALE CSS image for matching are the
SPACE COMPUTATION
maxima of its zero-crossing
AND MATCHING
contours. The matching of two
The CSS image is a
CSS images consists of finding
multi-scale organization of the
the optimal horizontal shift of
inflection points (or curvature
the maxima in one of the CSS
zero-crossing points) of the
images that would yield the best
contour as it evolves. Intuitively,
possible overlap with the
curvature is a local measure of
maxima of the other CSS image.
The matching cost is then a match with an image in the
defined as the sum of pair wise camera, the pedestrian can cross
distances (in CSS) between the road. The time in seconds
corresponding pairs of maxima. required to cross the road is also
So, if an image of a pedestrian detected based on the image of
light in the image database finds numbers in the database.

OBTAINED IMAGE IMAGE DATABASE

TIMING UNIT PEDESTRIAN

T < T1, the LIGHT DETECTION
system instructs the

The timing unit compares user to cross the road. Else it

the calculated value T, the time asks toTIMER

wait till UNIT
it is safe to cross

required by the user to cross the the road.

road with the time left to cross

the road T1, as identified from
the image (traffic signal time). If

PEDESTRIAN LIGHT
DETECTION

TIMER UNIT

sound pattern, consisting of a

VOICE SPEECH combination of frequencies

based on that specific set of
SYSTEM
Pixels. The result is an
AUDITORY IMAGE
auditory signature effectively
REPRESENTATION
an inverse spectrogram that
The images captured by characterizes the particular
the camera are swept from left to image.
right at little less than one image High-level scene
per second. The pixels in each interpretation applied to the
column generate a particular processed images will produce a
 symbolic description of the sound renderings. If we have a
scene. The symbolic description 64 * 64, 16 gray tone image, the
is then 64-channel sound synthesis
maps the image into an
exponentially distributed
frequency interval for a one
converted into verbal second visual sound. The
instructions appropriate to the VOICE mapping: vertical
needs of the user. positions of points in a visual
sound are represented by pitch,

VOICE VISION while horizontal positions are

represented by time-after-click.
The VOICE VISION technology
Brightness is represented by
for the totally blind offers the
loudness. In this manner, pixels
experience of live camera views
become... voices!
through sophisticated image-to-

FUNCTIONAL BLOCK START

CAPTURE
THE IMAGE
IMAGE
ANALYSER

ANALYSE
THE
IMAGE

IS
YES THERE ANY NO
CROSS ROADS

INFORM THE
USER THAT A CROSS
CROSSROAD IS ROAD
DETECTED DETECTOR

CALCULATE
THE WIDTH
OF THE ROAD
DETECT THE TRAFFIC
LIGHT

IS THE INFORM THE

YES LIGHT NO USER TO WAIT
FAVOURABLE? TRAFFIC
LIGHT
CALCULATE
THE TIME DETECTOR
TAKEN TO
CROSS THE
ROAD (A)
IDENTIFY THE TIME
LEFT IN THE
TRAFFIC SIGNAL (B)
YES IF NO
A<B
TIMER
INFORM THE INFORM THE UNIT
USER TO CROSS USER TO
THE ROAD WAIT

STOP

CONCLUSION potential solutions. A

sophisticated mechanism
The development of mobility designed to enhance the mobility
aids for the visually impaired is of the blind is intended to help
a challenging task that has many people who cannot recover their
eyesight by normal medical
procedures. Blind pedestrians in REFERENCES
the greatest danger are those
who must cross wide, busy  The HiPODS Board for
roads. This system along with Nanotechnology led by Dennis
the available low technology Quan
aids can relieve the visually  The Incubation Solutions
challenged of being dependent Team that owns the
on others and lead normal lives. Technological strategies led by
This effective navigation system Jose Vargas
would improve the mobility of  The Innovation Factory
millions of blind people all over team led by Jeff Coveyduc
the world.