International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT) - 2016

A Device Controlled Using Eye Movement

Prof. Prashant Salunkhe Miss. Ashwini R. Patil

Assistant Professor E&TC ME Student E&TC
Sandip Institude of Technology and Research Center, Sandip Institude of Technology and Research Center,
Nashik, MS, India Nashik, MS, India
[email protected] [email protected]

Abstract — Research carried out in this technical paper techniques are present in section 2. Section 3 shows system
suggests an application of the widely implemented eye tracking implementation and construction details. Experimental result
techniques. Traditionally, HCI uses mouse, keyboard as an input is present in section 4. Section 5 shows Algorithm. Section 6
devices but this paper presents hand free interface between includes conclusion and at last References.
computer and human. Here providing a novel idea to control
computer mouse cursor using human eyes movement. It controls
mouse moving by automatically affecting the position where II. EYE TRACKING TECHNICES
eyesight focused. The project mainly comprises of three sections An eye tracker is a device for measuring eye positions and
namely Image Capture, Image Processing, and Cursor Control. eye movement. In any study, the selection of the technique
After capturing image from webcam, the shape of pupil is rests with the actual demands of the application. During the
recognized using Hough Transform and the center coordinate analysis phase of this research, three techniques were
help to determine the exact point on the screen where the user is analyzed; the Limbus tracking, Pupil tracking, and third tech is
looking also coordinate will instruct the computer mouse to move Electrooculography. Each technique has its own Strong points
specific location. This system is very helpful for solving the HMI
and drawbacks.
problems of the disabled so that it can provide a way for them to
communicate with the outside world.
A. Limbus
Keywords— Eye tracking, Hough Transform, Eye-blinking Limbus Tracking explains a method of tracking the eye
detection, Image processing. using the limbus. The limbus is the boundary between the
white sclera of the eye and the darker iris.
I. INTRODUCTION Limbus Sclera
Disable people who cannot move anything except their
eyes. For these people eye movement and blinks are the only
way to communicate with outside word through computer.
This research aims in developing a system that can aid the
physically challenged by allowing them to interact with a
computer system using only their eyes. Human- computer
interaction has become an increasingly important part of our
daily lives. Here eyes as an input, the movement of user’s eyes
can provide a convenient, natural and high- bandwidth source
of input. There is no universal method to track the eye
movement. We are using eye tracking is technique for Iris
measuring the gaze or movement of an eye this helps identify Pupil
where the user is looking and has various applications [1]. The Fig. 1. Diagram of Eye
project mainly comprises of three sections namely Image
Capture, Image Processing, and Cursor Control. Project is As the sclera is white and the iris is darker, this boundary
composed of a webcam, infrared LED for image capture. can easily be optically detected and tracked. This technique is
MATLAB is a powerful tool for image processing because it based on the position and shape of the limbus relative to the
provides so many inbuilt functions in toolbox etc head, so either the head must be held quite still or the
implementing shape detection using Hough Transform has apparatus must be fixed to the user's head. This technique is
been made a lot easier by the updated image processing negatively affected by the eyelid often concealing all or part of
toolbox ver. 8.0 [2]. After track the movement of eye pupil, the limbus. This makes its uses restricted to horizontal
with the help of Matlab and co-ordinate of eye a computer tracking. Generally this method does not involve the use of
mouse to move to a specific location on the screen. The infra red light [3].
remainder of this paper is organized as follows eye tracking

978-1-4673-9939-5/16/$31.00 ©2016 IEEE

732
B. Pupile III. CONSTRACTION DETAILS
Pupil tracking is a method of gaze detection that is
commonly used often in conjunction with other forms of A. Hardware
tracking. There are several reasons for this, but the main This project is required minimum hardware. Camera,
advantage is the notion of the “bright spot”. Similar to the case infrared LEDs and a Laptop. The purpose of this project is to
of red eye when taking flash photographs at night, infrared can achieve complete functionality of the mouse, just by the
be used in pupil detection to create a high intensity bright spot movement of the eye.
that is easy to find with image processing. This bright spot
occurs when infrared is reflected off the back of the pupil and 1) Camera: In this project Logitech Webcam use for
magnified by the lens. The main advantage of pupil tracking is capture image which are having picture quality 8MP so that it
that as the border of the pupil is sharper than the limbus, a is provide good quality of picture in better light condition. But
higher resolution is achievable. Also, as the pupil is never in bad light condition image quality is also bad because of this
really covered by the eyelid, x-y tracking is more feasible as bad input system is not work properly. Solve this problem by
compared to Limbus tracking. The disadvantage is that the using IR LED [6].
difference in contrast is lower between the pupil and iris than 2) IR LED: The camera module includes a near-infrared
between the iris and sclera-thus making the border detection (NIR) LED to illuminate the eye. Under NIR illumination the
more difficult [4]. iris appears lighter than under visible light which has the
desirable effect of increasing the contrast between the pupil
C. Electro-oculography and the iris [7]. Using infrared also enables the camera to
The third category uses electric potentials measured with capture video in the dark where visible illumination would
electrodes placed around the eyes. The eyes are the origin of a otherwise interfere with the wearer’s vision. Some IR are
steady electric potential field, which can also be detected in harmful to human eye because of high current so here we use
total darkness and if the eyes are closed. It can be modeled to
be generated by a dipole with its positive pole at the cornea resister and connect this to IR to minimize the current.
and its negative pole at the retina. The electric signal that can
be derived using two pairs of contact electrodes placed on the
skin around one eye is called Electrooculogram. If the eyes
move from the centre position towards the periphery, the
retina approaches one electrode while the cornea approaches
the opposing one. This change in the orientation of the dipole
and consequently the electric potential field results in a change
in the measured EOG signal. Inversely, by analyzing these
changes in eye movement can be tracked. Due to the Fig. 2. The original frame, pic using IR LED
discretisation given by the common electrode setup two
separate movement components – a horizontal and a vertical – 3) Mount: In order to create a comfortably wearable
can be identified. A third EOG component is the radial EOG mount we can use the frame of a pair of any random
channel, which is the average of the EOG channels referenced sunglasses and attach the camera on it to detect the eye from a
to some posterior scalp electrode. This radial EOG channel is
close range.
sensitive to the saccadic spike potentials stemming from the
extra-ocular muscles at the onset of saccades.
Due to potential drifts and variable relations between the
EOG signal amplitudes and the saccade sizes make it
challenging to use EOG for measuring slow eye movement
and detecting gaze direction. EOG is, however, a very robust
technique for measuring saccadic eye movement associated
with gaze shifts and detecting blinks. Contrary to video-based
eye-trackers, EOG allows recording of eye movements even
with eyes closed, and can thus be used in sleep research. It is a
very light-weight approach that, in contrast to current video-
based eye trackers, only requires very low computational
power, works under different lighting conditions and can be
implemented as an embedded, self-contained wearable system. Fig. 3 Hardware Setup
It is thus the method of choice for measuring eye movement in
mobile daily-life situations and REM phases during sleep. The 4) Distance from the screen to the user: This distance
major disadvantage of EOG is its relatively poor gaze needs to be optimized appropriately in order to be able to
direction accuracy compared to a video tracker. accurately differentiate the different positions of the eye and
properly track the centre. If this distance is too large then the
movement of the pupil will be very less for traversing the
range of the screen hence it will become difficult to accurately

733
track the centre position. If the distance is too less than the eye
movement will be too large and because of the curvature of
the eye.
B. Software
This project has been implemented using MATLAB
version 2013a. We use basic principles of image processing
like Hough transform for shape recognition and circle
detection also colour recognition for the detection of an eye
Fig. 5 Gray Image of Eye
pupil and move cursor.
1) Hough transform: The Hough transform is a technique Convert Gay image to BW using Canny edge method and
which can be used to isolate features of a particular shape use Hough transform for circle detection for thresholding, a
within an image. Because it requires that the desired features proper threshold value needs to be selected to obtain a well
filtered binary image of the pupil so we are converting image
be specified in some parametric form, the classical Hough
to binary image. Using binary image find circle and center co-
transform is most commonly used for the detection of regular ordinate it is easy to track exact point where the user is
curves such as lines, circles, ellipses, etc. [8]. looking then MATLAB use these co-ordinate to instruct a
2) Canny edge detection: The Canny method finds edges computer mouse to move to a specific location on the screen.
by looking for local maxima of the gradient of image. The Here we are using java package. This package is used to
gradient is calculated dosing the derivative of a Gaussian generate native system input events for the purposes of test
filter. The method uses two thresholds, to detect strong and automation, self-running demos, and other applications where
weak edges, and includes the weak edges in the output only if control of the mouse and keyboard is needed.
they are connected to strong edges. This method is therefore
less likely than the others to be fooled by noise, and more
likely to detect true weak edges [9].
IV. EXPERIMENTAL RESULT
The experimental aimed is to move cursor just by the
movement of eye. The project is divided three section image
capture processing and mouse movement. Adequate
illumination need to be used for capturing an image which will
provide a well defined shape of the pupil and this is done
using IR LED which is mounted on either side of the camera Fig. 6 Binary Image of Eye using Hough Transform
lenses. It is invisible to the naked eye and hence will not cause
any strain to the user. Well-defined image of the eye pupil as For click action here give some time period i.e 0.5 to 0.10
shown in image below. sec. at this time cursor is stay any folder then this folder is
open or closed. Thus the user achieves complete control of the
cursor movement of the mouse through eye movement. Final
result image is represented by following way.

Fig. 4 Eye Image in RGB Format

Here first section is complete for next section i.e image

processing we use many inbuilt function in toolboxes like
Image processing toolbox, Computer system vision toolbox
etc. Capture image is rgb format convert this image to gray
image for finding center pt firstly finds size of image i.e row
and colume then easily find center by row/2 and colume/2. Fig. 7 Detect pupil in gray and binary image

734
 V. ALGORITHM interface includes hardware and software components. This
Input – Eye video input in RGB format. system also used in industrial control, robotics, medical,
advertising, Psychology and so on.
Output – Detect eye, Cursor move.
• Step 1 – Get video in RGB format by using function Acknowledgment
videoinput.
The authors thankful to Principal Prof. Dr. S. T. Gandhe,
• Step 2 – Get snapshot from camera in rgb format by H.O.D. Prof. Mrs. G. M. Phade of Sandip Institude of
using function getsnapshot. Technology and Research Center, for their support.
• Step 3 – Convert image rgb to gray using function
egb2gray. References
[1] Z. W. Zhu and Q. Ji.: Novel eye gaze tracking techniques under natural
• Step 4 – Find size of image using function size( ) after head movement. IEEE Trans. Biomed. Eng., vol. 54, no. 12, pp. 2246–
getting row and colume, for finding center pt use 2260, Dec. (2007)
function round(row/2) [2] Jianbin Xiong, Weichao Xu, Wei Liao, Qinruo Wang, Jianqi Liu, and
Qiong Liang.: Eye Control System Base on Ameliorated Hough
• Step 5 – Convert gray image to BW using canny edge Transform Algorithm. IEEE Trans. Vol. 13, no. 9, pp. 3421-3429,
method. September (2013)
[3] K. Abe, Shoich Ohi, M.Ohyama.: Eye – Gaze detection by image
• Step 6 – Hough transform are use for detect lines. analysis under natural light. Springer – verlag Berlin Heidelberg, pp.
• Step 7 - Plot x axis and y axis using subplot() function. 176-184, (2011)
[4] Kumar N, Kohlbechers, Schneider E.: A Novel Approach to video –
• Step 8 – Convert image to binary using function based Pupil Tracking. IEEE Int. Conf. on San Antonic, pp. 1255-1262,
im2bw. 14 oct 2009.
[5] Yingxi Chen.: Design and evaluation of a human-computer interface
• Step 9 – Use bwareaopen function for remove small based on electro-oculography. (2003)
objects from binary image. [6] C. H. Morimoto, A. Amir, and M. Flickner.: Detecting eye position and
gaze from a single camera and 2 light sources. in Proc. 16th Int. Conf.
• Step 10 – Rectangle function are use to find circle Pattern Recognit., pp. 314–317, (2002)
shape from eye image. [7] J. Daugman.: How iris recognition works. IEEE Trans. circuits syst.
video Techno 1, Vol. 14, no.1, pp. 21-30, Jan (2004)
• Step 11 – Use java package for cursor move according [8] Al-Rahayfeh, Faezipour. M.: Enhanced Frame rate for real–time eye
to x and y axis value. tracking using circle Hough Transform. LISAT, IEEE, pp. 1-6, 3 May
(2013)
• Step 12 – Assign the title of eye movement i.e top left,
[9] http://www.mathworks.in/help/images/edge-detection.html
top right, bottom left, bottom right.
[10] Kashani, Arani, Fini.: Eye detection and tracking in image with using
• Step 13 – Stop video using function stop( ) bag of pixel ”,Communication software and network (ICCSN). IEEE 3rd
Int. Conf., pp 64-68,May (2011)
• Step 14 – Flushdata use to remove buffered frames [11] Fini. MRR, Kashani. MAA, Rahmati, M.: Eye detection and tracking in
from memory. image with complex background. Electronic computer tech, 3rd Int.
Conf. IEEE, pp 57-61, 10 April (2011)
[12] Junghoon park, Taeyoung jung, Kangbin yim.: Implementation of an eye
VI. CONCLUSION gaze tracking system for the disabled people. AINA, IEEE 29th Int.
This paper focused on the analysis of the development of Conf., pp 904-908, 27 March (2015)
the ‘’Device controlled using eyes movement’’. Advantage of [13] Khairos Faizal, Noraini. A. J.: Eyes detection in facial image using
circular Hough Transform. SPIA, IEEE, pp. 238-242, 8 March (2009)
this system is providing computer access for people with
[14] J. H. Yim, Jeong Min Hwang, Jang. Moseo.: Wearable video-based eye
server disabilities. In this paper we describe Eye tracking tracking system with a webcam. Springer, pp. 267-270, (2009)
technology. The most unique aspect of this system is that it
[15] Mscislow Srutek and Lukasz Matblszak.: Eye tracking system for
does not require any complicated wearable attachments. This Human Computer Imteraction. Spriger.
makes the interaction more efficient and enjoyable. The

735