Center of Circle After Perspective Transformation
Center of Circle After Perspective Transformation
A BSTRACT
Video-based glint-free eye tracking commonly estimates gaze direc-
tion based on the pupil center. The boundary of the pupil is fitted
with an ellipse and the euclidean center of the ellipse in the image is
taken as the center of the pupil. However, the center of the pupil is
generally not mapped to the center of the ellipse by the projective
camera transformation. This error resulting from using a point that is
arXiv:1902.04541v1 [cs.CV] 12 Feb 2019
not the true center of the pupil directly affects eye tracking accuracy.
We investigate the underlying geometric problem of determining
the center of a circular object based on its projective image. The
main idea is to exploit two concentric circles – in the application (a) (b)
scenario these are the pupil and the iris. We show that it is possible
to computed the center and the ratio of the radii from the mapped
concentric circles with a direct method that is fast and robust in
practice. We evaluate our method on synthetically generated data
and find that it improves systematically over using the center of the p r
fitted ellipse. Apart from applications of eye tracking we estimate
that our approach will be useful in other tracking applications. R
It is becoming more and more common to include video cameras in Figure 1: A pair of ellipses (a) as the image of a pair of concentric
head mounted stereo displays to enable eye tracking in virtual reality circles (b) in perspective. Taking the data from a conventional ellipse
environments. In this setting, the optical axis of the eye is usually detection tool (shown in axes in (c)), a simple algorithm proposed in
determined based on the center of the pupil, which can be extracted this paper locates the true center p respecting the perspective, and
from the video images. Identification of the pupil in the image stream determines the radii ratio R/r of the inferred circles (d).
is commonly done by fitting an ellipse to the boundary between
the dark pupil and much lighter iris. This approach introduces two
sources of error: 1) the center of the ellipse in the image is not the however, all of them give rise to the same projective center, and also
center of the pupil, because the camera transformation is projective all of them agree on the ratio of the radii. In other words, based on
(and not just affine); 2) the refraction at the cornea distorts the the pair of ellipses, the center of the concentric circles and the ratio
pupil shape in addition to the projective camera transformation. The of the radii are uniquely determined.
problems from refraction can be circumvented in other situations The set of concentric circles is an instance of a pencil of conics.
by additional tracking equipment [7]. Without such equipment it One of the degenerated circles in the group corresponds to the true
may be possible to compensate the effect based on a computationally center and it is invariant under projective transformation. In Section 4
involved inverse model [6]. We focus on the first problem, namely we explain this concept and show how it leads to a simple formulation
the non-affine mapping of the center. To our knowledge, this is the for a computational approach.
first work providing a computational approach estimating the true We apply our method to estimate the true pupil center by using a
pupil center in the context of video-based eye tracking. pair of ellipses, i.e., the pupil ellipse and the iris ellipse. Tested with
The fact that the center of a circle is not mapped to the center of synthetic data, we show that our method provides robust estimation
an ellipse under projective transformations is illustrated in Figure 1: of pupil center with respect to projective distortion (see Section 5).
The pair of ellipses in (a) is the result of perspectively projecting Compared to the pupil center estimated from ellipse fitting our esti-
the two concentric circles in (b) to the image plane. The centers of mation shows significant improvement, less than one pixel distance
the two ellipses are not coincident (c), and neither coincides with to the true pupil center in most cases.
the projected center p of the concentric circles. As we explain later, Apart from eye tracking, our method can also be used for related
there are in fact many projective transformations that would map tasks in computer vision. The idea of exploring the projective
a pair of concentric circles to the two ellipses found in the image, invariant properties of concentric circles is not new. It has been used
for markers consisting of the two concentric circles specified by a
* Preliminary work.
† e-mail:[email protected]
ring [15] or localization in robotics applications [8]. Compared to the
‡ e-mail:[email protected]
iterative optimization techniques our formulation as an eigenproblem
§ e-mail:[email protected]
is more direct.
error (pixels)
error (pixels)
A0 = L−| AL| = Q02 Q0−1
1 = 0
1 0 . 1.00 1.00
0.75 0.75
0 0 R2/r2
0.50 0.50
0.25 0.25
Thus the invariant eigenvalue ratio of A is given by λ1 : λ2 : λ3 = 0.00
30 40 50 60 70
0.00
30 40 50 60 70
1 : 1 : R2/r2 . Therefore, the radii ratio R/r is encoded in the eigenvalues theta (degree) theta (degree)
λ1 = λ2 , λ3 of A. (a) (b)
In practice, when the two conics Q1 , Q2 are detected with mea- 2.00 2.00
Our estimation Our estimation
surement error, the spectrum of A may only have an approximated 1.75 Euclidean center 1.75 Euclidean center
1.50 Image fitting 1.50 Image fitting
double λ1 ≈ λ2 , λ3 . In that case we retrieve the radii ratios of the
1.25 1.25
error (pixels)
error (pixels)
concentric circles by one of the following symmetrizations 1.00 1.00
s s s 0.75 0.75
λ32 0.50 0.50
R λ2 λ3 λ1 λ3
≈ 2
≈ 2
≈ . 0.25 0.25
r λ1 λ2 λ1 λ2 0.00 0.00
30 40 50 60 70 30 40 50 60 70
theta (degree) theta (degree)
5 R ESULTS (c) (d)
We apply our method to estimate the true pupil center using the pupil Figure 2: Estimation errors in different camera positions. x axis
ellipse and the iris ellipse and evaluate it on synthetic data. corresponds to rotation angle θ in degree and y axis is the estimation
error measured in pixels. From (a) to (d), rotation angle φ changes
5.1 Experimental Setup from 10◦ to 40◦ . In each plot, θ varies from 30◦ to 70◦ .
Synthetic eye images are rendered using the 3D model proposed
in [21] with cornea refractive index set to be 1.0 (see Figure 3 for
examples). The true projected pupil center in the camera image is angle of the eye measured in degree. As expected, large pupil size
directly computed from the 3D model, and we also compute the leads to large estimation error. All estimation methods get worse
pupil ellipse and the iris ellipse in the image plane. The pupil center but our method consistently provides the most accurate estimation.
estimated by our method is compared to 1) the Euclidean center of Once again, image based ellipse fitting method fails when the eyes
the pupil ellipse in the image plane and 2) the pupil center estimated are tilted to some extend as shown by the second image in each plot.
by image based ellipse fitting following the method proposed in [20].
Eye camera is placed 3 cm in front of the eye, similar to the
camera position in a mobile eye tracker. We experiment with various 6 D ISCUSSION AND F UTURE W ORK
viewing angles (rotations of the camera), and three different pupil We introduce a simple algorithm to robustly estimate the true center
sizes when the eyes fixate at various targets. Fixation targets are position from a pair of ellipses projected from a pair of concentric
evenly sampled from a circle that is placed perpendicular to the circles. We apply our method in the context of eye tracking and use
ground. We compare the Euclidean distance in pixel unit between the it to find the true pupil center in the image plane. Evaluation based
true pupil center in the image plane and the estimated pupil center on synthetic data shows promising results, especially comparing to
using different methods. the estimated center of the fitted ellipse. Even though we did not
evaluate the performance of the method on the estimation of pupil
5.2 Camera Rotation
size, we believe it is possible to estimate the pupil size using the
With a fixed distance to the eye, the camera is placed at a set of radii ratio.
locations that are possible in practice. We use spherical coordinates Despite the fact that our method can accurately estimate the true
to describe the rotations. Polar angle φ defines the rotation angle pupil center under projective distortion, it does not consider refrac-
between the camera and the horizontal plane, and azimuthal angle θ tion, making it unsuitable for real-world eye tracking applications.
describes the camera rotation in the horizontal plane. However, note that our formulation allows us to estimate the radii
Figure 2 shows the estimation errors using different methods. ratio of two concentric circles, the pupil and the iris in this case,
We test with φ varying from 10◦ to 40◦ and θ varies from 30◦ to and the iris boundary is much less affected by the refraction at the
70◦ . In all tested scenarios, our method gives the best estimation corner. Theoretically we could use the estimated ratio to find the
with less than one pixel distance to the true pupil center. Estimated exact position of the pupil center as well as the iris center in the
Euclidean centers of the pupil ellipse deviate away from the true camera image. In other words, this would allow us to implicitly
pupil center. Image based ellipse fitting gives better estimation as model the corner refraction and find the true pupil center under both
more sample points from the boundary detection are used for ellipse distortions in the real scenarios.
fitting. However, the fitting fails when θ is large. In such cases, the
Beyond the scope of eye tracking, concentric circles pattern
projected pupil is small in the camera image, subsequently with less
has been commonly used as fiducial markers in computer vision
camera distortion. Therefore, the estimated pupil positions get closer
related tasks [4, 5]. Since detection accuracy and speed are crucial
to its true position. As shown in Figure 2, estimation errors decreases
for fiducial marker based real-time application, our method could
with an increasing θ from left to right in each plot.
provide another option for fiducial marker based tracking. We are
5.3 Pupil Size eager to investigate in this direction in future work.
As we see from previous test, the estimation accuracy of the Eu-
clidean ellipse center is correlated to the pupil size in the image plane. R EFERENCES
In this second experiment, we experiment with three different pupil [1] K. Alberto Funes Mora and J.-M. Odobez. Geometric generative gaze
sizes and compare the estimations when the eyes look at different estimation (g3e) for remote rgb-d cameras. In The IEEE Conference on
targets. We evenly place 36 targets on a circle that lies perpendicular Computer Vision and Pattern Recognition (CVPR), June 2014.
to the ground. Figure 3 shows the results where estimation error [2] D. A. Atchison, G. Smith, and G. Smith. Optics of the human eye.
measured in pixel is the y-axis and x-axis corresponds to the rotation Butterworth-Heinemann Oxford, 2000.
5 Our estimation 5 Our estimation 5 Our estimation
Euclidean center Euclidean center Euclidean center
4 Image fitting 4 Image fitting 4 Image fitting
error (pixels)
error (pixels)
error (pixels)
3 3 3
2 2 2
1 1 1
0 0 0
0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350 0 50 100 150 200 250 300 350
angle (degree) angle (degree) angle (degree)
Figure 3: Estimation errors with various pupil size. Camera is placed at a fixed position in front of the eye while fixation targets are evenly sampled
from a circle in front. Pupil size increases from left to right and embedded eye images show examples when the eyes are rotated to the left and
right. x axis corresponds to rotation angle of the eyes in degree and y axis is the estimation error measured in pixels.