CIRCULAR MARKERS FOR CAMERA POSE ESTIMATION

Alain Pagani, Johannes Koehler and Didier Stricker

Augmented Vision - DFKI

ABSTRACT
This papers presents a new system using circular markers to
estimate the pose of a camera. Contrary to most markers-
based systems using square markers, we advocate the use of
circular markers, as we believe that they are easier to detect
and provide a pose estimate that is more robust to noise. Un-
like existing systems using circular markers, our method com-
putes the exact pose from one single circular marker, and do
not need specific points being explicitly shown on the marker
(like center, or axes orientation). Indeed, the center and ori-
entation is encoded directly in the marker’s code. We can Fig. 1. Circular marker for Augmented Reality
thus use the entire marker surface for the code design. After
solving the back projection problem for one conic correspon- tion of the marker through its code, disambiguation in a small
dence, we end up with two possible poses. We show how to set of hypothetical poses and extraction of the correct local
find the marker’s code, rotation and final pose in one single orientation. These steps are completed efficiently thanks to a
step, by using a pyramidal cross-correlation optimizer. The pyramidal search of the best cross-correlation among possible
marker tracker runs at 100 frames/second on a desktop PC marker codes.
and 30 frames/second on a hand-held UMPC. Related work. Most systems use square markers [2, 1, 4,
3]. To the best of our knowledge, only two systems use cir-
1. INTRODUCTION cular markers: Intersense’s VIS-Tracker ([5]) and a subpart
of Cantag [3]. In [5], the system treats the markers as sin-
Markers can be used in many applications, like e.g. Aug- gle points, and at least four markers have to be recognized in
mented Reality (see Figure 1). Since the early years of marker- an image to recover the camera pose. Moreover, the marker
based pose estimation, several marker designs and pose esti- center is approximated by the center of the ellipse. The subse-
mation approaches have been proposed. Among them, a vast quent error can be neglected if the markers are small, but leads
majority uses square markers [1, 2, 3, 4], as the pose esti- to wrong poses when markers are seen in close up scenes.
mation algorithms for points or lines are easier to implement On the contrary, our method can recover the pose with any
and to maintain. However, we believe that the use of circular number of visible markers, and we use the correctly projected
markers has several advantages over square markers. First, center of the marker to recover the pose. Moreover the marker
the perspective projection of a circle is an ellipse, which is design proposed by Foxlin et al. explicitly shows the center of
easy to detect and to fit in the image. Second, the pose esti- the marker as a white disk, and the two axes of the ellipse are
mation of a square marker relies on a good estimation of only given by black disks (”eyes” in the marker). This inevitably
four points (the four corners) whereas the whole ellipse can be reduces the payload of the marker, as less space is left for the
used in the case of circular markers. Third, the case of occlu- marker’s code.
sion is easily treated for circular markers, as an ellipse fitting The use of circular markers is also possible in [3]. The
algorithm does not require the entire contour to be visible. marker design has a white disk in the center, which is used to
In this paper, we present our circular markers, and show recover the camera pose. Cantag has been specified to com-
how to use them to compute the pose of the camera. The pare different markers designs, and the authors suggest that
novelty of our approach resides in the fact that our markers circular markers give better results than square ones. Several
are defined only by their circular contour and a 16-bit code methods have been proposed to calibrate a camera and com-
(see Figure 2). No indication of the marker’s center nor the pute its pose from circular patterns. In [6], a camera is cali-
marker orientation on its underlying plane is needed. This brated using two concentric circles. [7] use the image of the
was made possible thanks to our new code extraction strat- absolute conic to compute the pose from one imaged circle.
egy, which solves several issues at the same time: identifica- However, the center of the circle has to be provided. In [8],
 These two steps are further detailed in the next subsections.
Contrast
enhancement
Contour extraction Dimensionality reduction from one imaged circle. We
4 3
5 2
now show how to reduce the pose computation to a one di-
6 14 13 1 u Partial pose
7 15 16 12
Ellipse fitting
estimation mensional problem when the image of a circle has been found.
8
9 10
11 In this derivation, we follow the work of [10], where further
Marker identification and pose refinement
(pyramidal cross correlation) details can be found. We assume that the camera follows the
pinhole camera model and is calibrated (the calibration ma-
trix K is known). Without loss of generality, we can therefore
Fig. 2. (left) Marker design. (right) Our pipeline assume that the image plane has equation Z = 1 in the cam-
era coordinate system and that the image is centered on the
principal point (see Figure 3). We define the Marker Coordi-
two concentric circles and a extra orientation mark are nec- nate System (MCS) as centered on the marker center, having
essary to find the pose of the camera. [9] proposes the com- a Z axis perpendicular to the supporting plane and a X axis
putation of the camera pose and calibration from the image defined by the marker’s orientation u. The Intermediate Coor-
of circles, but at least two circles have to be visible in the im- dinate System (ICS) is by definition a translation of the MCS
age. In [10], a camera is calibrated using two coplanar circles. so that the origin is at the camera optical center (see Figure
Our approach is different from these, in that we use only one 3).
circle without explicit orientation nor center to compute the
camera pose. The main idea is that a one-dimensional family Z
Rigid Body
of possible poses can be computed from one single ellipse. Coord. System

We then find the correct pose in this family while identifying X

Intermediate
the marker. Coord. System
The remainder of this paper is organized as follows: Sec- Z
Y
tion 4 presents the pose estimation algorithm from one marker. Image plane

Section 3 shows how the correct marker identification is found Z Camera

Coord. System
along with the final pose and rotation. In Section 4 we show Z
Y
how ton compute the pose of a set of markers. We show our X t
results in Section 5 before concluding. X Marker

Y Y

2. POSE ESTIMATION FROM ONE MARKER Marker Coord. System

X

Marker definition. Our marker is a black circle and consists

on three rings with equal width. The outer ring is always
black and serves for the marker localization. The inner ring is
divided into 4 angular bins and the middle ring into 12 angular Fig. 3. Two views of the oblique elliptical cone
bins. Each bin can be white or black. Thus, the code can store
16 bits of information. When designing markers, care has to Let us define the unknown transformation from the MCS
be taken that the code does not have a periodicity (other than to the CCS by the rotation matrix and the translation vector
2Π), as the marker orientation u has to be found from the (R, t), such as a point XM in the MCS transforms to XC in the
code itself. Figure 2 shows the marker with its 16 bits and the CCS as
principal orientation u. XC = RXM + t (1)
Image processing pipeline. In this section, we show how We will now show how to retrieve the correct R and t from
we can recover the marker pose pose from one single view image measurements. The idea of the derivation is that the
of the marker. Figure 2 shows the image processing pipeline oblique cone defined by the bundle of straight lines (i.e. the
we apply to each video frame. In order to better distinguish light rays) passing through the optical center, the ellipse and
the markers even under difficult light conditions, we first ap- the circle can be described in two different manners, once in
ply a contrast enhancement function to the image, based on the CCS, once in the ICS (see [10] for details).
homomorphic image processing [5]. Edges in the image are An ellipse found in the image is defined by a symmetric
then extracted using a Canny filter with non maximal suppres- matrix C. In the CCS, a point XC is on the cone if
sion (NMS). A SVD-based ellipse fitting algorithm is applied T
to all found closed contours, and we keep only the found el- XC CXC = 0 (2)
lipses for which the error returned by the fitting function is In the ICS, a 3D point XI is on the cone if
small. For each remaining ellipse, we then apply first a par-
tial pose computation, then a marker identification algorithm. XIT QXI = 0 (3)
where values on points regularly distributed over two rings on the
  marker. We find the position of these points in the image us-
1 0 −x0 /z0
ing the transformation (Rkj , tjk ) and the camera intrinsic ma-
Q= 0 1 −y0 /z0  (4)
trix K. Note that we know the pose of the marker only up
−x0 /z0 −y0 /z0 (x20 + y02 − r2 )/z02
to a rotation around its normal, so that we have to sample
and t = (x0 , y0 , z0 )T . By construction, the rotation matrix R several times using different values for the starting rotation
transforms XC to XI as following angle α. Sampling for a sufficient number of starting angles
α would require a prohibitive amount of computation time,
XC = RXI (5) and we therefore suggest to use a pyramidal normalized cross
correlation approach as follows: i, j and k being fixed, we
so that C and Q are related by following equation compute NCC (α) for 12 different values: α0 to α11 , starting
µRT CRc = Q (6) at zero and increasing with an angular increment δ = 2π/12.
We keep αmax = argmax αp NCC (αp ) and sample again
where µ is a scale factor. Equation (6) is solved using the with 5 values for α around αmax using half of the increment
eigendecomposition of the matrix C: if λ1 , λ2 , λ3 are the δ. We repeat the last step, halving each time the increment δ
(sorted) eigenvalues and V the matrix of eigenvectors, then until the computed NCC value does not change anymore. This
the one-dimensional family of solutions (R, t) is ([10]): pyramidal cross correlation is much faster than an exhaustive
  search and proves to produce the same results in terms of an-
g cos α S1 g sin α S2 h gular accuracy.
R(S1 , S2 , S3 , α) = V  sin α −S1 cos α 0  For each ellipse Ej -marker Mi combination, we compute
S1 S2 hqcos α S2 h sin α S1 g the score sij as the maximal NCC over all possible poses k
 
−S2 S3 (λ1 −λ−λ 2 )(λ2 −λ3 )
1 λ3
r cos α and angles α. This gives us a correlation matrix between el-
lipses and markers that we use as input to an assignment algo-
 q 
t(S1 , S2 , S3 , α) =  (λ1 −λ2 )(λ2 −λ3 )
 −S1 S2 S3 r sin α 

−λ1 λ3 rithm. In practice, we use the Hungarian assignment method,
λ2 r
S3 √−λ and reject the ellipses whose score is below a given threshold
1 λ3
(7) as non-marker. As a result, we have for each marker Mi in
where α is a free variable, S1 , S2 and S3 are undetermined the definition file: (1) a one-bit answer to the question is the
(discrete) signs, and g and h are defined by marker visible in the image and (2) if the answer is yes, the
q pose of the marker (Ri , ti ) as defined in equation (1), com-
g = λλ21 −λ 3
puted as the concatenation of the ellipse pose (Rkj max , tjkmax )
q −λ3 (8)
h = λλ11 −λ 2 and the rotation around the Z axis with the angle αmax .
−λ3

The free variable α reflects the rotation invariance of the 4. POSE OF A SET OF MARKERS
oblique cone, and fixing α boils down to fixing the rotation
of the marker around its normal. We chose to fix α = 0 In the previous section, we have shown how to compute the
and to find the true rotation of the marker in a later step. We pose of one single marker. Here, we address the more com-
further reduce the number of solutions by stating that among plex case where several markers are defined in the same rigid
the 8 possible combinations of the signs Si , exactly two meet body. A rigid body is a set of markers with a fixed relative
the visibility constraints (the center of the marker must be in position over time. Our system allows any combination of
front of the camera, i.e. in the CCS cz > 0 and the normal markers, including rigid bodies with markers lying on differ-
of the marker must be oriented towards the camera i.e. in the ent planes and any number of markers per plane. We propose
CCS nz < 0). We end up with exactly two possible poses to compute one global pose using all the available data (in
(R1 , t1 ) and (R2 , t2 ) for each ellipse. form of circle-ellipse correspondences). Our algorithm is as
follows: first, for each plane in a rigid body, we compute a ho-
3. MARKER IDENTIFICATION mography between the 3D plane and the imaged plane. Sec-
ond, we use the homographies from all the planes in a rigid
In order to identify the markers among the found ellipses, body for computing the global pose of the rigid body.
we use a one-dimensional Normalized Cross Correlation be- Homography of an imaged plane from circular mark-
tween the expected true markers and the image pixel values. ers. From the marker definition file, we can deduce which
In our application, the user can define which markers are used markers belong to the same 3D plane. For each plane of a
by the means of an editable XML file. For each ellipse Ej , rigid body, we count the number of visible markers on that
we go successively over the 2 poses (Rkj , tjk ), k ∈ [1, 2], plane.If one marker only is visible in the image, then we use
and then for each expected marker Mi we compute the nor- the retrieved pose (Ri,k , ti,k ) to compute the plane homogra-
malized cross correlation NCC (i, j, k, α) by sampling pixel phy H. If more than one marker is visible, we use the method
of [9] to compute the homography of the plane from conic 800

700
2

correspondences. Note that we use two different algorithms 600

1

500
depending on the number of correspondences (see [9] for de-

Distance (mm)
0

Angle (rad)
400
X alpha
tails). In case of multiple solutions, we apply visibility con- 300
Y -1

200
Z beta
straints and choose the remaining solutions that minimizes the 100
-2

reprojection error (reprojection of the marker center). -100

0
-3
gamma

Rigid body pose from planar homographies. If only -200

1 101 201 301 401 501
-4
Frame 1 101 201 301 401 501 Frame
one plane has visible markers within the rigid body, it is straight-
forward to recover the global pose from the homography, us-
ing the fact that if the plane has equation Z = 0, then the Fig. 4. Translation and rotation over 500 frames
homography is equal to the matrix K [r1 |r2 |t], where r1 and r2
are the two first columns of the final rotation matrix R. When
several planes have been found with visible marker(s), we ap-
ply following multiple plane pose estimation algorithm: Let
T be a transformation matrix transforming a given plane to a
plane with equation Z = 0 and hTi the same matrix without
its third column. Then we have the homogeneous equation:
Fig. 5. Examples of tracked frames
−1
[R | t]hTiH K ∼ I3×3
where the pose can be computed from one single marker. Our
where I3×3 is the 3 × 3 identity matrix.
method relies on a pose estimation from conic correspon-
Calling di , i = 1...3 the columns of the matrix hTiH−1 K and
dences and is accelerated by a pyramidal normalized cross
i the (unknown) rows of the matrix[R | t], we have:
qT
correlation algorithm. In case several markers are used in a
 T
q1 rigid setup, we have shown how to compute the global pose
 qT  d1 d2 d3 ∼ I3×3

using all conics in correspondence. Our system is fast and
2
qT robust, and can be used as alternative to square markers in
3
marker-based augmented reality applications.
This last 3 × 3 matrix homogeneous equation leads to Acknowledgment. This work has been partially funded
8 equations in the elements of the qi ’s. This system can be by the project CAPTURE (01IW09001) and the German BMBF
solved from 2 planes on. project AVILUSplus (01M08002).

7. REFERENCES
5. RESULTS
[1] H. Kato, M. Billinghurst, S. Weghorst, and T. Furness, “A mixed reality
Tracking one marker. In order to assess the developed theo- 3d conferencing application,” Tech. Rep., HIT lab, UW, 1999.
retical framework, we implemented the algorithms on a stan- [2] D. Schmalstieg, A. Fuhrmann, G. Hesina, Z. Szalavari, L.M. Encar-
dard PC. Figure 4 shows the results of tracking one marker nacao, M. Gervautz, and W. Purgathofer, “The studierstube augmented
reality project,” Presence: Teleoperators & Virtual Environments, vol.
over 500 consecutive frames. The translation vector is pro- 11, no. 1, pp. 33–54, 2002.
vided in mm and the rotation by means of Euler angles. From [3] A.. Rice, A. Beresford, and R. Harle, “Cantag: an open source software
the charts, we can see that the tracking is smooth and regular, toolkit for designing and deploying marker-based vision systems,” in
and we can track the pattern from a distance of 200 mm to Pervasive Computing and Communications, 2006.
750 mm (the marker radius was 35 mm in this experiment). [4] M. Fiala, “Artag, a fiducial marker system using digital techniques,” in
Time consumption. Our marker tracking algorithm works CVPR, 2005.
in a tracking-by-detection paradigm. This means that we search [5] L. Naimark and E. Foxlin, “Circular data matrix fiducial system and
robust image processing for a wearable vision-inertial self-tracker,” in
for all the markers and compute the pose independently in
ISMAR, 2002.
each frame, without taking the last frames into consideration.
[6] J. Kim, H. Kim, and I. Kweon, “A camera calibration method using
However, thanks to the fast ellipse detection method and to concentric circles for vision applications,” in ACCV, 2002.
the linear algorithms for pose computation, the tracker is very [7] G. Wang, Q. Wu, , and Z. Ji, “Pose estimation from circle or parallel
fast. We achieve about 100 frames per second on a standard lines in a single image,” in ACCV, 2007.
PC and 30 frames per second on a small Ultra Mobile PC. [8] F. Abad, E. Camahort, and R. Viv, “Camera calibration using two con-
centric circles,” in ICIAR, 2004.
6. CONCLUSION [9] J. Kannala, M. Salo, and J. Heikkil, “Algorithms for computing a planar
homography from conics in correspondence,” in BMVC, 2006.

In this paper, we presented a new marker design for marker- [10] Q. Chen, H. Wu, and T. Wada, “Camera calibration with two arbitrary
coplanar circles,” in ECCV, 2004.
based applications. This is the first circular marker design