Investigating The Limitations of Some of The Most Popular Lines and Points Based Camera Calibration Techniques in Photogrammetry and Computer Vision
Investigating The Limitations of Some of The Most Popular Lines and Points Based Camera Calibration Techniques in Photogrammetry and Computer Vision
ISSN No:-2456-2165
Abstract:- The need for camera calibration has been a The aim of camera calibration is to determine camera
fundamental requirement since the foundation of internal and external parameters that enabled the mapping of
photogrammetry. As the number of photogrammetry a 3D object space onto the 2D image space. Without
applications grows and the technology advances, camera accurate modelling of these camera internal and external
calibration became more complex to undertake. parameters it is impossible to achieve a good geometric
However, when measurements derived from the imagery description of the projection from the 3D world scene onto
is used for scene modelling purposes, the consequences of the 2D image plan. The accuracy of the modeled
small imaging errors can be significant on the accuracy transformation depends on a certain number of parameters
of derived models. Thus, the development of cheaper including the mathematical components of the distortion
lenses such as those of consumer grade cameras and model, the number of parameters considered by the model
their integration in the Photogrammetry process and the robustness of the mathematical solution used to
requires from camera calibration approaches to solve for the camera parameters. This study will try to
accurately model the projection process from the 3D investigate some of the mostly employed lines and points
scene onto the 2D image plan and also offer robust based camera calibration approaches in order to identify
solutions to derive with high accuracy the various their short-comings of the current camera calibration
camera parameters. Several line based and points based methods when dealing consumer grade digital cameras. The
camera calibration methods have been proposed in study will also investigate the suitability of Brown’s radial
literature and reported producing promising results but distortion model when it comes to modelling pincushion,
the majority of such approaches were found either barrel profiles that are not always symmetric with reference
numerically instable or suffer from serious limitations to the distortion center. Finally, the study will analyze the
when it comes to removing distortions at the edges of effects of the number of additional parameters in lens
imagery. The fact that these techniques rely of the distortion models, on the accuracy of the calibration
traditional brown’s model which assumes symmetric procedure.
radial distortions make them no suitable for consumer
grade digital cameras which are known for their instable II. CAMERA CALIBRATION APPROACHES
internal geometry. This study found undisputable that
new analytical camera calibration techniques more A. Ahmed and Farag Calibration Method
adapted to the internal geometry of consumer grade In Ahmed and Farag (2005), a new camera calibration
cameras are needed. approach based on the slope of a distorted line was
proposed. The camera model is based on the perspective
I. INTRODUCTION projection of a straight line in which every point on a line
satisfies the linear equation expressed by the following:
The need for camera calibration has been a
fundamental requirement since the foundation of ax by c 0 [1]
photogrammetry. With the increasing number of close range
photogrammetry applications and advances in imaging Where a, b and c are constants for a specific line l, and
technologies, camera calibration methods are becoming
a
more complex to perform. Indeed with the introduction of s the slope of the line. Considering the origin of the
consumer grade cameras which possess off the shelves b
lenses into the photogrammetry processes, the need for high image coordinate system O x0 , y 0 , the proposed model
quality metrics from photographs has grown over the past
decades, making camera calibration a very crucial task. relates a point Pu xu , yu on the undistorted line l and its
distorted corresponding Pd xd , yd on the distorted line by
using the traditional radial and decentring distortions models
originally proposed by Brown (1971) as follows:
2 2
f xu , y u a x d x d x0 k1 rd k 2 r 4 k 3 r 6 p1 rd 2x d x0 2 p 2 x d x0 y d y 0 1 p3 rd
2
2
c
[2]
b y b y d y 0 k1 rd k 2 rd k 3 rd p 2 rd 2 y d y 0 2 p1 x d x0 y d y 0 1 p3 rd
2 4 6 2 2 2
By calculating the elemental change of f f from With the distance from the origin of the line to the
equation [2] one obtains the following equation: end point and is the angle this line makes with the
horizontal axis. Once the equation of the undistorted line is
x x y y established, a search process to identify distorted points in
f a u xd u y d b u xd u y d [3] the neighborhood of the undistorted line is performed. This
xd y d xd y d process groups pixels based on their grey level and spatial
connectedness on Line Support Regions (LSRs). The set of
This equation represents the tangent to the distorted identified pixels are then used as input in a line fitting
curve and its slopes can be estimated by the equation [4] as process which leads to the estimation of radial distortion
follows: parameters using a Least Squares technique. The obtained
distortion parameters are then employed in a wrap function
yu yu yd which removes distortions from the imagery by mapping
distorted points onto their ideal locations. The wrap
x d y d xd
sxd , y d [4] parameters estimated by the technique are only limited to
xu xu yd the two first coefficients of radial distortion k1 , k 2 and the
x d y d xd coordinates of the distortion center xc , y c . However, a
limitation of this approach is the fact that it ignores other
Given a chain of edge points on the distorted
x
types of distortion such as decentering or film deformation
, yd , i 1,2, ....N , the error E between points
i i
line d
distortions. Moreover, the radial distortion model employed
positions on the distorted line can be estimated through the by the technique is not adequate for consumer grade
squared difference between slopes of the line. This can be cameras as it relies on consistent radial distance which
expressed as follows: assumes symmetric radial distortions. The other limitation
of the technique is the influence of image noise that can be
mistaken as a distorted point during the point search in the
i 2 i 1 i 1 2
E S xd , y d S xd
i
, yd [5] neighborhoods of the undistorted line model. Wang et al.,
(2009) proposed another line-based technique which
In the case where several lines are considered on the employs rational models model derived from the traditional
distorted image the error in [8] would be estimated as the Brown’s radial distortion model and described in Fitzgibbon
sum of errors as follows: (2001). The model relates the distorted and undistorted
points on an image by the following functions:
N 2
E S xd xd S xd
i 1 i 1
xd
i i
[6] xd yd
xu and y u , [8]
i 2
1 rd 1 rd
2 2
Under the correct values of the distortion parameters, Where is the first coefficient of radial distortion.
the error estimated in (6) should be zero. This error can be These produce the equation of the undistorted line as
minimized using non-linear optimization algorithm starting follows:
with initial guess values of distortion parameters. Although
the proposed projection model was reported robust, the yd xd
distortion model used in the complete model assumes a a b [9]
1 rd 1 rd
2 2
symmetry of radial distortion and produces strong
correlation between distortion parameters and the model
only models distortions at image corners (Jacobsen, 2003). After reformulation the relation in [8] gives:
A variation of the technique was earlier proposed in 1997 by
Prescott and McLean. The two stages techniques starts by
establishing an undistorted line model by joining two end
yd axd b b xd yd 2 2
[10]
points representing a line using an edge detector program.
The detected edge points’ coordinates are then used to build Which after development produces the following:
the equation of the undistorted line given by:
a 1 1
xd y d xd yd 0
2 2
[11]
x cos y sin [7] b b
B. Tsai Calibration Method
X u X d X d k1 r k 2 r ...
2 4
In 1987, Tsai proposed a calibration technique based
on a single image. The two stages calibration technique [13]
estimates the intrinsic camera parameters using a
Yu Yd Yd k1 r k 2 r ...
2 4
perspective projection before refining them in a second
stage by applying a non-linear optimisation technique. The
first stage of the camera model proposed by Tsai requires The proposed technique requires the position and
coordinates of 3D points to estimate the ideal coordinates of altitude of the calibration targets to be recovered with
projected 2D points. With known ideal 2D coordinates and respect to the camera coordinate system. For instance, if
the observed coordinates, the second phase of the model
estimates the radial distortion parameters (Horn, 2000). Tsai xs , ys are the coordinates of a point measured in the
scene coordinate system and xc , yc the coordinates
model start by estimating the relationship between camera
coordinates and ideal image coordinates (undistorted) by the
following expression: measured in the camera coordinate system, the perspective
projection model expressing the relationship between the
xc two coordinates is given as follows:
X u zc
f xc , yc R xs , ys t [14]
[12]
Yu yc
Where t is the translation and R.... the rotation
zc
matrix. This can be reformulated by the following
expression:
The conversion from distorted coordinates to image proposed by Tsai was reported viable for 3D machine vision
pixel is done by the following model: measurements and produced acceptable calibration results
(Horn, 2010), a certain number of limitations were
s x xd identified. Firstly, the technique is very laborious to
u cx implement as it requires very large amounts of points to
dx perform well. Moreover, the proposed model assumes the
[16]
y image centre to be the centre of projection, which does not
v d cy hold true for consumer grade cameras (Li et al., 2014). The
dx other limitation is the pinhole perspective projection model
which does not characterise the physical and optical
With s x the scale factor introduced by the image behaviour of consumer grade cameras with unstable internal
geometry. Furthermore, Tsai technique only works perfectly
capture hardware, d x , d y the horizontal and vertical for symmetric radial distortions as it assumes the radial
distances between centres of adjacent cells in the CCD distance constant. The model also ignores the decentring
array. The expression in [15] can be substituted in [14] to component of distortions.
estimate the coordinates in pixels. Although the model
Although the model is reported to perform better than Tsai’s model it relies on the perspective projection which does not
characterise the physical behaviour of consumer grades cameras. Moreover, the technique uses the traditional Brown’s model
which assume consistent radial distance.
x p 2 x p1 x1 f ( y p2 y p1 )
n y p2 y p2 y1 f ( x p2 x p1 ) [24]
0 f x2 y1 x1 y 2
The projection plane in the object space is composed of the vector P1 P2 and P1 PC , the vector normal N to the object
The equivalence between the two vectors can be obtained by equalling equation [24] and [25] and gives equation [26] as
follows:
However, due to the opposite directions between the two normal vectors and fact that the 3D object plane and the 2D image
are linked by a scale relationship, the equation [27] can be written as :
With is a scale factor and R is the rotation matrix defined by the sequence .R x , R y , R z of the rotation. This
relationship is now expressed as:
f ( y p2 y p1 )
(YP2 YP1 ) ( Z P1 Z PC ) YP1 YPC Z P2 Z P1
f ( x p2 x p1 )
R ( x p1 X PC ) ( Z P2 Z P1 ) X P2 X P1 ( Z P1 Z PC ) [28]
x p y p x p y p
2 1 1 2 ( Z P1 Z PC ) (YP1 YPC ) ( X P1 X PC ) (YP2 YP1 )
YP2 YP1 Z P1 Z PC YP1 Y pc Z P2 Z P1
N1
N X P1 X PC Z P2 Z P1 X P2 X P1 Z P1 Z PC
2 [29]
N 3
Z P1 Z PC YP1 YPC X P1 X Pc YP2 YP1
And by expending [30] one obtains:
f y p2 y p1 11 N 1 12 N 2 13 N 3
x p2 x p1 21 N 1 22 N 2 23 N 3
[30]
x p2 y p1 x p1 y 2
31 N 1 32 N 2 33 N 3
In order to eliminate the scale factor the first two equations in [28] are divided by the last equation and give:
f y p1 y p2 31 N1 32 N 2 33 N 3 x p1 y p2 x p2 y1 11 N1 12 N 2 33 N 3 0
[32]
f x p2 x p1 31 N1 32 N 2 33 N 3 x p1 y p2 x p2 y1 21 N1 22 N 2 33 N 3 0
To introduce radial and decentring distortions into the model, the authors considered the Conrady-Brown model of radial and
decentring distortions expressed by:
xu x0 xd x0 xd x0 k1 r k 2 r k 3 r p1 r 2xd x0 2 p 2 xd x0 y d y 0
2 4 6 2 2
[33]
2 4
6 2 2
y d y d y 0 y d y 0 k1 r k 2 r k 3 r p 2 r 2 yu y 0 2 p1 xd x0 yu y 0
Where xu , y u are the undistorted coordinates and model, the fourteen parameters Brown’s Physical model and
the sixteen parameters Borwon’s model.
xd , y d the observed distorted coordinates in the image
reference system, k1 , k 2 , k 3 are the coefficients of radial The Bauer model has three additional parameters
including two parameters that describes the extent of affine
distortions and P1 , P2 are the coefficients of decentring deformation on the image and one parameter that describes
the symmetric radial distortion (Anguilar et al., 2010;
distortion while x0 , y 0 are the coordinates of the principal
Blazquez and Colomnia, 2010). The distortions applied to
point. Although the technique was reported producing good the x and y coordinates are given by the following
estimates of distortion parameters, it does not guaranty the distortion model:
accuracy of the 3D points coordinates involved in the model
as any inaccuracy in points’ measurements can be
propagated into the estimated distortion parameters. a1 x r 2 r0 2 a 2 x
x
y
Moreover the Brown-Conrady model used is a model [34]
suitable for symmetric lens distortions as it assumes
consistent radial distance. 1
a y r 2
r0
2
a 2 y a 3
x
E. Additional Parameters Model. By combining the Bauer’s distortion model into the
Additional parameters are the term of a polynomial collinearity equation one obtains the complete distortion
expression incorporated in the collinearity equations in order model presented as follows:
to model various systematic errors including lens
distortions. Five types of additional parameters models have
been proposed in the literature. This includes the Bauer
simple model, the Jacobsen model, the Ebner’s orthogonal
xu x 0
2
r11 X w X 0 r12 Yw Y0 r13 Z w Z 0 a1 x r r0 a 2 x
2
y y f r31 X w X 0 r32 Yw Y0 r33 Z w Z 0 [35]
u 0
a1 y r 2 r0 a 2 y a3 x
2
With xu , yu the undistorted coordinates, x0 , y 0 the coordinates of the perspective centre and X 0 , Y0 , Z 0 the origin of the
3D coordinate system.
The Jacobsen simple model is similar to the Bauer’s model but presents one additional parameter and compensate for the
first and second order distortion associated with affine deformation and lens distortions (Passini and Jacobsen, 2008). The affine
deformations and lens distortion applied to the x and y coordinates are given by the distortion model as follows:
x
a1 x r 2 r0 2 a 2 x a3 y
y [36]
a1 y r r0 a 2 y a3 x a 4 x
2 2 2
Similar to Bauer’s model, the Jacobsen calibration approach relies on the collinearity equation to form its complete distortion
model.
The Ebners’s orthogonal model is a twelve additional parameters model which compensates for various types of systematic
errors such as scanner errors, affine deformation and film deformation. (Leica Geosystem User Guide, 2015). These additional
parameters are orthogonal to one another and to the exterior orientation parameters under circumstances where the ground surface
is flat. The model is given by the following:
2 4b 2 2b 2 2b 2 2b 2
a1 x a 2 y a3 2 x a 4 xy a5 y 2 a 7 x y 2 a9 y x 2
3 3 3 3
2
2
a11 x 2 2b y 2 2b
3 3
x [37]
y 2 4b 2 2 2b 2 2 2b 2 2 2b 2
a1 y a 2 x a3 xy a 4 2 y a6 x a8 y x a10 x y
3 3 3 3
2 2b 2 2 2b 2
a12 x y
3 3
The Brown’s fourteen additional parameters compensate for linear and non-linear forms of plate film deformation and lens
distortions. The total distortion applied to x and y image coordinates is given by the following distortion model.
x
1 a x a 2 y a 3 xy a 5 x 5
y a 6 xy 2
a 7 x 2 2
y a 13
x3 y
f
a14 x x 2 y 2
[38]
a xy a x 2 a x 2 y a xy 2 a x 2 y 2 a y x a y x 2 y 2
3 2
y 8 9 10 11 12 13 14
f
Like the previous models, Brown’s fourteen model relies on the collinearity equation to compose the complete distortion
model. Moreover, another Brown’s model presenting sixteen additional parameters has been proposed to deal with affine
distortion, film and lens distortions.
2
2
2
2 2
a11 x x 2 y 2
a12 x 3 y 2 a13 x x 4 y 4
1 a xy a 2 y a 3 x y a 4 xy a 5 x y
f f f
a xr r a xr r 2 a xr r 3
x 14 0 15 0 16 0
[39]
y
a 6 xy a 7 x 2 a8 x 2 y a9 xy 2 a10 x 2 y 2 11
a xx y 2 2
a x y
12
3 2
a xx y
13
4
2
f f f
a y r r a r r 2 a r r 3
14 0 15 0 16 0
The above model does not only differ from the for consumer grade cameras with instable internal geometry.
previous one in terms of the number of degrees of freedom This poor performance extended to other line based
but also in the functions composing those degrees of calibration approaches investigated in this study in
freedom. Although the additional parameter models occurrence of non-symmetric radial lens distortions as the
discussed above deal with variety of distortions, they suffer proposed algorithms assume a constant radial displacement
from numerical instability due to their large number of measure for all the distorted points within the image. Which
parameters. does not always hold true with distortion profiles such as
barrel and pincushion distortions that create a larger or small
III. CONCLUSION towards the edges of the photograph in comparison to the
measure around the image centre. Line based camera
From the lined-based calibration approaches studied in calibration approaches also rely on optimization algorithms
this investigation, the method proposed by Prescott and to perfect the numerical estimates of calibrated camera
McLean (1997) would produce the poorest camera parameters. Although point based calibration method offer
parameter accuracies due to the fact that it is very sensitive better numerical stability due to the fact that they do not
to image noise. In fact, during the search process, there is a need any sophisticated algorithm to measure the coordinates
high chance when dealing with low resolution images that of the projections of the 3D points onto the image plan, they
image noise could be mistaken as distorted pixel locations. require a large amount of calibration points in order to
This is associated to the dependence of the edge detection achieve an optimal calibration results. Moreover, most of
algorithm to pixel grey levels. Moreover, the radial point based calibration approaches require some results
distortion function employed by the technique is not suitable optimization through an iteration process and iterative