Triangulation in Presence of a Noise

review, implementation and comparison of different methods

1st Azadeh Hadadi

UBFC, Centre universitaire Condorcet
VIBOT program: Master in computer vision
without ORCID

Abstract—The objective of this paper is to review the Stereo- computes reconstruction as intersection of two rays as shown
Triangulation in computer vision, investigating the reconstruction in Figure 1.a. In order to solve this problem, it is necessary to
of three-dimensional data corresponding using the 2-D images in know the parameters of the camera projection function from
a situation where images contain noise. Finally four optimization
methods were implementation for Triangulation simulation in 3D to 2D for the cameras involved, in the simplest case repre-
the noisy condition. The cameras were modelled using Peter sented by the camera matrices. Triangulation is sometimes also
Corke’s MVTB toolbox in MATLAB and Gaussian noise was used referred to as reconstruction. Stereo -Triangulation method,
to simulate noisy condition. Comparison of the result for four which is used in navigation and surveying, is used to calculate
selected optimization methods, i.e., poly-abs, lindstrom, sampson depth. Basic triangulation uses the known distance of two
and mid-point shows that poly-abs performs better in practice
and create less error. separated points looking at the same scene point. From these
Index Terms—stereo-triangulation, optimization, noise, 3D re- parameters the distance to the scene point can be calculated
construction simply applying geometry relation. This same basic idea is
used in stereo vision to find depth information from two
I. I NTRODUCTION images. Figure 1.a graphically shows the geometry of a stereo
vision system and depth calculation.
in this work, we will practice following items and will
support our findings with related work review and some
experimental results.
1) “Recall the mathematical definition of a norm and
the definition of usual norms:L1 , L2 , Lp , L∞ , Frobenius
norm.
2) Search the scientific literature for articles that deal with
the problem of triangulation (in stereovision, not to
be confused with triangulation in computer graphics!).
Make a synthesis of the proposed methods.
3) Suggest and implement a triangulation method in simu-
lation (you can use Peter Corke’s toolbox) and propose a
noise-based analysis. In other words: test without noise
(on the image points), then gradually increase the noise Fig. 1. (a) triangulation in a Stereo Vision System (b)Stereo-Triangulation in
level (on the image points), give a graph with the 3D the presence of noise
error on the y-axis and the noise level on the x-axis. “
A binocular stereo vision system is composed of two In the above arrangement, two cameras ( C, C 0 ) see the same
cameras both of which observe the same scene [2]. The main feature point(Xi ). Assume that we know the camera matrices
task of stereo vision is to compute three dimensional data P , P 0 and 2D correspondences ui ↔ ui on each image. In
from 2D input images grabbed by cameras. Computer stereo order to determine the 3D point Xi it is tempting to back-
vision tries to imitate the human visual system. The human project the two image points and determines their intersection.
visual system obtains information about the world through two The location of the point in the two image planes is denoted by
planar images that are captured by the retina of each eye. The xi and x0i . When the cameras are separated by a distance T, the
position of a scene point in right view is horizontally shifted location of ui and u0i from the cameras normal axis will differ
in the left view. This displacement commonly referred to as (denoted by xi , x0i ). Using these differences, the distance(Xi )
disparity, human brain use to deduce the depth information of to the point can be calculated from the following formula:
the scene. Although this course of action appears simple, for T
Xi = f xi −x 0
computers is surprisingly difficult [1]. i

In a stereo-vision system, the depth is calculated by tri- We know the focal length of the camera (f ) and camera
angulation within 3D reconstruction process. Triangulation baseline distance (T) using the calibration procedure explained
by Zhang [12]. Thus, if we know disparity value, then 5) The Frobenius norm of matrix A denoted by |A|F and
the actual distance of the real world point is known. But, it is defined as follows
unfortunately, the real world environment is usually more  1/2
m X n
complicated due to various source of noise.In practice, the X 2 
1/2
= trace AT A

d= (aij ) 
coordinates of image points cannot be measured with arbitrary
i=1 j=1
accuracy. Instead, various types of noise, such as geometric
noise from lens distortion or interest point detection error, Disntance L1 , L2 are widely used in non-linear triangulation
lead to inaccuracies in the measured image coordinates. As a and all distances are used in linear triangualtion. The above
consequence, as shown in the Figure 1.b, the lines generated formulas can be extended to n-dimensional space. Non-linar
by the corresponding image points do not always intersect in triangulation methods are mainly trying to minimize Geomet-
3D space. As a result, we need to estimate and find a best ric and 3D errors while linear methods minimize algebraic
solution where two lines approximately intersect. error.
Several ways to approach the problem depending on what
we choose to optimize over which parameters. better saying, A. Non Linear Tiangulation (Minimizing a Geometric Error)
one of the following target in the optimisation objective. The idea is to estimate a 3D point X
bi which exactly satisfies
• Only errors in ui ↔ ui
0 the supplied camera geometry [5]. So it projects as
0 0
• Errors in ui ↔ ui , P and P
ûi = P X̂i , ûi = P 0 X̂i
The paper is organized as follows: section 2 summarizes lit-
erature review of methods used in stereo vision triangulation, a And the aim is to estimate X
b i from the image measurements
brief summary. The problem, then, is to find a 3D point which ui and u0i as shown in Figure 2.
optimally fits the image points. There are multiple proposals
for how to define optimality and how to find the optimal 3D
point. Since they are based on different optimization criteria,
the various methods produce different estimates of the 3D
point x when noise is involved. Mainly non-linear methods will
be discussed in this section. Some solutions for triangulation in
the presence of noise will be demonstrated in section 3 because
they are mainly focusing on linear triangulation. Then, for
development purpose, four non-linear methods will be selected
and triangulation implementation will be developed using pure
MATLAB code which will be detailed in section 4. Section
5 will demonstrate the result of the selected methods. In this
section, the result of the noisy and noise-free triangulation
will be presented and compared. The paper will end up to
conclusion and references.

II. L ITERATURE R EVIEW

Fig. 2. minimizing a geometric error
Many triangulation methods were presented and developed
during last decades. Among them, a few mehtods attracted
more attention and widely used in stereovision [12]. This 2 2
εi = d (ui , P Xi ) + d (u0i , P 0 Xi )
review will focus mostly on poly-abs [5], midpoint [9] [11],
Lindstrom [7], and finally Sampson [8] [10] methods. These This algorithms would be better to find the 30 point Xi that
methods generally minimize the error between two back- minimize a meaningful geometric error, like the reprojection
projected points and approximated point so that approximated error. It can be shown that if the measurement noise in
image
point, which is called reconstructed point, has the minimum points is Gaussian with mean equal to zero ∼ N 0, δ 2 , the
distance from both back projected rays. Therefore, each al- minimizing geometric error gives the Maximum Likelihood
gorithm uses certain kind of distance. Four distances, i.e.,L1 , estimate of Xi .
L2 , Lp and L∞ and Frobenius distances are widely used. The At first glance, this minimization appears to be over the three
definition of the distance for two given points with coordinate parameters in Xi , but under the assumption that P and P 0 are
(x1 , y1 ) and (x2 , y2 ) is defined as follows: error free the problem can be reduced to a minimization over
1) L1 : d = |x2 − x1 | + |y2 − y1 | one parameter.

2 2
1/2 If P and P 0 are error free, then the epipolar geometry is error
2) L2 : d = (x2 − x1 ) + (y2 − y1 ) free
p p 1/p
3) Lp : d = ((x2 − x1 ) + (y2 − y1 ) ) • We have a unique baseline which define all possible
4) L∞ : d = max {x2 − x1 , y2 − y1 } epipolar planes as a one parameter family
 • We have unique epipoles that all epipolar lines must pass difference becomes clear when Xi is much closer to one of the
through, so we have one parameter families of epipolar cameras than the other. Another disadvantage of this method
lines as well is that it does not extend naturally to situations when Xi is
By requiring that both reprojected points lie on the same observed by more than two cameras.
epipolar plane, the minimization problem can be reformulated Let P = (M | − M c) be a decomposition
 of the first camera
in terms of the one parameter families of epipolar lines as c
matrix. The centre of the camera is in homogeneous
shown in Figure 3. 1
coordinates. Furthermore, the point at infinity that maps to
M −1 u
a point u in the image is given by . Therefore,
0
any point
 on the−1ray mapping to u may be written in the
c + αM u
form or in non homogeneous coordinates
0
−1
c + αM u for some α. Given two images, the two rays
must meet in space, which leads to an equation αM −1 u −
α̂M̂ −1 û = ĉ − c. This gives three equations in two unknowns
(the values of α and α0 ) which we may solve using linear
least-squares methods. This minimizes the squared distance
between the  two rays. The midpoint between
 the two rays is
−1 −1
given by: c + αM u + ĉ + α̇M̂ û /2

C. Poly-abs
Fig. 3. epipolar plane
We suppose that a point x in R3 is visible in two images.
The two camera matrices P and P 0 corresponding to the two
images are supposed known. Let u and u0 be projections of the
2 2
εi = d (ui , l(t)) + d (u0i , l0 (t)) point x in the two images. Instead of minimizing the square
sum of image errors L1 it is possible to adapt the polynomial
l and l0 range over all choices of corresponding epipolar lines method to minimize the sum of absolute values of the distances
and ûi is the closest point on the line l to ui, same for L1 , instead of the squares of distances. This method is called
û0i as illustrated in Figure 4. To find t that minimizes the Poly-Abs.The quantity to be minimized is d(u, λ) + d(û + λ̂)
reprojection error, one has to minimize a 6-degree polynomial which, as a function of t, is expressed by
of t. |t| |ct + d|
s2 (t) = p +q
1 + (tf )2
(at + b)2 + fˆ(ct + d)2

Where λ and λ̂ range over all choices of corresponding

epipolar lines. The epipoles are placed on the x -axis at points
(1, 0, f ) and (1, 0, fˆ ), respectively. Constant a, b, c, d are
defined by the fundamental matrix. The first derivative is of
the above cost function is calculated to find best t. Setting the
Fig. 4. epipolar lines derivative equal to zero, separating the terms on opposite sides
of the equal sign and squaring to remove the square roots gives
Although this assumption is very likely to lead a much better a polynomial of degree 8 in t. We evaluate s2 (t) at the roots
approximation of the underlying uncertainty, the resulting cost of this polynomial to find the global minimum of s2 (t)
function is non-convex and extremely difficult to converge.
D. Lindstrom
B. 3D Mid-Point Method (Minimizing the 3D Error) This approach describes a simple and efficient algorithm
One natural estimate for Xi is to take the midpoint on for two-view triangulation of 3D points from approximate 2D
the shortest line between two back-projected rays [3]. This matches based on minimizing the L2 reprojection error. This
minimizes the 3D error, but typically not the reprojection iterative algorithm is an improvement on the algorithm original
error. This method is relatively easy to compute using a linear presented by Kanatani et al. [6]. The algorithm ensures that the
algorithm. However, ease of computation is almost its only epipolar constraint is satisfied each iteration. In the case where
virtue. This method is neither affine nor projective invariant. the two cameras are pointed in the same direction, the method
Since perpendicularity is not an affine and nor midpoint is a provably converges to an optimal solution in exactly two
projective concept. It is seen to behave very poorly indeed iterations. For more general camera poses, two iterations are
under projective and affine transformation and seems to be sufficient to achieve convergence to machine precision, which
the worst of the methods considered for triangulation. The is the reason why this method is sometimes referred to as fast
and non-iterative method. The resulting algorithm amounts to This can be rearranging as:
little more than solving a quadratic equation, and involves a
vi pT3 − pT2
 
fixed, small number of simple matrix-vector operations and no  ui pT3 − pT1 
conditional branches. The algorithm achieves higher numerical  vi p0T
 0T
 X̂i = 0
3 − p2

stability and 1-4 orders of magnitude greater speed.
ui p3 − p0T
0T
1

E. Sampson Combining these equations we get an over determined homo-

geneous system of linear equations that we can solve with
An alternative to the minimization of the geometric error
SVD. The minimized algebraic error is not geometrically
method is so called Sampson approximation. Without exam-
meaningful, but the method extends naturally to the case when
ining it in more details, we will make an overview of the
Xi is observed in more than two images.
method. The Sampson correction δc of the world point C is
T
expressed by (x, y, x0 , y0 ), where (x, y)T and (x0 , y0 ) are A. Lp Norm
the coordinates of the points c and c0 respectively. Logically
A triangulation problem takes as input:
Ĉ could be presented as the calculated C from the faulty
measurements plus the Sampson correction δc . After some T = {(ui , Pi ) |1 ≤ i ≤ M }
transformations (for details, please refer to [4]), the end result
looks like: and estimates the underlying unknown 3D world point U as

x̂
 follows:
M
 ŷ  X p
U = argx min kui − Pi (X)kp0
 x̂0  =
  b
i=1
yˆ0
Here, Pi denotes the projection operator corresponding to the
camera matrix Pi and the (ui , Pi ) pairs denote the camera
F T c 0
1
  

x matrices of the M cameras along with the projections of the
T
 y 
 − c0 F c  F T c0 
 2  unknown 3D world point U on their image planes.
 x̂  (F c)2 + (F c)2 + (F T c0 )2 + (F T c0 )2  (F c)1 
1 2 1 1 In the previous definition, p0 and p, are known as the
ŷ (F c)2 image-space and residual-space norms, respectively. When we

talk about algorithms that minimizes the (`p0 , `p) -norm of
Where, for example the expression F T ĉ 1 replaces the poly-
the reprojection error, for some particular p0 and p, we are
nomial f11 x̂ + f21 ŷ + f31 . The Sampson approximation is
referring to the image-space and residual-space norms in this
accurate only in case the needed correction is very small.
order. When p = ∞, we assume that becomes
Otherwise, there is a more stable algorithm, such as triangu-
lation with structured light, which results satisfy the epipolar b = argx min max kui − Pi (X)k 0
U p
i=1...
constraint.Triangulation with structure light can be done in
several ways, i.e., Light spot technique, Stripe projection, B. L∞ Norm
Projection of a static line pattern, Projection of encoded L∞ optimization can be regarded as something in between
patterns or Light spot stereo analysis. the L2 methods and linear algorithms. The L∞ framework
shares advantages with both of these alternative approaches.
III. M INIMIZING T HE A LGEBRAIC E RROR : L INEAR Optimal solutions are guaranteed with a geometrically mean-
T RIANGULATION ingful cost function and at a reasonable computational cost.
A potential disadvantage is that the L∞ norm is not robust to
This algorithm uses the two equations for perspective pro-
outliers.
jection to solve for the 3 D point that are optimal in a least
Let Pi , i = 1, . . . , m, denote several 3 × 4 camera matrices
squares sense [11]. Each perspective camera model gives rise T
and ui = (xi , yi ) the measured image points. For the present
to two equations on the three entries of Xi For the first camera
we are considering the calibrated triangulation problem, so
we have:
   T  the camera matrices Pi are correct in a Euclidean coordinate
p11 p12 p13 p14 p1 frame. Now, we wish to find the point U that maps most
P =  p21 p22 p23 p24  =  pT2  closely via the camera matrices Pi to the points ui , minimizing
p31 p32 p33 p34 pT3 the maximum error.
We assume that the camera matrices Pi are normalized ( by
Where pTi are the rows of P The unknown scale in λxi = P Xi multiplication by -1 if necessary) such that P is of the form
by forming a cross-product xi × P Xi = 0 [Mi |m] with det Mi > 0. In this case, the condition for a

T
point U = X T , 1 to lie in front of the camera is that
vi pT3 X̂i − pT2 X̂i = 0
the third coordinate of Pi U should be positive. This is the
ui pT3 X̂i − pT1 X̂i = 0 cheirality constraint, which we assume holds for the point U
ui pT2 X̂i − vi pT1 X̂i = 0 with respect to each camera.
 Let pjT
i denote the j− th row of the i− th cam-
era matrix. In this case,
a point U maps to the point
p1T 3T 2T 3T
i U/pi U, pi U/pi U in the i − th image. The error
is then: r 2  2
T T
1T
pi U 2T
pi U
(p1i −xi p3i ) U + (p2i −yi p3i ) U
p3T U − xi , p3T U − yi =

i i p3T U
i

IV. S TEREO -T RIANGULATION I MPLEMENTATION

The experiment was developed in MATLAB R2017a re-
vision 7.11 on HP PC in Microsoft windows 10 pro OS, Fig. 7. (a) overlap of noise free and noisy left image (b) overlap of noise
with Intel(R) Processor Core(TM) i7-8565U CPU @ 1.80GHz, free and noisy right image
1992 MHz, 4 Core(s), and 8 Logical Processor(s). Four
triangulation methods, i.e., polynomial (poly-abs), mid-point,
Lindstorm, and Sampson were selected and implemented as V. R ESULTS A ND D ISCUSSION
a way to correct the projection error of triangulation. A 3D
scene was defined as two orthogonal planes with regularly Following the detailed under section IV and using a triangu-
spaced points on it. The data.mat file includes 100 points lation function called stereoReconspts the 3D points are
as 3D model in the object coordinate system. To simulate the reconstructed. Depending on the triangulation method different
camera the Peter Corke’s toolbox, i.e. MVTB 4.3 was used. results are generated as shown in figures from Figure 8
The second camera was translated across the X axis as shown through 12. This function receives both camera matrices,
in the Figure 5. Figure 6 displays the projected images by the both left and right images and the method of correction.The
left and right cameras. noise free and with-noise reconstructions are displayed in
Figure 8(a) and in Figure 8(b) respectively.

Fig. 8. (a) triangulation of noise free image (b) triangulation with optimal
correction of noisy image

Fig. 5. 3D model without noise Min, max, average and variance of error over 100 recon-
structed points were calculated for 4 above selected methods.
The reconstruction was applied without using any optimiza-
tion, as the fifth method. The five results are demonstrated
and compared in Figure 9. In Figure 9, 1 through 5 refers
to without optimization, poly-abs, mid-point, lindstrom and
finally sampson. As seen in this Figure 9, average error, max
error and variation of the error for each pixel is minimum
for poly-abs. It means, poly-abs creates more uniform error
on all pixels with respect to other methods and is better to be
used and the behavior of the optimization is more regular. The
detail of the error for all 100 pints are shown in Figure 10. In
our study, the maximum error for all methods of optimization
Fig. 6. (a) projected image by the left camera (b) projected image by the
right camera appears for last two rows. While the minimum always appears
at the first two rows. The detail of the error for all 100 points
Then a Gaussian noise with mean equal to zero and variance is shown in Figure 11. The error curves were a little bit
equal to 5 ∼ N (0, 5) was created and added to the both smoothed for better visualization.
images. In the Figure 7 the noisy left and right images are Figure 12 shows the 3D reconstruction result in presence of
shown with ’x’ symbol compared to the noise free images. the noise without any optimization.
Fig. 9. min, max, average, variation of the error for all points presents for Fig. 12. error curve for all 100 points without optimization
five methods

VI. C ONCLUSION
The Stereo-Triangulation for 3D reconstruction was pre-
sented and the core issue under uncertainty was revealed.
Different optimization methods for triangulation in presence
of noise were reviewed and the principle of each method was
explained. Four methods, i.e., poly-abs, Lindstrom, Sampson
and mid-points were selected and the MATLAB function was
implemented for 3D reconstruction and triangualtion. The
result of the methods were extracted, anlayzed and compared.
3D points were reconstructed using above function adding
different level of the noise. The error for each method were
extracted and compared with triangulation without using any
optimization and without noise condition. The result shows
that poly-abs presents more accurate results with lower error
with respect to other three methods. Lindstrom and mid-
point presented faily poor result with a lot of varation. The
maximum average error belongs to mid-point.
Fig. 10. error curves for each method and maximum error
R EFERENCES
[1] HB Barlow. Temporal and spatial summation in human vision at
different background intensities. The Journal of physiology, 141(2):337–
350, 1958.
[2] Thomas R Dahl and George Spector. Binocular stereovision, January 1
1991. US Patent 4,982,278.
[3] Fredrik Kahl ; Richard Hartley. Multiple-view geometry under the l∞ -
norm. IEEE Transactions on Pattern Analysis and Machine Intelligence,
16 July 2008.
[4] Richard Hartley and Andrew Zisserman. Multiple view geometry in
computer vision. Cambridge university press, 2003.
[5] Richard I Hartley and Peter Sturm. Triangulation. Computer vision and
image understanding, 68(2):146–157, 1997.
[6] Kenichi Kanatani and Hirotaka Niitsuma. Optimal two-view planar
scene triangulation. IPSJ Transactions on Computer Vision and Ap-
plications, 3:67–79, 2011.
[7] Peter Lindstrom. Triangulation made easy. In 2010 IEEE Computer
Society Conference on Computer Vision and Pattern Recognition, pages
1554–1561. IEEE, 2010.
[8] Zlatka Mihaylova. Triangulation methods. seminar paper, 2009.
[9] N Ragot, R Rossi, X Savatier, JY Ertaud, and B Mazari. 3d volumetric
reconstruction with a catadioptric stereovision sensor. In 2008 IEEE
International Symposium on Industrial Electronics, pages 1306–1311.
Fig. 11. error curve for all 100 points and for five selected method
IEEE, 2008.
[10] Paul D Sampson. Fitting conic sections to “very scattered” data: An
iterative refinement of the bookstein algorithm. Computer graphics and
image processing, 18(1):97–108, 1982.
[11] Adam Scholefield, Alireza Ghasemi, and Martin Vetterli. Bound and
conquer: Improving triangulation by enforcing consistency. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 2019.
[12] Zhengyou Zhang. Flexible camera calibration by viewing a plane from
unknown orientations. In Proceedings of the Seventh IEEE International
Conference on Computer Vision, volume 1, pages 666–673. Ieee, 1999.