Institute of Informatics – Institute of Neuroinformatics

Lecture 03
Image Formation 2

Davide Scaramuzza
http://rpg.ifi.uzh.ch

1
Lab Exercise 2 - Today afternoon
 Room ETH HG E 1.1 from 13:15 to 15:00
 Work description: your first camera motion estimator using DLT

2
Goal of today’s lecture
• Study the algorithms behind robot-position control and augmented reality

3
Outline of this lecture
• (Geometric) Camera calibration
– PnP problem
• P3P for calibrated cameras
• DLT for uncalibrated cameras
• Omnidirectional cameras

4
Perspective from n Points (aka PnP Problem)
• Given known 3D landmarks positions in the world frame and given their image
correspondences in the camera frame, determine the 6DOF pose of the camera
in the world frame (including the intrinsinc parameters if uncalibrated)

Camera
Image

world

3D Landmarks

5
Perspective from n Points (aka PnP Problem)

PnP Problem
Calibrated camera Uncalibrated camera
(i.e., instrinc parameters are known) (i.e., intrinsic parameters unknown)

Works for any 3D point configurations Direct Linear Transform (DLT)

Minimum number of points: 3 Minimum number of points:

P3P (Perspective from Three Points) 4 if coplanar
6 if non coplanar

6
How Many Points are Enough?
• 1 Point: infinitely many solutions.
• 2 Points: infinitely many solutions, but bounded.
• 3 Points:
– (no 3 collinear) finitely many solutions (up to 4).
• 4 Points:
– Unique solution

7
1 Point

8
2 Points

C
B

9
Inscribed Angles are Equal

q C

q
q

A B
10
3 Points 𝐴
LC
C

𝑠2
𝐿𝐴
s2 𝐶 s3
𝜃𝐴𝐶 qCA
𝐿𝐶LB 𝑠3
𝑠1
qBC𝜃𝐴𝐵 𝜃𝐵𝐶
B s1
CP qAB
C
𝐿𝐵 𝐵
LA A

From Carnot’s Theorem:

2
s 1  LA  LB  LA LB cos q AB
2 2 C’
𝑠12 = 𝐿2𝐵 + 𝐿2𝐶 − 2𝐿𝐵 𝐿𝐶 cos 𝜃𝐵𝐶
s 2
 L 2
 L 2
 L L cosq BC
𝑠 2 = 𝐿2 2+ 𝐿2 −B 2𝐿 𝐿C cos 𝜃B C
2 𝐴 𝐶 𝐴 𝐶 𝐴𝐶 B’
2 2LA  LC  LA LC cos q AC
2 2 2 A’
𝑠32 s
= 𝐿𝐴 3+ 𝐿𝐵 − 2𝐿𝐴 𝐿𝐵 cos 𝜃𝐴𝐵

Image Plane
11
Algebraic Approach: reduce to 4th order equation
(Fischler and Bolles, 1981)

𝑠12 = 𝐿2𝐵 + 𝐿2𝐶 − 2𝐿𝐵 𝐿𝐶 cos 𝜃𝐵𝐶

𝑠22 = 𝐿2𝐴 + 𝐿2𝐶 − 2𝐿𝐴 𝐿𝐶 cos 𝜃𝐴𝐶

𝑠32 = 𝐿2𝐴 + 𝐿2𝐵 − 2𝐿𝐴 𝐿𝐵 cos 𝜃𝐴𝐵

• It is known that 𝑛 independent polynomial equations, in 𝑛 unknowns, can have no more solutions
than the product of their respective degrees. Thus, the system can have a maximum of 8 solutions.
However, because every term in the system is either a constant or of second degree, for every real
positive solution there is a negative solution.
• Thus, with 3 points, there are at most 4 valid (positive) solutions.
• A 4th point can be used to disambiguate the solutions.

By defining 𝑥 = 𝐿𝐵 /𝐿𝐴, it can be shown that the system can be reduced to a 4th order equation:

G0  G1 x  G2 x 2  G3 x 3  G4 x 4  0
12
Application to Monocular Visual Odometry:
camera pose estimation from known 3D-2D correspondences

Keyframe 1 Keyframe 2 Current frame

New keyframe

Initial point cloud New triangulated points

13
AR Application: Microsoft HoloLens

14
Outline of this lecture
• (Geometric) Camera calibration
– PnP problem
• P3P for calibrated cameras
• DLT for uncalibrated cameras
– From general 3D objects
– From planar grids

• Omnidirectional cameras

15
Camera calibration
• Calibration is the process to determine the intrinsic and extrinsic parameters of the
camera model
• A method proposed in 1987 by Tsai consists of measuring the 3D position of 𝑛 ≥ 6 control
points on a three-dimensional calibration target and the 2D coordinates of their projection
in the image. This problem is also called “Resection”, or “Perspective from 𝒏 Points (PnP)”,
or “Camera pose from 3D-to-2D correspondences”, and is one of the most widely used
algorithms in Computer Vision and Robotics
• Solution: The intrinsic and extrinsic parameters are computed directly from the
perspective projection equation; let’s see how!
3D position of control points is assigned
in a reference frame specified by the user
P c = Pw
u
v O Zw Xw
p
W Yw
Zc
C Xc
16
Yc
Camera calibration: Direct Linear Transform (DLT)
Our goal is to compute K, R, and T that satisfy the perspective projection equation (we
neglect the radial distortion)

X w
u~  u  Y 
p   v~     v   K R T   w  
~
~  Zw 
 w 1   
 1 

X w
u~   u 0 u0   r11 r12 r13 t1   
  v~    0  v v0   r21 r22   Yw 
r23 t2  
~  0 0 Z 
 w   1  r31 r32 r33 t3   w 
 1 
X w
u~   u r11  u0 r31  u r12  u0 r32  u r13  u0 r33  u t1  u0t3   
  v~    v r21  v0 r31  v r22  v0 r32  v r23  v0 r33  v t 2  v0t3    w 
Y
~  Zw 
 w  r31 r32 r33 t3   
 1 
17
Camera calibration: Direct Linear Transform (DLT)
Our goal is to compute K, R, and T that satisfy the perspective projection equation (we
neglect the radial distortion)

X w
u~  u  Y 
p   v~     v   K R T   w  
~
~  Zw 
 w 1   
 1 

X w
u~   u 0 u0   r11 r12 r13 t1   
  v~    0  v v0   r21 r22   Yw 
r23 t 2  
~  0 0 Z 
 w   1  r31 r32 r33 t3   w 
 1 
X 
u~   m11 m12 m13 m14   w 
  v~   m21 m22 m24    w 
Y
m23
~  m  Zw 
 w   31 m32 m33 m34   
 1 
18
Camera calibration: Direct Linear Transform (DLT)
Our goal is to compute K, R, and T that satisfy the perspective projection equation (we
neglect the radial distortion)

X 
u~   m11 m12 m13 m14   w 
  v~   m21 m24    w 
Y
m22 m23
~  m Z 
 w   31 m32 m33 m34   w 
 1 
X w
u~  Y 
  v~   M   w 
~  Zw 
 w  
 1 
X w
u~  m1   
T

  Y
  v~   m2T    w 
 Zw 
~  T
 w m3   
 1 
where miT is the i-th row of M
19
Camera calibration: Direct Linear Transform (DLT)

~ X w

u  m1  
T

 ~   T   Yw 
  v   m2   P
~   mT   Z w 
 w   3  1 
 

Conversion back from homogeneous coordinates to pixel coordinates leads to:

u~ m1T  P
u ~ T
w m3  P (m1T  ui m3T )  Pi  0

v~ m2T  P (m2T  vi m3T )  Pi  0
v ~  T
w m3  P
20
Camera calibration: Direct Linear Transform (DLT)
By re-arranging the terms, we obtain
 m1 
(m  ui m )  Pi  0
T T
P T
0T
 u P    0 
T
1 3
 1 1 1
 m2    
(m  vi m )  Pi  0
T
2
T
3
T
0 P1T  v P    0 
1 1
T

 m3 
For 𝑛 points, we can stack all these equations into a big matrix:

 P1T 0T  u1 P1T   0
 T 
T m 
 
 0 P1  v1 P1  1   0 
T

     m2     
 P T 0T  u P T    0 
 n n n  m3 
 
 0T P T  v P T   0
 n n n 

21
Camera calibration: Direct Linear Transform (DLT)
By re-arranging the terms, we obtain
 m1 
(m  ui m )  Pi  0
T T
P T
0 T
 u P    0 
T
1 3
  1
 m2    
1 1

(m  vi m )  Pi  0
T
2
T
3 0
T
P1T  v P    0 
1 1
T

 m3 
For 𝑛 points, we can stack all these equations into a big matrix:
 m11 
 
 12 
m
m 
 13 
m 
 X w1 Yw1 Z w1 1 0 0 0 0  u1 X w1  u1Yw1  u1Z w1  u1  m14   0 
  21  
0 X w1 Yw1 Z w1 1  v1 X w1  v1Yw1  v1Z w1  v1  m   0 
 QM  0
 0 0 0
     22     
 n  m23   0 
 X w Yw
n
Z w 1 0 0 0 0  un X wn  unYwn  un Z wn
n
 un    
0 0 0  vn  24   0 
m
 0 X wn Ywn Z wn 1  vn X wn  vnYwn  vn Z wn
 m31 
 
 m32 
 m33 
 
 34 
m
Q (this matrix is known) 22
M (this matrix is unknown)
Camera calibration: Direct Linear Transform (DLT)

QM  0
Minimal solution
• 𝑄(2𝑛×12) should have rank 11 to have a unique (up to a scale) non-zero solution 𝑀
• Each 3D-to-2D point correspondence provides 2 independent equations
1
• Thus, 5+ point correspondences are needed (in practice 6 point correspondences!)
2

Over-determined solution
• n ≥ 6 points
• A solution is to minimize | 𝑄𝑀 |2 subject to the constraint | 𝑀 |2 = 1.
It can be solved through Singular Value Decomposition (SVD). The solution is the
eigenvector corresponding to the smallest eigenvalue of the matrix 𝑄𝑇 𝑄
(because it is the unit vector 𝑥 that minimizes | 𝑄𝑥 |2 = 𝑥 𝑇 𝑄 𝑇 𝑄𝑥).
• Matlab instructions:
• [U,S,V] = svd(Q);
• M = V(:,12); 23
Camera calibration: Direct Linear Transform (DLT)

QM  0
Degenerate configurations

1. Points lying on a plane and/or along a single line passing through the center of projection

2. Camera and points on a twisted cubic (i.e., smooth curve in 3D space of degree 3)

𝐶
24
Camera calibration: Direct Linear Transform (DLT)
• Once we have determined M, we can recover the intrinsic and extrinsic
parameters by remembering that:

M  K(R | T)
 m11 m12 m13 m14   u 0 u0   r11 r12 r13 t1 
m m 22 m23 m24    0  v v0  r21 r 22 r23 t 2 
 21
m31 m32 m33 m34   0 0 1  r31 r32 r33 t3 

25
Camera calibration: Direct Linear Transform (DLT)
• Once we have determined M, we can recover the intrinsic and extrinsic
parameters by remembering that

M  K(R | T)
 m11 m12 m13 m14  r11  u0 r31 r12  u0 r32 r13  u0 r33 t1  u0t3 
m m 22 m23 m24   r21  v0 r31 r22  v0 r32 r23  v0 r33 t 2  v0t3 
 21
m31 m32 m33 m34   r31 r32 r33 t3 

• However, notice that we are not enforcing the constraint that 𝑅 is

orthogonal, i.e., 𝑅 ∙ 𝑅 𝑇 = 𝐼

• To do this, we can use the so-called QR factorization of 𝑀, which

decomposes 𝑀 into a 𝑅 (orthogonal), T, and an upper triangular matrix
(i.e., 𝐾)
26
Tsai’s (1987) Calibration example
1. Edge detection
2. Straight line fitting to the detected edges
3. Intersecting the lines to obtain the image corners (corner accuracy < 0.1 pixels)
4. Use more than 6 points (ideally more than 20) and not all lying on the same plane

Why is this ratio

not 1?
What are the «skew»
and «residuals»?

𝛼𝑢 𝛼𝑢 /𝛼𝑣 𝑢0 𝑣0

27
Tsai’s (1987) Calibration example
• The original Tsai calibration (1987) used to consider two different focal lengths 𝛼𝑢 , 𝛼𝑣
(which means that the pixels are not squared) and a skew factor (𝐾12 ≠ 0, which means
the pixels are parallelograms instead of rectangles) to account for possible misalignments
(small 𝑥, 𝑦 rotation) between image plane and lens
𝛼𝑢
• Most today’s cameras are well manufactured, thus, we can assume = 1 and 𝐾12 = 0
𝛼𝑣
• What is the residual? The residual is the average “reprojection error” (see Lecture 8). The
reprojection error is computed as the distance (in pixels) between the observed pixel point
and the camera-reprojected 3D point. The reprojection error gives as a quantitative
measure of the accuracy of the calibration (ideally it should be zero).

28
DLT algorithm applied to mutual robot localization

In this case, the camera has been pre-calibrated (i.e., K is known). Can you
think of how the DLT algorithm could be modified so that only R and T need
to be determined and not K? http://youtu.be/8Ui3MoOxcPQ
29
Outline of this lecture
• (Geometric) Camera calibration
– PnP problem
• P3P for calibrated cameras
• DLT for uncalibrated cameras
– From general 3D objects
– From planar grids

• Omnidirectional cameras

30
Camera calibration from planar grids: homographies

• Tsai’s calibration is based on DLT algorithm, which requires points not to lie
on the same plane
• An alternative method (today’s standard camera calibration method)
consists of using a planar grid (e.g., a chessboard) and a few images of it
shown at different orientations
• This method was invented by Zhang (1999) @Microsoft Research and is
implemented in both Matlab and the OpenCV C/C++ library

31
Camera calibration from planar grids: homographies

• Tsai’s calibration is based on DLT algorithm, which requires points not to lie
on the same plane
• An alternative method (today’s standard camera calibration method)
consists of using a planar grid (e.g., a chessboard) and a few images of it
shown at different orientations
• This method was invented by Zhang (1999) @Microsoft Research and is
implemented in both Matlab and the OpenCV C/C++ library

32
Camera calibration from planar grids: homographies
• Our goal is to compute K, R, and T, that satisfy the perspective projection equation
(we neglect the radial distortion)
• Since the points lie on a plane, we have 𝑍𝑤 = 0

~ X w
u  Y 
 v~   K R T   w  
   0 
~
 w   
 1 
X w
u~   u 0 u0   r11 r12 r13 t1   
  v~    0  v v0   r21 r22   Yw 
r23 t2  
~  0 0  0 
 w   1  r31 r32 r33 t3   
 1 

u~   u 0 u0   r11 r12 t1   X w 

  v~    0  v v0   r21 r22 t 2    Yw 
~  0 0
 w 1  r31 t3   1 
  r32
33
Camera calibration from planar grids: homographies
• Our goal is to compute K, R, and T, that satisfy the perspective projection equation
(we neglect the radial distortion)
• Since the points lie on a plane, we have 𝑍𝑤 = 0

u~   h11 h12 h13   X w 

  v~   h21 h22 h23    Yw 
~  h
 w h33   1 
  31 h32

u~  X w
  v~   H   Yw 
~
 w  1 
 This matrix is called
Homography
u~  h1   X w 
T

 
  v~   h2T    Yw 
~  hT   1 
 w   3  

where hiT is the i-th row of 𝐻

34
Camera calibration from planar grids: homographies

u~  h1   X w 
T

 
  v~   h2T    Yw 
~  hT   1 
 w   3  

Conversion back from homogeneous coordinates to pixel coordinates leads to:

u~ h1T  P
u ~ T
w h3  P (h1T  ui h3T )  Pi  0

v~ h2T  P (h2T  vi h3T )  Pi  0
v ~  T
w h3  P
where P = (𝑋𝑤 , 𝑌𝑤 , 1)𝑇
35
Camera calibration from planar grids: homographies
By re-arranging the terms, we obtain
 h1 
(h  u h )  Pi  0
T T
Pi  h1  0  h  ui Pi  h  0
T T T T
 Pi T
0 T
 u1 Pi    0 
T
1 i 3
 2 3
  T  h   
T  2 
(h  v h )  Pi  0 0  h1  Pi  h  vi Pi  h  0  v1 Pi    0 
T T T T
2 i 3
T T
2
T
3 0 Pi T
 h3 
For 𝑛 points, we can stack all these equations into a big matrix:

 P1T 0T  u1 P1T  0

 T 
T h 
 
 0 P1  v1 P1  1   0 
T

     h2     
 P T 0T  u P T    0 
 QH  0
 n n n  h3 
 
 0T P T  v P T   0
 n n n 

Q (this matrix is known) H (this matrix is unknown)

36
Camera calibration from planar grids: homographies

QH  0
Minimal solution
• 𝑄(2𝑛×9) should have rank 8 to have a unique (up to a scale) non-trivial solution 𝐻
• Each point correspondence provides 2 independent equations
• Thus, a minimum of 4 non-collinear points is required

Over-determined solution
• n ≥ 4 points
• It can be solved through Singular Value Decomposition (SVD) (same considerations as
before)

Solving for K, R and T  h11 h12 h13   u 0 u0   r11 r12 t1 

• H can be decomposed by recalling that h h22 h23    0  v v0   r21 r22 t 2 
 21
h31 h32 h33   0 0 1  r31 r32 37 t3 
 Won’t be asked
How to recover K, R, T from H? at the exam

1. Estimate the homography 𝐻𝑖 for each view, using the DLT algorithm.

2. Determine the intrinsics K of the camera from a set of homographies:

1. Each homography 𝐻𝑖 ∼ 𝐾 𝒓1 , 𝒓2 , 𝒕 provides two linear equations in the 6 entries of
the matrix B ≔ 𝐾 −⊤ 𝐾 −1 . Letting 𝒘1 ≔ 𝐾𝒓1 , 𝒘2 ≔ 𝐾𝒓2 , the rotation constraints
𝒓1⊤ 𝒓1 = 𝒓⊤ ⊤
2 𝒓2 = 1 and 𝒓1 𝒓2 = 0 become
𝒘1⊤ 𝐵𝒘1 − 𝒘⊤ ⊤
2 𝐵𝒘2 = 0 and 𝒘1 𝐵𝒘2 = 0.
2. Stack 2N equations from N views, to yield a linear system 𝐴𝒃 = 𝟎. Solve for b (i.e., B)
using the Singular Value Decomposition (SVD).
3. Use Cholesky decomposition to obtain K from B.

3. The extrinsic parameters for each view can be computed using K:

𝒓1 ∼ 𝜆𝐾 −1 𝐻𝑖 : , 1 , 𝒓2 ∼ 𝜆𝐾 −1 𝐻𝑖 : , 2 , 𝒓3 = 𝒓1 × 𝒓2 and 𝑇𝑖 = 𝜆𝐾 −1 𝐻𝑖 : , 3 ,
with 𝜆 = 1/𝐾 −1 𝐻𝑖 : , 1 . Finally, build 𝑅𝑖 = (𝒓1 , 𝒓2 , 𝒓3 ) and enforce rotation
matrix constraints.

38
Types of 2D Transformations

This transformation is called

Homography

39
Application of calibration from planar grids
• Today, there are thousands of application of this algorithm:
– Augmented reality

40
Most AR apps use AprilTag or ARuco Markers
Application of calibration from planar grids
• Today, there are thousands of application of this algorithm:
– Augmented reality
– Robotics (beacon-based localization)
• Do we need to know the metric size of the tag?
– For Augmented Reality?
– For Robotics?

My lab Marc Pollefeys’ lab

41
Most AR apps use AprilTag or ARuco Markers
DLT vs PnP: Accuracy vs noise
If the camera is calibrated, only R and T need to be determined. In this
case, should we use DLT (linear system of equations) or PnP (non
linear)?

Lepetit, Moreno Noguer, Fua, EPnP: An Accurate O(n) Solution to the PnP Problem, IJCV’09
42
DLT vs PnP: Accuracy vs number of points
If the camera is calibrated, only R and T need to be determined. In this
case, should we use DLT (linear system of equations) or PnP (non
linear)?

Lepetit, Moreno Noguer, Fua, EPnP: An Accurate O(n) Solution to the PnP Problem, IJCV’09
43
DLT vs PnP: Timing

Lepetit, Moreno Noguer, Fua, EPnP: An Accurate O(n) Solution to the PnP Problem, IJCV’09
44
 Won’t be asked
An Efficient Algebraic Solution to P3P at the exam

Ke and Roumeliotis (CVPR’17)

Similarly to Kneip (CVPR’11) and Maselli (ICPR’14), directly solve for

the camera’s pose (not the distances).
1. Eliminate the camera’s position and the features’ distances to yield
a system of 3 equations in the camera’s orientation alone.
3 points Pairwise subtraction and dot product

2. Successively eliminate two of the unknown 3-DOFs (angles)

algebraically and arrive at a quartic polynomial equation.
• Results: outperforms previous methods in terms of speed,
accuracy, and robustness to close-to-singular cases.
• Available in OpenCV > 3.3. solvePnP() with SOLVEPNP_AP3P 45
Recap

PnP Problem
Calibrated camera Uncalibrated camera
(i.e., instrinc parameters are known) (i.e., intrinsic parameters unknown)

Works for any 3D point configuration DLT (Direct Linear Transform)

Minimum number of points: 4 (i.e., 3+1) Minimum number of points:

P3P (Perspective from Three Points (plus 1)) 4 if coplanar
6 if non coplanar

46
Outline of this lecture
• (Geometric) Camera calibration
– PnP problem
• P3P for calibrated cameras
• DLT for uncalibrated cameras
– From general 3D objects
– From planar grids

• Omnidirectional cameras

47
Overview on Omnidirectional Cameras
Example scene viewed by three different camera models

Perspective Fisheye Catadioptric

Zhang et al., Benefit of Large Field-of-View Cameras for Visual Odometry, ICRA’16
http://rpg.ifi.uzh.ch/fov.html 49
Catadioptric Cameras

50
Catadioptric Cameras

51
Non Central Catadioptric cameras
Rays do not intersect in a single point

52
Central
CameraCatadioptric
Modelscameras
• Rays do not intersect in a single point
• Mirror must be surface of revolution of a conic

single effective viewpoint

53
Central Catadioptric
Types of central cameras
catadioptric cameras
Hyperbola Parabola
+ +
Perspective camera Orthographic lens

F1
F1

F2

NB: one of the foci of the hyperbola must lie in the

54
camera’s center of projection
Why do we prefer central cameras?
Because we can:
• Apply standard algorithms valid for perspective geometry.
• Unwarp parts of an image into a perspective one
• Transform image points into normalized vectors on the unit sphere

55
Unified Omnidirectional Camera Model
• We describe the image projection function by means of a
polynomial, whose coefficients are the parameters to be
estimated
• The coefficients, intrinsics, and extrinsics are then found via DLT p
𝛼
X w
u  u0  Y 
 
  p    v  v0    R | T   w 
 Z w 
 g (  )   
1 

g      a1   a2  2  ...  a N  N

  (u  u0 ) 2  (v  v0 ) 2
When 𝑎𝑖 = 0 then we get a pinhole camera
Scaramuzza, Martinelli, Siegwart, A toolbox for easily calibrating omnidirectional cameras, IROS’06. PDF
Scaramuzza, Omnidirectional Camera, chapter of Encyclopedia of Computer Vision, Springer’14. PDF
OcamCalib: Omnidirectional Camera Calibration
• Released in 2006, it is the standard toolbox for calibrating wide angle cameras
• Original url: https://sites.google.com/site/scarabotix/ocamcalib-toolbox
• Since 2015, included in the Matlab Computer Vision Toolbox:
https://ch.mathworks.com/help/vision/ug/fisheye-calibration-basics.html

Example calibration images of a catadioptric camera Example calibration images of a fisheye camera

D. Scaramuzza, A. Martinelli, R. Siegwart, A toolbox for easily calibrating omnidirectional cameras,

IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006. PDF. 57
Equivalence between Perspective and Omnidirectional model

58
Equivalence between Perspective and Omnidirectional model

Measures the ray direction (angles).

59
Equivalence between Perspective and Omnidirectional model

Measures the ray direction (angles).

60
Equivalence between Perspective and Omnidirectional model

Zc = optical axis
u Zc Pc
v

p

O = principal point f

Image plane

C Image plane
C = optical center = center of the lens

61
Representation of image points on the unit sphere
Always possible after the camera has been calibrated!

62
Summary (things to remember)
• P3P and PnP problems
• DLT algorithm
• Calibration from planar grid (Homography algorithm)
• Readings: Chapter 2.1 of Szeliski book

• Omnidirectional cameras
– Central and non central projection
– Dioptric
– Catadioptric (working principle of conic mirrors)
• Unified (spherical) model for perspective and omnidirectional cameras
• Reading: Chapter 4 of Autonomous Mobile Robots book: link

63
Understanding Check
Are you able to:
• Describe the general PnP problem and derive the behavior of its solutions?
• Explain the working principle of the P3P algorithm?
• Explain and derive the DLT? What is the minimum number of point
correspondences it requires?
• Define central and non central omnidirectional cameras?
• What kind of mirrors ensure central projection?

64