Photogrammetry & Robotics Lab 3D Point to Pixel:

Estimating the Parameters of P

Camera Calibration:
Direct Linear Transform

Cyrill Stachniss
pixel trans- world
coordinate formation coordinate

The slides have been created by Cyrill Stachniss.

1 2

Estimating Camera Parameters Estimate Ex- and Intrinsics

Given the Geometry
§ Wanted: Extrinsic and intrinsic
known control points
parameters of a camera
§ Given: Coordinates of object points
(control points)
§ Observed: Coordinates of those
known 3D object points in the image

3 4
Mapping Mapping
Direct linear transform (DLT) maps any Direct linear transform (DLT) maps any
object point to the image point object point to the image point

5 6

Mapping Camera Parameters

Direct linear transform (DLT) maps any
object point to the image point
§ Intrinsics
§ Camera-internal parameters
§ Given through
§ Extrinsics
§ Pose parameters of the camera
§ Given through and
§ Projection matrix
contains both, the in- and extrinsics
7 8
 Direct Linear Transform (DLT) How Many Points Are Needed?
Compute the 11 intrinsic and
extrinsic parameters

Each point gives ??? observation

control point
equations
observed coordinates
image point c, s, m, (given)
xH, yH 3 translations

3 rotations

9 10

How Many Points Are Needed? How Many Points Are Needed?

Each point gives ??? observation Each point gives ??? observation
equations equations

11 12
How Many Points Are Needed? Spatial Resection vs. DLT
§ Calibrated camera
§ 6 unknowns
§ We need at least 3 points
§ Problem solved by spatial resection
Each point gives two observation § Uncalibrated camera
equations, one for each image coordinate § 11 unknowns
§ We need at least 6 points
§ Assuming the model of an affine camera
§ Problem solved by DLT

13 14

DLT: Problem Specification

§ Task: Estimate the 11 elements of
§ Given:
DLT: Direct Linear Transform § 3D coordinates of object points
§ Observed image coordinates of an
uncalibrated camera with the mapping
Computing the Orientation
of an Uncalibrated Camera
Using ≥ 6 Known Points
§ Data association

15 16
Rearrange the DLT Equation Rearrange the DLT Equation

Define the tree vectors A, B, C as follows

17 18

Rearrange the DLT Equation Rearrange the DLT Equation

So what we can rewrite the equation as So what we can rewrite the equation as

19 20
Rearrange the DLT Equation Rearrange the DLT Equation

Leads to an system of equation, which is

linear in the parameters A, B and C

21 22

Estimating the Elements of P Estimating the Elements of P

§ Collect the elements of P within a § Rewrite
parameter vector
§ as
rows of P as
column-vectors,
one below the
other (12x1)

23 24
Estimating the Elements of P Verifying Correctness
§ Rewrite

§ as

§ with

25 26

Verifying Correctness Verifying Correctness

A A

B B

C C

27 28
Verifying Correctness Verifying Correctness

29 30

Estimating the Elements of P Solving the Linear System

(Homogeneous System => SVD)
§ For each point, we have
§ Solving a system of linear equations of
the form is equivalent to
finding the null space of A
§ Stacking everything together
§ Thus, we can apply the SVD to solve

§ Choose as the singular vector

belonging to the singular value of 0

31 32
Redundant Observations Redundant Observations
§ In case of redundant observations, § Singular value decomposition (SVD)
we will have contradictions :

§ Find such that it minimizes

§ Choosing (the singular vector

belonging to the smallest singular
value ) minimizes
with will have
33 34

Obtaining the Projection Matrix Critical Surfaces

§ Choosing minimizes and § M is of rank 11, if
thus is our estimate of P: § Number of points ≥6
§ Assumption: no gross errors
§ No solution, if all points are
located on a plane

e.g., assume all Zi=0

Does this always work?

35 36
Critical Surfaces Critical Surfaces
§ M is of rank 11, if § M is of rank 11, if
§ Number of points ≥6 § Number of points ≥6
§ Assumption: no gross errors § Assumption: no gross errors
§ No solution, if all points are § No solution, if
located on a plane § All points are located on a plane
§ (All points and the projection center
rank deficiency are located on a twisted cubic curve)

37 38

Decomposition of P
§ We have , how to obtain ?
§ Structure of the projection matrix

From to
§ with

39 40
Decomposition of P Decomposition of P

§ We get the projection center through § We get the projection center through

Rotation matrix:
§ Let’s look to the structure
§ What do we know about the matrices?

41 42

Decomposition of P Decomposition of P

Rotation matrix: Rotation matrix:

§ Let’s look to the structure § Let’s look to the structure
§ K is a triangular matrix § K is a triangular matrix
§ R is a rotation matrix § R is a rotation matrix

Is there a matrix decomposition into Is there a matrix decomposition into

a rotation matrix and a triangular on? a rotation matrix and a triangular on?
QR-decomposition
43 44
Decomposition of P Decomposition of P
§ The matrix is homogenous
§ We perform this for given the § Thus, is the calibration matrix
order of rotation and triangular matrix § Due to homogeneity normalize
§ QR decomposition of yields
rotation and calibration matrix

Q R

45 46

Decomposition of P DLT in a Nutshell

§ Decomposition results in 1. Build the for the linear system
with positive diagonal elements
§ To get negative camera constant,
choose
2Ix12 12x1

using with

decomposition still holds

47 48
DLT in a Nutshell DLT in a Nutshell
2. Solve by SVD 3. If individual parameters are needed
Solution is last column of

49 50

Discussion DLT Summary

§ We realize both ways § Direct linear transforms estimates
§ We are free to choose sign of c the intrinsic and extrinsic of a camera
§ Solution is instable if the control points § Computes the parameters for the
lie approximately on a plane mapping of the uncalibrated camera
§ Solution is statistically not optimal § Requires at least 6 known control
(no uncertainties of point coordinates) points in 3D
§ Direct solution (no initial guess)

51 52
Literature Slide Information
§ Förstner & Wrobel, Photogrammetric § The slides have been created by Cyrill Stachniss as part of the
photogrammetry and robotics courses.
Computer Vision, Chapter 11.2 § I tried to acknowledge all people from whom I used
images or videos. In case I made a mistake or missed
§ Förstner, Scriptum Photogrammetrie I, someone, please let me know.
Chapter 13.3 § The photogrammetry material heavily relies on the very well
written lecture notes by Wolfgang Förstner and the
Photogrammetric Computer Vision book by Förstner & Wrobel.
§ Parts of the robotics material stems from the great
Probabilistic Robotics book by Thrun, Burgard and Fox.
§ If you are a university lecturer, feel free to use the course
material. If you adapt the course material, please make sure
that you keep the acknowledgements to others and please
acknowledge me as well. To satisfy my own curiosity, please
send me email notice if you use my slides.

53 Cyrill Stachniss, [email protected] 54