Photogrammetry & Robotics Lab 3D Point to Pixel:

Estimating the Parameters of P

Camera Calibration:
Zhang’s Method

Cyrill Stachniss
pixel trans- world
coordinate formation coordinate

This time we only want the intrinsics!

The slides have been created by Cyrill Stachniss.
1 2

Mapping (Recap) Direct Linear Transform (Recap)

Direct linear transform (DLT) maps any Compute the 11 intrinsic and
object point to the image point extrinsic parameters

control point
observed coordinates
image point c, s, m, (given)
xH, yH 3 translations

3 rotations

3 4
 Zhang’s Method
Compute the 5 intrinsic parameters

Assumption:
You know how DLT works!
control point
observed
image point c, s, m,
xH, yH 3 translations

3 rotations

5 6

Camera Calibration
Zhang’s Method Using a 2D Checkerboard
for Camera Calibration
§ Observed 2D pattern (checkerboard)
Using a Checkerboard
§ Known size and structure

Zhang, A Flexible New Technique for

Camera Calibration, MSR-TR-98-71
7 8
Trick for Checkerboard Trick for Checkerboard
Calibration Calibration
§ Set the world coordinate system § Set the world coordinate system
to the corner of the checkerboard to the corner of the checkerboard
§ All points on the checkerboard
lie in the X/Y plane, i.e., Z=0

9 10

Simplification Simplification
§ The Z coordinate of each point on the § The Z coordinate of each point on the
checkerboard is equal to zero checkerboard is equal to zero

11 12
Simplification Simplification
§ The Z coordinate of each point on the § The Z coordinate of each point on the
checkerboard is equal to zero checkerboard is equal to zero
§ Deleting the 3rd column of the
extrinsic parameter matrix leads to

§ We can delete the 3rd column of the

extrinsic parameter matrix One point observed on the checkerboard
generates such an equation
13 14

Setting Up the Equations for Setting Up the Equations for

Determining the Parameter Determining the Parameter

One point generates the equation:

15 16
Setting Up the Equations for Setting Up the Equations for
Determining the Parameter Determining the Parameter
§ For multiple observed points on the § For multiple observed points on the
checkerboard (in the same image), checkerboard (in the same image),
we obtain we obtain

How to proceed? § Analogous to first steps of the DLT

17 18

DLT-Like Estimation DLT-Like Estimation

§ We estimate a 3x3 homography § We estimate a 3x3 homography
instead of a 3x4 projection matrix instead of a 3x4 projection matrix
§ Rest is identical § Rest is identical
§ We use § We use

§ with § with

19 20
DLT-Like Estimation DLT-Like Estimation
§ Solving the system of linear equations § Solving the system of linear equations
leads to an estimate of leads to an estimate of
§ How many points are needed to § We need to identify at least 4 points
estimate ? as has 8 DoF and each point consists
of 2 observations (x and y)

This provides an estimate of

21 22

DLT-Like Estimation Computing K Given H

§ Solving the system of linear equations
leads to an estimate of
§ We need to identify at least 4 points
as has 8 DoF and each point consists
of 2 observations (x and y)

no rotation matrix, thus

QR decomposition is not
applicable as for DLT
After we have estimated ,
we need to compute from
How to obtain the matrix decomposition?
23 24
Computing K Given H Computing K Given H
§ We need to extract from the matrix § We need to extract from the matrix
we computed via SVD we computed via SVD

Four step procedure:

1. Exploit constrains about
2. Define a matrix
3. This can be computed by solving
another homogeneous linear system
4. Decompose matrix
25 26

Computing K Given H is Exploiting Constraints for

Different from the DLT Solution Determining the Parameter
§ Homography has only 8 DoF
§ No direct decomposition as in DLT
§ Exploit constraints on the parameters

27 28
Exploiting Constraints for Exploiting Constraints
Determining the Parameter

As form an orthonormal basis

29 30

Exploiting Constraints Exploiting Constraints

31 32
Exploiting Constraints Exploiting Constraints

§ Define symmetric and positive definite

matrix

33 34

Exploiting Constraints Exploiting Constraints

§ Define symmetric and positive definite § Define symmetric and positive definite
matrix matrix
§ From B, the calibration matrix can be
recovered through Cholesky decomp.

35
If we know B, then we can compute K 36
Exploiting Constraints Next Step: Compute B
§ Define a vector
of unknowns
§ Define symmetric and positive definite
matrix
§ If we know B, we can compute K
§ Inspect equations above:
§ B consists of the unknowns
§ h are known
§ Two equations that relate B and h
37 38

Exploiting Constraints The Matrix V

§ Define a vector § The matrix is given as
of unknowns

with

§ Construct a system of linear equations

exploiting the prev constraints:
§ For one image, we obtain elements of

(first constraint) (second constraint)

39 40
The Matrix V Solving the Linear System
§ For multiple images, we stack the § The system has a trivial
matrices to a 2n x 6 matrix solution which (invalid matrix )
§ Impose additional constraint
image 1

image n

§ We need to solve the linear system

to obtain and thus
41 42

Solving the Linear System What is Needed?

§ The system has a trivial
solution which (invalid matrix )
§ Impose additional constraint
§ Real measurements are noisy § We need at least 4 points per plane
§ Find the solution that minimizes the to compute the matrix H
squares error § Each plane gives us two equations
§ Since has 5/6 DoF, we need at least
3 different views of a plane
§ Solve as in DLT computation § Solve to compute
43 44
 General Calibration Matrix
§ General calibration matrix is obtained
by combining the one of the affine
camera with the general mapping

Non-Linear Parameters?

45 see lecture on camera parameters 46

Lens Distortion Example Example: Barrel Distortion

§ A standard approach for wide angle
lenses is to model the barrel distortion

§ with being point as projected by

an ideal pin-hole camera
§ with being the distance of the pixel
in the image to the principal point
§ Additional non-linear parameters
47 see lecture on camera parameters 48
Error Minimization Example Results
Lens distortion can be calculated by § Before calibration:
minimizing a non-linear error function

…linearize to obtain a quadratic § After calibration:

function, compute derivative, set
it to 0, solve linear system, iterate…
(solved using Levenberg-Marquardt,
K by Zhang’s m. as initial value)
49 50

Summary on Camera Calibration Camera Calibration Summary

Using a Checkerboard
§ Calibration means estimating the
§ Pinhole camera model (first step) (intrinsic) parameters of a camera
§ Non-linear model for lens distortion § Linear and non-linear errors
(second step) § Linear: 5 parameters
§ Approach to camera calibration that § Zhang: estimate the 5 linear
§ accurately determines the camera parameters using a checkerboard
parameters § Estimate non-linear parameters in a
§ is relatively easy to realize in practice second step

51 52
Literature Slide Information
§ Zhang, A Flexible New Technique for § The slides have been created by Cyrill Stachniss as part of the
photogrammetry and robotics courses.
Camera Calibration, MSR-TR-98-71 § I tried to acknowledge all people from whom I used
(uses a slightly different notation) images or videos. In case I made a mistake or missed
someone, please let me know.
§ Förstner & Wrobel, Photogrammetric § The photogrammetry material heavily relies on the very well
written lecture notes by Wolfgang Förstner and the
Computer Vision, Chapter 11.2 Photogrammetric Computer Vision book by Förstner & Wrobel.
§ Parts of the robotics material stems from the great
Probabilistic Robotics book by Thrun, Burgard and Fox.
§ If you are a university lecturer, feel free to use the course
material. If you adapt the course material, please make sure
that you keep the acknowledgements to others and please
acknowledge me as well. To satisfy my own curiosity, please
send me email notice if you use my slides.

53 Cyrill Stachniss, [email protected] 54