Computer Vision

Projective Geometry
and Calibration
Professor Hager
http://www.cs.jhu.edu/~hager

Jason Corso
http://www.cs.jhu.edu/~jcorso
.
05/09/20 CS 441, Copyright G.D. Hager
 Pinhole cameras
• Abstract camera model - box • Pinhole cameras work in practice
with a small hole in it

05/09/20 CS 441, Copyright G.D. Hager

 Real Pinhole Cameras

Pinhole too big -

many directions are
averaged, blurring the
image

Pinhole too small-

diffraction effects blur
the image

Generally, pinhole
cameras are dark, because
a very small set of rays
from a particular point
hits the screen.

05/09/20 CS 441, Copyright G.D. Hager

 The reason for lenses

Lenses gather and

focus light, allowing
for brighter images.

05/09/20 CS 441, Copyright G.D. Hager

 The thin lens

Thin Lens Properties:

1. A ray entering parallel to optical axis
goes through the focal point.
2. A ray emerging from focal point is parallel 1 1 1
to optical axis − =
3. A ray through the optical center is unaltered z' z f
05/09/20 CS 441, Copyright G.D. Hager
 The thin lens

1 1 1
− = Note that, if the image plane is very
z' z f small and/or z >> z’, then z’ is about f

05/09/20 CS 441, Copyright G.D. Hager

 Field of View
• The effective diameter of a lens (d) is the portion of a lens actually reachable
by light rays.

• The effective diameter and the focal length determine the field of view:

tan w = d /(2 f )
• w is the half the total angular “view” of a lens system.

• Another fact is that in practice points at different distances are imaged,

leading to so-called “circles of confusion” of size d/z | z’-z| where z is the
nominal image plane and z’ is the focusing distance given by the thin lens
equation.

• The “depth of field” is the range of distances that produce acceptably focused
images. Depth of field varies inversely with focal length and lens diameter.

05/09/20 CS 441, Copyright G.D. Hager

 Lens Realities
Real lenses have a finite depth of field, and usually
suffer from a variety of defects

Spherical Aberration

vignetting

05/09/20 CS 441, Copyright G.D. Hager

 Standard Camera Coordinates
• By convention, we place the image in front of the optical center
– typically we approximate by saying it lies one focal distance from the center
– in reality this can’t be true for a finite size chip!

• Optical axis is z axis pointing outward

• X axis is parallel to the scanlines (rows) pointing to the right!

• By the right hand rule, the Y axis must point downward

• Note this corresponds with indexing an image from the upper left to the
lower right, where the X coordinate is the column index and the Y
coordinate is the row index.

05/09/20 CS 441, Copyright G.D. Hager

 The equation of projection
• Equating z’ and f x y
– We have, by similar triangles, (x,y,z) → ( f , f )
that (x, y, z) -> (-f x/z, -f y/z, -f) z z
– Ignore the third coordinate, and
flip the image around to get:

05/09/20 CS 441, Copyright G.D. Hager

 Distant objects are smaller

05/09/20 CS 441, Copyright G.D. Hager

 Parallel lines meet

common to draw film plane

in front of the focal point

A Good Exercise: Show this is the case!

05/09/20 CS 441, Copyright G.D. Hager

 Some Useful Geometry
• In 3D space
– points:
• Cartesian point (x,y,z)
• Projective pt (x,y,z,w) with convention that w is a scale factor
– lines:
• a point p on the line and unit vector v for direction
– for minimal parameterization, p is closest point to origin
• Alternative, a line is the intersection of two planes (see below)
– planes
• a point p on the plane and a unit normal n s.t. n . (p’ - p) = 0
• multiplying through, also n.p’ - d = 0, where d is distance of closest pt to
origin.
• any vector n . q = 0 where q is a projective pt
– note, for two planes, the intersection is two equations in 4 unknowns up to
scale --- i.e. a one-dimensional subspace, or a line
• Note that planes and points are dual --- in the above, I can equally think
of n or q as the normal (resp. point).

05/09/20 CS 441, Copyright G.D. Hager

 Some Useful Geometry
• In 2D space
– points:
• Cartesian point (x,y)
• Projective pt (x,y,w) with convention that w is a scale factor
– lines
• a point p on the line and a unit normal n s.t. n . (p’ - p) = 0
• multiplying through, also n.p’ - d = 0, where d is distance of closest pt to
origin.
• any vector n . q = 0 where q is a projective pt
– note, for two lines, the intersection is two equations in 3 unknowns up to
scale --- i.e. a one-dimensional subspace, or a point
• note that points and lines are dual --- I can think of n or q as the normal
(resp. point)

05/09/20 CS 441, Copyright G.D. Hager

 Some Projective Concepts
• The vector p = (x,y,z,w)’ is equivalent to the vector k p for nonzero k
– note the vector p = 0 is disallowed from this representation

• The vector v = (x,y,z,0)’ is termed a “point at infinity”; it corresponds to a

direction

• In P2,
– given two points p1 and p2, l = p1 £ p2 is the line containing them
– given two lines, l1, and l2, p = l1 £ l2 is point of intersection
– A point p lies on a line l if p ¢ l = 0 (note this is a consequence of the triple
product rule)
– l = (0,0,1) is the “line at infinity”
– it follows that, for any point p at infinity, l¢ p = 0, which implies that points at
infinity lie on the line at infinity.

05/09/20 CS 441, Copyright G.D. Hager

 Some Projective Concepts
• The vector p = (x,y,z,w)’ is equivalent to the vector k p for nonzero k
– note the vector p = 0 is disallowed from this representation

• The vector v = (x,y,z,0)’ is termed a “point at infinity”; it corresponds to a

direction

• In P3,
– A point p lies on a plane l if p ¢ l = 0 (note this is a consequence of the triple
product rule; there is an equivalent expression in determinants)
– l = (0,0,0,1) is the “plane at infinity”
– it follows that, for any point p at infinity, l¢ p = 0, which implies that points at
infinity lie on the line at infinity.

05/09/20 CS 441, Copyright G.D. Hager

 Some Projective Concepts
• The vector p = (x,y,z,w)’ is equivalent to the vector k p for nonzero k
– note the vector p = 0 is disallowed from this representation

• The vector v = (x,y,z,0)’ is termed a “point at infinity”; it corresponds to a direction

• Plucker coordinates
– In general, a representation for a line through points p 1 and p2 is given by all possible 2x2
determinants of [p1 p2] (an n by 2 matrix)
• u = (l4,1,l4,1, l4,3,l2,3,l3,1,l1,2) are the Plűcker coordinates of the line passing through the two points.
• if the points are not at infinity, then this is also the same as (p2 - p1, p1 x p2)
– The first 3 coordinates are the direction of the line
– The second 3 are the normal to the plane (in R 3) containing the origin and the points
– In general, a representation for a plane passing through three points p 1, p2 and p3 are the
determinants of all 3 by 3 submatrices [p1 p2 p3]
• let li,j mean the determinant of the matrix of matrix formed by the rows i and j
• P = (l234,l134,l142,l123)
• Note the three points are colinear if all four of these values are zero (hence the original 3x4 matrix
has rank 2, as we would expect).
– Two lines are colinear if we create the 4x4 matrix [p 1,p2,p’1,p’2] where the p’s come from
one line, and the p’s come from another.

05/09/20 CS 441, Copyright G.D. Hager

 Parallel lines meet
• First, show how lines project to images.

• Second, consider lines that have the same direction (are parallel)

• Third, consider the degenerate case of lines parallel in the image

– (by convention, the vanishing point is at infinity!)

A Good Exercise: Show this is the case!

05/09/20 CS 441, Copyright G.D. Hager

 Vanishing points
• Another good exercise (really follows from the
previous one): show the form of projection of *lines*
into images.

• Each set of parallel lines (=direction) meets at a

different point
– The vanishing point for this direction

• Sets of parallel lines on the same plane lead to

collinear vanishing points.
– The line is called the horizon for that plane

05/09/20 CS 441, Copyright G.D. Hager

 The Camera Matrix
• Homogenous coordinates for 3D
– four coordinates for 3D point
– equivalence relation (X,Y,Z,T) is the same as (k X, k Y, k Z,k T)
• Turn previous expression into HC’s
– HC’s for 3D point are (X,Y,Z,T)
– HC’s for point in image are (U,V,W)

⎛ X⎞
⎛U ⎞ ⎛1 0 0 0⎞
⎜Y ⎟ U V
⎜ V ⎟ =⎜0 1 ⎟
0 0⎜ (U ,V ,W ) → ( , ) = (u, v )
⎜ ⎟ ⎜ Z ⎟ W W
⎝W⎠ ⎝0 0 ⎟
1 0⎜ ⎟
f ⎠⎝ T ⎠

05/09/20 CS 441, Copyright G.D. Hager

 Orthographic projection

u=x
Suppose I let f go to infinity; then
v=y
05/09/20 CS 441, Copyright G.D. Hager
 The model for orthographic
projection

⎛X⎞
⎛U ⎞ ⎛1 0 0 0⎞
⎜Y ⎟
⎜ V ⎟ =⎜0 1 0 0⎟
⎜ ⎟ ⎜ ⎟⎜ Z⎟
⎝W⎠ ⎝0 0 0 1⎠⎜ ⎟
⎝T ⎠

05/09/20 CS 441, Copyright G.D. Hager

 Weak perspective
• Issue u = sx
– perspective effects, but not over
the scale of individual objects
– collect points into a group at
v = sy
s = f /Z*
about the same depth, then
divide each point by the depth of
its group
– Adv: easy
– Disadv: wrong

05/09/20 CS 441, Copyright G.D. Hager

 The model for weak perspective
projection

⎛X ⎞
⎛U ⎞ ⎛1 0 0 0 ⎞⎜ ⎟
⎜ ⎟ ⎜ ⎟⎜ Y ⎟
⎜V ⎟ = ⎜0 1 0 0 ⎟⎜ ⎟
⎜W ⎟ ⎜0 0 0 Z * / f ⎟⎜ Z ⎟
⎝ ⎠ ⎝ ⎠⎜ T ⎟
⎝ ⎠

05/09/20 CS 441, Copyright G.D. Hager

 The Affine Camera
• Choose a nominal point x0, y0, z0 and describe projection relative to
that point

• u = f [x0/z0 + (x-x0)/z0 - x0/z20 (z – z0) ] = f (a1 x + a2 z + d1)

• v = f [y0/z0 + (y – y0)/z0 – y0/z20 (z – z0) = f (a3 y + a4 z + d2)

• gathering up
alternatively:
• A = [a1 0 a2; 0 a3 a4]
⎛X ⎞
• d = [d1; d2] ⎛U ⎞ ⎛a1 0 a2 d1 ⎞⎜ ⎟
⎜ ⎟ ⎜ ⎟⎜ Y ⎟
⎜V ⎟ = ⎜ 0 a3 a4 d 2 ⎟⎜ ⎟
• u=AP+d ⎜W ⎟ ⎜ 0 ⎟ Z
⎝ ⎠ ⎝ 0 0 1 / f ⎠⎜ ⎟⎟
⎜
⎝T ⎠

05/09/20 CS 441, Copyright G.D. Hager

 Geometric Transforms

In general, a point in n-D space transforms by

P’ = rotate(point) + translate(point)

In 2-D space, this can be written as a matrix equation:

⎛ x ' ⎞ ⎛Cos(θ ) −Sin(θ ) ⎞⎛ x ⎞ ⎛tx ⎞

⎜⎜ ⎟⎟ = ⎜⎜ ⎟
⎟⎜
⎜ ⎟
⎟ +⎜
⎜ ⎟⎟
⎝ y ' ⎠ ⎝ Sin(θ ) Cos(θ ) ⎠⎝ y ⎠ ⎝ty ⎠

In 3-D space (or n-D), this can generalized as a matrix equation:

p’ = R p + T or p = Rt (p’ – T)

05/09/20 CS 441, Copyright G.D. Hager

 Geometric Transforms
Now, using the idea of homogeneous transforms,
we can write:

⎛ R T⎞
p'= ⎜
⎜0 0 0 1 ⎟
⎟p
⎝ ⎠

R and T both require 3 parameters. These correspond

to the 6 extrinsic parameters needed for camera calibration

05/09/20 CS 441, Copyright G.D. Hager

 Intrinsic Parameters
Intrinsic Parameters describe the conversion from
unit focal length metric to pixel coordinates (and the reverse)

xmm = - (xpix – ox) sx --> -1/sx xmm + ox = xpix

ymm = - (ypix – oy) sy --> -1/sy ymm + oy = ypix

or
⎛x ⎞ ⎛−1 / s x 0 ox ⎞⎛ x ⎞
⎜ ⎟ ⎜ ⎟⎜ ⎟
⎜y⎟ =⎜ 0 −1 / s y o y ⎟⎜ y ⎟ = K int p
⎜w ⎟ ⎜ 0 0 1 ⎟⎠⎜⎝w ⎟⎠mm
⎝ ⎠pix ⎝
It is common to combine scale and focal length together
as the are both scaling factors; note projection is unitless in this case!
05/09/20 CS 441, Copyright G.D. Hager
 Camera parameters
• Summary:
– points expressed in external frame
– points are converted to canonical camera coordinates
– points are projected
– points are converted to pixel units

⎛X ⎞
⎛U ⎞ ⎛Transforma tion ⎞⎛Transforma tion ⎞⎛Transforma tion ⎞⎜ ⎟
⎜ ⎟ ⎜ ⎟⎜ ⎟⎜ ⎟⎜ Y ⎟
⎜V ⎟ = ⎜representing ⎟⎜representing ⎟⎜representing ⎟⎜ ⎟
⎜W ⎟ ⎜intrinsic parameters ⎟⎜projection model ⎟⎜extrinsic parameters ⎟⎜ Z ⎟
⎝ ⎠ ⎝ ⎠⎝ ⎠⎝ ⎠⎜ T ⎟
⎝ ⎠
point in pixel point in metric point in cam.
point in
coords. image coords. coords.
05/09/20 CS 441, Copyright G.D. Hager world coords.
 Lens Distortion
• In general, lens introduce minor irregularities into images, typically
radial distortions:

x = xd(1 + k1r2 + k2r4)

y = yd(1 + k1r2 + k2r4)
r2 = xd2 + yd2

• The values k1 and k2 are additional parameters that must be

estimated in order to have a model for the camera system.

05/09/20 CS 441, Copyright G.D. Hager

 Summary: Other Models
• The orthographic and scaled orthographic cameras (also called weak perspective)
– simply ignore z
– differ in the scaling from x/y to u/v coordinates
– preserve Euclidean structure to a great degree

• The affine camera is a generalization of orthographic models.

– u=Ap+d
– A is 2 x 3 and d is 2x1
– This can be derived from scaled orthography or by linearizing perspective about a point not
on the optical axis

• The projective camera is a generalization of the perspective camera.

– u’ = M p
– M is 3x4 nonsingular defined up to a scale factor
– This just a generalization (by one parameter) from “real” model

• Both have the advantage of being linear models on real and projective spaces,
respectively.

05/09/20 CS 441, Copyright G.D. Hager

 Model Stratification
Euclidean Similarity Affine Projective
Transforms
rotation x x x x
translation x x x x
uniform scaling x x x
nonuniform scaling x x
shear x x
perspective x
composition of proj. x
Invariants
length x
angle x x
ratios x x
parallelism x x x
incidence/cross rat. x x x x
05/09/20 CS 441, Copyright G.D. Hager
 Why Projective (or Affine or ...)
• Recall in Euclidean space, we can define a change of coordinates by choosing a new origin
and three orthogonal unit vectors that are the new coordinate axes
– The class of all such transformation is SE(3) which forms a group
– One rendering is the class of all homogeneous transformations
– This does not model what happens when things are imaged (why?)
• If we allow a change in scale, we arrive at similarity transforms, also a group
– This sometimes can model what happens in imaging (when?)
• If we allow the 3x3 rotation to be an arbitrary member of GL(3) we arrive at affine
transformations (yet another group!)
– This also sometimes is a good model of imaging
– The basis is now defined by three arbitrary, non-parallel vectors

• The process of perspective projection does not form a group

– that is, a picture of a picture cannot in general be described as a perspective projection

• Projective systems include perspectivities as a special case and do form a group

– We now require 4 basis vectors (three axes plus an additional independent vector)
– A model for linear transformations (also called collineations or homographies) on P n is GL(n+1) which
is, of course, a group

05/09/20 CS 441, Copyright G.D. Hager

 Planar Homographies
• First Fundamental Theorem of Projective Geometry:
– There exists a unique homography that performs a change of basis
between two projective spaces of the same dimension.

– Projection Becomes

– Notice that the homography is defined up to scale (s).

05/09/20 CS 441, Copyright G.D. Hager
 Estimating A Homography
• Here is what looks like a reasonable recipe for computing
homographies:
– Planar pts (x1;y1;1, x2; y2; 1, .... xn;yn;1) = X
– Corresponding pts (u1;v1;1,u2;v2;1,...un;vn;1) = U
– U=HX
– U X’ (X X’)-1 = H

• The problem is that X will not be full rank (why?). So we’ll have
to work a little harder ...

05/09/20 CS 441, Copyright G.D. Hager

 An Example Using Homographies
• Image rectification is the computation of an image as seen by a
rotated camera
– The computation of the planar reprojection is a homography
– we’ll show later that depth doesn’t matter when rotating; for now we’ll
just use intuition

Original image plane

New image plane

05/09/20 CS 441, Copyright G.D. Hager

 Rectification Using Homographies
• Pick a rotation matrix R from old to new image

• Consider all points in the image you want to compute; then

– construct pixel coordinates x = (u,v,1)
– K maps unit focal length metric coordinates to pixel (normalized
camera)
– x’ = K Rt K-1 x  x’ = H x
pixel coordinates
pixel coordinates
rotation pixel to euclidean coords
euclidean to pixel coords

• Sample a point x’ in the original image for each point x in the

new.

05/09/20 CS 441, Copyright G.D. Hager

 Bilinear Interpolation
• A minor detail --- new value x’ = (u’,v’,1) may not be integer

• let u’ = i + fu and v’ = j+fv

• New image value b = (1-fu)((1-fv)I(j,i) + fv I(j+1,i)) +

fu((1-fv)I(j,i+1) + fv I(j+1,i+1))

05/09/20 CS 441, Copyright G.D. Hager

 Rectification: Basic Algorithm
• 1. Create a mesh of pixel coordinates for the rectified image
• 2. Turn the mesh into a list of homogeneous points
• 3. Project *backwards* through the intrinsic parameters to get unit
focal length values
• 4. Rotate these values back to the current camera coordinate
system.
• 5. Project them *forward* through the intrinsic parameters to get
pixel coordinates again.
– Note equivalently this is the homography K Rt K-1 where K is the
intrinsic parameter matrix
• 6. Sample at these points to populate the rectified image
– typically use bilinear interpolation in the sampling

05/09/20 CS 441, Copyright G.D. Hager

 Rectification Results
.2 rad

.4 rad

.6 rad
05/09/20 CS 441, Copyright G.D. Hager
 Camera calibration
• Issues: • Error minimization:
– what are intrinsic parameters of – Linear least squares
the camera? • easy problem numerically
– what is the camera matrix? • solution can be rather bad
(intrinsic+extrinsic)
– Minimize image distance
• General strategy: • more difficult numerical problem
– view calibration object
• solution usually rather good, but
– identify image points can be hard to find
– obtain camera matrix by – start with linear least squares
minimizing error – Numerical scaling is an issue
– obtain intrinsic parameters from
camera matrix
• Most modern systems employ
the multi-plane method
– avoids knowing absolute
coordinates of calibration poitns

05/09/20 CS 441, Copyright G.D. Hager

 Calibration – Problem Statement
The problem:
Compute the camera intrinsic (4 or more)
and extrinsic parameters (6) using only observed
camera data.

05/09/20 CS 441, Copyright G.D. Hager

 CAMERA CALIBRATION: A WARMUP

rki
= ( xi −ox ) sx
d
Known r
distance = ( xi +1 −xi ) s x
d d

known regular offset r

A simple way to get scale parameters; we can

compute the optical center as the numerical center
and therefore have the intrinsic parameters
05/09/20 CS 441, Copyright G.D. Hager
 Calibration: Another Warmup
• Suppose we want to calibrate the affine camera and we know
ui = A pi + d for many pairs i

• m is mean of u’s and q is mean of p’s; note m = A q + d

• U = [u1 - m,u2 - m, ... un-m] and P = [p1 - q,p2 -q, ... pn - q]

• U = A P  U P’ (P P’)-1 = A

• d is now mean of ui – A pi

05/09/20 CS 441, Copyright G.D. Hager

 Types of Calibration
• Photogrammetric Calibration
• Self Calibration
• Multi-Plane Calibration

05/09/20 CS 441, Copyright G.D. Hager

 Photogrammetric Calibration
• Calibration is performed through imaging a pattern whose
geometry in 3d is known with high precision.

• PRO: Calibration can be performed very efficiently

• CON: Expensive set-up apparatus is required; multiple
orthogonal planes.

• Approach 1: Direct Parameter Calibration

• Approach 2: Projection Matrix Estimation

05/09/20 CS 441, Copyright G.D. Hager

 The General Case
• Affine is “easy” because it is linear and unconstrained (note
orthographic is harder because of constraints)

• Perspective case is also harder because it is both nonlinear and

constrained

• Observation: optical center can be computed from the

orthocenter of vanishing points of orthogonal sets of lines.

05/09/20 CS 441, Copyright G.D. Hager

 Basic Equations

05/09/20 CS 441, Copyright G.D. Hager

 Basic Equations

05/09/20 CS 441, Copyright G.D. Hager

 Basic Equations

one of these for each point

05/09/20 CS 441, Copyright G.D. Hager

 Properties of SVD
• Recall the singular values of a matrix are related to its rank.

• Recall that Ax = 0 can have a nonzero x as solution only if A is

singular.

• Finally, note that the matrix V of the SVD is an orthogonal basis

for the domain of A; in particular the zero singular values are the
basis vectors for the null space.

• Putting all this together, we see that A must have rank 7 (in this
particular case) and thus x must be a vector in this subspace.

• Clearly, x is defined only up to scale.

05/09/20 CS 441, Copyright G.D. Hager

 Basic Equations

05/09/20 CS 441, Copyright G.D. Hager

 Basic Equations

We now know Rx and Ry up to a sign and .

Rz = Rx x Ry

We will probably use another SVD to orthogonalize

this system (R = U D V’; set D to I and multiply).
05/09/20 CS 441, Copyright G.D. Hager
 Last Details
• We still need to compute the correct sign.
– note that the denominator of the original equations must be positive
(points must be in front of the cameras)
– Thus, the numerator and the projection must disagree in sign.
– We know everything in numerator and we know the projection, hence
we can determine the sign.

• We still need to compute Tz and fx

– we can formulate this as a least squares problem on those two
values using the first equation.

05/09/20 CS 441, Copyright G.D. Hager

 Direct Calibration: The Algorithm
1. Compute image center from orthocenter
2. Compute the A matrix (6.8)
3. Compute solution with SVD
4. Compute gamma and alpha
5. Compute R (and normalize)
6. Compute fx and and Tz
7. If necessary, solve a nonlinear regression to get distortion
parameters

05/09/20 CS 441, Copyright G.D. Hager

 Indirect Calibration: The Basic Idea
• We know that we can also just write
– u h = M ph
– x = (u/w) and y = (v/w), uh = (u,v,1)’
– As before, we can multiply through (after plugging in for u,v, and w)

• Once again, we can write

– Am=0

• Once again, we use an SVD to compute m up to a scale factor.

05/09/20 CS 441, Copyright G.D. Hager

 Getting The Camera Parameters

05/09/20 CS 441, Copyright G.D. Hager

 Getting The Camera Parameters
FIRST:

|q3| is scale up to sign;

divide by this value

M3,4 is Tz up to sign, but

Tz must be positive; if not
THEN: divide M by -1

Ry = (q2 – oy Rz)/fy ox = q1 . q3
Rx = R y x Rz oy = q2 . q3
Tx = -(q4,1 – ox Tz)/fx fx = (q1 . q1 – ox2)1/2
Ty = -(q4,2 – oy Tz)/fy fy = (q2 . q2 – oy2)1/2

Finally, use SVD to orthogonalize the rotation,

05/09/20 CS 441, Copyright G.D. Hager
 Self-Calibration
• Calculate the intrinsic parameters solely from point
correspondences from multiple images.
• Static scene and intrinsics are assumed.
• No expensive apparatus.
• Highly flexible but not well-established.
• Projective Geometry – image of the absolute conic.

05/09/20 CS 441, Copyright G.D. Hager

 Multi-Plane Calibration
• Hybrid method: Photogrammetric and Self-Calibration.
• Uses a planar pattern imaged multiple times (inexpensive).
• Used widely in practice and there are many implementations.
• Based on a group of projective transformations called
homographies.

• m be a 2d point [u v 1]’ and M be a 3d point [x y z 1]’.

• Projection is

05/09/20 CS 441, Copyright G.D. Hager

 Computing the Intrinsics
• We know that

• From one homography, how many constraints on the intrinsic

parameters can we obtain?
– Extrinsics have 6 degrees of freedom.
– The homography has 8 degrees of freedom.
– Thus, we should be able to obtain 2 constraints per homography.

• Use the constraints on the rotation matrix columns…

05/09/20 CS 441, Copyright G.D. Hager

 Computing Intrinsics
• Rotation Matrix is orthogonal….

• Write the homography in terms of its columns…

05/09/20 CS 441, Copyright G.D. Hager

 Computing Intrinsics
• Derive the two constraints:

05/09/20 CS 441, Copyright G.D. Hager

 Closed-Form Solution

• Notice B is symmetric, 6 parameters can be written as a vector b.

• From the two constraints, we have

• Stack up n of these for n images and build a 2n*6 system.

• Solve with SVD (yet again).
• Extrinsics “fall-out” of the result easily.

05/09/20 CS 441, Copyright G.D. Hager

 Non-linear Refinement
• Closed-form solution minimized algebraic distance.
• Since full-perspective is a non-linear model
– Can include distortion parameters (radial, tangential)
– Use maximum likelihood inference for our estimated parameters.

05/09/20 CS 441, Copyright G.D. Hager

 Multi-Plane Approach In Action
• …if we can get matlab to work…

05/09/20 CS 441, Copyright G.D. Hager

 Calibration Summary
• Two groups of parameters:
– internal (intrinsic) and external (extrinsic)
• Many methods
– direct and indirect, flexible/robust
• The form of the equations that arise here and the way they are
solved is common in vision:
– bilinear forms
– Ax = 0
– Orthogonality constraints in rotations

• Most modern systems use the method of multiple planes (matlab

demo)
– more difficult optimization over a large # of parameters
– more convenient for the user

05/09/20 CS 441, Copyright G.D. Hager