Lectures 9-10: Imaging Geometry and Camera

Model

Dr. V Masilamani
[email protected]
Department of Computer Science and Engineering
IIITDM Kancheepuram
Chennai-127

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 1 / 37

Overview
1 Multiple View Imaging Geometry

2 Single View Imaging Geometry

3 Geometric Transformation in 3D
Translation in 3D
Scaling in 3D
Rotation in 3D
Shearing in 3D
Perspective Transformation in 3D

4 Camera Modeling

5 Camera Calibration
6 Acknowledgements

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 2 / 37

Multiple View Imaging Geometry

I What is multiple view imaging

geometry

• Geometry of images of the

same scene taken by many
camera

I Objective of multiple view

imaging geometry

• Given the multiple images

of the same scene, find the
geometry of 3D object

• The geometry of 3D object

means: shape, volume,
surface area, length,
position etc.

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 3 / 37

Multiple View Imaging Geometry (cont.)

Some Applications of Multiple

View Geometry

I Animating the real actor

I Depth computation

I Image Stitching

I 3D Model construction

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 4 / 37

Single View Imaging Geometry

I Coordinate System: • p can be written as a linear

(O, (X , Y , Z ) combination of those
orthonormal basis vectors
• O is a point in the space
called origin • Example:
(a, b, c) = a(1, 0, 0) +
• X , Y , Z are ortho-normal
b(0, 1, 0) + c(0, 0, 1)
basis vectors
I Example of a Coordinate
system
• O = (0, 0, 0)
• X = (1, 0, 0), Y = (0, 1, 0),
Z = (0, 0, 1)
I How to represent a point p in
a coordinate system

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 5 / 37

Single View Imaging Geometry (cont.)

How to relate a point on an object and the corresponding point on

its image

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 6 / 37

Single View Imaging Geometry (cont.)

The coordinate systems involved in Imaging geometry

I World Coordinate system(3D):
• The coordinate system used to locate the points of the object
• The coordinates (x, y , z) ∈ R 3

I Image Coordinate system (2D)

• Origin: The center of the image plane
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ R 2

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 7 / 37

Single View Imaging Geometry (cont.)

I Camera Coordinate system(3D):

• Origin: The center of the lens (also called as projection center)
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ R 3
I Pixel Coordinate system (2D)
• Origin: Top most-Left most point in the image plane
• X , Y axes are the X , Y axes of the image plane in the camera
• The coordinates (x, y , z) ∈ A × B, where A = {0, 1, 2, ..., (M − 1)
and B = {0, 1, 2, ..., (N − 1)

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 8 / 37

Geometric Transformation in 3D

I Definition of Geometric Transformation:

• Transformation that may change the geometry of the objects
I The geometry of object refers to
• Shape, size, distance, positions etc.
I Some Geometric Transformations
• Translation
• Scaling
• Rotation
• Shearing
• Perspective

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 9 / 37

Translation in 3D
   
x1 x2
I Translation of a 3D Point: y 1 → y 2
z1 z2
where
x2 = x1 + ∆x
y 2 = y 1 + ∆y
z2 = z1 + ∆z

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 10 / 37

Translation in 3D (cont.)

I HW: Prove that Translation is not linear in 3D Cartesian Coordinate

system
I To make translation linear, transform the Cartesian coordinate
system to homogeneous coordinate system
I Homogeneous Coordinatesystem:

kx
ky 
Coordinates of the form 
kz 


k
I Given Cartesian coordinate system, homogeneous coordinates can be
obtained
I Given homogeneous coordinates, Cartesian coordinate system can be
obtained

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 11 / 37

Translation in 3D (cont.)

I To make 3D translation linear in homogeneous system(4D),

 
  x
x y 
transform 3D point y  to  z 

z
1
I 3D translation in homogeneous
     coordinate
 system(4D):
x2 1 0 0 ∆x x1
y 2 0 1 0 ∆y   y 1
 
 =
z2 0 0 1 ∆z  z1
1 0 0 0 1 1
I Translation
 matrix: 
1 0 0 ∆x
0 1 0 ∆y 
T= 0 0 1 ∆z 


0 0 0 1

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 12 / 37

Translation in 3D (cont.)

 
1 0 0 −∆x
I T −1
0 1 0 −∆y 
= 
0 0 1 −∆z 
0 0 0 1
I Observation: Given the translation vector, T −1 can be computed
directly(without using algorithm for computing inverse)
I Where is translation used
• To model moving object

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 13 / 37

Scaling in 3D

I Scaling of 3D point:
   
x1 ax1
y1  → by1 
z1 cz1
I In
 Matrix
 equation
 form:
 
x2 a 0 0 x1
y 2 = 0 b 0 y 1
z2 0 0 c z1
I In hom. coord. system:

    
x2 a 0 0 0 x1
y 2 0 b 0 0 y 1
 =  
z2 0 0 c 0 z1
1 0 0 0 1 1

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 14 / 37

Scaling in 3D (cont.)

 0  
I x x
= S
y0 y
 
a 0
where, scaling S =
0 b
1 
I S −1 = a 01
0 b
I S −1 S = S −1 S = I

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 15 / 37

Rotation in 3D

I Defn. of Rotation of a point

about Z axis
• P1(x1, y 1, z1) is said to be
rotated about Z axis with
angle θ to result
inP2(x2, y 2, z2) if
z2 = z1, and x2, y 2 are
obtained using 2d Rotation
on XY plane in anti-clock
wise direction
I Similarly rotations of a point
about X and Y axes can be
defined

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 16 / 37

Rotation in 3D (cont.)

Rotations of objects about X, Y and Z directions

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 17 / 37

Rotation in 3D (cont.)

Is the rotation of a point about Z axis in 3D Cartesian coordinate

linear?
I Since the rotation is along Z axis, z1 = z2.
I Let (r , θ) represents P1 on XY plane, and hence, (r , θ + φ) will
represent P2 on XY plane
I sinΦ = y1
r ⇒ y 1 = rsinΦ
I cosΦ = x1
r ⇒ x1 = rcosΦ
I sin(Θ + Φ) = y2
r ⇒ y 2 = rsin(Θ + Φ)
I similarly x2 = rcos(Θ + Φ) x2 = r (cosΘcosΦ − sinΘsinΦ)

x2 = (rcosΦ)cosΘ − (rsinΦ)sinΘ

x2 = x1cosΘ − y 1sinΘ

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 18 / 37

Rotation in 3D (cont.)

I similarly y 2 = rsin(Θ + Φ) y 2 = r (sinΘcosΦ + cosΘsinΦ)

y 2 = (rcosΦ)sinΘ + (rsinΦ)cosΘ
y 2 = x1sinΘ + y 1cosΘ
I Rotation of P1 = (x1, y 1, z1) about Z axis which results in
P2 = (x2, y 2, z2) can be defined as a matrix equation
I 
In 3DCartesian
 coordinates:   
x2 cosΘ −sinΘ 0 x1
y 2 =  sinΘ cosΘ 0 y 1
z2 0 0 1 z1
I In homogeneous coord. system:
P2 = RΘz P1 where 
cosΘ −sinΘ 0 0
Z
 sinΘ cosΘ 0 0
RΘ =  0

0 1 0
0 0 0 1

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 19 / 37

Rotation in 3D (cont.)

I Similarly, rotation about X axis:

P2 = RΘX P1 where 
1 0 0 0
X
0 cosΘ −sinΘ 0
RΘ =   
0 sinΘ cosΘ 0
0 0 0 1
I Rotation about Y axis:
Y
P2 = RΘ P1 where 
cosΘ 0 −sinΘ 0
 0 1 0 0
RΘY =  sinΘ 0 cosΘ 0


0 0 0 1
I Hw: Derive transformation matrices for rotations about Y and Z
axes

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 20 / 37

Shearing in 2D

I Shearing in x Dir:
x 0 = x + sy ; y 0 = y ;

I Shearing in y Dir:
x 0 = x; y 0 = sx + y ;

I Shearing in x-y Directions:

x 0 = x + sy ; y 0 = tx + y ;

I Application: When the film in

the camera not fixed properly
-Shearing effect may happen

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 21 / 37

Shearing in 3D

I 3D-Shearing in x Dir: opposite faces

x 0 = x + by + cz;
y0 = y; z0 = z
I 3D-Shearing in y Dir:
x 0 = x; z 0 = z
y 0 = dx + y + fz;
I 3D-Shearing in Z Dir:
x 0 = x; y 0 = y ;
z 0 = gx + iy + z
I Application: To synthesize
funny cartoon characters
-3D sheering pulls the

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 22 / 37

Perspective Transformation in 3D

where

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 23 / 37

Perspective Transformation in 3D (cont.)

f - is the focal length

Yc - height of the object
-y - height of the image
I Camera coordinate system is considered
I Image plane and XY plane of camera are the same
I Using the property of similar triangles,

h2 f
=
h1 Zc
−y f
Sub value of h1 and h2, =
Yc Zc
−Yc f
Therefore, y =
Zc
−Xc f
Similarly , x =
Zc

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 24 / 37

Perspective Transformation in 3D (cont.)
I Hw: Prove that perspective transformation is not linear in Cartesian
coord. system.
I The perspective transformation in homogeneous coord. system
    
kXc 1 1 0 0 kXc
 kYc  0 1 0 0 kYc 
 kZc  = 0 0 1 0 kZc 
    
−kZc
f
0 0 −1 f 1 k
I To convert perspective transformed points in homogeneous coord.
system to Cartesian coord. sysem

kXc −Xc f
x= −kZc
⇒x =
f
Zc

kYc −Yc f
y= −kZc
⇒y =
f
Zc

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 25 / 37

Camera Modeling

I Observation: Any two coordinate systems can be aligned using 1

translation and 3 rotations.
I Therefore, Camera Coordinate System(CCS) can be obtain from
World Coordinate System(WCS) by translating with (∆x, ∆y , ∆z)
followed by the rotating with Θ1, Θ2, Θ3 about X,Y and Z axes
respectively.
I Let Pw and Pc be the representations of p in the WCS and CCS
respectively

Z −1 Y −1 X −1
Pc = ((RΘ3 ) (RΘ2 ) (RΘ1 ) T −1 ) Pw

Z
Pc = ((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 ) Pw

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 26 / 37

Camera Modeling (cont.)

I Show that (RΘZ )−1 = R−Θ

Z

 
cosΘ −sinΘ 0 0
 sinΘ cosΘ 0 0
RΘZ = 
 0

0 1 0
0 0 0 1
 
cosΘ sinΘ 0 0
−sinΘ cosΘ 0 0
(RΘZ )−1 =
 0

0 1 0
0 0 0 1
Z
= R−Θ
I Let Pi be the output of perspective transformation of pc

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 27 / 37

Camera Modeling (cont.)
I X,Y axes of CCS and ICS are the same, Pi can be considered as a
point in ICS

   
Ih1 Ch1
Ih2  Ch2 
Pi = 
Ih3  P = Ch3 
  

Ih4 Ch4

in hom. coord. system

I To obtain Cartesian coordinates Pp in PCS from Pi in ICS,

Ih1 Ih2
x= y=
Ih4 Ih4

I To obtain Cartesian coordinates of Pi in PCS

• Step 1: Translate Pi so that Left Most and Top Most point is origin
(call the transform as T1 )

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 28 / 37

Camera Modeling (cont.)
• Step 2: Scale the resultant of Step 1 so that coordinates are
Non-negative integers(call the transform as S)
Z
I Pc = ((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 ) Pw
Z
I Pi = PPc = (P((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 )) Pw
Z
I Pp = ST1 Pi = (ST1 P((R−Θ3 Y
) (R−Θ2 X
) (R−Θ1 ) T −1 )) Pw
I
 
Ih10
Ih20  Z Y X −1
Let Pp = 
Ih30  = APw ; A = (ST1 P((R−Θ3 ) (R−Θ2 ) (R−Θ1 ) T ))


Ih40

I The Cartesian coord. of Pp :

Ih10 Ih20
x0 = y0 =
Ih40 Ih40

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 29 / 37

Camera Calibration

I What is camera calibration?

• Camera calibration means finding a matrix A such that Pp = APw
• A is called as camera matrix

I Camera calibration is also called as computaion of camera matrix

How to find the camera matrix A:
I In Pp = APw , we can substitute the values for Pp and Pw
I As Pp = (x 0 , y 0 )T where

Ih10 Ih20
x0 = y0 =
Ih40 Ih40

Ih10 − x 0 Ih40 = 0
Ih20 − y 0 Ih40 = 0

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 30 / 37

Camera Calibration (cont.)
I Sub values of Ih10 , Ih40 , Ih20 , we get
a11 X + a12 Y + a13 Z + a14 − x 0 (a41 X + a42 Y + a43 Z + a44 ) = 0

a21 X + a22 Y + a23 Z + a24 − y 0 (a41 X + a42 Y + a43 Z + a44 ) = 0

I for each pair (Pw , Pp ), two equations are formed
   
a11 a12 a13 a14 X
a21 a22 a23 a24  Y 
I A =  Ph =  
− − − − Z 
a41 a42 a43 a44 1
I Since A is a 4*4 matrix, we have 16 unknowns requiring 16 equations,
thus we need 8 points.
I since Z is not required, we can use 6 points to form 12 equations for 12
unknowns
I But all the equations should be linearly independent which might not be
the case. Hence more points will be required.

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 31 / 37

Camera Calibration (cont.)

I Use n pairs of points, to get 2n equations as follows

a11 X1 + a12 Y1 + a13 Z1 + a14 − x1 (a41 X1 + a42 Y1 + a43 Z1 + a44 ) = 0
.
.
.
a11 Xn + a12 Yn + a13 Zn + a14 − xn (a41 Xn + a42 Yn + a43 Zn + a44 ) = 0

a21 X1 + a22 Y1 + a23 Z1 + a24 − y1 (a41 X1 + a42 Y1 + a43 Z1 + a44 ) = 0

.
.
.
a21 Xn + a22 Yn + a23 Zn + a24 − yn (a41 Xn + a42 Yn + a43 Zn + a44 ) = 0

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 32 / 37

Camera Calibration (cont.)

I To Convert homogeneous system of linear equations to

non-homogeneous system of linear equations
• Divide the equation by a44 on both sides
• Then the aij ← aij /a44
• rearrange the matrix equation as shown in the next slide

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 33 / 37

Camera Calibration (cont.)

I By solving the updated non-homogeneous system of linear

equations(as shown below), we shall get 11 parameters, the 12th
parameter namely a44 = 1

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 34 / 37

Camera Calibration (cont.)

C D = U
C T (C D) = C T U
(C T C ) D = C T U
D = (C T C ) C T U

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 35 / 37

Acknowledgements

I Images are downloaded from internet sources

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 36 / 37

 Thank You! :)

Computer Vision Lectures 9-10: Imaging Geometry and Camera Model 37 / 37