GEO1016

Photogrammetry and 3D Computer Vision

Lecture
Camera Models

Liangliang Nan
What is an image?
c

2
What is an image?
c

P = f (x, y)
f : R2 Þ R

Pixel

3
What is an image?
c

- Defined over a rectangle

- Intensity values
f(x, y) ∈ [0, 255]
height

f(x, y) = 97

f(x, y) = 213
f(x, y) = 0

width 4
What is an image?
c

• Projection of the scene on the image plane

• Digital (discrete) image
– A matrix of integer values

5
What is an image?
c

A color image: R, G, B channels

“vector-valued” function

6
Today’s agenda
c

• Images
• Camera Models

7
We see the world through our eyes…
c
cornea
(focus light)
lens retina
(captures image)

optic nerves
(transmit image)

pupil
(control amount of
light entering the eye)
8
Machines see the world through cameras…
c

• Camera is structurally the same as our eyes

– Lens does similarly to our lens and cornea
– Sensor receives the light signals to form images

9
Machines see the world through cameras…
c

• Camera is structurally the same as our eyes

– Lens does similarly to our lens and cornea
– Sensor receives the light signals to form images
– Aperture controls the amount of light

10
Camera vs. eye
c

• Similarities
– Image focusing (cornea & lens)
– Light adjustment (pupil & aperture)
• Differences (just name a few)
– Lens focus
• Camera: lens moves closer/further from the film
• Eye: lens changes shape to focus
– Sensitivity to light
• Camera: A film is designed to be uniformly sensitive to light
• Eye: retina is not; has greater sensitivity in dark 11
Imaging …
c

• Images are 2D projections of real-world scenes

• Images capture two kinds of information:
– Geometric: points, lines, curves, etc.
– Photometric: intensity, color
• Complex 3D-2D relationships
– Camera models approximate relationships

12
Camera models
c

• Pinhole camera model

• Perspective projection model
– Most commonly used model

13
Pinhole camera model
c

• One-to-one mapping

14
Pinhole camera model
c

• Assumption: aperture is a single point.

2 mm 1mm

0.6mm 0.35 mm

15
Pinhole camera model
c
Aperture
film (camera center)

What is the transformation between images on the two image planes 16

Pinhole camera model
c

• 3D point P = (X, Y, Z)T projected to 2D image p = (x, y)T

camera coordinate
system

Right-handed coordinate system

17
Pinhole camera model
c

• 3D point P = (X, Y, Z)T projected to 2D image p = (x, y)T

camera coordinate
system

𝑋 𝑌
𝑥=𝑓 , y=𝑓
𝑍 𝑍

18
Pinhole camera model
c

• 3D point P = (X, Y, Z)T projected to 2D image p = (x, y)T

camera coordinate 𝑋 𝑥 𝑌 𝑦
system = , =
𝑍 𝑓 𝑍 𝑓

𝑋 𝑌
𝑥=𝑓 , y=𝑓
𝑍 𝑍
Simplest form of perspective projection

19
Pinhole camera model
c

• Compromise between sharpness and brightness

2 mm 1mm

0.6mm 0.35 mm Can we develop cameras that

take images both sharp and bright

20
Perspective projection model
c

• Balancing sharpness and brightness using lenses

– Replace the pinhole with a lens
– The lens refracts light
– The lens converges all rays of light of same points to a single point

21
Perspective projection model
c

• Lens
– Converges all rays of light of same points to a single point
– Focuses parallel light rays to the focal point

22
Perspective projection model
c

• Pinhole camera model

𝑋 𝑌
𝑥=𝑓 , y=𝑓
𝑍 𝑍
Image plane coordinates in physical measurements (mm)

• Change of unit: physical measurements -> pixels

𝑋 𝑌
𝑥 = 𝑘𝑓 , y = 𝑙𝑓
𝑍 𝑍

- x, y: image coordinates (pixels)

What if k = l - k, l: scale parameters (pixels/mm)
- f : focal length (mm)
23
Perspective projection model
c

• Pinhole camera model

𝑋 𝑌
𝑥=𝑓 , y=𝑓
𝑍 𝑍
Image plane coordinates in physical measurements (mm)

• Change of unit: physical measurements -> pixels

𝑋 𝑌
𝑥 = 𝑘𝑓 , y = 𝑙𝑓
𝑍 𝑍
Denote 𝛼 = 𝑘𝑓, 𝛽 = 𝑙𝑓

𝑋 𝑌
𝑥=𝛼 , y=𝛽
𝑍 𝑍
24
Perspective projection model
c

• Change of coordinate system

– Image plane coordinates have origin at image center
– Digital image coordinates have origin at top-left corner

Image plane coordinates Digital image coordinates

25
Perspective projection model
c

• Change of coordinate system

– Image plane coordinates have origin at image center
– Digital image coordinates have origin at top-left corner

Image center (principal point): (cx , cy)

26
Perspective projection model
c

• Change of coordinate system

– Image plane coordinates have origin at image center
– Digital image coordinates have origin at top-left corner

Image center (principal point): (cx , cy)

27
Perspective projection model
c

• Image frame may not be exactly rectangular

– Due to sensor manufacturing errors

θ: skew angle between x- and y-axis

28
Perspective projection model
c

• Other types of distortions

– Common aberration: radial distortion
• Different portions of the lens have differing focal lengths
• Straight lines appear curved
• Errors by radial distortion << scanning resolution of image

29
Normal Pincushion Barrel
Perspective projection model
c

• Other types of distortions

– Common aberration: radial distortion
• Different portions of the lens have differing focal lengths
• Straight lines appear curved
• Errors by radial distortion << scanning resolution of image

30
Intrinsic parameters
c

• Rewrite in matrix-vector product form

𝐏 = 𝑋, 𝑌, 𝑍 T , 𝐩 = 𝑥, 𝑦, 1 T

(homogeneous coordinates)

𝐩 = 𝐾𝐏, 𝐾=

31
Intrinsic parameters
c

• Rewrite in matrix-vector product form

𝐏 = 𝑋, 𝑌, 𝑍 T , 𝐩 = 𝑥, 𝑦, 1 T

(homogeneous coordinates)

𝛼 −𝛼 cot 𝜃 𝑐𝑥
𝛽
𝐩 = 𝐾𝐏, 𝐾= 0 𝑐𝑦
sin 𝜃
0 0 1
Intrinsic parameter matrix 32
Intrinsic parameters
c

• For simplicity, people use a simpler form of K

𝛼 −𝛼 cot 𝜃 𝑐𝑥 𝑓𝑥 𝑠 𝑐𝑥 s: skew parameter
𝛽
𝐾= 0 𝑐𝑦 = 0 𝑓𝑦 𝑐𝑦
sin 𝜃
0 0 1 0 0 1

• Internal characteristics
– focal length, skew distortion, and image center.

33
Extrinsic parameters
c

• Camera motion
– World frame may not align with the camera frame
– Camera can move and rotate

camera frame

world frame
34
Extrinsic parameters
c

• Camera motion
– World frame may not align with the camera frame
– Camera can move and rotate
Camera frame

𝐏𝐶 = 𝑅𝑊
𝐶
𝐏𝑊 + 𝐭𝐶𝑊
World frame
1 3 2 4

1. Coordinates of 3D scene point in camera frame.

2. Coordinates of 3D scene point in world frame.
3. Rotation matrix of world frame in camera frame.
4. Position of world frame’s origin in camera frame.
35
Complete camera model
c

• Combine intrinsic and extrinsic parameters

𝐩 = 𝐾𝐏, 𝐏𝐶 = 𝑅𝑊
𝐶
⋅ 𝐏𝑊 + 𝐭𝐶𝑊

𝐶
𝐩 = 𝐾(𝑅𝑊 ⋅ 𝐏𝑊 + 𝐭𝐶𝑊 )

• Use a simpler notation

𝐩 = 𝐾 𝑅𝐏 + 𝐭
=𝐾 𝑅𝐭𝐏
= 𝑀𝐏
𝑀 = 𝐾 𝑅 𝐭 : 3 x 4 projection matrix 36
Summary
c

• Simplest camera model: pinhole model.

• Most commonly used model: perspective model.
• Intrinsic parameters:
– Focal length, principal point (image center), skew factor
• Extrinsic parameters:
– Camera rotation and translation.

Further reading:
R. Szeliski. Computer Vision: Algorithms and Applications. Springer, 2010.
- Camera models: Section 2.1.5
- Lens distortion: Section 2.1.6
37
Next lecture
c

• Camera calibration

38