CNG 483

Introduction to Computer Vision

Image Formation

Asst. Prof. Dr. Meryem Erbilek

Slide partially based on Stanford U. CS131

METU Image Formation 1

 Image formation
• How are objects in the world captured in an
image?

Slide partially based on Stanford U. CS131

METU Image Formation 2

 Image formation
• The image formation process produced a particular image
given a set of lighting conditions, scene geometry, surface
properties, camera optics and sensor.

Slide partially based on Stanford U. CS131

METU Image Formation 3

 Physical parameters of image formation
• Geometric
– Type of projection
– Camera pose
• Optical
– Sensor’s lens type
– focal length, field of view, aperture
• Photometric
– Type, direction, intensity of light reaching sensor
– Surfaces’ reflectance properties
• Sensor
– sampling, etc.

Slide partially based on Stanford U. CS131

METU Image Formation 4

 Image formation

• How to form an image of an object?

• Let’s design a camera!
– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?

Slide partially based on Stanford U. CS131 Slide by Steve Seitz

METU Image Formation 5

 Pinhole camera

• Add a barrier to block off most of the rays!

– This reduces blurring
– The opening is known as the aperture
– How does this transform the image?

Slide partially based on Stanford U. CS131 Slide by Steve Seitz https://www.youtube.com/watch?v=IcNEfwNeZss

METU Image Formation 6

 Pinhole camera
• Pinhole camera is a simple model to approximate
imaging process, perspective projection.

Image
plane

Virtual pinhole
image

→If we treat pinhole as a point, only one ray from any given point
can enter the camera.

Slide partially based on Stanford U. CS131 Figure from Forsyth and Ponce

METU Image Formation 7

 First Pinhole Camera: Obscura

In Latin, means
‘dark room’

"Reinerus Gemma-Frisius (an astronomer), had used the pinhole in a darkened room to
study the solar eclipse on January 24, 1544, and later he used this illustration of the event in
his book De Radio Astronomica et Geometrica, 1545. It is thought to be the first published
illustration of a camera obscura..." Hammond, John H., The Camera Obscura, A Chronicle

Slide partially based on Stanford U. CS131 http://www.acmi.net.au/AIC/CAMERA_OBSCURA.html

METU Image Formation 8

 Camera obscura

Jetty at Margate England, 1898.

An attraction in the late

Around 1870s
19th century
Slide partially based on Stanford U. CS131 http://brightbytes.com/cosite/collection2.html Adapted from R. Duraiswami

METU Image Formation 9

 Camera obscura at home

https://www.youtube.com/watch?v=B2aOs8RWntg

Slide partially based on Stanford U. CS131 Sketch from http://www.funsci.com/fun3_en/sky/sky.htm

METU Image Formation 10

 Physical parameters of image
formation
• Geometric
– Type of projection
– Camera pose
• Optical
– Sensor’s lens type
– focal length, field of view, aperture
• Photometric
– Type, direction, intensity of light reaching sensor
– Surfaces’ reflectance properties
• Sensor
– sampling, etc.
Slide partially based on Stanford U. CS131

METU Image Formation 11

 Perspective effects
Distortions / manipulations due to camera viewpoint!

Slide partially based on Stanford U. CS131

METU Image Formation 12

 Perspective effects
• Far away objects appear smaller!

A and C is half size of B

Slide partially based on Stanford U. CS131

METU Image Formation 13

 Perspective effects
Each set of parallel lines in the real 3D world will have a different
vanishing point in the image located on the horizon line.

Slide partially based on Stanford U. CS131

METU Image Formation 14

 Perspective effects
• Image plane is placed in front of the pinhole
• Parallel lines in the scene intersect in the image
• Converge in image on horizon line
Image plane

Pinhole

Scene

Slide partially based on Stanford U. CS131

METU Image Formation 15

 Projection properties
• Many-to-one: any points along same ray map to
same point in image
• Points → points
• Lines → lines
• Distances and angles are not preserved!

• Degenerate cases:
– Line through focal point projects to a point.
– Plane through focal point projects to line.
– Plane perpendicular to image plane projects to part of the
image.
Slide partially based on Stanford U. CS131

METU Image Formation 16

 Perspective and art
• Use of correct perspective projection indicated in 1st century
B.C. frescoes
• Skill resurfaces in Renaissance: artists develop systematic
methods to determine perspective projection (around 1480-
1515)

Raphael Durer, 1525

Slide partially based on Stanford U. CS131

METU Image Formation 17

 Perspective projection equations
• 3d world mapped to 2d projection in image plane
Image
plane
Focal
length

Optical
Camera axis
frame
‘
‘’ ’ Scene / world
points
Scene point Image coordinates

Slide partially based on Stanford U. CS131 Forsyth and Ponce

METU Image Formation 18

 Homogeneous coordinates
Is this (scene → image point) a linear transformation?
• no—division by z is nonlinear
Trick: add one more coordinate:

homogeneous image homogeneous scene

coordinates coordinates

Converting from homogeneous coordinates

Slide partially based on Stanford U. CS131 Slide by Steve Seitz

METU Image Formation 19

 Perspective Projection Matrix
• Projection is a matrix multiplication using homogeneous
coordinates:

divide by the third

coordinate to convert back
to non-homogeneous
coordinates

Slide partially based on Stanford U. CS131 Slide by Steve Seitz

METU Image Formation 20

 Weak perspective
• Approximation: treat magnification as constant
• Assumes scene depth << average distance to camera

Image World
plane points:

Slide partially based on Stanford U. CS131

METU Image Formation 21

 Orthographic projection
• Given camera at constant distance from scene
• World points projected along rays parallel to optical
access

Slide partially based on Stanford U. CS131

METU Image Formation 22

 Physical parameters of image
formation
• Geometric
– Type of projection
– Camera pose
• Optical
– Sensor’s lens type
– focal length, field of view, aperture
• Photometric
– Type, direction, intensity of light reaching sensor
– Surfaces’ reflectance properties
• Sensor
– sampling, etc.
Slide partially based on Stanford U. CS131

METU Image Formation 23

 Pinhole size / aperture
How does the size of the aperture affect the
image we’d get?
Larger

http://www.me.umn.edu/courses/me5286/vision/Notes/2015/ME5286-Lecture2.pdf

Diffraction: Here, light

does not travel in a
Smaller straight line through the
hole, gets scattered in
many directions.
Slide partially based on Stanford U. CS131

METU Image Formation 24

 Adding a lens

focal point

• A lens focuses light onto the film

– Rays passing through the center are not deviated
– All parallel rays converge to one point on a plane
located at the focal length f
Slide partially based on Stanford U. CS131 Slide by Steve Seitz

METU Image Formation 25

 Pinhole vs. lens

Slide partially based on Stanford U. CS131

METU Image Formation 26

 Cameras with lenses

focal point
optical center
(Center Of Projection)

• A lens focuses parallel rays onto a single focal point

• Gather more light, while keeping focus; make
pinhole perspective projection practical
Slide partially based on Stanford U. CS131

METU Image Formation 27

 Thin lens equation

• How to relate distance of object from optical

center (u) to the distance at which it will be in
focus (v), given focal length f?

Slide partially based on Stanford U. CS131

METU Image Formation 28

 Thin lens equation

• How to relate distance of object from optical

center (u) to the distance at which it will be in
focus (v), given focal length f?
Slide partially based on Stanford U. CS131

METU Image Formation 29

 Thin lens equation

• How to relate distance of object from optical

center (u) to the distance at which it will be in
focus (v), given focal length f?
Slide partially based on Stanford U. CS131

METU Image Formation 30

 Thin lens equation

• Any object point satisfying this equation is in focus

Slide partially based on Stanford U. CS131

METU Image Formation 31

 Focus and depth of field

Slide partially based on Stanford U. CS131 Image credit: cambridgeincolour.com

METU Image Formation 32

 Focus and depth of field
• Depth of field: distance between image planes
where blur is tolerable

Thin lens: scene points

at distinct depths come
in focus at different
image planes.
(Real camera lens
systems have greater
depth of field.)
“circles of confusion”

Slide partially based on Stanford U. CS131 Shapiro and Stockman

METU Image Formation 33

 Focus and depth of field
• How does the aperture affect the depth of field?

A smaller aperture increases the range in which the

object appears approximately in focus but reduces the
amount of light into the camera
Slide partially based on Stanford U. CS131 Flower images from Wikipedia http://en.wikipedia.org/wiki/Depth_of_field

METU Slide from S. Seitz Image Formation 34

 Depth of Field - Modern Tricks

Slide partially based on Stanford U. CS131 https://iphonephotographyschool.com/portrait-mode/

METU Image Formation 35

 Field of view

• Angular
measure of
portion of 3d
space seen by
the camera

Slide partially based on Stanford U. CS131 Images from http://en.wikipedia.org/wiki/Angle_of_view

METU Image Formation 36

 Field of view depends on focal length
• As f gets smaller, image
becomes more wide angle
– more world points project
onto the finite image plane
• As f gets larger, image
becomes more telescopic
– smaller part of the world
projects onto the finite
image plane

Slide partially based on Stanford U. CS131 from R. Duraiswami

METU Image Formation 37

 Physical parameters of image
formation
• Geometric
– Type of projection
– Camera pose
• Optical
– Sensor’s lens type
– focal length, field of view, aperture
• Photometric
– Type, direction, intensity of light reaching sensor
– Surfaces’ reflectance properties
• Sensor
– sampling, etc.
Slide partially based on Stanford U. CS131

METU Image Formation 39

 Digital cameras
• Film → sensor array
• Often an array of charge coupled
devices
• Each CCD is light sensitive diode that
converts photons (light energy) to
electrons

camera
CCD
array optics frame
computer
grabber

Slide partially based on Stanford U. CS131

METU Image Formation 40

 Summary
• Image formation affected by geometry, photometry,
and optics.
• Projection equations express how world points
mapped to 2d image.
– Homogenous coordinates allow linear system for
projection equations.
• Lenses make pinhole model practical.
• Parameters (focal length, aperture, lens diameter,…)
affect image obtained.

Slide partially based on Stanford U. CS131

METU Image Formation 41