Today: Cameras

Overview
• The pinhole projection model
• Qualitative properties
• Perspective projection matrix
• Cameras with lenses
• Depth of focus
• Field of view
• Lens aberrations
• Digital cameras
• Types of sensors
• Color
Let’s design a camera

Idea 1: put a piece of film in front of an object

Do we get a reasonable image?

Slide by Steve Seitz

Pinhole camera

Add a barrier to block off most of the rays

• This reduces blurring
• The opening is known as the aperture

Slide by Steve Seitz

Pinhole camera model

Pinhole model:
• Captures pencil of rays – all rays through a single point
• The point is called Center of Projection (focal point)
• The image is formed on the Image Plane

Slide by Steve Seitz

Dimensionality Reduction Machine (3D to 2D)

3D world 2D image

Point of observation

What have we lost?

• Angles
• Distances (lengths)
Slide by A. Efros
Figures © Stephen E. Palmer, 2002
Projection properties
• Many-to-one: any points along same visual
ray map to same point in image
• Points → points
• But projection of points on focal plane is undefined
• Lines → lines (collinearity is preserved)
• But line through focal point (visual ray) projects to a point
• Planes → planes (or half-planes)
• But plane through focal point projects to line
 Vanishing points
• Each direction in space has its own vanishing point
• All lines going in that direction converge at that point
• Exception: directions parallel to the image plane
• All directions in the same plane have vanishing points
on the same line
 Vanishing points
• Each direction in space has its own vanishing point
• All lines going in that direction converge at that point
• Exception: directions parallel to the image plane
• All directions in the same plane have vanishing points
on the same line
• How do we construct the vanishing point/line?
One-point perspective
Masaccio, Trinity, Santa
Maria Novella,
Florence, 1425-28

One of the first

consistent uses of
perspective in
Western art
Perspective distortion
• Problem for architectural photography:
converging verticals

Source: F. Durand
Perspective distortion
• Problem for architectural photography:
converging verticals

Tilting the camera Keeping the camera level, Shifting the lens
upwards results in with an ordinary lens, upwards results in a
converging verticals captures only the bottom picture of the entire
portion of the building subject

• Solution: view camera (lens shifted w.r.t. film)

http://en.wikipedia.org/wiki/Perspective_correction_lens Source: F. Durand

Perspective distortion
• Problem for architectural photography:
converging verticals
• Result:

Source: F. Durand
Perspective distortion
• However, converging verticals work quite well
for horror movies…
Perspective distortion
• What does a sphere project to?

Image source: F. Durand

Perspective distortion
• What does a sphere project to?
Perspective distortion
• The exterior columns appear bigger
• The distortion is not due to lens flaws
• Problem pointed out by Da Vinci

Slide by F. Durand
Perspective distortion: People
Modeling projection
y
f
z

The coordinate system

• The optical center (O) is at the origin
• The image plane is parallel to xy-plane (perpendicular to z axis)

Source: J. Ponce, S. Seitz

Modeling projection
y
f
z

Projection equations
• Compute intersection with image plane of ray from P = (x,y,z) to O
• Derived using similar triangles
x y
( x, y , z )  ( f , f , f )
z z
• We get the projection by throwing out the last coordinate:
x y
( x, y , z )  ( f , f )
z z Source: J. Ponce, S. Seitz
Homogeneous coordinates
x y
( x, y , z )  ( f , f )
z z
Is this a linear transformation?
• no—division by z is nonlinear

Trick: add one more coordinate:

homogeneous image homogeneous scene

coordinates coordinates

Converting from homogeneous coordinates

Slide by Steve Seitz

Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous
coordinates:

 x
1 0 0 0    x 
0 1 0  y   x y
 0     y   ( f , f )
 z z z
 0 0 1 / f 0    z / f  divide by the third
 1 coordinate
 Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous
coordinates:

 x
1 0 0 0    x 
0 1 0  y   x y
 0     y   ( f , f )
 z z z
 0 0 1 / f 0    z / f  divide by the third
 1 coordinate

In practice: lots of coordinate transformations…

Camera to World to 3D
2D Perspective
pixel coord.
point = trans. matrix
projection matrix camera coord.
trans. matrix
point
(3x1) (3x4) (4x1)
(3x3) (4x4)
Orthographic Projection
Special case of perspective projection
• Distance from center of projection to image plane is infinite

Image World

• Also called “parallel projection”

• What’s the projection matrix?

Slide by Steve Seitz

Building a real camera
Camera Obscura

• Basic principle known to

Mozi (470-390 BCE),
Aristotle (384-322 BCE)
• Drawing aid for artists:
described by Leonardo
da Vinci (1452-1519)
Gemma Frisius, 1558

Source: A. Efros
 Abelardo Morell

From Grand Images Through a Tiny Opening, Photo

District News, February 2005

Camera Obscura Image of Manhattan View

Looking South in Large Room, 1996

http://www.abelardomorell.net/camera_obscura1.html
 Home-made pinhole camera

Why so
blurry?

Slide by A. Efros http://www.debevec.org/Pinhole/

 Shrinking the aperture

Why not make the aperture as small as possible?

• Less light gets through
• Diffraction effects…

Slide by Steve Seitz

Shrinking the aperture
Adding a lens

A lens focuses light onto the film

• Rays passing through the center are not deviated

Slide by Steve Seitz

Adding a lens

focal point

A lens focuses light onto the film

• Rays passing through the center are not deviated
• All parallel rays converge to one point on a plane located at
the focal length f

Slide by Steve Seitz

Adding a lens

“circle of
confusion”

A lens focuses light onto the film

• There is a specific distance at which objects are “in focus”
– other points project to a “circle of confusion” in the image

Slide by Steve Seitz

Thin lens formula

D’ D
f

Frédo Durand’s slide

 Thin lens formula
Similar triangles everywhere!

D’ D
f

Frédo Durand’s slide

 Thin lens formula
Similar triangles everywhere! y’/y = D’/D

D’ D
f
y
y’

Frédo Durand’s slide

 Thin lens formula
Similar triangles everywhere! y’/y = D’/D
y’/y = (D’-f)/f
D’ D
f
y
y’

Frédo Durand’s slide

Thin lens formula
1 +1 =1 Any point satisfying the thin
lens equation is in focus.
D’ D f
D’ D
f

Frédo Durand’s slide

Depth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Slide by A. Efros
How can we control the depth of field?

Changing the aperture size affects depth of field

• A smaller aperture increases the range in which the object is
approximately in focus
• But small aperture reduces amount of light – need to
increase exposure
Slide by A. Efros
Varying the aperture

Large aperture = small DOF Small aperture = large DOF

Slide by A. Efros
Nice Depth of Field effect

Source: F. Durand
Manipulating the plane of focus
In this image, the plane of focus is almost at a
right angle to the image plane

Source: F. Durand
 Tilt-shift lenses
• Tilting the lens with respect to the image plane allows
to choose an arbitrary plane of focus

shift

tilt

plane of focus lens plane image plane

• Standard setup: plane of focus is parallel to image

plane and lens plane
 Tilt-shift lenses
• Tilting the lens with respect to the image plane allows
to choose an arbitrary plane of focus

shift

tilt

tilted image plane

plane of focus
lens plane

• Scheimpflug principle: plane of focus

passes through the line of intersection
between the lens plane and the image plane
 “Fake miniatures”

Olivo Barbieri: http://www.metropolismag.com/cda/story.php?artid=1760

Field of View

Slide by A. Efros
Field of View

Slide by A. Efros
 Field of View

f
f

FOV depends on focal length and size of the camera retina

Smaller FOV = larger Focal Length Slide by A. Efros

Field of View / Focal Length

Large FOV, small f

Camera close to car

Small FOV, large f

Camera far from the car
Sources: A. Efros, F. Durand
Same effect for faces

wide-angle standard telephoto

Source: F. Durand
Approximating an affine camera

Source: Hartley & Zisserman

The dolly zoom
• Continuously adjusting the focal length while
the camera moves away from (or towards)
the subject

http://en.wikipedia.org/wiki/Dolly_zoom
The dolly zoom
• Continuously adjusting the focal length while
the camera moves away from (or towards)
the subject
• “The Vertigo shot”

Examples of dolly zoom from movies (YouTube)

Real lenses
Lens Flaws: Chromatic Aberration
Lens has different refractive indices for different
wavelengths: causes color fringing

Near Lens Center Near Lens Outer Edge

Lens flaws: Spherical aberration
Spherical lenses don’t focus light perfectly
Rays farther from the optical axis focus closer
Lens flaws: Vignetting
Radial Distortion
• Caused by imperfect lenses
• Deviations are most noticeable near the edge of the lens

No distortion Pin cushion Barrel

 Digital camera

A digital camera replaces film with a sensor array

• Each cell in the array is light-sensitive diode that converts photons to electrons
• Two common types
– Charge Coupled Device (CCD)
– Complementary metal oxide semiconductor (CMOS)
• http://electronics.howstuffworks.com/digital-camera.htm

Slide by Steve Seitz

CCD vs. CMOS
CCD: transports the charge across the chip and reads it at one corner of the array.
An analog-to-digital converter (ADC) then turns each pixel's value into a digital
value by measuring the amount of charge at each photosite and converting that
measurement to binary form

CMOS: uses several transistors at each pixel to amplify and move the charge using
more traditional wires. The CMOS signal is digital, so it needs no ADC.
http://electronics.howstuffworks.com/digital-camera.htm

http://www.dalsa.com/shared/content/pdfs/CCD_vs_CMOS_Litwiller_2005.pdf
Color sensing in camera: Color filter array

Bayer grid
Estimate missing
components from
neighboring values
(demosaicing)

Why more green?

Human Luminance Sensitivity Function

Source: Steve Seitz

Assignment 1: Demosaicing
Problem with demosaicing: color moire

Slide by F. Durand
The cause of color moire

detector

Fine black and white detail in image

misinterpreted as color information

Slide by F. Durand
Color sensing in camera: Prism
• Requires three chips and precise alignment
• More expensive
CCD(R)

CCD(G)

CCD(B)
Color sensing in camera: Foveon X3
• CMOS sensor
• Takes advantage of the fact that red, blue and green
light penetrate silicon to different depths

http://www.foveon.com/article.php?a=67 http://en.wikipedia.org/wiki/Foveon_X3_sensor

better image quality

Source: M. Pollefeys
Digital camera artifacts

Noise
– low light is where you most notice noise
– light sensitivity (ISO) / noise tradeoff
– stuck pixels

In-camera processing
– oversharpening can produce halos

Compression
– JPEG artifacts, blocking

Blooming
– charge overflowing into neighboring pixels

Color artifacts
– purple fringing from microlenses,
– white balance

Slide by Steve Seitz

 Historic milestones
• Pinhole model: Mozi (470-390 BCE),
Aristotle (384-322 BCE)
• Principles of optics (including lenses):
Alhacen (965-1039 CE) Alhacen’s notes
• Camera obscura: Leonardo da Vinci
(1452-1519), Johann Zahn (1631-1707)
• First photo: Joseph Nicephore Niepce (1822)
• Daguerréotypes (1839)
• Photographic film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895) Niepce, “La Table Servie,” 1822

• Color Photography (Lumière Brothers, 1908)

• Television (Baird, Farnsworth, Zworykin, 1920s)
• First consumer camera with CCD:
Sony Mavica (1981)
• First fully digital camera: Kodak DCS100 (1990)
CCD chip
Early color photography
Sergey Prokudin-Gorsky (1863-1944)
Photographs of the Russian empire
(1909-1916)

Lantern
projector
http://en.wikipedia.org/wiki/Sergei_Mikhailovich_Prokudin-Gorskii
http://www.loc.gov/exhibits/empire/
First digitally scanned photograph
• 1957, 176x176 pixels

http://listverse.com/history/top-10-incredible-early-firsts-in-photography/