Lenses: Focus and Defocus

“circle of
confusion”

• A lens focuses light onto the film

– There is a specific distance at which objects are “in focus”
• other points project to a “circle of confusion” in the image
– Changing the shape of the lens changes this distance

Slide by Steve Seitz

Thin lenses

• Thin lens equation:

– Any object point satisfying this equation is in focus

– What is the shape of the focus region?
– How can we change the focus region?
– Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang )Slide by Steve Seitz
 Slide source: Seitz

Depth of field

f / 5.6

f / 32

Changing the aperture size or focal length

affects depth of field

Flower images from Wikipedia http://en.wikipedia.org/wiki/Depth_of_field

 Beyond Pinholes: Radial Distortion

Corrected Barrel Distortion

Image from Martin Habbecke

Lens Flaws: Chromatic Aberration
• Dispersion: wavelength-dependent refractive index
– (enables prism to spread white light beam into rainbow)
• Modifies ray-bending and lens focal length: f()

• color fringes near edges of image

• Corrections: add ‘doublet’ lens of flint glass, etc.
 Chromatic Aberration

Near Lens Center Near Lens Outer Edge

Aside: Hollywood’s Anamorphic Format
• http://en.wikipedia.org/wiki/Anamorphic_format
 09/12/2011

Light and Color Capture

Intro to Computer Vision

James Hays, Brown
Slides by Derek Hoiem and others.
Graphic:
Today’s Class: light, color, eyes, and pixels

• Review of lighting
– Color, Reflection, and absorption

• What is a pixel? How is an image

represented?
– Color spaces
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence ?
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffuse Reflection light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Specular Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ1

• Refraction λ2
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency t=1

• Refraction t=n
• Fluorescence
• Subsurface scattering
• Phosphorescence
• Interreflection
A photon’s life choices
• Absorption
• Diffusion light source
• Reflection
• Transparency λ

• Refraction
• Fluorescence
• Subsurface scattering
• Phosphorescence (Specular Interreflection)
• Interreflection
The Eye

The human eye is a camera!

• Iris - colored annulus with radial muscles
• Pupil - the hole (aperture) whose size is controlled by the iris
• What’s the “film”?
– photoreceptor cells (rods and cones) in the retina

Slide by Steve Seitz

The Retina
Cross-section of eye Cross section of retina

Pigmented
epithelium
Ganglion axons
Ganglion cell layer
Bipolar cell layer

Receptor layer
What humans don’t have: tapetum lucidum
Two types of light-sensitive receptors

Cones
cone-shaped
less sensitive
operate in high light
color vision

Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision cone

rod
© Stephen E. Palmer, 2002
Rod / Cone sensitivity

The famous sock-matching problem…

Distribution of Rods and Cones
.

Blind
Fovea
# Receptors/mm2

Spot
150,000 Rods Rods
100,000
50,000 Cones Cones
0
80 60 40 20 0 20 40 60 80
Visual Angle (degrees from fovea)

Night Sky: why are there more stars off-center?

Averted vision: http://en.wikipedia.org/wiki/Averted_vision
© Stephen E. Palmer, 2002
 Electromagnetic Spectrum

Human Luminance Sensitivity Function

http://www.yorku.ca/eye/photopik.htm
Visible Light

Why do we see light of these wavelengths?

10000 C
.

…because that’s where the

5000 C
Sun radiates EM energy
Energy

2000 C
700 C

0 400 700 1000 2000 3000

Visible Wavelength (nm)

Region © Stephen E. Palmer, 2002
 The Physics of Light

Any patch of light can be completely described

physically by its spectrum: the number of photons
(per time unit) at each wavelength 400 - 700 nm.

# Photons
(per ms.)

400 500 600 700

Wavelength (nm.)
© Stephen E. Palmer, 2002
 The Physics of Light

Some examples of the spectra of light sources

A. Ruby Laser B. Gallium Phosphide Crystal
.

# Photons

# Photons
400 500 600 700 400 500 600 700
Wavelength (nm.) Wavelength (nm.)

C. Tungsten Lightbulb D. Normal Daylight

# Photons

# Photons

400 500 600 700 400 500 600 700

© Stephen E. Palmer, 2002
 The Physics of Light

Some examples of the reflectance spectra of surfaces

% Photons Reflected

Red Yellow Blue Purple

400 700 400 700 400 700 400 700

Wavelength (nm) © Stephen E. Palmer, 2002
 Physiology of Color Vision

Three kinds of cones:

440 530 560 nm.

100
RELATIVE ABSORBANCE (%)

S M L

50

400 450 500 550 600 650

WAVELENGTH (nm.)

• Why are M and L cones so close?

• Why are there 3? © Stephen E. Palmer, 2002
Tetrachromatism

Bird cone
responses

Most birds, and many other animals, have

cones for ultraviolet light.
Some humans, mostly female, seem to have
slight tetrachromatism.
More Spectra

metamers
Surface orientation and light intensity

Why is (1) darker than (2)?

For diffuse reflection, will intensity change when viewing angle
changes?
Perception of Intensity

from Ted Adelson

Perception of Intensity

from Ted Adelson

Image Formation
Digital camera

A digital camera replaces film with a sensor array

• Each cell in the array is light-sensitive diode that converts photons to
electrons
• Two common types: Charge Coupled Device (CCD) and CMOS
• http://electronics.howstuffworks.com/digital-camera.htm

Slide by Steve Seitz

Sensor Array

CMOS sensor
The raster image (pixel matrix)
The raster image (pixel matrix)
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
 Color Images: Bayer Grid

Estimate RGB
at ‘G’ cells from
neighboring
values
http://www.cooldictionary.com/
words/Bayer-filter.wikipedia
Slide by Steve Seitz
Color Image
R

B
Images in Matlab
• Images represented as a matrix
• Suppose we have a NxM RGB image called “im”
– im(1,1,1) = top-left pixel value in R-channel
– im(y, x, b) = y pixels down, x pixels to right in the bth channel
– im(N, M, 3) = bottom-right pixel in B-channel
• imread(filename) returns a uint8 image (values 0 to 255)
– Convert to double format (values 0 to 1) with im2double
column
row R
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 G
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81
0.95
0.81
0.89
0.87
0.82
0.57
0.89
0.37
0.56
0.80
0.31
0.88
0.75
0.89
0.92
0.79
0.81
0.85
0.95 0.91
B
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.49 0.62 0.60 0.58 0.92
0.50 0.93
0.60 0.94
0.58 0.97
0.50 0.62
0.61 0.37
0.45 0.85
0.33 0.97 0.93 0.92 0.99
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.95
0.51 0.89
0.39 0.82
0.73 0.89
0.92 0.56
0.91 0.31
0.49 0.75
0.74 0.92 0.81 0.95 0.91
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.96 0.67 0.54 0.85 0.89
0.48 0.72
0.37 0.51
0.88 0.55
0.90 0.51
0.94 0.42
0.82 0.57
0.93 0.41 0.49 0.91 0.92
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.96
0.43 0.95
0.42 0.88
0.77 0.94
0.73 0.56
0.71 0.46
0.90 0.91
0.99 0.87 0.90 0.97 0.95
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.79 0.73 0.90 0.67 0.71
0.33 0.81
0.61 0.81
0.69 0.87
0.79 0.57
0.73 0.37
0.93 0.80
0.97 0.88 0.89 0.79 0.85
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.49
0.41 0.62
0.78 0.60
0.78 0.58
0.77 0.50
0.89 0.60
0.99 0.58
0.93 0.50 0.61 0.45 0.33
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
Color spaces
• How can we represent color?

http://en.wikipedia.org/wiki/File:RGB_illumination.jpg
 Color spaces: RGB
Default color space
0,1,0

R
(G=0,B=0)

G
1,0,0 (R=0,B=0)

0,0,1

B
Some drawbacks (R=0,G=0)

• Strongly correlated channels

• Non-perceptual
Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png
Color spaces: HSV
Intuitive color space

H
(S=1,V=1)

S
(H=1,V=1)

V
(H=1,S=0)
 Color spaces: YCbCr
Fast to compute, good for
compression, used by TV
Y=0 Y=0.5
Y
(Cb=0.5,Cr=0.5)

Cr

Cb
(Y=0.5,Cr=0.5)
Cb
Y=1

Cr
(Y=0.5,Cb=05)
Color spaces: L*a*b*
“Perceptually uniform”* color space

L
(a=0,b=0)

a
(L=65,b=0)

b
(L=65,a=0)
If you had to choose, would you rather go
without luminance or chrominance?
If you had to choose, would you rather go
without luminance or chrominance?
Most information in intensity

Only color shown – constant intensity

Most information in intensity

Only intensity shown – constant color

Most information in intensity

Original image
Back to grayscale intensity
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99
0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91
0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92
0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95
0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85
0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33
0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74
0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93
0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99
0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97
0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
Next classes: filtering!

• Image filters in spatial domain

– Filter is a mathematical operation of a grid of numbers
– Smoothing, sharpening, measuring texture

• Image filters in the frequency domain

– Filtering is a way to modify the frequencies of images
– Denoising, sampling, image compression

• Templates and Image Pyramids

– Filtering is a way to match a template to the image
– Detection, coarse-to-fine registration
 Image Pyramids

Known as a Gaussian Pyramid [Burt and Adelson, 1983]

• In computer graphics, a mip map [Williams, 1983]
• A precursor to wavelet transform

Slide by Steve Seitz