Image formation - 1

Image Formation in Man and

Machines
Image formation - 2
Overview
v Pinhole camera
v Refraction of light
v Thin-lens equation
v Optical power and accommodation
v Image irradiance and scene radiance
v Human eye
v Geometry of perspective imaging
Image formation - 3
Lens-less Imaging Systems - Pinhole
Optics
v Pinhole optics
projects images
without lens
with infinite depth of
field
v Smaller the
pinhole
better the focus
less the light energy
from any single point
v Good for tracking
solar eclipses
Pinhole
I mage
Scene
Blurred
I mage
Image formation - 4
Diffraction
v Two disadvantages to pinhole systems
Low light collecting power
diffraction
v Diffraction
Light bends as it passes by the edge of a narrow aperture
v Human vision
at high light levels, pupil (aperture) is small and blurring is
due to diffraction
at low light levels, pupil is open and blurring is due to lens
imperfections
D

=
D
Image formation - 5
Diffraction and pinhole optics
Image formation - 6
Lenses Collect More Light
v With a lens, diverging rays from a scene
point are converged back to an image point
Center of projection
I mage plane
Scene
Lens
Image formation - 7
incident ray
reflected ray
refracted ray
refracted ray
v If is the angle of incidence
and is the angle of refraction
then
where n and n are the
refractive indices of the two
media
v Refractive index is the ratio of
speed of light in a vacuum to
speed of light in the medium
Refraction: Snells law
Refractive indices
glass - 1.50
water - 1.333
air - 1.000

' sin ' sin n n =

Image formation - 8
Lens Equations
Image formation - 9
Thin-Lens Equation
v Thin-lens equation
relates the distance between the scene point being viewed and the lens
to the distance between the lens and the points image (where the rays
from that point are brought into focus by the lens)
Let M be a point being viewed
u pis the distance of M from the lens along the optical axis
u The thin lens focuses all the rays from M onto the same point, the
image point m at distance qfrom the lens.
M
O
F
m
f q-f
p q
H
h
Q
S
s
Image formation - 10
Thin-Lens Equation
v m can be determined by
intersecting two known
rays
MQ is parallel to the optical
axis, so it must be refracted to
pass through F.
MO passes through the lens
center, so it is not bent.
v Note two pairs of
similar triangles
MSO and Osm(yellow)
OQF and Fsm(green)
f q p
1 1 1
= +
q p
h H
q
h
p
H
+
+
= =
q
h H
f q
h
f
H +
=

=
q
q p
f
p +
=
M
O
F
m
f q -f
p q
H
h
Q
S
s
q p
q p
f
+
=
1
Divide 2 equations:
Image formation - 11
Thin-Lens Equation
v Notice that the distance behind the lens, q, at
which a point, M, is brought into focus
depends on p, the distance of that point from
the lens
familiar to us from rotating the focus ring of any camera
f q p
1 1 1
= +
M
O
F
m
f
q-f
p q
H
h
Q
S
s
Image formation - 12
Thin-Lens Equation
v As pgets large, qapproaches f
v As qapproaches f, papproaches infinity
f q p
1 1 1
= +
M
O
F
m
f
p q
Q
S
M
S
m
Image formation - 13
Optical Power and Accommodation
v Optical power of a lens - how strongly the
lens bends the incoming rays
Short focal length lens bends rays significantly
It images a point source at infinity (large p) at distance f
behind the lens. The smaller f, the more the rays must be
bent to bring them into focus sooner.
Optical power is 1/f, with f measured in meters. The unit is
called the diopter
Human vision: when viewing faraway objects the distance
from the lens to the retina is 0.017m. So the optical power of
the eye is 58.8 diopters
Image formation - 14
Accommodation
v How does the human eye bring nearby points
into focus on the retina?
by increasing the power of the lens
muscles attached to the lens change its shape to change the
lens power
accommodation: adjusting the focal length of the lens
bringing points that are nearby into focus causes faraway
points to go out of focus
depth-of-field: range of distances in focus
M
O
m
f
p
q
S
M
S
m
F F
Image formation - 15
Accommodation
Image formation - 16
Accommodation
Sources at > 1 meter are imaged at same distance
Sources closer than 1 m are imaged at different distances
Image formation - 17
Accommodation
v Physical cameras - mechanically change the
distance between the lens and the image
plane
M
O
F
m
f
p q
S
M
S
m
Image formation - 18
Pixel Brightness and Scene
Brightness
2 2
) cos / (
cos
cos / (
cos

Z
dA
f
da
=
)
2
cos
cos

=
f
Z
da
dA

cos cos
4
3
2

=
Z
D
LdA dP

cos cos
4
3
2

= =
Z
D
da
dA
L
da
dP
E
D
f
da

L
f
D
E

4
2
cos
4

=
dA
cos = LdA dP
Z
Image formation - 19
Image Irradiance and Scene Radiance
v Image irradiance E is proportional to scene radiance
v Brighter scene points produce brighter pixels
v Image irradiance is proportional to inverse of square
of f-number (f/D), is larger for small f-number
L
f
D
E

4
2
cos
4

=
Image formation - 20
Retina
Light
Image formation - 21
Retina
v Limitations of human
vision
Blood vessels and other
cells in front of
photoreceptors
shadows cast on
photoreceptors
non-uniform brightness
Light
Image formation - 22
Photoreceptor Mosaics
v The retina is covered with a mosaic of
photoreceptors
v Two different types of photoreceptors
Rods - approximately 100,000,000
Cones - approximately 5,000,000
v Rods
Sensitive to low levels of light: scotopic light levels
v Cones
Sensitive to higher levels of light: photopic light levels
v Mesopic light levels - both rods and cones
active
Image formation - 23
Photoreceptor Mosaics
v (A) Cones in the Fovea. Several neurons per cone
v (B) Cones and Rods in the periphery
v Rods are small but several rods per neuron
Image formation - 24
Photoreceptor Mosaics
v Fovea is area of highest concentration of
photoreceptors
fovea contains no rods, just cones
approximately 50,000 cones in the fovea
cannot see dim light sources (like stars) when we look
straight at them!
v TV camera photoreceptor mosaics
nearly square mosaic of approximately 800X640 elements for
complete field of view
Image formation - 25
The Human Eye
v Limitations of human
vision
the image is upside-down!
high resolution vision only in
the fovea
u only one small fovea in
man
u other animals (birds,
cheetas) have different
foveal organizations
blind spot
Image formation - 26
Blind Spot
v Close left eye
v Look steadily at white cross
v Move head slowly toward and away from figure
v At a particular head position, the white disk
disappears completely from view.
Image formation - 27
Cones and color
v There are three different types of cones
they differ in their sensitivity to different wavelengths of
light
Image formation - 28
Color Cameras
v Two types of color cameras
Single CCD array
u in front of each CCD element is a filter - red, green or
blue
u color values at each pixel are obtained by hardware
interpolation
subject to artifacts
lower intensity quality than a monochromatic
camera
3 CCD arrays packed together, each sensitive to different
wavelengths of light
Image formation - 29
3-CCD Camera
Image formation - 30
Cones, CCDs and space
v How much of the world does a cone see?
measured in terms of visual angle
the eye lens collects light over a total field of view of about
100
o
each cone collects light over a visual angle of about 8.5 x 10
-3
degrees (about 30 seconds)
v How much of the world does a single camera
CCD see
example: 30
o
lens
30/500 gives about 6 x10
-2
degrees per CCD
Eyes acuity is 10 times higher.
Image formation - 31
Perspective Imaging -
Pinhole Camera Model
pinhole at the
center of projection
Scene point
(x
s
, y
s
, z
s
)
Image point
(x
i
, y
i
, -f)
f
x
z
v The point on the image plane that corresponds to a particular
point in the scene is found by following the line that passes
through the scene point and the center of projection
z
s
y
Image center
Line of sight
Image plane
C
Image formation - 32
Perspective Imaging=Central Projection
v Line of sight to a point in the scene is the line
through the center of projection to that point
v Image plane is parallel to the x-y plane
distance to image plane is f - focal length
this inverts the image
move the image plane in front of the center of projection
Image formation - 33
Perspective Imaging
v Fundamental equations
for perspective
projection onto a plane
Scene point
(x
s
, y
s
, z
s
)
Image point
(x
i
, y
i
, f)
f
x
z
C
Image point
(x
i
, y
i
, -f)
f
s
s
i
s
s
i
z
y
f y
z
x
f x
=
=
Image plane
y
center of
projection
Image formation - 34
Field of View
v As f gets smaller, image
becomes more wide angle
(more world points
project onto the finite
image plane)
v As f gets larger, image
becomes more telescopic
(smaller part of the
world projects onto the
finite image plane)
f
3
f
2
f
1
Field of view
Image formation - 35
m
Vanishing Points and Lines
v We are looking at a scene on a plane
v To each point M on the scene plane we
associate an image point m
v The line, v, of the image plane that
belongs to the plane through C parallel
to the scene plane is called the
vanishing line, or the horizon line
Image plane
Scene plane
C
v
M
m
Image formation - 36
Vanishing Points and Lines
v The images of the points on rail L
1
belong
to a plane P
1
defined by C and L
1
The lines of sight for each point of the
track L
1
lie on this plane
v The image of rail L
2
belongs to a plane P
2
defined by C and track
v The image l
1
of L
1
belongs to intersection of
plane P
1
and image plane
v The image l
2
of L
2
belongs to intersection of
plane P
1
and image plane
v Planes P
1
and P
2
intersect along line CV
parallel to L
1
and L
2
v Point V belongs to P
1,
P
2
and image plane.
v Therefore, l
1
and l
2
intersect in V
v V is image of a scene point of L
1
and a
scene point of L
2
. Since CV is parallel to L
1
and L
2
, these points are at infinity.
m
Image plane
Scene plane
C
v
V
V
L
1
L
2
l
1
l
2
Image formation - 37
From Perspective Projection to
Orthographic Projection
v Select origin of
coordinate system at
image center
v World coordinates are
independent of focal
length
v = 1/f
v When = 0,
orthographic projection
Scene point
(x
s
, y
s
, z
s
)
(x
i
, y
i
, 0)
x
z
y
f
s
s
s
s
i
s
s
s
s
i
z
y
z f
y
f y
z
x
z f
x
f x

+
=
+
=
+
=
+
=
1
1
C
O
Image formation - 38
Orthographic Projection
Scene point
(x
s
, y
s
, z
s
)
(x
i
, y
i
, 0)
x
z
y
1/f = 0
s i
s i
y y
x x
=
=
C
O
Image plane
Image formation - 39
Depth of Field and f-number
v Depth of field is smaller for small f-number
M
F
m
S
Image formation - 40
Lens Imperfections
v Lens imperfections might cause rays not to
intersect at a point
Deviations in shape from the ideal lens
Material imperfections that might cause the refractive index
to vary within the lens
Image formation - 41
Refraction of Color
v Why does the prism
separate the light into its
spectral components?
Prism bends different
wavelengths of light by different
amounts
u Refractive index is a function
of wavelength
u Shorter wavelengths are
refracted more strongly than
longer wavelengths
Image formation - 42
Chromatic Aberration
v Chromatic aberration
Different wavelengths of light from the same point source
are focused at different distances behind the lens
When incident light is a mixture of wavelengths, we can
observe a chromatic fringe at edges
Accommodation can bring any wavelength into good focus,
but not all simultaneously
Human visual system has other mechanisms for reducing
chromatic aberration
Color cameras have similar problems
Image formation - 43
Chromatic Aberration
Image formation - 44
Compound Eyes
Many (small) animals have compound eyes
- each photoreceptor has its own lens
- images seen by these eyes are equally sharp in
all directions
-examples: flies and other insects
But these eyes do not scale well biologically
Image formation - 45
References
v Foundations of Vision, Brian Wandell, Sinauer
Associates, Sunderland MA, 1995
v Introductory Techniques for 3-D Computer
Vision, E. Trucco and A. Verri, Prentice-Hall,
pp. 18-28
v Computer Vision, Ballard & Brown, Prentice-
Hall, pp. 19-22, pp.206-208
v Robot Vision, B.K.P. Horn, MIT Press, pp. 18-27
v A Guided Tour of Computer Vision, V. Nalwa,
AT&T Press, pp. 3-49